I need to create a localized environment for a SSIS package that would only have the objects and entities needed by tasks inside it.
It's a large & complex SSIS package and it connects to eight databases on the same sql server.
Is there a quick way to list all the tables, stored procedures, functions etc that will be utilized(directly or indirectly) when I execute this package.
Good thought by Nick, but even that will only get you objects that are used directly. There is no quick easy way to get all the objects that will be used indirectly.
Related
We are presently developing an application, let's call it APP1, which uses a SQL Database which have about 800 stored procedures, 600 tables, etc. APP1 was originally created in order to replace another application, APP0, from which we do not have source code but only SQL tables, Stored Procedures, views, etc. Previous programers of APP1 used some DB objects from this same database and added some other objects specific to APP1 because it becomes bigger than APP0. And we do not need APP0 anymore as APP1 does all what we want, and more.
So, now, we are thinking about a way to find out which objects are used by APP1 in order to remove objects which are ONLY used by APP0.
What is the best approach to discover all objects used by APP1 without having to open every single class and form?
Once we will have a complete list of these objects, it will be easy to use a program we bought which detects all dependencies for all SQL Objects specified directly from SQL and remove objects which do not return from any dependencies. Any ideas of how I could get this list without having to go through all our program that have many, many, many classes and forms?
Thanks,
Note : I know, in a perfect world, all calls to PSs and tables should be in a DAL but in the case of the application we're presently working on ... this is not our case! Yippy! (sarcastic yippy) ;)
Note 2 : This application is not using any ORM. So all queries are directly using SqlCommand. So any call to any DB objects are in string format.
You mentioned you have all the Tables, Sprocs & etc from APP0. Presumably there is a BAK of them or you can grab the original SQL objects by installing APP0 on a fresh PC.
Then use SQL Compare from RedGate to compare the Database that APP1 uses to the original APP0 Database, then you can see which objects you've added and can strip out all the redundant APP0 db objects.
You could run a trace on the database whilst the application is in use. This is likely to create a rather large amount of data, but from that you can reduce it to the procedures and or SQL statements executed by your application.
Can you guarantee that you will, or can, use all the functionality? You might want to also run something like NCover to check how much of the application code you've exercised whilst using it.
I don't have an easy answer, but here's how I'd attack it. I admit up front this would take a fair amount of time, so I'll readily yield to someone who has a better answer.
It's a two-step problem.
Step 1: Find all the dependencies within SQL. That is, find all the tables that are used to make views, and find all the tables and views that are used in stored procedures and functions. MS SQL server has a function to do this for you. With other DBs you could write some queries against information_schema (or whatever their proprietary equivalent is).
Step 2: Get a list of all the SQL statements and stored procedures executed from within your code. This should be relatively straightforward if you do not build SQL statements on the fly. Just search for all your SQLCommand objects and find what you set the query to. You could write a little program to scan your source and dump this out.
Then parse through this dump and make a list of referenced sprocs, tables, and views. Sort alphabetically and eliminate duplicates. Then add any tables or views referenced from sprocs and any tables referenced from views. Sort and eliminate duplicates again. Then you have your list.
If you do geneate SQL on the fly, I think the complexity level multiplies greatly. Then you have to work your way through code that generates SQL and pick out the table names. If there are places where table names are passed from function to function, this could get very hard. (I could imagine real nightmare scenarios, like you ask the user to type in a table name or you build a table name from pieces. Like, "dim tablename = if(dept="B17","accounting", "manufacturing") & "_" & year".)
Let's imagine that I have 2 separate Databases on the same instance of SQL Server.
DB1 is a Database relating to Trading and Position Keeping.
DB2 is a Database relating to market pricing variables.
Both databases have the concept of working with time/date objects and I have created some convenience UDFs. Lets further imagine I build some convenient math functions that I would like to be called from all databases. What is the best way of creating and organising them?
i.e. Should I create the UDFs and SPs in the Master Database? How should I group all my CustomDateTime UDFs, is it best to create something in the Schema so that I replace .dbo with .myDateTime or .myMath?
Suggestions appreciated as I do not like how I currently have placed most of my functions in DB1 and I know that many will be relevant to DB2?
Create a Database Project using Visual Studio (I believe 2008 and 2010 support it). Keep object scripts organized within the project. Check the project and related files into some form of version control software (SVN, git, hg, SourceSafe...).
You should have a DBA or a dedicated person who is responsible for deploying code changes to your production environments. You can configure the database project with pre- and post-deployment scripts that can make this easier to work with.
I can't recommend keeping user-defined objects in master; IMHO you're better off with copies in each database. If you're managing your code as above it won't matter that they're duplicated, since the source is in a single, managed location.
I was told by someone that when you create procedures in Oracle you should create a Package with procedures in it. Is this a true statement? Are procedures in MS the same as Oracle?
You do not have to put your procedures and functions inside a package, but it is generally considered a best practice. Bundle them by function and they are a lot easier to organize.
One exception to this is the AUTHID clause. That can only be specified for an entire package or a standalone procedure/function. Different procedures inside a single package cannot have different privileges.
Procedures in Oracle and MS are similar, but yes, in Oracle, you make a package declaring the procedure, then you define the procedure in the package body
This has some good info on how to build an Oracle package
Stored procedures and functions are very similar between Oracle and SQL Server (or MySQL, PostgreSQL, etc). A SQL function is intended to always return a SQL data type; a stored procedure can return a SQL data type but doesn't by default.
An Oracle package is an awesome tool.
SQL Server has assemblies, but they aren't as immediately accessible as Oracle packages.
Besides being a container for functions and stored procedures, Oracle packages support variable declaration when other alternatives would only expose via functions.
Oracle packages also allow for logical grouping of functionality, without needing to setup schemas (for namespacing). Because of what's contained in a package, you deploy the package -- not the functions/stored procedures individually.
My only regret was that IME (Oracle 9i, 10g), you could not drill into the specific function/stored procedure/line when tracing references.
There's nothing requiring you to use Oracle Packages, but you really stand to benefit by using them vs not. I don't understand why other vendors haven't reproduced the functionality in their own way.
There are a ton of reasons to use packages over standalone procedures/functions, and a few situations when you wouldn't use packages. Instead of trying to rehash them from memory, I'll point you to an excellent article from pl/sql guru Steve Feuerstein.
Note: The article is several years old, but the rationale still holds imo
I have several different packages, one for each logical part of my application. Some packages are getting huge but I would like to keep all the procedures/functions grouped in some way rather than breaking them into separate packages. Is there any way to nest, or namespace, my packages?
So if I have MYSCHEMA.PKG_PEOPLE and it has 10 procedures and 10 functions, is there no way that I can for instance move the CRUD procedures to MYSCHEMA.PKG_PEOPLE.CRUD. I want to keep all these items inside of PKG_PEOPLE but I want to further sub-divide them.
Beyond Schema and Package, there is no multi-level namespace handling for PL/SQL packages in Oracle.
Within a package body you can define nested procedures but I would guess this isn't what you need.
I think the closest you'll get is to enforce a naming rule on your packages. For example:
MYSCHEMA.PKG_PEOPLE
MYSCHEMA.PKG_PEOPLE_CRUD
I have read these very good questions on SO about SQL stored procedures:
When should you use stored procedures? and
Are Stored Procedures more efficient, in general, than inline statements on modern RDBMS’s?
I am a beginner on integrating .NET/SQL though I have used basic SQL functionality for more than a decade in other environments. It's time to advance with regards to organization and deployment. I am using .NET C# 3.5, Visual Studio 2008 and SQL Server 2008; though this question can be regarded as language- and database- agnostic, meaning that it could easily apply to other environments that use stored procedures and a relational database.
Given that I have an application with inline SQL queries, and I am interested in converting to stored procedures for organizational and performance purposes, what are your recommendations for doing so?
Here are some additional questions in my mind related to this subject that may help shape the answers:
Should I create the stored procedures in SQL using SQL Management Studio and simply re-create the database when it is installed for a client?
Am I better off creating all of the stored procedures in my application, inside of a database initialization method?
It seems logical to assume that creating stored procedures must follow the creation of tables in a new installation. My database initialization method creates new tables and inserts some default data. My plan is to create stored procedures following that step, but I am beginning to think there might be a better way to set up a database from scratch (such as in the installer of the program). Thoughts on this are appreciated.
I have a variety of queries throughout the application. Some queries are incredibly simple (SELECT id FROM table) and others are extremely long and complex, performing several joins and accepting approximately 80 parameters. Should I replace all queries with stored procedures, or only those that might benefit from doing so?
Finally, as this topic obviously requires some research and education, can you recommend an article, book, or tutorial that covers the nuances of using stored procedures instead of direct statements?
Consider skipping stored procedures for an ORM. Consider using:
LINQ To SQL
Entity Framework
SubSonic
You'll be writing less boiler plate ListCustomer and GetCustomerByID code when you could be adding more value to your application.
IMO, there isn't any real compelling reason to choose stored procedures with the modern toolset that we have in the Microsoft stack.
The move away from inline SQL statements is good, and an ORM will help parameterize your queries for you. You don't have to think about it.
You don't have to mess with ADO.NET objects at all. Code your data access in an object oriented fashion.
There are several compelling reasons to avoid giving table access to very many logins, including application logins, and these drive the use of stored procedures. (I generally do not ascribe any importance to using SPs for performance reasons - SQL Server caches even adhoc query plans).
Stored procedures give your database much more capability in defining its interface boundaries. In many cases, views are not sufficient to control the interface.
Any framework built solely on tables and views (note that many frameworks can build on top of SP results) is going to be severely limited in letting your database protect itself and control itself.
As a simple example, neither tables nor views can be parameterized. If you have a very large table or view and you want to enforce all users to specify a certain set of filter criteria (for instance a snapshot date or effective date), there is no way to enforce this at the database call interface. The framework can submit queries for all time. If the table/view is not exposed, and the only interface is through an SP or table-valued UDF, then the parameters to that SP or UDF MUST be provided, thus satisfying your database's need to ensure that it is used properly.
Other examples, where views may or may not work, include hiding privacy information for certain users, hiding internal keys, hiding internal implementation details, and enforcing complex security rules.
As far as scripting the creation of your database schema, including objects in the correct dependency order, there are several tools to do this (and generate change scripts), including Red Gate SQL Compare and Apex SQLScript.
Use stored procedures if you really have a performance requeriment, particularly if one stored procedures will be called thousands of times per minute. This way sql engine avoids severals steps for processing the statement. GPS Tracking systems is an example. Say you have 10000 vehicles which reports a 3 positions per minute. In this case stored procedures helps performance.
If not, instead of CRUD sql statements, use ORM features.
You missed one:
When is it better to write "ad hoc sql" vs stored procedures
My answer is: don't use stored procedures at all.