I have a set of 10 tables in a database (DB1). And there are 10 tables in another database (DB2) with exact same schema on the same SQL Server 2008 R2 database server machine.
The 10 tables in DB1 are frequently updated with data.
I intend to write a stored procedure that would run once every day for synchronizing the 10 tables in DB1 with DB2. The stored procedure would make use of the MERGE statement.
Now, my aim is to make this as generic and parametrized as possible. That is, accommodate for more tables down the line... and accommodate different source and target DB names. Definitely no hard coding is intended.
This is my algorithm so far:
Have the database names as parameters
Have the first query within the stored procedure... result in giving the names of the 10 tables from a lookup table (this can be 10, 20 or whatever)
Have a generic MERGE statement that does the sync for each of the above set of tables (based on primary key?)
This is where I need more inputs on.
What is the best way to achieve this stored procedure? SQL syntax would be helpful.
I had to do something similar, to do that i used a string with a "skeleton" for the merge statement, then i retrieved the list of columns and the pks with a simple query on the sys views.
you could do something similar to build your merge statement, here's a sketch i wrote now as an example (I know it's horrible but i'm not going to write something decent at this hour, and it should give you a hint anyway :P )
SQLFiddle
then you just need to execute it with the usual sp_executesql stored procedure
by the way, always pay attention when building command strings this way, it's not that secure
Related
I know basic sql commands, and this is my first time working with stored procedures. In the stored procedure I am looking at, there are several temporary tables.
The procedure, is triggerred every morning, which then pulls a specific ID and then loops through each ID to grab certain parameters.
My question is: are temporary tables used in stored procedures so that when the procedure goes off the variables will be instantly passed into the parameters and loop, and then the temporary tables will be cleared, thus restarting the process for the next loop?
Temporary tables are used because as soon as the session that created them (or stored procedure) is closed the temporary table is gone. A Temp table with a single # in front of the name (also called a local temp table) is only visible in the session it was created in so a temp table with the same name can be created in multiple sessions without bumping into each other (SQL Server adds characters to the name to make it unique). If a temp table with two ## in front of it is created (a global temp table) then it is unique within SQL Server so other sessions can see it. Temp tables are the equivalent of a scratch pad. When SQL Server is restarted all temp tables and their values are gone. Temp tables can have indexes created against them and SQL Server can use statistics on Temp tables to create efficient query plans.
For stored procedures (SPs), they are the least restricted and most capable objects, for example:
Their usual alternatives, views and functions, are not allowed to utilize many things, like DML statements, temp. tables, transactions, etc.
SPs can avoid returning result sets. They also can return more than one result set.
For temp. tables:
Yes, once the SP is done, the table disappears along with its
contents (at least for single-# tables).
They have advantages &
disadvantages compared to their alternatives (actual tables, table
varaibles, various non-table solutions).
Stored Procedures, in my opinion, don't forcibly need Temporary tables. It's up to the SP's scope to decide if using a TempTable is the best approach.
For example, let's suppose that we want to retrieve a List of elements that come out from joining a few tables, then it's the best to have a TempTable to put the joined fields. On the other hand, if we're using a Stored Procedure to retrieve a single or field, I don't see the need for a Temp table.
Temp tables are only available during the usage of the Stored Procedure, once it's finished, the table goes out of scope.
I have a set of stored procedures that are stored in one database and all refer to tables in another database to snapshot data. They have a structure broadly like this (the actual queries are more complicated, but this reflects the database usage):
USE [Database_A]
GO
INSERT [Database_A].[dbo].[MyTable]
SELECT *
FROM [Database_B].[dbo].[YourTable]
When applying a column filter to the SQL server profiler, in order to see everything these queries are doing, should I filter on Database_A or Database_B or both?
It says "Name of the database in which the statement of the user is running", but are there parts of the above style of query running in each?
A statement runs in the context of a single database even if the referenced objects are in other databases. The database name will be Database_A for events such as SQL:BatchCompleted and SQL:StmtCompleted in your example. However, more granular events like SP:StmtCompleted will show the database name of the stored procedure containing the statement.
Consider the following abstract situation (just as an example):
I have two tables TableA and TableB. They have unique IDs and possibly other columns (which are irrelevant) The relatioship between them is many to many so I have a third table AssociationTable that is used to store the relationships between them. Basically, AssociationTable will have two columns (ID_A and ID_B - foreign keys).
If I delete a row in AssociationTable and the ID_A that was deleted was the last one, I would also like to delete the entry from TableA that corresponds to that ID.
I could do this:
a) From the application that uses the database
b) by using an SQL trigger
My question, basically, is the following:
Is there any good practice that says "if you can do something from both the application and from SQL, always prefer sql." ?
Or does it depend on the case? If so, what should I take into account?
Performance: The query plan for stored procedures is compiled onn DB Server and subsequent requests can run faster.
A stored procedure can execute multiple steps and the intermediate results need not go back to application layer, reducing traffic between an application and the DB server.
Security: Stored procedures are well defined database objects that can be locked down with security measures. Use of typed parameters can help prevent SQL injection attacks.
Code re-use: SQL queries can be written once and re-used across multiple clients without writing the same SQL commands over and over again.
Abstraction: By putting all the SQL code into a stored procedure, the application is completely abstracted from the field names, tables names, etc. So when a SQL query needs to be changed, there is almost zero or NO impact in the application code.
There are more benefits of doing it in the database.
Other client application code need not worry about data integrity.
The data logic should remain as close to data as possible
It could be faster if managed by DB (trigger invocation).
I have a particular SQL file in which i copy all contents from on table in a database to another table in another database.
The traditional INSERT statements are used to perform the same operation. However this table has 8.5 Million records and it fails. The queries succeed with a smaller database.
Also in when i run the select * query for that particular table the SQL query express shows out of memory exception.
In particular there is one table that has some many records. So this table alone i want to copy from the old Db to the new Db.
What are alternate ways to achieve this?
Is there any quick work around by which we can avoid this exception and make the queries succeed?
Let me put it this way. Why would this operation fail when there are a lot of records?
I don't know if this counts as "traditional INSERT", but have you tried "INSERT INTO"?
http://www.w3schools.com/sql/sql_select_into.asp
I read about temporary tables, global temporary tables and table variables. I understood it but could not imagine a condition when I have to use this. Please elaborate on when I should use the temporary table.
Most common scenario for using temporary tables is from within a stored procedure.
If there is logic inside a stored procedure which involves manipulation of data that cannot be done within a single query, then in such cases, the output of one query / intermediate results can be stored in a temporary table which then participates in further manipulation via joins etc to achieve the final result.
One common scenario in using temporary tables is to store the results of a SELECT INTO statement
The table variable is relatively new (introduced in SQL Server 2005 - as far as i can remember ) can be used instead of the temp table in most cases. Some differences between the two are discussed here
In a lot of cases, especially in OLTP applications, usage of temporary tables within your procedures means that you MAY possibly have business processing logic in your database and might be a consideration for you to re-look your design - especially in case of n tier systems having a separate business layer in their application.
The main difference between the three is a matter of lifetime and scope.
By a global table, I am assuming you mean a standard, run of the mill, table. Tables are used for storing persistent data. They are accessible to all logged in users. Any changes you make are visible to other users and vice versa.
A temporary table exist solely for storing data within a session. The best time to use temporary tables are when you need to store information within SQL server for use over a number of SQL transactions. Like a normal table, you'll create it, interact with it (insert/update/delete) and when you are done, you'll drop it. There are two differences between a table and a temporary table.
The temporary table is only visible to you. Even if someone else creates a temporary table with the same name, no one else will be able to see or affect your temporary table.
The temporary table exists for as long as you are logged in, unless you explicitly drop it. If you log out or are disconnected SQL Server will automatically clean it up for you. This also means the data is not persistent. If you create a temporary table in one session and log out, it will not be there when you log back in.
A table variable works like any variable within SQL Server. This is used for storing data for use in a single transaction. This is a relatively new feature of TSQL and is generally used for passing data between procedures - like passing an array. There are three differences between a table and a table variable.
Like a temporary table, it is only visible to you.
Because it is a variable, it can be passed around between stored procedures.
The temporary table only exists within the current transaction. Once SQL Server finishes a transaction (with the GO or END TRANSACTION statements) or it goes out of scope, it will be deallocated.
I personally avoid using temporary tables and table variables, for a few reasons. First, the syntax for them is Microsoft specific. If your program is going to interact with more than one RDBMS, don't use them. Also, temporary tables and table variables have a tendency to increase the complexity of some SQL queries. If your code can be accomplished using a simpler method, I'd recommend going with simple.