We have a need to (once per month) clear out the contents of a table with 50,000 records, and repopulate, using a Stored Procedure. The SP has a User Defined Table Type parameter which contains all of the new records to be inserted.
The current thought is as follows
ALTER PROCEDURE [ProcName]
#TableParm UserTableType READONLY
AS
[Set lock on table?]
BEGIN TRAN
DELETE FROM [table]
INSERT INTO [table](column, column, column)
SELECT (a.column, a.column, a.column) FROM #TableParm a
COMMIT TRAN
[Remove lock from table?]
I've read some solutions which suggest to set READ COMMITED or READ UNCOMMITED... but figured I'd turn to the pro's to steer me in the right direction, based on the situation.
thanks!
I'd use a serializable transaction
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE
Both the READ... type levels would allow data of some form to be read from the table, which is probably not what you want.
You may also be able to use TRUNCATE TABLE rather than DELETE, depending on your data structure.
If reducing the unavailability of this table is an issue, you may be able to reduce it by creating a new table, populating it, then renaming the old and new tables.
Related
I am trying to create a stored procedure to recreate a table from scratch, with a possible change of schema (including possible additions/removals of columns), by using a DROP TABLE followed by a SELECT INTO, like this:
BEGIN TRAN
DROP TABLE [MyTable]
SELECT (...) INTO [MyTable] FROM (...)
COMMIT
My concern is that errors could be generated if someone tries to access the table after it has been dropped but before the SELECT INTO has completed. Is there a way to lock [MyTable] in a way that will persist through the DROP?
Instead of DROP/SELECT INTO, I could TRUNCATE/INSERT INTO, but this would not allow the schema to be changed. SELECT INTO is convenient in my situation because it allows the new schema to be automatically determined. Is there a way to make this work safely?
Also, I would like to be sure that the source tables in "FROM (...)" are not locked during this process.
If you try to make a significant change to the table (like adding a column in the middle of existing columns, not at the end) using SSMS and see what script it generates, you'll see that SSMS uses sp_rename.
The general structure of the SSMS's script:
create a new table with temporary name
populate the new table with data
drop the old table
rename the new table to the correct name.
All this in a transaction.
This should keep the time when tables are locked to a minimum.
BEGIN TRANSACTION
SELECT (...) INTO dbo.Temp_MyTable FROM (...)
DROP TABLE dbo.MyTable
EXECUTE sp_rename N'dbo.Temp_MyTable', N'dbo.MyTable', 'OBJECT'
COMMIT
DROP TABLE MyTable acquires a schema modification (Sch-M) lock on it until the end of transaction, so all other queries using MyTable would wait. Even if other queries use the READ UNCOMMITTED isolation level (or the infamous WITH (NOLOCK) hint).
See also MSDN Lock Modes:
Schema Locks
The Database Engine uses schema modification (Sch-M)
locks during a table data definition language (DDL) operation, such as
adding a column or dropping a table. During the time that it is held,
the Sch-M lock prevents concurrent access to the table. This means the
Sch-M lock blocks all outside operations until the lock is released.
First of all sorry if my English is not good.
I'm facing a problem with respect to transaction isolation level. My current isolation level is read committed.But it leads table to dead lock some times.
For example
create table tmp(id int,name varchar(20))
insert into tmp(id,name)
values(1,'Binesh')
,(2,'Bijesh')
,(3,'Bibesh')
begin transaction
update tmp set name ='Harish' where id=2
And I'm trying to get in another query window
select * from tmp where id=1
It is locking the table so it is not giving any records until I rollback or commits the first one
I tried
ALTER DATABASE db
SET READ_COMMITTED_SNAPSHOT On
ALTER DATABASE db
SET ALLOW_SNAPSHOT_ISOLATION on
It not locks the table but it gives old value for id =2
select * from tmp where id=2
returns me Bijesh where I'm expecting a locking
I'm expecting a way like if id=1 it will work fine, but if id =2 it will wait until the other transaction overs.
Hopes your help.....
Thanks in advance
Binesh Nambiar C
This is not a deadlock you're experiencing.
Your second query is blocked because it has to scan the entire table. During the scan, it encountered the row being updated (exclusively locked), so it waits until the lock is released (ie transaction ends).
If engine knew that the id column is unique (primary key or unique constraint), you will not get blocking here, because it wouldn't have to do scan on entire table but would rather stop on first match.
Keep ypur transactions short, provide alternative access paths to data (indexes) and try not use "select *".
Also, think carefully whether you really want to use the READ UNCOMMITTED isolation level.
I'm currently working on a project where I insert or update a lot of data frequently to a remote database. The data volume is around 50 sets of data with 500-800 rows each, that goes into the same table.
Currently I have a stored procedure that I call for every row to insert or update (simplified for easier read):
ALTER PROCEDURE stat_memberstat_upsert
...
AS
BEGIN
UPDATE Memberstats ...
if(##ROWCOUNT = 0)
BEGIN
INSERT INTO Memberstats ...
END
END
This works, but as you can see it mounts to a lot of calls to the same stored procedure (worst case around 100,000 calls). I'm looking into User-defined table type, which sound like a good solution, because it decreases the calls to the database server, with a more bulk like structure. The problem is that when I look at the solutions, tutorials and documentations I find that no one mentions a way to do a insert/update routine with the table type; it's either insert or update.
Is there a way, when working with table types, to do a insert/update call?
Alternatively I have thought about two workaround solutions:
1: Using cursor
I could use a cursor to iterate through the table type value and call the stat_memberstat_upsert procedure above for each row. This will not prevent the many calls to the procedure, but since the calls are done from a local stored procedure the speed might increase.
How to do ForEach on user defined table type in SQL Server stored procedure? (answer "Why not use a cursor ???")
2: Pre validate data
Second solution is to retrieve the already inserted rows primary keys, validate them against the incoming data and sort them into 2 tables, where one is for inserts and the other one is for updates. Then execute both tables to the database. This means that I need to encapsulate this in a transaction so the table will not change during the time is take to validate and execute the insert and update.
Would any of these be a good solution?
Both the solution might slow down further.
You can simply have insert statements into a table
ALTER PROCEDURE stat_memberstat_upsert
...
AS
INSERT INTO Memberstats_temp ...
END
and have a batch process which will run at low traffic time to update/insert the Memberstats table from Memberstats_temp table after that truncating this temp table. This wouldn't be a solution if you need real time update to the table.
I have a number of tables that get updated through my app which return a lot of data or are difficult to query for changes. To get around this problem, I have created a "LastUpdated" table with a single row and have a trigger on these complex tables which just sets GetDate() against the appropriate column in the LastUpdated table:
CREATE TRIGGER [dbo].[trg_ListItem_LastUpdated] ON [dbo].[tblListItem]
FOR INSERT, UPDATE, DELETE
AS
UPDATE LastUpdated SET ListItems = GetDate()
GO
This way, the clients only have to query this table for the last updated value and then can decided whether or not they need to refresh their data from the complex tables. The complex tables are using snapshot isolation to prevent dirty reads.
In busy systems, around once a day we are getting errors writing or updating data in the complex tables due to update conflicts in "LastUpdated". Because this occurs in the statement executed by the trigger, the affected complex table fails to save data. The following error is logged:
Snapshot isolation transaction aborted due to update conflict. You
cannot use snapshot isolation to access table 'dbo.tblLastUpdated'
directly or indirectly in database 'devDB' to update, delete, or
insert the row that has been modified or deleted by another
transaction. Retry the transaction or change the isolation level for
the update/delete statement.
What should I be doing here in the trigger to prevent this failure? Can I use some kind of query hints on the trigger to avoid this - or can I just ignore errors in the trigger? Updating the data in LastUpdated is not critical, but saving the data correctly into the complex tables is.
This is probably something very simple that I have overlooked or am not aware of. As always, thanks for any info.
I would say that you should look into using Change Tracking (http://msdn.microsoft.com/en-gb/library/cc280462%28v=sql.100%29.aspx), which is lightweight builtin SQL Server functionality that you can use to monitor the fact that a table has changed, as opposed to logging each individual change (which you can also do with Change Data Capture). It needs Snapshot Isolation, which you are already using.
Because your trigger is running in your parent transaction, and your snapshot has become out of date, your whole transaction would need to start again. If this is a complex workload, maintaining this last updated data in this way would be costly.
Short answer - don't do that! Making the updated transactions dependent on one single shared row makes it prone to deadlocks and and update conflicts whole gammut of nasty things.
You can either use views to determine last update, e.g.:
SELECT
t.name
,user_seeks
,user_scans
,user_lookups
,user_updates
,last_user_seek
,last_user_scan
,last_user_lookup
,last_user_update
FROM sys.dm_db_index_usage_stats i JOIN sys.tables t
ON (t.object_id = i.object_id)
WHERE database_id = db_id()
Or, if you really insist on the solution with LastUpdate, you can implement it's update from the trigger in an autonomous transactions. Even though SQL Server doesn't support autonomous transactions, it could done using liked servers: How to create an autonomous transaction in SQL Server 2008
The schema needs to change. If you have to keep your update table, make a row for every table. That would greatly reduce your locks because each table could update their very own row and not competing for the sole row in a table.
LastUpdated
table_name (varchar(whatever)) pk
modified_date (datetime)
New Trigger for tblListItem
CREATE TRIGGER [dbo].[trg_ListItem_LastUpdated] ON [dbo].[tblListItem]
FOR INSERT, UPDATE, DELETE
AS
UPDATE LastUpdated SET modified_date = GetDate() WHERE table_name = 'tblListItem'
GO
Another option that I use a lot is having a modified_date column in every table. Then people know exactly which records to update/insert to sync with your data rather than dropping and reloading everything in the table each time one record changes or is inserted.
Alternatively, you can update the log table inside the same transaction which you use to update your complex tables inside your application & avoid the trigger altogether.
Update
You can also opt for inserting a new row instead of updating the same row in LastUpdated table. You can then query max timestamp for latest update. However, with this approach your LastUpdated table would grow each day which you need to take care of if volume of transactions is high.
two relating question: I have a table that is in use in a production or live environment. Is it appropriate and possible to begin a transaction on that table to ensure no one is using that table while it is dropped and re-generated and then committed or rolled back so that the users on live side will not notice? It may be better to delete rows and then repopulate rows. 2nd part of the question is how would I go about performing this drop or deletion of row in a transact statement, or should this be done on a tempory database table?
Thanks.
1)to lock all table when updating you can use Table Hints (see http://msdn.microsoft.com/en-us/library/ms187373%28v=SQL.100%29.aspx), for example
USE AdventureWorks;
GO
UPDATE Production.Product
WITH (TABLOCK)
SET ListPrice = ListPrice * 1.10
WHERE ProductNumber LIKE 'BK-%';
GO
2)to delete all rows in table you need to use TRUNCATE TABLE. AS BOL says, TRUNCATE TABLE always locks the table and page but not each row.
So, if somebody locks one of the record in this table, you cannot lock this table and TRUNCATE cannot begin.