Optimistic concurrency while in transaction - sql

In trying to fix data errors due to concurrency conflicts I realized I'm not completely sure how optimistic concurrency works in SQL Server. Assume READ_COMMITTED isolation level. A similar example:
BEGIN TRAN
SELECT * INTO #rows FROM SourceTable s WHERE s.New = 1
UPDATE d SET Property = 'HelloWorld' FROM DestinationTable d INNER JOIN #rows r ON r.Key = d.Key
UPDATE s SET Version = GenerateRandomVersion() FROM SourceTable s
INNER JOIN #rows r on r.Key = s.Key AND r.Version = s.Version
IF ##ROWCOUNT <> SELECT COUNT(*) FROM #rows
RAISEERROR
END IF
COMMIT TRAN
Is this completely atomic / thread safe?
The ON clause on UPDATE s should prevent concurrent updates via the Version and ROWCOUNT check. But is that really true? What about the following, similar query?
BEGIN TRAN
SELECT * INTO #rows FROM SourceTable s WHERE s.New = 1
UPDATE s SET New = 0 AND Version = GenerateRandomVersion() FROM SourceTable s
INNER JOIN #rows r on r.Key = s.Key AND r.Version = s.Version
IF ##ROWCOUNT <> SELECT COUNT(*) FROM #rows
RAISEERROR
END IF
UPDATE d SET Property = 'HelloWorld' FROM DestinationTable d INNER JOIN #rows r ON r.Key = d.Key
COMMIT TRAN
My worry here is that concurrent execution of the above script will reach the UPDATE s statement, get a ##ROWCOUNT that is transient / not actually committed to DB yet, so both threads / executions will continue past the IF statement and perform the important UPDATE d statement, which in this case is idempotent but not so in my original production case.

I think what you want to do is remove the very small race condition in your script by making it as set based as possible, e.g.
BEGIN TRAN
DECLARE #UpdatedSources Table (Key INT NOT NULL);
UPDATE s SET New = 0
FROM SourceTable s
WHERE s.New = 1
OUTPUT Inserted.Key into #UpdatedSources
UPDATE d SET Property = 'HelloWorld'
FROM DestinationTable d
INNER JOIN #UpdatedSources r ON r.Key = d.Key
COMMIT TRAN
I think the 'version' column in your table is confusing things - you're trying to build atomicity into your table rather than just letting the DB transactions handle that. With the script above, the rows where New=1 will be locked until the transaction commits, so subsequent attempts will only find either 'actually' new rows or rows where new=0.
Update after comment
To demonstrate the locking on the table, if it is something you want to see, then you could try and initiate a deadlock. If you were to run this query concurrently with the first one, I think you may eventually deadlock, though depending on how quickly these run, you may struggle to see it:
BEGIN TRAN
SELECT *
FROM DestinationTable d
INNER JOIN SourceTable ON r.Key = d.Key
WHERE s.New = 1
UPDATE s SET New = 0
FROM SourceTable s
WHERE s.New = 1
COMMIT TRAN

Related

SQL Trigger insert problems

I want to make a trigger where I can check if the value from the stock of a product 0 is.
Then the trigger should make the value 1.
My Trigger
CREATE TRIGGER [IfStockIsNull]
ON [dbo].[Products]
FOR DELETE, INSERT, UPDATE
AS
BEGIN
if [Products].Stock = 0
UPDATE [Products] SET [Stock] = 1
FROM [Products] AS m
INNER JOIN inserted AS i
ON m.ProductId = i.ProductId;
ELSE
raiserror('Niks aan het handje',10,16);
END
I'm getting the error:
Altering [dbo].[IfStockIsNull]...
(53,1): SQL72014: .Net SqlClient Data Provider: Msg 4104, Level 16, State 1,
Procedure IfStockIsNull, Line 7 The multi-part identifier "Products.Stock"
could not be bound.
(47,0): SQL72045: Script execution error. The executed script:
ALTER TRIGGER [IfStockIsNull]
ON [dbo].[Products]
FOR DELETE, INSERT, UPDATE
AS BEGIN
IF [Products].Stock = 0
UPDATE [Products]
SET [Stock] = 1
FROM [Products] AS m
INNER JOIN
inserted AS i
ON m.ProductId = i.ProductId;
ELSE
RAISERROR ('Niks aan het handje', 10, 16);
END
An error occurred while the batch was being executed.
Maybe you guys can help me out?
A number of issues and surprises in what you're trying to run here. But basically, don't try to run procedural steps when you're dealing with sets. inserted can contain 0, 1 or multiple rows, so which row's stock are you asking about in your IF?
Better to deal with it in the WHERE clause:
CREATE TRIGGER [IfStockIsNull]
ON [dbo].[Products]
FOR INSERT, UPDATE --Removed DELETE, because ?!?
AS
BEGIN
UPDATE [Products] SET [Stock] = 1
FROM [Products] AS m
INNER JOIN inserted AS i
ON m.ProductId = i.ProductId;
WHERE m.Stock = 0
--Not sure what the error is for - the above update may have updated
--some number of rows, between 0 and the number in inserted.
--What circumstances should produce an error then?
END
Simple demo script that the UPDATE is correctly targetting only matching rows from inserted:
declare #t table (ID int not null, Val int not null)
insert into #t(ID,Val) values (1,1),(2,2),(3,3)
update
#t
set
Val = 4
from
#t t
inner join
(select 2 as c) n
on
t.ID = n.c
select * from #t
Shows that only the row with ID of 2 is updated.
Even Microsoft's own example of UPDATE ... FROM syntax uses the UPDATE <table> ... FROM <table> <alias> ... syntax that Gordon asserts doesn't work:
USE AdventureWorks2012;
GO
UPDATE Sales.SalesPerson
SET SalesYTD = SalesYTD + SubTotal
FROM Sales.SalesPerson AS sp
JOIN Sales.SalesOrderHeader AS so
ON sp.BusinessEntityID = so.SalesPersonID
AND so.OrderDate = (SELECT MAX(OrderDate)
FROM Sales.SalesOrderHeader
WHERE SalesPersonID = sp.BusinessEntityID);
GO
This example does have an issue which is further explained below it but that relates to the "multiple matching rows in other tables will result in only a single update" flaw that anyone working with UPDATE ... FROM should make themselves aware of.
If you want to stop the insert if any rows are bad, then do that before making the changes:
ALTER TRIGGER [IfStockIsNull] ON [dbo].[Products]
FOR INSERT, UPDATE -- DELETE is not really appropriate
AS BEGIN
IF (EXISTS (SELECT 1
FROM Products p JOIN
inserted i
ON m.ProductId = i.ProductId
WHERE p.Stock = 0
)
)
BEGIN
RAISERROR ('Niks aan het handje', 10, 16);
END;
UPDATE p
SET Stock = 1
FROM Products p INNER JOIN
inserted AS i
ON p.ProductId = i.ProductId;
END;
Notes:
The comparison in the IF checks if any of the Product rows have a 0. If so, the entire transaction is rolled back.
You might complain that you cannot reject a single row. But that is how transactions work. If any row fails in an INSERT, the entire INSERT is rolled back, not just a single row.
The UPDATE is not correct, because you have Products in the FROM clause and the UPDATE clause. You need to use the alias for the first one.

Delete more than 6 million rows so SLOW

I have a SP that needs to delete more than 6 million rows.
I tried this approach but still the execution time is SLOW.
DECLARE #continue BIT = 1
-- delete all ids not between starting and ending ids
WHILE #continue = 1
BEGIN
SET #continue = 0
DELETE top (10000) u
FROM Table1 u WITH (READPAST)
WHERE ID = #ID
AND NID IN (SELECT NID FROM #Node GROUP BY NID)
IF ##ROWCOUNT > 0
SET #continue = 1
end
Any other suggestions?
Your loop logic looks OK
As other have said look at application design
Can you use staging tables and be able to truncate
If the table has FK and you are confident you are not going to violate them then disable the FK and re-enable
If you have a delete trigger and you need to delete 6 million every day then you really need to reevaluate the design
Look at optimizing the select then put it in the delete
select top (10000) u
FROM Table1 u WITH (READPAST)
JOIN #Node
on u.NIT = #Node.NIT
and u.ID = #ID

SQL Server : trigger deadlock

There is an InventoryCategory table and I have the following trigger attached to that table. The main purpose of trigger is to update the ProcessQueue table with InventoryCategory RowID when someone update or insert or delete a record in InventoryCategory table so that I know which records are updated recently and then I need to update other systems.
I have a multi threaded C# application updating InventoryCategory and this trigger is causing a deadlock. If I run the it with single thread I don't have any error.
CREATE trigger [dbo].[tr_InventoryCategory_Queue]
On [dbo].[InventoryCategory]
After Insert, Update, Delete
as
if ##ROWCOUNT = 0
return
-- inserted records need to be either inserted or deleted into the queue
if exists(select 1 from inserted)
begin
-- update existing queue records
update ProcessQueue
set ParentTable = 'InventoryCategory', UpdatedTime = getdate()
from inserted i
join ProcessQueue p on i.Id = p.RowID
where exists (select * from ProcessQueue q
where q.RowID = p.RowID and q.ParentTable = 'InventoryCategory')
and p.ParentTable = 'InventoryCategory'
-- insert new queue records
insert into ProcessQueue (ParentTable, RowID)
select
'InventoryCategory', i.id
from
inserted i
where
i.ID not in (select q.RowID from ProcessQueue q
where q.ParentTable = 'InventoryCategory')
end
-- deleted records need to be either inserted or updated into the queue
if exists(select 1 from deleted)
begin
-- update existing queue records
update ProcessQueue
set ParentTable = 'InventoryCategory', UpdatedTime = getdate()
from deleted d
join ProcessQueue p on d.Id = p.RowID
where exists (select * from ProcessQueue q
where q.RowID = p.RowID and q.ParentTable = 'InventoryCategory')
and p.ParentTable = 'InventoryCategory'
-- insert new queue records
insert into ProcessQueue (ParentTable, RowID)
select
'InventoryCategory', d.Id
from
deleted d
where
d.Id not in (select q.RowID from ProcessQueue q
where q.ParentTable = 'InventoryCategory')
end
Any suggestions?
Thanks in advance

SQL Server 2008 R2 Trigger multiple rows in a single batch update issue

I have implemented a trigger in SQL Server 2008 R2 in this way -
ALTER TRIGGER [dbo].[trgtblOrgStaffAssocLastUpdate] ON [dbo].[tblOrgStaffAssoc]
AFTER UPDATE
AS
IF EXISTS (SELECT i.* FROM INSERTED i inner join deleted d on i.ORG_ID = d.ORG_ID and isnull(i.StaffType, -1111) = isnull(d.StaffType, -1111)
where i.Deleted = 0
and (isnull(i.PER_ID, cast(cast(0 as binary) as uniqueidentifier)) <> isnull(d.PER_ID, cast(cast(0 as binary) as uniqueidentifier))
))
BEGIN
IF EXISTS (SELECT * FROM DELETED)
BEGIN
--UPDATE PER_ID
update l
set PER_ID = 1, UpdatedOn = GETDATE()
from dbo.tblOrgStaffAssocLastUpadate l inner join [dbo].[inserted] i on l.ORG_ID = i.ORG_ID and l.StaffType = i.StaffType
inner join [dbo].[deleted] d on i.ORG_ID = d.ORG_ID and i.StaffType = d.StaffType
where isnull(i.PER_ID, cast(cast(0 as binary) as uniqueidentifier)) <> isnull(d.PER_ID, cast(cast(0 as binary) as uniqueidentifier))
END
END
My purpose is to update tblOrgStaffAssocLastUpadate when tblOrgStaffAssoc is updated. For only one row, it works fine. However for multiple rows send over in one batch, it updates one row only in tblOrgStaffAssocLastUpadate while tblOrgStaffAssoc has multiple rows updated.
When I use a intermediate tables _Inserted and _deleted to buffer the INSERTED and DELETED data and use the permanent tables to update like this way -
ALTER TRIGGER [dbo].[trgtblOrgStaffAssocLastUpdate] ON [dbo].[tblOrgStaffAssoc]
AFTER UPDATE
AS
IF EXISTS (SELECT i.* FROM INSERTED i inner join deleted d on i.ORG_ID = d.ORG_ID and isnull(i.StaffType, -1111) = isnull(d.StaffType, -1111)
where i.Deleted = 0
and (isnull(i.PER_ID, cast(cast(0 as binary) as uniqueidentifier)) <> isnull(d.PER_ID, cast(cast(0 as binary) as uniqueidentifier))
))
BEGIN
IF EXISTS (SELECT * FROM DELETED)
BEGIN
IF EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[_inserted]') AND type in (N'U'))
insert into [dbo].[_inserted]
select * from inserted
else
select * into [dbo].[_inserted]
from inserted
IF EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[_deleted]') AND type in (N'U'))
insert into [dbo].[_deleted]
select * from deleted
else
select * into [dbo].[_deleted]
from deleted
--UPDATE PER_ID
update l
set PER_ID = 1, UpdatedOn = GETDATE()
from dbo.tblOrgStaffAssocLastUpadate l inner join [dbo].[_inserted] i on l.ORG_ID = i.ORG_ID and l.StaffType = i.StaffType
inner join [dbo].[_deleted] d on i.ORG_ID = d.ORG_ID and i.StaffType = d.StaffType
where isnull(i.PER_ID, cast(cast(0 as binary) as uniqueidentifier)) <> isnull(d.PER_ID, cast(cast(0 as binary) as uniqueidentifier))
END
END
It works fine. Changing permanent table _Inserted to use #inserted or table variable #inserted does not work either.
Apparently using permanent table is not a good idea. I don't know how the trigger works in this way. Can anyone help out?
Thanks
Edit to answer #usr's comment -
It does not work correctly if I use -
update l set PER_ID = 1, UpdatedOn = GETDATE() from dbo.tblOrgStaffAssocLastUpadate l inner join [dbo].[inserted] i on l.ORG_ID = i.ORG_ID and l.StaffType = i.StaffType inner join [dbo].[deleted] d on i.ORG_ID = d.ORG_ID and i.StaffType = d.StaffType where isnull(i.PER_ID, cast(cast(0 as binary) as uniqueidentifier)) <> isnull(d.PER_ID, cast(cast(0 as binary) as uniqueidentifier))
Only one row is updated. The rest rows are not updated at all. Even though I can see multiple rows returned if I use
select l.* from dbo.tblOrgStaffAssocLastUpadate l inner join [dbo].[inserted] i on l.ORG_ID = i.ORG_ID and l.StaffType = i.StaffType inner join [dbo].[deleted] d on i.ORG_ID = d.ORG_ID and i.StaffType = d.StaffType where isnull(i.PER_ID, cast(cast(0 as binary) as uniqueidentifier)) <> isnull(d.PER_ID, cast(cast(0 as binary) as uniqueidentifier))
right before the update statement. That is where I am totally confused why the next update statement only updates one row only, and why only permanent table can persist all the rows update but not temporary table and table variables.
Updated question -
Since multiple update rows are send to the table in a single batch. It seems that the trigger's INSERT and DELETE table only one and the last one update row only at the time when I reach " update l " statement. Before that it holds multiple update rows. I can see that when I use permanent tables. I just don't understand SQL Server behave in this way. Anyone saw the same thing?
You are seriously over-complicating this. You do not need all those checks and links between INSERTED and DELETED.
This is a trigger on UPDATE only, therefore, there will always be a DELETED table that contains for each changed row what was in the table and there will always be an INSERTED that contains for each changed row what will be in the table. So for your purposes you only need to deal with one of these tables. Rows that are not changed are in neither table.
If this was a trigger on INSERT there would only be an INSERTED table. Similarly, a DELETE trigger only has a DELETED table.
Secondly, you seem to think that NULL=NULL is true - it isn't. The statement NULL=NULL returns NULL. So does NULL<>NULL, NULL>=NULL etc. etc. In a database NULL means not a value and something that does not have a value is incomparable. So your where statement is also superfluous.
So I think the code you want is:
ALTER TRIGGER [dbo].[trgtblOrgStaffAssocLastUpdate] ON [dbo].[tblOrgStaffAssoc]
AFTER UPDATE
AS
UPDATE l
SET PER_ID = 1
,UpdatedOn = GETDATE()
FROM dbo.tblOrgStaffAssocLastUpadate l
INNER JOIN
inserted i ON l.ORG_ID = i.ORG_ID and l.StaffType = i.StaffType

How can I do a SQL UPDATE in batches, like an Update Top?

Is it possible to add a TOP or some sort of paging to a SQL Update statement?
I have an UPDATE query, that comes down to something like this:
UPDATE XXX SET XXX.YYY = #TempTable.ZZZ
FROM XXX
INNER JOIN (SELECT SomeFields ... ) #TempTable ON XXX.SomeId=#TempTable.SomeId
WHERE SomeConditions
This update will affect millions of records, and I need to do it in batches. Like 100.000 at the time (the ordering doesn't matter)
What is the easiest way to do this?
Yes, I believe you can use TOP in an update statement, like so:
UPDATE TOP (10000) XXX SET XXX.YYY = #TempTable.ZZZ
FROM XXX
INNER JOIN (SELECT SomeFields ... ) #TempTable ON XXX.SomeId=#TempTable.SomeId
WHERE SomeConditions
You can use SET ROWCOUNT { number | #number_var } it limits number of rows processed before stopping the specific query, example below:
SET ROWCOUNT 10000 -- define maximum updated rows at once
UPDATE XXX SET
XXX.YYY = #TempTable.ZZZ
FROM XXX
INNER JOIN (SELECT SomeFields ... ) #TempTable ON XXX.SomeId = #TempTable.SomeId
WHERE XXX.YYY <> #TempTable.ZZZ and OtherConditions
-- don't forget about bellow
-- after everything is updated
SET ROWCOUNT 0
I've added XXX.YYY <> #TempTable.ZZZ to where clause to make sure you will not update twice already updated value.
Setting ROWCOUNT to 0 turn off limits - don't forget about it.
You can do something like the following
declare #i int = 1
while #i <= 10 begin
UPDATE top (10) percent
masterTable set colToUpdate = lt.valCol
from masterTable as mt
inner join lookupTable as lt
on mt.colKey = lt.colKey
where colToUpdate is null
print #i
set #i += 1
end
--one final update without TOP (assuming lookupTable.valCol is mostly not null)
UPDATE --top (10) percent
masterTable set colToUpdate = lt.valCol
from masterTable as mt
inner join lookupTable as lt
on mt.colKey = lt.colKey
where colToUpdate is null
Depending on your ability to change the datastructure of the table, I would suggest that you add a field to your table that can hold some sort of batch-identificator. Ie. it can be a date-stamp if you do it daily, an incremenal value or basically any value that you can make unique for your batch. If you take the incremental approach, your update will then be:
UPDATE TOP (100000) XXX SET XXX.BATCHID = 1, XXX.YYY = ....
...
WHERE XXX.BATCHID < 1
AND (rest of WHERE-clause here).
Next time, you'll set the BATCHID = 2 and WHERE XXX.BATCHID < 2
If this is to be done repeatedly, you can set an index on the BATCHID and reduce load on the server.
DECLARE #updated_Rows INT;
SET #updated_Rows = 1;
WHILE (#updated_Rows > 0)
BEGIN
UPDATE top(10000) XXX SET XXX.YYY = #TempTable.ZZZ FROM XXX
INNER JOIN #TempTable ON XXX.SomeId=#TempTable.SomeId
WHERE SomeConditions
SET #updated_Rows = ##ROWCOUNT;
END