There is a table with IDU (PK) and stat columns. If first bit of stat is 1 I need to set it to 0 and run some stored procedure in this case only, otherwise I do nothing.
Here is the simple query for this
DECLARE #s INT
-- get the current value of status before update
SET #s = (SELECT stat FROM myTable
WHERE IDU = 999999999)
-- check it first bit is 1
IF (#s & 1) = 1
BEGIN
-- first bit is 1, set it to 0
UPDATE myTable
SET status = Stat & ~1
WHERE IDU = 999999999
-- first bit is one, in this case we run our SP
EXEC SOME_STORED_PROCEDURE
END
But I'm not sure that this query is optimal. I heard about OUTPUT parameter for UPDATE query but I found how to get inserted value. Is there a way to get a value that was before insert?
Yes, OUTPUT clause allows you to get the previous value before the update. You need to look at deleted and inserted tables.
DELETED
Is a column prefix that specifies the value deleted by the
update or delete operation. Columns prefixed with DELETED reflect the
value before the UPDATE, DELETE, or MERGE statement is completed.
INSERTED
Is a column prefix that specifies the value added by the insert or
update operation. Columns prefixed with INSERTED reflect the value
after the UPDATE, INSERT, or MERGE statement is completed but before
triggers are executed.
-- Clear the first bit without checking what it was
DECLARE #Results TABLE (OldStat int, NewStat int);
UPDATE myTable
SET Stat = Stat & ~1
WHERE IDU = 999999999
OUTPUT
deleted.Stat AS OldStat
,inserted.Stat AS NewStat
INTO #Results
;
-- Copy data from #Results table into variables for comparison
-- Assumes that IDU is a primary key and #Results can have only one row
DECLARE #OldStat int;
DECLARE #NewStat int;
SELECT #OldStat = OldStat, #NewStat = NewStat
FROM #Results;
IF #OldStat <> #NewStat
BEGIN
EXEC SOME_STORED_PROCEDURE;
END;
Regardless of optimal, this query is not 100% safe. This is because between SET #s =... and UPDATE myTable there is no guarantee the value of stat has not been changed. If this code runs multiple times it is possible to screw up if two cases execute deadly close for the same IDU. The first thread will do ok but the second one will not, since the first would change the stat after the second read it and before update it. A select statement does not lock beyond its own execution time even on SERIALIZABLE isolation.
To be safe, you need to lock the record BEFORE read it, and to do that you need an update statement, even fake:
DECLARE #s INT
BEGIN TRANSACTION
UPDATE myTable SET stat = stat WHERE IDU = 999999999 --now you row lock your row, make sure no other thread can move along
-- get the current value of status before update
SET #s = (SELECT stat FROM myTable
WHERE IDU = 999999999)
-- check it first bit is 1
IF (#s & 1) = 1
BEGIN
-- first bit is 1, set it to 0
UPDATE myTable
SET status = Stat & ~1
WHERE IDU = 999999999
-- first bit is one, in this case we run our SP
-- COMMIT TRANSACTION here? depends on what SOME_STORED_PROCEDURE does
EXEC SOME_STORED_PROCEDURE
COMMIT TRANSACTION --i believe here you release the row lock
I am not sure what you mean by "Is there a way to get a value that was before insert" because you only update and the only data, stat, you had already read from the old record regardless if you update or not.
You could do this with an INSTEAD OF UPDATE Trigger.
I am trying to take values from a table and insert them into another table. However, there is one database column that needs to increase by 1 value each time. This value though is not an identity insert column, the value comes from another table. There is another db column that acts as a counter. I wrote a couple of things but it just isnt helping:
(121 documents)
declare #count int;
set #count=0
while #count<=121
begin
insert into cabinet..DOCUMENT_NAMES_ROBBY (TAG,TAGORDER,ACTIVE,QUEUE,REASON,FORM,DIRECT_VIEW,GLOBAL,FU_SIGN,
SIGN_X,SIGN_Y,SIGN_W,SIGN_H,ANNOTATE,doctype_id,CODE,DOC_TYPE,SET_ID,SUSPEND_DELAY,Text_Editing,Restrict_Viewing,Viewing_Expire,
Viewing_Period,DocHdrLength,DocFtrLength,DocRuleId,Outbound,SigQueue,SigReason) select TAG,TAGORDER,ACTIVE,QUEUE,REASON,FORM,
DIRECT_VIEW,GLOBAL,FU_SIGN,
SIGN_X,SIGN_Y,SIGN_W,SIGN_H,ANNOTATE,(select nextid from cabinet..Wfe_NextValue where Name='documents')+1, CODE,DOC_TYPE,'2',SUSPEND_DELAY,Text_Editing,Restrict_Viewing,Viewing_Expire,
Viewing_Period,DocHdrLength,DocFtrLength,DocRuleId,Outbound,SigQueue,SigReason from cabinet..document_names where SET_ID ='1'
update cabinet..Wfe_NextValue set NextID=NextID+1 where Name='documents'
set #count=#count+1
end
That database column is the doctype_id. Above obviously comes out wrong and puts like 14,000 rows in the table. Basically I want to take every single entry from document_names and put it in document_names_robby...except the doctype_id column should take the value from wfe_nextvalue +1, while at the same time increasing that number in that table by 1 BEFORE inserting the next document name into document_Names_Robby. Any help is appreciated
Many popular databases support sequences. For the sequence, there is a function nextval that returns the sequence value and increments the sequence counter and currval that returns the latest previously returned value, also you can set the initial value and increment. Sequences are thread safe when storing counters in the table column is not.
Rewrite your code using sequences.
Assuming that you are using SQL Server database. Use IDENTITY function
SELECT *, IDENTITY(int, 1,1) AS IDCol FROM Cabinet.DocumentNames INTO #Tab1 WHERE Set_Id = '1';
insert into cabinet..DOCUMENT_NAMES_ROBBY (TAG,TAGORDER,ACTIVE,QUEUE,REASON,FORM,DIRECT_VIEW,GLOBAL,FU_SIGN,
SIGN_X,SIGN_Y,SIGN_W,SIGN_H,ANNOTATE,doctype_id,CODE,DOC_TYPE,SET_ID,SUSPEND_DELAY,Text_Editing,Restrict_Viewing,Viewing_Expire,
Viewing_Period,DocHdrLength,DocFtrLength,DocRuleId,Outbound,SigQueue,SigReason)
SELECT * FROM #Tab1;
DROP TABLE #Tab1;
declare #count int;
set #count=0
declare #nextId int;
select #nextId= nextid from cabinet..Wfe_NextValue where Name='documents'
while #count<=121
begin
insert into cabinet..DOCUMENT_NAMES_ROBBY (TAG,TAGORDER,ACTIVE,QUEUE,REASON,FORM,DIRECT_VIEW,GLOBAL,FU_SIGN,
SIGN_X,SIGN_Y,SIGN_W,SIGN_H,ANNOTATE,doctype_id,CODE,DOC_TYPE,SET_ID,SUSPEND_DELAY,Text_Edi ting,Restrict_Viewing,Viewing_Expire,
Viewing_Period,DocHdrLength,DocFtrLength,DocRuleId,Outbound,SigQueue,SigReason) select TAG,TAGORDER,ACTIVE,QUEUE,REASON,FORM,
DIRECT_VIEW,GLOBAL,FU_SIGN,
SIGN_X,SIGN_Y,SIGN_W,SIGN_H,ANNOTATE,(select nextid from cabinet..Wfe_NextValue where Name='documents')+1, CODE,DOC_TYPE,'2',SUSPEND_DELAY,Text_Editing,Restrict_Viewing,Viewing_Expire,
Viewing_Period,DocHdrLength,DocFtrLength,DocRuleId,Outbound,SigQueue,SigReason from cabinet..document_names where SET_ID ='1'
set #count=#count+1
end
update cabinet..Wfe_NextValue set NextID=NextID+121 where Name='documents'
I have an SQL Server locking question regarding an application we have in house. The application takes submissions of data and persists them into an SQL Server table. Each submission is also assigned a special catalog number (unrelated to the identity field in the table) which is a sequential alpha numeric number. These numbers are pulled from another table and are not generated at run time. So the steps are
Insert Data into Submission Table
Grab next Unassigned Catalog
Number from Catalog Table
Assign the Catalog Number to the
Submission in the Submission table
All these steps happen sequentially in the same stored procedure.
Its, rate but sometimes we manage to get two submission at the same second and they both get assigned the same Catalog Number which causes a localized version of the Apocalypse in our company for a small while.
What can we do to limit the over assignment of the catalog numbers?
When getting your next catalog number, use row locking to protect the time between you finding it and marking it as in use, e.g.:
set transaction isolation level REPEATABLE READ
begin transaction
select top 1 #catalog_number = catalog_number
from catalog_numbers with (updlock,rowlock)
where assigned = 0
update catalog_numbers set assigned = 1 where catalog_number = :catalog_number
commit transaction
You could use an identity field to produce the catalog numbers, that way you can safely create and get the number:
insert into Catalog () values ()
set #CatalogNumber = scope_identity()
The scope_identity function will return the id of the last record created in the same session, so separate sessions can create records at the same time and still end up with the correct id.
If you can't use an identity field to create the catalog numbers, you have to use a transaction to make sure that you can determine the next number and create it without another session accessing the table.
I like araqnid's response. You could also use an insert trigger on the submission table to accomplish this. The trigger would be in the scope of the insert, and you would effectively embed the logic to assign the catalog_number in the trigger. Just wanted to put your options up here.
Here's the easy solution. No race condition. No blocking from a restrictive transaction isolation level. Probably won't work in SQL dialects other than T-SQL, though.
I assume their is some outside force at work to keep your catalog number table populated with unassigned catalog numbers.
This technique should work for you: just do the same sort of "interlocked update" that retrieves a value, something like:
update top 1 CatalogNumber
set in_use = 1 ,
#newCatalogNumber = catalog_number
from CatalogNumber
where in_use = 0
Anyway, the following stored procedure just just ticks up a number on each execution and hands back the previous one. If you want fancier value, add a computed column that applies the transform of choice to the incrementing value to get the desired value.
drop table dbo.PrimaryKeyGenerator
go
create table dbo.PrimaryKeyGenerator
(
id varchar(100) not null ,
current_value int not null default(1) ,
constraint PrimaryKeyGenerator_PK primary key clustered ( id ) ,
)
go
drop procedure dbo.GetNewPrimaryKey
go
create procedure dbo.GetNewPrimaryKey
#name varchar(100)
as
set nocount on
set ansi_nulls on
set concat_null_yields_null on
set xact_abort on
declare
#uniqueValue int
--
-- put the supplied key in canonical form
--
set #name = ltrim(rtrim(lower(#name)))
--
-- if the name isn't already defined in the table, define it.
--
insert dbo.PrimaryKeyGenerator ( id )
select id = #name
where not exists ( select *
from dbo.PrimaryKeyGenerator pkg
where pkg.id = #name
)
--
-- now, an interlocked update to get the current value and increment the table
--
update PrimaryKeyGenerator
set #uniqueValue = current_value ,
current_value = current_value + 1
where id = #name
--
-- return the new unique value to the caller
--
return #uniqueValue
go
To use it:
declare #pk int
exec #pk = dbo.GetNewPrimaryKey 'foobar'
select #pk
Trivial to mod it to return a result set or return the value via an OUTPUT parameter.
I've got a table where I need to auto-assign an ID 99% of the time (the other 1% rules out using an identity column it seems). So I've got a stored procedure to get next ID along the following lines:
select #nextid = lastid+1 from last_auto_id
check next available id in the table...
update last_auto_id set lastid = #nextid
Where the check has to check if users have manually used the IDs and find the next unused ID.
It works fine when I call it serially, returning 1, 2, 3 ... What I need to do is provide some locking where multiple processes call this at the same time. Ideally, I just need it to exclusively lock the last_auto_id table around this code so that a second call must wait for the first to update the table before it can run it's select.
In Postgres, I can do something like 'LOCK TABLE last_auto_id;' to explicitly lock the table. Any ideas how to accomplish it in SQL Server?
Thanks in advance!
Following update increments your lastid by one and assigns this value to your local variable in a single transaction.
Edit
thanks to Dave and Mitch for pointing out isolation level problems with the original solution.
UPDATE last_auto_id WITH (READCOMMITTEDLOCK)
SET #nextid = lastid = lastid + 1
You guys have between you answered my question. I'm putting in my own reply to collate the working solution I've got into one post. The key seems to have been the transaction approach, with locking hints on the last_auto_id table. Setting the transaction isolation to serializable seemed to create deadlock problems.
Here's what I've got (edited to show the full code so hopefully I can get some further answers...):
DECLARE #Pointer AS INT
BEGIN TRANSACTION
-- Check what the next ID to use should be
SELECT #NextId = LastId + 1 FROM Last_Auto_Id WITH (TABLOCKX) WHERE Name = 'CustomerNo'
-- Now check if this next ID already exists in the database
IF EXISTS (SELECT CustomerNo FROM Customer
WHERE ISNUMERIC(CustomerNo) = 1 AND CustomerNo = #NextId)
BEGIN
-- The next ID already exists - we need to find the next lowest free ID
CREATE TABLE #idtbl ( IdNo int )
-- Into temp table, grab all numeric IDs higher than the current next ID
INSERT INTO #idtbl
SELECT CAST(CustomerNo AS INT) FROM Customer
WHERE ISNUMERIC(CustomerNo) = 1 AND CustomerNo >= #NextId
ORDER BY CAST(CustomerNo AS INT)
-- Join the table with itself, based on the right hand side of the join
-- being equal to the ID on the left hand side + 1. We're looking for
-- the lowest record where the right hand side is NULL (i.e. the ID is
-- unused)
SELECT #Pointer = MIN( t1.IdNo ) + 1 FROM #idtbl t1
LEFT OUTER JOIN #idtbl t2 ON t1.IdNo + 1 = t2.IdNo
WHERE t2.IdNo IS NULL
END
UPDATE Last_Auto_Id SET LastId = #NextId WHERE Name = 'CustomerNo'
COMMIT TRANSACTION
SELECT #NextId
This takes out an exclusive table lock at the start of the transaction, which then successfully queues up any further requests until after this request has updated the table and committed it's transaction.
I've written a bit of C code to hammer it with concurrent requests from half a dozen sessions and it's working perfectly.
However, I do have one worry which is the term locking 'hints' - does anyone know if SQLServer treats this as a definite instruction or just a hint (i.e. maybe it won't always obey it??)
How is this solution? No TABLE LOCK is required and works perfectly!!!
DECLARE #NextId INT
UPDATE Last_Auto_Id
SET #NextId = LastId = LastId + 1
WHERE Name = 'CustomerNo'
SELECT #NextId
Update statement always uses a lock to protect its update.
You might wanna consider deadlocks. This usually happens when multiple users use the stored procedure simultaneously. In order to avoid deadlock and make sure every query from the user will succeed you will need to do some handling during update failures and to do this you will need a try catch. This works on Sql Server 2005,2008 only.
DECLARE #Tries tinyint
SET #Tries = 1
WHILE #Tries <= 3
BEGIN
BEGIN TRANSACTION
BEGIN TRY
-- this line updates the last_auto_id
update last_auto_id set lastid = lastid+1
COMMIT
BREAK
END TRY
BEGIN CATCH
SELECT ERROR_NUMBER() AS ErrorNumber, ERROR_MESSAGE() as ErrorMessage
ROLLBACK
SET #Tries = #Tries + 1
CONTINUE
END CATCH
END
I prefer doing this using an identity field in a second table. If you make lastid identity then all you have to do is insert a row in that table and select #scope_identity to get your new value and you still have the concurrency safety of identity even though the id field in your main table is not identity.
I am working on a work queueing solution. I want to query a given row in the database, where a status column has a specific value, modify that value and return the row, and I want to do it atomically, so that no other query will see it:
begin transaction
select * from table where pk = x and status = y
update table set status = z where pk = x
commit transaction
--(the row would be returned)
it must be impossible for 2 or more concurrent queries to return the row (one query execution would see the row while its status = y) -- sort of like an interlocked CompareAndExchange operation.
I know the code above runs (for SQL server), but will the swap always be atomic?
I need a solution that will work for SQL Server and Oracle
Is PK the primary key? Then this is a non issue, if you already know the primary key there is no sport. If pk is the primary key, then this begs the obvious question how do you know the pk of the item to dequeue...
The problem is if you don't know the primary key and want to dequeue the next 'available' (ie. status = y) and mark it as dequeued (delete it or set status = z).
The proper way to do this is to use a single statement. Unfortunately the syntax differs between Oracle and SQL Server. The SQL Server syntax is:
update top (1) [<table>]
set status = z
output DELETED.*
where status = y;
I'm not familiar enough with Oracle's RETURNING clause to give an example similar to SQL's OUTPUT one.
Other SQL Server solutions require lock hints on the SELECT (with UPDLOCK) to be correct.
In Oracle the preffered avenue is use the FOR UPDATE, but that does not work in SQL Server since FOR UPDATE is to be used in conjunction with cursors in SQL.
In any case, the behavior you have in the original post is incorrect. Multiple sessions can all select the same row(s) and even all update it, returning the same dequeued item(s) to multiple readers.
As a general rule, to make an operation like this atomic you'll need to ensure that you set an exclusive (or update) lock when you perform the select so that no other transaction can read the row before your update.
The typical syntax for this is something like:
select * from table where pk = x and status = y for update
but you'd need to look it up to be sure.
I have some applications that follow a similar pattern. There is a table like yours that represents a queue of work. The table has two extra columns: thread_id and thread_date. When the app asks for work froom the queue, it submits a thread id. Then a single update statement updates all applicable rows with the thread id column with the submitted id and the thread date column with the current time. After that update, it selects all rows with that thread id. This way you dont need to declare an explicit transaction. The "locking" occurs in the initial update.
The thread_date column is used to ensure that you do not end up with orphaned work items. What happens if items are pulled from the queue and then your app crashes? You have to have the ability to try those work items again. So you might grab all items off the queue that have not been marked completed but have been assigned to a thread with a thread date in the distant past. Its up to you to define "distant."
Try this. The validation is in the UPDATE statement.
Code
IF EXISTS (SELECT * FROM sys.tables WHERE name = 't1')
DROP TABLE dbo.t1
GO
CREATE TABLE dbo.t1 (
ColID int IDENTITY,
[Status] varchar(20)
)
GO
DECLARE #id int
DECLARE #initialValue varchar(20)
DECLARE #newValue varchar(20)
SET #initialValue = 'Initial Value'
INSERT INTO dbo.t1 (Status) VALUES (#initialValue)
SELECT #id = SCOPE_IDENTITY()
SET #newValue = 'Updated Value'
BEGIN TRAN
UPDATE dbo.t1
SET
#initialValue = [Status],
[Status] = #newValue
WHERE ColID = #id
AND [Status] = #initialValue
SELECT ColID, [Status] FROM dbo.t1
COMMIT TRAN
SELECT #initialValue AS '#initialValue', #newValue AS '#newValue'
Results
ColID Status
----- -------------
1 Updated Value
#initialValue #newValue
------------- -------------
Initial Value Updated Value