This seemed so easy at first but everything I know seems to be wrong again.
Looking at PAQ's the consensus seems to be that EXEC does not start an implicit transaction, you can test this doing :
create procedure usp_foo
as
begin
select ##trancount;
end
go
exec usp_foo;
which returns 0.
If you however step through this with the T-SQL debugger ##Transaction is in fact 1 inside the procedure according to the watch, although it does return 0 ...
So I think this is a side-effect of the debugger, but then I write some code to test it, do an update in a table and then select max (id) out the, the classic :
create table nextid
(
id int
)
insert into nextid values (0)
create procedure nextid
as
BEGIN
UPDATE nextid set id = id + 1
select max(id) from nextid
END
So I am expecting this to give out duplicate id's as executed in parallel 2 updates can complete before the 2 selects fetching the last id and returning the same value, but try as I might hitting it from multiple machines I cannot make it break. When monitoring the locks and transactions on the machine it reports that the exec is happening in a transaction and importantly, all the statements inside the exec are treated as one unit of work/one transaction.
I would understand if the update was in a transaction and this was the cause of the lock, but the lock seems to remain until after the select.
If I trace with profiler I can see that a transaction ID is provided for the entire execution of the EXEC statement, and transaction ID is not 0 as I would expect while executing ...
Can someone please explain to me where I am missing the plot or am I wrong and it is in fact safe to generate ID's like that?
Your test must be giving you correct results because you are not fast enough to invoke the second call in between these two statements. Try adding a delay and you can see that the test would start failing.
CREATE TABLE NextID
(
ID int
)
GO
INSERT INTO NextID VALUES (0)
GO
CREATE PROC GetNextID
AS
BEGIN
UPDATE NextID SET ID = ID + 1
WAITFOR DELAY '00:00:05'
SELECT Max(ID) FROM NextID
END
Issue an EXEC GetNextID and issue another EXEC GetNextID as soon as you can from another session. About 5 seconds later both EXEC would return same result i.e. incorrect value. Now change the SP to
CREATE PROC GetNextID
AS
BEGIN
BEGIN TRAN
UPDATE NextID SET ID = ID + 1
WAITFOR DELAY '00:00:05'
SELECT Max(ID) FROM NextID
COMMIT TRAN
END
and repeat the above test. You would see that both calls would return correct value. In addition second call (if issued as soon as possible) would return result approximately in 10 seconds because the UPDATE is blocked and has to wait 5 seconds (for the first call to COMMIT) and then its own 5 second wait.
Related
There is a table with IDU (PK) and stat columns. If first bit of stat is 1 I need to set it to 0 and run some stored procedure in this case only, otherwise I do nothing.
Here is the simple query for this
DECLARE #s INT
-- get the current value of status before update
SET #s = (SELECT stat FROM myTable
WHERE IDU = 999999999)
-- check it first bit is 1
IF (#s & 1) = 1
BEGIN
-- first bit is 1, set it to 0
UPDATE myTable
SET status = Stat & ~1
WHERE IDU = 999999999
-- first bit is one, in this case we run our SP
EXEC SOME_STORED_PROCEDURE
END
But I'm not sure that this query is optimal. I heard about OUTPUT parameter for UPDATE query but I found how to get inserted value. Is there a way to get a value that was before insert?
Yes, OUTPUT clause allows you to get the previous value before the update. You need to look at deleted and inserted tables.
DELETED
Is a column prefix that specifies the value deleted by the
update or delete operation. Columns prefixed with DELETED reflect the
value before the UPDATE, DELETE, or MERGE statement is completed.
INSERTED
Is a column prefix that specifies the value added by the insert or
update operation. Columns prefixed with INSERTED reflect the value
after the UPDATE, INSERT, or MERGE statement is completed but before
triggers are executed.
-- Clear the first bit without checking what it was
DECLARE #Results TABLE (OldStat int, NewStat int);
UPDATE myTable
SET Stat = Stat & ~1
WHERE IDU = 999999999
OUTPUT
deleted.Stat AS OldStat
,inserted.Stat AS NewStat
INTO #Results
;
-- Copy data from #Results table into variables for comparison
-- Assumes that IDU is a primary key and #Results can have only one row
DECLARE #OldStat int;
DECLARE #NewStat int;
SELECT #OldStat = OldStat, #NewStat = NewStat
FROM #Results;
IF #OldStat <> #NewStat
BEGIN
EXEC SOME_STORED_PROCEDURE;
END;
Regardless of optimal, this query is not 100% safe. This is because between SET #s =... and UPDATE myTable there is no guarantee the value of stat has not been changed. If this code runs multiple times it is possible to screw up if two cases execute deadly close for the same IDU. The first thread will do ok but the second one will not, since the first would change the stat after the second read it and before update it. A select statement does not lock beyond its own execution time even on SERIALIZABLE isolation.
To be safe, you need to lock the record BEFORE read it, and to do that you need an update statement, even fake:
DECLARE #s INT
BEGIN TRANSACTION
UPDATE myTable SET stat = stat WHERE IDU = 999999999 --now you row lock your row, make sure no other thread can move along
-- get the current value of status before update
SET #s = (SELECT stat FROM myTable
WHERE IDU = 999999999)
-- check it first bit is 1
IF (#s & 1) = 1
BEGIN
-- first bit is 1, set it to 0
UPDATE myTable
SET status = Stat & ~1
WHERE IDU = 999999999
-- first bit is one, in this case we run our SP
-- COMMIT TRANSACTION here? depends on what SOME_STORED_PROCEDURE does
EXEC SOME_STORED_PROCEDURE
COMMIT TRANSACTION --i believe here you release the row lock
I am not sure what you mean by "Is there a way to get a value that was before insert" because you only update and the only data, stat, you had already read from the old record regardless if you update or not.
You could do this with an INSTEAD OF UPDATE Trigger.
It is old question but to be sure i am asking again.
Actually i have created sequence like in Oracle with a table and want to use with multiple threads and multiple JVM's all process will be hitting it parallel.
Following is sequence stored procedure just want to ask whether this will work with multiple JVM's and always provide unique number to threads in all jvm's or is there any slight chance of it returning same sequence number two more than one calls?
create table sequenceTable (id int)
insert into sequenceTable values (0)
create procedure mySequence
AS
BEGIN
declare #seqNum int
declare #rowCount int
select #rowCount = 0
while(#rowCount = 0)
begin
select #seqNum = id from sequenceTable
update sequenceTable set id = id + 1 where id = #seqNum
select #rowCount = ##rowcount
print 'numbers of rows update %1!', #rowCount
end
SELECT #seqNum
END
If you choose to maintain your current design of updating the sequenceTable.id column each time you want to generate a new sequence number, you need to make sure:
the 'current' process gets an exclusive lock on the row containing the desired sequence number
the 'current' process then updates the desired row and retrieves the newly updated value
the 'current' process releases the exclusive lock
While the above can be implemented via a begin tran + update + select + commit tran, it's actually a bit easier with a single update statement, eg:
create procedure mySequence
AS
begin
declare #seqNum int
update sequenceTable
set #seqNum = id + 1,
id = id + 1
select #seqNum
end
The update statement is its own transaction so the update of the id column and the assignment of #seqNum = id + 1 is performed under an exclusive lock within the update's transaction.
Keep in mind that the exclusive lock will block other processes from obtaining a new id value; net result is that the generation of new id values will be single-threaded/sequential
While this is 'good' from the perspective of ensuring all processes obtain a unique value, it does mean this particular update statement becomes a bottleneck if you have multiple processes hitting the update concurrently.
In such a situation (high volume of concurrent updates) you could alleviate some contention by calling the stored proc less often; this could be accomplished by having the calling processes request a range of new id values (eg, pass #increment as input parameter to the proc, then instead of id + 1 you use id + #increment), with the calling process then knowing it can use sequence numbers (#seqNum-#increment+1) to #seqNum.
Obviously (?) any process that uses a stored proc to generate 'next id' values only works if *ALL* processes a) always call the proc for a new id value and b) *ALL* processes only use the id value returned by the proc (eg, they don't generate their own id values).
If there's a possibility of applications not following this process (call proc to get new id value), you may want to consider pushing the creation of the unique id values out to the table where these id values are being inserted; in other words, modify the target table's id column to include the identity attribute; this eliminates the need for applications to call the stored proc (to generate a new id) and it (still) ensures a unique id is generated for each insert.
You can emulate sequences in ASE. Use the reserve_identity function to achieve required type of activity:
create table sequenceTable (id bigint identity)
go
create procedure mySequence AS
begin
select reserve_identity('sequenceTable', 1)
end
go
This solution is non-blocking and does generate minimal transaction log activity.
I have implemented a SQL tablockx locking in a Procedure.It is working fine when this is running on one server.But duplicate policyNumber occurs when request comes from two different servers.
declare #plantype varchar(max),
#transid varchar(max),
#IsStorySolution varchar(10),
#outPolicyNumber varchar(max) output,
#status int output -- 0 mean error and 1 means success
)
as
begin
BEGIN TRANSACTION
Declare #policyNumber varchar(100);
Declare #polseqid int;
-- GET POLICY NUMBER ON THE BASIS OF STORY SOLUTION..
IF (UPPER(#IsStorySolution)='Y')
BEGIN
select top 1 #policyNumber=Policy_no from PLAN_POL_NO with (tablockx, holdlock) where policy_no like '9%'
and pol_id_seq is null and status='Y';
END
ELSE
BEGIN
select top 1 #policyNumber=pp.Policy_no from PLAN_POL_NO pp with (tablockx, holdlock) ,PLAN_TYP_MST pt where pp.policy_no like PT.SERIES+'%'
and pt.PLAN_TYPE in (''+ISNULL(#plantype,'')+'') and pol_id_seq is null and pp.status='Y'
END
-- GET POL_SEQ_ID NUMBER
select #polseqid=dbo.Sequence();
--WAITFOR DELAY '00:00:03';
set #policyNumber= ISNULL(#policyNumber,'');
-- UPDATE POLICY ID INFORMATION...
Update PLAN_POL_NO set status='N',TRANSID =#transid , POL_ID_SEQ=ISNULL(#polseqid,0) where Policy_no =#policyNumber
set #outPolicyNumber=#policyNumber;
if(##ERROR<>0) begin GOTO Fail end
COMMIT Transaction
set #status=1;
return;
Fail:
If ##TRANCOUNT>0
begin
Rollback transaction
set #status=0;
return;
This is function which i have called::
CREATE function [dbo].[Sequence]()
returns int
as
begin
declare #POL_ID int
/***************************************
-- Schema name is added with table name in below query
-- as there are two table with same name (PLAN_POL_NO)
-- on different schema (dbo & eapp).
*****************************************/
select #POL_ID=isnull(MAX(POL_ID_SEQ),2354) from dbo.PLAN_POL_NO
return #POL_ID+1
end
The problem you are facing is because concurrent requests are both getting the same POL_ID_SEQ from your table dbo.PLAN_POL_NO.
There are multiple solutions to your problem, but I can think of two that might help you and require none/small code changes:
Using a higher transaction isolation level instead of table hints.
In your stored procedure you can use the following:
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE;
This will make sure that any data read/modified during the SP block is transactionally consistent and avoids phantom reads, duplicate records etc. It will potentially create higher deadlocks and if these tables are heavily queried/updated throughout your application you might have a whole new set of issues.
Make sure your update to the dbo.PLAN_POL_NO only succeeds if the sequence has not changed. If it has changed, error out (if it changed it means a concurrent transaction obtained the ID and completed first)
Something like this:
Update dbo.PLAN_POL_NO
SET status ='N',
TRANSID = #transid,
POL_ID_SEQ = ISNULL(#polseqid,0)
WHERE Policy_no = #policyNumber
AND POL_ID_SEW = #polseqid - 1
IF ##ROWCOUNT <> 1
BEGIN
-- Update failed, error out and let the SP rollback the transaction
END
A WITH (HOLDLOCK) option in the function might suffice.
The HOLDLOCK from the first queries may be being applied at a row or page level that then DOESN'T include the row of interest that is queried in the function.
However, the function is not reliable since it is not self-contained. I'd be looking to redesign this such that the Sequence can generate AND "take" the sequence number before returning it. The current design is fragile at best.
I've got a stored procedure that does the following (Simplified):
SET TRANSACTION ISOLATION LEVEL REPEATABLE READ
BEGIN TRANSACTION
DECLARE #intNo int
SET #intNo = (SELECT MAX(intNo) + 1 FROM tbl)
INSERT INTO tbl(intNo)
Values (#intNo)
SELECT intNo
FROM tbl
WHERE (intBatchNumber = #intNo - 1)
COMMIT TRANSACTION
My issue is that when two or more users execute this at the same time I am getting deadlocks. Now as I understand it the moment I do my first select in the proc this should create a lock in tbl. If the second procedure is then called while the first procedure is still executing it should wait for it to complete right?
At the moment this is causing a deadlock, any ideas?
The insert query requires a different lock than the select. The lock for select blocks a second insert, but it does not block a second select. So both queries can start with the select, but they both block on the other's insert.
You can solve this by asking the first query to lock the entire table:
SET #intNo = (SELECT MAX(intNo) + 1 FROM tbl with (tablockx))
^^^^^^^^^^^^^^^
This will make the second transaction's select wait for the complete first transaction to finish.
Make it simpler so you have one statement and no transaction
--BEGIN TRANSACTION not needed
INSERT INTO tbl(intNo)
OUTPUT INSERTED.intNo
SELECT MAX(intNo) + 1 FROM tbl WITH (TABLOCK)
--COMMIT TRANSACTION not needed
Although, why aren't you using IDENTITY...?
I've got a table where I need to auto-assign an ID 99% of the time (the other 1% rules out using an identity column it seems). So I've got a stored procedure to get next ID along the following lines:
select #nextid = lastid+1 from last_auto_id
check next available id in the table...
update last_auto_id set lastid = #nextid
Where the check has to check if users have manually used the IDs and find the next unused ID.
It works fine when I call it serially, returning 1, 2, 3 ... What I need to do is provide some locking where multiple processes call this at the same time. Ideally, I just need it to exclusively lock the last_auto_id table around this code so that a second call must wait for the first to update the table before it can run it's select.
In Postgres, I can do something like 'LOCK TABLE last_auto_id;' to explicitly lock the table. Any ideas how to accomplish it in SQL Server?
Thanks in advance!
Following update increments your lastid by one and assigns this value to your local variable in a single transaction.
Edit
thanks to Dave and Mitch for pointing out isolation level problems with the original solution.
UPDATE last_auto_id WITH (READCOMMITTEDLOCK)
SET #nextid = lastid = lastid + 1
You guys have between you answered my question. I'm putting in my own reply to collate the working solution I've got into one post. The key seems to have been the transaction approach, with locking hints on the last_auto_id table. Setting the transaction isolation to serializable seemed to create deadlock problems.
Here's what I've got (edited to show the full code so hopefully I can get some further answers...):
DECLARE #Pointer AS INT
BEGIN TRANSACTION
-- Check what the next ID to use should be
SELECT #NextId = LastId + 1 FROM Last_Auto_Id WITH (TABLOCKX) WHERE Name = 'CustomerNo'
-- Now check if this next ID already exists in the database
IF EXISTS (SELECT CustomerNo FROM Customer
WHERE ISNUMERIC(CustomerNo) = 1 AND CustomerNo = #NextId)
BEGIN
-- The next ID already exists - we need to find the next lowest free ID
CREATE TABLE #idtbl ( IdNo int )
-- Into temp table, grab all numeric IDs higher than the current next ID
INSERT INTO #idtbl
SELECT CAST(CustomerNo AS INT) FROM Customer
WHERE ISNUMERIC(CustomerNo) = 1 AND CustomerNo >= #NextId
ORDER BY CAST(CustomerNo AS INT)
-- Join the table with itself, based on the right hand side of the join
-- being equal to the ID on the left hand side + 1. We're looking for
-- the lowest record where the right hand side is NULL (i.e. the ID is
-- unused)
SELECT #Pointer = MIN( t1.IdNo ) + 1 FROM #idtbl t1
LEFT OUTER JOIN #idtbl t2 ON t1.IdNo + 1 = t2.IdNo
WHERE t2.IdNo IS NULL
END
UPDATE Last_Auto_Id SET LastId = #NextId WHERE Name = 'CustomerNo'
COMMIT TRANSACTION
SELECT #NextId
This takes out an exclusive table lock at the start of the transaction, which then successfully queues up any further requests until after this request has updated the table and committed it's transaction.
I've written a bit of C code to hammer it with concurrent requests from half a dozen sessions and it's working perfectly.
However, I do have one worry which is the term locking 'hints' - does anyone know if SQLServer treats this as a definite instruction or just a hint (i.e. maybe it won't always obey it??)
How is this solution? No TABLE LOCK is required and works perfectly!!!
DECLARE #NextId INT
UPDATE Last_Auto_Id
SET #NextId = LastId = LastId + 1
WHERE Name = 'CustomerNo'
SELECT #NextId
Update statement always uses a lock to protect its update.
You might wanna consider deadlocks. This usually happens when multiple users use the stored procedure simultaneously. In order to avoid deadlock and make sure every query from the user will succeed you will need to do some handling during update failures and to do this you will need a try catch. This works on Sql Server 2005,2008 only.
DECLARE #Tries tinyint
SET #Tries = 1
WHILE #Tries <= 3
BEGIN
BEGIN TRANSACTION
BEGIN TRY
-- this line updates the last_auto_id
update last_auto_id set lastid = lastid+1
COMMIT
BREAK
END TRY
BEGIN CATCH
SELECT ERROR_NUMBER() AS ErrorNumber, ERROR_MESSAGE() as ErrorMessage
ROLLBACK
SET #Tries = #Tries + 1
CONTINUE
END CATCH
END
I prefer doing this using an identity field in a second table. If you make lastid identity then all you have to do is insert a row in that table and select #scope_identity to get your new value and you still have the concurrency safety of identity even though the id field in your main table is not identity.