I have stored procedure in the sql server 2008, my stored procedure calculate and get the last number "not primary key" from column from table B and add one ( +1 ) to this number to use it on the next statement on the same stored procedure.
My issue that i have a duplicate number some times, i think this happened when multiple users call the stored procedure on the same time. is this the issue and how can i solve it
my code is like the below:-
DECLARE #ID AS NVARCHAR(10)
SET #ID = (
SELECT TOP 1 MyNo
FROM Employee
WHERE (
(TypeID = #TypeID) AND
(Year = #Year)
)
ORDER BY ID DESC
)
SET #ID = ISNULL(#ID,0) + 1
INSERT INTO Employee (name,lname,MyNo) VALUES (#name,#lname,#MyNo)
You can lock a table for the duration of a transaction with the WITH (TABLOCKX, HOLDLOCK) syntax:
BEGIN TRANSACTION
DECLARE #ID AS NVARCHAR(10)
SET #ID = (
SELECT TOP 1 MyNo
FROM Employee WITH (TABLOCKX, HOLDLOCK)
WHERE (
(TypeID = #TypeID) AND
(Year = #Year)
)
ORDER BY ID DESC
)
SET #ID = ISNULL(#ID,0) + 1
INSERT INTO Employee (name,lname,MyNo) VALUES (#name,#lname,#MyNo)
COMMIT TRANSACTION
You can find more information about TABLOCK and TABLOCKX here:
https://learn.microsoft.com/en-us/sql/t-sql/queries/hints-transact-sql-table
Per discussion, the best lock to use in this case would be:
(UPDLOCK,HOLDLOCK)
If you cannot use Identity column or the Table lock, another alternative is to use sp_getapplock
The advantage with this mechanism is that this kind of lock can be used across multiple stored procedures that should not run concurrently or for operations that span multiple tables. It also allows for handling timeout and other kinds of behavior if the lock is not available.
You have to be careful when using this feature and ensure you acquire and release locks properly or you will create more problems than you solve.
Related
I have a table with x number of rows. I want to create a stored procedure that always select a new row and return that row (when all rows has been returned it will start over from first row). My idea is to select top 1 row (ordered by a date time row) return that from the stored procedure and then set an datetime column so next time it will be a new row that is returned. It needs to be thread safe so I would expect some row locking is needed (I don't know if this is true). How would you create a stored procedure like that? I am not sure of you need to use variables or it can be done in a single query. Something like:
select top 1 *
from [dbo].[AppRegistrations]
order by LastUsed
update [dbo].[AppRegistrations]
set LastUsed = getdate()
In the comments it is stated that it cannot be done in a single query. If I added following to a stored procedure will it then be thread safe? Or do I need to add a lock? And does the query make sense or should it be done differently?
declare #id int
declare #name as nvarchar(256)
select top 1 #id=id,#name=name from [dbo].[AppRegistrations] order by LastUsed
Update [dbo].[AppRegistrations] set LastUsed=getdate() where id=#id
select #id,#name
It is important that another query cannot interrupt returning a unique row because it updates a row between the select and the update. That is why I wanted it in a single query.
I tried to gather everything up and added a row lock. Following sample works as expected, but I dont know whether the row lock is the right way, or I should expect some challenges. Can someone validate if this approach is correct?
BEGIN TRAN
declare #id int
declare #name as nvarchar(256)
select top 1 #id=id,#name=name from [dbo].[AppRegistrations] WITH (HOLDLOCK, ROWLOCK) order by LastUsed
Update [dbo].[AppRegistrations] set LastUsed=getdate() where id=#id
select #id as id,#name as name
COMMIT TRAN
I make a good number of assumptions here
UPDATE [dbo].[AppRegistrations]
SET LastSelected = CURRENT_TIMESTAMP
OUTPUT INSERTED.*
WHERE Id = (SELECT TOP (1) Id
FROM [dbo].[AppRegistrations]
ORDER BY LastSelected
)
Here is some background on the OUTPUT https://learn.microsoft.com/en-us/sql/t-sql/queries/output-clause-transact-sql?view=sql-server-ver15
Here is another reference where you can do slightly more complex things https://learn.microsoft.com/en-us/sql/t-sql/queries/update-transact-sql?view=sql-server-ver15#CaptureResults
I would like to ask if there is a quick way to keep the last 'x' inserted rows in a database.
For example, i have an application through which users can search for items and I want to keep the last 10 searches that each user has made. I do not want to keep all his searches into the database as this will increase db_size. Is there a quick way of keeping only the latest 10 searches on my db or shall i check every time:
A) how many searches has been stored on database so far
B) if (searches = 10) delete last row
C) insert new row
I think that this way will have an impact on performance as it will need 3 different accesses on the database: check, delete and insert
I don't think an easy/quick way to do this. Based on your conditions i created the below stored procedure.
I considered SearchContent which is going to store the data.
CREATE TABLE SearchContent (
Id INT IDENTITY (1, 1),
UserId VARCHAR (8),
SearchedOn DATETIME,
Keyword VARCHAR (40)
)
In the stored procedure passing the UserId, Keyword and do the calculation. The procedure will be,
CREATE PROCEDURE [dbo].[pub_SaveSearchDetails]
(
#UserId VARCHAR (8),
#Keyword VARCHAR (40)
)
AS
BEGIN
SET NOCOUNT ON;
DECLARE #Count AS INT = 0;
-- A) how many searches has been stored on database so far
SELECT #Count = COUNT(Keyword) FROM SearchContent WHERE UserId = #UserId
IF #Count >= 10
BEGIN
-- B) if (searches = 10) delete last row
DELETE FROM SearchContent WHERE Id IN ( SELECT TOP 1 Id FROM SearchContent WHERE UserId = #UserId ORDER BY SearchedOn ASC)
END
-- C) insert new row
INSERT INTO SearchContent (UserId, SearchedOn, Keyword)
VALUES (#UserId, GETDATE(), #Keyword)
END
Sample execution: EXEC pub_SaveSearchDetails 'A001', 'angularjs'
I have an arbitrary stored procedure usp_DoubleCheckLockInsert that does an INSERT for multiple clients and I want to give the stored procedure exclusive access to writing to a table SomeTable when it is within the critical section Begin lock and End lock.
CREATE PROCEDURE usp_DoubleCheckLockInsert
#Id INT
,#SomeValue INT
AS
BEGIN
IF (EXISTS(SELECT 1 FROM SomeTable WHERE Id = #Id AND SomeValue = #SomeValue)) RETURN
BEGIN TRAN
--Begin lock
IF (EXISTS(SELECT 1 FROM SomeTable WHERE Id = #Id AND SomeValue = #SomeValue)) ROLLBACK
INSERT INTO SomeTable(Id, SomeValue)
VALUES(#Id,#SomeValue);
--End lock
COMMIT
END
I have seen how Isolation Level relates to updates, but is there a way to implement locking in the critical section, give the transaction the writing lock, or does TSQL not work this way?
Obtain Update Table Lock at start of Stored Procedure in SQL Server
A second approach which works for me is to combine the INSERT and the SELECT into a single operation.
This index needed only for efficiently querying SomeTable. Note that there is NOT a uniqueness constraint. However, if I were taking this approach, I would actually make the index unique.
CREATE INDEX [IX_SomeTable_Id_SomeValue_IsDelete] ON [dbo].[SomeTable]
(
[Id] ASC,
[SomeValue] ASC,
[IsDelete] ASC
)
The stored proc, which combines the INSERT/ SELECT operations:
CREATE PROCEDURE [dbo].[usp_DoubleCheckLockInsert]
#Id INT
,#SomeValue INT
,#IsDelete bit
AS
BEGIN
-- Don't allow dirty reads
SET TRANSACTION ISOLATION LEVEL READ COMMITTED
BEGIN TRAN
-- insert only if data not existing
INSERT INTO dbo.SomeTable(Id, SomeValue, IsDelete)
SELECT #Id, #SomeValue, #IsDelete
where not exists (
select * from dbo.SomeTable WITH (HOLDLOCK, UPDLOCK)
where Id = #Id
and SomeValue = #SomeValue
and IsDelete = #IsDelete)
COMMIT
END
I did try this approach using multiple processes to insert data. (I admit though that I didn't exactly put a lot of stress on SQL Server). There were never any duplicates or failed inserts.
It seems all you are trying to do is to prevent duplicate rows from being inserted. You can do this by adding a unique index, with the option IGNORE_DUP_KEY = ON:
CREATE UNIQUE INDEX [IX_SomeTable_Id_SomeValue_IsDelete]
ON [dbo].[SomeTable]
(
[Id] ASC,
[SomeValue] ASC,
[IsDelete] ASC
) WITH (IGNORE_DUP_KEY = ON)
Any inserts with duplicate keys will be ignored by SQL Server. Running the following:
INSERT INTO [dbo].[SomeTable] ([Id],[SomeValue],[IsDelete])
VALUES(0,0,0)
INSERT INTO [dbo].[SomeTable] ([Id],[SomeValue],[IsDelete])
VALUES(1,1,0)
INSERT INTO [dbo].[SomeTable] ([Id],[SomeValue],[IsDelete])
VALUES(2,2,0)
INSERT INTO [dbo].[SomeTable] ([Id],[SomeValue],[IsDelete])
VALUES(0,0,0)
Results in:
(1 row(s) affected)
(1 row(s) affected)
(1 row(s) affected)
Duplicate key was ignored.
(0 row(s) affected)
I did not test the above using multiple processes (threads), but the results in that case should be the same - SQL Server should still ignore any duplicates, no matter which thread is attempting the insert.
See also Index Options at MSDN.
I think I may not understand the question but why couldn't you do this:
begin tran
if ( not exists ( select 1 from SomeTable where Id = #ID and SomeValue = #SomeValue ) )
insert into SomeTable ( Id, SomeValue ) values ( #ID, #SomeValue )
commit
Yes you have a transaction every time you do this but as long as your are fast that shouldn't be a problem.
I have a feeling I'm not understanding the question.
Jeff.
As soon as you start messing with sql preferred locking management, you are taking the burdon on, but if you're certain this is what you need, update your sp to select a test variable and replace your "EXISTS" check with that variable. When you query the variable use an exclusive table lock, and the table is yours till your done.
CREATE PROCEDURE usp_DoubleCheckLockInsert
#Id INT
,#SomeValue INT
AS
BEGIN
IF (EXISTS(SELECT 1 FROM SomeTable WHERE Id = #Id AND SomeValue = #SomeValue)) RETURN
BEGIN TRAN
--Begin lock
DECLARE #tId as INT
-- You already checked and the record doesn't exist, so lock the table
SELECT #tId
FROM SomeTable WITH (TABLOCKX)
WHERE Id = #Id AND SomeValue = #SomeValue
IF #tID IS NULL
BEGIN
-- no one snuck in between first and second checks, so commit
INSERT INTO SomeTable(Id, SomeValue)
VALUES(#Id,#SomeValue);
--End lock
COMMIT
END
If you execute this as a query, but don't hit the commit, then try selecting from the table from a different context, you will sit and wait till the commit is enacted.
Romoku, the answers you're getting are basically right, except
that you don't even need BEGIN TRAN.
you don't need to worry about isolation levels.
All you need is a simple insert ... select ... where not exists (select ...) as suggested by Jeff B and Chue X.
Your concerns about concurrency ("I'm talking about concurrency and your answer will not work.") reveal a profound misunderstanding of how SQL works.
SQL INSERT is atomic. You don't have to lock the table; that's what the DBMS does for you.
Instead of offering a bounty for misbegotten questions based on erroneous preconceived notions -- and then summarily dismissing right answers as wrong -- I recommend sitting down with a good book. On SQL. I can suggest some titles if you like.
I've got a table where I need to auto-assign an ID 99% of the time (the other 1% rules out using an identity column it seems). So I've got a stored procedure to get next ID along the following lines:
select #nextid = lastid+1 from last_auto_id
check next available id in the table...
update last_auto_id set lastid = #nextid
Where the check has to check if users have manually used the IDs and find the next unused ID.
It works fine when I call it serially, returning 1, 2, 3 ... What I need to do is provide some locking where multiple processes call this at the same time. Ideally, I just need it to exclusively lock the last_auto_id table around this code so that a second call must wait for the first to update the table before it can run it's select.
In Postgres, I can do something like 'LOCK TABLE last_auto_id;' to explicitly lock the table. Any ideas how to accomplish it in SQL Server?
Thanks in advance!
Following update increments your lastid by one and assigns this value to your local variable in a single transaction.
Edit
thanks to Dave and Mitch for pointing out isolation level problems with the original solution.
UPDATE last_auto_id WITH (READCOMMITTEDLOCK)
SET #nextid = lastid = lastid + 1
You guys have between you answered my question. I'm putting in my own reply to collate the working solution I've got into one post. The key seems to have been the transaction approach, with locking hints on the last_auto_id table. Setting the transaction isolation to serializable seemed to create deadlock problems.
Here's what I've got (edited to show the full code so hopefully I can get some further answers...):
DECLARE #Pointer AS INT
BEGIN TRANSACTION
-- Check what the next ID to use should be
SELECT #NextId = LastId + 1 FROM Last_Auto_Id WITH (TABLOCKX) WHERE Name = 'CustomerNo'
-- Now check if this next ID already exists in the database
IF EXISTS (SELECT CustomerNo FROM Customer
WHERE ISNUMERIC(CustomerNo) = 1 AND CustomerNo = #NextId)
BEGIN
-- The next ID already exists - we need to find the next lowest free ID
CREATE TABLE #idtbl ( IdNo int )
-- Into temp table, grab all numeric IDs higher than the current next ID
INSERT INTO #idtbl
SELECT CAST(CustomerNo AS INT) FROM Customer
WHERE ISNUMERIC(CustomerNo) = 1 AND CustomerNo >= #NextId
ORDER BY CAST(CustomerNo AS INT)
-- Join the table with itself, based on the right hand side of the join
-- being equal to the ID on the left hand side + 1. We're looking for
-- the lowest record where the right hand side is NULL (i.e. the ID is
-- unused)
SELECT #Pointer = MIN( t1.IdNo ) + 1 FROM #idtbl t1
LEFT OUTER JOIN #idtbl t2 ON t1.IdNo + 1 = t2.IdNo
WHERE t2.IdNo IS NULL
END
UPDATE Last_Auto_Id SET LastId = #NextId WHERE Name = 'CustomerNo'
COMMIT TRANSACTION
SELECT #NextId
This takes out an exclusive table lock at the start of the transaction, which then successfully queues up any further requests until after this request has updated the table and committed it's transaction.
I've written a bit of C code to hammer it with concurrent requests from half a dozen sessions and it's working perfectly.
However, I do have one worry which is the term locking 'hints' - does anyone know if SQLServer treats this as a definite instruction or just a hint (i.e. maybe it won't always obey it??)
How is this solution? No TABLE LOCK is required and works perfectly!!!
DECLARE #NextId INT
UPDATE Last_Auto_Id
SET #NextId = LastId = LastId + 1
WHERE Name = 'CustomerNo'
SELECT #NextId
Update statement always uses a lock to protect its update.
You might wanna consider deadlocks. This usually happens when multiple users use the stored procedure simultaneously. In order to avoid deadlock and make sure every query from the user will succeed you will need to do some handling during update failures and to do this you will need a try catch. This works on Sql Server 2005,2008 only.
DECLARE #Tries tinyint
SET #Tries = 1
WHILE #Tries <= 3
BEGIN
BEGIN TRANSACTION
BEGIN TRY
-- this line updates the last_auto_id
update last_auto_id set lastid = lastid+1
COMMIT
BREAK
END TRY
BEGIN CATCH
SELECT ERROR_NUMBER() AS ErrorNumber, ERROR_MESSAGE() as ErrorMessage
ROLLBACK
SET #Tries = #Tries + 1
CONTINUE
END CATCH
END
I prefer doing this using an identity field in a second table. If you make lastid identity then all you have to do is insert a row in that table and select #scope_identity to get your new value and you still have the concurrency safety of identity even though the id field in your main table is not identity.
I have a primary key that I don't want to auto increment (for various reasons) and so I'm looking for a way to simply increment that field when I INSERT. By simply, I mean without stored procedures and without triggers, so just a series of SQL commands (preferably one command).
Here is what I have tried thus far:
BEGIN TRAN
INSERT INTO Table1(id, data_field)
VALUES ( (SELECT (MAX(id) + 1) FROM Table1), '[blob of data]');
COMMIT TRAN;
* Data abstracted to use generic names and identifiers
However, when executed, the command errors, saying that
"Subqueries are not allowed in this
context. only scalar expressions are
allowed"
So, how can I do this/what am I doing wrong?
EDIT: Since it was pointed out as a consideration, the table to be inserted into is guaranteed to have at least 1 row already.
You understand that you will have collisions right?
you need to do something like this and this might cause deadlocks so be very sure what you are trying to accomplish here
DECLARE #id int
BEGIN TRAN
SELECT #id = MAX(id) + 1 FROM Table1 WITH (UPDLOCK, HOLDLOCK)
INSERT INTO Table1(id, data_field)
VALUES (#id ,'[blob of data]')
COMMIT TRAN
To explain the collision thing, I have provided some code
first create this table and insert one row
CREATE TABLE Table1(id int primary key not null, data_field char(100))
GO
Insert Table1 values(1,'[blob of data]')
Go
Now open up two query windows and run this at the same time
declare #i int
set #i =1
while #i < 10000
begin
BEGIN TRAN
INSERT INTO Table1(id, data_field)
SELECT MAX(id) + 1, '[blob of data]' FROM Table1
COMMIT TRAN;
set #i =#i + 1
end
You will see a bunch of these
Server: Msg 2627, Level 14, State 1, Line 7
Violation of PRIMARY KEY constraint 'PK__Table1__3213E83F2962141D'. Cannot insert duplicate key in object 'dbo.Table1'.
The statement has been terminated.
Try this instead:
INSERT INTO Table1 (id, data_field)
SELECT id, '[blob of data]' FROM (SELECT MAX(id) + 1 as id FROM Table1) tbl
I wouldn't recommend doing it that way for any number of reasons though (performance, transaction safety, etc)
It could be because there are no records so the sub query is returning NULL...try
INSERT INTO tblTest(RecordID, Text)
VALUES ((SELECT ISNULL(MAX(RecordID), 0) + 1 FROM tblTest), 'asdf')
I don't know if somebody is still looking for an answer but here is a solution that seems to work:
-- Preparation: execute only once
CREATE TABLE Test (Value int)
CREATE TABLE Lock (LockID uniqueidentifier)
INSERT INTO Lock SELECT NEWID()
-- Real insert
BEGIN TRAN LockTran
-- Lock an object to block simultaneous calls.
UPDATE Lock WITH(TABLOCK)
SET LockID = LockID
INSERT INTO Test
SELECT ISNULL(MAX(T.Value), 0) + 1
FROM Test T
COMMIT TRAN LockTran
We have a similar situation where we needed to increment and could not have gaps in the numbers. (If you use an identity value and a transaction is rolled back, that number will not be inserted and you will have gaps because the identity value does not roll back.)
We created a separate table for last number used and seeded it with 0.
Our insert takes a few steps.
--increment the number
Update dbo.NumberTable
set number = number + 1
--find out what the incremented number is
select #number = number
from dbo.NumberTable
--use the number
insert into dbo.MyTable using the #number
commit or rollback
This causes simultaneous transactions to process in a single line as each concurrent transaction will wait because the NumberTable is locked. As soon as the waiting transaction gets the lock, it increments the current value and locks it from others. That current value is the last number used and if a transaction is rolled back, the NumberTable update is also rolled back so there are no gaps.
Hope that helps.
Another way to cause single file execution is to use a SQL application lock. We have used that approach for longer running processes like synchronizing data between systems so only one synchronizing process can run at a time.
If you're doing it in a trigger, you could make sure it's an "INSTEAD OF" trigger and do it in a couple of statements:
DECLARE #next INT
SET #next = (SELECT (MAX(id) + 1) FROM Table1)
INSERT INTO Table1
VALUES (#next, inserted.datablob)
The only thing you'd have to be careful about is concurrency - if two rows are inserted at the same time, they could attempt to use the same value for #next, causing a conflict.
Does this accomplish what you want?
It seems very odd to do this sort of thing w/o an IDENTITY (auto-increment) column, making me question the architecture itself. I mean, seriously, this is the perfect situation for an IDENTITY column. It might help us answer your question if you'd explain the reasoning behind this decision. =)
Having said that, some options are:
using an INSTEAD OF trigger for this purpose. So, you'd do your INSERT (the INSERT statement would not need to pass in an ID). The trigger code would handle inserting the appropriate ID. You'd need to use the WITH (UPDLOCK, HOLDLOCK) syntax used by another answerer to hold the lock for the duration of the trigger (which is implicitly wrapped in a transaction) & to elevate the lock type from "shared" to "update" lock (IIRC).
you can use the idea above, but have a table whose purpose is to store the last, max value inserted into the table. So, once the table is set up, you would no longer have to do a SELECT MAX(ID) every time. You'd simply increment the value in the table. This is safe provided that you use appropriate locking (as discussed). Again, that avoids repeated table scans every time you INSERT.
use GUIDs instead of IDs. It's much easier to merge tables across databases, since the GUIDs will always be unique (whereas records across databases will have conflicting integer IDs). To avoid page splitting, sequential GUIDs can be used. This is only beneficial if you might need to do database merging.
Use a stored proc in lieu of the trigger approach (since triggers are to be avoided, for some reason). You'd still have the locking issue (and the performance problems that can arise). But sprocs are preferred over dynamic SQL (in the context of applications), and are often much more performant.
Sorry about rambling. Hope that helps.
How about creating a separate table to maintain the counter? It has better performance than MAX(id), as it will be O(1). MAX(id) is at best O(lgn) depending on the implementation.
And then when you need to insert, simply lock the counter table for reading the counter and increment the counter. Then you can release the lock and insert to your table with the incremented counter value.
Have a separate table where you keep your latest ID and for every transaction get a new one.
It may be a bit slower but it should work.
DECLARE #NEWID INT
BEGIN TRAN
UPDATE TABLE SET ID=ID+1
SELECT #NEWID=ID FROM TABLE
COMMIT TRAN
PRINT #NEWID -- Do what you want with your new ID
Code without any transaction scope (I use it in my engineer course as an exercice) :
-- Preparation: execute only once
CREATE TABLE increment (val int);
INSERT INTO increment VALUES (1);
-- Real insert
DECLARE #newIncrement INT;
UPDATE increment
SET #newIncrement = val,
val = val + 1;
INSERT INTO Table1 (id, data_field)
SELECT #newIncrement, 'some data';
declare #nextId int
set #nextId = (select MAX(id)+1 from Table1)
insert into Table1(id, data_field) values (#nextId, '[blob of data]')
commit;
But perhaps a better approach would be using a scalar function getNextId('table1')
Any critiques of this? Works for me.
DECLARE #m_NewRequestID INT
, #m_IsError BIT = 1
, #m_CatchEndless INT = 0
WHILE #m_IsError = 1
BEGIN TRY
SELECT #m_NewRequestID = (SELECT ISNULL(MAX(RequestID), 0) + 1 FROM Requests)
INSERT INTO Requests ( RequestID
, RequestName
, Customer
, Comment
, CreatedFromApplication)
SELECT RequestID = #m_NewRequestID
, RequestName = dbo.ufGetNextAvailableRequestName(PatternName)
, Customer = #Customer
, Comment = [Description]
, CreatedFromApplication = #CreatedFromApplication
FROM RequestPatterns
WHERE PatternID = #PatternID
SET #m_IsError = 0
END TRY
BEGIN CATCH
SET #m_IsError = 1
SET #m_CatchEndless = #m_CatchEndless + 1
IF #m_CatchEndless > 1000
THROW 51000, '[upCreateRequestFromPattern]: Unable to get new RequestID', 1
END CATCH
This should work:
INSERT INTO Table1 (id, data_field)
SELECT (SELECT (MAX(id) + 1) FROM Table1), '[blob of data]';
Or this (substitute LIMIT for other platforms):
INSERT INTO Table1 (id, data_field)
SELECT TOP 1
MAX(id) + 1, '[blob of data]'
FROM
Table1
ORDER BY
[id] DESC;