With the help of others on SO I've knocked up a couple of Tables and Stored Procedures, this morning, as I'm far from a DB programmer.
Would someone mind casting an eye over this and telling me if it's thread-safe? I guess that's probably not the term DBAs/DB developers use but I hope you get the idea: basically, what happens if this sp is executing and another comes along at the same time? Could one interfere with the other? Is this even an issue in SQL/SPs?
CREATE PROCEDURE [dbo].[usp_NewTicketNumber]
#ticketNumber int OUTPUT
AS
BEGIN
SET NOCOUNT ON;
INSERT INTO [TEST_Db42].[dbo].[TicketNumber]
([CreatedDateTime], [CreatedBy])
VALUES
(GETDATE(), SUSER_SNAME())
SELECT #ticketNumber = IDENT_CURRENT('[dbo].[TicketNumber]');
RETURN 0;
END
You probably do not want to be using IDENT_CURRENT - this returns the latest identity generated on the table in question, in any session and any scope. If someone else does an insert at the wrong time you will get their id instead!
If you want to get the identity generated by the insert that you just performed then it is best to use the OUTPUT clause to retrieve it. It used to be usual to use the SCOPE_IDENTITY() for this but there are problems with that under parallel execution plans.
The main SQL equivalent of thread safety is when multiple statements are executed that cause unexpected or undesirable behaviour. The two main types of such behaviour I can think of are locking (in particular deadlocks) and concurrency issues.
Locking problems occur when a statement stops other statements from accessing the rows it is working with. This can affect performance and in the worst scenario two statements make changes that cannot be reconciled and a deadlock occurs, causing one statement to be terminated.
However, a simple insert like the one you have should not cause locks unless something else is involved (like database transactions).
Concurrency issues (describing them very poorly) are caused by one set of changes to database records overwriting other changes to the same records. Again, this should not be a problem when inserting a record.
The safest way to go here would probably be to use the Output clause, since there is a known bug in scope_idendity under certain circumstances ( multi/parallel processing ).
CREATE PROCEDURE [dbo].[usp_NewTicketNumber]
AS
BEGIN
DECLARE #NewID INT
BEGIN TRANSACTION
BEGIN TRY
declare #ttIdTable TABLE (ID INT)
INSERT INTO
[dbo].[TicketNumber]([CreatedDateTime], [CreatedBy])
output inserted.id into #ttIdTable(ID)
VALUES
(GETDATE(), SUSER_SNAME())
SET #NewID = (SELECT id FROM #ttIdTable)
COMMIT TRANSACTION
END TRY
BEGIN CATCH
ROLLBACK TRANSACTION
SET #NewID = -1
END CATCH
RETURN #NewID
END
This way you should be thread safe, since the output clause uses the data that the insert actually inserts, and you won't have problems across scopes or sessions.
CREATE PROCEDURE [dbo].[usp_NewTicketNumber]
#NewID int OUTPUT
AS
BEGIN
SET NOCOUNT ON;
BEGIN TRY
BEGIN TRANSACTION
INSERT INTO
[dbo].[TicketNumber] ([CreatedDateTime], [CreatedBy])
VALUES
(GETDATE(), SUSER_SNAME())
SET #NewID = SCOPE_IDENTITY()
COMMIT TRANSACTION;
END TRY
BEGIN CATCH
IF XACT_STATE() <> 0
ROLLBACK TRANSACTION;
SET #NewID = NULL;
END CATCH
END
I would not use RETURN for meaningful use data: either recordset or output parameter. RETURN would normally be used for error states (like system stored procs do in most cases):
EXEC #rtn = EXEC dbo.uspFoo
IF #rtn <> 0
--do error stuff
You can also use the OUTPUT clause to return a recordset instead.
This is "thread safe", that is it can be run concurrently.
First off - why don't you just return the new ticket number instead of 0 all the time? Any particular reason for that?
Secondly, to be absolutely sure, you should wrap your INSERT and SELECT statement into a TRANSACTION so that nothing from the outside can intervene.
Thirdly, with SQL Server 2005 and up, I'd wrap my statements into a TRY....CATCH block and roll back the transaction if it fails.
Next, I would try to avoid specifying the database server (TestDB42) in my procedures whenever possible - what if you want to deploy that proc to a new server (TestDB43) ??
And lastly, I'd never use a SET NOCOUNT in a stored procedure - it can cause the caller to erroneously think the stored proc failed (see my comment to gbn below - this is a potential problem if you're using ADO.NET SqlDataAdapter objects only; see the MSDN docs on how to modify ADO.NET data with SqlDataAdapter for more explanations).
So my suggestion for your stored proc would be:
CREATE PROCEDURE [dbo].[usp_NewTicketNumber]
AS
BEGIN
DECLARE #NewID INT
BEGIN TRANSACTION
BEGIN TRY
INSERT INTO
[dbo].[TicketNumber]([CreatedDateTime], [CreatedBy])
VALUES
(GETDATE(), SUSER_SNAME())
SET #NewID = SCOPE_IDENTITY()
COMMIT TRANSACTION
END TRY
BEGIN CATCH
ROLLBACK TRANSACTION
SET #NewID = -1
END CATCH
RETURN #NewID
END
Marc
I agree with David Hall's answer, I just want to expand a bit on why ident_current is absolutely the wrong thing to use in this situation.
We had a developer here who used it. The insert from the client application happened at the same time the database was importing millions of records through an automated import. The id returned to him was from one of the records my process imported. He used this id to create records for some child tables which were now attached to the wrong record. Worse we now have no idea how many times this happened before someone couldn't find the information that should have been in the child tables (his change had been on prod for several months). Not only could my automated import have interfered with his code, but another user inserting a record at the smae time could have done the same thing. Ident_current should never be used to return the identity of a record just inserted as it is not limited to the process that calls it.
Related
Hear me out! I know this use case sounds suspect, but...
I have a stored procedure which checks a table (effectively a cache) for data for a given requested ID. If it doesn't find any data for that ID, or deems it out of date, it executes a second stored procedure which will pull data from a separate database (using dynamic SQL, source DB name is based on the requested ID) and insert it into the local table. It then selects from this table.
If the data is in the table, everything returns quickly (ms), but if it needs to be brought in from the other database, it takes about 10 seconds. We're seeing race conditions where two concurrent instances check the local cache, see something is missing, and queue up sequential ingestions of the remote data into the cache. To avoid double-insertion, the cache-populating procedure will clear whatever is already there for this id, but this just means the first instance of the procedure can selecting no rows because the second instance deleted the just-inserted records before re-inserting them itself.
I think I want to put a lock around the entire procedure (checking the cache, potentially populating the cache, selecting from the cache) - although I'm open to other solutions. I think the overall caching approach has to remain on-demand though, the remote databases come and go by the hundreds, and we only want to cache the ones actually requested by reporting as-needed.
BEGIN TRANSACTION;
BEGIN TRY
-- Take out a lock intended to prevent anyone else modifying the cache while we're reading and potentially modifying it
EXEC sp_getapplock #Resource = '[private].[cache_entries]', #LockOwner='Transaction', #LockMode = 'Exclusive', #LockTimeout = 120000;
-- Invoke a stored procedure that ingests any required data that is not already cached
EXEC [private].populate_cache #required_dbs
-- CALCULATIONS
-- ... SELECT FROM [private].cache_entries
COMMIT TRANSACTION; -- Free the lock
END TRY
BEGIN CATCH --Ensure we release our lock on failure
ROLLBACK TRANSACTION;
THROW
END CATCH;
The alternative to sp_getapplock is to use locking hints with your transaction. Both are reasonable approaches. Locking hints can be complex, but they protect the target object itself rather than a single code path. So sometimes necessary. sp_getapplock is simple (with Transaction as owner), and reliable.
You can do this without sp_getapplock, which tends to inhibit concurrency a lot.
The way to do this is to continue do your checks within a transaction, but to apply a HOLDLOCK hint, as well as a UPDLOCK hint.
HOLDLOCK aka the SERIALIZABLE isolation level, will place a lock not only on the ID you specify, but even on the absence of such data, in other words it will prevent anyone else inserting into that ID.
You must use both these hints, as well as have an index that matches that SELECT, otherwise you could run into major blocking and deadlocking problems due to full table scans.
Also, you don't need a CATCH and ROLLBACK. Just use SET XACT_ABORT ON; which ensures a rollback in any event of an error.
SET XACT_ABORT ON; -- always have this set
BEGIN TRANSACTION;
DECLARE #SomeData nvarchar(100) = (
SELECT ce.SomeColumn
FROM [private].cache_entries ce WITH (HOLDLOCK, UPDLOCK)
WHERE ce.SomeCondition = 1
);
IF #SomeData IS NULL
BEGIN
-- Invoke a stored procedure that ingests any required data that is not already cached
EXEC [private].populate_cache #required_dbs
END
-- CALCULATIONS
-- ... SELECT FROM [private].cache_entries
COMMIT TRANSACTION; -- Free the lock
I have created AFTER INSERT TRIGGER
Now if any case if an error occurs while executing Trigger. It should not effect Insert Operation on Triggered table.
In One word if any ERROR occurs in trigger it should Ignore it.
As I have used
BEGIN TRY
END TRY
BEGIN CATCH
END CATCH
But it give following error message and Rolled back Insert operation on Triggered table
An error was raised during trigger execution. The batch has been
aborted and the user transaction, if any, has been rolled back.
Interesting problem. By default, triggers are designed that if they fail, they rollback the command that fired it. So whenever trigger is executing there is an active transaction, whatever there was an explicit BEGIN TRANSACTION or not on the outside. And also BEGIN/TRY inside trigger will not work. Your best practice would be not to write any code in trigger that could possibly fail - unless it is desired to also fail the firing statement.
In this situation, to suppress this behavior, there are some workarounds.
Option A (the ugly way):
Since transaction is active at the beginning of trigger, you can just COMMIT it and continue with your trigger commands:
CREATE TRIGGER tgTest1 ON Test1 AFTER INSERT
AS
BEGIN
COMMIT;
... do whatever trigger does
END;
Note that if there is an error in trigger code this will still produce the error message, but data in Test1 table are safely inserted.
Option B (also ugly):
You can move your code from trigger to stored procedure. Then call that stored procedure from Wrapper SP that implements BEGIN/TRY and at the end - call Wrapper SP from trigger. This might be a bit tricky to move data from INSERTED table around if needed in the logic (which is in SP now) - probably using some temp tables.
SQLFiddle DEMO
You cannot, and any attempt to solve it is snake oil. No amount of TRY/CATCH or ##ERROR check will work around the fundamental issue.
If you want to use the tightly coupling of a trigger then you must buy into the lower availability induced by the coupling.
If you want to preserve the availability (ie. have the INSERT succeed) then you must give up coupling (remove the trigger). You must do all the processing you were planning to do in the trigger in a separate transaction that starts after your INSERT committed. A SQL Agent job that polls the table for newly inserted rows, an Service Broker launched procedure or even an application layer step are all going to fit the bill.
The accepted answer's option A gave me the following error: "The transaction ended in the trigger. The batch has been aborted.". I circumvented the problem by using the SQL below.
CREATE TRIGGER tgTest1 ON Test1 AFTER INSERT
AS
BEGIN
SET XACT_ABORT OFF
BEGIN TRY
SELECT [Column1] INTO #TableInserted FROM [inserted]
EXECUTE sp_executesql N'INSERT INTO [Table]([Column1]) SELECT [Column1] FROM #TableInserted'
END TRY
BEGIN CATCH
END CATCH
SET XACT_ABORT ON
END
I need to modify approx. 24 huge UDP and for production deployment i need to do a BEGIN TRANSACTION / ROLLBACK / COMMIT PROCESS.
How can I add the ALTER PROCEDURE my_proc between BEGIN TRANSACTION and COMMIT or ROLLBACK?
Note: EXEC('ALTER PROCEDURE..') can NOT be implemented.
Thanks
Update: there is a way to alter a procedure and rollback if it fails?
why can't you the regular way.
BEGIN TRANSACTION
GO
CREATE PROCEDURE testProcedure
AS
SELECT 1
GO
SELECT OBJECT_ID('testProcedure') ObjectID --this will return the object ID
GO
rollback TRANSACTION
SELECT OBJECT_ID('testProcedure') ObjectID --this will return NULL because the proc creation was rolled back
GO
You cannot have BEGIN TRY and BEGIN CATCH around batches. However you can use the last batch to check that all previous steps have succeeded (by examining the catalog views like sys.objects for example). Then you can decide if the batch all succeeded and either commit or roll back.
(Leandro, I’m adding a new answer because it would be too long for a compent)
I’ve been thinking. I don’t think this is a solution I would ever implement, but based on your requirements (and specially your restrictions), here is an idea that would work:
There is a modify_date on the sys.objects catalog so, why don’t you store the dates off all your objects before you run your updates and compare with the dates after you ran your updates. If ALL the dates are different, it means that all of them were updated correctly, if one of the dates is equal, it means that one failed and then you run a rollback script (you will need the rollback code, won’t be easy as just type ROLLBACK)
I need to provide an auto update feature to my application.
I am having problem in applying the SQL updates. I have the updated SQL statement in my .sql file and what i want to achieve is that if one statment fails then entire script file must be rolled back
Ex.
create procedure [dbo].[test1]
#P1 varchar(200),
#C1 int
as
begin
Select 1
end
GO
Insert into test (name) values ('vv')
Go
alter procedure [dbo].[test2]
#P1 varchar(200),
#C1 int
as
begin
Select 1
end
GO
Now in the above example, if i get the error in third statement of "alter procedure [dbo].[test2]" then i want to rollback the first two changes also which is creating SP of "test1" and inserting data into "test" table
How should i approach this task? Any help will be much appreciated.
If you need any more info then let me know
Normally, you would want to add a BEGIN TRAN at the beginning, remove the GO statements, and then handle the ROLLBACK TRAN/COMMIT TRAN with a TRY..CATCH block.
When dealing with DML though there are often statements that have to be at the start of a batch, so you can't wrap them in a TRY..CATCH block. In that case you need to put together a system that knows how to roll itself back.
A simple system would be just to backup the database at the start and restore it if anything fails (assuming that you are the only one accessing the database the whole time). Another method would be to log each batch that runs successfully and to have corresponding rollback scripts which you can run to put everything back should a later batch fail. This obviously requires much more work (writing an undo script for every script PLUS fully testing the rollbacks) and can also be a problem if people are still accessing the database while the upgrade is happening.
EDIT:
Here's an example of a simple TRY..CATCH block with transaction handling:
BEGIN TRY
BEGIN TRANSACTION
-- All of your code here, with `RAISERROR` used for any of your own error conditions
COMMIT TRANSACTION
END TRY
BEGIN CATCH
ROLLBACK TRANSACTION
END CATCH
However, the TRY..CATCH block cannot span batches (maybe that's what I was thinking of when I said transactions couldn't), so in your case it would probably be something more like:
IF (OBJECT_ID('dbo.Error_Happened') IS NOT NULL)
DROP TABLE dbo.Error_Happened
GO
BEGIN TRANSACTION
<Some line of code>
IF (##ERROR <> 0)
CREATE TABLE dbo.Error_Happened (my_id INT)
IF (OBJECT_ID('dbo.Error_Happened') IS NOT NULL)
BEGIN
<Another line of code>
IF (##ERROR <> 0)
CREATE TABLE dbo.Error_Happened (my_id INT)
END
...
IF (OBJECT_ID('dbo.Error_Happened) IS NOT NULL)
BEGIN
ROLLBACK TRANSACTION
DROP TABLE dbo.Error_Happened
END
ELSE
COMMIT TRANSACTION
Unfortunately, because of the separate batches from the GO statements you can't use GOTO, you can't use the TRY..CATCH, and you can't persist a variable across the batches. This is why I used the very kludgy trick of creating a table to indicate an error.
A better way would be to simply have an error table and look for rows in it. Just keep in mind that your ROLLBACK will remove those rows at the end as well.
I'm working on a procedure that will update a large number of items on a remote server, using records from a local database. Here's the pseudocode.
CREATE PROCEDURE UpdateRemoteServer
pre-processing
get cursor with ID's of records to be updated
while on cursor
process the item
No matter how much we optimize it, the routine is going to take a while, so we don't want the whole thing to be processed as a single transaction. The items are flagged after being processed, so it should be possible to pick up where we left off if the process is interrupted.
Wrapping the contents of the loop ("process the item") in a begin/commit tran does not do the trick... it seems that the whole statement
EXEC UpdateRemoteServer
is treated as a single transaction. How can I make each item process as a complete, separate transaction?
Note that I would love to run these as "non-transacted updates", but that option is only available (so far as I know) in 2008.
EXEC procedure does not create a transaction. A very simple test will show this:
create procedure usp_foo
as
begin
select ##trancount;
end
go
exec usp_foo;
The ##trancount inside usp_foo is 0, so the EXEC statement does not start an implicit transaction. If you have a transaction started when entering UpdateRemoteServer it means somebody started that transaction, I can't say who.
That being said, using remote servers and DTC to update items is going to perform quite bad. Is the other server also SQL Server 2005 at least? Maybe you can queue the requests to update and use messaging between the local and remote server and have the remote server perform the updates based on the info from the message. It would perform significantly better because both servers only have to deal with local transactions, and you get much better availability due to the loose coupling of queued messaging.
Updated
Cursors actually don't start transactions. The typical cursor based batch processing is usually based on cursors and batches updates into transactions of a certain size. This is fairly common for overnight jobs, as it allows for better performance (log flush throughput due to larger transaction size) and jobs can be interrupted and resumed w/o losing everithing. A simplified version of a batch processing loop is typically like this:
create procedure usp_UpdateRemoteServer
as
begin
declare #id int, #batch int;
set nocount on;
set #batch = 0;
declare crsFoo cursor
forward_only static read_only
for
select object_id
from sys.objects;
open crsFoo;
begin transaction
fetch next from crsFoo into #id ;
while ##fetch_status = 0
begin
-- process here
declare #transactionId int;
SELECT #transactionId = transaction_id
FROM sys.dm_tran_current_transaction;
print #transactionId;
set #batch = #batch + 1
if #batch > 10
begin
commit;
print ##trancount;
set #batch = 0;
begin transaction;
end
fetch next from crsFoo into #id ;
end
commit;
close crsFoo;
deallocate crsFoo;
end
go
exec usp_UpdateRemoteServer;
I ommitted the error handling part (begin try/begin catch) and the fancy ##fetch_status checks (static cursors actually don't need them anyway). This demo code shows that during the run there are several different transactions started (different transaction IDs). Many times batches also deploy transaction savepoints at each item processed so they can skip safely an item that causes an exception, using a pattern similar to the one in my link, but this does not apply to distributed transactions since savepoints and DTC don't mix.
EDIT: as pointed out by Remus below, cursors do NOT open a transaction by default; thus, this is not the answer to the question posed by the OP. I still think there are better options than a cursor, but that doesn't answer the question.
Stu
ORIGINAL ANSWER:
The specific symptom you describe is due to the fact that a cursor opens a transaction by default, therefore no matter how you work it, you're gonna have a long-running transaction as long as you are using a cursor (unless you avoid locks altogether, which is another bad idea).
As others are pointing out, cursors SUCK. You don't need them for 99.9999% of the time.
You really have two options if you want to do this at the database level with SQL Server:
Use SSIS to perform your operation; very fast, but may not be available to you in your particular flavor of SQL Server.
Because you're dealing with remote servers, and you're worried about connectivity, you may have to use a looping mechanism, so use WHILE instead and commit batches at a time. Although WHILE has many of the same issues as a cursor (looping still sucks in SQL), you avoid creating the outer transaction.
Stu
Are yo running this only from within sql server, or from an app? if so, get the list to be processed, then loop in the app to only process for the subsets as required.
Then the transaction should be handled by your app, and should only lock the items being updated/pages the items are in.
NEVER process one item at a time in a loop when you are doing transactional work. You can loop through records processing groups of them but never ever do one record at a time. Do set-based inserts instead and your performance will change from hours to minutes or even seconds. If you are using a cursor to insert update or delete and it isn't handling at least 1000 rowa in each statement (not one at atime) you are doing the wrong thing. Cursors are an extremely poor practice for such thing.
Just an idea ..
Only process a few items when the procedure is called (e.g. only get the TOP 10 items to process)
Process those
Hopefully, this will be the end of the transaction.
Then write a wrapper that calls the procedure as long as there is more work to do (either use a simple count(..) to see if there are items or have the procedure return true indicating that there is more work to do.
Don't know if this works, but maybe the idea is helpful.