I have a complex unit of work from an application that might commit changes to 10-15 tables as a single transaction. The unit of work executes under snapshot isolation.
Some of the tables have a trigger which executes a stored procedure to log messages into a queue. The message contains the Table Name, Key and Change Type. This is necessary to provide backwards compatibility with SQL2005, I can't use the built in queuing.
The problem is I am getting blocking and time-outs in the queue writing stored procedure. I either get a message saying:
Snapshot isolation transaction aborted due to update conflict. You cannot use snapshot isolation to access table 'dbo.tblObjectChanges' directly or indirectly in database
or I get a timeout writing to that table.
Is there a way to change the transaction isolation of the particular call to (or within) the stored procedure that does the message queue writing, from within the trigger? As a last resort, can I make the call to the delete or update parts of the stored procedure run asynchronously?
Here is the SQL for the Stored Procedure:
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
ALTER PROCEDURE [dbo].[usp_NotifyObjectChanges]
#ObjectType varchar(20),
#ObjectKey int,
#Level int,
#InstanceGUID varchar(50),
#ChangeType int = 2
AS
SET NOCOUNT ON
DECLARE #ObjectChangeID int
--Clean up any messages older than 10 minutes
DELETE from tblObjectChanges Where CreatedTime < DATEADD(MINUTE, -10, GetDate())
--If the object is already in the queue, change the time and instanceID
SELECT #ObjectChangeID = [ObjectChangeID] FROM tblObjectChanges WHERE [ObjectType] = #ObjectType AND [ObjectKey] = #ObjectKey AND [Level] = #Level
IF NOT #ObjectChangeID is NULL
BEGIN
UPDATE [dbo].[tblObjectChanges] SET
[CreatedTime] = GETDATE(), InstanceGUID = #InstanceGUID
WHERE
[ObjectChangeID] = #ObjectChangeID
END
ELSE
BEGIN
INSERT INTO [dbo].[tblObjectChanges] (
[CreatedTime],
[ObjectType],
[ObjectKey],
[Level],
ChangeType,
InstanceGUID
) VALUES (
GETDATE(),
#ObjectType,
#ObjectKey,
#Level,
#ChangeType,
#InstanceGUID
)
END
Definition of tblObjectChanges:
CREATE TABLE [dbo].[tblObjectChanges](
[CreatedTime] [datetime] NOT NULL,
[ObjectType] [varchar](20) NOT NULL,
[ObjectKey] [int] NOT NULL,
[Rowversion] [timestamp] NOT NULL,
[Level] [int] NOT NULL,
[ObjectChangeID] [int] IDENTITY(1,1) NOT NULL,
[InstanceGUID] [varchar](50) NULL,
[ChangeType] [int] NOT NULL,
CONSTRAINT [PK_tblObjectChanges] PRIMARY KEY CLUSTERED
(
[ObjectChangeID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, FILLFACTOR = 80) ON [PRIMARY]
) ON [PRIMARY]
GO
This line is almost certainly your problem:
DELETE from tblObjectChanges Where CreatedTime < DATEADD(MINUTE, -10, GetDate())
There are two BIG problems with this statement. First, according to your table definition, CreatedTime is not indexed. This means that in order to execute this statement, the entire table must be scanned, and that will cause the entire table to be locked for the duration of whatever transaction this happens to be a part of. So put an index on this column.
The second problem, is that even with an index, you really shouldn't be performing operational maintenance tasks like this from within a trigger. Besides slowing down the OLTP transactions that have to execute it, this statement only really needs to be executed once every 5-10 minutes. Instead, you are executing it any time (and every time) any of these tables are modified. That is a lot of additional load that gets worse as your system gets busier.
A better approach would be to take this statement out of the triggers entirely, and instead have a SQL Agent Job that runs every 5-10 minutes to execute this clean-up operation. If you do this along with adding the index, most of your problems should disappear.
An additional problem is this statement:
SELECT #ObjectChangeID = [ObjectChangeID] FROM tblObjectChanges WHERE [ObjectType] = #ObjectType AND [ObjectKey] = #ObjectKey AND [Level] = #Level
Unlike the first statement above, this statement belongs in the trigger. However, like the first statement, it too will have (and cause) serious performance and locking issues under load, because again, according to your posted table definition, none of the columns being searched are indexed.
The solution again is to put an additional index on these columns as well.
A few ideas:
Move the delete into a separate scheduled job if possible
Add an index on CreatedTime
Add an index on ObjectType, ObjectKey, Level
add WITH(UPDLOCK, ROWLOCK) to the SELECT
add WITH(ROWLOCK) to the INSERT and the UPDATE
You need to test all of these to see what helps. I would go through them in this order, but see the note below.
Even if you decide against all this, at least leave the WITH(UPDLOCK) on the SELECT as you otherwise might loose updates.
Related
I'm looking at a trivial query and struggle to understand why SQL Server cannot execute it.
Say I have a table
CREATE TABLE [dbo].[t2](
[id] [nvarchar](36) NULL,
[name] [nvarchar](36) NULL
)
And I want to add a new column and set some value to it. So I do the following:
BEGIN TRANSACTION
ALTER TABLE [t2] ADD [name2] [nvarchar](255) NULL
UPDATE [t2] SET [name2] = CONCAT(name, '-XXXX')
COMMIT TRANSACTION
And if I execute the query, I have
I know, it failing because the SQL Server executes the query in a different order for optimization purposes, and one way to fix it would be to separate those two sentences with GO statement. Thus the following query will pass without issues.
BEGIN TRANSACTION
ALTER TABLE [t2] ADD [name2] [nvarchar](255) NULL
GO
UPDATE [t2] SET [name2] = CONCAT(name, '-XXXX')
COMMIT TRANSACTION
Actually, not exactly without issues, as I have to use GO statement which will make transaction scope useless, as discussed on this Stackoverflow question
So I have two questions:
How to make that script work without using GO statement
Why SQL server is not smart enough to figure out such a trivial case? (it is more like a rhetorical question)
This is a parser error. When you run a statement it is parsed before hand, however, only certain DDL operations are "cached" by the parser so that it is aware of later. CREATE is something it will "cache" however, ALTER is not. That is why you can CREATE a table in the same batch and then reference it.
As you have an ALTER then when the parser parses the batch and it gets to the UPDATE statement it will fail, and the error you see is raised. One method is to defer to parsing of the statement:
BEGIN TRANSACTION;
ALTER TABLE [t2] ADD [name2] [nvarchar](255) NULL;
EXEC sys.sp_executesql N'UPDATE [t2] SET [name2] = CONCAT(name, N''-XXXX'');';
COMMIT TRANSACTION;
If, however, N'-XXXX' is meant to be the default value, you could qualify that in the DDL statement instead:
BEGIN TRANSACTION;
ALTER TABLE t2 ADD name2 nvarchar(255) NULL DEFAULT N'-XXXX' WITH VALUES;
COMMIT TRANSACTION;
I am trying to implement an event source in SQL Server, and have been experiencing deadlocking issues.
In my design, events are grouped by DatasetId and each event is written with a SequenceId, with the requirement that, for a given DatasetId, SequenceIds are serial, beginning at 1 and increasing one at a time with each event, never missing a value and never repeating one either.
My Events table looks something like:
CREATE TABLE [Events]
(
[Id] [BIGINT] IDENTITY(1,1) NOT NULL,
[DatasetId] [BIGINT] NOT NULL,
[SequenceId] [BIGINT] NOT NULL,
[Value] [NVARCHAR](MAX) NOT NULL
)
I also have a non-unique, non-clustered index on the DatasetId column.
In order to insert into this table with the above restrictions on SequenceId, I have been inserting rows under a transaction using Serializable isolation level, and calculating the required SequenceId manually within this transaction as the max of all existing SequenceIds plus one:
DECLARE #DatasetId BIGINT = 1, #Value NVARCHAR(MAX) = N'I am an event.';
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE
BEGIN TRANSACTION
BEGIN TRY
DECLARE #SequenceId BIGINT;
SELECT #SequenceId = ISNULL(MAX([SequenceId]), 0) + 1
FROM [Events]
WHERE [DatasetId] = #DatasetId;
INSERT INTO [Events] ([DatasetId], [SequenceId], [Value])
VALUES (#DatasetId, #SequenceId, #Value);
COMMIT TRANSACTION;
END TRY
BEGIN CATCH
ROLLBACK TRANSACTION;
END CATCH
This has worked fine in terms of the guarantees that I require on the SequenceId column. However, I have been experiencing deadlocking when trying to insert multiple rows in parallel, even when such rows are for different DatasetIds.
The behaviour seems to be that the query to generate a SequenceId in a first connection blocks the same query to generate a SequenceId in the second connection, and this second attempt blocks the first connections ability to insert the row, meaning neither transaction is able to complete, hence the deadlock.
Is there a means of avoiding this whilst still gaining the benefits of a Serializable transaction isolation level?
Another technique I have been considering is reducing the isolation level and instead using sp_getapplock to manually acquire a lock on the table for a given DatasetId, meaning I can then ensure I can generate a consistent SequenceId. Once the transaction has been committed/rolled back, the lock would automatically be released. Is this approach reasonable, or is manually managing locks like this considered an anti-pattern?
When I execute the following code (Case 1) I get the value 2 for the count. Which means inside the same transaction the chagnes made to the table are visible. So this behaves in the way I expect.
Case 1
begin tran mytran
begin try
CREATE TABLE [dbo].[ft](
[ft_ID] [int] IDENTITY(1,1) NOT NULL,
[ft_Name] [nvarchar](100) NOT NULL
CONSTRAINT [PK_FileType] PRIMARY KEY CLUSTERED
(
[ft_ID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
INSERT INTO [dbo].[ft]([ft_Name])
VALUES('xxxx')
INSERT INTO [dbo].[ft]([ft_Name])
VALUES('yyyy')
select count(*) from [dbo].[ft]
commit tran mytran
end try
begin catch
rollback tran mytran
end catch
However when I alter a column (e.g. add a new column within the transaction) it is not visible to the (self/same) transaction (Case 2). Let's assume there is a product table without a column called ft_ID and I am adding a column withing the same transaction and going to read it.
Case 2
begin tran mytran
begin try
IF NOT EXISTS (
SELECT *
FROM sys.columns
WHERE object_id = OBJECT_ID(N'dbo.Products')
AND name = 'ft_ID'
)
begin
alter table dbo.Products
add ft_ID int null
end
select ft_ID from dbo.Products
commit tran mytran
end try
begin catch
rollback tran mytran
end catch
When trying to execute Case 2 I get the error "Invalid column name 'ft_ID'" because the newly added column is not visible within the same transaction.
Why this discrepancy happens? Create table is atomic (Case 1) and works in the way I expect but alter table is not. Why changes made within the same transaction are not visible to the statements down (Case 2).
You get a compile errors. The batch is never launched into execution. See Understanding how SQL Server executes a query. Transaction visibility and boundaries has nothing to do with what you're seeing.
You should always separate DDL and DML into separate requests. Without going into too much details, due to the way recovery works, mixing DDL and DML in the same transaction is just asking for trouble. Just take my word on this one.
Rules for Using Batches
...
A table cannot be changed and then the new columns referenced in the same batch.
See this
Alternative is to spawn a child batch and reference your new column from there, like...
exec('select ft_ID from dbo.Products')
However, as Remus said, be very careful about mixing schema changes and selecting data from that schema, especially in one same transaction. Even WITHOUT transaction this code will have side-effects: try wrapping this exec() workaround in the stored procedure, and you will get a recompile every time you call it. Tough luck, but it simply works that way.
I want to create a simple queue with a sql database as backend.
the table have the fields, id, taskname,runat(datetime) and hidden(datetime).
I want to ensure a queue item is not run once and only once.
The idea is when a client want to dequeue, a stored procedure selects the first item(sorted by runat and hidden < now), sets the hidden field to current time + 2min and returns the item.
How does MS Sql (Azure to be precise) wokr, will two clients be able to run at the same time and both set the same item to hidden and return it? Or can i be sure that they are run one by one and the second one will not return the same item as the hidden field was changed with the first?
The key is to get a lock (Row or table) on the queue item you are receiving. You can use a couple of ways, my favorite being the UPDATE with OUTPUT clause. Either will produce serialized access to the table.
Example:
CREATE PROCEDURE spGetNextItem_output
BEGIN
BEGIN TRAN
UPDATE TOP(1) Messages
SET [Status] = 1
OUTPUT INSERTED.MessageID, INSERTED.Data
WHERE [Status] = 0
COMMIT TRAN
END
CREATE PROCEDURE spGetNextItem_tablockx
BEGIN
BEGIN TRAN
DECLARE #MessageID int, #data xml
SELECT TOP(1) #MessageID = MessageID, #Data = Data
FROM Messages WITH (ROWLOCK, XLOCK, READPAST) --lock the row, skip other locked rows
WHERE [Status] = 0
UPDATE Messages
SET [Status] = 1
WHERE MessageID = #MessageID
SELECT #MessageID AS MessageID, #Data as Data
COMMIT TRAN
END
Table definition:
CREATE TABLE [dbo].[Messages](
[MessageID] [int] IDENTITY(1,1) NOT NULL,
[Status] [int] NOT NULL,
[Data] [xml] NOT NULL,
CONSTRAINT [PK_Messages] PRIMARY KEY CLUSTERED
(
[MessageID] ASC
)
)
Windows Azure SQL Database is going to behave just like a SQL Server database in terms of concurrency. This is a database problem, not a Windows Azure problem.
Now: If you went with Windows Azure Queues (or Service Bus Queues) rather than implementing your own, then the behavior is well documented. For instance: with Azure Queues, first-in gets the queue item, and then the item is marked as invisible until it's either deleted or a timeout period has been reached.
I have a table in SQL Server that I inherited from a legacy system thats still in production that is structured according to the code below. I created a SP to query the table as described in the code below the table create statement. My issue is that, sporadically, calls from .NET to this SP both through the Enterprise Library 4 and through a DataReader object are slow. The SP is called through a loop structure in the Data Layer that specifies the params that go into the SP for the purpose of populating user objects. It's also important to mention that a slow call will not take place on every pass the loop structure. It will generally be fine for most of a day or more, and then start presenting which makes it extremely hard to debug.
The table in question contains about 5 million rows. The calls that are slow, for instance, will take as long as 10 seconds, while the calls that are fast will take 0 to 10 milliseconds on average. I checked for locking/blocking transactions during the slow calls, none were found. I created some custom performance counters in the data layer to monitor call times. Essentially, when performance is bad, it's really bad for that one call. But when it's good, it's really good. I've been able to recreate the issue on a few different developer machines, but not on our development and staging database servers, which of course have beefier hardware. Generally, the problem is resolved through restarting the SQL server services, but not always. There are indexes on the table for the fields I'm querying, but there are more indexes than I would like. However, I'm hesitant to remove any or toy with the indexes due to the impact it may have on the legacy system. Has anyone experienced a problem like this before, or do you have a recommendation to remedy it?
CREATE TABLE [dbo].[product_performance_quarterly](
[performance_id] [int] IDENTITY(1,1) NOT FOR REPLICATION NOT NULL,
[product_id] [int] NULL,
[month] [int] NULL,
[year] [int] NULL,
[performance] [decimal](18, 6) NULL,
[gross_or_net] [char](15) NULL,
[vehicle_type] [char](30) NULL,
[quarterly_or_monthly] [char](1) NULL,
[stamp] [datetime] NULL CONSTRAINT [DF_product_performance_quarterly_stamp] DEFAULT (getdate()),
[eA_loaded] [nchar](10) NULL,
[vehicle_type_id] [int] NULL,
[yearmonth] [char](6) NULL,
[gross_or_net_id] [tinyint] NULL,
CONSTRAINT [PK_product_performance_quarterly_4_19_04] PRIMARY KEY CLUSTERED
(
[performance_id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, FILLFACTOR = 80) ON [PRIMARY]
) ON [PRIMARY]
GO
SET ANSI_PADDING OFF
GO
ALTER TABLE [dbo].[product_performance_quarterly] WITH NOCHECK ADD CONSTRAINT [FK_product_performance_quarterlyProduct_id] FOREIGN KEY([product_id])
REFERENCES [dbo].[products] ([product_id])
GO
ALTER TABLE [dbo].[product_performance_quarterly] CHECK CONSTRAINT [FK_product_performance_quarterlyProduct_id]
CREATE PROCEDURE [eA.Analytics.Calculations].[USP.GetCalculationData]
(
#PRODUCTID INT, --products.product_id
#BEGINYEAR INT, --year to begin retrieving performance data
#BEGINMONTH INT, --month to begin retrieving performance data
#ENDYEAR INT, --year to end retrieving performance data
#ENDMONTH INT, --month to end retrieving performance data
#QUARTERLYORMONTHLY VARCHAR(1), --do you want quarterly or monthly data?
#VEHICLETYPEID INT, --what product vehicle type are you looking for?
#GROSSORNETID INT --are your looking gross of fees data or net of fees data?
)
AS
BEGIN
SET NOCOUNT ON
DECLARE #STARTDATE VARCHAR(6),
#ENDDATE VARCHAR(6),
#vBEGINMONTH VARCHAR(2),
#vENDMONTH VARCHAR(2)
IF LEN(#BEGINMONTH) = 1
SET #vBEGINMONTH = '0' + CAST(#BEGINMONTH AS VARCHAR(1))
ELSE
SET #vBEGINMONTH = #BEGINMONTH
IF LEN(#ENDMONTH) = 1
SET #vENDMONTH = '0' + CAST(#ENDMONTH AS VARCHAR(1))
ELSE
SET #vENDMONTH = #ENDMONTH
SET #STARTDATE = CAST(#BEGINYEAR AS VARCHAR(4)) + #vBEGINMONTH
SET #ENDDATE = CAST(#ENDYEAR AS VARCHAR(4)) + #vENDMONTH
--because null values for gross_or_net_id and vehicle_type_id are represented in
--multiple ways (true null, empty string, or 0) in the PPQ table, need to account for all possible variations if
--a -1 is passed in from the .NET code, which represents an enumerated value that
--indicates that the value(s) should be true null.
IF #VEHICLETYPEID = '-1' AND #GROSSORNETID = '-1'
SELECT
PPQ.YEARMONTH, PPQ.PERFORMANCE
FROM PRODUCT_PERFORMANCE_QUARTERLY PPQ
WITH (NOLOCK)
WHERE
(PPQ.PRODUCT_ID = #PRODUCTID)
AND (PPQ.YEARMONTH BETWEEN #STARTDATE AND #ENDDATE)
AND (PPQ.QUARTERLY_OR_MONTHLY = #QUARTERLYORMONTHLY)
AND (PPQ.VEHICLE_TYPE_ID IS NULL OR PPQ.VEHICLE_TYPE_ID = '0' OR PPQ.VEHICLE_TYPE_ID = '')
AND (PPQ.GROSS_OR_NET_ID IS NULL OR PPQ.GROSS_OR_NET_ID = '0' OR PPQ.GROSS_OR_NET_ID = '')
ORDER BY PPQ.YEARMONTH ASC
IF #VEHICLETYPEID <> '-1' AND #GROSSORNETID <> '-1'
SELECT
PPQ.YEARMONTH, PPQ.PERFORMANCE
FROM PRODUCT_PERFORMANCE_QUARTERLY PPQ
WITH (NOLOCK)
WHERE
(PPQ.PRODUCT_ID = #PRODUCTID)
AND (PPQ.YEARMONTH BETWEEN #STARTDATE AND #ENDDATE)
AND (PPQ.QUARTERLY_OR_MONTHLY = #QUARTERLYORMONTHLY)
AND (PPQ.VEHICLE_TYPE_ID = #VEHICLETYPEID )
AND (PPQ.GROSS_OR_NET_ID = #GROSSORNETID)
ORDER BY PPQ.YEARMONTH ASC
IF #VEHICLETYPEID = '-1' AND #GROSSORNETID <> '-1'
SELECT
PPQ.YEARMONTH, PPQ.PERFORMANCE
FROM PRODUCT_PERFORMANCE_QUARTERLY PPQ
WITH (NOLOCK)
WHERE
(PPQ.PRODUCT_ID = #PRODUCTID)
AND (PPQ.YEARMONTH BETWEEN #STARTDATE AND #ENDDATE)
AND (PPQ.QUARTERLY_OR_MONTHLY = #QUARTERLYORMONTHLY)
AND (PPQ.VEHICLE_TYPE_ID IS NULL OR PPQ.VEHICLE_TYPE_ID = '0' OR PPQ.VEHICLE_TYPE_ID = '')
AND (PPQ.GROSS_OR_NET_ID = #GROSSORNETID)
ORDER BY PPQ.YEARMONTH ASC
IF #VEHICLETYPEID <> '-1' AND #GROSSORNETID = '-1'
SELECT
PPQ.YEARMONTH, PPQ.PERFORMANCE
FROM PRODUCT_PERFORMANCE_QUARTERLY PPQ
WITH (NOLOCK)
WHERE
(PPQ.PRODUCT_ID = #PRODUCTID)
AND (PPQ.YEARMONTH BETWEEN #STARTDATE AND #ENDDATE)
AND (PPQ.QUARTERLY_OR_MONTHLY = #QUARTERLYORMONTHLY)
AND (PPQ.VEHICLE_TYPE_ID = #VEHICLETYPEID)
AND (PPQ.GROSS_OR_NET_ID IS NULL OR PPQ.GROSS_OR_NET_ID = '0' OR PPQ.GROSS_OR_NET_ID = '')
ORDER BY PPQ.YEARMONTH ASC
END
I have seen this happen with indexes that were out of date. It could also be a parameter sniffing problem, where a different query plan is being used for different parameters that come in to the stored procedure.
You should capture the parameters of the slow calls and see if they are the same ones each time it runs slow.
You might also try running the tuning wizard and see if it recommends any indexes.
You don't want to worry about having too many indexes until you can prove that updates and inserts are happening too slow (time needed to modify the index plus locking/contention), or you are running out of disk space for them.
Sounds like another query is running in the background that has locked the table and your innocent query is simply waiting for it to finish
A strange, edge case but I encountered it recently.
If the queries run longer in the application than they do when run from within Management Studio, you may want to check to make sure that Arithabort is set off. The connection parameters used by Management Studio are different from the ones used by .NET.
It seems like it's one of two things - either the parameters on the slow calls are different in some way than on the fast calls, and they're not able to use the indexes as well, or there's some type of locking contention that's holding you up. You say you've checked for blocking locks while a particular process is hung, and saw none - that would suggest that it's the first one. However - are you sure that your staging server (that you can't reproduce this error on) and the development servers (that you can reproduce it on) have the same database configuration? For example, maybe "READ COMMITTED SNAPSHOT" is enabled in production, but not in development, which would cause read contention issues to disappear in production.
If it's a difference in parameters, I'd suggest using SQL Profiler to watch the transactions and capture a few - some slow ones and some faster ones, and then, in a Management Studio window, replace the variables in that SP above with the parameter values and then get an execution plan by pressing "Control-L". This will tell you exactly how SQL Server expects to process your query, and you can compare the execution plan for different parameter combination to see if there's a difference with one set, and work from there to optimize it.
Good luck!