Looking for help solving this deadlock issue... I have a stored proc that is called rather frequently from a number of processes but on a small table (quantity of rows is in low thousands)... Once in a while, I'm getting a deadlock on the proc.
Proc's purpose is to return the next 'eligible for processing' rows. It is very important to not return the same row to two simultaneous calls of the proc.... (that part works fine)... but I can't understand why sometimes there would be a deadlock.
This is a SQL Azure database
Appreciate the help
CREATE PROCEDURE [dbo].[getScheduledAccounts_Monitor]
-- Add the parameters for the stored procedure here
#count int,
#timeout int = 1200,
#forcedAccountId uniqueidentifier = NULL
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE;
DECLARE #batchId uniqueidentifier
SELECT #batchId = NEWID()
BEGIN TRAN
-- Update rows
UPDATE Schedule
WITH (ROWLOCK)
SET
LastBatchId = #batchId,
LastStartedProcessingId = NEWID(),
LastStartedProcessingTime = GETUTCDATE(),
LastCompletedProcessingId = ISNULL(LastCompletedProcessingId, NEWID()),
LastCompletedProcessingTime = ISNULL(LastCompletedProcessingTime, GETUTCDATE())
WHERE
ActivityType = 'Monitor' AND
IsActive = 1 AND
AccountId IN (
SELECT TOP (#count) AccountId
FROM Schedule
WHERE
(LastStartedProcessingId = LastCompletedProcessingId OR LastCompletedProcessingId IS NULL OR DATEDIFF(SECOND, LastStartedProcessingTime, GETUTCDATE()) > #timeout) AND
IsActive = 1 AND ActivityType = 'Monitor' AND
(LastStartedProcessingTime IS NULL OR DATEDIFF(SECOND, LastStartedProcessingTime, GETUTCDATE()) > Frequency)
ORDER BY (DATEDIFF(SECOND, LastStartedProcessingTime, GETUTCDATE()) - Frequency) DESC
) AND
AccountId = ISNULL(#forcedAccountId, AccountID)
-- Return the changed rows
SELECT AccountId, LastStartedProcessingId, Frequency, LastProcessTime, LastConfigChangeTime, ActivityType
FROM Schedule
WHERE LastBatchId = #batchId
COMMIT TRAN
END
You can use an application lock to ensure single execution. The article below is for SQL 2005 but I am sure the solution applies for newer versions also.
http://www.sqlteam.com/article/application-locks-or-mutexes-in-sql-server-2005
Related
This stored procedure completes in between 10 - 20 milliseconds (usually) until it gets called 27 times in a load testing session and then it slows way down to 300 - 400 milliseconds per call. There's a clear jump.
After starting over (and running it a couple times with some time between) it's fast again.
What is causing it to jump up so dramatically? How can I get around it?
My requirements are to handle 50 active/concurrent users, but I'm going to blow through that fast if the big call is taking that long.
The stored procedure deals with a parent child relationship. First it clears it all, then it tries to repopulate it from a de-normalized table parameter (faster than nested DB calls in the code).
I would like to remove the FOR loop, but since I don't know the ID of the parent, I don't know how to preserve the FK for the child without procedurally creating the children immediately after the parent is created.
CREATE OR ALTER PROCEDURE dbo.sp_DeleteThenCreateRoleAndMembers
#PenguineHouseRoleMemberUpdate PenguineHouseRoleMemberUpdateType READONLY
AS
BEGIN
-- clear out all the existing roles and stuff
DELETE PenguinHouseRoleMember
FROM PenguinHouseRoleMember phrm
INNER JOIN PenguinHouseRole ph on phrm.PenguinHouseRoleId = ph.Id
WHERE ph.PenguinHouseId = (
SELECT TOP 1 PenguinHouseId from #PenguineHouseRoleMemberUpdate
)
-- cursor setup
DECLARE #PenguinHouseId int;
DECLARE #RoleType int;
DECLARE #SubRoleIds varchar;
DECLARE #RoleId int
DECLARE #RowCnt int = 0;
DECLARE #CounterId int = 1;
SELECT #RowCnt = COUNT(*) FROM #PenguineHouseRoleMemberUpdate;
WHILE #CounterId <= #RowCnt
BEGIN
SELECT #PenguinHouseId = PenguinHouseId,
#RoleType = RoleType,
#SubRoleIds = SubRoleIds
FROM #PenguineHouseRoleMemberUpdate
-- insert parent record
INSERT INTO PenguinHouseRole (RoleType, PenguinHouseId, CreatedAt, CreatedBy, UpdatedBy, UpdatedAt)
VALUES(#RoleType, #PenguinHouseId, GETDATE(), 'system', 'system', GETDATE())
SET #RoleId = SCOPE_IDENTITY()
-- identify children (subroles)
DROP TABLE IF EXISTS #TempSubRoleUpdateTable
SELECT sq.subRoleId INTO #TempSubRoleUpdateTable
FROM (
select convert(int, value) subRoleId FROM string_split(#SubRoleIds, ',')
) sq
-- insert children
IF (#SubRoleIds IS NOT NULL AND LEN(LTRIM(RTRIM(#SubRoleIds))) > 0)
BEGIN
INSERT INTO dbo.PenguinHouseRoleMember
(
workflowinstanceroleid,
subroleid,
AllTasksCompleted,
Approved,
SuggestedBySystem,
SelectedByCurator,
CreatedAt, CreatedBy, UpdatedBy, UpdatedAt
)
select
#RoleId,
subRoleId,
0,
0,
1,
0,
GETDATE(), 'system', 'system', GETDATE()
from
#TempSubRoleUpdateTable
END
SET #CounterId = #CounterId + 1
END -- loop
END
I'm writing a C++ application that is connecting to a SQL Server database via ODBC.
I need an Archive function so I'm going to write a stored procedure that takes a date. It will total up all the transactions and payments prior to that date for each customer, update the customer's starting balance accordingly, and then delete all transactions and payments prior to that date.
It occurs to me that it could be very bad if someone else is adding or deleting transactions or payments at the same time this stored procedure runs. Therefore, I'm thinking I should lock the entire database during execution, which would not happen that often.
I'm curious if my logic is good and what would be the best way to lock the entire database for such a purpose.
UPDATE:
Based on user12069178's answer, here's what I've come up with so far. Would appreciate any feedback on it.
ALTER PROCEDURE [dbo].[ArchiveData] #ArchiveDateTime DATETIME
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
DECLARE #TempTable TABLE
(
CustomerId INT,
Amount BIGINT
);
BEGIN TRANSACTION;
-- Archive transactions
DELETE Transactions WITH (TABLOCK)
OUTPUT deleted.CustomerId, deleted.TotalAmount INTO #TempTable
WHERE [TimeStamp] < #ArchiveDateTime;
IF EXISTS (SELECT 1 FROM #TempTable)
BEGIN
UPDATE Customers SET StartingBalance = StartingBalance +
(SELECT SUM(Amount) FROM #TempTable temp WHERE Id = temp.CustomerId)
END;
DELETE FROM #TempTable
-- Archive payments
DELETE Payments WITH (TABLOCK)
OUTPUT deleted.CustomerId, deleted.Amount INTO #TempTable
WHERE [Date] < #ArchiveDateTime;
IF EXISTS (SELECT 1 FROM #TempTable)
BEGIN
UPDATE Customers SET StartingBalance = StartingBalance -
(SELECT SUM(Amount) FROM #TempTable temp WHERE Id = temp.CustomerId)
END;
COMMIT TRANSACTION;
END
Generally the way to make sure that the rows you are deleting are the ones that you are totalling and inserting is to use the OUTPUT clause while deleting. It can output the rows that were selected for deletion.
Here's a setup that will give us some transactions:
USE tempdb;
GO
DROP TABLE IF EXISTS dbo.Transactions;
GO
CREATE TABLE dbo.Transactions
(
TransactionID int NOT NULL IDENTITY(1,1)
CONSTRAINT PK_dbo_Transactions
PRIMARY KEY,
TransactionAmount decimal(18,2) NOT NULL,
TransactionDate date NOT NULL
);
GO
SET NOCOUNT ON;
DECLARE #Counter int = 1;
WHILE #Counter <= 50
BEGIN
INSERT dbo.Transactions
(
TransactionAmount, TransactionDate
)
VALUES (ABS(CHECKSUM(NewId())) % 10 + 1, DATEADD(day, 0 - #Counter * 3, GETDATE()));
SET #Counter += 1;
END;
SELECT * FROM dbo.Transactions;
GO
Now the following code deletes the rows past a cutoff, and concurrently outputs the amounts into a table variable, and then inserts the total row into the transactions table.
DECLARE #CutoffDate date = DATEADD(day, 1, EOMONTH(DATEADD(month, -2, GETDATE())));
DECLARE #TransactionAmounts TABLE
(
TransactionAmount decimal(18,2)
);
BEGIN TRAN;
DELETE dbo.Transactions WITH (TABLOCK)
OUTPUT deleted.TransactionAmount INTO #TransactionAmounts
WHERE TransactionDate < #CutoffDate;
IF EXISTS (SELECT 1 FROM #TransactionAmounts)
BEGIN
INSERT dbo.Transactions (TransactionAmount, TransactionDate)
SELECT SUM(TransactionAmount), DATEADD(day, 1, #CutoffDate)
FROM #TransactionAmounts;
END;
COMMIT;
I usually try to avoid specifying locks whenever possible but based on your suggestion, I've added it. If you didn't have the table lock, it'd still be ok but would mean that even if someone adds in a new "old" row while you're doing this, it won't be in the total or deleted either. Making the transaction serializable would also achieve the outcome and would lock less than the table lock if the number of rows being deleted was less than the lock escalation threshold (defaults to 5000).
Hope that helps.
I have a table that acts somewhat of a queue except records are never removed from the table. What I am trying to attempt is have the stored procedure return a single record that is not currently being processed from a front end application. What we have is a "Locked" column we set to indicate this. The reason we are doing this is so only one call center agent can work on a record at a time. Here is what my sql looks like so far. The problem is if I run this query from two separate sessions (the second session comments out the waitfor statement) the second session does not return any records for 10 seconds. I've narrowed it down to the order by clause when selecting a record. If I remove the Order By it returns but I need the order by.
Or maybe my query is competently wrong? should I be using a transaction isolation level (serializable, snapshot)?? any guidance would be great!
DECLARE #WorkItemId INT;
BEGIN TRANSACTION
/* TODO: Skip Records That Have Been Completed. */
SET #WorkItemId = (SELECT TOP 1 CampaignSetDetailId FROM CampaignSetDetail WITH (XLOCK, READPAST) WHERE LockedBy IS NULL ORDER BY NEWID(), NumAttempts ASC);
/* */
UPDATE CampaignSetDetail SET LockedBy = 'MPAUL', LockedDTM = GETUTCDATE() WHERE CampaignSetDetailId = #WorkItemId;
/* */
SELECT * FROM CampaignSetDetail WHERE CampaignSetDetailId = #WorkItemId;
WAITFOR DELAY '00:00:10';
COMMIT TRANSACTION
Try the following - a set of retry attempts, and also using update lock.
DECLARE #RetryCount Int = 1
DECLARE #MaxRetries Int = 5;
SET #RetryCount = 1
WHILE #RetryCount < #MaxRetries
BEGIN
BEGIN TRY
/* TODO: Skip Records That Have Been Completed. */
SET #WorkItemId = (
SELECT TOP 1
CampaignSetDetailId
FROM CampaignSetDetail WITH (UPDLOCK)
WHERE LockedBy IS NULL
ORDER BY NEWID()
,NumAttempts ASC
);
UPDATE CampaignSetDetail WITH (UPDLOCK)
SET LockedBy = 'MPAUL'
,LockedDTM = GETUTCDATE()
WHERE CampaignSetDetailId = #WorkItemId;
SELECT *
FROM CampaignSetDetail WITH (UPDLOCK)
WHERE CampaignSetDetailId = #WorkItemId;
SELECT #RetryCount = #MaxRetries;
END TRY
BEGIN CATCH
IF ERROR_NUMBER() IN (1204, 1205, 1222)
BEGIN
SET #RetryCount += 1;
WAITFOR DELAY '00:00:02';
END
ELSE
THROW;
END CATCH
END
For more details about specific lock errors, see here. This should work even if updates on the same table are executed from other sessions.
You could do it in one statement
DECLARE #WorkItemId INT;
UPDATE [CampaignSetDetail]
SET
[LockedBy] = 'MPAUL',
[LockedDTM] = GETUTCDATE()
WHERE
[CampaignSetDetailId] = (
SELECT TOP 1
[CampaignSetDetailId],
#WorkItemId = [CampaignSetDetailId]
FROM
[CampaignSetDetail]
WHERE
[LockedBy] IS NULL
ORDER BY
[NumAttempts]);
SELECT
* -- You shouldn't do this.
FROM
[CampaignSetDetail]
WHERE
[CampaignSetDetailId] = #WorkItemId;
I'm assuming [CampaignSetDetailId] forms the clustered index on the table. You could consider an index like,
CREATE INDEX [IX_CampaignSetDetail_NumAttempts]
ON [CampaignSetDetail]([NumAttempts])
WHERE [LockedBy] IS NULL;
to optimize this operation.
i think i found a good solution but i am still evaluating all the answers. i am also trying to use a CTE and this is giving me the best performance right now.
DECLARE #WorkItemId INT;
WITH CTE AS (
SELECT TOP 1 [CampaignSetDetailId], LockedBy FROM [dbo].[CampaignSetDetail] WITH (ROWLOCK, READPAST) WHERE LockedBy IS NULL ORDER BY NEWID(), NumAttempts ASC
)
UPDATE CTE SET LockedBy = 'DIESEL', #WorkItemId = [CampaignSetDetailId]
SELECT #WorkItemId
WAITFOR DELAY '00:00:05';
Guys I have a stored procedure that inserts a new value in the table, only when the last inserted value is different.
CREATE PROCEDURE [dbo].[PutData]
#date datetime,
#value float
AS
IF NOT EXISTS(SELECT * FROM Sensor1 WHERE SensorTime <= #date AND SensorTime = (SELECT MAX(SensorTime) FROM Sensor1) AND SensorValue = #value)
INSERT INTO Sensor1 (SensorTime, SensorValue) VALUES (#date, #value)
RETURN 0
Now, since I'm doing this at a high frequency (say every 10ms), the IF NOT EXISTS (SELECT) statement is often getting old data, and because of this I'm getting duplicate data. Would it be possible to lock the entire table during the stored procedure, to make sure the SELECT statement always receives the latest data?
According to the poster's comments to the question, c# code receives a value from a sensor. The code is supposed to insert the value only if it is different from the previous value.
Rather than solving this in the database, why not have the code store the last value inserted and only invoke the procedure if the new value is different? Then the procedure will not need to check whether the value exists in the database; it can simply insert. This will be much more efficient.
You could write it like this :
CREATE PROCEDURE [dbo].[PutData]
#date datetime,
#value float
AS
BEGIN TRANSACTION
INSERT INTO Sensor1 (SensorTime, SensorValue)
SELECT SensorTime = #date,
SensorValue = #value
WHERE NOT EXISTS(SELECT *
FROM Sensor1 WITH (UPDLOCK, HOLDLOCK)
WHERE SensorValue = #value
AND SensorTime <= #date
AND SensorTime = (SELECT MAX(SensorTime) FROM Sensor1) )
COMMIT TRANSACTION
RETURN 0
Thinking a bit about it, you could probably write it like this too:
CREATE PROCEDURE [dbo].[PutData]
#date datetime,
#value float
AS
BEGIN TRANSACTION
INSERT INTO Sensor1 (SensorTime, SensorValue)
SELECT SensorTime = #date,
SensorValue = #value
FROM (SELECT TOP 1 SensorValue, SensorTime
FROM Sensor1 WITH (UPDLOCK, HOLDLOCK)
ORDER BY SensorTime DESC) last_value
WHERE last_value.SensorTime <= #date
AND last_value.SensorValue <> #value
COMMIT TRANSACTION
RETURN 0
Assuming you have a unique index (PK?) on SensorTime this should be quite fast actually.
i got a little problem in my SQL Query. The following error is :
Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
And here is my SQL Query
ALTER PROC sp_savepresence
#Username char(20),
#Image text
AS
BEGIN
------------
DECLARE #PresStatus CHAR,
#DateDiff INT,
#ClockIn DATETIME,
#InsertData varchar(20) = 'TranSavePresence';
IF NOT EXISTS(SELECT Username FROM PresenceTransaction WHERE Username=#Username AND ClockOut IS NULL)
BEGIN
BEGIN TRANSACTION #InsertData
INSERT INTO PresenceTransaction
(
Username,
[Image],
PresenceStatus,
WorkHour,
ClockIn,
ClockOut
)
VALUES
(
#Username,
#Image,
'N',
0,
getdate(),
NULL
)
END
ELSE
BEGIN
SELECT #ClockIn = ClockIn, #DateDiff = DateDiff(MINUTE, #ClockIn, getDate()) FROM PresenceTransaction WHERE Username=#Username AND ClockOut IS NULL AND PresenceStatus = 'N'
IF #DateDiff IS NOT NULL
BEGIN
SELECT #PresStatus = 'P'
END
ELSE
BEGIN
SELECT #PresStatus='N'
END
UPDATE PresenceTransaction
SET
PresenceStatus = #PresStatus,
WorkHour = #DateDiff,
ClockOut = getDate()
WHERE Username=#Username AND ClockOut IS NULL AND PresenceStatus = 'N'
END
------------
IF(##Error <> 0)
BEGIN
PRINT ##Error
Rollback Tran #InsertData
SELECT ##Error AS [Status]
END
ELSE
BEGIN
COMMIT TRAN #InsertData
SELECT 'True' AS [Status]
END
END
GO
I have already read from some articles over the Internet, and some of articles, tell me to tune up my query. But i don't know where's the error point or maybe deadlock point, and I don't know how to tune up my query code. Thanks :)
Your stored procedure code conditionally starts a transaction, but commits if there was no error, rather than also checking if a transaction is in progress. See ##TRANCOUNT.
The fact you are trying to use a named transaction suggests there are other transactions likely to be active. Unless you are a guru (and I'm not) I would strongly suggest not using named nested transactions. It is hard to get right and often leads to confusing, hard to maintain code.