i have couple of stored procedures in our sql server 2008 that gets executed one after another, im facing a sql error (Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding) at the last one which updates or inserts against a 1 million records table while it works with smaller files but when updating or inserting this number of records it stops and throws that error , how can i tune the stored procedure, i tried to clean the server with sp_updatestates but no luck, i'd appreciate any help, thank you, here is my stored procedure code :
USE [DBTest]
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
ALTER PROCEDURE [dbo].[SP_UpdateData]
#ItemsExt dbo.TT_ItemsExt READONLY
AS
BEGIN
SET NOCOUNT ON;
Declare
#ItemID nvarchar(36)
,#DescE nvarchar(50)
,etc
,etc
---------------------------------------
DECLARE Item_cursor CURSOR FOR
SELECT
ItemID
,DescE
,etc
,etc
FROM #ItemsExt
OPEN Item_cursor
FETCH NEXT FROM Item_cursor INTO
#ItemID
,#DescE
,etc
,etc
WHILE ##FETCH_STATUS = 0
BEGIN
If exists (select * from Items WHERE ItemID = #ItemID)
Begin
Update dbo.Items SET
DescEng = #DescE
,etc = #etc
WHERE ItemID = #ItemID
End
ElSE
Begin
Declare #ItemNumber nvarChar(36)
SET #ItemNumber = REPLACE((REPLACE(RIGHT(#ItemID, LEN(#ItemID) - 1))
insert into dbo.Items
(ItemID , etc)
VALUES (#ItemID, etc)
End
--ItemsReplacment Table
If exists (select * from dbo.ItemsReplacement WHERE ItemID = #ItemID)
Begin
Update dbo.ItemsReplacement SET
ItemAlter= #AlternativeItem
,etc
WHERE ItemID = #ItemID
End
ElSE
Begin
Insert Into dbo.ItemsReplacement Values (
#ItemID
,etc
,etc)
End
FETCH NEXT FROM Item_cursor INTO
#ItemID
,#DescE
,etc
,etc
END
CLOSE Item_cursor;
DEALLOCATE Item_cursor;
END
ps: the timeout of execution is set to 0 meaning infinite.
Here's a better aproach to those updates and inserts:
ALTER PROCEDURE [dbo].[SP_UpdateData]
#ItemsExt dbo.TT_ItemsExt READONLY
AS
BEGIN
UPDATE IT
SET
DescEng = ITE.DescE,
etc = ITE.etc
FROM
dbo.Items IT
INNER JOIN ItemsExt ITE ON (IT.ItemID = ITE.ItemID);
INSERT INTO
dbo.Items
(
ItemID,
etc
)
SELECT
ITE.ItemID,
ITE.etc
FROM
dbo.ItemsExt ITE;
WHERE
NOT EXISTS (SELECT 1 FROM dbo.Items WHERE ItemID = ITE.ItemID);
UPDATE ITR
SET
ItemAlt = 0, --AlternativeItem?
etc = ITE.etc
FROM
dbo.ItemsReplacement ITR
INNER JOIN ItemsExt ITE ON (ITR.ItemID = ITE.ItemID);
INSERT INTO
dbo.ItemsReplacement
(
ItemID,
etc
)
SELECT
ITE.ItemID,
ITE.etc
FROM
dbo.ItemsExt ITE;
WHERE
NOT EXISTS (SELECT 1 FROM dbo.ItemsReplacement WHERE ItemID = ITE.ItemID);
END
That way you avoid unecessary cursor-loops and let the Database do it's magic on planning the execution.
And by the way: Shouldn't it have any kind of "restriction" on those updates and inserts? Are you planning on run all those tables on all executions?
!! On your original code you were using #AlternativeItem that I couldn't find anywhere. Used 0 in my response.
Related
I have a table where item balances are stored.
CREATE TABLE itembalance (
ItemID VARCHAR(15),
RemainingQty INT,
Cost Money,
Id INT
)
I need to make sure that whenever an item is being sent out, the proper balances are deducted from the itembalance table. I do it this way:
DECLARE crsr CURSOR LOCAL FAST_FORWARD FOR
SELECT
itembalance.Cost,
itembalance.RemainingQty
itembalance.Id
FROM dbo.itembalance
WHERE itembalance.ItemID = #v_item_to_be_updated AND RemainingQty > 0
OPEN crsr
FETCH crsr
INTO
#cost,
#qty,
#id
WHILE ##FETCH_STATUS = 0
BEGIN
IF #qty >= #qty_to_be_deducted
BEGIN
UPDATE itembalance SET RemainingQty = RemainingQty - #qty_to_be_deducted WHERE Id = #id
/*do something with cost*/ BREAK
END
ELSE
BEGIN
UPDATE itembalance SET RemainingQty = 0 WHERE Id = #id
/*do something with cost*/ SET #qty_to_be_deducted = #qty_to_be_deducted - #qty
END
FETCH crsr
INTO
#cost,
#qty,
#id
END
CLOSE crsr
DEALLOCATE crsr
The table may contain same item code but with different cost. This code is okay for few items being updated at a time but whenever a lot of items/quantities are being sent out, the process becomes really slow. Is there a way to optimize this code? I am guessing the cursor is making it slow so I want to explore a different code for this process.
This looks like you just need a simple CASE expression:
UPDATE dbo.itembalance
SET Qty = CASE WHEN Qty >= #qty_to_be_deducted THEN Qty - #qty_to_be_deducted ELSE 0 END
WHERE ItemID = #v_item_to_be_updated
--What is the difference between Qty and RemainingQty?
--Why are you checking one and updating the other?
AND RemainingQty > 0;
You code is not very clear as to how and why the mechanism is required and works.
However assuming that you must have multiple records with an outstanding balance, and that you must consider multiple records sequentially as part of this mechanism, then you have two options to solve that within SQL (handling in client code is another option):
1) Use a cursor as you have done
2) Use a temp table or table variable and iterate over it - pretty similar to a cursor but might be faster - you'd have to try and see e.g.
declare #TableVariable table (Cost money, RemainingQty int, Id int, OrderBy int, Done bit default(0))
declare #Id int, #Cost money, #RemainingQty int
insert into #TableVariable (Cost, RemainingQty, Id, OrderBy)
SELECT
itembalance.Cost
, itembalance.RemainingQty
, itembalance.Id
, 1 /* Some order by condition */
FROM dbo.itembalance
WHERE itembalance.ItemID = #v_item_to_be_updated AND RemainingQty > 0
while exists (select 1 from #TableVariable where Done = 0) begin
select top 1 #Id = id, #Cost = Cost, #RemainingQty
from #TableVariable
where Done = 0
order by OrderBy
-- Do stuff here
update #TableVariable set Done = 1 where id = #Id
end
However the code you have shown doesn't appear that it should be slow - so it may be that you are lacking the appropriate indexes, and that a single ItemId update is locking too many rows in the ItemBalance table which is then affecting other ItemId updates.
I created a trigger as below
alter trigger sale_Trigger_Update on sale
after update
as
begin
Declare #old_value varchar(50)
Declare #new_value varchar(50)
Declare #sale_id UNIQUEIDENTIFIER
DECLARE new_cur CURSOR FORWARD_ONLY READ_ONLY LOCAL FOR
SELECT saleid
FROM INSERTED
open new_cur
Fetch Next from new_cur into #sale_id
while ##FETCH_STATUS = 0
Begin
set #old_value = (select enddate from deleted where SaleID = #sale_id)
set #new_value = (select enddate from inserted where SaleID = #sale_id)
insert into zzz (old_value,new_value) values(#old_value,#new_value)
end
CLOSE new_cur
DEALLOCATE new_cur
end
Then I do an update statement as below
update sale
set enddate = null
Sale table contain only 2 rows
and the execution is continuing unlimited.
I tried
update sale
set enddate = null
where saleid = 10
same problem.
Then I forcefully stopped the execution. Then checked the sale table and zzz table. No changes happened.
I am sure there is some issue in cursor. Can somebody show some light on it.
****Edited****
Actually I need to check enddate in deleted is null and enddate in inserted is not null
open new_cur
Fetch Next from new_cur into #sale_id
while ##FETCH_STATUS = 0
Begin
set #old_value = (select enddate from deleted where SaleID = #sale_id)
set #new_value = (select enddate from inserted where SaleID = #sale_id)
if #old_value = null and #new_value != null
begin
SELECT approval.*,
(select diag.*
from diag diag
where approval.id =diag.id
FOR XML PATH('diag'), TYPE
),
(select ser.*
from ser ser
where approval.id =ser.id
FOR XML PATH('ser'), TYPE
)
FROM approval approval,
where approval.id = 1
and approval.saleid =#saleid
FOR XML PATH, ELEMENTS,
root('Head')
end if
end
CLOSE new_cur
DEALLOCATE new_cur
Regarding the cursor.
Could you replace your trigger with?
alter trigger sale_Trigger_Update on sale
after update
as
begin
insert into zzz (old_value,new_value)
select
--i.SalesID,
d.enddate,
i.enddate
from inserted i
inner join deleted d on
i.SaleID = d.SaleID
where
d.enddate is null and
i.enddate is not null
end
I have declared a temp table to hold all the required values as follows:
DECLARE #temp TABLE
(
Password INT,
IdTran INT,
Kind VARCHAR(16)
)
INSERT INTO #temp
SELECT s.Password, s.IdTran, 'test'
from signal s inner join vefify v
on s.Password = v.Password
and s.IdTran = v.IdTran
and v.type = 'DEV'
where s.[Type] = 'start'
AND NOT EXISTS (SELECT * FROM signal s2
WHERE s.Password = s2.Password
and s.IdTran = s2.IdTran
and s2.[Type] = 'progress' )
INSERT INTO #temp
SELECT s.Password, s.IdTran, 'test'
FROM signal s inner join vefify v
on s.Password = v.Password
and s.IdTran = v.IdTran
and v.type = 'PROD'
where s.[Type] = 'progress'
AND NOT EXISTS (SELECT * FROM signal s2
WHERE s.Password = s2.Password
and s.IdTran = s2.IdTran
and s2.[Type] = 'finish' )
Now i need to loop through the rows in the #temp table and and for each row call a sp that takes all the parameters of #temp table as input.
How can I achieve this?
you could use a cursor:
DECLARE #id int
DECLARE #pass varchar(100)
DECLARE cur CURSOR FOR SELECT Id, Password FROM #temp
OPEN cur
FETCH NEXT FROM cur INTO #id, #pass
WHILE ##FETCH_STATUS = 0 BEGIN
EXEC mysp #id, #pass ... -- call your sp here
FETCH NEXT FROM cur INTO #id, #pass
END
CLOSE cur
DEALLOCATE cur
Try returning the dataset from your stored procedure to your datatable in C# or VB.Net. Then the large amount of data in your datatable can be copied to your destination table using a Bulk Copy. I have used BulkCopy for loading large datatables with thousands of rows, into Sql tables with great success in terms of performance.
You may want to experiment with BulkCopy in your C# or VB.Net code.
something like this?
DECLARE maxval, val, #ind INT;
SELECT MAX(ID) as maxval FROM table;
while (ind <= maxval ) DO
select `value` as val from `table` where `ID`=ind;
CALL fn(val);
SET ind = ind+1;
end while;
You can do something like this
Declare #min int=0, #max int =0 --Initialize variable here which will be use in loop
Declare #Recordid int,#TO nvarchar(30),#Subject nvarchar(250),#Body nvarchar(max) --Initialize variable here which are useful for your
select ROW_NUMBER() OVER(ORDER BY [Recordid] ) AS Rownumber, Recordid, [To], [Subject], [Body], [Flag]
into #temp_Mail_Mstr FROM Mail_Mstr where Flag='1' --select your condition with row number & get into a temp table
set #min = (select MIN(Rownumber) from #temp_Mail_Mstr); --Get minimum row number from temp table
set #max = (select Max(Rownumber) from #temp_Mail_Mstr); --Get maximum row number from temp table
while(#min <= #max)
BEGIN
select #Recordid=Recordid, #To=[To], #Subject=[Subject], #Body=Body from #temp_Mail_Mstr where Rownumber=#min
-- You can use your variables (like #Recordid,#To,#Subject,#Body) here
-- Do your work here
set #min=#min+1 --Increment of current row number
END
You always don't need a cursor for this. You can do it with a while loop. You should avoid cursors whenever possible. While loop is faster than cursors.
I am not good at SQL Server 2000. I have a comma-delimited list of ids. I need to see if that ID exists in a table. If it does, I want to break out of the loop with that ID saved in a variable that I can use in my stored procedure. This is what I am trying right now:
DECLARE #coreID INT
SET #coreID=NULL
DECLARE #itemID NVARCHAR(50)
DECLARE itemCursor CURSOR LOCAL FAST_FORWARD FOR
SELECT [String] AS 'itemID' FROM dbo.SplitListIntoTable(#myIDs)
OPEN itemCursor
FETCH NEXT FROM itemCursor INTO #itemID
WHILE ##FETCH_STATUS = 0 BEGIN
-- If #itemID EXISTS IN MyTable set #coreID=#itemID and Break. How do I do this?
FETCH NEXT FROM itemCursor INTO #itemID
END
CLOSE itemCursor
DEALLOCATE itemCursor
Thank you!
Ideally, you shouldn't use a cursor as performance won't be great. If you can do it as a set-based statement, do that instead, maybe like this:
SELECT TOP 1 #CoreID = [String]
FROM dbo.SplitListIntoTable(#myIDs) x
JOIN MyTable t ON x.[String] = t.ID
However, if you have a real reason to use a cursor, you can use the BREAK statement to break out of a WHILE loop
e.g.
WHILE ##FETCH_STATUS = 0
BEGIN
IF EXISTS(SELECT * FROM MyTable WHERE Id = #ItemID)
BEGIN
SET #CoreId = #ItemId
BREAK
END
FETCH NEXT FROM itemCursor INTO #itemID
END
I don't know how to do this using a cursor, but I supect you can do this much better (faster) with a a join. If the output of dbo.SplitListIntoTable(#myIDs) is actually an odered table, then you can output a table with another column what is say the string numer, 1, 2, 3, etc...
(I don't have sql in front of me to test this but something like)
create table t(itemNum int identity, itemId nvarchar(max))
insert into t (item id) select 1 from dbo.SplitListIntoTable(#myIDs)
Then join the two and take the top one
set #coreID =
select top 1 #itemID
from MyTable m
inner join t t.itemid = m.itemid
order by m.itemNum asc
of course you could use a CTE, table var or temp table too.
Before I go any further: Yes, I know that cursors perform poorly compared with set-based operations. In this particular case I'm running a cursor on a temporary table of 100 or so records, and that temporary table will always be fairly small, so performance is less crucial than flexibility.
My difficulty is that I'm having trouble finding an example of how to update a column fetched by a cursor. Previously when I've used cursors I've retrieved values into variables, then run an update query at each step based upon these values. On this occasion I want to update a field in the temporary table, yet I can't figure out how to do it.
In the example below, I'm trying to update the field CurrentPOs in temporary table #t1, based upon a query that uses #t1.Product_ID to look up the required value. You will see in the code that I have attempted to use the notation curPO.Product_ID to reference this, but it doesn't work. I have also attempted to use an update statement against curPO, also unsuccessfully.
I can make the code work by fetching to variables, but I'd like to know how to update the field directly.
I think I'm probably missing something obvious, but can anyone help?
declare curPO cursor
for select Product_ID, CurrentPOs from #t1
for update of CurrentPOs
open curPO
fetch next from curPO
while ##fetch_status = 0
begin
select OrderQuantity = <calculation>,
ReceiveQuantity = <calculation>
into #POs
from PurchaseOrderLine POL
inner join SupplierAddress SA ON POL.Supplier_ID = SA.Supplier_ID
inner join PurchaseOrderHeader POH ON POH.PurchaseOrder_ID = POL.PurchaseOrder_ID
where Product_ID = curPO.Product_ID
and SA.AddressType = '1801'
update curPO set CurrentPOs = (select sum(OrderQuantity) - sum(ReceiveQuantity) from #POs)
drop table #POs
fetch next from curPO
end
close curPO
deallocate curPO
After doing a bit more googling, I found a partial solution. The update code is as follows:
UPDATE #T1
SET CURRENTPOS = (SELECT SUM(ORDERQUANTITY) - SUM(RECEIVEQUANTITY)
FROM #POS)
WHERE CURRENT OF CURPO
I still had to use FETCH INTO, however, to retrieve #t1.Product_ID and run the query that produces #POs, so I'd still like to know if it's possible to use FETCH on it's own.
Is this what you want?
declare curPO cursor
for select Product_ID, CurrentPOs from #t1
for update of CurrentPOs
open curPO
fetch next from curPO
while ##fetch_status = 0
begin
update curPO set CurrentPOs =
(select sum(<OrderQuantityCalculation>)
from PurchaseOrderLine POL
inner join SupplierAddress SA ON POL.Supplier_ID = SA.Supplier_ID
inner join PurchaseOrderHeader POH ON POH.PurchaseOrder_ID = POL.PurchaseOrder_ID
where Product_ID = curPO.Product_ID
and SA.AddressType = '1801') -
(select sum(<ReceiveQuantityCalculation>)
from PurchaseOrderLine POL
inner join SupplierAddress SA ON POL.Supplier_ID = SA.Supplier_ID
inner join PurchaseOrderHeader POH ON POH.PurchaseOrder_ID = POL.PurchaseOrder_ID
where Product_ID = curPO.Product_ID
and SA.AddressType = '1801')
fetch next from curPO
end
close curPO
deallocate curPO
Maybe you need something like that:
update DataBaseName..TableName
set ColumnName = value
where current of your_cursor_name;
Here's an example to calculate one column based upon values from two others (note, this could be done during the original table select). This example can be copy / pasted into an SSMS query window to be run without the need for any editing.
DECLARE #cust_id INT = 2, #dynamic_val NVARCHAR(40), #val_a INT, #val_b INT
DECLARE #tbl_invoice table(Cust_ID INT, Cust_Fees INT, Cust_Tax INT)
INSERT #tbl_invoice ( Cust_ID, Cust_Fees, Cust_Tax ) SELECT 1, 111, 11
INSERT #tbl_invoice ( Cust_ID, Cust_Fees, Cust_Tax ) SELECT 2, 222, 22
INSERT #tbl_invoice ( Cust_ID, Cust_Fees, Cust_Tax ) SELECT 3, 333, 33
DECLARE #TblCust TABLE
(
Rec_ID INT
, Val_A INT
, Val_B INT
, Dynamic_Val NVARCHAR(40)
, PRIMARY KEY NONCLUSTERED (Rec_ID)
)
INSERT #TblCust(Rec_ID, Val_A, Val_B, Dynamic_Val)
SELECT Rec_ID = Cust_ID, Val_A = Cust_Fees, Val_B = Cust_Tax, NULL
FROM #tbl_invoice
DECLARE cursor_cust CURSOR FOR
SELECT Rec_ID, Val_A, Val_B, Dynamic_Val
FROM #TblCust
WHERE Rec_ID <> #cust_id
FOR UPDATE OF Dynamic_Val;
OPEN cursor_cust;
FETCH NEXT FROM cursor_cust INTO #cust_id, #val_a, #val_b, #dynamic_val;
WHILE ##FETCH_STATUS = 0
BEGIN
UPDATE #TblCust
SET Dynamic_Val = N'#c = "' + LTRIM(STR((#val_a + #val_b), 40)) + N'"'
WHERE CURRENT OF cursor_cust
FETCH NEXT FROM cursor_cust INTO #cust_id, #val_a, #val_b, #dynamic_val;
END
CLOSE cursor_cust
DEALLOCATE cursor_cust
SELECT * FROM #TblCust