I've added a new column, packageNo, to my table:
create table Packages(
id varchar(20) primary key, -- ok, I know :)
orderNo uniqueIdentifier not null,
orderlineNo int not null,
packageNo int not null default(0)
)
Now I want to generate the packageNo with the following rules:
reset it for each order
ascendantfor order, orderline
My problem is that the script I wrote uses 15 minutes for 26500 rows on my testServer.
Here it is:
set NoCount ON
declare #Counter int
declare #handledCounter int
declare #currentorder uniqueIdentifier
declare #fetchedOrder uniqueIdentifier
declare #fetchedId varchar(20) -- will using PK speed up things?
declare PackageNo_Cursor cursor for
select orderNo, id from packages order by orderNo, orderlineNo for update of packageNo
open PackageNo_Cursor
fetch next from PackageNo_Cursor into #fetchedOrder, #fetchedId
set #currentOrder = #fetchedOrder
set #counter = 0
set #handledCounter = 0
while ##fetch_status = 0
begin
if (#currentOrder <> #fetchedOrder)
begin -- reset counter for each order
set #currentOrder = #fetchedOrder
set #counter = 0
end
set #counter = #counter + 1
set #handledCounter = #handledCounter +1
if (#handledCounter % 50 = 0)
begin
print 'handled = ' + cast(#handledCounter as varchar)
end
update packages set packageNo = #counter where current of PackageNo_Cursor
fetch next from PackageNo_Cursor into #fetchedOrder, #fetchedId
end
close PackageNo_Cursor
deallocate PackageNo_Cursor
This should result in:
id - orderno - lineNo - packageNo (what I need to set)
ean1 - guid1 - 1 - 1
ean2 - guid1 - 2 - 2
ean3 - guid2 - 1 - 1
ean15- guid2 - 3 - 2
ean15- guid2 - 4 - 3
Can I make this run any faster?
You don't want to use a cursor, you want to do this as a set, like so:
update p set
packageNo = r.packageNo
from
packages p
inner join
(select orderNo, orderLineNo,
ROW_NUMBER() OVER(PARTITION BY orderNo ORDER BY orderLineNo) as packageNo
from packages) r on
p.orderNo = r.orderNo
and p.orderLineNo = r.orderLineNo
This will leverage SQL Server's ROW_NUMBER function to get you the correct count by each line. Using UPDATE...FROM, we can create an inner join from which to update. This is vastly more efficient than the cursor.
See, cursors add an iterative ability to a procedural and set based language (SQL). The two don't play well together. That update statement is being called in order for each row (and opening/committing a transaction, to boot). If you do it all in one statement, SQL can parallelize this to make it much faster.
The standard rule is this: Use cursors as sparingly as possible. There are some fringe cases where they're necessary, but if you aren't doing massive amounts of SQL administration a day, it's doubtful you'll ever come across those cases.
Something like, not tested
update
p
set
packageNo = p2.Ranking
from
packages p
JOIN
(SELECT
orderNo, orderlineNo,
ROW_NUMBER() OVER (PARTITION BY orderNo ORDER BY orderlineNo) AS Ranking
FROM
packages) p2 ON p.orderNo = p2.orderNo AND p.orderlineNo= p2.orderlineNo
WITH cteA AS (
SELECT
packageNo,
row_number() over (partition by orderNo order by orderNo, orderlineNo) rn
FROM packages
)
UPDATE cteA
SET packageNo = rn - 1;
You should also create a clustered index on orderNo, orderlineNo.
(i hope you are using sql2005 or newer)
In case you really need to use cursors, define your cursor with some attributes, like this :
DECLARE _cursor CURSOR LOCAL FAST_FORWARD FOR
....
Related
I've created a stored procedure that filters and paginates for a DataTable.
Problem: I need to set an OUTPUT variable for #TotalRecords found before an OFFSET occurs, otherwise it sets #TotalRecord to #RecordPerPage.
I've messed around with CTE's and also simply trying this:
SELECT *, #TotalRecord = COUNT(1)
FROM dbo
But that doesn't work either.
Here is my stored procedure, with most of the stuff pulled out:
ALTER PROCEDURE [dbo].[SearchErrorReports]
#FundNumber varchar(50) = null,
#ProfitSelected bit = 0,
#SortColumnName varchar(30) = null,
#SortDirection varchar(10) = null,
#StartIndex int = 0,
#RecordPerPage int = null,
#TotalRecord INT = 0 OUTPUT --NEED TO SET THIS BEFORE OFFSET!
AS
BEGIN
SET NOCOUNT ON;
SELECT *
FROM
(SELECT *
FROM dbo.View
WHERE (#ProfitSelected = 1 AND Profit = 1)) AS ERP
WHERE
((#FundNumber IS NULL OR #FundNumber = '')
OR (ERP.FundNumber LIKE '%' + #FundNumber + '%'))
ORDER BY
CASE
WHEN #SortColumnName = 'FundNumber' AND #SortDirection = 'asc'
THEN ERP.FundNumber
END ASC,
CASE
WHEN #SortColumnName = 'FundNumber' AND #SortDirection = 'desc'
THEN ERP.FundNumber
END DESC
OFFSET #StartIndex ROWS
FETCH NEXT #RecordPerPage ROWS ONLY
Thank you in advance!
You could try something like this:
create a CTE that gets the data you want to return
include a COUNT(*) OVER() in there to get the total count of rows
return just a subset (based on your OFFSET .. FETCH NEXT) from the CTE
So your code would look something along those lines:
-- CTE definition - call it whatever you like
WITH BaseData AS
(
SELECT
-- select all the relevant columns you need
p.ProductID,
p.ProductName,
-- using COUNT(*) OVER() returns the total count over all rows
TotalCount = COUNT(*) OVER()
FROM
dbo.Products p
)
-- now select from the CTE - using OFFSET/FETCH NEXT, get only those rows you
-- want - but the "TotalCount" column still contains the total count - before
-- the OFFSET/FETCH
SELECT *
FROM BaseData
ORDER BY ProductID
OFFSET 20 ROWS FETCH NEXT 15 ROWS ONLY
As a habit, I prefer non-null entries before possible null. I did not reference those in my response below, and limited a working example to just the two inputs you are most concerned with.
I believe there could be some more clean ways to apply your local variables to filter the query results without having to perform an offset. You could return to a temp table or a permanent usage table that cleans itself up and use IDs that aren't returned as a way to set pages. Smoother, with less fuss.
However, I understand that isn't always feasible, and I become frustrated myself with those attempting to solve your use case for you without attempting to answer the question. Quite often there are multiple ways to tackle any issue. Your job is to decide which one is best in your scenario. Our job is to help you figure out the script.
With that said, here's a potential solution using dynamic SQL.
I'm a huge believer in dynamic SQL, and use it extensively for user based table control and ease of ETL mapping control.
use TestCatalog;
set nocount on;
--Builds a temp table, just for test purposes
drop table if exists ##TestOffset;
create table ##TestOffset
(
Id int identity(1,1)
, RandomNumber decimal (10,7)
);
--Inserts 1000 random numbers between 0 and 100
while (select count(*) from ##TestOffset) < 1000
begin
insert into ##TestOffset
(RandomNumber)
values
(RAND()*100)
end;
set nocount off;
go
create procedure dbo.TestOffsetProc
#StartIndex int = null --I'll reference this like a page number below
, #RecordsPerPage int = null
as
begin
declare #MaxRows int = 30; --your front end will probably manage this, but don't trust it. I personally would store this on a table against each display so it can also be returned dynamically with less manual intrusion to this procedure.
declare #FirstRow int;
--Quick entry to ensure your record count returned doesn't excede max allowed.
if #RecordsPerPage is null or #RecordsPerPage > #MaxRows
begin
set #RecordsPerPage = #MaxRows
end;
--Same here, making sure not to return NULL to your dynamic statement. If null is returned from any variable, the entire statement will become null.
if #StartIndex is null
begin
set #StartIndex = 0
end;
set #FirstRow = #StartIndex * #RecordsPerPage
declare #Sql nvarchar(2000) = 'select
tos.*
from ##TestOffset as tos
order by tos.RandomNumber desc
offset ' + convert(nvarchar,#FirstRow) + ' rows
fetch next ' + convert(nvarchar,#RecordsPerPage) + ' rows only'
exec (#Sql);
end
go
exec dbo.TestOffsetProc;
drop table ##TestOffset;
drop procedure dbo.TestOffsetProc;
I have a table where item balances are stored.
CREATE TABLE itembalance (
ItemID VARCHAR(15),
RemainingQty INT,
Cost Money,
Id INT
)
I need to make sure that whenever an item is being sent out, the proper balances are deducted from the itembalance table. I do it this way:
DECLARE crsr CURSOR LOCAL FAST_FORWARD FOR
SELECT
itembalance.Cost,
itembalance.RemainingQty
itembalance.Id
FROM dbo.itembalance
WHERE itembalance.ItemID = #v_item_to_be_updated AND RemainingQty > 0
OPEN crsr
FETCH crsr
INTO
#cost,
#qty,
#id
WHILE ##FETCH_STATUS = 0
BEGIN
IF #qty >= #qty_to_be_deducted
BEGIN
UPDATE itembalance SET RemainingQty = RemainingQty - #qty_to_be_deducted WHERE Id = #id
/*do something with cost*/ BREAK
END
ELSE
BEGIN
UPDATE itembalance SET RemainingQty = 0 WHERE Id = #id
/*do something with cost*/ SET #qty_to_be_deducted = #qty_to_be_deducted - #qty
END
FETCH crsr
INTO
#cost,
#qty,
#id
END
CLOSE crsr
DEALLOCATE crsr
The table may contain same item code but with different cost. This code is okay for few items being updated at a time but whenever a lot of items/quantities are being sent out, the process becomes really slow. Is there a way to optimize this code? I am guessing the cursor is making it slow so I want to explore a different code for this process.
This looks like you just need a simple CASE expression:
UPDATE dbo.itembalance
SET Qty = CASE WHEN Qty >= #qty_to_be_deducted THEN Qty - #qty_to_be_deducted ELSE 0 END
WHERE ItemID = #v_item_to_be_updated
--What is the difference between Qty and RemainingQty?
--Why are you checking one and updating the other?
AND RemainingQty > 0;
You code is not very clear as to how and why the mechanism is required and works.
However assuming that you must have multiple records with an outstanding balance, and that you must consider multiple records sequentially as part of this mechanism, then you have two options to solve that within SQL (handling in client code is another option):
1) Use a cursor as you have done
2) Use a temp table or table variable and iterate over it - pretty similar to a cursor but might be faster - you'd have to try and see e.g.
declare #TableVariable table (Cost money, RemainingQty int, Id int, OrderBy int, Done bit default(0))
declare #Id int, #Cost money, #RemainingQty int
insert into #TableVariable (Cost, RemainingQty, Id, OrderBy)
SELECT
itembalance.Cost
, itembalance.RemainingQty
, itembalance.Id
, 1 /* Some order by condition */
FROM dbo.itembalance
WHERE itembalance.ItemID = #v_item_to_be_updated AND RemainingQty > 0
while exists (select 1 from #TableVariable where Done = 0) begin
select top 1 #Id = id, #Cost = Cost, #RemainingQty
from #TableVariable
where Done = 0
order by OrderBy
-- Do stuff here
update #TableVariable set Done = 1 where id = #Id
end
However the code you have shown doesn't appear that it should be slow - so it may be that you are lacking the appropriate indexes, and that a single ItemId update is locking too many rows in the ItemBalance table which is then affecting other ItemId updates.
How can one call a stored procedure for each row in a table, where the columns of a row are input parameters to the sp without using a Cursor?
Generally speaking I always look for a set based approach (sometimes at the expense of changing the schema).
However, this snippet does have its place..
-- Declare & init (2008 syntax)
DECLARE #CustomerID INT = 0
-- Iterate over all customers
WHILE (1 = 1)
BEGIN
-- Get next customerId
SELECT TOP 1 #CustomerID = CustomerID
FROM Sales.Customer
WHERE CustomerID > #CustomerId
ORDER BY CustomerID
-- Exit loop if no more customers
IF ##ROWCOUNT = 0 BREAK;
-- call your sproc
EXEC dbo.YOURSPROC #CustomerId
END
You could do something like this: order your table by e.g. CustomerID (using the AdventureWorks Sales.Customer sample table), and iterate over those customers using a WHILE loop:
-- define the last customer ID handled
DECLARE #LastCustomerID INT
SET #LastCustomerID = 0
-- define the customer ID to be handled now
DECLARE #CustomerIDToHandle INT
-- select the next customer to handle
SELECT TOP 1 #CustomerIDToHandle = CustomerID
FROM Sales.Customer
WHERE CustomerID > #LastCustomerID
ORDER BY CustomerID
-- as long as we have customers......
WHILE #CustomerIDToHandle IS NOT NULL
BEGIN
-- call your sproc
-- set the last customer handled to the one we just handled
SET #LastCustomerID = #CustomerIDToHandle
SET #CustomerIDToHandle = NULL
-- select the next customer to handle
SELECT TOP 1 #CustomerIDToHandle = CustomerID
FROM Sales.Customer
WHERE CustomerID > #LastCustomerID
ORDER BY CustomerID
END
That should work with any table as long as you can define some kind of an ORDER BY on some column.
DECLARE #SQL varchar(max)=''
-- MyTable has fields fld1 & fld2
Select #SQL = #SQL + 'exec myproc ' + convert(varchar(10),fld1) + ','
+ convert(varchar(10),fld2) + ';'
From MyTable
EXEC (#SQL)
Ok, so I would never put such code into production, but it does satisfy your requirements.
I'd use the accepted answer, but another possibility is to use a table variable to hold a numbered set of values (in this case just the ID field of a table) and loop through those by Row Number with a JOIN to the table to retrieve whatever you need for the action within the loop.
DECLARE #RowCnt int; SET #RowCnt = 0 -- Loop Counter
-- Use a table variable to hold numbered rows containg MyTable's ID values
DECLARE #tblLoop TABLE (RowNum int IDENTITY (1, 1) Primary key NOT NULL,
ID INT )
INSERT INTO #tblLoop (ID) SELECT ID FROM MyTable
-- Vars to use within the loop
DECLARE #Code NVarChar(10); DECLARE #Name NVarChar(100);
WHILE #RowCnt < (SELECT COUNT(RowNum) FROM #tblLoop)
BEGIN
SET #RowCnt = #RowCnt + 1
-- Do what you want here with the data stored in tblLoop for the given RowNum
SELECT #Code=Code, #Name=LongName
FROM MyTable INNER JOIN #tblLoop tL on MyTable.ID=tL.ID
WHERE tl.RowNum=#RowCnt
PRINT Convert(NVarChar(10),#RowCnt) +' '+ #Code +' '+ #Name
END
Marc's answer is good (I'd comment on it if I could work out how to!)
Just thought I'd point out that it may be better to change the loop so the SELECT only exists once (in a real case where I needed to do this, the SELECT was quite complex, and writing it twice was a risky maintenance issue).
-- define the last customer ID handled
DECLARE #LastCustomerID INT
SET #LastCustomerID = 0
-- define the customer ID to be handled now
DECLARE #CustomerIDToHandle INT
SET #CustomerIDToHandle = 1
-- as long as we have customers......
WHILE #LastCustomerID <> #CustomerIDToHandle
BEGIN
SET #LastCustomerId = #CustomerIDToHandle
-- select the next customer to handle
SELECT TOP 1 #CustomerIDToHandle = CustomerID
FROM Sales.Customer
WHERE CustomerID > #LastCustomerId
ORDER BY CustomerID
IF #CustomerIDToHandle <> #LastCustomerID
BEGIN
-- call your sproc
END
END
If you can turn the stored procedure into a function that returns a table, then you can use cross-apply.
For example, say you have a table of customers, and you want to compute the sum of their orders, you would create a function that took a CustomerID and returned the sum.
And you could do this:
SELECT CustomerID, CustomerSum.Total
FROM Customers
CROSS APPLY ufn_ComputeCustomerTotal(Customers.CustomerID) AS CustomerSum
Where the function would look like:
CREATE FUNCTION ComputeCustomerTotal
(
#CustomerID INT
)
RETURNS TABLE
AS
RETURN
(
SELECT SUM(CustomerOrder.Amount) AS Total FROM CustomerOrder WHERE CustomerID = #CustomerID
)
Obviously, the example above could be done without a user defined function in a single query.
The drawback is that functions are very limited - many of the features of a stored procedure are not available in a user-defined function, and converting a stored procedure to a function does not always work.
For SQL Server 2005 onwards, you can do this with CROSS APPLY and a table-valued function.
Using CROSS APPLY in SQL Server 2005
Just for clarity, I'm referring to those cases where the stored procedure can be converted into a table valued function.
This is a variation on the answers already provided, but should be better performing because it doesn't require ORDER BY, COUNT or MIN/MAX. The only disadvantage with this approach is that you have to create a temp table to hold all the Ids (the assumption is that you have gaps in your list of CustomerIDs).
That said, I agree with #Mark Powell though that, generally speaking, a set based approach should still be better.
DECLARE #tmp table (Id INT IDENTITY(1,1) PRIMARY KEY NOT NULL, CustomerID INT NOT NULL)
DECLARE #CustomerId INT
DECLARE #Id INT = 0
INSERT INTO #tmp SELECT CustomerId FROM Sales.Customer
WHILE (1=1)
BEGIN
SELECT #CustomerId = CustomerId, #Id = Id
FROM #tmp
WHERE Id = #Id + 1
IF ##rowcount = 0 BREAK;
-- call your sproc
EXEC dbo.YOURSPROC #CustomerId;
END
This is a variation of n3rds solution above. No sorting by using ORDER BY is needed, as MIN() is used.
Remember that CustomerID (or whatever other numerical column you use for progress) must have a unique constraint. Furthermore, to make it as fast as possible CustomerID must be indexed on.
-- Declare & init
DECLARE #CustomerID INT = (SELECT MIN(CustomerID) FROM Sales.Customer); -- First ID
DECLARE #Data1 VARCHAR(200);
DECLARE #Data2 VARCHAR(200);
-- Iterate over all customers
WHILE #CustomerID IS NOT NULL
BEGIN
-- Get data based on ID
SELECT #Data1 = Data1, #Data2 = Data2
FROM Sales.Customer
WHERE [ID] = #CustomerID ;
-- call your sproc
EXEC dbo.YOURSPROC #Data1, #Data2
-- Get next customerId
SELECT #CustomerID = MIN(CustomerID)
FROM Sales.Customer
WHERE CustomerID > #CustomerId
END
I use this approach on some varchars I need to look over, by putting them in a temporary table first, to give them an ID.
If you don't what to use a cursor I think you'll have to do it externally (get the table, and then run for each statement and each time call the sp)
it Is the same as using a cursor, but only outside SQL.
Why won't you use a cursor ?
I usually do it this way when it's a quite a few rows:
Select all sproc parameters in a dataset with SQL Management Studio
Right-click -> Copy
Paste in to excel
Create single-row sql statements with a formula like '="EXEC schema.mysproc #param=" & A2' in a new excel column. (Where A2 is your excel column containing the parameter)
Copy the list of excel statements into a new query in SQL Management Studio and execute.
Done.
(On larger datasets i'd use one of the solutions mentioned above though).
DELIMITER //
CREATE PROCEDURE setFakeUsers (OUT output VARCHAR(100))
BEGIN
-- define the last customer ID handled
DECLARE LastGameID INT;
DECLARE CurrentGameID INT;
DECLARE userID INT;
SET #LastGameID = 0;
-- define the customer ID to be handled now
SET #userID = 0;
-- select the next game to handle
SELECT #CurrentGameID = id
FROM online_games
WHERE id > LastGameID
ORDER BY id LIMIT 0,1;
-- as long as we have customers......
WHILE (#CurrentGameID IS NOT NULL)
DO
-- call your sproc
-- set the last customer handled to the one we just handled
SET #LastGameID = #CurrentGameID;
SET #CurrentGameID = NULL;
-- select the random bot
SELECT #userID = userID
FROM users
WHERE FIND_IN_SET('bot',baseInfo)
ORDER BY RAND() LIMIT 0,1;
-- update the game
UPDATE online_games SET userID = #userID WHERE id = #CurrentGameID;
-- select the next game to handle
SELECT #CurrentGameID = id
FROM online_games
WHERE id > LastGameID
ORDER BY id LIMIT 0,1;
END WHILE;
SET output = "done";
END;//
CALL setFakeUsers(#status);
SELECT #status;
A better solution for this is to
Copy/past code of Stored Procedure
Join that code with the table for which you want to run it again (for each row)
This was you get a clean table-formatted output. While if you run SP for every row, you get a separate query result for each iteration which is ugly.
In case the order is important
--declare counter
DECLARE #CurrentRowNum BIGINT = 0;
--Iterate over all rows in [DataTable]
WHILE (1 = 1)
BEGIN
--Get next row by number of row
SELECT TOP 1 #CurrentRowNum = extendedData.RowNum
--here also you can store another values
--for following usage
--#MyVariable = extendedData.Value
FROM (
SELECT
data.*
,ROW_NUMBER() OVER(ORDER BY (SELECT 0)) RowNum
FROM [DataTable] data
) extendedData
WHERE extendedData.RowNum > #CurrentRowNum
ORDER BY extendedData.RowNum
--Exit loop if no more rows
IF ##ROWCOUNT = 0 BREAK;
--call your sproc
--EXEC dbo.YOURSPROC #MyVariable
END
I had some production code that could only handle 20 employees at a time, below is the framework for the code. I just copied the production code and removed stuff below.
ALTER procedure GetEmployees
#ClientId varchar(50)
as
begin
declare #EEList table (employeeId varchar(50));
declare #EE20 table (employeeId varchar(50));
insert into #EEList select employeeId from Employee where (ClientId = #ClientId);
-- Do 20 at a time
while (select count(*) from #EEList) > 0
BEGIN
insert into #EE20 select top 20 employeeId from #EEList;
-- Call sp here
delete #EEList where employeeId in (select employeeId from #EE20)
delete #EE20;
END;
RETURN
end
I had a situation where I needed to perform a series of operations on a result set (table). The operations are all set operations, so its not an issue, but...
I needed to do this in multiple places. So putting the relevant pieces in a table type, then populating a table variable w/ each result set allows me to call the sp and repeat the operations each time i need to .
While this does not address the exact question he asks, it does address how to perform an operation on all rows of a table without using a cursor.
#Johannes offers no insight into his motivation , so this may or may not help him.
my research led me to this well written article which served as a basis for my solution
https://codingsight.com/passing-data-table-as-parameter-to-stored-procedures/
Here is the setup
drop type if exists cpRootMapType
go
create type cpRootMapType as Table(
RootId1 int
, RootId2 int
)
go
drop procedure if exists spMapRoot2toRoot1
go
create procedure spMapRoot2toRoot1
(
#map cpRootMapType Readonly
)
as
update linkTable set root = root1
from linktable lt
join #map m on lt.root = root2
update comments set root = root1
from comments c
join #map m on c.root = root2
-- ever growing list of places this map would need to be applied....
-- now consolidated into one place
here is the implementation
... populate #matches
declare #map cpRootMapType
insert #map select rootid1, rootid2 from #matches
exec spMapRoot2toRoot1 #map
I like to do something similar to this (though it is still very similar to using a cursor)
[code]
-- Table variable to hold list of things that need looping
DECLARE #holdStuff TABLE (
id INT IDENTITY(1,1) ,
isIterated BIT DEFAULT 0 ,
someInt INT ,
someBool BIT ,
otherStuff VARCHAR(200)
)
-- Populate your #holdStuff with... stuff
INSERT INTO #holdStuff (
someInt ,
someBool ,
otherStuff
)
SELECT
1 , -- someInt - int
1 , -- someBool - bit
'I like turtles' -- otherStuff - varchar(200)
UNION ALL
SELECT
42 , -- someInt - int
0 , -- someBool - bit
'something profound' -- otherStuff - varchar(200)
-- Loop tracking variables
DECLARE #tableCount INT
SET #tableCount = (SELECT COUNT(1) FROM [#holdStuff])
DECLARE #loopCount INT
SET #loopCount = 1
-- While loop variables
DECLARE #id INT
DECLARE #someInt INT
DECLARE #someBool BIT
DECLARE #otherStuff VARCHAR(200)
-- Loop through item in #holdStuff
WHILE (#loopCount <= #tableCount)
BEGIN
-- Increment the loopCount variable
SET #loopCount = #loopCount + 1
-- Grab the top unprocessed record
SELECT TOP 1
#id = id ,
#someInt = someInt ,
#someBool = someBool ,
#otherStuff = otherStuff
FROM #holdStuff
WHERE isIterated = 0
-- Update the grabbed record to be iterated
UPDATE #holdAccounts
SET isIterated = 1
WHERE id = #id
-- Execute your stored procedure
EXEC someRandomSp #someInt, #someBool, #otherStuff
END
[/code]
Note that you don't need the identity or the isIterated column on your temp/variable table, i just prefer to do it this way so i don't have to delete the top record from the collection as i iterate through the loop.
Before I go any further: Yes, I know that cursors perform poorly compared with set-based operations. In this particular case I'm running a cursor on a temporary table of 100 or so records, and that temporary table will always be fairly small, so performance is less crucial than flexibility.
My difficulty is that I'm having trouble finding an example of how to update a column fetched by a cursor. Previously when I've used cursors I've retrieved values into variables, then run an update query at each step based upon these values. On this occasion I want to update a field in the temporary table, yet I can't figure out how to do it.
In the example below, I'm trying to update the field CurrentPOs in temporary table #t1, based upon a query that uses #t1.Product_ID to look up the required value. You will see in the code that I have attempted to use the notation curPO.Product_ID to reference this, but it doesn't work. I have also attempted to use an update statement against curPO, also unsuccessfully.
I can make the code work by fetching to variables, but I'd like to know how to update the field directly.
I think I'm probably missing something obvious, but can anyone help?
declare curPO cursor
for select Product_ID, CurrentPOs from #t1
for update of CurrentPOs
open curPO
fetch next from curPO
while ##fetch_status = 0
begin
select OrderQuantity = <calculation>,
ReceiveQuantity = <calculation>
into #POs
from PurchaseOrderLine POL
inner join SupplierAddress SA ON POL.Supplier_ID = SA.Supplier_ID
inner join PurchaseOrderHeader POH ON POH.PurchaseOrder_ID = POL.PurchaseOrder_ID
where Product_ID = curPO.Product_ID
and SA.AddressType = '1801'
update curPO set CurrentPOs = (select sum(OrderQuantity) - sum(ReceiveQuantity) from #POs)
drop table #POs
fetch next from curPO
end
close curPO
deallocate curPO
After doing a bit more googling, I found a partial solution. The update code is as follows:
UPDATE #T1
SET CURRENTPOS = (SELECT SUM(ORDERQUANTITY) - SUM(RECEIVEQUANTITY)
FROM #POS)
WHERE CURRENT OF CURPO
I still had to use FETCH INTO, however, to retrieve #t1.Product_ID and run the query that produces #POs, so I'd still like to know if it's possible to use FETCH on it's own.
Is this what you want?
declare curPO cursor
for select Product_ID, CurrentPOs from #t1
for update of CurrentPOs
open curPO
fetch next from curPO
while ##fetch_status = 0
begin
update curPO set CurrentPOs =
(select sum(<OrderQuantityCalculation>)
from PurchaseOrderLine POL
inner join SupplierAddress SA ON POL.Supplier_ID = SA.Supplier_ID
inner join PurchaseOrderHeader POH ON POH.PurchaseOrder_ID = POL.PurchaseOrder_ID
where Product_ID = curPO.Product_ID
and SA.AddressType = '1801') -
(select sum(<ReceiveQuantityCalculation>)
from PurchaseOrderLine POL
inner join SupplierAddress SA ON POL.Supplier_ID = SA.Supplier_ID
inner join PurchaseOrderHeader POH ON POH.PurchaseOrder_ID = POL.PurchaseOrder_ID
where Product_ID = curPO.Product_ID
and SA.AddressType = '1801')
fetch next from curPO
end
close curPO
deallocate curPO
Maybe you need something like that:
update DataBaseName..TableName
set ColumnName = value
where current of your_cursor_name;
Here's an example to calculate one column based upon values from two others (note, this could be done during the original table select). This example can be copy / pasted into an SSMS query window to be run without the need for any editing.
DECLARE #cust_id INT = 2, #dynamic_val NVARCHAR(40), #val_a INT, #val_b INT
DECLARE #tbl_invoice table(Cust_ID INT, Cust_Fees INT, Cust_Tax INT)
INSERT #tbl_invoice ( Cust_ID, Cust_Fees, Cust_Tax ) SELECT 1, 111, 11
INSERT #tbl_invoice ( Cust_ID, Cust_Fees, Cust_Tax ) SELECT 2, 222, 22
INSERT #tbl_invoice ( Cust_ID, Cust_Fees, Cust_Tax ) SELECT 3, 333, 33
DECLARE #TblCust TABLE
(
Rec_ID INT
, Val_A INT
, Val_B INT
, Dynamic_Val NVARCHAR(40)
, PRIMARY KEY NONCLUSTERED (Rec_ID)
)
INSERT #TblCust(Rec_ID, Val_A, Val_B, Dynamic_Val)
SELECT Rec_ID = Cust_ID, Val_A = Cust_Fees, Val_B = Cust_Tax, NULL
FROM #tbl_invoice
DECLARE cursor_cust CURSOR FOR
SELECT Rec_ID, Val_A, Val_B, Dynamic_Val
FROM #TblCust
WHERE Rec_ID <> #cust_id
FOR UPDATE OF Dynamic_Val;
OPEN cursor_cust;
FETCH NEXT FROM cursor_cust INTO #cust_id, #val_a, #val_b, #dynamic_val;
WHILE ##FETCH_STATUS = 0
BEGIN
UPDATE #TblCust
SET Dynamic_Val = N'#c = "' + LTRIM(STR((#val_a + #val_b), 40)) + N'"'
WHERE CURRENT OF cursor_cust
FETCH NEXT FROM cursor_cust INTO #cust_id, #val_a, #val_b, #dynamic_val;
END
CLOSE cursor_cust
DEALLOCATE cursor_cust
SELECT * FROM #TblCust
How can one call a stored procedure for each row in a table, where the columns of a row are input parameters to the sp without using a Cursor?
Generally speaking I always look for a set based approach (sometimes at the expense of changing the schema).
However, this snippet does have its place..
-- Declare & init (2008 syntax)
DECLARE #CustomerID INT = 0
-- Iterate over all customers
WHILE (1 = 1)
BEGIN
-- Get next customerId
SELECT TOP 1 #CustomerID = CustomerID
FROM Sales.Customer
WHERE CustomerID > #CustomerId
ORDER BY CustomerID
-- Exit loop if no more customers
IF ##ROWCOUNT = 0 BREAK;
-- call your sproc
EXEC dbo.YOURSPROC #CustomerId
END
You could do something like this: order your table by e.g. CustomerID (using the AdventureWorks Sales.Customer sample table), and iterate over those customers using a WHILE loop:
-- define the last customer ID handled
DECLARE #LastCustomerID INT
SET #LastCustomerID = 0
-- define the customer ID to be handled now
DECLARE #CustomerIDToHandle INT
-- select the next customer to handle
SELECT TOP 1 #CustomerIDToHandle = CustomerID
FROM Sales.Customer
WHERE CustomerID > #LastCustomerID
ORDER BY CustomerID
-- as long as we have customers......
WHILE #CustomerIDToHandle IS NOT NULL
BEGIN
-- call your sproc
-- set the last customer handled to the one we just handled
SET #LastCustomerID = #CustomerIDToHandle
SET #CustomerIDToHandle = NULL
-- select the next customer to handle
SELECT TOP 1 #CustomerIDToHandle = CustomerID
FROM Sales.Customer
WHERE CustomerID > #LastCustomerID
ORDER BY CustomerID
END
That should work with any table as long as you can define some kind of an ORDER BY on some column.
DECLARE #SQL varchar(max)=''
-- MyTable has fields fld1 & fld2
Select #SQL = #SQL + 'exec myproc ' + convert(varchar(10),fld1) + ','
+ convert(varchar(10),fld2) + ';'
From MyTable
EXEC (#SQL)
Ok, so I would never put such code into production, but it does satisfy your requirements.
I'd use the accepted answer, but another possibility is to use a table variable to hold a numbered set of values (in this case just the ID field of a table) and loop through those by Row Number with a JOIN to the table to retrieve whatever you need for the action within the loop.
DECLARE #RowCnt int; SET #RowCnt = 0 -- Loop Counter
-- Use a table variable to hold numbered rows containg MyTable's ID values
DECLARE #tblLoop TABLE (RowNum int IDENTITY (1, 1) Primary key NOT NULL,
ID INT )
INSERT INTO #tblLoop (ID) SELECT ID FROM MyTable
-- Vars to use within the loop
DECLARE #Code NVarChar(10); DECLARE #Name NVarChar(100);
WHILE #RowCnt < (SELECT COUNT(RowNum) FROM #tblLoop)
BEGIN
SET #RowCnt = #RowCnt + 1
-- Do what you want here with the data stored in tblLoop for the given RowNum
SELECT #Code=Code, #Name=LongName
FROM MyTable INNER JOIN #tblLoop tL on MyTable.ID=tL.ID
WHERE tl.RowNum=#RowCnt
PRINT Convert(NVarChar(10),#RowCnt) +' '+ #Code +' '+ #Name
END
Marc's answer is good (I'd comment on it if I could work out how to!)
Just thought I'd point out that it may be better to change the loop so the SELECT only exists once (in a real case where I needed to do this, the SELECT was quite complex, and writing it twice was a risky maintenance issue).
-- define the last customer ID handled
DECLARE #LastCustomerID INT
SET #LastCustomerID = 0
-- define the customer ID to be handled now
DECLARE #CustomerIDToHandle INT
SET #CustomerIDToHandle = 1
-- as long as we have customers......
WHILE #LastCustomerID <> #CustomerIDToHandle
BEGIN
SET #LastCustomerId = #CustomerIDToHandle
-- select the next customer to handle
SELECT TOP 1 #CustomerIDToHandle = CustomerID
FROM Sales.Customer
WHERE CustomerID > #LastCustomerId
ORDER BY CustomerID
IF #CustomerIDToHandle <> #LastCustomerID
BEGIN
-- call your sproc
END
END
If you can turn the stored procedure into a function that returns a table, then you can use cross-apply.
For example, say you have a table of customers, and you want to compute the sum of their orders, you would create a function that took a CustomerID and returned the sum.
And you could do this:
SELECT CustomerID, CustomerSum.Total
FROM Customers
CROSS APPLY ufn_ComputeCustomerTotal(Customers.CustomerID) AS CustomerSum
Where the function would look like:
CREATE FUNCTION ComputeCustomerTotal
(
#CustomerID INT
)
RETURNS TABLE
AS
RETURN
(
SELECT SUM(CustomerOrder.Amount) AS Total FROM CustomerOrder WHERE CustomerID = #CustomerID
)
Obviously, the example above could be done without a user defined function in a single query.
The drawback is that functions are very limited - many of the features of a stored procedure are not available in a user-defined function, and converting a stored procedure to a function does not always work.
For SQL Server 2005 onwards, you can do this with CROSS APPLY and a table-valued function.
Using CROSS APPLY in SQL Server 2005
Just for clarity, I'm referring to those cases where the stored procedure can be converted into a table valued function.
This is a variation on the answers already provided, but should be better performing because it doesn't require ORDER BY, COUNT or MIN/MAX. The only disadvantage with this approach is that you have to create a temp table to hold all the Ids (the assumption is that you have gaps in your list of CustomerIDs).
That said, I agree with #Mark Powell though that, generally speaking, a set based approach should still be better.
DECLARE #tmp table (Id INT IDENTITY(1,1) PRIMARY KEY NOT NULL, CustomerID INT NOT NULL)
DECLARE #CustomerId INT
DECLARE #Id INT = 0
INSERT INTO #tmp SELECT CustomerId FROM Sales.Customer
WHILE (1=1)
BEGIN
SELECT #CustomerId = CustomerId, #Id = Id
FROM #tmp
WHERE Id = #Id + 1
IF ##rowcount = 0 BREAK;
-- call your sproc
EXEC dbo.YOURSPROC #CustomerId;
END
This is a variation of n3rds solution above. No sorting by using ORDER BY is needed, as MIN() is used.
Remember that CustomerID (or whatever other numerical column you use for progress) must have a unique constraint. Furthermore, to make it as fast as possible CustomerID must be indexed on.
-- Declare & init
DECLARE #CustomerID INT = (SELECT MIN(CustomerID) FROM Sales.Customer); -- First ID
DECLARE #Data1 VARCHAR(200);
DECLARE #Data2 VARCHAR(200);
-- Iterate over all customers
WHILE #CustomerID IS NOT NULL
BEGIN
-- Get data based on ID
SELECT #Data1 = Data1, #Data2 = Data2
FROM Sales.Customer
WHERE [ID] = #CustomerID ;
-- call your sproc
EXEC dbo.YOURSPROC #Data1, #Data2
-- Get next customerId
SELECT #CustomerID = MIN(CustomerID)
FROM Sales.Customer
WHERE CustomerID > #CustomerId
END
I use this approach on some varchars I need to look over, by putting them in a temporary table first, to give them an ID.
If you don't what to use a cursor I think you'll have to do it externally (get the table, and then run for each statement and each time call the sp)
it Is the same as using a cursor, but only outside SQL.
Why won't you use a cursor ?
I usually do it this way when it's a quite a few rows:
Select all sproc parameters in a dataset with SQL Management Studio
Right-click -> Copy
Paste in to excel
Create single-row sql statements with a formula like '="EXEC schema.mysproc #param=" & A2' in a new excel column. (Where A2 is your excel column containing the parameter)
Copy the list of excel statements into a new query in SQL Management Studio and execute.
Done.
(On larger datasets i'd use one of the solutions mentioned above though).
DELIMITER //
CREATE PROCEDURE setFakeUsers (OUT output VARCHAR(100))
BEGIN
-- define the last customer ID handled
DECLARE LastGameID INT;
DECLARE CurrentGameID INT;
DECLARE userID INT;
SET #LastGameID = 0;
-- define the customer ID to be handled now
SET #userID = 0;
-- select the next game to handle
SELECT #CurrentGameID = id
FROM online_games
WHERE id > LastGameID
ORDER BY id LIMIT 0,1;
-- as long as we have customers......
WHILE (#CurrentGameID IS NOT NULL)
DO
-- call your sproc
-- set the last customer handled to the one we just handled
SET #LastGameID = #CurrentGameID;
SET #CurrentGameID = NULL;
-- select the random bot
SELECT #userID = userID
FROM users
WHERE FIND_IN_SET('bot',baseInfo)
ORDER BY RAND() LIMIT 0,1;
-- update the game
UPDATE online_games SET userID = #userID WHERE id = #CurrentGameID;
-- select the next game to handle
SELECT #CurrentGameID = id
FROM online_games
WHERE id > LastGameID
ORDER BY id LIMIT 0,1;
END WHILE;
SET output = "done";
END;//
CALL setFakeUsers(#status);
SELECT #status;
A better solution for this is to
Copy/past code of Stored Procedure
Join that code with the table for which you want to run it again (for each row)
This was you get a clean table-formatted output. While if you run SP for every row, you get a separate query result for each iteration which is ugly.
In case the order is important
--declare counter
DECLARE #CurrentRowNum BIGINT = 0;
--Iterate over all rows in [DataTable]
WHILE (1 = 1)
BEGIN
--Get next row by number of row
SELECT TOP 1 #CurrentRowNum = extendedData.RowNum
--here also you can store another values
--for following usage
--#MyVariable = extendedData.Value
FROM (
SELECT
data.*
,ROW_NUMBER() OVER(ORDER BY (SELECT 0)) RowNum
FROM [DataTable] data
) extendedData
WHERE extendedData.RowNum > #CurrentRowNum
ORDER BY extendedData.RowNum
--Exit loop if no more rows
IF ##ROWCOUNT = 0 BREAK;
--call your sproc
--EXEC dbo.YOURSPROC #MyVariable
END
I had some production code that could only handle 20 employees at a time, below is the framework for the code. I just copied the production code and removed stuff below.
ALTER procedure GetEmployees
#ClientId varchar(50)
as
begin
declare #EEList table (employeeId varchar(50));
declare #EE20 table (employeeId varchar(50));
insert into #EEList select employeeId from Employee where (ClientId = #ClientId);
-- Do 20 at a time
while (select count(*) from #EEList) > 0
BEGIN
insert into #EE20 select top 20 employeeId from #EEList;
-- Call sp here
delete #EEList where employeeId in (select employeeId from #EE20)
delete #EE20;
END;
RETURN
end
I had a situation where I needed to perform a series of operations on a result set (table). The operations are all set operations, so its not an issue, but...
I needed to do this in multiple places. So putting the relevant pieces in a table type, then populating a table variable w/ each result set allows me to call the sp and repeat the operations each time i need to .
While this does not address the exact question he asks, it does address how to perform an operation on all rows of a table without using a cursor.
#Johannes offers no insight into his motivation , so this may or may not help him.
my research led me to this well written article which served as a basis for my solution
https://codingsight.com/passing-data-table-as-parameter-to-stored-procedures/
Here is the setup
drop type if exists cpRootMapType
go
create type cpRootMapType as Table(
RootId1 int
, RootId2 int
)
go
drop procedure if exists spMapRoot2toRoot1
go
create procedure spMapRoot2toRoot1
(
#map cpRootMapType Readonly
)
as
update linkTable set root = root1
from linktable lt
join #map m on lt.root = root2
update comments set root = root1
from comments c
join #map m on c.root = root2
-- ever growing list of places this map would need to be applied....
-- now consolidated into one place
here is the implementation
... populate #matches
declare #map cpRootMapType
insert #map select rootid1, rootid2 from #matches
exec spMapRoot2toRoot1 #map
I like to do something similar to this (though it is still very similar to using a cursor)
[code]
-- Table variable to hold list of things that need looping
DECLARE #holdStuff TABLE (
id INT IDENTITY(1,1) ,
isIterated BIT DEFAULT 0 ,
someInt INT ,
someBool BIT ,
otherStuff VARCHAR(200)
)
-- Populate your #holdStuff with... stuff
INSERT INTO #holdStuff (
someInt ,
someBool ,
otherStuff
)
SELECT
1 , -- someInt - int
1 , -- someBool - bit
'I like turtles' -- otherStuff - varchar(200)
UNION ALL
SELECT
42 , -- someInt - int
0 , -- someBool - bit
'something profound' -- otherStuff - varchar(200)
-- Loop tracking variables
DECLARE #tableCount INT
SET #tableCount = (SELECT COUNT(1) FROM [#holdStuff])
DECLARE #loopCount INT
SET #loopCount = 1
-- While loop variables
DECLARE #id INT
DECLARE #someInt INT
DECLARE #someBool BIT
DECLARE #otherStuff VARCHAR(200)
-- Loop through item in #holdStuff
WHILE (#loopCount <= #tableCount)
BEGIN
-- Increment the loopCount variable
SET #loopCount = #loopCount + 1
-- Grab the top unprocessed record
SELECT TOP 1
#id = id ,
#someInt = someInt ,
#someBool = someBool ,
#otherStuff = otherStuff
FROM #holdStuff
WHERE isIterated = 0
-- Update the grabbed record to be iterated
UPDATE #holdAccounts
SET isIterated = 1
WHERE id = #id
-- Execute your stored procedure
EXEC someRandomSp #someInt, #someBool, #otherStuff
END
[/code]
Note that you don't need the identity or the isIterated column on your temp/variable table, i just prefer to do it this way so i don't have to delete the top record from the collection as i iterate through the loop.