How can one call a stored procedure for each row in a table, where the columns of a row are input parameters to the sp without using a Cursor?
Generally speaking I always look for a set based approach (sometimes at the expense of changing the schema).
However, this snippet does have its place..
-- Declare & init (2008 syntax)
DECLARE #CustomerID INT = 0
-- Iterate over all customers
WHILE (1 = 1)
BEGIN
-- Get next customerId
SELECT TOP 1 #CustomerID = CustomerID
FROM Sales.Customer
WHERE CustomerID > #CustomerId
ORDER BY CustomerID
-- Exit loop if no more customers
IF ##ROWCOUNT = 0 BREAK;
-- call your sproc
EXEC dbo.YOURSPROC #CustomerId
END
You could do something like this: order your table by e.g. CustomerID (using the AdventureWorks Sales.Customer sample table), and iterate over those customers using a WHILE loop:
-- define the last customer ID handled
DECLARE #LastCustomerID INT
SET #LastCustomerID = 0
-- define the customer ID to be handled now
DECLARE #CustomerIDToHandle INT
-- select the next customer to handle
SELECT TOP 1 #CustomerIDToHandle = CustomerID
FROM Sales.Customer
WHERE CustomerID > #LastCustomerID
ORDER BY CustomerID
-- as long as we have customers......
WHILE #CustomerIDToHandle IS NOT NULL
BEGIN
-- call your sproc
-- set the last customer handled to the one we just handled
SET #LastCustomerID = #CustomerIDToHandle
SET #CustomerIDToHandle = NULL
-- select the next customer to handle
SELECT TOP 1 #CustomerIDToHandle = CustomerID
FROM Sales.Customer
WHERE CustomerID > #LastCustomerID
ORDER BY CustomerID
END
That should work with any table as long as you can define some kind of an ORDER BY on some column.
DECLARE #SQL varchar(max)=''
-- MyTable has fields fld1 & fld2
Select #SQL = #SQL + 'exec myproc ' + convert(varchar(10),fld1) + ','
+ convert(varchar(10),fld2) + ';'
From MyTable
EXEC (#SQL)
Ok, so I would never put such code into production, but it does satisfy your requirements.
I'd use the accepted answer, but another possibility is to use a table variable to hold a numbered set of values (in this case just the ID field of a table) and loop through those by Row Number with a JOIN to the table to retrieve whatever you need for the action within the loop.
DECLARE #RowCnt int; SET #RowCnt = 0 -- Loop Counter
-- Use a table variable to hold numbered rows containg MyTable's ID values
DECLARE #tblLoop TABLE (RowNum int IDENTITY (1, 1) Primary key NOT NULL,
ID INT )
INSERT INTO #tblLoop (ID) SELECT ID FROM MyTable
-- Vars to use within the loop
DECLARE #Code NVarChar(10); DECLARE #Name NVarChar(100);
WHILE #RowCnt < (SELECT COUNT(RowNum) FROM #tblLoop)
BEGIN
SET #RowCnt = #RowCnt + 1
-- Do what you want here with the data stored in tblLoop for the given RowNum
SELECT #Code=Code, #Name=LongName
FROM MyTable INNER JOIN #tblLoop tL on MyTable.ID=tL.ID
WHERE tl.RowNum=#RowCnt
PRINT Convert(NVarChar(10),#RowCnt) +' '+ #Code +' '+ #Name
END
Marc's answer is good (I'd comment on it if I could work out how to!)
Just thought I'd point out that it may be better to change the loop so the SELECT only exists once (in a real case where I needed to do this, the SELECT was quite complex, and writing it twice was a risky maintenance issue).
-- define the last customer ID handled
DECLARE #LastCustomerID INT
SET #LastCustomerID = 0
-- define the customer ID to be handled now
DECLARE #CustomerIDToHandle INT
SET #CustomerIDToHandle = 1
-- as long as we have customers......
WHILE #LastCustomerID <> #CustomerIDToHandle
BEGIN
SET #LastCustomerId = #CustomerIDToHandle
-- select the next customer to handle
SELECT TOP 1 #CustomerIDToHandle = CustomerID
FROM Sales.Customer
WHERE CustomerID > #LastCustomerId
ORDER BY CustomerID
IF #CustomerIDToHandle <> #LastCustomerID
BEGIN
-- call your sproc
END
END
If you can turn the stored procedure into a function that returns a table, then you can use cross-apply.
For example, say you have a table of customers, and you want to compute the sum of their orders, you would create a function that took a CustomerID and returned the sum.
And you could do this:
SELECT CustomerID, CustomerSum.Total
FROM Customers
CROSS APPLY ufn_ComputeCustomerTotal(Customers.CustomerID) AS CustomerSum
Where the function would look like:
CREATE FUNCTION ComputeCustomerTotal
(
#CustomerID INT
)
RETURNS TABLE
AS
RETURN
(
SELECT SUM(CustomerOrder.Amount) AS Total FROM CustomerOrder WHERE CustomerID = #CustomerID
)
Obviously, the example above could be done without a user defined function in a single query.
The drawback is that functions are very limited - many of the features of a stored procedure are not available in a user-defined function, and converting a stored procedure to a function does not always work.
For SQL Server 2005 onwards, you can do this with CROSS APPLY and a table-valued function.
Using CROSS APPLY in SQL Server 2005
Just for clarity, I'm referring to those cases where the stored procedure can be converted into a table valued function.
This is a variation on the answers already provided, but should be better performing because it doesn't require ORDER BY, COUNT or MIN/MAX. The only disadvantage with this approach is that you have to create a temp table to hold all the Ids (the assumption is that you have gaps in your list of CustomerIDs).
That said, I agree with #Mark Powell though that, generally speaking, a set based approach should still be better.
DECLARE #tmp table (Id INT IDENTITY(1,1) PRIMARY KEY NOT NULL, CustomerID INT NOT NULL)
DECLARE #CustomerId INT
DECLARE #Id INT = 0
INSERT INTO #tmp SELECT CustomerId FROM Sales.Customer
WHILE (1=1)
BEGIN
SELECT #CustomerId = CustomerId, #Id = Id
FROM #tmp
WHERE Id = #Id + 1
IF ##rowcount = 0 BREAK;
-- call your sproc
EXEC dbo.YOURSPROC #CustomerId;
END
This is a variation of n3rds solution above. No sorting by using ORDER BY is needed, as MIN() is used.
Remember that CustomerID (or whatever other numerical column you use for progress) must have a unique constraint. Furthermore, to make it as fast as possible CustomerID must be indexed on.
-- Declare & init
DECLARE #CustomerID INT = (SELECT MIN(CustomerID) FROM Sales.Customer); -- First ID
DECLARE #Data1 VARCHAR(200);
DECLARE #Data2 VARCHAR(200);
-- Iterate over all customers
WHILE #CustomerID IS NOT NULL
BEGIN
-- Get data based on ID
SELECT #Data1 = Data1, #Data2 = Data2
FROM Sales.Customer
WHERE [ID] = #CustomerID ;
-- call your sproc
EXEC dbo.YOURSPROC #Data1, #Data2
-- Get next customerId
SELECT #CustomerID = MIN(CustomerID)
FROM Sales.Customer
WHERE CustomerID > #CustomerId
END
I use this approach on some varchars I need to look over, by putting them in a temporary table first, to give them an ID.
If you don't what to use a cursor I think you'll have to do it externally (get the table, and then run for each statement and each time call the sp)
it Is the same as using a cursor, but only outside SQL.
Why won't you use a cursor ?
I usually do it this way when it's a quite a few rows:
Select all sproc parameters in a dataset with SQL Management Studio
Right-click -> Copy
Paste in to excel
Create single-row sql statements with a formula like '="EXEC schema.mysproc #param=" & A2' in a new excel column. (Where A2 is your excel column containing the parameter)
Copy the list of excel statements into a new query in SQL Management Studio and execute.
Done.
(On larger datasets i'd use one of the solutions mentioned above though).
DELIMITER //
CREATE PROCEDURE setFakeUsers (OUT output VARCHAR(100))
BEGIN
-- define the last customer ID handled
DECLARE LastGameID INT;
DECLARE CurrentGameID INT;
DECLARE userID INT;
SET #LastGameID = 0;
-- define the customer ID to be handled now
SET #userID = 0;
-- select the next game to handle
SELECT #CurrentGameID = id
FROM online_games
WHERE id > LastGameID
ORDER BY id LIMIT 0,1;
-- as long as we have customers......
WHILE (#CurrentGameID IS NOT NULL)
DO
-- call your sproc
-- set the last customer handled to the one we just handled
SET #LastGameID = #CurrentGameID;
SET #CurrentGameID = NULL;
-- select the random bot
SELECT #userID = userID
FROM users
WHERE FIND_IN_SET('bot',baseInfo)
ORDER BY RAND() LIMIT 0,1;
-- update the game
UPDATE online_games SET userID = #userID WHERE id = #CurrentGameID;
-- select the next game to handle
SELECT #CurrentGameID = id
FROM online_games
WHERE id > LastGameID
ORDER BY id LIMIT 0,1;
END WHILE;
SET output = "done";
END;//
CALL setFakeUsers(#status);
SELECT #status;
A better solution for this is to
Copy/past code of Stored Procedure
Join that code with the table for which you want to run it again (for each row)
This was you get a clean table-formatted output. While if you run SP for every row, you get a separate query result for each iteration which is ugly.
In case the order is important
--declare counter
DECLARE #CurrentRowNum BIGINT = 0;
--Iterate over all rows in [DataTable]
WHILE (1 = 1)
BEGIN
--Get next row by number of row
SELECT TOP 1 #CurrentRowNum = extendedData.RowNum
--here also you can store another values
--for following usage
--#MyVariable = extendedData.Value
FROM (
SELECT
data.*
,ROW_NUMBER() OVER(ORDER BY (SELECT 0)) RowNum
FROM [DataTable] data
) extendedData
WHERE extendedData.RowNum > #CurrentRowNum
ORDER BY extendedData.RowNum
--Exit loop if no more rows
IF ##ROWCOUNT = 0 BREAK;
--call your sproc
--EXEC dbo.YOURSPROC #MyVariable
END
I had some production code that could only handle 20 employees at a time, below is the framework for the code. I just copied the production code and removed stuff below.
ALTER procedure GetEmployees
#ClientId varchar(50)
as
begin
declare #EEList table (employeeId varchar(50));
declare #EE20 table (employeeId varchar(50));
insert into #EEList select employeeId from Employee where (ClientId = #ClientId);
-- Do 20 at a time
while (select count(*) from #EEList) > 0
BEGIN
insert into #EE20 select top 20 employeeId from #EEList;
-- Call sp here
delete #EEList where employeeId in (select employeeId from #EE20)
delete #EE20;
END;
RETURN
end
I had a situation where I needed to perform a series of operations on a result set (table). The operations are all set operations, so its not an issue, but...
I needed to do this in multiple places. So putting the relevant pieces in a table type, then populating a table variable w/ each result set allows me to call the sp and repeat the operations each time i need to .
While this does not address the exact question he asks, it does address how to perform an operation on all rows of a table without using a cursor.
#Johannes offers no insight into his motivation , so this may or may not help him.
my research led me to this well written article which served as a basis for my solution
https://codingsight.com/passing-data-table-as-parameter-to-stored-procedures/
Here is the setup
drop type if exists cpRootMapType
go
create type cpRootMapType as Table(
RootId1 int
, RootId2 int
)
go
drop procedure if exists spMapRoot2toRoot1
go
create procedure spMapRoot2toRoot1
(
#map cpRootMapType Readonly
)
as
update linkTable set root = root1
from linktable lt
join #map m on lt.root = root2
update comments set root = root1
from comments c
join #map m on c.root = root2
-- ever growing list of places this map would need to be applied....
-- now consolidated into one place
here is the implementation
... populate #matches
declare #map cpRootMapType
insert #map select rootid1, rootid2 from #matches
exec spMapRoot2toRoot1 #map
I like to do something similar to this (though it is still very similar to using a cursor)
[code]
-- Table variable to hold list of things that need looping
DECLARE #holdStuff TABLE (
id INT IDENTITY(1,1) ,
isIterated BIT DEFAULT 0 ,
someInt INT ,
someBool BIT ,
otherStuff VARCHAR(200)
)
-- Populate your #holdStuff with... stuff
INSERT INTO #holdStuff (
someInt ,
someBool ,
otherStuff
)
SELECT
1 , -- someInt - int
1 , -- someBool - bit
'I like turtles' -- otherStuff - varchar(200)
UNION ALL
SELECT
42 , -- someInt - int
0 , -- someBool - bit
'something profound' -- otherStuff - varchar(200)
-- Loop tracking variables
DECLARE #tableCount INT
SET #tableCount = (SELECT COUNT(1) FROM [#holdStuff])
DECLARE #loopCount INT
SET #loopCount = 1
-- While loop variables
DECLARE #id INT
DECLARE #someInt INT
DECLARE #someBool BIT
DECLARE #otherStuff VARCHAR(200)
-- Loop through item in #holdStuff
WHILE (#loopCount <= #tableCount)
BEGIN
-- Increment the loopCount variable
SET #loopCount = #loopCount + 1
-- Grab the top unprocessed record
SELECT TOP 1
#id = id ,
#someInt = someInt ,
#someBool = someBool ,
#otherStuff = otherStuff
FROM #holdStuff
WHERE isIterated = 0
-- Update the grabbed record to be iterated
UPDATE #holdAccounts
SET isIterated = 1
WHERE id = #id
-- Execute your stored procedure
EXEC someRandomSp #someInt, #someBool, #otherStuff
END
[/code]
Note that you don't need the identity or the isIterated column on your temp/variable table, i just prefer to do it this way so i don't have to delete the top record from the collection as i iterate through the loop.
I am trying to insert data to a table based on another table in SQL Management Studio 2014. I have a list of User having count 9500. For each user I want to insert multiple data (Detailed data) in another table. Here is my Case:
DECLARE #MAXID INT, #Counter INT
DECLARE #TEMP1 TABLE (
ROWID int identity(1,1) primary key,
userName nvarchar(50),
userEmail nvarchar(256)
)
insert into #TEMP1
select UserName, UserEmail from utblUsers
SET #COUNTER = 1
SELECT #MAXID = COUNT(*) FROM #TEMP1
declare #userEmail nvarChar(256),#UserName nvarchar(50)
WHILE (#COUNTER <= #MAXID/2)
BEGIN
SELECT #userEmail=UserEmail, #UserName=UserName
FROM #TEMP1 AS PT
WHERE ROWID = #COUNTER
exec sifms.dbo.sp_UserDetailInsert #UserName, #userEmail
SET #COUNTER = #COUNTER + 1
END;
Here inside the loop I am calling sp_UserDetailInsert which is inserting data to my detail table which looks like this:
DECLARE #cnt INT = 1,#sumAmount bigint, #deductedFromId int;
WHILE #cnt <= 5
BEGIN
if #cnt=1
begin
-------Doing some calculation for SumAmount and get the deductedFromID
Insert into utblUserDetails values (#UserName,#userEmail, #sumAmount, #deductedFromId)
end
else if #cnt=2
begin
-------Doing some calculation for SumAmount and get the deductedFromID
Insert into utblUserDetails values (#UserName,#userEmail, #sumAmount, #deductedFromId)
end
.
.
.
.
else #cnt=5
begin
-------Doing some calculation for SumAmount and get the deductedFromID
Insert into utblUserDetails values (#UserName,#userEmail, #sumAmount, #deductedFromId)
end
END
My query is executing absolutely fine for few minutes. But the problems are
Query is executing very slowly
SQL Management Studio is getting closed after few minutes of execution.
I have tried clearing cache using
DBCC FREESYSTEMCACHE ('ALL')
DBCC FREESESSIONCACHE
DBCC FREEPROCCACHE
But the result is still same. I have tried using cursor as well but cursor is much havier and it consume more memory than the loop. As the loop is light weight than cursor, I prefer loop in this case.I just want to get the user details at first and then I want to minimize the complexity.
In a SQL Server proc we may pass in a string list of values. Sometimes this can be an empty string. We break this string (which is a csv string) and store each value in a temp table. Again, it can be blank and so in that case the temp table is empty.
What we're trying to do in the where clause is have it run against the temp table but if there is no data run it against everything. There is a 'trick' that I've never used before and not sure if I fully understand that others at work have used but it's not working for this query (it kills performance and the query never comes back when it should in 4 seconds otherwise).
Below is an example query and the where part is the key:
DECLARE #myList AS NVARCHAR(MAX)
SET #myList = '73'
CREATE TABLE #TempList (data int)
INSERT INTO #TempList
SELECT * FROM BreakCSV(#myList,',')
DECLARE #Count int
SELECT #Count = COUNT(data) FROM #TempList sl WHERE ISNULL(sl.data,0) <> 0
SELECT *
FROM MyTable
WHERE MyDateField BETWEEN '1/1/2015' AND '3/1/2015'
AND (MyIdField IN (SELECT data FROM #TempList WHERE data <> 0) OR #Count = 0)
That OR part with the #Count is the trick. If TempList has no records then #Count would equal 0 and 0 = 0 should cause it to pull the entire table. However, even if TempList has a record in it, the mere fact of having the OR #Count = 0 makes the query not come back. If I comment out the #Count = 0 then it comes back in 4 seconds as expected.
I'm curious if someone could explain the logic in this thought process and if there is a different way to do something like this without an IF statement and duplicating this query with just different where clauses when you want some specific values or you want them all WITHOUT having to specify them all. Also, no dynamic sql.
Using your example code, this would seem to be a better way to optimize. Remember - sometimes, more code is better.
DECLARE #myList AS NVARCHAR(MAX)
DECLARE #Count int
SET #myList = '73'
CREATE TABLE #TempList (data int)
INSERT INTO #TempList
SELECT DISTINCT data
FROM BreakCSV(#myList,',')
WHERE data != 0
SET #Count = ##ROWCOUNT
CREATE UNIQUE INDEX ix1 ON #tempList(data)
IF #count = 0
SELECT *
FROM MyTable
WHERE MyDateField BETWEEN '1/1/2015' AND '3/1/2015'
ELSE
SELECT *
FROM MyTable
INNER JOIN #TempList
ON MyIdField = data
WHERE MyDateField BETWEEN '1/1/2015' AND '3/1/2015'
You overengineered this. Get rid of this:
DECLARE #Count int
SELECT #Count = COUNT(data) FROM #TempList sl WHERE ISNULL(sl.data,0) <> 0
Change this:
AND (MyIdField IN (SELECT data FROM #TempList WHERE data <> 0) OR #Count = 0)
to this:
AND (MyIdField IN (SELECT data FROM #TempList WHERE data <> 0) OR #myList is null)
This is a common approach when writing stored procedures with optional parameters.
I have this query..
Begin
declare #Col int,
#lev int,
#Plan int
select #Col = 411
select #lev = 16
select #Plan = 780
--Insert into baCodeLibrary(Plan_Num,Level_Num,Column_Num,Block_Num,Code_Id,Sort_Order,isactive,Added_By,DateTime_Added,Updated_By,DateTime_Updated)
Select Distinct
#plan,
#lev,
#col,
ba_Object_Id - 5539,
ba_Object_Id,
ba_OBject_Desc,
ba_Object_Id - 5539,
1,
'xyz',
GETDATE(),
'xyz',
GETDATE()
from baObject
where ba_OBject_Id > 5539
and ba_Object_Id < 5554
end
Here I have only for the #col = 411, but I want to loop for all the column until 489
Could any body help me out how to write the loop in this query to select all the columns from 411 to 489?
Thanks in advance
How about not thinking about this in terms of a loop, and instead think about it in terms of a set?
declare #lev int,
#Plan int;
select #lev = 16,
#Plan = 780;
;WITH n(n) AS
(
SELECT TOP (489-411) Number
FROM master.dbo.spt_values
WHERE type = N'P' ORDER BY Number
)
--Insert dbo.baCodeLibrary(...)
SELECT DISTINCT
#plan,
#lev,
n.n,
ba.ba_Object_Id - 5539,
ba.ba_Object_Id,
ba.ba_Object_Desc,
ba.ba_Object_Id - 5539,
1,
'xyz',
GETDATE(),
'xyz',
GETDATE()
FROM dbo.baObject AS ba CROSS JOIN n
where ba.ba_Object_Id > 5539
and ba.ba_Object_Id < 5554;
There's no actual looping mechanism in your query as it exists now. You'll need to implement something like a while loop in order to get that functionality.
See: http://technet.microsoft.com/en-us/library/ms178642.aspx
It'll look something like
DECLARE #counter int
SET #counter = 1
WHILE #counter < 10
BEGIN
-- loop logic
SET #counter = #counter + 1
END
Looping can be performed three ways in T-SQL; a WHILE loop, a SET based operation, or a CURSOR.
T-SQL / SQL Server are optimised for SET based operations, and they are easily the most efficient way to loop, but it all depends on what you're trying to achieve.
You may be warned away from cursors, with good reason, but they are perfectly acceptable as long as you understand what you're doing. Here's an example of a very simple, fast cursor:
DECLARE #myColumn VARCHAR(100)
DECLARE cTmp CURSOR FAST_FORWARD FOR
SELECT MyColumn
FROM MyTable (nolock)
OPEN cTmp
FETCH NEXT FROM cTmp INTO #myColumn
WHILE ##FETCH_STATUS = 0
BEGIN
-- Do something with #myColumn here
FETCH NEXT FROM cTmp INTO #myColumn
END
CLOSE cTmp
DEALLOCATE cTmp
GO
Please do not use this code without reading up on cursors first - they should only be used when SET based operations are not suitable. It depends entirely on what you're trying to achieve - cursors can be very resource hungry, and can result in locked records / tables if you're not careful with the hints you apply to them.
Want to change this piece of SQL query into a loop version which
Could Select-Insert records by 1000 items per iteration.
On each iteration want to get a print to see what was the last inserted item.
The Code :
INSERT INTO [tData2]
(
[Key],
Info1,
Info2
)
SELECT
[Key], --the Id
Info1,
Info2
FROM
[tData]
Edit :
(There were other conditions to limit the insertion, which aren't related to the question)
The Bulk-Insertion logic will not be changed, just we shrink the database pieces to be inserted.
Instead of inserting the whole table we Bulk-Insert 10000 items per iteration and could get a simple report on it also.
your help is really appreciated.
Here's some sample code I quickly put together for you to experiment with. This doesn't use a cursor (uses while instead) but you could build it that way as well. There are some performance optimizations that you could make but only you can judge that. YMMV
set nocount on
/* mock up source/dest tables */
declare #src table (id int, value varchar(max));
declare #dest table (id int, value varchar(max));
/* fill source with some data */
declare #rownum int;
set #rownum=0;
while (#rownum<5000) begin
insert into #src select #rownum,'test value ' + CONVERT(varchar(25),#rownum);
select #rownum=#rownum+1;
end
/* laod batched data */
declare #pagesize int;set #pagesize=1000;
declare #rowlow int;set #rowlow=0;
declare #rowmax int;set #rowmax=#pagesize;
declare #ct int;select #ct = COUNT(*) from #src;
declare #id int;
declare #value varchar(max);
while (#rowmax<=#ct) begin
WITH result_set AS (
SELECT ROW_NUMBER() OVER (ORDER BY id) AS [row_number], id,value
FROM #src
) insert into #dest
SELECT id,value FROM result_set
WHERE [row_number] BETWEEN #rowlow AND #rowmax
-- Output
print 'Copied rows ' + convert(varchar(25),#rowlow) + ' to ' + convert(varchar(25),#rowmax)
-- Increment batch counters
select #rowlow=#rowmax+1,#rowmax+=#pagesize;
end
select * from #dest