Related
This query (I've simulated it to show you. Otherwise, it is longer with many more lists of columns, some more tables joined; and finally to sort by every column as required by the user) takes too long, and can timeout from the UI sometimes. I was trying to optimize the stored procedure to reduce loading time without affecting the retrieved lists. I was considering to select * from the temporary table and then Order from there. That could help somehow. I also considered using a dynamic sql without specifying the #sortidr (DESC/ASC), but the user should have the flexibility to order either way.
I was wondering if anyone can suggest better ways of optimizing it.
CREATE PROCEDURE get_lists
#employeeid int
#ColumnName varchar(100) = NULL,
#sortidr varchar(4) = NULL,
#start_date datetime = NULL
AS
CREATE #tempemp
(emp_id int, first_name varchar(20), last_name varchar(20), SSN int)
INSERT INTO #tempemp
SELECT team_action, rec_id, total_records, cfirst_name, efirst_name, has_desc, category_name
FROM
(SELECT COALESCE(lt.description, lt.action) AS team_action
,cl.rec_id
,total_records = count(*) Over()
,c.cfirst_name
,e.efirst_name
, has_desc = cld.changelog_id IS NOT NULL THEN 1 ELSE 0 END
, ct.category_name
FROM claimll cll
JOIN itemtype lt ON cll.clLog_id = lt.clLog_id
LEFT JOIN categgory ct ON ct.id=clc.cat_id
LEFT JOIN clients c ON ct.client_id = c.client_id
LEFT JOIN employees e ON cll.emp_id = e.emp_id
LEFT JOIN detail d ON d.change_id = cll.change_id
WHERE cll.change_date > #start_date
AND cll.emp_id = #employeeid OR cll.by_employeeid = #employeeid
) t
ORDER BY
CASE WHEN #sortidr = 'asc' AND #ColumnName='team_action' THEN team_action END,
CASE WHEN #sortidr = 'desc' AND #ColumnName = 'team_action' THEN team_action END DESC,
CASE WHEN #sortidr = 'asc' AND #ColumnName='rec_id' THEN rec_id END,
CASE WHEN #sortidr = 'desc' AND #ColumnName = 'rec_id' THEN rec_id END DESC,
CASE WHEN #sortidr = 'asc' AND #ColumnName='total_records' THEN total_records END,
CASE WHEN #sortidr = 'desc' AND #ColumnName = 'total_records' THEN total_records END DESC,
CASE WHEN #sortidr = 'asc' AND #ColumnName='cfirst_name' THEN cfirst_name END,
CASE WHEN #sortidr = 'desc' AND #ColumnName = 'cfirst_name' THEN cfirst_name END DESC,
CASE WHEN #ColumnName IS NULL THEN first_name END DESC
I'm not at a PC so I won't be able to add code until later but you have provided enough information for me to provide some good insight.
First - the optimizer will never be able to leverage an index to prevent a sort when you order by a CASE statement. That said, there are many reasons your query could be slow; you should remove that logic then test performance to understand if that's the culprit.
Based on the limited info I have my suggestion would be update your proc to:
1. Use dynamic SQL to build your temp table, as you have done, but without the CASE statement.
Included in this ^^^ dynamic SQL should be code to build a clustered index with #columnName as the cluster key and using #sortDir to dictate the index sort order.
Now you have a temp table correctly indexed based on user supplied parameters.
Add a second dynamic SQL statement that does a SELECT * FROM #temp... You will again use #columnName and #sortOrder to build a SARGable ORDER BY (one which will leverage the index from Step #1.
Test with different parameters always looking at the Actual Execution plan to further tune your indexes.
Hope this helps. I'll add demo code a little later if you need.
Updated On 2/22/2019 to include examples:
Here's a stored proc that does what I'm describing (note the comments)--
SET NOCOUNT ON;
USE tempdb
GO
IF OBJECT_ID('dbo.DynamicSortProc','P') IS NOT NULL DROP PROC dbo.DynamicSortProc;
GO
CREATE PROC dbo.DynamicSortProc #sortby NVARCHAR(100), #sortorder NVARCHAR(4)
AS
BEGIN
-- 1. Populate the temp table (sample data)
IF OBJECT_ID('##tmp','U') IS NOT NULL DROP TABLE ##tmp;
SELECT SomeId = IDENTITY(INT,1,1), ColA, ColB
INTO ##tmp
FROM (VALUES(1,3),(6,5),(2,8),(5,5),(2,6),(5,1)) AS f(ColA,ColB);
-- NOTE: On SQL 2014+ SELECT INTO is often faster than the standard create+insert
;
-- 2. Setup #sortby
SET #sortby = QUOTENAME(#sortby)+' '+CASE #sortorder WHEN 'DESC' THEN 'DESC' ELSE '' END;
-- 3. Set up the Dynamic SQL
DECLARE #createIndex NVARCHAR(4000) = 'CREATE CLUSTERED INDEX x ON ##tmp ('+#sortby+') ';
DECLARE #executeQuery NVARCHAR(4000) = 'SELECT t.* FROM ##tmp AS t ORDER BY '+#sortby;
-- 4. Execute the queries
EXEC sys.sp_executesql #createIndex;
EXEC sys.sp_executesql #executeQuery;
END
GO
You can test like this:
EXEC dbo.DynamicSortProc 'ColA', 'ASC';
EXEC dbo.DynamicSortProc 'ColA', 'DESC';
EXEC dbo.DynamicSortProc 'ColB', 'ASC';
EXEC dbo.DynamicSortProc 'ColB', 'DESC';
In each case they sort as expected and there is no sort in the output query.
How can one call a stored procedure for each row in a table, where the columns of a row are input parameters to the sp without using a Cursor?
Generally speaking I always look for a set based approach (sometimes at the expense of changing the schema).
However, this snippet does have its place..
-- Declare & init (2008 syntax)
DECLARE #CustomerID INT = 0
-- Iterate over all customers
WHILE (1 = 1)
BEGIN
-- Get next customerId
SELECT TOP 1 #CustomerID = CustomerID
FROM Sales.Customer
WHERE CustomerID > #CustomerId
ORDER BY CustomerID
-- Exit loop if no more customers
IF ##ROWCOUNT = 0 BREAK;
-- call your sproc
EXEC dbo.YOURSPROC #CustomerId
END
You could do something like this: order your table by e.g. CustomerID (using the AdventureWorks Sales.Customer sample table), and iterate over those customers using a WHILE loop:
-- define the last customer ID handled
DECLARE #LastCustomerID INT
SET #LastCustomerID = 0
-- define the customer ID to be handled now
DECLARE #CustomerIDToHandle INT
-- select the next customer to handle
SELECT TOP 1 #CustomerIDToHandle = CustomerID
FROM Sales.Customer
WHERE CustomerID > #LastCustomerID
ORDER BY CustomerID
-- as long as we have customers......
WHILE #CustomerIDToHandle IS NOT NULL
BEGIN
-- call your sproc
-- set the last customer handled to the one we just handled
SET #LastCustomerID = #CustomerIDToHandle
SET #CustomerIDToHandle = NULL
-- select the next customer to handle
SELECT TOP 1 #CustomerIDToHandle = CustomerID
FROM Sales.Customer
WHERE CustomerID > #LastCustomerID
ORDER BY CustomerID
END
That should work with any table as long as you can define some kind of an ORDER BY on some column.
DECLARE #SQL varchar(max)=''
-- MyTable has fields fld1 & fld2
Select #SQL = #SQL + 'exec myproc ' + convert(varchar(10),fld1) + ','
+ convert(varchar(10),fld2) + ';'
From MyTable
EXEC (#SQL)
Ok, so I would never put such code into production, but it does satisfy your requirements.
I'd use the accepted answer, but another possibility is to use a table variable to hold a numbered set of values (in this case just the ID field of a table) and loop through those by Row Number with a JOIN to the table to retrieve whatever you need for the action within the loop.
DECLARE #RowCnt int; SET #RowCnt = 0 -- Loop Counter
-- Use a table variable to hold numbered rows containg MyTable's ID values
DECLARE #tblLoop TABLE (RowNum int IDENTITY (1, 1) Primary key NOT NULL,
ID INT )
INSERT INTO #tblLoop (ID) SELECT ID FROM MyTable
-- Vars to use within the loop
DECLARE #Code NVarChar(10); DECLARE #Name NVarChar(100);
WHILE #RowCnt < (SELECT COUNT(RowNum) FROM #tblLoop)
BEGIN
SET #RowCnt = #RowCnt + 1
-- Do what you want here with the data stored in tblLoop for the given RowNum
SELECT #Code=Code, #Name=LongName
FROM MyTable INNER JOIN #tblLoop tL on MyTable.ID=tL.ID
WHERE tl.RowNum=#RowCnt
PRINT Convert(NVarChar(10),#RowCnt) +' '+ #Code +' '+ #Name
END
Marc's answer is good (I'd comment on it if I could work out how to!)
Just thought I'd point out that it may be better to change the loop so the SELECT only exists once (in a real case where I needed to do this, the SELECT was quite complex, and writing it twice was a risky maintenance issue).
-- define the last customer ID handled
DECLARE #LastCustomerID INT
SET #LastCustomerID = 0
-- define the customer ID to be handled now
DECLARE #CustomerIDToHandle INT
SET #CustomerIDToHandle = 1
-- as long as we have customers......
WHILE #LastCustomerID <> #CustomerIDToHandle
BEGIN
SET #LastCustomerId = #CustomerIDToHandle
-- select the next customer to handle
SELECT TOP 1 #CustomerIDToHandle = CustomerID
FROM Sales.Customer
WHERE CustomerID > #LastCustomerId
ORDER BY CustomerID
IF #CustomerIDToHandle <> #LastCustomerID
BEGIN
-- call your sproc
END
END
If you can turn the stored procedure into a function that returns a table, then you can use cross-apply.
For example, say you have a table of customers, and you want to compute the sum of their orders, you would create a function that took a CustomerID and returned the sum.
And you could do this:
SELECT CustomerID, CustomerSum.Total
FROM Customers
CROSS APPLY ufn_ComputeCustomerTotal(Customers.CustomerID) AS CustomerSum
Where the function would look like:
CREATE FUNCTION ComputeCustomerTotal
(
#CustomerID INT
)
RETURNS TABLE
AS
RETURN
(
SELECT SUM(CustomerOrder.Amount) AS Total FROM CustomerOrder WHERE CustomerID = #CustomerID
)
Obviously, the example above could be done without a user defined function in a single query.
The drawback is that functions are very limited - many of the features of a stored procedure are not available in a user-defined function, and converting a stored procedure to a function does not always work.
For SQL Server 2005 onwards, you can do this with CROSS APPLY and a table-valued function.
Using CROSS APPLY in SQL Server 2005
Just for clarity, I'm referring to those cases where the stored procedure can be converted into a table valued function.
This is a variation on the answers already provided, but should be better performing because it doesn't require ORDER BY, COUNT or MIN/MAX. The only disadvantage with this approach is that you have to create a temp table to hold all the Ids (the assumption is that you have gaps in your list of CustomerIDs).
That said, I agree with #Mark Powell though that, generally speaking, a set based approach should still be better.
DECLARE #tmp table (Id INT IDENTITY(1,1) PRIMARY KEY NOT NULL, CustomerID INT NOT NULL)
DECLARE #CustomerId INT
DECLARE #Id INT = 0
INSERT INTO #tmp SELECT CustomerId FROM Sales.Customer
WHILE (1=1)
BEGIN
SELECT #CustomerId = CustomerId, #Id = Id
FROM #tmp
WHERE Id = #Id + 1
IF ##rowcount = 0 BREAK;
-- call your sproc
EXEC dbo.YOURSPROC #CustomerId;
END
This is a variation of n3rds solution above. No sorting by using ORDER BY is needed, as MIN() is used.
Remember that CustomerID (or whatever other numerical column you use for progress) must have a unique constraint. Furthermore, to make it as fast as possible CustomerID must be indexed on.
-- Declare & init
DECLARE #CustomerID INT = (SELECT MIN(CustomerID) FROM Sales.Customer); -- First ID
DECLARE #Data1 VARCHAR(200);
DECLARE #Data2 VARCHAR(200);
-- Iterate over all customers
WHILE #CustomerID IS NOT NULL
BEGIN
-- Get data based on ID
SELECT #Data1 = Data1, #Data2 = Data2
FROM Sales.Customer
WHERE [ID] = #CustomerID ;
-- call your sproc
EXEC dbo.YOURSPROC #Data1, #Data2
-- Get next customerId
SELECT #CustomerID = MIN(CustomerID)
FROM Sales.Customer
WHERE CustomerID > #CustomerId
END
I use this approach on some varchars I need to look over, by putting them in a temporary table first, to give them an ID.
If you don't what to use a cursor I think you'll have to do it externally (get the table, and then run for each statement and each time call the sp)
it Is the same as using a cursor, but only outside SQL.
Why won't you use a cursor ?
I usually do it this way when it's a quite a few rows:
Select all sproc parameters in a dataset with SQL Management Studio
Right-click -> Copy
Paste in to excel
Create single-row sql statements with a formula like '="EXEC schema.mysproc #param=" & A2' in a new excel column. (Where A2 is your excel column containing the parameter)
Copy the list of excel statements into a new query in SQL Management Studio and execute.
Done.
(On larger datasets i'd use one of the solutions mentioned above though).
DELIMITER //
CREATE PROCEDURE setFakeUsers (OUT output VARCHAR(100))
BEGIN
-- define the last customer ID handled
DECLARE LastGameID INT;
DECLARE CurrentGameID INT;
DECLARE userID INT;
SET #LastGameID = 0;
-- define the customer ID to be handled now
SET #userID = 0;
-- select the next game to handle
SELECT #CurrentGameID = id
FROM online_games
WHERE id > LastGameID
ORDER BY id LIMIT 0,1;
-- as long as we have customers......
WHILE (#CurrentGameID IS NOT NULL)
DO
-- call your sproc
-- set the last customer handled to the one we just handled
SET #LastGameID = #CurrentGameID;
SET #CurrentGameID = NULL;
-- select the random bot
SELECT #userID = userID
FROM users
WHERE FIND_IN_SET('bot',baseInfo)
ORDER BY RAND() LIMIT 0,1;
-- update the game
UPDATE online_games SET userID = #userID WHERE id = #CurrentGameID;
-- select the next game to handle
SELECT #CurrentGameID = id
FROM online_games
WHERE id > LastGameID
ORDER BY id LIMIT 0,1;
END WHILE;
SET output = "done";
END;//
CALL setFakeUsers(#status);
SELECT #status;
A better solution for this is to
Copy/past code of Stored Procedure
Join that code with the table for which you want to run it again (for each row)
This was you get a clean table-formatted output. While if you run SP for every row, you get a separate query result for each iteration which is ugly.
In case the order is important
--declare counter
DECLARE #CurrentRowNum BIGINT = 0;
--Iterate over all rows in [DataTable]
WHILE (1 = 1)
BEGIN
--Get next row by number of row
SELECT TOP 1 #CurrentRowNum = extendedData.RowNum
--here also you can store another values
--for following usage
--#MyVariable = extendedData.Value
FROM (
SELECT
data.*
,ROW_NUMBER() OVER(ORDER BY (SELECT 0)) RowNum
FROM [DataTable] data
) extendedData
WHERE extendedData.RowNum > #CurrentRowNum
ORDER BY extendedData.RowNum
--Exit loop if no more rows
IF ##ROWCOUNT = 0 BREAK;
--call your sproc
--EXEC dbo.YOURSPROC #MyVariable
END
I had some production code that could only handle 20 employees at a time, below is the framework for the code. I just copied the production code and removed stuff below.
ALTER procedure GetEmployees
#ClientId varchar(50)
as
begin
declare #EEList table (employeeId varchar(50));
declare #EE20 table (employeeId varchar(50));
insert into #EEList select employeeId from Employee where (ClientId = #ClientId);
-- Do 20 at a time
while (select count(*) from #EEList) > 0
BEGIN
insert into #EE20 select top 20 employeeId from #EEList;
-- Call sp here
delete #EEList where employeeId in (select employeeId from #EE20)
delete #EE20;
END;
RETURN
end
I had a situation where I needed to perform a series of operations on a result set (table). The operations are all set operations, so its not an issue, but...
I needed to do this in multiple places. So putting the relevant pieces in a table type, then populating a table variable w/ each result set allows me to call the sp and repeat the operations each time i need to .
While this does not address the exact question he asks, it does address how to perform an operation on all rows of a table without using a cursor.
#Johannes offers no insight into his motivation , so this may or may not help him.
my research led me to this well written article which served as a basis for my solution
https://codingsight.com/passing-data-table-as-parameter-to-stored-procedures/
Here is the setup
drop type if exists cpRootMapType
go
create type cpRootMapType as Table(
RootId1 int
, RootId2 int
)
go
drop procedure if exists spMapRoot2toRoot1
go
create procedure spMapRoot2toRoot1
(
#map cpRootMapType Readonly
)
as
update linkTable set root = root1
from linktable lt
join #map m on lt.root = root2
update comments set root = root1
from comments c
join #map m on c.root = root2
-- ever growing list of places this map would need to be applied....
-- now consolidated into one place
here is the implementation
... populate #matches
declare #map cpRootMapType
insert #map select rootid1, rootid2 from #matches
exec spMapRoot2toRoot1 #map
I like to do something similar to this (though it is still very similar to using a cursor)
[code]
-- Table variable to hold list of things that need looping
DECLARE #holdStuff TABLE (
id INT IDENTITY(1,1) ,
isIterated BIT DEFAULT 0 ,
someInt INT ,
someBool BIT ,
otherStuff VARCHAR(200)
)
-- Populate your #holdStuff with... stuff
INSERT INTO #holdStuff (
someInt ,
someBool ,
otherStuff
)
SELECT
1 , -- someInt - int
1 , -- someBool - bit
'I like turtles' -- otherStuff - varchar(200)
UNION ALL
SELECT
42 , -- someInt - int
0 , -- someBool - bit
'something profound' -- otherStuff - varchar(200)
-- Loop tracking variables
DECLARE #tableCount INT
SET #tableCount = (SELECT COUNT(1) FROM [#holdStuff])
DECLARE #loopCount INT
SET #loopCount = 1
-- While loop variables
DECLARE #id INT
DECLARE #someInt INT
DECLARE #someBool BIT
DECLARE #otherStuff VARCHAR(200)
-- Loop through item in #holdStuff
WHILE (#loopCount <= #tableCount)
BEGIN
-- Increment the loopCount variable
SET #loopCount = #loopCount + 1
-- Grab the top unprocessed record
SELECT TOP 1
#id = id ,
#someInt = someInt ,
#someBool = someBool ,
#otherStuff = otherStuff
FROM #holdStuff
WHERE isIterated = 0
-- Update the grabbed record to be iterated
UPDATE #holdAccounts
SET isIterated = 1
WHERE id = #id
-- Execute your stored procedure
EXEC someRandomSp #someInt, #someBool, #otherStuff
END
[/code]
Note that you don't need the identity or the isIterated column on your temp/variable table, i just prefer to do it this way so i don't have to delete the top record from the collection as i iterate through the loop.
I am stuck at a point.
I want to select based on the column entitytype if entitytype value is Booking or JOb then it will filter on its basis but if it is null or empty string('') then i want it to return all the rows containing jobs and bookings
create proc spproc
#entityType varchar(50)
as
begin
SELECT TOP 1000 [Id]
,[EntityId]
,[EntityType]
,[TenantId]
FROM [FutureTrakProd].[dbo].[Activities]
where TenantId=1 and EntityType= case #EntityType when 'BOOKING' then 'BOOKING'
when 'JOB' then 'JOB'
END
end
Any help would be appreciable
Thankyou
create proc spproc
#entityType varchar(50)
as
begin
SELECT TOP 1000 [Id]
,[EntityId]
,[EntityType]
,[TenantId]
FROM [FutureTrakProd].[dbo].[Activities]
where TenantId=1 and (#EntityType is null OR EntityType= #EntityType)
end
You don't need to use case expression you can do :
SELECT TOP 1000 [Id], [EntityId], [EntityType], [TenantId]
from [FutureTrakProd].[dbo].[Activities]
WHERE TenantId = 1 AND
(#EntityType IS NULL OR EntityType = #EntityType)
ORDER BY id; -- whatever order you want (asc/desc)
For your query procedure you need to state explicit ORDER BY clause otherwise TOP 1000 will give random Ids.
You don't need a CASE expression for this, you just need an OR. The following should put you on the right path:
WHERE TenantId=1
AND (EntityType = #EntityType OR #EntityType IS NULL)
Also, note it would also be wise to declare your parameter as NULLable:
CREATE PROC spproc #entityType varchar(50) = NULL
This means that someone can simply exclude the paramter, value than having to pass NULL (thus EXEc spproc; would work).
Finally, if you're going to have lots of NULLable parameters, then you're looking at a "catch-all" query; the solution would be different if that is the case. "Catch-all" queries can be notoriously slow.
You can execute a dynamic sql query.
Query
create proc spproc
#entityType varchar(50)
as
begin
declare #sql as nvarchar(max);
declare #condition as nvarchar(2000);
select = case when #entityType is not null then ' and [EntityType] = #entityType;' else ';' end;
select #sql = 'SELECT TOP 1000 [Id], [EntityId], [EntityType], [TenantId] FROM [FutureTrakProd].[dbo].[Activities] where TenantId = 1 ';
exec sp_executesql #sql,
N'#entityType nvarchar(1000)',
#entityType = #entityType
end
I'm working on an application for work that is going to query our employee database. The end users want the ability to search based on the standard name/department criteria, but they also want the flexibility to query for all people with the first name of "James" that works in the Health Department. The one thing I want to avoid is to simply have the stored procedure take a list of parameters and generate a SQL statement to execute, since that would open doors to SQL injection at an internal level.
Can this be done?
While the COALESCE trick is neat, my preferred method is:
CREATE PROCEDURE ps_Customers_SELECT_NameCityCountry
#Cus_Name varchar(30) = NULL
,#Cus_City varchar(30) = NULL
,#Cus_Country varchar(30) = NULL
,#Dept_ID int = NULL
,#Dept_ID_partial varchar(10) = NULL
AS
SELECT Cus_Name
,Cus_City
,Cus_Country
,Dept_ID
FROM Customers
WHERE (#Cus_Name IS NULL OR Cus_Name LIKE '%' + #Cus_Name + '%')
AND (#Cus_City IS NULL OR Cus_City LIKE '%' + #Cus_City + '%')
AND (#Cus_Country IS NULL OR Cus_Country LIKE '%' + #Cus_Country + '%')
AND (#Dept_ID IS NULL OR Dept_ID = #DeptID)
AND (#Dept_ID_partial IS NULL OR CONVERT(varchar, Dept_ID) LIKE '%' + #Dept_ID_partial + '%')
These kind of SPs can easily be code generated (and re-generated for table-changes).
You have a few options for handling numbers - depending if you want exact semantics or search semantics.
The most efficient way to implement this type of search is with a stored procedure. The statement shown here creates a procedure that accepts the required parameters. When a parameter value is not supplied it is set to NULL.
CREATE PROCEDURE ps_Customers_SELECT_NameCityCountry
#Cus_Name varchar(30) = NULL,
#Cus_City varchar(30) = NULL,
#Cus_Country varchar(30) =NULL
AS
SELECT Cus_Name,
Cus_City,
Cus_Country
FROM Customers
WHERE Cus_Name = COALESCE(#Cus_Name,Cus_Name) AND
Cus_City = COALESCE(#Cus_City,Cus_City) AND
Cus_Country = COALESCE(#Cus_Country,Cus_Country)
Taken from this page: http://www.sqlteam.com/article/implementing-a-dynamic-where-clause
I've done it before. It works well.
Erland Sommarskog's article Dynamic Search Conditions in T-SQL is a good reference on how to do this. Erland presents a number of strategies on how to do this without using dynamic SQL (just plain IF blocks, OR, COALESCE, etc) and even lists out the performance characteristics of each technique.
In case you have to bite the bullet and go through the Dynamic SQL path, you should also read Erland's Curse and Blessings of Dynamic SQL where he gives out some tips on how to properly write dynamic SQLs
It can be done, but usually these kitchen-sink procedures result in some poor query plans.
Having said all that, here is the tactic most commonly used for "optional" parameters. The normal approach is to treat NULL as "ommitted".
SELECT
E.EmployeeID,
E.LastName,
E.FirstName
WHERE
E.FirstName = COALESCE(#FirstName, E.FirstName) AND
E.LastName = COALESCE(#LastName, E.LastName) AND
E.DepartmentID = COALESCE(#DepartmentID, E.DepartmentID)
EDIT:
A far better approach would be parameterized queries.
Here is a blog post from one of the world's foremost authorities in this domain, Frans Bouma from LLBLGen Pro fame:
Stored Procedures vs. Dynamic Queries
Using the COALESCE method has a problem in that if your column has a NULL value, passing in a NULL search condition (meaning ignore the search condition) will not return the row in many databases.
For example, try the following code on SQL Server 2000:
CREATE TABLE dbo.Test_Coalesce (
my_id INT NOT NULL IDENTITY,
my_string VARCHAR(20) NULL )
GO
INSERT INTO dbo.Test_Coalesce (my_string) VALUES (NULL)
INSERT INTO dbo.Test_Coalesce (my_string) VALUES ('t')
INSERT INTO dbo.Test_Coalesce (my_string) VALUES ('x')
INSERT INTO dbo.Test_Coalesce (my_string) VALUES (NULL)
GO
DECLARE #my_string VARCHAR(20)
SET #my_string = NULL
SELECT * FROM dbo.Test_Coalesce WHERE my_string = COALESCE(#my_string, my_string)
GO
You will only get back two rows because in the rows where the column my_string is NULL you are effective getting:
my_string = COALESCE(#my_string, my_string) =>
my_string = COALESCE(NULL, my_string) =>
my_string = my_string =>
NULL = NULL
But of course, NULL does not equal NULL.
I try to stick with:
SELECT
my_id,
my_string
FROM
dbo.Test_Coalesce
WHERE
(#my_string IS NULL OR my_string = #my_string)
Of course, you can adjust that to use wild cards or whatever else you want to do.
Copying this from my blog post:
USE [AdventureWorks]
GO
CREATE PROCEDURE USP_GET_Contacts_DynSearch
(
-- Optional Filters for Dynamic Search
#ContactID INT = NULL,
#FirstName NVARCHAR(50) = NULL,
#LastName NVARCHAR(50) = NULL,
#EmailAddress NVARCHAR(50) = NULL,
#EmailPromotion INT = NULL,
#Phone NVARCHAR(25) = NULL
)
AS
BEGIN
SET NOCOUNT ON
DECLARE
#lContactID INT,
#lFirstName NVARCHAR(50),
#lLastName NVARCHAR(50),
#lEmailAddress NVARCHAR(50),
#lEmailPromotion INT,
#lPhone NVARCHAR(25)
SET #lContactID = #ContactID
SET #lFirstName = LTRIM(RTRIM(#FirstName))
SET #lLastName = LTRIM(RTRIM(#LastName))
SET #lEmailAddress = LTRIM(RTRIM(#EmailAddress))
SET #lEmailPromotion = #EmailPromotion
SET #lPhone = LTRIM(RTRIM(#Phone))
SELECT
ContactID,
Title,
FirstName,
MiddleName,
LastName,
Suffix,
EmailAddress,
EmailPromotion,
Phone
FROM [Person].[Contact]
WHERE
(#lContactID IS NULL OR ContactID = #lContactID)
AND (#lFirstName IS NULL OR FirstName LIKE '%' + #lFirstName + '%')
AND (#lLastName IS NULL OR LastName LIKE '%' + #lLastName + '%')
AND (#lEmailAddress IS NULL OR EmailAddress LIKE '%' + #lEmailAddress + '%')
AND (#lEmailPromotion IS NULL OR EmailPromotion = #lEmailPromotion)
AND (#lPhone IS NULL OR Phone = #lPhone)
ORDER BY ContactID
END
GO
We can use Generic #Search Parameter and pass any value to it for searching.
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
-- =============================================
-- Author: --
-- Create date:
-- Description: --
-- =============================================
CREATE PROCEDURE [dbo].[usp_StudentList]
#PageNumber INT = 1, -- Paging parameter
#PageSize INT = 10,-- Paging parameter
#Search VARCHAR(MAX) = NULL, --Generic Search Parameter
#OrderBy VARCHAR(MAX) = 'FirstName', --Default Column Name 'FirstName' for records ordering
#SortDir VARCHAR(MAX) = 'asc' --Default ordering 'asc' for records ordering
AS
BEGIN
SET NOCOUNT ON;
--Query required for paging, this query used to show total records
SELECT COUNT(StudentId) AS RecordsTotal FROM Student
SELECT Student.*,
--Query required for paging, this query used to show total records filtered
COUNT(StudentId) OVER (PARTITION BY 1) AS RecordsFiltered
FROM Student
WHERE
--Generic Search
-- Below is the column list to add in Generic Serach
(#Search IS NULL OR Student.FirstName LIKE '%'+ #Search +'%')
OR (#Search IS NULL OR Student.LastName LIKE '%'+ #Search +'%')
--Order BY
-- Below is the column list to allow sorting
ORDER BY
CASE WHEN #SortDir = 'asc' AND #OrderBy = 'FirstName' THEN Student.FirstName END,
CASE WHEN #SortDir = 'desc' AND #OrderBy = 'FirstName' THEN Student.FirstName END DESC,
CASE WHEN #SortDir = 'asc' AND #OrderBy = 'LastName' THEN Student.LastName END,
CASE WHEN #SortDir = 'desc' AND #OrderBy = 'LastName' THEN Student.LastName END DESC,
OFFSET #PageSize * (#PageNumber - 1) ROWS FETCH NEXT #PageSize ROWS ONLY;
END
My first thought was to write a query something like this...
SELECT EmpId, NameLast, NameMiddle, NameFirst, DepartmentName
FROM dbo.Employee
INNER JOIN dbo.Department ON dbo.Employee.DeptId = dbo.Department.Id
WHERE IdCrq IS NOT NULL
AND
(
#bitSearchFirstName = 0
OR
Employee.NameFirst = #vchFirstName
)
AND
(
#bitSearchMiddleName = 0
OR
Employee.NameMiddle = #vchMiddleName
)
AND
(
#bitSearchFirstName = 0
OR
Employee.NameLast = #vchLastName
)
AND
(
#bitSearchDepartment = 0
OR
Department.Id = #intDeptID
)
...which would then have the caller provide a bit flag if they want to search a particular field and then supply the value if they are to search for it, but I don't know if this is creating a sloppy WHERE clause or if I can get away with a CASE statement in the WHERE clause.
As you can see this particular code is in T-SQL, but I'll gladly look at some PL-SQL / MySQL code as well and adapt accordingly.
I would stick with the NULL/COALESCE method over AdHoc Queries, and then test to make sure you don't have performance problems.
If it turns out that you have slow running queries because it's doing a table scan when you're searching on columns that are indexed, you could always supplement the generic search stored procedure with additional specific ones that allow searching on these indexed fields. For instance, you could have a special SP that does searches by CustomerID, or Last/First Name.
Write a procedure to insert all employee data whose name start with A in table??
How can one call a stored procedure for each row in a table, where the columns of a row are input parameters to the sp without using a Cursor?
Generally speaking I always look for a set based approach (sometimes at the expense of changing the schema).
However, this snippet does have its place..
-- Declare & init (2008 syntax)
DECLARE #CustomerID INT = 0
-- Iterate over all customers
WHILE (1 = 1)
BEGIN
-- Get next customerId
SELECT TOP 1 #CustomerID = CustomerID
FROM Sales.Customer
WHERE CustomerID > #CustomerId
ORDER BY CustomerID
-- Exit loop if no more customers
IF ##ROWCOUNT = 0 BREAK;
-- call your sproc
EXEC dbo.YOURSPROC #CustomerId
END
You could do something like this: order your table by e.g. CustomerID (using the AdventureWorks Sales.Customer sample table), and iterate over those customers using a WHILE loop:
-- define the last customer ID handled
DECLARE #LastCustomerID INT
SET #LastCustomerID = 0
-- define the customer ID to be handled now
DECLARE #CustomerIDToHandle INT
-- select the next customer to handle
SELECT TOP 1 #CustomerIDToHandle = CustomerID
FROM Sales.Customer
WHERE CustomerID > #LastCustomerID
ORDER BY CustomerID
-- as long as we have customers......
WHILE #CustomerIDToHandle IS NOT NULL
BEGIN
-- call your sproc
-- set the last customer handled to the one we just handled
SET #LastCustomerID = #CustomerIDToHandle
SET #CustomerIDToHandle = NULL
-- select the next customer to handle
SELECT TOP 1 #CustomerIDToHandle = CustomerID
FROM Sales.Customer
WHERE CustomerID > #LastCustomerID
ORDER BY CustomerID
END
That should work with any table as long as you can define some kind of an ORDER BY on some column.
DECLARE #SQL varchar(max)=''
-- MyTable has fields fld1 & fld2
Select #SQL = #SQL + 'exec myproc ' + convert(varchar(10),fld1) + ','
+ convert(varchar(10),fld2) + ';'
From MyTable
EXEC (#SQL)
Ok, so I would never put such code into production, but it does satisfy your requirements.
I'd use the accepted answer, but another possibility is to use a table variable to hold a numbered set of values (in this case just the ID field of a table) and loop through those by Row Number with a JOIN to the table to retrieve whatever you need for the action within the loop.
DECLARE #RowCnt int; SET #RowCnt = 0 -- Loop Counter
-- Use a table variable to hold numbered rows containg MyTable's ID values
DECLARE #tblLoop TABLE (RowNum int IDENTITY (1, 1) Primary key NOT NULL,
ID INT )
INSERT INTO #tblLoop (ID) SELECT ID FROM MyTable
-- Vars to use within the loop
DECLARE #Code NVarChar(10); DECLARE #Name NVarChar(100);
WHILE #RowCnt < (SELECT COUNT(RowNum) FROM #tblLoop)
BEGIN
SET #RowCnt = #RowCnt + 1
-- Do what you want here with the data stored in tblLoop for the given RowNum
SELECT #Code=Code, #Name=LongName
FROM MyTable INNER JOIN #tblLoop tL on MyTable.ID=tL.ID
WHERE tl.RowNum=#RowCnt
PRINT Convert(NVarChar(10),#RowCnt) +' '+ #Code +' '+ #Name
END
Marc's answer is good (I'd comment on it if I could work out how to!)
Just thought I'd point out that it may be better to change the loop so the SELECT only exists once (in a real case where I needed to do this, the SELECT was quite complex, and writing it twice was a risky maintenance issue).
-- define the last customer ID handled
DECLARE #LastCustomerID INT
SET #LastCustomerID = 0
-- define the customer ID to be handled now
DECLARE #CustomerIDToHandle INT
SET #CustomerIDToHandle = 1
-- as long as we have customers......
WHILE #LastCustomerID <> #CustomerIDToHandle
BEGIN
SET #LastCustomerId = #CustomerIDToHandle
-- select the next customer to handle
SELECT TOP 1 #CustomerIDToHandle = CustomerID
FROM Sales.Customer
WHERE CustomerID > #LastCustomerId
ORDER BY CustomerID
IF #CustomerIDToHandle <> #LastCustomerID
BEGIN
-- call your sproc
END
END
If you can turn the stored procedure into a function that returns a table, then you can use cross-apply.
For example, say you have a table of customers, and you want to compute the sum of their orders, you would create a function that took a CustomerID and returned the sum.
And you could do this:
SELECT CustomerID, CustomerSum.Total
FROM Customers
CROSS APPLY ufn_ComputeCustomerTotal(Customers.CustomerID) AS CustomerSum
Where the function would look like:
CREATE FUNCTION ComputeCustomerTotal
(
#CustomerID INT
)
RETURNS TABLE
AS
RETURN
(
SELECT SUM(CustomerOrder.Amount) AS Total FROM CustomerOrder WHERE CustomerID = #CustomerID
)
Obviously, the example above could be done without a user defined function in a single query.
The drawback is that functions are very limited - many of the features of a stored procedure are not available in a user-defined function, and converting a stored procedure to a function does not always work.
For SQL Server 2005 onwards, you can do this with CROSS APPLY and a table-valued function.
Using CROSS APPLY in SQL Server 2005
Just for clarity, I'm referring to those cases where the stored procedure can be converted into a table valued function.
This is a variation on the answers already provided, but should be better performing because it doesn't require ORDER BY, COUNT or MIN/MAX. The only disadvantage with this approach is that you have to create a temp table to hold all the Ids (the assumption is that you have gaps in your list of CustomerIDs).
That said, I agree with #Mark Powell though that, generally speaking, a set based approach should still be better.
DECLARE #tmp table (Id INT IDENTITY(1,1) PRIMARY KEY NOT NULL, CustomerID INT NOT NULL)
DECLARE #CustomerId INT
DECLARE #Id INT = 0
INSERT INTO #tmp SELECT CustomerId FROM Sales.Customer
WHILE (1=1)
BEGIN
SELECT #CustomerId = CustomerId, #Id = Id
FROM #tmp
WHERE Id = #Id + 1
IF ##rowcount = 0 BREAK;
-- call your sproc
EXEC dbo.YOURSPROC #CustomerId;
END
This is a variation of n3rds solution above. No sorting by using ORDER BY is needed, as MIN() is used.
Remember that CustomerID (or whatever other numerical column you use for progress) must have a unique constraint. Furthermore, to make it as fast as possible CustomerID must be indexed on.
-- Declare & init
DECLARE #CustomerID INT = (SELECT MIN(CustomerID) FROM Sales.Customer); -- First ID
DECLARE #Data1 VARCHAR(200);
DECLARE #Data2 VARCHAR(200);
-- Iterate over all customers
WHILE #CustomerID IS NOT NULL
BEGIN
-- Get data based on ID
SELECT #Data1 = Data1, #Data2 = Data2
FROM Sales.Customer
WHERE [ID] = #CustomerID ;
-- call your sproc
EXEC dbo.YOURSPROC #Data1, #Data2
-- Get next customerId
SELECT #CustomerID = MIN(CustomerID)
FROM Sales.Customer
WHERE CustomerID > #CustomerId
END
I use this approach on some varchars I need to look over, by putting them in a temporary table first, to give them an ID.
If you don't what to use a cursor I think you'll have to do it externally (get the table, and then run for each statement and each time call the sp)
it Is the same as using a cursor, but only outside SQL.
Why won't you use a cursor ?
I usually do it this way when it's a quite a few rows:
Select all sproc parameters in a dataset with SQL Management Studio
Right-click -> Copy
Paste in to excel
Create single-row sql statements with a formula like '="EXEC schema.mysproc #param=" & A2' in a new excel column. (Where A2 is your excel column containing the parameter)
Copy the list of excel statements into a new query in SQL Management Studio and execute.
Done.
(On larger datasets i'd use one of the solutions mentioned above though).
DELIMITER //
CREATE PROCEDURE setFakeUsers (OUT output VARCHAR(100))
BEGIN
-- define the last customer ID handled
DECLARE LastGameID INT;
DECLARE CurrentGameID INT;
DECLARE userID INT;
SET #LastGameID = 0;
-- define the customer ID to be handled now
SET #userID = 0;
-- select the next game to handle
SELECT #CurrentGameID = id
FROM online_games
WHERE id > LastGameID
ORDER BY id LIMIT 0,1;
-- as long as we have customers......
WHILE (#CurrentGameID IS NOT NULL)
DO
-- call your sproc
-- set the last customer handled to the one we just handled
SET #LastGameID = #CurrentGameID;
SET #CurrentGameID = NULL;
-- select the random bot
SELECT #userID = userID
FROM users
WHERE FIND_IN_SET('bot',baseInfo)
ORDER BY RAND() LIMIT 0,1;
-- update the game
UPDATE online_games SET userID = #userID WHERE id = #CurrentGameID;
-- select the next game to handle
SELECT #CurrentGameID = id
FROM online_games
WHERE id > LastGameID
ORDER BY id LIMIT 0,1;
END WHILE;
SET output = "done";
END;//
CALL setFakeUsers(#status);
SELECT #status;
A better solution for this is to
Copy/past code of Stored Procedure
Join that code with the table for which you want to run it again (for each row)
This was you get a clean table-formatted output. While if you run SP for every row, you get a separate query result for each iteration which is ugly.
In case the order is important
--declare counter
DECLARE #CurrentRowNum BIGINT = 0;
--Iterate over all rows in [DataTable]
WHILE (1 = 1)
BEGIN
--Get next row by number of row
SELECT TOP 1 #CurrentRowNum = extendedData.RowNum
--here also you can store another values
--for following usage
--#MyVariable = extendedData.Value
FROM (
SELECT
data.*
,ROW_NUMBER() OVER(ORDER BY (SELECT 0)) RowNum
FROM [DataTable] data
) extendedData
WHERE extendedData.RowNum > #CurrentRowNum
ORDER BY extendedData.RowNum
--Exit loop if no more rows
IF ##ROWCOUNT = 0 BREAK;
--call your sproc
--EXEC dbo.YOURSPROC #MyVariable
END
I had some production code that could only handle 20 employees at a time, below is the framework for the code. I just copied the production code and removed stuff below.
ALTER procedure GetEmployees
#ClientId varchar(50)
as
begin
declare #EEList table (employeeId varchar(50));
declare #EE20 table (employeeId varchar(50));
insert into #EEList select employeeId from Employee where (ClientId = #ClientId);
-- Do 20 at a time
while (select count(*) from #EEList) > 0
BEGIN
insert into #EE20 select top 20 employeeId from #EEList;
-- Call sp here
delete #EEList where employeeId in (select employeeId from #EE20)
delete #EE20;
END;
RETURN
end
I had a situation where I needed to perform a series of operations on a result set (table). The operations are all set operations, so its not an issue, but...
I needed to do this in multiple places. So putting the relevant pieces in a table type, then populating a table variable w/ each result set allows me to call the sp and repeat the operations each time i need to .
While this does not address the exact question he asks, it does address how to perform an operation on all rows of a table without using a cursor.
#Johannes offers no insight into his motivation , so this may or may not help him.
my research led me to this well written article which served as a basis for my solution
https://codingsight.com/passing-data-table-as-parameter-to-stored-procedures/
Here is the setup
drop type if exists cpRootMapType
go
create type cpRootMapType as Table(
RootId1 int
, RootId2 int
)
go
drop procedure if exists spMapRoot2toRoot1
go
create procedure spMapRoot2toRoot1
(
#map cpRootMapType Readonly
)
as
update linkTable set root = root1
from linktable lt
join #map m on lt.root = root2
update comments set root = root1
from comments c
join #map m on c.root = root2
-- ever growing list of places this map would need to be applied....
-- now consolidated into one place
here is the implementation
... populate #matches
declare #map cpRootMapType
insert #map select rootid1, rootid2 from #matches
exec spMapRoot2toRoot1 #map
I like to do something similar to this (though it is still very similar to using a cursor)
[code]
-- Table variable to hold list of things that need looping
DECLARE #holdStuff TABLE (
id INT IDENTITY(1,1) ,
isIterated BIT DEFAULT 0 ,
someInt INT ,
someBool BIT ,
otherStuff VARCHAR(200)
)
-- Populate your #holdStuff with... stuff
INSERT INTO #holdStuff (
someInt ,
someBool ,
otherStuff
)
SELECT
1 , -- someInt - int
1 , -- someBool - bit
'I like turtles' -- otherStuff - varchar(200)
UNION ALL
SELECT
42 , -- someInt - int
0 , -- someBool - bit
'something profound' -- otherStuff - varchar(200)
-- Loop tracking variables
DECLARE #tableCount INT
SET #tableCount = (SELECT COUNT(1) FROM [#holdStuff])
DECLARE #loopCount INT
SET #loopCount = 1
-- While loop variables
DECLARE #id INT
DECLARE #someInt INT
DECLARE #someBool BIT
DECLARE #otherStuff VARCHAR(200)
-- Loop through item in #holdStuff
WHILE (#loopCount <= #tableCount)
BEGIN
-- Increment the loopCount variable
SET #loopCount = #loopCount + 1
-- Grab the top unprocessed record
SELECT TOP 1
#id = id ,
#someInt = someInt ,
#someBool = someBool ,
#otherStuff = otherStuff
FROM #holdStuff
WHERE isIterated = 0
-- Update the grabbed record to be iterated
UPDATE #holdAccounts
SET isIterated = 1
WHERE id = #id
-- Execute your stored procedure
EXEC someRandomSp #someInt, #someBool, #otherStuff
END
[/code]
Note that you don't need the identity or the isIterated column on your temp/variable table, i just prefer to do it this way so i don't have to delete the top record from the collection as i iterate through the loop.