I have a stored procedure that uses dynamic SQL. My company recently decided to get rid of dynamic code.
Here I greatly simplified it to explain my point; #PersonID and #MhnNum are parameters of the stored procedure:
Declare #sql Varchar(max)
Set #sql="Select from tableA p"
If #PersonID Is Not Null
Set #sql = #sql + ' Where p.[ID] = ' + cast(#PersonID as varchar(12))
If #MhnNum Is Not Null
Set #sql = #sql + ' Where p.[MhnNum] = ' + '''' + cast(#MhnNum as varchar(12)) + ''''
Is there an (easy) way to get rid of this dynamic SQL?
Right now my solution is to create 2 If's with repetitive code. There must be a more elegant way to do this.
Honestly, Dynamic SQL is probably the way to go, as you have a catch-all query, just not your dynamic SQL. It's a huge injection risk. Parametrise the statements:
DECLARE #SQL nvarchar(MAX),
#CRLF nchar(2) = CHAR(13) + CHAR(10);
SET #SQL = N'SELECT *' + #CRLF +
N'FROM TableA A' + #CRLF +
CASE WHEN #PersonID IS NOT NULL THEN N'WHERE A.ID = #PersonID;'
WHEN #MhnNum IS NOT NULL THEN N'WHERE A.MhnNum = #MhnNum;'
END;
EXEC sys.sp_executesql #SQL, N'#PersonID int, #MhnNum int', #PersonID, #MhnNum;
Using a non-dynamic approach will cause the caching of bad query plans, which isn't going to be desired.
If you must use non dynamic SQL (due to a pointless Company Policy, there is nothing wrong with Dynamic SQL if used correctly), then add OPTION RECOMPILE:
IF #PersonID IS NOT NULL AND #MhnNum IS NOT NULL
THROW 68542, N'Both #PersonID and #MhnNum cannot be non-NULL values.', 11;
SELECT *
FROM TableA A
WHERE (A.ID = #PersonID OR #PersonID IS NULL)
AND (A.MhnNum = #MhnNum OR #MhnNum IS NULL)
OPTION (RECOMPILE);
The THROW is in there, as your code will also error if you have 2 non-NULL values.
A simple combination of or and and can get you the same result:
Select *
From tableA
Where (#personId is null or id = #personId)
And (#MhnNum is null or whnNum =#mhnNum)
(writing in a comment would be a mess)
You could check the parameters and act accordingly. ie:
Select from tableA p
where (#PersonID IS NULL or p.[ID] = #PersonID) and
(#MhnNum IS NULL or p.[MhnNum] = #MhnNum);
Related
Think I am missing something really obvious but I want to use a parameter in a where statement but when I do I get the following error (This is being run as a stored procedure if that makes any difference)
Invalid column name 'John'
The where statement in question
USE [Reports]
GO
/****** Object: StoredProcedure [Reports].[Alarm_TestSignOffFull_Qry] Script Date: 25/10/2018 08:56:26 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
ALTER PROCEDURE [Reports].[CustomerSearch]
-- Add the parameters for the stored procedure here
#Database VARCHAR(20)
,#Schema VARCHAR(20)
,#Name VARCHAR(20)
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
-- Insert statements for procedure here
EXEC ('
USE ' + #Database + '
SELECT Customer.Name,
[Product Categories].[Product Number]
RIGHT JOIN ((' + #Schema +'.Customer INNER JOIN ' + #Schema +'.Client ON Customer.Name = Client.Name)
LEFT JOIN ' + #Schema +'.Product ON Client.Name = Product.Client)
ON [Product Categories].[Product Number] = Client.Product
WHERE (Customer.Name = ' + #Name + ' AND Customer.Order = ''Y'') OR (Client.Name = ' + #Name + ' AND Customer.Order = ''Y'');
' )
END
GO
The parameter #Name is declared as VARCHAR(20)
Dont think too much about it had to find and replace table names and column names and stuff to make sure I am allowed to post it.
The select on the real thing also brings through a lot more fields but not too worried about that for this question
To emphasize it is the where statements that are going wrong the rest of the query works and it all works when I do not use any parameters
I'm posting this as answer, as the comments aren't enough to explain, however, there is far too little detail here to actually answer the OP's question.
Firstly, the error the OP is getting would be impossible to get with the tiny amount of SQL they have provided. If we expand it into a full Pseudo-SQL query, we get:
SELECT {Columns}
FROM Customer
WHERE Customer.Name = (' + #Name + ');
This would return rows in the Customer table, where Name has the literal string value ' + #Name + '. Very unlikely.
I suspect that the OP simply needs to do:
SELECT {Columns}
FROM Customer
WHERE Customer.Name = #Name;
HOWEVER, the error the OP is getting strong implies they are using Dynamic SQL. This means they have a query like this:
DECLARE #SQL nvarchar(MAX);
DECLARE #Name varchar(20) = 'John';
SET #SQL = N'SELECT {Columns} FROM Customer WHERE Customer.Name = (' + #Name + ');';
EXEC (#SQL);
The problem here is that translates into the SQL:
SELECT {Columns} FROM Customer WHERE Customer.Name = (John);
Notice, no quotes around John, and why the error.
What you are doing here, however, is a very bad idea. Raw string concatenation like that leaves your SQL wide open to injection(SQL injection). Parametrise your SQL:
DECLARE #SQL nvarchar(MAX);
DECLARE #Name varchar(20) = 'John';
SET #SQL = N'SELECT {Columns} FROM Customer WHERE Customer.Name = #dName;'
EXEC sp_executesql #SQL, N'#dName varchar(20)',#dName = #Name;`
Edit: Ok, the OP has provided us with their query. Honestly, it's a mess, far worse that I wanted it to be. Like i said before, raw string concatenation is an awful idea, it leaves you open to injection. You can't get rid of concatenation for dynamic objects, but you can use QUOTENAME, which makes the code safe(r).
You're also missing a FROM; no idea what that needs to be, so i've left that as pseudo-sql. That RIGHT JOIN is very messy too, but I've no idea what you're trying to achieve there.
Anyway, this need to replace your EXEC command. Read the below, understand the below, and don't make the mistake of concatenating raw strings in your dynamic SQL:
DECLARE #SQL nvarchar(MAX)
SET #SQL = N' USE ' + QUOTENAME(#Database) + N';' + NCHAR(10) +
N'SELECT Customer.Name,' + NCHAR(10) +
N' [Product Categories].[Product Number]' + NCHAR(10) +
--where is your FROM?
N'FROM {Something}' + NCHAR(10) + --This needs fixing
N' RIGHT JOIN((' + QUOTENAME(#Schema) + N'.Customer' + NCHAR(10) + --This whole thing is messy, but i'm not fixing it, as I have no data
N' INNER JOIN ' + QUOTENAME(#Schema) + N'.Client ON Customer.Name = Client.Name)' + NCHAR(10) +
N' LEFT JOIN ' + QUOTENAME(#Schema) + N'.Product ON Client.Name = Product.Client)ON [Product Categories].[Product Number] = Client.Product' + NCHAR(10) +
N'WHERE (Customer.Name = #dName' + NCHAR(10) +
N' AND Customer.[Order] = ''Y'')' + NCHAR(10) +
N' OR (Client.Name = #dName' + NCHAR(10) +
N' AND Customer.[Order] = ''Y'');';
EXEC sp_executesql #SQL, N'#dName varchar(20)', #dName = #Name;
If you are directly write in sql then you can use as below
WHERE Customer.Name = #Name
You are using a Dynamic SQL, and if you print your query it will be like
WHERE Customer.Name = (John)
And that's wrong, you can do like
EXECUTE sp_executesql N' ..WHERE Customer.Name = #CName',
N'#CName VARCHAR(20)',
#CName = 'John';
Or even
--#Name should be declared and has 'John' value
EXECUTE sp_executesql N' ..WHERE Customer.Name = #CName',
N'#Name VARCHAR(20)',
#Name = #Name;
Why does SQL think that the parameter is a column and not just plain text?
Well, as you can see before, the parameter you pass will be John not 'John', thus SQL Server will think it's a column name.
try giving single quotes in where clause
WHERE (Customer.Name = ''' + #Name + ''' AND Customer.Order = ''Y'')
OR (Client.Name = ''' + #Name + ''' AND Customer.Order = ''Y'');
We have a stored procedure that is used to allow users to search in a table with 20 million records and 40 columns wide. There are about 20 different columns they can search on (any combination) from and all those columns are in the WHERE clause.
Furthermore each columns is checked for Null and needs to be able to search with just part of the data.
Here is an example
(
#FirstName IS NULL
OR (RTRIM(UPPER(FirstName)) LIKE RTRIM(UPPER(#FirstName)) + '%')
)
AND (#LastName IS NULL)
What is a best way to rewrite this stored procedure? Should I break this stored procedure into multiple small stored procedures? If so how? I will need to allow user to search
When I look at the execution plan, regardless of what columns are passed, it always does the index scan
I had exactly this situation years ago, millions of rows and numerous filter parameters and the best method is to use dynamic sql. Construct a SQL statement based on the parameters that have values, then execute the SQL statement. (EXEC sp_executesql #sql)
The select clause of the sql statement is static but the from clause and the where clause is based on the parameters.
CREATE PROCEDURE dbo.DynamicSearch
#FirstName VARCHAR(20),
#LastName VARCHAR(20),
#CompanyName VARCHAR(50)
AS
BEGIN
DECLARE #SQL NVARCHAR(MAX) = N''
DECLARE #Select NVARCHAR(MAX) = N'SELECT ColA, ColB, ColC, ColD '
DECLARE #From NVARCHAR(MAX) = N'From Person'
DECLARE #Where NVARCHAR(MAX) = N''
IF #FirstName IS NOT NULL
Begin
Set #Where = #Where + 'FirstName = ''' + #FirstName + ''''
End
IF #LastName IS NOT NULL
Begin
if len(#Where) > 0
Begin
Set #Where = #Where + ' AND '
End
Set #Where = #Where + 'LastName = ''' + #LastName + ''''
End
IF #CompanyName IS NOT NULL
Begin
if len(#Where) > 0
Begin
Set #Where = #Where + ' AND '
End
Set #From = #From + ' inner join Company on person.companyid = company.companyid '
Set #Where = #Where + 'company.CompanyName = ''' + #CompanyName + ''''
End
Set #SQL = #Select + #From + #Where
EXECUTE sp_executesql #sql
END
To go down the dynamic SQL route you would use something like:
CREATE PROCEDURE dbo.SearchSomeTable
#FirstName VARCHAR(20),
#LastName VARCHAR(20),
#AnotherCol INT
AS
BEGIN
DECLARE #SQL NVARCHAR(MAX) = N'SELECT SomeColumn FROM SomeTable WHERE 1 = 1',
#ParamDefinition NVARCHAR(MAX) = N'#FirstName VARCHAR(20),
#LastName VARCHAR(20),
#AnotherCol INT';
IF #FirstName IS NOT NULL
#SQL = #SQL + ' AND FirstName = #FirstName';
IF #LastName IS NOT NULL
#SQL = #SQL + ' AND LastName = #LastName';
IF #AnotherCol IS NOT NULL
#SQL = #SQL + ' AND AnotherCol = #AnotherCol';
EXECUTE sp_executesql #sql, #ParamDefinition, #FirstName, #LastName, #AnotherCol;
END
Otherwise you will need to use the OPTION (RECOMPILE) query hint to force the query to recompile each time it is run to get the optimal plan for the particular parameters you have passed.
Another possibility, though the above is perhaps the most logical, is to create one or more "search" criteria tables into which you insert the users selections, and then perform LEFT JOINS against the search criteria. You would only be able to do this for cases in which there is an equivalency and preferably an small datatype such as int. For string comparisons, these kind of joins would be potentially dreadful as far as performance goes. The only problem with the above suggestion (dynamic-sql) is the possibility of plan cache bloat as each execution in which there is only one difference with an existing plan would cause a new plan to be generated.
so i want to pass several values to a stored procedure. then bases on those values, add to a variable that would then be set as my where clause. but im stumped and google aint helping. here is what i have/my idea
CREATE PROCEDURE sp_RunReport #TransType varchar(255), #Accounts varchar(75)
AS
--declare a varchar WHERE clause variable here
IF #TransType <> ''
--add to WHERE clause variable
iF #Accounts <>''
--add to WHERE clause variable
SELECT *
FROM log
WHERE --my WHERE clause
GO
i dont see how this is possible. i can do it all in c sharp in the front end, but i feel like it should be done and can be done in the stored procedure. any help would be greatly appreciated
While dynamic SQL in previous answer is perhaps fine, I would suggest this "pure" SQL approach
WHERE TransType = ISNULL(#TransType, TransType)
AND Accounts = ISNULL(#Accounts, Accounts)
There is some room to optimize performance by getting rid of ISULL (not optimal when used in WHERE clause), but this should give you the idea.
Of course, insteead of AND your logic may reuire an OR and also you need to ensure that params are "properly" empty (NULL in my case or whatever will constitute "empty" if you re-write this to remove the ISNULL)
You can use dynamic SQL using EXEC:
CREATE PROCEDURE sp_RunReport #TransType varchar(255), #Accounts varchar(75)
AS
DECLARE #where VARCHAR(4000) = ' WHERE 1=1'
DECLARE #sql NVARCHAR(4000)
IF #TransType <> ''
SET #where = #where + ' AND TransType = ''' + #TransType + ''''
IF #Accounts <>''
SET #where = #where + ' AND Accounts = ''' + #Accounts + ''''
SET #sql = 'SELECT *
FROM log' + #where
exec (#sql)
Or using sp_executesql which is recommended:
CREATE PROCEDURE sp_RunReport #TransType varchar(255), #Accounts varchar(75)
AS
DECLARE #where NVARCHAR(4000) = N' WHERE 1=1'
DECLARE #sql NVARCHAR(4000)
DECLARE #ParmDefinition nvarchar(500) = N'#TransType varchar(255), #Accounts varchar(75)'
IF #TransType <> ''
SET #where = #where + N' AND TransType = #TransType'
IF #Accounts <>''
SET #where = #where + N' AND Accounts = #Accounts'
SET #sql = N'SELECT *
FROM log' + #where
EXECUTE sp_executesql #sql, #ParmDefinition,
#TransType = #TransType, #Accounts = #Accounts
I have a stored procedure that looks like this:
create stored procedure aaa
#columnName nvarchar(10),
#comparisonParam nvarchar(10),
#val nvarchar(100)
as
declare #date date
set #date = convert(#val, date)
exec('select * from Sheep where ' + #columnName + #comparisonParam + #date )
When actually the query is supposed to be like this:
select * from Sheep where birth_date = 12-12-2000
When I run the procedure it doesn't work with date value, but with string and int it works.
The date value must be quoted.
On a side note, I'd warn against doing this. If you need to build up dynamic sql you need to consider the risks such as: sql injection attacks, bad syntax, invalid semantics etc.
Consider using an existing component to build the query. A few examples:
.NET LINQ (to SQL/Entities) http://msdn.microsoft.com/en-us/library/bb397926.aspx
.NET SqlCommandBuilder http://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqlcommandbuilder.aspx
See Best way of constructing dynamic sql queries in C#/.NET3.5?
Your date literal needs to be surrounded in single quotes (I use CHAR(39) usually since it is easier to read and doesn't require escaping). Otherwise you are saying:
WHERE birth_date = (12) - (12) - (2000)
Which resolves to:
WHERE birth_date = -2000
Which resolves to DATEADD(DAY, -2000, '1900-01-01') or:
WHERE birth_date = '1894-07-11'
This is probably not going to yield the results you want.
With typical SQL injection warnings in place of course, and assuming that #columnName is always a string or date/time column, here is how I would re-write your stored procedure (though I would probably try to avoid the dynamic SQL altogether if I could).
ALTER PROCEDURE dbo.aaa
#columnName NVARCHAR(10),
#comparisonParam NVARCHAR(10),
#val NVARCHAR(100)
AS
BEGIN
SET NOCOUNT ON;
DECLARE #sql NVARCHAR(MAX);
SET #sql = N'SELECT * FROM dbo.Sheep WHERE '
+ QUOTENAME(#columnName) + #comparisonParam + CHAR(39)
+ REPLACE(#val, CHAR(39), CHAR(39) + CHAR(39))
+ CHAR(39);
EXEC sp_executesql #sql;
END
GO
In order to thwart potential issues you may want to add validation for columns and data types, and ensure that the operation is one you expect. e.g.
CREATE PROCEDURE dbo.bbb
#columnName NVARCHAR(10),
#comparisonParam NVARCHAR(10),
#val NVARCHAR(100)
AS
BEGIN
SET NOCOUNT ON;
DECLARE #delimiter CHAR(1);
SELECT #delimiter = CASE
WHEN [system_type_id] IN
(104,48,52,56,127,59,60,62,106,108,122) THEN '' -- numeric
WHEN [system_type_id] IN
(35,40,41,42,43,58,61,99,167,175,231,239) THEN CHAR(39) -- string
END FROM sys.columns WHERE [object_id] = OBJECT_ID(N'dbo.Sheep')
AND name = #columnName;
IF #delimiter IS NULL
BEGIN
RAISERROR('Column ''%s'' was not found or an unexpected data type.', 11, 1,
#columnName);
RETURN;
END
IF #comparisonParam NOT IN (N'=', N'>=', N'<=', N'<', N'>', N'LIKE')
BEGIN
RAISERROR('Comparison param ''%s'' was not valid.', 11, 1, #comparisonParam);
RETURN;
END
DECLARE #sql NVARCHAR(MAX);
SET #sql = N'SELECT * FROM dbo.Sheep WHERE '
+ QUOTENAME(#columnName) + ' ' + #comparisonParam + ' '
+ #delimiter + REPLACE(#val, CHAR(39), CHAR(39) + CHAR(39))
+ #delimiter;
EXEC sp_executesql #sql;
END
GO
Now make sure you use an unambiguous date format for your string literals. 12-12-2000 is not a good choice. 20001212 is much better.
There are possibly some ways to do this without dynamic SQL - I gave a very simplified answer here. This may be feasible depending on the data types, the number of potential columns, and the number of operations you want to support.
create stored procedure aaa
#columnName nvarchar(10),
#comparisonParam nvarchar(10),
#val nvarchar(100)
as
declare #date date
set #date = convert(#val, date)
exec('select * from Sheep where ' + #columnName + #comparisonParam + #date )
Build your dynamic SQL using a typed date parameter. Use sp_executesql which allows to pass parameter definitions and parameter values to the embedded SQL:
create procedure aaa
#columnName nvarchar(10),
#comparisonParam nvarchar(10),
#val nvarchar(100)
as
declare #date date, #sql nvarchar(max);
set #date = convert(#val, date);
-- Note how #date is a *variable* in the generated SQL:
set #sql =N'select * from Sheep where ' +
quotename(#columnName) + #comparisonParam + N'#date';
-- Use sp_executesql and define the type and value of the variable
exec sp_executesql #sql, N'#date date', #date;
You need to create table valued function for this rather than creating a stored procedure.
You can use any table valued function like
SELECT * from dbo.CallMyFunction(parameter1, parameter2
eg.
CREATE FUNCTION Sales.ufn_SalesByStore (#storeid int)
RETURNS TABLE
AS
RETURN
(
SELECT P.ProductID, P.Name, SUM(SD.LineTotal) AS 'Total'
FROM Production.Product AS P
JOIN Sales.SalesOrderDetail AS SD ON SD.ProductID = P.ProductID
JOIN Sales.SalesOrderHeader AS SH ON SH.SalesOrderID = SD.SalesOrderID
JOIN Sales.Customer AS C ON SH.CustomerID = C.CustomerID
WHERE C.StoreID = #storeid
GROUP BY P.ProductID, P.Name
);
GO
See this for reference http://msdn.microsoft.com/en-us/library/ms191165(v=sql.105).aspx
EDIT
Instead of using dynamic sql try giving a thought on
SELECT * FROM
FROM [dbo].[Person]
WHERE ([PersonID] = #PersonID
OR #AreaID IS NULL
)
AND (([Code] BETWEEN #Code AND CHAR(255))
OR #Code IS NULL
)
AND (([Name] BETWEEN #Name AND CHAR(255))
OR #Name IS NULL
)
AND (([Notes] BETWEEN #Notes AND CHAR(255))
OR #Notes IS NULL
)
I will preface this question by saying, I do not think it is solvable. I also have a workaround, I can create a stored procedure with an OUTPUT to accomplish this, it is just easier to code the sections where I need this checksum using a function.
This code will not work because of the Exec SP_ExecuteSQL #SQL calls. Anyone know how to execute dynamic SQL in a function? (and once again, I do not think it is possible. If it is though, I'd love to know how to get around it!)
Create Function Get_Checksum
(
#DatabaseName varchar(100),
#TableName varchar(100)
)
RETURNS FLOAT
AS
BEGIN
Declare #SQL nvarchar(4000)
Declare #ColumnName varchar(100)
Declare #i int
Declare #Checksum float
Declare #intColumns table (idRecord int identity(1,1), ColumnName varchar(255))
Declare #CS table (MyCheckSum bigint)
Set #SQL =
'Insert Into #IntColumns(ColumnName)' + Char(13) +
'Select Column_Name' + Char(13) +
'From ' + #DatabaseName + '.Information_Schema.Columns (NOLOCK)' + Char(13) +
'Where Table_Name = ''' + #TableName + '''' + Char(13) +
' and Data_Type = ''int'''
-- print #SQL
exec sp_executeSql #SQL
Set #SQL =
'Insert Into #CS(MyChecksum)' + Char(13) +
'Select '
Set #i = 1
While Exists(
Select 1
From #IntColumns
Where IdRecord = #i)
begin
Select #ColumnName = ColumnName
From #IntColumns
Where IdRecord = #i
Set #SQL = #SQL + Char(13) +
CASE WHEN #i = 1 THEN
' Sum(Cast(IsNull(' + #ColumnName + ',0) as bigint))'
ELSE
' + Sum(Cast(IsNull(' + #ColumnName + ',0) as bigint))'
END
Set #i = #i + 1
end
Set #SQL = #SQL + Char(13) +
'From ' + #DatabaseName + '..' + #TableName + ' (NOLOCK)'
-- print #SQL
exec sp_executeSql #SQL
Set #Checksum = (Select Top 1 MyChecksum From #CS)
Return isnull(#Checksum,0)
END
GO
It "ordinarily" can't be done as SQL Server treats functions as deterministic, which means that for a given set of inputs, it should always return the same outputs. A stored procedure or dynamic sql can be non-deterministic because it can change external state, such as a table, which is relied on.
Given that in SQL server functions are always deterministic, it would be a bad idea from a future maintenance perspective to attempt to circumvent this as it could cause fairly major confusion for anyone who has to support the code in future.
Here is the solution
Solution 1:
Return the dynamic string from Function then
Declare #SQLStr varchar(max)
DECLARE #tmptable table (<columns>)
set #SQLStr=dbo.function(<parameters>)
insert into #tmptable
Exec (#SQLStr)
select * from #tmptable
Solution 2:
call nested functions by passing parameters.
You can get around this by calling an extended stored procedure, with all the attendant hassle and security problems.
http://decipherinfosys.wordpress.com/2008/07/16/udf-limitations-in-sql-server/
http://decipherinfosys.wordpress.com/2007/02/27/using-getdate-in-a-udf/
Because functions have to play nicely with the query optimiser there are quite a few restrictions on them. This link refers to an article that discusses the limitations of UDF's in depth.
Thank you all for the replies.
Ron: FYI, Using that will throw an error.
I agree that not doing what I originally intended is the best solution, I decided to go a different route. My two choices were to use sum(cast(BINARY_CHECKSUM(*) as float)) or an output parameter in a stored procedure. After unit testing speed of each, I decided to go with sum(cast(BINARY_CHECKSUM(*) as float)) to get a comparable checksum value for each table's data.