Is there a way to get build a WHERE clause on the fly in a sql statement?
This code is within a Stored Procedure. I have x amount of parameters and each parameter's default value is NULL
SELECT *
FROM MyTable m
WHERE
IF(NOT(#Param1 IS NULL))
m.Col1 = #Param1
END IF
AND
IF(NOT(#Param2 IS NULL))
m.Col2 = #Param2
END IF
[EDIT:] I'm running SQL server 2005.
[EDIT:] The number of parameters are fixed, but can have a NULL value. If a parameter has a NULL value it shouldn't be included in the WHERE clause. Each parameter also correlates to a specific column.
Isn't this equivalent to the following, without any dynamic behavior in it?
SELECT *
FROM MyTable m
WHERE
(#Param1 IS NULL OR m.Col1 = #Param1)
AND
(#Param2 IS NULL OR m.Col2 = #Param2)
Or is there a possibility that the columns themselves might be missing?
Assuming SQL Server 2005+ syntax because the database wasn't specified... Highly recommended reading before addressing the query: The curse and blessings of dynamic SQL
DECLARE #SQL NVARCHAR(4000)
SET #SQL = N'SELECT m.*
FROM MyTable m
WHERE 1 = 1 '
SET #SQL = #SQL + CASE
WHEN #param1 IS NOT NULL THEN ' AND m.col1 = #param1 '
ELSE ' '
END
SET #SQL = #SQL + CASE
WHEN #param2 IS NOT NULL THEN ' AND m.col2 = #param2 '
ELSE ' '
END
BEGIN
EXEC sp_executesql #SQL,
N'#param1 [replace w/ data type], #param2 [replace w/ data type]'
#param1, #param2
END
You may be forced to use dynamic sql whether you're using a stored proc or not. Before you implement this stored proc, think about a few things.
Are the parameters themselves dynamic? Would you use 2 parameters one call, 10 the next? If this is the case you will need to create a ton of "placeholder" stored proc parameters that may not be used every call. What if you define 10 placeholder parameters but then need 11? This is not clean. Maybe you could use some sort of array parameter....
If parameters are dynamic, you will be forced to use dynamic SQL within your proc. This opens you up to injection attacks. You will have to manually escape all input within your stored proc. Using dymamaic sql in a proc defeats one of the main reasons for using procs.
If parameters are truly dynamic, I may actually favor generating sql from the code rather than the stored proc. Why? If you generate from the code you can use the ADO library to escape the input.
Just make sure you do something like....
sql = "select * from table where colA=#colA"; //dyanmically generated
SQlCommand.Parameters.add("#colA", valueA);
And not....
sql = "select * from table where colA=" + valueA; //dyanmically generated the wrong way
Also a 3rd thing to think about. What if the data type of the parameters is dynamic? Stored procs start to fall apart because they are a defined interface. If your operations do not conform to a interface, trying to squish them into a pre-set interface is going to be ugly.
If you're making some sort of open-ended graphical query tool, this situation will pop-up. Most of the time your data access will conform to an interface, but when it doesn't..... stored procs may not be the way to go.
Related
I have a stored procedure that can get the number of records in a table, in which the #tableName is the parameter of the stored procedure. Let's call it FastCount:
SELECT OBJECT_NAME(object_id), SUM(row_count) AS rows
FROM sys.dm_db_partition_stats
WHERE object_id = OBJECT_ID(#tableName)
AND index_id < 2
GROUP BY OBJECT_NAME(object_id);
Now, let's say I have 50 tables, like data_1950, data_1951, .....data_2000. I wrote a batch, query each table's records count, and put them into a temporary table. It works like a charm
CREATE TABLE #Temp
(
TableName varchar(30),
RecordsCount int
)
DECLARE #sql as varchar(max)
DECLARE #yearN as int = 1950
DECLARE #tbName as sysname
WHILE #yearN <= 2000
BEGIN
SET #tbName = QUOTEName(N'[dbo].data_' + Convert(varchar,#yearN))
SET #sql = N'Exec [dbo].FastCount #tableName=' + #tbName
INSERT INTO #Temp
EXEC (#sql)
SET #yearN = #yearN + 1
END
SELECT * FROM #Temp
DROP TABLE #Temp
However, if I replace the dynamic SQL string part
SET #sql = N'Exec [dbo].FastCount #tableName=' + #tbName
INSERT INTO #Temp
EXEC (#sql)
with a straightforward call
INSERT INTO #Temp
EXEC [dbo].FastCount #tableName = #tbName
Then the whole batch just not work...
So I don't understand why... Should I always use dynamic SQL string and exec(#sql) when programmatically using the stored procedure. A big thanks for taking the time to look.
OK, here is what is happening in the two scenarios that you posed in your original question. (Yes, the reality is that there are probably better ways to achieve your end result, but let's look at the actual problem that you posed .... why is the behaviour of your INSERT / EXEC different, depending on how you made the call).
First, you have your variable declared, that will contain your table name:
DECLARE #tbName as sysname
Then you have your looping block that incrementally increases the year number, to generate the different table names. There's nothing inherently wrong with the looping block, so let's just look at an example using one of the table names, to see what's happening within the WHILE block. Take the first table name as the example, which would be [dbo].data_1950.
Your statement:
set #tbName = QUOTEName(N'[dbo].data_' + Convert(varchar,#yearN))
ultimately takes the string "[dbo].data_1950" - which comes from concatenating '[dbo].data_' with the year number (in this case, 1950) converted to a string (varchar) - and passes it to the QUOTENAME() function. The QUOTENAME() function takes its input and a second parameter, which is the character that the input should be quoted with (if the 2nd parameter is not passed, then the default is []). Thus, if we then converted the #tbName variable to a string, it would appear like this:
[[dbo].data_1950]
Now we get to see the funky way that SQL deals with "sysname" data-types. (In fact, as you read further down, maybe the issue is not primarily tied to the "sysname" data-type, but anyhow, take away from this what you will). To be honest, "sysname" is, in itself, a little bit of a funky data-type anyway, which I tend to steer away from, unless absolutely necessary. But anyhow, on to the details of the issue that you were seeing.
Step 1 - I created a version of your stored proc, but I included a statement that would output the value of the #tableName parameter that was passed in. This gives us an opportunity to see what SQL is doing in the two different scenarios, and then explain why the results are different.
CREATE PROC [dbo].FastCount
(
#tableName varchar(100)
)
AS
BEGIN
PRINT #tableName;
SELECT OBJECT_NAME(object_id), SUM(row_count) AS rows
FROM sys.dm_db_partition_stats
WHERE object_id = OBJECT_ID(#tableName)
AND index_id < 2
GROUP BY OBJECT_NAME(object_id);
END
Step 2 - our first scenario is executing the dynamic SQL.
set #tbName = QUOTEName(N'[dbo].data_' + Convert(varchar,#yearN))
set #sql = N'Exec [dbo].FastCount #tableName=' + #tbName
Insert Into #Temp Exec(#sql)
Now, we know that the #tbName variable contains
[[dbo].data_1950]
and therefore we can then infer that the #sql variable contains
Exec [dbo].FastCount #tableName=[[dbo].data_1950]
so that is effectively the statement that is executed by the Exec(#sql) command.
When this runs, and we look at the output of the PRINT command, we see
[dbo].data_1950
and we see a result from our query (the table name and row count). This makes sense, of course, because our table name is "data_1950", and the schema of the table is "dbo", so the SELECT statement to get the row count is going to work as expected.
Step 3 - run the EXEC command directly, without the use of the #sql variable, ie.
Insert Into #Temp Exec [dbo].FastCount #tableName = #tbName
Now, when we look at the output of the PRINT command for this execution of the "FastCount" stored procedure, we see
[[dbo].data_1950]
Of course, this is now NOT going to produce the results that we expect, because we're telling SQL to find the row count for a table named "[dbo].data_1950" (in the absence of the specific schema, SQL will just assume the default schema. In this case, with a schema of [dbo], we'd be telling SQL to get the row count from a table named [dbo].[[dbo].data_1950] - which is clearly NOT the table name).
You should see the obvious difference - in one scenario, the parameter value that is passed into the stored is the "correct" reference to the table name, and in the other scenario it is not.
As a final step, let's look at how the "non-dynamic" SQL would be executed, to achieve the results that we need. In this instance, there's no need for the QUOTENAME() function:
set #tbName = N'[dbo].data_' + Convert(nvarchar,#yearN)
Insert Into #Temp Exec [dbo].FastCount #tableName = #tbName
When we run it in this way, we see the expected output ([dbo].data_1950) from the PRINT command, and we see the expected query results (containing the table name and row count).
Can I explain this behaviour, exactly? Errr, not necessarily ... maybe someone else will be able to explain specifically what is happening, and why. My only interpretation is that when the EXEC() statement is passed the dynamic sql (ie. #sql variable) it is first interpreting the entire string and stripping out identifiers (in the case, the surrounding [] ... on what basis is it making that decision, I don't know). As opposed to the non-dynamic execution, where the #tbName value ([[dbo].data_1950]) is just being passed straight in as the parameter, with no modification (and thus causing the unexpected end result that we saw).
Hopefully this information is useful to you (or, at least, to someone else in the future!).
In general you should avoid dynamic SQL, and you should avoid granting rights to execute dynamic SQL, unless absolutely necessary. This is for performance and security reasons.
The typical way to deal with this situation is to use a partitioned view:
CREATE VIEW DataView
AS
SELECT '1950' TableName, * FROM Data_1950
UNION ALL
SELECT '1951' TableName, * FROM Data_1951
UNION ALL
SELECT '1952' TableName, * FROM Data_1952
UNION ALL
SELECT '1953' TableName, * FROM Data_1953
UNION ALL
SELECT '1954' TableName, * FROM Data_1954
UNION ALL
SELECT '1955' TableName, * FROM Data_1955
(Keep adding select statements until you have covered all of your tables.)
Now to get your table counts all you need to do is execute this:
SELECT TableName, COUNT(*) RecordCount
FROM DataView
GROUP BY TableName
Isn't that much easier?
I have a stored procedure for selecting rows. I want to pass a parameter to filtering rows dynamically like this :
Create Procedure CustomerSelectAll
#FilterExpresion NVARCHAR(MAX)
DECLARE #CMD NVARCHAR(MAX)
SET #CMD = N'SELECT * FROM dbo.Customers '+#FilterExpresion;
EXEC(#CMD)
The above code works fine, but it is at risk for SQL injection, so I want to be able pass multiple columns with any WHERE statement such as:
exec CustomerSelectAll
#FilterExpresion = N' where Name = 'abc' and family = ''xyz'''
I am not aware if you can pass the entire where clause as parameter.But from the little amount of knowledge I have, I can suggest you use a sql parser to see if the where clause has just select statements and if the answer is affirmative then you can pass the where clause to your stored procedure. Hope this is of some help.
please help me with writing this search sql stored procedure
procedure may have different number of parameters at different time
so could any body help me with writing this query. I don't know how to concatenate parameters.
i am new to stored procedure
CREATE PROCEDURE searchStudent
-- Add the parameters for the stored procedure here
#course int=null,
#branch int=null,
#admissionYear varchar(max)=null,
#passingYear varchar(max)=null,
#userName varchar(max)=null,
#sex varchar(max)=null,
#studyGap varchar(max)=null,
#firstName varchar(max)=null,
#lastName varchar(max)=null
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
DECLARE query STR DEFAULT null
IF #course IS NOT NULL
THEN query=
SELECT * FROM [tbl_students] WHERE
END
GO
please complete the query so that it can have parameters which are having values and can search from database on the basis of parameters value. But parameter may vary every time depends on search criteria.
You would probably need to use Dynamic SQL to achieve this. First of all I would highly recommend reading this excellent article. http://www.sommarskog.se/dynamic_sql.html
You're dynamic sql would be something like this;
Declare #query varchar(max)
Set #query = 'Select * From dbo.MyTable Where '
If #Course Is Not Null
Begin
Set #query = #query + 'Course = ' + Convert(varchar(10), #Course)
end
If #Branch Is Not Null
Begin
Set #query = #query + ' and Branch = ' + Convert(varchar(10), #Branch )
end
This is only an example! You will need to build in some checks to ensure that you have one (and only one) Where clause, you must ensure that the integer values are converted to string values correctly. You must also check that the parameters don't have any special characters that could break the dynamic sql - like an apostrophe (')
Using dynamic SQL can be painful and very difficult to get right.
Good luck!
The key with a dynamic search conditions is to make sure an index is used, instead of how can I easily reuse code, eliminate duplications in a query, or try to do everything with the same query. Here is a very comprehensive article on how to handle this topic:
Dynamic Search Conditions in T-SQL by Erland Sommarskog
It covers all the issues and methods of trying to write queries with multiple optional search conditions. This main thing you need to be concerned with is not the duplication of code, but the use of an index. If your query fails to use an index, it will preform poorly. There are several techniques that can be used, which may or may not allow an index to be used.
here is the table of contents:
Introduction
The Case Study: Searching Orders
The Northgale Database
Dynamic SQL
Introduction
Using sp_executesql
Using the CLR
Using EXEC()
When Caching Is Not Really What You Want
Static SQL
Introduction
x = #x OR #x IS NULL
Using IF statements
Umachandar's Bag of Tricks
Using Temp Tables
x = #x AND #x IS NOT NULL
Handling Complex Conditions
Hybrid Solutions – Using both Static and Dynamic SQL
Using Views
Using Inline Table Functions
Conclusion
Feedback and Acknowledgements
Revision History
Sorry, I am having trouble understanding what you are asking. Do you mean the consumer of the sproc may specify some arbitrary subset of the parameters and you want to filter on those?
Assuming the above you have 2 options.
1.
use a where clause something like this:
WHERE ([tbl_students].firstName = ISNULL(#firstname,firstName)
AND ([tbl_students].lastName = ISNULL(#lastName ,lastName )
etc.
What this does is check if your parameter has a value, and, if so, it will compare it to the column. If the param is null, then it will compare the column to itself, which will never filter anything out.
use dynamic sql in your sproc and just include the line of the where clause you want if the param is not null.
I am trying to write a database script (SQL Server 2008) which will copy information from database tables on one server to corresponding tables in another database on a different server.
I have read that the correct way to do this is to use a sql statement in a format similar to the following:
INSERT INTO <linked_server>.<database>.<owner>.<table_name> SELECT * FROM <linked_server>.<database>.<owner>.<table_name>
As there will be several tables being copied, I would like to declare variables at the top of the script to allow the user to specify the names of each server and database that are to be used. These could then be used throughout the script. However, I am not sure how to use the variable values in the actual SQL statements. What I want to achieve is something like the following:
DECLARE #SERVER_FROM AS NVARCHAR(50) = 'ServerFrom'
DECLARE #DATABASE_FROM AS NVARCHAR(50) = 'DatabaseTo'
DECLARE #SERVER_TO AS NVARCHAR(50) = 'ServerTo'
DECLARE #DATABASE_TO AS NVARCHAR(50) = 'DatabaseTo'
INSERT INTO #SERVER_TO.#DATABASE_TO.dbo.TableName SELECT * FROM #SERVER_FROM.#DATABASE_FROM.dbo.TableName
...
How should I use the # variables in this code in order for it to work correctly?
Additionally, do you think my method above is correct for what I am trying to achieve and should I be using NVARCHAR(50) as my variable type or something else?
Thanks
There is probably a better way to do this, but what you are probably trying to do in your example is what's called dynamic SQL where you treat the statement as a string and the execute it. This would be section #2 here:
http://www.mssqltips.com/tip.asp?tip=1160
There are some major downsides to dynamic SQL. You see a couple other approaches that might be better in that article.
If you want to execute a dynamically generated query then you have to use sp_ExecuteSQL
HTH
For the nvarchar(50) - you'd be better using sysname. This is a synonym in SQL Server (for nvarchar(128)) and represents the maximum length of an object identifier.
have a look at http://msdn.microsoft.com/en-us/library/ms188001.aspx - sp_executesql takes a parameter that is a string and executes the sql in that string. so you'd need to concatenate #SERVER_FROM and other params with the INSERT INTO part to make the entire sql statement, and then pass to sp_executesql.
nvarchar(50) is fine, unless your server/database names are longer than that :)
You can create the select statement by concatenating all the information together and then use sp_executesql
so:
sp_executesql 'INSERT INTO ' + #SERVER_TO + '.' + #DATABASE_TO +
'.dbo.TableName SELECT * FROM ' + #SERVER_FROM + '.' +
#DATABASE_FROM+'.dbo.TableName'
I like to make templates for dynamic SQL things like this - it's a lot easier to maintain complex statements and also sometimes easier to handle nested quotes - and definitely easier when terms need to be repeated in multiple places (like column lists):
DECLARE #sql AS nvarchar(max);
SET #sql = 'INSERT INTO {#SERVER_TO}.{#DATABASE_TO}.dbo.TableName
SELECT *
FROM {#SERVER_FROM}.{#DATABASE_FROM}.dbo.TableName'
SET #sql = REPLACE(#sql, '{#SERVER_TO}', QUOTENAME(#SERVER_TO))
SET #sql = REPLACE(#sql, '{#DATABASE_TO}', QUOTENAME(#DATABASE_TO))
SET #sql = REPLACE(#sql, '{#SERVER_FROM}', QUOTENAME(#SERVER_FROM))
SET #sql = REPLACE(#sql, '{#DATABASE_FROM}', QUOTENAME(#DATABASE_FROM))
EXEC(#sql)
If I have this stored proc definition minus the body
ALTER PROCEDURE sp_AlloctionReport(#where NVARCHAR(1000), #alldate NVARCHAR(200), #alldateprevweek NVARCHAR(200))
AS
And I call like this.
sp_AllocationReport "ProductCode = 'FA' AND AllocationDate = '20090112' AND VenueInfo.VenueID In(SELECT vf.VenueID FROM VenueFilters vf INNER JOIN FilterTypes ft ON vf.FilterTypeID = ft.FilterTypeID WHERE ft.FilterDescription = 'Coke') AND State = 'NSW'","CampaignAllocations.AllocationDate = '20090112'","CampaignAllocations.AllocationDate = '20090105'"
Why do I get this error when my first argument is defined NVARCHAR(1000).
The identifier that starts with 'ProductCode = 'FA' AND AllocationDate = '20090112' AND VenueInfo.VenueID In(SELECT vf.VenueID FROM VenueFilters vf INNER JOIN Fi' is too long. Maximum length is 128.
Take the where clause and edit it in a new file
Replace ' with '' (single quote -> double single quote)
EXEC dbo.sp_AllocationReport #where= '<THE TEXT EDITED ABOVE>'
The maximum length of a object in sql server in 128 chars (I think). Perhaps a syntax problem when you build the query has made it think that the whole string is a table/view name?
You must use single quotes instead of double quotes unless the connection has QUOTED_IDENTIFIER turned ON
http://msdn.microsoft.com/en-us/library/ms174393.aspx
This is the default from SQL 2000 onwards, but if you've upgraded from an older version then the old default will still be active.
If we had the body of the stored proc, it might be clearer. But ... what it looks like is that SQL Server is interpreting your parameter as an indentifer (i.e., column name). The clue is the error message which states "The identifier that starts with 'ProductCode = 'FA' AND [...] is too long". i.e., SQL Server is looking for a column named "ProductCode = 'FA' AND [...etc...]"
So what I suspect is you've done this in within the stored proc:
SELECT col1, col2, col3, ... FROM table WHERE #where
...and your hoping the where clause to work just like that.
Assuming this is what you've done, it won't work. If this isn't what you've done, the rest of this answer may be completely bogus :-) If you can give an example of the body of the sproc, it might make things clearer.
So assuming my suspision is correct, you need to write it as a dynamic SQL statement like this:
DECLARE #sql NVARCHAR(2000)
SET #sql = 'SELECT col1, col2, col3, ... FROM table WHERE ' + #where
EXEC sp_ExecuteSQL #sql
HOWEVER ... even this isn't the end of the story as this is prone to injection attacks, which are a very bad thing. What you're better off doing is changing the params to your stored proc to make use of parameterised SQL which won't be prone to injection attacks. Something like this...
ALTER PROCEDURE sp_AllocationReport (#ProductCode VARCHAR(10), #AllocationDate DATETIME, {rest of your parameters})
AS
DECLARE #sql NVARCHAR(2000)
SET #sql = 'SELECT col1, col2, col3, ... FROM table WHERE 1 = 1'
IF ISNULL(#ProductCode, '') <> ''
SET #sql = #sql + ' AND ProductCode = #pProductCode'
IF #AllocationDate IS NOT NULL
SET #sql = #sql + ' AND AllocationDate = #pAllocationDate'
{other conditionals, depending on what you need to pass in}
EXEC sp_ExecuteSQL #sql, '#pProductCode VARCHAR(10),
#pAllocationDate DATETIME,
{other passed in params}
', #ProductCode, #AllocationDate
This code isn't prone to injection attacks. It's also more performant as SQL Server will cache execution plans more reliably. Read up about this; there's plenty out there on it.
Check your double quotes around "Coke". The syntax highligher makes the error obvious.
Use single quote for the main string, and double escape them where needed. Yes, it's a pain, but it's the right way to do it.
Prefix NVarchar literals with a capital N
This smells of an sql injection hack waiting to happen. Are you sure you need to pass sql around as string variables?