Is it possible to create a parameterized SQL statement that will taken an arbitrary number of parameters? I'm trying to allow users to filter a list based on multiple keywords, each separated by a semicolon. So the input would be something like "Oakland;City;Planning" and the WHERE clause would come out something equivalent to the below:
WHERE ProjectName LIKE '%Oakland%' AND ProjectName Like '%City%' AND ProjectName Like '%Planning%'
It's really easy to create such a list with concatenation, but I don't want to take that approach because of the SQL injection vulnerabilities. What are my options? Do I create a bunch of parameters and hope that users never try to use more parameters that I've defined? Or is there a way to create parameterized SQL on the fly safely?
Performance isn't much of an issue because the table is only about 900 rows right now, and won't be growing very quickly, maybe 50 to 100 rows per year.
A basic proof-of-concept... Actual code would be less, but since I don't know your table/field names, this is the full code, so anyone can verify it works, tweak it, etc.
--Search Parameters
DECLARE #SearchString VARCHAR(MAX)
SET #SearchString='Oakland;City;Planning' --Using your example search
DECLARE #Delim CHAR(1)
SET #Delim=';' --Using your deliminator from the example
--I didn't know your table name, so I'm making it... along with a few extra rows...
DECLARE #Projects TABLE (ProjectID INT, ProjectName VARCHAR(200))
INSERT INTO #Projects (ProjectID, ProjectName) SELECT 1, 'Oakland City Planning'
INSERT INTO #Projects (ProjectID, ProjectName) SELECT 2, 'Oakland City Construction'
INSERT INTO #Projects (ProjectID, ProjectName) SELECT 3, 'Skunk Works'
INSERT INTO #Projects (ProjectID, ProjectName) SELECT 4, 'Oakland Town Hall'
INSERT INTO #Projects (ProjectID, ProjectName) SELECT 5, 'Oakland Mall'
INSERT INTO #Projects (ProjectID, ProjectName) SELECT 6, 'StackOverflow Answer Planning'
--*** MAIN PROGRAM CODE STARTS HERE ***
DECLARE #Keywords TABLE (Keyword VARCHAR(MAX))
DECLARE #index int
SET #index = -1
--Each keyword gets inserted into the table
--Single keywords are handled, but I did not add code to remove duplicates
--since that affects performance only, not the result.
WHILE (LEN(#SearchString) > 0)
BEGIN
SET #index = CHARINDEX(#Delim , #SearchString)
IF (#index = 0) AND (LEN(#SearchString) > 0)
BEGIN
INSERT INTO #Keywords VALUES (#SearchString)
BREAK
END
IF (#index > 1)
BEGIN
INSERT INTO #Keywords VALUES (LEFT(#SearchString, #index - 1))
SET #SearchString = RIGHT(#SearchString, (LEN(#SearchString) - #index))
END
ELSE
SET #SearchString = RIGHT(#SearchString, (LEN(#SearchString) - #index))
END
--This way, only a project with all of our keywords will be shown...
SELECT *
FROM #Projects
WHERE ProjectID NOT IN (SELECT ProjectID FROM #Projects Projects INNER JOIN #Keywords Keywords ON CHARINDEX(Keywords.Keyword,Projects.ProjectName)=0)
I decided to mix a few different answers together into one :-P
This assumes you'll pass in a delimited string list of search keywords (passed in via #SearchString) as a VARCHAR(MAX), which -- realistically -- you won't run into a limit on for keyword searches.
Each keyword is broken down from the list and added into a keyword table. You'd probably want to add code to remove out duplicate keywords, but it won't hurt in my example. Just slightly less effective, since we only need to evaluate once per keyword, ideally.
From there, any keyword that isn't a part of the project name removes that project from the list...
So searching for "Oakland" gives 4 results but "Oakland;City;Planning" gives only 1 result.
You can also change the delimiter, so instead of a semi-colon, it can use a space. Or whatever floats your boat...
Also, because of the joins and what not instead of Dynamic SQL, it doesn't run the risk of SQL Injection like you were worried about.
You might also want to consider Full Text Search and using CONTAINS or CONTAINSTABLE for a more "natural" search capability.
May be overkill for 1K rows, but it is written and is not easily subverted by injection.
The trick would usually to simply pass the list as a string separated by comas (csv style), parse that string in a loop and dynamically build the query.
Above post is also right that maybe the best approach is not T-SQL but the business/application layer.
What about using an XML data type to contain the parameters?
It can be unbounded and assembled at run time...
I pass in an unknown number of PKs for a table update then pump them into a temp table.
It is easy to then update where PK in PKTempTable.
Here is the code to parse the XML data type...
INSERT INTO #ERXMLRead (ExpenseReportID)
SELECT ParamValues.ID.value('.','VARCHAR(20)')
FROM #ExpenseReportIDs.nodes('/Root/ExpenseReportID') as ParamValues(ID)
using a tool like NHibernate will allow you to dynamically construct your queries safely without the need for stored procedures.
Frans Bouma has an excellent article about stored procs vs dynamic sql and what some of the benefits of using an SQL generator are over using hand generated statements
If you use stored procs you can include a default value for the parameters, then you can elect to pass them or not pass them in client code, but you still have have to declare them individually in the stored procedure... Also only if you're using a stored proc, you can pass a single parameter as a delimited string of values, and parse out the individual values inside the sproc (There are some "standard" T-SQL functions available that will split out the records into a dynamic table variable for you)
If your using SQL server 2008 check out this artical passing a table valued parameter
Whatever way you go, watch out for SQL Server's parameter limit: ~2000 parameters.
Similar to some of the other answers, you can parse out a delimited string or an XML document. See this excellent link which demonstrates both methods with SQL Server.
Related
I have 2 tables
Account(AccountId, Encoding)
DeviceAccountMap(AccountId, DeviceId)
Now I need to fetch the devices from the DeviceAccountMap. I pass a list of AccountId to a stored procedure and while fetching the DeviceId from the DeviceAccountMap table I need to compare the Encoding value for each account with a particular value.
Which is the easy way to do this? I am totally lost.
The select clause in the stored procedure will look something like this:
DECLARE #Accounts [usp].[Array]
and [usp].[Array] is defined as below
CREATE TYPE [usp].[Array] AS TABLE
(
Value VARCHAR(36) NULL
)
SELECT
DeviceId,
AccountEncoding = A.Encoding
FROM
usp.DeviceControllerAccountMap DCAM
INNER JOIN
usp.Account A ON (DCAM.AccountId = A.AccountId)
WHERE
DCAM.AccountId IN (SELECT Value From #AccountIds)
AND DCAM.IsShared = 1
AND AccountEncoding LIKE A.Encoding + '.%'
In other words I need to fetch the encoding value for each account and use that in this where clause.
So you can look up information on Table-Valued Parameters (TVPs) in T-SQL.
Here is an article by Erland Sommarskog.
You can refer to this StackOverflow answer to see an example of C# code calling a stored procedure that uses a TVP. I believe TVPs require SQL Server 2008 or higher.
TVPs, as far as I understand, provide a way to make your own data type in sql server that gets treated as if it was a table. You're doing this when you declare your Array type and then when you use the #AccountIds in your stored procedure's select statement.
CREATE TYPE [usp].[Array] AS TABLE -- maybe choose a more descriptive name than 'Array'
(
Value VARCHAR(36) NULL -- choose a more descriptive name than 'Value'
)
CREATE PROCEDURE [usp].[your_procedure_name]
#AccountIds [usp].[Array] READONLY -- use TVP as a parameter
AS
SELECT …
It is not clear form your question details whether you also mean to have a parameter in the stored procedure for the Encoding. It seems like you're looking for accounts whose Encodings start with a period '.'.
So first, create your type, like you're doing.
Then create your stored procedure.
Then test your stored procedure, something like this:
DECLARE #mylist Array -- make TVP sample data
INSERT #mylist(Value) VALUES(1),(11),(27),(123) -- insert some values
exec your_procedure_name #mylist -- run stored procedure
The following line is completely unnecessary. The JOIN to Account does this filter for you.
DCAM.AccountId IN (SELECT Value From #AccountIds)
Or am I missing something?
I am using a stored procedure to insert records into a table. And do this at least 12 times in a loop to insert multiple records which is very inefficient.
here is the procedure as CREATED
Create PROC [dbo].[SP_INSERT_G_SAMPLING]
#GameID INT,
#ScoreID INT
as
begin
INSERT INTO GAMESCORE (GAMEID, SCOREID) VALUES
(#GameID, #ScoreID)
end
I pass on the values ex(1,3) and loop with more values from the website.
I want to however pass on all the values at one time like (1,3),(4,5),(8,9)
and then alter the above procedure to receive and insert multiple rows.
ALTER PROC [dbo].[SP_INSERT_G_SAMPLING]
#totalinsert nvarchar(Max)
INSERT INTO GAMESCORE (GAMEID, SCOREID) VALUES
(#totalinsert)
with #totalinsert being like (1,3),(4,5),(8,9) pushed from the webpage.
any help is greatly appreciated
What you're going to have to do is write a table valued function which accepts the multi-value string and breaks it out into a table object. If you can change your source to use a record delimiter instead of having comma sets it would be slightly easier to process. An example of that would look like this.
The below is pure psuedo and has not been validated in any way, just meant to give you a rough idea of where to go.
ex: #TotalInsert = 1,2|4,5|8,9
DECLARE #Results TABLE
(
value1 INT,
value2 INT
)
DECLARE #setlist VARCHAR(max);
WHILE Len(#TotalInsert) > 0
BEGIN
SET #setlist = LEFT(#totalinsert, Charindex('|', #totalinsert))
INSERT INTO #results
SELECT LEFT(#setlist, Charindex(',', #setlist) - 1),
RIGHT(#setlist, Charindex(',', Reverse(#setlist)) + 1)
SET #totalinsert = RIGHT(#totalinsert, Len(#totalinsert) - Len(#setlist))
END
I'm assuming you're using .NET for your website since you're also using SQL Server.
Have a look at table valued parameters, this page also includes a nice example of how to use the table valued parameters in .NET.
Check here for a better example of making a stored procedure with a table valued parameter in T-SQL.
Here is the full discussion:
http://www.sommarskog.se/arrays-in-sql-2005.html#XMLlist%20of%20values
Personally, I sent xml to the stored procedure, I "shred it" into #variable or #temp tables, then I do my INSERT/UPDATE/MERGE/DELETE from there.
Here is a fuller discussion on xml-shredding.
http://pratchev.blogspot.com/2007/06/shredding-xml-in-sql-server-2005.html
My personal trick is to create a strong dataset, populate the strong dataset with rows, and use the ds.GetXml() to send the xml down to the TSQL. With a strong dataset, I get strong-typing when populating the values. But at the end of the day, dataset is just some super fancy xml.
CREATE PROCEDURE Testing1
#Varaible nvarchar(50),
#value integer
AS
BEGIN
DECLARE #SUMM FLOAT
SET #SUMM=(#value*2.38/7.456)*2
PRINT #Varaible
PRINT 'EXPENSES IS'
PRINT #SUMM
END
Output is:
PETER
EXPENSES IS
24.2597
The above is my code where I am passing a single input parameter.
If I want to pass multiple values like peter,robber,licoln,mat
#varaible peter,robber,licoln,mat
#value 37 45 66 77
is it possible in SQL??
If you're only sending a few values in your delimited strings, I would suggest simply using the proper datatypes and calling the stored procedure a few times with individual values.
If, however, your delimited strings might contain hundreds or thousands of discrete values, then calling a proc that many times could be costly in terms of performance, especially if you can't send them all in one batch (I'm sure you want to use parameters rather than a giant concatenated command). If this is the case, you have a few options:
Use Table Valued Parameters. This is like passing arrays as arguments to your proc.
Pass XML to your proc, then shred and process it within the procedure.
Insert your data to staging/temp tables, then call the procedure to operate on those tables.
Take a step back and see if it makes sense to do more processing in your app. DB code usually doesn't scale nearly as well as app code.
Send these delimited strings to your proc, split/parse them, then loop over the results in SQL. This seems to be what you're asking about and is possibly the least elegant option, even though it's one of the more popular ways to abuse a relational database.
The table valued parameters approach seems very 'approachable', but is only available as of MSSQL 2008. If you are still stuck with MSSQL 2005 then, maybe, the temporary table approach works best for you?
Your code could look something like:
-- define your stored procedure
CREATE PROC sptest1 AS
BEGIN
-- do some stuff with #tmp, like join it to other tables
-- and UPDATE values in these tables with it!
-- or simply list a processed version of #tmp:
SELECT nam,val*(2.38/7.456)*2 FROM #tmp
END
-- prepare input values by creating a temporary table on the fly
SELECT 'Peter' nam,23 val INTO #tmp
UNION ALL SELECT 'Paul',27
UNION ALL SELECT 'Harry',16
UNION ALL SELECT 'Mary-Ann',45;
-- and call the procedure:
EXEC sptest1
So, your frontend will have to build the SELECT ... INTO #tmp ... string. After that the rest of the processing can be done inside your stored procedure.
I've just begun to learn how to write stored procedures and SQL code outside of the basic DML stuff. Something that I've recently come across is table value parameters. I've found a script to make a TVP and it works just fine but there are two things that I don't understand about it. One, when to use them. What's a typical real world scenario when a TVP would be beneficial. Two, how come when I remove the begin and end from the following script does it work the same; what's the difference between having those keywords and not? SQL Server 2008 R2
use [tempdb]
--this is the database table that will be populated
create table SampleTable
(
id int not null identity (1,1)
,SampleString varchar(50)
,SampleInt int null
)
go
--create the table data type
create type dbo.SampleDataType as table
(
SampleString varchar(50)
,SampleInt int
)
go
--the stored procedure takes the SampleDataType as an input parameter
create proc SampleProc
(
--accepts the TVP as the lone parameter
--make sure that the TVP is readonly
#Sample as dbo.SampleDataType readonly
)
as
begin
--we simply insert the values into the db table from the parameter
insert into SampleTable(SampleString,SampleInt)
select SampleString,SampleInt from #Sample
end
go
--this is the sample script to test that the sproc worked
declare #SampleData as dbo.SampleDataType
insert into #SampleData(SampleString,SampleInt) values ('one',1);
insert into #SampleData(SampleString,SampleInt) values ('two',2);
select * from #SampleData
One real world use is to parameterise an in clause.
Where a query has a filter on (x, y, z, ...) you no longer have to resort to one of the methods here such as passing it in as a comma delimited list and then splitting it.
The BEGIN ... END make no difference there. It defines a block. You might use that after an IF statement for example to group multiple statements together into one logical block.
I would like to create a SP or UDF where I supply a table and column name as a parameter and it does something to that target. I'm using Sql Server 2005
Trivial Example of what I'm trying to accomplish:
CREATE FUNCTION Example (#TableName AS VARCHAR(100))
RETURNS TABLE
AS
BEGIN
SELECT *
INTO #temp
FROM #TableName
RETURN #temp
END
The example is just something trivial to illustrate what I'm trying to accomplish in terms of passing the Table name as a parameter.
Is this possible to do w/o concatinating strings and calling the EXEC function?
Ultimately, I'm trying to convert the answer from this question into something reusable.
This reeks of SQL injection. You would still need to use EXEC to do this.
No. Can't do it. Sadly, there is no macro pre-complier in T-SQL. The closest you'll get is SQLCMD mode, but that's only for scripts, can't use it in object definitions.
Are you doing the same thing to the table each time?
You could dynamically redefine a synonym, but that still requires an EXEC and you lose concurrency. You could serialize execution with a queue, but at that point you may be better off w/ plain old dynamic SQL.
You might try temporary tables, not passed in as a variable, but created in the parent connection or calling procedure. eg.
create proc #proc as
select * from #table
go
create table #table (col1 int)
insert #table values (1)
insert #table values (2)
insert #table values (3)
go
exec #proc
go
For more ways to share data between stored procedures, see here: http://www.sommarskog.se/share_data.html