How hard would you try to make your SQL queries secure? - sql

I am in a situation where I am given a comma-separated VarChar as input to a stored procedure. I want to do something like this:
SELECT * FROM tblMyTable
INNER JOIN /*Bunch of inner joins here*/
WHERE ItemID IN ($MyList);
However, you can't use a VarChar with the IN statement. There are two ways to get around this problem:
(The Wrong Way) Create the SQL query in a String, like so:
SET $SQL = '
SELECT * FROM tblMyTable
INNER JOIN /*Bunch of inner joins here*/
WHERE ItemID IN (' + $MyList + ');
EXEC($SQL);
(The Right Way) Create a temporary table that contains the values of $MyList, then join that table in the initial query.
My question is:
Option 2 has a relatively large performance hit with creating a temporary table, which is less than ideal.
While Option 1 is open to an SQL injection attack, since my SPROC is being called from an authenticated source, does it really matter? Only trusted sources will execute this SPROC, so if they choose to bugger up the database, that is their prerogative.
So, how far would you go to make your code secure?

What database are you using? in SQL Server you can create a split function that can split a long string and return a table sub-second. you use the table function call like a regular table in a query (no temp table necessary)
You need to create a split function, or if you have one just use it. This is how a split function can be used:
SELECT
*
FROM YourTable y
INNER JOIN dbo.yourSplitFunction(#Parameter) s ON y.ID=s.Value
I prefer the number table approach to split a string in TSQL but there are numerous ways to split strings in SQL Server, see the previous link, which explains the PROs and CONs of each.
For the Numbers Table method to work, you need to do this one time table setup, which will create a table Numbers that contains rows from 1 to 10,000:
SELECT TOP 10000 IDENTITY(int,1,1) AS Number
INTO Numbers
FROM sys.objects s1
CROSS JOIN sys.objects s2
ALTER TABLE Numbers ADD CONSTRAINT PK_Numbers PRIMARY KEY CLUSTERED (Number)
Once the Numbers table is set up, create this split function:
CREATE FUNCTION [dbo].[FN_ListToTable]
(
#SplitOn char(1) --REQUIRED, the character to split the #List string on
,#List varchar(8000)--REQUIRED, the list to split apart
)
RETURNS TABLE
AS
RETURN
(
----------------
--SINGLE QUERY-- --this will not return empty rows
----------------
SELECT
ListValue
FROM (SELECT
LTRIM(RTRIM(SUBSTRING(List2, number+1, CHARINDEX(#SplitOn, List2, number+1)-number - 1))) AS ListValue
FROM (
SELECT #SplitOn + #List + #SplitOn AS List2
) AS dt
INNER JOIN Numbers n ON n.Number < LEN(dt.List2)
WHERE SUBSTRING(List2, number, 1) = #SplitOn
) dt2
WHERE ListValue IS NOT NULL AND ListValue!=''
);
GO
You can now easily split a CSV string into a table and join on it:
select * from dbo.FN_ListToTable(',','1,2,3,,,4,5,6777,,,')
OUTPUT:
ListValue
-----------------------
1
2
3
4
5
6777
(6 row(s) affected)
Your can use the CSV string like this, not temp table necessary:
SELECT * FROM tblMyTable
INNER JOIN /*Bunch of inner joins here*/
WHERE ItemID IN (select ListValue from dbo.FN_ListToTable(',',$MyList));

I would personally prefer option 2 in that just because a source is authenticated, does not mean you should be letting your guard down. You would leave yourself open to potential rights escalations where an authenticated low lvl user, is able to still execute commands against the database you had not intended.
The phrase you use of 'trusted sources' - it might be better if you assume an X-Files aproach and to trust no-one.

If someone buggers up the database you might still be getting a call.
A good option that is similar to option two is to use a function to create a table in memory from the CSV list. It is reasonably fast and offers the protections of option two. Then that table can be joined to the Inner Join, e.g.
CREATE FUNCTION [dbo].[simple_strlist_to_tbl] (#list nvarchar(MAX))
RETURNS #tbl TABLE (str varchar(4000) NOT NULL) AS
BEGIN
DECLARE #pos int,
#nextpos int,
#valuelen int
SELECT #pos = 0, #nextpos = 1
WHILE #nextpos > 0
BEGIN
SELECT #nextpos = charindex(',', #list, #pos + 1)
SELECT #valuelen = CASE WHEN #nextpos > 0
THEN #nextpos
ELSE len(#list) + 1
END - #pos - 1
INSERT #tbl (str)
VALUES (substring(#list, #pos + 1, #valuelen))
SELECT #pos = #nextpos
END
RETURN
END
Then in the join:
tblMyTable INNER JOIN
simple_strlist_to_tbl(#MyList) list ON tblMyTable.itemId = list.str

Option 3 is to confirm each item in the list is in fact an integer before concatenating the string to your SQL statement.
Do this by parsing the input string (e.g., split into an array), loop through and convert each value to an int, and then recreate the list yourself before concatenating back to the SQL statement. This will give you reasonable assurance that SQL injection cannot occur.
It is safer to concatenate strings that have been created by your application, because you can do things like check for int, but it also means your code is written in a way that a subsequent developer may modify slightly, thus opening back up the risk of SQL injection, because they do not realize that is what your code is protecting against. Make sure you comment well what you are doing if you go this route.

A third option: pass the values to the stored procedure in an array. Then you can either assemble the comma-separated string in your code and use the dynamic SQL option, or (if your flavour of RDBMS permits it) use the array directly in the SELECT statement.

Why don't You write an CLR split function, that will do all the job nice and easy? You can write user Defined table functions which will return a table doing string splitting with .Net infructure. Hell in SQL 2008 you can even give them hints if they return the strings sorted in any way... like ascending or something which can help the optimizer?
Or maybe You can't do CLR integration then You have to stick to the tsql but I personally would go for the CLR soluton

Related

SQL How to Split One Column into Multiple Variable Columns

I am working on MSSQL, trying to split one string column into multiple columns. The string column has numbers separated by semicolons, like:
190230943204;190234443204;
However, some rows have more numbers than others, so in the database you can have
190230943204;190234443204;
121340944534;340212343204;134530943204
I've seen some solutions for splitting one column into a specific number of columns, but not variable columns. The columns that have less data (2 series of strings separated by commas instead of 3) will have nulls in the third place.
Ideas? Let me know if I must clarify anything.
Splitting this data into separate columns is a very good start (coma-separated values are an heresy). However, a "variable number of properties" should typically be modeled as a one-to-many relationship.
CREATE TABLE main_entity (
id INT PRIMARY KEY,
other_fields INT
);
CREATE TABLE entity_properties (
main_entity_id INT PRIMARY KEY,
property_value INT,
FOREIGN KEY (main_entity_id) REFERENCES main_entity(id)
);
entity_properties.main_entity_id is a foreign key to main_entity.id.
Congratulations, you are on the right path, this is called normalisation. You are about to reach the First Normal Form.
Beweare, however, these properties should have a sensibly similar nature (ie. all phone numbers, or addresses, etc.). Do not to fall into the dark side (a.k.a. the Entity-Attribute-Value anti-pattern), and be tempted to throw all properties into the same table. If you can identify several types of attributes, store each type in a separate table.
If these are all fixed length strings (as in the question), then you can do the work fairly simply (at least relative to other solutions):
select substring(col, 1+13*(n-1), 12) as val
from t join
(select 1 as n union all select union all select 3
) n
on len(t.col) <= 13*n.n
This is a useful hack if all the entries are the same size (not so easy if they are of different sizes). Do, however, think about the data structure because semi-colon (or comma) separated list is not a very good data structure.
IF I were you, I would create a simple function that is dividing values separated with ';' like this:
IF EXISTS (SELECT * FROM sysobjects WHERE id = object_id(N'fn_Split_List') AND xtype IN (N'FN', N'IF', N'TF'))
BEGIN
DROP FUNCTION [dbo].[fn_Split_List]
END
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE FUNCTION [dbo].[fn_Split_List](#List NVARCHAR(512))
RETURNS #ResultRowset TABLE ( [Value] NVARCHAR(128) PRIMARY KEY)
AS
BEGIN
DECLARE #XML xml = N'<r><![CDATA[' + REPLACE(#List, ';', ']]></r><r><![CDATA[') + ']]></r>'
INSERT INTO #ResultRowset ([Value])
SELECT DISTINCT RTRIM(LTRIM(Tbl.Col.value('.', 'NVARCHAR(128)')))
FROM #xml.nodes('//r') Tbl(Col)
RETURN
END
GO
Than simply called in this way:
SET NOCOUNT ON
GO
DECLARE #RawData TABLE( [Value] NVARCHAR(256))
INSERT INTO #RawData ([Value] )
VALUES ('1111111;22222222')
,('3333333;113113131')
,('776767676')
,('89332131;313131312;54545353')
SELECT SL.[Value]
FROM #RawData AS RD
CROSS APPLY [fn_Split_List] ([Value]) as SL
SET NOCOUNT OFF
GO
The result is as the follow:
Value
1111111
22222222
113113131
3333333
776767676
313131312
54545353
89332131
Anyway, the logic in the function is not complicated, so you can easily put it anywhere you need.
Note: There is not limitations of how many values you will have separated with ';', but there are length limitation in the function that you can set to NVARCHAR(MAX) if you need.
EDIT:
As I can see, there are some rows in your example that will caused the function to return empty strings. For example:
number;number;
will return:
number
number
'' (empty string)
To clear them, just add the following where clause to the statement above like this:
SELECT SL.[Value]
FROM #RawData AS RD
CROSS APPLY [fn_Split_List] ([Value]) as SL
WHERE LEN(SL.[Value]) > 0

Splitting delimited values in a SQL column into multiple rows

I would really like some advice here, to give some background info I am working with inserting Message Tracking logs from Exchange 2007 into SQL. As we have millions upon millions of rows per day I am using a Bulk Insert statement to insert the data into a SQL table.
In fact I actually Bulk Insert into a temp table and then from there I MERGE the data into the live table, this is for test parsing issues as certain fields otherwise have quotes and such around the values.
This works well, with the exception of the fact that the recipient-address column is a delimited field seperated by a ; character, and it can be incredibly long sometimes as there can be many email recipients.
I would like to take this column, and split the values into multiple rows which would then be inserted into another table. Problem is anything I am trying is either taking too long or not working the way I want.
Take this example data:
message-id recipient-address
2D5E558D4B5A3D4F962DA5051EE364BE06CF37A3A5#Server.com user1#domain1.com
E52F650C53A275488552FFD49F98E9A6BEA1262E#Server.com user2#domain2.com
4fd70c47.4d600e0a.0a7b.ffff87e1#Server.com user3#domain3.com;user4#domain4.com;user5#domain5.com
I would like this to be formatted as followed in my Recipients table:
message-id recipient-address
2D5E558D4B5A3D4F962DA5051EE364BE06CF37A3A5#Server.com user1#domain1.com
E52F650C53A275488552FFD49F98E9A6BEA1262E#Server.com user2#domain2.com
4fd70c47.4d600e0a.0a7b.ffff87e1#Server.com user3#domain3.com
4fd70c47.4d600e0a.0a7b.ffff87e1#Server.com user4#domain4.com
4fd70c47.4d600e0a.0a7b.ffff87e1#Server.com user5#domain5.com
Does anyone have any ideas about how I can go about doing this?
I know PowerShell pretty well, so I tried in that, but a foreach loop even on 28K records took forever to process, I need something that will run as quickly/efficiently as possible.
Thanks!
If you are on SQL Server 2016+
You can use the new STRING_SPLIT function, which I've blogged about here, and Brent Ozar has blogged about here.
SELECT s.[message-id], f.value
FROM dbo.SourceData AS s
CROSS APPLY STRING_SPLIT(s.[recipient-address], ';') as f;
If you are still on a version prior to SQL Server 2016
Create a split function. This is just one of many examples out there:
CREATE FUNCTION dbo.SplitStrings
(
#List NVARCHAR(MAX),
#Delimiter NVARCHAR(255)
)
RETURNS TABLE
AS
RETURN (SELECT Number = ROW_NUMBER() OVER (ORDER BY Number),
Item FROM (SELECT Number, Item = LTRIM(RTRIM(SUBSTRING(#List, Number,
CHARINDEX(#Delimiter, #List + #Delimiter, Number) - Number)))
FROM (SELECT ROW_NUMBER() OVER (ORDER BY s1.[object_id])
FROM sys.all_objects AS s1 CROSS APPLY sys.all_objects) AS n(Number)
WHERE Number <= CONVERT(INT, LEN(#List))
AND SUBSTRING(#Delimiter + #List, Number, 1) = #Delimiter
) AS y);
GO
I've discussed a few others here, here, and a better approach than splitting in the first place here.
Now you can extrapolate simply by:
SELECT s.[message-id], f.Item
FROM dbo.SourceData AS s
CROSS APPLY dbo.SplitStrings(s.[recipient-address], ';') as f;
Also I suggest not putting dashes in column names. It means you always have to put them in [square brackets].
SQL Server 2016 include a new table function string_split(), similar to the previous solution.
The only requirement is Set compatibility level to 130 (SQL Server 2016)
You may use CROSS APPLY (available in SQL Server 2005 and above) and STRING_SPLIT function (available in SQL Server 2016 and above):
DECLARE #delimiter nvarchar(255) = ';';
-- create tables
CREATE TABLE MessageRecipients (MessageId int, Recipients nvarchar(max));
CREATE TABLE MessageRecipient (MessageId int, Recipient nvarchar(max));
-- insert data
INSERT INTO MessageRecipients VALUES (1, 'user1#domain.com; user2#domain.com; user3#domain.com');
INSERT INTO MessageRecipients VALUES (2, 'user#domain1.com; user#domain2.com');
-- insert into MessageRecipient
INSERT INTO MessageRecipient
SELECT MessageId, ltrim(rtrim(value))
FROM MessageRecipients
CROSS APPLY STRING_SPLIT(Recipients, #delimiter)
-- output results
SELECT * FROM MessageRecipients;
SELECT * FROM MessageRecipient;
-- delete tables
DROP TABLE MessageRecipients;
DROP TABLE MessageRecipient;
Results:
MessageId Recipients
----------- ----------------------------------------------------
1 user1#domain.com; user2#domain.com; user3#domain.com
2 user#domain1.com; user#domain2.com
and
MessageId Recipient
----------- ----------------
1 user1#domain.com
1 user2#domain.com
1 user3#domain.com
2 user#domain1.com
2 user#domain2.com
for table = "yelp_business", split the column categories values separated by ; into rows and display as category column.
SELECT unnest(string_to_array(categories, ';')) AS category
FROM yelp_business;

SQL separate string being passed

I have a product table with a tag column, each product has multiple tags stored in this format: "|technology|mobile|acer|laptop|" ...second product's tags could look like this "|computer|laptop|toshiba|"
I am using MS SQL Server 2008 and stored procedure, I would like to know how I could pass a string like "|computer|laptop|" and get both records returned as they both have the tag laptop in them and if I passed "|computer|" only the second record would return as it is the only one comtainning that tag.
What is the best way of doing this without performance penalties using stored procedure?
I have so far had no luck with different codes i have found on the internet, I really hope you guys can maybe help me with this, thank you.
I agree with the other posters that storing data in a column like that is going to cause headaches. You really want to store those tags in a child table so you can easily and efficiently join them. If it's an inherited system or something you can't refactor right away you can write a split function.
The typical sql split implementation uses a while loop and a table variable in a multi-statement TVF. Every iteration incurs more I/O and CPU overhead. Performance testing on SQL 2005 SP1 showed that this overhead is hidden from the I/O Stats and query plan. Profiling the code will reveal the true cost.
Rewriting that function into a inline TVF is much more efficient. The primary difference between an inline and multi-statement TVF is the Query Optimizer will merge the inline function into the query before processing; this eliminates the overhead from the function call. Also, since there is no table variable required, the additional I/O cost is eliminated. Finally, you avoid the costly iterative processing.
Here is the fastest, most scalable split function I could come up with including unit tests and summary.
This function requires a numbers table:
CREATE TABLE dbo.Numbers
(
NUM INT PRIMARY KEY CLUSTERED
)
;WITH Nbrs ( n ) AS
(
SELECT 1 UNION ALL
SELECT 1 + n FROM Nbrs WHERE n < 10000
)
INSERT INTO dbo.Numbers
SELECT n FROM Nbrs
OPTION ( MAXRECURSION 10000 )
The source of the function is here:
IF EXISTS (
SELECT 1
FROM dbo.sysobjects
WHERE id = object_id(N'[dbo].[ParseString]')
AND xtype in (N'FN', N'IF', N'TF'))
BEGIN
DROP FUNCTION [dbo].[ParseString]
END
GO
CREATE FUNCTION dbo.ParseString (#String VARCHAR(8000), #Delimiter VARCHAR(10))
RETURNS TABLE
AS
/*******************************************************************************************************
* dbo.ParseString
*
* Creator: MagicMike
* Date: 9/12/2006
*
*
* Outline: A set-based string tokenizer
* Takes a string that is delimited by another string (of one or more characters),
* parses it out into tokens and returns the tokens in table format. Leading
* and trailing spaces in each token are removed, and empty tokens are thrown
* away.
*
*
* Usage examples/test cases:
Single-byte delimiter:
select * from dbo.ParseString2('|HDI|TR|YUM|||', '|')
select * from dbo.ParseString('HDI| || TR |YUM', '|')
select * from dbo.ParseString(' HDI| || S P A C E S |YUM | ', '|')
select * from dbo.ParseString2('HDI|||TR|YUM', '|')
select * from dbo.ParseString('', '|')
select * from dbo.ParseString('YUM', '|')
select * from dbo.ParseString('||||', '|')
select * from dbo.ParseString('HDI TR YUM', ' ')
select * from dbo.ParseString(' HDI| || S P A C E S |YUM | ', ' ') order by Ident
select * from dbo.ParseString(' HDI| || S P A C E S |YUM | ', ' ') order by StringValue
Multi-byte delimiter:
select * from dbo.ParseString('HDI and TR', 'and')
select * from dbo.ParseString('Pebbles and Bamm Bamm', 'and')
select * from dbo.ParseString('Pebbles and sandbars', 'and')
select * from dbo.ParseString('Pebbles and sandbars', ' and ')
select * from dbo.ParseString('Pebbles and sand', 'and')
select * from dbo.ParseString('Pebbles and sand', ' and ')
*
*
* Notes:
1. A delimiter is optional. If a blank delimiter is given, each byte is returned in it's own row (including spaces).
select * from dbo.ParseString3('|HDI|TR|YUM|||', '')
2. In order to maintain compatibility with SQL 2000, ident is not sequential but can still be used in an order clause
If you are running on SQL2005 or later
SELECT Ident, StringValue FROM
with
SELECT Ident = ROW_NUMBER() OVER (ORDER BY ident), StringValue FROM
*
*
* Modifications
*
*
********************************************************************************************************/
RETURN (
SELECT Ident, StringValue FROM
(
SELECT Num as Ident,
CASE
WHEN DATALENGTH(#delimiter) = 0 or #delimiter IS NULL
THEN LTRIM(SUBSTRING(#string, num, 1)) --replace this line with '' if you prefer it to return nothing when no delimiter is supplied. Remove LTRIM if you want to return spaces when no delimiter is supplied
ELSE
LTRIM(RTRIM(SUBSTRING(#String,
CASE
WHEN (Num = 1 AND SUBSTRING(#String,num ,DATALENGTH(#delimiter)) <> #delimiter) THEN 1
ELSE Num + DATALENGTH(#delimiter)
END,
CASE CHARINDEX(#Delimiter, #String, Num + DATALENGTH(#delimiter))
WHEN 0 THEN LEN(#String) - Num + DATALENGTH(#delimiter)
ELSE CHARINDEX(#Delimiter, #String, Num + DATALENGTH(#delimiter)) - Num -
CASE
WHEN Num > 1 OR (Num = 1 AND SUBSTRING(#String,num ,DATALENGTH(#delimiter)) = #delimiter)
THEN DATALENGTH(#delimiter)
ELSE 0
END
END
)))
End AS StringValue
FROM dbo.Numbers
WHERE Num <= LEN(#String)
AND (
SUBSTRING(#String, Num, DATALENGTH(ISNULL(#delimiter,''))) = #Delimiter
OR Num = 1
OR DATALENGTH(ISNULL(#delimiter,'')) = 0
)
) R WHERE StringValue <> ''
)
For your case, you could use it like this:
--SAMPLE DATA
CREATE TABLE #products
(
productid INT IDENTITY PRIMARY KEY CLUSTERED ,
prodname VARCHAR(200),
tags VARCHAR(200)
)
INSERT INTO #products (prodname, tags)
SELECT 'toshiba laptop', '|laptop|toshiba|notebook|'
UNION ALL
SELECT 'toshiba netbook', '|netbook|toshiba|'
UNION ALL
SELECT 'Apple macbook', '|laptop|apple|notebook|'
UNION ALL
SELECT 'Apple mouse', '|apple|mouse'
--Actual solution
DECLARE #searchTags VARCHAR(200)
SET #searchTags = '|apple|laptop|' --This would the string that would get passed in if it were a stored procedure
--First we convert the supplied tags into a table for use later
--My (2005) dev box raised a severe error attempting to do the search in 1 step
--hence the temp table
CREATE TABLE #tags
(
tag VARCHAR(200) PRIMARY KEY CLUSTERED
)
INSERT INTO #tags --The function splits the string up into one record for each value
SELECT stringValue
FROM dbo.parsestring(#searchTags,'|') --SQL 2005 has a real problem joining to a TVF twice, apparently
SELECT DISTINCT p.*
FROM #products P --we join the products table with the function to get a row for each tag so we can compare with the temp table
CROSS APPLY (SELECT stringValue FROM dbo.parsestring(P.tags,'|')) T
WHERE EXISTS(SELECT * FROM #tags WHERE tag = T.stringValue) --we compare the rows with our temp table and if we get matches, the products are returned
/*This will return the Apple Macbook and the Toshiba Laptop because they both contain
the 'laptop' tag and the Apple mouse because it contains the 'apple' tag. The
toshiba netbook contains neither tag so it won't be returned.*/
But, with your tags in a separate table as suggested (1-many for a simplified example) It would look like this:
SELECT * FROM Products P
WHERE EXISTS (SELECT *
FROM tags T
INNER JOIN dbo.parsestring(#tags,'|') Q
ON T.tag = Q.StringValue
WHERE T.productid = P.productiId )
You have a many-to-many relationship between products and tags. The best way of doing this is to redesign your database. Create a table of tags and a junction table that links products with tags.
That's not a very good design. Combining like terms into one field and separating them with a delimiter such as a vertical bar does not scale well and it is very limiting.
I recommend you read up on how to design databases. The best book I ever bought regarding database design was Database Design for Mere Mortals by Michael Hernandez ISBN: 0-201-69471-9. Amazon Listing I noticed he has a second edition.
He walks you through the entire process of (from start to finish) of designing a database. I recommend you start with this book.
You have to learn to look at things in groups or chunks. Database design has simple building blocks just like programming does. If you gain a thorough understanding of these simple building blocks you can tackle any database design.
In programming you have:
If Constructs
If Else Constructs
Do While Loops
Do Until Loops
Case Constructs
With databases you have:
Data Tables
Lookup Tables
One to One relationships
One to Many Relationships
Many to Many relationships
Primary keys
Foreign keys
The simpler you make things the better. A database is nothing more than a place where you put data into cubbie holes. Start by identifying what these cubbie holes are and what kind of stuff you want in them.
You are never going to create the perfect database design the first time you try. This is a fact. Your design will go through several refinements during the process. Sometimes things won't seem apparent until you start entering data, and then you have an ahh ha moment.
The web brings it's own sets of challenges. Bandwith issues. Statelessness. Erroneous data from processes that start but never get finished.
make a split with CLR function return a table with the value or pass as xml and load it into a table varible an make a join
create procedure search
(
#data xml
)
AS
BEGIN
--declare #data xml
declare #LoadData table
(
dataToFind varchar(max)
)
--set #data= cast(
--'<data>
-- <item>computer</item>
-- <item>television</item>
--</data>' as xml)
insert into #LoadData
SELECT T2.Loc.value('.','varchar(max)')
FROM (select #data as data )T
CROSS APPLY data.nodes('/data/item') as T2(Loc)
select * from #LoadData--use for join
END
I would suggest you write an extra couplle of tables that with "proper design,
Populate those tables from the existing not well designed bit - this way y our search will work properly buy others using the old | pipe approach won't notice till you have time to refactor

how to write the store procedure for searching (CSV)?

how can i write the store procedure for searching particular string in a column of table, for given set of strings (CSV string).
like : select * from xxx where tags like ('oscar','rahman','slumdog')
how can i write the procedure for that combination of tags.
To create a comma seperated string...
You could then apply this list to Oded example to create the LIKE parts of the WHERE cluase on the fly.
DECLARE #pos int, #curruntLocation char(20), #input varchar(2048)
SELECT #pos=0
SELECT #input = 'oscar,rahman,slumdog'
SELECT #input = #input + ','
CREATE TABLE #tempTable (temp varchar(100) )
WHILE CHARINDEX(',',#input) > 0
BEGIN
SELECT #pos=CHARINDEX(',',#input)
SELECT #curruntLocation = RTRIM(LTRIM(SUBSTRING(#input,1,#pos-1)))
INSERT INTO #tempTable (temp) VALUES (#curruntLocation)
SELECT #input=SUBSTRING(#input,#pos+1,2048)
END
SELECT * FROM #tempTable
DR0P TABLE #tempTable
First off, the use of like for exact matches is sub-optimal. Might as well use =, and if doing so, you can use the IN syntax:
select * from xxx
where tags IN ('oscar', 'rahman', 'slumdog')
I am guessing you are not looking for an exact match, but for any record where the tags field contains all of the tags.
This would be something like this:
select * from xxx
where tags like '%oscar%'
and tags like '%rahman%'
and tags like '%slumdog%'
This would be not be very fast or performant though.
Think about moving this kind of logic into your application, where it is faster and easier to do.
Edit:
Following the comments - there are lots of examples on how to parse delimited strings out there. You can put these in a table and use dynamic sql to generate your query.
But, this will have bad performance and SQL Server will not be able to cache query plans for this kind of thing. As I said above - think about moving this kind of logic to application level.

Filtering With Multi-Select Boxes With SQL Server

I need to filter result sets from sql server based on selections from a multi-select list box. I've been through the idea of doing an instring to determine if the row value exists in the selected filter values, but that's prone to partial matches (e.g. Car matches Carpet).
I also went through splitting the string into a table and joining/matching based on that, but I have reservations about how that is going to perform.
Seeing as this is a seemingly common task, I'm looking to the Stack Overflow community for some feedback and maybe a couple suggestions on the most commonly utilized approach to solving this problem.
I solved this one by writing a table-valued function (we're using 2005) which takes a delimited string and returns a table. You can then join to that or use WHERE EXISTS or WHERE x IN. We haven't done full stress testing yet, but with limited use and reasonably small sets of items I think that performance should be ok.
Below is one of the functions as a starting point for you. I also have one written to specifically accept a delimited list of INTs for ID values in lookup tables, etc.
Another possibility is to use LIKE with the delimiters to make sure that partial matches are ignore, but you can't use indexes with that, so performance will be poor for any large table. For example:
SELECT
my_column
FROM
My_Table
WHERE
#my_string LIKE '%|' + my_column + '|%'
.
/*
Name: GetTableFromStringList
Description: Returns a table of values extracted from a delimited list
Parameters:
#StringList - A delimited list of strings
#Delimiter - The delimiter used in the delimited list
History:
Date Name Comments
---------- ------------- ----------------------------------------------------
2008-12-03 T. Hummel Initial Creation
*/
CREATE FUNCTION dbo.GetTableFromStringList
(
#StringList VARCHAR(1000),
#Delimiter CHAR(1) = ','
)
RETURNS #Results TABLE
(
String VARCHAR(1000) NOT NULL
)
AS
BEGIN
DECLARE
#string VARCHAR(1000),
#position SMALLINT
SET #StringList = LTRIM(RTRIM(#StringList)) + #Delimiter
SET #position = CHARINDEX(#Delimiter, #StringList)
WHILE (#position > 0)
BEGIN
SET #string = LTRIM(RTRIM(LEFT(#StringList, #position - 1)))
IF (#string <> '')
BEGIN
INSERT INTO #Results (String) VALUES (#string)
END
SET #StringList = RIGHT(#StringList, LEN(#StringList) - #position)
SET #position = CHARINDEX(#Delimiter, #StringList, 1)
END
RETURN
END
I've been through the idea of doing an
instring to determine if the row value
exists in the selected filter values,
but that's prone to partial matches
(e.g. Car matches Carpet)
It sounds to me like you aren't including a unique ID, or possibly the primary key as part of values in your list box. Ideally each option will have a unique identifier that matches a column in the table you are searching on. If your listbox was like below then you would be able to filter for specifically for cars because you would get the unique value 3.
<option value="3">Car</option>
<option value="4">Carpret</option>
Then you just build a where clause that will allow you to find the values you need.
Updated, to answer comment.
How would I do the related join
considering that the user can select
and arbitrary number of options from
the list box? SELECT * FROM tblTable
JOIN tblOptions ON tblTable.FK = ? The
problem here is that I need to join on
multiple values.
I answered a similar question here.
One method would be to build a temporary table and add each selected option as a row to the temporary table. Then you would simply do a join to your temporary table.
If you want to simply create your sql dynamically you can do something like this.
SELECT * FROM tblTable WHERE option IN (selected_option_1, selected_option_2, selected_option_n)
I've found that a CLR table-valued function which takes your delimited string and calls Split on the string (returning the array as the IEnumerable) is more performant than anything written in T-SQL (it starts to break down when you have around one million items in the delimited list, but that's much further out than the T-SQL solution).
And then, you could join on the table or check with EXISTS.