We are trying to update our classic asp search engine to protect it from SQL injection. We have a VB 6 function which builds a query dynamically by concatenating a query together based on the various search parameters. We have converted this to a stored procedure using dynamic sql for all parameters except for the keywords.
The problem with keywords is that there are a variable number words supplied by the user and we want to search several columns for each keyword. Since we cannot create a separate parameter for each keyword, how can we build a safe query?
Example:
#CustomerId AS INT
#Keywords AS NVARCHAR(MAX)
#sql = 'SELECT event_name FROM calendar WHERE customer_id = #CustomerId '
--(loop through each keyword passed in and concatenate)
#sql = #sql + 'AND (event_name LIKE ''%' + #Keywords + '%'' OR event_details LIKE ''%' + #Keywords + '%'')'
EXEC sp_executesql #sql N'#CustomerId INT, #CustomerId = #CustomerId
What is the best way to handle this and maintaining protection from SQL injection?
You may not like to hear this, but it might be better for you to go back to dynamically constructing your SQL query in code before issuing against the database. If you use parameter placeholders in the SQL string you get the protection against SQL injection attacks.
Example:
string sql = "SELECT Name, Title FROM Staff WHERE UserName=#UserId";
using (SqlCommand cmd = new SqlCommand(sql))
{
cmd.Parameters.Add("#UserId", SqlType.VarChar).Value = "smithj";
You can build the SQL string depending on the set of columns you need to query and then add the parameter values once the string is complete. This is a bit of a pain to do, but I think it is much easier than having really complicated TSQL which unpicks lots of possible permutations of possible inputs.
You have 3 options here.
Use a function that converts lists tables and join into it. So you will have something like this.
SELECT *
FROM calendar c
JOIN dbo.fnListToTable(#Keywords) k
ON c.keyword = k.keyword
Have a fixed set of params, and only allow the maximum of N keywords to be searched on
CREATE PROC spTest
#Keyword1 varchar(100),
#Keyword2 varchar(100),
....
Write an escaping string function in TSQL and escape your keywords.
Unless you need it, you could simply strip out any character that's not in [a-zA-Z ] - most of those things won't be in searches and you should not be able to be injected that way, nor do you have to worry about keywords or anything like that. If you allow quotes, however, you will need to be more careful.
Similar to sambo99's #1, you can insert the keywords into a temporary table or table variable and join to it (even using wildcards) without danger of injection:
This isn't really dynamic:
SELECT DISTINCT event_name
FROM calendar
INNER JOIN #keywords
ON event_name LIKE '%' + #keywords.keyword + '%'
OR event_description LIKE '%' + #keywords.keyword + '%'
You can actually generate an SP with a large number of parameters instead of coding it by hand (set the defaults to '' or NULL depending on your preference in coding your searches). If you found you needed more parameters, it would be simple to increase the number of parameters it generated.
You can move the search to a full-text index outside the database like Lucene and then use the Lucene results to pull the matching database rows.
You can try this:
SELECT * FROM [tablename] WHERE LIKE % +keyword%
Related
How can I create a function like this?
function FN_something (#entrada char(50))
declare #consulta table
declare #notificacao varchar(50)
declare #multa float
declare #saida varchar(50)
set #consulta as = (select num_notificacao,num_multa from table where field = #entrada)
set #notificacao = #consulta.num_notificacao
set #multa = #consulta.num_multa
set #saida = "resultado: "+ #notificacao +";"+#multa
return #saida
Thanks in advance
I would not use a function... Scalar functions tend to be a real performance killer. Try to use something like this inline
SELECT 'resultado: '
+ ISNULL(CAST(t.num_notificacao AS VARCHAR(MAX)),'???')
+ ';'
+ ISNULL(CAST(t.num_multa AS VARCHAR(MAX)),'???')
FROM SomeTable AS t WHERE t.SomeField=#entrada;
If you need a function it was much better to use an inlined TVF (syntax without BEGIN...END and bind it into your query with CROSS APPLY.
Might be simplified:
If your columns are NOT NULL you can go without ISNULL()-function. If your columns are strings, you can do without CAST()... My code is defensive proramming :-D
Hint
If this is something you need more often, you might introduce a VIEW carrying this calculated column and use it instead of your table. You might include this value into your table as computed column as well...
UPDATE
Great, the VIEW you show in the comment is an inline TVF actually, which is very good!
My magic crystall ball tells me, that you might need something like this:
SELECT cl.*
,'resultado: ' + t.num_notificacao + ';' + t.num_multa AS CalculatedResult
FROM dbo.[CampoLivre876]('SomeParameter') AS cl
LEFT JOIN SomeOtherTable AS t ON cl.entrada=t.SomeField --should be only one related row per main row!
This will call the iTFV and join it to the other Table, where the two columns are living. I Assume, that the CampoLivre876-row knows its entrada key.
Hint 2:
If this works for you, you might include this approach directly into your existing iTVF.
UPDATE 2
You might try to change your function like here:
ALTER FUNCTION [dbo].[CampoLivre876] ()
RETURNS TABLE
RETURN
Select cl.mul_numero_notificacao + ';' + CAST(cl.mul_valor_multa as varchar(max)) AS ExistingColumn
,'resultado: ' + t.num_notificacao + ';' + CAST(t.num_multa AS varchar(max)) AS CalculatedResult
From Campo_Livre AS cl With(NoLock)
INNER JOIN SomeOtherTable AS t ON cl.entrada=t.SomeField;
This should read all lines in one go. Reading 1 row after the other is - in almost all cases - something really, really bad...
Here is an example of a function with correct SQL Server syntax:
create function FN_something (
#entrada char(50) -- should probably be `varchar(50)` rather than `char(50)`
) returns varchar(50)
begin
declare #saida varchar(50);
select #saida = 'resultado: ' + num_notificacao + ';' + num_multa
from table
where field = #entrada;
return #saida;
end;
Note: This assumes that the num_ columns are strings, not numbers. If they are numbers, you need to convert them or use concat().
EDIT:
A function really isn't appropriate for this. Probably the best solution is a computed column:
alter table t add something as (concat('resultado: ', num_notificacao, ';', num_multa);
Then you can get the value directly from the table. In earlier versions of SQL Server, you would use a view rather than computed column.
I created a stored procedure (spBalanceRange) with 2 optional parameters. They've been set to a default value and the sp works fine when I pass only 1 value per parameter by position. However, I have a situation where I'm trying to pass, by position, two strings immediately followed by a wildcard. I want the user to be able to search for Vendor names that start with either 'C%' or 'F%'. Here's the gist of the CREATE PROC statement:
CREATE PROC spBalanceRange
#VendorVar varchar(40) = '%',
#BalanceMin money = 1.0
...
Here's what I've tried so far, but doesn't work:
EXEC spBalanceRange '(C%|F%)', 200.00;
EXEC spBalanceRange 'C%|F%', 200.00;
Is there a way to check for 2 or more string values with a wildcard when passed by position? Thanks.
EDIT: According to your comments you are looking for the first letter of a vendor's name only.
In this special case I could suggest an easy, not well performing but really simple approach. CHARINDEX returns a number greater than zero, if a character appears within a string. So you just have to pass in all your lookup-first-characters as a simple "chain":
DECLARE #DummyVendors TABLE(VendorName VARCHAR(100));
INSERT INTO #DummyVendors VALUES
('Camel Industries')
,('Fritz and Fox')
,('some other');
DECLARE #ListOfFirstLetters VARCHAR(100)='CF';
SELECT VendorName
FROM #DummyVendors AS dv
WHERE CHARINDEX(LEFT(dv.VendorName,1),#ListOfFirstLetters)>0
This was the former answer
Checking against more than one value needs either a dedicated list of compares
WHERE val=#prm1 OR val=#prm2 OR ... (you know the count before)
...or you use the IN-clause
WHERE LEFT(VenoderName,1) IN ('C','F', ...)
...but you cannot pass the IN-list with a parameter like ... IN(#allValues)
You might think about a created TYPE to pass in all your values like a table and use an INNER JOIN as filter: https://stackoverflow.com/a/337864/5089204 (and a lot of other examples there...)
Or you might think of dynamic SQL: https://stackoverflow.com/a/5192765/5089204
And last but not least you might think of one of the many split string approaches. This is one of my own answers, section "dynamic IN-statement": https://stackoverflow.com/a/33658220/5089204
I'm answering my own question, and maybe other solutions exist but here is what had to happen with my stored procedure in order to pass variables by position:
CREATE PROC spBalanceRange
#VendorVar varchar(40) = '%',
#BalanceMin money = 1.0
AS
IF (#VendorVar = '%' AND #BalanceMin IS NULL OR #BalanceMin = '')
BEGIN
PRINT 'BalanceMin cannot be null.';
END
IF (#VendorVar = % AND #BalanceMin IS NOT NULL)
BEGIN
(sql statement using parameters)
END
EXEC spBalanceRange '[C,F]%', 200.00;
That's what I know.
I am trying to write a SQL query in SQL Server 2008 R2 that will allow a user to search a database table by a number of parameters. The way this should work is, my user enters his criteria and the query looks for all close matches, while ignoring those criteria for which the user did not enter a value.
I've written my query using LIKE and parameters, like so:
select item
from [item]
where a like #a and b like #b and c like #c ...
where 'a', 'b', and 'c' are table columns, and my # parameters all default to '%' wildcards. This goes on for about twenty different columns, and that leads to my problem: if this query is entered as is, no input, just wildcards, it returns no results. Yet this table contains over 30,000 rows, so an all-wildcard query should return the whole table. Obviously I'm going about this the wrong way, but I don't know how to correct it.
I can't use 'contains' or 'freetext', as those look for whole words, and I need to match user input no matter where it occurs in the actual column value. I've tried breaking my query up into individual steps using 'intersect', but that doesn't change anything. Does anyone know a better way to do this?
To allow for null inputs, this is a good pattern:
select * from my table where ColA LIKE isnull(#a, ColA) AND ColB like isnull(#b, ColB)
This avoids having to construct and execute a dynamic SQL statement (and creating possible SQL injection issues.)
my # parameters all default to '%' wildcards
Don't do this. Default them to null. The way to disregard empty parameters is with a short circuit:
(#a IS NULL OR #a LIKE a)
Depending on how you want to handle missing data in the column, you might want a third term, because null will not match LIKE statements:
(#a IS NULL OR a IS NULL OR #a LIKE a)
How can I search a SQL database with multiple "%" wildcards?
Slowly. SQL is suboptimal for doing text comparisons. The best approach is to perform this search somewhere else, or at least structure your data to facilitate these kinds of queries. If you know you'll be performing a lot of these queries, consider redesigning your schema in the shape of a suffix tree. At an absolute bare minimum, do something so that every LIKE match is suffix-only, meaning LIKE 'xxx%' and never LIKE '%xxx' or LIKE 'x%x'. The latter two preclude the use of indexes. And put an index on every column you need to search.
Thanks for the guidance, all. It turns out that the table I'm querying can easily contain null values in the columns I'm searching against, so I expanded my query to say "where (a like #a or a is null) and ... " and it works now.
Personally, I'd do this in the application layer (assuming you have one), and build the query around the parameters the user supplies, eliminating the ones they don't.
For example, the following bit of code builds the query in SQL, where only the parameters the user has supplied (not null) are included in the where clause.
NOTE: This is very crude, as it doesn't take into account AND's if the first parameter if null, and it doesn't remove the WHERE clause if no parameters are supplied. If you let me know what language your application layer is built in, I'll provide a better example. (This is purely pseudo-code!)
DECLARE #a VARCHAR(100) = '''%SomeValue%''', #b VARCHAR(100)= '''%AnotherValue%''', #c VARCHAR(100)
DECLARE #SQL VARCHAR(MAX) = 'SELECT * FROM MyTable WHERE'
IF #a IS NOT NULL
BEGIN
SET #SQL += ' ColA LIKE ' + #a
END
IF #b IS NOT NULL
BEGIN
SET #SQL += ' AND ColA LIKE ' + #b
END
IF #c IS NOT NULL
BEGIN
SET #SQL += ' AND ColC LIKE ' + #c
END
PRINT #SQL
--EXEC(#SQL)
Output:
SELECT * FROM MyTable WHERE ColA = '%SomeValue%' AND ColB = '%AnotherValue%'
I need to execute a search query in SQL Server where I need to filter out data based upon an user input textfield.
The problem is, this query needs to be executed on several tables (so I only know the tablecolumns at runtime).
This is the query I have:
SELECT * FROM [BTcegeka.C2M].[dbo].[Lookup_Country] WHERE Name LIke '%test%'
Now the problem is I need to do the Like function on every column (I only know the columnname at runtime) in the table. I am calling this query from an ASP.NET website. The user selects a table from a dropdownlist and can then enter the search field.
This is what I really want to accomplish:
SELECT * FROM [BTcegeka.C2M].[dbo].[Lookup_Country] WHERE * LIke '%test%'
Obviously 'Where * Like' Fails. How can I accomplish this?
You can query all columns in a table like:
select name from sys.columns where object_id = object_id('YourTable')
Then you can construct a query that does a like for each column.
Another approach is to create a calculated column called SearchField that contains a concatenation of all strings you'd like to search for. Then you can search like:
create table #tmp (id int identity, col1 varchar(10), col2 varchar(10),
SearchField as col1 + '|' + col2 persisted)
insert #tmp (col1, col2) values
('alfa', 'beta'),
('gamma', 'DELTA'),
('GAMMA', 'delta')
select * from #tmp where SearchField like '%alfa%'
Try using your SQL query like this.
SELECT * FROM [BTcegeka.C2M].[dbo].[Lookup_Country]
WHERE
COL1 LIke '%test%'
OR COL2 LIke '%test%'
OR COL3 LIke '%test%'
You may use AND instead of OR if your requirement needs that.
If you know the column names at run time, then you should build you query in .NET before passing it to sql. You can build it with the correct column name. This way you can account also for the type of the column you search in.
Careful though this path you chose is prone to SQL injection so before sending a query to the SQL you should check it.
If you really need to do this you can search in sqlserver meta tables and find the description of selected user table. Make a good use of this data is easy and you can make any sql you want with this information, but performance may not the that good
you have to use dynamic sql for implementing this. Your column name needs to be passed as parameter to this stored procedure or if you dont want to create stored procedure just declare one paramter and assign the value selected from the drop down list to it and use that in the query.
create procedure sp_dynamicColumn
(
#columnName varchar(10)
)
as
begin
declare #DYNAMICSQL nvarchar(4000);
SET #DYNAMICSQL = 'Select * from [BTcegeka.C2M].[dbo].[Lookup_Country] where '+ #columnName + ' like ''%test%'''
EXECUTE SP_EXECUTESQL #DYNAMICSQL
end
go
I am passing a comma-delimited list of values into a stored procedure. I need to execute a query to see if the ID of an entity is in the comma-delimited list. Unfortunately, I think I do not understand something.
When I execute the following stored procedure:
exec dbo.myStoredProcedure #myFilter=N'1, 2, 3, 4'
I receive the following error:
"Conversion failed when converting the varchar value '1, 2, 3, 4' to data type int."
My stored procedure is fairly basic. It looks like this:
CREATE PROCEDURE [dbo].[myStoredProcedure]
#myFilter nvarchar(512) = NULL
AS
SET NOCOUNT ON
BEGIN
-- Remove the quote marks so the filter will work with the "IN" statement
SELECT #myFilter = REPLACE(#myFilter, '''', '')
-- Execute the query
SELECT
t.ID,
t.Name
FROM
MyTable t
WHERE
t.ID IN (#myFilter)
ORDER BY
t.Name
END
How do I use a parameter in a SQL statement as described above? Thank you!
You could make function that takes your parameter, slipts it and returns table with all the numbers in it.
If your are working with lists or arrays in SQL Server, I recommend that you read Erland Sommarskogs wonderful stuff:
Arrays and Lists in SQL Server 2005
You need to split the string and dump it into a temp table. Then you join against the temp table.
There are many examples of this, here is one at random.
http://blogs.microsoft.co.il/blogs/itai/archive/2009/02/01/t-sql-split-function.aspx
Absent a split function, something like this:
CREATE PROCEDURE [dbo].[myStoredProcedure]
#myFilter varchar(512) = NULL -- don't use NVARCHAR for a list of INTs
AS
SET NOCOUNT ON
BEGIN
SELECT
t.ID,
t.Name
FROM
MyTable t
WHERE
CHARINDEX(','+CONVERT(VARCHAR,t.ID)+',',#myFilter) > 0
ORDER BY
t.Name
END
Performance will be poor. A table scan every time. Better to use a split function. See: http://www.sommarskog.se/arrays-in-sql.html
I would create a function that takes your comma delimited string and splits it and returns a single column table variable with each value in its own row. Select that column from the returned table in your IN statement.
I found a cute way of doing this - but it smells a bit.
declare #delimitedlist varchar(8000)
set #delimitedlist = '|1|2|33|11|3134|'
select * from mytable where #delimitedlist like '%|' + cast(id as varchar) + '|%'
So... this will return all records with an id equal to 1, 2, 33, 11, or 3134.
EDIT:
I would also add that this is not vulnerable to SQL injection (whereas dynamic SQL relies on your whitelisting/blacklisting techniques to ensure it isn't vulnerable). It might have a performance hit on large sets of data, but it works and it's secure.
I have a couple of blog posts on this as well, with a lot of interesting followup comments and dialog:
More on splitting lists
Processing list of integers