a) What do we consider as a dynamic sql statement?
Any sql statement that dynamically adds a clause(s) or even just a part of a clause to a SQL string?
b)Aren’t then parameterized strings that use placeholders for dynamically supplied values also considered dynamic sql statements?
thanx
A dynamic SQL statement is a statement that is built at execution time. The emphasis lies on statement. So, it isn't dynamic SQL if you just supply a value at execution time.
Dynamic SQL statements generally refers those that are constructed using string concatenation.
"SELECT name FROM names WHERE id=" + this.id;
"SELECT name FROM names WHERE id=" + this.id + " AND age=" this.age;
Parameterized queries are also dynamic but not in terms of construct. You can only change parameters but you can't change the structure of the statement i.e add WHERE clauses.
Parameterized queries are often at the database level so the database can cache the execution plan of the query and use it over and over. Not quite possible in the first case since a simple change in the text or the order of the where clauses can cause the database to not recognize the previously cached execution plan and start over.
The first construct is also vulnerable to SQL injection since it is hard to validate input values for attempts to inject rogue SQL.
Certainly anything involving EXEC (#sql) or EXEC sp_ExecuteSQL #sql, ... (i.e. dynamic at the database itself) would qualify, but I guess you could argue that any SQL generated at runtime (rather than fixed at build / install) would qualify.
So yes, you could argue that a runtime-generated, yet correctly parameterized query is "dynamic" (for example, LINQ-to-SQL generated queries), but to be honest as long as it doesn't expose you to injection attacks I don't care about the name ;-p
Dynamic sql is basically just any sql that is not fully constructed until runtime. Its generated on-the-fly by concatenating runtime values into a statement. Could be any part of an sql statement
A. Anything that will cause the DB server to evaluate strings as SQL.
B. No, as they still go through the DB driver/provider and get cleaned up.
For point b) You already know the statement and you pass in known parameters (hopefully type safe and not string literals).
I consider a dynamic SQL statement to be one that accepts new values at runtime in order to return a different result. "New values", by my reckoning, can be a different ORDER BY, new WHERE criteria, different field selections, etc.
a) What do we consider as a dynamic sql statement?
Any sql statement that dynamically adds a clause(s) or even just a part of a clause to a SQL string?
Both - any query altered/tailored prior to execution.
b)Aren’t then parameterized strings that use placeholders for dynamically supplied values also considered dynamic sql statements?
Parameterized queries, AKA using bind variables, are supplying different filter criteria to the query. You can use (my_variable IS NULL OR ...), but OR performance is generally terrible on any db & the approach destroys sargability.
Dynamic SQL generally deals with tailoring the query to include other logic, like JOINs that only need to be included if a specific parameter is set. However, there are limitations like the IN clause not supporting converting a comma delimited string as it's list of options - for this you would have to use dynamic SQL, or handle the comma delimited list in another fashion (CLR, pipelined into a temp table, etc).
I see where you're going with this, but one somewhat objective criteria which defines a particular situation as Dynamic SQL vs. say a prepared statement is...
...the fact that dynamic statements cause the SQL server to fully evaluate the query, to define a query plan etc.
With prepared statements, SQL can (and does unless explicitly asked) cache the query plan (and in some cases, even gather statistics about the returns etc.).
[Edit: effectively, even dynamic SQL statements are cached, but such cached plans have a much smaller chance of being reused because the exact same query would need to be received anew for this to happen (unlike with parametrized queries where the plan is reused even with distinct parameter values and of course, unless "WITH RECOMPILE")]
So in the cases from the question, both a) and b) would be considered dynamic SQL, because, in the case of b), the substitution takes place outside of SQL. This is not a prepared statement. SQL will see it as a totally novel statement, as it doesn't know that you are merely changing the search values.
Now... beyond the SQL-centric definition of dynamic SQL, it may be useful to distinguish between various forms of dynamic SQL, such as say your a) and b) cases. Dynamic SQL has had a bad rep for some time, some of it related to SQL injection awareness. I've seen applications where dynamic SQL is produced, and executed, within a Stored Procedure, and because it is technically "SQL-side" some people tended to accept it (even though SQL-injecting in particular may not have been addressed)...
The point I'm trying to make here is that building a query, dynamically, from various contextual elements is often needed, and one may be better advised to implement this in "application" layers (well, call it application-side layers, for indeed, it can and should be separate from application per-se), where the programming language and associated data structures are typically easier and more expressive than say T-SQL and such. So jamming this into SQL for the sake of calling it "Data-side" isn't a good thing, in my opinion.
Related
If we take a database table, we can query all the rows or we can choose to apply a filter on it. The filter can vary depending on the user input. In cases when there are few options we can specify different queries for those few specific conditions. But if there are lots and lots of options that user might or might not specify, aforementioned method does not come handy. I know, I can compose the filter based upon the user input and send it as a string to the corresponding stored procedure as a parameter, build the query with that filter and finally execute the query string with the help of EXECUTE IMMEDIATE(In Oracle's case). Don't know why but I really don't like this way of query building. I think this way I leave the doors open for SQL injectors. And besides, that I always have trouble with the query itself as everything is just a string and I need to handle dates and numbers carefully.What is the best and most used method of forming the WHERE clause of a query against a database table?
Using database parameters instead of attempting to quote your literals is the way forward.
This will guard you against SQL injection.
A common way of approaching this problem is building expression trees that represent your query criteria, converting them to parameterized SQL (to avoid SQL injection risks), binding parameter values to the generated SQL, and executing the resultant query against your target database.
The exact approach depends on your client programming framework: .NET has Entity Framework and LINQ2SQL that both support expression trees; Java has Hibernate and JPA, and so on. I have seen several different frameworks used to construct customizable queries, with great deal of success. In situations when these frameworks are not available, you can roll your own, although it requires a lot more work.
My question is basically this: If I use a parameterized statement/prepared statement to insert a user input string into a table, then get that value later and use it for dynamically constructing a table's column values, does that leave me open to SQLInjection?
Specific example:
If i store a user's input string into a table using a parameterized statement, then select that TEXT from that table and store it in a local variable (String localVariable) in my program and CREATE a table with something like:
"CREATE TABLE InjectFree (" + localVariable + " TEXT)"
would my localVariable be free of injectable sql code? I know there are alternatives (and will probably use an alternative just to be on the safe side), but I guess I'm just wondering what parameterizing a value actually does and what effect it has on the data being stored in the table.
you will be in danger.
the parameterized insert will protect from injection on that otriginal insert statement, but not from the next use.
If you use parameterised queries the query and the data are supplied to the database separately. This has several effects...
It allows the RDBMS to see that the query is identical to previous instances of that query. (If you embed the data as static values in the query string, the RDBMS will not see that the query is the same and only the data has changed.) This allows execution plan re-use and other beneficial characteristic of the RDBMS.
It simplifies the data validation. This is relevant to injection attacks. No matter what values are substituted into the parameter, the data is always just data. It will never be treated as part of the query.
This latter point, however, also means that you can't do this...
INSERT INTO #tableName(#fieldName) VALUES (#dataValue)
Each parameter is treated as a data item. It isn't a loosely bound script, the value in #tableName won't be substituted into the script. The query must be hard-coded with the table and field names. Only true data items can be passed as parameters.
This often feels like a limitation to users of java script, etc. It is, however, the mecahnism that prtects you from SQL Injection attacks. It's a good thing :)
This means that to allow user defined Data Definition Lanaguage (Such as a CREATE TABLE) you need to concatenate the different parts of the string together yourself. And virtually no matter what you do to protect yourself from a SQL Injection Attack here, some-one will find a way through.
As soon as you allow a user to specify table names, field names, etc, you become immediately open to attack. The only safe way is to have a white-list of allowable strings.
I know this question has been asked and debated before. The answers I have read are mostly opinion, so I am looking for something more technical. More "under the hood".
I'm using SQL Server 2008. I am about to write a query that will need to accept at least 11 inputs. The user could specify 1 to 11(+) of these. They are of various types (IDs, text matching, geography, full text matches, record sets, etc.). It's as dynamic as you can get with SQL.
I know how to do this without using dynamic SQL:
/*for simple match*/ WHERE #Input IS NOT NULL OR t.col = #Input
This has drawbacks because of joins etc. I don't ever use dynamic SQL. But from what I've read, it seems this may be a case for dynamic SQL.
Can anyone give some technical pointers please?
http://www.sommarskog.se/index.html
Look at the curse and blessing of Dynamic SQl and the Dynamic Search Conditions links
You need to remember that if you don't want to use dynamic SQL (and I wouldn't either) then the DRY principle is sorta out-the-window.
Make effective use of TVP's and accept the fact that you will have somewhat high cyclomatic complexity (for a SQL statement)
If you want everything cacheable and sargable, you should consider how to design it in multiple procedures and then break out your conditions as needed.
Another very common approach is to include conditions such as:
WHERE (#pParam IS NULL OR ~condition~)
If your parameter is NULL then SQL will not bother resolving the other half of the condition.
EDIT: This last part only applies in SQL2008R2 and only if using the OPTION(RECOMPILE) command
I am working on a problem that I'm certain someone has seen before, but all I found across the net was how not to do it.
Fake table example and dynamic searching.
(Due to my low rating I cannot post images. I know I should be ashamed!!)
Clicking the add button automatically creates another row for adding more criteria choices.
(Note: My table is most definitely more complex)
Now to my issue, I thought I knew how to handle the SQL for this task, but I really don't. The only examples of what I should do are not meant for this sort of dynamic table querying. The examples didn't have the ability to create as as many search filters as a user pleases (or perhaps my understanding was lacking).
Please let me know if my uploaded image is not of good enough quality or if I have not given enough information.
I'm really curious about the best practice for this situation. Thank you in advance.
I had a similar question. You can use dynamic sql with the sp_executesql stored proc where you actually build your select statement as a string and pass it in.
Or you might be able to write a stored proc kinda like the one I created where you have all of the conditions in the where clause but the NULL values are ignored.
Here's the stored proc I came up with for my scenario:
How do I avoid dynamic SQL when using an undetermined number of parameters?
The advantage with the parameterized stored proc I wrote is that I'm able to avoid the SQL injection risks associated with dynamic SQL.
Two main choices:
Linq to Sql allows you to compose a query, add to it, add to it again, and it won't actually compile and execute a SQL statement until you iterate the results.
Or you can use dynamic SQL. The trick to making this easy is the "WHERE (1=1)" technique, but you do have to be careful to use parameters (to avoid SQL injection attacks) and build your sql statements carefully.
The original post:
Write a sql for searching with multiple conditions
select * from thetable
where (#name='' or [name]=#name) and (#age=0 or age=#age)
However, the above query forces table scan. For better performance and more complex scenario (I guess you simplified the question in you original post), consider use dynamic sql. By the way, Linq to SQL can help you build dynamic SQL very easily, like the following:
IQueryable<Person> persons = db.Persons;
if (!string.IsNullOrEmpty(name)) persons = persons.Where(p=>p.Name==name);
if (age != 0) persons = persons.Where(p=>p.Age=age);
Check out SqlBuilder, a utility for Dynamic SQL.
I'm overloading a vb.net search procedure which queries a SQL database.
One of the older methods i'm using as a comparison uses a Stored Procedure to perform the search and return the query.
My new method uses linq.
I'm slightly concerned about the performance when using contains queries with linq. I'm looking at equally comparable queries using both methods.
Basically having 1 where clause to
Here are some profiler results;
Where name = "ber10rrt1"
Linq query : 24reads
Stored query : 111reads
Where name = "%ber10%"
Linq query : 53174reads
Stored proc query : 23386reads
Forgetting for a moment, the indexes (not my database)... The fact of the matter is that both methods are fundamentally performing the same query (albeit the stored procedure does reference a view for [some] of the tables).
Is this consitent with other peoples experiance of linq to sql?
Also, interestingly enough;
Using like "BER10%"
resultset.Where(Function(c) c.ci.Name.StartsWith(name))
Results in the storedproc using 13125reads and linq using 8172reads
I'm not sure there is enough there for a complete analysis... I'm assuming we are talking about string.Contains/string.StartsWith here (not List<T>.Contains).
If the generated TSQL is similar, then the results should be comparable. There are a few caveats to this - for example, is the query column a calculated+persisted value? If so, the SET options must be exact matches for it to be usable "as is" (otherwise it has to re-calculate per row).
So: what is the TSQL from the SP and LINQ? Are they directly comparable?
You mention a VIEW - I'm guessing this could make a big difference if (for example) it filters out data (either via a WHERE or an INNER JOIN).
Also - LIKE clauses starting % are rarely a good idea - not least, it can't make effective use of any index. You might have better performance using "full text search"; but this isn't directly available via LINQ, so you'll have to wrap it in an SP and expose the SP via the LINQ data-context (just drag the SP into the designer).
My money is on the VIEW (and the other code in the SP) being the main difference here.