I use tons of dynamic SQL -
I figured that there must some good guidelines, frameworks and/or tools to help one use dynamic SQL queries.
I'm looking for any suggestions on how exactly should one compose dynamic SQL query (with out the obvious solution of simply writing it, then adding ' ' ect).
The big problem here that sometimes it gets way to messy (dynamic sql that contains another dynamic sql ect).
If it matters at all, I am using sql-server.
I'll take any advice I can get,
Thanks! ;)
Vague question.. But one piece of advice:
Use sp_executesql and pass in any variables/parameters whenever possible to prevent SQL injection
If possible generate the dynamic SQL as much as possible, don't write it yourself.
Execute the dynamic SQL without appending the parameters to the SQL string, use parameters and pass those to sp_executesql to avoid having to do double quoting (too tedious). It is also a solid guard against SQL Injection. Example:
DECLARE #stmt NVARCHAR(MAX)='SELECT * FROM your_table WHERE id=#par1 AND ... AND thename=#parn;';
EXECUTE sp_executesql #stmt, N'#par1 INT, ..., #parn VARCHAR(256)', #par1, ... , #parn;
Related
I've decided to use postgreSQL as database for a game project(C++).
At the moment I'm using Oracle and it's Pro*C precompiler at work and heard that postgreSQL also has something similar called ECPG.
It's also possible to access data from the the postgres database directly by using the SQL in a string.
So the difference between "normal" and using ECPG, is that you can write your SQL statements like code?, or are there any other differences I should be aware of?.
(PS: i know I'm using it at work, but I haven't noticed any other differences)
Looking forward to hearing from you guys.
Yes, ECPG is covered in the documentation.
So the difference between "normal" and using ECPG, is that you can
write your SQL statements like code?
Well, SQL statements are code. A SQL statement just looks like a SQL statement. This is what a CREATE TABLE statement might look like in ECPG.
EXEC SQL CREATE TABLE foo (number integer, ascii char(16));
ECPG allows variable substitution. (Maybe that's what you meant by "write your SQL statements like code".)
EXEC SQL INSERT INTO sometable VALUES (:v1, 'foo', :v2);
All this stuff is in the documentation.
I've got the SQL stored procedure from hell that I've created and all input parameters are parameterised for security but it's not running as quick as I'd like so I wanted to make it dynamic and so a bit more efficient.
I know I can keep my input parameters to my stored procedure, then within it create a dynamic SQL statement into which I can then pass the input parameters of the stored procedure, but are there any security implications I need to be aware of when doing this? I'm guessing not as it just another set of parameters and they should be treated the same as the parameters passed to the current stored procedure.
Obviously, producing code like this "WHERE OrderNo = ' + #orderno is asking for trouble - I will be doing 'WHERE OrderNo = #orderno' in the dynamic SQL, but is there anything else I need to be aware of?
Thx MH
PS - before anyone suggests it, I can't create the SQL dynamically at the client side using LINQ or similar - it all (for various reasons) has to be contained and controlled at the database level
There is a form of SQL injection that many people don't think about when doing dynamic SQL in stored procedures: SQL Truncation attacks.
With a SQL truncation attack, the attacker injects a long peace of text making the used text variable overflow and lose part of the query.
This article gives more information about this.
Where your parameters are always Data Items, both when being passed to the StoredProc and when used in yor DynamicSQL, everything will stay safe.
Should any of your StoredProc's parameters end up being table or field names, and so forming part of the structure of the DynamicSQL itself, you introduce a new risk : That the parameter can be used to inject rogue SQL Code.
To prevent against such an injection attack you should always validate any such parameters.
One example of how to do this would be to use the input parameter as a token, rather than substitute it directly into the DynamicSQL...
SET #SQL = #SLQ + CASE targetTable WHEN '1' THEN 'table1'
WHEN 'tx' THEN 'tableX'
END
Some people suggest you only need to validate on the client application. But that means that if someone becomes able to execute you SP's directly, the SP has become a point of attack. I always prefer to validate both on the client AND in the server.
EDIT Performance
Note that using DynamicSQL isn't always a guarnatee of performance increases. If you use parameterised queries, the execution plans can indeed be stored. But if the queries do vary greatly, you may still find a significant overhead in compiling the SQL.
There is also the fact that dependancy tracking is lost. It's not possible to see what tables the SP is dependant on, because the code is hidden away as strings.
I have very rarely found that DynamicSQL is needed. Often a complex query can be reformed as several optimised queries. Or the data can be re-structured to meet the new demands. Or even a rethink of both the data and the algorithm using the data. One might even be able to suggest that a dependancy on DynamicSQL is an indicator of another underlying problem.
Perhaps it's not in the scope of your question, but it would be interesting to see the actual puzzle you're facing; to see if anyone has any alternative approaches for you.
I have a question regarding SQL. I have the following SQL statement:
SELECT id, First, Last, E_Mail, Notes
FROM mytable
WHERE SOMETHING_SHOULD_BE_HERE IS NOT NULL;
I know that the SOMETHING_SHOULD_BE_HERE should be a column(attribute) in my table. Is their a way I can put a variable that can refer to the column I'm trying to access? In my case their are 30 columns. Can I have a string for SOMETHING_SHOULD_BE_HERE that can be assigned in my program to the column in which I want to search?
Thanks
No. Variables in SQL can refer to data, but not to object names (columns, functions or other database objects).
If you are building the SQL query, you'll need to use string operations to build your query.
The column can't be variable, but the value of the column can. The parser needs to know what to bind to.
If you elaborate on what you're trying to solve and which platform you're using it would allow for more complete answers.
You can have different SQLs queries in your code and use each one according to the case.
Another way is generate dynamically the query according the fields you want.
Without dynamic SQL, this is probably your best bet:
SELECT
id, first, last, email, notes
FROM
My_Table
WHERE
CASE #column_name_variable
WHEN 'column_1' THEN column_1
WHEN 'column_2' THEN column_2
...
ELSE 'not null'
END IS NOT NULL
There might be some issues with data type conversions, so you might need to explicitly cast all of the columns to one data type (VARCHAR is probably the best bet). Also, there's a good chance that performance will be horrendous on this query. I'd test it thoroughly before even thinking about implementing something like this.
I mentioned this in my comment, but for completeness I'll put it here too... you can probably also accomplish this with dynamic SQL, but how you do that will depend on your database server (MS SQL Server, Oracle, mySQL, etc.) and there are usually some caveats to using dynamic SQL.
In JDBC program, yes,the select statement can be composed like string operation.
for(String colName: colList)
{
String sql="Select id, First, Last, E_Mail, Notes From mytable where "+colName+" IS NOT NULL";
//execute the sql statement
}
It depends on how you are going to find out the value of SOMETHING_SHOULD_BE_HERE.
If you are in an Oracle PLS/SQL environment you could build up the WHERE clause using dynamic SQL and then use EXECUTE IMMEDIATE to execute it.
If you have a small set number of possibilities you could use CASE to workaround your problem possibly.
Your question is unclear.
However I am quite sure that what you have in mind is the so-called dynamic SQL (and related). "Dynamic SQL" allows you to dynamically build and submit queries at runtime. However such functionalities may not exist for your RDBMS.
There are several ways to do this.
When your query would return one and only one row
then you have to consider the EXECUTE IMMEDIATE statements (along with sp_executesql in tSQL : http://msdn.microsoft.com/en-us/library/ms188001.aspx ; or the USING clause in PL/SQL : http://docs.oracle.com/cd/B14117_01/appdev.101/b10807/13_elems017.htm to specify a list of input/output bind arguments) and/or PREPARED statements (http://rpbouman.blogspot.fr/2005/11/mysql-5-prepared-statement-syntax-and.html).
When your query can return more than one row
then you have to consider techniques such as the EXECUTE IMMEDIATE statement with the BULK COLLECT INTO clause or the OPEN-FOR, FETCH, and CLOSE statements (explicit cursors in PL/SQL :
http://docs.oracle.com/cd/B28359_01/appdev.111/b28370/dynamic.htm)
Please note that except in some particular cases, most conventional techniques like IF-THEN-ELSE and CASE statements should be preferred (along with a good algorithm). Furthermore they work with almost all RDBMS.
I'm using sqlite3_exec() function in order to execute an SQL Insert command. The problem starts when I need to insert strings that need to be encoded.
For example, I want to insert the following string: "f('hello')". If I want to insert this string I need to change "'" to "''".
My question is, how do I encode these strings? Is there a function I can count on? or a table that details all the needed encodes?
Thanks! :-)
Instead of manually escaping strings (which is error-prone and invites SQL injection attacks), I'd strongly recommend using prepared statements and bind values; read up on sqlite3_bind_XXX and sqlite3_prepare_v2
Using bind values will solve this problem and it will also make sqlite faster because it remembers previously executed sql statements and it can reuse their execution plans. This doesn't work when the sql statement is always slightly different because it hashes the complete sql statement.
sqlite_mprintf supports %q for that.
"Maybe" you should use something like a prepared statement. I am not an expert in SQLite, but I found this link (http://www.sqlite.org/c3ref/stmt.html) and it could help you. It is about SQL Statement Object.
As the topic suggests I wish to be able to pass table names as parameters using .NET (doesn't matter which language really) and SQL Server.
I know how to do this for values, e.g. command.Parameters.AddWithValue("whatever", whatever) using #whatever in the query to denote the parameter. The thing is I am in a situation where I wish to be able to do this with other parts of the query such as column and table names.
This is not an ideal situation but it's one I have to use, it's not really prone to SQL injection as only someone using the code can set these table names and not the end-user. It is messy however.
So, is what I am asking possible?
EDIT: To make the point about SQL injection clear, the table names are only passed in by source code, depending on the situation. It is the developer who specifies this. The developer will have access to the database layer anyway, so the reason I am asking is not so much for security but just to make the code cleaner.
You cannot directly parameterize the table name. You can do it indirectly via sp_ExecuteSQL, but you might just as well build the (parameterized) TSQL in C# (concatenating the table-name but not the other values) and send it down as a command. You get the same security model (i.e. you need explicit SELECT etc, and assuming it isn't signed etc).
Also - be sure to white-list the table name.
I don't think I've ever seen this capability in any SQL dialect I've seen, but it's not an area of expertise.
I would suggest restricting the characters to A-Z, a-z, 0-9, '.', '_' and ' ' - and then use whatever the appropriate bracketing is for the database (e.g. [] for SQL Server, I believe) to wrap round the whole thing. Then just place it directly in the SQL.
It's not entirely clear what you meant about it not being a SQL injection risk - do you mean the names will be in source code and only in source code? If so, I agree that makes things better. You may not even need to do the bracketing automatically, if you trust your developers not to be cretins (deliberately or not).
You can pass the table name as a parameter like any other parameter. the key is you have to build a dynamic sql statement, which then you should consider if it's easier to build it in your app tier or in the procs.
create procedure myProc
#tableName nvarchar(50)
as
sp_executesql N'select * from ' + #tablename
fyi this code sample is from memory have a look at BOL for the proper syntax of sp_executesql.
Also this is highly sucesptible to SQL injection as you indicated is not an issue for you but anyone reading this should be very wary of accepting input from a user to generate their queries like this.
SQL query parameters can only take the place of a literal value. You cannot use a parameter for a table name, column name, list of values, or other SQL syntax. That's standard SQL behavior across all brands of database.
The only way to make the table name dynamic is to interpolate a variable into your SQL query before you prepare that string as a statement.
BTW, you're fooling yourself if you think this isn't a risk for SQL injection. If you interpolate the table name into the query dynamically, you need to use delimited identifiers around the table name, just as you would use quotes around a string literal that is interpolated from a variable.
The idea that it is not prone to SQL injection is misguided. It may be less prone to SQL injection from front end users, but it is still very much prone to SQL injection. Most attacks on databases come from inside the company being attacked, not from end users.
Employees may have grudges, they may be dishonest, they may be disgruntled, or they may just be not so bright and think that it's ok to bypass security to do whatever it is that THEY think should be done to the database.
Please see this post answer by user Vimvq1987:
MySqlParameter as TableName
Essentially you first check the table name against the schema, in which the table name is used in a parameterized fashion. Then if all is ok, the table name is legit.
Paraphrased basic idea is:
SELECT table_name
FROM information_schema.tables
WHERE table_schema = 'databasename'
AND table_name = #table;
cmd.Parameters.AddWithValue("#table",TableName);
If this returns ok with the table name, go ahead with your main query...
I would just check
select OBJECT_ID(#tablename)
the idea is to prevent injection you know it has to be table name this was if this returns a number then i would run the actual query,