I am working on some reports (counts) and I have to fetch counts for different parameters. Pretty simple but tedious.
A sample query for one parameter :
qCountsEmployee = (
"select count(*) from %s where EmployeeName is not null"
% (tablename)
)
CountsEmployee = execute_query(qCountsEmployee)
Now I have few hundred such parameters!
What I did was : create a list of all the parameters and generate them using a quick Python script and then copy that text and put it in the main script to avoid the tedious lines.
columnList = ['a', 'b', ............'zzzz']
for each in columnList:
print (
'q' + each + ' ='
+ '"select count(*) from %s where' + each
+ 'is not null" % (tablename)'
)
print each + ' = execute_query(' + 'q' + each + ')'
While this approach works, I was wondering if instead of a separate script to generate lines of code and copy paste into main program, can I generate them in the main script directly and let the script treat them as lines of code? That will make the code much more readable is what I think. Hope I made sense! Thank you...
It would be possible, but is not useful here.
Just do something like
columnList = ['a', 'b', ............'zzzz']
results = {}
for column in columnList:
query = (
"select count(*) from " + tablename
+ " where " + column + " is not null"
)
result = execute_query(qCountsEmployee)
results[column] = result
You as well can put all this together in a generator function and do
def do_counting(column_list):
for column in column_list:
query = (
"select count(*) from " + tablename
+ " where " + column + " is not null"
)
result = execute_query(qCountsEmployee)
yield column, result
result_dict = dict(do_counting(['...']))
You can do:
cmd = compile( 'a = 5', '<string>', 'exec' )
exec( cmd )
That is the same as just writing:
a = 5
The string passed as the first argument to compile can be built dynamically.
To build on what glglgl said, you are probably better with dynamic SQL than dynamic Python (though Dynamic Python is definitely possible using things like eval ). When working with Dynamic SQL, you should be careful of sql injection. It seems like it would not come up in your particular use-case, but it certainly comes up more often than many developers realize.
I happen to have written an article about SQL Injection and Python which is available at Simple-talk.
Related
I'm working on a search tool, the words (to be searched) are in object array
first i search the database(about 450k rows) using FTS4 and sqlite. This works fast, the SQL im using is:
String s = "select * from fts_sggs where word_text MATCH ? ";
but then, i need some extra words which are a part of the same table.
The FTS4 doesnt seem to work here.the sql is
String s ="select * from fts_sggs where word_number BETWEEN " + wordnumlo + " AND " + wordnumhi + " and page_number =" + pgnum;
im a newbie to SQL Programming, I need a single query that could perform both tasks fast. this is being done in java which is terribly slow.
lcVillkor1 = "table.numbers > 1"
SELECT * FROM table WHERE lcVillkor1 ORDER BY table.numbers
I got the error - SQL: WHERE clause is invalid.
I've tried almost every possible combination
"table.numbers > 1", (table.numbers > 1), "(table.numbers > '1')" etc..
Im trying to get some posts (where a number is greater then 1) from a table to print out.
Use & before the variable to expand it:
lcVillkor1 = "table.numbers > 1"
SELECT * FROM table WHERE &lcVillkor1 ORDER BY table.numbers
You can't just put a variable name in a line of code and have it execute. Instead, build up your SQL command as a text variable, and then use something like macro substitution to execute it:
lcVillkor1 = "table.numbers > 1"
lcSql = "SELECT * FROM table WHERE " + lcVillkor1 + " ORDER BY table.numbers"
&lcSql
Also, for future reference, perhaps table and field names close to reserved words - eg "table" and "numbers" - may not be ideal.
Macro substitution works fine (in both the examples given previously). If you use VFP9 you can also use the function EXECSCRIPT() after you've built the query string:
lcVillkor1 = "table.numbers > 1"
lcSql = "SELECT * FROM table WHERE " + lcVillkor1 + " ORDER BY table.numbers"
EXECSCRIPT(lcsql)
Haven't come across this in ages and when I searched for the solution I couldn't find one. I think its called overloading in SQL. Basically when I have "" (an empty string) for any parameter in this SQL I don't want to set a value in the database...
NOTE: I want to do it at a SQL level not do it at a C# level because its sloppy that way.
string Sql = "IF NOT EXISTS (SELECT * FROM tbl_FileSystemReferences) "
+ "INSERT INTO tbl_FileSystemReferences (UploadDir) VALUES (null) "
+ "UPDATE tbl_FileSystemReferences SET "
+ "UploadDir=#UploadDir, "
+ "ThumbnailDir=#ThumbnailDir, "
+ "ArchiveDir=#ArchiveDir, "
+ "RealDir=#RealDir, "
+ "FlashDir=#FlashDir, "
+ "AssociatedFilesDir=#AssociatedFilesDir, "
+ "EnableArchiving=#EnableArchiving, "
+ "AppWideDir=#AppWideDir, "
+ "FFmpegDir=#FFmpegDir, "
+ "InstallationDir=#InstallationDir ";
SqlCommand Command = new SqlCommand(Sql);
Command.Parameters.AddWithValue("#UploadDir", f.UploadDir);
Command.Parameters.AddWithValue("#ThumbnailDir", f.ThumbnailDir);
Command.Parameters.AddWithValue("#ArchiveDir", f.ArchiveDir);
Command.Parameters.AddWithValue("#RealDir", f.RealDir);
Command.Parameters.AddWithValue("#FlashDir", f.FlashDir);
Command.Parameters.AddWithValue("#AssociatedFilesDir", f.AssociatedFilesDir);
Command.Parameters.AddWithValue("#EnableArchiving", f.EnableArchiving);
Command.Parameters.AddWithValue("#AppWideDir", f.AppWideDir);
Command.Parameters.AddWithValue("#FFmpegDir", f.FFmpegDir);
Command.Parameters.AddWithValue("#InstallationDir", f.InstallationDir);
ExecuteNonQuery(Command);
I know there is a way I used to do this with stored procedure I just cant remember how (I think it's called overloading)....
Cheers,
Can you create a stored procedure rather than passing the command as text?
That way you can break each of the lines like "UploadDir=#UploadDir," into its own variable and only add it to the command if it is not null or not empty string
one way would be on a stored procedure, where you would receive all those parameters, then before the query either:
you allow to pass null
you convert each parameter to null if they are empty as:
select #UploadDir = null where #UploadDir = ''
you would do that for all your parameters, then on update query:
IF NOT EXISTS (SELECT * FROM tbl_FileSystemReferences)
INSERT INTO tbl_FileSystemReferences (UploadDir) VALUES (null)
UPDATE tbl_FileSystemReferences SET
UploadDir=coalesce(#UploadDir, UploadDir),
ThumbnailDir=coalesce(#ThumbnailDir, ThumbnailDir),
ArchiveDir=coalesce(#ArchiveDir, ArchiveDir),
RealDir=coalesce(#RealDir, RealDir),
FlashDir=coalesce(#FlashDir, FlashDir),
AssociatedFilesDir=coalesce(#AssociatedFilesDir, AssociatedFilesDir),
EnableArchiving=coalesce(#EnableArchiving, EnableArchiving),
AppWideDir=coalesce(#AppWideDir, AppWideDir),
FFmpegDir=coalesce(#FFmpegDir, FFmpegDir),
InstallationDir=coalesce(#InstallationDir, InstallationDir)
I have the following table in a SQLite3 database:
CREATE TABLE overlap_results (
neighbors_of_annotation varchar(20),
other_annotation varchar(20),
set1_size INTEGER,
set2_size INTEGER,
jaccard REAL,
p_value REAL,
bh_corrected_p_value REAL,
PRIMARY KEY (neighbors_of_annotation, other_annotation)
);
I would like to perform the following query:
SELECT * FROM overlap_results WHERE
(neighbors_of_annotation, other_annotation)
IN (('16070', '8150'), ('16070', '44697'));
That is, I have a couple of tuples of annotation IDs, and I'd like to fetch
records for each of those tuples. The sqlite3 prompt gives me the following
error:
SQL error: near ",": syntax error
How do I properly express this as a SQL statement?
EDIT I realize I did not explain well what I am really after. Let me try another crack at this.
If a person gives me an arbitrary list of terms in neighbors_of_annotation that they're interested in, I can write a SQL statement like the following:
SELECT * FROM overlap_results WHERE
neighbors_of_annotation
IN (TERM_1, TERM_2, ..., TERM_N);
But now suppose that person wants to give me pairs of terms if the form (TERM_1,1, TERM_1,2), (TERM_2,1, TERM_2,2), ..., (TERM_N,1, TERM_N,2), where TERM_i,1 is in neighbors_of_annotation and TERM_i,2 is in other_annotation. Does the SQL language provide an equally elegant way to formulate the query for pairs (tuples) of interest?
The simplest solution seems to be to create a new table, just for these pairs,
and then join that table with the table to be queried, and select only the
rows where the first terms and the second terms match. Creating tons of AND /
OR statements looks scary and error prone.
I've never seen SQL like that. If it exists, I would suspect it's a non-standard extension. Try:
SELECT * FROM overlap_results
WHERE neighbors_of_annotation = '16070'
AND other_annotation = '8150'
UNION ALL SELECT * FROM overlap_results
WHERE neighbors_of_annotation = '16070'
AND other_annotation = '44697';
In other words, build the dynamic query from your tuples but as a series of unions instead, or as a series of ANDs within ORs:
SELECT * FROM overlap_results
WHERE (neighbors_of_annotation = '16070' AND other_annotation = '8150')
OR (neighbors_of_annotation = '16070' AND other_annotation = '44697');
So, instead of code (pseudo-code, tested only in my head so debugging is your responsibility) such as:
query = "SELECT * FROM overlap_results"
query += " WHERE (neighbors_of_annotation, other_annotation) IN ("
sep = ""
for element in list:
query += sep + "('" + element.noa + "','" + element.oa + "')"
sep = ","
query += ");"
you would instead have something like:
query = "SELECT * FROM overlap_results "
sep = "WHERE "
for element in list:
query += sep + "(neighbors_of_annotation = '" + element.noa + "'"
query += " AND other_annotation = '" + element.oa + "')"
sep = "OR "
query += ";"
I'm not aware of any SQL dialects that support tuples inside IN clauses. I think you're stuck with:
SELECT * FROM overlap_results WHERE (neighbors_of_annotation = '16070' and other_annotation = '8150') or (neighbors_of_annotation = '16070' and other_annotation = '44697')
Of course, this particular query can be simplified to something like:
SELECT * FROM overlap_results WHERE neighbors_of_annotation = '16070' and (other_annotation = '8150' or other_annotation = '44697')
Generally SQL WHERE-clause predicates only allow filtering on a single-column.
code:
string query1 = #"UPDATE global_mapping set escape_id = " +
dataGridView1.Rows[i].Cells[2].Value + ",function_id = " +
dataGridView1.Rows[i].Cells[3].Value + ",function_name = '" +
dataGridView1.Rows[i].Cells[4].Value + "',parameter_name = '" +
dataGridView1.Rows[i].Cells[5].Value + "',parameter_validity = '" +
dataGridView1.Rows[i].Cells[6].Value + "',statusparameter_id = " +
dataGridView1.Rows[i].Cells[7].Value + ",acb_datatype = '" +
dataGridView1.Rows[i].Cells[8].Value + "',data_type_id = " +
dataGridView1.Rows[i].Cells[9].Value + ",bit_size = " +
dataGridView1.Rows[i].Cells[10].Value + ",validity_status ='" +
dataGridView1.Rows[i].Cells[11].Value + "',validity_func = '" +
dataGridView1.Rows[i].Cells[12].Value + "'WHERE global_mapping.parameter_id =" +
dataGridView1.Rows[i].Cells[1].Value + "";
OleDbCommand cmd1 = new OleDbCommand(query1, conn);
cmd1.ExecuteNonQuery();
code ends:
When I execute the above code I get an error stating "Syntax error in Update statement".
Can someone please tell me how to resolve this?
It looks like you need to add a space before your WHERE clause.
Hope this helps,
Bill
Wow. Can we say... SQL Injection?
Try using Parameters. Not only will you protect yourself, but your SQL will become MUCH more readable.
Never use string concatenation for building SQL queries. Use SQL parameters.
Yikes!
Please provide the final query1 value and try to format it so we can get a better picture of it. My guess is a missing ' or something.
I'd say you're missing some quotes in there but your code is such a pig-sty I can't tell. If you won't fix your code then at the minimum give us a dump of query1 so we can read your actual query.
And use parameters or stored procedures like the previous responses said. All it takes is one of your variables to get overwritten with something nasty and your server will be wide open to anyone deleting your tables or worse.
Even if this is a local "safe" database you should unlearn your bad habits now.
Put
Console.WriteLine(query1)
before OleDbCommand cmd1 = new OleDbCommand(query1, conn);
See the value of query1 printed to console window.
Does the SQL Statement look OK? I guess not - you will now be able to find a field which is non-numeric and is blank in the grid.
And, use parameters as others have said.