I have a stored procedure that accepts an array argument and runs a dynamically created query using execute immediate. In the dynamic query, I would like to pass the array along with a set of columns in each row to a stored function.
For any other type of argument, this would be straightforward. For an integer, I would use format as follows:
execute immediate format(""" SELECT %d*value1 as scaled_value FROM (...) """)
But there does not seem to be a format specifier for arrays. One way to do this would be for me to write a pretty printer for arrays, which loops through an array and constructs the corresponding string [a, b, c, ...] but that approach seems cumbersome. Is there a supported way of doing this?
As mentioned by #Jaytiger , for using an array inside an execute immediate statement, %t can be used inside the FORMAT function. You can use it as FORMAT('%t', ['a', 'b', 'c']).
Related
I am querying a postgres db using python/pandas with sqlalchemy. I have an sql query that looks like this:
SELECT table_name
FROM fixed_602fcccd0f189c2434611b14.information_schema."tables" t
WHERE table_type='BASE TABLE'
and table_schema='di_602fccd10f189c2434611be9'
and (
table_name like 'general_journal%'
or table_name like 'output_journal_line%'
or table_name like 'output_journal_tx%'
or table_name like 'flats%'
)
I've tested it in dBeaver and it works perfectly. I am now trying to pass the same query through pandas read_sql as follows:
from_string = pg_db + '.information_schema."tables"'
print(from_string)
pg_query = queries.id_tables.format(from_string,di_id)
The idea is that I construct the query with variables 'pg_db' (string) and 'di_id' (string) as I make a series of queries. The problem is the query returns empty array when done this way. No error is thrown.
I suspected the challenge is the "tables" attribute that when pandas interprets the query eg. strips off the ", but that doesn't actually seem to matter. Any thoughts on how to make this work with pandas?
UPDATE:
I have tried parameterized and met with the same problem. It seems to boil down to the FROM parameter gets passed in with double quotes. I have tried to strip these but it looks like pandas appends them anyways. In principle double quotes should be fine according to postgres docs but that doesn't seem to be the case even when doing the query in dBeaver. If I pass in the query via pandas as it is written at the top of this post, no problem. The challenge is when I try to use variables for the FROM and table_schema parameters, I get syntax errors.
It turns out that the problem disappeared when I removed the parentheses I put around the 'or' statements. I think the message is to pay attention to how you construct the query eg. form and join all the strings and variables before passing them to pandas.
That said I have used parentheses with much more complex queries in pandas and they were not a problem.
I would first suggest that you use a parameterized query for input but in some cases its just easier to use a built in function repr()
s = "SQL sometimes likes \"here\" and %s \"now\" problems"
print(repr(s))
gives
'SQL sometimes likes "here" and %s "now" problems'
I'm reading sql code which has a line that looks like this:
SELECT INTO _user tag FROM login.user WHERE id = util.uuid_to_int(_user_id)::oid;
What exactly does this do? The usual way to use SELECT INTO requires specifying the columns to select after the SELECT token, e.g.
SELECT * INTO _my_new_table WHERE ...;
The database is postgresql.
This line must appear inside of a PL/pgSQL function. In that context the value from column tag is assigned to variable _user.
According to the documentation:
Tip: Note that this interpretation of SELECT with INTO is quite different from PostgreSQL's regular SELECT INTO command, wherein the INTO target is a newly created table.
and
The INTO clause can appear almost anywhere in the SQL command. Customarily it is written either just before or just after the list of select_expressions in a SELECT command, or at the end of the command for other command types. It is recommended that you follow this convention in case the PL/pgSQL parser becomes stricter in future versions.
I would want to add an optional parameter to my stored procedure with default *. If the list of columns is provided [delimited by a comma] these columns should be returned back by the procedure. If the wildcard character is provided [star] *, all columns should be returned. Please let me know how to implement it.
First thing - why stored procedure not table UDF?
Anyway it would be easier to pass null instead of "*" - tsql allows default values on UDF parameters.
You would have to construct query dynamically and then use sp_executesql().
The issue is that you should validate columns list to prevent errors.
I actually want to execute an SQL query in a C program, with the variables in a SELECT query to be the value stored in a string variable.
For example:
void fetch_data(char var[])
{
char COL1[]=var, COL2[]="Address", COL3[]="Name";
SELECT COL1, COL2, COL3 FROM TABLE WHERE COL4='some value';
}
Here as you can see I want want my code to be flexible so that I can have different column names depending on the variable var which is the parameter of the fetch_data function.
Please tell me if this is possible in C.
I have thought of another method if the above is not possible: can we store the whole SQL statement in a string and execute it, so that I can modify this string whenever I want according to the parameter's value that I get in the function fetch_data()?
The code below will make my point more clear that what I want:
void fetch_data(char var[])
{
char COL1[]="Name", COL2[]="Address", COL3[]=var;
char qry1[]="SELECT ", qry2[]=var, qry3=" COL2, COL3 FROM TABLE WHERE COL4='some value';";
char str[]=strcat(qry1,qry2);
char query[]=strcat(str,qry3);
//now query will be having "select (value of var), COL2, COL3 FROM TABLE WHERE COL4='some value';
}
Now in the above code, can I execute the query that is stored in the string query?
Please let me know if any of the 2 methods can work or if it can be achieved by any other method in 'C'.
The second method will certainly work, you'll just need to be careful with your string manipulation.
In fact you could simplify it using snprintf:
snprintf(queryStr, MAX_QUERY_LENGTH,
"SELECT %s, COL2, COL3 FROM TABLE WHERE COL4='some value';", var);
If the query needs to be more dynamic, i.e. 'some value' also needs to come from a variable, you can add additional format specifiers:
snprintf(queryStr, MAX_QUERY_LENGTH,
"SELECT %s, COL2, COL3 FROM TABLE WHERE COL4='%s';", var, someValue);
Once you've prepared such a query, just how you use it depends upon your toolchain. Are you using ODBC? If so, then you can use ODBC calls to handle your queries, and ODBC drivers to manage the connection to your database. Otherwise, which database are you using, and what API is provided to you? For instance, MySQL provides a pretty large C API, oracle a different one, and SQL Server another.
As for the first option, you'll need to make use of a SQL preprocessor to use something like raw SQL statements in your C code, and the syntax will depend upon which tool that you use.
I should also add that you can't copy non-literal C strings using a statement like: char COL1[]=var;. Instead you need to use a string library method like strncpy.
Essentially I a SSIS pkg with an Execute SQL statement that dynamically writes a REPLACE function based on some table values. (ie REPLACE(REPLACE(Col1,"*","",),"###","")
ExecuteSQL result is put to variable #Cleanse
In my Derived Column conversion Im trying to call #User::Cleanse as an expression in to replace the Value of the Col1 from the DataFlow.
The result appears to be pulling the result of #Cleanse and using it as a string value rather than applying it as the REPLACE function.
When Debugging the #Cleanse value appear to be putting \ in the string, which I dont think should matter as it seems to be appliying this to other values without a problem
The result is that when running out to the CSV this is putting
"REPLACE(REPLACE(Col1" in Col1
"*" in Col2
) in Col3 etc etc
How can I get the Derived column transformation to 'see' the variable as the function, not a string value?
Many thanks in advance
Set the EvaluateAsExpression property of the variable to True.
However, binding variables as parameters to an SQL query using the Variable Mappings pane of the Execute SQL task might be a better solution