Use macro variables in Proc SQL in SAS - sql

Two macro variables are defined for table schema and table name and they are used in the proc sql statement in SAS (postgresql interface to Amazon RedShift), sql statement can't be read correctly.
%let table_schema = 'sales';
%let table_name = 'summary_table';
proc sql;
connect to ODBC(DSN='redshift_db', user=masteruser password='*pwd');
create table column_names as
select * from connection to odbc(
select distinct(column_name) as col_name from information_schema.columns where table_schema = &table_schema and table_name = &table_name;
);
create table dt as
select * from connection to odbc(
select * from &table_schema..&table_name;
);
QUIT;
The first table creation throws an error:
ERROR: CLI describe error: ERROR: column "summary_table" does not exist in columns;
The "summary_table" actually exists.
The second table creation throws an error:
ERROR: CLI describe error: ERROR: syntax error at or near "'sales'"; No query has been executed with that handle
which is invalid either.

Check your syntax.
Look at the ERROR: column "summary_table" does not exist in columns;
What that is saying is that the COLUMN named "summary_table" does not exist on the table named "columns". The where clause is trying to compare 2 columns, not a column and a string.
Write the query without macros and get that to work. Then attempt to put the macros in.
This is a syntax issue with the PostGreSQL ODBC driver, not SAS.

I haven't used the PostGreSQL driver before but I know different DBs handle quotes differently. It may be that PostGreSQL wants double quotes instead of single. If this is the case then I would change your code like so:
%let table_schema = sales;
%let table_name = summary_table;
proc sql;
connect to ODBC(DSN='redshift_db', user=masteruser password='*pwd');
create table column_names as
select * from connection to odbc(
select distinct(column_name) as col_name from information_schema.columns where table_schema = "&table_schema" and table_name = "&table_name";
);
create table dt as
select * from connection to odbc(
select * from &table_schema..&table_name;
);
QUIT;
Note that I've removed the single quotes from the %let statments and added double quotes to the where clause in the first query.
Removing the singles quotes from the macro vars will also fix the syntax error you are experiencing with the second query as you were effectively trying to do the following:
select * from 'sales'.'summary_table';
... which is most likely not correct. Again I haven't used PostGreSQL so I can't say for certain - please correct me if I'm wrong and that is correct syntax for it.
Finally, I'd recommend that when working with macro variables in general, that you don't put single quotes around your strings. ie.
%let table_schema = 'sales'; * AVOID THIS WHEN POSSIBLE;
vs
%let table_schema = sales; * USE THIS APPROACH WHEN POSSIBLE;
Put double quotes around the places where macro vars are used instead and you'll probably find things a little easier. Of course there's always circumstances where you may want quotes in the value of the macro variable, but as a general rule of thumb I avoid doing this whenever possible.

Related

deleting tables from postgresql without raising cross-database references are not implemented: using pandas/psycopg2

I am trying to drop a table from a database.
As long as name_Table is a structured as
schema.table
it all works nicely. However, I do have one table in public schema.
When I try to delete it as:
public.subname.table
I get this answer:
cross-database references are not implemented: "public.subname.table"
How to drop public.subname.table?
print('Connecting to the PostgreSQL database...')
postgresConnection = psycopg2.connect(
host=XXXXXX,
port=YYYYYYYYY,
database="mydb",
user=os.environ['user'],
password=os.environ['pwd'])
cursor = postgresConnection.cursor()
dropTableStmt = "drop TABLE %s;"%name_Table;
# Create a table in PostgreSQL database
print(dropTableStmt)
cursor.execute(dropTableStmt)
postgresConnection.commit()
cursor.close();
print('Database cursor closed.')
postgresConnection.close()
print('Database connection closed.')
public.subname.table runs afoul of Identifier rules in that '.' is not a valid character. The way around that is to double quote the identifier e.g. "public.subname.table" or use the function quote_ident like quote_ident(public.subname.table). In your case drop TABLE quote_ident(%s).
UPDATE
Previous solution was not. I did not test it and just assumed. A tested solution:
--In psql
create table "public.subname.table"(id int);
select * from "public.subname.table";
id
----
(0 rows)
--In Python
import psycopg2
from psycopg2 import sql
con = psycopg2.connect(dbname="test", host='localhost', user='postgres')
cur = con.cursor()
cur.execute(sql.SQL("DROP table {table}").format(table=sql.Identifier("public.subname.table")))
con.commit()
--psql
select * from "public.subname.table";
ERROR: relation "public.subname.table" does not exist
LINE 1: select * from "public.subname.table";
This makes use of the psycopg2 sql module to properly and safely quote the table name in a query string.
DROP TABLE public."subname.table" does what you want.
sql.Identifier("public", "subname.table") is what you want as psycopg2 identifier.

is it possible to do "CREATE OR REPLACE TABLE foo_{#run_time|"%Y%m%d"} AS" in a scheduled query?

I am wondering if/how i can run a CTAS type statement via a scheduled query but wrangle the #run_date or #run_time param to be the yyyymmdd suffix of the table i want to create or replace.
So the scheduled query would look like:
CREATE OR REPLACE TABLE foo_{#run_date|"%Y%m%d"} AS
select 'hello'
Such that if i was to run it on date of '2021-07-01' i would create the table called foo_20210701
I'm just not quite sure for to wrangle the default #run_date param to include it in my table name.
As mentioned in a answer to a similar question and in the documentation:
Parameters cannot be used as substitutes for identifiers, column names, table names, or other parts of the query.
There is however a workaround. You could use EXECUTE IMMEDIATE to run a SQL script defined as a string. Eg.
EXECUTE IMMEDIATE "SELECT CURRENT_DATE()"
Although it is very hacky it could be used like this achieve what you need.
EXECUTE IMMEDIATE CONCAT('CREATE TABLE `some_project.some_dataset.foo_', CURRENT_DATE(), '` AS SELECT "hello" as column_name' );
And below an approach using a DECLARE statement to keep the table name as a variable
DECLARE table_name STRING DEFAULT CAST(CURRENT_DATE() AS STRING);
EXECUTE IMMEDIATE
CONCAT('CREATE TABLE `some_project.some_dataset.foo_', table_name, '` AS SELECT "hello" as column_name' );

Sas process sql

I am trying to select from a SQL server table that both has dashes in the name and is greater than 32 characters.
I have tried pass through and quotes but no joy.
It's very unlikely that I could get a view produced and only have read access.
proc sql;
drop table poss_gw1;
create table poss_gw1 as ( select * from cdb.'''form_Garden_waste_service_AF-Form-59fb9946-0f6e-4cd9-‌​9b30-82fc5d96ec71'''‌​n as agg);
quit;
proc sql;
connect to odbc(dsn=FirmstepReporting user=myname pwd=mypwd);
Create table work.tmp_gw as select * from connection to odbc (select * from "'form_Garden_waste_service_AF-Form-59fb9946-0f6e-4cd9-9b30-‌​82fc5d96ec71'"n);
disconnect from odbc;
quit;
Any one have any ideas?
You need to use SQL Server syntax in the pass thru code.
create table work.tmp_gw as
select * from connection to odbc
(select *
from "form_Garden_waste_service_AF-Form-59fb9946-0f6e-4cd9-9b30-‌​82fc5d96ec71"
);
If your variable names are also not valid for SAS then you will need to change the name in the pass thru code also.
create table work.tmp_gw as
select * from connection to odbc
(select id
, "invalid-name" as valid_name
from "form_Garden_waste_service_AF-Form-59fb9946-0f6e-4cd9-9b30-‌​82fc5d96ec71"
);

db2 use query result in variable

i want to use the result of a query as an input in another query.
What might make it difficult: The variable is the schema in the database.
CREATE or replace VARIABLE myschema varchar(15) ;
set myschema = (select owner from syscat.tables where tabname = 'xyz');
select count(name) as result from myschema.USR02 where USTYP = 'A';
DROP VARIABLE myschema;
This is my last try, after i failed using declare.
But i get an error, because "myschema" is used as a string, and of course there is no schema with name "myschema". The result of the first query is not used.
If I just run the first two lines, i get the schemaname as result. Do i have to mark the variable in a special way? The goal is just the result of the query in line 3 by using the dynamic value of "myschema".
Unfortunately, you have to use dynamic SQL (forming a custom SQL query through string manipulation) if you want to deal with table, schema, or column names dynamically:
This is the basic idea:
execute immediate 'select * from ' || myschema || '.USR02';
However, you can't just run a bare select in dynamic SQL; you have to put the result in something. And the whole thing must be in a compound SQL block. So the full example would look something like this (simplified query for space).
This query assumes that a table called "result" exists to store the result you are returning.
begin
declare myschema varchar(100) default '';
set myschema = (select owner from syscat.tables where tabname = 'xyz');
execute immediate 'insert into result select count(*) from ' || myschema || '.USR02';
end
select * from result;
Note that within the block, you can simply declare a variable (as shown in my example). So you don't have to declare a global variable for this purpose, unless you want it to persist beyond this one statement.

Insert rows from a table in another database

How can rows be inserted into a table from a table in a remote database?
We currently have a stored procedure which does this using a database link. However, we are having to delete the link because our company policy doesn't allow their usage.
begin
...
execute immediate 'insert into '|| table_name
|| ' (select * from schema_name.'|| table_name ||'#link)';
...
end;
I'm unable to use the COPY command as it appears to be not recognized.
COPY FROM username/pwd#SID
insert into table_name using (select *
from table_name);
Error report:
SQL Error: ORA-00926: missing VALUES keyword
00926. 00000 - "missing VALUES keyword"
*Cause:
*Action:
According to this SQL Plus page, COPY command is now obsolete.
Your Query syntax is slightly wrong, and you just need to specify INSERT/REPLACE/CREATE .. and INTO is NOT needed.
COPY is not obsolete, but it ends up with some encoding issues.
You would use the line continuation character to get around that.
COPY
FROM username/pwd#SID
TO username/pass#SID2
insert
table_name
using
select * from schema_name.table_name;
You can also, download the table data into a text file, and use SQL*Loader to load the data into the another Database.
I prefer the SQL*Loader option! Since maintenance is easy!
For download,
- you can either use SPOOL with delimiter option as mentioned Here
- Write a ProC/PLSQL to output the data into a File (Method 4 in ProC, OR select column names from dba_columns))