I have a query to find potential SSN in a table using regex pattern.
db_name.schema_name.Table name: db_name.schema_name.ABC
Column name with Sensitive data: senstve_col
select regexp_substr(senstve_col, '\\b[0-9]{3}[ -][0-9]{2}[ -]{4}\\b') as sensitive_data, * from db_name.schema_name.ABC)
I need to do this for 200 tables with 200 different column names. Also, the db_name and schema_name varies for each table.
Is there a way to pass the values dynamically and store the data into a new table in snowflake?
can someone help with the query to automate the above query for multiple tables?
This is how you call the .SQL in unix
snowsql -o variable_substitution=True --variable NEXT_TABLE=tblname --variable NEXT_COL=colname -f /home/sagar/snowflake/create_table.sql
And this is how you mention the variable name in the .sql
create or replace view ntgrpa_hist.vw_rt_satelliteinfo_latest as
select &NEXT_COL from public.&NEXT_TABLE;
Related
I'm working myself through this guide to how to convert several character variables at once to numeric using SAS. The guide uses PROC SQL, the SQL interface of SAS. What got me wondering is the part following the INTO-clause below:
proc sql noprint; select
trim(left(name)), trim(left(newname)),
trim(left(newname))||'='||trim(left(name))
into :c_list separated by ' ', :n_list separated by ' ',
:renam_list separated by ' '
from vars;
Essentially, the clause seems to select each variable into a different table. Are you allowed to do this in SQL, or is this only a feature of PROC SQL in SAS?
I tried the syntax at several SQL sandboxes, but couldn't get it to work.
It is a feature of PROC SQL in SAS to read values into SAS Macro Variables.
Only the CREATE TABLE clause specifies the name of the table the query should create from the result set delivered by the SELECT clause.
The INTO clause in SAS' implementation of SQL is different than then INTO clause implemented in other flavors of SQL.
From SAS documentation
INTO Clause
Stores the value of one or more columns for use later in another PROC SQL query or SAS statement.
Restriction: An INTO clause cannot be used in a CREATE TABLE statement.
...
INTO macro-variable-specification
< , … macro-variable-specification>
Compare the above with Microsoft Transact-SQL INTO clause
[ INTO new_table ]
...
new_table
Specifies the name of a new table to be created, based on the columns in the select list and the rows chosen from the data source.
For these two the general patterns are
SAS: CREATE TABLE abc as SELECT ...
MS: SELECT ... INTO abc
The SAS/MS comparison for selecting a column into a variable is
SAS (: indicates into variable)SELECT expression-1,...,expression-N
INTO :macro-variable-1,...,:macro-variable-N
MS (# indicates into variable)SELECT expression-1,...,expression-N
INTO #:tsql-variable-1*,...,#tsql-variable-N
The variables the values are selected into have different scope contexts depending on system or implementation.
AFAIK there is no method to create multiple tables with a single query in SAS or ANSI SQL.
However, this is trivial within a SAS data step using a variety of methods.
The most basic:
data male female;
set sashelp.class;
if sex='M' then output male;
else if sex='F' then output female;
run;
I am trying to write a BigQuery script that I can store as a procedure, I would like one of the arguments I pass to be used in the table name that is written out by the script, for example:
DECLARE id STRING;
SET id = '123';
CREATE OR REPLACE TABLE test.id AS(
SELECT * FROM dataset.table
)
However, in this example the table is created with the name id rather than the value of the "id" variable, 123. Is there any way I can dynamically create a table using the value of a declared variable in the BigQuery UI?
Why not just use Execute Immediate with concat if you know the table schema?
EXECUTE IMMEDIATE CONCAT('CREATE TABLE `', id, '` (column_name STRING)');
So far we have officially announced BigQuery scripting document, which is still in Beta phase, leveraging usage of dynamic parameters (variables) as a placeholders for values in SQL queries . However, according to Parameterized queries in BigQuery documentation, query parameters can't be used for SQL object identifiers:
Parameters cannot be used as substitutes for identifiers, column
names, table names, or other parts of the query.
Maybe you can use a wildcard table. You would create a wildcard table with all subtables you want to query and use the WHERE clause to select any subtable you want. Just be careful, the tables must have the same schema.
I am trying to write a BigQuery script that I can store as a procedure, I would like one of the arguments I pass to be used in the table name that is written out by the script, for example:
DECLARE id STRING;
SET id = '123';
CREATE OR REPLACE TABLE test.id AS(
SELECT * FROM dataset.table
)
However, in this example the table is created with the name id rather than the value of the "id" variable, 123. Is there any way I can dynamically create a table using the value of a declared variable in the BigQuery UI?
Why not just use Execute Immediate with concat if you know the table schema?
EXECUTE IMMEDIATE CONCAT('CREATE TABLE `', id, '` (column_name STRING)');
So far we have officially announced BigQuery scripting document, which is still in Beta phase, leveraging usage of dynamic parameters (variables) as a placeholders for values in SQL queries . However, according to Parameterized queries in BigQuery documentation, query parameters can't be used for SQL object identifiers:
Parameters cannot be used as substitutes for identifiers, column
names, table names, or other parts of the query.
Maybe you can use a wildcard table. You would create a wildcard table with all subtables you want to query and use the WHERE clause to select any subtable you want. Just be careful, the tables must have the same schema.
I need to export data from an SQL table to CSV format. But I also need:
1. Metadata inserted in the first row of the output. This will be static.
2. Header row after the metadata.
3. Data. BUT I need fields with multiple values (e.g. name JOHN SMITH) to be in "" and commas within the quotes to separate the values within the field.
Here is my first draft to get data in CSV:
EXEC xp_cmdshell 'bcp "SELECT ITN_USER, SITE_ID, TICKET_NUMBER, VALIDATING_CARRIER_CODE, TICKET_EXPIRATION_DATE, TICKET_CURR_CODE, RESIDUAL_TOTAL_AMT, TICKET_TOTAL_FARE, PASSENGER_NAME, FIRST_ORIG_APT_CODE, FIRST_DEST_APT_CODE, FIRST_DEPART_DATE, TICKET_ISSUE_DATE, CRS_LOCATOR, TICKET_STATUS_ID, TICKET_TYPE, RSVN_SYS_ID, TICKETING_LOCATION, TICKET_BASE_FARE, TICKET_TAX, FARE_CALC_LINE FROM GDSX.dbo.UnusedTickets WHERE INSERT_DATE = ''01-31-2018''" queryout "C:\Users\Public\Documents\filename1_filename2_date.csv" /c /t, -T'
Any helpful tips or suggestions would be greatly appreciated.
This is what I want to achieve: “josh#gmail.com,vbear#gmail.com"
ITN_USER,SITE_ID,TICKET_NUMBER,VALIDATING_CARRIER_CODE,TICKET_ EXPIRATION_DATE,TICKET_CURR_CODE,RESIDUAL_TOTAL_AMT,TICKET_TOT AL_FARE,PASSENGER_NAME,FIRST_ORIG_APT_CODE,FIRST_DEST_APT_CODE ,FIRST_DEPART_DATE,TICKET_ISSUE_DATE,CRS_LOCATOR,TICKET_STATUS _ID,TICKET_TYPE,RSVN_SYS_ID,TICKETING_LOCATION,TICKET_BASE_FAR E,TICKET_TAX,FARE_CALC_LINE vbear,abccorpus,0017845439769,AA,08MAY2009,USD,1226.57,1629.00 ,bear/vernon,MSY,ORD,17MAY2008,08MAY2008,,,electronic,,,,, jsmith,abccorpus,0167846739059,UA,19JUN2009,USD,354.00,354.00, smith/john,LAX,PDX,25JUN2008,19JUN2008,,,,,,,, dgarcia,abccorpmx,1327959759566,MX,03AUG2009,MXN,6828.06,6828. 06,garcia/diego,MEX,GUA,07AUG2008,03AUG2008,,,electronic,,,,,
Thanks!
Try Creating a view and use the view in your BCP statement. You can do all your calculation in the view. For metadata you need to do a union all with the data. SO your view will be something like...
Create view abc as
select 'Name' as Name, 'Age' as Age --Metadata
Union All
Select Name , Cast(Age as Varchar(X)) from your table
make sure you give cast varchar for all the column as you will be doing union.
INFORMIX-SQL or any other SQL-based DB:
Suppose I have an app where depending on the value of some columns, example:
company.code char(3) {abc}
company.branch char(2) {01}
Can I construct table name "abc01" for inclusion in SELECT * FROM abc01; ?
In other words, a variable table name.. same question applies for column names.
Only in a language which can manipulate character strings and handle Dynamic SQL. It has to create the statement on the fly.
You can only use placeholders in queries for values, not for the structural elements of the query such as the table name or column name.
Only if you use dynamic sql. It's like you build sql code in your app, then use something like execute immediate.
sprintf(cdb_text1, "create table %s (field1 char(3));", usr_db_id);
EXEC SQL execute immediate :cdb_text;
If you use dynamic sql, it's bad because of sql injections.