I'm new to Hive, trying to create Hive tables through SAS studio over a server. I have been using execute statement to run HQL, yet I'm stuck on creating a simple flag value. I just want to assign '1' to every row in the exiting table event1.
The code I have has no errors, but the new variable, eventflag, cannot be found:
proc sql;
connect using cen0;
execute (create temporary table event1 as
select x, y, z
from lab.t2_clctn
where z IN ('7','3')
)
by cen0;
execute (alter table event1 add columns(eventflag string)) by cen0;
execute (insert into table event1 (eventflag) value('1')) by cen0;
quit;
Related
Im trying to schedule my stored procedure to run every day and save the results into a table.
I got the stored procedure to work but im not able to create a table with the results.
Creating a table backed by the SP to run daily
You can use DDL statements where you're executing queries in your stored procedure, before the SQL of your output query.
CREATE OR REPLACE TABLE `datasetId.tableId`
CREATE TABLE statement
Whatever your result query is: try to use DDL of create/replace table to save the results.
something like :
BEGIN
EXECUTE IMMEDIATE CONCAT("create or replace table `",YOUR_TABLE_NAME_VARIABLE,"` as YOUR_QUERY_HERE");
EXCEPTION WHEN ERROR THEN
RAISE USING MESSAGE = CONCAT('StoredProc Error: ',##error.message);
END;
I am creating a function that will create a new table and insert informations about that table into other tables.
To create that table I am using the
CREATE TABLE IF NOT EXISTS
statement. Sadly it does not update the FOUND special variable in PostgreSQL nor can i find any other variable that would be updated.
Is there any way in PL/PGSQL to know whether that statement created a table or not?
The target of it is to not to double the informations in the other tables.
You may use CREATE TABLE AS in combination with ON_ERROR_ROLLBACK:
BEGIN;
-- Do inital stuff
\set ON_ERROR_ROLLBACK on
CREATE TABLE my_table AS
SELECT id, name FROM (VALUES (1, 'Bob'), (2, 'Mary')) v(id, name);
\set ON_ERROR_ROLLBACK off
-- Do remaining stuff
END;
To put it bluntly, with \set ON_ERROR_ROLLBACK on postgres will create a savepoint before each statement and automatically rollback to this savepoint or releasing it depending on the success of that statement.
The code above will execute initial and remaining stuff even if the table creation fails.
No, there are not any information if this command created table or not. The found variable is updated after query execution - not after DDL command. There is guaranteed so after this command, the table will be or this command fails to an exception.
I would just like to ask why this code does not create the table?
BEGIN
EXECUTE IMMEDIATE 'create table temp1 as (select * from table)';
COMMIT;
END;
when I try this, it creates the table but no record/data.
BEGIN
EXECUTE IMMEDIATE 'drop table temp1';
EXECUTE IMMEDIATE 'create table temp1 as (select * from table)';
COMMIT;
END;
the table is a global temp table that is why there are no data when I select, but when I run the report, the output has data.
I am trying to fix a duplicate data error that is why I need to create the temp table.
Data in a global temp table (GTT) is only accessible to the session that inserted it; other sessions will only see their own data. It's a common misunderstanding that the data in a global temporary table is not "global" - only the table definition is "global", the data in that table is local to the session that inserted it, and automatically cleared when the session ends (or commits, depending on what type of GTT it is).
If your report queries a GTT and finds data, there must have been a process which populated the global temp table first.
When you ran your "create table" statement (which, by the way, does not need to be explicitly committed), it queried the GTT and found no records because the session in which you created the table did not first populate the GTT with the data needed.
How do I update an Oracle table in SAS from a SAS dataset?
Here's the scenario:
Trough a libname I load an Oracle table into a SAS dataset.
Make some data processing during which I UPDATE some values, INSERT some new observations and DELETE some observations in the dataset.
I need to update the original Oracle table with the dataset I've modified in the previous step - so when there's a match between the keys of the oracle table and the dataset, then the values will be updated, when there's a missing key in the oracle table, then it will be inserted, and when there's a key which is in the Oracle table but already deleted from the dataset, then it will be deleted from the Oracle table.
NOTE: I can not create a new table in Oracle. I need to make the "updating" on the original table.
I was trying to do it in two step using MERGE INTO and DELETE, but there's no MERGE INTO in PROC SQL.
I would really appreciate any help.
EDIT: I was also thinking about just truncating the oracle table and inserting the rows (talking about 4-5000 rows per procedure run), but seems like there's no built in truncate statement in PROC SQL.
Please try using the below,
Method 1:
PROC SQL;
insert into <User_Defined_Oracle_table>
select variables
from <SAS_Tables>;
QUIT;
Above creates a table that resides in the same database and schema.
PROC SQL;
connect to oracle (user= oraclepwd=);
execute(
UPDATE <Oracle_table> a SET <Column to be updated> = (SELECT <Columns to update seperated by commas>
FROM <SAS_table> b
WHERE a.<VARIABLE>=b.<VARIABLE>)
WHERE exists (select * from <SAS_table> b
WHERE a.<VARIABLE>=b.<VARIABLE> ))
by oracle;
QUIT;
PROC SQL;
connect to oracle
(user= oraclepwd=};
execute (truncate table <SAS_table>) by
oracle;
QUIT;
This is one of the efficient ways to update the oracle table.
Please refer to Update Oracle using SAS for more information.
Method 2:
LIBNAME Sample oracle user= password= path= schema= ; run;
PROC SQL;
UPDATE Sample_Oracle.<Table_Name> as a SET <Variable_Name> = (SELECT <Varibales>
FROM <Sas_table> as b
WHERE <A.Variable_Name>=<B.Variable_Name>)
WHERE exists
(select * from <Sas_table> as b
WHERE <A.Variable_Name>=<B.Variable_Name>);
QUIT;
This method takes longer processing time of all methods.
Also,
Method 3:
%MACRO update_oracle (SAS_Table,Oracle_Table);
Proc sql ;
select count(*) into: Count_Obs from <SAS_Table> ; Quit;
%do i = 1 %to &Count_Obs;
Proc sql;
select <variables to update seperated by commas> into: <macros> ; Quit;
PROC SQL;
UPDATE &Oracle_Table as a
SET <Oracle_Variable_to_Update>=<Variable_macro_created_above>
WHERE <A.Variable_Name>=<B.Variable_Name>
QUIT;
%end;
%MEND update_oracle;
%update_oracle();
The macro variables SAS_Table and Oracle_Table represent the SAS Dataset that contains the records to update and records to be updated in oracle, respectively.
Method 3 uses less processing time than method 2 but not as efficient as method 1.
Surely there are UPDATE and INSERT methods in proc SQL. Also, check if SAS will allow you to do other SQL operations "execute immediate" (such as PL/SQL will allow) where you can construct the SQL statement as a string, then send it to Oracle to execute.
Suppose I have these simple sql statements
CREATE TABLE a AS (SELECT 1); -- query #1
CREATE TABLE b AS (SELECT 2); -- query #2
The two tables are created only when the two queries are both finished.
If query #2 runs into any error (or takes much longer than the query #1), neither table a nor b will be created (or table a will not be present until query #2 finishes).
I hope there is a way to create tables one by one, that is, after table a is created, then query #2 is allowed to run, table a thus will be saved (or present) even query #2 runs into errors (or takes much longer time to run).
I googled it with several keywords but in vain. Any solution?
If your connection has implicit transactions turned on, all statements must succeed, or they will all be rolled back. You probably know that's how it works when manipulating data (insert, update etc.) but you may not realize it also happens for table creation and many other DDL commands.
You can avoid this behavior by:
Adding an explicit COMMIT after each CREATE TABLE command
Turning on AUTOCOMMIT in your session
Many DBMSs only apply transactions to data manipulation, but some like Postgres do not.
I assume you are talking about transaction?
successful:
t=# begin;
BEGIN
t=# create table so56(i int);
CREATE TABLE
t=# create table so57 as select * from pg_tables limit 1;
SELECT 1
t=# end;
COMMIT
t=# select * from so56;
i
---
(0 rows)
not successful
t=# begin;
BEGIN
t=# create table so56(i int);
CREATE TABLE
t=# select * from so56;
i
---
(0 rows)
t=# create table so57 as select * from not_existing_table_to_raise_error;
ERROR: relation "not_existing_table_to_raise_error" does not exist
LINE 1: create table so57 as select * from not_existing_table_to_rai...
^
t=# end;
ROLLBACK
t=# select * from so56;
ERROR: relation "so56" does not exist
LINE 1: select * from so56;
second case first table creation rollback if first table creation fails