BigQuery create Temp table - google-bigquery

I am trying to create a temporary table with a specific name so i can use it in other queries. The problem is that when i create a temp table like this:
BEGIN
BEGIN TRANSACTION;
--delete test_temp where 1=1;
create temp table test_temp as SELECT * FROM `analytics.reports.analysis` ;
COMMIT TRANSACTION;
END;
the table name is indeed test_temp but if i will try to select * from test_temp
i get nothing. the table id always generates a new unique id.
something like analytics-prod:_script3991beb3e9868774854ac09d407f1c397428a4f6.test_temp
Is there a way to make this temp table with a pre-defined name?
What I am trying to achieve basically is create a table from a select every 15min.
I thought to create a scheduled query running this transaction that will delete + populate the temp table and then I can always reference to test_temp without changing the table id.

According to the docs:
You can refer to a temporary table by name for the duration of the current multi-statement query
So if you want to refer the table in other queries, an option could be use a permanent table in a separated dataset:
...
CREATE OR REPLACE TABLE `analytics.my_temp_dataset.test_temp` AS SELECT * FROM `analytics.reports.analysis` ;
...

In that case, you don't need a temp table but a permanent table you just replace on the next run using the CREATE OR REPLACE TABLE statement. If you are using the temp table as part of a scripting stage, then I suggest using running this instead:
BEGIN
CREATE OR REPLACE TEMP TABLE _SESSION.test_temp AS
SELECT * FROM `analytics.reports.analysis`;
-- SELECT COUNT(*) CNT FROM _SESSION.test_temp;
END;
Since it is a temp table you don't need a DROP query at the end.

You can create a temporary table in a session in BigQuery.
You can create a session via the API and then create your temporary tables in the session. The tables will then be visible to all queries using that session.
There are limitations on sessions you should be aware of:
They last up to 24 hours max
No concurrent jobs running in a session
I use temporary tables with sessions to do complex transformations in an ETL pipeline. For example:
Create session
Create temporary tables
Do joins, and other transformations, write results to temporary tables
Do final transformations and output results to permanent tables
End session
I am writing a Python API for this and will put it on github at some point. I could probably post a gist if that is helpful.
In the meantime, see Google's documentation for sessions in BigQuery:
https://cloud.google.com/bigquery/docs/sessions-intro

Related

How to create a temporary table as copy of another table in Oracle?

I already tried this:
CREATE GLOBAL TEMPORARY TABLE tempTable AS
SELECT * FROM realTable;
But then tempTable has only the structure of realTable, but not the elements themselves.
"But then tempTable has only the structure of realTable, but not the
elements themselves."
What makes a global temporary table temporary is that the data is transient. Firstly, the data is only visible within the session which inserts it; any other session will see an empty table. Secondly, the data can persist for either a transaction or the session, depending on the ON COMMIT clause; the default is ON COMMIT DELETE ROWS. Find out more.
Now the thing is, a DDL statement in Oracle issues two commits, one before and one after the statement in question. So a DDL statement is a complete, discrete transaction. Hence this ...
CREATE GLOBAL TEMPORARY TABLE tempTable AS
SELECT * FROM realTable;
... is a transaction and, as it doesn't specify the ON COMMIT clause, it will apply the default which is DELETE ROWS. So an empty table is the expected behaviour.
The solution is simple: specify the ON COMMIT statement with session-level retention:
SQL> select count(*) from t23;
COUNT(*)
----------
11
SQL> create global temporary table gtt23
2 as select * from t23
3 /
Table created.
SQL> select count(*) from gtt23;
COUNT(*)
----------
0
SQL> drop table gtt23;
Table dropped.
SQL> create global temporary table gtt23
2 on commit preserve rows
3 as select * from t23
4 /
Table created.
SQL> select count(*) from gtt23;
COUNT(*)
----------
11
SQL>
Generally, I think that a policy of CREATE GLOBAL TEMPORARY TABLE using SELECT * FROM indicates a misunderstanding of the construct. GTTs in Oracle are permanent data structures; only the records are temporary. They are not disposable objects like temporary tables in T-SQL. If that's the sort of thing you want, you should probably be using PL/SQL collections instead. Find out more.
Global temporary tables can have either transaction-level scope or session-level scope. The default is to have transaction-level scope which means that the data disappears after the transaction completes. If you do a CREATE TABLE AS SELECT to create your global temporary table, the data will be inserted but, since CREATE is DDL, the data will be removed as soon as the statement completes.
One option would be to create the structure using a query that doesn't return any data
CREATE GLOBAL TEMPORARY TABLE tempTable AS
SELECT *
FROM realTable
WHERE 1=0;
then insert the data
INSERT INTO tempTable
SELECT *
FROM realTable;
Of course, given how infrequently global temporary tables are used in Oracle (particularly in comparison to other databases), I'd want to be very certain that you really need to create a temporary table from a permanent table in the first place.

SQL Server Global Temporary Table Locking

How do I lock a global temporary table in a stored procedure that's getting created and populated by a SELECT INTO statement? For example:
SELECT *
INTO ##TempEmployee
FROM Employee
This stored procedure is executed for generating reports and it's there in every client database (multi-tenant architecture using different DB per client). I do not want data in this global temporary table to be shared between clients when the report is generated concurrently. I don't have a choice but to use global temp table because I use it for generating columns on the fly using PIVOT.
Why not include it inside a transaction block like
begin transaction
SELECT *
INTO ##TempEmployee
FROM Employee
Try this,
WorkDummySQL
create table rr(id integer,name varchar(20))
insert into rr values(1,'aa')
select * from rr
Tempdb
select * into ##ta from WorkDummySQL.dbo.rr

Select into tables in SQL. How are they stored?

When I run a script in PostgreSQL I usually do the following from psql:
my_database> \i my_script.sql
Where in my_script.sql I may have code like the following:
select a.run_uid, s.object_uid into temp_table from dt.table_run_group as a
inner join dt.table_segment as s on a.group_uid = s.object_uid;
In this particular case, I am only interested in creating temp_table with the results of the query.
Are these results in disk on the server? In memory? Is the table stored permanently?
Temporary tables are stored in RAM until the available memory is used up, at which time they spill onto disk. The relevant setting here is temp_buffers.
Either way, they live for the duration of a session and are dropped at the end automatically.
You can also drop them at the end of a transaction automatically (ON COMMIT DROP) or manually any time.
Temporary table are only visible to the the same user in the same session. Others cannot access it - and also not conflict with it.
Always use CREATE TABLE tbl AS .... The alternative form SELECT ... INTO tbl is discouraged since it conflicts with the INTO clause in plpgsql.
Your query could look like:
CREATE TEMP TABLE tbl AS
SELECT a.run_uid, s.object_uid
FROM dt.table_run_group a
JOIN dt.table_segment s ON a.group_uid = s.object_uid;
SELECT INTO table ... is the same as CREATE TABLE table AS ..., which creates a normal, permanent table.

How do you create a temporary table in an Oracle database?

I would like to create a temporary table in a Oracle database
something like
Declare table #table (int id)
In SQL server
And then populate it with a select statement
Is it possible?
Thanks
Yep, Oracle has temporary tables. Here is a link to an AskTom article describing them and here is the official oracle CREATE TABLE documentation.
However, in Oracle, only the data in a temporary table is temporary. The table is a regular object visible to other sessions. It is a bad practice to frequently create and drop temporary tables in Oracle.
CREATE GLOBAL TEMPORARY TABLE today_sales(order_id NUMBER)
ON COMMIT PRESERVE ROWS;
Oracle 18c added private temporary tables, which are single-session in-memory objects. See the documentation for more details. Private temporary tables can be dynamically created and dropped.
CREATE PRIVATE TEMPORARY TABLE ora$ptt_today_sales AS
SELECT * FROM orders WHERE order_date = SYSDATE;
Temporary tables can be useful but they are commonly abused in Oracle. They can often be avoided by combining multiple steps into a single SQL statement using inline views.
Just a tip.. Temporary tables in Oracle are different to SQL Server. You create it ONCE and only ONCE, not every session. The rows you insert into it are visible only to your session, and are automatically deleted (i.e., TRUNCATE, not DROP) when you end you session ( or end of the transaction, depending on which "ON COMMIT" clause you use).
CREATE GLOBAL TEMPORARY TABLE Table_name
(startdate DATE,
enddate DATE,
class CHAR(20))
ON COMMIT DELETE ROWS;
CREATE TABLE table_temp_list_objects AS
SELECT o.owner, o.object_name FROM sys.all_objects o WHERE o.object_type ='TABLE';

How can I create a copy of an Oracle table without copying the data?

I know the statement:
create table xyz_new as select * from xyz;
Which copies the structure and the data, but what if I just want the structure?
Just use a where clause that won't select any rows:
create table xyz_new as select * from xyz where 1=0;
Limitations
The following things will not be copied to the new table:
sequences
triggers
indexes
some constraints may not be copied
materialized view logs
This also does not handle partitions
I used the method that you accepted a lot, but as someone pointed out it doesn't duplicate constraints (except for NOT NULL, I think).
A more advanced method if you want to duplicate the full structure is:
SET LONG 5000
SELECT dbms_metadata.get_ddl( 'TABLE', 'MY_TABLE_NAME' ) FROM DUAL;
This will give you the full create statement text which you can modify as you wish for creating the new table. You would have to change the names of the table and all constraints of course.
(You could also do this in older versions using EXP/IMP, but it's much easier now.)
Edited to add
If the table you are after is in a different schema:
SELECT dbms_metadata.get_ddl( 'TABLE', 'MY_TABLE_NAME', 'OTHER_SCHEMA_NAME' ) FROM DUAL;
create table xyz_new as select * from xyz where rownum = -1;
To avoid iterate again and again and insert nothing based on the condition where 1=2
Using sql developer select the table and click on the DDL tab
You can use that code to create a new table with no data when you run it in a sql worksheet
sqldeveloper is a free to use app from oracle.
If the table has sequences or triggers the ddl will sometimes generate those for you too. You just have to be careful what order you make them in and know when to turn the triggers on or off.
You can do this
Create table New_table as select * from Old_table where 1=2 ;
but be careful
The table you create does not have any Index, PK and so on like the old_table.
DECLARE
l_ddl VARCHAR2 (32767);
BEGIN
l_ddl := REPLACE (
REPLACE (
DBMS_LOB.SUBSTR (DBMS_METADATA.get_ddl ('TABLE', 'ACTIVITY_LOG', 'OLDSCHEMA'))
, q'["OLDSCHEMA"]'
, q'["NEWSCHEMA"]'
)
, q'["OLDTABLSPACE"]'
, q'["NEWTABLESPACE"]'
);
EXECUTE IMMEDIATE l_ddl;
END;
Simply write a query like:
create table new_table as select * from old_table where 1=2;
where new_table is the name of the new table that you want to create and old_table is the name of the existing table whose structure you want to copy, this will copy only structure.
SELECT * INTO newtable
FROM oldtable
WHERE 1 = 0;
Create a new, empty table using the schema of another. Just add a WHERE clause that causes the query to return no data:
WHERE 1 = 0 or similar false conditions work, but I dislike how they look. Marginally cleaner code for Oracle 12c+ IMHO is
CREATE TABLE bar AS
SELECT *
FROM foo
FETCH FIRST 0 ROWS ONLY;
Same limitations apply: only column definitions and their nullability are copied into a new table.
If one needs to create a table (with an empty structure) just to EXCHANGE PARTITION, it is best to use the "..FOR EXCHANGE.." clause. It's available only from Oracle version 12.2 onwards though.
CREATE TABLE t1_temp FOR EXCHANGE WITH TABLE t1;
This addresses 'ORA-14097' during the 'exchange partition' seamlessly if table structures are not exactly copied by normal CTAS operation. I have seen Oracle missing some of the "DEFAULT" column and "HIDDEN" columns definitions from the original table.
ORA-14097: column type or size mismatch in ALTER TABLE EXCHANGE
PARTITION
See this for further read...
you can also do a
create table abc_new as select * from abc;
then truncate the table abc_new. Hope this will suffice your requirement.
Using pl/sql developer you can right click on the table_name either in the sql workspace or in the object explorer, than click on "view" and than click "view sql" which generates the sql script to create the table along with all the constraints, indexes, partitions etc..
Next you run the script using the new_table_name
copy without table data
create table <target_table> as select * from <source_table> where 1=2;
copy with table data
create table <target_table> as select * from <source_table>;
In other way you can get ddl of table creation from command listed below, and execute the creation.
SELECT DBMS_METADATA.GET_DDL('TYPE','OBJECT_NAME','DATA_BASE_USER') TEXT FROM DUAL
TYPE is TABLE,PROCEDURE etc.
With this command you can get majority of ddl from database objects.
Create table target_table
As
Select *
from source_table
where 1=2;
Source_table is the table u wanna copy the structure of.
create table xyz_new as select * from xyz;
-- This will create table and copy all data.
delete from xyz_new;
-- This will have same table structure but all data copied will be deleted.
If you want to overcome the limitations specified by answer:
How can I create a copy of an Oracle table without copying the data?
The task above can be completed in two simple steps.
STEP 1:
CREATE table new_table_name AS(Select * from old_table_name);
The query above creates a duplicate of a table (with contents as well).
To get the structure, delete the contents of the table using.
STEP 2:
DELETE * FROM new_table_name.
Hope this solves your problem. And thanks to the earlier posts. Gave me a lot of insight.