I am trying to truncate an existing table in GBQ but the below command fails when I run it. Is there any specific command or syntax to do that. I looked into GBQ documentation but no luck.
TRUNCATE TABLE [dw_test.test];
While BigQuery didn't used to support anything other than SELECTs, it now does as long as you uncheck "Use Legacy SQL" in the query options. There is no truncation, but you can delete:
DELETE from my_table WHERE 1=1
Note that BigQuery requires the use of WHERE in the DELETE, so if you want to delete everything you need to use a statement that will always be true.
Good news, TRUNCATE TABLE is supported by now: https://cloud.google.com/bigquery/docs/reference/standard-sql/dml-syntax#truncate_table_statement
TRUNCATE TABLE [[project_name.]dataset_name.]table_name
However, please note that this will not work / is not supported, if a partition filter is required through your table definition.
CREATE OR REPLACE TABLE <dataset>.<table>
AS SELECT * FROM <dataset>.<table> LIMIT 0;
For partitionned tables, assuming you have a day partition on field "created_on", then execute the following :
CREATE OR REPLACE TABLE <dataset>.<table> PARTITION BY created_on
AS SELECT * FROM <dataset>.<table> WHERE created_on = CURRENT_DATE() LIMIT 0;
EDIT (Nov 2020): BigQuery now supports other verbs, check other answers for newer solutions.
BigQuery doesn't support TRUNCATE as part of a query string. The only DDL/DML verb that BQ supports is SELECT.
One option is to run a job with WRITE_TRUNCATE write disposition (link is for the query job parameter, but it's supported on all job types with a destination table). This will truncate all data already in the table and replace it with the results of the job.
If you don't want to replace the contents with other data or start a job, your best option is probably to delete and recreate the table with the same schema.
Related
I have an Oracle 11g partitioned table with 10 partitions for ten years of data, each on its own tablespace partitioned by range. Each year-partition contains 12 monthly-partitions.
I would like to convert this table to a non-partitioned table, before migrating all the database to Postgresql 10.7 with ora2pg.
I've read that I could first backup this table by expdp and then import it using PARTITIONS_OPTIONS parameter option of impdp.
But is it also possible to use this following statement as a strict equivalent ?
CREATE TABLE IF NOT EXISTS non_partitioned_table AS SELECT * FROM partitioned_table
I would not lose any data, but what about the indexes ?
Is there other differences between these two procedures ?
Syntax you posted doesn't exist in Oracle (there's no if not exists clause there).
Therefore, you'd
create table non_partitioned_table as select * from partitioned_table;
If object whose name is non_partitioned_table already exists, that command would fail.
No indexes would be created automatically - you'd have to create them manually, but - you'd do that in PostgreSQL anyway, wouldn't you? Why bother in Oracle as you won't be using that table for anything (except migration purposes); right?
You can use expdp to export the PARTITION table.
Then use impdp with the MERGE option to import it back into a non partition table. How you get the data into postgres is up to you.
expdp TABLES=scott.part_tab USERID="' / as sysdba'" DIRECTORY=test_dir DUMPFILE=part_tab.dmp LOGFILE=part_tab.log
impdp USERID="'/ as sysdba'" TABLES=scott.part_tab DIRECTORY=test_dir DUMPFILE=part_tab.dmp LOGFILE=imp_part_tab.log REMAP_SCHEMA=scott:testuser
I am using DB2 and Oracle SQL Developer.
How to get the CREATE TABLE Statements from the existing tables?
There are too many tables and it will be a very lengthy process to do manually.
There is a special db2look utility for DDL extraction in Db2. You may refer to its options and their meaning at this link.
If you want SQL access to its capabilities, you may use the SYSPROC.DB2LK_GENERATE_DDL stored procedure supporting most of the utility's options. The routine has an output parameter getting "invocation number" int value after its call.
In case of a single table:
CALL SYSPROC.DB2LK_GENERATE_DDL ('-e -noview -t MY_SCHEMA.MY_TABLE', ?);
SELECT SQL_STMT
FROM SYSTOOLS.DB2LOOK_INFO_V
WHERE OP_TOKEN = <value_of_output_parameter_from_call_above>
ORDER BY OP_SEQUENCE;
In SQLDeveloper if you can see the table there's the initial Create Table Statement in the SQL Tab
You should do that for each table, this is a way to do it but I'm not sure it's fast enough for you.
At work we have a large Oracle SQL query designed to output two select statements based off of analysis and table combining in prior scripts. At the end of these select statements we truncate the temp tables that were created. My issue is that the tables are getting truncated before the select statement has time to run, resulting in 0 output for both queries and empty tables that now need the whole process to be run over again to populate the tables correctly. This is something I'm trying to help automate but I'm stuck on how to get Oracle to wait for the select statement to finish processing before triggering the truncate. Very simply it looks like:
Select * from temp;
Truncate Table temp;
commit;
TRUNCATE is a DDL sentence in Oracle which means it can modify the structure of tables and databases, instead of using TRUNCATE why don't you try to change it for a simple DELETE which is a simple DML sentence in Oracle that just change the data.
What you describe can only ne the case if the 2 select statements (one followed by a truncate) are running in separate sessions. Run in the same session and the problem would be solved although you may lose out on a performance benefit of runnng them in parallel.
I have a hive job that is scheduled to be executed. At the beginning of the job, I want to truncate a table by doing:
TRUNCATE TABLE SOMETABLE
The problem is that the table may be empty. In that case, I don't want to perform the truncate operation which will raise an exception. I know in MySQL you can do something like:
IF EXISTS(SELECT * FROM SOMETABLE)
BEGIN
TRUNCATE SOMETABLE
END
Is there a way I can achieve something similar in hive? Thanks a lot for your help!
If the table is empty also hive won't raise any exception.
You can also make use of Temporary tables in hive so that these tables only accessible through the session that established and very useful to manage intermediate data.
Once the session got closed then Hive deletes all temporary tables.
Please refer this and this links regarding hive temporary tables.
I am trying to delete records from one of the tables in BigQuery, but getting
DML over table dataset.tablename is not supported
I am using this command to delete
bq query --use_legacy_sql=false 'delete from dataset.table_name where Day = 20161215'
But it works when I tried to run it from the console like
delete from dataset.table_name where Day = 20161215
Please Help
Usually, you would see this error if you try DML against partitioned table, which is not yet supported by DML
If this table is not partitioned - check if it is under streaming inserts (check if it has streaming buffer or not)
See more about DML's Known Issues