I have a query that I want to execute daily that's to be partitioned by the date it's executed. The results of this query should be appended to a the same table.
My idea was ideally having something similar to the CREATE TABLE IF NOT EXISTS command for adding data by a new partition every day to the existing table if the partition doesn't already exist, but I can't figure out how I'd be able to integrate this in my query.
My query:
CREATE TABLE IF NOT EXISTS db_name.table_name
WITH (
external_location = 's3://my-query-results-location/',
format = 'PARQUET',
parquet_compression = 'SNAPPY',
partitioned_by = ARRAY['date_executed'])
AS
SELECT
{columns_that_I_am_selecting_here_including_'date_executed'}
What this does is create a new table for the first day it's executed but nothing happens for subsequent days, I'm assuming because of the CREATE TABLE IF NOT EXISTS validating that the table already exists and not proceeding with the logic.
Is there a way to modify my query to create a table for the first day executed and append the results by a new partition for each subsequent day?
I'm quite sure ALTER TABLE table_name ADD [IF NOT EXISTS] PARTITION would not apply to my use case here as I'm running a CTAS query.
You can simply use INSERT INTO existing_table SELECT....
Presumably your table is already partitioned, so include that partition column in the SELECT and Amazon Athena will automatically put the data in the correct directory.
For example, you might include hte column like this: SELECT ... CURRENT_DATE as date_executed
See: INSERT INTO - Amazon Athena
I've created a Stored Procedure that refreshes the data in a table. It first re-loads the entire table. Next, several filters are applied. (Example: the column 'Model' must equal 'W'; all rows with model 'B' are deleted.) This happens after the table has been loaded (and not during) because I want to log how many rows are deleted because of each individual filter. After the filters have been applied, some columns contain the same value in every row (the other values were deleted in the filtering process). These columns are now useless, so I want to delete them.
This seems to be problematic for SQL Server. When given the command to execute the SP, it indicates that the columns it is supposed to remove in its final step do not currently exist and refuses to run. That is technically correct, the columns currently don't exist, but they will be created by the SP itself.
Some mockup code:
CREATE PROCEDURE dbo.Procedure AS (
DROP TABLE dbo.Table
SELECT * INTO dbo.Table FROM dbo.View
INSERT INTO dbo.Log VALUES (GETDATE(),(SELECT COUNT(1) FROM dbo.Table))
DELETE FROM dbo.Table WHERE Model <> 'W'
INSERT INTO dbo.Log VALUES (GETDATE(),(SELECT COUNT(1) FROM dbo.Table))
ALTER TABLE dbo.Table DROP COLUMN Model
)
Error code when executing:
[2016-09-02 12:25:20] [S0001][207] Invalid column name 'Model'.
How do I circumvent this problem and get the SP to run?
If I understand correctly, you can use dynamic SQL:
exec sp_executesql 'ALTER TABLE dbo.Table DROP COLUMN Model';
Syntax to remove any column from table in SQL Server is
ALTER TABLE TableName DROP COLUMN ColumnName ;
This may be cause for issue.
Can you check one more time for the existency of the column 'Model' exists in the view.
because i have tried with the same scenario and its works for me..
Recently I have started learning Oracle-sql. I know that with the help of DELETE command we can delete a particular row(s). So, Is it possible to delete entire data from a particular column in a table using only DELETE command. (I know that using UPDATE command by setting null values to entire column we can achieve the functionality of DELETE).
DELETE
The DELETE statement removes entire rows of data from a specified
table or view
If you want to "remove" data from particular column update it:
UPDATE table_name
SET your_column_name = NULL;
or if column is NOT NULL
UPDATE table_name
SET your_column_name = <value_indicating_removed_data>;
You can also remove entire column using DDL:
ALTER TABLE table_name DROP COLUMN column_name;
In SQL, delete deletes rows not columns.
You have three options in Oracle:
Set all the values to NULL using update.
Remove the column from the table.
Set the column to unused.
The last two use alter table:
alter table t drop column col;
alter table t set unused (col);
Use Invisible Type, which is from an oracle 12cR2.
ALTER TABLE LOG1
MODIFY operation INVISIBLE
It is a better than drop of a particular column.If you need to visible you can get back by altering with an VISIBLE of a column name.
update employee set commission=nvl2(commission,'','')
this will remove all the data from the column
A simple query on the INTO clause. When i try the below statement, the items get pushed into CustomersBackup2013 whether the table exists or not.
SELECT *
INTO CustomersBackup2013
FROM Customers;
However, when i try using the into clause in a MERGE like
MERGE TargetTable tt
USING SyncTable st on <condition>
.
.
WHEN Not MATCHED BY SOURCE
DELETE
OUTPUT deleted.* INTO #Sometemptable;
I get an error saying invalid object name '#Sometemptable'
Isnt it supposed to create the table if it does not exist? Is there something I am doing wrong.
Is there any way i can modify the clause to push items into #Sometemptable?
No, the table has to exists for the output clause to work.
First create the #temp table and then you can output deleted values in it. Columns in the temp table must match columns from output by ordinal and type.
select into table creates a table if required. it is much like oracle's create table as select. see here select into table
merge only inserts, updates or deletes
I know the statement:
create table xyz_new as select * from xyz;
Which copies the structure and the data, but what if I just want the structure?
Just use a where clause that won't select any rows:
create table xyz_new as select * from xyz where 1=0;
Limitations
The following things will not be copied to the new table:
sequences
triggers
indexes
some constraints may not be copied
materialized view logs
This also does not handle partitions
I used the method that you accepted a lot, but as someone pointed out it doesn't duplicate constraints (except for NOT NULL, I think).
A more advanced method if you want to duplicate the full structure is:
SET LONG 5000
SELECT dbms_metadata.get_ddl( 'TABLE', 'MY_TABLE_NAME' ) FROM DUAL;
This will give you the full create statement text which you can modify as you wish for creating the new table. You would have to change the names of the table and all constraints of course.
(You could also do this in older versions using EXP/IMP, but it's much easier now.)
Edited to add
If the table you are after is in a different schema:
SELECT dbms_metadata.get_ddl( 'TABLE', 'MY_TABLE_NAME', 'OTHER_SCHEMA_NAME' ) FROM DUAL;
create table xyz_new as select * from xyz where rownum = -1;
To avoid iterate again and again and insert nothing based on the condition where 1=2
Using sql developer select the table and click on the DDL tab
You can use that code to create a new table with no data when you run it in a sql worksheet
sqldeveloper is a free to use app from oracle.
If the table has sequences or triggers the ddl will sometimes generate those for you too. You just have to be careful what order you make them in and know when to turn the triggers on or off.
You can do this
Create table New_table as select * from Old_table where 1=2 ;
but be careful
The table you create does not have any Index, PK and so on like the old_table.
DECLARE
l_ddl VARCHAR2 (32767);
BEGIN
l_ddl := REPLACE (
REPLACE (
DBMS_LOB.SUBSTR (DBMS_METADATA.get_ddl ('TABLE', 'ACTIVITY_LOG', 'OLDSCHEMA'))
, q'["OLDSCHEMA"]'
, q'["NEWSCHEMA"]'
)
, q'["OLDTABLSPACE"]'
, q'["NEWTABLESPACE"]'
);
EXECUTE IMMEDIATE l_ddl;
END;
Simply write a query like:
create table new_table as select * from old_table where 1=2;
where new_table is the name of the new table that you want to create and old_table is the name of the existing table whose structure you want to copy, this will copy only structure.
SELECT * INTO newtable
FROM oldtable
WHERE 1 = 0;
Create a new, empty table using the schema of another. Just add a WHERE clause that causes the query to return no data:
WHERE 1 = 0 or similar false conditions work, but I dislike how they look. Marginally cleaner code for Oracle 12c+ IMHO is
CREATE TABLE bar AS
SELECT *
FROM foo
FETCH FIRST 0 ROWS ONLY;
Same limitations apply: only column definitions and their nullability are copied into a new table.
If one needs to create a table (with an empty structure) just to EXCHANGE PARTITION, it is best to use the "..FOR EXCHANGE.." clause. It's available only from Oracle version 12.2 onwards though.
CREATE TABLE t1_temp FOR EXCHANGE WITH TABLE t1;
This addresses 'ORA-14097' during the 'exchange partition' seamlessly if table structures are not exactly copied by normal CTAS operation. I have seen Oracle missing some of the "DEFAULT" column and "HIDDEN" columns definitions from the original table.
ORA-14097: column type or size mismatch in ALTER TABLE EXCHANGE
PARTITION
See this for further read...
you can also do a
create table abc_new as select * from abc;
then truncate the table abc_new. Hope this will suffice your requirement.
Using pl/sql developer you can right click on the table_name either in the sql workspace or in the object explorer, than click on "view" and than click "view sql" which generates the sql script to create the table along with all the constraints, indexes, partitions etc..
Next you run the script using the new_table_name
copy without table data
create table <target_table> as select * from <source_table> where 1=2;
copy with table data
create table <target_table> as select * from <source_table>;
In other way you can get ddl of table creation from command listed below, and execute the creation.
SELECT DBMS_METADATA.GET_DDL('TYPE','OBJECT_NAME','DATA_BASE_USER') TEXT FROM DUAL
TYPE is TABLE,PROCEDURE etc.
With this command you can get majority of ddl from database objects.
Create table target_table
As
Select *
from source_table
where 1=2;
Source_table is the table u wanna copy the structure of.
create table xyz_new as select * from xyz;
-- This will create table and copy all data.
delete from xyz_new;
-- This will have same table structure but all data copied will be deleted.
If you want to overcome the limitations specified by answer:
How can I create a copy of an Oracle table without copying the data?
The task above can be completed in two simple steps.
STEP 1:
CREATE table new_table_name AS(Select * from old_table_name);
The query above creates a duplicate of a table (with contents as well).
To get the structure, delete the contents of the table using.
STEP 2:
DELETE * FROM new_table_name.
Hope this solves your problem. And thanks to the earlier posts. Gave me a lot of insight.