I am trying to create snapshot tables as part of script I run regularly.
At the moment I have to manually enter table names, but I would like to call on a field in the base data table to create the snapshot table name.
For example:
Base Data Table = base_data and contains a field for the month it was created in.
Snapshot table = base_data_month
I have already tried to run this to create an automatically named table...
create table base_data_snapshot_||(select month from base_data) as
select * from base_data
But this gets a syntax error. For reference there is only one month included in the base data.
Has anyone had any success with this before?
Although we cant combine a running create statement with a select but can use select. So, I guess below should work
create table
base_data_snapshot_||t.month
as
( select * from base_data) t
Related
I have a query that I want to execute daily that's to be partitioned by the date it's executed. The results of this query should be appended to a the same table.
My idea was ideally having something similar to the CREATE TABLE IF NOT EXISTS command for adding data by a new partition every day to the existing table if the partition doesn't already exist, but I can't figure out how I'd be able to integrate this in my query.
My query:
CREATE TABLE IF NOT EXISTS db_name.table_name
WITH (
external_location = 's3://my-query-results-location/',
format = 'PARQUET',
parquet_compression = 'SNAPPY',
partitioned_by = ARRAY['date_executed'])
AS
SELECT
{columns_that_I_am_selecting_here_including_'date_executed'}
What this does is create a new table for the first day it's executed but nothing happens for subsequent days, I'm assuming because of the CREATE TABLE IF NOT EXISTS validating that the table already exists and not proceeding with the logic.
Is there a way to modify my query to create a table for the first day executed and append the results by a new partition for each subsequent day?
I'm quite sure ALTER TABLE table_name ADD [IF NOT EXISTS] PARTITION would not apply to my use case here as I'm running a CTAS query.
You can simply use INSERT INTO existing_table SELECT....
Presumably your table is already partitioned, so include that partition column in the SELECT and Amazon Athena will automatically put the data in the correct directory.
For example, you might include hte column like this: SELECT ... CURRENT_DATE as date_executed
See: INSERT INTO - Amazon Athena
Is it possible to run query to create a table if it does not exist, and append to the table if the table already exists? I like to write a single query to create or append. Note: I am using Admin console for now, will be using API eventually.
I have following query:
CREATE TABLE IF NOT EXISTS `project_1.dataset_1.tabe_1`
OPTIONS(
description="Some desc"
) AS
SELECT *
FROM source_table
I get following error:
A table named project_1.dataset_1.tabe_1 already exists.
Above query creates a table named 'table_1' if it does not exist under 'project_1.dataset_1', and append to the table if the table already exists.
IF
(
SELECT
COUNT(1)
FROM
`project_1.dataset_1.__TABLES_SUMMARY__`
WHERE
table_id='table_1') = 0
THEN
CREATE OR REPLACE TABLE
`project_1.dataset_1.table_1` AS
SELECT
'apple' AS item,
'fruit' AS category
UNION ALL
SELECT
'leek',
'vegetable';
ELSE
INSERT
`project_1.dataset_1.table_1` ( item,
category )
SELECT
'lettuce' AS item,
'vegetable' AS category
UNION ALL
SELECT
'orange',
'fruit';
END IF;
This seems like it may be a good opportunity to leverage scripting within a single query to accomplish your needs.
See this page for adding control flow to a query to handle an error (e.g. if the table create fails due to existing). For the exception case, you could then INSERT ... SELECT statement as needed.
You can do this via the API as well if you prefer. Simply issue a tables.get equivalent appropriate to the particular library/language you choose and see if the table exists, and then insert the appropriate query based on that outcome of that check.
There is a requirement to rename the DB tables and column names,
so all the tools/application taking data from the source will have to change their queries. The solution we are planning to implement is that for every table name change we will create a VIEW with the original table name. Easy and simple to implement. No query change required, but there are cases where a table name remains the same but a column name changes within the table, so we can't create another view (any object with the same object name).
Is there a Column Synonym kind of thing which we can propose here?
Any solutions/ideas are welcome. Requirement is to have queries containing original column names referring to the new columns in the same tables.
For example:
Table Name: DATA_TABLE
Existing Column Name: PM_DATE_TIME
New Column Name: PM_DATETIME
Existing Query select pm_Date_time from Data_Table; should refer to new column pm_Datetime
You could consider renaming your original table, and then create a View in its place providing both the old and the new column-names:
CREATE TABLE Data_Table ( pm_Date_time DATE );
ALTER TABLE Data_Table RENAME TO Data_Table_;
CREATE VIEW Data_Table AS
(
SELECT pm_Date_time,
pm_Date_time AS pm_Datetime -- Alias to provide the new column name
FROM Data_table_
);
-- You can use both the old columnn-name...
INSERT INTO Data_Table( pm_Date_time ) VALUES ( SYSDATE );
-- ... or the new one
UPDATE Data_Table SET pm_Datetime = SYSDATE;
There are things that won't work the same way as before:
-- INSERT without stating column-names will fail.
INSERT INTO Data_Table VALUES ( SYSDATE );
-- SELECT * will return both columns (should not do this anyway)
SELECT * FROM Data_Table
Once you are done with your changes drop the view and rename the table and the columns.
You'll want to add virtual columns:
ALTER TABLE Data_Table ADD pm_Date_time as (pm_Datetime);
UPDATE: Oracle (11g at least) doesn't accept this and raises "ORA-54016: Invalid column expression was specified". Please use Peter Lang's solution, where he pseudo-adds zero days:
ALTER TABLE Data_Table ADD (pm_Datetime + 0) AS pm_Date_time;
This works like a view; when accessing pm_Date_time you are really accessing pm_Datetime.
Rextester demo: http://rextester.com/NPWFEW17776
And Peter is also right in this point that you can use it in queries, but not in INSERT/columns or UPDATE/SET clauses.
This was basically touched on in the answer by Thorsten Kettner, but what your looking for is a pseudocolumn.
This solution looks a little hacky because the syntax for a pseudocolumn requires an expression. The simplest expression I can think of is the case statement below. Let me know if you can make it more simple.
ALTER TABLE <<tablename>> ADD (
<<new_column_name>> AS (
CASE
WHEN 1=1 THEN <<tablename>>.<<old_column_name>>
END)
);
This strategy basically creates a new column on the fly by evaluating the case statement and copying the value of <old_column_name> to <new_column_name>. Because you are dynamically interpolating this column there is a performance penalty vs just selecting the original column.
One gotcha here is that this will only work if you are duplicating a column once. Multiple pseudocolumns cannot contain duplicate expressions in Oracle.
we cant create a another view (any object with the same object name).
That's true within a schema. Another somewhat messy approach is to create a new user/schema with appropriate privileges and create all your views in that, with those querying the modified tables in the original schema. You could include instead-of triggers if you need to do more than query. They would only need the old columns names (as aliases), not the new ones, so inserts that don't specify the columns (which is bad, of course) would still work too.
You could also create synonyms to packages etc. in the original schema if the applications/tools call any and their specifications haven't changed. And if they have changed you can create wrapper packages in your new schema.
Then your legacy tools/applications can connect to that new schema and if it's all set up right will see things apparently as they were before. That could potentially be done by setting current_schema, perhaps through a login trigger, if the way they connect or the account they connect to can't be modified.
As the tools and applications are upgraded to work with the new table/column names they can switch back to the original schema.
Actually, I want to move one table to another database.
But spark don't permit this.
Then, how to copy table by spark-sql?
I already tried this.
SELECT *
INTO table1 IN new_database
FROM old_database.table1
But it was not working.
maybe try:
CREATE TABLE new_db.new_table AS
SELECT *
FROM old_db.old_table;
To preserve partitioning and storage format do the following-
Get the complete schema of the existing table by running-
show create table db.old_table
The above query will output the table schema which you can just execute after changing the path name and table name.
Then insert all the rows into the new blank table using-
insert into db.new_table select * from db.old_table
The following snippet will create a new table while preserving the definition of the "old" table.
CREATE TABLE db.new_table LIKE db.old_table;
For more info, check the doc's CREATE TABLE.
I need a query to create a table which is the exact replica but with different table name and without any data from the source table using a sql query!
You can try this
SELECT * INTO Table_Copy
FROM Table
where 1=2
It will create a empty table with the same structure.
SQL Server Management Studio
Object Explorer
Connect -> Your server
Databases -> Choose Database
Tables
Right Click Your Table
Script Table as -> Create To -> New Query Editor Window
Jonathan has it (upvoted), and you should probably go with that because it's more portable. I normally use something similar:
SELECT TOP 0 * INTO [New_Table] FROM [Old_Table]
I think this better expresses what you're doing, but I like Jonathan's because 'TOP 0' is SQL Server specific, and so his is more portable.
For MySQL, you can call SHOW CREATE TABLE table_name;
It will display a CREATE TABLE query. Simply change the table name in that query and you're good to go.
http://dev.mysql.com/doc/refman/5.1/en/show-create-table.html
If you use Postgresql:
CREATE TABLE LIKE table_name
http://www.postgresql.org/docs/8.1/static/sql-createtable.html
SELECT * INTO Table_Copy
FROM Table
where 1=2
This worked very well, when i tried to create a replica of the table without any data's.
SELECT * INTO Table_Copy
FROM Table
This will create a replica with the data's too.
This can help you:
CREATE TABLE foo AS SELECT...
Read more here
select * into newtablename from sourcetablename
go
truncate newtablename
go
That will result in an exact copy but it also copies the data at first which you remove with the truncate statement.
create table <new table name> as select * from <old tale name from which you would like to extract data>
It will create a new table with a different name but will copy all existing data from the old table to new table.
in postgres you can use INHERITS or LIKE keyword to make replica of a table(only copies structure of the table)
CREATE TABLE client_new (LIKE client);
or
CREATE TABLE client_new () INHERITS (client)
Use of INHERITS creates a persistent relationship between the new child table and its parent table(s). Schema modifications to the parent(s) normally propagate to children as well, and by default the data of the child table is included in scans of the parent(s).
LIKE clause specifies a table from which the new table automatically copies all column names, their data types, and their not-null constraints.Unlike INHERITS, the new table and original table are completely decoupled after creation is complete. Changes to the original table will not be applied to the new table, and it is not possible to include data of the new table in scans of the original table.