I'm currently working on a project that uses a Redshift table with 51 columns. However, the person who made the table forgot to add a sortkey to our time column which will hurt performance for our use case if we don't add it.
How can I make a version of the table with our time column as the sortkey? I'm aware that you can't make a column a sortkey if its a member of an existing table, but I was hoping there's a way to do it that doesn't involve writing out the CREATE TABLE syntax by hand; for example, something like this would be nice:
timecube=# CREATE TABLE foo (like bar) sortkey(time);
ERROR: CREATE TABLE LIKE is not supported with DISTSTYLE, DISTKEY(), or SORTKEY() clauses
but as you can see its not supported. Is there another way? As we're still developing we don't need any of existing data.
Using traditional tools like pgdump didn't work well because they don't include any of the Redshift extras like encoding.
Redshift supports specifying the DIST and SORT keys as part of CREATE TABLE AS statements, as per the docs.
CREATE TABLE table_name
DISTSTYLE KEY
DISTKEY ( column )
SORTKEY ( column )
AS
(SELECT *
FROM source_table)
;
First step you need to do use get create table statement for existing table. Then create new table this time add sort key to new table.
Check encoding for old table ( when you load data using copy command it automatically adds compression encodings)
select "column", type, encoding
from pg_table_def where tablename = 'old_table'
When creating new table add encoding type for each column. Create table with Sort key .
Once new table is created use below command
insert into new table ( select * from old table order by time asc)
Related
There is a requirement to rename the DB tables and column names,
so all the tools/application taking data from the source will have to change their queries. The solution we are planning to implement is that for every table name change we will create a VIEW with the original table name. Easy and simple to implement. No query change required, but there are cases where a table name remains the same but a column name changes within the table, so we can't create another view (any object with the same object name).
Is there a Column Synonym kind of thing which we can propose here?
Any solutions/ideas are welcome. Requirement is to have queries containing original column names referring to the new columns in the same tables.
For example:
Table Name: DATA_TABLE
Existing Column Name: PM_DATE_TIME
New Column Name: PM_DATETIME
Existing Query select pm_Date_time from Data_Table; should refer to new column pm_Datetime
You could consider renaming your original table, and then create a View in its place providing both the old and the new column-names:
CREATE TABLE Data_Table ( pm_Date_time DATE );
ALTER TABLE Data_Table RENAME TO Data_Table_;
CREATE VIEW Data_Table AS
(
SELECT pm_Date_time,
pm_Date_time AS pm_Datetime -- Alias to provide the new column name
FROM Data_table_
);
-- You can use both the old columnn-name...
INSERT INTO Data_Table( pm_Date_time ) VALUES ( SYSDATE );
-- ... or the new one
UPDATE Data_Table SET pm_Datetime = SYSDATE;
There are things that won't work the same way as before:
-- INSERT without stating column-names will fail.
INSERT INTO Data_Table VALUES ( SYSDATE );
-- SELECT * will return both columns (should not do this anyway)
SELECT * FROM Data_Table
Once you are done with your changes drop the view and rename the table and the columns.
You'll want to add virtual columns:
ALTER TABLE Data_Table ADD pm_Date_time as (pm_Datetime);
UPDATE: Oracle (11g at least) doesn't accept this and raises "ORA-54016: Invalid column expression was specified". Please use Peter Lang's solution, where he pseudo-adds zero days:
ALTER TABLE Data_Table ADD (pm_Datetime + 0) AS pm_Date_time;
This works like a view; when accessing pm_Date_time you are really accessing pm_Datetime.
Rextester demo: http://rextester.com/NPWFEW17776
And Peter is also right in this point that you can use it in queries, but not in INSERT/columns or UPDATE/SET clauses.
This was basically touched on in the answer by Thorsten Kettner, but what your looking for is a pseudocolumn.
This solution looks a little hacky because the syntax for a pseudocolumn requires an expression. The simplest expression I can think of is the case statement below. Let me know if you can make it more simple.
ALTER TABLE <<tablename>> ADD (
<<new_column_name>> AS (
CASE
WHEN 1=1 THEN <<tablename>>.<<old_column_name>>
END)
);
This strategy basically creates a new column on the fly by evaluating the case statement and copying the value of <old_column_name> to <new_column_name>. Because you are dynamically interpolating this column there is a performance penalty vs just selecting the original column.
One gotcha here is that this will only work if you are duplicating a column once. Multiple pseudocolumns cannot contain duplicate expressions in Oracle.
we cant create a another view (any object with the same object name).
That's true within a schema. Another somewhat messy approach is to create a new user/schema with appropriate privileges and create all your views in that, with those querying the modified tables in the original schema. You could include instead-of triggers if you need to do more than query. They would only need the old columns names (as aliases), not the new ones, so inserts that don't specify the columns (which is bad, of course) would still work too.
You could also create synonyms to packages etc. in the original schema if the applications/tools call any and their specifications haven't changed. And if they have changed you can create wrapper packages in your new schema.
Then your legacy tools/applications can connect to that new schema and if it's all set up right will see things apparently as they were before. That could potentially be done by setting current_schema, perhaps through a login trigger, if the way they connect or the account they connect to can't be modified.
As the tools and applications are upgraded to work with the new table/column names they can switch back to the original schema.
I have a "source data" table with columns A,B,C,D,E,F
I use this table to populate a live table by using
INSERT INTO LIVETABLE
SELECT *
FROM SOURCEDATATABLE
Recently, a new column (C1) was added to the LIVETABLE
All I want to do is insert a C1 column into my SOURCEDATATABLE between C and D so that it now is A,B,C,C1,D,E,F. There is no need to populate with data as the LIVETABLE accepts NULLs
Is there any easy solution?
EDIT - MISSING INFORMATION
This table is one of many and my approach to using the INSERT INTO is due to having to use dynamic SQL (for various other reasons) so I cannot specify the column names
There is a reason for the Mantra "I shall not use SELECT *" and you ran straight into it. Add the column to SOURCEDATATABLE (if necessary) and enumerate the columns in the SELECT clause using NULL for the new one.
The only way to insert a new column between two columns is to create a new table with the columns in the order you want, copy the data into it, drop the old table and rename the new table with the old name. Make sure you remove primary key identities to maintain the identity column.
I'm trying to setup temporary tables for unit-testing purposes. So far I managed to create a temporary table which copies the structure of an existing table:
CREATE TEMP TABLE t_mytable (LIKE mytable INCLUDING DEFAULTS);
But this lacks the data from the original table. I can copy the data into the temporary table by using a CREATE TABLE AS statement instead:
CREATE TEMP TABLE t_mytable AS SELECT * FROM mytable;
But then the structure of t_mytable will not be identical, e.g. column sizes and default values are different. Is there a single statement which copies everything?
Another problem with the first query using LIKE is that the key column still references the SEQUENCE of the original table, and thus increments it on insertion. Is there an easy way to create the new table with its own sequence, or will I have to set up a new sequence by hand?
I'm using the following code to do it:
CREATE TABLE t_mytable (LIKE mytable INCLUDING ALL);
ALTER TABLE t_mytable ALTER id DROP DEFAULT;
CREATE SEQUENCE t_mytable_id_seq;
INSERT INTO t_mytable SELECT * FROM mytable;
SELECT setval('t_mytable_id_seq', (SELECT max(id) FROM t_mytable), true);
ALTER TABLE t_mytable ALTER id SET DEFAULT nextval('t_my_table_id_seq');
ALTER SEQUENCE t_mytable_id_seq OWNED BY t_mytable.id;
Postgres 10 or later
Postgres 10 introduced IDENTITY columns conforming to the SQL standard (with minor extensions). The ID column of your table would look something like:
id integer PRIMARY KEY GENERATED BY DEFAULT AS IDENTITY
Syntax in the manual.
Using this instead of a traditional serial column avoids your problem with sequences. IDENTITY columns use exclusive, dedicated sequences automatically, even when the specification is copied with LIKE. The manual:
Any identity specifications of copied column definitions will only be
copied if INCLUDING IDENTITY is specified. A new sequence is created
for each identity column of the new table, separate from the sequences
associated with the old table.
And:
INCLUDING ALL is an abbreviated form of INCLUDING DEFAULTS INCLUDING IDENTITY INCLUDING CONSTRAINTS INCLUDING INDEXES INCLUDING STORAGE INCLUDING COMMENTS.
The solution is simpler now:
CREATE TEMP TABLE t_mytable (LIKE mytable INCLUDING ALL);
INSERT INTO t_mytable TABLE mytable;
SELECT setval(pg_get_serial_sequence('t_mytable', 'id'), max(id)) FROM tbl;
As demonstrated, you can still use setval() to set the sequence's current value. A single SELECT does the trick. pg_get_serial_sequence()]6 gets the name of the sequence.
db<>fiddle here
Related:
How to reset postgres' primary key sequence when it falls out of sync?
Is there a shortcut for SELECT * FROM?
Creating a PostgreSQL sequence to a field (which is not the ID of the record)
Original (old) answer
You can take the create script from a database dump or a GUI like pgAdmin (which reverse-engineers database object creation scripts), create an identical copy (with separate sequence for the serial column), and then run:
INSERT INTO new_tbl
SELECT * FROM old_tbl;
The copy cannot be 100% identical if both tables reside in the same schema. Obviously, the table name has to be different. Index names would conflict, too. Retrieving serial numbers from the same sequence would probably not be in your best interest, either. So you have to (at least) adjust the names.
Placing the copy in a different schema avoids all of these conflicts. While you create a temporary table from a regular table like you demonstrated, that's automatically the case since temp tables reside in their own temporary schema.
Or look at Francisco's answer for DDL code to copy directly.
in postgres sql creating the table as select dropped the not null constraints on the table.
for example :
create table A (char a not null);
create table B as select * from a;
select * from B;-- no constraint is copied from A table
please let me know how to copy table data as well as constraints in postgres.
There is no single-command solution to this.
To create a table based on an existing one, including all constraints, use:
create table B ( like a including constraints);
Once you have done that, you can copy the data from the old one to the new one:
insert into b
select * from a;
If you do this in a single transaction, it looks like an atomic operation to all other sessions connected to the database.
very detailed and nicely explained tutorial for create table command in PostgreSQL 9.1
http://www.postgresql.org/docs/current/static/sql-createtable.html
Not null constraints are always copied (if creating table by giving reference of parent table in create table command) and even with including constraints, only check constraint will be copied.
I need a query to create a table which is the exact replica but with different table name and without any data from the source table using a sql query!
You can try this
SELECT * INTO Table_Copy
FROM Table
where 1=2
It will create a empty table with the same structure.
SQL Server Management Studio
Object Explorer
Connect -> Your server
Databases -> Choose Database
Tables
Right Click Your Table
Script Table as -> Create To -> New Query Editor Window
Jonathan has it (upvoted), and you should probably go with that because it's more portable. I normally use something similar:
SELECT TOP 0 * INTO [New_Table] FROM [Old_Table]
I think this better expresses what you're doing, but I like Jonathan's because 'TOP 0' is SQL Server specific, and so his is more portable.
For MySQL, you can call SHOW CREATE TABLE table_name;
It will display a CREATE TABLE query. Simply change the table name in that query and you're good to go.
http://dev.mysql.com/doc/refman/5.1/en/show-create-table.html
If you use Postgresql:
CREATE TABLE LIKE table_name
http://www.postgresql.org/docs/8.1/static/sql-createtable.html
SELECT * INTO Table_Copy
FROM Table
where 1=2
This worked very well, when i tried to create a replica of the table without any data's.
SELECT * INTO Table_Copy
FROM Table
This will create a replica with the data's too.
This can help you:
CREATE TABLE foo AS SELECT...
Read more here
select * into newtablename from sourcetablename
go
truncate newtablename
go
That will result in an exact copy but it also copies the data at first which you remove with the truncate statement.
create table <new table name> as select * from <old tale name from which you would like to extract data>
It will create a new table with a different name but will copy all existing data from the old table to new table.
in postgres you can use INHERITS or LIKE keyword to make replica of a table(only copies structure of the table)
CREATE TABLE client_new (LIKE client);
or
CREATE TABLE client_new () INHERITS (client)
Use of INHERITS creates a persistent relationship between the new child table and its parent table(s). Schema modifications to the parent(s) normally propagate to children as well, and by default the data of the child table is included in scans of the parent(s).
LIKE clause specifies a table from which the new table automatically copies all column names, their data types, and their not-null constraints.Unlike INHERITS, the new table and original table are completely decoupled after creation is complete. Changes to the original table will not be applied to the new table, and it is not possible to include data of the new table in scans of the original table.