I need to copy the contents of a table from one database to another database with an identical table which is currently empty.
I plan to dump the table data from the old table and then simply import it to the empty table in the new database. However, I ran into some behavior I don't understand using pg_dump.
I try to dump the table data to a file with the command:
pg_dump -a -t '"my_table"' my_database > /tmp/my_table.sql
This works, but I only get 8 records and there are over 1000 records in the table if I view the table like so:
SELECT * FROM my_table;
So, I tried to use the COPY command to generate a .csv file and I see similar behavior:
COPY my_table TO '/tmp/my_table.csv' WITH CSV HEADER;
I get the same 8 records as pg_dump. But, with:
COPY (SELECT * FROM my_table) TO '/tmp/my_table.csv' WITH CSV HEADER;
I get all 1266 records.
I would assume these commands should all return the same data, but obviously, I'm wrong. What is the difference?
Is it possible that my_table is part of an inheritance hierarchy? I ask because http://www.postgresql.org/docs/9.0/interactive/sql-copy.html#AEN58984 has this:
COPY only deals with the specific table named; it does not copy data to or from child tables. Thus for example COPY table TO shows the same data as SELECT * FROM ONLY table. But COPY (SELECT * FROM table) TO ... can be used to dump all of the data in an inheritance hierarchy.
You should be able to check by running:
SELECT * FROM ONLY my_table;
If that returns just the 8 records then we're on the right track, and we just need to find the child tables (for which How to find child tables that inherit from another table in PSQL will be helpful).
If not then I'm not sure - I wondered if maybe Rules or Triggers were getting involved, but I can't see how at the moment. Still, maybe it gives someone else an idea...?
Related
I'm new to PostgreSQL and and looking for some guidance and best practice.
I have created a table by importing data from a csv file. I then altered the table by creating multiple generated columns like this:
ALTER TABLE master
ADD office VARCHAR(50)
GENERATED ALWAYS AS (CASE WHEN LEFT(location,4)='Chic' THEN 'CHI'
ELSE LEFT(location,strpos(location,'_')-1) END) STORED;
But when I try to import new data into the table I get the following error:
ERROR: column "office" is a generated column
DETAIL: Generated columns cannot be used in COPY.
My goal is to be able to import new data each day to the table and have the generated columns automatically populate in order to transform the data as I would like. How can I do so?
CREATE TEMP TABLE master (location VARCHAR);
ALTER TABLE master
ADD office VARCHAR
GENERATED ALWAYS AS (
CASE
WHEN LEFT(location, 4) = 'Chic' THEN 'CHI'
ELSE LEFT(location, strpos(location, '_') - 1)
END
) STORED;
--INSERT INTO master (location) VALUES ('Chicago');
--INSERT INTO master (location) VALUES ('New_York');
COPY master (location) FROM $$d:\cities.csv$$ CSV;
SELECT * FROM master;
Is this the structure and the behaviour you are expecting? If not, please provide more details regarding your table structure, your importable data and your importing commands.
Also, maybe when you try to import the csv file, the columns are not linked properly, or maybe the delimiter is not properly set. Try to specify each column in the exact order that appear in your csv file.
https://www.postgresql.org/docs/12/sql-copy.html
Note: d:\cities.csv contains:
Chicago
New_York
EDIT:
If columns positions are mixed up between table and csv, the following operation may come in handy:
1. create temporary table tmp (csv_column1 <data_type>, csv_column_2 <data_type>, ...); (including ALL csv columns)
2. copy tmp from '/path/to/file.csv';
3. insert into master (location, other_info, ...) select csv_column_3 as location, csv_column_7 as other_info, ... from tmp;
Importing data using an intermediate table may slow things down a little, but gives you a lot of flexibility.
I was getting the same error when importing to PG from a csv - I found that even though my column was generated, I still had to have it in the imported data, just left it empty. Worked fine when the column name was in there and mapped to my DB col name.
We use copy command to copy data of one table to a file outside database.
Is it possible to copy data of one table to another table using command.
If yes can anyone please share the query.
Or is there any better approach like we can use pg_dump or something like that.
You cannot easily do that, but there's also no need to do so.
CREATE TABLE mycopy AS
SELECT * FROM mytable;
or
CREATE TABLE mycopy (LIKE mytable INCLUDING ALL);
INSERT INTO mycopy
SELECT * FROM mytable;
If you need to select only some columns or reorder them, you can do this:
INSERT INTO mycopy(colA, colB)
SELECT col1, col2 FROM mytable;
You can also do a selective pg_dump and restore of just the target table.
If the columns are the same (names and datatypes) in both tables then you can use the following
INSERT INTO receivingtable (SELECT * FROM sourcetable WHERE column1='parameter' AND column2='anotherparameter');
Suppose there is already a table and you want to copy all records from this table to another table which is not currently present in the database then following query will do this task for you:
SELECT * into public."NewTable" FROM public."ExistingTable";
In Teradata, the way I've been doing backups for tables is like this:
create table xxx_bak as xxx with data
Works great, but I have just discovered that this doesn't work for tables with identity columns.
I need a backup method that can duplicate a table with its data intact so that I can roll it back in case I mess up some data.
After over a year and a half, I've finally found a slick solution to this issue:
create table mydb.mytablebackup as
(select * from (select * from mydb.mytable) x)
with data;
Be sure to qualify the innermost subquery or it won't work.
If you just want a copy of the table, you can create one with the same structure but without making the key column an identity column. You can then insert into it from the original table. However, you wouuldn't be able to insert back into the old table from the backup while retaining the same keys.
The way to make a backup that you can later restore with the same keys is to use the archive/restore tool ARCMAIN.
Backup like this:
logon my_server/my_user, my_password;
archive data tables (my_database.my_table), release lock, file=backup_file;
Restore like this:
logon my_server/my_user, my_password;
restore data tables (my_database.my_table), release lock, file=backup_file;
This involves 3 steps:
1. SHOW TABLE orig_Table; (*Get the DDL*)
2. Replace orig_Table with bkp_Table name
3. INSERT INTO bkp_Table SELECT * FROM orig_Table;
I have two database in the same schema. My db is in Postgres. I want to copy data of any table (i.e product) of my 1st db into the same table of the 2nd db.
Is it possible to do so using query?
Can't do it as a single SQL command (at least not without dblink), but the easiest way is probably to just use a pipe between two psql's - use COPY on both ends, one sending the data out in CSV format the other one receiving it.
try
insert into db1.table1 select * from db2.table2
It's not possible in vanilla PostgreSQL installation.
If you are able to install contrib modules, use dblink:
INSERT
INTO product
SELECT *
FROM dblink
(
'dbname=sourcedb',
'
SELECT *
FROM product
'
) AS p (id INT, column1 INT, column2 TEXT, …)
This should be run in the target database.
What is the SQL command to copy a table from one database to another database?
I am using MySQL and I have two databases x and y. Suppose I have a table in x called a and I need to copy that table to y database.
Sorry if the question is too novice.
Thanks.
If the target table doesn't exist....
CREATE TABLE dest_table AS (SELECT * FROM source_table);
If the target table does exist
INSERT INTO dest_table (SELECT * FROM source_table);
Caveat: Only tested in Oracle
If your two database are separated, the simplest thing to do would be to create a dump of your table and to load it into the second database. Refer to your database manual to see how a dump can be performed.
Otherwise you can use the following syntax (for MySQL)
INSERT INTO database_b.table (SELECT * FROM database_a.table)
Since your scenario involves two different databases, the correct query should be...
INSERT INTO Y..dest_table (SELECT * FROM source_table);
Query assumes, you are running it using X database.
If you just want to copy the contents, you might be looking for select into:
http://www.w3schools.com/Sql/sql_select_into.asp. This will not create an identical copy though, it will just copy every row from one table to another.
At the command line
mysqldump somedb sometable -u user -p | mysql otherdb -u user -p
then type both passwords.
This works even if they are on different hosts (just add the -h parameter as usual), which you can't do with insert select.
Be careful not to accidentally pipe into the wrong db or you will end up dropping the sometable table in that db! (The dump will start with 'drop table sometable').
insert blah from select suggested by others is good for copying the data under mysql.
If you want to copy the table structure you might want to use the show create table Tablename; statement.