So I have a table, person_table:
name
age
gender
john
21
M
abraham
32
M
I want to copy name and gender columns of this table from one database to another on another server. So I have the following table in the other database.
name
gender
john
M
abraham
M
I tried to run the following command, but it does not seem to work.
psql -h localhost -d db_name -p 5432 -U postgres -c "copy(SELECT name, gender FROM person_table) to stdout" > dump.tsv
psql -h localhost -p 5431 -d leaves_db -U postgres -c "copy person_table from stdin" < dump.tsv
But I am getting this error
ERROR: relation "person_table" does not exist
Can anyone tell me what I am doing wrong? Also this is a dummy table. My actual table has millions of entries. Can anyone recommend some faster way to transfer data or my mentioned way is the fastest?
Related
I want to create a query like this:
create table genre (id int, name char(50));
insert into genre (id,name) values (1,"Test")
insert into genre (id,name) values (2,"Test2")
But I want to make it by TPC-H with a size of 1 GB and more. I tried to make the file in dbgn with this command:
dbgen -s 100 -S 1 -C 100 -T p -v
but the output is a table but I want it as an sql file like in this command:
1|155190|7706|1|17|21168.23|0.04|0.02|N|O|1996-03-13|1996-02-12|1996-03-22|DELIVER IN PERSON|TRUCK|egular courts above the|
1|67310|7311|2|36|45983.16|0.09|0.06|N|O|1996-04-12|1996-02-28|1996-04-20|TAKE BACK RETURN|MAIL|ly final dependencies: slyly bold |
1|63700|3701|3|8|13309.60|0.10|0.02|N|O|1996-01-29|1996-03-05|1996-01-31|TAKE BACK RETURN|REG AIR|riously. regular, express dep|
Can you help me figure out what is the command that creates the sql file and contains the information as above?
I have imported all tables using sqoop into a Hive database "sqoop_import" able to see all tables imported successfully as below :-
hive> use sqoop_import;
OK
Time taken: 0.026 seconds
hive> show tables;
OK
categories
customers
departments
order_items
orders
products
Time taken: 0.025 seconds, Fetched: 6 row(s)
hive>
But when I am trying the same from impala-shell or Hue using the same user, It's showing different results as below : -
[quickstart.cloudera:21000] > use sqoop_import;
Query: use sqoop_import
[quickstart.cloudera:21000] > show tables;
Query: show tables
+--------------+
| name |
+--------------+
| customers |
| customers_nk |
+--------------+
Fetched 2 row(s) in 0.01s
[quickstart.cloudera:21000] >
What am I doing wrong?
When you import a new table with sqoop to hive, in order to see it through Impala-Shell you should INVALIDATE METADATA of the specific table. So from the Impala-Shell run the following command : impala-shell -d DB_NAME -q "INVALIDATE METADATA table_name"; .
But if you append new data files to an existing table through sqoop you need to do REFRESH. So from the Impala-Shell run the following command:
impala-shell -d DB_NAME -q "REFRESH table_name";.
Is there anyway to:
Remove specific records from the table using a query?
Make a backup from specific records and restore them into another SQL Server instance somewhere else?
1) If ID is the table's PK (or it is unique) you can just use DELETE FROM TABLE_NAME WHERE ID IN (3, 4). You better check if this will not delete other items (or open a transaction, which is always good).
2) If it is just those 4 records and both databases are on the same server (and both tables have the same schema) you can just do (with the same worries that I have expressed in the answer above)
insert into DESTINATION
select * from SOURCE where id between 73 and 76;
Edit: If you really need to do something more like a row backup you can use the bcp utility:
bcp "select * from SOURCE where id between 73 and 76" queryout "file.dat" -T -c
bcp DESTINATION in file.dat -T -c
DELETE FROM ListsItems
WHERE ID = (3, 4);
It will remove your record.
Modify it....
I want to dump a database.
I have three tables:
table1
table2
table3
From table1 i want the schema plus data.
From table2 and table3 i just want the schema.
How do i do that?
To get data from just a few tables:
pg_dump myDatabase --inserts -a -t table1 -t table2 > backup.sql;
pg_dump myDatabase --inserts -a -t seq1 -t seq2 > backupSequences.sql;
Parameters descriptions:
-a, --data-only dump only the data, not the schema
-t, --table=TABLE dump the named table(s) only
--inserts dump data as INSERT commands, rather than
COPY
This is what i wanted :)
Thanks all!
Use pg_dump, which has both schema-only and schema + data output.
I need to sync two PostgreSQL databases (some tables from development db to production db) sometimes.
So I came up with this script:
[...]
pg_dump -a -F tar -t table1 -t table2 -U user1 dbname1 | \
pg_restore -a -U user2 -d dbname2
[...]
The problem is that this works just for newly added rows. When I edit non-PK column I get constraint error and row isn't updated. For each dumped row I need to check if it exists in destination database (by PK) and if so delete it before INSERT/COPY.
Thanks for advices.
Do this:
pg_dump -t table1 production_database > /tmp/old_production_database_table1.sql
pg_dump -t table1 devel_database > /tmp/devel_database_table1.sql
psql production_database
truncate table1
\i /tmp/devel_database_table1.sql
\i /tmp/old_production_database_table1.sql
You'll get a lot of duplicate primary key errors on second \i, but it'll do what you want: all rows from devel will be updated, all rows not in devel will not be updated nor deleted.
If you have any references to table1 then you'll have to drop them before and recreate them after importing. Especially check for on delete cascade, set null or set default references to table1 - you'd loose data in other tables if you have those.