I am trying to migrate files over from one remote database- Scratxh to another remote database. I am using pg_dump and psql to download an .sql file then using psql to recreate the table in the new database - SourceData. I want to copy the table only. I used -t to indicate this, but I still get these errors :
ERROR schema public does not exist
ERROR permission denied to set session authorization.
These are the commands I used.
pg_dump -t table -d Scratch -U me -h host.com > table.sql
psql -d SourceData -U me -h host.com < table.sql
I know that psql command uses the .sql text file to recreate the table, so I tried editing this file to get rid of any mentions of the schema 'public'.
It didn't help. I got the same error.
Has anyone else encountered this?
It is not clear from your comment when the error happens. I will assume it happens on the second command. In that case, the first error shown might be because the second database is not ready to receive the data, i.e.: the SQL contains INSERT statement to a table that doesn't exist yet in SourceData.
You need to create the table in the new database before being able to import data into it.
If you pg_dump the entire database, you would probably not encounter this exact problem.
Related
I need to play around with a data syncing program I wrote, but I want to copy the structure of the production database into a new table on my localhost Postgres database, without copying the data to my localhost db.
I was thinking along the lines of
CREATE TABLE new_table AS
TABLE existing_table
WITH NO DATA;
But I am unsure how to modify it to work with 2 different databases.
Any help would be appreciated
This boils down to the question "how to create the DDL script for a table" which can easily be done using pg_dump on the command line.
pg_dump -d some_db -h production_server -t existing_table --schema-only -f create.sql
The file create.sql then contains the CREATE TABLE script that you can run on your local Postgres installation.
I am new to Postgres and i am trying to learn from an online tutorial. One of the first thing is to load the data, as follows:
Finally, run psql -U <username> -f clubdata.sql -d postgres -x -q to
create the 'exercises' database, the Postgres 'pgexercises' user, the
tables, and to load the data in. Note that you may find that the sort
order of your results differs from those shown on the web site:
I am using pdAdmin4 and opened the SQL shell. However I wasn't able to load this database. First of all, how can i figure out what is my current username?
Secondly, I have never worked with command line before and am quite unsure how to do this. Could someone break this down step-by-step?
You can run "psql -h" for more help. You never have a current username as such, you have to specify it but start with "-U postgres" and ask again if that doesn't work.
Your sql file to load will need the folder path or you could try the cmd prompt and change to the folder where your clubdata file is. Your command line assumes there is already a database named postgres which there probably is. Try again;
psql -U postgres -f clubdata.sql -d postgres -x -q
The command psql is for the command line client. You need to run this in a terminal.
I wrestled with this input myself, despite a little CLI experience with psql. It may help to remove the -q flag in the end to make the output non-quiet, then you see what's going on.
Lastly, beware that the import creates a schema, so you need to read up on schemas. See this related question for a bit more background: https://dba.stackexchange.com/questions/264398/cant-find-any-tables-after-psql-dump-import-from-pgexercises-com
I have a dev version and a production version running in django.
I recently started populating it with a lot of data and found that the django loaddata tries to load everything into memory before adding it into the db and my files will be too big for that.
What is the proper way to push my data from my dev machine to my production?
I did...
pg_dump -U user -W db ./filename.sql
and then on the production server I did...
psql dbname < filename.sql
It seems like it worked, all the data is there, but it came up with some errors such as
relation xxx already exists
constrain xxx for relation xxx already exists
and there were quite a few of them, but like I said everything appears to be there. Is this the right way to do it?
Edit: I have in the production machine the database with information and I don't want truncate the tables before import.
This is the script that I use:
pg_dump -d DATABASE_NAME -U postgres --format plain --inserts > /FILE.sql
Edit: As you says in comments that you don't want truncate the tables before import, you can't do this type of import into your production database. I suggest empty your production database before import the dev database dump.
I have two Postgresql servers (Windows), and I am trying to transfer a table from server1 to server2. This table is around 200 MB size as it contains binary data.
I want to put the table into a usb stick and then move it to the second server. (assume the two servers are not connected by a LAN).
What is the simplest way to do that? Can you describe the way with command.
The easiest way would probably be to use pg_dump.
I haven't used it on Windows so I don't know the actual path to it, but it should be in the Postgres\bin directory and you need to execute it in a shell window (like PowerShell or CMD).
Assuming you have console access to each server, and that the table already exists in the second database:
pg_dump -a -b -Fc -t <tablename> <databasename> > <path to dump file>
Then when you have moved it to the new server.
pg_restore -a -Fc -d <databasename> <path to dump file>
If you don't have direct access to each server, then you need to add the connection parameters to each command:
-h <server> -U <username>
Quick description of the parameters:
-a : dumps only the data and not the schema definition. This should be removed if the table is not already in place on the new server
-b : dumps blobs. You mentioned there are binary data in the table, if they are stored as large objects, this parameter needs to be included, otherwise you can skip it.
-Fc : The format to dump the data as. c stands for Postgres custom format, which is better suited for moving binary data. You could change it to d to use a directory format since you're using 9.2, but I prefer the custom format still. d however is useful when dumping large databases since it stores each table in one file within the specified directory.
-t : Specifies that you want to dump a table and not the entire database.
-d : the database that you want to restore to (this parameter can be used in pg_dump as well, but not needed if specified as above)
There is a possibility that you need to add the -t parameter to the restore as well, but as far as I remember, it should not be necessary since you only have that table in the dump (however, if you had several tables in the dump, for instance if it is a complete dump of the database, this can be used to only restore parts of the database).
Is there a simple way to create a copy of a database or schema in PostgreSQL 8.1?
I'm testing some software which does a lot of updates to a particular schema within a database, and I'd like to make a copy of it so I can run some comparisons against the original.
If it's on the same server, you just use the CREATE DATABASE command with the TEMPLATE parameter. For example:
CREATE DATABASE newdb WITH TEMPLATE olddb;
pg_dump with the --schema-only option.
If you have to copy the schema from the local database to a remote database, you may use one of the following two options.
Option A
Copy the schema from the local database to a dump file.
pg_dump -U postgres -Cs database > dump_file
Copy the dump file from the local server to the remote server.
scp localuser#localhost:dump_file remoteuser#remotehost:dump_file
Connect to the remote server.
ssh remoteuser#remotehost
Copy the schema from the dump file to the remote database.
psql -U postgres database < dump_file
Option B
Copy the schema directly from the local database to the remote database without using an intermediate file.
pg_dump -h localhost -U postgres -Cs database | psql -h remotehost -U postgres database
This blog post might prove helpful for you if you want to learn more about options for copying the database using pg_dump.
This can be done by running the following command:
CREATE DATABASE [Database to create] WITH TEMPLATE [Database to copy] OWNER [Your username];
Once filled in with your database names and your username, this will create a copy of the specified database. This will work as long as there are no other active connections to the database you wish to copy. If there are other active connections you can temporarily terminate the connections by using this command first:
SELECT pg_terminate_backend(pg_stat_activity.pid)
FROM pg_stat_activity
WHERE pg_stat_activity.datname = '[Database to copy]'
AND pid <> pg_backend_pid();
A good article that I wrote for Chartio's Data School which goes a bit more in depth on how to do this can be found here:
https://dataschool.com/learn/how-to-create-a-copy-of-a-database-in-postgresql-using-psql