I have an issue with automatic database creation. I have created create_database.sql file with following code:
BEGIN;
\i schema.sql
\i domains.sql
\i sequences.sql
\i tables.sql
\i dane/insert_users.sql
\i dane/insert_default_expenses.sql
\i dane/insert_default_incomes.sql
\i dane/insert_default_payments.sql
\i dane/insert_user_incomes.sql
\i dane/insert_user_expenses.sql
\i dane/insert_user_payments.sql
\i dane/insert_expenses.sql
\i dane/insert_incomes.sql
\i procedures.sql
\i views.sql
COMMIT;
I would like to automatically create database using that transaction. So I am typing inside SQL SHELL:
psql budget < 'C:\Users\xxx\Desktop\yyy\create_database.sql'
There is no error returned but also DB is not being updated. Any ideas why ?
For running a SQL file you need to provide the file by using the following command
./psql -U postgres -d postgres -p 5432 -f 'C:\Users\xxx\Desktop\yyy\create_database.sql'
Also in create_database.sql you need to provide full path for all sql files that you are executing from inside the tracnsaction block e.g
\i C:\Users\xxx\Desktop\yyy\schema.sql
I wanted to run the alter table command using bash script. I managed to create the table, load the basemodel, create config tables and etc. The script will login to the postgres database before it is execute the alter table command. It stuck as (abcdb=> ) without proceed to the alter table command. Is there any way to make sure the alter table able to execute?
The login as
psql -h 191.169.51.10 -d abcdb -U myname
alter table attr_config rename regexp to regexp_val;
alter table class_action_config rename type to type_name;
alter table funcitem_config rename type to type_name;
In order to run a script like this you need to redirect the SQL/DML (alter table statements) into the psql command. Otherwise bash won't understand what to do with them.
psql -h 191.169.51.10 -d abcdb -U myname << EOF
alter table attr_config rename regexp to regexp_val;
alter table class_action_config rename type to type_name;
alter table funcitem_config rename type to type_name;
EOF
Alternatively you can put your SQL/DML into a separate file and have psql to read from that:
psql -h 191.169.51.10 -d abcdb -U myname < alter_statements.sql
Or
psql -h 191.169.51.10 -d abcdb -U myname -f alter_statements.sql
So what I want to do here is to run a script while connected to a database I already had using pgAdmin3. The script contains a create role, tablespace, database and a create schema and several tables under that schema.
The problem here is that when I run the script it creates the new role, tablespace and database correctly. It also creates the schema and the tables correctly but with a problem, the schema is created under the database, from which I ran the script, instead of the newly created database. The script is more or less like this.
CREATE ROLE "new_role" ... ;
CREATE TABLESPACE "new_space"
OWNER "new_role"
LOCATION '/home/...';
CREATE DATABASE "new_db"
WITH OWNER = "new_role"
TABLESPACE = "new_space";
CREATE SCHEMA "schema" AUTHORIZATION "new_role" ;
CREATE TABLE IF NOT EXISTS "schema"."new_table"(
...
) TABLESPACE "new_space";...
...
I already saw a solution with a \connect foo; but that is not what I wanted, I wanted it to somehow connect within the script without running things separately and running \connect foo in the terminal.
Can anyone tell me if there is anyway to do this and help me come out with a solution to this problem?
Use psql and split it up into two scripts . You can save the scripts in .sql files, and then run psql to connect to the DB you want to run each script against all on the same command line (with && in between each command). The two psql commands could be combined into one bash script so it's only one command that you need to run.
Something like this, if the script were named foo.sql:
psql -X -h <host> -U <user> -p <port> -f foo.sql <db_name>
The first script could have the create role, create tablespace and create database commands, connecting to the postgres db or a template DB, and the second script could have the rest of the commands.
You could also use createdb from the bash script instead of CREATE DATABASE.
Using pgAdminIV:
1- right click on default database "postgres"
2- select create database, give a name f.e. "newdatabase"
3- click on "newdatabase" (to establish connection)
4- open the query tool
5- import, write or paste your code
6- run your code f.e.: CREATE SCHEMA newschema;
It works for me...
I want to create a database which does not exist through JDBC. Unlike MySQL, PostgreSQL does not support create if not exists syntax. What is the best way to accomplish this?
The application does not know if the database exists or not. It should check and if the database exists it should be used. So it makes sense to connect to the desired database and if connection fails due to non-existence of database it should create new database (by connecting to the default postgres database). I checked the error code returned by Postgres but I could not find any relevant code that species the same.
Another method to achieve this would be to connect to the postgres database and check if the desired database exists and take action accordingly. The second one is a bit tedious to work out.
Is there any way to achieve this functionality in Postgres?
Restrictions
You can ask the system catalog pg_database - accessible from any database in the same database cluster. The tricky part is that CREATE DATABASE can only be executed as a single statement. The manual:
CREATE DATABASE cannot be executed inside a transaction block.
So it cannot be run directly inside a function or DO statement, where it would be inside a transaction block implicitly. SQL procedures, introduced with Postgres 11, cannot help with this either.
Workaround from within psql
You can work around it from within psql by executing the DDL statement conditionally:
SELECT 'CREATE DATABASE mydb'
WHERE NOT EXISTS (SELECT FROM pg_database WHERE datname = 'mydb')\gexec
The manual:
\gexec
Sends the current query buffer to the server, then treats each column of each row of the query's output (if any) as a SQL statement to be executed.
Workaround from the shell
With \gexec you only need to call psql once:
echo "SELECT 'CREATE DATABASE mydb' WHERE NOT EXISTS (SELECT FROM pg_database WHERE datname = 'mydb')\gexec" | psql
You may need more psql options for your connection; role, port, password, ... See:
Run batch file with psql command without password
The same cannot be called with psql -c "SELECT ...\gexec" since \gexec is a psql meta‑command and the -c option expects a single command for which the manual states:
command must be either a command string that is completely parsable by the server (i.e., it contains no psql-specific features), or a single backslash command. Thus you cannot mix SQL and psql meta-commands within a -c option.
Workaround from within Postgres transaction
You could use a dblink connection back to the current database, which runs outside of the transaction block. Effects can therefore also not be rolled back.
Install the additional module dblink for this (once per database):
How to use (install) dblink in PostgreSQL?
Then:
DO
$do$
BEGIN
IF EXISTS (SELECT FROM pg_database WHERE datname = 'mydb') THEN
RAISE NOTICE 'Database already exists'; -- optional
ELSE
PERFORM dblink_exec('dbname=' || current_database() -- current db
, 'CREATE DATABASE mydb');
END IF;
END
$do$;
Again, you may need more psql options for the connection. See Ortwin's added answer:
Simulate CREATE DATABASE IF NOT EXISTS for PostgreSQL?
Detailed explanation for dblink:
How do I do large non-blocking updates in PostgreSQL?
You can make this a function for repeated use.
another alternative, just in case you want to have a shell script which creates the database if it does not exist and otherwise just keeps it as it is:
psql -U postgres -tc "SELECT 1 FROM pg_database WHERE datname = 'my_db'" | grep -q 1 || psql -U postgres -c "CREATE DATABASE my_db"
I found this to be helpful in devops provisioning scripts, which you might want to run multiple times over the same instance.
For those of you who would like an explanation:
-c = run command in database session, command is given in string
-t = skip header and footer
-q = silent mode for grep
|| = logical OR, if grep fails to find match run the subsequent command
If you don't care about the data, you can drop database first and then recreate it:
DROP DATABASE IF EXISTS dbname;
CREATE DATABASE dbname;
PostgreSQL does not support IF NOT EXISTS for CREATE DATABASE statement. It is supported only in CREATE SCHEMA. Moreover CREATE DATABASE cannot be issued in transaction therefore it cannot be in DO block with exception catching.
When CREATE SCHEMA IF NOT EXISTS is issued and schema already exists then notice (not error) with duplicate object information is raised.
To solve these problems you need to use dblink extension which opens a new connection to database server and execute query without entering into transaction. You can reuse connection parameters with supplying empty string.
Below is PL/pgSQL code which fully simulates CREATE DATABASE IF NOT EXISTS with same behavior like in CREATE SCHEMA IF NOT EXISTS. It calls CREATE DATABASE via dblink, catch duplicate_database exception (which is issued when database already exists) and converts it into notice with propagating errcode. String message has appended , skipping in the same way how it does CREATE SCHEMA IF NOT EXISTS.
CREATE EXTENSION IF NOT EXISTS dblink;
DO $$
BEGIN
PERFORM dblink_exec('', 'CREATE DATABASE testdb');
EXCEPTION WHEN duplicate_database THEN RAISE NOTICE '%, skipping', SQLERRM USING ERRCODE = SQLSTATE;
END
$$;
This solution is without any race condition like in other answers, where database can be created by external process (or other instance of same script) between checking if database exists and its own creation.
Moreover when CREATE DATABASE fails with other error than database already exists then this error is propagated as error and not silently discarded. There is only catch for duplicate_database error. So it really behaves as IF NOT EXISTS should.
You can put this code into own function, call it directly or from transaction. Just rollback (restore dropped database) would not work.
Testing output (called two times via DO and then directly):
$ sudo -u postgres psql
psql (9.6.12)
Type "help" for help.
postgres=# \set ON_ERROR_STOP on
postgres=# \set VERBOSITY verbose
postgres=#
postgres=# CREATE EXTENSION IF NOT EXISTS dblink;
CREATE EXTENSION
postgres=# DO $$
postgres$# BEGIN
postgres$# PERFORM dblink_exec('', 'CREATE DATABASE testdb');
postgres$# EXCEPTION WHEN duplicate_database THEN RAISE NOTICE '%, skipping', SQLERRM USING ERRCODE = SQLSTATE;
postgres$# END
postgres$# $$;
DO
postgres=#
postgres=# CREATE EXTENSION IF NOT EXISTS dblink;
NOTICE: 42710: extension "dblink" already exists, skipping
LOCATION: CreateExtension, extension.c:1539
CREATE EXTENSION
postgres=# DO $$
postgres$# BEGIN
postgres$# PERFORM dblink_exec('', 'CREATE DATABASE testdb');
postgres$# EXCEPTION WHEN duplicate_database THEN RAISE NOTICE '%, skipping', SQLERRM USING ERRCODE = SQLSTATE;
postgres$# END
postgres$# $$;
NOTICE: 42P04: database "testdb" already exists, skipping
LOCATION: exec_stmt_raise, pl_exec.c:3165
DO
postgres=#
postgres=# CREATE DATABASE testdb;
ERROR: 42P04: database "testdb" already exists
LOCATION: createdb, dbcommands.c:467
I had to use a slightly extended version #Erwin Brandstetter used:
DO
$do$
DECLARE
_db TEXT := 'some_db';
_user TEXT := 'postgres_user';
_password TEXT := 'password';
BEGIN
CREATE EXTENSION IF NOT EXISTS dblink; -- enable extension
IF EXISTS (SELECT 1 FROM pg_database WHERE datname = _db) THEN
RAISE NOTICE 'Database already exists';
ELSE
PERFORM dblink_connect('host=localhost user=' || _user || ' password=' || _password || ' dbname=' || current_database());
PERFORM dblink_exec('CREATE DATABASE ' || _db);
END IF;
END
$do$
I had to enable the dblink extension, plus i had to provide the credentials for dblink.
Works with Postgres 9.4.
If you can use shell, try
psql -U postgres -c 'select 1' -d $DB &>dev/null || psql -U postgres -tc 'create database $DB'
I think psql -U postgres -c "select 1" -d $DB is easier than SELECT 1 FROM pg_database WHERE datname = 'my_db',and only need one type of quote, easier to combine with sh -c.
I use this in my ansible task
- name: create service database
shell: docker exec postgres sh -c '{ psql -U postgres -tc "SELECT 1" -d {{service_name}} &> /dev/null && echo -n 1; } || { psql -U postgres -c "CREATE DATABASE {{service_name}}"}'
register: shell_result
changed_when: "shell_result.stdout != '1'"
The best way is just running the SQL.
CREATE DATABASE MY_DATABASE;
if the database already exists, it throws "database already exists error" which you can do whatever you want to do, otherwise it creates the database. I do not think it will create a new database on top of yours. :D
After reading through all these in my opinion complicated solutions that are terrible work arounds for the lack of the IF NOT EXIST option for postgres user creation, I forgot almost there is a simply way to just handle it at the shell level. Even though it might not be what some want, I think a lot of folks want simplicity and not creating procedures and complicated constructs.
I'm using docker, here are the important snippets from my bash script that loads data in a devsetup:
execute_psql_command_pipe () {
$DOCKER_COMMAND exec -it $POSTGRES_CONTAINER bash -c "echo \"$1\"| psql -h localhost -U postgres || echo psql command failed - object likely exists"
}
read -r -d '' CREATE_USER_COMMANDS << EOM
create user User1 WITH PASSWORD 'password';
create user User2 WITH PASSWORD 'password';
EOM
execute_psql_command_pipe "$CREATE_USER_COMMANDS"
There are a few things wrong with it, but it's the simplest way I could find to make it do what I want: create on first pass of script, continue on second pass when existing.
By the way, the echo output does not show, but the commands continue because the echo command exits with 0.
The same can be done for any command (like db create).
This obviously fails (or succeeds, depending on perspective) for any other error that may occur too, but you get the psql output printer so more handling can be added.
Another flavor if running with psql
psql --quiet -d postgres -c "CREATE DATABASE $DB_DATABASE;" || :
Note, this will still output ERROR: database "" already exists but can be ignored.
One simple clean way to do it that I ended up using:
createdb $DATABASE 2> /dev/null || echo "database already exists"
If you expect other error than database "x" already exists that obviously won't work (e.g. permission denied). In any case, if that is a concern, one can always perform such checks prior to this point.
Don't forget to set the value for DATABASE, and to pass in the required switches for the createdb command. Preferably you can also do like:
export PGHOST=localhost
export PGUSER=user
export PGPASSWORD=p455w0rd
...
Just create the database using createdb CLI tool:
PGHOST="my.database.domain.com"
PGUSER="postgres"
PGDB="mydb"
createdb -h $PGHOST -p $PGPORT -U $PGUSER $PGDB
If the database exists, it will return an error:
createdb: database creation failed: ERROR: database "mydb" already exists
I'm searching for a simple way to delete all data from a database and keep the structure (table, relationship, etc...).
I using postgreSQL but I think, if there a command to do that, it's not specific to postgres.
Thanks,
Damien
Dump the schema using pg_dump. drop the database, recreate it and load the schema.
Dump you database schema (the -s tag) to a file:
pg_dump -s -f db.dump DB-NAME
Delete the database:
dropdb DB-NAME
Recreate it:
createdb DB-NAME
Restore the schema only:
pg_restore db.dump > psql DB-NAME
This should work on PostgreSQL; Other DBMS might have their own tools for that. I do no know of any generic tool to do it.
EDIT:
Following comments, you might want to skip the dropdb command, and simply create another database with the dumped schema. If all went through well, you can drop the old database:
pg_dump -s -f db.dump DB-NAME
createdb DB-NEW-NAME
pg_restore db.dump > psql DB-NEW-NAME
At this point, you have the full database at DB-NAME, and an empty schema at DB-NEW-NAME. after you're sure everything is OK, use dropdb DB-NAME.
You can do something like this:
export PGUSER=your_pg_user
export PGHOST=database.host
export PGPORT=port
export PGDATABASE=your_database
psql -qAtX -c "select 'TRUNCATE table ' || quote_ident(table_schema) || '.' || quote_ident(table_name) || ' CASCADE;' from information_schema.tables where table_type = 'BASE TABLE' and not table_schema ~ '^(information_schema|pg_.*)$'" | psql -qAtX
It will do what's necessary.
Of course these exports are not necessary, but they will make it simpler to run 2 copies of psql without having to givem them all standard -U, -d, and so on, switches.
One thing though - using TRUNCATE to do so, while faster than DELETE, has it's drowbacks - for example - it is not being replicated by Slony and any other replication system that works on triggers. Unless you are working on PostgreSQL 8.4, and your replication knows how to use triggers on TRUNCATE.
I'm not a Postgres guy, but one option would be to iterate through the tables and issue a Truncate command against each one. You'll have to take otable relationships into account, though - you will not be able to delete reference data before data that refers to it, for example.
In pgAdmin you can do:
Right-click database -> backup, select "Schema only"
Drop the database
Create a new database and name it like the former
Right-click the new database -> restore -> select the backup, select "Schema only"
your can delete all records of your database without restriction of foreign keys by following three steps
Take script of your Database
Right Click on your database (your DB Name)
click on task and then "Generate script"
Specify location
Delete your database base
recreate a database with the same name and run you generated script
This way you can empty all of your database