Logging SQL script errors - sql

I have a SQL script for postgres file with following command.
COPY product_master(productId, productName) FROM 'product.txt' DELIMITERS ',' CSV;
I want to handle errors of this command (log the error)
example
ERROR: duplicate key value violates.
Does the COPY command return any value ? If no, then how to log the output of the shell script?

You can log any and all messages (error, warning, ..) with a plethora of additional information in the database log file. This is standard behavior. Of course your database cluster has to be configured to do so. Read the fine manual here.
Dpending on your client you should also be able to get error messages as direct answer from the database server. Note that errors are reported on a different stream than data output. Like stout and stderr in the shell.
From the shell you would probably call psql -f to execute a script. See what happens in this demo:
Create a dummy SQL script in the shell:
vim test.sql
Put something like this into it:
CREATE temp table x (a int primary key, b int);
insert into x values (1,2),(3,4);
COPY x TO '/var/lib/postgres/dev/test.out';
COPY x FROM '/var/lib/postgres/dev/test.out';
Execute it:
psql mydb -f test.sql
Output depends on various settings like client_min_messages:
psql:test.sql:2: NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "x_pkey" for table "x"
CREATE TABLE
INSERT 0 2
psql:test.sql:4: ERROR: duplicate key value violates unique constraint "x_pkey"
KONTEXT: COPY x, line 1: "1 2"
Since I have configured log_statement = all (among others) my server log reads:
2011-11-15 22:36:23 CET postgres LOG: statement: CREATE temp table x (a int primary key, b int);
2011-11-15 22:36:23 CET postgres LOG: statement: insert into x values (1,2),(3,4);
2011-11-15 22:36:23 CET postgres LOG: statement: COPY x FROM '/var/lib/postgres/dev/test.out';
2011-11-15 22:36:23 CET postgres ERROR: duplicate key value violates unique constraint "x_pkey"
2011-11-15 22:36:23 CET postgres CONTEXT: COPY x, line 1: "1 2"
2011-11-15 22:36:23 CET postgres STATEMENT: COPY x FROM '/var/lib/postgres/dev/test.out';
I would not use log_statement = all on a productive server. That produces huge log files.

Related

Where does psql import import to?

I got a SQL dump file and try to import it into my database with the following command:
psql -U postgres timetable < /tmp/restores/2\ Restore\ Rooms.sql
It contains several SET commands, a CREATE TABLE timetable.rooms and several other commands.
I logged in via psql -U postgres before and created the schema timetable and ran the above import command, but it returned some errors. So I logged back in and dropped the schema timetable to start again. After that the import returned the following:
postgres#ubuntu:~$ psql -U postgres timetable < /tmp/restores/2\ Restore\ Rooms.sql
SET
SET
SET
SET
SET
set_config
------------
(1 row)
SET
SET
SET
SET
SET
SET
ERROR: relation "rooms" already exists
ALTER TABLE
ERROR: relation "rooms_id_seq" already exists
ALTER TABLE
ALTER SEQUENCE
ALTER TABLE
ERROR: duplicate key value violates unique constraint "rooms_pkey"
DETAIL: Key (id)=(50) already exists.
CONTEXT: COPY rooms, line 1
setval
--------
57
(1 row)
ERROR: multiple primary keys for table "rooms" are not allowed
postgres#ubuntu:~$ psql -U postgres
psql (12.4 (Ubuntu 12.4-0ubuntu0.20.04.1))
Type "help" for help.
I then tried to access the table timetable.rooms but it wasn't there:
postgres=# select * from
information_schema. pg_catalog. pg_temp_1. pg_toast. pg_toast_temp_1. public.
Where did it import the tables then and how do I access it, if not via psql -U postgres?

Drop table only if it exists, or ignore drop error

I have a table MYLOG and would like to try drop it before creating it using the SQL script below.
If the table does not exist yet, the error below is throw.
How could I bypass this error if the table does not exist?
The schema gets set in an earlier script, which is not available in the SQL script:
set current schema MYSCHEMA
SQL script:
DROP TABLE MYLOG;
CREATE TABLE MYLOG (
TIME_STARTED TIMESTAMP NOT NULL,
USER_EMAIL VARCHAR(254) NOT NULL,
CONSTRAINT PK_TIME_STARTED_USER_EMAIL PRIMARY KEY (TIME_STARTED, USER_EMAIL)) ORGANIZE BY ROW;
COMMIT;
Error:
DROP TABLE MYLOG
SQLError: rc = 0 (SQL_SUCCESS)
SQLGetDiagRec: SQLState : S0002
fNativeError : -204
szErrorMsg : [IBM][CLI Driver][DB2/6000] SQL0204N "MYSCHEMA.MYLOG" is an undefined name. SQLSTATE=42704
This is a FAQ
There's more than one way to do it.
You can use compound-SQL in your script with a continue-handler for the SQLSTATE corresponding to the error you get if the table is not found, but this requires that you also use an alternative statement delimiter like shown below
--#SET TERMINATOR #
set current schema myschema#
BEGIN
DECLARE CONTINUE HANDLER FOR SQLSTATE '42704'
BEGIN end;
EXECUTE IMMEDIATE 'DROP TABLE MYLOG';
END #
CREATE TABLE MYLOG(... )#
You can also change the abort-on-first-error logic (if you use +s when running your script via the command line). You can udate the Db2 CLP options on the fly inside your script via update command options using s off (to continue on error) or update command options using s on to abort on error.
by using this query
select tabname from syscat.tables where
tabschema='myschema' and tabname='MYLOG'
check that table in your schema
if exist then
drop table myschema.MYLOG
then create

Duplicating a SQLite table, indexes, and data [duplicate]

Is there an easy way to copy an existing table structure to a new one?
(dont need the data, only the structure -> like id INTEGER, name varchar(20) ...)
Thx
You could use a command like this:
CREATE TABLE copied AS SELECT * FROM mytable WHERE 0
but due to SQLite's dynamic typing, most type information would be lost.
If you need just a table that behaves like the original, i.e., has the same number and names of columns, and can store the same values, this is enough.
If you really need the type information exactly like the original, you can read the original SQL CREATE TABLE statement from the sqlite_master table, like this:
SELECT sql FROM sqlite_master WHERE type='table' AND name='mytable'
SQLite cannot clone table with PK, defaults and indices.
Hacking by another tool is necessary.
In shell, replace the table name by sed.
sqlite3 dbfile '.schema oldtable' | sed '1s/oldtable/newtable/' | sqlite3 dbfile
And you can check new table.
sqlite3 dbfile '.schema newtable'
Primary key, defaults and indices will be reserved.
I hope this command can help you.
sqlite> .schema
CREATE TABLE [About](
[id],
[name],
[value]);
.schema command will give you structure of About-table how it could be made by programming SQLite interpreter by hand, typing in commands.
Paste in and execute, the CREATE block giving the table new name:
sqlite> CREATE TABLE [AboutToo](
[id],
[name],
[value]);
.tables command now will show you have two tables, old and new, "copied".
sqlite> .tables
About AboutToo
p.s. sqlite> is command prompt you get in console after launching SQLite.exe interpreter. To get it go to www.sqlite.org
Just for the record - This worked for me:
CREATE TABLE mytable (
contact_id INTEGER PRIMARY KEY,
first_name TEXT NOT NULL,
last_name TEXT NOT NULL,
email TEXT NOT NULL UNIQUE,
phone TEXT NOT NULL UNIQUE
);
-- Two variations
INSERT INTO mytable VALUES ( 1, "Donald", "Duck", "noone#nowhere.com", "1234");
INSERT INTO mytable ( contact_id,first_name,last_name,email,phone ) VALUES ( 2, "Daisy", "Duck", "daisy#nowhere.com", "45678");
.output copied.sql
-- Add new table name
.print CREATE TABLE copied (
-- Comment out first line from SQL
SELECT "-- " || sql FROM sqlite_master WHERE type='table';
.output
.read copied.sql
.schema
select * from copied;
Beware that this only works if schema is wrapped after CREATE TABLE mytable (.
Otherwise you'll need some string replacement using .system
Yes by using the SQLiteStudio you can use the last icon in the structure table called create similar table from any existing table.
I would prefer :
> sqlite3 <db_file>
sqlite3 > .output <output_file>
sqlite3 > .dump <table_name>
The line above generates the dump of table that includes DDL and DML statement.
Make changes in this file, i.e. find and replace the table name with new table name
Also, replace "CREATE TRIGGER " with "CREATE TRIGGER <NEWTABLE>_" , this will replace existing triggers with trigger names with a new table name on it. That will make it unique and will not cause conflicts with existing triggers. Once all schema changes are implemented, read it back into database using .read
sqlite3 > .read output_file
This can be scripted in shell file using shell commands like :
echo ".dump <table>" | sqlite3 <db_file> > <table_file>
sed -i.bak "s/\b<table_name>\b/<new_table_name>/g" <table_file>
sed -i.bak "s/\bCREATE TRIGGER \b/CREATE TRIGGER <new_table_name_>/g" <table_file>
echo ".read <table_file>" | sqlite3 <db_file>
rm <table_name>.bak
For example :
If you have table T and new table is TClone in db file D with file F to be created : then
echo ".dump T" | sqlite3 D.sqlite > F
sed -i.bak "s/\bT\b/TClone/g" F
sed -i.bak "s/\bCREATE TRIGGER \b/CREATE TRIGGER TClone_>/g" F
echo ".read F" | sqlite3 D.sqlite
rm T.bak
Finally, you can generalize this script by creating a parameterized version where you can pass source_table, destination_table , db_file as parameters that can be used to clone any table.
I tested this and it works.
Testing :
sqlite3 <database_file>
sqlite3 > select * from <new_table>;
should give you same results as original table. and
sqlite3 > .schema <new_table>
should have same schema as that of original table with a new name.

Failing update table in db2 with SQLCODE: -668, SQLSTATE: 57016, SQLERRMC: 7;

I am using db2 9.5 i have created a column in table which is created successfully but i am not able to update table column and getting following error
[Error] Script lines: 1-1 --------------------------
DB2 SQL error: SQLCODE: -668, SQLSTATE: 57016, SQLERRMC: 7;DB2ADMIN.XCATENTRYEXT
Message: Operation not allowed for reason code "7" on table "DB2ADMIN.XCATENTRYEXT".
Following the some blog/sites on google i found the REORG command as solution as mentioned in following link
http://bytes.com/topic/db2/answers/508869-reorg-tablespace
i have tried the following queries to run on database to solve the problem.
Database["DB2"].ExecuteNonQuery("call SYSPROC.ADMIN_CMD ('REORG TABLE DB2ADMIN.XCATENTRYEXT index CATENTRY_ID INPLACE')")
REORG TABLE DB2ADMIN.XCATENTRYEXT index CATENTRY_ID INPLACE
REORG TABLE DB2ADMIN.XCATENTRYEXT
REORG INDEXES I0000908 FOR TABLE DB2ADMIN.XCATENTRYEXT
but all queries have the same error in result like
DB2 SQL error: SQLCODE: -104, SQLSTATE: 42601, SQLERRMC: Database;BEGIN-OF-STATEMENT;<variable_set>
Message: An unexpected token "Database" was found following "BEGIN-OF-STATEMENT". Expected tokens may include: "<variable_set>".
I am stuck on this error, I am not even able to update any column of that particular table.
It is possible to do REORG through an SQL statement:
CALL SYSPROC.ADMIN_CMD('REORG TABLE SCHEMA.TABLENAME');
It follows from the error message, that you somehow submit the entire string Database["DB2"].ExecuteNonQuery("call SYSPROC.ADMIN_CMD ('REORG TABLE DB2ADMIN.XCATENTRYEXT index CATENTRY_ID INPLACE')") as a SQL statement, which obviously is incorrect.
Simply issue these on the shell command line:
db2 connect to <your database name here>
db2 REORG TABLE DB2ADMIN.XCATENTRYEXT
If you are using tool like dbeaver , you can go to Schema --> table name --> right click --> select tools and you should see option for reorg table

Copy table structure to new table in sqlite3

Is there an easy way to copy an existing table structure to a new one?
(dont need the data, only the structure -> like id INTEGER, name varchar(20) ...)
Thx
You could use a command like this:
CREATE TABLE copied AS SELECT * FROM mytable WHERE 0
but due to SQLite's dynamic typing, most type information would be lost.
If you need just a table that behaves like the original, i.e., has the same number and names of columns, and can store the same values, this is enough.
If you really need the type information exactly like the original, you can read the original SQL CREATE TABLE statement from the sqlite_master table, like this:
SELECT sql FROM sqlite_master WHERE type='table' AND name='mytable'
SQLite cannot clone table with PK, defaults and indices.
Hacking by another tool is necessary.
In shell, replace the table name by sed.
sqlite3 dbfile '.schema oldtable' | sed '1s/oldtable/newtable/' | sqlite3 dbfile
And you can check new table.
sqlite3 dbfile '.schema newtable'
Primary key, defaults and indices will be reserved.
I hope this command can help you.
sqlite> .schema
CREATE TABLE [About](
[id],
[name],
[value]);
.schema command will give you structure of About-table how it could be made by programming SQLite interpreter by hand, typing in commands.
Paste in and execute, the CREATE block giving the table new name:
sqlite> CREATE TABLE [AboutToo](
[id],
[name],
[value]);
.tables command now will show you have two tables, old and new, "copied".
sqlite> .tables
About AboutToo
p.s. sqlite> is command prompt you get in console after launching SQLite.exe interpreter. To get it go to www.sqlite.org
Just for the record - This worked for me:
CREATE TABLE mytable (
contact_id INTEGER PRIMARY KEY,
first_name TEXT NOT NULL,
last_name TEXT NOT NULL,
email TEXT NOT NULL UNIQUE,
phone TEXT NOT NULL UNIQUE
);
-- Two variations
INSERT INTO mytable VALUES ( 1, "Donald", "Duck", "noone#nowhere.com", "1234");
INSERT INTO mytable ( contact_id,first_name,last_name,email,phone ) VALUES ( 2, "Daisy", "Duck", "daisy#nowhere.com", "45678");
.output copied.sql
-- Add new table name
.print CREATE TABLE copied (
-- Comment out first line from SQL
SELECT "-- " || sql FROM sqlite_master WHERE type='table';
.output
.read copied.sql
.schema
select * from copied;
Beware that this only works if schema is wrapped after CREATE TABLE mytable (.
Otherwise you'll need some string replacement using .system
Yes by using the SQLiteStudio you can use the last icon in the structure table called create similar table from any existing table.
I would prefer :
> sqlite3 <db_file>
sqlite3 > .output <output_file>
sqlite3 > .dump <table_name>
The line above generates the dump of table that includes DDL and DML statement.
Make changes in this file, i.e. find and replace the table name with new table name
Also, replace "CREATE TRIGGER " with "CREATE TRIGGER <NEWTABLE>_" , this will replace existing triggers with trigger names with a new table name on it. That will make it unique and will not cause conflicts with existing triggers. Once all schema changes are implemented, read it back into database using .read
sqlite3 > .read output_file
This can be scripted in shell file using shell commands like :
echo ".dump <table>" | sqlite3 <db_file> > <table_file>
sed -i.bak "s/\b<table_name>\b/<new_table_name>/g" <table_file>
sed -i.bak "s/\bCREATE TRIGGER \b/CREATE TRIGGER <new_table_name_>/g" <table_file>
echo ".read <table_file>" | sqlite3 <db_file>
rm <table_name>.bak
For example :
If you have table T and new table is TClone in db file D with file F to be created : then
echo ".dump T" | sqlite3 D.sqlite > F
sed -i.bak "s/\bT\b/TClone/g" F
sed -i.bak "s/\bCREATE TRIGGER \b/CREATE TRIGGER TClone_>/g" F
echo ".read F" | sqlite3 D.sqlite
rm T.bak
Finally, you can generalize this script by creating a parameterized version where you can pass source_table, destination_table , db_file as parameters that can be used to clone any table.
I tested this and it works.
Testing :
sqlite3 <database_file>
sqlite3 > select * from <new_table>;
should give you same results as original table. and
sqlite3 > .schema <new_table>
should have same schema as that of original table with a new name.