Bulk insert, update if on conflict (bulk upsert) on Postgres - sql

I am writing a data-mining program, which bulk inserts user data.
The current SQL is just a plain bulk insert:
insert into USERS(
id, username, profile_picture)
select unnest(array['12345']),
unnest(array['Peter']),
unnest(array['someURL']),
on conflict (id) do nothing;
How do I do an update if on conflict? I tried:
...
unnest(array['Peter']) as a,
unnest(array['someURL']) as b,
on conflict (id) do
update set
username = a,
profile_picture = b;
But it throws There is a column named "a" in table "*SELECT*", but it cannot be referenced from this part of the query. error.
EDIT:
Table of USERS is very simple:
create table USERS (
id text not null primary key,
username text,
profile_picture text
);

Turns out a special table named excluded contains the row-to-be-inserted
(strange name though)
insert into USERS(
id, username, profile_picture)
select unnest(array['12345']),
unnest(array['Peter']),
unnest(array['someURL'])
on conflict (id) do
update set
username = excluded.username,
profile_picture = excluded.profile_picture;
http://www.postgresql.org/docs/9.5/static/sql-insert.html#SQL-ON-CONFLICT
The SET and WHERE clauses in ON CONFLICT DO UPDATE have access to the existing row using the table's name (or an alias), and to rows proposed for insertion using the special excluded table...

For bulk insert from another table if they are identical you can do it like that :
INSERT INTO table_a (SELECT * FROM table_b)
ON CONFLICT ON CONSTRAINT "pk_guid"
DO UPDATE SET
column1 = excluded.column1,
column2 = excluded.column2,
column3 = excluded.column3,
...... ;

Related

Postgres - upsert on passed parameter

In my project I am using Postgres 12 and I want to use one sql query to INSERT OR UPDATE..
My syntax is not correct.
UPDATE: Insert works but updating does not.
ERROR: Invalid parameter number: :name"
'INSERT INTO user (
name, url
) VALUES (:name, :url)
ON CONFLICT (id)
WHERE id = :userId
DO UPDATE SET
name = :name,
url = :url'
I am using this EXAMPLE to do UPSERT and I want to UPDATE if userId is passed and if not to INSERT new row.
Thanks
BEGIN;
CREATE TABLE users (
user_id bigint PRIMARY KEY GENERATED BY DEFAULT AS IDENTITY,
name text,
url text
);
INSERT INTO users (name, url)
VALUES ('Hello', 'world');
COMMIT;
using psql: https://www.postgresql.org/docs/current/app-psql.html
set variable in psql: How do you use script variables in psql?
You can also set variable in an transaction.
BEGIN;
\set name 'hi'
\set url 'yech'
INSERT INTO users (user_id, name, url)
VALUES (1, :'name', :'url')
ON CONFLICT (user_id)
DO UPDATE SET
name = EXCLUDED.name, url = EXCLUDED.url
RETURNING
*;
TABLE users;
COMMIT;

SQL Server trigger validation to insert or update

I have to develop a trigger for multiple value update for the Change_Table that contains two columns Article_C (primary key) and Status_C. The trigger activates when the Status_C column is updated and it needs to either insert or update the Target_Tablewhich columns are Article_T (primary key) and Status_T.
The Status column for both tables is a NOT NULL int
For insert validation: insert all Change_Table values (Articles & Status) that do not exist into the Target_Table.
For update condition: if the articles exist in the Target_Table, pass the updated status value to that table.
Status values are 1 or 2.
My current implementation:
CREATE TRIGGER [dbo].[Article_Trigger]
ON [dbo].[Change_Table]
AFTER UPDATE
NOT FOR REPLICATION
AS
IF UPDATE (Status_C)
BEGIN
INSERT INTO Target_Table (Article_T, Status_T)
SELECT Article_C, Status_C
FROM Change_Table
WHERE NOT EXISTS (SELECT * FROM Target_Table
WHERE Change_Table.Article_C = Target_Table.Article_T
AND Change_Table.Status_C = Target_Table.Status_T)
END
This is a rough idea of the insert but it only works once when the Change_Table is first updated and the target table is empty. After that I get an error "Violation of PRIMARY KEY constraint" due to duplicate primary key after updating the status for a second time due to the insert condition.
Query that gave error after second execution:
update Change_Table set status_c = 1 where Article_C in (1000,1003)
How can I modify this query to implement the update status condition to this trigger?
I think you could use deleted or inserted tables. These 2 virtual tables are working nicely with trigger. In you example, you may not need deleted table as you are not logging the historical changes.
CREATE TRIGGER [dbo].[Article_Trigger]
ON [dbo].[Change_Table]
AFTER UPDATE
NOT FOR REPLICATION
AS
IF UPDATE (Status_C)
BEGIN
--insert if not exist
INSERT INTO Target_Table
SELECT A.Article_C, A.Status_C
FROM inserted as A --the updated records from the source table will exist in the inserted virtual table
LEFT JOIN Target_Table as B
ON B.Article_T = A.Article_C
WHERE B.Article_T IS NULL
--update if exist
UPDATE A
SET A.Status_T = B.Article_C
FROM Target_Table as A
INNER JOIN inserted as B --the updated records from the source table will exist in the inserted virtual table
ON B.Article_C = A.Article_T
END

How can I add prefix to SQLite auto_increment field?

Is there a way to add prefix to an auto increment field in SQLite so that when new records are added, the auto increment value contain the prefix?
Is there a way to add prefix to an auto increment field in sqlite so
that when new records are added, the auto increment value contain the
prefix?
No.
Such a column is a special column that is an alias of the rowid column. Such a column MUST be an integer value.
The intended use of such a column is really to be able to uniquely identify a row.
BUT.....
However, it would be possible to always return a value with a prefix e.g. by using
SELECT 'prefix'||mycolumn FROM mytable;
Working Example
:-
DROP TABLE IF EXISTS mytable;
CREATE TABLE IF NOT EXISTS mytable (mycolumn INTEGER PRIMARY KEY, myothercolumn TEXT);
INSERT INTO mytable (myothercolumn) VALUES('Fred'),('Mary'),('Jane'),('andsoon');
SELECT 'myprefix'||mycolumn, myothercolumn FROM mytable;
Note there is no need for the AUTOINCREMENT keyword that only imposes a constraint but is inefficient AUTOINCREMENT
Result
Mimicking what you want
It would be possible to replicate what you want e.g. :-
DROP TRIGGER IF EXISTS mytable2manager;
DROP TABLE IF EXISTS mytable2;
CREATE TABLE IF NOT EXISTS mytable2 (mycolumn TEXT, myothercolumn TEXT);
CREATE TRIGGER IF NOT EXISTS mytable2manager AFTER INSERT ON mytable2 BEGIN
UPDATE mytable2 SET mycolumn = 'myprefix'||(SELECT count() FROM mytable2) WHERE rowid = new.rowid;
END;
INSERT INTO mytable2 (myothercolumn) VALUES('Fred'),('Mary'),('Jane'),('andsoon');
SELECT * FROM mytable2;
This creates a table along with a TRIGGER and then inserts the same data as above.
The TRIGGER is actioned whenever a row is inserted and in this case it mimics what the SQLite auto-generation of the rowid value does prefixing the value
it could alternatively use SET mycolumn = 'myprefix'||rowid
Without the TRIGGER mycolumn would be null
Result
Additional
You could even have the TRIGGER in the mimicking method apply different prefixes. The following example uses a different prefix when the value in the myothercolumn starts with an M
:-
DROP TRIGGER IF EXISTS mytable2manager;
DROP TABLE IF EXISTS mytable2;
CREATE TABLE IF NOT EXISTS mytable2 (mycolumn TEXT, myothercolumn TEXT);
CREATE TRIGGER IF NOT EXISTS mytable2manager AFTER INSERT ON mytable2 BEGIN
UPDATE mytable2 SET mycolumn = CASE WHEN substr(new.myothercolumn,1,1) = 'M' THEN 'myprefix' ELSE 'myotherprefix' END||(SELECT count() FROM mytable2) WHERE rowid = new.rowid;
-- UPDATE mytable2 SET mycolumn = 'myprefix'||rowid WHERE rowid = new.rowid;
END;
INSERT INTO mytable2 (myothercolumn) VALUES('Fred'),('Mary'),('Jane'),('andsoon');
SELECT * FROM mytable2;
Result
The simplest method is probably to use a view:
create view v_t as
select ('prefix' || id) as new_id, t.*
from t;

SQLite renumber ID using cycle

Hello I have table with many inserted row. I need to renumber all row by id and order them.
I have found this code but it does not work for me.
SET #i = 100;
UPDATE "main"."Categories" SET ID = (#i := #i +1) WHERE "Name" = "White";
ALTER TABLE "main"."Categories" AUTO_INCREMENT = 1
So using code above I expected renumbered all records that have name - white and start insert them from 100 with increment 1. But it is not work for me. Maybe there is some problem in my code but maybe it is a difference between SQL and SQLite query.
This how I created table:
CREATE TABLE Categories (id INTEGER PRIMARY KEY, Name TEXT, Free NUMERIC)
I hope there is already made solution how to do it because I don't want to do it manually :)
That code is not standard SQL.
SQLite does not have many programming constructs because it is designed to be an embedded database where it is more natural to have the logic in the host language.
If you want to do this in SQL, try the following:
First, create a temporary table so that we have an autoincrement column that can be used for counting:
CREATE TEMPORARY TABLE new_ids(i INTEGER PRIMARY KEY, old_id INTEGER);
Insert a dummy record to ensure that the next new record starts at 100, then insert all the IDs of the Categories table that you want to change:
INSERT INTO new_ids VALUES(99, NULL);
INSERT INTO new_ids SELECT NULL, id FROM "Categories" WHERE "Name" = 'White';
DELETE FROM new_ids WHERE i = 99;
Then we can change all these IDs in the original table:
UPDATE "Categories"
SET id = (SELECT i FROM new_ids WHERE old_id = "Categories".id)
WHERE id IN (SELECT old_id FROM new_ids);
DROP TABLE new_ids;

SQL Trigger to update a field on insert if the inserted value conflicts with the unique constraint

I have a Table called Albums which contains a field IsDeleted and a field "Name".
Name is a unique key. My Question is:
Can I create a trigger which updates IsDeleted to False if it is set to True and the same name is inserted into the table by the user?
Of course the insert statement should somehow be changed to an update statement.
I'm Using MSSQL 2008 and I'm very new to triggers and SQL ingeneral
Thanks for your help!
Yes, you can do this with an instead of trigger. Example:
create trigger albums_ins_trig on albums instead of insert as begin
-- update albums which already exist and are being inserted again
update albums set IsDeleted=0 where IsDeleted=1
and exists (select * from inserted where albums.name=inserted.name)
-- insert new albums
insert into albums select * from inserted
where not exists (select * from albums where albums.id=inserted.id)
end