have an array of values. I wish to INSERT each of them as records if it does not already exist. can this be done in one statement in SQL?
traditional javascript approach:
[4,5,8]forEach( x => queryParams(
insert into t1( c1 ) values( $1 )
where not exists( select 1 from t1 where c1 = $1`,
[x]
);
what would be ideal is something like
queryParams(
`some fancy SQL here`,
[ `{${join[4,5,8]}}` ]
);
many reasons for this, including limiting the cost of server round trips and transactions.
You can use a correlated sub-query to find the values that don't exist matching a condition:
INSERT INTO Records (X)
SELECT X
FROM unnest(ARRAY[4,5,8]) T (X)
WHERE NOT EXISTS (SELECT * FROM Records WHERE X = T.X);
SQL Fiddle: http://sqlfiddle.com/#!15/e0334/29/0
Edited above to use unnest
Related
I am looking to insert or update values in an SQLite database (version > 3.35) avoiding multiple queries. upsert along with returning seems promising :
CREATE TABLE phonebook2(
name TEXT PRIMARY KEY,
phonenumber TEXT,
validDate DATE
);
INSERT INTO phonebook2(name,phonenumber,validDate)
VALUES('Alice','704-555-1212','2018-05-08')
ON CONFLICT(name) DO UPDATE SET
phonenumber=excluded.phonenumber,
validDate=excluded.validDate
WHERE excluded.validDate>phonebook2.validDate RETURNING name;
This helps me track names corresponding to inserted/modified rows. How to find rows where phonebook2 values conflict with values upserted in above statement, but no insert or update happened due to where clause?
The RETURNING clause can't be used to get non-affected rows.
What you can do is execute a SELECT statement before the UPSERT:
WITH cte(name, phonenumber, validDate) AS (VALUES
('Alice', '704-555-1212', '2018-05-08'),
('Bob','804-555-1212', '2018-05-09')
)
SELECT *
FROM phonebook2 p
WHERE EXISTS (
SELECT *
FROM cte c
WHERE c.name = p.name AND c.validDate <= p.validDate
);
In the CTE you may include as many tuples as you want
Let's imagine that we have a plpgsql (PostgreSQL 10.7) function where there is a query like
INSERT INTO "myTable"
SELECT * FROM "anotherTable"
INNER JOIN "otherTable"
...
So, this query will insert several rows into myTable. In the next query I want to collect the ids which were inserted with some condition. So, my idea was to do it the following:
INSERT INTO "resultTable" rt
SELECT FROM "myTable"
INNER JOIN ...
WHERE rt."id" >= firstInsertedId;
Now the question: how to find this firstInsertedId?
My solution:
select nextval(''"myTable.myTable_id_seq"'') into firstInsertedId;
if firstInsertedId > 1 then
perform setval(''"myTable.myTable_id_seq"'', (firstInsertedId - 1));
end if;
I don't really like the solution as I don't think that it is good for the performance to generate the id, then go back, then generate it again during the insertion.
Thoughts:
was thinking about inserting the ids into variable array and then find the minimum, but no luck.
was considering to use lastval() function, but it seems that it doesn'
t work for me even though in a very similar implementation in MySQL LAST_INSERT_ID() worked just fine.
Any suggestions?
You can do both things in a single statement using a data modifying common table expression. You don't really need PL/pgSQL for that.
with new_rows as (
INSERT INTO my_table
SELECT *
FROM anotherTable
JOIN "otherTable" ...
returning my_table.id
)
insert into resulttable (new_id)
select id
from new_rows;
Another option would be to store the generate IDs in an array.
declare
l_ids integer[];
begin
....
with new_rows as (
INSERT INTO my_table
SELECT *
FROM anotherTable
JOIN "otherTable" ...
returning my_table.id
)
select array_agg(id)
into l_ids
from new_rows;
....
end;
I'm trying to return a set of rows after doing UPDATE.
Something like this.
UPDATE Notis new_noti SET notis = '{}'::noti_record_type[]
FROM (SELECT * FROM Notis WHERE user_id = 2 FOR UPDATE) old_noti
WHERE old_noti.user_id = new_noti.user_id RETURNING unnest(old_noti.notis);
but postgres complains, rightly so:
set-valued function called in context that cannot accept a set
How am I supposed to go about implementing this?
That is, RETURNING a set of rows from SELECTed array after UPDATE?
I'm aware that a function can achieve this using RETURNS SETOF but rather prefer not to if possible.
Use WITH statement:
WITH upd AS (
UPDATE Notis new_noti SET notis = '{}'::noti_record_type[]
FROM (SELECT * FROM Notis WHERE user_id = 2 FOR UPDATE) old_noti
WHERE old_noti.user_id = new_noti.user_id RETURNING old_noti.notis
)
SELECT unnest(notis) FROM upd;
Use a data-modifying CTE.
You can use a set-returning function in the SELECT list, but it is cleaner to move it to the FROM list with a LATERAL subquery since Postgres 9.3. Especially if you need to extract multiple columns (from a row type like you commented). It would also be inefficient to call unnest() multiple times.
WITH upd AS (
UPDATE notis n
SET notis = '{}'::noti_record_type[] -- explicit cast optional
FROM (
SELECT user_id, notis
FROM notis
WHERE user_id = 2
FOR UPDATE
) old_n
WHERE old_n.user_id = n.user_id
RETURNING old_n.notis
)
SELECT n.*
FROM upd u, unnest(u.notis) n; -- implicit CROSS JOIN LATERAL
If the array can be empty and you want to preserve empty / NULL results use LEFT JOIN LATERAL ... ON true. See:
What is the difference between LATERAL JOIN and a subquery in PostgreSQL?
Call a set-returning function with an array argument multiple times
Also, multiple set-returning functions in the same SELECT can exhibit surprising behavior. Avoid that.
This has been sanitized with Postgres 10. See:
What is the expected behaviour for multiple set-returning functions in SELECT clause?
Alternative to unnest multiple arrays in parallel before and after Postgres 10:
Unnest multiple arrays in parallel
Related:
Return pre-UPDATE column values using SQL only
Behavior of composite / row values
Postgres has an oddity when assigning a row type (or composite or record type) from a set-returning function to a column list. One might expect that the row-type field is treated as one column and assigned to the respective column, but that is not so. It is decomposed automatically (one row-layer only!) and assigned element-by-element.
So this does not work as expected:
SELECT (my_row).*
FROM upd u, unnest(u.notis) n(my_row);
But this does (like #klin commented):
SELECT (my_row).*
FROM upd u, unnest(u.notis) my_row;
Or the simpler version I ended up using:
SELECT n.*
FROM upd u, unnest(u.notis) n;
Another oddity: A composite (or row) type with a single field is decomposed automatically. Thus, table alias and column alias end up doing the same in the outer SELECT list:
SELECT n FROM unnest(ARRAY[1,2,3]) n;
SELECT n FROM unnest(ARRAY[1,2,3]) n(n);
SELECT n FROM unnest(ARRAY[1,2,3]) t(n);
SELECT t FROM unnest(ARRAY[1,2,3]) t(n); -- except output column name is "t"
For more than one field, the row-wrapper is preserved:
SELECT t FROM unnest(ARRAY[1,2,3]) WITH ORDINALITY t(n); -- requires 9.4+
Confused? There is more. For composite types (the case at hand) like:
CREATE TYPE my_type AS (id int, txt text);
While this works as expected:
SELECT n FROM unnest(ARRAY[(1, 'foo')::my_type, (2, 'bar')::my_type]) n;
You are in for a surprise here:
SELECT n FROM unnest(ARRAY[(1, 'foo')::my_type, (2, 'bar')::my_type]) n(n);
And that's the error I had: When providing a column list, Postgres decomposes the row and assigns provided names one-by-one. Referring to n in the SELECT list does not return the composite type, but only the (renamed) first element. I had mistakenly expected the row type and tried to decompose with (my_row).* - which only returns the first element nonetheless.
Then again:
SELECT t FROM unnest(ARRAY[(1, 'foo')::my_type, (2, 'bar')::my_type]) t(n);
(Be aware that the first element has been renamed to "n"!)
With the new form of unnest() taking multiple array arguments (Postgres 9.4+):
SELECT *
FROM unnest(ARRAY[(1, 'foo')::my_type, (2, 'bar')::my_type]
, ARRAY[(3, 'baz')::my_type, (4, 'bak')::my_type]) n;
Column aliases only for the first two output columns:
SELECT *
FROM unnest(ARRAY[(1, 'foo')::my_type, (2, 'bar')::my_type]
, ARRAY[(3, 'baz')::my_type, (4, 'bak')::my_type]) n(a, b);
Column aliases for all output columns:
SELECT *
FROM unnest(ARRAY[(1,'foo')::my_type, (2,'bar')::my_type]
, ARRAY[(3,'baz')::my_type, (4,'bak')::my_type]) n(a,b,c,d);
db<>fiddle here
Old sqlfiddle
Probably
For:
SELECT *
FROM unnest (ARRAY[(1, 'foo')::my_type, (2, 'bar')::my_type]
, ARRAY[(3, 'baz')::my_type, (4, 'bak')::my_type]) n(a, b);
Use:
SELECT *
FROM unnest (ARRAY[(1, 'foo')::text, (2, 'bar')::text]
, ARRAY[(3, 'baz')::text, (4, 'bak')::text]) WITH ORDINALITY AS t(first_col, second_col);
CREATE TABLE object (
object_id serial,
object_attribute_1 integer,
object_attribute_2 VARCHAR(255)
)
-- primary key object_id
-- btree index on object_attribute_1, object_attribute_2
Here is what I currently have:
SELECT * FROM object
WHERE (object_attribute_1=100 AND object_attribute_2='Some String') OR
(object_attribute_1=200 AND object_attribute_2='Some other String') OR
(..another row..) OR
(..another row..)
When the query returns, I check for what is missing (thus, does not exist in the database).
Then I will make an multiple row insert:
INSERT INTO object (object_attribute_1, object_attribute_2)
VALUES (info, info), (info, info),(info, info)
Then I will select what I just inserted
SELECT ... WHERE (condition) OR (condition) OR ...
And at last, I will merge the two selects on the client side.
Is there a way that I can combine these 3 queries, into one single queries, where I will provide all the data, and INSERT if the records do not already exist and then do a SELECT in the end.
Your suspicion was well founded. Do it all in a single statement using a data-modifying CTE (Postgres 9.1+):
WITH list(object_attribute_1, object_attribute_2) AS (
VALUES
(100, 'Some String')
, (200, 'Some other String')
, .....
)
, ins AS (
INSERT INTO object (object_attribute_1, object_attribute_2)
SELECT l.*
FROM list l
LEFT JOIN object o1 USING (object_attribute_1, object_attribute_2)
WHERE o1.object_attribute_1 IS NULL
RETURNING *
)
SELECT * FROM ins -- newly inserted rows
UNION ALL -- append pre-existing rows
SELECT o.*
FROM list l
JOIN object o USING (object_attribute_1, object_attribute_2);
Note, there is a tiny time frame for a race condition. So this might break if many clients try it at the same time. If you are working under heavy concurrent load, consider this related answer, in particular the part on locking or serializable transaction isolation:
Postgresql batch insert or ignore
I Can't run this query with SQLite
if 0<(select COUNT(*) from Repetition where (Word='behnam' and Topic='mine'))
begin
update Repetition set Counts=1+ (select Counts from Repetition where (Word='behnam' and Topic='mine'))
end
else
begin
insert Repetition(Word,Topic,Counts)values('behnam','mine',1)
end
It says "Syntax error near IF"
How can I solve the problem
SQLite does not have an IF statement (see the list of supported queries)
Insetad, check out out ERIC B's suggestion on another thread. You're effectively looking at doing an UPSERT (UPdate if the record exists, INSERT if not). Eric B. has a good example of how to do this in SQLite syntax utilizing the "INSERT OR REPLACE" functionality in SQLite. Basically, you'd do something like:
INSERT OR REPLACE INTO Repetition (Word, Topic, Counts)
VALUES ( 'behnam', 'mine',
coalesce((select Counts + 1 from Repetition
where Word = 'behnam', AND Topic = 'mine)
,1)
)
Another approach is to INSERT ... SELECT ... WHERE ... EXISTS [or not] (SELECT ...);
I do this sort of thing all the time, and I use jklemmack's suggestion as well. And I do it for other purposes too, such as doing JOINs in UPDATEs (which SQLite3 does not support).
For example:
CREATE TABLE t(id INTEGER PRIMARY KEY, c1 TEXT NOT NULL UNIQUE, c2 TEXT);
CREATE TABLE r(c1 TEXT NOT NULL UNIQUE, c2 TEXT);
INSERT OR REPLACE INTO t (id, c1, c2)
SELECT t.id, coalesce(r.c1, t.c1), coalesce(r.c2, t.c2)
FROM r LEFT OUTER JOIN t ON r.c1 = t.c1
WHERE r.c2 = #param;
The WHERE there has the condition that you'd have in your IF. The JOIN in the SELECT provides the JOIN that SQLite3 doesn't support in UPDATE. The INSERT OR REPLACE and the use of t.id (which can be NULL if the row doesn't exist in t) together provide the THEN and ELSE bodies.
You can apply this over and over. If you'd have three statements (that cannot somehow be merged into one) in the THEN part of the IF you'd need to have three statements with the IF condition in their WHEREs.
This is called an UPSERT (i.e. UPdate or inSERT). It has its forms in almost every type of database. Look at this question for the SQLite version: SQLite - UPSERT *not* INSERT or REPLACE
One way that I've found is based on SQL WHERE clause true/false statement:
SELECT * FROM SOME_TABLE
WHERE
(
SELECT SINGLE_COLUMN_NAME FROM SOME_OTHER_TABLE
WHERE
SOME_COLUMN = 'some value' and
SOME_OTHER_COLUMN = 'some other value'
)
This actually means execute some QUERIES if some other QUERY returns 'any' result.