Multiple row insert or select if exists - sql

CREATE TABLE object (
object_id serial,
object_attribute_1 integer,
object_attribute_2 VARCHAR(255)
)
-- primary key object_id
-- btree index on object_attribute_1, object_attribute_2
Here is what I currently have:
SELECT * FROM object
WHERE (object_attribute_1=100 AND object_attribute_2='Some String') OR
(object_attribute_1=200 AND object_attribute_2='Some other String') OR
(..another row..) OR
(..another row..)
When the query returns, I check for what is missing (thus, does not exist in the database).
Then I will make an multiple row insert:
INSERT INTO object (object_attribute_1, object_attribute_2)
VALUES (info, info), (info, info),(info, info)
Then I will select what I just inserted
SELECT ... WHERE (condition) OR (condition) OR ...
And at last, I will merge the two selects on the client side.
Is there a way that I can combine these 3 queries, into one single queries, where I will provide all the data, and INSERT if the records do not already exist and then do a SELECT in the end.

Your suspicion was well founded. Do it all in a single statement using a data-modifying CTE (Postgres 9.1+):
WITH list(object_attribute_1, object_attribute_2) AS (
VALUES
(100, 'Some String')
, (200, 'Some other String')
, .....
)
, ins AS (
INSERT INTO object (object_attribute_1, object_attribute_2)
SELECT l.*
FROM list l
LEFT JOIN object o1 USING (object_attribute_1, object_attribute_2)
WHERE o1.object_attribute_1 IS NULL
RETURNING *
)
SELECT * FROM ins -- newly inserted rows
UNION ALL -- append pre-existing rows
SELECT o.*
FROM list l
JOIN object o USING (object_attribute_1, object_attribute_2);
Note, there is a tiny time frame for a race condition. So this might break if many clients try it at the same time. If you are working under heavy concurrent load, consider this related answer, in particular the part on locking or serializable transaction isolation:
Postgresql batch insert or ignore

Related

How to return ids of rows with conflicting values?

I am looking to insert or update values in an SQLite database (version > 3.35) avoiding multiple queries. upsert along with returning seems promising :
CREATE TABLE phonebook2(
name TEXT PRIMARY KEY,
phonenumber TEXT,
validDate DATE
);
INSERT INTO phonebook2(name,phonenumber,validDate)
VALUES('Alice','704-555-1212','2018-05-08')
ON CONFLICT(name) DO UPDATE SET
phonenumber=excluded.phonenumber,
validDate=excluded.validDate
WHERE excluded.validDate>phonebook2.validDate RETURNING name;
This helps me track names corresponding to inserted/modified rows. How to find rows where phonebook2 values conflict with values upserted in above statement, but no insert or update happened due to where clause?
The RETURNING clause can't be used to get non-affected rows.
What you can do is execute a SELECT statement before the UPSERT:
WITH cte(name, phonenumber, validDate) AS (VALUES
('Alice', '704-555-1212', '2018-05-08'),
('Bob','804-555-1212', '2018-05-09')
)
SELECT *
FROM phonebook2 p
WHERE EXISTS (
SELECT *
FROM cte c
WHERE c.name = p.name AND c.validDate <= p.validDate
);
In the CTE you may include as many tuples as you want

WITH returned table behave unexpectedly when used with RETURNING in Postgresql?

I'm stumped. Why does this work fine...
WITH sst AS (SELECT * FROM subtopics_results WHERE student_id = 2)
SELECT * FROM subtopics_results AS str
WHERE (subtopic_id,student_id) IN (SELECT subtopic_id,student_id FROM sst);
But the below doesn't - it returns an empty table. I can query the returned table sst just fine (e.g. SELECT * FROM sst), but when I try to use
WITH sst AS (
-- inside block is working fine
INSERT INTO subtopics_results (paper_id, student_id, subtopic_id, marks_scored)
(
WITH subtopics AS (
SELECT qbr.question_id, q.subtopic_id, qbr.student_id, q.paper_id, qbr.marks_scored
FROM question AS q
JOIN question_breakdown_result AS qbr
ON qbr.question_id = q.question_id
WHERE paper_id = 1 AND qbr.student_id = 2
)
SELECT paper_id, student_id, subtopic_id, SUM(marks_scored)
FROM subtopics GROUP BY subtopic_id, student_id, paper_id
HAVING SUM(marks_scored) > 2
)
RETURNING subtopic_id, student_id
)
-- this returns an empty table, even though identical to the above.
SELECT * FROM subtopics_results AS str
WHERE (subtopic_id,student_id) IN (SELECT subtopic_id,student_id FROM sst);
I'm trying to select all the pairs in the table subtopics_results (subtopic_id and student_id) after an insertion, but keep getting nothing back, even though I do expect to have results.
I have tried inner joins, no avail.
Is this to do with the RETURNING statement?
That behavior is documented in "7.8.2. Data-Modifying Statements in WITH":
(...)
The sub-statements in WITH are executed concurrently with each
other and with the main query. Therefore, when using data-modifying
statements in WITH, the order in which the specified updates
actually happen is unpredictable. All the statements are executed with
the same snapshot (see Chapter
13), so they
cannot "see" one another's effects on the target tables. (...)
(...)
Maybe using UNION ALL to merge the rows returned by RETURNING and the ones already in the table before the INSERT happens can do what you want here.
WITH
sst AS
(
INSERT INTO subtopics_results
(...)
RETURNING *
)
SELECT sst.*
FROM sst
UNION ALL
SELECT str.*
FROM subtopics_results AS str
WHERE (str.subtopic_id,
str.student_id) IN (SELECT sst.subtopic_id,
sst.student_id
FROM sst);
Or, if you just want the inserted rows back (that's not entirely clear to me) simply use:
INSERT INTO subtopics_results
(...)
RETURNING *;

Use inserted value as a parameter for other inserts

There is a db2 database with two tables. The first one, table1, has autoincrement column ID. It is the foreign key for the table2.
A am writing an HTML generator for SQL queries. So with some input parameters it generates a query or multiple queries. It is not connected to the database.
What I need is to get that autoincrement field and use it in next queries.
So basically, the scenario is:
insert into table1;
select autogenerated field ID;
insert into table2 using that ID;
insert into table2 using that ID;
...some more similar inserts...
insert into table2 using that ID;
And all that SQL query should be generated and then used as a single SQL script.
I was thinking about something like this:
SELECT ID FROM FINAL TABLE (INSERT INTO Table1 (t1column1, t1column2, etc.)
VALUES (t1value1, t1value2, etc.))
But I don't know, how I can write the result into a variable so I could use it in next queries like this:
INSERT INTO Table2 (foreignKeyCol, t2column1, t2column2, etc.)
VALUES ($ID, t2value1, t2value2, etc.)
I could just paste that select instead of $ID, but the second query can be used several times with the same $ID and different values.
EDIT: DB2 10.5 on Linux.
You can chain several inserts together using CTEs, like so:
WITH idcte (id) as (
SELECT ID FROM FINAL TABLE (
INSERT INTO Table1 (t1column1, t1column2, etc.)
VALUES (t1value1, t1value2, etc.)
)
),
ins1 (id) as (
SELECT foreignKeyCol FROM FINAL TABLE (
INSERT INTO Table2 (foreignKeyCol, t2column1, t2column2, etc.)
SELECT id, t2value1, t2value2, etc.
FROM idcte
)
),
-- more CTEs
SELECT foreignKeyCol FROM FINAL TABLE (
-- your last INSERT ... SELECT FROM
)
Essentially you will have to wrap each INSERT into a SELECT FROM FINAL TABLE for this to work.
Alternatively, you can use a global variable to keep the ID value:
CREATE VARIABLE myNewId INT;
SET myNewId = (SELECT ID FROM FINAL TABLE (
INSERT INTO Table1 (t1column1, t1column2, etc.)
VALUES (t1value1, t1value2, etc.)
));
INSERT INTO Table2 (foreignKeyCol, t2column1, t2column2, etc.)
VALUES (myNewId, t2value1, t2value2, etc.);
DROP VARIABLE myNewId;
This assumes a recent version of Db2 for LUW.

in Postgres, use array of values for INSERT

have an array of values. I wish to INSERT each of them as records if it does not already exist. can this be done in one statement in SQL?
traditional javascript approach:
[4,5,8]forEach( x => queryParams(
insert into t1( c1 ) values( $1 )
where not exists( select 1 from t1 where c1 = $1`,
[x]
);
what would be ideal is something like
queryParams(
`some fancy SQL here`,
[ `{${join[4,5,8]}}` ]
);
many reasons for this, including limiting the cost of server round trips and transactions.
You can use a correlated sub-query to find the values that don't exist matching a condition:
INSERT INTO Records (X)
SELECT X
FROM unnest(ARRAY[4,5,8]) T (X)
WHERE NOT EXISTS (SELECT * FROM Records WHERE X = T.X);
SQL Fiddle: http://sqlfiddle.com/#!15/e0334/29/0
Edited above to use unnest

IF-Statement in SQLite: update or insert?

I Can't run this query with SQLite
if 0<(select COUNT(*) from Repetition where (Word='behnam' and Topic='mine'))
begin
update Repetition set Counts=1+ (select Counts from Repetition where (Word='behnam' and Topic='mine'))
end
else
begin
insert Repetition(Word,Topic,Counts)values('behnam','mine',1)
end
It says "Syntax error near IF"
How can I solve the problem
SQLite does not have an IF statement (see the list of supported queries)
Insetad, check out out ERIC B's suggestion on another thread. You're effectively looking at doing an UPSERT (UPdate if the record exists, INSERT if not). Eric B. has a good example of how to do this in SQLite syntax utilizing the "INSERT OR REPLACE" functionality in SQLite. Basically, you'd do something like:
INSERT OR REPLACE INTO Repetition (Word, Topic, Counts)
VALUES ( 'behnam', 'mine',
coalesce((select Counts + 1 from Repetition
where Word = 'behnam', AND Topic = 'mine)
,1)
)
Another approach is to INSERT ... SELECT ... WHERE ... EXISTS [or not] (SELECT ...);
I do this sort of thing all the time, and I use jklemmack's suggestion as well. And I do it for other purposes too, such as doing JOINs in UPDATEs (which SQLite3 does not support).
For example:
CREATE TABLE t(id INTEGER PRIMARY KEY, c1 TEXT NOT NULL UNIQUE, c2 TEXT);
CREATE TABLE r(c1 TEXT NOT NULL UNIQUE, c2 TEXT);
INSERT OR REPLACE INTO t (id, c1, c2)
SELECT t.id, coalesce(r.c1, t.c1), coalesce(r.c2, t.c2)
FROM r LEFT OUTER JOIN t ON r.c1 = t.c1
WHERE r.c2 = #param;
The WHERE there has the condition that you'd have in your IF. The JOIN in the SELECT provides the JOIN that SQLite3 doesn't support in UPDATE. The INSERT OR REPLACE and the use of t.id (which can be NULL if the row doesn't exist in t) together provide the THEN and ELSE bodies.
You can apply this over and over. If you'd have three statements (that cannot somehow be merged into one) in the THEN part of the IF you'd need to have three statements with the IF condition in their WHEREs.
This is called an UPSERT (i.e. UPdate or inSERT). It has its forms in almost every type of database. Look at this question for the SQLite version: SQLite - UPSERT *not* INSERT or REPLACE
One way that I've found is based on SQL WHERE clause true/false statement:
SELECT * FROM SOME_TABLE
WHERE
(
SELECT SINGLE_COLUMN_NAME FROM SOME_OTHER_TABLE
WHERE
SOME_COLUMN = 'some value' and
SOME_OTHER_COLUMN = 'some other value'
)
This actually means execute some QUERIES if some other QUERY returns 'any' result.