How do I do a deep copy with a single PostgreSQL query? - sql

I have three tables:
CREATE TABLE offers
(
id serial NOT NULL PRIMARY KEY,
title character varying(1000) NOT NULL DEFAULT ''::character varying
);
CREATE TABLE items
(
id serial NOT NULL PRIMARY KEY,
offer_id integer NOT NULL,
title character varying(1000) NOT NULL DEFAULT ''::character varying,
CONSTRAINT items_offer_id_fkey FOREIGN KEY (offer_id)
REFERENCES offers (id)
);
CREATE TABLE sizes
(
id serial NOT NULL PRIMARY KEY,
item_id integer NOT NULL,
title character varying(1000) NOT NULL DEFAULT ''::character varying,
CONSTRAINT sizes_item_id_fkey FOREIGN KEY (item_id)
REFERENCES items (id)
);
I have 1 offer that has 2 items. Each item has 2 sizes:
INSERT INTO offers (title) VALUES ('My Offer');
INSERT INTO items (offer_id, title) VALUES (1, 'First Item');
INSERT INTO items (offer_id, title) VALUES (1, 'Second Item');
INSERT INTO sizes (item_id, title) VALUES (1, 'First Size of Item #1');
INSERT INTO sizes (item_id, title) VALUES (1, 'Second Size of Item #1');
INSERT INTO sizes (item_id, title) VALUES (2, 'First Size of Item #2');
INSERT INTO sizes (item_id, title) VALUES (2, 'Second Size of Item #2');
Is there a way to clone an offer with all its items and sizes with a single query?
I tried to solve it with CTE, here is my SQL:
WITH tmp_offers AS (
INSERT INTO offers (title)
SELECT title FROM offers WHERE id = 1
RETURNING id
), tmp_items AS (
INSERT INTO items (offer_id, title)
(SELECT (SELECT id FROM tmp_offers), title FROM items WHERE offer_id = 1)
RETURNING id
)
INSERT INTO sizes (item_id, title)
(SELECT (SELECT id FROM tmp_items), title FROM sizes WHERE id IN (
SELECT sizes.id FROM sizes
JOIN items ON items.id = sizes.item_id
WHERE items.offer_id = 1
));
But this SQL results to an error, that I can't resolve:
ERROR: more than one row returned by a subquery used as an expression
Your help is greatly appreciated.
P.S. I use PostgreSQL 9.5

This should work:
WITH tmp_offers AS (
INSERT INTO offers (title)
SELECT title
FROM offers
WHERE id = 1
RETURNING id
), tmp_items AS (
INSERT INTO items (offer_id, title)
SELECT o.id, i.title
FROM items i
cross join tmp_offers o
WHERE i.offer_id = 1
order by i.id
RETURNING items.id
), numbered_new as (
select ti.id,
row_number() over (order by ti.id) as rn
from tmp_items ti
), numbered_old as (
select i.id,
row_number() over (order by i.id) as rn
from items i
WHERE i.offer_id = 1
), item_mapper as (
select n.id as new_item_id,
o.id as old_item_id
from numbered_new n
join numbered_old o on n.rn = o.rn
)
INSERT INTO sizes (item_id, title)
select im.new_item_id, s.title
from sizes s
join item_mapper im on im.old_item_id = s.item_id;
Online example: http://rextester.com/RYQUS11008

You are quite close. It is the final query that needs work:
WITH tmp_offers AS (
INSERT INTO offers (title)
SELECT title FROM offers WHERE id = 1
RETURNING id
),
tmp_items AS (
INSERT INTO items (offer_id, title)
SELECT o.id, i.title
FROM items i CROSS JOIN
(SELECT id FROM tmp_offers) o
WHERE i.offer_id = 1
RETURNING id, title
)
INSERT INTO sizes (item_id, title)
SELECT i.id, i.title
FROM tmp_items i;
The major difference here is that tmp_items now has two columns -- and they appear to be the columns that you want for this purpose.

Related

Insert multiple rows using the same foreign key that needs to be selected

Assume that there are two tables:
CREATE TABLE products (id SERIAL, name TEXT);
CREATE TABLE comments (id SERIAL, product_id INT, txt TEXT);
I would like to insert multiple comments for the same product. But I don't know the product_id yet, only the product name.
So I could do:
INSERT INTO comments (txt, product_id) VALUES
( 'cool', (SELECT id from products WHERE name='My product name') ),
( 'great', (SELECT id from products WHERE name='My product name') ),
...
( 'many comments later', (SELECT id from products WHERE name='My product name') );
I'd like to reduce the repetition. How to do this?
I tried this but it inserts no rows:
INSERT INTO
comments (txt, product_id)
SELECT
x.txt,
p.id
FROM
(
VALUES
('Great product'),
('I love it'),
...
('another comment')
) x (txt)
JOIN products p ON p.name = 'My product name';
Your query works just fine. The only way it inserts zero rows is if there is no product in the table products for a given string - in your query named My product name. However, #a_horse_with_no_name's suggestion to use a CROSS JOIN might simplify your query a bit. You can combine it with a CTE to collect all comments and then CROSS JOIN it with the record you filtered in from table products.
CREATE TABLE products (id SERIAL, name TEXT);
CREATE TABLE comments (id SERIAL, product_id INT, txt TEXT);
INSERT INTO products VALUES (1, 'My product name'),(2,'Another product name');
WITH j (txt) AS (
VALUES ('Great product'),('I love it'),('another comment')
)
INSERT INTO comments (product_id,txt)
SELECT id,j.txt FROM products
CROSS JOIN j WHERE name = 'My product name';
SELECT * FROM comments;
id | product_id | txt
----+------------+-----------------
1 | 1 | Great product
2 | 1 | I love it
3 | 1 | another comment
Check this db<>fiddle

How to insert a column which sets unique id based on values in another column (SQL)?

I will create table where I will insert multiple values for different companies. Basically I have all values that are in the table below but I want to add a column IndicatorID which is linked to IndicatorName so that every indicator has a unique id. This will obviously not be a PrimaryKey.
I will insert the data with multiple selects:
CREATE TABLE abc
INSERT INTO abc
SELECT company_id, 'roe', roevalue, metricdate
FROM TABLE1
INSERT INTO abc
SELECT company_id, 'd/e', devalue, metricdate
FROM TABLE1
So, I don't know how to add the IndicatorID I mentioned above.
EDIT:
Here is how I populate my new table:
INSERT INTO table(IndicatorID, Indicator, Company, Value, Date)
SELECT [the ID that I need], 'NI_3y' as 'Indicator', t.Company, avg(t.ni) over (partition by t.Company order by t.reportdate rows between 2 preceding and current row) as 'ni_3y',
t.reportdate
FROM table t
LEFT JOIN IndicatorIDs i
ON i.Indicator = roe3 -- the part that is not working if I have separate indicatorID table
I am going to insert different indicators for the same companies. And I want indicatorID.
Your "indicator" is a proper entity in its own right. Create a table with all indicators:
create table indicators (
indicator_id int identity(1, 1) primary key,
indicator varchar(255)
);
Then, use the id only in this table. You can look up the value in the reference table.
Your inserts are then a little more complicated:
INSERT INTO indicators (indicator)
SELECT DISTINCT roevalue
FROM table1 t1
WHERE NOT EXISTS (SELECT 1 FROM indicators i2 WHERE i2.indicator = t1.roevalue);
Then:
INSERT INTO ABC (indicatorId, companyid, value, date)
SELECT i.indicatorId, t1.company, v.value, t1.metricdate
FROM table1 t1 CROSS APPLY
(VALUES ('roe', t1.roevalue), ('d/e', t1.devalue)
) v(indicator, value) JOIN
indicators i
ON i.indicator = v.indicator;
This process is called normalization and it is the typical way to store data in a database.
DDL and INSERT statement to create an indicators table with a unique constraint on indicator. Because the ind_id is intended to be a foreign key in the abc table it's created as a non-decomposable surrogate integer primary key using the IDENTITY property.
drop table if exists test_indicators;
go
create table test_indicators (
ind_id int identity(1, 1) primary key not null,
indicator varchar(20) unique not null);
go
insert into test_indicators(indicator) values
('NI'),
('ROE'),
('D/E');
The abc table depends on the ind_id column from indicators table as a foreign key reference. To populate the abc table company_id's are associated with ind_id's.
drop table if exists test_abc
go
create table test_abc(
a_id int identity(1, 1) primary key not null,
ind_id int not null references test_indicators(ind_id),
company_id int not null,
val varchar(20) null);
go
insert into test_abc(ind_id, company_id)
select ind_id, 102 from test_indicators where indicator='NI'
union all
select ind_id, 103 from test_indicators where indicator='ROE'
union all
select ind_id, 104 from test_indicators where indicator='D/E'
union all
select ind_id, 103 from test_indicators where indicator='NI'
union all
select ind_id, 105 from test_indicators where indicator='ROE'
union all
select ind_id, 102 from test_indicators where indicator='NI';
Query to get result
select i.ind_id, a.company_id, i.indicator, a.val
from test_abc a
join test_indicators i on a.ind_id=i.ind_id;
Output
ind_id company_id indicator val
1 102 NI NULL
2 103 ROE NULL
3 104 D/E NULL
1 103 NI NULL
2 105 ROE NULL
1 102 NI NULL
I was finally able to find the solution for my problem which seems to me very simple, although it took time and asking different people about it.
First I create my indicators table where I assign primary key for all indicators I have:
CREATE TABLE indicators (
indicator_id int identity(1, 1) primary key,
indicator varchar(255)
);
Then I populate easy without using any JOINs or CROSS APPLY. I don't know if this is optimal but it seems as the simplest choice:
INSERT INTO table(IndicatorID, Indicator, Company, Value, Date)
SELECT
(SELECT indicator_id from indicators i where i.indicator = 'NI_3y) as IndicatorID,
'NI_3y' as 'Indicator',
Company,
avg(ni) over (partition by Company order by reportdate rows between 2 preceding and current row) as ni_3y,
reportdate
FROM TABLE1

Adding a LEFT JOIN on a INSERT INTO....RETURNING

My query Inserts a value and returns the new row inserted
INSERT INTO
event_comments(date_posted, e_id, created_by, parent_id, body, num_likes, thread_id)
VALUES(1575770277, 1, '9e028aaa-d265-4e27-9528-30858ed8c13d', 9, 'December 7th', 0, 'zRfs2I')
RETURNING comment_id, date_posted, e_id, created_by, parent_id, body, num_likes, thread_id
I want to join the created_by with the user_id from my user's table.
SELECT * from users WHERE user_id = created_by
Is it possible to join that new returning row with another table row?
Consider using a WITH structure to pass the data from the insert to a query that can then be joined.
Example:
-- Setup some initial tables
create table colors (
id SERIAL primary key,
color VARCHAR UNIQUE
);
create table animals (
id SERIAL primary key,
a_id INTEGER references colors(id),
animal VARCHAR UNIQUE
);
-- provide some initial data in colors
insert into colors (color) values ('red'), ('green'), ('blue');
-- Store returned data in inserted_animal for use in next query
with inserted_animal as (
-- Insert a new record into animals
insert into animals (a_id, animal) values (3, 'fish') returning *
) select * from inserted_animal
left join colors on inserted_animal.a_id = colors.id;
-- Output
-- id | a_id | animal | id | color
-- 1 | 3 | fish | 3 | blue
Explanation:
A WITH query allows a record returned from an initial query, including data returned from a RETURNING clause, which is stored in a temporary table that can be accessed in the expression that follows it to continue work on it, including using a JOIN expression.
You were right, I misunderstood
This should do it:
DECLARE mycreated_by event_comments.created_by%TYPE;
INSERT INTO
event_comments(date_posted, e_id, created_by, parent_id, body, num_likes, thread_id)
VALUES(1575770277, 1, '9e028aaa-d265-4e27-9528-30858ed8c13d', 9, 'December 7th', 0, 'zRfs2I')
RETURNING created_by into mycreated_by
SELECT * from users WHERE user_id = mycreated_by

Postgres: upsert a row and update a primary key column

Suppose I have two tables in my Postgres database:
create table transactions
(
id bigint primary key,
doc_id bigint not null,
-- lots of other columns...
amount numeric not null
);
-- same columns
create temporary table updated_transactions
(
id bigint primary key,
doc_id bigint not null,
-- lots of other columns...
amount numeric not null
);
Both tables have just a primary key, and no unique indexes.
I need to upsert rows from updated_transactions into transactions using the following rules:
id column values in transactions and updated_transactions don't match
other columns like doc_id, etc (except of the amount) should match
when a matching row is found, update both amount and id columns
when a matching row is not found, insert it
id values in updated_transactions are taken from a sequence.
A business object just populates updated_transactions and then merges the
new or updated rows from it into transactions using an upsert query.
So my old unchanged transactions keep their ids intact, and the updated ones
are assigned new ids.
In MSSQL and Oracle, it would be a merge statement similar to this:
merge into transactions t
using updated_transactions ut on t.doc_id = ut.doc_id, ...
when matched then
update set t.id = ut.id, t.amount = ut.amount
when not matched then
insert (t.id, t.doc_id, ..., t.amount)
values (ut.id, ut.doc_id, ..., ut.amount);
In PostgreSQL, I suppose it should be something like this:
insert into transactions(id, doc_id, ..., amount)
select coalesce(t.id, ut.id), ut.doc_id, ... ut.amount
from updated_transactions ut
left join transactions t on t.doc_id = ut.doc_id, ....
on conflict
on constraint transactions_pkey
do update
set amount = excluded.amount, id = excluded.id
The problem is with the do update clause: excluded.id is an old value
from transactions table, while I need a new value from updated_transactions.
ut.id value is inaccessible for the do update clause, and the only thing I can
use is the excluded row. But the excluded row has only coalesce(t.id, ut.id)
expression which returns old id values for the existing rows.
Is it possible to update both id and amount columns using the upsert query?
Create unique index on those columns you use as key and pass its name in your upsert expression, so that it uses it instead of pkey.
Then it will insert row if no matches were found, using ID from updated_transactions. If it finds match, then you can use excluded.id to get ID from updated_transactions.
I think that left join transactions is redundant.
So it would look kinda like this:
insert into transactions(id, doc_id, ..., amount)
select ut.id, ut.doc_id, ... ut.amount
from updated_transactions ut
on conflict
on constraint transactions_multi_column_unique_index
do update
set amount = excluded.amount, id = excluded.id
Looks like the task can be accomplished using writable CTEs instead of the plain upsert.
First, I'll post the easier version of the query that answers the original question as it was asked. This solution assumes that doc_id, unit_id columns address a candidate key, but doesn't require a unique index on these columns.
Test data:
create temp table transactions
(
id bigint primary key,
doc_id bigint,
unit_id bigint,
amount numeric
);
create temp table updated_transactions
(
id bigint primary key,
doc_id bigint,
unit_id bigint,
amount numeric
);
insert into transactions(id, doc_id, unit_id, amount)
values (1, 1, 1, 10), (2, 1, 2, 15), (3, 1, 3, 10);
insert into updated_transactions(id, doc_id, unit_id, amount)
values (6, 1, 1, 11), (7, 1, 2, 15), (8, 1, 4, 20);
The query to merge updated_transactions into transactions:
with new_values as
(
select ut.id new_id, t.id old_id, ut.doc_id, ut.unit_id, ut.amount
from updated_transactions ut
left join transactions t
on t.doc_id = ut.doc_id and t.unit_id = ut.unit_id
),
updated as
(
update transactions tr
set id = nv.new_id, amount = nv.amount
from new_values nv
where id = nv.old_id
returning tr.*
)
insert into transactions(id, doc_id, unit_id, amount)
select ut.new_id, ut.doc_id, ut.unit_id, ut.amount
from new_values ut
where ut.new_id not in (select id from updated);
The results:
select * from transactions
-- id | doc_id | unit_id | amount
------+--------+---------+-------
-- 3 | 1 | 3 | 10 -- not changed
-- 6 | 1 | 1 | 11 -- updated
-- 7 | 1 | 2 | 15 -- updated
-- 8 | 1 | 4 | 20 -- inserted
In my real application doc_id, unit_id aren't always unique, so they don't represent a candidate key. To match the rows I take into account the row number, calculated for the rows sorted by their ids. So here's my second solution.
Test data:
-- the tables are the same as above
insert into transactions(id, doc_id, unit_id, amount)
values (1, 1, 1, 10), (2, 1, 1, 15), (3, 1, 3, 10);
insert into updated_transactions(id, doc_id, unit_id, amount)
values (6, 1, 1, 11), (7, 1, 1, 15), (8, 1, 4, 20);
The merge query:
with trans as
(
select id, doc_id, unit_id, amount,
row_number() over(partition by doc_id, unit_id order by id) row_num
from transactions
),
updated_trans as
(
select id, doc_id, unit_id, amount,
row_number() over(partition by doc_id, unit_id order by id) row_num
from updated_transactions
),
new_values as
(
select ut.id new_id, t.id old_id, ut.doc_id, ut.unit_id, ut.amount
from updated_trans ut
left join trans t
on t.doc_id = ut.doc_id and t.unit_id = ut.unit_id and t.row_num = ut.row_num
),
updated as
(
update transactions tr
set id = nv.new_id, amount = nv.amount
from new_values nv
where id = nv.old_id
returning tr.*
)
insert into transactions(id, doc_id, unit_id, amount)
select ut.new_id, ut.doc_id, ut.unit_id, ut.amount
from new_values ut
where ut.new_id not in (select id from updated);
The results:
select * from transactions;
-- id | doc_id | unit_id | amount
------+--------+---------+-------
-- 3 | 1 | 3 | 10 -- not changed
-- 6 | 1 | 1 | 11 -- updated
-- 7 | 1 | 1 | 15 -- updated
-- 8 | 1 | 4 | 20 -- inserted
References:
Insert on duplicate update in PostgreSQL
Upserting via Writeable CTE
Waiting for 9.1 — Writable CTE
Why is UPSERT so complicated?

How to find the need of materials from nested estimates in Postgres

Product estimates contain sub-products.
Sub-products can contain also sub-products etc.
Finally tree leafs contians materials.
Maximum nesting level is 10.
Orders contain also products, sub-products and materials with ordered quantities.
How to find the need of materials required to fullfill the orders?
Products, sub-products and materials are in single table:
create table toode (productid char(10) primary key );
Estimate table:
create table dok (
dokumnr serial primary key,
productid char(10) not null references toode
);
Sub-products and materials in estimates:
create table rid (
id serial primary key,
dokumnr int not null references dok,
itemid char(10) not null references toode,
quantity numeric(12,4) -- quantity required to make one product
);
Orders:
create table orderrows (
id serial primary key,
itemid char(10) not null references toode,
quantity numeric(12,4) -- ordered quantity
);
Result should be query which return the need of materials and sub-products:
itemid char(10) not null references toode,
requiredquantity numeric(12,4) -- total quantity of items required to make ordered products
How to implement this in Postgresql 9.2?
Described fields should remain in those tables. It is possible to add additional
columns and tables if this helps.
Is it possible to make some universal query which works with unilimited nesting level.
Or is it best way to create query which repeats some parts 10 times for maximum nensting level ?
Update
estimates
product1
material1 2 pcs
subproduct2 3 pcs
subproduct2
material2 4 pcs
are described as
insert into dok values (1,'product1');
insert into rid (dokumnr, itemid, quantity) values (1, 'material1', 2);
insert into rid (dokumnr, itemid, quantity) values (1, 'subproduct2', 3);
insert into dok values (2,'subproduct2');
insert into rid (dokumnr, itemid, quantity) values (2, 'material2', 4);
If 10 pieces of product1 are ordered this is described as:
insert into orderrows (itemid, quantity ) values ('product1', 10);
Result should be:
material1 20
material2 120
material1 quantity is calculated as 10*2.
material2 quantity is calculated as 10*3*4
Update 2
Joachim answer gives incorrect result on multi level estimates when last level contains more that one row. Last join LEFT JOIN rid rid2 ON rid2.dokumnr = dok2.dokumnr returns multiple rows and result table is duplicated.
Testcase http://sqlfiddle.com/#!12/e5c11/1/0 :
create table toode (productid char(15) primary key );
create table dok (
dokumnr serial primary key,
productid char(15) not null references toode
);
create table rid (
id serial primary key,
dokumnr int not null references dok,
itemid char(15) not null references toode,
quantity numeric(12,4) -- quantity required to make one product
);
create table orderrows (
id serial primary key,
itemid char(15) not null references toode,
quantity numeric(12,4) -- ordered quantity
);
INSERT INTO toode VALUES ('product1'),('material1'),('subproduct2'), ('material2'), ('material3');
insert into dok values (1,'product1');
insert into dok values (2,'subproduct2');
insert into rid (dokumnr, itemid, quantity) values (1, 'material1', 1);
insert into rid (dokumnr, itemid, quantity) values (1, 'subproduct2', 1);
insert into rid (dokumnr, itemid, quantity) values (2, 'material2', 1);
insert into rid (dokumnr, itemid, quantity) values (2, 'material3', 1);
insert into orderrows (itemid, quantity ) values ('product1', 1);
Expected:
Every quantity is 1 so result quantity must be 1 for every material.
Observed:
Material2 and matererial3 rows are duplicated.
How to fix this ? Query should determine leaf nodes itself. Leaf nodes are not marked specially in data.
This should do it using a recursive query;
WITH RECURSIVE t(itemid,qty) AS (
SELECT itemid,quantity,false isleaf FROM orderrows
UNION ALL
SELECT rid.itemid,(rid.quantity*t.qty)::NUMERIC(12,4),
dok2.productid IS NULL
FROM t
JOIN dok ON dok.productid=t.itemid
JOIN rid ON rid.dokumnr=dok.dokumnr
LEFT JOIN dok dok2 ON dok2.productid=rid.itemid
)
SELECT itemid, SUM(qty) FROM t WHERE isleaf GROUP BY itemid
An SQLfiddle to test with.
Try this query:
;with recursive cte as (
select r.itemid, r.quantity * o.quantity as quantity, false as is_material
from orderrows as o
inner join dok as d on d.productid = o.itemid
inner join rid as r on r.dokumnr = d.dokumnr
union
select r.itemid, r.quantity * o.quantity as quantity, itemid like 'material%'
from cte as o
inner join dok as d on d.productid = o.itemid
inner join rid as r on r.dokumnr = d.dokumnr
)
select * from cte as c
where c.itemid not in (select t.productid from dok as t);
here's SQL FIDDLE example to test it. Here I'm assuming that your define materials as products which name starts with 'material', but I think that you should have an attribute is_material or something like that in your DB, so you could change this condition.
update - test case sql fiddle