How to group by two tables with CTE - sql

Question

I have three tables (Oracle):
sales_order
-------------
int so_key (pk)
int part_key (fk)
int condition_key (fk)
number unit_price
int qty_ordered
number unit_cost
date entry_date
quote
-------------
int q_key (pk)
int part_key (fk)
int condition_key (fk)
number unit_price
int qty_quoted
date entry_date
stock
-------------
int stock_key (pk)
int part_key (fk)
int condition (fk)
int qty_available
number unit_cost
And all three have foreign key references to these two tables:
part
-------------
int part_key (pk)
condition
-------------
int condition_key (pk)
I am writing a query that will aggregate the data into rows grouped by part and condition. However, I am unable to figure out how to group by BOTH part and condition. Here is the (functional) query that I have that groups by part only:
WITH
ctePart_Quotes AS
(
SELECT q.part_key
, COUNT(*) AS quotes_count
, SUM(q.unit_price * q.qty_quoted) AS quotes_amt_total
, SUM(q.qty_quoted) AS quotes_qty_total
FROM quote q
WHERE q.entry_date BETWEEN TO_DATE('01-Jan-2011', 'dd-mm-yyyy') AND TO_DATE('31-Dec-2011', 'dd-mm-yyyy')
GROUP BY q.part_key
)
, ctePart_Sales AS
(
SELECT so.part_key
, COUNT(*) AS sales_count
, SUM(so.unit_price * so.qty_ordered) AS sales_amt_total
, SUM(so.qty_ordered) AS sales_qty_total
, SUM(so.qty_ordered * so.unit_cost) AS cost_total
FROM sales_order so
WHERE so.entry_date BETWEEN TO_DATE('01-Jan-2011', 'dd-mm-yyyy') AND TO_DATE('31-Dec-2011', 'dd-mm-yyyy')
GROUP BY so.part_key
)
, ctePart_Stock AS
(
SELECT stm.part_key
, SUM(stm.qty_available) AS total_available
, SUM(stm.qty_available * stm.unit_cost) AS inv_cost
FROM stock stm
GROUP BY stm.part_key
)
SELECT p.part_key,
part_stock.total_available,
part_stock.inv_cost,
sales.sales_amt_total,
sales.sales_qty_total,
sales.sales_count,
sales.cost_total,
quotes.quotes_amt_total,
quotes.quotes_qty_total,
quotes.quotes_count
FROM parts p
LEFT OUTER JOIN ctePart_Quotes quotes
ON quotes.part_key = p.part_key
LEFT OUTER JOIN ctePart_Sales sales
ON sales.part_key = p.part_key
LEFT OUTER JOIN ctePart_Stock part_stock
ON part_stock.part_key = p.part_key
WHERE NOT(sales_amt_total IS NULL
AND sales_qty_total IS NULL
AND sales_count IS NULL
AND cost_total IS NULL
AND quotes_amt_total IS NULL
AND quotes_qty_total IS NULL
AND quotes_count IS NULL)
AND SALES_AMT_TOTAL > 10000
This query produces this output (totals grouped by part_key):
part_key | total_available | inv_cost | sales_amt_total | ...
---------|-----------------|----------|-----------------| ...
234 | 59 | 4923.90 | 29403.48 | ...
185 | 21 | 192.64 | 9034.95 | ...
102 | 102 | 8738.34 | 50382.20 | ...
...
But I'm trying to modify the query to produce this (totals grouped by part_key and condition_key):
part_key | condition_key | total_available | inv_cost | sales_amt_total | ...
---------|---------------|-----------------|----------|-----------------| ...
234 | 3 | 24 | 2360.50 | 16947.18 | ...
234 | 7 | 35 | 2563.40 | 12456.30 | ...
...
How do you do this?
EDIT: for clarification:
The complexity lies in: how do you join the condition in the final select? Because you are selecting FROM part but the condition relationship is through the other tables (sales_order, etc.). So you'd have to join through each of the tables (LEFT OUTER JOIN condition cond ON quotes.condition_key = cond.condition_key, etc.) but those joins would each be separate columns.
EDIT #2: someone provided a good image of the data model that illustrates the (proper, legitimate) relationship between part/condition but also the subtle complexity faced in this problem:

Answer 1

The main problem here seems to be your data model. Converting the table "descriptions" of your question into DDL code, and reversing this to a relational model (using Oracle Datamodeler), we find something like this:
DDL code
create table part ( part_key number primary key ) ;
create table condition ( condition_key number primary key ) ;
create table sales_order (
so_key number generated always as identity start with 3000 primary key
, part_key number references part
, condition_key number references condition
, unit_price number
, qty_ordered number
, unit_cost number
, entry_date date ) ;
create table quote (
q_key number generated always as identity start with 4000 primary key
, part_key number references part
, condition_key number references condition
, qty_quoted number
, unit_price number
, entry_date date );
create table stock (
stock_key number generated always as identity start with 5000 primary key
, part_key number references part
, condition_key number references condition
, qty_available number
, unit_cost number ) ;
Relational model (Oracle SQL Developer Data Modeler)
Looking at the model, it becomes clear that each PART can have several CONDITIONs. Thus, it may be necessary (for you) to decide, which condition you are referring to. That may not be easy. Suppose we have a part (with part_key) 1000. Now, we can record 3 different conditions, and use a specific condition for each of your 3 tables mentioned in your query.
-- one part, 3 conditions
begin
insert into part ( part_key ) values ( 1000 ) ;
insert into condition( condition_key ) values ( 2001 ) ;
insert into condition( condition_key ) values ( 2002 ) ;
insert into condition( condition_key ) values ( 2003 ) ;
insert into sales_order ( part_key, condition_key ) values ( 1000, 2001 ) ;
insert into quote ( part_key, condition_key ) values ( 1000, 2002 ) ;
insert into stock ( part_key, condition_key ) values ( 1000, 2003 ) ;
end ;
/
Which one of the 3 condition is supposed to be used for the query? Hard to tell.
-- not using WITH (subquery factoring) here - for clarity
select
P.part_key
, SO.condition_key
, Q.condition_key
, ST.condition_key
from part P
join sales_order SO on SO.part_key = P.part_key
join quote Q on Q.part_key = P.part_key
join stock ST on ST.part_key = P.part_key
;
-- output
PART_KEY CONDITION_KEY CONDITION_KEY CONDITION_KEY
1000 2001 2002 2003
Well - we could pick one of the conditions, couldn't we. However, even more conditions can exist for one and the same part ...
begin
insert into condition( condition_key ) values ( 2004 ) ;
insert into condition( condition_key ) values ( 2005 ) ;
insert into condition( condition_key ) values ( 2006 ) ;
insert into sales_order ( part_key, condition_key ) values ( 1000, 2004 ) ;
insert into quote ( part_key, condition_key ) values ( 1000, 2005 ) ;
insert into stock ( part_key, condition_key ) values ( 1000, 2006 ) ;
end ;
/
-- Same query as above now gives us:
PART_KEY CONDITION_KEY CONDITION_KEY CONDITION_KEY
1000 2001 2005 2006
1000 2001 2005 2003
1000 2001 2002 2006
1000 2001 2002 2003
1000 2004 2005 2006
1000 2004 2005 2003
1000 2004 2002 2006
1000 2004 2002 2003
Conclusion: Fix your data model. (We know this is sometimes easier said than done ...) Then, it will make sense to do some more work on your query.
__Update__
Now that we know that nothing can be done about the tables and the constraints, maybe the following queries will give you a starting point. We do not have proper test data, so let's just add some random values to the tables ...
-- PART and CONDITION -> 1000 integers each
begin
for i in 1 .. 1000
loop
insert into part ( part_key ) values ( i ) ;
insert into condition( condition_key ) values ( i ) ;
end loop;
end ;
/
Table QUOTE
-- 2 12s, 2 18s
SQL> select * from quote ;
Q_KEY PART_KEY CONDITION_KEY QTY_QUOTED UNIT_PRICE ENTRY_DATE
4000 10 100 55 500 01-MAY-11
4001 12 120 55 500 01-MAY-11
4002 12 37 56 501 01-MAY-11
4003 14 140 55 500 01-MAY-11
4004 15 46 56 501 01-MAY-11
4005 16 160 55 500 01-MAY-11
4006 18 180 55 500 01-MAY-11
4007 18 55 56 501 01-MAY-11
4008 20 200 55 500 01-MAY-11
Table SALES_ORDER
SQL> select * from sales_order ;
SO_KEY PART_KEY CONDITION_KEY UNIT_PRICE QTY_ORDERED UNIT_COST ENTRY_DATE
3000 10 100 500 55 400 05-MAY-11
3001 12 120 500 55 400 05-MAY-11
3002 14 140 500 55 400 05-MAY-11
3003 16 160 500 55 400 05-MAY-11
3004 18 180 500 55 400 05-MAY-11
3005 20 200 500 55 400 05-MAY-11
Table STOCK
SQL> select * from stock ;
STOCK_KEY PART_KEY CONDITION_KEY QTY_AVAILABLE UNIT_COST
5000 10 100 10 400
5001 12 120 10 400
5002 14 140 10 400
5003 14 100 12 402
5004 16 160 10 400
5005 18 180 10 400
5006 20 200 10 400
Assuming that only valid part/condition combinations are recorded, we can use FULL OUTER JOINs to get a first picture.
SQL> select
2 Q.part_key q_part , Q.condition_key q_cond
3 , SO.part_key so_part, SO.condition_key so_cond
4 , ST.part_key st_part, ST.condition_key st_cond
5 from quote Q
6 full join sales_order SO
7 on SO.part_key = Q.part_key and SO.condition_key = Q.condition_key
8 full join stock ST
9 on ST.part_key = SO.part_key and ST.condition_key = SO.condition_key
10 ;
-- result
Q_PART Q_COND SO_PART SO_COND ST_PART ST_COND
10 100 10 100 10 100
12 120 12 120 12 120
12 37 NULL NULL NULL NULL
14 140 14 140 14 140
15 46 NULL NULL NULL NULL
16 160 16 160 16 160
18 180 18 180 18 180
18 55 NULL NULL NULL NULL
20 200 20 200 20 200
NULL NULL NULL NULL 14 100
Then, we can use Analytic Functions for the various calculations. Note that we do not use GROUP BY here, the grouping is done via ... partition by Q.part_key, Q.condition_key ... (More about analytic functions: Oracle documentation, and examples here).
-- Skeleton query ...
-- Note that you will have need to write over(...) several times.
-- Add a WHERE clause and conditions as required.
select
Q.part_key as q_part, Q.condition_key as q_cond,
count( Q.part_key ) over ( partition by Q.part_key, Q.condition_key ) as q_count
--
-- Q example sums
-- , sum( Q.unit_price * Q.qty_quoted )
-- over ( partition by Q.part_key, Q.condition_key ) as qat -- quotes_amt_total
-- , sum( Q.qty_quoted )
-- over ( partition by Q.part_key, Q.condition_key ) as qqt -- quotes_qty_total
--
, SO.part_key as so_part, SO.condition_key as so_cond
, count( SO.part_key ) over ( partition by SO.part_key, SO.condition_key ) as so_count
--
-- SO sums here
--
, ST.part_key as st_part, ST.condition_key as st_cond
, count( ST.part_key ) over ( partition by ST.part_key, ST.condition_key ) as st_count
from sales_order SO
full join quote Q
on SO.part_key = Q.part_key and SO.condition_key = Q.condition_key
full join stock ST
on ST.part_key = SO.part_key and ST.condition_key = SO.condition_key
-- where ...
;
Result
-- output
Q_PART Q_COND Q_COUNT SO_PART SO_COND SO_COUNT ST_PART ST_COND ST_COUNT
10 100 1 10 100 1 10 100 1
12 37 1 NULL NULL 0 NULL NULL 0
12 120 1 12 120 1 12 120 1
14 140 1 14 140 1 14 140 1
15 46 1 NULL NULL 0 NULL NULL 0
16 160 1 16 160 1 16 160 1
18 55 1 NULL NULL 0 NULL NULL 0
18 180 1 18 180 1 18 180 1
20 200 1 20 200 1 20 200 1
NULL NULL 0 NULL NULL 0 14 100 1

Answer 2

The trick is to first create a Carthesian product (The condition table only has ~30 rows), and maybe suppress the unwanted result rows later:
This may look sub-optimal, but it will avoid a join onCOALESCE()d keyfields, which could perform badly.
WITH
ctePart_Quotes AS
(
SELECT q.part_key, q.condition_key
, COUNT(*) AS quotes_count
, SUM(q.unit_price * q.qty_quoted) AS quotes_amt_total
, SUM(q.qty_quoted) AS quotes_qty_total
FROM quote q
WHERE q.entry_date BETWEEN TO_DATE('01-Jan-2011', 'dd-mm-yyyy') AND TO_DATE('31-Dec-2011', 'dd-mm-yyyy')
GROUP BY q.part_key, q.condition_key
)
, ctePart_Sales AS
(
SELECT so.part_key, so.condition_key
, COUNT(*) AS sales_count
, SUM(so.unit_price * so.qty_ordered) AS sales_amt_total
, SUM(so.qty_ordered) AS sales_qty_total
, SUM(so.qty_ordered * so.unit_cost) AS cost_total
FROM sales_order so
WHERE so.entry_date BETWEEN TO_DATE('01-Jan-2011', 'dd-mm-yyyy') AND TO_DATE('31-Dec-2011', 'dd-mm-yyyy')
GROUP BY so.part_key, so.condition_key
)
, ctePart_Stock AS
(
SELECT stm.part_key, stm.condition_key
, SUM(stm.qty_available) AS total_available
, SUM(stm.qty_available * stm.unit_cost) AS inv_cost
FROM stock stm
GROUP BY stm.part_key, stm.condition_key
)
SELECT p.part_key,
c.condition_key,
part_stock.total_available,
part_stock.inv_cost,
sales.sales_amt_total,
sales.sales_qty_total,
sales.sales_count,
sales.cost_total,
quotes.quotes_amt_total,
quotes.quotes_qty_total,
quotes.quotes_count
FROM parts p
CROSS JOIN condition c -- <<-- Here
LEFT OUTER JOIN ctePart_Quotes quotes
ON quotes.part_key = p.part_key
AND quotes.condition_key = c.condition_key -- <<-- Here
LEFT OUTER JOIN ctePart_Sales sales
ON sales.part_key = p.part_key
AND sales.condition_key = c.condition_key -- <<-- Here
LEFT OUTER JOIN ctePart_Stock part_stock
ON part_stock.part_key = p.part_key
AND part_stock.condition_key = c.condition_key -- <<-- Here
WHERE NOT(sales_amt_total IS NULL
AND sales_qty_total IS NULL
AND sales_count IS NULL
AND cost_total IS NULL
AND quotes_amt_total IS NULL
AND quotes_qty_total IS NULL
AND quotes_count IS NULL) -- <<-- And maybe Here, too
AND SALES_AMT_TOTAL > 10000
;

Answer 3

Maybe this works for you:
WITH
ctePart_Quotes AS
(
SELECT q.part_key,
q.condition_key
, COUNT(*) AS quotes_count
, SUM(q.unit_price * q.qty_quoted) AS quotes_amt_total
, SUM(q.qty_quoted) AS quotes_qty_total
FROM quote q
WHERE q.entry_date BETWEEN TO_DATE('01-Jan-2011', 'dd-mm-yyyy') AND TO_DATE('31-Dec-2011', 'dd-mm-yyyy')
GROUP BY q.part_key,
q.condition_key
)
, ctePart_Sales AS
(
SELECT so.part_key,
so.condition_key
, COUNT(*) AS sales_count
, SUM(so.unit_price * so.qty_ordered) AS sales_amt_total
, SUM(so.qty_ordered) AS sales_qty_total
, SUM(so.qty_ordered * so.unit_cost) AS cost_total
FROM sales_order so
WHERE so.entry_date BETWEEN TO_DATE('01-Jan-2011', 'dd-mm-yyyy') AND TO_DATE('31-Dec-2011', 'dd-mm-yyyy')
GROUP BY so.part_key,
so.condition_key
)
, ctePart_Stock AS
(
SELECT stm.part_key,
stm.condition_key
, SUM(stm.qty_available) AS total_available
, SUM(stm.qty_available * stm.unit_cost) AS inv_cost
FROM stock stm
GROUP BY stm.part_key,
stm.condition_key
)
SELECT p.part_key,
cte.condition_key,
cte.total_available,
cte.inv_cost,
cte.sales_amt_total,
cte.sales_qty_total,
cte.sales_count,
cte.cost_total,
cte.quotes_amt_total,
cte.quotes_qty_total,
cte.quotes_count
FROM parts p
LEFT OUTER JOIN (SELECT coalesce(quotes.part_key, sales.part_key, part_stock.part_key) part_key,
coalesce(quotes.condition_key, sales.condition_key, part_stock.condition_key) condition_key,
quotes.quotes_count,
quotes.quotes_amt_total,
quotes.quotes_qty_total,
sales.sales_count,
sales.sales_amt_total,
sales.sales_qty_total,
sales.cost_total,
part_stock.total_available,
part_stock.inv_cost
FROM ctePart_Quotes quotes
FULL JOIN ctePart_Sales sales
ON sales.part_key = quotes.part_key
AND sales.condition_key = quotes.condition_key
FULL JOIN ctePart_Stock part_stock
ON part_stock.part_key = sales.part_key
AND part_stock.condition_key = sales.condition_key) cte
ON cte.part_key = p.part_key
WHERE NOT(sales_amt_total IS NULL
AND cte.sales_qty_total IS NULL
AND cte.sales_count IS NULL
AND cte.cost_total IS NULL
AND cte.quotes_amt_total IS NULL
AND cte.quotes_qty_total IS NULL
AND cte.quotes_count IS NULL)
AND SALES_AMT_TOTAL > 10000;
It also groups by condition_key in the CTEs. Then it FULL JOINs the CTEs together using coalesce to compensate for null values of part_key or condition_key in the first tables (But maybe there is none, if every combination of part_keyand condition_key, that is present in one of the tables is also present in the respective two other tables.). The result is then LEFT JOINed to part using part_key.