Hey guys I've been struggling to get a simple query done in SnowFlake.
A sample of the data is explained below,
Number
ID
2559
23
2559
33
2559
55
2550
89
2499
14
2499
14
Based on the table above I would like to create a column that checks if (column Number) have duplicated values if yes; then check (column ID) if the values associated with that Number were duplicated or distinct. If values were duplicated then return 0 if values were distinct then return 1.
The desired outcome is as follows,
Number
ID
Check
2559
23
1
2559
33
1
2559
55
1
2550
89
0
2499
14
0
2499
14
0
and the last outcome should be on the Number column level (group by) and would be like the following table,
Number
Check
2559
1
2550
0
2499
0
Any suggestions are appreciated.
Edited tables
The scenario has changed based on new requirements
The original table have two IDs (ID1 and ID2) and those IDs must be linked to two different tables T-ID1 and T-ID2.
Number
ID1
ID2
2559
23
null
2559
33
null
2559
55
null
2550
89
null
2499
14
null
2499
14
null
2498
null
14
2498
null
50
2498
null
null
So the idea is if Number has duplicated IDs (in ID1 or ID2) then return 1, if Number has unique ID (in ID1 or ID2) then return ID1 or ID2. if Number have null values for ID1 and ID2 then ignore the entire row.
This is the query I am using - everything works based on solution provided below, but I am getting nulls along with ID values.
ID1 is linked to Supplier_ID_Ekuep and ID2 is linked to Supplier_ID_Qava.
select wbd.WAYBILL_ID, sup.WAYBILL_NO,
sup.Supplier_ID_Qava, supq.Supplier_Qava, supq.Supllier_City_Qava, supq.Supplier_Qava_Add,
sup.Supplier_ID_Ekuep, supc.Supplier_Ekuep, supc.Supllier_City_Ekuep, supc.Supplier_Ekuep_Add
from WAYBILL_PRODUCTS wbd
left join (select wb.WAYBILL_ID, ow.WAYBILL_NO,
case when mult.multi_supp_qava = 1 then '1'
when mult.multi_supp_qava = 0 then op.ID_Supplier_Qava
else null end as Supplier_ID_Qava,
case when mult.multi_supp_ekuep = 1 then '1'
when mult.multi_supp_ekuep = 0 then op.ID_Supplier_Ekuep
else null end as Supplier_ID_Ekuep
from WAYBILL_PRODUCTS wb
left join (select ID, ORDER_ID, WAYBILL_NO from ORDER_WAYBILLS) ow on wb.WAYBILL_ID = ow.id
left join (select opr.ID_ORDER, opr.ID_PRODUCT, ss.ID_Supplier_Ekuep, ss.ID_Supplier_Qava
from ORDER_PRODUCTS opr
left join (select s.id, supc.*, supq.*
from SUPPLIERS s
left join (select ID_SUPPLIER as ID_Supplier_Ekuep
from EKUEP_KSA_EKUEP_STAGING.PS_SUPPLIER where USED_SUPPLIER_ID is null) supc on supc.ID_Supplier_Ekuep = s.ID_SUPPLIER and s.STORE_ID = 1
left join (select ID_SUPPLIER as ID_Supplier_Qava
from QAVASHOP_KSA_QAVAISH_MAINSHOP.PS_SUPPLIER where USED_SUPPLIER_ID is null) supq on supq.ID_Supplier_Qava = s.ID_SUPPLIER and s.STORE_ID = 2) ss on opr.SUPPLIER_ID = ss.ID
group by opr.ID_ORDER, opr.ID_PRODUCT, ss.ID_Supplier_Ekuep, ss.ID_Supplier_Qava) op on op.ID_PRODUCT = wb.PRODUCT_ID and op.ID_ORDER = ow.ORDER_ID
------------------------------------------------------------------------------------------------------------------------
left join (select wb.WAYBILL_ID, ow.WAYBILL_NO,
(count(distinct op.ID_Supplier_Qava) > 1)::int as multi_supp_qava,
(count(distinct op.ID_Supplier_Ekuep) > 1)::int as multi_supp_ekuep
from WAYBILL_PRODUCTS wb
left join (select ID, ORDER_ID, WAYBILL_NO from ORDER_WAYBILLS) ow on wb.WAYBILL_ID = ow.id
left join (select opr.ID_ORDER, opr.ID_PRODUCT, ss.ID_Supplier_Ekuep, ss.ID_Supplier_Qava
from ORDER_PRODUCTS opr
left join (select s.id, supc.ID_Supplier_Ekuep, supq.ID_Supplier_Qava
from SUPPLIERS s
left join (select ID_SUPPLIER as ID_Supplier_Ekuep
from EKUEP_KSA_EKUEP_STAGING.PS_SUPPLIER where USED_SUPPLIER_ID is null) supc on supc.ID_Supplier_Ekuep = s.ID_SUPPLIER and s.STORE_ID = 1
left join (select ID_SUPPLIER as ID_Supplier_Qava
from QAVASHOP_KSA_QAVAISH_MAINSHOP.PS_SUPPLIER where USED_SUPPLIER_ID is null) supq on supq.ID_Supplier_Qava = s.ID_SUPPLIER and s.STORE_ID = 2
) ss on opr.SUPPLIER_ID = ss.ID
group by opr.ID_ORDER, opr.ID_PRODUCT, ss.ID_Supplier_Ekuep, ss.ID_Supplier_Qava) op on op.ID_PRODUCT = wb.PRODUCT_ID and op.ID_ORDER = ow.ORDER_ID
where _FIVETRAN_DELETED = 0
group by wb.WAYBILL_ID, ow.WAYBILL_NO) mult on mult.WAYBILL_NO = ow.WAYBILL_NO
------------------------------------------------------------------------------------------------------------------------
group by wb.WAYBILL_ID, ow.WAYBILL_NO, mult.multi_supp_qava, mult.multi_supp_ekuep, op.ID_Supplier_Qava, op.ID_Supplier_Ekuep) sup on wbd.WAYBILL_ID = sup.WAYBILL_ID
left join
(select ps.ID_SUPPLIER as ID_Supplier_Qava,
ps.name as Supplier_Qava,
listagg(distinct pa.CITY, ', ') as Supllier_City_Qava,
ps.DATE_ADD as Supplier_Qava_Add
from QAVASHOP_KSA_QAVAISH_MAINSHOP.PS_SUPPLIER ps
left join (select * from QAVASHOP_KSA_QAVAISH_MAINSHOP.PS_ADDRESS) pa on pa.ID_SUPPLIER = ps.ID_SUPPLIER and pa._FIVETRAN_DELETED = 0
where ps.USED_SUPPLIER_ID is null and ps._FIVETRAN_DELETED = 0
group by ID_Supplier_Qava, Supplier_Qava, Supplier_Qava_Add) supq on supq.ID_Supplier_Qava = sup.Supplier_ID_Qava
left join
(select ps.ID_SUPPLIER as ID_Supplier_Ekuep,
ps.name as Supplier_Ekuep,
listagg(distinct pa.CITY, ', ') as Supllier_City_Ekuep,
ps.DATE_ADD as Supplier_Ekuep_Add
from EKUEP_KSA_EKUEP_STAGING.PS_SUPPLIER ps
left join (select * from EKUEP_KSA_EKUEP_STAGING.PS_ADDRESS) pa on pa.ID_SUPPLIER = ps.ID_SUPPLIER and pa._FIVETRAN_DELETED = 0
where ps.USED_SUPPLIER_ID is null and ps._FIVETRAN_DELETED = 0
group by ID_Supplier_Ekuep, Supplier_Ekuep, Supplier_Ekuep_Add) supc on supc.ID_Supplier_Ekuep = sup.Supplier_ID_Ekuep
where _FIVETRAN_DELETED = 0
group by wbd.WAYBILL_ID, sup.WAYBILL_NO,
sup.Supplier_ID_Qava, supq.Supplier_Qava, supq.Supllier_City_Qava, supq.Supplier_Qava_Add,
sup.Supplier_ID_Ekuep, supc.Supplier_Ekuep, supc.Supllier_City_Ekuep, supc.Supplier_Ekuep_Add
The output I am getting for Waybill_No = 43265104 is as follows, I just want to get rid of the null row.
WITH data(num, id) AS (
SELECT * FROM VALUES
(2559, 23),
(2559, 33),
(2559, 55),
(2550, 89),
(2499, 14),
(2499, 14)
)
SELECT
num,
id,
count(*) over(partition by num) as num_dup,
count(distinct id)over(partition by num) as id_dup,
iff(num_dup > 1 AND num_dup = id_dup, 1, 0) as "check"
FROM data;
gives:
NUM
ID
NUM_DUP
ID_DUP
check
2559
23
3
3
1
2559
33
3
3
1
2559
55
3
3
1
2550
89
1
1
0
2499
14
2
1
0
2499
14
2
1
0
Extra Data:
WITH data(num, id) AS (
SELECT * FROM VALUES
(2559, 23),
(2559, 33),
(2559, 33),
(2559, 55),
(2550, 89),
(2499, 14),
(2499, 14)
)
SELECT
num,
id,
count(*) over(partition by num) as num_dup,
count(distinct id)over(partition by num) as id_dup,
iff(num_dup > 1 AND num_dup = id_dup, 1, 0) as "check"
FROM data;
if we make 2559 have two 33's then it show change to 0/false
NUM
ID
NUM_DUP
ID_DUP
check
2559
23
4
3
0
2559
33
4
3
0
2559
33
4
3
0
2559
55
4
3
0
2550
89
1
1
0
2499
14
2
1
0
2499
14
2
1
0
Last Step:
WITH data(num, id) AS (
SELECT * FROM VALUES
(2559, 23),
(2559, 33),
(2559, 55),
(2550, 89),
(2499, 14),
(2499, 14)
)
SELECT
num,
iff(count(*) > 1 AND count(*) = count(distinct id), 1, 0) as output
FROM data
GROUP BY 1;
gives:
NUM
OUTPUT
2559
1
2550
0
2499
0
left join (
select
ID,
ORDER_ID,
WAYBILL_NO
from ORDER_WAYBILLS
) ow
on wb.WAYBILL_ID = ow.id
should just be:
LEFT JOIN order_waybills AS ow
ON wb.waybill_id = ow.id
Gaint Refactoing Starts Here
So I start refactoring this SQL, the first part was to ignore my prior "this could be simpler" and move that into a CTE ob_cte, so it was "just simple" then I notice the first two sections had large block you noted as op that where the same, so that becomes a CTE op_cte also then that gives us:
WITH ob_cte AS (
SELECT
id,
order_id,
waybill_no
FROM order_waybills
), op_etc AS (
SELECT
opr.id_order,
opr.id_product,
ss.id_supplier_ekuep,
ss.id_supplier_qava
FROM order_products AS opr
LEFT JOIN (
SELECT
s.id,
supc.id_supplier_ekuep,
supq.id_supplier_qava
FROM suppliers AS s
LEFT JOIN (
SELECT
id_supplier AS id_supplier_ekuep
FROM ekuep_ksa_ekuep_staging.ps_supplier
WHERE used_supplier_id IS NULL
) AS supc
ON supc.id_supplier_ekuep = s.id_supplier
AND s.store_id = 1
LEFT JOIN (
SELECT
id_supplier AS id_supplier_qava
FROM qavashop_ksa_qavaish_mainshop.ps_supplier
WHERE used_supplier_id IS NULL
) AS supq
ON supq.id_supplier_qava = s.id_supplier
AND s.store_id = 2
) AS ss
ON opr.SUPPLIER_ID = ss.ID
GROUP BY opr.id_order, opr.id_product, ss.id_supplier_ekuep, ss.id_supplier_qava
)
SELECT
wbd.waybill_id,
sup.waybill_no,
sup.supplier_id_qava,
supq.supplier_qava,
supq.supllier_city_qava,
supq.supplier_qava_add,
sup.supplier_id_ekuep,
supc.supplier_ekuep,
supc.supllier_city_ekuep,
supc.supplier_ekuep_add
FROM waybill_products AS wbd
LEFT JOIN (
SELECT
wb.waybill_id,
ow.waybill_no,
CASE
WHEN mult.multi_supp_qava = 1 THEN '1'
WHEN mult.multi_supp_qava = 0 THEN op.ID_Supplier_Qava
ELSE NULL
end as supplier_id_qava,
case
WHEN mult.multi_supp_ekuep = 1 THEN '1'
WHEN mult.multi_supp_ekuep = 0 THEN op.ID_Supplier_Ekuep
ELSE NULL
END AS supplier_id_ekuep
FROM waybill_products AS wb
LEFT JOIN ob_cte AS ow
ON wb.waybill_id = ow.id
LEFT JOIN op_etc AS op
ON op.id_product = wb.product_id
AND op.id_order = ow.order_id
------------------------------------------------------------------------------------------------------------------------
left join (
select
wb.WAYBILL_ID,
ow.WAYBILL_NO,
(count(distinct op.ID_Supplier_Qava) > 1)::int as multi_supp_qava,
(count(distinct op.ID_Supplier_Ekuep) > 1)::int as multi_supp_ekuep
from WAYBILL_PRODUCTS wb
LEFT JOIN ob_cte AS ow
ON wb.waybill_id = ow.id
left join op_etc AS op
on op.ID_PRODUCT = wb.PRODUCT_ID
and op.ID_ORDER = ow.ORDER_ID
where _FIVETRAN_DELETED = 0
group by wb.WAYBILL_ID, ow.WAYBILL_NO
) AS mult
on mult.WAYBILL_NO = ow.WAYBILL_NO
------------------------------------------------------------------------------------------------------------------------
group by wb.WAYBILL_ID, ow.WAYBILL_NO, mult.multi_supp_qava, mult.multi_supp_ekuep, op.ID_Supplier_Qava, op.ID_Supplier_Ekuep
) AS sup
on wbd.WAYBILL_ID = sup.WAYBILL_ID
left join (
select
ps.id_supplier AS id_supplier_qava,
ps.name AS supplier_qava,
ps.date_add AS supplier_qava_add,
LISTAGG(DISTINCT pa.CITY, ', ') AS supllier_city_qava
from QAVASHOP_KSA_QAVAISH_MAINSHOP.PS_SUPPLIER ps
left join (
select *
from QAVASHOP_KSA_QAVAISH_MAINSHOP.PS_ADDRESS
) pa
ON pa.ID_SUPPLIER = ps.ID_SUPPLIER
AND pa._fivetran_deleted = 0
WHERE ps.USED_SUPPLIER_ID is null
AND ps._fivetran_deleted = 0
GROUP BY 1, 2, 3
) AS supq
on supq.ID_Supplier_Qava = sup.Supplier_ID_Qava
left join (
select
ps.id_supplier AS id_supplier_ekuep,
ps.name AS supplier_ekuep,
ps.date_add AS supplier_ekuep_add,
LISTAGG(DISTINCT pa.CITY, ', ') AS supllier_city_ekuep
FROM ekuep_ksa_ekuep_staging.ps_supplier AS ps
LEFT JOIN FROM ekuep_ksa_ekuep_staging.ps_address AS pa
ON pa.id_supplier = ps.id_supplier
and pa._fivetran_deleted = 0
WHERE ps.used_supplier_id IS NULL
and ps._fivetran_deleted = 0
GROUP BY 1, 2, 4
) AS supc
ON supc.id_supplier_ekuep = sup.supplier_id_ekuep
WHERE _fivetran_deleted = 0
GROUP BY wbd.waybill_id, sup.waybill_no,
sup.supplier_id_qava, supq.supplier_qava,
supq.supllier_city_qava, supq.supplier_qava_add,
sup.supplier_id_ekuep, supc.supplier_ekuep,
supc.supllier_city_ekuep, supc.supplier_ekuep_add;
Now op_etc itself, that GROUP BY at the end is really just a DISTINCT so we can swap that in. Also in ss of that op_cte
SELECT
s.id,
supc.id_supplier_ekuep,
supq.id_supplier_qava
FROM suppliers AS s
LEFT JOIN (
SELECT
id_supplier AS id_supplier_ekuep
FROM ekuep_ksa_ekuep_staging.ps_supplier
WHERE used_supplier_id IS NULL
) AS supc
ON supc.id_supplier_ekuep = s.id_supplier
AND s.store_id = 1
LEFT JOIN (
SELECT
id_supplier AS id_supplier_qava
FROM qavashop_ksa_qavaish_mainshop.ps_supplier
WHERE used_supplier_id IS NULL
) AS supq
ON supq.id_supplier_qava = s.id_supplier
AND s.store_id = 2
those to left joins, keep only values that are NULL and then LEFT JOIN of two requirements which means we are get a row per suppliers the IS NULL can be moved to the ON with no impact, which means the rename can be moved higher up, and the sub-select dropped, so that section can become:
SELECT
s.id,
supc.id_supplier AS id_supplier_ekuep,
supq.id_supplier AS id_supplier_qava
FROM suppliers AS s
LEFT JOIN ekuep_ksa_ekuep_staging.ps_supplier AS supc
ON supc.id_supplier = s.id_supplier
AND s.store_id = 1
AND supc.used_supplier_id IS NULL
LEFT JOIN qavashop_ksa_qavaish_mainshop.ps_supplier AS supq
ON supq.id_supplier = s.id_supplier
AND s.store_id = 2
AND supq.used_supplier_id IS NULL
thus the op_cte becomes:
), op_etc AS (
SELECT DISTINCT
opr.id_order,
opr.id_product,
ss.id_supplier_ekuep,
ss.id_supplier_qava
FROM order_products AS opr
LEFT JOIN (
SELECT
s.id,
supc.id_supplier AS id_supplier_ekuep,
supq.id_supplier AS id_supplier_qava
FROM suppliers AS s
LEFT JOIN ekuep_ksa_ekuep_staging.ps_supplier AS supc
ON supc.id_supplier = s.id_supplier
AND s.store_id = 1
AND supc.used_supplier_id IS NULL
LEFT JOIN qavashop_ksa_qavaish_mainshop.ps_supplier AS supq
ON supq.id_supplier = s.id_supplier
AND s.store_id = 2
AND supq.used_supplier_id IS NULL
) AS ss
ON opr.SUPPLIER_ID = ss.ID
)
which actually means the ss can just be left joined as first order things like:
), op_etc AS (
SELECT DISTINCT
opr.id_order,
opr.id_product,
supc.id_supplier AS id_supplier_ekuep,
supq.id_supplier AS id_supplier_qava
FROM order_products AS opr
LEFT JOIN suppliers AS s
opr.SUPPLIER_ID = s.ID
LEFT JOIN ekuep_ksa_ekuep_staging.ps_supplier AS supc
ON supc.id_supplier = s.id_supplier
AND s.store_id = 1
AND supc.used_supplier_id IS NULL
LEFT JOIN qavashop_ksa_qavaish_mainshop.ps_supplier AS supq
ON supq.id_supplier = s.id_supplier
AND s.store_id = 2
AND supq.used_supplier_id IS NULL
)
ah thats much more refreshing..
in the sub-select sup` I am worried, that you are grouping by values that are not in you output set, so you will get duplicate rows
in this case block
CASE
WHEN mult.multi_supp_qava = 1 THEN '1'
WHEN mult.multi_supp_qava = 0 THEN op.ID_Supplier_Qava
ELSE NULL
end as supplier_id_qava,
you are group not by the output supplier_id_qava or it's position 3 but mult.multi_supp_qava, op.ID_Supplier_Qava
so if op.ID_Supplier_Qava is not null when mult.multi_supp_qava = 1 and can have different values, the supplier_id_qava will output 1 on both rows, but they will not merge, as op.ID_Supplier_Qava is different. This is the root of my strong dislike for using the input to the SELECT section as the grouping clauses of the aggregation of those values. because through transformation, they can become the same on the output, but remain different on the input, and produce hard to debug deuplicates. And if the flip case that they are alway safe/null/constant in the odd case, this is not clear to the SQL reader. (on this point I have to say a strong thank you for have used the aliases on select etc, as that alway confidance of where those values are coming from).
So again the GROUP BY on sup has no agration of values so this is a hidden DISTINCT, but it not giving distinct values perhaps in all cases.
So after moving some sub-selects into CTE's as I find it makes each select layer show, just what it is doing, with have to leap over smaller details we get SQL like this:
WITH ob_cte AS (
SELECT
id,
order_id,
waybill_no
FROM order_waybills
), op_etc AS (
SELECT DISTINCT
opr.id_order,
opr.id_product,
supc.id_supplier AS id_supplier_ekuep,
supq.id_supplier AS id_supplier_qava
FROM order_products AS opr
LEFT JOIN suppliers AS s
opr.SUPPLIER_ID = s.ID
LEFT JOIN ekuep_ksa_ekuep_staging.ps_supplier AS supc
ON supc.id_supplier = s.id_supplier
AND s.store_id = 1
AND supc.used_supplier_id IS NULL
LEFT JOIN qavashop_ksa_qavaish_mainshop.ps_supplier AS supq
ON supq.id_supplier = s.id_supplier
AND s.store_id = 2
AND supq.used_supplier_id IS NULL
), mult_cte AS (
SELECT
wb.waybill_id,
ow.waybill_no,
(COUNT(DISTINCT op.id_supplier_qava) > 1)::int AS multi_supp_qava,
(COUNT(DISTINCT op.id_supplier_ekuep) > 1)::int AS multi_supp_ekuep
FROM waybill_products AS wb
LEFT JOIN ob_cte AS ow
ON wb.waybill_id = ow.id
LEFT JOIN op_etc AS op
ON op.id_product = wb.product_id
AND op.id_order = ow.order_id
WHERE _fivetran_deleted = 0
GROUP BY wb.waybill_id, ow.waybill_no
), supq_cte AS (
SELECT
ps.id_supplier AS id_supplier_qava,
ps.name AS supplier_qava,
ps.date_add AS supplier_qava_add,
LISTAGG(DISTINCT pa.city, ', ') AS supllier_city_qava
FROM qavashop_ksa_qavaish_mainshop.ps_supplier AS ps
LEFT JOIN qavashop_ksa_qavaish_mainshop.ps_address AS pa
ON pa.id_supplier = ps.id_supplier
AND pa._fivetran_deleted = 0
WHERE ps.used_supplier_id is null
AND ps._fivetran_deleted = 0
GROUP BY 1, 2, 3
), supc_cte AS (
SELECT
ps.id_supplier AS id_supplier_ekuep,
ps.name AS supplier_ekuep,
ps.date_add AS supplier_ekuep_add,
LISTAGG(DISTINCT pa.city, ', ') AS supllier_city_ekuep
FROM ekuep_ksa_ekuep_staging.ps_supplier AS ps
LEFT JOIN FROM ekuep_ksa_ekuep_staging.ps_address AS pa
ON pa.id_supplier = ps.id_supplier
AND pa._fivetran_deleted = 0
WHERE ps.used_supplier_id IS NULL
AND ps._fivetran_deleted = 0
GROUP BY 1, 2, 4
), sup_cte AS (
SELECT DISTINCT
wb.waybill_id,
ow.waybill_no,
CASE
WHEN mult.multi_supp_qava = 1 THEN '1'
WHEN mult.multi_supp_qava = 0 THEN op.ID_Supplier_Qava
ELSE NULL
end as supplier_id_qava,
case
WHEN mult.multi_supp_ekuep = 1 THEN '1'
WHEN mult.multi_supp_ekuep = 0 THEN op.ID_Supplier_Ekuep
ELSE NULL
END AS supplier_id_ekuep
FROM waybill_products AS wb
LEFT JOIN ob_cte AS ow
ON wb.waybill_id = ow.id
LEFT JOIN op_etc AS op
ON op.id_product = wb.product_id
AND op.id_order = ow.order_id
------------------------------------------------------------------------------------------------------------------------
LEFT JOIN mult_cte AS mult
on mult.WAYBILL_NO = ow.WAYBILL_NO
------------------------------------------------------------------------------------------------------------------------
)
SELECT
wbd.waybill_id,
sup.waybill_no,
sup.supplier_id_qava,
supq.supplier_qava,
supq.supllier_city_qava,
supq.supplier_qava_add,
sup.supplier_id_ekuep,
supc.supplier_ekuep,
supc.supllier_city_ekuep,
supc.supplier_ekuep_add
FROM waybill_products AS wbd
LEFT JOIN sup_cte AS sup
ON wbd.WAYBILL_ID = sup.WAYBILL_ID
LEFT JOIN supq_cte AS supq
ON supq.id_supplier_qava = sup.supplier_id_qava
LEFT JOIN supc_cte AS supc
ON supc.id_supplier_ekuep = sup.supplier_id_ekuep
WHERE _fivetran_deleted = 0
GROUP BY wbd.waybill_id, sup.waybill_no,
sup.supplier_id_qava, supq.supplier_qava,
supq.supllier_city_qava, supq.supplier_qava_add,
sup.supplier_id_ekuep, supc.supplier_ekuep,
supc.supllier_city_ekuep, supc.supplier_ekuep_add;
And at the end we see again you are suffering from duplicate data, that you are trying to clean away with yet another GROUP BY.. so we will switch to a DISTINCT
SELECT DISTINCT
wbd.waybill_id,
sup.waybill_no,
sup.supplier_id_qava,
supq.supplier_qava,
supq.supllier_city_qava,
supq.supplier_qava_add,
sup.supplier_id_ekuep,
supc.supplier_ekuep,
supc.supllier_city_ekuep,
supc.supplier_ekuep_add
FROM waybill_products AS wbd
LEFT JOIN sup_cte AS sup
ON wbd.WAYBILL_ID = sup.WAYBILL_ID
LEFT JOIN supq_cte AS supq
ON supq.id_supplier_qava = sup.supplier_id_qava
LEFT JOIN supc_cte AS supc
ON supc.id_supplier_ekuep = sup.supplier_id_ekuep
WHERE _fivetran_deleted = 0
Right so what was your question again?
you want to get rid of all the nulls... if this is so "why so many left joins" or returning null from CASE statements..
Right so in the picture to show a waybill_no with values, and another where supplier_id_qava is null, given the later comes from
LEFT JOIN sup_cte AS sup
ON wbd.WAYBILL_ID = sup.WAYBILL_ID
and everything else connect via sup you should change that from a LEFT JOIN to just a JOIN
I have some sample records below that I need to use the CASE WHEN statement to remove and identify the duplicate records in SQL.
Quantity Values Desc event ID
1 5 Blue 12550 577
1 5 bluee 12550 525
2 10 blu 12550 535
i would like to use a case statement to show the duplicate indicators such as:
Dup_Quantity Dup_Value Dup_Desc Quantity Values Desc event ID
Y Y N 1 5 Blue 12550 577
Y Y N 1 5 Bluee 12550 525
however, after using this script, the result still shows as:
Dup_Quantity Dup_Value Dup_Desc Quantity Values Desc event ID
Y Y N 1 5 Blue 12550 577
Y Y N 1 5 Bluee 12550 525
Y N N 2 10 Blu 12550 535
SELECT DISTINCT
CASE WHEN a.Quantity = b.Quantity THEN 'Y' ELSE 'N' END AS "Dup_Quantity",
CASE WHEN a.Values = b.Values THEN 'Y' ELSE 'N' END AS "Dup_Value",
CASE WHEN a.Desc = b.Desc THEN 'Y' ELSE 'N' END AS "Dup_Desc"
FROM Table1 a
INNER JOIN Table1 b ON a.event = b.event
WHERE (a.Quantity = b.Quantity OR a.Values = b.Values OR a.Desc = b.Desc)
AND a.ID <> b.ID
Basically, record with ID 535 stills shows up in the result. Would someone Please give me some pointers?
SQL Fiddle
MS SQL Server 2012 Schema Setup:
CREATE TABLE Table1
([Quantity] int, [Values] int, [Desc] varchar(5), [event] int, [ID] int)
;
INSERT INTO Table1
([Quantity], [Values], [Desc], [event], [ID])
VALUES
(1, 5, 'Blue', 12550, 577),
(1, 5, 'bluee', 12550, 525),
(2, 10, 'blu', 12550, 535)
;
Query 1:
SELECT
CASE WHEN (SELECT COUNT(*)
FROM Table1 t2
WHERE t1.Quantity = t2.Quantity AND
t1.ID <> t2.ID AND t1.event = t2.event) > 0
THEN 'Y' ELSE 'N' END AS Dup_Quantity,
CASE WHEN (SELECT COUNT(*)
FROM Table1 t2
WHERE t1."Values" = t2."Values" AND
t1.ID <> t2.ID AND t1.event = t2.event) > 0
THEN 'Y' ELSE 'N' END AS Dup_Value,
CASE WHEN (SELECT COUNT(*)
FROM Table1 t2
WHERE t1."Desc" = t2."Desc" AND
t1.ID <> t2.ID AND t1.event = t2.event) > 0
THEN 'Y' ELSE 'N' END AS Dup_Desc,
*
FROM Table1 t1
WHERE
(SELECT COUNT(*)
FROM Table1 t2
WHERE t1.Quantity = t2.Quantity AND
t1.ID <> t2.ID AND t1.event = t2.event) > 0
OR
(SELECT COUNT(*)
FROM Table1 t2
WHERE t1."Values" = t2."Values" AND
t1.ID <> t2.ID AND t1.event = t2.event) > 0
OR
(SELECT COUNT(*)
FROM Table1 t2
WHERE t1."Desc" = t2."Desc" AND
t1.ID <> t2.ID AND t1.event = t2.event) > 0
Results:
| DUP_QUANTITY | DUP_VALUE | DUP_DESC | QUANTITY | VALUES | DESC | EVENT | ID |
|--------------|-----------|----------|----------|--------|-------|-------|-----|
| Y | Y | N | 1 | 5 | Blue | 12550 | 577 |
| Y | Y | N | 1 | 5 | bluee | 12550 | 525 |
Your query returns:
Dup_Quantity Dup_Value Dup_Desc
Y Y N
However I don't get what you want to do here, the correct version of it is:
SELECT
CASE WHEN a."Quantity" = b."Quantity" THEN 'Y' ELSE 'N' END AS "Dup_Quantity",
CASE WHEN a."Values" = b."Values" THEN 'Y' ELSE 'N' END AS "Dup_Value",
CASE WHEN a."Desc" = b."Desc" THEN 'Y' ELSE 'N' END AS "Dup_Desc",
a.*
FROM Table1 a
INNER JOIN Table1 b ON b.event = a.event
WHERE (a."Quantity" = b."Quantity" OR a."Values" = b."Values" OR a."Desc" = b."Desc")
AND a.ID <> b.ID
If you want to get duplicated rows in terms of Quantity, Values and Desc:
SELECT
a.*
FROM Table1 a
INNER JOIN Table1 b ON b.event = a.event
WHERE (a."Quantity" = b."Quantity" AND a."Values" = b."Values" AND a."Desc" = b."Desc")
AND a.ID <> b.ID
Table I
ID LinkId
1 0
2 1
3 2
Table 2
ID IntersectionID X Y
1 5 100 200
2 6 300 400
3 7 800 500
Table 3
ID IntersectionID Sequence_number linkId LinkDirection
1 5 0 0 Positive
1 5 1 1 negative
2 6 0 0 negative
I want a single query giving me the following
ID LinkId X(start) Y(Start) X(End) Y(End)
1 0 100 200 300 400
For every row in Table 1, get its IntersectionID from table 3(using linked field),(link direction- positive means start point and negative means end point). Then go to table 2 and fill x,y values.
Help me achieve it.
Tried this query. Unable to get it.
select r1.id id, r1.linkid base_link_id, r2.X startX,r2.Y startY from table1 r1, table2 r2
where startX = (select X from table2 where id = r1.id and intersectionId =
(select intersectionId from table3 where id = r1.id and linkId= r1.linkid and LinkDirection = positive)) and startY
= (select Y from table2 where id = r1.id and intersectionId =
(select intersectionId from table3 where id = r1.id and linkId= r1.linkid and LinkDirection = negative));
With this query you should be able to do it.
Pay attention on LinkDirection if it's text.
SELECT
t1.ID
,t1.LinkId
,t2_start.X as X_Start
,t2_start.Y as Y_Start
,t2_end.X as X_End
,t2_end.Y as Y_End
FROM
table1 t1
LEFT JOIN table3 t3_start
on t1.linkId = t3_start.linkId
and t3_start.LinkDirection = 'Positive'
LEFT JOIN table2 t2_start
on t2_start.IntersectionID = t3_start.IntersectionID
LEFT JOIN table3 t3_end
on t1.linkId = t3_end.linkId
and t3_end.LinkDirection = 'negative'
LEFT JOIN table2 t2_end
on t2_end.IntersectionID = t3_end.IntersectionID
You will get null field in t2_start.X t2_start.Y t2_end.X t2_end.Y if there is no row in table2. You can change it to 0 or -1 with
CASE WHEN t2_start.X is not null THEN t2_start.X ELSE -1 END
Hope the below query is usefull, if you need clarification. Please ask
SELECT ID,LinkID,MAX(StartX) AS StartX ,MAX(Starty) AS StartY,
MAX(EndX) AS EndX,MAX(EndY) AS EndY FROM
(
SELECT a.ID,a.LinkID,X as StartX,Y as StartY,null as EndX,null as EndY
FROM
table1 a JOIN table3 b
ON a.Linkid=b.Linkid
JOIN table2 c
ON b.id=c.id WHERE LinkDirection='Positive'
UNION ALL
SELECT a.ID,a.LinkID,null as StartX,null as StartY,X as EndX,Y as EndYY
FROM
table1 a JOIN table3 b
ON a.Linkid=b.Linkid
JOIN table2 c
ON b.id=c.id WHERE LinkDirection='Negative'
) t
GROUP BY ID,LinkID
This will return
ID LinkID StartX StartY EndX EndY
1 0 100 200 300 400
2 1 NULL NULL 100 200
My result of is like this
date acc cr date acc dr
---------------------------------------------------
null null 0 13/3/12 A 1300
null null 0 13/3/12 c 1200
null null 0 13/3/12 D 1100
13/3/12 A 1000 null null 0
18/3/12 E 2000 null null 0
19/3/12 F 3000 null null 0
31/3/12 G 3000 null null 0
this result i got from following query bu joining 2 tables to get cash book
select(case when mli.voucher_type = 1 THEN tav.voucher_date end)pdate,
(case when mli.voucher_type = 1 THEN mli.description end) acc,
(case when tav.voucher_type_id = 1 then sum(tvl.amount) else 0 end) cr,
(case when mli.voucher_type = 2 THEN tav.voucher_dateend) rdate,
(case when mli.voucher_type = 2 THEN mli.description end) acc,
(case when tav.voucher_type_id = 2 then sum(tvl.amount) else 0 end) dr
from t_acc_voucher tav
join t_voucher_ledger tvl on tvl.voucher_id = tav.voucher_id
join m_ledger_index mli on mli.ledger_index_id = tvl.ledger_index_id
group by mli.description, mli.voucher_type, tav.voucher_type_id,tav.voucher_date
I want result like this
date acc cr date acc dr
---------------------------------------------------
13/3/12 A 1000 13/3/12 A 1300
18/3/12 E 2000 13/3/12 c 1200
19/3/12 F 3000 13/3/12 D 1100
31/3/12 G 3000 null null 0
can any body help me.or give some suggestion is it write way to get it or i can try with 2 diffrent query.
thanks in advance
Break your query into two separate queries, one for credit and another for debit and do a full outer join based on descending date, you can order by any column in fact. From the original query, I assumed that VOUCHER_TYPE = 1 is credits and VOUCHER_TYPE = 2 is debit.
Try this (not tested)
with CREDITS as (select TAV.VOUCHER_DATE PDATE,
MLI.DESCRIPTION ACC,
(case when TAV.VOUCHER_TYPE_ID = 1 then SUM(TVL.AMOUNT) else 0 end) CR
from t_acc_voucher tav
join t_voucher_ledger tvl
on tvl.voucher_id = tav.voucher_id
join M_LEDGER_INDEX MLI
on MLI.LEDGER_INDEX_ID = TVL.LEDGER_INDEX_ID
where MLI.VOUCHER_TYPE = 1
group by MLI.DESCRIPTION,
MLI.VOUCHER_TYPE,
TAV.VOUCHER_TYPE_ID,
TAV.VOUCHER_DATE),
debits as (select TAV.VOUCHER_DATE RDATE,
MLI.DESCRIPTION ACC,
(case when TAV.VOUCHER_TYPE_ID = 2 then SUM(TVL.AMOUNT) else 0 end) DR
from T_ACC_VOUCHER TAV
where mli.voucher_type = 2
join t_voucher_ledger tvl
on tvl.voucher_id = tav.voucher_id
join M_LEDGER_INDEX MLI
on MLI.LEDGER_INDEX_ID = TVL.LEDGER_INDEX_ID
group by MLI.DESCRIPTION,
MLI.VOUCHER_TYPE,
TAV.VOUCHER_TYPE_ID,
TAV.VOUCHER_DATE )
select T1.PDATE, T1.ACC, T1.CR, T2.RDATE, T2.ACC, T2.DR
from (select a.*, row_number() over (order by a.PDATE) RN
from credits a) T1
full outer join
select b.*, row_number() over (order by b.RDATE) RN
from debits b) T2
on (t1.rn = t2.rn);
select t1.date1, t1.acc1, t1.cr, t2.date2, t2.acc2, t2.dr
from (the table or query) t1
join (the table or query) t2 on t1.acc1=t2.acc2
where t1.acc1 is not null
we join the same query (or table) twice joinint to itself by acc.