Distinct on two columns (separately) and MIN on another column - sql

I've tried to strip this problem down the the bare bones; I hope I've still captured the essence of what I'm trying to achieve in the original query!
Code to generate the tables and data can be found here.
SQL flavour is Microsoft SQL Server 2000 (although I've been running this stripped down test case on MySQL)
The original table
+-----------+----------+----------+
| master_id | slave_id | distance |
+-----------+----------+----------+
| 1 | 1 | 0.1 |
| 1 | 3 | 10 |
| 2 | 2 | 3 |
| 3 | 2 | 2 |
+-----------+----------+----------+
Description of what is required
I would like to select slave_id master_id pairs with MIN(distance) with no duplicates of either master_id or slave_id.
The desired results table
+-----------+----------+----------+
| master_id | slave_id | distance |
+-----------+----------+----------+
| 1 | 1 | 0.1 |
| 3 | 2 | 2 |
+-----------+----------+----------+
My Attempt
SELECT
join_table.master_id,
join_table.slave_id,
join_table.distance
FROM join_table
INNER JOIN
(
SELECT
slave_id,
MIN(distance) AS distance
FROM join_table
GROUP BY slave_id
) AS self_join
ON self_join.slave_id = join_table.slave_id
AND self_join.distance = join_table.distance
What's wrong with my attempt
This query produces duplicates of master_id
Any help will be very much appreciated.

This should give the correct result:
select distinct t.master_id,
t.slave_id,
t.distance
from join_table t
inner join
(
SELECT ID, min(Distance) dist
FROM
(
SELECT master_ID ID, MIN(distance) AS Distance
FROM join_table
GROUP BY master_ID
UNION
SELECT slave_ID ID, MIN(distance) AS Distance
FROM join_table
GROUP BY slave_ID
) src
GROUP BY ID
) md
on t.distance = md.dist
and (t.master_id = md.id or t.slave_id = md.id)
See SQL Fiddle with Demo

If I got you right, here is what I would do:
SELECT DISTINCT t.master_id
,t.slave_id
,t.distance
FROM your_table t
INNER JOIN
(
SELECT master_id id, min(distance) distance
FROM your_table
GROUP BY master_id
UNION
SELECT slave_id id, min(distance) distance
FROM your_table
GROUP BY slave_id
) sub
ON (sub.id = t.master_id AND sub.distance = t.distance)
OR (sub.id = t.slave_id AND sub.distance = t.distance)

Related

How can i find all linked rows by array values in postgres?

i have a table like this:
id | arr_val | grp
-----------------
1 | {10,20} | -
2 | {20,30} | -
3 | {50,5} | -
4 | {30,60} | -
5 | {1,5} | -
6 | {7,6} | -
I want to find out which rows are in a group together.
In this example 1,2,4 are one group because 1 and 2 have a common element and 2 and 4. 3 and 5 form a group because they have a common element. 6 Has no common elments with anybody else. So it forms a group for itself.
The result should look like this:
id | arr_val | grp
-----------------
1 | {10,20} | 1
2 | {20,30} | 1
3 | {50,5} | 2
4 | {30,60} | 1
5 | {1,5} | 2
6 | {7,6} | 3
I think i need recursive cte because my problem is graphlike but i am not sure how to that.
Additional info and background:
The Table has ~2500000 rows.
In reality the problem i try to solve has more fields and conditions for finding a group:
id | arr_val | date | val | grp
---------------------------------
1 | {10,20} | -
2 | {20,30} | -
Not only do the element of a group need to be linked by common elements in arr_val. They all need to have the same value in val and need to be linked by a timespan in date (gaps and islands). I solved the other two but now the condition of my question was added. If there is an easy way to do all three together in one query that would be awesome but it is not necessary.
----Edit-----
While both answer work for the example of five rows they do not work for a table with a lot more rows. Both answers have the problem that the number of rows in the recursive part explodes and only reduce them at the end.
A solutiuon should work for data like this too:
id | arr_val | grp
-----------------
1 | {1} | -
2 | {1} | -
3 | {1} | -
4 | {1} | -
5 | {1} | -
6 | {1} | -
7 | {1} | -
8 | {1} | -
9 | {1} | -
10 | {1} | -
11 | {1} | -
more rows........
Is there a solution to that problem?
You can handle this as a recursive CTE. Define the edges between the ids based on common values. Then traverse the edges and aggregate:
with recursive nodes as (
select id, val
from t cross join
unnest(arr_val) as val
),
edges as (
select distinct n1.id as id1, n2.id as id2
from nodes n1 join
nodes n2
on n1.val = n2.val
),
cte as (
select id1, id2, array[id1] as visited, 1 as lev
from edges
where id1 = id2
union all
select cte.id1, e.id2, visited || e.id2,
lev + 1
from cte join
edges e
on cte.id2 = e.id1
where e.id2 <> all(cte.visited)
),
vals as (
select id1, array_agg(distinct id2 order by id2) as id2s
from cte
group by id1
)
select *, dense_rank() over (order by id2s) as grp
from vals;
Here is a db<>fiddle.
Here is an approach at this graph-walking problem:
with recursive cte as (
select id, arr_val, array[id] path from mytable
union all
select t.id, t.arr_val, c.path || t.id
from cte c
inner join mytable t on t.arr_val && c.arr_val and not t.id = any(c.path)
)
select c.id, c.arr_val, dense_rank() over(order by min(x.id)) grp
from cte c
cross join lateral unnest(c.path) as x(id)
group by c.id, c.arr_val
order by c.id
The common-table-expression walks the graph, recursively looking for "adjacent" nodes to the current node, while keeping track of already-visited nodes. Then the outer query aggregates, identifies groups using the least node per path, and finally ranks the groups.
Demo on DB Fiddle:
id | arr_val | grp
-: | :------ | --:
1 | {10,20} | 1
2 | {20,30} | 1
3 | {50,5} | 2
4 | {30,60} | 1
5 | {1,5} | 2
6 | {7,6} | 3
While Gordon Linoffs solution is the fastest i found for a small amount of data where the groups are not so big it will not work for bigger datasets and bigger groups.
I changed his solution to make it work.
I moved the edges to an indexed table:
create table edges
(
id1 integer not null,
id2 integer not null,
constraint staffel_group_nodes_pk
primary key (id1, id2)
);
insert into edges(id1, id2) with
nodes(id, arr_val) as (
select id, arr_val
from my_table
)
select n1.id as id1, n2.id as id2
from nodes n1
join
nodes n2
on n1.arr_val && n2.arr_val ;
Tha alone didnt help. I changed his recursive part too:
with recursive
cte as (
select id1, array [id1] as visited
from edges
where id1 = id2
union all
select unnested.id1, array_agg(distinct unnested.vis) as visited
from (
select cte.id1,
unnest(cte.visited || e.id2) as vis
from cte
join
staffel_group_edges e
on e.id1 = any (cte.visited)
and e.id2 <> all (cte.visited)) as unnested
group by unnested.id1
),
vals as (
select id1, array_agg(distinct vis) as id2s
from (
select cte.id1,
unnest(cte.visited) as vis
from cte) as unnested
group by unnested.id1
)
select id1,id2s, dense_rank() over (order by id2s) as grp
from vals;
Every step i group all searches by their starting point. This reduces the amount of parallel walked paths a lot and works surprisingly fast.

SQL select all rows in a single row's "history"

I have a table that looks like this:
ID | PARENT_ID
--------------
0 | NULL
1 | 0
2 | NULL
3 | 1
4 | 2
5 | 4
6 | 3
Being an SQL noob, I'm not sure if I can accomplish what I would like in a single command.
What I would like is to start at row 6, and recursively follow the "history", using the PARENT_ID column to reference the ID column.
The result (in my mind) should look something like:
6|3
3|1
1|0
0|NULL
I already tried something like this:
SELECT T1.ID
FROM Table T1, Table T2
WHERE T1.ID = 6
OR T1.PARENT_ID = T2.PARENT_ID;
but that just gave me a strange result.
With a recursive cte.
If you want to start from the maximum id:
with recursive cte (id, parent_id) as (
select t.*
from (
select *
from tablename
order by id desc
limit 1
) t
union all
select t.*
from tablename t inner join cte c
on t.id = c.parent_id
)
select * from cte
See the demo.
If you want to start specifically from id = 6:
with recursive cte (id, parent_id) as (
select *
from tablename
where id = 6
union all
select t.*
from tablename t inner join cte c
on t.id = c.parent_id
)
select * from cte;
See the demo.
Results:
| id | parent_id |
| --- | --------- |
| 6 | 3 |
| 3 | 1 |
| 1 | 0 |
| 0 | |

How to create a condition for this case?

Sample Table:
Id |Acc_Code|Description |Balance | Acclevel| Acctype| Exttype|
--- -------- ----------------- |-------- |-------- | -------| -------|
1 |SA |Sales | 0.00 | 1 | SA | |
2 |CS |Cost of Sales | 0.00 | 1 | CS | |
3 |5000/001|Revenue | 94.34 | 2 | SA | |
4 |5000/090|Sales(Local) | 62.83 | 2 | SA | |
5 |7000/000|Manufacturing Acc |-250.80 | 2 | CS | MA |
6 |7000/200|Manufacturing Acc | 178.00 | 2 | CS | |
This is a sample data of a temporary table which would be used to be inserted into another temporary table that would calculate the data for Profit and Loss Statement (For Manufacturing related Accounts only).
In this case, the acc_code for Manufacturing accounts start from 7000/000 and separated/partitioned for each following Exttype.
Eg: We start from the exttype of MA and based on its acclevel (could be 2 or more) until the next exttype.
The idea is we get the manufacturing accounts by SELECT FROM tmp_acc_list WHERE acc_code BETWEEN #start_acc_code (7000/000 in this case) AND #end_acc_code (the data before the next exttype)
I don't know what the exttype is, I'm still learning the tables.
How do we create the #end_acc_code part out from this sample table?
So here is a all in one script.
I created Your table for test:
create table #tmp_acc_list(
Id numeric,
Acc_Code nvarchar(100),
Acclevel numeric,
Acctype nvarchar(100),
Exttype nvarchar(100));
GO
insert into #tmp_acc_list(Id, Acc_Code, Acclevel, Acctype, Exttype)
select 1 , 'SA', 1,'SA', null union all
select 2 , 'CS', 1,'CS', null union all
select 3 , '5000/001', 2,'SA', null union all
select 4 , '5000/090', 2,'SA', null union all
select 5 , '7000/000', 2,'CS', 'MA' union all
select 6 , '7000/200', 2,'CS', null
;
Then comes the query:
with OrderedTable as -- to order the table is Id is not an order
(
select
t.*, ROW_NUMBER() over (
order by id asc --use any ordering You need here
)
as RowNum
from
#tmp_acc_list as t
),
MarkedTable as -- mark with common number
(
select
t.*,
Max(case when t.Exttype is null then null else t.RowNum end)
over (order by t.RowNum) as GroupRownum
from OrderedTable as t
),
GroupedTable as -- add group Exttype
(
select
t.Id, t.Acc_Code, t.Acclevel, t.Acctype, t.Exttype,
max(t.Exttype) over (partition by t.GroupRownum) as GroupExttype
from MarkedTable as t
)
select * from GroupedTable where GroupExttype = 'MA'
Is this what You need?
select *
from
(
select Id, Acc_Code
from tmp_acc_list
where Acc_Code = '7000/000'
) s
cross join tmp_acc_list a
cross apply
(
select top 1 x.Id, x.Acc_Code
from tmp_acc_list x
where x.Id >= a.Id
and x.AccLevel = a.AccLevel
and x.Acctype = a.Acctype
and x.Exttype = ''
order by Id desc
) e
where a.Id between s.Id and e.Id

SQL 1 row twice

I have SQL table what looks like:
+----------+-----------+
| ID | Direction |
+----------+-----------+
| 1 | left |
| 1 | null |
| 2 | left |
| 2 | null |
| 3 | null |
| 4 | left |
| 4 | null |
| 5 | null |
+----------+-----------+
I want to show each value only once:
If there will be ID 1 with Direction null and left, then show only ID 1 with direction left.
If there will be ID 1 only with null value, show it with null value.
Use a common table expression (cte):
with cte as
(
Your huge select...
)
select *
from cte t1
where t1.Direction = 'left'
or not exists (select * from cte t2
where t2.kanbanid = t1.kanbanid
and t2.Direction = 'left')
I.e. if your select has Direction 'left' for a kanbanid, return that row. Also return that row if same kanbanid has no Direction 'left' at all.
Why wont below query work:
select id,max(dir)
from #temp
group by id
below is test data:
create table #temp
(
id int,
dir char(10)
)
insert into #temp
select 1,'left'
union all
select 1,null
union all
select 2,null
union all
select 3,'right'
union all
select 3,null
union all
select 3,null
select id,max(dir)
from #temp
group by id
aggregate functions will ignore null,below is the output:
select distinct *,
row_number() over (partition by id order by ,Direction )as row1 into #any_table
from #your_table_name
select * from #any_table
where row1 =1

how to query range?

Raw Data
| ID | STATUS |
| 1 | A |
| 2 | A |
| 3 | B |
| 4 | B |
| 5 | B |
| 6 | A |
| 7 | A |
| 8 | A |
| 9 | C |
Result
| START | END |
| 1 | 2 |
| 6 | 8 |
Range of STATUS A
How to query ?
This should give you the correct ranges:
SELECT
STATUS,
MIN(ID),
max_id
FROM (
SELECT
t1.STATUS,
t1.ID,
COALESCE(MAX(t2.ID), t1.ID) max_id
FROM
yourtable t1 LEFT JOIN yourtable t2
ON t1.STATUS=t2.STATUS AND t1.ID<t2.ID
WHERE
NOT EXISTS (SELECT NULL
FROM yourtable t3
WHERE
t3.STATUS!=t1.STATUS
AND t3.ID>t1.ID AND t3.ID<t2.ID)
GROUP BY
t1.ID,
t1.STATUS
) s
WHERE
status = 'A'
GROUP BY
STATUS,
max_id
Please see fiddle here.
You are probably better off with a cursor-based solution or a client-side function.
However, if you were using Oracle - the following would work.
WITH LOWER_VALS AS
( -- All the Ids with no immediate predecessor
SELECT ROWNUM AS RN, STATUS, ID AS LOWER FROM
(
SELECT STATUS, ID
FROM RAWDATA RD1
WHERE RD1.ID -1 NOT IN
(SELECT ID FROM RAWDATA PRED_TABLE WHERE PRED_TABLE.STATUS = RD1.STATUS)
ORDER BY STATUS, ID
)
) ,
UPPER_VALS AS
( -- All the Ids with no immediate successor
SELECT ROWNUM AS RN, STATUS, ID AS UPPER FROM
(
SELECT STATUS, ID
FROM RAWDATA RD2
WHERE RD2.ID +1 NOT IN
(SELECT ID FROM RAWDATA SUCC_TABLE WHERE SUCC_TABLE.STATUS = RD2.STATUS)
ORDER BY STATUS, ID
)
)
SELECT
L.STATUS, L.LOWER, U.UPPER
FROM
LOWER_VALS L
JOIN UPPER_VALS U ON
U.RN = L.RN;
Results in the set
A 1 2
A 6 8
B 3 5
C 9 9
http://sqlfiddle.com/#!4/10184/2
There is not a lot to go on from what you put, but I think this might work. I am using T-SQL because I don't know what you are using?
SELECT
min(ID)
, max(ID)
FROM RawData
WHERE [Status] = 'A'