I have a tree-based structure of SKUs within a Product Hierarchy in SQL Server. The lowest level SKUs will only ever have values (these are consumption values). I then want to generate aggregates up the hierarchy at every level.
Here's is the sample table structure:
Id
ParentId
Name
Volume
IsSku
1
-1
All
0
0
2
1
Cat A
0
0
3
1
Cat B
0
0
4
2
Cat A.1
0
0
5
2
Cat A.2
0
0
6
3
Cat B.1
0
0
7
3
Cat B.2
0
0
8
4
SKU1
10
1
9
4
SKU2
5
1
10
5
SKU3
7
1
11
5
SKU4
4
1
12
6
SKU1
10
1
13
6
SKU2
5
1
14
7
SKU3
9
1
15
7
SKU4
7
1
I need a query that will start at the sku level (IsSku=1) and then working up, will sum the SKUs and carry the sum up the product category levels to get a cumulative running total.
I've seen several queries where there are recursive sums in a hierarchical structure where each level already has values, but I need one that will start at the lowest level that has values and recursively calculate the sum as it moves upward.
I was trying these, but they look like they are mainly summing hierarchical data where each node already has a value (Volume, in my case). I need to start at the lowest level and carry the aggregate up as I go up the hierarchy. I tried to emulate the answers in these posts with my data, but wasn't successful so far with my data setup.
24394601
29127163
11408878
The output for the query should be like so:
Id
ParentId
Name
Volume
IsSku
1
-1
All
54
0
2
1
Cat A
26
0
3
1
Cat B
28
0
4
2
Cat A.1
15
0
5
2
Cat A.2
11
0
6
3
Cat B.1
12
0
7
3
Cat B.2
16
0
8
4
SKU1
10
1
9
4
SKU2
5
1
10
5
SKU3
7
1
11
5
SKU4
4
1
12
6
SKU1
10
1
13
6
SKU2
2
1
14
7
SKU3
9
1
15
7
SKU4
7
1
I've got a start with a recursive CTE that returns the hierarchy can can aggregate the volume if that node has volume already, but can't seem to figure out how to start at the SKU levels and continue aggregating up the hierarchy.
Here's the start with my CTE:
DECLARE #tblData TABLE
(
[ID] INT NOT NULL,
[ParentId] INT NULL,
[Name] varchar(50) NOT NULL,
[Volume] int NOT NULL,
[IsSku] bit
)
INSERT INTO #tblData
VALUES
(1,-1,'All',0,0)
,(2,1,'Cat A',0,0)
,(3,1,'Cat B',0,0)
,(4,2,'Cat A.1',0,0)
,(5,2,'Cat A.2',0,0)
,(6,3,'Cat B.1',0,0)
,(7,3,'Cat B.2',0,0)
,(8,4,'SKU1',10,1)
,(9,4,'SKU2',5,1)
,(10,5,'SKU3',7,1)
,(11,5,'SKU4',4,1)
,(12,6,'SKU1',10,1)
,(13,6,'SKU2',5,1)
,(14,7,'SKU3',7,1)
,(15,7,'SKU4',4,1)
;WITH cte AS (
SELECT
a.ID
,a.ParentID
,a.Name
,a.Volume
,CAST('/' + cast(ID as varchar) + '/' as varchar) Node
,0 AS level
,IsSku
FROM #tblData AS a
WHERE a.ParentID = -1
UNION ALL
SELECT
b.ID
,b.ParentID
,b.Name
,b.Volume
,CAST(c.Node + CAST(b.ID as varchar) + '/' as varchar)
,level = c.level + 1
,b.IsSku
FROM #tblData AS b
INNER JOIN cte c
ON b.ParentId = c.ID
)
SELECT c1.ID, c1.ParentID, c1.Name, c1.Node
,ISNULL(SUM(c2.Volume),0)
FROM cte c1
LEFT OUTER JOIN cte c2
ON c1.Node <> c2.Node
AND LEFT(c2.Node, LEN(c1.Node)) = c1.Node
GROUP BY c1.ID, c1.ParentID, c1.Name, c1.Node
Any help is appreciated!
This should do it:
DECLARE #tbl TABLE(Id INT, ParentId INT, Name NVARCHAR(255), Volume INTEGER, IsSku BIT)
INSERT INTO #tbl
VALUES
(1,-1,'All',0,0)
,(2,1,'Cat A',0,0)
,(3,1,'Cat B',0,0)
,(4,2,'Cat A.1',0,0)
,(5,2,'Cat A.2',0,0)
,(6,3,'Cat B.1',0,0)
,(7,3,'Cat B.2',0,0)
,(8,4,'SKU1',10,1)
,(9,4,'SKU2',5,1)
,(10,5,'SKU3',7,1)
,(11,5,'SKU4',4,1)
,(12,6,'SKU1',10,1)
,(13,6,'SKU2',5,1)
,(14,7,'SKU3',7,1)
,(15,7,'SKU4',4,1)
SELECT * FROM #tbl
;
WITH cte AS (
SELECT
Id,ParentId, Name, Volume, IsSku, CAST(Id AS VARCHAR(MAX)) AS Hierarchy
FROM
#tbl
WHERE ParentId=-1
UNION ALL
SELECT
t.Id,t.ParentId, t.Name, t.Volume, t.IsSku, CAST(c.Hierarchy + '|' + CAST(t.Id AS VARCHAR(MAX)) AS VARCHAR(MAX))
FROM
cte c
INNER JOIN #tbl t
ON c.Id = t.ParentId
)
SELECT Id,ParentId, Name, ChildVolume AS Volume, IsSku
FROM (
SELECT c1.Id, c1.ParentId, c1.Name, c1. Volume, c1.IsSku, SUM(c2.Volume) AS ChildVolume
FROM cte c1
LEFT JOIN cte c2 ON c2.Hierarchy LIKE c1.Hierarchy + '%'
GROUP BY c1.Id, c1.ParentId, c1.Name, c1. Volume, c1.IsSku
) x
Basically the computation happens in three steps:
Capture the Hierarchy recursively for each descendant by concatenating the Ids: CAST(c.Hierarchy + '|' + CAST(t.Id AS VARCHAR(MAX)) AS VARCHAR(MAX))
Join the resulting table with itself so each record is joined with itself and all its descendants: FROM cte c1 LEFT JOIN cte c2 ON c2.Hierarchy LIKE c1.Hierarchy + '%'
Finally aggregate the Volume of each hierarchy by grouping: SUM(c2.Volume) AS ChildVolume
This is in reference to Ed Harper's answer to a similar question here: Hierarchy based aggregation
Due to the way that recursive CTEs work in SQL Server, it is very difficult to get this kind of logic working efficiently. It often either requires self-joining the whole resultset, or using something like JSON or XML.
The problem is that at each recursion of the CTE, although it appears you are working on the whole set at once, it actually only feeds back one row at a time. Therefore grouping is disallowed over the recursion.
Instead, it's much better to simply recurse with a WHILE loop and insert into a temp table or table variable, then read it back to aggregate
Use the OUTPUT clauses to view the intermediate results
DECLARE #tmp TABLE (
Id INTEGER,
ParentId INTEGER,
Name VARCHAR(7),
Volume INTEGER,
IsSku INTEGER,
Level INT,
INDEX ix CLUSTERED (Level, ParentId, Id)
);
INSERT INTO #tmp
(Id, ParentId, Name, Volume, IsSku, Level)
-- OUTPUT inserted.Id, inserted.ParentId, inserted.Name, inserted.Volume, inserted.IsSku, inserted.Level
SELECT
p.Id,
p.ParentId,
p.Name,
p.Volume,
p.IsSku,
1
FROM Product p
WHERE p.IsSku = 1;
DECLARE #level int = 1;
WHILE (1=1)
BEGIN
INSERT INTO #tmp
(Id, ParentId, Name, Volume, IsSku, Level)
-- OUTPUT inserted.Id, inserted.ParentId, inserted.Name, inserted.Volume, inserted.IsSku, inserted.Level
SELECT
p.Id,
p.ParentId,
p.Name,
t.Volume,
p.IsSku,
#level + 1
FROM (
SELECT
t.ParentID,
Volume = SUM(t.Volume)
FROM #tmp t
WHERE t.Level = #level
GROUP BY
t.ParentID
) t
JOIN Product p ON p.Id = t.ParentID;
IF (##ROWCOUNT = 0)
BREAK;
SET #level += 1;
END;
SELECT *
FROM #tmp
ORDER BY Id;
db<>fiddle
This solution does involve a blocking operator, due to Halloween protection (in my case I saw an "unnecessary" sort). You can avoid it by using Itzik Ben-Gan's Divide and Conquer method, utilizing two table variables and flip-flopping between them.
I have a huge table, where sometimes 1 product ID has multiple specifications. I want to select the newest but unfortunately, I don't have the date information. please consider this example dataset
Row ID Type Sn Sn_Ind
1 3 SLN SL20 20
2 1 SL SL 0
3 2 SL SL 0
4 1 M SL21 10
5 3 M SL21 10
6 1 SLN SL20 20
I used the below query to somehow group the products in give them row numbers like
with cleanedMasterData as(
SELECT *
FROM (
SELECT *,ROW_NUMBER() OVER(PARTITION BY ID ORDER BY Sn DESC, Sn_Ind DESC) AS rn
FROM `project.dataset.table`
)
-- where rn = 1
)
select * from cleanedMasterData
Please find below the example table after cleaning
Row ID Type Sn Sn_Ind rn
1 1 SL SL 0 1
2 1 M SL21 10 2
3 1 SLN SL20 20 3
4 2 SL SL 0 1
5 3 M SL21 10 1
6 3 SLN SL20 20 2
but if you see for ID 2 and 3, I can easily select the top row with where rn = 1
but for ID 1, my preferred row would be 2 because that is the newest.
My question here is how do I prioritise a value in column so that I can get the desired solution like :
Row ID Type Sn Sn_Ind rn
1 1 M SL21 10 1
2 2 SL SL 0 1
3 3 M SL21 10 1
As the values are fixed in Sn column - for ex SL, SL20, SL19, SL21 etc - If somehow I can give weightage to these values and create a new temp column with weightage and sort based on it, then?
Thank you for your support in advance!!
Consider below
SELECT *
FROM `project.dataset.table`
WHERE TRUE
QUALIFY ROW_NUMBER() OVER(PARTITION BY ID ORDER BY IF(Sn = 'SL', 0, 1) DESC, Sn DESC) = 1
If applied to sample data in your question - output is
It wasn't difficult, I tried a few things and it worked out. If anyone can optimize the below solution even more that would be awesome.
first the dataset
#standardSQL
WITH `project.dataset.table` AS (
SELECT 1 ID, 'SLN' Type, 'SL20' Sn, 20 Sn_Ind UNION ALL
SELECT 1 , 'SL' , 'SL' , 0 UNION ALL
SELECT 2 , 'SL' , 'SL' , 0 UNION ALL
SELECT 1 , 'M' , 'SL21' , 10 UNION ALL
SELECT 3 , 'M' , 'SL21' , 10 UNION ALL
SELECT 1 , 'SLN' , 'SL20' , 20
)
with weightage as(
SELECT
*,
MAX(CASE Sn WHEN 'SL' THEN 0 ELSE 1 END) OVER (PARTITION BY Sn) AS weightt,
FROM
`project.dataset.table`
ORDER BY
weightt DESC, Sn DESC
), main as (
select * EXCEPT(rn, weightt)
from (
select * ,ROW_NUMBER() OVER(PARTITION BY ID ORDER BY weightt DESC, Sn DESC) AS rn
from weightage )
where rn = 1
)
select * from main
after this, I can get the desired result
Row ID Type Sn Sn_Ind
1 1 M SL21 10
2 2 SL SL 0
3 3 M SL21 10
I have a table containing details on my company's chart of accounts - this data is essentially stored in nested sets (on SQL Server 2014), with each record having a left and right anchor - there are no Parent IDs.
Sample Data:
ID LeftAnchor RightAnchor Name
1 0 25 Root
2 1 16 Group 1
3 2 9 Group 1.1
4 3 4 Account 1
5 5 6 Account 2
6 7 8 Account 3
7 10 15 Group 1.2
8 11 12 Account 4
9 13 14 Account 5
10 17 24 Group 2
11 18 23 Group 2.1
12 19 20 Account 1
13 21 22 Account 1
I need to materialize the path for each record, so that my output looks like this:
ID LeftAnchor RightAnchor Name MaterializedPath
1 0 25 Root Root
2 1 16 Group 1 Root > Group 1
3 2 9 Group 1.1 Root > Group 1 > Group 1.1
4 3 4 Account 1 Root > Group 1 > Group 1.1 > Account 1
5 5 6 Account 2 Root > Group 1 > Group 1.1 > Account 2
6 7 8 Account 3 Root > Group 1 > Group 1.1 > Account 3
7 10 15 Group 1.2 Root > Group 1 > Group 1.2
8 11 12 Account 4 Root > Group 1 > Group 1.2 > Acount 4
9 13 14 Account 5 Root > Group 1 > Group 1.2 > Account 5
10 17 24 Group 2 Root > Group 2
11 18 23 Group 2.1 Root > Group 2 > Group 2.1
12 19 20 Account 1 Root > Group 2 > Group 2.1 > Account 10
13 21 22 Account 1 Root > Group 2 > Group 2.1 > Account 11
Whilst I've managed to achieve this using CTEs, the query is deathly slow. It takes just shy of two minutes to run with around 1200 records in the output.
Here's a simplified version of my code:
;with accounts as
(
-- Chart of Accounts
select AccountId, LeftAnchor, RightAnchor, Name
from ChartOfAccounts
-- dirty great where clause snipped
)
, parents as
(
-- Work out the Parent Nodes
select c.AccountId, p.AccountId [ParentId]
from accounts c
left join accounts p on (p.LeftAnchor = (
select max(i.LeftAnchor)
from accounts i
where i.LeftAnchor<c.LeftAnchor
and i.RightAnchor>c.RightAnchor
))
)
, path as
(
-- Calculate the Account path for each node
-- Root Node
select c.AccountId, c.LeftAnchor, c.RightAnchor, 0 [Level], convert(varchar(max), c.name) [MaterializedPath]
from accounts c
where c.LeftAnchor = (select min(LeftAnchor) from chart)
union all
-- Children
select n.AccountId, n.LeftAnchor, n.RightAnchor, p.level+1, p.path + ' > ' + n.name
from accounts n
inner join parents x on (n.AccountId=x.AccountId)
inner join path p on (x.ParentId=p.AccountId)
)
select * from path order by LeftAnchor
Ideally this query should only take a couple of seconds (max) to run. I can't make any changes to the database itself (read-only connection), so can anyone come up with a better way to write this query?
After your comments, I realized no need for the CTE... you already have the range keys.
Example
Select A.*
,Path = Replace(Path,'>','>')
From YourTable A
Cross Apply (
Select Path = Stuff((Select ' > ' +Name
From (
Select LeftAnchor,Name
From YourTable
Where A.LeftAnchor between LeftAnchor and RightAnchor
) B1
Order By LeftAnchor
For XML Path (''))
,1,6,'')
) B
Order By LeftAnchor
Returns
First you can try to rearrange your preparing CTEs (accounts and parents) to have it that each CTE contains all data from previous, so you only use the last one in path CTE - no need for multiple joins:
;with accounts as
(
-- Chart of Accounts
select AccountId, LeftAnchor, RightAnchor, Name
from ChartOfAccounts
-- dirty great where clause snipped
)
, parents as
(
-- Work out the Parent Nodes
select c.*, p.AccountId [ParentId]
from accounts c
left join accounts p on (p.LeftAnchor = (
select max(i.LeftAnchor)
from accounts i
where i.LeftAnchor<c.LeftAnchor
and i.RightAnchor>c.RightAnchor
))
)
, path as
(
-- Calculate the Account path for each node
-- Root Node
select c.AccountId, c.LeftAnchor, c.RightAnchor, 0 [Level], convert(varchar(max), c.name) [MaterializedPath]
from parents c
where c.ParentID IS NULL
union all
-- Children
select n.AccountId, n.LeftAnchor, n.RightAnchor, p.level+1, p.[MaterializedPath] + ' > ' + n.name
from parents n
inner join path p on (n.ParentId=p.AccountId)
)
select * from path order by LeftAnchor
This should give some improvement (50% in my test), but to have it really better, you can split first half of preparing data into #temp table, put clustered index on ParentID column in #temp table and use it in second part
if (Object_ID('tempdb..#tmp') IS NOT NULL) DROP TABLE #tmp;
with accounts as
(
-- Chart of Accounts
select AccountId, LeftAnchor, RightAnchor, Name
from ChartOfAccounts
-- dirty great where clause snipped
)
, parents as
(
-- Work out the Parent Nodes
select c.*, p.AccountId [ParentId]
from accounts c
left join accounts p on (p.LeftAnchor = (
select max(i.LeftAnchor)
from accounts i
where i.LeftAnchor<c.LeftAnchor
and i.RightAnchor>c.RightAnchor
))
)
select * into #tmp
from parents;
CREATE CLUSTERED INDEX IX_tmp1 ON #tmp (ParentID);
With path as
(
-- Calculate the Account path for each node
-- Root Node
select c.AccountId, c.LeftAnchor, c.RightAnchor, 0 [Level], convert(varchar(max), c.name) [MaterializedPath]
from #tmp c
where c.ParentID IS NULL
union all
-- Children
select n.AccountId, n.LeftAnchor, n.RightAnchor, p.level+1, p.[MaterializedPath] + ' > ' + n.name
from #tmp n
inner join path p on (n.ParentId=p.AccountId)
)
select * from path order by LeftAnchor
Hard to tell on small sample data, but it should be an improvement. Please tell if you try it.
Seems odd to me that you don't have a Parent ID, but with the aid of an initial OUTER APPLY, we can generate a Parent ID and then run a standard recursive CTE.
Example
Declare #Top int = null --<< Sets top of Hier Try 12 (Just for Fun)
;with cte0 as (
Select A.*
,B.*
From YourTable A
Outer Apply (
Select Top 1 Pt=ID
From YourTable
Where A.LeftAnchor between LeftAnchor and RightAnchor and LeftAnchor<A.LeftAnchor
Order By LeftAnchor Desc
) B
)
,cteP as (
Select ID
,Pt
,LeftAnchor
,RightAnchor
,Lvl=1
,Name
,Path = cast(Name as varchar(max))
From cte0
Where IsNull(#Top,-1) = case when #Top is null then isnull(Pt ,-1) else ID end
Union All
Select r.ID
,r.Pt
,r.LeftAnchor
,r.RightAnchor
,p.Lvl+1
,r.Name
,cast(p.path + ' > '+r.Name as varchar(max))
From cte0 r
Join cteP p on r.Pt = p.ID
)
Select *
From cteP
Order By LeftAnchor
Returns
I have a table [Departments] with 2 columns:
[IdDepartment]
[IdSubDepartment]
The table is a kind of hierarchy:
IdDepartment | IdSubDepartment
1 | 2
1 | 3
2 | 4
3 | 5
If I search for department 5 I want to get the following 5 -> 3 -> 1
(I only need the X level every time - not always the root).
I have written a query that gets a department ID and returns its 3rd level (say I enter ID 5 and get back 1). It works fast and good. the problem is when i do that for 7K departments, it gets stuck.
I want to convert the table to a pivot like this:
IdDepartment0 | IdDepartment1 | IdDepartment2 ...
1 2 4
1 3 5
important: I know the level of each department.
so, when I get department 5, I know it is on level 2 (IdDepartment2)
so I can query my new table in no time and get each department level I want.
How do I do convert to the new table?
thanks in advance
Eran
This snipped can be expanded to include deeper nesting.
It can propbably be optimized some.
;WITH cteLvl AS
(
SELECT IdDepartment, IdSubDepartment, 0 AS Lvl
FROM Department
WHERE IdDepartment NOT IN (SELECT IdSubDepartment FROM Department WHERE IdSubDepartment IS NOT NULL)
UNION ALL
SELECT B.IdDepartment, B.IdSubDepartment, A.Lvl + 1
FROM cteLvl A
INNER JOIN Department B ON B.IdDepartment = A.IdSubDepartment
)
, cteLeaf AS
(
SELECT *, ROW_NUMBER() OVER(ORDER BY IdDepartment) AS GroupId
FROM Department
WHERE IdSubDepartment IS NULL
UNION ALL
SELECT B.IdDepartment, B.IdSubDepartment, A.GroupId
FROM cteLeaf A
INNER JOIN Department B ON A.IdDepartment = B.IdSubDepartment
)
, cteCombined AS
(
SELECT A.IdDepartment, A.GroupId, B.Lvl FROM cteLeaf A
INNER JOIN (SELECT DISTINCT IdDepartment, Lvl FROM cteLvl) B ON A.IdDepartment = B.IdDepartment
)
--SELECT * FROM cteCombined
SELECT GroupId, [0] AS Dep0, [1] AS Dep1, [2] AS Dep2, [3] AS Dep3, [4] AS Dep4
FROM
(SELECT GroupId, Lvl, IdDepartment
FROM cteCombined) P
PIVOT
(
SUM(IdDepartment)
FOR Lvl IN
( [0], [1], [2], [3], [4] )
) AS V
Same effect without using the PIVOT construct:
SELECT
GroupId,
MAX(CASE Lvl WHEN 0 THEN IdDepartment END) AS Dep0,
MAX(CASE Lvl WHEN 1 THEN IdDepartment END) AS Dep1,
MAX(CASE Lvl WHEN 2 THEN IdDepartment END) AS Dep2,
MAX(CASE Lvl WHEN 3 THEN IdDepartment END) AS Dep3
FROM cteCombined
GROUP BY GroupId
I have this table in sql server 2005:
id student active
1 Bob 1
3 Rob 0
5 Steve 1
7 John 1
8 Mark 0
10 Dave 0
16 Nick 1
My select query returns an active student by a given id.
But I also want to return the ids of prev and next student who are active. If no prev, it will be 0 or null. Same for next.
Example: for id=5, my select would return
id student prev_id next_id
5 steve 1 7
Example: for id=7, my select would return
id student prev_id next_id
7 John 5 16
Example: for id=16, my select would return
id student prev_id next_id
16 Nick 7 0
How do I write this select query?
I have query but I just can't get the prev id correctly. It always returns the first active id.
Thanks
EDIT:
Here is the query I have right now.
select id, student,
(select top 1 id from test where id<7 and active=1) as prev,
(select top 1 id from test where id>7 and active=1) as next
from test where id=7--I used 7 just as an example. it will be a parameter
try something like this
SELECT ID,
Student,
( SELECT TOP 1
ID
FROM dbo.table AS pT
WHERE pT.ID < T.ID And Active = 1
ORDER BY ID DESC
) AS PrevID,
( SELECT TOP 1
ID
FROM dbo.table AS pT
WHERE pT.ID > T.ID And Active = 1
ORDER BY ID
) AS NextID
FROM dbo.table AS T
Working sample
DECLARE #T TABLE (id int, student varchar(10), active bit)
insert #t select
1 ,'Bob', 1 union all select
3 ,'Rob', 0 union all select
5 ,'Steve', 1 union all select
7 ,'John', 1 union all select
8 ,'Mark', 0 union all select
10 ,'Dave', 0 union all select
16 ,'Nick', 1
---- your query starts below this line
declare #id int set #id = 5
select id, student,
isnull((select top(1) Prev.id from #T Prev
where Prev.id < T.id and Prev.active=1
order by Prev.id desc),0) Prev,
isnull((select top(1) Next.id from #T Next
where Next.id > T.id and Next.active=1
order by Next.id),0) Next
from #T T
where id = #id
The ISNULLs are to return 0 when there is no match - NULL would have worked fine but your question has 0 when there is no Next.
You may want to take a look at Common Table Expression, a feature for only SQL Server for recursive queries, you can find a link here
But this sound like homework, and probebly not the right forum to ask it in.
Regards
You could use a nested query. I obviously can't test this out, but you should get the idea.
SELECT id, student ,
(SELECT C1.id FROM students S1 WHERE S1.active = 1 AND S1.id < S.id LIMIT 1) AS beforeActive,
(SELECT C2.id FROM categories S2 WHERE S2.active = 1 AND S2.id > S.id LIMIT 1) AS afterActive
FROM students S
Efficiency wise, I've no idea how well this query will perform
This will give you a little more control, especially since you are paginating.
WITH NumberedSet AS (
SELECT s.id,
s.student,
row_number() OVER (ORDER BY s.id) AS rownum
FROM dbo.students AS s
WHERE s.active = 1
)
SELECT cur.id,
cur.student,
isnull(prv.id,0) AS prev_id,
isnull(nxt.id,0) AS next_id
FROM NumberedSet AS cur
LEFT JOIN NumberedSet AS prv ON cur.rownum - 1 = prv.rownum
LEFT JOIN NumberedSet AS nxt ON cur.rownum + 1 = nxt.rownum
;