HiveQL - join multi-level subtotals to existing table - sql

Question

My goal is to determine the size of various organizations at various levels. Let's assume we have three organisations 'A', 'B', and 'C', each consisting of multiple department and having a further subdivision in teams with members., as outlined below:
Org. Dep. Tm. Member
A 1 I name1
A 1 I name2
A 1 I name3
A 1 II name4
A 2 I name5
A 2 I name6
B 1 I name7
B 1 II name8
B 1 II name9
B 1 II name10
B 2 I name11
B 2 I name12
B 2 II name13
B 2 II name14
B 2 III name15
B 2 III name16
C 1 I name17
C 1 I name18
C 1 I name19
C 1 I name20
C 1 I name21
Now, I'd like to know for each member how large their respective Org., Dep. and Tm. are, like this:
Org. Dep. Tm. Member org dep tm
A 1 I name1 6 4 3
A 1 I name2 6 4 3
A 1 I name3 6 4 3
A 1 II name4 6 4 1
A 2 I name5 6 2 2
A 2 I name6 6 2 2
B 1 I name7 10 4 1
B 1 II name8 10 4 3
B 1 II name9 10 4 3
B 1 II name10 10 4 3
B 2 I name11 10 6 2
B 2 I name12 10 6 2
B 2 II name13 10 6 2
B 2 II name14 10 6 2
B 2 III name15 10 6 2
B 2 III name16 10 6 2
C 1 I name17 5 5 5
C 1 I name18 5 5 5
C 1 I name19 5 5 5
C 1 I name20 5 5 5
C 1 I name21 5 5 5
My original idea was to do this with multiple LEFT JOINS to aggregate the different levels, but this scales very poorly as you need a new join for every aggregation level. Is there a way to do this efficiently in a single statement?

Answer 1

Use window functions:
select org, dep, tm,
count(*) over (partition by org) as org_cnt,
count(*) over (partition by org, dep) as dep_cnt,
count(*) over (partition by org, dep, tm) as tm_cnt
from t;
The columns are hierarchical so dep and tm need the higher levels of the hierarchy.
EDIT:
If Hive doesn't support count(distinct) and you need it, then you can do:
select org, dep, tm,
sum(case when seqnum_o = 1 then 1 else 0 end) over (partition by org) as org_cnt,
sum(case when seqnum_od = 1 then 1 else 0 end) over (partition by org, dep) as dep_cnt,
sum(case when seqnum_odt = 1 then 1 else 0 end) over (partition by org, dep, tm) as tm_cnt
from (select t.*,
row_number() over partition by org, memberid order by org) as seqnum_o,
row_number() over partition by org, dep, memberid order by org) as seqnum_od,
row_number() over partition by org, dep, tm, memberid order by org) as seqnum_odt
from t
) t;