SQL / HiveQL to assign values to buckets based on a table - hive

Question

I have a table "bucket" containing minimum int values for buckets, like this
min_value bucket_id
--------- ---------
0 1
12345 2
67890 3
i.e. any value >= 0 and < 12345 belongs in bucket 1, ..., any value >= 67890 belongs in bucket 3.
and a table of int values "value" like this:
id value
-- -----
11 10
22 20000
33 80000
I would like to figure out which bucket each value belongs to. So
select id, bucket_id
from (some join, or whatever, of bucket and value)
gives me
id bucket_id
-- ---------
11 1
22 2
33 3
I am trying to implement this in HiveQL. Any ideas?

Answer 1

I assumed that the condition for the bucket with largest min_value is min_value <= value (since there is no bucket with larger min_value) and I also assumed integer type for column value of table value and column min_value of table bucket (that's important because the query uses comparison which works differently in case of string type so you need to do typecasting for string).
The following query works for non-negative value of table value, in case of negative values involved, you have to replace
max(if(a.value >= b.min_value, b.min_value, 0))
with
max(if(a.value >= b.min_value, b.min_value, <minimum possible value that "value" field may have>)):
select
c.id,
if(d.bucket_id is null, 'not in bucket', d.bucket_id)
from
(
select
a.id,
max(if(a.value >= b.min_value, b.min_value, 0)) as bucket_min_value
from
value a
left join
bucket b
group by a.id
)
c
left join
bucket d
on c.bucket_min_value = d.min_value
;

Answer 2

You can use window functions to define ranges for the bucket ids and then join the bucket table. Check this out.
> select * from bucket;
+-------------------+-------------------+--+
| bucket.min_value | bucket.bucket_id |
+-------------------+-------------------+--+
| 0 | 1 |
| 12345 | 2 |
| 67890 | 3 |
+-------------------+-------------------+--+
> select * from buckvalue;
+---------------+------------------+--+
| buckvalue.id | buckvalue.value |
+---------------+------------------+--+
| 11 | 10 |
| 22 | 20000 |
| 33 | 80000 |
+---------------+------------------+--+
> select bucket_id, min_value, lead(min_value) over(order by bucket_id) as max1 from bucket;
INFO : OK
+------------+------------+--------+--+
| bucket_id | min_value | max1 |
+------------+------------+--------+--+
| 1 | 0 | 12345 |
| 2 | 12345 | 67890 |
| 3 | 67890 | NULL |
+------------+------------+--------+--+
> select t1.id, t1.value, t2.bucket_id from buckvalue t1 left outer join ( select bucket_id, min_value, lead(min_value) over(order by bucket_id) as max1 from bucket ) t2
where t1.value >= t2.min_value and t1.value < coalesce(t2.max1,99999);
+--------+-----------+---------------+--+
| t1.id | t1.value | t2.bucket_id |
+--------+-----------+---------------+--+
| 11 | 10 | 1 |
| 22 | 20000 | 2 |
| 33 | 80000 | 3 |
+--------+-----------+---------------+--+

Answer 3

I found a really simple query to do this. It works by finding all the bucket numbers for which the value is greater than the bucket's minimum value, and taking the maximum bucket_id.
create temporary table bucket as select * from (select 0 min_value, 1 bucket_id union select 12345, 2 union select 67890, 3) a;
create temporary table value as select * from (select 11 id, 10 value union select 22, 20000 union select 33, 80000) a;
select value.id, max(bucket.bucket_id) bucket_id
from value
join bucket
where value.value > bucket.min_value
group by value.id;