SQL: select top fewest rows with sum more than 0.7 - sql

Question

The raw data table is
+--------+--------+
| id | value |
+--------+--------+
| 1 | 0.1 |
| 1 | 0.2 |
| 1 | 0.3 |
| 1 | 0.2 |
| 1 | 0.2 |
| 2 | 0.4 |
| 2 | 0.5 |
| 2 | 0.1 |
| 3 | 0.5 |
| 3 | 0.5 |
+--------+--------+
For each id, its value sum is 1. I want to select the top fewest rows of each id with value sum is more than or equal with 0.7, like
+--------+--------+
| id | value |
+--------+--------+
| 1 | 0.3 |
| 1 | 0.2 |
| 1 | 0.2 |
| 2 | 0.5 |
| 2 | 0.4 |
| 3 | 0.5 |
| 3 | 0.5 |
+--------+--------+
How to solve this problem?

Answer 1

It's neither pretty nor efficient but it's the best I can come up with.
Disclaimer: I'm sure this will perform horribly on any real-world dataset.
with recursive calc (id, row_list, value_list, total_value) as (
select id, array[ctid], array[value]::numeric(6,2)[], value::numeric(6,2) as total_value
from data
union all
select c.id, p.row_list||c.ctid, (p.value_list||c.value)::numeric(6,2)[], (p.total_value + c.value)::numeric(6,2)
from data as c
join calc as p on p.id = c.id and c.ctid <> all(p.row_list)
)
select id, unnest(min(value_list)) as value
from (
select id,
value_list,
array_length(row_list,1) num_values,
min(array_length(row_list,1)) over (partition by id) as min_num_values
from calc
where total_value >= 0.7
) as result
where num_values = min_num_values
group by id
SQLFiddle example: http://sqlfiddle.com/#!15/8966b/1
How does this work?
The recursive CTE (thew with recursive) part creates all possible combinations of values from the table. To make sure that the same value is not counted twice I'm collecting the CTIDs (an Postgres internal unique identifier for each row) for each row already processed into an array. The recursive join condition (p.id = c.id and c.ctid <> all(p.row_list)) then makes sure only values for the same id are added and only those that have not yet processed.
The result of the CTE is then reduced to all rows where the total sum (the column total_value) is >= 0.7.
The final outer select (the alias result) is then filtered down to those where the number of values making up the total sum is the smallest. The distinct and unnest then transforms the arrays back into a proper "table". The distinct is necessary because the CTE collects all combinations so that for e.g. id=3 the value_list array will contain {0.40,0.50} and {0.50,0.40}. Without the distinct, the unnest would return both combinations making it a total of four rows for id=3.

Answer 2

This also isn't that pretty but I think it'd be more efficient (and more transferable between RDBMS')
with unique_data as (
select id
, value
, row_number() over ( partition by id order by value desc ) as rn
from my_table
)
, cumulative_sum as (
select id
, value
, sum(value) over ( partition by id order by rn ) as csum
from unique_data
)
, first_over_the_mark as (
select id
, value
, csum
, lag(csum) over ( partition by id order by csum ) as prev_value
from cumulative_sum
)
select *
from first_over_the_mark
where coalesce(prev_value, 0) < 0.7
SQL Fiddle
I've done it with CTEs to make it easier to see what's happening but there's no need to use them.
It uses a cumulative sum, the first CTE makes the data unique as without it 0.2 is the same value and so all rows that have 0.2 get summed together. The second works out the running sum. The third then works out the previous value. If the previous is strictly less than 0.7 pick up everything. The idea being that if the previous cumulative sum is less than 0.7 then the current value is more (or equal) to that number.
It's worth noting that this will break down if you have any rows in your table where the value is 0.

Answer 3

This is a variant on Ben's method, but it is simpler to implement. You just need a cumulative sum, ordered by value in reverse, and then to take everything where the cumulative sum is less then 0.7 plus the first one that exceeds that value.
select t.*
from (select t.*,
sum(value) over (partition by id order by value desc) as csum
from t
) t
where csum - value < 0.7;
The expression csum - value is the cumulative sum minus the current value (you can also get this using something like rows between unbounded preceding and 1 preceding). Your condition is that this value is less than some threshold.
EDIT:
Ben's comment is right about duplicate values. His solution is fine. Here is another solution:
select t.*
from (select t.*,
sum(value) over (partition by id order by value desc, random()) as csum
from t
) t
where csum - value < 0.7;