SQL Server 2012 buckets based on running total - sql-server-2012

Question

For SQL Server 2012, I am trying to assign given rows to sequential buckets based on the maximum size of the bucket (100 in the sample below) and running total of a column. Most of the solutions I found partition by known column changing value e.g. partition by department id etc. However, in this situation all I have is sequential id and size. The closest solution I have found is discussed in this thread for SQL Server 2008 and I tried it but the performance very slow for large row set much worse than cursor based solution. https://dba.stackexchange.com/questions/45179/how-can-i-write-windowing-query-which-sums-a-column-to-create-discrete-buckets
This table can contain up to 10 Million rows. With SQL Server 2012 supporting SUM OVER and LAG and LEAD functions, wondering if someone can suggest a solution based on 2012.
CREATE TABLE raw_data (
id INT PRIMARY KEY
, size INT NOT NULL
);
INSERT INTO raw_data
(id, size)
VALUES
( 1, 96) -- new bucket here, maximum bucket size is 100
, ( 2, 10) -- and here
, ( 3, 98) -- and here
, ( 4, 20)
, ( 5, 50)
, ( 6, 15)
, ( 7, 97)
, ( 8, 96) -- and here
;
--Expected output
--bucket_size is for illustration only, actual needed output is bucket only
id size bucket_size bucket
-----------------------------
1 100 100 1
2 10 10 2
3 98 98 3
4 20 85 4
5 50 85 4
6 15 85 4
7 97 98 5
8 1 98 5
TIA

Answer 1

You can achieve this quite easily in SQL Server 2012 using a window function and framing. The syntax looks quite complex, but the concept is simple - sum all the previous rows up to and including the current one. The cumulative_bucket_size column in this example is for demonstration purposes, as it is part of the equation used to derive the bucket number:
DECLARE #Bucket_Size AS INT;
SET #Bucket_Size = 100
SELECT
id,
size,
SUM(size) OVER (
PARTITION BY 1 ORDER BY id ASC
ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
) AS cumulative_bucket_size,
1 + SUM(size) OVER (
PARTITION BY 1 ORDER BY id ASC
ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
) / #Bucket_Size AS bucket
FROM
raw_data
The PARTITION BY clause is optional, but would be useful if you had different "bucket sets" for column groupings. I have added it here for completeness.
Results:
id size cumulative_bucket_size bucket
------------------------------------------
1 96 96 1
2 10 106 2
3 98 204 3
4 20 224 3
5 50 274 3
6 15 289 3
7 97 386 4
8 96 482 5
You can read more about windows framing in the following article:
https://www.simple-talk.com/sql/learn-sql-server/window-functions-in-sql-server-part-2-the-frame/

Answer 2

Before you can use the running total method to assign bucket numbers, you need to generate that bucket_size column, because the numbers would be produced based on that column.
Based on your expected output, the bucket ranges are
1..10
11..85
86..100
You could use a simple CASE expression like this to generate a bucket_size column like in your example:
CASE
WHEN size <= 10 THEN 10
WHEN size <= 85 THEN 85
ELSE 100
END
Then you would use LAG() to determine if a row starts a new sequence of sizes belonging to the same bucket:
CASE bucket_size
WHEN LAG(bucket_size) OVER (ORDER BY id) THEN 0
ELSE 1
END
These two calculations could be done in the same (sub)query with the help of CROSS APPLY:
SELECT
d.id,
d.size,
x.bucket_size, -- for illustration only
is_new_seq = CASE x.bucket_size
WHEN LAG(x.bucket_size) OVER (ORDER BY d.id) THEN 0
ELSE 1
END
FROM dbo.raw_data AS d
CROSS APPLY
(
SELECT
CASE
WHEN size <= 10 THEN 10
WHEN size <= 85 THEN 85
ELSE 100
END
) AS x (bucket_size)
The above query would produce this output:
id size bucket_size is_new_seq
-- ---- ----------- ----------
1 96 100 1
2 10 10 1
3 98 100 1
4 20 85 1
5 50 85 0
6 15 85 0
7 97 100 1
8 96 100 0
Now use that result as a derived table and apply SUM() OVER to is_new_seq to produce the bucket numbers, like this:
SELECT
id,
size,
bucket = SUM(is_new_seq) OVER (ORDER BY id)
FROM
(
SELECT
d.id,
d.size,
is_new_seq = CASE x.bucket_size
WHEN LAG(x.bucket_size) OVER (ORDER BY d.id) THEN 0
ELSE 1
END
FROM dbo.raw_data AS d
CROSS APPLY
(
SELECT
CASE
WHEN size <= 10 THEN 10
WHEN size <= 85 THEN 85
ELSE 100
END
) AS x (bucket_size)
) AS s
;