tsql grouping with duplication based on variable - sql

Question

I want to create some aggregations from a table but I am not able to figure out a solution.
Example table:
DECLARE #MyTable TABLE(person INT, the_date date, the_value int)
INSERT INTO #MyTable VALUES
(1,'2017-01-01', 10),
(1,'2017-02-01', 5),
(1,'2017-03-01', 5),
(1,'2017-04-01', 10),
(1,'2017-05-01', 2),
(2,'2017-04-01', 10),
(2,'2017-05-01', 10),
(2,'2017-05-01', 0),
(3,'2017-01-01', 2)
For each person existing at that time, I want to average the value for the last x (#months_back) months given some starting date (#start_date):
DECLARE #months_back int, #start_date date
set #months_back = 3
set #start_date = '2017-05-01'
SELECT person, avg(the_value) as avg_the_value
FROM #MyTable
where the_date <= #start_date and the_date >= dateadd(month, -#months_back, #start_date)
group by person
This works. I now want to do the same thing again but skip back some months (#month_skip) from the starting date. Then I want to union those two tables together. Then, I again want to skip back #month_skip months from this date and do the same thing. I want to continue doing this until I have skipped back to some specified date (#min_date).
DECLARE #months_back int, #month_skip int, #start_date date, #min_date date
set #months_back = 3
set #month_skip = 2
set #start_date = '2017-05-01'
set #min_date = '2017-03-01'
Using the above variables and the table #MyTable the result should be:
person | avg_the_value
1 | 5
2 | 6
1 | 6
3 | 2
Only one skip is made here since #min_date is 2 months back but I would like to be able to do multiple skips based on what #min_date is.
This example table is simple but the real one has many more automatically created columns and therefore it is not feasible to use a table variable where I would have to declare the scheme of the resulting table.
I asked a related question Here but did not manage to get any of the answers to work for this problem.

Answer 1

It sounds like what you're trying to do is the following:
Starting with a date (e.g. 2017-05-01), look back #months_back months and define a range of dates. For example, if we go 3 months back, we're defining a range from 2017-02-01 through 2017-05-01.
After we define this range, we go back to our starting date and define a new starting date, going back #month_skip months. For example, with an initial starting date of 2017-05-01, we might skip back 2 months, giving us a new starting date of 2017-03-01.
We take this new starting date, and define a range of corresponding dates (as we did above). This produces the range 2016-12-01 through 2017-03-01.
We repeat this as needed through the minimum date specified, to produce a list of date ranges we want to do calculations for:
2017-03-01 through 2017-05-01
2016-12-01 through 2017-03-01
... etc ...
For each of these periods, look at a person and calculate the average of their value.
The query below should do what is described above: rather than taking a value and iterating back to calculate previous values, we use a numbers table to calculate offsets on an interval, which is used to determine the ending and starting dates for each interval/period. This query was built using SQL Server 2008 R2 and should be compatible with future versions.
/* Table, data, variable declarations */
DECLARE #MyTable TABLE(person INT, the_date date, the_value int)
INSERT INTO #MyTable VALUES
(1,'2017-01-01', 10),
(1,'2017-02-01', 5),
(1,'2017-03-01', 5),
(1,'2017-04-01', 10),
(1,'2017-05-01', 2),
(2,'2017-04-01', 10),
(2,'2017-05-01', 10),
(2,'2017-05-01', 0),
(3,'2017-01-01', 2)
DECLARE #months_back int, #month_skip int, #start_date date, #min_date date
set #months_back = 3
set #month_skip = 2
set #start_date = '2017-05-01'
set #min_date = '2017-01-01'
/* Common table expression to build list of Integers */
/* reference http://www.itprotoday.com/software-development/build-numbers-table-you-need if you want more info */
declare #end_int bigint = 50
; WITH IntegersTableFill (ints) AS
(
SELECT
CAST(0 AS BIGINT) AS 'ints'
UNION ALL
SELECT (T.ints + 1) AS 'ints'
FROM IntegersTableFill T
WHERE ints <= (
CASE
WHEN (#end_int <= 32767) THEN #end_int
ELSE 32767
END
)
)
/* What we're going to do is define a series of periods.
These periods have a start date and an end date, and will simplify grouping
(in place of the calculate-and-union approach)
*/
/* Now, we start defining the periods
#months_Back_start defines the end of the range we need to calculate for.
#month_skip defines the amount of time we have to jump back for each period
*/
/* Using the number table we defined above and the data in our variables, calculate start and end dates */
,periodEndDates as
(
select ints as Period
,DATEADD(month, -(#months_back*ints), #start_date) as endOfPeriod
from IntegersTableFill itf
)
,periodStartDates as
(
select *
,DATEADD(month, -(#month_skip), endOfPeriod) as startOfPeriod
from periodEndDates
)
,finalPeriodData as
(
select (period) as period, startOfPeriod, endOfPeriod from periodStartDates
)
/* Link the entries in our original data to the periods they fall into */
/* NOTE: The join criteria originally specified allows values to fall into multiple periods.
You may want to fix this?
*/
,periodTableJoin as
(
select * from finalPeriodData fpd
inner join #MyTable mt
on mt.the_date >= fpd.startOfPeriod
and mt.the_date <= fpd.endOfPeriod
and mt.the_date >= #min_date
and mt.the_date <= #start_date
)
/* Calculate averages, grouping by period and person */
,periodValueAggregate as
(
select person, avg(the_value) as avg_the_value from
periodTableJoin
group by period, person
)
select * from periodValueAggregate

Answer 2

The method I propose is set-based, not iterative.
(I am not following your problem exactly, but please follow along and we can iron out any discrepancies)
Essentially, you are looking to divide a calendar up in to periods of interest. The periods are all equal in width and are sequential.
For this, I propose you build a calendar table and mark the periods using division as illustrated in the code;
DECLARE #CalStart DATE = '2017-01-01'
,#CalEnd DATE = '2018-01-01'
,#CalWindowSize INT = 2
;WITH Numbers AS
(
SELECT TOP (DATEDIFF(MONTH, #CalStart, #CalEnd)) N = CAST(ROW_NUMBER() OVER (ORDER BY (SELECT NULL)) AS INT) - 1
FROM syscolumns
)
SELECT CalWindow = N / #CalWindowSize
,CalDate = DATEADD(MONTH, N, #CalStart)
FROM Numbers
Once you have correctly configured the variables, you should have a calendar that represents the windows of interest.
It is then a matter of affixing this calendar to your dataset and grouping by not only the person but the CalWindow too;
DECLARE #MyTable TABLE(person INT, the_date date, the_value int)
INSERT INTO #MyTable VALUES
(1,'2017-01-01', 10),
(1,'2017-02-01', 5),
(1,'2017-03-01', 5),
(1,'2017-04-01', 10),
(1,'2017-05-01', 2),
(2,'2017-04-01', 10),
(2,'2017-05-01', 10),
(2,'2017-05-01', 0),
(3,'2017-01-01', 2)
----------------------------------
-- Build Calendar
----------------------------------
DECLARE #CalStart DATE = '2017-01-01'
,#CalEnd DATE = '2018-01-01'
,#CalWindowSize INT = 2
;WITH Numbers AS
(
SELECT TOP (DATEDIFF(MONTH, #CalStart, #CalEnd)) N = CAST(ROW_NUMBER() OVER (ORDER BY (SELECT NULL)) AS INT) - 1
FROM syscolumns
)
,Calendar AS
(
SELECT CalWindow = N / #CalWindowSize
,CalDate = DATEADD(MONTH, N, #CalStart)
FROM Numbers
)
SELECT TB.Person
,AVG(TB.the_value)
FROM #MyTable TB
JOIN Calendar CL ON TB.the_date = CL.CalDate
GROUP BY CL.CalWindow, TB.person
Hope I have understood your problem.