sql server rolling 12 months sum with date gaps - sql

Suppose I have a table that indicates the number of items sold in a particular month for each sales rep. However, there will not be a row for a particular person in months where there were no sales. Example
rep_id month_yr num_sales
1 01/01/2012 3
1 05/01/2012 1
1 11/01/2012 1
2 02/01/2012 2
2 05/01/2012 1
I want to be able to create a query that shows for each rep_id and all possible months (01/01/2012, 02/01/2012, etc. through current) a rolling 12 month sales sum, like this:
rep_id month_yr R12_Sum
1 11/01/2012 5
1 12/01/2012 5
1 01/01/2013 5
1 02/01/2013 2
I have found some examples online, but the problem I'm running into is I'm missing some dates for each rep_id. Do I need to cross join or something?

To solve this problem, you need a driver table that has all year/month combinations. Then, you need to create this for each rep.
The solution is then to left join the actual data to this driver and aggregate the period that you want. Here is the query:
with months as (
select 1 as mon union all select 2 union all select 3 union all select 4 union all
select 5 as mon union all select 6 union all select 7 union all select 8 union all
select 9 as mon union all select 10 union all select 11 union all select 12
),
years as (select 2010 as yr union all select 2011 union all select 2012 union all select 2013
),
monthyears as (
select yr, mon, yr*12+mon as yrmon
from months cross join years
),
rmy as (
select *
from monthyears my cross join
(select distinct rep_id from t
) r
)
select rmy.rep_id, rmy.yr, rmy.mon, SUM(t.num_sales) as r12_sum
from rmy join
t
on rmy.rep_id = t.rep_id and
t.year(month_yr)*12 + month(month_yr) between rmy.yrmon - 11 and rmy.yrmon
group by rmy.rep_id, rmy.yr, rmy.mon
order by 1, 2, 3
This hasn't been tested, so it may have syntactic errors. Also, it doesn't convert the year/month combination back to a date, leaving the values in separate columns.

Here is one solution:
SELECT
a.rep_id
,a.month_yr
,SUM(b.R12_Sum) AS R12_TTM
FROM YourTable a
LEFT OUTER JOIN YourTable b
ON a.rep_id = b.rep_id
AND a.month_yr <= b.month_yr
AND a.month_yr >= DATEADD(MONTH, -11, b.month_yr)
GROUP BY
a.rep_id
,a.month_yr

It's certainly not pretty but is more simple than a CTE, numbers table or self join:
DECLARE #startdt DATETIME
SET #startdt = '2012-01-01'
SELECT rep_id, YEAR(month_yr), MONTH(month_yr), SUM(num_sales)
FROM MyTable WHERE month_yr >= #startdt AND month_yr < DATEADD(MONTH,1,#startdt)
UNION ALL
SELECT rep_id, YEAR(month_yr), MONTH(month_yr), SUM(num_sales)
FROM MyTable WHERE month_yr >= DATEADD(MONTH,1,#startdt) AND month_yr < DATEADD(MONTH,2,#startdt)
UNION ALL
SELECT rep_id, YEAR(month_yr), MONTH(month_yr), SUM(num_sales)
FROM MyTable WHERE month_yr >= DATEADD(MONTH,2,#startdt) AND month_yr < DATEADD(MONTH,3,#startdt)
UNION ALL
SELECT rep_id, YEAR(month_yr), MONTH(month_yr), SUM(num_sales)
FROM MyTable WHERE month_yr >= DATEADD(MONTH,3,#startdt) AND month_yr < DATEADD(MONTH,4,#startdt)
UNION ALL
etc etc

The following demonstrates using a CTE to generate a table of dates and generating a summary report using the CTE. Sales representatives are omitted from the results when they have had no applicable sales.
Try jiggling the reporting parameters, e.g. setting #RollingMonths to 1, for more entertainment.
-- Sample data.
declare #Sales as Table ( rep_id Int, month_yr Date, num_sales Int );
insert into #Sales ( rep_id, month_yr, num_sales ) values
( 1, '01/01/2012', 3 ),
( 1, '05/01/2012', 1 ),
( 1, '11/01/2012', 1 ),
( 2, '02/01/2012', 1 ),
( 2, '05/01/2012', 2 );
select * from #Sales;
-- Reporting parameters.
declare #ReportEnd as Date = DateAdd( day, 1 - Day( GetDate() ), GetDate() ); -- The first of the current month.
declare #ReportMonths as Int = 6; -- Number of months to report.
declare #RollingMonths as Int = 12; -- Number of months in rolling sums.
-- Report.
-- A CTE generates a table of month/year combinations covering the desired reporting time period.
with ReportingIntervals as (
select DateAdd( month, 1 - #ReportMonths, #ReportEnd ) as ReportingInterval,
DateAdd( month, 1 - #RollingMonths, DateAdd( month, 1 - #ReportMonths, #ReportEnd ) ) as FirstRollingMonth
union all
select DateAdd( month, 1, ReportingInterval ), DateAdd( month, 1, FirstRollingMonth )
from ReportingIntervals
where ReportingInterval < #ReportEnd )
-- Join the CTE with the sample data and summarize.
select RI.ReportingInterval, S.rep_id, Sum( S.num_sales ) as R12_Sum
from ReportingIntervals as RI left outer join
#Sales as S on RI.FirstRollingMonth <= S.month_yr and S.month_yr <= RI.ReportingInterval
group by RI.ReportingInterval, S.rep_id
order by RI.ReportingInterval, S.rep_id

Related

Calculating per day in SQL

I have an sql table like that:
Id Date Price
1 21.09.09 25
2 31.08.09 16
1 23.09.09 21
2 03.09.09 12
So what I need is to get min and max date for each id and dif in days between them. It is kind of easy. Using SQLlite syntax:
SELECT id,
min(date),
max(date),
julianday(max(date)) - julianday(min(date)) as dif
from table group by id
Then the tricky one: how can I receive the price per day during this difference period. I mean something like this:
ID Date PricePerDay
1 21.09.09 25
1 22.09.09 0
1 23.09.09 21
2 31.08.09 16
2 01.09.09 0
2 02.09.09 0
2 03.09.09 12
I create a cte as you mentioned with calendar but dont know how to get the desired result:
WITH RECURSIVE
cnt(x) AS (
SELECT 0
UNION ALL
SELECT x+1 FROM cnt
LIMIT (SELECT ((julianday('2015-12-31') - julianday('2015-01-01')) + 1)))
SELECT date(julianday('2015-01-01'), '+' || x || ' days') as date FROM cnt
p.s. If it will be in sqllite syntax-would be awesome!
You can use a recursive CTE to calculate all the days between the min date and max date. The rest is just a left join and some logic:
with recursive cte as (
select t.id, min(date) as thedate, max(date) as maxdate
from t
group by id
union all
select cte.id, date(thedate, '+1 day') as thedate, cte.maxdate
from cte
where cte.thedate < cte.maxdate
)
select cte.id, cte.date,
coalesce(t.price, 0) as PricePerDay
from cte left join
t
on cte.id = t.id and cte.thedate = t.date;
One method is using a tally table.
To build a list of dates and join that with the table.
The date stamps in the DD.MM.YY format are first changed to the YYYY-MM-DD date format.
To make it possible to actually use them as a date in the SQL.
At the final select they are formatted back to the DD.MM.YY format.
First some test data:
create table testtable (Id int, [Date] varchar(8), Price int);
insert into testtable (Id,[Date],Price) values (1,'21.09.09',25);
insert into testtable (Id,[Date],Price) values (1,'23.09.09',21);
insert into testtable (Id,[Date],Price) values (2,'31.08.09',16);
insert into testtable (Id,[Date],Price) values (2,'03.09.09',12);
The SQL:
with Digits as (
select 0 as n
union all select 1
union all select 2
union all select 3
union all select 4
union all select 5
union all select 6
union all select 7
union all select 8
union all select 9
),
t as (
select Id,
('20'||substr([Date],7,2)||'-'||substr([Date],4,2)||'-'||substr([Date],1,2)) as [Date],
Price
from testtable
),
Dates as (
select Id, date(MinDate,'+'||(d2.n*10+d1.n)||' days') as [Date]
from (
select Id, min([Date]) as MinDate, max([Date]) as MaxDate
from t
group by Id
) q
join Digits d1
join Digits d2
where date(MinDate,'+'||(d2.n*10+d1.n)||' days') <= MaxDate
)
select d.Id,
(substr(d.[Date],9,2)||'.'||substr(d.[Date],6,2)||'.'||substr(d.[Date],3,2)) as [Date],
coalesce(t.Price,0) as Price
from Dates d
left join t on (d.Id = t.Id and d.[Date] = t.[Date])
order by d.Id, d.[Date];
The recursive SQL below was totally inspired by the excellent answer from Gordon Linoff.
And a recursive SQL is probably more performant for this anyway.
(He should get the 15 points for the accepted answer).
The difference in this version is that the datestamps are first formatted to YYYY-MM-DD.
with t as (
select Id,
('20'||substr([Date],7,2)||'-'||substr([Date],4,2)||'-'||substr([Date],1,2)) as [Date],
Price
from testtable
),
cte as (
select Id, min([Date]) as [Date], max([Date]) as MaxDate from t
group by Id
union all
select Id, date([Date], '+1 day'), MaxDate from cte
where [Date] < MaxDate
)
select cte.Id,
(substr(cte.[Date],9,2)||'.'||substr(cte.[Date],6,2)||'.'||substr(cte.[Date],3,2)) as [Date],
coalesce(t.Price, 0) as PricePerDay
from cte
left join t
on (cte.Id = t.Id and cte.[Date] = t.[Date])
order by cte.Id, cte.[Date];

Searching SQL table for two consecutive missing dates

I want to search through a SQL table and find two consecutive missing dates.
For example, person 1 inserts 'diary' entry on day 1 and day 2, misses day 3 and day 4, and enters an entry on day 5.
I am not posting code because I am not sure of how to do this at all.
Thanks!
This uses a LEVEL aggregate to build the list of calendar dates from the first entry to the last, then uses LAG() to check a given date with the previous date, and then checks that neither of those dates had an associated entry to find those two-day gaps:
With diary as (
select to_date('01/01/2016','dd/mm/yyyy') entry_dt from dual union all
select to_date('02/01/2016','dd/mm/yyyy') entry_dt from dual union all
select to_date('04/01/2016','dd/mm/yyyy') entry_dt from dual union all
--leave two day gap of 5th and 6th
select to_date('07/01/2016','dd/mm/yyyy') entry_dt from dual union all
select to_date('08/01/2016','dd/mm/yyyy') entry_dt from dual union all
select to_date('10/01/2016','dd/mm/yyyy') entry_dt from dual )
select calendar_dt -1, calendar_dt
FROM (
select calendar_dt, entry_dt, lag(entry_dt) over (order by calendar_dt) prev_entry_dt
from diary
RIGHT OUTER JOIN (select min(entry_dt) + lvl as calendar_dt
FROM diary
,(select level lvl
from dual connect by level < (select max(entry_dt) - min(entry_dt)+1 from diary))
group by lvl) ON calendar_dt = entry_dt
order by calendar_dt
)
where entry_dt is null and prev_entry_dt is null
returns:
CALENDAR_DT-1, CALENDAR_DT
05/01/2016, 06/01/2016
I am only doing the calendar building to simplify building all 2-day gaps, as if a person took three days off that would be two overlapping two-day gaps (day 1-2, and days 2-3). If you want a far simpler query that outputs the start and end point of any gap of two or more days, then the following works:
With diary as (
select to_date('01/01/2016','dd/mm/yyyy') entry_dt from dual union all
select to_date('02/01/2016','dd/mm/yyyy') entry_dt from dual union all
select to_date('04/01/2016','dd/mm/yyyy') entry_dt from dual union all
select to_date('07/01/2016','dd/mm/yyyy') entry_dt from dual union all
select to_date('08/01/2016','dd/mm/yyyy') entry_dt from dual union all
select to_date('10/01/2016','dd/mm/yyyy') entry_dt from dual )
select prev_entry_dt +1 gap_start, entry_dt -1 gap_end
FROM (
select entry_dt, lag(entry_dt) over (order by entry_dt) prev_entry_dt
from diary
order by entry_dt
) where entry_dt - prev_entry_dt > 2
My high level approach to this problem would be to select from a dynamic table of dates, using an integer counter to add or subtract from the current DateTime to get as many dates as you require into the future or past, then LEFT join your data table to this, order by date and select the first row, or N many rows which have a NULL join.
So your data ends up being
DATE ENTRY_ID
---- -----
2016-01-01 1
2016-01-02 2
2016-01-03 NULL
2016-01-04 3
2016-01-05 4
2016-01-06 NULL
2016-01-07 NULL
2016-01-08 NULL
And you can pick all of the values you need from this dataset
Try this your problem looks like similar to this :-
Declare #temp Table(id int identity(1,1) not null,CDate smalldatetime ,val int)
insert into #temp select '10/2/2012',1
insert into #temp select '10/3/2012',1
insert into #temp select '10/5/2012',1
insert into #temp select '10/7/2012',2
insert into #temp select '10/9/2012',2
insert into #temp select '10/10/2012',2
insert into #temp select '10/13/2012',2
insert into #temp select '10/15/2012',2
DECLARE #startDate DATE= '10/01/2012'
DECLARE #endDate DATE= '10/15/2012'
SELECT t.Id, X.[Date],Val = COALESCE(t.val,0)
FROM
(SELECT [Date] = DATEADD(Day,Number,#startDate)
FROM master..spt_values
WHERE Type='P'
AND DATEADD(day,Number,#startDate) <= #endDate)X
LEFT JOIN #temp t
ON X.[Date] = t.CDate
Alternative you can try this :-
WITH dates AS (
SELECT CAST('2009-01-01' AS DATETIME) 'date'
UNION ALL
SELECT DATEADD(dd, 1, t.date)
FROM dates t
WHERE DATEADD(dd, 1, t.date) <= '2009-02-01')
SELECT t.eventid, d.date
FROM dates d
JOIN TABLE t ON d.date BETWEEN t.startdate AND t.enddate

How to find the missing rows?

I have a table as shown in the image.
The column MONTH_NO should be having months from 1 to 12 for every year. For some years, we missed to load data for some months. I need a query which will fetch the years which doesn't have all the 12 months along with the missing month number.
Please help.
For example -
with mth
as (select level as month_no
from dual
connect by level <= 12),
yrs as (select distinct year from rag_month_dim)
select m.year, m.month_no
from (select year, month_no
from yrs, mth) m,
rag_month_dim r
where m.year = r.year(+)
and m.month_no = r.month_no(+)
group by m.year, m.month_no
having max(r.month_no) is null
order by year, month_no
Try it like this:
post this into an empty query window and adapt to your needs.
MyData contains a "full" year 2013, Sept is missing in 2014 and June and Sept are missing in 2015.
DECLARE #OneToTwelve TABLE(Nmbr INT)
INSERT INTO #OneToTwelve VALUES(1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12);
DECLARE #myData TABLE(yearNo INT, MonthNo INT)
INSERT INTO #myData VALUES
(2013,1),(2013,2),(2013,3),(2013,4),(2013,5),(2013,6),(2013,7),(2013,8),(2013,9),(2013,10),(2013,11),(2013,12)
,(2014,1),(2014,2),(2014,3),(2014,4),(2014,5),(2014,6),(2014,7),(2014,8),(2014,10),(2014,11),(2014,12)
,(2015,1),(2015,2),(2015,3),(2015,4),(2015,5),(2015,7),(2015,8),(2015,10),(2015,11),(2015,12);
WITH AllYears AS
(
SELECT DISTINCT yearNo FROM #myData
)
,AllCombinations AS
(
SELECT *
FROM #OneToTwelve AS months
CROSS JOIN AllYears
)
SELECT *
FROM AllCombinations
LEFT JOIN #myData AS md ON AllCombinations.Nmbr =md.MonthNo AND AllCombinations.yearNo=md.yearNo
WHERE md.MonthNo IS NULL
select distinct year, m.lev
from rag_month_dim a
join
(
select level lev
from dual
connect by level <= 12
) m
on 1=1
minus
select year, month_no
from rag_month_dim
order by 1, 2
select *
from (select count (-1) total, year from rag_month_dim group by year) as table
where total < 12.
you got a year that doesnt have 12 month data and total month record in your data.

Get SQL to process each row 1 by 1

I have been going around for while trying to get an anwswer to my issue, I think it revolves around cursors in SQL but I am not sure. I think I know how to write the loop for a single row of data but I don't know how to run it for all the records:
Hopefully there is an easy answer:
I have a table, let's call it A, that has Product_Code, Start_Date, End_Date and Value
I would need an output table B that has column: Product_Code, Month, Year, Value when Month * Year is in between Start_Date and End_date
Each record of A should then create several record into B. Hope that's fairly clear, I'm happy to elaborate if not! :)
CREATE TABLE YearMonth(
Year int not null,
Month int not null,
FirstDay date not null,
LastDay date not null
);
Fill this table with as many years and months that your range of data is covered (no problem if you have too much).
You could do this with a statement like this:
WITH y(year) AS (
SELECT 2007
union all
SELECT 2008
union all
SELECT 2009
union all
SELECT 2010
union all
SELECT 2011
union all
SELECT 2012
union all
SELECT 2013
union all
SELECT 2014
union all
SELECT 2015
union all
SELECT 2016
),
m(month) AS (
SELECT 1
union all
SELECT 2
union all
SELECT 3
union all
SELECT 4
union all
SELECT 5
union all
SELECT 6
union all
SELECT 7
union all
SELECT 8
union all
SELECT 9
union all
SELECT 10
union all
SELECT 11
union all
SELECT 12
)
INSERT INTO YearMonth(Year, Month, FirstDay, LastDay)
SELECT y.year
,m.month
,convert(date, convert(nvarchar(4), y.year) + '.' + convert(nvarchar(2), m.month) + '.01', 102)
,DateAdd(day, - 1,
CASE WHEN m.month = 12 THEN
convert(date, convert(nvarchar(4), y.year + 1) + '.01.01', 102)
ELSE
convert(date, convert(nvarchar(4), y.year) + '.' + convert(nvarchar(2), m.month + 1) + '.01', 102)
END)
FROM y CROSS JOIN m
The tricky part to calculate the LastDay works like this: create a date that is the first of the following month, then subtract one day from it. This handles the problem that the last day of the month can be 28, 29, 30, or 31.
Then just use a join:
INSERT INTO B(Product_Code, Month, Year, Value)
SELECT A.Product_Code
,YearMonth.Month
,YearMonth.Year
,A.Value
FROM A
JOIN YearMonth ON YearMonth.LastDay <= A.StartDate
AND YearMonth.FirstDay <= A.EndDate
Depending on the exact interpretation of "Month*Year is in between Start_Date and End_date", you might have to switch one or both of the <=s to <.

tsql month problem

I have a table like:
id month cost
------------------
1 Jan 200
1 Mar 204
1 May 200
1 Dec 201
I need an output like( order by month including the other months of a year-displaying all 12 months):
to month cost
------------------
1 Jan 200
NULL Feb NULL
1 Mar 204
....
....
....
1 Dec 201
any idea or solution how to do this in TSQL?
Thanks!
edit:: month is extracted from a datetime value.
in real world i'll have to show previous 12 months from last month in a DESC order! any suggestion for that?
Try building a reference table of months, and JOINing on it. It's the quickest way to do this with months in varchar datatype.
declare #foo table (id int, [mon] varchar(100), cost int)
declare #mon table (mon varchar(100), orderWeight int)
INSERT INTO #mon (mon, orderWeight)
VALUES ('Jan',1), ('Feb',2),('Mar',3),('Apr',4),('May',5),('Jun',6),('Jul',7),
('Aug',8),('Sep',9),('Oct',10),('Nov',11),('Dec',12)
INSERT INTO #foo(id, [mon], cost)
VALUES ( 1 ,'Jan' , 200),
( 1 ,'Mar', 204),
( 1 ,'May' , 200),
( 1 ,'Dec' , 201)
select f.id,
m.[mon] ,
f.cost
from #mon as m
left join #foo as f on m.mon = f.mon
order by m.orderWeight
Results:
Your ordering will now be guaranteed with the order by orderWeight.
Sample table
create table mytable(id int, dt datetime, cost money)
insert mytable values
(1,GETDATE()-10,200),
(1,GETDATE()-40,204),
(1,GETDATE()-100,200),
(1,GETDATE()-200,201);
The query, using SQL Server 2008 specific syntax, and sorted properly
select
t.id [to],
CONVERT(char(3),dateadd(month,-M.N,L.PVT),7) [Month],
sum(t.cost) totalCost
from (select PVT=dateadd(month,datediff(month,0,getdate())-1,0)) L
cross join (values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11)) M(N)
left join mytable t
on t.dt >= dateadd(month,-M.N,L.PVT)
and t.dt < dateadd(month,-M.N+1,L.PVT)
group by t.id, right(CONVERT(char(9),dt,6),6), M.N, L.PVT
order by M.N
What it does:
right(CONVERT(char(9),dt,6),6) converts a date into the format 'DD MMM YY', we only need the MMM YY part
In the SELECT, we further extract only the 3-char month from it, using LEFT( , 3)
The subquery L has a single record and column, PVT, which is the first date of the last month
The number series 0-11 is used to create the month values for the last 12 months, using the formula dateadd(month,-M.N,L.PVT)
The range t.dt >= .. and t.dt < .. finds data for a single month
How about this?
The result contains month and year, but you can strip it as you want.
;with months
as
(
select dateadd(month, -1, dateadd(day, datediff(day, 0, getdate()), 0)) as m
union all
select dateadd(month, -1, m)
from months
where m > dateadd(month, -12, getdate())
)
-- Testdata
,yourTable(id,somedate,cost)
as
(
select 1, '2011-01-03', 200
union all
select 1, '2011-03-06', 204
union all
select 1, '2010-05-09', 200
union all
select 1, '2010-05-19', 201
union all
select 1, '2010-12-02', 201
)
-- end testdata
select yt.id
,datename(month,coalesce(yt.somedate, m.m)) as [month]
,datename(year,coalesce(yt.somedate, m.m)) as [year]
--,yt.cost
,sum(yt.cost) as cost
from months m
left join yourTable yt
on datepart(year, yt.someDate) = DATEPART(year, m.m)
and datepart(month, yt.someDate) = DATEPART(month, m.m)
group by
yt.id
,datename(month,coalesce(yt.somedate, m.m))
,datename(year,coalesce(yt.somedate, m.m))
,m.m
order by m.m desc
Edit: Altered solution to support sum.
Remove the group by-section and alter the comment of cost, to get the old solution.