Dynamic 12 month rolling total for each previous month, going back 12 months - sql

I have the following sql Code (where clause just to limit rows currently)
select
month,
monthname,
year,
count(distinct case when a.dim_service_type_id_desc like '%Direct Payment%' then a.DIM_PERSON_ID else null end) as No_dp,
count(distinct a.DIM_PERSON_ID) as no_ppl
from
SERVICE_PROVISIONS a
inner join date_tbl d on CONVERT(VARCHAR(35),a.start_dttm,112) = d.dim_date_id
where
a.dim_person_id >0
and year = 2018
group by
month,
monthname,
year
my output is this
month monthname year No_dp no_ppl
1 January 2018 142 1604
2 February 2018 111 1526
3 March 2018 133 1636
4 April 2018 1107 3829
5 May 2018 140 1575
6 June 2018 131 1389
7 July 2018 200 893
8 August 2018 2 73
9 September 2018 1 32
10 October 2018 2 21
11 November 2018 2 21
12 December 2018 2 19
So my question is - the customer wants to see how many services were open (using start date and end date) during the previous 12 months (not how many were started, but how many were current and not ended). This is fine when using the current month, however they want to show this also for the previous 12 months as a rolling dynamic figure.
So for example this month in July they want to see how many services were open during the last 12 months. Last month June, they want to see how many services were open during the 12 months previous to June and so on for the previous 12 months.
The table needs to have the month name for the last 12 months and in a column show the number of services that were open in the previous 12 months next to that month.
I hope that makes sense, sorry if it doesn't, feel free to ask questions and I will try to clarify.
The output needs to look something like the current output table, but it is currently only showing how many services were started within that month, which isn't what we want.
The date table is a reference table which has different date formats etc. It can be used or added to if needed.

I've had to make several assumptions about your data. Hopefully the query I'll show in a minute will be easy for you to adjust if any of these are wrong:
I am guessing by its name that start_dttm is a datetime or datetime2 column.
I assume there is a corresponding column called end_dttm that gives the end date/time of a service, and that a null in this column would indicate that a service has not yet ended.
My best guess as to what it means for a service to be "open" in a given month is that it began sometime either within or prior to that month, and has not ended by the time that month is over.
I assume from your original query that multiple services having the same dim_person_id do not represent distinct services.
Since I don't know what's in your date_tbl, I'll show an example that doesn't require it. Consider the following query:
select
BeginDate = dateadd(month, -1, dateadd(day, 1, eomonth(getdate(), -Offset.X))),
EndDate = dateadd(day, 1, eomonth(getdate(), -Offset.X))
from
(values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11)) Offset(X)
This will give you 12 records, representing the current month and each of the 11 preceding months. Note that my EndDate here is not actually the last day of the month, but the first day of the following month. I've done this because of assumption 1 above; since your service dates may have time components, I'll determine whether they fall in a given month by checking if their dates are strictly earlier than the start of the following month. Here's what that query gives me:
BeginDate EndDate
2018-07-01 2018-08-01
2018-06-01 2018-07-01
2018-05-01 2018-06-01
2018-04-01 2018-05-01
2018-03-01 2018-04-01
2018-02-01 2018-03-01
2018-01-01 2018-02-01
2017-12-01 2018-01-01
2017-11-01 2017-12-01
2017-10-01 2017-11-01
2017-09-01 2017-10-01
2017-08-01 2017-09-01
Now I'll join the above result set to your SERVICE_PROVISIONS data, looking for records in each month that have dim_person_id > 0 (from your original query) and which satisfy assumption 3 above.
-- Some sample data (assumptions 1 & 2)
declare #SERVICE_PROVISIONS table (dim_person_id bigint, start_dttm datetime, end_dttm datetime);
insert #SERVICE_PROVISIONS values
(1, '20180101', '20180315'),
(1, '20180101', '20180315'),
(2, '20171215', '20180520');
-- The CTE defines the months we'll report on, as described earlier.
with MonthsCTE as
(
select
BeginDate = dateadd(month, -1, dateadd(day, 1, eomonth(getdate(), -Offset.X))),
EndDate = dateadd(day, 1, eomonth(getdate(), -Offset.X))
from
(values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11)) Offset(X)
)
-- This query matches the months from the CTE against the applicable services.
select
[Month] = datepart(month, M.BeginDate),
[MonthName] = datename(month, M.BeginDate),
[Year] = datepart(year, M.BeginDate),
ServicesOpen = count(distinct S.dim_person_id) -- Assumption 4
from
MonthsCTE M
left join #SERVICE_PROVISIONS S on
S.dim_person_id > 0 and
S.start_dttm < M.EndDate and -- Assumption 3
(
S.end_dttm >= M.EndDate or
S.end_dttm is null -- Assumption 2
)
group by
M.BeginDate,
M.EndDate
order by
M.BeginDate;
Note that I moved the dim_person_id > 0 from the WHERE clause to the JOIN so that each of the 12 months will still appear in the result set even if there were no services open during that time. Results:
Month MonthName Year ServicesOpen
8 August 2017 0
9 September 2017 0
10 October 2017 0
11 November 2017 0
12 December 2017 1
1 January 2018 2
2 February 2018 2
3 March 2018 1
4 April 2018 1
5 May 2018 0
6 June 2018 0
7 July 2018 0

something a bit like this - if you can write a query to get the value you want for a row in your ootput, then use cross apply to link to that query. Counting records that have an open record before the month, but no close record before the month seems feasible
SELECT IQ. *, OA.SERVICE_PROVISIONS FROM (select
month,
monthname,
year,
a.dim_person_id dim_person_id,
count(distinct case when a.dim_service_type_id_desc like '%Direct Payment%' then a.DIM_PERSON_ID else null end) as No_dp,
count(distinct a.DIM_PERSON_ID) as no_ppl
from
SERVICE_PROVISIONS a
inner join date_tbl d on CONVERT(VARCHAR(35),a.start_dttm,112) = d.dim_date_id
where
a.dim_person_id >0
and year = 2018
group by
month,
monthname,
year) IQ
CROSS APPLY
(SELECT count(0) OpenThings FROM SERVICE_PROVISIONS SP1 WHERE
(sp1.startdate < DATEFROMPARTS(IQ.year,iq.month,1)
AND
sp1.enddate is null or sp1.enddate > DATEFROMPARTS(IQ.year,iq.month,1)) and sp1.dim_person_id = iq.dim_person_id
) AS OA

Related

Can I calculate an aggregate duration over multiple rows with a single row per day?

I'm creating an Absence Report for HR. The Absence Data is stored in the database as a single row per day (the columns are EmployeeId, Absence Date, Duration). So if I'm off work from Tuesday 11 February 2020 to Friday 21 February 2020 inclusive, there will be 9 rows in the table:
11 February 2020 - 1 day
12 February 2020 - 1 day
13 February 2020 - 1 day
14 February 2020 - 1 day
17 February 2020 - 1 day
18 February 2020 - 1 day
19 February 2020 - 1 day
20 February 2020 - 1 day
21 February 2020 - 1 day
(see screenshot below)
HR would like to see a single entry in the report for a contiguous period of absence:
My question is - without using a cursor, how can I calculate the is in SQL (even more complicated because I have to do this using Linq to SQL, but I might be able to swap this out for a stored procedure. Note that the criterion for contiguous data is adjacent working days EXCLUDING weekends and bank holidays. I hope I've made myself clear ... apologies if not.
This is a form of gaps-and-islands. In this case, use lag() to see if two vacations overlap and then a cumulative sum:
select employee, min(absent_from), max(absent_to)
from (select t.*,
sum(case when prev_absent_to = dateadd(day, -1, absent_from) then 0 else 1
end) over (partition by employee order by absent_to) as grp
from (select t.*,
lag(absent_to) over (partition by employee order by absent_from) as prev_absent_to
from t
) t
) t
group by employee, grp;
If you need to deal with holidays and weekends, then you need a calendar table.

SQL Carryover from previous month

I have some data that I am trying to get some counts on. There are dates for when the record was entered and when it was closed, if it has been closed yet. I want to be able to get a count of how many records were still open from the previous month as of the first of the month. Here is an example. First table is the data, second table is the results I am looking for. In the second table, ignore the parenthesis, they are just the IDs of the records that make up that count.
Position DateEntered DateClosed
1 12/15/2017 12/20/2017
11 12/20/2017 1/7/2018
2 1/23/2018 2/3/2018
3 1/24/2018
4 2/15/2018
5 2/20/2018 5/16/2018
6 3/3/2018 3/15/2018
7 3/23/2018 4/12/2018
8 4/11/2018 5/10/2018
9 4/12/2018 4/25/2018
10 5/4/2018
Year Month Carried Over
2018 January 1 (11)
2018 February 2 (2,3)
2018 March 3 (3,4,5)
2018 April 4 (3,4,5,7)
2018 May 4 (3,4,5,8)
2018 June 3 (3,4,10)
2018 July 3 (3,4,10)
2018 August 3 (3,4,10)
Is this possible, and if so, how? Been racking my brain on this one for a few hours.
For each month, you want the number of rows that start before that month and end after. I'm thinking:
with dates as (
select cast('2018-01-01' as date) as dte
union all
select dateadd(month, 1, dte)
from dates
where dte < '2018-08-01'
)
select d.dte,
(select count(*)
from t
where t.dateentered < d.dte and
(t.dateclosed > d.dte or t.dateClosed is null)
) as carriedover
from dates d;
Note that this puts the date in a single column, rather than splitting the year and month into separate columns. That is easily arranged, but I prefer to keep date components together.

T-SQL Running Monthly Totals Including Missing Months

I know this sounds really simple but I just cannot seem to get my head around it.
I have a temporary table that holds for example, Handler, MonthName, MonthNumber and MTD, which is a total for that month. What I need to do with that data is then create a running total for each Handler, from April to March. Now, here is the bit I am struggling with. Not all Handlers will have data for all months.
For example.
Handler MonthName MonthNo MTD
Julian Slaughter April 1 10000
Julian Slaughter June 3 12000
Julian Slaughter July 4 10000
Julian Slaughter September 6 12000
Bob Monkhouse April 1 5000
Bob Monkhouse July 4 5000
So I want the results to look like this
Julian Slaughter April 1 10000
Julian Slaughter May 2 10000
Julian Slaughter June 3 22000
Julian Slaughter July 4 32000
Julian Slaughter August 5 32000
Julian Slaughter September 6 44000
...and so on until March
Bob Monkhouse April 1 5000
Bob Monkhouse May 2 5000
Bob Monkhouse June 3 5000
Bob Monkhouse July 4 10000
...and so on until March
I have tried LEFT JOIN onto a table of the Month Names\Numbers and I have had an attempt at
OVER(PARTITION ..... ORDER BY ..... RANGE\ROWS)
but can't get the missing months.
Thanks in advance, sorry for the poor formatting, not sure how to do tables on here.
EDIT - Here is my LEFT JOIN attempt
SELECT
Months.MonthNo,
Department,
Executive,
#8.MonthNo,
MTD = SUM([TY MTD Prem]) OVER (PARTITION BY Department, Executive, [Exec Code] ORDER BY #8.MonthNo RANGE UNBOUNDED PRECEDING)
FROM Months
LEFT JOIN #8 ON Months.MonthNo = #8.MonthNo
For one Executive, I only get 4 rows, not the 12 I need. Can't show you the results for Data Protection purposes.
DECLARE #start_date date, #end_date date
SELECT #start_date='2012-04-01',#end_date='2013-03-31'
;WITH xo AS
(
SELECT #start_date AS cte_start_date
UNION ALL
SELECT DATEADD(MONTH, 1, cte_start_date)
FROM xo
WHERE DATEADD(MONTH, 1, cte_start_date) <= #end_date
), x as (
select *,row_number() over (order by cte_start_date) monthno
from xo
)
, y as (
select distinct handler from test
)
SELECT y.handler, datename(mm,x.cte_start_date), x.monthno
,(select sum(mtd) from test a where a.handler=y.handler and a.monthno<=x.monthno) mtd
FROM y
cross join x
order by 1,3
see example on SQLFiddle http://sqlfiddle.com/#!3/7d483/15
Sorry for the delay. The proposed solution worked a treat. I had to use the same code several times in various other parts of my giant query but it worked great.

SQL Query to provide a count of jobs that were open grouped by week, month, year for the last 12 months

Given the following data:
Job CreatedDate ClosedDate
ID6 2014-06-04 01:51:47.060 NULL
ID7 2014-06-05 00:25:35.187 NULL
ID43 2014-06-16 05:17:18.803 2014-06-26 15:00:15.190
ID72 2014-06-20 04:00:07.733 2014-06-20 04:12:18.770
ID84 2014-06-27 16:01:18.953 NULL
ID74 2014-06-20 04:05:42.843 NULL
ID68 2014-06-20 03:46:52.653 2014-06-20 03:52:47.540
ID88 2014-07-03 03:47:55.407 NULL
ID64 2014-06-19 07:29:37.060 NULL
ID104 2014-07-08 02:59:58.337 2014-07-15 15:00:15.543
ID106 2014-07-08 03:02:29.710 2014-07-16 11:04:19.230
ID130 2014-07-10 04:30:20.900 NULL
ID132 2014-07-10 04:32:20.243 NULL
ID150 2014-07-15 20:59:06.077 2014-07-15 21:10:19.490
.
.
.
Etc.
I need to write a query that will group records by week, month, and year. A count of the number of records that were in an Open state at the end of each week (Sunday) needs to be returned.
The difficulty arises in determining if a job was in an Open state at the end of a particular week. I imagine the logic goes something like this: A job was open if the CreatedDate is less than or equal to the end of the week and the ClosedDate is after the end of the week or ClosedDate is NULL. A job can remain open over many weeks.
The output should look as follows.
Week Month Year Count
23 6 2014 80
24 6 2014 36
25 6 2014 71
26 6 2014 0
27 7 2014 25
28 7 2014 180
So, at the end of week 23 there were 80 jobs that were still open. Where a week has no records the count should return 0. Only records that were created 12 months from the time the query is executed should be returned. The query is executed every week and used to build a rolling 12 month report. I am using Microsoft SQL Server 2012.
I’ll give an example:
At the beginning of week 1 there are no jobs in the table. During week 1, 50 jobs are opened and 30 are closed which means that 20 are in an Open state at the end of the week.
During week 2, a further 50 jobs are opened and 40 are closed which means that 30 are in an Open state (when you add the 20 from last week and the 10 from this week) at the end of the week. Some of the jobs closed in week 2 were created in week 1 and some of the open jobs are still from week 1.
During week 3, a further 50 jobs are opened and 40 are closed which means that 40 are in an Open state at the end of the week. Some of the jobs closed in week 3 were created in weeks 1 and 2 and some of the open jobs are still from weeks 1 and 2.
And so on for a 12 month period.
When I run the query at the end of week 3 I want to know that at the end of week 1 there were 20 jobs open and at the end of week 2 there were 30 and at the end of week 3 there were 40. You can’t rely on the ClosedDate being NULL because even thought it was at the end of week 1 it may have been closed during week 3 and now has a ClosedDate. As I mentioned above, I think the logic that needs to be used is: A job was in the Open state at the end of a week if the CreatedDate is less than or equal to the end of the week and the ClosedDate is after the end of the week or ClosedDate is NULL.
So, at the end of week 3, when the query is run, the data that I would like to receive would look like:
Week Month Year Count
1 1 2014 20
2 1 2014 30
3 1 2014 40
A manual way of achieving what I need is to run a simple query at the end of each week that counts the number of jobs that have a ClosedDate of NULL. I then manually enter the week, month, year, and count into a table and use that table as a source for reporting. I was hoping that a more automated process could be achieved.
Thanks in advance
Hope this is what you are looking for..
DECLARE #S INT = (SELECT CAST(CAST(MIN([CREATEDDATE]) AS DATETIME) AS INT) FROM [DBO].[YOURTABLE] WHERE [CLOSEDDATE] IS NULL)
DECLARE #E INT = (SELECT CAST(CAST(MAX([CREATEDDATE]) AS DATETIME) AS INT) FROM [DBO].[YOURTABLE] WHERE [CLOSEDDATE] IS NULL)
DECLARE #TAB TABLE (ID INT PRIMARY KEY IDENTITY(1,1), DT DATETIME)
WHILE #S <= #E
BEGIN
INSERT INTO #TAB
SELECT CAST(#S AS DATETIME)
SET #S = #S + 1
END
SELECT MAIN.WEEK,Main.MONTH,Main.YEAR,
LU.COUNT
FROM (
SELECT DATEPART(YEAR,DT) YEAR,
DATEPART(MONTH,DT) MONTH,
DATEPART(WEEK,DT) WEEK
FROM #TAB
GROUP BY DATEPART(YEAR,DT),
DATEPART(MONTH,DT),
DATEPART(WEEK,DT)) MAIN
LEFT JOIN
(
SELECT DATEPART(YEAR,[CREATEDDATE]) YEAR,
DATEPART(MONTH,[CREATEDDATE]) MONTH,
DATEPART(WEEK,[CREATEDDATE]) WEEK,
COUNT(*) COUNT
FROM [DBO].[YOURTABLE]
WHERE [CLOSEDDATE] IS NULL
GROUP BY DATEPART(YEAR,[CREATEDDATE]),
DATEPART(MONTH,[CREATEDDATE]) ,
DATEPART(WEEK,[CREATEDDATE])) LU
ON MAIN.YEAR = LU.YEAR
AND MAIN.MONTH = LU.MONTH
AND MAIN.WEEK = LU.WEEK
ORDER BY MAIN.YEAR,
MAIN.MONTH,
MAIN.WEEK

Getting a variable end of year date and value from MS Access table using SQL

I have some data is that is daily (day on day) closing figures for a tracked supply and is in one MS Access table that has 2 columns - Dates (the date), PXLast(the day's closing figure)).
I have daily data from Jan 1991 to Aug 2013 and I wanted to get the percentage change of PXLast at every year end compared to last year year end as follows:
Year | Percentage Change of PXLast(Year on Year)
1991 | 15.2%
1992 | 9.2%
The year end date varies (not always 31st ) and I am going about getting the last PXLast value by:
1.Get the max date in Dec every year: results in MyYear, MyMonth, MyDay
2.Combine it using DateSerial(MyYear, MyMonth, MyDay)
3.Join the resulting query to the table and inner join on the date column
4.Get the PXLast value
SELECT EndDates.EndDates, NSE20.PX_LAST AS LookPoint
FROM NSE20 INNER JOIN
(SELECT DateSerial([MyYear],[MyMonth],[MyDay])
AS EndDates FROM (SELECT 12 AS MyMonth, MyDay, MyYear FROM
(SELECT Max(Day([Dates])) AS MyDay, Year([Dates]) AS MyYear
FROM NSE20 WHERE (((Month([Dates]))=12))
GROUP BY Year([Dates])) AS EndYearValues)
AS EndValueDates)
AS EndDates ON NSE20.Dates = EndDates.EndDates;
Could anyone assist me get the corresponding value using a query for previous year end
eg for 29 Dec 2006, it should show the current value and show the value for 31 Dec 2005
in the same row ie
Year | Current Year End| Previous Year End
2005 | 3449.00 | 4611.19
2006 | 9.2% |3449.00
Any help is appreciated.
Any suggestions to a better way of doing this is very very welcome....
Let's assume that you have some test data in a table named [NSE20] that looks like this
Dates PXLast
---------- ------
2010-07-01 131
2010-12-31 130
2011-11-12 123
2011-12-30 125
2012-01-03 127
2012-12-31 129
I'd start by creating a saved query in Access named [NSE20_year_ends] that identifies the year-end dates by (calendar) year:
SELECT Year(Dates) AS CalendarYear, Max(Dates) AS YearEndDate
FROM NSE20
GROUP BY Year(Dates)
That will produce
CalendarYear YearEndDate
------------ -----------
2010 2010-12-31
2011 2011-12-30
2012 2012-12-31
Then I'd create another saved query named [NSE20_year_end_balances] to extract the closing balances for each year:
SELECT NSE20_year_ends.CalendarYear, NSE20.PXLast
FROM
NSE20
INNER JOIN
NSE20_year_ends
ON NSE20.Dates = NSE20_year_ends.YearEndDate
That will give us
CalendarYear PXLast
------------ ------
2010 130
2011 125
2012 129
Now we can do a self-join on that query to calculate the percentage change
SELECT
y1.CalendarYear,
(y1.PXLast - y0.PXLast) / y0.PXLast * 100 AS PctChange
FROM
NSE20_year_end_balances y1
INNER JOIN
NSE20_year_end_balances y0
ON y0.CalendarYear = y1.CalendarYear - 1
resulting in
CalendarYear PctChange
------------ -----------------
2011 -3.84615384615385
2012 3.2