How do I include months that have no data? - sql

I am trying to create a report that shows how many training records will expire within a chosen date range however when I run the report it excludes months that have no training records going out of date. I have tried various solutions I've seen posted but I haven't been able to get any of them to work in my case.
This is my query:
SELECT COUNT(ISNULL(TRAININGRECORDID, 0)) AS NUMBEROFRECORDS
,DEPARTMENTNUMBER
,DATENAME( Month, EXPIRY ) + '-' + DATENAME( Year, EXPIRY ) AS [MONTHYEAR]
FROM Training_Records TR
JOIN Departments TD ON TR.DEPARTMENTID = TD.DEPARTMENTID
WHERE TR.EXPIRY IS NOT NULL
AND TD.DEPARTMENTNUMBER IN (#DEPTNO)
AND TR.EXPIRY BETWEEN #StartDate AND #EndDate
GROUP BY TD.DEPARTMENTNUMBER, DATENAME(Year, TR.EXPIRY), DATENAME(Month, TR.EXPIRY)
ORDER BY TD.DEPARTMENTNUMBER, [MONTHYEAR]
An example of results from this query looks like this:
NUMBEROFRECORDS DEPARTMENTNUMBER MONTHYEAR
1 21 April-2023
4 23 June-2023
1 83 August-2023
I am displaying the results of this query in a matrix with MONTHYEAR as the columns. In the example above the report will display April, June and August 2023 but will skip over the months May, July 2023 because there are no records going out of date in those months but despite that I still want them displayed in my report/returned in my query.
I've tried various solutions I've found on here but none of them have worked for me. How would I go about including these months with no records going out of date?

You need to first get all of the months, and then outer join to them (not using BETWEEN). Here is an example that gets April, May, June, and July, and then shows how you would outer join that against your table.
DECLARE #StartDate date = '20220405',
#EndDate date = '20220708';
;WITH Months(TheMonth) AS
(
SELECT DATEFROMPARTS(YEAR(#StartDate), MONTH(#StartDate), 1)
UNION ALL
SELECT DATEADD(MONTH, 1, TheMonth)
FROM Months
WHERE TheMonth < DATEFROMPARTS(YEAR(#EndDate), MONTH(#EndDate), 1)
)
SELECT TheMonth -- , COALESCE(SUM({your table}.{column}),0)
FROM Months AS m
-- LEFT OUTER JOIN {your table}
-- ON {your table}.{date column} >= m.TheMonth
-- AND {your table}.{date column} < DATEADD(MONTH, 1, m.TheMonth);
Output:
TheMonth
2022-04-01
2022-05-01
2022-06-01
2022-07-01
Example db<>fiddle
If your range could last more than 100 months, you'll need to add:
OPTION (MAXRECURSION 0);

Related

How to setup a cumulative count grouped by month with underlying conditions

I've come across somewhat of an interesting scenario where I'm needing to aggregate enrollment counts and group them by the individual month and all subsequent months leading up to the completion date. The starting counter will be placed into the month when the enrollment began, and now I'm needing to set up a cumulative sum to carry out the single count.
Here's a couple of test records I'm working with
I've set up the following query to compile the date_month CTE to compile the full 12 months derived from my Start/End Range variables. I've then joined it to my test table in order to establish the Counter placements.
DECLARE #EnrollmentDateStart DATETIME = '2020-01-01'
DECLARE #EnrollmentDateEnd DATETIME = '2020-12-01'
;WITH CTE_Months(year_month) AS
(
SELECT DATEADD(MONTH, n, DATEADD(MONTH, DATEDIFF(MONTH, 0, #EnrollmentDateStart), 0))
FROM ( SELECT TOP (DATEDIFF(MONTH, #EnrollmentDateStart, #EnrollmentDateEnd) + 1)
n = ROW_NUMBER() OVER (ORDER BY [object_id]) - 1
FROM sys.all_objects ORDER BY [object_id] ) AS n
)
SELECT
[Year] = YEAR(cm.year_month),
[Month] = DATENAME(MONTH, cm.year_month),
SUM(IIF(tt.[Enrollment Start Date] >= #EnrollmentDateStart,1,0)) AS EnrollmentCount
FROM CTE_Months cm
LEFT OUTER JOIN #TMP_Testing_Table tt
ON tt.[Enrollment Start Date] >= cm.year_month
AND tt.[Enrollment Start Date] < DATEADD(MONTH, 1, cm.year_month)
GROUP BY tt.Department, cm.year_month
At this stage, I'm pulling back the following results, so I now have the Enrollment Counts placed into the correct starting months derived from the Enrollment Start Date.
Now I'm trying to figure out what would be the best course of action to place the subsequent count for the additional months leading up to the Completion date?
For example - The first User (UserId: 1) was Enrolled in March, 2020, and Completed in August, 2020, so essentially I'm looking to produce the following result to reflect the number of months ranging between March <> July (Last month prior to Completion)
January: 0
February: 0
March: 1
April: 1
May: 1
June: 1
July: 1
August: 0
September: 0
October: 0
November: 0
December: 0
Thinking a cumulative total should be able to address the subsequent for the month by month range, however, I would then need to zero out the total for all subsequent months on and after the recorded Completion date for this record in question.
Seeing if I can get your thoughts/suggestions on how to address this scenario? Apologies if the information/explanation is confusing, but please let me know, and I'll do my best to elaborate.
....................
SELECT
[Year] = YEAR(cm.year_month),
[Month] = DATENAME(MONTH, cm.year_month),
count(tt.userid) AS EnrollmentCount
FROM CTE_Months cm
LEFT OUTER JOIN #TMP_Testing_Table tt on cm.year_month > eomonth([Enrollment Start Date], -1)
and cm.year_month <= tt.[Enrollment End Date]
GROUP BY cm.year_month

Data appear at least once for every month in the last X month

My problem:
Table: trans_detail:
PhoneNo | Datetime
01234 | 2013-01-05 20:40:10
01245 | 2013-04-02 21:00:13
05678 | 2013-04-16 01:24:07
04567 | 2013-07-23 07:00:00
etc | etc
I want to get all phoneNo that appears at least once for every month in the last X month (X month can be any month between 1-12).
For example: get all phone no. that appears at least once for Every Month in the last 3 months.
I am using SQL Server 2005.
Here is a quick query that comes close to what you want:
select PhoneNo
from trans_detail d
where d.datetime >= dateadd(mm, -#X, getdate())
group by PhoneNo
having count(distinct year(datetime)*12+month(datetime)) = #X
The where clause filters the data to only include rows in the last #X months. the having clause checks that each month is in the data, by counting the number of distinct months.
The above version of the query assumes that you mean calendar months. So, it has boundary condition problems. If you run it on June 16th, then it looks back one month and makes sure that the phone number appears at least once since May 16th. I am unclear on whether you want to insist that the number appear twice (once in May and once in June) or if once (once during the time period). The solution to this is to move the current date back to the end of the previous month:
select PhoneNo
from trans_detail d cross join
(select cast(getdate() - day(getdate) + 1 as date) as FirstOfMonth const
where d.datetime >= dateadd(mm, -#X, FirstOfMonth) and
d.datetime < FirstOfMonth
group by PhoneNo
having count(distinct year(datetime)*12+month(datetime)) = #X
Here it is. First two CTEs are to find and prepare last X months, third CTE is to group your data by phones and months. At the end just join the two and return where number of matching rows are equal to number of months.
DECLARE #months INT
SET #Months = 3
;WITH CTE_Dates AS
(
SELECT GETDATE() AS Dt
UNION ALL
SELECT DATEADD(MM,-1,Dt) FROM CTE_Dates
WHERE DATEDIFF(MM, Dt,GETDATE()) < #months-1
)
, CTE_Months AS
(
SELECT MONTH(Dt) AS Mn, YEAR(Dt) AS Yr FROM CTE_Dates
)
, CTE_Trans AS
(
SELECT PhoneNo, MONTH([Datetime]) AS Mn, YEAR([Datetime]) AS Yr FROM dbo.trans_detail
GROUP BY PhoneNo, MONTH([Datetime]), YEAR([Datetime])
)
SELECT PhoneNo FROM CTE_Months m
LEFT JOIN CTE_Trans t ON m.Mn = t.Mn AND m.Yr = t.Yr
GROUP BY PhoneNo
HAVING COUNT(*) = #months
SQLFiddle Demo - with added some more data that will match for last 3 months

Fiscal Year To-Date in Where Clause (T-SQL)

Company's Fiscal Year: July 1 - June 30
I have a query where I am trying to capture aggregate # of units and $ revenue by product and cost center for the fiscal year-to-date. It will run on the 1st of the month and look through the last day of the previous month. Fiscal year does not appear in the report - it is criteria.
Mix of pseudocode and SQL:
Where
If datepart(mm,getdate()) - 1 < 7
THEN
transaction_post_date BETWEEN 7/1/ previous year AND dateadd(day,-(day(getdate()),getdate())
Else
transaction_post_date BETWEEN 7/1/current year AND dateadd(day,-(day(getdate()),getdate())
Am I on the write track? How do I write the SQL for a specific date on a year that depends on SQL - 7/1/current year?
I am weak using variables and do not even know if I have access to create them on the SQL Server DB, which is rather locked down. Definitely can't create a function. (I'm a business analyst.)
UPDATE, Fiscal year goes forward, so July 1, 2010, is Fiscal Year 2011.
I think this works:
Year(dateadd(month,6,htx.tx_post_date)) = Year(DateAdd(Month, 5, GetDate()))
Feeback?
And now I've been asked to add Fiscal Year-To-Date fields for quantity and revenue to the following query which gave me totals for
Select
inv.ITEM_CODE
, inventory.ITEM_NAME
, cc.COST_CENTER_CODE
, tx.REV_CODE_ID
, tx.PRICE
, tx.ITEM_SALE_ID
, sum(tx.quantity)
, sum(tx.amount)
from
transactions tx
inner join inventory inv on inv.item_id = tx.item_id
left outer join cost_center cc on cc.cost_center_id = tx.cost_center_id
where
DATEPART(mm, tx.tx_date) = DATEPART(mm,dateadd(m,-1,getdate()))
and DATEPART(yyyy, tx.tx_date) = DATEPART(yyyy,dateadd(m,-1,getdate()))
group by
inv.ITEM_CODE
, inventory.ITEM_NAME
, cc.COST_CENTER_CODE
, tx.REV_CODE_ID
, tx.PRICE
, tx.ITEM_SALE_ID
I need to add the fiscal year-to-date quantity and and amount columns to this report. Would a correlated subquery by the way to go? Would the joins be tricky? I've never used a subquery with an aggregation/grouping query.
Thanks for all the previous help.
Here is how I would do it if I needed to group by Fiscal Year:
Group by Year(DateAdd(Month, -6, TransactionDate))
May be not exactly it, but you get the idea.
I would add a calculated column to your table called FiscalYear (with the proper calculation) and select based on that column
I believe the easiest way is to do this in two steps. Use the WHERE Clause to filter your YTD and then a GROUP BY to group by FY. Since your FY begins in July(7) then increment the FY if the month is greater than June(6).
WHERE CLAUSE:
WHERE
DATEDIFF(DAY, transaction_post_date, Cast(Month(GetDate()) as varchar) +
'/' + Cast(Day(GetDate()) as varchar) + '/' + CAST(Case WHEN
MONTH(transaction_post_date) > 6 then YEAR(transaction_post_date) + 1 else
Year(transaction_post_date) end as varchar)) >=0
GROUP BY CLAUSE:
GROUP BY CASE WHEN MONTH(transaction_post_date) > 6 then
Year(transaction_post_date) + 1 else YEAR(transaction_post_date) end

February 29 comparing to February 28 of previous year

I have a stored procedure that pulls data and joins last year's data on the Date. The problem is that the current year data has nothing to join on because there was no February 29th in 2011. Has anyone else experienced this problem? Anyone have any ideas on how to work around it?
Here is the stored procedure:
SELECT
--b.Date_Rep AS Date_Rep,
SUM(b.AccountsCreatedThisYear) AS AccountsCreatedThisYearTot,
SUM(a.AccountsCreatedThisYear) AS AccountsCreatedLastYearTot,
FROM Report2011.dbo.T_Report_01 b WITH (NOLOCK) --This year
LEFT JOIN Report2011.dbo.T_Report_01 a WITH (NOLOCK) ON DATEADD(yyyy,-1,b.Date_Rep) = a.date_rep --Last year
WHERE (a.Date_Rep BETWEEN DATEADD(year, -1,#StartDate) AND DATEADD(year, -1,#EndDate))
For starters, I wouldn't do a where clause on columns from a table on the outer side of a left join. Try this, instead:
SELECT SUM(b.AccountsCreatedThisYear) AS AccountsCreatedThisYearTot,
SUM(a.AccountsCreatedThisYear) AS AccountsCreatedLastYearTot,
FROM Report2011.dbo.T_Report_01 b WITH (NOLOCK) --This year
LEFT JOIN Report2011.dbo.T_Report_01 a WITH (NOLOCK)
ON DATEADD(yyyy,-1,b.Date_Rep) = a.date_rep --Last year
WHERE b.Date_Rep BETWEEN #StartDate AND #EndDate
Try using a FULL OUTER JOIN instead of a LEFT JOIN and use COALESCE:
SUM(COALESCE(b.AccountsCreatedThisYear, 0)) AS AccountsCreatedThisYearTot and
SUM(COALESCE(a.AccountsCreatedThisYear, 0)) AS AccountsCreatedLastYearTot
so you avoid the NULL's when the dates don't match.
declare #29Feb datetime = convert(datetime,'2012/02/29')
declare #28Feb datetime = convert(datetime,'2012/02/28')
select case when
dateadd(yy,-1,#29Feb) = dateadd(yy,-1,#28Feb)
then 1
else 0 end
This select statement outputs 1, so actually the Feb 29 and Feb 28 dates have only one corresponding date in the past year, Feb 28.
Now you are doing a sum for two periods in different years for which it happens that the first period has 1 day less than the current period.
How would someone answer to the following question:
"How many accounts have been created in the last year's February and how many this year?"
Does it matter the fact one February had 28 days and the other 29? I don't believe so, the reference is February, not the days.
So I see two problems with this query:
There might be a day this year when no accounts have been created, but in the last year, in the same period, some accounts did, so the left join doesn't catch the ones in the last year
For this year for two different dates, 28 and 29 correspond only one day, 28 so this one is summed twice.
SELECT (SUM(b.AccountsCreatedThisYear)
FROM Report2011.dbo.T_Report_01 WITH (NOLOCK) --This year
WHERE Date_Rep BETWEEN #StartDate and #EndDate ) as AccountsCreatedThisYearTot,
(SUM(b.AccountsCreatedThisYear)
FROM Report2011.dbo.T_Report_01 WITH (NOLOCK) -- Last Year
WHERE Date_Rep BETWEEN DATEADD(year, -1,#StartDate) AND DATEADD(year, -1,#EndDate)) as AccountsCreatedLastYearTot
There was no Feb 29 last year but March 1 was 365 days ago and Feb 28 was 366 days ago.
SELECT SUM(b.AccountsCreatedThisYear) AS AccountsCreatedThisYearTot,
SUM(a.AccountsCreatedThisYear) AS AccountsCreatedLastYearTot,
FROM Report2011.dbo.T_Report_01 b WITH (NOLOCK) --This year
LEFT JOIN Report2011.dbo.T_Report_01 a WITH (NOLOCK)
ON DATEADD(dd,-365,b.Date_Rep) = a.date_rep --Last year
WHERE b.Date_Rep BETWEEN #StartDate AND #EndDate
+1 Mark Bannister as I used his syntax

Calculating Open incidents per month

We have Incidents in our system with Start Time and Finish Time and project name (and other info) .
We would like to have report: How many Incidents has 'open' status per month per project.
Open status mean: Not finished.
If incident is created in December 2009 and closed in March 2010, then it should be included in December 2009, January and February of 2010.
Needed structure should be like this:
Project Year Month Count
------- ------ ------- -------
Test 2009 December 2
Test 2010 January 10
Test 2010 February 12
....
In SQL Server:
SELECT
Project,
Year = YEAR(TimeWhenStillOpen),
Month = DATENAME(month, MONTH(TimeWhenStillOpen)),
Count = COUNT(*)
FROM (
SELECT
i.Project,
i.Incident,
TimeWhenStillOpen = DATEADD(month, v.number, i.StartTime)
FROM (
SELECT
Project,
Incident,
StartTime,
FinishTime = ISNULL(FinishTime, GETDATE()),
MonthDiff = DATEDIFF(month, StartTime, ISNULL(FinishTime, GETDATE()))
FROM Incidents
) i
INNER JOIN master..spt_values v ON v.type = 'P'
AND v.number BETWEEN 0 AND MonthDiff - 1
) s
GROUP BY Project, YEAR(TimeWhenStillOpen), MONTH(TimeWhenStillOpen)
ORDER BY Project, YEAR(TimeWhenStillOpen), MONTH(TimeWhenStillOpen)
Briefly, how it works:
The most inner subselect, that works directly on the Incidents table, simply kind of 'normalises' the table (replaces NULL finish times with the current time) and adds a month difference column, MonthDiff. If there can be no NULLs in your case, just remove the ISNULL expression accordingly.
The outer subselect uses MonthDiff to break up the time range into a series of timestamps corresponding to the months where the incident was still open, i.e. the FinishTime month is not included. A system table called master..spt_values is also employed there as a ready-made numbers table.
Lastly, the main select is only left with the task of grouping the data.
A useful technique here is to create either a table of "all" dates (clearly that would be infinite so I mean a sufficiently large range for your purposes) OR create two tables: one of all the months (12 rows) and another of "all" years.
Let's assume you go for the 1st of these:
create table all_dates (d date)
and populate as appropriate. I'm going to define your incident table as follows
create table incident
(
incident_id int not null,
project_id int not null,
start_date date not null,
end_date date null
)
I'm not sure what RDBMS you are using and date functions vary a lot between them so the next bit may need adjusting for your needs.
select
project_id,
datepart(yy, all_dates.d) as "year",
datepart(mm, all_dates.d) as "month",
count(*) as "count"
from
incident,
all_dates
where
incident.start_date <= all_dates.d and
(incident.end_date >= all_dates.d or incident.end_date is null)
group by
project_id,
datepart(yy, all_dates.d) year,
datepart(mm, all_dates.d) month
That is not going to quite work as we want as the counts will be for every day that the incident was open in each month. To fix this we either need to use a subquery or a temporary table and that really depends on the RDBMS...
Another problem with it is that, for open incidents it will show them against all future months in your all_dates table. adding a all_dates.d <= today solves that. Again, different RDBMSs have different methods of giving back now/today/systemtime...
Another approach is to have an all_months rather than all_dates table that just has the date of first of the month in it:
create table all_months (first_of_month date)
select
project_id,
datepart(yy, all_months.first_of_month) as "year",
datepart(mm, all_months.first_of_month) as "month",
count(*) as "count"
from
incident,
all_months
where
incident.start_date <= dateadd(day, -1, dateadd(month, 1, first_of_month)
(incident.end_date >= first_of_month or incident.end_date is null)
group by
project_id,
datepart(yy, all_months.first_of_month),
datepart(mm, all_months.first_of_month)