Set based query to replace loop to populate all month end dates from given date for all records - sql

I have a table that stores patient lab test results. There can be results from multiple tests like Albumin, Potassium, Phosphorus etc. First reading for each patient from each of these categories is stored in a table called #MetricFirstGroupReading.
CREATE TABLE #MetricFirstGroupReading (Patient_Key INT, Metric_Group VARCHAR(100),
Observation_Date DATE)
ALTER TABLE #MetricFirstGroupReading
ADD CONSTRAINT UQ_MetricFirst UNIQUE (Patient_Key, Metric_Group);
INSERT INTO #MetricFirstGroupReading
SELECT 1, 'Albumin', '2018-11-15' UNION
SELECT 1, 'Potassium', '2018-12-10' UNION
SELECT 2, 'Albumin', '2018-10-20' UNION
SELECT 2, 'Potassium', '2018-11-25'
Now, I need to populate all month end dates upto current month into a new table, for each record from the #MetricFirstGroupReading table. Following is the expected result when the query run on December 2018.
I know how to do it using WHILE loops. How to do this without loops, using set based SQL queries, in SQL Server 2016?

Following worked. This is an expansion of the idea present in tsql: How to retrieve the last date of each month between given date range
Query
CREATE TABLE #AllMonthEnds (MonthEndDate DATE)
DECLARE #Start datetime
DECLARE #End datetime
SELECT #Start = '2000-01-01'
SELECT #End = DATEADD(MONTH,1,GETDATE())
;With CTE as
(
SELECT #Start as Date,Case When DatePart(mm,#Start)<>DatePart(mm,#Start+1) then 1 else 0 end as [Last]
UNION ALL
SELECT Date+1,Case When DatePart(mm,Date+1)<>DatePart(mm,Date+2) then 1 else 0 end from CTE
WHERE Date<#End
)
INSERT INTO #AllMonthEnds
SELECT [Date]
FROM CTE
WHERE [Last]=1
OPTION ( MAXRECURSION 0 )
SELECT T.Patient_Key, T.Metric_Group, T.Observation_Date AS First_Observation_Date,
DATEDIFF(MONTh,Observation_Date, MonthEndDate) AS MonthDiff,
A.MonthEndDate AS IterationDate
FROM #AllMonthEnds A
INNER JOIN
(
SELECT *, ROW_NUMBER() OVER(PARTITION BY Patient_Key, Metric_Group ORDER BY Observation_Date) AS RowVal
FROM #MetricFirstGroupReading M
)T
ON A.MonthEndDate >= T.Observation_Date
WHERE RowVal = 1
ORDER BY Patient_Key, Metric_Group, T.Observation_Date, A.MonthEndDate

How about:
select MetricFirstGroupReading.*, datediff(month, MetricFirstGroupReading.Observation_Date, months.monthendval) monthdiff, months.*
into allmonths
from
(
SELECT 1 patientid, 'Albumin' test, '2018-11-15' Observation_Date UNION
SELECT 1 patientid, 'Potassium' test, '2018-12-10' Observation_Date UNION
SELECT 2 patientid, 'Albumin' test, '2018-10-20' Observation_Date UNION
SELECT 2 patientid, 'Potassium' test, '2018-11-25' Observation_Date) MetricFirstGroupReading
join
(
select '2018-10-31' monthendval union
select '2018-11-30' monthendval union
select '2018-12-31' monthendval
) months on MetricFirstGroupReading.Observation_Date< months.monthendval
Replace the first select union with your table, and add or remove month ends from the second inner select.

Consider building a temp table of all 12 month end dates, then join to main table by date range. Use DateDiff for month difference:
CREATE TABLE #MonthEndDates (Month_End_Value DATE)
INSERT INTO #MonthEndDates
VALUES ('2018-01-31'),
('2018-02-28'),
('2018-03-31'),
('2018-04-30'),
('2018-05-31'),
('2018-04-30'),
('2018-06-30'),
('2018-07-31'),
('2018-08-31'),
('2018-09-30'),
('2018-10-31'),
('2018-11-30'),
('2018-12-31')
SELECT m.Patient_Key, m.Metric_Group, m.Observation_Date,
DateDiff(month, m.Observation_Date, d.Month_End_Value) AS Month_Diff,
d.Month_End_Value
FROM #MetricFirstGroupReading m
INNER JOIN #MonthEndDates d
ON m.Observation_Date < d.Month_End_Value
GO
Rextester Demo

Related

SQL Union as Subquery to create Date Ranges from Start Date

I have three tabels, each of them has a date column (the date column is an INT field and needs to stay that way). I need a UNION accross all three tables so that I get the list of unique dates in accending order like this:
20040602
20051215
20060628
20100224
20100228
20100422
20100512
20100615
Then I need to add a column to the result of the query where I subtract one from each date and place it one row above as the end date. Basically I need to generate the end date from the start date somehow and this is what I got so far (not working):
With Query1 As (
Select date_one As StartDate
From table_one
Union
Select date_two As StartDate
From table_two
Union
Select date_three e As StartDate
From table_three
Order By Date Asc
)
Select Query1.StartDate - 1 As EndDate
From Query1
Thanks a lot for your help!
Building on your existing union cte, we can use lead() in the outer query to get the start_date of the next record, and withdraw 1 from it.
with q as (
select date_one start_date from table_one
union select date_two from table_two
union select date_three from table_three
)
select
start_date,
dateadd(day, -1, lead(start_date) over(order by start_date)) end_date
from q
order by start_date
If the datatype the original columns are numeric, then you need to do some casting before applying date functions:
with q as (
select cast(cast(date_one as varchar(8)) as date) start_date from table_one
union select cast(cast(date_two as varchar(8)) as date) from table_two
union select cast(cast(date_three as varchar(8)) as date) from table_three
)
select
start_date,
dateadd(day, -1, lead(start_date) over(order by start_date)) end_date
from q
order by start_date

Find missing date as compare to calendar

I am explain problem in short.
select distinct DATE from #Table where DATE >='2016-01-01'
Output :
Date
2016-11-23
2016-11-22
2016-11-21
2016-11-19
2016-11-18
Now i need to find out missing date a compare to our calender dates from year '2016'
i.e. Here date '2016-11-20' is missing.
I want list of missing dates.
Thanks for reading this. Have nice day.
You need to generate dates and you have to find missing ones. Below with recursive cte i have done it
;WITH CTE AS
(
SELECT CONVERT(DATE,'2016-01-01') AS DATE1
UNION ALL
SELECT DATEADD(DD,1,DATE1) FROM CTE WHERE DATE1<'2016-12-31'
)
SELECT DATE1 MISSING_ONE FROM CTE
EXCEPT
SELECT * FROM #TABLE1
option(maxrecursion 0)
Using CTE and get all dates in CTE table then compare with your table.
CREATE TABLE #yourTable(_Values DATE)
INSERT INTO #yourTable(_Values)
SELECT '2016-11-23' UNION ALL
SELECT '2016-11-22' UNION ALL
SELECT '2016-11-21' UNION ALL
SELECT '2016-11-19' UNION ALL
SELECT '2016-11-18'
DECLARE #DATE DATE = '2016-11-01'
;WITH CTEYear (_Date) AS
(
SELECT #DATE
UNION ALL
SELECT DATEADD(DAY,1,_Date)
FROM CTEYear
WHERE _Date < EOMONTH(#DATE,0)
)
SELECT * FROM CTEYear
WHERE NOT EXISTS(SELECT 1 FROM #yourTable WHERE _Date = _Values)
OPTION(maxrecursion 0)
You need to generate the dates and then find the missing ones. A recursive CTE is one way to generate a handful of dates. Another way is to use master..spt_values as a list of numbers:
with n as (
select row_number() over (order by (select null)) - 1 as n
from master..spt_values
),
d as (
select dateadd(day, n.n, cast('2016-01-01' as date)) as dte
from n
where n <= 365
)
select d.date
from d left join
#table t
on d.dte = t.date
where t.date is null;
If you are happy enough with ranges of missing dates, you don't need a list of dates at all:
select date, (datediff(day, date, next_date) - 1) as num_missing
from (select t.*, lead(t.date) over (order by t.date) as next_date
from #table t
where t.date >= '2016-01-01'
) t
where next_date <> dateadd(day, 1, date);

Calculating per day in SQL

I have an sql table like that:
Id Date Price
1 21.09.09 25
2 31.08.09 16
1 23.09.09 21
2 03.09.09 12
So what I need is to get min and max date for each id and dif in days between them. It is kind of easy. Using SQLlite syntax:
SELECT id,
min(date),
max(date),
julianday(max(date)) - julianday(min(date)) as dif
from table group by id
Then the tricky one: how can I receive the price per day during this difference period. I mean something like this:
ID Date PricePerDay
1 21.09.09 25
1 22.09.09 0
1 23.09.09 21
2 31.08.09 16
2 01.09.09 0
2 02.09.09 0
2 03.09.09 12
I create a cte as you mentioned with calendar but dont know how to get the desired result:
WITH RECURSIVE
cnt(x) AS (
SELECT 0
UNION ALL
SELECT x+1 FROM cnt
LIMIT (SELECT ((julianday('2015-12-31') - julianday('2015-01-01')) + 1)))
SELECT date(julianday('2015-01-01'), '+' || x || ' days') as date FROM cnt
p.s. If it will be in sqllite syntax-would be awesome!
You can use a recursive CTE to calculate all the days between the min date and max date. The rest is just a left join and some logic:
with recursive cte as (
select t.id, min(date) as thedate, max(date) as maxdate
from t
group by id
union all
select cte.id, date(thedate, '+1 day') as thedate, cte.maxdate
from cte
where cte.thedate < cte.maxdate
)
select cte.id, cte.date,
coalesce(t.price, 0) as PricePerDay
from cte left join
t
on cte.id = t.id and cte.thedate = t.date;
One method is using a tally table.
To build a list of dates and join that with the table.
The date stamps in the DD.MM.YY format are first changed to the YYYY-MM-DD date format.
To make it possible to actually use them as a date in the SQL.
At the final select they are formatted back to the DD.MM.YY format.
First some test data:
create table testtable (Id int, [Date] varchar(8), Price int);
insert into testtable (Id,[Date],Price) values (1,'21.09.09',25);
insert into testtable (Id,[Date],Price) values (1,'23.09.09',21);
insert into testtable (Id,[Date],Price) values (2,'31.08.09',16);
insert into testtable (Id,[Date],Price) values (2,'03.09.09',12);
The SQL:
with Digits as (
select 0 as n
union all select 1
union all select 2
union all select 3
union all select 4
union all select 5
union all select 6
union all select 7
union all select 8
union all select 9
),
t as (
select Id,
('20'||substr([Date],7,2)||'-'||substr([Date],4,2)||'-'||substr([Date],1,2)) as [Date],
Price
from testtable
),
Dates as (
select Id, date(MinDate,'+'||(d2.n*10+d1.n)||' days') as [Date]
from (
select Id, min([Date]) as MinDate, max([Date]) as MaxDate
from t
group by Id
) q
join Digits d1
join Digits d2
where date(MinDate,'+'||(d2.n*10+d1.n)||' days') <= MaxDate
)
select d.Id,
(substr(d.[Date],9,2)||'.'||substr(d.[Date],6,2)||'.'||substr(d.[Date],3,2)) as [Date],
coalesce(t.Price,0) as Price
from Dates d
left join t on (d.Id = t.Id and d.[Date] = t.[Date])
order by d.Id, d.[Date];
The recursive SQL below was totally inspired by the excellent answer from Gordon Linoff.
And a recursive SQL is probably more performant for this anyway.
(He should get the 15 points for the accepted answer).
The difference in this version is that the datestamps are first formatted to YYYY-MM-DD.
with t as (
select Id,
('20'||substr([Date],7,2)||'-'||substr([Date],4,2)||'-'||substr([Date],1,2)) as [Date],
Price
from testtable
),
cte as (
select Id, min([Date]) as [Date], max([Date]) as MaxDate from t
group by Id
union all
select Id, date([Date], '+1 day'), MaxDate from cte
where [Date] < MaxDate
)
select cte.Id,
(substr(cte.[Date],9,2)||'.'||substr(cte.[Date],6,2)||'.'||substr(cte.[Date],3,2)) as [Date],
coalesce(t.Price, 0) as PricePerDay
from cte
left join t
on (cte.Id = t.Id and cte.[Date] = t.[Date])
order by cte.Id, cte.[Date];

Searching SQL table for two consecutive missing dates

I want to search through a SQL table and find two consecutive missing dates.
For example, person 1 inserts 'diary' entry on day 1 and day 2, misses day 3 and day 4, and enters an entry on day 5.
I am not posting code because I am not sure of how to do this at all.
Thanks!
This uses a LEVEL aggregate to build the list of calendar dates from the first entry to the last, then uses LAG() to check a given date with the previous date, and then checks that neither of those dates had an associated entry to find those two-day gaps:
With diary as (
select to_date('01/01/2016','dd/mm/yyyy') entry_dt from dual union all
select to_date('02/01/2016','dd/mm/yyyy') entry_dt from dual union all
select to_date('04/01/2016','dd/mm/yyyy') entry_dt from dual union all
--leave two day gap of 5th and 6th
select to_date('07/01/2016','dd/mm/yyyy') entry_dt from dual union all
select to_date('08/01/2016','dd/mm/yyyy') entry_dt from dual union all
select to_date('10/01/2016','dd/mm/yyyy') entry_dt from dual )
select calendar_dt -1, calendar_dt
FROM (
select calendar_dt, entry_dt, lag(entry_dt) over (order by calendar_dt) prev_entry_dt
from diary
RIGHT OUTER JOIN (select min(entry_dt) + lvl as calendar_dt
FROM diary
,(select level lvl
from dual connect by level < (select max(entry_dt) - min(entry_dt)+1 from diary))
group by lvl) ON calendar_dt = entry_dt
order by calendar_dt
)
where entry_dt is null and prev_entry_dt is null
returns:
CALENDAR_DT-1, CALENDAR_DT
05/01/2016, 06/01/2016
I am only doing the calendar building to simplify building all 2-day gaps, as if a person took three days off that would be two overlapping two-day gaps (day 1-2, and days 2-3). If you want a far simpler query that outputs the start and end point of any gap of two or more days, then the following works:
With diary as (
select to_date('01/01/2016','dd/mm/yyyy') entry_dt from dual union all
select to_date('02/01/2016','dd/mm/yyyy') entry_dt from dual union all
select to_date('04/01/2016','dd/mm/yyyy') entry_dt from dual union all
select to_date('07/01/2016','dd/mm/yyyy') entry_dt from dual union all
select to_date('08/01/2016','dd/mm/yyyy') entry_dt from dual union all
select to_date('10/01/2016','dd/mm/yyyy') entry_dt from dual )
select prev_entry_dt +1 gap_start, entry_dt -1 gap_end
FROM (
select entry_dt, lag(entry_dt) over (order by entry_dt) prev_entry_dt
from diary
order by entry_dt
) where entry_dt - prev_entry_dt > 2
My high level approach to this problem would be to select from a dynamic table of dates, using an integer counter to add or subtract from the current DateTime to get as many dates as you require into the future or past, then LEFT join your data table to this, order by date and select the first row, or N many rows which have a NULL join.
So your data ends up being
DATE ENTRY_ID
---- -----
2016-01-01 1
2016-01-02 2
2016-01-03 NULL
2016-01-04 3
2016-01-05 4
2016-01-06 NULL
2016-01-07 NULL
2016-01-08 NULL
And you can pick all of the values you need from this dataset
Try this your problem looks like similar to this :-
Declare #temp Table(id int identity(1,1) not null,CDate smalldatetime ,val int)
insert into #temp select '10/2/2012',1
insert into #temp select '10/3/2012',1
insert into #temp select '10/5/2012',1
insert into #temp select '10/7/2012',2
insert into #temp select '10/9/2012',2
insert into #temp select '10/10/2012',2
insert into #temp select '10/13/2012',2
insert into #temp select '10/15/2012',2
DECLARE #startDate DATE= '10/01/2012'
DECLARE #endDate DATE= '10/15/2012'
SELECT t.Id, X.[Date],Val = COALESCE(t.val,0)
FROM
(SELECT [Date] = DATEADD(Day,Number,#startDate)
FROM master..spt_values
WHERE Type='P'
AND DATEADD(day,Number,#startDate) <= #endDate)X
LEFT JOIN #temp t
ON X.[Date] = t.CDate
Alternative you can try this :-
WITH dates AS (
SELECT CAST('2009-01-01' AS DATETIME) 'date'
UNION ALL
SELECT DATEADD(dd, 1, t.date)
FROM dates t
WHERE DATEADD(dd, 1, t.date) <= '2009-02-01')
SELECT t.eventid, d.date
FROM dates d
JOIN TABLE t ON d.date BETWEEN t.startdate AND t.enddate

force number of rows to return in date range from SQL query

I'm running a query on our SQL (2012) database which returns a count of records in a given date range, grouped by the date.
For example:
Date Count
12/08 12
14/08 19
19/08 11
I need to fill in the blanks as the charts I plot get screwed up because there are missing values. Is there a way to force the SQL to report back a blank row, or a "0" value when it doesn't come across a result?
My query is
SELECT TheDate, count(recordID)
FROM myTable
WHERE (TheDate between '12-AUG-2013 00:00:00' and '20-AUG-2013 23:59:59')
GROUP BY TheDate
Would I need to create a temp table with the records in, then select from that and right join any records from myTable?
Thanks for any help!
If you create a (temporary or permanent) table of the date range, you can then left join to your results to create a result set including blanks
SELECT dates.TheDate, count(recordID)
FROM
( select
convert(date,dateadd(d,number,'2013-08-12')) as theDate
from master..spt_values
where type='p' and number < 9
) dates
left join yourtable on dates.thedate = convert(date,yourtable.thedate)
GROUP BY dates.TheDate
A temp table would do the job but for such a small date range you could go even simpler and use a UNION-ed subquery. E.g:
SELECT dates.TheDate, ISNULL(counts.Records, 0)
FROM
(SELECT TheDate, count(recordID) AS Records
FROM myTable
WHERE (TheDate between '12-AUG-2013 00:00:00' and '20-AUG-2013 23:59:59')
GROUP BY TheDate
) counts
RIGHT JOIN
(SELECT CAST('12-AUG-2013' AS DATETIME) AS TheDate
UNION ALL SELECT CAST('13-AUG-2013' AS DATETIME) AS TheDate
UNION ALL SELECT CAST('14-AUG-2013' AS DATETIME) AS TheDate
UNION ALL SELECT CAST('15-AUG-2013' AS DATETIME) AS TheDate
UNION ALL SELECT CAST('16-AUG-2013' AS DATETIME) AS TheDate
UNION ALL SELECT CAST('17-AUG-2013' AS DATETIME) AS TheDate
UNION ALL SELECT CAST('18-AUG-2013' AS DATETIME) AS TheDate
UNION ALL SELECT CAST('19-AUG-2013' AS DATETIME) AS TheDate
UNION ALL SELECT CAST('20-AUG-2013' AS DATETIME) AS TheDate
) dates
ON counts.TheDate = dates.TheDate
Here's a SQL Fiddle Demo.
If you need a more generic (but also more complex) solution, take a look at this excellent answer (by #RedFilter) to a similar question.