Calculating per day in SQL - sql

I have an sql table like that:
Id Date Price
1 21.09.09 25
2 31.08.09 16
1 23.09.09 21
2 03.09.09 12
So what I need is to get min and max date for each id and dif in days between them. It is kind of easy. Using SQLlite syntax:
SELECT id,
min(date),
max(date),
julianday(max(date)) - julianday(min(date)) as dif
from table group by id
Then the tricky one: how can I receive the price per day during this difference period. I mean something like this:
ID Date PricePerDay
1 21.09.09 25
1 22.09.09 0
1 23.09.09 21
2 31.08.09 16
2 01.09.09 0
2 02.09.09 0
2 03.09.09 12
I create a cte as you mentioned with calendar but dont know how to get the desired result:
WITH RECURSIVE
cnt(x) AS (
SELECT 0
UNION ALL
SELECT x+1 FROM cnt
LIMIT (SELECT ((julianday('2015-12-31') - julianday('2015-01-01')) + 1)))
SELECT date(julianday('2015-01-01'), '+' || x || ' days') as date FROM cnt
p.s. If it will be in sqllite syntax-would be awesome!

You can use a recursive CTE to calculate all the days between the min date and max date. The rest is just a left join and some logic:
with recursive cte as (
select t.id, min(date) as thedate, max(date) as maxdate
from t
group by id
union all
select cte.id, date(thedate, '+1 day') as thedate, cte.maxdate
from cte
where cte.thedate < cte.maxdate
)
select cte.id, cte.date,
coalesce(t.price, 0) as PricePerDay
from cte left join
t
on cte.id = t.id and cte.thedate = t.date;

One method is using a tally table.
To build a list of dates and join that with the table.
The date stamps in the DD.MM.YY format are first changed to the YYYY-MM-DD date format.
To make it possible to actually use them as a date in the SQL.
At the final select they are formatted back to the DD.MM.YY format.
First some test data:
create table testtable (Id int, [Date] varchar(8), Price int);
insert into testtable (Id,[Date],Price) values (1,'21.09.09',25);
insert into testtable (Id,[Date],Price) values (1,'23.09.09',21);
insert into testtable (Id,[Date],Price) values (2,'31.08.09',16);
insert into testtable (Id,[Date],Price) values (2,'03.09.09',12);
The SQL:
with Digits as (
select 0 as n
union all select 1
union all select 2
union all select 3
union all select 4
union all select 5
union all select 6
union all select 7
union all select 8
union all select 9
),
t as (
select Id,
('20'||substr([Date],7,2)||'-'||substr([Date],4,2)||'-'||substr([Date],1,2)) as [Date],
Price
from testtable
),
Dates as (
select Id, date(MinDate,'+'||(d2.n*10+d1.n)||' days') as [Date]
from (
select Id, min([Date]) as MinDate, max([Date]) as MaxDate
from t
group by Id
) q
join Digits d1
join Digits d2
where date(MinDate,'+'||(d2.n*10+d1.n)||' days') <= MaxDate
)
select d.Id,
(substr(d.[Date],9,2)||'.'||substr(d.[Date],6,2)||'.'||substr(d.[Date],3,2)) as [Date],
coalesce(t.Price,0) as Price
from Dates d
left join t on (d.Id = t.Id and d.[Date] = t.[Date])
order by d.Id, d.[Date];
The recursive SQL below was totally inspired by the excellent answer from Gordon Linoff.
And a recursive SQL is probably more performant for this anyway.
(He should get the 15 points for the accepted answer).
The difference in this version is that the datestamps are first formatted to YYYY-MM-DD.
with t as (
select Id,
('20'||substr([Date],7,2)||'-'||substr([Date],4,2)||'-'||substr([Date],1,2)) as [Date],
Price
from testtable
),
cte as (
select Id, min([Date]) as [Date], max([Date]) as MaxDate from t
group by Id
union all
select Id, date([Date], '+1 day'), MaxDate from cte
where [Date] < MaxDate
)
select cte.Id,
(substr(cte.[Date],9,2)||'.'||substr(cte.[Date],6,2)||'.'||substr(cte.[Date],3,2)) as [Date],
coalesce(t.Price, 0) as PricePerDay
from cte
left join t
on (cte.Id = t.Id and cte.[Date] = t.[Date])
order by cte.Id, cte.[Date];

Related

How to return 0 value if no record exists in bigquery

I need find out for which date record does not exits in BigQuery table.
Query pls find
select cast(creat_ts as date) as create,IFNULL(count(*) ,0)
FROM table
where cast(creat_ts as date)='2020-06-23' group by 1 )
Below is for BigQuery Standard SQL
#standardSQL
SELECT DISTINCT day
FROM UNNEST(GENERATE_DATE_ARRAY('2020-06-01', '2020-06-30')) day
LEFT JOIN `project.dataset.table` t
ON CAST(creat_ts AS DATE) = day
WHERE creat_ts IS NULL
You could try something like this:
with calendar as (
select * from unnest(generate_date_array('2020-01-01', '2020-07-01', interval 1 day)) date
),
temp as (
select cast(b.create_ts as date) as date from `project.dataset.table` b
),
daily_count as (
select
date,
count(date.temp) as ct
from calendar
left join temp using(date)
group by 1
)
select * from daily_count
where ct = 0
order by 1

Set based query to replace loop to populate all month end dates from given date for all records

I have a table that stores patient lab test results. There can be results from multiple tests like Albumin, Potassium, Phosphorus etc. First reading for each patient from each of these categories is stored in a table called #MetricFirstGroupReading.
CREATE TABLE #MetricFirstGroupReading (Patient_Key INT, Metric_Group VARCHAR(100),
Observation_Date DATE)
ALTER TABLE #MetricFirstGroupReading
ADD CONSTRAINT UQ_MetricFirst UNIQUE (Patient_Key, Metric_Group);
INSERT INTO #MetricFirstGroupReading
SELECT 1, 'Albumin', '2018-11-15' UNION
SELECT 1, 'Potassium', '2018-12-10' UNION
SELECT 2, 'Albumin', '2018-10-20' UNION
SELECT 2, 'Potassium', '2018-11-25'
Now, I need to populate all month end dates upto current month into a new table, for each record from the #MetricFirstGroupReading table. Following is the expected result when the query run on December 2018.
I know how to do it using WHILE loops. How to do this without loops, using set based SQL queries, in SQL Server 2016?
Following worked. This is an expansion of the idea present in tsql: How to retrieve the last date of each month between given date range
Query
CREATE TABLE #AllMonthEnds (MonthEndDate DATE)
DECLARE #Start datetime
DECLARE #End datetime
SELECT #Start = '2000-01-01'
SELECT #End = DATEADD(MONTH,1,GETDATE())
;With CTE as
(
SELECT #Start as Date,Case When DatePart(mm,#Start)<>DatePart(mm,#Start+1) then 1 else 0 end as [Last]
UNION ALL
SELECT Date+1,Case When DatePart(mm,Date+1)<>DatePart(mm,Date+2) then 1 else 0 end from CTE
WHERE Date<#End
)
INSERT INTO #AllMonthEnds
SELECT [Date]
FROM CTE
WHERE [Last]=1
OPTION ( MAXRECURSION 0 )
SELECT T.Patient_Key, T.Metric_Group, T.Observation_Date AS First_Observation_Date,
DATEDIFF(MONTh,Observation_Date, MonthEndDate) AS MonthDiff,
A.MonthEndDate AS IterationDate
FROM #AllMonthEnds A
INNER JOIN
(
SELECT *, ROW_NUMBER() OVER(PARTITION BY Patient_Key, Metric_Group ORDER BY Observation_Date) AS RowVal
FROM #MetricFirstGroupReading M
)T
ON A.MonthEndDate >= T.Observation_Date
WHERE RowVal = 1
ORDER BY Patient_Key, Metric_Group, T.Observation_Date, A.MonthEndDate
How about:
select MetricFirstGroupReading.*, datediff(month, MetricFirstGroupReading.Observation_Date, months.monthendval) monthdiff, months.*
into allmonths
from
(
SELECT 1 patientid, 'Albumin' test, '2018-11-15' Observation_Date UNION
SELECT 1 patientid, 'Potassium' test, '2018-12-10' Observation_Date UNION
SELECT 2 patientid, 'Albumin' test, '2018-10-20' Observation_Date UNION
SELECT 2 patientid, 'Potassium' test, '2018-11-25' Observation_Date) MetricFirstGroupReading
join
(
select '2018-10-31' monthendval union
select '2018-11-30' monthendval union
select '2018-12-31' monthendval
) months on MetricFirstGroupReading.Observation_Date< months.monthendval
Replace the first select union with your table, and add or remove month ends from the second inner select.
Consider building a temp table of all 12 month end dates, then join to main table by date range. Use DateDiff for month difference:
CREATE TABLE #MonthEndDates (Month_End_Value DATE)
INSERT INTO #MonthEndDates
VALUES ('2018-01-31'),
('2018-02-28'),
('2018-03-31'),
('2018-04-30'),
('2018-05-31'),
('2018-04-30'),
('2018-06-30'),
('2018-07-31'),
('2018-08-31'),
('2018-09-30'),
('2018-10-31'),
('2018-11-30'),
('2018-12-31')
SELECT m.Patient_Key, m.Metric_Group, m.Observation_Date,
DateDiff(month, m.Observation_Date, d.Month_End_Value) AS Month_Diff,
d.Month_End_Value
FROM #MetricFirstGroupReading m
INNER JOIN #MonthEndDates d
ON m.Observation_Date < d.Month_End_Value
GO
Rextester Demo

Teradata - How to account for missing hours in timestamp when using extract() function

I have the following statement to extract the date, hour and number of users from a table in a Teradata DB . . .
SELECT
CAST(end_time AS DATE) AS end_date,
EXTRACT(HOUR FROM end_time) AS end_hour,
COUNT(users) AS total_users
FROM table
GROUP BY end_date, end_hour
When using the extract() function, my resultset contains missing hours where there is no activity by users over a 24 hour period... I'm wondering is there any technique to account for these missing hours in my resultset?
I can't creat a lookup table to reference as I don't have the necessary permissions to create a table on this DB.
Any help would be appreciated!
sys_calendar.calendar to generate the requested dates (change the range as needed)
WITH RECURSIVE to generate the hours
with recursive cte_hours (hr)
as
(
select 0 from (select 1) t(c)
union all select hr + 1 from cte_hours where hr < 23
)
select c.calendar_date as dt
,h.hr as hr
,zeroifnull(t.total_users) as total_users
from sys_calendar.calendar as c
cross join cte_hours as h
left join (select cast(end_time as date) as end_date
,extract(hour from end_time) as end_hour
,count(users) as total_users
from mytable t
group by end_date
,end_hour
) t
on t.end_date = c.calendar_date
and t.end_hour = h.hr
where c.calendar_date between current_date - 10 and current_date
order by dt,hr
;
For #GordonLinoff
select 0
0
select 1
1
select 0
union all
select 1
[3888] A SELECT for a UNION,INTERSECT or MINUS must reference a table.
select 0 from (select 1 as c) t
union all
select 1 from (select 1 as c) t
0
1
or
select 0 from (select 1) t(c)
union all
select 1 from (select 1) t(c)
0
1
If you want all hours from all days in the database, then you can generate the rows using cross join and then use left join to bring in results:
SELECT d.end_date,
EXTRACT(HOUR FROM end_time) AS end_hour,
COUNT(t.users) AS total_users
FROM (select distinct CAST(end_time AS DATE) AS end_date from table) d CROSS JOIN
(select distinct EXTRACT(HOUR FROM end_time) AS end_hour from table) h LEFT JOIN
table t
ON t.end_date = d.end_date and t.end_hour = d.end_hour
GROUP BY e.end_date, h.end_hour;
If all hours are not represented, you can use an explicit list:
SELECT d.end_date,
EXTRACT(HOUR FROM end_time) AS end_hour,
COUNT(t.users) AS total_users
FROM (select distinct CAST(end_time AS DATE) AS end_date from table) d CROSS JOIN
(select * from (select 0 as end_hour) t UNION ALL
select * from (select 1 as end_hour) t UNION ALL
. . .
) h LEFT JOIN
table t
ON t.end_date = d.end_date and t.end_hour = d.end_hour
GROUP BY e.end_date, h.end_hour;

Find missing date as compare to calendar

I am explain problem in short.
select distinct DATE from #Table where DATE >='2016-01-01'
Output :
Date
2016-11-23
2016-11-22
2016-11-21
2016-11-19
2016-11-18
Now i need to find out missing date a compare to our calender dates from year '2016'
i.e. Here date '2016-11-20' is missing.
I want list of missing dates.
Thanks for reading this. Have nice day.
You need to generate dates and you have to find missing ones. Below with recursive cte i have done it
;WITH CTE AS
(
SELECT CONVERT(DATE,'2016-01-01') AS DATE1
UNION ALL
SELECT DATEADD(DD,1,DATE1) FROM CTE WHERE DATE1<'2016-12-31'
)
SELECT DATE1 MISSING_ONE FROM CTE
EXCEPT
SELECT * FROM #TABLE1
option(maxrecursion 0)
Using CTE and get all dates in CTE table then compare with your table.
CREATE TABLE #yourTable(_Values DATE)
INSERT INTO #yourTable(_Values)
SELECT '2016-11-23' UNION ALL
SELECT '2016-11-22' UNION ALL
SELECT '2016-11-21' UNION ALL
SELECT '2016-11-19' UNION ALL
SELECT '2016-11-18'
DECLARE #DATE DATE = '2016-11-01'
;WITH CTEYear (_Date) AS
(
SELECT #DATE
UNION ALL
SELECT DATEADD(DAY,1,_Date)
FROM CTEYear
WHERE _Date < EOMONTH(#DATE,0)
)
SELECT * FROM CTEYear
WHERE NOT EXISTS(SELECT 1 FROM #yourTable WHERE _Date = _Values)
OPTION(maxrecursion 0)
You need to generate the dates and then find the missing ones. A recursive CTE is one way to generate a handful of dates. Another way is to use master..spt_values as a list of numbers:
with n as (
select row_number() over (order by (select null)) - 1 as n
from master..spt_values
),
d as (
select dateadd(day, n.n, cast('2016-01-01' as date)) as dte
from n
where n <= 365
)
select d.date
from d left join
#table t
on d.dte = t.date
where t.date is null;
If you are happy enough with ranges of missing dates, you don't need a list of dates at all:
select date, (datediff(day, date, next_date) - 1) as num_missing
from (select t.*, lead(t.date) over (order by t.date) as next_date
from #table t
where t.date >= '2016-01-01'
) t
where next_date <> dateadd(day, 1, date);

sql server rolling 12 months sum with date gaps

Suppose I have a table that indicates the number of items sold in a particular month for each sales rep. However, there will not be a row for a particular person in months where there were no sales. Example
rep_id month_yr num_sales
1 01/01/2012 3
1 05/01/2012 1
1 11/01/2012 1
2 02/01/2012 2
2 05/01/2012 1
I want to be able to create a query that shows for each rep_id and all possible months (01/01/2012, 02/01/2012, etc. through current) a rolling 12 month sales sum, like this:
rep_id month_yr R12_Sum
1 11/01/2012 5
1 12/01/2012 5
1 01/01/2013 5
1 02/01/2013 2
I have found some examples online, but the problem I'm running into is I'm missing some dates for each rep_id. Do I need to cross join or something?
To solve this problem, you need a driver table that has all year/month combinations. Then, you need to create this for each rep.
The solution is then to left join the actual data to this driver and aggregate the period that you want. Here is the query:
with months as (
select 1 as mon union all select 2 union all select 3 union all select 4 union all
select 5 as mon union all select 6 union all select 7 union all select 8 union all
select 9 as mon union all select 10 union all select 11 union all select 12
),
years as (select 2010 as yr union all select 2011 union all select 2012 union all select 2013
),
monthyears as (
select yr, mon, yr*12+mon as yrmon
from months cross join years
),
rmy as (
select *
from monthyears my cross join
(select distinct rep_id from t
) r
)
select rmy.rep_id, rmy.yr, rmy.mon, SUM(t.num_sales) as r12_sum
from rmy join
t
on rmy.rep_id = t.rep_id and
t.year(month_yr)*12 + month(month_yr) between rmy.yrmon - 11 and rmy.yrmon
group by rmy.rep_id, rmy.yr, rmy.mon
order by 1, 2, 3
This hasn't been tested, so it may have syntactic errors. Also, it doesn't convert the year/month combination back to a date, leaving the values in separate columns.
Here is one solution:
SELECT
a.rep_id
,a.month_yr
,SUM(b.R12_Sum) AS R12_TTM
FROM YourTable a
LEFT OUTER JOIN YourTable b
ON a.rep_id = b.rep_id
AND a.month_yr <= b.month_yr
AND a.month_yr >= DATEADD(MONTH, -11, b.month_yr)
GROUP BY
a.rep_id
,a.month_yr
It's certainly not pretty but is more simple than a CTE, numbers table or self join:
DECLARE #startdt DATETIME
SET #startdt = '2012-01-01'
SELECT rep_id, YEAR(month_yr), MONTH(month_yr), SUM(num_sales)
FROM MyTable WHERE month_yr >= #startdt AND month_yr < DATEADD(MONTH,1,#startdt)
UNION ALL
SELECT rep_id, YEAR(month_yr), MONTH(month_yr), SUM(num_sales)
FROM MyTable WHERE month_yr >= DATEADD(MONTH,1,#startdt) AND month_yr < DATEADD(MONTH,2,#startdt)
UNION ALL
SELECT rep_id, YEAR(month_yr), MONTH(month_yr), SUM(num_sales)
FROM MyTable WHERE month_yr >= DATEADD(MONTH,2,#startdt) AND month_yr < DATEADD(MONTH,3,#startdt)
UNION ALL
SELECT rep_id, YEAR(month_yr), MONTH(month_yr), SUM(num_sales)
FROM MyTable WHERE month_yr >= DATEADD(MONTH,3,#startdt) AND month_yr < DATEADD(MONTH,4,#startdt)
UNION ALL
etc etc
The following demonstrates using a CTE to generate a table of dates and generating a summary report using the CTE. Sales representatives are omitted from the results when they have had no applicable sales.
Try jiggling the reporting parameters, e.g. setting #RollingMonths to 1, for more entertainment.
-- Sample data.
declare #Sales as Table ( rep_id Int, month_yr Date, num_sales Int );
insert into #Sales ( rep_id, month_yr, num_sales ) values
( 1, '01/01/2012', 3 ),
( 1, '05/01/2012', 1 ),
( 1, '11/01/2012', 1 ),
( 2, '02/01/2012', 1 ),
( 2, '05/01/2012', 2 );
select * from #Sales;
-- Reporting parameters.
declare #ReportEnd as Date = DateAdd( day, 1 - Day( GetDate() ), GetDate() ); -- The first of the current month.
declare #ReportMonths as Int = 6; -- Number of months to report.
declare #RollingMonths as Int = 12; -- Number of months in rolling sums.
-- Report.
-- A CTE generates a table of month/year combinations covering the desired reporting time period.
with ReportingIntervals as (
select DateAdd( month, 1 - #ReportMonths, #ReportEnd ) as ReportingInterval,
DateAdd( month, 1 - #RollingMonths, DateAdd( month, 1 - #ReportMonths, #ReportEnd ) ) as FirstRollingMonth
union all
select DateAdd( month, 1, ReportingInterval ), DateAdd( month, 1, FirstRollingMonth )
from ReportingIntervals
where ReportingInterval < #ReportEnd )
-- Join the CTE with the sample data and summarize.
select RI.ReportingInterval, S.rep_id, Sum( S.num_sales ) as R12_Sum
from ReportingIntervals as RI left outer join
#Sales as S on RI.FirstRollingMonth <= S.month_yr and S.month_yr <= RI.ReportingInterval
group by RI.ReportingInterval, S.rep_id
order by RI.ReportingInterval, S.rep_id