generate_series() equivalent in snowflake - sql

I'm trying to find the snowflake equivalent of generate_series() (the PostgreSQL syntax).
SELECT generate_series(timestamp '2017-11-01', CURRENT_DATE, '1 day')

Just wanted to expand on Marcin Zukowski's comment to say that these gaps started to show up almost immediately after using a date range generated this way in a JOIN.
We ultimately ended up doing this instead!
select
dateadd(
day,
'-' || row_number() over (order by null),
dateadd(day, '+1', current_date())
) as date
from table (generator(rowcount => 90))

I had a similar problem and found an approach, which avoids the issue of a generator requiring a constant value by using a session variable in addition to the already great answers here. This is closest to the requirement of the OP to my mind.
-- set parameter to be used as generator "constant" including the start day
set num_days = (Select datediff(day, TO_DATE('2017-11-01','YYYY-MM-DD'), current_date()+1));
-- use parameter in bcrowell's answer now
select
dateadd(
day,
'-' || row_number() over (order by null),
dateadd(day, '+1', current_date())
) as date
from table (generator(rowcount => ($num_days)));
-- clean up previously set variable
unset num_days;

WITH RECURSIVE rec_cte AS (
-- start date
SELECT '2017-11-01'::DATE as dt
UNION ALL
SELECT DATEADD('day',1,dt) as dt
FROM rec_cte
-- end date (inclusive)
WHERE dt < current_date()
)
SELECT * FROM rec_cte

Adding this answer for completitude, in case you have an initial and last date:
select -1 + row_number() over(order by 0) i, start_date + i generated_date
from (select '2020-01-01'::date start_date, '2020-01-15'::date end_date)
join table(generator(rowcount => 10000 )) x
qualify i < 1 + end_date - start_date

I found the generator function in Snowflake quite limiting for all but the simplest use cases. For example, it was not clear how to take a single row specification, explode it into a table of dates and join it back to the original spec table.
Here is an alternative that uses recursive CTEs.
-- A 2 row table that contains "specs" for a date range
create local temp table date_spec as
select 1 as id, '2022-04-01'::date as start_date, current_date() as end_date
union all
select 2, '2022-03-01', '2032-03-30'
;
with explode_date(id, date, next_date, end_date) as (
select
id
, start_date as date -- start_date is the first date
, date + 1 as next_date -- next_date is the date of for the subsequent row in the recursive cte
, end_date
from date_spec
union all
select
ds.id
, ed.next_date -- the current_date is the value of next_date from above
, ed.next_date + 1
, ds.end_date
from date_spec ds
join explode_date ed
on ed.id = ds.id
where ed.date <= ed.end_date -- keep running until you hit the end_date
)
select * from explode_date
order by id, date desc
;

This is how I was able to generate a series of dates in Snowflake. I set row count to 1095 to get 3 years worth of dates, you can of course change that to whatever suits your use case
select
dateadd(day, '-' || seq4(), current_date()) as dte
from
table
(generator(rowcount => 1095))
Originally found here
EDIT: This solution is not correct. seq4 does not guarantee a sequence without gaps. Please follow other answers, not this one. Thanks #Marcin Zukowski for pointing that out.

Related

postgres sql query to identify rows with same foreign key, but non consecutive dates

I have a table with a foreign_key_id column and a date column.
For each row that has the same foreign key, there is a different date, and if I order by foreign_key_id, date , 90% of the time all the dates are consecutive.
There are some edge cases though, where there are multiple entries with the same foreign_key that don't have consecutive dates.
Trying to come up with an easy way to identify all the foreign_key_id 's that don't have consecutive dates. Any ideas?
I was thinking of left joining on to a generated series, somehow partitioning by track id, but keep hitting a mental wall. My sql query editor keeps crashing, so that is adding some more unrelated frustration
EDIT:
I ended up doing an order by foreign_key_id, date , copying and pasting the result in excel, and then finding what I needed by doing this type of logic formula:
=IF( (B91 = B90), (F91 =(F90 + 1)) , 1 ) , where b is the foreign key column and F is the date column
but wondering if something similar could be done in sql. here's what I had when I gave up and went to excel:
select to_char(date_range.days, 'yyyy-mm-dd') as x
, data.*
from (
select generate_series('2019-04-30'::date,'2019-11-05'::date, '1 day')::date as days
) as date_range
left join(
select foreign_key_id, date
from table_a
order by foreign_key_id, date
) data on data.date = date_range.days
where foreign_key_id is null
You could do that, sure. No joins needed either. Use LAG(datecol) OVER(PARTITION BY foreignkeycol ORDER BY datecol) to get the date of the previous row for the same fk, diff it to the current date to show how many intervals (days? Minutes?) have passed since that date and then wrap it all in something that does WHERE thedifference <> 1 (Or however you define consecutive - if consecutive to you is "every 2 days" then it would be anything that doesn't have a difference of 2)
If you want both rows either side of the gap, use LEAD (same format as LAG) to get the next date and calc two diffs, then do WHERE difftoprev <> 1 or difftonext <>1 etc
It would look something like this (untested)
WITH cte AS (
SELECT foreignkeycol, datecol,
LAG(datecol) OVER(PARTITION BY foreignkeycol ORDER BY datecol) as prevdate,
LEAD(datecol) OVER(PARTITION BY foreignkeycol ORDER BY datecol) as nextdate
FROM table
)
SELECT *
FROM cte
WHERE
DATE_PART('day', datecol - prevdate) <> 1 OR
DATE_PART('day', nextdate - datecol) <> 1
I would use lead():
select t.*
from (select t.*,
lead(date) over (partition by foreign_key_id order by date) as next_date
from t
) t
where next_date <> date + interval '1 day';
This will provide each row where the next row does not have the expected date.

SQL how to write a query that return missing date ranges?

I am trying to figure out how to write a query that looks at certain records and finds missing date ranges between today and 9999-12-31.
My data looks like below:
ID |start_dt |end_dt |prc_or_disc_1
10412 |2018-07-17 00:00:00.000 |2018-07-20 00:00:00.000 |1050.000000
10413 |2018-07-23 00:00:00.000 |2018-07-26 00:00:00.000 |1040.000000
So for this data I would want my query to return:
2018-07-10 | 2018-07-16
2018-07-21 | 2018-07-22
2018-07-27 | 9999-12-31
I'm not really sure where to start. Is this possible?
You can do that using the lag() function in MS SQL (but that is available starting with 2012?).
with myData as
(
select *,
lag(end_dt,1) over (order by start_dt) as lagEnd
from myTable),
myMax as
(
select Max(end_dt) as maxDate from myTable
)
select dateadd(d,1,lagEnd) as StartDate, dateadd(d, -1, start_dt) as EndDate
from myData
where lagEnd is not null and dateadd(d,1,lagEnd) < start_dt
union all
select dateAdd(d,1,maxDate) as StartDate, cast('99991231' as Datetime) as EndDate
from myMax
where maxDate < '99991231';
If lag() is not available in MS SQL 2008, then you can mimic it with row_number() and joining.
select
CASE WHEN DATEDIFF(day, end_dt, ISNULL(LEAD(start_dt) over (order by ID), '99991231')) > 1 then end_dt +1 END as F1,
CASE WHEN DATEDIFF(day, end_dt, ISNULL(LEAD(start_dt) over (order by ID), '99991231')) > 1 then ISNULL(LEAD(start_dt) over (order by ID) - 1, '99991231') END as F2
from t
Working SQLFiddle example is -> Here
FOR 2008 VERSION
SELECT
X.end_dt + 1 as F1,
ISNULL(Y.start_dt-1, '99991231') as F2
FROM t X
LEFT JOIN (
SELECT
*
, (SELECT MAX(ID) FROM t WHERE ID < A.ID) as ID2
FROM t A) Y ON X.ID = Y.ID2
WHERE DATEDIFF(day, X.end_dt, ISNULL(Y.start_dt, '99991231')) > 1
Working SQLFiddle example is -> Here
This should work in 2008, it assumes that ranges in your table do not overlap. It will also eliminate rows where the end_date of the current row is a day before the start date of the next row.
with dtRanges as (
select start_dt, end_dt, row_number() over (order by start_dt) as rownum
from table1
)
select t2.end_dt + 1, coalesce(start_dt_next -1,'99991231')
FROM
( select dr1.start_dt, dr1.end_dt,dr2.start_dt as start_dt_next
from dtRanges dr1
left join dtRanges dr2 on dr2.rownum = dr1.rownum + 1
) t2
where
t2.end_dt + 1 <> coalesce(start_dt_next,'99991231')
http://sqlfiddle.com/#!18/65238/1
SELECT
*
FROM
(
SELECT
end_dt+1 AS start_dt,
LEAD(start_dt-1, 1, '9999-12-31')
OVER (ORDER BY start_dt)
AS end_dt
FROM
yourTable
)
gaps
WHERE
gaps.end_dt >= gaps.start_dt
I would, however, strongly urge you to use end dates that are "exclusive". That is, the range is everything up to but excluding the end_dt.
That way, a range of one day becomes '2018-07-09', '2018-07-10'.
It's really clear that my range is one day long, if you subtract one from the other you get a day.
Also, if you ever change to needing hour granularity or minute granularity you don't need to change your data. It just works. Always. Reliably. Intuitively.
If you search the web you'll find plenty of documentation on why inclusive-start and exclusive-end is a very good idea from a software perspective. (Then, in the query above, you can remove the wonky +1 and -1.)
This solves your case, but provide some sample data if there will ever be overlaps, fringe cases, etc.
Take one day after your end date and 1 day before the next line's start date.
DECLARE # TABLE (ID int, start_dt DATETIME, end_dt DATETIME, prc VARCHAR(100))
INSERT INTO # (id, start_dt, end_dt, prc)
VALUES
(10410, '2018-07-09 00:00:00.00','2018-07-12 00:00:00.000','1025.000000'),
(10412, '2018-07-17 00:00:00.00','2018-07-20 00:00:00.000','1050.000000'),
(10413, '2018-07-23 00:00:00.00','2018-07-26 00:00:00.000','1040.000000')
SELECT DATEADD(DAY, 1, end_dt)
, DATEADD(DAY, -1, LEAD(start_dt, 1, '9999-12-31') OVER(ORDER BY id) )
FROM #
You may want to take a look at this:
http://sqlfiddle.com/#!18/3a224/1
You just have to edit the begin range to today and the end range to 9999-12-31.

Find the date after a gap in date range in sql

I have these date ranges that represent start and end dates of subscription. There are no overlaps in date ranges.
Start Date End Date
1/5/2015 - 1/14/2015
1/15/2015 - 1/20/2015
1/24/2015 - 1/28/2015
1/29/2015 - 2/3/2015
I want to identify delays of more than 1 day between any subscription ending and a new one starting. e.g. for the data above, i want the output: 1/24/2015 - 1/28/2015.
How can I do this using a sql query?
Edit : Also there can be multiple gaps in the subscription date ranges but I want the date range after the latest one.
You do this using a left join or not exists:
select t.*
from t
where not exists (select 1
from t t2
where t2.enddate = dateadd(day, -1, t.startdate)
);
Note that this will also give you the first record in the sequence . . . which, strictly speaking, matches the conditions. Here is one solution to that problem:
select t.*
from t cross join
(select min(startdate) as minsd from t) as x
where not exists (select 1
from t t2
where t2.enddate = dateadd(day, -1, t.startdate)
) and
t.startdate <> minsd;
You can also approach this with window functions:
select t.*
from (select t.*,
lag(enddate) over (order by startdate) as prev_enddate,
min(startdate) over () as min_startdate
from t
) t
where minstartdate <> startdate and
enddate <> dateadd(day, -1, startdate);
Also note that this logic assumes that the time periods do not overlap. If they do, a clearer problem statement is needed to understand what you are really looking for.
You can achieve this using window function LAG() that would get value from previous row in ordered set for later comparison in WHERE clause. Then, in WHERE you just apply your "gapping definition" and discard the first row.
SQL FIDDLE - Test it!
Sample data:
create table dates(start_date date, end_date date);
insert into dates values
('2015-01-05','2015-01-14'),
('2015-01-15','2015-01-20'),
('2015-01-24','2015-01-28'), -- gap
('2015-01-29','2015-02-03'),
('2015-02-04','2015-02-07'),
('2015-02-09','2015-02-11'); -- gap
Query
SELECT
start_date,
end_date
FROM (
SELECT
start_date,
end_date,
LAG(end_date, 1) OVER (ORDER BY start_date) AS prev_end_date
FROM dates
) foo
WHERE
start_date IS DISTINCT FROM ( prev_end_date + 1 ) -- compare current row start_date with previous row end_date + 1 day
AND prev_end_date IS NOT NULL -- discard first row, which has null value in LAG() calculation
I assume that there are no overlaps in your data and that there are unique values for each pair. If that's not the case, you need to clarify this.

Counting business days between two dates for each row in a table

I'm trying to implement this askTom solution to count the business days between two date fields in a table.
select count(*)
from ( select rownum rnum
from all_objects
where rownum <= to_date('&1') - to_date('&2')+1 )
where to_char( to_date('&2')+rnum-1, 'DY' ) not in ( 'SAT', 'SUN' )
I don't know how I can pass values to toms code or how to do a work around so that each time the select executes with a different set of dates and that way obtaining a similar output :
rowid | bussiness_days
I guess this could be implemented with a PL/SQL block but I'd prefer to keep it down to a query if possible. Could it be possible to pass values using &1 parameters from a select above toms one?
This is not like the original askTom, but if you're using 11gR2 you can use a Recursive CTE:
with rcte(a,b, i, is_wd) as (
select from_dt , to_dt , id, case when (to_char(from_dt, 'DY') in ('SAT','SUN')) then 0 else 1 end from t
union all
select decode(a, null, a,a+1), b, i, case when (to_char(a, 'DY') in ('SAT','SUN')) then 0 else 1 end
from rcte
where a+1 <= b
)
select i id, sum(is_wd)
from rcte
group by i
where t is a table containing "from_dates" and "to_dates"
Here is a sqlfiddle demo
Try this one:
SELECT COUNT(*)
FROM ( SELECT ROWNUM rnum
FROM all_objects
WHERE ROWNUM <= TO_DATE('2014-02-07','yyyy-mm-dd') - TO_DATE('2014-02-01','yyyy-mm-dd')+1 )
WHERE TO_CHAR( TO_DATE('2014-02-01','yyyy-mm-dd')+rnum-1, 'DY' ) NOT IN ( 'SAT', 'SUN' )
However, it has at least two issues:
It presumes the session NLS_DATE_LANGUAGE is set to English
The query gives wrong result if number of rows in ALL_OBJECTS is smaller than the amount of days between your ranges.
The version is less error-prone and faster:
WITH t AS
(SELECT TO_DATE('2014-02-01','yyyy-mm-dd')+LEVEL-1 the_date
FROM dual
CONNECT BY TO_DATE('2014-02-01','yyyy-mm-dd')+LEVEL-1 <= TO_DATE('2014-02-07','yyyy-mm-dd'))
SELECT COUNT(*)
FROM t
WHERE TO_CHAR(the_date, 'DY', 'NLS_DATE_LANGUAGE = AMERICAN') NOT IN ( 'SAT', 'SUN' )
This is another situation where a calendar table comes in handy. A calendar table has a row for every date.
If Saturday and Sunday are not business days, it's quite likely that holidays are not either. A field in the calendar table to indicate holidays makes it a lot easier to exclude them from your business days calculation.

SQL Get the gaps in dateranges when a list of ranges is provided

I'm currently looking for a SQL solution for the following problem:
SQLFiddle as guidance:
I have a list of not-nullable startdates and nullable enddates. Based on this list I need the total gap time between a given start and enddate.
Based on the SQLFiddle
If I would only have situation 1 in my database the result should be 2 days.
If I would have situation 2 and 3 in my database the result should be 1 day.
I have been pondering this for a couple of days now... any help would be much appreciated!
Regards,
Kyor
Notes: I'm running SQL 2012 ( should any special new features be required )
The best solution will be to create 'Dates' table and start from there, otherwise solution will be unmaintainable. For each date in specified range you can check whether it is covered by ranges in 'dateranges' table and get a count of dates that are not.
Something like this:
SELECT COUNT(*)
FROM
Dates d
WHERE
d.Date BETWEEN #start AND #end
AND NOT EXISTS
(SELECT *
FROM dateranges r
WHERE d.date BETWEEN r.startdate and ISNULL(r.enddate, d.date)
)
CREATE TABLE Dates (
dt DATETIME NOT NULL PRIMARY KEY);
INSERT INTO Dates VALUES('20081204');
INSERT INTO Dates VALUES('20081205');
INSERT INTO Dates VALUES('20090608');
INSERT INTO Dates VALUES('20090609');
-- missing ranges
SELECT DATEADD(DAY, 1, prev) AS start_gap,
DATEADD(DAY, -1, next) AS end_gap,
DATEDIFF(MONTH, DATEADD(DAY, 1, prev),
DATEADD(DAY, -1, next)) AS month_diff
FROM (
SELECT dt AS prev,
(SELECT MIN(dt)
FROM Dates AS B
WHERE B.dt > A.dt) AS next
FROM Dates AS A) AS T
WHERE DATEDIFF(DAY, prev, next) > 1;
-- existing ranges
SELECT MIN(dt) AS start_range,
MAX(dt) AS end_range
FROM (
SELECT dt,
DATEDIFF(DAY, ROW_NUMBER() OVER(ORDER BY dt), dt) AS grp
FROM Dates) AS D
GROUP BY grp;
DROP TABLE Dates;