I have a table with the following columns :
sID, start_date and end_date
Some of the values are as follows:
1 1995-07-28 2003-07-20
1 2003-07-21 2010-05-04
1 2010-05-03 2010-05-03
2 1960-01-01 2011-03-01
2 2011-03-02 2012-03-13
2 2012-03-12 2012-10-21
2 2012-10-22 2012-11-08
3 2003-07-23 2010-05-02
I only want the 2nd and 3rd rows in my result as they are the overlapping date ranges.
I tried this but it would not get rid of the first row. Not sure where I am going wrong?
select a.sID from table a
inner join table b
on a.sID = b.sID
and ((b.start_date between a.start_date and a.end_date)
and (b.end_date between a.start_date and b.end_date ))
order by end_date desc
I am trying to do in SQL Server
One way of doing this reasonably efficiently is
WITH T1
AS (SELECT *,
MAX(end_date) OVER (PARTITION BY sID ORDER BY start_date) AS max_end_date_so_far
FROM YourTable),
T2
AS (SELECT *,
range_start = IIF(start_date <= LAG(max_end_date_so_far) OVER (PARTITION BY sID ORDER BY start_date), 0, 1),
next_range_start = IIF(LEAD(start_date) OVER (PARTITION BY sID ORDER BY start_date) <= max_end_date_so_far, 0, 1)
FROM T1)
SELECT SId,
start_date,
end_date
FROM T2
WHERE 0 IN ( range_start, next_range_start );
if you have an index on (sID, start_date) INCLUDE (end_date) this can perform the work with a single ordered scan.
Your logic is not totally correct, although it almost works on your sample data. The specific reason it fails is because between includes the end points, so any given row matches itself. That said, the logic still isn't correct because it doesn't catch this situation:
a-------------a
b----b
Here is correct logic:
select a.*
from table a
where exists (select 1
from table b
where a.sid = b.sid and
a.start_date < b.end_date and
a.end_date > b.start_date and
(a.start_date <> b.start_date or -- filter out the record itself
a.end_date <> b.end_date
)
)
order by a.end_date;
The rule for overlapping time periods (or ranges of any sort) is that period 1 overlaps with period 2 when period 1 starts before period 2 ends and period 1 ends after period 2 starts. Happily, there is no need or use for between for this purpose. (I strongly discourage using between with date/time operands.)
I should note that this version does not consider two time periods to overlap when one ends on the same day another begins. That is easily adjusted by changing the < and > to <= and >=.
Here is a SQL Fiddle.
Related
I have a table of customer transactions called cust_trans where each transaction made by a customer is stored as one row. I have another col called visit_date that contains the transaction date. I would like to filter the customers who transact atleast 3 times a year for the past 2 years.
The data looks like below
Id visit_date
---- ------
1 01/01/2019
1 01/02/2019
1 01/01/2019
1 02/01/2020
1 02/01/2020
1 03/01/2020
1 03/01/2020
2 01/02/2019
3 02/04/2019
I would like to know the customers who visited atleast 3 times every year for the past two years
ie. I want below output.
id
---
1
From the customer table only one person visited atleast 3 times for 2 years.
I tried with below query but it only checks if total visits greater than or equal to 3
select id
from
cust_scan
GROUP by
id
having count(visit_date) >= 3
and year(date(max(visit_date)))-year(date(min(visit_date))) >=2
I would appreciate any help, guidance or suggestions
One option would be to generate a list of distinct ids, cross join it with the last two years, and then bring the original table with a left join. You can then aggregate to count how many visits each id had each year. The final step is to aggregate again, and filter with a having clause
select i.id
from (
select i.id, y.yr, count(c.id) cnt
from (select distinct id from cust_scan) i
cross join (values
(date_trunc('year', current_date)),
(date_trunc('year', current_date) - interval '1' year)
) as y(yr)
left join cust_scan c
on i.id = c.id
and c.visit_date >= y.yr
and c.visit_date < y.yr + interval '1' year
group by i.id, y.yr
) t
group by i.id
having min(cnt) >= 3
Another option would be to use two correlated subqueries:
select distinct id
from cust_scan c
where
(
select count(*)
from cust_scan c1
where
c1.id = c.id
and c1.visit_date >= date_trunc('year', current_date)
and c1.visit_date < date_trunc('year', current_date) + interval '1' year
) >= 3
and (
select count(*)
from cust_scan c1
where
c1.id = c.id
and c1.visit_date >= date_trunc('year', current_date) - interval '1' year
and c1.visit_date < date_trunc('year', current_date)
) >= 3
I assume you mean calendar years. I think I would use two levels of aggregation:
select ct.id
from (select ct.id, year(visit_date) as yyyy, count(*) as cnt
from cust_trans ct
where ct.visit_date >= '2019-01-01' -- or whatever
group by ct.id
) ct
group by ct.id
having count(*) = 2 and -- both year
min(cnt) >= 3; -- at least three transactions
If you want the last two complete years, just change the where clause in the subquery.
You can use a similar idea -- of two aggregations -- if you want the last two years relative to the current date. That would be two full years, rather than 1 and some fraction of the current year.
SELECT oi.created_at, count(oi.id_order_item)
FROM order_item oi
The result is the follwoing:
2016-05-05 1562
2016-05-06 3865
2016-05-09 1
...etc
The problem is that I need information for all days even if there were no id_order_item for this date.
Expected result:
Date Quantity
2016-05-05 1562
2016-05-06 3865
2016-05-07 0
2016-05-08 0
2016-05-09 1
You can't count something that is not in the database. So you need to generate the missing dates in order to be able to "count" them.
SELECT d.dt, count(oi.id_order_item)
FROM (
select dt::date
from generate_series(
(select min(created_at) from order_item),
(select max(created_at) from order_item), interval '1' day) as x (dt)
) d
left join order_item oi on oi.created_at = d.dt
group by d.dt
order by d.dt;
The query gets the minimum and maximum date form the existing order items.
If you want the count for a specific date range you can remove the sub-selects:
SELECT d.dt, count(oi.id_order_item)
FROM (
select dt::date
from generate_series(date '2016-05-01', date '2016-05-31', interval '1' day) as x (dt)
) d
left join order_item oi on oi.created_at = d.dt
group by d.dt
order by d.dt;
SQLFiddle: http://sqlfiddle.com/#!15/49024/5
Friend, Postgresql Count function ignores Null values. It literally does not consider null values in the column you are searching. For this reason you need to include oi.created_at in a Group By clause
PostgreSql searches row by row sequentially. Because an integral part of your query is Count, and count basically stops the query for that row, your dates with null id_order_item are being ignored. If you group by oi.created_at this column will trump the count and return 0 values for you.
SELECT oi.created_at, count(oi.id_order_item)
FROM order_item oi
Group by io.created_at
From TechontheNet (my most trusted source of information):
Because you have listed one column in your SELECT statement that is not encapsulated in the count function, you must use a GROUP BY clause. The department field must, therefore, be listed in the GROUP BY section.
Some info on Count in PostgreSql
http://www.postgresqltutorial.com/postgresql-count-function/
http://www.techonthenet.com/postgresql/functions/count.php
Solution #1 You need Date Table where you stored all date data. Then do a left join depending on period.
Solution #2
WITH DateTable AS
(
SELECT DATEADD(dd, 1, CONVERT(DATETIME, GETDATE())) AS CreateDateTime, 1 AS Cnter
UNION ALL
SELECT DATEADD(dd, -1, CreateDateTime), DateTable.Cnter + 1
FROM DateTable
WHERE DateTable.Cnter + 1 <= 5
)
Generate Temporary table based on your input and then do a left Join.
I have these date ranges that represent start and end dates of subscription. There are no overlaps in date ranges.
Start Date End Date
1/5/2015 - 1/14/2015
1/15/2015 - 1/20/2015
1/24/2015 - 1/28/2015
1/29/2015 - 2/3/2015
I want to identify delays of more than 1 day between any subscription ending and a new one starting. e.g. for the data above, i want the output: 1/24/2015 - 1/28/2015.
How can I do this using a sql query?
Edit : Also there can be multiple gaps in the subscription date ranges but I want the date range after the latest one.
You do this using a left join or not exists:
select t.*
from t
where not exists (select 1
from t t2
where t2.enddate = dateadd(day, -1, t.startdate)
);
Note that this will also give you the first record in the sequence . . . which, strictly speaking, matches the conditions. Here is one solution to that problem:
select t.*
from t cross join
(select min(startdate) as minsd from t) as x
where not exists (select 1
from t t2
where t2.enddate = dateadd(day, -1, t.startdate)
) and
t.startdate <> minsd;
You can also approach this with window functions:
select t.*
from (select t.*,
lag(enddate) over (order by startdate) as prev_enddate,
min(startdate) over () as min_startdate
from t
) t
where minstartdate <> startdate and
enddate <> dateadd(day, -1, startdate);
Also note that this logic assumes that the time periods do not overlap. If they do, a clearer problem statement is needed to understand what you are really looking for.
You can achieve this using window function LAG() that would get value from previous row in ordered set for later comparison in WHERE clause. Then, in WHERE you just apply your "gapping definition" and discard the first row.
SQL FIDDLE - Test it!
Sample data:
create table dates(start_date date, end_date date);
insert into dates values
('2015-01-05','2015-01-14'),
('2015-01-15','2015-01-20'),
('2015-01-24','2015-01-28'), -- gap
('2015-01-29','2015-02-03'),
('2015-02-04','2015-02-07'),
('2015-02-09','2015-02-11'); -- gap
Query
SELECT
start_date,
end_date
FROM (
SELECT
start_date,
end_date,
LAG(end_date, 1) OVER (ORDER BY start_date) AS prev_end_date
FROM dates
) foo
WHERE
start_date IS DISTINCT FROM ( prev_end_date + 1 ) -- compare current row start_date with previous row end_date + 1 day
AND prev_end_date IS NOT NULL -- discard first row, which has null value in LAG() calculation
I assume that there are no overlaps in your data and that there are unique values for each pair. If that's not the case, you need to clarify this.
I am trying to pair up dates that I am getting from my SQL. The output at the moment looks something like this:
start_date end_date
2015-02-02 2015-02-02
2015-02-02 2015-02-03
2015-02-03 2015-02-03
2015-04-12 2015-02-12
I would like the ouput to be paired up so that the smallest and the biggest date of a date group is chosen, so that the output would look like this:
start_date end_date
2015-02-02 2015-02-03
2015-04-12 2015-02-12
Using the first response I get something like this, I believe I have formatted this wrong, I am getting the same date pairs as before, but it does run.
select min(date), max(date)
from (select date,
sum(case when sum(inc) = 0 then 1 else 0 end) over (order by date desc) as grp
from (select t1.datev as date, 1 as inc
from table2 t1,
table3 c,
table4 cr
where t1.datev between date(c.e_start_date) and date(c.e_end_date)
and t1.datev not in (select date(temp.datev) from mdmins11.temp temp where temp.number < 4000 and temp.organisation_id = 11111)
and c.tp_cd in (1,6)
and cr.from_id = c.id
and cr.organisation_id = 11111
union all
select t.datev as date, -1 as inc
from table1 t,
table3 c,
table4 cr
where t.datev between date(c.e_start_date) and date(c.e_end_date)
and t.datev not in (select date(temp.datev) from mdmins11.temp temp where temp.number < 4000 and temp.organisation_id = 11111)
and c.tp_cd in (1,6)
and cr.from_id = c.id
and cr.organisation_id = 11111
) t
group by date
) t
group by grp;
One method is to determine where groups of non-overlapping dates start. For this, you can use not exists. Then count up this flag over all records. This uses window functions. However, this poses problems because you have multiple starts on the same date.
Another method is to keep track of starts and stops and note where the sum is zero. These represent boundaries between groups. The following should work on your data:
select min(date), max(date)
from (select date,
sum(case when sum(inc) = 0 then 1 else 0 end) over (order by date desc) as grp
from (select start_date as date, 1 as inc
from table
union all
select end_date as date, -1 as inc
from table
) t
group by date
) t
group by grp;
This type of problem is made more complicated when duplicate values are allowed on a given date. Given only the dates, this is challenging. With a separate unique id for each row, then there are more robust solutions.
EDIT:
A more robust solution:
select min(start_date), max(end_date)
from (select t.*, sum(StartGroupFlag) over (order by start_date) as grp
from (select t.*,
(case when not exists (select 1
from table t2
where t2.start_date < t.start_date and
t2.end_date >= t.start_date
)
then 1 else 0
end) as StartGroupFlag
from table t
) t
) t
group by grp;
Whole setup on SQL Fiddle: http://sqlfiddle.com/#!4/1fd0e/5
I have some data containing persons id, level and the levels date range like shown below:
PID LVL START_DATE END_DATE
1 1 01.01.14 19.03.14
1 2 20.03.14 15.08.14
1 3 16.08.14 09.10.14
1 4 10.10.14 31.12.14
2 1 01.01.14 31.12.14
3 1 01.01.14 16.01.14
I need to set the start date to the first day of month and the end date to the last day of month. the last day rule applies only if it ist not the last row of data for that person.
what i've don so far:
select
pid, lvl,
trunc(start_date, 'month') as start_date,
case when lead(pid, 1) over (PARTITION BY pid order by end_date) is not null
then last_day(add_months(end_date, -1))
else last_day(end_date)
end as end_date
from date_tbl t;
gives me the desired output:
PID LVL START_DATE END_DATE
1 1 01.01.14 28.02.14
1 2 01.03.14 31.07.14
1 3 01.08.14 30.09.14
1 4 01.10.14 31.12.14
2 1 01.01.14 31.12.14
3 1 01.01.14 31.01.14
BUT: It just works well with my test-data. On my production data on a table containing 25k+ rows of data (witch is not too much data i'd say) it performs really slow.
Can anyone give me a hint how I could improve the query's performance? What indices to set on wich columns for example...? The only indexed column so far is the PID column.
Actually, as I understand, your script produces wrong result if person has only one record (case with pid = 3)
Please, could you try this one?
select
pid,
lvl,
trunc(start_date, 'month') as start_date,
last_day(add_months(end_date, case when lvl = max(lvl) over (partition by pid) then 0 else -1 end)) end_date
from date_tbl t;
I guess that you need to build index for columns (pid, lvl desc)
Ok guys, sorry for waisting your time. To make it short: it was my fault. In my procedure the query above makes a LEFT JOIN to another table in some subquery:
with dates as (
select
pid, lvl,
trunc(start_date, 'month') as start_date,
case when lead(pid, 1) over (PARTITION BY pid order by end_date) is not null
then last_day(add_months(end_date, -1))
else last_day(end_date)
end as end_date
from date_tbl t
),
some_other_table as (
select pid, (...some more columns)
from other_table
)
select * from (
select
b.pid, -- <== this has to be a.pid. b is much bigger than a!
a.start_date,
a.end_date
from dates a left join some_other_table b on a.pid = b.pid
)
The whole query is much bigger.
#jonearles
thx for your comment. "And what is the full query?" helped me to get back on track: split the query into pieces and check again what REALLY slows it down.