I have a legacy query in which I am looking data for six weeks as shown below. In my below AND condition I get data for past six weeks and it worked fine in 2020 middle and end. But since 2021 started, this stopped working because of obvious subtraction I am doing with 6.
AND data.week_col::integer BETWEEN DATE_PART(w, CURRENT_DATE) - 6 AND DATE_PART(w, CURRENT_DATE) - 1
There is a bug in above query because of which it stopped working in 2021. How can I change above condition so that it can work entire year without any issues and give me data for past 6 weeks.
Update
Below is my query which I am running:
select *,
dateadd(d, - datepart(dow, trunc(CONVERT_TIMEZONE('UTC','PST8PDT',client_date))), trunc(CONVERT_TIMEZONE('UTC','PST8PDT',client_date)) + 6) as day,
date_part(week, day) as week_col
from holder data
where data.week_col::integer BETWEEN DATE_PART(w, CURRENT_DATE) - 6 AND DATE_PART(w, CURRENT_DATE) - 1
client_date column has values like this - 2021-01-15 21:30:00.0. And from that I get value of day column and from day column I get value of
week_col column as shown above.
week_col column has values like 53, 52 .... It's a week number in general.
Because of my AND condition I am getting data for week 1 only but technically I want data for 49, 50, 51, 52, 53 and 1 as it is past six weeks. Can I use day column here to get correct past six weeks?
Would this serve as a solution? I do not know much about the redshirt syntax but I read it supports dateadd(). If you are normalizing client_date to a time zone converted day with no time then why not simply use that in the comparison to the current date converted to the same time zone.
WHERE
client_date BETWEEN
DATEADD(WEEK,-6,trunc(CONVERT_TIMEZONE('UTC','PST8PDT',CURRENT_DATE)))
AND
DATEADD(WEEK,-1,trunc(CONVERT_TIMEZONE('UTC','PST8PDT',CURRENT_DATE)))
If the above logic works out then you may want to convert the -6 and -1 week to variables, if that is supported.
Solution 2
This is a bit more verbose but involves virtualizing a calender table and then joining your current date parameter into the calender data, for markers. Finally, you can join your data against the calender which has been normalized by weeks in time chronologically.
This is SQL Server syntax, however, I am certain it can be converted to RS.
DECLARE #D TABLE(client_date DATETIME)
INSERT #D VALUES
('11/20/2020'),('11/27/2020'),
('12/4/2020'),('12/11/2020'),('12/18/2020'),('12/25/2020'),
('01/8/2021'),('01/8/2021'),('1/15/2021'),('1/22/2021'),('1/29/2021')
DECLARE #Date DATETIME = '1/23/2021'
DECLARE #StartDate DATETIME = '01/01/2010'
DECLARE #NumberOfDays INT = 6000
;WITH R1(N) AS (SELECT 1 FROM (VALUES (1),(1),(1),(1),(1),(1),(1),(1),(1),(1))dt(n)),
R2(N) AS (SELECT 1 FROM R1 a, R1 b),
R3(N) AS (SELECT 1 FROM R2 a, R2 b),
Tally(Number) AS (SELECT ROW_NUMBER() OVER (ORDER BY (SELECT NULL)) FROM R3)
,WithTally AS
(
SELECT CalendarDate = DATEADD(DAY,T.Number,#StartDate)
FROM Tally T
WHERE T.Number < #NumberOfDays
)
,Calendar AS
(
SELECT
CalendarDate,
WeekIndex = DENSE_RANK() OVER(ORDER BY DATEPART(YEAR, CalendarDate), DATEPART(WEEK, CalendarDate))
FROM
WithTally
),
CalendarAlignedWithCurrentDateParamater AS
(
SELECT *
FROM
Calendar
CROSS JOIN (SELECT WeekIndexForToday=WeekIndex FROM Calendar WHERE Calendar.CalendarDate=#Date ) AS X
)
SELECT
D.*,
C.WeekIndex,
C.WeekIndexForToday
FROM
CalendarAlignedWithCurrentDateParamater C
INNER JOIN #D D ON D.client_date = C.CalendarDate
WHERE
C.WeekIndex BETWEEN C.WeekIndexForToday-6 AND C.WeekIndexForToday-1
OPTION (MAXRECURSION 0)
Related
I'm having an issue in a query where SQL Server is throwing the error
Cannot construct data type date, some of the arguments have values which are not valid
when comparing two date objects that themselves are valid.
If I remove the where clause, it resolves without error, but the moment I try to compare them with any relational or equality operator it errors again.
Minimum query to reproduce the issue is as follows:
with Years as
(
select
YEAR(getdate()) + 1 Year,
DATEFROMPARTS(YEAR(getdate()) + 1, 1, 1) FirstOfTheYear,
0 YearOffset
union all
select
Year - 1,
DATEFROMPARTS(Year - 1, 1, 1),
YearOffset + 1
from Years
where YearOffset < 5
),
Months as
(
select 1 Month
union all
select Month + 1
from Months
where Month < 12
),
Days as
(
select 1 Day
union all
select Day + 1
from Days
where Day < 31
),
Dates as
(
select cast(DATEFROMPARTS(Year, Month, Day) as date) Date
from Years
cross join Months
cross join Days
where DAY(EOMONTH(FirstOfTheYear, Month - 1)) >= Day
)
select Dates.Date, cast ('2019-10-01' as date), CAST ('2019-10-11' as date)
from Dates
where Date = cast ('2019-10-01' as date) -- Comment this line out and the error goes away, occurs with any date construction pattern
--where Dates.[Date] >= datefromparts(2019, 10, 01) and Dates.[Date] <= DATEFROMPARTS(2019, 10, 11)
order by date
Commenting out the where clause returns results as expected, confirming that it is specifically the comparison that is triggering this issue.
Additionally, manually creating a handful of dates (first of the year, 2015-2019, the October dates in the query) and querying against that does not cause the error to show.
Edit: I want to emphasize that the code is already handling February and leap years correctly. The output of the Dates CTE is valid and outputs the full range without error. It is only when I reference the date in the where clause that it throws the error
Edit2: I was able to resolve my issue by switching to a different date generation pattern (adding a day, day by day, in a recursive), but I still am curious what causes this error.
The point of a couple of the other answers is that attacking the issue in the manner you are is not necessarily the most efficient way of generating a date's table. Most of the time when constrained with SQL server people will lead someone to use a Tally table for this purpose. Doing so will remain a SET based operation rather than requiring looping or recursion. Which means the recursion limit you mentioned in one of your comments simply doesn't apply.
A Tally table is a set of numeric values that you can then use to generate or produce the values you want. In this case that is approximately 1827 days (5 years + 1 day) but can differ by leap years. The leap years and February are likely the issues within your code. Anyway to generate a tally table you can start with 10 values then cross join till you get to an acceptable number of combinations. 3 cross joins will bring you to 10,000 values and ROW_NUMBER() - 1 can be used to generate a 0 based increment. After which you can use DATEADD() to actually create the dates:
;WITH cteTen AS (
SELECT n FROM (VALUES (0),(0),(0),(0),(0),(0),(0),(0),(0),(0)) T(n)
)
, cteTally AS (
SELECT
N = ROW_NUMBER() OVER (ORDER BY (SELECT NULL)) - 1
FROM
cteTen t10
CROSS JOIN cteTen t100
CROSS JOIN cteTen t1000
CROSS JOIN cteTen t10000
)
, cteStartOfNextYear AS (
SELECT
StartOfNextYear = s.[Date]
,NumOfDaysBetween = DATEDIFF(DAY,DATEADD(YEAR,-5,s.[Date]),s.[Date])
FROM
(VALUES (DATEFROMPARTS(YEAR(GETDATE()) + 1, 1, 1))) s([Date])
)
, cteDates AS (
SELECT
[Date] = DATEADD(DAY,- t.N, s.StartOfNextYear)
FROM
cteStartOfNextYear s
INNER JOIN cteTally t
ON t.N <= NumOfDaysBetween
)
SELECT *
FROM
cteDates
ORDER BY
[Date]
Per our conversation, I see why you would think that EOMONTH() would take care of the issue but it is an order of operations sort of. So the DATEFROMPARTS() portion is analyzed across the entirety of the dataset prior to interpreting the where clause. So it is trying to build the date of 29,30 of Feb. etc. before it is limiting it to the number of days defined by EOMONTH() where clause
I have no idea why you are using code like that to generate days. Why not start at the first date and just add one date at a time?
In any case Feb 29 or 30 or 31 is going to cause an error. You can fix this approach by changing the dates subquery:
Dates as (
select try_convert(date, concat(year, '-' month, '-', day)) as Date
from Years y cross join
Months m cross join
Days
where try_convert(date, concat(year, '-' month, '-', day)) and
DAY(EOMONTH(FirstOfTheYear, Month - 1)) >= Day
)
You're asking DATEFROMPARTS to convert invalid combinations of dates and times. That's what is throwing the error - not your CAST statement.
See Using T-SQL DATEFROMPARTS to return NULL instead of throw error to find your problem dates in general.
Your query creates dates including February 29th, 30th and 31st, as well as the 31st of April, June, September and November.
If you just want all the dates from 2015 through 2020, you can count off a bunch of days and add to a base date. SQL Server will handle the month issues for you:
-- Create up to 16 million integers
WITH N AS (SELECT 0 AS N FROM (VALUES (0),(1),(2),(3),(4),(5),(6),(7)) T(n))
, M AS (SELECT 0 AS N FROM N A, N B, N C, N D, N E, N F, N G, N H)
, Z AS (SELECT ROW_NUMBER() OVER (ORDER BY A.N) AS N FROM M A)
-- Filter only the integers you need; add to a start date
SELECT CAST(DATEADD(DAY, N-1, '2015-01-01') AS DATE) FROM Z
WHERE N < DATEDIFF(DAY, '2015-01-01', '2020-01-01')
I am trying to find the first and last business day for every month since 1986.
Using this, I can find the first day of any given month using, but just that month and it does not take into consideration whether it is a business day or not. To make it easier for now, business day is simply weekdays and does not consider public holiday.
SELECT DATEADD(s,0,DATEADD(mm, DATEDIFF(m,0,getdate()),0))
But I am not able to get the correct business day, so I created a calendar table consisting of all the weekdays and thought that I can extract the min(date) from each month, but I am currently stuck.
Date
---------------
1986-01-01
1986-01-02
1986-01-03
1986-01-06
...and so on
I have tried to get the first day of every month instead, but it does not take into account whether the day is a weekend or not. It just simply give the first day of each month
declare #DatFirst date = '20000101', #DatLast date = getdate();
declare #DatFirstOfFirstMonth date = dateadd(day,1-day(#DatFirst),#DatFirst);
select DatFirstOfMonth = dateadd(month,n,#DatFirstOfFirstMonth)
from (select top (datediff(month,#DatFirstOfFirstMonth,#DatLast)+1)
n=row_number() over (order by (select 1))-1
from (values (1),(1),(1),(1),(1),(1),(1),(1)) a (n)
cross join (values (1),(1),(1),(1),(1),(1),(1),(1)) b (n)
cross join (values (1),(1),(1),(1),(1),(1),(1),(1)) c (n)
cross join (values (1),(1),(1),(1),(1),(1),(1),(1)) d (n)
) x
I am wondering if anyone can perhaps shed some light as to how can I best approach this issue.
If you already have your calendar table with all available dates, then you just need to filter by weekday.
SET DATEFIRST 1 -- 1: Monday, 7: Sunday
SELECT
Year = YEAR(T.Date),
Month = MONTH(T.Date),
FirstBusinessDay = MIN(T.Date),
LastBusinessDay = MAX(T.Date)
FROM
Calendar AS T
WHERE
DATEPART(WEEKDAY, T.Date) BETWEEN 1 AND 5 -- 1: Monday, 5: Friday
GROUP BY
YEAR(T.Date),
MONTH(T.Date)
You should use the query to mark these days on your calendar table, so it's easy to access them afterwards.
This is how you can mix it up with the generation of the calendar table (with recursion).
SET DATEFIRST 1 -- 1: Monday, 7: Sunday
declare
#DatFirst date = '20000101',
#DatLast date = getdate();
;WITH AllDays AS
(
SELECT
Date = #DatFirst
UNION ALL
SELECT
Date = DATEADD(DAY, 1, D.Date)
FROM
AllDays AS D
WHERE
D.Date < #DatLast
),
BusinessLimitsByMonth AS
(
SELECT
Year = YEAR(T.Date),
Month = MONTH(T.Date),
FirstBusinessDay = MIN(T.Date),
LastBusinessDay = MAX(T.Date)
FROM
AllDays AS T
WHERE
DATEPART(WEEKDAY, T.Date) BETWEEN 1 AND 5 -- 1: Monday, 5: Friday
GROUP BY
YEAR(T.Date),
MONTH(T.Date)
)
SELECT
*
FROM
BusinessLimitsByMonth AS B
ORDER BY
B.Year,
B.Month
OPTION
(MAXRECURSION 0) -- 0: Unlimited
If you got already a table with all the weekdays only:
select min(datecol), max(datecol)
from BusinessOnlyCalendar
group by year(datecol), month(datecol)
But you should expand your calendar to include all those calculations you might do on date, like FirstDayOfWeek/Month/Quarter/Year, WeekNumber, etc.
When you got a column in your calendar indicating business day yes/no, it's a simple:
select min(datecol), max(datecol)
from calendar
where businessday = 'y'
group by year(datecol), month(datecol)
I would like to split up the year into 13 periods with 4 weeks in each
52 weeks a year / 4 = 13 even periods
I would like each period to start on a saturday and end on a friday.
It should look like the below image
Obviously I could do this manually, but the dates would change each year and I am looking for a way to automate this with SQL rather than manually do this for each upcoming year
Is there a way to produce this yearly split automatically?
In this previous answer I show an approach to create a numbers/date table. Such a table is very handsome in many places.
With this approach you might try something like this:
CREATE TABLE dbo.RunningNumbers(Number INT NOT NULL,CalendarDate DATE NOT NULL, CalendarYear INT NOT NULL,CalendarMonth INT NOT NULL,CalendarDay INT NOT NULL, CalendarWeek INT NOT NULL, CalendarYearDay INT NOT NULL, CalendarWeekDay INT NOT NULL);
DECLARE #CountEntries INT = 100000;
DECLARE #StartNumber INT = 0;
WITH E1(N) AS(SELECT 1 FROM(VALUES (1),(1),(1),(1),(1),(1),(1),(1),(1),(1))t(N)), --10 ^ 1
E2(N) AS(SELECT 1 FROM E1 a CROSS JOIN E1 b), -- 10 ^ 2 = 100 rows
E4(N) AS(SELECT 1 FROM E2 a CROSS JOIN E2 b), -- 10 ^ 4 = 10,000 rows
E8(N) AS(SELECT 1 FROM E4 a CROSS JOIN E4 b), -- 10 ^ 8 = 10,000,000 rows
CteTally AS
(
SELECT TOP(ISNULL(#CountEntries,1000000)) ROW_NUMBER() OVER(ORDER BY(SELECT NULL)) -1 + ISNULL(#StartNumber,0) As Nmbr
FROM E8
)
INSERT INTO dbo.RunningNumbers
SELECT CteTally.Nmbr,CalendarDate.d,CalendarExt.*
FROM CteTally
CROSS APPLY
(
SELECT DATEADD(DAY,CteTally.Nmbr,{ts'1900-01-01 00:00:00'})
) AS CalendarDate(d)
CROSS APPLY
(
SELECT YEAR(CalendarDate.d) AS CalendarYear
,MONTH(CalendarDate.d) AS CalendarMonth
,DAY(CalendarDate.d) AS CalendarDay
,DATEPART(WEEK,CalendarDate.d) AS CalendarWeek
,DATEPART(DAYOFYEAR,CalendarDate.d) AS CalendarYearDay
,DATEPART(WEEKDAY,CalendarDate.d) AS CalendarWeekDay
) AS CalendarExt;
GO
NTILE - SQL Server 2008+ will create (almost) even chunks.
This the actual query
SELECT *,NTILE(13) OVER(ORDER BY CalendarDate) AS Periode
FROM RunningNumbers
WHERE CalendarWeekDay=6
AND CalendarDate>={d'2017-01-01'} AND CalendarDate <= {d'2017-12-31'};
GO
--Carefull with existing data!
--DROP TABLE dbo.RunningNumbers;
Hint 1: Place indexes!
Hint 2: Read the link about NTILE, especially the Remark-section.
I think this will fit for this case. You might think about using Prdp's approach with ROW_NUMBER() in conncetion with INT division. But - big advantage! - NTILE would allow PARTITION BY CalendarYear.
Hint 3: You might add a column to the table
...where you set the period's number as a fix value. This will make future queries very easy and would allow manual correction on special cases (53rd week..)
Here is one way using Calendar table
DECLARE #start DATE = '2017-04-01',
#end_date DATE = '2017-12-31'
SET DATEFIRST 7;
WITH Calendar
AS (SELECT 1 AS id,
#start AS start_date,
Dateadd(dd, 6, #start) AS end_date
UNION ALL
SELECT id + 1,
Dateadd(week, 1, start_date),
Dateadd(week, 1, end_date)
FROM Calendar
WHERE end_date < #end_date)
SELECT id,
( Row_number()OVER(ORDER BY id) - 1 ) / 4 + 1 AS Period,
start_date,
end_date
FROM Calendar
OPTION (maxrecursion 0)
I have generated dates using Recursive CTE but it is better to create a physical calendar table use it in queries like this
Firstly, you will never get 52 even weeks in a year, there are overlap weeks in most calendar standards. You will occasionally get a week 53.
You can tell SQL to use Saturday as the first day of the week with datefirst, then running a datepart on today's date with getdate() will tell you the week of the year:
SET datefirst 6 -- 6 is Saturday
SELECT datepart(ww,getdate()) as currentWeek
You could then divide this by 4 with a CEILING command to get the 4-week split:
SET datefirst 6
SELECT DATEPART(ww,getdate()) as currentWeek,
CEILING(DATEPART(ww,getdate())/4) as four_week_split
I have two columns (Created and ResolutionDate) in a table with the datetime values
I need to get the difference between the columns created and resolutiondate to get the number of days it took to be resolved from created date.
And also I need to get the result only with the working days or network days i.e., Monday to Friday (not the weekends and holidays).
For example, if I take created:2015-09-22 and resolutiondate: 2015-09-30, then the result should be 6 days, because two days are saturday and sunday between the created and resolutiondate I choose.
Please let me know how can I work it out with SQL.
For calculating the difference between two dates in working days, you can use the following function. Be aware that this will only calculate without weekends, and if you have holidays in the middle, it will calculate them as ordinary days.
public double GetBusinessDays(DateTime startD, DateTime endD)
{
double calcBusinessDays = 1 + ((endD - startD).TotalDays * 5 - (startD.DayOfWeek - endD.DayOfWeek) * 2) / 7;
if ((int)endD.DayOfWeek == 6) calcBusinessDays--;
if ((int)startD.DayOfWeek == 0) calcBusinessDays--;
return calcBusinessDays;
}
Perhaps something like this... The Cross Apply portion could be a UDF
Declare #YourTable table (ID int,Created datetime, ResolutionDate datetime)
Insert Into #YourTable values
(1,'2015-09-22 13:35:38','2015-09-30 17:37:09'),
(2,'2016-02-28 12:55:22','2016-02-29 12:55:44'),
(3,'2015-09-22 13:30:31','2015-09-30 17:37:09')
Select A.*
,B.WorkingDays
From #YourTable A
Cross Apply (
Select WorkingDays=count(*)
From (Select Top (DateDiff(DD,A.Created,A.ResolutionDate)+1) D=DateAdd(DD,Row_Number() over (Order By (Select NULL))-1,cast(cast(A.Created as date) as datetime)) From master..spt_values N1) D
Where D >= A.Created and D<= A.ResolutionDate
and DatePart(DW,D) not in (7,1)
and Cast(D as Date) Not In (Select Date From (Values
('2016-01-01','New Year''s Day'),
('2016-01-18','Martin Luther King, Jr,'),
('2016-02-15','Washington''s Birthday'),
('2016-03-25','Good Friday'),
('2016-05-30','Memorial Day'),
('2016-07-04','Independence Day'),
('2016-09-05','Labor Day'),
('2016-11-24','Thanksgiving'),
('2016-11-25','Black Friday'),
('2016-12-26','Christmas Day')
) as H (Date,Name))
) B
Returns
ID Created ResolutionDate WorkingDays
1 2015-09-22 13:35:38.000 2015-09-30 17:37:09.000 6
2 2016-02-28 12:55:22.000 2016-02-29 12:55:44.000 1
3 2015-09-22 13:30:31.000 2015-09-30 17:37:09.000 6
If you need company dates then have a table with all company work days
select ticket.ID, count(companyWorkDay.dt)
from ticket
left join companyWorkDay
on companyWorkDay.dt between ticket.created and ticket.resolution
group by ticket.ID
For the hassle of populating the table once you get a lot easier queries
You could write a query to enter the weekdays and then just remove the company holidays
My table creates a new record with timestamp daily when an integration is successful. I am trying to create a query that would check (preferably automated) the number of days in a month vs number of records in the table within a time frame.
For example, January has 31 days, so i would like to know how many days in january my process was not successful. If the number of records is less than 31, than i know the job failed 31 - x times.
I tried the following but was not getting very far:
SELECT COUNT (DISTINCT CompleteDate)
FROM table
WHERE CompleteDate BETWEEN '01/01/2015' AND '01/31/2015'
Every 7 days the system executes the job twice, so i get two records on the same day, but i am trying to determine the number of days that nothing happened (failures), so i assume some truncation of the date field is needed?!
One way to do this is to use a calendar/date table as the main source of dates in the range and left join with that and count the number of null values.
In absence of a proper date table you can generate a range of dates using a number sequence like the one found in the master..spt_values table:
select count(*) failed
from (
select dateadd(day, number, '2015-01-01') date
from master..spt_values where type='P' and number < 365
) a
left join your_table b on a.date = b.CompleteDate
where b.CompleteDate is null
and a.date BETWEEN '01/01/2015' AND '01/31/2015'
Sample SQL Fiddle (with count grouped by month)
Assuming you have an Integers table*. This query will pull all dates where no record is found in the target table:
declare #StartDate datetime = '01/01/2013',
#EndDate datetime = '12/31/2013'
;with d as (
select *, date = dateadd(d, i - 1 , #StartDate)
from dbo.Integers
where i <= datediff(d, #StartDate, #EndDate) + 1
)
select d.date
from d
where not exists (
select 1 from <target> t
where DATEADD(dd, DATEDIFF(dd, 0, t.<timestamp>), 0) = DATEADD(dd, DATEDIFF(dd, 0, d.date), 0)
)
Between is not safe here
SELECT 31 - count(distinct(convert(date, CompleteDate)))
FROM table
WHERE CompleteDate >= '01/01/2015' AND CompleteDate < '02/01/2015'
You can use the following query:
SELECT DATEDIFF(day, t.d, dateadd(month, 1, t.d)) - COUNT(DISTINCT CompleteDate)
FROM mytable
CROSS APPLY (SELECT CAST(YEAR(CompleteDate) AS VARCHAR(4)) +
RIGHT('0' + CAST(MONTH(CompleteDate) AS VARCHAR(2)), 2) +
'01') t(d)
GROUP BY t.d
SQL Fiddle Demo
Explanation:
The value CROSS APPLY-ied, i.e. t.d, is the ANSI string of the first day of the month of CompleteDate, e.g. '20150101' for 12/01/2015, or 18/01/2015.
DATEDIFF uses the above mentioned value, i.e. t.d, in order to calculate the number of days of the month that CompleteDate belongs to.
GROUP BY essentially groups by (Year, Month), hence COUNT(DISTINCT CompleteDate) returns the number of distinct records per month.
The values returned by the query are the differences of [2] - 1, i.e. the number of failures per month, for each (Year, Month) of your initial data.
If you want to query a specific Year, Month then just simply add a WHERE clause to the above:
WHERE YEAR(CompleteDate) = 2015 AND MONTH(CompleteDate) = 1