I'm working on a recurrence application for events. I have a date range of say, January 1 2010 to December 31 2011. I want to return all of the 3rd Thursdays (arbitrary) of the each month, efficiently. I could do this pretty trivially in code, the caveat is that it must be done in a stored procedure. Ultimately I'd want something like:
CALL return_dates(event_id);
That event_id has a start_date of 1/1/2010 and end_date of 12/31/2011. Result set would be something like:
1/20/2010
2/14/2010
3/17/2010
4/16/2010
5/18/2010
etc.
I'm just curious what the most efficient method of doing this would be, considering I might end up with a very large result set in my actual usage.
One idea that comes to mind - you can create a table and store the dates you're interested in there.
Ok, I haven't tested it, but I think the most efficient way of doing it is via a tally table which is a useful thing to have in the db anyway:
IF EXISTS (SELECT * FROM sys.objects
WHERE object_id = OBJECT_ID(N'[dbo].[num_seq]') AND type in (N'U'))
DROP TABLE [dbo].[num_seq];
SELECT TOP 100000 IDENTITY(int,1,1) AS n
INTO num_seq
FROM MASTER..spt_values a, MASTER..spt_values b;
CREATE UNIQUE CLUSTERED INDEX idx_1 ON num_seq(n);
You can then use this to build up the date range between the two dates. It's fast because
it just uses the index (in fact often faster than a loop, so I'm led to believe)
create procedure getDates
#eventId int
AS
begin
declare #startdate datetime
declare #enddate datetime
--- get the start and end date, plus the start of the month with the start date in
select #startdate=startdate,
#enddate=enddate
from events where eventId=#eventId
select
#startdate+n AS date,
from
dbo.num_seq tally
where
tally.n<datediff(#monthstart, #enddate) and
Datepart(dd,#startdate+n) between 15 and 21 and
Datepart(dw, #startdate+n) = '<day>'
Aside from getting the start and end dates, the third x id each month must be between the 15th and the 21st inclusive.
The day names in that range must be unique, so we can locate it straight away.
If you wanted the second dayname, just modify the range appropriately or use a parameter to calculate it.
It constucts a date table using the startdate, and then adding days on (via the list of numbers in the tally table) until it reaches the end date.
Hope it helps!
Related
I'm a bit confused if there is a simple way to do this.
I have a field called receipt_date in my data table and I wish to add 10 working days to this (with bank holidays).
I'm not sure if there is any sort of query I could use to join onto this table from my original to calculate 10 working days from this, I've tried a few sub queries but I couldn't get it right or perhaps its not possible to do this. I didn't know if there was a way to extract the 10th rowcount after the receipt date to get the calendar date if I only include 'Y' into the WHERE?
Any help appreciated.
This is making several assumptions about your data, because we have none. One method, however, would be to create a function, I use a inline table value function here, to return the relevant row from your calendar table. Note that this assumes that the number of days must always be positive, and that if you provide a date that isn't a working day that day 0 would be the next working day. I.e. adding zero working days to 2021-09-05 would return 2021-09-06, or adding 3 would return 2021-09-09. If that isn't what you want, this should be more than enough for you to get there yourself.
CREATE FUNCTION dbo.AddWorkingDays (#Days int, #Date date)
RETURNS TABLE AS
RETURN
WITH Dates AS(
SELECT CalendarDate,
WorkingDay
FROM dbo.CalendarTable
WHERE CalendarDate >= #Date)
SELECT CalendarDate
FROM Dates
WHERE WorkingDay = 1
ORDER BY CalendarDate
OFFSET #Days ROWS FETCH NEXT 1 ROW ONLY;
GO
--Using the function
SELECT YT.DateColumn,
AWD.CalendarDate AS AddedWorkingDays
FROM dbo.YourTable YT
CROSS APPLY dbo.AddWorkingDays(10,YT.DateColumn) AWD;
I have a table that acts as a message log, with the two key tables being TIMESTAMP and TEXT. I'm working on a query that grabs all alerts (from TEXT) for the past 30 days (based on TIMESTAMP) and gives a daily average for those alerts.
Here is the query so far:
--goback 30 days start at midnight
declare #olderdate as datetime
set #olderdate = DATEADD(Day, -30, DATEDIFF(Day, 0, GetDate()))
--today at 11:59pm
declare #today as datetime
set #today = dateadd(ms, -3, (dateadd(day, +1, convert(varchar, GETDATE(), 101))))
print #today
--Grab average alerts per day over 30 days
select
avg(x.Alerts * 1.0 / 30)
from
(select count(*) as Alerts
from MESSAGE_LOG
where text like 'The process%'
and text like '%has alerted%'
and TIMESTAMP between #olderdate and #today) X
However, I want to add something that checks whether there were any alerts for a day and, if there are no alerts for that day, doesn't include it in the average. For example, if there are 90 alerts for a month but they're all in one day, I wouldn't want the average to be 3 alerts per day since that's clearly misleading.
Is there a way I can incorporate this into my query? I've searched for other solutions to this but haven't been able to get any to work.
This isn't written for your query, as I don't have any DDL or sample data, thus I'm going to provide a very simple example instead of how you would do this.
USE Sandbox;
GO
CREATE TABLE dbo.AlertMessage (ID int IDENTITY(1,1),
AlertDate date);
INSERT INTO dbo.AlertMessage (AlertDate)
VALUES('20190101'),('20190101'),('20190105'),('20190110'),('20190115'),('20190115'),('20190115');
GO
--Use a CTE to count per day:
WITH Tots AS (
SELECT AlertDate,
COUNT(ID) AS Alerts
FROM dbo.AlertMessage
GROUP BY AlertDate)
--Now the average
SELECT AVG(Alerts*1.0) AS DayAverage
FROM Tots;
GO
--Clean up
DROP TABLE dbo.AlertMessage;
You're trying to compute a double-aggregate: The average of daily totals.
Without using a CTE, you can try this as well, which is generalized a bit more to work for multiple months.
--get a list of events per day
DECLARE #Event TABLE
(
ID INT NOT NULL IDENTITY(1, 1)
,DateLocalTz DATE NOT NULL--make sure to handle time zones
,YearLocalTz AS DATEPART(YEAR, DateLocalTz) PERSISTED
,MonthLocalTz AS DATEPART(MONTH, DateLocalTz) PERSISTED
)
/*
INSERT INTO #Event(EntryDateLocalTz)
SELECT DISTINCT CONVERT(DATE, TIMESTAMP)--presumed to be in your local time zone because you did not specify
FROM dbo.MESSAGE_LOG
WHERE UPPER([TEXT]) LIKE 'THE PROCESS%' AND UPPER([TEXT]) LIKE '%HAS ALERTED%'--case insenitive
*/
INSERT INTO #Event(DateLocalTz)
VALUES ('2018-12-31'), ('2019-01-01'), ('2019-01-01'), ('2019-01-01'), ('2019-01-12'), ('2019-01-13')
--get average number of alerts per alerting day each month
-- (this will not return months with no alerts,
-- use a LEFT OUTER JOIN against a month list table if you need to include uneventful months)
SELECT
YearLocalTz
,MonthLocalTz
,AvgAlertsOfAlertingDays = AVG(CONVERT(REAL, NumDailyAlerts))
FROM
(
SELECT
YearLocalTz
,MonthLocalTz
,DateLocalTz
,NumDailyAlerts = COUNT(*)
FROM #Event
GROUP BY YearLocalTz, MonthLocalTz, DateLocalTz
) AS X
GROUP BY YearLocalTz, MonthLocalTz
ORDER BY YearLocalTz ASC, MonthLocalTz ASC
Some things to note in my code:
I use PERSISTED columns to get the month and year date parts (because I'm lazy when populating tables)
Use explicit CONVERT to escape integer math that rounds down decimals. Multiplying by 1.0 is a less-readable hack.
Use CONVERT(DATE, ...) to round down to midnight instead of converting back and forth between strings
Do case-insensitive string searching by making everything uppercase (or lowercase, your preference)
Don't subtract 3 milliseconds to get the very last moment before midnight. Change your semantics to interpret the end of a time range as exclusive, instead of dealing with the precision of your datatypes. The only difference is using explicit comparators (i.e. use < instead of <=). Also, DATETIME resolution is 1/300th of a second, not 3 milliseconds.
Avoid using built-in keywords as column names (i.e. "TEXT"). If you do, wrap them in square brackets to avoid ambiguity.
Instead of dividing by 30 to get the average, divide by the count of distinct days in your results.
select
avg(x.Alerts * 1.0 / x.dd)
from
(select count(*) as Alerts, count(distinct CAST([TIMESTAMP] AS date)) AS dd
...
I have a table storing a datetime column, which is indexed. I'm trying to find a way to compare ONLY the month and day (ignores the year totally).
Just for the record, I would like to say that I'm already using MONTH() and DAY(). But I'm encountering the issue that my current implementation uses Index Scan instead of Index Seek, due to the column being used directly in both functions to get the month and day.
There could be 2 types of references for comparison: a fixed given date and today (GETDATE()). The date will be converted based on time zone, and then have its month and day extracted, e.g.
DECLARE #monthValue DATETIME = MONTH(#ConvertDateTimeFromServer_TimeZone);
DECLARE #dayValue DATETIME = DAY(#ConvertDateTimeFromServer_TimeZone);
Another point is that the column stores datetime with different years, e.g.
1989-06-21 00:00:00.000
1965-10-04 00:00:00.000
1958-09-15 00:00:00.000
1965-10-08 00:00:00.000
1942-01-30 00:00:00.000
Now here comes the problem. How do I create a SARGable query to get the rows in the table that match the given month and day regardless of the year but also not involving the column in any functions? Existing examples on the web utilise years and/or date ranges, which for my case is not helping at all.
A sample query:
Select t0.pk_id
From dob t0 WITH(NOLOCK)
WHERE ((MONTH(t0.date_of_birth) = #monthValue AND DAY(t0.date_of_birth) = #dayValue))
I've also tried DATEDIFF() and DATEADD(), but they all end up with an Index Scan.
Adding to the comment I made, on a Calendar Table.
This will, probably, be the easiest way to get a SARGable query. As you've discovered, MONTH([YourColumn]) and DATEPART(MONTH,[YourColumn]) both cause your query to become non-SARGable.
Considering that all your columns, at least in your sample data, have a time of 00:00:00 this "works" to our advantage, as they are effectively just dates. This means we can easily JOIN onto a Calendar Table using something like:
SELECT dob.[YourColumn]
FROM dob
JOIN CalendarTable CT ON dob.DateOfBirth = CT.CalendarDate;
Now, if we're using the table from the above article, you will have created some extra columns (MonthNo and CDay, however, you can call them whatever you want really). You can then add those columns to your query:
SELECT dob.[YourColumn]
FROM dob
JOIN CalendarTable CT ON dob.DateOfBirth = CT.CalendarDate
WHERE CT.MonthNo = #MonthValue
AND CT.CDay = #DayValue;
This, as you can see, is a more SARGable query.
If you want to deal with Leap Years, you could add a little more logic using a CASE expression:
SELECT dob.[YourColumn]
FROM dob
JOIN CalendarTable CT ON dob.DateOfBirth = CT.CalendarDate
WHERE CT.MonthNo = #MonthValue
AND CASE WHEN DATEPART(YEAR, GETDATE()) % 4 != 0 AND CT.CDat = 29 AND CT.MonthNo = 2 THEN 28 ELSE CT.Cdat END = #DayValue;
This treats someone's birthday on 29 February as 28 February on years that aren't leap years (when DATEPART(YEAR, GETDATE()) % 4 != 0).
It's also, probably, worth noting that it'll likely be worth while changing your DateOfBirth Column to a date. Date of Births aren't at a given time, only on a given date; this means that there's no implicit conversion from datetime to date on your Calendar Table.
Edit: Also, just noticed, why are you using NOLOCK? You do know what that does, right..? Unless you're happy with dirty reads and ghost data?
So I'm new to SQL (I believe it's T-SQL) and I'm trying to convert a function I used in Excel to SQL.
L2 becomes Column 1
G2 becomes Column 2
=(INT(L2)-INT(G2))*("17:00"-"08:45")+MEDIAN(MOD(L2,1),"17:00","08:45")-MEDIAN(MOD(G2,1),"17:00","08:45")
What this does is calculate the business hours worked between 8:45AM and 05:00PM.
If work goes from 4:00PM to 9:00AM the next day, the result should be 01:15:00.
If it goes over several days (4:00PM on the 1st to 9:00AM on the 4th) it should be 17:45:00.
I'd prefer not to have a separate function because I don't know how to use them as I'm quite new to this - I'd prefer to have it as something I can write within the SELECT * , <code here here> FROM db.name section.
Thanks in advance
I know you said you don't want this in a function, but they really aren't hard to use and the logic you require for this is too complex in SQL Server to be sensibly contained inline (Though it can be, if you really want to be that guy).
This function has no error handling if any of your parameters are not suitable, though I will leave that up to you as a learning exercise on NULL values, process flows and fully thinking through all the possibilities that you may need to deal with:
-- This bit creates your function. You can rename the function from fnWorkingDays to anything you want, though try to keep your naming conventions sensible:
create function fnWorkingDays(#Start datetime
,#End datetime
)
returns decimal(10,2)
as
begin
-- Declare the start and end times of your working day:
declare #WorkingStart time = '08:45:00.000'
declare #WorkingEnd time = '17:00:00.000'
-- Work out the number of minutes outside the working day in 24 Hour Notation:
declare #OvernightMinutes int = datediff(minute -- Work out the difference in minutes,
,cast(#workingend as datetime) -- between the end of the working day (CASTing a TIME as DATETIME gives you 1900-01-01 17:00:00)
,dateadd(d,1,cast(#WorkingStart as datetime)) -- and the start of the next working day (CAST the TIME value as DATETIME [1900-01-01 08:45:00] and then add a day to it [1900-01-02 08:45:00])
)
-- There is no need to retain the minutes that fall outside your Working Day, to if the very start or very end of your given period fall outside your Working Day, discard those minutes:
declare #TrueStart datetime = (select case when cast(#Start as time) < #WorkingStart
then dateadd(d,datediff(d,0,#Start),0) + cast(#WorkingStart as datetime)
else #Start
end
)
declare #TrueEnd datetime = (select case when cast(#End as time) > #WorkingEnd
then dateadd(d,datediff(d,0,#End),0) + cast(#WorkingEnd as datetime)
else #End
end
)
-- You can now calculate the number of minutes in your true working period, and then subtract the total overnight periods in minutes to get your final value.
-- So firstly, if your Working Period is not long enough to stretch over two days, there is not need to do any more than calculate the difference between the True Start and End:
return (select case when datediff(minute,#Start,#End) < #OvernightMinutes
then datediff(minute,#TrueStart,#TrueEnd)
-- If you do need to calculate over more than one day, calculate the total minutes between your True Start and End, then subtract the number of Overnight Minutes multiplied by the number of nights.
-- This works because DATEDIFF calculated the number of boundaries crossed, so when using DAYS, it actually counts the number of midnights between your two dates:
else (datediff(minute,#TrueStart,#TrueEnd) - (datediff(d,#TrueStart,#TrueEnd) * #OvernightMinutes))/1440.
-- If you want to return your value in a slightly different format, you could use variations of these two, though you will need to change the RETURNS DECIMAL(10,2) at the top to RETURNS NVARCHAR(25) if you use the last one:
-- else datediff(minute,#TrueStart,#TrueEnd) - (datediff(d,#TrueStart,#TrueEnd) * #OvernightMinutes)
-- else cast((datediff(minute,#TrueStart,#TrueEnd) - (datediff(d,#TrueStart,#TrueEnd) * #OvernightMinutes))/60 as nvarchar(5)) + ' Hours ' + cast((datediff(minute,#TrueStart,#TrueEnd) - (datediff(d,#TrueStart,#TrueEnd) * #OvernightMinutes))%60 as nvarchar(5)) + ' Minutes'
end
)
end
go
And this is how you call the function:
select dbo.fnWorkingDays('2016-09-04 12:00:00.000', '2016-09-06 12:10:00.000') as WorkingDays
You can replace the two DATETIME values about with the appropriate column names to get your desired result inline:
select dbo.fnWorkingDays(Dates.StartDate, Dates.EndDate) as WorkingDays
from (select '2016-09-04 12:00:00.000' as StartDate
,'2016-09-06 12:10:00.000' as EndDate
) as Dates
I'm trying to model a DateRange concept for a reporting application. Some date ranges need to be absolute, March 1, 2011 - March 31, 2011. Others are relative to current date, Last 30 Days, Next Week, etc. What's the best way to store that data in SQL table?
Obviously for absolute ranges, I can have a BeginDate and EndDate. For relative ranges, having an InceptionDate and an integer RelativeDays column makes sense. How do I incorporate both of these ideas into a single table without implementing context into it, ie have all four columns mentioned and use XOR logic to populate 2 of the 4.
Two possible schemas I rejected due to having context-driven columns:
CREATE TABLE DateRange
(
BeginDate DATETIME NULL,
EndDate DATETIME NULL,
InceptionDate DATETIME NULL,
RelativeDays INT NULL
)
OR
CREATE TABLE DateRange
(
InceptionDate DATETIME NULL,
BeginDaysRelative INT NULL,
EndDaysRelative INT NULL
)
Thanks for any advice!
I don't see why your second design doesn't meet your needs unless you're in the "no NULLs never" camp. Just leave InceptionDate NULL for "relative to current date" choices so that your application can tell them apart from fixed date ranges.
(Note: not knowing your DB engine, I've left date math and current date issues in pseudocode. Also, as in your question, I've left out any text description and primary key columns).
Then, either create a view like this:
CREATE VIEW DateRangesSolved (Inception, BeginDays, EndDays) AS
SELECT CASE WHEN Inception IS NULL THEN Date() ELSE Inception END,
BeginDays,
EndDays,
FROM DateRanges
or just use that logic when you SELECT from the table directly.
You can even take it one step further:
CREATE VIEW DateRangesSolved (BeginDate, EndDate) AS
SELECT (CASE WHEN Inception IS NULL THEN Date() ELSE Inception END + BeginDays),
(CASE WHEN Inception IS NULL THEN Date() ELSE Inception END + EndDays)
FROM DateRanges
Others are relative to current date, Last 30 Days, Next Week, etc.
What's the best way to store that data in SQL table?
If you store those ranges in a table, you have to update them every day. In this case, you have to update each row differently every day. That might be a big problem; it might not.
There usually aren't many rows in that kind of table, often less than 50. The table structure is obvious. Updating should be driven by a cron job (or its equivalent), and you should run very picky exception reports every day to make sure things have been updated correctly.
Normally, these kinds of reports should produce no output if things are fine. You have the added complication that driving such a report from cron will produce no output if cron isn't running. And that's not fine.
You can also create a view, which doesn't require any maintenance. With a few dozen rows, it might be slower than a physical table, but it might still fast enough. And it eliminates all maintenance and administrative work for these ranges. (Check for off-by-one errors, because I didn't.)
create view relative_date_ranges as
select 'Last 30 days' as range_name,
(current_date - interval '30' day)::date as range_start,
current_date as range_end
union all
select 'Last week' as range_name,
(current_date - interval '7' day)::date as range_start,
current_date as range_end
union all
select 'Next week' as range_name,
(current_date + interval '7' day)::date as range_start,
current_date as range_end
Depending on the app, you might be able to treat your "absolute" ranges the same way.
...
union all
select 'March this year' as range_name,
(extract(year from current_date) || '-03-01')::date as range_start,
(extract(year from current_date) || '-03-31')::date as range_end
Put them in separate tables. There is absolutely no reason to have them in a single table.
For the relative dates, I would go so far as to simply make the table the parameters you need for the date functions, i.e.
CREATE TABLE RelativeDate
(
Id INT Identity,
Date_Part varchar(25),
DatePart_Count int
)
Then you can know that it is -2 WEEK or 30 DAY variance and use that in your logic.
If you need to see them both at the same time, you can combine them LOGICALLY in a query or view without needing to mess up your data structure by cramming different data elements into the same table.
Create a table containing the begindate and the offset. The precision of the offset is up to you to decide.
CREATE TABLE DateRange(
BeginDate DATETIME NOT NULL,
Offset int NOT NULL,
OffsetLabel varchar(100)
)
to insert into it:
INSERT INTO DateRange (BeginDate, Offset, OffsetLabel)
select '20110301', DATEDIFF(sec, '20110301', '20110331'), 'March 1, 2011 - March 31, 2011'
Last 30 days
INSERT INTO DateRange (BeginDate, Duration, OffsetLabel)
select '20110301', DATEDIFF(sec, current_timestamp, DATEADD(day, -30, current_timestamp)), 'Last 30 Days'
To display the values later:
select BeginDate, EndDate = DATEADD(sec, Offset, BeginDate), OffsetLabel
from DateRange
If you want to be able to parse the "original" vague descriptions you will have to look for a "Fuzzy Date" or "Approxidate" function. (There exists something like this in the git source code. )