To speed and simplify aggregation of data queries for a dashboard, I have added these additional date columns as computed columns derived from a date field when the record is inserted.
These 3 are created from (CONVERT([varchar],[DateCreated], (112))) or a substring of such. Persisting works. No problem.
YearUTC
YearMonthUTC
YearMonthDayUTC
However, with these 2, I receive the error
Cannot be persisted because the column is non-deterministic.
WEEK(DATEPART(WEEK, [DateCreated]))
DAYOFWEEK(DATEPART(WEEKDAY, [DateCreated]))
If cannot persist, I'm afraid the cost would be too great with large table (500K rows). Is there a work-around?
EIDT: Sorry, I was not using GetDate() in my real code. I should have changed that to a standard field, [DateCreated]. I updated my code above. Even with a standard date field that is known when the record is created, I get the same error.
Ok, I found a solution to the Week datepart. Use (datepart, ISO_WEEK, [datefield]) instead of (datepart, week, [datefield]) and you can persist the value. One more reason to hate on the nuances of time however, “The numbering systems of different countries/regions might not comply with the ISO standard.”
https://learn.microsoft.com/en-us/sql/t-sql/functions/datepart-transact-sql?view=sql-server-2017
The solution I went with for the DayOfWeek was to create an after update trigger as suggested by #Jeroen_Mostert
Now that you have the week part sorted, you can also create a (much simpler than it would be for the week) function for the weekday bit:
-- Monday is always 1 and so on
create function WeekdayAbsolute(#d datetime)
returns int
with schemabinding, returns null on null input
as
begin
declare #b int;
-- The zero date (i.e. cast(0 as datetime)) was Monday. Therefore...
set #b = 1 + ((datediff(d, #d - #d, #d)) % 7);
if #b > 7 set #b = #b - 7;
return #b;
end;
and use it in the column expression.
Related
I have a problem were I need to insert a set of dates (one row for each date) that come from a date range that comes in to me as an ical format. I also need to insert the date if the date does not exist.
I'm happy manipulating the date formats so converting the incoming 20131201T000000 to smalldatetime is ok and I can build the SQL Not Exits as separate bits but how do I go about listing all the dates between 20131201 and 20140101 and then go about inserting them into the table if Not Exists along with some other data.
One idea I had would be to count the days using DateDiff which will give me a total number of days in the range and then I could DateAdd one day at a time and insert for each date until the DateDiff total is reached. However, it seems to be that its a very messy and long winded way to go about it and I would find it challenging to say the least.
Is there another method that someone could walk me through or point me in the right direction of?
In my opinion you are much better off letting SQL Server determine the set of dates that are missing than to create a set of dates, then pull back all of the existing dates in the table to compare them to your set, then insert the ones that don't match. By doing this you are taking what SQL Server is really, really good at, tossing it in the mud, and re-inventing a wheel that is now quite non-circular.
So I suggest you do this in SQL Server.
Imagine you have an existing table with some dates:
CREATE TABLE dbo.MyTable(TheDate DATE);
And it has some existing days in the specified range:
INSERT dbo.MyTable(TheDate) VALUES
('20131203'),('20131205'),('20131209'),('20131230');
You can easily derive a set without using loops as follows.
CREATE PROCEDURE dbo.GenerateDates
#start CHAR(17),
#end CHAR(17)
AS
BEGIN
SET NOCOUNT ON;
DECLARE #s DATE = LEFT(#start, 8), #e DATE = LEFT(#end, 8);
DECLARE #d SMALLINT = DATEDIFF(DAY, #s, #e);
;WITH x(d) AS
(
SELECT TOP (#d) DATEADD(DAY, number, #s)
FROM master..spt_values
WHERE type = N'P' ORDER BY number
)
-- INSERT dbo.MyTable(TheDate)
SELECT d FROM x WHERE NOT EXISTS
(SELECT 1 FROM dbo.MyTable WHERE TheDate = x.d);
END
GO
Sample usage:
EXEC dbo.GenerateDates
#start = '20131201T000000',
#end = '20140101T000000';
When you are happy this is giving the output you expect, alter the procedure and uncomment the INSERT.
(Also it would be best if you just pass date values from your app, instead of these 17-character strings, but I was working with what you have.)
Well lets see. You have a date called beginDate and date called endDate. Lets get all they days between those dates into list:
List<DateTime> betweenDates = new List<DateTime>();
var currentDate = beginDate;
while(currentDate <= endDate)
{
betweenDates.Add(currentDate);
currentDate = currentDate.AddDays(1);
}
Then lets fetch all the rows that are already in database:
// var datesInDb = SELECT DATE FROM TABLE_X WHERE DATE BETWEEN (beginDate, EndDate);
Now we have dates from database and all the dates that should be in database. Lets substract dates from database from our betweenDates
var datesWhichShouldBeInsertedIntoDB = betweenDates.Any(day => datesInDb.Contains(day) == false);
then insert datesWhichShouldBeInsertedIntoDB
I have been trying this for a while and so far research has not gotten me far other than I need to use something called DATEPART (I have no idea how to use this). My SQL Server is not the greatest.
I have declare #DueDate datetime and through a cursor #DueDate will always been the current row's DueDate (datetime) column. This so far works perfectly without issue.
Now what I am trying to do is get the hour out of #DueDate, check to see if it is 0, and if the hour is 0, set the #DueDate hour to midnight and then update the rows DueDate column with this #DueDate variable.
As stated above I have the cursor and variables all working, I just don't know how to get the hour from #DueDate, check what that hour value is, and update the #DueDate variable so its hour is now midnight. I know how the update the table, that is the easy part.
Thanks in advance!
I think this does exactly what you want without using the cursor.
UPDATE [MyTable]
SET DueDate = DATEADD(dd, 1, DueDate) -- add one day (set to midnight)
WHERE DATEPART(hh, DueDate) = 0 -- do this for dates where hour is zero (midnight)
Note that you should avoid using cursors where possible. SQL development requires a different way of thinking. Don't think about iterations, think atomic. You can do most things in a single statement (one statement can be very long if complicated enough).
I need to write a store procedure that will return a next friday date on a given date? for example - if the date is 05/12/2011, then it should return next friday date as 05/13/2011. If you pass, 05/16/2011, then it should return the date is 5/20/2011 (Friday). If you pass friday as the date, then it should return the same date.
I'd make this a scalar UDF as it is easier to consume the output.
CREATE FUNCTION dbo.GetNextFriday(
#D DATETIME
)
RETURNS DATETIME
WITH SCHEMABINDING, RETURNS NULL ON NULL INPUT
AS
BEGIN
RETURN DATEADD(DAY,(13 - (##DATEFIRST + DATEPART(WEEKDAY,#D)))%7,#D)
END
This is for SQL Server 2008. To use in 2005, just change the date fields to your preference for datetime to date conversions. It also assumes you are not changing the default week begin value.
DECLARE #PassedDate date = '5/21/2011';
SELECT DATEADD(DAY,(CASE DATEPART(DW,#PassedDate) WHEN 7 THEN 6 ELSE 6 - DATEPART(DW,#PassedDate) END),#PassedDate);
Similar to the top answer, but without using ##DATEFIRST in the solution:
DECLARE #Today DATETIME = GETDATE(); -- any date
DECLARE #WeekdayIndex SMALLINT = DATEPART(WEEKDAY, #Today);
DECLARE #DaysUntilFriday SMALLINT = (13 - #WeekdayIndex) % 7;
DECLARE #UpcomingFridayDate DATETIME = DATEADD(DAY, #DaysUntilFriday, #Today);
SELECT #UpcomingFridayDate ;
Great solutions here, but I also recommend looking at time tables: you can generate them easily in Analysis server, and they can be indexed to be very fast, giving you lots of easy ways to get next week days (among other things).
You can find out more about them here
In our case, the same solution would be
Select MIN(PK_Date) from Time Where PK_Date > #SomeDate AND Day_Of_Week= 6
And of course when you're doing this for a large recordset, you can also do joins for maximum speed & efficiency.
SELECT Date_Received, DateAdd(Year, DateDiff(year, Cast('3/01/2010 12:00:00AM' as DateTime) ,
GetDate())-1, Cast('3/01/2010 12:00:00AM' as DateTime)) as minimum_date
FROM [Volunteers].[dbo].[Applications]
WHERE Date_received >= DateAdd(Year, DateDiff(year, Cast('3/01/2010 12:00:00AM' as DateTime),
GetDate())-1, Cast('3/01/2010 12:00:00AM' as DateTime))
In several subqueries where I need to check that a date is within an acceptable range. I need to avoid using a simple constant as I really don't want to update it or a config file each new school year.
My current solution is to enter the date into the query and use some complicated DATEADD tricks to get the current year(or previous year) into the date I am using in the comparison. The exact code is above. Is there a cleaner way for me to do this?
Thanks
Edit
The business requirement is to find applications submitted between 3/01 and 7/31.
We are running background checks and it costs us money for each check we do. Identifying applications submitted during this time period helps us determine if we should do a full, partial or no background check. I will also need to check if dates concerning the previous year.
We will be doing this every year and we need to know if they were in the current year. Maintaining the queries each year to update the dates is not something I want to do.
So I am looking for a good technique to keep the year parts of the dates relevant without having to update the query or a config file.
Old TSQL trick: cast the date to a string in a format that starts with the four-digit year, using substring to take the first four characters of that, cast it back to a date.
Actually, the reason that it's an old TSQL trick is that, if I recall correctly, there wasn't a year() function back then. Given that there's one now, using year( getdate() ) , as others' have answered, is probably the better answer.
SELECT YEAR(GETDATE())
will give you the current year.
If you need to query by month and year a lot, you should also consider making those properties into persisted, computed fields:
ALTER TABLE dbo.Applications
ADD DateReceivedMonth AS MONTH(Date_Received) PERSISTED
ALTER TABLE dbo.Applications
ADD DateReceivedYear AS YEAR(Date_Received) PERSISTED
SQL Server will now extract the MONTH and YEAR part of your Date_Received and place them into two new columns. Those are persisted, e.g. stored along side with your table data. SQL Server will make sure to keep them up to date automatically, e.g. if you change Date_Received, those two new columns will be recomputed (but not on every SELECT).
Now, your queries might be a lot easier:
SELECT (list of fields)
FROM dbo.Applications
WHERE DateReceivedYear = 2010 AND DateReceivedMonth BETWEEN 3 AND 7
Since these are persisted fields, you can even put an index on them to speed up queries against them!
Is there any reason you cannot simply use the Year function?
Select Date_Received
, Year(GetDate())
- Year('3/01/2010 12:00:00AM') - 1
+ Year('3/01/2010 12:00:00AM')
From [Volunteers].[dbo].[Applications]
Where Date_received >= ( Year(GetDate())
- Year('3/01/2010 12:00:00AM') - 1
+ Year('3/01/2080 12:00:00AM') )
Another way would be to use a common-table expression
With Years As
(
Select Year(GetDate()) As CurrentYear
, Year('3/01/2010 12:00:00AM') As ParamYear
, Year('3/01/2080 12:00:00AM') As BoundaryYear
)
Select Date_Received
, CurrentYear - Years.ParamYear - 1 + Years.ParamYear
From [Volunteers].[dbo].[Applications]
Cross Join Years
Where Date_received >= ( Years.CurrentYear
- Years.ParamYear - 1 + Years.BoundaryYear )
TSQL Function returns four digit year dependent on year. This behaves much like the standard SQL YEAR functions [Thomas - nod] which 'CAN' be tweaked using sp_configure on the advanced options, however, the code below is provided as a framework for CUSTOM requirements and can be modified as required. e.g. return as int, use with standard DATETIME functions in SQL to achieve what is needed. e.g. When working with "dirty" data I had to migrate, I used it with the PATINDEX() function to strip non-numeric values etc.
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
-- =============================================
-- Author: Andrew McLintock
-- Create date: 13 July 2016
-- Description: Return 4-digit YEAR
-- =============================================
/*
SELECT Staging.fn_4year('06')
SELECT Staging.fn_4year('56')
SELECT Staging.fn_4year('99')
SELECT Staging.fn_4year('1906')
SELECT Staging.fn_4year('2025')
*/
CREATE FUNCTION Staging.fn_4year
(
#year_in varchar (4)
)
RETURNS varchar(4)
AS
BEGIN
DECLARE #yeartmp int, #Retval varchar(4)
SET #yeartmp = CAST(REPLACE(#year_in,' ','') AS INT)
IF LEN(CAST(#yeartmp AS Varchar)) = 4
BEGIN
Return cast(#yeartmp as varchar(4))
END
IF LEN(#year_in) = 2
BEGIN
SET #Retval = CAST(iif(#yeartmp > 49, #yeartmp + 1900, #yeartmp + 2000) AS varchar(4))
END
RETURN #Retval
END
GO
Consider keeping a set of datetime variables help readability and maintainability. I'm not sure I've captured all your requirements, especially with reference to 'previous year'. If it's as simple as finding applications submitted between 3/01 and 7/31, then this should work. If you need to determine those that were submitted Aug 1 (last year) through Feb 28 (current year), this solution could be modified to suit.
DECLARE #Start smalldatetime, #End smalldatetime, #CurrYear char(4)
SELECT #CurrYear = YEAR(getdate())
SELECT #Start = CAST( 'mar 1 ' + #CurrYear as smalldatetime),
#End = CAST( 'jul 31 ' + #CurrYear as smalldatetime)
SELECT *
FROM Applications
WHERE Date_Received
BETWEEN #Start AND #End
I have to write an SQL view that returns the time part of a datetime column as a string in the format hhmmss (apparently SAP BW doesn't understand hh:mm:ss).
This code is the SAP recommended way to do this, but I think there must be a better, more elegant way to accomplish this
TIME = case len(convert(varchar(2), datepart(hh, timecolumn)))
when 1 then /* Hour Part of TIMES */
case convert(varchar(2), datepart(hh, timecolumn))
when '0' then '24' /* Map 00 to 24 ( TIMES ) */
else '0' + convert(varchar(1), datepart(hh, timecolumn))
end
else convert(varchar(2), datepart(hh, timecolumn))
end
+ case len(convert(varchar(2), datepart(mi, timecolumn)))
when 1 then '0' + convert(varchar(1), datepart(mi, timecolumn))
else convert(varchar(2), datepart(mi, timecolumn))
end
+ case len(convert(varchar(2), datepart(ss, timecolumn)))
when 1 then '0' + convert(varchar(1), datepart(ss, timecolumn))
else convert(varchar(2), datepart(ss, timecolumn))
end
This accomplishes the desired result, 21:10:45 is displayed as 211045.
I'd love for something more compact and easily readable but so far I've come up with nothing that works.
NOTE:
The question says that the column is of datatype DATETIME, not the newer (SQL Server 2008) TIME datatype.
ANSWER:
REPLACE(CONVERT(VARCHAR(8),timecolumn,8),':','')
Let's unpack that.
First, CONVERT formats the time portion of the datetime into a varchar, in format 'hh:mi:ss' (24-hour clock), as specified by the format style value of 8.
Next, the REPLACE function removes the colons, to get varchar in format 'hhmiss'.
That should be sufficient to get a usable string in the format you'd need.
FOLLOW-UP QUESTION
(asked by the OP question)
Is an inline expression faster/less server intensive than a user defined function?
The quick answer is yes. The longer answer is: it depends on several factors, and you really need to measure the performance to determine if that's actually true or not.
I created and executed a rudimentary test case:
-- sample table
create table tmp.dummy_datetimes (c1 datetime)
-- populate with a row for every minute between two dates
insert into tmp.dummy_datetimes
select * from udfDateTimes('2007-01-01','2009-01-01',1,'minute')
(1052641 row(s) affected)
-- verify table contents
select min(c1) as _max
, max(c1) as _min
, count(1) as _cnt
from tmp.dummy_datetimes
_cnt _min _max
------- ----------------------- -----------------------
1052641 2007-01-01 00:00:00.000 2009-01-01 00:00:00.000
(Note, the udfDateTimes function returns the set of all datetime values between two datetime values at the specified interval. In this case, I populated the dummy table with rows for each minute for two entire years. That's on the order of a million ( 2x365x24x60 ) rows.
Now, user defined function that performs the same conversion as the inline expression, using identical syntax:
CREATE FUNCTION [tmp].[udfStrippedTime] (#ad DATETIME)
RETURNS VARCHAR(6)
BEGIN
-- Purpose: format time portion of datetime argument to 'hhmiss'
-- (for performance comparison to equivalent inline expression)
-- Modified:
-- 28-MAY-2009 spencer7593
RETURN replace(convert(varchar(8),#ad,8),':','')
END
NOTE: I know the function is not defined to be DETERMINISTIC. (I think that requires the function be declared with schema binding and some other declaration, like the PRAGMA required Oracle.) But since every datetime value is unique in the table, that shouldn't matter. The function is going to have to executed for each distinct value, even if it were properly declared to be DETERMINISTIC.
I'm not a SQL Server 'user defined function' guru here, so there may be something else I missed that will inadvertently and unnecessarily slow down the function.
Okay.
So for the test, I ran each of these queries alternately, first one, then the other, over and over in succession. The elapsed time of the first run was right in line with the subsequent runs. (Often that's not the case, and we want to throw out the time for first run.) SQL Server Management Studio reports query elapsed times to the nearest second, in format hh:mi:ss, so that's what I've reported here.
-- elapsed times for inline expression
select replace(convert(varchar(8),c1,8),':','') from tmp.dummy_datetimes
00:00:10
00:00:11
00:00:10
-- elapsed times for equivalent user defined function
select tmp.udfStrippedTime(c1) from tmp.dummy_datetimes
00:00:15
00:00:15
00:00:15
For this test case, we observe that the user defined function is on the order of 45% slower than an equivalent inline expression.
HTH
you could use a user-defined function like:
create FUNCTION [dbo].[udfStrippedTime]
(
#dt datetime
)
RETURNS varchar(32)
AS
BEGIN
declare #t varchar(32)
set #t = convert( varchar(32), #dt, 108 )
set #t = left(#t,2) + substring(#t,4,2)
RETURN #t
END
then
select dbo.udfStrippedTime(GETDATE())
the logic for the seconds is left as an exercise for the reader
Here's a question. Does the formatting need to happen on the Db Server? The server itself really only care about, and is optimized for storing the data. Viewing the data is usually the responsibility of hte layer above the Db (in a strictly academic sense, the real world is a bit more messy)
For instance, if you are outputting the result into an ASP.NET page bound to a GridControl you would just specify your DataFormattingString when you bind to the column. If you were using c# to write it to a text file, when you pull the data you would just pass the format string to the .ToString() function.
If you need it to be on the DbServer specifically, then yeah pretty much every solution is going to be messy because the time format you need, while compact and logical, is not a time format the server will recognize so you will need to manipulate it as a string.
This handles the 00 - > 24 conversion
SELECT CASE WHEN DATEPART(hh,timecolumn) = 0
THEN '24' + SUBSTRING(REPLACE(CONVERT(varchar(8),timecolumn, 108),':',''),3,4)
ELSE REPLACE(CONVERT(varchar(8),timecolumn, 108),':','') END
Edit 2: updated to handle 0 --> 24 conversion, and a shorter version:
select replace(left(replace(convert(char,getdate(),8),':',''),2),'00','24') + right(replace(convert(char,getdate(),8),':',''),4)
Back to the slightly longer version :)
SELECT replace(convert(varchar(15),datetimefield, 108), ':','')
from Table
SELECT REPLACE('2009-05-27 12:49:19', ':', '')
2009-05-27 124919