SQL: Using DATEADD with bigints - sql

I have some SQL to convert javascript dates to SQL dates which works great. However, I've encoutered some data which is too large and is causing an exception:
Arithmetic overflow error converting expression to data type int
Here is the SQL in question:
DATEADD(MILLISECOND, cast(569337307200000 as bigint) % 1000, DATEADD(SECOND, cast(569337307200000 as bigint) / 1000, '19700101'))
I am running this on SQL Server 2008.

Just do the problematic DATEADD in two steps, starting with a coarser time unit (seconds, minutes, hours etc.), then dropping back to the fine grained one for the remainder.
Avoid going to the level of weeks and months though as that would require actual calendar calculations and we would prefer the system to handle that.
Example below needs to calculate a start time given a (possibly) large current duration in milliseconds.
-- large durations can overflow the integer argument needed for DATEADD
-- so do as two steps subtracting minutes (60000ms) and then remaining milliseconds.
DATEADD(ms, -large_duration_ms%60000, DATEADD(minute, -large_duration_ms/60000, GETDATE()))

One way I got around the Integer overflow issue was to subtract a more recent date from the microtime unix time stamp.
DATEADD(s, (CreationTimeStamp/1000-1384128000), '2013-11-11') AS CreateDate,
This will not fix the OP's problem because they will still overflow the max on the date column.

According to MSDN, in DATEADD (datepart , number , date )
number is an expression that can be resolved to an int that is added
to a datepart of date. User-defined variables are valid. If you
specify a value with a decimal fraction, the fraction is truncated and
not rounded.
Also notice that even if you give number as an integer, depending on your date & datepart, it could overflow the max range of the date which is 31-12-9999 for sql server 2008
Number has to be an integer. Here is a Test Demo

I had the same problem and I wanted to be meet the datetime range of mssql
Minimun datetime: 1753-01-01 00:00:00.000 (-6847804800)
Maximum datetime: 9999-12-31 23:59:59.997 (253402300799)
To achieve this the only solution I found was to loop to use DATEADD with int range values.
So based on this answer: https://stackoverflow.com/a/2904294/687490
CREATE FUNCTION dbo.fn_ConvertToBigDateTime (#Datetime BIGINT)
RETURNS DATETIME
AS
BEGIN
DECLARE #result datetime = Convert(datetime, '01/01/1970');
DECLARE #LocalTimeOffset BIGINT
,#AdjustedLocalDatetime BIGINT
,#MinIntValue INT
,#MaxIntValue INT
,#RemainingSeconds BIGINT;
-- define int limit
SET #MinIntValue = -2147483648;
SET #MaxIntValue = 2147483647;
-- compute the datetime with the offset
SET #LocalTimeOffset = DATEDIFF(second,GETDATE(),GETUTCDATE())
SET #AdjustedLocalDatetime = #Datetime - #LocalTimeOffset
-- going to the future
WHILE(#AdjustedLocalDatetime>#MaxIntValue)
BEGIN
SET #AdjustedLocalDatetime = #AdjustedLocalDatetime - #MaxIntValue;
SELECT #result = Convert(datetime, dateadd(ss, #MaxIntValue,#result));
END
-- going back in the past
WHILE(#AdjustedLocalDatetime<#MinIntValue)
BEGIN
SET #AdjustedLocalDatetime = #AdjustedLocalDatetime - #MinIntValue;
SELECT #result = Convert(datetime, dateadd(ss, #MinIntValue,#result));
END
RETURN (SELECT DATEADD(second,#AdjustedLocalDatetime, #result))
END;
You can then test the function with :
select dbo.fn_ConvertToBigDateTime(-6847804800) as 'min datetime',
dbo.fn_ConvertToBigDateTime(253402300799) as 'max datetime'
Hope it will help.

You can try converting the millis to days, add the days to beginning of EPOCH, and add the ms part to the date at the end. The problem is that you were trying to convert millis to seconds, which can still be too large number for INT for larger dates.
DATEADD(MILLISECOND,
CAST(myLongDateMs AS BIGINT) % 86400000,
DATEADD(day,
CAST(myLongDateMs AS BIGINT) / 86400000,
'19700101'
)
)

I faced this problem too. In my sql statement, the error occurred when the date time value is null.
My solution is to check whether the date time value is null using "CASE When". Only running the arithmetic when it is not null, and the problem solved.

Related

SQL Server DateDiff between two dates returns inaccurate values

I am trying to convert a string into DateTimeOffset (in SQL Server) through a ETL job. Basically, my string would look something like '2017-10-15' and I want this to be converted into a DatetimeOffset (from the current DB server).
SELECT
SWITCHOFFSET(DATEADD(mi, DATEDIFF(MI, GETDATE(), GETUTCDATE()), CAST(#DateInString + ' 00:00:00' AS DATETIMEOFFSET)), DATENAME(tzoffset, SYSDATETIMEOFFSET()))
I have been getting some weird issues with this statement as the final output would fall either +1 / -1 minute than the expected ones. This happens for at least every 10 records/million. I tried to nail down the issue and I could see the problem was with the DATEDIFF() method returning +/-1 minute.
SELECT DATEDIFF(MI, GETDATE(), GETUTCDATE())
This should exactly return -600 (since my DB server UTC is +10). However, it returns either -599 or 601 for few records. I execute them as a single select statement in my Stored Procedure and return it as a parameter.
This is weird on how SQL could detect two different datetime values for GETDATE() and GETUTCDATE() on the same select statement.
Is there a way to force SQL to get exactly same dates in those DATEDIFF parameters or am I missing something here? Thanks in advance
I am using SQL Server 2014 (v12.0).
Stored procedure:
CREATE PROCEDURE dbo.SPConvertDateTimeOffset
#DateInString VARCHAR(10),
#DateTimeOffset_Value DATETIMEOFFSET OUTPUT,
#Datediff_Value INT OUTPUT
AS
BEGIN
-- This line returns +/- 1
SELECT #Datediff_Value = DATEDIFF(MI, GETDATE(), GETUTCDATE())
SELECT #DateTimeOffset_Value = SWITCHOFFSET(DATEADD(mi, #Datediff_Value, CAST(#DateInString + ' 00:00:00' AS DATETIMEOFFSET)), DATENAME(tzoffset, SYSDATETIMEOFFSET()))
END
#GordonLinoff has explained why this happens: since functions are executed at slightly different times, they may return a different minute.
To work around, try this:
DECLARE #DateTimeOffset_Value Datetimeoffset
DECLARE #Datediff_Value INT, #DateInString VARCHAR( 10 )
SET #DateInString = CONVERT( VARCHAR, GETDATE(), 112 )
SET #DateTimeOffset_Value = TODATETIMEOFFSET( #DateInString, DATENAME(tzoffset,SYSDATETIMEOFFSET()))
SET #Datediff_Value = DATEDIFF( MI, #DateInString, #DateTimeOffset_Value )
SELECT #DateInString, #DateTimeOffset_Value, #Datediff_Value
It does not use current date comparisons.
Note: that during the time when day light saving changes you may get a different value from the expected, depending on when exactly the code was run.
Have a look at https://dba.stackexchange.com/questions/28187/how-can-i-get-the-correct-offset-between-utc-and-local-times-for-a-date-that-is for more solutions about how to handle DTS changes.
The two functions are not executed simultaneously. So about 1 time in 100,000 (in your test) the times are on opposite sides of a minute boundary.
If you just want the timezone, you could try:
select datepart(tz, SYSDATETIMEOFFSET())

T-SQL - Arithmetic overflow while working with milliseconds

I have a very simple script in TSQL that tries to convert a timestamp in milliseconds to a DATETIME data type. This also includes the local time offset.
DECLARE #Time AS BIGINT
SET #Time = 1413381394000
SELECT DATEADD(MILLISECOND,
#Time - DATEDIFF(MILLISECOND, GETDATE(), GETUTCDATE()), CAST('1970-01-01 00:00:00' AS DATETIME)) AS [Datetime]
The error it's giving me all the time is:
Arithmetic overflow error converting expression to data type int.
Now, I don't have any explicit int variables in this query, and any CAST() to BIGINT or DECIMAL(13,0) I've did, resulted in the same error.
Which is the wrong part in this query? Is int the default return data type of DATEDIFF()?
I know that I could just divide #Time by 1000 and work with SECONDS instead of MILLISECONDS, I just want to know if there is a way to work directly with milliseconds, since the idea is to use this script as an Inline Table-Valued Function (cannot use scalar ones for other reasons outside this query).
Your calculation for local offset has the potential to be wrong by an hour due to Daylight Savings Time. DATEDIFF(MILLISECOND, GETDATE(), GETUTCDATE()) will only get the current offset and not the offset for the given date. Conversions to and from UTC and local time are generally best handled in application or SQLCLR code due to SQL's lack of functionality for this purpose. See How can I get the correct offset between UTC and local times for a date that is before or after DST?.
In DATEADD (Transact-SQL) Microsoft states that:
number
Is an expression that can be resolved to an int that is added
to a datepart of date. User-defined variables are valid.
If you specify a value with a decimal fraction, the fraction is
truncated and not rounded.
Therefore, you cannot directly work with millisecond values larger than the maximum value for an int which supports a range of -2^31 (-2,147,483,648) to 2^31-1 (2,147,483,647) as stated in int, bigint, smallint, and tinyint (Transact-SQL). You'll have to do to separate date adds and some modulo division.
DECLARE #Time bigint
DECLARE #Seconds int
DECLARE #RemainingMilliseconds int
DECLARE #EpochDate datetime
SET #Time = 1413381394000
SET #EpochDate = '1970-01-01 00:00:00'
SET #Seconds = #Time / 1000
SET #RemainingMilliseconds = #Time % 1000
SELECT DATEADD(MILLISECOND, #RemainingMilliseconds, DATEADD(SECOND,#Seconds, #EpochDate))
DateDiff() does indeed return an int, but I suspect that it's DateAdd() that's giving you the error message.
You'll need to work in that precision, unfortunately, as you said you wanted to avoid, as you're wanting to work in milliseconds.
DATEADD (datepart , number , date )
number Is an expression that can be resolved to an int that is added to a datepart of date. User-defined variables are valid.
You could obviously code your away around it with loops or something, but obviously there's a cost/benefit there that you would need to go through.
The second parameter to DATEADD should be INT not BIGINT. Convert to seconds and then pass to date add.
DECLARE #Time AS BIGINT
SET #Time = 1413381394000
DECLARE #seconds AS INT = #Time / 1000
SELECT DATEADD(SECOND, #seconds, ...)
If you don't want to lose milliseconds, you can add a second step:
DECLARE #milliseconds INT = #Time % 1000
SELECT DATEADD(MILLISECOND, #milliseconds, <date from before>)

Datetime comparison error in sql?

I have used date and time validation for scheduling a report...I have to schedule that reports for future date and time only and not previous date and time..I have used this
declare #Dt varchar(50)
declare #Hr varchar(50)
declare #trandate_time_tmp as TIME(0)
select #trandate_time_tmp = getdate()
set #Dt = DATEDIFF (D,#schedule_date ,#trandate_tmp )
set #Hr = DATEDIFF (S,#schedule_date ,#trandate_time_tmp )
if ( #Dt > 0)
begin
raiserror('Schedule Date should not be earlier than system date',16,1)
return
end
if ( #Hr > 0)
begin
raiserror('Schedule Time should not be earlier than system time',16,1)
return
end
For date part it is checking correctly but for time it is throwing error as
The datediff function resulted in an overflow. The number of dateparts separating two date/time instances is too large. Try to use datediff with a less precise datepart.
Not exactly answering your question, but perhaps a solution to your problem. You don't need to use DATEDIFF and check the results, you could just compare the two dates.
IF ( #schedule_date <= GETDATE() )
BEGIN
RAISERROR('Schedule date should not be earlier than system date', 16, 1)
RETURN
END
I just ran into this same problem when trying to make a Unix timestamp from a date,
Here's an example of what I was trying to do:
select DATEDIFF(second,'1970-01-01','2200-01-11');
It overflows since DATEDIFF is trying to return a signed integer - which can only hold just over 68 years worth of seconds.
In order to get the Unix timestamp (which I need so I can feed it into Sphinx Search), you can get the difference in minutes first, then cast the result as a big integer and then multiply by 60 seconds:
select CAST(DATEDIFF(minute,'1970-01-01','2200-01-11') AS BIGINT) * 60;
Now we should be able to handle dates that vary in difference of up to 4000 years or so. If you need even more room, simply change out minute with bigger and bigger intervals, and change the seconds multiplier accordingly.

tsql getdate conversion

I know that when i do the following, it converts getdate to int
select cast (getdate() as int)
Getdate output on my server is "2010-06-11 14:42:20.100" and the int to which the above command is converting to is 40339. What is this integer? Did this int consider minutes and
seconds? i am confused. Please help.
Regards
Manjot
This number is the number of days since the reference date, which is 01/01/1900. You should get the same result from:
SELECT DATEDIFF(day, 0, GETDATE())
Internally, SQL stores datetimes as two 4-byte integers. The first integer (which you are getting in your statement) represents the number of days since the reference date. The second integer represents the number of 1/300 second intervals since midnight. You can find the second integer by converting first to binary, then to int:
SELECT CONVERT(int, CONVERT(binary(4), GETDATE()))

How can I compare time in SQL Server?

I'm trying to compare time in a datetime field in a SQL query, but I don't know if it's right. I don't want to compare the date part, just the time part.
I'm doing this:
SELECT timeEvent
FROM tbEvents
WHERE convert(datetime, startHour, 8) >= convert(datetime, #startHour, 8)
Is it correct?
I'm asking this because I need to know if 08:00:00 is less or greater than 07:30:00 and I don't want to compare the date, just the time part.
Thanks!
Your compare will work, but it will be slow because the dates are converted to a string for each row. To efficiently compare two time parts, try:
declare #first datetime
set #first = '2009-04-30 19:47:16.123'
declare #second datetime
set #second = '2009-04-10 19:47:16.123'
select (cast(#first as float) - floor(cast(#first as float))) -
(cast(#second as float) - floor(cast(#second as float)))
as Difference
Long explanation: a date in SQL server is stored as a floating point number. The digits before the decimal point represent the date. The digits after the decimal point represent the time.
So here's an example date:
declare #mydate datetime
set #mydate = '2009-04-30 19:47:16.123'
Let's convert it to a float:
declare #myfloat float
set #myfloat = cast(#mydate as float)
select #myfloat
-- Shows 39931,8244921682
Now take the part after the comma character, i.e. the time:
set #myfloat = #myfloat - floor(#myfloat)
select #myfloat
-- Shows 0,824492168212601
Convert it back to a datetime:
declare #mytime datetime
set #mytime = convert(datetime,#myfloat)
select #mytime
-- Shows 1900-01-01 19:47:16.123
The 1900-01-01 is just the "zero" date; you can display the time part with convert, specifying for example format 108, which is just the time:
select convert(varchar(32),#mytime,108)
-- Shows 19:47:16
Conversions between datetime and float are pretty fast, because they're basically stored in the same way.
convert(varchar(5), thedate, 108) between #leftTime and #rightTime
Explanation:
if you have varchar(5) you will obtain HH:mm
if you have varchar(8) you obtain HH:mm ss
108 obtains only the time from the SQL date
#leftTime and #rightTime are two variables to compare
If you're using SQL Server 2008, you can do this:
WHERE CONVERT(time(0), startHour) >= CONVERT(time(0), #startTime)
Here's a full test:
DECLARE #tbEvents TABLE (
timeEvent int IDENTITY,
startHour datetime
)
INSERT INTO #tbEvents (startHour) SELECT DATEADD(hh, 0, GETDATE())
INSERT INTO #tbEvents (startHour) SELECT DATEADD(hh, 1, GETDATE())
INSERT INTO #tbEvents (startHour) SELECT DATEADD(hh, 2, GETDATE())
INSERT INTO #tbEvents (startHour) SELECT DATEADD(hh, 3, GETDATE())
INSERT INTO #tbEvents (startHour) SELECT DATEADD(hh, 4, GETDATE())
INSERT INTO #tbEvents (startHour) SELECT DATEADD(hh, 5, GETDATE())
--SELECT * FROM #tbEvents
DECLARE #startTime datetime
SET #startTime = DATEADD(mi, 65, GETDATE())
SELECT
timeEvent,
CONVERT(time(0), startHour) AS 'startHour',
CONVERT(time(0), #startTime) AS '#startTime'
FROM #tbEvents
WHERE CONVERT(time(0), startHour) >= CONVERT(time(0), #startTime)
Just change convert datetime to time that should do the trick:
SELECT timeEvent
FROM tbEvents
WHERE convert(time, startHour) >= convert(time, #startHour)
if (cast('2012-06-20 23:49:14.363' as time) between
cast('2012-06-20 23:49:14.363' as time) and
cast('2012-06-20 23:49:14.363' as time))
One (possibly small) issue I have noted with the solutions so far is that they all seem to require a function call to process the comparison. This means that the query engine will need to do a full table scan to seek the rows you are after - and be unable to use an index. If the table is not going to get particularly large, this probably won't have any adverse affects (and you can happily ignore this answer).
If, on the other hand, the table could get quite large, the performance of the query could suffer.
I know you stated that you do not wish to compare the date part - but is there an actual date being stored in the datetime column, or are you using it to store only the time? If the latter, you can use a simple comparison operator, and this will reduce both CPU usage, and allow the query engine to use statistics and indexes (if present) to optimise the query.
If, however, the datetime column is being used to store both the date and time of the event, this obviously won't work. In this case if you can modify the app and the table structure, separate the date and time into two separate datetime columns, or create a indexed view that selects all the (relevant) columns of the source table, and a further column that contains the time element you wish to search for (use any of the previous answers to compute this) - and alter the app to query the view instead.
Using float does not work.
DECLARE #t1 datetime, #t2 datetime
SELECT #t1 = '19000101 23:55:00', #t2 = '20001102 23:55:00'
SELECT CAST(#t1 as float) - floor(CAST(#t1 as float)), CAST(#t2 as float) - floor(CAST(#t2 as float))
You'll see that the values are not the same (SQL Server 2005). I wanted to use this method to check for times around midnight (the full method has more detail) in which I was comparing the current time for being between 23:55:00 and 00:05:00.
Adding to the other answers:
you can create a function for trimming the date from a datetime
CREATE FUNCTION dbo.f_trimdate (#dat datetime) RETURNS DATETIME AS BEGIN
RETURN CONVERT(DATETIME, CONVERT(FLOAT, #dat) - CONVERT(INT, #dat))
END
So this:
DECLARE #dat DATETIME
SELECT #dat = '20080201 02:25:46.000'
SELECT dbo.f_trimdate(#dat)
Will return
1900-01-01 02:25:46.000
Use Datepart function: DATEPART(datepart, date)
E.g#
SELECT DatePart(#YourVar, hh)*60) +
DatePart(#YourVar, mi)*60)
This will give you total time of day in minutes allowing you to compare more easily.
You can use DateDiff if your dates are going to be the same, otherwise you'll need to strip out the date as above
You can create a two variables of datetime, and set only hour of date that your need to compare.
declare #date1 datetime;
declare #date2 datetime;
select #date1 = CONVERT(varchar(20),CONVERT(datetime, '2011-02-11 08:00:00'), 114)
select #date2 = CONVERT(varchar(20),GETDATE(), 114)
The date will be "1900-01-01" you can compare it
if #date1 <= #date2
print '#date1 less then #date2'
else
print '#date1 more then #date2'
SELECT timeEvent
FROM tbEvents
WHERE CONVERT(VARCHAR,startHour,108) >= '01:01:01'
This tells SQL Server to convert the current date/time into a varchar using style 108, which is "hh:mm:ss". You can also replace '01:01:01' which another convert if necessary.
I believe you want to use DATEPART('hour', datetime).
Reference is here:
http://msdn.microsoft.com/en-us/library/ms174420.aspx
I don't love relying on storage internals (that datetime is a float with whole number = day and fractional = time), but I do the same thing as the answer Jhonny D. Cano. This is the way all of the db devs I know do it. Definitely do not convert to string. If you must avoid processing as float/int, then the best option is to pull out hour/minute/second/milliseconds with DatePart()
I am assuming your startHour column and #startHour variable are both DATETIME; In that case, you should be converting to a string:
SELECT timeEvent
FROM tbEvents
WHERE convert(VARCHAR(8), startHour, 8) >= convert(VARCHAR(8), #startHour, 8)
below query gives you time of the date
select DateAdd(day,-DateDiff(day,0,YourDateTime),YourDateTime) As NewTime from Table
#ronmurp raises a valid concern - the cast/floor approach returns different values for the same time. Along the lines of the answer by #littlechris and for a more general solution that solves for times that have a minute, seconds, milliseconds component, you could use this function to count the number of milliseconds from the start of the day.
Create Function [dbo].[MsFromStartOfDay] ( #DateTime datetime )
Returns int
As
Begin
Return (
( Datepart( ms , #DateTime ) ) +
( Datepart( ss , #DateTime ) * 1000 ) +
( Datepart( mi , #DateTime ) * 1000 * 60 ) +
( Datepart( hh , #DateTime ) * 1000 * 60 * 60 )
)
End
I've verified that it returns the same int for two different dates with the same time
declare #first datetime
set #first = '1900-01-01 23:59:39.090'
declare #second datetime
set #second = '2000-11-02 23:56:39.090'
Select dbo.MsFromStartOfDay( #first )
Select dbo.MsFromStartOfDay( #second )
This solution doesn't always return the int you would expect. For example, try the below in SQL 2005, it returns an int ending in '557' instead of '556'.
set #first = '1900-01-01 23:59:39.556'
set #second = '2000-11-02 23:56:39.556'
I think this has to do with the nature of DateTime stored as float. You can still compare the two number, though. And when I used this approach on a "real" dataset of DateTime captured in .NET using DateTime.Now() and stored in SQL, I found that the calculations were accurate.
TL;DR
Separate the time value from the date value if you want to use indexes in your search (you probably should, for performance). You can: (1) use function-based indexes or (2) create a new column for time only, index this column and use it in you SELECT clause.
Keep in mind you will lose any index performance boost if you use functions in a SQL's WHERE clause, the engine has to do a scan search. Just run your query with EXPLAIN SELECT... to confirm this. This happens because the engine has to process EVERY value in the field for EACH comparison, and the converted value is not indexed.
Most answers say to use float(), convert(), cast(), addtime(), etc.. Again, your database won't use indexes if you do this. For small tables that may be OK.
It is OK to use functions in WHERE params though (where field = func(value)), because you won't be changing EACH field's value in the table.
In case you want to keep use of indexes, you can create a function-based index for the time value. The proper way to do this (and support for it) may depend on your database engine. Another option is adding a column to store only the time value and index this column, but try the former approach first.
Edit 06-02
Do some performance tests before updating your database to have a new time column or whatever to make use of indexes. In my tests, I found out the performance boost was minimal (when I could see some improvement) and wouldn't be worth the trouble and overhead of adding a new index.