I am trying to calculate the difference in seconds between 2 dates in hive. I found that one of the records is being calculated incorrectly, and I can't understand why or how to fix it.
The example is as follows:
select '2020-03-08 03:00:48' as stop_time,
UNIX_TIMESTAMP('2020-03-08 03:00:48') as stop_timestamp,
'2020-03-08 02:45:03' as start_time,
UNIX_TIMESTAMP('2020-03-08 02:45:03') as start_timestamp,
UNIX_TIMESTAMP('2020-03-08 03:00:48') - UNIX_TIMESTAMP('2020-03-08 02:45:03') as difference
I am getting a result of -2,655 instead of +945
Any advise?
Thank you!
This happens because of daylight savings time change. The place where you're located has day lights savings time that changed during 8 March, 2020 at the same hour. So, it is calculating different timestamp for start_timestamp.
Related
is it possible to have date_diff with specific start and end time?
let say my store are open from 8AM - 10PM, which is 14 Hours.
and I have a lot of stuff to sell during that time. One of the SKU is out of stock from 2022-11-01 06.00 PM until tomorrow 2022-11-02 11.00 AM.
Instead of calculate 24 hours, I just want to calculate only from opening store until it closed or until its restock. Meaning from 6PM to 11AM is 8 Hours
my query
select date_diff('2022-11-02 11.00 AM', '2022-11-02 06.00 PM', hour) from table
with the result 17 hours instead of 8 hours
There isn't a way to configure DATE_DIFF to do this for you, but it's possible to do what you want, with some effort.
You should convert your dates to timestamps (TIMESTAMP(yourdate) or CAST(yourdate AS TIMESTAMP)) and use TIMESTAMP_DIFF instead.
This will allow you to work with smaller intervals than days.
For your calculation, you ultimately need to find the total time difference between the two timestamps and then subtract the out-of-hours timeframe.
However, calculating the latter is not as simple as taking the difference in days and multiplying by 8 hours (10pm-6am), because your out-of-hours calculation has to account for weekends and possibly holidays etc. Hence it can get quite complex, which is where the solution in my first link might come in.
I have a grafana chart showing the data of the last 24 hours
But the data does not fit the time axis. There is missing 2 hours in the beginning of the 24 hour period. And the last value at 21:27:57 is 66.74 but at this time it was 73.50.
The time axis seems to be shifted by 2 hours. The data at time x shows the data of time x-2h.
The timestamp (datetime) in the SQL database is correct.
EDIT:
Changing the timezone doesn't help much. Using UTC (which is wrong for me) the most recent time on the time axis is about 20:40 (wrong)
Using UTC+2 (which fits my timezone) the most recent time is about 22:40, the correct local time when taking the screenshot.
The data is not affected and there is still 2 hours missing in the 24 hour period. And still the most recent value in the chart shows the value of 2 hours ago.
I don't really understand why, but I figured out that there is a UNIX_TIMESTAMP() needed:
SELECT
UNIX_TIMESTAMP(timestamp) AS "time",
humidity
FROM Sensor_BME280_01
WHERE
$__timeFilter(timestamp)
ORDER BY timestamp
instead of
SELECT
timestamp AS "time",
humidity
FROM Sensor_BME280_01
WHERE
$__timeFilter(timestamp)
ORDER BY timestamp
The value timestamp is of type DATETIME in a MariaDB.
I am having trouble pulling the below data in actual UTC time. I would like the day, hour, and ticket count but for some reason it does not seem to be pulling in UTC (data doesn't make sense). It is also pulling a few hours of 7-15 and only goes as far as 8-25
Does it have to do with a setting that I'm not aware of? Any ideas would be greatly appreciated!
Query below:
SET timezone "UTC";
SELECT
day,
CONCAT(cast(hour as STRING),':00') as hour,
COUNT(DISTINCT units) as count
FROM
tableABC
WHERE
created.timestamp BETWEEN TIMESTAMP("2018-07-16 00:00:00 UTC")
AND TIMESTAMP("2018-08-26 00:00:00 UTC")
I figured it out:
FORMAT_TIMESTAMP('%Y-%m-%d',timestamp, "UTC") as day,
CONCAT(FORMAT_TIMESTAMP('%H',timestamp, "UTC"),':00') as hour,
Thank you Zaynul.
dbms was bigquery
What does this do in a SQL Query? Can someone explain? What does the .5- represent?
WHERE ScheduleEntry.ScheduleDate >= getdate() and ScheduleEntry.ScheduleDate <= getDate() +.50
Think of date unit as 1 day. 0.50 of a day is 1/2 of a day. So this returns anything that has ScheduleDate within half a day from getdate() time forward.
It restricts the rows returned to rows where the ScheduleEntry.ScheduleDate is in the future and where it is not more than .50 units later than the current date. To find out how much time .50 units is equal to, run the following on your console:
SELECT getDate()
SELECT getDate() +.50
The difference between the dates should tell you the difference. Most likely, it is half a day.
GETDATE returns the current date and time in SQL.
You can use addition to "add days" to the current date.
Generally, it is better to use DATE_ADD instead of adding directly and is probably easier to read. Adding 0.5 is akin to adding half a day (or 12 hours).
To better illustrate, the .5 is half a day
SELECT DateDiff(HH,getDate(),getDate() +.50)
Returns 12 hours
I have problem converting number column to date, I did the following
SELECT to_date('12-30-1899 1:00:00','MM-DD-YYYY HH24:Mi:SS') + (createDate/1440)
FROM table_A;
and got the query result
10/17/5826 17:18
The month and date including hours and seconds is right but the year is different I got 5826. Its also the same for the other rows i got different results for year. I did follow some examples on this here. But still got wrong result. Can anyone help on this thanks.
The samples below are createDate column values:
1300844909778
1302831103113
1303210978316
1396963615616
Date arithmetic in Oracle assumes days. As it stands you are dividing a very large number by 1440 and adding that number of days to your starting date. That's why you're getting results in the far future.
So what value does createdate represent? It's clearly not an actual date. Your choice of 1440 as denominator suggests you think it's meant to be "number of minutes" but if the dates are so far out of expectation that is not it either.
I thought could be values represented in the Unix epoch because the numbers start with 13. Except that they're way too big. Current Unix timestamps should be ten digits. You've got thirteen digits.
Could they be Unix epoch plus milliseconds?
I have created a SQLfiddle to test this theory. Treating the first ten digits of your createdate values as seconds and adding that number to the Unix date produces sensible dates. Check it out.
So the theory holds water. But I doesn't help with your query. Adding two dates together doesn't make any sense. What are you actually trying to achieve? If your're looking for an interval you need to subtract the earlier date from the later one.
The createDate could be the number of milliseconds. It is just a guess. If so, then maybe this helps:
SELECT to_date('12-30-1899 1:00:00','MM-DD-YYYY HH24:Mi:SS') + (1300844909778/(1000*60*60*24))
FROM dual
/
3/21/1941 2:48:30 AM