Using Cloudera's Impala SQL, is there a way to truncate a timestamp by day?
i.e. go from:
2015-05-01 01:23:45 -> 2015-05-01 00:00:00
2015-05-01 12:34:56 -> 2015-05-01 00:00:00
2015-05-01 23:45:59 -> 2015-05-01 00:00:00
The default TRUNC options only seem to enable Week or Hour, not Day...
By my read of the tests, you should be able to pass DDD, DD, or J to TRUNC to get this behavior. I agree that the docs are not clear on this point though--I've filed an issue to get the documentation cleaned up.
Related
So I have this table where I've got two times for each line, but no date and need to get the interval between those two, all is fine when it's:
11:00:00 - 09:38:54
Returns: 01:21:06
As there's no dates, times are stored in "time without time zone" format.
The problem arises when the time enters the next day and the hour becomes 00h, as there's no date the interval will something absurd like -22:58:21
Example:
00:00:00 - 22:59:01
Returns: -22:59:01
00:00:00 - 22:44:06
Returns: -22:44:06
Is there anyway to make SQL understand 00:00:00 as 24:00:00 for the sake of math without date?
The hours only range between 8 and 0, and nothing from the previous day goes further than 0h30, a simple case for "00h" solves it, but I can't make SQL understand 00h as 24h so far. Any ideas?
Like:
select '24:00:00'::time - '22:59:01'::time;
?column?
----------
01:00:59
UPDATE
If the 00:00:00 is coming from somewhere you can't modify in place then:
select time_fld from time_test ;
time_fld
----------
00:00:00
01:30:00
select coalesce(nullif(time_fld, '00:00:00'::time), '24:00:00') from time_test;
coalesce
----------
24:00:00
01:30:00
Could someone help me understand why these two queries are returning different results in bigquery?
select FORMAT_TIMESTAMP('%F %H:%M:%E*S', "2018-10-01 00:00:00" , 'Europe/London')
returns 2018-10-01 01:00:00
select PARSE_TIMESTAMP('%F %H:%M:%E*S', "2018-10-0100:00:00", "Europe/London")
returns 2018-09-30 23:00:00 UTC
As 2018-10-01 is during british summer time (UTC +1), I would've expected both queries to return 2018-09-30 23:00:00 UTC
The first is given a timestamp which is in UTC. It then converts it to the corresponding time in Europe/London. The return value is a string representing the time in the local timezone.
The second takes a string representation and returns a UTC timestamp. The representation is assumed to be in Europe/London.
So, the two functions are going in different directions, one from UTC to the local time and the other from the local time to UTC.
i have a select statement where i am converting timezones
Select
from_tz(cast(DATE_TIME as timestamp), 'US/Eastern') at time zone 'UTC' DATE_TIME_UTC
From Table1
but for some rows i am getting error due to DST
ORA-01878: specified field not found in datetime or interval
i want to write a query like
select
if error then do something else do the time conversion from table1
As you're on 12c you can use the enhanced subquery factoring that provides to define a local function; that can attempt the conversion with US/Eastern, and fall back to -4:00 if that fails.
Using your sample data and a couple of extra rows that will convert anyway:
alter session set nls_date_format = 'YYYY-MM-DD HH24:MI:SS';
alter session set nls_timestamp_tz_format = 'YYYY-MM-DD HH24:MI:SS TZR TZD';
with
function get_tstz(p_date in date) return timestamp with time zone is
dst_exception exception;
pragma exception_init(dst_exception, -1878);
begin
return from_tz(cast(p_date as timestamp), 'US/Eastern');
exception
when dst_exception then
return from_tz(cast(p_date as timestamp), '-04:00');
end get_tstz;
select date_time,
get_tstz(date_time) as date_time_converted,
get_tstz(date_time) at time zone 'UTC' as date_time_utc
from table1
/
DATE_TIME DATE_TIME_CONVERTED DATE_TIME_UTC
------------------- ---------------------------------- ---------------------------
2018-03-11 01:59:00 2018-03-11 01:59:00 US/EASTERN EST 2018-03-11 06:59:00 UTC UTC
2018-03-11 02:06:00 2018-03-11 02:06:00 -04:00 -04:00 2018-03-11 06:06:00 UTC UTC
2018-03-11 02:08:00 2018-03-11 02:08:00 -04:00 -04:00 2018-03-11 06:08:00 UTC UTC
2018-03-11 02:21:00 2018-03-11 02:21:00 -04:00 -04:00 2018-03-11 06:21:00 UTC UTC
2018-03-11 02:48:00 2018-03-11 02:48:00 -04:00 -04:00 2018-03-11 06:48:00 UTC UTC
2018-03-11 02:06:00 2018-03-11 02:06:00 -04:00 -04:00 2018-03-11 06:06:00 UTC UTC
2018-03-11 02:33:00 2018-03-11 02:33:00 -04:00 -04:00 2018-03-11 06:33:00 UTC UTC
2018-03-11 03:00:00 2018-03-11 03:00:00 US/EASTERN EDT 2018-03-11 07:00:00 UTC UTC
I've adjusted my NLS settings so you can see the difference in the converted values, as either EST, EDT or a fixed -4:00.
As mentioend in comments, you're ignoring the underlying data issues, and it would be better to correct the data that you know is wrong - assuming you can be sure why it is wrong and therefore how it is safe to fix; or to confirm your assertion that the original data is all supposed to be US/Eastern.
Fundamentally, as some are clearly not really US/Eastern, it doesn't seem safe to trust any of the data. Without knowing how and why those specifc records have values you don't expect, you can't be sure that any other values are what you expect either. Whatever application, tool or process inserted those dates may have (and probably did) insert other times which look OK but are also not actually US/Eastern. The rest may all convert without error, but that doesn't mean the UTC times are necessarily representative.
You also have a secondary problem in that you don't know whether a date you have recorded as 2017-11-05 01:00:00 was originally 01:00 EST or 01:00 EDT, as that hour was repeated when summertime ended. Oracle will just choose for you though.
You can create a custom function and checks if its a valid timestamp with time zone and use that function in the where clause of your query as follows for example.
create table t(x varchar(100));
insert into t
select '21-FEB-2009 18:00:00'
from dual
union all
select '31-FEB-2009 18:00:00' /*Junk date here..*/
from dual;
create or replace function fn_test(dt in varchar2)
return int
as
l_timestamp timestamp with time zone;
begin
l_timestamp :=from_tz(to_timestamp(dt,'DD-MON-YYYY hh24:mi:ss'), 'US/Eastern') at time zone 'UTC';
return 1;
exception
when others then
return null;
end;
/
select from_tz(to_timestamp(x,'DD-MON-YYYY hh24:mi:ss'),'US/Eastern') at time zone 'UTC'
from t
where fn_test(x) is not null
I got this error message for one of these two reasons:
Most likely: you are trying to convert a local time that does not exist for reasons of DST switch, e.g. '28-MAR-21 02:34' does not exist in Germany (timezone Berlin/Europe), because clocks jump from 1:59 AM to 3:00 AM during night. Solution: add one hour, UTC will then reflect the correct time.
Less likely: typo in timezone string: "Europe/ Berlin" or "Berlin/Europe" - both is wrong, correct is "Europe/Berlin" (pattern is Continent/City)
I have a file that contains forecasted events for the next two weeks. There is a datetime column which has the date and each 30 minute interval, and a time zone column.
I am using pytz to convert the different time zones (around 30+ unique ones) to UTC before loading them into a database. However, for the forecast file I am receiving an error:
NonExistentTimeError: 2016-10-16 00:00:00
Is there a way to go about this?
date interval time_zone
10/26/2016 22:30 US/Central
10/26/2016 22:30 US/Eastern
10/26/2016 23:00 America/Bogota
10/26/2016 23:00 Asia/Calcutta
Current code:
for tz in df['time_zone'].unique():
df.loc[df['time_zone'] == tz, 'datetime_utc'] = df.loc[df['time_zone'] == tz, 'datetime'].dt.tz_localize(tz).dt.tz_convert('UTC')
df['datetime_utc'] = df['datetime_utc'].dt.tz_localize(None)
Due to changes in daylight saving happening on the 16th October, 2016-10-16 00:00:00 really is a local time that does not exist for Brazil (It should instead read 2016-10-16 01:00:00)
I want historic date convert UTC 0 to UTC local in SQL. Like;
2012-11-23
2013-01-08
2014-02-23
But we have 2 different time zone. We use UTC +2 after last sunday in March and use UTC +3 after last sunday October. I need solution immediately guys. Please help me...
Try this:
SELECT CONVERT_TZ('your date ','+your time zone','+time zone you want');
SELECT CONVERT_TZ('2004-01-01 12:00:00','+02:00' ,'+03:00'); // in your case, from +2 to +3
See this link
If you need a dynamic thing, you'll need the timezone of each history (maybe you can store it in a separeted column), so I think you can do something like this:
SELECT CONVERT_TZ('your_date_column','local_timezone' ,'time_zone_column');