date_trunc in hive is working incorrectly - hive

I am running below query:
select a.event_date,
date_format(date_trunc('month', a.event_date), '%m/%d/%Y') as date
from monthly_test_table a
order by 1;
Output:
2017-09-15 | 09/01/2017
2017-10-01 | 09/30/2017
2017-11-01 | 11/01/2017
Can anyone tell me why for date "2017-10-01" it is showing me date as "09/30/2017" after using date_trunc.
Thanks in Advance...!

You are reverse formatting so it is incorrect.
Use the below Code
select a.event_date,
date_format(date_trunc('month', a.event_date), '%Y/%m/%d') as date
from monthly_test_table a
order by 1;

You can use date_add with a logic to subtract 1-day(yourdate) to replicate trunc.
For eg:
2017-10-01 - day('2017-10-01') is 1 and you add 1-1=0 days
2017-08-30 - day('2017-08-30') is 30 and you add 1-30=-29 days
I faced the same issue recently and resorted to using this logic.
date_add(from_unixtime(unix_timestamp(event_date,'yyyy-MM-dd'),'yyyy-MM-dd'),
1-day(from_unixtime(unix_timestamp(event_date,'yyyy-MM-dd'),'yyyy-MM-dd'))
)
PS: As far as i know, there is no date_trunc function in Hive documentation.

As per the source code below: UTC_CHRONOLOGY time is translated w.r.t. locale, also in Description it is mentioned that session timezone will be the precision, also refer to below URL.
#Description("truncate to the specified precision in the session timezone")
#ScalarFunction("date_trunc")
#LiteralParameters("x")
#SqlType(StandardTypes.DATE)
public static long truncateDate(ConnectorSession session, #SqlType("varchar(x)") Slice unit, #SqlType(StandardTypes.DATE) long date)
{
long millis = getDateField(UTC_CHRONOLOGY, unit).roundFloor(DAYS.toMillis(date));
return MILLISECONDS.toDays(millis);
}
See https://prestodb.io/docs/current/release/release-0.66.html:::
Time Zones:
This release has full support for time zone rules, which are needed to perform date/time calculations correctly. Typically, the session time zone is used for temporal calculations. This is the time zone of the client computer that submits the query, if available. Otherwise, it is the time zone of the server running the Presto coordinator.
Queries that operate with time zones that follow daylight saving can
produce unexpected results. For example, if we run the following query
to add 24 hours using in the America/Los Angeles time zone:
SELECT date_add('hour', 24, TIMESTAMP '2014-03-08 09:00:00');
Output: 2014-03-09 10:00:00.000

Related

Convert UTC 'TimeGenerated' to local time in Azure monitor/log/analytics, when using "summarize by"

I have this simple query
MyLog
| summarize avg(executionTimeInMS_d) by bin(TimeGenerated, 5min)
I'd like the summary to be in my local time zone, not UTC. This does not work :
MyLog
| summarize avg(executionTimeInMS_d) by bin(TimeGenerated-5, 5min)
Can this be done?
datetime values are in UTC.
if you know the timezone offset (at the time you run the query), you can subtract/add it to your datetime values as explained here: https://learn.microsoft.com/en-us/azure/kusto/query/datetime-timespan-arithmetic
for example: print now() - 7h
There is now a "Display time zone" setting in the App Insights query page. This will convert the timestamp to the selected timezone. It will also show the timezone in the timestamp column heading.
This only seems to work if you output the timestamp directly. If you apply any operations it reverts to UTC.
Best to convert using the datetime_utc_to_local() function. This way you can dynamically handle daylight savings within the query and don't need to depend on the UI.
AzureDiagnostics
| extend CentralTime = datetime_utc_to_local(TimeGenerated, "US/Central")
You can do this by subtracting/adding the time different from UTC. For example, to convert to EST. I subtracted 5h from TimeGenerated which is in UTC.
AppServiceConsoleLogs
| extend EasternTime = TimeGenerated - 5h
| sort by EasternTime desc
| project Level, EasternTime, ResultDescription

Selecting where timestamp greater than with timezone

Suppose I have this table structure:
Table "test"
Column | Type | Modifiers
----------------+-----------------------------+-----------
uuid | uuid | not null
created | timestamp without time zone | not null
How would I select records after a certain date? But also factor in a specific timezone?
Since created is timestamp without zone, we can assume it's UTC
Example query:
select uuid from test where created >= '2017-07-20'
This would return all events that happend on, or after 2017-07-20 00:00:00.000 UTC How would I query for events that happend after say, 2017-07-20 00:00:00.000 GMT+2 ? Without having to add hours to my argument in created > arg
select uuid
from test
where created > '2017-07-20'::timestamp with time zone at time zone '+02';
I think one approach would be to compare the number of seconds since the epoch between your UTC timestamp in the table and some other input:
select uuid
from test
where
extract(epoch from created) >
extract(epoch from timestamp with time zone
'2017-07-19 23:59:59.999'::timestamp with time zone at time zone 'GMT')
The above syntax is verbose, and there may be a shorter way of writing this query. As a general rule, when you find yourself jumping through hoops to answer simple queries, it might mean you need to change your design. If you were storing your timestamps in UTC and running queries from an application, you would probably be passing in local timestamps already converted to UTC, which would make things much simpler.

SQL query in NodeJS hours are off when formatting date

I want to format the dtime2 field in my query: SELECT FORMAT(MAX(dTime),'yyyy-MM-dd hh:mm:ss') FROM triangulations
This gives the output { result: [ { '': '03:34:30' } ], rowcount: 1 }
The hours should be 15. This is also displayed when leaving the format out of the query. Query: SELECT MAX(dTime) FROM triangulations gives output:
{ result: [ { '': Mon Jul 17 2017 15:34:30 GMT+0000 (Coordinated Universal Time) } ],
rowcount: 1 }
I execute the query in NodeJs with the library node-mssql-connector.
Why is SQL giving my the wrong hours?
In your format string, yyyy-MM-dd hh:mm:ss, hh means you want the hours in the 12-hour-cycle format, so 3 and 15 are always 3 (AM and PM). Use HH to get them in the 24-hour-cycle format:
yyyy-MM-dd HH:mm:ss
Relevant docs, scroll down to the list of format specifiers.
You should use HH instead of hh:
SELECT FORMAT(MAX(dTime),'yyyy-MM-dd HH:mm:ss') FROM triangulations
Usually when you get bad hours while the minutes and dates are fine, in means that you're using the wrong time zone. This could mean that either the time in a wrong time zone was written to the database, or you're getting the date from the database in some other time zone that you're expecting.
You should always be explicit about time zones when working with dates (which you don't seem to be doing here when while putting the dates into the database), and it makes things much easier if you're always saving dates in UTC (which seems to be the case for the dates that you're reading here).
In Node you can convert the dates using the Moment module - Moment Timezone in particular. See:
https://momentjs.com/timezone/
See this answer for some examples - it's about Mongo instead of SQL server but you can use exactly the same conversion here:
Mongoose saving and retrieving dates with UTC time, change to server timezone

Oracle timestamp, timezone and utc

I have an application, using an Oracle 11g (11.2.0.2.0 64 bit) db.
I have a lot of entries in a Person table. To access data I'm using different application (same data).
In this example I'm using birth_time field of my person table.
Some application queries data with birth_time directly, some other with to_char to reformat it, and some other with UTC function.
The problem is this: with same data, same query, result are different.
In this screenshot you can see the result with Oracle Sql developer (3.2.20.09)
All the timestamp are inserted with midnight timestamp, and in fact the to_char(..) and birth_time result are at midnight. UTC hours are returned with one hour less (Correct according to my timezone!) but some entry (here one for example, the last one) is TWO HOURS less (only few on thousand are Three)!!
The same query executed with sql*plus return the correct result with one hour of difference for all the entries!
Does anyone have a suggestion to approach this problem?
The issue is born because one of our application made with adobe flex seems to execute queries with UTC time, and the problems appears when you look at data with this component.
ps.:
"BIRTH_TIME" is TIMESTAMP (6)
Would it be possible for you to change the query used? If so, you could use the AT TIME ZONE expression to tell Oracle that this date is in UTC time zone:
SELECT SYS_EXTRACT_UTC(CAST(TRUNC(SYSDATE) AS TIMESTAMP)) AS val FROM dual;
Output:
VAL
----------------------------
13/11/20 23:00:00,000000000
Now, using AT TIME ZONE 'UTC' gets you the date you need:
SELECT SYS_EXTRACT_UTC(
CAST(
TRUNC(SYSDATE) AS TIMESTAMP)
AT TIME ZONE 'UTC') AS val FROM dual;
Output:
VAL
----------------------------
13/11/21 00:00:00,000000000

Default implied time for comparison in Oracle SQL TO_DATE?

I'm doing a SQL query in Oracle 10g where I'm comparing against a cutoff date. So my query has this in it:
THING < TO_DATE('02/14/13','MM/DD/YY')
Now the THING can have a time component in it. I want to know how the cutoff date will interact with it. Does the TO_DATE function have some default implied time component in it? Does the date it creates have a default time of midnight on the specified date, or noon or some other time? Essentially my concern is if I have a column in the table like this:
THING
-------
2/4/13 11:13AM
2/13/13 3:36PM
2/14/13 2:00PM
2/15/13 1:52AM
Will I get 2 rows or 3 rows back?
The implied time is 00:00:00, so in your example you will get two rows back.
You can verify this with:
select to_char(TO_DATE('02/14/13','MM/DD/YY'),'YYYY-MM-DD HH24:MI:SS')
from dual;
You'll get two rows back. The implied time is 0:00:00 (midnight). Your dates with a 24-hour clock look like this:
2/13/13 3:36PM --> 2013-02-13 15:36:00
TO_DATE('02/14/13','MM/DD/YY') --> 2013-02-13 00:00:00