I'm currently trying to collect data that falls between 2 dates via Unix timestamp. All of our dates are stored as VARCHARs to the CAST function is used.
The line in my query reads as:
FROM_UNIXTIME(UNIX_TIMESTAMP(), '%Y %D %M %h:%i:%s %x') between
CAST(d.start_date AS TIMESTAMP) and CAST(d.end_date AS TIMESTAMP)
This returns as error:
Function unix_timestamp not registered
I also tried:
CAST(from_unixtime(unixtime) AS DATE) between
CAST(start_date AS DATE) and CAST(end_date AS DATE)
This produces the error:
Column unixtime cannot be resolved
Any suggestions?
Presto does not support unix_timestamp() function. You need to convert your varchar to date.
So:
now() BETWEEN
date_parse(start_date, '%Y-%m-%d %H:%i:%s') AND
date_parse(end_date, '%Y-%m-%d %H:%i:%s')
Adjust the date format string as per scenario.
For a full list of Presto date and time function, refer to: https://prestodb.io/docs/current/functions/datetime.html
I've used the code below to convert a unix timestamp to a date. You should then be able to compare it to the other two dates.
CAST(from_unixtime(unix_ts_col) AS DATE)
In the database I use, the unix timestamp has been stored as a string, so I had to cast it to an integer first.
CAST(from_unixtime(CAST(unix_ts_col AS INTEGER)) AS DATE);
Related
Hi I have a database SQLite with a table "data" with the column "time#timestamp" that is a REAL for example 1669729394.792
So I have to select a range of data using 2 dates (start date and end date) written by the operator in human datetime format (ex. 2022-11-29) and extract all my data
somehow I should convert my date from standard format to UNIX timestamp
I tried like this but it doesn't work for me:
SELECT * FROM data WHERE date([time#timestamp]) BETWEEN CAST(strftime('%s', '2022-11-29') AS REAL) AND CAST(strftime('%s', '2022-11-30') AS REAL)
The function date() with a numeric parameter and no modifiers considers the numeric value as a Julian day and returns that date in the text format YYYY-MM-DD.
But your datetime values are not Julian days, they are unix timestamps and you can transform them to a readable date format YYYY-MM-DD with the modifier 'unixepoch':
date([time#timestamp], 'unixepoch')
After that you can directly compare the result to any date in the format YYYY-MM-DD and no casting is needed:
SELECT *
FROM data
WHERE date([time#timestamp], 'unixepoch') BETWEEN '2022-11-29' AND '2022-11-30'
Or, keep the value of [time#timestamp] as it is and transform the 2 date boundaries to unix timestamps:
SELECT *
FROM data
WHERE [time#timestamp] BETWEEN strftime('%s', '2022-11-29') AND strftime('%s', '2022-11-30')
Or, if your version of SQLite is 3.38.0+:
SELECT *
FROM data
WHERE [time#timestamp] BETWEEN unixepoch('2022-11-29') AND unixepoch('2022-11-30')
The following function returns dates in this format, "2021-01-01T00:00:00.000Z" and all I need is just the date portion of "2021-01-01".
DATE_TRUNC(‘day’, timestamp)
The return value from your current call to DATE_TRUNC already functionally is the date 2021-01-01, which in timestamp form is at midnight. That being said, if you want to view as a date only, then maybe you want this:
SELECT FORMAT_DATETIME("%Y-%m-%d", DATE_TRUNC('day', timestamp))
FROM yourTable;
Another trick which might work on BigQuery is to cast the timestamp to a VARCHAR of the right length:
SELECT CAST(DATE_TRUNC('day', timestamp) AS VARCHAR(10))
FROM yourTable;
I have Oracle SQL query where it has been used TRUNC(04-Aug-2017 15:35:32)
What will be parameter in Hive to replace TRUNC?
Assuming you have a date/time, you can use the to_date() function:
select to_date(col)
If you have a timestamp, say ts, you can use trunc():
trunc(ts, 'day')
This returns a timestamp, with the time portion stripped off - which is similar to what trunc() does in Oracle when given one argument only.
On the other hand, you can also convert the timestamp to a date:
to_date(ts)
This returns a date rather than a timestamp: that's a different datatype, that has no time component (Oracle does not have such a datatype: both date and timestamp store the date and time).
As per Oracle docs, The TRUNC (date) function returns date with the time portion of the day truncated to the unit specified by the format model fmt. The value returned is always of datatype DATE, even if you specify a different datetime datatype for date. If you omit fmt, then date is truncated to the nearest day.
Similar is the function of to_date function in Hive.
It returns the date part of a timestamp string (pre-Hive 2.1.0): to_date("1970-01-01 00:00:00") = "1970-01-01".
If what you want is the timestamp(midnight timestamp : 00:00:00) along with the truncated date, you need to use some conversions as shown below:
cast(from_unixtime(unix_timestamp(to_date(<YOU_DATE_COL>), 'yyyy-MM-dd')) as timestamp)
I have a date/time stamp field, where I need to pull records just by date.
Example: All data were records are >= '01/01/2016'.
The data in the field is store in the following format '9/5/2012 7:34:59 AM'
I have tried the following but either I get an error or bad results:
where to_char(start_time) > '01/01/2016' (still gives 2012 records)
where trunc(start_time) > '01/01/2016' (Error: Not a valid month)
In mysql you should use
this is in string canonical format
select * from my_table
where start_time > '2016/01/01'
or
or converting by str_to_date using proper format
select * from my_table
where start_time > str_to_date('01/01/2016', '%d/%m/%Y')
Use to_date to convert the string to date and then do the comparison.
Try this:
where start_time >= to_date('01/01/2016', 'dd/mm/yyyy');
In Oracle (and perhaps other database products) you can use the ANSI date literal, as shown below. You could also use to_date(), but the benefit of the ANSI date literal is that it doesn't require a function call. (Function calls are overhead which consumes time and resources, although calling to_date() just once is not a concern.)
... where start_time >= date '2016-01-01'
Note that in the ANSI standard date literal, the date must be in the exact format YYYY-MM-DD (with dashes and not with slashes or any other separators), since the ANSI date literal does not take a format model.
We have a timestamp epoch column (BIGINT) stored in Hive.
We want to get Date 'yyyy-MM-dd' for this epoch.
Problem is my epoch is in milliseconds e.g. 1409535303522.
So select timestamp, from_unixtime(timestamp,'yyyy-MM-dd') gives wrong results for date as it expects epoch in seconds.
So i tried dividing it by 1000. But then it gets converted to Double and we can not apply function to it. Even CAST is not working when I try to Convert this double to Bigint.
Solved it by following query:
select timestamp, from_unixtime(CAST(timestamp/1000 as BIGINT), 'yyyy-MM-dd') from Hadoop_V1_Main_text_archieved limit 10;
The type should be double to ensure precision is not lost:
select from_unixtime(cast(1601256179170 as double)/1000.0, "yyyy-MM-dd hh:mm:ss.SSS") as event_timestamp
timestamp_ms is unixtime in milliseconds
SELECT from_unixtime(floor(CAST(timestamp_ms AS BIGINT)/1000), 'yyyy-MM-dd HH:mm:ss.SSS') as created_timestamp FROM table_name;
In the original answer you'll get string, but if you'd like to get date you need to call extra cast with date:
select
timestamp,
cast(from_unixtime(CAST(timestamp/1000 as BIGINT), 'yyyy-MM-dd') as date) as date_col
from Hadoop_V1_Main_text_archieved
limit 10;
Docs for casting dates and timestamps. For converting string to date:
cast(string as date)
If the string is in the form 'YYYY-MM-DD', then a date value corresponding to that year/month/day is returned. If the string value does not match this formate, then NULL is returned.
Date type is available only from Hive > 0.12.0 as mentioned here:
DATE (Note: Only available starting with Hive 0.12.0)