I have a column eventtime that only stores the time of day as string. Eg:
0445AM - means 04:45 AM. I am using the below query to convert to UNIX timestamp.
select unix_timestamp(eventtime,'hhmmaa'),eventtime from data_raw limit 10;
This seems to work fine for test data. I always thought unixtimestamp is a combination of date and time while here I only have the time. My question is what date does it consider while executing the above function? The timestamps seem to be quite small.
Unix timestamp is the bigint number of seconds from Unix epoch (1970-01-01 00:00:00 UTC). The unix time stamp is a way to track time as a running total of seconds.
select unix_timestamp('0445AM','hhmmaa') as unixtimestamp
Returns
17100
And this is exactly 4hrs, 45min converted to seconds.
select 4*60*60 + 45*60
returns 17100
And to convert it back use from_unixtime function
select from_unixtime (17100,'hhmmaa')
returns:
0445AM
If you convert using format including date, you will see it assumes the date is 1970-01-01
select from_unixtime (17100,'yyyy-MM-dd hhmmaa')
returns:
1970-01-01 0445AM
See Hive functions dosc here.
Also there is very useful site about Unix timestamp
Related
I want to md5 a timestamp column in Hive, without the millisecond.
If timestamp is before Epoch Unix Time (year 1970), timestamp is corrupted:
START_DATE=1915-07-15 23:25:26.290448384
select ID, START_DATE, MD5(START_DATE) from TABLE1
Result : START_DATE = 2500-02-02 00:00:00.0
No issue without adding MD5 function, or if the timestamp > 1970.
I've tried with vectorized parameter (https://cwiki.apache.org/confluence/display/hive/vectorized+query+execution#VectorizedQueryExecution-Limitations) but still the same issue.
Also tried : Cast as string, substr... before MD5.
How can we handle timestamp < 1970 ?
can you use somethign like this ?
select md5(from_unixtime(unix_timestamp(substring('1915-07-15 23:25:26.290448384',1,19)))) as md5_out
Unixtime will calculate any date before 1970 as negative number and calculate accordingly. so i think it should be alright.
I store date from Calendar.getTimeInMilliseconds() in SQLite DB.
I need to mark first rows by every month in SELECT statement, so I need convert time in milliseconds into any date format using SQLite function only. How can I avoid this?
One of SQLite's supported date/time formats is Unix timestamps, i.e., seconds since 1970.
To convert milliseconds to that, just divide by 1000.
Then use some date/time function to get the year and the month:
SELECT strftime('%Y-%m', MillisField / 1000, 'unixepoch') FROM MyTable
Datetime expects epochtime, which is in number of seconds while you are passing in milliseconds. Convert to seconds & apply.
SELECT datetime(1346142933585/1000, 'unixepoch');
Can verify this from this fiddle
http://sqlfiddle.com/#!5/d41d8/223
Do you need to avoid milliseconds to date conversion or function to convert milliseconds to date?
Since sqlite date functions work with seconds, then you can try to
convert milliseconds in your query, like this
select date(milliscolumn/1000,'unixepoch','localtime') from table1
convert millis to seconds before saving it to db, and then use date function in sql query
I have a value in my csv file for timetamp as '1522865628160'. When I load the data in bigQuery where this field type is timestamp, it saves the timestamp as '1522865628160000'. so when I query like
select * from <tablename> limit 1
it gives me error
Cannot return an invalid timestamp value of 1522865628160000000 microseconds relative to the Unix epoch. The range of valid timestamp values is [0001-01-1 00:00:00, 9999-12-31 23:59:59.999999]; error in writing field timestamp"
please help
I think the issue here is that you tried to load your UNIX timestamp data into a timestamp column in BigQuery. A BigQuery timestamp column is not the same thing as a UNIX timestamp. The latter is just a numerical value representing the number of seconds since the start of the UNIX epoch in 1970.
So the fix here would be to load your data into an INT64 (or INTEGER if you are using legacy) column. From there, you may convert your UNIX timestamp to a bona fide date or timestamp.
There is a MSEC_TO_TIMESTAMP() function which can convert an integer number of milliseconds since the UNIX epoch to a bona fide timestamp, e.g.
SELECT MSEC_TO_TIMESTAMP(1522865628160)
2018-04-04 11:13:48 UTC
why i get error with this code
SELECT EXTRACT(SECOND FROM TIME total_time) from tr_empl_quiz;
and i got error to with this code
SELECT EXTRACT(EPOCH FROM TIMESTAMP total_time) from tr_empl_quiz;
this is my table content tr_empl_quiz and type data is time without time zone
total_time
============
00:01:00
00:02:00
When you use the extract() function you are getting the value of the date/time part. In your examples, the seconds are zero, so that is what you get.
Postgres does support what you want, using the perhaps unintuitive name epoch. Epoch returns the number of seconds. For an date or datetime value, this is the number since 1970-01-01 (the beginning of Unix time). For a time or interval it is the total number of seconds during the period. So:
select extract(epoch from time '00:02:00')
returns 120.
Surprisingly, the documentation for epoch doesn't mention that it works on the time data type. The functionality is entirely consistent with what the function does. Either the documentation (which is generally quite excellent) overlooks time; or time is treated as an interval.
For a column in a table, you would just do:
SELECT EXTRACT(EPOCH FROM total_time)
FROM tr_empl_quiz;
or:
SELECT EXTRACT(EPOCH FROM CAST(total_time as time))
FROM tr_empl_quiz;
Depending on what you want.
I am storing a timestamp field in a SQLite3 column as TIMESTAMP DATETIME DEFAULT CURRENT_TIMESTAMP and I was wondering if there was any way for it to include milliseconds in the timestamp as well?
Instead of CURRENT_TIMESTAMP, use (STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW')) so that your column definition become:
TIMESTAMP DATETIME DEFAULT(STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW'))
For example:
CREATE TABLE IF NOT EXISTS event
(when_ts DATETIME DEFAULT(STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW')));
To get number of milliseconds since epoch you can use julianday() with some additional calculations:
-- Julian time to Epoch MS
SELECT CAST((julianday('now') - 2440587.5)*86400000 AS INTEGER);
The following method doesn't require any multiplies or divides and should always produce the correct result, as multiple calls to get 'now' in a single query should always return the same result:
SELECT strftime('%s','now') || substr(strftime('%f','now'),4);
The generates the number of seconds and concatenates it to the milliseconds part from the current second+millisecond.
Here's a query that will generate a timestamp as a string with milliseconds:
select strftime("%Y-%m-%d %H:%M:%f", "now");
If you're really bent on using a numeric representation, you could use:
select julianday("now");
The accepted answer only gives you UTC. If you need a local time instead of UTC, use this:
strftime('%Y-%m-%d %H:%M:%f', 'now', 'localtime')