I am using postgresql 8.0 wherein I have a column which is a timestamp in milliseconds from epoch (but stored as text in table). I want to identify the start of hour for this timestamp. I am unable to think of clean way to do
Table structure
id (varchar(52)) | Name(varchar(100)) | UpdateTime (varchar(20))
1 | Robin | 1598051512000
2 | Sally | 1628734800000
My thought process was to use to_timestamp(). However it does not accept a BigInt. So the idea was to use substring() to discard the last 3 characters of the value in updateTime and then pass it to to_timestamp()
e.g. using select substring('1598051512000', 1, length('1598051512000') - 3));
However this seems to be getting convoluted. IS there a cleaner way to get start of hour based on timestamp?
Expected Output for input
1598051512000 = 2020-08-21T23:00:00Z
1628734800000 = 2020-08-12T02:00:00Z
You could use date arithmetics, then date_trunc():
select
t.*,
date_trunc(
'hour',
date '1970-01-01' + '1598051512000'::bigint / 1000 * '1 second'::interval
) UpdateHour
from mytable t
Related
I have a hive table with following columns
session Timestamp
1 2018-05-18T00:00:00.000Z
1 2018-05-18T00:01:00.000Z
1 2018-05-18T00:02:00.000Z
1 2018-05-18T00:03:00.000Z
2 2018-05-18T00:05:00.000Z
2 2018-05-18T00:10:00.000Z
2 2018-05-18T00:15:00.000Z
For each session, I want to find the difference between the first timestamp and last timestamp, in seconds
Hence my output will be like this
session Period
1 180
2 600
The 1st difference, 180 (seconds) = 2018-05-18T00:03:00.000Z - 2018-05-18T00:00:00.000Z
The 2nd difference, 600 (seconds) = 2018-05-18T00:15:00.000Z - 2018-05-18T00:05:00.000Z
Both session and Timestamp is in string.
I am trying to convert string timestamp into unix timestamp using this command, however I am not successful
select from_unixtime(unix_timestamp(Timestamp, "yyyy-MM-dd'T'HH:mm:ss.SSSZ")) AS NEW_TIMESTAMP from TBL;
unix_timestamp converts a string to timestamp (seconds), so you just need to use this function and substract the min from the max :
select
max(unix_timestamp(Timestamp, "yyyy-MM-dd'T'HH:mm:ss.SSSZ"))
-
min(unix_timestamp(Timestamp, "yyyy-MM-dd'T'HH:mm:ss.SSSZ"))
from TBL
group by session_id
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF
I'll try to explain my problem as clear as possible. I would like to filter a table by date (selecting only the record have the date included in current month) and in Oracle SQL I'm using the following query to achieve such goal:
select * from table t1
where t1.DATE_COLUMN between TRUNC(SYSDATE, 'mm') and SYSDATE
How can I replicate the same filter in Hive SQL? The column I should use to apply the filter is a TIMESTAMP type column (e.g. 2017-05-15 00:00:00).
I'm using CDH 5.7.6-1.
Any advice?
Be aware that unix_timestamp is not fixed and is going to change during the query.
For that reason it cannot be used for partitions elimination.
For newer Hive versions use current_date / current_timestamp instead.
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF
select *
from table t1
where t1.DATE_COLUMN
between cast(from_unixtime(unix_timestamp(),'yyyy-MM-01 00:00:00') as timestamp)
and cast(from_unixtime(unix_timestamp()) as timestamp)
;
select cast (from_unixtime(unix_timestamp(),'yyyy-MM-01 00:00:00') as timestamp)
,cast (from_unixtime(unix_timestamp()) as timestamp)
;
+---------------------+---------------------+
| _c0 | _c1 |
+---------------------+---------------------+
| 2017-05-01 00:00:00 | 2017-05-16 01:04:55 |
+---------------------+---------------------+
You can format as strings:
where date_format(t1.DATE_COLUMN, 'y-m') = date_format(current_timestamp, 'y-m')
I realize that I don't have Hive accessible right now. The documentation suggests 'y-m', but the Java documentation suggests 'yyyy-mm'.
I'm pretty sure this is an easy one for you guys but it's driving me crazy.
I have a column with dates in a "YYYY-MM-DD" format and a column with small intergers values between 0 and 29. So I want to add the 2 columns together and get something like this:
Date | INT | NEW timestamp
2016-01-01 | 2 | 2016-01-01 00:02:00
2016-10-15 | 21 | 2015-10-15 00:21:00
so I tried the obvious like:
"Date" + "INT" as "NEW timestamp"
and stuff like
VARCHAR_FORMAT("INT",'MI')
or even
VARCHAR_FORMAT("Date",'YYYY-MM-DD HH24:MI:SS') + VARCHAR_FORMAT("INT",'MI')
but keep getting errors. I am doing this in dashDB
One option is to use:
select add_minutes(cast("date" as timestamp),"int") from yourTable
Another simple version is:
select cast("date" as timestamp) + "int" minutes from yourTable
On db2 iseries
select TIMESTAMP_FORMAT(Date , 'YYYY-MM-DD') + INT minute as Newtimestamp from yourtable
or
select cast(cast(Date as date) as timestamp) + int minute from yourtable
you can try this
Date_add(cast(`Date` as date),interval Int minute)
I have a table with 4 columns value , animal_id, food_id and timestamp. Time stamp records the data in unix time. Currently my script is working fine and produces a table like this. This is all great and all but this forces me to go edit the file and change the timestamp each day for today's midnight timestamp and this is quite time consuming. Is there a way to produce this line of code for every day without having to update it ? Thank you in advance. I use oracle 10g.
The line in the script I want to change
WHERE timestamp BETWEEN 1406073600 AND (1406073600 + 86400)
The Script
SELECT MAX(value),animal_id, food_id
FROM farm1
WHERE timestamp BETWEEN 1406073600 AND (1406073600 + 86400)
group by animal_id, food_id ;
The Table Results
MAX(VALUE) animal_id food_id
---------- ---------- ----------
9302 8 9081
8015 10 9081
You can convert a date to a Unix-style epoch timestamp number with:
(your_date - date '1970-01-01') * 60 * 60 * 24
... since the timestamp is the number of seconds since 1970-01-01.
So to run your query for all records yesterday (which your sample seems to be doing):
SELECT MAX(value),animal_id, food_id
FROM farm1
WHERE timestamp >= (trunc(sysdate - 1) - date '1970-01-01') * 60 * 60 * 24
AND timestamp < (trunc(sysdate) - date '1970-01-01') * 60 * 60 * 24
GROUP by animal_id, food_id ;
BETWEEN is inclusive so will catch both midnights, which could potentially lead to a value being included in two days' runs, or skewing the results for one of them. I've changed from that to explicit >= and < to make it inclusive. Or you could just subtract 1 from the later time of course, to make it 23:59:59.
SQL Fiddle demo (including a value being picked up from the wrong with your original query).
A way to solve it would be to convert each timestamp to a conceptual date and use the virtual column systimestamp:
SELECT MAX(value),animal_id, food_id
FROM farm1
WHERE TO_CHAR(timestamp, 'YYYYMMDD') = TO_CHAR(systimestamp, 'YYYYMMDD')
group by animal_id, food_id ;
Hi I have a column with number datatype
the data like 1310112000 this is a date, but I don't know how to make it in an understandable format:
ex: 10-mar-2013 12:00:00 pm
Can any one please help me.
That is EPOCH time: number of seconds since Epoch(1970-01-01). Use this:
SELECT CAST(DATE '1970-01-01' + ( 1 / 24 / 60 / 60 ) * '1310112003' AS TIMESTAMP) FROM DUAL;
Result:
08-JUL-11 08.00.03.000000000 AM
Please try
select from_unixtime(floor(EPOCH_TIMESTAMP/1000)) from table;
This will give the result like E.g: 2018-03-22 07:10:45
PFB refence from MYSQL
In Microsoft SQL Server, the previous answers did not work for me. But the following does work.
SELECT created_time AS created_time_raw,
dateadd( second, created_time, CAST( '1970-01-01' as datetime ) ) AS created_time_dt
FROM person
person is a database table, and created_time is an integer field whose value is a number of seconds since epoch.
There may be other ways to do the datetime arithmetic. But this is the first thing that worked. I do not know if it is MSSQL specific.