I have a table with three columns: number of ten digits, time and name.
1473522179,18:39,sukesh
1474209006,9:16,syam
1473820431,7:40,ram
How can i convert first column into date with GMT timeZone using hive and pig?
Use ToDate.Assuming your relation has fields f1,f2,f3
B = FOREACH A GENERATE ToDate(f1),f2,f3;
Hive
SELECT from_unixtime(f1),f2,f3 from A
It can be done by using from_unixtime, for example your first column date is:
to_utc_timestamp(from_unixtime(1473522179),"GMT")
2016-09-10 18:42:59.0
Related
in Hive table value for one column is like 01/12/17.But I need the value in the format as 12-2017(month-year).How to convert it?
Convert the string to a unix_timestamp and output the required format using from_unixtime.
select from_unixtime(unix_timestamp(col_name,'MM/dd/yy'),'MM-yyyy')
I am using sql for database connectivity. I am filtering records in database on date basis using sql query which consists of function convert(varchar(100),column name,106). I am comparing it with date having dd/MMM/yyyy format. As a result I get records of next all the dates.
Example: If I try to filter the table by 01/June/2017 then inspite of having records of that date, I get records of next all the dates. I am not able to get records of 01/June/2017.
I have used greater than or equal to expression.
It is better to convert it to this yyyymmdd number format and do the comparison as below:
select convert(int,convert(varchar(10),[your column],112))
20170529
Don't convert your dates into any format (=strings) when doing comparison. Always use the date (or datetime etc) format. That way you will be comparing the actual values, not string or numeric comparison. Here the problem is that format 106 is dd mon yyyy so it will never match your date because it's a string dd/MMM/yyyy but you will get any dates starting with 02 anyhow, for example 02/Jan/1990, because in alphabetical order that will come after your given date.
I have a column in hive table stored as text. The text looks as shown below
2007-01-01T00:00:00+00:00
I am trying to find difference in time between two timestamp value stored as text in the above format.
Suppose we've got an Hive table dateTest with two column date1 string, date2 string
and suppose that table containing a row with this values:
2007-01-01T00:00:00+00:00,2007-02-01T00:00:00+00:00
The dates are in ISO 8601 UTC format, so if you run this query:
select datediff(from_unixtime(unix_timestamp(date2, "yyyy-MM-dd'T'HH:mm:ss")),from_unixtime(unix_timestamp(date1, "yyyy-MM-dd'T'HH:mm:ss"))) as days
from datetest;
the result is 31
In Table have a Date field of type date and I get 2 date from and to as String.
I want to filter the recodrs that exist between these 2 dates.
What will be the best way to select
The best is to transform the string dates into dates using your DBMS provided transformation functions,
Using Oracle for instance, i would write :
where
your_date_field
between to_date(from,format_of_from) and to_date(to,format_of_to)
How can we match a date value like 2014-Dec-20 in unix_timestamp()? The pattern yyyy-MM-dd matches 2014-12-20 format. But yyyy-mmm-dd or yyyy-m-dd doesn't do the trick for matching the month name.
Got the answer from the hive community. "MMM" will match the three letter month code in unix_timestamp() in Hive.
eg: select request,unix_timestamp(time,'dd/MMM/yyyy:HH:mm:ss') from apachelogs;