hive date cast issue - sql

Hi in my Hive table I have a column with the date values like this .
cl1
31102019000000
30112019000000
31122019000000
I have tried to convert the column values to date format like this
Select from_unixtime(unix_timestamp(cl1,'yyyy/MM/dd'),'yyyy-MM-dd') from table1;
it prints NUll. Any help will be appreciated.

You said you have dates in dd-mm-yyyy but then posted data that doesn't have any hyphens in at all
Assuming 31102019000000 is 31-oct-2019 00:00:00
Select from_unixtime(unix_timestamp(cl1,'ddMMyyyyHHmmss'),'yyyy-MM-dd') from ...
Match the format string to the data..

Try using this
Select from_unixtime(unix_timestamp(REGEXP_REPLACE(cl1,'0+$',''),'ddMMyyyy'),'yyyy-MM-dd') from table1;
but this would fail for years 2020,2010.
So the below query would be a better alternative
Select from_unixtime(unix_timestamp(cast(cast(cl1/1000000 as bigint) as string),'ddMMyyyy'),'yyyy-MM-dd') from table1;

The UnixTimeStamp which you are trying to convert to date format is out of range.
Your Time stamp corresponds to :
GMT: Saturday, August 2, 2955 1:43:20 AM
Your time zone: Saturday, August 2, 2955 5:43:20 AM GMT+04:00
Relative: In 936 years
It is an open bug in MYSql community. Please check the below URL for reference.
https://forums.mysql.com/read.php?20,385047,385132#msg-385132
MySQL database's built-in functions like UNIX_TIMESTAMP() will return 0 after 03:14:07 UTC on 19 January 2038. (Source: https://en.wikipedia.org/wiki/Year_2038_problem)

Related

How to convert date format yyyyMMdd in Hive SQL and automate the query inside the where clause

I would like to get some help in regards of automating my query.
The goal is to get the latest data, everyday the data refreshes but the start date should remain the same.
For example: Start date is Jan 1 2023, but the todays end date should be Jan 13 2023, and next day it will be Jan 14 2023, but the start date stays the same (Jan 1st 2023).
I have tried the following query but it keeps running and do not provide any output. the query I am using is hive SQL and hoping if anyone can help this.
SELECT *
FROM table_a
WHERE cast(entered_date as date) >= '20230101'
and cast(entered_date as date) < date_add(current_date(),0)
You can use date_format(DATE|TIMESTAMP|STRING ts, STRING fmt) to convert value to string in the date format. You can find the description of the function from here. Therefore if you want the output between your desired date and not beyond the current date you can query like this
SELECT *
FROM table
WHERE DATE_FORMAT(entered_date, 'yyyyMMdd') >= '20230101'
AND DATE_FORMAT(entered_date, 'yyyyMMdd') <= DATE_FORMAT(CURRENT_DATE(), 'yyyyMMdd');

Convert date to dateTtime format in SQL

I am trying to convert a date column (ie. 2012-10-02) to the first day of the year with time (ie. 2012-01-01T00:00:00) in sql.
Is there a way to do so in the SELECT query?
for BigQuery use below
select timestamp_trunc('2012-10-02', year)
with output
2012-01-01 00:00:00 UTC
Note - if you column is of date type - the output will be
2012-01-01T00:00:00
and finally, you can use datetime_trunc instead of timestamp_trunc and you will get expected result - 2012-01-01T00:00:00
Look at the YEAR() function.
It would allow you to extract just the year, and then just as the date and time you need.

Amazon Athena unable to convert date to timestamp in query

I would like to convert date value to timestamp format using Amazon Athena Query and used below function to convert ,able to convert some of the dates into timestamp format but not working for some of the date values ,Please find below example
date_parse('10/20/1977','%m/%d/%Y') output is 1977-10-20 00:00:00:000
But for below type of dates showing error:
date_parse('02/29/1977''%m/%d/%Y') output is
cannot parse "02/29/1977":value 29 for dayOfMonth must be in range [1-28]
1977 (Gregorian calendar) was not a leap year, so there were no 29th of February - there were only 28 days. So you need to fix the data.

Invalid date error when modifying Big Query Table schema from string (YYYY-MM-DD HH:MM:SS TIMEZONE) to date

I have a table with a string format column where value are like this :
YYYY-MM-DD HH:MM:SS TIMEZONE so for example 2015-08-27 19:42:53 UTC
UTC is the only timezone. I want to rewrite this string column as a date column. I have an Invalid date error when I'm trying to run the following query to export the result in a new table
SELECT
CAST(my_date AS DATE),stuff_here,stuff_here
FROM
`table`
What should I do in order to properly change the type of this column from string to date ?
You appear to be trying to write this logic:
select CAST(substr('2015-08-27 19:42:53 UTC', 1, 10) AS DATE)
Because I am in New York, I would instead write:
select date(timestamp('2015-08-27 19:42:53 UTC'), 'America/New_York')
This distinction has been very important in our using data -- the difference between days at the Greenwich Meridian versus in our local area.
Try:
SELECT CAST(SUBSTR(my_date,1,19) AS DATE,stuff_here,stuff_here
FROM `table`
This assumes that you are not interested in the timezone part.

Convert UTC Seconds to String in SQL

I have a column with UTC Seconds in SQL.
How can I convert UTCSeconds to varchar in order to get this output in a select statement:
DD-MMM-YYYY HH:mm:ss
Example: 1400249277 in table
16-May-2014 15:07:57 as output of Select statement
I don't have a SQLServer available right now but this looks like it might be what you're looking for : DATEADD. You are probably considering the number of seconds since January the 1st 1970, so it would be something alike :
SELECT DATEADD (second, <your number of seconds>, '1970-01-01')