Convert String to Timestamp in Spark/Hive - sql

I need to convert string to Timestamp.
The problem is that the input is coming from a csv file and contains date-time values such as:
Mar 3 2022 8:30AM
Apr 27 2022 7:37AM
If I use the following conversion:
to_timestamp(to_timestamp(trim(DateColumn), 'MMM dd yyyy h:mma'), 'yyyy-MM-dd HH:mm:ss')
It converts the date Apr 27 2022 7:37AM correctly, but throws error while converting Mar 3 2022 8:30AM because of the extra space between the Month and Date values and that the date 3 is not 03.
Is there a way to convert these 2 strings formats into Datetime?

It is recommended that you first uniformly replace multiple spaces with a single space, and then convert to timestamp.
val df1 = df.withColumn("ts", to_timestamp(regexp_replace(trim(col("ts")), "\\s+", " "), "MMM d y h:mma"))

Related

How to convert a datetime ''May 30 2022 9:30PM" to this "2022-05-30 21:30:00" in Snowflake SQL?

I have a date column in text format of "May 30 2022 9:30PM" and it needs to convert to "2022-05-30 21:30:00" in Snowflake SQL
I tried below SQL and its not working.
SELECT cast('May 30 2022 9:30PM' as datetime);
For Snowflake, try the following:
select to_timestamp_ntz('May 30 2022 9:30PM', 'MON DD YYYY HH12:MIAM');
Reference:
https://docs.snowflake.com/en/sql-reference/functions-conversion.html

How to convert "Wed Jan 26 2022" to date in sql?

I have varchar like this "Wed Jan 26 2022"
I need to convert this to date in sql. How can i do this
for Sql Server:
convert(date, substring('Wed Jan 26 2022',5,11),9)
we ignore the Day name (superfluous), and convert the rest using format 9 indicating Mon dd yyyy format.
SQL*plus server (Here's my code) -
If I am right, you want such type of string which is an invalid one to convert that into a valid one so that you can store valid data into the database. then this code you can use->
SELECT TO_DATE('WED JAN 26 2022','DY MON DD YYYY')FROM DUAL;
(Explanation)->
Code will convert invalid date datatype to a valid date data type which is used in Oracle(SQL).
DY = Abbreviated Week Day
DD = Month day indicator
MON = Abbreviated month
YYYY = Four-digit year indicator

Convert Time and Date in string to date VB.net

I'm using RSS feeds from multiple sources and I'm sorting the feeds by date. However, the dates I receive are in different time zones.
This is the format of some of the date strings that I get:
Wed 08 Dec 2021 01:40:31 -0500
Wed 08 Dec 2021 11:11:19 -0600
The "-500' indicates the time zone which is UTC-5. I need to use that "-0500" to convert this string into a date and the date needs to be only in UTC.
Basically from "Wed 08 Dec 2021 01:40:31 -0500" to "12/08/2021 06:40:31"
I've managed to split the string up to sort by date but because of the time difference, it doesn't really help.
Is there coding which I can use to convert the string as-is into date and only UTC time? Or is there just a place where I can start?
Thank you in advance.
Using DateTime.ParseExact, you specify a format to convert from, and that can include "zzz" for the timezone offset. You can also specify that you want the result in UTC:
Dim s = "Wed 08 Dec 2021 01:40:31 -0500"
Dim d = DateTime.ParseExact(s, "ddd dd MMM yyyy HH:mm:ss zzz", Nothing, Globalization.DateTimeStyles.AdjustToUniversal)
Console.WriteLine(d.ToString("yyyy-MM-dd HH:mm:ss"))
Console.WriteLine(d.Kind.ToString())
Outputs:
2021-12-08 06:40:31
Utc
Of course, format the result as you desire.
Alternatively, if you need to, you can keep the offset by using a DateTimeOffset structure and adjust it to UTC for representation as a string:
Dim s = "Wed 08 Dec 2021 01:40:31 -0500"
Dim d = DateTimeOffset.ParseExact(s, "ddd dd MMM yyyy HH:mm:ss zzz", Nothing)
Console.WriteLine(d.ToUniversalTime.ToString("yyyy-MM-dd HH:mm:ss"))
Outputs:
2021-12-08 06:40:31

Unable to convert timestamp to date in Amazon Redshift

I am trying to convert a varchar field into date field using cast(date as date) but it is throwing an error as "Amazon Invalid operation: Error converting text to date".
I tried converting the same column using to_date(date,'Mon DD yyyy') function and it worked fine.
what could be causing this error in cast function and what can I do to rectify it?
I want to use cast specifically for this conversion.
SELECT DISTINCT cast(activity as date)
from akhil_date_conversion;
my activity field has values like:
Nov 24, 2002 9:02 AM
Jan 21, 2002 9:00 AM
Nov 17, 2002 9:00 AM
Nov 5, 2002 9:00 AM
Feb 17, 2002 10:00 AM
Jan 16, 2009 9:03 AM
Apr 20 2002 13:02
May 11 2002 19:34
Aug 11, 2002 12:00 PM
some have AM/PM and some do not!
There is a cluster parameter called datestyle that can be used to set the default date / time format for the cluster. The default is ISO and what you have is closest to POSTGES. This is used in outputting dates but also in interpreting them. However, you don't have a consistent datestyle in your data (12 and 24 hour format changes) so I don't think this will work in your case. You will need to write some SQL get your data into a single format before CAST will work by default.

Conditional DateTime.ParseExact format

I am parsing a date from a bank deposit report, and the format is like this:
Jul 9 2015
Jun 20 2015
Basically MMM dd yyyy except that the single digit day does not contain a leading zero. Is there a simple way to do conditional formatting in DateTime.ParseExact()? Or will I have to pre-process the date string and either add the leading zero or remove the extra space?? Here is what works for the single digit day dates:
Dim dtDepositDate As DateTime
dtDepositDate = DateTime.ParseExact(strDate, "MMM d yyyy", CultureInfo.InvariantCulture)
and obviously, MMM dd yyyy would work for the two digit dates, but would not work for the single digit dates with an extra space in between.
For single/double digit day part
Use single d which is good for both single and double digits day value.
Having single d would effect the values if the DateTime is converted to string. As far as parsing is concerned, it will work for both single and double digits day values, like 01 , 1 , 11, 20 etc. The same is true for M, H, m, specifier for Month, Hour Minutes etc.
For multiple spaces
For multiple spaces use DateTimeStyles.AllowWhiteSpaces in parsing.
DateTime dt = DateTime.ParseExact("Jul 9 2015", "MMM d yyyy",
CultureInfo.InvariantCulture,
DateTimeStyles.AllowWhiteSpaces);
or for double digit day part:
DateTime dt = DateTime.ParseExact("Jun 20 2015", "MMM d yyyy",
CultureInfo.InvariantCulture,
DateTimeStyles.AllowWhiteSpaces);