i have a DateTime below in string format and I want to convert using hive date function that will extract the date, hour and minutes
Convert ‘2010-11-01 16:30:15’ to 2010-11-01 16:30 in hive
please any suggestions will be appreciated
Use below syntax to extract date,hour and minutes.
select from_unixtime(unix_timestamp('2010-11-01 16:30:15',"yyyy-MM-dd HH:mm:ss"),'yyyy-MM-dd HH:mm');
+-------------------+--+
| _c0 |
+-------------------+--+
| 2010-11-01 16:30 |
+-------------------+--+
Related
I have this table:
Original_Date Date_ Date_1 YearMonth YearMonth_1
03-07-2020 2020-07-03 2020-06-03 03-0 2020
13-03-2020 2020-03-13 2020-02-13 13-0 2020
08-01-2020 2020-01-08 2019-12-08 08-0 2019
13-11-2020 2020-11-13 2020-10-13 13-1 2020
23-03-2020 2020-03-23 2020-02-23 23-0 2020
30-07-2020 2020-07-30 2020-06-30 30-0 2020
13-07-2020 2020-07-13 2020-06-13 13-0 2020
24-01-2020 2020-01-24 2019-12-24 24-0 2019
05-10-2020 2020-10-05 2020-09-05 05-1 2020
11-07-2020 2020-07-11 2020-06-11 11-0 2020
Where if I use TYPE() function on them they return:
Original Date: VARCHAR
Date_: DATE
Date_1: DATE
YearMonth_1: VARCHAR
YearMonth: VARCHAR
These columns are built based on Original_Date column, where:
SELECT
Original_Date
CAST(Original_Date AS DATE FORMAT 'DD-MM-YYYY') AS Date_, -- This is in a subquery, put for simplicity I put it here
ADD_MONTHS(CAST(Original_Date AS DATE FORMAT 'DD-MM-YYYY'),-1) AS Date_1, -- This is in a subquery, put for simplicity I put it here
LEFT(CAST(Date_ AS varchar(10)),4) AS YearMonth,
LEFT(CAST(Date_1 AS varchar(10)),4) AS YearMonth_1,
FROM TABLE
The problem is:
I can't understand why YearMonth takes the last (right) four characters instead of the first (left) four characters.
I was looking for a YearMonth with type DATE and format like YYYY-MM. But as I didn't find any solution with Teradata (if you can provide one, it will be better than the workaround I'm doing now), I'm going for YYYYMM as integer, but as you can see when I'm applying LEFT() function it seem to work like a RIGHT() function, even when both (Date_ and Date_1) have the same data types. why? How can I solve this?
EDIT
SELECT
LEFT(CAST(Date_ AS varchar(10)),4) AS YearMonth_v2,
LEFT(CAST(Date_1 AS varchar(10)),4) AS YearMonth_1_v2,
FROM TABLE
Results in:
YearMonth_v2 YearMonth_1_v2
03-07-2020 2020-06-03
13-03-2020 2020-02-13
08-01-2020 2019-12-08
13-11-2020 2020-10-13
23-03-2020 2020-02-23
30-07-2020 2020-06-30
13-07-2020 2020-06-13
24-01-2020 2019-12-24
05-10-2020 2020-09-05
11-07-2020 2020-06-11
Why transforming the data type of these columns behave differently even when both of them (Date and Date_1) have the same data type and date format?
The big miss here is that you are fighting nature, or rather then nature of date storage and formatting in Teradata. If you run, as an example, the following:
SELECT
'03-07-2020' AS Original_Date,
CAST(Original_Date AS DATE FORMAT 'DD-MM-YYYY') AS Date_, -- This is in a subquery, put for simplicity I put it here
ADD_MONTHS(CAST(Original_Date AS DATE FORMAT 'DD-MM-YYYY'),-1) Date_1, -- This is in a subquery, put for simplicity I put it here
CAST(Date_ AS varchar(10)),
CAST(Date_1 AS varchar(10)),
LEFT(CAST(Date_ AS varchar(10)),4) AS YearMonth,
LEFT(CAST(Date_1 AS varchar(10)),4) AS YearMonth_1
+-----------------+------------+------------+--------------+--------------+-------------+---------------+
| "Original_Date" | "Date_" | "Date_1" | "Date_" | "Date_1" | "YearMonth" | "YearMonth_1" |
+-----------------+------------+------------+--------------+--------------+-------------+---------------+
| "03-07-2020" | 2020-07-03 | 2020-06-03 | "03-07-2020" | "2020-06-03" | "03-0" | "2020" |
+-----------------+------------+------------+--------------+--------------+-------------+---------------+
You'll notice that the return of the ADD_MONTHS() function is a date formatted in the Teradata default of YYYY-MM-DD. This is why your attempt to use the LEFT() string function against both of these dates is yielding strange results.
To correct you can just flip that first YearMonth to use the RIGHT() function instead of LEFT or you can re-cast the date format of your DATE_ like:
SELECT
'03-07-2020' AS Original_Date,
CAST(CAST(Original_Date AS DATE FORMAT 'DD-MM_YYYY') AS DATE FORMAT 'YYYY-MM-DD') AS Date_, -- This is in a subquery, put for simplicity I put it here
ADD_MONTHS(CAST(Original_Date AS DATE FORMAT 'DD-MM-YYYY'),-1) Date_1, -- This is in a subquery, put for simplicity I put it here
CAST(Date_ AS varchar(10)),
CAST(Date_1 AS varchar(10)),
LEFT(CAST(Date_ AS varchar(10)),4) AS YearMonth,
LEFT(CAST(Date_1 AS varchar(10)),4) AS YearMonth_1
+-----------------+------------+------------+--------------+--------------+-------------+---------------+
| "Original_Date" | "Date_" | "Date_1" | "Date_" | "Date_1" | "YearMonth" | "YearMonth_1" |
+-----------------+------------+------------+--------------+--------------+-------------+---------------+
| "03-07-2020" | 2020-07-03 | 2020-06-03 | "2020-07-03" | "2020-06-03" | "2020" | "2020" |
+-----------------+------------+------------+--------------+--------------+-------------+---------------+
The take-away here is that in Teradata if you want to change the format of the date, you MUST use a cast statement so sometimes you'll find yourself casting multiple times, plus Teradata has the tendency to gravitate to the default 'YYYY-MM-DD'. I've found it best to just go with the default so there are no surprises.
If you want to format a date/timestamp, the easiest way is TO_CHAR plus a format, e.g.
TO_CHAR(date_col, 'YYYY-MM') AS YearMonth
To get a YYYYMM integer:
Extract(YEAR From date_col) * 100
+ Extract(MONTH From date_col)
Simple rules:
FORMAT clauses are only applied when converting some other data type to character string or vice versa. (This could be an explicit CAST, or an implicit conversion triggered by applying a function that only operates on strings to a different data type.)
Applying a FORMAT to something does not change how the value is stored.
When retrieving a value with a data type other than character string, the client is responsible for any formatting performed; the database FORMAT clause will be ignored. (In other words, if you want the database to format the data, you need something in your SQL that forces conversion to character string.)
hello my friends I got tired of looking for a solution I have a postgresql database that has a timestamp column I want to extract the values this way
2010-01-01 14:34:43
without the milliseconds how to do this I want it
2010-01-01 14:34:43
and he shows me like this
2010-01-01 14:34:43.267543
If you want to discard the milliseconds you can use date_trunc():
date_trunc('second', mytimestamp)
On the other hand if you want to round to the closest second, you can cast to timestamp(0):
mytimestamp::timestamp(0)
Demo on DB Fiddle - I used a timestamp whose tens of seconds is greater than 5 to make the test representative:
select
mytimestamp,
date_trunc('second', mytimestamp) trunc_mytimestamp,
mytimestamp::timestamp(0) round_mytimestamp
from (values('2010-01-01 14:34:43.567543'::timestamp)) as t(mytimestamp)
mytimestamp | trunc_mytimestamp | round_mytimestamp
:------------------------- | :------------------ | :------------------
2010-01-01 14:34:43.567543 | 2010-01-01 14:34:43 | 2010-01-01 14:34:44
I want to create a column using pyspark that contains the date which is 3 years prior to the date in a given column. The date column looks like this :
date
2018-08-01
2016-08-11
2014-09-18
2018-12-08
2011-12-18
And I want this result :
date past date
2018-08-01 2015-08-01
2016-08-11 2013-08-11
2014-09-18 2011-09-18
2018-12-08 2015-12-08
2011-12-18 2008-12-18
You can use date_sub function.
Here is Scala code which will be very to python.
df.withColumn("past_date",date_sub(col("date"), 1095))
Try with add_months function in pyspark and multiply 12 with -3!
Example:
l = l=[('2018-08-01',),('2016-08-11',)]
ll=["date"]
df=spark.createDataFrame(l,ll)
df.withColumn("past_date",add_months(col("`date`"),-3*12)).show()
RESULT:
+----------+----------+
| date| past_date|
+----------+----------+
|2018-08-01|2015-08-01|
|2016-08-11|2013-08-11|
+----------+----------+
I have a set of data where one column is date and time. I have been asked for all the data in the table, between two date ranges and within those dates, only certain time scale. For example, I was data between 01/02/2019 - 10/02/2019 and within the times 12:00 AM to 07:00 AM. (My real date ranges are over a number of months, just using these dates as an example)
I can cast the date and time into two different columns to separate them out as shown below:
select
name
,dateandtimetest
,cast(dateandtimetest as date) as JustDate
,cast(dateandtimetest as time) as JustTime
INTO #Test01
from [dbo].[TestTable]
I put this into a test table so that I could see if I could use a between function on the JustTime column, because I know I can do the between on the dates no problem. My idea was to get them done in two separate tables and perform an inner join to get the results I need
from #Test01
WHERE justtime between '00:00' and '05:00'
The above code will not give me the data I need. I have been racking my brain for this so any help would be much appreciated!
The test table I am using to try and get the correct code is shown below:
|Name | DateAndTimeTest
-----------------------------------------|
|Lauren | 2019-02-01 04:14:00 |
|Paul | 2019-02-02 08:20:00 |
|Bill | 2019-02-03 12:00:00 |
|Graham | 2019-02-05 16:15:00 |
|Amy | 2019-02-06 02:43:00 |
|Jordan | 2019-02-06 03:00:00 |
|Sid | 2019-02-07 15:45:00 |
|Wes | 2019-02-18 01:11:00 |
|Adam | 2019-02-11 11:11:00 |
|Rhodesy | 2019-02-11 15:16:00 |
I have now tried and got the data to show me information between the times on one date using the below code, but now I would need to make this piece of code run for every date over a 3 month period
select *
from dbo.TestTable
where DateAndTimeTest between '2019-02-11 00:00:00' and '2019-02-11 08:30:00'
You can use SQL similar to following:
select *
from dbo.TestTable
where (CAST(DateAndTimeTest as date) between '2019-02-11' AND '2019-02-11') AND
(CAST(DateAndTimeTest as time) between '00:00:00' and '08:30:00')
Above query will return all records where DateAndTimeTest value in date range 2019-02-11 to 2019-02-11 and with time between 12AM to 8:30AM.
This question already has answers here:
How do I get just the date when using MSSQL GetDate()? [duplicate]
(7 answers)
Closed 9 years ago.
How can I get the current date in MS-SQL Server 2008 R2?
The format of the column in my database is DATETIME and dates are stored in the following format:
+++++++++++++ Vrdate ++++++++++
| |
| 2012-11-18 00:00:00.000 |
| 2012-11-19 00:00:00.000 |
| 2013-11-18 00:00:00.000 |
| 2012-12-01 00:00:00.000 |
| 2010-10-09 00:00:00.000 |
| 2012-11-11 00:00:00.000 |
| |
+++++++++++++++++++++++++++++++
I searched but wasn't able to find the way to get the date in this format (i.e. which has the time associated with it in 00:00:00.00). I found GETDATE() function but that provides the current time along with the date as well, what I want is get the date in the following format: CurrentDate 00:00:00.00
How may I get this?
SELECT CAST(GETDATE() AS DATE)
Returns the current date with the time part removed.
DATETIMEs are not "stored in the following format". They are stored in a binary format.
SELECT CAST(GETDATE() AS BINARY(8))
The display format in the question is independent of storage.
Formatting into a particular display format should be done by your application.
As you are using SQL Server 2008, go with Martin's answer.
If you find yourself needing to do it in SQL Server 2005 where you don't have access to the Date column type, I'd use:
SELECT DATEADD(DAY, DATEDIFF(DAY, 0, GETDATE()), 0)
SQLFiddle