I am new to Hive, I have created hive table with date columns(2)
Create table test(start_date timestamp, end_time timestamp)
row format delimited fields terminated by ",";
But by default Hive date format is YYYY-MM-DD
but my data is Like this
Input
========
DDMMYYYY DD-MON-YYYY
01-02-2012 o1-JAN-2012
22-11-2013 02-FEB-2012
so I want to create table with two columns in Hive with 2 different Date formats
how can I do that one
Can any tell me the QUERY for creating table with custom date format.
create table custom_date(s_date timestamp(DD-MM-YYYY),E_date timestamp(DD-MON-YYYY)) ?
It's not possible to create different Timestamp formats in same table, but different timestamp formats can be displayed in output using unixtime.
select from_unixtime(unix_timestamp(start_date), 'dd-MM-yyyy'), from_unixtime(unix_timestamp(end_time), 'dd-MMM-yyyy') from test;
Related
Using SQL Server Management Studio - all date columns in date format.
Table name: dbo.[FP Data]
Date column (YYYY-MM-DD): order_date
New date column (YYYY-MM): order-month
I used this query:
SELECT
*,
FORMAT(order_date,'YYYY-MM') AS order_month
FROM
dbo.[FP Data]
to create a column in the format YYYY-MM. I now want to merge the new column with my table.
(I need in a YYYY-MM format to compare to other data I have in a YYYY-MM format)
Happy to scrap the above if there is a cleverer way to add the YYYY-MM column OR find a way to name the new column as a table and somehow merge the two tables.
Create a computed column:
alter table FPdata add order_month AS FORMAT(order_date,'yyyy-MM')
https://dbfiddle.uk/?rdbms=sqlserver_2019&fiddle=ad93628258b6c674d7403239f0744803
You may be able to use the "Alter Table" function to solve your question!
ALTER TABLE table_name
ADD column_name data_type column_constraint;
I have three table in the database two of them contain dates however the dates are in two format, first 20-02-2011 and second is 25/09/2018. Let say each table have 10000 records and mixed with these two types of dates format. This why I why I create the column like --- (Transaction_Date, Varchar(10) Not Null)
I tried convert (Varchar(10),Transaction_Date,105)
and also tried replace(convert(varchar(10),Transaction_Date,105),'/','-')
However date and year functions are still not working.
Please suggest a possible way.
How about this?
select replace(date, replace(Transaction_Date, '/', '-'), 105)
That is: (1) convert to a date and (2) replace the slash before converting.
You need to remember about your culture. Saved format vs server culture. But this is very possible
select Cast('2-22-2011' as datetime) f1,
Cast('2/22/2011' as datetime) f2
I other words just use Cast
select cast(Transaction_Date as datetime) . . .
But you should as soon as possible get rid of columns that saves date as string and create new date/time column, and insert your date values there
alter table tbl add column temp datetime
update tbl set temp = Cast(Transaction_Date as datetime)
alter table tbl drop column Transaction_Date
alter table tbl add column Transaction_Date datetime
update tbl set Transaction_Date = temp
alter table tbl drop column temp
I have a value of 12/31/18 and created table in snowflake:
create table my_date (a date);
insert into my_date values ('12/31/18');
select * from my_date;
Result: 0018-12-31
I want to get: 2018-12-31
I saw about 2 number format:
https://docs.snowflake.net/manuals/sql-reference/parameters.html#label-two-digit-century-start
but not sure if this is specification of a column type or data needs to be transformed before the insert?
The parameter two_digit_century_start seems not to be used when parameter date_input_format is set to AUTO. You can get your example working correctly by setting the date format with a parameter ("alter session..." statement on line 2 below). Your complete working example would look like this:
create table my_date (a date);
alter session set DATE_INPUT_FORMAT = 'MM/DD/YY';
insert into my_date values ('12/31/18');
select * from my_date;
This results in 2018-12-31.
Snowflake best-practices recommend to specify the format explicitly with to_date(value, 'format') or by setting the format in parameters. You can find the best practices for date/time functions from Snowflake documentation here: https://docs.snowflake.net/manuals/user-guide/date-time-input-output.html#date-time-function-format-best-practices
How to format dates during the process of creating Hive tables?
I've currently been dumping some data into a discovery environment at work and storing dates as string, because if I format them as a DATE or TIMESTAMP the values are null.
Here's what the raw data looks like:
12/07/2016 05:07:28 PM
My understanding is that Hive accepts dates in this format
yyyy-mm-dd hh:mm:ss
I can format these using a select statement:
select id, receipt_dt, from_unixtime(unix_timestamp(receipt_dt ,'MM/dd/yyyy'), 'yyyy-MM-dd') as app_dt from MySchema.MyTable where app_num='123456'
How can I add in the statement
from_unixtime(unix_timestamp(receipt_dt ,'MM/dd/yyyy'), 'yyyy-MM-dd')
How can I add this in to the generic CREATE EXTERNAL STATEMENT below so that I no longer have to store dates as a string, or use an ALTER TABLE statement to change the formatting?
CREATE EXTERNAL TABLE IF NOT EXISTS MySchema.My_New_Table
( Field1 Format,
Field2 Format,
Field 3 Format,
)
.......
Use MyTable as staging table with raw data and create final/target table my_new_table with transformations i.e, date format...it will be EDW kind of process...
example:
CREATE EXTERNAL TABLE IF NOT EXISTS MySchema.My_New_Table
( Field1 int,
Field2 string,
Field3 date
)
... more definitions....
AS
select id, receipt_dt,
cast(from_unixtime(unix_timestamp(receipt_dt ,'MM/dd/yyyy'), 'yyyy-MM-dd') as date) as app_dt
from MySchema.MyTable ;
NOTE: This is not tested statement. You may need to try and edit and try...but you got the idea...
Then inserting delta should be similar process...
INSERT INTO TABLE MySchema.My_New_Table
AS
select id, receipt_dt,
cast(from_unixtime(unix_timestamp(receipt_dt ,'MM/dd/yyyy'), 'yyyy-MM-dd') as date) as app_dt
from MySchema.MyTable where <<conditions>>;
we need to alter the table column data type from string to date. While am trying to do am getting the below error. Could you please help.
hive> describe sales_staging;
OK
cust_id string prod_num string
qty int sale_date string
sale_id string
Time taken: 0.151 seconds,
Fetched: 5 row(s)
hive> alter table sales_staging CHANGE sale_date sale_date DATE ;
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask. Unable to alter table. The
following columns have types incompatible with the existing columns in
their respective positions :sale_date
hive>
You can't give same name to column you wish to change datatype of. use like this
ALTER TABLE sales_staging CHANGE sale_date sale_date_new DATE;
refer this Apache Hive Wiki
you can't change the existing string type data to date type. but we can able to solve this issues in 2 ways.
create another table with the same columns count but the data type is date where the column you need string to date, then use insert command to export old table data to new table by casting the string to date.
add a new column to existing table with datatype as date. overwrite the table itself by casting the string to date into the new column.
ex:
I have orders table
describe orders;
order_id int
order_date string
order_customer_id int
order_status string
created another table ordersnew
describe ordersnew;
id int
odate date
cid int
ostatus string
now exported the orders data to ordersnew table
insert into ordersnew select order_id,cast(from_unixtime(unix_timestamp(substring(order_date,1,19), 'yyyy-MM-dd HH:mm:ss')) as timestamp) as strdate, order_customer_id,order_status from orders;
substring(order_date,1,19), 'yyyy-MM-dd HH:mm:ss' this is the place you have to check and alter your query as per your data.
please check here for date conversions
Do the following steps:
step-1) check all the dates in field "sale_date" are valid dates or not. If yes then go to step-2
step-2) Check the date format, for DATE datatype format should be 'yyyy-MM-dd'. If the date format is 'yyyy-MM-dd HH:mm:ss' then you should try the following syntax:
alter table sales_staging CHANGE sale_date sale_date TIMESTAMP;