I would like to create a table in hive, inside the comment include the creation date (current_timestamp function). Something like this:
CREATE TABLE IF NOT EXISTS ex.tb_test ( field1 int, field2 String) COMMENT current_timestamp STORED AS TEXTFILE;
But it returns error: ILED: ParseException line 2: 8 mismatched input 'current_timestamp' expecting StringLiteral near 'COMMENT'
Do you know any way to add to the comment the creation date of the table?
Functions are not supported in table DDL. You can pass pre-calculated timestamp as a --hiveconf parameter and use for example like this: comment '${hiveconf:ts}'(it should be quoted), such parameter will be resolved as a string literal before command execution.
BTW Hive stores CreateTime.
describe formatted table_name command outputs CreateTime along with other table info.
Related
For example, when using Parquet format, I'd like to be able to specify the compression scheme (("parquet.compression"="SNAPPY")). Running this query:
CREATE TABLE table_a_copy
STORED AS PARQUET
TBLPROPERTIES("parquet.compression"="SNAPPY")
AS
SELECT * FROM table_a
returns an error:
Error: Error while compiling statement: FAILED: ParseException line 1:69 cannot recognize input near 'parquet' '.' 'compression' in table properties list (state=42000,code=40000)
The same query without the TBLPROPERTIES works just fine.
This is similar to this question: Create hive table using "as select" or "like" and also specify delimiter. But I can't figure out how to make TBLPROPERTIES work with that approach. I'm using Hive 1.1.
I was able to run same exact statement in Hive 2.1.1 version.
Try with this workaround:
CREATE TABLE table_a_copy like table_a STORED AS PARQUET;
alter table set TBLPROPERTIES("parquet.compression"="SNAPPY");
insert into table table_a_copy select * from table_a ;
I want to get the current date as YYMMDD and then set it to variable in order to use it as table name.
Here is my code:
set dates= date +%Y-%m-%d;
CREATE EXTERNAL TABLE IF NOT EXISTS dates(
id STRING,
region STRING,
city STRING)
But this method doesn't work, because it seems the assignments are wrong. Any idea?
Hive does not calculate variables, it substitutes them as is, in your case it will be exactly this string 'date +%Y-%m-%d'. Also it is not possible to use UDF like current_date() in place of table name in DDL.
The solution is to calculate variable in the shell and pass it to Hive:
In the shell
dates=$(date +%Y_%m_%d);
hive --hivevar date="$dates" -f myscript.hql
In the script:
use mydb; create table if not exists tab_${hivevar:date} (id int);
Or you can execute hive script from command line using hive -e, in this case variable can be substituted using shell:
dates=$(date +%Y_%m_%d);
hive -e "use mydb; create table if not exists tab_${dates} (id int);"
create table Book_inf2(OID int, date timestamp, CUSTOMER_ID string, AMOUNT
int) row format delimited fields terminated by ',';
Error which I got:
FAILED: ParseException line 1:32 missing Identifier at 'date' near
'date' in create table statement line 1:37 mismatched input
'timestamp' expecting ) near 'date' in create table statement
Note: I am new to the Hive, please help me to get understand.
Date is a reserved keyword in hive that's the reason why you are facing issue
However hive allows to use reserved keywords as field names, but that's not the best practice to use them.
To fix the issue:
Surround date field name with backtick's
`
Try with below create table statement
hive> create table Book_inf2(OID int, `date` timestamp, CUSTOMER_ID string, AMOUNT int) row format delimited fields terminated by ',';
I have a hive table, where I want to replace the hyphen ('-') with underscore ('_').
The sample query is as:
CREATE TABLE test_${yearAndMonth} ......
INSERT OVERWRITE TABLE test_${yearAndMonth} ......
The 'yearAndMonth' contains value like: 2017-05; So, I want to have the table value name as test_2017_05; however, the 'yearAndMonth' will must contain the hyphen value.
I have tried with: regex replace
For example:
CREATE TABLE test_${regexp_replace(yearAndMonth, '-', '_')} ......
INSERT OVERWRITE TABLE test_${regexp_replace(yearAndMonth, '-', '_')} ......
However, I am getting error as:
cannot recognize input near 'test_' '$' '{' in table name
Any suggestions please.
Update:
Trying in this was way:
CREATE TABLE test_regexp_replace(${yearAndMonth}, "-", "_") ......
INSERT OVERWRITE TABLE test_regexp_replace(${yearAndMonth}, "-", "_") ......
I am getting this error:
missing EOF at '(' near 'test_regexp_replace'
Changing the variable format in hive is not a good idea, try to change the format before passing. Doing something similar to below will work (added id int as a sample column, you can add your own or pass them from another variable if required)
hive --hiveconf table_name=table_$(date '+%Y')_$(date '+%m') -e "create table \${hiveconf:table_name}(id int); insert overwrite table \${hiveconf:table_name}"
I am creating table in hive like:
CREATE TABLE SEQUENCE_TABLE(
SEQUENCE_NAME VARCHAR2(225) NOT NULL,
NEXT_VAL NUMBER NOT NULL
);
But, in result there is parse exception. Unable to read Varchar2(225) NOT NULL.
Can anyone guide me that how to create table like given above and any other process to provide path for it.
There's no such thing as VARCHAR, field width or NOT NULL clause in hive.
CREATE TABLE SEQUENCE_TABLE( SEQUENCE_TABLE string, NEXT_VAL bigint);
Please read this for CREATE TABLE syntax:
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-CreateTable
Anyway Hive is "SQL Like" but it's not "SQL". I wouldn't use it for things such as sequence table as you don't have support for transactions, locking, keys and everything you are familiar with from Oracle (though I think that in new version there is simple support for transactions, updates, deletes, etc.).
I would consider using normal OLTP database for whatever you are trying to achieve
only you have option here like:
CREATE TABLE SEQUENCE_TABLE(SEQUENCE_NAME String,NEXT_VAL bigint) row format delimited fields terminated by ',' stored as textfile;
PS:Again depends the types to data you are going to load in hive
Use following syntax...
CREATE [TEMPORARY] [EXTERNAL] TABLE [IF NOT EXISTS] [db_name.] table_name
[(col_name data_type [COMMENT col_comment], ...)]
[COMMENT table_comment]
[ROW FORMAT row_format]
[STORED AS file_format]
And Example of hive create table
CREATE TABLE IF NOT EXISTS employee ( eid int, name String,
salary String, destination String)
COMMENT ‘Employee details’
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ‘\t’
LINES TERMINATED BY ‘\n’
STORED AS TEXTFILE;