How to replace hyphen (dash) in table name? - sql

I have a hive table, where I want to replace the hyphen ('-') with underscore ('_').
The sample query is as:
CREATE TABLE test_${yearAndMonth} ......
INSERT OVERWRITE TABLE test_${yearAndMonth} ......
The 'yearAndMonth' contains value like: 2017-05; So, I want to have the table value name as test_2017_05; however, the 'yearAndMonth' will must contain the hyphen value.
I have tried with: regex replace
For example:
CREATE TABLE test_${regexp_replace(yearAndMonth, '-', '_')} ......
INSERT OVERWRITE TABLE test_${regexp_replace(yearAndMonth, '-', '_')} ......
However, I am getting error as:
cannot recognize input near 'test_' '$' '{' in table name
Any suggestions please.
Update:
Trying in this was way:
CREATE TABLE test_regexp_replace(${yearAndMonth}, "-", "_") ......
INSERT OVERWRITE TABLE test_regexp_replace(${yearAndMonth}, "-", "_") ......
I am getting this error:
missing EOF at '(' near 'test_regexp_replace'


Changing the variable format in hive is not a good idea, try to change the format before passing. Doing something similar to below will work (added id int as a sample column, you can add your own or pass them from another variable if required)
hive --hiveconf table_name=table_$(date '+%Y')_$(date '+%m') -e "create table \${hiveconf:table_name}(id int); insert overwrite table \${hiveconf:table_name}"

Related

Need to insert special character in string as insert script

I wanted to insert records having special character in snowflake.
Having record in source table :
order/date=2022-02-18/hour=12/85b3e2d8-0195-4238-b246-7ed6564ac464.json
I need to extract hour value i.e 12
I am able to extract the value using : cast(replace(substr(METADATA$FILENAME,28,2),'/','') as number)
But I need to create the insert script , I had tried :
'cast(replace(substr(METADATA$FILENAME,28,2),'/,'') as number)'
But getting error : FAILED CODE: 0 STATE: 22018 MESSAGE: Numeric value '5/' is not recognized

I tested your string in select and insert command as below:
select cast(replace(substr('order/date=2022-02-18/hour=12/85b3e2d8-0195-4238-b246-7ed6564ac464.json',28,2),'/','') as integer);
create table t1(c1 number);
insert into t1(c1) select cast(replace(substr('order/date=2022-02-18/hour=12/85b3e2d8-0195-4238-b246-7ed6564ac464.json',28,2),'/','') as integer);
If your issue is different, then share the exact command that you are executing and that's failing.

I got the solution :
Solution Snap shot
I wanted to insert this whole statement as string , I was facing issue due to special characters : / and '' .
Used backslash to resolve it.

Create a table in hive with timestamp as comment

I would like to create a table in hive, inside the comment include the creation date (current_timestamp function). Something like this:
CREATE TABLE IF NOT EXISTS ex.tb_test ( field1 int, field2 String) COMMENT current_timestamp STORED AS TEXTFILE;
But it returns error: ILED: ParseException line 2: 8 mismatched input 'current_timestamp' expecting StringLiteral near 'COMMENT'
Do you know any way to add to the comment the creation date of the table?

Functions are not supported in table DDL. You can pass pre-calculated timestamp as a --hiveconf parameter and use for example like this: comment '${hiveconf:ts}'(it should be quoted), such parameter will be resolved as a string literal before command execution.
BTW Hive stores CreateTime.
describe formatted table_name command outputs CreateTime along with other table info.

Error while running Hive command with DATE as of the colume name

create table Book_inf2(OID int, date timestamp, CUSTOMER_ID string, AMOUNT
int) row format delimited fields terminated by ',';
Error which I got:
FAILED: ParseException line 1:32 missing Identifier at 'date' near
'date' in create table statement line 1:37 mismatched input
'timestamp' expecting ) near 'date' in create table statement
Note: I am new to the Hive, please help me to get understand.

Date is a reserved keyword in hive that's the reason why you are facing issue
However hive allows to use reserved keywords as field names, but that's not the best practice to use them.
To fix the issue:
Surround date field name with backtick's
`
Try with below create table statement
hive> create table Book_inf2(OID int, `date` timestamp, CUSTOMER_ID string, AMOUNT int) row format delimited fields terminated by ',';

How to create an external Hive table if the field value has comma separated values

I had used sqoop-import command to sqoop the data into Hive from teradata. Sqoop-import command is creating a text file with comma(,) as the delimiter.
After Sqooping, I had created an external table as shown below:
CREATE EXTERNAL TABLE IF NOT EXISTS employee ( eid int, name String,
salary String, description String)
COMMENT ‘Employee details’
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ‘,’
LINES TERMINATED BY ‘\n’
STORED AS TEXTFILE;
But description column has values like this:"abc,xyz,mnl". Due to this,loading of data into a hive table is not proper. Then how to create a text file with a delimiter other than comma while sqooping.
Then how to delimit the fields while creating an external table of Hive?

Use --fields-terminated-by in your Sqoop job if you want to avoid the default delimiter.
--fields-terminated-by - This parameter is used for field separator character in output.
Example: --fields-terminated-by |
and then change fields separator in create table statement by FIELDS TERMINATED BY ‘|’

Load Data Infile - negative decimal truncated (to positive number)

I am having trouble loading decimal data into a database - specifically, my negative numbers are getting truncated, and I can't figure it out.
Here is what my query looks like:
> CREATE TABLE IF NOT EXISTS mytable (id INT(12) NOT NULL AUTO_INCREMENT,
mydecimal DECIMAL(13,2),PRIMARY KEY(id));
> LOAD DATA INFILE 'data.dat' INTO TABLE mytable FIELDS TERMINATED BY ';';
And the data.dat that I'm loading:
;000000019.50 ;
;000000029.50-;
;000000049.50 ;
When it completes, giving me a warning that "Data truncated for column 'mydecimal' at row 2." And when I look at the data, it's stored as positive number. Any ideas how to fix this?

The best way to handle data abnormalities like this in the input file is to load them into a local variable, then set the actual column value based on a transformation of the local variable.
In your case, you can load the strings into a local variable, then either leave it alone or multiply by negative one depending on whether it ends with a minus sign.
Something like this should work for you:
LOAD DATA INFILE 'data.dat'
INTO TABLE mytable FIELDS TERMINATED BY ';'
(id,#mydecimal)
set mydecimal = IF(#mydecimal like '%-',#mydecimal * -1,#mydecimal);

I'm not sure why you're putting the minus sign after the number rather than before it. Does it work when you place the '-' sign at the start of the line?

you can consider this
CREATE TABLE IF NOT EXISTS mytable (id INT(12) NOT NULL AUTO_INCREMENT,
mydecimal varchar(255),PRIMARY KEY(id));
LOAD DATA INFILE 'data.dat' INTO TABLE mytable FIELDS TERMINATED BY ';';
update mytable set mydecimal =
cast(mydecimal as decimal(13,2))*if (substring(mydecimal, -1)='-', -1, 1);
alter table mytable modify column mydecimal decimal(13,2) signed;