I was creating table in hive with
create table stores(
storeid integer,
area string,
city string,
state string,
country string)
Try using INT in place of Integer.
I have copied and pasted your ddl statement above into my hive cli and it worked perfectly.
You need to check your hive installation
INTEGER can be used from Hive version 2.2.0 onwards. For earlier versions, INT should be used.
Try using this DDL:
create table stores(
storeid int,
area string,
city string,
state string,
country string);
References: https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Types#LanguageManualTypes-NumericTypes
https://issues.apache.org/jira/browse/HIVE-14950
Related
I have a parquet file on gcp storage. File converted from simple json {"id":1,"name":"John"}.
Could you help me write the correct script? Is it possible to do that without schema?
create external table test (
id string,
name string
)
row format delimited
fields terminated by '\;'
stored as ?????
location '??????'
tblproperties ('skip.header.line.count'='1');
Hive is , as sql databases, working in a write-in schema architecture so you cannot create a table using HQL without using a schema ( not like other cases for NoSql like Hbase for example or others). I advise you to use a Hive version >= 0.14, it is easier:
CREATE TABLE table_name (
string1 string,
string2 string,
int1 int,
boolean1 boolean,
long1 bigint,
float1 float,
double1 double,
inner_record1 struct,
enum1 string,
array1 array,
map1 map,
union1 uniontype,
fixed1 binary,
null1 void,
unionnullint int,
bytes1 binary)
PARTITIONED BY (ds string);
I'd like to use this query to create a partitioned table in Amazon Athena:
CREATE TABLE IF NOT EXISTS
testing.partitioned_test(order_id bigint, name string, car string, country string)
PARTITIONED BY (year int)
ROW FORMAT SERDE 'parquet.hive.serde.ParquetHiveSerDe'
STORED AS 'PARQUET'
LOCATION 's3://testing-imcm-into/partitions'
Unfortunately I don't get the error message which tells me the following:
line 3:2: mismatched input 'partitioned' expecting {, 'with'}
The quotes around 'PARQUET' seemed to be causing a problem.
Try this:
CREATE EXTERNAL TABLE IF NOT EXISTS
partitioned_test (order_id bigint, name string, car string, country string)
PARTITIONED BY (year int)
STORED AS PARQUET
LOCATION 's3://testing-imcm-into/partitions/'
I have an IBM cloud where I have Hive/Hbase, I just create a "table" on Hive and I also load some data from a csv file.
My csv file contains information from google play store apps.
My commands for creating and upload data to my table are the following ones:
hive> create table if not exists app_desc (name string,
category string, rating int,
reviews int, installs string,
type string, price int,
content string, genres string,
last_update string, current_ver string,
android_ver string)
row format delimited fields terminated by ',';
hive > load data local inpath '/home/uamibm130/googleplaystore.csv' into table app_desc;
Ok, It works correctly and using a Select I obtain the data correctly.
Now what I want to do is to create a HBASE table, my problem is that I don't know how to do it correctly.
First of all I create a Hbase Db -> create google_db_ , google_data, info_data
Now I try to create an external table using this hive command, but what I am getting is an error that my table is not found.
This is the command I am using for the creation of the external hive table.
create external table uamibm130_hbase_google (name string, category string, rating int, reviews int, installs string, type string, price int, content string, genres string, last_update string, current_ver string, android_ver string)
stored by 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,
google_data:category,google_data:rating, info_data:reviews,
info_data:installs, info_data:type, info_data:price, info_data:content,
info_data:genres, info_data:last_update, info_data:current_ver,
info_data:android_ver") TBLPROPERTIES("hbase.table.name" = "google_db_");
I don't know the correct way for the creation of Hbase table based on an Hive schema, for uploading correctly my .csv data.
Any idea ? I am new on it.
Thanks!
Try with below create table statement in HBase,
Create Hbasetable:
hbase(main):001:0>create 'google_db_','google_data','info_data'
Create Hive External table on Hbase:
hive> create external table uamibm130_hbase_google (name string, category string, rating int, reviews int, installs string, type string, price int, content string, genres string, last_update string, current_ver string, android_ver string)
stored by 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,
google_data:category,google_data:rating, info_data:reviews,
info_data:installs, info_data:type, info_data:price, info_data:content,
info_data:genres, info_data:last_update, info_data:current_ver,
info_data:android_ver") TBLPROPERTIES("hbase.table.name" = "google_db_",
"hbase.mapred.output.outputtable" = "google_db_");
Then insert data into Hive-Hbase table(uamibm130_hbase_google) from Hive table(app_desc).
Insert data into Hive-Hbase table:
Hive> insert into table uamibm130_hbase_google select * from app_desc;
I am new to hadoop hive. I am using open source hadoop 2.7.1 hive 1.2.2. It is installed on ubuntu a single node cluster. I have 106 rows and 30 columns data in csv file. I import it into hive table using following code:
CREATE TABLE clinicaldatabc (comp_tcga_id String, gender String, age_inti_diag int, ER_status String, PR_status String, HER2_final_status String, Tumor String, Tumor_T1_code String, Node String, Node_coded String, Metastasis String, Metastasis_coded String, AJCC_Stage String, Converted_stage String, Survival_dt_from String, Vital_Status String, d_to_date_of_last_contact int, d_to_Day_of_Death int, OS_event int,OS_time int, PAM50_mRNA String, SigClust_unsupervised_mRNA int, SigClust_intrinsic_mRNA int, miRNA_clusters int, methylation_clusters int,RPPA_clusters int, CN_clusters int, integrated_clusters_with_PAM50 int, integrated_cluster_no_exp int, integrated_clusters_unsup_exp int)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ',';
Then I got null column name:
first half of returns
second half of returns
Please help me how to solve it. Thank you in advance!
Possible duplicate of NULL column names in Hive query result
The first thing to note here is that the NULL values occur in columns that are not of type string
Have a refer
CREATE EXTERNAL TABLE IF NOT EXISTS ejREGandTEST(
DBN STRING,
School_name STRING,
Year_of_SHST INT,
Grade_level INT,
Enrollment INT,
Number_of_registered INT,
Number_students_SHSAT INT)
row format delimited fields terminated by ','
location "/user/ebin/kaggleData/csv"
TBLPROPERTIES("skip.header.line.count"="1");
I am new to Hive and got an error when I tried to create a partitioned table.
here is my script:
CREATE　TABLE partitionedemp(emp_no int, birth_date string, first_name string, last_name string) Partitioned By(gender string, hire_date string) stored as sequencefile;
I got an error below:
bash: syntax error near unexpected token `('
What's wrong with my script? Thank you very much.
Everything is fine , Just you have ^M characters between CREATE and TABLE keywords. To check, paste this command in vi editor and run command like this:-
$ cat -v file.txt
CREATEM-cM-^#M-^#TABLE partitionedemp(emp_no int, birth_date string, first_name string, last_name string) Partitioned By(gender string, hire_date string) stored as sequencefile;
Solution:- Straightforward one - type full create table statement on hive prompt instead of copy and paste.