Is there a command in hive that would alter the serde properties of an existing table . Well the tables are created using com.bizo.hive.serde.csv.CSVSerde which needs to be changed to org.apache.hadoop.hive.serde2.OpenCSVSerde,I am looking for something like:
alter table table_X change serde
Thanks,
This will help:
ALTER TABLE TABLE_NAME SET SERDE 'org.apache.hadoop.hive.serde2.OpenCSVSerde';
Related
I have a Hive table that was originally created as transactional, but I want to disable transactions on the table because they are not actually needed.
I tried to disable them using ALTER TABLE, but I got an error:
hive> ALTER TABLE foo SET TBLPROPERTIES('transactional'='false');
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Unable to alter table. TBLPROPERTIES with 'transactional'='true' cannot be unset
I am using Hive 2.3.2
According to the documentation changing TBLPROPERTIES ("transactional"="false") is not allowed.
You can re-create the table.
Do table backup first:
create table bkp_table as
select * from your_table;
Then drop table and create again without transactional property. Reload data from backup.
Or make a new table, load data from old one, delete old, rename new.
You have to re-create the table.
First backup table if you want. then, DROP TABLE
Create Table with TBLPROPERTIES ( 'transactional'='false' )
CREATE TABLE your_table(
`col` string,
`col2` string)
ROW FORMAT SERDE
'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
STORED AS INPUTFORMAT
'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
TBLPROPERTIES (
'transactional'='false'
)
You can Choose Input and Output format
I'm planning to truncate the hive external table which has one partition. So, I have used the following command to truncate the table :
hive> truncate table abc;
But, it is throwing me an error stating : Cannot truncate non-managed table abc.
Can anyone please suggest me out regarding the same ...
Make your table MANAGED first:
ALTER TABLE abc SET TBLPROPERTIES('EXTERNAL'='FALSE');
Then truncate:
truncate table abc;
And finally you can make it external again:
ALTER TABLE abc SET TBLPROPERTIES('EXTERNAL'='TRUE');
By default, TRUNCATE TABLE is supported only on managed tables. Attempting to truncate an external table results in the following error:
Error: org.apache.spark.sql.AnalysisException: Operation not allowed: TRUNCATE TABLE on external tables
Action Required
Change applications. Do not attempt to run TRUNCATE TABLE on an external table.
Alternatively, change applications to alter a table property to set external.table.purge to true to allow truncation of an external table:
ALTER TABLE mytable SET TBLPROPERTIES ('external.table.purge'='true');
There is an even better solution to this, which is basically a one liner.
insert overwrite table table_xyz select * from table_xyz where 1=2;
This code will delete all the files and create a blank file in the external folder location with absolute zero records.
Look at https://issues.apache.org/jira/browse/HIVE-4367 : use
truncate table my_ext_table force;
I am trying to create an external table with tblproperties in Hive. The table gets created but it does not display the rows. Any ideas? Please find the scripts i am using below:
Thanks for your time and suggestions in advance.
Data is in a recursive folder: /user/test/test1/test2/samplefile.csv
use dw_raw;
drop table if exists temp_external_tab1;
create external table if not exists temp_external_tab1 (
col1 int,
col2 string,
col3 string,
col4 string
)
row format delimited fields terminated by ','
lines terminated by '\n'
stored as textfile
location '/user/test/test1/'
tblproperties ("hive.input.dir.recursive" = "TRUE",
"hive.mapred.supports.subdirectories" = "TRUE",
"hive.supports.subdirectories" = "TRUE",
"mapred.input.dir.recursive" = "TRUE");
These are not table properties, but global settings.
You should set these using 'set', i.e.:
set hive.mapred.supports.subdirectories=true;
set mapred.input.dir.recursive=true;
You've created a table but haven't put any data into it. Try
hive> LOAD DATA LOCAL INPATH '/user/test/test1/test2/samplefile.csv'
INTO TABLE temp_external_tab1;
If you are using ambari the set the following properties to hive advanced config inside custom hive-site.xml.
SET hive.input.dir.recursive=TRUE
SET hive.mapred.supports.subdirectories=TRUE
SET hive.supports.subdirectories=TRUE
SET mapred.input.dir.recursive=TRUE
And then restart the affected services. This will read all the data recursively.
I am trying to rename a columnName in Hive. Is there a way to rename column name in Hive .
tableA (column1 ,_c1,_c2)
to
tableA(column1,column2,column3)
??
Change Column Name/Type/Position/Comment:
ALTER TABLE table_name CHANGE [COLUMN] col_old_name col_new_name column_type [COMMENT col_comment] [FIRST|AFTER column_name]
Example:
CREATE TABLE test_change (a int, b int, c int);
// will change column a's name to a1
ALTER TABLE test_change CHANGE a a1 INT;
Command works only if "use" -command has been first used to define the database where working in. Table column renaming syntax using DATABASE.TABLE throws error and does not work. Version: HIVE 0.12.
EXAMPLE:
hive> ALTER TABLE databasename.tablename CHANGE old_column_name new_column_name;
MismatchedTokenException(49!=90)
at org.antlr.runtime.BaseRecognizer.recoverFromMismatchedToken(BaseRecognizer.java:617)
at org.antlr.runtime.BaseRecognizer.match(BaseRecognizer.java:115)
at org.apache.hadoop.hive.ql.parse.HiveParser.alterStatementSuffixExchangePartition(HiveParser.java:11492)
...
hive> use databasename;
hive> ALTER TABLE tablename CHANGE old_column_name new_column_name;
OK
alter table table_name change old_col_name new_col_name new_col_type;
Here is the example
hive> alter table test change userVisit userVisit2 STRING;
OK
Time taken: 0.26 seconds
hive> describe test;
OK
uservisit2 string
category string
uuid string
Time taken: 0.213 seconds, Fetched: 3 row(s)
In the comments #libjack mentioned a point which is really important. I would like to illustrate more into it. First, we can check what are the columns of our table by describe <table_name>; command.
there is a double-column called _c1 and such columns are created by the hive itself when we moving data from one table to another. To address these columns we need to write it inside backticks
`_c1`
Finally, the ALTER command will be,
ALTER TABLE <table_namr> CHANGE `<system_genarated_column_name>` <new_column_name> <data_type>;
I am trying to alter a datatype for a derby db column. The current price column is set as DECIMAL(5,0). I would like to alter it to DECIMAL(7,2). I did this :
alter table item alter column price set data type DECIMAL(7,2);
But it did not work, and showing the error:
Error: Only columns of type VARCHAR may have their length altered.
May I know how is it possible to alter it? Thank you.
Here is the Derby SQL script to change column MY_TABLE.MY_COLUMN from BLOB(255) to BLOB(2147483647):
ALTER TABLE MY_TABLE ADD COLUMN NEW_COLUMN BLOB(2147483647);
UPDATE MY_TABLE SET NEW_COLUMN=MY_COLUMN;
ALTER TABLE MY_TABLE DROP COLUMN MY_COLUMN;
RENAME COLUMN MY_TABLE.NEW_COLUMN TO MY_COLUMN;
I think you can do like this:
ALTER TABLE SCHEMA.TABLE ALTER "COLUMN-NAME" SET DATA TYPE VARCHAR(255);
(column-Name SET DATA TYPE VARCHAR(integer)) for Datatype String as an example...
Here's a slightly more complicated way to alter the column's data type in this fashion:
Add a new column, of the desired data type
Issue "update ... set new-column = old-column to copy the data from the old column to the new column
drop the old column
Rename the new column to have the name of the old column.
Slightly more steps, but in the end the effect will be the same.
If you have trouble working out the exact details of the SQL to do this, let us know and we'll help.
You can alter table like this:
ALTER TABLE [table] ALTER COLUMN [column] SET DATA TYPE [type];
Or in Rails, just use:
change_column :table_name, :column_name, :integer
Posgtes Solution :
ALTER TABLE prices_table ALTER price_column TYPE decimal (7,2 )