I am trying to rename a columnName in Hive. Is there a way to rename column name in Hive .
tableA (column1 ,_c1,_c2)
to
tableA(column1,column2,column3)
??
Change Column Name/Type/Position/Comment:
ALTER TABLE table_name CHANGE [COLUMN] col_old_name col_new_name column_type [COMMENT col_comment] [FIRST|AFTER column_name]
Example:
CREATE TABLE test_change (a int, b int, c int);
// will change column a's name to a1
ALTER TABLE test_change CHANGE a a1 INT;
Command works only if "use" -command has been first used to define the database where working in. Table column renaming syntax using DATABASE.TABLE throws error and does not work. Version: HIVE 0.12.
EXAMPLE:
hive> ALTER TABLE databasename.tablename CHANGE old_column_name new_column_name;
MismatchedTokenException(49!=90)
at org.antlr.runtime.BaseRecognizer.recoverFromMismatchedToken(BaseRecognizer.java:617)
at org.antlr.runtime.BaseRecognizer.match(BaseRecognizer.java:115)
at org.apache.hadoop.hive.ql.parse.HiveParser.alterStatementSuffixExchangePartition(HiveParser.java:11492)
...
hive> use databasename;
hive> ALTER TABLE tablename CHANGE old_column_name new_column_name;
OK
alter table table_name change old_col_name new_col_name new_col_type;
Here is the example
hive> alter table test change userVisit userVisit2 STRING;
OK
Time taken: 0.26 seconds
hive> describe test;
OK
uservisit2 string
category string
uuid string
Time taken: 0.213 seconds, Fetched: 3 row(s)
In the comments #libjack mentioned a point which is really important. I would like to illustrate more into it. First, we can check what are the columns of our table by describe <table_name>; command.
there is a double-column called _c1 and such columns are created by the hive itself when we moving data from one table to another. To address these columns we need to write it inside backticks
`_c1`
Finally, the ALTER command will be,
ALTER TABLE <table_namr> CHANGE `<system_genarated_column_name>` <new_column_name> <data_type>;
Related
I'm planning to truncate the hive external table which has one partition. So, I have used the following command to truncate the table :
hive> truncate table abc;
But, it is throwing me an error stating : Cannot truncate non-managed table abc.
Can anyone please suggest me out regarding the same ...
Make your table MANAGED first:
ALTER TABLE abc SET TBLPROPERTIES('EXTERNAL'='FALSE');
Then truncate:
truncate table abc;
And finally you can make it external again:
ALTER TABLE abc SET TBLPROPERTIES('EXTERNAL'='TRUE');
By default, TRUNCATE TABLE is supported only on managed tables. Attempting to truncate an external table results in the following error:
Error: org.apache.spark.sql.AnalysisException: Operation not allowed: TRUNCATE TABLE on external tables
Action Required
Change applications. Do not attempt to run TRUNCATE TABLE on an external table.
Alternatively, change applications to alter a table property to set external.table.purge to true to allow truncation of an external table:
ALTER TABLE mytable SET TBLPROPERTIES ('external.table.purge'='true');
There is an even better solution to this, which is basically a one liner.
insert overwrite table table_xyz select * from table_xyz where 1=2;
This code will delete all the files and create a blank file in the external folder location with absolute zero records.
Look at https://issues.apache.org/jira/browse/HIVE-4367 : use
truncate table my_ext_table force;
I have a hive table in XYZ db named ABC.
When I run describe formatted XYZ.ABC; from hue, I get the following..
that is
Table Type: MANAGED_TABLE
Table Parameters: EXTERNAL True
So is this actually an external or a managed/internal hive table?
This is treated as an EXTERNAL table. Dropping table will keep the underlying HDFS data. The table type is being shown as MANAGED_TABLE since the parameter EXTERNAL is set to True, instead of TRUE.
To fix this metadata, you can run this query:
hive> ALTER TABLE XYZ.ABC SET TBLPROPERTIES('EXTERNAL'='TRUE');
Some details:
The table XYZ.ABC must have been created via this kind of query:
hive> CREATE TABLE XYZ.ABC
<additional table definition details>
TBLPROPERTIES (
'EXTERNAL'='True');
Describing this table will give:
hive> desc formatted XYZ.ABC;
:
Location: hdfs://<location_of_data>
Table Type: MANAGED_TABLE
:
Table Parameters:
EXTERNAL True
Dropping this table will keep the data referenced in Location in describe output.
hive> drop table XYZ.ABC;
# does not drop table data in HDFS
The Table Type still shows as MANAGED_TABLE which is confusing.
Making the value for EXTERNAL as TRUE will fix this.
hive> ALTER TABLE XYZ.ABC SET TBLPROPERTIES('EXTERNAL'='TRUE');
Now, doing a describe will show it as expected:
hive> desc formatted XYZ.ABC;
:
Location: hdfs://<location_of_data>
Table Type: EXTERNAL_TABLE
:
Table Parameters:
EXTERNAL TRUE
Example -
Lets create a sample MANAGED table,
CREATE TABLE TEST_TBL(abc int, xyz string);
INSERT INTO TABLE test_tbl values(1, 'abc'),(2, 'xyz');
DESCRIBE FORMATTED test_tbl;
Changing type to EXTERNAL (in the wrong way using True, instead of TRUE):
ALTER TABLE test_tbl SET TBLPROPERTIES('EXTERNAL'='True');
This gives,
Now lets DROP the table,
DROP TABLE test_tbl;
The result:
Table is dropped but data on HDFS isn't. Showing correct external table behavior!
If we re-create the table we can see data exists:
CREATE TABLE test_tbl(abc int, xyz string);
SELECT * FROM test_tbl;
Result:
The describe shows it wrongly as MANAGED TABLE along with EXTERNAL True because of:
.equals check in the meta
Hive Issue JIRA: HIVE-20057
Proposed fix: Use case insensitive equals
I need to alter my table and change a VARCHAR2 column to be CLOB.
The following script is too slow (8 minutes) for me when having 1 million records:
ALTER TABLE SA.EVENT_PARAMS ADD COLUMN NEW_VALUE CLOB(65344);
UPDATE SA.EVENT_PARAMS SET NEW_VALUE=VALUE;
ALTER TABLE SA.EVENT_PARAMS DROP COLUMN VALUE;
RENAME COLUMN SA.EVENT_PARAMS.NEW_VALUE TO VALUE;
Is there another way to do this? Talking about apache derby.
Thanks
Hi All the table is partitioned on column 1 and column 2 both being INT types,I am using the following command to drop the partition,column1 is equal to null or HIVE_DEFAULT_PARTITION
ALTER TABLE Table_Name DROP IF EXISTS PARTITION(column1=__HIVE_DEFAULT_PARTITION__,column2=101);
but i am getting the following error :missing \' at ',' near ''
when i do:
show partitions Table_Name (this is how the partition to be dropped looks)
Output:
column1=__HIVE_DEFAULT_PARTITION__/column2=101
Thanks for your answers guys appreciate it ..actually figured out the problem
ALTER TABLE Table_Name DROP IF EXISTS PARTITION(column1<1,column2=101);
as column1 had a null value entry which i wanted to remove which was HIVE_DEFAULT_PARTITION or (null) so using conditions <1 worked for me
Have you tried putting HIVE_DEFAULT_PARTITION in quotes
ALTER TABLE Table_Name DROP IF EXISTS
PARTITION(column1='__HIVE_DEFAULT_PARTITION__',column2=101);
You can directly drop the partition on column2.
ALTER TABLE Table_Name DROP IF EXISTS PARTITION(column2=101);
I am trying to alter a datatype for a derby db column. The current price column is set as DECIMAL(5,0). I would like to alter it to DECIMAL(7,2). I did this :
alter table item alter column price set data type DECIMAL(7,2);
But it did not work, and showing the error:
Error: Only columns of type VARCHAR may have their length altered.
May I know how is it possible to alter it? Thank you.
Here is the Derby SQL script to change column MY_TABLE.MY_COLUMN from BLOB(255) to BLOB(2147483647):
ALTER TABLE MY_TABLE ADD COLUMN NEW_COLUMN BLOB(2147483647);
UPDATE MY_TABLE SET NEW_COLUMN=MY_COLUMN;
ALTER TABLE MY_TABLE DROP COLUMN MY_COLUMN;
RENAME COLUMN MY_TABLE.NEW_COLUMN TO MY_COLUMN;
I think you can do like this:
ALTER TABLE SCHEMA.TABLE ALTER "COLUMN-NAME" SET DATA TYPE VARCHAR(255);
(column-Name SET DATA TYPE VARCHAR(integer)) for Datatype String as an example...
Here's a slightly more complicated way to alter the column's data type in this fashion:
Add a new column, of the desired data type
Issue "update ... set new-column = old-column to copy the data from the old column to the new column
drop the old column
Rename the new column to have the name of the old column.
Slightly more steps, but in the end the effect will be the same.
If you have trouble working out the exact details of the SQL to do this, let us know and we'll help.
You can alter table like this:
ALTER TABLE [table] ALTER COLUMN [column] SET DATA TYPE [type];
Or in Rails, just use:
change_column :table_name, :column_name, :integer
Posgtes Solution :
ALTER TABLE prices_table ALTER price_column TYPE decimal (7,2 )