External Tables (HIVE) Choose only a few columns from a file - hive

Question

How can I create a external table setting only a few columns from a file?
Ex: In archive I have six columns, A,B,C,D,E,F. But in my table i want only A, C, F.
Is It possible?

Answer 1

I do not know of a way to selectively include columns from HDFS files for an external table. Depending on your use case, it may be sufficient to define a view based on the external table to only include the columns you want. For example, given the following silly example of an external table:
hive> CREATE EXTERNAL TABLE ext_table (
> A STRING,
> B STRING,
> C STRING,
> D STRING,
> E STRING,
> F STRING
> )
> ROW FORMAT DELIMITED FIELDS TERMINATED BY ','
> STORED AS TEXTFILE
> LOCATION '/tmp/ext_table';
OK
Time taken: 0.401 seconds
hive> SELECT * FROM ext_table;
OK
row_1_col_A row_1_col_B row_1_col_C row_1_col_D row_1_col_E row_1_col_F
row_2_col_A row_2_col_B row_2_col_C row_2_col_D row_2_col_E row_2_col_F
row_3_col_A row_3_col_B row_3_col_C row_3_col_D row_3_col_E row_3_col_F
Time taken: 0.222 seconds, Fetched: 3 row(s)
Then create a view to only include the columns you want:
hive> CREATE VIEW filtered_ext_table AS SELECT A, C, F FROM ext_table;
OK
Time taken: 0.749 seconds
hive> DESCRIBE filtered_ext_table;
OK
a string
c string
f string
Time taken: 0.266 seconds, Fetched: 3 row(s)
hive> SELECT * FROM filtered_ext_table;
OK
row_1_col_A row_1_col_C row_1_col_F
row_2_col_A row_2_col_C row_2_col_F
row_3_col_A row_3_col_C row_3_col_F
Time taken: 0.301 seconds, Fetched: 3 row(s)
Another way to achieve what you want would require that you have the ability to modify the HDFS files backing your external table - if the columns you are interested in are all near the beginning of each line, then you can define your external table to capture only the first 3 columns (without regard for how many more columns are actually in the file). For example, with the same data file as above:
hive> DROP TABLE IF EXISTS ext_table;
OK
Time taken: 1.438 seconds
hive> CREATE EXTERNAL TABLE ext_table (
> A STRING,
> B STRING,
> C STRING
> )
> ROW FORMAT DELIMITED FIELDS TERMINATED BY ','
> STORED AS TEXTFILE
> LOCATION '/tmp/ext_table';
OK
Time taken: 0.734 seconds
hive> SELECT * FROM ext_table;
OK
row_1_col_A row_1_col_B row_1_col_C
row_2_col_A row_2_col_B row_2_col_C
row_3_col_A row_3_col_B row_3_col_C
Time taken: 0.727 seconds, Fetched: 3 row(s)

Answer 2

I found answer here
create table tmpdc_ticket(
SERVICE_ID CHAR(144),
SERVICE_TYPE CHAR(50),
CUSTOMER_NAME CHAR(200),
TELEPHONE_NO CHAR(144),
ACCOUNT_NUMBER CHAR(144),
FAULT_STATUS CHAR(50),
BUSINESS_GROUP CHAR(100)
)
organization external(
type oracle_loader
default directory sample_directory
access parameters(
records delimited by newline
nologfile
skip 1
fields terminated by '|'
missing field values are null
(DUMMY_1,
DUMMY_2,
SERVICE_ID CHAR(144),
SERVICE_TYPE CHAR(50),
CUSTOMER_NAME CHAR(200),
TELEPHONE_NO CHAR(144),
ACCOUNT_NUMBER CHAR(144),
FAULT_STATUS CHAR(50),
BUSINESS_GROUP CHAR(100)
)
)
location(sample_directory:'sample_file.txt')
)
reject limit 1
noparallel
nomonitoring;