I have table A in teradata which has count of 40k records, have same table in hive which has only 37k records, so im trying to update the hive table with the missing records from TD. I created a stage table in hive, to get all the 40k records from TD and then do a INSERT OVERWRITE into the final table in HIVE to update the missing records. HOw should the insert overwrite syntax be?
What i did was.
INSERT OVERWRITE TABLE A PARTITION (column X)
select ( column A
column B
,,
column Z)
from stage table stg
left join final table f on stg.x= f.x and
where f.x is NULL
basically, im looking to insert records into the final HIVE table, which are present in Teradata table, but not the HIVE table.
Want to know if the approach is correct or am i wrong? Thanks
Tried this and worked:
INSERT OVERWRITE TABLE A PARTITION (column X)
select ( column A
column B
,,
column Z)
from final table f
union
select
column 1
column 2
from stage table stg
left join final table f on stg.x= f.x and
where f.x is NULL
Related
I have table A as truncate and load for every month file and table B will be append
So table A will be file to table in hive
Table B will be tableA Insert and append data
Issue here is table B is straight move select stmt from table A , and chances are it can be inserted with duplicate/ same data
How should I write a select query to insert data from Table A
Both tables will have file-date as the column
Left join A and B is giving wrong counts in this insert tables
And hive is not working for not exists code
Issue Is:
Append table script : partitioned by yearmonth
Insert into table dist.t2
Select
Person_sk,
Np_id,
Yearmonth,
Insert_date
File_date
From table raw.ma
Data in Table raw.ma —this is truncate and reload
File1 data:201902
File2data:201903
File3data:201904
File4data: if 201902 data gets loaded to table — this should not duplicate the file1 data.. it should either not get inserted or should overwrite that partition
Here I need a filter or where condition to append data into dist.t2
Can you please help with this ??
I tried alter drop table partition in hive, but it’s failing in the spark framework
Please help with avoiding duplicate entries insert
I have a non empty table A and want to append this, with data from table B (both have same #columns and data types).
Can I use normal Teradata sql insert syntax like
insert into table A
select * from table B;
Should I make the table A immutable while creating ?
if the sequence of column, data type and column is same in both the tables then you can use the syntax
insert into table A select * from B;
and if the sequence is not same then you have to select each column from the B in the same sequence as table A.
i want to insert all rows of one hive table to another hive table
insert into table <table_name> as select * from <table_bkp>
i have many rows in table but it is inserting only one row from to
Please suggest the solution for it
and i am using hive 1.2.1 version
In your query remove 'as' and write the query as follows
insert into table <table_name> select * from <table_bkp>
A table 'A'is there which is partitioned. The another table 'B' is not partitioned . How to insert the values of B into A? Will error be thrown?
Yes, you can insert from a non-partitioned table to a partitioned table. You will either have to define the partition you want to insert into or have Hive do it dynamically.
For example, to dynamically insert into partitions, you could run something similar to:
SET hive.exec.dynamic.partition.mode=nonstrict;
INSERT INTO TABLE A PARTITION (partition) SELECT col1, col2, ..., colN, partition FROM B WHERE .... ;
More information regarding Hive Partitions with dynamic inserts can be found here : https://cwiki.apache.org/confluence/display/Hive/DynamicPartitions. Take note, the last column in your SELECT is what is used for the partition insert. Another thing to note is that you need the number of columns to match between the two tables, otherwise you will have to fill in NULLs.
table A's structure is a subset of table B, that means the table A's all the columns are the first columns of table B, but table B has more columns than table A. My question, what's the SQL statment to copy all the rows from table A to table B(the missing columns in table B will be kept empty).
Use:
INSERT INTO TABLE_B
SELECT col1,
col2,
col3,
NULL
FROM TABLE_A
Use NULL as the placeholder for however many columns you can't populate from TABLE_A, assuming TABLE_B columns allow NULL values.