Hive : Cannot copy data from unpartitioned table to partitioned table - hive

I have an unpartitioned table
create table tabUn
(
col1 string,
col2 int
)
Lets say it has some data. Next I created a partitioned table
CREATE EXTERNAL TABLE tabPart
(
col1 string,
col2 int
)
PARTITIONED BY (col_date string)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t'
STORED AS TEXTFILE
LOCATION '/path/to/table';
Finally, I tried to copy the data over
set hive.exec.dynamic.partition=true;
set hive.exec.dynamic.partition.mode=nonstrict;
INSERT OVERWRITE TABLE tabPart PARTITION(data_date='2018-10-01')
SELECT
(
col1,
col2,
'2018-10-01' as col_date
) select * FROM tabUn;
but I get the below error
FAILED: NullPointerException null
What am I doing wrong?

Your select statement seems to be incorrect.
INSERT OVERWRITE TABLE tabPart PARTITION (data_date='2018-10-01')
SELECT col1,col2,'2018-10-01' as col_date from tabUn;

Related

Insert into a table with the data present in one table which match the first row of another table

I am new to sql.
I want to insert a data to backup table from main table that also matches the first record of another table.
suppose I have backup table with name "baktble" and main table with name "sales".
note : both tables have same columns c1,c2,c3,c4,c5,c6.
and I have a buffer table "buftble" with columns only the first 3 columns of bakup and sales table c1,c2,c3.
Now how to insert a data into backup table from sales table which matches the columns of first record.
I tired this but got error.
insert into baktble
select *
from sales
where col1,col2,col3 in (
select top 1 col1,col2,col3
from buftble).
So while nbk's as is the correct SQL syntax for your query:
create table baktble(col1 int, col2 int, col3 int, col4 int, col5 int, col6 int);
create or replace table sales(col1 int, col2 int, col3 int, col4 int, col5 int, col6 int);
create or replace table buftble(col1 int, col2 int, col3 int);
insert into buftble values
(1,10,100),
(2,20,200),
(3,30,300);
insert into sales values
(1,10,100, 101, 102, 103),
(2,20,200, 201, 202, 203),
(3,30,300, 301, 302, 303);
insert into baktble
select *
from sales
where (col1,col2,col3) in (
select top 1 col1,col2,col3
from buftble);
this only ever inserts zero or one row from sales into baktble.
Because the top 1 only selects one row from buftble
If I had to guess what you are wanting to do is ether:
insert all sales that match the distinct rows in buftble
insert the first sales row that matches the distinct rows in buftble
The first is done with:
insert into baktble
select *
from sales
where (col1,col2,col3) in (
select distinct col1,col2,col3
from buftble);
and the later (with the assumption that col4 is valid to sort duplicate values by) using QUALIFY and ROW_NUMBER like so:
insert into baktble
select *
from sales
where (col1,col2,col3) in (
select distinct col1,col2,col3
from buftble)
qualify row_number() over (partition by col1,col2,col3 order by col4 desc) = 1;

Create automatically generated timestamp column in table?

How to create a automatically generated timestamp column in table in Microsoft SQL Server 2019? Timestamp column should be automatically generated when I insert or update table.
In IBM Db2 database the syntax is the following:
create table myschema.mytable (col1 int, col2 timestamp not null generated always for each row on update as row change timestamp
insert into myschema.mytable (col1) values (1)
update myschema.mytable set col1 = 2
After insert/update of column col1, column col2 is automatically generated as current timestamp.
In Microsoft SQL Server you can try this code:
CREATE TABLE myschema.mytable
(
col1 int,
col2 datetime not null default(current_timestamp)
)
INSERT INTO myschema.mytable(col1) VALUES (1)
UPDATE myschema.mytable SET col1 = 2
SELECT * FROM myschema.mytable
Update:
Let's create temporary table for test
DECLARE #mytable TABLE
(
col1 int,
col2 datetime not null default(current_timestamp)
)
INSERT INTO #mytable(col1) VALUES (1)
SELECT * FROM #mytable
UPDATE #mytable SET col1 = 2
SELECT * FROM #mytable

Exclude column in redshift spectrum sql queries

In my table having columns col1,col2.....coln
I want to
select all columns except col1
instead of writing select col2,col3.... coln from I can specify
select * from <table name> except col1
Select all column excluding one column
You can do something like this-
-- create a temporary copy of your table
SELECT * INTO temp_table FROM original_table;
-- drop the column you don't need
ALTER TABLE temp_table DROP COLUMN col1;
-- select all columns
SELECT * FROM temp_table;
-- drop the temporary table
DROP TABLE temp_table;

Exchange column order of table after it is created

I have created a table with 4 columns. I need to change the structure of the table. I need to interchange the position of the 4th and 2nd columns permanently. Is this possible in Oracle?
Not possible. See this.
Oracle only allows columns to be added to the end of an existing
table.
So you must drop and recreate the tables.
You can run a script like this:
CREATE TABLE TMP_TBL as SELECT * FROM TBL_ORIG;
ALTER TABLE TBL_ORIG ADD COLUMN COL3;
DROP TABLE TBL_ORIG;
CREATE TABLE TBL_ORIG AS SELECT COL1, COL3, COL2 FROM TMP_TBL;
DROP TABLE TMP_TBL
You would need to consider indexes as well as storage concerns.
Why in the world is this necessary? Column order means nothing in SQL.
Swap of columns col1 and col2
It is assumed that col1 is int and col2 is varchar2(20)
-- drop all indexes and constraints concerning col1 and col2
alter table your_table add temp_col int; -- type of col1
update your_table set col1 = null, temp_col = col1;
alter table your_table modify col1 varchar2(20); -- type of col2
update your_table set col2 = null, col1 = col2;
alter table your_table modify col2 int; -- type of col1
update your_table set col2 = temp_col;
alter table your_table drop column temp_col;
alter table your_table rename column col1 to temp_col;
alter table your_table rename column col2 to col1;
alter table your_table rename column temp_col to col1;
-- recreate indexes and constraints
Simply rename table columns if they are the same datatype. If not then Alter - see Sean and Egor examples.
Rename:
http://docs.oracle.com/cd/E11882_01/server.112/e25494/tables006.htm#ADMIN11662
And on the interview they are looking for Sean's answer. Just FYI...

Inserting data into a temporary table

After having created a temporary table and declaring the data types like so;
CREATE TABLE #TempTable(
ID int,
Date datetime,
Name char(20))
How do I then insert the relevant data which is already held on a physical table within the database?
INSERT INTO #TempTable (ID, Date, Name)
SELECT id, date, name
FROM physical_table
To insert all data from all columns, just use this:
SELECT * INTO #TempTable
FROM OriginalTable
Don't forget to DROP the temporary table after you have finished with it and before you try creating it again:
DROP TABLE #TempTable
SELECT ID , Date , Name into #temp from [TableName]
My way of Insert in SQL Server. Also I usually check if a temporary table exists.
IF OBJECT_ID('tempdb..#MyTable') IS NOT NULL DROP Table #MyTable
SELECT b.Val as 'bVals'
INTO #MyTable
FROM OtherTable as b
SELECT *
INTO #TempTable
FROM table
I have provided two approaches to solve the same issue,
Solution 1: This approach includes 2 steps, first create a temporary table with
specified data type, next insert the value from the existing data
table.
CREATE TABLE #TempStudent(tempID int, tempName varchar(MAX) )
INSERT INTO #TempStudent(tempID, tempName) SELECT id, studName FROM students where id =1
SELECT * FROM #TempStudent
Solution 2: This approach is simple, where you can directly insert the values to
temporary table, where automatically the system take care of creating
the temp table with the same data type of original table.
SELECT id, studName INTO #TempStudent FROM students where id =1
SELECT * FROM #TempStudent
After you create the temp table you would just do a normal INSERT INTO () SELECT FROM
INSERT INTO #TempTable (id, Date, Name)
SELECT t.id, t.Date, t.Name
FROM yourTable t
The right query:
drop table #tmp_table
select new_acc_no, count(new_acc_no) as count1
into #tmp_table
from table
where unit_id = '0007'
group by unit_id, new_acc_no
having count(new_acc_no) > 1
insert into #temptable (col1, col2, col3)
select col1, col2, col3 from othertable
Note that this is considered poor practice:
insert into #temptable
select col1, col2, col3 from othertable
If the definition of the temp table were to change, the code could fail at runtime.
Basic operation of Temporary table is given below, modify and use as per your requirements,
-- CREATE A TEMP TABLE
CREATE TABLE #MyTempEmployeeTable(tempUserID varchar(MAX), tempUserName varchar(MAX) )
-- INSERT VALUE INTO A TEMP TABLE
INSERT INTO #MyTempEmployeeTable(tempUserID,tempUserName) SELECT userid,username FROM users where userid =21
-- QUERY A TEMP TABLE [This will work only in same session/Instance, not in other user session instance]
SELECT * FROM #MyTempEmployeeTable
-- DELETE VALUE IN TEMP TABLE
DELETE FROM #MyTempEmployeeTable
-- DROP A TEMP TABLE
DROP TABLE #MyTempEmployeeTable
INSERT INTO #TempTable(ID, Date, Name)
SELECT OtherID, OtherDate, OtherName FROM PhysicalTable
insert #temptable
select idfield, datefield, namefield from yourrealtable
All the above mentioned answers will almost fullfill the purpose. However, You need to drop the temp table after all the operation on it. You can follow-
INSERT INTO #TempTable (ID, Date, Name)
SELECT id, date, name
FROM physical_table;
IF OBJECT_ID('tempdb.dbo.#TempTable') IS NOT NULL
DROP TABLE #TempTable;