Inserting into multiple hive tables using WITH being ignored - hive

I have the query in the following form:-
WITH
stage1 AS
(
...
),
stage2 AS
(
...
),
stage3 AS
(
...
)
FROM stage3
INSERT OVERWRITE TABLE T1 Patition (category = 13)
select c1,c2,c3
INSERT OVERWRITE TABLE T2 Patition (category = 13)
select c1,c2,c3;
It succeeds, but the problem is that only the first insert seems to be executing, the second insert is being ignored. So only the table T1 is getting the data. If I swap the order of the insert statements, then T1 doesn't get data and rather T2 gets it. So whichever INSERT statement appears last is being ignored. How to make it work?

Common Table Expression (CTE) are not usable across 1 Select statements.
Thus you may have to use the CTE's twice to get the data into the second Select statement.
WITH
stage1 AS
(
...
),
stage2 AS
(
...
),
stage3 AS
(
...
)
FROM stage3
INSERT OVERWRITE TABLE T1 Patition (category = 13)
select c1,c2,c3
WITH
stage1 AS
(
...
),
stage2 AS
(
...
),
stage3 AS
(
...
)
FROM stage3
INSERT OVERWRITE TABLE T2 Patition (category = 13)
select c1,c2,c3

Related

How to join multiple CTE's into a temp table

I have been given a script to clean up which uses approx 85 temp tables, I have been advised to use Common Table Expressions.
I have 3 CTE's, the first is the result of 7 tables pulled together using Union all. Followed by 2 more CTE's. The script runs up to:
select * from CTE_1
Union all
select * from CTE_2
Union all
select * from CTE_3
I then want to put all these results into a reusable table so I can then add some joins with various case statement logic. How can I put these into a temp table so that I can reference it later.
I'm looking to reduce the amount of temp tables so rather than put each CTE into a temp table I would ideally put multiple CTE's into one temp table. I currently have:
; with [CTE One] as (
select 1 as col
),
[CTE Two] as (
select 2 as col
),
[CTE Three] as (
select 3 as col
)
select * from CTE_1
Union all
select * from CTE_2
Union all
select * from CTE_3
Alternatively..
IF ( OBJECT_ID('tempdb..#temptable') IS NOT NULL )
BEGIN
DROP TABLE #temptable
END
CREATE TABLE #temptable
(
val int
)
;
WITH [CTE One]
AS ( SELECT 1 AS col
),
[CTE Two]
AS ( SELECT 2 AS col
),
[CTE Three]
AS ( SELECT 3 AS col
)
INSERT INTO #temptable (val)
SELECT *
FROM ( SELECT *
FROM CTE_1
UNION ALL
SELECT *
FROM CTE_2
UNION ALL
SELECT *
FROM CTE_3
) T
Can't you just use into?
select *
into #temptable
from CTE_1
Union all
select * from CTE_2
Union all
select * from CTE_3;
I might also be inclined to use a table variable, if the code is structured appropriately.

How to add the sum of the 'sum of the two tables?'

I created Tables T1 and T2. I managed to add their sum, but I can't seem to add the sum of the T1 and T2 together (10+12 = 22) by adding a sum() in the beginning of the code.
CREATE TABLE T1(kW int)
CREATE TABLE T2(kW int)
SELECT T1C1, T2C1
FROM
( select SUM(Kw) T1C1 FROM T1 ) A
CROSS JOIN
( select SUM(Kw) T2C1 FROM T2 ) B
BEGIN
INSERT INTO T1 VALUES ('4');
INSERT INTO T1 VALUES ('1');
INSERT INTO T1 VALUES ('5');
INSERT INTO T2 VALUES ('7');
INSERT INTO T2 VALUES ('2');
INSERT INTO T2 VALUES ('3');
END
You should use union all to create a "virtual" column from the columns in the two tables:
SELECT SUM(kw)
FROM (SELECT kw FROM t1
UNION ALL
SELECT kw FROM t2) t
Try using a stored procedure. Doing so you will be able to store the sum of each table on a separated variable and then return the SUM of those two variables.
You can also make a UNION ALL and SUM the column you want. Notice that you should a UNION ALL to avoid eliminating duplicated values.
Another approach is to add the results of the two subqueries directly, using the built-in dummy table dual as the main driving table:
select ( select SUM(Kw) FROM T1 )
+ ( select SUM(Kw) FROM T2 ) as total
from dual;
TOTAL
----------
22

sql select only first 3 rows from second table

I only need 3 records from child table.
CREATE TABLE project
(prj_code int , prj_name varchar(255) );
CREATE TABLE task
(id int , prj_code int, task_name varchar(255) );
INSERT INTO project VALUES (1,'myproj1');
INSERT INTO project VALUES (2,'myproj2');
INSERT INTO task VALUES (1,1,'testing');
INSERT INTO task VALUES (2,1,'prod');
INSERT INTO task VALUES (3,1,'prod2');
INSERT INTO task VALUES (4,1,'testing2');
INSERT INTO task VALUES (5,2,'mytask1');
INSERT INTO task VALUES (6,2,'mytask2');
Please see the fiddle
EDIT:
Want to pivot table to retrieve the 3 smallests tasks Ids in columns for each project
This code does the trick. You need to use RANK to select the 3 smallests IDs and then pivot table:
WITH tasks AS (
select
A.prj_code,
prj_name,
RANK() OVER (PARTITION BY A.PRJ_CODE ORDER BY ID ) RANK,
ID,
TASK_NAME
from task A
inner join project B
on A.PRJ_CODE = B.PRJ_CODE
order by A.prj_code)
select PRJ_CODE, prj_name, sum(T1) T1, sum(T2) T2, sum(T3) T3
from tasks A
PIVOT (max(ID) for RANK IN ('1' as T1 ,'2' as T2 ,'3' as T3 )) B
GROUP BY PRJ_CODE, prj_name
OR
use the code bellow to have the tasks names
select PRJ_CODE, prj_name, max(T1) T1, max(T2) T2, max(T3) T3
from tasks A
PIVOT (max(TASK_NAME) for RANK IN ('1' as T1 ,'2' as T2 ,'3' as T3 )) B
GROUP BY PRJ_CODE, prj_name
SQLFiddle
This is a hideous solution, but I think it illustrates a way to accomplish this. I don't assume any pivoting capabilies, not knowing what version of Oracle you're running.
with tasks as (
select
a.prj_code, a.prj_name,
b.id, b.task_name,
row_number() over (partition by a.prj_code order by b.id) as rn
from
project a
join task b on
a.prj_code = b.prj_code
), pivotdata as (
select
prj_code, prj_name,
case when rn = 1 then task_name end as task_1,
case when rn = 2 then task_name end as task_2,
case when rn = 3 then task_name end as task_3
from tasks
)
select
prj_code, prj_name,
max (task_1) as task_1, max (task_2) as task_2,
max (task_3) as task_3
from pivotdata
group by
prj_code, prj_name
Output:
Prj Code Prj Name Task 1 Task 2 Task 3
1 myproj1 testing prod prod2
2 myproj2 mytask1 mytask2

How to get Distinct Records from Temp table to Sql one database table

i have temp table named "#Test" which have columns "T1", "T2", "T3" with data.
I have database table named "TestTbl" which have same columns.
I want to insert data from #Test table to TestTbl with distinct records of T1 column.
Do you have any idea how to insert distinct records in TestTbl table?
You Can Try Like this....
INSERT INTO TestTbl (T1,T2,T3) SELECT T1,T2,T3 from
(
Select Row_Number() over(Partition By T1 order By T1) as row,* from #Test
) a
where a.row=1;
INSERT INTO TestTbl (T1,T2,T3)
SELECT Distinct(T1), T2, T3 FROM #Test
EDIT After further explanation
INSERT INTO TestTbl
( T1 ,
T2 ,
T3
)
SELECT T1 ,
T2 ,
T3
FROM ( SELECT T1 ,
T2 ,
T3 ,
Row_Number() OVER ( PARTITION BY T1 ORDER BY T1) AS record
-- you need to select the relevant clause here for the order
-- do you want first or latest record?
FROM #Test
) tmp
WHERE tmp.record = 1 ;
Get distinct records
SELECT DISTINCT column_name,column_name
FROM table_name
Insert records
INSERT INTO table_name (column1,column2,column3,...)
VALUES (value1,value2,value3,...)

Using count as a condition in Oracle

I have two queries, q1 and q2. I want to return columns from q1 when q2 has no rows. Example:
select a, b, c from t1 where
count(select d, e, f from t2 where ...) == 0
and ...
Normally, I would just use a JOIN, but in this case, I have no related keys.
What is the best way to do this in Oracle?
I assume that those queries are entirely independant, like so:
create table table_q1 (
id number,
txt varchar2(10)
);
insert into table_q1 values ( 1, 'This');
insert into table_q1 values ( 2, 'data');
insert into table_q1 values ( 3, 'only');
insert into table_q1 values ( 4, 'selected');
insert into table_q1 values ( 5, 'if');
insert into table_q1 values ( 6, 'other');
insert into table_q1 values ( 7, 'query''s');
insert into table_q1 values ( 8, 'count');
insert into table_q1 values ( 9, 'greater');
insert into table_q1 values (10, 'zero');
create table table_q2 (
id number
);
insert into table_q2 values (1);
insert into table_q2 values (2);
insert into table_q2 values (3);
insert into table_q2 values (4);
You can now have a with-query q2 that selects the count of table_q2 and cross join it to table_q1 with the condition q2.cnt = 0 so that q1 only selects records if q2's count is != 0.
The following select statement returns no records:
with q2 as (select count(*) cnt from table_q2 where id > 2)
select q1.* from table_q1 q1, q2
where q2.cnt = 0
order by q1.id;
But this one does:
with q2 as (select count(*) cnt from table_q2 where id > 1000)
select q1.* from table_q1 q1, q2
where q2.cnt = 0
order by q1.id;
select <columns>
from table
where not exists (select <columns>
from table2
where ....)
should work. If there were some relationship between the inner query and the outer query, you would just add an additional predicate to the NOT EXISTS subquery that expressed that relationship (i.e. table.column_name = table2.column_name). But there is no need to make the subquery correlated.
You also don't need to specify the column names in the SELECT list of the subquery. It would only matter if adding the columns changed the query plan (say, by forcing the optimizer to query the table rather than using a covering index). You'll get the same result if you use something like this and it may be slightly faster.
select <columns>
from table
where not exists (select 1
from table2
where ....)
maybe you can try something like this
SELECT *
FROM TABLE1
WHERE DECODE((SELECT COUNT(T2.SOME_COLUMN)
FROM TABLE2 T2
WHERE T2.CONDITION_COLUMN = 'SOM_VAL'),
0,
'FALSE',
'TRUE') = 'TRUE'
Here the nested query within the DECODE will count the number of a certain column. In case it is ZERO, it will return false and query will return nothing or in case it returns anything more than ZERO, it will return TRUE and query will return values.
Hope it helps
Write a query that includes COUNT and GROUP BY without trying to filter out COUNT(x) = 0. You should see the zeros in your result set. That you want to eliminate.
Add a HAVING clause: HAVING COUNT(x) <> 0
If the tables are in fact joined on some field (let's name it id for both), it worth construct a query like
SELECT ... FROM table WHERE id NOT IN (SELECT id FROM table2 WHERE ...)
Check this query
Tested
select * from table1 where (SELECT count() FROM table2)=0