I'm getting the exception for SQL query on the hive
create table temp as
select t.type
from temp1 LATERAL VIEW posexplode(c1.array_of_struct_field1) dummy as alias1, t
union all
select t.type
from temp1 LATERAL VIEW posexplode(c1.array_of_struct_field2) dummy as alias1, t;
Below I'm adding the exception for the query.
Error: Error while compiling statement: FAILED: SemanticException Can
not find database1.temp in genColumnStatsTask
(state=42000,code=40000)
Use the union all query as below way.
create table temp as
select t.type from
(select t.type
from temp1 LATERAL VIEW posexplode(c1.array_of_struct_field1) dummy as alias1, t
union all
select t.type
from temp1 LATERAL VIEW posexplode(c1.array_of_struct_field2) dummy as alias1, t ) a;
try this one in your code
set hive.stats.column.autogather=false;
Related
I have a query like this:
SELECT MONTH_ID, 'Total' AS cola, colb
FROM
(
SELECT A.*, ROW_NUMBER()OVER(PARTITION BY MONTH_ID,col3 ORDER BY col4 DESC) AS ROWN
FROM
(
SELECT A.*, B.col3
FROM table1 A
LEFT JOIN table2 B
ON A.col1 = B.col1
) A
)
WHERE ROWN=1
GROUP BY MONTH_ID
If I create a intermediate table with the subqueries this query can work. But when I run entire thing Impala will raise: "AnalysisException: Syntax error in line 12:undefined: WHERE ROWN = 1 ^ Encountered: WHERE Expected: AS, DEFAULT, IDENTIFIER CAUSED BY: Exception: Syntax error"
I tried run this in Hive, different error shows: "Error while compiling statement: FAILED: ParseException line 20:4 cannot recognize input near 'WHERE' 'ROWN' '=' in subquery source"
Then I tried same query in oracle, it works...
Could anyone explain why this is happening and how to solve this?
Thank you for your help ;)
Subquery should have some alias like this (see comment in the code):
SELECT MONTH_ID, 'Total' AS cola, colb
FROM
(
SELECT A.*, ROW_NUMBER()OVER(PARTITION BY MONTH_ID,col3 ORDER BY col4 DESC) AS ROWN
FROM
(
SELECT A.*, B.col3
FROM table1 A
LEFT JOIN table2 B
ON A.col1 = B.col1
) A
) B ----------------------------Alias is a must --------------
WHERE ROWN=1
GROUP BY MONTH_ID, colb -----All columns which are not aggregated and not constants should be in GROUP BY
I have a query like
SELECT name, salary/ (SELECT max(money) from table_sal) FROM table_a;
I get an error saying
Unsupported SubQuery Expression Invalid subquery. Subquery in SELECT could only be top-level expression
Is there a way to resolve this?
Does this work with a CROSS JOIN?
SELECT name, salary / s.max_money
FROM table_a CROSS JOIN
(SELECT max(money) as max_money from table_sal) s
You can also do this as below, please let me know if it works for you.
Select t1.name
, t1.salary/T2.max_money
from
(SELECT name
, salary, 1 as dummy
from table_a ) t1
Join
(SELECT max(money) as max_money
, 1 as dummy
from table_sal) t2
on t1.dummy = t2.dummy ;
I am trying to make a query like this:
select * from tbl
LATERAL VIEW OUTER explode(column) temp_tbl as the_col
WHERE (the_col IN (select column from tbl2))
and it gives this error:
Unsupported SubQuery Expression: Correlating expression cannot contain
unqualified column references
I looked at this answer and changed the query to:
select * from tbl
LATERAL VIEW OUTER explode(column) temp_tbl as the_col
WHERE (tbl.the_col IN (select column from tbl2))
and now I get this error:
FAILED: SemanticException Line XX:XX Invalid column reference
'the_col' in definition of SubQuery sq_1
What's going on here and how to fix this?
try this,
SELECT * FROM (
SELECT * FROM tbl
LATERAL VIEW OUTER explode(column) temp_tbl as the_col ) a
WHERE a.the_col IN (select column from tbl2);
SELECT
xfqti_virtuemart_products_pt_pt.virtuemart_product_id,
xfqti_virtuemart_product_medias.virtuemart_media_id
INTO #tempTable
FROM xfqti_virtuemart_products_pt_pt
Gives syntax error, I'm about to pull my hair off
Being Virtuemart, I'm guessing this is a MySQL database. If so, the correct syntax for creating a temp table is:
CREATE TEMPORARY TABLE IF NOT EXISTS tempTableName AS
(
SELECT field1, field2
FROM yourtable;
)
That being said, your SELECT statement has two fields from two different tables, but only one of those tables is mentioned in the FROM clause of your statement. They should really both be in there and JOINed. Something like:
CREATE TEMPORARY TABLE IF NOT EXISTS tempTableName AS
(
SELECT
t1.virtuemart_product_id,
t2.virtuemart_media_id
FROM
xfqti_virtuemart_products_pt_pt as t1
INNER JOIN xfqti_virtuemart_product_medias as t2 ON
t1.product_id = t2.product_id
)
Or something.. I can't see your tables and it's been years since I used Virtuemart, so it's just a guess at the table relationship.
Insert Into and Selecthave these sintax
Insert into your_Table (col1,col2)
SELECT
xfqti_virtuemart_products_pt_pt.virtuemart_product_id,
xfqti_virtuemart_product_medias.virtuemart_media_id
FROM xfqti_virtuemart_products_pt_pt
for create table
Create your_Table as
SELECT
xfqti_virtuemart_products_pt_pt.virtuemart_product_id,
xfqti_virtuemart_product_medias.virtuemart_media_id
FROM xfqti_virtuemart_products_pt_pt
Failed to find the answer in the specs.
So, I wonder: Can I do something like that in hive?
insert into table my_table
with a as
(
select *
from ...
where ...
),
b as
(
select *
from ...
where ...
)
select
a.a,
a.b,
a.c,
b.a,
b.b,
b.c
from a join b on (a.a=b.a);
With is available in Hive as of version 0.13.0. Usage documented here.
Hadoop Hive WITH Clause Syntax and Examples
With the Help of Hive WITH clause you can reuse piece of query result in same query construct. You can also improve the Hadoop Hive query using WITH clause. You can simplify the query by moving complex, complicated repetitive code to the WITH clause and refer the logical table created in your SELECT statements.
Hive WITH clause example with the SELECT statement
WITH t1 as (SELECT 1),
t2 as (SELECT 2),
t3 as (SELECT 3)
SELECT * from t1
UNION ALL
SELECT * from t2
UNION ALL
SELECT * from t3;
Hive WITH Clause in INSERT Statements
You can use the WITH clause while inserting data to table. For example:
WITH t11 as (SELECT 10),
t12 as (SELECT 20),
t13 as (SELECT 3)
INSERT INTO t1
SELECT * from t11
UNION ALL
SELECT * from t12
UNION ALL
SELECT * from t13;
I guess you could always use subqueries:
insert into table my_table
select
a.a,
a.b,
a.c,
b.a,
b.b,
b.c
from
(
select *
from ...
where ...
) a
join
(
select *
from ...
where ...
) b
on a.a = b.a;