Two functions in one query give error in Hive - hive

I have a simple query:
select
substr(mos_inbound,1,6),
replace(calling_party_number_received,'F','') calling_party_number_received
from cdr;
that gives this error:
[Cloudera][HiveJDBCDriver](500051) ERROR processing query/statement. Error Code: 40000, SQL state: TStatus(statusCode:ERROR_STATUS, infoMessages:[*org.apache.hive.service.cli.HiveSQLException:Error while compiling statement: FAILED: SemanticException Line 0:-1 Invalid function 'replace':28:27, org.apache.hive.service.cli.operation.Operation:toSQLException:Operation.java:315, org.apache.hive.service.cli.operation.SQLOperation:prepare:SQLOperation.java:112, org.apache.hive.service.cli.operation.SQLOperation:runInternal:SQLOperation.java:181, org.apache.hive.service.cli.operation.Operation:run:Operation.java:257, org.apache.hive.service.cli.session.HiveSessionImpl:executeStatementInternal:HiveSessionImpl.java:419, org.apache.hive.service.cli.session.HiveSessionImpl:executeStatementAsync:HiveSessionImpl.java:406, sun.reflect.GeneratedMethodAccessor36:invoke::-1, sun.reflect.DelegatingMethodAccessorImpl:invoke:DelegatingMethodAccessorImpl.java:43, java.lang.reflect.Method:invoke:Method.java:498, org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:78, org.apache.hive.service.cli.session.HiveSessionProxy:access$000:HiveSessionProxy.java:36, org.apache.hive.service.cli.session.HiveSessionProxy$1:run:HiveSessionProxy.java:63, java.security.AccessController:doPrivileged:AccessController.java:-2, javax.security.auth.Subject:doAs:Subject.java:422, org.apache.hadoop.security.UserGroupInformation:doAs:UserGroupInformation.java:1709, org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:59, com.sun.proxy.$Proxy36:executeStatementAsync::-1, org.apache.hive.service.cli.CLIService:executeStatementAsync:CLIService.java:276, org.apache.hive.service.cli.thrift.ThriftCLIService:ExecuteStatement:ThriftCLIService.java:486, org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement:getResult:TCLIService.java:1317, org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement:getResult:TCLIService.java:1302, org.apache.thrift.ProcessFunction:process:ProcessFunction.java:39, org.apache.thrift.TBaseProcessor:process:TBaseProcessor.java:39, org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor:process:HadoopThriftAuthBridge.java:562, org.apache.thrift.server.TThreadPoolServer$WorkerProcess:run:TThreadPoolServer.java:285, java.util.concurrent.ThreadPoolExecutor:runWorker:ThreadPoolExecutor.java:1142, java.util.concurrent.ThreadPoolExecutor$Worker:run:ThreadPoolExecutor.java:617, java.lang.Thread:run:Thread.java:745, *org.apache.hadoop.hive.ql.parse.SemanticException:Line 0:-1 Invalid function 'replace':53:26, org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor:getXpathOrFuncExprNodeDesc:TypeCheckProcFactory.java:935, org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor:process:TypeCheckProcFactory.java:1316, org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher:dispatch:DefaultRuleDispatcher.java:90, org.apache.hadoop.hive.ql.lib.DefaultGraphWalker:dispatchAndReturn:DefaultGraphWalker.java:95, org.apache.hadoop.hive.ql.lib.DefaultGraphWalker:dispatch:DefaultGraphWalker.java:79, org.apache.hadoop.hive.ql.lib.DefaultGraphWalker:walk:DefaultGraphWalker.java:133, org.apache.hadoop.hive.ql.lib.DefaultGraphWalker:startWalking:DefaultGraphWalker.java:110, org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory:genExprNode:TypeCheckProcFactory.java:213, org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory:genExprNode:TypeCheckProcFactory.java:157, org.apache.hadoop.hive.ql.parse.SemanticAnalyzer:genAllExprNodeDesc:SemanticAnalyzer.java:10529, org.apache.hadoop.hive.ql.parse.SemanticAnalyzer:genExprNodeDesc:SemanticAnalyzer.java:10485, org.apache.hadoop.hive.ql.parse.SemanticAnalyzer:genSelectPlan:SemanticAnalyzer.java:3822, org.apache.hadoop.hive.ql.parse.SemanticAnalyzer:genSelectPlan:SemanticAnalyzer.java:3601, org.apache.hadoop.hive.ql.parse.SemanticAnalyzer:genPostGroupByBodyPlan:SemanticAnalyzer.java:8965, org.apache.hadoop.hive.ql.parse.SemanticAnalyzer:genBodyPlan:SemanticAnalyzer.java:8920, org.apache.hadoop.hive.ql.parse.SemanticAnalyzer:genPlan:SemanticAnalyzer.java:9765, org.apache.hadoop.hive.ql.parse.SemanticAnalyzer:genPlan:SemanticAnalyzer.java:9658, org.apache.hadoop.hive.ql.parse.SemanticAnalyzer:genOPTree:SemanticAnalyzer.java:10131, org.apache.hadoop.hive.ql.parse.CalcitePlanner:genOPTree:CalcitePlanner.java:329, org.apache.hadoop.hive.ql.parse.SemanticAnalyzer:analyzeInternal:SemanticAnalyzer.java:10142, org.apache.hadoop.hive.ql.parse.CalcitePlanner:analyzeInternal:CalcitePlanner.java:211, org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer:analyze:BaseSemanticAnalyzer.java:227, org.apache.hadoop.hive.ql.Driver:compile:Driver.java:459, org.apache.hadoop.hive.ql.Driver:compile:Driver.java:316, org.apache.hadoop.hive.ql.Driver:compileInternal:Driver.java:1189, org.apache.hadoop.hive.ql.Driver:compileAndRespond:Driver.java:1183, org.apache.hive.service.cli.operation.SQLOperation:prepare:SQLOperation.java:110], sqlState:42000, errorCode:40000, errorMessage:Error while compiling statement: FAILED: SemanticException Line 0:-1 Invalid function 'replace'), Query: select
substr(mos_inbound,1,6),
replace(calling_party_number_received,'F','') calling_party_number_received
from cdr.
with teh most important error:
FAILED: SemanticException Line 0:-1 Invalid function 'replace'), Query: select
substr(mos_inbound,1,6),
replace(calling_party_number_received,'F','') calling_party_number_received
from cdr.
I don't understand why, cause when I query the same functions separately it works:
select
--substr(mos_inbound,1,6),
replace(calling_party_number_received,'F','') calling_party_number_received
from cdr;
INT17867310AAA
INT49241572AAA
INT18053548AAA
INT17146971AAA
INT18169158AAA
select
substr(mos_inbound,1,6)
--replace(calling_party_number_received,'F','') calling_party_number_received
from cdr;
00.00|
04.30|
00.00|
Hive version;
> hive --version
WARNING: Use "yarn jar" to launch YARN applications.
Hive 1.2.1000.2.4.2.0-258
Subversion git://c66-slave-38a0b1a6-7/grid/0/jenkins/workspace/HDP-build-centos6/bigtop/build/hive/rpm/BUILD/hive-1.2.1000.2.4.2.0 -r 240760457150036e13035cbb82bcda0c65362f3a
Compiled by jenkins on Mon Apr 25 06:47:41 UTC 2016
From source with checksum 195fc2d28ec575146e342c1bff540639
What can be wrong?
There is no problem, when I use many replace functions in my query...
Regards
Pawel

Related

SQL0007N invalid character with sqlalchemy in jupyter-notebook

%%sql
Select Per_capita_Income, Community_Area_Name
from CENSUS_DATA
Where Per_Capita_Income < 11000;
ibm_db_sa://ztl80401:***#125f9f61-9715-46f9-9399-c8177b21803b.c1ogj3sd0tgtu0lqde00.databases.appdomain.cloud:30426/BLUDB
(ibm_db_dbi.ProgrammingError) ibm_db_dbi::ProgrammingError: SQLNumResultCols failed: [IBM][CLI Driver][DB2/LINUXX8664] SQL0007N The statement was not processed because a character that is not supported in SQL statements was included in the SQL statement. Invalid character: "\". Text preceding the invalid character: "ame from CENSUS_DATA". SQLSTATE=42601 SQLCODE=-7
[SQL: Select Per_capita_Income, Community_Area_Name
from CENSUS_DATA
Where Per_Capita_Income < 11000;]
(Background on this error at: http://sqlalche.me/e/13/f405)
Hello,
I can't get this query to pull the Country Area names where the Per capita Income is less than 11000. what is wrong with my query? I am using Jupyter Notebooks and my database is DB2. Please help. thanks!

SemanticException [Error 10128]: Line 1:33 Not yet supported place for UDAF 'sum'

My command was this in hive shell:
select name_of_state, max(total_confirmed_cases/sum(total_confirmed_cases)) from total_india_case;
I got this error:
FAILED: SemanticException [Error 10128]: Line 1:33 Not yet supported place for UDAF 'sum'
Plzzz help

You are getting error because you are not using group by clause. You can't use aggregate functions like this.
I am not sure what are you trying to achieve, but assuming that you want to fetch the state name having maximum ration of total_confirmed_cases to entire India.
With total_sum as (
Select sum(total_confirmed_cases) total from total_india_case)
Select name_of_state, max(total_confirmed_cases)/total from total_india_case
left outer join total_sum on 1=1
group by name_of_state, total limit;

Nested select statements in impala sql

I have the following SQL query in impala
SELECT currentdate,close
FROM ( SELECT * FROM spyprice)
Where currentdate between '2015-01-16' and '2016-06-17';
And it is giving me the error:
Starting Impala Shell without Kerberos authentication
ERROR: AnalysisException: Syntax error in line 15:
WHERE currentdate BETWEEN '2015-01-16' and '2016-06-17'
^
Encountered: WHERE
Expected: AS, DEFAULT, IDENTIFIER
CAUSED BY: Exception: Syntax error
Anyone knows what's going on?
Thanks in advance!

#James Xiang The right syntax of the query statement is :
SELECT a.currentdate, a.close FROM
(
SELECT * FROM spyprice
) a
Where a.currentdate between '2015-01-16' and '2016-06-17';

Impala query: Exception: Syntax error caused by cast function

I have the following Impala query
select session_id, max(cast(milli_ts) as integer), min(cast(milli_ts) as integer)from my_table group by session_id
But got the following errors:
HiveServer2Error: AnalysisException: Syntax error in line 10:
...sion_id, max(cast(milli_ts) as integer), min(cast(mill...
^
Encountered: )
Expected: AND, AS, BETWEEN, DIV, ILIKE, IN, IREGEXP, IS, LIKE, NOT, OR, REGEXP, RLIKE
CAUSED BY: Exception: Syntax error
Any idea what I missed? Thanks!

The correct syntax for CAST is as following:
cast(milli_ts as integer)

Hive will not recognize column name in the 2nd condition in the query

I ran the following and query and got the following error. Please see that error message mentions the column name (platform) that it thinks is not there. weird.
hive -S -e 'select * from devices.device_app_action where ds= '20160511'
AND platform= 'ios' limit 3;'
FAILED: SemanticException [Error 10004]: Line 1:73 Invalid table alias or column reference 'ios': (possible column names are: duid, id, dt, app, platform, app_level, tier1, tier2, tier3, tier4, tier5, tier6, first_geo, first_v, first_lang, total_events, min_ats, max_ats, ds)
Its telling me the column platform does not exist and its there in the list

could be you have to enclose the query in proper quotes
'select * from devices.device_app_action where ds= "20160511" AND platform= "ios" limit 3;'