Impala query SQL execution returns null, which feels very strange
24 14:13:26 205
[ACCESS_TIME,2021-08-24 14:12:13][ACTION,QUERY][BEGIN_TIME,1629785533][APPLICATION_ID,APP202009000005][GROUP_NAME,saas-merge][DATABASE_NAME,][INTERFACE_NAME,revFullLifeToB][SOURCE,100051#9.138.225.17][PROCESS_RESULT,ERROR][TOTAL_DURATION,67024][BEGIN,0][AUTH,0][INIT_SQL,0][QUERY_TOTAL_RECORDS,564][PREPARE_QUERY,6433][QUERY,8][SERIALIZE,60017][RESP_SIZE,0][REQUEST,RequestDTO(applicationId:APP202009000005, group:null, interfaceName:revFullLifeToB, connectionId:null, queryDSL:null, query:%5B%5D, displayNames:null, page:PageDTO(pageSize:30000, pageIndex:50), sortings:%5BSortingDTO(sortingType:ASC, sortingColName:id)%5D, requestSettings:RequestSettingsDTO(maxReturnSize:-1, gzip:false, timeout:3600, i18n:null, responseFormat:JSON, returnTotalCount:true, splitSheets:false, splitFiles:false, perSize:-1, features:%7Bskip.permission.user.name=SYSTEM%7D), tenantId:null, dimensionCode:null, appCode:null)][QUERY,SELECT t.* FROM ( SELECT PERIOD_ID, AFT_DIV_BG_CN_NAME, AFT_DIV_XBG_CN_NAME, AFT_DIV_DEPT_CN_NAME, AFT_DIV_COSTCENTER_CN_NAME, DATA_SOURCE_NAME, SETT_CHANNEL_CN_NAME, SETT_SUBCHANNEL_NAME, OU_CN_NAME, CUST_ACCT_ID, CUST_ACCT_NAME, BIZ_CUSTOMER_NAME, SETT_PRODUCT_CN_NAME, AFT_DIV_PRODUCT_CN_NAME, BGT_ACCT_CN_NAME, PROD_TYPE_NAME, DIV_NET_INCOMDE, DIV_NET_INCOMDE_AFTER_TAX, CURRENCY_CODE, INCOME_TYPE, REMARK, TOB_FLAG, aft_div_bg_code, aft_div_xbg_code, aft_div_dept_code, aft_div_costcenter_code, ou_code, biz_customer_id, sett_product_code, aft_div_product_code, bgt_account_code, idFROM data.getAccountData BY id ASC limit 1500000 ) t order by id ASC limit 30000 OFFSET 1470000][QUERY_ARGS,][PROCESS_MSG,JDBC error:com.tencent.oa.fm.jarvis.dataservice.provider.jdbc.JDBCDataProviderException: JDBC error:com.tencent.oa.fm.jarvis.dataservice.provider.jdbc.JDBCDataResultHandlerException: com.fasterxml.jackson.databind.JsonMappingException: Unexpected IOException (of type java.io.IOException): java.sql.SQLException: %5BCloudera%5D%5BImpalaJDBCDriver%5D(500312) Error in fetching data rows: null;][RETURN_ROWS,0]....
2021-08-24 14:13:21 219
Caused by: com.tencent.oa.fm.jarvis.dataservice.provider.jdbc.JDBCDataResultHandlerException: com.fasterxml.jackson.databind.JsonMappingException: Unexpected IOException (of type java.io.IOException): java.sql.SQLException: [Cloudera][ImpalaJDBCDriver](500312) Error in fetching data rows: null;....
2021-08-24 14:13:21 219
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2021-08-24 14:13:21 219
at com.tencent.oa.fm.jarvis.dataservice.provider.jdbc.JDBCDataResult.(JDBCDataResult.java:43)
2021-08-24 14:13:21 219
... 22 common frames omitted
Your SQL looks incorrect. In screenshot i showed where is the issue - the tablename should be correct and by clause is causing this issue.
Here is the corrected SQL.
SELECT t.*
FROM
(SELECT PERIOD_ID,
AFT_DIV_BG_CN_NAME,
AFT_DIV_XBG_CN_NAME,
AFT_DIV_DEPT_CN_NAME,
AFT_DIV_COSTCENTER_CN_NAME,
DATA_SOURCE_NAME,
SETT_CHANNEL_CN_NAME,
SETT_SUBCHANNEL_NAME,
OU_CN_NAME,
CUST_ACCT_ID,
CUST_ACCT_NAME,
BIZ_CUSTOMER_NAME,
SETT_PRODUCT_CN_NAME,
AFT_DIV_PRODUCT_CN_NAME,
BGT_ACCT_CN_NAME,
PROD_TYPE_NAME,
DIV_NET_INCOMDE,
DIV_NET_INCOMDE_AFTER_TAX,
CURRENCY_CODE,
INCOME_TYPE,
REMARK,
TOB_FLAG,
aft_div_bg_code,
aft_div_xbg_code,
aft_div_dept_code,
aft_div_costcenter_code,
ou_code,
biz_customer_id,
sett_product_code,
aft_div_product_code,
bgt_account_code,
id
FROM data.getAccountData LIMIT 1500000) t
ORDER BY id ASC
LIMIT 30000
OFFSET 1470000
Related
I am trying to calculate the percentage of faulty transaction statuses per IP address in Clickhouse.
SELECT
c.source_ip,
COUNT(c.source_ip) AS total,
(COUNT(c.source_ip) / t.total_calls) * 100 AS percent_faulty
FROM sip_transaction_call AS c
CROSS JOIN
(
SELECT count(*) AS total_calls
FROM sip_transaction_call
) AS t
WHERE (status = 8 OR status = 9 or status = 13)
GROUP BY c.source_ip
Unfortunately Clickhouse rejects this with:
"Received exception from server (version 20.8.3):
Code: 47. DB::Exception: Received from 127.0.0.1:9000. DB::Exception: Unknown identifier: total_calls there are columns: source_ip, COUNT(source_ip)."
I tried various workarounds for the "invisible" alias, but failed. Any help would be greatly appreciated.
SELECT
source_ip,
countIf(status = 8 OR status = 9 or status = 13) AS failed,
failed / count() * 100 AS percent_faulty
FROM sip_transaction_call
GROUP BY source_ip
If you have a GROUP BY clause, you can only use columns you are grouping by (ie. c.source_ip) - for others you need an aggregate function.
Clickhouse is not too helpful here - for almost any other engine you would get a more meaningful error. See https://learnsql.com/blog/not-a-group-by-expression-error/.
Anyway, change grouping to GROUP BY c.source_ip, t.total_calls to fix it.
I try to make a fulltext search with postgresql.
Query:
SELECT * ,
count(*) OVER(order by id DESC) AS full_count,
ts_headline(
'english',
"content",
websearch_to_tsquery('test1 test2 test3 test4'),
'StartSel=<mark>, StopSel=</mark>, HighlightAll=true, MaxWords=35, MinWords=1, ShortWord=3'
) AS content_highlighted,
ts_rank_cd(to_tsvector('english', content), websearch_to_tsquery('test1 test2 test3 test4')) AS rank
FROM book_c_bookitem bi
WHERE to_tsvector('english', content) ## websearch_to_tsquery('test1 test2 test3 test4')
order by id DESC, is_latest DESC , rank DESC
limit 30
offset 0
When I search a word with websearch_to_tsquery, everything is fine. But when I try to search two or more words, I got error:
canceling statement due to statement timeout
How I can solve this problem?
Update
Thanks to #LaurenzAlbe. I adding EXPLAIN (ANALYZE, BUFFERS) to query, I got this error:
Notice: word is too long to be indexed - set successfully executed
How I can fix this error?
I have the following query in Postgres 11
SELECT DISTINCT ON (account.name) *
FROM sys_accounts AS account
INNER JOIN sys_activities AS activity ON account.guid=activity.account_guid
WHERE activity.datefinish IS NOT NULL
AND (extract(day from NOW() - activity.datefinish)) > 60
AND dc_status_guid='df11a68c-20d3-46e2-840f-2e4548aacf1b'
ORDER BY account.name,activity.datefinish DESC
How can I run that through Hibernate (5.4)
I need get all accounts where last task closed > 60 days ago
Java structure like
Account (oneToMany){
Activity1 :finishdate |stage ....
Activity2 :finishdate |stage ....
......
}
I don't understand how to write this request with hibernate syntax HQL or CRITERIA. With HQL i get error:Caused by: org.hibernate.hql.internal.ast.QuerySyntaxException: unexpected token: ON near line 1, column 17 [SELECT DISTINCT ON (account.name)* FROM sys_accounts AS account INNER JOIN sys_activities AS activity ON account.guid=activity.account_guid WHERE activity.datefinish IS NOT NULL AND dc_status_guid='df11a68c-20d3-46e2-840f-2e4548aacf1b' ORDER BY account.name,activity.datefinish DESC]
I am trying to run one SQL query to find out the count per error code from the database. I have two table
sw_sms_events where the transaction id and sms that was sent is stored.
sw_events where the transaction id and error reason in case it is failed then it is stored otherwise the reason is always "Successfully Sent TariffText".
Total error count :- select count(*) from sw_sms_events where sms_text like '%Welkom in het buitenland%'
Total error count per error reason :-
select distinct count(*) over (partition by b.reason) , b.reason
from sw_sms_events a, sw_events b
where a.transaction_id= b.transaction_id
and a.sms_text like '%Welkom in het buitenland%'
and b.reason !='Successfully Sent TariffText'
order by (count(*) over (partition by b.reason)) desc
Normally these queries gives same result i.e. sum of individual error count = total number of errors.But in worst case scenarios where the same transaction is retried multiple times the results are not same .i.e. we have multiple rows in table with same transaction id.
below is one of the result in case of worst case :
Name 24-07-2015
Total Number of SMSWelcome Sent 156788
Total Number of Error SMSWelcome 1738
Total Number of SMSWelcome Sent with null Tariffs 286
Error Reason Error Count
Unknown error received :BEA-380000 , ErrorMessage : BSL-99999 1829
Backend system not available , ErrorMessage : BSL-50002 641
Remote Error 527
NativeQuery.executeQuery failed , ErrorMessage : BSL-11009 41
This service is available only for active products , ErrorMessage : BSL-15024 30
Unknown error received :BEA-382556 , ErrorMessage : BSL-99999 18
Customer information: Not retrieved. This action cannot continue without customer information. Please try later or contact your system administrator. , ErrorMessage : BSL-10004 13
OMS login failure: Problem in OMS UAMS login - Nested Exception/Error: java.net.ConnectException: Tried all: '1' addresses, but could not connect over HTTP to server: '195.233.102.177', port: '40123' , 12
t3://195.233.102.171:30101: Bootstrap to: 195.233.102.171/195.233.102.171:30101' over: 't3' got an error or timed out , ErrorMessage : BSL-11000 5
getTariffsAndAddOns, status: Failure , ErrorCode : An internal error occured , ErrorMessage : BSL-14005 3
Authorization failed of dealer market restrictions , ErrorMessage : BSL-50005 2
com.amdocs.cih.exception.InvalidUsageException: The input parameter AssignedProductRef is invalid. , ErrorMessage : BSL-10004 1
My question is how i can modify the current sql in such a way that the total count of error should always be equal to sum of individual error count when we have wrost cases where same transaction is multiple times in a table
I don't really understand why you are using an analytical query. Isn't a simpler group by sufficient ?
select count(*), b.reason
from sw_sms_events a, sw_events b
where a.transaction_id= b.transaction_id
and a.sms_text like '%Welkom in het buitenland%'
and b.reason !='Successfully Sent TariffText'
group by b.reason
order by count(*) desc
When you say we have multiple rows in table with same transaction id, do you mean in sw_events table only or in both sw_sms_events and sw_events tables ?
If so, events are counted multiple times because you are doing a cartesian product on all raws with the same transaction_id. You should use a stricter join clause.
You could also do something (quite ugly) like :
select count(distinct b.ROWID), b.reason
from sw_sms_events a, sw_events b
where a.transaction_id= b.transaction_id
and a.sms_text like '%Welkom in het buitenland%'
and b.reason !='Successfully Sent TariffText'
group by b.reason
order by count(distinct b.ROWID) desc
to ensure that each event is only counted once.
select distinct count(distinct b.ROWID) over (partition by b.reason) , b.reason
from sw_sms_events a, sw_events b
where a.transaction_id= b.transaction_id
and a.sms_text like '%Welkom in het buitenland%'
and b.reason !='Successfully Sent TariffText'
order by (count(distinct b.ROWID) over (partition by b.reason)) desc
SELECT
getOrgName(BC.ManageOrgID),
COUNT(CASE WHEN (EXISTS (SELECT FO.OBJECTNO FROM FLOW_OBJECT FO WHERE FO.ObjectNo=CR.SerialNo) AND NVL(CR.FinallyResult,'') IN ('01','02','03','04','05')) THEN BC.ManageOrgID ELSE NULL END)
FROM
BUSINESS_CONTRACT BC,
CLASSIFY_RECORD CR
WHERE
CR.ObjectType='BusinessContract'
AND CR.ObjectNo=BC.SerialNo
GROUP BY BC.ManageOrgID, CR.SerialNo, CR.FinallyResult
The error message I receive is:
11:01:32 [SELECT - 0 row(s), 0.000 secs] [Error Code: -112, SQL State: 42607] DB2 SQL Error: SQLCODE=-112, SQLSTATE=42607, SQLERRMC=SYSIBM.COUNT, DRIVER=3.57.82
... 1 statement(s) executed, 0 row(s) affected, exec/fetch time: 0.000/0.000 sec [0 successful, 0 warnings, 1 errors]
"The operand of the column function name (in your case, count) includes a column function, a scalar fullselect, or a subquery." DB2 doesn't allow this. See the documentation on SQL112 for more.
I'm not really sure how to fix your query but perhaps you can try the HAVING clause after GROUP BY.
Here is one way to rework the query:
SELECT
getOrgName( BC.ManageOrgID ),
COUNT( FO.ObjectNo ) AS objectcount
FROM BUSINESS_CONTRACT BC
INNER JOIN CLASSIFY_RECORD CR
ON CR.ObjectNo = BC.SerialNo
AND CR.ObjectType = 'BusinessContract'
AND CR.FinallyResult IN ( '01','02','03','04','05' )
INNER JOIN FLOW_OBJECT FO
ON FO.ObjectNo = CR.SerialNo
GROUP BY BC.ManageOrgID
;