I need to pass a parameter into an HQL at execution time. For this we have updated the .param file with the required value. Now I am facing issue in using it.
SELECT * FROM temp_table A
WHERE PRODT_CTGRY_CD = 'CAR' AND
(
(DATE(A.CUST_BRTH_DT) BETWEEN concat(${hiveconf:PRODYEAR} - 13 ,"-02","-28") and concat(${hiveconf:PRODYEAR} - 7 ,"-03","-01")
AND DATE(A.TERM_DT) >= concat(${hiveconf:PRODYEAR} - 1 ,"-03","-31")
)
In the above query if I don't include the condition of term_dt the query is running fine. However when I include that I am getting an error as below:
FAILED: ClassCastException org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableIntObjectInspector cannot be cast to org.apache.hadoop.hive.serde2.objectinspector.primitive.BooleanObjectInspector (state=42000,code=40000)
Can someone let me know what needs to be done here?
Related
I am using dbt-trino and for some reason, it doesn't understand the MySQL query that works fine by executing it directly on MySQL. In this query, I want to select and group records that have been created during the previous month.
The Query:
SELECT order_location, COUNT(*) as order_count
FROM {{ ref('x_stg_order_fields') }}
WHERE
created_at >= DATE_FORMAT( CURRENT_DATE - INTERVAL 1 MONTH, '%Y/%m/01' )
AND
created_at < DATE_FORMAT( CURRENT_DATE, '%Y/%m/01' )
GROUP BY order_location
While this query works fast and successfully directly on MySQL, it returns this error when executing with dbt run:
TrinoUserError(type=USER_ERROR, name=SYNTAX_ERROR, message="line 7:53: mismatched input 'COUNT'. Expecting: '*', <expression>")
Does this mean that dbt-trino doesn't support all MySQL functions?
That error is coming from your database, not from dbt itself. dbt does not parse your SQL commands, it just passes them through to your connected database.
My guess is that {{ ref('x_stg_order_fields' }} may be referring to an ephemeral model that contains a syntax error, or possibly a field named count that isn't quoted?
You can confirm or disprove that by looking at the SQL that dbt tried to run in your database, by inspecting the target directory in your project. Specifically, target/run/path/to/your_model.sql will show you the actual command being executed. You should be able to check line 7, col 53 in that file, and you will see the code that trino is erroring about.
Hey everyone im relativly new to Apache Drill and im having troubles converting my Oracle specific sql scripts (pl/sql) to Drill based querys.
For example i have a Scripts who checks for processed data in the last X Days.
In this script im using the the sysdate function.
Here is my old script:
SELECT i.id,i.status,status_text,i.kunnr,i.bukrs,i.belnr,i.gjahr,event,i.sndprn,i.createdate,executedate,tstamp,v.typ_text,i.docnum,i.description, i.*
FROM in_job i JOIN vstatus_injob v ON i.id= v.id
WHERE 1=1
AND i.createdate > sysdate - 30.5
order by i.createdate desc;
When i looked up in terms of drill specific Datetime Diff functions i found "TIMESTAMPDIFF".
So here is my "drillified" script:
SELECT i.id, i.status, status_text, i.kunnr, i.bukrs, i.belnr, i.gjahr, i.event, i.sndprn, i.createdate, i.executedate, i.tstamp,v.typ_text,i.docnum,i.description,i.*
FROM SchemaNAME.IN_JOB i JOIN SchemaNAME.VSTATUS_INJOB v ON i.id=v.id
WHERE TIMESTAMPDIFF(DAY, CURRENT_TIMESTAMP, i.createdate) >=30
And the Error that is returned reads like this:
DATA_READ ERROR: The JDBC storage plugin failed while trying setup the SQL query.
By further inspection i can see the Oracle specific error that reads:
Caused by: java.sql.SQLSyntaxErrorException: ORA-00904: "TIMESTAMPDIFF": invalid ID
So now my question:
I thought apache drill replaces the function "TIMSTAMPDIFF" at runtime. But from what i can see in the logs its more like that Drill Hands over the Function Call "TIMESTAMPDIFF" to the Oracle database.
If thats true, how could i change my script to calculate the time difference (in days) and compare it to an int (ie 30 in the script).
If i use sysdate like above Apache Drill jumps in and says it doesnt know "sysdate".
How would you guyes handle that?
Thanks in advance and so long
:)
I have found a solution...
Just in Case someone (or even me in the future) is having a similar problem.
{
"queryType": "SQL",
"query": "select to_char(SELECT CURRENT_TIMESTAMP - INTERVAL XX MONTH FROM (VALUES(1)),'dd.MM.yy')"
}
With some to_char and the use of the CURRENT_TIMESTAMP - Interval Function Calls i can get everything i needed.
I took the query above packed it into an Grafana Variable, named it "timeStmpDiff" and then queried everything with an json Api Call to my Drill instance.
Basically:
"query" : "SELECT i.id, i.status, status_text, i.kunnr, i.bukrs, i.belnr, i.gjahr, i.event, i.sndprn, i.createdate, i.executedate, i.tstamp,v.typ_text,i.docnum,i.description,i.* FROM ${Schema}.IN_JOB i JOIN ${Schema}.VSTATUS_INJOB v ON i.id=v.id WHERE i.createdate >= '${timeStmpDiff}' order by i.createdate desc"
You can, of course query it in on go with an subselect.
But because i use grafana it made sense to me to bundle that in a Variable.
I'm using h2 to inject the db during run time and testing, everything was working fine until I started trying to make the date field current, not hard-coded.
I thought it was related to the jdbc version I've and updated spring jdbc library to be the latest, but that didn't solve the problem.
This is the code I'm using to inject the data both on run-time and testing :
This code runs during run-time, and it works perfectly before I tried to make the date current.
ResourceDatabasePopulator resourceDatabasePopulator = new ResourceDatabasePopulator();
resourceDatabasePopulator.addScript(new ClassPathResource("schema.sql"));
resourceDatabasePopulator.addScript(new ClassPathResource("data.sql"));
DatabasePopulatorUtils.execute(resourceDatabasePopulator, dataSource); // This is what the DataSourceInitializer does.
For testing purpose, I'm using this code, as I mentioned, it was working perfectly before I tried to make the date current.
DataSource dataSource = new EmbeddedDatabaseBuilder().setType(EmbeddedDatabaseType.H2)
.addScript("classpath:schema.sql")
.addScript("classpath:data.sql")
.build();
data.sql file
INSERT INTO TABLE_X (
dayxx,
xxx
) VALUES
(CONVERT(char(50), DATEADD('DAY', -1,CURRENT_DATE()),126),'xxx')
Exception
Caused by: org.h2.jdbc.JdbcSQLNonTransientException: Unknown data type: "DATEADD"; SQL statement:
INSERT INTO TABLE_X ( dayxx, xxx ) VALUES (CONVERT(char(50), DATEADD('DAY', -1,CURRENT_DATE()),126),'XXX') [50004-200]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:505)
at org.h2.message.DbException.getJdbcSQLException(DbException.java:429)
at org.h2.message.DbException.get(DbException.java:205)
at org.h2.message.DbException.get(DbException.java:181)
at org.h2.command.Parser.parseColumnWithType(Parser.java:5971)
at org.h2.command.Parser.readFunctionParameters(Parser.java:3793)
at org.h2.command.Parser.readFunction(Parser.java:3772)
at org.h2.command.Parser.readTerm(Parser.java:4305)
at org.h2.command.Parser.readFactor(Parser.java:3343)
at org.h2.command.Parser.readSum(Parser.java:3330)
at org.h2.command.Parser.readConcat(Parser.java:3305)
at org.h2.command.Parser.readCondition(Parser.java:3108)
at org.h2.command.Parser.readExpression(Parser.java:3059)
at org.h2.command.Parser.parseValuesForCommand(Parser.java:1877)
at org.h2.command.Parser.parseInsertGivenTable(Parser.java:1817)
at org.h2.command.Parser.parseInsert(Parser.java:1749)
at org.h2.command.Parser.parsePrepared(Parser.java:954)
at org.h2.command.Parser.parse(Parser.java:843)
at org.h2.command.Parser.parse(Parser.java:815)
at org.h2.command.Parser.prepareCommand(Parser.java:738)
at org.h2.engine.Session.prepareLocal(Session.java:657)
at org.h2.engine.Session.prepareCommand(Session.java:595)
at org.h2.jdbc.JdbcConnection.prepareCommand(JdbcConnection.java:1235)
at org.h2.jdbc.JdbcStatement.executeInternal(JdbcStatement.java:212)
at org.h2.jdbc.JdbcStatement.execute(JdbcStatement.java:201)
at org.springframework.jdbc.datasource.init.ScriptUtils.executeSqlScript(ScriptUtils.java:472)
... 53 more
As reported in the docs, Convert function looks like this:
CONVERT ( value , dataType )
Converts a value to another data type.
Example:
CONVERT(NAME, INT)
Since you're passing
CONVERT(char(50), DATEADD('DAY', -1,CURRENT_DATE()),126)
The error states that DATEADD isn't a valid data type, since it's the value, not the type, so try using the right syntax.
I have been using the query below and it was working fine but now it's giving a SQL error.
There were no changes made that could cause this to stop working.
There is no Field 'APPLICATIONID' in the table.
QueueSQL=select distinct (convert(char(5),SkillsetID)+'='+Skillset) from iagentbySkillsetStat where ApplicationID > 10000
QueueMappingSQL=SELECT DISTINCT (convert(char(5),SkillsetID)+'='+Skillset), SkillsetID FROM iagentbySkillsetStat iagentbySkillsetStat WHERE (iagentbySkillsetStat.ApplicationID>10000)
DB- Intersystems Cache
Error details for the log files are below.
14/09/2016 11:20:05 a.m. > ERROR - SQL Error '[Cache ODBC][State : S1000][Native Code 29]
[C:\xlink\xlink Compiled Package\Xlink Application\Isps_Ul.exe]
[SQLCODE: <-29>:<Field not found in the applicable tables>]
[Cache Error: <<SYNTAX>errdone+2^%qaqqt>]
[Details: <Prepare>]
[%msg: < SQL ERROR #29: Field 'APPLICATIONID' not found in the applicable tables^select distinct ( convert ( char ( 5 ) , SkillsetID ) + :%qpar(1) + Skillset ) from iagentbySkillsetStat where ApplicationID >>]'
Question was also asked at InterSystems Developer Community, where I noticed that table names are different. So, the answer is, that table name is suddenly changed somehow.
I'm working on a IBM iseries v6r1m0 system.
I'm trying to execute a very simple query :
select * from XG.ART where DOS = 998 and (DES like 'ALB%' or DESABR like 'ALB%')
The columns are:
DOS -> numeric (3,0)
DES -> Graphic(80) CCSID 1200
DESABR -> Garphic(25) CCSID 1200
I get :
SQL State : 58004
SQL Code : -901
Message : [SQL0901] SQL System error.
Cause . . . . . : An SQL system error has occurred. The current SQL statement cannot be completed successfully. The error will not prevent other SQL statements from being processed. Previous messages may indicate that there is a problem with the SQL statement and SQL did not correctly diagnose the error. The previous message identifier was CPF4204. Internal error type 3107 has occurred. If precompiling, processing will not continue beyond this statement.
Recovery . . . : See the previous messages to determine if there is a problem with the SQL statement. To view the messages, use the DSPJOBLOG command if running interactively, or the WRKJOB command to view the output of a precompile. An application program receiving this return code may attempt further SQL statements. Correct any errors and try the request again.
If I change DES into REF (graphic(25)), it works...
EDIT :
I run some tests this afternoon, and it is very strange :
Just after the creation of the table/indexes, I have no errors.
If I insert some datas : error
If I clear the table : error
If I remove an index (see below) : it works (with or without datas)
!!
The index is :
create index XG.GTFAT_ART_B on XG.ART(
DOS,
DESABR,
ART_ID
)
Edit 2 :
Here is the job log (sorry, it is in French...)
It sais :
Function error X'1720' in machine instruction. Internal snapshot ID 01010054
Foo file created in library QTEMP.
*** stuff with the printer
DBOP *** FAILED open. Exception from call to SLIC$
Internal error in the query processor file
Sql system error
I finally contacted IBM.
It was an old bug from v5.
I have installed the latest PTF, and now, it works.
You need to use the GRAPHIC scalar function to convert your character literals on the LIKE predicate.
CREATE TABLE QTEMP/TEST (F1 GRAPHIC(80))
INSERT INTO QTEMP/TEST (F1) VALUES (GRAPHIC('TEST'))
SELECT * FROM QTEMP/TEST WHERE F1 LIKE GRAPHIC('TE%')
I know this guy got his problem fixed with an update. But here is something that worked for me that might work for the next guy here who has the problem.
My problem query had a lot of common table expressions. Most of them did not create tables with a whole lot of records. So if I figured that the maximum number of records a CTE would make was 1000, I added a "Fetch first 9999 rows only" to it. I knew that the CTE couldn't possibly have more rows than that. I guess the query optimizer had less to think about with that added.
If you have that problem and you don't have the option to upgrade or talk to IBM, I hope this help you.
For other people getting this errore, I encountered it on an IBM i Series v7r3, when tried an UPDATE, retrieving the value to be set on a field using a inner SELECT where multiple results where reduced to 1, using DISTINCT. I solved the problem removing DISTINCT and adding FETCH FIRST 1 ROW ONLY at the end of the inner SELECT.
E.g.: changed from
UPDATE MYTABLE AS T1
SET T1.FIELD1 = (
SELECT DISTINCT T2.FIELD5
FROM MYTABLE AS T2
WHERE T1.FIELD2 = T2.FIELD2
AND T1.FIELD3 = T2.FIELD3
)
WHERE T1.FIELD4 = 'XYZ'
to
UPDATE MYTABLE AS T1
SET T1.FIELD1 = (
SELECT T2.FIELD5
FROM MYTABLE AS T2
WHERE T1.FIELD2 = T2.FIELD2
AND T1.FIELD3 = T2.FIELD3
FETCH FIRST 1 ROW ONLY
)
WHERE T1.FIELD4 = 'XYZ'