how insert ODI step error message in to Oracle table, if the error message has single quotes and colons - sql

I'm trying to insert ODI step error message into oracle table.
I captured the error message using <%=odiRef.getPrevStepLog("MESSAGE")%>.
ODI-1226: Step PRC_POA_XML_synchronize fails after 1 attempt(s).
ODI-1232: Procedure PRC_POA_XML_synchronize execution fails.
ODI-1227: Task PRC_POA_XML_synchronize (Procedure) fails on the source XML connection XML_PFIZER_LOAD_POA_DB_DEV.
Caused By: java.sql.SQLException: class java.sql.SQLException
oracle.xml.parser.v2.XMLParseException: End tag does not match start tag 'tns3:ContctID'.
at com.sunopsis.jdbc.driver.xml.SnpsXmlFile.readDocument(SnpsXmlFile.java:459)
at com.sunopsis.jdbc.driver.xml.SnpsXmlFile.readDocument(SnpsXmlFile.java:469)
When I try to insert this into a table, I'm getting the following error:
Missing IN or OUT parameter at index:: 1
I tried with substr, replace. Nothing works as in middle of the error message we have a single quotes 'tns3:ContctID'.
Is there any way to insert this into a table?

that's a tough one if you want to use pure java BeanShell and you've given way too little details to get short and straight answer, like
how do you try to insert this (command on source/target, bean shell only, Oracle SQL +jBS, jython, groovy etc...)
The problem here is not only quotes but also newlines.
To replace them is even more difficult as every parsing step <%, <?, <# requires different trick to define those literals
What will work for sure is if you write Jython task for inserting log data (Jython in technology).
There you may use Python ability for multiline string literals
simply:
⋮
err_log = """
<?=odiRef.getPrevStepLog("MESSAGE")?>
"""
⋮

I faced this error few days back . I applied below mentioned solution in ODI ...
Use - q'#<%=odiRef.getPrevStepLog("MESSAGE")%>#'
This will escape inverted comma (') for INSERT statement.
I have used this in my code and it is working fine :)
For example -
select 'testing'abcd' from dual;
this query will give below error
"ORA-01756: quoted string not properly terminated"
select q'#testing'abcd#' from dual;
This query gives no error and we get below response in SQL Developer
testing'abcd

Related

Cannot query my container using the GridDB Shell?

I have a device container with the name 1cbfce15ec4d which houses some my data. I know for a fact there's data in there, but when I try a simple query in the griddb shell, I got the following error:
gs[public]> sql select * from 1cbfce15ec4d;
D20332: An unexpected error occurred while executing a SQL. : msg=[[240001:SQL_COMPILE_SYNTAX_ERROR] Parse SQL failed, reason = Syntax error: 1cbfce15ec4d; on executing query (sql="select * from 1cbfce15ec4d") (db='public') (user='admin') (appName='gs_sh') (clientId='a6d92f48-e558-440-86dd-a05e949fa726:1') (clientNd='{clientId=3, address=127.0.0.1:55744}') (address=127.0.0.1:20001, partitionId=983)]
I am not exactly sure what is going on here -- at first I assumed my data must be corrupt or empty, but that is not the case. It seems to be a case of the shell dying trying to process something about that container name.
Any ideas?
According to the manual :
"If the name of a table or a column contains characters other than ASCII alphanumeric characters and underscore, or if the first character of the name is a number in a SQL statement, enclose the name with double quotation marks."
Try select * from "1cbfce15ec4d"

Replace string output from query in results tab

I have a DB with a large number of CLOB columns. Each of these columns contains a repeated set of characters, which are fillers designed to designate a paragraph break (I didn't design the tables).
Is there a way to write the script so that each time the script finds these characters it enters a paragraph or line break into the results that are returned, while still saying in the same row of the results?
The data would look like
"Hello all,XYZ!£$I can't get ... to work.XYZ!£$The error meassge says ..."
As an example:
SELECT *
FROM ALERTS
REPLACE(Alert_Text, 'XYZ!£$', CH(13))
(Obviously the above returns errors)
The ideal output of the query would be:
"Hello all,
I can't get ... to work.
The error meassge says ..."
I am using SqlDbx to connect to an Oracle DB.
The obvious error is ORA-00904: "CH": invalid identifier
The reason is, that the function name is CHR
select REPLACE(txt, 'XYZ!£$', CHR(13)) from tab;
REPLACE(TXT,'XYZ!£$',CHR(13))
--------------------------------------------------------------------------------
Hello all,
I can't get ... to work.
The error meassge says ...

Hive -character '<EOF>' not supported here

Select * from mytable where field=
'ce7bd3d4-dbdd-407e-a3c3-ce093a65abc9;cdb597073;7cf6cda5fc'
Getting Below Error while running above query in Hive
FAILED: ParseException line 1:92 character '' not supported here
<EOF> here means End Of File. When you get an "unexpected End Of File" error it means the parser reached the end of the query unexpectedly. This typically happens when the parser is expecting to find a closing character, such as when you have started a string with ' or " but have not closed the string (with the closing ' or ").
When you come across these types of errors it is good to check that your query can be parsed correctly. In addition, the error gives you the location where the parser failed: line 1:92 in this case. You can usually look at this location (character 92 of the query) and work backwards to find the problem character.
Try adding the database name to the "from" statement as below.
Select * from my_db_name.mytable where field= 'ce7bd3d4-dbdd-407e-a3c3-
ce093a65abc9;cdb597073;7cf6cda5fc';
Hive uses the default database when no database was previously specified.

Inserting into Postgres Multidimensional Text Array from NodeJS Knex

I am attempting to insert a single row into a Postgres table from a NodeJS application using the Knex-Seed-File module for Knex.
Upon each attempt, I receive an error for only one column/field which is a multidimensional text array: photo_urls text[][] NULL,. The error states there is a malformed array literal.
Having gone through the official Postgres documentation, I've tried using double quotes:
(8.14.2. Array Value Input)
"To write an array value as a literal constant, enclose the element values within curly braces and separate them by commas...You can put double quotes around any element value, and must do so if it contains commas or curly braces."
I've also tried using ARRAY constructor syntax.
Here are various attempts Ive had at constructing the input as well as what was returned as being the actual SQL generated and the returned error:
Attempt 1:
array[['ext_profile:','ext_random:','int_random:'],['https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Profile','https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Random+1,https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Random+2','https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Int+Random+1,https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=In+Random+2,https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Int+Random+3']]
Result 1:
'array[[\'ext_profile:\',\'ext_random:\',\'int_random:\'],[\'https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Profile\',\'https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Random+1,https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Random+2\',\'https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Int+Random+1,https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=In+Random+2,https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Int+Random+3\']]'
Error 1:
- malformed array literal: "array[['ext_profile:','ext_random:','int_random:'],['https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Profile','https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Random+1,https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Random+2','https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Int+Random+1,https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=In+Random+2,https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Int+Random+3']]
Attempt 2:
$${"ext_profile:", "ext_random:", "int_random:"},{"https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Profile", "https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Random+1, https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Random+2", "https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Int+Random+1, https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=In+Random+2, https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Int+Random+3"}$$
Result 2:
'"$${""ext_profile:"", ""ext_random:"", ""int_random:""},{""https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Profile"", ""https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Random+1, https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Random+2"", ""https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Int+Random+1, https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=In+Random+2, https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Int+Random+3""}$$"'
Error 2:
- malformed array literal: ""$${""ext_profile:"", ""ext_random:"", ""int_random:""},{""https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Profile"", ""https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Random+1, https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Random+2"", ""https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Int+Random+1, https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=In+Random+2, https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Int+Random+3""}$$"
Attempt 3:
($${"ext_profile:", "ext_random:", "int_random:"},{"https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Profile", "https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Random+1, https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Random+2", "https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Int+Random+1, https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=In+Random+2, https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Int+Random+3"}$$)
Result 3:
'"($${""ext_profile:"", ""ext_random:"", ""int_random:""},{""https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Profile"", ""https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Random+1, https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Random+2"", ""https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Int+Random+1, https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=In+Random+2, https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Int+Random+3""}$$)"'
Error 3:
- malformed array literal: ""($${""ext_profile:"", ""ext_random:"", ""int_random:""},{""https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Profile"", ""https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Random+1, https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Random+2"", ""https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Int+Random+1, https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=In+Random+2, https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Int+Random+3""}$$)"
Attempt 4:
array[['ext_profile:','ext_random:','int_random:'],["https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Profile","https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Random+1,https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Random+2","https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Int+Random+1,https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=In+Random+2,https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Int+Random+3"]]
Result 4:
'"array[[\'ext_profile:\',\'ext_random:\',\'int_random:\'],[""https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Profile"",""https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Random+1,https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Random+2"",""https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Int+Random+1,https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=In+Random+2,https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Int+Random+3""]]"'
Error 4:
- malformed array literal: ""array[['ext_profile:','ext_random:','int_random:'],[""https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Profile"",""https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Random+1,https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Random+2"",""https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Int+Random+1,https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=In+Random+2,https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Int+Random+3""]]"
Attempt 5 (Post knex-seed-file upgrade):
[["ext_profile:","ext_random:","int_random:"],["https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Profile","https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Random+1,https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Random+2","https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Int+Random+1,https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=In+Random+2,https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Int+Random+3"]]
Result 5:
'"[[""ext_profile:"",""ext_random:"",""int_random:""],[""https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Profile"",""https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Random+1,https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Random+2"",""https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Int+Random+1,https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=In+Random+2,https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Int+Random+3""]]"'
Error 5:
- malformed array literal: ""[[""ext_profile:"",""ext_random:"",""int_random:""],[""https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Profile"",""https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Random+1,https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Random+2"",""https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Int+Random+1,https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=In+Random+2,https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Int+Random+3""]]"
There appear to be many bugs/issues reported as related to knex postgres integration:
#658, #828, #869, #1602,... which seem to have been closed and/or merged into #1661.
From what I can tell, it appears the issue was closed as resolved.
Can anyone help identify what I'm doing wrong or what I can do to resolve the issue?
The module is now upgraded (0.3.1) and should now handle arrays properly. To enter array value after updating the package, you should use following pattern:
[["ext_profile:","ext_random:","int_random:"],["https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Profile","https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Random+1,https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Ext+Random+2","https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Int+Random+1,https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=In+Random+2,https://dummyimage.com/300x250/0cb3f5/fcfcfc.png&text=Int+Random+3"]]
Please open an issue at https://github.com/tohalla/knex-seed-file, upon encountering more problems.
#touko correctly identified that the issue is a result of the default behavior for csv files.
When saving a csv file, double quotes are added to anything with embedded commas or double-quoted characters.
It is explained in these posts and articles:
superuser post 1
superuser post 2
csvreader.com
https://www.rfc-editor.org/rfc/rfc4180
In regards to the knex-seed-file module there is an issue opened in Github. Currently, I'm using the workaround of opening the csv file in a text editor and manually removing the undesired double quotes.
Example (note: I am using a pipe-delimited csv file):
find "[[ and replace with [[
find ]]" and replace with ]]
find "" and replace with "

Remove quote marks in Google Cloud Datalab SQL module parameters?

The parameterization example in the "SQL Parameters" IPython notebook in the datalab github repo (under datalab/tutorials/BigQuery/) shows how to change the value being tested for in a WHERE clause.
%%sql --module get_data
SELECT *
FROM
[myproject:mydataset.mytable]
WHERE
$query
However, this syntax always seems to insert quotation marks around the parameter. This breaks when I pass parameters that aren't just a simple value:
import gcp.bigquery as bq
query = "(bnf_code LIKE '1202%') OR (bnf_code LIKE '1203%')"
query = bq.Query(get_data, query=query)
print query.sql
This prints an invalid query:
SELECT * FROM [myproject:mydataset.mytable]
WHERE "(bnf_code LIKE '1202%') OR (bnf_code LIKE '1203%')"
Is there any way I can insert values that aren't wrapped in quotation marks?
I'm using the module repeatedly in my code, with variable numbers of OR clauses in the query parameter. So I do need a way to pass in more complicated queries.
Sorry, variables are meant to be simple scalars, or tables, or (soon) lists for use in IN clauses. They are not meant for expressions.
Passing unquoted arguments to SQL modules isn't possible, but it is possible to create a datalabs.data.SQLStatement with straight-up SQL in string form. With that you can use your own, Python-style placeholders to substitute values as you see fit:
import datalab.data._sql_statement as bqsql
statement = bqsql.SqlStatement(
"SELECT some-field FROM %s" % '[your-instance:some-table-name]')
query = bq.Query(statement)
I don't know if they're doing anything special with placeholders or the in-notebook command processing but... well, I didn't see any of that in my (admittedly limited) spelunking.