I am trying to write a Function/Macro in Hive. Instead of pushing it to UDF in java, I am trying to write within hql. Example below
CREATE TEMPORARY MACRO testing(m string, mv double)
(
'Testing',
named_struct('m', m, 'value', mv)
);
It is getting following error -
org.apache.hadoop.hive.ql.parse.ParseException: mismatched input ',' expecting ) near ''Testing'' in expression specification
I tried searching, but couldn't find good documentation on how to fix this.
Related
I have a condition currently in presto like below
replace(replace(replace(replace(replace(replace(replace(replace(UPPER(FIELDNAME),' ',''),'LLC',''),'INC',''),'INTERNATIONAL',''),'LTD',''),'.',''),',',''),'QMT','')
What will be the equivalent function in Hive that will serve the exact same prurpose? Will regexp_replace work for the above scenario?
Used regexp_replace in the above condition instead of replace, but it is throwing me the below error.
FAILED: SemanticException [Error 10014]: Line 1:255 Wrong arguments ''('': No matching method for class org.apache.hadoop.hive.ql.udf.UDFRegExpReplace with (string, string). Possible choices: _FUNC_(string, string, string) (state=42000,code=10014)
It would be greatful if someone can help on this. Thanks
That is true. regexp_replace is the command that you can use. You can give the list of substrings to be substituted as a list separated by '|' and you need to use '\' to give any special characters which might act as wildcards like - . or ? or * etc. Below is the sample usage.
SELECT regexp_replace(upper(fieldname),'LTD|QMT|INTERNATIONAL|ORG|INC|\\.|,| ','') from my_table;
I'm fairly new to coding and I was wondering if you could give me a hand writing some regular expression for BigQuery SQL.
Basically I would like to extract everything before the bar sign "|" for one of my column.
Example:
Source string:
bla-BLABLA-cid=123456_sept1220_blabla--potato-Blah|someMore_string_stuff-IDontNeed
Desired output:
bla-BLABLA-cid=123456_sept1220_blabla--potato-Blah
I thought about using the REGEXP_EXTRACT(string, delimiter) function but I'm totally unable to write some regex (LOL). Therefore I had a look over Stack, and have found stuff like:
SELECT REGEXP_EXTRACT( String_Name , "\S*\s*\|" ) ,
# or
SELECT REGEXP_EXTRACT( String_Name , '.+?(?=|)')
But every time I get error messages like " invalid perl operator: (?= " or "Illegal escape space"
Would you have any suggestions on why I get these messages and/or how could I proceed to extract these strings?
Many many thanks in advance <3
You can use SPLIT instead:
SELECT SPLIT("bla-BLABLA-cid=123456_sept1220_blabla--potato-Blah|someMore_string_stuff-IDontNeed", "|")[OFFSET(0)]
Prefix the pattern string with r:
SELECT REGEXP_EXTRACT(String_Name, r'\S*\s*\|')
This is the syntax for a raw string constant. You can review what this means in the documentation.
I am trying to do a simple regex pattern match in Redshift
I have this code and I get the following error:
REGEXP_COUNT ( "code", '^(?=.{8}$)[A-z]{2,5}[0-9]{3,6}$' )
ERROR: Invalid preceding regular expression prior to repetition operator. The error occured while parsing the regular expression fragment: '^(?>>>HERE>>>=.{8}$)[A-'.
The pattern works fine in testing in python and online checkers I'm guessing its a REGEX language problem. I have checked in PostgreSQL documentation on REGEX to try get help as I can't find much details on actual Redshift.
Thanks,
I am trying to use a UDF in hive. But when I try to create a temporary function using userdate as 'unixtimeToDate', I get this exception
hive> create temporary function userdate1 as 'unixtimeToDate';
FAILED: ParseException line 1:25 character ' ' not supported here
line 1:35 character ' ' not supported here
I am not sure why the character is not supported. Could I get some guidance on this please.
The exception is clear enough here, you have an error in your SQL. You have a full width space in your SQL. More about Halfwidth_and_fullwidth_forms
hive> create temporary function userdate1 as 'unixtimeToDate';
^^^here, you have a full width space
org.apache.spark.sql.AnalysisException: line .. character ' ' not supported here
In my situation, it's because the ' ' is not a normal space. I replaced them all and it's OK.
Below is syntax for adding jar in hive
ADD JAR absolute_path_of_jar_file;
CREATE TEMPORARY FUNCTION function_name AS 'packagename.ClassName';
I'm trying to insert ODI step error message into oracle table.
I captured the error message using <%=odiRef.getPrevStepLog("MESSAGE")%>.
ODI-1226: Step PRC_POA_XML_synchronize fails after 1 attempt(s).
ODI-1232: Procedure PRC_POA_XML_synchronize execution fails.
ODI-1227: Task PRC_POA_XML_synchronize (Procedure) fails on the source XML connection XML_PFIZER_LOAD_POA_DB_DEV.
Caused By: java.sql.SQLException: class java.sql.SQLException
oracle.xml.parser.v2.XMLParseException: End tag does not match start tag 'tns3:ContctID'.
at com.sunopsis.jdbc.driver.xml.SnpsXmlFile.readDocument(SnpsXmlFile.java:459)
at com.sunopsis.jdbc.driver.xml.SnpsXmlFile.readDocument(SnpsXmlFile.java:469)
When I try to insert this into a table, I'm getting the following error:
Missing IN or OUT parameter at index:: 1
I tried with substr, replace. Nothing works as in middle of the error message we have a single quotes 'tns3:ContctID'.
Is there any way to insert this into a table?
that's a tough one if you want to use pure java BeanShell and you've given way too little details to get short and straight answer, like
how do you try to insert this (command on source/target, bean shell only, Oracle SQL +jBS, jython, groovy etc...)
The problem here is not only quotes but also newlines.
To replace them is even more difficult as every parsing step <%, <?, <# requires different trick to define those literals
What will work for sure is if you write Jython task for inserting log data (Jython in technology).
There you may use Python ability for multiline string literals
simply:
⋮
err_log = """
<?=odiRef.getPrevStepLog("MESSAGE")?>
"""
⋮
I faced this error few days back . I applied below mentioned solution in ODI ...
Use - q'#<%=odiRef.getPrevStepLog("MESSAGE")%>#'
This will escape inverted comma (') for INSERT statement.
I have used this in my code and it is working fine :)
For example -
select 'testing'abcd' from dual;
this query will give below error
"ORA-01756: quoted string not properly terminated"
select q'#testing'abcd#' from dual;
This query gives no error and we get below response in SQL Developer
testing'abcd