I am trying to use a UDF in hive. But when I try to create a temporary function using userdate as 'unixtimeToDate', I get this exception
hive> create temporary function userdate1 as 'unixtimeToDate';
FAILED: ParseException line 1:25 character ' ' not supported here
line 1:35 character ' ' not supported here
I am not sure why the character is not supported. Could I get some guidance on this please.
The exception is clear enough here, you have an error in your SQL. You have a full width space in your SQL. More about Halfwidth_and_fullwidth_forms
hive> create temporary function userdate1 as 'unixtimeToDate';
^^^here, you have a full width space
org.apache.spark.sql.AnalysisException: line .. character ' ' not supported here
In my situation, it's because the ' ' is not a normal space. I replaced them all and it's OK.
Below is syntax for adding jar in hive
ADD JAR absolute_path_of_jar_file;
CREATE TEMPORARY FUNCTION function_name AS 'packagename.ClassName';
Related
I have a condition currently in presto like below
replace(replace(replace(replace(replace(replace(replace(replace(UPPER(FIELDNAME),' ',''),'LLC',''),'INC',''),'INTERNATIONAL',''),'LTD',''),'.',''),',',''),'QMT','')
What will be the equivalent function in Hive that will serve the exact same prurpose? Will regexp_replace work for the above scenario?
Used regexp_replace in the above condition instead of replace, but it is throwing me the below error.
FAILED: SemanticException [Error 10014]: Line 1:255 Wrong arguments ''('': No matching method for class org.apache.hadoop.hive.ql.udf.UDFRegExpReplace with (string, string). Possible choices: _FUNC_(string, string, string) (state=42000,code=10014)
It would be greatful if someone can help on this. Thanks
That is true. regexp_replace is the command that you can use. You can give the list of substrings to be substituted as a list separated by '|' and you need to use '\' to give any special characters which might act as wildcards like - . or ? or * etc. Below is the sample usage.
SELECT regexp_replace(upper(fieldname),'LTD|QMT|INTERNATIONAL|ORG|INC|\\.|,| ','') from my_table;
I'm trying to concatenate two strings in Hive like below, but it keeps complaining about a ParseException
select concat('', cast(77 as varchar), '') as page_url
;
The message says:
FAILED: ParseException line 1:13 cannot recognize input near 'concat' '(' ''<a href="'' in expression specification
I tried using backticks around the strings and also escaping any potential special characters, but no luck. How can I get the concatenation to work?
I'm using Hive version 2.0.4
In this particular case, the problem was arising because I was using cast as varchar when I should have been using string
Select * from mytable where field=
'ce7bd3d4-dbdd-407e-a3c3-ce093a65abc9;cdb597073;7cf6cda5fc'
Getting Below Error while running above query in Hive
FAILED: ParseException line 1:92 character '' not supported here
<EOF> here means End Of File. When you get an "unexpected End Of File" error it means the parser reached the end of the query unexpectedly. This typically happens when the parser is expecting to find a closing character, such as when you have started a string with ' or " but have not closed the string (with the closing ' or ").
When you come across these types of errors it is good to check that your query can be parsed correctly. In addition, the error gives you the location where the parser failed: line 1:92 in this case. You can usually look at this location (character 92 of the query) and work backwards to find the problem character.
Try adding the database name to the "from" statement as below.
Select * from my_db_name.mytable where field= 'ce7bd3d4-dbdd-407e-a3c3-
ce093a65abc9;cdb597073;7cf6cda5fc';
Hive uses the default database when no database was previously specified.
I am importing data from teradata to hive using Sqoop.
I added --map-column-hive in sqoop import command
--map-column-hive col1=int,col2=float,col3=decimal,col4=timestamp,col5=varchar
I got exception:
FAILED: ParseException line 1:234 mismatched input ',' expecting ( near 'varchar' in primitive type specification
Then I tried:
--map-column-hive col1=int,col2=float,col3=decimal,col4=timestamp,col5=varchar(255)
I got:
bash: syntax error near unexpected token `('
How to handle char, varchar and decimal in this?
After checking source code and tracking issue related to this. I found:
For --map-column-hive tag
Without precision and scale
key=value
e.g. col1=int
With Precision
key = "value(precision)" (can be in single quotes)
e.g. col2="varchar(255)"
With Scale
There is a bug for this, fixed in Sqoop 1.4.7
Fix is not straightforward.
For example, for a column col3=decimal(1,1) one need to write col3=decimal(1%2C1)
Check SQOOP-2103 issue for more details
I'm trying to insert ODI step error message into oracle table.
I captured the error message using <%=odiRef.getPrevStepLog("MESSAGE")%>.
ODI-1226: Step PRC_POA_XML_synchronize fails after 1 attempt(s).
ODI-1232: Procedure PRC_POA_XML_synchronize execution fails.
ODI-1227: Task PRC_POA_XML_synchronize (Procedure) fails on the source XML connection XML_PFIZER_LOAD_POA_DB_DEV.
Caused By: java.sql.SQLException: class java.sql.SQLException
oracle.xml.parser.v2.XMLParseException: End tag does not match start tag 'tns3:ContctID'.
at com.sunopsis.jdbc.driver.xml.SnpsXmlFile.readDocument(SnpsXmlFile.java:459)
at com.sunopsis.jdbc.driver.xml.SnpsXmlFile.readDocument(SnpsXmlFile.java:469)
When I try to insert this into a table, I'm getting the following error:
Missing IN or OUT parameter at index:: 1
I tried with substr, replace. Nothing works as in middle of the error message we have a single quotes 'tns3:ContctID'.
Is there any way to insert this into a table?
that's a tough one if you want to use pure java BeanShell and you've given way too little details to get short and straight answer, like
how do you try to insert this (command on source/target, bean shell only, Oracle SQL +jBS, jython, groovy etc...)
The problem here is not only quotes but also newlines.
To replace them is even more difficult as every parsing step <%, <?, <# requires different trick to define those literals
What will work for sure is if you write Jython task for inserting log data (Jython in technology).
There you may use Python ability for multiline string literals
simply:
⋮
err_log = """
<?=odiRef.getPrevStepLog("MESSAGE")?>
"""
⋮
I faced this error few days back . I applied below mentioned solution in ODI ...
Use - q'#<%=odiRef.getPrevStepLog("MESSAGE")%>#'
This will escape inverted comma (') for INSERT statement.
I have used this in my code and it is working fine :)
For example -
select 'testing'abcd' from dual;
this query will give below error
"ORA-01756: quoted string not properly terminated"
select q'#testing'abcd#' from dual;
This query gives no error and we get below response in SQL Developer
testing'abcd