Getting oozie EL_ERROR. Please find below the oozie workflow details. Please advise.
Error Code: EL_ERROR
Error Message: An exception occured trying to convert String "/tmp/dir" to type "java.lang.Double"
Expected Result: /tmp/dir/0007735-200221155317872-oozie-oozi-W/destination
workflow.xml
<decision name="On_Success">
<switch>
<case to="success">${fs:dirSize(hdfs_path_variable+wf:id()+hdfs_dir) gt 0 }</case>
<default to="fail"/>
</switch>
</decision>
job.properties
hdfs_path_variable=/tmp/dir
hdfs_dir=destination
Related
I tried to use dirSize() in oozie decision node. but it does not work.
When I use ${fs:dirSize(InputDir) gt 10 * KB} in the decision node. The oozie workflow status moves to failed state and no error is displayed.
Please find below the code snippet from workflow.
<action name="L1_check" cred="hcat_creds">
<hive xmlns="uri:oozie:hive-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<script>l1_check_code.sql</script>
<param>PI_DB=${PI_DB}</param>
<param>PRD_DB=${PRD_DB}</param>
<param>PROMO_START_DATE=${PROMO_START_DATE}</param>
<param>PROMO_END_DATE=${PROMO_END_DATE}</param>
</hive>
<ok to="decision-node" />
<error to="fail" />
</action>
<decision name="decision-node">
<switch>
<case to="L1_exists">
${fs:dirSize(InputDir) gt 10 * KB}
</case>
<default to="fail"/>
</switch>
</decision>
InputDir is a defined in the properties file and the path exists.
First action L1_check executes and transitioned to decision node. then workflow status changes to Failed state.
Is there any error in the way function is used? If yes, what should be right function to use?
Also is there a way to give a path with combination of parameter and string
eg: dirSize(InputDir/l1_check)
where InputDir is a parameter and l1_check is a static name
I am want to run sqoop action using apache oozie but it always giving me error:
Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.SqoopMain], main() threw exception, null
my workflow.xml is:
<workflow-app name="Datashop-Core-Sqoop" xmlns="uri:oozie:workflow:0.1">
<!-- <start to="inputAvailableCheckDecision"/>
<decision name="inputAvailableCheckDecision">
<switch>
<case to="sqoopAction">
${sqoopInputRecordCount gt minRequiredRecordCount}
</case>
<default to="end"/>
</switch>
</decision> -->
<start to="sqoopAction"/>
<action name="sqoopAction">
<sqoop xmlns="uri:oozie:sqoop-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<configuration>
<property>
<name>mapred.job.queue.name</name>
<value>${queueName}</value>
</property>
<property>
<name>oozie.libpath</name>
<value>${oozieLibPath}</value>
</property>
</configuration>
<command>import --connect jdbc:mysql://${mysqlServer}/${mysqlServerDB} --username ${mysqlServerDBUID} --password ${mysqlServerDBPwd} --tabltable ${mysqlTable} -m 1 --target-dir ${targetDir}</command>
</sqoop>
<ok to="end"/>
<error to="killJob"/>
</action>
<kill name="killJob">
<message>"Killed job due to error: ${wf:errorMessage(wf:lastErrorNode())}"</message>
</kill>
<end name="end" />
</workflow-app>
this always throwing sqoopmain class exception which is finally treating as Error:
#sqoopAction] ERROR is considered as FAILED for SLA
In some stack question i found that external library is need to be included with <file>
so i changed my workflow.xml and add a line below <command> tag:
<file>${oozieLibPath}/sqoop/mysql-connector-java-5.1.37-bin.jar#mysql-connector-java-5.1.37-bin.jar</file>
But still same error is producing what may be the reason for this error. is there any thing else i need to configure?
I am trying to run an Oozie workflow that calls EL function replaceAll(). The action that's using the replaceAll() is this
<action name="createSuccess">
<fs>
<configuration>
<property>
<name>runDate</name>
<value>${replaceAll(hdfsDir, nameNode + '/(.+)/' + region + '/([0-9\\-]+)/?', '$2')}</value>
</property>
</configuration>
<mkdir path="${nameNode}/path/run/${region}/${runDate}"/>
<touchz path="${nameNode}/path/run/${region}/${runDate}/success.txt"/>
</fs>
<ok to="end"/>
<error to="sendEmailKill"/>
</action>
hdfsDir is something like hdfs://nameNode:8020/some/path/region/2015-04-22 and I need to grab that date at the end as a property and use it.
But when I run the above action, I got this exception:
javax.servlet.jsp.el.ELException: An exception occured trying to convert String "hdfs://nameNode:8020" to type "java.lang.Double"
at org.apache.commons.el.Logger.logError(Logger.java:481)
at org.apache.commons.el.Logger.logError(Logger.java:498)
at org.apache.commons.el.Logger.logError(Logger.java:566)
at org.apache.commons.el.Coercions.coerceToPrimitiveNumber(Coercions.java:440)
at org.apache.commons.el.Coercions.applyArithmeticOperator(Coercions.java:852)
at org.apache.commons.el.ArithmeticOperator.apply(ArithmeticOperator.java:83)
at org.apache.commons.el.BinaryOperatorExpression.evaluate(BinaryOperatorExpression.java:170)
at org.apache.commons.el.FunctionInvocation.evaluate(FunctionInvocation.java:163)
at org.apache.commons.el.ExpressionString.evaluate(ExpressionString.java:114)
at org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:274)
at org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:190)
at org.apache.oozie.util.ELEvaluator.evaluate(ELEvaluator.java:203)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:175)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:60)
at org.apache.oozie.command.XCommand.call(XCommand.java:280)
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:326)
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:255)
at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:175)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Any ideas why I'm getting this exception and how to fix it?
After trial an error, I figured out that I can not use "+" to concatenate two strings. I have to use this:
${replaceAll(hdfsDumpDir, concat(concat(concat(nameNode, '/(.+)/'), region), '/'), '')}
How do you throw an error from within an MSBuild task and force the build to fail. Something like:
<Task>
<ThrowError Condition="$(SomeCondition)" Message="There was a problem with the build" />
</Task>
Use the Error Task
<Error Condition="$(SomeCondition)" Text="There was a problem with the build" />
The Error-task would do the trick.
<Error
Text="errormessage"
Condition="errorcondition" />
Have you tried Error instead of ThrowError ? It works for me ;)
I have a problem with the Xml.ModifyFile task which I do not understand. Can you guys help?
My goal is simply to manipulate an attribute in an xml document.
Im fairly new to the world of xml and especially msbuild hence I how a hard time interpreting the error message i am receiving. It seems to me that my build file is valid so I guess something is wrong in sdc.tasks dll file.
As it can be seen from the build file I have added a target called "ping" for the sake of testing. That target works with the sdc.task Ping without any problems
Can you guys suggest a fix or an alternative solution to the challenge of modifying xml files with msbuild.
An additional question - how do one declare multiple namespaces as argument to the Xml.ModifyFile sdc.task? The explanation of the namespace attribute is as follows:
An array of TaskItems specifiying "Prefix" and "Uri" attributes for use with the specified xPath. I have tried to find an explanation or example of the usage of taskitems but unfortunately without any luck.
thanks / derdres
I will list the following below:
build file
the xml file that I try to modify
the error message
1) build file
<Target Name="Go">
<CallTarget Targets="modify"></CallTarget>
<!--<CallTarget Targets="ping"></CallTarget>-->
</Target>
<Target Name="modify">
<Xml.ModifyFile
Path="C:\Users\Andreas\Desktop\MSBuild\Test_05_april\Test01\bookstore_adv.xml"
AttributeName="age"
Force="true"
XPath="/bookstore/book[#id=2]/#age"
NewValue="200"
ShowMatches="Yes"
>
</Xml.ModifyFile>
<Message Text="After modification"></Message>
</Target>
<!--<Target Name="ping">
<Ping
Machine="localhost"
Count="2"
Interval="1000"
Timeout="3000"
BufferSize="1024"
AllowFragmentation="false"
TimeToLive="128"
StopOnSuccess="true"
LogSuccess="true">
<Output TaskParameter="FailureCount" PropertyName="FailedPingCount" />
<Output TaskParameter="RoundTripTime" PropertyName="RoundTripDuration" />
</Ping>
<Message Text="FailedPingcount: $(FailedPingCount)"></Message>
<Message Text="RoundTripDuration: $(RoundTripDuration)"></Message>
</Target>-->
2) xml file
<?xml version="1.0" encoding="utf-8"?>
<!--<bookstore xmlns:hat="www.google.dk/hat" xmlns:briller="www.google.dk/briller">-->
<!--<bookstore xmlns:hat="www.google.dk/hat">-->
<bookstore>
<book id="1">
<title>Harry Potter</title>
<author>Rowling</author>
</book>
<book id="2" age="100">
<title>Lykke Per</title>
<author>Pontoppidan</author>
</book>
3) Build Error Message
Build FAILED.
"C:\Users\Andreas\Desktop\MSBuild\Test_05_april\Test01\Deploy.proj" (default target) (1) ->
(modify target) ->
C:\Users\Andreas\Desktop\MSBuild\Test_05_april\Test01\Deploy.proj(11,9): error : A task error has occured.\r
C:\Users\Andreas\Desktop\MSBuild\Test_05_april\Test01\Deploy.proj(11,9): error : Message = Object reference not set to
an instance of an object.\r
C:\Users\Andreas\Desktop\MSBuild\Test_05_april\Test01\Deploy.proj(11,9): error : Action = Replace\r
C:\Users\Andreas\Desktop\MSBuild\Test_05_april\Test01\Deploy.proj(11,9): error : Path = C:\Users\Andreas\Desktop\MS
Build\Test_05_april\Test01\bookstore_advanced.xml\r
C:\Users\Andreas\Desktop\MSBuild\Test_05_april\Test01\Deploy.proj(11,9): error : Namespace = <null>\r
C:\Users\Andreas\Desktop\MSBuild\Test_05_april\Test01\Deploy.proj(11,9): error : XPath = /bookstore/book[#id=2]/#age
\r
C:\Users\Andreas\Desktop\MSBuild\Test_05_april\Test01\Deploy.proj(11,9): error : RegularExpression = <String.Empty>\r
C:\Users\Andreas\Desktop\MSBuild\Test_05_april\Test01\Deploy.proj(11,9): error : NewValue = 200\r
C:\Users\Andreas\Desktop\MSBuild\Test_05_april\Test01\Deploy.proj(11,9): error : AttributeName = age\r
C:\Users\Andreas\Desktop\MSBuild\Test_05_april\Test01\Deploy.proj(11,9): error : Force = True\r
C:\Users\Andreas\Desktop\MSBuild\Test_05_april\Test01\Deploy.proj(11,9): error : TreatNewValueAsXml = False\r
C:\Users\Andreas\Desktop\MSBuild\Test_05_april\Test01\Deploy.proj(11,9): error : ShowMatches = Yes\r
C:\Users\Andreas\Desktop\MSBuild\Test_05_april\Test01\Deploy.proj(11,9): error : \r
C:\Users\Andreas\Desktop\MSBuild\Test_05_april\Test01\Deploy.proj(11,9): error : at Microsoft.Sdc.Tasks.Xml.ModifyFile.Interna
lExecute() in c:\projects\codeplex\sdctasks\Solutions\Main\Tasks\Xml\ModifyFile.cs:line 346\r
C:\Users\Andreas\Desktop\MSBuild\Test_05_april\Test01\Deploy.proj(11,9): error : at Microsoft.Sdc.Tasks.TaskBase.Execute() in
c:\projects\codeplex\sdctasks\Solutions\Main\Tasks\TaskBase.cs:line 66
0 Warning(s)
1 Error(s)
Time Elapsed 00:00:00.20
In your XPath you are searching for the attribute age /bookstore/book[#id=2]/#age but in your task you set the AttributeName to "age". So it is like you want the attribute age of the attribute age.
You just have to change your XPath to /bookstore/book[#id=2] to make it work.
<Target Name="modify">
<Xml.ModifyFile
Path="C:\Users\Andreas\Desktop\MSBuild\Test_05_april\Test01\bookstore_adv.xml"
AttributeName="age"
Force="true"
XPath="/bookstore/book[#id=2]"
NewValue="200"
ShowMatches="Yes">
</Xml.ModifyFile>
<Message Text="After modification"/>
</Target>
How do one declare multiple namespaces as argument to the Xml.ModifyFile sdc.task?
<ItemGroup>
<Namespace Include="www.google.dk/briller">
<Prefix>briller</Prefix>
<Uri>www.google.dk/briller</Uri>
</Namespace>
<Namespace Include="www.google.dk/hat">
<Prefix>hat</Prefix>
<Uri>www.google.dk/hat</Uri>
</Namespace>
</ItemGroup>
<Target Name="modify">
<Xml.ModifyFile
Path="C:\Users\Andreas\Desktop\MSBuild\Test_05_april\Test01\bookstore_adv.xml"
AttributeName="age"
Force="true"
XPath="/bookstore/book[#id=2]"
NewValue="200"
ShowMatches="Yes"
Namespace="#(Namespace)">
</Xml.ModifyFile>
<Message Text="After modification"/>
</Target>
Your XML File Is Invalid.
I Tried It On My Machine , And It Worked Fine.
Just Close The bookstore Tag.