I am trying to run an Oozie workflow that calls EL function replaceAll(). The action that's using the replaceAll() is this
<action name="createSuccess">
<fs>
<configuration>
<property>
<name>runDate</name>
<value>${replaceAll(hdfsDir, nameNode + '/(.+)/' + region + '/([0-9\\-]+)/?', '$2')}</value>
</property>
</configuration>
<mkdir path="${nameNode}/path/run/${region}/${runDate}"/>
<touchz path="${nameNode}/path/run/${region}/${runDate}/success.txt"/>
</fs>
<ok to="end"/>
<error to="sendEmailKill"/>
</action>
hdfsDir is something like hdfs://nameNode:8020/some/path/region/2015-04-22 and I need to grab that date at the end as a property and use it.
But when I run the above action, I got this exception:
javax.servlet.jsp.el.ELException: An exception occured trying to convert String "hdfs://nameNode:8020" to type "java.lang.Double"
at org.apache.commons.el.Logger.logError(Logger.java:481)
at org.apache.commons.el.Logger.logError(Logger.java:498)
at org.apache.commons.el.Logger.logError(Logger.java:566)
at org.apache.commons.el.Coercions.coerceToPrimitiveNumber(Coercions.java:440)
at org.apache.commons.el.Coercions.applyArithmeticOperator(Coercions.java:852)
at org.apache.commons.el.ArithmeticOperator.apply(ArithmeticOperator.java:83)
at org.apache.commons.el.BinaryOperatorExpression.evaluate(BinaryOperatorExpression.java:170)
at org.apache.commons.el.FunctionInvocation.evaluate(FunctionInvocation.java:163)
at org.apache.commons.el.ExpressionString.evaluate(ExpressionString.java:114)
at org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:274)
at org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:190)
at org.apache.oozie.util.ELEvaluator.evaluate(ELEvaluator.java:203)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:175)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:60)
at org.apache.oozie.command.XCommand.call(XCommand.java:280)
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:326)
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:255)
at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:175)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Any ideas why I'm getting this exception and how to fix it?
After trial an error, I figured out that I can not use "+" to concatenate two strings. I have to use this:
${replaceAll(hdfsDumpDir, concat(concat(concat(nameNode, '/(.+)/'), region), '/'), '')}
Related
I have upgraded a Spring boot service to Infinispan 9.4.16.Final from 5.2.20.Final. The service has two XML files. I used the conversion script to convert them. Both have local-cache entries and no other types of caches. One was left with empty transport element by the conversion tool.
When we deploy and run these services, we often see this warning at startup:
org.infinispan.manager.EmbeddedCacheManagerStartupException: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.globalstate.impl.GlobalConfigurationManagerImpl.start() on object of type GlobalConfigurationManagerImpl
The above is the first warning/error we see. There is no stack trace. Why would it be calling GlobalConfigurationManagerImpl when we're only using local cache?
A few lines later in the log, then I see many The cache has been stopped and invocations are not allowed! errors. The last error we see is as follows. The service fails to start up successfully.
Caused by: org.infinispan.commons.CacheException: Initial state transfer timed out for cache org.infinispan.CONFIG on <server_name>
Why are these errors/warnings happening on startup? Is there a problem in the config files? I've searched online and have not found a solution.
~~More Info~~~
Here is one of the two XML config files:
<infinispan
xmlns:xsi = "http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation = "urn:infinispan:config:9.4 http://www.infinispan.org/schemas/infinispan-config-9.4.xsd"
xmlns = "urn:infinispan:config:9.4">
<threads/>
<cache-container name = "TestCenterServiceCache">
<!-- The conversion tool added this empty "transport" element. It was not present in our old config file -->
<transport/>
<jmx domain = "org.infinispan.TestCenterServiceCache"/>
<local-cache name = "authorizedLocations">
<expiration lifespan = "3600000"/>
</local-cache>
<!--caching for 24 hours: 3,600,000 milliseconds/hr x 24 hours -->
<local-cache name = "proximitySearchConfiguration">
<expiration lifespan = "86400000"/>
</local-cache>
</cache-container>
</infinispan>
The above is instantiated via applicationContext.xml. The first warning (GlobalConfigurationManagerImpl.start()) is referencing these beans.
<bean id="infinispanCacheManager"
class="org.infinispan.spring.embedded.support.InfinispanEmbeddedCacheManagerFactoryBean"
p:configurationFileLocation="classpath:testCenterServices-cache-config.xml" />
<bean id="cacheManager"
class="org.infinispan.spring.embedded.provider.SpringEmbeddedCacheManager">
<constructor-arg ref="infinispanCacheManager" />
</bean>
Here is the second XML config file:
<?xml version="1.0" ?>
<infinispan
xmlns:xsi = "http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation = "urn:infinispan:config:9.4 http://www.infinispan.org/schemas/infinispan-config-9.4.xsd"
xmlns = "urn:infinispan:config:9.4">
<threads />
<cache-container name="AtlasServicesCacheManager">
<local-cache name="allLocaleCache" />
<local-cache name="localeCacheByID" />
<local-cache name="countryByCode" />
<local-cache name="allActiveCountries" />
<local-cache name="allCountries" />
<local-cache name="allStatesForCountryCode" />
<local-cache name="allActiveStatesForCountryCode" />
<local-cache name="stateForCountryCodeStateCode" />
</cache-container>
</infinispan>
The above is instantiated via java code.
#Bean(name="atlasServicesCacheManager")
public CacheManager makeCacheManager() throws IOException {
return new SpringEmbeddedCacheManager(new DefaultCacheManager("atlas-cache-config.xml"));
}
I don't know if it's meaningful, but only after the upgrade, we log messages that include "JGroups", such as Unable to use any JGroups configuration mechanisms provided in properties {}. Using default JGroups configuration!.
The service instances are running on Windows Server 2012 R2 Standard (Windows 8).
To fix this, remove the empty <transport /> element for local caches.
Adding that empty element seems to be a defect in the config-converter. With the empty transport element in place, it seems that Infinispan is partially configured for cluster synchronization. For details on the underlying issue, see bug report: https://issues.redhat.com/browse/ISPN-11854.
Getting oozie EL_ERROR. Please find below the oozie workflow details. Please advise.
Error Code: EL_ERROR
Error Message: An exception occured trying to convert String "/tmp/dir" to type "java.lang.Double"
Expected Result: /tmp/dir/0007735-200221155317872-oozie-oozi-W/destination
workflow.xml
<decision name="On_Success">
<switch>
<case to="success">${fs:dirSize(hdfs_path_variable+wf:id()+hdfs_dir) gt 0 }</case>
<default to="fail"/>
</switch>
</decision>
job.properties
hdfs_path_variable=/tmp/dir
hdfs_dir=destination
in my application im using infinispan to manage my data, and now i want to save my data in a flat file and it doesn't work.
here is my infinispan conf file :
http://infinispan.org/schemas/infinispan-config-9.3.xsd"
xmlns="urn:infinispan:config:9.3">
<loaders passivation="false" shared="false" preload="true">
<loader class="org.infinispan.loaders.file.FileCacheStore" fetchPersistentState="true"
ignoreModifications="false" purgeOnStartup="false">
<properties>
<property name="location" value="C:\infinispan-4.0.0.FINAL\store"/>
</properties>
</loader>
</loaders>
</namedCache>
when i run my class , i encounter this error :
org.infinispan.commons.CacheConfigurationException: javax.xml.stream.XMLStreamException: ParseError at [row,col]:[4,35]
Message: Unexpected element '{urn:infinispan:config:9.3}namedCache' encountered
Have you any idea please ?
You are using an old configuration format which we have abandoned since Infinispan 7.0. For 9.3 you should use something like the following:
<infinispan
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="urn:infinispan:config:9.3 http://www.infinispan.org/schemas/infinispan-config-9.3.xsd"
xmlns="urn:infinispan:config:9.3">
<cache-container default-cache="local">
<local-cache name="local">
<persistence>
<file-store path="path" shared="false" purge="false"/>
</persistence>
</local-cache>
</cache-container>
</infinispan>
I tried to use dirSize() in oozie decision node. but it does not work.
When I use ${fs:dirSize(InputDir) gt 10 * KB} in the decision node. The oozie workflow status moves to failed state and no error is displayed.
Please find below the code snippet from workflow.
<action name="L1_check" cred="hcat_creds">
<hive xmlns="uri:oozie:hive-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<script>l1_check_code.sql</script>
<param>PI_DB=${PI_DB}</param>
<param>PRD_DB=${PRD_DB}</param>
<param>PROMO_START_DATE=${PROMO_START_DATE}</param>
<param>PROMO_END_DATE=${PROMO_END_DATE}</param>
</hive>
<ok to="decision-node" />
<error to="fail" />
</action>
<decision name="decision-node">
<switch>
<case to="L1_exists">
${fs:dirSize(InputDir) gt 10 * KB}
</case>
<default to="fail"/>
</switch>
</decision>
InputDir is a defined in the properties file and the path exists.
First action L1_check executes and transitioned to decision node. then workflow status changes to Failed state.
Is there any error in the way function is used? If yes, what should be right function to use?
Also is there a way to give a path with combination of parameter and string
eg: dirSize(InputDir/l1_check)
where InputDir is a parameter and l1_check is a static name
I am want to run sqoop action using apache oozie but it always giving me error:
Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.SqoopMain], main() threw exception, null
my workflow.xml is:
<workflow-app name="Datashop-Core-Sqoop" xmlns="uri:oozie:workflow:0.1">
<!-- <start to="inputAvailableCheckDecision"/>
<decision name="inputAvailableCheckDecision">
<switch>
<case to="sqoopAction">
${sqoopInputRecordCount gt minRequiredRecordCount}
</case>
<default to="end"/>
</switch>
</decision> -->
<start to="sqoopAction"/>
<action name="sqoopAction">
<sqoop xmlns="uri:oozie:sqoop-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<configuration>
<property>
<name>mapred.job.queue.name</name>
<value>${queueName}</value>
</property>
<property>
<name>oozie.libpath</name>
<value>${oozieLibPath}</value>
</property>
</configuration>
<command>import --connect jdbc:mysql://${mysqlServer}/${mysqlServerDB} --username ${mysqlServerDBUID} --password ${mysqlServerDBPwd} --tabltable ${mysqlTable} -m 1 --target-dir ${targetDir}</command>
</sqoop>
<ok to="end"/>
<error to="killJob"/>
</action>
<kill name="killJob">
<message>"Killed job due to error: ${wf:errorMessage(wf:lastErrorNode())}"</message>
</kill>
<end name="end" />
</workflow-app>
this always throwing sqoopmain class exception which is finally treating as Error:
#sqoopAction] ERROR is considered as FAILED for SLA
In some stack question i found that external library is need to be included with <file>
so i changed my workflow.xml and add a line below <command> tag:
<file>${oozieLibPath}/sqoop/mysql-connector-java-5.1.37-bin.jar#mysql-connector-java-5.1.37-bin.jar</file>
But still same error is producing what may be the reason for this error. is there any thing else i need to configure?