Apache Ignite Cache put fails with in the IgniteRunnable run method - ignite

I am getting the below error during the igniteCache.put() in the IgniteRunnable run()
I have only 2 nodes (client and server) .
1) Client creates the cache
CacheConfiguration<Integer, LAttribute> cfg = new CacheConfiguration<Integer, LAttribute>();
cfg.setIndexedTypes(Integer.class, LoanAttribute.class);
cfg.setCacheMode(CacheMode.PARTITIONED);
cfg.setName("inv_result");
cfg.setCopyOnRead(false);
cfg.setAtomicityMode(CacheAtomicityMode.ATOMIC);
2) Client Submit the IgniteRunnable task to Server
3) Client exit the cluster
On the Server (with in run() method)
1) Get the cache and put a value
IgniteCache<Integer, LAttribute> iCache = Ignition.localIgnite().cache("inv_result");
System.out.println("Begin .. "+iCache.size(CachePeekMode.ALL));
iCache.put(la.getId(), la);
Error :
[21:41:14,859][SEVERE][pub-#67%null%][GridJobWorker] Failed to execute job due to unexpected runtime exception [jobId=f4606f39b51-21c994a7-6b35-49fa-b696-582fa7825c31, ses=GridJobSessionImpl [ses=GridTaskSessionImpl [taskName=com.test.ignite.compute.AssetRestrictionComputeJob, dep=GridDeployment [ts=1492836063447, depMode=SHARED, clsLdr=sun.misc.Launcher$AppClassLoader#73d16e93, clsLdrId=438a5f39b51-76a937b0-7831-458b-aee4-cec662f02b0d, userVer=0, loc=true, sampleClsName=o.a.i.i.processors.cache.distributed.dht.preloader.GridDhtPartitionMap2, pendingUndeploy=false, undeployed=false, usage=1], taskClsName=com.bfm.seclending.ignite.compute.AssetRestrictionComputeJob, sesId=c4606f39b51-21c994a7-6b35-49fa-b696-582fa7825c31, startTime=1492836072790, endTime=9223372036854775807, taskNodeId=21c994a7-6b35-49fa-b696-582fa7825c31, clsLdr=sun.misc.Launcher$AppClassLoader#73d16e93, closed=false, cpSpi=null, failSpi=null, loadSpi=null, usage=1, fullSup=false, internal=false, subjId=21c994a7-6b35-49fa-b696-582fa7825c31, mapFut=IgniteFuture [orig=GridFutureAdapter [resFlag=0, res=null, startTime=1492836072829, endTime=0, ignoreInterrupts=false, state=INIT]]], jobId=f4606f39b51-21c994a7-6b35-49fa-b696-582fa7825c31]]
javax.cache.CacheException: class org.apache.ignite.IgniteInterruptedException: Failed to wait for asynchronous operation permit (thread got interrupted).
at org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1440)
at org.apache.ignite.internal.processors.cache.IgniteCacheProxy.cacheException(IgniteCacheProxy.java:2183)
at org.apache.ignite.internal.processors.cache.IgniteCacheProxy.put(IgniteCacheProxy.java:1383)
at co.test.ignite.compute.AssetRestrictionComputeJob.run(AssetRestrictionComputeJob.java:110)
at org.apache.ignite.internal.processors.closure.GridClosureProcessor$C4V2.execute(GridClosureProcessor.java:2215)
at org.apache.ignite.internal.processors.job.GridJobWorker$2.call(GridJobWorker.java:556)
at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6564)
at org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:550)
at org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:479)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1180)
at org.apache.ignite.internal.processors.job.GridJobProcessor$JobExecutionListener.onMessage(GridJobProcessor.java:1894)
at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1082)
at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:710)
at org.apache.ignite.internal.managers.communication.GridIoManager.access$1700(GridIoManager.java:102)
at org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:673)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: class org.apache.ignite.IgniteInterruptedException: Failed to wait for asynchronous operation permit (thread got interrupted).
at org.apache.ignite.internal.util.IgniteUtils$3.apply(IgniteUtils.java:766)
at org.apache.ignite.internal.util.IgniteUtils$3.apply(IgniteUtils.java:764)
... 19 more
Caused by: java.lang.InterruptedException
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
at java.util.concurrent.Semaphore.acquire(Semaphore.java:312)
at org.apache.ignite.internal.processors.cache.GridCacheAdapter.asyncOpAcquire(GridCacheAdapter.java:4597)
at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.asyncOp(GridDhtAtomicCache.java:817)
at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAsync0(GridDhtAtomicCache.java:1148)
at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.putAsync0(GridDhtAtomicCache.java:618)
at org.apache.ignite.internal.processors.cache.GridCacheAdapter.putAsync(GridCacheAdapter.java:2541)
at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.put(GridDhtAtomicCache.java:595)
at org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2215)
at org.apache.ignite.internal.processors.cache.IgniteCacheProxy.put(IgniteCacheProxy.java:1376)
... 16 more


Most likely server node was stopped in the middle of execution. That's the only case when internal Ignite threads are interrupted. When this happens, job can be automatically failed over to another node: https://apacheignite.readme.io/docs/fault-tolerance


I found the reason for the InterruptedException , executionService that i am using on the client side to submit the jobs is not waiting for the job completion .
when i call future.get() .. all good now. Thanks

Related

Why is ignite failing to deserialize a GridClosureProcessor object?

I know a similar stack has been posted by others. My question is "it looks like Ignite is trying to deserialize a GridClosureProcessor (or a closure from it?) . If so, why is it doing that? I am trying to root cause this issue but none of my code is in the stack except MyCallable mentioned at the top (not actually in the stack).
No cache puts are happening in this code path unless they are internal. I mention this because of a comment on another post that "Unknown pair" can be caused by a cache put of the wrong type.
I am focusing on
Failed to deserialize object
[typeName=org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2]
Here's the rest.
[2019-04-08 22:20:23,724][ERROR][pub-#63][GridJobWorker] Failed to
initialize job
[jobId=800890ff961-7ff6a786-9d4d-43d8-91a0-70225c5e3a4a,
ses=GridJobSessionImpl [ses=GridTaskSessionImpl
[taskName=com.obfucorp.aa.project.core.jobs.MyCallable,
dep=GridDeployment [ts=1554761996013, depMode=SHARED,
clsLdr=sun.misc.Launcher$AppClassLoader#764c12b6,
clsLdrId=730290ff961-8d93b961-09f2-48c3-bd2f-49db31aae61e, userVer=0,
loc=true,
sampleClsName=o.a.i.i.processors.cache.GridCacheProcessor$RemovedItemsCleanupTask$1,
pendingUndeploy=false, undeployed=false, usage=1],
taskClsName=com.obfucorp.aa.project.core.jobs.MyCallable,
sesId=700890ff961-7ff6a786-9d4d-43d8-91a0-70225c5e3a4a,
startTime=1554762023663, endTime=9223372036854775807,
taskNodeId=7ff6a786-9d4d-43d8-91a0-70225c5e3a4a,
clsLdr=sun.misc.Launcher$AppClassLoader#764c12b6, closed=false,
cpSpi=null, failSpi=null, loadSpi=null, usage=1, fullSup=false,
internal=false, topPred=null,
subjId=7ff6a786-9d4d-43d8-91a0-70225c5e3a4a, mapFut=IgniteFuture
[orig=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=314803578]], execName=null],
jobId=800890ff961-7ff6a786-9d4d-43d8-91a0-70225c5e3a4a]] class
org.apache.ignite.IgniteCheckedException: Failed to deserialize object
[typeName=org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2]
at
org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:9908)
at
org.apache.ignite.internal.processors.job.GridJobWorker.initialize(GridJobWorker.java:438)
at
org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1117)
at
org.apache.ignite.internal.processors.job.GridJobProcessor$JobExecutionListener.onMessage(GridJobProcessor.java:1921)
at
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1555)
at
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1183)
at
org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:126)
at
org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1090)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748) Caused by: class
org.apache.ignite.binary.BinaryObjectException: Failed to deserialize
object
[typeName=org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2]
at
org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:875)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1762)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1714)
at
org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:310)
at
org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:99)
at
org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
at
org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:9902)
... 10 more Caused by: class
org.apache.ignite.binary.BinaryInvalidTypeException: Unknown pair
[platformId=0, typeId=-1409390795] at
org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:696)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1755)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1714)
at
org.apache.ignite.internal.binary.BinaryUtils.doReadObject(BinaryUtils.java:1799)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.readObject(BinaryReaderExImpl.java:1329)
at
org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2.readBinary(GridClosureProcessor.java:1872)
at
org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:834)
... 16 more Caused by: java.lang.ClassNotFoundException: Unknown pair
[platformId=0, typeId=-1409390795] at
org.apache.ignite.internal.MarshallerContextImpl.getClassName(MarshallerContextImpl.java:385)
at
org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:335)
at
org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:687)
... 22 more
update - wanted to note this is happening a fresh deployment. There are no old files or persisted anything around. All classes are either pulled from even or freshly compiled. ty.
Pavel, here is the (Scala) code (redacted)
object MyCallable {
type FooList = Array[Foo]
}
class MyCallable(cacheName: String) extends IgniteCallable[FooList] with Serializable with LazyLogging {
#IgniteInstanceResource
private var ignite: Ignite = _
override def call(): FooList = {
logger.debug("callable called.");
val fooCache = ignite.getOrCreateCache[String, Foo](cacheName)
val qry = new ScanQuery[String, Foo]()
qry.setLocal(true)
val cursor = fooCache.query(qry)
val ret = cursor.iterator().asScala.map(e => e.getValue).toArray
logger.info("load status array: {}", ret.mkString)
return ret
}
#IgniteInstanceResource
def setIgnite(ignite: Ignite): Unit = {
this.ignite = ignite
}
}

Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException: Unknown pair [platformId=0, typeId=-1409390795]
Seems like you have lost your marshaller cache (marshaller/ dir).
You can do ignite.marshaller().marshal(new WhateverTypeIsCausingThis()); once to make this error go away.

Missing header 'amqp_consumerQueue' from RabbitMQ message

We have an application that reads RabbitMQ messages from a number of queues using Spring AMPQ and a RabbitListener. Something like this
#RabbitListener(queues = {"#{'${rabbit.queues}'.split(',')}"})
public void processRabbitMessage(#Payload String data, #Header(AmqpHeaders.CONSUMER_QUEUE) String queue, #Header(AmqpHeaders.MESSAGE_ID) String messageId) throws Exception {
// Do some stuff
}
However I'm getting an intermittent error where the message header isn't set.
org.springframework.amqp.rabbit.listener.exception.ListenerExecutionFailedException: Listener method could not be invoked with the incoming message
Endpoint handler details:
Method [public void com.service.RabbitService.processRabbitMessage(java.lang.String,java.lang.String,java.lang.String) throws java.lang.Exception]
Bean [com.service.RabbitService#5b7a7f33]
at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.invokeHandler(MessagingMessageListenerAdapter.java:135)
at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.onMessage(MessagingMessageListenerAdapter.java:106)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:822)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:745)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.access$001(SimpleMessageListenerContainer.java:97)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$1.invokeListener(SimpleMessageListenerContainer.java:189)
at sun.reflect.GeneratedMethodAccessor90.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.retry.interceptor.RetryOperationsInterceptor$1.doWithRetry(RetryOperationsInterceptor.java:91)
at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:286)
at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:179)
at org.springframework.retry.interceptor.RetryOperationsInterceptor.invoke(RetryOperationsInterceptor.java:115)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
at com.sun.proxy.$Proxy110.invokeListener(Unknown Source)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.invokeListener(SimpleMessageListenerContainer.java:1276)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.executeListener(AbstractMessageListenerContainer.java:726)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.doReceiveAndExecute(SimpleMessageListenerContainer.java:1219)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.receiveAndExecute(SimpleMessageListenerContainer.java:1189)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.access$1500(SimpleMessageListenerContainer.java:97)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1421)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.springframework.messaging.MessageHandlingException: Missing header 'amqp_consumerQueue' for method parameter type [class java.lang.String]
at org.springframework.messaging.handler.annotation.support.HeaderMethodArgumentResolver.handleMissingValue(HeaderMethodArgumentResolver.java:100)
at org.springframework.messaging.handler.annotation.support.AbstractNamedValueMethodArgumentResolver.resolveArgument(AbstractNamedValueMethodArgumentResolver.java:103)
at org.springframework.messaging.handler.invocation.HandlerMethodArgumentResolverComposite.resolveArgument(HandlerMethodArgumentResolverComposite.java:112)
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.getMethodArgumentValues(InvocableHandlerMethod.java:135)
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:107)
at org.springframework.amqp.rabbit.listener.adapter.HandlerAdapter.invoke(HandlerAdapter.java:49)
at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.invokeHandler(MessagingMessageListenerAdapter.java:126)
... 25 common frames omitted
All the messages are written to the queues in exactly the same way (as below) and come from the same source so are generally the same structure.
MessageProperties msgprop = new MessageProperties()
msgprop.setContentType(MessageProperties.CONTENT_TYPE_TEXT_PLAIN)
msgprop.setAppId(routing)
rabbitTemplate.send(exchange, routing, new Message(message.getBytes(), msgprop))
We're not loosing data since we can catch all the rejected messages in a DLX but it's difficult to reprocess when we don't know what queue they came from.
We know that none of the messages are null/malformed (even the ones generating errors) and I was under the impression that this was a header set by Rabbit based on the queue a message is written to so how can it be null?
Any ideas as to how this header is null for some and not all messages and/or a possible fix to insure the headers are set?
EDIT: Added some DEBUG logs of the last attempt at reading in the message. We have a stateless retry policy in place so it's tried twice already.
00:46:17.577 [SimpleAsyncTaskExecutor-1051] DEBUG o.s.retry.support.RetryTemplate - Checking for rethrow: count=3
00:46:17.577 [SimpleAsyncTaskExecutor-1051] DEBUG o.s.retry.support.RetryTemplate - Retry failed last attempt: count=3
00:46:17.577 [SimpleAsyncTaskExecutor-1051] WARN o.s.a.r.r.RejectAndDontRequeueRecoverer - Retries exhausted for message (Body:'{Message:Body}' MessageProperties [headers={}, timestamp=null, messageId=a.b-209491726, userId=null, receivedUserId=null, appId=a.b, clusterId=null, type=null, correlationId=null, correlationIdString=null, replyTo=null, contentType=text/plain, contentEncoding=null, contentLength=0, deliveryMode=null, receivedDeliveryMode=PERSISTENT, expiration=null, priority=0, redelivered=false, receivedExchange=exch, receivedRoutingKey=a.b, receivedDelay=null, deliveryTag=561, messageCount=0, consumerTag=amq.ctag-fpVPl0EjHwdKew6eTEMoWA, consumerQueue=null])org.springframework.amqp.rabbit.listener.exception.ListenerExecutionFailedException: Listener method could not be invoked with the incoming message Endpoint handler details:
Method [public void com.service.RabbitService.processRabbitMessage(java.lang.String,java.lang.String,java.lang.String) throws java.lang.Exception]Bean [com.service.RabbitService#5b7a7f33]
at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.invokeHandler(MessagingMessageListenerAdapter.java:135)
at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.onMessage(MessagingMessageListenerAdapter.java:106)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:822)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:745)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.access$001(SimpleMessageListenerContainer.java:97)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$1.invokeListener(SimpleMessageListenerContainer.java:189)
at sun.reflect.GeneratedMethodAccessor90.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.retry.interceptor.RetryOperationsInterceptor$1.doWithRetry(RetryOperationsInterceptor.java:91)
at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:286)
at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:179)
at org.springframework.retry.interceptor.RetryOperationsInterceptor.invoke(RetryOperationsInterceptor.java:115)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
at com.sun.proxy.$Proxy110.invokeListener(Unknown Source)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.invokeListener(SimpleMessageListenerContainer.java:1276)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.executeListener(AbstractMessageListenerContainer.java:726)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.doReceiveAndExecute(SimpleMessageListenerContainer.java:1219)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.receiveAndExecute(SimpleMessageListenerContainer.java:1189)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.access$1500(SimpleMessageListenerContainer.java:97)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1421)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.springframework.messaging.MessageHandlingException: Missing header 'amqp_consumerQueue' for method parameter type [class java.lang.String]
at org.springframework.messaging.handler.annotation.support.HeaderMethodArgumentResolver.handleMissingValue(HeaderMethodArgumentResolver.java:100)
at org.springframework.messaging.handler.annotation.support.AbstractNamedValueMethodArgumentResolver.resolveArgument(AbstractNamedValueMethodArgumentResolver.java:103)
at org.springframework.messaging.handler.invocation.HandlerMethodArgumentResolverComposite.resolveArgument(HandlerMethodArgumentResolverComposite.java:112)
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.getMethodArgumentValues(InvocableHandlerMethod.java:135)
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:107)
at org.springframework.amqp.rabbit.listener.adapter.HandlerAdapter.invoke(HandlerAdapter.java:49)
at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.invokeHandler(MessagingMessageListenerAdapter.java:126)
... 25 common frames omitted
00:46:17.577 [SimpleAsyncTaskExecutor-1051] WARN o.s.a.r.l.ConditionalRejectingErrorHandler - Execution of Rabbit message listener failed.
org.springframework.amqp.rabbit.listener.exception.ListenerExecutionFailedException: Retry Policy Exhausted
at org.springframework.amqp.rabbit.retry.RejectAndDontRequeueRecoverer.recover(RejectAndDontRequeueRecoverer.java:45)
at org.springframework.amqp.rabbit.config.StatelessRetryOperationsInterceptorFactoryBean$1.recover(StatelessRetryOperationsInterceptorFactoryBean.java:66)
at org.springframework.amqp.rabbit.config.StatelessRetryOperationsInterceptorFactoryBean$1.recover(StatelessRetryOperationsInterceptorFactoryBean.java:59)
at org.springframework.retry.interceptor.RetryOperationsInterceptor$ItemRecovererCallback.recover(RetryOperationsInterceptor.java:141)
at org.springframework.retry.support.RetryTemplate.handleRetryExhausted(RetryTemplate.java:512)
at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:350)
at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:179)
at org.springframework.retry.interceptor.RetryOperationsInterceptor.invoke(RetryOperationsInterceptor.java:115)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
at com.sun.proxy.$Proxy110.invokeListener(Unknown Source)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.invokeListener(SimpleMessageListenerContainer.java:1276)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.executeListener(AbstractMessageListenerContainer.java:726)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.doReceiveAndExecute(SimpleMessageListenerContainer.java:1219)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.receiveAndExecute(SimpleMessageListenerContainer.java:1189)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.access$1500(SimpleMessageListenerContainer.java:97)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1421)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.springframework.amqp.AmqpRejectAndDontRequeueException: org.springframework.amqp.rabbit.listener.exception.ListenerExecutionFailedException: Listener method could not be invoked with the incoming message
Endpoint handler details:
Method [public void com.service.RabbitService.processRabbitMessage(java.lang.String,java.lang.String,java.lang.String) throws java.lang.Exception]
Bean [com.service.RabbitService#5b7a7f33]
... 18 common frames omitted
Caused by: org.springframework.amqp.rabbit.listener.exception.ListenerExecutionFailedException: Listener method could not be invoked with the incoming message
Endpoint handler details:
Method [public void com.service.RabbitService.processRabbitMessage(java.lang.String,java.lang.String,java.lang.String) throws java.lang.Exception]
Bean [com.service.RabbitService#5b7a7f33]
at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.invokeHandler(MessagingMessageListenerAdapter.java:135)
at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.onMessage(MessagingMessageListenerAdapter.java:106)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:822)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:745)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.access$001(SimpleMessageListenerContainer.java:97)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$1.invokeListener(SimpleMessageListenerContainer.java:189)
at sun.reflect.GeneratedMethodAccessor90.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.retry.interceptor.RetryOperationsInterceptor$1.doWithRetry(RetryOperationsInterceptor.java:91)
at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:286)
... 12 common frames omitted
Caused by: org.springframework.messaging.MessageHandlingException: Missing header 'amqp_consumerQueue' for method parameter type [class java.lang.String]
at org.springframework.messaging.handler.annotation.support.HeaderMethodArgumentResolver.handleMissingValue(HeaderMethodArgumentResolver.java:100)
at org.springframework.messaging.handler.annotation.support.AbstractNamedValueMethodArgumentResolver.resolveArgument(AbstractNamedValueMethodArgumentResolver.java:103)
at org.springframework.messaging.handler.invocation.HandlerMethodArgumentResolverComposite.resolveArgument(HandlerMethodArgumentResolverComposite.java:112)
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.getMethodArgumentValues(InvocableHandlerMethod.java:135)
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:107)
at org.springframework.amqp.rabbit.listener.adapter.HandlerAdapter.invoke(HandlerAdapter.java:49)
at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.invokeHandler(MessagingMessageListenerAdapter.java:126)
... 25 common frames omitted
00:46:17.577 [SimpleAsyncTaskExecutor-1051] DEBUG o.s.a.r.l.BlockingQueueConsumer - Rejecting messages (requeue=false)
00:46:17.577 [SimpleAsyncTaskExecutor-1051] DEBUG o.s.a.r.l.SimpleMessageListenerContainer - Cancelling Consumer#2f48a506: tags=[{}], channel=Cached Rabbit Channel: AMQChannel(amqp://exch#1.1.1.1//exch,1055), conn: Proxy#79239726 Shared Rabbit Connection: SimpleConnection#7d1c1c5d [delegate=amqp://exch#10.136.229.112:5672//exch, localPort= 45298], acknowledgeMode=AUTO local queue size=0
00:46:17.577 [SimpleAsyncTaskExecutor-1051] DEBUG o.s.a.r.l.BlockingQueueConsumer - Closing Rabbit Channel: Cached Rabbit Channel: AMQChannel(amqp://exch#10.136.229.112:5672//exch,1055), conn: Proxy#79239726 Shared Rabbit Connection: SimpleConnection#7d1c1c5d [delegate=amqp://exch#10.136.229.112:5672//exch, localPort= 45298]

I don't see how that's possible; we unconditionally set the consumerQueue property in the received message - which is the source for that header. We maintain a map of consumerTag -> queue name for this purpose.
A DEBUG log for a message delivery that exhibits this behavior would be useful.

Ignite Error : Failed to finish operation (too many remaps): 32

I am loading data from kafka to ignite data streamers in parallel in multiple ignite nodes. But when I kill one of these nodes(for checking fault tolerance), I am getting following error. Some Messages are also getting lost.
class org.apache.ignite.IgniteCheckedException: Failed to finish operation (too many remaps): 32
at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$5.apply(DataStreamerImpl.java:863)
at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$5.apply(DataStreamerImpl.java:828)
at org.apache.ignite.internal.util.future.GridFutureAdapter$ArrayListener.apply(GridFutureAdapter.java:456)
at org.apache.ignite.internal.util.future.GridFutureAdapter$ArrayListener.apply(GridFutureAdapter.java:439)
at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:271)
at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListeners(GridFutureAdapter.java:259)
at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:389)
at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:355)
at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:343)
at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$Buffer.submit(DataStreamerImpl.java:1716)
at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$Buffer.update(DataStreamerImpl.java:1416)
at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.load0(DataStreamerImpl.java:932)
at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addData(DataStreamerImpl.java:576)
at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addData(DataStreamerImpl.java:544)
at org.apache.ignite.stream.StreamAdapter.addMessage(StreamAdapter.java:184)
at org.apache.ignite.stream.kafka.KafkaStreamer.access$100(KafkaStreamer.java:47)
at org.apache.ignite.stream.kafka.KafkaStreamer$1.run(KafkaStreamer.java:156)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: class org.apache.ignite.internal.cluster.ClusterTopologyCheckedException: Failed to send request (node has left): 0a04411f-a797-4526-826a-96c52549b0d0
... 11 more
[21:45:24] (err) Failed to execute compound future reducer: GridCompoundFuture [rdc=null, initFlag=1, lsnrCalls=1, done=false, cancelled=false, err=null, futs=[false, false, true, false, false, true, false, false]]class org.apache.ignite.IgniteCheckedException: DataStreamer request failed [node=f48edaa8-74df-4503-bf9b-1fdc6d642744]
at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$Buffer.onResponse(DataStreamerImpl.java:1777)
at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$3.onMessage(DataStreamerImpl.java:335)
at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1215)
at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:843)
at org.apache.ignite.internal.managers.communication.GridIoManager.access$2100(GridIoManager.java:108)
at org.apache.ignite.internal.managers.communication.GridIoManager$6.run(GridIoManager.java:783)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Calling remotely tasks deployed via a GAR file

I use UriDeploymentSpi bean to load GAR files from a directory in one of my nodes
I have following GAR ignite.xml file (took me a while to figure this one out btw, nowhere documented?)
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:util="http://www.springframework.org/schema/util"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util-2.5.xsd">
<util:list id="myList" value-type="java.lang.String">
<value>myproject.HelloWorldTask</value>
<value>myproject.SimpleTask</value>
</util:list>
</beans>
HelloWorldTask:
package myproject;
public class HelloWorldTask extends ComputeTaskAdapter<String, Integer> {
static {
System.out.println("TheGlue: Loading HelloWorldTask ");
}
public HelloWorldTask() {
}
#Nullable
#Override
public Map<? extends ComputeJob, ClusterNode> map(List<ClusterNode> nodes, #Nullable String arg) throws IgniteException {
System.out.println("Hello from GAR file");
return null; //To change body of implemented methods use File | Settings | File Templates.
}
#Nullable
#Override
public Integer reduce(List<ComputeJobResult> results) throws IgniteException {
return null; //To change body of implemented methods use File | Settings | File Templates.
}
}
SimpleTask:
package myproject;
#ComputeTaskName("SimpleTaskName")
public class SimpleTask implements ComputeTask<String, Integer> {
static {
System.out.println("Loading SimpleTask");
}
public SimpleTask() {
}
#Override
public Map<? extends ComputeJob, ClusterNode> map(List<ClusterNode> subgrid, String arg) throws IgniteException {
System.out.println("Computing Job in SimpleTask ");
return null; //To change body of implemented methods use File | Settings | File Templates.
}
#Override
public ComputeJobResultPolicy result(ComputeJobResult res, List<ComputeJobResult> rcvd) throws IgniteException {
return null; //To change body of implemented methods use File | Settings | File Templates.
}
#Override
public Integer reduce(List<ComputeJobResult> results) throws IgniteException {
return null; //To change body of implemented methods use File | Settings | File Templates.
}
}
The 2 classes can be found by Ignite (debugged through GridUriDeploymentSpringDocument and GridUriDeploymentFileProcessor and they are found and loaded). Ignite says that it found the GAR, but as far as I can see, the classes are not instantiated. No errors in the log files, no indications that the Tasks are deployed either.
I am trying to execute the following code on a node where the GAR file is not deployed (ie. client node of the cluster), but the Task is not executed on the cluster:
public class _03GarTest {
public static void main(String[] args) {
System.out.println("Start urideployment test");
IgniteConfiguration cfg = new IgniteConfiguration();
cfg.setPeerClassLoadingEnabled(true); //needs to be the same as in the XML for the server
cfg.setClientMode(true);
try(Ignite ignite = Ignition.start(cfg)) {
ignite.compute(ignite.cluster().forRemotes()).execute("SimpleTaskName", null);
}
}
}
Log file where I execute the _03GarTest class (same if I run with "SimpleTaskName" or "myproject.SimpleTaskName"), dumps the following stacktraces on the client node:
Exception in thread "main" class org.apache.ignite.IgniteDeploymentException: Unknown task name or failed to auto-deploy task (was task (re|un)deployed?): SimpleTaskName
at org.apache.ignite.internal.util.IgniteUtils$7.apply(IgniteUtils.java:761)
at org.apache.ignite.internal.util.IgniteUtils$7.apply(IgniteUtils.java:759)
at org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:877)
at org.apache.ignite.internal.IgniteComputeImpl.execute(IgniteComputeImpl.java:154)
at _03GarTest.main(_03GarTest.java:55)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
Caused by: class org.apache.ignite.internal.IgniteDeploymentCheckedException: Unknown task name or failed to auto-deploy task (was task (re|un)deployed?): SimpleTaskName
at org.apache.ignite.internal.processors.task.GridTaskProcessor.startTask(GridTaskProcessor.java:515)
at org.apache.ignite.internal.processors.task.GridTaskProcessor.execute(GridTaskProcessor.java:447)
at org.apache.ignite.internal.IgniteComputeImpl.execute(IgniteComputeImpl.java:151)
... 6 more
And on the server, following logs are produced:
[13:13:33,057][INFO][disco-event-worker-#48%null%][GridDiscoveryManager] Added new node to topology: TcpDiscoveryNode [id=b70dce5e-c0fd-4ffe-8dc2-b72b18db76da, addrs=[0:0:0:0:0:0:0:1, 10.1.26.59, 127.0.0.1, 192.168.8.103, 192.168.99.1], sockAddrs=[/192.168.8.103:0, /0:0:0:0:0:0:0:1:0, /192.168.99.1:0, /10.1.26.59:0, /10.1.26.59:0, /127.0.0.1:0, /192.168.8.103:0, /192.168.99.1:0], discPort=0, order=12, intOrder=7, lastExchangeTime=1452600812926, loc=false, ver=1.5.0#20151229-sha1:f1f8cda2, isClient=true]
[13:13:33,063][INFO][disco-event-worker-#48%null%][GridDiscoveryManager] Topology snapshot [ver=12, servers=1, clients=1, CPUs=8, heap=1.5GB]
[13:13:33,085][WARNING][disco-event-worker-#48%null%][CourtesyConfigNotice]
>>> +-------------------------------------------------------------------+
>>> + Courtesy notice that joining node has inconsistent configuration. +
>>> + Ignore this message if you are sure that this is done on purpose. +
>>> +-------------------------------------------------------------------+
>>> Remote Node ID: B70DCE5E-C0FD-4FFE-8DC2-B72B18DB76DA
>>> Remote SPI with the same name is not configured: UriDeploymentSpi
>>> => Local node: o.a.i.spi.deployment.uri.UriDeploymentSpi
[13:13:33,103][INFO][exchange-worker-#51%null%][GridCachePartitionExchangeManager] Skipping rebalancing (nothing scheduled) [top=AffinityTopologyVersion [topVer=12, minorTopVer=0], evt=NODE_JOINED, node=b70dce5e-c0fd-4ffe-8dc2-b72b18db76da]
[13:13:33,907][INFO][disco-event-worker-#48%null%][GridDiscoveryManager] Node left topology: TcpDiscoveryNode [id=b70dce5e-c0fd-4ffe-8dc2-b72b18db76da, addrs=[0:0:0:0:0:0:0:1, 10.1.26.59, 127.0.0.1, 192.168.8.103, 192.168.99.1], sockAddrs=[/192.168.8.103:0, /0:0:0:0:0:0:0:1:0, /192.168.99.1:0, /10.1.26.59:0, /10.1.26.59:0, /127.0.0.1:0, /192.168.8.103:0, /192.168.99.1:0], discPort=0, order=12, intOrder=7, lastExchangeTime=1452600812926, loc=false, ver=1.5.0#20151229-sha1:f1f8cda2, isClient=true]
[13:13:33,908][INFO][disco-event-worker-#48%null%][GridDiscoveryManager] Topology snapshot [ver=13, servers=1, clients=0, CPUs=8, heap=1.0GB]
[13:13:33,918][INFO][exchange-worker-#51%null%][GridCachePartitionExchangeManager] Skipping rebalancing (nothing scheduled) [top=AffinityTopologyVersion [topVer=13, minorTopVer=0], evt=NODE_LEFT, node=b70dce5e-c0fd-4ffe-8dc2-b72b18db76da]
[13:14:03,193][INFO][grid-timeout-worker-#33%null%][IgniteKernal]
Any ideas on how to call a task deployed via a GAR file on another node?
----UPDATE----
As suggested in one of the answers, I have added the following code in the client
System.out.println("Start urideployment test");
IgniteConfiguration cfg = new IgniteConfiguration();
cfg.setPeerClassLoadingEnabled(true); //needs to be the same as in the XML for the server
cfg.setClientMode(true);
UriDeploymentSpi deploymentSpi = new UriDeploymentSpi();
deploymentSpi.setUriList(Arrays.asList("file:///Users/sbeaupre/Dropbox/prorabel/Projects/IgniteTests/ignite/gar"));
cfg.setDeploymentSpi(deploymentSpi);
try(Ignite ignite = Ignition.start(cfg)) {
...
But this doesn't work either, I got following stack trace on the client node and nothing on the server node:
Jan 14, 2016 5:42:23 PM org.apache.ignite.logger.java.JavaLogger info
INFO: Topology snapshot [ver=4, servers=1, clients=1, CPUs=8, heap=1.5GB]
Jan 14, 2016 5:42:23 PM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions
INFO: Loading XML bean definitions from resource loaded from byte array
Jan 14, 2016 5:42:23 PM org.apache.ignite.logger.java.JavaLogger info
INFO: User version is not explicitly defined (will use default version) [file=META-INF/ignite.xml, clsLdr=GridUriDeploymentClassLoader [urls=[file:/var/folders/t3/595tz_px2j9__wl37f0b5nw40000gn/T/gg.uri.deployment.tmp/301a4cb8-6fc7-4aa9-b050-3083183f4cd0/dirzip_Archive8035449106801616883.gar/]]]
Jan 14, 2016 5:42:23 PM org.apache.ignite.logger.java.JavaLogger info
INFO: Task locally deployed: class myproject.SimpleTask
Loading SimpleTask
Computing Job in SimpleTask
Jan 14, 2016 5:42:23 PM org.apache.ignite.logger.java.JavaLogger error
SEVERE: Failed to map task jobs to nodes: GridTaskSessionImpl [taskName=SimpleTaskName, dep=GridDeployment [ts=1452789743727, depMode=SHARED, clsLdr=GridUriDeploymentClassLoader [urls=[file:/var/folders/t3/595tz_px2j9__wl37f0b5nw40000gn/T/gg.uri.deployment.tmp/301a4cb8-6fc7-4aa9-b050-3083183f4cd0/dirzip_Archive8035449106801616883.gar/]], clsLdrId=cc234014251-301a4cb8-6fc7-4aa9-b050-3083183f4cd0, userVer=0, loc=true, sampleClsName=myproject.SimpleTask, pendingUndeploy=false, undeployed=false, usage=1], taskClsName=myproject.SimpleTask, sesId=bc234014251-301a4cb8-6fc7-4aa9-b050-3083183f4cd0, startTime=1452789743638, endTime=9223372036854775807, taskNodeId=301a4cb8-6fc7-4aa9-b050-3083183f4cd0, clsLdr=GridUriDeploymentClassLoader [urls=[file:/var/folders/t3/595tz_px2j9__wl37f0b5nw40000gn/T/gg.uri.deployment.tmp/301a4cb8-6fc7-4aa9-b050-3083183f4cd0/dirzip_Archive8035449106801616883.gar/]], closed=false, cpSpi=null, failSpi=null, loadSpi=null, usage=1, fullSup=false, subjId=301a4cb8-6fc7-4aa9-b050-3083183f4cd0, mapFut=IgniteFuture [orig=GridFutureAdapter [resFlag=0, res=null, startTime=1452789743739, endTime=0, ignoreInterrupts=false, lsnr=null, state=INIT]]]
class org.apache.ignite.IgniteCheckedException: Task map operation produced no mapped jobs: GridTaskSessionImpl [taskName=SimpleTaskName, dep=GridDeployment [ts=1452789743727, depMode=SHARED, clsLdr=GridUriDeploymentClassLoader [urls=[file:/var/folders/t3/595tz_px2j9__wl37f0b5nw40000gn/T/gg.uri.deployment.tmp/301a4cb8-6fc7-4aa9-b050-3083183f4cd0/dirzip_Archive8035449106801616883.gar/]], clsLdrId=cc234014251-301a4cb8-6fc7-4aa9-b050-3083183f4cd0, userVer=0, loc=true, sampleClsName=myproject.SimpleTask, pendingUndeploy=false, undeployed=false, usage=1], taskClsName=myproject.SimpleTask, sesId=bc234014251-301a4cb8-6fc7-4aa9-b050-3083183f4cd0, startTime=1452789743638, endTime=9223372036854775807, taskNodeId=301a4cb8-6fc7-4aa9-b050-3083183f4cd0, clsLdr=GridUriDeploymentClassLoader [urls=[file:/var/folders/t3/595tz_px2j9__wl37f0b5nw40000gn/T/gg.uri.deployment.tmp/301a4cb8-6fc7-4aa9-b050-3083183f4cd0/dirzip_Archive8035449106801616883.gar/]], closed=false, cpSpi=null, failSpi=null, loadSpi=null, usage=1, fullSup=false, subjId=301a4cb8-6fc7-4aa9-b050-3083183f4cd0, mapFut=IgniteFuture [orig=GridFutureAdapter [resFlag=0, res=null, startTime=1452789743739, endTime=0, ignoreInterrupts=false, lsnr=null, state=INIT]]]
at org.apache.ignite.internal.processors.task.GridTaskWorker.body(GridTaskWorker.java:497)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at org.apache.ignite.internal.processors.task.GridTaskProcessor.startTask(GridTaskProcessor.java:678)
at org.apache.ignite.internal.processors.task.GridTaskProcessor.execute(GridTaskProcessor.java:447)
at org.apache.ignite.internal.IgniteComputeImpl.execute(IgniteComputeImpl.java:151)
at _03GarTest.main(_03GarTest.java:55)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)

Sven,
You should configure URI deployment SPI on client node as well to make GAR deployment work properly.
When you call compute.execute("taskName"); a lot of things have to be done locally on client prior to first request is sent to any of the node in your topology and after results start coming back. At least, Ignite should be able to get mapped jobs and be able to process results from all remote jobs and reduce all the results - please see ComputeTask.map() and ComputeTask.result() and ComputeTask.reduce(). So, you should be able to instantiate task on client node and that is why you should have task classes available.
I think after you configure URI deployment on client nodes you should have your code work fine.
Please post a comment here if you need any additional info.
Thanks!
UPDATE Jan, 18 2016
This is update in response to question update.
Please note that task in question returns null from map() method which is illegal. You can refer to org.apache.ignite.examples.computegrid.ComputeTaskMapExample in binary release or directly via https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=blob;f=examples/src/main/java/org/apache/ignite/examples/computegrid/ComputeTaskMapExample.java;h=3de5293a814e527b57e3984f6d3ab96bb1b62daf;hb=HEAD

Unable to deploy ear which uses Hibernate 3.6 on Wildfly8.1

I have been trying to deploy an application which uses Hibernate 3.6, on Wildfly8.1.
My application is an EAR, which consists of a war and ejb.jar.
After trying out several other things, I found this solution which seemed promising:
https://developer.jboss.org/message/882769
As per this solution, I have specified the correct provider for hibernate 3.6 and included all the jar files needed in the lib directory, but this does not work!
I also found this link to be helpful:https://docs.jboss.org/author/display/WFLY8/JPA+Reference+Guide?_sscc=t#JPAReferenceGuide-PackagingtheHibernate3.5orgreater3.xJPApersistenceproviderwithyourapplication
Any help would be greatly appreciated!!
Please find the error in the server log below:
ERROR [org.jboss.msc.service.fail] (MSC service thread 1-8) MSC000001: Failed to start service jboss.deployment.unit."xyz.ear".FIRST_MODULE_USE: org.jboss.msc.service.StartException in service jboss.deployment.unit."xyz.ear".FIRST_MODULE_USE: JBAS018733: Failed to process phase FIRST_MODULE_USE of deployment "xyz.ear"
at org.jboss.as.server.deployment.DeploymentUnitPhaseService.start(DeploymentUnitPhaseService.java:166) [wildfly-server-8.1.0.Final.jar:8.1.0.Final]
at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1948) [jboss-msc-1.2.2.Final.jar:1.2.2.Final]
at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1881) [jboss-msc-1.2.2.Final.jar:1.2.2.Final]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_71]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_71]
at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_71]
Caused by: java.lang.LinkageError: Failed to link org/hibernate/ejb/HibernatePersistence (Module "deployment.xyz.ear:main" from Service Module Loader)
at org.jboss.modules.ModuleClassLoader.defineClass(ModuleClassLoader.java:487) [jboss-modules.jar:1.3.3.Final]
at org.jboss.modules.ModuleClassLoader.loadClassLocal(ModuleClassLoader.java:277) [jboss-modules.jar:1.3.3.Final]
at org.jboss.modules.ModuleClassLoader$1.loadClassLocal(ModuleClassLoader.java:92) [jboss-modules.jar:1.3.3.Final]
at org.jboss.modules.Module.loadModuleClass(Module.java:568) [jboss-modules.jar:1.3.3.Final]
at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:205) [jboss-modules.jar:1.3.3.Final]
at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:459) [jboss-modules.jar:1.3.3.Final]
at org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:408) [jboss-modules.jar:1.3.3.Final]
at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:389) [jboss-modules.jar:1.3.3.Final]
at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:134) [jboss-modules.jar:1.3.3.Final]
at org.jboss.as.jpa.processor.PersistenceProviderHandler.deploy(PersistenceProviderHandler.java:74)
at org.jboss.as.jpa.processor.PersistenceBeginInstallProcessor.deploy(PersistenceBeginInstallProcessor.java:49)
at org.jboss.as.server.deployment.DeploymentUnitPhaseService.start(DeploymentUnitPhaseService.java:159) [wildfly-server-8.1.0.Final.jar:8.1.0.Final]
... 5 more
Caused by: java.lang.NoClassDefFoundError: javax/persistence/spi/PersistenceProvider
at java.lang.ClassLoader.defineClass1(Native Method) [rt.jar:1.7.0_71]
at java.lang.ClassLoader.defineClass(ClassLoader.java:800) [rt.jar:1.7.0_71]
at org.jboss.modules.ModuleClassLoader.doDefineOrLoadClass(ModuleClassLoader.java:361) [jboss-modules.jar:1.3.3.Final]
at org.jboss.modules.ModuleClassLoader.defineClass(ModuleClassLoader.java:482) [jboss-modules.jar:1.3.3.Final]
... 16 more
Caused by: java.lang.ClassNotFoundException: javax.persistence.spi.PersistenceProvider from [Module "deployment.xyz.ear:main" from Service Module Loader]
at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:213) [jboss-modules.jar:1.3.3.Final]
at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:459) [jboss-modules.jar:1.3.3.Final]
at org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:408) [jboss-modules.jar:1.3.3.Final]
at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:389) [jboss-modules.jar:1.3.3.Final]
at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:134) [jboss-modules.jar:1.3.3.Final]
... 20 more
15:12:01,051 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) JBAS014613: Operation ("add") failed - address: ([("subsystem" => "jsr77")]) - failure description: {"JBAS014879: One or more services were unable to start due to one or more indirect dependencies not being available." => {
"Services that were unable to start:" => ["jboss.jsr77.ejb"],
"Services that may be the cause:" => [
"jboss.io.worker.default",
"jboss.remoting.remotingConnectorInfoService.remoting-connector",
"jboss.web.common.server"
]
}}
15:12:01,122 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) JBAS014613: Operation ("add") failed - address: ([("subsystem" => "remoting")]) - failure description: {"JBAS014771: Services with missing/unavailable dependencies" => ["jboss.remoting.endpoint.subsystem is missing [jboss.io.worker.default]"]}
15:12:01,135 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) JBAS014613: Operation ("deploy") failed - address: ([("deployment" => "xyz.ear")]) - failure description: {
"JBAS014671: Failed services" => {"jboss.deployment.unit.\"xyz.ear\".FIRST_MODULE_USE" => "org.jboss.msc.service.StartException in service jboss.deployment.unit.\"xyz.ear\".FIRST_MODULE_USE: JBAS018733: Failed to process phase FIRST_MODULE_USE of deployment \"xyz.ear\"
Caused by: java.lang.LinkageError: Failed to link org/hibernate/ejb/HibernatePersistence (Module \"deployment.xyz.ear:main\" from Service Module Loader)
Caused by: java.lang.NoClassDefFoundError: javax/persistence/spi/PersistenceProvider
Caused by: java.lang.ClassNotFoundException: javax.persistence.spi.PersistenceProvider from [Module \"deployment.xyz.ear:main\" from Service Module Loader]"},
"JBAS014879: One or more services were unable to start due to one or more indirect dependencies not being available." => {
"Services that were unable to start:" => [
"jboss.deployment.subunit.\"xyz.ear\".\"xyz-ejb.jar\".FIRST_MODULE_USE",
"jboss.deployment.subunit.\"xyz.ear\".\"xyz.war\".FIRST_MODULE_USE"
],
"Services that may be the cause:" => [
"jboss.io.worker.default",
"jboss.remoting.remotingConnectorInfoService.remoting-connector",
"jboss.web.common.server"
]
}
}
15:12:01,436 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) JBAS014613: Operation ("add") failed - address: ([("subsystem" => "webservices")]) - failure description: {"JBAS014771: Services with missing/unavailable dependencies" => ["jboss.ws.config is missing [jboss.web.common.server]"]}
15:12:01,450 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) JBAS014613: Operation ("add") failed - address: ([
("subsystem" => "naming"),
("service" => "remote-naming")
]) - failure description: {"JBAS014879: One or more services were unable to start due to one or more indirect dependencies not being available." => {
"Services that were unable to start:" => ["jboss.naming.remote"],
"Services that may be the cause:" => [
"jboss.io.worker.default",
"jboss.remoting.remotingConnectorInfoService.remoting-connector",
"jboss.web.common.server"
]
}}
15:12:01,940 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) JBAS014613: Operation ("add") failed - address: ([
("subsystem" => "ejb3"),
("service" => "remote")
]) - failure description: {"JBAS014771: Services with missing/unavailable dependencies" => ["jboss.ejb3.connector is missing [jboss.remoting.remotingConnectorInfoService.remoting-connector]"]}
15:12:02,393 INFO [org.jboss.as.server] (ServerService Thread Pool -- 28) JBAS018559: Deployed "xyz.ear" (runtime-name : "xyz.ear")
15:12:02,570 INFO [org.jboss.as.controller] (Controller Boot Thread) JBAS014774: Service status report
JBAS014775: New missing/unsatisfied dependencies:
service jboss.io.worker.default (missing) dependents: [service jboss.remoting.endpoint.subsystem]
service jboss.remoting.remotingConnectorInfoService.remoting-connector (missing) dependents: [service jboss.ejb3.connector]
service jboss.web.common.server (missing) dependents: [service jboss.ws.config]
JBAS014777: Services which failed to start: service jboss.deployment.unit."xyz.ear".FIRST_MODULE_USE: org.jboss.msc.service.StartException in service jboss.deployment.unit."xyz.ear".FIRST_MODULE_USE: JBAS018733: Failed to process phase FIRST_MODULE_USE of deployment "xyz.ear"

Ok, the application is now loading in wildfly 8.1.0.Final. We ended up upgrading Hibernate to 4.1 (packaged by default with Wildfly). And since this is the default, a 'provider' tag was not necessary. We changed, 2 XML files i.e. jboss-deployment-structure.xml & persistence.xml and a java file in the application (which consisted of ServiceLocator).