async scope is not working in Clustering in mule 3.4.2. we are getting below exception.
Message : Interrupted while queueing event for "SEDA Stage Main_Flow.async1". Message payload is of type: ConfirmReceiveMessageResponse
Code : MULE_ERROR--2
--------------------------------------------------------------------------------
Exception stack is:
1. com.sample.client.ReceiveMessageResponse (java.io.NotSerializableException)
java.io.ObjectOutputStream:1183 (null)
2. java.io.NotSerializableException: com.elexon.bmrs.ecp.client.ReceiveMessageResponse (org.apache.commons.lang.SerializationException)
org.apache.commons.lang.SerializationUtils:111 (null)
3. Interrupted while queueing event for "SEDA Stage Main_Flow.async1". Message payload is of type: ConfirmReceiveMessageResponse (org.mule.api.service.FailedToQueueEventException)
org.mule.processor.SedaStageInterceptingMessageProcessor:92 (http://www.mulesoft.org/docs/site/current3/apidocs/org/mule/api/service/FailedToQueueEventException.html)
--------------------------------------------------------------------------------
Root Exception stack trace:
java.io.NotSerializableException: com.sample.client.ReceiveMessageResponse
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1183)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
at org.apache.commons.collections.map.AbstractHashedMap.doWriteObject(AbstractHashedMap.java:1182)
+ 3 more (set debug level logging or '-Dmule.verbose.exceptions=true' for everything)
After removing the async scope we are able to test the application. Could please help us how to make the application works with async in cluster env?
If the flow ref is using an async processing strategy it will try and persist the event in a cluster I believe. And your object is not Serializable.
You can make com.sample.client.ReceiveMessageResponse implement java.io.Serializable if you want the message to be persisted.
Or you can try forcing the flow that you are flow-ref'ing processingStrategy="synchronous" maybe.
Related
We have a spring-webflux application running on spring-boot-starter-webflux:jar:2.1.5.RELEASE and reactor-netty:jar:0.8.8.RELEASE. When a reactive client goes away (k8s pod killed or the client subscription is disposed off) before the request is completed, we see the server simply stops processing the request and no more application logs related to that request are seen. However, reactive-netty prints a trace log indicating that the channel is inactive and will be terminated.
Is there anything we can do to handle this termination gracefully? We would ideally like to respond to a cancellation signal from reactor.
2020-03-18T16:46:40.372Z TRACE --- [reactor-http-epoll-2] r.n.c.ChannelOperations : [id: 0x4cad974b, L:/<SERVER_IP_ADDRESS>:8080 ! R:/<SOME_OTHER_IP_ADDRESS>:55436] Disposing ChannelOperation from a channel
java.lang.Exception: ChannelOperation terminal stack
at reactor.netty.channel.ChannelOperations.terminate(ChannelOperations.java:391)
at reactor.netty.channel.ChannelOperations.onInboundClose(ChannelOperations.java:360)
at reactor.netty.channel.ChannelOperationsHandler.channelInactive(ChannelOperationsHandler.java:72)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:257)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:243)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:236)
at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelInactive(CombinedChannelDuplexHandler.java:420)
at io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:393)
at io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:358)
at io.netty.channel.CombinedChannelDuplexHandler.channelInactive(CombinedChannelDuplexHandler.java:223)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:257)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:243)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:236)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1416)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:257)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:243)
at io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:912)
at io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:816)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:416)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:331)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at java.base/java.lang.Thread.run(Thread.java:834)
The doOnCancel operator allows for a callback to be provided when a subscription is cancelled. This is different from the doOnEach operator which is run only when the subscription completes or errors out.
https://projectreactor.io/docs/core/release/api/reactor/core/publisher/Mono.html#doOnCancel-java.lang.Runnable-
I thought that the default configuration of the objectstore module in Mule was In-Memory (http://mulesoft.github.io/mule-module-objectstore/mule/objectstore-config.html#config)
I have an objectstore configured as such in my app:
<objectstore:config name="sourceConfigStore" entryTtl="60000" ></objectstore:config>
I reference the store from a java component like so:
ObjectStoreModule objectStore = (ObjectStoreModule) eventContext.getMuleContext().getRegistry().lookupObject("sourceConfigStore");
objectStore.store((String)sourceConfig.get("url"), sourceConfig, true);
This works for the most part, except I discovered today that this was writing files to disk when I got the following error:
Message : Unable to create a canonical file for parent: C:\git-ucd\.mule\.mule\edus-esb-rss-aggregator\objectstore and child: DEFAULT_PARTITION\news.ucdavis.edu/xml/getnews.php?type=category&categories=General+Interest&format=rss.obj (org.mule.api.MuleRuntimeException)
Code : MULE_ERROR--2
--------------------------------------------------------------------------------
Exception stack is:
1. Invalid argument (java.io.IOException)
java.io.WinNTFileSystem:-2 (null)
2. Unable to create a canonical file for parent: C:\git-ucd\.mule\.mule\edus-esb-rss-aggregator\objectstore and child: DEFAULT_PARTITION\news.ucdavis.edu/xml/getnews.php?type=category&categories=General+Interest&format=rss.obj (org.mule.api.MuleRuntimeException)
org.mule.util.FileUtils:402 (http://www.mulesoft.org/docs/site/current3/apidocs/org/mule/api/MuleRuntimeException.html)
3. Unable to create a canonical file for parent: C:\git-ucd\.mule\.mule\edus-esb-rss-aggregator\objectstore and child: DEFAULT_PARTITION\news.ucdavis.edu/xml/getnews.php?type=category&categories=General+Interest&format=rss.obj (org.mule.api.MuleRuntimeException) (org.mule.api.store.ObjectStoreException)
org.mule.util.store.PartitionedPersistentObjectStore:278 (http://www.mulesoft.org/docs/site/current3/apidocs/org/mule/api/store/ObjectStoreException.html)
--------------------------------------------------------------------------------
Root Exception stack trace:
java.io.IOException: Invalid argument
at java.io.WinNTFileSystem.canonicalize0(Native Method)
at java.io.Win32FileSystem.canonicalize(Win32FileSystem.java:414)
at java.io.File.getCanonicalPath(File.java:618)
+ 3 more (set debug level logging or '-Dmule.verbose.exceptions=true' for everything)
So, my question is whether or not the default behavior of the objectstore module is in fact to use an in-memory store. If that is the case, I guess my next question would be 'how did I override that default behavior with my above config and code'?
The default implementation is in memory.
Then again if you're running your application from Mule studio that's not the case as in MuleStudio by default it persists things to a file. This is why on your run configuration in the general tab you have the option to delete these files in each run.
In any case the easiest way to force the in memory will be something like this:
<objectstore:all-keys config-ref="_defaultInMemoryObjectStore"/>
In my <catch-exception-strategy>, I write error payload to file. But sometimes when flow involves web-service calls and host is unavailable or unknown (for e.g. java.net.UnknownHostException is thrown), payload is not anymore an instance of InputStream or String. If I try to log error then to file, following exception is thrown:
exception.AbstractExceptionListener (AbstractExceptionListener.java:299) -
********************************************************************************
Message : Could not find a transformer to transform "SimpleDataType{type=org.apache.commons.httpclient.methods.PostMethod, mimeType='*/*'}" to "SimpleDataType{type=java.io.InputStream, mimeType='*/*'}".
Code : MULE_ERROR-65237
--------------------------------------------------------------------------------
Exception stack is:
1. Could not find a transformer to transform "SimpleDataType{type=org.apache.commons.httpclient.methods.PostMethod, mimeType='*/*'}" to "SimpleDataType{type=java.io.InputStream, mimeType='*/*'}". (org.mule.api.transformer.TransformerException)
org.mule.registry.MuleRegistryHelper:252 (http://www.mulesoft.org/docs/site/current3/apidocs/org/mule/api/transformer/TransformerException.html)
--------------------------------------------------------------------------------
Root Exception stack trace:
org.mule.api.transformer.TransformerException: Could not find a transformer to transform "SimpleDataType{type=org.apache.commons.httpclient.methods.PostMethod, mimeType='*/*'}" to "SimpleDataType{type=java.io.InputStream, mimeType='*/*'}".
at org.mule.registry.MuleRegistryHelper.lookupTransformer(MuleRegistryHelper.java:252)
at org.mule.DefaultMuleMessage.getPayload(DefaultMuleMessage.java:355)
at org.mule.DefaultMuleMessage.getPayload(DefaultMuleMessage.java:313)
+ 3 more (set debug level logging or '-Dmule.verbose.exceptions=true' for everything)
********************************************************************************
I am thinking of doing a choice block before writing to file to make sure payload is writable. Shall I do something like #[payload instanceof java.io.InputStream]? But then how about cases where payload is DOM or something else? Please advise.
I would use a transformer inside the catch exception strategy and encapsulate there the logic that would consider the input and produce a writable payload.
If you want to check whether there is a transformer available for a specific payload type and output type, I guess you could look up the available transformers from the registry. In Groovy:
transformers = message.getMuleContext().getRegistry().lookupTransformers(
new org.mule.transformer.types.SimpleDataType(payload.getClass()),
new org.mule.transformer.types.SimpleDataType(java.io.InputStream))
if (transformers.size() == 0) {
//set some variable or whatever
}
Using JOliver EventStore 3.0, and just getting started with simple samples.
I have a simple pub/sub CQRS implementation using NServiceBus. A client sends commands on the bus, a domain server recieves and processes the commands and stores events to the eventstore, which are then published on the bus by the eventstore's dispatcher. a read-model server then subscribes to those events to update the read-model. Nothing fancy, pretty much by-the-book.
It is working, but just in simple tests I am getting lots of concurrency exceptions (intermittantly) on the domain server when the event is stored to the EventStore. It properly retries, but sometimes it hits the 5 retry limit and the command ends up on the error queue.
Where could I start investigating to see what is causing the concurrency exception? I remove the dispatcher and just focus on storing events and it has the same issue.
I'm using RavenDB for persistence of my EventStore. I'm not doing anything fancy, just this:
using (var stream = eventStore.OpenStream(entityId, 0, int.MaxValue))
{
stream.Add(new EventMessage { Body = myEvent });
stream.CommitChanges(Guid.NewGuid());
}
The stack trace for the exception looks like this:
2012-03-17 18:34:01,166 [Worker.14] WARN
NServiceBus.Unicast.UnicastBus [(null)] <(null)> -
EmployeeCommandHandler failed handling message.
EventStore.ConcurrencyException: Exception of type
'EventStore.ConcurrencyException' was thrown. at
EventStore.OptimisticPipelineHook.PreCommit(Commit attempt) in
c:\Code\public\EventStore\src\proj\EventStore.Core\OptimisticPipelineHook.cs:line
55 at EventStore.OptimisticEventStore.Commit(Commit attempt) in
c:\Code\public\EventStore\src\proj\EventStore.Core\OptimisticEventStore.cs:line
90 at EventStore.OptimisticEventStream.PersistChanges(Guid
commitId) in
c:\Code\public\EventStore\src\proj\EventStore.Core\OptimisticEventStream.cs:line
168 at EventStore.OptimisticEventStream.CommitChanges(Guid
commitId) in
c:\Code\public\EventStore\src\proj\EventStore.Core\OptimisticEventStream.cs:line
149 at CQRSTest3.Domain.Extensions.StoreEvent(IStoreEvents
eventStore, Guid entityId, Object evt) in
C:\dev\test\CQRSTest3\CQRSTest3.Domain\Extensions.cs:line 13 at
CQRSTest3.Domain.ComandHandlers.EmployeeCommandHandler.Handle(ChangeEmployeeSalary
message) in
C:\dev\test\CQRSTest3\CQRSTest3.Domain\ComandHandlers\Emplo
yeeCommandHandler.cs:line 55
I figured it out. Had to dig through source code to find it though. I wish this was better documented! Here's my new eventstore wireup:
EventStore = Wireup.Init()
.UsingRavenPersistence("RavenDB")
.ConsistentQueries()
.InitializeStorageEngine()
.Build();
I had to add .ConsistentQueries() in order for the raven persistence provider to internally use WaitForNonStaleResults on the queries eventstore was making to raven.
Basically when I add a new event, and then try to add another before raven has caught up with indexing, the stream revision was not up to date. The second event would step on the first one.
I am not able to get the reason of exception from the message that is in the DLQ.
These are the steps i have followed:-
Message is sent to a 'Sample' Queue.
The Message Listener throws a Runtime Exception in the onMessage Function.(throw new RuntimeException("Exception Reason Test");)
The message goes to the DLQ.
I am trying to access the Exception reason via two approaches(i pass the DLQ Name and the JMS Message ID in both):-
Spring JMSTemplate browseSelected function
(ActiveMQMessage)message.getStringProperty(ActiveMQMessage.DLQ_DELIVERY_FAILURE_CAUSE_PROPERTY);
JMX QueueViewMBean browse function via the composite data map
Map datamap = (Map)dataMap.get("StringProperties");
datamap.containsKey(ActiveMQMessage.DLQ_DELIVERY_FAILURE_CAUSE_PROPERTY);
In both cases i am getting a null value for the property DLQ_DELIVERY_FAILURE_CAUSE_PROPERTY.
Please help.
Thanks