SimpleMessageListenerContainer Error Handling - error-handling

I'm using a SimpleMessageListenerContainer as a basis for remoting over AMQP. Everything goes smooth provided that the RabbitMQ broker can be reached at process startup. However, if by any reason it can't be reached (network down, permissions problem, etc...) the container just keeps retrying to connect forever. How can I set up a retry behaviour in this case (for example, try at most 5 times with an exponential backoff and then abort, killing the process)? I've had a look at this, but it doesn't seem to work for me on container startup. Can anyone please shed some light?
At the very least, I'd like to be able to catch the exception and provide a log message, instead of printing the exception itself as is the default behaviour.

How can I set up a retry behaviour in this case
There is no sophisticated connection retry, just a simple recoveryInterval. The assumption is that the broker unavailability is temporary. Fatal errors (such as bad credentials) stop the container.
You could use some external process to try connectionFactory.createConnection() and stop() the SimpleMessageListenerContainer when you deem it's time to give up.
You could also subclass CachingConnectionFactory, override createBareConnection catch the exception and increment the recoveryInterval, then call stop() when you want.
EDIT
Since 1.5, you can now configure a backOff. Here's an example using Spring Boot...
#SpringBootApplication
public class RabbitBackOffApplication {
public static void main(String[] args) {
SpringApplication.run(RabbitBackOffApplication.class, args);
}
#Bean(name = "rabbitListenerContainerFactory")
public SimpleRabbitListenerContainerFactory simpleRabbitListenerContainerFactory(
SimpleRabbitListenerContainerFactoryConfigurer configurer,
ConnectionFactory connectionFactory) {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
configurer.configure(factory, connectionFactory);
BackOff recoveryBackOff = new FixedBackOff(5000, 3);
factory.setRecoveryBackOff(recoveryBackOff);
return factory;
}
#RabbitListener(queues = "foo")
public void listen(String in) {
}
}
and
2018-04-16 12:08:35.730 INFO 84850 --- [ main] com.example.RabbitBackOffApplication : Started RabbitBackOffApplication in 0.844 seconds (JVM running for 1.297)
2018-04-16 12:08:40.788 WARN 84850 --- [cTaskExecutor-1] o.s.a.r.l.SimpleMessageListenerContainer : Consumer raised exception, processing can restart if the connection factory supports it. Exception summary: org.springframework.amqp.AmqpConnectException: java.net.ConnectException: Connection refused (Connection refused)
2018-04-16 12:08:40.788 INFO 84850 --- [cTaskExecutor-1] o.s.a.r.l.SimpleMessageListenerContainer : Restarting Consumer#57abad67: tags=[{}], channel=null, acknowledgeMode=AUTO local queue size=0
2018-04-16 12:08:40.789 INFO 84850 --- [cTaskExecutor-2] o.s.a.r.c.CachingConnectionFactory : Attempting to connect to: [localhost:1234]
2018-04-16 12:08:45.851 WARN 84850 --- [cTaskExecutor-2] o.s.a.r.l.SimpleMessageListenerContainer : Consumer raised exception, processing can restart if the connection factory supports it. Exception summary: org.springframework.amqp.AmqpConnectException: java.net.ConnectException: Connection refused (Connection refused)
2018-04-16 12:08:45.852 INFO 84850 --- [cTaskExecutor-2] o.s.a.r.l.SimpleMessageListenerContainer : Restarting Consumer#3479ea: tags=[{}], channel=null, acknowledgeMode=AUTO local queue size=0
2018-04-16 12:08:45.852 INFO 84850 --- [cTaskExecutor-3] o.s.a.r.c.CachingConnectionFactory : Attempting to connect to: [localhost:1234]
2018-04-16 12:08:50.935 WARN 84850 --- [cTaskExecutor-3] o.s.a.r.l.SimpleMessageListenerContainer : Consumer raised exception, processing can restart if the connection factory supports it. Exception summary: org.springframework.amqp.AmqpConnectException: java.net.ConnectException: Connection refused (Connection refused)
2018-04-16 12:08:50.935 INFO 84850 --- [cTaskExecutor-3] o.s.a.r.l.SimpleMessageListenerContainer : Restarting Consumer#2be60f67: tags=[{}], channel=null, acknowledgeMode=AUTO local queue size=0
2018-04-16 12:08:50.936 INFO 84850 --- [cTaskExecutor-4] o.s.a.r.c.CachingConnectionFactory : Attempting to connect to: [localhost:1234]
2018-04-16 12:08:50.938 WARN 84850 --- [cTaskExecutor-4] o.s.a.r.l.SimpleMessageListenerContainer : stopping container - restart recovery attempts exhausted

Related

Spring AMQP (RabbitMQ) Throwing Channel Shutdown Error

I am trying to do channel.basicReject() to requeue message based on some condition by creating an MethodInterceptor ConsumerAdvice and adding it to SMLC factor.setAdviceChain(new ConsumerAdvice()). I also have concurrentConsumer configuration which is set to 10. The moment my reject condition is met I issue basicReject command and it gets redelivered and processed by another consumer. During this redelivery process I get the below error,
2019-11-07 17:34:13.268 ERROR 29385 --- [ 127.0.0.1:5672] o.s.a.r.c.CachingConnectionFactory : Channel shutdown: channel error; protocol method: #method<channel.close>(reply-code=406, reply-text=PRECONDITION_FAILED - unknown delivery tag 1, class-id=60, method-id=80)
2019-11-07 17:34:13.268 DEBUG 29385 --- [ool-2-thread-13] o.s.a.r.listener.BlockingQueueConsumer : Received shutdown signal for consumer tag=amq.ctag-HUaN71TZUqMfLDR7k6LwGQ
com.rabbitmq.client.ShutdownSignalException: channel error; protocol method: #method<channel.close>(reply-code=406, reply-text=PRECONDITION_FAILED - unknown delivery tag 1, class-id=60, method-id=80)
at com.rabbitmq.client.impl.ChannelN.asyncShutdown(ChannelN.java:516)
at com.rabbitmq.client.impl.ChannelN.processAsync(ChannelN.java:346)
at com.rabbitmq.client.impl.AMQChannel.handleCompleteInboundCommand(AMQChannel.java:178)
at com.rabbitmq.client.impl.AMQChannel.handleFrame(AMQChannel.java:111)
at com.rabbitmq.client.impl.AMQConnection.readFrame(AMQConnection.java:670)
at com.rabbitmq.client.impl.AMQConnection.access$300(AMQConnection.java:48)
at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:597)
at java.lang.Thread.run(Thread.java:748)
My message is not getting lost but I am seeing a bunch of above errors and unable to understand why this is happening. If anyone has any clues please guide me.
Below are the trace logs,
2019-11-08 02:11:31.883 TRACE 8695 --- [askExecutor-138] o.s.a.r.c.CachingConnectionFactory : AMQChannel(amqp://guest#127.0.0.1:5672/,99) channel.getChannelNumber()
2019-11-08 02:11:31.883 INFO 8695 --- [askExecutor-138] c.g.s.w.consumer.advice.ArgumentUtils : Channel number before triggering redelivery : 99
2019-11-08 02:11:31.883 TRACE 8695 --- [askExecutor-138] o.s.a.r.c.CachingConnectionFactory : AMQChannel(amqp://guest#127.0.0.1:5672/,99) channel.basicReject([2, true])
2019-11-08 02:11:31.883 INFO 8695 --- [askExecutor-138] c.g.s.w.consumer.advice.ArgumentUtils : ==============================================================================
2019-11-08 02:11:31.883 INFO 8695 --- [askExecutor-138] c.g.s.w.consumer.advice.ConsumerAdvice : Requeue Message attempted, status : true
2019-11-08 02:11:31.884 TRACE 8695 --- [askExecutor-138] o.s.a.r.l.SimpleMessageListenerContainer : Waiting for message from consumer.
2019-11-08 02:11:31.884 TRACE 8695 --- [askExecutor-138] o.s.a.r.listener.BlockingQueueConsumer : Retrieving delivery for Consumer#7783912f: tags=[[amq.ctag-eY7LN-1pSXPX8FKRBgt-ug]], channel=Cached Rabbit Channel: AMQChannel(amqp://guest#127.0.0.1:5672/,99), conn: Proxy#37ffe4f3 Shared Rabbit Connection: SimpleConnection#708dfe10 [delegate=amqp://guest#127.0.0.1:5672/, localPort= 58638], acknowledgeMode=AUTO local queue size=0
2019-11-08 02:11:31.884 DEBUG 8695 --- [askExecutor-138] o.s.a.r.l.SimpleMessageListenerContainer : Consumer raised exception, processing can restart if the connection factory supports it
com.rabbitmq.client.ShutdownSignalException: channel error; protocol method: #method<channel.close>(reply-code=406, reply-text=PRECONDITION_FAILED - unknown delivery tag 1, class-id=60, method-id=80)
at org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.checkShutdown(BlockingQueueConsumer.java:436)
at org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.nextMessage(BlockingQueueConsumer.java:501)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.doReceiveAndExecute(SimpleMessageListenerContainer.java:843)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.receiveAndExecute(SimpleMessageListenerContainer.java:832)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.access$700(SimpleMessageListenerContainer.java:78)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1073)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.rabbitmq.client.ShutdownSignalException: channel error; protocol method: #method<channel.close>(reply-code=406, reply-text=PRECONDITION_FAILED - unknown delivery tag 1, class-id=60, method-id=80)
at com.rabbitmq.client.impl.ChannelN.asyncShutdown(ChannelN.java:516)
at com.rabbitmq.client.impl.ChannelN.processAsync(ChannelN.java:346)
at com.rabbitmq.client.impl.AMQChannel.handleCompleteInboundCommand(AMQChannel.java:178)
at com.rabbitmq.client.impl.AMQChannel.handleFrame(AMQChannel.java:111)
at com.rabbitmq.client.impl.AMQConnection.readFrame(AMQConnection.java:670)
at com.rabbitmq.client.impl.AMQConnection.access$300(AMQConnection.java:48)
at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:597)
... 1 common frames omitted
2019-11-08 02:11:31.884 ERROR 8695 --- [ 127.0.0.1:5672] o.s.a.r.c.CachingConnectionFactory : Channel shutdown: channel error; protocol method: #method<channel.close>(reply-code=406, reply-text=PRECONDITION_FAILED - unknown delivery tag 2, class-id=60, method-id=90)
You need to show your code and configuration.
It seems like the SMLC has its default configuration to automatically acknowledge messages and this failure is because you already rejected it; why are you interacting with the channel directly?
You can simply throw an exception and the container will reject the message on your behalf.
I don't know if it will helpful for someone.
I had the same error because of invalid input data. But when I added next properties:
"rabbit.listener.acknowledgeMode": "MANUAL",
"rabbit.listener.defaultRequeueRejected": "true",
"rabbit.listener.prefetchCount": "1",
the problem stop to break my program but only had stopping my listener

EasyNetQ not reconnecting after RabbitMQ server was unreachable

I am trying some edge cases in my software. So I have created a very simple test environment:
RabbitMQ server running on CentOS 7
Message consumer written against .NETCore 2.1 in C# running under CentOS 7
Message sender written against .NETCore 2.1 in C# running under CentOS 7
I send a simple text message every 5 seconds. The sender and receiver run on the same UNIX box, while the RabbitMQ server runs on a different machine in the network.
So far so good. Now I stop my RabbitMQ server with systemctl stop rabbitmq-server.
I get errors on the sender and the receiver which was expected.
I restart the RabbitMQ server using systemctl start rabbitmq-server.
And now the fun starts! The sender can recover and continues to send messages but the consumer CANNOT recover and does not receive messages. They are accumulated on the RabbitMQ server!
Here are my log entries from the sender (which works as expected):
2019-01-22 21:18:25.628 +01:00 [ERR] [EasyNetQ.PersistentConnection] [ThreadId 10] Failed to connect to broker infraserver-tbws2, port 5672, vhost testvh
RabbitMQ.Client.Exceptions.BrokerUnreachableException: None of the specified endpoints were reachable ---> System.AggregateException: One or more errors occurred. (Connection failed) ---> RabbitMQ.Client.Exceptions.ConnectFailureException: Connection failed ---> System.Net.Internals.SocketExceptionFactory+ExtendedSocketException: Connection refused 172.16.63.239:5672
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Exception source)
at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
at System.Net.Sockets.Socket.<>c.<ConnectAsync>b__272_0(IAsyncResult iar)
--- End of stack trace from previous location where exception was thrown ---
at RabbitMQ.Client.TcpClientAdapter.ConnectAsync(String host, Int32 port)
at RabbitMQ.Client.Impl.TaskExtensions.TimeoutAfter(Task task, Int32 millisecondsTimeout)
at RabbitMQ.Client.Impl.SocketFrameHandler.ConnectOrFail(ITcpClient socket, AmqpTcpEndpoint endpoint, Int32 timeout)
--- End of inner exception stack trace ---
at RabbitMQ.Client.Impl.SocketFrameHandler.ConnectUsingAddressFamily(AmqpTcpEndpoint endpoint, Func`2 socketFactory, Int32 timeout, AddressFamily family)
at RabbitMQ.Client.Impl.SocketFrameHandler.ConnectUsingIPv4(AmqpTcpEndpoint endpoint, Func`2 socketFactory, Int32 timeout)
at RabbitMQ.Client.Impl.SocketFrameHandler..ctor(AmqpTcpEndpoint endpoint, Func`2 socketFactory, Int32 connectionTimeout, Int32 readTimeout, Int32 writeTimeout)
at RabbitMQ.Client.Framing.Impl.IProtocolExtensions.CreateFrameHandler(IProtocol protocol, AmqpTcpEndpoint endpoint, Func`2 socketFactory, Int32 connectionTimeout, Int32 readTimeout, Int32 writeTimeout)
at RabbitMQ.Client.ConnectionFactory.CreateFrameHandler(AmqpTcpEndpoint endpoint)
at RabbitMQ.Client.EndpointResolverExtensions.SelectOne[T](IEndpointResolver resolver, Func`2 selector)
--- End of inner exception stack trace ---
at RabbitMQ.Client.EndpointResolverExtensions.SelectOne[T](IEndpointResolver resolver, Func`2 selector)
at RabbitMQ.Client.ConnectionFactory.CreateConnection(IEndpointResolver endpointResolver, String clientProvidedName)
--- End of inner exception stack trace ---
at RabbitMQ.Client.ConnectionFactory.CreateConnection(IEndpointResolver endpointResolver, String clientProvidedName)
at RabbitMQ.Client.ConnectionFactory.CreateConnection(String clientProvidedName)
at EasyNetQ.ConnectionFactoryWrapper.CreateConnection()
at EasyNetQ.PersistentConnection.TryToConnect()
2019-01-22 21:18:25.632 +01:00 [ERR] [EasyNetQ.PersistentConnection] [ThreadId 10] Failed to connect to any Broker. Retrying in 00:00:05
2019-01-22 21:18:35.444 +01:00 [INF] [] [ThreadId 1] Sucessfully sent Message 'Message: 'This is test message number 7.' | Num: 7 | Guid: 5345c7e4-61e6-4c79-8179-d4bef7864420'.
2019-01-22 21:18:40.452 +01:00 [INF] [] [ThreadId 1] Sucessfully sent Message 'Message: 'This is test message number 8.' | Num: 8 | Guid: 3cd8635c-cdfa-45f3-8495-2acb0713d47b'.
2019-01-22 21:18:45.457 +01:00 [INF] [] [ThreadId 1] Sucessfully sent Message 'Message: 'This is test message number 9.' | Num: 9 | Guid: 099462b8-cd66-40b9-ac10-89c3246819ec'.
2019-01-22 21:18:50.470 +01:00 [INF] [] [ThreadId 1] Sucessfully sent Message 'Message: 'This is test message number 10.' | Num: 10 | Guid: c25139b2-8e45-4771-9544-830014382e0c'.
2019-01-22 21:18:55.515 +01:00 [INF] [] [ThreadId 1] Sucessfully sent Message 'Message: 'This is test message number 11.' | Num: 11 | Guid: 90049d91-3805-4aaa-ac18-b61c09164afd'.
2019-01-22 21:19:00.526 +01:00 [INF] [] [ThreadId 1] Sucessfully sent Message 'Message: 'This is test message number 12.' | Num: 12 | Guid: 108ff318-6a34-4e64-94bd-dafa67aa6717'.
This shows the last error message and then one can see that EasyNetQ recovered and can deliver messages again.
The message consumer does NOT work! Here my log entries:
2019-01-22 21:18:25.623 +01:00 [ERR] [EasyNetQ.PersistentConnection] [ThreadId 12] Failed to connect to broker infraserver-tbws2, port 5672, vhost testvh
RabbitMQ.Client.Exceptions.BrokerUnreachableException: None of the specified endpoints were reachable ---> System.AggregateException: One or more errors occurred. (Connection failed) ---> RabbitMQ.Client.Exceptions.ConnectFailureException: Connection failed ---> System.Net.Internals.SocketExceptionFactory+ExtendedSocketException: Connection refused 172.16.63.239:5672
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Exception source)
at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
at System.Net.Sockets.Socket.<>c.<ConnectAsync>b__272_0(IAsyncResult iar)
--- End of stack trace from previous location where exception was thrown ---
at RabbitMQ.Client.TcpClientAdapter.ConnectAsync(String host, Int32 port)
at RabbitMQ.Client.Impl.TaskExtensions.TimeoutAfter(Task task, Int32 millisecondsTimeout)
at RabbitMQ.Client.Impl.SocketFrameHandler.ConnectOrFail(ITcpClient socket, AmqpTcpEndpoint endpoint, Int32 timeout)
--- End of inner exception stack trace ---
at RabbitMQ.Client.Impl.SocketFrameHandler.ConnectUsingAddressFamily(AmqpTcpEndpoint endpoint, Func`2 socketFactory, Int32 timeout, AddressFamily family)
at RabbitMQ.Client.Impl.SocketFrameHandler.ConnectUsingIPv4(AmqpTcpEndpoint endpoint, Func`2 socketFactory, Int32 timeout)
at RabbitMQ.Client.Impl.SocketFrameHandler..ctor(AmqpTcpEndpoint endpoint, Func`2 socketFactory, Int32 connectionTimeout, Int32 readTimeout, Int32 writeTimeout)
at RabbitMQ.Client.Framing.Impl.IProtocolExtensions.CreateFrameHandler(IProtocol protocol, AmqpTcpEndpoint endpoint, Func`2 socketFactory, Int32 connectionTimeout, Int32 readTimeout, Int32 writeTimeout)
at RabbitMQ.Client.ConnectionFactory.CreateFrameHandler(AmqpTcpEndpoint endpoint)
at RabbitMQ.Client.EndpointResolverExtensions.SelectOne[T](IEndpointResolver resolver, Func`2 selector)
--- End of inner exception stack trace ---
at RabbitMQ.Client.EndpointResolverExtensions.SelectOne[T](IEndpointResolver resolver, Func`2 selector)
at RabbitMQ.Client.ConnectionFactory.CreateConnection(IEndpointResolver endpointResolver, String clientProvidedName)
--- End of inner exception stack trace ---
at RabbitMQ.Client.ConnectionFactory.CreateConnection(IEndpointResolver endpointResolver, String clientProvidedName)
at RabbitMQ.Client.ConnectionFactory.CreateConnection(String clientProvidedName)
at EasyNetQ.ConnectionFactoryWrapper.CreateConnection()
at EasyNetQ.PersistentConnection.TryToConnect()
2019-01-22 21:18:25.625 +01:00 [ERR] [EasyNetQ.PersistentConnection] [ThreadId 12] Failed to connect to any Broker. Retrying in 00:00:05
Here it sits and waits forever! Looks like the thing is deadlocked in a way while the messages are accumulating on the RabbitMQ server:
When I stop my consumer application and restart it, the messages are picked up.
I connect in both applications (sender and consumer) with the following code:
private static IBus SetupRabbitMqConnection(string rabbitServer, string rabbitVHost, ushort rabbitPort, string rabbitUser, string rabbitPwd)
{
Log.Logger.Debug($"Creating a connection to RabbitMQ server '{rabbitServer}' on port {rabbitPort.ToString()} " +
$"using the EasyNetQ library....");
try
{
var connStr = $"host={rabbitServer}:{rabbitPort.ToString()};virtualHost={rabbitVHost};username={rabbitUser};" +
$"password={rabbitPwd};publisherConfirms=true;timeout=30;prefetchcount=1;requestedHeartbeat=30";
var msgBus = RabbitHutch.CreateBus(connStr, x => { });
if (!msgBus.IsConnected)
{
var errMsg = $"Currently not connected to RabbitMQ server '{rabbitServer}'.";
Log.Logger.Error(errMsg);
}
Log.Logger.Debug("Successfully connected to RabbitMQ server.");
return msgBus;
}
catch (Exception ex)
{
Log.Logger.Error($"Error to establish a connection to RabbitMQ server '{rabbitServer}'. Error: {ex}");
throw;
}
}
The consumer program registers the listeners as follows:
var msgBus = SetupRabbitMqConnection(rabbitServer, vhost, rabbitPort, rabbitUser, rabbitPwd);
RegisterMsgSubscriptions(msgBus);
private static void RegisterMsgSubscriptions(IBus msgBus)
{
Log.Logger.Debug("Starting to register RabbitMQ message subscriptions...");
try
{
#region Queue declarations
var advancedBus = msgBus.Advanced;
var testQueueOne = new EasyNetQ.Topology.Queue(TestQueueOneName, true);
Log.Logger.Debug("Finished declaring queues.");
#endregion
#region Message Queue Handler registrations
advancedBus.Consume(testQueueOne, registration => registration
.Add<RabbitMessage<TestTextMessageDto>>(MessageProcessor.ProcessRabbitTestMessage));
Log.Logger.Debug("Finished registration of message handlers for several queues.");
#endregion
}
catch (Exception ex)
{
Log.Logger.Error($"Error registering message handler. Error: {ex}");
}
}
Any idea what might go wrong here?? In production we have more than 100 servers consuming messages. These servers are on premise spread all over the country. The RabbitMQ server is in the data center. So the consuming servers MUST recover if the connection gets lost, otherwise this is not usable!
Just add subscription logic on connected event (It will be fired on every broker connected event).
advancedBus.Connected += (sender, args) => {
// subscribe logic
};
Of course You should handle not to call subscribe logice twice on any connect etc.

Not able to start RabbitMQ source in Flink 1.3.2

I want to start a RabbitMQ source and then sink, but I not able to perform first step i.e starting Rabbit MQ source . RabbitMQ server is running and I can see dashboard as well.
My code is as below
public class rabbitmq_source {
public static void main(String[] args) throws Exception {
StreamExecutionEnvironment envrionment = StreamExecutionEnvironment.getExecutionEnvironment();
RMQConnectionConfig connectionConfig = new RMQConnectionConfig.Builder()
.setHost("localhost")
.setPort(50000).
setUserName("root")
.setPassword("root").
setVirtualHost("/").build();
DataStream<String> stream = envrionment
.addSource(new RMQSource<String>(
connectionConfig, // config for the RabbitMQ connection
"queue", // name of the RabbitMQ queue to consume
new SimpleStringSchema()));
stream.print();
envrionment.execute();
}
}
I am not sure what username and pass should I set, should they be guest and guest. However, I am getting the following error
java.lang.RuntimeException: Cannot create RMQ connection with queue at localhost
at org.apache.flink.streaming.connectors.rabbitmq.RMQSource.open(RMQSource.java:172)
at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:111)
at org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:376)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:253)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:702)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at com.rabbitmq.client.impl.FrameHandlerFactory.create(FrameHandlerFactory.java:32)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:588)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:612)
Use LocalStreamEnvironment.createLocalEnvironment() with guest username and password.

SimpleMessageListenerContainer: Consumer raised exception, processing can restart if processing supports it

I have a Spring Boot application interfacing with a rabbitmq broker which manages to startup fine but I am getting a constant start/shutdown of the message consumer regardless of there being a message on the queue. Below is a copy of my application log and client class:
2016-10-19 11:40:25,909 WARN t:[SimpleAsyncTaskExecutor-106] SimpleMessageListenerContainer: Consumer raised exception, processing can restart if the connection factory supports it
java.lang.NullPointerException: null
at com.rabbitmq.client.impl.ChannelN.validateQueueNameLength(ChannelN.java:1232) ~[amqp-client-3.5.5.jar:na]
at com.rabbitmq.client.impl.ChannelN.queueDeclarePassive(ChannelN.java:884) ~[amqp-client-3.5.5.jar:na]
at com.rabbitmq.client.impl.ChannelN.queueDeclarePassive(ChannelN.java:61) ~[amqp-client-3.5.5.jar:na]
at sun.reflect.GeneratedMethodAccessor334.invoke(Unknown Source) ~[na:na]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_51]
at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_51]
at org.springframework.amqp.rabbit.connection.CachingConnectionFactory$CachedChannelInvocationHandler.invoke(CachingConnectionFactory.java:666) ~[spring-rabbit-1.4.6.RELEASE.jar:na]
at com.sun.proxy.$Proxy181.queueDeclarePassive(Unknown Source) ~[na:na]
at org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.attemptPassiveDeclarations(BlockingQueueConsumer.java:533) ~[spring-rabbit-1.4.6.RELEASE.jar:na]
at org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.start(BlockingQueueConsumer.java:453) ~[spring-rabbit-1.4.6.RELEASE.jar:na]
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1083) ~[spring-rabbit-1.4.6.RELEASE.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51]
2016-10-19 11:40:25,909 INFO t:[SimpleAsyncTaskExecutor-106] SimpleMessageListenerContainer: Restarting Consumer: tags=[{}], channel=Cached Rabbit Channel: AMQChannel(amqp://guest#127.0.0.1:5672/,1), acknowledgeMode=AUTO local queue size=0
2016-10-19 11:40:25,910 DEBUG t:[SimpleAsyncTaskExecutor-106] BlockingQueueConsumer: Closing Rabbit Channel: Cached Rabbit Channel: AMQChannel(amqp://guest#127.0.0.1:5672/,1)
2016-10-19 11:40:25,910 DEBUG t:[SimpleAsyncTaskExecutor-106] CachingConnectionFactory: Closing cached Channel: AMQChannel(amqp://guest#127.0.0.1:5672/,1)
2016-10-19 11:40:25,911 DEBUG t:[SimpleAsyncTaskExecutor-107] DefaultListableBeanFactory: Returning cached instance of singleton bean 'macRequestQueue'
2016-10-19 11:40:25,911 DEBUG t:[SimpleAsyncTaskExecutor-107] BlockingQueueConsumer: Starting consumer Consumer: tags=[{}], channel=null, acknowledgeMode=AUTO local queue size=0
2016-10-19 11:40:25,912 DEBUG t:[SimpleAsyncTaskExecutor-107] CachingConnectionFactory: Creating cached Rabbit Channel from AMQChannel(amqp://guest#127.0.0.1:5672/,1)
2016-10-19 11:40:25,912 DEBUG t:[SimpleAsyncTaskExecutor-107] SimpleMessageListenerContainer: Recovering consumer in 5000 ms.
Below is a copy of my client class:
public class RabbitClientConfiguration extends AbstractEMCRabbitConfiguration {
#Inject
private JacksonConfiguration jacksonConfiguration;
#Inject
private MessagingConfiguration messagingConfiguration;
//#Value("${data.core.pattern}")
//private String coreDataRoutingKey;
//#Inject
//private ClientHandler clientHandler;
#Override
public void configureRabbitTemplate(RabbitTemplate rabbitTemplate) {
rabbitTemplate.setRoutingKey(MAC_REQUEST_ROUTING_KEY);
}
#Bean
public SimpleMessageListenerContainer messageListenerContainer(){
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(connectionFactory());
container.setQueueNames(MAC_REQUEST_QUEUE_NAME);
container.setMessageListener(messageListenerAdapter());
container.setAcknowledgeMode(AcknowledgeMode.AUTO);
return container;
}
#Bean
MessageListenerAdapter messageListenerAdapter(){
return new MessageListenerAdapter(clientHandler(jacksonConfiguration.objectMapper(),messagingConfiguration.macMessageHandler(messagingConfiguration.messagingFactory())),jsonMessageConverter());
//return new MessageListenerAdapter(clientHandler,"receiveMessage");
}
#Bean
ClientHandler clientHandler(ObjectMapper objectMapper, IMessageHandler macMessageHandler){
ClientHandler clientHandler = new ClientHandler();
clientHandler.setObjectMapper(objectMapper);
clientHandler.setMacMessageHandler(macMessageHandler);
return clientHandler;
}
#Bean
public AmqpAdmin rabbitAdmin() { return new RabbitAdmin(connectionFactory());}
}
java.lang.NullPointerException: null at com.rabbitmq.client.impl.ChannelN.validateQueueNameLength
It means MAC_REQUEST_QUEUE_NAME contains null - it looks like the container doesn't check that when you call the setter.
I opened a JIRA Issue to detect this condition.

Cannot deploy MDB with durable subscription(topic) in EJB2.1 for Glassfish 4

I am in the process of migrate from OC4J to glassfish and struggles to deploy MDBs with durable subscriptions on Glassfish4
The class MyObserver implements MessageDrivenBean and MessageListener and is registered with a topic
Created topic:
asadmin create-jms-resource --restype=javax.jms.Topic --description="Topic for EventObjectMessage(s)" --property Name=PhysicalTopic jms/EventObjectMessageTopic
created topic factory:
asadmin create-jms-resource --restype=javax.jms.TopicConnectionFactory --description="Topic factory" --property ClientId=EventObjectMessageTopicConnectionFactoryID EventObjectMessageTopicConnectionFactory
created physical destination:
asadmin create-jmsdest --desttype topic EventObjectMessageTopic
in ejb-jar.xml:
<message-driven>
<ejb-name>MyObserver </ejb-name>
<ejb-class>com.firm.MyObserver </ejb-class>
<transaction-type>Container</transaction-type>
<message-selector>qualifiedName = 'com.firm.EventObjectMessage' AND objectsClassName = 'com.firm.SomeObject'</message-selector>
<acknowledge-mode>Auto-acknowledge</acknowledge-mode>
<message-driven-destination>
<destination-type>javax.jms.Topic</destination-type>
<subscription-durability>Durable</subscription-durability>
</message-driven-destination>
... a lot of refs
</message-driven>
and glassfish-ejb-jar.xml
<ejb>
<ejb-name>MyObserver </ejb-name>
<jndi-name>jms/EventObjectMessageTopic</jndi-name>
<mdb-resource-adapter>
<activation-config>
<activation-config-property>
<activation-config-property-name>clientId</activation-config-property-name>
<activation-config-property-value>EventObjectMessageTopicConnectionFactoryID</activation-config-property-value>
</activation-config-property>
<activation-config-property>
<activation-config-property-name>subscriptionName</activation-config-property-name>
<activation-config-property-value>ASubscriptionName</activation-config-property-value>
</activation-config-property>
</activation-config>
</mdb-resource-adapter>
</ejb>
This is part of the stacktrace:
[2014-01-07T08:44:44.308+0100] [glassfish 4.0] [SEVERE] [NCLS-CORE-00026] [javax.enterprise.system.core] [tid: _ThreadID=46 _ThreadName=admin-listener(3)] [timeMillis: 1389080684308] [levelValue: 1000] [[
Exception during lifecycle processing
java.lang.RuntimeException: EJB Container initialization error
at org.glassfish.ejb.startup.EjbApplication.loadContainers(EjbApplication.java:234)
at org.glassfish.ejb.startup.EjbDeployer.load(EjbDeployer.java:291)
at org.glassfish.ejb.startup.EjbDeployer.load(EjbDeployer.java:99)
...
Caused by: java.lang.Exception
at com.sun.enterprise.connectors.inbound.ConnectorMessageBeanClient.setup(ConnectorMessageBeanClient.java:216)
at org.glassfish.ejb.mdb.MessageBeanContainer.<init>(MessageBeanContainer.java:252)
at org.glassfish.ejb.mdb.MessageBeanContainerFactory.createContainer(MessageBeanContainerFactory.java:63)
at org.glassfish.ejb.startup.EjbApplication.loadContainers(EjbApplication.java:221)
... 68 more
Caused by: javax.resource.NotSupportedException: MQRA:EC:Error creating Direct Message Consumer:
createConsumer on JMSService:jmsdirect failed for connectionId:8664718337762585088 and sessionId:8664718337762587904 due to [B4135]: Cannot add durable consumer MyObserver. No ClientID was set on connection.
at com.sun.messaging.jms.ra.EndpointConsumer.createDirectMessageConsumer(EndpointConsumer.java:890)
at com.sun.messaging.jms.ra.EndpointConsumer._init(EndpointConsumer.java:345)
I am not sure if clientId should only be set on the connectionfactory and as a mdb-resource-adapter as I did.
Does anyone have a tip or could point out what i miss.
Best regards Geir