RabbitMQ throws AlreadyClosedException rather than IOException when the broker connection is lost - rabbitmq

I'm using rabbitmq java client 2.4.1 the newest version.
After a TCP connection lost, and still call a method on a channel over
this connection, a AlreadyClosedException will be thrown.
Is it a bug? I expected an IOException, but AlreadyClosedException I
got, and AlreadyClosedException is a RuntimeException.
If not, why all other errors cause an IOException.
#Test
public void testConnectionLost() throws IOException{
ConnectionFactory factory = new ConnectionFactory();
factory.setRequestedHeartbeat(60);
factory.setHost("<your rabbitmq host>");
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
connection.close();
try {
channel.queueDeclare("queueName", false, false, false, null);
Assert.fail("Exception expected.");
}catch (IOException e) {
//it will NOT reach here.
//Inner exception should be AlreadyClosedException
System.out.println(e);
}catch (AlreadyClosedException e) {
// it will reach here.
System.out.println(e);
//this is strange!
//I expected IOException , but AlreadyClosedException I got.
//And AlreadyClosedException is a RuntimeException.
}
Thank you.

If your client loses the TCP connection to your broker, the connection is considered "closed". Therefore it is appropriate (and not a bug) for the client library to throw an AlreadyClosedException.
In other words, a connection is considered "closed" no matter how it got closed (either through a graceful manner or through an unexpected failure).

Related

rabbitmq: channel level exception recovery

I am working on a worker service that consume from rabbitmq queue and I am trying to figure out how to handle channel shut down event, for example: lets say my consumer didn't ack the broker for 30 minutes and the broker shut down the channel for that.
I know the rabbitmq clinet library (I am using the C# library) will automatically try to re-connect on connection shut down, but what is the best practice for when the connection is alive but the channel was closed? I can register handler for the 'Channel Shut Down' event but what should I do inside this handler except for logging it? I want to keep consuming from the the relevant queue after all.
here is my code, i tried to create the channel again but i get timeout exception for that:
var consumer = ...
channel.BasicConsume(queue: queueName, autoAck: false, consumer: consumer);
channel.BasicQos(0, 100, false);
channel.ModelShutdown += (sender, args) =>
{
try
{
Log.Error($"channel was shut down");
channel = _connection.CreateModel();
channel.BasicConsume(queue: queueName, autoAck: false, consumer: consumer);
channel.BasicQos(0, 100, false)
}
catch (Exception exception)
{
Log.Error(exception);
}
As far as I understand, the problem is related to the fact that the event processing occurs in the same thread that is trying to access the RabbitMQ server, or there is a lock blocking the creation of the channel. The simplest solution I've found is to just use Task.Run(() => )
private async void RecreateChannel(object sender, ShutdownEventArgs e)
{
_logger.LogWarning($"Channel was shutdowned, whit reason: {e.ReplyText}, code: {e.ReplyCode}, trying to reconnect");
_channel.Dispose();
while (!_channel.IsOpen)
{
try
{
await Task.Run(() => _channel = InitChannel());
}
catch (Exception exception)
{
_logger.LogError(exception, "Failed to recreate channel, trying again");
}
}
_logger.LogInformation("Channel was recreated successfully");
}

Spring AMQP retry policy - wait with retrying until external system is reachable

Our application is storing data the user enters in SalesForce. We are using Spring AMQP (RabbitMQ) to be able to store data and run the application even if SalesForce is not reachable (scheduled downtimes or else).
So once SalesForce is not available the queue should wait with retrying messages until it is up again. Is this possible using the CircuitBreakerRetryPolicy or do i have to implement a custom RetryPolicy?
Currently i just have a simply retry policy which should be replaced by a complex one that blocks the retries until the external system is reachable again.
#Bean
RetryOperationsInterceptor salesForceInterceptor() {
return RetryInterceptorBuilder.stateless()
.recoverer(new RejectAndDontRequeueRecoverer())
.retryPolicy(simpleRetryPolicy())
.build();
}
Message listener:
#Override
public void onMessage(Message message) {
try {
.....
} catch (ResourceAccessException e) {
//TODO probablyy wrong exception
throw new AmqpTimeoutException("Salesforce can not be accessed, retry later");
} catch (Exception e) {
throw new AmqpRejectAndDontRequeueException("Fatal error with message to salesforce, do not retry", e);
}
}

What is the use case of BrokerService in ActiveMQ and how to use it correctly

I am new about ActiveMQ. I'm trying to study and check how it works by checking the example code provided by Apache at this link:-
http://activemq.apache.org/how-should-i-implement-request-response-with-jms.html
public class Server implements MessageListener {
private static int ackMode;
private static String messageQueueName;
private static String messageBrokerUrl;
private Session session;
private boolean transacted = false;
private MessageProducer replyProducer;
private MessageProtocol messageProtocol;
static {
messageBrokerUrl = "tcp://localhost:61616";
messageQueueName = "client.messages";
ackMode = Session.AUTO_ACKNOWLEDGE;
}
public Server() {
try {
//This message broker is embedded
BrokerService broker = new BrokerService();
broker.setPersistent(false);
broker.setUseJmx(false);
broker.addConnector(messageBrokerUrl);
broker.start();
} catch (Exception e) {
System.out.println("Exception: "+e.getMessage());
//Handle the exception appropriately
}
//Delegating the handling of messages to another class, instantiate it before setting up JMS so it
//is ready to handle messages
this.messageProtocol = new MessageProtocol();
this.setupMessageQueueConsumer();
}
private void setupMessageQueueConsumer() {
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory(messageBrokerUrl);
Connection connection;
try {
connection = connectionFactory.createConnection();
connection.start();
this.session = connection.createSession(this.transacted, ackMode);
Destination adminQueue = this.session.createQueue(messageQueueName);
//Setup a message producer to respond to messages from clients, we will get the destination
//to send to from the JMSReplyTo header field from a Message
this.replyProducer = this.session.createProducer(null);
this.replyProducer.setDeliveryMode(DeliveryMode.NON_PERSISTENT);
//Set up a consumer to consume messages off of the admin queue
MessageConsumer consumer = this.session.createConsumer(adminQueue);
consumer.setMessageListener(this);
} catch (JMSException e) {
System.out.println("Exception: "+e.getMessage());
}
}
public void onMessage(Message message) {
try {
TextMessage response = this.session.createTextMessage();
if (message instanceof TextMessage) {
TextMessage txtMsg = (TextMessage) message;
String messageText = txtMsg.getText();
response.setText(this.messageProtocol.handleProtocolMessage(messageText));
}
//Set the correlation ID from the received message to be the correlation id of the response message
//this lets the client identify which message this is a response to if it has more than
//one outstanding message to the server
response.setJMSCorrelationID(message.getJMSCorrelationID());
//Send the response to the Destination specified by the JMSReplyTo field of the received message,
//this is presumably a temporary queue created by the client
this.replyProducer.send(message.getJMSReplyTo(), response);
} catch (JMSException e) {
System.out.println("Exception: "+e.getMessage());
}
}
public static void main(String[] args) {
new Server();
}
}
My confusion about the messageBrokerUrl = "tcp://localhost:61616"; You know ActiveMQ service is running on port 61616 by default. Why does this example chooses same port. If I try to run the code thows eception as:
Exception: Failed to bind to server socket: tcp://localhost:61616 due to: java.net.BindException: Address already in use: JVM_Bind
Perhaps if I change the port number, I can execute the code.
Please let me know why it is like this in the example and how to work with BrokerService.
The BrokerService in this example is trying to create an in memory ActiveMQ broker for use in the example. Given the error you are seeing I'd guess you already have an ActiveMQ broker running on the machine that is bound to port 61616 as that's the default port and thus the two are conflicting. You could either stop the external broker and run the example or modify the example to not run the embedded broker and just rely on your external broker instance.
Embedded brokers are great for unit testing or for creating examples that don't require the user to have a broker installed and running.

Activemq VM Protocol - BrokerService Error

I am getting an error while opening connection to the broker service with VM protocol.
BrokerService broker = new BrokerService();
try {
broker.setPersistent(true);
broker.setUseJmx(false);
broker.setBrokerName("broker1");
broker.start();
while(true){
}
} catch (Exception e) {
e.printStackTrace();
}
And here is my producer;
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory("vm://broker1?create=false");
Connection connection = null;
Session session = null;
try {
connection = connectionFactory.createConnection();
connection.start();
I am getting this error;
javax.jms.JMSException: Could not create Transport. Reason: java.io.IOException: Broker named 'broker1' does not exist.
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:35)
at org.apache.activemq.ActiveMQConnectionFactory.createTransport(ActiveMQConnectionFactory.java:254)
at org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:267)
at org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:239)
at org.apache.activemq.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:185)
at Client.main(Client.java:22)
Tim you are right, connection was not created within the same jvm.I have realized my mistake

Reuse a client class in WCF after it is faulted

I use WCF for a client server system.
When I add a service reference to IService on the server, a proxy class ServiceClient is generated.
My code looks like the following:
ServiceClient client = new ServiceClient();
try
{
client.Operation1();
}
catch(Exception ex)
{
// Handle Exception
}
try
{
client.Operation2();
}
catch(Exception ex)
{
// Handle Exception
}
The problem is that if there is a communication exception in the first call, the client's state changes to Faulted, and I don't know how to reopen it to make the second call. Is there a way to reopen it? or should I create a new one and replace the instance (It doesn't seem like an elegant way)?
Once a an ICommunicationObject (your WCF client object) is in a faulted state, the only way to "re-open" it is to create a new one.
ServiceClient client = new ServiceClient();
try
{
client.Operation1();
}
catch(Exception ex)
{
if (client.State == CommunicationState.Faulted)
{
client.Abort();
client = new ServiceClient();
}
}
try
{
client.Operation2();
}
catch(Exception ex)
{
// Handle Exception
}
If there is a communication exception on the first call which is causing a faulted state, you have to basically "re-create" the WCF client proxy. In your example I would probably do something like:
if (client.State == CommunicationState.Faulted)
client = new ServiceClient();
This would allow you to "re-open" the connection if it is faulted. It may seem a little overkill, but if you're getting a communication exception on the client side, there's probably something else going on (i.e.: server dead? server not responding?)
Good luck
Agree with the last answers, once failed, you need to abort. We use a combination of lambdas and a method like the following to do this:
public static void Use<TServiceInterface>(TServiceInterface proxy, Action handler)
{
Type proxyType = typeof(TServiceInterface);
IClientChannel channel = (IClientChannel)proxy;
try
{
handler();
_logSource.Log(LogLevel.Debug, string.Format("Closing client channel for '{0}' ...", proxyType.Name));
channel.Close();
_logSource.Log(LogLevel.Debug, string.Format("Client channel for '{0}' closed.", proxyType.Name));
}
catch
{
if (channel.State == CommunicationState.Faulted)
{
_logSource.Log(LogLevel.Debug, string.Format("Aborting client channel for '{0}' ...", proxyType.Name));
channel.Abort();
_logSource.Log(LogLevel.Debug, string.Format("Client channel for '{0}' aborted.", proxyType.Name));
}
else
{
_logSource.Log(LogLevel.Debug, string.Format("Closing client channel for '{0}' ...", proxyType.Name));
channel.Close();
_logSource.Log(LogLevel.Debug, string.Format("Client channel for '{0}' closed.", proxyType.Name));
}
throw;
}
}
This is a slight modification of a solution that is already on the .net, but it works great for handling proxies. You then can put multiple service calls in the same lambda expression, and pass it into the method.
This is most likely caused by an unhandled exception on the server side. WCF runtime by default terminates your service instance and puts the channel in faulted state in case of unhandled exception and you can no longer communicate over that channel. So you will need to establish a new session with the service. You should catch exceptions on the server side and send the soap faults by raising FaultException or defining FaultContract. There is also returnUnknownExceptionsAsFaults service behavior that you can use.