RabbitMQ CachingConnectionFactory and publisherReturns configuration - rabbitmq

A continuation of Make Spring RabbitMQ fail on missing exchange
I register MessageListenerContainer for multiple queues.
Where and how should I configure channel returnListener? - I thing the way I have done it is wrong. I inserted CachingConnectionFactory configuration into createQueueBMessageListener(...) - method responsible for creating one of multiple MessageListeners.
1. How should be CachingConnectionFactory additional configuration done Spring and Rabbit way? By now I didn't configure it in Java (only by application.properties and admins in K8S environment). I only injected ConnectionFactory and set it as connectionFactory in SimpleMessageListenerContainer (as in createQueueAMessageListener(...)), I even didn't know it's CachingConnectionFactory.
Is there something like CachingConnectionFactoryConfigurer?
2. Why is ReturnListener.handleReturn(..) not executed? ChannelListener.onCreate(...) is executed.
3. Checking missing exchange exception in cachingConnectionFactory.setCloseExceptionLogger and doing System.exit(1) there seems wrong to me, isn't it? But this is all I managed to do by now. I want to application not start when there is no exchange during binding creation. When I throw exception there application still starts. ReturnListener.handleReturn seems a better place for it, but it isn't executed when configured as below.
4. How can I stop Spring Application Context gracefully instead of System.exit(1)? - throwing exception doesn't stop Application Context. How to make RabbitMq fail to start in such situation? - when a creation of #Bean Binding at Spring Application Context start fails.
#Bean
MessageListenerContainer createQueueAMessageListener(SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory,
ConnectionFactory connectionFactory) {
SimpleMessageListenerContainer container = rabbitListenerContainerFactory.createListenerContainer();
container.setConnectionFactory(connectionFactory);
container.setQueueNames("queueA");
MessageConverter jsonMessageConverter = null;
container.setMessageListener(new MessageListenerAdapter(new Object(), jsonMessageConverter));
return container;
}
#Bean
MessageListenerContainer createQueueBMessageListener(SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory,
ConnectionFactory connectionFactory,
CachingConnectionFactory cachingConnectionFactory) {
// I think configuring CachingConnectionFactory here is a lame, isn't it? Of course connectionFactory is redundant now, I left it to show how was it done earlier.
// Where and how should I add listeners to CachingConnectionFactory?
cachingConnectionFactory.setPublisherConfirmType(CachingConnectionFactory.ConfirmType.CORRELATED);
cachingConnectionFactory.setPublisherReturns(true);
cachingConnectionFactory.addChannelListener(new ChannelListener() {
#Override
public void onCreate(final Channel channel, final boolean transactional) {
log.info("channelListener onCreate - this is executed");
channel.addReturnListener(new ReturnListener() {
#Override
public void handleReturn(final int replyCode, final String replyText, final String exchange, final String routingKey,
final AMQP.BasicProperties properties,
final byte[] body) throws IOException
{
log.info("!!! Why is this not executed ?!!! handleReturn replyCode: " + replyCode + " replyText: " + replyText);
}
});
}
});
cachingConnectionFactory.addConnectionListener(new ConnectionListener() {
#Override
public void onCreate(final Connection connection) {
log.info("connectionListener onCreate - this is executed" + connection);
}
});
cachingConnectionFactory.setCloseExceptionLogger(new ConditionalExceptionLogger() {
#Override
public void log(final Log logger, final String message, final Throwable t) {
try {
logger.error(message + ": " + t.getMessage());
if (t.getMessage().contains("reply-code=404, reply-text=NOT_FOUND")) {
// throw new RuntimeException(); it doesn't stop Spring ApplicationContext from starting
log.error("Executing System.exit(1) command.");
// System.exit(1);
}
} catch (Exception e) {
log.error("err in listener ", e);
}
}
});
SimpleMessageListenerContainer container = rabbitListenerContainerFactory.createListenerContainer();
container.setConnectionFactory(connectionFactory);
container.setQueueNames("queueB");
MessageConverter jsonMessageConverter = null;
container.setMessageListener(new MessageListenerAdapter(new Object(), jsonMessageConverter));
return container;
}
#Bean
MessageListenerContainer createQueueCMessageListener(SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory,
ConnectionFactory connectionFactory) {
SimpleMessageListenerContainer container = rabbitListenerContainerFactory.createListenerContainer();
container.setConnectionFactory(connectionFactory);
container.setQueueNames("queueC");
MessageConverter jsonMessageConverter = null;
container.setMessageListener(new MessageListenerAdapter(new Object(), jsonMessageConverter));
return container;
}

// I think configuring CachingConnectionFactory here is a lame, isn't it?
It is not "lame"; that is the normal way of configuring beans with additional properties not exposed directly by Boot.
It should be called; have you tried debugging?
Why don't you do what I advised here Make Spring RabbitMQ fail on missing exchange - it's much simpler.
close() it - but, since you are using Spring Boot, it will do that for you - it registers a JVM shutdown hook that closes the context.
EDIT
Binding to a non-existent exchange will fail; you just need to force it to happen before the application is fully initialized, e.g. in an ApplicationRunner.
#SpringBootApplication
public class So70212347Application {
public static void main(String[] args) {
SpringApplication.run(So70212347Application.class, args);
}
#Bean
Binding binding() {
return new Binding("foo", DestinationType.QUEUE, "doesn't exist", "foo", null);
}
#Bean
Queue queue() {
return new Queue("foo");
}
#Bean
ApplicationRunner runner(ConnectionFactory cf) {
return args -> {
cf.createConnection().close();
};
}
}
Created new connection: rabbitConnectionFactory#6a0cbc6f:0/SimpleConnection#6cd164a6 [delegate=amqp://guest#127.0.0.1:5672/, localPort= 62884]
Shutdown Signal: channel error; protocol method: #method<channel.close>(reply-code=404, reply-text=NOT_FOUND - no exchange 'doesn't exist' in vhost '/', class-id=50, method-id=20)
Shutdown Signal: channel error; protocol method: #method<channel.close>(reply-code=404, reply-text=NOT_FOUND - no exchange 'doesn't exist' in vhost '/', class-id=50, method-id=20)
Shutdown Signal: channel error; protocol method: #method<channel.close>(reply-code=404, reply-text=NOT_FOUND - no exchange 'doesn't exist' in vhost '/', class-id=50, method-id=20)
Shutdown Signal: channel error; protocol method: #method<channel.close>(reply-code=404, reply-text=NOT_FOUND - no exchange 'doesn't exist' in vhost '/', class-id=50, method-id=20)
Shutdown Signal: channel error; protocol method: #method<channel.close>(reply-code=404, reply-text=NOT_FOUND - no exchange 'doesn't exist' in vhost '/', class-id=50, method-id=20)
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
Application run failed
java.lang.IllegalStateException: Failed to execute ApplicationRunner
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:761)
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:748)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:309)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1301)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1290)
at com.example.demo.So70212347Application.main(So70212347Application.java:16)
Caused by: org.springframework.amqp.AmqpIOException: java.io.IOException
at org.springframework.amqp.rabbit.support.RabbitExceptionTranslator.convertRabbitAccessException(RabbitExceptionTranslator.java:70)
at org.springframework.amqp.rabbit.connection.RabbitAccessor.convertRabbitAccessException(RabbitAccessor.java:113)
at org.springframework.amqp.rabbit.core.RabbitTemplate.doExecute(RabbitTemplate.java:2192)
at org.springframework.amqp.rabbit.core.RabbitTemplate.execute(RabbitTemplate.java:2138)
at org.springframework.amqp.rabbit.core.RabbitTemplate.execute(RabbitTemplate.java:2118)
at org.springframework.amqp.rabbit.core.RabbitAdmin.initialize(RabbitAdmin.java:691)
at org.springframework.amqp.rabbit.core.RabbitAdmin.lambda$null$10(RabbitAdmin.java:619)
at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:329)
at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:209)
at org.springframework.amqp.rabbit.core.RabbitAdmin.lambda$afterPropertiesSet$11(RabbitAdmin.java:618)
at org.springframework.amqp.rabbit.connection.CompositeConnectionListener.lambda$onCreate$0(CompositeConnectionListener.java:38)
at java.base/java.util.concurrent.CopyOnWriteArrayList.forEach(CopyOnWriteArrayList.java:807)
at org.springframework.amqp.rabbit.connection.CompositeConnectionListener.onCreate(CompositeConnectionListener.java:38)
at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.createConnection(CachingConnectionFactory.java:730)
at com.example.demo.So70212347Application.lambda$0(So70212347Application.java:33)
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:758)
... 5 common frames omitted
Caused by: java.io.IOException: null
at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:129)
at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:125)
at com.rabbitmq.client.impl.AMQChannel.exnWrappingRpc(AMQChannel.java:147)
at com.rabbitmq.client.impl.ChannelN.queueBind(ChannelN.java:1077)
at com.rabbitmq.client.impl.ChannelN.queueBind(ChannelN.java:46)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.amqp.rabbit.connection.CachingConnectionFactory$CachedChannelInvocationHandler.invoke(CachingConnectionFactory.java:1157)
at com.sun.proxy.$Proxy47.queueBind(Unknown Source)
at org.springframework.amqp.rabbit.core.RabbitAdmin.declareBindings(RabbitAdmin.java:870)
at org.springframework.amqp.rabbit.core.RabbitAdmin.lambda$initialize$12(RabbitAdmin.java:694)
at org.springframework.amqp.rabbit.core.RabbitTemplate.invokeAction(RabbitTemplate.java:2227)
at org.springframework.amqp.rabbit.core.RabbitTemplate.doExecute(RabbitTemplate.java:2186)
... 18 common frames omitted
Caused by: com.rabbitmq.client.ShutdownSignalException: channel error; protocol method: #method<channel.close>(reply-code=404, reply-text=NOT_FOUND - no exchange 'doesn't exist' in vhost '/', class-id=50, method-id=20)
...

Related

How to connect to FTPS server with data connection using same TLS session from Apache Camel using custom FTPSClient?

I would like to send files to FTPS server using Apache Camel. The problem is that this FTPS server requires that the TLS/SSL session is to be reused for the data connection. And I can't set 'TLSOptions NoSessionReuseRequired' option for security reason to solve the issue.
As far as I know, Apache Camel uses Apache Common Net class FTPSClient internally to communicate to FTPS servers and Apache Common Net doesn't support this feature as described here
So I has implemented this workaround. Here is code of my custom FTPSClient:
public class SSLSessionReuseFTPSClient extends FTPSClient {
// adapted from: https://trac.cyberduck.io/changeset/10760
#Override
protected void _prepareDataSocket_(final Socket socket) throws IOException {
if (socket instanceof SSLSocket) {
final SSLSession session = ((SSLSocket) _socket_).getSession();
final SSLSessionContext context = session.getSessionContext();
try {
final Field sessionHostPortCache = context.getClass().getDeclaredField("sessionHostPortCache");
sessionHostPortCache.setAccessible(true);
final Object cache = sessionHostPortCache.get(context);
final Method putMethod = cache.getClass().getDeclaredMethod("put", Object.class, Object.class);
putMethod.setAccessible(true);
// final Method getHostMethod = socket.getClass().getDeclaredMethod("getHost");
Method getHostMethod;
try {
getHostMethod = socket.getClass().getDeclaredMethod("getPeerHost");
} catch (NoSuchMethodException e) {
getHostMethod = socket.getClass().getDeclaredMethod("getHost");
}
getHostMethod.setAccessible(true);
Object host = getHostMethod.invoke(socket);
final String key = String.format("%s:%s", host, String.valueOf(socket.getPort()))
.toLowerCase(Locale.ROOT);
putMethod.invoke(cache, key, session);
} catch (Exception e) {
throw new RuntimeException(e);
}
}
}
}
It works brilliantly as standalone FTPS client in JDK 8 and JDK 11 as shown:
public class FTPSDemoClient {
public static void main(String[] args) throws IOException {
System.out.println("Java version is: " + System.getProperty("java.version"));
System.out.println("Java vendor is: " + System.getProperty("java.vendor"));
final SSLSessionReuseFTPSClient ftps = new SSLSessionReuseFTPSClient();
System.setProperty("jdk.tls.useExtendedMasterSecret", "false");
System.setProperty("jdk.tls.client.enableSessionTicketExtension", "false");
System.setProperty("jdk.tls.client.protocols", "TLSv1,TLSv1.1,TLSv1.2");
System.setProperty("https.protocols", "TLSv1,TLSv1.1,TLSv1.2");
//System.setProperty("javax.net.debug", "all");
ftps.setTrustManager(TrustManagerUtils.getAcceptAllTrustManager());
ftps.addProtocolCommandListener(new PrintCommandListener(new PrintWriter(System.out), true));
ftps.connect("my_ftps_server");
System.out.println("Connected to server");
ftps.login("user", "password");
System.out.println("Loggeded to server");
ftps.setFileType(FTP.BINARY_FILE_TYPE);
// Use passive mode as default because most of us are
// behind firewalls these days.
ftps.enterLocalPassiveMode();
ftps.setUseEPSVwithIPv4(true);
// Set data channel protection to private
ftps.execPROT("P");
for (final String s : ftps.listNames("directory1/directory2")) {
System.out.println(s);
}
// send file
try (final InputStream input = new FileInputStream("C:\\testdata\\olympus2.jpg")) {
ftps.storeFile("directory1/directory2/olympus2.jpg", input);
}
// receive file
try (final OutputStream output = new FileOutputStream("C:\\testdata\\ddd.txt")) {
ftps.retrieveFile(""directory1/directory2/ddd.txt", output);
}
ftps.logout();
if (ftps.isConnected()) {
try {
ftps.disconnect();
} catch (final IOException f) {
// do nothing
}
}
}
}
Now I am ready to use this custom FTPSClient in my Apache Camel route, first I create custom FTPSClient instance and make it available for Apache Camel:
public final class MyFtpClient {
public static void main(String[] args) {
RouteBuilder routeBuilder = new MyFtpClientRouteBuilder();
System.out.println("Java version is: " + System.getProperty("java.version"));
System.out.println("Java vendor is: " + System.getProperty("java.vendor"));
System.setProperty("jdk.tls.useExtendedMasterSecret", "false");
System.setProperty("jdk.tls.client.enableSessionTicketExtension", String.valueOf(false));
System.setProperty("jdk.tls.client.protocols", "TLSv1,TLSv1.1,TLSv1.2");
System.setProperty("https.protocols", "TLSv1,TLSv1.1,TLSv1.2");
SSLSessionReuseFTPSClient ftps = new SSLSessionReuseFTPSClient();
ftps.setTrustManager(TrustManagerUtils.getAcceptAllTrustManager());
// ftps.addProtocolCommandListener(new PrintCommandListener(new PrintWriter(System.out), true));
ftps.setRemoteVerificationEnabled(false);
ftps.setUseEPSVwithIPv4(true);
SimpleRegistry registry = new SimpleRegistry();
registry.bind("FTPClient", ftps);
// tell Camel to use our SimpleRegistry
CamelContext ctx = new DefaultCamelContext(registry);
try {
ctx.addRoutes(routeBuilder);
ctx.start();
Thread.sleep(5 * 60 * 1000);
ctx.stop();
}
catch (Exception e) {
e.printStackTrace();
}
}
}
And use it in Apache Camel Route:
public class MyFtpClientRouteBuilder extends RouteBuilder {
#Override
public void configure() throws Exception {
// lets shutdown faster in case of in-flight messages stack up
getContext().getShutdownStrategy().setTimeout(10);
from("ftps://my_ftps_server:21/directory1/directory2?username=user&password=RAW(password)"
+ "&localWorkDirectory=/tmp&autoCreate=false&passiveMode=true&binary=true&noop=true&resumeDownload=true"
+ "&bridgeErrorHandler=true&throwExceptionOnConnectFailed=true&maximumReconnectAttempts=0&transferLoggingLevel=OFF"
+ "&readLock=changed&disconnect=true&ftpClient=#FTPClient") // #FTPClient
.to("file://c:/testdata?noop=true&readLock=changed")
.log("Downloaded file ${file:name} complete.");
// use system out so it stand out
System.out.println("*********************************************************************************");
System.out.println("Use ctrl + c to stop this application.");
System.out.println("*********************************************************************************");
}
}
And it works!
But, when I add another route in the same java code by adding second from clause like this:
from("ftps://my_ftps_server/directory1/directory2?username=user&password=RAW(password)"
+ "&localWorkDirectory=/tmp&autoCreate=false&passiveMode=true&binary=true&noop=true&resumeDownload=true"
+ "&bridgeErrorHandler=true&throwExceptionOnConnectFailed=true&maximumReconnectAttempts=0&transferLoggingLevel=OFF"
+ "&readLock=changed&disconnect=true&ftpClient=#FTPClient") // #FTPClient
.to("file://c:/testdata?noop=true&readLock=changed")
.log("Downloaded file ${file:name} complete.");
from("file://c:/testdata?noop=true&readLock=changed&delay=30s")
.to("ftps://my_ftps_server/directory1/directory2?username=user&password=RAW(password)"
+ "&localWorkDirectory=/tmp&autoCreate=false&passiveMode=true&binary=true&noop=true&resumeDownload=true"
+ "&bridgeErrorHandler=true&throwExceptionOnConnectFailed=true&maximumReconnectAttempts=0&transferLoggingLevel=OFF"
+ "&readLock=changed&disconnect=true&stepwise=false&ftpClient=#FTPClient") // changed from FTPClient to FTPClient1
.log("Upload file ${file:name} complete.");
it ruins my code, it throws exception:
org.apache.camel.component.file.GenericFileOperationFailedException: File operation failed: null Socket is closed. Code: 226
...
Caused by: java.net.SocketException: Socket is closed
at java.net.Socket.setSoTimeout(Socket.java:1155) ~[?:?]
at sun.security.ssl.BaseSSLSocketImpl.setSoTimeout(BaseSSLSocketImpl.java:637) ~[?:?]
at sun.security.ssl.SSLSocketImpl.setSoTimeout(SSLSocketImpl.java:74) ~[?:?]
at org.apache.commons.net.ftp.FTP._connectAction_(FTP.java:426) ~[commons-net-3.8.0.jar:3.8.0]
at org.apache.commons.net.ftp.FTPClient._connectAction_(FTPClient.java:668) ~[commons-net-3.8.0.jar:3.8.0]
at org.apache.commons.net.ftp.FTPClient._connectAction_(FTPClient.java:658) ~[commons-net-3.8.0.jar:3.8.0]
at org.apache.commons.net.ftp.FTPSClient._connectAction_(FTPSClient.java:221) ~[commons-net-3.8.0.jar:3.8.0]
at org.apache.commons.net.SocketClient._connect(SocketClient.java:254) ~[commons-net-3.8.0.jar:3.8.0]
at org.apache.commons.net.SocketClient.connect(SocketClient.java:212) ~[commons-net-3.8.0.jar:3.8.0]
at org.apache.camel.component.file.remote.FtpOperations.doConnect(FtpOperations.java:125) ~[camel-ftp-3.4.1.jar:3.4.1]
Files, anyway are transferred to and from FTPS server by Apache Camel.
Interesting thing, when I don't share my custom FTPSClient and use one instance exactly for one route like this:
SSLSessionReuseFTPSClient ftps = new SSLSessionReuseFTPSClient();
...
SSLSessionReuseFTPSClient ftps1 = new SSLSessionReuseFTPSClient();
...
SimpleRegistry registry = new SimpleRegistry();
registry.bind("FTPClient", ftps);
registry.bind("FTPClient1", ftps1);
from("ftps://my_ftps_server/directory1/directory2?username=user&password=RAW(password)"
+ "&localWorkDirectory=/tmp&autoCreate=false&passiveMode=true&binary=true&noop=true&resumeDownload=true"
+ "&bridgeErrorHandler=true&throwExceptionOnConnectFailed=true&maximumReconnectAttempts=0&transferLoggingLevel=OFF"
+ "&readLock=changed&disconnect=true&ftpClient=#FTPClient") // #FTPClient
.to("file://c:/testdata?noop=true&readLock=changed")
.log("Downloaded file ${file:name} complete.");
from("file://c:/testdata?noop=true&readLock=changed&delay=30s")
.to("ftps://my_ftps_server/directory1/directory2?username=user&password=RAW(password)"
+ "&localWorkDirectory=/tmp&autoCreate=false&passiveMode=true&binary=true&noop=true&resumeDownload=true"
+ "&bridgeErrorHandler=true&throwExceptionOnConnectFailed=true&maximumReconnectAttempts=0&transferLoggingLevel=OFF"
+ "&readLock=changed&disconnect=true&stepwise=false&ftpClient=#FTPClient1")
.log("Upload file ${file:name} complete.");
it works perfectly!
So, I have couple of questions:
Why does Apache Camel (I mean Apache Common Net) developers refuse (or can't) to add usage of same TLS session functionality to FTPSClient class since 2011?
Am I the only person who uses Apache Camel to work with FTPS server with data connection using same TLS session? I haven't managed to find solution anywhere.
Is it possible to force Apache Camel not to share custom FTPSClient instance what, I suppose is the root of the problem, but to create new instance of FTPSClient every time then route are processed? My solution doesn't seem elegant.
What is wrong in my custom FTPSClient implementation that leads to this error then I use instance of this class in Apache Camel? Standard FTPClient hasn't this issue, of course.

First message to RabbitMQ queue causes channel shutdown

The first message to my queue always fails.
From the second one, everything works just fine!
Not sure if that's readable so :
Created new connection: rabbitConnectionFactory#1b940034:0/SimpleConnection#2c52fbff [delegate=amqp://guest#10.0.0.10:5672/, localPort= 36370]
Channel shutdown: channel error; protocol method: #method<channel.close>(reply-code=406, reply-text=PRECONDITION_FAILED - inequivalent arg 'auto_delete' for exchange 'rabbitmq_exchange' in vhost '/': received 'false' but current is 'true', class-id=40, method-id=10)
I'm not sure why this is happening, because I launch this on a fresh VM (AWS EC2 instance) every single time. How could "current be true"?
I suppose something is badly configured in the Spring Boot publisher:
Not sure if that's readable so :
#Configuration
public class RabbitMqConfig {
#Bean
Queue queue() {
return new Queue(System.getenv("RABBITMQ_QUEUE_NAME"), true,false, false);
}
#Bean
DirectExchange exchange() {
return new DirectExchange(System.getenv("RABBITMQ_EXCHANGE_NAME"), true, false);
}
#Bean
Binding binding(Queue queue, DirectExchange exchange) {
return BindingBuilder.bind(queue).to(exchange).with(System.getenv("RABBITMQ_ROUTING_KEY"));
}
#Bean
public MessageConverter jsonMessageConverter(){
return new Jackson2JsonMessageConverter();
}
public AmqpTemplate rabbitTemplate(ConnectionFactory connectionFactory) {
final RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
rabbitTemplate.setMessageConverter(jsonMessageConverter());
return rabbitTemplate;
}
}
So what's going wrong here?
Thanks!
The error is quite clear...
Channel shutdown: channel error; protocol method: #method(reply-code=406, reply-text=PRECONDITION_FAILED - inequivalent arg 'auto_delete' for exchange 'rabbitmq_exchange' in vhost '/': received 'false' but current is 'true', class-id=40, method-id=10)
When the connection is first opened, the framework looks for queues/bindings etc and declares them. If a queue exists already, it must have the same properties/arguments.
#Bean
Queue queue() {
return new Queue(System.getenv("RABBITMQ_QUEUE_NAME"), true, false, false);
}
Presumably, on the consumer side, it is being declared with different properties...
#Bean
Queue queue() {
return new Queue(System.getenv("RABBITMQ_QUEUE_NAME"), true, false, true);
}
(the auto_delete is inequivalent).
They must be the same.
Define a queue as below.
declare queue name=YourQueName durable=false --vhost="YourVirtualHostName" -u UsernameOfYourQueue -p PasswordOfYourQueue
In my case the queue was first created with wrong configurations, so each time I relaunched my application this error where shown.
Deleting the queue and letting the application recreated it, with the new configurations, solved this problem.

What is the use case of BrokerService in ActiveMQ and how to use it correctly

I am new about ActiveMQ. I'm trying to study and check how it works by checking the example code provided by Apache at this link:-
http://activemq.apache.org/how-should-i-implement-request-response-with-jms.html
public class Server implements MessageListener {
private static int ackMode;
private static String messageQueueName;
private static String messageBrokerUrl;
private Session session;
private boolean transacted = false;
private MessageProducer replyProducer;
private MessageProtocol messageProtocol;
static {
messageBrokerUrl = "tcp://localhost:61616";
messageQueueName = "client.messages";
ackMode = Session.AUTO_ACKNOWLEDGE;
}
public Server() {
try {
//This message broker is embedded
BrokerService broker = new BrokerService();
broker.setPersistent(false);
broker.setUseJmx(false);
broker.addConnector(messageBrokerUrl);
broker.start();
} catch (Exception e) {
System.out.println("Exception: "+e.getMessage());
//Handle the exception appropriately
}
//Delegating the handling of messages to another class, instantiate it before setting up JMS so it
//is ready to handle messages
this.messageProtocol = new MessageProtocol();
this.setupMessageQueueConsumer();
}
private void setupMessageQueueConsumer() {
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory(messageBrokerUrl);
Connection connection;
try {
connection = connectionFactory.createConnection();
connection.start();
this.session = connection.createSession(this.transacted, ackMode);
Destination adminQueue = this.session.createQueue(messageQueueName);
//Setup a message producer to respond to messages from clients, we will get the destination
//to send to from the JMSReplyTo header field from a Message
this.replyProducer = this.session.createProducer(null);
this.replyProducer.setDeliveryMode(DeliveryMode.NON_PERSISTENT);
//Set up a consumer to consume messages off of the admin queue
MessageConsumer consumer = this.session.createConsumer(adminQueue);
consumer.setMessageListener(this);
} catch (JMSException e) {
System.out.println("Exception: "+e.getMessage());
}
}
public void onMessage(Message message) {
try {
TextMessage response = this.session.createTextMessage();
if (message instanceof TextMessage) {
TextMessage txtMsg = (TextMessage) message;
String messageText = txtMsg.getText();
response.setText(this.messageProtocol.handleProtocolMessage(messageText));
}
//Set the correlation ID from the received message to be the correlation id of the response message
//this lets the client identify which message this is a response to if it has more than
//one outstanding message to the server
response.setJMSCorrelationID(message.getJMSCorrelationID());
//Send the response to the Destination specified by the JMSReplyTo field of the received message,
//this is presumably a temporary queue created by the client
this.replyProducer.send(message.getJMSReplyTo(), response);
} catch (JMSException e) {
System.out.println("Exception: "+e.getMessage());
}
}
public static void main(String[] args) {
new Server();
}
}
My confusion about the messageBrokerUrl = "tcp://localhost:61616"; You know ActiveMQ service is running on port 61616 by default. Why does this example chooses same port. If I try to run the code thows eception as:
Exception: Failed to bind to server socket: tcp://localhost:61616 due to: java.net.BindException: Address already in use: JVM_Bind
Perhaps if I change the port number, I can execute the code.
Please let me know why it is like this in the example and how to work with BrokerService.
The BrokerService in this example is trying to create an in memory ActiveMQ broker for use in the example. Given the error you are seeing I'd guess you already have an ActiveMQ broker running on the machine that is bound to port 61616 as that's the default port and thus the two are conflicting. You could either stop the external broker and run the example or modify the example to not run the embedded broker and just rely on your external broker instance.
Embedded brokers are great for unit testing or for creating examples that don't require the user to have a broker installed and running.

Apache camel write to rabbitmq

I am trying to read data from a file and write to a rabbitmq queue using apache camel but ending up with the error
Exception in thread "main" org.apache.camel.FailedToCreateRouteException: Failed to create route route1 at: >>> To[rabbitmq://localhost:15672?queue=hello] <<< in route: Route(route1)[[From[file://target/?fileName=doctor.txt&chars... because of Failed to resolve endpoint: rabbitmq://localhost:15672?queue=hello due to: No URI path as the exchangeName for the RabbitMQEndpoint, the URI is rabbitmq://localhost:15672?queue=hello
at org.apache.camel.model.RouteDefinition.addRoutes(RouteDefinition.java:945)
at org.apache.camel.model.RouteDefinition.addRoutes(RouteDefinition.java:187)
at org.apache.camel.impl.DefaultCamelContext.startRoute(DefaultCamelContext.java:794)
at org.apache.camel.impl.DefaultCamelContext.startRouteDefinitions(DefaultCamelContext.java:2184)
at org.apache.camel.impl.DefaultCamelContext.doStartCamel(DefaultCamelContext.java:1916)
at org.apache.camel.impl.DefaultCamelContext.doStart(DefaultCamelContext.java:1777)
at org.apache.camel.support.ServiceSupport.start(ServiceSupport.java:61)
at org.apache.camel.impl.DefaultCamelContext.start(DefaultCamelContext.java:1745)
at test.RMQCamelSender.main(RMQCamelSender.java:38)
Caused by: org.apache.camel.ResolveEndpointFailedException: Failed to resolve endpoint: rabbitmq://localhost:15672?queue=hello due to: No URI path as the exchangeName for the RabbitMQEndpoint, the URI is rabbitmq://localhost:15672?queue=hello
at org.apache.camel.impl.DefaultCamelContext.getEndpoint(DefaultCamelContext.java:545)
at org.apache.camel.util.CamelContextHelper.getMandatoryEndpoint(CamelContextHelper.java:71)
at org.apache.camel.model.RouteDefinition.resolveEndpoint(RouteDefinition.java:202)
at org.apache.camel.impl.DefaultRouteContext.resolveEndpoint(DefaultRouteContext.java:106)
at org.apache.camel.impl.DefaultRouteContext.resolveEndpoint(DefaultRouteContext.java:112)
at org.apache.camel.model.SendDefinition.resolveEndpoint(SendDefinition.java:61)
at org.apache.camel.model.SendDefinition.createProcessor(SendDefinition.java:55)
at org.apache.camel.model.ProcessorDefinition.makeProcessor(ProcessorDefinition.java:500)
at org.apache.camel.model.ProcessorDefinition.addRoutes(ProcessorDefinition.java:213)
at org.apache.camel.model.RouteDefinition.addRoutes(RouteDefinition.java:942)
... 8 more
Caused by: java.lang.IllegalArgumentException: No URI path as the exchangeName for the RabbitMQEndpoint, the URI is rabbitmq://localhost:15672?queue=hello
at org.apache.camel.component.rabbitmq.RabbitMQComponent.createEndpoint(RabbitMQComponent.java:50)
at org.apache.camel.component.rabbitmq.RabbitMQComponent.createEndpoint(RabbitMQComponent.java:31)
at org.apache.camel.impl.DefaultComponent.createEndpoint(DefaultComponent.java:122)
at org.apache.camel.impl.DefaultCamelContext.getEndpoint(DefaultCamelContext.java:525)
... 17 more
Following are my implementations of the classes which are creating camel context and rabbitmq queue.
RMQCamleSender.java
package test;
import org.apache.camel.CamelContext;
import org.apache.camel.Exchange;
import org.apache.camel.Processor;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.impl.DefaultCamelContext;
import test.Producer;
public class RMQCamelSender {
public static void main(String[] argv) throws Exception {
Producer queueProd = new Producer();
queueProd.setupConnection();
System.out.println(queueProd.toString());
CamelContext context = new DefaultCamelContext();
context.addRoutes(new RouteBuilder() {
public void configure() throws Exception {
System.out.println("hello world");
from("file://target/?fileName=doctor.txt&charset=utf-8")
.process(new Processor() {
public void process(Exchange msg) throws Exception {
System.out.println(msg.getIn().getBody(
String.class));
}
}).to("rabbitmq://localhost:15672?queue=hello");
}
});
context.start();
Thread.sleep(4000);
context.stop();
}
}
Producer.java
package test;
import java.io.IOException;
import java.util.concurrent.TimeoutException;
import com.rabbitmq.client.ConnectionFactory;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.Channel;
public class Producer {
public static final String QUEUE_NAME = "hello";
public static Connection connection;
public void setupConnection() throws IOException, TimeoutException {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("localhost");
connection = factory.newConnection();
Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
}
}
The program is working fine if I am not using Camel and trying to contact the queue using standard rabbitmq library.
The error message is pretty indicative: you are missing the exchange name in your endpoint URI, it cannot be blank, as the official Camel documentation says it has to follow the given format:
rabbitmq://hostname[:port]/exchangeName?[options]
I suggest you try using the amq.direct exchange (for more details, refer to the official RabbitMQ documentation), like so:
public void configure() throws Exception {
System.out.println("hello world");
from("file://target/?fileName=doctor.txt&charset=utf-8")
.process(new Processor() {
public void process(Exchange msg) throws Exception {
System.out.println(msg.getIn().getBody(String.class));
}
}).to("rabbitmq://localhost:5672/amq.direct?routingKey=hello");
}
Additionally, you are using 15672 port, which is by default the port for RabbitMQ's web management console, I guess you haven't changed the default settings so the port needs to be 5672. You also have to use the routingKey parameter instead of the queue one since in RabbitMQ semantics you publish to exchange:routingKey and only consume from queues.
You need to include the rabbitmq exchange name. It is missing from here;
.to("rabbitmq://localhost:15672?queue=hello");
It should be something like this:
.to("rabbitmq://localhost:15672/exchangeName?routingKey=hello");
Also, if you are sending to exchange, why do you want to specify a queue? You need only to specify the routing key and the exchange will send to that queue if there is a binding for that routing key.

Timeout of basicPublish when server is outofspace

My case is rabbitmq server got out of space, just as below
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/ramonubuntu--vg-root 6299376 5956336 0 100% /
The producer publishes message to server(the message needs to be persisted), and then will be blocked forever, it will keeping waiting the response of publishing. Sure we should avoid the situation of server out of space, but is there any timeout mechanism to let producer quit the waiting?
I have tried heartbeat and SO_TIMEOUT, they both don't work, as the network works fine. Below is my producer.
protected void publish(byte[] message) throws Exception {
// ConnectionFactory can be reused between threads.
ConnectionFactory factory = new SoTimeoutConnectionFactory();
factory.setHost(this.getHost());
factory.setVirtualHost("te");
factory.setPort(5672);
factory.setUsername("amqp");
factory.setPassword("amqp");
factory.setConnectionTimeout(10 * 1000);
// doesn't help if server got out of space
factory.setRequestedHeartbeat(1);
final Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
// declare a 'topic' type of exchange
channel.exchangeDeclare(this.exchangeName, "topic", true);
channel.addReturnListener(new ReturnListener() {
#Override
public void handleReturn(int replyCode, String replyText, String exchange, String routingKey,
AMQP.BasicProperties properties, byte[] body) throws IOException {
logger.warn("[X]Returned message(replyCode:" + replyCode + ",replyText:" + replyText
+ ",exchange:" + exchange + ",routingKey:" + routingKey + ",body:" + new String(body));
}
});
channel.confirmSelect();
channel.addConfirmListener(new ConfirmListener() {
#Override
public void handleAck(long deliveryTag, boolean multiple) throws IOException {
logger.info("Ack: " + deliveryTag);
// RabbitMessagePublishMain.this.release(connection);
}
#Override
public void handleNack(long deliveryTag, boolean multiple) throws IOException {
logger.info("Nack: " + deliveryTag);
// RabbitMessagePublishMain.this.release(connection);
}
});
channel.basicPublish(this.exchangeName, RabbitMessageConsumerMain.EXCHANGE_NAME + ".-1", true,
MessageProperties.PERSISTENT_BASIC, message);
channel.waitForConfirmsOrDie(10*1000);
// now we can close connection
connection.close();
}
It will block at 'channel.waitForConfirmsOrDie(10*1000);', and the SotimeoutConnectionFactory,
public class SoTimeoutConnectionFactory extends ConnectionFactory {
#Override
protected void configureSocket(Socket socket) throws IOException {
super.configureSocket(socket);
socket.setSoTimeout(10 * 1000);
}
}
Also I captured the network between producer and rabbimq,
Please help.
You need to implement Connection Block/Unblocked.
This is basically a way of notifying the publisher that the server is running out of resources. The advantage with this is that the publisher will also be notified once it is safe to publish again.
I would recommend that you take a look at this article. A simple way of implementing this is to have a flag that indicates if it is safe to publish, if it is not wait until it is.
As an example you can take a look on how I implemented this in one of my Python examples.