We need to know if the message from the producer has been delivered to the queue, not to the consumer. This is basically another way of checking if queue exists.
We are using Camel routing for our RabbitMQ messages and we need a publisher confirms. I know that RabbitMQ client provides publisher confirms but I'm wondering if Camel supports this feature. We are using Camel as follows.
#Produce(uri = "direct:event")
private ProducerTemplate producer;
void method() {
producer.sendBodyAndHeaders("content", HashMapHeaders);
//Confirm the acknowledgement
}
Camel does support publisher confirms (see the official documentation), you just have to enable them by setting publisherAcknowledgements property to true and optionally specifying a timeout interval through publisherAcknowledgementsTimeout property:
to("rabbitmq://localhost/A?routingKey=B&publisherAcknowledgements=true&publisherAcknowledgementsTimeout=10000")
Please note that support for this feature is available as of Camel 2.17.0.
Related
In Cloud Stream for RabbitMQ I want to set a consumer that will consume from a Queue only the messages that have been forwarded with a specific routing key. Is this possible?
Here are my application.properties.
Producer:
spring.cloud.stream.rabbit.bindings..consumer.bindQueue=false
spring.cloud.stream.rabbit.bindings..consumer.declareExchange=false
spring.cloud.stream.bindings.producer1-out-0.destination=exchange1
spring.cloud.stream.rabbit.bindings.producer1-out-0.producer.routingKeyExpression='routing.key.1'
Consumer:
spring.cloud.stream.rabbit.bindings..consumer.bindQueue=false
spring.cloud.stream.rabbit.bindings..consumer.declareExchange=false
spring.cloud.stream.bindings.consumer1-in-0.destination=exchange1
spring.cloud.stream.bindings.consumer1-in-0.group=reports
spring.cloud.stream.rabbit.bindings.consumer1-in-0.consumer.queueNameGroupOnly=false
spring.cloud.stream.rabbit.bindings.consumer1-in-0.consumer.bindingRoutingKeyDelimiter=,
spring.cloud.stream.rabbit.bindings.consumer1-in-0.consumer.bindingRoutingKey='routing.key.1'
Exchanges and bindings are defined directly in RabbitMQ.
My expectation was if I change spring.cloud.stream.rabbit.bindings.consumer1-in-0.consumer.bindingRoutingKey to something else the consumer will stop consuming but it's not the case.
No; unlike JMS, RabbitMQ has no concept of a message selector; you will get all messages.
The canonical way to solve this issue is to use a different queue for each RK.
bindingRoutingKey is meaningless when bindQueue is false and, in any case, all it does is specify which routing key to use when binding the queue to the exchange.
I am integrating a NestJS application with RabbitMQ and i followed this tutorial
From the tutorial i can see that message consumer can connect to the rabbitMQ broker using a queue name, but actually i want to create a temporary queue that connects to an exchange. I cannot find an option to specify the exchange name in the queue options specified in the tutorial. can someone please point to the right configuration to use to specify an exchange instead of queue?
is it possible to specify an exchange name inside the 'queueOptions' structure?
From what I understand from your tutorial (I don't know NestJS), createMicroservice only connects to a queue (or create it if needed). In other words, it performs assertQueue operation:
var queue = 'task_queue';
channel.assertQueue(queue, {
durable: false
});
If you want to bind a queue to an existing exchange, you need to perform bindQueue operation:
channel.bindQueue(queue, exchange, '');
Thus you can't bind to an exchange using queueOptions. Take a look at the official Rabbitmq documentation, especially the "Temporary queues" and following "Bindings" sections.
This document https://www.rabbitmq.com/reliability.html says it is possible to achieve at-least-once guarantee with RabbitMQ without using transactions. The problem with transactions is that they are slow, so the throughput drops drastically (see sections Publisher Confirms in https://www.rabbitmq.com/confirms.html). The option is to use Consumer Acknowledgements and Publisher Confirms.
On the other hand, Spring AMQP is able to handle Consumer Acknowledgements automatically using this configuration:
spring.rabbitmq.listener.acknowledgeMode=AUTO
For Publisher Confirms this is the configuration:
spring.rabbitmq.publisherConfirms=true
My doubt is whether these two properties are enough to guarantee at-least-once delivery or if I need to do anything else.
You will need to add a confirm callback to the rabbit template (either a custom one or the one configured by boot) to get the confirmations. You can add correlation data to the message sends so that you can correlate the confirm with the send.
See the documentation.
For Publisher Confirms (aka Publisher Acknowledgements), the template requires a CachingConnectionFactory that has its publisherConfirms property set to true. Confirms are sent to to the client by it registering a RabbitTemplate.ConfirmCallback by calling setConfirmCallback(ConfirmCallback callback). The callback must implement this method:
void confirm(CorrelationData correlationData, boolean ack, String cause);
The CorrelationData is an object supplied by the client when sending the original message. The ack is true for an ack and false for a nack. For nack s, the cause may contain a reason for the nack, if it is available when the nack is generated. An example is when sending a message to a non-existent exchange. In that case the broker closes the channel; the reason for the closure is included in the cause. cause was added in version 1.4.
Only one ConfirmCallback is supported by a RabbitTemplate.
I am integrating several .Net modules using pub/sub messaging using RabbitMQ and MassTransit. Most of the message subscription shall be durable. But some shall be transient. When a consumer dies the messages shall not be stored and already queued messages shall be discarded.
In each module I create 1 bus with 2 receive endpoints. One is configured as durable and non-auto-delete. The other one is configured as non-durable and auto-delete. Each gets its own set of consumers. This works as expected.
Now I am trying to implement request/response messages. Here comes the problem because now the sender has to decide to which exchange to route to. And that is wrong as I want receiver to decide whether to use durable or transient queue.
My questions:
Is there a better way how to support durable and transient subscription at the same time?
Why is MassTransit binding message exchange to an endpoint exchange that is bound to an endpoint queue? Why cannot the message exchange be directly bound to the endpoint queue?
Lets assume that all request consumers in one module are either durable or transient. Is it possible to declare one "module"-exchange which is then bound to either durable or transient queue? So the sender addresses the module exchange and module decides to which queue to bind. How to convince MassTransit to do so?
A module is using durable subscriptions that survive through restarts of module and also broker. After some time, admin (so in run-time of the system) decides to disconnect this module from the system. Can the module somehow unsubscribe everything and let MassTransit to remove the durable exchanges and queues?
Your question starts with request/response sent to an unknown endpoint, and ends with removing exchanges. These are different things, I suppose.
I cannot answer point-by-point, just will try to clear up things.
Request/response by definition requires you to know where you send stuff. As per MassTransit convention, the endpoint address is always an exchange/queue pair address. Therefore, you cannot let receiver decide who will handle this message, it will be delivered to the exchange/queue of the endpoint where you send it to.
About the "unsubscribe" - MassTransit deletes nothing. You have to clean up the binding that is not being used manually or by using the management API.
I am trying to connect from my Android app to one queue called "messages".
The producer (one webservices under AMQP protocol) is already connected, it can be check through RabbitMQ admin panel.
To connect from my Android device I am coding like this.
private void connect() throws Exception {
this.sampleClient = new MqttClient(this.broker, this.clientId);
MqttConnectOptions connOpts = new MqttConnectOptions();
connOpts.setUserName("user");
connOpts.setPassword("user".toCharArray());
/*connOpts.setConnectionTimeout(60 * 10);
connOpts.setKeepAliveInterval(60 * 5);*/
connOpts.setCleanSession(true);
this.sampleClient.connect(connOpts);
this.sampleClient.setCallback(this);
this.sampleClient.subscribe("messages");
if(!this.sampleClient.isConnected()){
System.out.println("Not Connected");
return;
}
System.out.println("Connected");
}
I have tried with "amq.topic", "amq.topic.*", "amq.topic.messages", etc... But when I look in the RabbitMQ queue section "messages" is with 0 consumers, and have been set one new queue called "mqtt-subscription-Sampleqos1" automatically.
What's happening? How can I susbscribe to "messages" queue?
There are two important points about this question.
According with the RabbitMQ MQTT documentation: http://www.rabbitmq.com/mqtt.html
Firstly, every queues are bound automatically to amq.topic exchange by the mqtt-plugin.
Secondly, every subscriber has his own queue which look like this, mqtt-subscription-{cliend_id}{qosX} (where X is the qos level of the subscription)
Therefore, producer must to publish the message to "amq.topic" exchange, and "amq.topic.." routing-key, and receiver must to subscribe to "amq.topic.." routing-key.
First, make sure MQTT plugin is enabled: rabbitmq-plugins enable rabbitmq_mqtt
From the client side (here is you Android app), you need subscriber to a topic, lets say, topic my/android/app/messages
this.sampleClient.subscribe("my/android/app/messages");
Then, from the server side, because of RabbitMQ's implementation, you need send the message to a special exchange 'amq.topic' with appropriate route key my.android.app.messages (notice the mapping between '/' and '.', MQTT use / and AMQP use .). For example if you publish by pika AMQP Python lib, the code will looks like following:
channel.basic_publish(
exchange='amq.topic',
routing_key='my.android.app.messages',
body='hello world'
)
In your case, you want to receive message from queue "messages", basically there is no way to directly subscriber message from that AMQP queue on your MQTT client. The work around is create a service running on your server side, work as AMQP subscriber, receive message from "messages" queue, and transparent forward message to exchange amq.topic with proper routing key.
Hope my answer helpful.