I am sending messages to rabbitmq queue using mqtt protocol. I want to consume the messages using AMQP protocol. Instead of consuming the messages one by one. I want to consume the messages in batches and process them together. Is it possible with with RabbitMQ AMQP?
To read a group of messages in the queue together instead of reading one by one.
Instead of leaving Rabbit push messages to my consumers, the consumer connects to a queue and fetches a batch of N message.
Related
When the consumer consumes messages from the queue, then messages get automatically deleted from the queue. But I want to store them in the queue as long as I want. So is there any way in Spring Boot to store the RabbitMQ messages in the queue?
Can we design pub-sub patterns in RabbitMQ where a consumer can also act as a producer and send messages to the message broker?
pub-sub with the same service
Did you try to use producer API in consumer code? It should work...
You can find API docs for many languages in Client Documentation
Regarding design, consumers may consume, do some processing and then produce - publish to some other exchange of the same or other messaging broker instance...
It's design decision...
Yes, the consumer can also act as a producer. It's a common use case that the consumer sends back a new message/task about something else once the first message has been processed.
Make sure that you separate the connections for the publisher and the consumer.
RabbitMQ can apply back pressure on the TCP connection when the publisher is sending too many messages for the server to handle. If you consume on the same TCP connection, the server might not receive the message acknowledgments from the client, thus effecting the consume performance. With a lower consume speed, the server will be overwhelmed.
I have a RabbitMQ message queue and I want to publish multiple messages to the queue from a web service call ServiceA. Meantime, there is an another web service called ServiceB which is implemented for consuming the messages from the RabbitMQ same queue in an interval of 10 seconds time period. Is this use case possible with the implementation of the RabbitMQ queues?
Does RabbitMQ support to access the same queue by the publisher and consumer at the same time (simultaneously)?
ServiceB which is implemented for consuming the messages from the RabbitMQ same queue in an interval of 10 seconds time period.
It's a little bit strange to implement this by RabbitMQ. In RabbitMQ, consumer channel will receive message immediately unless its unAck messages reach the prefetch limit. I recommend to add a Buffer Cache (flush every 10 seconds) between RabbitMQ consumer and ServiceB.
Does RabbitMQ support to access the same queue by the publisher and consumer at the same time (simultaneously)?
In RabbitMQ, publisher can't access queue directly, you can only publish message to exchange, RabbitMQ Daemon will route message by the exchange binding rule. In other words, publisher and consumer can work independently and simultaneously.
I'm using RabbitMQ and need a message to be queued and consumed by multiple servers, but only one of the servers could confirm this message and only it would remove this message from the queue, even if other servers have consumed the message, but do not have removed it from the queue. I wonder how RabbitMQ can do this scenario ?
I use ActiveMQ as a job dispatcher. Which means one master sends job messages to ActiveMQ, and multiple slaves grab job messages from ActiveMQ and process them. When slaves finish one job, they send a message with job_id back to ActiveMQ.
However, slaves are unreliable. If one slave doesn't respond before a period of time, we can assume the slave is down, and try redeliver the sent job message.
Are there any good ideas to realize this re-delivery?
Typically a consumer handles redelivery so that it can maintain message order while a message appears as inflight on the broker. This means that redelivery is limited to a single consumer unless that consumer terminates. In this way the broker is unaware of redelivery.
In ActiveMQ v5.7+ you have the option of using broker side redelivery, it is possible to have the broker redeliver a message after a delay using a resend. This is implemented by a broker plugin that handles dead letter processing by redelivery via the scheduler. This is useful when total message order is not important and where through put and load distribution among consumers is. With broker redelivery, messages that fail delivery to a given consumer can get immediately re-dispatched.
See the ActiveMQ documentation for an example of setting this up in the configuration file.