I understand below
prefetch simply controls how many messsages the broker allows to be outstanding at the consumer at a time. When set to 1, this means the broker will send 1 message, wait for the ack, then send the next.
but questions regarding following scenarios:
Lets say prefetch is 200, we have 2 consumers idle. Broker got 150 messages, I think broker will pick one random and will send all 150 messages? I think yes it wont do sharing between consumers.
Lets say one consumer is having 100 messages in unack and one is idle and prefetch again is 200. Now we got 50 more messages, again I think broker will give those 50 to either one randomly? Or it will not give to consumer who already have 100 messages that not acked yet
If prefetch is 200, one consumer got 200, will listener block that thread (spring rabbitmq listner method) to send ack until all 200 processed ? I think it will not send ack one by one and will wait until all prefetched messages processed. In other words if prefetch is 200 and if broker delivers 200 messages, when broker will start getting ack?
If there are two active consumers, the broker will distribute new messages fairly (until each instance has 200 outstanding).
If there are 150 messages in the queue and no consumers running; the first consumer to start will (likely) get all 150, but when both are running, the distribution is fair.
If there are 200 outstanding at each consumer, the broker will send new messages on demand as each one is ack'd. The consumer thread is not "blocked", it is just that the broker will send no more messages.
By default, spring will ack each message one-at-a-time. This behavior can be changed by setting the container's batchSize property. e.g. if it is set to 100, it will send an ack every 100 records; this improves performance, but adds the risk of duplicate deliveries after a failure. In this case, the broker will send up to 100 new messages after the ack.
In older versions, batchSize was called txSize.
EDIT
See this for an example; the default prefetch is 250 in recent versions.
#SpringBootApplication
public class So65201334Application {
public static void main(String[] args) {
SpringApplication.run(So65201334Application.class, args);
}
#RabbitListener(id = "foo", queues = "foo", autoStartup = "false")
#RabbitListener(id = "bar", queues = "foo", autoStartup = "false")
void listen(String in, #Header(AmqpHeaders.CONSUMER_TAG) String tag) throws InterruptedException {
System.out.println(tag);
Thread.sleep(240_000);
}
#Bean
public ApplicationRunner runner(RabbitTemplate template, RabbitListenerEndpointRegistry registry) {
return args -> {
for (int i = 0; i < 200; i++) {
template.convertAndSend("foo", "bar");
}
registry.getListenerContainer("foo").start();
System.out.println("Hit Enter to start the second listener and send more records");
System.in.read();
registry.getListenerContainer("bar").start();
Thread.sleep(2000);
for (int i = 0; i < 200; i++) {
template.convertAndSend("foo", "bar");
}
};
}
}
As expected, all 200 went to the first consumer:
When the second consumer is started, the records are sent to both consumers, not the one that has no backlog. With the distribution now looking like this:
When I increase the prefetch to 400, you can see that the new messages go 50% to each consumer.
Setting a right value for prefetch is important and it depends on your RTT for the comume deliver ack cycle, so if you have large processing time its better to have the higher prefetch count otherwise lower prefetchenter link description here
Related
I have a RabbitMQ queue to hold unprocessed messages. I happy path, I will read message from the queue, process it, and removes the message in the queue. But if certain criteria are met while processing I have to republish the message to the queue again. I am using a pollable channel adapter to fetch the message. since I want to fetch all the available messages in that queue while polling I have set the maxMessagesPerPoll to -1. This causes the code to go in an infinite loop. after republishing the message into the queue, the inbound polled adapter picks it up immediately. How can I prevent this situation?
Is there any way to delay the message delivery or can we restrict the message processing once per message in single polling of the InboundPolledAdapter. What will be the best approach?
The inboundPolledAdapter is,
#Bean
public IntegrationFlow inboundIntegrationFlowPaymentRetry() {
return IntegrationFlows
.from(Amqp.inboundPolledAdapter(connectionFactory, RetryQueue),
e -> e.poller(Pollers.fixedDelay(20_000).maxMessagesPerPoll(-1)).autoStartup(true))
.handle(message -> {
channelRequestFromQueue()
.send(MessageBuilder.withPayload(message.getPayload()).copyHeaders(message.getHeaders())
.setHeader(IntegrationConstants.QUEUED_MESSAGE, message).build());
}).get();
}
For the first posting of first message to the queue by,
#Bean
Binding bindingRetryQueue() {
return BindingBuilder.bind(queueRetry()).to(exchangeRetry())
.with(ProcessQueuedMessageService.RETRY_ROUTING_KEY);
}
#Bean
TopicExchange exchangeRetry() {
return new TopicExchange(ProcessQueuedMessageService.RETRY_EXCHANGE);
}
#Bean
Queue queueRetry() {
return new Queue(RetryQueue, false);
}
#Bean
#ServiceActivator(inputChannel = "channelAmqpOutbound")
public AmqpOutboundEndpoint outboundAmqp(AmqpTemplate amqpTemplate) {
final AmqpOutboundEndpoint outbound = new AmqpOutboundEndpoint(amqpTemplate);
outbound.setRoutingKey(RetryQueue);
return outbound;
}
Republishing message by,
StaticMessageHeaderAccessor.getAcknowledgmentCallback(requeueMessage).acknowledge(Status.REQUEUE);
Is there any way to delay the message delivery
See Delayed Exchange feature in Rabbit MQ and its API in Spring AMQP: https://docs.spring.io/spring-amqp/docs/current/reference/html/#delayed-message-exchange
restrict the message processing once per message
For this scenario you can take a look into Idempotent Receiver pattern and its implementation in Spring Integration: https://docs.spring.io/spring-integration/docs/current/reference/html/messaging-endpoints.html#idempotent-receiver.
The redelivered message is going to have an AmqpHeaders.REDELIVERED header.
See more in docs: https://www.rabbitmq.com/reliability.html#consumer-side
I wanted to find out the failed messages for my rabbitmq producers using some call back api.I have configured rabbitmq with [{rabbit, [{vm_memory_high_watermark, 0.001}]}]. and tried pushing lot of messages but all the messages are getting accepted and TimeoutException is coming later on and messages not getting send to Queueenter code here, Please tell me how to capture it.
Code for sending message:
// #create-sink - producer
final Sink<ByteString, CompletionStage<Done>> amqpSink =
AmqpSink.createSimple(
AmqpSinkSettings.create(connectionProvider)
.withRoutingKey(AkkaConstants.queueName)
.withDeclaration(queueDeclaration));
// #run-sink
//final List<String> input = Arrays.asList("one", "two", "three", "four", "five");
//Source.from(input).map(ByteString::fromString).runWith(amqpSink, materializer);
String filePath = "D:\\subrata\\code\\akkaAmqpTest-master\\akkaAmqpTest-master\\logs2\\dummy.txt";
Path path = Paths.get(filePath);
// List containing 78198 individual message
List<String> contents = Files.readAllLines(path);
System.out.println("********** file reading done ....");
int times = 5;
// Send 78198*times message to Queue [From console i can see 400000 number of messages being sent]
for(int i=0;i<times;i++) {
Source.from(contents).map(ByteString::fromString).runWith(amqpSink, materializer);
}
System.out.println("************* sending to queue is done");
Unfortunately currently that is not supported out of the box. Ideally the producer would be modeled as a Flow which would send all incoming messages to the AMQP broker and would emit the same message with a result weather it has been successfully sent to the broker or not. There is a ticket to track this possible improvement on the Alpakka issue tracker.
The System Layout
We have three systems:
An API Endpoint (Publisher and Consumer)
The RabbitMQ Server
The main application/processor (Publisher and consumer)
System 1 and 3 both use Laravel, and use PHPAMQPLIB for interaction with RabbitMQ.
The path of a message
System 1 (the API Endpoint) sends a serialized job to the RabbitMQ Server for System 3 to process. It then immediately declares a new randomly named queue, binds an exchange to that queue with a correlation ID - and starts to listen for messages.
Meanwhile, system 3 finishes the job, and once it does, responds back with details from that job to RabbitMQ, on the exchange, with the correlation ID.
The issue and what I've tried
I often find that this process fails. The job gets sent and received, and the response gets sent - but system 1 never reads this response, and I don't see it published in RabbitMQ.
I've done some extensive debugging of this without getting to a root cause. My current theory is that System 3 is so quick at returning a response, that the new queue and exchange binding hasn't even been declared yet from System 1. This means the response from System 3 has nowhere to go, and as a result vanishes. This theory is mainly based on the fact that if I set jobs to be processed at a lower frequency on System 3, the system becomes more reliable. The faster the jobs process, the more unreliable it becomes.
The question is: How can I prevent that? Or is there something else that I'm missing? I of course want these jobs to process quickly and efficiently without breaking the Request/Response-pattern.
I've logged output from both systems - both are working with the same correlation ID's, and System 3 gets an ACK upon publishing - whilst System 1 has a declared queue with no messages that eventually just times out.
Code Example 1: Publishing a Message
/**
* Helper method to publish a message to RabbitMQ
*
* #param $exchange
* #param $message
* #param $correlation_id
* #return bool
*/
public static function publishAMQPRouteMessage($exchange, $message, $correlation_id)
{
try {
$connection = new AMQPStreamConnection(
env('RABBITMQ_HOST'),
env('RABBITMQ_PORT'),
env('RABBITMQ_LOGIN'),
env('RABBITMQ_PASSWORD'),
env('RABBITMQ_VHOST')
);
$channel = $connection->channel();
$channel->set_ack_handler(function (AMQPMessage $message) {
Log::info('[AMQPLib::publishAMQPRouteMessage()] - Message ACK');
});
$channel->set_nack_handler(function (AMQPMessage $message) {
Log::error('[AMQPLib::publishAMQPRouteMessage()] - Message NACK');
});
$channel->confirm_select();
$channel->exchange_declare(
$exchange,
'direct',
false,
false,
false
);
$msg = new AMQPMessage($message);
$channel->basic_publish($msg, $exchange, $correlation_id);
$channel->wait_for_pending_acks();
$channel->close();
$connection->close();
return true;
} catch (Exception $e) {
return false;
}
}
Code Example 2: Waiting for a Message Response
/**
* Helper method to fetch messages from RabbitMQ.
*
* #param $exchange
* #param $correlation_id
* #return mixed
*/
public static function readAMQPRouteMessage($exchange, $correlation_id)
{
$connection = new AMQPStreamConnection(
env('RABBITMQ_HOST'),
env('RABBITMQ_PORT'),
env('RABBITMQ_LOGIN'),
env('RABBITMQ_PASSWORD'),
env('RABBITMQ_VHOST')
);
$channel = $connection->channel();
$channel->exchange_declare(
$exchange,
'direct',
false,
false,
false
);
list($queue_name, ,) = $channel->queue_declare(
'',
false,
false,
true,
false
);
$channel->queue_bind($queue_name, $exchange, $correlation_id);
$callback = function ($msg) {
return self::$rfcResponse = $msg->body;
};
$channel->basic_consume(
$queue_name,
'',
false,
true,
false,
false,
$callback
);
if (!count($channel->callbacks)) {
Log::error('[AMQPLib::readAMQPRouteMessage()] - No callbacks registered!');
}
while (self::$rfcResponse === null && count($channel->callbacks)) {
$channel->wait();
}
$channel->close();
$connection->close();
return self::$rfcResponse;
}
Grateful for any advise you can offer!
I may be missing something, but when I read this:
System 1 (the API Endpoint) sends a serialized job to the RabbitMQ Server for System 3 to process. It then immediately declares a new randomly named queue, binds an exchange to that queue with a correlation ID - and starts to listen for messages.
My first thought was "why do you wait until the message is sent before declaring the return queue?"
In fact, we have a whole series of separate steps here:
Generating a correlation ID
Publishing a message containing that ID to an exchange for processing elsewhere
Declaring a new queue to receive responses
Binding the queue to an exchange using the correlation ID
Binding a callback to the new queue
Waiting for responses
The response cannot come until after step 2, so we want to do that as late as possible. The only step that can't come before that is step 6, but it's probably convenient to keep steps 5 and 6 close together in the code. So I would rearrange the code to:
Generating a correlation ID
Declaring a new queue to receive responses
Binding the queue to an exchange using the correlation ID
Publishing a message containing the correlation ID to an exchange for processing elsewhere
Binding a callback to the new queue
Waiting for responses
This way, however quickly the response is published, it will be picked up by the queue declared in step 2, and as soon as you bind a callback and start waiting, you will process it.
Note that there is nothing that readAMQPRouteMessage knows that publishAMQPRouteMessage doesn't, so you can freely move code between them. All you need when you want to consume from the response queue is its name, which you can either save into a variable and pass around, or generate yourself rather than letting RabbitMQ name it. For instant, you could name it after the correlation ID it is listening for, so that you can always work out what it is with simple string manipulation, e.g. "job_response.{$correlation_id}"
I'm just starting to learn RabbitMQ so forgive me if my question is very basic.
My problem is actually the same with the one posted here:
RabbitMQ - Does one consumer block the other consumers of the same queue?
However, upon investigation, i found out that manual acknowledgement prevents other consumers from getting a message from the queue - blocking state. I would like to know how can I prevent it. Below is my code snippet.
...
var message = receiver.ReadMessage();
Console.WriteLine("Received: {0}", message);
// simulate processing
System.Threading.Thread.Sleep(8000);
receiver.Acknowledge();
public string ReadMessage()
{
bool autoAck = false;
Consumer = new QueueingBasicConsumer(Model);
Model.BasicConsume(QueueName, autoAck, Consumer);
_ea = (BasicDeliverEventArgs)Consumer.Queue.Dequeue();
return Encoding.ASCII.GetString(_ea.Body);
}
public void Acknowledge()
{
Model.BasicAck(_ea.DeliveryTag, false);
}
I modify how I get messages from the queue and it seems blocking issue was fixed. Below is my code.
public string ReadOneAtTime()
{
Consumer = new QueueingBasicConsumer(Model);
var result = Model.BasicGet(QueueName, false);
if (result == null) return null;
DeliveryTag = result.DeliveryTag;
return Encoding.ASCII.GetString(result.Body);
}
public void Reject()
{
Model.BasicReject(DeliveryTag, true);
}
public void Acknowledge()
{
Model.BasicAck(DeliveryTag, false);
}
Going back to my original question, I added the QOS and noticed that other consumers can now get messages. However some are left unacknowledged and my program seems to hangup. Code changes are below:
public string ReadMessage()
{
Model.BasicQos(0, 1, false); // control prefetch
bool autoAck = false;
Consumer = new QueueingBasicConsumer(Model);
Model.BasicConsume(QueueName, autoAck, Consumer);
_ea = Consumer.Queue.Dequeue();
return Encoding.ASCII.GetString(_ea.Body);
}
public void AckConsume()
{
Model.BasicAck(_ea.DeliveryTag, false);
}
In Program.cs
private static void Consume(Receiver receiver)
{
int counter = 0;
while (true)
{
var message = receiver.ReadMessage();
if (message == null)
{
Console.WriteLine("NO message received.");
break;
}
else
{
counter++;
Console.WriteLine("Received: {0}", message);
receiver.AckConsume();
}
}
Console.WriteLine("Total message received {0}", counter);
}
I appreciate any comments and suggestions. Thanks!
Well, the rabbit provides infrastructure where one consumer can't lock/block other message consumer working with the same queue.
The behavior you faced with can be a result of couple of following issues:
The fact that you are not using auto ack mode on the channel leads you to situation where one consumer took the message and still didn't send approval (basic ack), meaning that the computation is still in progress and there is a chance that the consumer will fail to process this message and it should be kept in rabbit queue to prevent message loss (the total amount of messages will not change in management consule). During this period (from getting message to client code and till sending explicit acknowledge) the message is marked as being used by specific client and is not available to other consumers. However this doesn't prevent other consumers from taking other messages from the queue, if there are more mossages to take.
IMPORTANT: to prevent message loss with manual acknowledge make sure
to close the channel or sending nack in case of processing fault, to
prevent situation where your application took the message from queue,
failed to process it, removed from queue, and lost the message.
Another reason why other consumers can't work with the same queue is QOS - parameter of the channel where you declare how many messages should be pushed to client cache to improve dequeue operation latency (working with local cache). Your code example lackst this part of code, so I am just guessing. In case like this the QOS can be so big that there are all messages on server marked as belonging to one client and no other client can take any of those, exactly like with manual ack I've already described.
Hope this helps.
Is there a way to receive multiple message using a single synchronous call ?
When I know that there are N messages( N could be a small value less than 10) in the queue, then I should be able to do something like channel.basic_get(String queue, boolean autoAck , int numberofMsg ). I don't want to make multiple requests to the server .
RabbitMQ's basic.get doesn't support multiple messages unfortunately as seen in the docs. The preferred method to retrieve multiple messages is to use basic.consume which will push the messages to the client avoiding multiple round trips. acks are asynchronous so your client won't be waiting for the server to respond. basic.consume also has the benefit of allowing RabbitMQ to redeliver the message if the client disconnects, something that basic.get cannot do. This can be turned off as well setting no-ack to true.
Setting basic.qos prefetch-count will set the number of messages to push to the client at any time. If there isn't a message waiting on the client side (which would return immediately) client libraries tend to block with an optional timeout.
You can use a QueueingConsumer implementation of Consumer interface which allows you to retrieve several messages in a single request.
QueueingConsumer queueingConsumer = new QueueingConsumer(channel);
channel.basicConsume(plugin.getQueueName(), false, queueingConsumer);
for(int i = 0; i < 10; i++){
QueueingConsumer.Delivery delivery = queueingConsumer.nextDelivery(100);//read timeout in ms
if(delivery == null){
break;
}
}
Not an elegant solution and does not solve making multiple calls but you can use the MessageCount method. For example:
bool noAck = false;
var messageCount = channel.MessageCount("hello");
BasicGetResult result = null;
if (messageCount == 0)
{
// No messages available
}
else
{
while (messageCount > 0)
{
result = channel.BasicGet("hello", noAck);
var message = Encoding.UTF8.GetString(result.Body);
//process message .....
messageCount = channel.MessageCount("hello");
}
First declare instance of QueueingBasicConsumer() wich wraps the model.
From the model execute model.BasicConsume(QueueName, false, consumer)
Then implement a loop that will loop around messages from the queue which will then processing
Next line - consumer.Queue.Dequeue() method - waiting for the message to be received from the queue.
Then convert byte array to a string and display it.
Model.BasicAck() - release message from the queue to receive next message
And then on the server side can start waiting for the next message to come through:
public string GetMessagesByQueue(string QueueName)
{
var consumer = new QueueingBasicConsumer(_model);
_model.BasicConsume(QueueName, false, consumer);
string message = string.Empty;
while (Enabled)
{
//Get next message
var deliveryArgs = (BasicDeliverEventArgs)consumer.Queue.Dequeue();
//Serialize message
message = Encoding.Default.GetString(deliveryArgs.Body);
_model.BasicAck(deliveryArgs.DeliveryTag, false);
}
return message;
}