It is not HasOut.
I understand that when I send a message (SendFrame), it does not mean it is really sent, but only queued until successfully sending it.
So when I would like to send another message there is no point in doing so if the first one was not actually sent (for me). In short -- I would like to work in such loop: send a message if the outgoing queue is empty, repeat.
I know how to send a message, but how to check if the queue is empty?
I use DealerSockect if this makes a difference.
Related
I am trying some stuff about Reliability of message delivery using ConfirmCallback.
So far I've done these:
1.When I send a message, I save it in the db (There is a field called status that indicates whether the message reached the broker successfully). Message id will stored in correlationData.
2.Using ConfirmCallback, if ack, i will update Message#status to success.( I can get message id from CorrelationData )
3.Using timed tasks to find the message that was not sent successfully, and resend.
I wonder why ConfirmCallback#confirm#CorrelationData has only one property id rather than the entire message so that I can resend message immediately.(In this way I don't need to persist messages).
Is there any other way to ensure that the message is sent successfully?
Any Suggestions would be appreciated.
You can sub-class CorrelationData to add the message.
I have exchange and queue. Producer doesn't need consumption confirmation, but messages in some cases can be un-processable by consumer in current moment, because of lack of other data. Because of this, I want to return those messages to the end of queue. How to do this? Or is it done automatically when I reject message?
Flow:
Message1 gets consumed and creates some record in database.
Message2 gets consumed and checks if there is record in database, and if yes, then it updates the record. If there is no record in database, message should be returned to the end of queue.
So there is message ordering problem and in general situation I get messages in order, because most of components deliver their messages correctly. I want to solve potential situation, when Producer of Message1 wasn't able to put message to exchange immediately because of heavy load or other reason. In this situation, Message2 will be consumed first but there will be no sufficient informations in database to process it. I want this message to be returned back to queue, but be sure that this Message2 will go to the tail of queue. If it will go to head, I will get infinite loop if I use only one queue.
Side question is, if it's possible to track how many times consumers tried to process message but returned it. If there is possibility to put message to the tail of queue like I described before, but for some reason Producer of Message1 died, and there will be no Message1, I want to make Message2 dead after some number of retries or some time.
RabbitMQ always puts rejected messages at the head of the queue. To put them at the tail, you will have to publish them yourself (e.g. using RabbitTemplate). You can add a header with a count of retries.
I'm trying to use RabbitMQ in a more unconventional way (though at this point i can pick any other message queue implementation if needed)
I have one queue (I can have more if needed) that where customers are fetching N messages asynchronous. After they do their work I send the results from the client to the db.
I have two problems: first I don't want that they will work on the same message, second I want to grantee that I wont lose messages in case that my customer will close the browser or just stop working.
I looked at the documentation and saw the TTL which was perfect for me if I could alter that message that got timeout isn't going to be deleted but to move to another queue. can't find a way to alter this.
Moreover I looked at the confirmation option which in the first glance looked what I wanted,that mechanism is working like this: when the consumer gets a message he send confirmation to queue, I thought I can delay this confirm and send it when the work is done on the client side.
my problem was that I can't program the queue that if any message didn't get confirm then return it to the queue (or to another).
I also find how to do a scheduled message but it didn't help either because I don't want that the message will be inserted to the queue in five min,I want that when a customer will receive a message it will be locked in the queue for 5 min until confirm to delete is set otherwise return it to the queue.
Can I do temporary queue that enables my mechanism?
If someone can help with one of the problems or suggest another architecture or option to do it in another MQ it would be great.
Resources:
confirmation:
http://www.rabbitmq.com/blog/2011/02/10/introducing-publisher-confirms/
post about locks but his problem was a batcher component:
Locks and batch fetch messages with RabbitMq
TTL:
https://www.rabbitmq.com/ttl.html
Schedule a message:
https://www.rabbitmq.com/blog/2015/04/16/scheduling-messages-with-rabbitmq/
my problem was that I can't program the queue that if any message
didnt get confirm then return it to the queue (or to another).
RabbitMQ does this anyhow, so all you have to do is switch off the auto-ack flag, you figured this out
I thought I can delay this confirm and send it when the work is done
on the client side.
so just send the ACK once you've finished with processing the message.
All the unacknowledged messages remain in the queue and are re-delivered to next consumer (or the same one when it's up again, depending on your setup)
I have this issue, I want to know my rabbit is working great.
I am not gonna send the message, so, Im not 100% sure is being sent correctly. But the problem is this.
After all is configured and all....
I see at the RabbitMQ web manager
And when I supposedly send a message the I see activity on the "message rates" chart but nothing at the "queued messages" .
I frankly dont know whats going on, is it too fast that doesnt need to queue the messages? Or something is misconfigured?
Any idea of the difference?
Thanks.
In case RabbitMQ receive non-routable message it drop it. So while message was received, it was not queued.
You may configure Alternate Exchanges to catch such messages.
In my case,
Situation1:
when my Exchange in rabbitTemplate.convertAndSend was not set properly -- the message was not sent to the correct queue -- the Queued messages was empty all time.
however, Message rates is not zero, it does show there are message get sent.
Which correspond to what the other answer is saying:
In case RabbitMQ receive non-routable message it drop it.
Situation2:
when my Exchange in rabbitTemplate.convertAndSend was indeed set properly -- the message was sent to the correct queue -- the Queued messages was queuing up the message.
Everything seems fine.
Situation3:
(continue from Situation2)
And now, I turn on the receiver service which has the #RabbitListener.
The Queued messages immediately drops down to 0, and never goes up again.
But the transporting of messages is still working fine.
Situation4:
(continue from Situation2)
And now, I change the receiver service to use the rabbitTemplate.receiveAndConvert.
Which I manually receive the message from the queue every 2s by using a loop.
(message is also sent from sender service every 2s by using a loop, same as the situations before.)
Now, the Queued messages stays at constant -- a straight line
(depends on how many message you have queued up, in my case 1, before the receiver service is up, then it stays at 1).
Conclusion:
I suspect that, when the message is consumed too fast, the Queued messages will just show 0.
Which correspond to what the OP is saying:
is it too fast that doesnt need to queue the messages?
(or, I could screw up some setting in RabbitMQ and led to wrong conclusion. I dont think so, but idk, I am not familiar with RabbitMQ.)
I have a service which tasks worker processes via RabbitMQ. The messages are sent with a TTL, and the worker will not ack the message until it successfully completes the task sent in the message.
The tasking process will monitor workers for timeouts, and if a worker exceeds the timeout it will be terminated. Since the message isn't ack'd, the message is re-queued immediately and the next worker will pick up the message (this is useful in my scenario, as workers are unreliable and may fail but subsequent attempts typically succeed.
However, I would also like the ability to cancel a message. Terminating and re-creating the worker process is the normal procedure (it's single threaded, so I can't send a separate 'cancel' message to the worker). However, doing so leads to the message immediately re-queueing if the TTL has not been exceeded.
The only suggested solution I've found is here, which suggested a separate data source which checks if a message is still valid. However, that answer is both a) old and b) inconvenient.
Does RabbitMQ offer a means to cancel a message once it's been placed into the queue?
Unfortunately rabbitmq does not have a way to cancel a message.
Without the ability to send a "cancel" message to your consumer, you may have to do something like what that other post suggests.
Another option to consider: message processing should be idempotent. That is, processing the same message more than once should only cause the desired result to occur once (the first time it is processed).
Idempotence is often achieved through the use of a correlationid in messaging. You can attach a correlationid to your message, then check a database or other service to see if that message should still be processed. If you want to "cancel" the message, you would update the other database/service with that specific correlationid to say "this one has been processed already" or "has been canceled" or something like that.