I need the broker to delete all messages that dont have any consumers waiting. I seem to recall that this should be a default behavior, shouldn't it?
There are two approcahes to define a time-to-live for ativeMQ messages,if in this time period message is not consumed then it will expire.
you can have two approaches for this
producer.setTimeToLive(Long timeToLiveInMilliSeconds)
Above approach will apply to all messages from this producer or if you want for specific messages you can do it like this
producer.send(Destination destination,Message message,int deliveryMode,int priority,long timeToLive)
apart from this,there is no default configuration that message gets deleted as soon as it is placed on a queue and there is no consumer for that queue
hope this helps!
Good luck!
Related
This morning I found that my rabbitmq instances does not have several queues that are usually there.
What I noticed is a pattern, that the remaining queues are the ones that had consumers attached to those.
The queues that are gone are mostly retry and DLQs. How does one investigate what happened? What do I look for and where?
Update:
This is my queue details :
The expires is your problem there, if the queue has not been used for some time, it will get dropped.
Also I would get rid of the message-ttl unless you want your messages to be dropped after certain amount of time.
More info here: http://www.rabbitmq.com/ttl.html
These are rabbitmq configuration settings you must change, here some more info https://www.rabbitmq.com/parameters.html
This is due to a feature of TTL or Time to Live for a queue .
As shown in this example below
The value against expires correspond to 28 days. So If a queue hasn't had any consumer for 28 days it gets deleted.
It could also be possible that the auto-delete property is set to true in which case the queue will automatically get deleted the moment the last consumer gets disconnected.
You should be able to get both these parameters in the rabbitmq console where you check the queue properties .
I'm trying to use RabbitMQ in a more unconventional way (though at this point i can pick any other message queue implementation if needed)
I have one queue (I can have more if needed) that where customers are fetching N messages asynchronous. After they do their work I send the results from the client to the db.
I have two problems: first I don't want that they will work on the same message, second I want to grantee that I wont lose messages in case that my customer will close the browser or just stop working.
I looked at the documentation and saw the TTL which was perfect for me if I could alter that message that got timeout isn't going to be deleted but to move to another queue. can't find a way to alter this.
Moreover I looked at the confirmation option which in the first glance looked what I wanted,that mechanism is working like this: when the consumer gets a message he send confirmation to queue, I thought I can delay this confirm and send it when the work is done on the client side.
my problem was that I can't program the queue that if any message didn't get confirm then return it to the queue (or to another).
I also find how to do a scheduled message but it didn't help either because I don't want that the message will be inserted to the queue in five min,I want that when a customer will receive a message it will be locked in the queue for 5 min until confirm to delete is set otherwise return it to the queue.
Can I do temporary queue that enables my mechanism?
If someone can help with one of the problems or suggest another architecture or option to do it in another MQ it would be great.
Resources:
confirmation:
http://www.rabbitmq.com/blog/2011/02/10/introducing-publisher-confirms/
post about locks but his problem was a batcher component:
Locks and batch fetch messages with RabbitMq
TTL:
https://www.rabbitmq.com/ttl.html
Schedule a message:
https://www.rabbitmq.com/blog/2015/04/16/scheduling-messages-with-rabbitmq/
my problem was that I can't program the queue that if any message
didnt get confirm then return it to the queue (or to another).
RabbitMQ does this anyhow, so all you have to do is switch off the auto-ack flag, you figured this out
I thought I can delay this confirm and send it when the work is done
on the client side.
so just send the ACK once you've finished with processing the message.
All the unacknowledged messages remain in the queue and are re-delivered to next consumer (or the same one when it's up again, depending on your setup)
I'm positive I'm missing a nuance of MassTranist and/or RabbitMQ, but how long do durable (permanent?) messages stay on queues?
The situation I'm thinking of is one in which all consumers of a certain type of event are unavailable - obviously when they come back up, you want them to be able to take the appropriate actions based on the events they "missed" while they were offline.
However, what about the case when a new consumer starts reading off of the same queue after days/months/years? Is that consumer now going to be pulling in all events since the beginning of time? I'm almost certain that's not the case, but how is durability balanced with timeliness?
As I know MassTransit doesn't control message lifetime. RabbitMQ doing the same, thus message will stay in queue forever. The only exception from this is request/response model in which you can set up timeout period in which you want accept response.
In common way if you need to control lifetime you can store creating time in the message and check it in consumers.
I'm trying to achieve load balancing between different types of messages. I would not know in advance what the messages coming in might be until they hit the queue. I know I can try resequencing the messages, but I was thinking that maybe if there was a way to have the various consumers round robin between either queues or between topics, this would solve my problem.
The main problem i'm trying to solve is that I have many services sending messages to one queue with many consumers feeding off one queue. I do not want one type of service monopolizing the entire worker cluster. Again I don't know in advance what the messages that are going to hit the queue are going to be.
To try to clearly repeat my question:
Is there a way to tell the consumers to round robin between either existing queues or topics?
Thank you in advance.
I found the answer to my question on another post just had to know to look there. I resolved my problem by not creating AMQ consumer but a JMS listener with a composite destination as specified in this post: jms-listener-dynamically-choose-destinations. It turns out the JMS listener automatically round robins though all the queues you assign to it.
Consumers on a Queue will already do round robin processing of the messages on the Queue. The one thing to keep in mind is consumer prefetch which can allow one consumer to grab many messages before others arrive on the Queue so you may need to adjust prefetch depending on your scenario.
Read up on the differences between Queue and Topic here.
I am starting with ActiveMQ and I have a usecase. i have n producers sending messages into a Queue Q1. I want to stop the delivery of messages (i.e. i do not want consumers to consume those messages). I want to store the messages for sometime without those being consumed.
I was looking at ways this can be achieved. These two things came into my mind based on what i browsed through.
Using Mirrored queues, so that I can wiretap the messages and save into a virtual queue.
Possibly stop consumers from doing a PULL on the queue.
Another dirty way of doing this is by making consumers not send ack messages once its consumed a message from the queue.
We are currently not happy with either of these.
Any other way you can suggest.
Thanks in advance.
If you always want message delivery to be delayed you can use the scheduler feature of ActiveMQ to delay delivery until a set time or a fixed delay etc.
Other strategies might also work but it really up to you to design something that fits your use case. You can try to use Apache Camel to define a route that implements the logic of your use case to either dispatch a message to a Queue or send it to the scheduler for delayed processing. It all really depends on your use case and requirements.