Did I understood it right that in Windows Azure Queue storage a message can live for 7 days maximum?
What happens with the message after 7 days?
What happens with the message if you read it once during the 7 days and you don't delete it? I mean you read the message from the queue and set the visibility timeout to let's say 1 minute. What happens after you have read the message (without deleting it) and the 1 minutes timeout visibility expires? I know is back and available on the queue but what happens with the 7 days TTL of that message, is it renewed to 7 days from that point or is 7 days totally since message was created?
You can now opt-in to infinite TTL for Azure Storage Queue messages by specifying an expiry time of -1 seconds when initially queuing the message.
The time-to-live interval for the message, in seconds. Prior to version 2017-07-29, the maximum time-to-live allowed is 7 days. For version 2017-07-29 or later, the maximum time-to-live can be any positive number, as well as -1 indicating that the message does not expire. If this parameter is omitted, the default time-to-live is 7 days.
https://learn.microsoft.com/en-us/rest/api/storageservices/put-message#uri-parameters
Yes, you understood it correctly. Messages from queues are deleted automatically within 7 days time and there is nothing called 're-newed'. Once inserted into the queue storage, it will be deleted automatically and there is no way you can recover it back(there are no recovery tools either).
When adding a message, users specify a time to live (< 7 days) after which the message is automatically deleted if it still exists in the queue.
Maybe you need to use Windows Azure Service Bus Queue. Take a look at a great comparison between the two Queues available at Azure:
http://msdn.microsoft.com/en-us/library/windowsazure/hh767287.aspx
Yes 7 day is maximum. There is feeeback feature request here . You can go and comment
https://feedback.azure.com/forums/217298-storage/suggestions/4134167-infinite-ttl-for-queue-messages
Related
This morning I found that my rabbitmq instances does not have several queues that are usually there.
What I noticed is a pattern, that the remaining queues are the ones that had consumers attached to those.
The queues that are gone are mostly retry and DLQs. How does one investigate what happened? What do I look for and where?
Update:
This is my queue details :
The expires is your problem there, if the queue has not been used for some time, it will get dropped.
Also I would get rid of the message-ttl unless you want your messages to be dropped after certain amount of time.
More info here: http://www.rabbitmq.com/ttl.html
These are rabbitmq configuration settings you must change, here some more info https://www.rabbitmq.com/parameters.html
This is due to a feature of TTL or Time to Live for a queue .
As shown in this example below
The value against expires correspond to 28 days. So If a queue hasn't had any consumer for 28 days it gets deleted.
It could also be possible that the auto-delete property is set to true in which case the queue will automatically get deleted the moment the last consumer gets disconnected.
You should be able to get both these parameters in the rabbitmq console where you check the queue properties .
Currently, we have 2 systems that are communicating directly.
Service A continuously (but not in periodical manner) sends messages to service B. The Message is in simple Key/Value format. Key is an integer number and Value is current local date and time.
Service B, in order to decide whether to process the request, has this logic to examine the last incoming request; If there is a time difference against the system time (for each key) and the difference is more than 10 minutes, then it starts processing the request.
Now that we are bringing RabbitMQ into our solution, we need to revise this communication model as well. I was thinking to use a delayed message exchange for the 10 minutes time window, and then rewrite and reset the time (re-schedule for another 10 minutes) for duplicate messages incoming from service A.
Could share your ideas about this proposed solution?
Well, after reading the documents I'm certain that such logic should be implemented in application layer (in my situation consumer software)
We have a Java application that gets messages from rabbitmq using Spring AMQP.
For some of the queues, the number of consumers are not increasing resulting in slower messages delivery rate.
e.g. even though the max consumers is set to 50, number of consumers remained 6 for most of the time for the load of 9000 messages.
However, this is not the case with other queues. i..e consumers count reached till 35 for other queues.
We are using SimpleMessageListenerContainer's setMaxConcurrentConsumers API for setting max consumers.
Can someone please help me to understand this?
Configuration:
number of concurrent consumers: 4
number of max concurrent consumers: 50
When asking questions like this, you must always show configuration. Edit your question with complete details.
It depends on your configuration. By default, a new consumer is only added once every 10 seconds, and only if an existing consumer receives 10 messages without any gaps.
If that still doesn't answer your question, turn on DEBUG logging. If you can't figure it out from that, post the log (covering at least startConsumerMinInterval milliseconds) someplace like pastebin or dropbox.
I'm doing some stress tests on a saga that uses 2 timeouts. During the test about 21K saga's get created. So that would mean 42K timeouts, but I notice that the timeoutsdispatcher queue of the saga is getting flooded with 100's of thousands of messages until it crashes because the MSMQ storage limit is hit.
I'm seeing this behavior since I switched the persistence mechanism from RavenDB to SQL Server.
Does anyone have an idea what could be wrong?
Transport: MSMQ
Persistence: NHibernate
Packages used:
NHibernate version 4.0.4.4000
NServiceBus version 5.2.14
NServiceBus.Host version 6.0.0
NServiceBus.Log4Net version 1.0.0
NServiceBus.NHibernate version 6.2.7
Test setup:
* endpoint 1 is sending 22000 messages to endpoint 2.
* endpoint 2 hosts a saga that is started by that message.
* each saga publishes an event and then requests 2 timeouts: 1 at 4 minutes, 1 at 10 minutes.
Observed behavior:
* endpoint 1 sends the 22K messages in under a minute.
* endpoint 2 (the saga) processes 5 to 10 messages per second.
* after 4 minutes the first timeouts are fired, while endpoint 2 is still processing messages from its queue and thus is still creating new saga instances.
* from that moment on, the timeoutsdispatcher queue of the saga endpoint is getting filled with messages.
* after 10 minutes or so, the timeoutsdispatcher queue already contains over 170K messages and is still filling up.
* That continues until endpoint 2 crashes because the MSMQ storage limit is hit, or all messages from the input queue are processed. If the latter occurs first, the timeoutsdispatcher queue message count starts to decrease until it eventually reaches 0.
Did you perform the same stress test with RavenDB? And is SQL Server on a machine that's more-or-less equally powerful, with fast drives?
Update
Some checks for your saga
Is the [Unique] attribute used and is it used properly? In other words, do you use unique ids for every incoming message? So that every incoming message that is spawning 2 timeouts, will create a unique saga instance? If every incoming message is accessing the same Saga, this would be a great case for extremely limiting throughput. Imagine the Saga instance was created already once, else the explanation would become to complex. So Message1 comes in, tries to find the row in the database, finds and locks it. The second message comes in at the same time, finds the row but it's locked. It will go into retry. Message3 up until Message100 come in (if concurrency is set to 100) and all try to do the same thing, immediately failing. You can see this will limit throughput for a while :)
Are the correct indexes on your Saga table(s) and Timeout tables?
What is your maximum concurrency level set to?
Based on the number of message, you say you send 22k messages, resulting in 44k timeout messages. Image all of these timeouts are in MSMQ. Imagine messages are really, really small, like 1Kb. Header information added by NServiceBus might take up 2Kb. That's 44.000 times 3Kb is roughly 135 megabytes. So there's no way that can fill up a default MSMQ installation which has a quota of 1GB by default.
This probably means your deadletter queue is filled up completely. Find more information on MSMQ connectionstrings and set the appropriate connectionstring. For example
<connectionStrings>
<add name="NServiceBus/Transport"
connectionString="deadLetter=false;journal=false;"/>
</connectionStrings>
Messages with TimeToBeReceived attribute set (link) end up in deadletter queue. Also purging queues will make all messages go to deadletter queue. Unless you set the proper connectionstring.
I'm sending messages over GCM with TTL=15 - and they arrive just fine. Despite that fact, the developer console (where GCM messages can be tracked) show status=expired.
According to Google's docs, expired means:
Reached their time-to-live (TTL) and expired.
Am I doing something wrong? Perhaps I'm not acking the message on my Android app?
As Reference;
time_to_live: This parameter specifies how long (in seconds) the message should be
kept in GCM storage if the device is offline. The maximum time to live
supported is 4 weeks, and the default value is 4 weeks. For more
information, see Setting the lifespan of a message.
So 15 seconds are too short to track you may want to increase this value.