Is there a way to define a "sorting" function depending on the priority in ActiveMQ (or any other queuing system)? I know I can set a priority number and that messages with higher priority will be sent before, but I would like to define a function so that I can compare the messages that are already waiting with the new one.
Something like a compareTo function that activeMQ will use to define which is the next message.
I need to take into account several parameters and depending on the time and the length of the message, the priority might change over time.
Is there a way to do this? I have not been able to find info about this online, but I guess this is quite a common problem, isn't it?
haven't seen any way to do this w/o dequeing/requeueing messages...based on your use case, you might be able to use the camel resequencer to consume in batches and reorder as you go (or populate another queue, etc)
Related
Can anyone explain in which cases I need to create multiple queues (one user -> one queue name), and when one queue name for all clients with different routing keys (one user -> one routing key) and why?
A user should not be able to read messages intended for another user.
I'm using direct exchange type.
First off I am going to assume that when you say "user" you are interchangeably referring to a consumer or producer, and they aren't the same thing so I would read up on that here in rabbitmq's simplest explanation. Walking through that tutorial will definitely help solidify your understanding of rabbit a bit more overall too, which is always good.
In any case, I would recommend doing this:
Create multiple queue's, each one linked to a single consumer. The reason for doing this instead of using a single queue with multiple is discussed here but if you don't want a bunch of programmer jargon, it pretty much says that a single queue is super slow because only one message can be consumed at a time from the queue.
Also, there is a built in "default exchange" that you can use instead of setting up another direct exchange which it sounds like you're putting effort into that you might not need to, obviously I'm not sure what you are doing but I would take that into consideration... hope this helps!
I'm working on creating DB with Redis.
One of my recruitments is that all the clients in the system will be able to listen to set events and get information about both key and value change.
I know that publishing value may be big(512 MB) but I know that in my system the size of value will not be more than 100 chars.
I have 3 possible solutions and I wonder which one will be better or consider other solutions:
1) After each set operation client will also publish it (PUB/SUB)
2)Edit setGenericCommand function to publish the value as well and use keyspace binding.
3)After client receive keyspace notification it will get the value with get operation.
I would like to understand which approach will be better?
Thank you!
So, 1st and foremost, remember that PubSub is at-most-once delivery. If you really need to process every change in the client, you should consider a more resilient way to do so.
That said, assuming you're ok with PubSub's promises, 1 is the simplest and I'd go with that. At most, I'd provide the clients with a Lua wrapper that combines the SET and PUBLISH commands. This, of course, removes the need to actually listen to Keyspace notifications as you basically implementing it yourself.
2 means hacking Redis, which is great but means you'll have to maintain your own which is meh--;
3 is also simple enough, but with 1 you get away with a single round trip instead of 2.
Another (4) approach is to write a custom module, but IMO too complex for this need. Go with 1 and Lua, and may the force be with you.
I have part of my application that receives string messages from remote clients and decodes these into a _Message class. I then want to pass these messages into a queue for immediate processing. The FIFO method is exactly what I require as I would particularly prefer to process these messages in order of receiving.
These messages come in fairly constant (24 hours a day, maybe 1 every couple of seconds or so...) so I need to ensure that I capture them all and no messages get lost or rejected.
Each _Message will then run through a Routine which will decode and action various parts of the message content.
Therefore, what would be the best way of handling a constant message pool? I started to go down the path of Queues with Queue.Enqueue and Queue.Dequeue but I'm not sure how to constantly poll for items within the Queue without affecting performance or resources.
I then came across ThreadPooling (something very new to me) which sounds like it could be down the right path, but I'm not 100% sure on how it works or how to set it up correctly.
Or....can I use ThreadPooling in conjunction with a Queue? And simply add items into my Queue and have the ThreadPool automatically detect new items?
Any help or guidance would be appreciated. Thanks
In out product we need to count occasionally the number of messages in a queue that hold a specific property value.
The straight forward way is to use a queue browser with a selector. But:
Is there a way to get this statistics without browsing all relevant messages? We don't need the data, only the count.
If we do go for the queue browser approach, will it also browse messages that were consumed by some consumer, but not acknowledged yet?
There is not guaranteed way to count messages in a queue based on a selector. There are a number of reasons for this one of which is that the JMS Spec does not require that a QueueBrowser even return every message on a Queue. ActiveMQ won't return more than the number of messages that it can fit into memory so if you have a deep queue you won't have an accurate count. None of the management tools will do this for you either since it's not really ActiveMQ's job to be a database.
Here's some articles on messaging anti-patterns that you should read so you are better prepared for adventures in messaging.
Article 1 and Article 2
I am considering using AMQP for an application where delivery order is paramount.
I cannot therefore use the normal re-delivery features, as undelivered messages are re-queued out of order.
It looks like what I must do is to leave the message on the queue until it has been processed, and then specifically delete it. It is then possible that the same message is processed twice in order, but that is easy to trap and deal with.
However, I don't see how to do this. What I am looking for is some sort of peek and delete message methods, giving me direct control, but they don't seem to exist.
Am I missing something, or trying to solve the problem in the wrong way?
You cannot have peek-and-delete in AMQP. Actually, you cannot browse the messages on a queue without consuming them and Rabbit does not provide any extension to enable this.
The general response to your problem is "Think very carefully if you actually need that in-order constraint", because, for instance, with that constraint in place, you cannot have multiple consumers on a queue.
I have been solving the same problem. In my solutions I have been wrapping the messages into one single message where the outer message has been processed first and then I have processed the remaining inner messages in the wrapped order. This has some disadvantages, for example big messages (once your wrapping hierarchy contains many messages), more difficult serialization, ..., but for me the solution was suitable enough.