i would like to publish/produce a message with some message id, like each message has a particular id..And at the consumer end i would like to retrieve the messages by prividing the ID. suppose we have multiple consumers than each one should get only those messages which they requested through the message ids using java. (i hope i am clear enough).
Thanks
This is not exactly how a queue Functions you should be looking at Hashtables (key,value) stores like Redis.
A queue is not supposed to have look up features.
One way is the consumers consume messages and remove those only those messages from the Queue, whose UID they want.
Related
I am using ActiveMQ to store messages to be used later. It is working as expected, but there is a specific scenario I need to fit which I cannot figure out.
The short question is this.
Is there a way to run a query on the queue to find out all messages with certain header values?
The problem in detail is this :
So there is a set of data that is coming in multiple messages and the requirement is to use that data only after all messages for that has come in.
So if the dataset has lets say 50 messages i need to wait for those 50 messages and then read them in.
I am adding headers to each message to denote they belong to a certain set.
Like "TotalSets"=50 , "SetId"=39 .
I would like to write a thread that keeps tracing if all sets for a particular batch has arrived.
NMS is a .NET equivalent to the JMS messaging API so the means of filtering messages is the same as in JMS, your subscription applies a JMS Message Selector when created to tell the broker what messages it is interested in. The session methods to create MessageConsumer instances have variants that accept the selector using the JMS defined syntax which is your means of filtering messages.
i would like to know what is the best practice for processing messages from the queue if i need to make sure that another message (has same product Id) at the same time is not being processed by another consumer.
my problem is that if someone places and order with product ids and another message comes with same products i want to process them one by one, but not in parallel.
i am thinking of using redis where i would save the ids that are being processed and clear them after the processing is over. But maybe there are some better solutions for this kind of situation.
In rabbitmq, messages in same queue are ordered by "publish order". It means, messages will be received in the same order that they were sent to queue.
Take a look about ordering of messages : https://www.rabbitmq.com/semantics.html#ordering
If you want to consume messages exactly in publish order, you should use a "direct exchange" with one queue.
Also you must have exactly one consumer, with concurrency limit set to one.
I am looking for a way to achieve this.
say I have 10 nodes subscribed to the same queue. one of them publishes the message. but I want the message to be delivered to only one node among all connected.
Further, that node will decide if it wants more nodes to process the same thing then it will again publish it and get extra help.
Is it even possible with ActiveMQ? if not, what else is suitable for this requirement?
Thanks
This looks like the default behavior of an ActiveMQ queue. Just put the message there and one connected consumer will get it.
Don't confuse it with publish/subscribe, that is, topics. If you connect multiple subscribers to a topic, then all would get the same message.
I am reading RabbitMQ in Action book, stil in the chapter 2, but one thing authors says puzzling me. You setup a exchange and send a message, two subscribers are listening to the queue. When the first message comes in, the first subscriber gets it and the message is removed once it is acknowledged. When the next messages arrives it goes to the next listener in round robin way. I thought, if I am sending a message, I want all the subscribers to get it. Is my understanding wrong?
This is simple. If you want all subscribers to get a copy of the message, then create multiple queues with a wildcard binding.
Assuming that you have a topic exchange, and you publish all messages with a routing key like fred.interesting or fred.boring, then if each subscriber declares a queue with the binding key of fred.*, then each queue will get a copy of every message. The only issue is how to name the queues, although RabbitMQ can generate unique names for you if you wish.
If I were doing this I would have a supervisor process that starts and monitors the message consumer processes. The supervisor would assign each consumer process a queue name like fred0001, fred0002 and keep track of which names are in play. Using specified names like this makes it easier to use management tools or write management and monitoring scripts.
In our scenario I'm thinking of using the pub sub technique. However I don't know which is the better option.
1 ########
A web service of ours will publish a message that something has happened when it is called externally, ExternalPersonCreatedMessage!
This message will contain a field that represents the destinations to process the message into (multiple allowed).
Various subscribers will subscribe. These subscribers will filter the message to see if any action is required by checking the destination field.
2 ########
A web service of ours will parse the incoming call and publish specific types of messages depending on the destinations supplied in the field. i.e. many Destination[n]PersonCreatedMessage messages would be created.
Subscribers will subscribe to only the specific message they care for. i.e. not having to filter any messages
QUESTIONS
Which of the above is the better option and why? And how do I stop myself from making RequestMessages. From what I've read/seen I should be trying to structure this in a way of PersonCreated, PersonDeleted i.e. SOMETHING HAS HAPPENED and NOT in the REQUEST SOMETHING TO HAPPEN form such as CreatePerson or DeletePerson
Are my thoughts correct? I've been looking for guidance on how to structure messages and making sure I don't go down a wrong path but have found no guidance out there on do's and dont's. Can any one help and guide? I want to try and get this correct from the off :)
Based on the integration scenario in the referenced article, it appears to me that you may need a Saga to complete the workflow of accept message -> operate on message -> send confirmation. In the case that the confirmation is sent immediately after the operation, you could use NSBs message handler pipeline feature which allows you to chain handlers in a specified sequence such as...
First<FilterHandler>.Then<DoWorkHandler>().AndThen<SendConfirmationHandler>();
In terms of the content filtering, you can do this although you incur some transport overhead, meaning the queue will have to accept the message and the process will always call the first handler on every message(you can short-circuit the above pipeline at any point). It may be the case that what you really want is a Distributor/Worker setup where all Workers are the same and you can handle some load.
If you truly have different endpoints with completely different logic, then I would have the Publisher process(only accepts and Publishes message) do the work of translating the inbound message to something else a Subscriber can then be interested in. If then you find that a given Published message only ever has 1 Subscriber, then you don't need to Publish at all, you need to just Bus.Send() to the correct endpoint.
The way NServiceBus handles pub-sub is more like your option two.
A publisher service has an input queue and a subscription store.
A subscriber service has an input queue
The subscriber, on start-up will send a subscription message to the input queue of the publisher
The subscription message contains the type of message subscriber is interested in and the subscribers queue address
The publisher records the subscription in the subscription store.
The publisher receives a message.
The publisher evaluates the message type against the list of subscriptions
For each match found the publisher sends the message to the queue address.
In my opinion, you should stop thinking about destinations. Messages are messages. They should not have any inherent destination information in them. The subscription mechanism defines the addressing/routing requirements for the solution.