We consume a list of messages from the queue inside ActiveMQ broker and present it to the user so the user can select one message, we are using the .net client implementation based on AMQP.Net Lite (.net 2.1).
AMQ queue messages
If the user wants to select a specific message from the queue, not the one which is the next to consume from the queue, is there any possibility to access that message by the 'MessageId' parameter or some other property?
Something like the code bellow :
Message msgToSend = new Message();
msgToSend.Properties = new Properties() { MessageId = "8589942532"};
receiver.Accept(msgToSend);
The list of messages the user can see from the queue are not accepted so they are still visible in the queue. The idea is to accept the message when the user performese a specific action like clicking a 'save' button.
You can create additional consumer with filter expression, which naturally would filter by message ID, to receive that message. After receiving the message, this consumer would have to go as it would be of no use.
BTW, you can also use non-destructive queues or consumers to browse the queue without removing messages from it (at least this is available in Artemis, not sure about ActiveMQ Classic).
Related
any body can help me finding how can I send the message to differents queues (depending on business logic) to differents queues in RabbitMQ, using Masstransit
I have read the documentation I didn't found how I can specify the queue destination name
thank you
You might want to read the documentation again. If you want to send to a specific queue, you actually have to send to the exchange for that queue created by MassTransit.
Use the ISendEndpointProvider (or ConsumeContext):
Call await GetSendEndpoint(new Uri("exchange:name")) or await GetSendEndpoint(new Uri("queue:name")) to get the ISendEndpoint.
Call Send(...) to send the message to the exchange or queue.
I'm using Masstransit with RabbitMq to communicate between two web applications .net core.
In the publisher application I publish a message to an exchange 'producer'.
rabbitConfigurator.Message<TMessage>(x => x.SetEntityName("producer"));
In the same application I have a consumer-A of said messages and that works fine (the consumer has an exchange and a queue as recommended by rabbit/masstransit connected to the producer exchange)
rabbitConfigurator.ReceiveEndpoint("consumer-A", x =>
{
x.Consumer<TConsumer>(context);
x.Bind("producer");
});
In a second application I'm trying to setup another consumer-B of the same message type connected to the exchange 'producer'.
rabbitConfigurator.ReceiveEndpoint("consumer-B", x =>
{
x.Consumer<TConsumer>(context);
x.Bind("producer");
});
However messages destined for this second application are getting sent to a skipped queue. From what I can see this second consumer is correctly configured in RabbitMq though.
I can't see what I'm missing.
Edit: In the producing application I can also successfully receive messages in a second consumer. But from another web application it doesn't work - perhaps something to do with the fact that there's a differetn connection/channel ?
Messages sent to the producer exchange should be in a format that can be deserialized by MassTransit. If they aren't, you'll need to add a message deserializer which can support the message type. If it is application/json then you'll need to add the RawJsonMessageDeserializer to the receive endpoint.
If the messages are in the correct format, verify that you have a consumer on the receive endpoint that can consume that message type. Message types must match entirely, including namespace, to be consumed. More details are available in the documentation.
Requirement
A system undergoes some state change, and multiple other parts of the system has to know this(lets call them observers) so that they can perform some actions based on the current state, the actions of the observers are important, if some of the observers are not online(not listening currently due to some trouble, but will be back soon), the message should not be discarded till all the observers gets the message.
Trying to accomplish this with pub/sub model, here are my findings, (please correct if this understanding is wrong) -
The publisher creates an event on specific topic, and multiple subscribers can consume the same message. This model either provides no delivery guarantee(in redis), or delivery is guaranteed once(with messaging queues), ie. when one of the consumer acknowledges a message, the message is discarded(rabbitmq).
Example
A new Person Profile entity gets created in DB
Now,
A background verification service has to know this to trigger the verification process.
Subscriptions service has to know this to add default subscriptions to the user.
Now both the tasks are important, unrelated and can run in parallel.
Now In Queue model, if subscription service is down for some reason, a BG verification process acknowledges the message, the message will be removed from the queue, or if it is fire and forget like most of pub/sub, the delivery is anyhow not guaranteed for both the services.
One more point is both the tasks are unrelated and need not be triggered one after other.
In short, my need is to make sure all the consumers gets the same message and they should be able to acknowledge them individually, the message should be evicted only after all the consumers acknowledged it either of the above approaches doesn't do this.
Anything I am missing here ? How should I approach this problem ?
This scenario is explicitly supported by RabbitMQ's model, which separates "exchanges" from "queues":
A publisher always sends a message to an "exchange", which is just a stateless routing address; it doesn't need to know what queue(s) the message should end up in
A consumer always reads messages from a "queue", which contains its own copy of messages, regardless of where they originated
Multiple consumers can subscribe to the same queue, and each message will be delivered to exactly one consumer
Crucially, an exchange can route the same message to multiple queues, and each will receive a copy of the message
The key thing to understand here is that while we talk about consumers "subscribing" to a queue, the "subscription" part of a "pub-sub" setup is actually the routing from the exchange to the queue.
So a RabbitMQ pub-sub system might look like this:
A new Person Profile entity gets created in DB
This event is published as a message to an "events" topic exchange with a routing key of "entity.profile.created"
The exchange routes copies of the message to multiple queues:
A "verification_service" queue has been bound to this exchange to receive a copy of all messages matching "entity.profile.#"
A "subscription_setup_service" queue has been bound to this exchange to receive a copy of all messages matching "entity.profile.created"
The consuming scripts don't know anything about this routing, they just know that messages will appear in the queue for events that are relevant to them:
The verification service picks up the copy of the message on the "verification_service" queue, processes, and acknowledges it
The subscription setup service picks up the copy of the message on the "subscription_setup_service" queue, processes, and acknowledges it
If there are multiple consuming scripts looking at the same queue, they'll share the messages on that queue between them, but still completely independent of any other queue.
Here's a screenshot from this interactive visualisation tool that shows this scenario:
As you mentioned it is not something that you can control with Redis Pub/Sub data structure.
But you can do it easily with Redis Streams.
Streams will allow you to post messages using the XADD command and then control which consumers are dealing with the message and acknowledge that message has been processed.
You can look at these sample application that provides (in Java) example about:
posting and consuming messages
create multiple consumer groups
manage exceptions
Links:
Getting Started with Redis Streams and Java
Redis Streams in Action ( Project that shows how to use ADD/ACK/PENDING/CLAIM and build an error proof streaming application with Redis Streams and SpringData )
I am using RabbitMQ to queue up all the messages and send the messages as SMS to respective consumers. I am using a Direct exchange and I have correctly created a binding to a queue with a routing key. The problem is, when I try to publish a message, I get some activity in the Message rates chart, but the message doesn't show up in the queue
Could certainly use some help here. I am sure the binding is done correctly.
Am I missing some other configuration?
I would recommend to "use specific exchanges", not sending message without specified exchange. I had same issue, when I published it to amq.direct or amq.fanout it worked as I wanted to.
If your configuration is correct, and you also have an active consumer that listens to that queue, I don't think anything is wrong. Doesn't those metrics depicts that the event was published and then delivered and acknowledged by the consumer ? So of course you won't have any queued events since it was consumed as soon as it was published.
It looks like the message is delivered to a consumer (as you can see in the chart). Remove the consumer and try to publish the message again, and you will see that it ends up in the queue instead.
In my case I was creating custom queue so I had to provide custom queue ID as a routing key.
I am getting a little confused with NServiceBus. It seems like a lot of examples that I see, they always use publish() and subscribe(). What I am trying to do is that I have a publisher that polling from its queue and distributes the message to subscriber’s queue. The messages are being generated by other application and the body of message will contain a text, which will be parsed later.
Do I still need to call publish() and subsribe() to transfer the messages from publisher's queue to subscriber's queue? The way I understood was that I only need to configure the queue names in both config file and call LoadAllMessages() on subscriber side, will take above scenario. I don't even have to handle the message on the subscriber side.
Thanks.
Your Publisher will still need to call Publish. What this does is the Publisher then looks into Subscription Storage to find out who is interested in that message type. It then will send a message to each Subscriber. On the Subscriber side you need to implement message handlers to do something with those messages. This is done via implementing the IHandleMessages<T> interface in the Subscriber assembly. NSB will discover this and autowire everything up. Be aware by default, the Subscriber will subscriber to all message types. If you want to only subscribe to certain messages, use the .DoNotAutoSubscribe setting in the manual configuration.