I'm using RabbitMQ with node-amqp lib. I'm publishing messages with mandatory flag set, and when there is no route to any queue, RabbitMQ responds with basic.return as in specification.
My problem is that, as far as I can tell, basic.return is asynchronous and does not contain any information about for which message no queue was found. Even when exchange is in confirm mode). How the hell am I supposed to tell which message was returned?
node-amqp emits 'basic-return' event on receiving the basic.return from amqp. Only thing of any use there is routing key. Since all messages with the same routing key are routed the same way. I assumed that once I get a basic.return about a specific routing key, all messages with this routing key can be considered undelivered
function deliver(routing_key, message, exchange, resolve, reject){
var failed_delivery = function(ret){
if(ret.routingKey == routing_key){
exchange.removeListener('basic-return', failed_delivery);
reject(new Error('failed to deliver'));
}
};
exchange.on('basic-return', failed_delivery);
exchange.publish(
routing_key,
message,
{ deliveryMode: 1, //non-persistent
mandatory: true
}, function(error_occurred, error){
exchange.removeListener('basic-return', failed_delivery);
if(error_occurred){
reject(error);
} else {
resolve();
}
});
}
I read the AMQP spec, because I've used the Basic Return without a problem before, but I'm also using the .NET client. I looked through the documentation on node-amqp, and I can't even see that it implements Basic.Return.
In any event, the server does respond with the full message when it could not be published. You may consider switching to a different Node.js library (for example, amqplib does have this feature (marked as Channel#on('return', function(msg) {...})).
Related
In StartUp of the project, I make the following settings for MassTransit.ActiveMQ. But when I run, it creates two queues for me, one is event-listener and the other is called Generation.
When I publish information, the information goes into the queues generated by the system.
But I want the information to be published inside queue event-listener that I set.
Please guide me
services.AddMassTransit(x =>
{
x.AddConsumer<EventConsumer>();
x.UsingActiveMq((context, cfg) =>
{
cfg.Host("localhost", h =>
{
h.Username("admin");
h.Password("admin");
});
cfg.ReceiveEndpoint("event-listener", e =>
{
e.ConfigureConsumer<EventConsumer>(context);
});
});
});
MassTransit will only create queues for configured consumers, or explicitly configured receive endpoints. In the code above, the only queue created would be called event-listener. For each message type consumed by the consumer, a topic is created and a virtual topic consumer is created so that the receive endpoint can consume messages of each type.
When messages are published, a topic is created for each published message type.
If you want to send a message directly to a queue, instead of publishing:
var provider = serviceProvider.GetRequiredService<ISendEndpointProvider>();
var endpoint = await provider.GetSendEndpoint(new Uri("queue:event-listener"));
await endpoint.Send(...);
We are migrating our Vertx code base from 3.6.3 to 3.8.1.
And I am not sure if the redis pub/sub and the event bus consumers still work together. I couldn't find any documentation in the new version either.
from the 3.5.1 documentation:
vertx.eventBus().<JsonObject>consumer("io.vertx.redis.channel1", received -> {
// do whatever you need to do with your message
JsonObject value = received.body().getJsonObject("value");
// the value is a JSON doc with the following properties
// channel - The channel to which this message was sent
// pattern - Pattern is present if you use psubscribe command and is the pattern that matched this message channel
// message - The message payload
});
RedisClient redis = RedisClient.create(vertx, new RedisOptions());
redis.subscribe("channel1", res -> {
if (res.succeeded()) {
// so something...
}
});
How does this work now with the new Redis client / RedisAPI?
The documentation is not very descriptive. (There s not a subscribe command.. nor a trace of channel1)
You should be able to register a handler for subscriptions io.vertx.redis.client.Redis#handler. Which will be called with an io.vertx.redis.client.Response reference (with corresponding information) for published messages.
Using SpringBoot.
I have created an TopicExchange which accepts messages and directs them to two queues based on a routingKey present in the message.
Messages are sent via :
rabbitTemplate.convertAndSend('in-out-topic', 'inbound.queue.route.key', payload)
Messages are received:
#RabbitListener(queues = "inbound-queue")
def onInboundMessage(def message) {
try {
log.debug("Received inbound message: ${message.messageId} on inbound queue listener", message)
} catch (Exception ex) {
log.error("Inbound message exception: ${ex.getMessage()}")
return;
}
return message.payload
}
But when my listener (consumer) receives a message I get the following exception:
org.springframework.amqp.AmqpException: Cannot determine ReplyTo message property value: Request message does not contain reply-to property, and no default response Exchange was set.
Should I create a dummy response exchange via RabbitMQ dashboard?
Hardcode a non existent replyTo property?
Configure the existing topicExchange or Queues somehow?
I just want the message being removed from the corresponding queue when consumed by my message listener.
Your problem is in the end of method, here:
return message.payload
If you really are not going to send reply and we indeed see that by expectations via convertAndSend(), then you shouldn’t return anything from the #RabbitListener method. Otherwise, as you are experiencing, the return from such a method is treated as an attempt to send a reply.
See more info in the Reference Manual: https://docs.spring.io/spring-amqp/docs/2.0.3.RELEASE/reference/html/_reference.html#async-annotation-driven. Pay attention to the Reply Management paragraph.
I use Masstransit in C# project.
I have a publisher and consumer services, and when both of them are up, then there are no problems. But if the consumer goes offline, published messages don't go to the queue. They just disappear.
The expected behavior is to keep messages in the queue until the consumer is started, and then send them to it. I've found several topics in google groups with same questions, but it wasn't clear for me how to solve that problem.
It seems strange to me that this functionality isn't provided out of the box because, in my understanding, it is the main purpose of RabbitMQ and MT.
The way I create publisher bus:
public static IBusControl CreateBus()
{
return Bus.Factory.CreateUsingRabbitMq(sbc =>
{
var host = sbc.Host(new Uri("rabbitmq://RMQ-TEST"), h =>
{
h.Username("test");
h.Password("test");
});
sbc.ReceiveEndpoint(host, "test_queue", ep =>
{
ep.Handler<IProductDescriptionChangedEvent>(
content => content.CompleteTask);
});
});
}
And the consumer:
public static void StartRmqBus()
{
var bus = Bus.Factory.CreateUsingRabbitMq(cfg =>
{
var host = cfg.Host(new Uri("rabbitmq://RMQ-TEST"), h =>
{
h.Username("test");
h.Password("test");
});
cfg.ReceiveEndpoint(host, "test_queue", ep =>
{
ep.Consumer<ProductChangedConsumer>();
});
});
bus.Start();
}
EDIT:
Here is one more interesting feature: if I stop both services and manually put a message to the queue via admin interface of MT, the message is waiting in test_queue. But when I start publisher or consumer service, it falls to test_queue_error queue.
You use the same queue for published and consumer, plus publisher has a consumer for this message type, as you pointed out in your own answer.
If your publisher does not consume messages, it is better to remove the receiving endpoint from it at all and then your service will be send-only.
If you have several services, where each of them need to have their own consumers for the same message type - this is how pub-sub works and you must have different queues per service. This is described in the Common Gotchas section of the documentation. In such scenario, each service will get it's own copy of the published message.
If you have one queue - you get competing consumers and this scenario is only valid for horizontal scalability, where you run several instance of the same services to increase the number of processed messages if the processing is too slow. In such case all these instances will consume messages from the same queue. In this scenario only one instance will get a message.
It seems like my publisher was set up incorrectly. After removing this part:
sbc.ReceiveEndpoint(host, "test_queue", ep =>
{
ep.Handler<IProductDescriptionChangedEvent>(
content => content.CompleteTask);
});
it started to work as expected. Looks like it consumed its own messages, that's why I didn't see messages in the queue when the consumer was down.
MassTransit.3.1.2 MassTransit.Autofac.3.1.1 MassTransit.RabbitMQ.3.1.1 RabbitMQ.Client.3.6.0 Topshelf.3.3.1
One Topshelf Windows service, create a bus instance like this:
var builder = new ContainerBuilder();
builder.RegisterConsumers(Assembly.GetExecutingAssembly());
builder.Register<IBusControl>(context =>
{
return Bus.Factory.CreateUsingRabbitMq(rbmq =>
{
var host = rbmq.Host(new Uri("rabbitmq://" + BusConfig.Instance.Host + ":" + BusConfig.Instance.Port + "/" + BusConfig.Instance.VHost), h =>
{
h.Username(BusConfig.Instance.UserName);
h.Password(BusConfig.Instance.Password);
});
rbmq.UseJsonSerializer();
rbmq.UseNLog();
rbmq.ReceiveEndpoint(BusConfig.Instance.Queue, edp =>
{
edp.UseRetry(Retry.Incremental(5, TimeSpan.FromSeconds(5), TimeSpan.FromSeconds(5)));
edp.LoadFrom(context);
});
});
}).SingleInstance().As<IBusControl>().As<IBus>();
return builder.Build().Resolve<IBusControl>();
One console application like this ：
var bus = Bus.Factory.CreateUsingRabbitMq(rbmq =>
{
var host = rbmq.Host(new Uri("rabbitmq://" + BusConfig.Instance.Host + ":" + BusConfig.Instance.Port + "/" + BusConfig.Instance.VHost), h =>
{
h.Username(BusConfig.Instance.UserName);
h.Password(BusConfig.Instance.Password);
});
});
bus.Start();
var msg = new OrderCreatedMessage() { OrderId = 10102 };
bus.Publish<OrderCreatedMessage>(msg);
When mesasge published by console application ，Rabbitmq queue named by “BusConfig.Instance.Queue” received the OrderCreatedMessage：
Rabbitmq queue view when message published
Then i started the topshelf service ，The OrderCreatedMessage auto removed to _skipped queue ：
Rabbitmq queue view when topshelf service started
Masstransit log like this:
MOVE:rabbitmq:\/\/192.168.12.217:5672\/zst\/zst.order.queue?prefetch=8:rabbitmq:\/\/192.168.12.217:5672\/zst\/zst.order.queue_skipped?bind=true&queue=zst.order.queue_skipped:N\/A:Moved
But,When i publish message and consume the message with the same bus (Topshelf service), it works!!!
Any help or additional insight on this architecture would be appreciated!
Look at the bindings between the exchange for the published message type, and the exchange for the input queue of the service endpoint. Make sure that the proper type exchanges are bound correctly. Since the message is being delivered, I'm guessing that part is correct.
For the receive endpoint, it seems like the consumer was correct at one point (which explains why the binding exists) but is perhaps not consuming the correct message type currently. The message type must be the same message contract in the consumer and the publisher for the message to be consumed by the consumer.
When the message is moved to _skipped, there is no consumer on that endpoint actually consuming the message types in the message itself. I'd recommend posting the following output for review:
bus.GetProbeResult().ToJsonString()
This will show the consumers that are being registered and the message types which are being consumed. It will also help immensely in troubleshooting the issue you're seeing.
I guess its having problems creating the consumer for OrderCreatedMessage message.
Have you tried to just resolve the consumer on its own within the builder?
var test = builder.Resolve<OrderCreatedMessageConsumer>();
I'm just getting started with Masstransit and RabbitMQ. The answers above did send me in the right direction but for future reference. I was getting extra queues in RabbitMQ because the exhange had incorrect (duplicate bindings on) and I didn't notice this since the code seemed correct but the incorrect settings in RabbitMQ persisted. Deleting the queues and starting fresh solved it.