In StartUp of the project, I make the following settings for MassTransit.ActiveMQ. But when I run, it creates two queues for me, one is event-listener and the other is called Generation.
When I publish information, the information goes into the queues generated by the system.
But I want the information to be published inside queue event-listener that I set.
Please guide me
services.AddMassTransit(x =>
{
x.AddConsumer<EventConsumer>();
x.UsingActiveMq((context, cfg) =>
{
cfg.Host("localhost", h =>
{
h.Username("admin");
h.Password("admin");
});
cfg.ReceiveEndpoint("event-listener", e =>
{
e.ConfigureConsumer<EventConsumer>(context);
});
});
});
MassTransit will only create queues for configured consumers, or explicitly configured receive endpoints. In the code above, the only queue created would be called event-listener. For each message type consumed by the consumer, a topic is created and a virtual topic consumer is created so that the receive endpoint can consume messages of each type.
When messages are published, a topic is created for each published message type.
If you want to send a message directly to a queue, instead of publishing:
var provider = serviceProvider.GetRequiredService<ISendEndpointProvider>();
var endpoint = await provider.GetSendEndpoint(new Uri("queue:event-listener"));
await endpoint.Send(...);
Related
I would like all the messaging for a particular saga to all take place on the same topic.
I set my saga up like so:
public void ConfigureServices(IServiceCollection services)
{
services.AddMassTransit(x =>
{
var machine = new MyStateMachine();
var repository = new InMemorySagaRepository<MyState>();
x.UsingAzureServiceBus((ctx,cfg) => {
cfg.Host(config.ServiceBusConnectionString);
cfg.SubscriptionEndpoint("mySub", "myTopic", e =>
{
e.StateMachineSaga(machine, repository);
});
});
});
}
In the state machine I do this:
Initially(
When(FirstEvent)
.PublishAsync(context => context.Init<SecondMessage>(new { TestParam = "test"}))
.TransitionTo(FirstState));
I would like the SecondMessage to be published on the topic 'myTopic' but instead MassTransit creates a new topic for this message.
MassTransit creates a topic per message type, and publishes messages to their corresponding topics. Assigning the same topic name to multiple message types is not recommended.
If you want to use a subscription endpoint for a saga, you would need to configure the saga on the corresponding topic for each event. By default, MassTransit will forward the message topics for each saga event to the receive endpoint queue.
An example topology is shown below:
I'm working with 2 .NET Core console applications in a producer/consumer scenario with MassTransit/RabbitMQ. I need to ensure that even if NO consumers are up-and-running, the messages from the producer are still queued up successfully. That didn't seem to work with Publish() - the messages just disappeared, so I'm using Send() instead. The messages at least get queued up, but without any consumers running the messages all end up in the "_skipped" queue.
So that's my first question: is this the right approach based on the requirement (even if NO consumers are up-and-running, the messages from the producer are still queued up successfully)?
With Send(), my consumer does indeed work, but still many messages are falling through the cracks and getting dumped into to the "_skipped" queue. The consumer's logic is minimal (just logging the message at the moment) so it's not a long-running process.
So that's my second question: why are so many messages still getting dumped into the "_skipped" queue?
And that leads into my third question: does this mean my consumer needs to listen to the "_skipped" queue as well?
I am unsure what code you need to see for this question, but here's a screenshot from the RabbitMQ management UI:
Producer configuration:
static IHostBuilder CreateHostBuilder(string[] args)
{
return Host.CreateDefaultBuilder()
.ConfigureServices((hostContext, services) =>
{
services.Configure<ApplicationConfiguration>(hostContext.Configuration.GetSection(nameof(ApplicationConfiguration)));
services.AddMassTransit(cfg =>
{
cfg.AddBus(ConfigureBus);
});
services.AddHostedService<CardMessageProducer>();
})
.UseConsoleLifetime()
.UseSerilog();
}
static IBusControl ConfigureBus(IServiceProvider provider)
{
var options = provider.GetRequiredService<IOptions<ApplicationConfiguration>>().Value;
return Bus.Factory.CreateUsingRabbitMq(cfg =>
{
var host = cfg.Host(new Uri(options.RabbitMQ_ConnectionString), h =>
{
h.Username(options.RabbitMQ_Username);
h.Password(options.RabbitMQ_Password);
});
cfg.ReceiveEndpoint(host, typeof(CardMessage).FullName, e =>
{
EndpointConvention.Map<CardMessage>(e.InputAddress);
});
});
}
Producer code:
Bus.Send(message);
Consumer configuration:
static IHostBuilder CreateHostBuilder(string[] args)
{
return Host.CreateDefaultBuilder()
.ConfigureServices((hostContext, services) =>
{
services.AddSingleton<CardMessageConsumer>();
services.Configure<ApplicationConfiguration>(hostContext.Configuration.GetSection(nameof(ApplicationConfiguration)));
services.AddMassTransit(cfg =>
{
cfg.AddBus(ConfigureBus);
});
services.AddHostedService<MassTransitHostedService>();
})
.UseConsoleLifetime()
.UseSerilog();
}
static IBusControl ConfigureBus(IServiceProvider provider)
{
var options = provider.GetRequiredService<IOptions<ApplicationConfiguration>>().Value;
return Bus.Factory.CreateUsingRabbitMq(cfg =>
{
var host = cfg.Host(new Uri(options.RabbitMQ_ConnectionString), h =>
{
h.Username(options.RabbitMQ_Username);
h.Password(options.RabbitMQ_Password);
});
cfg.ReceiveEndpoint(host, typeof(CardMessage).FullName, e =>
{
e.Consumer<CardMessageConsumer>(provider);
});
//cfg.ReceiveEndpoint(host, typeof(CardMessage).FullName + "_skipped", e =>
//{
// e.Consumer<CardMessageConsumer>(provider);
//});
});
}
Consumer code:
class CardMessageConsumer : IConsumer<CardMessage>
{
private readonly ILogger<CardMessageConsumer> logger;
private readonly ApplicationConfiguration configuration;
private long counter;
public CardMessageConsumer(ILogger<CardMessageConsumer> logger, IOptions<ApplicationConfiguration> options)
{
this.logger = logger;
this.configuration = options.Value;
}
public async Task Consume(ConsumeContext<CardMessage> context)
{
this.counter++;
this.logger.LogTrace($"Message #{this.counter} consumed: {context.Message}");
}
}
In MassTransit, the _skipped queue is the implementation of the dead letter queue concept. Messages get there because they don't get consumed.
MassTransit with RMQ always delivers a message to an exchange, not to a queue. By default, each MassTransit endpoint creates (if there's no existing queue) a queue with the endpoint name, an exchange with the same name and binds them together. When the application has a configured consumer (or handler), an exchange for that message type (using the message type as the exchange name) also gets created and the endpoint exchange gets bound to the message type exchange. So, when you use Publish, the message is published to the message type exchange and gets delivered accordingly, using the endpoint binding (or multiple bindings). When you use Send, the message type exchange is not being used, so the message gets directly to the destination exchange. And, as #maldworth correctly stated, every MassTransit endpoint only expects to get messages that it can consume. If it doesn't know how to consume the message - the message is moved to the dead letter queue. This, as well as the poison message queue, are fundamental patterns of messaging.
If you need messages to queue up to be consumed later, the best way is to have the wiring set up, but the endpoint itself (I mean the application) should not be running. As soon as the application starts, it will consume all queued messages.
When the consumer starts the bus bus.Start(), one of the things it does is create all exchanges and queues for the transport. If you have a requirement that publish/send happens before the consumer, your only option is to run DeployTopologyOnly. Unfortunately this feature is not documented in official docs, but the unit tests are here: https://github.com/MassTransit/MassTransit/blob/develop/src/MassTransit.RabbitMqTransport.Tests/BuildTopology_Specs.cs
The skipped queue happens when messages are sent to a consumer that doesn't know how to process.
For example if you have a consumer that can process IConsumer<MyMessageA> which is on receive endpoint name "my-queue-a". But then your message producer does Send<MyMessageB>(Uri("my-queue-a")...), Well this is a problem. The consumer only understands the A, it doesn't know how to process B. And so it just moves it to a skipped queue and continues on.
In my case, the same queue listens to multiple consumers at the same time
I use Masstransit in C# project.
I have a publisher and consumer services, and when both of them are up, then there are no problems. But if the consumer goes offline, published messages don't go to the queue. They just disappear.
The expected behavior is to keep messages in the queue until the consumer is started, and then send them to it. I've found several topics in google groups with same questions, but it wasn't clear for me how to solve that problem.
It seems strange to me that this functionality isn't provided out of the box because, in my understanding, it is the main purpose of RabbitMQ and MT.
The way I create publisher bus:
public static IBusControl CreateBus()
{
return Bus.Factory.CreateUsingRabbitMq(sbc =>
{
var host = sbc.Host(new Uri("rabbitmq://RMQ-TEST"), h =>
{
h.Username("test");
h.Password("test");
});
sbc.ReceiveEndpoint(host, "test_queue", ep =>
{
ep.Handler<IProductDescriptionChangedEvent>(
content => content.CompleteTask);
});
});
}
And the consumer:
public static void StartRmqBus()
{
var bus = Bus.Factory.CreateUsingRabbitMq(cfg =>
{
var host = cfg.Host(new Uri("rabbitmq://RMQ-TEST"), h =>
{
h.Username("test");
h.Password("test");
});
cfg.ReceiveEndpoint(host, "test_queue", ep =>
{
ep.Consumer<ProductChangedConsumer>();
});
});
bus.Start();
}
EDIT:
Here is one more interesting feature: if I stop both services and manually put a message to the queue via admin interface of MT, the message is waiting in test_queue. But when I start publisher or consumer service, it falls to test_queue_error queue.
You use the same queue for published and consumer, plus publisher has a consumer for this message type, as you pointed out in your own answer.
If your publisher does not consume messages, it is better to remove the receiving endpoint from it at all and then your service will be send-only.
If you have several services, where each of them need to have their own consumers for the same message type - this is how pub-sub works and you must have different queues per service. This is described in the Common Gotchas section of the documentation. In such scenario, each service will get it's own copy of the published message.
If you have one queue - you get competing consumers and this scenario is only valid for horizontal scalability, where you run several instance of the same services to increase the number of processed messages if the processing is too slow. In such case all these instances will consume messages from the same queue. In this scenario only one instance will get a message.
It seems like my publisher was set up incorrectly. After removing this part:
sbc.ReceiveEndpoint(host, "test_queue", ep =>
{
ep.Handler<IProductDescriptionChangedEvent>(
content => content.CompleteTask);
});
it started to work as expected. Looks like it consumed its own messages, that's why I didn't see messages in the queue when the consumer was down.
I'm using massTransit with RabbitMQ. Publishing messages with massTransit will create an exchange for my message type. And a masstransit consumer will create queues and bindings to an exchange. Great, make things easy.
Before looking at massTransit I used rabbitMQ's api to create queues, exchanges and binding. I would get both publisher and consumers to run the same setup code. So no matter who ran first all queues, exchanges and binding will be created no matter which part of the application ran first. This was great when running in a development environment.
I was wondering if something similar could be achieved with massTransit?
With MassTransit should be the same: consumers will create queues bound to the exchanges of the messages they consume (with names equal to the messages types).
Publishers will create the exchanges with same names of the types of the messages they publish.
Remember that if the messages published or consumed have super classes or implement interfaces, MassTransit will create the same hierarchy, creating and binding as many exchanges as your message class hierarchy has.
You could use HareDu 2 to achieve this with the below code. This works with both Autofac and .NET Core DI. Check the docs here: https://github.com/ahives/HareDu2
// Create a queue
var result = _container.Resolve<IBrokerObjectFactory>()
.Object<Queue>()
.Create(x =>
{
x.Queue("fake_queue");
x.Configure(c =>
{
c.IsDurable();
c.AutoDeleteWhenNotInUse();
c.HasArguments(arg =>
{
arg.SetQueueExpiration(1000);
arg.SetPerQueuedMessageExpiration(2000);
});
});
x.Targeting(t =>
{
t.VirtualHost("fake_vhost");
t.Node("fake_node");
});
});
// Create an exchange
var result = _container.Resolve<IBrokerObjectFactory>()
.Object<Exchange>()
.Create(x =>
{
x.Exchange("fake_exchange");
x.Configure(c =>
{
c.IsDurable();
c.IsForInternalUse();
c.HasRoutingType(ExchangeRoutingType.Fanout);
c.HasArguments(arg =>
{
arg.Set("fake_arg", "fake_arg_value");
});
});
x.Targeting(t => t.VirtualHost("fake_vhost"));
});
// Create a binding
var result = _container.Resolve<IBrokerObjectFactory>()
.Object<Binding>()
.Create(x =>
{
x.Binding(b =>
{
b.Source("fake_exchange");
b.Destination("fake_queue");
b.Type(BindingType.Exchange);
});
x.Configure(c =>
{
c.HasRoutingKey("your_routing_key");
c.HasArguments(arg =>
{
arg.Set("your_arg", "your_arg_value");
});
});
x.Targeting(t => t.VirtualHost("fake_vhost"));
});
MassTransit.3.1.2 MassTransit.Autofac.3.1.1 MassTransit.RabbitMQ.3.1.1 RabbitMQ.Client.3.6.0 Topshelf.3.3.1
One Topshelf Windows service, create a bus instance like this:
var builder = new ContainerBuilder();
builder.RegisterConsumers(Assembly.GetExecutingAssembly());
builder.Register<IBusControl>(context =>
{
return Bus.Factory.CreateUsingRabbitMq(rbmq =>
{
var host = rbmq.Host(new Uri("rabbitmq://" + BusConfig.Instance.Host + ":" + BusConfig.Instance.Port + "/" + BusConfig.Instance.VHost), h =>
{
h.Username(BusConfig.Instance.UserName);
h.Password(BusConfig.Instance.Password);
});
rbmq.UseJsonSerializer();
rbmq.UseNLog();
rbmq.ReceiveEndpoint(BusConfig.Instance.Queue, edp =>
{
edp.UseRetry(Retry.Incremental(5, TimeSpan.FromSeconds(5), TimeSpan.FromSeconds(5)));
edp.LoadFrom(context);
});
});
}).SingleInstance().As<IBusControl>().As<IBus>();
return builder.Build().Resolve<IBusControl>();
One console application like this ：
var bus = Bus.Factory.CreateUsingRabbitMq(rbmq =>
{
var host = rbmq.Host(new Uri("rabbitmq://" + BusConfig.Instance.Host + ":" + BusConfig.Instance.Port + "/" + BusConfig.Instance.VHost), h =>
{
h.Username(BusConfig.Instance.UserName);
h.Password(BusConfig.Instance.Password);
});
});
bus.Start();
var msg = new OrderCreatedMessage() { OrderId = 10102 };
bus.Publish<OrderCreatedMessage>(msg);
When mesasge published by console application ，Rabbitmq queue named by “BusConfig.Instance.Queue” received the OrderCreatedMessage：
Rabbitmq queue view when message published
Then i started the topshelf service ，The OrderCreatedMessage auto removed to _skipped queue ：
Rabbitmq queue view when topshelf service started
Masstransit log like this:
MOVE:rabbitmq:\/\/192.168.12.217:5672\/zst\/zst.order.queue?prefetch=8:rabbitmq:\/\/192.168.12.217:5672\/zst\/zst.order.queue_skipped?bind=true&queue=zst.order.queue_skipped:N\/A:Moved
But,When i publish message and consume the message with the same bus (Topshelf service), it works!!!
Any help or additional insight on this architecture would be appreciated!
Look at the bindings between the exchange for the published message type, and the exchange for the input queue of the service endpoint. Make sure that the proper type exchanges are bound correctly. Since the message is being delivered, I'm guessing that part is correct.
For the receive endpoint, it seems like the consumer was correct at one point (which explains why the binding exists) but is perhaps not consuming the correct message type currently. The message type must be the same message contract in the consumer and the publisher for the message to be consumed by the consumer.
When the message is moved to _skipped, there is no consumer on that endpoint actually consuming the message types in the message itself. I'd recommend posting the following output for review:
bus.GetProbeResult().ToJsonString()
This will show the consumers that are being registered and the message types which are being consumed. It will also help immensely in troubleshooting the issue you're seeing.
I guess its having problems creating the consumer for OrderCreatedMessage message.
Have you tried to just resolve the consumer on its own within the builder?
var test = builder.Resolve<OrderCreatedMessageConsumer>();
I'm just getting started with Masstransit and RabbitMQ. The answers above did send me in the right direction but for future reference. I was getting extra queues in RabbitMQ because the exhange had incorrect (duplicate bindings on) and I didn't notice this since the code seemed correct but the incorrect settings in RabbitMQ persisted. Deleting the queues and starting fresh solved it.