I want to webapplication and in the backend I open a new thread and start a listener.What I want ,open connection one time and rabbitmq listener start and when new message is coming ,it is processed in background.
What is the best approach for this?
Why do you need to listen events in web application?
Instead of it, write a windows service using topshelf and masstransit as a rabbitmq client.
You can connect to rabbitmq and register listener consumers in Program.cs as below:
IBusControl busControl = Bus.Factory.CreateUsingRabbitMq(cfg =>
{
IRabbitMqHost host = cfg.Host(new Uri(RabbitMQConstants.RabbitMQUri),
hst =>
{
hst.Username(RabbitMQConstants.RabbitMQUserName);
hst.Password(RabbitMQConstants.RabbitMQPassword);
});
cfg.ReceiveEndpoint(host,
RabbitMQConstants.YourQueueName,
endPointConfigurator => { endPointConfigurator.Consumer<SomeConsumer>(); });
});
And start bus configurator :
busControl.Start();
Example consumer:
public class SomeConsumer :
IConsumer<YourMessageClass>
{
public async Task Consume(ConsumeContext<YourMessageClass> context)
{
await Console.Out.WriteLineAsync($"Message consumed: {context.Message.YourValue}");
}
}
For more information and example take a look at offical masstransit documentation : http://masstransit-project.com/MassTransit/usage/message-consumers.html
Related
I'm trying to implement scheduling mechanism by the masstransit/rabbitmq.
I've added the configuration as stated in the docs:
Uri schedulerEndpoint = new (Constants.MassTransit.SchedulerEndpoint);
services.AddMassTransit(mtConfiguration =>
{
mtConfiguration.AddMessageScheduler(schedulerEndpoint);
mtConfiguration.AddSagaStateMachine<ArcStateMachine, ArcProcess>(typeof(ArcSagaDefinition))
.Endpoint(e => e.Name = massTransitConfiguration.SagaQueueName)
.MongoDbRepository(mongoDbConfiguration.ConnectionString, r =>
{
r.DatabaseName = mongoDbConfiguration.DbName;
r.CollectionName = mongoDbConfiguration.CollectionName;
});
mtConfiguration.UsingRabbitMq((context, cfg) =>
{
cfg.UseMessageScheduler(schedulerEndpoint);
cfg.Host(new Uri(rabbitMqConfiguration.Host), hst =>
{
hst.Username(rabbitMqConfiguration.Username);
hst.Password(rabbitMqConfiguration.Password);
});
cfg.ConfigureEndpoints(context);
});
});
Then I'm sending a scheduled message using the Bus:
DateTime messageScheduleTime = DateTime.UtcNow + TimeSpan.FromMinutes(1);
await _MessageScheduler.SchedulePublish<ScheduledMessage>(messageScheduleTime, new
{
ActivationId = context.Data.ActivationId
});
_MessageCheduler is the IMessageScheduler instance.
I do see the Scheduler queue receive the scheduled message and I see the correct scheduledTime property in it but the message does not reach the state machine whenever its schedule should fire. Seems like I'm missing something in the configuration or some MassTransit service that is not started.
Please, assist.
If you actually read the documentation you would see that UseDelayedMessageScheduler is the proper configuration to use RabbitMQ for scheduling. And AddDelayedMessageScheduler for the container-based IMessageScheduler registration.
I created a microservice application that microservices using MassTransit and RabbitMQ for communication.
Each microservice developed using clean architecture, so we have MediatR inside each microservice.
Is it possible to use MassTransit for inside communication as well? so I can use the same signature for all services and when I want to expose a service to be used inter-microservice, it will be doable with ease.
So MediatR used for intra-communication and RabbitMQ used for inter-communication, and whole universe is on MassTransit system.
[Update] My question is how we can configure consumers so some can be used for inside communication (via MediatR) and some can be used for external communication (via RabbitMQ) and easily change them from inside to outside.
[Update2] for example here is my MassTransit registration:
services.AddMassTransit(x =>
{
x.AddConsumers(Assembly.GetExecutingAssembly());
x.AddBus(provider =>
Bus.Factory.CreateUsingRabbitMq(cfg =>
{
cfg.Host(new Uri(config.RabbitMQ.Address), h =>
{
h.Username(config.RabbitMQ.Username);
h.Password(config.RabbitMQ.Password);
});
cfg.ReceiveEndpoint("my-queue", ep => { ep.ConfigureConsumers(provider); });
}));
x.AddMediator((provider, cfg) => { cfg.ConfigureConsumers(provider); });
});
How can I differ in internal communication and external communication? in other words, how can I register some consumers to MediatR and some to RabbitMQ?
They can be used together, and MassTransit has its own Mediator implementation as well so you can write your handlers once and use them either via the mediator or via a durable transport such as RabbitMQ.
There are videos available that take you through the capabilities, starting with mediator and moving to RabbitMQ.
I found that I should create a separate bus for each. then external services inherit from an interface like IExternalConsumer, so I can separate them form internal ones and add them to related bus:
UPDATED for version 7
// find consumers
var types = AssemblyTypeCache.FindTypes(new[]{Assembly.GetExecutingAssembly()},TypeMetadataCache.IsConsumerOrDefinition).GetAwaiter().GetResult();
var consumers = types.FindTypes(TypeClassification.Concrete | TypeClassification.Closed).ToArray();
var internals = new List<Type>();
var externals = new List<Type>();
foreach (Type type in consumers)
{
if (type.HasInterface<IExternalConsumer>())
externals.Add(type);
else
internals.Add(type);
}
services.AddMediator(x =>
{
x.AddConsumers(internals.ToArray());
x.ConfigureMediator((provider, cfg) => cfg.UseFluentValidation());
});
services.AddMassTransit<IExternalBus>(x =>
{
x.AddConsumers(externals.ToArray());
x.AddBus(provider =>
Bus.Factory.CreateUsingRabbitMq(cfg =>
{
cfg.Host(new Uri(config.RabbitMQ.Address), h =>
{
h.Username(config.RabbitMQ.Username);
h.Password(config.RabbitMQ.Password);
});
cfg.ReceiveEndpoint(apiProviderName, ep => { ep.ConfigureConsumers(provider); });
}));
});
services.AddMassTransitHostedService();
I'm working with 2 .NET Core console applications in a producer/consumer scenario with MassTransit/RabbitMQ. I need to ensure that even if NO consumers are up-and-running, the messages from the producer are still queued up successfully. That didn't seem to work with Publish() - the messages just disappeared, so I'm using Send() instead. The messages at least get queued up, but without any consumers running the messages all end up in the "_skipped" queue.
So that's my first question: is this the right approach based on the requirement (even if NO consumers are up-and-running, the messages from the producer are still queued up successfully)?
With Send(), my consumer does indeed work, but still many messages are falling through the cracks and getting dumped into to the "_skipped" queue. The consumer's logic is minimal (just logging the message at the moment) so it's not a long-running process.
So that's my second question: why are so many messages still getting dumped into the "_skipped" queue?
And that leads into my third question: does this mean my consumer needs to listen to the "_skipped" queue as well?
I am unsure what code you need to see for this question, but here's a screenshot from the RabbitMQ management UI:
Producer configuration:
static IHostBuilder CreateHostBuilder(string[] args)
{
return Host.CreateDefaultBuilder()
.ConfigureServices((hostContext, services) =>
{
services.Configure<ApplicationConfiguration>(hostContext.Configuration.GetSection(nameof(ApplicationConfiguration)));
services.AddMassTransit(cfg =>
{
cfg.AddBus(ConfigureBus);
});
services.AddHostedService<CardMessageProducer>();
})
.UseConsoleLifetime()
.UseSerilog();
}
static IBusControl ConfigureBus(IServiceProvider provider)
{
var options = provider.GetRequiredService<IOptions<ApplicationConfiguration>>().Value;
return Bus.Factory.CreateUsingRabbitMq(cfg =>
{
var host = cfg.Host(new Uri(options.RabbitMQ_ConnectionString), h =>
{
h.Username(options.RabbitMQ_Username);
h.Password(options.RabbitMQ_Password);
});
cfg.ReceiveEndpoint(host, typeof(CardMessage).FullName, e =>
{
EndpointConvention.Map<CardMessage>(e.InputAddress);
});
});
}
Producer code:
Bus.Send(message);
Consumer configuration:
static IHostBuilder CreateHostBuilder(string[] args)
{
return Host.CreateDefaultBuilder()
.ConfigureServices((hostContext, services) =>
{
services.AddSingleton<CardMessageConsumer>();
services.Configure<ApplicationConfiguration>(hostContext.Configuration.GetSection(nameof(ApplicationConfiguration)));
services.AddMassTransit(cfg =>
{
cfg.AddBus(ConfigureBus);
});
services.AddHostedService<MassTransitHostedService>();
})
.UseConsoleLifetime()
.UseSerilog();
}
static IBusControl ConfigureBus(IServiceProvider provider)
{
var options = provider.GetRequiredService<IOptions<ApplicationConfiguration>>().Value;
return Bus.Factory.CreateUsingRabbitMq(cfg =>
{
var host = cfg.Host(new Uri(options.RabbitMQ_ConnectionString), h =>
{
h.Username(options.RabbitMQ_Username);
h.Password(options.RabbitMQ_Password);
});
cfg.ReceiveEndpoint(host, typeof(CardMessage).FullName, e =>
{
e.Consumer<CardMessageConsumer>(provider);
});
//cfg.ReceiveEndpoint(host, typeof(CardMessage).FullName + "_skipped", e =>
//{
// e.Consumer<CardMessageConsumer>(provider);
//});
});
}
Consumer code:
class CardMessageConsumer : IConsumer<CardMessage>
{
private readonly ILogger<CardMessageConsumer> logger;
private readonly ApplicationConfiguration configuration;
private long counter;
public CardMessageConsumer(ILogger<CardMessageConsumer> logger, IOptions<ApplicationConfiguration> options)
{
this.logger = logger;
this.configuration = options.Value;
}
public async Task Consume(ConsumeContext<CardMessage> context)
{
this.counter++;
this.logger.LogTrace($"Message #{this.counter} consumed: {context.Message}");
}
}
In MassTransit, the _skipped queue is the implementation of the dead letter queue concept. Messages get there because they don't get consumed.
MassTransit with RMQ always delivers a message to an exchange, not to a queue. By default, each MassTransit endpoint creates (if there's no existing queue) a queue with the endpoint name, an exchange with the same name and binds them together. When the application has a configured consumer (or handler), an exchange for that message type (using the message type as the exchange name) also gets created and the endpoint exchange gets bound to the message type exchange. So, when you use Publish, the message is published to the message type exchange and gets delivered accordingly, using the endpoint binding (or multiple bindings). When you use Send, the message type exchange is not being used, so the message gets directly to the destination exchange. And, as #maldworth correctly stated, every MassTransit endpoint only expects to get messages that it can consume. If it doesn't know how to consume the message - the message is moved to the dead letter queue. This, as well as the poison message queue, are fundamental patterns of messaging.
If you need messages to queue up to be consumed later, the best way is to have the wiring set up, but the endpoint itself (I mean the application) should not be running. As soon as the application starts, it will consume all queued messages.
When the consumer starts the bus bus.Start(), one of the things it does is create all exchanges and queues for the transport. If you have a requirement that publish/send happens before the consumer, your only option is to run DeployTopologyOnly. Unfortunately this feature is not documented in official docs, but the unit tests are here: https://github.com/MassTransit/MassTransit/blob/develop/src/MassTransit.RabbitMqTransport.Tests/BuildTopology_Specs.cs
The skipped queue happens when messages are sent to a consumer that doesn't know how to process.
For example if you have a consumer that can process IConsumer<MyMessageA> which is on receive endpoint name "my-queue-a". But then your message producer does Send<MyMessageB>(Uri("my-queue-a")...), Well this is a problem. The consumer only understands the A, it doesn't know how to process B. And so it just moves it to a skipped queue and continues on.
In my case, the same queue listens to multiple consumers at the same time
I am currently working with micro service architecture and .net core.
Rabbit MQ + MassTransit are being used to send and receive data between the micro services.
I have a host application in IIS and from 2 separate browsers I send the same request to micro service and that microservice calls other service using RabbitMQ.
I expect to get 2 separate requests hitting the consumer but instead get an internal server error.
Startup:
services.AddScoped<OrderCompletedEventConsumer>();
services.AddMassTransit(x =>
{
x.AddConsumer<Controllers.OrderController>();
});
services.AddSingleton(provider => Bus.Factory.CreateUsingRabbitMq(cfg =>
{
var host = cfg.Host(new Uri("http://192.168.101.111:5672"),
"/", h =>
{
h.Username("Test");
h.Password("test");
});
cfg.ReceiveEndpoint(host, "TestQUE", e =>
{
e.Consumer<Controllers.OrderController>(provider);
});
}));
//Register Publish Endpoint of RabbitMQ bus service
services.AddSingleton<IPublishEndpoint>(provider => provider.GetRequiredService<IBusControl>());
//Register Send Endpoint of RabbitMQ bus service
services.AddSingleton<ISendEndpointProvider>(provider => provider.GetRequiredService<IBusControl>());
//Register Bus control for RabbitMQ
services.AddSingleton<IBus>(provider => provider.GetRequiredService<IBusControl>());
//Regster Bus Service hosting
services.AddSingleton<IHostedService, BusService>();
Request From One microservice:-
IRequestClient<IAddRequest<IOrder>, IAddResponse<IOrder>> orderClient =
new MessageRequestClient<IAddRequest<IOrder>, IAddResponse<IOrder>>(_bus,
EndpointAddress("orderQue"), TimeSpan.FromSeconds(Convert.ToDouble("150")));
var addResponse = orderClient.Request(new
{
entity = order
});
await Task.WhenAll(addResponse);
Consumer
public async Task Consume(ConsumeContext<IGetRequest<IOrder>> context)
{
// Operation and return result
await context.RespondAsync<IGetResposne<IOffice>>(new
{
// send result.
});
}
In the consumer the 2 separate requests from different browser arrive but both are unsuccessful. However if I do one request at a time then it will work, why is this?
Please give any idea, suggestion or hint.
Thank you,
I use this way to start subscribing to messages:
var client = BusClientFactory.CreateDefault();
client.SubscribeAsync<BasicMessage>(async (msg, context) =>
{
Console.WriteLine($"Recieved: {msg.Prop}.");
});
... but how do I "unsubscribe" and/or disconnect?
I have the code in a windows service and I want to be able to unsubscrite/disconnect when shuting down the Windows Service.