MassTransit 6.3.2 Consumer is never called when using IBusControl.ConnectConsumer - rabbitmq

When I attach consumers during intial message bus config, the consumers are called as expected.
When I attach the consumers after bus config, using ConnectConsumer the consumers are never called; The temporary queue/exchange is created, but it doesn't seen to know of the consumers that are supposed to be attached to that queue.
There is another service/consumer on the bus that is receiving Request messages being published here and publishing Response messages that should be consumed here.
Any idea why this is not working?
NOTE: I know that the "preferred" way of connecting consumers to the bus in the bus config (as in the working example); this is not an option for me as in practice, the bus is being creating/configed in a referenced assembly and the end-user programmers that are adding consumers to the bus do not have access to the bus configuration method. This is something that used to be trivial in version 2; it seems later version make such usecases much more difficult - not all use-cases have easy access to the bus creation/config methods.
Ex.
public class TestResponseConsumer : IConsumer<ITestResponse>
{
public Task Consume(ConsumeContext<ITestResponse> context)
{
Console.WriteLine("TestResponse received");
return Task.CompletedTask;
}
}
...
This works (consumer gets called):
public IBusControl ServiceBus;
public IntegrationTestsBase()
{
ServiceBus = Bus.Factory.CreateUsingRabbitMq(cfg =>
{
cfg.Host("vmdevrab-bld", "/", h => {
h.Username("guest");
h.Password("guest");
});
cfg.ReceiveEndpoint("Int_Test", e =>
{
e.Consumer<TestResponseConsumer>();
});
cfg.AutoStart = true;
});
ServiceBus.Start();
}
~IntegrationTestsBase()
{
ServiceBus.Stop();
}
}
This does not work:
[TestMethod]
public void Can_Receive_SampleResponse()
{
try
{
ITestRequest request = new TestRequest(Guid.NewGuid(), Guid.NewGuid(), Guid.NewGuid());
ServiceBus.ConnectConsumer<TestResponseConsumer>();
ServiceBus.Publish<ITestRequest>(request);
mre.WaitOne(60000);
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
Console.WriteLine(ex.StackTrace);
Assert.Fail();
}
}

It doesn't work because as explained in the documentation when you connect a consumer to the bus endpoint, no exchange bindings are created. Published messages will not be delivered to the bus endpoint.
If you want to connect consumers to the bus after it has been started, you should use ConnectReceiveEndpoint() instead, which is also covered in the documentation.
var handle = bus.ConnectReceiveEndpoint("secondary-queue", x =>
{
x.Consumer<TestResponseConsumer>();
})
var ready = await handle.Ready;
The endpoint can be stopped when it is no longer needed, otherwise it will be stopped when the bus is stopped.

Related

Is it possible to use the reactive redis client for subscribing to redis events (publish/subscribe) in quarkus?

There seems to exist a solution for the blocking redis client, but I couldn't find a working example for the reactive one.
In the following example the publishing works, but the listening doesn't (it's not invoked).
#Inject
ReactiveRedisClient redisClient;
#Inject
#RedisClientName("second")
ReactiveRedisClient redisClient2;
Uni<Void> onStart(#Observes StartupEvent ev) {
logger.info("subscribing to redis channel: {}", "test");
return redisClient2.subscribe(List.of("test")).map(response -> {
String responseString = response.toString();
logger.info("got redis event: {}", responseString);
return responseString;
}).replaceWithVoid();
}
#Scheduled(every = "2s")
Uni<Void> publishScheduled() {
return redisClient.publish("test", "ping").replaceWithVoid();
}

Issues reading events from RabbitMQ queue on Axon 3.3.5

Using Axon 3.3.5, I am trying to read events from an AMQP queue.
/**
* AMQP subscribing
*/
#Bean
SpringAMQPMessageSource notificationsEventsQueue(Serializer serializer) {
return new SpringAMQPMessageSource(serializer) {
#Override
#Transactional
#RabbitListener(id = "eventsQueue", queues = "notificationsEventsQueue")
public void onMessage(Message message, Channel channel) {
super.onMessage(message, channel);
}
};
}
#Autowired
public void configure(EventProcessingConfiguration conf, SpringAMQPMessageSource src) {
conf.registerSubscribingEventProcessor("notificationsServiceEventProcessor", c -> src);
}
I debugged the onMessage method and when a new message comes in, the eventProcessors list is always empty, so the message isn't processed by my application.
What am I missing out?
Event Handlers receive their events from the Event Bus by default. If you want handlers to receive events from another source (such as RabbitMQ), you need to explicitly assign that source to a processor, and assign that handler to that processor as well.
The easiest way is to put #ProcessingGroup('notificationsServiceEventProcessor') on the eventhandler class.

Second level retries in Rebus with Rabbitmq

I have a scenario where I call an api in one of my handlers and that Api can go down for like 6 hours per month. Therefore, I designed a retry logic with 1sec retry, 1 minute retry and a 6 hour retry. This all works fine but then I found that long delay retries are not a good option.Could you please give me your experience about this?
Thank you!
If I were you, I would use Rebus' ability to defer messages to the future to implement this functionality.
You will need to track the number of failed delivery attempts manually though, by attaching and updating headers on the deferred message.
Something like this should do the trick:
public class YourHandler : IHandleMessages<MakeExternalApiCall>
{
const string DeliveryAttemptHeaderKey = "delivery-attempt";
public YourHandler(IMessageContext context, IBus bus)
{
_context = context;
_bus = bus;
}
public async Task Handle(MakeExternalApiCall message)
{
try
{
await MakeCallToExternalWebApi();
}
catch(Exception exception)
{
var deliveryAttempt = GetDeliveryAttempt();
if (deliveryAttempt > 5)
{
await _bus.Advanced.TransportMessage.Forward("error");
}
else
{
var delay = GetNextDelay(deliveryAttempt);
var headers = new Dictionary<string, string> {
{DeliveryAttemptHeaderKey, (deliveryAttempt+1).ToString()}
};
await bus.Defer(delay.Value, message, headers);
}
}
}
int GetDeliveryAttempt() => _context.Headers.TryGetValue(DeliveryAttemptHeaderKey, out var deliveryAttempt)
? deliveryAttempt
: 0;
TimeSpan GetNextDelay() => ...
}
When running in production, please remember to configure some kind of persistent subscription storage – e.g. SQL Server – otherwise, your deferred messages will be lost in the event of a restart.
You can configure it like this (after having installed the Rebus.SqlServer package):
Configure.With(...)
.(...)
.Timeouts(t => t.StoreInSqlServer(...))
.Start();

MassTransit with RabbitMq Request/Response wrong reply address because of network segments

I have a web app that uses a request/response message in Masstransit.
This works on out test environment, no problem.
However on the customer deployment we face a problem. At the customer site we do have two network segments A and B. The component doing the database call is in segment A, the web app and the RabbitMq server in segment B.
Due to security restrictions the component in segment A has to go through a loadbalancer with a given address. The component itself can connect to RabbitMQ with Masstransit. So far so good.
The web component on segment B however uses the direct address for the RabbitMq server. When the web component now is starting the request/response call, I can see that the message arrives at the component in segment A.
However I see that the consumer tries to call the RabbitMQ server on the "wrong" address. It uses the address the web component uses to issue the request. However the component in segment A should reply on the "loadbalancer" address.
Is there a way to configure or tell the RespondAsync call to use the connection address configured for that component?
Of course the easiest would be to have the web component also connect through the loadbalancer, but due to the network segments/security setup the loadbalancer is only reachable from segment A.
Any input/help is appreciated.
I had a similar problem with rabbitmq federation. Here's what I did.
ResponseAddressSendObserver
class ResponseAddressSendObserver : ISendObserver
{
private readonly string _hostUriString;
public ResponseAddressSendObserver(string hostUriString)
{
_hostUriString = hostUriString;
}
public Task PreSend<T>(SendContext<T> context)
where T : class
{
if (context.ResponseAddress != null)
{
// Send relative response address alongside the message
context.Headers.Set("RelativeResponseAddress",
context.ResponseAddress.AbsoluteUri.Substring(_hostUriString.Length));
}
return Task.CompletedTask;
}
...
}
ResponseAddressConsumeFilter
class ResponseAddressConsumeFilter : IFilter<ConsumeContext>
{
private readonly string _hostUriString;
public ResponseAddressConsumeFilter(string hostUriString)
{
_hostUriString = hostUriString;
}
public Task Send(ConsumeContext context, IPipe<ConsumeContext> next)
{
var responseAddressOverride = GetResponseAddress(_hostUriString, context);
return next.Send(new ResponseAddressConsumeContext(responseAddressOverride, context));
}
public void Probe(ProbeContext context){}
private static Uri GetResponseAddress(string host, ConsumeContext context)
{
if (context.ResponseAddress == null)
return context.ResponseAddress;
object relativeResponseAddress;
if (!context.Headers.TryGetHeader("RelativeResponseAddress", out relativeResponseAddress) || !(relativeResponseAddress is string))
throw new InvalidOperationException("Message has ResponseAddress but doen't have RelativeResponseAddress header");
return new Uri(host + relativeResponseAddress);
}
}
ResponseAddressConsumeContext
class ResponseAddressConsumeContext : BaseConsumeContext
{
private readonly ConsumeContext _context;
public ResponseAddressConsumeContext(Uri responseAddressOverride, ConsumeContext context)
: base(context.ReceiveContext)
{
_context = context;
ResponseAddress = responseAddressOverride;
}
public override Uri ResponseAddress { get; }
public override bool TryGetMessage<T>(out ConsumeContext<T> consumeContext)
{
ConsumeContext<T> context;
if (_context.TryGetMessage(out context))
{
// the most hackish part in the whole arrangement
consumeContext = new MessageConsumeContext<T>(this, context.Message);
return true;
}
else
{
consumeContext = null;
return false;
}
}
// all other members just delegate to _context
}
And when configuring the bus
var result = MassTransit.Bus.Factory.CreateUsingRabbitMq(cfg =>
{
var host = cfg.Host(new Uri(hostAddress), h =>
{
h.Username(...);
h.Password(...);
});
cfg.UseFilter(new ResponseAddressConsumeFilter(hostAddress));
...
});
result.ConnectSendObserver(new ResponseAddressSendObserver(hostAddress));
So now relative response addresses are sent with the messages and used on the receiving side.
Using observers to modify anything is not recommended by the documentation, but should be fine in this case.
Maybe three is a better solution, but I haven't found one. HTH

Adding values to header in MassTransit.RabbitMq

I am using MassTransit 3.0.0.0 and I have a hard time understanding how to intercept messages in a Request-Response scenario on their way out and add some information to the headers field that I can read on the receiver's end.
I was looking at the Middleware, as recommended in the MassTransit docs - see Observers warning - but the context you get on the Send is just a Pipe context that doesn't have access to the Headers field so I cannot alter it. I used the sample provided in Middleware page.
I then, looked at IPublishInterceptor
public class X<T> : IPublishInterceptor<T> where T : class, PipeContext
{
public Task PostPublish(PublishContext<T> context)
{
return new Task(() => { });
}
public Task PostSend(PublishContext<T> context, SendContext<T> sendContext)
{
return new Task(() => { });
}
public Task PrePublish(PublishContext<T> context)
{
context.Headers.Set("ID", Guid.NewGuid().ToString());
return new Task(() => { });
}
public Task PreSend(PublishContext<T> context, SendContext<T> sendContext)
{
context.Headers.Set("ID", Guid.NewGuid().ToString());
return new Task(() => { });
}
}
Which is very clear and concise. However, I don't know where it is used and how to link it to the rest of the infrastructure. As it stands, this is just an interface that is not really linked to anything.
If you need to add headers when a message is being sent, you can add middleware components to either the Send or the Publish pipeline as shown below. Note that Send filters will apply to all messages, whereas Publish filters will only apply to messages which are published.
// execute a synchronous delegate on send
cfg.ConfigureSend(x => x.Execute(context => {}));
// execute a synchronous delegate on publish
cfg.ConfigurePublish(x => x.Execute(context => {}));
The middleware can be configured on either the bus or individual receive endpoints, and those configurations are local to where it's configured.
You can also add headers in the consumer class:
public async Task Consume(ConsumeContext<MyMessage> context)
{
....
await context.Publish<MyEvent>(new { Data = data }, c => AddHeaders(c));
}
public static void AddHeaders(PublishContext context)
{
context.Headers.Set("CausationId", context.MessageId);
}
http://masstransit-project.com/MassTransit/advanced/middleware/custom.html
Shows adding an extension method to make it clear what you're setup. That's a big help if it's an interceptor that will be used a lot, so it's clear that purpose. You can skip that step if you want.
Basically, just...
cfg.AddPipeSpecification(new X<MyMessage>());
When configuring the transport.