Adding values to header in MassTransit.RabbitMq - rabbitmq

I am using MassTransit 3.0.0.0 and I have a hard time understanding how to intercept messages in a Request-Response scenario on their way out and add some information to the headers field that I can read on the receiver's end.
I was looking at the Middleware, as recommended in the MassTransit docs - see Observers warning - but the context you get on the Send is just a Pipe context that doesn't have access to the Headers field so I cannot alter it. I used the sample provided in Middleware page.
I then, looked at IPublishInterceptor
public class X<T> : IPublishInterceptor<T> where T : class, PipeContext
{
public Task PostPublish(PublishContext<T> context)
{
return new Task(() => { });
}
public Task PostSend(PublishContext<T> context, SendContext<T> sendContext)
{
return new Task(() => { });
}
public Task PrePublish(PublishContext<T> context)
{
context.Headers.Set("ID", Guid.NewGuid().ToString());
return new Task(() => { });
}
public Task PreSend(PublishContext<T> context, SendContext<T> sendContext)
{
context.Headers.Set("ID", Guid.NewGuid().ToString());
return new Task(() => { });
}
}
Which is very clear and concise. However, I don't know where it is used and how to link it to the rest of the infrastructure. As it stands, this is just an interface that is not really linked to anything.

If you need to add headers when a message is being sent, you can add middleware components to either the Send or the Publish pipeline as shown below. Note that Send filters will apply to all messages, whereas Publish filters will only apply to messages which are published.
// execute a synchronous delegate on send
cfg.ConfigureSend(x => x.Execute(context => {}));
// execute a synchronous delegate on publish
cfg.ConfigurePublish(x => x.Execute(context => {}));
The middleware can be configured on either the bus or individual receive endpoints, and those configurations are local to where it's configured.

You can also add headers in the consumer class:
public async Task Consume(ConsumeContext<MyMessage> context)
{
....
await context.Publish<MyEvent>(new { Data = data }, c => AddHeaders(c));
}
public static void AddHeaders(PublishContext context)
{
context.Headers.Set("CausationId", context.MessageId);
}

http://masstransit-project.com/MassTransit/advanced/middleware/custom.html
Shows adding an extension method to make it clear what you're setup. That's a big help if it's an interceptor that will be used a lot, so it's clear that purpose. You can skip that step if you want.
Basically, just...
cfg.AddPipeSpecification(new X<MyMessage>());
When configuring the transport.

Related

Why is JobConsumer not being hit/run?

I am trying out the new MassTransit IJobConsumer implementation, and although I've tried to follow the documentation, the JobConsumer I have written is never being run/hit.
I have:
created the JobConsumer which has a run method that runs the code I need it to
public class CalculationStartRunJobConsumer : IJobConsumer<ICalculationStartRun>
{
private readonly ICalculationRunQueue runQueue;
public CalculationStartRunJobConsumer(ICalculationRunQueue runQueue)
{
this.runQueue = runQueue;
}
public Task Run(JobContext<ICalculationStartRun> context)
{
return Task.Run(
() =>
{
var longRunningJob = new LongRunningJob<ICalculationStartRun>
{
Job = context.Job,
CancellationToken = context.CancellationToken,
JobId = context.JobId,
};
runQueue.StartSpecial(longRunningJob);
},
context.CancellationToken);
}
}
I have registered that consumer trying both ConnectReceiveEndpoint and AddConsumer
Configured the ServiceInstance as shown in the documentation
services.AddMassTransit(busRegistrationConfigurator =>
{
// TODO: Get rid of this ugly if statement.
if (consumerTypes != null)
{
foreach (var consumerType in consumerTypes)
{
busRegistrationConfigurator.AddConsumer(consumerType);
}
}
if(requestClientType != null)
{
busRegistrationConfigurator.AddRequestClient(requestClientType);
}
busRegistrationConfigurator.UsingRabbitMq((context, cfg) =>
{
cfg.UseNewtonsoftJsonSerializer();
cfg.UseNewtonsoftJsonDeserializer();
cfg.ConfigureNewtonsoftJsonSerializer(settings =>
{
// The serializer by default omits fields that are set to their default value, but this causes unintended effects
settings.NullValueHandling = NullValueHandling.Include;
settings.DefaultValueHandling = DefaultValueHandling.Include;
return settings;
});
cfg.Host(
messagingHostInfo.HostAddress,
hostConfigurator =>
{
hostConfigurator.Username(messagingHostInfo.UserName);
hostConfigurator.Password(messagingHostInfo.Password);
});
cfg.ServiceInstance(instance =>
{
instance.ConfigureJobServiceEndpoints(serviceCfg =>
{
serviceCfg.FinalizeCompleted = true;
});
instance.ConfigureEndpoints(context);
});
});
});
Seen that the queue for the job does appear in the queue for RabbitMQ
When I call .Send to send a message to that queue, it does not activate the Run method on the JobConsumer.
public async Task Send<T>(string queueName, T message) where T : class
{
var endpointUri = GetEndpointUri(messagingHostInfo.HostAddress, queueName);
var sendEndpoint = await bus.GetSendEndpoint(endpointUri);
await sendEndpoint.Send(message);
}
Can anyone help?
Software
MassTransit 8.0.2
MassTransit.RabbitMq 8.0.2
MassTransit.NewtonsoftJson 8.0.2
.NET6
Using in-memory for JobConsumer
The setup of any type of repository for long running jobs is missing. We needed to either:
explicitly specify that it was using InMemory (missing from the docs)
Setup saga repositories using e.g. EF Core.
As recommended by MassTransit, we went with the option of setting up saga repositories by implementing databases and interacting with them using EF Core.

Using Attribute and ActionFilters for logging request and response of controller and actions

I am trying to find an elegant way of logging every request and response in my Web API using Filters in Asp.net Core 3.1 rather than have them in each action and each controller.
Haven't found a nice solution that seems performable well to deploy in production.
I've been trying to do something like this (below) but no much success.
Any other suggestion would be appreciated.
public class LogFilter : IAsyncActionFilter
{
private readonly ILogger _logger;
public LogFilter(ILogger logger)
{
_logger = logger;
}
public async Task OnActionExecutionAsync(ActionExecutingContext context, ActionExecutionDelegate next)
{
var requestBodyData = context.ActionArguments["request"];
var responseBodyData = "";//how to get the response result
_logger.LogInformation($"{AppDomain.CurrentDomain.FriendlyName} Endpoint: {nameof(context.ActionDescriptor.DisplayName)} - Request Body: {requestBodyData}");
await next();
_logger.LogInformation($"{AppDomain.CurrentDomain.FriendlyName} Endpoint: {nameof(context.ActionDescriptor.DisplayName)} - Response Body: {responseBodyData}");
}
}
I think logging the response should be done in debugging mode only and really can be done at your service API (by using DI interception). That way you don't need to use IActionFilter which actually can provide you only a wrapper IActionResult which wraps the raw value from the action method (which is usually the result returned from your service API). Note that at the phase of action execution (starting & ending can be intercepted by using IActionFilter or IAsyncActionFilter), the HttpContext.Response may have not been fully written (because there are next phases that may write more data to it). So you cannot read the full response there. But here I suppose you mean reading the action result (later I'll show you how to read the actual full response body in a correct phase). When it comes to IActionResult, you have various kinds of IActionResult including custom ones. So it's hard to have a general solution to read the raw wrapped data (which may not even be exposed in some custom implementations). That means you need to target some specific well-known action results to handle it correctly. Here I introduce code to read JsonResult as an example:
public async Task OnActionExecutionAsync(ActionExecutingContext context, ActionExecutionDelegate next)
{
var requestBodyData = context.ActionArguments["request"];
_logger.LogInformation($"{AppDomain.CurrentDomain.FriendlyName} Endpoint: {nameof(context.ActionDescriptor.DisplayName)} - Request Body: {requestBodyData}");
var actionExecutedContext = await next();
var responseBodyData = "not supported result";
//sample for JsonResult
if(actionExecutedContext.Result is JsonResult jsonResult){
responseBodyData = JsonSerializer.Serialize(jsonResult.Value);
}
//check for other kinds of IActionResult if any ...
//...
_logger.LogInformation($"{AppDomain.CurrentDomain.FriendlyName} Endpoint: {nameof(context.ActionDescriptor.DisplayName)} - Response Body: {responseBodyData}");
}
IActionResult has a method called ExecuteResultAsync which can trigger the next processing phase (result execution). That's when the action result is fully written to the HttpContext.Response. So you can try creating a dummy pipeline (starting with a dummy ActionContext) on which to execute the action result and get the final data written to the response body. However that's what I can imagine in theory. It would be very complicated to go that way. Instead you can just use a custom IResultFilter or IAsyncResultFilter to try getting the response body there. Now there is one issue, the default HttpContext.Response.Body is an HttpResponseStream which does not support reading & seeking at all (CanRead & CanSeek are false), we can only write to that kind of stream. So there is a hacky way to help us mock in a readable stream (such as MemoryStream) before running the code that executes the result. After that we swap out the readable stream and swap back the original HttpResponseStream in after copying data from the readable stream to that stream. Here is an extension method to help achieve that:
public static class ResponseBodyCloningHttpContextExtensions
{
public static async Task<Stream> CloneBodyAsync(this HttpContext context, Func<Task> writeBody)
{
var readableStream = new MemoryStream();
var originalBody = context.Response.Body;
context.Response.Body = readableStream;
try
{
await writeBody();
readableStream.Position = 0;
await readableStream.CopyToAsync(originalBody);
readableStream.Position = 0;
}
finally
{
context.Response.Body = originalBody;
}
return readableStream;
}
}
Now we can use that extension method in an IAsyncResultFilter like this:
//this logs the result only, to write the log entry for starting/beginning the action
//you can rely on the IAsyncActionFilter as how you use it.
public class LoggingAsyncResultFilterAttribute : Attribute, IAsyncResultFilter
{
//missing code to inject _logger here ...
public async Task OnResultExecutionAsync(ResultExecutingContext context, ResultExecutionDelegate next)
{
var readableStream = await context.HttpContext.CloneBodyAsync(() => next());
//suppose the response body contains text-based content
using (var sr = new StreamReader(readableStream))
{
var responseText = await sr.ReadToEndAsync();
_logger.LogInformation($"{AppDomain.CurrentDomain.FriendlyName} Endpoint: {nameof(context.ActionDescriptor.DisplayName)} - Response Body: {responseText}");
}
}
}
You can also use an IAsyncResourceFilter instead, which can capture result written by IExceptionFilter. Or maybe the best, use an IAsyncAlwaysRunResultFilter which can capture the result in all cases.
I assume that you know how to register IAsyncActionFilter so you should know how to register IAsyncResultFilter as well as other kinds of filter. It's just the same.
starting with dotnet 6 asp has HTTP logging built in. Microsoft has taken into account redacting information and other important concepts that need to be considered when logging requests.
var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();
/* enabled HttpLogging with this line */
app.UseHttpLogging();
if (!app.Environment.IsDevelopment())
{
app.UseExceptionHandler("/Error");
}
app.UseStaticFiles();
app.MapGet("/", () => "Hello World!");
app.Run();
https://learn.microsoft.com/en-us/aspnet/core/fundamentals/http-logging/?view=aspnetcore-6.0#enabling-http-logging

Apikey authentication webapi core 2.1

I have some service that is going to send some messages to my endpoint. But I need to validate these messages by checking if the http header consist out of a fixed (for a period) api key and id. I can do this by check the header but I don't think this is good practice. Anybody a clue on how to verify that the message send from the service?
I have found something but it is for core2.2 and I need to use 2.1... (https://github.com/mihirdilip/aspnetcore-authentication-apiKey)
Thanks in advance
If you have quite a few endpoints, maybe even multiple controllers i would suggest writing a middleware to handle this.
But if this apikey check is only needed to one endpoint. Since you said "my endpoint".
I would recommend just checking the header value in the controller action/endpoint
Example:
[HttpGet]
public async Task<IActionResult> ExampleEndpoint() {
var headerValue = Request.Headers["Apikey"];
if(headerValue.Any() == false)
return BadRequest(); //401
//your endpoint code
return Ok(); //200
}
You can check the request header in custom middleware as shown here . Or you can use action filter to check the api key , see code sample here .
Like I said I want to do this via the middleware and not in the end of the http pipeline. In the meantime I figured out a solution, it is a simple one but it works.
I created a class called MiddelWareKeyValidation with the following async method:
public async Task Invoke(HttpContext context)
{
if (!context.Request.Headers.Keys.Contains("X-GCS-Signature") || !context.Request.Headers.Keys.Contains("X-GCS-KeyId"))
{
context.Response.StatusCode = 400;
await context.Response.WriteAsync("User Key is missing");
return;
}
else
{
var apiKey = new ApiKey { Signature = context.Request.Headers["X-GCS-Signature"], Key = context.Request.Headers["X-GCS-KeyId"] };
if (!ContactsRepo.CheckValidUserKey(apiKey))
{
context.Response.StatusCode = 401;
await context.Response.WriteAsync("Invalid User Key");
return;
}
}
await _next.Invoke(context);
}
Then I go to my Startup.cs in the Configure method where I add a new middleware like so:
app.UseMiddleware<MiddelWareKeyValidation>();
A good resource and credits goes to this article: https://www.mithunvp.com/write-custom-asp-net-core-middleware-web-api/

FluentValidation errors to Logger

I'm registering my FluentValidation validators as follows:
services.AddFluentValidation(fv => fv.RegisterValidatorsFromAssemblyContaining<CustomRequestValidator>());
How can I get a handle of any validation errors to log them via my logging implementation, e.g. Serilog?
For users stumbling into this problem (like me) I'd like to provide the solution I found.
The actual problem is that ASP.NET Core, and APIs in particular, make Model State errors automatically trigger a 400 error response, which doesn't log the validation errors.
Below that same documentation they include instructions on how to enable automatic logging for that feature which I believe should be the correct solution to this problem.
To jump onto what #mdarefull said, I did the following:
public void ConfigureServices(IServiceCollection services)
{
// Possible other service configurations
services.AddMvc()
.ConfigureApiBehaviorOptions(options =>
{
options.InvalidModelStateResponseFactory = context =>
{
if (!context.ModelState.IsValid)
{
LogAutomaticBadRequest(context);
}
return new BadRequestObjectResult(context.ModelState);
};
});
// Possible other service configurations
}
The LogAutomaticBadRequest method is as follows:
private static void LogAutomaticBadRequest(ActionContext context)
{
// Setup logger from DI - as explained in https://github.com/dotnet/AspNetCore.Docs/issues/12157
var loggerFactory = context.HttpContext.RequestServices.GetRequiredService<ILoggerFactory>();
var logger = loggerFactory.CreateLogger(context.ActionDescriptor.DisplayName);
// Get error messages
var errorMessages = string.Join(" | ", context.ModelState.Values
.SelectMany(x => x.Errors)
.Select(x => x.ErrorMessage));
var request = context.HttpContext.Request;
// Use whatever logging information you want
logger.LogError("Automatic Bad Request occurred." +
$"{System.Environment.NewLine}Error(s): {errorMessages}" +
$"{System.Environment.NewLine}|{request.Method}| Full URL: {request.Path}{request.QueryString}");
}

Resource Authorization with DTOs and Bus scenario

On an ASP.NET Core I have the following controller:
public class MessageApiController : Controller {
private readonly IMediator _mediator;
public MessageApiController(IMediator mediator) {
_mediator = mediator;
}
[HttpGet("messages")]
public async Task<IActionResult> Get(MessageGetQuery query) {
MessageGetReply reply = await _mediator.SendAsync(query);
return Ok(reply);
}
[HttpDelete("messages")]
public async Task<IActionResult> Delete(MessageDeleteModel model) {
MessageDeleteReply reply = await _mediator.SendAsync(model);
return Ok(reply);
}
}
I have handlers classes with a method handle to perform this actions:
GET (short code for sake of simplicity)
public async MessageGetReply Handle(MessageGetQuery query) {
IQueryable<Message> messages = _context.Messages.AsQueryable();
messages = messages.Include(x => x.Author).Include(x => x.Recipients);
// Omitted: Filter messages according to query
List<Message> result = await messages.ToListAsync();
// Omitted: Create MessageGetReply from result
} // Handle
DELETE (short code for sake of simplicity)
public async MessageDeleteReply Handle(MessageDeleteModel model) {
Message message = await _context.Messages.FirstOrDefaultAsync(x => x.Id == model.Id);
if (message != null) {
_context.Remove(message);
await _context.SaveChangesAsync();
}
// Omitted: Return reply
} // Handle
The authorization scenario is the following:
GET
1. The user must be authenticated
2. User.Id must equal Message.RecipientId;
DELETE
1. The user must be authenticated
2. User.Id must equal Message.AuthorId;
So I created the following resource authorization handler:
public class MessageAuthorizationHandler : AuthorizationHandler<OperationAuthorizationRequirement, Message> {
protected override void Handle(AuthorizationContext context, OperationAuthorizationRequirement requirement, Message resource) {
if (requirement == Operations.Delete) {
if (resource.AuthorId.ToString() == context.User.FindFirstValue(ClaimTypes.NameIdentifier))
context.Succeed(requirement);
}
if (requirement == Operations.Read) {
if (resource.RecipientId.ToString() == context.User.FindFirstValue(ClaimTypes.NameIdentifier)))
context.Succeed(requirement);
}
} // Handle
}
There are a few problems that arise:
In GET should I pass all messages to Authorization Handler?
In fact the messages are filtered by MessageGetQuery.AuthorId ...
So the authorization handler could receive MessageGetQuery.AuthorId and not a List ...
But feels strange since the resource is the List of Messages.
MessageGetQuery is simply a DTO.
Authorization could be coupled to DTOs (Query and Model) but does it make sense?
The problem arises when the DTO has less information than the Entity and I need that information to take decisions on authorization ...
If using the entities as resources I loose the ability to do projection:
IQueryable<Message> messages = _context.Messages.AsQueryable();
messages = messages.Include(x => x.Author).Include(x => x.Recipients);
// Omitted: Filter messages according to query
List<Message> result = await messages.ToListAsync();
// CALL authorization and send the resource messages ...
// Omitted: Create MessageGetReply from result
One solution would be to have a AuthorizationHandler for Read Messages, one AuthorizationHandler for Delete message ... The first one would take a list of messages, the second one message, ... I would however have to many classes.
Everything becomes simpler when using Entities directly in controllers and no DTOs but that, IMHO, should not be done ...
In RC2 we removed the object restriction in auth handlers, so you'll be able to use, for example, ints. So you could inject your repo into a handler and pull out your DTOs as you wish.