Why is JobConsumer not being hit/run? - rabbitmq

I am trying out the new MassTransit IJobConsumer implementation, and although I've tried to follow the documentation, the JobConsumer I have written is never being run/hit.
I have:
created the JobConsumer which has a run method that runs the code I need it to
public class CalculationStartRunJobConsumer : IJobConsumer<ICalculationStartRun>
{
private readonly ICalculationRunQueue runQueue;
public CalculationStartRunJobConsumer(ICalculationRunQueue runQueue)
{
this.runQueue = runQueue;
}
public Task Run(JobContext<ICalculationStartRun> context)
{
return Task.Run(
() =>
{
var longRunningJob = new LongRunningJob<ICalculationStartRun>
{
Job = context.Job,
CancellationToken = context.CancellationToken,
JobId = context.JobId,
};
runQueue.StartSpecial(longRunningJob);
},
context.CancellationToken);
}
}
I have registered that consumer trying both ConnectReceiveEndpoint and AddConsumer
Configured the ServiceInstance as shown in the documentation
services.AddMassTransit(busRegistrationConfigurator =>
{
// TODO: Get rid of this ugly if statement.
if (consumerTypes != null)
{
foreach (var consumerType in consumerTypes)
{
busRegistrationConfigurator.AddConsumer(consumerType);
}
}
if(requestClientType != null)
{
busRegistrationConfigurator.AddRequestClient(requestClientType);
}
busRegistrationConfigurator.UsingRabbitMq((context, cfg) =>
{
cfg.UseNewtonsoftJsonSerializer();
cfg.UseNewtonsoftJsonDeserializer();
cfg.ConfigureNewtonsoftJsonSerializer(settings =>
{
// The serializer by default omits fields that are set to their default value, but this causes unintended effects
settings.NullValueHandling = NullValueHandling.Include;
settings.DefaultValueHandling = DefaultValueHandling.Include;
return settings;
});
cfg.Host(
messagingHostInfo.HostAddress,
hostConfigurator =>
{
hostConfigurator.Username(messagingHostInfo.UserName);
hostConfigurator.Password(messagingHostInfo.Password);
});
cfg.ServiceInstance(instance =>
{
instance.ConfigureJobServiceEndpoints(serviceCfg =>
{
serviceCfg.FinalizeCompleted = true;
});
instance.ConfigureEndpoints(context);
});
});
});
Seen that the queue for the job does appear in the queue for RabbitMQ
When I call .Send to send a message to that queue, it does not activate the Run method on the JobConsumer.
public async Task Send<T>(string queueName, T message) where T : class
{
var endpointUri = GetEndpointUri(messagingHostInfo.HostAddress, queueName);
var sendEndpoint = await bus.GetSendEndpoint(endpointUri);
await sendEndpoint.Send(message);
}
Can anyone help?
Software
MassTransit 8.0.2
MassTransit.RabbitMq 8.0.2
MassTransit.NewtonsoftJson 8.0.2
.NET6
Using in-memory for JobConsumer

The setup of any type of repository for long running jobs is missing. We needed to either:
explicitly specify that it was using InMemory (missing from the docs)
Setup saga repositories using e.g. EF Core.
As recommended by MassTransit, we went with the option of setting up saga repositories by implementing databases and interacting with them using EF Core.

Related

MassTransit rabbitMq - Why are all my messages skipped

I am working on a .net core 3.1 app, for some reason my messages are not getting consumed.
service configuration :
services.AddMassTransit(x =>
{
x.AddConsumer<ItemAddedConsumer>();
x.AddBus(provider => Bus.Factory.CreateUsingRabbitMq(cfg =>
{
cfg.UseHealthCheck(provider);
cfg.Host(new Uri($"rabbitmq://{rMqSettings.host}:{rMqSettings.port}"), h =>
{
h.Username(rMqSettings.username);
h.Password(rMqSettings.password);
});
cfg.ReceiveEndpoint("items-service.ItemAdded", ep =>
{
ep.ConfigureConsumeTopology=false;
ep.Bind("ItemAdded");
ep.PrefetchCount = 15;
ep.Consumer<ItemAddedConsumer>(provider);
});
consumer class :
public class ItemAddedConsumer : IConsumer<ItemAdded>
{
private readonly IMediator _mediator;
public ItemAddedConsumer(IMediator mediator)
{
_mediator = mediator;
}
public async Task Consume(ConsumeContext<ItemAdded> context)
{
await _mediator.Send(new ItemAdded(context.Message.Id));
}
}
and this is how i am sending the messages :
Uri uri = new Uri("exchange:ItemAdded?bind=true&queue=items-service.ItemAdded");
var endPoint = await _bus.GetSendEndpoint(uri);
await endPoint.Send(#event);
all messages get sent to a new queue called items-service.ItemAdded_skipped
queues
Make sure the sent message and the consumer are using the same message type, including namespace, as outlined in the docs.
Also, why the overly complicated send endpoint address and receive endpoint configuration? You can change the EntityName of the message (via attribute or the publish topology) and simply use Publish from your message producer.

FluentValidation errors to Logger

I'm registering my FluentValidation validators as follows:
services.AddFluentValidation(fv => fv.RegisterValidatorsFromAssemblyContaining<CustomRequestValidator>());
How can I get a handle of any validation errors to log them via my logging implementation, e.g. Serilog?
For users stumbling into this problem (like me) I'd like to provide the solution I found.
The actual problem is that ASP.NET Core, and APIs in particular, make Model State errors automatically trigger a 400 error response, which doesn't log the validation errors.
Below that same documentation they include instructions on how to enable automatic logging for that feature which I believe should be the correct solution to this problem.
To jump onto what #mdarefull said, I did the following:
public void ConfigureServices(IServiceCollection services)
{
// Possible other service configurations
services.AddMvc()
.ConfigureApiBehaviorOptions(options =>
{
options.InvalidModelStateResponseFactory = context =>
{
if (!context.ModelState.IsValid)
{
LogAutomaticBadRequest(context);
}
return new BadRequestObjectResult(context.ModelState);
};
});
// Possible other service configurations
}
The LogAutomaticBadRequest method is as follows:
private static void LogAutomaticBadRequest(ActionContext context)
{
// Setup logger from DI - as explained in https://github.com/dotnet/AspNetCore.Docs/issues/12157
var loggerFactory = context.HttpContext.RequestServices.GetRequiredService<ILoggerFactory>();
var logger = loggerFactory.CreateLogger(context.ActionDescriptor.DisplayName);
// Get error messages
var errorMessages = string.Join(" | ", context.ModelState.Values
.SelectMany(x => x.Errors)
.Select(x => x.ErrorMessage));
var request = context.HttpContext.Request;
// Use whatever logging information you want
logger.LogError("Automatic Bad Request occurred." +
$"{System.Environment.NewLine}Error(s): {errorMessages}" +
$"{System.Environment.NewLine}|{request.Method}| Full URL: {request.Path}{request.QueryString}");
}

HttpParameterBinding without code coverage

OpenCover On Cake script does not detect coverage on my Owin.Testing usage applying HttpPArameterBiding to some ApiController action parameter.
I have Created a new type of my ApiController that as an action with my ParameterBindingAttribute that I called FromHeaderAttribute. After that I created my Owin Test Server and respective HttpClient and did the requests and the proper asserts to validate that the Binding is working properly. The tests pass with sucess.
This is my unit tests Cake Task
Task("UnitTests")
.IsDependentOn("Build")
.IsDependentOn("RestoreNugets")
.DoesForEach(GetFiles($"{testsPath}/**/*.csproj"), (file) =>
{
var openCoverSettings = new OpenCoverSettings
{
OldStyle = true,
MergeOutput = true,
Register = "user",
LogLevel = OpenCoverLogLevel.Verbose,
ArgumentCustomization = args => args.Append("-coverbytest:*.Tests.dll").Append("-mergebyhash")
}
.WithFilter("+[AppRootName.*]*");
var projectName = file.GetFilename().ToString().Replace(".csproj",string.Empty);
var dotNetTestSettings = new DotNetCoreTestSettings
{
Configuration = "Release",
DiagnosticOutput = true,
Verbosity = DotNetCoreVerbosity.Normal,
ArgumentCustomization = (args)=>
{
args.Append($"--logger \"trx;LogFileName={projectName}-TestsResults.trx\"");
args.Append("--filter \"TestCategory=Unit|Category=Unit\"");
return args;
}
};
OpenCover(context => context.DotNetCoreTest(file.FullPath, dotNetTestSettings), new FilePath($"CoverageResults.xml"), openCoverSettings);
})
.Finally(()=>
{
Information($"Copying test reports to ${outputDir}/TestsResults .... ");
CopyFiles($"{testsPath}/**/TestResults/*.trx",$"{outputDir}/TestsResults");
ReportGenerator($"*-CoverageResults.xml", $"{outputDir}/Reports");
});
this is my XUnit test:
[Fact]
[Trait("Category", "Unit")]
public async Task WhenHeadersArePresent_SettingsShouldBeSetted()
{
HttpConfiguration configuration = new HttpConfiguration();
var container = new SimpleInjector.Container();
Mock<IApiControllerValidation> mockValidationInterface = new Mock<IApiControllerValidation>();
ManualResetEvent resetEvent = new ManualResetEvent(false);
Settings settingsReceived = null;
mockValidationInterface.Setup((validator) => validator.Assert(It.IsAny<object>(), It.IsAny<object>(), It.IsAny<IDictionary<string, string>>(), It.IsAny<IHttpActionResult>()))
.Callback<object, object, IDictionary<string, string>, IHttpActionResult>((header, body, parameters, result) =>
{
settingsReceived = header as Settings;
resetEvent.Set();
});
container.RegisterInstance(mockValidationInterface.Object);
using (var server = TestServer.Create(app =>
{
configuration.DependencyResolver = new SimpleInjectorWebApiDependencyResolver(container);
configuration.MapHttpAttributeRoutes();
app.Use((owinContext, nextHandler)=> nextHandler());
app.UseWebApi(configuration);
}))
{
var client = server.HttpClient;
client.DefaultRequestHeaders.Add("header1", new List<string>() { "headervalue1" } );
client.DefaultRequestHeaders.Add("header2", new List<string>() { "headervalue2" });
var result = await client.PostAsync<Payload>("optionalHeader", new Payload("value1"), new JsonMediaTypeFormatter());
Assert.Equal(HttpStatusCode.OK,result.StatusCode);
};
resetEvent.WaitOne();
Assert.NotNull(settingsReceived);
Assert.Equal("headervalue1", settingsReceived.Header1);
Assert.Equal("headervalue2", settingsReceived.Header2);
}
And this is my Api Action were I want to test the FromHEader attribute that I have implement.
[HttpPost]
[Route("optionalHeader",Name = "PostValidation")]
public IHttpActionResult OptionalHeaders([FromHeader]Settings settings, [FromBody]Payload payload)
{
var result = Ok();
validation.Assert(settings,payload, null, result);
return result;
}
I expect that the code coverage of the test detects the usage of This type but its not because the report is showing 0 code coverage on my type.
I figured out what was the problem, and it was not related to anything related to asp.net framework HttpParameterBinding component.
instead of execute the code cover like this:
OpenCover(context => context.DotNetCoreTest(file.FullPath, dotNetTestSettings), new FilePath($"CoverageResults.xml"), openCoverSettings);
I changed that to be like this:
OpenCover(tool => {
tool.XUnit2($"{testsPath}/**/**/**/**/{projectName}.dll",xUnit2Settings);
}, new FilePath("./OpenCoverCoverageResults.xml"),openCoverSettings);
Also the Build must be done with configuration in debug mode so the OpenCover can use the pdbs.
The only thing that I still dont like is the path to the dlls to be explicit by the number of levels that are explicit and I also did not want to copy the dlls because that will take more time.

Second level retries in Rebus with Rabbitmq

I have a scenario where I call an api in one of my handlers and that Api can go down for like 6 hours per month. Therefore, I designed a retry logic with 1sec retry, 1 minute retry and a 6 hour retry. This all works fine but then I found that long delay retries are not a good option.Could you please give me your experience about this?
Thank you!
If I were you, I would use Rebus' ability to defer messages to the future to implement this functionality.
You will need to track the number of failed delivery attempts manually though, by attaching and updating headers on the deferred message.
Something like this should do the trick:
public class YourHandler : IHandleMessages<MakeExternalApiCall>
{
const string DeliveryAttemptHeaderKey = "delivery-attempt";
public YourHandler(IMessageContext context, IBus bus)
{
_context = context;
_bus = bus;
}
public async Task Handle(MakeExternalApiCall message)
{
try
{
await MakeCallToExternalWebApi();
}
catch(Exception exception)
{
var deliveryAttempt = GetDeliveryAttempt();
if (deliveryAttempt > 5)
{
await _bus.Advanced.TransportMessage.Forward("error");
}
else
{
var delay = GetNextDelay(deliveryAttempt);
var headers = new Dictionary<string, string> {
{DeliveryAttemptHeaderKey, (deliveryAttempt+1).ToString()}
};
await bus.Defer(delay.Value, message, headers);
}
}
}
int GetDeliveryAttempt() => _context.Headers.TryGetValue(DeliveryAttemptHeaderKey, out var deliveryAttempt)
? deliveryAttempt
: 0;
TimeSpan GetNextDelay() => ...
}
When running in production, please remember to configure some kind of persistent subscription storage – e.g. SQL Server – otherwise, your deferred messages will be lost in the event of a restart.
You can configure it like this (after having installed the Rebus.SqlServer package):
Configure.With(...)
.(...)
.Timeouts(t => t.StoreInSqlServer(...))
.Start();

NancyFx Authentication per Route

From what I saw in the source code RequiresAuthentication() does an Authentication check for the whole module. Is there any way to do this per Route?
I had the same problem. However it turns out the RequiresAuthentication works at both the module level and the route level. To demonstrate, here is some code ripped out my current project (not all routes shown for brevity).
public class RegisterModule : _BaseModule
{
public RegisterModule() : base("/register")
{
Get["/basic-details"] = _ => View["RegisterBasicDetailsView", Model];
Get["/select"] = _ =>
{
this.RequiresAuthentication();
return View["RegisterSelectView", Model];
};
}
}
Of course the only problem with doing it this way is that all the protected routes in the module need to call RequiresAuthentication. In the case of my module above, I have another 5 routes (not shown) all of which need protecting, so that makes six calls to RequiresAuthentication instead of one at the module level. The alternative would be to pull the unprotected route into another module, but my judgement was that a proliferation of modules is worse than the additional RequiresAuthentication calls.
namespace Kallist.Modules {
#region Namespaces
using System;
using Nancy;
#endregion
public static class ModuleExtensions {
#region Methods
public static Response WithAuthentication(this NancyModule module, Func<Response> executeAuthenticated) {
if ((module.Context.CurrentUser != null) && !string.IsNullOrWhiteSpace(module.Context.CurrentUser.UserName)) {
return executeAuthenticated();
}
return new Response { StatusCode = HttpStatusCode.Unauthorized };
}
#endregion
}
}
I ran into the same issue, here's how I solved it.
var module = new MyModule();
module.AddBeforeHookOrExecute(context => null, "Requires Authentication");
_browser = new Browser(with =>
{
with.Module(module);
with.RequestStartup((container, pipelines, ctx) =>
{
ctx.CurrentUser = new User { UserId = "1234", UserName = "test"};
});
});
I can now use this.RequiresAuthentication() at the module level and run my unit tests.