How to control log level in ASP.Net core Integration tests - asp.net-core

I am trying to limit the amount the log information being printed when running integration tests for asp.net core. Currently, everything down to debug level is being printed and it obscures useful information. I would really like to restrict it to warning and above.
I am running integration tests using the example at https://learn.microsoft.com/en-us/aspnet/core/test/integration-tests?view=aspnetcore-2.2#customize-webapplicationfactory
public class CustomWebApplicationFactory<TStartup>
: WebApplicationFactory<TStartup> where TStartup: class
{
protected override void ConfigureWebHost(IWebHostBuilder builder)
{
builder.ConfigureServices(services =>
{
// Create a new service provider.
var serviceProvider = new ServiceCollection()
.AddEntityFrameworkInMemoryDatabase()
.BuildServiceProvider();
// Add a database context (ApplicationDbContext) using an in-memory
// database for testing.
services.AddDbContext<ApplicationDbContext>(options =>
{
options.UseInMemoryDatabase("InMemoryDbForTesting");
options.UseInternalServiceProvider(serviceProvider);
});
// Build the service provider.
var sp = services.BuildServiceProvider();
// Create a scope to obtain a reference to the database
// context (ApplicationDbContext).
using (var scope = sp.CreateScope())
{
var scopedServices = scope.ServiceProvider;
var db = scopedServices.GetRequiredService<ApplicationDbContext>();
var logger = scopedServices
.GetRequiredService<ILogger<CustomWebApplicationFactory<TStartup>>>();
// Ensure the database is created.
db.Database.EnsureCreated();
try
{
// Seed the database with test data.
Utilities.InitializeDbForTests(db);
}
catch (Exception ex)
{
logger.LogError(ex, "An error occurred seeding the database. Error: {Message}", ex.Message);
}
}
});
}
}
Logs:
dbug: Microsoft.AspNetCore.Hosting.Internal.WebHost[4]
Hosting started
dbug: Microsoft.AspNetCore.Hosting.Internal.WebHost[0]
Loaded hosting startup assembly BackEnd.Api
info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
Request starting HTTP/2.0 GET http://localhost/test
dbug: Microsoft.AspNetCore.HostFiltering.HostFilteringMiddleware[0]
Wildcard detected, all requests with hosts will be allowed.
...lots more dbug logs...
Things I have tried in the ConfigureWebHost:
builder.ConfigureLogging(o =>
{
o.SetMinimumLevel(LogLevel.Warning);
});
No effect. I could not come up with any other combination of levels and filters that had any effect.
builder.ConfigureLogging(o =>
{
o.ClearProviders();
});
Stops all logging, but that's not really what I want.

Try to use AddFilter like
builder.ConfigureLogging(o=> {
//o.SetMinimumLevel(LogLevel.Warning);
o.AddFilter(logLevel => logLevel >= LogLevel.Warning);
});

I resolved this for my case. In the appsettings.json for the asp.net core app I was testing, was the following config:
"Console": {
"LogLevel": {
"Default": "Debug"
}
}
It seems that this would override any of the filtering or level changes I made in the code for the integration tests. Changing this level to Warning resolved my issue.

Related

Application Insights: Console Application HttpClient correlation not working

I have a Console Application which I plan to use Application Insights to start telemetry. This Console App calls a Web API within it.
Operation correlation works, but the Parent hierarchy does not. Essentially, the Parent of the Web API call is not the initial call from Console Application.
Below is my code:
Console App
static async Task SendHttpOnly()
{
//Create TelemetryClient
TelemetryConfiguration configuration = TelemetryConfiguration.CreateDefault();
configuration.InstrumentationKey = "<id>";
var telemetryClient = new TelemetryClient(configuration);
RequestTelemetry requestTelemetry = new RequestTelemetry { Name = "ConsoleTest" };
var operation = telemetryClient.StartOperation(requestTelemetry);
try
{
using (var client = new HttpClient())
{
client.BaseAddress = new Uri("http://localhost:37970/");
var responseTask = await client.PostAsJsonAsync<MessageDto>("MessageReceiver", new MessageDto() { Body = "Test" });
}
}
catch (Exception e)
{
operation.Telemetry.Success = false;
telemetryClient.TrackException(e);
throw;
}
finally
{
telemetryClient.StopOperation(operation);
telemetryClient.Flush();
Task.Delay(5000).Wait();
}
}
Web API
[HttpPost]
public string Post([FromBody] MessageDto dto)
{
_telemetryClient.TrackTrace($"Service Bus Message Processed: Message: {dto.Body}");
return $"Processed { dto.Body }";
}
Weird thing is, if I do a Web API to Web API call, it logs it properly. Even with the same code; the 2nd Web API call parent is the 1st Web API call.
Thankyou Water. Glad that you resolved your issue and posting the same as an answer so that it will be helpful for other community members.
Application HttpClient correlation not working because of using wrong Nuget package instead of using below package
Microsoft.ApplicationInsights.AspNetCore
We need to use the below package
Microsoft.ApplicationInsights.WorkerService
Below is the sample code for using SDK in application insights.
<ItemGroup>
<PackageReference Include="Microsoft.ApplicationInsights.WorkerService" Version="2.13.1" />
</ItemGroup>
For more information check the Application Insight worker service.

Programmatic configuration of NLog in ASP.NET Core application to filter unwanted chatter?

I'm attempting to filter out all the unnecessary chatter from the Microsoft hosting assemblies, while still allowing Debug-level messages from our own code to pass through to our logging targets.
Here's code that configures NLog and initializes a web app. Unfortunately, the Microsoft.* and System.* namespace filtering does not functional at all. The result is that all messages Debug and higher are logged.
I do not want to use the old nlog.config process. I want to be able to build the LoggingConfiguration in code.
I really must be missing something easy here, but I don't see it!
public static void Main(string[] args)
{
// Read configuration from a variety of sources to compose our final IConfiguration
var config = ReadOurConfigFromLotsOfPlaces();
var nLogConfig = new LoggingConfiguration();
var consoleTarget = new ColoredConsoleTarget("console")
{
AutoFlush = true,
ErrorStream = true,
Layout = #"${date:format=yyyy-MM-dd HH\:mm\:ss.fff} ${level} [${logger}] - ${message}${onexception:${newline}}${exception:format=shortType,message,stackTrace:maxInnerExceptionLevel=5}"
};
nLogConfig.AddTarget(consoleTarget);
nLogConfig.LoggingRules.Add(new LoggingRule("Microsoft.*", LogLevel.Warn, consoleTarget) {Final = true});
nLogConfig.LoggingRules.Add(new LoggingRule("System.*", LogLevel.Warn, consoleTarget) {Final = true});
nLogConfig.LoggingRules.Add(new LoggingRule("*", LogLevel.Debug, consoleTarget));
var logger = NLogBuilder.ConfigureNLog(nLogConfig).GetCurrentClassLogger();
logger.Debug("NLog initialization complete");
try
{
logger.Debug("init main");
CreateHostBuilder(config).Build().Run();
}
catch (Exception exception)
{
logger.Error(exception, "Stopped program because of exception");
throw;
}
finally
{
// Ensure to flush and stop internal timers/threads before application-exit (Avoid segmentation fault on Linux)
LogManager.Shutdown();
}
}
public static IWebHostBuilder CreateHostBuilder(IConfiguration config)
{
// we don't call CreateDefaultBuilder() because we already have assembled the configuration we want to use
return new WebHostBuilder()
.UseConfiguration(config)
.UseKestrel(options => options.UseSystemd())
.UseStartup<Startup>()
.ConfigureLogging(logging =>
{
logging.ClearProviders();
logging.SetMinimumLevel(Microsoft.Extensions.Logging.LogLevel.Trace);
})
.UseNLog();
}
Any tips would be greatly appreciated.
Ugh. I understand my problem now: The 'LoggingRules' is looking for a positive match between logger name and the min/max level values, so my first two LoggingRule objects did not match messages coming from the Microsoft.* and System.* namespaces, so the rules did nothing with those messages.
In order to accomplish the filtering I want, this is the solution:
var nLogConfig = new LoggingConfiguration();
var consoleTarget = new ColoredConsoleTarget("console")
{
AutoFlush = true,
ErrorStream = true,
Layout = #"${date:format=yyyy-MM-dd HH\:mm\:ss.fff} ${level} [${logger}] - ${message}${onexception:${newline}}${exception:format=shortType,message,stackTrace:maxInnerExceptionLevel=5}"
};
nLogConfig.AddTarget(consoleTarget);
var nullTarget = new NullTarget("null");
nLogConfig.AddTarget(nullTarget);
// matches every Microsoft.* logger with a LogLevel less than LogLevel.Warn
nLogConfig.LoggingRules.Add(new LoggingRule("Microsoft.*", LogLevel.Trace, LogLevel.Info, nullTarget) {Final = true});
// matches every System.* logger with a LogLevel less than LogLevel.Warn
nLogConfig.LoggingRules.Add(new LoggingRule("System.*", LogLevel.Trace, LogLevel.Info, nullTarget) {Final = true});
// and everything else, LogLevel.Debug and higher
nLogConfig.LoggingRules.Add(new LoggingRule("*", LogLevel.Debug, consoleTarget));

Blazor Web Assembly App with Azure B2C is always trying to authenticate as soon as page is loaded

I am adding support for Azure AD B2C to a Blazor WebAssembly App, I followed the instructions here
https://learn.microsoft.com/en-us/aspnet/core/blazor/security/webassembly/hosted-with-azure-active-directory-b2c?view=aspnetcore-3.1#client-app-configuration
however, the application is always trying to authenticate as soon as I load the page,
which does not allow for a public anonymous section of the site.
Is there any solution to this problem?
The default httpClient requires authorization so even making a call to see if a person is authorized causes the code to prompt the user to log in kicks in.
So to get around this, in the Program.cs file (in the Client project), I created a httpClient that allows anonymous requests
// This allows anonymous requests
// See: https://learn.microsoft.com/en-us/aspnet/core/security/blazor/webassembly/additional-scenarios?view=aspnetcore-3.1#unauthenticated-or-unauthorized-web-api-requests-in-an-app-with-a-secure-default-client
builder.Services.AddHttpClient("ServerAPI.NoAuthenticationClient", client => client.BaseAddress = new Uri(builder.HostEnvironment.BaseAddress));
This example should help:
https://github.com/ADefWebserver/SyncfusionHelpDeskClient/blob/main/Client/Pages/Index.razor
It calls the NoAuthenticationClient httpClient
protected override void OnInitialized()
{
// Create a httpClient to use for non-authenticated calls
NoAuthenticationClient =
ClientFactory.CreateClient(
"ServerAPI.NoAuthenticationClient");
}
public async Task HandleValidSubmit(EditContext context)
{
try
{
// Save the new Help Desk Ticket
// Create a new GUID for this Help Desk Ticket
objHelpDeskTicket.TicketGuid =
System.Guid.NewGuid().ToString();
await NoAuthenticationClient.PostAsJsonAsync(
"SyncfusionHelpDesk", objHelpDeskTicket);
// Send Email
HelpDeskEmail objHelpDeskEmail = new HelpDeskEmail();
objHelpDeskEmail.EmailType = "Help Desk Ticket Created";
objHelpDeskEmail.EmailAddress = "";
objHelpDeskEmail.TicketGuid = objHelpDeskTicket.TicketGuid;
await NoAuthenticationClient.PostAsJsonAsync(
"Email", objHelpDeskEmail);
// Clear the form
objHelpDeskTicket = new HelpDeskTicket();
// Show the Toast
ToastContent = "Saved!";
StateHasChanged();
await this.ToastObj.Show();
}
catch (Exception ex)
{
ToastContent = ex.Message;
StateHasChanged();
await this.ToastObj.Show();
}
}

NServiceBus Send-Only endpoints not generating heartbeats

I'm using NServiceBus.Core v6.4.3 and NServiceBus.Heartbeat v2.0.0
I have a console application running as a Scheduled Task, it extracts data and send commands to an endpoint for processing.
The console application is configured as a SendOnly endpoint.
My code is as follows:
Main
// Local NServiceBus Configuration
var endpointConfiguration = EndpointConfiguration();
// Global NServiceBus & Ninject configuration
var conventions = new NServiceBusConventions();
conventions.Customize(endpointConfiguration);
// Create and start endpoint
var endpointInstance = await Endpoint.Start(endpointConfiguration).ConfigureAwait(false);
EndpointConfiguration
private static EndpointConfiguration EndpointConfiguration()
{
var configuration = new EndpointConfiguration("EndpointName");
// To ensure OctopusDeploy doesn't cause ServicePulse to think multiple services have been deployed
// http://docs.particular.net/nservicebus/hosting/override-hostid
configuration.UniquelyIdentifyRunningInstance()
.UsingNames("EndpointName", Environment.MachineName);
configuration.SendOnly();
return configuration;
}
Conventions
public class NServiceBusConventions
{
public IKernel Kernel;
public void Customize(EndpointConfiguration configuration)
{
// Custom Logging Factory implementation
LogManager.UseFactory(new NServiceBusTraceLoggerFactory());
Kernel = NinjectCommon.Start();
configuration.UseContainer<NinjectBuilder>(b => b.ExistingKernel(Kernel));
configuration.UsePersistence<NHibernatePersistence>();
configuration.UseSerialization<JsonSerializer>();
configuration.UseTransport<MsmqTransport>();
var transport = configuration.UseTransport<MsmqTransport>();
// Enabled by default in MsmqTransport, but to ensure we have it
transport.Transactions(TransportTransactionMode.TransactionScope);
configuration.DefineCriticalErrorAction(NServiceBusOnCriticalError.OnCriticalError);
configuration.EnableInstallers();
configuration.Conventions()
.DefiningCommandsAs(t => t.Namespace != null && t.Namespace.Equals("Contracts.Commands"))
.DefiningEventsAs(t => t.Namespace != null && t.Namespace.Equals("Contracts.Interfaces.Events"));
configuration.AuditProcessedMessagesTo(ConfigurationManager.AppSettings["Messaging.NServiceBus.QueueNames.AuditQueue"]);
configuration.SendFailedMessagesTo(ConfigurationManager.AppSettings["Messaging.NServiceBus.QueueNames.ErrorQueue"]);
configuration.SendHeartbeatTo(ConfigurationManager.AppSettings["Messaging.NServiceBus.QueueNames.ServiceControlQueue"]);
var scanner = configuration.AssemblyScanner();
var excludeRegexs = new List<string>
{
#"DevExpress.*\.dll"
};
var baseDirectory = AppDomain.CurrentDomain.BaseDirectory;
foreach (var fileName in Directory.EnumerateFiles(baseDirectory, "*.dll").Select(Path.GetFileName))
{
foreach (var pattern in excludeRegexs)
{
if (Regex.IsMatch(fileName, pattern, RegexOptions.IgnoreCase))
{
scanner.ExcludeAssemblies(fileName);
break;
}
}
}
}
}
Removing the configuration.SendOnly(); line in EndpointConfiguration makes the endpoint appear in ServicePulse, but it doesn't appear otherwise.
I knew this was an issue in previous versions, but I thought this had been fixed in NServiceBus V5.
I don't have to configure the endpoint as Send-Only, but I was just for completeness.
The reason behind the missing heartbeats was:
I have a console application running as a Scheduled Task, it extracts
data and send commands to an endpoint for processing.
The process would start fresh each time and the time taken to extract data, process and send the commands was too short for NServiceBus to get the heartbeat messages sent.
Putting an await Task.Delay(10000) at the end of the application was enough to allow NServiceBus to complete its necessary bootstrapping and didn't impact our SLA.
Thanks to Sean Farmar for his help in diagnosing

NServiceBus test client not receiving Messages

I am very new to NService Bus, so I am trying to get it working with a simple test solution using LearningPersistence, obviously this will be changed soon!
So I have 3 projects:
IceDataExtractor - Client which sends a message
IceProcessManager - Processes messages
Messages - Contains a single Message class Messages
I am using the standard code generated by NServiceBus.Bootstrap.WindowsService 2.0.1
Here is page I used as to get sample
I then modified as follows
Ice Data Extractor
private async Task AsyncOnStart()
{
try
{
var endpointConfiguration = new EndpointConfiguration("IceDataExtractor");
var transport = endpointConfiguration.UseTransport<LearningTransport>();
transport.Routing().RouteToEndpoint(typeof(TestMessage), "IceProcessManager");
endpointConfiguration.UseSerialization<JsonSerializer>();
//TODO: optionally choose a different error queue. Perhaps on a remote machine
// https://docs.particular.net/nservicebus/recoverability/
endpointConfiguration.SendFailedMessagesTo("error");
//TODO: optionally choose a different audit queue. Perhaps on a remote machine
// https://docs.particular.net/nservicebus/operations/auditing
endpointConfiguration.AuditProcessedMessagesTo("audit");
endpointConfiguration.DefineCriticalErrorAction(OnCriticalError);
//TODO: For production use select a durable persistence.
// https://docs.particular.net/nservicebus/persistence/
endpointConfiguration.UsePersistence<LearningPersistence>();
//TODO: For production use script the installation.
endpointConfiguration.EnableInstallers();
endpointConfiguration.Conventions()
.DefiningCommandsAs(t => t.Namespace != null && t.Namespace.StartsWith("Messages") &&
t.Namespace.EndsWith("Commands"));
endpoint = await Endpoint.Start(endpointConfiguration)
.ConfigureAwait(false);
PerformStartupOperations();
**var testMessage = new TestMessage {Id = Guid.NewGuid()};
await endpoint.Send(testMessage).ConfigureAwait(false);**
}
catch (Exception exception)
{
logger.Fatal("Failed to start", exception);
Environment.FailFast("Failed to start", exception);
}
}
Ice Process Manager
private async Task AsyncOnStart()
{
try
{
var endpointConfiguration = new EndpointConfiguration("IceDataExtractor");
var transport = **endpointConfiguration.UseTransport<LearningTransport>();
transport.Routing().RouteToEndpoint(typeof(TestMessage), "IceProcessManager");**
endpointConfiguration.UseSerialization<JsonSerializer>();
//TODO: optionally choose a different error queue. Perhaps on a remote machine
// https://docs.particular.net/nservicebus/recoverability/
endpointConfiguration.SendFailedMessagesTo("error");
//TODO: optionally choose a different audit queue. Perhaps on a remote machine
// https://docs.particular.net/nservicebus/operations/auditing
endpointConfiguration.AuditProcessedMessagesTo("audit");
endpointConfiguration.DefineCriticalErrorAction(OnCriticalError);
//TODO: For production use select a durable persistence.
// https://docs.particular.net/nservicebus/persistence/
endpointConfiguration.UsePersistence<LearningPersistence>();
//TODO: For production use script the installation.
endpointConfiguration.EnableInstallers();
**endpointConfiguration.Conventions()
.DefiningCommandsAs(t => t.Namespace != null && t.Namespace.StartsWith("Messages") &&
t.Namespace.EndsWith("Commands"));**
endpoint = await Endpoint.Start(endpointConfiguration)
.ConfigureAwait(false);
PerformStartupOperations();
var testMessage = new TestMessage {Id = Guid.NewGuid()};
await endpoint.Send(testMessage).ConfigureAwait(false);
}
catch (Exception exception)
{
logger.Fatal("Failed to start", exception);
Environment.FailFast("Failed to start", exception);
}
}
TestMessage class
using System;
namespace Messages.Commands
{
public class TestMessage
{
public Guid Id { get; set; }
}
}
This all compiles and runs fine, other than performance warnings which I dont think matter
I have a message handler
TestMessageHandler
using System;
using System.Threading.Tasks;
using Messages.Commands;
using NServiceBus;
namespace IceProcessManager
{
public class TestMessageHandler : IHandleMessages<TestMessage>
{
public Task Handle(TestMessage message, IMessageHandlerContext context)
{
Console.WriteLine("Handled TEst MEssage ID:{0}", message.Id);
return Task.CompletedTask;
}
}
}
As you can see from the screenshot, no message is being received by the IceProcessManager. What am I doing wrong? I was thinking initially that I am sending the message too early, i.e. before the ProcessManager is up and running, but this not the problem because if I leave the ProcessManager running (i.e. run from explorer) then run the extractor, no message is receieved
Ideally I would like to have sent lots of messages to test this but I am not familiar with async stuff yet!
Can someone help please?
Paul
If I am not missing something you are using the same endpoint name for both instances?
var endpointConfiguration = new EndpointConfiguration("IceDataExtractor");
While you are routing the message to "IceDataManager" which doesn't exist.
I guess you might have pasted the wrong code?