Why am I getting this error from NLog during integration testing: LayoutRenderer cannot be found - asp.net-core

I added some aspnet layout renderers to my NLog configuration, and everything works fine for my website, but when I try to run my integration tests, they fail with this error message:
NLog.NLogConfigurationException : Error when setting property 'Layout' on NLog.Targets.DatabaseParameterInfo
----> System.ArgumentException : LayoutRenderer cannot be found: 'aspnet-request-cookie'. Is NLog.Web not included?
My initial instinct was to follow their suggestion and add the NLog.Web.AspNetCore package to my integration Test project. That did not do anything.
Here's what my integration test setup looks like:
[OneTimeSetUp]
public static void BaseOneTimeSetUp()
{
var environment = Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT") ?? "Development";
Assume.That(environment, Is.AnyOf("Development", "CI"), "Integration tests can only run in one of those environments");
var serviceCollection = new ServiceCollection();
serviceCollection.AddSingleton<IHostingEnvironment>(p =>
{
var mock = new Mock<IHostingEnvironment>();
mock.SetupGet(m => m.ContentRootPath).Returns(TestContext.CurrentContext.TestDirectory);
mock.SetupGet(m => m.EnvironmentName).Returns(environment);
return mock.Object;
});
Startup.AddConfiguration(serviceCollection); //throws NLogConfigurationException
}
And here's my implementation for the Startup AddConfiguration method:
public static void AddConfiguration(IServiceCollection services)
{
services.AddSingleton<IConfiguration>(s =>
{
var env = s.GetService<IHostingEnvironment>();
var builder = new ConfigurationBuilder()
.SetBasePath(env.ContentRootPath)
.AddJsonFile("appsettings.json", false, true)
.AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true)
.AddEnvironmentVariables();
var config = builder.Build();
// This next line throws the NLogConfigurationException
LogManager.Configuration = new NLogLoggingConfiguration(config.GetSection("NLog"));
return config;
});
}

The integration test is not calling the UseNLog method for the asp.net WebHost, as shown in their example code:
WebHost.CreateDefaultBuilder(args)
.UseStartup<Startup>()
.ConfigureLogging(logging =>
{
logging.ClearProviders();
logging.SetMinimumLevel(Microsoft.Extensions.Logging.LogLevel.Trace);
})
.UseNLog(); // NLog: setup NLog for Dependency injection
If you don't want to set up a WebHost for your tests, you can make do with a Mock<IWebHostBuilder>.
All you need to do is add this to your test setup before configuring NLog:
var mockHostBuilder = new Mock<IWebHostBuilder>();
mockHostBuilder.Object.UseNLog();

Related

I cannot Access the currently logged in user in my controllers actions (Blazor ASP.NET Core hosted .NET 6 and IdentityServer JWT)

I created a Hosted ASP.NET core Blazor WASM app using the Individual Authentication VS 2022 template which also includes Duende Identity Server configuration, apart from having to switch from Blazor UI to .cshtml for authentication views, it has done what is expected of it. A big problem I'm having is that I can't seem to access the currently logged in user from within my controllers neither using the User property from ControllerBase, nor via the IHttpContextAccessor, the Claims Principle instances all seem to be null when inspected via the debugging mode, meanwhile on the client-side WASM I can access my subject Id, email or whatever I specify in the ProfileService just fine (Which is not really useful unless I would want to fetch the user from the server via the subject ID sent from the client using a parameter or something... which would be disastrous I know..)
Here's my Client/Program.cs:
using System.Globalization;
using CurrieTechnologies.Razor.SweetAlert2;
using Microsoft.AspNetCore.Components.Web;
using Microsoft.AspNetCore.Components.WebAssembly.Authentication;
using Microsoft.AspNetCore.Components.WebAssembly.Hosting;
using Microsoft.JSInterop;
using Proj.Client;
using Proj.Client.Auth;
using Proj.Client.Helpers;
using Proj.Client.Repository;
var builder = WebAssemblyHostBuilder.CreateDefault(args);
var services = builder.Services;
builder.RootComponents.Add<App>("#app");
builder.RootComponents.Add<HeadOutlet>("head::after ");
services.AddHttpClient<IHttpService>("Proj.ServerAPI",
client => client.BaseAddress = new Uri(builder.HostEnvironment.BaseAddress))
.AddHttpMessageHandler<BaseAddressAuthorizationMessageHandler>();
// Supply HttpClient instances that include access tokens when making requests to the server project
services.AddScoped(sp => sp.GetRequiredService<IHttpClientFactory>().CreateClient("Proj.ServerAPI"));
// SERVICES
services.AddOptions();
services.AddLocalization();
// builder.Services.AddScoped<DialogService>();
// builder.Services.AddScoped<NotificationService>();
// builder.Services.AddScoped<TooltipService>();
// builder.Services.AddScoped<ContextMenuService>();
services.AddApiAuthorization()
.AddAccountClaimsPrincipalFactory<CustomUserFactory>();
var host = builder.Build();
var js = host.Services.GetRequiredService<IJSRuntime>();
var culture = await js.InvokeAsync<string>("getFromLocalStorage", "culture");
CultureInfo selectedCulture;
selectedCulture = culture == null ? new CultureInfo("en") : new CultureInfo(culture);
CultureInfo.DefaultThreadCurrentCulture = selectedCulture;
CultureInfo.DefaultThreadCurrentUICulture = selectedCulture;
await host.RunAsync();
And the Server/Program.cs:
using System.IdentityModel.Tokens.Jwt;
using Duende.IdentityServer.Services;
using Microsoft.AspNetCore.ApiAuthorization.IdentityServer;
using Microsoft.AspNetCore.Authentication;
using Microsoft.AspNetCore.Authentication.JwtBearer;
using Microsoft.AspNetCore.Identity;
using Microsoft.AspNetCore.Identity.UI.Services;
using Microsoft.EntityFrameworkCore;
using Proj.Server.Data;
using Proj.Server.DbInitializer;
using Proj.Server.Models;
using Proj.Server.Services;
using Proj.Server.Utils;
var builder = WebApplication.CreateBuilder(args);
var services = builder.Services;
// Add services to the container.
var connectionString = builder.Configuration.GetConnectionString("MySQLConnectionLocal");
var serverVersion = new MySqlServerVersion(ServerVersion.AutoDetect(connectionString));
services.AddDbContext<ApplicationDbContext>(
dbContextOptions => dbContextOptions
.UseMySql(connectionString, serverVersion)
// The following three options help with debugging, but should
// be changed or removed for production.
.LogTo(Console.WriteLine, LogLevel.Information)
.EnableSensitiveDataLogging()
.EnableDetailedErrors()
);
services.AddAutoMapper(typeof(MappingConfig));
services.AddSingleton<IHttpContextAccessor, HttpContextAccessor>();
services.AddDatabaseDeveloperPageExceptionFilter();
services.AddDefaultIdentity<ApplicationUser>(options =>
{
options.SignIn.RequireConfirmedAccount = true;
options.Stores.MaxLengthForKeys = 80;
options.User.RequireUniqueEmail = true;
options.Tokens.ProviderMap.Add("CustomEmailConfirmation",
new TokenProviderDescriptor(
typeof(CustomEmailConfirmationTokenProvider<ApplicationUser>)));
options.Tokens.EmailConfirmationTokenProvider = "CustomEmailConfirmation";
})
.AddRoles<IdentityRole>()
.AddEntityFrameworkStores<ApplicationDbContext>();
services.AddTransient<CustomEmailConfirmationTokenProvider<ApplicationUser>>();
services.AddTransient<IEmailSender, EmailSender>();
services.Configure<AuthMessageSenderOptions>(builder.Configuration);
services.AddHttpContextAccessor();
services.AddIdentityServer()
.AddApiAuthorization<ApplicationUser, ApplicationDbContext>()
.AddDeveloperSigningCredential();
builder.Services.AddTransient<IProfileService, ProfileService>();
builder.Services.AddTransient<IFileStorageService, InAppStorageService>();
builder.Services.AddDataProtection();
JwtSecurityTokenHandler.DefaultInboundClaimTypeMap.Remove("role");
services.AddAuthentication()
.AddIdentityServerJwt()
.AddGoogle(googleOptions =>
{
googleOptions.ClientId = builder.Configuration["Authentication:Google:ClientId"];
googleOptions.ClientSecret = builder.Configuration["Authentication:Google:ClientSecret"];
}).AddFacebook(facebookOptions =>
{
facebookOptions.AppId = builder.Configuration["Authentication:Facebook:AppId"];
facebookOptions.AppSecret = builder.Configuration["Authentication:Facebook:AppSecret"];
});
services.Configure<DataProtectionTokenProviderOptions>(o =>
o.TokenLifespan = TimeSpan.FromHours(3));
services.ConfigureApplicationCookie(o =>
{
o.ExpireTimeSpan = TimeSpan.FromDays(5);
o.SlidingExpiration = true;
});
services.AddControllersWithViews();
services.AddRazorPages();
var app = builder.Build();
// Configure the HTTP request pipeline.
if (app.Environment.IsDevelopment())
{
app.UseMigrationsEndPoint();
app.UseWebAssemblyDebugging();
}
else
{
app.UseExceptionHandler("/Error");
// The default HSTS value is 30 days. You may want to change this for production scenarios, see https://aka.ms/aspnetcore-hsts.
app.UseHsts();
}
app.UseHttpsRedirection();
app.UseBlazorFrameworkFiles();
app.UseStaticFiles();
app.UseRouting();
app.UseIdentityServer();
app.UseAuthentication();
app.UseAuthorization();
app.MapRazorPages();
app.MapControllers();
app.MapFallbackToFile("index.html");
app.Run();
And I think that the ProfileService Impelementation would also be Useful:
using IdentityModel;
using Duende.IdentityServer.Models;
using Duende.IdentityServer.Services;
namespace Proj.Server.Services;
public class ProfileService : IProfileService
{
public ProfileService()
{
}
public async Task GetProfileDataAsync(ProfileDataRequestContext context)
{
var nameClaim = context.Subject.FindAll(JwtClaimTypes.Name);
context.IssuedClaims.AddRange(nameClaim);
var subClaim = context.Subject.FindAll(JwtClaimTypes.Subject);
context.IssuedClaims.AddRange(subClaim);
var roleClaims = context.Subject.FindAll(JwtClaimTypes.Role);
context.IssuedClaims.AddRange(roleClaims);
await Task.CompletedTask;
}
public async Task IsActiveAsync(IsActiveContext context)
{
await Task.CompletedTask;
}
}
I'd be grateful if anyone could point me in the right direction,
Thanks :)
Set the options in AddApiAuthorization :
builder.Services.AddIdentityServer()
.AddApiAuthorization<ApplicationUser, ApplicationDbContext>(options =>
{
const string OpenId = "openid";
options.IdentityResources[OpenId].UserClaims.Add(JwtClaimTypes.Email);
options.ApiResources.Single().UserClaims.Add(JwtClaimTypes.Email);
options.IdentityResources[OpenId].UserClaims.Add(JwtClaimTypes.Id);
options.ApiResources.Single().UserClaims.Add(JwtClaimTypes.Id);
options.IdentityResources[OpenId].UserClaims.Add(JwtClaimTypes.Name);
options.ApiResources.Single().UserClaims.Add(JwtClaimTypes.Name);
options.IdentityResources[OpenId].UserClaims.Add(JwtClaimTypes.Role);
options.ApiResources.Single().UserClaims.Add(JwtClaimTypes.Role);
});
JwtSecurityTokenHandler.DefaultInboundClaimTypeMap.Remove("role");
In your controller:
var name = User.FindFirst(ClaimTypes.Name)?.Value;
Turns out this question also describes the same problem I have, By removing
.AddIdentityServerJwt()
From the line
services.AddAuthentication()
.AddIdentityServerJwt()
Everything started working as expected...
Now being the beginner I am, I'm not certain whether or not I should've commented out that line, it may have been important, so if anyone knows a better solution by all means share...
Thanks.

Programmatic configuration of NLog in ASP.NET Core application to filter unwanted chatter?

I'm attempting to filter out all the unnecessary chatter from the Microsoft hosting assemblies, while still allowing Debug-level messages from our own code to pass through to our logging targets.
Here's code that configures NLog and initializes a web app. Unfortunately, the Microsoft.* and System.* namespace filtering does not functional at all. The result is that all messages Debug and higher are logged.
I do not want to use the old nlog.config process. I want to be able to build the LoggingConfiguration in code.
I really must be missing something easy here, but I don't see it!
public static void Main(string[] args)
{
// Read configuration from a variety of sources to compose our final IConfiguration
var config = ReadOurConfigFromLotsOfPlaces();
var nLogConfig = new LoggingConfiguration();
var consoleTarget = new ColoredConsoleTarget("console")
{
AutoFlush = true,
ErrorStream = true,
Layout = #"${date:format=yyyy-MM-dd HH\:mm\:ss.fff} ${level} [${logger}] - ${message}${onexception:${newline}}${exception:format=shortType,message,stackTrace:maxInnerExceptionLevel=5}"
};
nLogConfig.AddTarget(consoleTarget);
nLogConfig.LoggingRules.Add(new LoggingRule("Microsoft.*", LogLevel.Warn, consoleTarget) {Final = true});
nLogConfig.LoggingRules.Add(new LoggingRule("System.*", LogLevel.Warn, consoleTarget) {Final = true});
nLogConfig.LoggingRules.Add(new LoggingRule("*", LogLevel.Debug, consoleTarget));
var logger = NLogBuilder.ConfigureNLog(nLogConfig).GetCurrentClassLogger();
logger.Debug("NLog initialization complete");
try
{
logger.Debug("init main");
CreateHostBuilder(config).Build().Run();
}
catch (Exception exception)
{
logger.Error(exception, "Stopped program because of exception");
throw;
}
finally
{
// Ensure to flush and stop internal timers/threads before application-exit (Avoid segmentation fault on Linux)
LogManager.Shutdown();
}
}
public static IWebHostBuilder CreateHostBuilder(IConfiguration config)
{
// we don't call CreateDefaultBuilder() because we already have assembled the configuration we want to use
return new WebHostBuilder()
.UseConfiguration(config)
.UseKestrel(options => options.UseSystemd())
.UseStartup<Startup>()
.ConfigureLogging(logging =>
{
logging.ClearProviders();
logging.SetMinimumLevel(Microsoft.Extensions.Logging.LogLevel.Trace);
})
.UseNLog();
}
Any tips would be greatly appreciated.
Ugh. I understand my problem now: The 'LoggingRules' is looking for a positive match between logger name and the min/max level values, so my first two LoggingRule objects did not match messages coming from the Microsoft.* and System.* namespaces, so the rules did nothing with those messages.
In order to accomplish the filtering I want, this is the solution:
var nLogConfig = new LoggingConfiguration();
var consoleTarget = new ColoredConsoleTarget("console")
{
AutoFlush = true,
ErrorStream = true,
Layout = #"${date:format=yyyy-MM-dd HH\:mm\:ss.fff} ${level} [${logger}] - ${message}${onexception:${newline}}${exception:format=shortType,message,stackTrace:maxInnerExceptionLevel=5}"
};
nLogConfig.AddTarget(consoleTarget);
var nullTarget = new NullTarget("null");
nLogConfig.AddTarget(nullTarget);
// matches every Microsoft.* logger with a LogLevel less than LogLevel.Warn
nLogConfig.LoggingRules.Add(new LoggingRule("Microsoft.*", LogLevel.Trace, LogLevel.Info, nullTarget) {Final = true});
// matches every System.* logger with a LogLevel less than LogLevel.Warn
nLogConfig.LoggingRules.Add(new LoggingRule("System.*", LogLevel.Trace, LogLevel.Info, nullTarget) {Final = true});
// and everything else, LogLevel.Debug and higher
nLogConfig.LoggingRules.Add(new LoggingRule("*", LogLevel.Debug, consoleTarget));

Serilog dotnet core function app and sql sink

I need to use dotnet5 with Azure Functions so followed the guidance to create a new solution: https://learn.microsoft.com/en-us/azure/azure-functions/dotnet-isolated-process-guide.
This worked great so next job was to add in serilog with sinks for console and sql server.
I have added nuget packages:
Serilog.AspNetCore v4.1.0
Serilog.Sinks.MSSqlServer v5.6.0
Here is the Program.Main:
static void Main(string[] args)
{
string EventName = "Main";
var columnOptions = new ColumnOptions
{
AdditionalColumns = new Collection<SqlColumn>
{
new SqlColumn
{ColumnName = "EventName", DataType = SqlDbType.NVarChar, DataLength = 32, NonClusteredIndex = true}
}
};
Log.Logger = new LoggerConfiguration()
.MinimumLevel.Override("Microsoft.Azure", LogEventLevel.Warning)
.Enrich.FromLogContext()
.WriteTo.Console()
.WriteTo.MSSqlServer(
logEventFormatter: new RenderedCompactJsonFormatter(),
restrictedToMinimumLevel: LogEventLevel.Debug,
connectionString: "Data Source=(localdb)\\MSSQLLocalDB;Initial Catalog=SmsRouter",
sinkOptions: new MSSqlServerSinkOptions
{
TableName = "LogEvents",
AutoCreateSqlTable = true,
},
columnOptions: columnOptions)
.CreateLogger();
try
{
Log.Information("Starting up {EventName}", EventName);
var host = new HostBuilder()
.UseSerilog()
.ConfigureFunctionsWorkerDefaults()
.ConfigureServices(s =>
{
//services configured here
})
.Build();
host.Run();
}
catch (Exception ex)
{
Log.Fatal(ex, "Application start-up failed");
}
finally
{
Log.CloseAndFlush();
}
}
You can see the line Log.Information("Starting up {EventName}", EventName); This works and is logged to both console and Sql Server :)
After the App is started it will sit and wait for a Http request - as shown below:
[Function("SendSimpleSms")]
public async Task<QueueAndHttpOutputType> RunSimple([HttpTrigger(AuthorizationLevel.Anonymous, "get", "post")] HttpRequestData req,
FunctionContext executionContext)
{
string EventName = "SendSimpleSms";
try
{
Log.Information("Starting: {EventName}", EventName);
My problem is, this log request "Starting: SendSimpleSms" is logged to the console window but not to Sql Server.
Anyone see what I have wrong please?
Thanks to Panagiotis Kanavos for making me aware of the Serilog self-logging.
I added the following into program.main, after the LoggerConfiguration:
Serilog.Debugging.SelfLog.Enable(Console.Error);
This then made me aware that the sql sink was unable to log because the length of a custom property exceeded that of its column

Serilog Elasticsearch

i have a microservice api where I try to log all the request which come in...so the elasticsaerch service and kibana are on a different server. I'm using the serilog.sinks.elasticsearch package to send data to the elasticsearch.
Both servers are not running with docker, they are just normal windows server.
My code looks like this to setup the logging...
public static Logger Create(IConfiguration configuration)
{
var elasticsearchSection = configuration.GetSection("Elasticsearch");
if (elasticsearchSection != null)
{
return CreateLoggerConfiguration(elasticsearchSection).CreateLogger();
}
return null;
}
private static LoggerConfiguration CreateLoggerConfiguration(IConfigurationSection section)
{
var loggerConfiguration = new LoggerConfiguration();
var url = section.GetValue<string>("Url");
var minimumLogLevel = section.GetValue<string>("MinimumLogLevel");
var minimumLogEventLevel = section.GetValue<string>("MinimumLogEventLevel");
SetLoggerConfigurationMinimumLogLevel(minimumLogLevel, loggerConfiguration);
loggerConfiguration.WriteTo.Elasticsearch(new ElasticsearchSinkOptions(GetLoggingUri(url))
{
MinimumLogEventLevel = ReturnLogEventLevel(minimumLogEventLevel),
AutoRegisterTemplate = true
});
loggerConfiguration.Enrich.FromLogContext();
return loggerConfiguration;
}
And in my startup,cs I'm using
services.AddLogging(loggingBuilder => loggingBuilder.AddSerilog(dispose: true));
in the ConfigureServices Method...
But apparently I cant create an Index inside my Kibana.
Any ideas why this isnt working?

How to control log level in ASP.Net core Integration tests

I am trying to limit the amount the log information being printed when running integration tests for asp.net core. Currently, everything down to debug level is being printed and it obscures useful information. I would really like to restrict it to warning and above.
I am running integration tests using the example at https://learn.microsoft.com/en-us/aspnet/core/test/integration-tests?view=aspnetcore-2.2#customize-webapplicationfactory
public class CustomWebApplicationFactory<TStartup>
: WebApplicationFactory<TStartup> where TStartup: class
{
protected override void ConfigureWebHost(IWebHostBuilder builder)
{
builder.ConfigureServices(services =>
{
// Create a new service provider.
var serviceProvider = new ServiceCollection()
.AddEntityFrameworkInMemoryDatabase()
.BuildServiceProvider();
// Add a database context (ApplicationDbContext) using an in-memory
// database for testing.
services.AddDbContext<ApplicationDbContext>(options =>
{
options.UseInMemoryDatabase("InMemoryDbForTesting");
options.UseInternalServiceProvider(serviceProvider);
});
// Build the service provider.
var sp = services.BuildServiceProvider();
// Create a scope to obtain a reference to the database
// context (ApplicationDbContext).
using (var scope = sp.CreateScope())
{
var scopedServices = scope.ServiceProvider;
var db = scopedServices.GetRequiredService<ApplicationDbContext>();
var logger = scopedServices
.GetRequiredService<ILogger<CustomWebApplicationFactory<TStartup>>>();
// Ensure the database is created.
db.Database.EnsureCreated();
try
{
// Seed the database with test data.
Utilities.InitializeDbForTests(db);
}
catch (Exception ex)
{
logger.LogError(ex, "An error occurred seeding the database. Error: {Message}", ex.Message);
}
}
});
}
}
Logs:
dbug: Microsoft.AspNetCore.Hosting.Internal.WebHost[4]
Hosting started
dbug: Microsoft.AspNetCore.Hosting.Internal.WebHost[0]
Loaded hosting startup assembly BackEnd.Api
info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
Request starting HTTP/2.0 GET http://localhost/test
dbug: Microsoft.AspNetCore.HostFiltering.HostFilteringMiddleware[0]
Wildcard detected, all requests with hosts will be allowed.
...lots more dbug logs...
Things I have tried in the ConfigureWebHost:
builder.ConfigureLogging(o =>
{
o.SetMinimumLevel(LogLevel.Warning);
});
No effect. I could not come up with any other combination of levels and filters that had any effect.
builder.ConfigureLogging(o =>
{
o.ClearProviders();
});
Stops all logging, but that's not really what I want.
Try to use AddFilter like
builder.ConfigureLogging(o=> {
//o.SetMinimumLevel(LogLevel.Warning);
o.AddFilter(logLevel => logLevel >= LogLevel.Warning);
});
I resolved this for my case. In the appsettings.json for the asp.net core app I was testing, was the following config:
"Console": {
"LogLevel": {
"Default": "Debug"
}
}
It seems that this would override any of the filtering or level changes I made in the code for the integration tests. Changing this level to Warning resolved my issue.