Only write logs for custom logging - asp.net-core

I am following this article:
https://learn.microsoft.com/en-us/aspnet/core/fundamentals/logging/?view=aspnetcore-3.1
I am trying to only log my own custom logging in a asp.net core 3.1 API. And not all logs generated from asp.net core. I have created a blank weather forecast service:
[HttpGet]
public IEnumerable<WeatherForecast> Get()
{
_logger.LogTrace("LogTrace");
_logger.LogDebug("LogDebug");
_logger.LogInformation("LogInformation");
_logger.LogError("LogError");
var rng = new Random();
return Enumerable.Range(1, 5).Select(index => new WeatherForecast
{
Date = DateTime.Now.AddDays(index),
TemperatureC = rng.Next(-20, 55),
Summary = Summaries[rng.Next(Summaries.Length)]
})
.ToArray();
}
appsettings.json:
{
"Logging": {
"LogLevel": {
"Default": "Trace",
"Microsoft": "Trace",
"TestAPI": "Trace"
},
"ApplicationInsights": {
"InstrumentationKey": "xx-xx-x-x-xx",
"LogLevel": {
"Default": "Trace",
"Microsoft": "Trace",
"TestAPI": "Trace"
}
}
},
"AllowedHosts": "*"
}
I have following nuget installed both version 2.14.0:
Microsoft.Extensions.Logging.ApplicationInsights
Microsoft.ApplicationInsights.AspNetCore
Now I try to run the app, but gets no logs.
I try adding services.AddApplicationInsightsTelemetry(); to startup:
public void ConfigureServices(IServiceCollection services)
{
services.AddApplicationInsightsTelemetry();
services.AddControllers();
}
Same no logs.

First, please note that the ApplicationInsights key means two very different things depending on where it's located in the JSON file.
If the key is on the JSON root level (ie. what you call "outside"), it's used to configure Application Insights, and it's where you specify your instrumentation key. It looks like this:
{
"ApplicationInsights": {
"Instrumentationkey":"xxx-36a5-4687-b1fc-xxxxxx"
}
}
Second, if it's located inside the Logging section, it's used to configure the ApplicationInsightsLoggerProvider, which determines which log level is sent to Application Insights. That's the ILogger log filtering mechanism.
By default, only log levels warning or higher are sent to app insights. If you only want to send all your logs to application insights, you can either configure it for your namespaces, or ignore the messages coming from the System and Microsoft namespaces:
{
"Logging": {
"ApplicationInsights": {
"LogLevel": {
"Default": "Trace"
"System": "None",
"Microsoft": "None"
}
}
}

First of all, there is no need to put those provider settings outside "Logging" (you shouldn't).
Every logging setting you want should be put inside only ( unless there is a provider configured to specifically read that)
Now to answer your question, let's say that your application's root namespace is MyNetCore. ( it would be similar to the name of the project).
If you are using Visual Studio, You can view your Project's root namespace from Project Properties -> Application -> Default Namespace
To view logs from your application only, you have to set the default logging level to None and logging level of your project MyNetCore to Trace
[Edit: You have set the logging levels for ApplicationInsights (or any other provider) separately. The default one is for kestrel.]
"Logging": {
"LogLevel": {
"Default": "None",
"MyNetCore": "Trace"
},
"ApplicationInsights": {
"InstrumentationKey": "xxx-36a5-4687-b1fc-xxxxxx",
"LogLevel": {
"Default": "None",
"MyNetCore": "Trace"
}
}
}
The Logging level set here is the minimum level. If you set it to Trace (0), all the log levels (greater than 0) will be shown. i.e. From Information to Critical
If you set it to None (6), no logs will be shown
Reference to different Log Levels : https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.logging.loglevel
If you want to view only errors from asp.net core, and every log level from your application, then you can do this.
"Logging": {
"LogLevel": {
"Default": "None",
"Microsoft": "Error",
"MyNetCore": "Trace"
},
"ApplicationInsights": {
"InstrumentationKey": "xxx-36a5-4687-b1fc-xxxxxx",
"LogLevel": {
"Default": "None",
"Microsoft": "Error",
"MyNetCore": "Trace"
}
}
}
Edit: To read the above ApplicationInsights configuration, you need to have Microsoft.Extensions.Logging.ApplicationInsights nuget package installed. Otherwise the config will be totally ignored.

Related

Logging based on environment in .NetCore 2.2 using ILoggerProvider

I am working on .Net Core 2.2 logging mechanism.
While I was experimenting the logging based on environment.
So, I created two new appSettings.json files with respective environments along with appSettings.json present in the solution. One for Development, other for Production environment.
appsettings.json
{
"Logging": {
"LogLevel": {
//"Default": "Debug",
//"System": "Information",
//"Microsoft": "Error"
}
}
}
appsettings.Development.json
{
"Logging": {
"LogLevel": {
"Microsoft": "Information"
}
}
}
appSettings.Production.json
{
"Logging": {
"Console": {
"LogLevel": {
"Microsoft": "Critical"
}
}
}
}
Changed the StartUp.cs file
public Startup(IConfiguration configuration, IHostingEnvironment env)
{
Configuration = configuration;
var currentEnvironment = env.EnvironmentName;
var builder = new ConfigurationBuilder()
.SetBasePath(env.ContentRootPath)
.AddJsonFile("appsettings.json", optional: true, reloadOnChange: true)
.AddJsonFile($"appsettings.{currentEnvironment}.json", optional: true);
}
Logging Method
public void LogExceptionToConsole()
{
_logger.LogError("This is raised by error");
_logger.LogCritical("This is raised by critical ");
}
And launchSettings.json
{
"iisSettings": {
"windowsAuthentication": false,
"anonymousAuthentication": true,
"iisExpress": {
"applicationUrl": "http://localhost:2131",
"sslPort": 44388
}
},
"profiles": {
"IIS Express": {
"commandName": "IISExpress",
"launchBrowser": true,
"environmentVariables": {
"ASPNETCORE_ENVIRONMENT": "Production"
}
},
"DemoLoggingApplication": {
"commandName": "Project",
"launchBrowser": true,
"applicationUrl": "https://localhost:5001;http://localhost:5000",
"environmentVariables": {
"ASPNETCORE_ENVIRONMENT": "Development"
}
}
}
}
I understand that environment specific appSettings would be taken the highest precedence.
And I have two questions
1) Does environment specific logging happens? If yes, then what could be the changes in the above logic.
2) When I ran the application in Development mode on DemoLoggingApplication profile. I can see all the Information,Error & Critical logs.
But when I changed the ASPNETCORE_ENVIRONMENT value from Development to Production for DemoLoggingApplication profile and ran the application I could again see both the logs of type Error & Critical.
As, I've set that Console provider should only display Critical logs of type Microsoft category .I've been displayed Errors logs also.
Though I've read the Microsoft documents, I couldn't understand the prioritization. Could anyone explain me detail why I'm seeing both the logs.
Am I missing any understanding. Please help me.
Thanks in advance
Updated the question after Bob's answer
It worked after changing the appsettings.json, appsettings.Development.json & appSettings.Production.json
appSettings.json
{
"Logging": {
"LogLevel": {
"Default": "Information"
}
}
}
appsettings.Development.json
{
"Logging": {
"Console": {
"LogLevel": {
"Microsoft": "Information",
"Default": "Information" // newly
}
}
}
}
appSettings.Production.json
{
"Logging": {
"Console": {
"LogLevel": {
"Microsoft": "Critical",
"Default" : "Critical" // newly added line
}
}
}
}
Now when I changed the environment to Development, I could log from Information but only after adding Default category to both Development and Production.
I just wanted to know why is this behavior?
And what's the impact of having maintaining appsettings.json -> Logging when we are having Development & Production settings.
Thanks
Environment specific logging happens.
In your example, both appsettings.Development.json and appsettings.Production.json defines LogLevel only for "Microsoft" category. But, the logging done from your code falls in a different Log Category whose LogLevel is not defined in code/config files. Hence, it takes default minimum log level as "Information" in both environments.
To see a difference, add "Default" key with different LogLevel settings in different environments as below:
appsettings.Development.json
{
"Logging": {
"LogLevel": {
"Microsoft": "Information",
"Default": "Error"
}
}
}
appSettings.Production.json
{
"Logging": {
"Console": {
"LogLevel": {
"Microsoft": "Critical",
"Default": "Critical"
}
}
}
}

Log information is displayed though "LogLevel" Property is commented

I am pretty new to .Net Core. And I'm working on basic logging of information by the default ILogger provider.
Initially my AppSetting.json was uncommented and able to see the logs that I've written.
{
"Logging": {
"LogLevel": {
"Default": "Debug",
"System": "Information",
"Microsoft": "Information"
}
},
"AllowedHosts": "*"
}
Later, I commented theLogLevel properties and ran the application on Krestel server. Then I can still see the logged information in the console.
AppSettings.json
{
"Logging": {
//"LogLevel": {
// "Default": "Debug",
// "System": "Information",
// "Microsoft": "Information"
//}
},
"AllowedHosts": "*"
}
Program.cs
public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
WebHost.CreateDefaultBuilder(args)
.ConfigureLogging((context, logging)
=>
{
logging.ClearProviders();
logging.AddConfiguration(context.Configuration.GetSection("Logging"));
logging.AddConsole();
})
.UseStartup<Startup>();
Sample logging method in HomeController.cs
public void LogExceptionToConsole()
{
_logger.LogError("This is raised by logger information");
}
Console
Why is this behavior? If this the behavior why it has been designed like this?
Could anyone explain me.
Thanks in advance
ASP.NET Core defines the following log levels, ordered here from lowest to highest severity:
Trace = 0 (Disabled by default)
Debug = 1
Information = 2
Warning = 3
Error = 4
Critical = 5
When no LogLevel is set in the application (either in startup.cs or program.cs or appsettings.json or appsettings.development.json) for a LogCategory, then Minimum LogLevel is set as Information by default.
Hence, in your example, the Error is logged since it is higher than the default Minimum LogLevel.
You can refer to this link for more details

Date and time in console log

Is their a way to mention the date and time in the console log of an asp.net core 2.0 project for production and development environment?
I have following in my startup:
services.AddLogging(builder =>
{
builder.AddConfiguration(Configuration.GetSection("Logging"+Environment.EnvironmentName))
.AddConsole()
.AddDebug();
});
Appsettings.json:
"LoggingDevelopment": {
"IncludeScopes": false,
"Console": {
"LogLevel": {
"Default": "Debug",
"System": "Information",
"Microsoft": "Information"
}
}
},
"LoggingProduction": {
"IncludeScopes": false,
"Console": {
"LogLevel": {
"Default": "Error",
"System": "Error",
"Microsoft": "Error"
}
}
},
Current [development] log layout (without a date or time of the log line):
info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[2]
Executed action a.Controller (project) in 531.2457ms
info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2]
Request finished in 532.5812ms 200 text/html; charset=utf-8
A date and time would be extremly handy in production mode for error lines.
Default Console logger is pretty limited. Of course there is always the possibility to use lambda formatter like said in Github issue provided by serpent5. But it's accessible only if using raw ILogger.Log() method.
It's not available for ILogger extensions like LogInformation or LogError. This option is also not available if you don't control logging call. It's actually your case when logging is done by ASP.NET Core internal classes.
So you need some more flexible implementation of logger for .Net Core. I suggest using of Serilog. It's pretty simple but very flexible and powerful at the same time.
To integrate Serilog into ASP.Net Core application do the following:
Install following NuGet packages:
Serilog.AspNetCore
Serilog.Sinks.Console
Besides Console, Serilog supports many other targets like Rolling File, E-mail, SQL Server, etc. See this list of other Serilog targets.
Configure Serilog on application startup:
Here is a sample:
public class Program
{
public static void Main(string[] args)
{
Log.Logger = new LoggerConfiguration()
.MinimumLevel.Debug()
.MinimumLevel.Override("Microsoft", LogEventLevel.Information)
.Enrich.FromLogContext()
.WriteTo.Console(outputTemplate: "{Timestamp:yyyy-MM-dd HH:mm:ss} [{Level}] {Message}{NewLine}{Exception}")
.CreateLogger();
BuildWebHost(args).Run();
}
public static IWebHost BuildWebHost(string[] args) =>
WebHost.CreateDefaultBuilder(args)
.UseStartup<Startup>()
.UseSerilog()
.Build();
}
Provided output template adds date and time to logged message:
To expand on CodeFuller's response, here's how you can modify the Serilog Console output formatter purely from configuration:
{
"Serilog": {
"MinimumLevel": {
"Default": "Debug",
"Override": {
"System": "Information",
"Microsoft": "Information"
}
},
"WriteTo:Sublogger": {
"Name": "Logger",
"Args": {
"configureLogger": {
"WriteTo": [
{
"Name": "Console",
"Args": {
"outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss.fff zzz} [{Level:u3}] [{TraceId}] {Message:lj}{NewLine}{Exception}"
}
}
]
}
}
}
}
}
If you take this approach, remove the WriteTo.Console logger method from your startup configuration. This line:
.WriteTo.Console(outputTemplate: "{Timestamp:yyyy-MM-dd HH:mm:ss} [{Level}] {Message}{NewLine}{Exception}")
I've also modified the output format template slightly. Either template will work fine.

Filtering logs in Azure Web App Diagnostics Logs

Is it possible to filter logs using Azure's Web App Diagnostic logs to capture application logs?
I want to capture information-level logs reported from our assemblies, but only warnings for MS/System libraries.
Startup.cs looks as follows:
loggerFactory
.WithFilter(new FilterLoggerSettings
{
{ "Microsoft", LogLevel.Warning },
{ "System", LogLevel.Warning },
{ "MyAssembly", LogLevel.Information }
})
.AddAzureWebAppDiagnostics();
But in the Azure Portal there is only an option to set the level:
You can also apply filters in your appsettings.json file like this
"Logging": {
"IncludeScopes": false,
"AzureAppServicesBlob": {
"LogLevel": {
"Default": "Warning",
"Microsoft": "Warning",
"System": "Warning",
"{custom-category}": "Information"
}
},
"LogLevel": {
"Default": "Warning"
}
}
There's also a provider aliases for AzureAppServicesFile.
According to your scenario, I have tested this issue and you could just configure your logger in Configure method of Startup.cs as follows:
loggerFactory
.WithFilter(new FilterLoggerSettings
{
{ "Microsoft", LogLevel.Warning },
{ "System", LogLevel.Warning },
{ "{custom-category}", LogLevel.Information}
})
.AddAzureWebAppDiagnostics();
Note: The category would be the fully qualified name of the class from which the logs are written. If the logger is under TodoApi.Controllers.TodoController, you could configure {custom-category} as TodoApi to limit framework logs level to informations for the logs from your assemblies.
The Azure App Service provider
This provider is available only for apps that target ASP.NET Core 1.1.0 or higher. The provider only works when your project runs in the Azure environment. It has no effect when you run locally -- it does not write to local files or local development storage for blobs.
When using Azure App Service Logging, the available log level would be the larger one between the level you set in your filtering rules and the application level your configured on Azure Portal. In order to capture information-level logs reported from your assemblies, the Application Level you configured on Azure Portal need to less than or equal information-level, you could configure it to verbose or information. For more details, you could refer to this official tutorial.
UPDATE:
Here are the details about my test, you could refer to them:
Log filtering
loggerFactory
.WithFilter(new FilterLoggerSettings
{
{ "Microsoft", LogLevel.Warning },
{ "System", LogLevel.Warning },
{ "WebApplication_FilterLogging", LogLevel.Information }
})
.AddConsole()
.AddDebug()
.AddAzureWebAppDiagnostics();
HomeController.cs
public class HomeController : Controller
{
private readonly ILogger _logger;
public HomeController(ILogger<HomeController> logger)
{
_logger = logger;
}
public IActionResult Index()
{
_logger.LogWarning($"Index {DateTime.UtcNow}");
return View();
}
public IActionResult About()
{
_logger.LogInformation($"About {DateTime.UtcNow}");
return View();
}
public IActionResult Contact()
{
_logger.LogError($"Contact {DateTime.UtcNow}");
return View();
}
}
Result
1) Logs from my output window:
2) Logs from my application log stored in Blob Storage:
When I set information-level logs for Microsoft/System libraries, then I could retrieve the following logs:
I faced the same issue recently by using below log filters in appsettings.json.
"Logging": {
"LogLevel": {
"Microsoft": "Warning",
"System": "Warning",
"Default": "Information"
}
}
I had to specify the log levels for each target as given below.
"Logging": {
"AzureAppServicesBlob": {
"IncludeScopes": false,
"LogLevel": {
"Microsoft": "Warning",
"System": "Warning",
"Default": "Information"
}
},
"AzureAppServicesFile": {
"IncludeScopes": false,
"LogLevel": {
"Microsoft": "Warning",
"System": "Warning",
"Default": "Information"
}
},
"LogLevel": {
"Microsoft": "Warning",
"System": "Warning",
"Default": "Information"
}
}
Log level in Azure portal was set to Information.

EF-core with PostgreSQL migration error

In my appsettings.json:
{
"Data": {
"DefaultConnection": {
"ConnectionString": "Host=localhost;Username={{postgres}};Password={{postgres}};Database={{asp_trial_api}};Port=5432;Pooling=true;"
}
},
"Logging": {
"IncludeScopes": false,
"LogLevel": {
"Default": "Debug",
"System": "Information",
"Microsoft": "Information"
}
}
}
And in my startup.cs:
public void ConfigureServices(IServiceCollection services)
{
// Add framework services.
services.AddApplicationInsightsTelemetry(Configuration);
services.AddDbContext<WebAPIDataContext>(options => {
options.UseNpgsql(Configuration["Data:DefaultConnection:ConnectionString"]);
});
services.AddMvc();
services.AddSwaggerGen();
}
My dotnet ef migrations add initialwas successful however dotnet ef database updategives me following error:
28P01: password authentication failed for user "{{postgres}}"
The username and password I provided in my connection string is however correct. I have already created a blank database with the name asp_trial_api in my PostgresSQL.
What is going wrong here?
The problem seems like have some missing attributes in the connection string such as User ID and Server. Try the following CS
"ConnectionStrings": {
"DefaultConnection": "User ID=postgres;Password={{pass}};Server=localhost;Port=1234;Database={{database}};Integrated Security=true;Pooling=true;"
},
I have a GitHub Repo for a working example of an ASP.NET Core application with a Postgres database.