Whenever an error occurs within our API, we end up getting multiple emails for a single error. Based on the log messages; we can see that these other emails seem to be getting generated because various Microsoft libraries are calling something like _logger.LogError as well as our own _logger.LogError that happens when we handle the error.
For example, when there is a database timeout, we see 4 emails from these different classes:
Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware
Microsoft.EntityFrameworkCore.Database.Command
Microsoft.EntityFrameworkCore.Query
Web.Controllers.ErrorController (Our own exception handler; this is
the only one we want to see)
The last one is the only one that has our own error formatting with helpful information in it such as current user, etc. The others just contain a stack trace, which is already within our own formatted email.
We can't figure out for sure where these other log messages are coming from; but the most likely thing I can think of is that within Microsoft's libraries, they are calling _logger.LogError(), and our own NLog configuration is handling all instances of LogError; instead of just handling our own.
How can we prevent these other log statements from being logged and especially emailed to us?
This is our setup in Program.cs:
public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
WebHost.CreateDefaultBuilder(args)
.UseStartup<Startup>()
.ConfigureLogging(logging =>
{
logging.ClearProviders();
logging.SetMinimumLevel(Microsoft.Extensions.Logging.LogLevel.Trace);
})
.UseNLog();
You could filter this in .NET Core - because your using the Microsoft.Extensions.Logging integration and Microsoft send the messages to that - and in NLog.
Configure in .NET Core
Modify you config, e.g. appsettings.json. For example at least an error for all Microsoft.* and Warning for Microsoft.EntityFrameworkCore.*
{
"Logging": {
"IncludeScopes": false,
"LogLevel": {
"Default": "Trace",
"Microsoft": "Error",
"Microsoft.EntityFrameworkCore": "Warning"
}
}
}
Note, this isn't NLog specific so you can't use the NLog level names. possible level names
Read more about this approach here.
Configure NLog
Or you could configure in the NLog config.
In the NLog.config, edit the <rules> .Those rules are processed from top to bottom.
You could filter the namespace with the name attribute (* are allowed)
Without a writeTo attribute, the logs are discarded
If a rule has final="true" and matching the events, the next rule won't be processed.
For example:
<rules>
<!--All logs, including from Microsoft-->
<logger name="*" minlevel="Trace" writeTo="allfile" />
<!--Skip non-critical Microsoft logs and so log only own logs-->
<logger name="Microsoft.*" maxlevel="Info" final="true" /> <!-- BlackHole without writeTo -->
<logger name="*" minlevel="Trace" writeTo="ownFile-web" />
</rules>
You could read about the nlog.config rules here.
This is also possible with the same approach from code, see here.
Related
We are using NLog in our .net 5.0 Web API and feel like log levels are set in multiple places. Is there a way we can configure nLog to use the app settings and ignore nLog.config log levels?
nlog.config:
<rules>
<logger name="*" levels="Trace,Debug,Info,Warn,Error,Fatal" writeTo="NLogTarget"/>
</rules>
AppSettings.json:
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft": "Warning",
"Microsoft.Hosting.Lifetime": "Debug"
}
}
Thanks in advance.
If only using Microsoft ILogger then you can just use this in NLog (and rely on Microsoft LoggerFactory filtering from appsettings.json):
<rules>
<logger name="*" writeTo="NLogTarget"/>
</rules>
Notice when upgrading to NLog 5.0, then remember to configure NLogProviderOptions.RemoveLoggerFactoryFilter = false on AddNLog or UseNLog (With latest NLog then the setting is already there, so you can set it upfront already now).
Most users gets confused by having 2 log-filtering systems, and expects when using NLog that it is NLog filters that applies. Therefore changed it to to ignore Microsoft LoggerFactory filters by default.
Notice you can also have the NLog config in appsettings.json. See also https://github.com/NLog/NLog.Extensions.Logging/wiki/NLog-configuration-with-appsettings.json
I am about to log information of the Spider in some IActionResult to a dividual file with the date.
For example:
C:\20210317spider.txt
C:\20210318spider.txt
C:\20210319spider.txt
I don't want to use some method such as Logger.LogInformation to log all information into a file.
How can I achieve this? Thank you.
You can use the ${shortdate} layoutrenderer inside the FileName-Layout for the NLog FileTarget:
<nlog>
<targets>
<target type="file" name="spiderFile" fileName="C:/${shortdate}spider.txt" />
</targets>
<rules>
<logger name="Spider" writeTo="spiderFile" final="true" />
</rules>
</nlog>
See also the NLog Tutorial and the list of available layoutrenders that can be used with NLog Layout.
Then you can do this with Microsoft ILoggerFactory:
public class MyClass
{
private readonly ILogger _spiderLogger;
public MyClass(ILoggerFactory loggerFactory)
{
_spiderLogger = loggerFactory.CreateLogger("Spider");
_spiderLogger.LogInformation("Hello little spider");
}
}
I have multiple WCF services hosted in IIS to which I'm applying the same custom service behavior. I'm looking for a way to pass several parameters to the behavior through configuration, such as in behaviorExtensions or behavior elements.
If it helps, I'm also adding custom message inspector in ApplyDispatchBehavior, so I will need to pass parameters to the inspector:
public void ApplyDispatchBehavior(ServiceDescription serviceDescription, ServiceHostBase serviceHostBase)
{
foreach (ChannelDispatcher channelDispatcher in serviceHostBase.ChannelDispatchers)
{
foreach (EndpointDispatcher endpointDispatcher in channelDispatcher.Endpoints)
{
endpointDispatcher.DispatchRuntime.MessageInspectors.Add(
new ValidatingMessageInspector(<custom parameters>));
}
}
}
Would just creating a configuration section in web.config with the parameters be valid? If so you can just read the config there and apply it, or even just use appSettings if the parameters are basic.
For example, if you have few parameters of basic types, maybe a simple approach could be:
<appSettings>
<add key="Shared.CommonParameter" value="A value" />
<add key="Service1.OneParameter" value="False" />
<add key="Service1.AnotherParameter" value="Some Value" />
<add key="Service2.ADifferentParameter" value="42" />
</appSettings>
That way it would be easy to differentiate what setting belongs to which service (notice the service name prefix in the key), and also have some shared parameters if needed.
If you need something more complex in structure you might want to look into defining custom configuration sections for each service, as is shown here: http://msdn.microsoft.com/en-us/library/2tw134k3%28v=vs.140%29.aspx
But that might be overkill, depending on your needs and expected flexibility.
I have a small set of ServiceStack REST services that is using NLog 2.1 (from NuGet) for logging.
My test server is running:
Windows 7
IIS 7.5
.NET 4.5
NLog config:
<nlog throwExceptions="true" internalLogToConsole="true" internalLogLevel="Debug" xmlns="http://www.nlog-project.org/schemas/NLog.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<targets>
<target name="c" xsi:type="Console" />
<target name="f1" xsi:type="File" fileName="C:\logs\test.log" />
</targets>
<rules>
<logger name="*" writeTo="c,f1" />
</rules>
</nlog>
My NLog config is exceedingly simple... just trying to get it working...
and in this configuration, everything works fine. NLog creates the log files correctly.
On my DEVELOPMENT machine, I am using:
Windows 7
Xamarin Studio / XSP4
Mono 3.2.3
Here is my Application_Start...
protected void Application_Start() {
LogManager.LogFactory = new NLogFactory();
ILog log = LogManager.GetLogger(typeof(Global));
log.Info("Application_Start called");
try {
new AppHost().Init();
} catch (Exception e) {
log.Error("Exception caught initializing AppHost");
}
}
In this configuration, my service's AppHost().Init() throws an exception as ServiceStack is registering my services in ServiceController.cs. I believe that part is irrelevant except that it is the first time something is logged outside of Application_Start (because both of the calls in Application_Start work... the log.info before the exception and the log.error after the exception).
Here is the exception that is shown:
The most relevant bit is that there was a System.NotImplementedException thrown at NLog.Internal.FileAppenders.BaseFileAppender.WindowsCreateFile (System.String fileName, Boolean allowConcurrentWrite).
I have found a workaround (in the accepted answer below). Hopefully anyone else who runs into this will quickly come upon this solution.
Some digging around on Google led me to this NLog pull request:
Avoid Win32-specific file functions in Mono where parts not implemented.
It appears that this change tries to use the preprocessor to NOT call WindowsCreateFile at all. However, for some reason, this still executes.
So I then went to check the newest version of BaseFileAppender.cs in the NLog GitHub repository to make sure someone didn't at some later point break this again.
#if !NET_CF && !SILVERLIGHT && !MONO
try
{
if (!this.CreateFileParameters.ForceManaged && PlatformDetector.IsDesktopWin32)
{
return this.WindowsCreateFile(this.FileName, allowConcurrentWrite);
}
}
catch (SecurityException)
{
InternalLogger.Debug("Could not use native Windows create file, falling back to managed filestream");
}
#endif
Hmmm... it's still there. What gives? Why doesn't MONO seem to be defined by the preprocessor, thus allowing this block to execute? I'm not sure. Before I started down that path of investigation, I noticed another change...
if (!this.CreateFileParameters.ForceManaged && ...
So after following that tracing that ForceManaged boolean back to it's origin, it appeared that I could set forceManaged="true" on my FileTarget declaration in my NLog config. Could it really be that simple?!
<nlog throwExceptions="true" internalLogToConsole="true" internalLogLevel="Debug" xmlns="http://www.nlog-project.org/schemas/NLog.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<targets>
<target name="c" xsi:type="Console" />
<target name="f1" xsi:type="File" forceManaged="true" fileName="C:\logs\test.log" />
</targets>
<rules>
<logger name="*" writeTo="c,f1" />
</rules>
</nlog>
Once that change was made, everything worked... the call to WindowsCreateFile that was throwing the exception was skipped & a managed filestream was used instead, which works great. Hallelujah!
Lastly, I could not find that forceManaged flag in the NLog documentation anywhere. Would likely have found this solution sooner if it was.
On Server I'm throwing the exception like this.
catch(SqlException exception)
{
if (exception.Message.Contains("Custom error from stored proc"))
{
//Exception to be thrown when authentication fails.
throw new FaultException<MyServiceFault>(new MyServiceFault { MessageText = exception.Message });
}
}
And on client end I'm catching the exception
catch(FaultException<MyServiceFault> faultException)
{
}
Here is my MyServiceFault
[DataContract]
public class MyServiceFault
{
[DataMember]
public string MessageText { get; set; }
[DataMember]
public Guid Id { get; set; }
}
The problem is that on client, it doesn't go to MyServiceFault catch block instead it goes to communication exception catch block and throws this error
System.ServiceModel.CommunicationException: The underlying connection was closed: The connection was closed unexpectedly. ---> System.Net.WebException
I've also decorated my service method [FaultContract(typeof(MyServiceFault))] in the interface which is implemented by my service.
In my web.config servicebehaviour tag consist
<serviceDebug includeExceptionDetailInFaults="true" />
Any idea where I'm going wrong.
This problem is occurring on windows 7. Can there be a reason related to it?
IMPORTANT UPDATE
According to answerers, any unhandled exception on the server can cause throwing of the Communication exception on the client side and it may not have anything to do with the custom fault exception you have thrown on the server. So the solution is log the errors on the server and find what error is causing this behavior. Here is a very useful open source logging functionality I found and implemented which can even be more useful after your project goes into production environment. Many thanks to the answerers.
A better way of logging exceptions in WCF
Add logging to your WCF calls by including this in your app.config.
<system.diagnostics>
<trace autoflush="true" />
<sources>
<source name="System.ServiceModel" switchValue="Information, ActivityTracing">
<listeners>
<add name="sdt" type="System.Diagnostics.XmlWriterTraceListener" initializeData="c:\LogPath\LogFile.svclog" />
</listeners>
</source>
</sources>
</system.diagnostics>
(You can do this for both server and client, obviously specifying different log files)
Once you've generated some logs, look through for exceptions or warnings. I often find this produces some very useful information that helps me solve WCF problems.
To read the log file, you'll need to use SvcTraceViewer.exe. Unfortunately the only way to get this is with the windows SDK which is a bit of a big download for one tiny little tool.
It's also worth bearing in mind that WCF can through the CommunctionException when it is being closed, and this is expected behaviour. You shouldn't just do using with client WCF comm channels. Instead, you should follow a pattern like this:
try{
x.Close()
}
catch(Comms ex){
x.Abort()
}
One thing i learned from working with wcf is that the error CommunicationException is thrown alot of times. It is even possible that the error you get has nothing to do with the Exception you throw, but with something else causing it. At the moment something is being sent between a client and server, it's hard to track down what is causing the exception.
I could solve some of my issues by adjusting some settings in the app.config. Setting timeout, etc...
Maybe that can help?
As a follow-up, one reason for getting CommunicationException is that the FaultException could not be serialized correctly. In the log (see Simon's answer on how to set up logging), this would show up as "Handling exception" followed by "Replying to an operation threw an exception".
In my case, this was caused by not initializing an enumeration value:
[DataContract]
public class MyCustomWebServiceFault
{
public MyCustomWebServiceFault()
{
}
[DataMember]
public EMyCustomWebServiceFaultReason Reason { get; set; }
[...]
The log then revealed it:
Enum value '0' is invalid for type 'EMyCustomWebServiceFaultReason' and cannot be serialized. Ensure that the necessary enum values are present and are marked with EnumMemberAttribute attribute if the type has DataContractAttribute attribute.
Long story short, use logging or unit test exception serialization.