I'm trying to set up a Windows Server 2016 instance to automatically create a crash dump when a .Net process exits with an unhandled exception. I've tried the following:
Windows Error Reporting (as described here: https://learn.microsoft.com/en-us/windows/win32/wer/wer-settings) [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps\MyApplication.exe]
"DumpType"=dword:00000002
"DumpCount"=dword:00000005
"DumpFolder"="C:\dumps"
WinDbg as postmortem debugger (windbg -I, as descibed here: https://learn.microsoft.com/en-us/windows-hardware/drivers/debugger/enabling-postmortem-debugging)
procdump as postmortem debugger (procdump -ma -i C:\dumps, as described also in the above link.)
I've tried running my application in a way that will generate an exception. All I have to do is start it twice in the same configuration, and the second instance crashes on a socket bind exception. However, in none of the cases above does the crash generate a dump file. Is there something special I have to do on Windows Server 2016, or something special I need to do to my .Net application to get it to trigger a dump on an unhandled exception?
I found the reason: we are using TopShelf to host our process. By default TopShelf handles uncaught exceptions and exits cleanly.
For any who come afterwards, please note that TopShelf has an undocumented feature from late last year to disable this behavior:
x.UnhandledExceptionPolicy = Topshelf.Runtime.UnhandledExceptionPolicyCode.TakeNoAction;
in the service configuration will do it. With this code, I was able to get a crash dump.
Related
I have a WCF application which consists in some async communications with ecternal services. When we start a new expedient, a new instance is created; it process data and send an xml to a external service and waits for the response. This response requires that a person review the xml and send the response so it usually it is delayed for a long time. For this reason, the workflow go to idle and we use persistence with AppFabric.
The fact is that sometime, when we receive the response, the next exception is raised:
The execution of the InstancePersistenceCommand named {urn:schemas-microsoft-com:System.Activities.Persistence/command}LoadWorkflowByInstanceKey was interrupted by an error.
Normally this error does not occur, it can occur very sporadically. However, we are trying to update the app to include a new functionality (it does not modify the workflow) but when the application is deployed to the server, the instances that were created with the old deployment and were waiting for the response, throw this exception when they receive the response from the external service. However, the instances initiated with the new deployment process the response without problem.
I have been looking for information about this problem but I haven't found much. Anybody can help me?
SOLUTION:
Thanks a lot for your answer, it may be helpful for me in the future. In this case, the problem was that I was updating an assembly version of one of the implicated project (to upload a nuget package) and for a reason that I don’t understand, the instances created with an old version raised this exception when the service with the new version had to manipulate the mentioned instances.
If I change the assembly version to upload the nuget and then set the original version and deploy with this version, everything works ok. Anybody knows what is the reason?
Thanks a lot.
This may be because there is a program running in the background and trying to extend the lock on the instance store every 30 seconds, and it seems that whenever the connection to the SQL service fails, it marks the instance store as invalid.
You can try <workflowIdle timeToUnload="0"/>, if it doesn't work you can look at the methods provided by other links.
Windows workflow 4.0 InstancePersistenceCommand Error
Why do I get exception "The execution of the InstancePersistenceCommand named LoadWorkflowByInstanceKey was interrupted by an error"
WF4 InstancePersistenceCommand interrupted
Our web app process is restarting regularly and we are unable to determine the reason.
When looking into Application Events (using the 'Diagnostics and solve problems' blade in the Azure Portal), there exists a bunch of the following Info logs by 'IIS AspNetCore Module'
Event ID 1005:
Failed to gracefully shutdown process '14040'.
Event ID 1001:
Application 'MACHINE/WEBROOT/APPHOST/myapplication__xxxx' started process '31628' successfully and is listening on port '17663'.
There is nothing fishy with general resource usage and nothing in our application logs.
What is the best way to troubleshoot the reason behind these process restarts?
EDIT 1:
After fiddling around with web logging in the Web App's Diagnostic Logs, I now get an error logged from W3SVC-WP after each restart, but the message is nonsense:
1<br/>5<br/>50000780
EDIT 2:
Event Id 2284 refers to this:
FailedRequestTracing module failed to write buffered events to log
file for the request that matched failure definition. No logs will be
generated until this condition is corrected. The problem happened at
least %1 times in the last %2 minutes. The data is the error.
I'm not sure if this could be related to our Diagnostic Logs configuration, but seems unlikely.
EDIT 3:
As per Brando Zhang's suggestion, I've used the Web App Crash Diagnoser extension and tried monitoring 2nd Chance Unhandled Exceptions on both my application process AND on w3wp, but nothing is dumped.
From how I understand it, 1st Chance Exceptions will not crash the process, so no need to monitor these.
Very likely application is crashing due to fatal exception and causing the restarts.
On Azure App Service platform.You can use the Diagnostics as a
Service (DaaS) to troubleshoot this
It can also do an analysis and tell you the root cause most of the time.More step by step infofrmation can be found on this msdn blog .Also refer tips for using crash diagnoser
While using Windows Azure Table Storage in WCFService WebRole, tried to create CloudStorageAccount by the following way:
storageAccount =
CloudStorageAccount.Parse(Microsoft.WindowsAzure.CloudConfigurationManager.GetSetting("[Setting name]"))
Get exception:
ConfigurationErrorsException "Could not create Microsoft.WindowsAzure.Diagnostics.DiagnosticMonitorTraceListener, Microsoft.WindowsAzure.Diagnostics, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35."
MSDN help says that 1) Visual Studio must be run as an administrator. 2) A role must be running under full trust (change the .NET trust level option to Full Trust).
All Done, but I still have the same exception.
One thing that can cause this error is running the web role itself, instead of running the containing cloud project. If this is the issue, you could fix it by ensuring that the cloud project is set as the startup project for debugging, and not the web role.
It's possible, and sometimes useful, to run the ASP.NET project that defines the web role on its own. This can be a lot quicker than running things in the Azure Compute Emulator. It may also enable you to develop your project without having to run VS elevated. Also, I've found that the emulator tends to cause Visual Studio to report an invalid memory access error from time to time, at which point you need to restart VS. Running the web role directly avoids all these problems.
However, there are some things that can prevent this from working, and the exception you describe is a symptom of one of these problems. If your web role's Web.config includes configuration for Azure's DiagnosticMonitorTraceListener (and Visual Studio adds that by default when you create a web role) then the first thing that tries to generate trace output will crash with the error you describe if you run outside the emulator. And as it happens, retrieving a setting from the CloudConfigurationManager appears to do this.
This isn't peculiar to the CloudConfigurationManager by the way. All it's doing is producing some trace output. VS configures web roles to send all trace output to the Azure diagnostic listener, and because that listener can only run in either the compute emulator or an actual Azure instance, the first thing that tries to produce trace output will crash. CloudConfigurationManager is a common candidate because it happens to produce trace output, and it typically gets used early on when a role starts up. But in principle, anything that produces trace output could hit this exception.
A simple way to avoid this is to remove the relevant section from the configuration file. When you create a new web role, Visual Studio adds a <system.diagnostics> section that configures the default trace output to go to the Azure diagnostic listener. You could just comment that out. That will enable you to debug the web role directly in Visual Studio without using the compute emulator (assuming you aren't doing anything else that depends on being in a role environment).
Of course, the problem with that is that you'll no longer get any diagnostic traces when running in Azure. One way to solve that is to move the relevant configuration to the Web.config.Release file (adding the necessary xdt: attributes).
This change will also stop the Azure diagnostic trace listener from running when you use the local compute emulator. (That's less of a problem, because the trace messages will still appear in the debugger. It just means you won't get persistent copies of the traces copied to table storage like you would when running for real.) The obvious way to fix this would seem to be to make a similar modification to Web.config.Debug (or to run the release build in the emulator), but there's a snag: apparently cloud projects do not apply configuration file transforms when packaging for the emulator by default. Fortunately, you can fix this: http://blog.hill-it.be/2011/03/07/no-web-config-transformation-in-local-azure/ shows how to enable transforms for local debugging in the compute emulator. (Transforms are never applied when debugging an ASP.NET project directly from within VS, by the way.)
I've found that this error is caused by the wrong version in your web.config
Ie., you may not have
Version=1.0.0.0
Microsoft.WindowsAzure.Diagnostics is up to version 1.8.0.0 as of now
Try updating to the current version
Remove the lines in Web.config < add type="Microsoft.WindowsAzure.Diagnostics.DiagnosticMonitorTraceListener
I have a ASPMVC project (vb.net; ASPMVC-2 if it makes any difference) in VS2010. It's been fine for some years, but now I have a weird situation. If I do the following:
Start debug mode in VS
Go through the site
Stop debugging but leave the asp.net development server running so that i can still test.
Put VS back into debug mode
Then the site runs fine for the first couple of steps but when it gets to a certain line of code it throws a System.ExecutionEngineException. The development server then stops (I only see the execption becuase VS is in debug mode - the web browser just says "cannot connect" and the action filter which usually logs errors to a database doesn't run either because the whole web server process bails out). If I, without changing anything, simply put VS back into debug mode then it works perfectly, without throwing any exception on the problematic command. This happens almost every time (the exception that is, it's always OK if I do a clean start of the development server).
The row which causes the problem is an if statement using .ContainsValue() on an object which is of type SortedDictionary(Of Integer, String).
The full exception message is:
System.ExecutionEngineException was unhandled
Message=Exception of type 'System.ExecutionEngineException' was thrown.
InnerException:
All I can find on MSDN is that this exception should never happen! Googling throws up a couple of people meeting this exception, sometimes tied to a specific machine, but no real explanations. I haven't had a problem before and have been developing this site on this machine for some years now. The row which triggers the exception is new in this version so that may not say much.
Does anyone have any ideas? The fact that it always works on a clean start is slightly comforting, but I'm not happy about shipping the code until I know why it does this and can prevent it.
I'm receiving the following error after I attach the visual studio 2010 profiler to my wcf service.
The requested service,
'net.tcp://host:port/path/myservice.svc'
could not be activated. See the
server's diagnostic trace logs for
more information.
The service works great otherwise, and as soon as I stop profiling, I can connect and use the service again.
The closest I could find to a similar situation online was here, but the Net.Tcp Listener Adapter does not stop on my server, and restarting it does not help either. Neither does an iisreset.
I'm receiving the following entry in the event log:
Failed to initialize the
AppDomain:/LM/W3SVC/1/ROOT/MyService
Exception: System.Web.HttpException
Message: Could not load file or
assembly 'MyService' or one of its
dependencies. An attempt was made to
load a program with an incorrect
format. InnerException:
System.Configuration.ConfigurationErrorsException
Message: Could not load file or
assembly 'MyService' or one of
its dependencies. An attempt was made
to load a program with an incorrect
format.
InnerException:
System.BadImageFormatException
Message: Could not load file or
assembly 'MyService' or one of
its dependencies. An attempt was made
to load a program with an incorrect
format.
Does this sound like the profiler is creating a bad dll when instrumenting? If so, how would I get around that?
There were a couple issues at work here.
1) the binaries were being instrumented as 32 bit instead of 64 bit. This seemed to be a problem with the visual studio profiler, but also could have been a symptom of issue 2) I was attempting to profile remotely instead of on the physical machine which isn't supported.
I installed the command line tools on the test server and am looking to profile that way.