How should I investigate a cause of crash on Bluemix? - crash

I got the following error messages when I pushed an app of Liberty for Java into Bluemix.
[Console]
an instance of the app crashed: failed to start exit status: -1, CRASHED
[cf events APPNAME]
index: 1, reason: CRASHED, exit_description: failed to start, exit_status: -1
index: 2, reason: CRASHED, exit_description: failed to start, exit_status: -1
a) Is there exit status code list? I can not find it.
b) This error, exit status = -1, sometimes occurred. What is it? How should I investigate this cause?

On bluemix, you can use cf logs appName command to investigate the logs.
You can also use Monitoring and analytics for investigating the logs.
Read this article to get an idea of logging mechanism on bluemix -
https://www.ibm.com/blogs/bluemix/2014/10/accessing-application-logs-bluemix/

Try cf events appname to see if there's any more crash related info. Ensure that you deployed the app with enough memory, as insufficient memory will cause a crash on startup.

Related

Azure Dataflow fails with error: Job failed due to reason: Not started

I have an azure pipeline with several dataflows and this one consistently fails with this bizarre error:
"Job failed due to reason: Not started"
I have no idea why this is happening and have not seen it addressed anywhere online. Has anyone seen this? How do I get past this?

Automatic Crash Dump for .Net process

I'm trying to set up a Windows Server 2016 instance to automatically create a crash dump when a .Net process exits with an unhandled exception. I've tried the following:
Windows Error Reporting (as described here: https://learn.microsoft.com/en-us/windows/win32/wer/wer-settings) [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps\MyApplication.exe]
"DumpType"=dword:00000002
"DumpCount"=dword:00000005
"DumpFolder"="C:\dumps"
WinDbg as postmortem debugger (windbg -I, as descibed here: https://learn.microsoft.com/en-us/windows-hardware/drivers/debugger/enabling-postmortem-debugging)
procdump as postmortem debugger (procdump -ma -i C:\dumps, as described also in the above link.)
I've tried running my application in a way that will generate an exception. All I have to do is start it twice in the same configuration, and the second instance crashes on a socket bind exception. However, in none of the cases above does the crash generate a dump file. Is there something special I have to do on Windows Server 2016, or something special I need to do to my .Net application to get it to trigger a dump on an unhandled exception?
I found the reason: we are using TopShelf to host our process. By default TopShelf handles uncaught exceptions and exits cleanly.
For any who come afterwards, please note that TopShelf has an undocumented feature from late last year to disable this behavior:
x.UnhandledExceptionPolicy = Topshelf.Runtime.UnhandledExceptionPolicyCode.TakeNoAction;
in the service configuration will do it. With this code, I was able to get a crash dump.

Troubleshooting Web App process restarting

Our web app process is restarting regularly and we are unable to determine the reason.
When looking into Application Events (using the 'Diagnostics and solve problems' blade in the Azure Portal), there exists a bunch of the following Info logs by 'IIS AspNetCore Module'
Event ID 1005:
Failed to gracefully shutdown process '14040'.
Event ID 1001:
Application 'MACHINE/WEBROOT/APPHOST/myapplication__xxxx' started process '31628' successfully and is listening on port '17663'.
There is nothing fishy with general resource usage and nothing in our application logs.
What is the best way to troubleshoot the reason behind these process restarts?
EDIT 1:
After fiddling around with web logging in the Web App's Diagnostic Logs, I now get an error logged from W3SVC-WP after each restart, but the message is nonsense:
1<br/>5<br/>50000780
EDIT 2:
Event Id 2284 refers to this:
FailedRequestTracing module failed to write buffered events to log
file for the request that matched failure definition. No logs will be
generated until this condition is corrected. The problem happened at
least %1 times in the last %2 minutes. The data is the error.
I'm not sure if this could be related to our Diagnostic Logs configuration, but seems unlikely.
EDIT 3:
As per Brando Zhang's suggestion, I've used the Web App Crash Diagnoser extension and tried monitoring 2nd Chance Unhandled Exceptions on both my application process AND on w3wp, but nothing is dumped.
From how I understand it, 1st Chance Exceptions will not crash the process, so no need to monitor these.
Very likely application is crashing due to fatal exception and causing the restarts.
On Azure App Service platform.You can use the Diagnostics as a
Service (DaaS) to troubleshoot this
It can also do an analysis and tell you the root cause most of the time.More step by step infofrmation can be found on this msdn blog .Also refer tips for using crash diagnoser

Repeated IBM bluemix Node Red app crashing; status 1

My Node Red application in IBM BlueMix is repeatedly crashing - once an hour - with no real error message other than "exited with status: 1."
How can I troubleshoot this issue?
Is there someone from IBM BlueMix support that monitors this that could take a look?
I looked at my logs and there's nothing in there that really says what's going on.
Edit per requests:
The regular log for "OUT/ERR" is scrolling so fast with HTTPD logs that I can't get it to copy/paste. Filtering to "ERR" Channel the only thing I see is below. I believe this is an error which occurs during deploy when the application restarts.
[App/0] ERR js-bson: Failed to load c++ bson extension, using pure JS version
My Node Red application is gathering data from Wink, LIFX, and other IoT services and compiles them together into a Freeboard dashboard.
Caught crash on screenshot here -- not enough cred to post images so it'll only post as a link
The zlib error was fixed in the 0.13.2 Node-RED release (that shipped 19/02/16).
If you re-stage your application is should pick up the new version of Node-RED
You can re-stage the application using the cf command line management application:
cf restage <app name>

Web service timeout error

We are running SQL Server 8. Every day when our users try to run a report for the first time it takes too long and eventually displays the error message below (other subsequent runs for the report work fine). Here is the error message.
Server Error in '/' Application.
The operation has timed out
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.
Exception Details: System.Net.WebException: The operation has timed out
Can anyone help with this issue?
Thanks.
If it is only slow on the first run, you can fix it by running a "warm-up" script. This can be as simple as setting a timer to call a web address (a report in your SSRS). The Windows Task Scheduler is pretty easy for this purpose. I've seen people do it for web servers, DBs, reports, nearly anything that is slow on the first run (due to caching or just-in-time compliling).