Asp.Net Core 2.2.0
For a failed health-check how to return
For example :
using Microsoft.Extensions.Diagnostics.HealthChecks;
public class SqlConnectionHealthCheck : IHealthCheck
{
public async Task<HealthCheckResult> CheckHealthAsync(HealthCheckContext context, CancellationToken cancellationToken = default(CancellationToken))
{
try
{
// attempt connection to db
}
catch(DbException ex)
{
return new HealthCheckResult(status: context.Registration.FailureStatus, exception: ex);
}
When this fails the browser shows Unhealthy
How to display the exception message and the stack trace ?
How to display the exception message
// one option:
catch(DbException ex)
{
return new HealthCheckResult(status: context.Registration.FailureStatus, description: exception.Message, exception: ex);
}
and the stack trace
Are you sure you want to do that?
Related
I've implemented healthchecks in my asp.net core application.
One healthcheck does 2 checks - DbContext connection and custom one that checks NpgsqlConnection.
Everything works fine in over 99% of cases. Occasionally healthcheck fails throwing TaskCanceledException or OperationCanceledException. From my logs I can see that this exceptions are thrown after around 2ms-25ms (so there is no chance any timeout happened).
Important hint:
When I hit healtchecks many times (simple F5 in browser) it throws the exception. Looks like you can't hit /health endpoint before previous healthcheck is completed. If this is the case - why? Even if I put Thread.Sleep(5000); in custom healthcheck (an no DB connection check at all) it will fail if I hit /health endpoint before 5 seconds passes.
QUESTION: Is healtheck somehow 'magically' single-threaded (when you hit that endpoint again, it cancels previous healthcheck invocation)?
Startup.cs ConfigureServices
services
.AddHealthChecks()
.AddCheck<StorageHealthCheck>("ReadOnly Persistance")
.AddDbContextCheck<MyDbContext>("EFCore persistance");
Startup.cs Configure
if (env.IsDevelopment())
{
app.UseDeveloperExceptionPage();
}
else
{
app.UseHsts();
}
app.UseHttpsRedirection();
app.UseCors(options => options.AllowAnyOrigin().AllowAnyMethod().AllowAnyHeader());
app.UseMiddleware<RequestLogMiddleware>();
app.UseMiddleware<ErrorLoggingMiddleware>();
if (!env.IsProduction())
{
app.UseSwagger();
app.UseSwaggerUI(c =>
{
c.SwaggerEndpoint("/swagger/v1/swagger.json", "V1");
c.SwaggerEndpoint($"/swagger/v2/swagger.json", $"V2");
});
}
app.UseHealthChecks("/health", new HealthCheckOptions()
{
ResponseWriter = WriteResponse
});
app.UseMvc();
StorageHealthCheck.cs
public class StorageHealthCheck : IHealthCheck
{
private readonly IMediator _mediator;
public StorageHealthCheck(IMediator mediator)
{
_mediator = mediator;
}
public async Task<HealthCheckResult> CheckHealthAsync(HealthCheckContext context, CancellationToken cancellationToken = default(CancellationToken))
{
var isReadOnlyHealthy = await _mediator.Send(new CheckReadOnlyPersistanceHealthQuery());
return new HealthCheckResult(isReadOnlyHealthy ? HealthStatus.Healthy : HealthStatus.Unhealthy, null);
}
}
CheckReadOnlyPersistanceHealthQueryHandler:
NpgsqlConnectionStringBuilder csb = new NpgsqlConnectionStringBuilder(_connectionString.Value);
string sql = $#"
SELECT * FROM pg_database WHERE datname = '{csb.Database}'";
try
{
using (IDbConnection connection = new NpgsqlConnection(_connectionString.Value))
{
connection.Open();
var stateAfterOpening = connection.State;
if (stateAfterOpening != ConnectionState.Open)
{
return false;
}
connection.Close();
return true;
}
}
catch
{
return false;
}
TaskCanceledException:
System.Threading.Tasks.TaskCanceledException: A task was canceled.
at Npgsql.TaskExtensions.WithCancellation[T](Task`1 task, CancellationToken cancellationToken)
at Npgsql.NpgsqlConnector.ConnectAsync(NpgsqlTimeout timeout, CancellationToken cancellationToken)
at Npgsql.NpgsqlConnector.RawOpen(NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken)
at Npgsql.NpgsqlConnector.Open(NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken)
at Npgsql.NpgsqlConnection.<>c__DisplayClass32_0.<<Open>g__OpenLong|0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlDatabaseCreator.ExistsAsync(CancellationToken cancellationToken)
at Microsoft.Extensions.Diagnostics.HealthChecks.DbContextHealthCheck`1.CheckHealthAsync(HealthCheckContext context, CancellationToken cancellationToken)
at Microsoft.Extensions.Diagnostics.HealthChecks.DefaultHealthCheckService.CheckHealthAsync(Func`2 predicate, CancellationToken cancellationToken)
at Microsoft.AspNetCore.Diagnostics.HealthChecks.HealthCheckMiddleware.InvokeAsync(HttpContext httpContext)
at Microsoft.AspNetCore.Builder.Extensions.MapWhenMiddleware.Invoke(HttpContext context)
OperationCanceledException:
System.OperationCanceledException: The operation was canceled.
at System.Threading.CancellationToken.ThrowOperationCanceledException()
at Microsoft.Extensions.Diagnostics.HealthChecks.DefaultHealthCheckService.CheckHealthAsync(Func`2 predicate, CancellationToken cancellationToken)
at Microsoft.AspNetCore.Diagnostics.HealthChecks.HealthCheckMiddleware.InvokeAsync(HttpContext httpContext)
at Microsoft.AspNetCore.Builder.Extensions.MapWhenMiddleware.Invoke(HttpContext context)
I've finally found the answer.
The initial reason is that when the HTTP request is aborted, then httpContext.RequestAborted CancellationToken is triggered, and it throws an exception (OperationCanceledException).
I have a global exception handler in my application, and I have been converting every unhandled exception to a 500 error. Even though the client aborted the request, and never got the 500 response, my logs kept logging this.
The solution I implemented is like that:
public async Task Invoke(HttpContext context)
{
try
{
await _next(context);
}
catch (Exception ex)
{
if (context.RequestAborted.IsCancellationRequested)
{
_logger.LogWarning(ex, "RequestAborted. " + ex.Message);
return;
}
_logger.LogCritical(ex, ex.Message);
await HandleExceptionAsync(context, ex);
throw;
}
}
private static Task HandleExceptionAsync(HttpContext context, Exception ex)
{
var code = HttpStatusCode.InternalServerError; // 500 if unexpected
//if (ex is MyNotFoundException) code = HttpStatusCode.NotFound;
//else if (ex is MyUnauthorizedException) code = HttpStatusCode.Unauthorized;
//else if (ex is MyException) code = HttpStatusCode.BadRequest;
var result = JsonConvert.SerializeObject(new { error = ex.Message });
context.Response.ContentType = "application/json";
context.Response.StatusCode = (int)code;
return context.Response.WriteAsync(result);
}
hope it helps to somebody.
My best theory, after testing in a large production environment, is that you need to await any writers to the http context output stream in the health check. I was getting this error in a method where I returned a task that was not awaited. Awaiting the task appears to have solved the problem. The nice thing about await is that you could also catch a TaskCancelledException and just eat it.
Example:
// map health checks
endpoints.MapHealthChecks("/health-check", new HealthCheckOptions
{
ResponseWriter = HealthCheckExtensions.WriteJsonResponseAsync,
Predicate = check => check.Name == "default"
});
/// <summary>
/// Write a json health check response
/// </summary>
/// <param name="context">Http context</param>
/// <param name="report">Report</param>
/// <returns>Task</returns>
public static async Task WriteJsonResponseAsync(HttpContext context, HealthReport report)
{
try
{
HealthReportEntry entry = report.Entries.Values.FirstOrDefault();
context.Response.ContentType = "application/json; charset=utf-8";
await JsonSerializer.SerializeAsync(context.Response.Body, entry.Data,entry.Data.GetType());
}
catch (TaskCancelledException)
{
}
}
I have a RestEasy based service in which I am doing some cleanup work in a ContainerResponseFilter. The problem is that if an unknown runtime exception (i.e. an exception for which I do not have a mapper) is thrown by a resource, the ContainerResponseFilter is never executed.
Is this the expected behavior? Is there a workaround to this? I was looking at the following question (answer by Jonas):
How should I log uncaught exceptions in my RESTful JAX-RS web service?
and that made it seem like the ContainerResponseFilter is executed even when an exception is thrown in the controller?
Am I missing something?
Didn't work for me either. This claims it should work: https://github.com/Graylog2/graylog2-server/issues/1826
I didn't want to investigate further, and simply use a plain old javax.servlet.Filter, but of course there it's hard to set the reponses-headers (after chain.doFilter(), ... grr..
So used a Spring solution:
public static class MyFilter extends OncePerRequestFilter {
#Override
protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain filterChain) throws ServletException, IOException {
response.setHeader("MyHeader", "MyValue");
filterChain.doFilter(request, response);
}
}
Base on the source code of SynchronousDispatcher in RestEasy.
protected void writeResponse(HttpRequest request, HttpResponse response, Response jaxrsResponse) {
try {
ServerResponseWriter.writeNomapResponse((BuiltResponse)jaxrsResponse, request, response, this.providerFactory);
} catch (Exception var5) {
this.writeException(request, response, var5);
}
}
public void writeException(HttpRequest request, HttpResponse response, Throwable e) {
if (response.isCommitted()) {
throw new UnhandledException("Response is committed, can't handle exception", e);
} else {
Response handledResponse = (new ExceptionHandler(this.providerFactory, this.unwrappedExceptions)).handleException(request, e);
if (handledResponse == null) {
throw new UnhandledException(e);
} else {
try {
ServerResponseWriter.writeNomapResponse((BuiltResponse)handledResponse, request, response, this.providerFactory);
} catch (Exception var6) {
throw new UnhandledException(var6);
}
}
}
}
ContainerResponseFilter will not execute.
If you want to set headers when exceptions happen, you need an exception handler to deal with it.
I am working on WCF Services from Couple of Days. I had a written a service with an Exception as
public String Login(string Vendorname, string VendorAccessCode)
{
try
{
if()
{
}
else
{
UserIdentityToken = string.Empty;
this.ErrorMessage = "Authentication failed. Please contact administrator";
throw new FaultException(this.ErrorMessage);
}
}
catch (FaultException ex)
{
logger.Error(ex.Message);
}
catch (Exception ex)
{
logger.Error(ex.Message);
}
return UserIdentityToken;
}
After this i am handling the exceptions in client side in a Messgae Inspector as
public class MessageInspector : IClientMessageInspector
{
public void AfterReceiveReply(ref Message reply, object correlationState)
{
if (reply.IsFault)
{
MessageFault fault = MessageFault.CreateFault(new FaultCode("Receiver"), new FaultReason(reply.ToString()));
throw new FaultException(fault);
}
}
}
I am handling my Client Side Code as
try
{ objVendorServiceClient.Login(txtuserName.Text.Trim(),
txtAccessCode.Text.Trim());
}
catch (FaultException ex)
{
lblAuthenticationMessage.Text = ex.Message;
throw ex;
}
But when ever the authentication fails in Service, the Reply.IsFault is returning false only. could any one explain me what is reply.Isfault and how exactly it is useful to me?
Message.IsFault Property:
Gets a value that indicates whether this message generates any SOAP faults.
You say:
when ever the authentication fails in Service, the Reply.IsFault is returning false
That is because you catch the fault yourself, log it, and then return an empty string. You'll have to throw the FaultException in order for WCF to catch it and create a SOAP fault from it.
I am trying to catch an exception in a method of the service and bubble this exception to the client.
My code is:
CONTRACT:
[OperationContract(AsyncPattern = true)]
[FaultContractAttribute(typeof(Exception))]
IAsyncResult BeginUpdateUsers(List<Users> paramUsers, AsyncCallback callback, object state);
List<Users> EndUpdateUsers(IAsyncResult result);
IMPLEMENTATION INTERFACE
public IAsyncResult BeginUpdateUsers(List<Users> paramUsers, AsyncCallback callback, object state)
{
Task<List<Users>> task= Task<List<Users>>.Factory.StartNew(p => aupdateUsersMethod(paramUsers), state);
return task.ContinueWith(res => callback(task));
}
public List<Users> EndUpdateUsers(IAsyncResult result)
{
try
{
return ((Task<List<Users>>)result).Result;
}
catch (Exception ex)
{
throw new Exception(ex.Message);
}
}
private updateUsersMethod(List<Users> paramUsers)
{
try
{
}
catch(Exception ex)
{
throw new Exception(ex.Message);
}
}
I get an error in runtime in the catch of the method updateUsersMethod. It enters in the catch, but in the line when I throw new Exception, I get the following error: the usr's code does not control the exception.
EDIT1: I solved the problem, I must throw FaultException instead of the Exception. However, the exception dos not arrive to the client. which could be the reason?
EDIT2: Well, I must to use faultException, but I am having the same problem, it says me that the faultException is not handle by the user's code. I have try/catch in all the methods, Begin, End and the auxiliar method updateUsersMethod.
I'm trying to create some custom FaultException. I've made a DataContract class called CreateFault.
[DataContract]
public class CreateFault
{
private string report;
public CreateFault(string message)
{
this.report = message;
}
[DataMember]
public string Message
{
get { return this.report; }
set { this.report = value; }
}
}
I'm then throwing the fault in a service method.
In IService1.cs
[OperationContract]
[FaultContract(typeof(CreateFault))]
void TestFaultException();
and in Service1.cs
public void TestFaultException()
{
throw new FaultException<CreateFault>(new CreateFault("CreateFault message"), "Message abt exception");
}
I catch the FaultException in my client.
private void btnTest_Click(object sender, RoutedEventArgs e)
{
try
{
ServiceReference1.Service1Client client = new ServiceReference1.Service1Client();
client.TestFaultException();
}
catch (FaultException<CreateFault> ex)
{
MessageBox.Show(ex.Detail.Message, "Success", MessageBoxButton.OK, MessageBoxImage.Error);
}
catch (FaultException ex)
{
MessageBox.Show(ex.Message, "Failure", MessageBoxButton.OK, MessageBoxImage.Error);
}
catch (Exception ex)
{
}
}
Now here comes the problem. When I create a WCF Service Application project in Visual Studio 2010 it works like expected. The error is caught in:
catch (FaultException<CreateFault> ex)
But when I create a WCF Service Library project with my custom FaultExceptions the client does not recognize my custom exception. It instead catches the error in:
catch (FaultException ex)
Why does it not work with WCF Service Application Project?
Edit:
This is what i get during debugging when it catches the exception in
catch (FaultException ex)
(typed ?ex in Immediate window)
{"Message abt exception"}
[System.ServiceModel.FaultException<WpfApplication1.ServiceReference2.CreateFault>]: {"Message abt exception"}
base {System.ServiceModel.CommunicationException}: {"Message abt exception"}
Action: "http://tempuri.org/IService1/TestFaultExceptionCreateFaultFault"
Code: {System.ServiceModel.FaultCode}
Message: "Message abt exception"
Reason: {Message abt exception}
Edit2:
Found the problem. I had two Service references who both had the CreateFault DataContract. And it was using the wrong one when i ran the program.
When i changed to
catch (FaultException<ServiceReference2.CreateFault> ex)
it worked
Found the problem. I had two Service references who both had the CreateFault DataContract. And it was using the wrong one when i ran the program.
When i changed to
catch (FaultException<ServiceReference2.CreateFault> ex)
it worked