Is there an inbuilt health check for service fabric? I have a guest executable written in NET Core 2.2 and utilising the health check feature within it. For example, I have a simple health check that returns unhealthy state:
services
.AddHealthChecks()
.AddCheck<DocumentDbHealthCheck>("cosmos-database");
internal class DocumentDbHealthCheck : IHealthCheck
{
public Task<HealthCheckResult> CheckHealthAsync(HealthCheckContext context, CancellationToken cancellationToken = default(CancellationToken))
{
return Task.FromResult(HealthCheckResult.Unhealthy());
}
}
I have hooked this up using:
app.UseHealthChecks(#"/foo/bar/v1/healthcheck");
However, when I locally startup my service fabric instance the state is healthy, I was expecting this to be in an errored / unhealthy state.
Is it possible to have service fabric hit the API healthcheck route?
The Healthcheck introduced on AspNetCore is a mechanism to return data about the status of some service, it does not affect the actual state of the service.
In Service Fabric, If you want to report the health from within the service to the Service Fabric health system, you can use the ReportReplicaHealth() API. something like this:
HealthInformation healthInformation = new HealthInformation("ServiceCode", "StateDictionary", HealthState.Error);
this.Partition.ReportReplicaHealth(healthInformation);
This will show in the SF Explorer as an Error.
You can also report issues using the FabricClient as described here, in this case, you would create a service to monitor other services and then report their status, aka Watchdog.
AFAIK, Service Fabric does not have an HTTP Probe mechanism to check for the health of a service, it uses the internal metrics reported by the service direct to the health sub system.
If you are planning to use it to validate if a service is healthy before sending request to it, you could user the load balancer http probes or you could just put it behind a Proxy that handles failures and forward the request to a valid node, like the Built-in Reverse Proxy as described here.
Related
I have experienced unexpected behavior from Service Fabric Reverse Proxy.
When I abort a long-running request the proxy request to the Service Fabric Service is not being aborted and the whole request is executed.
If we make requests directly to the service, requests are canceled as expected.
An uninterrupted run of the long request - directly to the service
An interrupted run of the long request - directly to the service
Is there a setting that we need to enable so Service Fabric Reverse Proxy handles the requests as we expect it to?
Consider using Traefik instead, which is a more mature product.
It comes with an active community and request termination support.
It also doesn't have the undesired side effect of exposing all SF services by default.
I Have a ASP.net Core Web API and the end point goes something like this
https://{domainname}/v1/{ControllerName} and it is hosted on AWS Fargate Conatainer
and we scheduled the endpoint to be invoked at particular time of the day, but to our surprise we see that this is
getting invoked from other sources and we decided to find the remoteip from where the API end point is getting invoked
tried adding with below snippet..
var ip = this.HttpContext.Features.Get()?.RemoteIpAddress?.ToString();
and when we invoked the API with Postman from my local machine the client id is not matching with my local machineIP, Any ideas here whether is my approach right or wrong?
As I know, in Azure service fabric, service communication can be possibles either by directly accessing the endpoints (or) reverse-proxy.
Currently I'm directly accessing the other service endpoints using below code:
var service = $"fabric://myapp/**service1**";
var servicePartitionResolver = ServicePartitionResolver.GetDefault();
var partition = await servicePartitionResolver.ResolveAsync(new System.Uri(service),
new ServicePartitionKey(), default(CancellationToken));
var serviceEndpointJson = partition.GetEndpoint().Address;
string endpointUrl = JObject.Parse(serviceEndpointJson)["Endpoints"][string.Empty].Value<string>();
Will this approach work if service1 instance is not available on expected node1 but available on node2 of the SF cluster?
Is directly accessing service endpoints - approach uses 'Naming Service' like reverse-proxy approach to identify the requested service address via service url?
Any disadvantages of using above direct access call implementation to discover services for internal service communication?
The resolved endpoint should work, but you can do this in a simpler way:
Use the built-in DNS to communicate between services using HTTP.
Use SF Remoting to communicate between services.
The DNS allows you to get an endpoint by passing in the application and target service details.
SF remoting works in a similar way. The main advantage of the last, is that the SF remoting client has built-in transient error handling (it retries calls whenever the remote service is temporarily unavailable).
I have .net core API inside the web app and that web app is backend pool for azure application gateway. while trying to access the web app got below error.
"502 - Web server received an invalid response while acting as a gateway or proxy server."
On app GW, health prob for that web app in unhealthy but while access the API as a https://abc.azurewebsites.net/api/values then it works.
When we deploy API in Web App Service then apiname.azurewebsites.net does not work give any probes to application gateway and treat unhealthy. API works like xxx.azurewebsites.net/api/values and Application Gateway also know this path. We have to put /api/values in override backend path of http settings. Same have to do in health probes.
Yes, you can first verify if the backend API could access directly without app gateway. Then this error may happen due to the following main reasons:
NSG, UDR or Custom DNS is blocking access to backend pool members.
Back-end VMs or instances of virtual machine scale set are not responding to the default health probe.
Invalid or improper configuration of custom health probes.
Azure Application Gateway's back-end pool is not configured or empty.
None of the VMs or instances in virtual machine scale set are healthy.
Request time-out or connectivity issues with user requests.
Generally, the Backend healthy status and details could point it out and show some clues. You could also verify all of the above reasons one by one according to this DOC.
I have a Managed Discovery Service hosted with a known URI. I have a discoverable service that when it starts, it announces itself using an AnnouncementEndpoint added to the ServiceDiscoveryBehavior of the service.
The specific use case I would like to solve is the following:
Managed Discovery service starts.
A discoverable service starts and announces itself to the Managed Discovery service.
The Managed Discovery service is restarted (for any various possible reasons).
How then does the discoverable service refresh itself (re-announce) to the Managed Discovery service?
I know the Managed Discovery service can persist endpoints and restore them upon start but I want everything to be dynamic and self repairing so that there's no chance of stale endpoint information.
An alternative use case would be:
An existing discoverable service is running.
A new Managed Discovery service is brought online.
How do we force or invoke the same Announcement service contract call to the new Managed Discovery service?
I hope this is enough information about what I want to accomplish.
I found the answer myself. In the scenario where you need to control announcements outside of the ServiceDiscoveryBehavior , you would use the AnnouncementClient class.
AnnouncementClient client = new AnnouncementClient(announcementEndpoint);
var endpointDiscoveryMetadata = EndpointDiscoveryMetadata.FromServiceEndpoint(netTcpEndpoint);
client.AnnounceOnline(endpointDiscoveryMetadata);