Request Based Sticky session with Spring Cloud Loadbalancer not working as expected - load-balancing

I am using Spring reactor and WebClient for load balance purposes. I want to use the Request-based Sticky Session for LoadBalancer as follows:
Have service A that can invoke REST end points on service B or service C. Both B and C can have multiple instances of them running at a given time.
I want to use the standard round robin balancer for service B.
I want to use the request-based sticky session balancer for service C.
This is my code in the service A (that can invokes both service B and service C) through the WebClient. But it always ends up with round robin algorithm for service C. Can someone please
help me to figure out what I am doing wrong or missed to do? I referred to the below link
https://docs.spring.io/spring-cloud-commons/docs/current/reference/html/#request-based-sticky-session-for-loadbalancer
and also looked into the below SO
Request-based Sticky Session configuration with Spring Cloud LoadBalancer
#Configuration
public class WebClientConfiguration {
#Bean
#LoadBalanced
#Qualifier("loadBalancedWebClientBuilder")
WebClient.Builder builder() {
return WebClient.builder();
}
#Bean
#Qualifier("webClientBuilder")
WebClient webClient(WebClient.Builder builder) {
return builder.build();
}
}
application.yml (I am not very sure where the sticky-session definition should be added; i.e. service A or service C and hence repeated the below in both the services)
spring:
application:
name: service-A
cloud:
loadbalancer:
ribbon:
enabled: 'false'
sticky-session:
instanceIdCookieName: sc-lb-instance-id
addServiceInstanceCookie: true
Code in the service A to invoke service C with request based sticky session. I have a different file that does the same for service B (without the sticky session but uses the same WebClient Builder).
I know the service instance id that I want the WebClient to use to route the call.
#Component
public class ServiceCClient {
#Autowired
#Qualifier("loadBalancedWebClientBuilder")
private WebClient.Builder webClientBuilder;
public Mono<Void> triggerServiceCEndPoint(String paramA, String svcInstanceId) {
return webClientBuilder.baseUrl("http://service-C").build()
.post().uri(uriBuilder -> uriBuilder.path("/v1/example/end/point")
.queryParam("param-a",paramA).build())
.headers(httpHeaders -> {
httpHeaders.set("sc-lb-instance-id",svcInstanceId);
})
.cookie("sc-lb-instance-id", svcInstanceId)
.accept(MediaType.APPLICATION_JSON)
.retrieve()
.bodyToMono(Void.class);
}
}

Related

How to integrate Prometheus with ReactiveFeignClient to meter response time/hits of each status code returned from client call

I am with a Spring Boot project with WebFlux + Spring Reactor, and it calls other services with ReactiveFeignClient.
How can I integrate Prometheus so that I could monitor response time (with a #Timer) and the percentage of each status code returned by feign call? Like 200, 400, 404... I have only found ways to:
monitor endpoint response time/status code(http_server_requests_seconds)
monitor RestTemplate (as explained here: https://docs.spring.io/spring-metrics/docs/current/public/prometheus, but I use feign)
After all the work I have done, I have seen no reactive feign clients meters in Prometheus output, even though I defined the bean of logger like:
#Bean
public MetricsWebClientFilterFunction metricsWebClientFilterFunction(PrometheusMeterRegistry meterRegistry,
WebClientExchangeTagsProvider provider) {
return new MetricsWebClientFilterFunction(
meterRegistry,
provider,
APP_NAME + "reactive-client-request",
AutoTimer.ENABLED
);
}
#Bean
public MicrometerReactiveLogger feignReactiveLogger(Clock clock,
PrometheusMeterRegistry meterRegistry) {
return new MicrometerReactiveLogger(
clock,
meterRegistry,
APP_NAME + ".feign.client_metrics",
MetricsTag.getMandatory()
);
}
Also, I found it impossible to enable /actuator/prometheus, but only /_system/check/prometheus. I did enable and expose the endpoints of metrics and prometheus.
management:
health:
diskspace:
enabled: false
endpoint:
metrics.enabled: true
prometheus.enabled: true
endpoints:
web:
base-path: /_system/check
exposure:
include: info,health,loggers,metrics,prometheus
path-mapping:
health: /simple
I found the problem: the metrics related with reactive feign clients will only show after you actually do feign calls. Before that they are hidden.
Reactive Feign Client uses WebClient underneath and will be measured automatically with presence of Spring Actuator.

Can I add a service info / health check endpoint to my Identity Server 3-based service?

I have a set of AspNet WebApi-based web services and an IdentityServer3-based authentication service. All of the web services support a simple service info endpoint that we use for monitoring and diagnosis. It reports the service version and the server name. The only service that currently does not support the service info endpoint is the IdentityServer3-based authentication service.
Is there a way to add a simple endpoint to an IdentityServer3-based service? In GitHub issue 812 Brock Allen says "We have a way to add custom controllers, but it's undocumented, current unsupported, and not really done." I'd rather not take that indocumented, unsupported route.
Is there a way to modify/extend the discovery endpoint to include additional information?
Here's how I ended up coding this up. At a high level, basically I added a Controllers folder, created a AuthenticationServiceInfoController class with a single GET action method and then registered that controller during Startup. As noted in comment above, my solution had some extra complexity because my AuthenticationServiceInfoController inherited from a base ServiceInfoController defined elsewhere, but I've tried to eliminate that from this sample. So, the controller code looks like this:
[RoutePrefix("api/v1/serviceinfo")]
public class AuthencticationServiceInfoController : IServiceInfoController
{
[Route("")]
[Route("~/api/serviceinfo")]
public IHttpActionResult Get()
{
try
{
ServiceInformation serviceInfo = new ServiceInformation();
serviceInfo.ServiceVersion = Global.serviceVersion;
return Ok(serviceInfo);
}
catch (Exception ex)
{
return InternalServerError(ex);
}
}
}
It implements a simple interface:
public interface IServiceInfoController
{
IHttpActionResult Get();
}
And in my Startup.Configuration method where I configure Identity Server, I've got:
var idSrvFactory = new IdentityServerServiceFactory();
idSrvFactory.Register(new Registration<IServiceInfoController, Controllers.AuthencticationServiceInfoController>());
I think that's all that it took. It's in place and working in my Identity Server 3-based service.

Call Service Fabric service from console application using WCF HTTPS endpoint

I have a service hosted in a Service Fabric cluster in Azure (not locally) and I'm trying to call a method in it using a console application on my local machine. Using WCF for communication, I have a HTTPS endpoint set up in my application on a specific port, and have configured load balancing rules for the port in the Azure portal. The cluster has 6 nodes and the application is the only one deployed on the cluster.
Have followed the ServiceFabric.WcfCalc on GitHub (link), which works on a local cluster using HTTP endpoints, but can't call a method on the service using HTTPS endpoints once it has been deployed. What do I need to do to get it working? Have tried following the example here but don't know how to configure this for HTTPS with a service on multiple nodes for a console application to access.
Thanks in advance.
EDIT Here's my client code which I am using to call the service method. I pass the fabric:/ URI into the constructor here.
public class Client : ServicePartitionClient<WcfCommunicationClient<IServiceInterface>>, IServiceInterface
{
private static ICommunicationClientFactory<WcfCommunicationClient<IServiceInterface>> communicationClientFactory;
static Client()
{
communicationClientFactory = new WcfCommunicationClientFactory<IServiceInterface>(
clientBinding: new BasicHttpBinding(BasicHttpSecurityMode.Transport));
}
public Client(Uri serviceUri)
: this(serviceUri, ServicePartitionKey.Singleton)
{ }
public Client(
Uri serviceUri,
ServicePartitionKey partitionKey)
: base(
communicationClientFactory,
serviceUri,
partitionKey)
{ }
public Task<bool> ServiceMethod(DataClass data)
{
try
{
//It hangs here
return this.InvokeWithRetry((c) => c.Channel.ServiceMethod(data));
}
catch (Exception)
{
throw;
}
}
}
When debugging my console application on my local machine, the application hangs on the InvokeWithRetry call which calls the method in my service in Service Fabric. The application does not throw any exceptions and does not return to the debugger in Visual Studio.
Make sure you run every service instance /replica with a unique url.
Make sure you call the WebHttpBinding constructor using WebHttpSecurityMode.Transport.
Make sure you register the url using the same port number (443 likely) as in you service manifest endpoint declaration.
Make sure the endpoint is configured as HTTPS.
The warning you see in Service Fabric is telling you that there is already another service registered to listen on port 443 on your nodes. This means that Service Fabric fails to spin up your service (since it throws an exception internally when it is trying to register the URL with http.sys). You can change the port for your service to something else that will not conflict with the existing service, e.g.:
<Resources>
<Endpoint Name="CalculatorEndpoint" Protocol="https" Type="Input" Port="44330" />
</Endpoints>
If you log in to Service Fabric Explorer on https://{cluster_name}.{region}.cloudapp.azure.com:19080 you should be able to see what other applications and services are running there. If you expand services all the way down to node you should be able to see the registered endpoints, including ports, for existing services.
Bonus
You can query the cluster using FabricClient for all registered endpoints
var fabricClient = new FabricClient();
var applicationList = fabricClient.QueryManager.GetApplicationListAsync().GetAwaiter().GetResult();
foreach (var application in applicationList)
{
var serviceList = fabricClient.QueryManager.GetServiceListAsync(application.ApplicationName).GetAwaiter().GetResult();
foreach (var service in serviceList)
{
var partitionListAsync = fabricClient.QueryManager.GetPartitionListAsync(service.ServiceName).GetAwaiter().GetResult();
foreach (var partition in partitionListAsync)
{
var replicas = fabricClient.QueryManager.GetReplicaListAsync(partition.PartitionInformation.Id).GetAwaiter().GetResult();
foreach (var replica in replicas)
{
if (!string.IsNullOrWhiteSpace(replica.ReplicaAddress))
{
var replicaAddress = JObject.Parse(replica.ReplicaAddress);
foreach (var endpoint in replicaAddress["Endpoints"])
{
var endpointAddress = endpoint.First().Value<string>();
Console.WriteLine($"{service.ServiceName} {endpointAddress} {endpointAddress}");
}
}}}}}
Just run that with the proper FabricClient credentials (if it is a secured cluster) and you should see it listing all endpoints for all services there. That should help you find the one that has an endpoint for :443

Limit connection in RESTful web service

Is there a way to configure a rest web service to allow only one connection at a time?
I am using Wildfly 9.0.1-Final with the resteasy 3.0.11.Final implementation.
You can use synchronized block on static field:
private static final Object LOCK = new Object();
#GET
#Path("find")
#Produces(MediaType.APPLICATION_JSON)
public Response find(){
synchronized(LOCK){
//your code
}
}

WCF Discovery and DataService V3

I would like to expose discovery endpoints (both TCP and UDP) for my Data Services v3 and enable services to be discoverable from the client and discover them in another application. The main point in the discovery is to get the service endpoint address at the client.
I have tried to adapt the samples that Microsoft have provided for WCF Discovery, but so far I failed to achieve my goal.
I have created a custom Data Service Host Factory on server side:
public class CustomDataServiceHostFactory : System.Data.Services.DataServiceHostFactory
{
protected override System.ServiceModel.ServiceHost CreateServiceHost(Type serviceType, Uri[] baseAddresses)
{
var serviceHost = base.CreateServiceHost(serviceType, baseAddresses);
EndpointDiscoveryBehavior endpointDiscoveryBehavior = new EndpointDiscoveryBehavior();
// Create XML metadata to add to the service endpoint
XElement endpointMetadata = new XElement(
"Root",
new XElement("Information", "This endpoint is Data Service v3!"),
new XElement("Time", System.DateTime.Now.ToString("MM/dd/yyyy HH:mm")));
// Add the XML metadata to the endpoint discovery behavior.
endpointDiscoveryBehavior.Extensions.Add(endpointMetadata);
//may be this is not the safest way to set the behaviour
foreach (var endpoint in serviceHost.Description.Endpoints)
{
endpoint.Behaviors.Add(endpointDiscoveryBehavior);
}
// Make the service discoverable over UDP multicast
serviceHost.Description.Behaviors.Add(new ServiceDiscoveryBehavior());
serviceHost.AddServiceEndpoint(new UdpDiscoveryEndpoint());
return serviceHost;
}
}
On the client side I have tried the following code:
DiscoveryClient discoveryClient = new DiscoveryClient(new UdpDiscoveryEndpoint());
// Find service endpoints
// ServiceReference.DataModel is the generated class for the Data Service client proxy
FindCriteria findCriteria = new FindCriteria(typeof(ServiceReference.DataModel));
findCriteria.Duration = TimeSpan.FromSeconds(30);
FindResponse findResponse = discoveryClient.Find(findCriteria);
// Check to see if endpoints were found & print the XML metadata in them.
if (findResponse.Endpoints.Count > 0)
{
foreach (XElement xElement in findResponse.Endpoints[0].Extensions)
{
Console.WriteLine("Printing Metadata from ServiceEndpoint:");
Console.WriteLine("Endpoint Information: " + xElement.Element("Information").Value);
Console.WriteLine("Endpoint Started at Time: " + xElement.Element("Time").Value);
Console.WriteLine();
}
}
Unfortunately this does not work. I get InvalidOperationException:
Attempted to get contract type for DataModel, but that type is
not a ServiceContract, nor does it inherit a ServiceContract.
If I am heading in the right direction I need a way to express the type for the service contract for the discovery. Too bad I am not sure that it will even work like the normal WCF Discovery...
Please share your ideas or even better - working solutions.
I think exception message is clear enough.
For service discovery You try to use type of your data model, while You must use type of your WCF service implementation - this is different things.
Basically DataServicesV3 service adapter uses your data model and exposes it as a WCF service with it's own service contract.
Look at DataServiceV3 type declaration see that it is implementing some interface, i just don't remember name, in this interface declaration you will find [ServiceContract] and [ServiceOperation] attributes. This is Your SERVICE CONTRACT for all ancestors of DataServiceV3. They use THE SAME contract. Here stands another problem I haven't managed to solve yet - how to make WS-Discovery work with DataServices if they share same contract. You'd better dig in this way.