Need help in applying ConnectionKeepAliveStrategy for ApacheConnector in jersey client.
For a standalone Apache Client, we can do it this way -
ConnectionKeepAliveStrategy myStrategy = new ConnectionKeepAliveStrategy() {
public long getKeepAliveDuration(HttpResponse response, HttpContext context) {
//your strategy here..
}
};
CloseableHttpClient client = HttpClients.custom()
.setKeepAliveStrategy(myStrategy)
.build();
But I couldn't find any way to apply strategy while using Jersey Client.
Tried registering the above feature class to WebClient but that did not help.
Any leads ?
Jersey 2.29.1 added support for Apache HTTP Client ConnectionKeepAliveStrategy and ConnectionReuseStrategy. Should work like this with ApacheConnector:
clientConfig.property(ApacheClientProperties.KEEPALIVE_STRATEGY, myStrategy);
Related
I am trying to create Spring WebClient for making HTTP calls to other services.
I am using ConnectionProvider for setting the connection-related configuration for example maxConnections, maxIdleTime etc.
I need to set the custom value of maxConnections for a particular host. Is there a way to do this in Spring WebClient?
There is a method called forRemoteHost but I am not really clear on how to use that ?
You need to customize ConnectionProvider while building WebClient
WebClient createWebClient(Map<SocketAddress, Integer> hosts) {
ConnectionProvider.Builder builder = ConnectionProvider.builder("http");
hosts.entrySet().forEach(entry ->
builder.forRemoteHost(entry.getKey(), spec -> spec.maxConnections(entry.getValue()))
);
HttpClient httpClient = HttpClient.create(builder.build());
ClientHttpConnector connector = new ReactorClientHttpConnector(httpClient);
return WebClient.builder()
.baseUrl(...)
.clientConnector(connector)
.build();
}
I have a barebone Service Fabric Application hosting a Asp.net Core 1.1 Web API with Azure Application Gateway as reverse proxy on a Virtual Machine scale set of 5 DS3_V2.
The API have 10 HttpClients with different URLs injected via Dependency Injection.
A simple foreach cycle in a method call 10 Httpclients in parallel:
var cts = new CancellationTokenSource();
cts.CancelAfter(600);
//Logic for asyncronously parallel calling the Call method below
public async Task<MyResponse> Call(CancellationTokenSource cts, HttpClient client, string endpoint )
{
var endpoint = "finalpartOfThendpoint";
var jsonRequest = "jsonrequest";
try
{
var content = new StringContent(jsonRequest, Encoding.UTF8, "application/json");
await content.LoadIntoBufferAsync();
if (cts.Token.IsCancellationRequested)
{
return new MyResponse("Token Canceled");
}
var response = await client.PostAsync(endpoint, content, cts.Token);
if (response.IsSuccessStatusCode && ((int)response.StatusCode != 204))
{
//do something with response and return
return MyResponse("Response Ok")
}
return MyResponse("No response")
}
catch (OperationCanceledException e)
{
return new MyResponse("Timeout");
}
}
There is a single CancellationToken for all calls.
After 600ms, the still pending HttpCalls are canceled and a response is sent back anyway.
In local and in production all works perfectly, all endpoints are called and return in time, rarely one is canceled before the timeout.
But when the number of concurrent connections reach 30+, ALL calls timeout no matter what, until I reduce the load.
Does Asp.net Core have a connection limit?
This is how I create the HttpClients in a custom factory for injection in the main Controller:
public static HttpClient CreateClient(string endpoint)
{
var client = new HttpClient
{
BaseAddress = new Uri(endpoint)
};
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
return client;
}
All the Httpclients are reused and static.
The same exact code works perfectly on a Asp.net Web API 2 hosted on OWIN in Service Fabric. The problem is only with Asp.net Core 1.1
I saw online to create a HttpClientHandler, but there is no parameter for concurrent connections.
What can I do to investigate further?
No exception are thrown but the OperationcanceledException and If I remove the CancellationToken the calls are stuck and the CPU goes to 100%, basically 30 connections destroy the power of 5 quad core servers.
This has something to do to the number of calls going out of Kestrel.
UPDATE
I tried with WebListener and the problem is still present, so it's not Kestrel, but Asp.net Core
I figured it out.
Asp.net core still have some HttpClient limits for the connection to the same server like the old Asp.net WebAPI.
It's poor documented but the old ServicepointManager option for maxconnections must now be passed via HttpClientHandler.
I just create HttpClient like this and the problem vanished.
var config = new HttpClientHandler()
{
MaxConnectionsPerServer = int.MaxValue
};
var client = new HttpClient(config)
{
BaseAddress = new Uri('url here')
};
Really if someone of the team is reading, this should be the default.
I am trying to implement OAuth2 in my JAX-RS application, using Apache Oltu. I have found this:
https://github.com/apache/oltu/tree/trunk/oauth-2.0/integration-tests/src/test/java/org/apache/oltu/oauth2/integration/endpoints
#POST
#Consumes("application/x-www-form-urlencoded")
#Produces("application/json")
public Response authorize(#Context HttpServletRequest request) throws OAuthSystemException
{
OAuthTokenRequest oauthRequest = null;
OAuthIssuer oauthIssuerImpl = new OAuthIssuerImpl(new MD5Generator());
try {
oauthRequest = new OAuthTokenRequest(request);
} catch (OAuthProblemException e) {
OAuthResponse res = OAuthASResponse.errorResponse(HttpServletResponse.SC_BAD_REQUEST).error(e)
.buildJSONMessage();
return Response.status(res.getResponseStatus()).entity(res.getBody()).build();
}
This works fine with application/x-www-form-urlencoded. However I want to have support for application/json too. I can't find any documentation how I extend or implement this. Is anyone familiar with this problem?
Preferably I want to reuse OAuthTokenRequest
The #Consumes and #Produces annotations from the JAX-RS api have support for an Array of Strings by default. You can declare your REST endpoint like this to support multiple formats: #Consumes({"application/x-www-form-urlencoded", "application/json"})
I am using apache cxf java client to connect my WS. I am also using NTLM for authentication.
Now problem I am facing due to credential caching. First time i tried user which does not have privileges to access WS method. when I changed the user , it is still using same user to access WS method.
I am running in tomcat, so cannot kill my JVM .. tried all possible combination on httpClientPolicy.
Any help will be appreciated.
This is NTLM specific problem. sun.net.www.protocol.https.HttpsURLConnectionImpl is getting serverAuthorization via java.net.Authenticator. requestPasswordAuthentication(). This authorization info is maintained in sun.net.www.protocol.http.AuthCacheValue.cache.
So if we override sun.net.www.protocol.http.AuthCacheValue means we can fix this issue.
AuthCacheValue.setAuthCache(new AuthCache()
{
#Override
public void remove(String arg0, AuthCacheValue arg1) { }
#Override
public void put(String arg0, AuthCacheValue arg1) { }
#Override
public AuthCacheValue get(String arg0, String arg1)
{
return null;
}
});
Reference :
http://web.archiveorange.com/archive/v/ACbGtycfTs2dqbRNpy6d
http://tigrou.nl/2011/06/11/cached-credentials-in-http-basic-authentication/
I googled and tried a lot of solutions to this problem.. apparently the simplest code that worked is as below using the JCIFS library
//Set the jcifs properties
jcifs.Config.setProperty("jcifs.smb.client.domain", "domainname");
jcifs.Config.setProperty("jcifs.netbios.wins", "xxx.xxx.xxx.xxx");
jcifs.Config.setProperty("jcifs.smb.client.soTimeout", "300000"); // 5 minutes
jcifs.Config.setProperty("jcifs.netbios.cachePolicy", "1200"); // 20 minutes
jcifs.Config.setProperty("jcifs.smb.client.username", "username");
jcifs.Config.setProperty("jcifs.smb.client.password", "password");
//Register the jcifs URL handler to enable NTLM
jcifs.Config.registerSmbURLHandler();
Apparently CXF 3.0 doesnt have a valid way of configuring the HTTP Client (4.3.x) with NTCredentials instance. Please refer to bug https://issues.apache.org/jira/browse/CXF-5671
By the way, if you have a simple message which needs to be transmitted, just use HTTP Client (I worked using 4.3.4.. not sure of the earlier versions) with NTCredentials Instance. That too did the magic for me.. The sample is as below:
final NTCredentials ntCredentials = new NTCredentials("username", "Passworrd","destination", "domain");
CredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(AuthScope.ANY, ntCredentials);
CloseableHttpClient httpclient = HttpClientBuilder.create()
.setDefaultCredentialsProvider(credsProvider)
.build();
We have an issue in our environment when using Websphere to attempt to connect to an external system with HttpClient 4.x (current version is 4.2.1). Connecting to the external system is fine with their certificate being installed in Websphere with no additional configuration of HttpClient. However, when they enabled mutual authentication, it no longer works and we get a SSLPeerUnverifiedException exception:
javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated,
at com.ibm.jsse2.SSLSessionImpl.getPeerCertificates(SSLSessionImpl.java:105),
at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:128),
at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:572),
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180),
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294),
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:640),
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:479),
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906),
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:1066),
I was provided the following code sample, and I was wondering if there's any way to configure HttpClient to use an explicit alias like this code sample does. I've tried to find good documentation on using SSL mutual authentication with HttpClient 4 and haven't been able to find much.
Here's the code sample:
private HttpURLConnection getConnection(String server, String machine,
String port) throws Exception {
URL u = new URL(server);
HttpsURLConnection connection = (HttpsURLConnection) u.openConnection();
String alias = "CellDefaultSSLSettings";
final HashMap connectionInfo = new HashMap();
connectionInfo.put(JSSEHelper.CONNECTION_INFO_DIRECTION,
JSSEHelper.DIRECTION_OUTBOUND);
connectionInfo.put(JSSEHelper.CONNECTION_INFO_REMOTE_HOST, machine);
connectionInfo.put(JSSEHelper.CONNECTION_INFO_REMOTE_PORT, port);
javax.net.ssl.SSLSocketFactory sslFact = JSSEHelper.getInstance()
.getSSLSocketFactory(alias, connectionInfo, null);
connection.setSSLSocketFactory(sslFact);
connection.setDoOutput(true);
connection.setDoInput(true);
connection.setRequestMethod("POST");
return connection;
}
Basically, how do I make HttpClient use "CellDefaultSSLSettings"?
Fundamentally this problem has nothing to do with HttpClient. HttpClient can be configured to establish HTTPS connections using any custom SSLContext or SSLSocketFactory instance. This is basically about how to use JSSE APIs to configure SSLContext in the right way. In your particular case JSSEHelper does all the hard work for you.
// JSSE socket factory
javax.net.ssl.SSLSocketFactory jssesf = JSSEHelper.getInstance().getSSLSocketFactory(alias, connectionInfo, null);
// HC socket factory
SSLSocketFactory hcsf = new SSLSocketFactory(jssesf, SSLSocketFactory.BROWSER_COMPATIBLE_HOSTNAME_VERIFIER);
This will give a connection socket factory that can be registered with the connection manager.
HttpClient 4.3 also comes with SSLContextBuilder class which can be used to assemble custom SSL configurations using fluid builder API.
https://svn.apache.org/repos/asf/httpcomponents/httpclient/trunk/httpclient/src/main/java/org/apache/http/conn/ssl/SSLContextBuilder.java
oleg's answer helped me out.
What I did was extend the DefaultHttpClient, and each constructor takes a String argument for the destination URL and calls a method setupScheme:
private void setupScheme(final String url) throws Exception {
Scheme scheme = new Scheme("https", 443, retrieveWebsphereSSLConnectionFactory(url));
getConnectionManager().getSchemeRegistry().register(scheme);
}
The method retrieveWebsphereSSLConnectionFactory essentially combines the code from the sample with the code oleg provided:
private SchemeSocketFactory retrieveWebsphereSSLConnectionFactory(final String url)
throws SSLException, URISyntaxException {
final String alias = "CellDefaultSSLSettings";
final HashMap<String, String> connectionInfo = new HashMap<String, String>();
connectionInfo.put(JSSEHelper.CONNECTION_INFO_DIRECTION, JSSEHelper.DIRECTION_OUTBOUND);
connectionInfo.put(JSSEHelper.CONNECTION_INFO_REMOTE_HOST,
URIUtils.extractHost(new URI(url)).getHostName());
connectionInfo.put(JSSEHelper.CONNECTION_INFO_REMOTE_PORT, "443");
return new SSLSocketFactory(JSSEHelper.getInstance().getSSLSocketFactory(alias, connectionInfo, null),
SSLSocketFactory.BROWSER_COMPATIBLE_HOSTNAME_VERIFIER);
}