I am trying to create Spring WebClient for making HTTP calls to other services.
I am using ConnectionProvider for setting the connection-related configuration for example maxConnections, maxIdleTime etc.
I need to set the custom value of maxConnections for a particular host. Is there a way to do this in Spring WebClient?
There is a method called forRemoteHost but I am not really clear on how to use that ?
You need to customize ConnectionProvider while building WebClient
WebClient createWebClient(Map<SocketAddress, Integer> hosts) {
ConnectionProvider.Builder builder = ConnectionProvider.builder("http");
hosts.entrySet().forEach(entry ->
builder.forRemoteHost(entry.getKey(), spec -> spec.maxConnections(entry.getValue()))
);
HttpClient httpClient = HttpClient.create(builder.build());
ClientHttpConnector connector = new ReactorClientHttpConnector(httpClient);
return WebClient.builder()
.baseUrl(...)
.clientConnector(connector)
.build();
}
Related
I am facing some trouble with the WebClient when using proxy, i.e. the code below does not work
WebClient webClient = WebClient.builder().baseUri("BASEURL").proxy(getProxy()).build();
Single<WebClientResponse> res = webClient.get().path("/MY/SUB/PATH").addHeader("Authorization", "Bearer " + MY_TOKEN).request();
WebClientResponse webClientRes = res.get();
String resContent = webClientRes.content().as(String.class).get();
public Proxy getProxy(){
return Proxy.builder().type(Proxy.ProxyType.HTTP).host(host).port(port).password("SECRET_PASSWORD".toCharArray()).username(username).build();
}
However the if we use Apache HttpClient the code works (working code below)
HttpHost proxy = new HttpHost(host, port);
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(new AuthScope(proxy), new UsernamePasswordCredentials(username, "SECRET_PASSWORD"));
HttpGet request = new HttpGet("BASEURL" + "/MY/SUB/PATH");
request.setHeader(HttpHeaders.AUTHORIZATION, "Bearer " + MY_TOKEN);
CloseableHttpClient httpClient = HttpClients.custom().setProxy(proxy).setDefaultCredentialsProvider(credentialsProvider).build();
String resContent = EntityUtils.toString(httpClient.execute(request).getEntity());
could anyone let us know if we are overlooking something basic?
We are using helidon MP 2.5.2
When you set a proxy in WebClient, it will use absolute URI in the request because of changes made in https://github.com/helidon-io/helidon/issues/2302 and https://github.com/helidon-io/helidon/issues/3438. The use of absolute URI was implemented because of section 5.1.2 Request-URI in https://www.w3.org/Protocols/rfc2616/rfc2616-sec5.html which states:
The absoluteURI form is REQUIRED when the request is being made to a
proxy.
The problem is that some hosts have issue processing a request with absolute URI as they expect relative URI instead. I have encountered while working on issue https://github.com/helidon-io/helidon/issues/4644 where my testcase has a client that is connecting to KeyCloak as an OIDC server, and KeyCloak will return a 404 because it cannot handle the absoluteURI.
There is a special Webclient config property called relative-uris that you can use to force the request URI to use the relative form rather than absolute. So you can try adding config() in your WebClient.builder() and set that property like this:
.config(Config.create(ConfigSources.create(Map.of("relative-uris", "true")))
where Config needs to be imported as io.helidon.config.Config and ConfigSources as io.helidon.config.ConfigSources. As an alternative, you can also add something like this in your application.yaml:
force-relative-uris:
relative-uris: true
and add config() in the WebClient.builder() like this:
.config(config.get("force-relative-uris"))
where config is instantiated prior to WebClient.builder() like this:
Config config = Config.create();
In the upcoming Helidon v2.5.5 (and v3.0.3), there will be a new relativeUris(boolean relativeUris) in WebClient.builder() so that you don’t have to use config() as in my examples above, which is slightly cumbersome.
I'm trying to attempt bulk loading Neptune using the loader endpoint, and Apache HTTP client by making the POST call like below.
My lambda is in the same VPC as Neptune and has the same sub-nets and Security group as well.
However i'm seeing :
`org.apache.http.NoHttpResponseException: cluster-endpoint-url:8182 failed to respond`
ObjectMapper objectMapper = new ObjectMapper();
String requestBodyJSON = objectMapper
.writeValueAsString(values);
CloseableHttpClient httpclient = HttpClients.createDefault();
HttpPost httpPost = new HttpPost("http://endpont:8182/loader");
log.info("Initiated HTTP Client ");
HttpEntity stringEntity = new StringEntity(requestBodyJSON, ContentType.APPLICATION_JSON);
httpPost.setEntity(stringEntity);
CloseableHttpResponse response2 = httpclient.execute(httpPost);
log.info("RESPONSE is "+ response2);
The likely issue is the use of http:// versus https://. Neptune does not allow for non-SSL/TLS connections. Also check your security group configuration. The security group for Neptune should be allowing inbound traffic from the subnets or the security group used by the Lambda function.
Need help in applying ConnectionKeepAliveStrategy for ApacheConnector in jersey client.
For a standalone Apache Client, we can do it this way -
ConnectionKeepAliveStrategy myStrategy = new ConnectionKeepAliveStrategy() {
public long getKeepAliveDuration(HttpResponse response, HttpContext context) {
//your strategy here..
}
};
CloseableHttpClient client = HttpClients.custom()
.setKeepAliveStrategy(myStrategy)
.build();
But I couldn't find any way to apply strategy while using Jersey Client.
Tried registering the above feature class to WebClient but that did not help.
Any leads ?
Jersey 2.29.1 added support for Apache HTTP Client ConnectionKeepAliveStrategy and ConnectionReuseStrategy. Should work like this with ApacheConnector:
clientConfig.property(ApacheClientProperties.KEEPALIVE_STRATEGY, myStrategy);
We have an issue in our environment when using Websphere to attempt to connect to an external system with HttpClient 4.x (current version is 4.2.1). Connecting to the external system is fine with their certificate being installed in Websphere with no additional configuration of HttpClient. However, when they enabled mutual authentication, it no longer works and we get a SSLPeerUnverifiedException exception:
javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated,
at com.ibm.jsse2.SSLSessionImpl.getPeerCertificates(SSLSessionImpl.java:105),
at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:128),
at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:572),
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180),
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294),
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:640),
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:479),
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906),
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:1066),
I was provided the following code sample, and I was wondering if there's any way to configure HttpClient to use an explicit alias like this code sample does. I've tried to find good documentation on using SSL mutual authentication with HttpClient 4 and haven't been able to find much.
Here's the code sample:
private HttpURLConnection getConnection(String server, String machine,
String port) throws Exception {
URL u = new URL(server);
HttpsURLConnection connection = (HttpsURLConnection) u.openConnection();
String alias = "CellDefaultSSLSettings";
final HashMap connectionInfo = new HashMap();
connectionInfo.put(JSSEHelper.CONNECTION_INFO_DIRECTION,
JSSEHelper.DIRECTION_OUTBOUND);
connectionInfo.put(JSSEHelper.CONNECTION_INFO_REMOTE_HOST, machine);
connectionInfo.put(JSSEHelper.CONNECTION_INFO_REMOTE_PORT, port);
javax.net.ssl.SSLSocketFactory sslFact = JSSEHelper.getInstance()
.getSSLSocketFactory(alias, connectionInfo, null);
connection.setSSLSocketFactory(sslFact);
connection.setDoOutput(true);
connection.setDoInput(true);
connection.setRequestMethod("POST");
return connection;
}
Basically, how do I make HttpClient use "CellDefaultSSLSettings"?
Fundamentally this problem has nothing to do with HttpClient. HttpClient can be configured to establish HTTPS connections using any custom SSLContext or SSLSocketFactory instance. This is basically about how to use JSSE APIs to configure SSLContext in the right way. In your particular case JSSEHelper does all the hard work for you.
// JSSE socket factory
javax.net.ssl.SSLSocketFactory jssesf = JSSEHelper.getInstance().getSSLSocketFactory(alias, connectionInfo, null);
// HC socket factory
SSLSocketFactory hcsf = new SSLSocketFactory(jssesf, SSLSocketFactory.BROWSER_COMPATIBLE_HOSTNAME_VERIFIER);
This will give a connection socket factory that can be registered with the connection manager.
HttpClient 4.3 also comes with SSLContextBuilder class which can be used to assemble custom SSL configurations using fluid builder API.
https://svn.apache.org/repos/asf/httpcomponents/httpclient/trunk/httpclient/src/main/java/org/apache/http/conn/ssl/SSLContextBuilder.java
oleg's answer helped me out.
What I did was extend the DefaultHttpClient, and each constructor takes a String argument for the destination URL and calls a method setupScheme:
private void setupScheme(final String url) throws Exception {
Scheme scheme = new Scheme("https", 443, retrieveWebsphereSSLConnectionFactory(url));
getConnectionManager().getSchemeRegistry().register(scheme);
}
The method retrieveWebsphereSSLConnectionFactory essentially combines the code from the sample with the code oleg provided:
private SchemeSocketFactory retrieveWebsphereSSLConnectionFactory(final String url)
throws SSLException, URISyntaxException {
final String alias = "CellDefaultSSLSettings";
final HashMap<String, String> connectionInfo = new HashMap<String, String>();
connectionInfo.put(JSSEHelper.CONNECTION_INFO_DIRECTION, JSSEHelper.DIRECTION_OUTBOUND);
connectionInfo.put(JSSEHelper.CONNECTION_INFO_REMOTE_HOST,
URIUtils.extractHost(new URI(url)).getHostName());
connectionInfo.put(JSSEHelper.CONNECTION_INFO_REMOTE_PORT, "443");
return new SSLSocketFactory(JSSEHelper.getInstance().getSSLSocketFactory(alias, connectionInfo, null),
SSLSocketFactory.BROWSER_COMPATIBLE_HOSTNAME_VERIFIER);
}
I use this simple way to attach username and password to the SOAP request header. This works fine inside Java boundaries, but I want to be able to call it with my WCF client. How do I do this?
I've tried the following code, but it does not include the credentials in the header:
wsClient.ClientCredentials.UserName.UserName = "Hello";
wsClient.ClientCredentials.UserName.Password = "World";
Thanks in advance!
That is quite awful non-standardized way. It uses custom HTTP Headers so you cannot expect that built in WCF mechanism will magically support such approach. How should WCF know that you want to add custom non-standard HTTP header to HTTP request (not SOAP header)?
Use this:
var proxy = new YourServiceClient();
using (var scope = new OperationContextScope(proxy.InnerChannel))
{
var prop = new HttpRequestMessageProperty();
prop.Headers.Add("UserName", "Hello");
prop.Headers.Add("Password", "World");
OperationContext context = OperationContext.Current;
context.OutgoingMessageProperties[HttpRequestMessageProperty.Name] = prop;
proxy.CallYourOperation();
}