I'm trying to attempt bulk loading Neptune using the loader endpoint, and Apache HTTP client by making the POST call like below.
My lambda is in the same VPC as Neptune and has the same sub-nets and Security group as well.
However i'm seeing :
`org.apache.http.NoHttpResponseException: cluster-endpoint-url:8182 failed to respond`
ObjectMapper objectMapper = new ObjectMapper();
String requestBodyJSON = objectMapper
.writeValueAsString(values);
CloseableHttpClient httpclient = HttpClients.createDefault();
HttpPost httpPost = new HttpPost("http://endpont:8182/loader");
log.info("Initiated HTTP Client ");
HttpEntity stringEntity = new StringEntity(requestBodyJSON, ContentType.APPLICATION_JSON);
httpPost.setEntity(stringEntity);
CloseableHttpResponse response2 = httpclient.execute(httpPost);
log.info("RESPONSE is "+ response2);
The likely issue is the use of http:// versus https://. Neptune does not allow for non-SSL/TLS connections. Also check your security group configuration. The security group for Neptune should be allowing inbound traffic from the subnets or the security group used by the Lambda function.
Related
I am facing some trouble with the WebClient when using proxy, i.e. the code below does not work
WebClient webClient = WebClient.builder().baseUri("BASEURL").proxy(getProxy()).build();
Single<WebClientResponse> res = webClient.get().path("/MY/SUB/PATH").addHeader("Authorization", "Bearer " + MY_TOKEN).request();
WebClientResponse webClientRes = res.get();
String resContent = webClientRes.content().as(String.class).get();
public Proxy getProxy(){
return Proxy.builder().type(Proxy.ProxyType.HTTP).host(host).port(port).password("SECRET_PASSWORD".toCharArray()).username(username).build();
}
However the if we use Apache HttpClient the code works (working code below)
HttpHost proxy = new HttpHost(host, port);
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(new AuthScope(proxy), new UsernamePasswordCredentials(username, "SECRET_PASSWORD"));
HttpGet request = new HttpGet("BASEURL" + "/MY/SUB/PATH");
request.setHeader(HttpHeaders.AUTHORIZATION, "Bearer " + MY_TOKEN);
CloseableHttpClient httpClient = HttpClients.custom().setProxy(proxy).setDefaultCredentialsProvider(credentialsProvider).build();
String resContent = EntityUtils.toString(httpClient.execute(request).getEntity());
could anyone let us know if we are overlooking something basic?
We are using helidon MP 2.5.2
When you set a proxy in WebClient, it will use absolute URI in the request because of changes made in https://github.com/helidon-io/helidon/issues/2302 and https://github.com/helidon-io/helidon/issues/3438. The use of absolute URI was implemented because of section 5.1.2 Request-URI in https://www.w3.org/Protocols/rfc2616/rfc2616-sec5.html which states:
The absoluteURI form is REQUIRED when the request is being made to a
proxy.
The problem is that some hosts have issue processing a request with absolute URI as they expect relative URI instead. I have encountered while working on issue https://github.com/helidon-io/helidon/issues/4644 where my testcase has a client that is connecting to KeyCloak as an OIDC server, and KeyCloak will return a 404 because it cannot handle the absoluteURI.
There is a special Webclient config property called relative-uris that you can use to force the request URI to use the relative form rather than absolute. So you can try adding config() in your WebClient.builder() and set that property like this:
.config(Config.create(ConfigSources.create(Map.of("relative-uris", "true")))
where Config needs to be imported as io.helidon.config.Config and ConfigSources as io.helidon.config.ConfigSources. As an alternative, you can also add something like this in your application.yaml:
force-relative-uris:
relative-uris: true
and add config() in the WebClient.builder() like this:
.config(config.get("force-relative-uris"))
where config is instantiated prior to WebClient.builder() like this:
Config config = Config.create();
In the upcoming Helidon v2.5.5 (and v3.0.3), there will be a new relativeUris(boolean relativeUris) in WebClient.builder() so that you don’t have to use config() as in my examples above, which is slightly cumbersome.
I am trying to create Spring WebClient for making HTTP calls to other services.
I am using ConnectionProvider for setting the connection-related configuration for example maxConnections, maxIdleTime etc.
I need to set the custom value of maxConnections for a particular host. Is there a way to do this in Spring WebClient?
There is a method called forRemoteHost but I am not really clear on how to use that ?
You need to customize ConnectionProvider while building WebClient
WebClient createWebClient(Map<SocketAddress, Integer> hosts) {
ConnectionProvider.Builder builder = ConnectionProvider.builder("http");
hosts.entrySet().forEach(entry ->
builder.forRemoteHost(entry.getKey(), spec -> spec.maxConnections(entry.getValue()))
);
HttpClient httpClient = HttpClient.create(builder.build());
ClientHttpConnector connector = new ReactorClientHttpConnector(httpClient);
return WebClient.builder()
.baseUrl(...)
.clientConnector(connector)
.build();
}
I am trying to get AWS Secrets Manager to work on an older Java 7 platform. Unfortunately we're locked on Java 7 for now.
The issue I have is that Java 7 had some security issues with SSL, and most modern Java platforms are using newer cipherSuites. Thus I get the error
javax.net.ssl.SSLHandshakeException: No negotiable cipher suite
In other interfaces I've been able to solve the issue by doing an .setEnabledCipherSuites on the SSL socket.
The problem here is that the Secrets Manager client does not expose the socket (AFAICT), nor does it expose the SocketFactory. I've been trying to create a new SSLContext wrapping the stock SSLContext that will provide a custom SocketFactory but creating and installing a custom SSLContext has proven to be quite complicated.
Before I end up pulling out the rest of my hair, is there an easier way to do this?
AWS Secrets Manager uses Apache HTTP Client (httpclient-4.5.7) under the covers. Is there a static way of hooking the Apache client with a custom Socket, SocketFactory, or SSLContext? One that does not require access to the HTTPClient object (which is not exposed either).
After much head banging I came up with the following code:
final String ciphers[] =
{ "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA", "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
"TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA", "TLS_RSA_WITH_AES_128_CBC_SHA", "TLS_RSA_WITH_AES_128_CBC_SHA256",
"TLS_RSA_WITH_AES_256_CBC_SHA256" };
final String[] protocols = new String[]
{ "TLSv1.2" };
// create and initialize an SSLContext for a custom socket factory
final SSLContext sslcontext = SSLContext.getInstance("SSL");
sslcontext.init(null, null, new SecureRandom());
// and here's our SocketFactory
final SSLConnectionSocketFactory secureSocketFactory = new SSLConnectionSocketFactory(sslcontext, protocols,
ciphers, new DefaultHostnameVerifier());
// Create a custom AWS Client Configuration with our socket factory
final ClientConfiguration cc = new ClientConfiguration();
final ApacheHttpClientConfig acc = cc.getApacheHttpClientConfig();
acc.setSslSocketFactory(secureSocketFactory);
// Create a Secrets Manager client with our custom AWS Client Configuration
final AWSSecretsManager client = AWSSecretsManagerClientBuilder //
.standard() //
.withRegion(region) //
.withClientConfiguration(cc) //
.build();
client is then used for the requests.
We have an issue in our environment when using Websphere to attempt to connect to an external system with HttpClient 4.x (current version is 4.2.1). Connecting to the external system is fine with their certificate being installed in Websphere with no additional configuration of HttpClient. However, when they enabled mutual authentication, it no longer works and we get a SSLPeerUnverifiedException exception:
javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated,
at com.ibm.jsse2.SSLSessionImpl.getPeerCertificates(SSLSessionImpl.java:105),
at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:128),
at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:572),
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180),
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294),
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:640),
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:479),
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906),
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:1066),
I was provided the following code sample, and I was wondering if there's any way to configure HttpClient to use an explicit alias like this code sample does. I've tried to find good documentation on using SSL mutual authentication with HttpClient 4 and haven't been able to find much.
Here's the code sample:
private HttpURLConnection getConnection(String server, String machine,
String port) throws Exception {
URL u = new URL(server);
HttpsURLConnection connection = (HttpsURLConnection) u.openConnection();
String alias = "CellDefaultSSLSettings";
final HashMap connectionInfo = new HashMap();
connectionInfo.put(JSSEHelper.CONNECTION_INFO_DIRECTION,
JSSEHelper.DIRECTION_OUTBOUND);
connectionInfo.put(JSSEHelper.CONNECTION_INFO_REMOTE_HOST, machine);
connectionInfo.put(JSSEHelper.CONNECTION_INFO_REMOTE_PORT, port);
javax.net.ssl.SSLSocketFactory sslFact = JSSEHelper.getInstance()
.getSSLSocketFactory(alias, connectionInfo, null);
connection.setSSLSocketFactory(sslFact);
connection.setDoOutput(true);
connection.setDoInput(true);
connection.setRequestMethod("POST");
return connection;
}
Basically, how do I make HttpClient use "CellDefaultSSLSettings"?
Fundamentally this problem has nothing to do with HttpClient. HttpClient can be configured to establish HTTPS connections using any custom SSLContext or SSLSocketFactory instance. This is basically about how to use JSSE APIs to configure SSLContext in the right way. In your particular case JSSEHelper does all the hard work for you.
// JSSE socket factory
javax.net.ssl.SSLSocketFactory jssesf = JSSEHelper.getInstance().getSSLSocketFactory(alias, connectionInfo, null);
// HC socket factory
SSLSocketFactory hcsf = new SSLSocketFactory(jssesf, SSLSocketFactory.BROWSER_COMPATIBLE_HOSTNAME_VERIFIER);
This will give a connection socket factory that can be registered with the connection manager.
HttpClient 4.3 also comes with SSLContextBuilder class which can be used to assemble custom SSL configurations using fluid builder API.
https://svn.apache.org/repos/asf/httpcomponents/httpclient/trunk/httpclient/src/main/java/org/apache/http/conn/ssl/SSLContextBuilder.java
oleg's answer helped me out.
What I did was extend the DefaultHttpClient, and each constructor takes a String argument for the destination URL and calls a method setupScheme:
private void setupScheme(final String url) throws Exception {
Scheme scheme = new Scheme("https", 443, retrieveWebsphereSSLConnectionFactory(url));
getConnectionManager().getSchemeRegistry().register(scheme);
}
The method retrieveWebsphereSSLConnectionFactory essentially combines the code from the sample with the code oleg provided:
private SchemeSocketFactory retrieveWebsphereSSLConnectionFactory(final String url)
throws SSLException, URISyntaxException {
final String alias = "CellDefaultSSLSettings";
final HashMap<String, String> connectionInfo = new HashMap<String, String>();
connectionInfo.put(JSSEHelper.CONNECTION_INFO_DIRECTION, JSSEHelper.DIRECTION_OUTBOUND);
connectionInfo.put(JSSEHelper.CONNECTION_INFO_REMOTE_HOST,
URIUtils.extractHost(new URI(url)).getHostName());
connectionInfo.put(JSSEHelper.CONNECTION_INFO_REMOTE_PORT, "443");
return new SSLSocketFactory(JSSEHelper.getInstance().getSSLSocketFactory(alias, connectionInfo, null),
SSLSocketFactory.BROWSER_COMPATIBLE_HOSTNAME_VERIFIER);
}
I am attempting to host a service that serves up basic web content (HTML, javascript, json) using a WebHttpBinding with minimal administrator involvement.
Thus far I have been successful, the only admin priviledges necessary are at install time (register the http reservation for the service account and to create the service itself). However, now I am running into issues with SSL. Ideally I would like to support a certificate outside the windows certificate store. I found this article - http://www.codeproject.com/KB/WCF/wcfcertificates.aspx - which seems to indicate you can specify the certificate on the service host, however at runtime navigating a browser to https://localhost/Dev/MyService results in a 404.
[ServiceContract]
public interface IWhoAmIService
{
[OperationContract]
[WebInvoke(
Method = "GET",
UriTemplate = "/")]
Stream WhoAmI();
}
public class WhoAmIService : IWhoAmIService
{
public Stream WhoAmI()
{
string html = "<html><head><title>Hello, world!</title></head><body><p>Hello from {0}</p></body></html>";
html = string.Format(html, WindowsIdentity.GetCurrent().Name);
WebOperationContext.Current.OutgoingResponse.ContentType = "text/html";
return new MemoryStream(Encoding.UTF8.GetBytes(html));
}
}
static void Main(string[] args)
{
ServiceHost host = new ServiceHost(typeof(WhoAmIService), new Uri("https://localhost:443/Dev/WhoAmI"));
host.Credentials.ServiceCertificate.Certificate = new X509Certificate2(#"D:\dev\Server.pfx", "private");
WebHttpBehavior behvior = new WebHttpBehavior();
behvior.DefaultBodyStyle = WebMessageBodyStyle.Bare;
behvior.DefaultOutgoingResponseFormat = WebMessageFormat.Json;
behvior.AutomaticFormatSelectionEnabled = false;
WebHttpBinding secureBinding = new WebHttpBinding();
secureBinding.Security.Mode = WebHttpSecurityMode.Transport;
secureBinding.Security.Transport.ClientCredentialType = HttpClientCredentialType.None;
ServiceEndpoint secureEndpoint = host.AddServiceEndpoint(typeof(IWhoAmIService), secureBinding, "");
secureEndpoint.Behaviors.Add(behvior);
host.Open();
Console.WriteLine("Press enter to exit...");
Console.ReadLine();
host.Close();
}
If I change my binding security to none and the base uri to start with http, it serves up okay. This post seems to indicate that an additional command needs to be executed to register a certificate with a port with netsh (http://social.msdn.microsoft.com/Forums/en-US/wcf/thread/6907d765-7d4c-48e8-9e29-3ac5b4b9c405/). When I try this, it fails with some obscure error (1312).
C:\Windows\system32>netsh http add sslcert ipport=0.0.0.0:443 certhash=0b740a29f
29f2cc795bf4f8730b83f303f26a6d5 appid={00112233-4455-6677-8899-AABBCCDDEEFF}
SSL Certificate add failed, Error: 1312
A specified logon session does not exist. It may already have been terminated.
How can I host this service using HTTPS without the Windows Certificate Store?
It is not possible. HTTPS is provided on OS level (http.sys kernel driver) - it is the same as providing HTTP reservation and OS level demands certificate in certificate store. You must use netsh to assign the certificate to selected port and allow accessing the private key.
The article uses certificates from files because it doesn't use HTTPS. It uses message security and message security is not possible (unless you develop your own non-interoperable) with REST services and webHttpBinding.
The only way to make this work with HTTPS is not using built-in HTTP processing dependent on http.sys = you will either have to implement whole HTTP yourselves and prepare new HTTP channel for WCF or you will have to find such implementation.