Send HTTP GET/POST via a reverse proxy that requires client certificates - reverse-proxy

I am trying to send http GET/POST requests to applications that are hidden behind a reverse proxy. Communication with the reverse proxy is via https and the proxy requires a client certificate.
It looks like that the keystore certificate (gatling.http.ssl.keyStore.file) is not used to authenticate with the reverse proxy. I assume this because:
if I request https://reverse-proxy-address without specifying a proxy, I receive an answer (basically the same as if I access the URL within a browser with the client certificate) -> certificate is used for the request.
if I specify a proxy with http.proxy(Proxy("reverse-proxy-address", port)) and sent a request to http://hidden-url I receive a "org.asynchttpclient.exception.RemotelyClosedException: Remotely closed" (Gatling 2.3.1) or "java.io.IOException: Premature close" (Gatling 3.0.3)
I haven't found a hint how I can specify that the client certificate is used for authentication with the reverse proxy. Maybe the client certificate is already used to authenticate with the reverse proxy and something else is not configured correctly. I don't know how to analyze further...
Hope that someone else already faced the same issue and know the solution. Also hints so that I can dig deeper are more than welcome!
Thanks

I was doing that with Gatling 2.x on OSX. It requires a few steps more. Setting cert path in gatling.conf is not enough.
I received CRT.pem and KEY.pem files. I created p12 cert based on the key pair.
openssl pkcs12 -export -in client1-crt.pem -inkey client1-key.pem -out cert.p12
Then I created store and imported the cert to keystore.
keytool -importkeystore -deststorepass mycert -destkeystore filename-new-keystore.jks -srckeystore cert.p12 -srcstoretype PKCS12
The next step is to set correct path in gatling.conf (it depends on OS)
my gatling conf:
gatling {
http {
ssl {
keyStore {
type = "PKCS12"
file = "/Users/lukasz/cert.p12"
password = ""
}
trustStore {
type = ""
file = "/Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/lib/security/cacerts"
password = "changeit"
}
}
}
}
This way I was able to use custom certificate with Gatling. I'm not sure if this is a workaround or this is a proper way to handle custom certificate by JVM.

Related

Serving content over HTTPS

Can GraphDB Free be configured in any way to allow content to be served over HTTPS?
I'm creating a simple front-end web application using Angular2 that makes HTTP requests to the GraphDB SPARQL endpoint.
I'm using Windows IIS and the server has been setup not to trust any content this is not served over HTTPS.
Any insights would be helpful.
According the instructions in GraphDB's configuration file $GDB_HOME/conf/graphdb.properties, to enable HTTPS uncomment the following lines:
# Enable SSL (uncomment to enable)
graphdb.connector.SSLEnabled = true
graphdb.connector.scheme = https
graphdb.connector.secure = true
# GraphDB uses the Java implementation of SSL, which requires a configured key in the Java keystore.
# To setup keystore uncomment the following properties and set keystorePass and keyPass to the actual values.
graphdb.connector.keyFile = <path to the keystore file if different from ${graphdb.home}/keystore>
graphdb.connector.keystorePass = <secret>
graphdb.connector.keyAlias = graphdb
graphdb.connector.keyPass = <secret>
If you use a self-signed certificate add it to the keystore with:
keytool -genkey -alias graphdb -keyalg RSA
If you have a third-party trusted OpenSSL certificate, you would need to convert it to PKCS12 key and then import to the Java keystore with:
keytool -importkeystore -deststorepass MYPASS -srckeystore mypkcs12.p12 -srcstoretype PKCS12
For additional information on how to deal with the third party trusted certificates check this excellent link explaining in great details how to convert import private key and certificate into java keystore
Edited
To locate the $GDB_HOME directory for the desktop installation, check the GraphDB documentation.

Securing a private IP address (https certificate)

I have an unusual use case :
a web server on the Internet is serving pages through HTTPS,
inside those web pages, there are calls to XMLHttpRequests to a locally connected device (IP over USB)
the device supports both HTTP and HTTPS,
the device is accessible on http(s)://192.168.0.1
the http calls fail because of insecure content in a https page,
the https calls fail because the certificate is not trusted (self-signed),
Side question: Since the device is locally connected to the PC, the encryption is pretty useless: Does a http header exists that allows insecure connections to a specific URL ? (like CORS for cross domain)
Main question: Is it possible to obtain a certificate for a private IP address ?
Edit: it seems that Plex had a similar problem and solved it the way described on this blog. This is a way too big for me.
Is it possible to obtain a certificate for a private IP address ?
A certificate can be bound to an IP address (see this). You can issue a self-signed certificate to a private address, but a trusted CA will not issue a certificate to a private address because it can not verify its identity.
For example a certificate issued to 192.168.0.1 would be theoretically valid in any context, and this should not be allowed by a trusted CA
Plex solves the problem with a Dynamic DNS and a wildcard certificate. The connection are done using the name (not the IP) of the device which is resolved to the private IP
Does a http header exists that allows insecure connections to a specific URL ? (like CORS for cross domain)
No, it does not exist. The browser blocks your XHR connections because they are HTTP connections initiated from a HTTPS page (mixed-content warning). Non-secure content can theoretically be read or modified by attackers, even though the parent page is served over HTTPs, so is normal and recommended that the browser warns the user.
To fix the mixed-content and https errors, you could serve the content through HTTPS and a self-signed certificate, and request users to import your root CA at browser.
An SSL certificate cannot be issued for Reserved IP addresses (RFC 1918 and RFC 4193 range)/ private IP addresses (IPv4, IPv6), Intranet for Internal Server Name, local server name with a non-public domain name suffix.
You could however use a 'self-signed' certificate. Here's how to create one:
Creating a Self-signed Certificate for a private IP
(example https://192.168.0.1) :
You need OpenSSL installed.
For example, on Ubuntu, you could install it by: sudo apt-get install openssl
(It may already be installed. Type "openssl version" to find out)
For Windows, you could try this: https://slproweb.com/products/Win32OpenSSL.html
Once OpenSSL is installed, go to OpenSSL prompt by entering 'openssl' on the console (LINUX), or the cmd prompt (WINDOWS).
$ openssl
OpenSSL>
Now do the following steps to create: Private key, Certificate Request, Self-signing the certificate, and putting it all together, by using the below commands:
i) Create KEY called mydomain.key:
OpenSSL> genrsa -out mydomain.key 2048
ii) Use the key to create a Certificate request called mydomain.csr
You could accept the default options, or specify your own information:
OpenSSL> req -new -key mydomain.key -out mydomain.csr
iii) use the above to create a certificate:
OpenSSL> x509 -req -days 1825 -in mydomain.csr -signkey mydomain.key -out mydomain.crt
iv) Put all the above to create a PEM certificate:
exit OpenSSL (OpenSSL> q) and go to certificate location and do:
$ sudo cat mydomain.key mydomain.crt >> mylabs.com.pem
mylabs.com.pem is your self-signed certificate. You can use this in requests like https://192.168.0.1 if your server supports https. Remember to check the port number for https(443).

How to configure SoapUI mock service with non self signed certificate X509 for SSL

I have no problem to set up SoapUI mock service with self signed certificate by following the steps at
http://www.soapui.org/Service-Mocking/securing-mockservices-with-ssl.html .
But now I want to test the mock service with a "real" certificate issue by Entrust Certification Authority - L1C.
Other information for this certificate is
Ensures the identity of a remote computer
1.2.840.113533.7.75.2
2.23.140.1.2.2
It is valid until 2016.
I used the command to import the certificate to server.keystore .
c:\Program Files\Java\jre6\bin\keytool.exe -import -alias server -keystore server.keystore -storepass mypasss -file server_cer_issued_by_entrust.cer
(if I replace server_cer_issued_by_entrust.cer with a self signed (created) certificate it works)
I got the error in SoapUI which says ssh handshake error.
I just realize that it maybe not possible since the mock service is 127.0.0.1 but the actual certificate is issue for a web service like "mydomain.com", is that right?
SOAPUI runs over java, and when java connects to SSL url performs two validations:
Validate that the server certificate is trusted.
Validate that the common name in subject distinguished name of the server certificate matches the domain.
I assume that the first validation pass since in the link instructions which you provide its specify the same keystore in mock service for both: keystore and truststore, and since server certificate is imported in this one there is no problem to pass this validation.
So probably as you said the problem is that your mock service is on 127.0.0.1 and your certificate is issued to be used in mydomain.com.
A possible workaround is to disable SSL validation however this has no sense since as you said in the question you want to test the service with "real" certificate, instead maybe you can try editing host file and adding the mapping for 127.0.0.1 ip address for your certificate server domain, in your case:
127.0.0.1 mydomain.com
And then change the SOAPUI endpoint for your mock service to https://mydomain.com.
Hope this helps,

SSL client verification failure

I have a backend server with SSL only as defaut public interface. This server listens on 443 with cert, key and ca files on, ssl verify client is set to true.
On client side, I have client cert, key and a ca file made of 2 CA files catted together (intermediate CA, root CA) in a single ca.pem file. When I do curl request on my backend server, it works fine. I also tested the server / client cert and key with gnutls-serv and openssl s_server to ensure all was valid.
But when I create a keystore on Apigee (client cert and key file [pem format]) and a trustore (ca.pem file), there is an error:
- target.name fsbca-test
- Properties
- Expression ("fsbca-test" equals target.name)
- ExpressionResult true
- Tree TARGET_fsbca-test
- error The Service is temporarily unavailable
- error.cause General SSLEngine problem
- error.cause.cause General SSLEngine problem
- error.class com.apigee.messaging.adaptors.http.HttpAdaptorException
- state TARGET_REQ_FLOW
- type ErrorPoint
If I put SSL verify client to false on my backend, then the request is correctly diverted by Apigee and I get the response.
If I put IgnoreValidationErrors to true in the target endpoints property, then the request is diverted to my backend server but I can see an error in the server's log: "client sent no required SSL certificate while reading c...".
Any ideas about what could be wrong in what I am doing?
Additional track: could it be an issue on Apigee side with the CA file made of 2 certs (it may ignore trailing certificates found in a .pem). If wanted to test pkcs12 and jks but I failed to upload them to Apigee (the API doc page only describes .pem, JAR and cert action). I wrote a small Java client with pkcs12 keystore and jks trustore and it worked fine from my local workstation.
Thank you in advance for any piece of information that could help me.
Regards
Fr
You should upload your certificates separately (one per cert), and you need the entire trust chain of certificates to be stored in your truststore.
Here is the page about SSL to your backend.
EDIT:
Here is a method I know works:
1) Separate certs into separate PEM files. Do not put more than once cert in a file.
2) Validate each cert using openssl:
openssl x509 -noout -text -in <cert file name>
Validate that no certs are expired, and that the Subject and Issuer fields create a chain of all the certs, with identical names.
The server's certificate, the root certificate, and all certificates in between need to be in the truststore.

Apache HTTP Client javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated

We are developing an application using tomcat and jersey.
Within this webapplication we need to connect to a https Website with a valid, not expired certificate.
If I do connect to this website locally via my chrome browser, everything works fine!
Unfortunately the tomcat server with our webapp throws an exception. We are using the Apache HttpClient (4.0) to connect to the https site:
javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
at sun.security.ssl.SSLSessionImpl.getPeerCertificates(SSLSessionImpl.java:371)
at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:126)
at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:572)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
The server certificate is absolutely valid and from thawte.
Three different online tools validated the certificate successfully.
Openssl has an issue, too and showing me three certificates but throwing a simple error:
Verify return code: 20 (unable to get local issuer certificate)
The problem with openssl seems to be that it uses the wrong path /usr/lib/sslinstead of /etc/ssl/certs. If I use the CApath argument pointing to the proper path, openssl works fine so may this be an issue with the httpClient?
So our code for the default client is quite simple:
client = new DefaultHttpClient();
response = client.execute(url); //this throws the exception
EntityUtils.consume(response.getEntity());
It's not an option to allow any certificates by implementing a custom TrustedManager!
Futher I read, that some CA's are not part of the JDK/JRE and so it's certificates should be imported manually into the keystore or use a custom one, but thawte is a well known CA and shouldn't it work on default?
EDIT
I did set the javax.debug properties in catalina.sh so that I have further information about the problem:
http-bio-8080-exec-1, handling exception: javax.net.ssl.SSLHandshakeException:
sun.security.validator.ValidatorException: PKIX path validation failed:
java.security.cert.CertPathValidatorException: basic constraints check failed:
pathLenConstraint violated - this cert must be the last cert in the certification path
I would appreciate any help!
Thanks in advance!
Okay, I got it working!
Although thawte is a well known CA it seems that Java SSL did have some problems with it.
After downloading the ssl Certificate via openssl:
echo |\
openssl s_client -connect ${REMHOST}:${REMPORT} 2>&1 |\
sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p'
and saving it into an pem file, I did the manual import into the java keystore:
keytool -import -alias myAlias -file theCert.pem -keystore lib/security/cacerts
I have no idea why java ssl was not able to validate the thawte certificate properly.
Listing the keystore showed me, that there are 7 thawte trusted certificates in the standard keystore but bizarrely it did not work until I manually imported the pem file
I am trying to understand your setup. You have a SSL certificate (issued by Thwate), installed in tomcat and you can access your site just fine over SSL using say IE or Firefox or Chrome.
But when you try to access it using HttpClient, you receive the above error ?
Is that correct ?
The error clearly indicates that your client does not trust the CA. But if the cert is signed by Thwate (and is installed correctly and is acessible via IE/Firefox etc), then it should work fine.