Bigquery SSL error while doing streaming insert api call [duplicate] - google-bigquery

Hi I am working on android app in which I have integrated bigquery. I see sometimes we are getting a lot of SSL exceptions while inserting data to big query tables. I don't know how to handle this . Please help what exactly is the cause of this problem. Here is the same thread but no answer Bigquery SSL error while doing streaming insert api call
javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:946) ~[na:1.7.0_51]
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312) ~[na:1.7.0_51]
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339) ~[na:1.7.0_51]
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323) ~[na:1.7.0_51]
at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:563) ~[na:1.7.0_51]
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185) ~[na:1.7.0_51]
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1091) ~[na:1.7.0_51]
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:250) ~[na:1.7.0_51]
at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:77) ~[google-http-client-1.19.0.jar:1.19.0]
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:965) ~[google-http-client-1.19.0.jar:1.19.0]
at com.google.api.client.googleapis.batch.BatchRequest.execute(BatchRequest.java:241) ~[google-api-client-1.19.1.jar:1.19.1]
at com.livestream.analytics.datastorage.worker.InsertApiActor$$anonfun$2$$anonfun$4.apply(InsertApiActor.scala:131) ~[analytics-data-storage-worker_2.11-1.0.0.jar:1.0.0]
at com.livestream.analytics.datastorage.worker.InsertApiActor$$anonfun$2$$anonfun$4.apply(InsertApiActor.scala:118) ~[analytics-data-storage-worker_2.11-1.0.0.jar:1.0.0]
at com.livestream.analytics.common.store.bigquery.api.BigQueryApi$.withSyncClient(BigQueryApi.scala:71) ~[analytics-common_2.11-1.0.0.jar:1.0.0]
at com.livestream.analytics.datastorage.worker.InsertApiActor$$anonfun$2.apply$mcV$sp(InsertApiActor.scala:118) ~[analytics-data-storage-worker_2.11-1.0.0.jar:1.0.0]
at com.livestream.analytics.datastorage.worker.InsertApiActor$$anonfun$2.apply(InsertApiActor.scala:115) ~[analytics-data-storage-worker_2.11-1.0.0.jar:1.0.0]
at com.livestream.analytics.datastorage.worker.InsertApiActor$$anonfun$2.apply(InsertApiActor.scala:115) ~[analytics-data-storage-worker_2.11-1.0.0.jar:1.0.0]
at com.livestream.analytics.common.monitoring.Timer.time(Timer.scala:15) ~[analytics-common_2.11-1.0.0.jar:1.0.0]
at com.livestream.analytics.datastorage.worker.InsertApiActor.com$livestream$analytics$datastorage$worker$InsertApiActor$$insertDataRowsToBigQueryTable(InsertApiActor.scala:115) [analytics-data-storage-worker_2.11-1.0.0.jar:1.0.0]
at com.livestream.analytics.datastorage.worker.InsertApiActor$$anonfun$receive$1.applyOrElse(InsertApiActor.scala:80) [analytics-data-storage-worker_2.11-1.0.0.jar:1.0.0]
at akka.actor.Actor$class.aroundReceive(Actor.scala:465) [akka-actor_2.11-2.3.9.jar:na]
at com.livestream.analytics.datastorage.worker.InsertApiActor.aroundReceive(InsertApiActor.scala:54) [analytics-data-storage-worker_2.11-1.0.0.jar:1.0.0]
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516) [akka-actor_2.11-2.3.9.jar:na]
at akka.actor.ActorCell.invoke(ActorCell.scala:487) [akka-actor_2.11-2.3.9.jar:na]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:254) [akka-actor_2.11-2.3.9.jar:na]
at akka.dispatch.Mailbox.run(Mailbox.scala:221) [akka-actor_2.11-2.3.9.jar:na]
at akka.dispatch.Mailbox.exec(Mailbox.scala:231) [akka-actor_2.11-2.3.9.jar:na]
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [scala-library-2.11.5.jar:na]
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [scala-library-2.11.5.jar:na]
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [scala-library-2.11.5.jar:na]
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [scala-library-2.11.5.jar:na]
Caused by: java.io.EOFException: SSL peer shut down incorrectly
at sun.security.ssl.InputRecord.read(InputRecord.java:482) ~[na:1.7.0_51]
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927) ~[na:1.7.0_51]
... 30 common frames omitted

Probably the server is asking for a client certificate and you aren't providing one. The server will provide a list of trusted signers, and your client certificate needs to be signed by one of those. You can't use a self-signed certificate for the client unless you've made special arrangements with the server, i.e. imported your client certificate into its trusted certificate list. Your SSL client won't send a certificate if it can't find one, or if the one(s) that it finds don't have trusted signers.
It doesn't have anything to do with what the SSL connection was going to do after it was established, e.g. SQL queries, updates, etc.

If you are using (https) requests you need to add Security Certificate in your app. here is link how to add it.
http://stackoverflow.com/questions/4065379/how-to-create-a-bks-bouncycastle-format-java-keystore-that-contains-a-client-c
You can test if indeed security certificate is causing it, Add this class in your project.
import java.security.KeyManagementException;
import java.security.NoSuchAlgorithmException;
import java.security.SecureRandom;
import java.security.cert.X509Certificate;
import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.HttpsURLConnection;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSession;
import javax.net.ssl.TrustManager;
import javax.net.ssl.X509TrustManager;
public class HttpsTrustManager implements X509TrustManager {
private static TrustManager[] trustManagers;
private static final X509Certificate[] _AcceptedIssuers = new X509Certificate[]{};
#Override
public void checkClientTrusted(
java.security.cert.X509Certificate[] x509Certificates, String s)
throws java.security.cert.CertificateException {
}
#Override
public void checkServerTrusted(
java.security.cert.X509Certificate[] x509Certificates, String s)
throws java.security.cert.CertificateException {
}
public boolean isClientTrusted(X509Certificate[] chain) {
return true;
}
public boolean isServerTrusted(X509Certificate[] chain) {
return true;
}
#Override
public X509Certificate[] getAcceptedIssuers() {
return _AcceptedIssuers;
}
public static void allowAllSSL() {
HttpsURLConnection.setDefaultHostnameVerifier(new HostnameVerifier() {
#Override
public boolean verify(String arg0, SSLSession arg1) {
return true;
}
});
SSLContext context = null;
if (trustManagers == null) {
trustManagers = new TrustManager[]{new HttpsTrustManager()};
}
try {
context = SSLContext.getInstance("TLS");
context.init(null, trustManagers, new SecureRandom());
} catch (NoSuchAlgorithmException e) {
e.printStackTrace();
} catch (KeyManagementException e) {
e.printStackTrace();
}
HttpsURLConnection.setDefaultSSLSocketFactory(context
.getSocketFactory());
}
}
and call
HttpsTrustManager.allowAllSSL(); // Allow all SSL connections
before an API call.
NOTE:
This code skips verification and allows any certificate to work.
This method should not be used for secure communication.
This is just to check if Certificate authentication is causing the error.

Related

Enable SSL certificate revocation checking in OpenJDK 11

Is there some quick "declarative" way in Java 11, instead of a tedious manual implementation, to enable checking if a certificate is revoked?
I tried to use properties from this answer:
Check X509 certificate revocation status in Spring-Security before authenticating
with this dummy revoked certificate: https://revoked.badssl.com
but the code always accepts the certificate. Am I doing something wrong or these properties are no more actual for Java 11? If so, do we have any alternatives?
Below is my code:
public static void validateOnCertificateRevocation(boolean check) {
if (check) {
System.setProperty("com.sun.net.ssl.checkRevocation", "true");
System.setProperty("com.sun.security.enableCRLDP", "true");
Security.setProperty("ocsp.enable", "true");
}
try {
new URL("https://revoked.badssl.com").openConnection().connect();
} catch (IOException e) {
e.printStackTrace();
}
}
It seems like those options have to be set before the first request has been performed.
Therefore the following code as standalone Java program throws an CertPathValidatorException: Certificate has been revoked (tested using OpenJDK 11.0.2 x64 on Windows):
public static void main(String[] args) {
validateOnCertificateRevocation(true); // throws CertPathValidatorException
}
However the following code does not cause any errors/Exceptions:
public static void main(String[] args) {
validateOnCertificateRevocation(false);
validateOnCertificateRevocation(true); // nothing happens
}
You can see the changing the options after the first request has been processed isn't effective. I assume that those options are processed in a static { ... } block of some certificate validation related class.
If you still want to enable/disable certificate revocation checking on a per-request base you can do so by implementing your own X509TrustManager that uses CertPathValidator (for which you can enable/disable certificate revocation checking via PKIXParameters.setRevocationEnabled(boolean).
Alternatively there is the solution to globally enable certificate revocation checking and explicitly handle the CertificateRevokedException:
private boolean checkOnCertificateRevocation;
#Override
public void checkServerTrusted(X509Certificate[] certs, String authType) throws CertificateException {
try {
getDefaultTrustManager().checkServerTrusted(certs, authType);
} catch (CertificateException e) {
if (checkOnCertificateRevocation) {
if (getRootCause(e) instanceof CertificateRevokedException) {
throw e;
}
}
}
}

Fabric Crashlytics Error javax.net.ssl.SSLHandshakeException: java.security.cert.CertPathValidatorException

This Error is related to Fabric Crashlytics, not direct RestFul API consumptions.
I'm getting this error when initiating the Fabric Crashlytics on Android Emulator
E/Fabric: Settings request failed.
io.fabric.sdk.android.services.network.HttpRequest$HttpRequestException: javax.net.ssl.SSLHandshakeException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.
at io.fabric.sdk.android.services.network.HttpRequest.code(HttpRequest.java:1355)
at io.fabric.sdk.android.services.settings.DefaultSettingsSpiCall.handleResponse(DefaultSettingsSpiCall.java:104)
at io.fabric.sdk.android.services.settings.DefaultSettingsSpiCall.invoke(DefaultSettingsSpiCall.java:88)
at io.fabric.sdk.android.services.settings.DefaultSettingsController.loadSettingsData(DefaultSettingsController.java:80)
at io.fabric.sdk.android.services.settings.DefaultSettingsController.loadSettingsData(DefaultSettingsController.java:64)
at io.fabric.sdk.android.services.settings.Settings.loadSettingsData(Settings.java:153)
at io.fabric.sdk.android.Onboarding.retrieveSettingsData(Onboarding.java:126)
at io.fabric.sdk.android.Onboarding.doInBackground(Onboarding.java:99)
at io.fabric.sdk.android.Onboarding.doInBackground(Onboarding.java:45)
at io.fabric.sdk.android.InitializationTask.doInBackground(InitializationTask.java:63)
at io.fabric.sdk.android.InitializationTask.doInBackground(InitializationTask.java:28)
at io.fabric.sdk.android.services.concurrency.AsyncTask$2.call(AsyncTask.java:311)
at java.util.concurrent.FutureTask.run(FutureTask.java:237)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:428)
at java.util.concurrent.FutureTask.run(FutureTask.java:237)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607)
at java.lang.Thread.run(Thread.java:761)
Caused by: java.security.cert.CertificateException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.
at com.android.org.conscrypt.TrustManagerImpl.verifyChain(TrustManagerImpl.java:563)
at com.android.org.conscrypt.TrustManagerImpl.checkTrustedRecursive(TrustManagerImpl.java:444)
at com.android.org.conscrypt.TrustManagerImpl.checkTrustedRecursive(TrustManagerImpl.java:508)
at com.android.org.conscrypt.TrustManagerImpl.checkTrusted(TrustManagerImpl.java:401)
at com.android.org.conscrypt.TrustManagerImpl.checkTrusted(TrustManagerImpl.java:375)
at com.android.org.conscrypt.TrustManagerImpl.getTrustedChainForServer(TrustManagerImpl.java:304)
at android.security.net.config.NetworkSecurityTrustManager.checkServerTrusted(NetworkSecurityTrustManager.java:94)
E/Fabric: at android.security.net.config.RootTrustManager.checkServerTrusted(RootTrustManager.java:88)
at com.android.org.conscrypt.Platform.checkServerTrusted(Platform.java:178)
at com.android.org.conscrypt.OpenSSLSocketImpl.verifyCertificateChain(OpenSSLSocketImpl.java:596)
at com.android.org.conscrypt.NativeCrypto.SSL_do_handshake(Native Method)
at com.android.org.conscrypt.OpenSSLSocketImpl.startHandshake(OpenSSLSocketImpl.java:357)
... 30 more
Caused by: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.
... 42 more
E/Answers: Failed to retrieve settings
Any Idea how to resolve this, or add custom SSlSocketFactory to Fabric HttpRequest class?
For anyone else experiencing this issue: This error occurred for me when the wifi password was outdated and my device switched to a different (open) network. Try switching to another (closed) network.
Ran into the same issue, the same Crashlytics setup in Fabric and Firebase, tried with different versions of Crashlytics up to 2.9.2, it worked fine on API 22+, but not on API 21 and below. In the end, it is resolved by a workaround, not ideal and not recommended for production, which is bypassing the ssl check all together by calling the function below in the onCreate() in app's Application class.
public void trustAllCertificates() {
try {
TrustManager[] trustAllCerts = new TrustManager[]{
new X509TrustManager() {
public X509Certificate[] getAcceptedIssuers() {
X509Certificate[] myTrustedAnchors = new X509Certificate[0];
return myTrustedAnchors;
}
#Override
public void checkClientTrusted(X509Certificate[] certs, String authType) {
}
#Override
public void checkServerTrusted(X509Certificate[] certs, String authType) {
}
}
};
SSLContext sc = SSLContext.getInstance("SSL");
sc.init(null, trustAllCerts, new SecureRandom());
HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
HttpsURLConnection.setDefaultHostnameVerifier(new HostnameVerifier() {
#Override
public boolean verify(String arg0, SSLSession arg1) {
return true;
}
});
} catch (Exception e) {
}
}
References: https://androidlad.blogspot.com/2017/08/how-to-trust-all-certificates-or-bypass.html

How to ignore some path with Spring Security and SSL?

This is my Spring configuration:
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.security.config.annotation.method.configuration.EnableGlobalMethodSecurity;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.builders.WebSecurity;
import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity;
import org.springframework.security.config.annotation.web.configuration.WebSecurityConfigurerAdapter;
import org.springframework.security.config.http.SessionCreationPolicy;
import org.springframework.security.core.authority.AuthorityUtils;
import org.springframework.security.core.userdetails.User;
import org.springframework.security.core.userdetails.UserDetails;
import org.springframework.security.core.userdetails.UserDetailsService;
import org.springframework.security.core.userdetails.UsernameNotFoundException;
#Configuration
#EnableWebSecurity
#EnableGlobalMethodSecurity(prePostEnabled = true)
public class X509Configuration extends WebSecurityConfigurerAdapter {
#Override
public void configure(WebSecurity web) throws Exception {
web.ignoring().antMatchers("/ws/items.wsdl");
}
#Override
public void configure(final HttpSecurity http) throws Exception {
http
.sessionManagement()
.sessionCreationPolicy(SessionCreationPolicy.NEVER)
.and()
.authorizeRequests()
.antMatchers("/ws/items.wsdl").permitAll()
.and()
// require authorization for everything else
.authorizeRequests()
.anyRequest().authenticated()
.and()
.x509()
.subjectPrincipalRegex("CN=(.*?)(?:,|$)")
.userDetailsService(userDetailsService())
.and()
.csrf().disable()
// configure form login
.formLogin().disable()
// configure logout
.logout().disable();
}
#Bean
public UserDetailsService userDetailsService() {
return new UserDetailsService() {
#Override
public UserDetails loadUserByUsername(String username) {
if (username.equals("cid")) {
return new User(username, "",
AuthorityUtils
.commaSeparatedStringToAuthorityList("ROLE_USER"));
}
throw new UsernameNotFoundException("User not found!");
}
};
}
}
These are SSL settings:
server:
port: 8443
ssl:
enabled: true
key-store: 'classpath:keystore.jks'
key-store-password: 'changeit'
key-password: 'changeit'
trust-store: 'classpath:truststore.jks'
trust-store-password: 'changeit'
client-auth: need
What I want is to ignore for SSL or to permit for all my WSDL site.
But with this configuration it doesn't work. I get following error:
http https://localhost:8443/ws/items.wsdl http: error: SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645) while doing GET request to URL: https://localhost:8443/ws/items.wsdl
I don't want to use ca signed certificates. (With signed certificates it works.)
How can I do this? What options do I have?
Remove this :
.authorizeRequests()
.antMatchers("/ws/items.wsdl").permitAll()
The web config is saying ignore that path, yet your http security is saying to make requests to that path go through auth.

java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.NETWORK

java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.NETWORK
Hi i got this error while i am calling one API service from retrofit , i am searching a lot and found answer like
private static void setupRestClient() {
RestAdapter restAdapter = new RestAdapter.Builder()
.setLogLevel(RestAdapter.LogLevel.FULL)
.setEndpoint(ROOT)
//.setClient(new OkClient(new com.squareup.okhttp.OkHttpClient()))
//.setClient(getOkClient())
.setClient(setSSLFactoryForClient(new com.squareup.okhttp.OkHttpClient()))
.setRequestInterceptor(new SessionRequestInterceptor())
.setLogLevel(RestAdapter.LogLevel.FULL)
.setLog(new AndroidLog(NetworkUtil.APP_TAG))
.build();
REST_CLIENT = restAdapter.create(Restapi.class);
}
// SET SSL
public static OkClient setSSLFactoryForClient(OkHttpClient client) {
try {
// Create a trust manager that does not validate certificate chains
final TrustManager[] trustAllCerts = new TrustManager[]{
new X509TrustManager() {
#Override
public void checkClientTrusted(java.security.cert.X509Certificate[] chain, String authType) throws CertificateException {
}
#Override
public void checkServerTrusted(java.security.cert.X509Certificate[] chain, String authType) throws CertificateException {
}
#Override
public java.security.cert.X509Certificate[] getAcceptedIssuers() {
return null;
}
}
};
// Install the all-trusting trust manager
final SSLContext sslContext = SSLContext.getInstance("SSL");
sslContext.init(null, trustAllCerts, new java.security.SecureRandom());
// Create an ssl socket factory with our all-trusting manager
final SSLSocketFactory sslSocketFactory = sslContext.getSocketFactory();
client.setSslSocketFactory(sslSocketFactory);
client.setHostnameVerifier(new HostnameVerifier() {
#Override
public boolean verify(String hostname, SSLSession session) {
return true;
}
});
} catch (Exception e) {
throw new RuntimeException(e);
}
return new OkClient(client);
}
After using setSSLFactoryForClient method it work fine but i couldn't understand whats going wrong and what this method does i know the problem is related to SSL Certificate Authentication but Can any one explain me this in brief Please
This is disabling the security of SSL. This is ok for local testing but not appropriate for use with real users.
If you run your local dev server with a self signed cert then this is how you can tell it to connect to it with minimal pain.
More generally any user agent (Firefox on Windows, Safari on Mac, Android) will have a list of root CAs it trusts to verify a sites certificates. Some newer services like let's encrypt will not be trusted on older platforms so you can add your own certificates that you know ahead of time.
The hostname verification means that the cert it serves could be for a different site even.
For real traffic this code means your users are susceptible to man in the middle attacks.

Specifying trust store information in spring boot application.properties

I am using springBootVersion 1.2.0.RELEASE.
I'm trying to have my keystore and truststore configured through application.properties.
When I add the following settings, I can get the keystore to work, but not the truststore.
server.ssl.key-store=classpath:foo.jks
server.ssl.key-store-password=password
server.ssl.key-password=password
server.ssl.trust-store=classpath:foo.jks
server.ssl.trust-store-password=password
However, if I add the truststore through gradle:
bootRun {
jvmArgs = [ "-Djavax.net.ssl.trustStore=c://foo.jks", "-Djavax.net.ssl.trustStorePassword=password"]
}
it works just fine.
Has anyone used the application.properties for trust stores?
In case if you need to make a REST call you can use the next way.
This will work for outgoing calls through RestTemplate.
Declare the RestTemplate bean like this.
#Configuration
public class SslConfiguration {
#Value("${http.client.ssl.trust-store}")
private Resource keyStore;
#Value("${http.client.ssl.trust-store-password}")
private String keyStorePassword;
#Bean
RestTemplate restTemplate() throws Exception {
SSLContext sslContext = new SSLContextBuilder()
.loadTrustMaterial(
keyStore.getURL(),
keyStorePassword.toCharArray()
).build();
SSLConnectionSocketFactory socketFactory =
new SSLConnectionSocketFactory(sslContext);
HttpClient httpClient = HttpClients.custom()
.setSSLSocketFactory(socketFactory).build();
HttpComponentsClientHttpRequestFactory factory =
new HttpComponentsClientHttpRequestFactory(httpClient);
return new RestTemplate(factory);
}
}
Where http.client.ssl.trust-store and http.client.ssl.trust-store-password points to truststore in JKS format and the password for the specified truststore.
This will override the RestTemplate bean provided with Spring Boot and make it use the trust store you need.
I had the same problem with Spring Boot, Spring Cloud (microservices) and a self-signed SSL certificate. Keystore worked out of the box from application properties, and Truststore didn't.
I ended up keeping both keystore and trustore configuration in application.properties, and adding a separate configuration bean for configuring truststore properties with the System.
#Configuration
public class SSLConfig {
#Autowired
private Environment env;
#PostConstruct
private void configureSSL() {
//set to TLSv1.1 or TLSv1.2
System.setProperty("https.protocols", "TLSv1.1");
//load the 'javax.net.ssl.trustStore' and
//'javax.net.ssl.trustStorePassword' from application.properties
System.setProperty("javax.net.ssl.trustStore", env.getProperty("server.ssl.trust-store"));
System.setProperty("javax.net.ssl.trustStorePassword",env.getProperty("server.ssl.trust-store-password"));
}
}
I have the same problem, I'll try to explain it a bit more in detail.
I'm using spring-boot 1.2.2-RELEASE and tried it on both Tomcat and Undertow with the same result.
Defining the trust-store in application.yml like:
server:
ssl:
trust-store: path-to-truststore...
trust-store-password: my-secret-password...
Doesn't work, while:
$ java -Djavax.net.debug=ssl -Djavax.net.ssl.trustStore=path-to-truststore... -Djavax.net.ssl.trustStorePassword=my-secret-password... -jar build/libs/*.jar
works perfectly fine.
The easiest way to see the difference at rutime is to enable ssl-debug in the client. When working (i.e. using -D flags) something like the following is written to the console (during processing of the first request):
trustStore is: path-to-truststore...
trustStore type is : jks
trustStore provider is :
init truststore
adding as trusted cert:
Subject: C=..., ST=..., O=..., OU=..., CN=...
Issuer: C=..., ST=..., O=..., OU=..., CN=...
Algorithm: RSA; Serial number: 0x4d2
Valid from Wed Oct 16 17:58:35 CEST 2013 until Tue Oct 11 17:58:35 CEST 2033
Without the -D flags I get:
trustStore is: /Library/Java/JavaVirtualMachines/jdk1.8.0_11.jdk/Contents/Home/jre/lib/security/cacerts
trustStore type is : jks
trustStore provider is :
init truststore
adding as trusted cert: ... (one for each CA-cert in the defult truststore)
...and when performing a request I get the exception:
sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Hope it helps to understand the issue better!
java properties "javax.net.ssl.trustStore" and "javax.net.ssl.trustStorePassword" do not correspond to "server.ssl.trust-store" and "server.ssl.trust-store-password" from Spring boot "application.properties" ("application.yml")
so you can not set "javax.net.ssl.trustStore" and "javax.net.ssl.trustStorePassword" simply by setting "server.ssl.trust-store" and "server.ssl.trust-store-password" in "application.properties" ("application.yml")
an alternative of setting "javax.net.ssl.trustStore" and "javax.net.ssl.trustStorePassword" is by Spring boot Externalized Configuration
below are excerpts of my implementation :
Params class holds the external settings
#Component
#ConfigurationProperties("params")
public class Params{
//default values, can be override by external settings
public static String trustStorePath = "config/client-truststore.jks";
public static String trustStorePassword = "wso2carbon";
public static String keyStorePath = "config/wso2carbon.jks";
public static String keyStorePassword = "wso2carbon";
public static String defaultType = "JKS";
public void setTrustStorePath(String trustStorePath){
Params.trustStorePath = trustStorePath;
}
public void settrustStorePassword(String trustStorePassword){
Params.trustStorePassword=trustStorePassword;
}
public void setKeyStorePath(String keyStorePath){
Params.keyStorePath = keyStorePath;
}
public void setkeyStorePassword(String keyStorePassword){
Params.keyStorePassword = keyStorePassword;
}
public void setDefaultType(String defaultType){
Params.defaultType = defaultType;
}
KeyStoreUtil class undertakes the settings of "javax.net.ssl.trustStore" and "javax.net.ssl.trustStorePassword"
public class KeyStoreUtil {
public static void setTrustStoreParams() {
File filePath = new File( Params.trustStorePath);
String tsp = filePath.getAbsolutePath();
System.setProperty("javax.net.ssl.trustStore", tsp);
System.setProperty("javax.net.ssl.trustStorePassword", Params.trustStorePassword);
System.setProperty("javax.net.ssl.keyStoreType", Params.defaultType);
}
public static void setKeyStoreParams() {
File filePath = new File(Params.keyStorePath);
String ksp = filePath.getAbsolutePath();
System.setProperty("Security.KeyStore.Location", ksp);
System.setProperty("Security.KeyStore.Password", Params.keyStorePassword);
}
}
you get the setters executed within the startup function
#SpringBootApplication
#ComponentScan("com.myapp.profiles")
public class ProfilesApplication {
public static void main(String[] args) {
KeyStoreUtil.setKeyStoreParams();
KeyStoreUtil.setTrustStoreParams();
SpringApplication.run(ProfilesApplication.class, args);
}
}
Edited on 2018-10-03
you may also want to adopt the annotation "PostConstruct" as as an alternative to execute the setters
import javax.annotation.PostConstruct;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
#SpringBootApplication(scanBasePackages={"com.xxx"})
public class GateApplication {
public static void main(String[] args) {
SpringApplication.run(GateApplication.class, args);
}
#PostConstruct
void postConstruct(){
setTrustStoreParams();
setKeyStoreParams();
}
private static void setTrustStoreParams() {
File filePath = new File( Params.trustStorePath);
String tsp = filePath.getAbsolutePath();
System.setProperty("javax.net.ssl.trustStore", tsp);
System.setProperty("javax.net.ssl.trustStorePassword", Params.trustStorePassword);
System.setProperty("javax.net.ssl.keyStoreType", Params.defaultType);
}
private static void setKeyStoreParams() {
File filePath = new File(Params.keyStorePath);
String ksp = filePath.getAbsolutePath();
System.setProperty("Security.KeyStore.Location", ksp);
System.setProperty("Security.KeyStore.Password", Params.keyStorePassword);
}
}
the application.yml
---
params:
trustStorePath: config/client-truststore.jks
trustStorePassword: wso2carbon
keyStorePath: config/wso2carbon.jks
keyStorePassword: wso2carbon
defaultType: JKS
---
finally, within the running environment(deployment server), you create a folder named "config" under the same folder where the jar archive is stored .
within the "config" folder, you store "application.yml", "client-truststore.jks", and "wso2carbon.jks". done!
Update on 2018-11-27 about Spring boot 2.x.x
starting from spring boot 2.x.x, static properties are no longer supported, please see here. I personally do not think it a good move, because complex changes have to be made along the reference chain...
anyway, an implementation excerpt might look like this
the 'Params' class
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.stereotype.Component;
import lombok.Data;
/**
* Params class represent all config parameters that can
* be external set by spring xml file
*/
#Component
#ConfigurationProperties("params")
#Data
public class Params{
//default values, can be override by external settings
public String trustStorePath = "config/client-truststore.jks";
public String trustStorePassword = "wso2carbon";
public String keyStorePath = "config/wso2carbon.jks";
public String keyStorePassword = "wso2carbon";
public String defaultType = "JKS";
}
the 'Springboot application class' (with 'PostConstruct')
import java.io.File;
import javax.annotation.PostConstruct;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
#SpringBootApplication(scanBasePackages={"com.xx.xx"})
public class BillingApplication {
#Autowired
Params params;
public static void main(String[] args) {
SpringApplication.run(BillingApplication.class, args);
}
#PostConstruct
void postConstruct() {
// set TrustStoreParams
File trustStoreFilePath = new File(params.trustStorePath);
String tsp = trustStoreFilePath.getAbsolutePath();
System.setProperty("javax.net.ssl.trustStore", tsp);
System.setProperty("javax.net.ssl.trustStorePassword", params.trustStorePassword);
System.setProperty("javax.net.ssl.keyStoreType", params.defaultType);
// set KeyStoreParams
File keyStoreFilePath = new File(params.keyStorePath);
String ksp = keyStoreFilePath.getAbsolutePath();
System.setProperty("Security.KeyStore.Location", ksp);
System.setProperty("Security.KeyStore.Password", params.keyStorePassword);
}
}
I was also having the same issue with Spring Boot and embedded Tomcat.
From what I understand these properties only set the Tomcat configuration parameters. According to the Tomcat documentation this is only used for Client authentication (i.e. for two-way SSL) and not for verifying remote certificates:
truststoreFile - The trust store file to use to validate client certificates.
https://tomcat.apache.org/tomcat-8.0-doc/config/http.html
In order to configure the trust store for HttpClient it largely depends on the HttpClient implementation you use. For instance for RestTemplate by default Spring Boot uses a SimpleClientHttpRequestFactory based on standard J2SE classes like java.net.HttpURLConnection.
I've come up with a solution based on the Apache HttpClient docs and these posts:
http://vincentdevillers.blogspot.pt/2013/02/configure-best-spring-resttemplate.html
http://literatejava.com/networks/ignore-ssl-certificate-errors-apache-httpclient-4-4/
Basically this allows for a RestTemplate bean that only trusts certificates signed by the root CA in the configured truststore.
#Configuration
public class RestClientConfig {
// e.g. Add http.client.ssl.trust-store=classpath:ssl/truststore.jks to application.properties
#Value("${http.client.ssl.trust-store}")
private Resource trustStore;
#Value("${http.client.ssl.trust-store-password}")
private char[] trustStorePassword;
#Value("${http.client.maxPoolSize}")
private Integer maxPoolSize;
#Bean
public ClientHttpRequestFactory httpRequestFactory() {
return new HttpComponentsClientHttpRequestFactory(httpClient());
}
#Bean
public HttpClient httpClient() {
// Trust own CA and all child certs
Registry<ConnectionSocketFactory> socketFactoryRegistry = null;
try {
SSLContext sslContext = SSLContexts
.custom()
.loadTrustMaterial(trustStore.getFile(),
trustStorePassword)
.build();
// Since only our own certs are trusted, hostname verification is probably safe to bypass
SSLConnectionSocketFactory sslSocketFactory = new SSLConnectionSocketFactory(sslContext,
new HostnameVerifier() {
#Override
public boolean verify(final String hostname,
final SSLSession session) {
return true;
}
});
socketFactoryRegistry = RegistryBuilder.<ConnectionSocketFactory>create()
.register("http", PlainConnectionSocketFactory.getSocketFactory())
.register("https", sslSocketFactory)
.build();
} catch (Exception e) {
//TODO: handle exceptions
e.printStackTrace();
}
PoolingHttpClientConnectionManager connectionManager = new PoolingHttpClientConnectionManager(socketFactoryRegistry);
connectionManager.setMaxTotal(maxPoolSize);
// This client is for internal connections so only one route is expected
connectionManager.setDefaultMaxPerRoute(maxPoolSize);
return HttpClientBuilder.create()
.setConnectionManager(connectionManager)
.disableCookieManagement()
.disableAuthCaching()
.build();
}
#Bean
public RestTemplate restTemplate() {
RestTemplate restTemplate = new RestTemplate();
restTemplate.setRequestFactory(httpRequestFactory());
return restTemplate;
}
}
And then you can use this custom Rest client whenever you need to, e.g.:
#Autowired
private RestTemplate restTemplate;
restTemplate.getForEntity(...)
This assumes your trying to connect to a Rest endpoint, but you can also use the above HttpClient bean for whatever you want.
If you execute your Spring Boot application as a linux service (e.g. init.d script or similar), then you have the following option as well:
Create a file called yourApplication.conf and put it next to your executable war/jar file. It's content should be something similar:
JAVA_OPTS="
-Djavax.net.ssl.trustStore=path-to-your-trustStore-file
-Djavax.net.ssl.trustStorePassword=yourCrazyPassword
"
Although I am commenting late. But I have used this method to do the job. Here when I am running my spring application I am providing the application yaml file via -Dspring.config.location=file:/location-to-file/config-server-vault-application.yml which contains all of my properties
config-server-vault-application.yml
***********************************
server:
port: 8888
ssl:
trust-store: /trust-store/config-server-trust-store.jks
trust-store-password: config-server
trust-store-type: pkcs12
************************************
Java Code
************************************
#SpringBootApplication
public class ConfigServerApplication {
public static void main(String[] args) throws IOException {
setUpTrustStoreForApplication();
SpringApplication.run(ConfigServerApplication.class, args);
}
private static void setUpTrustStoreForApplication() throws IOException {
YamlPropertySourceLoader loader = new YamlPropertySourceLoader();
List<PropertySource<?>> applicationYamlPropertySource = loader.load(
"config-application-properties", new UrlResource(System.getProperty("spring.config.location")));
Map<String, Object> source = ((MapPropertySource) applicationYamlPropertySource.get(0)).getSource();
System.setProperty("javax.net.ssl.trustStore", source.get("server.ssl.trust-store").toString());
System.setProperty("javax.net.ssl.trustStorePassword", source.get("server.ssl.trust-store-password").toString());
}
}
Here my extended version of Oleksandr Shpota's answer, including the imports. The package org.apache.http.* can be found in org.apache.httpcomponents:httpclient. I've commented the changes:
import org.apache.http.client.HttpClient;
import org.apache.http.conn.ssl.NoopHostnameVerifier;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.conn.ssl.TrustSelfSignedStrategy;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.ssl.SSLContexts;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.core.io.Resource;
import org.springframework.http.client.HttpComponentsClientHttpRequestFactory;
import org.springframework.web.client.RestTemplate;
#Value("${http.client.ssl.key-store}")
private Resource keyStore;
#Value("${http.client.ssl.trust-store}")
private Resource trustStore;
// I use the same pw for both keystores:
#Value("${http.client.ssl.trust-store-password}")
private String keyStorePassword;
// wasn't able to provide this as a #Bean...:
private RestTemplate getRestTemplate() {
try {
SSLContext sslContext = SSLContexts.custom()
// keystore wasn't within the question's scope, yet it might be handy:
.loadKeyMaterial(
keyStore.getFile(),
keyStorePassword.toCharArray(),
keyStorePassword.toCharArray())
.loadTrustMaterial(
trustStore.getURL(),
keyStorePassword.toCharArray(),
// use this for self-signed certificates only:
new TrustSelfSignedStrategy())
.build();
HttpClient httpClient = HttpClients.custom()
// use NoopHostnameVerifier with caution, see https://stackoverflow.com/a/22901289/3890673
.setSSLSocketFactory(new SSLConnectionSocketFactory(sslContext, new NoopHostnameVerifier()))
.build();
return new RestTemplate(new HttpComponentsClientHttpRequestFactory(httpClient));
} catch (IOException | GeneralSecurityException e) {
throw new RuntimeException(e);
}
}
If you are in Spring, try just add properties for it (use needed properties), and it should work for total JVM
javax:
net:
ssl:
key-store-password: ${KEYSTORE_SECRET}
key-store-type: PKCS12
trust-store-password: ${TRUSTSTORE_SECRET}
trust-store-type: PKCS12
In a microservice infrastructure (does not fit the problem, I know ;)) you must not use:
server:
ssl:
trust-store: path-to-truststore...
trust-store-password: my-secret-password...
Instead the ribbon loadbalancer can be configuered:
ribbon:
TrustStore: keystore.jks
TrustStorePassword : example
ReadTimeout: 60000
IsSecure: true
MaxAutoRetries: 1
Here https://github.com/rajaramkushwaha/https-zuul-proxy-spring-boot-app you can find a working sample. There was also a github discussion about that, but I didn't find it anymore.