Mutual Authentication with Reactive Netty on restricted urls - ssl

I am using spring cloud gateway to front a legacy application so that we can start migrating things behind the scenes. Some of the urls that are hosted by the application are public facing and some are device restricted. We control the devices and they use a browser client to access the restricted urls. We have mutual authentication setup for the device restricted urls on the server using tomcat and security constraints like this in web.xml:
<security-constraint>
<web-resource-collection>
<web-resource-name>Certificate Content</web-resource-name>
<!-- URL for authentication endpoint - this is locked down with the role assigned by tomcat -->
<url-pattern>/rest/secure/url1</url-pattern>
<url-pattern>/rest/secure/url2</url-pattern>
<url-pattern>/rest/secure/url3</url-pattern>
</web-resource-collection>
<auth-constraint>
<role-name>certificate</role-name>
</auth-constraint>
<user-data-constraint>
<transport-guarantee>CONFIDENTIAL</transport-guarantee>
</user-data-constraint>
</security-constraint>
<!-- All other endpoints- force the switch from http to https with transport-guarantee -->
<security-constraint>
<web-resource-collection>
<web-resource-name>Protected Context</web-resource-name>
<url-pattern>/*</url-pattern>
</web-resource-collection>
<user-data-constraint>
<transport-guarantee>CONFIDENTIAL</transport-guarantee>
</user-data-constraint>
</security-constraint>
<login-config>
<auth-method>CLIENT-CERT</auth-method>
</login-config>
<security-role>
<role-name>certificate</role-name>
</security-role>
That is coupled with a truststore setup in tomcat's server.xml (I can add it, but I don't think that is relevant to this conversation).
My goal is to implement a similar setup in spring cloud gateway which is using reactive netty under-the-hood and remove the web.xml restrictions from the legacy application. I think I could switch it to using tomcat and probably get the web.xml from above to work, but I'd rather stick to the performance benefits of using reactive netty.
Key Goals:
Only deploy one api gateway for the app. The number of urls that
require mutual auth is very small so I'd rather not include a whole
other container to manage just to support them.
Do not ask for a client cert on the public urls.
Require valid client certs for the restricted urls.
I've setup mutual authentication and can get it to work with need/want/none as expected (truststores setup, etc), but it applies to ALL urls. I've also setup X509 security restrictions and that all seems to work.
I think what I want to setup is tsl renegotiation using the SslHandler after the http request is decrypted (so that I can access the url) based on the path. But I'm having trouble with the details and I've failed at finding any examples that incorporate spring-boot applications using reactive netty to do a tsl renegotiation. Any tips on how to perform a renegotiation of the ssl connection with needClientAuth set to true would be appreciated. I think I need to invalidate the session or something because when I try to do it manually it appears that it is skipping negotiation because the connection is already marked as negotiated in the ssl engine.
This is one of the iterations I've tried (this doesn't restrict on urls, but I plan to add that after I get this working):
#Component
public class NettyWebServerFactoryGatewayCustomizer implements WebServerFactoryCustomizer<NettyReactiveWebServerFactory> {
private static final Logger LOG = LoggerFactory.getLogger(NettyWebServerFactoryGatewayCustomizer.class);
#Override
public void customize(NettyReactiveWebServerFactory serverFactory) {
serverFactory.addServerCustomizers(httpServer -> {
httpServer = httpServer.wiretap(true);
return httpServer.tcpConfiguration(tcpServer -> {
tcpServer = tcpServer.doOnConnection(connection ->
connection.addHandler("request client cert",
new SimpleChannelInboundHandler<HttpRequest>() {
#Override
protected void channelRead0(ChannelHandlerContext ctx, HttpRequest httpRequest) {
LOG.error("HttpRequest: {}", httpRequest);
final ChannelPipeline pipeline = ctx.pipeline();
final SslHandler sslHandler = pipeline.get(SslHandler.class);
final SSLEngine sslEngine = sslHandler.engine();
sslEngine.setNeedClientAuth(true);
sslHandler.renegotiate()
.addListener(future -> ctx.fireChannelRead(httpRequest));
}
}
)
);
return tcpServer;
});
});
}
}
I see it performing the renegotiation in the debugger, but it still seems to be set to client auth none (as set in the application.properties) instead of need as set in the code before renegotiation. I've tried sslEngine.getSession().invalidate(); but that didn't help. I've also tried generating a new ssl handler from the ssl provider but that seemed to really screw things up.
Thank you for any help provided.
Edit: Doing more research it appears that this approach is not appropriate going forward since ssl renegotiation is being dropped entirely in tsl 1.3 (see https://security.stackexchange.com/a/230327). Is there a way to perform the equivalent of SSL verify client post handshake as described here: https://www.openssl.org/docs/manmaster/man3/SSL_verify_client_post_handshake.html ?
Edit2: Looks like this was an issue where TLS1.3 post handshake was not supported by the browser I was testing with. Setting the server to just accept TLS 1.2 seemed to work. Not sure if there is a better way to solve this but this is what I added to my application.properties:
server.ssl.enabled-protocols=TLSv1.2

Here is what I used to get it to work. I'm going to leave out the spring security side of it since that is separate from requesting the certificate from the client.
There are so many ways to configure the child pipeline that is used to process the request. Please let me know if there is a more accepted way to configure it.
Configure the HttpServer by adding to the bootstrap pipeline that is applied when a connection is established with the client:
#Component
public class NettyWebServerFactoryGatewayCustomizer implements WebServerFactoryCustomizer<NettyReactiveWebServerFactory> {
private static final HttpRenegotiateClientCertHandler HTTP_RENEGOTIATE_CLIENT_CERT_HANDLER =
new HttpRenegotiateClientCertHandler(SecurityConfig.X509_PROTECTED_ENDPOINTS);
#Override
public void customize(NettyReactiveWebServerFactory serverFactory) {
serverFactory.addServerCustomizers(NettyWebServerFactoryGatewayCustomizer::addRenegotiateHandlerToHttpServer);
}
private static HttpServer addRenegotiateHandlerToHttpServer(HttpServer httpServer) {
return httpServer.tcpConfiguration(NettyWebServerFactoryGatewayCustomizer::addRenegotiateHandlerToTcpServer);
}
private static TcpServer addRenegotiateHandlerToTcpServer(TcpServer server) {
return server.doOnBind(NettyWebServerFactoryGatewayCustomizer::addRenegotiateHandlerToServerBootstrap);
}
private static void addRenegotiateHandlerToServerBootstrap(ServerBootstrap serverBootstrap) {
BootstrapHandlers.updateConfiguration(
serverBootstrap,
HttpRenegotiateClientCertHandler.NAME,
NettyWebServerFactoryGatewayCustomizer::addRenegotiateHandlerToChannel
);
}
private static void addRenegotiateHandlerToChannel(ConnectionObserver connectionObserver, Channel channel) {
final ChannelPipeline pipeline = channel.pipeline();
pipeline.addLast(HttpRenegotiateClientCertHandler.NAME, HTTP_RENEGOTIATE_CLIENT_CERT_HANDLER);
}
}
Child Handler that performs the renegotiation:
#ChannelHandler.Sharable
public class HttpRenegotiateClientCertHandler extends SimpleChannelInboundHandler<HttpRequest> {
public static final String NAME = NettyPipeline.LEFT + "clientRenegotiate";
private static final PathPatternParser DEFAULT_PATTERN_PARSER = new PathPatternParser();
private final Collection<PathPattern> pathPatterns;
public HttpRenegotiateClientCertHandler(String ... antPatterns) {
Assert.notNull(antPatterns, "patterns cannot be null");
Assert.notEmpty(antPatterns, "patterns cannot be empty");
Assert.noNullElements(antPatterns, "patterns cannot have null items");
pathPatterns = Arrays.stream(antPatterns)
.map(DEFAULT_PATTERN_PARSER::parse)
.collect(Collectors.toSet());
}
#Override
protected void channelRead0(ChannelHandlerContext ctx, HttpRequest request) {
if (shouldNotRenegotiate(request)) {
ctx.fireChannelRead(request);
return;
}
final ChannelPipeline pipeline = ctx.pipeline();
final SslHandler sslHandler = pipeline.get(SslHandler.class);
final SSLEngine sslEngine = sslHandler.engine();
sslEngine.setNeedClientAuth(true);
sslHandler.renegotiate()
.addListener(renegotiateFuture -> ctx.fireChannelRead(request));
}
/**
* Determine if the request uri matches the configured uris for this handler.
* #param request to match the path from.
* #return true if any of the path patterns are matched.
*/
private boolean shouldNotRenegotiate(HttpRequest request) {
final String requestUri = request.uri();
final PathContainer path = PathContainer.parsePath(requestUri);
return pathPatterns.stream()
.noneMatch(matcher -> matcher.matches(path));
}
}
And these configurations in application.properties:
# Setup Client Auth Truststore:
server.ssl.trust-store=<path to truststore>
server.ssl.trust-store-password=<truststore password>
server.ssl.trust-store-type=<truststore type>
# Set to none by default so we do not ask for client auth until needed.
server.ssl.client-auth=none
# This is specifically not including TLSv1.3 because there are issues
# with older browsers' implementation of TLSv1.3 that prevent verify
# client post handshake client from working.
server.ssl.enabled-protocols=TLSv1.2
Edit: Updated because handler gateway route code wasn't being invoked properly.

Related

S3A client and local S3 mock

To create end-to-end local tests of data workflow I utilize "mock S3" container (e.g adobe/S3Mock). Seems to work just fine. However, some parts of the system rely on S3A client. As far as I see, its format does not allow to point to particular nameserver or endpoint.
Is it possible to make S3A work in local environment?
you talking about the ASF Hadoop S3A Connector? Nobody has tested against S3 mock AFAIK (never seen it before!), but it does work with non-AWS endpoints
set fs.s3a.endpoint to the URL of your S3 connection. There's some settings about switching from https to http (fs.s3a.connection.ssl.enabled = false) and moving from virtual hosts to directories (fs.s3a.path.style.access = true) which will also be needed.
further reading
Like I said: nobody has done this. We developers just go against the main AWS endpoints with its problems (latency, inconsistency, error reporting, etc), precisely because its what you get in production. But for your local testing, it will simplify your life (and you can run it under jenkins without having to give it any secrets)
Answer by #stevel worked for me. Here is the code if someone wants to refer
class S3WriterTest {
private static S3Mock api;
private static AmazonS3 mockS3client;
#BeforeAll
public static void setUp() {
//start mock s3 service using findify
api = new S3Mock.Builder().withPort(8001).withInMemoryBackend().build();
api.start();
/* AWS S3 client setup.
* withPathStyleAccessEnabled(true) trick is required to overcome S3 default
* DNS-based bucket access scheme
* resulting in attempts to connect to addresses like "bucketname.localhost"
* which requires specific DNS setup.
*/
EndpointConfiguration endpoint = new EndpointConfiguration("http://localhost:8001", "us-west-2");
mockS3client = AmazonS3ClientBuilder
.standard()
.withEndpointConfiguration(endpoint)
.withPathStyleAccessEnabled(true)
.withCredentials(new AWSStaticCredentialsProvider(new AnonymousAWSCredentials()))
.build();
mockS3client.createBucket("test-bucket");
}
#AfterAll
public static void tearDown() {
api.shutdown();
}
#Test
void unitTestForHadoopCodeWritingUsingS3A {
Configuration hadoopConfig = getTestConfiguration();
........
}
private static Configuration getTestConfiguration() {
Configuration config = new Configuration();
config.set("fs.s3a.endpoint", "http://127.0.0.1:8001");
config.set("fs.s3a.connection.ssl.enabled", "false");
config.set("fs.s3a.path.style.access", "true");
config.set("fs.s3a.aws.credentials.provider", "org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider");
config.set("fs.s3a.access.key", "foo");
config.set("fs.s3a.secret.key", "bar");
return config;
}
}

Apache camel SSL connection to restful service

I am busy with a project where I have to do a GET on an exposed rest service using specific certificates. I am using the apache camel framework with the https4 component. I created a keystore and tested it using soapUI and it connected successfully, but I am however unable to connect through my project.
I used the following page as reference: http://camel.apache.org/http4.html
I set up the SSL for the HTTP Client through the following configuration:
<spring:sslContextParameters id="sslContextParameters">
<spring:keyManagers keyPassword="xxxx">
<spring:keyStore resource="classpath:certificates/keystore.jks" password="xxxx"/>
</spring:keyManagers>
</spring:sslContextParameters>
<setHeader headerName="CamelHttpMethod">
<simple>GET</simple>
</setHeader>
My endpoint is configured as:
<to uri="https4://endpointUrl:9007/v1/{id}?sslContextParametersRef=sslContextParameters"/>
The stacktrace I am receiving:
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1904)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:279)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:273)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1446)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:209)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:901)
at sun.security.ssl.Handshaker.process_record(Handshaker.java:837)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1023)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1332)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1359)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1343)
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:394)
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:353)
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:141)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.camel.component.http4.HttpProducer.executeMethod(HttpProducer.java:301)
at org.apache.camel.component.http4.HttpProducer.process(HttpProducer.java:173)
at org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:145)
at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:163)
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:468)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:197)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:121)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:83)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:197)
at org.apache.camel.component.direct.DirectProducer.process(DirectProducer.java:62)
at org.apache.camel.impl.InterceptSendToEndpoint$1.process(InterceptSendToEndpoint.java:164)
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:145)
at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:163)
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:468)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:197)
at org.apache.camel.processor.ChoiceProcessor.process(ChoiceProcessor.java:117)
at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:163)
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:468)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:197)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:121)
at org.apache.camel.processor.Pipeline.access$100(Pipeline.java:44)
at org.apache.camel.processor.Pipeline$1.done(Pipeline.java:139)
at org.apache.camel.processor.CamelInternalProcessor$InternalCallback.done(CamelInternalProcessor.java:257)
at org.apache.camel.processor.RedeliveryErrorHandler$1.done(RedeliveryErrorHandler.java:480)
at org.apache.camel.processor.interceptor.TraceInterceptor$1.done(TraceInterceptor.java:180)
at org.apache.camel.processor.SendProcessor$1.done(SendProcessor.java:155)
at org.apache.camel.processor.CamelInternalProcessor$InternalCallback.done(CamelInternalProcessor.java:257)
at org.apache.camel.processor.Pipeline$1.done(Pipeline.java:148)
at org.apache.camel.processor.CamelInternalProcessor$InternalCallback.done(CamelInternalProcessor.java:257)
at org.apache.camel.processor.RedeliveryErrorHandler$1.done(RedeliveryErrorHandler.java:480)
at org.apache.camel.processor.interceptor.TraceInterceptor$1.done(TraceInterceptor.java:180)
at org.apache.camel.processor.SendProcessor$1.done(SendProcessor.java:155)
at org.apache.camel.component.cxf.CxfClientCallback.handleResponse(CxfClientCallback.java:61)
at org.apache.cxf.endpoint.ClientImpl.onMessage(ClientImpl.java:827)
at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleResponseInternal(HTTPConduit.java:1672)
at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream$1.run(HTTPConduit.java:1168)
at org.apache.cxf.workqueue.AutomaticWorkQueueImpl$3.run(AutomaticWorkQueueImpl.java:428)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at org.apache.cxf.workqueue.AutomaticWorkQueueImpl$AWQThreadFactory$1.run(AutomaticWorkQueueImpl.java:353)
at java.lang.Thread.run(Thread.java:745)
Any help would be much appreciated !
Just same: I followed documented instructions and got too stuck on "PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target". There's a quick fix, but if you want to link the configuration to the client HTTP session at stake, it becomes a complex set-up.
Method 1:
Doc pages, forums, and this other article would tell you that setting JVM launch options "-Djavax.net.ssl.trustStore=myKeystore.jks -Djavax.net.ssl.trustStorePassword=mystorepass" do solve the issue, provided the remote parties' certificates (self signed, or signed by a CA but then with all the full certificate chain) were all fetched as Trusted certificates in the supplied keystore. Fact is, HTTP4 is based on JSSE, and these java launch options do configure the stack JVM-wide.
As an alternative, you can also fetch peers' certificates (complete chains) in the default JVM keystore jre\lib\security\cacerts (initial password: "changeit") and thus not even need JVM options.
If you have a few outgoing client connections and few peer certificates, this is the simplest way.
Method 2:
In our context, with above 100 remote parties, each requiring certificate updates every 2 years in average, that method implies a JVM reboot on an updated keystore about every week. Our highly available gateway is no longer highly available. So I searched a dynamic/per-connexion/programmatic way.
Below is a simplified excerpt of code from a CAMEL Processor that we use to remotely connect as REST or plain-vanilla HTTP client, with or without SSL/TLS, and with or without client-side certificate (i.e. 2-way SSL/TLS versus 1-way SSL/TLS), as well as combine HTTP Basic Auth as required by peers.
For various reasons the now old CAMEL version 2.16.3 is still used in our context. I have not tested yet newer versions. I suspect no changes given the libraries at stake under the Apache CAMEL layer.
I have added in the code below many comments detailling variant API's to the same effect. So you have clues below to further simplify the code or try alternatives with newer HTTP4 versions. As is, the code works with 2.16, as a CAMEL Processor bean within a Spring application context that contains the entire CAMEL route definition in DSL.
In our context we use java code for configuring entirely dynamic SSL/TLS outbound connexions per session. You should have no difficulties freezing part of the configuration that we set below dynamically via java, into the CAMEL XML DSL as suitable to your context.
Maven dependencies at stake:
<properties>
<camel-version>2.16.3</camel-version>
</properties>
...
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-core</artifactId>
<version>${camel-version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-http4</artifactId>
<version>${camel-version}</version>
<scope>provided</scope>
</dependency>
Code extracted from our org.apache.camel.Processor (I have removed many Exception handling and simplified the code below in order to focus on the solution):
// relevant imports (partial)
import java.security.KeyStore;
import java.security.SecureRandom;
import java.security.Security;
import java.security.cert.CertificateException;
import java.security.cert.X509Certificate;
import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.KeyManagerFactory;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSession;
import javax.net.ssl.TrustManager;
import javax.net.ssl.TrustManagerFactory;
import javax.net.ssl.X509TrustManager;
import org.apache.camel.Exchange;
import org.apache.camel.Processor;
import org.apache.camel.component.http4.HttpClientConfigurer;
import org.apache.camel.component.http4.HttpComponent;
import org.apache.http.config.Registry;
import org.apache.http.config.RegistryBuilder;
import org.apache.http.conn.HttpClientConnectionManager;
import org.apache.http.conn.socket.ConnectionSocketFactory;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.impl.conn.BasicHttpClientConnectionManager;
...
#Override
public void process(Exchange exchange) throws Exception {
// assume here that we have previously fetched all dynamic connexion parameters in set of java Properties. Of course you can use numerous means to inject connection parameters
Properties params= ... ;
// Trick! 'targetURL' is the URI of the http server to call. Its not the same as the Camel endpoint URI (see further "httpUrlToken" placeHolder), on which you configure endpoint options
// Fact is, we prefer to pass just the target URL as parameter and keep full control on building the CAMEL endpoint URI in java
String targetURL= params.getProperty("targetURL"); // URL to call, e.g. "http://remoteHost.com/some/servlet/path". Will override the placeholder URL set on the endpoint.
// default plain HTTP without SSL/TLS:
String endPointURI = "http4://httpUrlToken?throwExceptionOnFailure=false"; // with option to prevent exceptions from being thrown for failed response codes. It allows us to process all the response codes in a response Processor
// Oh yes! we have to manage a map of HttpComponent instances, because the CAMEL doc clearly tells that each instance can only support a single configuration
// and our true connector is multithreading where each request may go to a different (dynamic) destination with different SSL settings,
// so we actually use a Map of HttpComponent instances of size MAX_THREADS and indexed by the thread ID plus ageing and re-use strategies... but this brings us too far.
// So, for a single thread per client instance, you can just do:
HttpComponent httpComponent = exchange.getContext().getComponent("http4", HttpComponent.class);
// overload in case of SSL/TLS
if (targetURL.startsWith("https")) {
try {
endPointURI = "https4://httpUrlToken?throwExceptionOnFailure=false";
httpComponent = exchange.getContext().getComponent("https4", HttpComponent.class); // well: "https4" and "http4" are the same, so you may skip this line! (our true HttpComponent map is common to secured and unsecured client connexions)
// basic SSL context setup as documented elsewhere, should be enough in theory
SSLContext sslctxt = getSSLContext(exchange, params.getProperty("keystoreFilePath"), params.getProperty("keystorePassword"), params.getProperty("authenticationMode")); // cfr helper method below
HttpClientConfigurer httpClientConfig = getEndpointClientConfigurer(sslctxt); // cfr helper method below
httpComponent.setHttpClientConfigurer(httpClientConfig);
// from here, if you skip the rest of the configuration, you'll get the exception "sun.security.provider.certpath.SunCertPathBuilderException:unable to find valid certification path to requested target"
// the SSL context covers certificate validation but not the host name verification process
// we de-activate here at the connection factory level (systematically... you may not want that), and link the later to the HTTP component
HostnameVerifier hnv = new AllowAll();
SSLConnectionSocketFactory sslSocketFactory = new SSLConnectionSocketFactory(sslctxt, hnv);
// You may choose to enforce the BasicHttpClientConnectionManager or PoolingHttpClientConnectionManager, cfr CAMEL docs
// In addition, the following linkage of the connection factory through a Registry that captures the 'https' scheme to your factory is required
Registry<ConnectionSocketFactory> lookup = RegistryBuilder.<ConnectionSocketFactory>create().register("https", sslSocketFactory).build();
HttpClientConnectionManager connManager = new BasicHttpClientConnectionManager(lookup);
// Does not work in 2.16, as documented at http://camel.apache.org/http4.html#HTTP4-UsingtheJSSEConfigurationUtility
// ... keystore and key manager setup ...
// SSLContextParameters scp = new SSLContextParameters();
// scp.setKeyManagers(...);
// httpComponent.setSslContextParameters(scp);
// Not as good as using a connection manager on the HTTP component, although same effects in theory
// HttpClientBuilder clientBuilder = HttpClientBuilder.create();
// clientBuilder.set... various parameters...
// httpClientConfig.configureHttpClient(clientBuilder);
// Commented-out alternative method to set BasicAuth with user and password
// HttpConfiguration httpConfiguration = new HttpConfiguration();
// httpConfiguration.setAuthUsername(authUsername);
// ... more settings ...
// httpComponent.setHttpConfiguration(httpConfiguration);
// setClientConnectionManager() is compulsory to prevent "SunCertPathBuilderException: unable to find valid certification path to requested target"
// if instead we bind the connection manager to a clientBuilder, that doesn't work...
httpComponent.setClientConnectionManager(connManager);
} catch (Exception e) { ... ; }
}
// (back to code common to secured and unsecured client sessions)
// additional parameters on the endpoint as needed, cfr API docs
httpComponent.set...(...) ;
// you may want to append these 3 URI options in case of HTTP[S] with Basic Auth
if (... basic Auth needed ...)
endPointURI += "&authUsername="+params.getProperty("user")+"&authPassword="+params.getProperty("password")+"&authenticationPreemptive=true";
// *********** ACTUAL TRANSMISSION ********************
exchange.getIn().setHeader(Exchange.HTTP_URI, targetURL); // needed to overload the "httpUrlToken" placeholder in the endPointURI
// Next, there are many ways to get a CAMEL Producer or ProducerTemplate
// e.g. httpComponent.createEndpoint(endPointURI).createProducer()
// ... in our case we use a template injected from a Spring application context (i.e. <camel:template id="producerTemplate"/>) via constructor arguments on our Processor bean
try {
producerTemplate.send(httpComponent.createEndpoint(endPointURI),exchange);
} catch (Exception e) { ...; }
// you can then process the HTTP response here, or better dedicate the next
// Processor on the CAMEL route to such handlings...
...
}
Supporting helper methods, invoked by above code
private HttpClientConfigurer getEndpointClientConfigurer(final SSLContext sslContext) {
return new HttpClientConfigurer(){
#Override
public void configureHttpClient(HttpClientBuilder clientBuilder) {
// I put a logger trace here to see if/when the ssl context is actually applied, the outcome was ... weird, try it!
clientBuilder.setSSLContext(sslContext);
}
};
}
/**
* Build a SSL context with keystore and other parameters according to authentication mode.
* The keystore may just contain a trusted peer's certificate for 1way cases, and the associated certificate chain up to a trusted root as applicable.
* The keystore shall too contain one single client private key and certificate for 2way modes. We assume here a same password on keystore and private key.
* #param authenticationMode one of "1waySSL" "1wayTLS" "2waySSL" "2wayTLS" each possibly suffixed by "noCHECK" as in "1waySSLnoCHECK"
* #param keystoreFilePath can be null for "noCHECK" modes
* #param keystorePassword would be null if above is null
*/
private SSLContext getSSLContext(Exchange exchange, String keystoreFilePath, String keystorePassword, String authenticationMode) throws GeneralSecurityException, FileNotFoundException, IOException {
SSLContext sslContext = SSLContext.getInstance(authenticationMode.substring(4,7).toUpperCase(),"SunJSSE");
//enforce Trust ALL ? pass a trust manager that does not validate certificate chains
if (authenticationMode.endsWith("noCHECK")) {
TrustManager[] trustAllCerts = new TrustManager[]{ new TrustALLManager()};
sslContext.init(null , trustAllCerts, null);
return sslContext;
}
// we use https, and validate remote cert's by default, henceforth keystore and password become compulsory
if (null == keystoreFilePath || null == keystorePassword)
throw new GeneralSecurityException("Config ERROR: using https://... and implicit default AUTHMODE=1waySSL altogether requires to supply keystore parameters");
KeyStore trustStore = KeyStore.getInstance(KeyStore.getDefaultType());
TrustManagerFactory tmf = TrustManagerFactory.getInstance("SunX509");
trustStore.load(new FileInputStream(keystoreFilePath), keystorePassword.toCharArray());
tmf.init(trustStore);
KeyStore keyStore = KeyStore.getInstance(KeyStore.getDefaultType());
KeyManagerFactory kmf = KeyManagerFactory.getInstance("SunX509");
if (authenticationMode.charAt(0)=='2') { // our authenticationMode starts with 1way.. or 2way...
// 2way... case: set the keystore parameters accordingly
keyStore.load(new FileInputStream(keystoreFilePath), keystorePassword.toCharArray());
kmf.init(keyStore, keystorePassword.toCharArray());
sslContext.init(kmf.getKeyManagers() , tmf.getTrustManagers(), new SecureRandom());
} else { // 1way... case
sslContext.init(null , tmf.getTrustManagers(), new SecureRandom());
}
return sslContext;
}
// Create a trust manager that does not validate certificate chains
private class TrustALLManager implements X509TrustManager {
#Override
public void checkClientTrusted(X509Certificate[] chain, String authType) throws CertificateException { }
#Override
public void checkServerTrusted(X509Certificate[] chain, String authType) throws CertificateException { }
#Override
public X509Certificate[] getAcceptedIssuers() {
return new X509Certificate[0];
}
}
private static class AllowAll implements HostnameVerifier
{
#Override
public boolean verify(String arg0, SSLSession arg1) {
return true;
}
}
}
Hope this helps. I spent many hours trying to get it working (although I know well about SSL/TLS principles, security, X509, etc) ... This code is far from my taste for clean and lean java code. In addition I assumed that you do know how to build a keystore, supply all needed certificate chains, define a CAMEL route, etc. As such, it works with Camel 2.16 within a Spring Application Context, and has no other pretention than providing clues that would save you hours.

Bluemix force HTTPS on Spring Boot application

I have a Spring Boot application that is pushed on Bluemix as a CF app.
It works efficiently with the http protocol. However if i tried to force https, I get a 502 error.
I have:
#Configuration
class WebSecurityConfig extends WebSecurityConfigurerAdapter {
#Override
protected void configure(HttpSecurity http) throws Exception {
http.requiresChannel().anyRequest().requiresSecure();
//http.csrf().disable();
}
}
And I have an application.properties file with those entries:
server.ssl.key-store = classpath:**.jks
server.ssl.key-store-password = *******
server.ssl.key-password = ******
server.tomcat.remote_ip_header=x-forwarded-for
server.tomcat.protocol_header=x-forwarded-proto
I am aware that Bluemix performs SSL termination; in fact it sets correctly x-forwarded-proto and x-forwarded-for. I looked for solutions like 1 and 2 but without any luck.
I then tried with the following solution, as suggested in this article but a received a redirect loop insted:
#Bean
public TomcatEmbeddedServletContainerFactory tomcatEmbeddedServletContainerFactory(){
return new TomcatEmbeddedServletContainerFactory() {
#Override
protected void postProcessContext(Context context) {
SecurityConstraint securityConstraint = new SecurityConstraint();
securityConstraint.setUserConstraint("CONFIDENTIAL");
SecurityCollection collection = new SecurityCollection();
collection.addPattern("/*");
securityConstraint.addCollection(collection);
context.addConstraint(securityConstraint);
}
};
}
What did I miss in my approach? Many thanks for any tips/suggestions you may provide me
For the sake of the community, it would be good to see Rob's comment accepted as the answer. Rob, feel free to add your own answer if you would rather see that accepted instead.
Tomcat is not detecting the x-forwarded headers as being a trusted proxy. Try setting server.tomcat.internal-proxies=.* and logging.level.org.apache.catalina.valves=DEBUG

Handling multiple certificates in Netty's SSL Handler used in Play Framework 1.2.7

I have a Java Key Store where I store certificates for each of my customer's sub-domain. I am planning to use the server alias to differentiate between multiple customers in the key store as suggested here. Play framework 1.2.7 uses Netty's SslHandler to support SSL on the server-side. I tried implementing a custom SslHttpServerContextFactory that uses this solution.
import play.Play;
import javax.net.ssl.*;
import java.io.FileInputStream;
import java.net.InetAddress;
import java.net.Socket;
import java.security.KeyStore;
import java.security.Principal;
import java.security.PrivateKey;
import java.security.Security;
import java.security.cert.X509Certificate;
import java.util.Properties;
public class CustomSslHttpServerContextFactory {
private static final String PROTOCOL = "SSL";
private static final SSLContext SERVER_CONTEXT;
static {
String algorithm = Security.getProperty("ssl.KeyManagerFactory.algorithm");
if (algorithm == null) {
algorithm = "SunX509";
}
SSLContext serverContext = null;
KeyStore ks = null;
try {
final Properties p = Play.configuration;
// Try to load it from the keystore
ks = KeyStore.getInstance(p.getProperty("keystore.algorithm", "JKS"));
// Load the file from the conf
char[] certificatePassword = p.getProperty("keystore.password", "secret").toCharArray();
ks.load(new FileInputStream(Play.getFile(p.getProperty("keystore.file", "conf/certificate.jks"))),
certificatePassword);
// Set up key manager factory to use our key store
KeyManagerFactory kmf = KeyManagerFactory.getInstance(algorithm);
kmf.init(ks, certificatePassword);
TrustManagerFactory tmf = TrustManagerFactory.getInstance(algorithm);
tmf.init(ks);
final X509KeyManager origKm = (X509KeyManager) kmf.getKeyManagers()[0];
X509KeyManager km = new X509KeyManagerWrapper(origKm);
// Initialize the SSLContext to work with our key managers.
serverContext = SSLContext.getInstance(PROTOCOL);
serverContext.init(new KeyManager[]{km}, tmf.getTrustManagers(), null);
} catch (Exception e) {
throw new Error("Failed to initialize the server-side SSLContext", e);
}
SERVER_CONTEXT = serverContext;
}
public static SSLContext getServerContext() {
return SERVER_CONTEXT;
}
public static class X509KeyManagerWrapper implements X509KeyManager {
final X509KeyManager origKm;
public X509KeyManagerWrapper(X509KeyManager origKm) {
this.origKm = origKm;
}
public String chooseServerAlias(String keyType,
Principal[] issuers, Socket socket) {
InetAddress remoteAddress = socket.getInetAddress();
//TODO: Implement alias selection based on remoteAddress
return origKm.chooseServerAlias(keyType, issuers, socket);
}
#Override
public String chooseClientAlias(String[] keyType,
Principal[] issuers, Socket socket) {
return origKm.chooseClientAlias(keyType, issuers, socket);
}
#Override
public String[] getClientAliases(String s, Principal[] principals) {
return origKm.getClientAliases(s, principals);
}
#Override
public String[] getServerAliases(String s, Principal[] principals) {
return origKm.getServerAliases(s, principals);
}
#Override
public X509Certificate[] getCertificateChain(String s) {
return origKm.getCertificateChain(s);
}
#Override
public PrivateKey getPrivateKey(String s) {
return origKm.getPrivateKey(s);
}
}
}
But, this approach did not work for some reason. I get this message in my SSL debug log.
X509KeyManager passed to SSLContext.init(): need an X509ExtendedKeyManager for SSLEngine use
This is the SSL trace, which fails with "no cipher suites in common". Now, I switched the wrapper to:
public static class X509KeyManagerWrapper extends X509ExtendedKeyManager
With this change, I got rid of the warning, but I still see the same error as before "no cipher suites in common" and here is the SSL trace. I am not sure why the delegation of key manager won't work.
Some more information that may be useful in this context.
Netty uses javax.net.ssl.SSLEngine to support SSL in NIO server.
As per the recommendation in this bug report, it is intentional that X509ExtendedKeyManager must be used with an SSLEngine. So, the wrapper must extend X509ExtendedKeyManager.
This is hindering me to move further with the custom alias selection logic in X509KeyManagerWrapper. Any clues on what might be happening here? Is there any other way to implement this in Netty/Play? Appreciate any suggestions.
SSLEngine uses the chooseEngineServerAlias method to pick the certificate to use (in server mode) - not the chooseServerAlias method.
The default chooseEngineServerAlias implementation actually returns null, which is what causes the "no cipher suites in common" message - you need a certificate to know which cipher suites can be used (e.g. ECDSA can only be used for authentication if the certificate has an ECC public key, etc.) There are actually some cipher suites which can be used without a certificate, however, these are typically disabled as they are vulnerable to MITM attacks.
Therefore, you should also override chooseEngineServerAlias, and implement your logic to select the certificate based on the IP address there. As Netty only uses SSLEngine, what chooseServerAlias does doesn't matter - it'll never be called.
Java 8 also has support for server-side SNI, which allows you to use several certificates across many hostnames with a single IP address. Most web browsers support SNI - the notable exceptions are IE running on Windows XP and some old versions of Android, however, usage of these is declining. I have created a small example application demonstrating how to use SNI in Netty on GitHub. The core part of how it works is by overriding chooseEngineServerAlias - which should give you enough hints, even if you want to use the one certificate per IP address technique instead of SNI.
(I posted a similar answer to this on the Netty mailing list, where you also asked this question - however, my post seems to have not yet been approved, so I thought I'd answer here too so you can get an answer sooner.)

Can I self-host an HTTPS service in WCF without the certificate store and without using netsh http add sslcert?

I am attempting to host a service that serves up basic web content (HTML, javascript, json) using a WebHttpBinding with minimal administrator involvement.
Thus far I have been successful, the only admin priviledges necessary are at install time (register the http reservation for the service account and to create the service itself). However, now I am running into issues with SSL. Ideally I would like to support a certificate outside the windows certificate store. I found this article - http://www.codeproject.com/KB/WCF/wcfcertificates.aspx - which seems to indicate you can specify the certificate on the service host, however at runtime navigating a browser to https://localhost/Dev/MyService results in a 404.
[ServiceContract]
public interface IWhoAmIService
{
[OperationContract]
[WebInvoke(
Method = "GET",
UriTemplate = "/")]
Stream WhoAmI();
}
public class WhoAmIService : IWhoAmIService
{
public Stream WhoAmI()
{
string html = "<html><head><title>Hello, world!</title></head><body><p>Hello from {0}</p></body></html>";
html = string.Format(html, WindowsIdentity.GetCurrent().Name);
WebOperationContext.Current.OutgoingResponse.ContentType = "text/html";
return new MemoryStream(Encoding.UTF8.GetBytes(html));
}
}
static void Main(string[] args)
{
ServiceHost host = new ServiceHost(typeof(WhoAmIService), new Uri("https://localhost:443/Dev/WhoAmI"));
host.Credentials.ServiceCertificate.Certificate = new X509Certificate2(#"D:\dev\Server.pfx", "private");
WebHttpBehavior behvior = new WebHttpBehavior();
behvior.DefaultBodyStyle = WebMessageBodyStyle.Bare;
behvior.DefaultOutgoingResponseFormat = WebMessageFormat.Json;
behvior.AutomaticFormatSelectionEnabled = false;
WebHttpBinding secureBinding = new WebHttpBinding();
secureBinding.Security.Mode = WebHttpSecurityMode.Transport;
secureBinding.Security.Transport.ClientCredentialType = HttpClientCredentialType.None;
ServiceEndpoint secureEndpoint = host.AddServiceEndpoint(typeof(IWhoAmIService), secureBinding, "");
secureEndpoint.Behaviors.Add(behvior);
host.Open();
Console.WriteLine("Press enter to exit...");
Console.ReadLine();
host.Close();
}
If I change my binding security to none and the base uri to start with http, it serves up okay. This post seems to indicate that an additional command needs to be executed to register a certificate with a port with netsh (http://social.msdn.microsoft.com/Forums/en-US/wcf/thread/6907d765-7d4c-48e8-9e29-3ac5b4b9c405/). When I try this, it fails with some obscure error (1312).
C:\Windows\system32>netsh http add sslcert ipport=0.0.0.0:443 certhash=0b740a29f
29f2cc795bf4f8730b83f303f26a6d5 appid={00112233-4455-6677-8899-AABBCCDDEEFF}
SSL Certificate add failed, Error: 1312
A specified logon session does not exist. It may already have been terminated.
How can I host this service using HTTPS without the Windows Certificate Store?
It is not possible. HTTPS is provided on OS level (http.sys kernel driver) - it is the same as providing HTTP reservation and OS level demands certificate in certificate store. You must use netsh to assign the certificate to selected port and allow accessing the private key.
The article uses certificates from files because it doesn't use HTTPS. It uses message security and message security is not possible (unless you develop your own non-interoperable) with REST services and webHttpBinding.
The only way to make this work with HTTPS is not using built-in HTTP processing dependent on http.sys = you will either have to implement whole HTTP yourselves and prepare new HTTP channel for WCF or you will have to find such implementation.