HAPROXY ingress controller setup using mTLS with configmap with just the ingress load balancer because it's ssl offloaded. No need for backend check - ssl-certificate

I was able to achieve ssl offloading with Haproxy. So great product and appreciate that capability!
With that said, I need to doing mutual TLS but am a little confused on how that will work with the ingress controller configmap.
Going through this reference i've created a client cert, intermediate cert and root cert.
To note, I am terminating the ssl cert (which is from letsencrpt) on the load balancer currently.
However, the client cert and org CA are different than the lesencrypt tls/ssl cert that I have assigned as the SSL now; does that matter?
So, the first question I would have is does the ssl-certificate have to be set to the CA that will sign the client and server certs or can I just use the new ones I created in the instruction.
Setting up the configmap.
This is the part i'm confused on.
You can setup server-ca and server-crt but I don't think that applys here because after the ssl offloading there is nothing meant to be checked. However, I do want mTLS via the ssl termination.
So there is an configuration client-ca
Sets the client certificate authority enabling HAProxy to check clients certificate (TLS authentication), thus enabling client mTLS.
NB, ssl-offloading should be enabled for TLS authentication to work.
The client in this case being the actual client I want which is the device/frontend. Not the loadbalancer acting as a client to the backend server.
When I look at how this is setup:
frontend mysite
bind 192.168.56.20:80
bind 192.168.56.20:443 ssl crt /etc/haproxy/certs/ssl.crt verify required ca-file /etc/haproxy/certs/intermediate-ca.crt ca-verify-file /etc/haproxy/certs/root-ca.crt
http-request redirect scheme https unless { ssl_fc }
default_backend apiservers
Is it possible to do the same with the controller configmap as what is listed here below? There's a lot more going on that what I am seeing as flags / configurations that are in this methodology of applying client mTLS. Is there a way to achieve this in kubernetes without configmap?
The ssl parameter enables SSL termination for this listener. The crt parameter identifies the location of the PEM-formatted SSL certificate. This certificate should contain both the public certificate and private key.
You can restrict who can access your application by giving trusted clients a certificate that they must present when connecting. HAProxy will check for this if you add a verify required parameter to the bind line, as shown:
the ssl argument enables HTTPS
the crt argument specifies the server SSL certificate, which you will typically obtain from a certificate provider like Let’s Encrypt
the verify required argument requires clients to send a client certificate
the ca-file argument specifies the intermediate certificate with which we will verify that the client’s certificate has been signed with our organization’s CA
the ca-verify-file argument (introduced in HAProxy 2.2) includes the root CA certificate, allowing HAProxy to send a shorter list of CAs to the client in the SERVER HELLO message that will be used for verification, but keeping upper level CAs, such as the root, out of that list. HAProxy requires the root CA to be set with this argument or else included in the intermediate-ca.crt file (compatibility with older versions of HAProxy).
Also, my reasoning for now wanting to use letsencrypt and rather a private CA is because I can't renew device certificates every 60 - 90 days. That would not be efficient. In this case, and please let me know otherwise, I think it better to use either a real key/cert provider or in development testing utilize the openssl certs like in the HAProxy instruction.
It's odd but you really have to think about what a "client" is with these abstractions because I would never use this for a normal web page login but rather the server to server communication and in that sense this server is a client to this server. Or in my case this device is a client to this loadbalancer.

Related

Allow kubernetes storageclass resturl HTTPS with self-signed certificate

I'm currently trying to setup GlusterFS integration for a Kubernetes cluster. Volume provisioning is done with Heketi.
GlusterFS-cluster has a pool of 3 VMs
1st node has Heketi server and client configured. Heketi API is secured with a self-signed certificate OpenSSL and can be accessed.
e.g. curl https://heketinodeip:8080/hello -k
returns the expected response.
StorageClass definition sets the "resturl" to Heketi API https://heketinodeip:8080
When storageclass was created successfully and I try to create a PVC, this fails:
"x509: certificate signed by unknown authority"
This is expected, as ususally one has to allow this insecure HTTPS-connection or explicitly import the issuer CA (e.g. a file simply containing the pem-String)
But: How is this done for Kubernetes? How do I allow this insecure connection to Heketi from Kubernetes, allowing insecure self-signed cert HTTPS or where/how do I import a CA?
It is not an DNS/IP problem, this was resolved with correct subjectAltName settings.
(seems that everybody is using Heketi, and it seems to be still a standard usecase for GlusterFS integration, but always without SSL, if connected to Kubernetes)
Thank you!
To skip verification of server cert, caller just need specify InsecureSkipVerify: true. Refer this github issue for more information (https://github.com/heketi/heketi/issues/1467)
In this page, they have specified a way to use self signed certificate. Not explained thoroughly but still can be useful (https://github.com/gluster/gluster-kubernetes/blob/master/docs/design/tls-security.md#self-signed-keys).

Confusion about HTTPS --> How is SSL handshake happing

I've always been an end consumer of HTTPS and have never really understood it that well but am looking to change that.
I am calling a RESTful web service over HTTPS. For example...
curl -X GET \
https://myCompanydns/rest/connect/v1.4/myEndpoint
With all my requests I send a basic authentication header i.e a username and password.
When I make these calls via my application I was expecting to have to add a certificate into like a jks (which I've had to do in the past) but on this occasion I've found that I can call the HTTPS web service without that.
For HTTPS to work I believe there is an SSL handshake? How is that happening successfully is this scenario without a jks?
Again, sorry for this beginner type question.
When doing a https://... request the client needs to verify that the servers certificate is the expected one - and not some man in the middle. This is done (among other things) by making sure that the servers certificate was issued by a trusted certificate authority (CA). Which CA is trusted is setup in the local trust store (i.e. local to the client). In the above call where no explicit trust store is given curl is using its default trust store. In the case where you've explicitly gave a jks you've provided the application with a specific trust store it should use.
For more on how the server certificates gets validated see SSL Certificate framework 101: How does the browser actually verify the validity of a given server certificate?.

How to setup IIS to verify a client certificate and pass it to the backend as an http header?

I've read quite a few articles on the client certificate authentication in IIS but they mostly talk about the Required mode for the client certificate. In this case, the authentication is fully done on the web server side and the user is redirected to the error page in case the certificate is missing or invalid. What I need is to configure IIS to check the client certificate if it exists and pass the results as http headers to my backend. It can be easily done with nginx or apache web servers. The client authentication can be set up as optional. Then, the web server checks the certificate and pass the results of the check as SSL_CLIENT_CERT (PEM representation of the certificate) and SSL_CLIENT_VERIFY (the result of the check - SUCCESS, NONE,...) headers to the application backend. So, on the backend part, I can read the headers and use the values to do the actual authentication -- find a user in the database, issue an auth token. Is it possible in IIS? If yes, is there any documentation on how to bypass the certificate and the check state as http headers?
After a while, I'm writing an answer to my question.
IIS has to be set up with ARR extension to act as a reverse proxy. Then, the client certificate authentication can be enabled for the default web site. Here are several links to instructions how to achieve it:
https://blogs.msdn.microsoft.com/benjaminperkins/2014/06/02/configure-application-request-routing-arr-with-client-certificates/
https://blogs.msdn.microsoft.com/asiatech/2014/01/27/configuring-arr-with-client-certificate/
and couple of links to very solid explanations of the SSL/TLS handshake and certificates in general:
https://blogs.msdn.microsoft.com/kaushal/2013/08/02/ssl-handshake-and-https-bindings-on-iis/
https://blogs.msdn.microsoft.com/kaushal/2013/01/09/self-signed-root-ca-and-intermediate-ca-certificates/
https://blogs.msdn.microsoft.com/kaushal/2015/05/27/client-certificate-authentication-part-1/
Basically, the Application Request Routing (ARR) extension should be enabled for the IIS to act as a proxy. Then, you set up proxy bypass routes. Next, you need to set up a valid server certificate for the IIS server and use it in the Default Web Site https bindings. The certificate should be issued by a CA which certificate should be placed into the Trusted Root Certification Authorities and Intermediate Certification Authorities of the Local Computer. Then, you should require SSL in the SSL settings of the Default Web Site with the client certificates setting equal to Accept. In this case, any client that connects to the web server will be asked for a valid client certificate that has been issued by the same CA as the server certificate.
Actually, IIS sends a list of distinguished names of root issuers that are trusted by the web server to the client browser. The browser finds an intersection of this names with client trusted certificate issuers and looks for valid certificates that have been issued by the issuers in the intersection. After that, the user selects one of them (or none) and the selected certificate is checked against the CA certificate. If the certificate passes the check the request is "redirected" to the backend application with the certificate in the X-ARR-ClientCert header. The name of the header can be changed in the Server -> Configuration Editor -> system.webServer/proxy -> clientCertHeaderName IIS parameter. In case the user selects (or has) none of the required certificates, the request is "redirected" to the app backend without the header.
Seems, there is no need in the SSL_CLIENT_VERIFY header with the state of the check at all. If the certificate is valid it is passed in the header. If the certificate is missing the header is empty. If the certificate is provided by the client but is invalid then the request fails and is not "redirected" to the backend app server at all. Seems, it is a rare case but I have an example.
Imagine, a server certificate is issued by a CA with a distinguished name XXX, and there is a client certificate YYY (on the client computer) that is issued by a CA with the distinguished name XXX but those CAs are not the same (one or both of them are self-signed). In this case, the YYY certificate is present in the certificate selection dialog that is shown by a browser but the certificate doesn't pass the further validation against the real web server CA.
That's it. Also, seems IIS has no way to require (or accept) a client certificate for some app endpoints (addresses) only. I haven't found any other option apart from enabling in for the whole web site.
IIS ARR can also be configured with a client certificate for the backend or upstream server. In this case users connecting to this IIS do not need to provide the certificate as it will be attached by IIS automatically.
This configuration is available on IIS server level only.

simple Akka ssl encryption

There are several questions on stackoverflow regarding Akka, SSL and certificate management to enable secure (encrypted) peer to peer communication between Akka actors.
The Akka documentation on remoting (http://doc.akka.io/docs/akka/current/scala/remoting.html)
points readers to this resource as an example of how to Generate X.509 Certificates.
http://typesafehub.github.io/ssl-config/CertificateGeneration.html#generating-a-server-ca
Since the actors are running on internal servers, the Generation of a server CA for example.com (or really any DNS name) seems unrelated.
Most servers (for example EC2 instances running on Amazon Web Services) will be run in a VPC and the initial Akka remotes will be private IP addresses like
remote = "akka.tcp://sampleActorSystem#172.16.0.10:2553"
My understanding, is that it should be possible to create a self signed certificate and generate a trust store that all peers share.
As more Akka nodes are brought online, they should (I assume) be able to use the same self signed certificate and trust store used by all other peers. I also assume, there is no need to trust all peers with an ever growing list of certificates, even if you don't have a CA, since the trust store would validate that certificate, and avoid man in the middle attacks.
The ideal solution, and hope - is that it possible to generate a single self signed certificate, without the CA steps, a single trust store file, and share it among any combination of Akka remotes / (both the client calling the remote and the remote, i.e. all peers)
There must be a simple to follow process to generate certificates for simple internal encryption and client authentication (just trust all peers the same)
Question: can these all be the same file on every peer, which will ensure they are talking to trusted clients, and enable encryption?
key-store = "/example/path/to/mykeystore.jks"
trust-store = "/example/path/to/mytruststore.jks"
Question: Are X.509 instructions linked above overkill - Is there a simple self signed / trust store approach without the CA steps? Specifically for internal IP addresses only (no DNS) and without an ever increasing web of IP addresses in a cert, since servers could autoscale up and down.
First, I have to admit that I do not know Akka, but I can give you the guidelines of identification with X509 certificates in the SSL protocol.
akka server configuration require a SSL certificate bound to a hostname
You will need a server with a DNS hostname assigned, for hostname verification. In this example, we assume the hostname is example.com.
A SSL certificate can be bound to a DNS name or an IP (not usual). In order for the client verification to be correct, it must correspond to the IP / hostname of the server
AKKA requires a certificate for each server, issued by a common CA
CA
- server1: server1.yourdomain.com (or IP1)
- server2: server2.yourdomain.com (or IP2)
To simplify server deployment, you can use a wildcard *.yourdomain.com
CA
- server1: *.yourdomain.com
- server2: *.yourdomain.com
On the client side you need to configure a truststore including the public key of the CA certificate in the JKS. The client will trust in any certificate issued by this CA.
In the schema you have described I think you do not need the keystore. It is needed when you also want to identify the client with a certificate. The SSL encrypted channel will be stablished in both cases.
If you do not have a domain name like yourdomain.com and you want to use internal IP, I suggest to issue a certificate for each server and bound it to the IP address.
Depending on how akka is verifying the server certificate, it would be possible to use a unique self-signed certificate for all servers. Akka probably relies trust configuration to JVM defaults. If you include a self-signed certificate in the truststore (not the CA), the ssl socket factory will trust connections presenting this certificate, even if it is expired or if the hostname of the server and the certificate will not match. I do not recomend it

Is it possible to automatically select correct client side certificate?

I have configured an Apache httpd website with SSL client side certificates so that only users who have installed the correct certificate in their web browsers can access the website.
If there is only one client side certificate installed the web browser will automatically select it (it is not the default, but it can be configured somewhere in the settings dialog). But if a user has more than one certificate installed, the web browser presents a list of certificates and the user has to pick the right one to continue.
The question is: Is there a way to configure httpd to send a hint so that the web browser can automatically select the required certificate?
The SSL (TLS) protocol only allows the server to specify two constraints on the client certificate:
The type of certificate (RSA, DSA, etc.)
The trusted certificate authorities (CAs) that signed the client certificate
You can use "openssl s_client" to see which CAs your Apache server trusts for client certs. I do not know how to configure Apache to change that list (sorry), but I bet there is a way. So if you can limit the list to (say) your own organization's CA alone, then you will have done all you can to allow a Web browser to select the client cert automatically.
As Eugene said, whether the browser actually does so is up to the particular browser.
I'd say that as selection of the certificate is a client-side task, there's no definite way to force the client use this or that certificate from the server side.
In addition to what #Nemo and #Eugene said, by default, Apache Httpd will send the list of CAs it gets from its SSLCACertificateFile or SSLCACertificatePath configuration directives.
However, you can force it to send a different list in certificate_authorities using the SSLCADNRequestFile or SSLCADNRequestPath directives and pointing them to another set of certificates. Only the Subject DN of these certificates is used (and send in the list). If you want to force certain names, you can even self-sign these certificates with whichever name you want. I've tried this (in conjunction with SSLVerifyClient optional_no_ca, and you can get clients to send certificates for CA certificates that the server doesn't actually have. (This isn't necessarily useful, but it works.)