I am trying to setup Traefik on Kubernetes with Let's Encrypt enabled. I managed yesterday to retrieve the first SSL certificated from Let's Encrypt but am a little bit stuck on how to store the SSL certificates.
I am able to create a Volume to store the Traefik certificates but that would mean that I am limited to a single replica (when having multiple replicas am I unable to retrieve a certificate since the validation goes wrong most of the times due to that the volume is not shared).
I read that Traefik is able to use something like Consul but I am wondering if I have to setup/run a complete Consul cluster to just store the fetched certificates etc.?
You can store the certificate in a kubernetes secret and you reference to this secret in your ingress.
spec:
tls:
- secretName: testsecret
The secret has to be in same namespace the ingress is running in.
See also https://docs.traefik.io/user-guide/kubernetes/#add-a-tls-certificate-to-the-ingress
You can set up the ingress with controller and apply for the SSL certificate of let's encrypt.
You can use cluster issuer to manage the SSL certificates and store that tls certificate on ingress.you can also use different ingress controllers like nginx also can use service mess istio.
For more details you can check : https://docs.traefik.io/user-guide/kubernetes/
Related
I am using an Ingress using Google-managed SSL certs mostly similar to what is described here:
https://cloud.google.com/kubernetes-engine/docs/how-to/managed-certs#setting_up_a_google-managed_certificate
However my backend service is a grpc service that is using HTTP2. According to the same documentation if I am using HTTP2 my backend needs to be "configured with SSL".
This sounds like I need a separate set of certificates for my backend service to configure it with SSL.
Is there a way to use the same Google managed certs here as well?
What are my other options here? I am using, Google managed certs for the Ingress not to manage any certs on my own, if I then use self signed certificates for my service, that kind of defeats the purpose.
i don't think it's required to create SSL for the backend services if you are terminating the HTTPS at LB level. You can attach your certs to at LB level and the backed-end will be HTTPS > HTTP.
You might need to create SSL/TLS new cert in case there is diff version ssl-protocols: TLSv1.2 TLSv1.3, Cipher set in your ingress controller configmap which you are using Nginx ingress controller, Kong etc.
If you are looking for End to End HTTPS traffic definitely you need to create a cert for the backend service.
You can also create/manage the Managed certificate or Custom cert with Cert manager the K8s secret and mount to deployment which will be used further by the service, in that case, no need to manage or create the certs. Ingress will passthrough the HTTPS request to service directly.
In this case, it will be an end-to-end HTTPS setup.
Update :
Note: To ensure the load balancer can make a correct HTTP2 request to
your backend, your backend must be configured with SSL. For more
information on what types of certificates are accepted, see Encryption
from the load balancer to the backends ." end to end tls seems to be a
requirement for HTTP2
This is my site https://findmeip.com it's running on HTTP2 and terminating the SSL/TLS at the Nginx level only.
Definitely, it's good to go with the suggested practice so you can use the ESP option from the Google, setting GKE ingress + ESP + grpc stack.
https://cloud.google.com/endpoints/docs/openapi/specify-proxy-startup-options?hl=tr
If not want to use ESP check above suggested :
You can Mount Managed certificate to
deployment which will be used further by the service, in that case, no
need to manage or create the certs. In other words, cert-manager will create/manage/re-new SSL/TLS on behalf of you in K8s secret which will used by service.
Google Managed Certificates can only be used for the frontend portion of the load balancer (aka client to LB). If you need encryption from the LB to the backends you will have use self-signed certificates or some other way to store said certificates on GKE as secrets and configuring the Ingress to connect to the backend using these secrets.
Like this https://cloud.google.com/kubernetes-engine/docs/concepts/ingress-xlb#setting_up_https_tls_between_client_and_load_balancer
I'd like to deploy an ldap server on my kubernetes cluster. The server itself is up and running, but I'd like to enable SSL encryption for it as well.
I already have cert-manager up and running and I also use a multitude of SSL certificates with my ingresses with my HTTP traffic. It would be really nice if I could just use a CertificateRequest with my ldap server as well, managed and updated by cert-manager.
My problem is I have no idea how to mount a Certificate to my kubernetes pod. I know that cert-manager creates a secret and puts the certificate data in it. The problem with that is I have no idea of the validity of that certificate this way, and can't remount/reapply the new certificate.
Has anybody done anything like this? Is there a non-hacky way to incorporate ingresses to terminate SSL encryption?
Sorry for my bad English but I don't know how to solve my problem.
So...
Introduction:
I have 2 microservices (I called them gRPCClient and gRPCServer, although it doesn’t matter what exactly). They need to communicate via TLS. Without Kubernets, everything is quite simple. I create my CA via cfssl in a docker container, then I get the root certificate from CA and I put it in trust for my grpc applications (I do this in Dockerfile), so that any certificate signed by my CA passes the test.
Now Kubernetes is included in the game. I'm playing locally with minikube. I create local cluster "minikube start" on mac (maybe this is important, I don’t know ...)
Problem:
How will this flow work with the Kubernetes? As I understand it, there is already a CA inside the Kubernetes (correct me if this is not so). I read many articles, but I really didn’t understand anything. I tried the examples from this article https://kubernetes.io/docs/tasks/tls/managing-tls-in-a-cluster/
Step by step:
Create a signature request
cat <<EOF | cfssl genkey - | cfssljson -bare server
{
"hosts": [
"my-svc.my-namespace.svc.cluster.local",
"my-pod.my-namespace.pod.cluster.local",
"192.0.2.24",
"10.0.34.2"
],
"CN": "my-pod.my-namespace.pod.cluster.local",
"key": {
"algo": "ecdsa",
"size": 256
}
}
EOF
The first thing I did not understand was the hosts. For example, my-svc.my-namespace.svc.cluster.local is the full name of my service? (I mean the service in Kubernetes as kind: Service). I have it in the namespace "dev" and its name is user-app-sesrvice. Should I specify user-app-sesrvice.dev.svc.cluster.local then? or it just user-app-sesrvice. Or is there some kind of command to get the full name of the service? 192.0.2.24 - as I understand it, is the IP of service, it is also unclear whether it is mandatory to specify it or is it possible only the name of the service? What if I have clusterIP: None installed, then I don't have IP for it. my-pod.my-namespace.pod.cluster.local - Should I specify this? If I have several pods, should I list them all? Then the problem is in the dynamics, because the pods are recreated, deleted and added, and I need to send a new request for signature each time. The same questions that I asked about service including some parts "my-pod" and "namespace"? Is it possible to see the full name of the pod with all this data. 10.0.34.2 - pods' IP. The same question about pods' IP.
I tried to specify the host and CN as name of my service name "user-app-service" (as if I was working without a Kubernetes). I created a signature and a key. Then all the steps, created a request object for signature in the Kubernetes
cat <<EOF | kubectl apply -f -
apiVersion: certificates.k8s.io/v1beta1
kind: CertificateSigningRequest
metadata:
name: my-svc.my-namespace
spec:
request: $(cat server.csr | base64 | tr -d '\n')
usages:
- digital signature
- key encipherment
- server auth
EOF
Then I made it and I received a certificate
Further, based on security, I need to store the key and a certificate in secrets and then get it in the container (for the purposes of the test, I just put them in the container in the docker file, hard-coded), this is in the gRPC server. I run the deployment and created a client on golang, specifying config: = &tls.Config{} in the code so that it would pull the trusted certificates from the system itself, I thought that the Kubernetes has a CA, but did not find how to get its certificate in the docs. I thought the Kubernetes adds them to all the containers himself. But I got the error Unavailable desc = connection error: desc = "transport: authentication handshake failed: x509: certificate signed by unknown authority". How should all this work? Where can I get a CA certificate from a Kubernetes? And then, do I need to add it to each container with my hands in dockerfile? or is this not the right tactic and is there some kind of automation from the Kubernetes?
I found another way, this is to try to deploy cfssl https://hub.docker.com/r/cfssl/cfssl/ on the Kubernetes and already work with it, like there was no Kubernetes (I have not tried this method yet)
How to put all this into a working system, what options to use and why? Maybe there are some full articles. I wrote a lot, but I hope it’s clear. I really need the help.
I am going to break down my answer into a couple of parts:
Kubernetes Services and DNS Discovery
In general, it is recommended to put a Service in front of a Deployment that manages pods in Kubernetes.
The Service creates a stable DNS and IP endpoint for pods that may be deleted and be assigned a different
IP address when recreated. DNS service discovery is automatically enabled with a ClusterIP type service and
is in the format: <service name>.<kubernetes namespace>.svc.<cluster domain> where cluster domain is usually
cluster.local. This means that we can use the autocreated DNS and assigned ClusterIP in our altnames for our
certificate.
Kubernetes Internal CA
Kubernetes does have an internal CA along with API methods to post CSRs and have those CSRs signed
by the CA however I would not use the internal CA for securing microservices. The internal CA is
primarily used by the kubelet and other internal cluster processes to authenticate to the Kubernetes
API server. There is no functionality for autorenewal and I think the cert will always be signed for 30 days.
Kubernetes-native Certificate Management
You can install and use cert-manager to have the cluster automatically create and manage certificates
for you using custom resources. They have excellent examples on their website so I would encourage you
to check that out if it is of interest. You should be able to use the CA Issuer Type and create
Certificate Resources that will create a certificate as a Kubernetes Secret. For the altnames, refer
to the below certificate generation steps in the manual section of my response.
Manually Create and Deploy Certificates
You should be able to achieve they same result using your "without Kubernetes" approach using cfssl:
generate CA using cfssl
add CA as trusted in image (using your Dockerfile approach)
create Kubernetes Service (for example purposes I will use kubectl create)
$ kubectl create service clusterip grpcserver --tcp=8000
describe the created Kubernetes Service, note IP will most likely be different in your case
$ kubectl describe service/grpcserver
Name: grpcserver
Namespace: default
Labels: app=grpcserver
Annotations: <none>
Selector: app=grpcserver
Type: ClusterIP
IP: 10.108.125.158
Port: 8000 8000/TCP
TargetPort: 8000/TCP
Endpoints: <none>
Session Affinity: None
Events: <none>
generate certificate for gRPCServer with a CN of grpcserver.default.svc.cluster.local the following altnames:
grpcserver
grpcserver.default.svc
grpcserver.default.svc.cluster.local
10.108.125.158
generate the client certificate with cfssl
put both certificates into Secret objects
kubectl create secret tls server --cert=server.pem --key=server.key
kubectl create secret tls client --cert=client.pem --key=client.key
mount the secret into the podspec
There is a lot of boilerplate work that you need to do with this bespoke approach. If you have an option I would suggest exploring service mesh such as istio or linkerd to secure communication between micro-services using TLS in kubernetes.
I use cert-manager to generate TLS certificates for my application on Kubernetes with Let's Encrypt.
It is running and I can see "ca.crt", "tls.crt" and "tsl.key" inside the container of my application (in /etc/letsencrypt/).
But "ca.crt" is empty, and the application complains about it (Error: Unable to load CA certificates. Check cafile "/etc/letsencrypt/ca.crt"). The two other files look like normal certificates.
What does that mean?
With cert-manager you have to use the nginx-ingress controller which will work as expose point.
ingress nginx controller will create one load balancer and you can setup your application tls certificate there.
There is nothing regarding certificate inside the pod of cert-manager.
so setup nginx ingress with cert-manager that will help to manage the tls certificate. that certificate will be stored in kubernetes secret.
Please follow this guide for more details:
https://www.digitalocean.com/community/tutorials/how-to-set-up-an-nginx-ingress-with-cert-manager-on-digitalocean-kubernetes
I noticed this:
$ kubectl describe certificate iot-mysmartliving -n mqtt
...
Status:
Conditions:
...
Message: Certificate issuance in progress. Temporary certificate issued.
and a related line in the docs:
https://docs.cert-manager.io/en/latest/tasks/issuing-certificates/index.html?highlight=gce#temporary-certificates-whilst-issuing
They explain that the two existing certificates are generated for some compatibility, but they are not valid until the issuer has done its work.
So that suggests that the issuer is not properly set up.
Edit: yes it was. The DNS challenge was failing, the debug line that helped was
kubectl describe challenge --all-namespaces=true
More generally,
kubectl describe clusterissuer,certificate,order,challenge --all-namespaces=true
According to the documentation, cafile is for something else (trusted root certificates), and it would probably be more correct to use capath /etc/ssl/certs on most systems.
You can follow this guide if you have Windows Operating System:
tls.
Article is about how to enable Mosquitto and clients to use the TLS protocol.
Establishing a secure TLS connection to the Mosquitto broker requires key and certificate files. Creating all these files with the correct settings is not the easiest thing, but is rewarded with a secure way to communicate with the MQTT broker.
If you want to use TLS certificates you've generated using the Let's Encrypt service.
You need to be aware that current versions of mosquitto never update listener settings when running, so when you regenerate the server certificates you will need to completely restart the broker.
If you use DigitalOcean Kubernetes try to follow this instruction: ca-ninx, you can use Cert-Manager and ingress nginx controller, they will work like certbot.
Another solution is to create the certificate locally on your machine and then upload it to kubernetes secret and use secret on ingress.
I want to generate SSL certificate automatically on Kubernetes. My site is already up and running but it is on HTTP:// not on HTTPS:// so which is the best way to provide them and generate the SSL certificate automatically.
I am new to k8s and learning. If the first time I manually install it then also the next time if the pod gets recreated the certificate will be deleted.
So suggest me easy way to manage certificate manager on K8s
A few of options I can think of:
You can try the certificate manager with Letsencrypt certificates.
You can try an a Kubernetes Ingress with an ingress controller like nginx also with Letsencrypt and this described here.
You can try a Traefik ingress controller also with Letsencrypt.