I am using Traefik like a reverse proxy on my Kubernetes cluster and I'm looking a way to use multiple SSL Certificates.
According to the doc, is not possible to use multiple TLSStore, I must use only the default store.
For example, I have two domains app1.com and app2.com and two certificates for *.app1.com and *.app2.com known by Traefik using secret app1-com-ssl and app2-com-ssl, is it possible to store these two certificates in the same default TLSStore?
Here my TLSStore definition:
apiVersion: traefik.containo.us/v1alpha1
kind: TLSStore
metadata:
name: default
namespace: app1
spec:
defaultCertificate:
secretName: app1-com-ssl
Here my IngressRoute for app1:
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: routes
namespace: app1
spec:
entryPoints:
- websecure
- web
routes:
- match: Host(`test.app1.com`)
kind: Rule
services:
- name: test-service
port: 8080
tls:
store:
name: default
Here my IngressRoute for app2:
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: routes
namespace: app2
spec:
entryPoints:
- websecure
- web
routes:
- match: Host(`test.app2.com`)
kind: Rule
services:
- name: test-service
port: 8080
tls:
store:
name: default
Thanks!
Good question. The TLSStore doesn't currently support multiple certificates it just supports a 'default certificate'. And as you have already seen you can only specify a 'default' TLSStore.
The Traefik configs do allow you to specify multiple certificates for the 'default' TLSStore (But not the K8s CRD) I have opened a feature request for the Traekik TLSStore CRD to add support for multiple certificates.
✌️
Related
I have a Kubernetes (v1.25.2) cluster running with cert-manager 1.11.0 and Traefik 2.9.6.
For some services I want Let's Encrypt to auto sign certificates. For some reason, it feels nicer, to use IngressRoute instead of Ingress. I just can't get the IngressRoute to create the certificate.
Now, I have the a ClusterIssuer:
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: my#email.com
privateKeySecretRef:
name: letsencrypt-prod
solvers:
- http01:
ingress:
class: traefik
and, working, corresponding Ingress:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: myapp-name-websecure
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
ingressClassName: traefik
rules:
- host: my.host.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: linkingservice
port:
number: 80
tls:
- hosts:
- my.host.com
secretName: some-secret-name-tls
This works, nice. Instead, with IngressRoute the base resource is this:
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: myapp-other-name-websecure
spec:
entryPoints:
- websecure
routes:
- match: Host(`other.host.com`)
kind: Rule
services:
- name: linkingservice
port: 80
tls:
# certResolver: ??? # resolve what? Doesn't link with the ClusterIssuer
# issuerRef: ??? # doesn't exist (anymore)
Now, I've tried to:
just as for the Ingress to use the annotations: cert-manager.io/cluster-issuer: letsencrypt-prod. Which is being ignored
use the tls.certResolver, which doesn't work, because it doesn't exist. Should I create one? I expect The ClusterIssuer to create the certificate and secret, just as it does for Ingress.
I also saw the issuerRef as option in the tls section, but that doesn't appear to exist.
I thought I read that the IngressRoute is like a layer on top of the k8s default Ingress, so it should be something logical/similar
FYI: the ClusterIssuer and Ingress will also work for Nginx, when you replace the solvers.http01.ingress.class with nginx, likewise for the Ingress's spec.ingressClassName. (perhaps also without, but I can't test)
Now, I did find a way, but still feels like more work than should be necessary. The thing here is to create the Certificate and link that to the ClusterIssuer, that certificate then creates a Secret. This secret needs to be added to the spec.tls.secretName, like:
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: my.host.com-cert
spec:
secretName: my.host.com-secret
issuerRef:
name: letsencrypt-prod
kind: ClusterIssuer
dnsNames:
- my.host.com
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
spec:
...
tls:
secretName: my.host.com-secret
I've also tried for, certResolver. But the CertificateResolver from cert-manager was discontinued in v0.15.0; alternatively the CertificateRequest, but this was also discontinued in cert-manager version 0.9.0. The suggested way seems to be the previous described one.
I revisited the Traefik deployment values.yaml where I found there is a field with certResolvers, from their file:
certResolvers:
letsencrypt:
# for challenge options cf. https://doc.traefik.io/traefik/https/acme/
email: email#example.com
dnsChallenge:
# also add the provider's required configuration under env
# or expand then from secrets/configmaps with envfrom
# cf. https://doc.traefik.io/traefik/https/acme/#providers
provider: digitalocean
# add futher options for the dns challenge as needed
# cf. https://doc.traefik.io/traefik/https/acme/#dnschallenge
delayBeforeCheck: 30
resolvers:
- 1.1.1.1
- 8.8.8.8
tlsChallenge: true
httpChallenge:
entryPoint: "web"
# It has to match the path with a persistent volume
storage: /data/acme.json
Which makes me wonder. If you set this up, then probably the:
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
spec:
tls:
certResolver: letsencrypt
can work
I have set up a TLS kubernetes secret which is operating normally on the non-traefik workloads.
When I try to use it to one of my ingress routes the certificate seems to have not been applied and the "TRAEFIK DEFAULT CERT" is assigned.
Below is my IngressRoute .yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: example
namespace: example-ns
spec:
entryPoints:
- websecure
routes:
- match: Host(`example.com`)
kind: Rule
services:
- name: example-svc
port: 9090
tls:
secretName: example-tls
Found it!
Indeed issue was related with passing the TLS Store to both namespaces and traefik pod logged the below:
level=error msg="Default TLS Stores defined in multiple namespaces: [example traefik]" providerName=kubernetescrd
I removed the TLS store from the "example" namespace and everything worked fine!
I kind of new on Kubernetes stuff and I'm trying to improve one current system we have here.
The Application is developed using Spring Boot and until now it was using HTTP (Port 8080) without any encryption. The system requirement is to enable e2e-encryption for all Data In-Transit. So here is the problem.
Currently, we have GCE Ingress with TLS enabled using Let's Encrypt to provide the Certificates on Cluster entrance. This is working fine. Our Ingress has some Path Rules to redirect the traffic to the correct microservice and those microservices are not using TLS on the communication.
I managed to create a Self-Signed certificate and embedded it inside the WAR and this is working on the Local machine just fine (using certificate validation disabled). When I deploy this on GKE, the GCP Health Check and Kubernetes Probes are not working at all (I can't see any communication attempt on the Application logs).
When I try to configure the Backend and Health Check on GCP changing both to HTTPS, they don't show any error, but after some time they quietly switch back to HTTP.
Here are my YAML files:
admin-service.yaml
---
apiVersion: v1
kind: Service
metadata:
name: admin-service
namespace: default
spec:
type: NodePort
selector:
app: admin
ports:
- port: 443
targetPort: 8443
name: https
protocol: TCP
admin-deployment.yaml
---
apiVersion: "apps/v1"
kind: "Deployment"
metadata:
name: "admin"
namespace: "default"
labels:
app: "admin"
spec:
replicas: 1
selector:
matchLabels:
app: "admin"
template:
metadata:
labels:
app: "admin"
spec:
containers:
- name: "backend-admin"
image: "gcr.io/my-project/backend-admin:X.Y.Z-SNAPSHOT"
livenessProbe:
httpGet:
path: /actuator/health/liveness
port: 8443
scheme: HTTPS
initialDelaySeconds: 8
periodSeconds: 30
readinessProbe:
httpGet:
path: /actuator/health/readiness
port: 8443
scheme: HTTPS
initialDelaySeconds: 8
periodSeconds: 30
env:
- name: "FIREBASE_PROJECT_ID"
valueFrom:
configMapKeyRef:
key: "FIREBASE_PROJECT_ID"
name: "service-config"
---
apiVersion: "autoscaling/v2beta1"
kind: "HorizontalPodAutoscaler"
metadata:
name: "admin-etu-vk1a"
namespace: "default"
labels:
app: "admin"
spec:
scaleTargetRef:
kind: "Deployment"
name: "admin"
apiVersion: "apps/v1"
minReplicas: 1
maxReplicas: 3
metrics:
- type: "Resource"
resource:
name: "cpu"
targetAverageUtilization: 80
ingress.yaml
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: my-ingress
annotations:
kubernetes.io/ingress.global-static-ip-name: my-ingress-addr
kubernetes.io/ingress.class: "gce"
kubernetes.io/tls-acme: "true"
cert-manager.io/cluster-issuer: "letsencrypt-prod"
acme.cert-manager.io/http01-edit-in-place: "true"
kubernetes.io/ingress.allow-http: "false"
spec:
tls:
- hosts:
- my-domain.com
secretName: mydomain-com-tls
rules:
- host: my-domain.com
http:
paths:
- path: /admin/v1/*
backend:
serviceName: admin-service
servicePort: 443
status:
loadBalancer:
ingress:
- ip: XXX.YYY.WWW.ZZZ
Reading this document from GCP I understood that Loadbalancer it's compatible with Self-signed certificates.
I would appreciate any insight or new directions you guys can provide.
Thanks in advance.
EDIT 1: I've added here the ingress YAML file which may help to a better understanding of the issue.
EDIT 2: I've updated the deployment YAML with the solution I found for liveness and readiness probes (scheme).
EDIT 3: I've found the solution for GCP Health Checks using annotation on Services declaration. I will put all the details on the response to my own question.
Here is what I found on how to fix the issue.
After reading a lot of documentation related to Kubernetes and GCP I found a document on GCP explaining to use annotations on Service declaration. Take a look at lines 7-8.
---
apiVersion: v1
kind: Service
metadata:
name: admin-service
namespace: default
annotations:
cloud.google.com/app-protocols: '{"https":"HTTPS"}'
spec:
type: NodePort
selector:
app: iteam-admin
ports:
- port: 443
targetPort: 8443
name: https
protocol: TCP
This will hint GCP to create the backend-service and health-check using HTTPS and everything will work as expected.
Reference: https://cloud.google.com/kubernetes-engine/docs/concepts/ingress-xlb#https_tls_between_load_balancer_and_your_application
I have set up a Kubernetes cluster on GCP/GKE and it's all working well except for one thing. When I access the external IP for the service the (default?) "Kubernetes Ingress Controller Fake Certificate" is served.
I am trying to use the NGINX Ingress (https://kubernetes.github.io/ingress-nginx/) and have followed what I believe are the correct instructions for associating a TLS secret with the Ingress. For example:
https://estl.tech/configuring-https-to-a-web-service-on-google-kubernetes-engine-2d71849520d
https://kubernetes.github.io/ingress-nginx/user-guide/tls/
I have created a secret like this:
apiVersion: v1
kind: Secret
metadata:
name: example-tls
namespace: default
data:
tls.crt: [removed]
tls.key: [removed]
type: kubernetes.io/tls
And associated that secret (which I can confirm is applied correctly and I can see in the cluster config) with the Ingress like this:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: example-ingress
namespace: default
annotations:
kubernetes.io/ingress.class: "nginx"
kubernetes.io/ingress.allow-http: "false"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
nginx.ingress.kubernetes.io/affinity: "cookie"
spec:
backend:
serviceName: example-service
servicePort: 80
tls:
- secretName: example-tls
From the documentation I feel that this should work (but, barring a bug, I am obviously mistaken!).
I've also seen some documentation around requiring target proxies for HTTPS. Perhaps that is the way that I should be doing this?
Many thanks for your help in advance.
Cheers,
Ben
PS: This is my load balancer configuration:
kind: Service
apiVersion: v1
metadata:
name: ingress-nginx
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
externalTrafficPolicy: Cluster
loadBalancerIP: [removed]
sessionAffinity: ClientIP
type: LoadBalancer
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
ports:
- name: http
port: 80
targetPort: http
- name: https
port: 443
targetPort: https
Edit 1:
Looking at my Ingress I can see this:
➜ gke git:(develop) ✗ kubectl describe ing example-tls-ingress
Name: example-tls-ingress
Namespace: default
Address: [removed]
Default backend: example-webapp-service:80 ([removed])
TLS:
example-tls terminates
Rules:
Host Path Backends
---- ---- --------
* * example-webapp-service:80 ([removed])
So it looks like the secret is picked up.
And this makes me think that there is a difference between Ingress-terminated TLS and Load Balancer-terminated TLS?
You can just refer to this stackoverflow post.
You need to install jetstack cert-Manager, create clusterissuer/issuer, along with a certificate in which you have to pass domain name / hostname and jetstack will automatically create the secret for you, by the name you mentioned in the 'Certificate'.
That secret has to be patched to TLS in ingress rule.
I have a setup that is not too much different than the user guide for use with k8s. For some reason I can only access http://app.minikube and not https://app.minikube.
Can someone look at my setup and see what I am obviously missing?
apiVersion: v1
kind: Service
metadata:
name: myapp
labels:
app: myapp
spec:
ports:
- name: http
port: 80
targetPort: 7777
selector:
app: myapp
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: myingress
annotations:
kubernetes.io/ingress.class: traefik
spec:
rules:
- host: app.minikube
http:
paths:
- path: /
backend:
serviceName: myapp
servicePort: http
tls:
- secretName: mytls
FYI, according to the Traefik user guide, the hosts definition in tls is unneeded, which is why I left it out.
The field hosts in the TLS configuration is ignored. Instead, the domains provided by the certificate are used for this purpose. It is recommended to not use wildcard certificates as they will match globally)
You're missing the hosts section:
tls:
- hosts:
- my-host.example.com
secretName: my-secret