I have set up a TLS kubernetes secret which is operating normally on the non-traefik workloads.
When I try to use it to one of my ingress routes the certificate seems to have not been applied and the "TRAEFIK DEFAULT CERT" is assigned.
Below is my IngressRoute .yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: example
namespace: example-ns
spec:
entryPoints:
- websecure
routes:
- match: Host(`example.com`)
kind: Rule
services:
- name: example-svc
port: 9090
tls:
secretName: example-tls
Found it!
Indeed issue was related with passing the TLS Store to both namespaces and traefik pod logged the below:
level=error msg="Default TLS Stores defined in multiple namespaces: [example traefik]" providerName=kubernetescrd
I removed the TLS store from the "example" namespace and everything worked fine!
Related
I have a Kubernetes (v1.25.2) cluster running with cert-manager 1.11.0 and Traefik 2.9.6.
For some services I want Let's Encrypt to auto sign certificates. For some reason, it feels nicer, to use IngressRoute instead of Ingress. I just can't get the IngressRoute to create the certificate.
Now, I have the a ClusterIssuer:
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: my#email.com
privateKeySecretRef:
name: letsencrypt-prod
solvers:
- http01:
ingress:
class: traefik
and, working, corresponding Ingress:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: myapp-name-websecure
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
ingressClassName: traefik
rules:
- host: my.host.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: linkingservice
port:
number: 80
tls:
- hosts:
- my.host.com
secretName: some-secret-name-tls
This works, nice. Instead, with IngressRoute the base resource is this:
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: myapp-other-name-websecure
spec:
entryPoints:
- websecure
routes:
- match: Host(`other.host.com`)
kind: Rule
services:
- name: linkingservice
port: 80
tls:
# certResolver: ??? # resolve what? Doesn't link with the ClusterIssuer
# issuerRef: ??? # doesn't exist (anymore)
Now, I've tried to:
just as for the Ingress to use the annotations: cert-manager.io/cluster-issuer: letsencrypt-prod. Which is being ignored
use the tls.certResolver, which doesn't work, because it doesn't exist. Should I create one? I expect The ClusterIssuer to create the certificate and secret, just as it does for Ingress.
I also saw the issuerRef as option in the tls section, but that doesn't appear to exist.
I thought I read that the IngressRoute is like a layer on top of the k8s default Ingress, so it should be something logical/similar
FYI: the ClusterIssuer and Ingress will also work for Nginx, when you replace the solvers.http01.ingress.class with nginx, likewise for the Ingress's spec.ingressClassName. (perhaps also without, but I can't test)
Now, I did find a way, but still feels like more work than should be necessary. The thing here is to create the Certificate and link that to the ClusterIssuer, that certificate then creates a Secret. This secret needs to be added to the spec.tls.secretName, like:
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: my.host.com-cert
spec:
secretName: my.host.com-secret
issuerRef:
name: letsencrypt-prod
kind: ClusterIssuer
dnsNames:
- my.host.com
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
spec:
...
tls:
secretName: my.host.com-secret
I've also tried for, certResolver. But the CertificateResolver from cert-manager was discontinued in v0.15.0; alternatively the CertificateRequest, but this was also discontinued in cert-manager version 0.9.0. The suggested way seems to be the previous described one.
I revisited the Traefik deployment values.yaml where I found there is a field with certResolvers, from their file:
certResolvers:
letsencrypt:
# for challenge options cf. https://doc.traefik.io/traefik/https/acme/
email: email#example.com
dnsChallenge:
# also add the provider's required configuration under env
# or expand then from secrets/configmaps with envfrom
# cf. https://doc.traefik.io/traefik/https/acme/#providers
provider: digitalocean
# add futher options for the dns challenge as needed
# cf. https://doc.traefik.io/traefik/https/acme/#dnschallenge
delayBeforeCheck: 30
resolvers:
- 1.1.1.1
- 8.8.8.8
tlsChallenge: true
httpChallenge:
entryPoint: "web"
# It has to match the path with a persistent volume
storage: /data/acme.json
Which makes me wonder. If you set this up, then probably the:
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
spec:
tls:
certResolver: letsencrypt
can work
I have 4 yaml file
Deployment.yaml
Service.yaml
Ingress.yaml
issuer.yaml
I want to use letsencrypt-prod for my service for certification . But it doesn't work.
When I use to be sure ingress is working or issuer is working both of them are done!
kubectl get ing
kubectl get issuer
But when I run:
kubectl get cert
Cert is not readt during 2 days . Like below:
it creates problem like below. certification is not binding mandrakee.xyz.Mandrakee.xyz looks still not secure! how can I make my website secyre via cert manager?
Deployment.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
name: echo-deployment
spec:
replicas: 1
selector:
matchLabels:
app: echo-server
template:
metadata:
labels:
app: echo-server
spec:
containers:
- name: httpapi-host
image: jmalloc/echo-server
imagePullPolicy: Always
resources:
requests:
memory: "128Mi"
cpu: "500m"
ports:
- containerPort: 80
Service.yaml:
apiVersion: v1
kind: Service
metadata:
name: echo-service
spec:
ports:
- name: http-port
port: 80
targetPort: 8080
selector:
app: echo-server
Ingress.yaml:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: ambassador
cert-manager.io/issuer: letsencrypt-prod
name: test-ingress
spec:
tls:
- hosts:
- mandrakee.xyz
secretName: letsencrypt-prod
rules:
- host: mandrakee.xyz
http:
paths:
- backend:
service:
name: echo-service
port:
number: 80
path: /
pathType: Prefix
issuer.yaml:
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: letsencrypt-prod
spec:
acme:
email: ykaratoprak#sphereinc.com
server: https://acme-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name: letsencrypt-prod
solvers:
- dns01:
digitalocean:
tokenSecretRef:
name: digitalocean-dns
key: ce28952b5b4e33ea7d98de190f3148a7cc82d31f030bde966ad13b22c1abc524
If you have setup your issuer correctly, which you have assured us, you will see in your namespace a pod belonging to cert manager. This creates a pod that will validate that the server requesting the certificate resolves to the DNS record.
In your case, you would need to point your DNS towards your ingress.
If this is done successfully, then the next stage of debugging is to validate that both 443 and 80 can be resolved. The Validation Pod created by Cert Manager uses port 80 to validate the communication. A common mistake people make is assuming that they will only use port 443 for ssl and disable 80 for security reasons to find out later that letsencrypt can't validate the hostname without port 80.
Otherwise, the common scenario is that cert-manager is installed in the namespace cert-manager and so you should check the logs of the manager. This will provided a limited amount of logs and can be sometimes cryptic to finding the remedy to your issues.
To find the direct error, the pod spawned by cert-manager in the namespace you have deployed the ingress is a good place to focus.
A test I would run is to setup the ingress with both 80 and 443, if you use your domain from your browser you should get some invalid kubernetes generic certificates response on the port 443 and just "Not Found" on port 80. If this is successful, it rules out the limitation I have mentioned before.
I have been following this tutorial: https://cert-manager.io/docs/ , and after I have installed my cert manager and made sure they are running with kubectl get pods --namespace cert-manager,
cert-manager-5597cff495-l5hjs 1/1 Running 0 91m
cert-manager-cainjector-bd5f9c764-xrb2t 1/1 Running 0 91m
cert-manager-webhook-5f57f59fbc-q5rqs 1/1 Running 0 91m
I then configured my cert-manager using ACME issuer by following this tutorial https://cert-manager.io/docs/configuration/acme/ .
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-staging
spec:
acme:
# You must replace this email address with your own.
# Let's Encrypt will use this to contact you about expiring
# certificates, and issues related to your account.
email: aidenhsy#gmail.com
server: https://acme-staging-v02.api.letsencrypt.org/directory
privateKeySecretRef:
# Secret resource that will be used to store the account's private key.
name: letsencrypt-staging
# Add a single challenge solver, HTTP01 using nginx
solvers:
- http01:
ingress:
class: nginx
Here is my full ingress config file:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-srv
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/use-regex: 'true'
spec:
rules:
- host: www.hyhaus.xyz
http:
paths:
- path: /api/?(.*)
backend:
serviceName: devback-srv
servicePort: 4000
- path: /?(.*)
backend:
serviceName: devfront-srv
servicePort: 3000
---
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/do-loadbalancer-enable-proxy-protocol: 'true'
service.beta.kubernetes.io/do-loadbalancer-hostname: 'www.hyhaus.xyz'
labels:
helm.sh/chart: ingress-nginx-2.0.3
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 0.32.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: controller
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
type: LoadBalancer
externalTrafficPolicy: Local
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
- name: https
port: 443
protocol: TCP
targetPort: https
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/component: controller
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-staging
spec:
acme:
# You must replace this email address with your own.
# Let's Encrypt will use this to contact you about expiring
# certificates, and issues related to your account.
email: aidenhsy#gmail.com
server: https://acme-staging-v02.api.letsencrypt.org/directory
privateKeySecretRef:
# Secret resource that will be used to store the account's private key.
name: letsencrypt-staging
# Add a single challenge solver, HTTP01 using nginx
solvers:
- http01:
ingress:
class: nginx
However when I browse to my site, the browser warns: security certificate is not trusted by your computer's operating system. And when I took a look a my certificate, it shows self-assigned, which is not really what I want. Am I doing something wrong here?
This is a certificate placeholder provided by nginx ingress controller. When you see it, it means there is no other (dedicated) certificate for the endpoint.
Now the first reason why this happened is that your Ingress doesn't have necessary data. Update it with this:
metadata:
annotations:
# which issuer to use
cert-manager.io/cluster-issuer: "letsencrypt-staging"
spec:
tls: # placing a host in TLS config indicates that a certificate should be created
- hosts:
- example.org
- www.example.org
- xyz.example.org
secretName: myingress-cert # cert-manager will store the created certificate in this secret
Documentation for ingress objects is here.
If the above didn't help, try the troubleshooting steps offered by the documentation. In my experience checking CertificateRequest and Certificate resources was enough in most cases to determine the problem.
$ kubectl get certificate
$ kubectl describe certificate <certificate-name>
$ kubectl get certificaterequest
$ kubectl describe certificaterequest <CertificateRequest name>
Remember that these objects are namespaced, meaning that they'll be in the same namespace as the ingress object.
To secure Ingress, First you have to add ClusterIssuer to your Ingress resources and cert-manager will then pick it up and create the Certificate resource for you .
Kind : ingress metadata: annotations : cert-manager.io/cluster-issuer: nameOfClusterIssuer .
Second you have to add tls <= this indicates the creation of certificate(key/cert pair) by Cert-manager via The ClusterIssuer.
Third you have to add secretName: myingress <= here the cert manager will store the tls secrets ( after creating key/cert pair and store them for you)..
I am using Traefik like a reverse proxy on my Kubernetes cluster and I'm looking a way to use multiple SSL Certificates.
According to the doc, is not possible to use multiple TLSStore, I must use only the default store.
For example, I have two domains app1.com and app2.com and two certificates for *.app1.com and *.app2.com known by Traefik using secret app1-com-ssl and app2-com-ssl, is it possible to store these two certificates in the same default TLSStore?
Here my TLSStore definition:
apiVersion: traefik.containo.us/v1alpha1
kind: TLSStore
metadata:
name: default
namespace: app1
spec:
defaultCertificate:
secretName: app1-com-ssl
Here my IngressRoute for app1:
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: routes
namespace: app1
spec:
entryPoints:
- websecure
- web
routes:
- match: Host(`test.app1.com`)
kind: Rule
services:
- name: test-service
port: 8080
tls:
store:
name: default
Here my IngressRoute for app2:
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: routes
namespace: app2
spec:
entryPoints:
- websecure
- web
routes:
- match: Host(`test.app2.com`)
kind: Rule
services:
- name: test-service
port: 8080
tls:
store:
name: default
Thanks!
Good question. The TLSStore doesn't currently support multiple certificates it just supports a 'default certificate'. And as you have already seen you can only specify a 'default' TLSStore.
The Traefik configs do allow you to specify multiple certificates for the 'default' TLSStore (But not the K8s CRD) I have opened a feature request for the Traekik TLSStore CRD to add support for multiple certificates.
✌️
I have set up a Kubernetes cluster on GCP/GKE and it's all working well except for one thing. When I access the external IP for the service the (default?) "Kubernetes Ingress Controller Fake Certificate" is served.
I am trying to use the NGINX Ingress (https://kubernetes.github.io/ingress-nginx/) and have followed what I believe are the correct instructions for associating a TLS secret with the Ingress. For example:
https://estl.tech/configuring-https-to-a-web-service-on-google-kubernetes-engine-2d71849520d
https://kubernetes.github.io/ingress-nginx/user-guide/tls/
I have created a secret like this:
apiVersion: v1
kind: Secret
metadata:
name: example-tls
namespace: default
data:
tls.crt: [removed]
tls.key: [removed]
type: kubernetes.io/tls
And associated that secret (which I can confirm is applied correctly and I can see in the cluster config) with the Ingress like this:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: example-ingress
namespace: default
annotations:
kubernetes.io/ingress.class: "nginx"
kubernetes.io/ingress.allow-http: "false"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
nginx.ingress.kubernetes.io/affinity: "cookie"
spec:
backend:
serviceName: example-service
servicePort: 80
tls:
- secretName: example-tls
From the documentation I feel that this should work (but, barring a bug, I am obviously mistaken!).
I've also seen some documentation around requiring target proxies for HTTPS. Perhaps that is the way that I should be doing this?
Many thanks for your help in advance.
Cheers,
Ben
PS: This is my load balancer configuration:
kind: Service
apiVersion: v1
metadata:
name: ingress-nginx
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
externalTrafficPolicy: Cluster
loadBalancerIP: [removed]
sessionAffinity: ClientIP
type: LoadBalancer
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
ports:
- name: http
port: 80
targetPort: http
- name: https
port: 443
targetPort: https
Edit 1:
Looking at my Ingress I can see this:
➜ gke git:(develop) ✗ kubectl describe ing example-tls-ingress
Name: example-tls-ingress
Namespace: default
Address: [removed]
Default backend: example-webapp-service:80 ([removed])
TLS:
example-tls terminates
Rules:
Host Path Backends
---- ---- --------
* * example-webapp-service:80 ([removed])
So it looks like the secret is picked up.
And this makes me think that there is a difference between Ingress-terminated TLS and Load Balancer-terminated TLS?
You can just refer to this stackoverflow post.
You need to install jetstack cert-Manager, create clusterissuer/issuer, along with a certificate in which you have to pass domain name / hostname and jetstack will automatically create the secret for you, by the name you mentioned in the 'Certificate'.
That secret has to be patched to TLS in ingress rule.