Kubernetes: cert-manager certificate is keep in pending state - ssl

I have installed cert-manager 0.12.0 for SSL certificate.
My Issuer file is
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: my#email.com
privateKeySecretRef:
name: letsencrypt-prod
http01: {}
My certificate file
apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
name: tls-secret
spec:
secretName: tls-secret-prod
dnsNames:
- mydomain.com
acme:
config:
- http01:
ingressClass: nginx
domains:
- mydomain.com
issuerRef:
name: letsencrypt-prod
kind: ClusterIssuer
Ingress configuration is
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: cms
annotations:
kubernetes.io/ingress.class: nginx
cert-manager.io/cluster-issuer: letsencrypt-prod
kubernetes.io/tls-acme: "true"
spec:
tls:
- hosts:
- mydomain.com
secretName: tls-secret-prod
rules:
- host: mydomain.com
http:
paths:
- backend:
serviceName: apostrophe
servicePort: 80
path: /
But still, SSL certificated is not valid. And Common name is “Kubernetes Ingress Controller Fake Certificate”.
The following result to show orders and challenges
kubectl get orders, challenges -o wide
NAME STATE DOMAIN REASON AGE
challenge.certmanager.k8s.io/tls-secret-155743219-0 pending mydomain.com pods "cm-acme-http-solver-gk2zx" is forbidden: minimum cpu usage per Container is 100m, but request is 10m. 26m
I have updated the resources limit the range and reinstalled cert-manager with helm. I am still getting this error.
I am not sure what goes wrong or show how to fix this.
Please let me know if you need anything. Thanks in advance!

The problem lays in cpu limits defined for specific pod.
You have to change minimum CPU limit in deployment configuration file. As you can see pod (cm-acme-http-solver) is requesting 100m CPU usage while minimum CPU usage defined for specific pod is *10m**. So change CPU limits in deployment configuration file from 100m to 10m or less or you can also increase CPU requests.
Take a look here: cert-manager-kubernetes, pod-min-cpu-request.
Useful article: resources-limits-kubernetes.

Related

From Ingress to IngressRoute with CertManager, HTTP01 Challenge and Let's Encrypt ClusterIssuer

I have a Kubernetes (v1.25.2) cluster running with cert-manager 1.11.0 and Traefik 2.9.6.
For some services I want Let's Encrypt to auto sign certificates. For some reason, it feels nicer, to use IngressRoute instead of Ingress. I just can't get the IngressRoute to create the certificate.
Now, I have the a ClusterIssuer:
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: my#email.com
privateKeySecretRef:
name: letsencrypt-prod
solvers:
- http01:
ingress:
class: traefik
and, working, corresponding Ingress:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: myapp-name-websecure
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
ingressClassName: traefik
rules:
- host: my.host.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: linkingservice
port:
number: 80
tls:
- hosts:
- my.host.com
secretName: some-secret-name-tls
This works, nice. Instead, with IngressRoute the base resource is this:
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: myapp-other-name-websecure
spec:
entryPoints:
- websecure
routes:
- match: Host(`other.host.com`)
kind: Rule
services:
- name: linkingservice
port: 80
tls:
# certResolver: ??? # resolve what? Doesn't link with the ClusterIssuer
# issuerRef: ??? # doesn't exist (anymore)
Now, I've tried to:
just as for the Ingress to use the annotations: cert-manager.io/cluster-issuer: letsencrypt-prod. Which is being ignored
use the tls.certResolver, which doesn't work, because it doesn't exist. Should I create one? I expect The ClusterIssuer to create the certificate and secret, just as it does for Ingress.
I also saw the issuerRef as option in the tls section, but that doesn't appear to exist.
I thought I read that the IngressRoute is like a layer on top of the k8s default Ingress, so it should be something logical/similar
FYI: the ClusterIssuer and Ingress will also work for Nginx, when you replace the solvers.http01.ingress.class with nginx, likewise for the Ingress's spec.ingressClassName. (perhaps also without, but I can't test)
Now, I did find a way, but still feels like more work than should be necessary. The thing here is to create the Certificate and link that to the ClusterIssuer, that certificate then creates a Secret. This secret needs to be added to the spec.tls.secretName, like:
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: my.host.com-cert
spec:
secretName: my.host.com-secret
issuerRef:
name: letsencrypt-prod
kind: ClusterIssuer
dnsNames:
- my.host.com
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
spec:
...
tls:
secretName: my.host.com-secret
I've also tried for, certResolver. But the CertificateResolver from cert-manager was discontinued in v0.15.0; alternatively the CertificateRequest, but this was also discontinued in cert-manager version 0.9.0. The suggested way seems to be the previous described one.
I revisited the Traefik deployment values.yaml where I found there is a field with certResolvers, from their file:
certResolvers:
letsencrypt:
# for challenge options cf. https://doc.traefik.io/traefik/https/acme/
email: email#example.com
dnsChallenge:
# also add the provider's required configuration under env
# or expand then from secrets/configmaps with envfrom
# cf. https://doc.traefik.io/traefik/https/acme/#providers
provider: digitalocean
# add futher options for the dns challenge as needed
# cf. https://doc.traefik.io/traefik/https/acme/#dnschallenge
delayBeforeCheck: 30
resolvers:
- 1.1.1.1
- 8.8.8.8
tlsChallenge: true
httpChallenge:
entryPoint: "web"
# It has to match the path with a persistent volume
storage: /data/acme.json
Which makes me wonder. If you set this up, then probably the:
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
spec:
tls:
certResolver: letsencrypt
can work

ArgoCD with nginx ingress and cert manager not working

I am trying to expose ArgoCD using ingress and cert-manager in GKE cluster(version 1.21.5-gke.1302) but the certificate is not issued.
Steps to reproduce:
Install cert-manager applying this yaml
Install nginx ingress-controller with helm running:
helm install my-release nginx-stable/nginx-ingress
Create clusterIssuer applying the following:
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-cluster-issuer
spec:
acme:
email: example#email.com
server: https://acme-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name: letsencrypt-cluster-issuer-key
solvers:
- http01:
ingress:
class: nginx
Applied ingress using this guide and the file:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: argocd-server-ingress
namespace: argocd
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
kubernetes.io/ingress.class: nginx
kubernetes.io/tls-acme: "true"
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
# If you encounter a redirect loop or are getting a 307 response code
# then you need to force the nginx ingress to connect to the backend using HTTPS.
#
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
spec:
rules:
- host: argocd.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: argocd-server
port:
name: https
tls:
- hosts:
- argocd.example.com
secretName: argocd-secret # do not change, this is provided by Argo CD
Map the ip of ingress-controller to your host name.
Expected behaviour: I was expecting a certificate to be created successfully and have access to the app.
Current status:
Certificate describe gives me this:
Conditions:
Last Transition Time: 2022-01-18T14:10:14Z
Message: Existing issued Secret is not up to date for spec: [spec.dnsNames]
Observed Generation: 3
Reason: SecretMismatch
Status: False
Type: Ready
Last Transition Time: 2022-01-18T14:10:14Z
Message: Issuing certificate as Secret was previously issued by Issuer.cert-manager.io/
Observed Generation: 1
Reason: IncorrectIssuer
Status: True
Type: Issuing
Next Private Key Secret Name: argocd-secret-ccjtv
Not After: 2023-01-18T13:39:24Z
Not Before: 2022-01-18T13:39:24Z
Renewal Time: 2022-09-18T21:39:24Z
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Requested 16m cert-manager Created new CertificateRequest resource "argocd-secret-qm469"
Normal Requested 15m cert-manager Created new CertificateRequest resource "argocd-secret-9ctn4"
Normal Reused 7m19s (x2 over 45h) cert-manager Reusing private key stored in existing Secret resource "argocd-secret"
Finally I can access the provided url by the challenge but status is pending with reason:
Waiting for HTTP-01 challenge propagation: failed to perform self check GET request
Does anyone have any idea what might be wrong? It would be highly appreciated.
Thanks!
Looks like you have a different name for cluster issue in your ingress rule for ArgoCD.
From your example in the ClusterIssuer manifest:
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-cluster-issuer
And from ingress rule
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
I think you need to specify:
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-cluster-issuer
spec:
acme:
email: example#email.com
server: https://acme-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name: argocd-secret # HERE use secrets created by ArgoCD
solvers:
- http01:
ingress:
class: nginx

Kubernetes cert-manager certificate is created but can not get vertified

I am working on a DO kubernetes cluster and install ingress nginx and argocd on it, All seems fine and I can easily use the ingress as long as they are accessing the services via http.
I have also installed certmanager and here are the main files regarding my ingress, certificate and issuer:
Ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: rancher-demo
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /
certmanager.k8s.io/cluster-issuer: "letsencrypt-production"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
tls:
- hosts:
- {sub-domain}
secretName: ssl-cert-production
rules:
- host: {sub-domain}
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: rancher-demo
port:
number: 80
Issuer
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-production
namespace: default
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: {my-email}
privateKeySecretRef:
name: letsencrypt-production
solvers:
- selector: {}
http01:
ingress:
class: nginx
Certificate
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: ssl-cert-production
namespace: default
spec:
secretName: ssl-cert-production
issuerRef:
name: letsencrypt-production
kind: ClusterIssuer
commonName: {sub-domain}
dnsNames:
- {sub-domain}
I went through some other samples on github and questions on stackoverflow and unfortunetly I can not figure out where I am doing it wrong.
Thank you in advance for your attentions
I finally managed to fix the issue, what I have done was as follow:
creating a new kubernete instance
installing cert-manager manaully
installing ingress-nginx manaully
creating the issuer (waiting for it to complete)
creating deployment and cluster
creating ingress config for my application
creating the certificate (waiting for it to complete)
I was working on ArgoCD and had to first do these setups myself before handling CD with Argo. It was my own fault that I did not properly go through their documentations. The order is important, but the way I sat up Argo, it was provisioing everything in parallel so for example certificate was being provision before ingress be up or issuer be in place
Also for anyone who is interested in a detailed version please checkout the github repo I created below:
https://github.com/mehdiamenein/cert-manager-nginx-ingress-do
I hope this can be helpful to someone else as well :)
Huge thanks to marcel.dempers for his wonderful video https://www.youtube.com/watch?v=hoLUigg4V18
and many thanks to #justin and #harsh-manvar for their comments

How can I use Cert manager letsencrypt-prod in my kubernetes service?

I have 4 yaml file
Deployment.yaml
Service.yaml
Ingress.yaml
issuer.yaml
I want to use letsencrypt-prod for my service for certification . But it doesn't work.
When I use to be sure ingress is working or issuer is working both of them are done!
kubectl get ing
kubectl get issuer
But when I run:
kubectl get cert
Cert is not readt during 2 days . Like below:
it creates problem like below. certification is not binding mandrakee.xyz.Mandrakee.xyz looks still not secure! how can I make my website secyre via cert manager?
Deployment.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
name: echo-deployment
spec:
replicas: 1
selector:
matchLabels:
app: echo-server
template:
metadata:
labels:
app: echo-server
spec:
containers:
- name: httpapi-host
image: jmalloc/echo-server
imagePullPolicy: Always
resources:
requests:
memory: "128Mi"
cpu: "500m"
ports:
- containerPort: 80
Service.yaml:
apiVersion: v1
kind: Service
metadata:
name: echo-service
spec:
ports:
- name: http-port
port: 80
targetPort: 8080
selector:
app: echo-server
Ingress.yaml:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: ambassador
cert-manager.io/issuer: letsencrypt-prod
name: test-ingress
spec:
tls:
- hosts:
- mandrakee.xyz
secretName: letsencrypt-prod
rules:
- host: mandrakee.xyz
http:
paths:
- backend:
service:
name: echo-service
port:
number: 80
path: /
pathType: Prefix
issuer.yaml:
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: letsencrypt-prod
spec:
acme:
email: ykaratoprak#sphereinc.com
server: https://acme-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name: letsencrypt-prod
solvers:
- dns01:
digitalocean:
tokenSecretRef:
name: digitalocean-dns
key: ce28952b5b4e33ea7d98de190f3148a7cc82d31f030bde966ad13b22c1abc524
If you have setup your issuer correctly, which you have assured us, you will see in your namespace a pod belonging to cert manager. This creates a pod that will validate that the server requesting the certificate resolves to the DNS record.
In your case, you would need to point your DNS towards your ingress.
If this is done successfully, then the next stage of debugging is to validate that both 443 and 80 can be resolved. The Validation Pod created by Cert Manager uses port 80 to validate the communication. A common mistake people make is assuming that they will only use port 443 for ssl and disable 80 for security reasons to find out later that letsencrypt can't validate the hostname without port 80.
Otherwise, the common scenario is that cert-manager is installed in the namespace cert-manager and so you should check the logs of the manager. This will provided a limited amount of logs and can be sometimes cryptic to finding the remedy to your issues.
To find the direct error, the pod spawned by cert-manager in the namespace you have deployed the ingress is a good place to focus.
A test I would run is to setup the ingress with both 80 and 443, if you use your domain from your browser you should get some invalid kubernetes generic certificates response on the port 443 and just "Not Found" on port 80. If this is successful, it rules out the limitation I have mentioned before.

SSL Certificates on Kubernetes Using ACME

I have been following this tutorial: https://cert-manager.io/docs/ , and after I have installed my cert manager and made sure they are running with kubectl get pods --namespace cert-manager,
cert-manager-5597cff495-l5hjs 1/1 Running 0 91m
cert-manager-cainjector-bd5f9c764-xrb2t 1/1 Running 0 91m
cert-manager-webhook-5f57f59fbc-q5rqs 1/1 Running 0 91m
I then configured my cert-manager using ACME issuer by following this tutorial https://cert-manager.io/docs/configuration/acme/ .
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-staging
spec:
acme:
# You must replace this email address with your own.
# Let's Encrypt will use this to contact you about expiring
# certificates, and issues related to your account.
email: aidenhsy#gmail.com
server: https://acme-staging-v02.api.letsencrypt.org/directory
privateKeySecretRef:
# Secret resource that will be used to store the account's private key.
name: letsencrypt-staging
# Add a single challenge solver, HTTP01 using nginx
solvers:
- http01:
ingress:
class: nginx
Here is my full ingress config file:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-srv
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/use-regex: 'true'
spec:
rules:
- host: www.hyhaus.xyz
http:
paths:
- path: /api/?(.*)
backend:
serviceName: devback-srv
servicePort: 4000
- path: /?(.*)
backend:
serviceName: devfront-srv
servicePort: 3000
---
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/do-loadbalancer-enable-proxy-protocol: 'true'
service.beta.kubernetes.io/do-loadbalancer-hostname: 'www.hyhaus.xyz'
labels:
helm.sh/chart: ingress-nginx-2.0.3
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 0.32.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: controller
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
type: LoadBalancer
externalTrafficPolicy: Local
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
- name: https
port: 443
protocol: TCP
targetPort: https
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/component: controller
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-staging
spec:
acme:
# You must replace this email address with your own.
# Let's Encrypt will use this to contact you about expiring
# certificates, and issues related to your account.
email: aidenhsy#gmail.com
server: https://acme-staging-v02.api.letsencrypt.org/directory
privateKeySecretRef:
# Secret resource that will be used to store the account's private key.
name: letsencrypt-staging
# Add a single challenge solver, HTTP01 using nginx
solvers:
- http01:
ingress:
class: nginx
However when I browse to my site, the browser warns: security certificate is not trusted by your computer's operating system. And when I took a look a my certificate, it shows self-assigned, which is not really what I want. Am I doing something wrong here?
This is a certificate placeholder provided by nginx ingress controller. When you see it, it means there is no other (dedicated) certificate for the endpoint.
Now the first reason why this happened is that your Ingress doesn't have necessary data. Update it with this:
metadata:
annotations:
# which issuer to use
cert-manager.io/cluster-issuer: "letsencrypt-staging"
spec:
tls: # placing a host in TLS config indicates that a certificate should be created
- hosts:
- example.org
- www.example.org
- xyz.example.org
secretName: myingress-cert # cert-manager will store the created certificate in this secret
Documentation for ingress objects is here.
If the above didn't help, try the troubleshooting steps offered by the documentation. In my experience checking CertificateRequest and Certificate resources was enough in most cases to determine the problem.
$ kubectl get certificate
$ kubectl describe certificate <certificate-name>
$ kubectl get certificaterequest
$ kubectl describe certificaterequest <CertificateRequest name>
Remember that these objects are namespaced, meaning that they'll be in the same namespace as the ingress object.
To secure Ingress, First you have to add ClusterIssuer to your Ingress resources and cert-manager will then pick it up and create the Certificate resource for you .
Kind : ingress metadata: annotations : cert-manager.io/cluster-issuer: nameOfClusterIssuer .
Second you have to add tls <= this indicates the creation of certificate(key/cert pair) by Cert-manager via The ClusterIssuer.
Third you have to add secretName: myingress <= here the cert manager will store the tls secrets ( after creating key/cert pair and store them for you)..