I am trying to get Let's Encrypt to work with cert-manager on GKE. I have followed the following procedure:
Install the CustomResourceDefinition resources separately
kubectl apply -f https://raw.githubusercontent.com/jetstack/cert-manager/release-0.7/deploy/manifests/00-crds.yaml
Create the namespace for cert-manager
kubectl create namespace cert-manager
Label the cert-manager namespace to disable resource validation
kubectl label namespace cert-manager certmanager.k8s.io/disable-validation=true
Add the Jetstack Helm repository
helm repo add jetstack https://charts.jetstack.io
Update your local Helm chart repository cache
helm repo update
Install the cert-manager Helm chart
helm install \
--name cert-manager \
--namespace cert-manager \
--version v0.7.0 \
jetstack/cert-manager
This results in (in the cert-manager namespace)
kubectl -n cert-manager get all
NAME READY STATUS
RESTARTS AGE
pod/cert-manager-6d8fc95f98-57c55 1/1 Running 0 26m
pod/cert-manager-cainjector-7c789f4fcc-jdqfs 1/1 Running 0 26m
pod/cert-manager-webhook-86bc6ff498-kcxj8 1/1 Running 0 26m
NAME TYPE CLUSTER-IP EXTERNAL-IP
PORT(S) AGE
service/cert-manager-webhook ClusterIP 10.39.251.139 <none> 443/TCP 26m
...
kubectl -n cert-manager get secrets
NAME TYPE DATA AGE
cert-manager-cainjector-token-mvmsx kubernetes.io/service-account-token 3 30m
cert-manager-token-gk2sp kubernetes.io/service-account-token 3 30m
cert-manager-webhook-ca kubernetes.io/tls 3 30m
cert-manager-webhook-token-6l6k7 kubernetes.io/service-account-token 3 30m
cert-manager-webhook-webhook-tls kubernetes.io/tls 3 30m
default-token-rx6sp kubernetes.io/service-account-token 3 30m
letsencrypt-prod Opaque 1 30m
Afterwards I install the webapp (in default) and (also in default) issuer.yml
apiVersion: certmanager.k8s.io/v1alpha1
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: 'me#me.com'
privateKeySecretRef:
name: letsencrypt-prod
https01: {}
and certificate.yml
apiVersion: certmanager.k8s.io/v1alpha1
kind: Certificate
metadata:
name: test-tls
spec:
secretName: test-me
issuerRef:
name: letsencrypt-prod
kind: ClusterIssuer
commonName: test.me
dnsNames:
- test.me
- www.test.me
acme:
config:
- http01:
ingressClass: nginx
domains:
- test.me
- www.test.me
Here, I seem to be having an issue as I get:
...
Issuer Ref:
Kind: ClusterIssuer
Name: letsencrypt-prod
Secret Name: test-me
Status:
Conditions:
Last Transition Time: 2019-03-27T16:35:40Z
Message: Certificate issuance in progress. Temporary certificate issued.
Reason: TemporaryCertificate
Status: False
Type: Ready
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning IssuerNotFound 4m (x2 over 4m) cert-manager clusterissuer.certmanager.k8s.io "letsencrypt-prod" not found
Warning IssuerNotReady 4m cert-manager Issuer letsencrypt-prod not ready
Normal Generated 4m cert-manager Generated new private key
Normal GenerateSelfSigned 4m cert-manager Generated temporary self signed certificate
Normal OrderCreated 4m cert-manager Created Order resource "test-me-tls-202592384"
It does move beyond this. No certificate gets verified...
Ingress looks like
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-service
annotations:
kubernetes.io/ingress.class: nginx
kubernetes.io/tls-acme: "true"
nginx.ingress.kubernetes.io/add-base-url: "true"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
certmanager.k8s.io/cluster-issuer: "letsencrypt-prod"
spec:
tls:
- hosts:
- test.me
- www.test.me
secretName: test-me
rules:
- host: test.me
http:
paths:
- path: /
backend:
serviceName: web-cluster-ip-service
servicePort: 80
- host: www.test.me
http:
paths:
- path: /
backend:
serviceName: web-cluster-ip-service
servicePort: 80
In the end my site remains insecure with a invalid certificate.
Issued to:
Common Name (CN) test.me
Organization (O) cert-manager
Organizational Unit (OU) <Not Part Of Certificate>
Issued by:
Common Name (CN) cert-manager.local
Organization (O) cert-manager
Organizational Unit (OU)
What am I missing that the certificate is not valid.
https01 (in issuer.yml) is a typo: this should have been http01
Related
I am trying to expose ArgoCD using ingress and cert-manager in GKE cluster(version 1.21.5-gke.1302) but the certificate is not issued.
Steps to reproduce:
Install cert-manager applying this yaml
Install nginx ingress-controller with helm running:
helm install my-release nginx-stable/nginx-ingress
Create clusterIssuer applying the following:
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-cluster-issuer
spec:
acme:
email: example#email.com
server: https://acme-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name: letsencrypt-cluster-issuer-key
solvers:
- http01:
ingress:
class: nginx
Applied ingress using this guide and the file:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: argocd-server-ingress
namespace: argocd
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
kubernetes.io/ingress.class: nginx
kubernetes.io/tls-acme: "true"
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
# If you encounter a redirect loop or are getting a 307 response code
# then you need to force the nginx ingress to connect to the backend using HTTPS.
#
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
spec:
rules:
- host: argocd.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: argocd-server
port:
name: https
tls:
- hosts:
- argocd.example.com
secretName: argocd-secret # do not change, this is provided by Argo CD
Map the ip of ingress-controller to your host name.
Expected behaviour: I was expecting a certificate to be created successfully and have access to the app.
Current status:
Certificate describe gives me this:
Conditions:
Last Transition Time: 2022-01-18T14:10:14Z
Message: Existing issued Secret is not up to date for spec: [spec.dnsNames]
Observed Generation: 3
Reason: SecretMismatch
Status: False
Type: Ready
Last Transition Time: 2022-01-18T14:10:14Z
Message: Issuing certificate as Secret was previously issued by Issuer.cert-manager.io/
Observed Generation: 1
Reason: IncorrectIssuer
Status: True
Type: Issuing
Next Private Key Secret Name: argocd-secret-ccjtv
Not After: 2023-01-18T13:39:24Z
Not Before: 2022-01-18T13:39:24Z
Renewal Time: 2022-09-18T21:39:24Z
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Requested 16m cert-manager Created new CertificateRequest resource "argocd-secret-qm469"
Normal Requested 15m cert-manager Created new CertificateRequest resource "argocd-secret-9ctn4"
Normal Reused 7m19s (x2 over 45h) cert-manager Reusing private key stored in existing Secret resource "argocd-secret"
Finally I can access the provided url by the challenge but status is pending with reason:
Waiting for HTTP-01 challenge propagation: failed to perform self check GET request
Does anyone have any idea what might be wrong? It would be highly appreciated.
Thanks!
Looks like you have a different name for cluster issue in your ingress rule for ArgoCD.
From your example in the ClusterIssuer manifest:
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-cluster-issuer
And from ingress rule
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
I think you need to specify:
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-cluster-issuer
spec:
acme:
email: example#email.com
server: https://acme-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name: argocd-secret # HERE use secrets created by ArgoCD
solvers:
- http01:
ingress:
class: nginx
I can't seem to get cert-manager working:
$ kubectl get certificates -o wide
NAME READY SECRET ISSUER STATUS AGE
tls-secret False tls-secret letsencrypt Issuing certificate as Secret does not exist 115m
$ kubectl get CertificateRequest -o wide
NAME READY ISSUER STATUS AGE
tls-secret-xxxx False letsencrypt Referenced "ClusterIssuer" not found: clusterissuer.cert-manager.io "letsencrypt" not found 113m
my certificate.yaml is :
apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
name: tls-secret
namespace: default
spec:
secretName: tls-secret
dnsNames:
- aks-xxxx.xxxxx.xxxx.aksapp.io
acme:
config:
- http01:
ingress:
name: xxxxxx
domains:
- aks-xxxx.xxxxx.xxxx.aksapp.io
issuerRef:
name: letsencrypt-staging
kind: ClusterIssuer
When i get cluster issuers
$ kubectl get clusterissuers
No resources found
any idea whats wrong?
You have not created the clusterissuers so it wont be there.
As you have created the certificate you can try the
kubectl get certificate
Your error is clearly saying the issue you have to create the clusterissuers
Referenced "ClusterIssuer" not found: clusterissuer.cert-manager.io
"letsencrypt" not found
Cert-manager site : https://cert-manager.io/docs/
Installation : https://cert-manager.io/docs/installation/
in single line just apply :
kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.5.3/cert-manager.yaml
how to configure & setup the clusterissuer : https://cert-manager.io/docs/configuration/acme/
Example of cluster issuer & ingress
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
name: cluster-issuer-name
namespace: development
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: harsh#example.com
privateKeySecretRef:
name: secret-name
solvers:
- http01:
ingress:
class: nginx-class-name
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx-class-name
cert-manager.io/cluster-issuer: cluster-issuer-name
nginx.ingress.kubernetes.io/rewrite-target: /
name: example-ingress
spec:
rules:
- host: sub.example.com
http:
paths:
- path: /api
backend:
serviceName: service-name
servicePort: 80
tls:
- hosts:
- sub.example.com
secretName: secret-name
Try with the latest cert-manager.
You'll also need issuer.yaml if you haven't set it up already
I have been following this tutorial: https://cert-manager.io/docs/ , and after I have installed my cert manager and made sure they are running with kubectl get pods --namespace cert-manager,
cert-manager-5597cff495-l5hjs 1/1 Running 0 91m
cert-manager-cainjector-bd5f9c764-xrb2t 1/1 Running 0 91m
cert-manager-webhook-5f57f59fbc-q5rqs 1/1 Running 0 91m
I then configured my cert-manager using ACME issuer by following this tutorial https://cert-manager.io/docs/configuration/acme/ .
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-staging
spec:
acme:
# You must replace this email address with your own.
# Let's Encrypt will use this to contact you about expiring
# certificates, and issues related to your account.
email: aidenhsy#gmail.com
server: https://acme-staging-v02.api.letsencrypt.org/directory
privateKeySecretRef:
# Secret resource that will be used to store the account's private key.
name: letsencrypt-staging
# Add a single challenge solver, HTTP01 using nginx
solvers:
- http01:
ingress:
class: nginx
Here is my full ingress config file:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-srv
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/use-regex: 'true'
spec:
rules:
- host: www.hyhaus.xyz
http:
paths:
- path: /api/?(.*)
backend:
serviceName: devback-srv
servicePort: 4000
- path: /?(.*)
backend:
serviceName: devfront-srv
servicePort: 3000
---
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/do-loadbalancer-enable-proxy-protocol: 'true'
service.beta.kubernetes.io/do-loadbalancer-hostname: 'www.hyhaus.xyz'
labels:
helm.sh/chart: ingress-nginx-2.0.3
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 0.32.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: controller
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
type: LoadBalancer
externalTrafficPolicy: Local
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
- name: https
port: 443
protocol: TCP
targetPort: https
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/component: controller
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-staging
spec:
acme:
# You must replace this email address with your own.
# Let's Encrypt will use this to contact you about expiring
# certificates, and issues related to your account.
email: aidenhsy#gmail.com
server: https://acme-staging-v02.api.letsencrypt.org/directory
privateKeySecretRef:
# Secret resource that will be used to store the account's private key.
name: letsencrypt-staging
# Add a single challenge solver, HTTP01 using nginx
solvers:
- http01:
ingress:
class: nginx
However when I browse to my site, the browser warns: security certificate is not trusted by your computer's operating system. And when I took a look a my certificate, it shows self-assigned, which is not really what I want. Am I doing something wrong here?
This is a certificate placeholder provided by nginx ingress controller. When you see it, it means there is no other (dedicated) certificate for the endpoint.
Now the first reason why this happened is that your Ingress doesn't have necessary data. Update it with this:
metadata:
annotations:
# which issuer to use
cert-manager.io/cluster-issuer: "letsencrypt-staging"
spec:
tls: # placing a host in TLS config indicates that a certificate should be created
- hosts:
- example.org
- www.example.org
- xyz.example.org
secretName: myingress-cert # cert-manager will store the created certificate in this secret
Documentation for ingress objects is here.
If the above didn't help, try the troubleshooting steps offered by the documentation. In my experience checking CertificateRequest and Certificate resources was enough in most cases to determine the problem.
$ kubectl get certificate
$ kubectl describe certificate <certificate-name>
$ kubectl get certificaterequest
$ kubectl describe certificaterequest <CertificateRequest name>
Remember that these objects are namespaced, meaning that they'll be in the same namespace as the ingress object.
To secure Ingress, First you have to add ClusterIssuer to your Ingress resources and cert-manager will then pick it up and create the Certificate resource for you .
Kind : ingress metadata: annotations : cert-manager.io/cluster-issuer: nameOfClusterIssuer .
Second you have to add tls <= this indicates the creation of certificate(key/cert pair) by Cert-manager via The ClusterIssuer.
Third you have to add secretName: myingress <= here the cert manager will store the tls secrets ( after creating key/cert pair and store them for you)..
I'm running into an issue handling tls certificates with cert-manager, I'm following the documentation and added some extras to work with Traefik as an ingress.
Currently, I have this YAML files:
cluster-issuer.yaml
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
name: letsencrypt-staging
namespace: secure-alexguedescom
spec:
acme:
email: user#gmail.com
server: https://acme-staging-v02.api.letsencrypt.org/directory
privateKeySecretRef:
# Secret resource used to store the account's private key.
name: letsencrypt-staging
# Add a single challenge solver, HTTP01 using nginx
solvers:
- selector: {}
http01:
ingress:
class: traefik-cert-manager
traefik-ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
# add an annotation indicating the issuer to use.
cert-manager.io/cluster-issuer: letsencrypt-staging
name: secure-alexguedescom-ingress-http
namespace: secure-alexguedescom
spec:
rules:
- host: secure.alexguedes.com
http:
paths:
- backend:
serviceName: secure-alexguedescom-nginx
servicePort: 80
path: /
tls:
- hosts:
- secure.alexguedes.com
secretName: secure-alexguedescom-cert
cert-staging.yaml
apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
name: secure-alexguedescom-cert
namespace: secure-alexguedescom
spec:
commonName: secure.alexguedes.com
secretName: letsencrypt-staging
dnsNames:
- secure.alexguedes.com
issuerRef:
name: letsencrypt-staging
kind: ClusterIssuer
Inspecting the certs I have this error message:
Message: Issuing certificate as Secret does not contain a certificate
Reason: MissingData
Also inspecting the certificaterequest I have this log messages:
Status:
Conditions:
Last Transition Time: 2020-08-16T00:32:01Z
Message: Waiting on certificate issuance from order secure-alexguedescom/secure-alexguedescom-cert-q8w5p-1982372682: "pending"
Reason: Pending
Status: False
Type: Ready
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal OrderCreated 11m cert-manager Created Order resource secure-alexguedescom/secure-alexguedescom-cert-q8w5p-1982372682
Normal OrderPending 11m cert-manager Waiting on certificate issuance from order secure-alexguedescom/secure-alexguedescom-cert-q8w5p-1982372682: ""
I'm not sure which piece is wrong, using Helm v2 with Tiller and k8s v1.7
Any ideas?
Thanks in advance
The typical problem with letsencrypt certs is the letsencrypt itself not being able to validate who you are and that you own the domain. In this case, alexguedes.com.
With cert-manager you can do Domain Validation and HTTP Validation. Based on the posted ClusterIssuer you are doing HTTP Validation. So you need to make sure that secure.alexguedes.com resolves to a globally available IP address and that Traefik port 443 is listening on that IP address.
I have installed cert-manager 0.12.0 for SSL certificate.
My Issuer file is
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: my#email.com
privateKeySecretRef:
name: letsencrypt-prod
http01: {}
My certificate file
apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
name: tls-secret
spec:
secretName: tls-secret-prod
dnsNames:
- mydomain.com
acme:
config:
- http01:
ingressClass: nginx
domains:
- mydomain.com
issuerRef:
name: letsencrypt-prod
kind: ClusterIssuer
Ingress configuration is
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: cms
annotations:
kubernetes.io/ingress.class: nginx
cert-manager.io/cluster-issuer: letsencrypt-prod
kubernetes.io/tls-acme: "true"
spec:
tls:
- hosts:
- mydomain.com
secretName: tls-secret-prod
rules:
- host: mydomain.com
http:
paths:
- backend:
serviceName: apostrophe
servicePort: 80
path: /
But still, SSL certificated is not valid. And Common name is “Kubernetes Ingress Controller Fake Certificate”.
The following result to show orders and challenges
kubectl get orders, challenges -o wide
NAME STATE DOMAIN REASON AGE
challenge.certmanager.k8s.io/tls-secret-155743219-0 pending mydomain.com pods "cm-acme-http-solver-gk2zx" is forbidden: minimum cpu usage per Container is 100m, but request is 10m. 26m
I have updated the resources limit the range and reinstalled cert-manager with helm. I am still getting this error.
I am not sure what goes wrong or show how to fix this.
Please let me know if you need anything. Thanks in advance!
The problem lays in cpu limits defined for specific pod.
You have to change minimum CPU limit in deployment configuration file. As you can see pod (cm-acme-http-solver) is requesting 100m CPU usage while minimum CPU usage defined for specific pod is *10m**. So change CPU limits in deployment configuration file from 100m to 10m or less or you can also increase CPU requests.
Take a look here: cert-manager-kubernetes, pod-min-cpu-request.
Useful article: resources-limits-kubernetes.