can't get certs working with cert-manager - ssl

Question

I can't seem to get cert-manager working:
$ kubectl get certificates -o wide
NAME READY SECRET ISSUER STATUS AGE
example-ingress False example-ingress letsencrypt-prod Waiting for CertificateRequest "example-ingress-2556707613" to complete 6m23s
$ kubectl get CertificateRequest -o wide
NAME READY ISSUER STATUS AGE
example-ingress-2556707613 False letsencrypt-prod Referenced "Issuer" not found: issuer.cert-manager.io "letsencrypt-prod" not found 7m7s
and in the logs i see:
I1025 06:22:00.117292 1 sync.go:163] cert-manager/controller/ingress-shim "level"=0 "msg"="certificate already exists for ingress resource, ensuring it is up to date" "related_resource_kind"="Certificate" "related_resource_name"="example-ingress" "related_resource_namespace"="default" "resource_kind"="Ingress" "resource_name"="example-ingress" "resource_namespace"="default"
I1025 06:22:00.117341 1 sync.go:176] cert-manager/controller/ingress-shim "level"=0 "msg"="certificate resource is already up to date for ingress" "related_resource_kind"="Certificate" "related_resource_name"="example-ingress" "related_resource_namespace"="default" "resource_kind"="Ingress" "resource_name"="example-ingress" "resource_namespace"="default"
I1025 06:22:00.117382 1 controller.go:135] cert-manager/controller/ingress-shim "level"=0 "msg"="finished processing work item" "key"="default/example-ingress"
I1025 06:22:00.118026 1 sync.go:361] cert-manager/controller/certificates "level"=0 "msg"="no existing CertificateRequest resource exists, creating new request..." "related_resource_kind"="Secret" "related_resource_name"="example-ingress" "related_resource_namespace"="default" "resource_kind"="Certificate" "resource_name"="example-ingress" "resource_namespace"="default"
I1025 06:22:00.147147 1 controller.go:129] cert-manager/controller/certificaterequests-issuer-venafi "level"=0 "msg"="syncing item" "key"="default/example-ingress-2556707613"
I1025 06:22:00.147267 1 sync.go:373] cert-manager/controller/certificates "level"=0 "msg"="created certificate request" "related_resource_kind"="Secret" "related_resource_name"="example-ingress" "related_resource_namespace"="default" "resource_kind"="Certificate" "resource_name"="example-ingress" "resource_namespace"="default" "request_name"="example-ingress-2556707613"
I1025 06:22:00.147284 1 controller.go:129] cert-manager/controller/certificaterequests-issuer-acme "level"=0 "msg"="syncing item" "key"="default/example-ingress-2556707613"
I1025 06:22:00.147273 1 conditions.go:200] Setting lastTransitionTime for CertificateRequest "example-ingress-2556707613" condition "Ready" to 2019-10-25 06:22:00.147254385 +0000 UTC m=+603.871617341
I1025 06:22:00.147392 1 conditions.go:200] Setting lastTransitionTime for CertificateRequest "example-ingress-2556707613" condition "Ready" to 2019-10-25 06:22:00.147380513 +0000 UTC m=+603.871743521
E1025 06:22:00.147560 1 pki.go:128] cert-manager/controller/certificates "msg"="error decoding x509 certificate" "error"="error decoding cert PEM block" "related_resource_kind"="Secret" "related_resource_name"="example-ingress" "related_resource_namespace"="default" "resource_kind"="Certificate" "resource_name"="example-ingress" "resource_namespace"="default" "secret_key"="tls.crt"
I1025 06:22:00.147620 1 conditions.go:155] Setting lastTransitionTime for Certificate "example-ingress" condition "Ready" to 2019-10-25 06:22:00.147613112 +0000 UTC m=+603.871976083
I1025 06:22:00.147731 1 controller.go:129] cert-manager/controller/certificaterequests-issuer-ca "level"=0 "msg"="syncing item" "key"="default/example-ingress-2556707613"
I1025 06:22:00.147765 1 conditions.go:200] Setting lastTransitionTime for CertificateRequest "example-ingress-2556707613" condition "Ready" to 2019-10-25 06:22:00.14776244 +0000 UTC m=+603.872125380
I1025 06:22:00.147912 1 controller.go:129] cert-manager/controller/certificaterequests-issuer-selfsigned "level"=0 "msg"="syncing item" "key"="default/example-ingress-2556707613"
I1025 06:22:00.147942 1 conditions.go:200] Setting lastTransitionTime for CertificateRequest "example-ingress-2556707613" condition "Ready" to 2019-10-25 06:22:00.147938966 +0000 UTC m=+603.872301909
I1025 06:22:00.147968 1 controller.go:129] cert-manager/controller/certificaterequests-issuer-vault "level"=0 "msg"="syncing item" "key"="default/example-ingress-2556707613"
I1025 06:22:00.148023 1 conditions.go:200] Setting lastTransitionTime for CertificateRequest "example-ingress-2556707613" condition "Ready" to 2019-10-25 06:22:00.148017945 +0000 UTC m=+603.872380906
i deployed cert-manager via the manifest:
https://github.com/jetstack/cert-manager/releases/download/v0.11.0/cert-manager.yaml
$ kubectl get clusterissuer letsencrypt-prod -o yaml
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"cert-manager.io/v1alpha2","kind":"ClusterIssuer","metadata":{"annotations":{},"name":"letsencrypt-prod"},"spec":{"acme":{"email":"me#me.com","privateKeySecretRef":{"name":"letsencrypt-prod"},"server":"https://acme-staging-v02.api.letsencrypt.org/directory","solvers":[{"http01":{"ingress":{"class":"nginx"}},"selector":{}}]}}}
creationTimestamp: "2019-10-25T06:27:06Z"
generation: 1
name: letsencrypt-prod
resourceVersion: "1759784"
selfLink: /apis/cert-manager.io/v1alpha2/clusterissuers/letsencrypt-prod
uid: 05831417-b359-42de-8298-60da553575f2
spec:
acme:
email: me#me.com
privateKeySecretRef:
name: letsencrypt-prod
server: https://acme-staging-v02.api.letsencrypt.org/directory
solvers:
- http01:
ingress:
class: nginx
selector: {}
status:
acme:
lastRegisteredEmail: me#me.com
uri: https://acme-staging-v02.api.letsencrypt.org/acme/acct/11410425
conditions:
- lastTransitionTime: "2019-10-25T06:27:07Z"
message: The ACME account was registered with the ACME server
reason: ACMEAccountRegistered
status: "True"
type: Ready
and my ingress is:
$ kubectl get ingress example-ingress -o yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
cert-manager.io/issuer: letsencrypt-prod
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"cert-manager.io/issuer":"letsencrypt-prod","kubernetes.io/ingress.class":"nginx","kubernetes.io/tls-acme":"true"},"name":"example-ingress","namespace":"default"},"spec":{"rules":[{"host":"example-ingress.example.com","http":{"paths":[{"backend":{"serviceName":"apple-service","servicePort":5678},"path":"/apple"},{"backend":{"serviceName":"banana-service","servicePort":5678},"path":"/banana"}]}}],"tls":[{"hosts":["example-ingress.example.com"],"secretName":"example-ingress"}]}}
kubernetes.io/ingress.class: nginx
kubernetes.io/tls-acme: "true"
creationTimestamp: "2019-10-25T06:22:00Z"
generation: 1
name: example-ingress
namespace: default
resourceVersion: "1758822"
selfLink: /apis/extensions/v1beta1/namespaces/default/ingresses/example-ingress
uid: 921b2e91-9101-4c3c-a0d8-3f871dafdd30
spec:
rules:
- host: example-ingress.example.com
http:
paths:
- backend:
serviceName: apple-service
servicePort: 5678
path: /apple
- backend:
serviceName: banana-service
servicePort: 5678
path: /banana
tls:
- hosts:
- example-ingress.example.com
secretName: example-ingress
status:
loadBalancer:
ingress:
- ip: x.y.z.a
any idea whats wrong? cheers,

Answer 1

Your ingress is referring to an issuer, but the issuer is a ClusterIssuer. Could that be the reason? I have a similar setup with Issuer instead of a ClusterIssuer and it is working.

Answer 2

I have done this implementation, you can follow this way -
Install jetstack from here
Then follow these steps from this stackoverflow post
Make one clusterIssuer or you can make individual issuer too, once you patch the hostname to ingress, then the tls-certificate in that namespace will be autogenerated by Jetstack after the acme-challenge validation
Kindly make sure to map the IP of loadbalancer nginx/traefik etc to DNS/hostname