Cert-Manager Certificate creation stuck at Created new CertificateRequest resource - ssl

Question

I am using cert-manager v1.0.0 on GKE, I tried to use the staging environment for acme and it worked fine but when shifting to production I can find the created certificate stuck at Created new CertificateRequest resource and nothing changes after that
I expect to see the creation of the certificate to be succeeded and change the status of the certificate from false to true as happens in staging
Environment details::
Kubernetes version (v1.18.9):
Cloud-provider/provisioner (GKE):
cert-manager version (v1.0.0):
Install method (helm)
Here is my clusterIssuer yaml file
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: i-storage-ca-issuer-prod
namespace: default
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: MY_EMAIL_HERE
privateKeySecretRef:
name: i-storage-ca-issuer-prod
solvers:
- http01:
ingress:
class: gce
And here is my ingress yaml file
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: i-storage-core
namespace: i-storage
annotations:
kubernetes.io/ingress.global-static-ip-name: i-storage-core-ip
cert-manager.io/cluster-issuer: i-storage-ca-issuer-prod
labels:
app: i-storage-core
spec:
tls:
- hosts:
- i-storage.net
secretName: i-storage-core-prod-cert
rules:
- host: i-storage.net
http:
paths:
- path: /*
backend:
serviceName: i-storage-core-service
servicePort: 80
describe certificateRequest output
Name: i-storage-core-prod-cert-stb6l
Namespace: i-storage
Labels: app=i-storage-core
Annotations: cert-manager.io/certificate-name: i-storage-core-prod-cert
cert-manager.io/certificate-revision: 1
cert-manager.io/private-key-secret-name: i-storage-core-prod-cert-2pw26
API Version: cert-manager.io/v1
Kind: CertificateRequest
Metadata:
Creation Timestamp: 2020-10-31T15:44:57Z
Generate Name: i-storage-core-prod-cert-
Generation: 1
Managed Fields:
API Version: cert-manager.io/v1
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.:
f:cert-manager.io/certificate-name:
f:cert-manager.io/certificate-revision:
f:cert-manager.io/private-key-secret-name:
f:generateName:
f:labels:
.:
f:app:
f:ownerReferences:
.:
k:{"uid":"f3442651-3941-49af-81de-dcb937e8ba40"}:
.:
f:apiVersion:
f:blockOwnerDeletion:
f:controller:
f:kind:
f:name:
f:uid:
f:spec:
.:
f:issuerRef:
.:
f:group:
f:kind:
f:name:
f:request:
f:status:
.:
f:conditions:
Manager: controller
Operation: Update
Time: 2020-10-31T15:44:57Z
Owner References:
API Version: cert-manager.io/v1
Block Owner Deletion: true
Controller: true
Kind: Certificate
Name: i-storage-core-prod-cert
UID: f3442651-3941-49af-81de-dcb937e8ba40
Resource Version: 18351251
Self Link: /apis/cert-manager.io/v1/namespaces/i-storage/certificaterequests/i-storage-core-prod-cert-stb6l
UID: 83412862-903f-4fff-a736-f170e840748e
Spec:
Issuer Ref:
Group: cert-manager.io
Kind: ClusterIssuer
Name: i-storage-ca-issuer-prod
Request: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQ2ZUQ0NBV1VDQVFBd0FEQ0NBU0l3RFFZSktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQU5HcQovRDRVZlRhV0xFa01GUzdsdVN1RmRlR0NNVjJ4czREcG5Pem1HbjJxSlRUTlBnS2hHbGVEd0p2TkZIaTc5WWxHCmpYcjhjNDFHU1JUT2U4UDdUS3AvWXpBSUtxSXpPMllIeHY5VzA5bEZDWWQ4MTByMUNsOG5jb2NYa3BGZlAxMzAKZURlczZ6SUkwZW9ZTW1uRXQ3cmRUNk52dHhuZ1ZZVmlnai9VcXpxSkZ4NmlLa0R6V1VHK3lNcWtQM1ZKa1lYeApZUFNTNWZsWXlTdkI4emdxb3pnNUNJUndra09KTU1aRlNoWHVxYkpNZnJvQmR2YW9nQWtEYmZYSWs0SVRIaXlYCkV4UDFBaFdieGhPbndDd2h5bXpGWmgzSkZUZHhzeFdtRDZJMmp3MzV1SXZ1WWlIWEJ4VTBCMG50K3FYMVVWaWwKSkRlOFdNcTdjT3AzWmtlT2FHa0NBd0VBQWFBNE1EWUdDU3FHU0liM0RRRUpEakVwTUNjd0dBWURWUjBSQkJFdwpENElOYVMxemRHOXlZV2RsTG01bGREQUxCZ05WSFE4RUJBTUNCYUF3RFFZSktvWklodmNOQVFFTEJRQURnZ0VCCkFLMkhhSEQxd3dDZVFqS1diU1N0SFkxMm1Da1A1amQ0RnFmZFFYRG5XR3grK3FCWExGY0F4TVZhbVF2cStQK0gKLzExQjhvdlUydU9icGRHRktoak9aNDJsdjNNMVllRWk5UG5nS0RFdndCbER0Q0Vsa0lHQzV4T1ZENCtheVlmaApEMUI2L20vdEJsdlhYNS8zRDlyejJsTWNRSzRnSTNVQ3Mxd0Y0bmduQ3JYMEhoSDJEendheXI5d2QvY1V1clZlClloYS9HZjcyaEFCcGQxSmkrR2hKaGxzVDlGbTNVZVNUTi9OYkpVWmk4NkM1S1dTRW1DblNjV3dzWGNoVW1vVisKVHpGQmNhOEhqOUxsVFdJVVBSYVl0bFQ2TEhrUjVLUW1EL2tJRTZDajlidTNXMG9oWDZ2UC9CQ012SWdaTVZEUgoyeFVwY3lhUmJad2ttWTQ2MktNZ25wUT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUgUkVRVUVTVC0tLS0tCg==
Status:
Conditions:
Last Transition Time: 2020-10-31T15:44:57Z
Message: Waiting on certificate issuance from order i-storage/i-storage-core-prod-cert-stb6l-177980933: "pending"
Reason: Pending
Status: False
Type: Ready
Events: <none>
describe order output
Name: i-storage-core-prod-cert-stb6l-177980933
Namespace: i-storage
Labels: app=i-storage-core
Annotations: cert-manager.io/certificate-name: i-storage-core-prod-cert
cert-manager.io/certificate-revision: 1
cert-manager.io/private-key-secret-name: i-storage-core-prod-cert-2pw26
API Version: acme.cert-manager.io/v1
Kind: Order
Metadata:
Creation Timestamp: 2020-10-31T15:44:57Z
Generation: 1
Managed Fields:
API Version: acme.cert-manager.io/v1
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.:
f:cert-manager.io/certificate-name:
f:cert-manager.io/certificate-revision:
f:cert-manager.io/private-key-secret-name:
f:labels:
.:
f:app:
f:ownerReferences:
.:
k:{"uid":"83412862-903f-4fff-a736-f170e840748e"}:
.:
f:apiVersion:
f:blockOwnerDeletion:
f:controller:
f:kind:
f:name:
f:uid:
f:spec:
.:
f:dnsNames:
f:issuerRef:
.:
f:group:
f:kind:
f:name:
f:request:
f:status:
.:
f:authorizations:
f:finalizeURL:
f:state:
f:url:
Manager: controller
Operation: Update
Time: 2020-10-31T15:44:57Z
Owner References:
API Version: cert-manager.io/v1
Block Owner Deletion: true
Controller: true
Kind: CertificateRequest
Name: i-storage-core-prod-cert-stb6l
UID: 83412862-903f-4fff-a736-f170e840748e
Resource Version: 18351252
Self Link: /apis/acme.cert-manager.io/v1/namespaces/i-storage/orders/i-storage-core-prod-cert-stb6l-177980933
UID: 92165d9c-e57e-4d6e-803d-5d28e8f3033a
Spec:
Dns Names:
i-storage.net
Issuer Ref:
Group: cert-manager.io
Kind: ClusterIssuer
Name: i-storage-ca-issuer-prod
Request: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQ2ZUQ0NBV1VDQVFBd0FEQ0NBU0l3RFFZSktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQU5HcQovRDRVZlRhV0xFa01GUzdsdVN1RmRlR0NNVjJ4czREcG5Pem1HbjJxSlRUTlBnS2hHbGVEd0p2TkZIaTc5WWxHCmpYcjhjNDFHU1JUT2U4UDdUS3AvWXpBSUtxSXpPMllIeHY5VzA5bEZDWWQ4MTByMUNsOG5jb2NYa3BGZlAxMzAKZURlczZ6SUkwZW9ZTW1uRXQ3cmRUNk52dHhuZ1ZZVmlnai9VcXpxSkZ4NmlLa0R6V1VHK3lNcWtQM1ZKa1lYeApZUFNTNWZsWXlTdkI4emdxb3pnNUNJUndra09KTU1aRlNoWHVxYkpNZnJvQmR2YW9nQWtEYmZYSWs0SVRIaXlYCkV4UDFBaFdieGhPbndDd2h5bXpGWmgzSkZUZHhzeFdtRDZJMmp3MzV1SXZ1WWlIWEJ4VTBCMG50K3FYMVVWaWwKSkRlOFdNcTdjT3AzWmtlT2FHa0NBd0VBQWFBNE1EWUdDU3FHU0liM0RRRUpEakVwTUNjd0dBWURWUjBSQkJFdwpENElOYVMxemRHOXlZV2RsTG01bGREQUxCZ05WSFE4RUJBTUNCYUF3RFFZSktvWklodmNOQVFFTEJRQURnZ0VCCkFLMkhhSEQxd3dDZVFqS1diU1N0SFkxMm1Da1A1amQ0RnFmZFFYRG5XR3grK3FCWExGY0F4TVZhbVF2cStQK0gKLzExQjhvdlUydU9icGRHRktoak9aNDJsdjNNMVllRWk5UG5nS0RFdndCbER0Q0Vsa0lHQzV4T1ZENCtheVlmaApEMUI2L20vdEJsdlhYNS8zRDlyejJsTWNRSzRnSTNVQ3Mxd0Y0bmduQ3JYMEhoSDJEendheXI5d2QvY1V1clZlClloYS9HZjcyaEFCcGQxSmkrR2hKaGxzVDlGbTNVZVNUTi9OYkpVWmk4NkM1S1dTRW1DblNjV3dzWGNoVW1vVisKVHpGQmNhOEhqOUxsVFdJVVBSYVl0bFQ2TEhrUjVLUW1EL2tJRTZDajlidTNXMG9oWDZ2UC9CQ012SWdaTVZEUgoyeFVwY3lhUmJad2ttWTQ2MktNZ25wUT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUgUkVRVUVTVC0tLS0tCg==
Status:
Authorizations:
Challenges:
Token: EMTpMo_Jt5YkITiwk_lOuL66Xu_Q38scNMf1o0LPgvs
Type: http-01
URL: https://acme-v02.api.letsencrypt.org/acme/chall-v3/8230128790/0EcdqA
Token: EMTpMo_Jt5YkITiwk_lOuL66Xu_Q38scNMf1o0LPgvs
Type: dns-01
URL: https://acme-v02.api.letsencrypt.org/acme/chall-v3/8230128790/9chkYQ
Token: EMTpMo_Jt5YkITiwk_lOuL66Xu_Q38scNMf1o0LPgvs
Type: tls-alpn-01
URL: https://acme-v02.api.letsencrypt.org/acme/chall-v3/8230128790/BaReZw
Identifier: i-storage.net
Initial State: pending
URL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/8230128790
Wildcard: false
Finalize URL: https://acme-v02.api.letsencrypt.org/acme/finalize/100748195/5939190036
State: pending
URL: https://acme-v02.api.letsencrypt.org/acme/order/100748195/5939190036
Events: <none>

Answer 1

List all certificates that you have:
kubectl get certificate --all-namespaces
Try to figure out the problem using describe command:
kubectl describe certificate CERTIFICATE_NAME -n YOUR_NAMESPACE
The output of the above command contains the name of the associated certificate request. Dig into more details using describe command once again:
kubectl describe certificaterequest CERTTIFICATE_REQUEST_NAME -n YOUR_NAMESPACE
You may also want to troubleshoot challenges with the following command:
kubectl describe challenges --all-namespaces
In my case, to make it work, I had to replace ClusterIssuer with just Issuer for reasons explained in the comment.
Here is my Issuer manifest:
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: cert-manager-staging
namespace: YOUR_NAMESPACE
spec:
acme:
# You must replace this email address with your own.
# Let's Encrypt will use this to contact you about expiring
# certificates, and issues related to your account.
email: example#example.com
server: https://acme-staging-v02.api.letsencrypt.org/directory
privateKeySecretRef:
# Secret resource that will be used to store the account's private key.
name: cert-manager-staging-private-key
# Add a single challenge solver, HTTP01 using nginx
solvers:
- http01:
ingress:
class: nginx
Here is my simple Ingress manifest:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
cert-manager.io/issuer: cert-manager-staging
name: YOUR_NAME
namespace: YOUR_NAMESPACE
spec:
tls:
- hosts:
- example.com
secretName: example-com-staging-certificate
rules:
- host: example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: example.com
port:
number: 80