Kubernetes Cert-Manager can't get http01 ACME challange to work - asp.net-core

Hello I m struggling to get the Cert-Manager work with let'sencrypt on my Azure AKS to secure and asp.net core web app.
I have a ClusterIssuer like that:
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
name: letsencryptstaging-issuer
spec:
acme:
server: https://acme-staging-v02.api.letsencrypt.org/directory
email: letsencryptstaging#prodibi.com
privateKeySecretRef:
name: letsencryptstaging-secret
solvers:
- http01:
ingress:
class: nginx
and I request a certificate like that:
apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
name: aks-prodibiv2-com-staging
spec:
secretName: aks-prodibiv2-com-staging-secret
duration: 2160h
renewBefore: 480h
organization:
- prodibiv2
dnsNames:
- aks.prodibiv2.com
issuerRef:
name: letsencryptstaging-issuer
kind: ClusterIssuer
I also have added the annotations to the ingress controller I would like to use
certmanager.k8s.io/acme-challenge-type: http01
certmanager.k8s.io/cluster-issuer: letsencryptstaging-issuer
In the following screenshot we can see that the certificate request is "Waiting to complete"
We can see also that we have two ingress controller and the one for the challenge seems to not have an IP, and the domain is pointing to the ingress-prodibiweb
if I try to put the domain in front of the .well-known path I get a 404 not found error.
So my guess is that the cert-manager is not configured properly to use the ingress-prodibiweb (that point to asp.net core webapp) or something like that. Any idea on what can I try to get it working?

Your ingress ip is private. No way for Let's encrypt to access.
You need to make sure you are using Nginx Ingress which you are using
- http01:
ingress:
class: nginx
Make sure your domain is point to right IP. (Ingress IP and also a Load Balancer IP)
Now is 20.50.42.93
https://mxtoolbox.com/SuperTool.aspx?action=a%3aaks.prodibiv2.com&run=toolpage
And dns01 also a solution to request SSL. You can give it a try if you have enough permissions.

Related

Cert manager doesn't get the challenge done

I'm setting up a k3s cluster for local development.
To be clear, I do not have a public IP address.
At this moment I'm looking for a solution to get the certificate process automated (via cert-manager).
In order to get this to work I've did the following:
Deployed k3s
Deployed cert-manager
Deployed traefik
Purchased a domain
Created a cloudflare account and added the domain there
Created an API token to do the acme challenge (based on https://cert-manager.io/docs/configuration/acme/dns01/cloudflare/)
Created a simple test website
When a add the test website I get the following error:
Found no Zones for domain _acme-challenge.. (neither in
the sub-domain noir in the SLD) please make sure your domain-entries
in the config are correct and the API is correctly setup with
Zone.read rights.
I have the following configuration:
ClusterIssuer
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt
spec:
acme:
email: my#emailaddress.com
server: https://acme-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name: letsencrypt
solvers:
- dns01:
cloudflare:
email: my#emailaddress.com
apiKeySecretRef:
name: cloudflare-api-key-secret
key: api-key
Test website
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: test-ingress
namespace: test
annotations:
kubernetes.io/ingress.class: "traefik"
cert-manager.io/cluster-issuer: letsencrypt
spec:
rules:
- host: test.<mydomain>
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: nginx-svc
port:
number: 80
tls:
- secretName: test.<mydomain>

Kubernetes Ingress can't find SSL secret created by CertManager

I have been trying to genereate an SSL certificate from Let's Encrypt to my domain but cert-manager creates the secret and adds a random suffix, causing the NGINX ingress controller to not be able to find it:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
cert-manager.io/issuer: scaleway
kubernetes.io/tls-acme: "true"
spec:
rules:
- host: <DOMAIN>
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: my-svc
port:
number: 80
tls:
- hosts:
- <DOMAIN>
secretName: tls-cert
Here is the error in the NGINX logs :
Error obtaining X.509 certificate: no object matching key "default/tls-cert" in local store
Error getting SSL certificate "default/storek8s.igesa.it": local SSL certificate default/storek8s.igesa.it was not found. Using default certificate
The secret is always created like this :
NAME TYPE DATA AGE
...
tls-secret-fjrm5 Opaque 1 60m
How do I disable adding the random suffix for the secret generated by Ingress ? Is there any workaround to solve this ?
At the moment I have too little information to give an exact answer. Are you for example using this Helm chart, what values does your values.yaml contain, and how did you setup your issuer?
Also, I know that tls secrets created by cert-manager should be of type kubernetes.io/tls and not Opaque as is the case in your situation. So there is definitly something going wrong there.

Kubernetes: Why are my acme challenges getting EOF/no response?

I'm setting up a Kubernetes cluster in AWS using Kops. I've got an nginx ingress controller, and I'm trying to use letsencrypt to setup tls. Right now I can't get my ingress up and running because my certificate challenges get this error:
Waiting for http-01 challenge propagation: failed to perform self check GET request 'http://critsit.io/.well-known/acme-challenge/[challengeId]': Get http://critsit.io/.well-known/acme-challenge/[challengeId]: EOF
I've got a LoadBalancer service that's taking public traffic, and the certificate issuer automatically creates 2 other services which don't have public IPs.
What am I doing wrong here? Is there some networking issue preventing the pods from finishing the acme flow? Or maybe something else?
Note: I have setup an A record in Route53 to direct traffic to the LoadBalancer.
> kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
cm-acme-http-solver-m2q2g NodePort 100.69.86.241 <none> 8089:31574/TCP 3m34s
cm-acme-http-solver-zs2sd NodePort 100.67.15.149 <none> 8089:30710/TCP 3m34s
default-http-backend NodePort 100.65.125.130 <none> 80:32485/TCP 19h
kubernetes ClusterIP 100.64.0.1 <none> 443/TCP 19h
landing ClusterIP 100.68.115.188 <none> 3001/TCP 93m
nginx-ingress LoadBalancer 100.67.204.166 [myELB].us-east-1.elb.amazonaws.com 443:30596/TCP,80:30877/TCP 19h
Here's my ingress setup:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: critsit-ingress
namespace: default
annotations:
kubernetes.io/ingress.class: "nginx"
cert-manager.io/acme-challenge-type: "http01"
cert-manager.io/cluster-issuer: "letsencrypt-prod"
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
tls:
- hosts:
- critsit.io
- app.critsit.io
secretName: letsencrypt-prod
rules:
- host: critsit.io
http:
paths:
- path: /
backend:
serviceName: landing
servicePort: 3001
And my certificate issuer:
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
spec:
acme:
# The ACME server URL
server: https://acme-v02.api.letsencrypt.org/directory
# Email address used for ACME registration
email: michael.vegeto#gmail.com
# Name of a secret used to store the ACME account private key
privateKeySecretRef:
name: letsencrypt-prod
# Enable the HTTP-01 challenge provider
solvers:
- http01:
ingress:
class: nginx
selector: {}
Update: I've noticed that my load balancer has all of the instances marked as OutOfOrder because they're failing health checks. I wonder if that's related to the issue.
Second update: I abandoned this route altogether, and rebuilt my networking/ingress system using Istio
The error message you are getting can mean a wide variety of issues. However, there are few things you can check/do in order to make it work:
Delete the Ingress, the certificates and the cert-manager fully. After that add them all back to make sure it installs clean. Sometimes stale certs or bad/multi Ingress pathing might be the issue. For example you can use Helm:
helm install my-nginx-ingress stable/nginx-ingress
helm repo add jetstack https://charts.jetstack.io
helm repo update
helm install cert-manager jetstack/cert-manager --namespace cert-manager --version v0.15.0 --set installCRDs=true
Make sure your traffic allows HTTP or has HTTPS with a trusted cert.
Check if hairpin mode of your loadbalancer and make sure it is working.
Add: nginx.ingress.kubernetes.io/ssl-redirect: "false" annotation to the Ingress rule. Wait a moment and see if valid cert will be created.
You can manually manually issue certificates in your Kubernetes cluster. To do so, please follow this guide.
The problem can solve itself in time. Currently if the self check fails, it
updates the status information with the reason (like: self check failed) and than
tries again later (to allow for propagation). This is an expected behavior.
This is an ongoing issue that is being tracked here and here.

Using ssl in Ingress without cert-manager

I'm trying to uses ssl in several Ingress on k8s. A first look up lead me to cert-manager but I can't make it work and I suspect that the cause is because my Cloud provider (ovh) is not supported.
I'm using kubernetes 1.17 and cert-manager 0.13.0
The first error I encountered was related to the web hook secret and no solutions worked for me.
Because of this I deployed cert-manager without web-hook but I still couldn't get a ClusterIssuer up and running. When I apply the following :
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
name: letsencrypt-staging
spec:
acme:
email: leonard.panichi#gmail.com
server: https://acme-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name: letsencrypt-staging
solvers:
- http01:
ingress:
class: nginx
a clusterissuer is created but there is no Status when I run describe on it.
So, after new searches (and more than a day of struggling), I found that ovh might not be compatible with cert-manager
which let me think that I'm loosing my time with this strategy.
Hence I'm looking for a new strategy.
How can I use certbot, there is a docker image certbot/certbot, to create and renew a few ssl certificates in kubernetes secrets in order to use them in my Ingress ? Is there any other way that do not requiere GKE, AWS, something simple, portable, production ready, etc...
Sincerely,
me

Cert-Manager Certificate Renewal process - How it is performed?

I am using cert-manager-v0.10.0 installed from its helm chart
I am using kong like ingress controller to manage the ingress operations.
So I have created a ClusterIssuer resource in order it could be contacted from an Ingress resource via kong-ingress controller.
The ClusterIssuer is this:
apiVersion: certmanager.k8s.io/v1alpha1
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
spec:
acme:
# The ACME server URL
server: https://acme-v02.api.letsencrypt.org/directory
# Email address used for ACME registration
email: username#mydomain.org
# Name of a secret used to store the ACME account private key
privateKeySecretRef:
name: letsencrypt-prod
# Enable the HTTP-01 challenge provider
solvers:
- http01:
ingress:
class: kong
The ingress resource that I am using is this.
You can see here, that I am pointing it to the ClusterIssuer created previously and
also I am pointing it to kong as an ingress controller, according to the kubernetes.io/ingress.class: "kong" annotation included there:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
# add an annotation indicating the issuer to use.
certmanager.k8s.io/cluster-issuer: letsencrypt-prod # letsencrypt-staging
kubernetes.io/ingress.class: "kong"
plugins.konghq.com: swaggerapi-customer-production-basic-auth, swaggerapi-customer-production-acl
name: production-customer-ingress-app
# namespace: default
spec:
rules:
- host: appprod.mydomain.org
http:
paths:
- backend:
serviceName: customer-production-app
servicePort: 80
path: /comcustomerpr
tls: # < placing a host in the TLS config will indicate a cert should be created
- hosts:
- appprod.mydomain.org
secretName: letsencrypt-prod # < cert-manager will store the created certificate in this secret.
So, when I create the Ingress resource above, the secretName referenced above in my ingress is created and also a certificate resource with the same name ... that is letsencrypt-prod.
It will be the certificate resource which receive the LetsEncrypt validation successful process ...
I got TLS encryption and everything is OK here.
But now, I want to know how will be the renewal process. Because I am pretty sure at the moment this renewal certificate process it does not to happen automatically ...
I was reading something here https://docs.cert-manager.io/en/latest/reference/certificates.html?highlight=renewal#certificate-duration-and-renewal-window and this documentation says that is necessary attach to the certificate resource created (kind:Certificate) the spec.duration and spec.renewBefore attributes of this way
spec:
secretName: example-tls
duration: 24h
renewBefore: 12h
If my certificate issued by LetsEncrypt has a 90 days as a default duration, how can I specify these spec.duration and spec.renewBefore attributes?
I want to get into in this concern, because my main question is I am not creating the certificate, it is created when the Ingress resource (above referenced) is executed.
How can I address the renewal process here with this approach that I am doing?
UPDATE
Do I need to create a specific kind:Certificate resource, referencing the secret that I got from LetsEncrypt?
I mean, something like this?
apiVersion: certmanager.k8s.io/v1alpha1
kind: Certificate
metadata:
name: letsencrypt-prod
spec:
secretName: letsencrypt-prod
dnsNames:
- mydomain.com
acme:
config:
- http01:
ingressClass: kong
domains:
- mydomain.com
issuerRef:
name: letsencrypt-prod
kind: ClusterIssuer
I have the doubt here, because currently I am not getting the certificate renewal action
since you have used the letsencrypt-prod issuer, and haven't done anything special/non-standard, the certificate renewal process will be completely automatic for you.
By default the letsencrypt certificates are valid fro 90-days, and renewed automatically every 30-days. If you don't have some strict requirements to use purchased certificates, or use some other specific Certificate Authority, this is a great option to use.
If you still have doubts then you can do the following to see for yourself. First decode the current certificates secret data and inspect the certificate contents with the openssl command. You'll be able to see the certificate expiry date, and make a note of that. Now if you subtract 59-days from that expiry date that should give you roughly the date that cert-manager will attempt to renew the certificate on. I add an extra day just to be safe we aren't too early. Then on that date repeat this process again; decoding the certificate secret, inspecting the certificate with the openssl command, and checking the certificate expiry date. You'll notice the expiry date for the certificate is different than before, hence it's was automatically renewed as we expected.
Hope this helps.