I am trying to deploy the "cert-manager" (https://github.com/jetstack/cert-manager) project which is the successor to "kube-lego". I'm finding that the certificates don't match what is being created, and I'm wondering if anybody else has tried this before.
I am creating a tls secretName with "monitoring-xxx-com", and in the ingress-nginx logs I find that it's trying to search for namespace/monitoring-xxx-com and not finding what it expects.
I am wondering whether this is because ingress-nginx is trying to use the pods namespace automatically and cert-manager is creating certs without a namespace, therefore that's why the cert can never be found.
error obtaining PEM from secret kube-system/monitoring-xxx-com: error
retrieving secret kube-system/monitoring-xxx-com: secret kube-
system/monitoring-xxx-com was not found
and in the certificate created by "cert-manager":
Issuer Ref:
Kind: ClusterIssuer
Name: letsencrypt-staging
Secret Name: monitoring-xxx-com
The secret and the nginx ingress controller are in a different namespace, there is an option where you can set the certificate from another namespace.
https://github.com/kubernetes/ingress-nginx/blob/master/docs/user-guide/cli-arguments.md
--default-ssl-certificate string Name of the secret
that contains a SSL certificate to be used as default for a HTTPS catch-all server.
Takes the form <namespace>/<secret name>.
To find the namespace of your secret:
kubectl describe secrets/monitoring-xxx-com
Using the default-ssl-certificate in the deployment template
spec:
template:
spec:
containers:
- args:
- /nginx-ingress-controller
- "--default-backend-service=$(POD_NAMESPACE)/default-http-backend"
- "--default-ssl-certificate=$(POD_NAMESPACE)/tls-certificate"
Related
I've created a certificate for the commonName "vault-lab.company.com" within Certificate Manager in the Istio namespace.
I've then used Reflector to copy that secret across to the Vault namespace, like so:
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: vault-lab.company.com-cert
namespace: istio-system
spec:
secretName: vault-lab.company.com-cert
commonName: vault-lab.company.com
dnsNames:
- vault-lab.company.com
issuerRef:
name: letsencrypt-prod-istio
kind: ClusterIssuer
secretTemplate:
annotations:
reflector.v1.k8s.emberstack.com/reflection-allowed: "true"
reflector.v1.k8s.emberstack.com/reflection-allowed-namespaces: "vault" # Control destination namespaces
reflector.v1.k8s.emberstack.com/reflection-auto-enabled: "true" # Auto create reflection for matching namespaces
reflector.v1.k8s.emberstack.com/reflection-auto-namespaces: "vault" # Control auto-reflection namespaces
The secret is successfully mounted via the volumes and volumeMounts section of the values.yaml for Vault:
volumes:
- name: vault-lab-cert
secret:
secretName: vault-lab.company.com-cert
volumeMounts:
mountPath: /etc/tls
readOnly: true
And in reading https://github.com/hashicorp/vault/issues/212, I've set the following as well in the listener configuration:
config: |
ui = false
listener "tcp" {
tls_disable = false
address = "0.0.0.0:8200"
tls_cert_file = "/etc/tls/tls.crt"
tls_key_file = "/etc/tls/tls.key"
}
api_addr = "https://vault-lab.company.com:8200"
cluster_addr = "https://vault-lab.company.com:8201"
However, I'm still seeing:
Get "https://127.0.0.1:8200/v1/sys/seal-status": x509: cannot validate certificate for 127.0.0.1 because it doesn't contain any IP SANs
When running:
kubectl exec vault-0 -- vault status
Interestingly, if I describe the vault-0 pod via kubectl, I see:
VAULT_ADDR: https://127.0.0.1:8200
VAULT_API_ADDR: https://$(POD_IP):8200
What could I be missing?
Is there something more I need to do if the certificate is configured via Cert Manager?
There's not a great deal of documentation on how to set this up at all.
When you run vault status, the binary runs as a client, unaware of the server configuration, even if it's running on the same machine or container. That means vault status can't read the listener stanza of your configuration file. It defaults to https://127.0.0.1:8200 that is missing from your certificate. The solution is not to add this IP address, but to tell Vault CLI where to find the server.
You can confirm that this is the problem with this command (should work, assuming your certificate is OK):
kubectl exec vault-0 -- vault status --address https://vault-lab.company.com:8200
For the client to pick up the API address automatically, set the VAULT_ADDR environment variable in your container to:
VAULT_ADDR=https://vault-lab.company.com:8200
I have been trying to genereate an SSL certificate from Let's Encrypt to my domain but cert-manager creates the secret and adds a random suffix, causing the NGINX ingress controller to not be able to find it:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
cert-manager.io/issuer: scaleway
kubernetes.io/tls-acme: "true"
spec:
rules:
- host: <DOMAIN>
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: my-svc
port:
number: 80
tls:
- hosts:
- <DOMAIN>
secretName: tls-cert
Here is the error in the NGINX logs :
Error obtaining X.509 certificate: no object matching key "default/tls-cert" in local store
Error getting SSL certificate "default/storek8s.igesa.it": local SSL certificate default/storek8s.igesa.it was not found. Using default certificate
The secret is always created like this :
NAME TYPE DATA AGE
...
tls-secret-fjrm5 Opaque 1 60m
How do I disable adding the random suffix for the secret generated by Ingress ? Is there any workaround to solve this ?
At the moment I have too little information to give an exact answer. Are you for example using this Helm chart, what values does your values.yaml contain, and how did you setup your issuer?
Also, I know that tls secrets created by cert-manager should be of type kubernetes.io/tls and not Opaque as is the case in your situation. So there is definitly something going wrong there.
I'm missing some form of configuration/knowledge when configuring an ingress through Helm for a chart.
I want to enable TLS for an existing chart which has the following values:
ingress:
enabled: false
annotations: {}
# kubernetes.io/ingress.class: nginx
# kubernetes.io/tls-acme: "true"
path: /
hosts:
- chart-example.local
tls: []
# - secretName: chart-example-tls
# hosts:
# - chart-example.local
From what I've read on the Traefik docs you can't use LetsEncrypt certificates in k8s secrets:
Only TLS certificates provided by users can be stored in Kubernetes Secrets. Let's Encrypt certificates cannot be managed in Kubernets Secrets yet.
What are my options here, do I overwrite the ingress and potentially mess with future helm upgrades or is annotations the way to go? If so an example would be great.
Only TLS certificates provided by users can be stored in Kubernetes
Secrets. Let's Encrypt certificates cannot be managed in Kubernets
Secrets yet.
It was discussed few times here, on stack. Maybe it will help you.
The most recommended solution is to use cert-manager instead of LetsEncrypt.
1. Traefik Ingress (Kubernetes) not receiving letsencrypt certificates
2. Let's Encrypt on Traefik with Helm
Also here is How to easily(ish!) get SSL/TLS configured for your web hosting needs using Traefik and cert-manager on Kubernetes article for you that shows everything is in details.
I'm trying to uses ssl in several Ingress on k8s. A first look up lead me to cert-manager but I can't make it work and I suspect that the cause is because my Cloud provider (ovh) is not supported.
I'm using kubernetes 1.17 and cert-manager 0.13.0
The first error I encountered was related to the web hook secret and no solutions worked for me.
Because of this I deployed cert-manager without web-hook but I still couldn't get a ClusterIssuer up and running. When I apply the following :
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
name: letsencrypt-staging
spec:
acme:
email: leonard.panichi#gmail.com
server: https://acme-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name: letsencrypt-staging
solvers:
- http01:
ingress:
class: nginx
a clusterissuer is created but there is no Status when I run describe on it.
So, after new searches (and more than a day of struggling), I found that ovh might not be compatible with cert-manager
which let me think that I'm loosing my time with this strategy.
Hence I'm looking for a new strategy.
How can I use certbot, there is a docker image certbot/certbot, to create and renew a few ssl certificates in kubernetes secrets in order to use them in my Ingress ? Is there any other way that do not requiere GKE, AWS, something simple, portable, production ready, etc...
Sincerely,
me
I am using cert-manager-v0.10.0 installed from its helm chart
I am using kong like ingress controller to manage the ingress operations.
So I have created a ClusterIssuer resource in order it could be contacted from an Ingress resource via kong-ingress controller.
The ClusterIssuer is this:
apiVersion: certmanager.k8s.io/v1alpha1
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
spec:
acme:
# The ACME server URL
server: https://acme-v02.api.letsencrypt.org/directory
# Email address used for ACME registration
email: username#mydomain.org
# Name of a secret used to store the ACME account private key
privateKeySecretRef:
name: letsencrypt-prod
# Enable the HTTP-01 challenge provider
solvers:
- http01:
ingress:
class: kong
The ingress resource that I am using is this.
You can see here, that I am pointing it to the ClusterIssuer created previously and
also I am pointing it to kong as an ingress controller, according to the kubernetes.io/ingress.class: "kong" annotation included there:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
# add an annotation indicating the issuer to use.
certmanager.k8s.io/cluster-issuer: letsencrypt-prod # letsencrypt-staging
kubernetes.io/ingress.class: "kong"
plugins.konghq.com: swaggerapi-customer-production-basic-auth, swaggerapi-customer-production-acl
name: production-customer-ingress-app
# namespace: default
spec:
rules:
- host: appprod.mydomain.org
http:
paths:
- backend:
serviceName: customer-production-app
servicePort: 80
path: /comcustomerpr
tls: # < placing a host in the TLS config will indicate a cert should be created
- hosts:
- appprod.mydomain.org
secretName: letsencrypt-prod # < cert-manager will store the created certificate in this secret.
So, when I create the Ingress resource above, the secretName referenced above in my ingress is created and also a certificate resource with the same name ... that is letsencrypt-prod.
It will be the certificate resource which receive the LetsEncrypt validation successful process ...
I got TLS encryption and everything is OK here.
But now, I want to know how will be the renewal process. Because I am pretty sure at the moment this renewal certificate process it does not to happen automatically ...
I was reading something here https://docs.cert-manager.io/en/latest/reference/certificates.html?highlight=renewal#certificate-duration-and-renewal-window and this documentation says that is necessary attach to the certificate resource created (kind:Certificate) the spec.duration and spec.renewBefore attributes of this way
spec:
secretName: example-tls
duration: 24h
renewBefore: 12h
If my certificate issued by LetsEncrypt has a 90 days as a default duration, how can I specify these spec.duration and spec.renewBefore attributes?
I want to get into in this concern, because my main question is I am not creating the certificate, it is created when the Ingress resource (above referenced) is executed.
How can I address the renewal process here with this approach that I am doing?
UPDATE
Do I need to create a specific kind:Certificate resource, referencing the secret that I got from LetsEncrypt?
I mean, something like this?
apiVersion: certmanager.k8s.io/v1alpha1
kind: Certificate
metadata:
name: letsencrypt-prod
spec:
secretName: letsencrypt-prod
dnsNames:
- mydomain.com
acme:
config:
- http01:
ingressClass: kong
domains:
- mydomain.com
issuerRef:
name: letsencrypt-prod
kind: ClusterIssuer
I have the doubt here, because currently I am not getting the certificate renewal action
since you have used the letsencrypt-prod issuer, and haven't done anything special/non-standard, the certificate renewal process will be completely automatic for you.
By default the letsencrypt certificates are valid fro 90-days, and renewed automatically every 30-days. If you don't have some strict requirements to use purchased certificates, or use some other specific Certificate Authority, this is a great option to use.
If you still have doubts then you can do the following to see for yourself. First decode the current certificates secret data and inspect the certificate contents with the openssl command. You'll be able to see the certificate expiry date, and make a note of that. Now if you subtract 59-days from that expiry date that should give you roughly the date that cert-manager will attempt to renew the certificate on. I add an extra day just to be safe we aren't too early. Then on that date repeat this process again; decoding the certificate secret, inspecting the certificate with the openssl command, and checking the certificate expiry date. You'll notice the expiry date for the certificate is different than before, hence it's was automatically renewed as we expected.
Hope this helps.