We are working on a on-prim k8 cluster (No native Load Balancer like cloud )
and exploring traefik for SSL termination and routing. We have few questions
1) does it support more than 1 certs ( no wild card for us) , can we configure 1 cert per route ?
2) can we listen on low ports , 443 for all the ingress traffic to the cluster ? We plan front-end the nodes with a on-prim global load balancer.
3) does all ingress configuration needs to be in ONE yaml ? can can we split configurations(route and ssl info) per application
4) We are using istio for east-west traffic , any issues with integration ?
Got response from #nicomengin
I hope these responses will help you:
1) does it support more than 1 certs ( no wild card for us),
Yes you can define many services per entrypoints statically (in the Traefik configuration) or dynamically (thanks to the TLS secrets in K8s)
can we configure 1 cert per route ?
The certificates are linked to entrypoints. As you can define many certificates per entrypoints, you can define a certificate for each (sub) domains. So, I'll say yes you can do that.
2) can we listen on low ports , 443 for all the ingress traffic to the cluster ? We plan front-end the nodes with a on-prim global load balancer.
Yes, you can define entrypoints on all the ports you need, then you can define if they are TLS or not
3) does all ingress configuration needs to be in ONE yaml ? can can we split configurations(route and ssl info) per application
Yes you can split your ingress rules in many YAML files
4) We are using istio for east-west traffic , any issues with integration ?
No issues you can use together Traefik for nourth-south and Istio for esat-west
Related
I am using an Ingress using Google-managed SSL certs mostly similar to what is described here:
https://cloud.google.com/kubernetes-engine/docs/how-to/managed-certs#setting_up_a_google-managed_certificate
However my backend service is a grpc service that is using HTTP2. According to the same documentation if I am using HTTP2 my backend needs to be "configured with SSL".
This sounds like I need a separate set of certificates for my backend service to configure it with SSL.
Is there a way to use the same Google managed certs here as well?
What are my other options here? I am using, Google managed certs for the Ingress not to manage any certs on my own, if I then use self signed certificates for my service, that kind of defeats the purpose.
i don't think it's required to create SSL for the backend services if you are terminating the HTTPS at LB level. You can attach your certs to at LB level and the backed-end will be HTTPS > HTTP.
You might need to create SSL/TLS new cert in case there is diff version ssl-protocols: TLSv1.2 TLSv1.3, Cipher set in your ingress controller configmap which you are using Nginx ingress controller, Kong etc.
If you are looking for End to End HTTPS traffic definitely you need to create a cert for the backend service.
You can also create/manage the Managed certificate or Custom cert with Cert manager the K8s secret and mount to deployment which will be used further by the service, in that case, no need to manage or create the certs. Ingress will passthrough the HTTPS request to service directly.
In this case, it will be an end-to-end HTTPS setup.
Update :
Note: To ensure the load balancer can make a correct HTTP2 request to
your backend, your backend must be configured with SSL. For more
information on what types of certificates are accepted, see Encryption
from the load balancer to the backends ." end to end tls seems to be a
requirement for HTTP2
This is my site https://findmeip.com it's running on HTTP2 and terminating the SSL/TLS at the Nginx level only.
Definitely, it's good to go with the suggested practice so you can use the ESP option from the Google, setting GKE ingress + ESP + grpc stack.
https://cloud.google.com/endpoints/docs/openapi/specify-proxy-startup-options?hl=tr
If not want to use ESP check above suggested :
You can Mount Managed certificate to
deployment which will be used further by the service, in that case, no
need to manage or create the certs. In other words, cert-manager will create/manage/re-new SSL/TLS on behalf of you in K8s secret which will used by service.
Google Managed Certificates can only be used for the frontend portion of the load balancer (aka client to LB). If you need encryption from the LB to the backends you will have use self-signed certificates or some other way to store said certificates on GKE as secrets and configuring the Ingress to connect to the backend using these secrets.
Like this https://cloud.google.com/kubernetes-engine/docs/concepts/ingress-xlb#setting_up_https_tls_between_client_and_load_balancer
We have 10 different kubernetes pods which runs inside a private VPN, this pods are HTTP serving endpoints(not HTTPS). But this services would interact with HTTPS serving endpoints. Logically to make call to HTTP-S serving endpoints from a HTTP serving pod , the SSL server certificate trust is required. Hence we decided to store the SSL certificates inside each HTTP Service pods to make call to HTTPS serving pods.
I am wondering is there are any alternative approaches for managing SSL certificates across different pods in Kubernetes cluster? How about kubeadm for K8s certificate management ... any suggestions ?
This is more of a general SSL certificate question rather than specific to Kubernetes.
If the containers/pods providing the HTTPS endpoint already have their SSL correctly configured and the SSL certificate you are using was purchased/generated from a known, trusted CA (like letsencrypt or any one of the known, trusted certificate companies out there) then there is no reason your other container apps that are making connections to your HTTPS endpoint serving pods would need anything special stored in them.
The only exception to this is if you have your own private CA and you've generated certificates on that internally and are installing them in your HTTPS serving containers. (Or if you are generating self-signed certs). Your pods/containers connecting to the https endpoints would then need to know about the CA certificate. Here is a stackoverflow question/answer that deals with this scenario:
How do I add a CA root certificate inside a docker image?
Lastly, there are better patterns to manage SSL in containers and container schedulers like Kubernetes. It all depends on your design/architecture.
Some general ideas:
Terminate SSL at a load balancer before traffic hits your pods. The load balancer then handles the traffic from itself to the pods as HTTP, and your clients terminate SSL at the Load Balancer. (This doesn't really tackle your specific use case though)
Use something like Hashicorp Vault as an internal CA, and use automation around this product and Kubernetes to manage certificates automatically.
Use something like cert-manager by jetstack to manage SSL in your kubernetes environment automatically. It can connect to a multitude of 'providers' such as letsencrypt for free SSL. https://github.com/jetstack/cert-manager
Hope that helps.
I have a k8s cluster that sits behind a load balancer. The request for myapisite.com passes through the LB and is routed by k8s to the proper deployment, getting the SSL cert from the k8s load balancer ingress, which then routes to the service ingress, like so:
spec:
rules:
- host: myapisite.com
http:
paths:
- backend:
serviceName: ingress-605582265bdcdcee247c11ee5801957d
servicePort: 80
path: /
tls:
- hosts:
- myapisite.com
secretName: myapisitecert
status:
loadBalancer: {}
So my myapisite.com resolves on HTTPS correctly.
My problem is that, while maintaining the above setup (if possible), I need to be able to go to my local service endpoints within the same namespace on HTTPS, i.e. from another pod I should be able to curl or wget the following without a cert error:
https:\\myapisite.namespace.svc.cluster.local
Even if I were interested in not terminating SSL until the pod level, creating a SAN entry on the cert for a .local address is not an option, so that solution is not viable.
Is there some simple way I'm missing to make all local DNS trusted in k8s? Or some other solution here that's hopefully not a reinvention of the wheel? I am using kubernetes version 1.11 with CoreDNS.
Thanks, and sorry in advance if this is a dumb question.
If your application can listen on both HTTP and HTTPS, you can configure both. Meaning you will be able to access via both HTTP and HTTPS by your preference. Now, how you create and distribute certificate is a different story, but you must solve it on your own (probably by using your own CA and storing cert/key in secret). Unless you want to use something like Istio and its mutual tls support to secure traffic between services.
While you write what you want to achieve, we don't really know why. The reason for this need might actually help to suggest the best solution
So I want to have this:
/ Nginx1 (SSL)
HAProxy-- Nginx2 (SSL)
\ Nginx3 (SSL)
But I have questions:
How do I update Letsencrypt certs on all nodes?
If I can't do this with certbot (+some config) - how do you do this? Maybe some distributed k/v storages?
The best thing is to use HTTP only services (not HTTPS) on Nginx nodes and configure SSL on balancer.
Options:
Traefik. Can be configured to auto update LetsEncrypt certs.
Fabio. Also can be configured to use SSL certs. (I've used Hashicorp Vault to store them). Need to configure updates myself.
Those 2 integrate well with service discovery tools like Consul.
Background:
I had 1 instance created for EC2
I had my domain pointing to this instance
I had SSL installed for this instance (things were running great)
Furthermore:
I opted to create a second instance (using custom AMI from first instance)
I create a load balancer (things were working great -- from what i can tell...)
Question:
Why do I need to install the SSL certificate on the load balancer when it seems to have already been working?
I would presume when you say load balancer, you are refering to AWS ELB. If this is no the case, then disregard my answer.
Well the Best practice is to install SSL certs on load balancer and do the SSL termination there. Let the load balacner do SSL encryption/decryption so that your web server can do what they do the BEST...serving hte web pages.
Why do I need to install the SSL certificate on the load balancer
Now, technically You set is fine and you don't have to install SSL on load balancer. But then you have to use TCP Load Balancing feature of AWS ELB where in ELB will simply accespt traffic on 443 and will forward it to Web servers on 443. And then let your web servers do the SSL work.
I think this what you are looking for.