Traefik V2 get wildcard certificate - traefik

I have basic setup running after following this tutorial
But struggling to get wildcard certificate for domain from Let's Encrypt.
traefik configuration :
traefik.toml: |
## static configuration
[global]
checkNewVersion = true
[entryPoints]
[entryPoints.web]
address = ":80"
[entryPoints.websecure]
address = ":443"
[providers]
[providers.kubernetesCRD]
[providers.file]
directory = "/etc/traefik/providers/"
watch = true
[log]
level = "INFO"
[accessLog]
[api]
insecure = true
dashboard = true
debug = true
[metrics]
[metrics.prometheus]
buckets = [0.1,0.3,1.2,5.0]
addEntryPointsLabels = true
addServicesLabels = true
entryPoint = "web"
[ping]
entryPoint = "web"
[certificatesResolvers]
[certificatesResolvers.default]
[certificatesResolvers.default.acme]
email = "admin#domain.com"
caServer = "https://acme-v02.api.letsencrypt.org/directory"
storage = "acme.json"
[certificatesResolvers.default.acme.dnsChallenge]
provider = "route53"
delayBeforeCheck = 0
resolvers = ["1.1.1.1:53", "8.8.8.8:53"]
dynamic.toml: |
## dynamic configuration
(Empty)
and route config :
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: traefik-admin
namespace: kube-system
spec:
entryPoints:
- web
routes:
- match: Host(`traefik.domain.ca`)
kind: Rule
services:
- name: traefik
port: 8080
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: whoami-notls
namespace: kube-system
spec:
entryPoints:
- web
routes:
- match: Host(`traefik.domain.ca`) && PathPrefix(`/notls`)
kind: Rule
services:
- name: whoami
port: 80
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: whoami-tls
namespace: kube-system
spec:
entryPoints:
- websecure
routes:
- match: Host(`traefik.domain.ca`) && PathPrefix(`/tls`)
kind: Rule
services:
- name: whoami
port: 80
tls:
certResolver: default
I am able to get certificate for traefik.domain.ca but need to get wildcard certificate for entire domain (*.domain.ca). I am not able find any direct reference config for this.
What I am missing here ?

Updating the tls block worked.
tls:
certResolver: default
domains:
- main: dev.domain.ca
sans:
- "dev.domain.ca"
- "*.dev.domain.ca"

Related

Securing grafana ingress with tls in kube-prometheus-stack values.yaml and make grafana available via https

I am using kube-prometheus-stack to monitor my system in gcp. Due to new requirements all my ingress need to be secured with tls. As a first step I wanted to make the grafana webpage available via https.
I created a tls secret and updated my values.yaml. After helm upgrade everything seems to work fine but page is still available via http only.
Hope you can support me here.
grafana:
enabled: true
namespaceOverride: ""
## Deploy default dashboards.
##
defaultDashboardsEnabled: true
adminPassword: prom-operator
ingress:
## If true, Grafana Ingress will be created
##
enabled: true
## Annotations for Grafana Ingress
##
# annotations: {
# kubernetes.io/ingress.class: gce-internal
# kubernetes.io/tls-acme: "true"
# }
## Labels to be added to the Ingress
##
labels: {}
## Hostnames.
## Must be provided if Ingress is enable.
##
# hosts:
# - grafana.domain.com
hosts: []
## Path for grafana ingress
# path: /*
## TLS configuration for grafana Ingress
## Secret must be manually created in the namespace
##
tls:
- secretName: monitoring-tls-secret
# hosts:
# - grafana.example.com
in the meantime I decided to create the ingress a different way.
I created a ssl-certificate and try to use that instead.
When starting up I get the failure down below. Which is strange as kubernetes.io/ingress.allow-http is configured.
kubectl describe ingress monitoring-cl2-grafana -n monitoring-cl2
Name: monitoring-cl2-grafana
Namespace: monitoring-cl2
Address: x.x.x.x
Default backend: default-http-backend:80 (y.y.y.y:8080)
Rules:
Host Path Backends
---- ---- --------
*
/* monitoring-cl2-grafana:80 (<deleted>)
Annotations: ingress.gcp.kubernetes.io/pre-shared-cert: monitoring-ssl
ingress.kubernetes.io/backends:
{"k8s1-613c3440-kube-system-default-http-backend-80-240d1018":"HEALTHY","k8s1-613c3440-mtx-monitoring--mtx-monitoring-cl2-gra-8-f146f2b2":...
ingress.kubernetes.io/https-forwarding-rule: k8s2-fs-3s1rnwzg-monitoring--monitoring-cl2-gr-hgx28ojy
ingress.kubernetes.io/https-target-proxy: k8s2-ts-3s1rnwzg-monitoring--monitoring-cl2-gr-hgx28ojy
ingress.kubernetes.io/ssl-cert: monitoring-ssl
ingress.kubernetes.io/url-map: k8s2-um-3s1rnwzg-monitoring--monitoring-cl2-gr-hgx28ojy
kubernetes.io/ingress.allow-http: false
kubernetes.io/ingress.class: gce-internal
kubernetes.io/ingress.global-static-ip-name: grafana-cl2
meta.helm.sh/release-name: monitoring-cl2
meta.helm.sh/release-namespace: monitoring-cl2
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning Sync 34m (x12 over 35m) loadbalancer-controller Error syncing to GCP: error running load balancer syncing routine: loadbalancer 3s1rnwzg-mtx-monitoring--monitoring-cl2-gr-hgx28ojy does not exist: error invalid internal ingress https config
Warning WillNotConfigureFrontend 26m (x18 over 35m) loadbalancer-controller gce-internal Ingress class does not currently support both HTTP and HTTPS served on the same IP (kubernetes.io/ingress.allow-http must be false when using HTTPS).
Normal Sync 3m34s loadbalancer-controller TargetProxy "k8s2-ts-3s1rnwzg-monitoring--monitoring-cl2-gr-hgx28ojy" certs updated
Normal Sync 3m29s (x9 over 35m) loadbalancer-controller Scheduled for sync
grafana:
image:
repository: europe-west3-docker.pkg.dev/<deleted info>/grafana
tag: 7.5.5
sha: ""
sidecar:
image:
repository: europe-west3-docker.pkg.dev/<deleted info>/prometheus/k8s-sidecar
tag: 1.10.7
sha: ""
imagePullPolicy: IfNotPresent
service:
enabled: true
type: NodePort
annotations: {
cloud.google.com/neg: '{"ingress": true}'
}
labels: {}
portName: service
ingress:
enabled: true
path: /*
pathType: ImplementationSpecific
annotations: {
ingress.gcp.kubernetes.io/pre-shared-cert: "monitoring-ssl",
kubernetes.io/ingress.allow-http: "false",
kubernetes.io/ingress.class: "gce-internal",
kubernetes.io/ingress.global-static-ip-name: "grafana-cl2"
}
WORKING NOW WITH FOLLOWING CONFIG
grafana:
image:
repository: europe-west3-docker.pkg.dev/del/mtx-monitoring/prometheus/grafana
tag: 7.5.5
sha: ""
sidecar:
image:
repository: europe-west3-docker.pkg.dev/del/mtx-monitoring/prometheus/k8s-sidecar
tag: 1.10.7
sha: ""
imagePullPolicy: IfNotPresent
service:
enabled: true
type: NodePort
# port: 80
# targetPort: 3000
annotations: {
cloud.google.com/neg: '{"ingress": true}'
}
labels: {}
portName: service
ingress:
enabled: true
path: /*
pathType: ImplementationSpecific
annotations: {
ingress.gcp.kubernetes.io/pre-shared-cert: "monitoring-ssl",
kubernetes.io/ingress.allow-http: "false",
kubernetes.io/ingress.class: "gce-internal",
kubernetes.io/ingress.global-static-ip-name: "grafana-cl2"
}
spec:
rules:
- host: grafana.monitoring.com
http:
paths:
- backend:
service:
name: mtx-monitoring-cl2-grafana
port:
number: 80

how to config traefik 2.2 forward url and remove keywords

I want to forward my request using treafik 2.2 and config my ingress like this:
kind: Ingress
apiVersion: extensions/v1beta1
metadata:
name: soa-illidan-forward
namespace: dabai-pro
selfLink: /apis/extensions/v1beta1/namespaces/dabai-pro/ingresses/soa-illidan-forward
uid: b0778435-4fff-4e78-b85f-a534833b73e0
resourceVersion: '322554'
generation: 1
creationTimestamp: '2020-06-07T11:04:43Z'
spec:
rules:
- host: manage.example.net
http:
paths:
- path: /service
backend:
serviceName: zuul-service
servicePort: 8088
status:
loadBalancer: {}
but the forward still using the key word service and forward to backend, how to remove the keywards service when forward request? I also tried this:
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: pro-manage-route
namespace: dabai-pro
spec:
entryPoints:
- web
routes:
- match: Host(`manage.example.net`)
kind: Rule
services:
- name: be-illidan-frontend
port: 80
- match: Host(`manage.example.net`) && PathPrefix(`/service`)
kind: Rule
services:
- name: zuul-service
port: 8088
but still not work. and this is my request path:
https://manage.example.net/service/illidan/report/user/captcha
and I want forward to backend zuul-service's url is /illidan/report/user/captcha. not /service/illidan/report/user/captcha.
just using traefik StripPrefix middleware, for example:
http:
middlewares:
test-stripprefix:
stripPrefix:
prefixes:
- "/foobar"
- "/fiibar"
for more detail, read the traefik docs.

Kubernetes tls for tcp subdomain

I am trying to create TCP entrypoint with Traefik v2. I have following in traefik.toml
[global]
checkNewVersion = true
[entryPoints]
[entryPoints.web]
address = ":80"
[entryPoints.websecure]
address = ":443"
[entryPoints.tcp5050]
address = ":5050"
and following IngressRouteTCP
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:
name: mosquitto-external
namespace: dev
spec:
entryPoints:
- tcp5050
routes:
- match: HostSNI(`*`)
kind: Rule
services:
- name: mosquitto
port: 5050
tls:
Above one works perfectly. But can't reuse entryPoints tcp5050 for any other host. I tried to make IngressRouteTCP specific to HostSNI like this
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:
name: mosquitto-external
namespace: dev
spec:
entryPoints:
- tcp5050
routes:
- match: HostSNI(`mosquitto.dev.domain.com`)
kind: Rule
services:
- name: mosquitto
port: 5050
tls:
passthrough: true
But it this doesn't work at all.
What I am missing here ?

Traefik v2 Middlewares not being detected

Middlewares are not being detected and therefore paths are not being stripped resulting in 404s in the backend api.
Middleware exists in k8s apps namespace
$ kubectl get -n apps middlewares
NAME AGE
traefik-middlewares-backend-users-service 1d
configuration for middleware and ingress route
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
annotations:
kubernetes.io/ingress.class: traefik
name: apps-services
namespace: apps
spec:
entryPoints:
- web
routes:
- kind: Rule
match: Host(`example.com`) && PathPrefix(`/users/`)
middlewares:
- name: traefik-middlewares-backend-users-service
priority: 0
services:
- name: backend-users-service
port: 8080
---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
name: traefik-middlewares-backend-users-service
namespace: apps
spec:
stripPrefix:
prefixes:
- /users
Static configuration
global:
checkNewVersion: true
sendAnonymousUsage: true
entryPoints:
http:
address: :80
traefik:
address: :8080
providers:
providersThrottleDuration: 2s
kubernetesIngress: {}
api:
# TODO: make this secure later
insecure: true
ping:
entryPoint: http
log: {}
Traefik dasboard has no middlewares
Spring Boot 404 page. Route is on example.com/actuator/health
The /users is not being stripped. This worked for me in traefik v1 perfectly fine.
Note: actual domain has been replaced with example.com and domain.com in the examples.
To get this working, I had to:
Add the Kubernetes CRD provider with the namespaces where the custom k8s CRDs for traefik v2 exist
Add TLSOption resource definition
Update cluster role for traefik to have permissions for listing and watching new v2 resources
Make sure all namespaces with new resources are configured
Traefik Static Configuration File
providers:
providersThrottleDuration: 2s
kubernetesCRD:
namespaces:
- apps
- traefik
TLSOption CRD
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: tlsoptions.traefik.containo.us
spec:
group: traefik.containo.us
version: v1alpha1
names:
kind: TLSOption
plural: tlsoptions
singular: tlsoption
scope: Namespaced
Updated Static Configuration for Traefik
global:
checkNewVersion: true
sendAnonymousUsage: true
entryPoints:
http:
address: :80
traefik:
address: :8080
providers:
providersThrottleDuration: 2s
kubernetesCRD:
namespaces:
- apps
- traefik
api:
# TODO: make this secure later
insecure: true
ping:
entryPoint: http
log: {}

Can't get real user's IP from X-Forwarded-For

I'm running Traefik 1.7.3 on a single node Kubernetes cluster and I'm trying to get the real user IP from the X-Forwarded-For header but what I get instead is X-Forwarded-For: 10.244.0.1 which is an IP in my k8s cluster.
Here's my Traefik deployment and service:
---
apiVersion: v1
kind: ConfigMap
metadata:
name: traefik-conf
data:
traefik.toml: |
# traefik.toml
debug = true
logLevel = "DEBUG"
defaultEntryPoints = ["http","https"]
[entryPoints]
[entryPoints.http]
address = ":80"
compress = true
[entryPoints.http.forwardedHeaders]
trustedIPs = [ "0.0.0.0/0" ]
entryPoint = "https"
[entryPoints.https]
address = ":443"
compress = true
[entryPoints.https.forwardedHeaders]
trustedIPs = [ "0.0.0.0/0" ]
[entryPoints.https.tls]
[acme]
email = "xxxx"
storage = "/acme/acme.json"
entryPoint = "https"
onHostRule = true
#caServer = "https://acme-staging-v02.api.letsencrypt.org/directory"
acmeLogging = true
[[acme.domains]]
main = "xxxx"
[acme.dnsChallenge]
provider = "route53"
delayBeforeCheck = 0
[persistence]
enabled = true
existingClaim = "pvc0"
annotations = {}
accessMode = "ReadWriteOnce"
size = "1Gi"
[kubernetes]
namespaces = ["default"]
[accessLog]
filePath = "/acme/access.log"
[accessLog.fields]
defaultMode = "keep"
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: traefik-ingress-controller
namespace: default
labels:
k8s-app: traefik-ingress-lb
spec:
replicas: 1
selector:
matchLabels:
k8s-app: traefik-ingress-lb
template:
metadata:
labels:
k8s-app: traefik-ingress-lb
name: traefik-ingress-lb
spec:
serviceAccountName: traefik-ingress-controller
terminationGracePeriodSeconds: 60
containers:
- image: traefik
name: traefik-ingress-lb
env:
- name: AWS_ACCESS_KEY_ID
value: xxxx
- name: AWS_SECRET_ACCESS_KEY
value: xxxx
- name: AWS_REGION
value: us-west-2
- name: AWS_HOSTED_ZONE_ID
value: xxxx
ports:
- name: http
containerPort: 80
- name: admin
containerPort: 8080
args:
- --api
- --kubernetes
- --configfile=/config/traefik.toml
volumeMounts:
- mountPath: /config
name: config
- mountPath: /acme
name: acme
volumes:
- name: config
configMap:
name: traefik-conf
- name: acme
persistentVolumeClaim:
claimName: "pvc0"
---
kind: Service
apiVersion: v1
metadata:
name: traefik-ingress-service
namespace: default
spec:
externalIPs:
- x.x.x.x
externalTrafficPolicy: Local
selector:
k8s-app: traefik-ingress-lb
ports:
- protocol: TCP
port: 80
name: web
- protocol: TCP
port: 443
name: https
- protocol: TCP
port: 8080
name: admin
type: NodePort
And here's my ingress:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: headers-test
namespace: default
annotations:
ingress.kubernetes.io/proxy-body-size: 500m
kubernetes.io/ingress.class: traefik
spec:
rules:
- host: xxxx
http:
paths:
- path: /
backend:
serviceName: headers-test
servicePort: 8080
I'd read that I only needed to add [entryPoints.http.forwardedHeaders] and a list of trustedIPs but that doesn't seem to work. Am I missing something?
If you use NodePort for the Traefik Ingress Service, you will have to set service.spec.externalTrafficPolicy to "Local". Otherwise you will have a SNAT when your connection enters the K8s-cluster. This SNAT is necessary to forward the incoming connection to your pod if it is not running on the same node.
But be aware that having set service.spec.externalTrafficPolicy to "Local", only the node on which the Traefik pod is executed will accept requests on 80, 443, 8080. There is no forwarding to the pod from the other nodes anymore. This can result in odd delays when connecting to your service. To avoid that your Traefik would need to run in a HA setup (DaemonSet). Just keep in mind that you need a K/V-Store for a distributed Traefik setup to make Letsencrypt work well.
If the service.spec.externalTrafficPolicy setting does not yet resolve your problem you might also need to configure the kubernetes overlay network to not do any SNAT.
service.spec.externalTrafficPolicy is nicely explained here:
https://kubernetes.io/docs/tutorials/services/source-ip/