We are using Traefik 2.6.2 in Docker Swarm. We have the following configuration for the application running behind Traefik reverse proxy:
version: "3.7"
services:
web:
image: blog:0.0.1
networks:
- traefik-net
deploy:
replicas: 1
labels:
# enable Traefik for this service
- "traefik.enable=false"
- "traefik.docker.network=traefik-net"
# router
- "traefik.http.routers.blog.rule=Host(`blog.company.com`)"
- "traefik.http.routers.blog.entrypoints=web-secure"
- "traefik.http.routers.blog.tls=true"
- "traefik.http.routers.blog.service=blog"
# service
- "traefik.http.services.blog.loadbalancer.server.port=443"
- "traefik.http.services.blog.loadbalancer.server.scheme=https"
- "traefik.http.services.blog.loadbalancer.healthCheck.path=/health"
- "traefik.http.services.blog.loadbalancer.healthCheck.interval=30s"
- "traefik.http.services.blog.loadbalancer.healthCheck.timeout=3s"
networks:
traefik-net:
external: true
name: traefik-net
We also have another low-priority router which listens on the same URL and entrypoint as the original blog service. It catches the request only if application is not deployed, kind of a fallback to static error page:
http:
routers:
blog-fallback:
rule: "Host(`blog.company.com`)"
priority: 1
middlewares:
- "redirect-to-generic-error-page"
tls: {}
service: "noop#internal"
middlewares:
redirect-to-generic-error-page:
redirectRegex:
regex: "(.*)"
replacement: "https://www.company.com/under-maintenance.html?returnURL=${1}"
permanent: false
Is it possible that the low-priority blog-fallback router takes precedence over blog router in case all of the backends are failing Traefik health checks in blog service?
Related
So long ago I started studying the data of the solution, but I am so stupid that I have not found a solution to how to configure file docker-compose for a simple ASP.NET Core for traefik proxy.
I took an example of a simple ASP.NET Core application from the Microsoft site, which, after deployment, is available at localhost:8443 using https, because earlier I released a self-signed (aspnetapp.pfx), ok.
Then I unfolded the traefik and configured the dashboard I see that traefik gets information about the aspnet_demo container, but at web app addresses, or webapp.mydomen.com/ or localhost nothing is available - maximum I get the error ERR_TOO_MANY_REDIRECTS in browser.
In logs traefik when referring to webapp.mydomen.com I get "RequestURI ": "/ "
What did you forget to point out?
I understand that the content aspnet_demo get on 443 port, so I tell Traefik where to look, but nothing...
Help me please understanding this. Thank you
My docker compose ASP.NET Core app looks like this:
version: "3.8"
services:
aspnet_demo:
image: mcr.microsoft.com/dotnet/core/samples:aspnetapp
container_name: aspnet_sample
ports:
- 8080:80
- 8443:443
environment:
- ASPNETCORE_ENVIRONMENT=Development
- ASPNETCORE_URLS=https://+:443;http://+:80
- ASPNETCORE_Kestrel__Certificates__Default__Password=password
- ASPNETCORE_Kestrel__Certificates__Default__Path=/https/aspnetapp.pfx
volumes:
- ~/.aspnet/https:/https:ro
networks:
- traefik-reverse-proxy
labels:
- traefik.enable=true
- traefik.http.routers.aspnet.entrypoints=web
- traefik.http.routers.aspnet.rule=Host(`webapp`)
- traefik.http.routers.aspnet_secure.entrypoints=web-secure
- traefik.http.routers.aspnet_secure.rule=Host(`webapp.mydomen.com`)
- traefik.http.routers.aspnet_secure.tls=true
- traefik.http.services.aspnet.loadbalancer.server.port=443
networks:
traefik-reverse-proxy:
external: true
My docker compose Traefik looks like this:
version: "3.8"
services:
traefik:
image: traefik:v2.9
ports:
- "80:80"
- "443:443"
- "8080:8080"
networks:
- traefik-reverse-proxy
volumes:
- "/var/run/docker.sock:/var/run/docker.sock:ro"
- "./configuration/dynamic.yaml:/traefik_conf/dynamic.yaml"
- "./configuration/traefik.yml:/traefik.yml:ro"
- "./cert/:/traefik_conf/cert/"
labels:
- "traefik.enable=true"
- "traefik.http.routers.traefik.service=api#internal"
- "traefik.http.routers.traefik.entrypoints=web-secure"
- "traefik.http.routers.traefik.rule=Host(`traefiklocal.mydomen.com`)"
- "traefik.http.routers.traefik.tls=true"
- "traefik.http.routers.traefik.tls.certresolver=tls"
- "traefik.http.routers.traefik.middlewares=traefik-auth"
- "traefik.http.middlewares.traefik-auth.basicauth.users=unixhost:$$apr1$$vqyMX723$$6nZ1lC3/2JN6QJyeEhJB8/"
networks:
traefik-reverse-proxy:
external: true
My static config Traefik looks like this:
api:
dashboard: true
insecure: true
log:
level: DEBUG
entryPoints:
web:
address: ":80"
forwardedHeaders:
insecure: true
http:
redirections:
entryPoint:
to: web-secure
web-secure:
address: ":443"
providers:
docker:
watch: true
exposedbydefault: false
file:
directory: /traefik_conf/
watch: true
filename: dynamic.yaml
My dynamic config Traefik:
tls:
certificates:
# first certificate
- certFile: "/traefik_conf/cert/pem_com_2022.pem"
keyfile: "/traefik_conf/cert/star_com_2022.key"
# second certificate
- certFile: "/traefik_conf/cert/aspnetapp.pem"
keyfile: "/traefik_conf/cert/aspnetapp.key"
stores:
- default
I'm starting up Traefik version 2.1.4 with the following configuration:
defaultEntryPoints:
- http
entryPoints:
hole-1:
address: ':663'
frontends:
hole-frontend-1:
backend: hole-backend-1
entrypoints:
- hole-1
backends:
hole-backend-1:
servers:
hole-server-1:
url: 'http://11.23.24.1:3000'
weight: 10
When I hit the http://11.23.24.1:663 I'm hit with the Traefik page 404 pages not found. When I go to http://11.23.24.1:3000 in the browser, it works, it displays the page.
I can't seem to figure out how to set up a reverse proxy in Traefik to point http://11.23.24.1:663 to http://11.23.24.1:3000.
I tried this, but I get the error
2020/02/18 11:39:43 command traefik.exe error: no valid configuration found in file: C:\config\traefik.yaml
http:
routers:
hole-router-1:
rule: "Host(`11.23.24.1`) && PathPrefix(`/`)"
service: hole-service
services:
hole-service:
loadBalancer:
servers:
- url: http://11.23.24.1:3000
C:\config\traefik\traefik.yaml:
entryPoints:
hole-1:
address: ":661"
providers:
file:
filename: C:\config\traefik\dynamic.yaml
C:\config\traefik\dynamic.yaml:
http:
routers:
hole-router-1:
rule: "Path(`/`)"
entryPoints:
- hole-1
service: hole-service
services:
hole-service:
loadBalancer:
servers:
- url: http://10.23.24.10:3000
I started Traefik with: traefik.exe --configFile=C:/config/traefik/traefik.yaml
Traefik 2.1 do not support frontend and backends any more.
This repository provides some examples to deploy Traefik 2.1 https://github.com/wshihadeh/traefik_v2
Example to do it with Docker:
version: '3.7'
networks:
traefik:
external: true
volumes:
db_data:
services:
proxy:
image: traefik:v2.1
command:
- '--providers.docker=true'
- '--entryPoints.http.address=:80'
- '--providers.providersThrottleDuration=2s'
- '--providers.docker.watch=true'
- '--providers.docker.exposedbydefault=false'
- '--providers.docker.defaultRule=Host("local.me")'
- '--accessLog.bufferingSize=0'
volumes:
- '/var/run/docker.sock:/var/run/docker.sock:ro'
ports:
- '663:80'
deploy:
restart_policy:
condition: any
delay: 5s
max_attempts: 3
window: 120s
update_config:
delay: 10s
order: start-first
parallelism: 1
rollback_config:
parallelism: 0
order: stop-first
logging:
driver: json-file
options:
'max-size': '10m'
'max-file': '5'
networks:
- traefik
hole-backend:
image: hole-backend:demo-v1
command: 'web'
deploy:
labels:
- traefik.enable=true
- traefik.http.services.hole.loadbalancer.server.port=8080
- traefik.http.routers.hole.rule=Host(`hole.local.me`)
- traefik.http.routers.hole.service=blog
- traefik.http.routers.hole.entrypoints=web
- traefik.docker.network=traefik
restart_policy:
condition: any
delay: 5s
max_attempts: 3
window: 120s
update_config:
delay: 10s
order: start-first
parallelism: 1
rollback_config:
parallelism: 0
order: stop-first
logging:
driver: json-file
options:
'max-size': '10m'
'max-file': '5'
networks:
- traefik
I currently access the V2 dashboard through http://traefik.my.server:8080/dashboard/ (Traefik runs in a docker container and 8080 is exposed to the host).
I would like to change that so that the dashboard is available at http://traefik.my.server/dashboard
I tried to add the following labels to configure this behavior but I get a 404 when accessing http://traefik.my.server/dashboard
- traefik.http.routers.dashboard.rule=Host(`traefik.my.server:`) && Path(`/dashboard`)
- traefik.http.services.dashboard.loadbalancer.server.port=8080
- traefik.http.routers.dashboard.entryPoints=http
(the http entrypoint is port 80)
What is the correct way to set up such redirectio
Recommend read:
https://docs.traefik.io/v2.1/operations/dashboard/#secure-mode
https://blog.containo.us/traefik-2-0-docker-101-fc2893944b9d
https://github.com/containous/blog-posts/tree/master/2019_09_10-101_docker
FYI it's not redirection but a routing.
https://community.containo.us/t/how-to-redirect-to-the-dashboard-from-a-url/4082/2
Following up on #Idez help at https://community.containo.us/t/how-to-redirect-to-the-dashboard-from-a-url/4082, a working configuration is
The docker-compose file:
services:
traefik:
container_name: traefik
image: traefik
ports:
- 80:80
- 443:443
restart: unless-stopped
volumes:
- /etc/docker/container-data/traefik:/etc/traefik
- /var/run/docker.sock:/var/run/docker.sock
- /etc/localtime:/etc/localtime:ro
labels:
- traefik.http.routers.api.rule=Host(`traefik.mydomain.org`)
- traefik.http.routers.api.service=api#internal
- traefik.http.routers.api.middlewares=lan
- traefik.http.middlewares.lan.ipwhitelist.sourcerange=192.168.10.0/24, 192.168.20.0/24
- traefik.enable=true
version: "3"
Configuration file
global:
sendAnonymousUsage: true
entryPoints:
http:
address: ":80"
https:
address: ":443"
api:
dashboard: true
providers:
docker:
endpoint: "unix:///var/run/docker.sock"
exposedByDefault: false
defaultRule: "Host(`{{ index .Labels \"com.docker.compose.service\" }}.mydomain.org`)"
log:
level: INFO
#level: DEBUG
certificatesResolvers:
le:
acme:
email: le#mydomain.org
storage: /etc/traefik/acme.json
tlsChallenge: {}
#caServer: "https://acme-staging-v02.api.letsencrypt.org/directory"
I've got a service up and running on traefik with LetsEncrypt at grpc.mydomain.com. However, traefik doesn't support routing grpc-web request due to some issue with CORS (https://github.com/containous/traefik/issues/4210). Envoy appears to be an alternative to traefik which works with grpc-web, but I don't want to go about reconfiguring everything.
If I put envoy at envoy.mydomain.com then it actually hits traefik first and traefik can't route the grpc-web requests to envoy. So this doesn't work.
If I put envoy outside of traefik (mydomain.com:9091) then envoy doesn't have the TLS support that traefik has.
Do I need to switch everything to envoy? Is there an alternative I haven't considered? Any guidance welcome :)
Current Traefik Setup:
traefik:
image: traefik:v2.0.0
container_name: traefik
command:
- --entrypoints.web.address=:80
- --entrypoints.websecure.address=:443
- --entrypoints.grpc.address=:8090
- --providers.docker
- --api
- --serversTransport.rootCAs=/certs/grpc.cert
# Lets Encrypt Resolvers
- --certificatesresolvers.leresolver.acme.email=${EMAIL}
- --certificatesresolvers.leresolver.acme.storage=/etc/acme/cert.json
- --certificatesresolvers.leresolver.acme.tlschallenge=${TLS_CHALLENGE}
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- /etc/acme/:/etc/acme/
- ./secrets/grpc.cert:/certs/grpc.cert
# Dynamic Configuration
labels:
# Dashboard
- "traefik.http.routers.traefik.rule=Host(`traefik.${DOMAIN}`)"
- "traefik.http.routers.traefik.service=api#internal"
- "traefik.http.routers.traefik.tls.certresolver=leresolver"
- "traefik.http.routers.traefik.entrypoints=websecure"
- "traefik.http.routers.traefik.middlewares=authtraefik"
# https://docs.traefik.io/middlewares/basicauth/
# password generated from `echo $(htpasswd -nb admin $PASSWORD) | sed -e s/\\$/\\$\\$/g`
- "traefik.http.middlewares.authtraefik.basicauth.users=admin:$$apr1$$6VzI3S0N$$29FC82dYEbjFN9tPSfWLX1"
# global redirect to https
- "traefik.http.routers.http-catchall.rule=hostregexp(`{host:.+}`)"
- "traefik.http.routers.http-catchall.entrypoints=web"
- "traefik.http.routers.http-catchall.middlewares=redirect-to-https"
# middleware redirect
- "traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https"
ports:
- 80:80
- 443:443
- 8090:8090
networks:
- internal
- proxied
grpc_server:
image: ${GRPC_IMAGE}
container_name: grpc_server
volumes:
- /tmp/keyset.json:/tmp/keyset.json
- ./secrets/:/secrets/
working_dir: /app/__main__/
labels:
- "traefik.http.routers.combined_server.rule=Host(`grpc.${DOMAIN}`)"
- "traefik.http.routers.combined_server.entrypoints=grpc"
- "traefik.http.routers.combined_server.tls=true"
- "traefik.http.routers.combined_server.tls.certresolver=leresolver"
# http
- "traefik.http.services.grpc-svc.loadbalancer.server.scheme=h2c"
- "traefik.http.services.grpc-svc.loadbalancer.server.port=8090"
expose:
- 8090
networks:
- internal
- proxied
I also tried setting these to fix the CORS error but got nowhere.
- "traefik.http.middlewares.testheader.headers.accesscontrolallowmethods=GET,PUT,DELETE,POST,OPTIONS"
- "traefik.http.middlewares.testheader.headers.accesscontrolallowheaders=keep-alive,user-agent,cache-control,content-type,content-transfer-encoding,custom-header-1,x-accept-content-transfer-encoding,x-accept-response-streaming,x-user-agent,x-grpc-web,grpc-timeout"
- "traefik.http.middlewares.testheader.headers.accesscontrolmaxage=100"
- "traefik.http.middlewares.testheader.headers.addvaryheader=true"
- "traefik.http.middlewares.testheader.headers.alloworigin=*"
A way to approach this issue is by using 2 different URLs that are both being handled by traefik at first. One URL is being used for "direct grpc" (grpc.mydomain.com), the other one for grpc-web (let's call it grpc-web.mydomain.com). Traefik does TLS termination for both.
The grpc.mydomain.com traffic is directly passed to the container running the grpc_server. The grpc-web.mydomain.com traffic is passed to envoy which acts as a grpc-web-proxy and then passes the traffic to the grpc_server.
So as you are using docker-compose, you would need to add an envoy service to your docker-compose.yml:
---
version: '3'
services:
traefik:
# traefik configuration from your question
# ...
grpc-server:
# grpc_server configuration from your question
# ...
envoy:
image: envoyproxy/envoy:v1.14.1
restart: unless-stopped
volumes:
- ./envoy.yaml:/etc/envoy/envoy.yaml
labels:
- traefik.enable=true
- traefik.http.routers.envoy.rule=Host(`grpc-web.mydomain.com`)
- traefik.http.services.envoy.loadbalancer.server.port=8080
- traefik.http.routers.envoy.tls=true
- traefik.http.routers.envoy.tls.certresolver=leresolver
The envoy.yaml configuration (mounted in the volumes section above) looks like this:
admin:
access_log_path: /tmp/admin_access.log
address:
socket_address: { address: 0.0.0.0, port_value: 9901 }
static_resources:
listeners:
- name: listener_0
address:
socket_address: { address: 0.0.0.0, port_value: 8080 }
filter_chains:
- filters:
- name: envoy.http_connection_manager
config:
codec_type: auto
stat_prefix: ingress_http
route_config:
name: local_route
virtual_hosts:
- name: local_service
domains: ["*"]
routes:
- match: { prefix: "/" }
route:
cluster: grpc_service
max_grpc_timeout: 0s
cors:
allow_origin_string_match:
- prefix: "*"
allow_methods: GET, PUT, DELETE, POST, OPTIONS
allow_headers: keep-alive,user-agent,cache-control,content-type,content-transfer-encoding,custom-header-1,x-accept-content-transfer-encoding,x-accept-response-streaming,x-user-agent,x-grpc-web,grpc-timeout
max_age: "1728000"
expose_headers: custom-header-1,grpc-status,grpc-message
http_filters:
- name: envoy.grpc_web
- name: envoy.cors
- name: envoy.router
clusters:
- name: grpc_service
connect_timeout: 0.25s
type: logical_dns
http2_protocol_options: {}
lb_policy: round_robin
hosts: [{ socket_address: { address: grpc-server, port_value: 8090 }}]
This is a pretty basic grpc-web config for envoy. The important part to notice is that we set address: grpc-server, port_value: 8090 in the configuration of the "grpc_service" cluster configuration to the service name from the docker-compose.yml and to the port your grpc-server is listening on. Please note I renamed your service from grpc_server to grpc-server as the underscore is not a valid charater in hostnames.
On the client side, use:
"grpc-web.mydomain.com" in your javascript (grpc-web) code.
"grpc.mydomain.com" when writing a client in another language (like golang for example).
I created a working example, which can be found under: https://github.com/rbicker/greeter
If you want to get rid of deprecated warnings in envoy, you can update envoy.yaml from this answer with those three changes:
replace:
- name: envoy.http_connection_manager
config:
with:
- name: envoy.filters.network.http_connection_manager
typed_config:
"#type": type.googleapis.com/envoy.config.filter.network.http_connection_manager.v2.HttpConnectionManager
replace
- name: envoy.grpc_web
- name: envoy.cors
- name: envoy.router
with
- name: envoy.filters.http.grpc_web
- name: envoy.filters.http.cors
- name: envoy.filters.http.router
replace
hosts: [{ socket_address: { address: grpc-server, port_value: 8090 }}]
with
load_assignment:
cluster_name: cluster_0
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: grpc-server
port_value: 8090
Middlewares are not being detected and therefore paths are not being stripped resulting in 404s in the backend api.
Middleware exists in k8s apps namespace
$ kubectl get -n apps middlewares
NAME AGE
traefik-middlewares-backend-users-service 1d
configuration for middleware and ingress route
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
annotations:
kubernetes.io/ingress.class: traefik
name: apps-services
namespace: apps
spec:
entryPoints:
- web
routes:
- kind: Rule
match: Host(`example.com`) && PathPrefix(`/users/`)
middlewares:
- name: traefik-middlewares-backend-users-service
priority: 0
services:
- name: backend-users-service
port: 8080
---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
name: traefik-middlewares-backend-users-service
namespace: apps
spec:
stripPrefix:
prefixes:
- /users
Static configuration
global:
checkNewVersion: true
sendAnonymousUsage: true
entryPoints:
http:
address: :80
traefik:
address: :8080
providers:
providersThrottleDuration: 2s
kubernetesIngress: {}
api:
# TODO: make this secure later
insecure: true
ping:
entryPoint: http
log: {}
Traefik dasboard has no middlewares
Spring Boot 404 page. Route is on example.com/actuator/health
The /users is not being stripped. This worked for me in traefik v1 perfectly fine.
Note: actual domain has been replaced with example.com and domain.com in the examples.
To get this working, I had to:
Add the Kubernetes CRD provider with the namespaces where the custom k8s CRDs for traefik v2 exist
Add TLSOption resource definition
Update cluster role for traefik to have permissions for listing and watching new v2 resources
Make sure all namespaces with new resources are configured
Traefik Static Configuration File
providers:
providersThrottleDuration: 2s
kubernetesCRD:
namespaces:
- apps
- traefik
TLSOption CRD
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: tlsoptions.traefik.containo.us
spec:
group: traefik.containo.us
version: v1alpha1
names:
kind: TLSOption
plural: tlsoptions
singular: tlsoption
scope: Namespaced
Updated Static Configuration for Traefik
global:
checkNewVersion: true
sendAnonymousUsage: true
entryPoints:
http:
address: :80
traefik:
address: :8080
providers:
providersThrottleDuration: 2s
kubernetesCRD:
namespaces:
- apps
- traefik
api:
# TODO: make this secure later
insecure: true
ping:
entryPoint: http
log: {}