Getting "Gateway Timeout" for Mautic behind Traefik proxy - traefik

I deployed an multicontainer application with Mautic behind a Traefik reverse proxy.
However, I am getting a "Gateway timeout".
The reverse proxy's configuration seems OK as other containers within the application work fine.
I also changed the mautic settings to the mautics domain name.
Any idea?
docker-compose.yml
version: '3.3'
services:
reverse-proxy:
image: traefik:v2.4
restart: always
ports:
- '80:80'
- '443:443'
volumes:
- ./traefik:/etc/traefik
- /var/run/docker.sock:/var/run/docker.sock
mautic-app:
restart: always
image: mautic/mautic:v3
volumes:
- mautic_data:/var/www/html
environment:
- MAUTIC_DB_HOST=mautic-database
- MAUTIC_DB_USER=${MAUTIC_DB_USER}
- MAUTIC_DB_PASSWORD=${MAUTIC_DB_PASSWORD}
- MAUTIC_DB_NAME=mautic3
ports:
- 80
labels:
- 'traefik.enable=true'
- 'traefik.http.routers.mautic.tls=true'
- 'traefik.http.routers.mautic.tls.domains[0].main=optin.${SITE}'
- 'traefik.http.routers.mautic.tls.certresolver=lets-encrypt'
- 'traefik.http.routers.mautic.rule=Host(`optin.${SITE}`)'
depends_on:
- mautic-database
networks:
- mautic-net
mautic-database:
image: powertic/percona-docker
restart: always
environment:
MYSQL_ROOT_PASSWORD: ${MAUTIC_DB_PASSWORD}
ports:
- 3306
volumes:
- database:/var/lib/mysql
command: --character-set-server=utf8mb4 --collation-server=utf8mb4_general_ci --sql-mode=""
networks:
- mautic-net
volumes:
database:
driver: local
mautic_data:
driver: local
networks:
mautic-net:
driver: bridge
traefik.toml
[log]
level = "DEBUG"
[providers]
[providers.docker]
exposedByDefault = false
[providers.file]
directory = "/etc/traefik/dynamic"
[entryPoints]
[entryPoints.http]
address = ":80"
[entryPoints.https]
address = ":443"
[certificatesResolvers.lets-encrypt.acme]
storage = "/etc/traefik/acme.json"
email = "jenslaufer#jenslaufer.com"
[certificatesResolvers.lets-encrypt.acme.tlsChallenge]
force-https.toml
[http.routers]
[http.routers.force-https]
entryPoints = ["http"]
middlewares = ["force-https"]
rule = "HostRegexp(`{any:.+}`)"
service = "noop"
[http.middlewares]
[http.middlewares.force-https.redirectScheme]
scheme = "https"
[http.services]
[http.services.noop.loadBalancer]

Not sure if you're using Apache or PHP FPM behind the proxy.
But... Traefik does not support fastCgi Proxy
We encountered this issue with Traefik and set up our mautic kubernetes with nginx-proxy for that reason.

If you have docker running in swam mode, containers will have multiple networks attached. Traefik needs pointing to which network to use.
You must set label traefik.docker.network on the backend service container.
labels:
traefik.docker.network: <traefik network>
Otherwise, you will be getting success sometimes but most of the time traefik will return Gateway Timeout

It's a "misconfiguration" in docker-compose:
Traefik and Maurice are in different networks.
Moving Traefik to the mautic-net network fixed the problem.

Related

Traefik and Harbor

I have changed the docker-compose.yml after the default Conf-Generation. My Proxy configuration looks like:
( I had only touched the proxy part )
proxy:
image: goharbor/nginx-photon:v1.9.4
container_name: nginx
restart: always
cap_drop:
- ALL
cap_add:
- CHOWN
- SETGID
- SETUID
- NET_BIND_SERVICE
volumes:
- ./common/config/nginx:/etc/nginx:z
networks:
- harbor
- web
dns_search: .
expose:
- 8080
ports:
- 8081:8080 #only to test if its online
labels:
- "traefik.enable=true"
- "traefik.backend=harbor-backend"
- "traefik.docker.network=web"
- "traefik.frontend.rule=Host:harbor.example.com"
- "traefik.frontend.entryPoints=https,http"
- "traefik.port=8080"
depends_on:
- registry
- core
- portal
- log
logging:
driver: "syslog"
options:
syslog-address: "tcp://127.0.0.1:1514"
tag: "proxy"
networks:
harbor:
external: false
web:
external: true
In my Harbor.yml i had leaved the http port to 80 and disabled https.
My Proxy configuration is the default one.
The Web UI works fine but when I try to login with the docker client i get this response:
root#h2791112:~# docker login registry.example.com
Username: admin
Password:
Error response from daemon: Get https://registry.example.com/v2/: unauthorized: authentication required
And that are the crendentials that work with the ui
did I miss a label ?
This worked for me: https://github.com/goharbor/harbor/issues/3114#issuecomment-432962664
Follow the steps

Reverse proxy NGINX front-end to redirect Dockerized Apache

I am coming from this. I am new in web-servers. I am setting up a server (example.de) serving several dockerized applications. One of them (app3) is running on behind an Apache server (port 80 inside the container mapped to 8484). I want to use a NGINX docker container to redirect the url example.de/app3 to example.de:8484. Other two applications will be also redirected to example.de/app?.
When I request the url example.de:8484 it works fine. However, when I use example.de/app3 only the "It works" webpage appears.
The docker-compose file is something similar to:
version: '3'
services:
app3:
build:
context: app3
container_name: app3
app3_apache:
build:
context: app3/apache
container_name: app3_apache
ports:
- "8484:80"
volumes:
- vol_app3:/vol/app3
depends_on:
- app3
links:
- app3
nginx-proxy:
container_name: nginx
build:
context: nginx/
ports:
- "80:80"
depends_on:
- app3_apache
links:
- app3_apache
volumes:
- vol_nginx:/etc/nginx/conf.d
volumes:
vol_app3:
driver_opts:
type: none
device: /vol/app3
o: bind
vol_nginx:
driver_opts:
type: none
device: /vol/nginx
o: bind
NGINX default.conf file looks like:
server {
listen 80;
listen [::]:80;
server_name example.de;
location /app3 {
proxy_pass http://app3_apache:80;
}
}
Additional information:
It creates and run the containers. The webpage in the APACHE container is served in the NGINX container using the xxx.xxx.xxx.xxx:80 but not app3_apache:80 <h1>Bad Request (400)</h1>.
I can reach the app3_apache container from the NGINX container:
traceroute app3_apache
traceroute to app3_apache (xxx.xxx.xxx.xxx), 30 hops max, 60 byte packets
1 app3_apache.server_default (xxx.xxx.xxx.xxx) 0.351 ms 0.054 ms 0.035 ms
You should create a docker network to link the container.
You can check docs here: https://docs.docker.com/network/bridge/
After you create the network, you will have to change configuration for nginx proxy_pass to point out the internal IP of the web server application.
As for example I setup this network configuration to connect 3 containers with fixed IP addresses.
version: '2'
services:
zabbix-server4:
container_name: zabbix-server4
image: zabbix/zabbix-server-mysql:alpine-4.0.5
networks:
zbx_net:
aliases:
- zabbix-server4
ipv4_address: 172.16.238.5
zabbix-web4:
container_name: zabbix-web4
image: zabbix/zabbix-web-nginx-mysql:alpine-4.0.5
ports:
- 127.0.0.1:11011:80
links:
- zabbix-server4
environment:
- ZBX_SERVER_HOST=172.16.238.5
networks:
zbx_net:
aliases:
- zabbix-web4
ipv4_address: 172.16.238.10
zabbix-agent4:
container_name: zabbix-agent4
image: zabbix/zabbix-agent:alpine-4.0.5
links:
- zabbix-server4
environment:
- ZBX_SERVER_HOST=172.16.238.5
networks:
zbx_net:
aliases:
- zabbix-agent4
ipv4_address: 172.16.238.15
networks:
zbx_net:
driver: bridge
driver_opts:
com.docker.network.enable_ipv6: "false"
ipam:
driver: default
config:
- subnet: 172.16.238.0/24
gateway: 172.16.238.1
In your case, you could set a new network and assign new network's IPs to both the nginx and web container, then in nginx proxy_pass configuration you should put web server's IP.
E.g:
version: '3'
services:
app3:
build:
context: app3
container_name: app3
app3_apache:
build:
context: app3/apache
container_name: app3_apache
ports:
- "8484:80"
volumes:
- vol_app3:/vol/app3
depends_on:
- app3
links:
- app3
networks:
my_net:
aliases:
- zabbix-agent4
ipv4_address: 172.16.11.10
nginx-proxy:
container_name: nginx
build:
context: nginx/
ports:
- "80:80"
depends_on:
- app3_apache
links:
- app3_apache
volumes:
- vol_nginx:/etc/nginx/conf.d
networks:
my_net:
aliases:
- zabbix-agent4
ipv4_address: 172.16.11.20
volumes:
vol_app3:
driver_opts:
type: none
device: /vol/app3
o: bind
vol_nginx:
driver_opts:
type: none
device: /vol/nginx
o: bind
networks:
my_net:
driver: bridge
driver_opts:
com.docker.network.enable_ipv6: "false"
ipam:
driver: default
config:
- subnet: 172.16.11.0/24
gateway: 172.16.11.1
And proxy pass should be sety to
location /app3 {
proxy_pass http://172.16.11.10:80;
}
To prevent having this trouble, I run nginx on host, not in a container, then I split traffic in containers using same approach.
Having nginx installed in the host machine allow me to avoid this kind of configuration overhead.

Traefik and WildCard Certificates Issue By Third party

I have traefik set up and running, but experiencing issues with https, (wildcard certificate). Im able to get a redirect from 80 to 443 but nothing will load. I donot see much information in the logs and they have been set to debug level
Checked the permissions for the cert and set logs to debug
0644 on crt and 0600 on key
Tarefik TOML
defaultEntryPoints = ["http", "https"]
[entryPoints]
[entryPoints.http]
address = ":80"
[entryPoints.http.redirect]
entryPoint = "https"
[entryPoints.https]
address = ":443"
[entryPoints.https.tls]
[[entryPoints.https.tls.certificates]]
certFile = "/etc/traefik/certs/server.crt"
keyFile = "/etc/traefik/certs/server.key"
Docker Compose
services:
reverse-proxy:
image: traefik # The official Traefik docker image
command: --api --docker # Enables the web UI and tells Traefik to listen to docker
ports:
- "80:80" # The HTTP port
- "8080:8080" # The Web UI (enabled by --api)
volumes:
- /var/run/docker.sock:/var/run/docker.sock # So that Traefik can listen to the Docker events
- /home/traefik/traefik.toml:/traefik.toml
- /home/traefik/certs:/etc/traefik/certs
networks:
- traefik-net
Sample NGINX server being run
nginx-docker:
image: nginx:latest
labels:
- "traefik.frontend.rule=Host:<hostname>.eogresources.com"
networks:
- traefik-net
Exposing 443 on traefik resolved my issue. Added additional port docker-compose file, change reflected below. Silly mistake. Also added the network in compose file
services:
reverse-proxy:
image: traefik # The official Traefik docker image
command: --api --docker # Enables the web UI and tells Traefik to listen to docker
ports:
- "80:80" # The HTTP port
- "8080:8080" # The Web UI (enabled by --api)
- "443:443"
volumes:
- /var/run/docker.sock:/var/run/docker.sock # So that Traefik can listen to the Docker events
- /home/traefik/traefik.toml:/traefik.toml
- /home/traefik/certs:/etc/traefik/certs
networks:
- traefik-net
networks:
traefik-net:
driver: "overlay"

SSL passthrough with Traefik

I need to send the SSL connections directly to the backend, not decrypt at my Traefik. The backend needs to receive https requests.
I tried the traefik.frontend.passTLSCert=true option but getting "404 page not found" error when I access my web app and also get this error on Traefik container
traefik | time="2018-09-16T10:47:41Z" level=error msg="Failed to create TLSClientConfig: no TLS provided"
traefik | time="2018-09-16T10:47:41Z" level=error msg="Failed to create RoundTripper for frontend frontend-Host-dev-mydomain-com-0: no TLS provided"
traefik | time="2018-09-16T10:47:41Z" level=error msg="Skipping frontend frontend-Host-dev-mydomain-com-0..."
Could you suggest any solution? Thank you.
I'm using Traefik version 1.6.6.
Here is my docker-compose.yml for the app container.
version: '3'
services:
app:
image: webdevops/php-nginx-dev:7.2
networks:
- proxy
volumes:
- ./:/app
- ../traefik/ssl/*.mydomain.com.crt:/opt/docker/etc/nginx/ssl/server.crt
- ../traefik/ssl/*.mydomain.com.key:/opt/docker/etc/nginx/ssl/server.key
environment:
- WEB_DOCUMENT_ROOT=/app
labels:
- traefik.enable=true
- traefik.frontend.rule=Host:dev.mydomain.com
- traefik.docker.network=proxy
- traefik.port=443
networks:
proxy:
external: true
The docker-compose.yml of my Traefik container.
version: "3"
services:
traefik:
image: traefik
container_name: traefik
command:
- --api
- --docker
- --docker.exposedbydefault=false
restart: always
ports:
- 80:80
- 443:443
networks:
- proxy
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./traefik.toml:/traefik.toml
- ./ssl:/sslcert
networks:
proxy:
external: true
Finally, my traefik.toml file.
debug = true
logLevel = "ERROR"
defaultEntryPoints = ["http","https"]
[entryPoints]
[entryPoints.http]
address = ":80"
[entryPoints.https]
address = ":443"
[entryPoints.https.tls]
[[entryPoints.https.tls.certificates]]
certFile = "/sslcert/*.mydomain.com.crt"
keyFile = "/sslcert/*.mydomain.com.key"
[retry]
Traefik now has TCP support in its new 2.0 version - which is still in alpha at this time (Apr 2019).
See PR https://github.com/containous/traefik/pull/4587
and the release notes of v2.0.0-alpha1 at https://github.com/containous/traefik/releases/tag/v2.0.0-alpha1 showing this TCP support PR being included
Hence once 2.0 is released (probably within 2-3 months), HTTPS passthrough will become possible.
Answer for traefik 1.0 (outdated)
passTLSCert forwards the TLS Client certificate to the backend, that is, a client that sends a certificate in the TLS handshake to prove it's identity.
Traefik is an HTTP reverse proxy. To establish the SSL connection directly with the backend, you need to reverse proxy TCP and not HTTP, and traefik doesn't (yet ?) support tcp (but there are issues for that on github).
Traefik won't fit your usecase, there are different alternatives, envoy is one of them.
The new passthrough for TCP routers is already available: https://docs.traefik.io/routing/routers/#passthrough

Redis connection refused when using traefik

I'm having a hard time trying to configure one redis container for all my applications using traefik. This is my configuration:
1 - Docker compose for Traefik and Redis:
version: '2'
services:
proxy:
container_name: traefik
image: traefik:1.3.6-alpine
command: --docker
ports:
- 80:80
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./traefik.toml:/traefik.toml
networks:
- proxy
labels:
- traefik.frontend.rule=Host:monitor.company.dev
- traefik.port=8080
redis:
container_name: main_redis
image: redis:3.2
restart: always
volumes:
- ./data/redis:/data
networks:
- proxy
labels:
- traefik.backend=main-redis
- traefik.default.protocol=http
- traefik.frontend.rule=Host:main-redis.company.dev
- traefik.docker.network=proxy
- traefik.port=6379
networks:
proxy:
external: true
2 - Docker compose for my PHP Application.
version: '2'
services:
...
php:
container_name: myapp_php
build: ./docker/php # php:7.1-fpm base image
networks:
- internal
- proxy
labels:
- traefik.enable=false
- traefik.docker.network=proxy
expose:
- 9000
networks:
proxy:
external: true
internal:
external: false
I tried to connect my php application to main-redis.company.dev on both ports 6379 and 80 but I get a Redis::connect(): connect() failed: Connection refused message.
I also changed these stuff in my redis.conf:
Commented the line with bind 127.0.0.1
And changed protected-mode to no
My docker containers are staying in the same network, so I think it should work. Anyone knows why am I having this problem?
2022 UPDATE to #djeeg answer
For some time now you can use TCP mode for your routers. To do that you need to define the labels with TCP instead of HTTP
labels:
- "traefik.enable=true"
- "traefik.tcp.routers.redis.rule=HostSNI(`redis.example.com`)"
- "traefik.tcp.routers.redis.entrypoints=redis" //6379 entrypoint
- "traefik.tcp.routers.redis.tls.certresolver=myresolver" //let's encrypt resolver
- "traefik.tcp.routers.redis.service=redis"
- "traefik.tcp.services.redis.loadbalancer.server.port=6379"
once you got that working in order to connect assuming you are using TSL you will get error Error: Protocol error, got "H" as reply type byte.
To prevent this you need to do things.
Allow for tls connection in the connection string
Setup SNI for your DNS name or provide public certificate file via cert pr cacert
redis-cli -u redis://redis.example.com:6379 --tls --sni redis.example.com
First off, remove the traefik labels from your redis service definition, traefik is currently (Nov 2017) a HTTP proxy, so you can't expose the endpoint like that
See here:
https://github.com/containous/traefik/issues/10
https://github.com/containous/traefik/issues/1611
Then to connect the php service to the redis service, that looks like you are trying to do that within the same docker instance (rather than externally)
Instead of main-redis.company.dev:6379, it should be like on of these:
redis:6379
main_redis:6379
%PROJECT_NAME%_redis:6379
depending upon how you are deploying the container