Running Graylog behind Traefik - reverse-proxy

I have been using Traefik as a reverse proxy for several containers managed through docker compose. It has been working well but I'm having some difficulty getting graylog running properly behind it.
This config (excerpt of the complete file, removed irrelevant services) works for me
version: '3'
services:
traefik:
image: traefik
command: --docker
networks:
- web
ports:
- "80:80"
- "8080:8080"
- "443:443"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
graylog:
image: graylog/graylog:3.0
environment:
- GRAYLOG_PASSWORD_SECRET=somepasswordpepper
- GRAYLOG_ROOT_PASSWORD_SHA2=somestuffhere
- GRAYLOG_HTTP_BIND_ADDRESS=0.0.0.0:9000
- GRAYLOG_HTTP_EXTERNAL_URI=http://graylog.localhost/
links:
- mongodb:mongo
- elasticsearch
depends_on:
- mongodb
- elasticsearch
networks:
- web
- default
ports:
- 9000:9000
- 1514:1514
- 1514:1514/udp
- 12201:12201
- 12201:12201/udp
labels:
- "traefik.docker.network=web"
- "traefik.web.frontend.rule=Host:graylog.localhost"
- "traefik.protocol=http"
- "traefik.port=9000"
- "traefik.enable=true"
I can then access graylog in my browser at http://graylog.localhost/
The problem occurs because I can't use domains like this in my staging env. What I have had to use is paths rather than domains e.g. the swagger service in my staging env is defined as
swagger:
image: swaggerapi/swagger-ui
environment:
SWAGGER_JSON: /swagger/staging-openapi.yaml
BASE_URL: /swagger
volumes:
- /home/ubuntu/src/carelink_swagger_service:/swagger
networks:
- web
- default
labels:
- "traefik.docker.network=web"
- "traefik.frontend.rule=Host:my.staging.domain.org; PathPrefix: /swagger"
- "traefik.protocol=http"
- "traefik.port=8080"
- "traefik.enable=true"
and I access services like this https://my.staging.domain.org/swagger/
Trying to adopt this approach for graylog
graylog:
image: graylog/graylog:3.0
environment:
- GRAYLOG_PASSWORD_SECRET=somepasswordpepper
- GRAYLOG_ROOT_PASSWORD_SHA2=somestuffhere
- GRAYLOG_HTTP_BIND_ADDRESS=0.0.0.0:9000
- GRAYLOG_HTTP_EXTERNAL_URI=http://localhost/graylog/
links:
- mongodb:mongo
- elasticsearch
depends_on:
- mongodb
- elasticsearch
networks:
- web
- default
ports:
- 9000:9000
- 1514:1514
- 1514:1514/udp
- 12201:12201
- 12201:12201/udp
labels:
- "traefik.docker.network=web"
- "traefik.web.frontend.rule=Host:localhost; PathPrefix: /graylog"
- "traefik.protocol=http"
- "traefik.port=9000"
- "traefik.enable=true"
and I get the following errors
I have gone through this Q/A, similar entries on the graylog forums etc. but I feel like I'm chasing my tail.

It's for Traefik 2.x but I think that you'll manage to convert your rules:
labels:
- "traefik.http.routers.graylog.rule=PathPrefix(`/graylog`)"
- "traefik.http.routers.graylog.middlewares=graylog3#docker,graylog2#docker,graylog#docker"
- "traefik.http.middlewares.graylog3.stripprefix.prefixes=/graylog"
- "traefik.http.middlewares.graylog2.redirectregex.regex=^(.*)/graylog$$"
- "traefik.http.middlewares.graylog2.redirectregex.replacement=$$1/graylog/"
- "traefik.http.middlewares.graylog.replacepathregex.regex=^/graylog/(.*)"
- "traefik.http.middlewares.graylog.replacepathregex.replacement=^/$$1"
- "traefik.http.routers.graylog.service=graylog"
- "traefik.http.services.graylog.loadbalancer.server.port=9000"
- "traefik.docker.network=traefik"

Related

Trouble starting influxdb & chronograf with certificates. [open /etc/letsencrypt/certs/influx.domain.com.crt: no such file or directory]

While trying to run part of the TICK stack (influx and chronograf) through a nginx proxy with letsencrypt certificates i am getting an error that the certs files are not found.
Here is my docker-compose file:
version: '3'
services:
influxdb:
container_name: influxdb
image: quay.io/influxdb/influxdb:v2.0.2
restart: always
ports:
- '8086:8086'
volumes:
- './data:/root/.influxdbv2'
- /root/new/letsencrypt/certs:/etc/letsencrypt/certs:ro
environment:
- INFLUXD_TLS_CERT=/etc/letsencrypt/certs/influx.lizardsolutions.com.crt
- INFLUXD_TLS_KEY=/etc/letsencrypt/certs/influx.lizardsolutions.com.key
chronograf:
image: chronograf:1.7.14
container_name: chronograf
restart: always
hostname: chronograf
network_mode: bridge
ports:
- 3000:8888/tcp
volumes:
- /var/lib/chronograf:/var/lib/chronograf
- /root/new/letsencrypt/certs:/etc/letsencrypt/certs:ro
environment:
- VIRTUAL_HOST=influx.lizardsolutions.com# adjust to match your domain name
- VIRTUAL_PROTO=https
- VIRTUAL_PORT=3000
- LETSENCRYPT_HOST=influx.lizardsolutions.com # adjust to match your domain name -
- LETSENCRYPT_EMAIL=admin#lizardsolutions.com # adjust to match your email
- BOLT_PATH=/var/lib/chronograf/chronograf-v1.db
- CANNED_PATH=/usr/share/chronograf/canned
- RESOURCES_PATH=/usr/share/chronograf/resources
- PROTOBOARDS_PATH=/usr/share/chronograf/protoboards
# - INFLUXDB_URL=http://127.0.0.1:8086
#INFLUX_CREDENTIALS
# - INFLUXDB_USERNAME=grafanam9k5bXbBj7GPGdf3
# - INFLUXDB_PASSWORD=k9H4TV8ifRo7E6mNyv3L
- TLS_CERTIFICATE=/etc/letsencrypt/certs/influx.domain.com.crt
- TLS_PRIVATE_KEY=/etc/letsencrypt/certs/influx.domain.com.key
- REPORTING_DISABLED='false'
# Set the logging level. Valid values: debug | info | error
- LOG_LEVEL=info
- TOKEN_SECRET=hxDEXBPAc49muW3gLoot7YamJ
- AUTH_DURATION=0
- PUBLIC_URL=https://ec-influx.domain.com
nginx-proxy:
image: jwilder/nginx-proxy
container_name: nginx
restart: always
labels:
- com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy
ports:
- "80:80"
- "443:443"
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
- ./nginx/html:/usr/share/nginx/html
- ./nginx/vhost.d:/etc/nginx/vhost.d
- /root/new/letsencrypt/certs:/etc/nginx/certs:ro
letsencrypt:
image: jrcs/letsencrypt-nginx-proxy-companion
container_name: letsencrypt
restart: always
environment:
- NGINX_PROXY_CONTAINER=nginx-proxy
- REUSE_PRIVATE_KEYS=true
volumes_from:
- nginx-proxy
volumes:
- /root/new/letsencrypt/certs:/etc/nginx/certs:rw
- /var/run/docker.sock:/var/run/docker.sock:ro
Here is the error from chronograf:
chronograf | time="2022-08-29T00:22:02Z" level=error msg="open /etc/letsencrypt/certs/influx.domain.com.crt: no such file or directory" component=server
chronograf | 2022/08/29 00:22:02 open /etc/letsencrypt/certs/influx.domain.com.crt: no such file or directory
The same error shows for influxdb as well
From where can i start debuging ? While navigating to the mounted volume (./letsencrypt/certs/) i can confirm the key and cert file is there.

How to use ssl with docker desktop on windows

I have a dev Environment in docker and I'm using Nifi 1.13.2 and it's working fine (http).
I did an upgrade of Nifi to use version 1.15.3 that starts up with HTTPS only and it isn't working anymore.
docker-compose :
services:
nifi:
image: apache/nifi:1.15.3
container_name: nifi
environment:
- JAVA_TOOL_OPTIONS=-Dfile.encoding=utf8
- NIFI_WEB_HTTPS_PORT=8443
ports:
- 10000:8443
- "9001-9100:9001-9100"
volumes:
- ./data/nifi/content:/opt/nifi/nifi-current/content_repository:rw
- ./data/nifi/database:/opt/nifi/nifi-current/database_repository:rw
- ./data/nifi/flowfile:/opt/nifi/nifi-current/flowfile_repository:rw
- ./data/nifi/provenance:/opt/nifi/nifi-current/provenance_repository:rw
- ./data/nifi/state:/opt/nifi/nifi-current/state:rw
- ./data/db/driver:/opt/driver
Logs from nifi-app.log :
nifi-app_2022-03-16_14.0.log:36362:2022-03-16 14:36:43,370 INFO [main] org.apache.nifi.web.server.JettyServer Created HostHeaderHandler [HostHeaderHandler for 2dec0477131a:8443}]
nifi-app_2022-03-16_14.0.log:139435:2022-03-16 14:37:07,253 INFO [main] o.eclipse.jetty.server.AbstractConnector Started ServerConnector#36b53f08{SSL, (ssl, http/1.1)}{2dec0477131a:8443}
nifi-app_2022-03-16_14.0.log:139933:2022-03-16 14:37:07,292 INFO [main] org.apache.nifi.web.server.JettyServer https://2dec0477131a:8443/nifi
Error in logs when I go to https://localhost:10000/nifi :
2022-03-16 14:39:15,572 WARN [NiFi Web Server-16] o.a.nifi.web.server.HostHeaderHandler Request host header [localhost:10000] different from web hostname [2dec0477131a(:8443)]. Overriding to [2dec0477131a:8443/nifi]
2022-03-16 14:39:15,655 WARN [NiFi Web Server-20] o.a.nifi.web.server.HostHeaderHandler Request host header [localhost:10000] different from web hostname [2dec0477131a(:8443)]. Overriding to [2dec0477131a:8443/favicon.ico]
[![enter image description here][1]][1]
I see that on docker desktop when I do open in browser it opens http://localhost:10000/nifi , how can I change this url ?
Update1 :
I found this article : https://www.javahelps.com/2022/01/run-apache-nifi-in-docker-with-ssl.html so my docker-compose looks like this :
services:
nifi:
image: apache/nifi:1.15.3
container_name: nifi
environment:
- JAVA_TOOL_OPTIONS=-Dfile.encoding=utf8
- NIFI_WEB_HTTPS_PORT=8443
- SINGLE_USER_CREDENTIALS_USERNAME=jhiihihhu
- SINGLE_USER_CREDENTIALS_PASSWORD=e9o6dQhlEDusiq6oKUpzp8ZtW/Bjim6q2U0SxyBO6JA
- NIFI_SENSITIVE_PROPS_KEY=rHkWR1gDNW3R
- NIFI_WEB_PROXY_HOST=0.0.0.0:8443
- NIFI_WEB_HTTPS_HOST=0.0.0.0
- NIFI_CLUSTER_ADDRESS=0.0.0.0
- NIFI_REMOTE_INPUT_HOST=0.0.0.0
- AUTH=tls
- KEYSTORE_PATH=/opt/certs/keystore.jks
- KEYSTORE_TYPE=JKS
- KEYSTORE_PASSWORD=cjwC2UU/ZBxhlNyjFbK1QCE/bqNxq9WONwCPq8HbyRk
- TRUSTSTORE_PATH=/opt/certs/truststore.jks
- TRUSTSTORE_TYPE=JKS
- TRUSTSTORE_PASSWORD=1ab4TYRgYw7HMY4Dr+7s+7gVDrPBbZPGp4g/t5VhJG4
- NIFI_SECURITY_USER_AUTHORIZER=single-user-authorizer
- NIFI_SECURITY_USER_LOGIN_IDENTITY_PROVIDER=single-user-provider
ports:
- 10000:8443
- "9001-9100:9001-9100"
volumes:
- ./data/nifi/content:/opt/nifi/nifi-current/content_repository:rw
- ./data/nifi/database:/opt/nifi/nifi-current/database_repository:rw
- ./data/nifi/flowfile:/opt/nifi/nifi-current/flowfile_repository:rw
- ./data/nifi/provenance:/opt/nifi/nifi-current/provenance_repository:rw
- ./data/nifi/state:/opt/nifi/nifi-current/state:rw
- ./data/db/driver:/opt/driver
- ./keystore.jks:/opt/certs/keystore.jks
- ./truststore.jks:/opt/certs/truststore.jks
I still have the same error .
[1]: https://i.stack.imgur.com/OGqE6.png
Update 2:
It worked with the following docker compose :
services:
nifi:
image: apache/nifi:1.15.3
container_name: nifi
environment:
- JAVA_TOOL_OPTIONS=-Dfile.encoding=utf8
- NIFI_WEB_HTTPS_PORT=8443
- SINGLE_USER_CREDENTIALS_USERNAME=khiuiuiuh
- SINGLE_USER_CREDENTIALS_PASSWORD=e9o6dQhlEDusiq6oKUpzp8ZtW/Bjim6q2U0SxyBO6JA
- NIFI_SENSITIVE_PROPS_KEY=rHkWR1gDNW3R
- NIFI_WEB_PROXY_HOST='localhost:8443,localhost:10000'
- NIFI_WEB_HTTPS_HOST=0.0.0.0
- NIFI_CLUSTER_ADDRESS=0.0.0.0
- NIFI_REMOTE_INPUT_HOST=0.0.0.0
- AUTH=tls
- KEYSTORE_PATH=/opt/certs/keystore.jks
- KEYSTORE_TYPE=JKS
- KEYSTORE_PASSWORD=cjwC2UU/ZBxhlNyjFbK1QCE/bqNxq9WONwCPq8HbyRk
- TRUSTSTORE_PATH=/opt/certs/truststore.jks
- TRUSTSTORE_TYPE=JKS
- TRUSTSTORE_PASSWORD=1ab4TYRgYw7HMY4Dr+7s+7gVDrPBbZPGp4g/t5VhJG4
- NIFI_SECURITY_USER_AUTHORIZER=single-user-authorizer
- NIFI_SECURITY_USER_LOGIN_IDENTITY_PROVIDER=single-user-provider
ports:
- 8443:8443
- "9001-9100:9001-9100"
volumes:
- ./data/nifi/content:/opt/nifi/nifi-current/content_repository:rw
- ./data/nifi/database:/opt/nifi/nifi-current/database_repository:rw
- ./data/nifi/flowfile:/opt/nifi/nifi-current/flowfile_repository:rw
- ./data/nifi/provenance:/opt/nifi/nifi-current/provenance_repository:rw
- ./data/nifi/state:/opt/nifi/nifi-current/state:rw
- ./data/db/driver:/opt/driver
- ./keystore.jks:/opt/certs/keystore.jks
- ./truststore.jks:/opt/certs/truststore.
Update 30/03/2022:
I've been using the new version for few days now and each time I start Nifi, I have the error message : The supplied username and password are not valid.
I have to run docker-compose and start all over.
Also I want to enable anonymous authentication and it's not working even if I set NIFI_SECURITY_ALLOW_ANONYMOUS_AUTHENTICATION to true .
Try adding NIFI_WEB_PROXY_HOST=localhost:8443 in the environment section in docker-compose. Also, you have to check if you can use port 10000 for Nifi.

traefik - simple modification of URL

I want to modify the endpoints of my URL, while it goes from traefik to one of my containers. What I want is this.
My URL looks like this - http://backend/asd and it should point to one of my containers with different endpoint like this - http://asd/dfg
What I tried -
asd:
image: asd
container_name: "asd"
labels:
- "traefik.backend=asd"
- "traefik.frontend.rule=Host:backend;PathPrefixStrip:/asd,PathPrefix:/dfg"
- "traefik.frontend.entryPoints=http"
- "traefik.enable=true"
- "traefik.port=80"
But this didn't work. Any suggestions are welcome.
Regards,
Ashutosh
Are you including a Traefik-docker-image in your docker-compose, like so?
traefik:
image: traefik
ports:
- 8080:80
volumes:
- /var/run/docker.sock:/var/run/docker.sock
command:
- "--docker"
For your path-replacement "/asd" -> "/dfg", Traefik's ReplacePath-modifier should do the trick. This following docker-label is the only, you will need:
labels:
- "traefik.frontend.rule=Path: /asd; ReplacePath: /dfg"
Having this setup, doing
curl http://localhost:8080/asd
should return the HTTP-response from your "asd"-container on path "/dfg"

bad gateway or gateway timeout alternatively with traefik and directus

I try to run a directus container with a reverse-proxy.
First try without reverse-proxy: it works on http://my.ip:8585
version: '2'
#
# [ server stack ]
#
services:
mysql:
image: mariadb:latest
container_name: directus-mrdb
restart: always
environment:
- MYSQL_ROOT_PASSWORD=directus
- MYSQL_DATABASE=directus
- MYSQL_USER=directus
- MYSQL_PASSWORD=directus
# If you need to specify a persistent volume for mysql
volumes:
- ./data/db:/var/lib/mysql
directus:
depends_on:
- mysql
image: getdirectus/directus:6.4
container_name: directus
# links:
# - mysql:mysql
ports:
- 8585:8080
restart: always
volumes:
# Switch between named storage, path storage
- ./data/storage:/var/www/html/storage
environment:
- MYSQL_USER=directus
- MYSQL_PASSWORD=directus
- MYSQL_ENV_MYSQL_DATABASE=directus
- MYSQL_PORT_3306_TCP_ADDR=mysql
- MYSQL_PORT_3306_TCP_PORT=3306
- MYSQL_ENV_MYSQL_PASSWORD=directus
- MYSQL_ENV_MYSQL_USER=directus
Second try: i get an gateway timeout error on https://admin.domain.com
version: '2'
#
# [ server stack ]
#
services:
mysql:
image: mariadb:latest
container_name: directus-mrdb
restart: always
environment:
- MYSQL_ROOT_PASSWORD=directus
- MYSQL_DATABASE=directus
- MYSQL_USER=directus
- MYSQL_PASSWORD=directus
# If you need to specify a persistent volume for mysql
volumes:
- ./data/db:/var/lib/mysql
directus:
depends_on:
- mysql
image: getdirectus/directus:6.4
container_name: directus
# links:
# - mysql:mysql
ports:
- 8585:8080
restart: always
volumes:
# Switch between named storage, path storage
- ./data/storage:/var/www/html/storage
environment:
- MYSQL_USER=directus
- MYSQL_PASSWORD=directus
- MYSQL_ENV_MYSQL_DATABASE=directus
- MYSQL_PORT_3306_TCP_ADDR=mysql
- MYSQL_PORT_3306_TCP_PORT=3306
- MYSQL_ENV_MYSQL_PASSWORD=directus
- MYSQL_ENV_MYSQL_USER=directus
labels:
- "traefik.frontend.rule=Host:admin.domain.com"
- "traefik.backend=admin"
- "traefik.port=8585"
- "traefik.frontend.entryPoints=http,https"
- "traefik.enable=true"
- "traefik.docker.network=config_traefik_network"
I understand because directus container is not connected with traefik container.
Third try: i get an bad gateway error on https://admin.domain.com
version: '2'
#
# [ server stack ]
#
services:
mysql:
image: mariadb:latest
container_name: directus-mrdb
restart: always
environment:
- MYSQL_ROOT_PASSWORD=directus
- MYSQL_DATABASE=directus
- MYSQL_USER=directus
- MYSQL_PASSWORD=directus
# If you need to specify a persistent volume for mysql
volumes:
- ./data/db:/var/lib/mysql
directus:
depends_on:
- mysql
image: getdirectus/directus:6.4
container_name: directus
# links:
# - mysql:mysql
ports:
- 8585:8080
restart: always
volumes:
# Switch between named storage, path storage
- ./data/storage:/var/www/html/storage
networks:
- config_traefik_network
environment:
- MYSQL_USER=directus
- MYSQL_PASSWORD=directus
- MYSQL_ENV_MYSQL_DATABASE=directus
- MYSQL_PORT_3306_TCP_ADDR=mysql
- MYSQL_PORT_3306_TCP_PORT=3306
- MYSQL_ENV_MYSQL_PASSWORD=directus
- MYSQL_ENV_MYSQL_USER=directus
labels:
- "traefik.frontend.rule=Host:admin.domain.com"
- "traefik.backend=admin"
- "traefik.port=8585"
- "traefik.frontend.entryPoints=http,https"
- "traefik.enable=true"
- "traefik.docker.network=config_traefik_network"
#
# [ networks definition ]
#
networks:
config_traefik_network:
external: true
I don't understand in this case.
There is a connection between the container.
Backend url is http://172.18.0.7:8585. The certificate seems to be well generated.
Any help will be appreciate.
Thank you in advance.
jB

Traefik : how to redirect using only docker-compose.yml?

I use traefik without traefik.toml, only a docker-compose.ymal file. I've managed to do whatever I want except one thing : automatically redirect one service from http to https.
Is it possible ?
Here is my docker-compose (extract)
version: '2'
services:
###################################
# TRAEFIK
###################################
traefik:
image: traefik:latest
command: --web --docker --docker.domain=docker.localhost --logLevel=DEBUG --entryPoints='Name:https Address::443 TLS' --entryPoints='Name:http Address::80' --acme.entrypoint=https --acme=true --acme.domains="${BASE_URL}, ${ADMIN_URL}" --acme.email="${MAIL_ADDRESS}" --acme.ondemand=true acme.onhostrule=true --acme.storage=/certs/acme.json
ports:
- "80:80"
- "443:443"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /dev/null:/traefik.toml
- $CONFIG_DIR/traefik:/etc/traefik/acme
- $CONFIG_DIR/certs:/certs
labels:
- "traefik.backend=traefik"
- "traefik.frontend.rule=Host:${ADMIN_URL}"
- "traefik.backend.port=8080"
- "traefik.frontend.auth.basic=admin:${passwd_admin}"
- "traefik.frontend.entryPoints=https"
rutorrent-steph:
image: xataz/rtorrent-rutorrent
ports:
- "45001:45001" #UserPort
labels:
- "traefik.backend=rutorrent_steph"
- "traefik.frontend.rule=Host:${BASE_URL};PathPrefix:/steph_rutorrent"
- "traefik.backend.port=8080"
- "traefik.frontend.auth.basic=steph:${passwd_steph}"
- "traefik.frontend.entryPoints=https,http"
environment:
- UID=${MYUID}
- GID=${MYGID}
- PORT_RTORRENT=45001
- WEBROOT=/steph_rutorrent
volumes:
- $DATA_DIR/steph/data:/data
- $DATA_DIR/steph/config/torrent:/config
dns: 8.8.8.8
When I use http://{ BASE_URL}/steph_rutorrent or https://{ BASE_URL}/steph_rutorrent, it works every time. But I'd like to make sure that if I type http://{ BASE_URL}/steph_rutorrent it will redirect to https://{ BASE_URL}/steph_rutorrent
Try to add this to the command:
--entryPoints='Name:http Address::80 Redirect.EntryPoint:https'
instead of
--entryPoints='Name:http Address::80'