How to use ssl with docker desktop on windows - ssl

I have a dev Environment in docker and I'm using Nifi 1.13.2 and it's working fine (http).
I did an upgrade of Nifi to use version 1.15.3 that starts up with HTTPS only and it isn't working anymore.
docker-compose :
services:
nifi:
image: apache/nifi:1.15.3
container_name: nifi
environment:
- JAVA_TOOL_OPTIONS=-Dfile.encoding=utf8
- NIFI_WEB_HTTPS_PORT=8443
ports:
- 10000:8443
- "9001-9100:9001-9100"
volumes:
- ./data/nifi/content:/opt/nifi/nifi-current/content_repository:rw
- ./data/nifi/database:/opt/nifi/nifi-current/database_repository:rw
- ./data/nifi/flowfile:/opt/nifi/nifi-current/flowfile_repository:rw
- ./data/nifi/provenance:/opt/nifi/nifi-current/provenance_repository:rw
- ./data/nifi/state:/opt/nifi/nifi-current/state:rw
- ./data/db/driver:/opt/driver
Logs from nifi-app.log :
nifi-app_2022-03-16_14.0.log:36362:2022-03-16 14:36:43,370 INFO [main] org.apache.nifi.web.server.JettyServer Created HostHeaderHandler [HostHeaderHandler for 2dec0477131a:8443}]
nifi-app_2022-03-16_14.0.log:139435:2022-03-16 14:37:07,253 INFO [main] o.eclipse.jetty.server.AbstractConnector Started ServerConnector#36b53f08{SSL, (ssl, http/1.1)}{2dec0477131a:8443}
nifi-app_2022-03-16_14.0.log:139933:2022-03-16 14:37:07,292 INFO [main] org.apache.nifi.web.server.JettyServer https://2dec0477131a:8443/nifi
Error in logs when I go to https://localhost:10000/nifi :
2022-03-16 14:39:15,572 WARN [NiFi Web Server-16] o.a.nifi.web.server.HostHeaderHandler Request host header [localhost:10000] different from web hostname [2dec0477131a(:8443)]. Overriding to [2dec0477131a:8443/nifi]
2022-03-16 14:39:15,655 WARN [NiFi Web Server-20] o.a.nifi.web.server.HostHeaderHandler Request host header [localhost:10000] different from web hostname [2dec0477131a(:8443)]. Overriding to [2dec0477131a:8443/favicon.ico]
[![enter image description here][1]][1]
I see that on docker desktop when I do open in browser it opens http://localhost:10000/nifi , how can I change this url ?
Update1 :
I found this article : https://www.javahelps.com/2022/01/run-apache-nifi-in-docker-with-ssl.html so my docker-compose looks like this :
services:
nifi:
image: apache/nifi:1.15.3
container_name: nifi
environment:
- JAVA_TOOL_OPTIONS=-Dfile.encoding=utf8
- NIFI_WEB_HTTPS_PORT=8443
- SINGLE_USER_CREDENTIALS_USERNAME=jhiihihhu
- SINGLE_USER_CREDENTIALS_PASSWORD=e9o6dQhlEDusiq6oKUpzp8ZtW/Bjim6q2U0SxyBO6JA
- NIFI_SENSITIVE_PROPS_KEY=rHkWR1gDNW3R
- NIFI_WEB_PROXY_HOST=0.0.0.0:8443
- NIFI_WEB_HTTPS_HOST=0.0.0.0
- NIFI_CLUSTER_ADDRESS=0.0.0.0
- NIFI_REMOTE_INPUT_HOST=0.0.0.0
- AUTH=tls
- KEYSTORE_PATH=/opt/certs/keystore.jks
- KEYSTORE_TYPE=JKS
- KEYSTORE_PASSWORD=cjwC2UU/ZBxhlNyjFbK1QCE/bqNxq9WONwCPq8HbyRk
- TRUSTSTORE_PATH=/opt/certs/truststore.jks
- TRUSTSTORE_TYPE=JKS
- TRUSTSTORE_PASSWORD=1ab4TYRgYw7HMY4Dr+7s+7gVDrPBbZPGp4g/t5VhJG4
- NIFI_SECURITY_USER_AUTHORIZER=single-user-authorizer
- NIFI_SECURITY_USER_LOGIN_IDENTITY_PROVIDER=single-user-provider
ports:
- 10000:8443
- "9001-9100:9001-9100"
volumes:
- ./data/nifi/content:/opt/nifi/nifi-current/content_repository:rw
- ./data/nifi/database:/opt/nifi/nifi-current/database_repository:rw
- ./data/nifi/flowfile:/opt/nifi/nifi-current/flowfile_repository:rw
- ./data/nifi/provenance:/opt/nifi/nifi-current/provenance_repository:rw
- ./data/nifi/state:/opt/nifi/nifi-current/state:rw
- ./data/db/driver:/opt/driver
- ./keystore.jks:/opt/certs/keystore.jks
- ./truststore.jks:/opt/certs/truststore.jks
I still have the same error .
[1]: https://i.stack.imgur.com/OGqE6.png
Update 2:
It worked with the following docker compose :
services:
nifi:
image: apache/nifi:1.15.3
container_name: nifi
environment:
- JAVA_TOOL_OPTIONS=-Dfile.encoding=utf8
- NIFI_WEB_HTTPS_PORT=8443
- SINGLE_USER_CREDENTIALS_USERNAME=khiuiuiuh
- SINGLE_USER_CREDENTIALS_PASSWORD=e9o6dQhlEDusiq6oKUpzp8ZtW/Bjim6q2U0SxyBO6JA
- NIFI_SENSITIVE_PROPS_KEY=rHkWR1gDNW3R
- NIFI_WEB_PROXY_HOST='localhost:8443,localhost:10000'
- NIFI_WEB_HTTPS_HOST=0.0.0.0
- NIFI_CLUSTER_ADDRESS=0.0.0.0
- NIFI_REMOTE_INPUT_HOST=0.0.0.0
- AUTH=tls
- KEYSTORE_PATH=/opt/certs/keystore.jks
- KEYSTORE_TYPE=JKS
- KEYSTORE_PASSWORD=cjwC2UU/ZBxhlNyjFbK1QCE/bqNxq9WONwCPq8HbyRk
- TRUSTSTORE_PATH=/opt/certs/truststore.jks
- TRUSTSTORE_TYPE=JKS
- TRUSTSTORE_PASSWORD=1ab4TYRgYw7HMY4Dr+7s+7gVDrPBbZPGp4g/t5VhJG4
- NIFI_SECURITY_USER_AUTHORIZER=single-user-authorizer
- NIFI_SECURITY_USER_LOGIN_IDENTITY_PROVIDER=single-user-provider
ports:
- 8443:8443
- "9001-9100:9001-9100"
volumes:
- ./data/nifi/content:/opt/nifi/nifi-current/content_repository:rw
- ./data/nifi/database:/opt/nifi/nifi-current/database_repository:rw
- ./data/nifi/flowfile:/opt/nifi/nifi-current/flowfile_repository:rw
- ./data/nifi/provenance:/opt/nifi/nifi-current/provenance_repository:rw
- ./data/nifi/state:/opt/nifi/nifi-current/state:rw
- ./data/db/driver:/opt/driver
- ./keystore.jks:/opt/certs/keystore.jks
- ./truststore.jks:/opt/certs/truststore.
Update 30/03/2022:
I've been using the new version for few days now and each time I start Nifi, I have the error message : The supplied username and password are not valid.
I have to run docker-compose and start all over.
Also I want to enable anonymous authentication and it's not working even if I set NIFI_SECURITY_ALLOW_ANONYMOUS_AUTHENTICATION to true .

Try adding NIFI_WEB_PROXY_HOST=localhost:8443 in the environment section in docker-compose. Also, you have to check if you can use port 10000 for Nifi.

Related

Trouble starting influxdb & chronograf with certificates. [open /etc/letsencrypt/certs/influx.domain.com.crt: no such file or directory]

While trying to run part of the TICK stack (influx and chronograf) through a nginx proxy with letsencrypt certificates i am getting an error that the certs files are not found.
Here is my docker-compose file:
version: '3'
services:
influxdb:
container_name: influxdb
image: quay.io/influxdb/influxdb:v2.0.2
restart: always
ports:
- '8086:8086'
volumes:
- './data:/root/.influxdbv2'
- /root/new/letsencrypt/certs:/etc/letsencrypt/certs:ro
environment:
- INFLUXD_TLS_CERT=/etc/letsencrypt/certs/influx.lizardsolutions.com.crt
- INFLUXD_TLS_KEY=/etc/letsencrypt/certs/influx.lizardsolutions.com.key
chronograf:
image: chronograf:1.7.14
container_name: chronograf
restart: always
hostname: chronograf
network_mode: bridge
ports:
- 3000:8888/tcp
volumes:
- /var/lib/chronograf:/var/lib/chronograf
- /root/new/letsencrypt/certs:/etc/letsencrypt/certs:ro
environment:
- VIRTUAL_HOST=influx.lizardsolutions.com# adjust to match your domain name
- VIRTUAL_PROTO=https
- VIRTUAL_PORT=3000
- LETSENCRYPT_HOST=influx.lizardsolutions.com # adjust to match your domain name -
- LETSENCRYPT_EMAIL=admin#lizardsolutions.com # adjust to match your email
- BOLT_PATH=/var/lib/chronograf/chronograf-v1.db
- CANNED_PATH=/usr/share/chronograf/canned
- RESOURCES_PATH=/usr/share/chronograf/resources
- PROTOBOARDS_PATH=/usr/share/chronograf/protoboards
# - INFLUXDB_URL=http://127.0.0.1:8086
#INFLUX_CREDENTIALS
# - INFLUXDB_USERNAME=grafanam9k5bXbBj7GPGdf3
# - INFLUXDB_PASSWORD=k9H4TV8ifRo7E6mNyv3L
- TLS_CERTIFICATE=/etc/letsencrypt/certs/influx.domain.com.crt
- TLS_PRIVATE_KEY=/etc/letsencrypt/certs/influx.domain.com.key
- REPORTING_DISABLED='false'
# Set the logging level. Valid values: debug | info | error
- LOG_LEVEL=info
- TOKEN_SECRET=hxDEXBPAc49muW3gLoot7YamJ
- AUTH_DURATION=0
- PUBLIC_URL=https://ec-influx.domain.com
nginx-proxy:
image: jwilder/nginx-proxy
container_name: nginx
restart: always
labels:
- com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy
ports:
- "80:80"
- "443:443"
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
- ./nginx/html:/usr/share/nginx/html
- ./nginx/vhost.d:/etc/nginx/vhost.d
- /root/new/letsencrypt/certs:/etc/nginx/certs:ro
letsencrypt:
image: jrcs/letsencrypt-nginx-proxy-companion
container_name: letsencrypt
restart: always
environment:
- NGINX_PROXY_CONTAINER=nginx-proxy
- REUSE_PRIVATE_KEYS=true
volumes_from:
- nginx-proxy
volumes:
- /root/new/letsencrypt/certs:/etc/nginx/certs:rw
- /var/run/docker.sock:/var/run/docker.sock:ro
Here is the error from chronograf:
chronograf | time="2022-08-29T00:22:02Z" level=error msg="open /etc/letsencrypt/certs/influx.domain.com.crt: no such file or directory" component=server
chronograf | 2022/08/29 00:22:02 open /etc/letsencrypt/certs/influx.domain.com.crt: no such file or directory
The same error shows for influxdb as well
From where can i start debuging ? While navigating to the mounted volume (./letsencrypt/certs/) i can confirm the key and cert file is there.

Dockware with Traefik

I try to proxy a dockware container through traefik.
The error is an internal server error (500).
Is it necessary to change the domainname at all? If so, how can I change the domain name?
Docker compose for shopware
version: "3"
services:
shopwaretest:
image: dockware/play:latest
container_name: shopwaretest
restart: always
volumes:
- "db_shopwaretest:/var/lib/mysql"
- "shopwaretest:/var/www/html"
- ./hosts:/etc/hosts
networks:
- proxy
environment:
- XDEBUG_ENABLED=0
- PHP_VERSION=8.0
labels:
- "traefik.enable=true"
- "traefik.http.routers.shopwaretest-http.rule=Host(`example.com`)"
- "traefik.http.routers.shopwaretest-http.entrypoints=http"
- "traefik.http.routers.shopwaretest-http.service=shopwaretest-http-service"
- "traefik.http.services.shopwaretest-http-service.loadbalancer.server.port=80"
- "traefik.http.routers.shopwaretest-https.rule=Host(`example.com`)"
- "traefik.http.routers.shopwaretest-https.entrypoints=https"
- "traefik.http.routers.shopwaretest-https.service=shopwaretest-https-service"
- "traefik.http.services.shopwaretest-https-service.loadbalancer.server.port=80"
- "traefik.http.routers.shopwaretest-https.tls=true"
- "traefik.http.routers.shopwaretest-http.middlewares=redirect#file"
- "traefik.http.routers.shopwaretest-https.tls.certresolver=http"
volumes:
db_shopwaretest:
driver: local
shopwaretest:
driver: local
networks:
proxy:
external: true
If you get an internal server error, please check the server logs.
You can manually change the domain name in the sales_channel_domain table.
The problem might be, that SSL is terminated on traefik and Shopware does not detect this - if this is the problem, you might need to set the TRUESTED_PROXIES variable to the IP of your traefik server/container.

Portainer doesn't show icons anymore since upgrading to v2 (Traefik Proxy)

Since upgrading to Portainer v2, the icons would suddenly not load anymore. I can still access Portainer (which is proxied by Traefik), but after a bit of testing, I noticed, only / would be forwarded. If a path was given, Traefik would throw a 404 error. This is a problem because Portainer loads the fonts from eg. /b15db15f746f29ffa02638cb455b8ec0.woff2.
There is one issue about this on Github, but I don't really know what to do with that information: https://github.com/portainer/portainer/issues/3706
My Traefik configuration
version: "2"
# Manage domain access to services
services:
traefik:
container_name: traefik
image: traefik
command:
- --api.dashboard=true
- --certificatesresolvers.le.acme.email=${ACME_EMAIL}
- --certificatesresolvers.le.acme.storage=acme.json
# Enable/Disable staging by commenting/uncommenting the next line
# - --certificatesresolvers.le.acme.caserver=https://acme-staging-v02.api.letsencrypt.org/directory
- --certificatesresolvers.le.acme.dnschallenge=true
- --certificatesresolvers.le.acme.dnschallenge.provider=cloudflare
- --entrypoints.http.address=:80
- --entrypoints.https.address=:443
- --global.sendAnonymousUsage
- --log.level=INFO
- --providers.docker=true
- --providers.docker.exposedbydefault=false
- --providers.docker.network=traefik_proxy
restart: always
networks:
- traefik_proxy
ports:
- "80:80"
- "443:443"
dns:
- 1.1.1.1
- 1.0.0.1
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- ./acme.json:/acme.json
# - ./acme-staging.json:/acme.json
environment:
CF_API_EMAIL: ${CLOUDFLARE_EMAIL}
CF_API_KEY: ${CLOUDFLARE_API_KEY}
labels:
- traefik.enable=true
- traefik.http.routers.traefik0.entrypoints=http
- traefik.http.routers.traefik0.rule=Host(`${TRAEFIK_URL}`)
- traefik.http.routers.traefik0.middlewares=to_https
- traefik.http.routers.traefik.entrypoints=https
- traefik.http.routers.traefik.rule=Host(`${TRAEFIK_URL}`)
- traefik.http.routers.traefik.middlewares=traefik_auth
- traefik.http.routers.traefik.tls=true
- traefik.http.routers.traefik.tls.certresolver=le
- traefik.http.routers.traefik.service=api#internal
# Declaring the user list
#
# Note: all dollar signs in the hash need to be doubled for escaping.
# To create user:password pair, it's possible to use this command:
# echo $(htpasswd -nb user password) | sed -e s/\\$/\\$\\$/g
- traefik.http.middlewares.traefik_auth.basicauth.users=${TRAEFIK_USERS}
# Standard middleware for other containers to use
- traefik.http.middlewares.to_https.redirectscheme.scheme=https
- traefik.http.middlewares.to_https_perm.redirectscheme.scheme=https
- traefik.http.middlewares.to_https_perm.redirectscheme.permanent=true
networks:
traefik_proxy:
external: true
And my Portainer configuration
version: "2"
# Manage docker containers
services:
portainer:
container_name: portainer
image: portainer/portainer-ce
restart: always
networks:
- traefik_proxy
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./data/:/data/
labels:
- traefik.enable=true
- traefik.http.services.portainer.loadbalancer.server.port=9000
- traefik.http.routers.portainer0.entrypoints=http
- traefik.http.routers.portainer0.rule=Host(`${PORTAINER_URL}`)
- traefik.http.routers.portainer0.middlewares=to_https
- traefik.http.routers.portainer.entrypoints=https
- traefik.http.routers.portainer.rule=Host(`${PORTAINER_URL}`)
- traefik.http.routers.portainer.tls=true
- traefik.http.routers.portainer.tls.certresolver=le
networks:
traefik_proxy:
external: true
What do I have to change to make Traefik be able to forward the paths so that Portainer can load the icons?
Could you try flush your DNS Cache?
In Chrome 'chrome://net-internals/#dns' into URL bar and pressed enter.
Then click on 'Clear host cache'
Then refresh your portainer page
I noticed that there is also an Alpine version of Portainer.
After switching to that (image: portainer/portainer-ce:alpine), the icons seem to be working again. I don't know what the issue is with the regular image, but this solves it for now.
PS: I had tried to use the Access-Control header on Traefik, but that didn't help. I guess it's a problem with Portainer's code itself.
If someone else is facing this issue, I resolved this by deleting my Browser Cache or just do a full Refresh with CTRL+Shift+R

Running Graylog behind Traefik

I have been using Traefik as a reverse proxy for several containers managed through docker compose. It has been working well but I'm having some difficulty getting graylog running properly behind it.
This config (excerpt of the complete file, removed irrelevant services) works for me
version: '3'
services:
traefik:
image: traefik
command: --docker
networks:
- web
ports:
- "80:80"
- "8080:8080"
- "443:443"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
graylog:
image: graylog/graylog:3.0
environment:
- GRAYLOG_PASSWORD_SECRET=somepasswordpepper
- GRAYLOG_ROOT_PASSWORD_SHA2=somestuffhere
- GRAYLOG_HTTP_BIND_ADDRESS=0.0.0.0:9000
- GRAYLOG_HTTP_EXTERNAL_URI=http://graylog.localhost/
links:
- mongodb:mongo
- elasticsearch
depends_on:
- mongodb
- elasticsearch
networks:
- web
- default
ports:
- 9000:9000
- 1514:1514
- 1514:1514/udp
- 12201:12201
- 12201:12201/udp
labels:
- "traefik.docker.network=web"
- "traefik.web.frontend.rule=Host:graylog.localhost"
- "traefik.protocol=http"
- "traefik.port=9000"
- "traefik.enable=true"
I can then access graylog in my browser at http://graylog.localhost/
The problem occurs because I can't use domains like this in my staging env. What I have had to use is paths rather than domains e.g. the swagger service in my staging env is defined as
swagger:
image: swaggerapi/swagger-ui
environment:
SWAGGER_JSON: /swagger/staging-openapi.yaml
BASE_URL: /swagger
volumes:
- /home/ubuntu/src/carelink_swagger_service:/swagger
networks:
- web
- default
labels:
- "traefik.docker.network=web"
- "traefik.frontend.rule=Host:my.staging.domain.org; PathPrefix: /swagger"
- "traefik.protocol=http"
- "traefik.port=8080"
- "traefik.enable=true"
and I access services like this https://my.staging.domain.org/swagger/
Trying to adopt this approach for graylog
graylog:
image: graylog/graylog:3.0
environment:
- GRAYLOG_PASSWORD_SECRET=somepasswordpepper
- GRAYLOG_ROOT_PASSWORD_SHA2=somestuffhere
- GRAYLOG_HTTP_BIND_ADDRESS=0.0.0.0:9000
- GRAYLOG_HTTP_EXTERNAL_URI=http://localhost/graylog/
links:
- mongodb:mongo
- elasticsearch
depends_on:
- mongodb
- elasticsearch
networks:
- web
- default
ports:
- 9000:9000
- 1514:1514
- 1514:1514/udp
- 12201:12201
- 12201:12201/udp
labels:
- "traefik.docker.network=web"
- "traefik.web.frontend.rule=Host:localhost; PathPrefix: /graylog"
- "traefik.protocol=http"
- "traefik.port=9000"
- "traefik.enable=true"
and I get the following errors
I have gone through this Q/A, similar entries on the graylog forums etc. but I feel like I'm chasing my tail.
It's for Traefik 2.x but I think that you'll manage to convert your rules:
labels:
- "traefik.http.routers.graylog.rule=PathPrefix(`/graylog`)"
- "traefik.http.routers.graylog.middlewares=graylog3#docker,graylog2#docker,graylog#docker"
- "traefik.http.middlewares.graylog3.stripprefix.prefixes=/graylog"
- "traefik.http.middlewares.graylog2.redirectregex.regex=^(.*)/graylog$$"
- "traefik.http.middlewares.graylog2.redirectregex.replacement=$$1/graylog/"
- "traefik.http.middlewares.graylog.replacepathregex.regex=^/graylog/(.*)"
- "traefik.http.middlewares.graylog.replacepathregex.replacement=^/$$1"
- "traefik.http.routers.graylog.service=graylog"
- "traefik.http.services.graylog.loadbalancer.server.port=9000"
- "traefik.docker.network=traefik"

Traefik : how to redirect using only docker-compose.yml?

I use traefik without traefik.toml, only a docker-compose.ymal file. I've managed to do whatever I want except one thing : automatically redirect one service from http to https.
Is it possible ?
Here is my docker-compose (extract)
version: '2'
services:
###################################
# TRAEFIK
###################################
traefik:
image: traefik:latest
command: --web --docker --docker.domain=docker.localhost --logLevel=DEBUG --entryPoints='Name:https Address::443 TLS' --entryPoints='Name:http Address::80' --acme.entrypoint=https --acme=true --acme.domains="${BASE_URL}, ${ADMIN_URL}" --acme.email="${MAIL_ADDRESS}" --acme.ondemand=true acme.onhostrule=true --acme.storage=/certs/acme.json
ports:
- "80:80"
- "443:443"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /dev/null:/traefik.toml
- $CONFIG_DIR/traefik:/etc/traefik/acme
- $CONFIG_DIR/certs:/certs
labels:
- "traefik.backend=traefik"
- "traefik.frontend.rule=Host:${ADMIN_URL}"
- "traefik.backend.port=8080"
- "traefik.frontend.auth.basic=admin:${passwd_admin}"
- "traefik.frontend.entryPoints=https"
rutorrent-steph:
image: xataz/rtorrent-rutorrent
ports:
- "45001:45001" #UserPort
labels:
- "traefik.backend=rutorrent_steph"
- "traefik.frontend.rule=Host:${BASE_URL};PathPrefix:/steph_rutorrent"
- "traefik.backend.port=8080"
- "traefik.frontend.auth.basic=steph:${passwd_steph}"
- "traefik.frontend.entryPoints=https,http"
environment:
- UID=${MYUID}
- GID=${MYGID}
- PORT_RTORRENT=45001
- WEBROOT=/steph_rutorrent
volumes:
- $DATA_DIR/steph/data:/data
- $DATA_DIR/steph/config/torrent:/config
dns: 8.8.8.8
When I use http://{ BASE_URL}/steph_rutorrent or https://{ BASE_URL}/steph_rutorrent, it works every time. But I'd like to make sure that if I type http://{ BASE_URL}/steph_rutorrent it will redirect to https://{ BASE_URL}/steph_rutorrent
Try to add this to the command:
--entryPoints='Name:http Address::80 Redirect.EntryPoint:https'
instead of
--entryPoints='Name:http Address::80'