I have a domain of the form api.mydomain.com with a letsencrypt certificate that I'm trying to renew.
root#prod-app-1:/home/ninesalt# certbot certificates
Saving debug log to /var/log/letsencrypt/letsencrypt.log
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Found the following certs:
Certificate Name: api.mydomain.com
Domains: api.mydomain.com
Expiry Date: 2019-06-17 11:25:52+00:00 (VALID: 27 days)
Certificate Path: /etc/letsencrypt/live/api.mydomain.com/fullchain.pem
Private Key Path: /etc/letsencrypt/live/api.mydomain.com/privkey.pem
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
However when I try to renew it with certbot renew I get this error:
Attempting to renew cert (api.mydomain.com) from /etc/letsencrypt/renewal/api.mydomain.com.conf produced an unexpected error: The manual plugin is not working; there may be problems with your existing configuration.
The error was: PluginError('An authentication script must be provided with --manual-auth-hook when using the manual plugin non-interactively.',). Skipping.
All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/api.mydomain.com/fullchain.pem (failure)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/api.mydomain.com/fullchain.pem (failure)
Certificates that are created using --manual (and without an authentication hook) cannot be automatically renewed.
This is because it involves you performing the the authorization steps by hand, which is not something that Certbot can automatically repeat at renewal time.
Usually you will want to use an authenticator other than the manual one (such as --apache, --nginx, --webroot, --standalone), so that Certbot can perform automatic renewal.
So, you should probably manually create the certificate again:
certbot certonly --manual -d xxx.com
Related
TLDR; The issue is that LAM cannot successfully authenticate to mailcow (Dovecot) with the DOVECOT_MASTER_USER and DOVECOT_MASTER_PASSWORD setup, the connection is aborted and hence no mailbox was provisioned.
So, I am trying to connect a directory service with mailcow, in order to provision and manage mailboxes for users and groups from LDAP Account Manager (LAM).
The directory service is comprised of bitnamit/OpenLDAP image with LDAP Account Manager (LAM) as Directory Management frontend run as a docker-compose stack behind Traefik as Reverse Proxy just as mailcow.
version: '3.5'
volumes:
lam-app:
lam-data:
openldap-data:
networks:
openldap-net:
driver: bridge
web:
external: true
services:
ldap-account-manager:
image: ldapaccountmanager/lam:latest
restart: unless-stopped
volumes:
- lam-data:/var/lib/ldap-account-manager/
- lam-app:/etc/ldap-account-manager
environment:
# - LAM_PASSWORD=${LAM_PASSWORD}
# - LAM_LANG=${LAM_LANG}
# - LDAP_SERVER=${LDAP_SERVER}
# - LDAP_DOMAIN=${LDAP_DOMAIN}
# - LDAP_BASE_DN=${LDAP_BASE_DN}
# - ADMIN_USER=cn=${LDAP_ADMIN_NAME},${LDAP_BASE_DN}
- LAM_PASSWORD=${LAM_PASSWORD}
- LAM_LANG=${LAM_LANG}
- LDAP_SERVER=${LDAP_SERVER}
- LDAP_DOMAIN=${LDAP_DOMAIN}
- LDAP_BASE_DN=${LDAP_BASE_DN}
- ADMIN_USER=cn=admin,${LDAP_BASE_DN}
# - DEBUG=true
hostname: directory
domainname: "${LDAP_DOMAIN}"
networks:
- openldap-net
- web
labels:
- traefik.enable=true
- traefik.docker.network=web
- traefik.http.routers.openldap-lam.rule=Host(`directory.${LDAP_DOMAIN}`)
- traefik.http.routers.openldap-lam.entrypoints=websecure
- traefik.http.routers.openldap-lam.tls=true
- traefik.http.routers.openldap-lam.tls.certresolver=myresolver
- traefik.http.routers.openldap-lam.service=openldap-lam-svc
- traefik.http.services.openldap-lam-svc.loadbalancer.server.port=80
- traefik.http.routers.openldap-lam.middlewares=ldap-auth
- traefik.http.middlewares.ldap-auth.basicauth.users=adam:$$2y$$05$$aio7rx3j9w87ky2uw9r
certdumper:
image: humenius/traefik-certs-dumper
container_name: ldap_certdumper
network_mode: none
# env_file:
# - ldap.env
volumes:
# mount the folder which contains Traefik's `acme.json' file
- ../../proxy/letsencrypt/:/traefik:ro
- ./ldap/certs/:/output:rw
environment:
- DOMAIN=ldap.${LDAP_DOMAIN}
# Bitnami OpenLDAP container (actively maintained)
ldap:
image: bitnami/openldap:latest
ports:
- '389:1389' # Expose port 389
# - '1636:1636'
networks:
- openldap-net
- web
volumes:
- openldap-data:/bitnami/openldap
- ./ldap/schemas:/schemas # Additional LDIF schemas
labels:
- traefik.enable=true
- traefik.docker.network=web
- traefik.http.routers.ldap.rule=Host(`ldap.${LDAP_DOMAIN}`)
- traefik.http.routers.ldap.tls=true
- traefik.http.routers.ldap.tls.certresolver=myresolver
- traefik.http.routers.ldap.entrypoints=ldap
- traefik.http.routers.ldap.service=ldap-svc
- traefik.http.services.ldap-svc.loadbalancer.server.port=1389
environment:
- LDAP_ADMIN_USERNAME=${LDAP_ADMIN_NAME}
- LDAP_ADMIN_PASSWORD=${LDAP_ADMIN_PASSWORD}
- LDAP_ROOT=${LDAP_BASE_DN}
- LDAP_SKIP_DEFAULT_TREE=${LDAP_SKIP_DEFAULT_TREE}
- LDAP_GROUP=${LDAP_GROUP}
- LDAP_USER_DC=${LDAP_USER_DC}
- LDAP_CONFIG_ADMIN_ENABLED=${LDAP_CONFIG_ADMIN_ENABLED}
- LDAP_CONFIG_ADMIN_USERNAME=${LDAP_CONFIG_ADMIN_NAME}
- LDAP_CONFIG_ADMIN_PASSWORD=${LDAP_CONFIG_ADMIN_PASSWORD}
- LDAP_LOGLEVEL=${LDAP_LOGLEVEL}
- LDAP_PORT_NUMBER=${LDAP_PORT}
- LDAP_USERS=${LDAP_USERS}
- LDAP_PASSWORDS=${LDAP_PASSWORDS}
As LAM documentation states there are no additional schemas needed on the LDAP side, in order to setup and use IMAP mailboxes. But as can be seen I mount several LDIF schemas to extend the basic LDAP functionality. Namely:
asterisk.ldif: Software-PBX
dhcp.schema: DHCP from LDAP
freeradius.ldif: RADIUS from LDAP
ldapns.ldif: Hosts and Allowed Services Management
misc.ldif: E-Mail aliases among other things.
pyroka: printer access and management
IMAP settings in LAM are set as follows:
On the mailcow side, I adjusted mailcow.conf and set DOVECOT_MASTER_USER and DOVECOT_MASTER_PASSWORD. On restart I can find the new user and password created in /etc/dovecot/dovecot-master.userdb and /etc/dovecot/dovecot-master.passdb.
So far so good; LAM connect to Dovecot service of the respective mailcow instance and tries to authenticate with the username / password setup for the DOVECOT_MASTER_USER. But authentication fails with the following logs:
mailcowdockerized-dovecot-mailcow-1 | Feb 4 23:53:53 374f0ac661be dovecot: imap-login: Disconnected: Aborted login by logging out (auth failed, 1 attempts in 2 secs): user=<dovecot-master#mailcow.local>, method=PLAIN, rip=fd4d:6169:6c63:6f77::1, lip=fd4d:6169:6c63:6f77::f, TLS, TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
Could it be due to TLS and certificates? OpenLDAP has TLS turned off internally. TLS termination is done with Traefik, which obtains and manages certificates via Letencrypt. Could this setup not meet Mailcow’s encryption requirements for successful authentication?
Would be grateful for hints and help 🙂
Luke
Is it possible to create new certificate with certbot if I accidentally lost the one I just created?
I was testing certbot with docker and created certificate on my test server. Then I wanted to make sure that running same command for creating certificate, will exit normally. But instead it would get stuck on (which is problematic when running ansible as it would just stuck without any output at all):
Saving debug log to /var/log/letsencrypt/letsencrypt.log
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/myhost.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Certificate not yet due for renewal
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
The following certificates are not due for renewal yet:
/etc/letsencrypt/live/myhost/fullchain.pem expires on 2023-02-28 (skipped)
No renewals were attempted.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
^CERROR: 130
So I decided to just delete certificate and private key and create new certificate (should have made a backup..). But now I always get this:
docker-compose run --rm -p 80:80 certbot certonly --register-unsafely-without-email --agree-tos --standalone --preferred-challenge http -d myhost
Creating app_certbot_run ... done
Saving debug log to /var/log/letsencrypt/letsencrypt.log
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
No renewals were attempted.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
^CERROR: 130
So as I understand letsencrypt knows there is valid certificate registered by them and refuses to create new one. Is there a way to create new one instead of waiting till it expires?..:)
My mistake. I forgot that I added this in docker compose for certbot as entrypoint..
"/bin/sh -c 'trap exit TERM; while :; do certbot renew; sleep 12h & wait $${!}; done;'"
So it would try to renew and be stuck.
I have a month of data stored in prometheus s3 generated with thanos, configure a new thanos with this s3 with thanos with the same configuration as the previous one, but the data is not loaded
the settings are as follows:
prometheus:
- --web.console.templates=/etc/prometheus/consoles
- --web.console.libraries=/etc/prometheus/console_libraries
- --config.file=/etc/prometheus/config_out/prometheus.env.yaml
- --storage.tsdb.path=/prometheus
- --storage.tsdb.retention.time=548d
- --web.enable-lifecycle
- --web.enable-admin-api
- --web.route-prefix=/
- --web.config.file=/etc/prometheus/web_config/web-config.yaml
- --storage.tsdb.max-block-duration=2h
- --storage.tsdb.min-block-duration=2h
config-reloader:
- --listen-address=:8080
- --reload-url=http://127.0.0.1:9090/-/reload
- --config-file=/etc/prometheus/config/prometheus.yaml.gz
- --config-envsubst-file=/etc/prometheus/config_out/prometheus.env.yaml
- --watched-dir=/etc/prometheus/rules/prometheus-prometheus-kube-prometheus-prometheus-rulefiles-0
thanos-sidecar
- sidecar
- --prometheus.url=http://127.0.0.1:9090/
- --grpc-address=:10901
- --http-address=:10902
- --objstore.config=$(OBJSTORE_CONFIG)
- --tsdb.path=/prometheus
- --log.level=info
- --log.format=logfmt
this is deployed with the kube-prometheus-stack heml in kubernetes
I also have the thanos helm deployed with the gateway storage, compactor and querier
I have a dev Environment in docker and I'm using Nifi 1.13.2 and it's working fine (http).
I did an upgrade of Nifi to use version 1.15.3 that starts up with HTTPS only and it isn't working anymore.
docker-compose :
services:
nifi:
image: apache/nifi:1.15.3
container_name: nifi
environment:
- JAVA_TOOL_OPTIONS=-Dfile.encoding=utf8
- NIFI_WEB_HTTPS_PORT=8443
ports:
- 10000:8443
- "9001-9100:9001-9100"
volumes:
- ./data/nifi/content:/opt/nifi/nifi-current/content_repository:rw
- ./data/nifi/database:/opt/nifi/nifi-current/database_repository:rw
- ./data/nifi/flowfile:/opt/nifi/nifi-current/flowfile_repository:rw
- ./data/nifi/provenance:/opt/nifi/nifi-current/provenance_repository:rw
- ./data/nifi/state:/opt/nifi/nifi-current/state:rw
- ./data/db/driver:/opt/driver
Logs from nifi-app.log :
nifi-app_2022-03-16_14.0.log:36362:2022-03-16 14:36:43,370 INFO [main] org.apache.nifi.web.server.JettyServer Created HostHeaderHandler [HostHeaderHandler for 2dec0477131a:8443}]
nifi-app_2022-03-16_14.0.log:139435:2022-03-16 14:37:07,253 INFO [main] o.eclipse.jetty.server.AbstractConnector Started ServerConnector#36b53f08{SSL, (ssl, http/1.1)}{2dec0477131a:8443}
nifi-app_2022-03-16_14.0.log:139933:2022-03-16 14:37:07,292 INFO [main] org.apache.nifi.web.server.JettyServer https://2dec0477131a:8443/nifi
Error in logs when I go to https://localhost:10000/nifi :
2022-03-16 14:39:15,572 WARN [NiFi Web Server-16] o.a.nifi.web.server.HostHeaderHandler Request host header [localhost:10000] different from web hostname [2dec0477131a(:8443)]. Overriding to [2dec0477131a:8443/nifi]
2022-03-16 14:39:15,655 WARN [NiFi Web Server-20] o.a.nifi.web.server.HostHeaderHandler Request host header [localhost:10000] different from web hostname [2dec0477131a(:8443)]. Overriding to [2dec0477131a:8443/favicon.ico]
[![enter image description here][1]][1]
I see that on docker desktop when I do open in browser it opens http://localhost:10000/nifi , how can I change this url ?
Update1 :
I found this article : https://www.javahelps.com/2022/01/run-apache-nifi-in-docker-with-ssl.html so my docker-compose looks like this :
services:
nifi:
image: apache/nifi:1.15.3
container_name: nifi
environment:
- JAVA_TOOL_OPTIONS=-Dfile.encoding=utf8
- NIFI_WEB_HTTPS_PORT=8443
- SINGLE_USER_CREDENTIALS_USERNAME=jhiihihhu
- SINGLE_USER_CREDENTIALS_PASSWORD=e9o6dQhlEDusiq6oKUpzp8ZtW/Bjim6q2U0SxyBO6JA
- NIFI_SENSITIVE_PROPS_KEY=rHkWR1gDNW3R
- NIFI_WEB_PROXY_HOST=0.0.0.0:8443
- NIFI_WEB_HTTPS_HOST=0.0.0.0
- NIFI_CLUSTER_ADDRESS=0.0.0.0
- NIFI_REMOTE_INPUT_HOST=0.0.0.0
- AUTH=tls
- KEYSTORE_PATH=/opt/certs/keystore.jks
- KEYSTORE_TYPE=JKS
- KEYSTORE_PASSWORD=cjwC2UU/ZBxhlNyjFbK1QCE/bqNxq9WONwCPq8HbyRk
- TRUSTSTORE_PATH=/opt/certs/truststore.jks
- TRUSTSTORE_TYPE=JKS
- TRUSTSTORE_PASSWORD=1ab4TYRgYw7HMY4Dr+7s+7gVDrPBbZPGp4g/t5VhJG4
- NIFI_SECURITY_USER_AUTHORIZER=single-user-authorizer
- NIFI_SECURITY_USER_LOGIN_IDENTITY_PROVIDER=single-user-provider
ports:
- 10000:8443
- "9001-9100:9001-9100"
volumes:
- ./data/nifi/content:/opt/nifi/nifi-current/content_repository:rw
- ./data/nifi/database:/opt/nifi/nifi-current/database_repository:rw
- ./data/nifi/flowfile:/opt/nifi/nifi-current/flowfile_repository:rw
- ./data/nifi/provenance:/opt/nifi/nifi-current/provenance_repository:rw
- ./data/nifi/state:/opt/nifi/nifi-current/state:rw
- ./data/db/driver:/opt/driver
- ./keystore.jks:/opt/certs/keystore.jks
- ./truststore.jks:/opt/certs/truststore.jks
I still have the same error .
[1]: https://i.stack.imgur.com/OGqE6.png
Update 2:
It worked with the following docker compose :
services:
nifi:
image: apache/nifi:1.15.3
container_name: nifi
environment:
- JAVA_TOOL_OPTIONS=-Dfile.encoding=utf8
- NIFI_WEB_HTTPS_PORT=8443
- SINGLE_USER_CREDENTIALS_USERNAME=khiuiuiuh
- SINGLE_USER_CREDENTIALS_PASSWORD=e9o6dQhlEDusiq6oKUpzp8ZtW/Bjim6q2U0SxyBO6JA
- NIFI_SENSITIVE_PROPS_KEY=rHkWR1gDNW3R
- NIFI_WEB_PROXY_HOST='localhost:8443,localhost:10000'
- NIFI_WEB_HTTPS_HOST=0.0.0.0
- NIFI_CLUSTER_ADDRESS=0.0.0.0
- NIFI_REMOTE_INPUT_HOST=0.0.0.0
- AUTH=tls
- KEYSTORE_PATH=/opt/certs/keystore.jks
- KEYSTORE_TYPE=JKS
- KEYSTORE_PASSWORD=cjwC2UU/ZBxhlNyjFbK1QCE/bqNxq9WONwCPq8HbyRk
- TRUSTSTORE_PATH=/opt/certs/truststore.jks
- TRUSTSTORE_TYPE=JKS
- TRUSTSTORE_PASSWORD=1ab4TYRgYw7HMY4Dr+7s+7gVDrPBbZPGp4g/t5VhJG4
- NIFI_SECURITY_USER_AUTHORIZER=single-user-authorizer
- NIFI_SECURITY_USER_LOGIN_IDENTITY_PROVIDER=single-user-provider
ports:
- 8443:8443
- "9001-9100:9001-9100"
volumes:
- ./data/nifi/content:/opt/nifi/nifi-current/content_repository:rw
- ./data/nifi/database:/opt/nifi/nifi-current/database_repository:rw
- ./data/nifi/flowfile:/opt/nifi/nifi-current/flowfile_repository:rw
- ./data/nifi/provenance:/opt/nifi/nifi-current/provenance_repository:rw
- ./data/nifi/state:/opt/nifi/nifi-current/state:rw
- ./data/db/driver:/opt/driver
- ./keystore.jks:/opt/certs/keystore.jks
- ./truststore.jks:/opt/certs/truststore.
Update 30/03/2022:
I've been using the new version for few days now and each time I start Nifi, I have the error message : The supplied username and password are not valid.
I have to run docker-compose and start all over.
Also I want to enable anonymous authentication and it's not working even if I set NIFI_SECURITY_ALLOW_ANONYMOUS_AUTHENTICATION to true .
Try adding NIFI_WEB_PROXY_HOST=localhost:8443 in the environment section in docker-compose. Also, you have to check if you can use port 10000 for Nifi.
I have a web server on which I apply different rewrite rules in order to redirect all http(s)://*.website.com requests to the same URL : https://website.com. Everything is working fine as expected.
Now, I would like to make the renewing of certificates (built by letsencrypt) in an automatic way.
Unfortunately, for the moment, I have to do it manually and the ideal is to do it by a simple script called from a crontab (I would like to renew certificates every 2 months).
Currently, I am using the following script certbot-auto (from https://github.com/certbot/certbot/blob/master/certbot-auto) this way :
./certbot-auto certonly --no-bootstrap --no-self-upgrade --renew-by-default -d website.com -d *.website.com --manual --preferred-challenges dns-01 --server https://acme-v02.api.letsencrypt.org/directory --rsa-key-size 4096
Here the output I get from this command :
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator manual, Installer None
Attempting to parse the version 1.2.0 renewal configuration file found at /etc/letsencrypt/renewal/website.com.conf with version 0.31.0 of Certbot. This might not work.
Renewing an existing certificate
Performing the following challenges:
dns-01 challenge for website.com
dns-01 challenge for website.com
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
NOTE: The IP of this machine will be publicly logged as having requested this
certificate. If you're running certbot in manual mode on a machine that is not
your server, please ensure you're okay with that.
Are you OK with your IP being logged?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: Y
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Please deploy a DNS TXT record under the name
_acme-challenge.website.com with the following value:
j-iC2Fywptdjn_MX4UQCTzDJ5FiuyyZiosVaZYTZxNA
Before continuing, verify the record is deployed.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Press Enter to Continue
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Please deploy a DNS TXT record under the name
_acme-challenge.website.com with the following value:
eO1eVTi4cPTpZYcuAkrqqYpRcPIobayhPawvtLNqBpU
Before continuing, verify the record is deployed.
(This must be set up in addition to the previous challenges; do not remove,
replace, or undo the previous challenge tasks yet. Note that you might be
asked to create multiple distinct TXT records with the same name. This is
permitted by DNS standards.)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Press Enter to Continue
Waiting for verification...
Cleaning up challenges
IMPORTANT NOTES:
- Congratulations! Your certificate and chain have been saved at:
/etc/letsencrypt/live/website.com/fullchain.pem
Your key file has been saved at:
/etc/letsencrypt/live/website.com/privkey.pem
Your cert will expire on 2020-06-29. To obtain a new or tweaked
version of this certificate in the future, simply run certbot-auto
again. To non-interactively renew *all* of your certificates, run
"certbot-auto renew"
- If you like Certbot, please consider supporting our work by:
Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate
Donating to EFF: https://eff.org/donate-le
(23) Failed writing body
As you can see, I have to put manually on my VPS DNS account 2 TXT fields named _acme-challenge.website.com and containing :
j-iC2Fywptdjn_MX4UQCTzDJ5FiuyyZiosVaZYTZxNA
and
eO1eVTi4cPTpZYcuAkrqqYpRcPIobayhPawvtLNqBpU
I wonder if there is a way to automate this registration of TXT fields or if there is another way to launch a simple command or script in crontab (executed every 2 months) to avoid all these manual operations ?
All of this seem to be relatively heavy to do each time I want to renew these certificates (4 files below) :
$ ls /etc/letsencrypt/live/website.com/
privkey.pem# fullchain.pem# chain.pem# cert.pem#
The registration of DNS TXT-Records can be automated if you access to programmatically modify records - so you own the DNS server or your provider has an API for you.
Here are some:
https://cloud.google.com/dns/docs/reference/v1/
https://learn.microsoft.com/en-us/rest/api/dns/
https://developer.rackspace.com/docs/cloud-dns/v1/
https://api.cloudflare.com/