Is it possible to create new certificate with certbot if I accidentally lost the one I just created?
I was testing certbot with docker and created certificate on my test server. Then I wanted to make sure that running same command for creating certificate, will exit normally. But instead it would get stuck on (which is problematic when running ansible as it would just stuck without any output at all):
Saving debug log to /var/log/letsencrypt/letsencrypt.log
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/myhost.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Certificate not yet due for renewal
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
The following certificates are not due for renewal yet:
/etc/letsencrypt/live/myhost/fullchain.pem expires on 2023-02-28 (skipped)
No renewals were attempted.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
^CERROR: 130
So I decided to just delete certificate and private key and create new certificate (should have made a backup..). But now I always get this:
docker-compose run --rm -p 80:80 certbot certonly --register-unsafely-without-email --agree-tos --standalone --preferred-challenge http -d myhost
Creating app_certbot_run ... done
Saving debug log to /var/log/letsencrypt/letsencrypt.log
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
No renewals were attempted.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
^CERROR: 130
So as I understand letsencrypt knows there is valid certificate registered by them and refuses to create new one. Is there a way to create new one instead of waiting till it expires?..:)
My mistake. I forgot that I added this in docker compose for certbot as entrypoint..
"/bin/sh -c 'trap exit TERM; while :; do certbot renew; sleep 12h & wait $${!}; done;'"
So it would try to renew and be stuck.
Related
While trying to run part of the TICK stack (influx and chronograf) through a nginx proxy with letsencrypt certificates i am getting an error that the certs files are not found.
Here is my docker-compose file:
version: '3'
services:
influxdb:
container_name: influxdb
image: quay.io/influxdb/influxdb:v2.0.2
restart: always
ports:
- '8086:8086'
volumes:
- './data:/root/.influxdbv2'
- /root/new/letsencrypt/certs:/etc/letsencrypt/certs:ro
environment:
- INFLUXD_TLS_CERT=/etc/letsencrypt/certs/influx.lizardsolutions.com.crt
- INFLUXD_TLS_KEY=/etc/letsencrypt/certs/influx.lizardsolutions.com.key
chronograf:
image: chronograf:1.7.14
container_name: chronograf
restart: always
hostname: chronograf
network_mode: bridge
ports:
- 3000:8888/tcp
volumes:
- /var/lib/chronograf:/var/lib/chronograf
- /root/new/letsencrypt/certs:/etc/letsencrypt/certs:ro
environment:
- VIRTUAL_HOST=influx.lizardsolutions.com# adjust to match your domain name
- VIRTUAL_PROTO=https
- VIRTUAL_PORT=3000
- LETSENCRYPT_HOST=influx.lizardsolutions.com # adjust to match your domain name -
- LETSENCRYPT_EMAIL=admin#lizardsolutions.com # adjust to match your email
- BOLT_PATH=/var/lib/chronograf/chronograf-v1.db
- CANNED_PATH=/usr/share/chronograf/canned
- RESOURCES_PATH=/usr/share/chronograf/resources
- PROTOBOARDS_PATH=/usr/share/chronograf/protoboards
# - INFLUXDB_URL=http://127.0.0.1:8086
#INFLUX_CREDENTIALS
# - INFLUXDB_USERNAME=grafanam9k5bXbBj7GPGdf3
# - INFLUXDB_PASSWORD=k9H4TV8ifRo7E6mNyv3L
- TLS_CERTIFICATE=/etc/letsencrypt/certs/influx.domain.com.crt
- TLS_PRIVATE_KEY=/etc/letsencrypt/certs/influx.domain.com.key
- REPORTING_DISABLED='false'
# Set the logging level. Valid values: debug | info | error
- LOG_LEVEL=info
- TOKEN_SECRET=hxDEXBPAc49muW3gLoot7YamJ
- AUTH_DURATION=0
- PUBLIC_URL=https://ec-influx.domain.com
nginx-proxy:
image: jwilder/nginx-proxy
container_name: nginx
restart: always
labels:
- com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy
ports:
- "80:80"
- "443:443"
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
- ./nginx/html:/usr/share/nginx/html
- ./nginx/vhost.d:/etc/nginx/vhost.d
- /root/new/letsencrypt/certs:/etc/nginx/certs:ro
letsencrypt:
image: jrcs/letsencrypt-nginx-proxy-companion
container_name: letsencrypt
restart: always
environment:
- NGINX_PROXY_CONTAINER=nginx-proxy
- REUSE_PRIVATE_KEYS=true
volumes_from:
- nginx-proxy
volumes:
- /root/new/letsencrypt/certs:/etc/nginx/certs:rw
- /var/run/docker.sock:/var/run/docker.sock:ro
Here is the error from chronograf:
chronograf | time="2022-08-29T00:22:02Z" level=error msg="open /etc/letsencrypt/certs/influx.domain.com.crt: no such file or directory" component=server
chronograf | 2022/08/29 00:22:02 open /etc/letsencrypt/certs/influx.domain.com.crt: no such file or directory
The same error shows for influxdb as well
From where can i start debuging ? While navigating to the mounted volume (./letsencrypt/certs/) i can confirm the key and cert file is there.
I have a dev Environment in docker and I'm using Nifi 1.13.2 and it's working fine (http).
I did an upgrade of Nifi to use version 1.15.3 that starts up with HTTPS only and it isn't working anymore.
docker-compose :
services:
nifi:
image: apache/nifi:1.15.3
container_name: nifi
environment:
- JAVA_TOOL_OPTIONS=-Dfile.encoding=utf8
- NIFI_WEB_HTTPS_PORT=8443
ports:
- 10000:8443
- "9001-9100:9001-9100"
volumes:
- ./data/nifi/content:/opt/nifi/nifi-current/content_repository:rw
- ./data/nifi/database:/opt/nifi/nifi-current/database_repository:rw
- ./data/nifi/flowfile:/opt/nifi/nifi-current/flowfile_repository:rw
- ./data/nifi/provenance:/opt/nifi/nifi-current/provenance_repository:rw
- ./data/nifi/state:/opt/nifi/nifi-current/state:rw
- ./data/db/driver:/opt/driver
Logs from nifi-app.log :
nifi-app_2022-03-16_14.0.log:36362:2022-03-16 14:36:43,370 INFO [main] org.apache.nifi.web.server.JettyServer Created HostHeaderHandler [HostHeaderHandler for 2dec0477131a:8443}]
nifi-app_2022-03-16_14.0.log:139435:2022-03-16 14:37:07,253 INFO [main] o.eclipse.jetty.server.AbstractConnector Started ServerConnector#36b53f08{SSL, (ssl, http/1.1)}{2dec0477131a:8443}
nifi-app_2022-03-16_14.0.log:139933:2022-03-16 14:37:07,292 INFO [main] org.apache.nifi.web.server.JettyServer https://2dec0477131a:8443/nifi
Error in logs when I go to https://localhost:10000/nifi :
2022-03-16 14:39:15,572 WARN [NiFi Web Server-16] o.a.nifi.web.server.HostHeaderHandler Request host header [localhost:10000] different from web hostname [2dec0477131a(:8443)]. Overriding to [2dec0477131a:8443/nifi]
2022-03-16 14:39:15,655 WARN [NiFi Web Server-20] o.a.nifi.web.server.HostHeaderHandler Request host header [localhost:10000] different from web hostname [2dec0477131a(:8443)]. Overriding to [2dec0477131a:8443/favicon.ico]
[![enter image description here][1]][1]
I see that on docker desktop when I do open in browser it opens http://localhost:10000/nifi , how can I change this url ?
Update1 :
I found this article : https://www.javahelps.com/2022/01/run-apache-nifi-in-docker-with-ssl.html so my docker-compose looks like this :
services:
nifi:
image: apache/nifi:1.15.3
container_name: nifi
environment:
- JAVA_TOOL_OPTIONS=-Dfile.encoding=utf8
- NIFI_WEB_HTTPS_PORT=8443
- SINGLE_USER_CREDENTIALS_USERNAME=jhiihihhu
- SINGLE_USER_CREDENTIALS_PASSWORD=e9o6dQhlEDusiq6oKUpzp8ZtW/Bjim6q2U0SxyBO6JA
- NIFI_SENSITIVE_PROPS_KEY=rHkWR1gDNW3R
- NIFI_WEB_PROXY_HOST=0.0.0.0:8443
- NIFI_WEB_HTTPS_HOST=0.0.0.0
- NIFI_CLUSTER_ADDRESS=0.0.0.0
- NIFI_REMOTE_INPUT_HOST=0.0.0.0
- AUTH=tls
- KEYSTORE_PATH=/opt/certs/keystore.jks
- KEYSTORE_TYPE=JKS
- KEYSTORE_PASSWORD=cjwC2UU/ZBxhlNyjFbK1QCE/bqNxq9WONwCPq8HbyRk
- TRUSTSTORE_PATH=/opt/certs/truststore.jks
- TRUSTSTORE_TYPE=JKS
- TRUSTSTORE_PASSWORD=1ab4TYRgYw7HMY4Dr+7s+7gVDrPBbZPGp4g/t5VhJG4
- NIFI_SECURITY_USER_AUTHORIZER=single-user-authorizer
- NIFI_SECURITY_USER_LOGIN_IDENTITY_PROVIDER=single-user-provider
ports:
- 10000:8443
- "9001-9100:9001-9100"
volumes:
- ./data/nifi/content:/opt/nifi/nifi-current/content_repository:rw
- ./data/nifi/database:/opt/nifi/nifi-current/database_repository:rw
- ./data/nifi/flowfile:/opt/nifi/nifi-current/flowfile_repository:rw
- ./data/nifi/provenance:/opt/nifi/nifi-current/provenance_repository:rw
- ./data/nifi/state:/opt/nifi/nifi-current/state:rw
- ./data/db/driver:/opt/driver
- ./keystore.jks:/opt/certs/keystore.jks
- ./truststore.jks:/opt/certs/truststore.jks
I still have the same error .
[1]: https://i.stack.imgur.com/OGqE6.png
Update 2:
It worked with the following docker compose :
services:
nifi:
image: apache/nifi:1.15.3
container_name: nifi
environment:
- JAVA_TOOL_OPTIONS=-Dfile.encoding=utf8
- NIFI_WEB_HTTPS_PORT=8443
- SINGLE_USER_CREDENTIALS_USERNAME=khiuiuiuh
- SINGLE_USER_CREDENTIALS_PASSWORD=e9o6dQhlEDusiq6oKUpzp8ZtW/Bjim6q2U0SxyBO6JA
- NIFI_SENSITIVE_PROPS_KEY=rHkWR1gDNW3R
- NIFI_WEB_PROXY_HOST='localhost:8443,localhost:10000'
- NIFI_WEB_HTTPS_HOST=0.0.0.0
- NIFI_CLUSTER_ADDRESS=0.0.0.0
- NIFI_REMOTE_INPUT_HOST=0.0.0.0
- AUTH=tls
- KEYSTORE_PATH=/opt/certs/keystore.jks
- KEYSTORE_TYPE=JKS
- KEYSTORE_PASSWORD=cjwC2UU/ZBxhlNyjFbK1QCE/bqNxq9WONwCPq8HbyRk
- TRUSTSTORE_PATH=/opt/certs/truststore.jks
- TRUSTSTORE_TYPE=JKS
- TRUSTSTORE_PASSWORD=1ab4TYRgYw7HMY4Dr+7s+7gVDrPBbZPGp4g/t5VhJG4
- NIFI_SECURITY_USER_AUTHORIZER=single-user-authorizer
- NIFI_SECURITY_USER_LOGIN_IDENTITY_PROVIDER=single-user-provider
ports:
- 8443:8443
- "9001-9100:9001-9100"
volumes:
- ./data/nifi/content:/opt/nifi/nifi-current/content_repository:rw
- ./data/nifi/database:/opt/nifi/nifi-current/database_repository:rw
- ./data/nifi/flowfile:/opt/nifi/nifi-current/flowfile_repository:rw
- ./data/nifi/provenance:/opt/nifi/nifi-current/provenance_repository:rw
- ./data/nifi/state:/opt/nifi/nifi-current/state:rw
- ./data/db/driver:/opt/driver
- ./keystore.jks:/opt/certs/keystore.jks
- ./truststore.jks:/opt/certs/truststore.
Update 30/03/2022:
I've been using the new version for few days now and each time I start Nifi, I have the error message : The supplied username and password are not valid.
I have to run docker-compose and start all over.
Also I want to enable anonymous authentication and it's not working even if I set NIFI_SECURITY_ALLOW_ANONYMOUS_AUTHENTICATION to true .
Try adding NIFI_WEB_PROXY_HOST=localhost:8443 in the environment section in docker-compose. Also, you have to check if you can use port 10000 for Nifi.
I have a web server on which I apply different rewrite rules in order to redirect all http(s)://*.website.com requests to the same URL : https://website.com. Everything is working fine as expected.
Now, I would like to make the renewing of certificates (built by letsencrypt) in an automatic way.
Unfortunately, for the moment, I have to do it manually and the ideal is to do it by a simple script called from a crontab (I would like to renew certificates every 2 months).
Currently, I am using the following script certbot-auto (from https://github.com/certbot/certbot/blob/master/certbot-auto) this way :
./certbot-auto certonly --no-bootstrap --no-self-upgrade --renew-by-default -d website.com -d *.website.com --manual --preferred-challenges dns-01 --server https://acme-v02.api.letsencrypt.org/directory --rsa-key-size 4096
Here the output I get from this command :
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator manual, Installer None
Attempting to parse the version 1.2.0 renewal configuration file found at /etc/letsencrypt/renewal/website.com.conf with version 0.31.0 of Certbot. This might not work.
Renewing an existing certificate
Performing the following challenges:
dns-01 challenge for website.com
dns-01 challenge for website.com
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
NOTE: The IP of this machine will be publicly logged as having requested this
certificate. If you're running certbot in manual mode on a machine that is not
your server, please ensure you're okay with that.
Are you OK with your IP being logged?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: Y
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Please deploy a DNS TXT record under the name
_acme-challenge.website.com with the following value:
j-iC2Fywptdjn_MX4UQCTzDJ5FiuyyZiosVaZYTZxNA
Before continuing, verify the record is deployed.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Press Enter to Continue
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Please deploy a DNS TXT record under the name
_acme-challenge.website.com with the following value:
eO1eVTi4cPTpZYcuAkrqqYpRcPIobayhPawvtLNqBpU
Before continuing, verify the record is deployed.
(This must be set up in addition to the previous challenges; do not remove,
replace, or undo the previous challenge tasks yet. Note that you might be
asked to create multiple distinct TXT records with the same name. This is
permitted by DNS standards.)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Press Enter to Continue
Waiting for verification...
Cleaning up challenges
IMPORTANT NOTES:
- Congratulations! Your certificate and chain have been saved at:
/etc/letsencrypt/live/website.com/fullchain.pem
Your key file has been saved at:
/etc/letsencrypt/live/website.com/privkey.pem
Your cert will expire on 2020-06-29. To obtain a new or tweaked
version of this certificate in the future, simply run certbot-auto
again. To non-interactively renew *all* of your certificates, run
"certbot-auto renew"
- If you like Certbot, please consider supporting our work by:
Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate
Donating to EFF: https://eff.org/donate-le
(23) Failed writing body
As you can see, I have to put manually on my VPS DNS account 2 TXT fields named _acme-challenge.website.com and containing :
j-iC2Fywptdjn_MX4UQCTzDJ5FiuyyZiosVaZYTZxNA
and
eO1eVTi4cPTpZYcuAkrqqYpRcPIobayhPawvtLNqBpU
I wonder if there is a way to automate this registration of TXT fields or if there is another way to launch a simple command or script in crontab (executed every 2 months) to avoid all these manual operations ?
All of this seem to be relatively heavy to do each time I want to renew these certificates (4 files below) :
$ ls /etc/letsencrypt/live/website.com/
privkey.pem# fullchain.pem# chain.pem# cert.pem#
The registration of DNS TXT-Records can be automated if you access to programmatically modify records - so you own the DNS server or your provider has an API for you.
Here are some:
https://cloud.google.com/dns/docs/reference/v1/
https://learn.microsoft.com/en-us/rest/api/dns/
https://developer.rackspace.com/docs/cloud-dns/v1/
https://api.cloudflare.com/
I have a domain of the form api.mydomain.com with a letsencrypt certificate that I'm trying to renew.
root#prod-app-1:/home/ninesalt# certbot certificates
Saving debug log to /var/log/letsencrypt/letsencrypt.log
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Found the following certs:
Certificate Name: api.mydomain.com
Domains: api.mydomain.com
Expiry Date: 2019-06-17 11:25:52+00:00 (VALID: 27 days)
Certificate Path: /etc/letsencrypt/live/api.mydomain.com/fullchain.pem
Private Key Path: /etc/letsencrypt/live/api.mydomain.com/privkey.pem
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
However when I try to renew it with certbot renew I get this error:
Attempting to renew cert (api.mydomain.com) from /etc/letsencrypt/renewal/api.mydomain.com.conf produced an unexpected error: The manual plugin is not working; there may be problems with your existing configuration.
The error was: PluginError('An authentication script must be provided with --manual-auth-hook when using the manual plugin non-interactively.',). Skipping.
All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/api.mydomain.com/fullchain.pem (failure)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/api.mydomain.com/fullchain.pem (failure)
Certificates that are created using --manual (and without an authentication hook) cannot be automatically renewed.
This is because it involves you performing the the authorization steps by hand, which is not something that Certbot can automatically repeat at renewal time.
Usually you will want to use an authenticator other than the manual one (such as --apache, --nginx, --webroot, --standalone), so that Certbot can perform automatic renewal.
So, you should probably manually create the certificate again:
certbot certonly --manual -d xxx.com
I have been using Traefik as a reverse proxy for several containers managed through docker compose. It has been working well but I'm having some difficulty getting graylog running properly behind it.
This config (excerpt of the complete file, removed irrelevant services) works for me
version: '3'
services:
traefik:
image: traefik
command: --docker
networks:
- web
ports:
- "80:80"
- "8080:8080"
- "443:443"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
graylog:
image: graylog/graylog:3.0
environment:
- GRAYLOG_PASSWORD_SECRET=somepasswordpepper
- GRAYLOG_ROOT_PASSWORD_SHA2=somestuffhere
- GRAYLOG_HTTP_BIND_ADDRESS=0.0.0.0:9000
- GRAYLOG_HTTP_EXTERNAL_URI=http://graylog.localhost/
links:
- mongodb:mongo
- elasticsearch
depends_on:
- mongodb
- elasticsearch
networks:
- web
- default
ports:
- 9000:9000
- 1514:1514
- 1514:1514/udp
- 12201:12201
- 12201:12201/udp
labels:
- "traefik.docker.network=web"
- "traefik.web.frontend.rule=Host:graylog.localhost"
- "traefik.protocol=http"
- "traefik.port=9000"
- "traefik.enable=true"
I can then access graylog in my browser at http://graylog.localhost/
The problem occurs because I can't use domains like this in my staging env. What I have had to use is paths rather than domains e.g. the swagger service in my staging env is defined as
swagger:
image: swaggerapi/swagger-ui
environment:
SWAGGER_JSON: /swagger/staging-openapi.yaml
BASE_URL: /swagger
volumes:
- /home/ubuntu/src/carelink_swagger_service:/swagger
networks:
- web
- default
labels:
- "traefik.docker.network=web"
- "traefik.frontend.rule=Host:my.staging.domain.org; PathPrefix: /swagger"
- "traefik.protocol=http"
- "traefik.port=8080"
- "traefik.enable=true"
and I access services like this https://my.staging.domain.org/swagger/
Trying to adopt this approach for graylog
graylog:
image: graylog/graylog:3.0
environment:
- GRAYLOG_PASSWORD_SECRET=somepasswordpepper
- GRAYLOG_ROOT_PASSWORD_SHA2=somestuffhere
- GRAYLOG_HTTP_BIND_ADDRESS=0.0.0.0:9000
- GRAYLOG_HTTP_EXTERNAL_URI=http://localhost/graylog/
links:
- mongodb:mongo
- elasticsearch
depends_on:
- mongodb
- elasticsearch
networks:
- web
- default
ports:
- 9000:9000
- 1514:1514
- 1514:1514/udp
- 12201:12201
- 12201:12201/udp
labels:
- "traefik.docker.network=web"
- "traefik.web.frontend.rule=Host:localhost; PathPrefix: /graylog"
- "traefik.protocol=http"
- "traefik.port=9000"
- "traefik.enable=true"
and I get the following errors
I have gone through this Q/A, similar entries on the graylog forums etc. but I feel like I'm chasing my tail.
It's for Traefik 2.x but I think that you'll manage to convert your rules:
labels:
- "traefik.http.routers.graylog.rule=PathPrefix(`/graylog`)"
- "traefik.http.routers.graylog.middlewares=graylog3#docker,graylog2#docker,graylog#docker"
- "traefik.http.middlewares.graylog3.stripprefix.prefixes=/graylog"
- "traefik.http.middlewares.graylog2.redirectregex.regex=^(.*)/graylog$$"
- "traefik.http.middlewares.graylog2.redirectregex.replacement=$$1/graylog/"
- "traefik.http.middlewares.graylog.replacepathregex.regex=^/graylog/(.*)"
- "traefik.http.middlewares.graylog.replacepathregex.replacement=^/$$1"
- "traefik.http.routers.graylog.service=graylog"
- "traefik.http.services.graylog.loadbalancer.server.port=9000"
- "traefik.docker.network=traefik"