So I'm a Puppet noob, and I foolishly did a cert clean --all. I'm now in the process of re-doing all of the certificates but I have a problem.
So what I've tried to do to re-create the master certificate is stop the puppet service, back-up and deleted the ssh directory on the master (/var/lib/puppet/ssh for me), then I ran the following command to recreate the master certificates: -
puppet master --verbose --no-daemonise
Then I start the puppet service again.
However, then when I try and sign a client (after cleaning out the SSL directory on the client side), when I run puppet agent --test I get the following error:-
Error: Could not request certificate: Connection reset by peer - SSL_connect
Or sometimes: -
Error: Could not request certificate: Connection refused - connect(2)
Does anybody know what's causing these errors? Any help would be greatly appreciated. It's puppet version 3.6 by the way.
Related
For several months now I've had issues with gitlab-runner which is randomly failing with the following log:
Running with gitlab-runner 13.7.0 (943fc252)
on <gitlab-runner-name> <gitlab-runner-id>
Preparing the "shell" executor
00:00
Using Shell executor...
Preparing environment
00:00
Running on <hostname>...
Getting source from Git repository
00:00
Fetching changes...
Reinitialized existing Git repository in /var/gitlab-runner/builds/<gitlab-runner-id>/0/<gtlab-group>/<gitlab-project>/.git/
fatal: unable to access 'https://gitlab-ci-token:[MASKED]#<hostname>/<gtlab-group>/<gitlab-project>.git/': Problem with the SSL CA cert (path? access rights?)
ERROR: Job failed: exit status 1
This line is the crucial one:
fatal: unable to access 'https://gitlab-ci-token:[MASKED]#<hostname>/<gtlab-group>/<gitlab-project>.git/': Problem with the SSL CA cert (path? access rights?)
I tried unregistering the runner and registering a new one. It also failed with the same error after a while (the first run usually worked well).
Furthermore, runners on other machines are working correctly and never fail with the error message above.
I believe the issue is caused by the missing CI_SERVER_TLS_CA_FILE file in:
/var/gitlab-runner/builds/<gitlab-runner-id>/0/<gtlab-group>/<gitlab-project>.tmp/CI_SERVER_TLS_CA_FILE
I tried doing a git pull in the faulty directory and I got the same message. After I copied this missing file from another directory which had it, I got the following:
remote: HTTP Basic: Access denied
fatal: Authentication failed for 'https://gitlab-ci-token:<gitlab-runner-token>#gitlab.lab.sk.alcatel-lucent.com/<gtlab-group>/<gitlab-project>.git/'
As far as I know, these tokens are generated for a one-time use and are discarded after the job finishes. This leads me to believe the missing file is the issue.
Where is this file copied from? Why is it missing? What can I do to fix this issue?
I've been looking through the GitLab issues without luck.
It sounds like one or more of your runners doesn't trust the certificate on your gitlab host. You'll have to track down the root and intermediate certs used to sign your TLS cert, and add it to your runners' hosts.
For my runners on CentOS, I follow this guide (for CentOS, the commands are the same for higher versions): https://manuals.gfi.com/en/kerio/connect/content/server-configuration/ssl-certificates/adding-trusted-root-certificates-to-the-server-1605.html.
I am installing Chef Server version 12.8.0-1 on Debian 8.5.
By downloading the .deb package files direct from the chef.io website I have successfully got the chef-server and chef-manage modules installed, configured and running.
I have got stuck trying to install the push jobs server. I used the command below...
chef-server-ctl install opscode-push-jobs-server
when the command runs I get the following errors...
Chef Client failed. 0 resources updated in 06 seconds
[2016-07-12T12:02:23+01:00] FATAL: Stacktrace dumped to /var/opt/opscode/local-mode-cache/chef-stacktrace.out
[2016-07-12T12:02:23+01:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2016-07-12T12:02:24+01:00] FATAL: OpenSSL::SSL::SSLError: SSL_connect returned=1 errno=0 state=error: certificate verify failed
I believe the cause of the problem is a self signed certificate used on our corporate firewall to allow the security team to decode SSL traffic.
What I need to know is how to either get Chef to accept this certificate or get it to ignore self signed certs.
I know I could manually download and install the module but this issue will affect other things like installing cookbooks from the Chef supermarket so I'd rather find a solution that lets me use the Chef tools as intended.
Can anyone advise please?
Tensibai gave you the path for fixing Chef Server, you'll probably need to do it for the client too which is fortunately easier. Just drop the extra root cert in /etc/chef/trusted_certs.
I am setting up Chef workstation by configuring knife.rb using "knife configure -i" configure command. After PROPERLY answering all question, I get the following error :
ERROR: SSL Validation failure connecting to host: 172.xx.x.xx - SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed
ERROR: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed
My goal is to disable this SSL certificate verification forever and use knife utility to bootstrap my all nodes.
I had the same issue running chef-client after upgrading to the version 12.xx. Steps to solve:
Pull crt from server. Run on node:
knife ssl fetch -s https://yourchefserver01.com:443
Note: If fetch doesnt work copy from yourchefserver01.com:/var/opt/chef-server/nginx/ca/yourchefserver01.com.crt to client:/root/.chef/trusted_certs/yourchefserver01.com.crt
Verify it pulled:
knife ssl check -s https://yourchefserver01.com:443
export SSL_CERT_FILE="/root/.chef/trusted_certs/yourchefserver01.com.crt"
Run chef-client
Your problem is the validation of the chef server certificate.
Install a proper certificate on the chef server
or add your chef server certificate (located in /etc/chef-server/hostname.crt) to your workstation cacert.pem (located by default in <install path>/opscode/chef/embedded/ssl/certs).
With chef 12 you'll have to ditribute it too on your nodes to validate the chef API server or you'll have a warning at the start of each chef-client run about it.
Issue seems to be concerned with the .pem validator. your validation are misconfigured. Try create new validation key from chef server and place it under the node.
If you are running Chef Server on-premise, it will easier in the long run to install a third-party SSL cert, e.g. Verisign, on the Chef Server (or load balancer). chef-client and knife come with OpenSSL which will trust a valid third-party cert automatically with no configuation required on each node.
Please don't turn off SSL cert validation. SSL validation is additional protection that the server you are trusting with root access to your Chef nodes is the real Chef server, not a man-in-the-middle attack.
I am trying to copy a current Puppet Master server on one domain and move it to another. Im finding that its very hard to try to change all the config remanence. Is there an easy way to do this, or a step by step best practice? I have grepped most of the old fqdn name and changed it to the new one, yet when I delete all certs, and re-issue new ones on the master, it wants to keep pulling a cert for the old FQDN.
Edit 1: I have resolved many of the issues I was previously getting. However I can not get past this SSL issue for the life of me.
[root#puppet lib]# puppet resource service apache2 ensure=running
Error: Could not run: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [unable to get local issuer certificate for /CN=puppet.foundry.test]
I have attempted to completely purge all certs from the master, using this link, and then regenerate all. But I still keep getting the same errors:
Error: Could not run: SSL_connect SYSCALL returned=5 errno=0 state=SSLv3 read finished A
Now Im not sure if I am having puppet SSL issues, or SSL issues in general.
Most likely you're connecting to a wrong server (default is hostname puppet).
Check your agent's config, you're mostly interested in server variable
puppet config print --section agent | grep "server = "
Also it's good to know where is puppet agent looking for its config:
$ puppet config print --section agent | grep "^config = "
config = /etc/puppetlabs/puppet/puppet.conf
Edit your config, set correct puppet master:
[agent]
server=puppet4.example.com
Just for sure, you can clean your cerfificate (on agent):
find /etc/puppetlabs/puppet/ssl -name $(hostname -f).pem -delete
on puppet server:
puppet cert clean {broken hostname}
And finally run puppet agent -t
You can use this link: http://bitcube.co.uk/content/puppet-errors-explained
Did you try to change the puppet master dns?
Try looking if the puppet master cert is the same as what you are writing in server on the node.
If not you can always use dns_alt_names = puppet_hostname.your_domain and all the names you want for the puppet master & CA.
Then try to restart the puppet master service, clean the slave certname from the master, remove all /var/lib/puppet/ssl/ folder from the slave, and run puppet again.
What puppet isn't telling you is that there is a cert mismatch. The master disconnects as soon as it determines that the cert is invalid or a mismatch. Because the disconnect is so sudden, puppet isn't told why it happens.
When this happens puppet could, for example, change that error message to be, "Hey! Here's a list of things you might check." and then suggest things like verify the cert expiration date, cert mismatch, etc. However why would anyone do that?
Here's one way you can get into this situation: Set up two puppet client machines with the same name by mistake. The second machine to use that name will work, but the first machine will no longer work.
How might someone get into that situation? Two machines can't have the same name! Of course not. But we have seen situations like this:
Machine A, B, C, D, E are all Puppet clients.
Machine C gets wiped and reloaded. The technician accidentally calls it "B". To get it working with Puppet, they "puppet cert clean B".
The technician realizes their mistake and reconfigures machine C with the proper name, performs "puppet cert clean C", and machine C now works fine.
A week later someone notices that machine B hasn't been able to talk to the master. It gets this error message. After hours of debugging they see that the client cert has one serial number but the master expects that client to have a very different serial number. Machine B's cert is cleaned, regenerated, etc. and everything continues.
Should Puppet Labs update the error message to hint that this may be the problem? They could, but then I wouldn't get rep points for writing this awesome answer. Besides, technicians should never make such a mistake, so why handle a case that obviously should never happen... except when it does.
Make sure that you are running puppet as root, or with sudo. I have received this exact error when I was my normal user and ran "puppet agent -t" without elevating my privileges.
I've tried to build a master/agent system with puppet.
My master host name is snspay.cn, I followed the document, and everything was right until I tried to get the catalog from the master. My command is
puppet agent --server snspay.cn --no-daemonize --test onetime --verbose
and the output from the agent
Error: Could not request certificate: SSL_connect returned=1
errno=0 state=SSLv3 read server certificate B: certificate verify
failed: [self signed certificate in certificate chain for /CN=Puppet
CA: snspay.cn]
and the master's log is like
[2014-08-11 14:39:14] ERROR OpenSSL::SSL::SSLError: SSL_accept
returned=1 errno=0 state=SSLv3 read client certificate A: tlsv1
alert unknown ca
I think it is wrong with the ssl instead of puppet it self, but I'm not very familiar with ssl, any ideas?
well I have added another agent node(ubuntu) with a total different environment and everything is so well, so the problems is with the original agent node, I am now running yum update in that node and try later
Your agent has not established trust with the master.
What basically needs to happen is for the agent to import the master's CA certificate to the agent. However, since the agent's cert is obviously signed by an obsolete CA, you will have to replace all SSL data.
On the agent, find the $ssldir (usually /var/lib/puppet/ssl) using
puppet agent --configprint ssldir
and rename or remove it.
Upon the next puppet agent --test run, the agent should request a new certificate, and cache th correct CA.