The classic was of dumping a certificate with openssl s_client -showcerts -connect HOST:443 does not work if you want to debug a https_proxy based connection.
Assuming that https_proxy=http://192.168.1.1:3128 how can I dump the certificate returned by the proxy when making a http request to, lets say https://pypi.org
Clearly the returned certificate is supposed to created on the fly by the proxy using its own CA authority.
I want to dump this certificate because while curl accepts it as valid without any problems I found that python-requests chokes.
Related
I am working with an external company. Lets call them evilcorp.com. I want to use openssl to debug a two way SSL handshake.
https://evilcorp.com is setup to not require client authentication.
https://evilcorp.com/webservices is setup to require client authentication.
How can I specify this path in openssl. So basically this works:
openssl s_client -connect evilcorp.com:443
But this does not work and gives me gethostbyname failure
openssl s_client -connect evilcorp.com/webservices:443
How can I get this to work (if possible)
You have a very simple error in the address. Here's the fix:
"openssl s_client -connect evilcorp.com:443/webservice"
You had the 443 at the end - it needs to go directly after to the domain name.
I'm not sure if this can be done at all but if it can be done then you first have to use openssl to connect to the clients host and already specify the client certificates. Then inside the successful connection you need to speak HTTP to access the relevant page.
I.e. you first connect:
$ openssl s_client -connect host:port -cert cert.pem -key key.pem
... CONNECTED
... Verify return code...
---
And then access the URL using the HTTP protocol
GET /protected_page/ HTTP/1.0
Host: example.org
<empty line>
Note that the last line must be an empty line according to the HTTP protocol. It might also that you need to use the -crlf option in openssl to get the line ends correct in case you have a strict web server. If all goes right the server should now issue a renegotiation request to the client, i.e another TLS handshake is done.
I am developing a client/server application with TLS. My idea is to use a certificate on the client so it is authenticated by the server. Also another certificate on the server so the client is also able to authenticate that it is connecting to the right server.
I want first to test and use openssl s_server and openssl s_client to validate the proposal.
Until now I have created a CA private key on the server, I have created a root certificate. With the root certificate I have signed two CSR, so I get one certificate for the server and one certificate for the client.
I also have installed the client certificate + root certificate on the client, and the server certificate + root certificate on the server.
I want now to try to establish a connection between openssl s_server and openssl s_client and verify that they get both authenticated mutually, but I cannot wrap my mind with the documentation on how to do it. Any help or any guide on that?
Once I have that set up, the next step is to test the own developed client against that server, and our own developed server against the s_client. Can we use that for testing?
Thanks
It looks like you are trying to set up a root of trust with (1) s_client and s_server for testing; and (2) programmatically within your code using OpenSSL.
To ensure openssl s_client (or openssl s_server) uses your root, use the following options:
-CAfile option to specify the root
-cert option for the certificate to use
-key option for the private key of the certificate
See the docs on s_client(1) and s_server(1) for details.
To do the same programmatically on the client, you would use:
SSL_CTX_load_verify_locations to load the trusted root
SSL_CTX_use_certificate to specify the client certificate
SSL_CTX_use_PrivateKey to load the private key for the client certificate
To do the same programmatically on the server, you would use:
SSL_CTX_load_verify_locations to load the trusted root
SSL_CTX_use_certificate_chain_file to specify the server certificate
SSL_CTX_use_PrivateKey to load the private key for the server certificate
SSL_CTX_set_client_CA_list to tell the client to send its client certificate
If you don't want to use the parameters for every connection (i.e. the common context), then set it for each SSL connection with, for example, SSL_use_certificate and SSL_use_PrivateKey.
A lot goes on with SSL_CTX_set_client_CA_list. It (1) loads the CA's to the server uses to verify a client, (2) it causes the server to send a list of CAs it accepts when verifing a client, and (3) it triggers the ClientCertificate message at the client if the client has a certificate that satisfies the server's accepted CAs list.
Also see the docs on SSL_CTX_load_verify_locations(3), SSL_CTX_use_certificate(3), SSL_CTX_set_client_CA_list and friends.
The easiest certificate and key format to use is PEM. PEM is the one that uses, for example, ----- BEGIN CERTIFICATE -----. For the server certificate, be sure the file is a concatenation of the server's certificate and any intermediates needed by the client to build the chain.
Having the server send all required certificates is standard practice for a problem known as the "which directory" problem. Its a well known problem in PKI, and its essentially the problem that clients don't know where to go to fetch missing intermediate certificates.
In general, you now know the functions that you need to use. Download a small server like nginx, and see how a production server uses them in practice. You could even use a SQL server like Postgres since it sets up a SSL/TLS server. Simply search the source files for SSL_CTX_load_verify_locations or SSL_load_verify_locations, and you will find the right place.
Though I don't recommend it, you could even look at s_client.c and s_server.c. They are located in <openssl dir>/apps. But the code can be difficult to read at times.
Generate two pairs of certificates/keys, one for the server and one for the client. Also create test.txt with any content.
To set up an SSL server that checks a client certificate, run the following command:
openssl s_server -cert server_cert.pem -key server_key.pem -WWW -port 12345 -CAfile client_cert.pem -verify_return_error -Verify 1
To test the server with client certificate, run the following command:
echo -e 'GET /test.txt HTTP/1.1\r\n\r\n' | openssl s_client -cert client_cert.pem -key client_key.pem -CAfile server_cert.pem -connect localhost:12345 -quiet
Alternatively you can use curl command:
curl -k --cert client_cert.pem --key client_key.pem https://localhost:12345/test.txt
I'm using cUrl to request data from a corporate website site using a .cer certificate that they sent me.
This is the command:
cUrl --header "Content-Type: text/xml;charset=UTF-8" \
--data #bustaRequestISEE2015ConsultazioneAttestazione.xml \
-o bustaResponseISEE2015ConsultazioneAttestazione.xml \
--cert ./caaffabisrl.cer \
https://istitutonazionaleprevidenzasociale.spcoop.gov.it/PD
When I run it, I get this error message:
curl: (58) could not load PEM client certificate, OpenSSL error error:0906D06C:PEM routines:PEM_read_bio:no start line, (no key found, wrong pass phrase, or wro
ng file format?)
Is there anybody who can help me?
Tks, Cristiano.
It is not possible to connect to a TLS server with curl using only a client certificate, without the client private key. Either they forgot to send you the private key file, or, what they sent you was not the client certificate but the server certificate for verification.
The first thing I would try is using --cacert instead of --cert. That is, tell curl that this is the server's certificate that curl can use to verify that the server is who you think it is.
You can also try removing --cert and not using --cacert, and you will probably get an error that the server is not trusted. Then add the --insecure argument and see if that works. I would not keep that argument, as then you have no proof of who you are talking to.
My guess is that it is the server cert, and that using --cacert instead of --cert will solve the problem.
My guess is that your certificate file is a DER encoded binary certificate instead of base-64 encoded certificate. To covert the from binary to base-64, you can use OpenSSL.
openssl x509 -inform der -in certificate.cer -out certificate.pem
I always forget all the arguments and have the following site bookmarked, as it gives examples of how to convert pretty much any certificate format. https://www.sslshopper.com/ssl-converter.html
First, you need to specify whether you're expected to perform two-way TLS/SSL or MTLS (mutual TLS). This would typically be the reason for sending a certificate. If they sent the server certificate, but you can connect to the server with a browser, you can down load the certificate. If their server is configured to send the server certificate and CA chain, then you can get the entire chain in a single request using "openssl s_client -connect [hostname:port] -showcerts". Save the certs in the console to a file, copying the cert blob(s) to individual cert files (cert1.crt, cert2.crt). However, if they are expecting MTLS and attempting to send a client certificate to you, either you've already generated a private key and CSR (certificate signing request) and send them the CSR. They would have then signed a certificate with their CA certificate using the CSR. The cert they returned would then need to be paired with the private key used to generate the CSR. They should not be generating the public/private key pair and sending them over mail. The private key should be stored security on the one system used to establish the connection. If it's one-way (server ssl only), then your client system (assuming it's not the browser), needs a truststore file, with the CA certificate chain installed and set to trusted. If the platform is Java, read Java's keytool documentation. Note, a keystore is for your systems public/private keypair. A truststore is for the CA certificates that you trust to sign public certificates that your system should trust as being authentic. You need to read any of the PKI x509 overviews by DigiCert, SSLABS, Sectigo, etc.
I'm having two coldfusion applications which runs on Apache web server
https://site1.com
https://site2.com
Both having its own SSL certificate and are configured in httpd-ssl.conf file with Name based virtual host for each site.
When I'm doing a HTTPS call from Site1.com to Site2.com,
httpService = new http();
httpService.setMethod("get");
httpService.setUrl("https://site2.com/comp.cfc?method=amethod&ID=12");
result = httpService.send().getPrefix();
it gives the following error
I/O Exception: hostname in certificate didn't match: <site2.com> != <site1.com>
Actually it should use site2's SSL certificate. But not sure why it is using Site1's SSL certificate and giving the error.
This looks like a Server Name Indication (SNI) issue. SNI is a TLS extension allowing to host several HTTPS servers on the same server.
You can confirm this issue using:
echo "" | openssl s_client -connect site2.com:443 | openssl x509 -noout -subject
If you see something like CN=site1.com try this:
echo "" | openssl s_client -connect site2.com:443 -servername site2.com | openssl x509 -noout -subject
if you get CN=site2.com, this is a SNI issue.
You can look at this bug, more specifically this comment:
The SNI support has been added in ColdFusion 11. The change required for supporting this is quite big and therefore it can't be backported to ColdFusion 10.
Other workarounds could be to host your 2 HTTPS sites on 2 separate servers, to set up a unique SSL certificate valid for both names (using X509 SubjectAltName extension) or to disable certificate CN validation (if possible).
You need to import the SSL certificate into ColdFusion/Java keystore. If this doesn't help, add -Djavax.net.debug=all in jvm.config for ColdFusion. This would require a CF service restart. Then try the SSL call.
I am developing a client/server application with TLS. My idea is to use a certificate on the client so it is authenticated by the server. Also another certificate on the server so the client is also able to authenticate that it is connecting to the right server.
I want first to test and use openssl s_server and openssl s_client to validate the proposal.
Until now I have created a CA private key on the server, I have created a root certificate. With the root certificate I have signed two CSR, so I get one certificate for the server and one certificate for the client.
I also have installed the client certificate + root certificate on the client, and the server certificate + root certificate on the server.
I want now to try to establish a connection between openssl s_server and openssl s_client and verify that they get both authenticated mutually, but I cannot wrap my mind with the documentation on how to do it. Any help or any guide on that?
Once I have that set up, the next step is to test the own developed client against that server, and our own developed server against the s_client. Can we use that for testing?
Thanks
It looks like you are trying to set up a root of trust with (1) s_client and s_server for testing; and (2) programmatically within your code using OpenSSL.
To ensure openssl s_client (or openssl s_server) uses your root, use the following options:
-CAfile option to specify the root
-cert option for the certificate to use
-key option for the private key of the certificate
See the docs on s_client(1) and s_server(1) for details.
To do the same programmatically on the client, you would use:
SSL_CTX_load_verify_locations to load the trusted root
SSL_CTX_use_certificate to specify the client certificate
SSL_CTX_use_PrivateKey to load the private key for the client certificate
To do the same programmatically on the server, you would use:
SSL_CTX_load_verify_locations to load the trusted root
SSL_CTX_use_certificate_chain_file to specify the server certificate
SSL_CTX_use_PrivateKey to load the private key for the server certificate
SSL_CTX_set_client_CA_list to tell the client to send its client certificate
If you don't want to use the parameters for every connection (i.e. the common context), then set it for each SSL connection with, for example, SSL_use_certificate and SSL_use_PrivateKey.
A lot goes on with SSL_CTX_set_client_CA_list. It (1) loads the CA's to the server uses to verify a client, (2) it causes the server to send a list of CAs it accepts when verifing a client, and (3) it triggers the ClientCertificate message at the client if the client has a certificate that satisfies the server's accepted CAs list.
Also see the docs on SSL_CTX_load_verify_locations(3), SSL_CTX_use_certificate(3), SSL_CTX_set_client_CA_list and friends.
The easiest certificate and key format to use is PEM. PEM is the one that uses, for example, ----- BEGIN CERTIFICATE -----. For the server certificate, be sure the file is a concatenation of the server's certificate and any intermediates needed by the client to build the chain.
Having the server send all required certificates is standard practice for a problem known as the "which directory" problem. Its a well known problem in PKI, and its essentially the problem that clients don't know where to go to fetch missing intermediate certificates.
In general, you now know the functions that you need to use. Download a small server like nginx, and see how a production server uses them in practice. You could even use a SQL server like Postgres since it sets up a SSL/TLS server. Simply search the source files for SSL_CTX_load_verify_locations or SSL_load_verify_locations, and you will find the right place.
Though I don't recommend it, you could even look at s_client.c and s_server.c. They are located in <openssl dir>/apps. But the code can be difficult to read at times.
Generate two pairs of certificates/keys, one for the server and one for the client. Also create test.txt with any content.
To set up an SSL server that checks a client certificate, run the following command:
openssl s_server -cert server_cert.pem -key server_key.pem -WWW -port 12345 -CAfile client_cert.pem -verify_return_error -Verify 1
To test the server with client certificate, run the following command:
echo -e 'GET /test.txt HTTP/1.1\r\n\r\n' | openssl s_client -cert client_cert.pem -key client_key.pem -CAfile server_cert.pem -connect localhost:12345 -quiet
Alternatively you can use curl command:
curl -k --cert client_cert.pem --key client_key.pem https://localhost:12345/test.txt