What can I do to troubleshoot/fix why my curl command cannot find my certificate when referencing to it by its nickname
I am trying to get a secure connection to a bank server, that is handling our online payments, using curl. The server is using NSS protocol. I installed the certificate from the bank into the nssdb with a nickname "foo" using:
certutil -A -d sql:/etc/pki/nssdb -i ccapi.pem -n "foo" -t "P,P,P"
Listing the cert to check that it has been created, I use
certutil -L -d sql:/etc/pki/nssdb/
with the output confirming a nickname "foo" for the cert.:
Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI
foo P,P,P
When I am trying to connect using curl referencing to the nickname foo for the certificate like this:
curl -k -v --cert "foo" https://ccapi.client.qvalent.com/payway/ccapi
This is PART of the output:
Connected to proxy.ourdomain.com.au (172.21.0.58) port 3128 (#0)
Establish HTTP proxy tunnel to ccapi.client.qvalent.com:443
CONNECT ccapi.client.qvalent.com:443 HTTP/1.1
Host: ccapi.client.qvalent.com:443
User-Agent: curl/7.37.0
Proxy-Connection: Keep-Alive
HTTP/1.1 200 Connection established
Proxy replied OK to CONNECT request
Initializing NSS with certpath: sql:/etc/pki/nssdb
skipping SSL peer certificate verification
NSS: client certificate not found: foo
The last line: certificate not found: foo, is my problem. I seem to be doing everything right when creating the cert in the nssdb with a nickname but when using a curl command to connect it cannot find the certificate. If I link to the actual cert like this and not the nickname:
curl -k -v --cert "./ccapi.pem" https://ccapi.client.qvalent.com/payway/ccapi
Then the certificate is recognized but I cannot connect to bank server. This could be because I need to use NSS but this is irrelevant for what I am trying to get an answer to. I have found similar questions on stackoverflow eg. CURL NSS client certificate not found myCert but they fall short of answering the question. I do not have/use PKCS #12 certificate.
What can I do to troubleshoot/fix why my curl command cannot find my certificate when referencing to it by its nickname.
Appreciate any help
-M
Related
I have an issue when using certificate when using curl. I'm running centOS7. We managed to get the curl going in other places, but not our dev machine:
What we are trying to do:
sudo curl -X 'GET' 'https://webpage/document' --cert '/localization.crt.pem' --key '/localization.key.pem' -H 'accept: */*' -k
Im getting this error:
curl: (58) SSL peer cannot verify your certificate.
What I tried to do?(from centOS documentation)
https://access.redhat.com/documentation/en-us/red_hat_certificate_system/9/html/administration_guide_common_criteria_edition/importing_certificate_into_nssdb
# PKICertImport -d . -n "client name" -t ",," -a -i certificate.crt.pem -u C
after echo $? we get a 0, so i think it is installed properly?
Any idea on whats wrong would be great.
I have run into this recently on our linux environments. I've found that this tends to happen if you have an SSL Certificate issued that also includes a chain certificate. If that chain is not also configured on your server OpenSSL considers the certificate invalid.
I would test this using this command:
openssl s_client -showcerts -verify 5 -connect website.com:443
If you see a block like this that means you are missing the certificate chain in your server configuration:
---
SSL handshake has read 2162 bytes and written 401 bytes
Verification error: unable to verify the first certificate
---
Windows fills in the gaps and doesn't mind this type of configuration, but openssl is very particular.
I managed to solve the issue. Recompiled curl with openSSL with following tutorial:
Install curl with openssl
Works like a charm :)
I need to do curl uploading behind company proxy. and I've getting the following two type of problems depending on the site that I try,
curl: (35) error:1408F10B:SSL routines:ssl3_get_record:wrong version number
curl: (60) SSL certificate problem: unable to get local issuer certificate
Here are the details:
Case 1:
. . .
< HTTP/1.1 200 Connection established
< Proxy-agent: CCProxy
<
* Proxy replied 200 to CONNECT request
* CONNECT phase completed!
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* CONNECT phase completed!
* CONNECT phase completed!
* error:1408F10B:SSL routines:ssl3_get_record:wrong version number
* Closing connection 0
curl: (35) error:1408F10B:SSL routines:ssl3_get_record:wrong version number
Case 2:
$ curl -vX POST -d "userId=5&title=Hello World&body=Post body." https://jsonplaceholder.typicode.com/posts
Note: Unnecessary use of -X or --request, POST is already inferred.
* Uses proxy env variable https_proxy == 'http://10.xx.xx.xx:808/'
* Trying 10.xx.xx.xx:808...
* TCP_NODELAY set
* Connected to 10.xx.xx.xx port 808 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to jsonplaceholder.typicode.com:443
> CONNECT jsonplaceholder.typicode.com:443 HTTP/1.1
> Host: jsonplaceholder.typicode.com:443
> User-Agent: curl/7.68.0
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 Connection established
< Proxy-agent: CCProxy
<
* Proxy replied 200 to CONNECT request
* CONNECT phase completed!
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* CONNECT phase completed!
* CONNECT phase completed!
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS alert, unknown CA (560):
* SSL certificate problem: unable to get local issuer certificate
* Closing connection 0
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
The problem is not the above CCProxy, but our company is using the Zscaler transparent proxy which is intercepting SSL requests with its own certificate.
Is there any way to fix it pls?
$ curl --version
curl 7.68.0 (x86_64-pc-linux-gnu) libcurl/7.68.0 OpenSSL/1.1.1g zlib/1.2.11 brotli/1.0.7 libidn2/2.3.0 libpsl/0.21.0 (+libidn2/2.3.0) libssh2/1.8.0 nghttp2/1.40.0 librtmp/2.3
Release-Date: 2020-01-08
$ lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux bullseye/sid
Release: testing
Codename: bullseye
Step 1 in both options will extract the Zscaler certificates.
OPTION 1 Direct curl
Download the certificates (all certificates are included in a single file)
Execute the curl command passing the certificateS you want to use.
# 1
openssl s_client -showcerts \
-connect jsonplaceholder.typicode.com:443 </dev/null 2>/dev/null \
| sed -n '/-----BEGIN CERTIFICATE-----/,/-----END CERTIFICATE-----/p' > typicode.crt
# 2
curl --cacert typicode.crt -v \
-d "userId=5&title=Hello World&body=Post body." \
https://jsonplaceholder.typicode.com/posts
OPTION 2 (installer script)
In case the curl command is executed by an installer you don't have control, then, update your certificates:
Extract the certificates from server (use the FQDN or IP and PORT, i.e: jsonplaceholder.typicode.com:443)
Move the XXX.crt certificate to your certificates directory
Update certificates
Execute installation script
# 1
openssl s_client -showcerts \
-connect jsonplaceholder.typicode.com:443 </dev/null 2>/dev/null \
| sed -n '/-----BEGIN CERTIFICATE-----/,/-----END CERTIFICATE-----/p' > typicode.crt
# 2
sudo mv typicode.crt /usr/local/share/ca-certificates/
# 3
sudo update-ca-certificates
# 4 execute your installer script
Bonus
In case you need/want to get the Zscaler certificates only, get the IP from: https://ip.zscaler.com
openssl s_client -showcerts -servername server -connect 165.225.216.33:443 > </dev/null 2>/dev/null | sed -n '/-----BEGIN CERTIFICATE-----/,/-----END CERTIFICATE-----/p' | grep -m1 -B-1 -- '-----END CERTIFICATE-----' > zscaler.crt
UPDATED (11/19/21):
Adding option 1, when is a direct curl and no need of install the certificates.
Optimized the command for extracting the certificates (creating the file)
Bonus: Getting the Zscaler IP
Tested on Ubuntu 20 and 18 behind Zscaler proxy.
Without certificate
With certificate
References:
How to install certificates for command line
unable to connect to server: x509: certificate signed by unknown authority
The answer is to "add that proxy's certificate to the CA bundle", thanks to Daniel Stenberg's answer. Then I guess I am suppose to fill in the rest. So here it is my attempt solving the remaining of the problems/questions --
Q: What is the easiest way to get that Zscaler certificate?
A: From here:
Go to Policy > SSL Inspection. In the Intermediate Root Certificate Authority for SSL Interception section, click Download Zscaler Root Certificate. Navigate to the ZscalerRootCerts. zip file and unzip it.
Q: How to add that certificate to the CA bundle?
A: See How to install company proxy certificate:
You can use curl --cacert <CA certificate> to supply your company CA cert.
Or you can add your company CA cert to /etc/pki/tls/certs/ and run make there to make it available system-wide.
This error (SSL certificate problem) means that the CA store that curl uses to verify the server's peer did not contain the cert and therefore the server couldn't be verified.
If want curl to work with a transparent proxy that terminates TLS you must add that proxy's certificate to the CA bundle or completely ignore the certificate check (which I recommend against).
A transparent proxy for TLS will of course make the connection completely unreliable and have broken security properties.
Right now I have to do an initial test of a mqtt broker (ssl).
However right now I don't have the valid truststore certificates, however I would like to test the basic connectivity, ignoring SSL errors regarding hostname verification, certificate validation etc.
Unfortunately I am not successful, even with a broker I know it's working.
What I'm doing:
mosquitto_sub -h the_host -p 8883 -t '#' -v -u myUser -P myPass --insecure -d --capath /etc/ssl/certs
According to the manpage I just use the --capath to identify it's a TLS connection, well knowing that the necessary root certificate is not available here.
What I get is this:
Client mosqsub|11262-csbox sending CONNECT
Error: A TLS error occurred.
Any idea what I'm doing wrong?
Using --insecure just disables the verification of the hostname in the certificate presented by the broker. It does not remove the need to have a copy of the CA certificate that signed the brokers certificate.
So if /etc/ssl/certs doesn't contain a matching CA certificate then the connection will fail.
If needed you should be able to use something like openssl s_client to download the certificate chain directly from the broker, you can then point to that file with the --cafile option instead of the --capath option.
I tried following the guide shown by mosquitto but once I launch the mosquitto
mosquitto -c mosquitto.conf
which defines port, location of ca.crt, server.crt, server.key
then I followed similar step using the same CA file, to sign the client key and certificate.
Then launched client
mosquitto_pub -p [port] -h localhost --cafile [ca.crt filepath] -t "hello" -m "hello world"
when I do it like this without key and certificate I get
Error: Success
but when I do it with key and certificate
mosquitto_pub -p [port] -h localhost --cafile [ca.crt filepath] --cert [client.crt path] --key [client1.key path] t "hello" -m "hello world"
I get
Error:Success
On the server side I See the following errors
... routines:SSL3_READ_BYTES: sslv3 alert certificate unknown
... routines:SSL3_READ_BYTES: ssl handshake failure
I ran openssl commands to verify CA approves of both generated certificates, and it did.
It turns out, when entering the detail of the certificate, i mistake the common name section's purpose. After I set it to the ip address of the server, it all worked well
I was getting the same error. I tried to subscribe like this:
mosquitto_sub -h ip_address -p 8883 -t topic --cafile /etc/mosquitto/ca_certificates/ca.crt -d.
Replace ip_addres with your ip address that you wrote when you created certificate. In your question, you wrote localhost. If you replace it with ip address it will be work.
I'm developing a program where I have a virtual development server that runs with a self signed certificate. My program uses curl to connect to the server and pull information, but needs to do so with SSL. When I try to connect I get the error "SSL certificate problem, verify that the CA cert is OK." When running firefox I can add the certificate to just firefox, but that doesn't help me with curl. How do I add the certificate for curl to recognize?
curl 7.19.7 (x86_64-pc-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8k zlib/1.2.3.3 libidn/1.15
Protocols: tftp ftp telnet dict ldap ldaps http file https ftps
Features: GSS-Negotiate IDN IPv6 Largefile NTLM SSL libz
Ubuntu 10.04 Lucid Lynx 64bit
This is one way that worked for me:
First, get the CA cert from the development domain and save it to a file called 'logfile'. (Assumes port 443 for SSL)
openssl s_client -connect xxxxx.com:443 |tee logfile
Then, use the --cacert curl option to use the saved certificate file.
curl --cacert logfile **THE REST OF YOUR CURL COMMAND**
Source:
http://curl.haxx.se/docs/sslcerts.html
I'd copy the certificate to /usr/local/share/ca-certificates/.
Let me quote the man page for update-ca-certificates:
Furthermore all certificates with a .crt extension found below /usr/local/share/ca-certificates are also included as implicitly trusted.
Add your rootCA.pem in /usr/share/ca-certificates directory.
After that update your certificates with: update-ca-certificates --fresh command.
I just did that, and works fine.
First, in your Linux, you should add your CERTIFICATE.cert to /usr/local/share/ca-certificates/.
After that by adding --cacert CERTIFICATE.cert to your command, curl will automatically use this certificate in that request.
Exp:
curl --cacert CERTIFICATE.cert GET "URL".
Obviously, you can edit the request to have your desired request.