Is it possible to use pip with a requirements file to install from a Mercurial repository which needs to be accessed using --insecure in order to work.
The repository is accessed via SSL, and due to pip ignoring Server Name Indication (SNI), a valid SSL certificate is not being honoured (it uses the main certificate for the IP address). --insecure gets round this issue
ie:
hg clone https://username#domain.org/username/app_name --insecure
So where you can usually do something like this:
pip install -e hg+https://username#domain.org/username/app_name#egg=app_name
or have the following in a pip requirements file:
hg+https://username#domain.org/username/app_name#egg=app_name
Neither of these work due to the SSL SNI issue.
Does anyone have any good suggestions?
Why don't you try to use [hostfingerprints] in your ~/.hgrc?
For example:
[hostfingerprints]
example.com = 38:76:52:7c:87:26:9a:8f:4a:f8:d3:de:08:45:3b:ea:d6:4b:ee:cc
To obtain the self-signed certificate, follow the steps suggested in https://www.mercurial-scm.org/wiki/CACertificates
The root CA certificate for a server can for example be retrieved with
Firefox. Browse to https://example.com/repo and verify that this is the
repository you trust, click the lock symbol in the lower right corner,
View Certificate, Details, select the certificate at the top of the
Certificate Hierarchy, Export, "X.509 Certificate (PEM)" and save
somewhere for example as server.pem. With other browsers on Windows
you have to view the certificate at the top of the Certification Path
and "Copy to File" as "Base-64 encoded X.509 (.CER)". Several such
files can be concatenated into one cacerts file.
Related
After the obligatory installation of Zscaler through out the Company my Anaconda started giving me the SSL verification Error while installing modules and using requests to get the urls
Error(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1076)'))': /simple/'some_module'/
SSLError: HTTPSConnectionPool(host='www.amazon.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])")))
With Zscaler being turned off it all works great, but the company policy does not allow that.....
I found some bypasses like setting verify to False but it is not what I want.
I would like to install the Zscaler certificate (which was provided to me by our IT department) to Anaconda
Now the problem seems to be that it uses conda’s generic certificates.
import ssl
print(ssl.get_default_verify_paths())
Output :
DefaultVerifyPaths(cafile=None, capath=None, openssl_cafile_env='SSL_CERT_FILE', openssl_cafile='C:\ci\openssl_1581353098519\_h_env\Library/cert.pem', openssl_capath_env='SSL_CERT_DIR', openssl_capath='C:\ci\openssl_1581353098519\_h_env\Library/certs')
Any idea what could I possibly do to point conda to the Zscaler certificate that I have??
system inf: Windows 10, Anaconda3 -2020.02, Python 3.7
Thanks a lot in advance
What you can do is :
Open a browser and go to www.google.com
Next to the reload page button, you will see a lock (see picture below). click on it
Click on : Certificat
Click on the tab: Certification Path
Select Zsclaer Root CA5 and the click on View Certificat button
Click on the tab: Details and then click on Copy to file button
Export the certificat choosing the base-64 encoded X.509 (.CER)
Choose a path where to save the file
Open Anaconda Prompt
conda config -set ssl_verify path_of_the_file_that_you_just_saved
background
I had this same issue, but ran into a similar with my work laptop except where Zscaler blocked my curl, git, and anaconda traffic. The temporary fix was to disable ssl verification, but this introduces a number of security vulnerabilities such as man-in-the-middle attacks.
From what I could gather and my limited research, WSL2 doesn't have a automatic way of importing ssl certificates from the system.
https://github.com/microsoft/WSL/issues/5134
Solution
The long term solution is to get the Zscaler certificate and add it to your shell file. Run the following commands in WSL after getting the certificate and navigating to the directory.
echo "export SSL_CERT_FILE=<Path to Certificate>/ZscalerRootCA.pem" >> $HOME/.bashrc
which I got from
https://help.zscaler.com/zia/adding-custom-certificate-application-specific-trusted-store#curl-SSL_CERT_FILE
They have more commands for other applications
If you use any other shells, make sure to change .bashrc to the directory of the configuration of that file. In my case I use fish, so I replaced $HOME/.bashrc with $HOME/.config/fish/config.fish
echo "export SSL_CERT_FILE=<Path to Certificate>/ZscalerRootCA.pem" >> $HOME/.config/fish/config.fish
After adding the certificate, make sure to reload the shell. In my case, I ran using instructions from jeffmcneil
source ~/.config/fish/config.fish
for bash, you would want to run
source ~/.bashrc
or
. ~/.bashrc
from
https://stackoverflow.com/a/2518150/16150356
Solution for Windows OS
After your Zscaler root cert is installed in the Windows trust root store, just install pip-system-certs the successor to python-certifi-win32 which is no longer maintained. Both packages are available from either pypi or conda-forge, so use either pip, conda, or mamba to install pip-system-certs into every Python environment in which you use the Requests package. The pip-system-certs package patches certifi at runtime to use the Windows trusted root store. This solves the issue for the requests package without resorting to setting $REQUESTS_CA_BUNDLE and/or editing your cacert.pem files.
Solution for Ubuntu
Copy the Zscaler root certificate file, it must have .crt ending and be in PEM format, to /usr/local/share/ca-certificates and use sudo update-ca-certificates to update your /etc/ssl/certs/ca-certificates.crt file. However, even then, pip-system-certs doesn't quite seem to work, so add export $REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt to your .profile and restart your shell.
For more information read the following:
Requests uses the certifi CA Certificate bundle
Certifi, a "carefully curated" bundle of CA certs
install CA certificate in Ubuntu trusted root store
Zscaler help, adding custom certificate root stores
installing custom root stores
WARNING: I do not recommend editing any Python cacert.pem files. Note that they are all linked so editing one edits all, and your mamba/conda solver may complain that your package cache is invalid because the file size changed due to your edits. Look in each environments ssl/ folder including base env, and in the base env's pkgs/ca-certificates-<date> files. On Windows OS, cacert.pem is in Library\ssl instead of ssl/. Finally the cacert.pem file will be overwritten if/when you install or update the Python certifi package, so editing it is really not the ideal solution. A better alternative would be to put your Zscaler root cert in a new ssl/ folder in your home directory and set $REQUESTS_CA_BUNDLE to that location. If your company is using Zscaler, then I think it's the only root cert you need.
I have an issue after I installed the SSL Certificate in WorldClient, MDaemon 12.
I followed the link in installing the SSL, SSL installation was fine. Then, I followed the link to use the SSL in WorldClient. I use 888 for TCP port and 443 for https port.
When I call the webmail (http://mail.mydomain.com.sg:888) in browser, it opens properly. But, if https://mail.mydomain.com.sg:888, Secure Connection Failed error message is shown.
Any Ideas, please? Thanks.
I have solved this problem.
If you read all instruction from the link you given in your post carefully until finished, you'll get the answer.
Note: MDaemon will only display certificates that have private keys
using the Personal Information Exchange format (PKCS #12). If your
imported certificate does not appear in the list then you may need to
import a *.PEM file, which contains both a certificate key and private
key. Importing this file using the same process outlined above will
convert it to the PKCS #12 format.
You should make a PKCS#12 file from generating the (certificate) .csr and private key (.key) you have. If you on windows, install the openssl first on your PC. After successful generate the PKCS#12 file, follow the instruction again to install the SSL certification on MDaemon.
If you success, you'll see the certificate appear in the MDaemon.
Remember to restart your server or webserver to apply the changes.
Regards
I am using wget in my program to get some file using HTTP protocol. Here i need to set security so we moved HTTP protocol to HTTPS.
After changing to HTTPS how to perform wget. I mean how to make trusted connection between two machines then perform wget.
I want to make sure that wget can be performed from certain system only.
Step 1: SSL Certificates
First things first, if this machine is on the internet and the SSL certificate is signed by a trusted source, there is no need to specify a certificate.
However, if there is a self signed certificate involved things get a little more interesting.
For example:
if this machine uses a self signed certificate, or
if you are on a network with a proxy that re-encrypts all https connections
Then you need to trust the public key of the self signed certificate. You will need to export the public key as a .CER file. How you got the SSL certificate will determine how you get the public key as a .CER
Once you have the .CER then...
Step 2: Trust the Certificate
I suggest two options:
option one
wget --ca-certificate={the_cert_file_path} https://www.google.com
option two
set the option on ~/.wgetrc
ca_certificate={the_cert_file_path}
Additional resources
Blog post about this wget and ssl certificates
wget manual
macOS users can use the cert.pem file:
wget --ca-certificate=/etc/ssl/cert.pem
or set in your ~/.wgetrc:
ca_certificate = /etc/ssl/cert.pem
On Linux (at least on my Debian and Ubuntu distributions), you can do the following to install your cert to be trusted system-wide.
Assuming your certificate is ~/tmp/foo.pem, do the following:
Install the ca-certificates package, if it is not already present, then do the following to install foo.pem:
$ cd ~/tmp
$ chmod 444 foo.pem
$ sudo cp foo.pem /usr/local/share/ca-certificates/foo.crt
$ sudo update-ca-certificates
Once this is done, most apps (including wget, Python and others) should automatically use it when it is required by the remote site.
The only exception to this I've found has been the Firefox web browser. It has its own private store of certificates, so you need to manually install the cert via its Settings interface if you require it there.
At least this has always worked for me (to install a corporate certificate needed for Internet access into the Linux VMs I create).
On Debian, using GitLab, I ran into issues with my self-signed certificate.
Reading through the code after a lot of searching on the Internet (I guess, it's the last resort, FOSS is helpful), I found the following lines in gitlab-shell/lib/gitlab_net.rb which left me... perplexed.
if config.http_settings['self_signed_cert']
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
end
Most Stack Overflow responses about the diverse issues I've had until now have led me to believe that VERIFY_NONE, as you'd expect, doesn't verify anything. VERIFY_PEER seems, based on my reading, to be the correct setting for self-signed.
As I read it, it feels like taking steps to secure my connection using a certificate, and then just deciding to not use it? Is it a bug, or am I misreading the source?
gitlab-shell (on the GitLab server) has to communicate to the GitLab instance through an HTTPS or SSH URL API.
If it is a self-signed certificate, it doesn't want any error/warning when trying to access those GitLab URLs, hence the SSL::VERIFY_NONE.
But, that same certificate is also used by clients (outside of the GitLab server), using those same GitLab HTTPS URLs from their browser.
For them, the self-signed certificate is useful, provided they install it in their browser keystore.
For those transactions (clients to GitLab), the certificate will be "verified".
The OP Kheldar point's out in Mislav's post:
OpenSSL expects to find each certificate in a file named by the certificate subject’s hashed name, plus a number extension that starts with 0.
That means you can’t just drop My_Awesome_CA_Cert.pem in the directory and expect it to be picked up automatically.
However, OpenSSL ships with a utility called c_rehash which you can invoke on a directory to have all certificates indexed with appropriately named symlinks.
(See for instance OpenSSL Verify location)
cd /some/where/certs
c_rehash .
I am getting the below error while making ssl connection with self signed certificate.
"Peer certificate cannot be authenticated with known CA certificates"
It is working fine with CA signed certificate.
I am setting the below using curl_easy_setopt().
curl_easy_setopt(MyContext, CURLOPT_CAPATH, CA_CERTIFICATE_PATH)
curl_easy_setopt(MyContext, CURLOPT_SSL_VERIFYPEER,TRUE);
The curl version:
libcurl-7.19.7-26
Openssl version is:
0_9_8u
Please let me know how to solve this issue.
By default CURL will generally verify the SSL certificate to see if its valid and issued by an accepted CA. To do this, curl uses a bundled set of CA certificates.
If you'd like to turn off curl's verification of the certificate, use the -k (or --insecure) option. Here's an example:
curl --noproxy -k \* -D - https://127.0.0.1:443/some-secure-endpoint
Security issue: This answer disables a security feature. Do not use this in production!
For php it is possible to switch off curl's verification of the certificate (see warning below) e.g. for curl_exec
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
http://php.net/manual/en/function.curl-setopt.php
(evaluate the security risk yourself, in my case it was on a partner company's server and the file required contained no secure information - just happened to be on a secure server)
We fixed a similar issue on CentOS 6 by updating curl to the latest version available in the standard repositories and installing the newest ca-certificates bundle:
yum update curl
yum install ca-certificates
libcurl performs peer SSL certificate verification by default. This is done
by using CA cert bundle that the SSL library can use to make sure the peer's
server certificate is valid.
If you communicate with HTTPS or FTPS servers using certificates that are
signed by CAs present in the bundle, you can be sure that the remote server
really is the one it claims to be.
Until 7.18.0, curl bundled a severely outdated ca bundle file that was
installed by default. These days, the curl archives include no ca certs at
all. You need to get them elsewhere. See below for example.
For more to know about Peer SSL Certificate Verification visit http://curl.haxx.se/docs/sslcerts.html
Though this error happened in the case of using git clone rather than with using curl, I've recently stumbled across an identical error message:
Peer certificate cannot be authenticated with known CA certificates
Similar to Arth's findings, something that worked for CentOS 6 (in order to successfully use HTTPS URLs with git clone for related GitLab repositories) involved updating the trusted certificates on the server (i.e., the server that is using HTTPS), using the following steps:
sudo yum install ca-certificates
sudo update-ca-trust enable
sudo cp /path/to/your_new_cert.crt /etc/pki/ca-trust/source/anchors/
sudo update-ca-trust extract
Perhaps the same certificate steps can be applied for the case of curl (or other similar scenarios) for users on CentOS in the future.
Security issue: This answer disables a security feature. Do not use this in production!
In 'C'
curl_easy_setopt(curl_handle, CURLOPT_SSL_VERIFYPEER, 0);
worked for me
As we checked and observed/ Found in Centos 8 .
Due to Proxy issue your packages not allowing you to get accessible to update or download any packages.
try to add sslverify=0 in file /etc/dnf/dnf.conf
Its worked for me.
Also make sure you must have proper internet acess on your server.