certificate verify failed using pip install, main problem, (_ssl.c:1108) - ssl

I've been trying to install python packages using:
pip install <package>
and all I get is:
"WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
certificate verify failed: unable to get local issuer certificate (_ssl.c:1108)'))': /simple/pip/"
I know the code that can fix the problem, for which is (seldomly works):
pip3 install --trusted-host=pypi.org --trusted-host=files.pythonhosted.org --user <package>
but I want to know what is happening behind the scene, what's the core problem and how can I use pip like always!
I'd be tremendously happy if you could help me out.

first write this
pip.ini
and now try to install packages
still if you are getting error then type this
pip install certifi
then write this
pip install Scrapy

Related

Buildozer error while compiling apk - pip ssl

I am trying to compile a simple program into an apk for Android with Buildozer but have run into the following problem. Can you please help? I tried upgrading pip but that didn't help:python3 -m pip install --upgrade pip
So, I reverted back to the original pip version.
Installed Cython separately: pip3 install Cython
But the same issue persists. I am at a loss. :-(
Command: buildozer android debug
RAN: /bin/bash -c 'venv/bin/pip install Cython'
STDOUT:
WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
How can I fix this?
Could not fetch URL https://pypi.org/simple/cython/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/cython/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.")) - skipping
ERROR: Could not find a version that satisfies the requirement Cython (from versions: none)
ERROR: No matching distribution found for Cython
WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
This is a recent bug, you need to install libssl-dev using apt install libssl-dev
Then you might also want to clean your buildozer directory by running rm -rf .buildozer in the directory that contains your buildozer.spec file.
That should do it!

How to fix matplotlib installation errors

I do not know how to fix this, any help is appreciated. Thanks in advance
-J
I have tried various ways of installing and importing matplotlib (though command prompt, the text editor I've been using (geany). and neither option has worked. I've also installed the .whl file from https://matplotlib.org/ and it still has not worked (gives the same error)
I originally used the code:
import matplotlib.pyplot as plt
in my text editor (geany) to install matplotlib but this gave the error shown below
Expected Results:
I expect it to import and install the right version of matplotlib (I think the right version for me is matplotlib 3.0 because I'm using python 3.7).
Actual Results:
When I type 'python -m pip install -U matplotlib' into my command prompt (I'm on windows 10, using python 3.7) it displays:
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)'))': /simple/matplotlib/
and repeats the message for a total of 5 times then displays:
Could not fetch URL https://pypi.org/simple/matplotlib/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/matplotlib/ (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)'))) - skipping
and finishes with the error:
ERROR: Could not find a version that satisfies the requirement matplotlib (from versions: none)
ERROR: No matching distribution found for matplotlib
I have tried to fix this SSLError but I haven't made any progress in this simply because I don't know where to start.
Thanks again for any help
From this topic:
You should add the pypi host to the list of trusted host. To do so, enter those commands:
pip install --trusted-host pypi.python.org pytest-xdist
pip install --trusted-host pypi.python.org --upgrade pip
This could happen if you're behind a firewall and console is blocked from internet. You can download the package to local and run
python -m pip --install -e /path/to/package/folder
or download the wheel file and run
python -m pip install /path/to/wheel_file.whl
A huge thank you to #Louis 'LYRO' Dupont and #Quang Hoang for helping me work through this. What I ended up doing was going to pypi.python.org and downloading all of the requirements as .whl files in order to run matplotlib. This took a bit of work as I had to try to install matplotlib after installing each requirement to see what the next one I needed was. Thanks again to both you guys and have a nice week!

Install library from downloaded pip files

I want to download and install some specific library (it does not have to be tensorflow, but i ll use it as an example)
I have a ubuntu virtual_machine_1 with internet connection.
I do:
pip download tensorflow
After downloading all the files in current folder, I transfer them into virtual machine_2
There I unpack files into folder_with_tf_dependencies and do:
pip install --user --find-links=../folder_with_tf_dependencies
The error I get is:
Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7ff735bbe090>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',)': /simple/tensorflow/

pip is unable to install selenium package

I am getting the below error while trying to install selenium package for Python.
Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionRefused Error(10061, 'No connection could be made because the target machine actively refused it', None, 10061, None))': /simple/selenium/
The command I am using is pip.exe install selenium.
I am now able to resolve the proxy/protocol issue by using below command
python.exe -m pip install selenium --proxy="proxy:port"
But with Python 3.5.1 I am still unable to install selenium package and now the error is:
Could not find a version that satisfies the requirement selenium (from versions: ) No matching distribution found for selenium

pip always fails ssl verification

Pip always fails ssl even when I do pip install dedupe or pip install --trusted-host pypi.python.org dedupe
The output is always the same no matter what:
Collecting dedupe
Retrying (Retry(total=4, connect=None, read=None,
redirect=None, status=None)) after connection broken by
'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate
verify failed (_ssl.c:777)'),)': /simple/dedupe/
Retrying...
skipping
Could not find a version that satisfies the requirement dedupe (from versions: ) No matching distribution found for dedupe
So I uninstalled anaconda and reinstalled it. Same thing.
Do you think the problem is that my _ssl.c file (which I have no idea where it is) must be corrupt or something? Why would pip need to reference that if I'm telling it to bypass ssl verification anyway?
It may be related to the 2018 change of PyPI domains.
Please ensure your firewall/proxy allows access to/from:
pypi.org
files.pythonhosted.org
So you could give a try to something like:
$ python -m pip install --trusted-host files.pythonhosted.org --trusted-host pypi.org --trusted-host pypi.python.org [--proxy ...] [--user] <packagename>
Please see $ pip help install for the --user option description (omit if in a virtualenv).
The --trusted-host option doesn't actually bypass SSL/TLS, but allows to mark host as trusted when (and only when) it does not have valid (or any) HTTPS. It shouldn't really matter with PiPY because pypi.org (formerly pypi.python.org) does use HTTPS and there is CDN in front of it which always enforces TLSv1.2 handshake requirement regardless of the connecting pip client options.. But if you had your own local mirrors of pypi.org with HTTP-only access, then --trusted-host could be handy. Oh, and if you are behind a proxy, please also make sure to also specify: --proxy [user:passwd#]proxyserver:port
Some corporate proxies may even go as far as to replace the certificates of HTTPS connections on the fly. And if your system clock is out of sync, it could break SSL verification process as well.
If firewall / proxy / clock isn't a problem, then check SSL certificates being used in pip's SSL handshake. In fact, you could just get a current cacert.pem (Mozilla's CA bundle from curl) and try it using the pip option --cert:
$ pip --cert ~/cacert.pem install --user <packagename>
where --cert argument is system path to your alternate CA bundle in PEM format. (regarding the --user option, please see below).
Or, it's possible to create a custom config ~/.pip/pip.conf and point the option at a valid system cert (or your cacert.pem) as a workaround, for example:
[global]
cert = /etc/pki/tls/external-roots/ca_bundle.pem
(or another pem file)
It's even possible to manually replace the original cacert.pem found in pip with your trusty CA bundle (if your pip is very old for example). Older pip versions knew to fallback between pip/_vendor/requests/cacert.pem and system stores like /etc/ssl/certs/ca-certificates.crt or /etc/pki/tls/certs/ca-bundle.crt in case of cert issues, but in recent pip it's no longer the case, as it seems to rely solely on pip/_vendor/certifi/cacert.pem
Basically, pip package uses requests which uses urllib3 which, among other things, verifies SSL certificates; and all of them are shipped (vendored) within pip, along with the certifi package (also included, since pip 9.0.2) that provides current CA bundle (cacert.pem file) required for TLS verification. Requests itself uses urllib3 and certifi internally, and before 9.0.2, pip used cacert.pem from requests or the system. What it all means is that actually updating pip may help fix the CERTIFICATE_VERIFY_FAILED error, particularly if the OS and pip were deployed long ago:
The OP used anaconda, so they could try:
$ conda update pip - because issues can arise if conda and pip are both used together in the same environment. If there's no pip version update available, they could try:
$ conda config --add channels conda-forge; conda update pip
Alternatively, it's possible to use conda alone to directly install / manage python packages: it is a tool completely separate from pip, but provides similar features in terms of package and venv management. Its packages come not from PyPI, but from anaconda's own repositories.
The problem is, if you mix both and run conda after pip, the former can overwrite and break packages (and their dependencies) installed via pip, and render it all unusable. So it's recommended to only use one or the other, or, if you have to, use only pip after conda (and no conda after pip), and only in isolated conda environments.
On normal Linux Python installations without conda:
If you are using a version of pip supplied by your OS distribution, then use vendor-supplied upgrades for a system-wide pip update:
$ sudo apt-get install python-pip or: $ sudo yum install python27-pip
Some updates may not be readily available because distros usually lag behind PyPI. In this case, it's possible to upgrade pip at your user level (right in your $HOME dir), or inside a virtualenv, like:
$ python -m pip install --user --trusted-host files.pythonhosted.org --trusted-host pypi.org --trusted-host pypi.python.org --upgrade pip
(omit --user if in a virtualenv)
The --user switch will upgrade pip only for the current user (in your home ~/.local/lib/) rather than for the whole OS, which is a good practice to avoid interfering with the system python packages. It's enabled by default in a pip distributed in recent Ubuntu/Fedora versions. Be aware of how to solve ImportError if you don't use this option and happen to overwrite the OS-level system pip.
Alternatively (also at a user level) you could try:
$ curl -LO https://bootstrap.pypa.io/get-pip.py && python get-pip.py --user
The PyPA script contains a wrapper that extracts the .pem SSL bundle from pip._vendor.certifi.
Otherwise, if still no-go, try running pip with -vvv option to add verbosity to the output and check if there is now another SSLError caused by tlsv1 alert protocol version.
This worked for me, try this:
pip install --trusted-host=pypi.org --trusted-host=files.pythonhosted.org --user {name of whatever I'm installing}
My way is a simplification of #Alex C's answer:
python -m pip install --trusted-host pypi.python.org --trusted-host files.pythonhosted.org --trusted-host pypi.org --upgrade pip
I experienced the same issue because I have Zscaler (a cloud security software) installed and was causing:
URL host for python packages being blocked
invalid SSL certificate warnings popping up
SSL inspection certificate not trusted
As mentioned by others, the below will fix individual package installations. pypi.python.org is not required since it has been replaced by pypi.org.
pip install --trusted-host pypi.org --trusted-host files.pythonhosted.org <package to install>
I permanently fixed the issue by creating pip.ini file (pip.conf in Unix) and adding the below:
[global]
trusted-host = pypi.python.org
pypi.org
files.pythonhosted.org
See pip configuration files for how to locate your pip.ini, or where to put it if you need to create one.
The error above or one like it was caused by the virtual machine (VM) not be time synchronized, my guest Ubuntu VM was several days in the past.
I ran this commend to get the VM to pick up the correct network time:
sudo timedatectl set-ntp on
This makes the Ubuntu guest OS get the network time. (You may have to provide a network time source... I used this article: Digital Ocean - How to set time on Ubuntu)
Check the time is correct:
timedatectl
Re-run the failing pip command.