Building wheel for pandas on Ubuntu 20.04 takes more than 20 minutes, but not on 18.04 - pandas

I have an installation script for ERPNext that works just fine on Ubuntu 18.04.
When I run the same script on 20.04 I am obliged to wait more than 20 minutes for it to complete where it takes around 30 secs on 18.04.
My script includes these two lines:
./env/bin/pip install numpy==1.18.5
./env/bin/pip install pandas==0.24.2
Their output is:
Collecting numpy==1.18.5
Downloading numpy-1.18.5-cp38-cp38-manylinux1_x86_64.whl (20.6 MB)
|████████████████████████████████| 20.6 MB 138 kB/s
Installing collected packages: numpy
Successfully installed numpy-1.18.5
Collecting pandas==0.24.2
Downloading pandas-0.24.2.tar.gz (11.8 MB)
|████████████████████████████████| 11.8 MB 18.0 MB/s
Requirement already satisfied: python-dateutil>=2.5.0 in ./env/lib/python3.8/site-packages (from pandas==0.24.2) (2.8.1)
Requirement already satisfied: pytz>=2011k in ./env/lib/python3.8/site-packages (from pandas==0.24.2) (2019.3)
Requirement already satisfied: numpy>=1.12.0 in ./env/lib/python3.8/site-packages (from pandas==0.24.2) (1.18.5)
Requirement already satisfied: six>=1.5 in ./env/lib/python3.8/site-packages (from python-dateutil>=2.5.0->pandas==0.24.2) (1.13.0)
Building wheels for collected packages: pandas
Building wheel for pandas (setup.py) ... done
Created wheel for pandas: filename=pandas-0.24.2-cp38-cp38-linux_x86_64.whl size=43655329 sha256=0067caf3a351f263bec1f4aaa3e11c5857d0434db7f56bec7135f3c3f16c8c2b
Stored in directory: /home/erpdev/.cache/pip/wheels/3d/17/1e/85f3aefe44d39a0b4055971ba075fa082be49dcb831db4e4ae
Successfully built pandas
Installing collected packages: pandas
Successfully installed pandas-0.24.2
The line "Building wheel for pandas (setup.py) ... /" is where the 20 min delay occurs.
This is all run from within the Frappe/ERPnext command directory, which has an embedded copy of pip3, like this:
erpdev#erpserver:~$ cd ~/frappe-bench/
erpdev#erpserver:~/frappe-bench$ ./env/bin/pip --version
pip 20.1.1 from /home/erpdev/frappe-bench/env/lib/python3.8/site-packages/pip (python 3.8)
erpdev#erpserver:~/frappe-bench$
I would be most grateful for any suggestions how to speed it up.

I just update pip using pip install --upgrade pip and it solved.

Your issue may be less to do with your distribution and more to be with the Python version in your virtualenv. Ubuntu 20.04 has its default Python pointing to 3.8.
From the pandas project listing on PyPI, your pip searches for a version that's compatible with your system, as provided by the project maintainers.
It seems you're using CPython3.8. pandas==0.24.2 does not wheels built for your version, so your system builds them for itself each time. You can check the available download files from here.
Possible Solutions:
While creating your env, check out this answer to generate a virtual environment for a different version. Seems like your options are between 3.5, 3.6 and 3.7.
Build a wheel for CPython3.8 and ship it along with your script. You can install your package from using that.

Related

ModuleNotFoundError: No module named 'pandas' (Problem with Python3.9)

Initially, I ran experiments on python3, which supports Pandas perfectly. Since our lab recently upgraded the server to an RTX 3000 series GPU, it supports Python 3.9, not previous versions. For that, when I ran the tests in python3.9 (python3.9 code file.py), the following error occurred:
ModuleNotFoundError: No module named 'pandas'
Then I ran 'pip install pandas' and found that the requirements had already been met; here's further information:
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: pandas in ./.local/lib/python3.6/site-packages (1.1.5)
Requirement already satisfied: pytz>=2017.2 in ./.local/lib/python3.6/site-packages (from pandas) (2021.1)
Requirement already satisfied: python-dateutil>=2.7.3 in ./.local/lib/python3.6/site-packages (from pandas) (2.8.2)
Requirement already satisfied: numpy>=1.15.4 in ./.local/lib/python3.6/site-packages (from pandas) (1.19.5)
Requirement already satisfied: six>=1.5 in ./.local/lib/python3.6/site-packages (from python-dateutil>=2.7.3->pandas) (1.15.0)
But when I ran the file again, I received the same issue: ModuleNotFoundError: No module named 'pandas'
How can we install solve this issue?
You need to make sure you are running pip with the correct python version. ./.local/lib/python3.6/site-packages in your error indicates that you are still using python3.6.
If you are using IPython, you can type pip install pandas directly in the interactive mode (not in the terminal), which uses the proper Python version then.
In general, I would create a virtual environment using /path/to/python39 -m venv ~/myvenv, then active it using source ~/myvenv/bin/activate and install packages then - especially if using a graphics card you may need different package versions for different projects.
According to my understanding, our lab server supports various Python versions, and I was executing my code files from a directory with a Python version of 3.6, which caused the issue. Therefore, we must either change our project directory to Python 3.9 and install Pandas there or use the following command as instructed here.
python3.9 -m pip install pandas

Install numpy for Python 3.7 in Ubuntu 18.04

I'm having trouble installing numpy on Ubuntu 18.04 for Python 3.7... here's what I tried:
(venv) root#servername:/path$ pip3.7 install numpy --no-cache-dir
Collecting numpy
Downloading numpy-1.19.5-cp37-cp37m-manylinux2010_x86_64.whl (14.8 MB)
|████████████████████████████████| 14.8 MB 8.6 MB/s
Installing collected packages: numpy
Successfully installed numpy-1.19.5
I get this ImportError:
Original error was: No module named 'numpy.core._multiarray_umath'
... but I've been focusing on this as the probable cause, from that same error message:
* The Python version is: Python3.6 from "/path/venv/bin/python"
... that says Python 3.6, but I'm trying to use Python3.7, and that python is on 3.7:
(venv) root#servername:/path$ /path/venv/bin/python --version
Python 3.7.9
I saw in another answer that upgrading can help, so I tried this - it looks like I'm already on the latest:
(venv) root#servername:/path$ pip install numpy --upgrade
Requirement already satisfied: numpy in ./venv/lib/python3.7/site-packages (1.19.5)
How can I get this working?
This was failing for me while using mod-wsgi with Apache - I had to load mod-wsgi for Python 3.7 instead of the system default of 3.6. The instructions are at https://github.com/GrahamDumpleton/mod_wsgi/issues/467, but in case that goes away:
Using your venv, run "mod_wsgi-express module-config"
Put the resulting load command into your apache's mods-enabled/wsgi.load

pandas installation error using pip installer

I am getting following error repeatedly while installing pandas through pip installer for python 3.7 in command prompt
Using cached https://files.pythonhosted.org/packages/26/fc/d0509d445d2724fbc5f9c9a6fc9ce7da794873469739b6c94afc166ac2a2/pandas-0.23.4-cp37-cp37m-win32.whl
Collecting pytz>=2011k (from pandas)
Downloading https://files.pythonhosted.org/packages/61/28/1d3920e4d1d50b19bc5d24398a7cd85cc7b9a75a490570d5a30c57622d34/pytz-2018.9-py2.py3-none-any.whl (510kB)
100% |████████████████████████████████| 512kB 204kB/s
Collecting python-dateutil>=2.5.0 (from pandas)
Downloading https://files.pythonhosted.org/packages/74/68/d87d9b36af36f44254a8d512cbfc48369103a3b9e474be9bdfe536abfc45/python_dateutil-2.7.5-py2.py3-none-any.whl (225kB)
100% |████████████████████████████████| 235kB 187kB/s
Collecting numpy>=1.9.0 (from pandas)
Downloading https://files.pythonhosted.org/packages/94/b5/f4bdf7bce5f8b35a2a83a0b70c545ca061a50b54724b5287505064906b14/numpy-1.16.0-cp37-cp37m-win32.whl (10.0MB)
100% |████████████████████████████████| 10.0MB 139kB/s
Could not install packages due to an EnvironmentError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\LENOVO\\AppData\\Local\\Temp\\pip-req-tracker-hwub07hg\\29e06807d5aed8dd372ea37c64d1e88dc172ee212d473a412d5e638c'
Consider using the `--user` option or check the permissions.
I have run the command through administrator but it didnt workout.
Try to upgrade as Jonny suggested.Restart your computer,Upgrade pip and try installing pandas with Unofficial Windows Binaries for Python Extension Packages or pip install --user <package_name>.
Also take a look at PermissionError: [WinError 32] The process cannot access the file because it is being used by another process.
Hope it was helpful.

Make colab use the latest installation of a library

I am trying to use the bleeding edge version of sklearn installing it from their github as shown on line 2 in the below image. Line 5 imports some functions from this version of sklearn. This line works in my local and not on Google Colab. Am I missing something to hint the tool to use the latest installed version and not its cached version?
I am not sure why that's happening but if you uninstall scikit-learn before installing the latest dev, it would work:
[1] !pip uninstall scikit-learn -y
Uninstalling scikit-learn-0.19.1:
Successfully uninstalled scikit-learn-0.19.1
[2]!pip install Cython
!pip install git+git://github.com/scikit-learn/scikit-learn.git
Requirement already satisfied: Cython in /usr/local/lib/python3.6/dist-packages (0.28.2)
Collecting git+git://github.com/scikit-learn/scikit-learn.git
Cloning git://github.com/scikit-learn/scikit-learn.git to /tmp/pip-req-build-d59ukisw
Requirement already satisfied: numpy>=1.8.2 in /usr/local/lib/python3.6/dist-packages (from scikit-learn==0.20.dev0) (1.14.3)
Requirement already satisfied: scipy>=0.13.3 in /usr/local/lib/python3.6/dist-packages (from scikit-learn==0.20.dev0) (0.19.1)
Building wheels for collected packages: scikit-learn
Running setup.py bdist_wheel for scikit-learn ... done
Stored in directory: /tmp/pip-ephem-wheel-cache-is88dk15/wheels/a1/50/0e/316ef2ff8d4cfade292bd20b49efda94727688a153382745a6
Successfully built scikit-learn
Installing collected packages: scikit-learn
Successfully installed scikit-learn-0.20.dev0
[3] !pip freeze | grep scikit
scikit-image==0.13.1
scikit-learn==0.20.dev0
[4] from sklearn.preprocessing import CategoricalEncoder
[5] import sklearn
sklearn.__version__
'0.20.dev0'

tensorflow not supported wheel on this platform

I've searched around and non of the solutions seem to pertain to me, so here I am.
I installed anaconda 5.1 for python 3.6, I downloaded and installed 64-Bit(x86)Installer(551 MB)
from
https://www.anaconda.com/download/#linux
I followed the directions here
https://docs.anaconda.com/anaconda/install/linux
I had the install prepend the path and install microsoft VS code.
I then attempt to install the CPU only tensorflow using anaconda as suggested here
https://www.tensorflow.org/install/install_linux#InstallingAnaconda
I try to install the binary for python 3.6 CPU only
https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.6.0-cp36-cp36m-linux_x86_64.whl
I get the following error
tensorflow-1.6.0-cp36-cp36m-linux_x86_64.whl is not a supported wheel on this platform.
I am running a Ubuntu 16.04 VM on windows 10.
edit: when I run this command
pip install --ignore-installed --upgrade tfBinaryURL
outside of the tensorflow environment it worked.
2nd edit:
Additionally I explored my tensorflow environment in my anaconda3 folder, and I noticed it only has python 2.7, so when I tried to install the cpu only tensorflow while in the enviroment for python 2.7 it worked.