When I do pip search numpy I got the following:
numpy (1.19.2) - NumPy is the fundamental package for array computing with Python.
INSTALLED: 1.19.1
LATEST: 1.19.2
....
intel-numpy (1.15.1) - NumPy optimized with Intel(R) MKL library
....
I'm on ubuntu 18.04 with pip 20.2.3 and anaconda 4.6.8. Is that the numpy w/ MKL version in pip is only up to that 1.15.1 version, and we have to build from source if we want 1.19.x version?
I tried using conda install numpy and it gave me openBLAS version for 1.19.2 and BLAS for 1.19.1
Related
I am using Anaconda and Ubuntu 18.04. I installed TensorFlow probability base on Anaconda's command:
conda install -c conda-forge tensorflow-probability
but when I wanted to import TensorFlow probability, I faced with below error:
ImportError: cannot import name 'compiler' from 'tensorflow.python.autograph.pyct' (/home/alireza/anaconda3/envs/tf-gpu/lib/python3.8/site-packages/tensorflow/python/autograph/pyct/init.py)
I check Anaconda by conda list tensorflow and TensorFlow probability version 0.8 was installed.
I appreciate your suggestion.
I think it's been a long time since the conda recipe for TFP was updated (TFP is now at 0.12), so you would need to ensure the version of TensorFlow you have is compatible. FWIW from the 0.8 release notes: "It is tested and stable against TensorFlow version 2.0.0 and 1.15.0rc1."
I'm having trouble installing numpy on Ubuntu 18.04 for Python 3.7... here's what I tried:
(venv) root#servername:/path$ pip3.7 install numpy --no-cache-dir
Collecting numpy
Downloading numpy-1.19.5-cp37-cp37m-manylinux2010_x86_64.whl (14.8 MB)
|████████████████████████████████| 14.8 MB 8.6 MB/s
Installing collected packages: numpy
Successfully installed numpy-1.19.5
I get this ImportError:
Original error was: No module named 'numpy.core._multiarray_umath'
... but I've been focusing on this as the probable cause, from that same error message:
* The Python version is: Python3.6 from "/path/venv/bin/python"
... that says Python 3.6, but I'm trying to use Python3.7, and that python is on 3.7:
(venv) root#servername:/path$ /path/venv/bin/python --version
Python 3.7.9
I saw in another answer that upgrading can help, so I tried this - it looks like I'm already on the latest:
(venv) root#servername:/path$ pip install numpy --upgrade
Requirement already satisfied: numpy in ./venv/lib/python3.7/site-packages (1.19.5)
How can I get this working?
This was failing for me while using mod-wsgi with Apache - I had to load mod-wsgi for Python 3.7 instead of the system default of 3.6. The instructions are at https://github.com/GrahamDumpleton/mod_wsgi/issues/467, but in case that goes away:
Using your venv, run "mod_wsgi-express module-config"
Put the resulting load command into your apache's mods-enabled/wsgi.load
I have a conda environment with PyTorch and Tensorflow, which both require CUDA 9.0 (~cudatoolkit 9.0 from conda). After installing pytorch with torchvision and the cudatoolkit (like they provided on their website) I wanted to install Tensorflow, the problem here is that I get this error:
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: /
Found conflicts! Looking for incompatible packages.
This can take several minutes. Press CTRL-C to abort.
failed
UnsatisfiableError: The following specifications were found
to be incompatible with the existing python installation in your environment:
Specifications:
- tensorflow==1.12.0 -> python[version='2.7.*|3.6.*']
- tensorflow==1.12.0 -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0']
Your python: python=3.5
If python is on the left-most side of the chain, that's the version you've asked for.
When python appears to the right, that indicates that the thing on the left is somehow
not available for the python version you are constrained to. Note that conda will not
change your python version to a different minor version unless you explicitly specify
that.
The following specifications were found to be incompatible with your system:
- feature:/linux-64::__cuda==10.2=0
- feature:|#/linux-64::__cuda==10.2=0
Your installed version is: 10.2
If I run nvcc or nvidia-smi on my host or the activated conda environment, I get that I have installed CUDA 10.2, even though conda list shows me that cudatoolkit 9.0 is installed. Any solution to this?
EDIT:
When running this code sample:
# setting device on GPU if available, else CPU
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
print('Using device:', device)
print()
#Additional Info when using cuda
if device.type == 'cuda':
print(torch.cuda.get_device_name(0))
print('Memory Usage:')
print('Allocated:', round(torch.cuda.memory_allocated(0)/1024**3,1), 'GB')
print('Cached: ', round(torch.cuda.memory_cached(0)/1024**3,1), 'GB')
print(torch.version.cuda)
I get this output:
GeForce GTX 1050
Memory Usage:
Allocated: 0.0 GB
Cached: 0.0 GB
9.0.176
So PyTorch does get the correct CUDA version, I just cant get tensorflow-gpu installed.
If I run nvcc or nvidia-smi on my host or the activated conda environment, I get that I have installed CUDA 10.2, even though conda list shows me that cudatoolkit 9.0 is installed. Any solution to this?
cudatoolkit doesn't ship with compiler (nvcc), thus when you run nvcc you start compiler from system wide installation. That's why it prints 10.2 istead of 9.0, and pytorch sees the local cudatoolkit.
anaconda / packages / cudatoolkit :
This CUDA Toolkit includes GPU-accelerated libraries, and the CUDA runtime for the Conda ecosystem. For the full CUDA Toolkit with a compiler and development tools visit https://developer.nvidia.com/cuda-downloads
From your comment above I understood that you are using python=3.5.6. So, first of all you should search for available tensorflow py35 builds using:
conda search tensorflow | grep py35
I have the following output:
tensorflow 1.9.0 eigen_py35h8c89287_1 pkgs/main
tensorflow 1.9.0 gpu_py35h42d5ad8_1 pkgs/main
tensorflow 1.9.0 gpu_py35h60c0932_1 pkgs/main
tensorflow 1.9.0 gpu_py35hb39db67_1 pkgs/main
tensorflow 1.9.0 mkl_py35h5be851a_1 pkgs/main
tensorflow 1.10.0 eigen_py35h5ed898b_0 pkgs/main
tensorflow 1.10.0 gpu_py35h566a776_0 pkgs/main
tensorflow 1.10.0 gpu_py35ha6119f3_0 pkgs/main
tensorflow 1.10.0 gpu_py35hd9c640d_0 pkgs/main
tensorflow 1.10.0 mkl_py35heddcb22_0 pkgs/main
As you can see there is no tensorflow 1.12.0 builds for py35, and that's why you are getting that error. You can try to inspect other conda channels, for example, conda-forge:
conda search tensorflow -c conda-forge | grep py35
But that wasn't helpful:
tensorflow 0.9.0 py35_0 conda-forge
tensorflow 0.10.0 py35_0 conda-forge
tensorflow 0.11.0rc0 py35_0 conda-forge
tensorflow 0.11.0rc2 py35_0 conda-forge
tensorflow 0.11.0 py35_0 conda-forge
tensorflow 0.12.1 py35_0 conda-forge
tensorflow 0.12.1 py35_1 conda-forge
tensorflow 0.12.1 py35_2 conda-forge
tensorflow 1.0.0 py35_0 conda-forge
tensorflow 1.1.0 py35_0 conda-forge
tensorflow 1.2.0 py35_0 conda-forge
tensorflow 1.2.1 py35_0 conda-forge
tensorflow 1.3.0 py35_0 conda-forge
tensorflow 1.4.0 py35_0 conda-forge
tensorflow 1.5.0 py35_0 conda-forge
tensorflow 1.5.1 py35_0 conda-forge
tensorflow 1.6.0 py35_0 conda-forge
tensorflow 1.8.0 py35_0 conda-forge
tensorflow 1.8.0 py35_1 conda-forge
tensorflow 1.9.0 eigen_py35h8c89287_1 pkgs/main
tensorflow 1.9.0 gpu_py35h42d5ad8_1 pkgs/main
tensorflow 1.9.0 gpu_py35h60c0932_1 pkgs/main
tensorflow 1.9.0 gpu_py35hb39db67_1 pkgs/main
tensorflow 1.9.0 mkl_py35h5be851a_1 pkgs/main
tensorflow 1.9.0 py35_0 conda-forge
tensorflow 1.10.0 eigen_py35h5ed898b_0 pkgs/main
tensorflow 1.10.0 gpu_py35h566a776_0 pkgs/main
tensorflow 1.10.0 gpu_py35ha6119f3_0 pkgs/main
tensorflow 1.10.0 gpu_py35hd9c640d_0 pkgs/main
tensorflow 1.10.0 mkl_py35heddcb22_0 pkgs/main
tensorflow 1.10.0 py35_0 conda-forge
So, the possible solutions are:
Install one of the older available tensorflow 1.10.0 gpu_py35 builds.
Switch to python 3.6.
conda search tensorflow | grep py36
...
tensorflow 1.11.0 gpu_py36h4459f94_0 pkgs/main
tensorflow 1.11.0 gpu_py36h9c9050a_0 pkgs/main
...
tensorflow 1.12.0 gpu_py36he68c306_0 pkgs/main
tensorflow 1.12.0 gpu_py36he74679b_0 pkgs/main
...
Note that versions >=1.13.1 doesn't support CUDA 9.
Use pip install inside conda env to install missing tensorflow build, because pip hosts more build combinations: Tested build configurations
Here is some best practices from Anaconda how to use pip w/ conda: Using Pip in a Conda Environment
The last option is to build your own missing conda package with conda-build
My experience is that even though the detected cuda version is incorrect by conda, what matters is the cudatoolkit version.
The actual problem for me was the incompatible python version.
E.g. You might have Python 3.7/3.8 but this version of tensorflow doesn't support that:
tensorflow==1.12.0 -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0']
Try an older conda environment or a newer tensorflow, the latest version before 2.0 is 1.15
If I haven't misunderstood, you installed the packages using the
conda install
from https://pytorch.org/get-started/locally/
I once had problems with the conda installation of PyTorch and Cuda: i solved by removing the packages installed with conda and re-installing it via pip.
If you are afraid to mess up the conda enviroment using pip, I suggest you to create another enviroment in order to test this solution.
Use conda to remove pytorch and cuda. See Removing Packages at Conda Managing packages
Install the cuda toolkit you need. Note that pytorch supports only cuda 9.2, 10.1 and 10.2, as you can see on the Pytorch download page.
If your OS is ubuntu 19, follow the CUDA instructions for ubuntu 18. Also note that not all gpus support the latest version of the toolkit for driver reasons (the 1050 should be recent enough to support them all, up to 10.1 sure because I used it).
Follow the instructions to install pytorch with the appropiate cuda support via pip at Pytorch download page. See Using pip in an enviroment at Conda Managing packages
I have the tensorflow and tensorflow-gpu 1.8.0 conda (not pip) packages installed in a conda environment on Ubuntu 16.04.4:
conda list t.*flow
# packages in environment at /home/lebedov/miniconda3/envs/TF:
#
# Name Version Build Channel
_tflow_180_select 1.0 gpu
tensorflow 1.8.0 py36_1 conda-forge
tensorflow-gpu 1.8.0 h7b35bdc_0
I have CUDA 9.0 installed on my system, which has a Quadro M2200 GPU. I can see the GPU listed in the output of nvidia-smi and can also access the GPU using other deep learning frameworks such as PyTorch 0.4.0, but for some reason TensorFlow doesn't seem to see it:
Python 3.6.5 | packaged by conda-forge | (default, Apr 6 2018, 13:39:56)
Type 'copyright', 'credits' or 'license' for more information
IPython 6.4.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]: import tensorflow as tf
...: sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
2018-07-11 23:21:11.827064: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
Device mapping: no known devices.
2018-07-11 23:21:11.827942: I tensorflow/core/common_runtime/direct_session.cc:284] Device mapping:
If I downgrade to tensorflow-gpu 1.7.0, however, I can see the GPU. Any thoughts as to why the GPU isn't being detected by TensorFlow 1.8.0?
The tensorflow 1.8.0 packages from Anaconda appear to support GPUs properly, but those from conda-forge do not; see this issue.
I also had similar instances where the tensorflow-gpu won't run on default.
As for me, I solved it by just uninstalling tensorflow and only keeping tensorflow-gpu. That way, it would always run with GPU as there was no other option.
As far as compatibility is concerned, I would recommend the new conda way of installing tensorflow-gpu, which automatically installs the relevant cudnn files for you. The code would be as follows:
conda create -n [EnvironmentName] python=3.6
conda activate [EnvironmentName]
conda install -c conda-forge tensorflow-gpu==1.14
it will assess which version (CUDA,CUDNN, etc.) you require and download and install it directly to your environment. Then run your python file from this environment.
Do check afterwards with "conda list" to ensure that tensorflow-gpu is installed for you.
I downloaded the tensorflow package for python3.6.0 from http://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy then installed it by pip, it showed successful . when I import tensorflow as tf,it failed like following
import tensorflow as tf
From Documentation
Install TensorFlow Python dependencies
To install TensorFlow, you must install the following packages:
numpy, which is a numerical processing package that TensorFlow requires.
dev, which enables adding extensions to Python.
pip, which enables you to install and manage certain Python packages.
wheel, which enables you to manage Python compressed packages in the wheel (.whl) format.
Install the dependency packages First.
or direct from command prompt.
pip3 install --upgrade tensorflow
for CPU version only for TensorFlow.
pip3 install --upgrade tensorflow-gpu
for GPU version of TensorFlow