When I install tensorflow-gpu through Conda; it gives me the following output:
conda install tensorflow-gpu
Collecting package metadata (current_repodata.json): done
Solving environment: done
## Package Plan ##
environment location: /home/psychotechnopath/anaconda3/envs/DeepLearning3.6
added / updated specs:
- tensorflow-gpu
The following packages will be downloaded:
package | build
---------------------------|-----------------
_tflow_select-2.1.0 | gpu 2 KB
cudatoolkit-10.1.243 | h6bb024c_0 347.4 MB
cudnn-7.6.5 | cuda10.1_0 179.9 MB
cupti-10.1.168 | 0 1.4 MB
tensorflow-2.1.0 |gpu_py36h2e5cdaa_0 4 KB
tensorflow-base-2.1.0 |gpu_py36h6c5654b_0 155.9 MB
tensorflow-gpu-2.1.0 | h0d30ee6_0 3 KB
------------------------------------------------------------
Total: 684.7 MB
The following NEW packages will be INSTALLED:
cudatoolkit pkgs/main/linux-64::cudatoolkit-10.1.243-h6bb024c_0
cudnn pkgs/main/linux-64::cudnn-7.6.5-cuda10.1_0
cupti pkgs/main/linux-64::cupti-10.1.168-0
tensorflow-gpu pkgs/main/linux-64::tensorflow-gpu-2.1.0-h0d30ee6_0
I see that installing tensorflow-gpu automatically triggers the installation of the cudatoolkit and cudnn. Does this mean that I no longer need to install CUDA and CUDNN manually anymore to be able to use tensorflow-gpu? Where does this conda installation of CUDA reside?
I first installed CUDA and CuDNN the old way (e.g. by following these installation instructions: https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html )
And then I noticed that tensorflow-gpu was also installing cuda and cudnn
Do i now have two versions of CUDA/CuDNN installed and how do I check this?
Do i now have two versions of CUDA installed and how do I check this?
No.
conda installs the bare minimum redistributable library components required to support the CUDA accelerated packages they offer. The package name cudatoolkit is a complete misnomer. It is nothing of the sort. Even though it is now greatly expanded in scope from what it used to be (literally 5 files -- I think at some point they must have gotten a licensing deal from NVIDIA because some of this wasn't/isn't on the official "freely redistributable" list AFAIK), it still is basically just a handful of libraries.
You can check this for yourself:
cat /opt/miniconda3/conda-meta/cudatoolkit-10.1.168-0.json
{
"build": "0",
"build_number": 0,
"channel": "https://repo.anaconda.com/pkgs/main/linux-64",
"constrains": [],
"depends": [],
"extracted_package_dir": "/opt/miniconda3/pkgs/cudatoolkit-10.1.168-0",
"features": "",
"files": [
"lib/cudatoolkit_config.yaml",
"lib/libcublas.so",
"lib/libcublas.so.10",
"lib/libcublas.so.10.2.0.168",
"lib/libcublasLt.so",
"lib/libcublasLt.so.10",
"lib/libcublasLt.so.10.2.0.168",
"lib/libcudart.so",
"lib/libcudart.so.10.1",
"lib/libcudart.so.10.1.168",
"lib/libcufft.so",
"lib/libcufft.so.10",
"lib/libcufft.so.10.1.168",
"lib/libcufftw.so",
"lib/libcufftw.so.10",
"lib/libcufftw.so.10.1.168",
"lib/libcurand.so",
"lib/libcurand.so.10",
"lib/libcurand.so.10.1.168",
"lib/libcusolver.so",
"lib/libcusolver.so.10",
"lib/libcusolver.so.10.1.168",
"lib/libcusparse.so",
"lib/libcusparse.so.10",
"lib/libcusparse.so.10.1.168",
"lib/libdevice.10.bc",
"lib/libnppc.so",
"lib/libnppc.so.10",
"lib/libnppc.so.10.1.168",
"lib/libnppial.so",
"lib/libnppial.so.10",
"lib/libnppial.so.10.1.168",
"lib/libnppicc.so",
"lib/libnppicc.so.10",
"lib/libnppicc.so.10.1.168",
"lib/libnppicom.so",
"lib/libnppicom.so.10",
"lib/libnppicom.so.10.1.168",
"lib/libnppidei.so",
"lib/libnppidei.so.10",
"lib/libnppidei.so.10.1.168",
"lib/libnppif.so",
"lib/libnppif.so.10",
"lib/libnppif.so.10.1.168",
"lib/libnppig.so",
"lib/libnppig.so.10",
"lib/libnppig.so.10.1.168",
"lib/libnppim.so",
"lib/libnppim.so.10",
"lib/libnppim.so.10.1.168",
"lib/libnppist.so",
"lib/libnppist.so.10",
"lib/libnppist.so.10.1.168",
"lib/libnppisu.so",
"lib/libnppisu.so.10",
"lib/libnppisu.so.10.1.168",
"lib/libnppitc.so",
"lib/libnppitc.so.10",
"lib/libnppitc.so.10.1.168",
"lib/libnpps.so",
"lib/libnpps.so.10",
"lib/libnpps.so.10.1.168",
"lib/libnvToolsExt.so",
"lib/libnvToolsExt.so.1",
"lib/libnvToolsExt.so.1.0.0",
"lib/libnvblas.so",
"lib/libnvblas.so.10",
"lib/libnvblas.so.10.2.0.168",
"lib/libnvgraph.so",
"lib/libnvgraph.so.10",
"lib/libnvgraph.so.10.1.168",
"lib/libnvjpeg.so",
"lib/libnvjpeg.so.10",
"lib/libnvjpeg.so.10.1.168",
"lib/libnvrtc-builtins.so",
"lib/libnvrtc-builtins.so.10.1",
"lib/libnvrtc-builtins.so.10.1.168",
"lib/libnvrtc.so",
"lib/libnvrtc.so.10.1",
"lib/libnvrtc.so.10.1.168",
"lib/libnvvm.so",
"lib/libnvvm.so.3",
"lib/libnvvm.so.3.3.0"
]
.....
i.e. what you get is (keeping in mind most of those "files" above are just symlinks)
CUBLAS runtime
The CUDA runtime library
CUFFT runtime
CUrand runtime
CUsparse rutime
CUsolver runtime
NPP runtime
nvblas runtime
NVTX runtime
NVgraph runtime
NVjpeg runtime
NVRTC/NVVM runtime
The CUDNN package that conda installs is the redistributable binary distribution which is identical to what NVIDIA distribute -- which is exactly two files, a header file and a library.
You would still require a supported NVIDIA driver installation to make the tensorflow which conda installs work.
If you want to actually compile and build CUDA code, you need to install a separate CUDA toolkit which contains all the the development components which conda deliberately omits from their distribution.
I am trying to install Rakudo on my Android with armv7l processor architecture using Termux.
I tried compiling from source, but it didn't work. Then someone pointed out the Termux user its-pointless and his package for this, but that package does not work on my phone.
How can I run Raku on my phone while it is offline? I'm open to solutions not using Termux.
Termux on SSH results:
u0_a74#localhost ~/rakudo [100]> pkg show rakudo -a
Package: rakudo Version: 2020.05 Maintainer: Termux members #termux
Installed-Size: 37.7 MB Depends: moarvm Homepage: https://rakudo.org
Download-Size: 5062 kB APT-Manual-Installed: yes APT-Sources:
https://its-pointless.github.io/files/24 termux/extras arm Packages
Description: Perl 6 implementation on top of Moar virtual machine
Package: rakudo Version: 2020.01-1 Maintainer: Fredrik Fornwall
#fornwall Installed-Size: 93.1 MB Depends: moarvm Homepage:
https://rakudo.org Download-Size: 10.9 MB APT-Sources:
https://its-pointless.github.io/files/24 termux/extras arm Packages
Description: Perl 6 implementation on top of Moar virtual machine
u0_a74#localhost ~/rakudo> raku
CANNOT LINK EXECUTABLE "raku": cannot locate symbol "ffi_type_double"
referenced by "/data/data/com.termux/files/usr/lib/libmoar.so"...
u0_a74#localhost ~/rakudo> raku --version
CANNOT LINK EXECUTABLE "raku": cannot locate symbol "ffi_type_double"
referenced by "/data/data/com.termux/files/usr/lib/libmoar.so"...
u0_a74#localhost ~/rakudo> raku --help
CANNOT LINK EXECUTABLE "raku": cannot locate symbol "ffi_type_double"
referenced by "/data/data/com.termux/files/usr/lib/libmoar.so"...
u0_a74#localhost ~/rakudo> uname -a
Linux localhost 3.4.42-g3d041de #1 SMP PREEMPT Sat Dec 24 19:56:29 PST
2016 armv7l Android
Does it have to be on Termux? I have successfully installed Raku on Android via UserLand, using Debian SSH. sudo apt-get install rakudo works.
I am trying to address the issue in the title:
Loaded runtime CuDNN library: 7.1.2 but source was compiled with: 7.6.0. CuDNN library major and minor version needs to match or have higher minor version in case of CuDNN 7.0 or later version
I have read several other posts (example: Loaded runtime CuDNN library: 5005 (compatibility version 5000) but source was compiled with 5103 (compatibility version 5100))
that basically tells me that my machine has CuDNN 7.1.2 but I need 7.6.0. The answer is then to download and install 7.6.*
the only issue is that I thought I did that by following the instructions on nvidia archive (https://developer.nvidia.com/rdp/cudnn-archive)
and if I go to /usr/local/cuda/include and read cudnn.h it shows
#if !defined(CUDNN_H_)
#define CUDNN_H_
#define CUDNN_MAJOR 7
#define CUDNN_MINOR 6
#define CUDNN_PATCHLEVEL 4
Currently I have CUDA-10.0, 10.1, and 10.2 installed with my .bashrc set to 10.0 (although nvcc --version states I have cuda 9.1 --another issue I cant seem to fix).
Any suggestions? I have been trying to tackle this for days but no luck.
UPDATE:
Here are the paths I have
export PATH=$PATH:/usr/local/cuda-10.0/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-10.0/lib64
export CUDA_HOME=/usr/local/cuda
Before this is closed could you help with either suggesting a proper path to set or to find old cudnn please?
I hit a very similar error:
Loaded runtime CuDNN library: 7.1.4 but source was compiled with: 7.6.5. CuDNN library major and minor version needs to match or have higher minor version in case of CuDNN 7.0 or later version. If using a binary install, upgrade your CuDNN library. If building from sources, make sure the library loaded at runtime is compatible with the version specified during compile configuration.
and tracked it down to accidentally having an older CuDNN in my ldconfig:
$ sudo ldconfig -p | grep libcudnn
libcudnn.so.7 (libc6,x86-64) => /usr/local/cuda-9.0/lib64/libcudnn.so.7
libcudnn.so.7 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libcudnn.so.7
libcudnn.so (libc6,x86-64) => /usr/local/cuda-9.0/lib64/libcudnn.so
libcudnn.so (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libcudnn.so
The libcudnn.so.7 file in the cuda-9.0 directory was pointing to the older version:
ls -alh /usr/local/cuda-9.0/lib64/libcudnn.so.7
lrwxrwxrwx 1 root root 17 Dec 16 2018 /usr/local/cuda-9.0/lib64/libcudnn.so.7 -> libcudnn.so.7.1.4
But I had compiled tensorflow against the newer version:
ls -alh /usr/lib/x86_64-linux-gnu/libcudnn.so.7
lrwxrwxrwx 1 root root 17 Oct 27 2019 /usr/lib/x86_64-linux-gnu/libcudnn.so.7 -> libcudnn.so.7.6.5
Since ldconfig uses /etc/ld.so.conf to determine where to look for libraries (I guess in conjunction with LD_LIBRARY_PATH), I checked it and it showed:
include /etc/ld.so.conf.d/*.conf
When I listed the files in that directory, I spotted the problem file and removed it:
$ cat /etc/ld.so.conf.d/cuda9.conf
/usr/local/cuda-9.0/lib64
$ sudo rm /etc/ld.so.conf.d/cuda9.conf
After that I re-ran ldconfig to reload the config, and then everything worked as expected and the error disappeared.
I am unable to import matplotlib._png:
import matplotlib._png as _png ImportError:
/home/james/opt/python/virtualenvs/work/lib/python2.7/site-packages/matplotlib-1.3.x-py2.7-linux-x86_64.egg/matplotlib/_png.so:
undefined symbol: png_set_longjmp_fn
This error prevents me from running import pylab (sincce this ultimately imports matplotlib._png).
I installed matplotlib from source, and made sure to add the path with local installations (/home/james/local) to basedir in setupext.py before running python setup.py install.
REQUIRED DEPENDENCIES AND EXTENSIONS
numpy: yes [version 1.7.1]
dateutil: yes [using dateutil version 2.1]
tornado: yes [using tornado version 3.0.1]
pyparsing: yes [using pyparsing version 1.5.7]
pycxx: yes [Couldn't import. Using local copy.]
libagg: yes [pkg-config information for 'libagg' could not
be found Using local copy.]
freetype: yes [version 16.0.10]
png: yes [version 1.2.10]
My research so far:
As can be seen above, matplotlib seems to find version 1.2.10 even though the version that I have under /home/james/local is 1.6.2:
$ find . -iname '*libpng*'
./libpng16.so.16.1.0
./libpng16.so
./libpng16.so.16
./libpng16.a
./libpng.a
./libpng.so
./libpng16.la
./pkgconfig/libpng.pc
./pkgconfig/libpng16.pc
./libpng.la
More specifically, I modified the following line in setupext.py with:
return basedir_map.get(sys.platform, ['/home/james/local', '/usr/local', '/usr'])
but matplotlib seems to have found the system version:
$ locate libpng
/usr/lib/libpng.so
/usr/lib/libpng.so.3
/usr/lib/libpng.so.3.10.0
/usr/lib/libpng12.a
/usr/lib/libpng12.so
/usr/lib/libpng12.so.0
/usr/lib/libpng12.so.0.10.0
Could this be the problem? Why am I unable to import matplotlib._png?
Update:
Looking at setupext.py, it looks like python setup install queries pkg-config through the SetupPackage method _check_for_pkg_config to determine the version of libpng I have installed. It turns out that pkg-config is returning the system installation:
$ pkg-config --libs libpng
-lpng12
even though I have updated basedir in matplotlib's setupext.py, and LD_LIBRARY_PATH to make them point to the the more recent version of libpng that I have locally installed.
Any ideas on how to have pkg-config return the right version?
It's a pkg-config issue; matplotlib's installation is (unfortunately, or perhaps not) relying too much on pkg-config's output.
Assuming you have build libpng the normal way, there should be a pkgconfig subdirectory in your /home/james/local/lib, which contains libpng.pc (and libpng16.pc). When setupext.py runs pkg-config, the latter should of course try and pick up the correct .pc file for libpng. For that, use the PKG_CONFIG_PATH variable and point it to the pkgconfig subdirectory:
$ export PKG_CONFIG_PATH=/home/james/local/lib/pkgconfig
Then, install matplotlib again, and see that it now finds the correct libpng version:
$ python setup.py build
basedirlist is: ['/usr/local', '/usr']
============================================================================
BUILDING MATPLOTLIB
matplotlib: 1.1.0
python: 2.7.4 (default, Apr 8 2013, 16:36:47) [GCC 4.4.5]
platform: linux2
REQUIRED DEPENDENCIES
numpy: 1.7.0
freetype2: 12.0.6
OPTIONAL BACKEND DEPENDENCIES
libpng: 1.6.1
Tkinter: Tkinter: 81008, Tk: 8.4, Tcl: 8.4
(For me, with a different PKG_CONFIG_PATH of course. Yes, I may want to upgrade some dependencies.)
Note that I didn't even alter basedirlist; it's just at its default.
In case pkg-config fails to now pick up some other package, just add more directories to PKG_CONFIG_PATH with colons in between. But I guess this should be enough.
Try
export LD_LIBRARY_PATH=/home/james/local/lib
and then execute Matplotlib... that would point matplotlib to your local version.
Presenting symptom: autogen disables the build of slideshowshuffle and pythonconsole, reporting "no python support." Platform is Ubuntu 9.04, Jaunty Jackalope; Gnome 2.26.1.
Log extract:
checking for a Python interpreter with version >= 2.3... python
checking for python... /usr/bin/python
checking for python version... 2.6
checking for python platform... linux2
checking for python script directory... ${prefix}/lib/python2.6/site-packages
checking for python extension module directory... ${exec_prefix}/lib/python2.6/site-packages
checking for PYGTK... no
configure: WARNING: Python not found, disabling python support
Evidence that both python and pygtk are installed:
Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41)
[GCC 4.3.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pygtk
>>>
I note the capitalization of PYGTK, which is common for environment variables. There is no PYGTK environment variable.
Your search - "PYGTK environment
variable" - did not match any
documents.
A grep for PYGTK in the tree rooted from /usr/share/doc/python-gtk2-doc/html returned no rows.
Try installing "python-gtk2-dev" package. You can make sure you have it with
pkg-config --list-all | grep pygtk-2.0
I think, the one you're using from python is "python-gtk2".