When I install tensorflow-gpu through Conda; it gives me the following output:
conda install tensorflow-gpu
Collecting package metadata (current_repodata.json): done
Solving environment: done
## Package Plan ##
environment location: /home/psychotechnopath/anaconda3/envs/DeepLearning3.6
added / updated specs:
- tensorflow-gpu
The following packages will be downloaded:
package | build
---------------------------|-----------------
_tflow_select-2.1.0 | gpu 2 KB
cudatoolkit-10.1.243 | h6bb024c_0 347.4 MB
cudnn-7.6.5 | cuda10.1_0 179.9 MB
cupti-10.1.168 | 0 1.4 MB
tensorflow-2.1.0 |gpu_py36h2e5cdaa_0 4 KB
tensorflow-base-2.1.0 |gpu_py36h6c5654b_0 155.9 MB
tensorflow-gpu-2.1.0 | h0d30ee6_0 3 KB
------------------------------------------------------------
Total: 684.7 MB
The following NEW packages will be INSTALLED:
cudatoolkit pkgs/main/linux-64::cudatoolkit-10.1.243-h6bb024c_0
cudnn pkgs/main/linux-64::cudnn-7.6.5-cuda10.1_0
cupti pkgs/main/linux-64::cupti-10.1.168-0
tensorflow-gpu pkgs/main/linux-64::tensorflow-gpu-2.1.0-h0d30ee6_0
I see that installing tensorflow-gpu automatically triggers the installation of the cudatoolkit and cudnn. Does this mean that I no longer need to install CUDA and CUDNN manually anymore to be able to use tensorflow-gpu? Where does this conda installation of CUDA reside?
I first installed CUDA and CuDNN the old way (e.g. by following these installation instructions: https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html )
And then I noticed that tensorflow-gpu was also installing cuda and cudnn
Do i now have two versions of CUDA/CuDNN installed and how do I check this?
Do i now have two versions of CUDA installed and how do I check this?
No.
conda installs the bare minimum redistributable library components required to support the CUDA accelerated packages they offer. The package name cudatoolkit is a complete misnomer. It is nothing of the sort. Even though it is now greatly expanded in scope from what it used to be (literally 5 files -- I think at some point they must have gotten a licensing deal from NVIDIA because some of this wasn't/isn't on the official "freely redistributable" list AFAIK), it still is basically just a handful of libraries.
You can check this for yourself:
cat /opt/miniconda3/conda-meta/cudatoolkit-10.1.168-0.json
{
"build": "0",
"build_number": 0,
"channel": "https://repo.anaconda.com/pkgs/main/linux-64",
"constrains": [],
"depends": [],
"extracted_package_dir": "/opt/miniconda3/pkgs/cudatoolkit-10.1.168-0",
"features": "",
"files": [
"lib/cudatoolkit_config.yaml",
"lib/libcublas.so",
"lib/libcublas.so.10",
"lib/libcublas.so.10.2.0.168",
"lib/libcublasLt.so",
"lib/libcublasLt.so.10",
"lib/libcublasLt.so.10.2.0.168",
"lib/libcudart.so",
"lib/libcudart.so.10.1",
"lib/libcudart.so.10.1.168",
"lib/libcufft.so",
"lib/libcufft.so.10",
"lib/libcufft.so.10.1.168",
"lib/libcufftw.so",
"lib/libcufftw.so.10",
"lib/libcufftw.so.10.1.168",
"lib/libcurand.so",
"lib/libcurand.so.10",
"lib/libcurand.so.10.1.168",
"lib/libcusolver.so",
"lib/libcusolver.so.10",
"lib/libcusolver.so.10.1.168",
"lib/libcusparse.so",
"lib/libcusparse.so.10",
"lib/libcusparse.so.10.1.168",
"lib/libdevice.10.bc",
"lib/libnppc.so",
"lib/libnppc.so.10",
"lib/libnppc.so.10.1.168",
"lib/libnppial.so",
"lib/libnppial.so.10",
"lib/libnppial.so.10.1.168",
"lib/libnppicc.so",
"lib/libnppicc.so.10",
"lib/libnppicc.so.10.1.168",
"lib/libnppicom.so",
"lib/libnppicom.so.10",
"lib/libnppicom.so.10.1.168",
"lib/libnppidei.so",
"lib/libnppidei.so.10",
"lib/libnppidei.so.10.1.168",
"lib/libnppif.so",
"lib/libnppif.so.10",
"lib/libnppif.so.10.1.168",
"lib/libnppig.so",
"lib/libnppig.so.10",
"lib/libnppig.so.10.1.168",
"lib/libnppim.so",
"lib/libnppim.so.10",
"lib/libnppim.so.10.1.168",
"lib/libnppist.so",
"lib/libnppist.so.10",
"lib/libnppist.so.10.1.168",
"lib/libnppisu.so",
"lib/libnppisu.so.10",
"lib/libnppisu.so.10.1.168",
"lib/libnppitc.so",
"lib/libnppitc.so.10",
"lib/libnppitc.so.10.1.168",
"lib/libnpps.so",
"lib/libnpps.so.10",
"lib/libnpps.so.10.1.168",
"lib/libnvToolsExt.so",
"lib/libnvToolsExt.so.1",
"lib/libnvToolsExt.so.1.0.0",
"lib/libnvblas.so",
"lib/libnvblas.so.10",
"lib/libnvblas.so.10.2.0.168",
"lib/libnvgraph.so",
"lib/libnvgraph.so.10",
"lib/libnvgraph.so.10.1.168",
"lib/libnvjpeg.so",
"lib/libnvjpeg.so.10",
"lib/libnvjpeg.so.10.1.168",
"lib/libnvrtc-builtins.so",
"lib/libnvrtc-builtins.so.10.1",
"lib/libnvrtc-builtins.so.10.1.168",
"lib/libnvrtc.so",
"lib/libnvrtc.so.10.1",
"lib/libnvrtc.so.10.1.168",
"lib/libnvvm.so",
"lib/libnvvm.so.3",
"lib/libnvvm.so.3.3.0"
]
.....
i.e. what you get is (keeping in mind most of those "files" above are just symlinks)
CUBLAS runtime
The CUDA runtime library
CUFFT runtime
CUrand runtime
CUsparse rutime
CUsolver runtime
NPP runtime
nvblas runtime
NVTX runtime
NVgraph runtime
NVjpeg runtime
NVRTC/NVVM runtime
The CUDNN package that conda installs is the redistributable binary distribution which is identical to what NVIDIA distribute -- which is exactly two files, a header file and a library.
You would still require a supported NVIDIA driver installation to make the tensorflow which conda installs work.
If you want to actually compile and build CUDA code, you need to install a separate CUDA toolkit which contains all the the development components which conda deliberately omits from their distribution.
I'm trying to run a Vulkan-based graphical application on a headless Ubuntu 19.10 virtual machine, via xvfb.
Starting from a bare Ubuntu 19.10 image (created using lxc), I prep the machine as follows:
> sudo apt update
> sudo apt install -y xvfb mesa-vulkan-drivers vulkan-tools
I then run the following two commands:
In Terminal 1: Start XVFB
Xvfb :1 -screen 0 1024x768x24
In Terminal 2: Run vulkaninfo
> DISPLAY=:1 vulkaninfo
==========
VULKANINFO
==========
Vulkan Instance Version: 1.1.114
/build/vulkan-tools-IZAxVX/vulkan-tools-1.1.114.0+dfsg1/vulkaninfo/vulkaninfo.c:5884:
failed with VK_ERROR_INITIALIZATION_FAILED
(Running my own custom Vulkan application yields a similar failure.)
From what I understand, Xvfb depends on mesa, and I'm using Mesa 19.2.1
And, based on the Mesa 19.2.1 release notes, it is supposed to support Vulkan:
"Mesa 19.2.1 implements the Vulkan 1.1 API, but the version reported by the apiVersion property of the VkPhysicalDeviceProperties struct depends on the particular driver being used."
Question: Is it reasonable to expect that vulkan apps would work with xvfb on Ubuntu 19.10, especially since the latest mesa releases claim to support Vulkan?
Versions of relevant some packages that I'm using:
> apt-cache show xvfb | grep Version
Version: 2:1.20.5+git20191008-0ubuntu1
> apt-cache show mesa-vulkan-drivers | grep Version
Version: 19.2.1-1ubuntu1
> apt-cache show libvulkan1 | grep Version
Version: 1.1.114.0-1
> apt-cache show vulkan-tools | grep Version
Version: 1.1.114.0+dfsg1-1
I'm running Mac OS 10.13.5 and struggling to install Matplotlib on Python 3.7 any help is greatly appreciated.
Here is the error that I get when I use pip3 install matplotlib:
BUILDING MATPLOTLIB
matplotlib: yes [2.2.2]
python: yes [3.7.0 (v3.7.0:1bf9cc5093, Jun 26 2018,
23:26:24) [Clang 6.0 (clang-600.0.57)]]
platform: yes [darwin]
REQUIRED DEPENDENCIES AND EXTENSIONS
numpy: yes [version 1.14.5]
install_requires: yes [handled by setuptools]
libagg: yes [Requires patches that have not been merged
upstream. Using local copy.]
freetype: no [The C/C++ header for freetype2 (ft2build.h)
could not be found. You may need to install the
development package.]
png: yes [version 1.6.34]
However I have already installed and linked freetype via Homebrew:
Ocean-Gypsys-MacBook-Pro:~ Aysegul$ more /usr/X11/lib/pkgconfig/freetype2.pc
prefix=/opt/X11
exec_prefix=/opt/X11
libdir=/opt/X11/lib
includedir=/opt/X11/include
Name: FreeType 2
URL: http://freetype.org
Description: A free, high-quality, and portable font engine.
Version: 18.6.12
Requires:
Requires.private:
Libs: -L${libdir} -lfreetype
Libs.private: -lz -lbz2
Cflags: -I${includedir}/freetype2
/usr/X11/lib/pkgconfig/freetype2.pc (END)
I know intel has already written a document for numpy build with mkl. but my machine CPU is power8(IBM) (OS: centos), I download in this link: https://registrationcenter.intel.com/en/products/postregistration/?sn=3VGW-J93Z886P&EmailID=mac16%40tsinghua.edu.cn&Sequence=2115993&dnld=t
but when I run install.sh
[root#power8 intel_mkl]# sh install.sh
install.sh: line 50: [: -lt: unary operator expected
install.sh: line 53: [: -eq: unary operator expected
The IA-32 architecture host installation is no longer supported.
The product cannot be installed on this system.
Please refer to product documentation for more information.
Does intel mkl support power8 machine? and how to build numpy link against mkl in power8 machine exactly?
I'm trying to compile the latest stable (2.8.19) version of Redis. Build is successfull as well as all tests, but unexpectedly server runs on 32bit arch.
Log entries:
# Warning: 32 bit instance detected but no memory limit set. Setting 3 GB maxmemory limit with 'noeviction' policy now.
Redis 2.8.19 (00000000/0) 32 bit
Running in stand alone mode
Port: 6582
PID: 2381
Redis-cli INFO display arch_bits:32. Previous instance (version 2.4.6) works well on arch_bits 64, but I don't know which way it was installed.
OS version info:
root:~# uname -a
Linux localhost 2.6.32-5-amd64 #1 SMP Tue Mar 8 22:49:26 UTC 2011 x86_64 GNU/Linux
root:~# lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux 6.0.1 (squeeze)
Release: 6.0.1
Codename: squeeze
root:~# arch
x86_64
What are the ways to fix this issue and run latest redis as 64bit?
UPD
Despite above commands output, dpkg --print-architecture returns i386 and all packages in system are all or i386. Only redis-server 2.4.*, installed as a package, is strangely ia64.
What can I do in this situation? The server was setup long time ago by another person, and I is still too newbie in Unix.
It seems, my server needs a full migration from 32 to 64-bit architeture.
Current task solved by downloading compiled 64-bit DEB-package and installing it manually.