I understand that Tensorflow requires (for GPU computation) a GPU with Nvidia Compute Capability >= 3.0. There are many such GPUs to choose from. The gaming oriented GPUs, e.g. GeForce models, are much less expensive than the compute-oriented models, e.g. Tesla. My limited undertanding is that the compute-oriented models may lack video output (not needed for computation) and that the gaming models may be doing 32-bit math instead of 64. Assuming that Tensorflow uses (or prefers) 64-bit, does this mean that the gaming models will not work or will produce deficient results if used with Tensorflow? What attributes should one look for in choosing a GPU to use with Tensorflow?
The GPU-enabled version of TensorFlow has the following requirements:
64-bit Linux
Python 2.7
NVIDIA CUDA® 7.5 (CUDA 8.0 required for Pascal GPUs)
NVIDIA cuDNN v4.0 (minimum) or v5.1 (recommended)
TensorFlow GPU support requires having a GPU card with NVidia Compute Capability >= 3.0. Supported cards include but are not limited to:
NVidia Titan
NVidia Titan X
NVidia K20
NVidia K40
You can see their official docs Tensorflow GPU support
Gaming GPUs can work quite well. You want a very recent GPU with lots of memory and CUDA cores. Most people training neural nets these days on GPU use 32 bit floats.
Related
I recently ran in to a bit of a glitch where my detection model running on two different GPUs (a Quadro RTX4000 and RTX A4000) on two different systems utilize the GPU differently.
The model uses only 0.2% of GPU on the Quadro system and uses anywhere from 50 to 70% on the A4000 machine. I am curious about why this is happening. The rest of the hardware on both the machines are the same.
Additional information: The model uses a 3D convolution and is built on tensorflow.
Looks like the Quadro RTX4000 does not use GPU.
The method tf.test.is_gpu_available() is deprecated and can still return True although the GPU is not used.
The correct way to verify the usage of the GPU availability + usage is to check the output of the snippet:
tf.config.list_physical_devices('GPU')
On the Quadro machine you should also run (in terminal):
watch -n 1 nvidia-smi
to see real-time the amount of GPU memory used.
Okay, so I've worked on a bunch of Deep Learning projects and internships now and I've never had to do heavy training. But lately I've been thinking of doing some Transfer Learning for which I'll need to run my code on a GPU. Now I have a system with Windows 10 and a dedicated NVIDIA GeForce 940M GPU. I've been doing a lot of research online, but I'm still a bit confused. I haven't installed the NVIDIA Cuda Toolkit or cuDNN or tensorflow-gpu on my system yet. I currently use tensorflow and pytorch to train my DL models. Here are my queries -
When I define a tensor in tf or pytorch, it is a cpu tensor by default. So, all the training I've been doing so far has been on the CPU. So, if I make sure to install the correct versions of Cuda and cuDNN and tensorflow-gpu (specifically for tensorflow), I can run my models on my GPU using tf-gpu and pytorch and that's it? (I'm aware of the torch.cuda.is_available() in pytorch to ensure pytorch can access my GPU and the device_lib module in tf to check if my gpu is visible to tensorflow)(I'm also aware of the fact that tf doesnt support all Nvidia GPUs)
Why does tf have a separate module for GPU support? PyTorch doesnt seem to have that and all you need to do is cast your tensor from cpu() to cuda() to switch between them.
Why install cuDNN? I know it is a high-level API CUDA built for support to train Deep Neural Nets on the GPU. But do tf-gpu and torch use these in the backend while training on the gpu?
After tf == 1.15, did they combine CPU and GPU support all into one package?
First of all unfortunately 940M is a kinda weak GPU for training. I suggest you use Google colab for faster training but of course, it would be faster than the CPU. So here my answers to your four questions.
1-) Yes if you install the requirements correctly, then you can run on GPU. You can manually place your data to your GPU as well. You can check implementations on TensorFlow. In PyTorch, you should specify the device that you want to use. As you said you should do device = torch.device("cuda" if args.cuda else "cpu") then for models and data you should always call .to(device) Then it will automatically use GPU if available.
2-) PyTorch also needs extra installation (module) for GPU support. However, with recent updates both TF and PyTorch are easy to use for GPU compatible code.
3-) Both Tensorflow and PyTorch is based on cuDNN. You can use them without cuDNN but as far as I know, it hurts the performance but I'm not sure about this topic.
4-) No they are still different packages. tensorflow-gpu==1.15 and tensorflow==1.15 what they did with tf2, was making the tensorflow more like Keras. So it is more simplified then 1.15 or before.
Rest was already answered by regarding 3) cudNN optimizes layer and such operations on hardware level and those implementations are pure black magic. It is incredibly hard to write CUDA code that properly utilizes your GPU (how load data into the GPU, how to actually perform them using matrices etc. )
how do i find out if tensorflow uses the gpu?
when I check the GPU in the task manager it says that it is 1% full. I find that a little bit, but I do not know whether the display may also be incorrect for the information.
I find the calculation too fast for only CPU, but actually too slow for GPU ...
is installed
tensorflow and tensorflow-gpu with version 1.15
This code will confirm that tensorflow using GPU or CPU
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
I am developing robot with computer vision on Raspberry Pi 3 with Tensorflow. Can I use gpu for deep learning on raspberry pi 3?
UPDATE :
Here is an alternative backend for Keras called plaidml that is not Tensorflow. The major selling feature is a speedup on non-Nvidia graphics cards. It still isn't Tensorflow, but it may be a viable option.
HERE IS MY OLD ANSER PRIOR TO 2018-09 :
The short answer is no, it isn't possible at this time since Tensorflow leverages Nvidia drivers to power Nvidia GPUs and Raspberry Pi does not have Nvidia hardware.
One of two things have to change for you to have access to GPUs for a small form computing, Tensorflow has to support OpenCl (tracked here), or you have to switch platforms to something that has a Nvidia GPU like this
Sorry to be the bringer of bad news.
Reading :
https://www.tensorflow.org/versions/r0.10/resources/faq.html it states :
Does TensorFlow make use of all the devices (GPUs and CPUs) available
on my machine?
TensorFlow supports multiple GPUs and CPUs. See the how-to
documentation on using GPUs with TensorFlow for details of how
TensorFlow assigns operations to devices, and the CIFAR-10 tutorial
for an example model that uses multiple GPUs.
Note that TensorFlow only uses GPU devices with a compute capability
greater than 3.5.
Does this mean Tensorflow can automatically make use of all CPU's on given machine or does it ned to be explicitly configured ?
CPUs are used via a "device" which is just a threadpool. You can control the number of threads if you feel like you need more:
sess = tf.Session(config=tf.ConfigProto(
intra_op_parallelism_threads=NUM_THREADS))