I have the latest tensorflow installed. When running
print(get_available_devices())
I get
['/device:CPU:0', '/device:GPU:0']
But then every time I run my model the kernel dies. However if I use
import os
os.environ["CUDA_VISIBLE_DEVICES"] = "-1"
to disable GPU use, the model runs just fine on the CPU.
Related
I made neural nets with tensorflow
but tnesorflow-gpu is slower than cpu!
total running(training) time is 130sec in tensorflow 2.1
and 330sec in tensorflow-gpu 2.1
My CPU is i7-7th gen and GPU is geforce-930M(laptop environments)
It's because my GPU is slower than CPU?
If so, Can I setup to run GPU automatically in appropriate situation only?
(CUDA environments seems appropriately setup, also I manually checked that tensorflow 2.1 is using CPU only and tensorflow-gpu 2.1 is using both CPU and GPU.)
update : My neural network's size is 64 x 32 x 16 x 1(maybe not fitting in parallel execution) and in tensorflow 2.1, I turn off GPU by this commands as follows.
import os
os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"] = ""
As of TensorFlow 2.1, the GPU support is also available in the tensorflow package, not only tensorflow-gpu; if you use import tensorflow as tf it defaults to GPU usage if it finds one; I would personally first uninstall tensorflow-gpu and leave only the plain tensorflow package.
I am training a CNN on GCP's notebook using a Tesla V100. I've trained a simple yolo on my own custom data and it was pretty fast but not very accurate. So, I decided to write my own code from scratch to solve the specific aspects of the problem that I want to tackle.
I have tried to run my code on Google Colab prior to GCP, and it went well. Tensorflow detects the GPU and is able to use it whether it was a Tesla K80 or T4.
from tensorflow.python.client import device_lib
print(device_lib.list_local_devices())
tf.test.is_gpu_available() #>>> True
My problem is that, this same function returns a False on GCP notebook, as if Tensorflow is unable to use the GPU it detected on GCP VM. I don't know of any command that forces Tensorflow to use the GPU over CPU, since it does that automatically.
I have already tried to install or uninstall and then install some versions of tensorflow, tensorflow-gpu and tf-nightly-gpu (1.13 and 2.0dev for instance) but it yielded nothing.
output of nvidia-smi
Have you tried using GCP's AI Platform Notebooks instead? They offer VMs that are pre-configured with Tensorflow and have all required GPU drivers installed.
I am setting up Multi-GPU-model using Tensorflow on Ubuntu 18.04 LTS desktop. I run the code on 4 NVIDIA RTX 2080 TI and compile the model using the CPU. The same code functions on Windows 10 OS, in case of the Ubuntu, it crashes and the system goes to reboot. Where do I check or change? is it is OS/code?
with tf.device("/cpu:0"):
model = create_image_model()
# make the model parallel
model = multi_gpu_model(model, gpus=G)
Try adding tf.ConfigProto(allow_soft_placement=True) to your session or estimator config. In the unlikely event this doesn't help, try switching IOMMU off in UEFI.
https://www.tensorflow.org/guide/using_gpu
I have previously asked if it is possible to run tensor flow with gpu support on a cpu. I was told that it is possible and the basic code to switch which device I want to use but not how to get the initial code working on a computer that doesn't have a gpu at all. For example I would like to train on a computer that has a NVidia gpu but program on a laptop that only has a cpu. How would I go about doing this? I have tried just writing the code as normal but it crashes before I can even switch which device I want to use. I am using Python on Linux.
This thread might be helpful: Tensorflow: ImportError: libcusolver.so.8.0: cannot open shared object file: No such file or directory
I've tried to import tensorflow with tensorflow-gpu loaded in the uni's HPC login node, which does not have GPUs. It works well. I don't have Nvidia GPU in my laptop, so I never go through the installation process. But I think the cause is it cannot find relevant libraries of CUDA, cuDNN.
But, why don't you just use cpu version? As #Finbarr Timbers mentioned, you still can run a model in a computer with GPU.
What errors are you getting? It is very possible to train on a GPU but develop on a CPU- many people do it, including myself. In fact, Tensorflow will automatically put your code on a GPU if possible.
If you add the following code to your model, you can see which devices are being used:
# Creates a session with log_device_placement set to True.
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
This should change when you run your model on a computer with a GPU.
I am using Windows 7. After i tested my GPU in tensorflow, which was awkwardly slowly on a already tested model on cpu, i switched to cpu with:
tf.device("/cpu:0")
I was assuming that i can switch back to gpu with:
tf.device("/gpu:0")
However i got the following error message from windows, when i try to rerun with this configuration:
The device "NVIDIA Quadro M2000M" is not exchange device and can not be removed.
With "nvida-smi" i looked for my GPU, but the system said the GPU is not there.
I restarted my laptop, tested if the GPU is there with "nvida-smi" and the GPU was recogniced.
I imported tensorflow again and started my model again, however the same error message pops up and my GPU vanished.
Is there something wrong with the configuration in one of the tensorflow configuration files? Or Keras files? What can i change to get this work again? Do you know why the GPU is so much slower that the 8 CPUs?
Solution: Reinstalling tensorflow-gpu worked for me.
However there is still the question why that happened and how i can switch between gpu and cpu? I dont want to use a second virtual enviroment.