When using TensorFlow's Keras sequential API is there any way to force my model to be trained on a certain piece of hardware? My understanding is that if there is a GPU to use (and I have tensorflow-gpu installed) I will, by default, do my training on the GPU.
Do I have to switch to a different API to gain more control over where my model is deployed?
I am a keras user and I work on ubuntu. I specify a certain GPU as follows:
import os
os.environ["CUDA_VISIBLE_DEVICES"] = "0"
where 0 is the number of GPU. By default, tensorflow uses the first GPU (whose number is 0) if there are several ones on your computer. You can obtain the information of GPUs by typing the following command on your terminal:
nvidia-smi
or
watch -n 1 -d nvidia-smi
if you want to refresh your terminal every second. The following picture shows the information of my GPU, and the number of it has been circled by a red box.
Related
hello~ I am very confused with this situation.
first, both my tf and pytorch can detect my gpu (use torch.cuda,is_available())
but my model which runs just fine on gpus just few days before can only run on cpus today.
it seems pytorch and tf skip passing model to gpu directly.
second, I have test in python interactive mode with:
import torch
x = torch.randn(10000,1000).cuda()
this line worked fine, and when I type
x.device
python shows me that x is on gpu device index 0
but at the same time.
NO GPU MEMORY in use in nvidia-smi
third, when I monitor my gpu states with
watch -n 1 nvidia-smi
I found temperature or power of my gpus does not vary at all for a long while.
any help will be appreciated!!
We have a DGX-1 in Lab.
I see many tasks are running on different GPU.
For MLperf docker application, I can use NV_GPU=x to assign which GPU to use.
However, I have a python Keras/TensorFlow code, I used this same way, the loading doesn't go to the specified GPU.
You could use CUDA_VISIBLE_DEVICES to specify the GPU to be used by your model:
import os
os.environ["CUDA_VISIBLE_DEVICES"] = 0,1 #Will assign GPUs 0 and 1 to the model
I want to train spacy model on custom dataset but its take too much time for training, is there any way to speed up the training.
I passed device=0 in ner.begin_training() but it takes same amount of time as before.
Yes, it is possible.
Go to [EDIT] -> [Notebook settings] -> Select GPU under Hardware Acceleration (It restarts the runtime, so all your cell states get lost)
Use !pip install -U spacy[cuda100] to install spacy with Cuda Support
Run the following
Script:
import spacy
gpu = spacy.prefer_gpu()
print('GPU:', gpu)
It returns:
GPU: True
I am new to Tensorflow, I am using retrain.py to train some images. In case I have a larger data base of 10000 images and I have a GPU capable system. How can i use retrain.py to run on my Nvidia GPU. So that training will be done faster.
I am following the steps from the link below
https://www.tensorflow.org/hub/tutorials/image_retraining
To get GPU support, be sure to install the PIP package tensorflow-gpu instead of plain tensorflow. You should see some performance benefits from that for retrain.py. That said, retrain.py shows its age (far predating TF Hub) and does not utilize GPUs so well, because it does not properly batch images when extracting bottleneck values.
If you are ready to live on the cutting edge of TF 2.0.0alpha0 (announced last week), take a look at Hub's
examples/colab/tf2_image_retraining.ipynb which is considerably smaller, faster (if you use a GPU), and even supports fine-tuning the image module.
I have a linux system with three gpus. I am using keras with theano to run cnn's, In the past when I was using Theano 8.+ , I was able to assign a particular gpu to jupyter notebook window using the following:
import theano.sandbox.cuda
theano.sandbox.cuda.use("gpu2")
This allowed me to run three versions of the same cnn model using different hyper-parameters.
I very recently updated both keras (to 2.0) and theano ( to 0.9). This required me to setup the gpuarray backend.
Running just one jupyter notebook with a model works fine. gpu1 is selected by theano. However when I startup a second notebook with the same model, theano tries to use the gpu assigned to the first notebook, causing a memory usage problem and ultimately causing the cnn model to run on the cpu rather than using one of the available two remaining gpus.
Is there a way to select the gpu that I wish the run on each jupyter notebook in theano 0.9 as I was able in theano 8.+