I am comparing two pre-trained models, one is in Tensorflow and one is in Pytorch, on a machine that has multiple GPUs. Each model fits on one GPU. They are both loaded in the same Python script. How can I assign one GPU to the Tensorflow model and another GPU to the Pytorch model?
Setting CUDA_VISIBLE_DEVICES=0,1 only tells both models that these GPUs are available - how can I (within Python I guess), make sure that Tensorflow takes GPU 0 and Pytorch takes GPU 1?
You can refer to torch.device. https://pytorch.org/docs/stable/tensor_attributes.html?highlight=device#torch.torch.device
In particular do
device=torch.device("gpu:0")
tensor = tensor.to(device)
or to load a pretrained model
device=torch.device("gpu:0")
model = model.to(device)
to put tensor/model on gpu 0.
Similarly tensorflow has tf.device. https://www.tensorflow.org/api_docs/python/tf/device. Its usage is described here https://www.tensorflow.org/guide/using_gpu
for tensorflow to load model on gpu:0 do,
with tf.device("gpu:0"):
load_model_function(model_path)
Related
After choosing and downloading a model from TensorFlow 2 Detection Model Zoo, it can be loaded as followed:
import tensorflow as tf
model = tf.saved_model.load(f'./efficientdet_d0_coco17_tpu-32/saved_model/')
However, it looks like one cannot extract the number of trainable variables directly/indirectly from the model variable, according to this investigation.
Nevertheless, the model training can continue, with new data, as this is a typical use-case of a pre-trained model. There must be a way to get the number of trainable variables. But I don't know how.
I tried:
tf.trainable_variables
# AttributeError: module 'tensorflow' has no attribute 'trainable_variables'
Environment:
Tensorflow 2.7.0 (implying CUDA 11.2, cuDNN 8.1).
Windows 10 x64
Python 3.9.7
NVIDIA GeForce MX150, Compute capability: 6.1
I have looked through many articles and posts about using multiple GPUs with TensorFlow. It helps me more here on "how to use parallel GPUs to train NN (neural network)". But I have a different question. Can a separate GPU be used to train different NNs at the same time?
More details:
I have neural networks A, B, and GPU1, GPU2. I want to train A NN on GPU1 and B NN on GPU2 at the same time. Is it possible?
I suggest using two separate python scripts to train both networks, such as trainA.py and trainB.py.
In the first two lines of trainA.py you select your preferred GPU.
import os
os.environ["CUDA_VISIBLE_DEVICES"] = "0"
For trainB.py you select the other GPU:
import os
os.environ["CUDA_VISIBLE_DEVICES"] = "1"
Now you should be able to run both train scripts at the same time.
In tensorflow 1.x this can be done using a graph and a session, which is quite tedious.
Is there an easier way to manually assign pretrained weights to a specific convolution in tensorflow 2.x?
If you are working with Keras inside Tensorflow 2.x, every layer has a method called set_weights that you can use to substitute weights or assign new ones from Numpy arrays.
Say, for example, that you are doing distillation knowledge. Then you could assign weights of the teacher to the student by:
conv.set_weights(teacher.convx.get_weights())
where conv is a particular layer of the student and convx the homologue of the teacher.
You can check the documentation for more details:
Documentation - set_weights()
Assume that the model trained in tensorflow uses two methods to convert the models available to tensorflowjs:
1) Use the tf.saved_model.simple_save method to save the model in tensorflow, then use tf.loadFrozenModel to load the model in tensorflowjs and predict the result using model.predict
2) Use keras(sequence) to save the model in tensorflow, then use tf.loadModel to load the model in tensorflowjs and predict the result using model.predict
If you train the same model in tensorflow, but different save methods. In the tensorflowjs to use the above 2 load model method to predict the results, will the time difference?
If you have the same architecture in both tensorflowJs and keras, the inference time using tensorflowJs will be alike. tensorflowJs converter will just construct a graph of your topology and the weights. So in both cases the processing time is roughly the same
I trained a keras model that uses CuDNNLSTM cells, and now wish to load the model on a host device that lacks a GPU. Because CuDNNLSTM cells require a GPU, though, the loading process bombs out, throwing:
No OpKernel was registered to support Op 'CudnnRNN' with these attrs.
Is there some backdoor that will allow me to load the model on a host without a GPU? Any suggestions would be very helpful!
Note: I am using Keras 2.2.4 and TensorFlow 1.12.0.
I was able to solve the issue with the following steps:
1) Train the model with CudnnLSTM and save the model (model_GPU.json) and the weights (*.h5).
2) Define the same model changing CudnnLSTM for LSTM, this must be done in a system/computer with no GPU, and then you can save the model (model_CPU.json).
2*) In the LSTM cell set activation='tanh',recurrent_activation='sigmoid'. Since these are the default ones in CudnnLSTM.
3) Then you can load model_CPU.json with the weights trained with CudnnLSTM.
Specifically, I used the following
CPU:
from keras.layers import LSTM
Bidirectional(LSTM(hidden_units_LSTM, return_sequences=True,activation='tanh',recurrent_activation='sigmoid'))(output)
GPU:
from keras.layers import CuDNNLSTM
Bidirectional(CuDNNLSTM(hidden_units_LSTM, return_sequences=True))(output)