I running DQN training tutorial with tf.agents(https://www.tensorflow.org/agents/tutorials/1_dqn_tutorial), and I am trying to change the model they use with just dense layers to have some convolutional ones on top
When I run this on colab without GPU its all fine, but when I connect to GPU I get this error.
Is it because conv layers are not available yet with GPU with tf.agents or are there some dependencies extra that I have to install to be able to do this?
Related
I have a quick question: when using Google Colab with the GPU enabled, does all of the code already run on the GPU then or is there some setting in the code that we must change to make it run on the GPU? Specifically, if I am training a neural network model in Keras/TF, do I need to edit my code in any way to ensure that the model is trained on the GPU?
Thanks!
as showed in the Tensorflow with Gpu example notebook you can run your model in the following way to make sure it is running on the chosen device:
def gpu():
with tf.device('/device:GPU:0'):
random_image_gpu = tf.random.normal((100, 100, 100, 3))
net_gpu = tf.keras.layers.Conv2D(32, 7)(random_image_gpu)
return tf.math.reduce_sum(net_gpu)
I am able to do this with their GPU, but with their TPU it retrieves me an error...
Does anybody around here know what I'm missing, please?
Does it make sense to actually use the TPU with CuDNNLSTM? Or is CuDNNLSTM just tailored for GPU?
Thanks a lot in advance.
keras.layers.CuDNNLSTM is only supported on GPUs. But in Tensorflow 2 built-in LSTM and GRU layers have been updated to leverage CuDNN kernels by default when a GPU is available.
Below is the details from Performance optimization and CuDNN kernels:
In TensorFlow 2.0, the built-in LSTM and GRU layers have been updated to leverage CuDNN kernels by default when a GPU is available.
With this change, the prior keras.layers.CuDNNLSTM/CuDNNGRU layers have been deprecated, and you can build your model without worrying about the hardware it will run on.
You can just use the built-in LSTM layer: tf.keras.layers.LSTM and it will work on both TPUs and GPUs.
I am playing around with Progressive Growing of Gans network from Karras et al. (NVIDIA). I trained the network on a different dataset on 2x 1080 Ti cards, using the NCHW mode for all convolutions as seen here.
Now I have a trained model and I want to use the code snippet from the project's readme to import the trained network - which is mentioned in the Importing and using pre-trained networks section.
However, when I try to run it on a PC with a CPU only, the network fails with the error:
InvalidArgumentError (see above for traceback): Conv2DCustomBackpropInputOp only supports NHWC.
[[node Gs/Run/Gs/cond/8x8/Conv0_up/conv2d_transpose (defined at <string>:93) ]]
I know that the CPU does not have NCHW implemented, but I would like to know how can I work around this? I see several options out of which I do not want any:
The first thing that comes to mind is to just use the NHWC mode for generating a new image, even though the network was trained in NCHW. This, however, if I am correct, will mess up the layer's weights, since it was trained on NCHW.
I do not want to train the whole network in NHWC mode, since it should be slower than NCHW? I do not use the cuDNN from NVidia, does it mean that those two modes are the same speed then, according to this?
What else can I do? Thank you!
I am currently training a bidirectional RNN model using the CudnnRNNRelu on tensorflow. I want to use these checkpoints for inference on a CPU machine. The documentation suggests that models trained with Cudnn specific cells (CudnnRNNRelu, CudnnRNNTanh, CudnnLSTM etc ..) cannot be restored directly using platform independent rnn cells ( tf.contrib.rnn.BasicRNNCell etc.. ). While there are separate cells to restore LSTM and GRU cells ( CudnnComaptibleLSTMCell and CudnnCompatibleGRUCell) , there is no indication of how to use CudnnRNNRelu trained checkpoints with platform independent rnn cells.
Restoring works properly on a GPU machine. Is there a way to restore the CudnnRNNRelu checkpoints?
I wanna use the inception v3 model in tensor flow for feature extraction. But the number of the images I am using is a lot, so it takes long time to run. So, I am going to use GPU. I have installed Cuda 7.5 and cuDnn correctly.
I am using following code in the CPU mode for one image:
with tf.Session as sess:
softmax_tensor =sess.graph.get_tensor_by_name('pool_3:0')
feat_vect = numpy.squeeze(sess.run(softmax_tensor,{'DecodeJpeg:0': in_image}))
So, my question is that how should I change my code so I can use it for many batches by GPU?