I trained a keras model that uses CuDNNLSTM cells, and now wish to load the model on a host device that lacks a GPU. Because CuDNNLSTM cells require a GPU, though, the loading process bombs out, throwing:
No OpKernel was registered to support Op 'CudnnRNN' with these attrs.
Is there some backdoor that will allow me to load the model on a host without a GPU? Any suggestions would be very helpful!
Note: I am using Keras 2.2.4 and TensorFlow 1.12.0.
I was able to solve the issue with the following steps:
1) Train the model with CudnnLSTM and save the model (model_GPU.json) and the weights (*.h5).
2) Define the same model changing CudnnLSTM for LSTM, this must be done in a system/computer with no GPU, and then you can save the model (model_CPU.json).
2*) In the LSTM cell set activation='tanh',recurrent_activation='sigmoid'. Since these are the default ones in CudnnLSTM.
3) Then you can load model_CPU.json with the weights trained with CudnnLSTM.
Specifically, I used the following
CPU:
from keras.layers import LSTM
Bidirectional(LSTM(hidden_units_LSTM, return_sequences=True,activation='tanh',recurrent_activation='sigmoid'))(output)
GPU:
from keras.layers import CuDNNLSTM
Bidirectional(CuDNNLSTM(hidden_units_LSTM, return_sequences=True))(output)
Related
When I trained a Keras model using QAT (Quantization aware training),
There are some non-compatible problems like not support BatchNormalization, or UpSampling2D, etc.
How to prevent it directly without apply each layer with tfmot.quantization.keras.quantize_annotate_layer on each layer? (especially when building model with tensorflow keras functional API (instead of tf.keras.Sequential))
Supported layers for QAT module can be found here
Then, to quantize some layers instead of whole Model, just followed the official tutorial, then add what layer you wanna use to quantize.
#added layers here
supported_layers = [tf.keras.layers.Conv2D, tf.keras.layers.Dense, tf.keras.layers.ReLU]
def apply_quantization_to_dense(layer):
for supported_layer in supported_layers:
if isinstance(layer, tf.keras.layers.Dense):
return tfmot.quantization.keras.quantize_annotate_layer(layer)
return layer
I have keras pretrained model(model.h5). And I want to prune that model with tensorflow Magnitude-based weight pruning with Keras. One curious things is that my pretrained model is built with original keras model > I mean that is not from tensorflow.keras. Inside tensorflow Magnitude-based weight pruning with Keras example, they show how to do with tensorflow.keras model. I want to ask is that can I use their tool to prune my original keras pretrained model?
inside their weight pruning toolkit ,there is two way. one is pruned the model layer by layer while training and second is pruned the whole model. I tried the second way to prune the whole pretrained model. below is my code.
inside their weight pruning toolkit ,there is two way. one is pruned the model layer by layer while training and second is pruned the whole model. I tried the second way to prune the whole pretrained model. below is my code.
For my original pretrained model, I load the weight from model.h5 and can call model.summary() after I apply prune_low_magnitude() none of the method from model cannot call including model.summary() method. And show the error like AttributeError: 'NoneType' object has no attribute 'summary'
model = get_training_model(weight_decay)
model.load_weights('model/keras/model.h5')
model.summary()
epochs = 1
end_step = np.ceil(1.0 * 100 / 2).astype(np.int32) * epochs
print(end_step)
new_pruning_params = {
'pruning_schedule': tfm.sparsity.keras.PolynomialDecay(initial_sparsity=0.1,
final_sparsity=0.90,
begin_step=40,
end_step=end_step,
frequency=30)
}
new_pruned_model = tfm.sparsity.keras.prune_low_magnitude(model, **new_pruning_params)
print(new_pruned_model.summary())
Inside their weight pruning toolkit
enter link description here ,there is two way. one is pruned the model layer by layer while training and second is pruned the whole model. I tried the second way to prune the whole pretrained model. below is my code.
For my original pretrained model, I load the weight from model.h5 and can call model.summary() after I apply prune_low_magnitude() none of the method from model cannot call including model.summary() method. And show the error like
AttributeError: 'NoneType' object has no attribute 'summary'
I hope this answer still helps, but I recently had the same issue that prune_low_magnitude() returns an object of type 'None'. Also new_pruned_model.compile() would not work.
The model I had been using was a pretrained model that could be imported from tensorflow.python.keras.applications.
For me this worked:
(0) Import the libraries:
from tensorflow_model_optimization.python.core.api.sparsity import keras as sparsity
from tensorflow.python.keras.applications.<network_type> import <network_type>
(1) Define the pretrained model architecture
# define model architecture
loaded_model = <model_type>()
loaded_model.summary()
(2) Compile the model architecture and load the pretrained weights
# compile model
opt = SGD(lr=learn_rate, momentum=momentum)
loaded_model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
loaded_model.load_weights('weight_file.h5')
(3) set pruning parameters and assign pruning schedule
# set pruning parameters
pruning_params = {
'pruning_schedule': sparsity.PolynomialDecay(...)
}
# assign pruning schedule
model_pruned = sparsity.prune_low_magnitude(loaded_model, **pruning_params)
(4) compile model and show summary
# compile model
model_pruned.compile(
loss=tf.keras.losses.categorical_crossentropy,
optimizer='SGD',
metrics=['accuracy'])
model_pruned.summary()
It was important to import the libraries specifically from tensorflow.python.keras and use this keras model from the TensorFlow library.
Also, it was important to use the TensorFlow Beta Release (pip install tensorflow==2.0.0b1), otherwise still an object with type 'None' would be returned by prune_low_magnitude.
I am using PyCharm 2019.1.3 (x64) as IDE. Here is the link that led me to this solution: https://github.com/tensorflow/model-optimization/issues/12#issuecomment-526338458
I'm using the Keras VGG16 model.
I've seen it there is a preprocess_input method to use in conjunction with the VGG16 model. This method appears to call the preprocess_input method in imagenet_utils.py which (depending on the case) calls _preprocess_numpy_input method in imagenet_utils.py.
The preprocess_input has a mode argument which expects "caffe", "tf", or "torch". If I'm using the model in Keras with TensorFlow backend, should I absolutely use mode="tf"?
If yes, is this because the VGG16 model loaded by Keras was trained with images which underwent the same preprocessing (i.e. changed input image's range from [0,255] to input range [-1,1])?
Also, should the input images for testing mode also undergo this preprocessing? I'm confident the answer to the last question is yes, but I would like some reassurance.
I would expect Francois Chollet to have done it correctly, but looking at https://github.com/fchollet/deep-learning-models/blob/master/vgg16.py either he is or I am wrong about using mode="tf".
Updated info
#FalconUA directed me to the VGG at Oxford which has a Models section with links for the 16-layer model. The information about the preprocessing_input mode argument tf scaling to -1 to 1 and caffe subtracting some mean values is found by following the link in the Models 16-layer model: information page. In the Description section it says:
"In the paper, the model is denoted as the configuration D trained with scale jittering. The input images should be zero-centered by mean pixel (rather than mean image) subtraction. Namely, the following BGR values should be subtracted: [103.939, 116.779, 123.68]."
The mode here is not about the backend, but rather about on what framework the model was trained on and ported from. In the keras link to VGG16, it is stated that:
These weights are ported from the ones released by VGG at Oxford
So the VGG16 and VGG19 models were trained in Caffe and ported to TensorFlow, hence mode == 'caffe' here (range from 0 to 255 and then extract the mean [103.939, 116.779, 123.68]).
Newer networks, like MobileNet and ShuffleNet were trained on TensorFlow, so mode is 'tf' for them and the inputs are zero-centered in the range from -1 to 1.
In my experience in training VGG16 in Keras, the inputs should be from 0 to 255, subtracting the mean [103.939, 116.779, 123.68]. I've tried transfer learning (freezing the bottom and stack a classifier on top) with inputs centering from -1 to 1, and the results are much worse than 0..255 - [103.939, 116.779, 123.68].
Trying to use VGG16 myself again lately, i had troubles getting descent results by just importing preprocess_input from vgg16 like this:
from keras.applications.vgg16 import VGG16, preprocess_input
Doing so, preprocess_input by default is set to 'caffe' mode but having a closer look at keras vgg16 code, i noticed that weights name
'https://github.com/fchollet/deep-learning-models/releases/download/v0.1/vgg16_weights_tf_dim_ordering_tf_kernels.h5'
is referring to tensorflow twice. I think that preprocess mode should be 'tf'.
processed_img = preprocess_input(img, mode='tf')
Is there a way to load a pretrained model in Tensorflow and remove the top layers in the network? I am looking at Tensorflow release r1.10
The only documentation I could find is with tf.keras.Sequential.pop
https://www.tensorflow.org/versions/r1.10/api_docs/python/tf/keras/Sequential#pop
I want to manually prune a pretrained network by removing bunch of top convolution layers and add a custom fully convoluted layer.
EDIT:
The model is ssd_mobilenet_v1_coco downloaded from Tensorflow Model Zoo. I have access to both the frozen_inference_graph.pb model file and checkpoint file.
I donot have access to the python code which is used to construct the model.
Thanks.
From inspecting the code, SSDMobileNetV1FeatureExtractor.extract_features redirects research.slim.nets:
from nets import mobilenet_v1 # nets will have to be on your PYTHONPATH
with tf.variable_scope('MobilenetV1',
reuse=self._reuse_weights) as scope:
with slim.arg_scope(
mobilenet_v1.mobilenet_v1_arg_scope(
is_training=None, regularize_depthwise=True)):
with (slim.arg_scope(self._conv_hyperparams_fn())
if self._override_base_feature_extractor_hyperparams
else context_manager.IdentityContextManager()):
_, image_features = mobilenet_v1.mobilenet_v1_base(
ops.pad_to_multiple(preprocessed_inputs, self._pad_to_multiple),
final_endpoint='Conv2d_13_pointwise',
min_depth=self._min_depth,
depth_multiplier=self._depth_multiplier,
use_explicit_padding=self._use_explicit_padding,
scope=scope)
The mobilenet_v1_base function takes a final_endpoint argument. Rather than prune the constructed graph, just construct the graph up until the endpoint you want.
I am comparing two pre-trained models, one is in Tensorflow and one is in Pytorch, on a machine that has multiple GPUs. Each model fits on one GPU. They are both loaded in the same Python script. How can I assign one GPU to the Tensorflow model and another GPU to the Pytorch model?
Setting CUDA_VISIBLE_DEVICES=0,1 only tells both models that these GPUs are available - how can I (within Python I guess), make sure that Tensorflow takes GPU 0 and Pytorch takes GPU 1?
You can refer to torch.device. https://pytorch.org/docs/stable/tensor_attributes.html?highlight=device#torch.torch.device
In particular do
device=torch.device("gpu:0")
tensor = tensor.to(device)
or to load a pretrained model
device=torch.device("gpu:0")
model = model.to(device)
to put tensor/model on gpu 0.
Similarly tensorflow has tf.device. https://www.tensorflow.org/api_docs/python/tf/device. Its usage is described here https://www.tensorflow.org/guide/using_gpu
for tensorflow to load model on gpu:0 do,
with tf.device("gpu:0"):
load_model_function(model_path)