I have trained a multi_gpu_model using tensorflow 1.13/1.14 and saved them with keras.model.save('<.hdf5>').
Now, after migrating to tensorflow 2.4.1, in which Keras is integrated as tensorflow.keras, I cannot tensorflow.keras.models.load_model as I did before, due to the following error:
AttributeError: module 'tensorflow.python.keras.backend' has no attribute 'slice'
After trying to import keras.models.load_model, and trying different versions of keras (2.2.4 -> 2.4.1) and tensorflow (2.2 -> 2.4.1), I cannot load_model from my .hdf5 file using my TF 2.2+ code.
I do know that in TF 2.X + we can train using distributed machines by implementing the "strategy" scope, and it does work, but I have a lot of "old" models that I need to work on the same code base which is now being migrated to TF 2.4.1
Apparently the problem was not the TF versions, but the way I was saving my models on my TF 1.X code versions.
I used the keras.multi_gpu_model class for both training and saving, while this practice is wrong, as clearly stated on Keras documentation:
"To save the multi-gpu model, use .save(fname) or .save_weights(fname)
with the template model (the argument you passed to multi_gpu_model),
rather than the model returned by multi_gpu_model."
So, after figuring this out a method for model conversion, using TF 1.X code, was adopted:
build you model from scratch, namely new_model
load your pre-trained weights from the multi_gpu_model, namely 'old_model'
copy your old_model's weights, which is old_model.layers[3] (due to the wrong usage of multi_gpu_model) to your new_model
save new_model as .hdf5 file
use new_model.hdf5 everywhere - TF 1.X and TF 2.X
Related
While converting model to tflite getting this error
"""
Some of the operators in the model are not supported by the standard TensorFlow Lite runtime and are not recognized by TensorFlow. If you have a custom implementation for them you can disable this error with --allow_custom_ops, or by setting allow_custom_ops=True when calling tf.lite.TFLiteConverter(). Here is a list of builtin operators you are using: ABS, ADD, CONV_2D, MAX_POOL_2D, MUL, RELU, SOFTMAX, SQUEEZE, SUB. Here is a list of operators for which you will need custom implementations: AdjustContrastv2, AdjustHue, AdjustSaturation, RandomUniform.
"""
How to resolve this?
tensorflow version: 1.13.1
You can use TF ops directly by selecting TF ops.
I've confirmed that AdjustContrastv2, AdjustHue, AdjustSaturation are available via FlexDelegate.
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/delegates/flex/allowlisted_flex_ops.cc#L35
To use this feature, you need to use TF 2.4 or higher. Since TF 2.4 is not available yet, you need to use tf-nightly release.
FYI, regarding migration TF1 to TF2, please check https://www.tensorflow.org/guide/migrate
You may try adding following lines to specify your model can use ops in both TF Lite built in and in TF.
converter.experimental_new_converter=True
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS,
tf.lite.OpsSet.SELECT_TF_OPS]
Or better you should rewrite ops not supported in TF Lite built in by ops available in TF built in
I'm having a problem similar to the one described here:
ValueError: Unknown layer: Functional
import tensorflow as tf
model = tf.keras.models.load_model("model.h5")
which throws: ValueError: Unknown layer: Functional.
I'm pretty sure this is because the h5 file was saved in TF 2.3.0 and I'm trying to load it in 2.2.0. I'd rather not convert using tf 2.3.0 directly, and I'm hoping to find a way of manually fixing the h5py file itself, or passing the right custom object to the model loader. I've noticed that it seems like it's just an extra key wherever the config file is stored, e.g. https://github.com/tensorflow/tensorflow/issues/41929
The problem is, I'm not sure how to manually get rid of the Functional layer in the h5 file. Specifically, I've tried:
import h5py
f = h5py.File("model.h5",'r')
print(f['model_weights'].keys())
which gives:
<KeysViewHDF5 ['concatenate_1', 'conv1d_3', 'conv1d_4', 'conv1d_5', 'dense_1', 'dropout_4', 'dropout_5', 'dropout_6', 'dropout_7', 'embedding_1', 'global_average_pooling1d_1', 'global_max_pooling1d_1', 'input_2']>
and I don't see the Functional layer anywhere. Where exactly is the config for the model stored in this file? E.g. I'm looking for something like {"class_name": "Functional", "config": {"name": "model", "layers":...}}
Question: is there a way I can manually edit the h5 file using h5py to get rid of the Functional layer?
Alternatively, can I pass a specific custom_obects={'Functiona':???} to the load_model function?
I've tried {'Functional':tf.keras.models.Model} but that returns ('Keyword argument not understood:', 'groups') because I think it's trying to load a model into weights?
I had a similar problem. The only way I could solve it without changing the Tensorflow version and retraining the model is by building the model structure again using Keras API in TensorFlow 2.2.0 and then call:
model.load_weights(<h5 file>)
where the original h5 file was created using TensorFlow 2.3.0. If you already have the code that builds the model structure then this method should be relatively easy since all you have to do is replace load_model(<h5 file>) with the line above.
Just change
keras.models import load_model
tensorflow.keras.models import load_model
then
load_model('model.h5', compile = False)
Summary
My question is composed by:
A context in which I present my project, my working environment and my workflow
The detailed problem
The concerned parts of my code
The solutions I tried to solve my problem
The question reminder
Context
I've written a Python Keras implementation of a downgraded version of the original Super-Resolution GAN. Now I want to test it using Google Firebase Machine Learning Kit, by hosting it in the Google servers. That's why I have to convert my Keras program to a TensorFlow Lite one.
Environment and workflow (with the problem)
I'm training my program on Google Colab working environment: there, I've installed TF 2.0.0-beta1 (this choice is motivated by this uncorrect answer: https://datascience.stackexchange.com/a/57408/78409).
Workflow (and problem):
I write locally my Python Keras program, keeping in mind that it will run on TF 2. So I use TF 2 imports, for example: from tensorflow.keras.optimizers import Adam and also from tensorflow.keras.layers import Conv2D, BatchNormalization
I send my code to my Drive
I run without any problem my Google Colab Notebook: TF 2 is used.
I get the output model in my Drive, and I download it.
I try to convert this model to the TFLite format by executing the following CLI: tflite_convert --output_file=srgan.tflite --keras_model_file=srgan.h5: here the problem appears.
The problem
Instead of outputing the TF Lite converted model from the TF (Keras) model, the previous CLI outputs this error:
ValueError: Unknown loss function:build_vgg19_loss_network
The function build_vgg19_loss_network is a custom loss function that I've implemented and that must be used by the GAN.
Parts of code that rise this problem
Presenting the custom loss function
The custom loss function is implemented like that:
def build_vgg19_loss_network(ground_truth_image, predicted_image):
loss_model = Vgg19Loss.define_loss_model(high_resolution_shape)
return mean(square(loss_model(ground_truth_image) - loss_model(predicted_image)))
Compiling the generator network with my custom loss function
generator_model.compile(optimizer=the_optimizer, loss=build_vgg19_loss_network)
What I've tried to do in order to solve the problem
As I read it on StackOverflow (link at the beginning of this question), TF 2 was thought to be sufficient to output a Keras model which would be correctly processed by my tflite_convert CLI. But it's not, obviously.
As I read it on GitHub, I tried to manually set my custom loss function among Keras' loss functions, by adding these lines: import tensorflow.keras.losses
tensorflow.keras.losses.build_vgg19_loss_network = build_vgg19_loss_network. It didn't work.
I read on GitHub I could use custom objects with load_model Keras function: but I only want to use compile Keras function. Not load_model.
My final question
I want to do only minor changes to my code, since it works fine. So I don't want, for example, to replace compile with load_model. With this constraint, could you help me, please, to make my CLI tflite_convert works with my custom loss function?
Since you are claiming that TFLite conversion is failing due to a custom loss function, you can save the model file without keep the optimizer details. To do that, set include_optimizer parameter to False as shown below:
model.save('model.h5', include_optimizer=False)
Now, if all the layers inside your model are convertible, they should get converted into TFLite file.
Edit:
You can then convert the h5 file like this:
import tensorflow as tf
model = tf.keras.models.load_model('model.h5') # srgan.h5 for you
converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)
Usual practice to overcome the unsupported operators in TFLite conversion is documented here.
I had the same error. I recommend changing the loss to "mse" since you already have a well-trained model and you don't need to train with the .tflite file.
I have a Problem converting Keras Models into Layers API format models to use with tensorflowjs
I use the command:
$ tensorflowjs_converter --input_format keras kerasModels/vgg16_weights_tf_dim_ordering_tf_kernels.h5 convertedModels/
I get an error "KeyError: Can't open attribute (can't locate attribute 'keras version')"
Here is an image of the error log:
I assume you are trying to convert the model downloaded from here, which is possibly outdated now.
You can download the VGG16 model fresh from keras-applications using the following python script:
from keras.applications.vgg16 import VGG16
model = VGG16(include_top=True, weights='imagenet')
model.save("VGG16.h5")
I used Keras to build a model and trained it. Then I saved the model as an h5 file, i.e. model.save('name.h5'). Now I want to reload the model in tensorflow such that I have access to .meta file, for example I want to import the computational graph from the .meta file, i.e., tf.train.import_meta_graph('name_of_the_file.meta').
So, the question is how to convert .h5 file of Keras to the following four files of TensorFlow:
.meta
checkpoint
.data-00000-of-00001
.index
You can use 3rd party packages, for example keras_to_tensorflow
keras_to_tensorflow: General code to convert a trained keras model into an inference tensorflow model
The conversion can be done by
python3 keras_to_tensorflow.py -input_model_file model.h5
Tensorflow 2.x will do that automatically. The function you are using to save (see also) is:
save(
filepath,
overwrite=True,
include_optimizer=True,
save_format=None
)
The save format let's you choose either 'h5' or 'tf'. However, for tensorflow 1.x is not implemented yet (and probably never will).
save_format: Either 'tf' or 'h5', indicating whether to save the model
to Tensorflow SavedModel or HDF5. The default is currently 'h5', but
will switch to 'tf' in TensorFlow 2.0. The 'tf' option is currently
disabled (use tf.keras.experimental.export_saved_model instead).
You can do as it says and use the tf.keras.experimental.export_saved_model but it will still not create the .meta file.