When i am using "optimizer = keras.optimizers.Adam(learning_rate)" i am getting this error
"AttributeError: module 'keras.optimizers' has no attribute 'Adam". I am using python3.8 keras 2.6 and backend tensorflow 1.13.2 for running the program. Please help to resolve !
Use tf.keras.optimizers.Adam(learning_rate) instead of keras.optimizers.Adam(learning_rate)
As per the documentation , try to import keras into your code like this,
>>> from tensorflow import keras
This has helped me as well.
Make sure you've imported tensorflow:
import tensorflow as tf
Then use
tf.optimizers.Adam(learning_rate)
There are ways to solve your problem as you are using keras 2.6 and tensorflow too:
use (from keras.optimizer_v2.adam import Adam as Adam) but go through the function documentation once to specify your learning rate and beta values
you can also use (Adam = keras.optimizers.Adam).
(import tensorflow as tf) then (Adam = tf.keras.optimizers.Adam)
Use the form that is useful for the environment you set
I think you are using Keras directly. Instead of giving as from keras.distribute import —> give as from tensorflow.keras.distribute import
Hope this would help you.. It is working for me.
I have trained a multi_gpu_model using tensorflow 1.13/1.14 and saved them with keras.model.save('<.hdf5>').
Now, after migrating to tensorflow 2.4.1, in which Keras is integrated as tensorflow.keras, I cannot tensorflow.keras.models.load_model as I did before, due to the following error:
AttributeError: module 'tensorflow.python.keras.backend' has no attribute 'slice'
After trying to import keras.models.load_model, and trying different versions of keras (2.2.4 -> 2.4.1) and tensorflow (2.2 -> 2.4.1), I cannot load_model from my .hdf5 file using my TF 2.2+ code.
I do know that in TF 2.X + we can train using distributed machines by implementing the "strategy" scope, and it does work, but I have a lot of "old" models that I need to work on the same code base which is now being migrated to TF 2.4.1
Apparently the problem was not the TF versions, but the way I was saving my models on my TF 1.X code versions.
I used the keras.multi_gpu_model class for both training and saving, while this practice is wrong, as clearly stated on Keras documentation:
"To save the multi-gpu model, use .save(fname) or .save_weights(fname)
with the template model (the argument you passed to multi_gpu_model),
rather than the model returned by multi_gpu_model."
So, after figuring this out a method for model conversion, using TF 1.X code, was adopted:
build you model from scratch, namely new_model
load your pre-trained weights from the multi_gpu_model, namely 'old_model'
copy your old_model's weights, which is old_model.layers[3] (due to the wrong usage of multi_gpu_model) to your new_model
save new_model as .hdf5 file
use new_model.hdf5 everywhere - TF 1.X and TF 2.X
Summary
My question is composed by:
A context in which I present my project, my working environment and my workflow
The detailed problem
The concerned parts of my code
The solutions I tried to solve my problem
The question reminder
Context
I've written a Python Keras implementation of a downgraded version of the original Super-Resolution GAN. Now I want to test it using Google Firebase Machine Learning Kit, by hosting it in the Google servers. That's why I have to convert my Keras program to a TensorFlow Lite one.
Environment and workflow (with the problem)
I'm training my program on Google Colab working environment: there, I've installed TF 2.0.0-beta1 (this choice is motivated by this uncorrect answer: https://datascience.stackexchange.com/a/57408/78409).
Workflow (and problem):
I write locally my Python Keras program, keeping in mind that it will run on TF 2. So I use TF 2 imports, for example: from tensorflow.keras.optimizers import Adam and also from tensorflow.keras.layers import Conv2D, BatchNormalization
I send my code to my Drive
I run without any problem my Google Colab Notebook: TF 2 is used.
I get the output model in my Drive, and I download it.
I try to convert this model to the TFLite format by executing the following CLI: tflite_convert --output_file=srgan.tflite --keras_model_file=srgan.h5: here the problem appears.
The problem
Instead of outputing the TF Lite converted model from the TF (Keras) model, the previous CLI outputs this error:
ValueError: Unknown loss function:build_vgg19_loss_network
The function build_vgg19_loss_network is a custom loss function that I've implemented and that must be used by the GAN.
Parts of code that rise this problem
Presenting the custom loss function
The custom loss function is implemented like that:
def build_vgg19_loss_network(ground_truth_image, predicted_image):
loss_model = Vgg19Loss.define_loss_model(high_resolution_shape)
return mean(square(loss_model(ground_truth_image) - loss_model(predicted_image)))
Compiling the generator network with my custom loss function
generator_model.compile(optimizer=the_optimizer, loss=build_vgg19_loss_network)
What I've tried to do in order to solve the problem
As I read it on StackOverflow (link at the beginning of this question), TF 2 was thought to be sufficient to output a Keras model which would be correctly processed by my tflite_convert CLI. But it's not, obviously.
As I read it on GitHub, I tried to manually set my custom loss function among Keras' loss functions, by adding these lines: import tensorflow.keras.losses
tensorflow.keras.losses.build_vgg19_loss_network = build_vgg19_loss_network. It didn't work.
I read on GitHub I could use custom objects with load_model Keras function: but I only want to use compile Keras function. Not load_model.
My final question
I want to do only minor changes to my code, since it works fine. So I don't want, for example, to replace compile with load_model. With this constraint, could you help me, please, to make my CLI tflite_convert works with my custom loss function?
Since you are claiming that TFLite conversion is failing due to a custom loss function, you can save the model file without keep the optimizer details. To do that, set include_optimizer parameter to False as shown below:
model.save('model.h5', include_optimizer=False)
Now, if all the layers inside your model are convertible, they should get converted into TFLite file.
Edit:
You can then convert the h5 file like this:
import tensorflow as tf
model = tf.keras.models.load_model('model.h5') # srgan.h5 for you
converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)
Usual practice to overcome the unsupported operators in TFLite conversion is documented here.
I had the same error. I recommend changing the loss to "mse" since you already have a well-trained model and you don't need to train with the .tflite file.
I'm unable to import this module
import keras.applications.resnet
ModuleNotFoundError
in ()
----> 1 import keras.applications.resnet
ModuleNotFoundError: No module named 'keras.applications.resnet'
keras resnet link
Keras team hasn't included resnet, resnet_v2 and resnext in the current module, they will be added from Keras 2.2.5, as mentioned here.
For a workaround, you can use keras_applications module directly to import all ResNet, ResNetV2 and ResNeXt models, as given below
from keras_applications.resnet import ResNet50
Or if you just want to use ResNet50
from keras.applications.resnet50 import ResNet50
Alternatively, you can always build from source as mentioned here.
try to use
from tensorflow.keras.applications.resnet50 import ResNet50
Found a workaround to use ResNeXt in Keras 2.2.4 here.
ResNeXt50() function needs 4 more arguments: backend, layers, models and utils.
import keras
from keras_applications.resnext import ResNeXt50
model = ResNeXt50(weights='imagenet',
backend=keras.backend,
layers=keras.layers,
models=keras.models,
utils=keras.utils)
In Keras there are multiple flavours of ResNet, you will have to specify the version of ResNet that you want e.g. You wish to load the ResNet50.
Use
from keras.applications import ResNet50
Edit 2 This is the list you get when you use dir() command on applications
['DenseNet121', 'DenseNet169', 'DenseNet201', 'InceptionResNetV2', 'InceptionV3', 'MobileNet', 'MobileNetV2', 'NASNetLarge', 'NASNetMobile', 'ResNet50', 'VGG16', 'VGG19', 'Xception', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', 'absolute_import', 'backend', 'densenet', 'division', 'inception_resnet_v2', 'inception_v3', 'keras_applications', 'keras_modules_injection', 'layers', 'mobilenet', 'mobilenet_v2', 'models', 'nasnet', 'print_function', 'resnet50', 'utils', 'vgg16', 'vgg19', 'xception'], the models visible here can be laoded like this, There are some models like ResNet101 missing here, let me see if I can come up with a way to fix this.
Edit Proof that this works too
To see all the available versions of the Resnet models, visit https://keras.io/applications/#resnet
There is a python package named 'keras-resnet' which has ResNet50, ResNet101, ResNet152 and many more variants of ResNet. (https://pypi.org/project/keras-resnet/)
Installation is also quite easy. Just type
pip install keras-resnet
It will install this module and then use it like:
from keras_resnet.models import ResNet50, ResNet101, ResNet152
backbone = ResNet50(inputs=image_input, include_top=False, freeze_bn=True)
C2, C3, C4, C5 = backbone.outputs # this will give you intermediate
# outputs of four blocks of resnet if you want to merge low and high level features
I am using backbones from this module and is working fine for me!
Check the versions:
pip list | grep Keras
If it's already installed, uninstall and upgrade:
pip uninstall Keras
pip install Keras==2.3.1
pip uninstall Keras-Applications
pip install Keras-Applications==1.0.8
pip uninstall Keras-Preprocessing
pip install Keras-Preprocessing==1.1.0
from keras.applications.resnet
import ResNet101
tf.keras.backend.clear_session
model=VGG19()
model.summary()
tf.keras.utils.plot_model(model,show_shapes=True)
visualkeras.layered_view(model,legend=True)
Before running
tensorflow.keras.applications.resnet50 import ResNet50
you need to run
from tensorflow import keras
I am learning Keras and Thensorflow for deep learning and I have a question.
With this imports:
import tensorflow as tf
from keras import backend as K
Is there a difference between these two calls:
K.foo
and
tf.foo
?
In which conditions are they equivalent ?
Yes, there may be a difference.
Keras is built on top of a backend. This backend may be Tensorflow, Theano or CNTK.
So, a function from keras will call a function from the backend, something like this:
#at keras.backend
def foo(args**):
#there may be some preprocessing or inversion in dimensions
return tf.foo(args_that_may_be_different**)
It's impossible to have an answer for all functions. Some are indeed exactly the same, some may have a difference.
You can search the backend codes, speficially the tensorflow backend and see how keras handles each function.