I am trying to create a new keras model from another trained keras model
Sample Code for model training referred from:
#TF version 2.2.0
from tensorflow.python.keras import models, layers
from tensorflow.python.keras.models import Sequential
from tensorflow.python import keras
import tensorflow as tf
from tensorflow.python.keras.layers import Dense
from tensorflow.keras.datasets import boston_housing
(x_train,y_train), (x_test,y_test) = boston_housing.load_data()
model = Sequential()
model.add(Dense(2, activation='relu', input_shape=(13,)))
model.add(Dense(1, activation='softmax'))
model.compile(optimizer='rmsprop',loss='categorical_crossentropy',metrics=['accuracy'])
model.fit(x_train, y_train,batch_size= 64,epochs= 1,validation_split=0.2)
Saving the model as json
json_obj = model.to_json()
new_model = keras.models.model_from_json(json_obj)
But after creating the new_model the weights are different:
model.get_weights() != new_model.get_weights()
This is the same case if I create new_model using from_config(). The question is, shouldn't the weight be the same for both model and new_model as I am creating new_model from model or my understanding is wrong? Any suggestions are helpful
to_json doesn't save the model's weights, but only the architecture.
Check here: to_json method
I recommend you to use save_model method.
If you want to copy the model to another directly, do the following:
new_model = tf.keras.models.clone_model(model)
new_model.set_weights(model.get_weights())
Related
I am following the official Keras transfer learning and fine-tuning tutorial. It consists of loading the Xception model with include_top=False, and adding a new classifier part on top.
I am then saving the model with model.save() and loading with load_model().
So this is what I see when I do model.summary()
My problem is that I would like to iterate through the layers, while now Xception layers are somehow folded (on the picture: xception(Functional)). Is there a way to somehow unfold it, to see all the layers (including those that are creating Xception)?
For model. summary(), you can unfold that as follows:
from tensorflow import keras
base_model = keras.applications.Xception(
weights='imagenet', # Load weights pre-trained on ImageNet.
input_shape=(150, 150, 3),
include_top=False) # Do not include the ImageNet classifier at the top.
inputs = keras.Input(shape=(150, 150, 3))
x = base_model(inputs, training=False)
x = keras.layers.GlobalAveragePooling2D()(x)
outputs = keras.layers.Dense(1)(x)
model = keras.Model(inputs, outputs)
model.summary() # full model
model.layers[1].summary() # only xception model
Note, you cal also see the layer using the plot_model utility.
keras.utils.plot_model(model, expand_nested=True)
I have the following code, I am retrieving error when I try to add my own classifier.
import keras
from keras import layers,Model
from keras.layers import Input,GlobalAveragePooling2D,Flatten,Dense
MobileNetV2_model= tf.keras.applications.MobileNetV2(input_shape=None, alpha=1.0, include_top=False,
weights='imagenet')
#MobileNetV2_model.summary()
x= MobileNetV2_model.output
x = layers.GlobalAveragePooling2D()(x)
final_output=layers.Dense(2, activation='sigmoid')(x)
model = keras.Model(inputs=MobileNetV2.input, outputs = final_output)
model.compile(optimizer="adam", loss='BinaryCrossentropy', metrics=['accuracy'],loss_weights=0.1)
Error
TypeError: Cannot convert a symbolic Keras input/output to a numpy array. This error may indicate that
you're trying to pass a symbolic value to a NumPy call, which is not supported. Or, you may be trying
to pass Keras symbolic inputs/outputs to a TF API that does not register dispatching, preventing Keras from automatically converting the API call to a lambda layer in the Functional Model.
You should never mix keras and tf.keras. You can refer working code as shown below
import tensorflow as tf
from tensorflow.keras import layers, Model
MobileNetV2_model= tf.keras.applications.MobileNetV2(input_shape=(224,224,3), alpha=1.0, include_top=False, weights='imagenet')
#MobileNetV2_model.summary()
x= MobileNetV2_model.output
x = layers.GlobalAveragePooling2D()(x)
final_output=layers.Dense(2, activation='sigmoid')(x)
model = Model(inputs=MobileNetV2_model.input, outputs = final_output)
model.compile(optimizer="adam", loss='BinaryCrossentropy', metrics=['accuracy'],loss_weights=0.1)
I am building a neural network using keras and tensorflow and I get a error at this place
def create_model():
model = Sequential()
model.add(Dense(4, input_dim=2, kernel_initializer='normal', activation='tanh'))
model.add(Dense(6, activation='relu'))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer=RAdam(learning_rate), metrics=['accuracy'])
return model
model = create_model()
And I get the following error when I run my code in jupyter notebook,
TypeError Traceback (most recent call last)
<ipython-input-14-2358feb9246f> in <module>
1 # make a shallow neural network
----> 2 model = create_model()
3 model.summary()
<ipython-input-13-7c6ab8b2130e> in create_model()
10
11 # Compile model
---> 12 model.compile(loss='binary_crossentropy', optimizer=RAdam(learning_rate), metrics=['accuracy'])
13 return model
~\anaconda3\envs\tf\lib\site-packages\keras_radam\optimizers.py in __init__(self, learning_rate, beta_1, beta_2, epsilon, decay, weight_decay, amsgrad, total_steps, warmup_proportion, min_lr, **kwargs)
32 total_steps=0, warmup_proportion=0.1, min_lr=0., **kwargs):
33 learning_rate = kwargs.pop('learning_rate', learning_rate)
---> 34 super(RAdam, self).__init__(**kwargs)
35 with K.name_scope(self.__class__.__name__):
36 self.iterations = K.variable(0, dtype='int64', name='iterations')
TypeError: __init__() missing 1 required positional argument: 'name'
And these are the imports I have used for my code to run. I think I have most of the codes imported to build a shallow neural network
import numpy as np
import keras
import tensorflow as tf
from keras.models import Sequential
from keras.layers import Dense
from keras import backend as K
from keras.wrappers.scikit_learn import KerasClassifier
from keras_radam import RAdam
For others who may be looking for another solution.
RAdam is not in tensorflow.keras.optimizers and neither in keras by default, but in tensorflow-addons package, which is a better alternative (IMHO) than the external keras_radam library, considerably less prone to errors.
What you are looking for is here: https://www.tensorflow.org/addons/api_docs/python/tfa/optimizers/RectifiedAdam
#pip install tensorflow-addons
import tensorflow_addons as tfa
optimizer = tfa.optimizers.RectifiedAdam(lr=1e-3)
I was able to reproduce your problem. It happened when you have tf. keras but you load keras-radam with old keras. But this implementation supports both versions of keras or tf. keras. To use it with the new version, as also mentioned here, all you need to do as follows:
import os
os.environ['TF_KERAS']='1'
from keras_radam import RAdam
The package will choose the tf. keras compatible version of RAdam()
from .backend import TF_KERAS
__all__ = ['RAdam']
if TF_KERAS:
from .optimizer_v2 import RAdam
else:
from .optimizers import
So, RAdam() will be imported from this script. But there is another issue. In the very latest version of tf, the following import has been updated
# from
from tensorflow.python import os, math_ops, state_ops, control_flow_ops
# to
from tensorflow.python.ops import math_ops, state_ops, control_flow_ops
From this point, you need to modify this import from the source script and it will solve this issue. Just modify the source script by replacing the above imports.
from keras import Sequential
from keras.layers import Dense
def create_model():
model = Sequential()
model.add(Dense(4, input_dim=2, kernel_initializer='normal', activation='tanh'))
model.add(Dense(6, activation='relu'))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy',
optimizer=RAdam(learning_rate),
metrics=['accuracy'])
return model
model = create_model()
I am using Anaconda Navigator, Jupyter to be precised.
import tensorflow as tf
from tensorflow import keras
print(tf.__version__)
>>> 1.14.0
This is my model
def create_model():
model = tf.keras.Sequential([
keras.layers.Dense(86, activation='relu', kernel_regularizer=keras.regularizers.l2(0.0001),input_shape=(129,)),
keras.layers.Dropout(0.2),
keras.layers.Dense(142, activation='relu', kernel_regularizer=keras.regularizers.l2(0.0001)),
keras.layers.Dropout(0.2),
keras.layers.Dense(4, activation='softmax')
])
return model
model = create_model()
# Display the model's architecture
model.summary()
After training,predicting and evaluating my model, I decided to save it using
model.save('/Users/Jennifer/myproject/my_model.h5')
I checked the directory and folder with the h5py file. And I decided to load it using
new_model1 = tf.keras.models.load_model('/Users/Jennifer/myproject/my_model.h5')
I got an Error
ValueError: Unknown entries in loss dictionary: ['class_name', 'config']. Only expected following keys: ['dense_17']
Please help me. What should I do? I have almost spent the whole day trying to solve this issue. Thanks
Here is a bit of a work around that just loads the weights:
#!/usr/bin/env python3
from tensorflow import keras
import os
def create_model():
model = keras.Sequential([
keras.layers.Dense(86, activation='relu', kernel_regularizer=keras.regularizers.l2(0.0001),input_shape=(129,)),
keras.layers.Dropout(0.2),
keras.layers.Dense(142, activation='relu', kernel_regularizer=keras.regularizers.l2(0.0001)),
keras.layers.Dropout(0.2),
keras.layers.Dense(4, activation='softmax')
])
return model
if os.path.exists("junk.h5"):
model = create_model()
model.load_weights("junk.h5")
else:
model = create_model()
model.compile(optimizer=keras.optimizers.Adam(0.0001), loss=keras.losses.CategoricalCrossentropy(from_logits=True), metrics=['accuracy'])
model.save("junk.h5")
Another workaround would be to save the model without the optimizer
model.save("junk.h5", include_optimizer=False)
It looks like the loss function you're using creates a dictionary that has invalid keys. This sounds like a bug in keras/tensorflow. That is why the colab one probably worked because it was using a newer version.
I've implemented a model using LSTM program in keras. I am trying to get the representations of hidden nodes of the LSTM layer. Is this the right way to get the representation (stored in activations variable) of hidden nodes?
model = Sequential()
model.add(LSTM(50, input_dim=sample_index))
activations = model.predict(testX)
model.add(Dense(no_of_classes, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adagrad', metrics=['accuracy'])
hist=model.fit(trainX, trainY, validation_split=0.15, nb_epoch=5, batch_size=20, shuffle=True, verbose=1)
Edit: your way to get the hidden representation is also correct.
Reference: https://github.com/fchollet/keras/issues/41
After training your model, you can save your model and the weights. Like this:
from keras.models import model_from_json
json_model = yourModel.to_json()
open('yourModel.json', 'w').write(json_model)
yourModel.save_weights('yourModel.h5', overwrite=True)
And then you can visualize the weights of your LSTM layers. Like this:
from keras.models import model_from_json
import matplotlib.pyplot as plt
model = model_from_json(open('yourModel.json').read())
model.load_weights('yourModel.h5')
layers = model.layers[1] # find the LSTM layer you want to visualize, [1] is just an example
weights, bias = layers.get_weights()
plt.matshow(weights, fignum=100, cmap=plt.cm.gray)
plt.show()