I have built an image classifier with 2 classes, say 'A' and 'B'. I have also saved this model, using model.save().
Now, after a certain time, the requirement arose to add one more class 'C'. Is it possible to load_model() and then add only one class to the previously saved model so that we have the final model with 3 classes ('A','B' and 'C'), without having to retrain the whole model, for classes 'A and 'B' again?
Can anyone help?
I have tried this:
I used vgg16 as a base model and pop out its last layer, freeze weights and added one dense layer (DL2), trained it for predicting 2 classes.
Then I added one more dense layer on top of DL2 say DL3, freeze weights and train with class C only but now its always predicting class C.
I think you should check this tutorial:
https://www.tensorflow.org/tutorials/image_retraining.
In short:
You can not take trained model, and add new classes.
You should make some additional 'finetuning', may be not retrain the model from scratch, but at least to train classifier (and some additional layers).
You can also simply change the number of output classes in the last layer and freeze weights for the remaining layer. Retrain the weights for only the last layer.
Just use a transfer learning, and create a new model.
model = VGG16(weights='imagenet',
include_top=False,
input_shape=(150, 150, 3))
model.pop()
base_model_layers = model.output
pred = Dense(11, activation='softmax')(base_model_layers)
model = Model(inputs=model.input, outputs=pred)
# Freeze the first layers, before train it
for layer in model.layers[:-2]:
layer.trainable = False
Related
I was working on an image recognition problem. After training the model, I saved the architecture as well as weights. Now I want to use the model for extracting features from other images and perform SVM on that. For this, I want to remove the last two layers of my model and get the values calculated by the CNN and fully connected layers till then. How can I do that in Keras?
# a simple model
model = keras.models.Sequential([
keras.layers.Input((32,32,3)),
keras.layers.Conv2D(16, 3, activation='relu'),
keras.layers.Flatten(),
keras.layers.Dense(10, activation='softmax')
])
# after training
feature_only_model = keras.models.Model(model.inputs, model.layers[-2].output)
feature_only_model take a (32,32,3) for input and the output is the feature vector
If your model is subclassed - just change call() method.
If not:
if your model is complicated - wrap your model by subclassed model and change forward pass in call() method, or
if your model is simple - create model without the last layers, load weights to every layer separately
I'm trying to implement a SimGAN in Keras, but I believe the question is more generally related to GANs and freezing layers.
As I understand it I need three models:
The refiner, which processes a synthetic image to make it more realistic. (In other GAN architectures, this might be the generator, which accepts random noise.)
The discriminator, which processes an image and classifies it as synthetic or real.
A combined model which passes the synthetic image through the refiner, and then the refined image into the discriminator.
In the combined model, we want the discriminator layers to be frozen, so that when the combined model is trained, we update only the refiner layers.
Separately, we train the discriminator-only model, where obviously the layers should not be frozen. The layers of the discriminator model and the combined model should share weights so that both get updated.
Here's what I have so far:
refiner_model = make_refiner_model(input_shape=(img_height, img_width, img_channels))
discriminator_model = make_discriminator_model(input_shape=refiner_model.output_shape[1:])
# create combined model with frozen discriminator layers
synthetic_image_tensor = layers.Input(refiner_model.input_shape[1:])
refiner_model_output = refiner_model(synthetic_image_tensor)
combined_output = discriminator_model(refiner_model_output)
combined_model = models.Model(
inputs=synthetic_image_tensor,
outputs=[refiner_model_output, combined_output],
name='combined'
)
How do I freeze the discriminator layers in the combined model without freezing them in the discriminator-only model?
The Keras FAQ explicitly suggests the following:
refiner_model.compile(...)
discriminator_model.compile(...)
discriminator_model.trainable = False
combined_model.compile(...)
But then when I print out discriminator_model.summary(), the number of parameters has doubled?
Total params: 151,812
Trainable params: 75,906
Non-trainable params: 75,906
Then I'll get warnings about changing .trainable without recompiling, and eventually fail with this error.
I use the already trained(pre-trained) data-set for object detection using yolo+tensorflow.
My inference results are great but now I want to "add" a new class to pre-trained data-set.
There are 80 classes in pre-trained data-set how can I add my custom classes and made it 81 or 82 in total?
Inference git-hub "https://github.com/thtrieu/darkflow".
In case of transfer learning, pre-trained weights on famous datasets like 'Imagenet', 'fashion-mnist' etc are used. These datasets have defined number of classes and labels which may or may not be same as our dataset. The best practice is to add layers above the output layer of the pre-trained model output. For example in keras:
from tensorflow.keras.applications import mobilenet
from tensorflow.keras.layers import Dense, Flatten
output = mobilenet(include_top=False)
flatten = Flatten()(output)
predictions = Dense(number_of_classes, activation='softmax')(layer)
In this case you need to train(or better call it fine tune) the model using your dataset. The mobilenet network will use pretrained weights and the last layer will be only trained as per your dataset with the your defined number of classes.
You may also use:
from tensorflow.keras.applications import mobilenet
preds = mobilenet(include_top=Flase, classes=number_of_classes, weights='imagenet')
for more information you can refer: keras-applications
and these blog1, blog2
If you have already trained your model for 80 classes and need to add another class, then it would be better to re-train the model starting from previously saved checkpoints.(The network architecture should be designed for the total number of classes since the beginning cause at the output layer you will have neurons equal to the number of classes, if that is not the case you cannot add other class to the data as network has not been designed for it.) This will make use of initial training done on previous classes. The data that you are using for re-training, should now contain all the classes (including all the previous class and the new classes that you want to add). It's similar to initializing the weights from last trained checkpoint(on 80 classes) and then again train using more data (including all the classes 80 + more that you want to add) allowing back propagation through all the layers.
I am trying to use transfer learning in tensorflow. I know the high level paradigm
base_model=MobileNet(weights='imagenet',include_top=False) #imports the
mobilenet model and discards the last 1000 neuron layer.
x=base_model.output
x=GlobalAveragePooling2D()(x)
x=Dense(1024,activation='relu')(x) #we add dense layers so that the model can learn more complex functions and classify for better results.
x=Dense(1024,activation='relu')(x) #dense layer 2
x=Dense(512,activation='relu')(x) #dense layer 3
preds=Dense(120,activation='softmax')(x) #final layer with softmax activation
and then one compiles it by
model=Model(inputs=base_model.input,outputs=preds)
However i want the there to be a few other layers before the base_model.input. I want to add adversarial noise to the images that come in and a few other things. So effectively i want to know how to :
base_model=MobileNet(weights='imagenet',include_top=False) #imports the
mobilenet model and discards the last 1000 neuron layer
x = somerandomelayers(x_in)
base_model.input = x_in
x=base_model.output
x=GlobalAveragePooling2D()(x)
x=Dense(1024,activation='relu')(x) #we add dense layers so that the model can learn more complex functions and classify for better results.
x=Dense(1024,activation='relu')(x) #dense layer 2
x=Dense(512,activation='relu')(x) #dense layer 3
preds=Dense(120,activation='softmax')(x) #final layer with softmax activation
model=Model(inputs=x_in,outputs=preds)
but the line base_model.input = x_in is apparently not the way to do it as it throws can't set attribute error. How do i go about achieving the desired behavior?
You need to define input layer. It's rather straightforward, just be sure to set right shapes. For example, you can use any predefined model from Keras.
base_model = keras.applications.any_model(...)
input_layer = keras.layers.Input(shape)
x = keras.layers.Layer(...)(input_layer)
...
x = base_model(x)
...
output = layers.Dense(num_classes, activation)(x)
model = keras.Model(inputs=input_layer, outputs=output)
I'm trying to save and load the weights of my model which has merged layer. Since my model code is bit long, let me shorten it by using a simple example and pseudo-style code.
First, my model looks like this:
def some_model():
model1 = Sequential()
model1.add(...)
model2 = Sequential()
model2.add(..)
final_model = Sequential()
final_model.add(Merge[model1, model2], mode='concat')
return final_model
So, after the training, I only saved final_mode's weight.
final_model.save_weights('w_final_model.h5')
And it had no problem when I load the weight for further training or testing.
final_model = some_model()
final_model.load_weights('w_final_model.h5')
So far so good. Yet, my curiosity comes when I tried to investigate the shape of the final_model's layers.
Obviously, the final_model only has its own layers. In other words, it doesn't look carrying all the weights vectors of model1 and model2. But, it still works. I wonder how this can be possible. Or, is it only loading the weights for the final_model layers while the model1 and model2 weights are initialized again? Yet, the network's output is too good to assume that model1 and model2 weights are newly initialized. Indeed, do I need to save each model's weight separately?