I was working on an image recognition problem. After training the model, I saved the architecture as well as weights. Now I want to use the model for extracting features from other images and perform SVM on that. For this, I want to remove the last two layers of my model and get the values calculated by the CNN and fully connected layers till then. How can I do that in Keras?
# a simple model
model = keras.models.Sequential([
keras.layers.Input((32,32,3)),
keras.layers.Conv2D(16, 3, activation='relu'),
keras.layers.Flatten(),
keras.layers.Dense(10, activation='softmax')
])
# after training
feature_only_model = keras.models.Model(model.inputs, model.layers[-2].output)
feature_only_model take a (32,32,3) for input and the output is the feature vector
If your model is subclassed - just change call() method.
If not:
if your model is complicated - wrap your model by subclassed model and change forward pass in call() method, or
if your model is simple - create model without the last layers, load weights to every layer separately
Related
I was reading the data augmentation article on Keras and they allow one to make preprocessing layer a part of the model:
model = tf.keras.Sequential([
resize_and_rescale,
data_augmentation,
layers.Conv2D(16, 3, padding="same", activation="relu"),
layers.MaxPooling2D(),
# Rest of your model
])
I'm wondering whether one or both of the resize_and_rescale and data_augmentation layers are also applied to the validation set during training?
It depends on which type of augmentation you are using. For example if you use resizing layer or a rescale layer they are applied even during inference mode, that is they would be applied to the valiation data in model.fit. For other augmentation layers like RandomFlip layer the documentation states:
During inference time, the output will be identical to input.
So you have to look up the information on the type of layer you are using. Documentation is here. From what I could gather I think only the resizing and rescaling layers remain active during inference mode.
I'm using Faster-RCNN, Yolo, and SSD models on GluonCV (mxnet) to predict on some medical images. However, the training result isn't ideal because the number of images in the dataset is small. As a result, I decide to use transfer learning and unfreeze the output layer with the 'reset_class' method to train my models. But the result is still below expectation. Thus, I'm trying to unfreeze more layers to improve the accuracy of the training result, but I couldn't find any build-in function to achieve this.
Basically, I have two questions as follow:
First, is it possible to unfreeze more layers on GluonCV?
Second, if not, is there any other way that I could use to further improve the accuracy of my result?
To be honest, I'm not sure why you believe that model parameters are frozen by default and reset_class will unfreeze an output layer. Still, if you want to unfreeze specific parameters, you need to choose them with collect_params method and set their grad_req attribute to 'write'. For example, having the following convolutional network
import mxnet as mx
class ConvNet(mx.gluon.nn.HybridSequential):
def __init__(self, n_classes, params=None, prefix=None):
super().__init__(params=params, prefix=prefix)
self.features = mx.gluon.nn.HybridSequential()
self.features.add(mx.gluon.nn.Conv2D(channels=6, kernel_size=5, padding=2,
activation='relu'))
self.add(mx.gluon.nn.MaxPool2D(pool_size=2, strides=2))
self.add(mx.gluon.nn.Flatten())
self.output = mx.gluon.nn.Dense(units=n_classes)
def hybrid_forward(self, F, x):
x = self.features(x)
return self.output(x)
net = ConvNet(10)
you can unfreeze convolution (features) block with
net.features.collect_params().setattr('grad_req', 'write')
Also, if your model is not composed of features, output, and/or other separate blocks (e.g. it's a single sequential block):
net = mx.gluon.nn.HybridSequential()
net.add(mx.gluon.nn.Conv2D(channels=6, kernel_size=5, padding=2,
activation='relu'))
net.add(mx.gluon.nn.MaxPool2D(pool_size=2, strides=2))
net.add(mx.gluon.nn.Flatten())
net.add(mx.gluon.nn.Dense(units=10))
to unfreeze the convolution block try this:
net.collect_params('conv*').setattr('grad_req', 'write')
I am trying to use efficientnet to custom train my dataset.
And I find out with all other code/data/config the same.
efficientnet.tfkeras.EfficientNetB0 can gives ~90% training/prediction accruacy and tf.keras.applications.efficientnet.EfficientNetB0 only gives ~70% accuracy.
But I guess both should be the same implementation of the efficient net, or I am missing something here?
I am using latest efficientnet and Tensorflow 2.3.0
with strategy.scope():
model = tf.keras.Sequential([
efficientnet.tfkeras.EfficientNetB0( #tf.keras.applications.efficientnet.EfficientNetB0
input_shape=(IMAGE_SIZE, IMAGE_SIZE, 3),
weights='imagenet',
include_top=False
),
L.GlobalAveragePooling2D(),
L.Dense(1, activation='sigmoid')
])
model.compile(
optimizer='adam',
loss='binary_crossentropy',
metrics=['binary_crossentropy']
)
model.summary()
I did run into the same problem for EfficientNetB4 and did encounter the following:
The number of total parameters are not equal. The trainable parameters are equal, but the non-trainable parameters aren't. The efficientnet.tfkeras has 7 fewer non-trainable parameters than the tf.keras.applications model.
The number of layers are not equal, the efficientnet.tfkeras has fewer layers than tf.keras.application model.
The different layers are at the very beginning, the most noteworthy are the normalization and rescaling layers, which are in the tf.keras.applications model, but not in the efficientnet.tfkeras model. You can observe this yourself using the model.summary() method.
When applying this layer, by using model.layers[i](array), it turn out these layers do rescale the image by dividing it by 255 and applying normalization according to:
(input_image - IMAGENET_MEAN) / square_root(IMAGENET_STD)
Thus, it turns out the image normalization is build into the model. When you perform this normalization yourself to the input image, the image will be normalized twice resulting in extremely small pixel values. The model will therefore have a hard time learning.
TLDR: Do not normalize the input image as it is build into the tf.keras.application model, input images should have values in the range 0-255.
I want to use ResNet50 with Imagenet weights.
The last layer of ResNet50 is (from here)
x = layers.Dense(1000, activation='softmax', name='fc1000')(x)
I need to keep the weights of this layer but remove the softmax function.
I want to manually change it so my last layer looks like this
x = layers.Dense(1000, name='fc1000')(x)
but the weights stay the same.
Currently I call my net like this
resnet = Sequential([
Input(shape(224,224,3)),
ResNet50(weights='imagenet', input_shape(224,224,3))
])
I need the Input layer because otherwise the model.compile says that placeholders aren't filled.
Generally there are two ways of achievieng this:
Quick way - supported functions:
To change the final layer's activation function, you can pass an argument classifier_activation.
So in order to get rid of activation all together, your module can be called like:
import tensorflow as tf
resnet = tf.keras.Sequential([
tf.keras.layers.Input(shape=(224,224,3)),
tf.keras.applications.ResNet50(
weights='imagenet',
input_shape=(224,224,3),
pooling="avg",
classifier_activation=None
)
])
This however, is not going to work if the you want a different function, that is not supported by Keras classifer_activation parameter (e. g. custom activation function).
To achieve this you can use the workaround solution:
Long way - copy the model's weights
This solution proposes copying the original model's weights onto your custom one. This approach works because apart from the activation function you are not chaning the model's architecture.
You need to:
1. Download original model.
2. Save it's weights.
3. Declare your modified version of the model (in your case, without the activation function).
4. Set the weights of the new model.
Below snippet explains this concept in more detail:
import tensorflow as tf
# 1. Download original resnet
resnet = tf.keras.Sequential([
tf.keras.layers.Input(shape=(224,224,3)),
tf.keras.applications.ResNet50(
weights='imagenet',
input_shape=(224,224,3),
pooling="avg"
)
])
# 2. Hold weights in memory:
imagenet_weights = resnet.get_weights()
# 3. Declare the model, but without softmax
resnet_no_softmax = tf.keras.Sequential([
tf.keras.layers.Input(shape=(224,224,3)),
tf.keras.applications.ResNet50(
include_top=False,
weights='imagenet',
input_shape=(224,224,3),
pooling="avg"
),
tf.keras.layers.Dense(1000, name='fc1000')
])
# 4. Pass the imagenet weights onto the second resnet
resnet_no_softmax.set_weights(imagenet_weights)
Hope this helps!
I have built an image classifier with 2 classes, say 'A' and 'B'. I have also saved this model, using model.save().
Now, after a certain time, the requirement arose to add one more class 'C'. Is it possible to load_model() and then add only one class to the previously saved model so that we have the final model with 3 classes ('A','B' and 'C'), without having to retrain the whole model, for classes 'A and 'B' again?
Can anyone help?
I have tried this:
I used vgg16 as a base model and pop out its last layer, freeze weights and added one dense layer (DL2), trained it for predicting 2 classes.
Then I added one more dense layer on top of DL2 say DL3, freeze weights and train with class C only but now its always predicting class C.
I think you should check this tutorial:
https://www.tensorflow.org/tutorials/image_retraining.
In short:
You can not take trained model, and add new classes.
You should make some additional 'finetuning', may be not retrain the model from scratch, but at least to train classifier (and some additional layers).
You can also simply change the number of output classes in the last layer and freeze weights for the remaining layer. Retrain the weights for only the last layer.
Just use a transfer learning, and create a new model.
model = VGG16(weights='imagenet',
include_top=False,
input_shape=(150, 150, 3))
model.pop()
base_model_layers = model.output
pred = Dense(11, activation='softmax')(base_model_layers)
model = Model(inputs=model.input, outputs=pred)
# Freeze the first layers, before train it
for layer in model.layers[:-2]:
layer.trainable = False