There are many examples about how to do fine-tuning with tensorflow. Almost all these examples are try to resize our images to the specified size that the existing model needs. Like for example, 224×224 is the input size that vgg19 needs. However, in keras, we can change the input size by setting the include_top to false:
base_model = VGG19(include_top=False, weights="imagenet", input_shape=(input_size, input_size, input_channels))
Then we do not have to fix the image size to be 224×224 anymore. Can we do such kind of fine-tuning by using official pre-trained models in tensorflow? I cannot find the solutions up till now, anyone help me?
Yes, it is possible to do this kind of fine-tuning. You would just have to ensure that you also fine-tune some of the first few layers (to account for changed input) of the original network in addition to the last few layers (to account for changed output).
I work with TensorFlow using Keras. If you are open to that, then there is a code snippet that shows the general fine-tuning flow here:
https://keras.io/applications/
Specifically, I had to write the following code to make it work for my case:
#img_width,img_height is the size of your new input, 3 is the number of channels
input_tensor = Input(shape=(img_width, img_height, 3))
base_model =
keras.applications.vgg19.VGG19(include_top=False,weights='imagenet', input_tensor=input_tensor)
#instantiate whatever other layers you need
model = Model(inputs=base_model.inputs, outputs=predictions)
#predictions is the new logistic layer added to account for new classes
Hope this helps.
Related
This is my first time doing image classification, I followed this tutorial:
https://www.tensorflow.org/tutorials/images/classification
I'm wondering, how do I take that model, and actually use it to make predictions?
I would just to put one image into the model, and would ideally like to get a prediction % of whether it thinks its a dog or a cat.
I saved the model using:
model.save(my_model.h5)
But am really lost at the next steps.
There's another Tensorflow tutorial which uses model.predict() specifically: Basic classification: Classify images of clothing
Not sure if my code is correct all the way but I tried to extend the prediction part of the cats/dogs tutorial using model.predict_generator() though I can't seem to entirely understand the results I get. Adapted code from this second tutorial: Tutorial on using Keras flow_from_directory and generators
# Preparing the testing dataset
test_dir = os.path.join(os.getcwd(), 'cat_dog_testing') # directory with test images
test_image_generator = ImageDataGenerator(rescale=1./255) # rescaling pixels 0 to 1
test_generator = test_image_generator.flow_from_directory(batch_size=6,
directory=test_dir,
shuffle=False,
target_size=(IMG_HEIGHT,IMG_WIDTH),
class_mode=None)
STEP_SIZE_TEST=test_generator.n//test_generator.batch_size
test_generator.reset()
pred=model_new.predict_generator(test_generator, steps=STEP_SIZE_TEST, verbose=1)
I built a tensorflow image classification workflow so that you can both train and classify images with no code. It's on FlyteHub if you want to see it
https://flytehub.org/trainandclassifyimages
Happy to collaborate if you have improvements you want to make to the codebase :)
I'm dealing with Keras functional API.
Specifically for my experiments, I'm using Keras resnet50 model obtained with:
model = resnet50.ResNet50(weights='imagenet')
Obviously, to get the final output of the network we need to feed a value to the placeholder input_1.
My question is, can I somehow start inferencing this graph from the relu layer which is depicted at the bottom of the picture below, provided that I feed a value of the appropriate dimensions into it?
I tried to achieve this with Keras functions. Something like:
self.inp = model.input
self.outputs = [layer.output for layer in model.layers]
self.functor = K.function([self.inp, K.learning_phase()], [self.outputs[6], self.outputs[17]])
But this approach will not work, because again to inference any output I need to feed value into tensor.
Is recreating graph from scratch my best option here?
Thanks
If I got you right, you can just specify input and output nodes
base_model = tf.keras.applications.ResNet50(weights='imagenet')
inference_model = tf.keras.Model(inputs=base_model.input, outputs=base_model.get_layer('any_layer_name').output)
You can set the output to any layer name
I would like to fine tune the VGG16 model using my own grayscale images. I know I can fine tune/add my own top layers by doing something like:
base_model = keras.applications.vgg16.VGG16(include_top=False, weights='imagenet', input_tensor=None, input_shape=(im_height,im_width,channels))
but only when channels = 3 according to the documentation.
I have thought of simply adding two redundant channels to my image, but this seems like a waste of computation/could make the classification worse. I could also replicate the same image across three channels, but I am similarly unsure of how it would preform.
Keras pre-trained models have trained on color images and if you want to use their full power, you should use color images for fine-tuning. However, if you have grayscale images you can still use these pre-trained models by repeating your grayscale image over three channels. But obviously, it will not as well as using color images as input.
The VGG keras model uses the function: keras.applications.imagenet_utils._obtain_input_shape.
This function was tailored for ImageNet data thus it enforces the input channel to be 3. One possible workaround will be to copy the VGG16 module and replace the line:
input_shape = _obtain_input_shape(input_shape, default_size=224, min_size=48, data_format=K.image_data_format(), include_top=include_top)
with:
input_shape = (im_height, im_width, 1)
As a side note, you will not be able to load ImageNet weights since your input space has changed and the first layer convolutions will not match.
I am still relatively new to the world of Deep Learning. I wanted to create a Deep Learning model (preferably using Tensorflow/Keras) for image anomaly detection. By anomaly detection I mean, essentially a OneClassSVM.
I have already tried sklearn's OneClassSVM using HOG features from the image. I was wondering if there is some example of how I can do this in deep learning. I looked up but couldn't find one single code piece that handles this case.
The way of doing this in Keras is with the KerasRegressor wrapper module (they wrap sci-kit learn's regressor interface). Useful information can also be found in the source code of that module. Basically you first have to define your Network Model, for example:
def simple_model():
#Input layer
data_in = Input(shape=(13,))
#First layer, fully connected, ReLU activation
layer_1 = Dense(13,activation='relu',kernel_initializer='normal')(data_in)
#second layer...etc
layer_2 = Dense(6,activation='relu',kernel_initializer='normal')(layer_1)
#Output, single node without activation
data_out = Dense(1, kernel_initializer='normal')(layer_2)
#Save and Compile model
model = Model(inputs=data_in, outputs=data_out)
#you may choose any loss or optimizer function, be careful which you chose
model.compile(loss='mean_squared_error', optimizer='adam')
return model
Then, pass it to the KerasRegressor builder and fit with your data:
from keras.wrappers.scikit_learn import KerasRegressor
#chose your epochs and batches
regressor = KerasRegressor(build_fn=simple_model, nb_epoch=100, batch_size=64)
#fit with your data
regressor.fit(data, labels, epochs=100)
For which you can now do predictions or obtain its score:
p = regressor.predict(data_test) #obtain predicted value
score = regressor.score(data_test, labels_test) #obtain test score
In your case, as you need to detect anomalous images from the ones that are ok, one approach you can take is to train your regressor by passing anomalous images labeled 1 and images that are ok labeled 0.
This will make your model to return a value closer to 1 when the input is an anomalous image, enabling you to threshold the desired results. You can think of this output as its R^2 coefficient to the "Anomalous Model" you trained as 1 (perfect match).
Also, as you mentioned, Autoencoders are another way to do anomaly detection. For this I suggest you take a look at the Keras Blog post Building Autoencoders in Keras, where they explain in detail about the implementation of them with the Keras library.
It is worth noticing that Single-class classification is another way of saying Regression.
Classification tries to find a probability distribution among the N possible classes, and you usually pick the most probable class as the output (that is why most Classification Networks use Sigmoid activation on their output labels, as it has range [0, 1]). Its output is discrete/categorical.
Similarly, Regression tries to find the best model that represents your data, by minimizing the error or some other metric (like the well-known R^2 metric, or Coefficient of Determination). Its output is a real number/continuous (and the reason why most Regression Networks don't use activations on their outputs). I hope this helps, good luck with your coding.
In classify_image.py, the input image is fed with a loaded image in
predictions = sess.run(softmax_tensor,{'DecodeJpeg/contents:0': image_data})
What if I want to add new layers to the inception model and train the whole model again? Are the variables loaded from classify_image_graph_def.pb trainable? I saw that freeze_graph.py used convert_variables_to_constants to produce freezed graph. So can those loaded weights be trained again, are they constants? And how can I connect the input('shuffle_batch:0') to the inception model to the output of tf.train.shuffle_batch?
The model used in classify_image.py has its variables frozen into constants, and doesn't have any gradient ops, so it's not easy to turn it back into something trainable. You can see how we remove one layer and replace it with something trainable here:
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/image_retraining/retrain.py
It's hard to generalize though. You'd be better off looking at some examples of fine-tuning here:
https://github.com/tensorflow/models/tree/master/inception#how-to-fine-tune-a-pre-trained-model-on-a-new-task