I have trained a model for classification using TensorFlow slim model vgg, using CASIA(a face recognition dataset) as training dataset.
I want to test the model by using LFW dataset, it is a face matching task. so I need to extract the net features like fc7/fc8, not the softmax layer, and compare the distance between the features, to determine whether they are the same person.
How can I extract the features of a slim model?
Here is part of the training code.
import tensorflow as tf
from tensorflow.contrib.slim.python.slim.nets import vgg
slim = tf.contrib.slim
FLAGS = tf.app.flags.FLAGS
def tower_loss(scope):
images, labels = read_and_decode()
with slim.arg_scope(vgg.vgg_arg_scope()):
logits, end_points = vgg.vgg_16(images, num_classes=FLAGS.num_classes)
_ = cal_loss(logits, labels)
losses = tf.get_collection('losses', scope)
total_loss = tf.add_n(losses, name='total_loss')
return total_loss
You can try using tf.get_default_graph().get_tensor_by_name("VGG16/fc16:0") or whatever tensor name of the specific feature you want to extract.
To verify the name of the tensors you are extracting, you can try
for operation in graph.get_operations():
print operation.values()
Remember to put :0 at the end of the names as they indicate the item you're retrieving is a tensor.
Get end_points of slim model and extract the feature.
Related
I want to use a pre-trained model and add a segmentation head at the end of that, but the problem is that I just have the 'frozen_inference_graph.pb'. These are the files that I have from the model:
I have tried several ways:
1. loading the pre-trained model into a Keras model:
It seems to be impossible with the files that I have. It just gives me an AutoTrackable object instead of a model.
2. Accessing the Tensor Objects of the frozen model and make the model with tensors:
I found out how to access the tensors but couldn't make a Keras model with Tensor objects.
with self.graph.as_default():
graph = tf.import_graph_def(graph_def, name='')
graph = tf.compat.v1.import_graph_def(graph_def)
tf.compat.v1.Graph.as_default(graph)
self.sess = tf.Session(graph=self.graph)
self.tensors = [tensor for op in tf.compat.v1.get_default_graph().get_operations() for tensor in op.values()]
Here I can get the tensors but I can't use the tensors in the model:
model = tf.keras.models.Model(inputs=self.tensors[0], outputs=self.tensors[-1])
Is there any way to convert this frozen graph to a Keras model?
Or If there is another approach which I can train the model, I would be glad to know.
P.S. The pre-trained model is 'ssd_mobilenet_v3_small_coco_2020_01_14' which can be found Here.
You can use two methods:
The file 'frozen_inference_graph.pb' contains all necessary information about the weights and the model architecture. Use the following snippet to read the model and add a new layer:a
customModel = tf.keras.models.load_model('savedModel')
# savedModel is the folder with .pb data
pretrainedOutput = customModel.layers[-1].output
newOutput = tf.keras.layers.Dense(2)(pretrainedOutput) # change layer as needed
new_model = tf.keras.Model(inputs=customModel.inputs, outputs=[newOutput])
# create a new model with input of old model and new output tensors
where 'savedModel' is the name of the folder with 'frozen_inference_graph.pb' and other meta data. See details about using .pb files and finetuning the custom models in the TFguide.
Try using .meta file with model architecture and .ckpt to restore the weights in TF 1.x:
with tf.Session() as sess:
new_saver = tf.train.import_meta_graph('my_test_model-1000.meta')
new_saver.restore(sess, tf.train.latest_checkpoint('./'))
Refer to the tutorial on how to load and customize restored models in TF 1.x.
I am using keras to build a multi-output classification model. My dataset is such as
[x1,x2,x3,x4,y1,y2,y3]
x1,x2,x3 are the features, and y1,y2,y3 are the labels, the y1,y2,y3 are multi-classes.
And I already built a model (I ingore some hidden layers):
def baseline_model(input_dim=23,output_dim=3):
model_in = Input(shape=(input_dim,))
model = Dense(input_dim*5,kernel_initializer='uniform',input_dim=input_dim)(model_in)
model = Activation(activation='relu')(model)
model = Dropout(0.5)(model)
...................
model = Dense(output_dim,kernel_initializer='uniform')(model)
model = Activation(activation='sigmoid')(model)
model = Model(model_in,model)
model.compile(optimizer='adam',loss='binary_crossentropy', metrics=['accuracy'])
return model
And then I try to use the method of keras to make it support classification:
estimator = KerasClassifier(build_fn=baseline_model)
estimator.fit()
estimator.predict(df[0:10])
But I found that the result is not multi-output, only one dimension is output.
[0,0,0,0,0,0,0,0,0,0]
So for the multi-output classification problem, we can not use KerasClassifier function to learn it?
You do not need to wrap the model in KerasClassifier. That wrapper is so that you can use the Keras model with Scikit-Learn. The type of model (classifier, regression, multiclass classifier, etc) is ultimately determined by the shape and activation of the final layer of your model.
You can simply use model.fit() function that is part of Keras. Make sure that you pass the data into the function. You can see more info on the fit function here: https://keras.io/models/model/#fit
Also your loss is setup as binary_crossentropy. For a multi-class problem you will want to use categorical_crossentropy.
model.compile(optimizer='adam',loss='categorical_crossentropy', metrics=['accuracy'])
This model isn't really what Keras refers to as multi-output as far as I can tell. With multi-output you are trying to get the output from several different layers and possibly apply different loss functions to them.
Base on the setup in your question you would be able to use the Keras Sequential model instead of the Functional model if you wanted. Keras recommends using the Sequential model if you can because its simpler. https://keras.io/getting-started/sequential-model-guide/
I'm building image processing network in tensorflow and I want to make use of texture loss. Texture loss seems simple to implement if you have pretrained model loaded.
I'm using TF to build the computational graph for my model and I want to incorporate Keras.application.VGG19 model to get output from layer 'block4_conv4'.
The problem is: I have two TF tensors target and result from my main model, how to feed them into keras VGG19 in the same session to compute their diff and use it in main loss for my model?
It seems following code does the trick
with tf.variable_scope("") as scope:
phi_func = VGG19(include_top=False, weights=None, input_shape=(128, 128, 3))
text_1 = phi_func(predicted)
scope.reuse_variables()
text_2 = phi_func(x)
text_loss = tf.reduce_mean((text_1 - text_2)**2)
right after session created I call phi_func.load_weights(path) to initiate weights
I have a trained Tensorflow model and weights vector which have been exported to protobuf and weights files respectively.
How can I convert these to JSON or YAML and HDF5 files which can be used by Keras?
I have the code for the Tensorflow model, so it would also be acceptable to convert the tf.Session to a keras model and save that in code.
I think the callback in keras is also a solution.
The ckpt file can be saved by TF with:
saver = tf.train.Saver()
saver.save(sess, checkpoint_name)
and to load checkpoint in Keras, you need a callback class as follow:
class RestoreCkptCallback(keras.callbacks.Callback):
def __init__(self, pretrained_file):
self.pretrained_file = pretrained_file
self.sess = keras.backend.get_session()
self.saver = tf.train.Saver()
def on_train_begin(self, logs=None):
if self.pretrian_model_path:
self.saver.restore(self.sess, self.pretrian_model_path)
print('load weights: OK.')
Then in your keras script:
model.compile(loss='categorical_crossentropy', optimizer='rmsprop')
restore_ckpt_callback = RestoreCkptCallback(pretrian_model_path='./XXXX.ckpt')
model.fit(x_train, y_train, batch_size=128, epochs=20, callbacks=[restore_ckpt_callback])
That will be fine.
I think it is easy to implement and hope it helps.
Francois Chollet, the creator of keras, stated in 04/2017 "you cannot turn an arbitrary TensorFlow checkpoint into a Keras model. What you can do, however, is build an equivalent Keras model then load into this Keras model the weights"
, see https://github.com/keras-team/keras/issues/5273 . To my knowledge this hasn't changed.
A small example:
First, you can extract the weights of a tensorflow checkpoint like this
PATH_REL_META = r'checkpoint1.meta'
# start tensorflow session
with tf.Session() as sess:
# import graph
saver = tf.train.import_meta_graph(PATH_REL_META)
# load weights for graph
saver.restore(sess, PATH_REL_META[:-5])
# get all global variables (including model variables)
vars_global = tf.global_variables()
# get their name and value and put them into dictionary
sess.as_default()
model_vars = {}
for var in vars_global:
try:
model_vars[var.name] = var.eval()
except:
print("For var={}, an exception occurred".format(var.name))
It might also be of use to export the tensorflow model for use in tensorboard, see https://stackoverflow.com/a/43569991/2135504
Second, you build you keras model as usually and finalize it by "model.compile". Pay attention that you need to give you define each layer by name and add it to the model after that, e.g.
layer_1 = keras.layers.Conv2D(6, (7,7), activation='relu', input_shape=(48,48,1))
net.add(layer_1)
...
net.compile(...)
Third, you can set the weights with the tensorflow values, e.g.
layer_1.set_weights([model_vars['conv7x7x1_1/kernel:0'], model_vars['conv7x7x1_1/bias:0']])
Currently, there is no direct in-built support in Tensorflow or Keras to convert the frozen model or the checkpoint file to hdf5 format.
But since you have mentioned that you have the code of Tensorflow model, you will have to rewrite that model's code in Keras. Then, you will have to read the values of your variables from the checkpoint file and assign it to Keras model using layer.load_weights(weights) method.
More than this methodology, I would suggest to you to do the training directly in Keras as it claimed that Keras' optimizers are 5-10% times faster than Tensorflow's optimizers. Other way is to write your code in Tensorflow with tf.contrib.keras module and save the file directly in hdf5 format.
Unsure if this is what you are looking for, but I happened to just do the same with the newly released keras support in TF 1.2. You can find more on the API here: https://www.tensorflow.org/api_docs/python/tf/contrib/keras
To save you a little time, I also found that I had to include keras modules as shown below with the additional python.keras appended to what is shown in the API docs.
from tensorflow.contrib.keras.python.keras.models import Sequential
Hope that helps get you where you want to go. Essentially once integrated in, you then just handle your model/weight export as usual.
I am using TensorFlow to make predictions on time-series data. So it is like I have 50 tags and I want to find out the next possible 5 tags.
As shown in the following picture, I want to make it like the 4th structure.
I went through the tutorial demo: Recurrent Neural Networks
But I found it can provide like the 5th one in the above picture, which is different.
I am wondering which model could I use? I am thinking of the seq2seq models, but not sure if it is the right way.
You are right that you can use a seq2seq model. For brevity I've written up an example of how you can do it in Keras which also has a Tensorflow backend. I've not run the example so it might need tweaking. If your tags are one-hot you need to use cross-entropy loss instead.
from keras.models import Model
from keras.layers import Input, LSTM, RepeatVector
# The input shape is your sequence length and your token embedding size
inputs = Input(shape=(seq_len, embedding_size))
# Build a RNN encoder
encoder = LSTM(128, return_sequences=False)(inputs)
# Repeat the encoding for every input to the decoder
encoding_repeat = RepeatVector(5)(encoder)
# Pass your (5, 128) encoding to the decoder
decoder = LSTM(128, return_sequences=True)(encoding_repeat)
# Output each timestep into a fully connected layer
sequence_prediction = TimeDistributed(Dense(1, activation='linear'))(decoder)
model = Model(inputs, sequence_prediction)
model.compile('adam', 'mse') # Or categorical_crossentropy
model.fit(X_train, y_train)