Is the TensorFlow Serving (TFS) Predict API output the same as the tf.keras.model.predict method (i.e. the outputs of the model according to the compiled metrics)?
For example, if we have a tf.keras.model compiled with BinaryAccuracy metric, will the output of the TFS predict API be a list of binary accuracy values for each one of the inputs of the predict request?
Thanks in advance!
I am not able to clearly get your question about compiled metrics and the output prediction of the model. But here's the comparision of outputs from Keras predict method and TF Serving's Predict API.
The output format of prediction for both Keras and TF Serving Predict API is similar, which emits a list of probability values of the data point belonging to each class.
Consider that you have a 10 class classification model and you're sending 4 data points to predict method, The output will be of shape 4x10, wherein for each data point the predicted result contains the probability of that data point belonging to each classes(0–9).
Here's a sample prediction
predictions = [
[8.66183618e-05 1.06925681e-05 1.40683464e-04 4.31487868e-09
7.31811961e-05 6.07917445e-06 9.99673367e-01 7.10965661e-11
9.43153464e-06 1.98050812e-10],
[6.35617238e-04 9.08200348e-10 3.23482091e-05 4.98994159e-05
7.29685112e-08 4.77315152e-05 4.25152575e-06 4.23201502e-10
9.98981178e-01 2.48882337e-04],
[9.99738038e-01 3.85520025e-07 1.05982785e-04 1.47284098e-07
5.99268958e-07 2.26216093e-06 1.17733900e-04 2.74483864e-05
3.30203284e-06 4.03360673e-06],
[3.42538192e-06 2.30619257e-09 1.29460409e-06 7.04832928e-06
2.71432992e-08 1.95419183e-03 9.96945918e-01 1.80040043e-12
1.08795590e-03 1.78136176e-07]]
You can take a look at the output of make_prediction() function in this reference to understand about how the Predict API in TF Serving works. Thank you!
Related
I am using Keras with a tensorflow backend to train some CNNs for semantic segmentation of biomedical images. I am trying to weight every pixel in my input images during training and believe I am doing so with the data generator I am passing to model.fit.
However, I am a little confused about the meaning of 'sample_weights' vs. 'sample_weight' in the documentation for model.fit.
'sample_weights' is the third optional output from your dataset or image generator - i.e. the output of the generator can either be the tuple (inputs, targets) or the tuple (inputs, targets, sample_weights). I believe this lets me create a mask that weights my samples pixel-by-pixels, but this isn't super clear from the documentation.
'sample_weight' is a separate field that seems to be pretty clearly defined as a weight you can give to every sample. If I understand, this would allow me to give more or less weight to particular images in my training set.
Do I have this right? Thanks.
I am trying to convert my CNN written with tensorflow layers to use the keras api in tensorflow (I am using the keras api provided by TF 1.x), and am having issue writing a custom loss function, to train the model.
According to this guide, when defining a loss function it expects the arguments (y_true, y_pred)
https://www.tensorflow.org/guide/keras/train_and_evaluate#custom_losses
def basic_loss_function(y_true, y_pred):
return ...
However, in every example I have seen, y_true is somehow directly related to the model (in the simple case it is the output of the network). In my problem, this is not the case. How do implement this if my loss function depends on some training data that is unrelated to the tensors of the model?
To be concrete, here is my problem:
I am trying to learn an image embedding trained on pairs of images. My training data includes image pairs and annotations of matching points between the image pairs (image coordinates). The input feature is only the image pairs, and the network is trained in a siamese configuration.
I am able to implement this successfully with tensorflow layers and train it sucesfully with tensorflow estimators.
My current implementations builds a tf Dataset from a large database of tf Records, where the features is a dictionary containing the images and arrays of matching points. Before I could easily feed these arrays of image coordinates to the loss function, but here it is unclear how to do so.
There is a hack I often use that is to calculate the loss within the model, by means of Lambda layers. (When the loss is independent from the true data, for instance, and the model doesn't really have an output to be compared)
In a functional API model:
def loss_calc(x):
loss_input_1, loss_input_2 = x #arbirtray inputs, you choose
#according to what you gave to the Lambda layer
#here you use some external data that doesn't relate to the samples
externalData = K.constant(external_numpy_data)
#calculate the loss
return the loss
Using the outputs of the model itself (the tensor(s) that are used in your loss)
loss = Lambda(loss_calc)([model_output_1, model_output_2])
Create the model outputting the loss instead of the outputs:
model = Model(inputs, loss)
Create a dummy keras loss function for compilation:
def dummy_loss(y_true, y_pred):
return y_pred #where y_pred is the loss itself, the output of the model above
model.compile(loss = dummy_loss, ....)
Use any dummy array correctly sized regarding number of samples for training, it will be ignored:
model.fit(your_inputs, np.zeros((number_of_samples,)), ...)
Another way of doing it, is using a custom training loop.
This is much more work, though.
Although you're using TF1, you can still turn eager execution on at the very beginning of your code and do stuff like it's done in TF2. (tf.enable_eager_execution())
Follow the tutorial for custom training loops: https://www.tensorflow.org/tutorials/customization/custom_training_walkthrough
Here, you calculate the gradients yourself, of any result regarding whatever you want. This means you don't need to follow Keras standards of training.
Finally, you can use the approach you suggested of model.add_loss.
In this case, you calculate the loss exaclty the same way I did in the first answer. And pass this loss tensor to add_loss.
You can probably compile a model with loss=None then (not sure), because you're going to use other losses, not the standard one.
In this case, your model's output will probably be None too, and you should fit with y=None.
I'm using a pre-trained deeplab model (from here) to obtain segmentations for an input image. I'm able to obtain the sematic labels (i.e. SemanticPredictions) which is argmax applied to logits (link).
I was wondering if there is an easy way to obtain the logits before argmax? I was hoping to find the output tensor name and simply pass it into my tfsession
as in the following:
tf_session.run(
self.OUTPUT_TENSOR_NAME,
feed_dict={self.INPUT_TENSOR_NAME: [np.asarray(input_image)]})
But I have not been able to locate such tensor name in the code that reveals the logits, or softmax outputs.
For a model trained from MobileNet_V2 setting self.OUTPUT_TENSOR_NAME = 'ResizeBilinear_2:0' retrieves the logits before the argmax is performed.
I suspect this is the same for xception, but have not verified it.
I arrived at this answer by loading my model in tensorflow. Then, printing the name of all layers in the loaded graph. Finally, I took the name of the final output layer before the last 'ArgMax' layer and ran some inferencing using that.
Here is a link to a stackoverflow question on printing the names of the layers in a graph. I found the answer by Ted to be most helpful.
By the way, the output layers of DeeplabV3 models does not apply SoftMax. So you cannot simply take the raw value of the elements of output vectors as a confidence.
I am trying to build a regression model, for which I have a nominal variable with very high cardinality. I am trying to get the categorical embedding of the column.
Input:
df["nominal_column"]
Output:
the embeddings of the column.
I want to use the op of the embedding column alone since I would require that as a input to my traditional regression model. Is there a way to extract that output alone.
P.S I am not asking for code, any suggestion on the approach would be great.
If the embedding is part of the model and you train it, then you can use functional API of keras to get output of any intermediate operation in your graph:
x=Input((number_of_categories,))
y=Embedding(parameters_of_your_embeddings)(x)
output=Rest_of_your_model()(y)
model=Model(inputs=[x],outputs=[output,y])
if you do it before you train the model, you'll have to define custom loss function, that deals only with part of the output. The other way is to train the model with just one output, then create identical model with two outputs and set the weights of the second model from the trained one.
If you want to get the embedding matrix from your model, you can just use method get_weights of the embedding layer which returns the weights in numpy array.
I am using tensorflow 1.3.0 to train a CNN classification model. However I need to get access to the prelogits layer to evaluate my method (i.e. while this is casted as a classification problem, the method is not a classification problem but is used to extract CNN features, i.e. to produce a point in an N-dimensional vector space for an input image test)
I am using both the dataset API (with TFRecord files) and the estimator API to train the model. However, I don't see how I can get access/return the prelogits value using the Estimator API, i.e. estimator.train(), .evaluate() or .predict() since model_fn() needs to return a specific tf.estimator.EstimatorSpec object.
Previously (i.e. using the standard sess=tf.Session() method) I could train the model and get access to the prelogits layer while training (or by loading the model after training) and feed the network with a specific input to get the specific layer output with a sess.run(specific_layer) as long as the layer was named specific_layer.
I have tried to use the prediction output of EstimatorSpec but it did not work. Any ideas/suggestions?