I exported a faster_rcnn_resnet101 model with custom classes for serving predictions and deployed it on Cloud ML platform so that I can use Cloud ML prediction engine. Online prediction works but the results fail when I try batch prediction. It appears that the official documentation is outdated and needs an update.
I tried formatting my data in both ways mentioned here. In addition I also tried the request format mentioned here.
I also tried the steps mentioned in the google cloud blog.
Local prediction and online prediction work but the batch prediction fails. Any help will be much appreciated.
Error Log:
('Exception during running the graph: assertion failed: [Unable to decode bytes as JPEG, PNG, GIF, or BMP]\n\t [[node map/while/decode_image/cond_jpeg/cond_png/cond_gif/Assert_1/Assert (defined at /usr/local/lib/python2.7/dist-packages/google/cloud/ml/prediction/frameworks/tf_prediction_lib.py:210) = Assert[T=[DT_STRING], summarize=3, _device="/job:localhost/replica:0/task:0/device:CPU:0"](map/while/decode_image/cond_jpeg/cond_png/cond_gif/is_bmp, map/while/decode_image/cond_jpeg/cond_png/cond_gif/Assert_1/Assert/data_0)]]', 1)
Our apologies, but batch prediction is not currently supported for custom models.
Related
As tf.data augmentations are executed only on CPUs. I need a way to run certain augmentations on the TPU for an audio project.
For example,
CPU: tf.recs read -> audio crop -> noise addition.
TPU: spectogram -> Mixup Augmentation.
Most augmentations can be done as a Keras Layer on top of the model, but MixUp requires both changes in input as well as label.
Is there a way to do it using tf keras APIs.
And if there is any way we can transfer part of tf.data to run on TPU that will also be helpful.
As you have rightly mentioned and as per the Tensorflow documentation also the preprocessing of tf.data is done on CPU only.
However, you can do some workaround to preprocess your tf.data using TPU/GPU by directly using transformation function in your model with something like below code.
input = tf.keras.layers.Input((512,512,3))
x = tf.keras.layers.Lambda(transform)(input)
You can follow this Kaggle post for detailed discussion on this topic.
See the Tensorflow guide that discusses preprocessing data before the model or inside the model. By including preprocessing inside the model, the GPU is leveraged instead of the CPU, it makes the model portable, and it helps reduce the training/serving skew. The guide also has multiple recipes to get you started too. It doesn't explicitly state this works for a TPU but it can be tried.
I followed the small tutorial here: https://github.com/tensorflow/models/tree/master/official/recommendation
to train a recommendation model based on the ml-1m movielens dataset. How would I go about deploying this to start using it?
I've tried adding my own code to convert the keras model into tflite to put on firebase, but converter.convert() throws a value error. I've looked into Tensorflow serving, but the checkpoint that it outputs does not follow the format needed from what it appears. I'm not even sure how to format the input data to get recommendations.
I am new to ml and tensorflow, so I appreciate details. Thank you.
The content recommendation codelab from Firebase ML codelabs has detailed steps on how to train, convert, and deploy a TFLite recommendations model to a mobile app.
I have retrained image classification (poet) tensor flow model, and it's working fine. Now, i want to get it deployed on the google cloud platform, saw some links but not getting step by step doc which explains how retrained mode can be deployed with my own custom data (which is shelf filled with products). So, can you please provide any doc which explains the deployment process.
https://codelabs.developers.google.com/codelabs/tensorflow-for-poets/#4
Thanks
Amit
I have locally retrained the Inception model using the retrain.py file from Google Code Lab TensorFlow for Poets and want to use Google Cloud machine Learning service to make predictions.
Specifically, I want to modify the retrain.py file, so my TensorFlow application is prepared for
gcloud beta ml predict --instances=INSTANCES --model=MODEL
(i.e., prediction only; no need for Google Cloud ML training ala gloud beta ml jobs submit training).
I understand conceptually that the retrain.py file must be modified as described in Preparing a Model.
But there is no complete answer showing all the lines of code in the retrain.py file after being modified. And the popularity of Google Code Lab TensorFlow for Poets and Pete Warden’s screencasts about retraining Inception makes one expect this to be a very common example of image classification among the TensorFlow community; which means an answer will benefit many in the community.
Will someone please answer with their version of the retrain.py file after being modified as described in Preparing a Model?
Note 1:
I have researched my question to confirm it has not been answered…
… The question asked by Davide Biraghi and answered by JoshGC “Q: How predict an image in google machine learning” does not show any modifications to the retrain.py file that retrains the Inception model in Google Code Lab TensorFlow for Poets.
… The question asked by KlezFromSpace and answered by rhaertel80 (with helpful comments by Robert Lacok) “Q: Deploy Retrained inception model on Google cloud machine learning” does not show all the lines of code in the retrain.py file after being modified for: Defining outputs; Creating inputs; Supporting variable batch sizes; Using instance keys; Adding input and output collections to the graph; and Exporting (saving) the final model. (See above Preparing a Model.)
… The question asked by Vinkeet Kaushik and answered by Robert Lacok (with helpful comments by mrry) “Q: Export a basic Tensorflow model to Google Cloud ML” is not specific to the retrain.py file that retrains the Inception model in Google Code Lab TensorFlow for Poets.
Note 2:
I assume the jpeg image for which prediction is to be made is
gcloud beta ml predict --instances=INSTANCES --model=MODEL
where INSTANCES is the path to a JSON file with information about the image as per the question asked by Davide Biraghi and answered by rhaertel80 “Q: How convert a jpeg image into json file in Google machine learning”
Note 3:
I assume I will manually store the EXPORT and EXPORT.META files saved by the modified retrain.py file at the URL I use to create MODEL in Google Cloud Console.
This posting yesterday by Google's Slaven Bilac appears to be the answer.
Has anyone incorporated the ability to save a graphdef protobuf along with each checkpoint in the inception-v3 model?
Lively discussions at https://github.com/tensorflow/tensorflow/issues/616 provided some solutions to export graph protobufs, but I cannot get any of them working using the inception training model (inception_train.py).
I am trying to implement graph.util.convery_variables_to_constants, but I am failing to capture trainable variables, as I always get assertion errors: [variable] is not in graph. I am using output names from tensorboard, which may be incorrect procedure.
Any solutions to this issue would definitely be of interest to the general community, not only for learning but deployment.
Thanks!