I have locally retrained the Inception model using the retrain.py file from Google Code Lab TensorFlow for Poets and want to use Google Cloud machine Learning service to make predictions.
Specifically, I want to modify the retrain.py file, so my TensorFlow application is prepared for
gcloud beta ml predict --instances=INSTANCES --model=MODEL
(i.e., prediction only; no need for Google Cloud ML training ala gloud beta ml jobs submit training).
I understand conceptually that the retrain.py file must be modified as described in Preparing a Model.
But there is no complete answer showing all the lines of code in the retrain.py file after being modified. And the popularity of Google Code Lab TensorFlow for Poets and Pete Warden’s screencasts about retraining Inception makes one expect this to be a very common example of image classification among the TensorFlow community; which means an answer will benefit many in the community.
Will someone please answer with their version of the retrain.py file after being modified as described in Preparing a Model?
Note 1:
I have researched my question to confirm it has not been answered…
… The question asked by Davide Biraghi and answered by JoshGC “Q: How predict an image in google machine learning” does not show any modifications to the retrain.py file that retrains the Inception model in Google Code Lab TensorFlow for Poets.
… The question asked by KlezFromSpace and answered by rhaertel80 (with helpful comments by Robert Lacok) “Q: Deploy Retrained inception model on Google cloud machine learning” does not show all the lines of code in the retrain.py file after being modified for: Defining outputs; Creating inputs; Supporting variable batch sizes; Using instance keys; Adding input and output collections to the graph; and Exporting (saving) the final model. (See above Preparing a Model.)
… The question asked by Vinkeet Kaushik and answered by Robert Lacok (with helpful comments by mrry) “Q: Export a basic Tensorflow model to Google Cloud ML” is not specific to the retrain.py file that retrains the Inception model in Google Code Lab TensorFlow for Poets.
Note 2:
I assume the jpeg image for which prediction is to be made is
gcloud beta ml predict --instances=INSTANCES --model=MODEL
where INSTANCES is the path to a JSON file with information about the image as per the question asked by Davide Biraghi and answered by rhaertel80 “Q: How convert a jpeg image into json file in Google machine learning”
Note 3:
I assume I will manually store the EXPORT and EXPORT.META files saved by the modified retrain.py file at the URL I use to create MODEL in Google Cloud Console.
This posting yesterday by Google's Slaven Bilac appears to be the answer.
Related
What I'm trying to do
I'm trying to learn TensorFlow object recognition and as usual with new things, I scoured the web for tutorials. I don't want to involve any third party cloud service or web development framework, I want to learn to do it with just native JavaScript, Python, and the TensorFlow library.
What I have so far
So far, I've followed a TensorFlow object detection tutorial (accompanied by a 5+ hour video) to the point where I've trained a model in Tensorflow (python) and want to convert it to run in a browser via TensorflowJS. I've also tried other tutorials and haven't seemed to find one that explains how to do this without a third party cloud / tool and React.
I know in order to use this model with tensorflow.js my goal is to get files like:
group1-shard1of2.bin
group1-shard2of2.bin
labels.json
model.json
I've gotten to the point where I created my TFRecord files and started training:
py Tensorflow\models\research\object_detection\model_main_tf2.py --model_dir=Tensorflow\workspace\models\my_ssd_mobnet --pipeline_config_path=Tensorflow\workspace\models\my_ssd_mobnet\pipeline.config --num_train_steps=100
It seems after training the model, I'm left with:
files named checkpoint, ckpt-1.data-00000-of-00001, ckpt-1.index, pipeline.config
the pre-trained model (which I believe isn't the file that changes during training, right?) ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8
I'm sure it's not hard to get from this step to the files I need, but I honestly browsed a lot of documentation and tutorials and google and didn't see an example of doing it without some third party cloud service. Maybe it's in the documentation, I'm missing something obvious.
The project directory structure looks like this:
Where I've looked for an answer
For some reason, frustratingly, every single tutorial I've found (including the one linked above) for using a pre-trained Tensorflow model for object detection via TensorFlowJS has required the use of IBM Cloud and ReactJS. Maybe they're all copying from some tutorial they found and now all the tutorials include this, I don't know. What I do know is I'm building an Electron.js desktop app and object detection shouldn't require network connectivity assuming the compute is happening on the user's device. To clarify: I'm creating an app where the user trains the model, so it's not just a matter of one time conversion. I want to be able to train with Python Tensorflow and convert the model to run on JavaScript Tensorflow without any cloud API.
So I stopped looking for tutorials and tried looking directly at the documentation at https://github.com/tensorflow/tfjs.
When you get to the section about importing pre-trained models, it says:
Importing pre-trained models
We support porting pre-trained models from:
TensorFlow SavedModel
Keras
So I followed that link to Tensorflow SavedModel, which brings us to a project called tfjs-converter. That repo says:
This repository has been archived in favor of tensorflow/tfjs.
This repo will remain around for some time to keep history but all
future PRs should be sent to tensorflow/tfjs inside the tfjs-core
folder.
All history and contributions have been preserved in the monorepo.
Which sounds a bit like a circular reference to me, considering it's directing me to the page that just told me to go here. So at this point you're wondering well is this whole library deprecated, will it work or what? I look around in this repo anyway, into: https://github.com/tensorflow/tfjs-converter/tree/master/tfjs-converter
It says:
A 2-step process to import your model:
A python pip package to convert a TensorFlow SavedModel or TensorFlow Hub module to a web friendly format. If you already have a converted model, or are using an already hosted model (e.g. MobileNet), skip this step.
JavaScript API, for loading and running inference.
And basically says to create a venv and do:
pip install tensorflowjs
tensorflowjs_converter \
--input_format=tf_saved_model \
--output_format=tfjs_graph_model \
--signature_name=serving_default \
--saved_model_tags=serve \
/mobilenet/saved_model \
/mobilenet/web_model
But wait, are the checkpoint files I have a "TensorFlow SavedModel"? This doesn't seem clear, the documentation doesn't explain. So I google it, find the documentation, and it says:
You can save and load a model in the SavedModel format using the
following APIs:
Low-level tf.saved_model API. This document describes how to use this
API in detail. Save: tf.saved_model.save(model, path_to_dir)
The linked syntax extrapolates somewhat:
tf.saved_model.save(
obj, export_dir, signatures=None, options=None
)
with an example:
class Adder(tf.Module):
#tf.function(input_signature=[tf.TensorSpec(shape=[], dtype=tf.float32)])
def add(self, x):
return x + x
model = Adder()
tf.saved_model.save(model, '/tmp/adder')
But so far, this isn't familiar at all. I don't understand how to take the results of my training process so far (the checkpoints) to load it into a variable model so I can pass it to this function.
This passage seems important:
Variables must be tracked by assigning them to an attribute of a
tracked object or to an attribute of obj directly. TensorFlow objects
(e.g. layers from tf.keras.layers, optimizers from tf.train) track
their variables automatically. This is the same tracking scheme that
tf.train.Checkpoint uses, and an exported Checkpoint object may be
restored as a training checkpoint by pointing
tf.train.Checkpoint.restore to the SavedModel's "variables/"
subdirectory.
And it might be the answer, but I'm not really clear on what it means as far as being "restored", or where I go from there, if that's even the right step to take. All of this is very confusing to someone learning TF which is why I looked for a tutorial that does it, but again, I can't seem to find one without third party cloud services / React.
Please help me connect the dots.
You can convert your model to TensorFlowJS format without any cloud services. I have laid out the steps below.
I'm sure it's not hard to get from this step to the files I need.
The checkpoints you see are in tf.train.Checkpoint format (relevant source code that creates these checkpoints in the object detection model code). This is different from the SavedModel and Keras formats.
We will go through these steps:
Checkpoint (current) --> SavedModel --> TensorFlowJS
Converting from tf.train.Checkpoint to SavedModel
Please see the script models/research/object_detection/export_inference_graph.py to convert the Checkpoint files to SavedModel.
The code below is taken from the docs of that script. Please adjust the paths to your specific project. --input_type should remain as image_tensor.
python export_inference_graph.py \
--input_type image_tensor \
--pipeline_config_path path/to/ssd_inception_v2.config \
--trained_checkpoint_prefix path/to/model.ckpt \
--output_directory path/to/exported_model_directory
In the output directory, you should see a savedmodel directory. We will use this in the next step.
Converting SavedModel to TensorFlowJS
Follow the instructions at https://github.com/tensorflow/tfjs/tree/master/tfjs-converter, specifically paying attention to the "TensorFlow SavedModel example". The example conversion code is copied below. Please modify the input and output paths for your project. The --signature_name and --saved_model_tags might have to be changed, but hopefully not.
tensorflowjs_converter \
--input_format=tf_saved_model \
--output_format=tfjs_graph_model \
--signature_name=serving_default \
--saved_model_tags=serve \
/mobilenet/saved_model \
/mobilenet/web_model
Using the TensorFlowJS model
I know in order to use this model with tensorflow.js my goal is to get files like:
group1-shard1of2.bin
group1-shard2of2.bin
labels.json
model.json
The steps above should create these files for you, though I don't think labels.json will be created. I am not sure what that file should contain. TensorFlowJS will use model.json to construct the inference graph, and it will load the weights from the .bin files.
Because we converted a TensorFlow SavedModel to a TensorFlowJS model, we will need to load the JS model with tf.loadGraphModel(). See the tfjs converter page for more information.
Note that for TensorFlowJS, there is a difference between a TensorFlow SavedModel and a Keras SavedModel. Here, we are dealing with a TensorFlow SavedModel.
The Javascript code to run the model is probably out of scope for this answer, but I would recommend reading this TensorFlowJS tutorial. I have included a representative javascript portion below.
import * as tf from '#tensorflow/tfjs';
import {loadGraphModel} from '#tensorflow/tfjs-converter';
const MODEL_URL = 'model_directory/model.json';
const model = await loadGraphModel(MODEL_URL);
const cat = document.getElementById('cat');
model.execute(tf.browser.fromPixels(cat));
Extra notes
... Which sounds a bit like a circular reference to me,
The TensorFlowJS ecosystem has been consolidated in the tensorflow/tfjs GitHub repository. The tfjs-converter documentation lives there now. You can create a pull request to https://github.com/tensorflow/tfjs to fix the SavedModel link to point to the tensorflow/tfjs repository.
I'm looking into training an object detection network using Tensorflow, and I had a look at the TF2 Model Zoo. I noticed that there are noticeably less models there than in the directory /models/research/models/, including the MobileDet with SSDLite developed for the jetson xavier.
To clarify, the readme says that there is a MobileDet GPU with SSDLite, and that the model and checkpoints trained on COCO are provided, yet I couldn't find them anywhere in the repo.
How is one supposed to use those models?
I already have a custom-trained MobileDetv3 for image classification, and I was hoping to see a way to turn the network into an object detection network, in accordance with the MobileDetv3 paper. If this is not straightforward, training one network from scratch could be ok too, I just need to know where to even start from.
If you plan to use the object detection API, you can't use your existing model. You have to choose from a list of models here for v2 and here for v1
The documentation is very well maintained and the steps to train or validate or run inference (test) on custom data is very well explained here by the TensorFlow team. The link is meant for TensorFlow version v2. However, if you wish to use v1, the process is fairly similar and there are numerous blogs/videos explaining how to go about it
I followed the small tutorial here: https://github.com/tensorflow/models/tree/master/official/recommendation
to train a recommendation model based on the ml-1m movielens dataset. How would I go about deploying this to start using it?
I've tried adding my own code to convert the keras model into tflite to put on firebase, but converter.convert() throws a value error. I've looked into Tensorflow serving, but the checkpoint that it outputs does not follow the format needed from what it appears. I'm not even sure how to format the input data to get recommendations.
I am new to ml and tensorflow, so I appreciate details. Thank you.
The content recommendation codelab from Firebase ML codelabs has detailed steps on how to train, convert, and deploy a TFLite recommendations model to a mobile app.
I am starting to work with Intel movidius neural compute stick.
To start working, in my case, it is necessary to download pretrained models.
In the tutorials they refer to http://download.tensorflow.org/models/.
However, there is not a list that shows all the models available for download.
If the latest version of a net, lets say inception_v4, is not known, I cannot download the corresponding .tar.gz file.
Does anyone know a method to have an updated list of the .tar.gz files of the pretrained models available for download?
Thanks
The following two links may help
detection_model_zoo
TensorFlow-Slim image classification model library
keras.applications makes it easy to load models and their pretrained weights if you can use Keras
Considering that most of the posted linked outdated, I suggest looking into the TensorFlow hub (https://www.tensorflow.org/hub) for more recent pre-trained models.
I am trying to follow the transfer learning examples 1 and 2, both use a pretrained Inception v3 model. Both refer to a graph definition file, classify_image_graph_def.pb.
Where can I download the original Inception v3 .pb file?
Any help would be appreciated.
Thanks in advance.
I seems you should follow the instructions from here:
https://www.tensorflow.org/tutorials/image_retraining
to get sustainable results - the script employed within the lab loads the pre-trained Inception v3 model, removes the old top layer, and trains a new one on the images you've applied.
There is more recent version of Inception-v3 model available:
http://download.tensorflow.org/models/inception_v3_2016_08_28.tar.gz
From the official Github page
curl -O http://download.tensorflow.org/models/image/imagenet/inception-v3-2016-03-01.tar.gz