I am reading tensorflow tutorial on saving and restoring model and came across the following statement:
If assets need to be saved and written or copied to disk,
they can be provided when the first MetaGraphDef is added. If multiple
MetaGraphDefs are associated with an asset of the same name,
only the first version is retained.
What does assets in this context mean?
Also another paragraphs says:
We provide a Python implementation of the SavedModel builder.
The SavedModelBuilder class provides functionality to save multiple MetaGraphDefs.
A MetaGraph is a dataflow graph, plus its associated variables, assets,
and signatures. A MetaGraphDef is the protocol buffer representation of
a MetaGraph. A signature is the set of inputs to and outputs from a graph.
What is dataflow graph and how is it different from graph?
Here is the tutorial
TL;DR
All you need to know when saving a Tensorflow model is that there are two files that are created (potentially more if you're using checkpoints):
file
file.meta
you save 'file' and restore 'file.meta'.
More info here: Tensorflow: how to save/restore a model?
#########
More to your question:
what you define in Tensorflow before you run your session is called a graph.
When you are saving your graph, a MetaGraph is created. This is the graph itself, and all the other metadata necessary for computations in this graph, as well as some user info that can be saved and version specification.
Assets are external files, such as vocabularies that were created with the graph.
Related
I am trying to modify a tensorflow project so that it becomes compatible with TPU.
For this, I started with the code explained on this site.
Here COCO dataset is downloaded and first its features are extracted using InceptionV3 model.
I wanted to modify this code so that it supports TPU.
For this, I added the mandatory code for TPU as per this link.
Withe TPU strategy scope, I created the InceptionV3 model using keras library and loaded model with ImageNet weights as per existing code.
Now, since TPU needs data to be stored on Google Cloud storage, I created a tf records file using tf.Example with the help of this link.
Now, I tried to create this file in several ways so that it will have the data that TPU will find through TFRecordDataset.
At first I directly added image data and image path to the file and uploaded it to GCP bucket but while reading this data, I realized that this image data is not useful as it does not contain shape/size information which it will need and I had not resized it to the required dimension before storage. This file size became 2.5GB which was okay.
Then I thought lets only keep image path at cloud, so I created another tf records file with only image path, then I thought that this may not be an optimized code as TPU will have to open the image individually resize it to 299,299 and then feed to model and it will be better if I have image data through .map() function inside TFRecordDataset, so I again tried, this time by using this link, by storing R, G and B along with image path inside tf records file.
However, now I see that the size of tf records file is abnormally large, like some 40-45GB and ultimately, I stopped the execution as my memory was getting filled up on Google Colab TPU.
The original size of COCO dataset is not that large. It almost like 13GB.. and from that the dataset is being created with only first 30,000 records. so 40GB looks weird number.
May I know what is the problem with this way of feature storage? Is there any better way to store image data in TF records file and then extract through TFRecordDataset.
I think the COCO dataset processed as TFRecords should be around 24-25 GB on GCS. Note that TFRecords aren't meant to act as a form of compression, they represent data as protobufs so it can be optimally loaded into TensorFlow programs.
You might have more success if you refer to: https://cloud.google.com/tpu/docs/coco-setup (corresponding script can be found here) for converting COCO (or a subset) into TFRecords.
Furthermore, we have implemented detection models for COCO using TF2/Keras optimized for GPU/TPU here which you might find useful for optimal input pipelines. An example tutorial can be found here. Thanks!
Tensorflow has several types of model formats:
TensorFlow SavedModel 2. Frozen Model 3. Session Bundle 4. Tensorflow Hub module
How can you distinguish between them on-disk? (to later use with tensorflowjs-converter)
And how is each model created?
Yup, there are a LOT of different model types, and they all have good reasons. I'm not going to claim that I have perfect clarity of each, but here's what I know (I think I know).
The .pb file: PB stands for protobuff or Protocol Buffer. This is the model structure, generally without the trained weights and is stored in a binary format.
The .pbtxt file: Nonbinary of the pb file for human reading.
Protobuff files that aren't frozen will need a checkpoint .ckpt file, too. The Checkpoint file is the missing set of weights that the pb needs.
The .h5 file: The model + weights from a Keras save
The .tflite file would be a TensorflowLite model
Frozen Model: A frozen model combines the pb with the weights file, so you don't have to manage two of them. Usually, this means adding the word frozen to the filename. I'm sure this can be inferred when loading the file, but on disk they are a bit more on the honor system and no ckpt file. This strips out extraneous graph info; it's basically like the "Production Ready" version of the model.
Session Bundle: Are a directory. They are nolonger used, and rare.
Tensorflow Hub Module: These are pre-existing popular models that are very likely already exported to TFJS and don't require you to manually convert them. I assume they are supported for Google's benefit, more so than ours. But it's nice to know if you use hub, you can always convert it.
A multi-exported grouping of files looks like this image. From here, you can see quite a few that you could turn into TFJS.
I'm newest to tensorflow serving. Still confused with servable after read official tutorial. It seems like a servable is a tensorflow session, but has versions. It also looks like a model, but a servable may also correspond to a fraction of a model.
So what is servable exactly?
I'll try to answer it in a simple way.
A Servable, as per my understanding, is nothing but a Directory generated by Saving the Model, either using SavedModelBuilder or using export_saved_model. The contents of a Servable are shown below.
assets/
assets.extra/
variables/
variables.data-?????-of-?????
variables.index
saved_model.pb
If we use SavedModelBuilder to Save a Model, we need to explicitly mention the Version Numbers. Suppose if the Name of the Model is export, then we should mention the Export Directory as /usr/local/google/home/mothukuru/Jupyter_Notebooks/export/1 or
/usr/local/google/home/mothukuru/Jupyter_Notebooks/export/2, etc..
That is, we need to explicitly mention the version numbers.
If we use export_saved_model, versioning will be taken care automatically, mostly, based on time stamps. Example,
/usr/local/google/home/mothukuru/Jupyter_Notebooks/export/1554294699
/usr/local/google/home/mothukuru/Jupyter_Notebooks/export/1554736143
So, to answer your questions:
Q1. It seems like a servable is a tensorflow session, but has versions.
A1. Yes, it is a Session which has Versions. In that Session, we run the Model and we Save it
Q2. It also looks like a model
A2. Yes, it is a Model, which is Saved using SavedModelBuilder or using export_saved_model.
Q3. But a servable may also correspond to a fraction of a model.
A3. Yes. This Answer has 2 parts.
i. saved_model.pb comprises of Training Graph and Inference Graph. Inference Graph is the Key to TF Serving.
ii. There are 2 folders named assets. The folder assets contains auxiliary files such as vocabularies, etc. That also will be useful during Serving.
There's a fairly clear difference between a model and a frozen model. As described in model_files, relevant part: Freezing
...so there's the freeze_graph.py script that takes a graph definition and a set of checkpoints and freezes them together into a single file.
Is a "saved_model" most similar to a "frozen_model" (and not a saved
"model_checkpoint")?
Is this defined somewhere in docs I'm missing?
In prior versions of tensorflow we would save and restore model
weights, but this seems to be in context of a "model_checkpoint" not
a "saved_model", is that still correct?
I'm asking more for the design overview here, not implementation specifics.
Checkpoint file only contains variables for specific model and should be loaded with either exactly same, predefined graph or with specific assignment_map to load only chosen variables. See https://www.tensorflow.org/api_docs/python/tf/train/init_from_checkpoint
Saved model is more broad cause it contains graph that can be loaded within a session and training could be continued. Frozen graph, however, is serialized and could not be used to continue training.
You can find all the info here https://www.tensorflow.org/guide/saved_model
When training in model folder there are auto saved meta graph files.
So what's the diff between graph and meta graph.
If I want to load model and do inference witout building graph from scratch,
use tf.train.import_meta_grah is fine?
Well, I find the answer from
https://www.tensorflow.org/versions/r0.9/how_tos/meta_graph/index.html
and
How to load several identical models from save files into one session in Tensorflow
And you can also refer to this:
https://github.com/tensorflow/tensorflow/issues/4658