Tensorflow has several types of model formats:
TensorFlow SavedModel 2. Frozen Model 3. Session Bundle 4. Tensorflow Hub module
How can you distinguish between them on-disk? (to later use with tensorflowjs-converter)
And how is each model created?
Yup, there are a LOT of different model types, and they all have good reasons. I'm not going to claim that I have perfect clarity of each, but here's what I know (I think I know).
The .pb file: PB stands for protobuff or Protocol Buffer. This is the model structure, generally without the trained weights and is stored in a binary format.
The .pbtxt file: Nonbinary of the pb file for human reading.
Protobuff files that aren't frozen will need a checkpoint .ckpt file, too. The Checkpoint file is the missing set of weights that the pb needs.
The .h5 file: The model + weights from a Keras save
The .tflite file would be a TensorflowLite model
Frozen Model: A frozen model combines the pb with the weights file, so you don't have to manage two of them. Usually, this means adding the word frozen to the filename. I'm sure this can be inferred when loading the file, but on disk they are a bit more on the honor system and no ckpt file. This strips out extraneous graph info; it's basically like the "Production Ready" version of the model.
Session Bundle: Are a directory. They are nolonger used, and rare.
Tensorflow Hub Module: These are pre-existing popular models that are very likely already exported to TFJS and don't require you to manually convert them. I assume they are supported for Google's benefit, more so than ours. But it's nice to know if you use hub, you can always convert it.
A multi-exported grouping of files looks like this image. From here, you can see quite a few that you could turn into TFJS.
Related
I have a Tensorflow model trained in Python, exported to a .pb file and then used with Tensorflow Serving.
I have written a custom op that greatly speeds up the inference of some operators in this Tensorflow model, but only works for inference -- I can't use this custom op during training time.
I am wondering if it's possible for me to use this custom op with the .pb file in Tensorflow serving. I figure I will probably have to edit the .pb file such that it uses my custom op in place of the original op, and Tensorflow serving should then go about looking for the custom op implementation which I can link against its runtime.
So -- how does one go about modifying a Tensorflow .pb file and swap out operators? Are there example codes doing this that I can refer to?
Your best bet, if you for some reason can't train with the original ops, is probably proto surgery. I would look for some tools that let you convert a proto to ascii format, modify it, and convert it back to binary format. I found this gist of someone doing just that for saved model. You could then write tooling on top to try to replace with your custom op.
Does a tflite file contain data about the model architecture? A graph that shows what operations there where between the weights and features and biases, what kind of layers (linear or convolutional etc), size of layers, and what activation functions are there in-between the layers?
For example a graph you get with graphviz, that contains all the information, or does a tflite file only contain the final weights of the model after training?
I am working on a project with image style transfer. I wanted to do some research on an existing project, and see what parameters work better. The project I am looking at is here:
https://tfhub.dev/sayakpaul/lite-model/arbitrary-image-stylization-inceptionv3-dynamic-shapes/int8/transfer/1
I can download a tflite file, but I don't know much about these files. If they have the architecture I need, how do I read it?
TFLite flatbuffer files contain the model structure as well. For example, there are a subgraph concept in TFLite, which corresponds to the function concept in the programming language and the operator nodes also represent a graph node, which takes inputs and generates outputs. By using the Netron application, the model architecture can be visualized.
I am trying to use this for my object detection task. The problems I am facing are:
On running the saved_model_cli command, I am getting the following output. There is no signature defined with tag-set "serve" also the method name is empty
The variable folder in the model directory only contains a few bytes of data which means the weights are not actually written to disk.
The model format seems to be HubModule V1 which seems to be the issue, any tips on making the above model servable are highly appreciated.
TF2 SavedModels should not have this problem, only Hub.Modules from TF1 since Hub.Modules use the signatures for other purposes. You can take a hub.Module and build a servable SavedModel, but it's quite complex and involves building the signatures yourself.
Instead, I recommend checking out the list of TF2 object detection models on TFHub.dev for a model you can use instead of the model you are using: https://tfhub.dev/s?module-type=image-object-detection&tf-version=tf2
These models should be servable with TF Serving
What is the main difference between .pb format of tensorflow and .h5 format of keras to store models? Is there any reason to choose one over the other?
Different file formats with different characteristics, both used by tensorflow to save models (.h5 specifically by keras).
.pb - protobuf
It is a way to store some structured data (in this case a neural network),project is open source and currently overviewed by Google.
Example
person {
name: "John Doe"
email: "jdoe#example.com"
}
Simple class containing two fields, you can load it in one of multiple supported languages (e.g. C++, Go), parse, modify and send to someone else in binary format.
Advantages
really small and efficient to parse (when compared to say .xml), hence often used for data transfer across the web
used by Tensorflow's Serving when you want to take your model to production (e.g. inference over the web)
language agnostic - binary format can be read by multiple languages (Java, Python, Objective-C, and C++ among others)
advised to use since tf2.0 , you can see official serializing guide
saves various metadata (optimizers, losses etc. if using keras's model)
Disadvantages
SavedModel is conceptually harder to grasp than single file
creates folder where weights are
Sources
You can read about this format here
.h5 - HDF5 binary data format
Used originally by keras to save models (keras is now officially part of tensorflow). It is less general and more "data-oriented", less programmatic than .pb.
Advantages
Used to save giant data (so some neural networks would fit well)
Common file saving format
Everything saved in one file (weights, losses, optimizers used with keras etc.)
Disadvantages
Cannot be used with Tensorflow Serving but you can simply convert it to .pb via keras.experimental.export_saved_model(model, 'path_to_saved_model')
All in all
Use the simpler one (.h5) if you don't need to productionize your model (or it's reasonably far away). Use .pb if you are going for production or just want to standardize on single format across all tensorflow provided tools.
I am reading tensorflow tutorial on saving and restoring model and came across the following statement:
If assets need to be saved and written or copied to disk,
they can be provided when the first MetaGraphDef is added. If multiple
MetaGraphDefs are associated with an asset of the same name,
only the first version is retained.
What does assets in this context mean?
Also another paragraphs says:
We provide a Python implementation of the SavedModel builder.
The SavedModelBuilder class provides functionality to save multiple MetaGraphDefs.
A MetaGraph is a dataflow graph, plus its associated variables, assets,
and signatures. A MetaGraphDef is the protocol buffer representation of
a MetaGraph. A signature is the set of inputs to and outputs from a graph.
What is dataflow graph and how is it different from graph?
Here is the tutorial
TL;DR
All you need to know when saving a Tensorflow model is that there are two files that are created (potentially more if you're using checkpoints):
file
file.meta
you save 'file' and restore 'file.meta'.
More info here: Tensorflow: how to save/restore a model?
#########
More to your question:
what you define in Tensorflow before you run your session is called a graph.
When you are saving your graph, a MetaGraph is created. This is the graph itself, and all the other metadata necessary for computations in this graph, as well as some user info that can be saved and version specification.
Assets are external files, such as vocabularies that were created with the graph.