For a given graph, how can we visualize the graph using tensorboard for tf.compat.v1 ?
Sharing this here after searching everywhere. Most of the documentations explains tf.keras and not for tf.compat.v1 static graphs
First export the graph to a logdir that tensorboard can use!
import tensorflow as tf
# Get the default graph
graph = tf.compat.v1.get_default_graph()
writer = tf.compat.v1.summary.FileWriter("logs", graph)
After that simply open tensorboard in the specified directory (here logs/)
python -m tensorboard.main --logdir logs/
Related
I need to export a custom object detection model, fine-tuned on a custom dataset, to TensorFlow Lite, so that it can run on Android devices.
I'm using TensorFlow 2.4.1 on Ubuntu 18.04, and so far this is what I did:
fine-tuned an 'ssd_mobilenet_v2_fpnlite_640x640_coco17_tpu-8' model, using a dataset of new images. I used the 'model_main_tf2.py' script from the repository;
I exported the model using 'exporter_main_v2.py'
python exporter_main_v2.py --input_type image_tensor --pipeline_config_path .\models\custom_model\pipeline.config --trained_checkpoint_dir .\models\custom_model\ --output_directory .\exported-models\custom_model
which produced a Saved Model (.pb file);
3. I tested the exported model for inference, and everything works fine. In the detection routine, I used:
def get_model_detection_function(model):
##Get a tf.function for detection
#tf.function
def detect_fn(image):
"""Detect objects in image."""
image, shapes = model.preprocess(image)
prediction_dict = model.predict(image, shapes)
detections = model.postprocess(prediction_dict, shapes)
return detections, prediction_dict, tf.reshape(shapes, [-1])
return detect_fn
and the shape of the produced image object is 640x640, as expected.
Then, I tried to convert this .pb model to tflite.
After updating to the nightly version of tensorflow (with the normal version, I got an error), I was actually able to produce a .tflite file by using this code:
import tensorflow as tf
from tflite_support import metadata as _metadata
saved_model_dir = 'exported-models/custom_model/'
## Convert the model
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.experimental_new_converter = True
converter.target_spec.supported_ops = [
tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS]
tflite_model = converter.convert()
# Save the model.
with open('tflite/custom_model.tflite', 'wb') as f:
f.write(tflite_model)
I tried to use this model in AndroidStudio, following the instructions given here.
However, I'm getting a couple of errors:
something regarding 'Not a valid Tensorflow lite model' (have to check better on this);
the error:
java.lang.IllegalArgumentException: Cannot copy to a TensorFlowLite tensor (serving_default_input_tensor:0) with 3 bytes from a Java Buffer with 270000 bytes.
The second error seems to indicate there's something weird with the input expected from the tflite model.
I examined the file with Netron, and this is what I got:
the input is expected to have...1x1x1x3 shape, or am I misinterpreting the graph?
Should I somehow set the tensor input size when using the tflite exporter?
Anyway, what is the right way to export my custom model so that it can run on Android?
TF Ops are supported via the Flex delegate. I bet that is the problem. If you want to check if it is that, you can do:
Download benchmark app with flex delegate support for TF Ops. You can find it here, in the section Native benchmark binary: https://www.tensorflow.org/lite/performance/measurement. For example, for android is https://storage.googleapis.com/tensorflow-nightly-public/prod/tensorflow/release/lite/tools/nightly/latest/android_aarch64_benchmark_model_plus_flex
Connect your phone to your computer and where you have downloaded the apk, do adb push <apk_name> /data/local/tmp
Push your model adb push <tflite_model> /data/local/tmp
Open shell adb shell and go to folder cd /data/local/tmp. Then run the app with ./<apk_name> --graph=<tflite_model>
Info from:
https://www.tensorflow.org/lite/guide/ops_select
https://www.tensorflow.org/lite/performance/measurement
I have downloaded a pre-trained PoseNet model for Tensorflow.js (tfjs) from Google, so its a json file.
However, I want to use it on Android, so I need the .tflite model. Although someone has 'ported' a similar model from tfjs to tflite here, I have no idea what model (there are many variants of PoseNet) they converted. I want to do the steps myself. Also, I don't want to run some arbitrary code someone uploaded into a file in stackOverflow:
Caution: Be careful with untrusted code—TensorFlow models are code. See Using TensorFlow Securely for details. Tensorflow docs
Does anyone know any convenient ways to do this?
You can find out what tfjs format you have by looking in the json file. It often says "graph-model". The difference between them are here.
From tfjs graph model to SavedModel (more common)
Use tfjs-to-tf by Patrick Levin.
import tfjs_graph_converter.api as tfjs
tfjs.graph_model_to_saved_model(
"savedmodel/posenet/mobilenet/float/050/model-stride16.json",
"realsavedmodel"
)
# Code below taken from https://www.tensorflow.org/lite/convert/python_api
converter = tf.lite.TFLiteConverter.from_saved_model("realsavedmodel")
tflite_model = converter.convert()
# Save the TF Lite model.
with tf.io.gfile.GFile('model.tflite', 'wb') as f:
f.write(tflite_model)
From tfjs layers model to SavedModel
Note: This will only work for layers model format, not graph model format as in the question. I've written the difference between them here.
Install and use tensorflowjs-convert to convert the .json file into a Keras HDF5 file (from another SO thread).
On mac, you'll face issues running pyenv (fix) and on Z-shell, pyenv won't load correctly (fix). Also, once pyenv is running, use python -m pip install tensorflowjs instead of pip install tensorflowjs, because pyenv did not change python used by pip for me.
Once you've followed the tensorflowjs_converter guide, run tensorflowjs_converter to verify it works with no errors, and should just warn you about Missing input_path argument. Then:
tensorflowjs_converter --input_format=tfjs_layers_model --output_format=keras tfjs_model.json hdf5_keras_model.hdf5
Convert the Keras HDF5 file into a SavedModel (standard Tensorflow model file) or directly into .tflite file using the TFLiteConverter. The following runs in a Python file:
# Convert the model.
model = tf.keras.models.load_model('hdf5_keras_model.hdf5')
converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()
# Save the TF Lite model.
with tf.io.gfile.GFile('model.tflite', 'wb') as f:
f.write(tflite_model)
or to save to a SavedModel:
# Convert the model.
model = tf.keras.models.load_model('hdf5_keras_model.hdf5')
tf.keras.models.save_model(
model, filepath, overwrite=True, include_optimizer=True, save_format=None,
signatures=None, options=None
)
I am following Tensorflow serving documentation to convert my trained model into a format that can be served in Docker container. As I'm new to Tensorflow, I am struggling to convert this trained model into a form that will be suitable for serving.
The model is already trained and I have the checkpoint file and .meta file. So, I need to get the .pb file and variables folder from the above two files. Can anyone please suggest me an approach on how to get this done for serving the models?
.
|-- tensorflow model
| -- 1
| |-- saved_model.pb
| -- variables
| |-- variables.data-00000-of-00001
| -- variables.index
There is multiple ways of doing this, and other methods could be required for more complex models.
I am currently using the method described here, which works great for tf.keras.models.Model and tf.keras.Sequential models (not sure for tensorflow subclassing?).
Below is a minimal working example, including creating a model using python (it seems like you have already completed this by your folder structure and can ignore the first step)
import tensorflow as tf
from tensorflow.keras.layers import Input, Dense
from tensorflow.keras.models import Model
import tensorflow.keras.backend as K
inputs = Input(shape=(2,))
x = Dense(128, activation='relu')(inputs)
x = Dense(32, activation='relu')(x)
outputs = Dense(1)(x)
model = Model(inputs=inputs, outputs=outputs)
model.compile(optimizer='adam', loss='mse')
# loading existing weights, model architectural must be the same as the existing model
#model.load_weights(".//MODEL_WEIGHT_PATH//WEIGHT.h5")
export_path = 'SAVE_PATH//tensorflow_model//1'
with K.get_session() as sess:
tf.saved_model.simple_save(
sess,
export_path,
inputs={'inputs': model.input}, # for single input
#inputs={t.name[:-5]: t for t in model.input}, # for multiple inputs
outputs={'outputs': model.output})
I suggest you use folder name "tensorflow_model" instead of "tensorflow model", to avoid possible problems with spaces.
Then we can build the docker image in terminal by (for windows, use ^ instead of \ for line brake, and use //C/ instead of C:\ in path):
docker run -p 8501:8501 --name tfserving_test \
--mount type=bind,source="SAVE_PATH/tensorflow_model",target=/models/tensorflow_model \
-e MODEL_NAME=tensorflow_model -t tensorflow/serving
Now the container should be up and running, and we can test the serving with python
import requests
import json
#import numpy as np
payload = {
"instances": [{'inputs': [1.,1.]}]
}
r = requests.post('http://localhost:8501/v1/models/tensorflow_model:predict', json=payload)
print(json.loads(r.content))
# {'predictions': [[0.121025]]}
The container is working with our model, giving the prediction 0.121025 for the input [1., 1.]
I hope this helps:
import tensorflow as tf
from tensorflow.contrib.keras import backend as K
from tensorflow.python.client import device_lib
K.set_learning_phase(0)
model = tf.keras.models.load_model('my_model.h5')
export_path = './'
with K.get_session() as sess:
tf.saved_model.simple_save(
sess,
export_path,
inputs={'input_image': model.input},
outputs={t.name: t for t in model.outputs}
)
print('Converted to SavedModel!!!')
From your question, do you mean you no more have access to Model and you have only Check Point files and .meta files?
If that is the case, you can refer the below links which has the code for converting those files into '.pb' file.
Tensorflow: How to convert .meta, .data and .index model files into one graph.pb file
https://github.com/petewarden/tensorflow_makefile/blob/master/tensorflow/python/tools/freeze_graph.py
If you have access to the Trained Model, then I guess you are saving it currently using tf.train.Saver. Instead of that, you can Save the Model and Export it using any of the three (commonly used) functions mentioned below:
tf.saved_model.simple_save => In this case, only Predict API is supported during Serving. Example of this is mentioned by KrisR89 in his answer.
tf.saved_model.builder.SavedModelBuilder => In this case, you can define the SignatureDefs, i.e., the APIs which you want to access during Serving.
You can find example on how to use it in the below link,
https://github.com/tensorflow/serving/blob/master/tensorflow_serving/example/mnist_saved_model.py
Third way is shown below:
classifier = tf.estimator.DNNClassifier(config=training_config, feature_columns=feature_columns,hidden_units=[256, 32], optimizer=tf.train.AdamOptimizer(1e-4),n_classes=NUM_CLASSES,dropout=0.1, model_dir=FLAGS.model_dir)
classifier.export_savedmodel(FLAGS.saved_dir,
serving_input_receiver_fn=serving_input_receiver_fn)
The Example on how to save model using Estimators can be found in the below link. This supports Predict and Classification APIs.
https://github.com/yu-iskw/tensorflow-serving-example/blob/master/python/train/mnist_premodeled_estimator.py
Let me know if this information helps or if you need any further help.
I am trying use an embeddings module from tensorflow hub as servable. I am new to tensorflow. Currently, I am using Universal Sentence Encoder embeddings as a lookup to convert sentences to embeddings and then using those embeddings to find a similarity to another sentence.
My current code to convert sentences into embeddings is:
with tf.Session() as session:
session.run([tf.global_variables_initializer(), tf.tables_initializer()])
sen_embeddings = session.run(self.embed(prepared_text))
Prepared_text is a list of sentences. How do I take this model and make it a servable?
Right now you probably need to do this by hand. Here is my solution, similar to previous answer but more general - show how to use any other module without guessing input parameters, as well as extended with verification and usage:
import tensorflow as tf
import tensorflow_hub as hub
from tensorflow.saved_model import simple_save
export_dir = "/tmp/tfserving/universal_encoder/00000001"
with tf.Session(graph=tf.Graph()) as sess:
module = hub.Module("https://tfhub.dev/google/universal-sentence-encoder/2")
input_params = module.get_input_info_dict()
# take a look at what tensor does the model accepts - 'text' is input tensor name
text_input = tf.placeholder(name='text', dtype=input_params['text'].dtype,
shape=input_params['text'].get_shape())
sess.run([tf.global_variables_initializer(), tf.tables_initializer()])
embeddings = module(text_input)
simple_save(sess,
export_dir,
inputs={'text': text_input},
outputs={'embeddings': embeddings},
legacy_init_op=tf.tables_initializer())
Thanks to module.get_input_info_dict() you know what tensor names you need to pass to the model - you use this name as a key for inputs={} in simple_save method.
Remember that to serve the model it needs to be in directory path ending with version, that's why '00000001' is the last path in which saved_model.pb resides.
After exporting your module, quickest way to see if your model is exported properly for serving is to use saved_model_cli API:
saved_model_cli run --dir /tmp/tfserving/universal_encoder/00000001 --tag_set serve --signature_def serving_default --input_exprs 'text=["what this is"]'
To serve the model from docker:
docker pull tensorflow/serving
docker run -p 8501:8501 -v /tmp/tfserving/universal_encoder:/models/universal_encoder -e MODEL_NAME=universal_encoder -t tensorflow/serving
Currently, the hub modules cannot be consumed by Tensorflow Serving directly. You will have to load the module into an empty graph and then export it using the SavedModelBuilder. For example:
import tensorflow as tf
import tensorflow_hub as hub
with tf.Graph().as_default():
module = hub.Module("http://tfhub.dev/google/universal-sentence-encoder/2")
text = tf.placeholder(tf.string, [None])
embedding = module(text)
init_op = tf.group([tf.global_variables_initializer(), tf.tables_initializer()])
with tf.Session() as session:
session.run(init_op)
tf.saved_model.simple_save(
session,
"/tmp/serving_saved_model",
inputs = {"text": text},
outputs = {"embedding": embedding},
legacy_init_op = tf.tables_initializer()
)
This will export your model (to the folder /tmp/serving_saved_model) in the desired format for serving. After this, you can follow the instructions given in the documentation here: https://www.tensorflow.org/serving/serving_basic
Note that the other answers are for TensorFlow 1. Most TF Hub models for TensorFlow 2 will already be compatible with TF Serving. For example, to deploy the USE-Large model:
Download the model, either via the tensorflow_hub library or just https://tfhub.dev/google/universal-sentence-encoder-large/5
Put the content into folders representing the model name and version, e.g. models/use-large/5
Run the TF Serving application, e.g. via Docker:
docker run -t --rm -p 8501:8501 \
-v "$PATH_TO_YOUR_WORKSPACE/models:/models" \
-e MODEL_NAME="use-large" \
tensorflow/serving
The model will be available at localhost:8501/v1/models/use-large:
curl -d '{"instances": ["Hey!"]}' \
-X POST http://localhost:8501/v1/models/use-large:predict
I want to see keras model like this. I used K.get_session().graph and get
tensorflow.python.framework.ops.Graph at 0x7f2a8b809400
but i to see this graph and save it. I am using tensorflow backend
Install tensorboard
Import it
from keras.callbacks import TensorBoard
Load it into a variable
tbCallBack = TensorBoard(log_dir='Graph',
histogram_freq=10,
write_graph=True,
write_images=True)
And then use that as a callback at training:
model.fit(x, y, ...
callbacks=[tbCallBack])
Make sure you have a made a directory called 'Graph' or whatever you want. Then before training run in terminal:
tensorboard --logdir Graph
And then you can see your graph in your browser