I successfully created a server that receives a TF saved_model, but now I want to send it queries and get predictions.
However, I'm having a hard time of understanding how the client works and how to implement it.
All I found online is the basic tutorial, but they only give the client code for mnist, and it doesn't fit my own mdoel.
So can anyone refer me to how to use or implement the client for a different model?
Thanks
I really thank google to make tensorflow serving open source, it is so helpful for people like me to put prediction models into production. But I have to admit tensorflow serving did poorly in documentation, or, they assume people who use it should already have pretty good knowledge in tensorflow. I stuck for a long time in order to get understand how it works. In their website they introduced concepts and examples well, but there is something missing in between.
I will recommend the tutorial here. This is the first part, and you can also follow the second part, the link will be in that article.
In general, when you export your .ckpt files to a servable model(.pb file and variables folder), you have to define the input, output and method name of your model and save them as a signature in tf.saved_model.signature_def_utils.build_signature_def
In the article, you will find what I said above in this part:
tf.saved_model.signature_def_utils.build_signature_def(
inputs={‘images’: predict_tensor_inputs_info},
outputs={‘scores’: predict_tensor_scores_info},
method_name=\
tf.saved_model.signature_constants.PREDICT_METHOD_NAME)
You can follow how the author defined input and output in the article, and do the same thing to your customized model.
After that, you have to in your client script to call the signature and feed input to server, server then will recognize which method to use and return the output. You can check how the author wrote the client script and find corresponding part of calling signature and feeding input.
Related
I am currently trying to train a custom model for use in Unity (Barracuda) for object detection and I am struggling near what I believe to be the last part of the pipeline. Following various tutorials and git-repos I have done the following...
Using Darknet, I have trained a custom-model using the Tiny-Yolov2 model. (model tested successfully on a webcam python script)
I have taken the final weights from that training and converted them
to a Keras (h5) file. (model tested successfully on a webcam python
script)
From Keras, I then use tf.save_model to turn it into a
save_model.pd.
From save_model.pd I then convert it using tf2onnx.convert to change
it to an onnx file.
Supposedly from there it can then work in one of a few Unity sample
projects...
...however, this project fails to read in the Unity Sample projects I've tried to use. From various posts it seems that I may need to use a 'frozen' save_model.pd before converting it to ONNX. However all the guides and python functions that seem to be used for freezing save_models require a lot more arguments than I have awareness of or data for after going through so many systems. https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/tools/freeze_graph.py - for example, after converting into Keras, I only get left with a h5 file, with no knowledge of what an input_graph_def, or output_node_names might refer to.
Additionally, for whatever reason, I cannot find any TF version (1 or 2) that can successfully run this python script using 'from tensorflow.python.checkpoint import checkpoint_management' it genuinely seems like it not longer exists.
I am not sure why I am going through all of these conversions and steps but every attempt to find a cleaner process between training and unity seemed to lead only to dead ends.
Any help or guidance on this topic would be sincerely appreciated, thank you.
I'm trying to dig into mediapipe and adapt it to perform inference using a custom tflite model. However, this task seems to be harder than expected. Modifying existing examples is also not practical as they seem to be either too complicated (like pose tracking) or too simple (like hello world).
Can someone provide me with a simple toy example?
Task could be
send a float array to the graph (say of length = 1 and all values = 0 for simplicity)
tflite model adds 1 to each element of the input tensor
output tensor is send back to output and logged to console
Having such a code would surely be a great step ahead while experimenting with the framework
Meanwhile I solved the issue myself. The solution is also part of MediaPipeUnityPlugin authored by homuler. Here is an example driver code, that shows an example how to use it.
Once I get time I can also post the pure mediapipe solution here (but it's a little ugly)
In https://www.tensorflow.org/js/guide/save_load, it describes the format for saving model files as one that uses model.json and the corresponding model.weights.bin. I'm not sure if there's a name for talking about this format (I think it's the same as https://js.tensorflow.org/api/latest/#class:LayersModel but not entirely certain), and I'm wondering if there's a way to visualize them as a graph.
I was expecting to be able to load and view them in TensorBoard but don't see any way to do this with its "Graphs" tool, so perhaps no one has made anything like this yet.
One minimal way to do this is with the summary method on a loaded model, which will log to the console:
In the example I tried, this output doesn't match what I would expect from looking at the model.json though. It looks like this might only look at the outermost Sequential layer but I didn't look more closely.
You can examine models, layers and tensors and etc via Visor API https://js.tensorflow.org/api_vis/latest/. You just need to install it npm i #tensorflow/tfjs-vis in order to use.
I haven't tried Tensorflow yet but still curious, how does it store, and in what form, data type, file type, the acquired learning of a machine learning code for later use?
For example, Tensorflow was used to sort cucumbers in Japan. The computer used took a long time to learn from the example images given about what good cucumbers look like. In what form the learning was saved for future use?
Because I think it would be inefficient if the program should have to re-learn the images again everytime it needs to sort cucumbers.
Ultimately, a high level way to think about a machine learning model is three components - the code for the model, the data for that model, and metadata needed to make this model run.
In Tensorflow, the code for this model is written in Python, and is saved in what is known as a GraphDef. This uses a serialization format created at Google called Protobuf. Common serialization formats include Python's native Pickle for other libraries.
The main reason you write this code is to "learn" from some training data - which is ultimately a large set of matrices, full of numbers. These are the "weights" of the model - and this too is stored using ProtoBuf, although other formats like HDF5 exist.
Tensorflow also stores Metadata associated with this model - for instance, what should the input look like (eg: an image? some text?), and the output (eg: a class of image aka - cucumber1, or 2? with scores, or without?). This too is stored in Protobuf.
During prediction time, your code loads up the graph, the weights and the meta - and takes some input data to give out an output. More information here.
Are you talking about the symbolic math library, or the idea of tensor flow in general? Please be more specific here.
Here are some resources that discuss the library and tensor flow
These are some tutorials
And here is some background on the field
And this is the github page
If you want a more specific answer, please give more details as to what sort of work you are interested in.
Edit: So I'm presuming your question is more related to the general field of tensor flow than any particular application. Your question still is too vague for this website, but I'll try to point you toward a few resources you might find interesting.
The tensorflow used in image recognition often uses an ANN (Artificial Neural Network) as the object on which to act. What this means is that the tensorflow library helps in the number crunching for the neural network, which I'm sure you can read all about with a quick google search.
The point is that tensorflow isn't a form of machine learning itself, it more serves as a useful number crunching library, similar to something like numpy in python, in large scale deep learning simulations. You should read more here.
I'm trying to send image input over http to classify using tensorflow. I have looked in detail in the c++ code for https://www.tensorflow.org/versions/r0.9/tutorials/image_recognition/index.html
I have implemented the inception-v3 example model using C++ API. It takes image input in the following form:
bazel-bin/tensorflow/examples/label_image/label_image --image=my_image.png
However, I want to add the case of:
bazel-bin/tensorflow/examples/label_image/label_image --image=http://www.somewebsite.com/my_image.png
This is due to the fact that it only accepts local image files. I want to add the functionality to take file pointers from online images and classify it in memory. I'm currently working on this, but so far no luck. Can anyone offer some insight how I would go about implementing this?