• Hardware Platform (Jetson / GPU)
Jetson Nano 4GB, Ubuntu 18.4
• DeepStream Version
marketplace.azurecr.io/nvidia/deepstream-iot2-l4t:latest
• JetPack Version
4.3
• Issue Type
Output inference class is different from Model class
• How to reproduce the issue ?
On DeepStream, deploy a object detection ONNX model. My model is ONNX model exported from Azure Custom Vision. My label file has 2 classes - 'Mask', 'No_Mask'. Deployment works fine and I am able to execute my model using DeepStream. However, output inference class I am getting as 'Vehicle' and 'No_Mask'. Can you please help me understand why I am getting output inference label as "Vehicle" when it is not there in my Model.
Sample output inference log
{"log":" "1|324|23|380|61|Vehicle|#|||||||0"\n","stream":"stdout","time":"2021-01-05T16:15:15.614591738Z"}
{"log":" "1|324|23|380|61|Vehicle|#|||||||0"\n","stream":"stdout","time":"2021-01-05T16:15:15.614790179Z"}
{"log":" "2|141|15|365|161|No Mask"\n","stream":"stdout","time":"2021-01-05T16:15:15.614221209Z"}
You've most probably specified wrong labels file or the classes in it are wrong. It's provided in labelfile-path as
labelfile-path=labels.txt
Related
I'm trying to run a converted RepNet model in TFLite on mobile (iOS and Android) using Flutter, and the tf_lite_flutter package.
I have successfully converted the model to TFLite by adapting the colab provided by the authors. The notebook I used can be found in this repository, along with the converted model.
To make sure everything was working before attempting to run on an edge device, I checked everything with the Python TFLite API (this notebook). Everything indeed worked well - the output for the TFLite model matches the output of the Google colab provided by the authors.
I created a Flutter project to get the model running on mobile. I've tried passing in the default input and output, resulting from calls to interpreter.getInputTensors() and interpreter.getOutputTensors() respectively. When using this project to try to run the model, I encounter the following error:
E/tflite (26540): tensorflow/lite/kernels/reshape.cc:69 num_input_elements != num_output_elements (1 != 0)
E/tflite (26540): Node number 1 (RESHAPE) failed to prepare.
I'm admittedly pretty new to Tensorflow and Tensorflow Lite, so my debugging ability is somewhat limited. It does seem strange to me that the expected output shape is 0. Considering it is working with the Python API, I'm not sure why it isn't working on-device. The only thing I might suspect it could be is the batch_size not being configured properly. Using the shape_signature field, as in interpreter.get_input_details()[0]['shape_signature'], I can see that the batch size is dynamic (value -1).
The model was converted using Tensorflow==2.5 in Python, and is being run using the standard TFLite 2.5 binaries (no GPUDelegate).
Any suggestions for fixing this error would be appreciated!
Im trying to use a TensorFlow Lite Model I trained using Microsoft Custom Vision with Tensorflow Task Library to run object detection inference on an Android app. However when I try run the model I get this error:
"Error occurred when initializing ObjectDetector: Mobile SSD models are expected to have exactly 4 outputs, found 3"
Ive inspected the model and compared it to models I have created using TFLite_model_maker and the custom vision model has 3 outputs compared to the 4 that the TFLite_model_maker has. The custom vision model has 'number of detections' missing.
Is there a way I can add the extra output to the custom vision model?
Or is there any other way anyone can recommend getting this to work?
I have trained my custom data-set using the Tensor Flow Object Detection API. I run my "prediction" script and it works fine on the GPU. Now , I want to convert the model to lite and run it on the Google Coral Edge TPU Board to detect my custom objects. I have gone through the documentation that Google Coral Board Website provides but I found it very confusing.
How to convert and run it on the Google Coral Edge TPU Board?
Thanks
Without reading the documentation, it will be very hard to continue. I'm not sure what your "prediction script" means, but I'm assuming that the script loaded a .pb tensorflow model, loaded some image data, and run inference on it to produce prediction results. That means you have a .pb tensorflow model at the "Frozen graph" stage of the following pipeline:
Image taken from coral.ai.
The next step would be to convert your .pb model to a "fully quantized .tflite model" using the post training quantization technique. The documentation to do that are given here. I also created a github gist, containing an example of Post Training Quantization here. Once you have produced the .tflite model, you'll need to compile the model via the edgetpu_compiler. Although everything you need to know about the edgetpu compiler is in that link, for your purpose, compiling a model is as simple as:
$ edgetpu_compiler your_model_name.tflite
Which will creates a your_model_name_edgetpu.tflite model that is compatible with the EdgeTPU. Now, if at this stage, instead of creating an edgetpu compatible model, you are getting some type of errors, then that means your model did not meets the requirements that are posted in the models-requirements section.
Once you have produced a compiled model, you can then deploy it on an edgetpu device. Currently are 2 main APIs that can be use to run inference with the model:
EdgeTPU API
python api
C++ api
tflite API
C++ api
python api
Ultimately, there are many demo examples to run inference on the model here.
The previous answer works with general classification models, but not with TF object detection API trained models.
You cannot do post-training quantization with TF Lite converter on TF object detection API models.
In order to run object detection models on EdgeTPU-s:
You must train the models in quantized aware training mode with this addition in model config:
graph_rewriter {
quantization {
delay: 48000
weight_bits: 8
activation_bits: 8
}
}
This might not work with all the models provided in the model-zoo, try a quantized model first.
After training, export the frozen graph with: object_detection/export_tflite_ssd_graph.py
Run tensorflow/lite/toco tool on the frozen graph to make it TFLite compatible
And finally run edgetpu_complier on the .tflite file
You can find more in-depth guide here:
https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/running_on_mobile_tensorflowlite.md
I have trained a model for detection, which is doing great when embedded in tensorflow sample app.
After freezing with export_tflite_ssd_graph and conversion to tflite using toco the results do perform rather bad and have a huge "variety".
Reading this answer on a similar problem with loss of accuracy I wanted to try tflite_diff_example_test on a tensorflow docker machine.
As the documentation is not that evolved right now, I build the tool referencing this SO Post
using:
bazel build tensorflow/contrib/lite/testing/tflite_diff_example_test.cc which ran smooth.
After figuring out all my needed input parameters I tried the testscript with following commands:
~/.cache/bazel/_bazel_root/68a62076e91007a7908bc42a32e4cff9/external/bazel_tools/tools/test/test-setup.sh tensorflow/contrib/lite/testing/tflite_diff_example_test '--tensorflow_model=/tensorflow/shared/exported/tflite_graph.pb' '--tflite_model=/tensorflow/shared/exported/detect.tflite' '--input_layer=a,b,c,d' '--input_layer_type=float,float,float,float' '--input_layer_shape=1,3,4,3:1,3,4,3:1,3,4,3:1,3,4,3' '--output_layer=x,y'
and
bazel-bin/tensorflow/contrib/lite/testing/tflite_diff_example_test --tensorflow_model="/tensorflow/shared/exported/tflite_graph.pb" --tflite_model="/tensorflow/shared/exported/detect.tflite" --input_layer=a,b,c,d --input_layer_type=float,float,float,float --input_layer_shape=1,3,4,3:1,3,4,3:1,3,4,3:1,3,4,3 --output_layer=x,y
Both ways are failing. Errors:
way:
tflite_diff_example_test.cc:line 1: /bazel: Is a directory
tflite_diff_example_test.cc: line 3: syntax error near unexpected token '('
tflite_diff_example_test.cc: line 3: 'Licensed under the Apache License, Version 2.0 (the "License");'
/root/.cache/bazel/_bazel_root/68a62076e91007a7908bc42a32e4cff9/external/bazel_tools/tools/test/test-setup.sh: line 184: /tensorflow/: Is a directory
/root/.cache/bazel/_bazel_root/68a62076e91007a7908bc42a32e4cff9/external/bazel_tools/tools/test/test-setup.sh: line 276: /tensorflow/: Is a directory
way:
2018-09-10 09:34:27.650473: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Failed to create session. Op type not registered 'TFLite_Detection_PostProcess' in binary running on d36de5b65187. Make sure the Op and Kernel are registered in the binary running in this process. Note that if you are loading a saved graph which used ops from tf.contrib, accessing (e.g.)tf.contrib.resamplershould be done before importing the graph, as contrib ops are lazily registered when the module is first accessed.
I would really appreciate any help, that enables me to compare the output of two graphs using tensorflows given tests.
The second way you mentioned is the correct way to use tflite_diff. However, the object detection model containing the TFLite_Detection_PostProcess op cannot be run via tflite_diff.
tflite_diff runs the provided TensorFlow (.pb) model in the TensorFlow runtime and runs the provided TensorFlow Lite (.tflite) model in the TensorFlow Lite runtime. In order to run the .pb model in the TensorFlow runtime, all of the operations must be implemented in TensorFlow.
However, in the model you provided, the TFLite_Detection_PostProcess op is not implemented in TensorFlow runtime - it is only available in the TensorFlow Lite runtime. Therefore, TensorFlow cannot resolve the op. Therefore, you unfortunately cannot use the tflite_diff tool with this model.
I am running sample program which comes packaged with Tensorflow object detection API(object_detection_tutorial.ipynb).
Program runs fine with no error, but bounding boxes are not diaplayed at all.
My environment is as follows:
Windows 10
Python 3.6.3
What can be the reason?
With regards
Manish
It seems that the latest version of the model ssd_mobilenet_v1_coco_2017_11_08 doesn't work and outputs abnormally low value. Replacing it in the Jupyter Notebook with an older version of the model works fine for me:
# What model to download.
MODEL_NAME = 'ssd_mobilenet_v1_coco_11_06_2017'
Ref: https://github.com/tensorflow/models/issues/2773
Please try updated SSD models in the detection zoo : https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md. This should be fixed.