I am using a custom object detection model with TensorFlow Lite on native Android. At this point I'm only detecting 2 custom objects. I am using the TensorFlow Object Detection API, and I have a pipeline in place that produces optimized .tflite files.
However, at inference time, the model only returns up to 10 individual detections. According to https://www.tensorflow.org/lite/models/object_detection/overview, this is expected. The problem is that my images have a relatively large object density. I need to be able to detect up to 30 individual objects per image/inference call.
If I change NUM_DETECTIONS in the sample Android app from the TF repo from 10 to, say, 20, I get a runtime exception due to shape mismatch. How can I produce .tflite files capable of yielding more than 10 object detection instances?
Thank you!
Unfortunately, since TFLite prefers static-shaped Input/Outputs, you would need to re-export a TFLite SSD graph with the required number of outputs. Instructions are here. While invoking object_detection/export_tflite_ssd_graph.py, you would need to pass in the parameter --max_detections=20. Then, your change of NUM_DETECTIONS should work as expected.
Related
I am trying to use an object detection model, that was created using the TF Object Detection API, in TensorFlow.js.
I converted the model using:
tensorflowjs_converter --input_format=tf_saved_model inference_graph/saved_model inference_graph/web_model
It gets converted without any problems and loads in my javascript code.
Now I am a bit unsure about what my next steps should be.
I have to translate the Python into JavaScript, but certain areas I am unsure about.
With the object detection API in python, there are many steps; (1)preprocessing the image, such as convert to RGB, numpy array reshape, expand dimensions (I have an idea of how I would approach it) and (2) the run inference for single image function, I am not sure how I would go about it in tensorflow.js.
I tried to find some general information about deploying an object detection model in tensorflow.js, but I could not find much, except with pre-trained models.
Any information about this topic would be great!
Thanks!
as mentioned by #edkeveked you will need to perform similar input processing and output processing in JavaScript as is being done in Python. i can't say exactly what you will need to do since i am not familiar with the model. however, you can find an example using a specific object detection model here:
https://github.com/vabarbosa/tfjs-model-playground/blob/master/object-detector/demo/object-detector.js
see also
https://medium.com/codait/bring-machine-learning-to-the-browser-with-tensorflow-js-part-iii-62d2b09b10a3
You would need to replicate the same process in javascript before giving it to the model. In js, the image use by default the RGB channel, so there is no need to make that conversion again.
I am trying to understand what I need from any pre-trained model used in the API regardless of any additional code found on the Tensorflow object detection API.
For example: ssd_mobilenet_v1_coco_2017_11_17, depending on what I have understood: it is a model that is already trained to detect objects (there is a classification to know the category of the object + Regression to bound the objects with rectangles and those rectangles are actually the x,y,w,h coordinates on the object).
How do we benefit from the regression output of that model (x,y,w,h coordinates) to use them in another model?
Let's assume we want to print out just the coordinates x,y,w,h of a detected object on an image without any need of the code of Tensorflow object detection API, how can we do that?
Certainly you can use the pretrained model provided in tensorflow object detection model zoo without installing object detection api. The alternative solution is to use opencv.
Opencv has provided both c++ and python api to call .pb models generated by tensorflow. Here is a nice tutorial.
I trained a model using Tensorflow object detection API for detecting swimming pools using satellite images. I used 'faster_rcnn_inception_v2_coco_2018_01_28' model for training. I generated a frozen inference graph (.pb). I want to evaluate the precision and recall of the model. Can someone tell me how I can do that, preferably without using pycocotools as I was facing some issues with that. Any suggestions are welcome :)
From the Object Detection API you can run "eval.py" from "models/research/object_detection/legacy/".
Your have to define an evaluation metric in your config file (see the supported evaluation protocols)
For example:
eval_config: {metrics_set: "coco_detection_metrics"}
The Pascal VOC e.g. then gives you the mean Average Precsion (mAP)
I am training a pre built tensorflow based model for custom object detection.
I want to detect only 1 type of object. I have taken lot of images from different angles and in different light conditions. I am training on K80 Nvidia GPU. Everything is working and when I train I can see the loss function falling to 0.3. But the loss values drops very quickly to under 1 when I start training. I am using SSD mobile Net as the base configuration for the model. When I try to test the model, it just draws a big square on the input image, rather than detecting the desired object in the image. Basically, it fails to detect the object.
I tried to train the model with a different set of images of mac n chesse which had lot of variations. Then the model worked fine and detected images of mac n chesse in the input image. But when I have pictures of single object then the model fails to detect. Please help me understand what I am doing wrong here
The issue was with my training dataset. I was not properly cropping the object from the original image. Also I needed around 300 images to properly train the model. SSD worked well after giving a well cropped images.
I used tensorflow object detection api with RFCN_resnet101 for little objects, but sometimes the detection result is not good, it will detect the object with offset, and sometimes it detects an object by mistake. Does anyone knows how to deal with it?
Debugging object detection can be tricky. I recommend checking the input data (does it make sense):
Do object bounding boxes get displayed correctly when overlayed with images?
Are you using pixel box coordinates (vs normalized) when preparing the training data?
Do you have boxes that are too small or outside the image boundary that cause tensor NaN errors?
Do you have images that are too large and cause CUDA out of memory errors?
Once you are satisfied with input data and able to successfully generate TF records files for training and evaluation. I recommend asking the following questions:
Are you training the network for sufficient number of global iterations (e.g. 200K) on preferrably multiple GPUs with a sufficiently large batch size?
Are you getting resonable detections when evaluating on a few images, e.g. by specifying the following config file:
eval_config: {
num_examples: 1000
num_visualizations: 16
min_score_threshold: 0.15
# Note: The below line limits the evaluation process to 10 evaluations.
# Remove the below line to evaluate indefinitely.
max_evals: 1
}
Here num_visualizations will create an images tab in tensorboard when you run eval.py script, and you'll be able to visualize detections and vary the IoU min_score_threshold.
Are you fine-tuning a pre-trained model, e.g. check to make sure you have
fine_tune_checkpoint: "/path/to/model.ckpt"
from_detection_checkpoint: true
Finally, the beauty of TensorFlow object detection API is that you can try different object detection models: Faster R-CNN, YOLO, SSD that have different speed-accuracy tradeoffs without much extra work. You may find a different object detector works better for your application.
why not use faster_rcnn_inception_resnet_v2_atrous_coco ... for small objects its my goto option