I have a custom object detection model that I can call with model = MyModel() and model.loadweights(checkpoint) and I want to evaluate it using the Object Detection API.
From what I understood there are two possibilities, either I use the legacy eval.py, there I don't know, what to put into the pipeline_config file
Or I use the newer version that is implemented in model_main_tf2.py, but there I would have to save my model as model.config and I don't know what to put the pipeline file either.
Since my model is a YOLO model, it is not included in the sample once yet.
https://github.com/tensorflow/models/tree/master/research/object_detection/configs/tf2
Would really appreciate the help!
You can't calculate the mAP using the Object Detection API because there's no pipeline.config file for Yolo.
However, you can check this repo out. It's a Tensorflow based implementation of YoloV3. They have working code for calculating mAP. You can modify this accordingly to calculate the mAP of your model.
Related
If I use the Pre-Trained TFLite Object detection model in MLKit, I get the following error:
CalculatorGraph::Run() failed in Run:
Calculator::Open() for node "BoxClassifierCalculator" failed: #vk Unexpected number of dimensions for output index 0: got 3D, expected either 2D (BxN with B=1) or 4D (BxHxWxN with B=1, W=1, H=1).
Any Idea what I might be doing wrong?
ML Kit does not currently support custom object detection model yet. ML Kit currently only allows developers to use custom image classification models. All TFLite models that are compatible with ML Kit are listed here:
https://tfhub.dev/ml-kit/collections/image-classification/1
If you want to do object detection, you can try out ML Kit's Object Detection API: https://developers.google.com/ml-kit/vision/object-detection
If you want to use a custom object detection model, you can try TFLite task library:
https://www.tensorflow.org/lite/inference_with_metadata/task_library/overview.
Well, since I found out through this post here on SO, I went on to look for other options.
And the tutorial posted here: Tensor Flow Lite OBJ detection did the trick just nicely.
First of all added the
implementation 'org.tensorflow:tensorflow-lite-task-vision:0.2.0'
To build.gradle, and this simple code worked like a charm. Obviously I have to make some changes to reflect my specific needs, but it's detecting without errors.
val image = TensorImage.fromBitmap(bitmap)
val options = ObjectDetector.ObjectDetectorOptions.builder()
.setMaxResults(5)
.setScoreThreshold(0.5f)
.build()
val detector = ObjectDetector.createFromFileAndOptions(
this, // the application context
"model.tflite", // must be same as the filename in assets folder
options
)
A more in depth explanation is given in that link, too much to be thrown here.
Hope this helps somebody else.
You can now create MLKit Custom Models with Google's Vertex AI/AutoML Cloud service and it works well with the CameraX API using an ImageAnalysis Analyzer. Follow the guide here to make and train the TFLite model with your own images via browser (or use the Google Cloud CLI if you have many to upload). After that, this guide walks you through pretty much everything you need to add the model to your project, and how detect and label objects with MLKit.
I am trying to use an object detection model, that was created using the TF Object Detection API, in TensorFlow.js.
I converted the model using:
tensorflowjs_converter --input_format=tf_saved_model inference_graph/saved_model inference_graph/web_model
It gets converted without any problems and loads in my javascript code.
Now I am a bit unsure about what my next steps should be.
I have to translate the Python into JavaScript, but certain areas I am unsure about.
With the object detection API in python, there are many steps; (1)preprocessing the image, such as convert to RGB, numpy array reshape, expand dimensions (I have an idea of how I would approach it) and (2) the run inference for single image function, I am not sure how I would go about it in tensorflow.js.
I tried to find some general information about deploying an object detection model in tensorflow.js, but I could not find much, except with pre-trained models.
Any information about this topic would be great!
Thanks!
as mentioned by #edkeveked you will need to perform similar input processing and output processing in JavaScript as is being done in Python. i can't say exactly what you will need to do since i am not familiar with the model. however, you can find an example using a specific object detection model here:
https://github.com/vabarbosa/tfjs-model-playground/blob/master/object-detector/demo/object-detector.js
see also
https://medium.com/codait/bring-machine-learning-to-the-browser-with-tensorflow-js-part-iii-62d2b09b10a3
You would need to replicate the same process in javascript before giving it to the model. In js, the image use by default the RGB channel, so there is no need to make that conversion again.
I use (https://github.com/matterport/Mask_RCNN) MASKRCNN for object detection and everything works fine. Is there a way to detect only certain objects? There are more than 80 classes of which I only need 2 pieces (like car and person). I would like to have the remaining classes not detected. How can they be removed?
I guess you are using the default pre-trained coco model for detection which comes with 80 classes. You can train your own model.
First of all you have to use VIA (VGG image annotator) to label the classes you want to predict.
Once that is done, you have to make some changes to the code of the model. For example, if you are using the file "balloon.py", you have to add classes, make some changes in load_mask() function and few other parts of the codes. After that you can start training your model, then use for detection and segmentation.
Yes, and for VIA, try using the version 1.0.0 as the format of the .json file is slightly changed in in the updated versions, which generally makes them incompatible with training on your custom data sets.
Check one example here
Is it possible to use different feature extractor with SSD meta-architecture in Tensorflow's Object Detection API? I know that .config files for mobilenets and inception are provided but is it possible to use a different architecture like AlexNet or VGG?
It's possible but with a little bit of work, as explained here, you should read this page for detailed explanation and links to examples.
In short, you'll need to create a custom FasterRCNNFeatureExtractor class, corresponding to VGG or AlexNet (it may require a bit of knowledge about these, for instance the amount of subsampling invovled). In this class, you'll code how your data should be preprocessed, how to retrieve the 1st and 2nd stage features in it (typically how is the last convolutional layer called), and how to load it.
Then you'll need to register your feaure extractor (tell the object detection API that it exists) by modifying the file object_detection/builders/model_builder.py.
Finally you should be able to make a config file with your custom feature extractor, et voilà !
I am very new to CNTK.
I wanted to train a set of images (to detect objects like alcohol glasses/bottles) using CNTK - ResNet/Fast-R CNN.
I am trying to follow below documentation from GitHub; However, it does not appear to be a straight forward procedure. https://github.com/Microsoft/CNTK/wiki/Object-Detection-using-Fast-R-CNN
I cannot find proper documentation to generate ROI's for the images with different sizes and shapes. And how to create object labels based on the trained models? Can someone point out to a proper documentation or training link using which I can work on the cntk model? Please see the attached image in which I was able to load a sample image with default ROI's in the script. How do I properly set the size and label the object in the image ? Thanks in advance!
sample image loaded for training
Not sure what you mean by proper documentation. This is an implementation of the paper (https://arxiv.org/pdf/1504.08083.pdf). Looks like you are trying to generate ROI's. Can you look through the helper functions as documented at the site to parse what you might need:
To run the toy example, make sure that in PARAMETERS.py the datasetName is set to "grocery".
Run A1_GenerateInputROIs.py to generate the input ROIs for training and testing.
Run A2_RunCntk_py3.py to train a Fast R-CNN model using the CNTK Python API and compute test results.
The algo will work on several candidate regions and then generate outputs: one for the classes of objects and another one that generates the bounding boxes for the objects belonging to those classes. Please refer to the code for getting the details of the implementation.
Can someone point out to a proper documentation or training link using which I can work on the cntk model?
You can take a look at my repository on GitHub.
It will guide you through all the steps required to train your own model for object detection and classification with CNTK.
But in short the proper steps should look something like this:
Setup environment
Prepare data
Tag images (ground truth)
Download pretrained model and create mappings for your custom dataset
Run training
Evaluate the model on test set