Unable to detect custom object after training using tensorflow object detection. There are No bounding boxes drawn at all. The image is exactly same as input. Even for the training example image its not able to recognize anything
I am trying to detect a specific object like car by doing following
190 training samples
40 test samples
Number of classes is 1
My training loss begins around 14.0 and ends with 0.7 after 1000 iterations. I am using ssd_mobilenet_v1_coco and only additionally changing the config file, as follows as my images are all "around" this size, apart from running for 1000 iterations.
image_resizer {
fixed_shape_resizer {
height: 450
width: 300
}
}
Please help how to debug this. I dont know where exactly to start.
I have annotated (drawn bounding box) images twice without any help. The images are of sizes varying from 425x300 to 450x300.
Related
I want to retrain quantized Mobilenet-SSD V2 model so i downloaded the unlabeled folder from COCO. This model requires input size of 300x300 but i succeeded retrainig it once on pictures of a different size and it worked (poorly, but worked).
Also, the code that uses the retrained model resizes the input from the camera to 500x500 and it works. So my question is, why is it written that the required input is 300x300 if it works with other sizes too? Do I need to resize all the dataset to 300x300 before I label them? I know it does convolution on the input so i don't think the size really matters (fix me if im wrong). As I know, the convolution occoure until we reach the end of the input.
Thanks for helping!
If I understand correctly you are using TF Object Detection API.
A given model, as mobilenet-v2-ssd, contains 3 main blocks:
[prepeocessing (normalizing and resizing] --> [Detector (backbone + detection heads)] --> [Postprocessing(bbox decoding+nms)]
When they talk about required input, it is for the detector.. The checkpoint itself contain the full pipeline, which means that the preprocessing unit will do the work for you - so there is no need to resize it to 300x300 beforehand.
if for some reason you intend to inject the input by yourself directly to the detector you have do the same preprocessing what was done in the training.
BTW:
in the config file of the training (https://github.com/tensorflow/models/blob/master/research/object_detection/samples/configs/ssd_mobilenet_v2_coco.config)
you can see the resize that was defined:
image_resizer {
fixed_shape_resizer {
height: 300
width: 300
}
}
- the normalization is mobilenet normalization (changing the dynamic range of the input from [0,255] to [-1,1]
I have been trying to train an object detection model using the tensorflow object detection API.
The network trains well when batch_size is 1. However, increasing the batch_size leads to the following error after some steps.
Network : Faster RCNN
train_config: {
batch_size: 1
optimizer {
momentum_optimizer: {
learning_rate: {
manual_step_learning_rate {
initial_learning_rate: 0.0002
schedule {
step: 25000
learning_rate: .00002
}
schedule {
step: 50000
learning_rate: .000002
}
}
}
momentum_optimizer_value: 0.9
}
use_moving_average: false
}
Error:
INFO:tensorflow:Error reported to Coordinator: , ConcatOp : Dimensions of inputs should match: shape[0] = [1,841,600,3] vs. shape[3] = [1,776,600,3]
[[node concat (defined at /home/<>/.virtualenvs/dl4cv/lib/python3.6/site-packages/object_detection-0.1-py3.6.egg/object_detection/legacy/trainer.py:190) ]]
Errors may have originated from an input operation.
Input Source operations connected to node concat:
Preprocessor_3/sub (defined at /home/<>/.virtualenvs/dl4cv/lib/python3.6/site-packages/object_detection-0.1-py3.6.egg/object_detection/models/faster_rcnn_inception_v2_feature_extractor.py:100)
The training with increased batch_size works on SSD mobilenet however.
While, I have solved the issue for my use-case at the moment, posting this question in SO to understand the reason for this behavior.
The reason you get an error is because you cannot technically train Faster RCNN in batch mode on a single GPU. This is due to its two stage architecture. SSD is single stage and hence can be parallelized to give larger batch sizes. If you still want to train F-RCNN with batch size>1, you can do so with multiple GPUs. There is a --num_clones parameter that you need to set to the number of GPUs available to you. set the num_clones and the batchsize to save values (It should be equal to the number of GPUs you have available)
I have used batchsizes of 4,8 and 16 in my application.
--num_clones=2 --ps_tasks=1
Check this link for more details
https://github.com/tensorflow/models/issues/1744
Just from the error it seems like your individual inputs have different sizes. I suppose it tries to concatenate (ConcatOp) 4 single inputs into one tensor to build a mini batch as the input.
While trying to concatenate it has one input with 841x600x3 and one input with 776x600x3 (ignored the batch dimension). So obviously 841 and 776 are not equal but they should be. With a batch size of 1 the concat function is probably not called, since you don't need to concatenate inputs to get a minibatch. There also seems to be no other component that relies on a pre defined input size, so the network will train normally or at least doesn't crash.
I would check the dataset you are using and check if this is supposed to be this way or you have some faulty data samples. If the dataset is ok and this can in fact happen you need to resize all inputs to some kind of pre defined resolution to be able to combine them probably into a minibatch.
You don't need to resize every image in your dataset. Tensorflow can handle it if you specify in your config file.
Default frcnn and ssd config is:
## frcnn
image_resizer {
keep_aspect_ratio_resizer {
min_dimension: 600
max_dimension: 1024
}
}
## ssd
image_resizer {
fixed_shape_resizer {
height: 300
width: 300
}
}
If you change image resizer of frcnn as fixed_shape_resizer like in ssd, You can increase the batch size.
I implemented it and training went well. Unfortunately, my loss didn't decrease as I expected. Then, I switched back to batch size 4 with 4 workers (it means batch size 1 for each worker). Latter is better for my case, but maybe it can be different for your case.
When increasing the batch size, the images loaded in the Tensors should all be of the same size.
This is how you may get the images to be all of the same size:
image_resizer {
keep_aspect_ratio_resizer {
min_dimension: 896
max_dimension: 896
pad_to_max_dimension: true
}
}
Padding the images to the maximum dimension, making that "true", that will cause the images to be all of the same size. This enables you to have a batch size larger than one.
So let's say my original raw dataset has 100 images. And I apply random_horizontal_flip data augmentation, which by default horizontally flips with 50% probability. So just for the sake of example, lets say it flips 50 of the 100 images. So,
Does that mean my algorithm will now be trained with 150 images (100 original and 50 flipped versions) or does it mean it will be trained with 100 images still, but 50 of them will be the flipped versions of the originals?
Is the answer to question #1 generalizable to all data augmentation options provided by Tensorflow object detection API?
I read as much official documentation as possible, and looked into preprocessor code, but couldn't find my answer.
Default augmentation probability, which is 50%, is independenetly applied to each image. Number of images that your model/algorithm is trained on depends on the number of epochs.
Let's say your batch size is 1 and total number of epoch is 100:
Your algoirthm will be trained on 100 images, 50 of them will be flipped version of the original images. In this case, model will not see the original 50 images because your epoch is too low.
Let's say your batch size is 1 and total number of epoch is 200:
Your algoirthm will be trained on 200 images, 100 of them will be flipped version of the original images.
As a result, as long as your epoch size is not limiting your dataset, with the probability of 50%, you will see an effect as if you have doubled the dataset by flipping each item.
In addition to the horizontal flip, if you add the vertical flip (random_vertical_flip) too, you triple your dataset.
I am using tensorflow object detection API to train and detect my dataset. This dataset has 5 classes(every class has 50 images), and it contains two very similar classes(red and black). After the train process, I detected the test images and found that, the model always detects a target of red class as a target of black class, the other classes is detected as correct class.
I trained the model with faster_rcnn_resnet101_breads.congfig, and using fine_tune_checkpoint.I set the learning_rate to 0.003(the original is 0.0003).
Can you tell me what is wrong with my model, and how much the learning_rate should I set?
The compare result of my config file and samples config file:compare result
train curves:train curves
> black class: https://i.stack.imgur.com/eWrlK.jpg
> red class: https://i.stack.imgur.com/TuRjg.jpg
I am running tutorial from object detection API and I am using Oxford dataset with ResNet Faster-RCNN.
When I evaluate my trained model by running (eval.py), Tensorboard returns about 0.95 smoothed precision value.
My question is how many image set does it evaluate? Because from Tensorboard and their tutorial link (https://github.com/tensorflow/models/blob/master/object_detection/g3doc/running_pets.md), Tensorboard only shows 10 images.
Does it mean that they check precision only with 10 images?
My Oxford dataset validation jpg counts should be about 2,200.
In my configuration, I specified input path correctly like this:
eval_input_reader: {
tf_record_input_reader {
input_path: "my_path/pet_val.record"
}
label_map_path: "my_path/pet_label_map.pbtxt"
shuffle: false
num_readers: 1
}
And does eval.py prints mAP at the end?
I run eval.py about three days ago on my 1 GPU local machine, but it does not print anything.
Finally, does this API provide F-value and fps (frame per second)? Anyone has experience with this?
edit: it seems that we can setup eval size limit from configuration such as /object_detection/samples/configs/faster_rcnn_resnet101_pets.config#L131. When I print len(result_lists) from https://github.com/tensorflow/models/blob/master/object_detection/eval_util.py#L404, it prints 2000, which was my eval num_examples.
I was also able to generate fps by comparing timestamp manually.
By default, we only visualize 10 images on Tensorboard (to avoid overwhelming it with images) but this is configurable from the eval_config. You can also change the number of images evaluated (defaults to 5000) in the config too.