I am using the tensorflow object detection API for the object detection task. However, I have objects that are captured from a high angle (camera at 10 m) and in a very small size where the size of images is 1920 x 1080.
Questions:
1) What is the best way to detect small objects under this condition?
2) What are the features of suitable dataset? Images from the same views (maybe!)?
I appreciate all of your answers, Thanks :)
You have to consider object detector's input size, even if you use high resolution image such as 1920x1080.
Because object detector resize input image to their architecture size(ex. general YOLO use 410x410 input in their architecture)
On the other hand, if you use 1920x1080 image as it is, your API will resize it to small resolution like 410x410.
It means your small objects in images will be disappeared during passing through convolution filter.
In my opinion,
1) If you know where small objects is located in whole image, CROP&SEPARATE image and USE as an input image.
Even though you do not know where small objects is, you can make several candidate list that is separated by some method.
2) I don't understand what you want to know, please let me know more specific.
I think you should try "faster_rcnn_resnet101" model with kitti dataset, this has the max image size of 1987. but this model is very slow compared to any other SSD models. The configuration link is below -
https://github.com/tensorflow/models/blob/001a2a61285e378fef5f45386f638cb5e9f153c7/research/object_detection/samples/configs/faster_rcnn_resnet101_kitti.config
Also the Faster rcnn models do better job compared to yolo in small object detection, not sure of performance with ssd model.
Related
I am trying to build CNN model using TensorFlow at my own data set. But i faced with problem that is i have many pictures with different sizes. There are one kind of object in my pictures. If i make all pictures with same size, objects at pictures are not same size. In order to run CNN model with TensorFlow how to fix this problem? I heard one thing from others that is no matter having different size of input data, using tf.reduce_max, tf.reduce_mean is the best solution. if it is true that best solution to fix my problem, how to use this in my CNN model?
If i make all pictures with same size, objects at pictures are not same size.
If you know already how to make your input images to have the same size, you are ready for your task to train your CNN model. Unless you have a strict need to make the object for the picture to have the same size, it does not matter to the network.
Usual approach is to resize the images to a fixed size that is accepted by the network as input. This means distorting the aspect ratio of objects.
If that bothers you, you could try padding the images to a square (supposing the network input is a square) and then resize. This will keep the aspect ratio, but add some extra-information (the padding).
Another option is to crop the image to a square, if you are confident you are not losing important information and your task allows it.
I have a working object detection model (fined-tuned MobileNet SSD) that detects my custom small robot. I'll feed it some webcam footage (which will be tied to a drone) and use the real-time bounding box information.
So, I am about to purchase the camera.
My questions: since SSD resizes the input images into 300x300, is the camera resolution very important? Does higher resolution mean better accuracy (even when it gets resized to 300x300 anyway)? Should I crop the camera footage into 1:1 aspect ratio at every frame before running my object detection model on it? Should I divide the image into MxN cropped segments and run inference one by one?
Because my robot is very small and the drone will be at a 4 meter altitude, so I'll effectively be trying to detect a very tiny spot on my input image.
Any sort of wisdom is greatly appreciated, thank you.
These are quite a few questions, I'll try to answer all of them. The detection model resizes the input images before feeding it to the network by some resizing method, e.g. bilinear. It would be better of course if the input image would be equal or larger than the input size to the network rather than smaller. A rule of thumb is that indeed higher resolution means better accuracy, but it highly depends on the setup and the task. If you're trying to detect a small object, and let's say for example that the original resolution is 1920x1080. Then after resizing the image, the small object would be even smaller (pixels-wise), and might be too small to detect. Therefore, indeed, it would be better to either split the image to smaller images (possibly with some overlap to avoid misdetection due to object splitting) and applying detection on each, or using a model with higher input resolution. Be aware that while the first is possible with your current model, you'll need to train a new model possibly with some architectural changes (e.g. adding SSD layers and modifying anchors, depends on the scales you want to detect) for the latter. Regarding the aspect ratio matter, you mostly need to stay consistent. It doesn't matter if you don't keep the original aspect ratio, but if you don't - do it both in training and evaluation/test/deployment.
The Tensorflow Object Detection API offers a variety of models. These are trained at 600x600 image size. Suppose I have a 6000x4000 satellite image, and I want to detect objects continuously throughout the image. What is the best practice for adapting a TFODI model to this image size? I don't care about the running time per image for object detection. I have a GPU with 9GB of RAM.
I know I can fit a single 6000x4000 image onto this GPU. I'm not sure if I can fit an image processing neural net for that size onto the GPU. I can think of a few alternatives:
Chip the image into 600x600 blocks, which risks losing features that cross the blocks, but then everything should work out of the box.
Change the image dimensions in the model definition from 600x600 to 6000x4000. Can I retrain from the Model Zoo checkpoint, or do I have to start from scratch if I do this?
Compress the image to smaller size. This distorts the image dimensions and also loses feature detail. For say a picture of a city, the resulting detail would not be adequate to pick out cars and small houses.
You need to try with different sizes and see using what size during training you don't run out of memory. The memory consumption also depends how many images you have that you are training on. From what you described you will end up using a intermediate size of the image
Is there an optimal size on which to run the object detection networks available in the object detection API? The API seems to accept images of all sizes, but it is unclear to me what type and how the image is being rescaled before feeding to the network.
Could you please clarify?
Thanks!
There is a script called preprocessor_builder which is responsible for that. So whenever you feed an image to the network it has to go through this preprocessing and makes sure that the image is resized properly to match the network depending on your network configuration file.
And actual resizing is happening here.
The answer is dependent on which model you're running. For our SSD models, we will reshape the image to 300x300 pixels. For FasterRCNN or RFCN, we'll reshape between 600-1024 pixels.
The images the user should add into the TFRecord can be any size, but we recommend users keep sizes as small as possible (ie. ~400-600px max per dimension for SSD, or ~1500px max per dimension for FasterRCNN or R-FCN) for memory reasons.
I have trained a model (faster rcnn based) to identify 80x80 sized objects in 1000x600 images.
Inference works well when presented with 1000x600 test image.
However, my final goal is to be able to detect such objects (80x80) in very high res photographs (5000x4000 or higher, sometimes 10x of that).
What options do I have?
One way I am thinking is to split the large image into smaller images of 1000x600 and do inference on them. But there are challenges in that approach.
Anyone has tried this use case and found any workable solution?
--
What I would do is:
Reduce the size of the image 5000x4000 -> 1000x600
Predict the objects; you will get the xminx, xmaxs, ymins, ymaxs -> normalize them by width and height to get a value space of 0 and 1
Take the original image and re-normalized the object boxes by the original width and height
Your suggested approach to split the image should work as well but will be computationally more expensive.
You can either:
do patch-by-patch inference and use non-maximum-suppression to handle border cases, or
make your training images the same size as your testing images by padding.
Let us know what you ended up doing!