I have lidar data and stitched 360-degree images. My task is to detect the individual object and localize them on the lidar data.
I have done object detection on stitched images but am unable to map those objects into lidar data.
Any help will be highly appreciated.
Thanks in advance.
Related
I am a beginner to machine learning , but I've been using yolov5 and yolov7 to train and detect objects from images. Trainings was done by annotating the objects using various annotation tools such as roboflow etc. I now wanted to enter the domain of lidar and point cloud data. I was wondering if there was such a way to annotate the objects from lidar, and then train and save objects for future detection of objects from point cloud data. Is there anyway this can be done using such algorithms or any other ways?
I am using the tensorflow object detection API for the object detection task. However, I have objects that are captured from a high angle (camera at 10 m) and in a very small size where the size of images is 1920 x 1080.
Questions:
1) What is the best way to detect small objects under this condition?
2) What are the features of suitable dataset? Images from the same views (maybe!)?
I appreciate all of your answers, Thanks :)
You have to consider object detector's input size, even if you use high resolution image such as 1920x1080.
Because object detector resize input image to their architecture size(ex. general YOLO use 410x410 input in their architecture)
On the other hand, if you use 1920x1080 image as it is, your API will resize it to small resolution like 410x410.
It means your small objects in images will be disappeared during passing through convolution filter.
In my opinion,
1) If you know where small objects is located in whole image, CROP&SEPARATE image and USE as an input image.
Even though you do not know where small objects is, you can make several candidate list that is separated by some method.
2) I don't understand what you want to know, please let me know more specific.
I think you should try "faster_rcnn_resnet101" model with kitti dataset, this has the max image size of 1987. but this model is very slow compared to any other SSD models. The configuration link is below -
https://github.com/tensorflow/models/blob/001a2a61285e378fef5f45386f638cb5e9f153c7/research/object_detection/samples/configs/faster_rcnn_resnet101_kitti.config
Also the Faster rcnn models do better job compared to yolo in small object detection, not sure of performance with ssd model.
I need to use a multimodal dataset in order to detect objects for an autonomous vehicle project.
The NuScenes Dataset just came with a lot of data : https://www.nuscenes.org/. Did anyone know how to project the lidar pointcloud into camera image plane with projection matrix just like for the Kitti Dataset?
Have a look at the official nuscenes SDK and the function map_pointcloud_to_image.
I believe, it does exactly what you have in mind.
https://github.com/nutonomy/nuscenes-devkit/blob/master/python-sdk/nuscenes/nuscenes.py
You can call the map_pointcloud_to_image function by creating an instance of NuSceneseExplorer.
from nuscenes import NuScenesExplorer
I want to train Sagemaker Object detection algorythm with my own dataset and want to know if the dataset must have images without object or not.
My dataset comes from Ground Truth with images from an outdoor camera, somes with object labelled, some without label (no object).
yes, absolutely. This will help you model learn better. For example, the Caltech-256 dataset has a clutter class (#257) with random images containing no objects: http://www.vision.caltech.edu/Image_Datasets/Caltech256/images/
I obtain depth & reflectance maps from Lidar (2D images) and I have also camera images (2D images). Image have the same size.
I want to use CNN to perform object detection using both images. It is a sort of "fusion CNN"
How am I suppose to do it? Did I am suppose to use a pre-train model? But the is no pre-train model using lidar images..
Which is the best CNN algorithm to do it? ie for performing fusion of modalities for object detection
Thanks you in advance
Did I am suppose to use a pre-train model?
Yes you should, unless you are super confident that you can find a working model directly by urself.
But the is no pre-train model using lidar image
First I`m pretty sure there are LIDAR based network .e.g
L Caltagirone , LIDAR-Camera Fusion for Road Detection Using Fully
Convolutional ... arxiv, 2018
Second, even if there is no open source implementation for direct LIDAR-based, You can always convert the LIDAR to the depth image. For Depth image based CNN, there are hundreds of implementation for segmentation and detection.
How am I suppose to do it?
First, you can place them side by side parallel, for RGB and depth/LIDAR 3d pointcloud. Feed them separately
Second, you can also combine them by merging the input to 4D tensor and transfer the initial weight to the single model. At last perform transfer learning in your given dataset.
best CNN algorithm?
Totally depends on your task and hardware. Do you need best in processing speed or best in accuracy? Define your "best", please.
ALso Are you using it for autonomous car or for in-house nurse care system? different CNN system customizes the weight for different purposes.
Generally, for real-time multiple object detection using a cheap PC e.g DJI manifold, I would suggest Yolo-tiny