This is my first time to use D435 depth image to detect object distance. As I know, we can use Realsense SDK depthframe.getDistance(x, y) to get point(x,y) distance. However, the distance seems not correspond to depth image.
For example, I detect the red point(260, 300) in and get the correct distance 0.747m and the body part in depth image is blue color because it is closed to the camera.
But when I sit down, the corresponded distance of red point(260, 300) in shows 5.17m and the point still on my body. I don't know why D435 detects the wrong distance since its depth image still shows blue color in my body part. Any good suggestion to improve the precision of depth distance.
Related
I have a camera at a known fixed location and orientation.
I also have a plane at a known location whose z position changes.
I want to turn the image from the camera into a top down view of the plane.
I can do this without knowing any positions by using the 4 points of the plane for a homography matrix and warping the image but each time the plane moves in Z I have to repeat this process.
After searching around online most methods seem to center on finding features of the image (using SIFT or something like it) then computing a homography matrix.
With the problem so constrained I thought there may be a simple linear algebra based approach.
When taking a depth pixel buffer from ARKit session like:
CVPixelBufferRef depthDataMap = frame.capturedDepthData.depthDataMap;
many pixels are in black. What do they represent? Why are they black? This doesn't reflect the real depth of the real world.
How to interpret this?
It's some sort of "trail" artifact.
Those black pixels in a Depth channel are digital artifacts. The appearance of this black outline became possible due to the fact that the depth data is captured at a frequency of 15 fps (at every fourth frame), in contrast to the RGB image which is captured at a frequency of 60 fps. Also, a Depth channel image is of a low quality (low-res). Thus, if the subject quickly or abruptly moves its head at the time of AR session running, when every three out of four depth frames contain a completely black background – this leads to a situation when RGBA and Depth data don't match each other, so we get these nasty digital artifacts.
Look at a picture in a post How to improve People Occlusion in ARKit 3.0 (click on a link below).
If you need an additional info about a depth map and its trailing effect, please read this post.
This means no depth data for this pixel (depth = 0)
I have a flat pan and using kinetic v1.
I want to receive the angle of the pan using kinetic camera.
for eg: If I put the angle in 45 degrees so kinetic will read the closet or exact angle it placed.
is this possible or any solutions ?
Thanks.
I don't know exactly how the data comes back in Kinect V1 but I believe this methodology should work for you.
First: You have to assume that the Kinect is your level of reference, if it is necessary to get the pans angle relative to the ground then make sure the Kinect is level with the ground.
Second: Separate the pan data from all other data. This should be straight forward, the pan should be the closets object so transmit the closest measurements into 3D coordinate points (array of x,y,z).
Third: Assuming you wish for horizontal angle find the highest and lowest grounds of data and average their depth from the camera. Then save both those depths and the vertical distance they are away from each other.
Fourth: Now you can essentially do the math for a triangle. Given you know the width of the pan(saves steps to know the objects size otherwise you have to estimate that too) you can solve for a triangle with sides a: distance to point 1, side b: distance to point 2, side c: size of pan and finding the angle of where points a and c or b and c meet will give you the horizontal angle of the pan relative to the Kinect.
Fifth: For verification your measurements came back correct you can then use the angle you found to calculate the width of the pan given the angle and distance of the top and bottom most points.
Needless to say, you need to make sure that your understanding of trig is solid for this task.
I'm using Kinect v2 and Kinect SDK v2.
I have couple of questions about coordinate mapping:
How to transfer a camera space point (point in 3d coordinate system) to depth space with depth value?
Current MapCameraPointToDepthSpace method can only return the depth space coordinate.
But without depth value, this method is useless.
Did anyone know how to get the depth value?
How to get the color camera intrinsic?
There is only a GetDepthCameraIntrinsics methos to get depth camera intrinsic.
But how about color camera?
How to use the depth camera intrinsic?
Seems that the Kinect 2 consider the radial distortion.
But how to use these intrinsic to do the transformation between depth pixel and 3d point?
Is there any example code can do this?
Regarding 1: The depth value of your remapped world coordinate is the same as in the original world coordinate's Z value. Read the description of the depth buffer and world coordinate space: this value in both is simply the distance from the point to Kinect's plane, in meters. Now, if you want the depth value of the object being seen on the depth frame directly behind your remapped coordinate, you have to read the depth image buffer in that position.
Regarding 3: You use the camera's intrinsic when you have to manually construct a CoordinateMapper object (i.e. when you don't have a Kinect available). When you get the CoordinateMapper associated to a Kinect (using Kinect object's CoordinateMapper property), it already contains that Kinect's intrinsics... that's why you have a GetDepthCameraIntrinsics method which returns that specific Kinect's intrinsics (they can vary from device to device).
Regarding 2: There is now way to get the color camera intrinsic. You have to evaluate them by camera calibration.
I have a robot and a camera. The robot is just a 3D printer where I changed the extruder for a tool, so it doesn't print but it moves every axis independently. The bed is transparent, and below the bed there is a camera, the camera never moves. It is just a normal webcam (playstation eye).
I want to calibrate the robot and the camera, so that when I click on a pixel on a image provided by the camera, the robot will go there. I know I can measure the translation and the rotation between the two frames, but that will probably return lots of errors.
So that's my question, how can I relate the camera and a robot. The camera is already calibrated using chessboards.
In order to make everything easier, the Z-axis can be ignored. So the calibration will be over X and Y.
It depends of what error is acceptable for you.
We have similar setup where we have camera which looks at some plane with object on it that can be moved.
We assume that the image and plane are parallel.
First lets calculate the rotation. Put the tool in such position that you see it on the center of the image, move it on one axis select the point on the image that is corresponding to tool position.
Those two points will give you a vector in the image coordinate system.
The angle between this vector and original image axis will give the rotation.
The scale may be calculated in the similar way, knowing the vector length (in pixels) and the distance between the tool positions(in mm or cm) will give you the scale factor between the image and real world axis.
If this method won't provide enough accuracy you may calibrate the camera for distortion and relative position to the plane using computer vision techniques. Which is more complicated.
See the following links
http://opencv.willowgarage.com/documentation/camera_calibration_and_3d_reconstruction.html
http://dasl.mem.drexel.edu/~noahKuntz/openCVTut10.html