I've a question regarding a line in camera coordinate.
Suppose the pixel/screen coordinate of a point is (u,v). And the camera coordinate
(coordinate system relative to camera) of (u,v) is (p,q,r) where (u,v) is given and a
line L goes through the point (0,0,0) [origin camera location] and (p,q,r) where r is
given. Is it possible to find (p,q)?
I know that the parametric equation of a line is:
(x-a, y-b, z-c)= t(x_0, y_0, z_0)
But I know only (a,b,c) which is (0,0,0) and z_0 which is r. Can anyone kindly tell me if it is possible to find
the value of (p,q)? Can I use (u,v) in some way?
It's not possible until you have more information about what something like (u, v) represents. Think of it this way. Suppose you claimed you could figure it out just based on (u, v) and r. Now, what if I just relabeled your pixels? A pixel doesn't have to represent any specific distance, so if I said (125, 100) was (250, 200) instead, that would make sense too. Suppose I just swap in a higher resolution chip for a lower resolution chip.
To actually recover (p, q), you'd have to know what physical distance a pixel corresponds to. You'd also have to know whether the pinhole in your camera model is (0,0) in your pixel reference frame, etc.
Related
Let's say I know two persons are standing at GPS location A and B. A is looking at B.
I would like to know B's (x, y, z) coordinates based on A, where the +y axis is the direction to B (since A is looking at B), +z is the vertically to the sky. (therefore +x is right-hand side of A)
I know how to convert a GPS coordinate to UTM, but in this case, a coordinate system rotation and translation seem needed. I am going to come up with a calculation, but before that, will there be some codes to look at?
I think this must be handled by many applications, but I could not find so far.
Convert booth points to 3D Cartesian
GPS suggest WGS84 so see How to convert a spherical velocity coordinates into cartesian
Construct transform matrix with your desired axises
see Understanding 4x4 homogenous transform matrices. So you need 3 perpendicular unit vectors. The Y is view direction so
Y = normalize(B-A);
one of the axises will be most likely up vector so you can use approximation
Z = normalize(A);
and as origin you can use point A directly. Now just exploit cross product to create X perpendicular to both and make also Y perpendicular to X and Z (so up stays up). For more info see Representing Points on a Circular Radar Math approach
Transfrom B to B' by that matrix
Again in the QA linked in #1 is how to do it. It is simple matrix/vector multiplication.
Recently I'm struggling with a pose estimation problem with a single camera. I have some 3D points and the corresponding 2D points on the image. Then I use solvePnP to get the rotation and translation vectors. The problem is, how can I determine whether the vectors are right results?
Now I use an indirect way to do this:
I use the rotation matrix, the translation vector and the world 3D coordinates of a certain point to obtain the coordinates of that point in Camera system. Then all I have to do is to determine whether the coordinates are reasonable. I think I know the directions of x, y and z axes of Camera system.
Is Camera center the origin of the Camera system?
Now consider the x component of that point. Is x equavalent to the distance of the camera and the point in the world space in Camera's x-axis direction (the sign can then be determined by the point is placed on which side of the camera)?
The figure below is in world space, while the axes depicted are in Camera system.
========How Camera and the point be placed in the world space=============
|
|
Camera--------------------------> Z axis
| |} Xw?
| P(Xw, Yw, Zw)
|
v x-axis
My rvec and tvec results seems right and wrong. For a specified point, the z value seems reasonable, I mean, if this point is about one meter away from the camera in the z direction, then the z value is about 1. But for x and y, according to the location of the point I think x and y should be positive but they are negative. What's more, the pattern detected in the original image is like this:
But using the points coordinates calculated in Camera system and the camera intrinsic parameters, I get an image like this:
The target keeps its pattern. But it moved from bottom right to top left. I cannot understand why.
Yes, the camera center is the origin of the camera coordinate system, which seems to be right following to this post.
In case of camera pose estimation, value seems reasonable can be named as backprojection error. That's a measure of how well your resulting rotation and translation map the 3D points to the 2D pixels. Unfortunately, solvePnP does not return a residual error measure. Therefore one has to compute it:
cv::solvePnP(worldPoints, pixelPoints, camIntrinsics, camDistortion, rVec, tVec);
// Use computed solution to project 3D pattern to image
cv::Mat projectedPattern;
cv::projectPoints(worldPoints, rVec, tVec, camIntrinsics, camDistortion, projectedPattern);
// Compute error of each 2D-3D correspondence.
std::vector<float> errors;
for( int i=0; i < corners.size(); ++i)
{
float dx = pixelPoints.at(i).x - projectedPattern.at<float>(i, 0);
float dy = pixelPoints.at(i).y - projectedPattern.at<float>(i, 1);
// Euclidean distance between projected and real measured pixel
float err = sqrt(dx*dx + dy*dy);
errors.push_back(err);
}
// Here, compute max or average of your "errors"
An average backprojection error of a calibrated camera might be in the range of 0 - 2 pixel. According to your two pictures, this would be way more. To me, it looks like a scaling problem. If I am right, you compute the projection yourself. Maybe you can try once cv::projectPoints() and compare.
When it comes to transformations, I learned not to follow my imagination :) The first thing I Do with the returned rVec and tVec is usually creating a 4x4 rigid transformation matrix out of it (I posted once code here). This makes things even less intuitive, but instead it is compact and handy.
Now I know the answers.
Yes, the camera center is the origin of the camera coordinate system.
Consider that the coordinates in the camera system are calculated as (xc,yc,zc). Then xc should be the distance between the camera and
the point in real world in the x direction.
Next, how to determine whether the output matrices are right?
1. as #eidelen points out, backprojection error is one indicative measure.
2. Calculate the coordinates of the points according to their coordinates in the world coordinate system and the matrices.
So why did I get a wrong result(the pattern remained but moved to a different region of the image)?
Parameter cameraMatrix in solvePnP() is a matrix supplying the parameters of the camera's external parameters. In camera matrix, you should use width/2 and height/2 for cx and cy. While I use width and height of the image size. I think that caused the error. After I corrected that and re-calibrated the camera, everything seems fine.
Say I want to construct a 3D cubic Bézier curve, and I already have both end-points, and the direction (normal vector) for both control points. How can I choose the distance of both control points to their respective end-points in order to make the curve as 'nicely rounded' as possible?
To formalize 'nicely rounded': I think that means maximizing the smallest angle between any two segments in the curve. For example, having end-points (10, 0, 0) and (0, 10, 0) with respective normal vectors (0, 1, 0) and (1, 0, 0) should result in a 90° circular arc. For the specific case of 2D circular arcs, I've found articles like this one. But I haven't been able to find anything for my more general case.
(Note that these images are just to illustrate the 'roundness' concept. My curves are not guaranteed to be plane-aligned. I may replace the images later to better illustrate that point.)
This is a question of aesthetics, and if the real solution is unknown or too complicated, I would be happy with a reasonable approximation. My current approximation is too simplistic: choosing half the distance between the two end-points for both control point distances. Someone more familiar with the math will probably be able to come up with something better.
(PS: This is for open-source software, and I would be happy to give credit on GitHub.)
Edit: Here are some other images to illustrate a 3D case (jsfiddle):
Edit 2: Here's a screenshot of an unstable version of ApiNATOMY to give you an idea of what I'm trying to do. I'm creating 3D tubes to represent blood-vessels, connecting different parts of an anatomical schematic:
(They won't let me put in a jsfiddle link if I don't include code...)
What you are basically asking is to have curvature over the spline as constant as possible.
A curve with constant curvature is just a circular arc, so it makes sense to try to fit such an arc to your input parameters. In 2D, this is easy: construct the line which goes through your starting point and is orthogonal to the desired direction vector. Do the same for the ending point. Now intersect these two lines: the result is the center of the circle which passes through the two points with the desired direction vectors.
In your example, this intersection point would just be (0,0), and the desired circular arc lies on the unit circle.
So this gives you a circular arc, which you can either use directly or use the approximation algorithm which you have already cited.
This breaks down when the two direction vectors are collinear, so you'd have to fudge it a bit if this ever comes up. If they point at each other, you can simply use a straight line.
In 3D, the same construction gives you two planes passing through the end points. Intersect these, and you get a line; on this line, choose the point which minimizes the sum of squared distances to the two points. This gives you the center of a sphere which touches both end points, and now you can simply work in the plane spanned by these three points and proceed as in 2D.
For the special case where your two end points and the two known normal vector for the control points happen to make the Bezier curve a planar one, then basically you are looking for a cubic Bezier curve that can well approximate a circular arc. For this special case, you can set the distance (denoted as L) between the control point and their respective end point as L = (4/3)*tan(A/4) where A is the angle of the circular arc.
For the general 3D case, perhaps you can apply the same formula as:
compute the angle between the two normal vectors.
use L=(4/3)*tan(A/4) to decide the location of your control points.
if your normals are aligned in a plane
What you're basically doing here is creating an elliptical arc, in 3D, where the "it's in 3D" part is completely irrelevant, since it's just a 2D curve, rotated/translated to sit in your 3D space. So let's just solve the 2D case, and then the RT is entirely up to you.
Creating the "perfect" cubic Bezier between two points on an arc comes with limitations. You basically can't create good looking arcs that span more than a quarter circle. So, with that said: your start and end point normals give you a 2D angle between your normal vectors, which is the same angle as between your start and end tangents (since normals are perpendicular to tangents). So, let's:
align our curve so that the tangent at the start is 0
plug the angle between tangents into the formula given in the section on Circle approximation in the Primer on Bezier curves. This is basically just dumb "implementing the formula for c1x/c1y/c2x/c2y as a function that takes an angle as argument, and spits out four values as c1(x,y) and c2(x,y) coordinats".
There is no step 3, we're done.
After step 2, you have your control points in 2D to create the most circular arc between a start and end point. Now you just need to scale/rotate/translate it in 3D so that it lines up with where you needed your start and end point to begin with.
if your normals are not aligned in a plane
Now we have a problem, although one that we can deal with by treating the dimensions as separate things entirely. Instead of creating a single 2D curve, we're going to create three: one that's the X/Y projection, one that's the X/Z projection, and one that's the Y/Z projection. For all three of these, we're going to abstract the control points in exactly the same way as before, and then we simply take the projective control points (three for each control point), and then go "okay, we now have X, Y, and Z projective coordinates. That means we have (X,Y,Z) coordinates", and done again.
I'm a libgdx NOOB - apologies if this is an obvious question..
Using libgdx I have set up a Perspective camera, looking at origin (camera near = 1f, far = 300f). Viewport extends across the entire screen.
Basically, I would like to know how to convert a 2D screen coordinate (x,y) to the 3D world coordinate (x, y, z) where the Z value is clamped to the camera's near plane.
I think I should use the camera.getPickRay method to get a picking ray for the screen coordinate. I should then get the intersection point of this ray and the camera's near plane to get the world coordinate of the point on the near plane.
I thought that the resulting Ray object's origin property was the near-plane intersection point I was after, but this doesn't seem to be the case.
Am I on the right track?
Camera#unproject() returns, depending on the z-value, the value between the near (z=0) and far plane (z=1), see the javadocs. Camera#getPickRay() sets the origin member to the unprojected value at z=0, thus on the near plane, see the code. If you don't need the ray (including the direction) then you don't have to calculate the pick ray, instead you can call the unproject method directly.
Vector3 pointOnNearPlane = camera.unproject(new Vector3(touchX, touchY, 0f));
Likewise, for the point on the far plane:
Vector3 pointOnFarPlane = camera.unproject(new Vector3(touchX, touchY, 1f));
I am currently trying to figure out a way to calcute the size of a given object with kinect
since I have the following data
angular field of view of the lens
distance
and width in pixels from a 800*600 resolution
I believe this can be possible to calculate. Does anyone has math skills to give me a little help?
With some trigonometry, it should be possible to approximate.
If you draw a right trangle ABC, with the camera at one of the legs (A), and the object at the far end (edge BC), where the right angle is (C), then the height of the object is going to be the height of leg BC. the distance to the pixel might be the distance of leg AC or AB. The Kinect sensor specifications are going to regulate that. If you get distance to the center of a pixel, then it will be AC. if you have distances to pixel corners then the distance will be AB.
With A representing the angle at the camera that the pixel takes up, d is the distance of the hypotenuse of a right angle and y is the distance of the far leg (edge BC):
sin(A) = y / d
y = d sin(A)
y is the length of the pixel projected into the object plane. You calculate it by multiplying the sin of the angel by the distance to the object.
Here I confess I do not know the API of the kinect, and what level of detail it provides. You say you have the angle of the field of vision. You might assume each pixel of your 800x600 pixel grid takes up an equal angle of your camera's field of vision. If you do, then you can break up that field of vision into equal pieces to measure the linear size of your object in each pixel.
You also mentioned that you have the distance to the object. I was assuming that you have a distance map for each pixel of the 800x600 grid. If this is incorrect, some calculations can be done to approximate a distance grid for the pixels involving the object of interest if you make some assumptions about the object being measured.