I'm a libgdx NOOB - apologies if this is an obvious question..
Using libgdx I have set up a Perspective camera, looking at origin (camera near = 1f, far = 300f). Viewport extends across the entire screen.
Basically, I would like to know how to convert a 2D screen coordinate (x,y) to the 3D world coordinate (x, y, z) where the Z value is clamped to the camera's near plane.
I think I should use the camera.getPickRay method to get a picking ray for the screen coordinate. I should then get the intersection point of this ray and the camera's near plane to get the world coordinate of the point on the near plane.
I thought that the resulting Ray object's origin property was the near-plane intersection point I was after, but this doesn't seem to be the case.
Am I on the right track?
Camera#unproject() returns, depending on the z-value, the value between the near (z=0) and far plane (z=1), see the javadocs. Camera#getPickRay() sets the origin member to the unprojected value at z=0, thus on the near plane, see the code. If you don't need the ray (including the direction) then you don't have to calculate the pick ray, instead you can call the unproject method directly.
Vector3 pointOnNearPlane = camera.unproject(new Vector3(touchX, touchY, 0f));
Likewise, for the point on the far plane:
Vector3 pointOnFarPlane = camera.unproject(new Vector3(touchX, touchY, 1f));
Related
Recently I'm struggling with a pose estimation problem with a single camera. I have some 3D points and the corresponding 2D points on the image. Then I use solvePnP to get the rotation and translation vectors. The problem is, how can I determine whether the vectors are right results?
Now I use an indirect way to do this:
I use the rotation matrix, the translation vector and the world 3D coordinates of a certain point to obtain the coordinates of that point in Camera system. Then all I have to do is to determine whether the coordinates are reasonable. I think I know the directions of x, y and z axes of Camera system.
Is Camera center the origin of the Camera system?
Now consider the x component of that point. Is x equavalent to the distance of the camera and the point in the world space in Camera's x-axis direction (the sign can then be determined by the point is placed on which side of the camera)?
The figure below is in world space, while the axes depicted are in Camera system.
========How Camera and the point be placed in the world space=============
|
|
Camera--------------------------> Z axis
| |} Xw?
| P(Xw, Yw, Zw)
|
v x-axis
My rvec and tvec results seems right and wrong. For a specified point, the z value seems reasonable, I mean, if this point is about one meter away from the camera in the z direction, then the z value is about 1. But for x and y, according to the location of the point I think x and y should be positive but they are negative. What's more, the pattern detected in the original image is like this:
But using the points coordinates calculated in Camera system and the camera intrinsic parameters, I get an image like this:
The target keeps its pattern. But it moved from bottom right to top left. I cannot understand why.
Yes, the camera center is the origin of the camera coordinate system, which seems to be right following to this post.
In case of camera pose estimation, value seems reasonable can be named as backprojection error. That's a measure of how well your resulting rotation and translation map the 3D points to the 2D pixels. Unfortunately, solvePnP does not return a residual error measure. Therefore one has to compute it:
cv::solvePnP(worldPoints, pixelPoints, camIntrinsics, camDistortion, rVec, tVec);
// Use computed solution to project 3D pattern to image
cv::Mat projectedPattern;
cv::projectPoints(worldPoints, rVec, tVec, camIntrinsics, camDistortion, projectedPattern);
// Compute error of each 2D-3D correspondence.
std::vector<float> errors;
for( int i=0; i < corners.size(); ++i)
{
float dx = pixelPoints.at(i).x - projectedPattern.at<float>(i, 0);
float dy = pixelPoints.at(i).y - projectedPattern.at<float>(i, 1);
// Euclidean distance between projected and real measured pixel
float err = sqrt(dx*dx + dy*dy);
errors.push_back(err);
}
// Here, compute max or average of your "errors"
An average backprojection error of a calibrated camera might be in the range of 0 - 2 pixel. According to your two pictures, this would be way more. To me, it looks like a scaling problem. If I am right, you compute the projection yourself. Maybe you can try once cv::projectPoints() and compare.
When it comes to transformations, I learned not to follow my imagination :) The first thing I Do with the returned rVec and tVec is usually creating a 4x4 rigid transformation matrix out of it (I posted once code here). This makes things even less intuitive, but instead it is compact and handy.
Now I know the answers.
Yes, the camera center is the origin of the camera coordinate system.
Consider that the coordinates in the camera system are calculated as (xc,yc,zc). Then xc should be the distance between the camera and
the point in real world in the x direction.
Next, how to determine whether the output matrices are right?
1. as #eidelen points out, backprojection error is one indicative measure.
2. Calculate the coordinates of the points according to their coordinates in the world coordinate system and the matrices.
So why did I get a wrong result(the pattern remained but moved to a different region of the image)?
Parameter cameraMatrix in solvePnP() is a matrix supplying the parameters of the camera's external parameters. In camera matrix, you should use width/2 and height/2 for cx and cy. While I use width and height of the image size. I think that caused the error. After I corrected that and re-calibrated the camera, everything seems fine.
How do i calculate point D for triangle 2?
I have the the following coordinates for triangle 1:
a(0,0) b(0,78) c(18,39)
point D is located at (0,39) in triangle 1.
now I change the perspective on my triangle by for example moving coordinate b and c.
the new triangle formed is called triangle 2 with coordinates:
a(0,0) b(11,72) c(37,42)
AS YOU CAN SEE POINT D IS NOT IN THE MIDDLE OF LINE a<-->b BECAUSE OF THE CHANGE IN PERSPECTIVE/SKEW.
How do i calculate point d? I have the coordinates abc of triangle 1 & 2.
Preferably answer in programcode rather than using math signs, since i am not a hero at reading math :)
You need to convert point D to barycentric coordinates using the original triangle coordinates, then convert it back to cartesian coordinates using the modified triangle coordinates.
This looks like a good introduction to triangular barycentric coordinates: http://blogs.msdn.com/b/rezanour/archive/2011/08/07/barycentric-coordinates-and-point-in-triangle-tests.aspx
Also, explicit formulae for converting a point in a triangle to barycentric coordinates are given at the end of the Converting to Barycentric Coordinates section of the Wikipedia article “Barycentric coordinate system”.
I guess there are more ways of calculating a coordinate from one perspective to another.
more on the triangle way is written by culebrón here: Transforming captured co-ordinates into screen co-ordinates
At the same link there is another way by using SVD and calculate an H-matrix which can be used to translate any coordinate from one perspective to another. I am going to use this way because i could solve this way in matlab. Next step in objective-c! i had some trouble calculating the same in objective-c. more on that here: calculate the V from A = USVt in objective-C with SVD from LAPACK in xcode
I would like to know how to solve the triangle way too! i could not figure out what a1 and a2 were in culebron's post: https://stackoverflow.com/a/1690300/1568532 neither the width and height made much sense to me.
Also i would like to know how to calculate the EYE's point of view on a triangle or quadrangle based on 3 or 4 coordinates. if you know the original size of the object.
any ideas on this?
when i search for eye or camera's point of view. there is load of result about photography.
what do i need to use in order to calculate this? maybe some example anyone?
I've a question regarding a line in camera coordinate.
Suppose the pixel/screen coordinate of a point is (u,v). And the camera coordinate
(coordinate system relative to camera) of (u,v) is (p,q,r) where (u,v) is given and a
line L goes through the point (0,0,0) [origin camera location] and (p,q,r) where r is
given. Is it possible to find (p,q)?
I know that the parametric equation of a line is:
(x-a, y-b, z-c)= t(x_0, y_0, z_0)
But I know only (a,b,c) which is (0,0,0) and z_0 which is r. Can anyone kindly tell me if it is possible to find
the value of (p,q)? Can I use (u,v) in some way?
It's not possible until you have more information about what something like (u, v) represents. Think of it this way. Suppose you claimed you could figure it out just based on (u, v) and r. Now, what if I just relabeled your pixels? A pixel doesn't have to represent any specific distance, so if I said (125, 100) was (250, 200) instead, that would make sense too. Suppose I just swap in a higher resolution chip for a lower resolution chip.
To actually recover (p, q), you'd have to know what physical distance a pixel corresponds to. You'd also have to know whether the pinhole in your camera model is (0,0) in your pixel reference frame, etc.
Working with iPhone and Objective C.
I am working on a game and I need to correctly reflect a ball off a circle object. I am trying to do it as a line and circle intersection. I have my ball position outside the circle and I have the new ball position that would be inside the circle at the next draw update. I know the intersect point of the line (ball path) and the circle. Now I want to rotate the ending point of the ball path about the intersection point to get the correct angle of reflection off the tangent.
The following are known:
ball current x,y
ball end x,y
ball radius
circle center x,y
circle radius
intersection point of ball path and circle x and y
I know I need to find the angle of incidence between the tangent line and the incoming ball path which will also equal my angle of reflection. I think once I know those two angles I can subtract them from 180 to get my rotation angle then rotate my end point about the angle of intersection by that amount. I just don't know how.
First, you should note that the center of the ball doesn't have to be inside of the circle to indicate that there's a reflection or bounce. As long as the distance between ball center and circle is less than the radius of the ball, there will be a bounce.
If the radius of the circle is R and the radius of the ball is r, things are simplified if you convert to the case where the circle has radius R+r and the ball has radius 0. For the purposes of collision detection and reflection/bouncing, this is equivalent.
If you have the point of intersection between the (enlarged) circle and the ball's path, you can easily compute the normal N to the circle at that point (it is the unit vector in the direction from the center of the circle to the collision point).
For an incoming vector V the reflected vector is V-2(N⋅V) N, where (N⋅V) is the dot product. For this problem, the incoming vector V is the vector from the intersection point to the point inside the circle.
As for the reflection formula given above, it is relatively easy to derive using vector math, but you can also Google search terms like "calculate reflection vector". The signs in the formula will vary with the assumed directions of V and N. Mathworld has a derivation although, as noted, the signs are different.
I only know the solution to the geometry part.
Let:
r1 => Radius of ball
r2 => Radius of circle
You can calculate the distance between the two circles by using Pythagoras theorem.
If the distance is less than the r1+r2 then do the collision.
For the collision,I would refer you Here. It's in python but I think it should give you an idea of what to do. Hopefully, even implement it in Objective C. The Tutorial By PeterCollingRidge.
Let's say I have circle bouncing around inside a rectangular area. At some point this circle will collide with one of the surfaces of the rectangle and reflect back. The usual way I'd do this would be to let the circle overlap that boundary and then reflect the velocity vector. The fact that the circle actually overlaps the boundary isn't usually a problem, nor really noticeable at low velocity. At high velocity it becomes quite clear that the circle is doing something it shouldn't.
What I'd like to do is to programmatically take reflection into account and place the circle at it's proper position before displaying it on the screen. This means that I have to calculate the point where it hits the boundary between it's current position and it's future position -- rather than calculating it's new position and then checking if it has hit the boundary.
This is a little bit more complicated than the usual circle/rectangle collision problem. I have a vague idea of how I should do it -- basically create a bounding rectangle between the current position and the new position, which brings up a slew of problems of it's own (Since the rectangle is rotated according to the direction of the circle's velocity). However, I'm thinking that this is a common problem, and that a common solution already exists.
Is there a common solution to this kind of problem? Perhaps some basic theories which I should look into?
Since you just have a circle and a rectangle, it's actually pretty simple. A circle of radius r bouncing around inside a rectangle of dimensions w, h can be treated the same as a point p at the circle's center, inside a rectangle (w-r), (h-r).
Now position update becomes simple. Given your point at position x, y and a per-frame velocity of dx, dy, the updated position is x+dx, y+dy - except when you cross a boundary. If, say, you end up with x+dx > W (letting W = w-r), then you do the following:
crossover = (x+dx) - W // this is how far "past" the edge your ball went
x = W - crossover // so you bring it back the same amount on the correct side
dx = -dx // and flip the velocity to the opposite direction
And similarly for y. You'll have to set up a similar (reflected) check for the opposite boundaries in each dimension.
At each step, you can calculate the projected/expected position of the circle for the next frame.
If this lies outside the rectangle, then you can then use the distance from the old circle position to the rectangle's edge and the amount "past" the rectangle's edge that the next position lies at (the interpenetration) to linearly interpolate and determine the precise time when the circle "hits" the rectangle edge.
For example, if the circle is 10 pixels away from the rectangle's edge, then is predicted to move to 5 pixels beyond it, you know that for 2/3rds of the timestep (10/15ths) it moves on its orginal path, then is reflected and continues on its new path for the remaining 1/3rd of the timestep (5/15ths). By calculating these two parts of the motion and "adding" the translations together, you can find the correct new position.
(Of course, it gets more complicated if you hit near a corner, as there may be several collisions during the timestep, off different edges. And if you have more than one circle moving, things get a lot more complex. But that's where you can start for the case you've asked about)
Reflection across a rectangular boundary is incredibly simple. Just take the amount that the object passed the boundary and subtract it from the boundary position. If the position without reflecting would be (-0.8,-0.2) for example and the upper left corner is at (0,0), the reflected position would be (0.8,0.2).