I use UIImagePickerController as camera. Now I want to integrate with ogre 3d render to make an ar application. Does someone know how to get the custom projection matrix of UIImagePickerController.
I convert cameracontroller.cameraviewtransform to a matrix. and the view matrix I just convert 2dar affinetransform to transform3d.
but it doesn't work.
Worse comes to worst, you can generate the projection transform from what you know about the camera, which is primarily just the FOV angle. Things like the orientation and location of the camera in an AR application are simply offsets from whatever your defaults are, and this data can be retrieved from the gyro and accelerometer.
Related
I'm working on an application for Android & iOS to show points of interest over the camera. ARkit & ARcore has poor compatibility nowdays.
Could you recommend me some library to do this? If it comes with an example, better! I know viro-media, but I don't understand how to do this using that library.
I don't want 3D models, just markers over the camera, similar to the attachment image.
To do this with Viro React -- and in AR in general -- the trick is to recognize that there are two coordinate systems:
The local coordinate system of your device, which we'll call 'AR space'. In Viro, this is centered at the user's initial position when the application starts, and is in meters.
Geographic coordinates (latitude and longitude).
To position the overlays, you have to convert your content from geographic coordinates into AR space. This is a two-step process. First project the spherical geographic coordinates onto a 2D plane -- the Web Mercator is great for this. Then translate the projected coordinates by the device's initial projected position.
The device's initial projected position can be derived by projecting its initial geographic position. In Viro React, you can use the Geolocation module to grab this when the user starts the app.
Finally, you'll need to do a similar transformation for the user's bearing: converting from compass direction to device orientation in AR space.
For this to work well you'll likely have to figure out how handle inaccurate geolocation lookups (e.g. what happens if the location retrieved from the device is inaccurate), and may also have to account for drift: over time the two coordinate systems may start to fall out of sync.
The last part, creating the info cards, is easy with Viro -- you either pre-bake the images with text and use ViroImage, or if the cards need to be more dynamic you can use a ViroFlexView.
I am also interested in this one and I'm trying out ViroReact!
I find a bit difficult to understand how to make this work when the lat's and long's have been converted to x-y-values. What should the z-value be?
Let's say you have the lat-lon coordinates [59, 10] as the user location you want to show where [59, 11] is relative to your location. How to you build that in a ViroARScene?
<ViroNode position={ **userLocationFromLatLonCartesian** }>
<ViroBox position={ **poiLocationFromLatLonToCartesian** }/>
</ViroNode>
So how do you calculate the scale, position and rotation, so that the object will be visible?
Seem like https://github.com/proj4js/proj4js is a library that could provide conversions from latlon to x-y values
I found that both android and ios AR sdk support location base AR View refererence:
https://developers.google.com/ar/develop/ios/geospatial/quickstart and https://developer.apple.com/documentation/arkit/argeoanchor
I've been searching around the web about how to do this and I know that it needs to be done with OpenCV. The problem is that all the tutorials and examples that I find are for separated shapes detection or template matching.
What I need is a way to detect the contents between 3 circles (which can be a photo or something else). From what I searched, its not to difficult to find the circles with the camera using contours but, how do I extract what is between them? The circles work like a pattern on the image to grab what is "inside the pattern".
Do I need to use the contours of each circle and measure the distance between them to grab my contents? If so, what if the image is a bit rotated/distorted on the camera?
I'm using Xamarin.iOS for this but from what I already saw, I believe I need to go native for this and any Objective C example is welcome too.
EDIT
Imagining that the image captured by the camera is this:
What I want is to match the 3 circles and get the following part of the image as result:
Since the images come from the camera, they can be rotated or scaled up/down.
The warpAffine function will let you map the desired area of the source image to a destination image, performing cropping, rotation and scaling in a single go.
Talking about rotation and scaling seem to indicate that you want to extract a rectangle of a given aspect ratio, hence perform a similarity transform. To define such a transform, three points are too much, two suffice. The construction of the affine matrix is a little tricky.
Here's the task:
We have an Mesh, drawn in position POS with rotation ROT
Also we have a camera Which position and rotation is relative to Mesh For example camera point is CPOS and camera rotation is CROT.
How to calculate resulting angle for camera? I was assuming that it something like:
camera.rotation.x = mesh.rotation.x + viewport.rotation.x
camera.rotation.y = mesh.rotation.y + viewport.rotation.y
camera.rotation.z = mesh.rotation.z + viewport.rotation.z
That worked strange and wrong.
Then I decided to read about it on docs and completely dissapointed.
There are several kind of rotation structures (Euler, Quaternion). But What a want is something different.
Imagine, like you are on spaceship. And it moves in space. You are sitting at starboard turret and looking at objects. They seems like passing by...
Then you want to turn your head - Angel of your head is known to you (in raw opengl, I'd just multiplied head rotation matrix on ship's rotation matrix and got my projection matrix).
In other words I want only x and y axis for camera rotations, combined in matrix. Then I want to multiply it with position-rotation matrix of an object. And this final matrix would be my projection matrix.
How could I do the same in THREE.js?
-----EDIT-----
Thank you for the answer.
Which coords should I give to a camera? It should be local, mesh relative coords, or something absolute?
I understand, that this questions are obvious, but there's no any description about relative objects in THREE.JS docs (besides api description). And the answer might be ambiguous.
Add the camera as a child of the mesh like so:
mesh.add( camera );
When the camera is a child of an object, the camera's position and orientation are specified relative to the parent object.
You can set the camera's orientation by setting either the camera's quaternion or Euler rotation -- your choice.
Please note that the renderer updates the object's matrix and matrixWorld for you. You do not need to do that manually.
three.js r.63
I'm having difficulties finding any documentation about cropping images using OpenGL ES on the iPhone or iPad.
Specifically, I am capturing video frames at a mildly rapid pace (20 FPS), and need something quick that will crop an image. Is it feasible to use OpenGL here? If so, will it perform faster than cropping using Core Image and its associated methods?
It seems that using Core Image methods, I can't achieve faster than about 10-12 FPS output, and I'm looking for a way to hit 20. Any suggestions or pointers to usage of OpenGL for this?
Obviously, using OpenGl ES will be faster than Core Image Framework. Cropping image will be done by set Texture Coordinate, in generally, Texture Coordinate always like this,
{
0.0f,1.0f,
1.0f,1.0f,
0.0f,0.0f,
1.0f.0.0f
}
The whole image will be drawed with Texture Coordinate above. If you just want upper right part of a image, you can set Texture Coordinate like this,
{
0.5f,1.0f,
1.0f,1.0f,
0.5f,0.5f,
1.0f.0.5f
}
This will get a quater of the whole image at upper right. You never forget that the Coordinate origin of OpenGl ES is at the lower left corner
I am trying to develop an iOS app to make any given image (UIImage) warp on selected locations.
So for this task to be accomplished what should be the rightmost way going forward, for now i'm doing some research on doing this on OpenGL (frankly any heads up on the framework would be nice too).
So finally the requirement is to get the UIImage warp on some given locations. (If x, y coordinates are there)
If you're sufficiently familiar with (or willing to learn) OpenGL, then you could do this:
Create a flat, rectangular grid of points to be a mesh that will be displayed with OpenGL.
Apply the image to the mesh as a texture.
When distorting the image at a particular location, you can just decide which points on the mesh will be affected by the distortion, and move them.
You can push points out from the center, or in toward a center, or shift them all in the same direction. If the distortion affects a large area, then you change a lot of points (possibly changing those in the center by more than those near the edges of the affected area).
Not sure what you mean by 'warp'. Do you mean skew it in 3 dimensions? If so you can adjust the CGAffineTransform for the UIImageView you are displaying it in to get that effect.
If you mean some kind of image processing warp, and you are using iOS 5, you can use Core Image for that.