Given that a user is static in a VR environment, which of the two camera types below would be better to create a more 'real' looking representation of an live-streamed presenter in the VR world?
1) Kinect (can measure depth)
2) Normal 2D camera such as a high end webcam (maybe something like the pointgrey Flea3) (software assisted 3D illusion from a static angle)
Would be grateful if anyone with any experience with the relevant technologies or fields would be able to help out!
Your question lacks the necessary information to provide a single correct answer. Is it your intent to provide a full 3D VR experience, or are you content with just 2D content? Is the presenter static, or are they moving around the viewer? Towards them? Away from them? Will you be using full spherical projection or something less complete, like cylindrical projection? And what sort of lighting do you think you'll need? These are all nontrivial questions, because the answers determine the best camera package to get your content.
You also fail to consider capturing with a 360º camera, which would be advantageous if the presenter is indeed moving around in the 360º space. My personal bias is towards capturing with these, but there's no single production solution unless you constrain the problem more thoroughly.
Related
I'm in the process of buying a 7.5 acre plot of land in a wooded, hilly area. I would estimate that the elevation varies about 50 feet from the bottom of the creek to the top of the hill. I would like to find a good method for measuring the topography of the land so I can create a 3D model. It would be tremendously useful to be able to try out different land development ideas and to simulate locations for future buildings.
My low-tech version of doing this would be to set up a laser level and go around taking elevation measurements in a 3' or so grid pattern. As I thought about that, I realized that smartphones and similar devices have quite a few sensors built in that might make this a lot easier.
I learned about software that will use a drone to capture data and images to automatically generate a topo map and 3D model. Drone Deploy is one such tool. I do have a DJI Phantom 4, but I don't know if it's feasible to fly such an intricate path among trees to scan the entire property. I wonder if there's another way to use this amazing modern hardware (phone or drone) to make my task easy.
I would appreciate hearing any thoughts and ideas about this!
The thing with dronedeploy is that you fly above the trees usually 30meters is ok. In a cross pattern.
Why do you want to fly between the trees? You have to explain that first.
This is entirely a theoretical question because I understand the time it would take to do such a thing would be ridiculous
I've been working with "voxels" a lot lately and the only way I can display them to a user is to either triangulate the visible surfaces or make a CPU ray-tracer but both come with their own problems.
Simply put, if we dismiss the storage space needed for voxel meshs and targeted a very specific GPU would someone who was wanting to create a graphics API like OpenGL but with "true" voxel primitives that don't need to be converted be able to make such thing or are GPUs designed specifically for triangles with no way to introduce a new base primitive?
Its possible and it was already done many times
games like Minecraft,SpaceEngineers...
3D printing tools and slicers
MRI/PET scans tools
Yes rendering on GPU is possible with the two base methods you mention. Games usually use the transform to boundary representation 3D geometry. With rise of shaders even ray tracers are now possible here mine:
simple GLSL voxel ray tracer
using native OpenGL architecture and passing geometry as 3D texture. In order to obtain speed you need to add BVH or similar spatial subdivision of geometry...
However voxel based tools have been here for quite some time. For example many isometric games/engines are voxel based (tile is a voxel) like this one:
Improving performance of click detection on a staggered column isometric grid
Also do you remember UFO ? It was playable on x286 and it was also "voxel/tile" based isometric.
Background
I'm working on a project where a user gets scanned by a Kinect (v2). The result will be a generated 3D model which is suitable for use in games.
The scanning aspect is going quite well, and I've generated some good user models.
Example:
Note: This is just an early test model. It still needs to be cleaned up, and the stance needs to change to properly read skeletal data.
Problem
The problem I'm currently facing is that I'm unsure how to place skeletal data inside the generated 3D model. I can't seem to find a program that will let me insert the skeleton in the 3D model programmatically. I'd like to do this either via a program that I can control programmatically, or adjust the 3D model file in such a way that skeletal data gets included within the file.
What have I tried
I've been looking around for similar questions on Google and StackOverflow, but they usually refer to either motion capture or skeletal animation. I know Maya has the option to insert skeletons in 3D models, but as far as I could find that is always done by hand. Maybe there is a more technical term for the problem I'm trying to solve, but I don't know it.
I do have a train of thought on how to achieve the skeleton insertion. I imagine it to go like this:
Scan the user and generate a 3D model with Kinect;
1.2. Clean user model, getting rid of any deformations or unnecessary information. Close holes that are left in the clean up process.
Scan user skeletal data using the Kinect.
2.2. Extract the skeleton data.
2.3. Get joint locations and store as xyz-coordinates for 3D space. Store bone length and directions.
Read 3D skeleton data in a program that can create skeletons.
Save the new model with inserted skeleton.
Question
Can anyone recommend (I know, this is perhaps "opinion based") a program to read the skeletal data and insert it in to a 3D model? Is it possible to utilize Maya for this purpose?
Thanks in advance.
Note: I opted to post the question here and not on Graphics Design Stack Exchange (or other Stack Exchange sites) because I feel it's more coding related, and perhaps more useful for people who will search here in the future. Apologies if it's posted on the wrong site.
A tricky part of your question is what you mean by "inserting the skeleton". Typically bone data is very separate from your geometry, and stored in different places in your scene graph (with the bone data being hierarchical in nature).
There are file formats you can export to where you might establish some association between your geometry and skeleton, but that's very format-specific as to how you associate the two together (ex: FBX vs. Collada).
Probably the closest thing to "inserting" or, more appropriately, "attaching" a skeleton to a mesh is skinning. There you compute weight assignments, basically determining how much each bone influences a given vertex in your mesh.
This is a tough part to get right (both programmatically and artistically), and depending on your quality needs, is often a semi-automatic solution at best for the highest quality needs (commercial games, films, etc.) with artists laboring over tweaking the resulting weight assignments and/or skeleton.
There are algorithms that get pretty sophisticated in determining these weight assignments ranging from simple heuristics like just assigning weights based on nearest line distance (very crude, and will often fall apart near tricky areas like the pelvis or shoulder) or ones that actually consider the mesh as a solid volume (using voxels or tetrahedral representations) to try to assign weights. Example: http://blog.wolfire.com/2009/11/volumetric-heat-diffusion-skinning/
However, you might be able to get decent results using an algorithm like delta mush which allows you to get a bit sloppy with weight assignments but still get reasonably smooth deformations.
Now if you want to do this externally, pretty much any 3D animation software will do, including free ones like Blender. However, skinning and character animation in general is something that tends to take quite a bit of artistic skill and a lot of patience, so it's worth noting that it's not quite as easy as it might seem to make characters leap and dance and crouch and run and still look good even when you have a skeleton in advance. That weight association from skeleton to geometry is the toughest part. It's often the result of many hours of artists laboring over the deformations to get them to look right in a wide range of poses.
I have a project where I have to recognize an entire room so I can calculate the distances between objects (like big ones eg. bed, table, etc.) and a person in that room. It is possible something like that using Microsoft Kinect?
Thank you!
Kinect provides you following
Depth Stream
Color Stream
Skeleton information
Its up to you how you use this data.
To answer your question - Official Micorosft Kinect SDK doesnt provides shape detection out of the box. But it does provide you skeleton data/face tracking with which you can detect distance of user from kinect.
Also with mapping color stream to depth stream you can detect how far a particular pixel is from kinect. In your implementation if you have unique characteristics of different objects like color,shape and size you can probably detect them and also detect the distance.
OpenCV is one of the library that i use for computer vision etc.
Again its up to you how you use this data.
Kinect camera provides depth and consequently 3D information (point cloud) about matte objects in the range 0.5-10 meters. With this information it is possible to segment out the floor (by fitting a plane) of the room and possibly walls and the ceiling. This step is important since these surfaces often connect separate objects making them a one big object.
The remaining parts of point cloud can be segmented by depth if they don't touch each other physically. Using color one can separate the objects even further. Note that we implicitly define an object as 3D dense and color consistent entity while other definitions are also possible.
As soon as you have your objects segmented you can measure the distances between your segments, analyse their shape, recognize artifacts or humans, etc. To the best of my knowledge however a Skeleton library can recognize humans after they moved for a few seconds. Below is a simple depth map that was broken on a few segments using depth but not color information.
I'm pretty much grabbing at straws here cause I have no idea what I'm asking, but here is the question.
I've been looking at 3D modeling out of pure interest and came across the concept of bones.
Now, I am not too sure what bones are even after looking it up on wiki, but they seem like an abstraction of real-life skeletons and whatnot, so in a model of say a human I just think of them as the skeleton.
To my understanding, a bone is defined by a translation, rotation, and a scale on the x, y and z axis'. (Isn't that just a single point?)
I am interested in taking a model in blender or max and export the information (whatever they may be) that is used to define these bones. I can definitely see the bones in these programs, but I want to get that out into a text file Is there a way to export this?
I think you need to seperate these ideas:
Bones - which as you correcly say have a position and rotation. They are the objects that you can control and will effect the skin of the model. They are usually in a hierarchy so that if you move one bone then it will affect all of the bones connected to it, like a human skeleton.
Skin - this is the polygonal mesh that you can usually see. It is given a base position by you in the editor and the skeleton operates on the skin to move it around.
Animation - This is data to pass to the bones. Usually a rotation, for example to make an arm bend.
http://gpwiki.org/index.php/OpenGL:Tutorials:Basic_Bones_System gives a good explanation.
Hope that helps :3