How to reconstruct MediaPipe landmark coordinates from extracted coordinate values? - mediapipe

I am able to extract the landmark coordinates from results.pose_landmarks. Now, I want to send it to another device and there, using the x, y, z and visibility values, the datatype will be reconstructed so as to build the pose on a frame.

Store the iterated values in a list. Then use this snippet below,
from mediapipe.framework.formats import landmark_pb2
reconstructed = landmark_pb2.NormalizedLandmarkList(extracted_points_list)
This will reconstruct the entire datatype.

Related

Interpolate 3D surface input into 4th dimension

My source data consists of a set of (x,y,z,e) samples. It can be visualized like this, where the dots are the (x,y,z) samples in 3D space and the color reflects the e value. The (x,y,z) samples compose a surface.
I want a kind of interpolation method, that I can feed with random (x,y,z) coordinates that are close to the surface and that can output an interpolated e value.
I tried the scipy LinearNDInterpolator. It works fine, but only for input (x,y,z) points that lie inside the convex hull of the surface. When the input is only slightly outside, the interpolator returns 'nan'.
I'm a bit out of ideas how to solve this.
I can only think of iterating the each line in the grid to find the points closest to the random (x,y,z) input and do linear interpolations from these points. But if I could somehow reconstruct the surface, that would be more accurate.

How to fill a line in 2D image along a given radius with the data in a given line image?

I want to fill a 2D image along its polar radius, the data are stored in a image where each row or column corresponds to the radius in target image. How can I fill the target image efficiently? Such as with iradius or some functions? I do not prefer a pix-pix operation.
Are you looking for something like this?
number maxR = 100
image rValues := realimage("I(r)",4,maxR)
rValues = 10 + trunc(100*random())
image plot :=realimage("Ring",4,2*maxR,2*maxR)
rValues.ShowImage()
plot.ShowImage()
plot = rValues.warp(iradius,0)
You might also want to check out the relevant example code from the F1 help documentation of GMS itself:
Explaining warp a bit:
plot = rValues.warp(iradius,0)
Assigns values to plot based on a value-lookup in rValues.
For each pixel in plot a coordinate position in rValues is computed, and the value is simply looked up. If the computed coordinate is non-integer, bilinear interpolation between the 4 closest points is used.
In the example, the two 'formulas' for the coordinate calculation are simple x' = iradius and y' = 0 where iradius is an expression computed from the coordinate in plot, for convenience.
You can feed any expression into the parameters for warp( ) and the command is closely related to just using the square bracket notation of addressing values. In fact, the only difference is that warp performs the bilinear interpolation of values instead of truncating the coordinates to integer values.

Using matplotlib to plot a matrix with the third variable as source for a color map

Say you have the matrix given by three arrays, being:
x = N-dimensional array.
y = M-dimensional array.
And z is a set of "somewhat random" values from -0.3 to 0.3 in a NxM shape. I need to create a plot in which the x values are in the x-axis, y values are in the y-axis and using z as the source to indicate the intensity of each pixel with a color map.
So far, I have tried using
plt.contourf(x,y,z)
and the resulting plot is very nice for me (attached at the end of this paragraph), but a smoothing is automatically applied to the plot! I need to be able to distinguish the pixels and I cannot find a way to do it.
contourf result
I have also studied the possibility of using
ax.matshow(z)
in order to sucesfully see the pixels... but then I am struggling trying to personalize the x and y axis, since only the index of the pixel is shown (see below).
matshow result
Would you please give me some ideas? Thank you.
Without more information on your x,y data it's hard to know, but I would guess you are looking for pcolormesh.
plt.pcolormesh(x,y,z)
This would take the x and y data as input and hence shows the z data at the appropriate coordinates.
You can use imshow with the keyword interpolation='nearest'.
plt.imshow(z, interpolation='nearest')

How do I modify mesh attributes to send custom information in Blender?

I have a mesh in 3DS format. I imported this mesh to blender and now, I want to export this mesh back to 3DS but, I want to associate a number (say id) with each vertex of this mesh. Now, I only need the x, y and z coordinates of this newly exported 3DS, and I don't really care about the normals or the texture coordinates.
So the way of keeping the IDs intact could be to insert that number in an un-required attribute, let's say the x coordinate of each vertex normal or the first texture coordinate of each vertex.
Here's what I tried with normals:
import bpy
import bmesh
object_reference = bpy.context.active_object
bm = bmesh.new()
bm.from_mesh(object_reference.data)
for vert in bm.verts:
vert.normal[0] = vert.index
bm.to_mesh(object_reference.data)
But, the normals reverted back to default on export. So, how do I do this?
I couldn't figure out a way to set the texture coordinates, how can I do so? If I can't, then how can I make the vertex normal hack work? Is there a less-hacky way of doing this?

Put pcolormesh and contour onto same grid?

I'm trying to display 2D data with axis labels using both contour and pcolormesh. As has been noted on the matplotlib user list, these functions obey different conventions: pcolormesh expects the x and y values to specify the corners of the individual pixels, while contour expects the centers of the pixels.
What is the best way to make these behave consistently?
One option I've considered is to make a "centers-to-edges" function, assuming evenly spaced data:
def centers_to_edges(arr):
dx = arr[1]-arr[0]
newarr = np.linspace(arr.min()-dx/2,arr.max()+dx/2,arr.size+1)
return newarr
Another option is to use imshow with the extent keyword set.
The first approach doesn't play nicely with 2D axes (e.g., as created by meshgrid or indices) and the second discards the axis numbers entirely
Your data is a regular mesh? If it doesn't, you can use griddata() to obtain it. I think that if your data is too big, a sub-sampling or regularization always is possible. If the data is too big, maybe your output image always will be small compared with it and you can exploit this.
If you use imshow() with "extent" and "interpolation='nearest'", you will see that the data is cell-centered, and extent provided the lower edges of cells (corners). On the other hand, contour assumes that the data is cell-centered, and X,Y must be the center of cells. So, you need to be care about the input domain for contour. The trivial example is:
x = np.arange(-10,10,1)
X,Y = np.meshgrid(x,x)
P = X**2+Y**2
imshow(P,extent=[-10,10,-10,10],interpolation='nearest',origin='lower')
contour(X+0.5,Y+0.5,P,20,colors='k')
My tests told me that pcolormesh() is a very slow routine, and I always try to avoid it. griddata and imshow() always is a good choose for me.