apply bind pose to a kinect skeleton - kinect

Question

I want to normalize a skeleton in order to make it invariant to the size of the person
in front of the kinect; in the same way as the aveteering example.
But I don't want to animate a 3D model using XNA, the only thing I need is to normalize an
skeleton.
So in order to do this task, I have divided it in two functions:
(a) apply a bind pose to an skeleton in order to see how to work this matrix. Obviously this is not what i want to do, but it is a first step in order to
know how to work whit matrix, and so on.
(b) apply any arbitrary pose to a normalized-size-skeleton
First of all, I want to apply a bind pose to an skeleton (a).
First, I have to load the matrix that describe the bone length/ offset between bones and store it in
List BindPose.
Due to I have no idea how to do it, I modified the Aveteering example and write in a file all the Matrix that define
the BindPose, InverseBindPose and SkeletonHierarchy of the dude. I only need BindPose to this first task, but I have the
code prepared in order to do the second task (b)
The file looks like this:
1,331581E-06;-5,551115E-17;1;0;1;-4,16881E-11;-1,331581E-06;0;4,16881E-11;1;8,153579E-23;0;0,03756338;37,46099;2,230549;1
1,110223E-16;-4,435054E-22;1;0;1;1,426127E-06;-2,220446E-16;0;-1,426127E-06;1;-7,654181E-22;0;-0,9558675;-4,079016E-08;-6,266987E-12;1
0,9954988;-0,09477358;1,501821E-06;0;0,09477358;0,9954988;-4,019565E-06;0;-1,114112E-06;4,143805E-06;1;0;3,786007;-0,003599779;5,107028E-06;1
0,9948416;-0,101441;-3,23556E-07;0;0,101441;0,9948416;-2,266755E-08;0;3,241862E-07;-1,027114E-08;1;0;4,543321;-0,00359975;-1,33061E-07;1
0,9950595;0,09927933;2,388133E-07;0;-0,09927933;0,9950595;-2,333792E-08;0;-2,399506E-07;-4,86646E-10;1;0;4,544049;-0,003599948;6,324596E-08;1
0,9992647;0,02747673;0,02674458;0;-0,02928042;0,9971476;0,06956656;0;-0,02475683;-0,07029849;0,9972187;0;4,543965;-0,004398902;2,258555E-07;1
0,9154034;0,4025377;1,107153E-06;0;-0,4025377;0,9154033;-2,437432E-07;0;-1,109319E-06;-2,115673E-07;1;0;5,536249;-0,00288291;1,332601E-07;1
0,9812952;-0,1925096;-4,732622E-07;0;0,1925095;0,9812951;-3,00921E-08;0;4,697166E-07;-5,889972E-08;1;0;3,953898;1,702301E-07;4,88653E-08;1
.......
So each line is a 4X4 matrix defining the BindPose.
To generate this file, the code is like this:
private void ViewSkinningData(SkinningData data)
{
string nameFile = "bind_pose_transformations";
bool append = false;
// The using statement automatically closes the stream and calls IDisposable.Dispose on the stream object.
using (System.IO.StreamWriter file = new System.IO.StreamWriter(#nameFile, append))
{
for (int i = 0; i < data.BindPose.Count; i++)
{
Matrix m = data.BindPose[i];
string matrixString = MatrixToString(m);
file.WriteLine(matrixString);
}
for (int i = 0; i < data.InverseBindPose.Count; i++)
{
Matrix m = data.InverseBindPose[i];
string matrixString = MatrixToString(m);
file.WriteLine(matrixString);
}
for (int i = 0; i < data.SkeletonHierarchy.Count; i++)
{
file.Write(data.SkeletonHierarchy[i] + ";");
}
}
}
string MatrixToString(Matrix m)
{
string result;
result = m.M11 + ";" + m.M12 + ";" + m.M13 + ";" + m.M14 + ";" + m.M21 + ";" + m.M22 + ";" + m.M23 + ";" + m.M24 + ";" + m.M31 + ";" + m.M32 + ";" + m.M33 + ";" + m.M34 + ";" + m.M41 + ";" + m.M42 + ";" + m.M43 + ";" + m.M44;
return result;
}
Next step is to load all this Skinning data in my program:
private void InitializeSkinningDataFromFile()
{
string filename = "bind_pose_transformations";
int number_avatar_joints = 58;
List<Matrix> binpose = new System.Collections.Generic.List<Matrix>();
List<Matrix> inversebindpose = new System.Collections.Generic.List<Matrix>();
List<int> skeletonhierarchy = new System.Collections.Generic.List<int>();
// The using statement automatically closes the stream and calls IDisposable.Dispose on the stream object.
using (System.IO.StreamReader file = new System.IO.StreamReader(filename))
{
string s;
int count = 0;
while (!String.IsNullOrEmpty(s = file.ReadLine()))
{
string[] values = s.Split(';');
Matrix m = BuildMatrix(values);
binpose.Add(m);
count++;
if (count == number_avatar_joints)
{
break;
}
}
count = 0;
while (!String.IsNullOrEmpty(s = file.ReadLine()))
{
string[] values = s.Split(';');
Matrix m = BuildMatrix(values);
inversebindpose.Add(m);
count++;
if (count == number_avatar_joints)
{
break;
}
}
string[] skeletonHierarchy = file.ReadLine().Split(';'); //lee un caracter de separacion al final...
//for (int i = 0; i < skeletonHierarchy.Count(); i++)
for (int i = 0; i < number_avatar_joints; i++)
{
skeletonhierarchy.Add(int.Parse(skeletonHierarchy[i]));
}
}
skinningDataValue = new SkinningData(binpose, inversebindpose, skeletonhierarchy);
}
After, I have to construct boneTransforms structure:
// Bone matrices for the "dude" model
this.boneTransforms = new Matrix[skinningDataValue.BindPose.Count];
this.skinningDataValue.BindPose.CopyTo(this.boneTransforms, 0);
Now boneTransforms have the transformation for my skeleton. So now, i have to apply these trasnformations to an skeleton
Skeleton skeleton = new Skeleton();
foreach (Joint joint in skeleton.Joints)
{
int indexMatrix = AvatarBoneToNuiJointIndex(joint.JointType);
Matrix transform;
if (indexMatrix >= 0)
{
transform = this.boneTransforms[indexMatrix];
}
else
{
transform = Matrix.Identity;
}
Joint aux = ApplyMatrixTransformationToJoint(joint, transform);
normalizeSkel.Joints[joint.JointType] = aux;
}
This is a helper function AvatarBoneToNuiJointIndex:
public int AvatarBoneToNuiJointIndex(JointType jointType)
{
switch (jointType)
{
case JointType.HipCenter:
return 1;
case JointType.Spine:
return 4;
case JointType.ShoulderCenter:
return 6;
case JointType.Head:
return 7;
case JointType.ShoulderLeft:
return 12;
case JointType.ElbowLeft:
return 13;
case JointType.WristLeft:
return 14;
case JointType.HandLeft:
return 15;
case JointType.ShoulderRight:
return 31;
case JointType.ElbowRight:
return 32;
case JointType.WristRight:
return 33;
case JointType.HandRight:
return 34;
case JointType.KneeLeft:
return 50;
case JointType.AnkleLeft:
return 51;
case JointType.FootLeft:
return 52;
case JointType.KneeRight:
return 54;
case JointType.AnkleRight:
return 55;
case JointType.FootRight:
return 56;
default: return -1;
}
}
This is a helper function ApplyMatrixTransformationToJoint:
public Joint ApplyMatrixTransformationToJoint(Joint skeletonJoint, Matrix tranformations)
{
Vector3 pos = SkeletonPointToVector3(skeletonJoint.Position);
Vector3 result = ApplyMatrixTransformationToVector(pos, tranformations);
SkeletonPoint newPosition = new SkeletonPoint()
{
X = result.X,
Y = result.Y,
Z = result.Z
};
skeletonJoint.Position = newPosition;
return skeletonJoint;
}
This is the code for ApplyMatrixTransformationToVector:
static Vector3 ApplyMatrixTransformationToVector(Vector3 v, Matrix m)
{
return Vector3.Transform(v, m);
}
But the problem is that I can't see anything.
I don't know if this approach is correct.
Any help would be fantastic.
Many thanks!