How to acquire skeletal joint data to text file from two kinect cameras in SkeletalViewer? (C++) - c++-cli

I am currently working on a project to use multiple kinect cameras to acquire x,y,z coordinates of skeletal data in SkeletalViewer. I have an idea to use the KinectID or index of different kinect cameras to extract the sets of skeletal joints data into 2 different text files. But I am not sure if I am doing right. Please help to take a look at the modification below, or I will appreciate all your kind advice on other method to solve this problem.
In SkeletalViewer.h, I modified as following:
public:
FILE* mp3DFile0;
FILE* mp3DFile1;
char mText[1024];
INuiSensor * m_pNuiSensor;
BSTR m_instanceId;
array SensorIndex;
In NuiImpl.cpp, I modified as following:
1) define array
ref class MyClass {
public:
int m_i;
};
array<MyClass^>^ arrSensor() {
int i;
array< MyClass^ >^ local = gcnew array< MyClass^ >(2);
for (i = 0; i < 2; i++) {
local[i] = gcnew MyClass;
local[i]->m_i = i;
}
return local;
}
2) create array to store sensor index to do for loop later
HRESULT CSkeletalViewerApp::Nui_Init( )
{
HRESULT hr;
bool result;
//create an array to store two file pointers
FILE* mp3DFile[] = { mp3DFile0, mp3DFile1 };
fopen_s(mp3DFile0, "D:/Kinect/KinectCam0.txt", "w+");
fopen_s(mp3DFile1, "D:/Kinect/KinectCam1.txt", "w+");
.
.
if ( !m_pNuiSensor )
{
hr = NuiCreateSensorByIndex(0, &m_pNuiSensor);
//I am not sure about how to utilize this index in this case
.
.
}
if (NuiGetSensorCount(&m_pNuiSensor) > 1)
{
array< MyClass^ >^ SensorIndex;
SensorIndex = arrSensor();
}
}
3) use for loop to store data to different text file using index
void CSkeletalViewerApp::Nui_DrawSkeleton( const NUI_SKELETON_DATA & skel, int windowWidth, int windowHeight )
{
int i;
int h;
for (h = 0; h < 2; h++)
{
//when index point to the current kinect
if (SensorIndex[h] == &m_pNuiSensor)
{
for (i = 0; i < NUI_SKELETON_POSITION_COUNT; i++)
{
m_Points[i] = SkeletonToScreen(skel.SkeletonPositions[i], windowWidth, windowHeight);
memset(mText, 0, 1024);
sprintf(mText, "(%0.3f,%0.3f,%0.3f)", skel.SkeletonPositions[i].x, skel.SkeletonPositions[i].y, skel.SkeletonPositions[i].z);
if (mp3DFile[h]) {
fputs((const char*)mText, mp3DFile[h]);
}
}
if (mp3DFile[h]) {
fputs("\n", mp3DFile[h]);
}
}
}
.
.
}
I am a newbie in this Kinect programming. Thank you very much for your help! :)

Related

how can i get all process name in os x programmatically? not just app processes

I want to get a snapshot of the process info in the os x system.
The 'NSProcessInfo' can only get info of the calling process.
The ps cmd can be one solution, but i'd like a c or objective-c program.
Here's an example using using libproc.h to iterate over all the processes on the system and determine how many of them belong to the effective user of the process. You can easily modify this for your needs.
- (NSUInteger)maxSystemProcs
{
int32_t maxproc;
size_t len = sizeof(maxproc);
sysctlbyname("kern.maxproc", &maxproc, &len, NULL, 0);
return (NSUInteger)maxproc;
}
- (NSUInteger)runningUserProcs
{
NSUInteger maxSystemProcs = self.maxSystemProcs;
pid_t * const pids = calloc(maxSystemProcs, sizeof(pid_t));
NSAssert(pids, #"Memory allocation failure.");
const int pidcount = proc_listallpids(pids, (int)(maxSystemProcs * sizeof(pid_t)));
NSUInteger userPids = 0;
uid_t uid = geteuid();
for (int *pidp = pids; *pidp; pidp++) {
struct proc_bsdshortinfo bsdshortinfo;
int writtenSize;
writtenSize = proc_pidinfo(*pidp, PROC_PIDT_SHORTBSDINFO, 0, &bsdshortinfo, sizeof(bsdshortinfo));
if (writtenSize != (int)sizeof(bsdshortinfo)) {
continue;
}
if (bsdshortinfo.pbsi_uid == uid) {
userPids++;
}
}
free(pids);
return (NSUInteger)userPids;
}

Implementing qsort in Objective-C

I'm trying to use qsort to sort a C array in descending order based on what this website is suggesting.
Here is the relevant code:
int x = 3;
- (IBAction)CaptureButton:(id)sender
{
x++;
if (x % 3 == 1)
{
int areas[detectedBlobs.size()];
for (int i = 0; i < detectedBlobs.size(); i++)
{
areas[i] = detectedBlobs[i].getWidth() * detectedBlobs[i].getHeight();
}
int compareInts(void const *item1, void const *item2)
{ // first error
int const *int1 = item1;
int const *int2 = item2;
return (*int2 - *int1);
}
qsort(areas, detectedBlobs.size(), sizeof(int), compareInts); // second error
}
}
Here are the two errors I'm getting:
First error:
Function definition is not allowed here
Second error:
Use of undeclared identifier 'compareInts'
If I cannot define the comparator (compareInts) function here, where do I have to define it? Also, how can I get the qsort function to recognize the comparator?
Objective-C does not allow function definitions inside methods. Move compareInts outside of the method, and make it static to hide from other translation units:
static int compareInts(const void* item1, const void* item2) {
const int* int1 = (const int*)item1;
const int* int2 = (const int*)item2;
return (*int2 - *int1);
}
- (IBAction)CaptureButton:(id)sender {
x++;
if (x % 3 == 1) {
int areas[detectedBlobs.size()];
for (int i = 0; i < detectedBlobs.size(); i++) {
areas[i] = detectedBlobs[i].getWidth() * detectedBlobs[i].getHeight();
}
qsort(areas, detectedBlobs.size(), sizeof(int), compareInts);
}
}

OpenKinect acquire raw depth image

I am trying to use the example code from here.
I have made some changes in order to save the images to the computer. When I read the data in MATLAB it seems like values that should be 0 are set to 2047, and overall it does not seem to be correct when I reconstruct the 3D points using the default intrinsic camera parameters.
What I want to achieve is to save the images so that I can use
img = single(imread(depth.png'))/ 1000
and have the depth values in meters, and pixels with no measurements should be zero.
It is the Kinect V1 by the way.
Here is the code with comments where I have tried to change.
#include "libfreenect.hpp"
#include <iostream>
#include <vector>
#include <cmath>
#include <pthread.h>
#include <cv.h>
#include <cxcore.h>
#include <highgui.h>
using namespace cv;
using namespace std;
class myMutex {
public:
myMutex() {
pthread_mutex_init( &m_mutex, NULL );
}
void lock() {
pthread_mutex_lock( &m_mutex );
}
void unlock() {
pthread_mutex_unlock( &m_mutex );
}
private:
pthread_mutex_t m_mutex;
};
// Should one use FREENECT_DEPTH_REGISTERED instead of FREENECT_DEPTH_11BIT?
class MyFreenectDevice : public Freenect::FreenectDevice {
public:
MyFreenectDevice(freenect_context *_ctx, int _index)
: Freenect::FreenectDevice(_ctx, _index), m_buffer_depth(FREENECT_DEPTH_11BIT),
m_buffer_rgb(FREENECT_VIDEO_RGB), m_gamma(2048), m_new_rgb_frame(false),
m_new_depth_frame(false), depthMat(Size(640,480),CV_16UC1),
rgbMat(Size(640,480), CV_8UC3, Scalar(0)),
ownMat(Size(640,480),CV_8UC3,Scalar(0)) {
for( unsigned int i = 0 ; i < 2048 ; i++) {
float v = i/2048.0;
v = std::pow(v, 3)* 6;
m_gamma[i] = v*6*256;
}
}
// Do not call directly even in child
void VideoCallback(void* _rgb, uint32_t timestamp) {
std::cout << "RGB callback" << std::endl;
m_rgb_mutex.lock();
uint8_t* rgb = static_cast<uint8_t*>(_rgb);
rgbMat.data = rgb;
m_new_rgb_frame = true;
m_rgb_mutex.unlock();
};
// Do not call directly even in child
void DepthCallback(void* _depth, uint32_t timestamp) {
std::cout << "Depth callback" << std::endl;
m_depth_mutex.lock();
uint16_t* depth = static_cast<uint16_t*>(_depth);
// Here I use memcpy instead so I can use uint16
// memcpy(depthMat.data,depth,depthMat.rows*depthMat.cols*sizeof(uint16_t));
depthMat.data = (uchar*) depth;
m_new_depth_frame = true;
m_depth_mutex.unlock();
}
bool getVideo(Mat& output) {
m_rgb_mutex.lock();
if(m_new_rgb_frame) {
cv::cvtColor(rgbMat, output, CV_RGB2BGR);
m_new_rgb_frame = false;
m_rgb_mutex.unlock();
return true;
} else {
m_rgb_mutex.unlock();
return false;
}
}
bool getDepth(Mat& output) {
m_depth_mutex.lock();
if(m_new_depth_frame) {
depthMat.copyTo(output);
m_new_depth_frame = false;
m_depth_mutex.unlock();
return true;
} else {
m_depth_mutex.unlock();
return false;
}
}
private:
// Should it be uint16_t instead or even higher?
std::vector<uint8_t> m_buffer_depth;
std::vector<uint8_t> m_buffer_rgb;
std::vector<uint16_t> m_gamma;
Mat depthMat;
Mat rgbMat;
Mat ownMat;
myMutex m_rgb_mutex;
myMutex m_depth_mutex;
bool m_new_rgb_frame;
bool m_new_depth_frame;
};
int main(int argc, char **argv) {
bool die(false);
string filename("snapshot");
string suffix(".png");
int i_snap(0),iter(0);
Mat depthMat(Size(640,480),CV_16UC1);
Mat depthf (Size(640,480),CV_8UC1);
Mat rgbMat(Size(640,480),CV_8UC3,Scalar(0));
Mat ownMat(Size(640,480),CV_8UC3,Scalar(0));
// The next two lines must be changed as Freenect::Freenect
// isn't a template but the method createDevice:
// Freenect::Freenect<MyFreenectDevice> freenect;
// MyFreenectDevice& device = freenect.createDevice(0);
// by these two lines:
Freenect::Freenect freenect;
MyFreenectDevice& device = freenect.createDevice<MyFreenectDevice>(0);
namedWindow("rgb",CV_WINDOW_AUTOSIZE);
namedWindow("depth",CV_WINDOW_AUTOSIZE);
device.startVideo();
device.startDepth();
while (!die) {
device.getVideo(rgbMat);
device.getDepth(depthMat);
// Here I save the depth images
std::ostringstream file;
file << filename << i_snap << suffix;
cv::imwrite(file.str(),depthMat);
cv::imshow("rgb", rgbMat);
depthMat.convertTo(depthf, CV_8UC1, 255.0/2048.0);
cv::imshow("depth",depthf);
if(iter >= 1000) break;
iter++;
}
device.stopVideo();
device.stopDepth();
return 0;
}
Thanks in advance!
Erik
I dont have any experience with OpenKinect in particular; but should your depth buffer be uint16?
std::vector<uint8_t> m_buffer_depth;
Also; for Matlab, do check if the image that you are reading is a uint16 or uint8. If its the latter then convert it to uint16
uint16(imread('depth.png'));
Sorry couldn't help more. Hope this helps.
The values you have are the raw depth values. You need to remap those into MM for the numbers to make sense. Kinect 1 can see up to 10 meters. So I would go with raw_values/2407*10000.
If the values are saturated at 2047, you are probably using the FREENECT_DEPTH_11BIT_PACKED depth format.
For work in Matlab, it is always easier to use FREENECT_DEPTH_MM or FREENECT_DEPTH_REGISTERED.
Enjoy.

Returning an int array in C++/CLI to c# .NET

I'm using a C++/CLI wrapper to call a c++ library from c# .NET. While this particular code "works," I suspect that I'm doing something wrong with respect to memory. (I run into problems after running this code about 20 times in a row.)
c# side:
public void ExportModelToImage(int[] myImage, int imageWidth, int imageHeight)
{
View.ExportModelToImage(ref myImage, imageWidth, imageHeight);
}
C++/CLI side:
void ExportModelToImage(array<int>^% myImage, int imageWidth, int imageHeight)
{
if (myView().IsNull())
{
return;
}
myView()->Redraw();
Image_PixMap theImage;
myView()->ToPixMap(theImage, imageWidth, imageHeight);
const int totalBytes = imageWidth * imageHeight;
int byteIndex = 0;
Standard_Integer si = 0;
Quantity_Color aColor;
Quantity_Parameter aDummy;
for (Standard_Size aRow = 0; aRow < theImage.SizeY(); ++aRow)
{
for (Standard_Size aCol = 0; aCol < theImage.SizeX(); ++aCol)
{
aColor = theImage.PixelColor((Standard_Integer )aCol, (Standard_Integer )aRow, aDummy);
aColor.Color2argb(aColor, si);
myImage[byteIndex] = (int) si;
byteIndex++;
if (byteIndex > totalBytes) return;
}
}
}
Ideally, I would prefer if ExportModelToImage() returned an int array instead of returning by reference, but I've had problems figuring out the correct way to do that in C++/CLI. Any suggestions would be greatly appreciated. Thanks!
To return an int array, have array<int>^ as your return type, and initialize your local variable with gcnew. Don't forget to leave off the ^ when you call gcnew.
array<int>^ ExportModelToImage(int imageWidth, int imageHeight)
{
array<int>^ result = gcnew array<int>(imageWidth * imageHeight);
if (myView().IsNull())
{
return nullptr;
// could also return a zero-length array, or the current
// result (which would be an all-black image).
}
myView()->Redraw();
Image_PixMap theImage;
myView()->ToPixMap(theImage, imageWidth, imageHeight);
int byteIndex = 0;
Standard_Integer si = 0;
Quantity_Color aColor;
Quantity_Parameter aDummy;
for (Standard_Size aRow = 0; aRow < theImage.SizeY(); ++aRow)
{
for (Standard_Size aCol = 0; aCol < theImage.SizeX(); ++aCol)
{
aColor = theImage.PixelColor((Standard_Integer )aCol, (Standard_Integer )aRow, aDummy);
aColor.Color2argb(aColor, si);
result[byteIndex] = (int) si;
byteIndex++;
}
}
return result;
}
Now, that said, there are other possibilities you could do here. In particular, you may want to construct a .Net image type of some sort and return that, rather than returning an array of integers.

How do i copy to a List?

I have this code in CLI
List<Codec^> ^GetCodecs()
{
List<Codec^> ^l = gcnew List<Codec^>;
bool KeepLooping = Encoder_MoveToFirstCodec();
while (KeepLooping)
{
Codec ^codec = gcnew Codec(); // here... and that call encoder_init many times... which call register codec many times... which is a mass...
codec->Name = gcnew String(Encoder_GetCurrentCodecName());
codec->Type = Encoder_GetCurrentCodecType();
char pix_fmts[200]; // array of 200 is probably enough
int actual_pix_fmts_sz = Encoder_GetCurrentCodecPixFmts( pix_fmts , 200 );
for (int i = 0 ; i < actual_pix_fmts_sz ; i++)
{
//copy from pix_fmts to the :List
codec->SupportedPixelFormats->Add(pix_fmts[i]);
}
This is the Encoder_GetCurrentCodecPixFmts function in C:
int Encoder_GetCurrentCodecPixFmts( char *outbuf , int buf_sz )
{
int i=0;
while ( (i<buf_sz) && (codec->pix_fmts[i]!=-1) )
{
outbuf[i] = codec->pix_fmts[i];
i++;
}
return i;
}
This is a new class i did:
#pragma once
using namespace System;
using namespace System::Collections::Generic;
public ref class Codec
{
public:
String^ Name;
int ID; // this is the index
int Type; // this is the type
List<int> ^SupportedPixelFormats;
Codec(void)
{
SupportedPixelFormats = gcnew List<int>;
// do nothing in the constructor;
}
};
Which contain also the: SupportedPixelFormats
The constructor in this new class should be empty but i needed somewhere to make an instance for the List make a NEW for the List.
Now in the C++ i need to transfer from pix_fmts char array to codec->Supported
Or to copy from pix_fmts to the :List
So i did as above:
codec->SupportedPixelFormats->Add(pix_fmts[i]);
But i'm not sure if this the meaning of copy.
Is that right what i did ?
It works, it's a kind of a deep copy. What makes you think it doesn't work? Do the results turn out wrong? If they do, put a breakpoint in there and try to get what is wrong.
Instead of copying one by one perhaps you can use the Enumerable::ToList extension method.
I hope this helped you.