How can I read individual pixels from a CVPixelBuffer - objective-c

Question

AVDepthData gives me a CVPixelBuffer of depth data. But I can't find a way to easily access the depth information in this CVPixelBuffer. Is there a simple recipe in Objective-C to do so?

Answer 1

You have to use the CVPixelBuffer APIs to get the right format to access the data via unsafe pointer manipulations. Here is the basic way:
CVPixelBufferRef pixelBuffer = _lastDepthData.depthDataMap;
CVPixelBufferLockBaseAddress(pixelBuffer, 0);
size_t cols = CVPixelBufferGetWidth(pixelBuffer);
size_t rows = CVPixelBufferGetHeight(pixelBuffer);
Float32 *baseAddress = CVPixelBufferGetBaseAddress( pixelBuffer );
// This next step is not necessary, but I include it here for illustration,
// you can get the type of pixel format, and it is associated with a kCVPixelFormatType
// this can tell you what type of data it is e.g. in this case Float32
OSType type = CVPixelBufferGetPixelFormatType( pixelBuffer);
if (type != kCVPixelFormatType_DepthFloat32) {
NSLog(#"Wrong type");
}
// Arbitrary values of x and y to sample
int x = 20; // must be lower that cols
int y = 30; // must be lower than rows
// Get the pixel. You could iterate here of course to get multiple pixels!
int baseAddressIndex = y * (int)cols + x;
const Float32 pixel = baseAddress[baseAddressIndex];
CVPixelBufferUnlockBaseAddress( pixelBuffer, 0 );
Note that the first thing you need to determine is what type of data is in the CVPixelBuffer - if you don't know this then you can use CVPixelBufferGetPixelFormatType() to find out. In this case I am getting depth data at Float32, if you were using another type e.g. Float16, then you would need to replace all occurrences of Float32 with that type.
Note that it's important to lock and unlock the base address using CVPixelBufferLockBaseAddress and CVPixelBufferUnlockBaseAddress.