Render pixel buffer with Cocoa - objective-c

Question

I've searched through various Apple's docs and StackOverflow answers, but nothing really helped, still have a blank app's window. I'm trying to display the content of a pixel buffer in the NSWindow, to do that I've allocated a buffer:
UInt8* row = (UInt8 *) malloc(WINDOW_WIDTH * WINDOW_HEIGHT * bytes_per_pixel);
UInt32 pitch = (WINDOW_WIDTH * bytes_per_pixel);
// For each row
for (UInt32 y = 0; y < WINDOW_HEIGHT; ++y) {
Pixel* pixel = (Pixel *) row;
// For each pixel in a row
for (UInt32 x = 0; x < WINDOW_WIDTH; ++x) {
*pixel++ = 0xFF000000;
}
row += pitch;
}
This should prepare a buffer with red pixels. Then I'm creating a NSBitmapImageRep:
NSBitmapImageRep *imageRep = [[NSBitmapImageRep alloc] initWithBitmapDataPlanes:(u8 *) row
pixelsWide:WINDOW_WIDTH
pixelsHigh:WINDOW_HEIGHT
bitsPerSample:8
samplesPerPixel:4
hasAlpha:YES
isPlanar:NO
colorSpaceName:NSDeviceRGBColorSpace
bytesPerRow:WINDOW_WIDTH * 4
bitsPerPixel:32];
Which then converted into NSImage:
NSSize imageSize = NSMakeSize(CGImageGetWidth([imageRep CGImage]), CGImageGetHeight([imageRep CGImage]));
NSImage *image = [[NSImage alloc] initWithSize:imageSize];
[image addRepresentation:imageRep];
Then I'm configuring the view:
NSView *view = [window contentView];
[view setWantsLayer: YES];
[[view layer] setContents: image];
Sadly this doesn't give me the result I expect.

Answer 1

Here are some problems with your code:
You are incrementing row by pitch at the end of each y-loop. You never saved the pointer to the beginning of the buffer. When you create your NSBitmapImageRep, you pass a pointer that is past the end of the buffer.
You are passing row as the first (planes) argument of initWithBitmapDataPlanes:..., but you need to pass &row. The documentation says
An array of character pointers, each of which points to a buffer containing raw image data.[…]
An “array of character pointers” means (in C) you pass a pointer to a pointer.
You say “This should prepare a buffer with red pixels.” But you filled the buffer with 0xFF000000, and you said hasAlpha:YES. Depending on the byte order used by the initializer, either you have set the alpha channel to 0, or you have set the alpha channel to 0xFF but set all of the color channels to 0.
As it happens, you have set each pixel to opaque black (alpha = 0xFF, colors all zero). Try setting each pixel to 0xFF00007F and you'll get a dimmed red (alpha = 0xFF, red = 0x7F).
Thus:
typedef struct {
uint8_t red;
uint8_t green;
uint8_t blue;
uint8_t alpha;
} Pixel;
#implementation AppDelegate
- (void)applicationDidFinishLaunching:(NSNotification *)aNotification {
size_t width = self.window.contentView.bounds.size.width;
size_t height = self.window.contentView.bounds.size.height;
Pixel color = { .red=127, .green=0, .blue=0, .alpha=255 };
size_t pitch = width * sizeof(Pixel);
uint8_t *buffer = malloc(pitch * height);
for (size_t y = 0; y < height; ++y) {
Pixel *row = (Pixel *)(buffer + y * pitch);
for (size_t x = 0; x < width; ++x) {
row[x] = color;
}
}
NSBitmapImageRep *rep = [[NSBitmapImageRep alloc] initWithBitmapDataPlanes:&buffer
pixelsWide:width pixelsHigh:height
bitsPerSample:8 samplesPerPixel:4 hasAlpha:YES isPlanar:NO
colorSpaceName:NSDeviceRGBColorSpace
bytesPerRow:pitch bitsPerPixel:sizeof(Pixel) * 8];
NSImage *image = [[NSImage alloc] initWithSize:NSMakeSize(width, height)];
[image addRepresentation:rep];
self.window.contentView.wantsLayer = YES;
self.window.contentView.layer.contents = image;
}
#end
Result:
Note that I didn't free buffer. If you free buffer before rep is destroyed, things will go wrong. For example, if you just add free(buffer) to the end of applicationDidFinishLaunching:, the window appears gray.
This is a thorny problem to solve. If you use Core Graphics instead, the memory management is all handled properly. You can ask Core Graphics to allocate the buffer for you (by passing NULL instead of a valid pointer), and it will free the buffer when appropriate.
You have to release the Core Graphics objects you create to avoid memory leaks, but you can do that as soon as you're done with them. The Product > Analyze command can also help you find leaks of Core Graphics objects, but will not help you find leaks of un-freed malloc blocks.
Here's what a Core Graphics solution looks like:
typedef struct {
uint8_t red;
uint8_t green;
uint8_t blue;
uint8_t alpha;
} Pixel;
#implementation AppDelegate
- (void)applicationDidFinishLaunching:(NSNotification *)aNotification {
size_t width = self.window.contentView.bounds.size.width;
size_t height = self.window.contentView.bounds.size.height;
CGColorSpaceRef rgb = CGColorSpaceCreateWithName(kCGColorSpaceLinearSRGB);
CGContextRef gc = CGBitmapContextCreate(NULL, width, height, 8, 0, rgb, kCGImageByteOrder32Big | kCGImageAlphaPremultipliedLast);
CGColorSpaceRelease(rgb);
size_t pitch = CGBitmapContextGetBytesPerRow(gc);
uint8_t *buffer = CGBitmapContextGetData(gc);
Pixel color = { .red=127, .green=0, .blue=0, .alpha=255 };
for (size_t y = 0; y < height; ++y) {
Pixel *row = (Pixel *)(buffer + y * pitch);
for (size_t x = 0; x < width; ++x) {
row[x] = color;
}
}
CGImageRef image = CGBitmapContextCreateImage(gc);
CGContextRelease(gc);
self.window.contentView.wantsLayer = YES;
self.window.contentView.layer.contents = (__bridge id)image;
CGImageRelease(image);
}
#end

Answer 2

Note sure what's going on, but here's code that has been working for years:
static NSImage* NewImageFromRGBA( const UInt8* rawRGBA, NSInteger width, NSInteger height )
{
size_t rawRGBASize = height*width*4/* sizeof(RGBA) = 4 */;
// Create a bitmap representation, allowing NSBitmapImageRep to allocate its own data buffer
NSBitmapImageRep* imageRep = [[NSBitmapImageRep alloc] initWithBitmapDataPlanes:NULL
pixelsWide:width
pixelsHigh:height
bitsPerSample:8
samplesPerPixel:4
hasAlpha:YES
isPlanar:NO
colorSpaceName:NSCalibratedRGBColorSpace
bytesPerRow:0
bitsPerPixel:0];
NSCAssert(imageRep!=nil,#"failed to create NSBitmapImageRep");
NSCAssert((size_t)[imageRep bytesPerPlane]==rawRGBASize,#"alignment or size of CGContext buffer and NSImageRep do not agree");
// Copy the raw bitmap image into the new image representation
memcpy([imageRep bitmapData],rawRGBA,rawRGBASize);
// Create an empty NSImage then add the bitmap representation to it
NSImage* image = [[NSImage alloc] initWithSize:NSMakeSize(width,height)];
[image addRepresentation:imageRep];
return image;
}