I want to do some scientific image processing on iOS in Obj-C or of course C, all I require to do this is to get a 3D array of the bit values of all the pixels' RGBA. UIImage doesn't seem to have a built in function. Do any of you know how to get the pixel values or more preferably a predefined library with those functions in there?
Thanks in advance, JEM
You'd normally either create a CGBitmapContext, telling it to use some memory you'd allocated yourself (so, you know where it is and how to access it) or let Core Graphics figure that out for you and call CGBitmapContextGetData if you're targeting only iOS 4/OS X 10.6 and later, then draw whatever you want to inspect to it.
E.g. (error checking and a few setup steps deliberately omitted for brevity; look for variables I use despite not defining and check the function documentation)
CGBitmapInfo bitmapInfo =
kCGImageAlphaPremultipliedFirst | kCGBitmapByteOrder32Host;
context =
CGBitmapContextCreate(
NULL,
width, height,
8,
width * 4,
rgbColourSpace,
bitmapInfo);
CGContextDrawImage(
context,
CGRectMake(0.0f, 0.0f, (CGFloat)width, (CGFloat)height),
[image CGImage]);
uint8_t *pixelPointer = CGBitmapContextGetData(context);
for(size_t y = 0; y < height; y++)
{
for(size_t x = 0u; x < width; x++)
{
if((bitmapInfo & kCGBitmapByteOrder32Little))
{
NSLog(#"rgba: %02x %02x %02x %02x",
pixelPointer[2], pixelPointer[1],
pixelPointer[0], pixelPointer[3]);
}
else
{
NSLog(#"rgba: %02x %02x %02x %02x",
pixelPointer[1], pixelPointer[2],
pixelPointer[3], pixelPointer[0]);
}
pixelPointer += 4;
}
}
Related
I'm trying to optimize the performance in one of my components. The component needs to draw some (10 to 200) rectangles in it's drawRect method, which is triggered about 20 times per second.
Everything works when I use the CGContextFillRect method on each CGRect separately. I want to test if grouping the drawing into one single call with CGContextFillRects on an array of CGRects would increase performance.
The method CGContextFillRects gives me a compiler error No matching function for call to 'CGContextFillRects'.
This code is inside a .mm file. Should I import something before the CGContextFillRects method can be used?
This is what i'm trying to do:
- (void) drawRect:(CGRect)rect{
CGContextRef context = UIGraphicsGetCurrentContext();
CGContextSaveGState(context);
CGContextSetFillColorWithColor(context, self.fillColor.CGColor);
//check if some objects are present
if (self.leftDrawBuffer && self.rightDrawBuffer){
UInt32 xPosForRect = self.leftPadding;
NSMutableArray *rectsToFill = [[NSMutableArray alloc] init];
for (int drawBufferLRIndex = 0; drawBufferLRIndex < 2; drawBufferLRIndex++){
Float32 *drawBuffer_ptr = self.leftDrawBuffer;
if (drawBufferLRIndex > 0){
drawBuffer_ptr = self.rightDrawBuffer;
}
for (int i=0; i< kAmountOfBarsPerChannel; i=i+1){
Float32 amp = drawBuffer_ptr[i];
Float32 blockNumber = 1.0f;
UInt32 yPosForRect = self.bounds.size.height - self.heightPerBlock;
while (blockNumber <= self.blocksPerLine && blockNumber / self.blocksPerLine < amp){
CGRect rect= CGRectMake(xPosForRect, yPosForRect, self.widthPerBlock, self.heightPerBlock);
[rectsToFill addObject:[NSValue valueWithCGRect:rect]];
//Using the method below works and gives me the expected result
//CGContextFillRect(context, rect);
blockNumber++;
yPosForRect -= self.heightPerBlock + self.vPaddingPerBlock;
}
xPosForRect += self.widthPerBlock + self.hPaddingPerBlock;
}
}
//This is the added code where i try to use CGContextFillRects
//1 -> transform to a c array of CGRects
const CGRect *cRects[rectsToFill.count];
for (int i = 0; i < rectsToFill.count; ++i) {
CGRect rect = [[rectsToFill objectAtIndex:i] CGRectValue];
cRects[i] = ▭
}
size_t size = rectsToFill.count;
//2 -> trigger the method to fill all rects at once
//this method gives me the compiler error 'No matching function for call to 'CGContextFillRects''
CGContextFillRects(context, cRects, size);
}
CGContextRestoreGState(context);
}
The problem is how you convert the rects to a C array. You make pointers to the rects that are temporarily stored on the stack. There are two problems with this. First, the rects are gone with each loop iteration, so you can't do that. Second, You should pass a pointer to an array of CGRects, not an array of pointers to CGRect.
This will likely solve it:
CGRect cRects[rectsToFill.count]; // Replace your lines from this
for (int i = 0; i < rectsToFill.count; ++i) {
CGRect rect = [[rectsToFill objectAtIndex:i] CGRectValue];
cRects[i] = rect;
}
size_t size = rectsToFill.count;
CGContextFillRects(context, cRects, size); // To this
Please note the re-declaration of the cRects array and the change in the assignment.
This is best I've come up with for blitting a 24-bit BGR image out to an NSView.
I did trim a significant amount of CPU time by ensuring that the NSWindow host also had the same colorSpace.
I think there are 4 or 5 pixel copies going on here:
in the vImage conversion (required)
calling CGDataProviderCreateWithData
calling CGImageCreate
creating the NSBitmapImageRep bitmap
in the final blit with drawInRect (required)
Anyone want to chime in on improving it?
Any help would be much appreciated.
{
// one-time setup code
CGColorSpaceRef useColorSpace = nil;
int w = 1920;
int h = 1080;
[theWindow setColorSpace: [NSColorSpace genericRGBColorSpace]];
// setup vImage buffers (not listed here)
// srcBuffer is my 24-bit BGR image (malloc-ed to be w*h*3)
// dstBuffer is for the resulting 32-bit RGBA image (malloc-ed to be w*h*4)
...
// this is called # 30-60fps
if (!useColorSpace)
useColorSpace = CGColorSpaceCreateWithName(kCGColorSpaceGenericRGB);
vImage_Error err = vImageConvert_BGR888toRGBA8888(srcBuffer, NULL, 0xff, dstBuffer, NO, 0);
CGDataProviderRef newProvider = CGDataProviderCreateWithData(NULL,dstBuffer->data,w*h*4,myReleaseProvider);
CGImageRef myImageRGBA = CGImageCreate(w, h, 8, 32, w*4, useColorSpace, kCGBitmapByteOrderDefault | kCGImageAlphaLast, newProvider, NULL, false, kCGRenderingIntentDefault);
CGDataProviderRelease(newProvider);
// store myImageRGBA in an array of frames (using NSObject wrappers) for later access (setNeedsDisplay:)
...
}
- (void)drawRect:(NSRect)dirtyRect
{
// this is called # 30-60fps
CGImageRef storedImage = ...; // retrieve from array
NSBitmapImageRep *repImg = [[NSBitmapImageRep alloc] initWithCGImage:storedImage];
CGRect myFrame = CGRectMake(0,0,CGImageGetWidth(storedImage),CGImageGetHeight(storedImage));
[repImg drawInRect:myFrame fromRect:myFrame operation:NSCompositeCopy fraction:1.0 respectFlipped:TRUE hints:nil];
// free image from array (not listed here)
}
// this is called when the CGDataProvider is ready to release its data
void myReleaseProvider (void *info, const void *data, size_t size)
{
if (data) {
free((void *)data);
data=nil;
}
}
Use CGColorSpaceCreateDeviceRGB instead of genericRGB to avoid colorspace conversion inside CG. Use kCGImageAlphaNoneSkipLast instead of kCGImageAlphaLast since we know alpha is opaque to allow for a copy instead of a blend.
After you make those changes, it would be useful to run an Instruments time profile on it to show where the time is going.
I tried to extract all 3 channels from an image with vImageConvert_RGB888toPlanar8 and then put them back together with vImageConvert_Planar8toRGB888 but the image gets totally messed up. Why is that?
vImage_Buffer blueBuffer;
blueBuffer.data = (void*)blueImageData.bytes;
blueBuffer.width = size.width;
blueBuffer.height = size.height;
blueBuffer.rowBytes = [blueImageData length]/size.height;
vImage_Buffer rBuffer;
rBuffer.width = size.width;
rBuffer.height = size.height;
rBuffer.rowBytes = size.width;
void *rPixelBuffer = malloc(size.width * size.height);
if(rPixelBuffer == NULL)
{
NSLog(#"No pixelbuffer");
}
rBuffer.data = rPixelBuffer;
vImage_Buffer gBuffer;
gBuffer.width = size.width;
gBuffer.height = size.height;
gBuffer.rowBytes = size.width;
void *gPixelBuffer = malloc(size.width * size.height);
if(gPixelBuffer == NULL)
{
NSLog(#"No pixelbuffer");
}
gBuffer.data = gPixelBuffer;
vImage_Buffer bBuffer;
bBuffer.width = size.width;
bBuffer.height = size.height;
bBuffer.rowBytes = size.width;
void *bPixelBuffer = malloc(size.width * size.height);
if(bPixelBuffer == NULL)
{
NSLog(#"No pixelbuffer");
}
bBuffer.data = bPixelBuffer;
vImageConvert_RGB888toPlanar8(&blueBuffer, &rBuffer, &gBuffer, &bBuffer, kvImageNoFlags);
size_t destinationImageBytesLength = size.width*size.height*3;
const void* destinationImageBytes = valloc(destinationImageBytesLength);
NSData* destinationImageData = [[NSData alloc] initWithBytes:destinationImageBytes length:destinationImageBytesLength];
vImage_Buffer destinationBuffer;
destinationBuffer.data = (void*)destinationImageData.bytes;
destinationBuffer.width = size.width;
destinationBuffer.height = size.height;
destinationBuffer.rowBytes = [destinationImageData length]/size.height;
vImage_Error result = vImageConvert_Planar8toRGB888(&rBuffer, &gBuffer, &bBuffer, &destinationBuffer, 0);
NSImage* image = nil;
if(result == kvImageNoError)
{
//TODO: If you need color matching, use an appropriate colorspace here
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
CGDataProviderRef dataProvider = CGDataProviderCreateWithCFData((__bridge CFDataRef)(destinationImageData));
CGImageRef finalImageRef = CGImageCreate(size.width, size.height, 8, 24, destinationBuffer.rowBytes, colorSpace, kCGBitmapByteOrder32Big|kCGImageAlphaNone, dataProvider, NULL, NO, kCGRenderingIntentDefault);
CGColorSpaceRelease(colorSpace);
CGDataProviderRelease(dataProvider);
image = [[NSImage alloc] initWithCGImage:finalImageRef size:NSMakeSize(size.width, size.height)];
CGImageRelease(finalImageRef);
}
free((void*)destinationImageBytes);
return image;
Working with vImage means to work with pixels only. So you must never use the size of an image (or imageRep), you only use pixelsWide and pixelsHigh. Replace all size.width with pixelsWide and all size.height with pixelsHigh. Apple has example code for vImage and they use size values! Don't believe them! Not all Apple example codes are correct.
The size of an image or imageRep determines how big an image shall be drawn on the screen (or a printer). Size values have the dimension of a length and the units are meter, cm, inch or (as in Cocoa) 1/72 inch aka point. They are represented as float values.
PixelsWide and pixelsHigh have no dimension and no unit (they are simply numbers) and are represented as int values.
There may be more bugs in your code, but the first step should be to replace all size values.
Strictly speaking, you want kCGBitmapByteOrderDefault instead of kCGBitmapByteOrder32Big. 32Big doesn't make much sense for a 24 bit pixel format.
This seems like a weak link:
destinationBuffer.rowBytes = [destinationImageData length]/size.height;
Check to see it is the right number.
A picture of the output would help diagnose the problem. Also check the console to see if CG is giving you any spew. Did you try vImageCreateCGImageFromBuffer() with kvImagePrintDiagnosticsToConsole to see if it has anything to say?
Working with vImage means to work with pixels only. So you must never use the size of an image (or imageRep), you only use pixelsWide and pixelsHigh. Replace all size.width with pixelsWide and all size.height with pixelsHigh. Apple has example code for vImage and they use size values! Don't believe them! Not all Apple example codes are correct.
The size of an image or imageRep determines how big an image shall be drawn on the screen (or a printer). Size values have the dimension of a length and the units are meter, cm, inch or (as in Cocoa) 1/72 inch aka point. They are represented as float values.
PixelsWide and pixelsHigh have no dimension and no unit (they are simply numbers) and are represented as int values.
There may be more bugs in your code, but the first step should be to replace all size values.
I want to read a PNG file such that I can:
a) Access the raw bitmap data of the file, with no color space adjustment or alpha premultiply.
b) Based on that bitmap, display bit slices (any single bit of R, G, B, or A, across the whole image) in an image in the window. If I have the bitmap I can find the right bits, but what can I stuff them into to get them onscreen?
c) After some modification of the bitplanes, write a new PNG file, again with no adjustments.
This is only for certain specific images. The PNG is not expected to have any data other than simply RGBA-32.
From reading some similar questions here, I'm suspecting NSBitmapImageRep for the file read/write, and drawing in an NSView for the onscreen part. Does this sound right?
1.) You can use NSBitmapImageRep's -bitmapData to get the raw pixel data. Unfortunately, CG (NSBitmapImageRep's backend) does not support native unpremultiplication so you would have to unpremultiply yourself. The colorspace used in this will be the same as present in the file. Here is how to unpremultiply the image data:
NSBitmapImageRep *imageRep = [NSBitmapImageRep imageRepWithData:data];
NSInteger width = [imageRep pixelsWide];
NSInteger height = [imageRep pixelsHigh];
unsigned char *bytes = [imageRep bitmapData];
for (NSUInteger y = 0; y < width * height * 4; y += 4) { // bgra little endian + alpha first
uint8_t a, r, g, b;
if (imageRep.bitmapFormat & NSAlphaFirstBitmapFormat) {
a = bytes[y];
r = bytes[y+1];
g = bytes[y+2];
b = bytes[y+3];
} else {
r = bytes[y+0];
g = bytes[y+1];
b = bytes[y+2];
a = bytes[y+3];
}
// unpremultiply alpha if there is any
if (a > 0) {
if (!(imageRep.bitmapFormat & NSAlphaNonpremultipliedBitmapFormat)) {
float factor = 255.0f/a;
b *= factor;
g *= factor;
r *= factor;
}
} else {
b = 0;
g = 0;
r = 0;
}
bytes[y]=a; // for argb
bytes[y+1]=r;
bytes[y+2]=g;
bytes[y+3]=b;
}
2.) I couldn't think of a simple way to do this. You could make your own image drawing method that loops through the raw image data and generates a new image based on the values. Refer above to see how to start doing it.
3.) Here is a method to get a CGImage from raw data places (you can write the png to a file using native CG functions or convert it to NSBitmapImageRep if CG makes you uncomfortable)
static CGImageRef cgImageFrom(NSData *data, uint16_t width, uint16_t height) {
CGDataProviderRef provider = CGDataProviderCreateWithCFData((CFDataRef)data);
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
CGBitmapInfo bitmapInfo = kCGImageAlphaFirst;
CGImageRef cgImage = CGImageCreate(width, height, 8, 32, 4 * width, colorSpace, bitmapInfo, provider, NULL, NO, kCGRenderingIntentDefault);
CGDataProviderRelease(provider);
CGColorSpaceRelease(colorSpace);
return cgImage;
}
You can create the NSData object out of the raw data object with +dataWithBytes:length:
I haven't ever worked in this area, but you may be able to use Image IO for this.
I have an array that contains the RGB colour values for each pixel in a 320 x 180 display. I would like to be able to set individual pixel values in the a bitmap context of the same size offscreen then display the bitmap context in a view.
It appears that I have to create 1x1 rects and either put a stroke on them or a line of length 1 at the point in question. Is that correct? I'm looking for a very efficient way of getting the array data onto the graphics context as you can imagine this is going to be an image buffer that cycles at 25 frames per second and drawing in this way seems inefficient.
I guess the other question is should I use OPENGL ES instead?
Thoughts/best practice would be much appreciated.
Regards
Dave
OK, have come a short way, but can't make the final hurdle and I am not sure why this isn't working:
- (void) displayContentsOfArray1UsingBitmap: (CGContextRef)context
{
long bitmapData[WIDTH * HEIGHT];
// Build bitmap
int i, j, h;
for (i = 0; i < WIDTH; i++)
{
for (j = 0; j < HEIGHT; j++)
{
h = frameBuffer01[i][j];
bitmapData[i * j] = h;
}
}
// Blit the bitmap to the context
CGDataProviderRef providerRef = CGDataProviderCreateWithData(NULL, bitmapData,4 * WIDTH * HEIGHT, NULL);
CGColorSpaceRef colorSpaceRef = CGColorSpaceCreateDeviceRGB();
CGImageRef imageRef = CGImageCreate(WIDTH, HEIGHT, 8, 32, WIDTH * 4, colorSpaceRef, kCGImageAlphaFirst, providerRef, NULL, YES, kCGRenderingIntentDefault);
CGContextDrawImage(context, CGRectMake(0.0, HEIGHT, WIDTH, HEIGHT), imageRef);
CGImageRelease(imageRef);
CGColorSpaceRelease(colorSpaceRef);
CGDataProviderRelease(providerRef);
}
Read the documentation for CGImageCreate(). Basically, you have to create a CGDataProvider from your pixel array (using CGDataProviderCreateDirect()), then create a CGImage with this data provider as a source. You can then draw the image into any context. It's a bit tedious to get this right because these functions expect a lot of arguments, but the documentation is quite good.
Dave,
The blitting code works fine, but your code to copy from the frame buffer is incorrect.
// Build bitmap
int i, j, h;
for (i = 0; i < WIDTH; i++)
{
for (j = 0; j < HEIGHT; j++)
{
h = frameBuffer01[i][j];
bitmapData[/*step across a line*/i + /*step down a line*/j*WIDTH] = h;
}
}
Note my changes to the assignment to elements of bitmapData.
Not knowing the layout of frame, this may still be incorrect, but from your code, this looks closer to the intent.