I've noticed in Apple's sample code that they often provide a value of 0 in the bytesPerRow parameter of CGBitmapContextCreate. For example, this comes out of the Reflection sample project.
CGContextRef gradientBitmapContext = CGBitmapContextCreate(NULL, pixelsWide, pixelsHigh,
8, 0, colorSpace, kCGImageAlphaNone);
That seemed odd to me, since I've always gone the route of multiplying the image width by the number of bytes per pixel. I tried swapping in a zero into my own code and tested it out. Sure enough, it still works.
size_t bitsPerComponent = 8;
size_t bytesPerPixel = 4;
size_t bytesPerRow = reflectionWidth * bytesPerPixel;
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
CGContextRef context = CGBitmapContextCreate(NULL,
reflectionWidth,
reflectionHeight,
bitsPerComponent,
0, // bytesPerRow ??
colorSpace,
kCGImageAlphaPremultipliedLast);
According to the docs, bytesPerRow should be "The number of bytes of memory to use per row of the bitmap."
So whats the deal? When can I supply a zero and when must I calculate the exact value? Are there any performance implications of doing it one way or the other?
My understanding is that if you pass in zero, it calculates the bytes-per-row based on the bitsPerComponent and width arguments. You might want additional padding at the end of each row of bytes (if your device required it, or some other constraint). In this case, you could pass a value that was more than just width * (bytes per pixel). I would imagine this is probably never needed in modern i/MacOS development, except for some weird edge-case optimizations.
Related
I am trying to scale an image using vImage_Buffer and the below code works for me. My trouble is I want to maintain the aspect ratio of the source image, so I might need to add a xOffset or yOffset. Below code only works for yOffset. How can I scale the image with xOffset as well. I can not do scaling with CGContext since that affect the performance.
CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
size_t finalWidth = 1080;
size_t finalHeight = 720;
size_t sourceWidth = CVPixelBufferGetWidth(imageBuffer);
size_t sourceHeight = CVPixelBufferGetHeight(imageBuffer);
CGRect aspectRect = AVMakeRectWithAspectRatioInsideRect(CGSizeMake(sourceWidth, sourceHeight), CGRectMake(0, 0, finalWidth, finalHeight));
size_t bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer);
size_t startY = aspectRect.origin.y;
size_t yOffSet = (finalWidth*startY*4);
CVPixelBufferLockBaseAddress(imageBuffer, 0);
void *baseAddress = CVPixelBufferGetBaseAddress(imageBuffer);
void* destData = malloc(finalHeight * finalWidth * 4);
vImage_Buffer srcBuffer = { (void *)baseAddress, sourceHeight, sourceWidth, bytesPerRow};
vImage_Buffer destBuffer = { (void *)destData+yOffSet, aspectRect.size.height, aspectRect.size.width, aspectRect.size.width * 4};
vImage_Error err = vImageScale_ARGB8888(&srcBuffer, &destBuffer, NULL, 0);
No pun intended, but you should really read Accelerate.framework documentation.
Replace malloc with calloc ...
void *destData = calloc(finalHeight * finalWidth * 4);
... to zero all the bytes (or use any other way).
What does vImage_Buffer.rowBytes documentation say?
The distance, in bytes, between the start of one pixel row and the next in an image, including any unused space between them.
The rowBytes value must be at least the width multiplied by the pixel size, where the pixel size depends on the image format. You can provide a larger value, in which case the extra bytes will extend beyond the end of each row of pixels. You may want to do so either to improve performance, or to describe an image within a larger image without copying the data. The extra bytes aren't considered part of the image represented by the vImage buffer.
When allocating floating-point data for images, keep the data 4-byte aligned by allocating bytes as integer multiples of 4. For best performance, allocate bytes as integer multiples of 16.
Look at the following image:
Red circle (top/left corner) is offset from the buffer start, let's calculate it (assuming 4 bytes per pixel):
size_t startY = aspectRect.origin.y;
size_t startX = aspectRect.origin.x;
size_t offset = 4 * (finalWidth * startY + startX);
The distance, in bytes, between the start of one pixel row and the next in an image, including any unused space between them is finalWidth * 4 (red line between two other circles).
Let's fix the destBuffer:
vImage_Buffer destBuffer = {
(void *)destData+offset,
aspectRect.size.height,
aspectRect.size.width,
finalWidth * 4
};
I am converting an Objective-C class that uses calloc() to create a buffer to Swift 3. Here is the pertinent part of the code I'm having issue with. In particular, the rawData assignment and usage.
CGImageRef imageRef = [capturedImage CGImage];
NSUInteger width = CGImageGetWidth(imageRef);
NSUInteger height = CGImageGetHeight(imageRef);
unsigned char *rawData = (unsigned char *)calloc(height * width * 4, sizeof(unsigned char));
NSUInteger bytesPerPixel = 4;
NSUInteger bytesPerRow = bytesPerPixel * width;
NSUInteger bitsPerComponent = 8;
CGContextRef context = CGBitmapContextCreate(rawData, width, height,
bitsPerComponent, bytesPerRow, colorSpace,
kCGImageAlphaPremultipliedLast | kCGBitmapByteOrder32Big);
NSUInteger byteIndex = (bytesPerRow * y) + x * bytesPerPixel;
CGFloat red = (CGFloat)rawData[byteIndex];
CGFloat green = (CGFloat)rawData[byteIndex + 1];
CGFloat blue = (CGFloat)rawData[byteIndex + 2];
From the CGBitmapContextCreate documentation:
data : UnsafeMutableRawPointer?
A pointer to the destination in memory where the drawing is to be rendered. The size of this memory block should be at least (bytesPerRow*height) bytes.
Pass NULL if you want this function to allocate memory for the bitmap. This frees you from managing your own memory, which reduces memory leak issues.
While you could figure out how to allocate the required memory block and obtain an UnsafeMutableRawPointer to it unless you have a good reason to allocate your own buffer just follow the documentation and pass NULL - no need to use calloc() at all.
HTH
I'm currently working on a project that envolves working with PNGs that have custom RGBA values. This is a preview of the code I use to create a PNG NSData that contains my custom RGBA values (all of this is working as it should):
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
CGContextRef gtx = CGBitmapContextCreate(&pixelData, width, height, bitsPerComponent, BytesPerRow, colorSpace, (CGBitmapInfo)kCGImageAlphaPremultipliedLast);
CGImageRef toCGImage = CGBitmapContextCreateImage(gtx);
NSBitmapImageRep *newRep = [[NSBitmapImageRep alloc] initWithCGImage:toCGImage];
NSData *pngData = [newRep representationUsingType:NSPNGFileType properties:nil];
When I'm creating the pixelData (which is an uint8_t array that contains all the RGBA values) and I set the Alpha index of each pixel as 255, the int values of each R/G/B of each pixel are the same when I was creating them.
Here's an RGBA example (Alpha = 255) -> (72, 101, 114, 255)
Now, if I set the Alpha of all pixels to be, let's say, 100, instead of 255, the above RGBA example will look like: (184, 255, 255, 100).
As you can see, the RGB values are totally different from what I created initially, and I really need to preserve the original values (through a custom property when creating the NSData or something like that) or a way to calculate them back, no matter what the Alpha value is. Is there any way of doing this?
Thanks!
Your data has un-premultiplied (by alpha) color components. Core Graphics is expecting premultiplied alpha.
Depending on how your build your pixelData array, you can either pre-multiply it in your own code as you go (e.g. premultipliedRed = unpremultipliedRed * alpha / 255.0) or you can use the Accelerate framework to convert it after the fact. You would use the function vImagePremultiplyData_RGBA8888() to do that:
vImage_Buffer buffer = { pixelData, width, height, BytesPerRow };
vImagePremultiplyData_RGBA8888(&buffer, &buffer, 0);
If, later, you get back premultiplied data and you need to convert back to un-premultiplied, you can do the reverse using vImageUnpremultiplyData_RGBA8888(). There may be some loss of precision in the round trip, so you're not absolutely guaranteed to get back the original source data bit-for-bit. It's an inevitable consequence of converting between unpremultiplied and premultiplied. If that's a problem, you need to keep the original source data or not use Core Graphics.
I'm confused about one strange thing....I have an unsigned char array.... I allocate it using calloc and record some bytes data in it... but when I free this unsigned char and allocate it again, I see that it reserves the same address in memory which was allocated previous time. I understand why....But I cannot understand why the data that I'm trying to write there second time is not written...There is written the data that was written first time....Can anybody explain me this???????
unsigned char *rawData = (unsigned char*) calloc(height * width * 4, sizeof(unsigned char));
This is how I allocate it....
Actually my problem is that because of this allocation , which happens once every 2 secs I have memory leak...But when I try to free the allocated memory sector happens thing described above....:(
Please if anybody can help me....I would be so glad...
Here is the code...
- (unsigned char*) createBitmapContext:(UIImage*)anImage{
CGImageRef imageRef = [anImage CGImage];
NSUInteger width = CGImageGetWidth(imageRef);
NSUInteger height = CGImageGetHeight(imageRef);
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
unsigned char *rawData = (unsigned char*) calloc(height * width * 4, sizeof(unsigned char));
bytesPerPixel = 4;
bytesPerRow = bytesPerPixel * width;
NSUInteger bitsPerComponent = 8;
CGContextRef context = CGBitmapContextCreate(rawData, width, height,
bitsPerComponent, bytesPerRow, colorSpace,
kCGImageAlphaPremultipliedLast | kCGBitmapByteOrder32Big);
CGColorSpaceRelease(colorSpace);
CGContextDrawImage(context, CGRectMake(0, 0, width, height), imageRef);
CGContextRelease(context);
imageRef=nil;
return rawData; }
in this code there is no the part where I free(rawData), and because I cannot free it inside this method I tried to define rawData globally and free it after calling this method...but nothing interesting....
Please if anybody can help me....I would be so glad...
Ok, so this method is rendering a UIImage into a freshly allocated byte buffer and returning the buffer to the caller. Since you're allocating it with calloc, it will be initialised to 0, then overwritten with the image contents.
when I free this unsigned char and allocate it again, I see that it reserves the same address in memory which was allocated previous time
Yes, there are no guarantees about the location of the buffer in memory. Assuming you call free() on the returned memory, requesting the exact same size is quite likely to give you the same buffer back. But - how are you verifying the contents are not written over a second time? What is in the buffer?
my problem is that because of this allocation , which happens once every 2 secs I have memory leak...But when I try to free the allocated memory sector happens thing described above....:(
If there is a leak, it is likely in the code that calls this method, since there is no obvious leakage here. The semantics are obviously such that the caller is responsible for freeing the buffer. So how is that done?
Also, are you verifying that the CGBitmapContext is being correctly created? It is possible that some creation flags or parameters may result in an error. So add a check for context being valid (at least not nil). That could explain why the content is not being overwritten.
One easy way to ensure your memory is being freshly updated is to write your own data to it. You could fill the buffer with a counter, and verify this outside the method. For example, just before you return rawData:
static unsigned updateCounter = 0;
memset(rawData, updateCounter & 0xff, width*height*4);
This will cycle through writing 0-255 into the buffer, which you can easily verify.
Another thing - what are you trying to achieve with this code? There might be an easier way to achieve what you're trying to achieve. Returning bare buffers devoid of metadata is not necessarily the best way to manage your images.
So guys I solved this issue...First thing I've changed createBitmapContext method to this
- (void) createBitmapContext:(UIImage*)anImage andRawData:(unsigned char *)theRawData{
CGImageRef imageRef = [anImage CGImage];
NSUInteger width = CGImageGetWidth(imageRef);
NSUInteger height = CGImageGetHeight(imageRef);
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
// unsigned char *rawData = (unsigned char*) calloc(height * width * 4, sizeof(unsigned char));
bytesPerPixel = 4;
bytesPerRow = bytesPerPixel * width;
NSUInteger bitsPerComponent = 8;
CGContextRef context = CGBitmapContextCreate(theRawData, width, height,
bitsPerComponent, bytesPerRow, colorSpace,
kCGImageAlphaPremultipliedLast | kCGBitmapByteOrder32Big);
CGColorSpaceRelease(colorSpace);
CGContextDrawImage(context, CGRectMake(0, 0, width, height), imageRef);
CGContextRelease(context);
imageRef=nil;
// return theRawData;}
then...besides this I missed the part where I assign newRawData to oldRawData and by this I was having two pointers to the same memory address...So from here came the issue... I changed this assignment part to this memcpy(rawDataForOldImage, rawDataForNewImage,newCapturedImage.size.width*newCapturedImage.size.height*4); and here the problem is solved....Thanks to all
I am working through some existing code for a project i am assigned to.
I have a successful call to glTexImage2D like this:
glTexImage2D(GL_TEXTURE_2D, 0, texture->format, texture->widthTexture, texture->heightTexture, 0, texture->format, texture->type, texture->data);
I would like create an image (preferably a CGImage or UIImage) using the variables passed to glTexImage2D, but don't know if it's possible.
I need to create many sequential images(many of them per second) from an OpenGL view and save them for later use.
Should i be able to create a CGImage or UIImage using the variables i use in glTexImage2D?
If i should be able to, how should i do it?
If not, why can't i and what do you suggest for my task of saving/capturing the contents of my opengl view many times per second?
edit: i have already successfully captured images using some techniques provided by apple with glReadPixels, etc etc. i want something faster so i can get more images per second.
edit: after reviewing and adding the code from Thomson, here is the resulting image:
the image very slightly resembles what the image should look like, except duplicated ~5 times horizontally and with some random black space underneath.
note: the video(each frame) data is coming over an ad-hoc network connection to the iPhone. i believe the camera is shooting over each frame with the YCbCr color space
edit: further reviewing Thomson's code
I have copied your new code into my project and got a different image as result:
width: 320
height: 240
i am not sure how to find the number of bytes in texture-> data. it is a void pointer.
edit: format and type
texture.type = GL_UNSIGNED_SHORT_5_6_5
texture.format = GL_RGB
Hey binnyb, here's the solution to creating a UIImage using the data stored in texture->data. v01d is certainly right that you're not going to get the UIImage as it appears in your GL framebuffer, but it'll get you an image from the data before it has passed through the framebuffer.
Turns out your texture data is in 16 bit format, 5 bits for red, 6 bits for green, and 5 bits for blue. I've added code for converting the 16 bit RGB values into 32 bit RGBA values before creating a UIImage. I'm looking forward to hearing how this turns out.
float width = 512;
float height = 512;
int channels = 4;
// create a buffer for our image after converting it from 565 rgb to 8888rgba
u_int8_t* rawData = (u_int8_t*)malloc(width*height*channels);
// unpack the 5,6,5 pixel data into 24 bit RGB
for (int i=0; i<width*height; ++i)
{
// append two adjacent bytes in texture->data into a 16 bit int
u_int16_t pixel16 = (texture->data[i*2] << 8) + texture->data[i*2+1];
// mask and shift each pixel into a single 8 bit unsigned, then normalize by 5/6 bit
// max to 8 bit integer max. Alpha set to 0.
rawData[channels*i] = ((pixel16 & 63488) >> 11) / 31.0 * 255;
rawData[channels*i+1] = ((pixel16 & 2016) << 5 >> 10) / 63.0 * 255;
rawData[channels*i+2] = ((pixel16 & 31) << 11 >> 11) / 31.0 * 255;
rawData[channels*4+3] = 0;
}
// same as before
int bitsPerComponent = 8;
int bitsPerPixel = channels*bitsPerComponent;
int bytesPerRow = channels*width;
CGColorSpaceRef colorSpaceRef = CGColorSpaceCreateDeviceRGB();
CGBitmapInfo bitmapInfo = kCGBitmapByteOrderDefault;
CGColorRenderingIntent renderingIntent = kCGRenderingIntentDefault;
CGDataProviderRef provider = CGDataProviderCreateWithData(NULL,
rawData,
channels*width*height,
NULL);
free( rawData );
CGImageRef imageRef = CGImageCreate(width,
height,
bitsPerComponent,
bitsPerPixel,
bytesPerRow,
colorSpaceRef,
bitmapInfo,
provider,NULL,NO,renderingIntent);
UIImage *newImage = [UIImage imageWithCGImage:imageRef];
The code for creating a new image comes from Creating UIImage from raw RGBA data thanks to Rohit. I've tested this with our original 320x240 image dimension, having converted a 24 bit RGB image into 5,6,5 format and then up to 32 bit. I haven't tested it on a 512x512 image but I don't expect any problems.
You could make an image from the data you are sending to GL, but I doubt that's really what you want to achieve.
My guess is you want the output of the Frame Buffer. To do that you need glReadPixels(). Bare in mind for a large buffer (say 1024x768) it will take seconds to read the pixels back from GL, you wont get more than 1 per second.
You should be able to use the UIImage initializer imageWithData for this. All you need is to ensure that the data in texture->data is in a structured format that is recognizable to the UIImage constructor.
NSData* imageData = [NSData dataWithBytes:texture->data length:(3*texture->widthTexture*texture->heightTexture)];
UIImage* theImage = [UIImage imageWithData:imageData];
The types that imageWithData: supports are not well documented, but you can create NSData from .png, .jpg, .gif, and I presume .ppm files without any difficulty. If texture->data is in one of those binary formats I suspect you can get this running with a little experimentation.