Generating thumbnails causes leak (MacOS, Obj C) - objective-c

I'm working on a MacOS program in Objective C that needs to produce in-memory thumbnails to send to a server. The following code is used to perform this operation. As the program runs, a leak of about 40mb is induced each time this method is called. I'm missing something really basic, I suspect, but I don't see the source of the problem.
I should add that I've also tried creating one context to use over the life of the program and the problem, if anything, seems somewhat worse.
When I run Instruments, the allocations for the category "VM: ImageIO_JPEG_Data" are growing by one allocation of 40mb each time it's called. The responsible library is "ImageIO" and the responsible caller is "ImageIO_Malloc".
- (void) createPhotoThumbnail
{
NSURL* fileURL = [NSURL fileURLWithPath : _imagePath];
CGColorSpaceRef colorspace = CGColorSpaceCreateDeviceRGB();
CGContextRef bitmapContext = CGBitmapContextCreate(NULL, MAX_THUMB_DIM, MAX_THUMB_DIM, 8, 0,
colorspace, (CGBitmapInfo)kCGImageAlphaNoneSkipLast);
CIContext *ciContext = [CIContext contextWithCGContext: bitmapContext options: #{}];
if (fileURL)
{
CIImage *image = [[CIImage alloc] initWithContentsOfURL: fileURL];
if (image)
{
// scale the image
CIFilter *scaleFilter = [CIFilter filterWithName: #"CILanczosScaleTransform"];
[scaleFilter setValue: image forKey: #"inputImage"];
NSNumber *scaleFactor = [[NSNumber alloc] initWithFloat: ((float) MAX_THUMB_DIM) /
((float)MAX(_processedWidth, _processedHeight))];
[scaleFilter setValue: scaleFactor forKey: #"inputScale"];
[scaleFilter setValue: #1.0 forKey: #"inputAspectRatio"];
CIImage *scaledImage = [scaleFilter valueForKey: #"outputImage"];
NSMutableData* thumbJpegData = [[NSMutableData alloc] init];
CGImageDestinationRef dest = CGImageDestinationCreateWithData((__bridge CFMutableDataRef)thumbJpegData,
(__bridge CFStringRef)#"public.jpeg",
1,
NULL);
if (dest)
{
CGImageRef img = [ciContext createCGImage:scaledImage
fromRect:[scaledImage extent]];
CGImageDestinationAddImage(dest, img, nil);
if (CGImageDestinationFinalize(dest))
{
// encode it as a string for later
_thumbnail = [thumbJpegData base64EncodedStringWithOptions: 0];
}
else
{
DDLogError(#"Failed to generate photo thumbnail");
}
CGImageRelease(img);
CFRelease(dest);
}
else
{
DDLogError(#"Failed to finalize photo thumbnail image");
}
thumbJpegData = nil;
}
}
CGContextRelease(bitmapContext);
CGColorSpaceRelease(colorspace);
ciContext = nil;
}
UPDATE: I switched the code to use a CGAffineTransform instead of the filter with "CILanczosScaleTransform" and the symptom did not change. Next I used a completely new method (snippet below) and yet the problem persists.
NSImage *thumbnail = [[NSImage alloc] initWithSize: newSize];
[thumbnail lockFocus];
[sourceImage setSize: newSize];
[[NSGraphicsContext currentContext] setImageInterpolation:NSImageInterpolationHigh];
[sourceImage compositeToPoint: NSZeroPoint operation: NSCompositeCopy];
[thumbnail unlockFocus];
NSData *tiff = [thumbnail TIFFRepresentation];
NSBitmapImageRep *imageRep = [NSBitmapImageRep imageRepWithData: tiff];
NSDictionary *imageProps = [NSDictionary dictionaryWithObject:[NSNumber numberWithFloat:0.9] forKey:NSImageCompressionFactor];
NSData *thumbJpegData = [imageRep representationUsingType:NSJPEGFileType properties:imageProps];
This is making me think the problem is perhaps related to something inherent in the way I'm doing this. I find it hard to believe two different methods of image scaling are going to exhibit the same sort of leak.

Thanks to this answer I was able to identify the need for an autorelease pool, something I was completely unaware of. The code in the question is one of a series of methods that are called repeatedly from inside a tight loop. This apparently prevents the OS from having a chance to do some cleanup. The block now looks like this:
#autoreleasepool {
[self findRelevantAdjustments];
[self adjustForStraightenCrop];
[self moveFacesRelativeToTopLeftOrigin];
[self createPhotoThumbnail];
[self sendPhotoToServer];
}
Moral of the story: even with ARC there are more things to pay attention to when it comes to the memory lifecycle.

The problem is not in the CGImageDestinationRef logic, because it still leaks even if you replace that with something far simple, such as:
NSBitmapImageRep *rep = [[NSBitmapImageRep alloc] initWithCIImage:scaledImage];
NSData *data = [rep representationUsingType:NSJPEGFileType properties:nil];
Digging a little further, it would appear that the problem appears to be an issue within CILanczosScaleTransform. If you use an inputScale of #1.0, then the leak disappears. But use something less than #1.0 (even #0.5) and it leaks.
I'd suggest you consider finding a different method for resizing the image.

Related

Creating an animated GIF in Cocoa - defining frame type

I've been able to adapt some code found on SO to produce an animated GIF from the "screenshots" of my view, but the results are unpredictable. GIF frames are sometimes full images, full frames ("replace" mode, as GIMP marks it), other times are just a "diff" from previous layer ("combine" mode).
From what I've seen, when there are fewer and/or smaller frames involved, the CG writes the GIF in "combine" mode, but failing to get the colors right. Actually, the moving parts are colored correctly, the background is wrong.
When CG saves the GIF as full frames, the colors are ok. The file size is larger, but hey, obviously you cannot have the best of both worlds. :)
Is there a way to either:
a) force CG to create "full frames" when saving the GIF
b) fix the colors (color table?)
What I do is (ARC mode):
capture the visible part of the view with
[[scrollView contentView] dataWithPDFInsideRect:[[scrollView contentView] visibleRect]];
convert and resize it to NSImageBitmapRep of PNG type
-(NSMutableDictionary*) pngImageProps:(int)quality {
NSMutableDictionary *pngImageProps;
pngImageProps = [[NSMutableDictionary alloc] init];
[pngImageProps setValue:[NSNumber numberWithBool:NO] forKey:NSImageInterlaced];
double compressionF = 1;
[pngImageProps setValue:[NSNumber numberWithFloat:compressionF] forKey:NSImageCompressionFactor];
return pngImageProps;
}
-(NSData*) resizeImageToData:(NSData*)data toDimX:(int)xdim andDimY:(int)ydim withQuality:(int)quality{
NSImage *image = [[NSImage alloc] initWithData:data];
NSRect inRect = NSZeroRect;
inRect.size = [image size];
NSRect outRect = NSMakeRect(0, 0, xdim, ydim);
NSImage *outImage = [[NSImage alloc] initWithSize:outRect.size];
[outImage lockFocus];
[image drawInRect:outRect fromRect:inRect operation:NSCompositeCopy fraction:1];
NSBitmapImageRep* bitmapRep = [[NSBitmapImageRep alloc] initWithFocusedViewRect:outRect];
[outImage unlockFocus];
NSMutableDictionary *imageProps = [self pngImageProps:quality];
NSData* imageData = [bitmapRep representationUsingType:NSPNGFileType properties:imageProps];
return [imageData copy];
}
get the array of BitmapReps and create the GIF
-(CGImageRef) pngRepDataToCgImageRef:(NSData*)data {
CFDataRef imgData = (__bridge CFDataRef)data;
CGDataProviderRef imgDataProvider = CGDataProviderCreateWithCFData (imgData);
CGImageRef image = CGImageCreateWithPNGDataProvider(imgDataProvider, NULL, true, kCGRenderingIntentDefault);
return image;
}
////////// create GIF from
NSArray *images; // holds all BitmapReps
CGImageDestinationRef destination = CGImageDestinationCreateWithURL((__bridge CFURLRef)[NSURL fileURLWithPath:pot],
kUTTypeGIF,
allImages,
NULL);
// set frame delay
NSDictionary *frameProperties = [NSDictionary
dictionaryWithObject:[NSDictionary
dictionaryWithObject:[NSNumber numberWithFloat:0.2f]
forKey:(NSString *) kCGImagePropertyGIFDelayTime]
forKey:(NSString *) kCGImagePropertyGIFDictionary];
// set gif color properties
NSMutableDictionary *gifPropsDict = [[NSMutableDictionary alloc] init];
[gifPropsDict setObject:(NSString *)kCGImagePropertyColorModelRGB forKey:(NSString *)kCGImagePropertyColorModel];
[gifPropsDict setObject:[NSNumber numberWithBool:YES] forKey:(NSString *)kCGImagePropertyGIFHasGlobalColorMap];
// set gif loop
NSDictionary *gifProperties = [NSDictionary
dictionaryWithObject:gifPropsDict
forKey:(NSString *) kCGImagePropertyGIFDictionary];
// loop through frames and add them to GIF
for (int i=0; i < [images count]; i++) {
NSData *imageData = [images objectAtIndex:i];
CGImageRef imageRef = [self pngRepDataToCgImageRef:imageData];
CGImageDestinationAddImage(destination, imageRef, (__bridge CFDictionaryRef) (frameProperties));
}
// save the GIF
CGImageDestinationSetProperties(destination, (__bridge CFDictionaryRef)(gifProperties));
CGImageDestinationFinalize(destination);
CFRelease(destination);
I've checked the ImageBitmapReps, when saved as PNG individually, they are just fine.
As I understood, the color tables should be handled by CG or am I responsible to produce the dithered colors? How to do that?
Even when doing the same animation repeatedly, the GIFs produced may vary.
This is a single BitmapRep
(source: andraz.eu)
And this is the GIF with the invalid colors ("combine" mode)
(source: andraz.eu)
I read your code. Please double check the "allImages" while you are creating the CGImageDestinationRef, and the "[images count]".
the follow test code works fine:
NSDictionary *prep = [NSDictionary dictionaryWithObject:[NSDictionary dictionaryWithObject:[NSNumber numberWithFloat:0.2f] forKey:(NSString *) kCGImagePropertyGIFDelayTime] forKey:(NSString *) kCGImagePropertyGIFDictionary];
CGImageDestinationRef dst = CGImageDestinationCreateWithURL((__bridge CFURLRef)(fileURL), kUTTypeGIF, [filesArray count], nil);
for (int i=0;i<[filesArray count];i++)
{
//load anImage from array
...
CGImageRef imageRef=[anImage CGImageForProposedRect:nil context:nil hints:nil];
CGImageDestinationAddImage(dst, imageRef,(__bridge CFDictionaryRef)(prep));
}
bool fileSave = CGImageDestinationFinalize(dst);
CFRelease(dst);

NSURL of UIImage from UIImagePickerController returns (null)

- (void)imagePickerController:(UIImagePickerController *)picker didFinishPickingImage:(UIImage *)image editingInfo:(NSDictionary *)editInfo {
userURL = [editInfo objectForKey:UIImagePickerControllerMediaURL];
userImage = image;
userImageView.image=userImage;
[self dismissViewControllerAnimated:YES completion:nil];}
I then take NSURL userURL, and put it in an UIActivityViewController to use to upload the image. However this never works, and always fails as it's trying to upload (null). However, when I use a preset image included in the xcode project and the following code, it always works and uploads correctly:
NSURL *url = [[NSBundle mainBundle] URLForResource:#"kitten.jpg" withExtension:nil];
If it helps, I'm using https://github.com/goosoftware/GSDropboxActivity
When I use UIImagePickerControllerReferenceURL instead of UIImagePickerControllerMediaURL, I get the following error:
[WARNING] DropboxSDK: File does not exist (/asset.JPG)
Failed to upload assets-library://asset/asset.JPG?id=EECAF4D0-A5ED-40E7-8E6F-3A586C0AB06E&ext=JPG
In theory, UIImagePickerControllerMediaURL is only populated for a movie, not an image. If you use the UIImagePickerControllerReferenceURL then the URL that you're given is not a URL for the file system, but rather the assets library. To get the file from that you'll need to use the asset library. Use something like the following :
typedef void (^ALAssetsLibraryAssetForURLResultBlock)(ALAsset *asset);
typedef void (^ALAssetsLibraryAccessFailureBlock)(NSError *error);
ALAssetsLibraryAssetForURLResultBlock resultblock = ^(ALAsset *myasset){
ALAssetRepresentation *rep = [myasset defaultRepresentation];
CGImageRef iref = [rep fullResolutionImage];
if (iref){
UIImage *myImage = [UIImage imageWithCGImage:iref scale:[rep scale] orientation:(UIImageOrientation)[rep orientation]];
// Do whatever you now want with your UIImage
}
};
ALAssetsLibraryAccessFailureBlock failureblock = ^(NSError *myerror){
//failed to get image.
};
ALAssetsLibrary* assetslibrary = [[ALAssetsLibrary alloc] init];
[assetslibrary assetForURL:[filePath objectAtIndex:0] resultBlock:resultblock failureBlock:failureblock];
This takes care of the blocks that will handle your request, but you still need to actually request the resource from the assets library. For that, try this :
ALAssetsLibrary* assetslibrary = [[[ALAssetsLibrary alloc] init] autorelease];
NSURL myAssetUrl = [NSURL URLWithString:[editInfo objectForKey:UIImagePickerControllerMediaURL]];
[assetslibrary assetForURL:myAssetUrl resultBlock:resultblock failureBlock:failureblock];
And in theory, you should get what you're after :)
The code for this was given by Ramshad and further discussion of it can be found here
Hopefully this will help you with what you're after. Sorry if it's a bit too late :(
Edit
Note that if you're not using ARC, you'll need to tidy up the memory in this example as I haven't included any memory management at all.

Mysterious CoreImage memory leak using ARC

I'm experiencing some massive memory leaks that don't show up using the "leaks" instrument. I pop up a Modal View Controller and apply 2 CoreImage filters to 4 or 5 different images. Using Instruments I can see the memory jump up about 40-50 MB as these images are created, but even after I dismiss the Modal View Controller, I never get that memory back, and the application will crash after repeating this process 2 or 3 times. I'm happy for any advice you can provide because this is driving me absolutely crazy. Below is the method in question:
UIView *finalView = [[UIView alloc] initWithFrame:CGRectMake(1024, 0, 1792, 1345)];
UIImageView *templateImageView = [[UIImageView alloc] initWithFrame:CGRectMake(0, 0, 1792, 1345)];
templateImageView.image = [UIImage imageNamed:[NSString stringWithFormat:#"%#.png",[theme objectForKey:#"template_background"]]];
//CI background Setup
NSString *filePath5 = [[NSBundle mainBundle] pathForResource:[theme objectForKey:#"template_background"] ofType:#"png"];
NSURL *fileNameAndPath5 = [NSURL fileURLWithPath:filePath5];
#autoreleasepool {
finalBackBeginImage = [CIImage imageWithContentsOfURL:fileNameAndPath5];
finalBackImage = [CIFilter filterWithName:#"CIHueAdjust" keysAndValues:#"inputAngle", [NSNumber numberWithFloat:[[boothPrefs objectForKey:#"templateBackground_hue"] floatValue]*6.28], #"inputImage", finalBackBeginImage, nil].outputImage;
finalBackImage = [CIFilter filterWithName:#"CIColorControls" keysAndValues:#"inputSaturation", [NSNumber numberWithFloat:([[boothPrefs objectForKey:#"templateBackground_saturation"] floatValue] * 5)], #"inputImage", finalBackImage, nil].outputImage;
finalBackContent = [CIContext contextWithOptions:nil];
CGImageRef cgimgFinalBack =
[finalBackContent createCGImage:finalBackImage fromRect:[finalBackImage extent]];
UIImage *newFinalBackImg = [UIImage imageWithCGImage:cgimgFinalBack];
[templateImageView setImage:newFinalBackImg];
CGImageRelease(cgimgFinalBack);
}
[finalView addSubview:templateImageView];
I've switched from using imageNamed to using imageWithData using the code below. In 5 minutes of testing (sorry, spent close to 12 hours on this issue now), I see that my real memory usage for the same operation is up to 50% lower (115mb versus up to 230 mb) and the mysterious "Push +80mb, pop -30mb" real memory issue appears to be solved.
I'm keeping my fingers crossed though.
//use images like this as base images for CIFilters
NSData* imageData = [NSData dataWithContentsOfFile:[[NSBundle mainBundle] pathForResource:self.frameName ofType:nil]];
;
UIImage* imageForFilter =[UIImage imageWithData: imageData];

How do I set the pixels per inch for an exported JPEG image in a Cocoa app?

I have a Cocoa Mac image editing app which lets users export JPEG images. I'm currently using the following code to export these images as JPEG files:
//this is user specified
NSInteger resolution;
NSImage* savedImage = [[NSImage alloc] initWithSize:NSMakeSize(600, 600)];
[savedImage lockFocus];
//draw here
[savedImage unlockFocus];
NSBitmapImageRep* savedImageBitmapRep = [NSBitmapImageRep imageRepWithData:[savedImage TIFFRepresentationUsingCompression:NSTIFFCompressionNone factor:1.0]];
NSDictionary* properties = [NSDictionary dictionaryWithObjectsAndKeys:[NSNumber numberWithFloat:1.0], NSImageCompressionFactor, nil];
//holds the jpeg file
NSData * imageData = nil;
imageData = [savedImageBitmapRep representationUsingType:NSJPEGFileType properties:properties];
However, I would like for the user to be able to provide the pixels per inch for this JPEG image (like you can in Photoshop's export options). What would I need to modify in the above code to adjust this value for the exported JPEG?
I couldn't find a way to do it with the NSImage APIs but CGImage can by setting kCGImagePropertyDPIHeight/Width.
I also set kCGImageDestinationLossyCompressionQuality which I think is the same as NSImageCompressionFactor.
//this is user specified
NSInteger resolution = 100;
NSImage* savedImage = [[NSImage alloc] initWithSize:NSMakeSize(600, 600)];
[savedImage lockFocus];
//draw here
[savedImage unlockFocus];
NSBitmapImageRep* savedImageBitmapRep = [NSBitmapImageRep imageRepWithData:[savedImage TIFFRepresentationUsingCompression:NSTIFFCompressionNone factor:1.0]];
NSDictionary* properties = [NSDictionary dictionaryWithObjectsAndKeys:
[NSNumber numberWithFloat:1.0], kCGImageDestinationLossyCompressionQuality,
[NSNumber numberWithInteger:resolution], kCGImagePropertyDPIHeight,
[NSNumber numberWithInteger:resolution], kCGImagePropertyDPIWidth,
nil];
NSMutableData* imageData = [NSMutableData data];
CGImageDestinationRef imageDest = CGImageDestinationCreateWithData((CFMutableDataRef) imageData, kUTTypeJPEG, 1, NULL);
CGImageDestinationAddImage(imageDest, [savedImageBitmapRep CGImage], (CFDictionaryRef) properties);
CGImageDestinationFinalize(imageDest);
// Do something with imageData
if (![imageData writeToFile:[#"~/Desktop/test.jpg" stringByExpandingTildeInPath] atomically:NO])
NSLog(#"Failed to write imageData");
For NSImage or NSImageRep you do not set the resolution directly but set the size instead.
For size, numberOfPixels and resolution the following equation holds:
size = numberOfPixels * 72.0 / resolution
size is a length and is expressed in dots with the unit inch/72.
(size and resolution are floats). You can see that for an image with dpi=72 size and numberOfPixels are numerally the same (but the meaning is very different).
After creating an NSBitmapImageRep the size with the desired resolution can be set:
NSBitmapImageRep* savedImageBitmapRep = . . . ; // create the new rep
NSSize newSize;
newSize.width = [savedImageBitmapRep pixelsWide] * 72.0 / resolution; // x-resolution
newSize.height = [savedImageBitmapRep pixelsHigh] * 72.0 / resolution; // y-resolution
[savedImageBitmapRep setSize:newSize];
// save the rep
Two remarks: do you really need the lockFocus / unlockFocus way? The preferred way to build a new NSBitmapImageRep is to use NSGraphicsContext. see : http://www.mail-archive.com/cocoa-dev#lists.apple.com/msg74857.html
And: to use TIFFRepresentation for an NSBitmapImageRep is very time and space consuming. Since 10.6 another
way exists and costs nothing, because lockFocus and unlockFocus create an object of class NSCGImageSnapshotRep which under the hood is a CGImage. (In OS versions before 10.6 it was an NSCachedImageRep.) The following does it:
[anImage lockFocus];
// draw something
[anImage unlockFocus];
// now anImage contains an NSCGImageSnapshotRep
CGImageRef cg = [anImage CGImageForProposedRect:NULL context:nil hints:nil];
NSBitmapImageRep *newRep = [[NSBitmapImageRep alloc] initWithCGImage:cg];
// set the resolution
// here you may NSLog anImage, cg and newRep
// save the newRep
// release the newRep if needed

Garbage Collection Crash using NSImage

This piece of code was split off from a project I am working on. It consistently reproduces a garbage collection error on my Mac OS 10.5.7 and sometimes crashes. I have been looking at it for too long so my question is: does anybody else see why this would give errors when garbage collection is on?
- (void) doCrash: (id) sender
{
NSArray *lURLArray = [ NSArray arrayWithObjects:
#"http://userserve-ak.last.fm/serve/300x300/23621007.jpg",
#"http://userserve-ak.last.fm/serve/300x300/26675609.png",
#"http://userserve-ak.last.fm/serve/300x300/26675609.png",
nil ];
NSString *lImageURL = nil;
for (lImageURL in lURLArray)
{
NSImage *lImage = [[NSImage alloc] initWithContentsOfURL: [NSURL URLWithString: lImageURL]];
NSSize targetSize = NSMakeSize(80,80);
NSImage *newImage = [[NSImage alloc] initWithSize:targetSize];
[newImage lockFocus];
NSRect thumbnailRect = NSMakeRect(0,0,80,80);
NSRect sourceRect = NSMakeRect(0,0,[lImage size].width,[lImage size].height);
[lImage drawInRect: thumbnailRect
fromRect: sourceRect
operation: NSCompositeSourceOver
fraction: 1.0];
[newImage unlockFocus];
}
}
When playing around with the URLs in the lURLArray I get different behavior: sometimes crashes, sometimes the error message.
The garbage collection error message is triggered when the garbage collector is freeing one of the images and goes like this:
reference count underflow for <address>, break on auto_refcount_underflow_error to debug.
Any help is much appreciated,
thanks,
Kristof
This has been confirmed to me by someone from Apple as a bug in OX X 10.5.7.
rdar://problem/6938657