iPhone iOS6 how to return Image IO type "resident dirty memory" to the OS? - objective-c

I"m looking at the WWDC 2010 video which deals with advanced memory analysis( session 311):
At around 45:00 into the video, the performance engineer discusses what to do with "Resident Dirty memory" that your app has loaded in RAM. The engineer suggests that in response to memory warnings, your app should clear this. The engineer pastes in his custom class "flush" method into didReceiveMemoryWarning and everything is fine, but the code does not really offer any examples of HOW the memory is to be freed.
The question that I have is - how to do I flush large chunks of dirty memory used by "Image IO"? :
Here's around 74 mb of memory just sitting around dirty ( for close to 6 minutes now), waiting for someone to return it to iOS6. Nothing is happening to it. Since it does not go away on its own, I need to know how to return it to iOS.
These blocks appear to originate from code like this and (maybe other image related operations).
UIImage *screenshot = nil;
#autoreleasepool {
if ([[UIScreen mainScreen] respondsToSelector:#selector(scale)])
UIGraphicsBeginImageContextWithOptions(iPhoneRetinaIconSize, NO, [UIScreen mainScreen].scale);
else
UIGraphicsBeginImageContext(iPhoneRetinaIconSize);
[self.view.layer renderInContext:UIGraphicsGetCurrentContext()];
screenshot = UIGraphicsGetImageFromCurrentImageContext();
UIGraphicsEndImageContext();
}
The issue is that there's a lot of memory sitting around, loaded in RAM, unable to be returned to the operating system until the app crashes.
For webview-related dirty memory, I found that this may work:
- (void)didReceiveMemoryWarning
{
[super didReceiveMemoryWarning];
[[NSURLCache sharedURLCache] removeAllCachedResponses];
[[NSURLCache sharedURLCache] setDiskCapacity:0];
[[NSURLCache sharedURLCache] setMemoryCapacity:0];
// Dispose of any resources that can be recreated.
}
Is there an equivalent for UIImage, CALayer or UIGraphics ?

I am far from an expert in these issues, but based on the tests I conducted with the code you provided, I'd say you just have to release the UIImages created in these blocks of code.
As far as I understand, the Image IO or GC raster data labeled chunks of memory are really just the underlying data of your images (UIImage being a UIKit wrapper on top of these). So to release the memory, release the image.
I tested this by creating a bunch of UIImages using your code, simulating a memory warning which released all of the created images:
The images speak for themselves. Releasing my UIImages (at ~00:08) removed the big GC raster data chunk from resident memory.
Because removing completely an image from your UI may not be the best solution for user experience, maybe you could try to downsize your largest images when receiving a memory warning, a poorer resolution resulting in a smaller memory footprint. Another idea (again this depends on what your images are used for) could be to dump the images to disk, and load them latter, when needed.
Hope that helps.

Related

UIImagePickerController terminated due to Memory Pressure

I have a UIViewController that has :
#property UIImagePickerController* mainPicker;
and with a button, I'm presenting that mainPicker like :
- (IBAction)takePhoto:(UIButton *)sender
{
// Take a photo.
if(![UIImagePickerController isSourceTypeAvailable:UIImagePickerControllerSourceTypeCamera])
{
// No camera is available, show an alert.
UIAlertView* newAlert = [[UIAlertView alloc] initWithTitle:#"Warning"
message:#"Camera is not available."
delegate:self
cancelButtonTitle:#"OK"
otherButtonTitles:nil];
[newAlert show];
return;
}
if(mainPicker == nil)
{
mainPicker = [[UIImagePickerController alloc]init];
mainPicker.delegate = self;
mainPicker.allowsEditing = YES; //I've tried without this line, didn't affect at all.
mainPicker.sourceType = UIImagePickerControllerSourceTypeCamera;
}
[self presentViewController:mainPicker animated:YES completion:nil];
}
The first problem is ;
Snapshotting a view that has not been rendered results in an empty snapshot.
Ensure your view has been rendered at least once before snapshotting or snapshot after screen updates.
Also, whenever that view controller is presented, there are at least two memory warnings.
After I take a photo, it gets worse. It's literally spamming "Received memory warning.".
Here is an Instrument screenshot, hope it would help.
The memory is about 4 MB at the beginning. After taking a photo, it goes up to 10 MB. While dismissing, I'm saving the UIImage, so it's nearly 30 MB after the dismissal.
(That peak of the memory is probably caused by writeToFile:. Also, that leak there is about 600 bytes only).
Currently, I'm testing on iPhone 5S, with iOS 7.
I've tried enabling zombies, dispatching the picker after a while, allowing/disallowing editing, etc. None of them worked. Also, I'm not trying to present picker view instantly after loading the view controller.
Additional note, I've used the functions in the answer, and here is the logs;
Memory used 9588.7 (+9589), free 32063.5 kb
Memory used 10281.0 ( +692), free 18448.4 kb
Watching memory usage in iOS
Isn't it a bit weird to see 32 MB free memory in the device, whilst Instruments is telling another story?
Here are a few explanations to help you solve your problem.
First of all, the Zombies diagnostic tool is meant to debug crashes in which memory that was already freed is being accessed. This doesn't seem to be your problem here and thus the Zombies tool will be useless to you for this particular problem.
Second, the screenshot you have provided us is showing the Leaks instruments. The elements you see in that list are objects that your program has allocated and forgotten about without releasing them beforehand. This means that you do not have any single remaining pointer towards that memory that Instruments knows about. Fixing these leaks is a first step towards fixing your memory warnings.
Third, fixing your leaks probably won't be enough to fix your memory warning problems. These warnings indicate that you are using too much memory to iOS's liking. Considering your leaks account for a mere 600 bytes, the problem seems to be your abandoned memory. Abandoned memory is memory that you have allocated, and towards which you still have live references even though they probably won't ever be used again by your application.
In order to help you in fixing your problems, here is some relevant documentation to fix both memory leaks and abandoned memory using Instruments :
https://developer.apple.com/library/mac/recipes/Instruments_help_articles/FindingMemoryLeaksinYourApp/FindingMemoryLeaksinYourApp.html#//apple_ref/doc/uid/TP40012965-CH32-SW1
https://developer.apple.com/library/mac/recipes/Instruments_help_articles/FindingAbandonedMemory/FindingAbandonedMemory.html
Also, here is a useful blog post about abandoned memory :
http://www.friday.com/bbum/2010/10/17/when-is-a-leak-not-a-leak-using-heapshot-analysis-to-find-undesirable-memory-growth/

set UIImage:nil in scrollview doesn't release memory with ARC if already referenced outside of scrollview

Here's what my app does.
It requests about 100 images from the server when it starts up. It stores
those images in model objects, we'll call them modelObject.storedImage
The modelObject also has a viewObject, which is a UIImageView subclass
The app has a 2D scrollview that sets all 100 viewObjects as subviews, with up to 16 of those shown in the scrollview content area at any give time
When you scroll to a given area, it finds the 16 images from the appropriate
modelObject.storedImage and does
modelObject.viewObject.image = modelObjects.storedImage;
When a viewObject scrolls out of the content area, it is purged from the view
[modelObject.viewObject setImage:nil];
[modelObject.viewObject removeFromSuperview];
The problem is, despite the purging, memory keeps growing as you scroll to new areas of the scrollview content area. Memory from putting the former modelObject.viewObject.images on the scrollview is not released. So if you scroll to enough new areas, memory use gets out of control.
To be clear, when the app first loads the 100 images into the modelObject.storedImages, memory does not get out of control. Furthermore, if I comment this out
//modelObject.viewObject.image = modelObjects.storedImage;
then memory also stays fine (but of course the scrollviews subviews don't show any image).
I did see this question "Setting uiimage to nil doesn't release memory with ARC" but I believe it's different because in my app I can't afford to keep requesting images over and over from the server. They must sit at modelObject.storedImage once they've been requested, always ready to be displayed when their corresponding viewObject is scrolled to.
Surely there must be a way to tell ARC to release the memory for these nil images? Thanks in advance for your help!
If images are created by imageNamed:, then no way to manually to release memory, you can optimize it by imageWithContentsOfFile, thanks to you are using images for server, I think you used imageWithContentsOfFile. :)
You can try use #autoreleasepool {} at scrolling method to release memory in real time.
You can try release viewObject instead of removeFromSuperview, maybe what you should store is viewFrame rather than viewObject.
Hope offer you some help.
So I went around this problem and did this as a solution. I'd be curious to see if someone has a more robust idea. I made a copy of the image and set the copy on the scrollview. Now when I purge that copy, its memory is actually released (thank you!!), while the original image still sits comfortably in the model. Note, it did not work to simply copy the image's CGImage and make a copy off that, in which case memory did not release. I had to make a new copy using a UIGraphicsImageContext.
Code to set the image on Scrollview:
UIImage *imageToCopy = modelObject.imageStore;
CGSize size = imageToCopy.size;
UIGraphicsBeginImageContext(size);
[imageToCopy drawInRect:CGRectMake(0, 0, imageToCopy.size.width, imageToCopy.size.height)];
UIImage *copiedImage = UIGraphicsGetImageFromCurrentImageContext();
UIGraphicsEndImageContext();
[modelObject.viewObject setImage:copiedImage];
Code to purge when it's off the scrollview's content area:
[modelObject.viewObject setImage:nil];
[modelObject.viewObject removeFromSuperview];
The only problem was that drawing those image contexts used a lot of memory, so I added this modification to make the drawing smaller:
...
CGSize size = imageToCopy.size;
size.width *= .2;
size.height *= .2;
UIGraphicsBeginImageContext(size);
[imageToCopy drawInRect:CGRectMake(0, 0, imageToCopy.size.width*.2, imageToCopy.size.height*.2)];
...

Memory warning and crash (ARC) - how to identify why it's happening?

I've started to use the ARC recently and since then I blame it for every single memory problem. :) Perhaps, you could help me better understand what I'm doing wrong.
My current project is about CoreGraphics a lot - charts drawing, views filled with thumbnails and so on. I believe there would be no problem while using manual memory management, except maybe a few zombies... But as of now, application simply crashes every time I try to either create a lot of thumbnails or redraw a bit more complicated chart.
While profiling with Instruments I can see an awfully high value in resident memory as well as in the dirty one. Heap analysis shows rather alarming irregular grow...
When drawing just a few thumbnails, resident memory grows for about 200 MB. When everything is drawn, memory drops back on almost the same value as before drawing. However, with a lot of thumbnails, a value in resident memory is higher than 400 MB and that obviously crashes the app. I've tried to limit number of thumbnails drawn at the same time (NSOperationQueue and its maxConcurrentOperationCount), but as releasing so much memory seems to take a bit more time, it didn't really solve the issue.
Right now my app basically doesn't work as the real data works with a lot of complicated charts = lot of thumbnails.
Every thumbnail is created with this code I got from around here: (category on UIImage)
+ (void)beginImageContextWithSize:(CGSize)size
{
if ([[UIScreen mainScreen] respondsToSelector:#selector(scale)]) {
if ([[UIScreen mainScreen] scale] == 2.0) {
UIGraphicsBeginImageContextWithOptions(size, YES, 2.0);
} else {
UIGraphicsBeginImageContext(size);
}
} else {
UIGraphicsBeginImageContext(size);
}
}
+ (void)endImageContext
{
UIGraphicsEndImageContext();
}
+ (UIImage*)imageFromView:(UIView*)view
{
[self beginImageContextWithSize:[view bounds].size];
BOOL hidden = [view isHidden];
[view setHidden:NO];
[[view layer] renderInContext:UIGraphicsGetCurrentContext()];
UIImage *image = UIGraphicsGetImageFromCurrentImageContext();
[self endImageContext];
[view setHidden:hidden];
return image;
}
+ (UIImage*)imageFromView:(UIView*)view scaledToSize:(CGSize)newSize
{
UIImage *image = [self imageFromView:view];
if ([view bounds].size.width != newSize.width ||
[view bounds].size.height != newSize.height) {
image = [self imageWithImage:image scaledToSize:newSize];
}
return image;
}
+ (UIImage*)imageWithImage:(UIImage*)image scaledToSize:(CGSize)newSize
{
[self beginImageContextWithSize:newSize];
[image drawInRect:CGRectMake(0,0,newSize.width,newSize.height)];
UIImage *newImage = UIGraphicsGetImageFromCurrentImageContext();
[self endImageContext];
return newImage;
}
Is there some other way which wouldn't eat so much memory or is something really wrong with the code when using ARC?
The other place where memory warning + crash is happening is when there is too much redrawing of any view. It doesn't need to be quick, just many times. Memory stacks up until it crashes and I'm not able to find anything really responsible for it. (I can see a growing resident/dirty memory in VM Tracker and a heap growth in Allocations instrument)
My question basically is: how to find why it is even happening? My understanding is when there is no owner for given object, it's released ASAP. My inspection of code suggests a lot of objects are not released at all even though I don't see any reason for it to happen. I don't know about any retain cycles...
I've read through the Transitioning to ARC Release Notes, bbum's article about heap analysis and probably a dozen of others. Differs somehow heap analysis with and without ARC? I can't seem to do anything useful with its output.
Thank you for any ideas.
UPDATE: (to not force everybody read all the comments and to hold my promise)
By carefully getting through my code and adding #autoreleasepool, where it had any sense, memory consumption got lowered. The biggest problem was calling UIGraphicsBeginImageContext from background thread. After fixing it (see #Tammo Freese's answer for details) deallocation occurred soon enough to not crash an app.
My second crash (caused by many redrawing of the same chart), was completely solved by adding CGContextFlush(context) at the end of my drawing method. Shame on me.
A small warning for anyone trying to do something similar: use OpenGL. CoreGraphics is not quick enough for animating big drawings, especially not on an iPad 3. (first one with retina)
To answer your question: Identifying problems with memory warnings and crashes with ARC basically works like before with manual retain-release (MRR). ARC uses retain, release and autorelease just like MRR, it only inserts the calls for you, and has some optimizations in place that should even lower the memory consumption in some cases.
Regarding your problem:
In the screenshot of Instruments you posted, there are allocation spikes visible. In most cases I encountered so far, these spikes were caused by autoreleased objects hanging around too long.
You mentioned that you use NSOperationQueue. If you override -[NSOperationQueue main], make sure that you wrap the whole content of the method in #autoreleasepool { ... }. An autorelease pool may already be in place, but it is not guaranteed (and even if there is one, it may be around for longer than you think).
If 1. has not helped and you have a loop that processes the images, wrap the inner part of the loop in #autoreleasepool { ... } so that temporary objects are cleaned up immediately.
You mentioned that you use NSOperationQueue. Since iOS 4, drawing to a graphics context in UIKit is thread-safe, but if the documentation is right, UIGraphicsBeginImageContext should still only be called on the main thread! Update: The docs now state that since iOS 4, the function can be called from any thread, to the following is actually unnecessary! To be on the safe side, create the context with CGBitmapContextCreate and retrieve the image with CGBitmapContextCreateImage. Something along these lines:
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
CGContextRef context = CGBitmapContextCreate(NULL, width, height, 8, 0, colorSpace, kCGImageAlphaPremultipliedLast);
CGColorSpaceRelease(colorSpace);
// draw to the context here
CGImageRef newCGImage = CGBitmapContextCreateImage(context);
CGContextRelease(context);
UIImage *result = [UIImage imageWithCGImage:newCGImage scale:scale orientation: UIImageOrientationUp];
CGImageRelease(newCGImage);
return result;
So, nothing you are doing relative to memory management (there is none!) looks improper. However, you mention using NSOperationQueue. Those UIGraphics... calls are marked as not thread safe, but others have stated they are as of iOS 4 (I cannot find a definitive answer to this, but recall that this is true.
In any case, you should not be calling these class methods from multiple threads. You could create a serial dispatch queue and feed all the work through that to insure single threaded usage.
What's missing here of course is what you do with the images after using them. Its possible you are retaining them in some way that is not obvious. Here are some tricks:
in any of your classes that use lots of images, add a dealloc() method that just logs its name and some identifier.
you can try to add a dealloc to UIImage to do the same.
try to drive your app using the simplest possible setup - fewest images etc - so you can verify that in fact that the images and their owners are getting dealloc'd.
when you want to make sure something is released, set the ivar or property to nil
I converted a 100 file project to ARC last summer and it worked perfectly out of the box. I have converted several open source projects to ARC as well with only one problem when I improperly used bridging. The technology is rock solid.
This is not an answer to your question but I was trying to solve similar problems long before ARC was introduced.
Recently I was working on an application that was caching images in memory and releasing them all after receiving memory warning. This worked fine as long as I was using the application at a normal speed (no crazy tapping). But when I started to generate a lot of events and many images started to load, the application did not manage to get the memory warning and it was crashing.
I once wrote a test application that was creating many autoreleased objects after tapping a button. I was able to tap faster (and create objects) than the OS managed to release the memory. The memory was slowly increasing so after a significant time or simply using bigger objects I would surely crash the application and cause device to reboot (looks really effective ;)). I checked that using Instruments which unfortunately affected the test and made everything slower but I suppose this is true also when not using Instruments.
On the other occasion I was working on a bigger project that is quite complex and has a lot of UI created from code. It also has a lot of string processing and nobody cared to use release - there were few thousands of autorelease calls when I checked last time. So after 5 minutes of slightly extensive usage of this application, it was crashing and rebooting the device.
If I'm correct then the OS/logic that is responsible for actually deallocating memory is not fast enough or has not high enough priority to save an application from crashing when a lot of memory operations are performed. I never confirmed these suspicions and I don't know how to solve this problem other than simply reducing allocated memory.

How to speed up saving a UIImagePickerController image from the camera to the filesystem via UIImagePNGRepresentation()?

I'm making an applications that let users take a photo and show them both in thumbnail and photo viewer.
I have NSManagedObject class called photo and photo has a method that takes UIImage and converts it to PNG using UIImagePNGRepresentation() and saves it to filesystem.
After this operation, resize the image to thumbnail size and save it.
The problem here is UIImagePNGRepresentation() and conversion of image size seems to be really slow and I don't know if this is a right way to do it.
Tell me if anyone know the best way to accomplish what I want to do.
Thank you in advance.
Depending on the image resolution, UIImagePNGRepresentation can indeed be quite slow, as can any writing to the file system.
You should always execute these types of operations in an asynchronous queue. Even if the performance seems good enough for your application when testing, you should still do it an asynch queue -- you never know what other processes the device might have going on which might slow the save down once your app is in the hands of users.
Newer versions of iOS make saving asynchronously really, really easy using Grand Central Dispatch (GCD). The steps are:
Create an NSBlockOperation which saves the image
In the block operation's completion block, read the image from disk & display it. The only caveat here is that you must use the main queue to display the image: all UI operations must occur on the main thread.
Add the block operation to an operation queue and watch it go!
That's it. And here's the code:
// Create a block operation with our saves
NSBlockOperation* saveOp = [NSBlockOperation blockOperationWithBlock: ^{
[UIImagePNGRepresentation(image) writeToFile:file atomically:YES];
[UIImagePNGRepresentation(thumbImage) writeToFile:thumbfile atomically:YES];
}];
// Use the completion block to update our UI from the main queue
[saveOp setCompletionBlock:^{
[[NSOperationQueue mainQueue] addOperationWithBlock:^{
UIImage *image = [UIImage imageWithContentsOfFile:thumbfile];
// TODO: Assign image to imageview
}];
}];
// Kick off the operation, sit back, and relax. Go answer some stackoverflow
// questions or something.
NSOperationQueue *queue = [[NSOperationQueue alloc] init];
[queue addOperation:saveOp];
Once you are comfortable with this code pattern, you will find yourself using it a lot. It's incredibly useful when generating large datasets, long operations on load, etc. Essentially, any operation that makes your UI laggy in the least is a good candidate for this code. Just remember, you can't do anything to the UI while you aren't in the main queue and everything else is cake.
Yes, it does take time on iPhone 4, where the image size is around 6 MB. The solution is to execute UIImagePNGRepresentation() in a background thread, using performSelectorInBackground:withObject:, so that your UI thread does not freeze.
It will probably be much faster to do the resizing before converting to PNG.
Try UIImageJPEGRepresentation with a medium compression quality. If the bottleneck is IO then this may prove faster as the filesize will generally be smaller than a png.
Use Instruments to check whether UIImagePNGRepresentation is the slow part or whether it is writing the data out to the filesystem which is slow.

CGImage, NSArray and Memory

This is a multiple part question, mostly because my ignorance on the matter has multiple layers.
First, I put together a caching system for caching CGImageRef objects. I keep it at the CGImageRef level (rather than UIImage) as I am loading images in background threads. When an image is loaded I put it into a NSMutableDictionary. I had to do a bit of arm twisting to get CGImageRef's into the array:
//Bunch of stuff drawing into a context
CGImageRef imageRef = CGBitmapContextCreateImage(context);
CGContextRelease(context);
[(id)imageRef autorelease];
[self.cache setObject:(id)imageRef forKey:#"SomeKey"];
So, as you can see, I'm trying to treat the Image Ref as an NSObject, setting it to autorelease then placing it in the dictionary. My expectation is this will allow the image to be cleaned up after being removed from the dictionary. Now, I am beginning to have my doubts.
My application clears the cache array when the user "restarts" to play with different images. Running the application in Instruments shows that the memory is not dropping back to the "start" level on restart, but instead remains steady. My gut tells me that when the array has all objects removed the CGImageRef is not being cleared.
However, I'm unable to confirm this as I don't quite know how to track down the actual source of the memory in instruments. It's just a list of (Malloc 16 Bytes, Malloc 32 Bytes, etc), drilling into them just show a list of dyld callers. Not sure how to properly read it.
So, first question, is my way of caching CGImageRef objects completely flawed? And is there a better way to confirm such things in instruments?
First of all, caching CGImages is OK and I don't see any problems with the code you posted.
Am I correctly assuming you use an NSMutableDictionary as the cache? If so, you can clear it by sending it -removeAllObjects, which should release all the keys and values. If you just set different images for the same keys, memory usage may remain roughly the same because you replace previous images with new ones. If the images have the same size, memory usage should be constant except brief spikes when you create a new batch of images.
As for Instruments, I've seen it both report false positives and miss real leaks. Try running it several times, making pauses, if possible, for the Leaks instrument to "catch up". This sounds crazy, but I think it may make it a bit more reliable.
If all else fails, you can log the contents of the cache before and after loading a set of images to make sure the cache itself works as expected.
Why not just cache UIImage objects; you can make them fine on a background thread?
It's UIImageView objects that you have to be more careful with and even they are OK for most operations in the background.