This is a multiple part question, mostly because my ignorance on the matter has multiple layers.
First, I put together a caching system for caching CGImageRef objects. I keep it at the CGImageRef level (rather than UIImage) as I am loading images in background threads. When an image is loaded I put it into a NSMutableDictionary. I had to do a bit of arm twisting to get CGImageRef's into the array:
//Bunch of stuff drawing into a context
CGImageRef imageRef = CGBitmapContextCreateImage(context);
CGContextRelease(context);
[(id)imageRef autorelease];
[self.cache setObject:(id)imageRef forKey:#"SomeKey"];
So, as you can see, I'm trying to treat the Image Ref as an NSObject, setting it to autorelease then placing it in the dictionary. My expectation is this will allow the image to be cleaned up after being removed from the dictionary. Now, I am beginning to have my doubts.
My application clears the cache array when the user "restarts" to play with different images. Running the application in Instruments shows that the memory is not dropping back to the "start" level on restart, but instead remains steady. My gut tells me that when the array has all objects removed the CGImageRef is not being cleared.
However, I'm unable to confirm this as I don't quite know how to track down the actual source of the memory in instruments. It's just a list of (Malloc 16 Bytes, Malloc 32 Bytes, etc), drilling into them just show a list of dyld callers. Not sure how to properly read it.
So, first question, is my way of caching CGImageRef objects completely flawed? And is there a better way to confirm such things in instruments?
First of all, caching CGImages is OK and I don't see any problems with the code you posted.
Am I correctly assuming you use an NSMutableDictionary as the cache? If so, you can clear it by sending it -removeAllObjects, which should release all the keys and values. If you just set different images for the same keys, memory usage may remain roughly the same because you replace previous images with new ones. If the images have the same size, memory usage should be constant except brief spikes when you create a new batch of images.
As for Instruments, I've seen it both report false positives and miss real leaks. Try running it several times, making pauses, if possible, for the Leaks instrument to "catch up". This sounds crazy, but I think it may make it a bit more reliable.
If all else fails, you can log the contents of the cache before and after loading a set of images to make sure the cache itself works as expected.
Why not just cache UIImage objects; you can make them fine on a background thread?
It's UIImageView objects that you have to be more careful with and even they are OK for most operations in the background.
Related
My app utilises tableviews that all have associated UIImages.
I show my UIImages in the app using:
cell.foodImage.image = UIImage(named: foodImageArray[indexPath.row] + ".jpg")
I know in objective c something such as
UIImage initWithContentsOfFile
Could be used in order to keep the memory footprint low.
Is there something like this I can use in swift?
UIImage(named:) caches once loaded image and keeps it in run-time until app quits or 'out of memory' notification from system, on which cache somehow cleaned up automatically.
Thus it is better to use this constructor only for small images, which are either always visible in UI or used often.
For other cases (probably your table view case is here) it is better to use UIImage(contentsOfFile:) constructor with implemented some in-app cache logic to keep it for some workflow only but clean when go out it (other mode, scenario, etc. including force cleaning for 'out of memory' notifications)
My class is rendering images offscreen. I thought reusing the CGContext instead of creating the same context again and again for every image would be a good thing. I set a member variable _imageContext so I would only have to create a new context if _imageContext is nil like so:
if(!_imageContext)
_imageContext = [self contextOfSize:imageSize];
instead of:
CGContextRef imageContext = [self contextOfSize:imageSize];
Of course I do not release the CGContext anymore.
These are the only changes I made, turns out that reusing the context slowed down rendering from about 10ms to 60ms. Have I missed something? Do I have to clear the context or something before drawing into it again? Or is it the correct way to recreate the context for each image?
EDIT
Found the weirdest connection..
While I was searching for the reason why the app's memory is incredibly increasing when the app starts rendering the images, I found the problem was where I set the rendered image to an NSImageView.
imageView.image = nil;
imageView.image = [[NSImage alloc] initWithCGImage:_imageRef size:size];
It looks like ARC is not releasing the previous NSImage. First way to avoid that was to draw the new image into the old one.
[imageView.image lockFocus];
[[[NSImage alloc] initWithCGImage:_imageRef size:size] drawInRect:NSMakeRect(0, 0, size.width, size.height) fromRect:NSZeroRect operation:NSCompositeSourceOver fraction:1.0];
[imageView.image unlockFocus];
[imageView setNeedsDisplay];
The memory problem was gone and what happened to the CGContext-reuse problem?
Not reusing the context now takes 20ms instead of 10ms - of course drawing into an image takes longer than just setting it.
Reusing the context also takes 20ms instead of 60ms. But why? I don't see that there could be any connection, but I can reproduce the old state where reusing takes more time just by setting the NSImageView's image instead of drawing it.
I investigated this, and I observe the same slowdown. Looking with Instruments set to sample kernel calls as well as userland calls shows the culprit. #RyanArtecona's comment was on the right track. I focused Instruments in on the bottom most userland call CGSColorMaskCopyARGB8888_sse in two test runs (one reusing contexts, the other making a new one every time), and then inverted the resulting call tree. In the case where the context is not reused, I see that the heaviest kernel trace is:
Running Time Self Symbol Name
668.0ms 32.3% 668.0 __bzero
668.0ms 32.3% 0.0 vm_fault
668.0ms 32.3% 0.0 user_trap
668.0ms 32.3% 0.0 CGSColorMaskCopyARGB8888_sse
This is the kernel zeroing out pages of memory that are being faulted in by virtue of CGSColorMaskCopyARGB8888_sse accessing them. What this means is that the CGContext maps VM pages to back the bitmap context but the kernel doesn't actually do the work associated with that operation until someone actually accesses that memory. The actual mapping/fault happens on first access.
Now let's look at the heaviest kernel trace when we DO reuse the context:
Running Time Self Symbol Name
1327.0ms 35.0% 1327.0 bcopy
1327.0ms 35.0% 0.0 user_trap
1327.0ms 35.0% 0.0 CGSColorMaskCopyARGB8888_sse
This is the kernel copying pages. My money would be on this being the underlying copy-on-write mechanism that delivers the behavior #RyanArtecona was talking about in his comment:
In the Apple docs for CGBitmapContextCreateImage, it says the actual
bit-copying operation doesn't happen until more drawing is done on the
original context.
In the contrived case I used to test, the non-reuse case took 3392ms to execute and the reuse case took 4693ms (significantly slower). Considering just the single heaviest trace from each case, the kernel trace indicates that we spend 668.0ms zero filling new pages on the first access, and 1327.0ms writing into the copy-on-write pages on the first write after the image gets a reference to those pages. This is a difference of 659ms. This one difference alone accounts for ~50% of the gap between the two cases.
So, to distill it down a little, the non-reused context is faster because when you create the context it knows the pages are empty, and there's no one else with a reference to those pages to force them to be copied when you write to them. When you reuse the context, the pages are referenced by someone else (the image you created) and must be copied on the first write, so as to preserve the state of the image when the state of the context changes.
You could further explore what's going on here by looking at the virtual memory map of the process as you step through in the debugger. vmmap is the helpful tool for that.
Practically speaking, you should probably just create a new CGContext every time.
To complement #ipmcc's excellent and thorough answer, here is an instructional overview.
In the Apple docs for CGBitmapContextCreateImage it is stated:
The CGImage object returned by this function is created by a copy
operation. In some cases the copy
operation actually follows copy-on-write semantics, so that the actual
physical copy of the bits occur only if the underlying data in the
bitmap graphics context is modified.
So, when this function is called, the image's underlying bits may not be copied right away, and may instead wait to be copied when the bitmap context is next modified. This bit-copying may be expensive (depending on the size and colorspace of the context), and may disguise itself in an Instruments profile as part of whatever CGContext... drawing function that gets called next on the context (when the bits are forced to copy). This is probably what is happening here with CGContextDrawImage.k
However, the docs go on to say this:
As a consequence, you may want to use the resulting image and release
it before you perform additional drawing into the bitmap graphics
context. In this way, you can avoid the actual physical copy of the
data.
This implies that if you will be finished using the in-memory created image (i.e. it has been saved to disk, sent over the network, etc.) by the time you need to do more drawing in the context, the image would never need to be physically copied at all!
TL;DR
If at some point you need to pull a CGImage out of a bitmap context, and you won't need to keep any references to it (including setting it as a UIImageView's image) before you do any more drawing in the context, then it is probably a good idea to use CGBitmapContextCreateImage. If not, your image will be physically copied at some point, which may take a while, and it may be better to just use a new context each time.
I've started to use the ARC recently and since then I blame it for every single memory problem. :) Perhaps, you could help me better understand what I'm doing wrong.
My current project is about CoreGraphics a lot - charts drawing, views filled with thumbnails and so on. I believe there would be no problem while using manual memory management, except maybe a few zombies... But as of now, application simply crashes every time I try to either create a lot of thumbnails or redraw a bit more complicated chart.
While profiling with Instruments I can see an awfully high value in resident memory as well as in the dirty one. Heap analysis shows rather alarming irregular grow...
When drawing just a few thumbnails, resident memory grows for about 200 MB. When everything is drawn, memory drops back on almost the same value as before drawing. However, with a lot of thumbnails, a value in resident memory is higher than 400 MB and that obviously crashes the app. I've tried to limit number of thumbnails drawn at the same time (NSOperationQueue and its maxConcurrentOperationCount), but as releasing so much memory seems to take a bit more time, it didn't really solve the issue.
Right now my app basically doesn't work as the real data works with a lot of complicated charts = lot of thumbnails.
Every thumbnail is created with this code I got from around here: (category on UIImage)
+ (void)beginImageContextWithSize:(CGSize)size
{
if ([[UIScreen mainScreen] respondsToSelector:#selector(scale)]) {
if ([[UIScreen mainScreen] scale] == 2.0) {
UIGraphicsBeginImageContextWithOptions(size, YES, 2.0);
} else {
UIGraphicsBeginImageContext(size);
}
} else {
UIGraphicsBeginImageContext(size);
}
}
+ (void)endImageContext
{
UIGraphicsEndImageContext();
}
+ (UIImage*)imageFromView:(UIView*)view
{
[self beginImageContextWithSize:[view bounds].size];
BOOL hidden = [view isHidden];
[view setHidden:NO];
[[view layer] renderInContext:UIGraphicsGetCurrentContext()];
UIImage *image = UIGraphicsGetImageFromCurrentImageContext();
[self endImageContext];
[view setHidden:hidden];
return image;
}
+ (UIImage*)imageFromView:(UIView*)view scaledToSize:(CGSize)newSize
{
UIImage *image = [self imageFromView:view];
if ([view bounds].size.width != newSize.width ||
[view bounds].size.height != newSize.height) {
image = [self imageWithImage:image scaledToSize:newSize];
}
return image;
}
+ (UIImage*)imageWithImage:(UIImage*)image scaledToSize:(CGSize)newSize
{
[self beginImageContextWithSize:newSize];
[image drawInRect:CGRectMake(0,0,newSize.width,newSize.height)];
UIImage *newImage = UIGraphicsGetImageFromCurrentImageContext();
[self endImageContext];
return newImage;
}
Is there some other way which wouldn't eat so much memory or is something really wrong with the code when using ARC?
The other place where memory warning + crash is happening is when there is too much redrawing of any view. It doesn't need to be quick, just many times. Memory stacks up until it crashes and I'm not able to find anything really responsible for it. (I can see a growing resident/dirty memory in VM Tracker and a heap growth in Allocations instrument)
My question basically is: how to find why it is even happening? My understanding is when there is no owner for given object, it's released ASAP. My inspection of code suggests a lot of objects are not released at all even though I don't see any reason for it to happen. I don't know about any retain cycles...
I've read through the Transitioning to ARC Release Notes, bbum's article about heap analysis and probably a dozen of others. Differs somehow heap analysis with and without ARC? I can't seem to do anything useful with its output.
Thank you for any ideas.
UPDATE: (to not force everybody read all the comments and to hold my promise)
By carefully getting through my code and adding #autoreleasepool, where it had any sense, memory consumption got lowered. The biggest problem was calling UIGraphicsBeginImageContext from background thread. After fixing it (see #Tammo Freese's answer for details) deallocation occurred soon enough to not crash an app.
My second crash (caused by many redrawing of the same chart), was completely solved by adding CGContextFlush(context) at the end of my drawing method. Shame on me.
A small warning for anyone trying to do something similar: use OpenGL. CoreGraphics is not quick enough for animating big drawings, especially not on an iPad 3. (first one with retina)
To answer your question: Identifying problems with memory warnings and crashes with ARC basically works like before with manual retain-release (MRR). ARC uses retain, release and autorelease just like MRR, it only inserts the calls for you, and has some optimizations in place that should even lower the memory consumption in some cases.
Regarding your problem:
In the screenshot of Instruments you posted, there are allocation spikes visible. In most cases I encountered so far, these spikes were caused by autoreleased objects hanging around too long.
You mentioned that you use NSOperationQueue. If you override -[NSOperationQueue main], make sure that you wrap the whole content of the method in #autoreleasepool { ... }. An autorelease pool may already be in place, but it is not guaranteed (and even if there is one, it may be around for longer than you think).
If 1. has not helped and you have a loop that processes the images, wrap the inner part of the loop in #autoreleasepool { ... } so that temporary objects are cleaned up immediately.
You mentioned that you use NSOperationQueue. Since iOS 4, drawing to a graphics context in UIKit is thread-safe, but if the documentation is right, UIGraphicsBeginImageContext should still only be called on the main thread! Update: The docs now state that since iOS 4, the function can be called from any thread, to the following is actually unnecessary! To be on the safe side, create the context with CGBitmapContextCreate and retrieve the image with CGBitmapContextCreateImage. Something along these lines:
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
CGContextRef context = CGBitmapContextCreate(NULL, width, height, 8, 0, colorSpace, kCGImageAlphaPremultipliedLast);
CGColorSpaceRelease(colorSpace);
// draw to the context here
CGImageRef newCGImage = CGBitmapContextCreateImage(context);
CGContextRelease(context);
UIImage *result = [UIImage imageWithCGImage:newCGImage scale:scale orientation: UIImageOrientationUp];
CGImageRelease(newCGImage);
return result;
So, nothing you are doing relative to memory management (there is none!) looks improper. However, you mention using NSOperationQueue. Those UIGraphics... calls are marked as not thread safe, but others have stated they are as of iOS 4 (I cannot find a definitive answer to this, but recall that this is true.
In any case, you should not be calling these class methods from multiple threads. You could create a serial dispatch queue and feed all the work through that to insure single threaded usage.
What's missing here of course is what you do with the images after using them. Its possible you are retaining them in some way that is not obvious. Here are some tricks:
in any of your classes that use lots of images, add a dealloc() method that just logs its name and some identifier.
you can try to add a dealloc to UIImage to do the same.
try to drive your app using the simplest possible setup - fewest images etc - so you can verify that in fact that the images and their owners are getting dealloc'd.
when you want to make sure something is released, set the ivar or property to nil
I converted a 100 file project to ARC last summer and it worked perfectly out of the box. I have converted several open source projects to ARC as well with only one problem when I improperly used bridging. The technology is rock solid.
This is not an answer to your question but I was trying to solve similar problems long before ARC was introduced.
Recently I was working on an application that was caching images in memory and releasing them all after receiving memory warning. This worked fine as long as I was using the application at a normal speed (no crazy tapping). But when I started to generate a lot of events and many images started to load, the application did not manage to get the memory warning and it was crashing.
I once wrote a test application that was creating many autoreleased objects after tapping a button. I was able to tap faster (and create objects) than the OS managed to release the memory. The memory was slowly increasing so after a significant time or simply using bigger objects I would surely crash the application and cause device to reboot (looks really effective ;)). I checked that using Instruments which unfortunately affected the test and made everything slower but I suppose this is true also when not using Instruments.
On the other occasion I was working on a bigger project that is quite complex and has a lot of UI created from code. It also has a lot of string processing and nobody cared to use release - there were few thousands of autorelease calls when I checked last time. So after 5 minutes of slightly extensive usage of this application, it was crashing and rebooting the device.
If I'm correct then the OS/logic that is responsible for actually deallocating memory is not fast enough or has not high enough priority to save an application from crashing when a lot of memory operations are performed. I never confirmed these suspicions and I don't know how to solve this problem other than simply reducing allocated memory.
I'm working on a kiosk style slideshow app. I have a UIScrollView which shows the slides, and a factory class, which generates the slides. The "slides" themselves are UIViewController subclasses, which are loaded out from XIB files and customized by the factory class. In my main view controller, I set up the scroll view and start a timer. The timer calls a "reload" method every N seconds, which handles the reload and call to the factory class.
The method that the factory class uses looks something like this:
- (SlideViewController *)slideFromManagedObject:(Slide *)managedObject{
NSInteger slideType = [managedObject slideType];
switch(slideType){
case kSlideTypeA:
{
//
// configure arguments here
//
return [[SlideViewController alloc] initWithArgument:argument] autorelease];
break;
}
//
// More types here...
//
default:
break;
}
}
I haven't yet gotten to the point of defining all of my cases, but the ones that are filled out seem to cause jumps in memory usage. If I add return [[[UIViewController alloc] init] autorelease]; right before the switch/case, I get no visible view, as expected, but I also don't see those memory increases. I'm not sure, but I suspect that it's the "C blocks" that I'm wrapping my slide generation code in.
Some things to note:
When the app starts, I see the memory plateau from about 400 kilobytes to around double that. Then, when the slides progress, any of the slides whose generation code is contained in curly braces is called, the memory plateaus upwards again.
This behavior only seems to happen once per launch - when the app loops through all of the slides, the plateaus to_not_ happen again. However if the app is backgrounded and then relaunched, the plateaus do occur again, consuming even more memory.
When I left the app to run overnight, for about 10 hours and forty minutes, the memory usage had slowly climbed from about 1.44 megabytes to somewhere closer to 1.57 megabytes. I suspect that there are/were some other leaks in there that may have been fixed by my tweaking, but the main jump from about 800 kilobytes to somewhere between 1.4 and 1.5 megabytes is still an issue.
Instruments does not report any leaks, but the plateauing concerns me.
What could be causing the increased memory?
EDIT:
So I don't think it's the blocks, since using an if/else seems to do the same thing.
Here's a screenshot of the Allocations instrument running:
Where could possibly be holding on to these views?
One possible explanation for what you are seeing is some caching that UIKit (I assume) is doing of your objects (don't know what they are, but I think of images mostly).
Caching is often used during transitions and for other internalities of UIKit.
UIKit empties its caches usually when a memory warning is received, so you could try and send one to see what happens. In actuality, I suspect that results of sending a memory warning will not be very easy to analyze, since all of your views are also unloaded, hence memory will go down forcibly. But you can try...
As to how sending a memory warning to the device (as opposed to the simulator), here you find an useful S.O. post.
When I use UIImage imagenamed: should I set the variables which hold UIImages to nil before exit? I notice that sometimes when I switch between views which have UIImages, the memory keeps growing and growing with each switch back and forth.
Setting variables to nil is not necessary.
Setting properties to nil (self.property = nil;) will release them if they're declared #property (retain).
Since +imageNamed: doesn't start with "alloc", "copy", "new", or "retain", you don't have to release it. It's possible things are staying in memory because the space isn't needed. Are you seeing any leaks or just memory usage?
Setting UIImage variables to nil won't do anything particularly useful. Also, you shouldn't release the image returned from +imageNamed:, since the method name does not imply that you have ownership of the returned object.
Cocoa maintains an image cache. Subsequent calls to imageNamed: will return the same UIImage object if it is loaded already (since UIImage objects are immutable), otherwise it will load it again into the cache. The lifetime of the image in the cache is up to Cocoa to decide. In low-memory situations the image data may be purged. Even though the actual internal image data is purged from the cache, the object that you have is still able to reference the image (Cocoa will reload the image data if it was purged from the cache). It is explained throughout the UIImage documentation.
If your memory usage is consistently growing then the leak is probably coming from somewhere else.