How to speed up saving a UIImagePickerController image from the camera to the filesystem via UIImagePNGRepresentation()? - objective-c

I'm making an applications that let users take a photo and show them both in thumbnail and photo viewer.
I have NSManagedObject class called photo and photo has a method that takes UIImage and converts it to PNG using UIImagePNGRepresentation() and saves it to filesystem.
After this operation, resize the image to thumbnail size and save it.
The problem here is UIImagePNGRepresentation() and conversion of image size seems to be really slow and I don't know if this is a right way to do it.
Tell me if anyone know the best way to accomplish what I want to do.
Thank you in advance.

Depending on the image resolution, UIImagePNGRepresentation can indeed be quite slow, as can any writing to the file system.
You should always execute these types of operations in an asynchronous queue. Even if the performance seems good enough for your application when testing, you should still do it an asynch queue -- you never know what other processes the device might have going on which might slow the save down once your app is in the hands of users.
Newer versions of iOS make saving asynchronously really, really easy using Grand Central Dispatch (GCD). The steps are:
Create an NSBlockOperation which saves the image
In the block operation's completion block, read the image from disk & display it. The only caveat here is that you must use the main queue to display the image: all UI operations must occur on the main thread.
Add the block operation to an operation queue and watch it go!
That's it. And here's the code:
// Create a block operation with our saves
NSBlockOperation* saveOp = [NSBlockOperation blockOperationWithBlock: ^{
[UIImagePNGRepresentation(image) writeToFile:file atomically:YES];
[UIImagePNGRepresentation(thumbImage) writeToFile:thumbfile atomically:YES];
}];
// Use the completion block to update our UI from the main queue
[saveOp setCompletionBlock:^{
[[NSOperationQueue mainQueue] addOperationWithBlock:^{
UIImage *image = [UIImage imageWithContentsOfFile:thumbfile];
// TODO: Assign image to imageview
}];
}];
// Kick off the operation, sit back, and relax. Go answer some stackoverflow
// questions or something.
NSOperationQueue *queue = [[NSOperationQueue alloc] init];
[queue addOperation:saveOp];
Once you are comfortable with this code pattern, you will find yourself using it a lot. It's incredibly useful when generating large datasets, long operations on load, etc. Essentially, any operation that makes your UI laggy in the least is a good candidate for this code. Just remember, you can't do anything to the UI while you aren't in the main queue and everything else is cake.

Yes, it does take time on iPhone 4, where the image size is around 6 MB. The solution is to execute UIImagePNGRepresentation() in a background thread, using performSelectorInBackground:withObject:, so that your UI thread does not freeze.

It will probably be much faster to do the resizing before converting to PNG.

Try UIImageJPEGRepresentation with a medium compression quality. If the bottleneck is IO then this may prove faster as the filesize will generally be smaller than a png.
Use Instruments to check whether UIImagePNGRepresentation is the slow part or whether it is writing the data out to the filesystem which is slow.

Related

UIImage to raw NSData / avoid compression

I have my own image downloader class, it holds a queue and downloads images one (or a certain amount) at a time, writes them to the cache folder and retrieves them from the cache folder when necessary. I also have a UIImageView subclass to which I can pass a URL, through the image downloader class it will look if the image already exists on the device and show it if it does, or download and show it after it finished.
After an image finishes downloading I do the following. I create a UIImage from the downloaded NSData, save the downloaded NSData to disk and return the UIImage.
// This is executed in a background thread
downloadedImage = [UIImage imageWithData:downloadedData];
BOOL saved = [fileManager createFileAtPath:filePath contents:downloadedData attributes:attributes];
// Send downloadedImage to the main thread and do something with it
To retrieve an existing image I do this.
// This is executed in a background thread
if ([fileManager fileExistsAtPath:filePath])
{
NSData* imageData = [fileManager contentsAtPath:filePath];
retrievedImage = [UIImage imageWithData:imageData];
// Send retrievedImage to the main thread and do something with it
}
As you can see, I always create a UIImage directly from the downloaded NSData, I never create NSData using UIImagePNGRepresentation so the image never gets compressed. When you create a UIImage from compressed NSData, UIImage will decompress it right before rendering on the main thread and thus block the UI. Since I'm now having a UITableView with a ton of small images in it that have to be downloaded or retrieved from disk, this would be unacceptable as it would slow down my scrolling immensely.
Now my problem. The user is also able to select a photo from the camera roll, save it and it also has to appear in my UITableView. But I can't seem to find a way to turn the UIImage from the camera roll into NSData without using UIImagePNGRepresentation. So here's my question.
How can I convert a UIImage into uncompressed NSData so I can convert it back to a UIImage later using imageWithData so that it doesn't have to be decompressed before rendering?
or
Is there any way I can do the decompression before sending the UIImage to the main thread and cache it so it only has to be decompressed once?
Thanks in advance.
How can I convert a UIImage into uncompressed NSData so I can convert it back to a UIImage later using imageWithData so that it doesn't have to be decompressed before rendering?
What you're really asking here, I take it, is how to store the UIImage on disk in such a way that you can later read the UIImage from disk as fast as possible. You don't really care whether it is stored as NSData; you just want to be able to read it quickly. I suggest you use the ImageIO framework. Save by way of an image destination and fetch later by way of an image source.
http://developer.apple.com/library/ios/#documentation/GraphicsImaging/Conceptual/ImageIOGuide/ikpg_dest/ikpg_dest.html
Is there any way I can do the decompression before sending the UIImage to the main thread and cache it so it only has to be decompressed once?
Yes, good question. That was going to be my second suggestion: use threading. This is what people have to do with tables all the time. When the table asks for the image, you either have the image already or you don't. If you don't, you supply a filler image and, in the background, fetch the real image. When the real image is ready, you have arranged to get a notification. Back on the main thread, you tell the table view to ask for the data for that row again; this time you've got the image and you supply it. The user will thus see a slight delay before the image appears. I'm sure you've seen lots of apps that behave this way (New York Times is a good example).
I have one further suggestion, and it may be the best of all. You speak of it taking time to decompress the image from disk. But this should take no time at all if the image is small. But the image should be small, because it's going to go into a small place - a table cell. In other words, you should shrink the images beforehand, when you first receive them, so that you are ready with the small version of each image when asked. It is a huge waste of time and memory to supply a large image that is to go into a small space.
ADDED LATER: Of course you do understand that a lot of this worry would be unnecessary if you weren't saving the images to disk. I'm not at all clear on why you need to do that. I hope you have a good reason for it; but it's a heck of a lot faster, obviously, if you just hold the images ready in memory.
I found solution:
CGImageRef downloadedImageRef = downloadedImage.CGImage;
CGDataProviderRef provider = CGImageGetDataProvider(downloadedImageRef);
NSData *data = CFBridgingRelease(CGDataProviderCopyData(provider));
// Then you can save the data
IF you download the data and save it to disk, then the data is compressed in either PNG, JPEG, or GIF format. You are not going to be downloading uncompressed image data. So, the root of your question about doing the decompression first needs to be addressed before you save the file to disk. Decompressing before you save will make the file a lot bigger, but it means that decompression is not needed before the data is read back into a CGImageRef or UIImage. It is the loading and then decompressing a bunch of images that is slowing down your CPU and making scrolling slow. But, it is not a solution to simply hold everything in memory already decompressed, because that will use up all your app memory and crash your phone before long. You might be able to get away with it for some small number of images, but this is a basic design flaw that you need to address when first writing your code. If you like, you can have a look at my blog post on this topic video-and-memory-usage-on-ios-devices, the post deals with video, but you have the exact same issue when dealing with lots of different images. I would suggest that you write your small images to disk in an uncompressed format like TIFF or BMP, that way reading them back in is easy as long as ImageIO supports that specific format.

switching views with grand central dispatch

I've looked at a lot of topics but I still can't figure it out.
I have a UITableview which downloads its content online. Each cell has an image, and I use GCD to let the image download. The downloaded image will be saved to disk, and before each time a cell is loaded there is checked if the file already exist, if not -> gcd, nsdata etc.
All goes well if someone has a good internet connection (wifi), but if I'm going to hop from View to View (back and forth), with my crappy 3G connection, what happens is that it wants to finish its queue (about 4 cells), but already gets assigned a new one, and a new one, and a new one and eventually the user has to wait a looong time before the others are executed (which he doesnt see) before the actual UITableview gets populated. With NSLog I can see that even I'm in a different view, it's still downloading and making uiimages that were visible on the screen. Each task is approximately 100 kb, and with a slow (or even no internet connection?!) it can take a while if you have a lot.
I know it's not possible to cancel it, but I read in other topics about using a BOOL variable but I don't really get it. Even if the BOOL variable change when the user leaves the screen, the cells are already in queue right?
Is it possible that when a user taps the back button in my Navigationcontroller, so he leaves the view, I change the data the blocks in queue use (empty it), so there is nothing to download and the blocks will be executed right away (there is nothing to do). So something like, making every value in array newsitems nil? Is it possible to change the datasource, or will the blocks that are waiting already have their datasource with them while waiting?
Then there is another problem, this doesn't have effect on the the currently executed block.
Can someone point me in a good direction?
Thank you.
Prastow
You can make use of NSBlockOperation and NSOperationQueue to create a cancellable download task. You create an NSBlockOperation by giving it a block which performs some work. In your case the block would download the contents of the URL.
In your view controller, you would store a list of the operations that have been submitted to the queue. If the user decides to leave the current view, you can then call cancel on each of the pending operations to prevent any needless work from taking place. The currently running operation will run to completion however. In order to cancel the currently running operation, you need to store a weak reference to the NSOperation object in the block doing teh work. Then at appropriate intervals within the body of the block, you can check to see if the operation has been cancelled and exit early.
// Create a queue on which to run the downloads
NSOperationQueue* queue = [NSOperationQueue new];
// Create an operation without any work to do
NSBlockOperation* downloadImageOperation = [NSBlockOperation new];
// Make a weak reference to the operation. This is used to check if the operation
// has been cancelled from within the block
__weak NSBlockOperation* operation = downloadImageOperation;
// The url from which to download the image
NSURL* imageURL = [NSURL URLWithString:#"http://www.someaddress.com/image.png"];
// Give the operation some work to do
[downloadImageOperation addExecutionBlock: ^() {
// Download the image
NSData* imageData = [NSData dataWithContentsOfURL:imageURL];
// Make sure the operation was not cancelled whilst the download was in progress
if (operation.isCancelled) {
return;
}
// Do something with the image
}];
// Schedule the download by adding the download operation to the queue
[queue addOperation:imageDownloadOperation];
// As necessary
// Cancel the operation if it is not already running
[imageDownloadOperation cancel];
A good talk on this exact topic was given at WWDC this year entitled "Building Concurrent User Interfaces on iOS". You can find the video and slides here
I faced similar issues with an app I developed a while back and found that the best way to do everything you require, and more, is to use https://github.com/MugunthKumar/MKNetworkKit
It took me the best part of a day to learn and understand the conversion and then a couple more days to tweak it to exactly what I needed.
If you do decide to use it or would like a thorough overview of the capabilities start here
http://blog.mugunthkumar.com/products/ios-framework-introducing-mknetworkkit/

Memory warning and crash (ARC) - how to identify why it's happening?

I've started to use the ARC recently and since then I blame it for every single memory problem. :) Perhaps, you could help me better understand what I'm doing wrong.
My current project is about CoreGraphics a lot - charts drawing, views filled with thumbnails and so on. I believe there would be no problem while using manual memory management, except maybe a few zombies... But as of now, application simply crashes every time I try to either create a lot of thumbnails or redraw a bit more complicated chart.
While profiling with Instruments I can see an awfully high value in resident memory as well as in the dirty one. Heap analysis shows rather alarming irregular grow...
When drawing just a few thumbnails, resident memory grows for about 200 MB. When everything is drawn, memory drops back on almost the same value as before drawing. However, with a lot of thumbnails, a value in resident memory is higher than 400 MB and that obviously crashes the app. I've tried to limit number of thumbnails drawn at the same time (NSOperationQueue and its maxConcurrentOperationCount), but as releasing so much memory seems to take a bit more time, it didn't really solve the issue.
Right now my app basically doesn't work as the real data works with a lot of complicated charts = lot of thumbnails.
Every thumbnail is created with this code I got from around here: (category on UIImage)
+ (void)beginImageContextWithSize:(CGSize)size
{
if ([[UIScreen mainScreen] respondsToSelector:#selector(scale)]) {
if ([[UIScreen mainScreen] scale] == 2.0) {
UIGraphicsBeginImageContextWithOptions(size, YES, 2.0);
} else {
UIGraphicsBeginImageContext(size);
}
} else {
UIGraphicsBeginImageContext(size);
}
}
+ (void)endImageContext
{
UIGraphicsEndImageContext();
}
+ (UIImage*)imageFromView:(UIView*)view
{
[self beginImageContextWithSize:[view bounds].size];
BOOL hidden = [view isHidden];
[view setHidden:NO];
[[view layer] renderInContext:UIGraphicsGetCurrentContext()];
UIImage *image = UIGraphicsGetImageFromCurrentImageContext();
[self endImageContext];
[view setHidden:hidden];
return image;
}
+ (UIImage*)imageFromView:(UIView*)view scaledToSize:(CGSize)newSize
{
UIImage *image = [self imageFromView:view];
if ([view bounds].size.width != newSize.width ||
[view bounds].size.height != newSize.height) {
image = [self imageWithImage:image scaledToSize:newSize];
}
return image;
}
+ (UIImage*)imageWithImage:(UIImage*)image scaledToSize:(CGSize)newSize
{
[self beginImageContextWithSize:newSize];
[image drawInRect:CGRectMake(0,0,newSize.width,newSize.height)];
UIImage *newImage = UIGraphicsGetImageFromCurrentImageContext();
[self endImageContext];
return newImage;
}
Is there some other way which wouldn't eat so much memory or is something really wrong with the code when using ARC?
The other place where memory warning + crash is happening is when there is too much redrawing of any view. It doesn't need to be quick, just many times. Memory stacks up until it crashes and I'm not able to find anything really responsible for it. (I can see a growing resident/dirty memory in VM Tracker and a heap growth in Allocations instrument)
My question basically is: how to find why it is even happening? My understanding is when there is no owner for given object, it's released ASAP. My inspection of code suggests a lot of objects are not released at all even though I don't see any reason for it to happen. I don't know about any retain cycles...
I've read through the Transitioning to ARC Release Notes, bbum's article about heap analysis and probably a dozen of others. Differs somehow heap analysis with and without ARC? I can't seem to do anything useful with its output.
Thank you for any ideas.
UPDATE: (to not force everybody read all the comments and to hold my promise)
By carefully getting through my code and adding #autoreleasepool, where it had any sense, memory consumption got lowered. The biggest problem was calling UIGraphicsBeginImageContext from background thread. After fixing it (see #Tammo Freese's answer for details) deallocation occurred soon enough to not crash an app.
My second crash (caused by many redrawing of the same chart), was completely solved by adding CGContextFlush(context) at the end of my drawing method. Shame on me.
A small warning for anyone trying to do something similar: use OpenGL. CoreGraphics is not quick enough for animating big drawings, especially not on an iPad 3. (first one with retina)
To answer your question: Identifying problems with memory warnings and crashes with ARC basically works like before with manual retain-release (MRR). ARC uses retain, release and autorelease just like MRR, it only inserts the calls for you, and has some optimizations in place that should even lower the memory consumption in some cases.
Regarding your problem:
In the screenshot of Instruments you posted, there are allocation spikes visible. In most cases I encountered so far, these spikes were caused by autoreleased objects hanging around too long.
You mentioned that you use NSOperationQueue. If you override -[NSOperationQueue main], make sure that you wrap the whole content of the method in #autoreleasepool { ... }. An autorelease pool may already be in place, but it is not guaranteed (and even if there is one, it may be around for longer than you think).
If 1. has not helped and you have a loop that processes the images, wrap the inner part of the loop in #autoreleasepool { ... } so that temporary objects are cleaned up immediately.
You mentioned that you use NSOperationQueue. Since iOS 4, drawing to a graphics context in UIKit is thread-safe, but if the documentation is right, UIGraphicsBeginImageContext should still only be called on the main thread! Update: The docs now state that since iOS 4, the function can be called from any thread, to the following is actually unnecessary! To be on the safe side, create the context with CGBitmapContextCreate and retrieve the image with CGBitmapContextCreateImage. Something along these lines:
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
CGContextRef context = CGBitmapContextCreate(NULL, width, height, 8, 0, colorSpace, kCGImageAlphaPremultipliedLast);
CGColorSpaceRelease(colorSpace);
// draw to the context here
CGImageRef newCGImage = CGBitmapContextCreateImage(context);
CGContextRelease(context);
UIImage *result = [UIImage imageWithCGImage:newCGImage scale:scale orientation: UIImageOrientationUp];
CGImageRelease(newCGImage);
return result;
So, nothing you are doing relative to memory management (there is none!) looks improper. However, you mention using NSOperationQueue. Those UIGraphics... calls are marked as not thread safe, but others have stated they are as of iOS 4 (I cannot find a definitive answer to this, but recall that this is true.
In any case, you should not be calling these class methods from multiple threads. You could create a serial dispatch queue and feed all the work through that to insure single threaded usage.
What's missing here of course is what you do with the images after using them. Its possible you are retaining them in some way that is not obvious. Here are some tricks:
in any of your classes that use lots of images, add a dealloc() method that just logs its name and some identifier.
you can try to add a dealloc to UIImage to do the same.
try to drive your app using the simplest possible setup - fewest images etc - so you can verify that in fact that the images and their owners are getting dealloc'd.
when you want to make sure something is released, set the ivar or property to nil
I converted a 100 file project to ARC last summer and it worked perfectly out of the box. I have converted several open source projects to ARC as well with only one problem when I improperly used bridging. The technology is rock solid.
This is not an answer to your question but I was trying to solve similar problems long before ARC was introduced.
Recently I was working on an application that was caching images in memory and releasing them all after receiving memory warning. This worked fine as long as I was using the application at a normal speed (no crazy tapping). But when I started to generate a lot of events and many images started to load, the application did not manage to get the memory warning and it was crashing.
I once wrote a test application that was creating many autoreleased objects after tapping a button. I was able to tap faster (and create objects) than the OS managed to release the memory. The memory was slowly increasing so after a significant time or simply using bigger objects I would surely crash the application and cause device to reboot (looks really effective ;)). I checked that using Instruments which unfortunately affected the test and made everything slower but I suppose this is true also when not using Instruments.
On the other occasion I was working on a bigger project that is quite complex and has a lot of UI created from code. It also has a lot of string processing and nobody cared to use release - there were few thousands of autorelease calls when I checked last time. So after 5 minutes of slightly extensive usage of this application, it was crashing and rebooting the device.
If I'm correct then the OS/logic that is responsible for actually deallocating memory is not fast enough or has not high enough priority to save an application from crashing when a lot of memory operations are performed. I never confirmed these suspicions and I don't know how to solve this problem other than simply reducing allocated memory.

CGImage, NSArray and Memory

This is a multiple part question, mostly because my ignorance on the matter has multiple layers.
First, I put together a caching system for caching CGImageRef objects. I keep it at the CGImageRef level (rather than UIImage) as I am loading images in background threads. When an image is loaded I put it into a NSMutableDictionary. I had to do a bit of arm twisting to get CGImageRef's into the array:
//Bunch of stuff drawing into a context
CGImageRef imageRef = CGBitmapContextCreateImage(context);
CGContextRelease(context);
[(id)imageRef autorelease];
[self.cache setObject:(id)imageRef forKey:#"SomeKey"];
So, as you can see, I'm trying to treat the Image Ref as an NSObject, setting it to autorelease then placing it in the dictionary. My expectation is this will allow the image to be cleaned up after being removed from the dictionary. Now, I am beginning to have my doubts.
My application clears the cache array when the user "restarts" to play with different images. Running the application in Instruments shows that the memory is not dropping back to the "start" level on restart, but instead remains steady. My gut tells me that when the array has all objects removed the CGImageRef is not being cleared.
However, I'm unable to confirm this as I don't quite know how to track down the actual source of the memory in instruments. It's just a list of (Malloc 16 Bytes, Malloc 32 Bytes, etc), drilling into them just show a list of dyld callers. Not sure how to properly read it.
So, first question, is my way of caching CGImageRef objects completely flawed? And is there a better way to confirm such things in instruments?
First of all, caching CGImages is OK and I don't see any problems with the code you posted.
Am I correctly assuming you use an NSMutableDictionary as the cache? If so, you can clear it by sending it -removeAllObjects, which should release all the keys and values. If you just set different images for the same keys, memory usage may remain roughly the same because you replace previous images with new ones. If the images have the same size, memory usage should be constant except brief spikes when you create a new batch of images.
As for Instruments, I've seen it both report false positives and miss real leaks. Try running it several times, making pauses, if possible, for the Leaks instrument to "catch up". This sounds crazy, but I think it may make it a bit more reliable.
If all else fails, you can log the contents of the cache before and after loading a set of images to make sure the cache itself works as expected.
Why not just cache UIImage objects; you can make them fine on a background thread?
It's UIImageView objects that you have to be more careful with and even they are OK for most operations in the background.

Significant lag when loading image using UIImage from URL asynchronously

I am trying to write an iPad app that loads an image from a URL. I am using the following image loading code:
url = [NSURL URLWithString:theURLString];
NSData *data = [NSData dataWithContentsOfURL:url];
img = [[UIImage alloc] initWithData:data];
[imageView setImage:img];
[img release];
NSLog(#"Image reloaded");
All of that code gets added to a NSOperationQueue as an operation so it will load asynchronously and not cause my app to lock up if the image's websever is slow. I added the NSLog line so I could see in the console when this code finished executing.
I have noticed consistently that the image is updated in my app about 5 seconds AFTER the code finishes executing. However if I use this code on it's own without putting it in the NSOperationQUeue it seems to update the image almost immediately.
The lag is not caused entirely by a slow web server... I can load the image URL in Safari and it takes less than a second to load, or I can load it with the same code without the NSOperationQueue and it loads much more quickly.
Is there any way to reduce the lag before my image is displayed but keep using a NSOperationQueue?
According to the documentation, the code you have written is invalid. UIKit objects may not be called anywhere but on the main thread. I'll bet that what you're doing happens to work in most respects but doesn't successfully alter the display, with the screen being updated by coincidence for some other reason.
Apple strongly recommend that threads are not the way to perform asynchronous URL fetches if you want to remain battery efficient. Instead you should be using NSURLConnection and allowing the runloop to organise asynchronous behaviour. It's not that hard to write a quick method that just accumulates data to an NSData as it comes then posts the whole thing on to a delegate when the connection is complete but assuming you'd rather stick with what you've got I'd recommend:
url = [NSURL URLWithString:theURLString];
NSData *data = [NSData dataWithContentsOfURL:url];
[self performSelectorOnMainThread:#selector(setImageViewImage:) withObject:data waitUntilDone:YES];
...
- (void)setImageViewImage:(NSData *)data
{
img = [[UIImage alloc] initWithData:data];
[imageView setImage:img];
[img release];
NSLog(#"Image reloaded");
}
performSelectorOnMainThread does what the name says — the object is sent to will schedule the selector requested with the object given as a single parameter on the main thread as soon as the run loop can get to it. In this case 'data' is an autoreleased object on the pool in the thread implicitly created by the NSOperation. Because you need it to remain valid until it has been used, I've used waitUntilDone:YES. An alternative would be to make data something that you explicitly own and have the main thread method release it.
The main disadvantage of this method is that if the image returns in a compressed form (such as a JPEG or a PNG), it'll be decompressed on the main thread. To avoid that without making empirical guesses about the behaviour of UIImage that go above and beyond what is documented to be safe, you'd need to drop to the C level and use CoreGraphics. But I'm taking it as given that doing so is beyond the scope of this question.
Tommy is correct about needing to do all UIKit stuff on the main thread. However, if you're running the fetch on a background operation queue, there's no need to use the NSURLConnection asynchronous loading. Also, by keeping the image decoding work on the background operation, you'll keep the main thread from blocking while decoding the image.
You should be able to use your original code as is, but just change [imgView setImage:img] to:
[imageView performSelectorOnMainThread:#selector(setImage:)
withObject:img
waitUntilDone:NO];