Memory warning after downloading and unzip files with ARC - objective-c

Im downloading some gzipped xml files from a server, save it to the documents folder and unzip every file. After that I delete the .gz file. I do that in a loop for more or less 500 files. When im using instruments, I see that the live bytes more or less are 470MB after this process. When Im waiting some seconds, the ARC clears it and the application is going to 5mb live bytes. But because its a synchronization process my app gets a memory warning right after that when I dont stop after the downloading and unzipping. At least I think it should be possible to force the ARC to release the memory? Or do I have a real bad code and I am just still dont see that?
Any help or hint is really appreciated.
Downloading and unzipping:
for(NSString *filePath in filePaths){
NSString *localPath = [[DownloadManager sharedInstance] downloadFile:filePath];
if(localPath){
//downloaded correctly
if([self unzipFileAtPath:localPath]){
[FileUtility deleteFileAtPath:localPath];
}
}
}
Unzip method:
+ (BOOL)unzipFileAtPath:(NSString *)path
{
NSData *gzData = [NSData dataWithContentsOfFile:path];
NSData *ungzippedData = [gzData gunzippedData];
BOOL success = [ungzippedData writeToFile:[FormatUtility pathWithoutGz:path] atomically:NO];
ungzippedData = nil;
gzData = nil;
return success;
}

Wrap the inside of your for-loop with an autorelease pool:
for (NSString* filePath in filePaths) {
#autoreleasepool {
// do work
}
}
The problem actually has nothing to do with ARC. Methods like dataWithContentsOfFile: will return a new autoreleased object instance. These objects will not be released until the enclosing autorelease pool is drained, which by default only happens at the end of your thread/operation or when you return to the run-loop.
When you allocate many temporary objects in a loop, like you're doing, you should use your own autorelease pool to ensure that these temporary objects do not accumulate needlessly.

I am facing a similar problem when downloading a very large zip file (of 5GB!) using AFNetworking 2.6.4 and AFDownloadRequestOperation 2.0.1. I find the such memory issue is caused by the Flipboard FLEX 2.1.1 framework. After I commented out the line [[FLEXManager sharedManager] setNetworkDebuggingEnabled:YES]; it works perfectly fine.

Related

OS X faster file system API than repetitively calling [NSFileManager attributesOfItemAtPath...]?

Is there a faster file system API that I can use if I only need to know if a file is a folder/symlink and its size. I'm currently using [NSFileManager attributesOfItemAtPath...] and only NSFileSize and NSFileType.
Are there any bulk filesystem enumeration APIs I should be using? I suspect this could be faster without having to jump in and out of user code.
My goal is to quickly recurse through directories to get a folders true file size and currently calling attributesOfItemAtPath is my 95% bottleneck.
Some of the code I'm currently using:
NSDictionary* properties = [fileManager attributesOfItemAtPath:filePath error:&error];
long long fileSize = [[properties objectForKey:NSFileSize] longLongValue];
NSObject* fileType = [[properties objectForKey:NSFileType isEqual:NSFileTypeDirectory];
If you want to get really hairy, the Mac OS kernel implements a unique getdirentriesattr() system call which will return a list of files and attributes from a specified directory. It's messy to set up and call, and it's not supported on all filesystems (!), but if you can get it to work for you, it can speed things up significantly.
There's also a closely related searchfs() system call which can be used to rapidly search for files. It's subject to most of the same gotchas.
You can use stat and lstat. Take a look at this answer for calculating directory size.
CPU raises with attributesOfItemAtPath:error:
Whether it's faster or not I'm not certain, but NSURL will give you this information via getResourceValue:forKey:error:
NSError * e;
NSNumber * isDirectory;
BOOL success = [URLToFile getResourceValue:&isDirectory
forKey:NSURLIsDirectoryKey
error:&e];
if( !success ){
// error
}
NSNumber * fileSize;
BOOL success = [URLToFile getResourceValue:&fileSize
forKey:NSURLFileSizeKey
error:&e];
You might also find it convenient to wrap this up if you don't really care about the error:
#implementation NSURL (WSSSimpleResourceValueRetrieval)
- (id)WSSResourceValueForKey: (NSString *)key
{
id value = nil;
BOOL success = [self getResourceValue:&value
forKey:key
error:nil];
if( !success ){
value = nil;
}
return value;
}
#end
This is given as the substitute for the deprecated File Manager function FSGetCatalogInfo(), which is used in a solution in an old Cocoa-dev thread that Dave DeLong gives the thumbs up to.
For the enumeration part, the File System Programming Guide has a section "Getting the Contents of a Directory in a Single Batch Operation", which discusses using contentsOfDirectoryAtURL:includingPropertiesForKeys:options:error:

Can't release componentsSeparetedByString array

I have a simple method run in background thread which open txt file and split it on lines. After that I'm trying to release memory, but something goes wrong. I'm using ARC. Here's code:
#autoreleasepool {
NSString* file = [NSString stringWithContentsOfFile:resourcePath encoding:NSWindowsCP1251StringEncoding error:&error];
NSArray* test = [file componentsSeparatedByString:#"\n"];
test = nil;
}
String released fine, but array still in memory. What I've missed?
UPD: Hm... Just tried to duplicate array few times, and after end of the method array really deallocates. But there is memory leak if I create this array. Where it could be?
// test = nil;
Dismiss it, and ARC will work fine.

Correct way to multithread in objective-c?

I have a UITableView which displays images. Every cell has an image and every time a cell loads, I call a selector (from the cellForRowAtIndexPath) in the background like this:
[self performSelectorInBackground:#selector(lazyLoad:) withObject:aArrayOfData];
The only problem is that sometimes I get a crash (because I am changing data in the background while it's trying to be read elsewhere). Here's the error:
*** Terminating app due to uncaught exception 'NSGenericException', reason: '*** Collection <CALayerArray: 0xce1a920> was mutated while being enumerated.'
When updating the data in the background, should I move it to the main selector and change it? Or should I call the #selector() differently?
Thanks!
If you can leave the operation on the main thread and have no lagginess nor problems you are done.
However: Let's assume you've already done that and encounter problems. The answer is: don't modify the array in the lazy load. Switch to the main thread to modify the array. See Brad's answer here:
https://stackoverflow.com/a/8186206/8047
for a way to do it with blocks, so you can send your objects over to the main queue (you should probably also use GCD for the call to the lazy load in the first place, but it's not necessary).
You can use #synchronized blocks to keep the threads from walking over each other. If you do
#synchronized(array)
{
id item = [array objectAtIndex:row];
}
in the main thread and
#synchronized(array)
{
[array addObject:item];
}
in the background, you're guaranteed they won't happen at the same time. (Hopefully you can extrapolate from that to your code—I'm not sure what all you're doing with the array there..)
It seems, though, like you'd have to notify the main thread anyway that you've loaded the data for a cell (via performSelectorOnMainThread:withObject:waitUntilDone:, say), so why not pass the data along, too?
Given the term 'lazy load' I am assuming that means you are pulling your images down from a server. (If the images are local then there is really no need for multithreading).
If you are downloading images off a server I would suggest using something along these lines (using ASIHTTPRequest)
static NSCache *cellCache; //Create a Static cache
if (!cellCache)//If the cache is not initialized initialize it
{
cellCache = [[NSCache alloc] init];
}
NSString *key = imageURL;
//Look in the cache for image matching this url
NSData *imageData = [cellCache objectForKey:key];
if (!imageData)
{
//Set a default image while it's loading
cell.icon.image = [UIImage imageNamed:#"defaultImage.png"];'
//Create an async request to the server to get the image
__unsafe_unretained ASIHTTPRequest *request = [ASIHTTPRequest requestWithURL:[NSURL URLWithString:imageURL]];
//This code will run when the request finishes
[request setCompletionBlock:^{
//Put downloaded image into the cache
[cellCache setObject:[request responseData] forKey:key];
//Display image
cell.icon.image = [UIImage imageWithData:[request responseData]];
}];
[request startAsynchronous];
}
else
{
//Image was found in the cache no need to redownload
cell.icon.image = [UIImage imageWithData:imageData];
}

NSBundle pathForResource failing in shell tool

I've noticed some weird behavior with NSBundle when using it in a
command-line program. If, in my program, I take an existing bundle and
make a copy of it and then try to use pathForResource to look up
something in the Resources folder, nil is always returned unless the
bundle I'm looking up existed before my program started. I created a
sample app that replicates the issue and the relevant code is:
int main(int argc, char *argv[])
{
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
NSString *exePath = [NSString stringWithCString:argv[0]
encoding:NSASCIIStringEncoding];
NSString *path = [exePath stringByDeletingLastPathComponent];
NSString *templatePath = [path stringByAppendingPathComponent:#"TestApp.app"];
// This call works because TestApp.app exists before this program is run
NSString *resourcePath = [NSBundle pathForResource:#"InfoPlist"
ofType:#"strings"
inDirectory:templatePath];
NSLog(#"NOCOPY: %#", resourcePath);
NSString *copyPath = [path stringByAppendingPathComponent:#"TestAppCopy.app"];
[[NSFileManager defaultManager] removeItemAtPath:copyPath
error:nil];
if ([[NSFileManager defaultManager] copyItemAtPath:templatePath
toPath:copyPath
error:nil])
{
// This call will fail if TestAppCopy.app does not exist before
// this program is run
NSString *resourcePath2 = [NSBundle pathForResource:#"InfoPlist"
ofType:#"strings"
inDirectory:copyPath];
NSLog(#"COPY: %#", resourcePath2);
[[NSFileManager defaultManager] removeItemAtPath:copyPath
error:nil];
}
[pool release];
}
For the purpose of this test app, let's assume that TestApp.app
already exists in the same directory as my test app. If I run this,
the 2nd NSLog call will output: COPY: (null)
Now, if I comment out the final removeItemAtPath call in the if
statement so that when my program exits TestAppCopy.app still exists
and then re-run, the program will work as expected.
I've tried this in a normal Cocoa application and I can't reproduce
the behavior. It only happens in a shell tool target.
Can anyone think of a reason why this is failing?
BTW: I'm trying this on 10.6.4 and I haven't tried on any other
versions of Mac OS X.
I can confirm that it is a bug in CoreFoundation, not Foundation. The bug is due to CFBundle code relying on a directory contents cache containing stale data. The code apparently assumes that neither the bundle directories nor their immediate parent directories will change during application runtime.
The CoreFoundation call corresponding to +[NSBundle pathForResource:ofType:inDirectory:] is CFBundleCopyResourceURLInDirectory(), and it exhibits the same misbehavior. (This is unsurprising, as -pathForResource:ofType:inDirectory: itself uses this call.)
The problem ultimately lies with _CFBundleCopyDirectoryContentsAtPath(). This is called during bundle loading and during all resource lookup. It caches information about the directories it looks up in contentsCache.
Here's the problem: When it comes time to get the contents of TestAppCopy.app, the cached contents of the directory containing TestApp.app don't include TestAppCopy.app. Because the cache ostensibly has the contents of that directory, only the cached contents are searched for TestAppCopy.app. When TestAppCopy.app is not found, the function takes that as a definitive "this path does not exist" and doesn't bother trying to open the directory:
__CFSpinLock(&CFBundleResourceGlobalDataLock);
if (contentsCache) dirDirContents = (CFArrayRef)CFDictionaryGetValue(contentsCache, dirName);
if (dirDirContents) {
Boolean foundIt = false;
CFIndex dirDirIdx, dirDirLength = CFArrayGetCount(dirDirContents);
for (dirDirIdx = 0; !foundIt && dirDirIdx < dirDirLength; dirDirIdx++) if (kCFCompareEqualTo == CFStringCompare(name, CFArrayGetValueAtIndex(dirDirContents, dirDirIdx), kCFCompareCaseInsensitive)) foundIt = true;
if (!foundIt) tryToOpen = false;
}
__CFSpinUnlock(&CFBundleResourceGlobalDataLock);
So, the contents array remains empty, gets cached for this path, and lookup continues. We now have cached the (incorrectly empty) contents of TestAppCopy.app, and as lookup drills down into this directory, we keep hitting bad cached information. Language lookup takes a stab when it finds nothing and hopes there's an en.lproj hanging around, but we still won't find anything, because we're looking in a stale cache.
CoreFoundation includes SPI functions to flush the CFBundle caches. The only place public API calls into them in CoreFoundation is __CFBundleDeallocate(). This flushes all cached information about the bundle's directory itself, but not its parent directory: _CFBundleFlushContentsCacheForPath(), which actually removes the data from the cache, removes only keys matching an anchored, case-insensitive search for the bundle path.
It would seem the only public way a client of CoreFoundation could flush bad information about TestApp.app's parent directory would be to make the parent directory a bundle directory (so TestApp.app lived alongside Contents), create a CFBundle for the parent bundle directory, then release that CFBundle. But, it seems that if you made the mistake of trying to work with the TestAppCopy.app bundle prior to flushing it, the bad data about TestAppCopy.app would not be flushed.
That sounds like a bug in the Foundation. The one key difference between a command line tool like that one and a Cocoa application is the run loop. Try refactoring the above into something like:
#interface Foo:NSObject
#end
#implementation Foo
- (void) doIt { .... your code from main() here .... }
#end
... main(...) {
Foo *f = [Foo new];
[f performSelector: #selector(doIt) withObject: nil afterDelay: 0.1 ...];
[[NSRunLoop currentRunLoop] run];
return 0; // not reached, I'd bet.
}
And see if that "fixes" it. It might. It might not (there are couple of other significant differences, obviously). In any case, do please file a bug via http://bugreport.apple.com/ and add the bug # as a comment.

Objc memory crash with autorelase

I have been hunting all over my code and can't find the source of this crash:
I am trying to decode an object with an NSKeyedUnarchiver and it crashes on it every time and says:
*** __NSAutoreleaseFreedObject(): release of previously deallocated object (0x1008ad200) ignored
*** __NSAutoreleaseFreedObject(): release of previously deallocated object (0x1008ab200) ignored
*** __NSAutoreleaseFreedObject(): release of previously deallocated object (0x1008a8c00) ignored
Ha my bad the reason why initWithCoder was not getting called was it was having issues with the [super initWithCoder:]; This is still driving me crazy. I looked and the pointers and the NSData objects are what are going wrong:
vertices = malloc(size_point3D * vertexCount);
textureCoords = malloc(size_point2D * textureCount);
normals = malloc(size_point3D * normalCount);
faces = malloc(sizeof(GLuint) * faceCount);
NSData *vertexData = [[NSData alloc] initWithData:[coder decodeObjectForKey:#"vertices"]];
NSData *textureData = [[NSData alloc] initWithData:[coder decodeObjectForKey:#"textureCoords"]];
NSData *normalData = [[NSData alloc] initWithData:[coder decodeObjectForKey:#"normals"]];
NSData *faceData = [[NSData alloc] initWithData:[coder decodeObjectForKey:#"faces"]];
memcpy(vertices, [vertexData bytes], sizeof(point3D) * vertexCount);
memcpy(textureCoords, [textureData bytes], sizeof(point2D) * textureCount);
memcpy(normals, [normalData bytes], sizeof(point3D) * normalCount);
memcpy(faces, [faceData bytes], sizeof(GLuint) * faceCount);
[vertexData release];
[textureData release];
[normalData release];
[faceData release];
I have tried retaining everything in this part (even the string) but it does not help.
This was a hard one to solve partly because debugging memory behaves inconsistently.
I have 2 classes JGStaticModel and JGModel. For some reason, the unarchiver would pick one of those at random so sometimes initWithCoder was sent to JGModel and not JGStaticModel. This led me to think it was not being called. Also since their structure is slightly different it had issues and crashed. The reason why I got the autorelease problem was I patched some memory problems in JGStaticModel but not JGModel so it would crash on the memory because I had not fixed it there.
Thanks for all the help!
Try turning on NSZombieEnabled, this should help you track down the problem.
If neither tools (Run -> Run with Performance tools) and NSZombiesEnabled helps, you can override - (id)retain and - (void)release methods of the class that caused the exception. Call super implementation and log retain/release. You can break in this methods to see the call stack. This way isn't beautiful, however, it helped me few time to figure out where was an extra release/autorelease call
This problem is very easy to solve with the solution given here.
The relevant part is:
If an environment variable named "NSAutoreleaseHaltOnFreedObject" is set with string value "YES", the function will automatically break in the debugger