"Can't allocate region" malloc error when running millions of iterations of loop - objective-c

Question

Thanks to a lot of help I've received here on SO, I've gotten an algorithm to check a list of around 15,000 8-letter words for any partial anagrams, against a list of around 50,000 total words (so I suppose a total of 108 million iterations). I call this method once for each comparison (so 750 million times). I'm getting the following error, always somewhere in the midst of the 119th iteration through the 1,350 there should be:
AnagramFINAL(2960,0xac8c7a28) malloc: *** mmap(size=2097152) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
I've narrowed the memory issue down to being a huge number of allocated CFStrings (immutable). Any idea what I can do to remedy the issue? I'm using ARC and an #autoreleasepool, not sure what else I could do, it seems something isn't being released when it should be.
AnagramDetector.h
#import <Foundation/Foundation.h>
#interface AnagramDetector : NSObject {
NSDictionary *allEightLetterWords;
NSDictionary *allWords;
NSFileManager *fileManager;
NSArray *paths;
NSString *documentsDirectory;
NSString *filePath;
}
- (BOOL) does: (NSString *) longWord contain: (NSString *) shortWord;
- (NSDictionary *) setupAllWordList;
- (NSDictionary *) setupEightLetterWordList;
- (void) saveDictionary: (NSMutableDictionary *)currentArray;
#end
AnagramDetector.m
#implementation AnagramDetector
- (id) init {
self = [super init];
if (self) {
fileManager = [NSFileManager defaultManager];
paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
documentsDirectory = [paths objectAtIndex:0];
}
return self;
}
- (BOOL) does: (NSString *) longWord contain: (NSString *) shortWord {
#autoreleasepool {
NSMutableString *longerWord = [longWord mutableCopy];
for (int i = 0; i < [shortWord length]; i++) {
NSString *letter = [shortWord substringWithRange: NSMakeRange(i, 1)];
NSRange letterRange = [longerWord rangeOfString: letter];
if (letterRange.location != NSNotFound) {
[longerWord deleteCharactersInRange: letterRange];
} else {
return NO;
}
}
return YES;
}
}
- (NSDictionary *) setupAllWordList {
#autoreleasepool {
NSString *fileWithAllWords = [[NSBundle mainBundle] pathForResource:#"AllDefinedWords" ofType:#"plist"];
allWords = [[NSDictionary alloc] initWithContentsOfFile: fileWithAllWords];
NSLog(#"Total number of words: %d.", [allWords count]);
}
return allWords;
}
- (NSDictionary *) setupEightLetterWordList {
#autoreleasepool {
NSString *fileWithEightWords = [[NSBundle mainBundle] pathForResource:#"AllDefinedEights" ofType:#"plist"];
allEightLetterWords = [[NSDictionary alloc] initWithContentsOfFile: fileWithEightWords];
NSLog(#"Total number of words: %d.", [allEightLetterWords count]);
}
return allEightLetterWords;
}
- (void) saveDictionary: (NSMutableDictionary *)currentArray {
#autoreleasepool {
filePath = [documentsDirectory stringByAppendingPathComponent: #"A.plist"];
[fileManager createFileAtPath:filePath contents: nil attributes: nil];
[currentArray writeToFile: filePath atomically:YES];
[currentArray removeAllObjects];
}
}
#end
Code running on launch (inside AppDelegate for now, since no VC):
#autoreleasepool {
AnagramDetector *detector = [[AnagramDetector alloc] init];
NSDictionary *allWords = [[NSDictionary alloc] initWithDictionary:[detector setupAllWordList]];
NSDictionary *eightWords = [[NSDictionary alloc] initWithDictionary:[detector setupEightLetterWordList]];
int remaining = [eightWords count];
for (NSString *currentEightWord in eightWords) {
if (remaining % 10 == 0) NSLog(#"%d ::: REMAINING :::", remaining);
for (NSString *currentAllWord in allWords) {
if ([detector does: [eightWords objectForKey: currentEightWord] contain: [allWords objectForKey: currentAllWord]]) {
// NSLog(#"%# ::: CONTAINS ::: %#", [eightWords objectForKey: currentEightWord], [allWords objectForKey: currentAllWord]);
}
}
remaining--;
}
}

Answer 1

The problem seems to be that a lot of autoreleased objects fill up the memory waiting to be released. So a solution is to add your own autorelease pool scope to collect autoreleased objects and release them sooner.
I suggest that you do something like this:
for (NSString *currentEightLetterWord in [eightLetterWordsDictionary allKeys]) {
#autoreleasepool {
for (NSString *currentWord in [allWordsDictionary allKeys]) {
}
}
}
Now all autoreleased objects inside #autoreleasepool { .. } will be released for each iteration of the outer loop.
As you see ARC might save you from thinking about most reference counting and memory management issues but objects can still end up in autorelease pools with ARC when using methods that directly or indirectly create autoreleased objects.
An alternative solution that I don't really recommend is to try to avoid using method that will use autorelease. Then does:contain: could awkwardly be rewritten to something like this:
- (BOOL) does: (NSString* ) longWord contain: (NSString *) shortWord {
NSMutableString *haystack = [longWord mutableCopy];
NSMutableString *needle = [shortWord mutableCopy];
while([haystack length] > 0 && [needle length] > 0) {
NSMutableCharacterSet *set = [[NSMutableCharacterSet alloc] init];
[set addCharactersInRange:NSMakeRange([needle characterAtIndex:0], 1)];
if ([haystack rangeOfCharacterFromSet:set].location == NSNotFound) return NO;
haystack = [haystack mutableCopy];
[haystack deleteCharactersInRange:NSMakeRange(0, [haystack rangeOfCharacterFromSet: set].location)];
needle = [needle mutableCopy];
[needle deleteCharactersInRange:NSMakeRange(0, 1)];
}
return YES;
}