How can I optimise out this nested for loop? - objective-c

Question

How can I optimise out this nested for loop?
The program should go through each word in the array created from the word text file, and if it's greater than 8 characters, add it to the goodWords array. But the caveat is that I only want the root word to be in the goodWords array, for example:
If greet is added to the array, I don't want greets or greetings or greeters, etc.
NSString *string = [NSString stringWithContentsOfFile:#"/Users/james/dev/WordParser/word.txt" encoding:NSUTF8StringEncoding error:NULL];
NSArray *words = [string componentsSeparatedByString:#"\r\n"];
NSMutableArray *goodWords = [NSMutableArray array];
BOOL shouldAddToGoodWords = YES;
for (NSString *word in words)
{
NSLog(#"Word: %#", word);
if ([word length] > 8)
{
NSLog(#"Word is greater than 8");
for (NSString *existingWord in [goodWords reverseObjectEnumerator])
{
NSLog(#"Existing Word: %#", existingWord);
if ([word rangeOfString:existingWord].location != NSNotFound)
{
NSLog(#"Not adding...");
shouldAddToGoodWords = NO;
break;
}
}
if (shouldAddToGoodWords)
{
NSLog(#"Adding word: %#", word);
[goodWords addObject:word];
}
}
shouldAddToGoodWords = YES;
}

Answer 1

How about something like this?
//load the words from wherever
NSString * allWords = [NSString stringWithContentsOfFile:#"/usr/share/dict/words"];
//create a mutable array of the words
NSMutableArray * words = [[allWords componentsSeparatedByCharactersInSet:[NSCharacterSet newlineCharacterSet]] mutableCopy];
//remove any words that are shorter than 8 characters
[words filterUsingPredicate:[NSPredicate predicateWithFormat:#"length >= 8"]];
//sort the words in ascending order
[words sortUsingSelector:#selector(caseInsensitiveCompare:)];
//create a set of indexes (these will be the non-root words)
NSMutableIndexSet * badIndexes = [NSMutableIndexSet indexSet];
//remember our current root word
NSString * currentRoot = nil;
NSUInteger count = [words count];
//loop through the words
for (NSUInteger i = 0; i < count; ++i) {
NSString * word = [words objectAtIndex:i];
if (currentRoot == nil) {
//base case
currentRoot = word;
} else if ([word hasPrefix:currentRoot]) {
//word is a non-root word. remember this index to remove it later
[badIndexes addIndex:i];
} else {
//no match. this word is our new root
currentRoot = word;
}
}
//remove the non-root words
[words removeObjectsAtIndexes:badIndexes];
NSLog(#"%#", words);
[words release];
This runs very very quickly on my machine (2.8GHz MBP).

Answer 2

A Trie seems suitable for your purpose. It is like a hash, and is useful for detecting if a given string is a prefix of an already seen string.

Answer 3

I used an NSSet to ensure that you only have 1 copy of a word added at a time. It will add a word if the NSSet does not already contain it. It then checks to see if the new word is a substring for any word that has already been added, if true then it won't add the new word. It's case-insensitive as well.
What I've written is a refactoring of your code. It's probably not that much faster but you really do want a tree data structure if you want to make it a lot faster when you want to search for words that have already been added to your tree.
Take a look at RedBlack Trees or B-Trees.
Words.txt
objective
objectively
cappucin
cappucino
cappucine
programme
programmer
programmatic
programmatically
Source Code
- (void)addRootWords {
NSString *textFile = [[NSBundle mainBundle] pathForResource:#"words" ofType:#"txt"];
NSString *string = [NSString stringWithContentsOfFile:textFile encoding:NSUTF8StringEncoding error:NULL];
NSArray *wordFile = [string componentsSeparatedByString:#"\n"];
NSMutableSet *goodWords = [[NSMutableSet alloc] init];
for (NSString *newWord in wordFile)
{
NSLog(#"Word: %#", newWord);
if ([newWord length] > 8)
{
NSLog(#"Word '%#' contains 8 or more characters", newWord);
BOOL shouldAddWord = NO;
if ( [goodWords containsObject:newWord] == NO) {
shouldAddWord = YES;
}
for (NSString *existingWord in goodWords)
{
NSRange textRange = [[newWord lowercaseString] rangeOfString:[existingWord lowercaseString]];
if( textRange.location != NSNotFound ) {
// newWord contains the a substring of existingWord
shouldAddWord = NO;
break;
}
NSLog(#"(word:%#) does not contain (substring:%#)", newWord, existingWord);
shouldAddWord = YES;
}
if (shouldAddWord) {
NSLog(#"Adding word: %#", newWord);
[goodWords addObject:newWord];
}
}
}
NSLog(#"***Added words***");
int count = 1;
for (NSString *word in goodWords) {
NSLog(#"%d: %#", count, word);
count++;
}
[goodWords release];
}
Output:
***Added words***
1: cappucino
2: programme
3: objective
4: programmatic
5: cappucine