Custom NSTextView insertText:replacementRange breaks Spell Checking - objective-c

Question

I have a custom NSTextView subclass, with a custom NSTextStorage component as well. The NSTextStorage modifies the text entered by the user based on context.
Because it's possible that the final text will be shorter than the text originally entered by the user, I had to override insertText:replacementRange in my NSTextView. A minimum example is:
- (void) insertText:(id)string replacementRange:(NSRange)replacementRange {
if ([self hasMarkedText]) {
[[self textStorage] replaceCharactersInRange:[self markedRange] withString:string];
} else {
[[self textStorage] replaceCharactersInRange:[self selectedRange] withString:string];
}
[self didChangeText];
}
This works fine in extensive testing over several months.... Except that automatic spell checking and correction is disabled. The "squigglies" don't appear under misspelled words, unless I stop typing, move the mouse, and switch focus to and from my app. After several seconds, the entire textview is spellcheck'ed. Because it happens after the fact, automatic correction is disabled of course.
If I disable my custom insertText:replacementRange: method, everything else works fine, and automatic spelling functionality returns. I just have to be careful not to trigger a change that results in shortening the text, as it triggers attribute out of range errors (the original reason for my custom method in the first place.)
Apparently Apple's implementation of insertText:replacementRange: does much more than mine. I have tried multiple variations on [self checkTextInRange...], [self checkTextInSelection:], etc. None of them restore proper functionality.
Searching Apple's documentation doesn't help point me towards what I am leaving out from my method that is causing spell checking to break. Any pointers or ideas would be much appreciated!!
Thanks in advance!
EDIT: Here are some examples of the sorts of behavior my NSTextStorage provides. (| represents the insertion caret)
Starting with:
* item
* |
If I hit the return key, I end up with the following (deleting *<space>):
* item
|
Another example, if "Change Tracking" is enabled:
this is thee| time
If I hit delete:
this is the|{--e--} time
As you can see, a single keystroke may result in the addition or deletion of multiple characters from the text.
EDIT 2: FYI -- the issue I have with attributes being out of range occur when the shortening happens while pressing return at the end of the document -- NSTextview attempts to set a new paragraph style only to find that the document is shorter than expected. I can find no way to change the range NSTextview targets.

Answer 1

I have a partial solution.
In my custom insertText:replacementRange: method, prior to didChangeText:
NSinteger wordCount;
NSOrthography * orthography;
static NSInteger theWordCount;
NSOrthography * orthography;
NSRange spellingRange = <range to check>
NSArray * results = [[NSSpellChecker sharedSpellChecker] checkString:[[self textStorage] string]
range:spellingRange
types:[self enabledTextCheckingTypes]
options:NULL
inSpellDocumentWithTag:0
orthography:&orthography
wordCount:&theWordCount];
if (results.count) {
[self handleTextCheckingResults:results forRange:spellingRange types:[self enabledTextCheckingTypes] options:#{} orthography:orthography wordCount:theWordCount];
}
However, this is incomplete:
Spell check and Grammar check works fine
Automatic spelling correction and text replacement do not work (even when enabled)

Answer 2

(EDITED 2018-05-30)
Updated response (2018-05-22):
This issue reared its ugly head again, and I really needed to figure it out.
My custom NSTextStorage is fundamentally the same as described, and still works.
I use a custom insertText:replacementRange: on my NSTextView, but it calls [super insertText:replacementRange:] to take advantage of Apple's behind-the-scenes work that makes spelling, etc. work better. My custom method only needs to set a boolean.
When shortening the text, I still get requests from Apple's insertText:replacementRange: for attributes in a non-existent part of the text. Previously, I would get stuck here, because everything I tried either caused a crash, or caused Apple's code to repeatedly request the non-existing attributes indefinitely.
Finally, I tried returning fake attributes with a NULL rangepointer, and this seems to make Apple's code happy:
- (NSDictionary *) attributesAtIndex:(NSUInteger)location effectiveRange:(nullable NSRangePointer)range {
if (location > _backingAttributes.length) {
// This happens if we shrink the text before the textview is aware of it.
// For example, if we expand "longtext" -> "short" in our smart string, then
// The textview may set and request attributes past the end of our
// _backing string.
// Initially this was due to error in my code, but now I had to add
// This error checking back
NSLog(#"get attributes at (%lu) in (%lu)", (unsigned long)location, (unsigned long)_backingAttributes.length);
NSLog(#"error");
// Apparently returning fake attributes satisfies [NSTextView insertText:replacementRange:]
range = NULL;
return #{
NSForegroundColorAttributeName : [BIColor redColor],
NSFontAttributeName : [BIFont fontWithName:#"Helvetica" size:14.0]
};
} else {
return [_backingAttributes attributesAtIndex:location effectiveRange:range];
}
}
With further testing, this turned out to not be quite enough. I ended up adding the following to the setter to store the invalid attributes and range that macOS was trying to set:
- (void) setAttributes:(NSDictionary<NSString *,id> *)attrs range:(NSRange)range {
if (NSMaxRange(range) > _backingAttributes.length) {
_invalidAttrs = attrs;
_invalidRange = range;
} else {
[self beginEditing];
[_backingAttributes setAttributes:attrs range:range];
[self edited:NSTextStorageEditedAttributes range:range changeInLength:0];
[self endEditing];
}
}
I updated `attributesAtIndex:effectiveRange: to return the following when called with an invalid range, rather than returning the fake attributes above:
// Apparently returning fake attributes satisfies [NSTextView insertText]
*range = _invalidRange;
return _invalidAttrs;
This seems to work under various conditions that would previously trigger an exception or an infinite loop.