Hang in mergeChangesFromContextDidSaveNotification (and merge conflicts) - objective-c

In summary: My app is hanging when I call [myMOC mergeChangesFromContextDidSaveNotification:notification] in a multithreaded scenario.
The detailed situation is this:
My app downloads a whole bunch of data from a server and stores it in Core Data on first launch. It comes in several parts. Parsing the whole thing takes several seconds, and most of that time is spent on two of the chunks. So in order to speed it up, I've parallelized those two chunks. This is what it looks like:
NSArray *arr = [self parseJsonData:downloadedNSData]; //turns NSData into JSON array
//using NSJSONSerialization
NSMutableArray __block *first = [[NSMutableArray alloc]init];
NSMutableArray __block *second = [[NSMutableArray alloc]init];
//put half of arr in first and half in second with a loop
dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
dispatch_group_t group = dipatch_group_create();
dispatch_group_async(group, queue, ^
{
for (id element in first)
{
[MyDataClass parseData:element]; //create NSManagedObject subclass, save
}
[self saveContext];
});
dispatch_group_async(group, queue, ^
{
for (id element in second)
{
[MyDataClass parseData:element]; //create NSManagedObject subclass, save
}
[self saveContext];
});
dispatch_group_wait(group, DISPATCH_TIME_FOREVER);
[self saveContext] calls save on the MOC. This is of course done on multiple threads, so I need to create a separate ManagedObjectContext for each thread. This is accomplished by exposing the MOC via a property on this object (self) that maintains an NSMutableDictionary of thread names (call description on NSThread) to NSManagedObjectContexts. When accessed, if there isn't a MOC for [NSThread currentThread], it creates a new one, adds it to the dictionary, and stores it. When each MOC is created, I subscribe to its change notifications:
NSNotificationCenter *center = [NSNotificationCenter defaultCenter];
[center addObserver:self selector:#selector(mergeChanges:)
name:NSManagedObjectContextDidSaveNotification object:createdMOC];
In mergeChanges, I loop through my dictionary of contexts and call mergeChangesFromContextDidSaveNotification on all of them except the one for the thread that this is happening on. More specifically, I use [performSelector:onThread:withObject:waitUntilDone:] to have each MOC do this on the thread it's created for.
I'm also locking using NSLock around both the [myMOC save] and my mergeChanges method. If I didn't lock around those, I got "Cocoa error 133020", which apparently means an error merging changes.
So, this is what my logging tells me is happening:
Thread 1 acquires the lock to save and begins saving
A merge context notification comes on thread 1. Thread 1 acquires the lock for merging changes and begins that process. It merges changes for the MOC for the main thread and then hangs when doing one of the MOCs for the background threads.
Thread 2 starts to save but never acquires the lock for saving, because the other thread is stuck trying to merge changes.
So, why is it hanging when merging changes? Is there a better way to handle this scenario?
Update: I have tried using [persistentStoreCoordinator lock] around my [MOC save] call instead of just a lock with NSLock, to no avail. I also tried adding [NSThread sleepForTimeInterval:2] before the call to [self saveContext] in one of the dispatch_group_async calls, and it didn't help.
Update 2: Perhaps the better question here is why I was getting merge conflicts (Cocoa Error 133020). Is that expected? Am I doing the merge right (merging to all contexts except the one saving)?
Update 3: I've posted another question to address the larger context of how I'm doing the multithreading.

When you're creating an NSManagedObject subclass, you're inserted it into a context and therefore you can only do that on the queue / thread that belongs to the context.
The same goes for calling -save on the context.
It seems to me from your code that you're inserting two objects — each on their own thread. That will not work.

I've posted this question documenting my situation a little better. This current question I think is a little narrow in scope.

Related

NSFecthedResultsController and NSPredicate update issue [duplicate]

I am presenting table view contents using NSFetchedResultsController which has a predicate:
[NSPredicate predicateWithFormat:#"visible == %#", [NSNumber numberWithBool:YES]]
On background thread using separate NSManagedObjectContext I update few of the entities and change theirs visible value from NO to YES. Save, merge changes in main thread's NSManagedObjectContext. But NSFetchedResultsController's fetchedObjects doesn't change. Also controller doesn't call -controller:didChangeObject:... on delegate. If entities are updated on main thread in identical manner (my test app calls the same method), everything works as expected.
Also Notification's NSUpdatedObjectsKey contains those objects.
Currently the only solutions I've found is to call for each of NSUpdatedObjectsKey entities:
NSManagedObjectContext *context = ... // main thread context
[context existingObjectWithID:[object objectID] error:nil]
This issue is only with updated objects which previously didn't match the predicate.
Am I missing something obvious?
Turns out main NSManagedObjectContext didn' t event fire NSManagedObjectContextObjectsDidChangeNotification for updated objects because it is not done for faulted objects.
Generic fix (or keep a track of object IDs that needs this treatment):
NSManagedObjectContext *context = [self managedObjectContext];
for(NSManagedObject *object in [[notification userInfo] objectForKey:NSUpdatedObjectsKey]) {
[[context objectWithID:[object objectID]] willAccessValueForKey:nil];
}
[context mergeChangesFromContextDidSaveNotification:notification];
From NSManagedObject Class Reference:
You can invoke this method with the
key value of nil to ensure that a
fault has been fired, as illustrated
by the following example.
You have to call processPendingChanges on your Background-NSManagedObjectContext after you merged the changes from an other NSManagedObjectContext.
See CoreData Programming Guide:
Note that the change notification is sent in NSManagedObjectContext’s
processPendingChanges method. The main thread is tied into the event
cycle for the application so that processPendingChanges is invoked
automatically after every user event on contexts owned by the main
thread. This is not the case for background threads—when the method is
invoked depends on both the platform and the release version, so you
should not rely on particular timing. If the secondary context is not
on the main thread, you should call processPendingChanges yourself at
appropriate junctures.

CoreData import not caught by NSFetchedResultsController [duplicate]

I am presenting table view contents using NSFetchedResultsController which has a predicate:
[NSPredicate predicateWithFormat:#"visible == %#", [NSNumber numberWithBool:YES]]
On background thread using separate NSManagedObjectContext I update few of the entities and change theirs visible value from NO to YES. Save, merge changes in main thread's NSManagedObjectContext. But NSFetchedResultsController's fetchedObjects doesn't change. Also controller doesn't call -controller:didChangeObject:... on delegate. If entities are updated on main thread in identical manner (my test app calls the same method), everything works as expected.
Also Notification's NSUpdatedObjectsKey contains those objects.
Currently the only solutions I've found is to call for each of NSUpdatedObjectsKey entities:
NSManagedObjectContext *context = ... // main thread context
[context existingObjectWithID:[object objectID] error:nil]
This issue is only with updated objects which previously didn't match the predicate.
Am I missing something obvious?
Turns out main NSManagedObjectContext didn' t event fire NSManagedObjectContextObjectsDidChangeNotification for updated objects because it is not done for faulted objects.
Generic fix (or keep a track of object IDs that needs this treatment):
NSManagedObjectContext *context = [self managedObjectContext];
for(NSManagedObject *object in [[notification userInfo] objectForKey:NSUpdatedObjectsKey]) {
[[context objectWithID:[object objectID]] willAccessValueForKey:nil];
}
[context mergeChangesFromContextDidSaveNotification:notification];
From NSManagedObject Class Reference:
You can invoke this method with the
key value of nil to ensure that a
fault has been fired, as illustrated
by the following example.
You have to call processPendingChanges on your Background-NSManagedObjectContext after you merged the changes from an other NSManagedObjectContext.
See CoreData Programming Guide:
Note that the change notification is sent in NSManagedObjectContext’s
processPendingChanges method. The main thread is tied into the event
cycle for the application so that processPendingChanges is invoked
automatically after every user event on contexts owned by the main
thread. This is not the case for background threads—when the method is
invoked depends on both the platform and the release version, so you
should not rely on particular timing. If the secondary context is not
on the main thread, you should call processPendingChanges yourself at
appropriate junctures.

NSManagedObjectContext: Is this a good pattern of thread confinement?

With Core Data there's a constant dance around thread safety issues. Executing fetches on one NSManagedObjectContext in two different threads guarantees a deadlock. Apple's Core Data Programming Guide suggests using thread confinement, but doesn't provide a pattern by which you may accomplish this. I have a quick and dirty solution below:
- (NSManagedObjectContext *) managedObjectContext
{
NSManagedObjectContext *moc = objc_getAssociatedObject([NSThread currentThread], _cmd);
if (!moc && self.persistentStoreCoordinator)
{
moc = [[NSManagedObjectContext alloc] init];
moc.mergePolicy = NSOverwriteMergePolicy;
moc.persistentStoreCoordinator = self.persistentStoreCoordinator;
objc_setAssociatedObject([NSThread currentThread], _cmd, moc, OBJC_ASSOCIATION_RETAIN);
}
return moc;
}
This method would appear in a proprietary class that manages the data stack in my app. It associates a MOC with the current thread. This seems like a plausible solution to me, but it also appears fast and loose.
What should I be worried about attempting a solution like this?
You should go the other way around: Use
[NSManagedObjectContext alloc] initWithConcurrencyType:concurrencyType]]
to create a context of
NSMainQueueConcurrencyType - if you need a context linked to controllers and UI objects that are required to be used only on the main thread, or
NSPrivateQueueConcurrencyType - for a context that runs on a private background queue.
Use performBlock or performBlockAndWait for all operations on the context, this ensures that the operations are executed on the queue specified for the context.
See Concurrency Support for Managed Object Contexts in the "Core Data Release Notes for OS X v10.7 and iOS 5.0" for more information.

Core Data Multithreading: Code Examples

I've been having problems with my multi-threaded Core Data enabled app, and I figured I should take a hard look at what I'm doing and how. Please let me know if the following should work.
I have a singleton DataManager class that handles the Core Data stuff. It has a property managedObjectContext that returns a different MOC for each thread. So, given NSMutableDictionary *_threadContextDict (string thread names to contexts) and NSMutableDictionary *_threadDict (string thread names to threads), it looks something like this:
-(NSManagedObjectContext *)managedObjectContext
{
if ([NSThread currentThread] == [NSThread mainThread])
{
MyAppDelegate *delegate = [[UIApplication sharedApplication] delegate];
return delegate.managedObjectContext; //MOC created in delegate code on main thread
}
else
{
NSString *thisThread = [[NSThread currentThread] description];
{
if ([_threadContextDict objectForKey:thisThread] != nil)
{
return [_threadContextDict objectForKey:thisThread];
}
else
{
NSManagedObjectContext *context = [[NSManagedObjectContext alloc]init];
MyAppDelegate *delegate = [[UIApplication sharedApplication] delegate];
[context setPersistentStoreCoordinator:delegate.persistentStoreCoordinator];
[_threadContextDict setObject:context forKey:thisThread];
[_threadDict setObject:[NSThread currentThread] forKey:thisThread];
//merge changes notifications
NSNotificationCenter *center = [NSNotificationCenter defaultCenter];
[center addObserver:self selector:#selector(mergeChanges:)
name:NSManagedObjectContextDidSaveNotification object:context];
return context;
}
}
}
}
In the mergeChanges method, I merge the changes from the incoming notification to all contexts except the one that generated the notification. It looks like this:
-(void)mergeChanges:(NSNotification *)notification
{
MyAppDelegate *delegate = [[UIApplication sharedApplication] delegate];
NSManagedObjectContext *context = delegate.managedObjectContext;
[context performSelectorOnMainThread:#selector(mergeChangesFromContextDidSaveNotification)
withObject:notification waitUntilDone:YES];
for (NSString *element in [_threadContextDict allKeys])
{
if (![element isEqualToString:[[NSThread currentThread] description]])
{
NSThread *thread = [_threadDict objectForKey:element];
NSManagedObjectContext *threadContext = [_threadContextDict objectForKey:element];
[threadContext performSelector:#selector(mergeChangesFromContextDidSaveNotification)
onThread:thread withObject:notification waitUntilDone:YES];
}
}
}
Whenever I save changes on a MOC, it's done with a call to a saveContext method on this shared DataManager, which calls save on a context obtained from the aforementioned property:
-(void)saveContext
{
NSManagedObjectContext *context = self.managedObjectContext;
NSError *err = nil;
[context save:&err];
//report error if necessary, etc.
}
Given my understanding of the Core Data multithreading rules, I feel like this should work. I'm using a separate context for each thread, but the same persistent store for all of them. But when I use this, I get a lot of merge conflicts, even though my threads aren't working on the same objects (NSManagedObject subclasses). I'm just downloading data from the network, parsing the results, and saving them to Core Data.
Am I doing something wrong? I've tried using NSLock instances to lock around some things, but then I just get hangs.
UPDATE/RESOLUTION: I was able to make this work by adding one simple thing: a way to remove a thread/MOC pair from my dictionary when I'm finished with it. At the end of each block in each call to dispatch_async where I do Core Data stuff, I call [self removeThread], which removes the current thread and its MOC from the dictionary. I also only merge changes to the main thread MOC. Effectively, this means that every time I do work on a background thread, I get a fresh new MOC.
I also distinguish threads by adding a number to userInfoDict, instead of calling description. The number is obtained by a readonly property on my class that returns a higher number each time it's called.
With all due respect, your approach is a nightmare, and it should be even worse to debug it to solve anything if there is a problem with it. First problem is this:
I have a singleton DataManager
Do not have a singleton object that manages core data manipulation with different entities on different threads. Singletons are tricky to deal with, especially on multithreading environment, and is even a worse approach to use it with core data.
Second thing, do not use NSThread to work on multithreading. There are more modern APIs. Use Grand central dispatch or NSOperation/NSOperationQueue. Apple has encouraged people to move from NSThread since the introduction of blocks (iOS 4). And for future reference, do not use the description of an object the way you are using it. Descriptions are usually/mostly used for debugging purposes. The information there should not be used to compare. Not even the pointer value (which is why you should use isEqual instead of ==).
This is what you need to know about core data and multithreading:
Create one context per thread. The core data template has already created a main thread context for you. At the start of the execution of the background thread (inside the block, or on the main method of your NSOperation subclass), initialize your context.
Once your context is initialize, and has the right persistentStoreCoordinator, listen to the NSManagedObjectContextObjectsDidChangeNotification. The object listening to the notification will receive the notification in the same thread the context was being saved. Since this is different than the main thread, do the merge call with the merging context on the thread the receiving context is being used. Let's say that you are using a context inside a thread different than the main thread, and you want to merge with the main thread, you need to call the merge method inside the main thread. You can do that with
dispatch_async(dispatch_get_main_queue(), ^{//code here});
Do not use an NSManagedObject outside the thread where its managedObjectContext lives.
With these and other, simple rules, managing core data under a multithreading environment is easier. Your approach more difficult to implement, and worse to debug. Make some changes to your architecture. Manage the context depending on the thread you are working with (instead of centralized). Do not keep references to context outside of their scope. Once your first context is created, it is not expensive to be creating contexts on your threads. You can reuse the same context, as long as it's inside the same block/NSOperation execution.

Is Objective-C's NSMutableArray thread-safe?

I've been trying to fix this crash for almost a week. The application crashes without any exception or stack-trace. The application does not crash in any way while running through instruments in zombie mode.
I have a method that gets called on a different thread.
The solution that fixed the crash was replacing
[self.mutableArray removeAllObjects];
with
dispatch_async(dispatch_get_main_queue(), ^{
[self.searchResult removeAllObjects];
});
I thought it might be a timing issue, so I tried to synchronize it, but it still crashed:
#synchronized(self)
{
[self.searchResult removeAllObjects];
}
Here is the code
- (void)populateItems
{
// Cancel if already exists
[self.searchThread cancel];
self.searchThread = [[NSThread alloc] initWithTarget:self
selector:#selector(populateItemsinBackground)
object:nil];
[self.searchThread start];
}
- (void)populateItemsinBackground
{
#autoreleasepool
{
if ([[NSThread currentThread] isCancelled])
[NSThread exit];
[self.mutableArray removeAllObjects];
// Populate data here into mutable array
for (loop here)
{
if ([[NSThread currentThread] isCancelled])
[NSThread exit];
// Add items to mutableArray
}
}
}
Is this problem with NSMutableArray not being thread-safe?
No.
It is not thread safe and if you need to modify your mutable array from another thread you should use NSLock to ensure everything goes as planned:
NSLock *arrayLock = [[NSLock alloc] init];
[...]
[arrayLock lock]; // NSMutableArray isn't thread-safe
[myMutableArray addObject:#"something"];
[myMutableArray removeObjectAtIndex:5];
[arrayLock unlock];
As others already said, NSMutableArray is not thread safe. In case anyone want to achieve more than removeAllObject in a thread-safe environment, I will give another solution using GCD besides the one using lock. What you have to do is to synchronize the read/update(replace/remove) actions.
First get the global concurrent queue:
dispatch_queue_t concurrent_queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
For read:
- (id)objectAtIndex:(NSUInteger)index {
__block id obj;
dispatch_sync(self.concurrent_queue, ^{
obj = [self.searchResult objectAtIndex:index];
});
return obj;
}
For insert:
- (void)insertObject:(id)obj atIndex:(NSUInteger)index {
dispatch_barrier_async(self.concurrent_queue, ^{
[self.searchResult insertObject:obj atIndex:index];
});
}
From Apple Doc about dispatch_barrier_async:
When the barrier block reaches the front of a private concurrent queue, it is not executed immediately. Instead, the queue waits until its currently executing blocks finish executing. At that point, the barrier block executes by itself. Any blocks submitted after the barrier block are not executed until the barrier block completes.
Similar for remove:
- (void)removeObjectAtIndex:(NSUInteger)index {
dispatch_barrier_async(self.concurrent_queue, ^{
[self.searchResult removeObjectAtIndex:index];
});
}
EDIT: Actually I found another simpler way today to synchronize access to a resource by using a serial queue provided by GCD.
From Apple Doc Concurrency Programming Guide > Dispatch Queues:
Serial queues are useful when you want your tasks to execute in a specific order. A serial queue executes only one task at a time and always pulls tasks from the head of the queue. You might use a serial queue instead of a lock to protect a shared resource or mutable data structure. Unlike a lock, a serial queue ensures that tasks are executed in a predictable order. And as long as you submit your tasks to a serial queue asynchronously, the queue can never deadlock.
Create your serial queue:
dispatch_queue_t myQueue = dispatch_queue_create("com.example.MyQueue", NULL);
Dispatch tasks async to the serial queue:
dispatch_async(myQueue, ^{
obj = [self.searchResult objectAtIndex:index];
});
dispatch_async(myQueue, ^{
[self.searchResult removeObjectAtIndex:index];
});
Hope it helps!
As well as NSLock can also use #synchronized(condition-object) you just have to make sure every access of the array is wrapped in a #synchronized with the same object acting as the condition-object , if you only want to modify the contents of the same array instance then you can use the array itself as the condition-object, other wise you will have to use something else you know will not go away, the parent object, i.e self, is a good choice because it will always be the same one for the same array.
atomic in #property attributes will only make setting the array thread safe not modifying the contents, i.e. self.mutableArray = ... is thread safe but [self.mutableArray removeObject:] is not.
__weak typeof(self)weakSelf = self;
#synchronized (weakSelf.mutableArray) {
[weakSelf.mutableArray removeAllObjects];
}
Since serial queues were mentioned: With a mutable array, just asking "is it thread safe" isn't enough. For example, making sure that removeAllObjects doesn't crash is all good and fine, but if another thread tries to process the array at the same time, it will either process the array before or after all elements are removed, and you really have to think what the behaviour should be.
Creating one class + object that is responsible for this array, creating a serial queue for it, and doing all operations through the class on that serial queue is the easiest way to get things right without making your brain hurt through synchronisation problems.
All the NSMutablexxx classes are not thread-safe. Operations including get,insert,remove,add and replace should be used with NSLock.This is a list of thread-safe and thread-unsafe classes given by apple: Thread Safety Summary
Almost NSMutable classes object is not thread safe.