I have an NSManagedObjectContext declared like so:
- (NSManagedObjectContext *) backgroundMOC {
if (backgroundMOC != nil) {
return backgroundMOC;
}
backgroundMOC = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSPrivateQueueConcurrencyType];
return backgroundMOC;
}
Notice that it is declared with a private queue concurrency type, so its tasks should be run on a background thread. I have the following code:
-(void)testThreading
{
/* ok */
[self.backgroundMOC performBlock:^{
assert(![NSThread isMainThread]);
}];
/* CRASH */
[self.backgroundMOC performBlockAndWait:^{
assert(![NSThread isMainThread]);
}];
}
Why does calling performBlockAndWait execute the task on the main thread rather than background thread?
Tossing in another answer, to try an explain why performBlockAndWait will always run in the calling thread.
performBlock is completely asynchronous. It will always enqueue the block onto the queue of the receiving MOC, and then return immediately. Thus,
[moc performBlock:^{
// Foo
}];
[moc performBlock:^{
// Bar
}];
will place two blocks on the queue for moc. They will always execute asynchronously. Some unknown thread will pull blocks off of the queue and execute them. In addition, those blocks are wrapped within their own autorelease pool, and also they will represent a complete Core Data user event (processPendingChanges).
performBlockAndWait does NOT use the internal queue. It is a synchronous operation that executes in the context of the calling thread. Of course, it will wait until the current operations on the queue have been executed, and then that block will execute in the calling thread. This is documented (and reasserted in several WWDC presentations).
Furthermore, performBockAndWait is re-entrant, so nested calls all happen right in that calling thread.
The Core Data engineers have been very clear that the actual thread in which a queue-based MOC operation runs is not important. It's the synchronization by using the performBlock* API that's key.
So, consider 'performBlock' as "This block is being placed on a queue, to be executed at some undetermined time, in some undetermined thread. The function will return to the caller as soon as it has been enqueued"
performBlockAndWait is "This block will be executed at some undetermined time, in this exact same thread. The function will return after this code has completely executed (which will occur after the current queue associated with this MOC has drained)."
EDIT
Are you sure of "performBlockAndWait does NOT use the internal queue"?
I think it does. The only difference is that performBlockAndWait will
wait until the block's completion. And what do you mean by calling
thread? In my understanding, [moc performBlockAndWait] and [moc
performBloc] both run on its private queue (background or main). The
important concept here is moc owns the queue, not the other way
around. Please correct me if I am wrong. – Philip007
It is unfortunate that I phrased the answer as I did, because, taken by itself, it is incorrect. However, in the context of the original question it is correct. Specifically, when calling performBlockAndWait on a private queue, the block will execute on the thread that called the function - it will not be put on the queue and executed on the "private thread."
Now, before I even get into the details, I want to stress that depending on internal workings of libraries is very dangerous. All you should really care about is that you can never expect a specific thread to execute a block, except anything tied to the main thread. Thus, expecting a performBlockAndWait to not execute on the main thread is not advised because it will execute on the thread that called it.
performBlockAndWait uses GCD, but it also has its own layer (e.g., to prevent deadlocks). If you look at the GCD code (which is open source), you can see how synchronous calls work - and in general they synchronize with the queue and invoke the block on the thread that called the function - unless the queue is the main queue or a global queue. Also, in the WWDC talks, the Core Data engineers stress the point that performBlockAndWait will run in the calling thread.
So, when I say it does not use the internal queue, that does not mean it does not use the data structures at all. It must synchronize the call with the blocks already on the queue, and those submitted in other threads and other asynchronous calls. However, when calling performBlockAndWait it does not put the block on the queue... instead it synchronizes access and runs the submitted block on the thread that called the function.
Now, SO is not a good forum for this, because it's a bit more complex than that, especially w.r.t the main queue, and GCD global queues - but the latter is not important for Core Data.
The main point is that when you call any performBlock* or GCD function, you should not expect it to run on any particular thread (except something tied to the main thread) because queues are not threads, and only the main queue will run blocks on a specific thread.
When calling the core data performBlockAndWait the block will execute in the calling thread (but will be appropriately synchronized with everything submitted to the queue).
I hope that makes sense, though it probably just caused more confusion.
EDIT
Furthermore, you can see the unspoken implications of this, in that the way in which performBlockAndWait provides re-entrant support breaks the FIFO ordering of blocks. As an example...
[context performBlockAndWait:^{
NSLog(#"One");
[context performBlock:^{
NSLog(#"Two");
}];
[context performBlockAndWait:^{
NSLog(#"Three");
}];
}];
Note that strict adherence to the FIFO guarantee of the queue would mean that the nested performBlockAndWait ("Three") would run after the asynchronous block ("Two") since it was submitted after the async block was submitted. However, that is not what happens, as it would be impossible... for the same reason a deadlock ensues with nested dispatch_sync calls. Just something to be aware of if using the synchronous version.
In general, avoid sync versions whenever possible because dispatch_sync can cause a deadlock, and any re-entrant version, like performBlockAndWait will have to make some "bad" decision to support it... like having sync versions "jump" the queue.
Why not? Grand Central Dispatch's block concurrency paradigm (which I assume MOC uses internally) is designed so that only the runtime and operating system need to worry about threads, not the developer (because the OS can do it better than you can do to having more detailed information). Too many people assume that queues are the same as threads. They are not.
Queued blocks are not required to run on any given thread (the exception being blocks in the main queue must execute on the main thread). So, in fact, sometimes sync (i.e. performBlockAndWait) queued blocks will run on the main thread if the runtime feels it would be more efficient than creating a thread for it. Since you are waiting for the result anyway, it wouldn't change the way your program functioned if the main thread were to hang for the duration of the operation.
This last part I am not sure if I remember correctly, but in the WWDC 2011 videos about GCD, I believe that it was mentioned that the runtime will make an effort to run on the main thread, if possible, for sync operations because it is more efficient. In the end though, I suppose the answer to "why" can only be answered by the people who designed the system.
I don't think that the MOC is obligated to use a background thread; it's just obligated to ensure that your code will not run into concurrency issues with the MOC if you use performBlock: or performBlockAndWait:. Since performBlockAndWait: is supposed to block the current thread, it seems reasonable to run that block on that thread.
The performBlockAndWait: call only makes sure that you execute the code in such a way that you don't introduce concurrency (i.e. on 2 threads performBlockAndWait: will not run at the same time, they will block each other).
The long and the short of it is that you can't depend on which thread a MOC operation runs on, well basically ever. I've learned the hard way that if you use GCD or just straight up threads, you always have to create local MOCs for each operation and then merge them to the master MOC.
There is a great library (MagicalRecord) that makes that process very simple.
Related
I'm programming an application that makes use of asynchronous web requests using NSURLConnection, so I have multiple threads running. To ensure that the main logic of my app happens on one thread, I am making heavy use of performSelectorOnMainThread:waitUntilDone:. Sometimes though, I am running this on the main thread, which piqued my curiosity.
If performSelectorOnMainThread:waitUntilDone: is called while in the main thread? Does it act the same as just performSelector:? What if waitUntilDone: is YES? What if it is NO?
EDIT: I have found that when waitUntilDone: is YES, the selector is executed (almost) immediately, but I cannot figure out when it is executed if waitUntilDone: is NO.
performSelectorOnMainThread:withObject:waitUntilDone:
is a method to deliver message on main thread of your application. Here boolean value in parameter waitUntilDone: specifies that whether you want to block your main thread to execute specified selector or not.
for example -
if you written these two lines-
[self performSelectorOnMainThread:#selector(print) withObject:nil waitUntilDone:YES];
NSLog(#"Hello iPhone");
and this is the print method -
- (void) print
{
NSLog(#"Hello World");
}
then you will get this o/p
Hello World
Hello iPhone
so it first pause the execution of your main thread and print "Hello World" and then execute main thread again and print "Hello iPhone" because you specified YES in waitUntilDone:
but if you specified NO in waitUntilDone: then it will print like this -
Hello iPhone
Hello World
it clearly indicates that it put your request of executing the specified selector in a queue and as OS gets its main thread free it executed you request.
Calling performSelectorOnMainThread:withObject:waitUntilDone: either from main thread or a secondary thread doesn't make any difference in it's execution, it depends on what you specified in waitUntilDone:
for more info -
NSObject Class Reference
If the current thread is also the main thread, and you pass YES,
the message is performed immediately, otherwise the perform is
queued to run the next time through the run loop.
If YES, it can be performed before performSelectorOnMainThread:withObject:waitUntilDone: returns.
I have found that when waitUntilDone: is YES, the selector is executed (almost) immediately, but I cannot figure out when it is executed if waitUntilDone: is NO.
The bit about the run loop: Your main thread has a run loop. This more or less prevents a thread from exiting. A run loop manages a todo list. When its work is complete, it suspends execution of that thread for some time. Then it wakes up later and sees if has work to do.
The amount of work can vary greatly (e.g. it may do some really heavy drawing or file i/o between the time it awakes and the point your selector is performed. Therefore, it's not a good tool for really precise timing, but it should be enough to know how it works and how the implementations adds the work to the run loop.
http://developer.apple.com/library/ios/#documentation/cocoa/Conceptual/Multithreading/RunLoopManagement/RunLoopManagement.html
If waitUntilDone: is YES it acts as an immediate function call.
If waitUntilDone: is NO then it queues the call along with all other threads' calls.
This method queues the message on the run loop of the main thread
using the common run loop modes—that is, the modes associated with the
NSRunLoopCommonModes constant. As part of its normal run loop
processing, the main thread dequeues the message (assuming it is
running in one of the common run loop modes) and invokes the desired
method.
As noted above, things like drawing and I/O are prioritized over anything in queues. Once the main thread gets around to having time for queue service in the next event loop, there's a couple other details that make it not quite as simple as counting on first in first out:
1) dispatch_async() blocks ignore modes.
2) The performSelector variants with a specific mode argument -- event tracking, say -- may take precedence over ones with the default common modes argument in a loop running in that specific mode.
As a general rule, if you want predictable timing behaviours you should use the low level GCD dispatch functions that don't take account of higher level considerations like run loop modes.
Is it safe to add an NSOperationQueue to an NSOperation, and then add this operation to another NSOperationQueue?
Here is some code to visualize what I am trying to do.
NSOperationQueue *mainQueue = [NSOperationQueue alloc] init];
// Here I declare some NSBlockOperation's, i.e. parseOperation1-2-3
// and also another operation called zipOperation, which includes
// an NSOperationQueue itself. This queue takes the processed (parsed) files
// and write them to a single zip file. Each operation's job is to write the data
// stream and add it to the zip file. After all operations are done,
// it closes the zip.
[zipOperation addDependency:parseOperation1];
[zipOperation addDependency:parseOperation2];
[zipOperation addDependency:parseOperation3];
[mainQueue addOperation:parseOperation1];
[mainQueue addOperation:parseOperation2];
[mainQueue addOperation:parseOperation3];
[mainQueue addOperation:zipOperation];
I have used this approach and have it running in live code deployed on the App Store. I haven't experienced any issues during development or in the last 2 months since the code has been live.
In my case I had a high level series of operations, some of which contained a set of sub operations. Rather than expose the detail of each sub operation into the high level code, I created NSOperations which themselves contained NSOperationQueues and enqueued their own sub operations. The code I ended up with was much cleaner and easier to maintain.
I read extensively into NSOperation and have not seen any commentary that warns against this approach. I reviewed a lot of information online, the Apple documentation, and WWDC videos.
The only possible "drawback" might be the added complexity of understanding and implementing a Concurrent operation. Embedding an NSOperationQueue in an NSOperation means that operation becomes Concurrent.
So that's a 'YES' from me.
Additional details about concurrent operations:
An NSOperationQueue calls the start method on a normal (non-concurrent) NSOperation and expects the operation to be finished by the time the start call returns. For instance some piece of code you supplied to NSBlockOperation is complete at the end of the block.
If the work will not be finished by the time the start call returns then you configure the NSOperation as a Concurrent operation, so the NSOperationQueue knows that it has to wait until you tell it that the operation is finished at some later point in time.
For example, concurrent operations are often used to run asynchronous network calls; the start method only starts the network call, which then runs in the background, and calls back to the operation when its finished. You then change the isFinished property of the NSOperation to flag that the work is now complete.
So.... Normally when you add operations to an NSOperationQueue that queue runs those operations in the background. So if you put an NSOperationQueue inside an NSOperation then that operations work will be done in the background. Therefore the operation is concurrent and you need to flag when the internal NSOperationQueue has finished processing all it's operations.
Alternatively there are some methods on NSOperationQueue such as waitUntilAllOperationsAreFinished which could be used to ensure all the work was done before the start call returns, however these involve blocking threads and I avoided them, you may feel more comfortable with that approach, and making sure you don't have any side effects from blocking threads.
In my case I was already familiar with Concurrent operations so it was straightforward just to set it up as a Concurrent operation.
Some documentation about concurrent operations:
Concurrency Programming Guide: Configuring Operations for Concurrent Execution
In this example they are detaching a thread to perform work in the background, in our case we would be starting the NSOperationQueue here.
I just want to confirm my reasoning that doing a dispatch_sync from the UI thread(main thread) using the main queue is utterly useless.
To add to that, async would be just as useless but with the illusion that it is useful because it doesn't block.
Let me know.
Thanks
The dispatch_sync to the same queue is not only useless, but it will lock your app. dispatch_sync says "dispatch something, but don't proceed on this queue until the other queue responds". That obviously can't happen if the "other" queue (that we're waiting for it to complete the dispatched block) is the same one as "this" queue (which is blocked until that other queue responds). Your app will freeze, waiting for itself!
On the other hand, dispatch_async to the same queue you're currently on is not generally a very useful construction, but at least it won't freeze. I've seen some awkward code that used dispatch_async to itself as a way of saying "as soon as I finish some series of actions, I then want to do something else". I've rarely seen this dispatching asynchronously to itself in situations where it couldn't be done more elegantly another way, but I have seen it.
I am modifying my program to use the new iOS5 style.
So I simply use this code:
NSManagedObjectContext *threadContext = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSPrivateQueueConcurrencyType];
threadContext.parentContext = [self managedObjectContextMainThread];
//threadContext.persistentStoreCoordinator= [self persistentStoreCoordinator]; //moc.persistentStoreCoordinator;// [moc persistentStoreCoordinator];
My new background ManagedObjectContext doesn't have a persistentStore but have parent store instead.\
After that I suppose I am supposed to add
performBlockAndWait on all operation where I use all operation that use the new MOC.
I don't use that and doing just fine at least so far
performBlockAndWait is done by executing the block at the same thread and wait till it's complete.
What's the difference between that and just type the code like usual?
I mean there has to be some used, but I am totally missing here.
I can understand performBlock. That'll be like executing something in back ground. Even then it's superseded with Global Central Dyspatch.
Yes there is this new thing called Queue. Okay, if we do something on the same thread, of course everything is done consecutively. Duh.... So why the queue?
Anyone care to explain?
It is possible that the thread that execute the block is not the same with the thread that call performBlockAndWait.
For example, some core data object may only be able to be executed at main thread.
Hence, the performBlockAndWait would do it on a main thread (different thread) and block the current thread.
Also it's saver. Core data would lock things up appropriately preventing collision. If you have several thread accessing the same managed object context, you need to pull this up.
The reason for performBlockAndWait: is it will get and hold the concurrency lock to access Core Data. You can consider it a modernization of the lock/unlock approach, but that's undocumented implementation detail.
If you just execute the code directly, it won't do proper concurrency locking. This is interesting for a number of reasons:
Requests to Core Data won't be properly serialized. That is, if you performBlock: (no wait) the code could end up executing at the same time as other Core Data code, which would probably cause a problem in the coordinator or persistent store.
It… well, I actually don't think it should work. It seems to most of the time in practice, but you're running Core Data without necessary locks. Pretty sure you're into undocumented behaviour here at a minimum.
So:
performBlockAndWait: sets up an environment where your block can access Core Data via the context and waits for the block to complete.
The documentation says nothing about the thread. It's not actually documented as running on the current thread.
Even if it doesn't now, it could be changed in the future to go to secondary threads in at least some circumstances.
Read the parent point again: That's what you're supposed to rely on. The rest is just details.
performBlock: sets up an environment where your block and access Core Data via the context and does not wait for the block to complete.
The documentation says nothing about the thread. It's not actually documented as running on a different thread.
Although unlikely, a future version of the OS could decide to run the block on the current thread at a later time.
Again, the parent point is what you're to rely on. The rest is undocumented details.
I hope that helps. Basically, you're supposed to play dumber than you are when touching these calls. Let the OS do the right thing, just try not to make assumptions about what it's doing. :)
The NSPrivateQueueConcurrencyType constant sets up too many expectations for how this works.
I am using GCD to start a long-running background process ('run_loop') that creates an NSManagedObjectContext ('MOC'), monitors CoreData objects, and sometimes (when they're ready) uploads a serialization of them to a webserver and then deletes them.
I am using AFNetworking for the HTTP calls. The problem is in the request completion handler blocks, as the blocks run in a different thread to the owner of the MOC, which isn't supported by CoreData.
I have tried storing the NSThread from the start of the GCD run_loop block, and using performSelector:onThread:run_thread but this just doesn't seem to actually call the selector at all.
I have tried using dispatch_sync(run_queue) but this doesn't guarantee the thread is the same, only the GCD queue. A different MOC save in the main thread later hangs.
Eventually the only thing that worked was to set a boolean in the completion callback handler, and to introduce extra logic to detect the boolean switch and to perform the MOC work from the main run_loop.
Would anyone be able to suggest a more elegant fix? Or is CoreData simply not compatible with an AFNetworking request started from a GCD queue, and I should look at a lower-level thread control from the start?
Hmm .. the recommended way to deal with MOC and threads is to always make a new MOC that is a sub-moc of your main thread's MOC. Let the main thread do all the saving, but your GCD threads can basically merge changes to the main MOC.
I've had pretty good success working with https://github.com/magicalpanda/MagicalRecord/ to facilitate this in a simpler fashion.