Recently I came to a point where I needed some block of code to execute always on the main thread synchronously. This block can be called from any thread. I solved this problem with the code that was already suggested in this SO answer by #Brad Larson
As the comments to this answer it is evident that the deadlock can occur, but I got into the deadlock very very easily. Please have a look at this code.
-(IBAction) buttonClicked
{
// Dispatch on the global concurrent queue async.
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), ^{
NSString* data = [self getTheString];
NSLog(#"From Background Thread: %#", data);
};
// Dispatch on the main queue async.
dispatch_async(dispatch_get_main_queue(), ^{
NSString* data = [self getTheString];
NSLog(#"From Main Thread: %#", data);
};
}
// This method can be called from any thread so synchronize it.
// Also the code that sets the string variable based on some logic need to execute on main thread.
-(NSString*) getTheString
{
__block NSString* data = nil;
#synchronized(self)
{
// Have some code here that need to be synchronized between threads.
// .......
//
// Create a block to be executed on the main thread.
void (^blockToBeRunOnMainThread)(void) = ^{
// This is just a sample.
// Determining the actual string value can be more complex.
data = #"Tarun";
};
[self dispatchOnMainThreadSynchronously:blockToBeRunOnMainThread];
}
}
- (void) dispatchOnMainThreadSynchronously:(void(^)(void))block
{
if([NSThread isMainThread])
{
if (block)
{
block();
}
}
else
{
dispatch_sync(dispatch_get_main_queue(), ^{
if (block)
{
block();
}
});
}
}
In this piece of code there are two simultaneous asynchronous requests to function getTheString (Assume you have no control over the buttonClicked method and how it calls getTheString api) . Suppose the request from global queue comes first and it is trying to run the block on the main thread synchronously, till that time background thread in waiting for main thread to execute the block synchronously, at the same time request from main queue comes and trying the acquire the lock from background thread, but as background thread in not complete main thread waiting for background thread to complete. Here we have a deadlock on main thread as main thread waiting for background thread to finish, and background thread is waiting for main thread to execute block.
If I remove the #synchronize statement everything works fine as expected. May be I don't need a #synchronize statement here but in same case you may need to have this. Or it can even happen from some other parts of the code.
I tried to search the whole net for the solution and also tried dispatch_semaphore but couldn't solve the issue. May be I am just not doing things the right way.
I assume this is classic problem of deadlock and faced by developers again and again, and probably have solved it to some extent. Can anyone help with this, or point me to right direction?
I would create a synchronous queue (NSOperationQueue would be simplest) and submit the block to be run on the main thread to that queue. The queue would dispatch the blocks in the order received, maintaining the ordering you desire. At the same time, it disassociates the synchronicity between calling the getTheString method and the dispatch to the main thread.
Help me out here or just shed some light on the problem.
I have a scenario where I perform a sync of archived messages on a openfire server and I handle and store all incoming messages with NSOperations and NSOperationQueue.
I want to get notified when the NSOperationQueue is done, but I can't simply count the number of operations it has running. At times the NSOperationQueue has 0 operations because it depends on data to arrive form the server.
The NSOperations start methods
- (void)startArchiveSyncStore:(XMPPIQ *)iq operationID:(NSString *)xmlID {
#autoreleasepool {
if (![self.pendingOperations.archiveStoreInProgress.allKeys containsObject:xmlID]) {
ArchiveStoreOperation *storeOperation = [[ArchiveStoreOperation alloc] initWithMessagesToArchive:iq withID:xmlID delegate:self];
[self.pendingOperations.archiveStoreInProgress setObject:storeOperation forKey:xmlID];
[self.pendingOperations.archiveStoreQueue addOperation:storeOperation];
}
}
}
- (void)startArchiveSycnDownload:(XMPPIQ *)iq operationID:(NSString *)xmlID {
#autoreleasepool {
if (![self.pendingOperations.archiveDownloadInProgress.allKeys containsObject:xmlID]) {
ArchiveDownloadOperation *downloadOperation = [[ArchiveDownloadOperation alloc] initWithMessagesToDownload:iq withID:xmlID delegate:self];
[self.pendingOperations.archiveDownloadInProgress setObject:downloadOperation forKey:xmlID];
[self.pendingOperations.archiveDownloadQueue addOperation:downloadOperation];
}
}
}
And this is the main thread callback performed by the NSOperation:
- (void)archiveStoreDidFinish:(ArchiveStoreOperation *)downloader {
NSString *xmlID = downloader.xmlnsID;
DDLogInfo(#"%# %#", THIS_METHOD, xmlID);
[self.pendingOperations.archiveStoreInProgress removeObjectForKey:xmlID];
}
These operations start when I receive iq stanzas containing lists of the chat history from the openfire server. Then I handle these lists like so:
- (BOOL)xmppStream:(XMPPStream *)sender didReceiveIQ:(XMPPIQ *)iq {
if ([iq isResultIQ]) {
if ([iq elementForName:#"list" xmlns:#"urn:xmpp:archive"]) {
[self startArchiveSycnDownload:iq operationID:[[iq attributeForName:#"id"] stringValue]];
}
if ([iq elementForName:#"chat" xmlns:#"urn:xmpp:archive"]) {
[self startArchiveSyncStore:iq operationID:[[iq attributeForName:#"id"] stringValue]];
}
}
return NO;
}
Any ideas folks ? Thanks in advance...
From my understanding each NSOperation has an isFinished property that you can check for. But, there is a caveat - isFinished doesn't guarantee that the operation has completed successfully. It is set to true if it succeeds but also if it has been cancelled or an error has occurred.
Obviously each queue has a count of the operations [queue.operations count] but as you've said that won't be of use here.
One alternative is to use KVO. You could try setting this up between the other object that you're using and the NSOperationQueue. You could add an observer to the queue and check that no other operations are in effect.
Also, check this SO post here if you haven't already.
I use NSNotificationCenter and post whenever a NSOperation in the last queue finishes. I assume that there is a "last" queue, aka the one that gets spun up after other queue operations have finished.
When you receive the notification check the count of all your NSOperationQueues to see if they are empty.
It's not clear from your question exactly what condition you do consider to be "done" (no operations in the queue and… what?).
One approach is to create a completion operation. As you create the other operations, add each as a dependency of the completion operation. The completion operation can be in some other queue, possibly [NSOperationQueue mainQueue]. When there are no other operations outstanding, the completion operation will execute.
If you have some other condition than other operations outstanding that means the queue is not "done", then you need to explain. If it's that network downloading is in progress, then maybe you need to wrap such downloading in an operation.
You could also use a custom subclass of NSOperation for the completion operation and override -isReady to use whatever criteria it wants to augment the superclass's notion of readiness. Of course, if you do that, you need to generate KVO change notifications when those other criteria change.
From the docs:
The completion block you provide is executed when the value returned by the isFinished method changes to YES. Thus, this block is executed by the operation object after the operation’s primary task is finished or cancelled.
I'm using RestKit/AFNetworking, if that matters.
I have multiple dependencies in my NSOperation in a OperationQueue. I use the completion block to set some variables (appending the results to an array) that my child requires.
(task1,...,taskN) -> taskA
taskA addDependency: task1-taskN
Will taskA receive incomplete data since the child can execute before the completion block is fired?
Reference
Do NSOperations and their completionBlocks run concurrently?
I did a simple test by adding a sleep in my completion block and I had a different result. The completion block runs in the main thread. While all the completion block are sleeping, the child task ran.
As I discuss below under "a few observations", you have no assurances that this final dependent operation will not start before your other sundry AFNetworking completion blocks have finished. It strikes me that if this final operation really needs to wait for these completion blocks to finish, then you have a couple of alternatives:
Use semaphores within each of the n the completion blocks to signal when they're done and have the completion operation wait for n signals; or
Don't queue this final operation up front, but rather have your completion blocks for the individual uploads keep track of how many pending uploads are still incomplete, and when it falls to zero, then initiate the final "post" operation.
As you pointed out in your comments, you could wrap your invocation of the AFNetworking operation and its completion handler in your own operation, at which point you can then use the standard addDependency mechanism.
You could abandon the addDependency approach (which adds an observer on the isFinished key of the operation upon which this operation is dependent, and once all those dependencies are resolved, performs the isReady KVN; the problem being that this can theoretically happen before your completion block is done) and replace it with your own isReady logic. For example, imagine you had a post operation which you could add your own key dependencies and remove them manually in your completion block, rather than having them removed automatically upon isFinished. Thus, you custom operation
#interface PostOperation ()
#property (nonatomic, getter = isReady) BOOL ready;
#property (nonatomic, strong) NSMutableArray *keys;
#end
#implementation PostOperation
#synthesize ready = _ready;
- (void)addKeyDependency:(id)key {
if (!self.keys)
self.keys = [NSMutableArray arrayWithObject:key];
else
[self.keys addObject:key];
self.ready = NO;
}
- (void)removeKeyDependency:(id)key {
[self.keys removeObject:key];
if ([self.keys count] == 0)
self.ready = YES;
}
- (void)setReady:(BOOL)ready {
if (ready != _ready) {
[self willChangeValueForKey:#"isReady"];
_ready = ready;
[self didChangeValueForKey:#"isReady"];
}
}
- (void)addDependency:(NSOperation *)operation{
NSAssert(FALSE, #"You should not use addDependency with this custom operation");
}
Then, your app code could do something like, using addKeyDependency rather than addDependency, and explicitly either removeKeyDependency or cancel in the completion blocks:
PostOperation *postOperation = [[PostOperation alloc] init];
for (NSInteger i = 0; i < numberOfImages; i++) {
NSURL *url = ...
NSURLRequest *request = [NSURLRequest requestWithURL:url];
NSString *key = [url absoluteString]; // or you could use whatever unique value you want
AFHTTPRequestOperation *operation = [[AFHTTPRequestOperation alloc] initWithRequest:request];
[operation setCompletionBlockWithSuccess:^(AFHTTPRequestOperation *operation, id responseObject) {
// update your model or do whatever
// now inform the post operation that this operation is done
[postOperation removeKeyDependency:key];
} failure:^(AFHTTPRequestOperation *operation, NSError *error) {
// handle the error any way you want
// perhaps you want to cancel the postOperation; you'd either cancel it or remove the dependency
[postOperation cancel];
}];
[postOperation addKeyDependency:key];
[queue addOperation:operation];
}
[queue addOperation:postOperation];
This is using AFHTTPRequestOperation, and you'd obviously replace all of this logic with the appropriate AFNetworking operation for your upload, but hopefully it illustrates the idea.
Original answer:
A few observations:
As I think you concluded, when your operation completes, it (a) initiates its completion block; (b) makes the queue available for other operations (either operations that had not yet started because of maxConcurrentOperationCount, or because of dependencies between the operations). I do not believe that you have any assurances that the completion block will be done before that next operation commences.
Empirically, it looks like the dependent operation does not actually trigger until after the completion blocks are done, but (a) I don't see that documented anywhere and (b) this is moot because if you're using AFNetworking's own setCompletionBlockWithSuccess, it ends up dispatching the block asynchronously to the main queue (or the defined successCallbackQueue), thereby thwarting any (undocumented) assurances of synchrony.
Furthermore, you say that the completion block runs in the main thread. If you're talking about the built in NSOperation completion block, you have no such assurances. In fact, the setCompletionBlock documentation says:
The exact execution context for your completion block is not guaranteed but is typically a secondary thread. Therefore, you should not use this block to do any work that requires a very specific execution context. Instead, you should shunt that work to your application’s main thread or to the specific thread that is capable of doing it. For example, if you have a custom thread for coordinating the completion of the operation, you could use the completion block to ping that thread.
But if you're talking about one of AFNetworking's custom completion blocks, e.g. those that you might set with AFHTTPRequestOperation's setCompletionBlockWithSuccess, then, yes, it's true that those are generally dispatched back to the main queue. But AFNetworking does this using the standard completionBlock mechanism, so the above concerns still apply.
It matters if your NSOperation is a subclass of AFHTTPRequestOperation. AFHTTPRequestOperation uses the NSOperation's property completionBlock for its own purpose in method setCompletionBlockWithSuccess:failure. In that case, don't set the property completionBlock yourself!
It seems, AFHTTPRequestOperation's success and failure handler will run on the main thread.
Otherwise, the execution context of NSOperation's completion block is "undefined". That means, the completion block can execute on any thread/queue. In fact it executes on some private queue.
IMO, this is the preferred approach, unless the execution context shall be explicitly specified by the call-site. Executing completion handlers on threads or queues which instances are accessible (the main thread for example) can easily cause dead locks by an unwary developer.
Edit:
If you want to start a dependent operation after the completion block of the parent operation has been finished, you can solve that by making the completion block content itself a NSBlockOperation (a new parent) and add this operation as a dependency to the children operation and start it in a queue. You may realize, that this quickly becomes unwieldy, though.
Another approach would require an utility class or class library which is especially suited to solve asynchronous problems in a more concise and easy way. ReactiveCocoa would be capable to solve such (an easy) problem. However, it's unduly complex and it actually has a "learning curve" - and a steep one. I wouldn't recommend it, unless you agree to spend a few weeks in learning it and have a lot other asynchronous use cases and even much more complex ones.
A simpler approach would utilize "Promises" which are pretty common in JavaScript, Python, Scala and a few other languages.
Now, please read carefully, the (easy) solution is actually below:
"Promises" (sometimes called Futures or Deferred) represent the eventual result of an asynchronous task. Your fetch request is such asynchronous task. But instead specifying a completion handler, the asynchronous method/task returns a Promise:
-(Promise*) fetchThingsWithURL:(NSURL*)url;
You obtain the result - or the error - with registering a success handler block or a failure handler block like so:
Promise* thingsPromise = [self fetchThingsWithURL:url];
thingsPromise.then(successHandlerBlock, failureHandlerBlock);
or, the blocks inlined:
thingsPromise.then(^id(id things){
// do something with things
return <result of success handler>
}, ^id(NSError* error){
// Ohps, error occurred
return <result of failure handler>
});
And shorter:
[self fetchThingsWithURL:url]
.then(^id(id result){
return [self.parser parseAsync:result];
}, nil);
Here, parseAsync: is an asynchronous method which returns a Promise. (Yes, a Promise).
You might wonder how to get the result from the parser?
[self fetchThingsWithURL:url]
.then(^id(id result){
return [self.parser parseAsync:result];
}, nil)
.then(^id(id parserResult){
NSLog(#"Parser returned: %#", parserResult);
return nil; // result not used
}, nil);
This actually starts async task fetchThingsWithURL:. Then when finished successfully, it starts async task parseAsync:. Then when this finished successfully, it prints the result, otherwise it prints the error.
Invoking several asynchronous tasks sequentially, one after the other, is called "continuation" or "chaining".
Note that the whole statement above is asynchronous! That is, when you wrap the above statement into a method, and execute it, the method returns immediately.
You might wonder how to catch any errors, say fetchThingsWithURL: fails, or parseAsync::
[self fetchThingsWithURL:url]
.then(^id(id result){
return [self.parser parseAsync:result];
}, nil)
.then(^id(id parserResult){
NSLog(#"Parser returned: %#", parserResult);
return nil; // result not used
}, nil)
.then(/*succes handler ignored*/, ^id (NSError* error){
// catch any error
NSLog(#"ERROR: %#", error);
return nil; // result not used
});
Handlers execute after the corresponding task has been finished (of course). If the task succeeds, the success handler will be called (if any). If the tasks fails, the error handler will be called (if any).
Handlers may return a Promise (or any other object). For example, if an asynchronous task finished successfully, its success handler will be invoked which starts another asynchronous task, which returns the promise. And when this is finished, yet another one can be started, and so force. That's "continuation" ;)
You can return anything from a handler:
Promise* finalResult = [self fetchThingsWithURL:url]
.then(^id(id result){
return [self.parser parseAsync:result];
}, nil)
.then(^id(id parserResult){
return #"OK";
}, ^id(NSError* error){
return error;
});
Now, finalResult will either eventually become the value #"OK" or an NSError.
You can save the eventual results into an array:
array = #[
[self task1],
[self task2],
[self task3]
];
and then continue when all tasks have been finished successfully:
[Promise all:array].then(^id(results){
...
}, ^id (NSError* error){
...
});
Setting a promise's value will be called: "resolving". You can resolve a promise only ONCE.
You may wrap any asynchronous method with a completion handler or completion delegates into a method which returns a promise:
- (Promise*) fetchUserWithURL:(NSURL*)url
{
Promise* promise = [Promise new];
HTTPOperation* op = [[HTTPOperation alloc] initWithRequest:request
success:^(NSData* data){
[promise fulfillWithValue:data];
}
failure:^(NSError* error){
[promise rejectWithReason:error];
}];
[op start];
return promise;
}
Upon completion of the task, the promise can be "fulfilled" passing it the result value, or it can be "rejected" passing it the reason (error).
Depending on the actual implementation, a Promise can also be cancelled. Say, you hold a reference to a request operation:
self.fetchUserPromise = [self fetchUsersWithURL:url];
You can cancel the asynchronous task as follows:
- (void) viewWillDisappear:(BOOL)animate {
[super viewWillDisappear:animate];
[self.fetchUserPromise cancel];
self.fetchUserPromise = nil;
}
In order to cancel the associated async task, register a failure handler in the wrapper:
- (Promise*) fetchUserWithURL:(NSURL*)url
{
Promise* promise = [Promise new];
HTTPOperation* op = ...
[op start];
promise.then(nil, ^id(NSError* error){
if (promise.isCancelled) {
[op cancel];
}
return nil; // result unused
});
return promise;
}
Note: you can register success or failure handlers, when, where and as many as you want.
So, you can do a lot with promises - and even more than in this brief introduction. If you read up to here, you might get an idea how to solve your actual problem. It's right there - and it's a few lines of code.
I admit, that this short introduction into promises was quite rough and it's also quite new to Objective-C developers, and may sound uncommon.
You can read a lot about promises in the JS community. There are one or three implementations in Objective-C. The actual implementation won't exceed a few hundred lines of code. It happens, that I'm the author of one of it:
RXPromise.
Take it with a grain of salt, I'm probably totally biased, and apparently all others ever dealt with Promises, too. ;)
I wanted to clean up one of my projects and extracted parts of my source that I often reuse, in a single class.
This class handles some requests to a web service, everything is fine so far ;). Until I extracted the code to its own class, I handled those requests with threads and callbacks in the calling class.
Now I have a "best practice" question:
In my code I do something like(simplified):
(void)foo{
Helper *h =[[Helper alloc]init];
[h doRequest];
}
doRequest performs a network action(in its own class)and I have to wait until this is request is finished. So I need a callback or something like this.
Should I simply thread doRequest incl. waituntildone=YES?
Do I have to thread the networking in the Helper class too? Or is it enough to call the method threaded something like this:
[NSThread detachNewThreadSelector:#selector(h doRequest) toTarget:self withObject:nil];
What is the best practice to get a callback from doRequest to the caller class after it has completed it’s tasks so that I can handle the returned values from the web service?
Thanks in advance.
Johannes
Given doRequest does not return until the request is done you could do
- (void)fooCompletion:(void (^)(void))completion {
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
Helper *h =[[Helper alloc]init];
[h doRequest];
if (completion) {
dispatch_async(dispatch_get_main_queue(), ^{
// doRequest is done
completion();
});
}
});
}
To call the method:
[self fooCompletion:^{
// do something after doRequest is done
}];
I personally prefer calling performSelectorOnMainThread:withObject:waitUntilDone: at the end of any helper threads that need to send information back.
[self performSelectorOnMainThread:#selector(infoFromService:) withObject:aDictionaryWithInfo waitUntilDone:NO];
- (void)infoFromService:(NSDictionary *)aDictionary {
//Process all the information and update UI
}
Be sure to always use the main thread for any UI updates even if they happen in the middle of the worker thread, for example updating a count of how much information has been downloaded. Use the same technique to call the main thread with the relevant information.
In my application, I let a progress indicator starts animation before I send a HTTP request.
The completion handler is defined in a block. After I get the response data, I hide the progress indicator from inside the block. My question is, as I know, UI updates must be performed in the main thread. How can I make sure it?
If I define a method in the window controller which updates UI, and let the block calls the method instead of updating UI directly, is it a solution?
Also, if your app targets iOS >= 4 you can use Grand Central Dispatch:
dispatch_async(dispatch_get_main_queue(), ^{
// This block will be executed asynchronously on the main thread.
});
This is useful when your custom logic cannot easily be expressed with the single selector and object arguments that the performSelect… methods take.
To execute a block synchronously, use dispatch_sync() – but make sure you’re not currently executing on the main queue or GCD will deadlock.
__block NSInteger alertResult; // The __block modifier makes alertResult writable
// from a referencing block.
void (^ getResponse)() = ^{
NSAlert *alert = …;
alertResult = [NSAlert runModal];
};
if ([NSThread isMainThread]) {
// We're currently executing on the main thread.
// We can execute the block directly.
getResponse();
} else {
dispatch_sync(dispatch_get_main_queue(), getResponse);
}
// Check the user response.
if (alertResult == …) {
…
}
You probably misunderstood something. Using blocks doesn't mean that your code is running in a background thread. There are many plugins that work asynchronously (in another thread) and use blocks.
There are a few options to solve your problem.
You can check if your code is running in the main thread my using [NSThread isMainThread]. That helps you to make sure that you're not in the background.
You can also perform actions in the main or background by using performSelectorInMainThread:SEL or performSelectorInBackground:SEL.
The app immediately crashes when you're trying to call the UI from a bakcground thread so it's quite easy to find a bug.