I think I might have an async problem going on here, which bites cause I thought I had solved it. Anyway, I am making a bunch of web service calls like so:
//get the client data
__block NSArray* arrClientPAs;
[dataManager getJSONData:strWebService withBlock:^(id results, NSError* error) {
if (error) {
UIAlertView* alert = [[UIAlertView alloc] initWithTitle:#"Getting Client Data Error!" message:error.description delegate:nil cancelButtonTitle:NSLocalizedString(#"Okay", nil) otherButtonTitles:nil, nil];
[alert show];
} else {
arrClientPAs = results;
}
}];
and getJSONData is like so:
- (void) getJSONData : (NSString*) strQuery withBlock:(void (^)(id, NSError *))completion {
NSDictionary* dictNetworkStatus = [networkManager checkNetworkConnectivity];
NetworkStatus networkStatus = [[dictNetworkStatus objectForKey:#"Status"] intValue];
if (networkStatus != NotReachable) {
//set up the url for webservice
NSURL* url = [NSURL URLWithString:strQuery];
NSMutableURLRequest* urlRequest = [NSMutableURLRequest requestWithURL:url];
//set up the url connection
__block id results;
[NSURLConnection sendAsynchronousRequest:urlRequest queue:[NSOperationQueue mainQueue] completionHandler:
^(NSURLResponse* response, NSData* jsonData, NSError* error) {
if (error) {
completion(nil, error);
return;
}
results = [NSJSONSerialization JSONObjectWithData:jsonData options:NSJSONReadingMutableContainers | NSJSONReadingMutableLeaves | NSJSONReadingAllowFragments error:&error];
completion(results, nil);
}];
} else {
//not connected to a network - data is going to have to come from coredata
}
}
In the first block, if I log arrClientData I can see the data that I am expecting but when I log arrClientData after it it is nil. I was following this SO thread - How to return a BOOL with asynchronous request in a method? (Objective-C) and a couple of others.
Obviously I am trying to get the data after the async call is made. Any help would be appreciated.
The problem lies, I think, in what "asynchronous" means. Here's a diagram:
Step One
__block result;
Step Two - do something asynchonous, including e.g. setting result
Step Three
What order do things happen in here? Step Three happens before Step Two gets finished. That is what asynchronous means: it means, "go right on with this code, don't wait for the asynchronous stuff to finish." So at the time Step Three happens, the result variable has not yet been set to anything.
So, you are just misleading the heck out of yourself with your __block result. __block or no __block, there is no way you are going to find out out what the result is afterwards, because there is no "afterwards". Your code has completed before your __block result is even set. That is why asynchronous code uses a callback (eg. your completion block) which does run afterwards, because it is sequentially part of (appended to) the asynchronous code. You can hand your result downwards through the callback, but you cannot usefully set it upwards from within the block and expect to retrieve it later.
So, your overall structure is like this:
__block NSArray* arrClientPAs; // it's nil
[call getJSONdata] = step one
[call sendAsynchronousRequest]
do the block _asynchronously_ = step two, tries to set arrClientPAs somehow
step three! This happens _before_ step two, ...
... and this entire method ends and is torn down ...
... and arrClientPAs is still nil! 🌻
I repeat: you cannot pass any information UP out of an asynchronous block. You can only go DOWN. You need your asynchronous block to call some method of some independently persistent object to hand it your result and tell it to use that result (and do it carefully, on the main thread, or you will cause havoc). You cannot use any automatic variable for this purpose, such as your declared NSArray variable arrClientPAs; there is no automatic scope any more, the method is over, the automatic variable is gone, there is no more code to run.
Check the value of the 'error 'variable after call:
results = [NSJSONSerialization JSONObjectWithData:jsonData options:NSJSONReadingMutableContainers | NSJSONReadingMutableLeaves | NSJSONReadingAllowFragments error:&error];
If 'error' isn't nil there is a problem with data which you get in your completion block.
You are mixing styles and confusing the purpose of __block.
Note: When you call a method that will be executed asynchronously you are creating a new execution path which will be executed at some point in the future (which includes immediately) on some thread.
In your getJSONData method you use a __block qualified variable, results, when you should not. The variable is only required within the block and should be declared there:
//set up the url connection
[NSURLConnection sendAsynchronousRequest:urlRequest queue:[NSOperationQueue mainQueue] completionHandler:
^(NSURLResponse* response, NSData* jsonData, NSError* error)
{
if (error) {
completion(nil, error);
return;
}
id results = [NSJSONSerialization JSONObjectWithData:jsonData options:NSJSONReadingMutableContainers | NSJSONReadingMutableLeaves | NSJSONReadingAllowFragments error:&error];
completion(results, nil);
}];
Declaring the variable outside of the block and adding __block just adds pointless complexity. After the call to sendAsynchronousRequest, returns before the request has been performed, the value of results would not be the value assigned in the block. The call to the completion block is performed on a different execution path and probably will not even be executed until after the call to getJSONData has returned.
However what is correct about your getJSONData method is its model - it takes a completion block which sendAsynchronousRequest's own completion handler will call. This is what is incorrect about your call to getJSONData - the completion block you pass does not pass on the results to another block or pass them to some object, but instead assigns them a local variable, arrClientPAs, declared before the call. This is the same situation as described above for getJSONData and will fail for the same reasons - it is not the arrClientPAs fails to "retain the data" but that you are reading it on in the current execution path before another execution path has written any data to it.
You can address this problem the same way getJSONData does - the enclosing method (not included in your question) can take a completion block (code entered directly into answer, expect typos!):
- (void) getTheClientData: ... completionHandler:(void (^)(id))handler
{
...
//get the client data
[dataManager getJSONData:strWebService withBlock:^(id results, NSError* error) {
if (error) {
UIAlertView* alert = [[UIAlertView alloc] initWithTitle:#"Getting Client Data Error!" message:error.description delegate:nil cancelButtonTitle:NSLocalizedString(#"Okay", nil) otherButtonTitles:nil, nil];
[alert show];
} else {
handler(results); // "return" the result to the handler
}
}];
There is another approach. If and only if getClientData is not executing on the main thread and you wish its behaviour to be synchronous and to return the result of the request then you can issue a sendSynchronousRequest:returningResponse:error: instead of an asynchronous one. This will block the thread getClientData is executing on until the request completes.
In general if you have an asynchronous method which you cannot replace by a synchronous one but require synchronous behaviour you can use semaphores to block your current thread until the asynchronous call completes. For an example of how to do this see this answer.
HTH
I have been asking and trying to understand how completion handlers work. Ive used quite a few and I've read many tutorials. i will post the one I use here, but I want to be able to create my own without using someone else's code as a reference.
I understand this completion handler where this caller method:
-(void)viewDidLoad{
[newSimpleCounter countToTenThousandAndReturnCompletionBLock:^(BOOL completed){
if(completed){
NSLog(#"Ten Thousands Counts Finished");
}
}];
}
and then in the called method:
-(void)countToTenThousandAndReturnCompletionBLock:(void (^)(BOOL))completed{
int x = 1;
while (x < 10001) {
NSLog(#"%i", x);
x++;
}
completed(YES);
}
Then I sorta came up with this one based on many SO posts:
- (void)viewDidLoad{
[self.spinner startAnimating];
[SantiappsHelper fetchUsersWithCompletionHandler:^(NSArray *users) {
self.usersArray = users;
[self.tableView reloadData];
}];
}
which will reload the tableview with the received data users after calling this method:
typedef void (^Handler)(NSArray *users);
+(void)fetchUsersWithCompletionHandler:(Handler)handler {
NSURL *url = [NSURL URLWithString:#"http://www.somewebservice.com"];
NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:url cachePolicy:NSURLRequestReloadIgnoringLocalAndRemoteCacheData timeoutInterval:10];
[request setHTTPMethod: #"GET"];
**// We dispatch a queue to the background to execute the synchronous NSURLRequest**
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^{
// Perform the request
NSURLResponse *response;
NSError *error = nil;
NSData *receivedData = [NSURLConnection sendSynchronousRequest:request
returningResponse:&response
error:&error];
if (error) { **// If an error returns, log it, otherwise log the response**
// Deal with your error
if ([response isKindOfClass:[NSHTTPURLResponse class]]) {
NSHTTPURLResponse *httpResponse = (NSHTTPURLResponse*)response;
NSLog(#"HTTP Error: %d %#", httpResponse.statusCode, error);
return;
}
NSLog(#"Error %#", error);
return;
}
**// So this line won't get processed until the response from the server is returned?**
NSString *responseString = [[NSString alloc] initWithData:receivedData encoding:NSUTF8StringEncoding];
NSArray *usersArray = [[NSArray alloc] init];
usersArray = [NSJSONSerialization JSONObjectWithData:[responseString dataUsingEncoding:NSASCIIStringEncoding] options:0 error:nil];
// Finally when a response is received and this line is reached, handler refers to the block passed into this called method...so it dispatches back to the main queue and returns the usersArray
if (handler){
dispatch_sync(dispatch_get_main_queue(), ^{
handler(usersArray);
});
}
});
}
I can see it in the counter example, that the called method (with the passed block) will never exit the loop until it is done. Thus the 'completion' part actually depends on the code inside the called method, not the block passed into it?
In this case the 'completion' part depends on the fact that the call to NSURLRequest is synchronous. What if it was asynchronous? How would I be able to hold off calling the block until my data was populated by the NSURLResponse?
Your first example is correct and complete and the best way to understand completion blocks. There is no further magic to them. They do not automatically get executed ever. They are executed when some piece of code calls them.
As you note, in the latter example, it is easy to call the completion block at the right time because everything is synchronous. If it were asynchronous, then you need to store the block in an instance variable, and call it when the asynchronous operation completed. It is up to you to arrange to be informed when the operation completes (possibly using its completion handler).
Do be careful when you store a block in an ivar. One of your examples includes:
self.usersArray = users;
The call to self will cause the block to retain self (the calling object). This can easily create a retain loop. Typically, you need to take a weak reference to self like this:
- (void)viewDidLoad{
[self.spinner startAnimating];
__weak typeof(self) weakSelf = self;
[SantiappsHelper fetchUsersWithCompletionHandler:^(NSArray *users) {
typeof(self) strongSelf = weakSelf;
if (strongSelf) {
[strongSelf setUsersArray:users];
[[strongSelf tableView] reloadData];
}
}];
}
This is a fairly pedantic version of the weakSelf/strongSelf pattern, and it could be done a little simpler in this case, but it demonstrates all the pieces you might need. You take a weak reference to self so that you don't create a retain loop. Then, in the completely block, you take a strong reference so that self so that it can't vanish on you in the middle of your block. Then you make sure that self actually still exists, and only then proceed. (Since messaging nil is legal, you could have skipped the strongSelf step in this particular case, and it would be the same.)
Your first example (countToTenThousandAndReturnCompletionBLock) is actually a synchronous method. A completion handler doesn't make much sense here: Alternatively, you could call that block immediately after the hypothetical method countToTenThousand (which is basically the same, just without the completion handler).
Your second example fetchUsersWithCompletionHandler: is an asynchronous method. However, it's actually quite suboptimal:
It should somehow signal the call-site that the request may have failed. That is, either provide an additional parameter to the completion handler, e.g. " NSError* error or us a single parameter id result. In the first case, either error or array is not nil, and in the second case, the single parameter result can be either an error object (is kind of NSError) or the actual result (is kind of NSArray).
In case your request fails, you miss to signal the error to the call-site.
There are code smells:
As a matter of fact, the underlying network code implemented by the system is asynchronous. However, the utilized convenient class method sendSynchronousRequest: is synchronous. That means, as an implementation detail of sendSynchronousRequest:, the calling thread is blocked until after the result of the network response is available. And this_blocking_ occupies a whole thread just for waiting. Creating a thread is quite costly, and just for this purpose is a waste. This is the first code smell. Yes, just using the convenient class method sendSynchronousRequest: is by itself bad programming praxis!
Then in your code, you make this synchronous request again asynchronous through dispatching it to a queue.
So, you are better off using an asynchronous method (e.g. sendAsynchronous...) for the network request, which presumable signals the completion via a completion handler. This completion handler then may invoke your completion handler parameter, taking care of whether you got an actual result or an error.
When using methods which return blocks they can be very convenient.
However, when you have to string a few of them together it gets messy really quickly
for instance, you have to call 4 URLs in succession:
[remoteAPIWithURL:url1 success:^(int status){
[remoteAPIWithURL:url2 success:^(int status){
[remoteAPIWithURL:url3 success:^(int status){
[remoteAPIWithURL:url2 success:^(int status){
//succes!!!
}];
}];
}];
}];
So for every iteration I go one level deeper, and I don't even handle errors in the nested blocks yet.
It gets worse when there is an actual loop. For instance, say I want to upload a file in 100 chunks:
- (void) continueUploadWithBlockNr:(int)blockNr
{
if(blocknr>=100)
{
//success!!!
}
[remoteAPIUploadFile:file withBlockNr:blockNr success:^(int status)
{
[self continueUploadWithBlockNr:blockNr];
}];
}
This feels very unintuitive, and gets very unreadable very quick.
In .Net they solved all this using the async and await keyword, basically unrolling these continuations into a seemingly synchronous flow.
What is the best practice in Objective C?
Your question immediately made me think of recursion. Turns out, Objective-c blocks can be used in recursion. So I came up with the following solution, which is easy to understand and can scale to N tasks pretty nicely.
// __block declaration of the block makes it possible to call the block from within itself
__block void (^urlFetchBlock)();
// Neatly aggregate all the urls you wish to fetch
NSArray *urlArray = #[
[NSURL URLWithString:#"http://www.google.com"],
[NSURL URLWithString:#"http://www.stackoverflow.com"],
[NSURL URLWithString:#"http://www.bing.com"],
[NSURL URLWithString:#"http://www.apple.com"]
];
__block int urlIndex = 0;
// the 'recursive' block
urlFetchBlock = [^void () {
if (urlIndex < (int)[urlArray count]){
[self remoteAPIWithURL:[urlArray objectAtIndex:index]
success:^(int theStatus){
urlIndex++;
urlFetchBlock();
}
failure:^(){
// handle error.
}];
}
} copy];
// initiate the url requests
urlFetchBlock();
One way to reduce nesting is to define methods that return the individual blocks. In order to facilitate the data sharing which is done "auto-magically" by the Objective C compiler through closures, you would need to define a separate class to hold the shared state.
Here is a rough sketch of how this can be done:
typedef void (^WithStatus)(int);
#interface AsyncHandler : NSObject {
NSString *_sharedString;
NSURL *_innerUrl;
NSURL *_middleUrl;
WithStatus _innermostBlock;
}
+(void)handleRequest:(WithStatus)innermostBlock
outerUrl:(NSURL*)outerUrl
middleUrl:(NSURL*)middleUrl
innerUrl:(NSURL*)innerUrl;
-(WithStatus)outerBlock;
-(WithStatus)middleBlock;
#end
#implementation AsyncHandler
+(void)handleRequest:(WithStatus)innermostBlock
outerUrl:(NSURL*)outerUrl
middleUrl:(NSURL*)middleUrl
innerUrl:(NSURL*)innerUrl {
AsyncHandler *h = [[AsyncHandler alloc] init];
h->_innermostBlock = innermostBlock;
h->_innerUrl = innerUrl;
h->_middleUrl = middleUrl;
[remoteAPIWithURL:outerUrl success:[self outerBlock]];
}
-(WithStatus)outerBlock {
return ^(int success) {
_sharedString = [NSString stringWithFormat:#"Outer: %i", success];
[remoteAPIWithURL:_middleUrl success:[self middleBlock]];
};
}
-(WithStatus)middleBlock {
return ^(int success) {
NSLog("Shared string: %#", _sharedString);
[remoteAPIWithURL:_innerUrl success:_innermostBlock];
};
}
#end
Note: All of this assumes ARC; if you are compiling without it, you need to use Block_copy in the methods returning blocks. You would also need to do a copy in the calling code below.
Now your original function can be re-written without the "Russian doll" nesting, like this:
[AsyncHandler
handleRequest:^(int status){
//succes!!!
}
outerUrl:[NSURL #"http://my.first.url.com"]
middleUrl:[NSURL #"http://my.second.url.com"]
innerUrl:[NSURL #"http://my.third.url.com"]
];
Iterative algorithm:
Create a __block variable (int urlNum) to keep track of the current URL (inside an NSArray of them).
Have the onUrlComplete block fire off the next request until all URLs have been loaded.
Fire the first request.
When all URLs have been loaded, do the "//success!" dance.
Code written without the aid of XCode (meaning, there may be compiler errors -- will fix if necessary):
- (void)loadUrlsAsynchronouslyIterative:(NSArray *)urls {
__block int urlNum = 0;
void(^onUrlComplete)(int) = nil; //I don't remember if you can call a block from inside itself.
onUrlComplete = ^(int status) {
if (urlNum < urls.count) {
id nextUrl = urls[urlNum++];
[remoteAPIWithURL:nextUrl success:onUrlComplete];
} else {
//success!
}
}
onUrlComplete(0); //fire first request
}
Recursive algorithm:
Create a method to load all the remaining URLs.
When remaining URLs is empty, fire "onSuccess".
Otherwise, fire request for the next URL and provide a completion block that recursively calls the method with all but the first remaining URLs.
Complications: we declared the "onSuccess" block to accept an int status parameter, so we pass the last status variable down (including a "default" value).
Code written without the aid of XCode (bug disclaimer here):
- (void)loadUrlsAsynchronouslyRecursive:(NSArray *)remainingUrls onSuccess:(void(^)(int status))onSuccess lastStatus:(int)lastStatus {
if (remainingUrls.count == 0) {
onSuccess(lastStatus);
return;
}
id nextUrl = remainingUrls[0];
remainingUrls = [remainingUrls subarrayWithRange:NSMakeRange(1, remainingUrls.count-1)];
[remoteAPIWithUrl:nextUrl onSuccess:^(int status) {
[self loadUrlsAsynchronouslyRecursive:remainingUrls onSuccess:onSuccess lastStatus:status];
}];
}
//fire first request:
[self loadUrlsAsynchronouslyRecursive:urls onSuccess:^(int status) {
//success here!
} lastStatus:0];
Which is better?
The iterative algorithm is simple and concise -- if you're comfortable playing games with __block variables and scopes.
Alternatively, the recursive algorithm doesn't require __block variables and is fairly simple, as recursive algorithms go.
The recursive implementation is more re-usable that the iterative one (as implemented).
The recursive algorithm might leak (it requires a reference to self), but there are several ways to fix that: make it a function, use __weak id weakSelf = self;, etc.
How easy would it be to add error-handling?
The iterative implementation can easily be extended to check the value of status, at the cost of the onUrlComplete block becoming more complex.
The recursive implementation is perhaps not as straight-forward to extend -- primarily because it is re-usable. Do you want to cancel loading more URLs when the status is such-and-such? Then pass down a status-checking/error-handling block that accepts int status and returns BOOL (for example YES to continue, NO to cancel). Or perhaps modify onSuccess to accept both int status and NSArray *remainingUrls -- but you'll need to call loadUrlsAsynchronouslyRecursive... in your onSuccess block implementation.
You said (in a comment), “asynchronous methods offer easy asynchronisity without using explicit threads.” But your complaint seems to be that you're trying to do something with asynchronous methods, and it's not easy. Do you see the contradiction here?
When you use a callback-based design, you sacrifice the ability to express your control flow directly using the language's built-in structures.
So I suggest you stop using a callback-based design. Grand Central Dispatch (GCD) makes it easy (that word again!) to perform work “in the background”, and then call back to the main thread to update the user interface. So if you have a synchronous version of your API, just use it in a background queue:
- (void)interactWithRemoteAPI:(id<RemoteAPI>)remoteAPI {
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
// This block runs on a background queue, so it doesn't block the main thread.
// But it can't touch the user interface.
for (NSURL *url in #[url1, url2, url3, url4]) {
int status = [remoteAPI syncRequestWithURL:url];
if (status != 0) {
dispatch_async(dispatch_get_main_queue(), ^{
// This block runs on the main thread, so it can update the
// user interface.
[self remoteRequestFailedWithURL:url status:status];
});
return;
}
}
});
}
Since we're just using normal control flow, it's straightforward to do more complicated things. Say we need to issue two requests, then upload a file in chunks of at most 100k, then issue one more request:
#define AsyncToMain(Block) dispatch_async(dispatch_get_main_queue(), Block)
- (void)uploadFile:(NSFileHandle *)fileHandle withRemoteAPI:(id<RemoteAPI>)remoteAPI {
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
int status = [remoteAPI syncRequestWithURL:url1];
if (status != 0) {
AsyncToMain(^{ [self remoteRequestFailedWithURL:url1 status:status]; });
return;
}
status = [remoteAPI syncRequestWithURL:url2];
if (status != 0) {
AsyncToMain(^{ [self remoteRequestFailedWithURL:url2 status:status]; });
return;
}
while (1) {
// Manage an autorelease pool to avoid accumulating all of the
// 100k chunks in memory simultaneously.
#autoreleasepool {
NSData *chunk = [fileHandle readDataOfLength:100 * 1024];
if (chunk.length == 0)
break;
status = [remoteAPI syncUploadChunk:chunk];
if (status != 0) {
AsyncToMain(^{ [self sendChunkFailedWithStatus:status]; });
return;
}
}
}
status = [remoteAPI syncRequestWithURL:url4];
if (status != 0) {
AsyncToMain(^{ [self remoteRequestFailedWithURL:url4 status:status]; });
return;
}
AsyncToMain(^{ [self uploadFileSucceeded]; });
});
}
Now I'm sure you're saying “Oh yeah, that looks great.” ;^) But you might also be saying “What if RemoteAPI only has asynchronous methods, not synchronous methods?”
We can use GCD to create a synchronous wrapper for an asynchronous method. We need to make the wrapper call the async method, then block until the async method calls the callback. The tricky bit is that perhaps we don't know which queue the async method uses to invoke the callback, and we don't know if it uses dispatch_sync to call the callback. So let's be safe by calling the async method from a concurrent queue.
- (int)syncRequestWithRemoteAPI:(id<RemoteAPI>)remoteAPI url:(NSURL *)url {
__block int outerStatus;
dispatch_semaphore_t sem = dispatch_semaphore_create(0);
[remoteAPI asyncRequestWithURL:url completion:^(int status) {
outerStatus = status;
dispatch_semaphore_signal(sem);
}];
dispatch_semaphore_wait(sem, DISPATCH_TIME_FOREVER);
dispatch_release(sem);
return outerStatus;
}
UPDATE
I will respond to your third comment first, and your second comment second.
Third Comment
Your third comment:
Last but not least, your solution of dedicating a separate thread to wrap around the synchronous version of a call is more costly than using the async alternatives. a Thread is an expensive resource, and when it is blocking you basically have lost one thread. Async calls (the ones in the OS libraries at least) are typically handled in a much more efficient way. (For instance, if you would request 10 urls at the same time, chances are it will not spin up 10 threads (or put them in a threadpool))
Yes, using a thread is more expensive than just using the asynchronous call. So what? The question is whether it's too expensive. Objective-C messages are too expensive in some scenarios on current iOS hardware (the inner loops of a real-time face detection or speech recognition algorithm, for example), but I have no qualms about using them most of the time.
Whether a thread is “an expensive resource” really depends on the context. Let's consider your example: “For instance, if you would request 10 urls at the same time, chances are it will not spin up 10 threads (or put them in a threadpool)”. Let's find out.
NSURL *url = [NSURL URLWithString:#"http://1.1.1.1/"];
NSURLRequest *request = [NSURLRequest requestWithURL:url];
for (int i = 0; i < 10; ++i) {
[NSURLConnection sendAsynchronousRequest:request queue:[NSOperationQueue mainQueue] completionHandler:^(NSURLResponse *response, NSData *data, NSError *error) {
NSLog(#"response=%# error=%#", response, error);
}];
}
So here I am using Apple's own recommended +[NSURLConnection sendAsynchronousRequest:queue:completionHandler:] method to send 10 requests asynchronously. I've chosen the URL to be non-responsive, so I can see exactly what kind of thread/queue strategy Apple uses to implement this method. I ran the app on my iPhone 4S running iOS 6.0.1, paused in the debugger, and took a screen shot of the Thread Navigator:
You can see that there are 10 threads labeled com.apple.root.default-priority. I've opened three of them so you can see that they are just normal GCD queue threads. Each calls a block defined in +[NSURLConnection sendAsynchronousRequest:…], which just turns around and calls +[NSURLConnection sendSynchronousRequest:…]. I checked all 10, and they all have the same stack trace. So, in fact, the OS library does spin up 10 threads.
I bumped the loop count from 10 to 100 and found that GCD caps the number of com.apple.root.default-priority threads at 64. So my guess is the other 36 requests I issued are queued up in the global default-priority queue, and won't even start executing until some of the 64 “running” requests finish.
So, is it too expensive to use a thread to turn an asynchronous function into a synchronous function? I'd say it depends on how many of these you plan to do simultaneously. I would have no qualms if the number's under 10, or even 20.
Second Comment
Which brings me to your second comment:
However, when you have: do these 3 things at the same time, and when 'any' of them is finished then ignore the rest and do these 3 calls at the same time and when 'all' of them finish then succes.
These are cases where it's easy to use GCD, but we can certainly combine the GCD and async approaches to use fewer threads if you want, while still using the languages native tools for control flow.
First, we'll make a typedef for the remote API completion block, just to save typing later:
typedef void (^RemoteAPICompletionBlock)(int status);
I'll start the control flow the same way as before, by moving it off the main thread to a concurrent queue:
- (void)complexFlowWithRemoteAPI:(id<RemoteAPI>)remoteAPI {
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
First we want to issue three requests simultaneously and wait for one of them to succeed (or, presumably, for all three to fail).
So let's say we have a function, statusOfFirstRequestToSucceed, that issues any number of asynchronous remote API requests and waits for the first to succeed. This function will provide the completion block for each async request. But the different requests might take different arguments… how can we pass the API requests to the function?
We can do it by passing a literal block for each API request. Each literal block takes the completion block and issues the asynchronous remote API request:
int status = statusOfFirstRequestToSucceed(#[
^(RemoteAPICompletionBlock completion) {
[remoteAPI requestWithCompletion:completion];
},
^(RemoteAPICompletionBlock completion) {
[remoteAPI anotherRequestWithCompletion:completion];
},
^(RemoteAPICompletionBlock completion) {
[remoteAPI thirdRequestWithCompletion:completion];
}
]);
if (status != 0) {
AsyncToMain(^{ [self complexFlowFailedOnFirstRoundWithStatus:status]; });
return;
}
OK, now we've issued the three first parallel requests and waited for one to succeed, or for all of them to fail. Now we want to issue three more parallel requests and wait for all to succeed, or for one of them to fail. So it's almost identical, except I'm going to assume a function statusOfFirstRequestToFail:
status = statusOfFirstRequestToFail(#[
^(RemoteAPICompletionBlock completion) {
[remoteAPI requestWithCompletion:completion];
},
^(RemoteAPICompletionBlock completion) {
[remoteAPI anotherRequestWithCompletion:completion];
},
^(RemoteAPICompletionBlock completion) {
[remoteAPI thirdRequestWithCompletion:completion];
}
]);
if (status != 0) {
AsyncToMain(^{ [self complexFlowFailedOnSecondRoundWithStatus:status]; });
return;
}
Now both rounds of parallel requests have finished, so we can notify the main thread of success:
[self complexFlowSucceeded];
});
}
Overall, that seems like a pretty straightforward flow of control to me, and we just need to implement statusOfFirstRequestToSucceed and statusOfFirstRequestToFail. We can implement them with no extra threads. Since they are so similar, we'll make them both call on a helper function that does the real work:
static int statusOfFirstRequestToSucceed(NSArray *requestBlocks) {
return statusOfFirstRequestWithStatusPassingTest(requestBlocks, ^BOOL (int status) {
return status == 0;
});
}
static int statusOfFirstRequestToFail(NSArray *requestBlocks) {
return statusOfFirstRequestWithStatusPassingTest(requestBlocks, ^BOOL (int status) {
return status != 0;
});
}
In the helper function, I'll need a queue in which to run the completion blocks, to prevent race conditions:
static int statusOfFirstRequestWithStatusPassingTest(NSArray *requestBlocks,
BOOL (^statusTest)(int status))
{
dispatch_queue_t completionQueue = dispatch_queue_create("remote API completion", 0);
Note that I will only put blocks on completionQueue using dispatch_sync, and dispatch_sync always runs the block on the current thread unless the queue is the main queue.
I'll also need a semaphore, to wake up the outer function when some request has completed with a passing status, or when all requests have finished:
dispatch_semaphore_t enoughJobsCompleteSemaphore = dispatch_semaphore_create(0);
I'll keep track of the number of jobs not yet finished and the status of the last job to finish:
__block int jobsLeft = requestBlocks.count;
__block int outerStatus = 0;
When jobsLeft becomes 0, it means that either I've set outerStatus to a status that passes the test, or that all jobs have completed. Here's the completion block where I'll the work of tracking whether I'm done waiting. I do it all on completionQueue to serialize access to jobsLeft and outerStatus, in case the remote API dispatches multiple completion blocks in parallel (on separate threads or on a concurrent queue):
RemoteAPICompletionBlock completionBlock = ^(int status) {
dispatch_sync(completionQueue, ^{
I check to see if the outer function is still waiting for the current job to complete:
if (jobsLeft == 0) {
// The outer function has already returned.
return;
}
Next, I decrement the number of jobs remaining and make the completed job's status available to the outer function:
--jobsLeft;
outerStatus = status;
If the completed job's status passes the test, I set jobsLeft to zero to prevent other jobs from overwriting my status or singling the outer function:
if (statusTest(status)) {
// We have a winner. Prevent other jobs from overwriting my status.
jobsLeft = 0;
}
If there are no jobs left to wait on (because they've all finished or because this job's status passed the test), I wake up the outer function:
if (jobsLeft == 0) {
dispatch_semaphore_signal(enoughJobsCompleteSemaphore);
}
Finally, I release the queue and the semaphore. (The retains will be later, when I loop through the request blocks to execute them.)
dispatch_release(completionQueue);
dispatch_release(enoughJobsCompleteSemaphore);
});
};
That's the end of the completion block. The rest of the function is trivial. First I execute each request block, and I retain the queue and the semaphore to prevent dangling references:
for (void (^requestBlock)(RemoteAPICompletionBlock) in requestBlocks) {
dispatch_retain(completionQueue); // balanced in completionBlock
dispatch_retain(enoughJobsCompleteSemaphore); // balanced in completionBlock
requestBlock(completionBlock);
}
Note that the retains aren't necessary if you're using ARC and your deployment target is iOS 6.0 or later.
Then I just wait for one of the jobs to wake me up, release the queue and the semaphore, and return the status of the job that woke me:
dispatch_semaphore_wait(enoughJobsCompleteSemaphore, DISPATCH_TIME_FOREVER);
dispatch_release(completionQueue);
dispatch_release(enoughJobsCompleteSemaphore);
return outerStatus;
}
Note that the structure of statusOfFirstRequestWithStatusPassingTest is fairly generic: you can pass any request blocks you want, as long as each one calls the completion block and passes in an int status. You could modify the function to handle a more complex result from each request block, or to cancel outstanding requests (if you have a cancellation API).
While researching this myself I bumped into a port of Reactive Extensions to Objective-C. Reactive Extensions is like having the ability to querying a set of events or asynchronous operations. I know it has had a big uptake under .Net and JavaScript, and now apparently there is a port for Objective-C as well
https://github.com/blog/1107-reactivecocoa-for-a-better-world
Syntax looks tricky. I wonder if there is real world experience with it for iPhone development and if it does actually solve this issue elegantly.
I tend to wrap big nested block cluster f**** like you describe in subclasses of NSOperation that describe what the overall behaviour that your big nest block cluster f*** is actually doing (rather than leaving them littered throughout other code).
For example if your following code:
[remoteAPIWithURL:url1 success:^(int status){
[remoteAPIWithURL:url2 success:^(int status){
[remoteAPIWithURL:url3 success:^(int status){
[remoteAPIWithURL:url2 success:^(int status){
//succes!!!
}];
}];
}];
}];
is intended to get an authorise token and then sync something perhaps it would be an NSAuthorizedSyncOperation… I'm sure you get the gist. Benefits of this are nice tidy bundles of behaviour wrapped up in a class with one place to edit them if things change down the line. My 2¢.
In NSDocument the following methods are available for serialization:
Serialization
– continueActivityUsingBlock:
– continueAsynchronousWorkOnMainThreadUsingBlock:
– performActivityWithSynchronousWaiting:usingBlock:
– performAsynchronousFileAccessUsingBlock:
– performSynchronousFileAccessUsingBlock:
I'm just digging into this, but it seems like this would be a good place to start.
Not sure if that is want you where looking for? Though all objects in the array need different times to complete the all appear in the order the where submitted to the queue.
typedef int(^SumUpTill)(int);
SumUpTill sum = ^(int max){
int i = 0;
int result = 0;
while (i < max) {
result += i++;
}
return result;
};
dispatch_queue_t queue = dispatch_queue_create("com.dispatch.barrier.async", DISPATCH_QUEUE_CONCURRENT);
NSArray *urlArray = #[ [NSURL URLWithString:#"http://www.google.com"],
#"Test",
[sum copy],
[NSURL URLWithString:#"http://www.apple.com"]
];
[urlArray enumerateObjectsUsingBlock:^(id obj, NSUInteger idx, BOOL *stop) {
dispatch_barrier_async(queue, ^{
if ([obj isKindOfClass:[NSURL class]]) {
NSURLRequest *request = [NSURLRequest requestWithURL:obj];
NSURLResponse *response = nil;
NSError *error = nil;
[NSURLConnection sendSynchronousRequest:request returningResponse:&response error:&error];
NSLog(#"index = %d, response=%# error=%#", idx, response, error);
}
else if ([obj isKindOfClass:[NSString class]]) {
NSLog(#"index = %d, string %#", idx, obj);
}
else {
NSInteger result = ((SumUpTill)obj)(1000000);
NSLog(#"index = %d, result = %d", idx, result);
}
});
}];
I'm using an AFNetworking client object which makes an asynchronous request for an XML document and parses it.
Also using NSNotificationCenter to post a notification when the document has finished parsing.
Is there a way to wait for a notification to be posted without blocking the main thread?
E.g code:
-(void)saveConfiguration:(id)sender {
TLHTTPClient *RESTClient = [TLHTTPClient sharedClient];
// Performs the asynchronous fetching....this works.
[RESTClient fetchActiveUser:[usernameTextField stringValue] withPassword:[passwordTextField stringValue]];
/*
* What do I need here ? while (xxx) ?
*/
NSLog(#"Fetch Complete.");
}
Basically I'm wondering what sort of code I need in the above specified area to ensure that the function waits until the fetch has been completed ?
As it is right now I'll see "Fetch Complete." in the debug console before the fetch has been completed.
I tried adding a BOOL flag to the TLHTTPClient class:
BOOL fetchingFlag;
and then trying:
while([RESTClient fetchingFlag]) { NSLog(#"fetching...); }
When this class receives the notification it sets RESTClient.fetchingFlag = FALSE; which should technically kill the while loop right? Except my while loop runs infinitely ?!
Basically I'm wondering what sort of code I need in the above specified area to ensure that the function waits until the fetch has been completed ?
You need no code. Don't put anything in the method after you start the fetch, and nothing will happen. Your program will "wait" (it will actually be processing other input) until the notification is recieved.
In the notification handler method, put all the stuff that you need to do when the fetch is completed. This is (one of) the point(s) of notifications and other callback schemes -- your object won't do anything further until it gets the notification that it's time to act.
Is there a way to wait for a notification to be posted without blocking the main thread?
That's exactly how it works already.
If you don't need to inform multiple objects upon completion of the task, you could add a completion handler (objc block) to the -fetchActiveUser:withPassword: method (so it would become something like -fetchActiveUser:withPassword:completionHandler: and add the code to be executed in the completion handler.
An example, lets say your -fetchActiveUser:withPassword:completionHandler: method looks like the following:
- (void)fetchActiveUser:(NSString *)user
withPassword:(NSString *)pass
completionHandler:(void (^)(TLUser *user, NSError *error))handler
{
NSURL *URL = [NSURL URLWithString:#"http://www.website.com/page.html"];
NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:URL];
NSOperationQueue *queue = [NSOperationQueue currentQueue];
[NSURLConnection sendAsynchronousRequest:request
queue:queue
completionHandler:^ (NSURLResponse *response, NSData *data, NSError *error)
{
if (!handler)
{
return
};
if (data)
{
TLUser *user = [TLUser userWithData:data];
if (user)
{
handler(user, nil);
}
else
{
NSError *error = // 'failed to create user' error ...
handler(nil, error);
}
}
else
{
handler(nil, error);
}
}];
}
The completion handler will be called whenever the task is finished. It will either return a TLUser object or an Error if something went wrong (bad connection, data format changed while parsing, etc...).
You'll be able to call the method like this:
- (void)saveConfiguration:(id)sender
{
TLHTTPClient *RESTClient = [TLHTTPClient sharedClient];
// Performs the asynchronous fetching
[RESTClient fetchActiveUser:[usernameTextField stringValue]
withPassword:[passwordTextField stringValue]
completionHandler:^ (TLUser *user, NSError *error)
{
if (user)
{
NSLog(#"%#", user);
}
else
{
NSLog(#"%#", error);
}
}];
}
Of course, in this example I've used the build in asynchronous methods of NSURLConnection in stead of AFNetworking, but you should be able to get the general idea.
Using iOS 5's new TWRequest API, I've ran into a brick wall related with block usage.
What I need to do is upon receiving a successful response to a first request, immediately fire another one. On the completion block of the second request, I then notify success or failure of the multi-step operation.
Here's roughly what I'm doing:
- (void)doRequests
{
TWRequest* firstRequest = [self createFirstRequest];
[firstRequest performRequestWithHandler:^(NSData* responseData,
NSHTTPURLResponse* response,
NSError* error) {
// Error handling hidden for the sake of brevity...
TWRequest* secondRequest = [self createSecondRequest];
[secondRequest performRequestWithHandler:^(NSData* a,
NSHTTPURLResponse* b,
NSError* c) {
// Notify of success or failure - never reaches this far
}];
}];
}
I am not retaining either of the requests or keeping a reference to them anywhere; it's just fire-and-forget.
However, when I run the app, it crashes with EXC_BAD_ACCESS on:
[secondRequest performRequestWithHandler:...];
It executes the first request just fine, but when I try to launch a second one with a handler, it crashes. What's wrong with that code?
The methods to create the requests are as simple as:
- (TWRequest*)createFirstRequest
{
NSString* target = #"https://api.twitter.com/1/statuses/home_timeline.json";
NSURL* url = [NSURL URLWithString:target];
TWRequest* request = [[TWRequest alloc]
initWithURL:url parameters:params
requestMethod:TWRequestMethodGET];
// _twitterAccount is the backing ivar for property 'twitterAccount',
// a strong & nonatomic property of type ACAccount*
request.account = _twitterAccount;
return request;
}
Make sure you're keeping a reference/retaining the ACAccountStore that owns the ACAccount you are using to sign the TWRequests.
If you don't, the ACAccount will become invalid and then you'll get EXC_BAD_ACCESS when trying to fire a TWRequest signed with it.
I'm not familiar with TW*, so consider this a wild guess ... try sending a heap-allocated block:
[firstRequest performRequestWithHandler:[^ (NSData *responseData, ...) {
...
} copy]];
To clarify, I think the block you're sending is heap-allocated, so while TW* might be retaining it, it won't make any difference if it has already gone out of scope.