I have an async NSOperation to download data for multiple ImageFile objects. Since this is all happening asynchronously I am using a dispatch group to track the requests and then dispatch_group_notify to complete the operation when they are all done.
The question I have is what happen when the operation is ended prematurely, either by cancelation or by some other error. The dispatch group will be left with unmatched dispatch_group_enter and dispatch_group_leave so dispatch_group_notify will never be called. Is it the block retained somewhere by the system forever waiting, or will it get released when the NSOperation gets released?
Or is my approach not ideal, how else should I do this?
- (void)main
{
if (self.cancelled) {
[self completeOperation];
return;
}
NSManagedObjectContext *context = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSPrivateQueueConcurrencyType];
context.persistentStoreCoordinator = self.persistentStoreCoordinator;
context.mergePolicy = NSMergeByPropertyObjectTrumpMergePolicy;
[context performBlock:^{
NSFetchRequest *request = [ImageFile fetchRequest];
request.predicate = ....
request.sortDescriptors = ...
NSError *error;
NSArray *imageFiles = [context executeFetchRequest:request error:&error];
if (!imageFiles) {
// Error handling...
[self completeOperation];
return;
}
dispatch_group_t group = dispatch_group_create();
for (ImageFile *imageFile in imageFiles) {
dispatch_group_enter(group);
#autoreleasepool {
[self.webService requestImageWithId:imageFile.id completionHandler:^(NSData * _Nullable data, NSURLResponse * _Nullable response, NSError * _Nullable error) {
if (self.cancelled) {
[self completeOperation];
return;
}
[context performBlock:^{
if (data && !error) {
imageFile.data = data;
NSError *error;
if (![context save:&error]) {
// Error handling...
[self completeOperation];
return;
}
}
dispatch_group_leave(group);
}];
}];
}
}
dispatch_group_notify(group, dispatch_get_main_queue(), ^{
[self completeOperation];
});
}];
}
From the docs for dispatch_group_enter():
A call to this function must be balanced with a call to dispatch_group_leave.
From the docs for dispatch_group_t:
The dispatch group keeps track of how many blocks are outstanding, and GCD retains the group until all its associated blocks complete execution.
It talks about outstanding blocks, but what it really means is unmatched calls to dispatch_group_enter().
So, the answer to your question about what happens is that the dispatch group object effectively leaks. The block object passed to dispatch_group_notify() and any objects it has strong references to also leak. In your case, that includes self.
The answer to your question of whether your approach is "ideal" is: no, it's not ideal. It's not even valid by the design contract of GCD. You must balance all calls to dispatch_group_enter() with calls to dispatch_group_leave().
If you want to somehow distinguish between success and failure or normal completion and cancellation, you should set some state that's available to the notify block and then code the notify block to consult that state to decide what to do.
In your case, though, the code paths where you fail to call dispatch_group_leave() just do the same thing the notify block would do. So I'm not even sure why you're not just calling dispatch_group_leave() rather than calling [self completeOperation] in those cases.
I have several methods that have the following structure:
- (void) doSomethingWithCompletion: (void (^)(NSError *error)) completion {
__block NSError *fetchError = nil;
dispatch_group_t dispatchGroup = dispatch_group_create();
for (Item* item in self.items)
{
dispatch_group_enter(dispatchGroup);
// fetchError = fetch online data
}
dispatch_group_notify(dispatchGroup, dispatch_get_main_queue(),^{
if (completion)
completion(fetchError);
});
}
My goal is to run several doSomethings after each other, so I could so something like this:
[self doSomethingAWithCompletion: ^(NSArray *results NSError *error) {
if (error == nil) {
[self doSomethingBWithArray: results withCompletion: ^(NSError *error) {
if (error == nil) {
[self doSomethingCWithCompletion: ^(NSError *error) {
if (error == nil) {
// done!!
}
}];
}];
}];
What I am struggling with is the second code block (no pun); is nesting all the methods the way to go, or are there other solutions?
The important thing is, is that doSomethingBWithCompletion cannot begin before doSomethingAWithCompletion is done, and doSomethingCWithCompletion needs to wait until doSomethingBWithCompletion is complete, etc.
Also, doSomethingBWithCompletion uses data that is generated in doSomethingAWithCompletion, etc.
EDIT: After a lot of thinking, refactoring, and simplifying my code, I was able to end up with only two functions, using the nested approach as I outlined above and with a #property for the results array.
The important thing is, is that doSomethingBWithCompletion cannot begin before doSomethingAWithCompletion is done, and doSomethingCWithCompletion needs to wait until doSomethingBWithCompletion is complete, etc.
According to the comments:
The Results of the block are not depending on the result of the first aren't they?
And
Yes they are. For instance, in the first doSomething I determine which items are outdated, in the second doSomething I download and parse the updated items, and in the third doSomething I save them to the store.
(BTW: You should really add this information to your Q.)
If an action depends on the result (not only execution) of a previous action, you have to nest the blocks. Your code does not look like this, because there is no data passed to the completion blocks.
If you do not have such a dependency, you could use a private serial dispatch queue. However, this is a solution in your case, too, if you have akin of a manager class holding the data passed from block to block. But this seems to be highly anticonceptual.
There may be community attempt to add promises to objective-c, and it would be nice to have, because that's just what's needed here. Without committing to a whole new library, you can handle the nesting (which I agree is a bummer) by doing the async tasks recursively... something like this for your example code:
Start with an operation that takes no params and results in an array...
- (void)firstOpWithCompletion:(void (^)(NSArray *, NSError *))completion {
dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
dispatch_async(queue, ^{
NSArray *components = [#"this is an array of strings from the FIRST op" componentsSeparatedByString:#" "];
if (completion) {
completion(components, nil);
}
});
}
Here are a couple that take an array param and result in an array...
- (void)secondOpWithParam:(NSArray *)array completion:(void (^)(NSArray *, NSError *))completion {
dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
dispatch_async(queue, ^{
if (completion) {
NSArray *components = [#"these strings are from the SECOND op" componentsSeparatedByString:#" "];
NSArray *result = [array arrayByAddingObjectsFromArray:components];
if (completion) {
completion(result, nil);
}
}
});
}
- (void)thirdOpWithParam:(NSArray *)array completion:(void (^)(NSArray *, NSError *))completion {
dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
dispatch_async(queue, ^{
if (completion) {
NSArray *components = [#"these strings are from the THIRD op" componentsSeparatedByString:#" "];
NSArray *result = [array arrayByAddingObjectsFromArray:components];
if (completion) {
NSLog(#"we did it. returning %#", result);
completion(result, nil);
}
}
});
}
// ...as many as these as you need
Now, as in my answer prior to this edit, we just add a param pass initially and in the intermediate calls...
- (void)doSeveralThingsInSequence:(NSArray *)todo param:(NSArray *)param {
if (todo.count == 0) return;
// you could generalize further here, by passing a "final" block and run that before the return
NSString *nextTodo = todo[0];
SEL sel = NSSelectorFromString(nextTodo);
IMP imp = [self methodForSelector:sel];
void (*func)(id, SEL, NSArray *, void (^)(NSArray *, NSError *)) = (void *)imp;
func(self, sel, param, ^(NSArray *result, NSError *error) {
if (!error) {
NSArray *remainingTodo = [todo subarrayWithRange:NSMakeRange(1, todo.count-1)];
[self doSeveralThingsInSequence:remainingTodo param:result];
}
});
}
Stepping through the code: this method bails if there's nothing to do, otherwise it takes the next selector name from the passed array, gets the C function implementation for it and invokes it, placing a completion block on the call stack that starts the process over for the remaining selectors.
Finally, doEverything calls the first operation to get started, then starts running a list of operations (which can be an arbitrarily long list) passing the array output from one as the array input to the next. (You could generalize this further by passing id's along the chain
- (void)doEverything {
[self firstOpWithCompletion:^(NSArray *array, NSError *error) {
NSArray *todo = #[ #"secondOpWithParam:completion:", #"thirdOpWithParam:completion:" ];
[self doSeveralThingsInSequence:todo param:array];
}];
}
I tested this exactly as posted and saw the expected output:
(
this,
is,
an,
array,
of,
strings,
from,
the,
FIRST,
op,
these,
strings,
are,
from,
the,
SECOND,
op,
these,
strings,
are,
from,
the,
THIRD,
op
)
The following is my code using GCD to fetch data from the network and then pass it to a response block on the main queue.
+ (void)allData:(void(^)(NSArray*))responseBlock
{
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
__block NSArray *data = [[self all] retain]; // get data from network
dispatch_async(dispatch_get_main_queue(), ^{
responseBlock(data); // 2
});
});
}
The [[self all] retain] is to prevent the object from being released . But now the responseBlock has to release it . Is this the correct way ?
you can do like this
+ (void)allData:(void(^)(NSArray*))responseBlock
{
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
__block NSArray *data = [[self all] retain]; // get data from network
dispatch_async(dispatch_get_main_queue(), ^{
responseBlock(data); // 2
[data release]; // you can release here
data = nil;
});
});
}
or simply remove __block, the inner block (the one you pass to main queue) will retain/release data automatically and because data does not retaining anything else so I can't see any possible retain cycle here (since self here is a Class).
I am trying to use MKNetworkKit to fetch an array of links from a web service, then parse each response on a background thread, and use the dispatch_group_t of GCD to wait until all threads are finished processing. Where I'm stuck is I can't figure out why my dispatch_group_notify is not waiting for all threads in the group to complete. Running this code will print:
results count: 0
added into results, count: 1
added into results, count: 2
The dispatch group is not waiting on its threads. I have also tried dispatch_group_wait but that gave me a crash. I don't know if MKNetworkKit's use of NSOperation is conflicting with this issue. Thanks for any help!
- (MKNetworkOperation *)getABunchOfMovies:(NSArray *)movies onCompletion:(CastResponseBlock)completionBlock onError:(MKNKErrorBlock)errorBlock
{
MKNetworkOperation *operation;
dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
dispatch_group_t group = dispatch_group_create();
block NSMutableArray *results = [[NSMutableArray alloc] initWithCapacity:[movies count]];
for (NSString *movieTitle in movies) {
operation = [self operationWithPath:REQUEST_URL(API_KEY, [movieTitle urlEncodedString])];
[operation onCompletion:^(MKNetworkOperation *completedOperation) {
dispatch_group_async(group, queue, ^{
NSDictionary *response = [completedOperation responseJSON];
id result = [self processResponse:response withMovieTitle:movieTitle];
#synchronized (results) {
[results addObject:result];
NSLog(#"added into results, count: %d", [results count]);
}
});
}
onError:^(NSError *error) {
errorBlock(error);
}];
[self enqueueOperation:operation];
}
dispatch_group_notify(group, dispatch_get_main_queue(), ^{
NSLog(#"results count: %d", [results count]);
// Return array here
completionBlock(results);
});
dispatch_release(group);
return operation;
}
Edit:
I still can't figure out why, but if I change it to use dispatch_group_enter(group); and match it with a dispatch_group_leave(group); at the end of the completion block, it works. Does anyone have any idea why this is happening?
At the moment MKNetworkKit doesn't support queue completion handlers.
You should consider adding operation dependency instead of this hack.
[lastOperation addDependency:op1];
[lastOperation addDependency:op2];
and assume that when "lastOperation" completes, the queue has indeed completed.
Another way is to KVO the "operationCount" keypath on the engine and check if it reaches zero.
MKNetworkEngine has a code block that does this to show and hide the network activity indicator.
I have some code which is similar to the following code:
dispatch_queue_t queue = dispatch_queue_create("", 0);
dispatch_queue_t inner_queue = dispatch_queue_create("", 0);
dispatch_async(queue,
^{
NSAutoreleasePool* autoreleasePool = [[NSAutoreleasePool alloc] init];
NSArray* objects = [self.otherObject getObjectsFromSlowQuery];
[objects enumerateObjectsWithUsingBlock:^(id anObject, NSUInteger idx, BOOL *stop)
{
[anObject prepare];
dispatch_async(inner_queue,
^{
InnerObject* innerObject = anObject.innerObject;
[self.helper doSomethingExpensiveWithObject:innerObject];
});
dispatch_sync(self.syncQueue,
^{
[self insertIntoCollection:anObject];
});
}];
dispatch_release(inner_queue);
[autoreleasePool drain];
});
dispatch_release(queue);
Where [anObject.innerObject] is a nonatomic property.
I got a crash report from a user which shows an EXC_BAD_ACCESS in objc_msgSend() when trying to access a property of innerObject within the doSomethingExpensiveWithObject: call.
At first I was thinking that perhaps the autoreleasePool was drained so the innerObject instance was released before returning from doSomethingExpensiveWithObject: but as far as I know anObject should be retained by the inner dispatch_async call which should also keep the innerObject alive.
What am I missing?
Instruments will make quick work of this - run with zombies and review the reference counts when it stops.