I have an asynchronous dispatch queue which reads data from large files in the background. During the course of that it does a few other things including some NSTask operations. The problem I'm facing is that I populate some variables with the result of those operations and the background queue has already moved on by the time those variables are ready (not NULL).
My code looks similar to the following:
dispatch_queue_t backgroundQueue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
dispatch_async(backgroundQueue, ^{
// reading large files
...
NSTaskClass *operation = [[NSTaskClass alloc] init];
NSString *result = [operation doTask:filePath];
NSLog(#"result: %#" result); // returns NULL since task isn't done yet.
...
// continue large file operations
});
Would would be the best way to handle this? I looked into creating a callback but I couldn't figure it out, and I'm not sure if that's even the right approach. Any advice on best practices is appreciated, thanks.
Just give you an idea, hope to solve your problem.
Use a dispatch_sempahore to handle the operations in your NSTaskClass
Create a handler for doTask method, use - (void)doTask:(NSString *)filePath handler:(void(^)(NSString *result))handler instead of - (NSString *)doTask:(NSString *)filePath
Then, your code should look something like this:
dispatch_queue_t backgroundQueue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
dispatch_async(backgroundQueue, ^{
// reading large files
dispatch_semaphore_t sema = dispatch_semaphore_create(0);
...
NSTaskClass *operation = [[NSTaskClass alloc] init];
[operation doTask:filePath handler:^(NSString * _Nonnull result) {
// Get your result here
result = [NSString stringWithString:result];
...
dispatch_semaphore_signal(sema);
}];
dispatch_semaphore_wait(sema, DISPATCH_TIME_FOREVER);
NSLog(#"result: %#" result); // Here should be fine (not NULL)
...
// continue large file operations
});
Related
I have several methods that have the following structure:
- (void) doSomethingWithCompletion: (void (^)(NSError *error)) completion {
__block NSError *fetchError = nil;
dispatch_group_t dispatchGroup = dispatch_group_create();
for (Item* item in self.items)
{
dispatch_group_enter(dispatchGroup);
// fetchError = fetch online data
}
dispatch_group_notify(dispatchGroup, dispatch_get_main_queue(),^{
if (completion)
completion(fetchError);
});
}
My goal is to run several doSomethings after each other, so I could so something like this:
[self doSomethingAWithCompletion: ^(NSArray *results NSError *error) {
if (error == nil) {
[self doSomethingBWithArray: results withCompletion: ^(NSError *error) {
if (error == nil) {
[self doSomethingCWithCompletion: ^(NSError *error) {
if (error == nil) {
// done!!
}
}];
}];
}];
What I am struggling with is the second code block (no pun); is nesting all the methods the way to go, or are there other solutions?
The important thing is, is that doSomethingBWithCompletion cannot begin before doSomethingAWithCompletion is done, and doSomethingCWithCompletion needs to wait until doSomethingBWithCompletion is complete, etc.
Also, doSomethingBWithCompletion uses data that is generated in doSomethingAWithCompletion, etc.
EDIT: After a lot of thinking, refactoring, and simplifying my code, I was able to end up with only two functions, using the nested approach as I outlined above and with a #property for the results array.
The important thing is, is that doSomethingBWithCompletion cannot begin before doSomethingAWithCompletion is done, and doSomethingCWithCompletion needs to wait until doSomethingBWithCompletion is complete, etc.
According to the comments:
The Results of the block are not depending on the result of the first aren't they?
And
Yes they are. For instance, in the first doSomething I determine which items are outdated, in the second doSomething I download and parse the updated items, and in the third doSomething I save them to the store.
(BTW: You should really add this information to your Q.)
If an action depends on the result (not only execution) of a previous action, you have to nest the blocks. Your code does not look like this, because there is no data passed to the completion blocks.
If you do not have such a dependency, you could use a private serial dispatch queue. However, this is a solution in your case, too, if you have akin of a manager class holding the data passed from block to block. But this seems to be highly anticonceptual.
There may be community attempt to add promises to objective-c, and it would be nice to have, because that's just what's needed here. Without committing to a whole new library, you can handle the nesting (which I agree is a bummer) by doing the async tasks recursively... something like this for your example code:
Start with an operation that takes no params and results in an array...
- (void)firstOpWithCompletion:(void (^)(NSArray *, NSError *))completion {
dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
dispatch_async(queue, ^{
NSArray *components = [#"this is an array of strings from the FIRST op" componentsSeparatedByString:#" "];
if (completion) {
completion(components, nil);
}
});
}
Here are a couple that take an array param and result in an array...
- (void)secondOpWithParam:(NSArray *)array completion:(void (^)(NSArray *, NSError *))completion {
dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
dispatch_async(queue, ^{
if (completion) {
NSArray *components = [#"these strings are from the SECOND op" componentsSeparatedByString:#" "];
NSArray *result = [array arrayByAddingObjectsFromArray:components];
if (completion) {
completion(result, nil);
}
}
});
}
- (void)thirdOpWithParam:(NSArray *)array completion:(void (^)(NSArray *, NSError *))completion {
dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
dispatch_async(queue, ^{
if (completion) {
NSArray *components = [#"these strings are from the THIRD op" componentsSeparatedByString:#" "];
NSArray *result = [array arrayByAddingObjectsFromArray:components];
if (completion) {
NSLog(#"we did it. returning %#", result);
completion(result, nil);
}
}
});
}
// ...as many as these as you need
Now, as in my answer prior to this edit, we just add a param pass initially and in the intermediate calls...
- (void)doSeveralThingsInSequence:(NSArray *)todo param:(NSArray *)param {
if (todo.count == 0) return;
// you could generalize further here, by passing a "final" block and run that before the return
NSString *nextTodo = todo[0];
SEL sel = NSSelectorFromString(nextTodo);
IMP imp = [self methodForSelector:sel];
void (*func)(id, SEL, NSArray *, void (^)(NSArray *, NSError *)) = (void *)imp;
func(self, sel, param, ^(NSArray *result, NSError *error) {
if (!error) {
NSArray *remainingTodo = [todo subarrayWithRange:NSMakeRange(1, todo.count-1)];
[self doSeveralThingsInSequence:remainingTodo param:result];
}
});
}
Stepping through the code: this method bails if there's nothing to do, otherwise it takes the next selector name from the passed array, gets the C function implementation for it and invokes it, placing a completion block on the call stack that starts the process over for the remaining selectors.
Finally, doEverything calls the first operation to get started, then starts running a list of operations (which can be an arbitrarily long list) passing the array output from one as the array input to the next. (You could generalize this further by passing id's along the chain
- (void)doEverything {
[self firstOpWithCompletion:^(NSArray *array, NSError *error) {
NSArray *todo = #[ #"secondOpWithParam:completion:", #"thirdOpWithParam:completion:" ];
[self doSeveralThingsInSequence:todo param:array];
}];
}
I tested this exactly as posted and saw the expected output:
(
this,
is,
an,
array,
of,
strings,
from,
the,
FIRST,
op,
these,
strings,
are,
from,
the,
SECOND,
op,
these,
strings,
are,
from,
the,
THIRD,
op
)
I've used AFNetworking to download and cache images in my project, but I wanted to replace the framework with a lightest in-house category, obviously I'm facing a lot of issues with dispatch queues, and I'm having a hard time debugging them.
This is the category I'm using right now, It works with no problem with a small number of simultaneous downloads but, if I start a lot of downloads it looks like the application hangs an an endless load.
I need some help debugging and I would like to understand what I'm missing here.
Here are the two main functions:
- (void) imageWithUrl: (NSURL*) imageUrl placeHolderImage: (UIImage *) placeHolderImage shouldAlwaysRefresh: (BOOL) shouldRefresh {
self.image= placeHolderImage;
TMCache *sharedCache = [TMCache sharedCache];
[sharedCache trimToDate: [NSDate dateWithTimeIntervalSinceNow: -(60.0*24.0*7.0)]];
UIImage *cachedImage = [sharedCache objectForKey: [imageUrl absoluteString]];
if(cachedImage){
self.image = cachedImage;
if(shouldRefresh)
[self fetchImageFromUrl: imageUrl];
}
else{
[self fetchImageFromUrl: imageUrl];
}
}
- (void) fetchImageFromUrl: (NSURL *) imageUrl{
dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0);
dispatch_async(queue, ^{
NSData *imageData = [NSData dataWithContentsOfURL:imageUrl];
UIImage *image = [UIImage imageWithData:imageData];
if(image){
[[TMCache sharedCache] setObject:image forKey: [imageUrl absoluteString]];
dispatch_sync(dispatch_get_main_queue(), ^{
self.image = image;
});
}
});
}
Your problem is most likely the dispatch_sync back to the main queue creating a deadlock when the image has finished loading.
The easiest way to solve that is to simply replace your dispatch_sync with dispatch_async.
if(image) {
[[TMCache sharedCache] setObject:image forKey: [imageUrl absoluteString]];
dispatch_async(dispatch_get_main_queue(), ^{
self.image = image;
});
}
I don't see any reason to use a dispatch_sync there since you're already in a dispatch_async block; this shouldn't have any negative effects on your code. If fact, you should usually try to avoid using dispatch_sync if at all possible, precisely for the issue you're running in to here.
Sidenote: AFNetworking actually gives you quite a bit for not all that much code. I'd seriously think twice before using your own solution over AFNetworking.
I have the code below to load a group of images and notify me via completionHandler when they are all done loading. However, I find that certain dispatch_group_leave won't be called at times and my guess is imageLoader is deallocated before the block gets to run. If I put a reference of imageLoader within the loadImageWithURL:completionHandler block, then everything works as intended.
Is my guess of the cause correct? What's the correct fix for this issue? I know ARC does block copy in most cases automatically, should I do a block copy in this case?
- (void)loadGroupImagesAsyncWithCompletion:(void(^)(NSError *))completionHandler {
dispatch_group_t group = dispatch_group_create();
int index = 0;
for (Item *item in items) {
char queueLabel[30] = {0};
sprintf(queueLabel, "loader%d", index);
dispatch_queue_t queue = dispatch_queue_create(queueLabel, NULL);
dispatch_group_enter(group);
dispatch_async(queue, ^{
ImageLoader *imageLoader = [[ImageLoader alloc] init];
[imageLoader loadImageWithURL:url completionHandler:^(UIImage *image, NSError *error) {
if (image) {
item.image = image;
}
//NOTE: if item object is referenced in this block,
//then there is no missed dispatch_group_leave call.
dispatch_group_leave(group);
}];
});
}
// Non-blocking wait
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
// shouldn't take more than 5 secs to load all images
dispatch_group_wait(group, dispatch_time(DISPATCH_TIME_NOW, (int64_t)(5.0 * NSEC_PER_SEC)));
dispatch_async(dispatch_get_main_queue(), ^{
completionHandler(nil);
});
});
}
Here's my guess. It's only a guess because you haven't posted the code of anything to do with ImageLoader.
If -loadImageWithURL:completionHandler: operates asynchronously i.e. it uses an async dispatch queue itself then you could be right that it is being deallocated before the load finishes. This is because its lifetime is just the scope of the for block it is declared in.
In fact, there is no reason why that method needs to do stuff asynchronously, because you have already got it in an asynchronous block. Just have a synchronous method and call dispatch_group_leave() after the method finishes.
EDIT
Given that you have no control over ImageLoader and -loadImageWithURL:completionHandler: operates asynchronously without any help from you, you should remove the dispatch_async wrapper around the call. You'll still have the problem of the ImageLoader being deallocated, but that can be avoided by putting each ImageLoader in a array when you create it.
The code would look something like this:
- (void)loadGroupImagesAsyncWithCompletion:(void(^)(NSError *))completionHandler
{
NSMutableArray* loaders = [[NSMutableArray alloc] init];
dispatch_group_t group = dispatch_group_create();
int index = 0;
for (Item *item in items) {
char queueLabel[30] = {0};
sprintf(queueLabel, "loader%d", index);
dispatch_queue_t queue = dispatch_queue_create(queueLabel, NULL);
dispatch_group_enter(group);
ImageLoader *imageLoader = [[ImageLoader alloc] init];
[imageLoader loadImageWithURL:url completionHandler:^(UIImage *image, NSError *error) {
if (image) {
item.image = image;
}
//NOTE: if item object is referenced in this block,
//then there is no missed dispatch_group_leave call.
dispatch_group_leave(group);
}];
[loaders addObject: imageLoader];
}
// Non-blocking wait
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
// shouldn't take more than 5 secs to load all images
dispatch_group_wait(group, dispatch_time(DISPATCH_TIME_NOW, (int64_t)(5.0 * NSEC_PER_SEC)));
dispatch_async(dispatch_get_main_queue(), ^{
completionHandler(nil);
});
[loaders removeAllObjects];
});
}
I am trying to use MKNetworkKit to fetch an array of links from a web service, then parse each response on a background thread, and use the dispatch_group_t of GCD to wait until all threads are finished processing. Where I'm stuck is I can't figure out why my dispatch_group_notify is not waiting for all threads in the group to complete. Running this code will print:
results count: 0
added into results, count: 1
added into results, count: 2
The dispatch group is not waiting on its threads. I have also tried dispatch_group_wait but that gave me a crash. I don't know if MKNetworkKit's use of NSOperation is conflicting with this issue. Thanks for any help!
- (MKNetworkOperation *)getABunchOfMovies:(NSArray *)movies onCompletion:(CastResponseBlock)completionBlock onError:(MKNKErrorBlock)errorBlock
{
MKNetworkOperation *operation;
dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
dispatch_group_t group = dispatch_group_create();
block NSMutableArray *results = [[NSMutableArray alloc] initWithCapacity:[movies count]];
for (NSString *movieTitle in movies) {
operation = [self operationWithPath:REQUEST_URL(API_KEY, [movieTitle urlEncodedString])];
[operation onCompletion:^(MKNetworkOperation *completedOperation) {
dispatch_group_async(group, queue, ^{
NSDictionary *response = [completedOperation responseJSON];
id result = [self processResponse:response withMovieTitle:movieTitle];
#synchronized (results) {
[results addObject:result];
NSLog(#"added into results, count: %d", [results count]);
}
});
}
onError:^(NSError *error) {
errorBlock(error);
}];
[self enqueueOperation:operation];
}
dispatch_group_notify(group, dispatch_get_main_queue(), ^{
NSLog(#"results count: %d", [results count]);
// Return array here
completionBlock(results);
});
dispatch_release(group);
return operation;
}
Edit:
I still can't figure out why, but if I change it to use dispatch_group_enter(group); and match it with a dispatch_group_leave(group); at the end of the completion block, it works. Does anyone have any idea why this is happening?
At the moment MKNetworkKit doesn't support queue completion handlers.
You should consider adding operation dependency instead of this hack.
[lastOperation addDependency:op1];
[lastOperation addDependency:op2];
and assume that when "lastOperation" completes, the queue has indeed completed.
Another way is to KVO the "operationCount" keypath on the engine and check if it reaches zero.
MKNetworkEngine has a code block that does this to show and hide the network activity indicator.
I have a UITableView consisting of roughly 10 subclassed UITableViewCells named TBPostSnapCell. Each cell, when initialised, sets two of its variables with UIImages downloaded via GCD or retrieved from a cache stored in the user's documents directory.
For some reason, this is causing a noticeable lag on the tableView and therefore disrupting the UX of the app & table.
Please can you tell me how I can reduce this lag?
tableView... cellForRowAtIndexPath:
if (post.postType == TBPostTypeSnap || post.snaps != nil) {
TBPostSnapCell *snapCell = (TBPostSnapCell *) [tableView dequeueReusableCellWithIdentifier:snapID];
if (snapCell == nil) {
snapCell = [[[NSBundle mainBundle] loadNibNamed:#"TBPostSnapCell" owner:self options:nil] objectAtIndex:0];
[snapCell setPost:[posts objectAtIndex:indexPath.row]];
[snapCell.bottomImageView setImage:[UIImage imageNamed:[NSString stringWithFormat:#"%d", (indexPath.row % 6) +1]]];
}
[snapCell.commentsButton setTag:indexPath.row];
[snapCell.commentsButton addTarget:self action:#selector(comments:) forControlEvents:UIControlEventTouchDown];
[snapCell setSelectionStyle:UITableViewCellSelectionStyleNone];
return snapCell;
}
TBSnapCell.m
- (void) setPost:(TBPost *) _post {
if (post != _post) {
[post release];
post = [_post retain];
}
...
if (self.snap == nil) {
NSString *str = [[_post snaps] objectForKey:TBImageOriginalURL];
NSURL *url = [NSURL URLWithString:str];
[TBImageDownloader downloadImageAtURL:url completion:^(UIImage *image) {
[self setSnap:image];
}];
}
if (self.authorAvatar == nil) {
...
NSURL *url = [[[_post user] avatars] objectForKey:[[TBForrstr sharedForrstr] stringForPhotoSize:TBPhotoSizeSmall]];
[TBImageDownloader downloadImageAtURL:url completion:^(UIImage *image) {
[self setAuthorAvatar:image];
}];
...
}
}
TBImageDownloader.m
+ (void) downloadImageAtURL:(NSURL *)url completion:(TBImageDownloadCompletion)_block {
if ([self hasWrittenDataToFilePath:filePathForURL(url)]) {
[self imageForURL:filePathForURL(url) callback:^(UIImage * image) {
_block(image); //gets UIImage from NSDocumentsDirectory via GCD
}];
return;
}
dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0);
dispatch_async(queue, ^{
UIImage *image = [UIImage imageWithData:[NSData dataWithContentsOfURL:url]];
dispatch_async(dispatch_get_main_queue(), ^{
[self writeImageData:UIImagePNGRepresentation(image) toFilePath:filePathForURL(url)];
_block(image);
});
});
}
First thing to try is converting DISPATCH_QUEUE_PRIORITY_HIGH (aka ONG MOST IMPORTANT WORK EVER FORGET EVERYTHING ELSE) to something like DISPATCH_QUEUE_PRIORITY_LOW.
If that doesn't fix it you could attempt to do the http traffic via dispatch_sources, but that is a lot of work.
You might also just try to limit the number of in flight http fetches with a semaphore, the real trick will be deciding what the best limit is as the "good" number will depend on the network, your CPUs, and memory pressure. Maybe benchmark 2, 4, and 8 with a few configurations and see if there is enough pattern to generalize.
Ok, lets try just one, replace the queue = ... with:
static dispatch_once_t once;
static dispatch_queue_t queue = NULL;
dispatch_once(&once, ^{
queue = dispatch_queue_create("com.blah.url-fetch", NULL);
});
Leave the rest of the code as is. This is likely to be the least sputtery, but may not load the images very fast.
For the more general case, rip out the change I just gave you, and we will work on this:
dispatch_async(queue, ^{
UIImage *image = [UIImage imageWithData:[NSData dataWithContentsOfURL:url]];
dispatch_async(dispatch_get_main_queue(), ^{
[self writeImageData:UIImagePNGRepresentation(image) toFilePath:filePathForURL(url)];
_block(image);
});
});
Replacing it with:
static dispatch_once_t once;
static const int max_in_flight = 2; // Also try 4, 8, and maybe some other numbers
static dispatch_semaphore_t limit = NULL;
dispatch_once(&once, ^{
limit = dispatch_semaphore_create(max_in_flight);
});
dispatch_async(queue, ^{
dispatch_semaphore_wait(limit, DISPATCH_TIME_FOREVER);
UIImage *image = [UIImage imageWithData:[NSData dataWithContentsOfURL:url]];
// (or you might want the dispatch_semaphore_signal here, and not below)
dispatch_async(dispatch_get_main_queue(), ^{
[self writeImageData:UIImagePNGRepresentation(image) toFilePath:filePathForURL(url)];
_block(image);
dispatch_semaphore_signal(limit);
});
});
NOTE: I haven't tested any of this code, even to see if it compiles. As written it will only allow 2 threads to be executing the bulk of the code in your two nested blocks. You might want to move the dispatch_semaphore_signal up to the commented line. That will limit you to two fetches/image creates, but they will be allowed to overlap with writing the image data to a file and calling your _block callback.
BTW you do a lot of file I/O which is faster on flash then any disk ever was, but if you are still looking for performance wins that might be another place to attack. For example maybe keeping the UIImage around in memory until you get a low memory warning and only then writing them to disk.