I've never used background threads before. I have a time consuming computation currently running on the main thread which appends the data output to a TERecord. My workflow essentially goes:
run long process…
update GUI…
run long process…
update GUI…
and so on.
At several places where the code produces (string) output I update the UI by calling my 'addToRecord' method shown here:
-(void)addToRecord:(NSString*)passedStr:(BOOL)updateUI
{
NSRange endRange;
// add the passed text...
endRange.location = [[theOutputView textStorage] length];
endRange.length = 0;
[theOutputView replaceCharactersInRange:endRange withString:passedStr];
if(updateUI) // immediate GUI update needed...
{
// scroll window contents to BOTTOM of page...
endRange = NSMakeRange([[theOutputView string] length],0);
[theOutputView scrollRangeToVisible:endRange];
[theOutputView display];
}
}
While it does the job, my entire UI remains unresponsive until the process completes, of course. I know I should be doing the heavy lifting on a background thread which I've never used before. I've figured out part of the problem in creating a background thread like below:
-(IBAction)readUserInput:(id)sender
{
// irrelevant code snipped for brevity
if([self checkForErrors] == NO)
{
[runButton setEnabled:NO];
[self performSelectorInBackground:#selector(runWorkThread) withObject:nil];
}
}
-(void)runWorkThread
{
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc]init];
[self runLongProcess];
[pool drain];
}
but i just don't understand how to call the main thread every time the code encounters my 'addToRecord' method, then how to return control to the background thread?
Another possibility might be to remove the updateUI code from my 'addToRecord' method and just have have the main thread calling this code every second or so on a timer?
Any advice and sample code would be greatly appreciated. Thanks!
Instead of using performSelectorInBackground you can use the Dispatch framework (also called GCD), which is the preferred way of handling concurrent work. The Dispatch already has a pool of background threads set up that you can use. To switch thread you call dispatch_async() like this:
dispatch_async(dispatch_get_global_queue(QOS_CLASS_BACKGROUND, 0), ^{
// :
// Do your background work here
// :
dispatch_async(dispatch_get_main_queue(), ^{
// :
// Now you are back in the main thread
// :
});
});
The first parameter is the queue identifier which is supplied to you by either dispatch_get_global_queue() which returns one of the "worker" queues, or dispatch_get_main_queue() which returns the main queue. The last parameter is a code block that is executed on the selected queue.
When requesting a concurrent queue using dispatch_get_global_queue() you specify a Quality of Service, which determines the priority your code will have in relation to other work. See the documentation for more information and possible values.
Read more on the Dispatch
I am testing a method that runs in background and executes a code block when it finishes. I am using expectations to handle the asynchronous execution of the tests. I wrote simple a test that shows the behaviour:
- (void) backgroundMethodWithCallback: (void(^)(void)) callback {
dispatch_queue_t backgroundQueue;
backgroundQueue = dispatch_queue_create("background.queue", NULL);
dispatch_async(backgroundQueue, ^(void) {
callback();
});
}
- (void) testMethodWithCallback {
XCTestExpectation *expectation = [self expectationWithDescription:#"Add collection bundle"];
[self backgroundMethodWithCallback:^{
[expectation fulfill];
usleep(50);
XCTFail(#"fail test");
}];
[self waitForExpectationsWithTimeout: 2 handler:^(NSError *error) {
if (error != nil) {
XCTFail(#"timeout");
}
}];
}
The XCTFail(#"fail test"); line should fail for this test but the test is passing.
I also noticed that this only happens when the code ran on the callback takes an amount of time (in my case, I was checking some files on the file system). This is why the usleep(50); line is necessary to reproduce the case.
The expectation must be fulfilled after all the test checks. Moving the line to the end of the callback block is enough to make the test fail:
- (void) testMethodWithCallback {
XCTestExpectation *expectation = [self expectationWithDescription:#"Add collection bundle"];
[self backgroundMethodWithCallback:^{
usleep(50);
XCTFail(#"fail test");
[expectation fulfill];
}];
[self waitForExpectationsWithTimeout: 2 handler:^(NSError *error) {
if (error != nil) {
XCTFail(#"timeout");
}
}];
}
I did not find explicit documentation about this but in the apple developer guides, the fulfill message is sent at the end of the block and it makes a lot of sense.
Note: I first found an example in swift where the fulfill method is called at the start of the callback. What I don't know is if the example is not correct or there is a difference with Objective-C.
The block called by backgroundMethodWithCallback is immediately fulfilling the expectation, thereby letting the test finish before XCTFail is called. If the block fulfills the expectation before it finishes performing other actions, you end up with race condition, in which the behavior of the test is conditional upon the speed with with the rest of the block is performed. But one shouldn't reasonably expect XCTFail to be captured if the test, itself, has already finished.
Bottom line, if you move the [expectation fulfill] to the end of the block, this race condition is eliminated.
From the docs:
The completion block you provide is executed when the value returned by the isFinished method changes to YES. Thus, this block is executed by the operation object after the operation’s primary task is finished or cancelled.
I'm using RestKit/AFNetworking, if that matters.
I have multiple dependencies in my NSOperation in a OperationQueue. I use the completion block to set some variables (appending the results to an array) that my child requires.
(task1,...,taskN) -> taskA
taskA addDependency: task1-taskN
Will taskA receive incomplete data since the child can execute before the completion block is fired?
Reference
Do NSOperations and their completionBlocks run concurrently?
I did a simple test by adding a sleep in my completion block and I had a different result. The completion block runs in the main thread. While all the completion block are sleeping, the child task ran.
As I discuss below under "a few observations", you have no assurances that this final dependent operation will not start before your other sundry AFNetworking completion blocks have finished. It strikes me that if this final operation really needs to wait for these completion blocks to finish, then you have a couple of alternatives:
Use semaphores within each of the n the completion blocks to signal when they're done and have the completion operation wait for n signals; or
Don't queue this final operation up front, but rather have your completion blocks for the individual uploads keep track of how many pending uploads are still incomplete, and when it falls to zero, then initiate the final "post" operation.
As you pointed out in your comments, you could wrap your invocation of the AFNetworking operation and its completion handler in your own operation, at which point you can then use the standard addDependency mechanism.
You could abandon the addDependency approach (which adds an observer on the isFinished key of the operation upon which this operation is dependent, and once all those dependencies are resolved, performs the isReady KVN; the problem being that this can theoretically happen before your completion block is done) and replace it with your own isReady logic. For example, imagine you had a post operation which you could add your own key dependencies and remove them manually in your completion block, rather than having them removed automatically upon isFinished. Thus, you custom operation
#interface PostOperation ()
#property (nonatomic, getter = isReady) BOOL ready;
#property (nonatomic, strong) NSMutableArray *keys;
#end
#implementation PostOperation
#synthesize ready = _ready;
- (void)addKeyDependency:(id)key {
if (!self.keys)
self.keys = [NSMutableArray arrayWithObject:key];
else
[self.keys addObject:key];
self.ready = NO;
}
- (void)removeKeyDependency:(id)key {
[self.keys removeObject:key];
if ([self.keys count] == 0)
self.ready = YES;
}
- (void)setReady:(BOOL)ready {
if (ready != _ready) {
[self willChangeValueForKey:#"isReady"];
_ready = ready;
[self didChangeValueForKey:#"isReady"];
}
}
- (void)addDependency:(NSOperation *)operation{
NSAssert(FALSE, #"You should not use addDependency with this custom operation");
}
Then, your app code could do something like, using addKeyDependency rather than addDependency, and explicitly either removeKeyDependency or cancel in the completion blocks:
PostOperation *postOperation = [[PostOperation alloc] init];
for (NSInteger i = 0; i < numberOfImages; i++) {
NSURL *url = ...
NSURLRequest *request = [NSURLRequest requestWithURL:url];
NSString *key = [url absoluteString]; // or you could use whatever unique value you want
AFHTTPRequestOperation *operation = [[AFHTTPRequestOperation alloc] initWithRequest:request];
[operation setCompletionBlockWithSuccess:^(AFHTTPRequestOperation *operation, id responseObject) {
// update your model or do whatever
// now inform the post operation that this operation is done
[postOperation removeKeyDependency:key];
} failure:^(AFHTTPRequestOperation *operation, NSError *error) {
// handle the error any way you want
// perhaps you want to cancel the postOperation; you'd either cancel it or remove the dependency
[postOperation cancel];
}];
[postOperation addKeyDependency:key];
[queue addOperation:operation];
}
[queue addOperation:postOperation];
This is using AFHTTPRequestOperation, and you'd obviously replace all of this logic with the appropriate AFNetworking operation for your upload, but hopefully it illustrates the idea.
Original answer:
A few observations:
As I think you concluded, when your operation completes, it (a) initiates its completion block; (b) makes the queue available for other operations (either operations that had not yet started because of maxConcurrentOperationCount, or because of dependencies between the operations). I do not believe that you have any assurances that the completion block will be done before that next operation commences.
Empirically, it looks like the dependent operation does not actually trigger until after the completion blocks are done, but (a) I don't see that documented anywhere and (b) this is moot because if you're using AFNetworking's own setCompletionBlockWithSuccess, it ends up dispatching the block asynchronously to the main queue (or the defined successCallbackQueue), thereby thwarting any (undocumented) assurances of synchrony.
Furthermore, you say that the completion block runs in the main thread. If you're talking about the built in NSOperation completion block, you have no such assurances. In fact, the setCompletionBlock documentation says:
The exact execution context for your completion block is not guaranteed but is typically a secondary thread. Therefore, you should not use this block to do any work that requires a very specific execution context. Instead, you should shunt that work to your application’s main thread or to the specific thread that is capable of doing it. For example, if you have a custom thread for coordinating the completion of the operation, you could use the completion block to ping that thread.
But if you're talking about one of AFNetworking's custom completion blocks, e.g. those that you might set with AFHTTPRequestOperation's setCompletionBlockWithSuccess, then, yes, it's true that those are generally dispatched back to the main queue. But AFNetworking does this using the standard completionBlock mechanism, so the above concerns still apply.
It matters if your NSOperation is a subclass of AFHTTPRequestOperation. AFHTTPRequestOperation uses the NSOperation's property completionBlock for its own purpose in method setCompletionBlockWithSuccess:failure. In that case, don't set the property completionBlock yourself!
It seems, AFHTTPRequestOperation's success and failure handler will run on the main thread.
Otherwise, the execution context of NSOperation's completion block is "undefined". That means, the completion block can execute on any thread/queue. In fact it executes on some private queue.
IMO, this is the preferred approach, unless the execution context shall be explicitly specified by the call-site. Executing completion handlers on threads or queues which instances are accessible (the main thread for example) can easily cause dead locks by an unwary developer.
Edit:
If you want to start a dependent operation after the completion block of the parent operation has been finished, you can solve that by making the completion block content itself a NSBlockOperation (a new parent) and add this operation as a dependency to the children operation and start it in a queue. You may realize, that this quickly becomes unwieldy, though.
Another approach would require an utility class or class library which is especially suited to solve asynchronous problems in a more concise and easy way. ReactiveCocoa would be capable to solve such (an easy) problem. However, it's unduly complex and it actually has a "learning curve" - and a steep one. I wouldn't recommend it, unless you agree to spend a few weeks in learning it and have a lot other asynchronous use cases and even much more complex ones.
A simpler approach would utilize "Promises" which are pretty common in JavaScript, Python, Scala and a few other languages.
Now, please read carefully, the (easy) solution is actually below:
"Promises" (sometimes called Futures or Deferred) represent the eventual result of an asynchronous task. Your fetch request is such asynchronous task. But instead specifying a completion handler, the asynchronous method/task returns a Promise:
-(Promise*) fetchThingsWithURL:(NSURL*)url;
You obtain the result - or the error - with registering a success handler block or a failure handler block like so:
Promise* thingsPromise = [self fetchThingsWithURL:url];
thingsPromise.then(successHandlerBlock, failureHandlerBlock);
or, the blocks inlined:
thingsPromise.then(^id(id things){
// do something with things
return <result of success handler>
}, ^id(NSError* error){
// Ohps, error occurred
return <result of failure handler>
});
And shorter:
[self fetchThingsWithURL:url]
.then(^id(id result){
return [self.parser parseAsync:result];
}, nil);
Here, parseAsync: is an asynchronous method which returns a Promise. (Yes, a Promise).
You might wonder how to get the result from the parser?
[self fetchThingsWithURL:url]
.then(^id(id result){
return [self.parser parseAsync:result];
}, nil)
.then(^id(id parserResult){
NSLog(#"Parser returned: %#", parserResult);
return nil; // result not used
}, nil);
This actually starts async task fetchThingsWithURL:. Then when finished successfully, it starts async task parseAsync:. Then when this finished successfully, it prints the result, otherwise it prints the error.
Invoking several asynchronous tasks sequentially, one after the other, is called "continuation" or "chaining".
Note that the whole statement above is asynchronous! That is, when you wrap the above statement into a method, and execute it, the method returns immediately.
You might wonder how to catch any errors, say fetchThingsWithURL: fails, or parseAsync::
[self fetchThingsWithURL:url]
.then(^id(id result){
return [self.parser parseAsync:result];
}, nil)
.then(^id(id parserResult){
NSLog(#"Parser returned: %#", parserResult);
return nil; // result not used
}, nil)
.then(/*succes handler ignored*/, ^id (NSError* error){
// catch any error
NSLog(#"ERROR: %#", error);
return nil; // result not used
});
Handlers execute after the corresponding task has been finished (of course). If the task succeeds, the success handler will be called (if any). If the tasks fails, the error handler will be called (if any).
Handlers may return a Promise (or any other object). For example, if an asynchronous task finished successfully, its success handler will be invoked which starts another asynchronous task, which returns the promise. And when this is finished, yet another one can be started, and so force. That's "continuation" ;)
You can return anything from a handler:
Promise* finalResult = [self fetchThingsWithURL:url]
.then(^id(id result){
return [self.parser parseAsync:result];
}, nil)
.then(^id(id parserResult){
return #"OK";
}, ^id(NSError* error){
return error;
});
Now, finalResult will either eventually become the value #"OK" or an NSError.
You can save the eventual results into an array:
array = #[
[self task1],
[self task2],
[self task3]
];
and then continue when all tasks have been finished successfully:
[Promise all:array].then(^id(results){
...
}, ^id (NSError* error){
...
});
Setting a promise's value will be called: "resolving". You can resolve a promise only ONCE.
You may wrap any asynchronous method with a completion handler or completion delegates into a method which returns a promise:
- (Promise*) fetchUserWithURL:(NSURL*)url
{
Promise* promise = [Promise new];
HTTPOperation* op = [[HTTPOperation alloc] initWithRequest:request
success:^(NSData* data){
[promise fulfillWithValue:data];
}
failure:^(NSError* error){
[promise rejectWithReason:error];
}];
[op start];
return promise;
}
Upon completion of the task, the promise can be "fulfilled" passing it the result value, or it can be "rejected" passing it the reason (error).
Depending on the actual implementation, a Promise can also be cancelled. Say, you hold a reference to a request operation:
self.fetchUserPromise = [self fetchUsersWithURL:url];
You can cancel the asynchronous task as follows:
- (void) viewWillDisappear:(BOOL)animate {
[super viewWillDisappear:animate];
[self.fetchUserPromise cancel];
self.fetchUserPromise = nil;
}
In order to cancel the associated async task, register a failure handler in the wrapper:
- (Promise*) fetchUserWithURL:(NSURL*)url
{
Promise* promise = [Promise new];
HTTPOperation* op = ...
[op start];
promise.then(nil, ^id(NSError* error){
if (promise.isCancelled) {
[op cancel];
}
return nil; // result unused
});
return promise;
}
Note: you can register success or failure handlers, when, where and as many as you want.
So, you can do a lot with promises - and even more than in this brief introduction. If you read up to here, you might get an idea how to solve your actual problem. It's right there - and it's a few lines of code.
I admit, that this short introduction into promises was quite rough and it's also quite new to Objective-C developers, and may sound uncommon.
You can read a lot about promises in the JS community. There are one or three implementations in Objective-C. The actual implementation won't exceed a few hundred lines of code. It happens, that I'm the author of one of it:
RXPromise.
Take it with a grain of salt, I'm probably totally biased, and apparently all others ever dealt with Promises, too. ;)
So I am attempting to throw together a simple test to verify that I am receiving frequency values from my audioController correctly.
In my view I am making a call like this to setup up a block callback:
- (void) registerVolumeCallback {
NSNumberBlock frequencyCallback = ^(NSNumber *frequency) {
self.currentFrequency = frequency;
};
self.audioController.frequencyCallback = frequencyCallback;
}
In my audio controller the frequency callback block is called with an nsnumber containing the frequency.
In my tests file I have the following:
- (void) testFrequencyAudioServiceCallbackActive {
OCMockObject *mockEqualizer = [OCMockObject partialMockForObject:self.testEqualizer];
[[[mockEqualizer stub] andCall:#selector(mockDidUpdateFrequency:)
onObject:self] setCurrentFrequency:[OCMArg any]];
[self.testEqualizer startAnimating];
[ mockEqualizer verify];
}
And:
- (void) mockDidUpdateFrequency: (NSNumber *) frequency {
GHAssertTrue((frequency!= nil), #"Audio Service is messing up");
}
Where test equalizer is an an instance of the aforementioned view. So Im trying to do some swizzling here. Problem is, mockDidUpdateFrequency is never called. I tried putting:
self.currentFrequency = frequency;
outside of the block, and the swizzling does happen and I do get a call to mockDidUpdateFrequency. I also tried:
- (void) registerVolumeCallback {
__block UIEqualizer *blockSafeSelf = self;
NSNumberBlock frequencyCallback = ^(NSNumber *frequency) {
blockSafeSelf.currentFrequency = frequency;
};
self.audioController.frequency = frequencyCallback;
}
No luck. Some weird instance stuff is going on here in the block context that I am not aware of. Anyone know whats happening?
You'll need to provide some more details for a definitive answer. For example, how is registerVolumeCallback invoked? And is frequencyCallback your own code, or a third-party API?
With what you've provided, I suspect that frequencyCallback is an asynchronous call. So even though startAnimating might create the condition where it will eventually be invoked, you immediately verify the mock before the callback has had a chance to be invoked. To get your test to do what you want as written, you need to understand what queue that block is executed on, and you need to give it a chance to execute.
If it's invoked asynchronously on the main queue, you can let the main run loop spin before calling verify:
[self.testEqualizer startAnimating];
[[NSRunLoop currentRunLoop] runUntilDate:[NSDate dateWithTimeIntervalSinceNow:.1]];
[mockEqualizer verify];
If it's invoked on a different queue, you have some different options, but it would help to have a clearer picture how your code is structured first.
I’m writing a library to export assets to a file using AVFoundation. I create a reader, a writer, connect the inputs and outputs to these and then call the requestMediaDataWhenReadyOnQueue method on the inputs to start pulling the data. The block callback supplied to this method looks a bit like this:
[input requestMediaDataWhenReadyOnQueue:queue usingBlock:^{
while ([input isReadyForMoreMediaData]) {
CMSampleBufferRef buffer;
// The track has some more data for us
if ([reader status] == AVAssetReaderStatusReading
&& (buffer = [output copyNextSampleBuffer])) {
BOOL result = [input appendSampleBuffer:buffer];
CFRelease(buffer);
if (!result) {
// handle error
break;
}
// The track is finished, for whatever reason
} else {
[input markAsFinished]; ⬅
switch ([reader status]) {
// inspect the status and act accordingly
}
}
}
}];
This works perfectly on iOS 5, but on iOS 4 the code dies from EXC_BAD_ACCESS after the line marked with the ⬅ arrow. After some poking around I feel like the block was somehow destroyed immediately after marking the input as finished. The self pointer that’s perfectly valid before executing the bad line somehow turns into 0xfff… or some garbage value as reported by the debugger. But the object pointed to it before is fine, as confirmed by the zombies tool, it does not get deallocated.
What am I missing?
Seeing the same (similar) issue. iOS5 happy, iOS4.3.5, not happy. Interested to learn what you ultimately find.
Got around it by explicitly retaining writer, writer input, reader, reader output before the requestMedatWhenReadyOnQueue block and explicitly releasing all four at the very end of the else clause.
The doc does say that after marking finished, "The block should then exit." Maybe they are not kidding. If you do anything other than exit, it is an error. The above workaround seems to work though.
UPDATE: I still found that it occasionally crashed even after retaining and releasing all of the asset objects. As your question observes, it crashes shortly after you mark the writer input as finished it is as if the block itself is being deallocated. Rather than just pass the block as part of the function. I create a copied block property that is part of a long lived object. I initialize it with Block_copy and only release it in the destructor of the long lived object. This seems to do the trick. I haven't seen any 4.3.5 crashes since.
Try [self retain] as the first line of the block and [self release] as the last line.
Another critical issue is that if the App is suspended (enters background) using requestMediaDataWhenReadyOnQueue you need to explicitly cover all of the [reader status] values as it will fail when the app restarts. In some cases I found the block ran more than once with a fail status flag. In other posts with similar code there's a lot of [retain]ing of the AV variables, which are then released at the end of the block. Because the block can run more than once this approach doesn't work in cases when the app enters the background state.
I found the following to work well in the "switch" (above):
case AVAssetReaderStatusReading:
break;
case AVAssetReaderStatusCompleted:
[videoWriterInput markAsFinished];
//do something else, like add an audio stream
[videoWriter finishWriting];
break;
case AVAssetReaderStatusFailed:
[videoWriterInput markAsFinished];
[videoWriter finishWriting];
break;
case AVAssetReaderStatusCancelled:
case AVAssetReaderStatusUnknown:
[videoWriterInput markAsFinished];
[videoWriter cancelWriting];
break;
}
dispatch_sync(dispatch_get_main_queue(), ^{
//hide any progress indicators
});
break;
other than "self", nothing is retained. The block should retain the variables automatically if they are required.