I’m working in a new codebase and I don’t have many people who understand it, so I’m hoping I can get some help. I am updating an interface and some of the synchronous methods are now async which is making it difficult to fit into the current architecture for resolving data.
Currently we have a function map which stores these synchronous methods, then when we want the data we do “call” which executes the block/method and returns the value.
Some code below shows how it currently is.
fnMap[#“vid”] = [[Callback alloc] initWithBlock:^id(id param) {
return #([services getVisitorID]);
}];
… later, to resolve the data
id fnMapVal = [fnMap[key] call:nil];
Here is how a callback and callback block are defined.
typedef id (^CallbackBlock)(id);
#interface Callback : NSObject
#property(copy, nonatomic, readonly) CallbackBlock block;
- (instancetype)initWithBlock:(CallbackBlock)block
- (id)call:(id)param
{
return self.block(param);
}
Now the service needs to call an async method to get the ID so I had to change it to:
- (void)getVisitorID: (nullable void (^) (NSString* __nullable visitorIdentifier)) callback
{
[SDK getUserIdentifier:^(NSString * _Nullable userIdentifier) {
callback(userIdentifier);
}];
}
So the call is:
[services getVisitorID:^(NSString * _Nullable visitorIdentifier) {
}];
I haven’t been able to find a way to fit this into the current architecture. Some options I’ve explored is using a run loop to wait for the async method to finish and keep my interface synchronous but this sounds like a bad idea. I’m for some suggestions on how to fit this in as I’ve never seen something like this before.
You need to use dispatch_queues or NSOperationQueue to run your code off the main thread. Dispatch queues are very low level and good to just fire off async tasks:
// we're going to run the getVisitorID method on a background queue
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
[services getVisitorID:^(NSString * _Nullable visitorIdentifier) {
dispatch_async(dispatch_get_main_queue(), ^(void){
// update the user interface on the main queue
});
}];
});
I prefer using NSOperationQueue because the API is cleaner, and they allow you to do more advanced things like make asynchronous task cancellable:
// create a background queue
NSOperationQueue *queue = [[NSOperationQueue alloc] init];
[queue addOperationWithBlock:^{
[services getVisitorID:^(NSString * _Nullable visitorIdentifier) {
// get the main queue and add your UI update code to it
[[NSOperationQueue mainQueue] addOperationWithBlock:^{
// update the UI from here
}];
}];
}];
Queues are much easier to manage than messing around with run loops. For more info see here: https://developer.apple.com/library/archive/documentation/General/Conceptual/ConcurrencyProgrammingGuide/OperationQueues/OperationQueues.html
What you need is the Future/Promise concept. Please don't reinvent the wheel, you will have to cover marathon distance trying to achieve the same functionality of requesting value and asynchronously consuming it.
As one of the favourites please consider PromiseKit. However don't be shy to explore the alternatives
You will quickly find it super delightful building chains of async tasks, mapping their results to different values, combining futures together to get a tuple of multiple resolved values once all are available - all that in a concise, well designed form, live tested in millions of applications.
The part of a method that I am trying to test is as follows:
- (void)configureTableFooterView {
dispatch_async(dispatch_get_main_queue(), ^{
self.tableView.tableFooterView = nil;
if ([self.parser.resultSet isLastPage]) {
return;
}
});
}
I have written the unit test as follows:
- (void)testTableFooterViewConfigurationAfterLastPageLoaded {
id mockTableView = OCMClassMock([GMGFlatTableView class]);
OCMExpect([mockTableView setTableFooterView:[OCMArg isNil]]);
id resultSet = OCMClassMock([GMGResultSetInfo class]);
OCMStub([resultSet isLastPage]).andReturn(YES);
OCMStub([self.mockParser resultSet]).andReturn(resultSet);
id partialMockSUT = OCMPartialMock(self.sut);
OCMStub([partialMockSUT tableView]).andReturn(mockTableView);
[self.sut configureTableFooterView];
OCMVerifyAllWithDelay(mockTableView, 2.0);
//OCMVerifyAllWithDelay(partialMockSUT, 2.0);
}
I have another test in the same class which is testing the same things from with in the dispatch_async call on the main thread. The test expectations and verification setup in that test match this one. While that test passes, this one gets stuck in an infinite loop at the delayed verification step.
Interestingly, if I only run this 1 test, it passes with out any problems. Its only when this test is run with other tests that I see the problem.
UPDATE:
In unit test, execute the block passed in queue with dispatch_asyc
This is a much more relevant post. However, this fails almost in the exact same way as the original test method:
- (void)testTableFooterViewConfigurationAfterLastPageLoaded {
id mockTableView = OCMClassMock([GMGFlatTableView class]);
OCMExpect([mockTableView setTableFooterView:[OCMArg isNil]]);
id resultSet = OCMClassMock([GMGResultSetInfo class]);
OCMStub([resultSet isLastPage]).andReturn(YES);
OCMStub([self.mockParser resultSet]).andReturn(resultSet);
id partialMockSUT = OCMPartialMock(self.sut);
OCMStub([partialMockSUT tableView]).andReturn(mockTableView);
[self.sut configureTableFooterView];
[[NSRunLoop mainRunLoop] runUntilDate:[NSDate dateWithTimeIntervalSinceNow:0.01]];
OCMVerifyAll(mockTableView);
}
The line with NSRunLoop crashes with EXC_BAD_ACCESS when run as suite but runs fine alone!
You can make class wrapper around dispatch_async, and pass it as dependency. Also you can make fake wrapper, and pass it in tests. If you interested in, I can provide much more detailed explanation.
From the docs:
The completion block you provide is executed when the value returned by the isFinished method changes to YES. Thus, this block is executed by the operation object after the operation’s primary task is finished or cancelled.
I'm using RestKit/AFNetworking, if that matters.
I have multiple dependencies in my NSOperation in a OperationQueue. I use the completion block to set some variables (appending the results to an array) that my child requires.
(task1,...,taskN) -> taskA
taskA addDependency: task1-taskN
Will taskA receive incomplete data since the child can execute before the completion block is fired?
Reference
Do NSOperations and their completionBlocks run concurrently?
I did a simple test by adding a sleep in my completion block and I had a different result. The completion block runs in the main thread. While all the completion block are sleeping, the child task ran.
As I discuss below under "a few observations", you have no assurances that this final dependent operation will not start before your other sundry AFNetworking completion blocks have finished. It strikes me that if this final operation really needs to wait for these completion blocks to finish, then you have a couple of alternatives:
Use semaphores within each of the n the completion blocks to signal when they're done and have the completion operation wait for n signals; or
Don't queue this final operation up front, but rather have your completion blocks for the individual uploads keep track of how many pending uploads are still incomplete, and when it falls to zero, then initiate the final "post" operation.
As you pointed out in your comments, you could wrap your invocation of the AFNetworking operation and its completion handler in your own operation, at which point you can then use the standard addDependency mechanism.
You could abandon the addDependency approach (which adds an observer on the isFinished key of the operation upon which this operation is dependent, and once all those dependencies are resolved, performs the isReady KVN; the problem being that this can theoretically happen before your completion block is done) and replace it with your own isReady logic. For example, imagine you had a post operation which you could add your own key dependencies and remove them manually in your completion block, rather than having them removed automatically upon isFinished. Thus, you custom operation
#interface PostOperation ()
#property (nonatomic, getter = isReady) BOOL ready;
#property (nonatomic, strong) NSMutableArray *keys;
#end
#implementation PostOperation
#synthesize ready = _ready;
- (void)addKeyDependency:(id)key {
if (!self.keys)
self.keys = [NSMutableArray arrayWithObject:key];
else
[self.keys addObject:key];
self.ready = NO;
}
- (void)removeKeyDependency:(id)key {
[self.keys removeObject:key];
if ([self.keys count] == 0)
self.ready = YES;
}
- (void)setReady:(BOOL)ready {
if (ready != _ready) {
[self willChangeValueForKey:#"isReady"];
_ready = ready;
[self didChangeValueForKey:#"isReady"];
}
}
- (void)addDependency:(NSOperation *)operation{
NSAssert(FALSE, #"You should not use addDependency with this custom operation");
}
Then, your app code could do something like, using addKeyDependency rather than addDependency, and explicitly either removeKeyDependency or cancel in the completion blocks:
PostOperation *postOperation = [[PostOperation alloc] init];
for (NSInteger i = 0; i < numberOfImages; i++) {
NSURL *url = ...
NSURLRequest *request = [NSURLRequest requestWithURL:url];
NSString *key = [url absoluteString]; // or you could use whatever unique value you want
AFHTTPRequestOperation *operation = [[AFHTTPRequestOperation alloc] initWithRequest:request];
[operation setCompletionBlockWithSuccess:^(AFHTTPRequestOperation *operation, id responseObject) {
// update your model or do whatever
// now inform the post operation that this operation is done
[postOperation removeKeyDependency:key];
} failure:^(AFHTTPRequestOperation *operation, NSError *error) {
// handle the error any way you want
// perhaps you want to cancel the postOperation; you'd either cancel it or remove the dependency
[postOperation cancel];
}];
[postOperation addKeyDependency:key];
[queue addOperation:operation];
}
[queue addOperation:postOperation];
This is using AFHTTPRequestOperation, and you'd obviously replace all of this logic with the appropriate AFNetworking operation for your upload, but hopefully it illustrates the idea.
Original answer:
A few observations:
As I think you concluded, when your operation completes, it (a) initiates its completion block; (b) makes the queue available for other operations (either operations that had not yet started because of maxConcurrentOperationCount, or because of dependencies between the operations). I do not believe that you have any assurances that the completion block will be done before that next operation commences.
Empirically, it looks like the dependent operation does not actually trigger until after the completion blocks are done, but (a) I don't see that documented anywhere and (b) this is moot because if you're using AFNetworking's own setCompletionBlockWithSuccess, it ends up dispatching the block asynchronously to the main queue (or the defined successCallbackQueue), thereby thwarting any (undocumented) assurances of synchrony.
Furthermore, you say that the completion block runs in the main thread. If you're talking about the built in NSOperation completion block, you have no such assurances. In fact, the setCompletionBlock documentation says:
The exact execution context for your completion block is not guaranteed but is typically a secondary thread. Therefore, you should not use this block to do any work that requires a very specific execution context. Instead, you should shunt that work to your application’s main thread or to the specific thread that is capable of doing it. For example, if you have a custom thread for coordinating the completion of the operation, you could use the completion block to ping that thread.
But if you're talking about one of AFNetworking's custom completion blocks, e.g. those that you might set with AFHTTPRequestOperation's setCompletionBlockWithSuccess, then, yes, it's true that those are generally dispatched back to the main queue. But AFNetworking does this using the standard completionBlock mechanism, so the above concerns still apply.
It matters if your NSOperation is a subclass of AFHTTPRequestOperation. AFHTTPRequestOperation uses the NSOperation's property completionBlock for its own purpose in method setCompletionBlockWithSuccess:failure. In that case, don't set the property completionBlock yourself!
It seems, AFHTTPRequestOperation's success and failure handler will run on the main thread.
Otherwise, the execution context of NSOperation's completion block is "undefined". That means, the completion block can execute on any thread/queue. In fact it executes on some private queue.
IMO, this is the preferred approach, unless the execution context shall be explicitly specified by the call-site. Executing completion handlers on threads or queues which instances are accessible (the main thread for example) can easily cause dead locks by an unwary developer.
Edit:
If you want to start a dependent operation after the completion block of the parent operation has been finished, you can solve that by making the completion block content itself a NSBlockOperation (a new parent) and add this operation as a dependency to the children operation and start it in a queue. You may realize, that this quickly becomes unwieldy, though.
Another approach would require an utility class or class library which is especially suited to solve asynchronous problems in a more concise and easy way. ReactiveCocoa would be capable to solve such (an easy) problem. However, it's unduly complex and it actually has a "learning curve" - and a steep one. I wouldn't recommend it, unless you agree to spend a few weeks in learning it and have a lot other asynchronous use cases and even much more complex ones.
A simpler approach would utilize "Promises" which are pretty common in JavaScript, Python, Scala and a few other languages.
Now, please read carefully, the (easy) solution is actually below:
"Promises" (sometimes called Futures or Deferred) represent the eventual result of an asynchronous task. Your fetch request is such asynchronous task. But instead specifying a completion handler, the asynchronous method/task returns a Promise:
-(Promise*) fetchThingsWithURL:(NSURL*)url;
You obtain the result - or the error - with registering a success handler block or a failure handler block like so:
Promise* thingsPromise = [self fetchThingsWithURL:url];
thingsPromise.then(successHandlerBlock, failureHandlerBlock);
or, the blocks inlined:
thingsPromise.then(^id(id things){
// do something with things
return <result of success handler>
}, ^id(NSError* error){
// Ohps, error occurred
return <result of failure handler>
});
And shorter:
[self fetchThingsWithURL:url]
.then(^id(id result){
return [self.parser parseAsync:result];
}, nil);
Here, parseAsync: is an asynchronous method which returns a Promise. (Yes, a Promise).
You might wonder how to get the result from the parser?
[self fetchThingsWithURL:url]
.then(^id(id result){
return [self.parser parseAsync:result];
}, nil)
.then(^id(id parserResult){
NSLog(#"Parser returned: %#", parserResult);
return nil; // result not used
}, nil);
This actually starts async task fetchThingsWithURL:. Then when finished successfully, it starts async task parseAsync:. Then when this finished successfully, it prints the result, otherwise it prints the error.
Invoking several asynchronous tasks sequentially, one after the other, is called "continuation" or "chaining".
Note that the whole statement above is asynchronous! That is, when you wrap the above statement into a method, and execute it, the method returns immediately.
You might wonder how to catch any errors, say fetchThingsWithURL: fails, or parseAsync::
[self fetchThingsWithURL:url]
.then(^id(id result){
return [self.parser parseAsync:result];
}, nil)
.then(^id(id parserResult){
NSLog(#"Parser returned: %#", parserResult);
return nil; // result not used
}, nil)
.then(/*succes handler ignored*/, ^id (NSError* error){
// catch any error
NSLog(#"ERROR: %#", error);
return nil; // result not used
});
Handlers execute after the corresponding task has been finished (of course). If the task succeeds, the success handler will be called (if any). If the tasks fails, the error handler will be called (if any).
Handlers may return a Promise (or any other object). For example, if an asynchronous task finished successfully, its success handler will be invoked which starts another asynchronous task, which returns the promise. And when this is finished, yet another one can be started, and so force. That's "continuation" ;)
You can return anything from a handler:
Promise* finalResult = [self fetchThingsWithURL:url]
.then(^id(id result){
return [self.parser parseAsync:result];
}, nil)
.then(^id(id parserResult){
return #"OK";
}, ^id(NSError* error){
return error;
});
Now, finalResult will either eventually become the value #"OK" or an NSError.
You can save the eventual results into an array:
array = #[
[self task1],
[self task2],
[self task3]
];
and then continue when all tasks have been finished successfully:
[Promise all:array].then(^id(results){
...
}, ^id (NSError* error){
...
});
Setting a promise's value will be called: "resolving". You can resolve a promise only ONCE.
You may wrap any asynchronous method with a completion handler or completion delegates into a method which returns a promise:
- (Promise*) fetchUserWithURL:(NSURL*)url
{
Promise* promise = [Promise new];
HTTPOperation* op = [[HTTPOperation alloc] initWithRequest:request
success:^(NSData* data){
[promise fulfillWithValue:data];
}
failure:^(NSError* error){
[promise rejectWithReason:error];
}];
[op start];
return promise;
}
Upon completion of the task, the promise can be "fulfilled" passing it the result value, or it can be "rejected" passing it the reason (error).
Depending on the actual implementation, a Promise can also be cancelled. Say, you hold a reference to a request operation:
self.fetchUserPromise = [self fetchUsersWithURL:url];
You can cancel the asynchronous task as follows:
- (void) viewWillDisappear:(BOOL)animate {
[super viewWillDisappear:animate];
[self.fetchUserPromise cancel];
self.fetchUserPromise = nil;
}
In order to cancel the associated async task, register a failure handler in the wrapper:
- (Promise*) fetchUserWithURL:(NSURL*)url
{
Promise* promise = [Promise new];
HTTPOperation* op = ...
[op start];
promise.then(nil, ^id(NSError* error){
if (promise.isCancelled) {
[op cancel];
}
return nil; // result unused
});
return promise;
}
Note: you can register success or failure handlers, when, where and as many as you want.
So, you can do a lot with promises - and even more than in this brief introduction. If you read up to here, you might get an idea how to solve your actual problem. It's right there - and it's a few lines of code.
I admit, that this short introduction into promises was quite rough and it's also quite new to Objective-C developers, and may sound uncommon.
You can read a lot about promises in the JS community. There are one or three implementations in Objective-C. The actual implementation won't exceed a few hundred lines of code. It happens, that I'm the author of one of it:
RXPromise.
Take it with a grain of salt, I'm probably totally biased, and apparently all others ever dealt with Promises, too. ;)
Following TDD I'm developing an iPad app that downloads some info from the internet and displays it on a list, allowing the user to filter that list using a search bar.
I want to test that, as the user types in the search bar, the internal variable with the filter text is updated, the filtered list of items is updated, and finally the table view receives a "reloadData" message.
These are my tests:
- (void)testSutChangesFilterTextWhenSearchBarTextChanges
{
// given
sut.filterText = #"previous text";
// when
[sut searchBar:nil textDidChange:#"new text"];
// then
assertThat(sut.filterText, is(equalTo(#"new text")));
}
- (void)testSutReloadsTableViewDataAfterChangeFilterTextFromSearchBar
{
// given
sut.tableView = mock([UITableView class]);
// when
[sut searchBar:nil textDidChange:#"new text"];
// then
[verify(sut.tableView) reloadData];
}
NOTE: Changing the "filterText" property triggers right now the actual filtering process, which has been tested in other tests.
This works OK as my searchBar delegate code was written as follows:
- (void)searchBar:(UISearchBar *)searchBar textDidChange:(NSString *)searchText
{
self.filterText = searchText;
[self.tableView reloadData];
}
The problem is that filtering this data is becoming a heavy process that right now is being done on the main thread, so during that time the UI is blocked.
Therefore, I thought of doing something like this:
- (void)searchBar:(UISearchBar *)searchBar textDidChange:(NSString *)searchText
{
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
NSArray *filteredData = [self filteredDataWithText:searchText];
dispatch_async(dispatch_get_main_queue(), ^{
self.filteredData = filteredData;
[self.tableView reloadData];
});
});
}
So that the filtering process occurs in a different thread and when it has finished, the table is asked to reload its data.
The question is... how do I test these things inside dispatch_async calls?
Is there any elegant way of doing that other than time-based solutions? (like waiting for some time and expect that those tasks have finished, not very deterministic)
Or maybe I should put my code on a different way to make it more testable?
In case you need to know, I'm using OCMockito and OCHamcrest by Jon Reid.
Thanks in advance!!
There are two basic approaches. Either
Make things synchronous only while testing. Or,
Keep things asynchronous, but write an acceptance test that does resynchronizing.
To make things synchronous for testing only, extract the code that actually does work into their own methods. You already have -filteredDataWithText:. Here's another extraction:
- (void)updateTableWithFilteredData:(NSArray *)filteredData
{
self.filteredData = filteredData;
[self.tableView reloadData];
}
The real method that takes care of all the threading now looks like this:
- (void)searchBar:(UISearchBar *)searchBar textDidChange:(NSString *)searchText
{
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
NSArray *filteredData = [self filteredDataWithText:searchText];
dispatch_async(dispatch_get_main_queue(), ^{
[self updateTableWithFilteredData:filteredData];
});
});
}
Notice that underneath all that threading fanciness, it really just calls two methods. So now to pretend that all that threading was done, have your tests just invoke those two methods in order:
NSArray *filteredData = [self filteredDataWithText:searchText];
[self updateTableWithFilteredData:filteredData];
This does mean that -searchBar:textDidChange: won't be covered by unit tests. A single manual test can confirm that it's dispatching the right things.
If you really want an automated test on the delegate method, write an acceptance test that has its own run loop. See Pattern for unit testing async queue that calls main queue on completion. (But keep acceptance tests in a separate test target. They're too slow to include with unit tests.)
Albite Jons options are very good options most of the time, sometime it creates less cluttered code when doing the following. For example if your API has a lot small methods that are synchronised using a dispatch queue.
Have a function like this (it could be a method of your class as well).
void dispatch(dispatch_queue_t queue, void (^block)())
{
if(queue)
{
dispatch_async(queue, block);
}
else
{
block();
}
}
Then use this function to call the blocks in your API methods
- (void)anAPIMethod
{
dispatch(dispQueue, ^
{
// dispatched code here
});
}
You would usually initialise the queue in your init method.
#implementation MyAPI
{
dispatch_queue_t dispQueue;
}
- (instancetype)init
{
self = [super init];
if (self)
{
dispQueue = dispatch_queue_create("myQueue", DISPATCH_QUEUE_SERIAL);
}
return self;
}
Then have a private method like this, to set this queue to nil. It is not part of your interface, the API consumer will never see this.
- (void) disableGCD
{
dispQueue = nil;
}
In your test target you create a category to expose the GCD disabling method:
#interface TTBLocationBasedTrackStore (Testing)
- (void) disableGCD;
#end
You call this in your test setup and your blocks will be called directly.
The advantage in my eyes is debugging. When a test case involves a runloop so that blocks are actually called, the problem is that there has to be a timeout involved. This timeout is usually quite short because you don't want to have tests that last long if the they run into the timeout. But having a short timeout means your test runs into the timeout when debugging.
So I am attempting to throw together a simple test to verify that I am receiving frequency values from my audioController correctly.
In my view I am making a call like this to setup up a block callback:
- (void) registerVolumeCallback {
NSNumberBlock frequencyCallback = ^(NSNumber *frequency) {
self.currentFrequency = frequency;
};
self.audioController.frequencyCallback = frequencyCallback;
}
In my audio controller the frequency callback block is called with an nsnumber containing the frequency.
In my tests file I have the following:
- (void) testFrequencyAudioServiceCallbackActive {
OCMockObject *mockEqualizer = [OCMockObject partialMockForObject:self.testEqualizer];
[[[mockEqualizer stub] andCall:#selector(mockDidUpdateFrequency:)
onObject:self] setCurrentFrequency:[OCMArg any]];
[self.testEqualizer startAnimating];
[ mockEqualizer verify];
}
And:
- (void) mockDidUpdateFrequency: (NSNumber *) frequency {
GHAssertTrue((frequency!= nil), #"Audio Service is messing up");
}
Where test equalizer is an an instance of the aforementioned view. So Im trying to do some swizzling here. Problem is, mockDidUpdateFrequency is never called. I tried putting:
self.currentFrequency = frequency;
outside of the block, and the swizzling does happen and I do get a call to mockDidUpdateFrequency. I also tried:
- (void) registerVolumeCallback {
__block UIEqualizer *blockSafeSelf = self;
NSNumberBlock frequencyCallback = ^(NSNumber *frequency) {
blockSafeSelf.currentFrequency = frequency;
};
self.audioController.frequency = frequencyCallback;
}
No luck. Some weird instance stuff is going on here in the block context that I am not aware of. Anyone know whats happening?
You'll need to provide some more details for a definitive answer. For example, how is registerVolumeCallback invoked? And is frequencyCallback your own code, or a third-party API?
With what you've provided, I suspect that frequencyCallback is an asynchronous call. So even though startAnimating might create the condition where it will eventually be invoked, you immediately verify the mock before the callback has had a chance to be invoked. To get your test to do what you want as written, you need to understand what queue that block is executed on, and you need to give it a chance to execute.
If it's invoked asynchronously on the main queue, you can let the main run loop spin before calling verify:
[self.testEqualizer startAnimating];
[[NSRunLoop currentRunLoop] runUntilDate:[NSDate dateWithTimeIntervalSinceNow:.1]];
[mockEqualizer verify];
If it's invoked on a different queue, you have some different options, but it would help to have a clearer picture how your code is structured first.