Barrier operations in NSOperationQueue - objective-c

How can we implement dispatch_barrier_async's equivalent behavior using NSOperationQueue or any user-defined data-structure based on NSOperationQueue?
The requirement is, whenever a barrier operation is submitted it should wait until all non-barrier operations submitted earlier finish their execution and blocks other operations submitted after that.
Non-barrier operations should be able to perform concurrently.
Barrier operations should execute serially.
NB: Not using GCD,as it doesn't provide(or atleast difficult) much access over the operations, like cancelling single operation, etc.

This is more or less what jeffamaphone was saying, but I put up a gist that should, in rough outline, do what you ask.
I create a NSMutableArray of NSOperationQueues, which serves as a "queue of queues". Every time you add a BarrierOperation object, you tack a fresh suspended op queue on the end. That becomes the addingQueue, to which you add subsequent operations.
- (void)addOperation:(NSOperation *)op {
#synchronized (self) {
if ([op isKindOfClass:[BarrierOperation class]]) {
[self addBarrierOperation:(id)op];
} else {
[[self addingQueue] addOperation:op];
}
}
}
// call only from #synchronized block in -addOperation:
- (void)addBarrierOperation:(BarrierOperation *)barrierOp {
[[self addingQueue] setSuspended:YES];
for (NSOperation *op in [[self addingQueue] operations]) {
[barrierOp addDependency:op];
}
[[self addingQueue] addOperation:barrierOp];
// if you are free to set barrierOp.completionBlock, you could skip popCallback and do that
__block typeof(self) weakSelf = self;
NSOperation *popCallback = [NSBlockOperation blockOperationWithBlock:^{
[weakSelf popQueue];
}];
[popCallback addDependency:barrierOp];
[[self addingQueue] addOperation:popCallback];
[[self addingQueue] setSuspended:NO];
NSOperationQueue *opQueue = [[NSOperationQueue alloc] init];
[opQueue setSuspended:YES];
[_queueOfQueues addObject:opQueue]; // fresh empty queue to add to
}
When one NSOperationQueue finishes, it gets popped and the next one starts running.
- (void)popQueue
{
#synchronized (self) {
NSAssert([_queueOfQueues count], #"should always be one to pop");
[_queueOfQueues removeObjectAtIndex:0];
if ([_queueOfQueues count]) {
// first queue is always running, all others suspended
[(NSOperationQueue *)_queueOfQueues[0] setSuspended:NO];
}
}
}
I might have missed something crucial. The devil's in the details.
This smells a bit like a homework assignment to me. If so, tell me what grade I get. :)
Addendum: Via abhilash1912's comment, a different but similar approach. That code is tested, so it already wins. But it is a bit stale (2 years or so as of today; some deprecated method usage). Moreover, I question whether inheriting from NSOperationQueue is the best path, though it has the virtue of retaining familiarity. Regardless, if you've read this far, it's probably worth looking over.
If you create or find the world's greatest BarrierQueue class, please let us know in the comments or otherwise, so it can be linked up.

Create an NSOperation that is your barrier, then use:
- (void)addDependency:(NSOperation *)operation
To make that barrier operation dependent on all the ones you want to come before it.

I don't think it's possible to create an NSOperation object that's gives you the same sort of functionality, barriers have more to do with the way the queue operates.
The main difference between using a barrier and the dependency mechanism of NSOperations is, in the case of a barrier, the thread queue waits until all running concurrent operations have completed, and then it runs your barrier block, while making sure that any new blocks submitted and any blocks waiting do not run until the critical block has passed.
With an NSOperationQueue, it's impossible to set up the queue in such a way that it'll enforce a proper barrier: all NSOperations added to the queue before your critical NSOperation must be explicitly registered as a dependency with the critical job, and once the critical job has started, you must explicitly guard the NSOperationQueue to make sure no other clients push jobs onto it before the critical job has finished; you guard the queue by adding the critical job as a dependency for the subsequent operations.
(In the case where you know there's only one critical job at a time, this sounds sorta easy, but there will probably be n critical jobs waiting at any one time, which means keeping track of the order jobs are submitted, managing the relative dependency of critical jobs relative to their dependent jobs -- some critical jobs can wait for others, some must be executed in a particular order relative to others... yikes.)
It might be possible to get this level of functionality by setting up an NSOperationQueue with a concurrent job max of one, but that sorta defeats the purpose of doing this, I think. You might also be able to make this work by wrapping an NSOperationQueue in a facade object that protects NSOperations that are submitted "critically."

Just another way... don't hurt me.
Todo: save origin completion and self.maxConcurrentOperationCount = 1 sets queue to serial on adding. But should before execution.
#import "NSOperationQueue+BarrierOperation.h"
#implementation NSOperationQueue (BarrierOperation)
- (void)addOperationAsBarrier:(NSOperation *)op
{
//TODO: needs to save origin completion
// if (op.completionBlock)
// {
// originBlock = op.completionBlock;
// }
NSOperationQueue* qInternal = [NSOperationQueue new];
NSInteger oldMaxConcurrentOperationCount = self.maxConcurrentOperationCount;
op.completionBlock = ^{
self.maxConcurrentOperationCount = oldMaxConcurrentOperationCount;
NSLog(#"addOperationAsBarrier maxConcurrentOperationCount restored");
};
[self addOperationWithBlock:^{
self.maxConcurrentOperationCount = 1;
NSLog(#"addOperationAsBarrier maxConcurrentOperationCount = 1");
}];
[qInternal addOperationWithBlock:^{
NSLog(#"waitUntilAllOperationsAreFinished...");
[self waitUntilAllOperationsAreFinished];
}];
NSLog(#"added OperationAsBarrier");
[self addOperation:op];
}
#end

Related

NSOperationQueue notification

Help me out here or just shed some light on the problem.
I have a scenario where I perform a sync of archived messages on a openfire server and I handle and store all incoming messages with NSOperations and NSOperationQueue.
I want to get notified when the NSOperationQueue is done, but I can't simply count the number of operations it has running. At times the NSOperationQueue has 0 operations because it depends on data to arrive form the server.
The NSOperations start methods
- (void)startArchiveSyncStore:(XMPPIQ *)iq operationID:(NSString *)xmlID {
#autoreleasepool {
if (![self.pendingOperations.archiveStoreInProgress.allKeys containsObject:xmlID]) {
ArchiveStoreOperation *storeOperation = [[ArchiveStoreOperation alloc] initWithMessagesToArchive:iq withID:xmlID delegate:self];
[self.pendingOperations.archiveStoreInProgress setObject:storeOperation forKey:xmlID];
[self.pendingOperations.archiveStoreQueue addOperation:storeOperation];
}
}
}
- (void)startArchiveSycnDownload:(XMPPIQ *)iq operationID:(NSString *)xmlID {
#autoreleasepool {
if (![self.pendingOperations.archiveDownloadInProgress.allKeys containsObject:xmlID]) {
ArchiveDownloadOperation *downloadOperation = [[ArchiveDownloadOperation alloc] initWithMessagesToDownload:iq withID:xmlID delegate:self];
[self.pendingOperations.archiveDownloadInProgress setObject:downloadOperation forKey:xmlID];
[self.pendingOperations.archiveDownloadQueue addOperation:downloadOperation];
}
}
}
And this is the main thread callback performed by the NSOperation:
- (void)archiveStoreDidFinish:(ArchiveStoreOperation *)downloader {
NSString *xmlID = downloader.xmlnsID;
DDLogInfo(#"%# %#", THIS_METHOD, xmlID);
[self.pendingOperations.archiveStoreInProgress removeObjectForKey:xmlID];
}
These operations start when I receive iq stanzas containing lists of the chat history from the openfire server. Then I handle these lists like so:
- (BOOL)xmppStream:(XMPPStream *)sender didReceiveIQ:(XMPPIQ *)iq {
if ([iq isResultIQ]) {
if ([iq elementForName:#"list" xmlns:#"urn:xmpp:archive"]) {
[self startArchiveSycnDownload:iq operationID:[[iq attributeForName:#"id"] stringValue]];
}
if ([iq elementForName:#"chat" xmlns:#"urn:xmpp:archive"]) {
[self startArchiveSyncStore:iq operationID:[[iq attributeForName:#"id"] stringValue]];
}
}
return NO;
}
Any ideas folks ? Thanks in advance...
From my understanding each NSOperation has an isFinished property that you can check for. But, there is a caveat - isFinished doesn't guarantee that the operation has completed successfully. It is set to true if it succeeds but also if it has been cancelled or an error has occurred.
Obviously each queue has a count of the operations [queue.operations count] but as you've said that won't be of use here.
One alternative is to use KVO. You could try setting this up between the other object that you're using and the NSOperationQueue. You could add an observer to the queue and check that no other operations are in effect.
Also, check this SO post here if you haven't already.
I use NSNotificationCenter and post whenever a NSOperation in the last queue finishes. I assume that there is a "last" queue, aka the one that gets spun up after other queue operations have finished.
When you receive the notification check the count of all your NSOperationQueues to see if they are empty.
It's not clear from your question exactly what condition you do consider to be "done" (no operations in the queue and… what?).
One approach is to create a completion operation. As you create the other operations, add each as a dependency of the completion operation. The completion operation can be in some other queue, possibly [NSOperationQueue mainQueue]. When there are no other operations outstanding, the completion operation will execute.
If you have some other condition than other operations outstanding that means the queue is not "done", then you need to explain. If it's that network downloading is in progress, then maybe you need to wrap such downloading in an operation.
You could also use a custom subclass of NSOperation for the completion operation and override -isReady to use whatever criteria it wants to augment the superclass's notion of readiness. Of course, if you do that, you need to generate KVO change notifications when those other criteria change.

When will completionBlock be called for dependencies in NSOperation

From the docs:
The completion block you provide is executed when the value returned by the isFinished method changes to YES. Thus, this block is executed by the operation object after the operation’s primary task is finished or cancelled.
I'm using RestKit/AFNetworking, if that matters.
I have multiple dependencies in my NSOperation in a OperationQueue. I use the completion block to set some variables (appending the results to an array) that my child requires.
(task1,...,taskN) -> taskA
taskA addDependency: task1-taskN
Will taskA receive incomplete data since the child can execute before the completion block is fired?
Reference
Do NSOperations and their completionBlocks run concurrently?
I did a simple test by adding a sleep in my completion block and I had a different result. The completion block runs in the main thread. While all the completion block are sleeping, the child task ran.
As I discuss below under "a few observations", you have no assurances that this final dependent operation will not start before your other sundry AFNetworking completion blocks have finished. It strikes me that if this final operation really needs to wait for these completion blocks to finish, then you have a couple of alternatives:
Use semaphores within each of the n the completion blocks to signal when they're done and have the completion operation wait for n signals; or
Don't queue this final operation up front, but rather have your completion blocks for the individual uploads keep track of how many pending uploads are still incomplete, and when it falls to zero, then initiate the final "post" operation.
As you pointed out in your comments, you could wrap your invocation of the AFNetworking operation and its completion handler in your own operation, at which point you can then use the standard addDependency mechanism.
You could abandon the addDependency approach (which adds an observer on the isFinished key of the operation upon which this operation is dependent, and once all those dependencies are resolved, performs the isReady KVN; the problem being that this can theoretically happen before your completion block is done) and replace it with your own isReady logic. For example, imagine you had a post operation which you could add your own key dependencies and remove them manually in your completion block, rather than having them removed automatically upon isFinished. Thus, you custom operation
#interface PostOperation ()
#property (nonatomic, getter = isReady) BOOL ready;
#property (nonatomic, strong) NSMutableArray *keys;
#end
#implementation PostOperation
#synthesize ready = _ready;
- (void)addKeyDependency:(id)key {
if (!self.keys)
self.keys = [NSMutableArray arrayWithObject:key];
else
[self.keys addObject:key];
self.ready = NO;
}
- (void)removeKeyDependency:(id)key {
[self.keys removeObject:key];
if ([self.keys count] == 0)
self.ready = YES;
}
- (void)setReady:(BOOL)ready {
if (ready != _ready) {
[self willChangeValueForKey:#"isReady"];
_ready = ready;
[self didChangeValueForKey:#"isReady"];
}
}
- (void)addDependency:(NSOperation *)operation{
NSAssert(FALSE, #"You should not use addDependency with this custom operation");
}
Then, your app code could do something like, using addKeyDependency rather than addDependency, and explicitly either removeKeyDependency or cancel in the completion blocks:
PostOperation *postOperation = [[PostOperation alloc] init];
for (NSInteger i = 0; i < numberOfImages; i++) {
NSURL *url = ...
NSURLRequest *request = [NSURLRequest requestWithURL:url];
NSString *key = [url absoluteString]; // or you could use whatever unique value you want
AFHTTPRequestOperation *operation = [[AFHTTPRequestOperation alloc] initWithRequest:request];
[operation setCompletionBlockWithSuccess:^(AFHTTPRequestOperation *operation, id responseObject) {
// update your model or do whatever
// now inform the post operation that this operation is done
[postOperation removeKeyDependency:key];
} failure:^(AFHTTPRequestOperation *operation, NSError *error) {
// handle the error any way you want
// perhaps you want to cancel the postOperation; you'd either cancel it or remove the dependency
[postOperation cancel];
}];
[postOperation addKeyDependency:key];
[queue addOperation:operation];
}
[queue addOperation:postOperation];
This is using AFHTTPRequestOperation, and you'd obviously replace all of this logic with the appropriate AFNetworking operation for your upload, but hopefully it illustrates the idea.
Original answer:
A few observations:
As I think you concluded, when your operation completes, it (a) initiates its completion block; (b) makes the queue available for other operations (either operations that had not yet started because of maxConcurrentOperationCount, or because of dependencies between the operations). I do not believe that you have any assurances that the completion block will be done before that next operation commences.
Empirically, it looks like the dependent operation does not actually trigger until after the completion blocks are done, but (a) I don't see that documented anywhere and (b) this is moot because if you're using AFNetworking's own setCompletionBlockWithSuccess, it ends up dispatching the block asynchronously to the main queue (or the defined successCallbackQueue), thereby thwarting any (undocumented) assurances of synchrony.
Furthermore, you say that the completion block runs in the main thread. If you're talking about the built in NSOperation completion block, you have no such assurances. In fact, the setCompletionBlock documentation says:
The exact execution context for your completion block is not guaranteed but is typically a secondary thread. Therefore, you should not use this block to do any work that requires a very specific execution context. Instead, you should shunt that work to your application’s main thread or to the specific thread that is capable of doing it. For example, if you have a custom thread for coordinating the completion of the operation, you could use the completion block to ping that thread.
But if you're talking about one of AFNetworking's custom completion blocks, e.g. those that you might set with AFHTTPRequestOperation's setCompletionBlockWithSuccess, then, yes, it's true that those are generally dispatched back to the main queue. But AFNetworking does this using the standard completionBlock mechanism, so the above concerns still apply.
It matters if your NSOperation is a subclass of AFHTTPRequestOperation. AFHTTPRequestOperation uses the NSOperation's property completionBlock for its own purpose in method setCompletionBlockWithSuccess:failure. In that case, don't set the property completionBlock yourself!
It seems, AFHTTPRequestOperation's success and failure handler will run on the main thread.
Otherwise, the execution context of NSOperation's completion block is "undefined". That means, the completion block can execute on any thread/queue. In fact it executes on some private queue.
IMO, this is the preferred approach, unless the execution context shall be explicitly specified by the call-site. Executing completion handlers on threads or queues which instances are accessible (the main thread for example) can easily cause dead locks by an unwary developer.
Edit:
If you want to start a dependent operation after the completion block of the parent operation has been finished, you can solve that by making the completion block content itself a NSBlockOperation (a new parent) and add this operation as a dependency to the children operation and start it in a queue. You may realize, that this quickly becomes unwieldy, though.
Another approach would require an utility class or class library which is especially suited to solve asynchronous problems in a more concise and easy way. ReactiveCocoa would be capable to solve such (an easy) problem. However, it's unduly complex and it actually has a "learning curve" - and a steep one. I wouldn't recommend it, unless you agree to spend a few weeks in learning it and have a lot other asynchronous use cases and even much more complex ones.
A simpler approach would utilize "Promises" which are pretty common in JavaScript, Python, Scala and a few other languages.
Now, please read carefully, the (easy) solution is actually below:
"Promises" (sometimes called Futures or Deferred) represent the eventual result of an asynchronous task. Your fetch request is such asynchronous task. But instead specifying a completion handler, the asynchronous method/task returns a Promise:
-(Promise*) fetchThingsWithURL:(NSURL*)url;
You obtain the result - or the error - with registering a success handler block or a failure handler block like so:
Promise* thingsPromise = [self fetchThingsWithURL:url];
thingsPromise.then(successHandlerBlock, failureHandlerBlock);
or, the blocks inlined:
thingsPromise.then(^id(id things){
// do something with things
return <result of success handler>
}, ^id(NSError* error){
// Ohps, error occurred
return <result of failure handler>
});
And shorter:
[self fetchThingsWithURL:url]
.then(^id(id result){
return [self.parser parseAsync:result];
}, nil);
Here, parseAsync: is an asynchronous method which returns a Promise. (Yes, a Promise).
You might wonder how to get the result from the parser?
[self fetchThingsWithURL:url]
.then(^id(id result){
return [self.parser parseAsync:result];
}, nil)
.then(^id(id parserResult){
NSLog(#"Parser returned: %#", parserResult);
return nil; // result not used
}, nil);
This actually starts async task fetchThingsWithURL:. Then when finished successfully, it starts async task parseAsync:. Then when this finished successfully, it prints the result, otherwise it prints the error.
Invoking several asynchronous tasks sequentially, one after the other, is called "continuation" or "chaining".
Note that the whole statement above is asynchronous! That is, when you wrap the above statement into a method, and execute it, the method returns immediately.
You might wonder how to catch any errors, say fetchThingsWithURL: fails, or parseAsync::
[self fetchThingsWithURL:url]
.then(^id(id result){
return [self.parser parseAsync:result];
}, nil)
.then(^id(id parserResult){
NSLog(#"Parser returned: %#", parserResult);
return nil; // result not used
}, nil)
.then(/*succes handler ignored*/, ^id (NSError* error){
// catch any error
NSLog(#"ERROR: %#", error);
return nil; // result not used
});
Handlers execute after the corresponding task has been finished (of course). If the task succeeds, the success handler will be called (if any). If the tasks fails, the error handler will be called (if any).
Handlers may return a Promise (or any other object). For example, if an asynchronous task finished successfully, its success handler will be invoked which starts another asynchronous task, which returns the promise. And when this is finished, yet another one can be started, and so force. That's "continuation" ;)
You can return anything from a handler:
Promise* finalResult = [self fetchThingsWithURL:url]
.then(^id(id result){
return [self.parser parseAsync:result];
}, nil)
.then(^id(id parserResult){
return #"OK";
}, ^id(NSError* error){
return error;
});
Now, finalResult will either eventually become the value #"OK" or an NSError.
You can save the eventual results into an array:
array = #[
[self task1],
[self task2],
[self task3]
];
and then continue when all tasks have been finished successfully:
[Promise all:array].then(^id(results){
...
}, ^id (NSError* error){
...
});
Setting a promise's value will be called: "resolving". You can resolve a promise only ONCE.
You may wrap any asynchronous method with a completion handler or completion delegates into a method which returns a promise:
- (Promise*) fetchUserWithURL:(NSURL*)url
{
Promise* promise = [Promise new];
HTTPOperation* op = [[HTTPOperation alloc] initWithRequest:request
success:^(NSData* data){
[promise fulfillWithValue:data];
}
failure:^(NSError* error){
[promise rejectWithReason:error];
}];
[op start];
return promise;
}
Upon completion of the task, the promise can be "fulfilled" passing it the result value, or it can be "rejected" passing it the reason (error).
Depending on the actual implementation, a Promise can also be cancelled. Say, you hold a reference to a request operation:
self.fetchUserPromise = [self fetchUsersWithURL:url];
You can cancel the asynchronous task as follows:
- (void) viewWillDisappear:(BOOL)animate {
[super viewWillDisappear:animate];
[self.fetchUserPromise cancel];
self.fetchUserPromise = nil;
}
In order to cancel the associated async task, register a failure handler in the wrapper:
- (Promise*) fetchUserWithURL:(NSURL*)url
{
Promise* promise = [Promise new];
HTTPOperation* op = ...
[op start];
promise.then(nil, ^id(NSError* error){
if (promise.isCancelled) {
[op cancel];
}
return nil; // result unused
});
return promise;
}
Note: you can register success or failure handlers, when, where and as many as you want.
So, you can do a lot with promises - and even more than in this brief introduction. If you read up to here, you might get an idea how to solve your actual problem. It's right there - and it's a few lines of code.
I admit, that this short introduction into promises was quite rough and it's also quite new to Objective-C developers, and may sound uncommon.
You can read a lot about promises in the JS community. There are one or three implementations in Objective-C. The actual implementation won't exceed a few hundred lines of code. It happens, that I'm the author of one of it:
RXPromise.
Take it with a grain of salt, I'm probably totally biased, and apparently all others ever dealt with Promises, too. ;)

NSOperationQueue INSIDE an NSOperation

I created an NSOperation which goal is to download a few images (like 20) from 20 URLs.
So inside this NSOperation I create 20 AFImageRequestOperation add them in an NSOperationQueue and call -waitUntilAllOperationsAreFinished on the queue.
Problem is, it doesn't wait, it returns instantly. Here is the code
- (void)main {
NSArray *array = [I have the 20 links stored in this array];
self.queue = [[NSOperationQueue alloc]init];
self.queue.maxConcurrentOperationCount = 1;
for (int i = 0; i < array.count; i++) {
NSURL *url = [NSURL URLWithString:[array objectAtIndex:i]];
NSURLRequest *request = [NSURLRequest requestWithURL:url];
AFImageRequestOperation *op = [AFImageRequestOperation imageRequestOperationWithRequest:request imageProcessingBlock:^UIImage *(UIImage *image) {
return image;
} success:^(NSURLRequest *request, NSHTTPURLResponse *response, UIImage *image) {
// SUCCESS BLOCK (not relevant)
} failure:^(NSURLRequest *request, NSHTTPURLResponse *response, NSError *error) {
// FAILURE BLOCK (not relevant)
}];
[self.queue addOperation:op];
}
[self.queue waitUntilAllOperationsAreFinished]; // Here is the problem it doesn't wait
DDLogWarn(#"-- %# FINISHED --", self);
}
In the console, the DDLogWarn(#"-- %# FINISHED --", self); prints before every operations even started, so my guess was that the waitUntilAllOperationsAreFinished didn't do its job and didn't wait, but the 20 Operations are still running after that which may means that the main didn't return yet, so I don't know what to think anymore.
EDIT 1 : Actually I'm wondering, if my DDLog inside the success and failure blocks because waitUntilAllOperationsAreFinished is blocking the thread until all operations complete. That may explain why I don't see anything in happening and everything suddenly.
I have found it useful to create an NSOperation that contains other NSOperations for similar reasons to you, i.e. I have a lot of smaller tasks that make up a bigger task and I would like to treat the bigger task as a single unit and be informed when it has completed. I also need to serialise the running of the bigger tasks, so only one runs at a time, but when each big task runs it can perform multiple concurrent operations within itself.
It seemed to me, like you, that creating an NSOperation to manage the big task was a good way to go, plus I hadn't read anything in the documentation that says not to do this.
It looks like your code may be working after all so you could continue to use an NSOperation.
Depending on your circumstances blocking the thread may be reasonable. If blocking isn't reasonable but you wanted to continue using an NSOperation you would need to create a "Concurrent" NSOperation see Concurrency Programming Guide: Configuring Operations for Concurrent Execution
If you only allow one image download at a time, you could use #jackslashs suggestion to signal the end of the operation, or if you want to allow concurrent image downloads then you could use a single NSBlockOperation as the final operation and use -[NSOperation addDependency:] to make it dependant on all the other operations so it would run last.
When you get the signal that everything is finished and you can set the isFinished and isExecuting flags appropriately as described in the documentation to finalise your main NSOperation.
Admittedly this has some level of complexity, but you may find it useful because once that complexity is hidden inside an NSOperation the code outside may be simpler as was the case for me.
If you do decide to create a Concurrent NSOperation you may find the Apple sample code LinkedImageFetcher : QRunLoopOperation useful as a starting point.
This code doesn't need to be inside an NSOperation. Instead make a class, perhaps a singleton, that has an operation queue and make a method called
-(void)getImagesFromArray:(NSArray *)array
or something like that and your code above will work fine enqueueing onto that queue. You don't need to call waitUntilAllOperationsAreFinished. Thats a blocking call. If your queue has a max operation count of 1 you can just add another operation to it once you have added all the network operations and then when it executes you know all the others have finished. You could just add a simple block operation on to the end:
//add all the network operations in a loop
[self.operationQueue addOperation:[NSBlockOperation blockOperationWithBlock:^{
//this is the last operation in the queue. Therefore all other network operations have finished
}]];

The most common and right pattern for performing synchronous tasks, asynchronous tasks to create a composite operation

I am rather new to iOS and Objective-C.
I am developing a project aiming to make a composition of complex (or "composite") operations as easy as possible: https://github.com/stanislaw/SACompositeOperations
I have four kinds of operations there: two single operations (sync and async) and two complex operations (cascaded and "transactional"). Single operations are used as "atoms" for complex operations, so I want to make them as good as possible in compliance with Objective-C best practices.
What I am interested in is: what code should I choose for these single operations?
By calling the first of two single operations "sync" I mean: run something probably asynchronous with completion handler and lock the flow waiting until it is done. "Async" means truly asynchronous operation - just run operation block in asynchronous fashion.
Here is the single operations code I currently use:
Sync operation
- (void)run:(syncOperationBlock)block {
self.semaphore = dispatch_semaphore_create(0);
block(self);
while (dispatch_semaphore_wait(self.semaphore, DISPATCH_TIME_NOW))
[[NSRunLoop currentRunLoop] runMode:NSDefaultRunLoopMode
beforeDate:[NSDate dateWithTimeIntervalSinceNow:10]];
}
- (void)finish {
dispatch_semaphore_signal(self.semaphore);
}
#end
Async operation
- (void)run:(asyncOperationBlock)block {
dispatch_queue_t queue = dispatch_queue_create("async_operation.queue", 0);
dispatch_async(queue, ^{
block(self);
});
}
I will be thankful, if someone can suggest any solutions for these single sync- and async- operations: more generic, more common, just better for the cases I've described
Shortly: what is the best code for forced-synchronous operation, for async operation?
Thanks.
My project did evolve significantly since I've asked this question.
From the structural point of view, I can't say much interesting about asynchronous operations: in my project they are all based on dispatch_async and the most valuable functionality is implemented with intensive usage of blocks (mainly: yielding operations themselves to the blocks they execute, allows to control a flow of these operations).
The more interesting thing to note here is the code I currently use for synchronous (or "forced-synchronous") operations - though their usage should be avoided in real apps, they are still applicable in unit tests: one example is a need to straighten the curly asynchronous flow of asynchronous net request to test its result in place, in the context of unit test case.
Though I am posting this as an answer to my non-specific question I would still really like to see any authoritative answer about whether my SASyncOperation is good enough for forced-sync operations (in the sense I've described it in this question) or maybe it could be improved more.
SASyncOperation as it currently written
Does not block main thread if it is run on main thread.
Does not produce excessive CPU polling as simple approaches with while(!completed){}; do.
This is the most important part of the code:
#import "SASyncOperation.h"
#interface SASyncOperation () {
BOOL _isOnMainThread;
dispatch_semaphore_t _semaphore;
}
#end
#implementation SASyncOperation
// …
- (void)start {
_semaphore = dispatch_semaphore_create(0);
_operation(self);
if (_finished) return;
if ((_isOnMainThread = [NSThread isMainThread])) {
while (dispatch_semaphore_wait(_semaphore, DISPATCH_TIME_NOW)) {
CFRunLoopRunInMode(kCFRunLoopDefaultMode, [[NSDate distantFuture] timeIntervalSinceNow], NO);
}
} else {
dispatch_semaphore_wait(_semaphore, DISPATCH_TIME_FOREVER);
}
_finished = YES;
}
- (void)run:(^(void)(SASyncOperation *)block {
_operation = [block copy];
[self start];
}
- (void)finish {
_finished = YES;
dispatch_semaphore_signal(_semaphore);
if (_isOnMainThread) {
dispatch_async(dispatch_get_main_queue(), ^{
CFRunLoopStop(CFRunLoopGetMain());
});
}
}
// …
#end
This is how SASyncOperation should be used:
SASyncOperation *syncOperation = [[SASyncOperation alloc] init];
[syncOperation run:^(SASyncOperation *so) {
doSomeThingMultiQueuedPossiblyMultiblockedAndAsynchronous(^{
soOver = YES;
[so finish];
});
}]; // <- (*)
// (*) The flow will be stopped at this point waiting when 'so' operation will finish itself

Locked up waiting for #synchronized

I have this (rare) odd case where my objective-c iOS program is locking up. When I break into the debugger, there are two threads and both of them are stuck at a #synchronized().
Unless I am completely misunderstanding #synchronized, I didn't think that was possible and the whole point of the command.
I have a main thread and worker thread that both need access to a sqlite database, so I wrap the chunks of code that are accessing the db in #synchronized(myDatabase) blocks. Not much else happens in these blocks except the db access.
I'm also using the FMDatabase framework to access sqlite, I don't know if that matters.
The myDatabase is a global variable that contains the FMDatabase object. It is created once at the start of the program.
I know I'm late to the party with this, but I've found a strange combination of circumstances that #synchronized handles poorly and is probably responsible for your problem. I don't have a solution to it, besides to change the code to eliminate the cause once you know what it is.
I will be using this code below to demonstrate.
- (int)getNumberEight {
#synchronized(_lockObject) {
// Point A
return 8;
}
}
- (void)printEight {
#synchronized(_lockObject) {
// Point B
NSLog(#"%d", [self getNumberEight]);
}
}
- (void)printSomethingElse {
#synchronized(_lockObject) {
// Point C
NSLog(#"Something Else.");
}
}
Generally, #synchronized is a recursively-safe lock. As such, calling [self printEight] is ok and will not cause deadlocks. What I've found is an exception to that rule. The following series of events will cause deadlock and is extremely difficult to track down.
Thread 1 enters -printEight and acquires the lock.
Thread 2 enters -printSomethingElse and attempts to acquire the lock. The lock is held by Thread 1, so it is enqueued to wait until the lock is available and blocks.
Thread 1 enter -getNumberEight and attempts to acquire the lock. The lock is held already and someone else is in the queue to get it next, so Thread 1 blocks. Deadlock.
It appears that this functionality is an unintended consequence of the desire to bound starvation when using #synchronized. The lock is only recursively safe when no other thread is waiting for it.
The next time you hit deadlock in your code, examine the call stacks on each thread to see if either of the deadlocked threads already holds the lock. In the sample code above, by adding long sleeps at Point A, B, and C, the deadlock can be recreated with almost 100% consistency.
EDIT:
I'm no longer able to demonstrate the previous issue, but there is a related situation that still causes issues. It has to do with the dynamic behavior of dispatch_sync.
In this code, there are two attempts to acquire the lock recursively. The first calls from the main queue into a background queue. The second calls from the background queue into the main queue.
What causes the difference in behavior is the distinction between dispatch queues and threads. The first example calls onto a different queue, but never changes threads, so the recursive mutex is acquired. The second changes threads when it changes queues, so the recursive mutex cannot be acquired.
To emphasize, this functionality is by design, but it behavior may be unexpected to some that do not understand GCD as well as they could.
dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
NSObject *lock = [[NSObject alloc] init];
NSTimeInterval delay = 5;
NSLog(#"Example 1:");
dispatch_async(queue, ^{
NSLog(#"Starting %d seconds of runloop for example 1.", (int)delay);
[[NSRunLoop currentRunLoop] runUntilDate:[NSDate dateWithTimeIntervalSinceNow:delay]];
NSLog(#"Finished executing runloop for example 1.");
});
NSLog(#"Acquiring initial Lock.");
#synchronized(lock) {
NSLog(#"Acquiring recursive Lock.");
dispatch_sync(queue, ^{
NSLog(#"Deadlock?");
#synchronized(lock) {
NSLog(#"No Deadlock!");
}
});
}
NSLog(#"\n\nSleeping to clean up.\n\n");
sleep(delay);
NSLog(#"Example 2:");
dispatch_async(queue, ^{
NSLog(#"Acquiring initial Lock.");
#synchronized(lock) {
NSLog(#"Acquiring recursive Lock.");
dispatch_sync(dispatch_get_main_queue(), ^{
NSLog(#"Deadlock?");
#synchronized(lock) {
NSLog(#"Deadlock!");
}
});
}
});
NSLog(#"Starting %d seconds of runloop for example 2.", (int)delay);
[[NSRunLoop currentRunLoop] runUntilDate:[NSDate dateWithTimeIntervalSinceNow:delay]];
NSLog(#"Finished executing runloop for example 2.");
I stumbled into this recently, assuming that #synchronized(_dataLock) does what it's supposed to do, since it is such a fundamental thing after all.
I went on investigating the _dataLock object, in my design I have several Database objects that will do their locking independently so I was simply creating _dataLock = [[NSNumber numberWithInt:1] retain] for each instance of Database.
However the [NSNumber numberWithInt:1] returns the same object, as in same pointer!!!
Which means what I thought was a localized lock for only one instance of Database is not a global lock for all instances of Database.
Of course this was never the intended design and I am sure this was the cause of issues.
I will change the
_dataLock = [[NSNumber numberWithInt:1] retain]
with
_dataLock = [[NSUUID UUID] UUIDString] retain]