Why does ARC only sometimes retain a __block out pointer? - objective-c

Question

1) Why does this retain its __block var:
{
void (^blockWithOutPointer)(NSObject * __autoreleasing *) = ^(NSObject * __autoreleasing * outPointer) {
*outPointer = [NSObject new];
};
NSObject * __block blockVar1;
dispatch_after(dispatch_time(DISPATCH_TIME_NOW,
(int64_t)(1 * NSEC_PER_SEC)),
dispatch_get_main_queue(),
^{
NSLog(#"blockVar1: %#",
blockVar1);
// prints non-nil. WHY????
});
blockWithOutPointer(&blockVar1);
}
2) But this doesn't?
void (^blockWithOutPointerThatDispatchesLater)(NSObject * __autoreleasing *,
dispatch_block_t) = ^(NSObject * __autoreleasing * outPointer,
dispatch_block_t block) {
dispatch_after(dispatch_time(DISPATCH_TIME_NOW,
(int64_t)(1 * NSEC_PER_SEC)),
dispatch_get_main_queue(),
block);
*outPointer = [NSObject new];
};
{
NSObject * __block blockVar2;
blockWithOutPointerThatDispatchesLater(&blockVar2,
^{
NSLog(#"blockVar2: %#",
blockVar2);
});
// prints nil, which is expected.
}
3) If I instead use an __autoreleasing variable as my out pointer destination, and then assign that variable to my __block pointer, everything works fine.
{
NSObject * __autoreleasing autoreleasingVar;
NSObject * __block blockVar3;
blockWithOutPointerThatDispatchesLater(&autoreleasingVar,
^{
NSLog(#"blockVar3: %#",
blockVar3);
});
blockVar3 = autoreleasingVar;
// prints non-nil, which is expected.
}
I've read CRD's answer about ARC pointer-to-pointer issues, it makes sense that #2 would print nil because ARC assumes that blockVar2 is __autoreleasing, and doesn't retain its value. Thus, in #3, when we assign autoreleasingVar to blockVar3, ARC properly retains the value. However, there is no such assignment for #1. Why does #1 retain its value?
Even more amazingly, #1 is unaffected if I wrap the out-pointer assignment in an #autoreleasepool:
{
void (^blockWithOutPointer)(NSObject * __autoreleasing *) = ^(NSObject * __autoreleasing * outPointer) {
#autoreleasepool {
*outPointer = [NSObject new];
}
};
NSObject * __block blockVar1;
dispatch_after(dispatch_time(DISPATCH_TIME_NOW,
(int64_t)(1 * NSEC_PER_SEC)),
dispatch_get_main_queue(),
^{
NSLog(#"blockVar1: %#",
blockVar1);
// still prints non-nil. WHY???
});
blockWithOutPointer(&blockVar1);
}
Whereas #3 crashes, as expected since the #autoreleasepool released the out-pointer's object, and I guess ARC doesn't set __autoreleasing variables to nil.
void (^blockWithOutPointerThatDispatchesLater)(NSObject * __autoreleasing *,
dispatch_block_t) = ^(NSObject * __autoreleasing * outPointer,
dispatch_block_t block) {
dispatch_after(dispatch_time(DISPATCH_TIME_NOW,
(int64_t)(1 * NSEC_PER_SEC)),
dispatch_get_main_queue(),
block);
#autoreleasepool {
*outPointer = [NSObject new];
}
};
{
NSObject * __autoreleasing autoreleasingVar;
NSObject * __block blockVar3;
blockWithOutPointerThatDispatchesLater(&autoreleasingVar,
^{
NSLog(#"blockVar3: %#",
blockVar3);
// crashes on the NSLog!
});
blockVar3 = autoreleasingVar;
}
I filed a radar about this.

Answer 1

You have discovered a "feature" of the block implementation and (at least for cases 1 & 2, I haven't checked further) it is nothing to do with __autoreleasing per se.
First let's look at your case 1. You seem surprised this prints a non-nil value. It works exactly as expected:
blockWithOutPointer(&blockVar1); executes, assigning a value to blockVar1; then
the block ^{ NSLog(#"blockVar1: %#", blockVar1); } is executed by GCD and the stored by (1) is printed.
(Note: You can removed the __autoreleasing qualifiers and it will work the same as this is the inferred mode for pass-by-writeback parameters.)
Now your case 2. This is where you hit the "feature":
In Objective-C objects are heap allocated; and
Objective-C blocks are objects; so
Ergo blocks are heap allocated...
Except when they are not...
As an optimisation the block specification allows blocks and their captured __block variables to be stack allocated and only moved onto the heap when their lifetime needs to be longer than the stack frame are in.
As an optimisation it should (a) be essentially invisible to the programmer - apart from any perf benefit and (b) not change the semantics in anyway. However Apple decided to initially introduce it as a "programmer assisted optimisation" and have then slowly improved matters.
The behaviour of your case 2 is all down to when the block and __block variable get copied onto the heap. Let's look at the code:
NSObject * __block blockVar2;
This declares blockVar2 to have __block storage duration, which allows a block to alter the value of this locally declared variable. At this point the compiler does not know whether a block may access it, in which case blockVar2 will need to be on the heap, or whether it won't, in which case it may be on the stack.
The compiler decides it prefers the stack and allocates blockVar2 there.
blockWithOutPointerThatDispatchesLater(&blockVar2,
Now the compiler needs to pass the address of blockVar2 as the first argument, the variable is currently on the stack, and the compiler emits code to calculate its address. That address is on the stack.
^{
NSLog(#"blockVar2: %#",
blockVar2);
});
Now the compiler gets to the second argument. It sees the block, that the block accesses blockVar2, and that blockVar2 is qualified by __block so it must capture the variable itself and not the variables value.
The compiler decides the block should go on the heap. For this to work it needs to migrate blockVar2 onto the heap, so it does that along with its current value nil...
Oops!
The first argument is the address of the original blockVar2 on the stack, while the second argument is a block which in turn references the cloned blockVar2 on the heap.
When the code is executed blockWithOutPointerThatDispatchesLater() allocates an object and stores its address in the stack blockVar2; then GCD executes the delayed block which prints of the value of the heap blockVar2, which is nil.
A “Fix”
Just change your code to:
NSObject * __block blockVar2;
dispatch_block_t afterBlock = ^{
NSLog(#"blockVar2: %#", blockVar2);
};
blockWithOutPointerThatDispatchesLater(&blockVar2, afterBlock);
i.e. pre-calculate the second argument expression. Now the compiler sees the block before it sees &blockVar2, moves the block and blockVar2 to the heap, and the value generated for &blockVar2 is the address of the heap version of blockVar2.
Expanded Conclusion: Bug or Feature?
The original answer simply stated this is clearly a bug, not a feature, and suggest you file a bug report, noting it's been reported before but another report won't harm.
However that might be a little unfair, yes it is a bug, the question is what the bug actually is. Consider:
In (Objective-)C variables, allocated on the stack or statically, and allocated dynamic memory blocks do not move during their lifetime.
Compiler optimisations in general should not alter the meaning or correctness of a program.
Apple has implemented a compiler optimisation – storing blocks and captured __block variables on the stack – in a way which can move __block attributed variables during their lifetime and in doing so the meaning and correctness of a program can be altered.
The result is a simple re-ordering of program statements in a way which should not alter program meaning or correctness in fact does. This is bad!
Given the history of the optimisation, as implemented by Apple, which by-design relied on programmer assistance (though it has since been made more automatic) for its correctness this could simply be seen as another “feature” of the chosen implementation.
Recommendation
Never, ever, apply the address-of (&) operator to a variable with __block storage duration. If may work if you do, but it just as easily may not.
If you need to use a __block variable as a pass-by-writeback argument then copy it to a local temporary first, make the call, and finally copy it back again.
HTH

Answer 2

There's probably a misconception in 1):
If a variable will be imported into a block, which is marked with __block the compiler generates a helper struct containing various fields and the variable corresponding to the given source. That is, there's no pointer blockVar1, instead there's a whole struct instead.
If a block imports this variable and needs to be copied (for example, when asynchronously submitted) it also "moves" this helper struct onto the heap before "moving" the block itself onto the heap.
This statement
NSObject * __block blockVar1;
will initialise the helper struct which embeds the actual variable and initialises it to nil. The corresponding address of the variable points into the stack.
When the compiler parses this statement:
dispatch_after(dispatch_time(DISPATCH_TIME_NOW,
(int64_t)(1 * NSEC_PER_SEC)),
dispatch_get_main_queue(),
^{
NSLog(#"blockVar1: %#",
blockVar1);
// prints non-nil. WHY????
});
The compiler generates code for the compound statement, specifically it creates a struct representing the block which initially exists on the stack, too. Since this block requires to be moved to the heap before executing dispatch_async, it also moves the helper struct onto the heap in subsequently generated code.
Additionally, since the imported variable is a NSObject pointer, it also assigns function pointers (located in the helper struct) which "keep" and "dispose" the object, which will be called when the helper struct is "moved" to the heap, respectively when it is destroyed.
When you finally execute this statement
blockWithOutPointer(&blockVar1);
the address of the variable has already been changed: the variable is now located on the heap (since the helper struct has been "moved" to the heap") and exists as long as the block.
Now, to the point where you declare (and define) the following block:
void (^blockWithOutPointer)(NSObject * __autoreleasing *)
What causes gripes here is the __autoreleasing modifier, specifically Storage duration of __autoreleasing objects:
That is, __autoreleasing can only be applied to variables with automatic storage duration.
Edit:
This would assume that the given variable is on the stack. However, it is located on the heap. Thus, your program is illformed and the behaviour is undefined.
Regarding the __autoreleasing storage qualifier, the code sample 1) is correct (the _autoreleasing storage qualifier refers to a temporary that is created when passing the variable as a parameter, see comments below).
What I've said previously above should be correct, and what the user is experiencing is thus expected.
For further reference see: http://clang.llvm.org/docs/Block-ABI-Apple.html
However:
There's still a subtle potential issue with a "data race": the statement
blockWithOutPointer(&blockVar1);
will modify a pointer variable which is located on the heap, while executing on the current thread. Later, on the main thread the same variable will be read. Unless current thread equals main thread, this exhibits a classic data race.
While this is not the question and no a complete answer - it shows, how quickly such code becomes more complex than desired and I would recommend to strive for simpler, comprehensible code.
End Edit
I haven't analysed the other code samples
- but at the first glance it seems, using __autoreleasing leads also to an illformed program.