I have a property declared on an object in Objective-C that's called from multiple threads:
#property (atomic, strong) NSNumber *validLock;
In my setter, I do the following:
#synchronized(self.validLock) {
self.validLock = #YES;
}
In my getter, I do the following:
#synchronized(self.validLock) {
if (self.validLock.boolValue) {
...
}
}
Despite all these precautions, the thread sanitizer in Xcode still breaks on the #synchronized getter line with a comment "Data Race detected."
I even ditched the '#synchronized' idea and went all-in with NSLock to no avail.
This is becoming laughable, as this also gets tagged on [myLock lock] by the Xcode thread sanitizer:
NSLock *myLock = [[NSLock alloc] init];
...
[myLock lock];
self.validLock = #NO;
[myLock unlock];
What am I doing wrong?
This isn't particularly meaningful:
#synchronized(self.validLock) {
self.validLock = #YES;
}
I think you're expecting this to synchronize on "the validLock property of self" but that's not what the parameter means. You're synchronizing on the NSNumber that is currently pointed to by validLock (an NSNumber that you then replace). Since this is not the same object that you synchronize against elsewhere, this doesn't do much.
Similarly, if this code is all in the same context (which is what your example suggests), it doesn't do anything either:
NSLock *myLock = [[NSLock alloc] init];
...
[myLock lock];
self.validLock = #NO;
[myLock unlock];
Each thread would have its own lock.
First, you really want to avoid both NSLock and #synchronized. They've been superseded by GCD for many years now. See the Concurrency Programming Guide for full details.
If you really want to use #synchronized (realizing that it is pretty inflexible and quite slow), then typically you call it on self.
#synchronized(self) {
self.validLock = #YES;
}
But in any case, it has to be on the same actual object (not just the same name of an object).
If you're going to use NSLock, you need to make sure everyone is sharing the same actual lock, so for a single instance it needs to be a property. For multiple instances it needs to be a global or at least a class property.
Related
Question
In my ARC project I have a class that manages objects, called LazyMutableArray. Some of the objects are actually nil, but users of my collection will never know about this; therefore I made it a subclass of NSMutableArray, and it tries to do "the same thing". In particular, objects are retained when added.
Now let's take a look at a memory behavior of other methods. It turns out that the NSArray destruction methods are documented by Apple to be an exception to this rule, in that they release, not autoreleased object.
There is some debate as to whether the combination of addObject: + objectAtIndex: + array destruction is documented by Apple to be never autoreleasing or simply happens to be in the examples I tested and in the example Apple includes.
How can I create in my subclass a method with exact same memory semantics?
Last update
After some thought, I've decided implementation based on NSMutableArray is more appropriate in this case compared to NSPointerArray. The new class, I should note, has the same retain/autorelease pair as the previous implementation.
Thanks to Rob Napier I see that no modification of my objectAtIndex: method would change this behavior, which answers my original question about this method.
On a practical level, several people said that any method can tackle an extra retain/autorelease pair for no reason; it's not reasonable to expect otherwise and not reasonable to try to find out which methods do this and which do not. It's been therefore a great learning opportunity for me on several levels.
Code (based on NSMutableArray) is available at GitHub: implementation, header, test (that's -testLazyMutableMemorySemantics).
Thank you all for participating.
Why I try to subclass NSMutableArray:
Subclassing foundation objects, I agree, is not always an appropriate solution. In tho case I have objects (in fact, OData resources), most of which have subobjects. The most natural class for an array of subobjects is obviously NSArray. Using a different class doesn't seem to make sense to me.
But for an OData collection this "array of sub objects", while, being an NSArray, must have a different implementation. Specifically, for a collection of 1000 elements, servers are encouraged to return collection in batches of (say)20, instead of all at once. If there is another pattern appropriate in this case, I'm all ears.
Some more detail in how I found this
I unit test the hell out of this collection, and values can be put into array, read from the array, and so forth. So far, so good. However, I realized that returning the object increases its retain count.
How do I see it? Suppose I insert two objects into lazy array lazy, one held weakly, one held strongly (*see the code *). Then retain count of weakSingleton is, as expected, 1. But now I read element:
XCTAssertEqual(weakSingleton, lazy[0], #"Correct element storage"); // line B
And in the debugger I see the retain count go up to 2. Of course, -retainCount may give me wrong information, so let's try to destroy the reference in array by
lazy[0] = nil; // yep, does the right thing
XCTAssertNil(weakSingleton, #"Dropped by lazy array"); // line C <-- FAIL
indeed, we see that weakSingleton is not released.
By now you probably guess that it's not just a retain, it's an autoreleased retain — putting an #autorelease around line B releases the weakSingleton. The exact source of this pair is not obvious, but seems to come from NSPointerArray -addPointer: (and unfortunately not from ARC's [[object retain] autorelease]). However, I don't want to return an autoreleased object and make method semantics different from its superclass!
After all, the method I'm overriding, NSMutableArray -objectAtIndex:`, doesn't do that; the object it returns will dealloc immediately if an array is released, as noted in the Apple's example. That's what I want: modify the method around line A so that the object it returns does not have an extra retain/autorelease pair. I'm not sure the compiler should even let me do it :)
Note 1 I could turn off ARC for a single file, but this would be my first non-ARC Objective-C code. And in any case the behavior may not some from ARC.
Note 2 What the fuss? Well, in this case I could change my unit tests, but still, the fact is that by adding or deleting line B, I'm changing the result of unit test at line C.
In other words, the described behavior of my method [LazyMutableArray -objectAtIndex] is essentially that by reading an object at index 0, I'm actually changing the retain count of this object, which means I could encounter unexpected bugs.
Note 3 Of course, if nothing is to be done about this, I'll document this behavior and move on; perhaps, this indeed should be considered an implementation detail, not to be included into tests.
Relevant methods from implementation
#implementation LazyMutableArray {
NSPointerArray *_objects;
// Created lazily, only on -setCount:, insert/add object.
}
- (id)objectAtIndex:(NSUInteger)index {
#synchronized(self) {
if (index >= self.count) {
return nil;
}
__weak id object = [_objects pointerAtIndex:index];
if (object) {
return object;
}
}
// otherwise do something else to compute a return value
// but this branch is never called in this test
[self.delegate array:self missingObjectAtIndex:index];
#synchronized(self) {
if (index >= self.count) {
return nil;
}
__weak id object = [_objects pointerAtIndex:index];
if (object) {
return object;
}
}
#throw([NSException exceptionWithName:NSObjectNotAvailableException
reason:#"Delegate was not able to provide a non-nil element to a lazy array"
userInfo:nil]);
}
- (void)createObjects {
if (!_objects) {
_objects = [NSPointerArray strongObjectsPointerArray];
}
}
- (void)addObject:(id)anObject {
[self createObjects];
[_objects addPointer:(__bridge void*)anObject];
}
The complete test code:
// Insert two objects into lazy array, one held weakly, one held strongly.
NSMutableArray * lazy = [LazyMutableArray new];
id singleton = [NSMutableArray new];
[lazy addObject:singleton];
__weak id weakSingleton = singleton;
singleton = [NSMutableDictionary new];
[lazy addObject:singleton];
XCTAssertNotNil(weakSingleton, #"Held by lazy array");
XCTAssertTrue(lazy.count == 2, #"Cleaning and adding objects");
// #autoreleasepool {
XCTAssertEqual(weakSingleton, lazy[0], #"Correct element storage");
XCTAssertEqual(singleton, lazy[1], #"Correct element storage");
// }
lazy = nil;
XCTAssertNotNil(singleton, #"Not dropped by lazy array");
XCTAssertNil(weakSingleton, #"Dropped by lazy array");
The last line fails, but it succeeds if I change first line to lazy = [NSMutableArray new] or if I uncomment #autoreleasepool.
First, I would not make this subclass. This is exactly what NSPointerArray is for. Wrapping that into an NSArray obscures important details that this approach can break. For example, what is the correct behavior for [NSArray arrayWithArray:lazyMutableArray] if lazyMutableArray includes NULLs? Algorithms that assume that NSArray can never include NULL need to be wary of the fact that this one can. It's true that you can get similar issues treating a non-retaining CFArray as an NSArray; I speak from experience that this is exactly why this kind of subclass can be very dangerous (and why I stopped doing that years ago). Don't create a subclass that cannot be used in every case that its superclass can be used (LSP).
If you have a collection with new semantics, I would subclass it from NSObject, and have it conform to <NSFastEnumeration>. See how NSPointerArray is not a subclass of NSArray. This was not an accident. Faced with the same problem, note the direction Apple chose.
By now you probably guess that it's not just a retain, it's an autoreleased retain — putting an #autorelease around line B releases the weakSingleton. This seems to be because line A under ARC translates to [[object retain] autorelease]. However, I don't want to return an autoreleased object and make caller remember this!
The caller should never assume anything else. The caller is never free to assume that a method does not add balanced autoreleases. If a caller wants the autorelease pool to drain, that is their responsibility.
All that said, there is some benefit to avoiding an extra autorelease if it's not required, and it's an interesting learning opportunity.
I would start by reducing this code to the simplest form, without your subclass at all. Just explore how NSPointerArray works:
__weak id weakobject;
#autoreleasepool
{
NSPointerArray *parray = [NSPointerArray strongObjectsPointerArray];
{
id object = [NSObject new];
[parray addPointer:(__bridge void*)object];
weakobject = object;
}
parray = nil;
}
NSAssert(!weakobject, #"weakobject still exists");
My structure here (such as the extra nesting block) is designed to try to avoid accidentally creating strong references I don't mean to make.
In my experiments, this fails without the autoreleasepool and succeeds with it. That indicates that the extra retain/autorelease is being added around or by the call to addPointer:, not by ARC modifying your interface.
If you're not using this implementation for addObject:, I'd be interested in digging deeper. It is an interesting question, even if I don't believe you should be subclassing this way.
I'm going to elaborate on why I said this "looks a lot like a homework assignment." This will likely earn me many down votes, but it will also server as a good learning case for others who later find this question.
Subclassing NSMutableArray not a goal of a program. It is a means to achieve something else. If I were to venture a guess, I expect you were trying to create an array that lazily creates the object when they are accessed. There are better ways to do this without dealing with memory management yourself.
Here's an example of how I would implement a lazy loading array.
#interface LazyMutableArray : NSMutableArray
- (id)initWithCreator:(id(^)(int))creator;
#end
#interface LazyMutableArray ( )
#property (nonatomic, copy) id (^creator)(int);
#property (nonatomic, assign) NSUInteger highestSet;
#end
#implementation LazyMutableArray
- (id)initWithCreator:(id(^)(int))creator
{
self = [super init];
if (self) {
self.highestSet = NSNotFound;
self.creator = creator;
}
return self;
}
- (id)objectAtIndex:(NSUInteger)index
{
id obj = nil;
if ((index < self.highestSet) && (self.highestSet != NSNotFound)) {
obj = [super objectAtIndex:index];
if ([obj isKindOfClass:[NSNull class]]) {
obj = self.creator(index);
[super replaceObjectAtIndex:index withObject:obj];
}
} else {
if (self.highestSet == NSNotFound) {
self.highestSet = 0;
}
while (self.highestSet < index) {
[super add:[NSNull null]];
self.highestSet += 1;
}
obj = self.creator(index);
[super add:obj];
self.highestSet += 1;
}
return obj;
}
Fair Warning: I'm not compiling or syntax checking any of this code. It probably has a few bugs in it, but it should generally work. Additionally, this implementation is missing an implementation of add:, count, removeObjectAtIndex:, insertObject:atIndex:, and possibly replaceObjectAtIndex:withObject:. What I show here is just to get you started.
There is a similar question here, which doesn't explain exactly what I want: Objective C Blocks as Async-callbacks & BAD ACCESS
I have a view controller, which calls a service with an async callback. The callback is done using a block, which references variables on the view controller to populate them.
It looks like so:
- (void) loadData {
__block MyViewController *me = self;
[self.service executeWithCompletion:^(NSArray *result, NSError *error) {
if (!error) {
me.data = result;
}
}];
}
However, if I dealloc the view controller, 'me' is then badly accessed by the callback.
What is the simplest way of making 'me' NULL? If i put it as an iVar, it then brings back the circular reference... i think?
I think I'm missing something obvious....
Thanks
Are you targeting iOS 5.0 or later (or Mac OS X 10.7 or later)? If so, you can use ARC and a __weak variable (instead of a __block one). This will automatically zero out when the referenced object is deallocated. Your code would look like
- (void)loadData {
__weak MyViewController *me = self;
[self.service executeWithCompletion:^(NSArray *result, NSError *error) {
if (!error) {
MyViewController *strongMe = me; // load __weak var into strong
if (strongMe) {
strongMe.data = result;
}
}
}];
}
If you need support for an older OS then you need to find a different solution. One solution is to just go ahead and let the block retain self. If the service is guaranteed to execute the completion block (and then release it), this will only produce a temporary cycle that will break automatically when the completion block is run. Alternatively if you have some way to cancel the service (in a way that guarantees the block cannot be called after the cancellation), you can stick with the __block and just be sure to cancel the service in your -dealloc. There's other alternatives too but they're more complicated.
I did a combination of things above from the suggestions. Including nilling the blocks. Although, my objects are still not getting released immediately. i.e. I'd put a breakpoint on dealloc of MyViewController, and without the __block variable it would get called at a much later point in time (probably due to the async connection) and sometimes not at all.
The code is fairly complex - so I imagine there are other things going on for it to not work as suggested above.
What I have also done, is used Mike Ash's MAZeroingWeakRef, which i guess is the same as using __weak - which #KevinBallard suggested.
Below is how I've implemented it, and it appears to be working. Dealloc is called immediately on disposal of the view controller, which i want. And I can't get it to crash... and with the log comment that i've put in, I can already see that I'm dodging bullets.
- (void) loadData {
__block MAZeroingWeakRef *zeroWeakRef = [[MAZeroingWeakRef alloc] initWithTarget:self];
[zeroWeakRef setCleanupBlock: ^(id target) {
[zeroWeakRef autorelease];
}];
[self.service executeWithCompletion:^(NSArray *result, NSError *error) {
MyViewController *me = [zeroWeakRef target];
if (!me) {
DULog(#"dodged a bullet");
}
if (!error) {
me.data = result;
}
}];
}
Is there a real retain cycle problem that you're trying to avoid? Is there a reason that self should not simply be retained until -executeWithCompletion: completes? Is there any real chance that it won't complete?
So long as it really will eventually complete (even with failure) and so long as it releases the block after invoking it (perhaps by setting a property to nil), then the retain cycle will eventually be broken and all will be well.
In objective c, suppose I have an object Obj stored in a NSMutableArray, and the array's pointer to it is the only strong pointer to Obj in the entire program. Now suppose I call a method on Obj and I run this method in another thread. In this method, if Obj sets the pointer for itself equal to nil will it essentially delete itself? (Because there will be no more strong pointers left) I suspect the answer is no, but why? If this does work, is it bad coding practice (I assume its not good coding, but is it actually bad?)
It is highly unlikely that an object would be in a position to cause its own release/deallocation if your code is designed properly. So yes, the situation you describe is indicative of bad coding practice, and can in fact cause the program to crash. Here is an example:
#interface Widget : NSObject
#property (retain) NSMutableArray *array;
#end
#implementation Widget
#synthesize array;
- (id)init
{
self = [super init];
if(self) {
array = [[NSMutableArray alloc] init];
[array addObject:self];
}
return self;
}
- (void)dealloc
{
NSLog(#"Deallocating!");
[array release];
[super dealloc];
}
- (void)removeSelf
{
NSLog(#"%d", [array count]);
[array removeObject:self];
NSLog(#"%d", [array count]);
}
#end
and then this code is in another class:
Widget *myWidget = [[Widget alloc] init];
[myWidget release]; // WHOOPS!
[myWidget removeSelf];
The second call to NSLog in removeSelf will cause an EXC_BAD_ACCESS due to the fact that array has been deallocated at that point and can't have methods called on it.
There are at least a couple mistakes here. The one that ultimately causes the crash is the fact that whatever class is creating and using the myWidget object releases it before it is finished using it (to call removeSelf). Without this mistake, the code would run fine. However, MyWidget shouldn't have an instance variable that creates a strong reference to itself in the first place, as this creates a retain cycle. If someone tried to release myWidget without first calling removeSelf, nothing would be deallocated and you'd probably have a memory leak.
If your back-pointer is weak (which it should be since a class should never try to own it's owner, you will end up with a retain-cycle) and you remove the strong pointer from the array the object will be removed from the heap. No strong pointers = removed from memory.
You can always test this.
If you need a class to bring to a situation where its deleted, the best practice is to first retain/autorelease it and then make the situation happen. In this case the class won't be deleted in a middle of its method, but only afterwards.
I think we can say it might be bad coding practice, depending on how you do it. There are ways you could arrange to do it safely, or probably safely.
So let's assume we have a global:
NSMutableArray *GlobalStore;
One approach is to remove yourself as your final action:
- (void) someMethod
{
...
[GlobalStore removeObject:self];
}
As this is the final action there should be no future uses of self and all should be well, probably...
Other options include scheduling the removal with a time delay of 0 - which means it will fire next time around the run loop (only works of course if you have a run loop, which in a thread you may not). This should always be safe.
You can also have an object keep a reference to itself, which produces a cycle and so will keep it alive. When its ready to die it can nil out its own reference, if there are no other references and that is a final action (or a scheduled action by another object) then the object is dead.
I am coding Objective-C using the Cocos2D framework, and I have a singleton used for multiple purposes. One new purposes is to get and set character's "states" which are strings. I've recently made an NSDictionary for this purpose, but I have issues with the program freezing up when a method inside the singleton is called.
Here's the singleton code. I'm just leaving in the character state stuff:
.h
#interface ExGlobal : NSObject {
NSArray *charStates_keys;
NSArray *charStates_objects;
NSMutableDictionary *charStates;
}
#property(nonatomic, retain) NSMutableDictionary *charStates;
+(ExGlobal*)sharedSingleton;
- (NSString *)charState:(NSString *)charName;
- (void)set_charState:(NSString *)value forCharName:(NSString *)charName;
#end
.m
#import "ExGlobal.h"
#implementation ExGlobal
#synthesize charStates;
static ExGlobal* _sharedSingleton = nil;
+(ExGlobal*)sharedSingleton {
#synchronized([ExGlobal class]) {
if (!_sharedSingleton) {
[[self alloc] init];
}
return _sharedSingleton;
}
return nil;
}
+(id)alloc {
#synchronized([ExGlobal class]) {
NSAssert(_sharedSingleton == nil, #"Attempted to allocate a second instance of a singleton.");
_sharedSingleton = [super alloc];
return _sharedSingleton;
}
return nil;
}
-(id)init {
self = [super init];
if (self != nil) {
// initialize stuff here
exitName = #"ruinsSkyMid";
sceneChangeKind = #"reborn";
charStates = [[NSMutableDictionary alloc] init];
charStates_keys = [NSArray arrayWithObjects:#"Feathers", #"Hummus", nil];
charStates_objects = [NSArray arrayWithObjects:#"at wall", #"with Feathers", nil];
charStates = [NSMutableDictionary dictionaryWithObjects:charStates_objects forKeys:charStates_keys];
}
return self;
}
- (NSString *)charState:(NSString *)charName{
NSString *value = [charStates objectForKey:charName];
return value;
}
- (void)set_charState:(NSString *)charState forCharName:(NSString *)charName{
[charStates setObject:charState forKey:charName];
}
- (void)dealloc {
//I know it doesn't get called, but just in case
[charStates release];
[super dealloc];
}
#end
It's unclear to me what exactly the issue is when it freezes. When this happens, all I get in the console is:
Program received signal: “EXC_BAD_ACCESS”.
warning: Unable to read symbols for /Developer/Platforms/iPhoneOS.platform/DeviceSupport/4.3.5 (8L1)/Symbols/Developer/usr/lib/libXcodeDebuggerSupport.dylib (file not found).
Previous frame inner to this frame (gdb could not unwind past this frame)
Previous frame inner to this frame (gdb could not unwind past this frame)
Which I'm sure doesn't help finding the issue. I found if I redefine charStates_keys, charStates_objects and charStates inside both the charState and set_charState methods, it seems to work without freezing, except set_charState does not change the state.
It isn't freezing, it is crashing. Hence the EXC_BAD_ACCESS. It looks like your Xcode installation is borked, too, as the two messages following should not happen.
Note that methods should not have _s in the name; not a cause of the problem, but a comment on following convention.
You aren't retaining charStates and that is likely the cause of the crash.
Not an answer as such but I didn't have enough space in the comments field above to post this, but it might be useful.
As bbum already said, your lack of retaining charStates is likely the problem.
If you are confused about when to retain and not retain objects there's a really good book called "Learn Objective-C on the Mac" and I know it's a Mac book but most of it applies to iPhone too. On page 171 of chapter 9 (Memory Management) it talks about the "Memory Management Rules" and how if you are confused about when to retain or not then you don't understand the simple rules of Objective C memory management.
Essentially if you create an object using new, alloc or copy, then the retain count is automatically set to 1 so the object is retained and does not require you to retain it and will require a subsequent release to deallocate.
If you create the object any other way then the object will be an autoreleased object.
Obviously these rules only apply within the standard iOS libraries and can't necessarily be applied to third party libraries.
I recommend anyone who doesn't fully understand memory management in Objective C read this book. I found highly enlightening even for my iPhone work.
Hope that helps/.
Does #synchronized not use "lock" and "unlock" to achieve mutual exclusion? How does it do lock/unlock then?
The output of the following program is only "Hello World".
#interface MyLock: NSLock<NSLocking>
#end
#implementation MyLock
- (id)init {
return [super init];
}
- (void)lock {
NSLog(#"before lock");
[super lock];
NSLog(#"after lock");
}
- (void)unlock {
NSLog(#"before unlock");
[super unlock];
NSLog(#"after unlock");
}
#end
int main (int argc, const char * argv[]) {
NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];
MyLock *lock = [[MyLock new] autorelease];
#synchronized(lock) {
NSLog(#"Hello World");
}
[pool drain];
}
The Objective-C language level synchronization uses the mutex, just like NSLock does. Semantically there are some small technical differences, but it is basically correct to think of them as two separate interfaces implemented on top of a common (more primitive) entity.
In particular with a NSLock you have an explicit lock whereas with #synchronized you have an implicit lock associated with the object you are using to synchronize. The benefit of the language level locking is the compiler understands it so it can deal with scoping issues, but mechanically they behave basically the same.
You can think of #synchronized as a compiler rewrite:
- (NSString *)myString {
#synchronized(self) {
return [[myString retain] autorelease];
}
}
is transformed into:
- (NSString *)myString {
NSString *retval = nil;
pthread_mutex_t *self_mutex = LOOK_UP_MUTEX(self);
pthread_mutex_lock(self_mutex);
retval = [[myString retain] autorelease];
pthread_mutex_unlock(self_mutex);
return retval;
}
That is not exactly correct because the actual transform is more complex and uses recursive locks, but it should get the point across.
In Objective-C, a #synchronized block handles locking and unlocking (as well as possible exceptions) automatically for you. The runtime dynamically essentially generates an NSRecursiveLock that is associated with the object you're synchronizing on. This Apple documentation explains it in more detail. This is why you're not seeing the log messages from your NSLock subclass — the object you synchronize on can be anything, not just an NSLock.
Basically, #synchronized (...) is a convenience construct that streamlines your code. Like most simplifying abstractions, it has associated overhead (think of it as a hidden cost), and it's good to be aware of that, but raw performance is probably not the supreme goal when using such constructs anyway.
Actually
{
#synchronized(self) {
return [[myString retain] autorelease];
}
}
transforms directly into:
// needs #import <objc/objc-sync.h>
{
objc_sync_enter(self)
id retVal = [[myString retain] autorelease];
objc_sync_exit(self);
return retVal;
}
This API available since iOS 2.0 and imported using...
#import <objc/objc-sync.h>
Apple's implementation of #synchronized is open source and it can be found here. Mike ash wrote two really interesting post about this subject:
Locks, Thread Safety, and Swift
Let's Build #synchronized
In a nutshell it has a table that maps object pointers (using their memory addresses as keys) to pthread_mutex_t locks, which are locked and unlocked as needed.
It just associates a semaphore with every object, and uses that.