I have a data structure that I wanted to enumerate. I tried to implement my object's NSFastEnumerator as follows:
- (NSUInteger)countByEnumeratingWithState:(NSFastEnumerationState *)state
objects:(__unsafe_unretained id [])buffer
count:(NSUInteger)len {
NSUInteger c = 0;
while (c < len) {
id obj = [self objectAtIndex:state->state];
if (obj == nil) break;
buffer[c] = obj;
c++;
state->state++;
}
state->itemsPtr = buffer;
state->mutationsPtr = nil;
return c;
}
If I use objectAtIndex directly, my object works properly. I get a nil when the index doesn't exist. But when I then use the for loop:
for (Pin *pin in coll) { ... }
the code runs through the above function fine and fills in state with what appears to be valid values and returns the number of objects, then I get an EXC_BAD_ACCESS failure at the for statement itself.
What am I doing wrong in this implementation?
I just had a similar issues, and after looking more closely into Apple's FastEnumerationSample, this part (that I had overlooked) jumped at me:
// We are not tracking mutations, so we'll set state->mutationsPtr to point into one of our extra values,
// since these values are not otherwise used by the protocol.
// If your class was mutable, you may choose to use an internal variable that is updated when the class is mutated.
// state->mutationsPtr MUST NOT be NULL.
state->mutationsPtr = &state->extra[0];
The important part being: state->mutationsPtr MUST NOT be NULL. I just used the example line provided and it worked like a charm!
I'm assuming you're using ARC. The problem may be that the buffer is an array of __unsafe_unretained objects, so ARC might be over-releasing them. But what does your objectAtIndex: method look like? This shouldn't be a problem if you are returning objects that are guaranteed to be alive at least as long as your object itself.
Instead of:
id obj = [self objectAtIndex:state->state];
use
__unsafe_unretained id = [self objectAtIndex:state->state];
Related
I use the following as a getter for a property in one of my classes:
- (NSString *)version
{
if (_version == nil) {
_version = [[[NSBundle mainBundle] infoDictionary] objectForKey:#"CFBundleVersion"];
}
return _version;
}
This works well. However, when I try the same for an int property I obviously get an error since int are never nil. What is the best way around this?
- (int)numberOfDays
{
if (_numberOfDays == nil) {
// relatively memory intense calculation that works out numberOfDays:
_numberOfDays = X;
}
return _numberOfDays;
}
Firstly, using int is not recommended Objective-C if possible. If you need to use a primitive integer type, you should use NSInteger. The size of NSInteger is determined at compile time based on the architecture(s) being built for. int is a static size that will not widen for different architectures. It's OK to use it, just be aware.
Using NSInteger, you still face the same problem, it can't be nil. You should therefore make your property an NSNumber which you can init with the result of your computation with [NSNumber numberWithInteger:anInteger];. That way, you can keep you nil check on your property and only do the computation once to create your NSNumber.
Add another boolean instance variable _numberOfDaysCalculated.
A thread-safe version would be
- (int)numberOfDays
{
#synchronized(self) {
if (!_numberOfDaysCalculated) {
// relatively memory intense calculation that works out numberOfDays:
_numberOfDays = X;
_numberOfDaysCalculated = YES;
}
}
return _numberOfDays;
}
Alternatively, if there is some "invalid" value of the property, you can use that
as a "not yet computed" marker. For example, if the computed value of numberOfDays has to be non-negative, you could initialize _numberOfDays = -1 in the init method,
and then test for if (_numberOfDays == -1) in the lazy getter method.
Use GCD.
static dispatch_once_t tok;
dispatch_once(&tok, ^{ memory_intensive_computation(); });
No, don't use GCD, I missed the point. In an instance method, you want to tie information to each instance, so using a static dispatch token is not appropriate. Maybe you should just stick with the "boolean flag as instance variable" approach.
Alternatively, you can initialize the int to a value which is known to be out of its valid range (for example, I suppose that numberOfDays can never be negative) and use that as a condition for performing the calculation.
Use a NSNumber to store the int value.
- (int)numberOfDays
{
if (_numberOfDays == nil) {
// relatively memory intense calculation that works out numberOfDays:
_numberOfDays = #(X);
}
return [_numberOfDays intValue];
}
I would initialize the _numberOfDays in the -init with NSNotFound and test for that in the getter.
Question
In my ARC project I have a class that manages objects, called LazyMutableArray. Some of the objects are actually nil, but users of my collection will never know about this; therefore I made it a subclass of NSMutableArray, and it tries to do "the same thing". In particular, objects are retained when added.
Now let's take a look at a memory behavior of other methods. It turns out that the NSArray destruction methods are documented by Apple to be an exception to this rule, in that they release, not autoreleased object.
There is some debate as to whether the combination of addObject: + objectAtIndex: + array destruction is documented by Apple to be never autoreleasing or simply happens to be in the examples I tested and in the example Apple includes.
How can I create in my subclass a method with exact same memory semantics?
Last update
After some thought, I've decided implementation based on NSMutableArray is more appropriate in this case compared to NSPointerArray. The new class, I should note, has the same retain/autorelease pair as the previous implementation.
Thanks to Rob Napier I see that no modification of my objectAtIndex: method would change this behavior, which answers my original question about this method.
On a practical level, several people said that any method can tackle an extra retain/autorelease pair for no reason; it's not reasonable to expect otherwise and not reasonable to try to find out which methods do this and which do not. It's been therefore a great learning opportunity for me on several levels.
Code (based on NSMutableArray) is available at GitHub: implementation, header, test (that's -testLazyMutableMemorySemantics).
Thank you all for participating.
Why I try to subclass NSMutableArray:
Subclassing foundation objects, I agree, is not always an appropriate solution. In tho case I have objects (in fact, OData resources), most of which have subobjects. The most natural class for an array of subobjects is obviously NSArray. Using a different class doesn't seem to make sense to me.
But for an OData collection this "array of sub objects", while, being an NSArray, must have a different implementation. Specifically, for a collection of 1000 elements, servers are encouraged to return collection in batches of (say)20, instead of all at once. If there is another pattern appropriate in this case, I'm all ears.
Some more detail in how I found this
I unit test the hell out of this collection, and values can be put into array, read from the array, and so forth. So far, so good. However, I realized that returning the object increases its retain count.
How do I see it? Suppose I insert two objects into lazy array lazy, one held weakly, one held strongly (*see the code *). Then retain count of weakSingleton is, as expected, 1. But now I read element:
XCTAssertEqual(weakSingleton, lazy[0], #"Correct element storage"); // line B
And in the debugger I see the retain count go up to 2. Of course, -retainCount may give me wrong information, so let's try to destroy the reference in array by
lazy[0] = nil; // yep, does the right thing
XCTAssertNil(weakSingleton, #"Dropped by lazy array"); // line C <-- FAIL
indeed, we see that weakSingleton is not released.
By now you probably guess that it's not just a retain, it's an autoreleased retain — putting an #autorelease around line B releases the weakSingleton. The exact source of this pair is not obvious, but seems to come from NSPointerArray -addPointer: (and unfortunately not from ARC's [[object retain] autorelease]). However, I don't want to return an autoreleased object and make method semantics different from its superclass!
After all, the method I'm overriding, NSMutableArray -objectAtIndex:`, doesn't do that; the object it returns will dealloc immediately if an array is released, as noted in the Apple's example. That's what I want: modify the method around line A so that the object it returns does not have an extra retain/autorelease pair. I'm not sure the compiler should even let me do it :)
Note 1 I could turn off ARC for a single file, but this would be my first non-ARC Objective-C code. And in any case the behavior may not some from ARC.
Note 2 What the fuss? Well, in this case I could change my unit tests, but still, the fact is that by adding or deleting line B, I'm changing the result of unit test at line C.
In other words, the described behavior of my method [LazyMutableArray -objectAtIndex] is essentially that by reading an object at index 0, I'm actually changing the retain count of this object, which means I could encounter unexpected bugs.
Note 3 Of course, if nothing is to be done about this, I'll document this behavior and move on; perhaps, this indeed should be considered an implementation detail, not to be included into tests.
Relevant methods from implementation
#implementation LazyMutableArray {
NSPointerArray *_objects;
// Created lazily, only on -setCount:, insert/add object.
}
- (id)objectAtIndex:(NSUInteger)index {
#synchronized(self) {
if (index >= self.count) {
return nil;
}
__weak id object = [_objects pointerAtIndex:index];
if (object) {
return object;
}
}
// otherwise do something else to compute a return value
// but this branch is never called in this test
[self.delegate array:self missingObjectAtIndex:index];
#synchronized(self) {
if (index >= self.count) {
return nil;
}
__weak id object = [_objects pointerAtIndex:index];
if (object) {
return object;
}
}
#throw([NSException exceptionWithName:NSObjectNotAvailableException
reason:#"Delegate was not able to provide a non-nil element to a lazy array"
userInfo:nil]);
}
- (void)createObjects {
if (!_objects) {
_objects = [NSPointerArray strongObjectsPointerArray];
}
}
- (void)addObject:(id)anObject {
[self createObjects];
[_objects addPointer:(__bridge void*)anObject];
}
The complete test code:
// Insert two objects into lazy array, one held weakly, one held strongly.
NSMutableArray * lazy = [LazyMutableArray new];
id singleton = [NSMutableArray new];
[lazy addObject:singleton];
__weak id weakSingleton = singleton;
singleton = [NSMutableDictionary new];
[lazy addObject:singleton];
XCTAssertNotNil(weakSingleton, #"Held by lazy array");
XCTAssertTrue(lazy.count == 2, #"Cleaning and adding objects");
// #autoreleasepool {
XCTAssertEqual(weakSingleton, lazy[0], #"Correct element storage");
XCTAssertEqual(singleton, lazy[1], #"Correct element storage");
// }
lazy = nil;
XCTAssertNotNil(singleton, #"Not dropped by lazy array");
XCTAssertNil(weakSingleton, #"Dropped by lazy array");
The last line fails, but it succeeds if I change first line to lazy = [NSMutableArray new] or if I uncomment #autoreleasepool.
First, I would not make this subclass. This is exactly what NSPointerArray is for. Wrapping that into an NSArray obscures important details that this approach can break. For example, what is the correct behavior for [NSArray arrayWithArray:lazyMutableArray] if lazyMutableArray includes NULLs? Algorithms that assume that NSArray can never include NULL need to be wary of the fact that this one can. It's true that you can get similar issues treating a non-retaining CFArray as an NSArray; I speak from experience that this is exactly why this kind of subclass can be very dangerous (and why I stopped doing that years ago). Don't create a subclass that cannot be used in every case that its superclass can be used (LSP).
If you have a collection with new semantics, I would subclass it from NSObject, and have it conform to <NSFastEnumeration>. See how NSPointerArray is not a subclass of NSArray. This was not an accident. Faced with the same problem, note the direction Apple chose.
By now you probably guess that it's not just a retain, it's an autoreleased retain — putting an #autorelease around line B releases the weakSingleton. This seems to be because line A under ARC translates to [[object retain] autorelease]. However, I don't want to return an autoreleased object and make caller remember this!
The caller should never assume anything else. The caller is never free to assume that a method does not add balanced autoreleases. If a caller wants the autorelease pool to drain, that is their responsibility.
All that said, there is some benefit to avoiding an extra autorelease if it's not required, and it's an interesting learning opportunity.
I would start by reducing this code to the simplest form, without your subclass at all. Just explore how NSPointerArray works:
__weak id weakobject;
#autoreleasepool
{
NSPointerArray *parray = [NSPointerArray strongObjectsPointerArray];
{
id object = [NSObject new];
[parray addPointer:(__bridge void*)object];
weakobject = object;
}
parray = nil;
}
NSAssert(!weakobject, #"weakobject still exists");
My structure here (such as the extra nesting block) is designed to try to avoid accidentally creating strong references I don't mean to make.
In my experiments, this fails without the autoreleasepool and succeeds with it. That indicates that the extra retain/autorelease is being added around or by the call to addPointer:, not by ARC modifying your interface.
If you're not using this implementation for addObject:, I'd be interested in digging deeper. It is an interesting question, even if I don't believe you should be subclassing this way.
I'm going to elaborate on why I said this "looks a lot like a homework assignment." This will likely earn me many down votes, but it will also server as a good learning case for others who later find this question.
Subclassing NSMutableArray not a goal of a program. It is a means to achieve something else. If I were to venture a guess, I expect you were trying to create an array that lazily creates the object when they are accessed. There are better ways to do this without dealing with memory management yourself.
Here's an example of how I would implement a lazy loading array.
#interface LazyMutableArray : NSMutableArray
- (id)initWithCreator:(id(^)(int))creator;
#end
#interface LazyMutableArray ( )
#property (nonatomic, copy) id (^creator)(int);
#property (nonatomic, assign) NSUInteger highestSet;
#end
#implementation LazyMutableArray
- (id)initWithCreator:(id(^)(int))creator
{
self = [super init];
if (self) {
self.highestSet = NSNotFound;
self.creator = creator;
}
return self;
}
- (id)objectAtIndex:(NSUInteger)index
{
id obj = nil;
if ((index < self.highestSet) && (self.highestSet != NSNotFound)) {
obj = [super objectAtIndex:index];
if ([obj isKindOfClass:[NSNull class]]) {
obj = self.creator(index);
[super replaceObjectAtIndex:index withObject:obj];
}
} else {
if (self.highestSet == NSNotFound) {
self.highestSet = 0;
}
while (self.highestSet < index) {
[super add:[NSNull null]];
self.highestSet += 1;
}
obj = self.creator(index);
[super add:obj];
self.highestSet += 1;
}
return obj;
}
Fair Warning: I'm not compiling or syntax checking any of this code. It probably has a few bugs in it, but it should generally work. Additionally, this implementation is missing an implementation of add:, count, removeObjectAtIndex:, insertObject:atIndex:, and possibly replaceObjectAtIndex:withObject:. What I show here is just to get you started.
I am trying to reason out in my head how to implement a thread-safe caching mechanism for a reference counted value with an API roughly like this: (Note: I'm using Objective-C syntax, but the problem is not language-specific)
typedef id (^InvalidatingLazyGenerator)();
#interface InvalidatingLazyObject : NSObject
- (id)initWithGenerator: (InvalidatingLazyGenerator)generator;
#property (readonly) id value;
- (void)invalidate;
#end
When someone requests -value, if it has an existing cached value, it should return a -retain/-autoreleased version of that value. If it doesn't have a value, or if the value isn't valid, it should generate one using a generation block passed in at init time, then it should cache that value for any future reads until someone calls -invalidate.
Let's assume we don't care if the generator block is called multiple times (i.e. a second reader arrives while the first reader is in the generator block), as long as the objects it returns aren't leaked when that happens. A first pass, non-wait-free implementation of this might look something like:
- (id)value
{
id retVal = nil;
#synchronized(self)
{
retVal = [mValue retain];
}
if (!retVal)
{
retVal = [[mGenerator() retain] retain]; // Once for the ivar and once for the return value
id oldVal = nil;
#synchronized(self)
{
oldVal = mValue;
mValue = retVal;
}
[oldVal release];
}
return [retVal autorelease];
}
- (void)invalidate
{
id val = nil;
#synchronized(self)
{
val = mValue;
mValue = nil;
}
[val release];
}
Naturally, this results in crappy read performance because concurrent reads are serialized by the lock. A reader/writer lock improves things from this, but is still quite slow in the read path. The performance goal here is for cached reads to be as fast as possible (hopefully lock-free). It's OK for reads to be slow if we have to calculate a new value, and it's OK for -invalidate to be slow.
So... I am trying to figure out a way to make reads lock/wait-free. My first (flawed - see below) thought involved adding an invalidation counter whose value is atomically, monotonically incremented and read using memory barriers. It looked like this:
- (id)value
{
// I think we don't need a memory barrier before this first read, because
// a stale read of the count can only cause us to generate a value unnecessarily,
// but should never cause us to return a stale value.
const int64_t startCount = mWriteCount;
id retVal = [mValue retain];
OSMemoryBarrier(); // But we definitely want a "fresh" read here.
const int64_t endCount = mWriteCount;
if (retVal && startCount == endCount)
{
return [retVal autorelease];
}
// Now we're in the slow path
retVal = [mGenerator() retain]; // we assume generator has given us an autoreleased object
#synchronized(self)
{
mValue = retVal;
OSAtomicIncrement64Barrier(&mWriteCount);
}
return retVal;
}
- (void)invalidate
{
id value = nil;
#synchronized(self)
{
value = mValue;
mValue = nil;
OSAtomicIncrement64Barrier(&mWriteCount);
}
[value release];
}
But I can already see problems here. For instance, the [mValue retain] in the read path: we need the value to be retained, but it's possible that in the time between the read of mValue and the call to -retain another thread could -invalidate, causing the value to have been dealloc'ed by the time the retain call is made. So this approach won't work as is. There may be other problems too.
Has anyone already worked something like this out and care to share? Or have a pointer to something similar in the wild?
I ended up taking this approach to the problem: Read-Copy-Update. It seems to work quite well -- the read performance is 50x faster than the lock-based approach (but that's hardly surprising.)
I have a class method in a category to construct a Cocoa collection in some way that the built-in initializers don't allow. Due to the limited initializer functionality, I have to use the mutable version of the collection to actually build it. Here's an example for NS{Mutable}IndexSet:
#implementation NSIndexSet (WSSNonContiguous)
+ (instancetype)WSSIndexSetFromMask:(NSUInteger)mask
{
NSMutableIndexSet * set = [NSMutableIndexSet indexSet];
for( NSUInteger i = 0; i < (sizeof(NSUInteger) * 8); i++ ){
if( mask & (1l << i) ){
[set addIndex:i];
}
}
return set;
}
My return type is sometimes a lie here -- there's always a mutable collection being returned, whether the user is requesting an immutable version or not.
Is it still appropriate to use instancetype in cases like this, or should I go with id? If I do use instancetype, should I also be explicitly re-creating the collection:
// Ick?
return [[self alloc] initWithIndexSet:set];
to make sure an immutable copy is returned when the call is +[NSIndexSet WSSIndexSetFromMask:]?
Everything is okay:
NSIndexSet *set = [[NSIndexSet WSSIndexSetFromMask:0] addIndex:0];
No visible #interface for 'NSIndexSet' declares the selector 'addIndex:'
instancetype says to the sender, that you return a instance of the receivers type even it is a subtype. For the sender it is a NSIndexSet, because it is send to the class object of NSIndexSet.
An introspection that way, that someone looks to the return type and sees a subclass and takes any advantage out of this information, is malformed. The contract is build with the return type and this is in this case NSIndexSet.
Similar things have been asked before, but I was unable to solve my current problem with any of these answers.
Situation:
CustomType *Object;
BOOL (^doAverage)(CustomType *, int, int) = ^(CustomType *Trigger, int Total, int Pulse) {
//Calculate Average from Total and Pulse
Total /= Pulse;
[Trigger setValue:Total];
};
Object = [CustomType CreateObject]; //Autoreleased Object
[Object addCallback:^{ return doAverage(Object, 56, 32); }];
[Array addObject:Object]; //Adds to collection.
The issue at hand is a retain cycle as you probably already figured.
Object keeps a reference to the block in addCallback, and the block doAverage retains the reference to Object.
Using an instance variable is out of the question, because I want to reuse the variable Object for multiple objects. (Temporary variable).
Using a local variable results in the retain count.
And using __block CustomType *Object doesn't work either, because for whatever reason Trigger ends up as nil once the callback is actually called.
Any ideas?
I have a makeshift solution, but it seems rather...hacky.
Several things. First, I would like to see your addCallback: method. It's possible that you've implemented it incorrectly. For example, if you store a block for use later, you must copy it. If it's incorrect, all bets are off on the rest of the stuff.
And using __block CustomType *Object doesn't work either, because for
whatever reason Trigger ends up as nil once the callback is actually
called.
So if it's nil, then that means you assigned nil to Object somewhere.
As already stated, this answer is rather hacky and I'd be very happy if someone could point me in a better direction.
Apparently a primitive datatype in combination with a __block variable does the trick, though this is a bit complicated.
void *Ptr; //Variable for Block.
__block CustomType *Obj; //Function variable, mutable by block.
BOOL (^doAverage)(void *, int, int) = ^(void *Trigger, int Total, int Pulse) {
CustomType *T = (CustomType *)Trigger; //Conversion
//Calculate Average from Total and Pulse
Total /= Pulse;
[T setValue:Total];
};
//Convenience method.
CustomObject *(^Add)(CustomObject *) = ^(CustomObject *)NewObject {
[Array addObject:NewObject];
Obj = NewObject; //Assigns to the local block-Variable.
return Obj;
};
Ptr = Add([CustomObject CreateObject]); //Creates the Object, and adds it to the array.
[Obj addCallback:^{ return doAverage(Ptr, 56, 32); }];
Since Ptr is a primitive type, it will NOT be retained and does not have to be released. At the same time, it assumes the address of the Object in question and thereby doubles as it.
Once the object is released, so is the block with the pointer and everything is good.
Once the block is called, the pointer needs to be cast to the type in question, but that's only a minor problem.
Add is optional of course, but I don't like the syntax Ptr = Obj = [CustomObject CreateObject];
CustomType *Object;
BOOL (^doAverage)(CustomType *, int, int) = ^(CustomType *Trigger, int Total, int Pulse) {
//Calculate Average from Total and Pulse
Total /= Pulse;
[Trigger setValue:Total];
};
Object = [CustomType CreateObject]; //Autoreleased Object
[Object addCallback:^{ return doAverage(Object, 56, 32); }];
Taylor said -> "What I want however is a 'non-retained copy of the object' in question, which will be 'lost' once the corresponding object is deallocated."
This code doesn't seem to cause a retain-cyle unless you use copy on or inside of addCallback([^{}copy]);..
Where is it exactly the copy is used in your code? Inside of addCallback? if this is like:
addCallback(o) {
o = [o copy];
o();
then do a [o release]; when you done with a block object.. do not release it in dealloc()
}
if you have never used copy anywhere, nothing to be worried about.. It all happens in the stack that means no retain cyles at all unless it is not a global one!
In case there is a retail-cyle, do not use __block __weak etc instead do release whatever object it is in the end of the block.. and bear in mind that no copy no retain cycle..
If your deployment target is at least iOS 5 (or OS X 10.7), you can use "zeroing weak references":
CustomType *object = [CustomType makeObject];
__weak CustomType *weakObject = object;
[object addCallback:^{
CustomType *strongObject = weakObject;
if (strongObject)
return doAverage(weakObject, 56, 32);
else
return 0;
}];
(I have used a makeObject instead of CreateObject for the name of the "factory method", because methods with "create" in their name are expected to return a (+1) retain count object, not an autoreleased object.)
The __weak reference does not increment the retain count, therefore no retain cycle is created. If the object is destroyed because the last strong reference to it is gone, then weakSelf is set to nil. Inside the block a strong reference is created, which either points to the object, if it still exists, or is nil, if it does not exist anymore.
If I understand you code correctly, the callback will not be called if the object has been released. In that case a __unsafe_unretained reference is sufficient (which works also on iOS 4):
CustomType *object = [CustomType makeObject];
__unsafe_unretained CustomType *unsafeObject = object;
[object addCallback:^{
return doAverage(unsafeObject, 56, 32);
}];
Try declaring the Object as
__weak CustomType *Object