I have a large NSArray (wordDictionary) and I am creating smaller sub-arrays from it inside a for-loop. If the for-loop is set to 20,000 iterations everything works just fine. But if I increase the for-loop iterations to 200,000 iterations I get a malloc error... Why is that?
I noticed that if I move the sub-array assignment from inside the loop to outside the loop, it solves the problem!(?) Note that all sub-arrays are identical in both cases (this is just to demonstrate the issue). Here is the code with the assignment inside the loop (which causes the malloc error):
NSArray *subArray;
//subArray = [wordDictionary subarrayWithRange:(NSRange){50000,20000}];
for (int i=0;i<200000;i++)
{
subArray = [wordDictionary subarrayWithRange:(NSRange){50000,20000}];
testBool = [subArray containsObject:#"hello"];
}
NSLog(#"Done");
The code above works if the sub-array assignment is moved outside the loop (as shown by the commented line)
In the error message I get the following is included:
* mach_vm_map(size=8388608) failed (error code=3)
error: can't allocate region
Terminating app due to uncaught exception 'NSMallocException'
reason: '* NSAllocateObject(): attempt to allocate object of class '__NSArrayI' failed'
libc++abi.dylib: terminating with uncaught exception of type NSException
Any hints as to what could be causing this and how to fix it are welcome!! Thanks!!
My guess is that you run out of memory. Every time you call -subarrayWithRange you allocate some memory (depends on the implementation, but could be 20000 * something).
Modern Objective-C uses automatic reference counting instead of garbage collection. That means memory does not get freed instantly, even though you assign another object to subArray.
Try moving it inside a local autorelease pool:
for (int i=0;i<200000;i++)
{
#autoreleasepool {
// your code
}
}
Even if you follow the advice given and create an autoreleasepool, this is awfully inefficient. You extract a range of 20,000 objects each time, which means 20,000 objects will be retained and later released, for no good reason at all. It's so easily avoided:
for (int i=0;i<200000;i++)
{
NSRange range = NSMakeRange (50000,20000);
testBool = [wordDictionary indexOfObject:#"hello" inRange:range] != NSNotFound;
}
NSLog(#"Done");
You might also consider whether an NSSet or NSOrderedSet wouldn't be the right data structure.
Related
I have written a very simple test application to try to help with a larger project I'm working on.
Simply put, the test app loops a predetermined number of times and appends "1" to a string on each loop. When the loop hits a multiple of 1000, the string is reset and the process starts over again.
The code is below; but what I am finding is that the memory usage is much higher than I would expect. Each iteration adds about .5MB.
It appears that the newString is not reused, but is discarded and a new instance created, without recovering the memory it was using.
Ultimately, the software needs to count much much higher than 100000.
As a test, if I change the iteration to 10million, it takes over 5GB memory!
Has anybody got any suggestions? So far I have various ways of writing the clearing of the string and turning off ARC and releasing it/recreating manually, but none seem to be reclaiming the amount of memory I would expect.
Thank you for any help!
*ps. Yes this actual software is totally pointless, but as I say, its a test app that will be migrated into a useful piece of code once fixed.
int targetCount = 100000;
NSString * newString;
int main(int argc, const char * argv[]) {
#autoreleasepool {
process();
return 0;
}
}
void process() {
for (int i=0; i<targetCount; i++) {
calledFunction(i);
}
}
void calledFunction(count) {
if ((count % 1000) == 0) {
newString = nil;
newString = #"";
} else {
newString = [NSString stringWithFormat:#"%#1", newString];
}
}
Your calledFunction function creates an autoreleased NSString that won't be released until the current autorelease pool gets drained.
Your process function calls the calledFunction 100,000 times in a loop. During the duration of this loop, the current autorelease pool is not given a chance to drain. By the time the end of the process method is reached, all 100,000 instances of the NSString objects created in calledFunction are still in memory.
A common solution to avoid a build-up of autoreleased objects when using a large loop is to add an additional autorelease pool as shown below:
void process() {
for (int i=0; i<targetCount; i++) {
#autoreleasepool {
calledFunction(i);
}
}
}
Your problem stems from the auto release pool, a somewhat anachronistic feature in the days of ARC.
When an object is created with an alloc/init combination the resultant object is owned by the caller. The same is true for the standard new method, it is defined as alloc followed by init.
For each init... method a class by have a matching <type>... method, this is defined as alloc/init/autorelease and returns an unowned object to the caller. For example your code uses stringWithFormat: which is defined as alloc/initWithFormat/autorelease.
The unowned returned object is in the auto release pool and unless ownership is taken it will be release automatically the next time that pool is emptied, which for the main autorelease pool is once per iteration of the main event loop. In many programs iterations of the event loop are frequent enough to reclaim objects from the auto release pool quick enough that memory usage does not climb. However if objects are created and then discarded a lot with a single iteration of the event loop, as in your example of a large for loop, the auto release pool can end up with a lot of needed objects before it is emptied.
A Common Solution
A common solution to this problem is to use a local auto release pool, which is emptied as soon as it is exited. By judicious placement of such local auto release pools memory use can be minimised. A common place for them is wrapping the body of loops which generate a lot of garbage, in your code this would be:
void process()
{
for (int i=0; i<targetCount; i++)
{ #autoreleasepool
{
calledFunction(i);
}
}
}
here all auto released and discarded objects created by calledFunction() will be reclaimed on every iteration of the loop.
A disadvantage of this approach is determining the best placement of the #autoreleasepool constructs. Surely in these days of automatic referencing count (ARC) this process can be simplified? Of course...
Another Solution: Avoid The Auto Release Pool
The problem you face is objects ending up in the auto release pool for too long, a simple solution to this is to never put the objects in the pool in the first place.
Objective-C has a third object creation pattern new..., it is similar to the <type>... but without the autorelease. Originating from the days of manual memory management and heavy auto release pool use most classes only implement new - which is just alloc/init - and no other members of the family, but you can easily add them with a category. Here is newWithFormat:
#interface NSString (ARC)
+ (instancetype)newWithFormat:(NSString *)format, ... NS_FORMAT_FUNCTION(1,2);
#end
#implementation NSString (ARC)
+ (instancetype)newWithFormat:(NSString *)format, ...
{
va_list args;
va_start(args, format);
id result = [[self alloc] initWithFormat:format arguments:args];
va_end(args);
return result;
}
#end
(Due to the variable arguments this is more involved than most new family methods would be.)
Add the above to your application and then replace calls to stringWithFormat with newWithFormat and the returned strings will be owned by the caller, ARC will manage them, they won't fill up the auto release pool - they will never enter it, and you won't need to figure out where to place #autoreleasepool constructs. Win, win, win :-)
HTH
I tried something out in my code to see the effect on memory utilization. I wanted to find out if the line inside the loop was leaking. Running this loop took utilization up to 100MB and it didn't go back down again. Does this indicate a memory leak? If so why? (I'm using ARC)
for (i = 0; i < 10000000; i++)
{
self.accounts = [[NSArray alloc] initWithArray:[_dal accounts] copyItems:YES];
}
(accounts is an array of AccountSummary objects which implements NSCopying like this: name city state phone are all NSStrings, isLocal is BOOL)
- (id)copyWithZone:(NSZone *)zone {
AccountSummary *newAccount = [[AccountSummary allocWithZone:zone] init];
newAccount.name = [self.name copyWithZone:zone];
newAccount.city = [self.city copyWithZone:zone];
newAccount.state = [self.state copyWithZone:zone];
newAccount.phone = [self.phone copyWithZone:zone];
newAccount.isLocal = self.isLocal;
return newAccount;
}
There's no leak here that I can see. There will, however, be quite a bit of peak memory usage.
The exact behaviour of things that should logically release memory varies. Quite often, instead of releasing the memory it's autorelased. (With MRR, there used to be a method called autorelease.) When you autorelease something, it isn't really released but is instead scheduled for release later, when your code is finished because it's returned to the main event loop.
If part of this is being autoreleased — and my guess is that the property assignment is autoreleasing, because autorelease is "safer" than hard releasing — that memory won't be deallocated until your next autoreleasepool flush. Code on the main thread has an autoreleasepool set up by the OS itself, so each time you return to the main event loop everything that's been autoreleased gets flushed out. Here, that probably means that all 10,000,000 copies are kept in memory until you return to the main event loop. Darn right that'll crash a real device. :)
(That's assuming you're on the main thread; if you're not, you may not even have an autorelasepool set up, which means you probably will get a leak. But I think you get warnings to console in this case, so you'd already have a hint about which way to go.)
You can reduce this peak memory usage by using #autoreleasepool:
for (i = 0; i < 10000000; i++) #autoreleasepool {
self.accounts = [[NSArray alloc] initWithArray:[_dal accounts] copyItems:YES];
}
What will happen now is that the memory scheduled for release later in each iteration of the loop will actually be released each iteration of the loop. That should solve your immediate problem.
That said, i can't imagine why you're doing this except to check the behaviour. And if that's the case, this is unlikely your core problem.
Assuming your core problem is a leak, with ARC you're not really looking for leaks. You're looking for circular references. That means your answer likely lies elsewhere in your code. If you're sure it's self's accounts rather than dsl's accounts that are leaking, look for self being involved in a circular loop.
Also, keep in mind that calling copyWithZone: on a NSString will probably not copy the string. (There's no need to copy a read-only string, as the read-only string can't be changed. Both "copies" can be the same object.) So if you're leaking just strings, they could be associated with the original objects.
When creating lots of objects inside a loop, you should do that inside an auto release pool.
#autoreleasepool {
for (i = 0; i < 10000000; i++) {
self.accounts = [[NSArray alloc] initWithArray:[_dal accounts] copyItems:YES];
}
}
or, more likely in the real world...
for (i = 0; i < 10000000; i++) {
#autoreleasepool {
self.accounts = [[NSArray alloc] initWithArray:[_dal accounts] copyItems:YES];
}
}
I am trying to implement a modelling class for a Physics project with finite difference methods for simulating a simple pendulum. I want to be able to make this class as generic as possible so I can do whatever I want with the values on each iteration of the method. For this reason I have given my methods callback blocks which can also be used to stop the method if we want to.
For example my Euler method loop looks like so:
for (NSInteger i = 0; i < n; i++) {
if (callBack) {
if(!callBack(NO, currentTheta, currentThetaDot, currentT, (CGFloat)i/n)) break;
}
currentTheta += self.dt*f_single_theta(currentThetaDot);
currentThetaDot += self.dt*f_single_thetaDot(currentTheta, currentThetaDot, gamma);
currentT += self.dt;
}
And in the callBack block I run the code
^BOOL (BOOL complete, double theta, double thetaDot, CGFloat timeElapsed, CGFloat percentComplete){
eulerT = [eulerT stringByAppendingFormat:#"%.8f\n",timeElapsed];
eulerTheta = [eulerTheta stringByAppendingFormat:#"%.8f\n",theta];
if ((currentThetaDot*currentThetaDot + cos(currentTheta)) > 0.5) {
return 0; // stops running if total E > 0.5
}
return 1;
}];
Where eulerT and eulerTheta are strings which I later save to a file. This callback method quickly results in a massive build up of memory, even for n of order 10,000 I end up with about 1Gb of RAM usage. As soon as I comment out calling the callBack block this drops right off. Is there anyway I can keep this nice functionality without the massive memory problems?
Many people who are new to Objective C do not realize the difference between [NSArray array] and [[NSArray alloc] init]. In the days before ARC, the difference was much more obvious now. Both create a new object, but the former allocates the object, assigns it to the current NSAutoreleasePool, and leaves it with a retain count of 0 while the latter allocates it and leaves it with a retain count of 1.
Objects that are assigned to an NSAutoreleasePool do not get deallocated immediately when the retain count reaches 0. Instead, they get deallocated when the OS gets time to. Generally this can be assumed to be when control returns to the current run loop, but it can also be when drain is called on the NSAutoreleasePool.
With ARC, the difference is less obvious, but still significant. Many, if not most, of the objects your allocate are assigned to an autorelease pool. This means that you don't get them back just because you're done using them. That leads to the memory usage spiking in tight loops, such as what you posted. The solution is to explicitly drain your autorelease pool, like this:
for (NSInteger i = 0; i < n; i++) {
if (callBack) {
#autoreleasepool {
if(!callBack(NO, currentTheta, currentThetaDot, currentT, (CGFloat)i/n))
break;
}
}
currentTheta += self.dt*f_single_theta(currentThetaDot);
currentThetaDot += self.dt*f_single_thetaDot(currentTheta, currentThetaDot, gamma);
currentT += self.dt;
}
You should wrap the inside of your loop in #autoreleasepool{} to clean up temporary objects.
I'm currently using GLKit to do some OpenGL drawing. I created a normal UIViewController and then added a GLKViewController subclass inside a container to do my drawing. While everything runs fine initially, if I let my program run for a period of time (perhaps 15-30 minutes), eventually it crashes and gives me the following error.
malloc: *** mmap(size=2097152) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
So I turned on the malloc_error_break breakpoint, and the stack trace points to the following code.
-(NSArray*)meshes:(NSArray *)meshes sortedFromFrontToBack:(BOOL)sortedFromFrontToBack
{
NSMutableArray *sortedMeshes = meshes.mutableCopy;
[sortedMeshes sortUsingComparator:^NSComparisonResult(id obj1, id obj2) {
DSNode *mesh1 = obj1;
DSNode *mesh2 = obj2;
GLKVector3 depth1 = isnan(mesh1.boundingSphere.radius) ? mesh1.transformationState.position : mesh1.boundingSphere.center;
GLKVector3 depth2 = isnan(mesh2.boundingSphere.radius) ? mesh2.transformationState.position : mesh2.boundingSphere.center;
GLKMatrix4 mesh1ToCameraSpace = [mesh1 nodeToOtherNodeTransform:self];
GLKMatrix4 mesh2ToCameraSpace = [mesh2 nodeToOtherNodeTransform:self];
GLKVector3 depth1InCameraSpace = GLKMatrix4MultiplyVector3WithTranslation(mesh1ToCameraSpace, depth1);
GLKVector3 depth2InCameraSpace = GLKMatrix4MultiplyVector3WithTranslation(mesh2ToCameraSpace, depth2);
NSNumber *n1 = [NSNumber numberWithFloat:depth1InCameraSpace.z];
NSNumber *n2 = [NSNumber numberWithFloat:depth2InCameraSpace.z]; /* Breakpoint triggered here */
if(sortedFromFrontToBack)
{
return [n2 compare:n1];
}
return [n1 compare:n2];
}];
return sortedMeshes;
}
As I commented, the [NSNumber numberWithFloat:] call throws the malloc error. This method gets called once each frame from my GLKViewController's drawInRect method. Essentially, I have a class which keeps track of my cameras and the meshes which are going to be drawn by OpenGL, and it sorts them in camera space from either front to back for opaque meshes or back to front for transparent before drawing them.
- (void)glkView:(GLKView *)view drawInRect:(CGRect)rect
{
glClearColor(self.clearColor.r, self.clearColor.g, self.clearColor.b, self.clearColor.a);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
DSDirector *director = [DSDirector sharedDirector];
for(DSMesh *mesh in director.opaqueMeshes)
{
[mesh draw];
}
/* The director class keeps track of my scene's cameras and meshes and calls the above method to return the scene's transparent meshes properly sorted */
for(DSMesh *mesh in director.transparentMeshes)
{
[mesh draw];
}
}
From what I've read, the autorelease pool should drain at the end of each run loop, so I wouldn't think that creating a bunch of autoreleased objects every frame is an issue, they should all get flushed each frame. I've profiled my program and checked for any leaks and can't find any, and I'm using ARC as well, which should minimize the risk. When I profile it, the live bytes total never budges, although the overall bytes rises quite quickly, and no leaks are found. In addition, didReceiveMemoryWarning never fires. I'm stumped.
Something smells wrong about the NSNumber creation being the cause of your malloc error. Testing with Instruments might help find the real culprit.
However, you're doing two things you don't need to, so there's some chance that eliminating them might help with your problem.
First, you don't need to wrap floats in an NSNumber to compare them. The basic comparison and mathematical operators work just fine, and don't require extra time or memory for object creation:
if (depth1InCameraSpace.z < depth2InCameraSpace.z)
return NSOrderedAscending;
else if (depth1InCameraSpace.z > depth2InCameraSpace.z)
return NSOrderedDescending;
else
return NSOrderedSame;
Second, the Tile-Based Deferred Rendering strategy implemented by the GPU hardware on iOS devices does its own hidden surface removal optimizations -- sorting opaque geometry front to back is redundant, so all it does is waste CPU time. (There's a decent explanation of this is in the OpenGL ES Hardware Platform Guide for iOS.) You should still sort translucent geometry back to front (and draw it after opaque geometry) for proper blending, though.
So, I think I've figured out what was going on. At some prior point I'd enabled NSZombies and then forgot about it, and so all of my objects that I expected to be deallocated were actually still hanging around. When profiling in Instruments, zombies must not be counted as live, which is why I couldn't figure out why I was seemingly running out of memory when the number of live bytes wasn't climbing. However, when I upgraded to XCode 5, the new memory gauge showed that I was quickly racking up memory, and then when I dropped into Instruments, Leaks was covered up by a warning that said that it would not work properly since NSZombies were enabled. So, I disabled zombies, and now my memory usage is staying constant, just like I expected it to.
I have the following code:
for (int i = 1; i <= [nmaSpread count];)
{
[nmaUserName addObjectsFromArray:[nmaSpread objectAtIndex:i]];
[nmaSpread removeObjectAtIndex:i];
i += 2;
}
I have declared all variables as global, nmaUserName and nmaSpread are both NSMutableArrays, and have been allocated in viewDidLoad.
I want to store all the odd objects from nmaSpread into nmaUsername and then delete the active object at nmaSpread.
However it keeps crashing with this error:
[NSMutableArray addObjectsFromArray:]: array argument is not an NSArray
2011-12-11 21:08:55.123 appName[15671:f803] * Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '* -[NSMutableArray addObjectsFromArray:]: array argument is not an NSArray'
nmaSpread itself is an NSMutableArray, but it looks like the objects it contains aren't. When you do [nmaSpread objectAtIndex:i], that returns an object from nmaSpread. This object isn't an array, so to add it, you'd just use addObject:, not addObjectsFromArray:.
There are a few problems with your code.
First off, you've miss-understood what -addObjectsFromArray: does. The actual method you want is just addObject:.
Second, it is dangerous to modify an array while looping through it. The official line from Apple is "this may work in some situations, but don't do it because it might stop working at any point in the future". You need to wait until after you have finished looping through the array, and then delete them. Your current code is definitely doing it wrong.
You could sit down with pen/paper and work out the math to keep it all intact, but it's easier and safer to just do this:
NSMutableIndexSet *indexesToRemove = [NSMutableIndexSet indexSet];
for (int i = 1; i <= [nmaSpread count];)
{
[nmaUserName addObject:[nmaSpread objectAtIndex:i]];
[indexesToRemove addIndex:i];
i += 2;
}
[mmaSpread removeObjectsAtIndexes:indexesToRemove];
NSMutableArray's addObjectsFromArray expects you to pass another array, but you are passing a single object (the one at index 'i')
You can try switching
[nmaUserName addObjectsFromArray:[nmaSpread objectAtIndex:i]];
to
[nmaUserName addObject:[nmaSpread objectAtIndex:i]];
and that will remove the error you are seeing, but then you are likely to run into another problem because you are removing objects from nmaSpread as you go and as a result the indexes for items later in the array get shifted. You should probably change your logic around to deal with that problem.