Is there a good tutorial on interpretation and problem-solving with Guard Malloc?
I'm getting message like "Failed to VM allocate 262144 bytes", and I have no idea what this means. Initially I thought it was the lack of RAM in the system, but maybe not so. If it is a problem I desperately need to learn how to interpret and catch the error.
Another question I have with Guard Malloc is whether it guards memory allocated in C codes of the project (it should right? considering the name) or only applying only to Objective-C? The reason I asked is that I just found out NSZombieEnabled only applies to Obj-C.
Help very much appreciated. I've been messing with likely memory errors for days. And I've not been able to compile Valgrind for iOS yet.
1) I've been chalking up allocation failures with guard malloc on to address space exhaustion -- every allocation takes up at least a page of address space that can't be reused. Uses of memory that is not currently allocated will crash in guard malloc, not cause allocation failures.
2) as the name suggests, guard malloc replaces the implementation of malloc(3), so C code that uses malloc will be checked.
Note that guard malloc is not a silver bullet. You still have to expose your app's bugs through testing; guard malloc just causes crashes to happen earlier and more reliably.
You might also want to read "man libgmalloc".
I was seeing this running on the ios simulator with Guard Malloc set. Choosing the 64-bit device for the simulator stopped the error coming up.
"Failed to VM allocate" is lack of available RAM, as you suspected.
I can only reliably use Guard Malloc when I close every other program on my mac, and even then it sometimes fails with greedy programs that use a lot of memory.
You'll need to:
Buy more RAM
Close all other running programs on your mac
Reduce the memory used by your program through profiling/optimization.
Related
I'm working on a Mac app. Initially monitoring Xcode's memory report while I ran my app showed showed the memory was just ramping up crazy. I used Instruments and profiled my app for allocations and leaks. Turned out there wasn't much leaked memory as you would expect due to strong reference cycles etc. However there was a lot of abandoned memory. By following the stack trace that lead to my code I have fixed by 70% using autorelease pools etc. Still the remaining 30% of abandoned memory seems to point system calls.
Now I have two questions based on that I have two questions
1) I want to fix the remaining 30%. How can I get rid of abandoned memory? I have already used Instruments and know exactly where those system calls are spawned but still dont know what to do to have that memory be cleaned up. (using ARC no manual retain/ release and autorelease doesn't seem to make a diff.)
2) After I know whatever my application was doing has completed and there is no need for any memory to be there (just like the application first started) I want to get rid of all memory that my app has used up. This I plan to use as a brute force approach to clean up all memory just like the system would if the user closes the app or turns off the system.
Basically if I know where my apps memory is in the file system I'll just programmatically call purge command on that or something similar. Because at this point I'm 100% sure nothing needs to be in memory for the app except for the first screen that you would expect the first time you launch the app.
I read this, this, this and this but they weren't helpful.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
How do malloc() and free() work?
I have encountered a weird problem and I'm really not sure why it doesn't work.
I have the following code in Xcode:
void *ptr = malloc(1024 * 1024 * 100);
memset(ptr, 0, 1024 * 1024 * 100);
free (ptr); //trace this line
ptr = malloc (1024 * 1024 * 100);
memset(ptr, 0, 1024 * 1024 * 100);
free (ptr); //trace this line
I put a breakpoint on each of the free() line, and when I traced the program, free didn't really free up the 100mb. However, if I change the number from 100 to 500 (allocate 500mb twice), memset 500mb, free() works fine. Why?
free can never fail(it does not have a return value) unless you call it with a improper address, which gives you undefined behavior.
You do not have to bother whether free actually frees memory or not you just have to ensure that you call free on the correct address after you are done with dynamic memory usage, rest the compiler should take care for you.
This is one of those things that you should just believe on your compiler to handle correctly.
Also, free just marks the memory being deallocated free(as name says) for reuse. It does not zero out or initialize the memory being deallocated.
When you pass a block of memory to free, that memory does not necessarily get returned to the operating system right away. In fact, based on the wording in the C standard, some argue that the memory can't be returned to the OS until the program exits.
The wording in question is (C99, ยง7.20.3.2/2): "The free function causes the space pointed to by ptr to be deallocated, that is, made available for further allocation." Their argument is that when/if a block of memory is allocated and then freed, it should be available for allocation again -- but if it's returned to the OS, some other process might take it, so it's no longer available for further allocation, as the standard requires. Personally, I don't find that argument completely convincing (I think "allocated by another process" is still allocation), but such is life.
Most libraries allocate large chunks of memory from the OS, and then sub-allocate pieces of those large chunks to the program. When memory is freed by the program, the put that block of memory on an "available" list for further allocation. Most also (at least at times) walk through the list of free blocks, merging free blocks that are adjacent addresses.
Many also follow some heuristics about what memory to keep after it's been freed. First, the keep an entire block as long as any of the memory in that block remains in use. If, however, all the memory in a block has been freed, they look at its size, and (often) at how much free memory they have available. If the amount available and/or size of the free block exceeds some threshold, they'll usually release it back to the OS.
Rather than having fixed thresholds, some try to tailor their behavior to the environment by (for example) basing their thresholds on percentages of available memory instead of fixed sizes. Without that, programs written (say) ten years ago when available memory was typically a lot smaller would often do quite a bit of "thrashing" -- repeatedly allocating and releasing the same (or similar) size blocks to/from the OS.
free() does not have to immediately unmap and return to the OS the pages backing up previously but no longer allocated buffers. It may keep them around so you can allocate memory quickly again. When the program finishes, the pages will be unmapped and returned to the OS.
As others already said, free() doesn't have to return memory to the OS. But I reject an idea that you should never care whether the memory is returned. There should be a good reason to care, but there are valid reasons.
If you do want to return memory to OS, use a platform-specific way which provides this guarantee:
mmap with MAP_ANONYMOUS on systems supporting it (there are many, but MAP_ANONYMOUS is not POSIX): mmap instead of malloc, munmap instead of free.
VirtualAlloc and VirtualFree on Windows.
[Shoul I add something here for other systems? Feel free to suggest.]
These ways of allocating memory work with big memory units (system page size or more).
I am playing around with Instruments. And I just recorded/profiled for memory leaks, I had very few memory leaks, but an overwhelming amount of allocations just keep going even when my app just opened. Here is a screenshot after using the app for less than 10 seconds.
And as I keep using the app it just keeps increasing and increasing.
The weirdest part is most of the allocations are coming from classes I don't know like:
Foundation
Altitude
lbdispatch.dylib
But it could be from the SBJson and the other classes I imported and added for JSon and XML.
But is this a lot of memory allocations? Is too much bad???
Yes and no, it depends on what you are doing, if you allocate for example a lot of strings, lets say you allocate 1000 strings these allocation perse are not bad, but it depends on your logic view of your application, if you really need all the strings at once and you need them to be allocated and alive through all the steps of your application, then you dont have anything to do, your application just needs alot of memory,
However on the other hand, you may find some other ways to logically structure your application, like for example you could only allocate each of the 1000 string once you need it.
A very abstract answer is, if your app requires a lot of memory and there is no way to use some techniques such as lazy loading, or caching, then you dont have any other solution
But if you can restructure your application to use lazy loading, caching, allocation pools it would bee better
Please note: that you could let iOS sdk help you, by implementing correctly the memory warning callbacks in your application, in such a way that whenever you receive a warning, you start releasing any resource that you dont currently need
Also, do you have Zombies on? Zombies default to not actually removing any allocations, so memory is never deallocated. Always test for memory leaks with Zombies off.
I'm trying to debug a mysterious app crash that happens after running the app for a few hours.
My hunch is that this may be memory related, as I've not done any memory optimization since starting to build my app.
I'm looking at my application with the "allocations" instrument and I see these numbers:
All allocations 4.70mb
Living 51072
Transitory 357280
Overall Bytes 100.23mb
Overall 408000
Which is the important number here? Does my app take 4.7 Mb of memory or 100 Mb? At which point should I be concerned about my app being killed for memory reasons? I want to avoid premature optimization :)
Since I'm using ARC and TabBarController, most of the controller related memory seems to be out of my hands, unless I find out how to create a tab and lazily init a controller when that tab is first touched.
Thank you!
i've been playing some time with different builds of my application and there seem strange things to happen:
my app has a 5mb idle footprint. when uploading a file memory in size of the file is reserved. after the upload the reserved memory should be freed. now there are differences in the builds (gc = garbage collector):
32bit i386 no-GC: all memory is freed instantly.
32bit i386 GC: almost all memory is freed instantly. the rest some time later.
64bit x86_64 no-GC: minimal memory is freed. like 10%
64bit x86_64 GC: no memory at all is freed. the memory stays reserved for hours. (activity mon)
i'm using LLVM with CLANG. i have been running today instruments all the time and was checking for leaks/zombies/etc. and everything seems to be clean. (the app is rather simple.)
is there an explanation for this behavior?
Update:
That's some weird stuff. I've boiled the problem to this:
I load a 20mb file into a NSData and release it. I'm doing this without any garbage collection enabled. The code is:
NSData *bla = [[NSData alloc] initWithContentsOfFile:#"/bigshit"];
[bla release];
When I build for i386 32bit the 20mb are allocated and released. When I switch the build to 64bit x86_64 the release does nothing. The 20mb stay allocated.
upper pic 32bit lower 64 http://kttns.org/zguxn
There is no difference between the two apps except that the upper one is built for 32bit and the lower one 64bit. There is no GC running. (With GC enabled the same problem appears.)
Update 2:
The Same behavior can be observed when I create a new cocoa app from scratch with only the upper code in applicationDidFinishLaunching:. In 64bit mode the memory is not released. i386 works as expected.
The same problem appears with NSString instead of NSData. It also appears when I boot the 64bit kernel. (Holding 64 at startup.)
OS is 10.6.0
First, use Instrument's Object Graph instrument to verify that the memory is no longer considered to be in use; does not have a retain count or a strong reference somewhere.
If it is no longer in use, then the memory is sticking around simply because you haven't hit the threshold at which the collector cares.
However, this statement:
64bit x86_64 no-GC: minimal memory is
freed. like 10%
Makes me wary. Specifically, if your code is designed to work in non-GC -- with retain/release -- then either (a) you have a memory leak and, if using CFRetain or some kind of global cache, that might impact GC or (b) you aren't using the right tools to figure out whether or not you have a memory leak.
So, how are you determining that you are leaking memory?
Update; you are using Activity Monitor to monitor the RSIZE/VSIZE of the process. This won't actually tell you anything useful beyond "is my process growing over time".
More likely than not (I haven't looked at the source), this code:
NSData *bla = [[NSData alloc] initWithContentsOfFile:#"/bigpoop"];
Will cause the 20MB file to be mmap()'d in to the process. There isn't a malloc() style allocation involved at all. Instead, the OS hands 20MB of contiguous address space to your process and maps the file's contents into it. As you read the contents of the NSData, it'll page fault in the file as you go.
When you release bla, the mapping is destroyed. But that doesn't mean that the VM subsystem is going to reduce your application's address space by 20MB.
So, you are burning up a bunch of address space, but not actual memory. Since your process is 64 bits, address space is pretty much an infinite resource and there is very little cost to using addresses, thus the reason why the OS is implemented this way.
I.e. there is no leak and your app is behaving correctly, GC or no.
This is a common misconception and, thus, star'd the question.
A garbage collector doesn't necessarily release memory immediately.
In the case of the Objective-C garbage collector, you can send Cocoa's garbage collector object a collectIfNeeded message to suggest that now may be a good time to do some collection, or collectExhaustively to order it to immediately start collecting any and all garbage (but even this is interruptible). See the docs.
I have a very similar problem in iPhoneOS 3.2 and I really don't think that the memory is being reclaimed -- I eventually trigger memory warnings. There is a small chance that I have overlooked a mistake of my own but I have been very thorough.
I use NSKeyedUnarchiver's unarchiveObjectWithFile: to load a custom object that contains a single large NSData and another much smaller object. The dealloc method in my custom object gets called, the NSData object is released, its retainCount == 1 just before. Physical memory does not decrement by any amount, let alone a fraction of the NSData size, and with repetition memory warnings are reliably generated: I have test until I actually received level 2 warnings. =(
Before release:
(gdb) p (int) [(NSData *) pastItsWelcomeData retainCount]
$1 = 1
After release:
(gdb) p (int) [(NSData *) pastItsWelcomeData retainCount]
Target does not respond to this message selector.