I am just getting started again with Mac development.
I am using CoreMIDI which is a C API which allows me to define a C callback function to be called from the MIDI server on a separate thread whenever a MIDI message arrives. The registration of this callback is done in Objective-C/C code triggered by an awakeFromNib call.
It seems to work fine, except when I get my first callback I get the following warning message on the console:
MidiList(6685,0x103ddb000) malloc: *** auto malloc[6685]: error: GC operation on unregistered thread. Thread registered implicitly. Break on auto_zone_thread_registration_error() to debug.
I read up on this online and it sounds like a harmless error. But the weird thing to me is I don't understand how a "GC operation" occurred? My project does have GC enabled, but I thought that was only for the Cocoa parts. And my callback doesn't use any Cocoa code, it is just a free function which uses some CoreMIDI and CoreFoundation functionality (including CFSTR if that matters). Why would a GC operation occur on that thread if i'm not using any Cocoa objects?
Because the garbage collector doesn't know you're not using Cocoa on this thread.
The garbage collector that Cocoa sports is a conservative garbage collector: it actually knows next to nothing about how your program is structured. All it does is scanning the stack of each thread, and objects on the heap, for bit patterns that look like pointers, and if there's an object at this would-be location, it keeps it alive.
There is, obviously, a possibility for false positives. You can have an integer that has a value looking like a pointer and the garbage collector will believe it's one.
Besides, you may allocate garbage-collected memory in a malloc-like way with NSAllocateCollectable. The GC must take them into account too, especially since the returned pointer could be passed into C code that's not even aware of garbage collection.
EDIT Aside from NSAllocateCollectable, Core Foundation objects (CF-prefixed types) can be made garbage-collected with the help of CFMakeCollectable. Once it's used, the garbage collector will take care of them.
If you are creating pthread, add this objc_registerThreadWithCollector(); to your pthread.
In case you cannot find a symbol or linking error, use following code,
#include <dlfcn.h>
void (*registerThreadWithCollector_fn)(void);
registerThreadWithCollector_fn = (void(*)(void)) dlsym(RTLD_NEXT, "objc_registerThreadWithCollector");
if (registerThreadWithCollector_fn) {
(*registerThreadWithCollector_fn)();
} else {
// do something else
}
Related
Is there a way to prevent an EXC_BAD_ACCESS from crashing an app, like with #try..#catch you can handle an exception gracefully.
Update:
The code crashes when it attempts to dereference an invalid pointer. This is a third party library and it interfaces with external hardware so I can't debug locally. I am trying to prevent it from crashing and output data to a debugging console on my app.
In ObjC, try/catch do not handle exceptions particularly gracefully. You will still leak memory and leave the system in an undefined state. With rare exception, the expectation is that you are simply catching so you can log some things before crashing. And in general, you should not use #catch anywhere but at the top level of your program for this purpose. There are some extraordinary situations where limited use of exceptions may be appropriate, but they are rare in ObjC. See the Exception Programming Guide for some more information. See especially the following from the ObjC ARC documentation:
The standard Cocoa convention is that exceptions signal programmer error and are not intended to be recovered from. Making code exceptions-safe by default would impose severe runtime and code size penalties on code that typically does not actually care about exceptions safety. Therefore, ARC-generated code leaks by default on exceptions, which is just fine if the process is going to be immediately terminated anyway. Programs which do care about recovering from exceptions should enable the option [-fobjc-arc-exceptions, which imposes speed and memory penalties on your program].
The same is true of EXC_BAD_ACCESS. You can catch it with a signal handler for the purpose of recording some information and then finishing your crash. For a good tool for doing this see PLCrashReporter. It is very difficult to write such a handler correctly, so I strongly recommend using an existing framework. You can easily get into deadlocks that drain the user's battery if you catch EXC_BAD_ACCESS incorrectly.
You get EXC_BAD_ACCESS often because you sent a message to a released object. Then you can examine the NSZombie. What is an NSZombie? You can see : this. It will catch the
EXC_BAD_ACCESS because of sent a message to the released object.
You can set NSZombie like this : Check the Enable Zombie Objects
And you can also get EXC_BAD_ACCESS because of the memory warnings level is too high , or your memory is too high , so the apple will shut your app down. This EXC_BAD_ACCESS is too hard to prevent . I think the only way is to manage your memory as low as you can , sometimes you can see the log where is receive memory warning , when the level is high, it may getEXC_BAD_ACCESS
You can rewrite your code to not have these errors. Try not to reference any null pointers and keep references to any object that you want to have access to.
STOP PRESS OK before you see the word retainCount in the following question, please skip to the EDIT at the bottom where I have stated that I have stopped using it.
My Cocoa App, which uses MRR, creates many global resources, which I am loading in main(), before NSApplicationMain() is called. As NSApplicationMain() doesn't return, I have hooked the clean-up of these resources using atexit(), like this:
atexit(cleanup);
if (![CocoaUtil initCocoaUtil] ||
![PreferenceController initPreferenceController] ||
![ResourceManager initResourceManager])
{
criticalAlertPanel(#"Failed to initialize application",
#"Failed to initialize application");
return 4;
}
retval = NSApplicationMain(argc, (const char **)argv);
However cleanup() is getting called before any of the views in my NSDocument subclass are dealloc'd (I have lack of log message to show this) and hence the reference counts of the objects in the global resources is sometimes > 1. I am being over-cautious and attempting to pre-empt memory leaks by using this method to release my global resources:
+ (void)fullRelease:(id)obj
format:(NSString *)format, ...
{
if (obj == nil)
return;
NSUInteger retainCount = [obj retainCount];
if (retainCount > 1)
{
va_list va;
va_start(va, format);
NSString *objDesc = [[NSString alloc] initWithFormat:format arguments:va];
logwrn(#"%# has a reference count of %lu", objDesc, retainCount);
[objDesc release];
}
while (retainCount > 0)
{
[obj release];
retainCount--;
}
}
My log shows the following:
12:15:04.954 INF -[AppController applicationDidFinishLaunching:] Application launched
12:15:06.702 INF -[AppController applicationShouldTerminate:] Application terminating
12:15:06.703 INF -[AppController applicationWillTerminate:] Application terminating
12:15:06.705 DBG cleanup Cleaning-up
12:15:06.705 INF +[ResourceManager finiResourceManager] Cleaning up
12:15:06.709 WRN +[CocoaUtil fullRelease:format:] _images[2] has a reference count of 2
12:15:06.709 WRN +[CocoaUtil fullRelease:format:] _images[3] has a reference count of 2
12:15:06.709 WRN +[CocoaUtil fullRelease:format:] _images[4] has a reference count of 2
12:15:06.710 WRN +[CocoaUtil fullRelease:format:] _images[5] has a reference count of 2
12:15:06.710 WRN +[CocoaUtil fullRelease:format:] _images[6] has a reference count of 2
12:15:06.710 WRN +[CocoaUtil fullRelease:format:] _images[7] has a reference count of 2
12:15:06.711 WRN +[CocoaUtil fullRelease:format:] _images[8] has a reference count of 2
12:15:06.711 WRN +[CocoaUtil fullRelease:format:] _images[9] has a reference count of 2
12:15:06.721 DBG +[PreferenceController finiPreferenceController] Cleaning up
12:15:06.721 DBG +[CocoaUtil finiCocoaUtil] Cleaning up
My question (finally!) is:
Is there a way to ensure I clean-up my global resource after all the NSDocument instances have been destroyed and stop getting these false negatives?
EDIT: I have unhooked the fullRelease call and just performed a normal release on my resources and Instruments did not detect any memory leaks, so things are OK, but I am curious as to why the NSDocument objects don't appear to be being released before atexit() is called.
Do not release something you do not own!
Every retain belongs to somebody else. Only send release to an object to balance your calls to new, alloc, copy, or retain (NARC.) This sort of behaviour will inevitably cause a crash in production code.
It looks like you want to make sure an object is deallocated rather than simply taken care of. If you own the object, release it once; the other references to it belong to other objects. It's possible you do have memory leaks in your code (we can't tell just from this code sample) but those can usually be found with the Static Analyzer, Instruments, and a bit of elbow grease.
More importantly: the operating system will free all your memory for you when your process exits. This is not part of the C standard but it is simply how OS X and iOS operate, and it is the expected behaviour on other platforms that support Objective-C. So you don't have to do anything special to clean up when your process exits, except perhaps writing files to disk or similar. In fact, many Cocoa applications don't bother to release anything owned by their app delegates, because it's faster to let the operating system dump the memory than it is to call -release on thousands of objects.
Do not call -retainCount!
It lies. Plain and simple. It includes temporary references used by Cocoa, most importantly, and you should never attempt to interfere with those. -retainCount is a poisoned symbol.
Some notes:
don't use retainCount; see the links on http://whentouseretaincount.com for lots of details
atexit() handlers are useless in high level programming. When called, the app is in a relatively undefined state. The frameworks will have torn down some stuff but, as you noted, there will be a ton of objects that will never be deallocated. atexit() may not be called at all, in some cases.
you cannot rely on app termination to do any kind of required state cleanup. The user may force quit your app. The system might, too, or it might be force-rebooted. At-termination behavior should be treated as an optimization; can you do some stuff that'll make the next launch faster.
during app termination, there is no need to deallocate anything. The system will reclaim all app resources upon termination, regardless of app state. In other words, objects left in memory at termination are not leaks.
in general, the "Leaks" detection tools can only be used to fix obvious problems. Leaks only detects memory leaks. It cannot detect memory accretion where the objects accreted are still connected to a global somehow. Accretion aren't technically leaks, but they can easily be a huge source of problems.
HeapShot analysis will detect both leaks and memory accretion.
Consider migrating your app to use ARC. One of the guiding patterns of ARC is that the compiler has complete knowledge of the lifespan of objects. Expressions that are ambiguous under MRR are disallowed under ARC (without proper markup, in some cases). As well, the ARC compiler and analyzer can do a much deeper analysis of your code, finding many issues that can be quite subtle.
Disable sudden termination, don't use globals, and just use the regular reference counting rules. In some cases, you will need to break strong circular references or manually clear out your objects' instance variables. Finally, it may be more helpful to pause/break before main returns, and run heap to see what's really alive (a leak) at this stage.
I am being over-cautious and attempting to pre-empt memory leaks by using this method to release my global resources.
You should not purge like this - you know better :)
However, locating and destroying all references/objects/allocations which you have control over is actually a very good practice, to ensure your programs work well, are reusable, and as good metrics for monitoring regressions as your apps change.
As all the others have said: Release only what you own. Your app may not crash for you now, but you're just getting lucky. It may crash on another Mac when you do this, or in two hours, or whatever.
If you have a scarce resource that some object is holding on to that really needs to be unregistered from (e.g. a network session to another server, where not correctly sending it a sign-off message will cause the server to wait for you, keeping some internal state for your return), you should build that into your objects instead of doing that when your objects are released.
E.g. there is a notification that Mac OS X sends before your application quits, NSApplicationWillTerminate. Have your object register for that, and when it gets it, have it send the server its goodbye (and remember that it already did that). That way, when your application quits, but e.g. the application delegate is still holding on to that object, you're still sure that the sign-off will happen.
But you should probably still also check if you've signed off already in -dealloc. One day you might add support for multiple servers, and then when you release one server, you want it to send a goodbye even though your app will not quit. But only do the sign off if you haven't done it already.
Same goes for when you have some data in memory that you want to write down before you quit (e.g. that's what NSUserDefaults does to make sure access to preferences is fast, but it still gets written to disk before you quit).
But cases like that are REALLY rare. Usually NSDocument will call you to write to disk when needed, and generally servers notice when the connection drops and clean up by themselves. So you really shouldn't release stuff that doesn't belong to you. You'll just cause crashes on quit for some of your users. It gives a really bad impression, and can cause your app to actually fail before it got the chance to save some data, and as such make things worse.
I am using Garbage collector in my Cocoa based application on Mac OS X. It has 100s of threads running and synchronization is done using Operation Queue.
After a long run, one of the object is garbaged and application will crash.
Checking to see if the object is non nil also fails as the the object is invalid and contains some garbage value. Calling a method on the object leads to crash.
Anyone please help me in debugging the issue.
Thank you......................
I am using Garbage collector in my
Cocoa based application on Mac OS X.
It has 100s of threads running and
synchronization is done using
Operation Queue.
More likely than not, the bug lies within the seemingly rather overly concurrent nature of your code. Running 100s of threads on a machine with "only" double digits worth of cores (if that) is unlikely to be very efficient and, of course, keeping everything properly in sync is going to be rather difficult.
The best place to start is to turn on Malloc stack logging and use malloc_history to find out what events occurred at the address that went south.
Also, there were fixes in 10.6.5 that impacted GC correctness.
If you can change the code of the garbage collected object, then override the finalize method like this, to get some information:
- (void) finalize
{
NSLog(#"Finalizing!\n%#", [[NSThread callStackSymbols] componentsJoinedByString:#"\n"]);
//if you put here a breakpoint, you can check the supposed references to this object
[super finalize];
}
Sorry for long description, however the questions aren't so easy...
My project written without GC. Recently I found a memory leak that I can't find. I did use new Xcode Analyzer without a result. I did read my code line by line and verified all alloc/release/copy/autorelease/mutableCopy/retain and pools... - still nothing.
Preamble: Standard Instruments and Omni Leak Checker don't work for me by some reason (Omin Tool rejects my app, Instruments.app (Leaks) eats too many memory and CPU so I have no chance to use it).
So I wanna write and use my own code to hook & track "all" alloc/allocWithZone:/dealloc messages statistics to write some simple own leaks checking library (the main goal is only to mark objects' class names with possible leaks).
The main hooking technique that I use:
Method originalAllocWithZone = class_getClassMethod([NSObject class],#selector(allocWithZone:));
if (originalAllocWithZone)
{
imp_azo = (t_impAZOriginal)method_getImplementation(originalAllocWithZone);
if (imp_azo)
{
Method hookedAllocWithZone = class_getClassMethod([NSObject class],#selector(hookedAllocWithZone:));
if (hookedAllocWithZone)
{
method_setImplementation(originalAllocWithZone,method_getImplementation(hookedAllocWithZone));
fprintf(stderr,"Leaks Hook: allocWithZone: ; Installed\n");
}
}
}
code like this for hook the alloc method, and dealloc as NSObject category method.
I save IMP for previous methods implementation then register & calculate all alloc/allocWithZone: calls as increment (+1) stat-array NSInteger values, and dealloc calls as decrement (-1).
As end point I call previous implementation and return value.
In concept all works just fine.
If it needs, I can even detect when class are part of class cluster (like NSString, NSPathStore2; NSDate, __NSCFDate)... via some normalize-function (but it doesn't matter for the issues described bellow).
However this technique has some issues:
Not all classes can be caught, for
example, [NSDate date] doesn't catch
in alloc/allocWithZone: at all, however, I can see alloc call in GDB
Since I'm trying to use auto singleton detection technique (based on retainCount readind) to auto exclude some objects from final statistics, NSLocale creation freezes on pre-init stage when starting of full Cocoa application (actually, even simple Objective-C command line utility with the Foundation framework included has some additional initialization before main()) - by GDB there is allocWithZone: calls one after other,....
Full Concept-Project draft sources uploaded here: http://unclemif.com/external/DILeak.zip (3.5 Kb)
Run make from Terminal.app to compile it, run ./concept to show it in action.
The 1st Question: Why I can't catch all object allocations by hooking alloc & allocWithZone: methods?
The 2nd Question: Why hooked allocWithZone: freezes in CFGetRetainCount (or [inst retainCount]) for some classes...
Holy re-inventing the wheel, batman!
You are making this way harder than it needs to be. There is absolutely no need whatsoever to roll your own object tracking tools (though it is an interesting mental exercise).
Because you are using GC, the tools for tracking allocations and identifying leaks are all very mature.
Under GC, a leak will take one of two forms; either there will be a strong reference to the object that should long ago been destroyed or the object has been CFRetain'd without a balancing CFRelease.
The collector is quite adept at figuring out why any given object is remaining beyond its welcome.
Thus, you need to find some set of objects that are sticking around too long. Any object will do. Once you have the address of said object, you can use the Object Graph instrument in Instruments to figure out why it is sticking around; figure out what is still referring to it or where it was retained.
Or, from gdb, use info gc-roots 0xaddr to find all of the various things that are rooting the object. If you turn on malloc history (see the malloc man page), you can get the allocation histories of the objects that are holding the reference.
Oh, without GC, huh...
You are still left with a plethora of tools and no need to re-invent the wheel.
The leaks command line tool will often give you some good clues. Turn on MallocStackLoggingNoCompact to be able to use malloc_history (another command line tool).
Or use the ObjectAlloc instrument.
In any case, you need to identify an object or two that is being leaked. With that, you can figure out what is hanging on to it. In non-GC, that is entirely a case of figuring out why it there is a retain not balanced by a release.
Even without the Leaks instrument, Instruments can still help you.
Start with the Leaks template, then delete the Leaks instrument from it (since you say it uses too much memory). ObjectAlloc alone will tell you all of your objects' allocations and deallocations, and (with an option turned on, which it is by default in the Leaks template) all of their retentions and releases as well.
You can set the ObjectAlloc instrument to only show you objects that still exist; if you bring the application to the point where no objects (or no objects of a certain class) should exist, and such objects do still exist, then you have a leak. You can then drill down to find the cause of the leak.
This video may help.
Start from the Xcode templates. Don't try to roll your own main() routine for a cocoa app until you know what you're doing.
Was wondering if anyone knows, or has pointers to good documentation that discusses, the low-level implementation details of Cocoa's 'performSelectorOnMainThread:' method.
My best guess, and one I think is probably pretty close, is that it uses mach ports or an abstraction on top of them to provide intra-thread communication, passing selector information along as part of the mach message.
Right? Wrong? Thanks!
Update 09:39AMPST
Thank you Evan DiBiase and Mecki for the answers, but to clarify: I understand what happens in the run loop, but what I'm looking for an answer to is; "where is the method getting queued? how is the selector information getting passed into the queue?" Looking for more than Apple's doc info: I've read 'em
Update 14:21PST
Chris Hanson brings up a good point in a comment: my objective here is not to learn the underlying mechanisms in order to take advantage of them in my own code. Rather, I'm just interested in a better conceptual understanding of the process of signaling another thread to execute code. As I said, my own research leads me to believe that it's takes advantage of mach messaging for IPC to pass selector information between threads, but I'm specifically looking for concrete information on what is happening, so I can be sure I'm understanding things correctly. Thanks!
Update 03/06/09
I've opened a bounty on this question because I'd really like to see it answered, but if you are trying to collect please make sure you read everything, including all currently posed answers, comments to both these answers and to my original question, and the update text I posted above. I'm look for the lowest-level detail of the mechanism used by performSelectorOnMainThread: and the like, and as I mentioned earlier, I suspect it has something to do with Mach ports but I'd really like to know for sure. The bounty will not be awarded unless I can confirm the answer given is correct. Thanks everyone!
Yes, it does use Mach ports. What happens is this:
A block of data encapsulating the perform info (the target object, the selector, the optional object argument to the selector, etc.) is enqueued in the thread's run loop info. This is done using #synchronized, which ultimately uses pthread_mutex_lock.
CFRunLoopSourceSignal is called to signal that the source is ready to fire.
CFRunLoopWakeUp is called to let the main thread's run loop know it's time to wake up. This is done using mach_msg.
From the Apple docs:
Version 1 sources are managed by the run loop and kernel. These sources use Mach ports to signal when the sources are ready to fire. A source is automatically signaled by the kernel when a message arrives on the source’s Mach port. The contents of the message are given to the source to process when the source is fired. The run loop sources for CFMachPort and CFMessagePort are currently implemented as version 1 sources.
I'm looking at a stack trace right now, and this is what it shows:
0 mach_msg
1 CFRunLoopWakeUp
2 -[NSThread _nq:]
3 -[NSObject(NSThreadPerformAdditions) performSelector:onThread:withObject:waitUntilDone:modes:]
4 -[NSObject(NSThreadPerformAdditions) performSelectorOnMainThread:withObject:waitUntilDone:]
Set a breakpoint on mach_msg and you'll be able to confirm it.
One More Edit:
To answer the question of the comment:
what IPC mechanism is being used to
pass info between threads? Shared
memory? Sockets? Mach messaging?
NSThread stores internally a reference to the main thread and via that reference you can get a reference to the NSRunloop of that thread. A NSRunloop internally is a linked list and by adding a NSTimer object to the runloop, a new linked list element is created and added to the list. So you could say it's shared memory, the linked list, that actually belongs to the main thread, is simply modified from within a different thread. There are mutexes/locks (possibly even NSLock objects) that will make sure editing the linked list is thread-safe.
Pseudo code:
// Main Thread
for (;;) {
lock(runloop->runloopLock);
task = NULL;
do {
task = getNextTask(runloop);
if (!task) {
// function below unlocks the lock and
// atomically sends thread to sleep.
// If thread is woken up again, it will
// get the lock again before continuing
// running. See "man pthread_cond_wait"
// as an example function that works
// this way
wait_for_notification(runloop->newTasks, runloop->runloopLock);
}
} while (!task);
unlock(runloop->runloopLock);
processTask(task);
}
// Other thread, perform selector on main thread
// selector is char *, containing the selector
// object is void *, reference to object
timer = createTimerInPast(selector, object);
runloop = getRunloopOfMainThread();
lock(runloop->runloopLock);
addTask(runloop, timer);
wake_all_sleeping(runloop->newTasks);
unlock(runloop->runloopLock);
Of course this is oversimplified, most details are hidden between functions here. E.g. getNextTask will only return a timer, if the timer should have fired already. If the fire date for every timer is still in the future and there is no other event to process (like a keyboard, mouse event from UI or a sent notification), it would return NULL.
I'm still not sure what the question is. A selector is nothing more than a C string containing the name of a method being called. Every method is a normal C function and there exists a string table, containing the method names as strings and function pointers. That are the very basics how Objective-C actually works.
As I wrote below, a NSTimer object is created that gets a pointer to the target object and a pointer to a C string containing the method name and when the timer fires, it finds the right C method to call by using the string table (hence it needs the string name of the method) of the target object (hence it needs a reference to it).
Not exactly the implementation, but pretty close to it:
Every thread in Cocoa has a NSRunLoop (it's always there, you never need to create on for a thread). PerformSelectorOnMainThread creates a NSTimer object like this, one that fires only once and where the time to fire is already located in the past (so it needs firing immediately), then gets the NSRunLoop of the main thread and adds the timer object there. As soon as the main thread goes idle, it searches for the next event in its Runloop to process (or goes to sleep if there is nothing to process and being woken up again as soon as an event is added) and performs it. Either the main thread is busy when you schedule the call, in which case it will process the timer event as soon as it has finished its current task or it is sleeping at the moment, in which case it will be woken up by adding the event and processes it immediately.
A good source to look up how Apple is most likely doing it (nobody can say for sure, as after all its closed source) is GNUStep. Since the GCC can handle Objective-C (it's not just an extension only Apple ships, even the standard GCC can handle it), however, having Obj-C without all the basic classes Apple ships is rather useless, the GNU community tried to re-implement the most common Obj-C classes you use on Mac and their implementation is OpenSource.
Here you can download a recent source package.
Unpack that and have a look at the implementation of NSThread, NSObject and NSTimer for details. I guess Apple is not doing it much different, I could probably prove it using gdb, but why would they do it much different than that approach? It's a clever approach that works very well :)
The documentation for NSObject's performSelectorOnMainThread:withObject:waitUntilDone: method says:
This method queues the message on the run loop of the main thread using the default run loop modes—that is, the modes associated with the NSRunLoopCommonModes constant. As part of its normal run loop processing, the main thread dequeues the message (assuming it is running in one of the default run loop modes) and invokes the desired method.
As Mecki said, a more general mechanism that could be used to implement -performSelectorOn… is NSTimer.
NSTimer is toll-free bridged to CFRunLoopTimer. An implementation of CFRunLoopTimer – although not necessarily the one actually used for normal processes in OS X – can be found in CFLite (open-source subset of CoreFoundation; package CF-476.14 in the Darwin 9.4 source code. (CF-476.15, corresponding to OS X 10.5.5, is not yet available.)