What is the best practice for mixing Objective-C ARC with longjmp?
I am using Lua as scripting language, and my platform exports custom library for scripts. Entry points do check arguments with luaL_checkinteger(L, 2) (among others), which, in turn, may call luaL_typerror(L, 2, ...), that is implemented in Lua with setjmp/longjmp. As far as I know, ARC simply auto-generates retain/release code, but what happens if it longjmps out of scope? Will this code leak on mistyped arguments?
static int
set_tag(lua_State *L)
{
NSControl *control = (__bridge NSControl *)lua_topointer(L, 1);
[control setTag:(NSInteger)luaL_checkinteger(L, 2)]; // may longjmp!
return 0;
}
In the snippet above, control will be temporarily retained by ARC, but with longjmps uncatchable nature, corresponding release call may never happen. On the other hand, all arguments may be checked before assigning to control variable.
static int
set_tag(lua_State *L)
{
NSInteger tag = luaL_checkinteger(L, 2); // may longjmp!
NSControl *control = (__bridge NSControl *)lua_topointer(L, 1);
[control setTag:tag];
return 0;
}
Does it resolve [potential] leak above? Are there better ways to do this?
UPDATE: longjmp only unwinds to Lua internals, and never crosses any system code, except for Lua source (which is aware), and my entry points (which I hope are aware).
I'm pretty sure that second snippet does right, but I need kind of formal proof.
LATE UPDATE:
LuaJIT implements dwarf2-compatible errors, so they are just like C++ exceptions. Pass -fobjc-arc-exceptions compiler flag to arc-enabled sources with Lua code and any retained object will be released on any lua_error. Nothing to worry about now! You are still not allowed to throw errors across Cocoa runtime, though.
I recall that original Lua may be compiled with exceptions too, but I'm not sure.
Doesn't really matter if ARC is in use or not; any setjmp/longjmp that jumps over any frame of code from the system frameworks will yield undefined behavior (for the same reason that exceptions cannot be used for recoverable error handling).
So, yes, that code might leak. Might because it depends on whether the compiler emits a retain/release in that block and where. Might also because whether the compiler emits retain/release will be impacted by the optimization level and, over time, the version of the compiler.
longjmp only unwinds to Lua internals, and never crosses any system
code, except for Lua source (which is aware), and my entry points
(which I hope are aware).
That is helpful. As long as you structure your entry points such that they never intermingle system scope with Lua jumpable scopes, you should be OK. I would recommend turning off ARC in the source files where you have to manage this (and, of course, put the ObjC->Lua interface into a nicely encapsulated bit of implementation so the rest of your code can be ARC clean).
Consider, though, that there is non-obvious risk:
for(id x in anArray) {
... lua call that causes longjmp ...
}
The above would cause lua to "jump over" system code. Same goes for enumerateWithBlock:, KVO, Notification handlers, etc...
You're going to have to think very very carefully about every potential stack trigger by a call from Lua into your code. If that call triggers any kind of automated behavior on the part of the system that could then call Lua API that could trigger a longjmp, all bets are off.
longjmp() may cause crashes or leaks in ARC. Arranging the code so longjmp() and ARC don't interfere is difficult.
If the longjmp() is only for a fatal error path and you expect to halt the process in response then you may be able to ignore the problem. This is what ARC does with C++/ObjC exceptions by default. ARC code is expected to leak when exceptions are thrown. There's a compiler option to enable the clean up code for exceptions, but that hurts performance.
If the longjmp() is not a process-killing error then your best option is to turn off ARC in any code that may be skipped by a longjmp() call.
Related
I am thinking about pros and cons of Try-Catch in Objective-C.
According to this article Dispelling NSException Myths in iOS: Can We Use #try…#catch, #finally?, try-catch isn't that bad, except for it leaks memory in ARC.
So how does try-catch cause memory leak?
First of all: Exceptions have different semantics in Objective-C. An exception means that something went completely wrong because of a programming mistake and the further execution of the application is not useful. Terminate it! To handle "expected errors" (like insufficient user input or not responding servers et al.) use Cocoa's error handling pattern. (The reason for this is that exceptions seems to be convenient in many situation, but are very hard to handle in other situations, i. e. while object construction. Read about exceptions in C++. It is painful.)
To your Q: ARC adds additional code to handle memory management. This code has to be executed to handle the memory management, esp. to release objects. If an exception occurs before this is done, the control flow never reaches the release statement. Memory leaks.
- (void)method
{
id reference = …;
// Some ARC code to retain the object, reference points to.
…
#throw …
…
// reference loses its extent, because of method termination
// Some ARC code to release the object, reference points to.
}
If you have an exception, the method is left immediately and the ARC code and the end of the method to release the object is never executed. This is the leak.
You can change this behavior by compiling the source with -fobjc-arc-exceptions option.
http://clang.llvm.org/docs/AutomaticReferenceCounting.html#exceptions
This will add code to make ARC exception-safe causing a runtime penalty. But there is little reason to do so in Cocoa development, as explained at the beginning of this answer.
Coming from the world of managed memory, wondering what would be the proper way to clean up objects when using ARC.
For example: if declaring an instance variable in C#, .NET will allow the GC to pick it up once it leaves scope (method/loop body, etc)
What's the proper way to clean-up in Objective-C? Just set the reference/pointer to nil or call dealloc or will ARC detect that no external references are pointing to the instance once execution leaves scope and do the job for you?
ARC means "Automatic Reference Counting" and is just a way to let the compiler add the calls to retain/release/autorelease for you. It's not the same as GC but in most cases, you can consider that objects lifetime is automatically managed for you, like in GC.
If you want more information, you should read LLVM document on ARC
Last note: never call dealloc yourself. dealloc is the object's finalizer which is called once the ObjC runtime determines that the object reference count has reached 0. This method is only meant to be overriden by subclasses. In ARC mode, you generally don't need to do that, except if your object references non-object ivars that need to be finalized once the object itself is finalized.
will ARC detect that no external references are pointing to the
instance once execution leaves scope and do the job for you
Basically, yes, that's exactly what ARC will do. You don't need to clean up objects when you're using ARC; in fact, you can't (it stops you from trying to perform manual memory management).
You might want to consult the relevant discussion in my book:
http://www.apeth.com/iOSBook/ch12.html#_memory_management
It explains what's really happening behind the scenes (how memory is actually managed) and then goes on to describe how ARC shields you from most of it.
Note that (as I explain in the URL referenced above) it mostly isn't done by anything like garbage collection: it's done by inserting invisible explicit memory management throughout your code.
Well, in the past, iOS programmers were responsible for telling the system when they were done using an object that they allocated by sending the object a release message. That was done in accordance with a memory management system known as manual reference counting. As of Xcode 4.2, programmers no longer have to worry about this and can rely on the system to take care of releasing memory as necessary. This is done through a mechanism known as Automatic Reference Counting, or ARC for short. ARC is enabled by default when you compile new applications using Xcode 4.2 or later.
You can also disable ARC, in your Xcode interface, go to your main project (not main.h) your actual Xcode project, and select it, you should see a window in Xcode that displays the settings for your project, there will be one that says 'Objective-C Automatic Reference Counting' and it will be set to 'Yes', deactivate it (to 'No') and you shouldn't worry about the ARC, if you come from the world of data management and memory as you said, but keep in mind that it would be easier to you to keep updated to the iOS new features system, that are easier to the programmer to program, it just makes our life easier.
And now, the 'proper way to clean-up in Xcode' with ARC is with 'alloc' and 'init'.
With ARC in Xcode you do not need to worry for 'cleaning' that's the job of Xcode now, you just need to:
1) Create a variable.
2) Allocate.
3) Initialize.
That's it.
An example here:
int main (int argc, char * argv[])
{
#autoreleasepool {
Variable *myVariable;
// Create an instance of a Variable and initialize it
myVariable = [Variable alloc];
myVariable = [myVariable init];
// Set variable to 4/20
[myVariable setNumerator: 4];
[myVariable setDenominator: 20];
// Display the variable using the print method
NSLog (#"The value of myVariable is:");
[myVariable print];
}
return 0;
}
Just allocate and then initialize, yo do not need to do any thing else.
Keep in mind getters and setters.
How would you write a unit test—using OCUnit, for instance—to ensure that objects are being released/retained properly in Cocoa/Objective-C?
A naïve way to do this would be to check the value of retainCount, but of course you should never use retainCount. Can you simply check whether an object's reference is assigned a value of nil to indicate that it has been released? Also, what guarantees do you have about the timing at which objects are actually deallocated?
I'm hoping for a concise solution of only a few lines of code, as I will probably use this extensively. There may actually be two answers: one that uses the autorelease pool, and another that does not.
To clarify, I'm not looking for a way to comprehensively test every object that I create. It's impossible to unit test any behavior comprehensively, let alone memory management. At the very least, though, it would be nice to check the behavior of released objects for regression testing (and ensure that the same memory-related bug doesn't happen twice).
About the Answers
I accepted BJ Homer's answer because I found it to be the easiest, most concise way of accomplishing what I had in mind, given the caveat that the weak pointers provided with Automatic Reference Counting aren't available in production versions of XCode (prior to 4.2?) as of July 23rd, 2011. I was also impressed to learn that
ARC can be enabled on a per-file basis; it does not require that your
entire project use it. You could compile your unit tests with ARC and
leave your main project on manual retain-release, and this test would
still work.
That being said, for a far more detailed exploration of the potential issues involved with unit testing memory management in Objective-C, I highly recommend Peter Hosey's in-depth response.
Can you simply check whether an object's reference is assigned a value of nil to indicate that it has been released?
No, because sending a release message to an object and assigning nil to a variable are two different and unrelated things.
The closest you can get is that assigning anything to a strong/retaining or copying property, which translates to an accessor message, causes the previous value of the property to be released (which is done by the setter). Even so, watching the value of the property—using KVO, say—does not mean you will know when the object is released; most especially, when the owning object is deallocated, you will not get a notification when it sends release directly to the owned object. You will also get a warning message in your console (because the owning object died while you were observing it), and you do not want noisy warning messages from a unit test. Plus, you would have to specifically observe every property of every object to pull this off—miss one, and you may be missing a bug.
A release message to an object has no effect on any variables that point to that object. Neither does deallocation.
This changes slightly under ARC: Weak-referencing variables will be automatically assigned nil when the referenced object goes away. That doesn't help you much, though, because strongly-referencing variables, by definition, will not: If there's a strong reference to the object, the object won't (well, shouldn't) go away, because the strong reference will (should) keep it alive. An object dying before it should is one of the problems you're looking for, not something you'll want to use as a tool.
You could theoretically create a weak reference to every object you create, but you would have to refer to every object specifically, creating a variable for it manually in your code. As you can imagine, a tremendous pain and certain to miss objects.
Also, what guarantees do you have about the timing at which objects are actually released?
An object is released by sending it a release message, so the object is released when it receives that message.
Perhaps you meant “deallocated”. Releasing merely brings it closer to that point; an object can be released many times and still have a long life ahead of it if each release merely balanced out a previous retain.
An object is deallocated when it is released for the last time. This happens immediately. The infamous retainCount doesn't even go down to 0, as many a clever person who tried to write while ([obj retainCount] > 0) [obj release]; has found out.
There may actually be two answers: one that uses the autorelease pool, and another that does not.
A solution that uses the autorelease pool only works for objects that are autoreleased; by definition, objects not autoreleased do not go into the pool. It is entirely valid, and occasionally desirable, to never autorelease certain objects (particularly those you create many thousands of). Moreover, you can't look into the pool to see what's in it and what's not, or attempt to poke each object to see if it's dead.
How would you write a unit test—using OCUnit, for instance—to ensure that objects are being released/retained properly in Cocoa/Objective-C?
The best you could do is to set NSZombieEnabled to YES in setUp and restore its previous value in tearDown. This will catch over-releases/under-retains, but not leaks of any kind.
Even if you could write a unit test that thoroughly tests memory management, it would still be imperfect because it can only test the testable code—model objects and maybe certain controllers. You could still have leaks and crashes in your application caused by view code, nib-borne references and certain options (“Release When Closed” comes to mind), and so on.
There's no out-of-application test you can write that will ensure that your application is memory-bug-free.
That said, a test like you're imagining, if it were self-contained and automatic, would be pretty cool, even if it couldn't test everything. So I hope that I'm wrong and there is a way.
If you can use the newly-introduced Automatic Reference Counting (not yet available in production versions of Xcode, but documented here), then you could use weak pointers to test whether anything was over-retained.
- (void)testMemory {
__weak id testingPointer = nil;
id someObject = // some object with a 'foo' property
#autoreleasepool {
// Point the weak pointer to the thing we expect to be dealloc'd
// when we're done.
id theFoo = [someObject theFoo];
testingPointer = theFoo;
[someObject setTheFoo:somethingElse];
// At this point, we still have a reference to 'theFoo',
// so 'testingPointer' is still valid. We need to nil it out.
STAssertNotNil(testingPointer, #"This will never happen, since we're still holding it.")
theFoo = nil;
}
// Now the last strong reference to 'theFoo' should be gone, so 'testingPointer' will revert to nil
STAssertNil(testingPointer, #"Something didn't release %# when it should have", testingPointer);
}
Note that this works under ARC because of this change to the language semantics:
A retainable object pointer is either a null pointer or a pointer to a valid object.
Thus, the act of setting a pointer to nil is guaranteed to release the object it points to, and there's no way (under ARC) to release an object without removing a pointer to it.
One thing to note is that ARC can be enabled on a per-file basis; it does not require that your entire project use it. You could compile your unit tests with ARC and leave your main project on manual retain-release, and this test would still work.
The above does not detect over-releasing, but that's fairly easy to catch with NSZombieEnabled anyway.
If ARC is simply not an option, you may be able to do something similar with Mike Ash's MAZeroingWeakRef. I haven't used it much, but it seems to provide similar functionality to __weak pointers in a backwards-compatible way.
this is possibly not what you're looking for, but as a thought experiment I wondered if this might do something close to what you want: what if you created a mechanism to track the retain/release behavior for particular objects you wanted to test. Work it something like this:
create an override of NSObject dealloc
create a CFMutableSetRef and set up a custom retain/release functions to do nothing
make a unit test routine like registerForRRTracking: (id) object
make a unit test routine like clearRRTrackingReportingLeaks: (BOOL) report that will report any object in the set at that point in time.
call [tracker clearRRTrackignReportingLeaks: NO]; at the start of your unit test
call the register method in your unit test for every object you want to track and it'll be removed automatically on dealloc.
At the end of your test call the [tracker clearRRTrackingReportingLeaks: YES]; and it'll list all the objects that were not disposed of properly.
you could override NSObject alloc as well and just track everything but I imagine your set would get overly large (!!!).
Even better would be to put the CFMutableSetRef in a separate process and thus not have it impact your program runtime memory footprint overly much. Adds the complexity and runtime hit of inter-process communication though. Could use a private heap ( or zone - do those still exist?) to isolate it to a lesser degree.
I have recently read Apple's sample code for MVCNetworking written by Apple's Developer Technical Support guru Quinn "The Eskimo!". The sample is really nice learning experience with what I guess are best development practices for iOS development.
What surprised me, coming from JVM languages, are extremely frequent assertions like this:
syncDate = [NSDate date];
assert(syncDate != nil);
and this:
photosToRemove = [NSMutableSet setWithArray:knownPhotos];
assert(photosToRemove != nil);
and this:
photoIDToKnownPhotos = [NSMutableDictionary dictionary];
assert(photoIDToKnownPhotos != nil);
Is that really necessary? Is that coding style worth emulating?
If you're used to Java, this may seem strange. You'd expect an object creation message to throw an exception when it fails, rather than return nil. However, while Objective-C on Mac OS X has support for exception handling; it's an optional feature that can be turned on/off with a compiler flag. The standard libraries are written so they can be used without exception handling turned on: hence messages often return nil to indicate errors, and sometimes require you to also pass a pointer to an NSError* variable. (This is for Mac development, I'm not sure whether you can even turn exception handling support on for iOS, considering you also can't turn on garbage collection for iOS.)
The section "Handling Initialization Failure" in the document "The Objective-C Programming Language" explains how Objective-C programmers are expected to deal with errors in object initialization/creation: that is, return nil.
Something like [NSData dataWithContentsOfFile: path] may definitely return nil: the documentation for the method explicitly says so. But I'm honestly not sure whether something like [NSMutableArray arrayWithCapacity: n] ever returns nil. The only situation I can think of when it might is when the application is out of memory. But in that case I'd expect the application to be aborted by the attempt to allocate more memory. I have not checked this though, and it may very well be that it returns nil in this case. While in Objective-C you can often safely send messages to nil, this could then still lead to undesirable results. For example, your application may try to make an NSMutableArray, get nil instead, and then happily continue sending addObject: to nil and write out an empty file to disk rather than one with elements of the array as intended. So in some cases it's better to check explicitly whether the result of a message was nil. Whether doing it at every object creation is necessary, like the programmer you're quoting is doing, I'm not sure. Better safe than sorry perhaps?
Edit: I'd like to add that while checking that object creation succeeded can sometimes be a good idea, asserting it may not be the best idea. You'd want this to be also checked in the release version of your application, not just in the debug version. Otherwise it kind of defeats the point of checking it, since you don't want the application end user to, for example, wind up with empty files because [NSMutableArray arrayWithCapacity: n] returned nil and the application continued sending messages to the nil return value. Assertions (with assert or NSAssert) can be removed from the release version with compiler flags; Xcode doesn't seem to include these flags by default in the "Release" configuration though. But if you'd want to use these flags to remove some other assertions, you'd also be removing all your "object creation succeeded" checks.
Edit: Upon further reflection, it seems more plausible than I first thought that [NSMutableArray arrayWithCapacity: n] would return nil rather than abort the application when not enough memory is available. Basic C malloc also doesn't abort but returns a NULL pointer when not enough memory is available. But I haven't yet found any clear mention of this in the Objective-C documentation on alloc and similar methods.
Edit: Above I said I wasn't sure checking for nil is necessary at every object creation. But it shouldn't be. This is exactly why Objective-C allows sending messages to nil, which then return nil (or 0 or something similar, depending on the message definition): this way, nil can propagate through your code somewhat similar to an exception so that you don't have to explicitly check for nil at every single message that might return it. But it's a good idea to check for it at points where you don't want it to propagate, like when writing files, interacting with the user and so on, or in cases where the result of sending a message to nil is undefined (as explained in the documentation on sending messages to nil). I'd be inclined to say this is like the "poor man's" version of exception propagation&handling, though not everyone may agree that the latter is better; but nil doesn't tell you anything about why an error occurred and you can easily forget to check for it where such checks are necessary.
Yup. I think it's a good idea.. It helps to filter out the edge cases (out of memory, input variables empty/nil) as soon as the variables are introduced. Although I am not sure the impact on speed because of the overhead!
I guess it's a matter of personal choice. Usually asserts are used for debugging purpose so that the app crashes at the assert points if the conditions are not met. You'd normally like to strip them out on your app releases though.
I personally am too lazy to place asserts around every block of code as you have shown. I think it's close to being a bit too paranoid. Asserts might be pretty handy in case of conditions where some uncertainity is involved.
I have also asked this on Apple DevForums. According to Quinn "The Eskimo!" (author of the MVCNetworking sample in question) it is a matter of coding style and his personal preference:
I use lots of asserts because I hate debugging. (...)
Keep in mind that I grew up with traditional Mac OS, where a single rogue pointer could bring down your entire machine (similarly to kernel programming on current systems). In that world it was important to find your bugs sooner rather than later. And lots of asserts help you do that.
Also, even today I spend much of my life dealing with network programs. Debugging network programs is hard because of the asynchrony involved. Asserts help to with this, because they are continually checking the state of your program as it runs.
However, I think you have a valid point with stuff like +[NSDate date]. The chances of that returning nil are low. The assert is there purely from habit. But I think the costs of this habit (some extra typing, learning to ignore the asserts) are small compared to the benefits.
From this I gather that asserting that every object creation succeeded is not strictly necessary.
Asserts can be valuable to document the pre-conditions in methods, during development, as design aid for other maintainers (including the future self). I personally prefer the alternative style - to separate the specification and implementation using TDD/BDD practices.
Asserts can be used to double-check runtime types of method arguments due to the dynamic nature of Objective C:
assert([response isKindOfClass:[NSHTTPURLResponse class]]);
I'm sure there are more good uses of assertions. All Things In Moderation...
Sorry for long description, however the questions aren't so easy...
My project written without GC. Recently I found a memory leak that I can't find. I did use new Xcode Analyzer without a result. I did read my code line by line and verified all alloc/release/copy/autorelease/mutableCopy/retain and pools... - still nothing.
Preamble: Standard Instruments and Omni Leak Checker don't work for me by some reason (Omin Tool rejects my app, Instruments.app (Leaks) eats too many memory and CPU so I have no chance to use it).
So I wanna write and use my own code to hook & track "all" alloc/allocWithZone:/dealloc messages statistics to write some simple own leaks checking library (the main goal is only to mark objects' class names with possible leaks).
The main hooking technique that I use:
Method originalAllocWithZone = class_getClassMethod([NSObject class],#selector(allocWithZone:));
if (originalAllocWithZone)
{
imp_azo = (t_impAZOriginal)method_getImplementation(originalAllocWithZone);
if (imp_azo)
{
Method hookedAllocWithZone = class_getClassMethod([NSObject class],#selector(hookedAllocWithZone:));
if (hookedAllocWithZone)
{
method_setImplementation(originalAllocWithZone,method_getImplementation(hookedAllocWithZone));
fprintf(stderr,"Leaks Hook: allocWithZone: ; Installed\n");
}
}
}
code like this for hook the alloc method, and dealloc as NSObject category method.
I save IMP for previous methods implementation then register & calculate all alloc/allocWithZone: calls as increment (+1) stat-array NSInteger values, and dealloc calls as decrement (-1).
As end point I call previous implementation and return value.
In concept all works just fine.
If it needs, I can even detect when class are part of class cluster (like NSString, NSPathStore2; NSDate, __NSCFDate)... via some normalize-function (but it doesn't matter for the issues described bellow).
However this technique has some issues:
Not all classes can be caught, for
example, [NSDate date] doesn't catch
in alloc/allocWithZone: at all, however, I can see alloc call in GDB
Since I'm trying to use auto singleton detection technique (based on retainCount readind) to auto exclude some objects from final statistics, NSLocale creation freezes on pre-init stage when starting of full Cocoa application (actually, even simple Objective-C command line utility with the Foundation framework included has some additional initialization before main()) - by GDB there is allocWithZone: calls one after other,....
Full Concept-Project draft sources uploaded here: http://unclemif.com/external/DILeak.zip (3.5 Kb)
Run make from Terminal.app to compile it, run ./concept to show it in action.
The 1st Question: Why I can't catch all object allocations by hooking alloc & allocWithZone: methods?
The 2nd Question: Why hooked allocWithZone: freezes in CFGetRetainCount (or [inst retainCount]) for some classes...
Holy re-inventing the wheel, batman!
You are making this way harder than it needs to be. There is absolutely no need whatsoever to roll your own object tracking tools (though it is an interesting mental exercise).
Because you are using GC, the tools for tracking allocations and identifying leaks are all very mature.
Under GC, a leak will take one of two forms; either there will be a strong reference to the object that should long ago been destroyed or the object has been CFRetain'd without a balancing CFRelease.
The collector is quite adept at figuring out why any given object is remaining beyond its welcome.
Thus, you need to find some set of objects that are sticking around too long. Any object will do. Once you have the address of said object, you can use the Object Graph instrument in Instruments to figure out why it is sticking around; figure out what is still referring to it or where it was retained.
Or, from gdb, use info gc-roots 0xaddr to find all of the various things that are rooting the object. If you turn on malloc history (see the malloc man page), you can get the allocation histories of the objects that are holding the reference.
Oh, without GC, huh...
You are still left with a plethora of tools and no need to re-invent the wheel.
The leaks command line tool will often give you some good clues. Turn on MallocStackLoggingNoCompact to be able to use malloc_history (another command line tool).
Or use the ObjectAlloc instrument.
In any case, you need to identify an object or two that is being leaked. With that, you can figure out what is hanging on to it. In non-GC, that is entirely a case of figuring out why it there is a retain not balanced by a release.
Even without the Leaks instrument, Instruments can still help you.
Start with the Leaks template, then delete the Leaks instrument from it (since you say it uses too much memory). ObjectAlloc alone will tell you all of your objects' allocations and deallocations, and (with an option turned on, which it is by default in the Leaks template) all of their retentions and releases as well.
You can set the ObjectAlloc instrument to only show you objects that still exist; if you bring the application to the point where no objects (or no objects of a certain class) should exist, and such objects do still exist, then you have a leak. You can then drill down to find the cause of the leak.
This video may help.
Start from the Xcode templates. Don't try to roll your own main() routine for a cocoa app until you know what you're doing.