Stack overflow in release binary but not in Xcode-Debugger - objective-c

Question

My (Cocoa)-App runs perfectly fine when I start it from within Xcode. However when I archive / release it, that version will crash. The error reporter that pops open says:
[...]
Crashed Thread: 0 Dispatch queue: com.apple.main-thread
Exception Type: EXC_CRASH (SIGABRT)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Application Specific Information:
[9082] stack overflow
Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0 libsystem_kernel.dylib 0x00007fff944a7212 __pthread_kill + 10
1 libsystem_c.dylib 0x00007fff9290caf4 pthread_kill + 90
2 libsystem_c.dylib 0x00007fff92950e9e __abort + 159
3 libsystem_c.dylib 0x00007fff92951d17 __stack_chk_fail + 195
[...]
While it also gives me the line of code where execution stopped, this really does not help me as the exact same code path is executed in debug mode (but succeedes).
So I'm wondering: might it actually be that the stack sizes are different for a release and a debug version? And how big is the stack after all on a Mac (64 bit / Mountain Lion)? I'm not aware of putting insanely much data on the stack...
If I have too much data on the stack, what patterns would I need to avoid to reduce my stack load?
[Update]
ok, I got my App running by adding the -fno-stack-protector flag. (BTW: I'm compiling with LLVM)
Before that, I stepped through the crashing code line by line and found the following bahaviour which I don't understand:
the method foo(x) calls bacon(x). x is 8 and is handed from foo to bacon without modification. Yet when I step into bacon(x), x suddenly is 4295939448 (each time). If I set -fno-stack-protector the value is correct.
To my naive eyes, this looks as if the stack-protector sets the magic value 4295939448 somewhere in the stack and makes it read-only. And while my functions put their parameters on the stack, at some point parameter x happens to be put on that magic address and thus cannot be written (subsequent parameters seem to be written correctly). In my case x is a buffer length parameter, which naturally leads to a buffer-overflow and crash.
Does somebody have a deeper understanding of the stack-protector? Why is this happening? And in what cases is it safe and legal to disable the stack-protector and in what cases is it dangerous?
[Update 2: Original Code]
This method calls the other Decrypt below. stIVLen at that point is 8
BOOL CEracomSE::Decrypt(
PBYTE pMsg, size_t stLen,
const CSK* pKey /* = NULL */,
PBYTE pIV /* = NULL */, size_t stIVLen /* = 0 */,
FBM fbm /* = FBM_CBC */,
PADDING padding /* = NO_PADDING */
)
{
//stIVLen == 8
return Decrypt( (uint64_t)0, pMsg, stLen, pKey, pIV, stIVLen, fbm, padding );
}
When Decrypt is called stIVLen is 4295939448, the other parameter are still correct
BOOL CEracomSE::Decrypt(
uint64_t qwOffset,
PBYTE pMsg, size_t stLen,
const CSK* pKey /* = NULL */,
PBYTE pIV /* = NULL */, size_t stIVLen /* = 0 */,
FBM fbm /* = FBM_CBC */,
PADDING padding /* = NO_PADDING */
)
{
//stIVLen now is 4295939448
BYTE a_iv[16] = {0};
size_t a_iv_len;
BYTE a_key[32] = {0};
size_t a_key_len = 0;
size_t nBytes;
size_t nDataOffset;
size_t nRemainingData = stLen;
bool ret;
//[...]
}

Answer 1

I recently faced this situation with my app. I know its an old thread but responding anyways with the intent that someone else could benefit from the findings.
Passing -fno-stack-protector did solve the problem as suggested in the question. However, digging a little deeper we found that, changing all occurrences of literal array declaration to a longer declaration did solve the problem without passing the compiler flag.
so changing all occurrences of
#[#(1), #(2)]
to
[NSArray arrayWithObjects:#(1), #(2), nil]
May be its specific to our app only but hope it helps others as well.