I am creating a method for executing shell commands. It looks like this:
NSString *cShellStr(NSString *command, int maxBufferSize) {
if (maxBufferSize<1) {
maxBufferSize = INT_MAX;
}
NSString *newCommand = [NSString stringWithFormat:#"%# 2>&1", command];
const char *cStrCommand = [newCommand cStringUsingEncoding:NSUTF8StringEncoding];
FILE *fp;
fp = popen(cStrCommand, "r");
if (fp == NULL) {
NSLog(#"Failed to open process");
return nil;
}
char *buffer = NULL;
buffer = (char*)malloc(4);
if (buffer == NULL) {
NSLog(#"Failed to allocate memory");
return nil;
}
while (!feof(fp)) {
buffer = realloc(buffer, sizeof(buffer)+1);
sprintf(buffer, "%s%c", buffer, fgetc(fp));
}
NSString *ret = [NSString stringWithCString:buffer encoding:NSASCIIStringEncoding];
fclose(fp);
free(buffer);
return ret;
}
However, it usually (strangely not always) gets this error at [NSString stringWithCString:buffer encoding:NSASCIIStringEncoding]:
objc[33674]: Method cache corrupted. This may be a message to an invalid object, or a memory error somewhere else.
objc[33674]: receiver 0x7fff7d62aec0, SEL 0x7fff99ac69f5, isa 0x7fff7d62aee8, cache 0x7fff7d62aef8, buckets 0x100200780, mask 0x3, occupied 0x2, wrap bucket 0x100200780
objc[33674]: receiver 0 bytes, buckets 64 bytes
objc[33674]: selector 'class'
objc[33674]: isa 'NSString'
objc[33674]: Method cache corrupted.
objc[33674]: Method cache corrupted. This may be a message to an invalid object, or a memory error somewhere else.
objc[33674]: receiver 0x7fff7d62aec0, SEL 0x7fff99aea2d4, isa 0x7fff7d62aee8, cache 0x7fff7d62aef8, buckets 0x100200780, mask 0x3, occupied 0x2, wrap bucket 0x100200780
objc[33674]: receiver 0 bytes, buckets 64 bytes
objc[33674]: selector 'stringWithCString:encoding:'
objc[33674]: isa 'NSString'
objc[33674]: Method cache corrupted.
I have stepped through it with lldb and confirmed that there are no other errors and that buffer contains exactly the text I would expect. Am I incorrectly using stringWithCString?
Extra info:
Xcode version:
Laptop info:
Clang version:
Apple LLVM version
6.0 (clang-600.0.51) (based on LLVM 3.5svn)
Target: x86_64-apple-darwin13.3.0
Thread model: posix
while (!feof(fp)) {
buffer = realloc(buffer, sizeof(buffer)+1);
sprintf(buffer, "%s%c", buffer, fgetc(fp));
}
This loop should be rewritten as:
size_t len = 0, cap = 4;
buffer = malloc(cap); // TODO check NULL
int c;
while (EOF != (c = fgetc(fp))) {
if (len >= cap) {
cap += cap;
buffer = realloc(buffer, cap); // TODO check NULL
}
buffer[len++] = c;
}
// check feof/ferror
sprintf(buffer, "%s%c", buffer, fgetc(fp));
This is undefined behaviour because you are using buffer twice. Anything could go wrong, and lucky for you, it does. I'm saying "lucky for you" because with a bit of bad luck it would only go wrong in the hand of customers.
I didn't watch out; the issue that user3125367 found is ten times worse.
Related
I noticed that, while generating a texture from an MTLBuffer created from mmap() via newBufferWithBytesNoCopy, if the size requested by the len argument to mmap, page aligned, is larger than the actual size of the file, page aligned, the mmap call succeeds, and the newBufferWithBytesNoCopy message does not result in a nil return or error, but when I pass the buffer to the GPU to copy the data to an MTLTexture, the following is printed to the console, and all GPU commands fail to perform any action:
Execution of the command buffer was aborted due to an error during execution. Internal Error (IOAF code -536870211)
Here is code to demonstrate the problem:
static id<MTLDevice> Device;
static id<MTLCommandQueue> Queue;
static id<MTLTexture> BlockTexture[3];
#define TEX_LEN_1 1 // These are all made 1 in this question for simplicity
#define TEX_LEN_2 1
#define TEX_LEN_4 1
#define TEX_SIZE ((TEX_LEN_1<<10)+(TEX_LEN_2<<11)+(TEX_LEN_4<<12))
#define PAGE_ALIGN(S) ((S)+PAGE_SIZE-1&~(PAGE_SIZE-1))
int main(void) {
if (!(Queue = [Device = MTLCreateSystemDefaultDevice() newCommandQueue]))
return EXIT_FAILURE;
#autoreleasepool {
const id<MTLBuffer> data = ({
void *const map = ({
NSFileHandle *const file = [NSFileHandle fileHandleForReadingAtPath:[NSBundle.mainBundle pathForResource:#"Content" ofType:nil]];
if (!file)
return EXIT_FAILURE;
mmap(NULL, TEX_SIZE, PROT_READ, MAP_SHARED, file.fileDescriptor, 0);
});
if (map == MAP_FAILED)
return errno;
[Device newBufferWithBytesNoCopy:map length:PAGE_ALIGN(TEX_SIZE) options:MTLResourceStorageModeShared deallocator:^(void *const ptr, const NSUInteger len){
munmap(ptr, len);
}];
});
if (!data)
return EXIT_FAILURE;
const id<MTLCommandBuffer> buffer = [Queue commandBuffer];
const id<MTLBlitCommandEncoder> encoder = [buffer blitCommandEncoder];
if (!encoder)
return EXIT_FAILURE;
{
MTLTextureDescriptor *const descriptor = [MTLTextureDescriptor new];
descriptor.width = descriptor.height = 32;
descriptor.mipmapLevelCount = 6;
descriptor.textureType = MTLTextureType2DArray;
descriptor.storageMode = MTLStorageModePrivate;
const enum MTLPixelFormat format[] = {MTLPixelFormatR8Unorm, MTLPixelFormatRG8Unorm, MTLPixelFormatRGBA8Unorm};
const NSUInteger len[] = {TEX_LEN_1, TEX_LEN_2, TEX_LEN_4};
for (NSUInteger i = 3, off = 0; i--;) {
descriptor.pixelFormat = format[i];
const NSUInteger l = descriptor.arrayLength = len[i];
const id<MTLTexture> texture = [Device newTextureWithDescriptor:descriptor];
if (!texture)
return EXIT_FAILURE;
const NSUInteger br = 32<<i, bi = 1024<<i;
for (NSUInteger j = 0; j < l; off += bi)
[encoder copyFromBuffer:data sourceOffset:off sourceBytesPerRow:br sourceBytesPerImage:bi sourceSize:(const MTLSize){32, 32, 1} toTexture:texture destinationSlice:j++ destinationLevel:0 destinationOrigin:(const MTLOrigin){0}];
[encoder generateMipmapsForTexture:BlockTexture[i] = texture];
}
}
[encoder endEncoding];
[buffer commit];
}
// Rest of code to initialize application (omitted)
}
In this case, the command will fail if the size of the actual Content file is less than 4097 bytes, assuming a 4096 page size. What is the most strange is that neither the mmap() nor the newBufferWithBytesNoCopy fails in this case, but the GPU execution fails so badly that any/all subsequent GPU calls also fail.
Is there a way to cause predictable behavior? I thought that mmap() space beyond the file was just valid 0 memory. Why is this apparently not the case if the space is being used by the GPU? At the very least, how can I detect GPU execution errors or invalid buffers like this to handle them gracefully, besides manually checking if the file is too small? Am I using these functions incorrectly somehow? Is something here undefined behavior?
My research efforts including Google searching for terms such as newBufferWithBytesNoCopy and/or mmap together with 536870211, and got absolutely no results. Now, this question is the only result for such searches.
My guess is this problem has to do with the inner workings of the GPU and/or the MTLBuffer implementation and/or mmap() and its underlying facilities. Not having access to these inner workings, I have no idea how to even start figuring out a solution. I would appreciate an expert to enlighten me as to what is actually going on behind the scenes causing this error, and how to avoid it (besides manually checking if the file is too big, as this is a 'workaround' but does not really fix the problem at its base, or at the very least how to gracefully detect GPU crashes of this type and abort the application gracefully.
I'm dealing with the garmin GDL90 protocol which sends across various types of messages in binary to my IOS device. I'm going through and trying to process all these messages but have been running into an issue. Specifically the messages are byte packed so that if you ever see an occurrence of
0x7d 0x5e or 0x7d 0x5d you have to convert them to 0x7d or 0x7e
I've set my code up so that I detect the message type I'm parsing and then call a function:
- (void) parseMessage:(NSMutableData *)message
to do my data parsing. My individual message parsing functions call the parent function [super parseMessage:message]; which handles both the parsing of common elements as well as dealing with my byte-stuffing. Each of these function calls takes an NSData * so shouldn't a modification made in my super function return back out the same data?
My top level class gets a parse message call and the NSMutableData pointer's address is: 0x170048f10
Once I step into the parent's parseData call my address is still 0x170048f10
After I make modifications to the data I'm now pointing at the memory address 0x17805e840
Once I return from this function, however, I'm back pointing at 0x170048f10 again with the wrong data.
Should I be using pass by reference or something? Any suggestions?
I have two variations of my function - unstuff1 throws an error and unstuff2 doesn't work.
- (NSMutableData *)unstuff1:(NSMutableData *)mutableData {
int dataLength = [mutableData length];
char *bytes = [mutableData bytes];
// Scan bytes ignoring 1st and last byte because they will be 7e's
for (int i = dataLength - 1; i > 0; i--) {
bytes[i + 1] ^= 0x20;
if (i + 1 == dataLength) {
NSLog(#"Terminal character padding detected on character %d with length %d", i, dataLength);
} else {
/* Replace 2 bytes with a single byte should remove the flag when you do this */
[mutableData replaceBytesInRange:NSMakeRange(i, 2) withBytes:&bytes[i + 1] length:1];
dataLength--;
}
}
return mutableData;
}
- (NSMutableData *)unstuff2:(NSMutableData *)data {
NSMutableData *mutableData = [[NSMutableData alloc] initWithData:data];
int dataLength = [mutableData length];
char *bytes = [mutableData bytes];
// Scan bytes ignoring 1st and last byte because they will be 7e's
for (int i = dataLength - 1; i > 0; i--) {
bytes[i + 1] ^= 0x20;
if (i + 1 == dataLength) {
NSLog(#"Terminal character padding detected on character %d with length %d", i, dataLength);
} else {
/* Replace 2 bytes with a single byte should remove the flag when you do this */
[mutableData replaceBytesInRange:NSMakeRange(i, 2) withBytes:&bytes[i + 1] length:1];
dataLength--;
}
}
return mutableData;
}
In unstuff2 obviously i'm making a new MutableData so I guess that accounts for the memory address change (that is the function i was using that gave me the error specified).
unstuff1 throws the following exception:
-[_NSInlineData replaceBytesInRange:withBytes:length:]: unrecognized selector sent to instance 0x178250d40
Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[_NSInlineData replaceBytesInRange:withBytes:length:]: unrecognized selector sent to instance
Unlike languages like C++ and C# (just to name two), Objective C has no concept of "pass by reference". However, passing a copy of a pointer to your NSMutableData is functionally equivalent to passing the object by reference. That is, if you pass in an NSMutableData (or NSMutableAnything for that matter) to a function and that function modifies it, the calling function will see the changes reflected in the object that it passed in.
Ok looks like I tracked down the problem. I realized the compiler was sending out warnings:
incompatible pointer types initializing 'NSMutableData *' with an expression of type 'NSData *'
It turns out I had some code
NSMutableData *message = [data subdataWithRange:NSMakeRange(5, len - 5)];
Which i needed to convert into:
NSMutableData *message = [NSMutableData dataWithData:[data subdataWithRange:NSMakeRange(5, len - 5)]];
And then things all work out. Moral of the story (read your warnings!!!!)
I'm reading a file on my disk (which can be few GB in size) in 10MB chunks to verify MD5 for it. Method fetchRecords has been simplified as it is a bi
t long. The problem is that the data is released when fetchRecords method returns, by then I have few GB in memory. If file is big enough, it causes a crash. [dataChunk release] at the end does not help. Getting a lot of inactive memory until it returns.
- (void)fetchRecords
{
for (DownloadChunkInfo *downloadChunkInfo in [downloadFileInfo chunk])
{
NSData *dataChunk = [NSData dataWithContentsOfFile:fileDownloadPath withStartOffset:[downloadChunk startingByte] andEndOffset:[downloadChunk endingByte]];
if ([dataChunk length] == [downloadChunk length])
{
if ([downloadChunk md5] && [[dataChunk MD5] isEqualToString:[downloadChunk md5]])
{
// Some code
}
else
{
// Some code
}
}
[dataChunk release];
}
}
+ (NSData *)dataWithContentsOfFile:(NSString *)path withStartOffset:(off_t)startOffset andEndOffset:(off_t)endOffset
{
FILE *file = fopen([path UTF8String], "rb");
if(file == NULL)
return nil;
uint64_t size = (endOffset - startOffset) + 1;
void *data = malloc(size); // check for NULL!
fseeko(file, startOffset, SEEK_SET);
size_t realSize = fread(data, 1, size, file); // check return value, in case read was short!
fclose(file);
// NSData takes ownership and will call free(data) when it's released
return [NSData dataWithBytesNoCopy:data length:realSize];
}
[dataChunk release] is actually wrong, because you don't "own" the object returned by
NSData *dataChunk = [NSData dataWithContentsOfFile:...];
The return value is (subject to possible optimizations made by the compiler)
an "autoreleased" object, which is released only when the current autorelease pool
is destroyed.
Therefore, using a local autorelease pool should help:
for (DownloadChunkInfo *downloadChunkInfo in [downloadFileInfo chunk])
{
#autoreleasepool {
NSData *dataChunk = [NSData dataWithContentsOfFile:fileDownloadPath withStartOffset:[downloadChunk startingByte] andEndOffset:[downloadChunk endingByte]];
// do something with dataChunk ...
}
}
For more information, see
"Basic Memory Management Rules"
"Using Autorelease Pool Blocks"
in the "Advanced Memory Management Programming Guide" for more information.
I'm creating an Objective-C library to talk with some external devices through USB.
When calling a certain method, it crashes at a random place inside the method or inside some C system functions (related to malloc or pthread) with one of the following error "invalid checksum for freed object", "autorelease pool page 0x1102032 corrupted", or even an unknown selector error (whereas the selector does exist).
Using Guard Malloc feature, it stops on this line with an EXEC_BAD_ACCESS error:
- (void)theMethod {
// some code
NSMutableData *payloads_pool = [NSMutableData dataWithLength:0x800];
NSUInteger payloads_pool_length = [payloads_pool length];
void *buffer = [payloads_pool mutableBytes];
memset(buffer, 0xCC, payloads_pool_length);
for (i = 0; i < 0x800; i += 0x40) {
unsigned int *buf = [payloads_pool mutableBytes];
(buf+i)[0] = 0x405; <==== STOP ON THIS LINE
(buf+i)[1] = 0x101;
(buf+i)[2] = 0x8402B001;
(buf+i)[3] = 0x8402EB01;
}
// some code
}
Since buf is unsigned int*, isn’t the pointer + in buf+i adding unsigned integers instead of bytes, thus seeking too far in memory?
Then I'd guess that the payloads_pool has fewer than 0x800 bytes in.
What is the value of i when it crashes?
Why isn't your for loop from 0 to payloads_pool.length?
Sometimes when trying to get a path to a keychain returned by SecKeychainCopySearchList I get error with code -25301 which from the list of errors stands for errSecBufferTooSmall. The SecCopyErrorMessageString states:
There is not enough memory available to use the specified item.
Weird thing is that it doesn't always return the error on the very same keychain reference.
Here's how I try to get the path to the keychain:
- (NSString *)getKeychainPath:(SecKeychainRef)keychain {
char *pathName = malloc(sizeof(*pathName) * 1024);
UInt32 pathLength;
OSStatus errCode = SecKeychainGetPath(keychain, &pathLength, pathName);
if (errCode != errSecSuccess) {
NSString *errString = (NSString *)SecCopyErrorMessageString(errCode, NULL);
DLog(#"%d: %#", errCode, errString);
}
NSData *d = [NSData dataWithBytes:pathName length:pathLength];
return [[[NSString alloc] initWithData:d encoding:NSUTF8StringEncoding] autorelease];
}
I'm interested in what buffer does the function use? I've tried outputting the pathLength variable but it's way bellow the 1K bytes. What am I doing wrong? What should I do to avoid these errors? Can they be bypassed by any way at all?
From the SecKeychainGetPath documentation:
ioPathLength
On entry, a pointer to a variable containing the length (in bytes) of the buffer specified by pathName.
On return, the string length of pathName, not including the null termination.
You're not doing the "on input" part. You need to initialize pathLength to the size of the pathName buffer. For example:
UInt32 pathLength = 1024;
char *pathName = malloc(pathLength);