We have an NSMutableData object that frequently has data appended to it. We also frequently pull out data via the bytes method for reading.
We've synchronized access to this NSMutableData object via a pthread mutex:
pthread_mutex_t _mutex;
pthread_mutexattr_t attributes;
pthread_mutexattr_settype( &attributes, PTHREAD_MUTEX_DEFAULT );
pthread_mutex_init( &_mutex, &attributes );
and then every time we access this object we:
pthread_mutex_lock(&_mutex);
const UInt8* rawData = [_coverage bytes];
//code that works with the raw bytes
pthread_mutex_unlock(&_mutex);
Also, every addData method we have locks the mutex before adding data to the NSMutableData object.
The problem is we still get the occasional EXC_BAD_ACCESS while working with rawData. I understand that NSMutableBytes will grow its byte array as data gets added to it. I also understand that I shouldn't expect rawData to magically grow also.
I'm just wondering how we can ever get into this situation where rawData has been free'd from underneath us when we have explicitly locked access for both read and write?
Are we doing something wrong with the mutex or the way we are accessing the bytes?
EDIT
I discovered the real reason why I was getting an EXC_BAD_ACCESS. I was not initializing the mutex attributes, so locking the mutex did nothing. Here is the corrected code:
pthread_mutex_t _mutex;
pthread_mutexattr_t attributes;
pthread_mutexattr_init(&attributes);
pthread_mutex_init(&_mutex, &attributes);
pthread_mutexattr_destroy(&attributes);
Yes it is possible that is being freed from underneath you.
According to the documentation:
bytes
Returns a pointer to the receiver’s contents.
You should make copies of the data to ensure that it will not be changed or freed from underneath you. When your done with your copy make sure to free() it.
pthread_mutex_lock(&_mutex);
const UInt8 *origData = [_coverage bytes];
UInt8 *rawData;
memmove(rawData, origData, [_coverage length]);
//code that works with the raw bytes
free(rawData);
pthread_mutex_unlock(&_mutex);
Related
I have a passed in unmanaged buffer with a type like this BYTE*, so this is a byte buffer and length information is also known, and now I want to use byte buffer to create a vector like this std::vector<BYTE>,
I know I could use copy and manually delete this buffer later like following,
foo(BYTE* in, int size) {
...
std::vector<BYTE> vec;
std::copy(in, in + size, std::back_inserter(vec));
...
}
My question is what is the best way to avoid a copy here? I feel there should be a good way to create this vector so that this vector takes ownership of this buffer using given pointer, aside from avoid a copy, another advantage is that I don't have to manually delete this buffer after copy. I think there is definitely some elegant way to do this safely.
How can I remove memory allocated by const char* arrayname in iOS 7.
here is my code
const char *bytes = [encodedString UTF8String];
Now I want to release bytes with ARC enabled. How can I do that.
You don't need to do anything to ensure it will be released eventually. Quoting the documentation:
The returned C string is a pointer to a structure inside the string object, which may have a lifetime shorter than the string object and will certainly not have a longer lifetime. Therefore, you should copy the C string if it needs to be stored outside of the memory context in which you called this method.
In other words, you don't know the lifetime of the array, but you know it may not live long and it will be freed automatically.
You should not release memory pointed by bytes pointer directly as it is managed by encodedString object. To free that memory get rid of all strong references to encodedString and memory should get released
Both return the same pointer. I know - bytes belongs to NSData, why does NSMutableData introduce - mutableBytes? Is it just for code clarity so it is more obvious you are accessing mutable data? Does it really matter which one is used?
NSMutableData* mydata = [[NSMutableData alloc] init];
[mydata appendData: [#"hello" dataUsingEncoding:NSUTF8StringEncoding]];
NSLog(#"%p", [mydata mutableBytes]);
NSLog(#"%p", [mydata bytes]);
Thanks.
There are a couple of reasons why NSMutableData might provide a separate mutableBytes method:
As you suggested in your question, using mutableBytes makes it clear to the reader that you want to change the data.
The bytes method returns a const void *. The mutableBytes method returns a void *. If you want to change the bytes, you need a void * with no const qualifier. The mutableBytes method eliminates the need to cast away the const qualifier.
In theory there could be a third reason: the -[NSData mutableCopy] method could return an NSMutableData that points to the same buffer as the original NSData, and only create a new, mutable copy of the buffer when you call mutableBytes. However, I don't think it's implemented this way based on my very limited testing.
One addition to the rob's answer and his comment:
#Dabbu NSData and NSMutableData store their contents as one contiguous
array of bytes.
The thing to keep in mind here is that this behavior was changed in iOS7: now NSData/NSMutableData are not guaranteed to keep contents as one contiguous array. It could be stored as multiple chunks.
So when you call bytes/mutableBytes, they will copy and flatten contents into one contiguous array of bytes, if needed, and then return a pointer to this contiguous chunk.
Depending of what you're trying to do, it may cause an unexpected performance penalty or excessive memory consumption for large buffers.
I am slightly bemused by the following property points of class MKMultiPoint in MapKit:
#property (nonatomic, readonly) MKMapPoint *points
It returns an array of struct. One can know the number of elements in the array with the pointCount property.
With my limited knowledge of C, I always thought C-arrays could only be "sort of returned" if passed by reference to a function because the caller is responsible for allocating the memory and then releasing it.
If I were to write a similar property, who would allocate the memory for the array (presumably the callee) and more importantly who would free it (presumably the caller)? That sounds a bit risky to me. Besides, the documentation for the property above doesn't say anything about having to free memory.
What am I missing?
(The sample code is in C).
The good practice is to allocate and free a resource at the same level. There are two ways to define a function that returns an array of things:
// `points` are allocated and freed by the caller.
void MakePoints (MKMapPoint *points, size_t number_of_points);
// usage:
size_t count = 10;
MKMapPoint *points = malloc (sizeof (MKMapPoint) * 10);
MakePoints (points, count);
// Use points
free (points);
// or simply
MKMapPoint points[10];
MakePoints (points, 10);
// Use points
The second way is to let the library function manage the memory:
MKMapPoint *MakePoints (size_t number_of_points);
void FreePoints (MKMapPoint *points);
// Usage:
MKMapPoint *points = MakePoints (10);
// Use points
// The library need not necessarily call free() on points,
// it might reuse it in further calls to MakePoints().
FreePoints (points);
The receiver, in most cases, would handle allocating the memory. Who frees it depends on how you determine ownership. Is the memory allocated by the receiver no longer needed once it's returned? If so, you should probably note in your documentation that the caller needs to free the returned array. If the receiver can reuse the returned memory, leave deallocation to it.
If you wanted to leave it up to the callee to handle memory allocation, you probably wouldn't use a property and instead opt for a message and property like so:
- (NSUInteger) mapPointCount;
- (void) getMapPoints:(MKMapPoint *)pointsOut;
where the sender should provide an existing buffer to store obj.mapPointCount number of MKMapPoints in. Then, you've placed the responsibility for allocation/deallocation on the caller.
If you don't want to go that route, and since the memory in question can't be retained/released/autoreleased by the receiver, I would leave it up to the caller to release it. If you want to make it somewhat clear that the memory is not to be freed, return it with the type const MKMapPoint * and note it in some form of documentation (which should hopefully make it clear that the memory is not owned by whoever accesses the data).
Alternatively, store it in an NSData or something and make it clear that once the next autorelease pool is drained, the pointer is invalid. However, that's a little less friendly, and possibly not safe with garbage collection. Probably wrong about that last bit, but I don't know enough to say, so I'll prefer caution for now.
I'm working with Objective-C and I need to add int's from a NSArray to a NSMutableData (I'm preparing a to send the data over a connection). If I wrap the int's with NSNumber and then add them to NSMutableData, how would I find out how many bytes are in the NSNumber int? Would it be possible to use sizeof() since according to the apple documentation, "NSNumber is a subclass of NSValue that offers a value as any C scalar (numeric) type."?
Example:
NSNumber *numero = [[NSNumber alloc] initWithInt:5];
NSMutableData *data = [[NSMutableData alloc] initWithCapacity:0];
[data appendBytes:numero length:sizeof(numero)];
numero is not a numeric value, it is a pointer to a an object represting a numeric value. What you are trying to do won't work, the size will always be equal to a pointer (4 for 32 bit platforms and 8 for 64 bit), and you will append some garbage pointer value to your data as opposed to the number.
Even if you were to try to dereference it, you cannot directly access the bytes backing an NSNumber and expect it to work. What is going on is an internal implementation detail, and may vary from release to release, or even between different configurations of the same release (32 bit vs 64 bit, iPhone vs Mac OS X, arm vs i386 vs PPC). Just packing up the bytes and sending them over the wire may result in something that does not deserialize properly on the other side, even if you managed to get to the actual data.
You really need to come up with an encoding of an integer you can put into your data and then pack and unpack the NSNumbers into that. Something like:
NSNumber *myNumber = ... //(get a value somehow)
int32_t myInteger = [myNumber integerValue]; //Get the integerValue out of the number
int32_t networkInteger = htonl(myInteger); //Convert the integer to network endian
[data appendBytes:&networkInteger sizeof(networkInteger)]; //stuff it into the data
On the receiving side you then grab out the integer and recreate an NSNumber with numberWithInteger: after using ntohl to convert it to native host format.
It may require a bit more work if you are trying to send minimal representations, etc.
The other option is to use an NSCoder subclass and tell the NSNumber to encode itself using your coder, since that will be platform neutral, but it may be overkill for what you are trying to do.
First, NSNumber *numero is "A pointer to a NSNumber type", and the NSNumber type is an Objective-C object. In general, unless specifically stated somewhere in the documentation, the rule of thumb in object-oriented programming is that "The internal details of how an object chooses to represent its internal state is private to the objects implementation, and should be treated as a black box." Again, unless the documentation says you can do otherwise, you can't assume that NSNumber is using a C primitive type of int to store the int value you gave it.
The following is a rough approximation of what's going on 'behind the scenes' when you appendBytes:numero:
typedef struct {
Class isa;
double dbl;
long long ll;
} NSNumber;
NSNumber *numero = malloc(sizeof(NSNumber));
memset(numero, 0, sizeof(NSNumber));
numero->isa = objc_getClass("NSNumber");
void *bytes = malloc(1024);
memcpy(bytes, numero, sizeof(numero)); // sizeof(numero) == sizeof(void *)
This makes it a bit more clear that what you're appending to the NSMutableData object data is the first four bytes of what ever numero is pointing to (which, for an object in Obj-C is always isa, the objects class). I suspect what you "wanted" to do was copy the pointer to the instantiated object (the value of numero), in which case you should have used &numero. This is a problem if you're using GC as the buffer used by NSMutableData is not scanned (ie, the GC system will no longer "see" the object and reclaim it, which is pretty much a guarantee for a random crash at some later point.)
It's hopefully obvious that even if you put the pointer to the instantiated NSNumber object in to data, that pointer only has meaning in the context of the process that created it. A pointer to that object is even less meaningful if you send that pointer to another computer- the receiving computer has no (practical, trivial) way to read the memory that the pointer points to in the sending computer.
Since you seem to be having problems with this part of the process, let me make a recommendation that will save you countless hours of debugging some extremely difficult implementation bugs you're bound to run in to:
Abandon this entire idea of trying to send raw binary data between machines and just send simple ASCII/UTF-8 formatted information between them.
If you think that this is some how going to be slow, or inefficient, then let me recommend that you bring every thing up using a simplified ASCII/UTF-8 stringified version first. Trust me, debugging raw binary data is no fun, and the ability to just NSLog(#"I got: %#", dataString) is worth its weight in gold when you're debugging your inevitable problems. Then, once everything has gelled, and you're confident that you don't need to make any more changes to what it is you need to exchange, "port" (for lack of a better word) that implementation to a binary only version if, and only if, profiling with Shark.app identifies it as a problem area. As a point of reference, these days I can scp a file between machines and saturate a gigabit link with the transfer. scp probably has to do about five thousand times as much processing per byte to compress and encrypt the data than this simple stringification all while transferring 80MB/sec. Yet on modern hardware this is barely enough to budge the CPU meter running in my menu bar.