I have a very peculiar problem here. I'm building a spider to grab hyperlinks from a webpage and put them into a table and I'm using NSRanges to parse the HTML document, but I've run into an issue.
I have the following line of code:
NSLog(#"%lu", [dataString rangeOfString:#"contents.asp?year1" options:0 range:NSMakeRange(index, dataString.length - index)].length);
This echoes 18 to the log, as it should, but if I put that into a boolean statement, seeing if that length is greater than -1:
NSLog(#"%d", ([dataString rangeOfString:#"contents.asp?year1" options:0 range:NSMakeRange(index, dataString.length - index)].length > -1));
This echoes 0, or false. 18 is clearly greater than -1, so what's the problem? If I switch it to < -1, it returns true. Does this have something to do with type-casting the unsigned long?
Here's the definition of NSRange:
typedef struct _NSRange {
NSUInteger location;
NSUInteger length;
} NSRange;
Notice that both fields are of type NSUInteger, an unsigned type. In fact, NSUInteger is unsigned long.
Since there is no wider integer type than unsigned long, the compiler promotes -1 to unsigned. I can't recall whether this is undefined behavior, but on iOS and Mac OS X it has the effect of treating the 2's complement bit pattern of -1 as an unsigned integer. That bit pattern, as an unsigned integer, is the maximum unsigned integer value.
Thus your comparison can never be true.
If you think -1 means "not found", you are mistaken. The correct way to check whether rangeOfString:options:range: failed to find the target is to check whether the location of the returned range is NSNotFound:
NSUInteger location = [dataString rangeOfString:#"contents.asp?year1"
options:0 range:NSMakeRange(index, dataString.length - index)].location
BOOL foundIt = location != NSNotFound;
I have function to convert an integer into byte array (for iPhone). To add dynamicity I have allocate the array using malloc. But I think this will leak memory. What's best way to manage this memory,
+ (unsigned char *) intToByteArray:(int)num{
unsigned char * arr = (unsigned char *)
malloc(sizeof(num) * sizeof(unsigned char));
for (int i = sizeof(num) - 1 ; i >= 0; i --) {
arr[i] = num & 0xFF;
num = num >> 8;
}
return arr;
}
When calling,
int x = 500;
unsigned char * bytes = [Util intToByteArray:x];
I want to avoid the call free(bytes) since, the calling function do not know or explicitly knows, the memory is allocated and not freed.
A few things:
The char type (and signed char and unsigned char) all have a size of 1 by definition, so sizeof(unsigned char) is unnecessary.
It looks like you just want to get the byte representation of an int object, if this is the case, it is not necessary to allocate more space for it, simply take the address of the int and cast it to a pointer to unsigned char *. If the byte order is wrong you can use the NSSwapInt function to swap the order of the bytes in the int and then take the address and cast to unsigned char *. For example:
int someInt = 0x12345678;
unsigned char *bytes = (unsigned char *) &someInt;
This cast is legal and reading from bytes is legal up until sizeof(int) bytes are read. This is accessing the “object representation”.
If you insist on using malloc, then you simply need to pass the buffer to free when you are done, as in:
free(bytes);
The name of your method does not imply the correct ownership of the returned buffer. If your method returns something that the caller is responsible for freeing, it is conventional to name the method using new, copy, or sometimes create. A more suitable name would be copyBytesFromInt: or something similar. Otherwise you could have the method accept a pre-allocated buffer and call the method getBytes:fromInt:, for example:
+ (void) getBytes:(unsigned char *) bytes fromInt:(int) num
{
for (int i = sizeof(num) - 1 ; i >= 0; i --) {
bytes[i] = num & 0xFF;
num = num >> 8;
}
}
You could wrap your bytes into a NSData instance:
NSData *data = [NSData dataWithBytesNoCopy:bytes length:sizeof(num) freeWhenDone:YES];
Make sure your method follows the usual object ownership rules.
Just call free(bytes); when you are done with the bytes (either at the end of method or in dealloc of the class)
since you want to avoid the free call, you could wrap your byte[] in a NSData object:
NSData *d = [NSData dataWithBytesNoCopy:bytes length:num freeWhenDone:YES];
The conventional way of handling this is for the caller to pass in an allocated byte buffer. That way the caller is responsible for freeing it. Something like:
int x = 500;
char *buffer = malloc(x * sizeof(char));
[Util int:x toByteArray:buffer];
…
free(buffer);
I would also consider creating an NSData to hold the bytes, this would take care of memory management for you, while still allowing you to alter the byte buffer:
+ (NSData *) intToByteArray:(int)num {
unsigned char * arr = (unsigned char *)
malloc(sizeof(num) * sizeof(unsigned char));
for (int i = sizeof(num) - 1 ; i >= 0; i --) {
arr[i] = num & 0xFF;
num = num >> 8;
}
return [NSData dataWithBytesNoCopy:arr length:num freeWhenDone:YES];
}
I need to do bit operations on representations of arbitrary precision numbers in Objective C. So far I have been using NSData objects to hold the numbers - is there a way to bit shift the content of those? If not, is there a different way to achieve this?
Using NSMutableData you can fetch the byte in a char, shift your bits and replace it with -replaceBytesInRange:withBytes:.
I don't see any other solution except for writing your own date holder class using a char * buffer to hold the raw data.
As you'll have spotted, Apple doesn't provide arbitrary precision support. Nothing is provided larger than the 1024-bit integers in vecLib.
I also don't think NSData provides shifts and rolls. So you're going to have to roll your own. E.g. a very naive version, which may have some small errors as I'm typing it directly here:
#interface NSData (Shifts)
- (NSData *)dataByShiftingLeft:(NSUInteger)bitCount
{
// we'll work byte by byte
int wholeBytes = bitCount >> 3;
int extraBits = bitCount&7;
NSMutableData *newData = [NSMutableData dataWithLength:self.length + wholeBytes + (extraBits ? 1 : 0)];
if(extraBits)
{
uint8_t *sourceBytes = [self bytes];
uint8_t *destinationBytes = [newData mutableBytes];
for(int index = 0; index < self.length-1; index++)
{
destinationBytes[index] =
(sourceBytes[index] >> (8-extraBits)) |
(sourceBytes[index+1] << extraBits);
}
destinationBytes[index] = roll >> (8-extraBits);
}
else
/* just copy all of self into the beginning of newData */
return newData;
}
#end
Of course, that assumes the number of bits you want to shift by is itself expressible as an NSUInteger, amongst other sins.
If I have an unsigned char *data pointer and I want to check whether size_t length of the data at that pointer is NULL, what would be the fastest way to do that? In other words, what's the fastest way to make sure a region of memory is blank?
I am implementing in iOS, so you can assume iOS frameworks are available, if that helps. On the other hand, simple C approaches (memcmp and the like) are also OK.
Note, I am not trying to clear the memory, but rather trying to confirm that it is already clear (I am trying to find out whether there is anything at all in some bitmap data, if that helps). For example, I think the following would work, though I have not tried it yet:
- BOOL data:(unsigned char *)data isNullToLength:(size_t)length {
unsigned char tester[length] = {};
memset(tester, 0, length);
if (memcmp(tester, data, length) != 0) {
return NO;
}
return YES;
}
I would rather not create a tester array, though, because the source data may be quite large and I'd rather avoid allocating memory for the test, even temporarily. But I may just being too conservative there.
UPDATE: Some Tests
Thanks to everyone for the great responses below. I decided to create a test app to see how these performed, the answers surprised me, so I thought I'd share them. First I'll show you the version of the algorithms I used (in some cases they differ slightly from those proposed) and then I'll share some results from the field.
The Tests
First I created some sample data:
size_t length = 1024 * 768;
unsigned char *data = (unsigned char *)calloc(sizeof(unsigned char), (unsigned long)length);
int i;
int count;
long check;
int loop = 5000;
Each test consisted of a loop run loop times. During the loop some random data was added to and removed from the data byte stream. Note that half the time there was actually no data added, so half the time the test should not find any non-zero data. Note the testZeros call is a placeholder for calls to the test routines below. A timer was started before the loop and stopped after the loop.
count = 0;
for (i=0; i<loop; i++) {
int r = random() % length;
if (random() % 2) { data[r] = 1; }
if (! testZeros(data, length)) {
count++;
}
data[r] = 0;
}
Test A: nullToLength. This was more or less my original formulation above, debugged and simplified a bit.
- (BOOL)data:(void *)data isNullToLength:(size_t)length {
void *tester = (void *)calloc(sizeof(void), (unsigned long)length);
int test = memcmp(tester, data, length);
free(tester);
return (! test);
}
Test B: allZero. Proposal by Carrotman.
BOOL allZero (unsigned char *data, size_t length) {
bool allZero = true;
for (int i = 0; i < length; i++){
if (*data++){
allZero = false;
break;
}
}
return allZero;
}
Test C: is_all_zero. Proposed by Lundin.
BOOL is_all_zero (unsigned char *data, size_t length)
{
BOOL result = TRUE;
unsigned char* end = data + length;
unsigned char* i;
for(i=data; i<end; i++) {
if(*i > 0) {
result = FALSE;
break;
}
}
return result;
}
Test D: sumArray. This is the top answer from the nearly duplicate question, proposed by vladr.
BOOL sumArray (unsigned char *data, size_t length) {
int sum = 0;
for (int i = 0; i < length; ++i) {
sum |= data[i];
}
return (sum == 0);
}
Test E: lulz. Proposed by Steve Jessop.
BOOL lulz (unsigned char *data, size_t length) {
if (length == 0) return 1;
if (*data) return 0;
return memcmp(data, data+1, length-1) == 0;
}
Test F: NSData. This is a test using NSData object I discovered in the iOS SDK while working on all of these. It turns out Apple does have an idea of how to compare byte streams that is designed to be hardware independent.
- (BOOL)nsdTestData: (NSData *)nsdData length: (NSUInteger)length {
void *tester = (void *)calloc(sizeof(void), (unsigned long)length);
NSData *nsdTester = [NSData dataWithBytesNoCopy:tester length:(NSUInteger)length freeWhenDone:NO];
int test = [nsdData isEqualToData:nsdTester];
free(tester);
return (test);
}
Results
So how did these approaches compare? Here are two sets of data, each representing 5000 loops through the check. First I tried this on the iPhone Simulator running on a relatively old iMac, then I tried this running on a first generation iPad.
On the iPhone 4.3 Simulator running on an iMac:
// Test A, nullToLength: 0.727 seconds
// Test F, NSData: 0.727
// Test E, lulz: 0.735
// Test C, is_all_zero: 7.340
// Test B, allZero: 8.736
// Test D, sumArray: 13.995
On a first generation iPad:
// Test A, nullToLength: 21.770 seconds
// Test F, NSData: 22.184
// Test E, lulz: 26.036
// Test C, is_all_zero: 54.747
// Test B, allZero: 63.185
// Test D, sumArray: 84.014
These are just two samples, I ran the test many times with only slightly varying results. The order of performance was always the same: A & F very close, E just behind, C, B, and D. I'd say that A, F, and E are virtual ties, on iOS I'd prefer F because it takes advantage of Apple's protection from processor change issues, but A & E are very close. The memcmp approach clearly wins over the simple loop approach, close to ten times faster in the simulator and twice as fast on the device itself. Oddly enough, D, the winning answer from the other thread performed very poorly in this test, probably because it does not break out of the loop when it hits the first difference.
I think you should do it with an explicit loop, but just for lulz:
if (length == 0) return 1;
if (*pdata) return 0;
return memcmp(pdata, pdata+1, length-1) == 0;
Unlike memcpy, memcmp does not require that the two data sections don't overlap.
It may well be slower than the loop, though, because the un-alignedness of the input pointers means there probably isn't much the implementation of memcmp can do to optimize, plus it's comparing memory with memory rather than memory with a constant. Easy enough to profile it and find out.
Not sure if it's the best, but I probably would do something like this:
bool allZero = true;
for (int i = 0; i < size_t; i++){
if (*data++){
//Roll back so data points to the non-zero char
data--;
//Do whatever is needed if it isn't zero.
allZero = false;
break;
}
}
If you've just allocated this memory, you can always call calloc rather than malloc (calloc requires that all the data is zeroed out). (Edit: reading your comment on the first post, you don't really need this. I'll just leave it just in case)
If you're allocating the memory yourself, I'd suggest using the calloc() function. It's just like malloc(), except it zeros out the buffer first. It's what's used to allocate memory for Objective-C objects and is the reason that all ivars default to 0.
On the other hand, if this is a statically declared buffer, or a buffer you're not allocating yourself, memset() is the easy way to do this.
Logic to get a value, check it, and set it will be at least as expensive as just setting it. You want it to be null, so just set it to null using memset().
This would be the preferred way to do it in C:
BOOL is_all_zero (const unsigned char* data, size_t length)
{
BOOL result = TRUE;
const unsigned char* end = data + length;
const unsigned char* i;
for(i=data; i<end; i++)
{
if(*i > 0)
{
result = FALSE;
break;
}
}
return result;
}
(Though note that strictly and formally speaking, a memory cell containing a NULL pointer mustn't necessarily be 0, as long as a null pointer cast results in the value zero, and a cast of a zero to a pointer results in a NULL pointer. In practice, this shouldn't matter as all known compilers use 0 or (void*) 0 for NULL.)
Note the edit to the initial question above. I did some tests and it is clear that the memcmp approach or using Apple's NSData object and its isEqualToData: method are the best approaches for speed. The simple loops are clearer to me, but slower on the device.
Using foundation and cocoa frameworks on Mac, I am trying to convert an NSData object in humanly understandable number.
Let say the NSData object is an image of NPIXEL. I know the binary data are coded in big endian and represent 32 bit integer (to be more precise 32 bit two complements integer). I write the piece of code bellow to convert the NSData into an int array. But the value I got are completely wrong (this does not means the measurement are bad, I used a special software to read the data and the value given by the software are different from the one I got with my code).
-(int *) GetArrayOfLongInt
{
//Get the total number of element into the Array
int Nelements=[self NPIXEL];
//CREATE THE ARRAY
int array[Nelements];
//FILL THE ARRAY
int32_t intValue;
int32_t swappedValue;
double Value;
int Nbit = abs(BITPIX)*GCOUNT*(PCOUNT + Nelements); Nbit/=sizeof(int32_t);
int i=0;
int step=sizeof(int32_t);
for(int bit=0; bit < Nbit; bit+=step)
{
[Img getBytes:&swappedValue range:NSMakeRange(bit,step)];
intValue= NSSwapBigIntToHost(swappedValue);
array[i]=intValue;
i++;
}
return array;
}
This piece of code (with minor change) work perfectly when the binary data represent float or double, but I dont when it is 16,32 or 64 bit integer. I also tried changingNSSapBigIntToHostintoNSSwapLittleInttoHost`. I even tried with long, but the results is still the same, I got bad values. What wrong I am doing ?
PS: Some of the variable in my code are already set elsewhere in my program. BITPIX is the bit size of each pixel. In this case 32. GCOUNT is equal to 1, PCOUNT 0 and Nelements is the total number of pixel I should have in my image.
Returning a pointer to a local variable is a very bad idea. array could get overwritten at any time (or if you were to write through the pointer, you could corrupt the stack). You probably want something like:
// CREATE THE ARRAY
int *array = malloc(Nelements * sizeof(int));
Your algorithm seems a bit overkill, too. Why not just copy out the whole array from the NSData object, and then byteswap the entries in place? Something like:
int32_t length = [Img length];
int32_t *array = malloc(length);
[Img getBytes:array length:length];
for (i = 0; i < length/sizeof(int32_t); i++)
{
array[i] = NSSwapBigIntToHost(array[i]);
}