I'm studying Cocoa collections currently and my research has brought to Mike Ash's post on object equality and hashing.
Here's an exerpt from the post:
Because of the semantics of hash, if you override isEqual: then you must override hash. If you don't, then you risk having two objects which are equal but which don't have the same hash. If you use these objects in a dictionary, set, or something else which uses a hash table, then hilarity will ensue.
Unfortunately the author doesn't get further in details of what the hilarity will occur and my curiosity doesn't let me just leave it without trying to dig deeper. So the question is: what exactly will happen if i have two equal objects with different hash values and i put these objects into one collection? What sort of problem i will run into?
The answer is in this section from Mike's post
A hash table is basically a big array with special indexing. Objects are placed into an array with an index that corresponds to their hash. The hash is essentially a pseudorandom number generated from the object's properties. The idea is to make the index random enough to make it unlikely for two objects to have the same hash, but have it be fully reproducible. When an object is inserted, the hash is used to determine where it goes. When an object is looked up, its hash is used to determine where to look.
In more formal terms, the hash of an object is defined such that two objects have an identical hash if they are equal. Note that the reverse is not true, and can't be: two objects can have an identical hash and not be equal. You want to try to avoid this as much as possible, because when two unequal objects have the same hash (called a collision) then the hash table has to take special measures to handle this, which is slow. However, it's provably impossible to avoid it completely.
What it means is that you will have your 2 objects which claim to be equal. You add the first as the key in a dictionary with some value. Then you try to extract that value using the other object as the key. And it doesn't work. It should, because your objects are equal. But the initial hash lookup failed.
To be clear, this might not happen. It might work fine for some objects and fail for others. The point is, if you don't implement both methods, you don't know what's going to happen.
Putting aside the desire to know "why", you should just look at Apple's documentation.
http://developer.apple.com/library/mac/#documentation/Cocoa/Reference/Foundation/Protocols/NSObject_Protocol/Reference/NSObject.html%23//apple_ref/occ/intfm/NSObject/isKindOfClass:
If two objects are equal, they must have the same hash value.
All other discussion is interesting from an academic perspective, but fundamentally whether you agree with Apples rules or not, you must abide by them if you want to use the Foundation frameworks.
What Mike and the above poster say seem to be true, for the current incarnation of NSDictionary - there is no guarantee that the same implementation will remain in-place for future releases. However, whatever Apple might replace it with, it will (probably) retain all of the same guarantees and restrictions.
Why should someone ever use the non-NSMutable equivalents of the data structures in Objective-C? When it's a situation when you need a const object that should not be modified? Does using non-NSMutable classes improve performance in any way? Any other situations?
The two main reasons off the top of my head:
An object returning a property can be certain nobody will alter it if it's immutable. The object can therefore return the original instead of making copies all the time. So it's a memory and performance benefit.
When writing your own immutable objects, it's very easy to be thread safe. That naturally flows into being able to write multi-threaded functional-style code which is reasonably efficient and error free.
You also tend to see arguments in favour of the inherent preservation of the original value being useful, especially in terms of semantics and design patterns.
Immutable classes don't tend to be much more efficient in and of themselves with one exception — if you take an immutable copy of a mutable array, for example, then it's clear exactly how much storage is needed and exactly that much can be allocated. Because memory allocation costs time, mutable collections tend to keep some spare storage around because they can't predict how they're going to grow.
const is not directly related to non-mutable objects; I'm more familiar with the latter, so that's what I'll talk about.
A non-mutable object is like a reservation. Imagine that you work at a busy restaurant that only works on a reservation basis—all guests must make a reservation. When someone calls and makes a reservation for eight people at six, you know that you'll be expecting eight people at 6. Of course, this keeps things predictable. You know to set out one table that can sit eight people (it wouldn't make sense to use more than one table, especially at a busy restaurant). You notify the kitchen and tell them to expect eight orders a few minutes after six (okay, maybe you won't, but you might as well). In this way, everything runs smoothly and there are no delays. When the party of eight arrives promptly at six (because everyone in this world is perfectly punctual), you lead them right over to their seats, they order, and enjoy their meal. No problems whatsoever.
A problem arises if the reservation never specifies the number of people or the time. Imagine someone calls and tells you to expect a group of people for dinner. In this case, you have no information. A group could be a couple on a date, a four-person family, or two dozen people for a corporate function. They might arrive late because they were at a movie, really early because they have a young child, or at different times because it was impossible to coordinate everyone. In this case, you would have to scramble to find seating for everyone and the kitchens might suddenly be swamped with a large number of orders. Or you could have blocked off to many seats and the kitchen might find itself with nothing to do. In either case, where you over-estimate or under-estimate, there are delays and lost potential. Anything could happen.
In this metaphor, the restaurant would be the runtime system, and the reservations are the objects. In the first scenario, you have a non-mutable object, like an NSArray. The system knows how much data it'll hold, how many elements there are, and by runtime, what type they are. The system knows that the size won't change, so it can optimize RAM to go around that array, without leaving any precautionary bits. Everything runs smoothly because everything is known.
By contrast, nothing is known with an NSMutableArray. The user might add more elements, so the system has to scramble to find more RAM, rather than using those same clock cycles to crunch some operation; the user might replace an element in the middle with a larger one, having to offset all the later elements—which involves copying all tho elements after. In certain cases, it could involve copying all the elements of the array or string or whatever to a new location, a (potentially) expensive operation. This can impart a significant performance overhead, especially when you use a lot of them. In Java for example, concatenating a string involves copying the entire existing string to a new memory location, and leaving the garbage collector to deal with the old string.
Another compelling reason is that you make it a bit harder to change the data. Users (of the class) have to explicitly make a mutable copy, which helps to ensure that they know what they're doing. This advantage is particularly notable with multiple threads—you don't want to pass a mutable object to something that's running on a background thread, because the foreground thread (or any other) could then be modifying the object, as it's being modified by the original thread, leading to very interesting results.
I'm refactoring a project that involves passing around a lot of arrays. Currently, each method that returns an array sorts it right before returning it. This isn't ideal for a couple reasons -- there's lots of duplicated code, it's inefficient to sort an array two or three times, and it's too easy to write a new function but to forget to sort the array before returning it.
I'm looking for a way to guarantee that the array always kept in alphabetical order. My current thought is to subclass NSMutableArray and/or NSArray to create an alphabetized array class. I would need to override all of the methods that create or modify the array to call super and then sort itself.
Does this sound reasonable, or is there a better approach?
EDIT:
Since performance issues have been mentioned, I'll include the relevant information from my project. Speed is not an important concern. The whole process only takes a few seconds, and the tool is only used every so often. So simplicity and obvious correctness is more important.
Also, the use case for arrays is specific. When an array is returned, the caller always accesses every element in the array at least once.
A balanced binary tree is the standard and efficient way to keep items sorted. Almost any way to do random access with a plain array will be slow. A skip list is also efficient and you may be able to add the functionality to the array class.
Check out CHDataStructures. It's a framework that has a lot of self-sorting datastructures, like balanced binary trees and whatnot.
Note: Although my particular context is Objective-C, my question actually transcends programming language choice. Also, I tagged it as "subjective" since someone is bound to complain otherwise, but I personally think it's almost entirely objective. Also, I'm aware of this related SO question, but since this was a bigger issue, I thought it better to make this a separate question. Please don't criticize the question without reading and understanding it fully. Thanks!
Most of us are familiar with the dictionary abstract data type that stores key-value associations, whether we call it a map, dictionary, associative array, hash, etc. depending on our language of choice. A simple definition of a dictionary can be summarized by three properties:
Values are accessed by key (as opposed to by index, like an array).
Each key is associated with a value.
Each key must be unique.
Any other properties are arguably conveniences or specializations for a particular purpose. For example, some languages (especially scripting languages such as PHP and Python) blur the line between dictionaries and arrays and do provide ordering for dictionaries. As useful as this can be, such additions are not a fundamental characteristics of a dictionary. In a pure sense, the actual implementation details of a dictionary are irrelevant.
For my question, the most important observation is that the order in which keys are enumerated is not defined — a dictionary may provide keys in whatever order it finds most convenient, and it is up to the client to organize them as desired.
I've created custom dictionaries that impose specific key orderings, including natural sorted order (based on object comparisons) and insertion order. It's obvious to name the former some variant on SortedDictionary (which I've actually already implemented), but the latter is more problematic. I've seen LinkedHashMap and LinkedMap (Java), OrderedDictionary (.NET), OrderedDictionary (Flash), OrderedDict (Python), and OrderedDictionary (Objective-C). Some of these are more mature, some are more proof-of-concept.
LinkedHashMap is named according to implementation in the tradition of Java collections — "linked" because it uses a doubly-linked list to track insertion order, and "hash" because it subclasses HashMap. Besides the fact that user shouldn't need to worry about that, the class name doesn't really even indicate what it does. Using ordered seems like the consensus among existing code, but web searches on this topic also revealed understandable confusion between "ordered" and "sorted", and I feel the same. The .NET implementation even has a comment about the apparent misnomer, and suggests that it should be "IndexedDictionary" instead, owing to the fact that you can retrieve and insert objects at a specific point in the ordering.
I'm designing a framework and APIs and I want to name the class as intelligently as possible. From my standpoint, indexed would probably work (depending on how people interpret it, and based on the advertised functionality of the dictionary), ordered is imprecise and has too much potential for confusion, and linked "is right out" (apologies to Monty Python). ;-)
As a user, what name would make the most sense to you? Is there a particular name that says exactly what the class does? (I'm not averse to using slightly longer names like InsertionOrderDictionary if appropriate.)
Edit: Another strong possibility (discussed in my answer below) is IndexedDictionary. I don't really like "insertion order" because it doesn't make sense if you allow the user to insert keys at a specific index, reorder the keys, etc.
I vote OrderedDictionary, for the following reasons:
"Indexed" is never used in Cocoa classes, except in one instance. It always appears as a noun (NSIndexSet, NSIndexPath, objectAtIndex:, etc). There is only one instance when "Index" appears as a verb, which is on NSPropertyDescription's "indexed" property: isIndexed and setIndexed. NSPropertyDescription is roughly analogous to a table column in a database, where "indexing" refers to optimizing to speed up search times. It would therefore make sense that with NSPropertyDescription being part of the Core Data framework, that "isIndexed" and "setIndexed" would be equivalent to an index in a SQL database. Therefore, to call it "IndexedDictionary" would seem redundant, since indices in databases are created to speed up lookup time, but a dictionary already has O(1) lookup time. However, to call it "IndexDictionary" would also be a misnomer, since an "index" in Cocoa refers to position, not order. The two are semantically different.
I understand your concern over "OrderedDictionary", but the precedent has already been set in Cocoa. When users want to maintain a specific sequence, they use "ordered": -[NSApplication orderedDocuments], -[NSWindow orderedIndex], -[NSApplication orderedWindows], etc. So, John Pirie has mostly the right idea.
However, you don't want to make insertion into the dictionary a burden on your users. They'll want to create a dictionary once and then have it maintain an appropriate order. They won't even want to request objects in a specific order. Order specification should be done during initialization.
Therefore, I recommend making OrderedDictonary a class cluster, with private subclasses of InsertionOrderDictionary and NaturalOrderDictionary and CustomOrderDictionary. Then, the user simply creates an OrderedDictionary like so:
OrderedDictionary * dict = [[OrderedDictionary alloc] initWithOrder:kInsertionOrder];
//or kNaturalOrder, etc
For a CustomOrderDictionary, you could have them give you a comparison selector, or even (if they're running 10.6) a block. I think this would provide the most flexibility for future expansion while still maintain an appropriate name.
I vote for InsertionOrderDictionary. You nailed it.
Strong vote for OrderedDictionary.
The word "ordered" means exactly what you are advertising: that in iterating through a list of items, there is a defined order to selection of those items. "Indexed" is an implementation word -- it talks more to how the ordering is achieved. Index, linked list, tree... the user doesn't care; that aspect of the data structure should be hidden. "Ordered" is the exact word for the additional feature you are offering, regardless of how you get it done.
Further, it seems like the choice of ordering could be at the user's option. Any reason why you couldn't create methods on your datatype that allow the user to switch from, say, alphabetical ordering to insertion-time ordering? In the default case, a user would choose a particular ordering and stick with it, in which case implementation would be no less efficient than if you created specialized subclasses for each ordering method. And in some less-used cases, the developer might actually wish to use any of a number of different orderings for the same data, depending on app context. (I can think of specific projects I've worked on where I would have loved to have such a data structure available.)
Call it OrderedDictionary, because that's precisely what it is. (Frankly, I have more of a problem with the use of the word "Dictionary", because that word heavily implies ordering, where popular implementations of such don't provide it, but that's my pet peeve. You really should just be able to say "Dictionary" and know that the ordering is alphabetical -- because that's what a dictionary IS -- but that argument is too late for existing implementations in the popular languages.) And allow the user to access in what order he chooses.
Since posting this question, I'm starting to lean towards something like IndexedDictionary or IndexableDictionary. While it is useful to be able to maintain arbitrary key ordering, limiting that to insertion ordering only seems like a needless restriction. Plus, my class already supports indexOfKey: and keyAtIndex:, which are (purposefully) analagous to NSArray's indexOfObject: and objectAtIndex:. I'm strongly considering adding insertObject:forKey:atIndex: which matches up with NSMutableArray's insertObject:atIndex:.
Everyone knows that inserting in the middle of an array is inefficient, but that doesn't mean we shouldn't be allowed to on the rare occasions that it's truly useful. (Besides, the implementation could secretly use a doubly-linked list or any other suitable structure for tracking the ordering if needed...)
The big question: is "indexed" or "indexable" as vague or potentially confusing as "ordered"? Would people think of database indexes, or book indexes, etc.? Would it be detrimental if they assumed it was implemented with an array, or might that simplify user understanding of the functionality?
Edit: This name makes even more sense given the fact that I'm considering adding methods that work with an NSIndexSet in the future. (NSArray has -objectsAtIndexes: as well as methods for adding/removing observers for objects at given indexes.)
What about KeyedArray?
As you said in your last paragraph, I think that InsertionOrder(ed)Dict(ionary) is pretty unambiguous; I don't see how it could be interpreted in any way other than that the keys would be returned in the order they were inserted.
By decoupling the indexed order from the insertion order, doesn't this simply boil down to keeping an array and Dictionary in a single object? I guess my vote for this type of object is IndexedKeyDictionary
In C#:
public class IndexedKeyDictionary<TKey, TValue> {
List<TKey> _keys;
Dictionary<TKey, TValue> _dictionary;
...
public GetValueAtIndex(int index) {
return _dictionary[_keys[index]];
}
public Insert(TKey key, TValue val, int index) {
_dictionary.Add(key, val);
// do some array massaging (splice, etc.) to fit the new key
_keys[index] = key;
}
public SwapKeyIndexes(TKey k1, TKey k2) {
// swap the indexes of k1 and k2, assuming they exist in _keys
}
}
What would be really cool is indexed values...so we have a way to sort the values and get the new key order. Like if the values were graph coordinates, and we could read the keys (bin names) as we move up/down along the coordinate plane. What would you call that data structure? An IndexedValueDictionary?
At first glance I'm with the first reply -- InsertionOrderDictionary, though it's a bit ambiguous as to what "InsertionOrder" means at first glance.
What you're describing sounds to me almost exactly like a C++ STL map. From what I understand, a map is a dictionary that has additional rules, including ordering. The STL simply calls it "map", which I think is fairly apt. The trick with map is you can't really give the inheritance a nod without making it redundant -- i.e. "MapDictionary". That's just too redundant. "Map" is a bit too basic and leaves a lot of room for misinterpretation.
Though "CHMap" might not be a bad choice after looking at your documentation link.
Maybe "CHMappedDictionary"? =)
Best of luck.
Edit: Thanks for the clarification, you learn something new every day. =)
Is the only difference that allKeys returns keys in a specific order? If so, I would simply add allKeysSorted and allKeysOrderdByInsertion methods to the standard NSDictionary API.
What is the goal of this insertion order dictionary? What benefits does it give the programmer vs. an array?
This is really a three-part question, but I've answered the first question myself:
I'm working on the iPhone, with many objects (up to 200) on the screen. Each object needs to look if it's overlapping other objects, and act accordingly. My original naive implementation was for each object to run through the list of each other object to check their bounding boxes (using CGRectInsersectsRect).
So my question (and answer) is what's a better method? My new implementation is to sort the array every frame using an insertion sort (since the data will be mostly sorted already) on the y-position of each object, then check only the nearest object on either side of the one searching, to see if it's in range vertically, then check horizontally.
First question: Is insertion sort the method I want to use for an array of objects that tend to move around randomly but only to a small extent so they mostly stay in order based on the last frame? Also: what sort algorithm does the NSArray use when I call
- sortedArrayUsingSelector:
I would sort of assume that it uses a quick sort, since it's the most useful in the general case. Does anybody know if I'm wrong? Does anybody know if I can change the sort method or if I will have to write my own sorting function?
Second question: is there a function for retrieving items from a sorted array using a binary search, rather than the naive approach that I assume is used by
- indexOfObject:
or would I have to write my own?
NSArray uses many different data structures internally depending on how many objects are in the array. See a Peter Ammon blog entry for more information. But basically this means you can't expect a certain kind of sort to happen. Sometimes it is worth it to write your own array implementation using C arrays so you can control the sorts yourself.
There are definitely much faster ways to implement collision detection. Look into Bounding Volume Hierarchies like KD Trees or similar.
As far as I know indexOfObject: is the only way, but it's potentially not as dumb as you think. Everything is hashable for NSDictionary so they can use some of those smarts in NSArray.