I am working on an applet which has to share some keys of type AESKey with different terminals. The thing is I don't know in advance how many terminals it will have to handle.
As there is no structure like HashTable in Java Card, it's getting complicated. I can still fix an upper bound and instanciate as much objects AESKey but I would like to search for another way to do.
I thought I could do something with byte arrays, but is it a bad practice to store keys in byte[]?
I think the answer is yes and it is only recommanded to store it in transient arrays to make computations. Otherwise, I don't understand the role of AESKey objects. Just want to be sure.
Important security-relevant data like keys and PINs shall always be stored in the therefore designated objects from the Javacard API, e.g. AESKey. The smartcard operating system will perform additional internal operations to protect there values from leaking. If you don't know how many terminals the card will encounter you could encapsulate the Keys in an Object which is part of a linked list:
class KeyElement{
KeyElement next;
AESKey key;
}
Technically, it is possible to store key values in a byte[] with some 'unknown level of security' by using the following scheme:
Store only wrapped (i.e. encrypted) values of the key in the persistent byte array using some persistent wrapping key.
Prior to the key use, unwrap the desired key using the same wrapping key into a transient key object. Then use it at will.
Advantage: Probably more memory efficient than the 'many AESKey objects approach'.
Drawback: It is quite weird. I would do my best not to implement it this way.
Desclaimer: I am no crypto expert, so please do validate my thoughts.
Desclaimer 2: Of course the most reasonable way is to use key derivation as Maarten Bodewes noted...
In fact, creating AESKey array is possible in Java Card. I thought that only byte arrays (byte[]) were authorized but no.
So nothing forbids me to declare an AESKey array (AESKey[]) if I consider that I have to fix an upperbound to limit the number of keys in my applet.
Related
There is one question that I often ask myself while designing a program, and I am never quite sure how to answer it.
Let's say I have an object with multiple fields, amongst which there is one serving as the identifier to that specific object. Let's also say that I need to keep track of a List of such objects somewhere else.
I now have three, and probably even more, options on how to go about it:
Have my object contain its own identifier, and all its other fields. I now use a simple array (or whatever simple list collection) of my objects where I need it. When I am looking for one specific object, I loop through my list and check for identifier equality.
Pros: 1. "Clarity" for each object instance. 2.?
Cons: Manipulating a collection of these objects gets annoying
Have my object contain all fields beside its identifier. I now use a Map with identifier as key, and object as value. When looking for one specific object, I just lookup the identifier in the map.
Pros: easy lookups and insertions,?
Cons: object instance itself doesnt know what it is,?
Combination of both: use a map with identifier as key and object having its own identifier as a field as value.
Pros: mentioned above.
Cons: looks redundant to me.
What situations would call for what? Let's use the standard hello-world example of networking for example, a chat server: how would I handle multiple "groups/channels" people are in?
What about other applications?
Your question is very wide and, actually, contains two questions.
First is “Which data structure is better — dictionary or list?”. The answer is: it depends on performance you want to achieve on insertion and search operations. Basically if you need to look through the collection, then list is ok, and if you need to have fast look-up, then dictionary is better. Dictionary has more memory overhead than list.
The second is “Do I need to have an Id field inside an entity or can I use built in hash code?”. The answer is: it depends on how you will use your object. If you want Id just to store it in a dictionary, then, most likely, you can go with hash code. There is nothing wrong with storing Id of an entity inside that entity. Either you use Id or hash code, you need to be sure that this entity will be uniquely identified by id or hash. That's the main concern with it.
You can override GetHashCode method and make it return Id of your entity. Sometimes you can find such implementation when hash code is required for collection and Id is required for database.
So, it really doesn't matter what you will choose in the end if both approaches are working for you right now.
A map<Identifier, Object> will offer you O(1) performance when retrieving an object based on its identifier. There certainly are situations where you want to achieve that.
However, in other cases it might be redundant to use this approach. It all depends on the situation at hand.
Two guidelines may answer this question:
A use case that calls for a lookup where there is an expectation of a 1:1 relationship between the key and value implies a Map structure.
OOP implies that a key which is so closely related to an object as to preform a lookup should be encapsulated within that object.
Regarding the question of redundancy, consider the key in a map is nothing but an index. Indexes are as common in data as in books.
NSOrderedSet seems to be able to give O(1) lookup speed of hash-tables and array like ordering of objects? What is the data structure used to achieve this? Is it a combination of two, e.g: Hashtable and a separate Array where index i has the key corresponding to it's object in the Hashtable?
We don't know how Apple chose to implement this data structure. The only way to find out would be to reverse-engineer the Foundation framework. But this is not an useful thing to do, Apple could change the implementation and underlying data structure with every update. So relying on this for a production app would be very stupid since it could break the app at any time.
If you wanted to implement this yourself your approach with a hash table and an array will work. The best way would be to store the objects in the array and have the hash map store the array indices keyed by objects.
Of course there are other possible ways how this could be implemented with different performance characteristics. It could be just an array making the containsObject: test O(n), or it could just be a hash table (with the object as key and index as value) making the objectAtIndex: operation O(n).
Why do we need to use serialization?
If we want to send an object or piece of data through a network we can use streams of bytes. If we want to save some data to the disk, we can again use the binary mode along with the byte streams and save it.
So what's the advantage of using serialization?
Technically on the low-level, your serialized object will also end up as a stream of bytes on your cable or your filesystem...
So you can also think of it as a standardized and already available way of converting your objects to a stream of bytes. Storing/transferring object is a very common requirement, and it has less or little meaning to reinvent this wheel in every application.
As other have mentioned, you also know that this object->stream_of_bytes implementation is quite robust, tested, and generally architecture-independent.
This does not mean it is the only acceptable way to save or transfer an object: in some cases, you'll have to implement your own methods, for example to avoid saving unnecessary/private members (for example for security or performance reasons). But if you are in a simple case, you can make your life easier by using the serialization/deserialization of your framework, language or VM instead of having to implement it by yourself.
Hope this helps.
Quoting from Designing Data Intensive Applications book:
Programs usually work with data in (at least) two different
representations:
In memory, data is kept in objects, structs, lists, arrays, hash tables, trees, and so on. These data structures are optimized for
efficient access and manipulation by the CPU (typically using
pointers).
When you want to write data to a file or send it over the network, you have to encode it as some kind of self-contained sequence of bytes
(for example, a JSON document). Since a pointer wouldn’t make sense to
any other process, this sequence-of-bytes representation looks quite
different from the data structures that are normally used in memory.
Thus, we need some kind of translation between the two
representations. The translation from the in-memory representation to
a byte sequence is called encoding (also known as serialization or
marshalling), and the reverse is called decoding (parsing,
deserialization, unmarshalling).
Among other reasons to be compatible between architecture. An integer doesn't have the same number of bytes from one architecture to another, and sometimes from one compiler to another.
Plus what you're talking about is still serialization. Binary Serialization. You're putting all the bytes of your object together in order to store them and be able to reconvert them as an object later. This is serializing.
More info on wikipedia
Serialization is the process of converting an object into a stream so that it can be saved in any physical file like (XML) or can be saved in Database. The main purpose of Serialization in C# is to persist an object and save it in any specified storage medium like stream, physical file or DataBase.
In General, serialization is a method to persist an object's state, but i suggest you to read this wiki page, it is pretty detailed and correct in my opinion:
http://en.wikipedia.org/wiki/Serialization
In serialization, the point is not turning an object into bits and bytes, objects ARE bits and bytes already. Serialization is the process of making the object's "state" persistent. Notice the word "state", which means the values of the instance variables of the entire object graph (the target object and all the objects it references either directly or indirectly) WITHOUT methods and other extra runtime stuff stuck to them (and of course plus a little more info that JVM needs for restoration of these objects, such as their class types).
So this is the main reason of its necessity: Storing the whole bytes of objects would be expensive, and for all intents and purposes, unnecessary.
This may be overkill, but I am trying to reduce the network consumption of a client/server protocol, by having both sides keep copies of previously transferred URIs, so as to use 2-4 byte placeholders instead of the full URIs on subsequent chatter.
Problem is I think it will be quite expensive to reflect through all the complex objects being transferred to locate the URIs that need processing, whereas the serializer is already visiting all these fields and probably using a mechanism much faster than reflection.
Can this be done in protobuf-net?
If this is part of a single call to Serialize/Deserialize (i.e. your data has the same uri repeated at multiple locations), then you can already do this, simply by telling it to treat those strings as references (it has special handling of strings, so two different references of the same string contents count as equal):
[ProtoMember(7, AsReference=true)]
public string Uri {get;set;}
During serialization, the first time it spots a new string value (decorated with AsReference=true) it will generate a unique token to represent the string; all subsequent usages of that same string will serialize only the token.
If this is in separate calls to Serialize/Deserialize, then no: you would have to do it manually. I can think of some ways of doing it, but I think this would be better handled outside of the serialization layer.
Could you possibly customise the Objects that you are using that you want to Tokenise your URIs and have them inherit or implement an interface that you can check to see if that particular object is a Tokenizer.
Then if that's the case you might be able to use the BeforeSerialization / AfterDeserialization to make your transformations.
Note: Although my particular context is Objective-C, my question actually transcends programming language choice. Also, I tagged it as "subjective" since someone is bound to complain otherwise, but I personally think it's almost entirely objective. Also, I'm aware of this related SO question, but since this was a bigger issue, I thought it better to make this a separate question. Please don't criticize the question without reading and understanding it fully. Thanks!
Most of us are familiar with the dictionary abstract data type that stores key-value associations, whether we call it a map, dictionary, associative array, hash, etc. depending on our language of choice. A simple definition of a dictionary can be summarized by three properties:
Values are accessed by key (as opposed to by index, like an array).
Each key is associated with a value.
Each key must be unique.
Any other properties are arguably conveniences or specializations for a particular purpose. For example, some languages (especially scripting languages such as PHP and Python) blur the line between dictionaries and arrays and do provide ordering for dictionaries. As useful as this can be, such additions are not a fundamental characteristics of a dictionary. In a pure sense, the actual implementation details of a dictionary are irrelevant.
For my question, the most important observation is that the order in which keys are enumerated is not defined — a dictionary may provide keys in whatever order it finds most convenient, and it is up to the client to organize them as desired.
I've created custom dictionaries that impose specific key orderings, including natural sorted order (based on object comparisons) and insertion order. It's obvious to name the former some variant on SortedDictionary (which I've actually already implemented), but the latter is more problematic. I've seen LinkedHashMap and LinkedMap (Java), OrderedDictionary (.NET), OrderedDictionary (Flash), OrderedDict (Python), and OrderedDictionary (Objective-C). Some of these are more mature, some are more proof-of-concept.
LinkedHashMap is named according to implementation in the tradition of Java collections — "linked" because it uses a doubly-linked list to track insertion order, and "hash" because it subclasses HashMap. Besides the fact that user shouldn't need to worry about that, the class name doesn't really even indicate what it does. Using ordered seems like the consensus among existing code, but web searches on this topic also revealed understandable confusion between "ordered" and "sorted", and I feel the same. The .NET implementation even has a comment about the apparent misnomer, and suggests that it should be "IndexedDictionary" instead, owing to the fact that you can retrieve and insert objects at a specific point in the ordering.
I'm designing a framework and APIs and I want to name the class as intelligently as possible. From my standpoint, indexed would probably work (depending on how people interpret it, and based on the advertised functionality of the dictionary), ordered is imprecise and has too much potential for confusion, and linked "is right out" (apologies to Monty Python). ;-)
As a user, what name would make the most sense to you? Is there a particular name that says exactly what the class does? (I'm not averse to using slightly longer names like InsertionOrderDictionary if appropriate.)
Edit: Another strong possibility (discussed in my answer below) is IndexedDictionary. I don't really like "insertion order" because it doesn't make sense if you allow the user to insert keys at a specific index, reorder the keys, etc.
I vote OrderedDictionary, for the following reasons:
"Indexed" is never used in Cocoa classes, except in one instance. It always appears as a noun (NSIndexSet, NSIndexPath, objectAtIndex:, etc). There is only one instance when "Index" appears as a verb, which is on NSPropertyDescription's "indexed" property: isIndexed and setIndexed. NSPropertyDescription is roughly analogous to a table column in a database, where "indexing" refers to optimizing to speed up search times. It would therefore make sense that with NSPropertyDescription being part of the Core Data framework, that "isIndexed" and "setIndexed" would be equivalent to an index in a SQL database. Therefore, to call it "IndexedDictionary" would seem redundant, since indices in databases are created to speed up lookup time, but a dictionary already has O(1) lookup time. However, to call it "IndexDictionary" would also be a misnomer, since an "index" in Cocoa refers to position, not order. The two are semantically different.
I understand your concern over "OrderedDictionary", but the precedent has already been set in Cocoa. When users want to maintain a specific sequence, they use "ordered": -[NSApplication orderedDocuments], -[NSWindow orderedIndex], -[NSApplication orderedWindows], etc. So, John Pirie has mostly the right idea.
However, you don't want to make insertion into the dictionary a burden on your users. They'll want to create a dictionary once and then have it maintain an appropriate order. They won't even want to request objects in a specific order. Order specification should be done during initialization.
Therefore, I recommend making OrderedDictonary a class cluster, with private subclasses of InsertionOrderDictionary and NaturalOrderDictionary and CustomOrderDictionary. Then, the user simply creates an OrderedDictionary like so:
OrderedDictionary * dict = [[OrderedDictionary alloc] initWithOrder:kInsertionOrder];
//or kNaturalOrder, etc
For a CustomOrderDictionary, you could have them give you a comparison selector, or even (if they're running 10.6) a block. I think this would provide the most flexibility for future expansion while still maintain an appropriate name.
I vote for InsertionOrderDictionary. You nailed it.
Strong vote for OrderedDictionary.
The word "ordered" means exactly what you are advertising: that in iterating through a list of items, there is a defined order to selection of those items. "Indexed" is an implementation word -- it talks more to how the ordering is achieved. Index, linked list, tree... the user doesn't care; that aspect of the data structure should be hidden. "Ordered" is the exact word for the additional feature you are offering, regardless of how you get it done.
Further, it seems like the choice of ordering could be at the user's option. Any reason why you couldn't create methods on your datatype that allow the user to switch from, say, alphabetical ordering to insertion-time ordering? In the default case, a user would choose a particular ordering and stick with it, in which case implementation would be no less efficient than if you created specialized subclasses for each ordering method. And in some less-used cases, the developer might actually wish to use any of a number of different orderings for the same data, depending on app context. (I can think of specific projects I've worked on where I would have loved to have such a data structure available.)
Call it OrderedDictionary, because that's precisely what it is. (Frankly, I have more of a problem with the use of the word "Dictionary", because that word heavily implies ordering, where popular implementations of such don't provide it, but that's my pet peeve. You really should just be able to say "Dictionary" and know that the ordering is alphabetical -- because that's what a dictionary IS -- but that argument is too late for existing implementations in the popular languages.) And allow the user to access in what order he chooses.
Since posting this question, I'm starting to lean towards something like IndexedDictionary or IndexableDictionary. While it is useful to be able to maintain arbitrary key ordering, limiting that to insertion ordering only seems like a needless restriction. Plus, my class already supports indexOfKey: and keyAtIndex:, which are (purposefully) analagous to NSArray's indexOfObject: and objectAtIndex:. I'm strongly considering adding insertObject:forKey:atIndex: which matches up with NSMutableArray's insertObject:atIndex:.
Everyone knows that inserting in the middle of an array is inefficient, but that doesn't mean we shouldn't be allowed to on the rare occasions that it's truly useful. (Besides, the implementation could secretly use a doubly-linked list or any other suitable structure for tracking the ordering if needed...)
The big question: is "indexed" or "indexable" as vague or potentially confusing as "ordered"? Would people think of database indexes, or book indexes, etc.? Would it be detrimental if they assumed it was implemented with an array, or might that simplify user understanding of the functionality?
Edit: This name makes even more sense given the fact that I'm considering adding methods that work with an NSIndexSet in the future. (NSArray has -objectsAtIndexes: as well as methods for adding/removing observers for objects at given indexes.)
What about KeyedArray?
As you said in your last paragraph, I think that InsertionOrder(ed)Dict(ionary) is pretty unambiguous; I don't see how it could be interpreted in any way other than that the keys would be returned in the order they were inserted.
By decoupling the indexed order from the insertion order, doesn't this simply boil down to keeping an array and Dictionary in a single object? I guess my vote for this type of object is IndexedKeyDictionary
In C#:
public class IndexedKeyDictionary<TKey, TValue> {
List<TKey> _keys;
Dictionary<TKey, TValue> _dictionary;
...
public GetValueAtIndex(int index) {
return _dictionary[_keys[index]];
}
public Insert(TKey key, TValue val, int index) {
_dictionary.Add(key, val);
// do some array massaging (splice, etc.) to fit the new key
_keys[index] = key;
}
public SwapKeyIndexes(TKey k1, TKey k2) {
// swap the indexes of k1 and k2, assuming they exist in _keys
}
}
What would be really cool is indexed values...so we have a way to sort the values and get the new key order. Like if the values were graph coordinates, and we could read the keys (bin names) as we move up/down along the coordinate plane. What would you call that data structure? An IndexedValueDictionary?
At first glance I'm with the first reply -- InsertionOrderDictionary, though it's a bit ambiguous as to what "InsertionOrder" means at first glance.
What you're describing sounds to me almost exactly like a C++ STL map. From what I understand, a map is a dictionary that has additional rules, including ordering. The STL simply calls it "map", which I think is fairly apt. The trick with map is you can't really give the inheritance a nod without making it redundant -- i.e. "MapDictionary". That's just too redundant. "Map" is a bit too basic and leaves a lot of room for misinterpretation.
Though "CHMap" might not be a bad choice after looking at your documentation link.
Maybe "CHMappedDictionary"? =)
Best of luck.
Edit: Thanks for the clarification, you learn something new every day. =)
Is the only difference that allKeys returns keys in a specific order? If so, I would simply add allKeysSorted and allKeysOrderdByInsertion methods to the standard NSDictionary API.
What is the goal of this insertion order dictionary? What benefits does it give the programmer vs. an array?