Proto3 release notes states:
The
deterministic serialization is, however, NOT canonical across languages; it
is also unstable across different builds with schema changes due to unknown
fields. Users who need canonical serialization, e.g. persistent storage in
a canonical form, fingerprinting, etc, should define their own
canonicalization specification and implement the serializer using reflection
APIs rather than relying on this API.
What I would like to achieve is to have a deterministic serialisation of protobuf message to carry a crypto signature along with it. As I understand due to differences in serialisers binary data could differ and signature would become invalid.
package Something
message Request {
Payload payload = 1;
// signature of serialised payload
bytes signature = 2;
message Payload {
string user_id_from = 1;
uint64 amount = 2;
string user_id_to = 3;
}
}
What is the way to do this using protobuf reflection?
This doesn't answer the question directly, but may solve your issue: don't store the payload as a message, but store the serialized bytes alongside with the signature.
message Request {
// Serialized Payload message.
bytes payload = 1;
// signature of serialised payload
bytes signature = 2;
}
message Payload {
string user_id_from = 1;
uint64 amount = 2;
string user_id_to = 3;
}
This may be a little less convenient to work with in code, but has the advantage of preserving all the forwards and backwards-compatibility guarantees of protobuf.
It also frees you from serializing the message twice when writing it (once as a subfield, once to get the signature).
RFC4511 (section 4.5.3.1) shows that if a directory is split over several servers, then the client needs to wade through several redirections in order to get a definitive answer. It seems silly that every client would need to do this. Is there any (free) library that does all of this logic and just returns a GOOD/BAD/UNKNOWN result?
http://linux.die.net/man/3/ldap_set_option
LDAP_OPT_REFERRAL_URLS
Sets/gets an array containing the referral URIs associated to the LDAP handle. outvalue must be a char *, and the caller is responsible of freeing the returned string by calling ldap_memvfree(3), while invalue must be a NULL-terminated char *const *; the library duplicates the corresponding string. This option is OpenLDAP specific.
LDAP_OPT_REFERRALS
Determines whether the library should implicitly chase referrals or not. invalue must be const int *; its value should either be LDAP_OPT_OFF or LDAP_OPT_ON. outvalue must be int *.
I am writing an encryption application that requires a 64 bit key. I am currently using the following code to automatically generate a key.
Function GenerateKey() As String
' Create an instance of a symmetric algorithm. The key and the IV are generated automatically.
Dim desCrypto As DESCryptoServiceProvider = DESCryptoServiceProvider.Create()
' Use the automatically generated key for encryption.
Return ASCIIEncoding.ASCII.GetString(desCrypto.Key)
End Function
I am wanting the user to create their own key. Can I convert a user defined password (a string) into a 64 bit key that can be used?
The answer depends on how secure you want it to be, I'm no security expert so I wouldn't give advice on it.
I did see this though: http://msdn.microsoft.com/en-us/library/system.security.cryptography.rfc2898derivebytes.aspx It can be used to derives bytes from a string key and salt in the way Jodrell eluded to, and would be far better than rolling yor own.
The other constructor that might be suited after that stage is detailed here: http://msdn.microsoft.com/en-us/library/51cy2e75.aspx
I'm sure if you searched for that on the web you could find examples of how to use it.
I have a C structure that allow users to configure options in an embedded system. Currently the GUI we use for this is custom written for every different version of this configuration structure. What I'd like for is to be able to describe the structure members in some format that can be read by the client configuration application, making it universal across all of our systems.
I've experimented with describing the structure in XML and having the client read the file; this works in most cases except those where some of the fields have inter-dependencies. So the format that I use needs to have a way to specify these; for instance, member A must always be less than or equal to half of member B.
Thanks in advance for your thoughts and suggestions.
EDIT:
After reading the first reply I realized that my question is indeed a little too vague, so here's another attempt:
The embedded system needs to have access to the data as a C struct, running any other language on the processor is not an option. Basically, all I need is a way to define metadata with the structure, this metadata will be downloaded onto flash along with firmware. The client configuration utility will then read the metadata file over RS-232, CAN etc. and populate a window (a tree-view) that the user can then use to edit options.
The XML file that I mentioned tinkering with was doing exactly that, it contained the structure member name, data type, number of elements etc. The location of the member within the XML file implicitly defined its position in the C struct. This file resides on flash and is read by the configuration program; the only thing lacking is a way to define dependencies between structure fields.
The code is generated automatically using MATLAB / Simulink so I do have access to a scripting language to help with the structure creation. For example, if I do end up using XML the structure will only be defined in the XML format and I'll use a script to create the C structure during code generation.
Hope this is clearer.
For the simple case where there is either no relationship or a relationship with a single other field, you could add two fields to the structure: the "other" field number and a pointer to a function that compares the two. Then you'd need to create functions that compared two values and return true or false depending upon whether or not the relationship is met. Well, guess you'd need to create two functions that tested the relationship and the inverse of the relationship (i.e. if field 1 needs to be greater than field 2, then field 2 needs to be less than or equal to field 1). If you need to place more than one restriction on the range, you can store a pointer to a list of function/field pairs.
An alternative is to create a validation function for every field and call it when the field is changed. Obviously this function could be as complex as you wanted but might require more hand coding.
In theory you could generate the validation functions for either of the above techniques from the XML description that you described.
I would have expected you to get some answers by now, but let me see what I can do.
Your question is a bit vague, but it sounds like you want one of
Code generation
An embedded extension language
A hand coded run-time mini language
Code Generation
You say that you are currently hand tooling the configuration code each time you change this. I'm willing to bet that this is a highly repetitive task, so there is no reason that you can't write program to do it for you. Your generator should consume some domain specific language and emit c code and header files which you subsequently build into you application. An example of what I'm talking about here would be GNU gengetopt. There is nothing wrong with the idea of using xml for the input language.
Advantages:
the resulting code can be both fast and compact
there is no need for an interpreter running on the target platform
Disadvantages:
you have to write the generator
changing things requires a recompile
Extension Language
Tcl, python and other languages work well in conjunction with c code, and will allow you to specify the configuration behavior in a dynamic language rather than mucking around with c typing and strings and and and...
Advantages:
dynamic language probably means the configuration code is simpler
change configuration options without recompiling
Disadvantages:
you need the dynamic language running on the target platform
Mini language
You could write your own embedded mini-language.
Advantages:
No need to recompile
Because you write it it will run on your target
Disadvantages:
You have to write it yourself
How much does the struct change from version to version? When I did this kind of thing I hardcoded it into the PC app, which then worked out what the packet meant from the firmware version - but the only changes were usually an extra field added onto the end every couple of months.
I suppose I would use something like the following if I wanted to go down the metadata route.
typedef struct
{
unsigned char field1;
unsigned short field2;
unsigned char a_string[4];
} data;
typedef struct
{
unsigned char name[16];
unsigned char type;
unsigned char min;
unsigned char max;
} field_info;
field_info fields[3];
void init_meta(void)
{
strcpy(fields[0].name, "field1");
fields[0].type = TYPE_UCHAR;
fields[0].min = 1;
fields[0].max = 250;
strcpy(fields[1].name, "field2");
fields[1].type = TYPE_USHORT;
fields[1].min = 0;
fields[1].max = 0xffff;
strcpy(fields[2].name, "a_string");
fields[2].type = TYPE_STRING;
fields[2].min = 0 // n/a
fields[2].max = 0 // n/a
}
void send_meta(void)
{
rs232_packet packet;
memcpy(packet.payload, fields, sizeof(fields));
packet.length = sizeof(fields);
send_packet(packet);
}
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
So after CodingHorror's fun with encryption and the thrashing comments, we are reconsidering doing our own encryption.
In this case, we need to pass some information that identifies a user to a 3rd party service which will then call back to a service on our website with the information plus a hash.
The 2nd service looks up info on that user and then passes it back to the 3rd party service.
We want to encrypt this user information going into the 3rd party service and decrypt it after it comes out. So it is not a long lived encryption.
On the coding horror article, Coda Hale recommended BouncyCastle and a high level abstraction in the library to do the encryption specific to a particular need.
My problem is that the BouncyCastle namespaces are huge and the documentation is non-existant. Can anyone point me to this high level abstraction library? (Or another option besides BouncyCastle?)
High level abstraction? I suppose the highest level abstractions in the Bouncy Castle library would include:
The BlockCipher interface (for symmetric ciphers)
The BufferedBlockCipher class
The AsymmetricBlockCipher interface
The BufferedAsymmetricBlockCipher class
The CipherParameters interface (for initializing the block ciphers and asymmetric block ciphers)
I am mostly familiar with the Java version of the library. Perhaps this code snippet will offer you a high enough abstraction for your purposes (example is using AES-256 encryption):
public byte[] encryptAES256(byte[] input, byte[] key) throws InvalidCipherTextException {
assert key.length == 32; // 32 bytes == 256 bits
CipherParameters cipherParameters = new KeyParameter(key);
/*
* A full list of BlockCiphers can be found at http://www.bouncycastle.org/docs/docs1.6/org/bouncycastle/crypto/BlockCipher.html
*/
BlockCipher blockCipher = new AESEngine();
/*
* Paddings available (http://www.bouncycastle.org/docs/docs1.6/org/bouncycastle/crypto/paddings/BlockCipherPadding.html):
* - ISO10126d2Padding
* - ISO7816d4Padding
* - PKCS7Padding
* - TBCPadding
* - X923Padding
* - ZeroBytePadding
*/
BlockCipherPadding blockCipherPadding = new ZeroBytePadding();
BufferedBlockCipher bufferedBlockCipher = new PaddedBufferedBlockCipher(blockCipher, blockCipherPadding);
return encrypt(input, bufferedBlockCipher, cipherParameters);
}
public byte[] encrypt(byte[] input, BufferedBlockCipher bufferedBlockCipher, CipherParameters cipherParameters) throws InvalidCipherTextException {
boolean forEncryption = true;
return process(input, bufferedBlockCipher, cipherParameters, forEncryption);
}
public byte[] decrypt(byte[] input, BufferedBlockCipher bufferedBlockCipher, CipherParameters cipherParameters) throws InvalidCipherTextException {
boolean forEncryption = false;
return process(input, bufferedBlockCipher, cipherParameters, forEncryption);
}
public byte[] process(byte[] input, BufferedBlockCipher bufferedBlockCipher, CipherParameters cipherParameters, boolean forEncryption) throws InvalidCipherTextException {
bufferedBlockCipher.init(forEncryption, cipherParameters);
int inputOffset = 0;
int inputLength = input.length;
int maximumOutputLength = bufferedBlockCipher.getOutputSize(inputLength);
byte[] output = new byte[maximumOutputLength];
int outputOffset = 0;
int outputLength = 0;
int bytesProcessed;
bytesProcessed = bufferedBlockCipher.processBytes(
input, inputOffset, inputLength,
output, outputOffset
);
outputOffset += bytesProcessed;
outputLength += bytesProcessed;
bytesProcessed = bufferedBlockCipher.doFinal(output, outputOffset);
outputOffset += bytesProcessed;
outputLength += bytesProcessed;
if (outputLength == output.length) {
return output;
} else {
byte[] truncatedOutput = new byte[outputLength];
System.arraycopy(
output, 0,
truncatedOutput, 0,
outputLength
);
return truncatedOutput;
}
}
Edit: Whoops, I just read the article you linked to. It sounds like he is talking about even higher level abstractions than I thought (e.g., "send a confidential message"). I am afraid I don't quite understand what he is getting at.
Assuming that you write your application in Java I'd recommend that you don't use a specific provider, but that you develop your application on top of Sun's JCE (Java Cryptography Extension). Doing so can make you independent of any underlying providers, I.e., you can switch providers easily as long as you use ciphers that are widely implemented. It does give you a certain level of abstraction as you don't have to know all the details of the implementations and may protect you a little from using the wrong classes (e.g. such as using raw encryption without proper padding etc) Furthermore, Sun provides a decent amount of documentation and code samples.
I've actually found that this sample uses default 128 bit encryption instead of 256 bit. I've made a little change:
BlockCipher blockCipher = new AESEngine();
now becomes:
BlockCipher blockCipher = new RijndaelEngine(256);
and it works together with my client application C++ AES256 encryption
One example of a high(er)-level API in BouncyCastle would be the CMS (Cryptographic Message Syntax) package. This ships in a separate jar (bcmail) from the provider itself, and is written to the JCE (The C# version is written against the lightweight API however).
"Send a confidential message" is implemented, roughly speaking, by the CMSEnvelopedDataGenerator class, and all you really need to do is give it the message, choose an encryption algorithm (all details handled internally), and then specify one or more ways that a recipient will be able to read the message: this can be based on a public key/certificate, a shared secret, a password, or even a key agreement protocol. You can have more than one recipient on a message, and you can mix and match types of recipient.
You can use CMSSignedDataGenerator to similarly send a verifiable message. If you want to sign and encrypt, the CMS structures are nestable/composable (but order could be important). There's also CMSCompressedDataGenerator and recently added CMSAuthenticatedData.
You may use:
byte[] process(bool encrypt, byte[] input, byte[] key)
{
var cipher = CipherUtilities.GetCipher("Blowfish");
cipher.Init(false, new KeyParameter(key));
return cipher.DoFinal(input);
}
// Encrypt:
byte[] encrypted = process(true, clear, key);
// Decrypt:
byte[] decrypted = process(false, encrypted, key);
See: https://github.com/wernight/decrypt-toolbox/blob/master/dtDecrypt/Program.cs
JCE won't work for me because we want 256 bit strength and can't change the java configuration on the system to allow it. Too bad the Bouncy Castle doesn't have an API as high-level as JCE.
"Note however that bouncycastle consists of two libraries, the lightweight crypto library and the JCE provider interface library. The keysize restrictions are enforced by the JCE layer, but you don't need to use this layer. If you just use the lightweight crypto API directly you don't have any restrictions, no matter what policy files are or are not installed."
http://www.coderanch.com/t/420255/Security/AES-cryptoPerms-Unlimited-Cryptography
The book Beginning Cryptography with Java contains very helpful examples and explanations based on the bouncycastle library