auto generate code based on a packet structure - embedded

Question

We have data packets with different structures. They are supposed to be read/written in different languages. Example:
| ClassId | Data |
ClassId = "datapoint" (Data structure):
temperature - 1bytes
elevation - 2bytes
gradient - 1bytes
ClassId = "config" (Data structure):
frequency - 1bytes
deviceId - 3bytes
ClassId = "accelerometer" (Data structure):
time - 2bytes
x - 2bytes
y - 2bytes
z - 2bytes
Rather than manually writing the code that parses each data packet based on its class (which is error prone and time-consuming), I would expect to have a configuration file and then the code (python/c/etc.) is generated automatically that can read and write packets. Something along these lines:
lib.set(packet, "datapoint", {
elevation: 933,
temperature: 18,
gradient: 20
});
lib.get(packet, "datapoint");
=>
{
elevation: 933,
temperature: 18,
gradient: 20
}
Googling it did not bring me anywhere. Any pointers would be very helpful.

Answer 1

You need a code generation system, that compiles a packet specification into code to parse/unparse the packets.
You can build one ad hoc using a parser generator, and write ad hoc code to procedurally walk a parse tree and spit out relevant code.
Or you can use a program transformation system (PTS), which treats your packet specifications like source code, and transforms to source code in your target language. You explain the syntax of the packets to the PTS pretty much the same way you explain the syntax to a parser generator.
But with a PTS, you can write transformation rules in surface syntax notation that recognizes the packet-system syntax and maps it to target langauge function syntax. That makes writing and maintaining such a tool a lot easier, especially if the packet syntax changes, and/or you change the target language infrastructure to parse the packets in different ways.
EDIT 10/3: OP asks for a concrete example, presumably with a PTS.
I'm going to show what this looks like for our DMS Software Reengineering Toolkit (see bio for more about DMS).
First you need an (DMS-compatible) grammar for the packet languages. Based on what I see, it is pretty simple:
Packets = Packet ;
Packets = Packets Packet ; -- allow a list of packet defintions
Packet = 'ClassID' '=' STRING members ;
members = ;
members = members member ; -- allow list of members
member = IDENTIFIER '-' NATURAL 'bytes' ;
I think this grammar is naive in that packet members in practice may have different types (maybe strings, floats, booleans, ...); OP's example only shows what I'm assuming are N-byte binary integer numbers. You also need grammars for your various target languages. I'm going to assume you have these grammars (and that's quite the assumption); lets work with C for the moment. [DMS does have many of these].
We also have to assume a representation of the transmitted data. OP suggests something but I think he is trying to hint at generated code ("lib.set...").
Instead I'm going to assume that packet content is being read from a Stream as binary bytes simply appended together; this makes for the smallest packet size possible and thus fast transmission times.
So, now we to specify our code generator, as set of rewrite rules that map packet definitions to code.
For background, a rewrite rule for a PTS typically looks like this:
if you see *this*, replace it by *that*
So you are essentially replacing one structure by another. These typically operate on ASTs, but use surface syntax for this and that for readability.
What follows are DMS's source to source rewrite rules; they look like they operate on text but in fact they operate on ASTs produced by DMS's parser.
DMS has its own syntax for rules but it essentially follows the typical style above:
rule rule_name( pattern_variables ):
source_syntax_category -> target_syntax_category =
" this_pattern " -> " that_pattern " ;
Source and target patterns are enclosed in *metaquotes" "; actual literal quote characters are thus escaped as \".
For DMS rules this is always a fragment of the Packet notation, and that is always a fragment of our chosen target language (C). Pattern variable names in the rule head are given syntactic types and can only match the corresponding type in the AST. Pattern variables found inside metaquotes are written as \variable. Metafunctions can compute derived results; they are invoked inside patterns as "\function( args )". See DMS Rewrite Rules for more details.
source domain Packet; -- the little language we defined
target domain C; -- what we will generate code for
-- you'll write one of these rulesets for each target language
rule top_level(pl: Packets): Packets -> Statements =
" \pl "
-> " ReadPacketType(stream, packet_type);
switch(packet_type) {
\pl
default: Panic(\"unrecognized packet type\");
}" if IsRoot(pl); -- do this once [at root of tree]
rule translate_packet_definitions(p: Packet, pl: packet_list): Packets -> switch_case_list
" \p \pl ";
rule translate_packet_definition(s:STRING, ms: members, pl: Packets): Packets -> switch_case =
" ClassID = \s \m \pl "
-> " case \concatenate\(\"enum_\"\,\string_to_identifier\(\s\)\): {
\string_to_identifier\(\s\)* p=malloc(sizeof(\string_to_identifier\(\s\)));
\m
return p;
}
";
rule translate_members(m: member, ms: members) : members -> Statements
= " \m \ms ";
rule translate_member(i: IDENTIFIER, n: NATURAL) = member -> StatementList =
" \i - \n bytes " ->
" p-> \toCIdentifer\(\i\) = ReadNByteValue(stream,\toCNatural\(\n\)) ; "
This isn't complete (in particular, I need anther set of rules to generate the enum declaration for the set of packet types) and I doubt if it is exactly right but it give the flavor of the rules. With these rules, OP's example input would generate this C code:
ReadPacketType(stream, packet_type);
switch(packet_type) {
case enum_datapoint: {
datapoint* p=malloc(sizeof(datapoint));
p->temperature=ReadNByteValue(stream,1);
p->elevation=ReadNByteValue(stream,2);
p->gradient=ReadNByteValue(stream,2);
return p;
}
case enum_config: {
config* p=malloc(sizeof(config));
p->frequency=ReadNByteValue(stream,1);
p->deviceId=ReadNByteValue(stream,3);
return p;
}
case enum_accelerometer: {
accelerometer* p=malloc(sizeof(accelerometer));
p-time>=ReadNByteValue(stream,2);
p->x=ReadNByteValue(stream,2);
p->y=ReadNByteValue(stream,2);
p->z=ReadNByteValue(stream,2);
return p;
}
default: Panic(\"unrecognized packet type\");
}