What's the format of boost.serialization's output - serialization

Question

I tried to serialize a vector and a map container and output their value by cout. However, it is hard for me to get the meaning of boost's output. My code looks like this:
#include <iostream>
#include <boost/serialization/vector.hpp>
#include <boost/serialization/map.hpp>
#include <boost/assign.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
#include <sstream>
#include <fstream>
using namespace std;
int main()
{
vector<int> v = boost::assign::list_of(1)(3)(5);
map<int, string> m = boost::assign::map_list_of(1,"one")(2,"two");
std::stringstream ss;
boost::archive::text_oarchive oa(ss);
oa<<v<<m;
vector<int> v_;
map<int,string> m_;
boost::archive::text_iarchive ia(ss);
ia>>v_>>m_;
boost::archive::text_oarchive ib(cout);
ib<<v_<<m_;
return 0;
}
The output looks like this:
22 serialization::archive 9 3 0 1 3 5 0 0 2 0 0 0 1 3 one 2 3 two
What's the meaning of the numbers 9 3 0 before the value 1 3 5 I compose? How about the 0 0 2 0 0 0 ? Does the '3' between '1' and 'one' mean there are 3 characters ?

Answer 1

I'm not sure about some zeros in the map (maybe some version number or tracking levels) but for the rest :
22 (length of the signature)
serialization::archive (signature)
9 (archive version, 10 on boost 1.53)
3 (vector size)
0 (item version)
1 3 5 (vector items)
0 (map class tracking level ?)
0 (map class version ?)
2 (map size)
0 (item class tracking _level ?)
0 (item class version ?)
0 (item version)
1 (key) 3 (value length) one (value)
2 (key) 3 (value length) two (value)
Note that the content and format of the text output is Boost's internal business and may change with future Boost revisions, so your application shouldn't rely on the internal archive contents.

Answer 2

If you put these lines at the end of your code, you will obtain a human readable XML version of the archive.
boost::archive::xml_oarchive ib(cout);
ib<< boost::serialization::make_nvp("v", v_) << boost::serialization::make_nvp("m", m_); // ib<<v_<<m_;
return 0;
you will get this output, which describes it self:
<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<!DOCTYPE boost_serialization>
<boost_serialization signature="serialization::archive" version="10">
<v>
<count>3</count>
<item_version>0</item_version>
<item>1</item>
<item>3</item>
<item>5</item>
</v>
<m class_id="1" tracking_level="0" version="0">
<count>2</count>
<item_version>0</item_version>
<item class_id="2" tracking_level="0" version="0">
<first>1</first>
<second>one</second>
</item>
<item>
<first>2</first>
<second>two</second>
</item>
</m>
</boost_serialization>
So #zacinter is correct and the three 0 after the 2 are: 1) item_version (of std::pair, the value type of the map) 2) tracking_level of std::pair and 3) version of std::pair.

Answer 3

22 is the length of the text "serialization::archive".
Every text that is archived has such a text-length-number in front of it I believe.

Answer 4

I'm aware that one should normally not try to parse the text_oarchive format of boost::serialization because the format is subject to change and should only ever be consumed by boost itself. In my case though, the software producing this output is a proprietary binary blob that uses messages on TCP and UDP encoded with boost::serialization. My (FOSS) client needs to be able to talk to it without having boost available. So I needed to figure out how all of this works and this SO question was the most useful I've found so far on this platform. So for any other poor souls in a similar situation, I wanted to share some more discoveries I made about the text_oarchive format.
I'll talk about an example as it would typically be produced by the proprietary application I need to inferface with. So here is some code with dummy classes A1, A2, A3, A4 and A5. A1 is the base class and all of the following classes are sub-classes inheriting from the one before.
#include "boost/serialization/export.hpp"
#include "boost/serialization/extended_type_info.hpp"
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/xml_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
#include <iostream>
#include <sstream>
class A1 {
public:
A1(void) {}
virtual ~A1(void) {}
private:
unsigned int mem1 = 101;
friend class boost::serialization::access;
template <class Archive> void serialize(Archive & ar, const unsigned int version) {
ar & BOOST_SERIALIZATION_NVP(mem1);
}
};
BOOST_CLASS_EXPORT_KEY(A1);
class A2 : public A1 {
public:
A2(void) {}
private:
unsigned int mem2 = 102;
friend class boost::serialization::access;
template <class Archive> void serialize(Archive & ar, const unsigned int version) {
ar & BOOST_SERIALIZATION_BASE_OBJECT_NVP(A1);
ar & BOOST_SERIALIZATION_NVP(mem2);
}
};
BOOST_CLASS_EXPORT_KEY(A2);
class A3 : public A2 {
public:
A3(void) {}
private:
virtual void foo() = 0;
unsigned int mem3 = 103;
friend class boost::serialization::access;
template <class Archive> void serialize(Archive & ar, const unsigned int version) {
ar & BOOST_SERIALIZATION_BASE_OBJECT_NVP(A2);
ar & BOOST_SERIALIZATION_NVP(mem3);
}
};
BOOST_CLASS_EXPORT_KEY(A3);
class A4 : public A3 {
public:
A4(void) {}
private:
void foo() {}
unsigned int mem4 = 104;
friend class boost::serialization::access;
template <class Archive> void serialize(Archive & ar, const unsigned int version) {
ar & BOOST_SERIALIZATION_BASE_OBJECT_NVP(A3);
ar & BOOST_SERIALIZATION_NVP(mem4);
}
};
BOOST_CLASS_EXPORT_KEY(A4);
class A5 : public A4 {
public:
A5(void) {}
private:
unsigned int mem5 = 105;
friend class boost::serialization::access;
template <class Archive> void serialize(Archive & ar, const unsigned int version) {
ar & BOOST_SERIALIZATION_BASE_OBJECT_NVP(A4);
ar & BOOST_SERIALIZATION_NVP(mem5);
}
};
BOOST_CLASS_EXPORT_KEY(A5);
BOOST_CLASS_EXPORT_IMPLEMENT(A1);
BOOST_CLASS_EXPORT_IMPLEMENT(A2);
BOOST_CLASS_EXPORT_IMPLEMENT(A3);
BOOST_CLASS_EXPORT_IMPLEMENT(A4);
BOOST_CLASS_EXPORT_IMPLEMENT(A5);
BOOST_CLASS_VERSION(A1, 11)
BOOST_CLASS_VERSION(A2, 12)
BOOST_CLASS_VERSION(A3, 13)
BOOST_CLASS_VERSION(A4, 14)
BOOST_CLASS_VERSION(A5, 15)
void func(const A1 * const packet) {
std::string outpacket;
std::ostringstream stream;
boost::archive::xml_oarchive oa_xml(stream);
oa_xml << BOOST_SERIALIZATION_NVP(packet);
boost::archive::text_oarchive oa_text(stream);
oa_text << BOOST_SERIALIZATION_NVP(packet);
outpacket = stream.str();
std::cout << outpacket << std::endl;
}
int main() {
auto data = new A5();
func(data);
}
At the end, the main() function creates an A5 object and passes it to the function called func which is able to handle arguments of type A1 of which A5 is a sub-class. This is important as the boost::serialization output would be different if func took A5 as input directly.
Another important bit is the virtual destructor of A1. This one is required as well or otherwise one will not get the expected output for reasons that I do not yet understand.
To be able to better understand the text format, func encodes the data as XML as well as in the text format. To be able to use the XML output, BOOST_SERIALIZATION_BASE_OBJECT_NVP and BOOST_SERIALIZATION_NVP are applied wherever necessary. This does not seem to inferfere with the text output at all, which does not need these.
Each of the classes A1 to A5 has a member called mem1 to mem5, respectively. These are filled with the values 101 to 105, respectively, to better debug which integer in the text output stands for what.
To understand the output even better, BOOST_CLASS_VERSION is used to give all classes a unique class version. This helps because by default the class version is zero and then it's not clear what all the zeroes in the output stand for.
Without further ado, this is the output of compiling and running above code:
<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<!DOCTYPE boost_serialization>
<boost_serialization signature="serialization::archive" version="18">
<packet class_id="1" class_name="A5" tracking_level="1" version="15" object_id="_0">
<A4 class_id="2" tracking_level="1" version="14" object_id="_1">
<A3 class_id="3" tracking_level="0" version="13">
<A2 class_id="4" tracking_level="1" version="12" object_id="_2">
<A1 class_id="0" tracking_level="1" version="11" object_id="_3">
<mem1>101</mem1>
</A1>
<mem2>102</mem2>
</A2>
<mem3>103</mem3>
</A3>
<mem4>104</mem4>
</A4>
<mem5>105</mem5>
</packet>
22 serialization::archive 18 1 2 A5 1 15
0 1 14
1 0 13 1 12
2 1 11
3 101 102 103 104 105
Some general observations:
in general, the values in the XML output seem to be in the same order as in the text output when reading the xml sequentially from top to bottom and left to right
consequently, the member variables of each of the sub-classes are bundled together all at the end of the output
class A3, which is the only class with a virtual function, is the only one with tracking level 0 (all the others are tracking level 1) and without an object id
I do not understand where the newlines in front of the object-ids come from in the text output but boost seems to be able to parse a message with newlines replaced by spaces just the same, so they do not seem to have any semantic meaning
Lets look at the values one-by-one by having each of them in their own line:
22 -- length of the string "serialization::archive"
serialization::archive -- BOOST_ARCHIVE_SIGNATURE defined in src/basic_archive.cpp
18 -- BOOST_ARCHIVE_VERSION as defined in src/basic_archive.cpp
1 -- class-id of A5
2 -- length of class name "A5"
A5 -- class name of A5
1 -- tracking level of A5
15 -- class version of A5
0 -- object-id of A5
1 -- tracking level of A4
14 -- class version of A4
1 -- object-id of A4
0 -- tracking level of A3 (the only "0" tracking level -- A3 is abstract class)
13 -- class version of A3 (notice the absence of object-id for abstract class A3 as well)
1 -- tracking level of A2
12 -- class version of A2
2 -- object-id of A2
1 -- tracking level of A1
11 -- class version of A1
3 -- object-id of A1
101 -- member of A1
102 -- member of A2
103 -- member of A3
104 -- member of A4
105 -- member of A5