I developed a simple pintool to list all the sections of the program's main executable image (iterating over all of its sections), and also its low and high limits using IMG_HighAddress and IMG_LowAddress; according to Pin, these return the definite limits of the image.
To my surprise, the sections went way beyond the low and high limits reported by these functions. Have I done something wrong, or are these functions inaccurate?
My image load function:
VOID ImageLoad(IMG img, VOID *v)
{
if (!IMG_IsMainExecutable(img))
return;
ADDRINT mainExeImageLowAddr = IMG_LowAddress(img);
ADDRINT mainExeImageHighAddr = IMG_HighAddress(img);
cout << "Image limits " << hex << mainExeImageLowAddr << " - " << mainExeImageHighAddr << endl;
for (SEC sec = IMG_SecHead(img); SEC_Valid(sec); sec = SEC_Next(sec))
{
cout << "Section " << SEC_Name(sec) << " at addresses 0x" << hex << SEC_Address(sec) << " - 0x" << SEC_Address(sec)+SEC_Size(sec)-1 << endl;
}
}
The results for running it on /bin/ls:
Image limits 400000 - 418b23
Section .interp at addresses 0x400200 - 0x40021b
Section .note.ABI-tag at addresses 0x40021c - 0x40023b
Section .note.gnu.build-id at addresses 0x40023c - 0x40025f
Section .dynsym at addresses 0x4002c8 - 0x400eaf
Section .rela.dyn at addresses 0x401618 - 0x4017c7
Section .rela.plt at addresses 0x4017c8 - 0x402157
Section .init at addresses 0x402158 - 0x40216f
Section .plt at addresses 0x402170 - 0x4027df
Section .text at addresses 0x4027e0 - 0x412347
Section .fini at addresses 0x412348 - 0x412355
Section .rodata at addresses 0x412360 - 0x415e86
Section .eh_frame_hdr at addresses 0x415e88 - 0x41653b
Section .eh_frame at addresses 0x416540 - 0x41851b
Section .dynstr at addresses 0x41851c - 0x418b23
Section .ctors at addresses 0x619000 - 0x61900f
Section .dtors at addresses 0x619010 - 0x61901f
Section .jcr at addresses 0x619020 - 0x619027
Section .data.rel.ro at addresses 0x619040 - 0x619a87
Section .dynamic at addresses 0x619a88 - 0x619c57
Section .got at addresses 0x619c58 - 0x619cef
Section .got.plt at addresses 0x619cf0 - 0x61a037
Section .data at addresses 0x61a040 - 0x61a23f
Section .bss at addresses 0x61a240 - 0x61af5f
Section .gnu.conflict at addresses 0x61af60 - 0x61b6f7
Section .gnu_debuglink at addresses 0x0 - 0xf
Section .gnu.prelink_undo at addresses 0x0 - 0x8ff
Section .shstrtab at addresses 0x0 - 0x12d
Have I done something wrong
Not necessarily.
It looks like Pin is trying to map ELF concepts to Windows-specific concepts, and there isn't a one-to-one mapping.
Intel documentation for IMG_HighAddress says:
Tells the highest address of any code or data loaded by the image.
This is the address of the last byte loaded by the image.
But what exactly does that mean?
ELF image loading is defined by PT_LOAD segments. You can see the segments, as well as section-to-segment mapping, in output from readelf -Wl a.out.
Usually there will be two LOAD segments: one with r-x protections covering .text, .rodata, and other read-only sections, and a second with rw- protections, covering .data, .bss and other writable sections.
It looks (from your output) like IMG_HighAddress is describing only the first LOAD segment.
You should also note that not all sections are LOADable, and the non-LOADable sections are usually not covered by any segment (and do not occupy memory at runtime). Various .debug* sections are usually not LOADable.
Related
I have been looking at some linker scripts for embedded ARM processors. In one of them, there's something like this (minimal example):
MEMORY {
REGION : ORIGIN = 0x1000, LENGTH = 0x1000
}
SECTIONS {
.text : {
/* ... */
. = 0x20;
/* ... */
} > MEMORY
}
This linker script states that the section .text should go in the memory region REGION, which starts at 0x1000. However, within the section contents, the location is explicitly set to 0x20.
Is this location assignment relative to the start of the region that the section is in? Or absolute? In general, how do regions and location assignments work together?
I did a test. I created an assembly file with the following contents:
.text
.word 0x1234
Then I wrote a basic linker script as detailed in the question:
MEMORY {
REGION : ORIGIN = 0x100, LENGTH = 0x100
}
SECTIONS {
.text : {
. = 0x20;
*(.text);
} > REGION
}
I compiled the assembly file to an object file with GCC, then linked the object file into an "executable" with ld. Running objdump -s on the result, I found that 0x1234 was at address 0x120. This means that the location assignment is relative to the start of the memory region.
The arbitrary precision libraries GMP and MPFR use heap-allocated arrays of machine word-sized integers to store the limbs that make up the high precision number/mantissa.
How should this array of limbs be interpreted to recover the arbitrary precision integer number? In other words: for N limbs holding B bits each, how should I interpret them to recover the N*B bit number?
Does the limb size really affect the in-memory representation (see below)? If so, what is the rationale behind this?
Background:
I wrote a small program to look inside the representation, but I was confused by what I saw. The limbs seem to be ordered in most significant digit order, whereas the limbs themselves are in native least significant digit format. When representing the 64-bit word 0xAAAABBBBCCCCDDDD using 32-bit words and precision fixed to 128 bits, I see:
% c++ limbs.cpp -lgmp -lmpfr -o limbs && ./limbs
ccccdddd|aaaabbbb|00000000|00000000
00000000|00000000|ccccdddd|aaaabbbb
This seems to imply that the in-memory representation can not be read back as a string of bits to recover the arbitrary precision number (e.g., if loaded this into a register on a machine that supported N*B sized words). Furthermore, this also seems to suggest that the limb size changes the representation, so that I would not be able to deserialize a number without knowing which limb size was used to serialize it.
Here's my test program (uses 32-bit limbs with the __GMP_SHORT_LIMB macro):
#define __GMP_SHORT_LIMB
#include <gmp.h>
#include <mpfr.h>
#include <iomanip>
#include <iostream>
constexpr int PRECISION = 128;
void PrintLimbs(mp_limb_t const *const limbs) {
std::cout << std::hex;
constexpr int NUM_LIMBS = PRECISION / (8 * sizeof(mp_limb_t));
for (int i = 0; i < NUM_LIMBS; ++i) {
std::cout << std::setfill('0') << std::setw(2 * sizeof(mp_limb_t)) << limbs[i];
if (i < NUM_LIMBS - 1) {
std::cout << "|";
}
}
std::cout << "\n";
}
int main() {
{ // GMP
mpz_t num;
mpz_init2(num, PRECISION);
mpz_set_ui(num, 0xAAAABBBBCCCCDDDD);
PrintLimbs(num->_mp_d);
mpz_clear(num);
}
{ // MPFR
mpfr_t num;
mpfr_init2(num, PRECISION);
mpfr_set_ui(num, 0xAAAABBBBCCCCDDDD, MPFR_RNDN);
PrintLimbs(num->_mpfr_d);
mpfr_clear(num);
}
return 0;
}
3 things that matter for the byte representation:
The limb size depends on your machine and the chosen ABI. The real size is also affected by the optional presence of nails (an experimental feature, thus it is unlikely that limbs have nails). MPFR does not support the presence of nails.
The limb representation in memory follows the endianness of the machine.
Limbs are stored least significant limb first (a.k.a. little endian).
Note that from the last two points, on a same big-endian machine, the byte representation of the array will depend on the limb size.
Concerning the size of the array of limbs, it depends on the type. For instance, with the mpn layer of GMP, it is entirely handled by the user.
For MPFR, the size is deduced from the precision of the mpfr_t object; and if the precision is not a multiple of the limb bitsize, the trailing bits are always set to 0. Note also that more memory may be allocated than the one actually used, and it must not be confused with the size of the array; you can ignore this fact, as the unused data are always after the actual array of limbs.
EDIT concerning the rationale: Manipulating limbs instead of bytes is for speed reasons. Then I suppose that little endian has been chosen to represent the array of limbs for two reasons. First, it makes the basic operations (addition, subtraction, multiplication) easier to implement and potentially faster. Second, this is much better to implement arithmetic modulo 2^K, in particular when K may change.
It finally clicked for me. The limb size does not affect the in-memory representation.
The data in GMP/MPFR is stored consistently in little-endian format, even when interpreted as a string of bytes across limbs. But registers on x86 are big-endian.
The inconsistent outcome when printing the limbs comes from how words are interpreted when read back from memory. When loaded into a register, memory is reinterpreted from little-endian (how it is stored in memory) to big-endian (how it is stored in registers).
I've modified the example below to show how it is in fact the word size with which we reinterpret memory that affects how the content is printed, as the output will be the same no matter if 32-bit or 64-bit limbs are used:
#define __GMP_SHORT_LIMB
#include <gmp.h>
#include <mpfr.h>
#include <iomanip>
#include <iostream>
#include <cstdint>
constexpr int PRECISION = 128;
template <typename InterpretAs>
void PrintLimbs(mp_limb_t const *const limbs) {
constexpr int LIMB_BITS = 8 * sizeof(InterpretAs);
constexpr int NUM_LIMBS = PRECISION / LIMB_BITS;
std::cout << LIMB_BITS << "-bit: ";
for (int i = 0; i < NUM_LIMBS; ++i) {
const auto limb = reinterpret_cast<InterpretAs const *>(limbs)[i];
for (int b = 0; b < LIMB_BITS; ++b) {
if (b > 0 && b % 16 == 0) {
std::cout << " ";
}
uint64_t bit = (limb >> (LIMB_BITS - 1 - b)) & 0x1;
std::cout << bit;
}
if (i < NUM_LIMBS - 1) {
std::cout << "|";
}
}
std::cout << "\n";
}
int main() {
uint64_t literal = 0b1111000000000000000000000000000000000000000000000000000000001001;
{ // GMP
mpz_t num;
mpz_init2(num, PRECISION);
mpz_set_ui(num, literal);
std::cout << "GMP where limbs are interpreted as:\n";
PrintLimbs<uint64_t>(num->_mp_d);
PrintLimbs<uint32_t>(num->_mp_d);
PrintLimbs<uint16_t>(num->_mp_d);
mpz_clear(num);
}
{ // MPFR
mpfr_t num;
mpfr_init2(num, PRECISION);
mpfr_set_ui(num, literal, MPFR_RNDN);
std::cout << "MPFR where limbs are interpreted as:\n";
PrintLimbs<uint64_t>(num->_mpfr_d);
PrintLimbs<uint32_t>(num->_mpfr_d);
PrintLimbs<uint16_t>(num->_mpfr_d);
mpfr_clear(num);
}
return 0;
}
This prints (regardless of limb size):
GMP where limbs are interpreted as:
64-bit: 1111000000000000 0000000000000000 0000000000000000 0000000000001001|0000000000000000 0000000000000000 0000000000000000 0000000000000000
32-bit: 0000000000000000 0000000000001001|1111000000000000 0000000000000000|0000000000000000 0000000000000000|0000000000000000 0000000000000000
16-bit: 0000000000001001|0000000000000000|0000000000000000|1111000000000000|0000000000000000|0000000000000000|0000000000000000|0000000000000000
MPFR where limbs are interpreted as:
64-bit: 0000000000000000 0000000000000000 0000000000000000 0000000000000000|1111000000000000 0000000000000000 0000000000000000 0000000000001001
32-bit: 0000000000000000 0000000000000000|0000000000000000 0000000000000000|0000000000000000 0000000000001001|1111000000000000 0000000000000000
16-bit: 0000000000000000|0000000000000000|0000000000000000|0000000000000000|0000000000001001|0000000000000000|0000000000000000|1111000000000000
CONTEXT
I'm using a code written to work with a GPS module that connects to the Arduino through serial communication. The module starts each packet with a header (0xb5, 0x62), continues with the information you requested and ends with to bytes of checksum, CK_A, and CK_B. I don't understand the code that calculates that checksum. More info about the algorithm of checksum (8-Bit Fletcher Algorithm) in the module protocol (https://www.u-blox.com/sites/default/files/products/documents/u-blox7-V14_ReceiverDescriptionProtocolSpec_%28GPS.G7-SW-12001%29_Public.pdf), page 74 (87 with index).
MORE INFO
Just wanted to understand the code, it works fine. In the UBX protocol, I mentioned there is also a piece of code that explains how it works (isn't write in c++)
struct NAV_POSLLH {
//Here goes the struct
};
NAV_POSLLH posllh;
void calcChecksum(unsigned char* CK) {
memset(CK, 0, 2);
for (int i = 0; i < (int)sizeof(NAV_POSLLH); i++) {
CK[0] += ((unsigned char*)(&posllh))[i];
CK[1] += CK[0];
}
}
In the link you provide, you can find a link to RFC 1145, containing that Fletcher 8 bit algorithm as well and explaining
It can be shown that at the end of the loop A will contain the 8-bit
1's complement sum of all octets in the datagram, and that B will
contain (n)*D[0] + (n-1)*D[1] + ... + D[n-1].
n = sizeof byte D[];
Quote adjusted to C syntax
Try it with a couple of bytes, pen and paper, and you'll see :)
I am rewriting codes that were using the class CGAL::Convex_hull_d, that is now deprecated. My need is to compute dynamically the convex hull of a set of points, and next to access to a a point and the normal of each facet.
The way to traverse the facets of a convex hull obtained by a triangulation is perfectly clear in the documentation, the problem is next to access to the informations related to the facets.
For the three-dimensional case, I have used the class CGAL::Delaunay_triangulation_3. After inserting points in the triangulation (called T), it is possible to obtain the convex hull using the CGAL::Surface_mesh class and the function CGAL::convex_hull_3_to_face_graph. Next, we can use facet iterators to traverse the facets, and next vertex iterators to access to the vertices of a facet. There is also a direct access to the normal to a facet using the function CGAL::Polygon_mesh_processing::compute_face_normal. This is described by the following portion of code.
Surface_mesh chull;
CGAL::convex_hull_3_to_face_graph(T,chull);
Surface_mesh::Face_range mesFaces = chull.faces();
Surface_mesh::Face_range::iterator debut,fin;
CGAL::Vertex_around_face_iterator<Surface_mesh> vbegin,vend;
debut = mesFaces.begin();
fin = mesFaces.end();
while(debut != fin)
{
Vector_3 p = CGAL::Polygon_mesh::compute_face_normal(*debut,chull);
std::cout << "Normal to the facet: (" << p[0] << "," << p[1] << "," << p[2] << ")" << std:endl;
boost::tie(vbegin,vend) = vertices_around_face(chull.halfedge(*debut),chull);
Point_3 po = chull.point(*vbegin);
std::cout << "One point of the facet: (" << po[0] << "," << po[1] << "," << po[2] << ")" << std:endl;
debut++;
}
However, the classes CGAL::Delaunay_Triangulation_3 and CGAL::Surface_mesh class cannot be used to compute the convex hull of a set of points with 4 dimensions. I have therefore considered the (recommended) use of dD Triangulation (in particular, the class CGAL::Triangulation). To traverse the facets of the convex hull is not a problem as a two full examples are given in the documention. Below, the first one is given.
{ int i=0;
typedef Triangulation::Full_cell_iterator Full_cell_iterator;
typedef Triangulation::Facet Facet;
for( Full_cell_iterator cit = t.full_cells_begin();cit != t.full_cells_end(); ++cit )
{
if( ! t.is_infinite(cit) )
continue;
Facet ft(cit, cit->index(t.infinite_vertex()));
++i;// |ft| is a facet of the convex hull
}
std::cout << "There are " << i << " facets on the convex hull."<< std::endl;
}
Here, the Facet ft is created but not used, the facets are only counted. I have not found a way to access to any information related to the facet in the documentation. My need would be the same as for the three-dimensional case.
Although I don't know how to write C++ I am trying to use CGAL for trying to derive building shapes from LiDAR point clouds using Point Set Shape Detection. Using the examples I can read points and normals from a file, whereupon CGAL detects shapes. The program is set to detect only planar shapes.
I would like to save the planar shapes to a file, so that I can use them in other software. But I was unable to find examples of how that can be achieved. The test program I use is based on the efficient_RANSAC_parameters.cpp code. It has a part when it iterates through all detected shapes. Could it be possible to add something there that will write the planar shapes to a file? I see the OFF format is a popular and simple way (in CGAL) to save polygons to a file, so that could be a good candidate file format.
A colleague who does know how to write C++ has helped me with this problem. He came up with the following:
while (it != shapes.end()) {
if (Plane* plane = dynamic_cast<Plane*>(it->get()))
{
std::cout << "PLANE_" << count++ << "[" << std::endl;
const std::vector<size_t> indices = it->get()->indices_of_assigned_points();
std::vector<size_t>::const_iterator iti = indices.begin();
while (iti != indices.end()) {
// Retrieves point
Point_with_normal pw = *(points.begin() + (*iti));
Kernel::Point_3 p = pw.first;
std::cout << "POINT[" << p.x() << "," << p.y() << "," << p.z() << "]" << std::endl;
// Proceeds with next point.
iti++;
}
std::cout << "]" << std::endl;
}
// Proceeds with next detected shape.
it++;
}
This block can replace loop in the efficient_RANSAC_parameters.cpp example. The output looks like this:
PLANE_0[
POINT[34.96,584.49,0.47]
POINT[34.97,585.24,0.54]
POINT[34.88,584.51,0.49]
POINT[34.98,584.75,0.49]
]
That gives me something to work with. In my case, I use sed to transform this output to SQL insert queries that allow me to transfer the data to a relational database for further processing.
In the example in the user manual you can see that once you have a plane shape object
if(Plane* plane = dynamic_cast<Plane*>(it->get())){..} you can obtain from the plane shape object a CGAL::Plane_3, from which you can obtain a point and a normal, or the coefficients of the plane.