I am new to Embedded field. One doubt came up in my mind about hex file downloading:
As output of the linker and locator is a binary file having various sections as .text,.bss,.data etc. and .text resides in Flash, .bss goes to RAM, .data goes to RAM ...
so my question is that
how .bss and .data are written to RAM as i am using FLASH Loader for burning my program onto flash.
Is there any index kind of thing in the final binary which discriminates between .text and .bss segments.
Is there any utility in the linker/locator which converts our simple binary into hex format.
How can I discriminate between .text and .bss from the contents of hex file?
Thanks in advance. Please help.
1.) how .bss and .data are written to RAM as i am using FLASH Loader for burning my program onto flash?
Code, constant data and initialized data are all written to FLASH. At runtime, initialized data is copied to bss during startup. Constant data is usually accessed directly (you declare it with the "const" keyword).
2.) Is there any index kind of thing in the final binary which discriminates between .text and .bss segments?
I think you mean by "binary" the linker output. This is usually referred to as an object file and is different from a binary image. The object file includes all code, data, symbol, debug information and memory addresses. For the GCC tool chain, the linker output is usually an .elf file.
Your linker uses a "link script" or other definition file to locate the various segments at the appropriate memory addresses. Your tool chain should have docs on how to change that.
3.) Is there any utility in the linker/locator which converts our simple binary into hex format?
The "objcopy" utility will read linker output and can write output files in a variety of formats, including Intel-hex. For human readable output see "objdump".
4.) How can I discriminate between .text and .bss from the contents of hex file?
By memory address. GCC uses the "initialized data" segment for data which is copied to the bss. It is located according to your linker script.
Intel-hex format: http://en.wikipedia.org/wiki/Intel_HEX
GCC: http://gcc.gnu.org/onlinedocs/
Your programming tools just write the program image (hex or binary file) into Flash, at the specified address.
When the compiler compiles the program, it adds information (look-up tables) containing information about the .bss and .data etc into the hex file, along with some C start-up code. The C start-up code, which runs before the C main() function begins, is responsible for initialising .bss and .data according to the information in the look-up tables in Flash memory. Typically, that means:
Initialising .bss to zeros.
Copying initialisation values for .data from the Flash memory into the appropriate locations in RAM.
(On some platforms, the initialisation values might even be stored compressed to save space in Flash, and the start-up code decompresses it as it writes it to RAM).
On some platforms, you might want to run code in RAM, for various reasons. The C start-up code can also copy code from Flash to RAM in a similar way.
The C start-up code can also do other essential things:
Initialise stack(s) and initial stack pointer
Initialise heap (if your program uses it)
Set some essential processor configuration, such as processor operating mode, clock speeds, Flash wait states, memory management unit, etc
Typically these issues are solved by the embedded tools you use to manage the target device.
For small embedded devices (less than 16K to 128K of RAM and ROM/NVRAM), code executes directly from NVRAM. The initialization code copies initialized data to RAM, perhaps decompressing it, and initializes uninitialized RAM, usually by clearing it. The locator is responsible for making all data references access the proper target addresses and not the address of ROM data.
Large embedded devices run much faster than NVRAM, so they tend to be implemented where NVRAM content (code plus data) is copied from NVRAM to RAM for execution.
To answer your other points:
2) The final binary is carefully structured to distinguish between program sections. There are many different file formats. See the corresponding Wikipedia article about the format for details.
3) You many not need to convert to hex format, though that depends on your target loader. For example, U-Boot based systems support kermit file transfers of the binary data.
4) See 2).
Related
I have some functioning firmware that I am deploying to an STM32 part with 64K of Flash memory (starting at address 0x8000000). I want to use some amount of this memory towards the end of the flash memory space to store a gzipped file on the STM32 part itself.
What is the best way to write this file to a location in the flash with openOCD? Should I somehow bundle it into the firmware ELF? Should I flash firmware and then follow that up with another flash command to write the file to a specific portion of the flash space?
Once written how do i use openOCD to pull the file from this location and get it back in a format that gzip can uncompress?
For the first question, there is this thread discussing how to embed some resources in an executable.
It would be probably easier to directly load the binary with openOCD in a command line.
For the second question, you will have direct access to the binary content of your file in Flash. The tricky point is that you have to implement an unzipper to uncompress it. There are probably plenty of source code to do so, but perhaps you may want to evaluate if this is the best solution: you would save memory by storing a zipped file but you would "waste" some for the code to unzip it ?
I need a way to send some data to the ucontroller through Trace32. I heard that this is possible some way, but I have no idea where to start.
What I am actually trying to do is run a piece of code on a Aurix TC297 ucontroller to do some measurements (runtime, RAM, etc.). This piece of code is actually a Kalman filter that needs as input a vector of structs that I have too send from the computer through Trace32. Please help !
"A way to send some data to the ucontroller through Trace32" is a little bit vague. There are various possibilities depending on what your actually try to achieve and might also depend on the used CPU family and target OS. Anyhow one of the following might work:
Simply writing some raw data to the target memory can be achieved with the Data.Set command.
To transfer a big amount of data (or even a whole application) from a file to the target memory the Data.LOAD commands might be the right choice. E.g. Data.LOAD.Binary command for a raw binary file.
To set variables in your application or even initiate C-style data arrays use the Var.Set command.
To write data to NOR flash or onchip flash memory you'll need the FLASH.AUTO command in addition to the previously mentioned commands (after declaring the flash memory to TRACE32).
To write data to a NAND, SPI or other serial flash memory you probably should use the FLASHFILE.Set command (after initialization of the FLASHFILE programming system).
To transfer data from TRACE32 to your target while the CPU is running you might have to configure correctly SYStem.MemAccess and use the memory access class prefix "E". E.g. Data.Set E:<addr> <data> or Var.Set %E <expression>.
You can use FDX for a bidirectional data transfer between debugger and a running target application.
To enable the target application to open and read files from the computer running TRACE32, you have to compile your application with suitable semihosting code and initiate semihosting in TRACE32 with TERM.GATE command.
I'm a total n00b in embedded programming. Suppose I'm building a firmware using a compiler. The result of this operation is a file that will be flashed into (I guess) the flash memory of a MCU such an ARM or a AVR.
My question is: What common structures (if any) are used for such generated files containing the firmware?
I came from desktop development and I understand that for example for Windows the compiler will most likely generate a PE or PE+, while Unix-like systems I may get a ELF and COFF, but have no idea for embedded systems.
I also understand that this highly depends on many factors (Compiler, ISA, MCU vendor, OS, etc.) so I'm fine with at least one example.
Update: I will up vote all answers providing examples of used structures and will select the one I feel best surveys the state of the art.
The firmware file is the Executable and Linkable File, usually processed to a binary (.bin) or text represented binary (.hex).
This binary file is the exact memory that is written to the embedded flash. When you first power the board, an internal bootloader will redirect the execution to your firmware entry point, normally at the address 0x0.
From there, it is your code that is running, this is why you have a startup code (usually startup.s file) that will configure clock, stack pointer registers, vector table, load the data section to RAM (your initialized variables), clear the zero initialized section, maybe you will want to copy your code to RAM and jump to the copy to avoid running code from FLASH (can be faster on some platforms), and so on.
When running over an Operational System, all these platform choices and resources are not in control of user code, there you can only link to the OS libraries and use the provided API to do low level actions. In embedded, it is 100% user code, you access the hardware and manage its resources.
Not surprisingly, Operational Systems are booted in a similar manner as firmware, since both are there in touch with the processor, memory and I/Os.
All of that, to say: the structure of a firmware is similar to the structure of any compiled program. There's the data sections and code sections that are organized in memory during the load by the Operational System, or by the program itself when running on embedded.
One main difference is the memory addressing in the firwmare binary, usually addresses are physical RAM address, since you do not have memory mapping feature on most of micro-controllers. This is transparent to the user, the compiler will abstract it.
Other significant difference is the stack pointer, on OSs user code will not reserve memory for the stack by itself, it relays on OS for that. When on firmware, you have to do it in user code for the same reason as before, there's no middle man to manage it for you. The linker script of the compiler will reserve Stack and Heap memory accordingly configured and there will be a stack_pointer symbol on your .map file letting you know where it points to. You won't find it in OSs program's map files.
Most tools output either an ELF, or a COFF, or something similar that can eventually boil down to a HEX/bin file.
That isn't necessarily what your target wants to see, however. Every vendor has their own format of "firmware" files. Sometimes they're encrypted and signed, sometimes plain text. Sometimes there's compression, sometimes it's raw. It might be a simple file, or something complex that is more than just your program.
An integral part of doing embedded work is the build flow and system startup/booting procedure, plus getting your code onto the part. Don't underestimate the effort.
Ultimately the data written to the ROM is normally just the code and constant data from which your application is composed, and therefore has no structure other than perhaps being segmented into code and data, and possibly custom segments if you have created them. The structure in this sense is defined by the linker script or configuration used to build the code. The file containing this code/data may be raw binary, or an encoded binary format such as Intel Hex or Motorola S-Record for example.
Typically also your toolchain will generate an object code file that contains not only the code/data, but also symbolic and debug information for use by a debugger. In this case when the debugger runs, it loads the code to the target (as in the binary file case above) and the symbol/debug information to the host to allow source level debugging. These files may be in a proprietary object file format specific to the toolchain, but are often standard "open" formats such as ELF. However strictly the meta-data components of an object file are not part of the firmware since they are not loaded on the target.
I've recently run across another firmware format not listed here. It's a binary format, might be called ".EEP" but might not. I think it is used by NXP. I've seen it used for ARM THUMB2 and for mystery stuff that may be a DSP/BSP.
All the following are 32-bit values, all stored in reverse endian except for CAFEBABE (so... BEBAFECA?):
CAFEBABE
length_in_16_bit_words(yes, 16-bit...?!)
base_addr
CRC32
length*2 bytes of data
FFFF (optional filler if the length is an odd number)
If there are more data blocks:
length
base
checksum that is not a CRC but something bizarre
data
FFFF (optional filler if the length is an odd number)
...
When no more data blocks remain:
length == 0
base == 0
checksum that is not a CRC but something bizarre
Then all of that is repeated for another memory bank/device.
I've learned that a process has the following structure in memory:
(Image from Operating System Concepts, page 82)
However, it is not clear to me what decides that a process looks like this. I guess processes could (and do?) look different if you have a look at non-standard OS / architectures.
Is this structure decided by the OS? By the compiler of the program? By the computer architecture? A combination of those?
Related and possible duplicate: Why do stacks typically grow downwards?.
On some ISAs (like x86), a downward-growing stack is baked in. (e.g. call decrements SP/ESP/RSP before pushing a return address, and exceptions / interrupts push a return context onto the stack so even if you wrote inefficient code that avoided the call instruction, you can't escape hardware usage of at least the kernel stack, although user-space stacks can do whatever you want.)
On others (like MIPS where there's no implicit stack usage), it's a software convention.
The rest of the layout follows from that: you want as much room as possible for downward stack growth and/or upward heap growth before they collide. (Or allowing you to set larger limits on their growth.)
Depending on the OS and executable file format, the linker may get to choose the layout, like whether text is above or below BSS and read-write data. The OS's program loader must respect where the linker asks for sections to be loaded (at least relative to each other, for executables that support ASLR of their static code/data/BSS). Normally such executables use PC-relative addressing to access static data, so ASLRing the text relative to the data or bss would require runtime fixups (and isn't done).
Or position-dependent executables have all their segments loaded at fixed (virtual) addresses, with only the stack address randomized.
The "heap" isn't normally a real thing, especially in systems with virtual memory so each process can have their own private virtual address space. Normally you have some space reserved for the stack, and everything outside that which isn't already mapped is fair game for malloc (actually its underlying mmap(MAP_ANONYMOUS) system calls) to choose when allocating new pages. But yes even modern glibc's malloc on modern Linux does still use brk() to move the "program break" upward for small allocations, increasing the size of "the heap" the way your diagram shows.
I think this is recommended by some committee and then tools like GCC conform to that recommendation. Binary format defines these segments and operating system and its tools facilitate the process of that format to run on system. lets say ELF is recommended by system V and then adopted by unix; and gcc produce the ELF binaries to be run on unix. so i feel story may start from binary format as it decides about memory mappings(code, data/heap/stack). binary format,among other hacks, defines memory mappings to be mapped for loading programs. As for example ELF defines segments (arrange code in text,data,stack to be loaded in memory), GCC generates that segments of ELF binary while loader loads these segments. operating system also has freedom in adjusting the values of these segments like stack size. These are debatable loud thoughts which I try to consolidate.
That figure represents a a specific implementation or an idealized one. A process does not necessarily have that structure. On many systems a process looks only somewhat similar to what is in the diagram.
I've been searching for why for two hours. Now I'm using IAR Embedded Workbench Evaluation 5.30 to debug on a CC430F6137 dev kit board. As I have declared some global static variable, I realize that I am not able to watch that correctly. They are shown as either FF or 3F.
I tried to look them up in the memory table, the pattern looks like ff 3f ff 3f .... This pattern last from 0x1C00-0x1CFF; Data shown in 0x1D00 - 0x2BFF. According to the datasheet of CC430F6137, the section 0 of the RAM ranges from 0x1C00 - 0x23FF, which is 2KB in total. Section 1 ranges from 0x2400 to 0x2BFF.
For example, on static variable is located at 0x1CED, according to the watch window. However, the value is 0x3F. When I use a local variable to copy the value from that static var, it is however not 0x3F. My static variable falls in a small boundary area(that has the strange pattern) in section 0.
My hardware: USB-MSP430 debug interface by TI. The eval board is EM430F6137F900.
I tried the simulator, no problem. I also created a simple piece of code utilizing static variable for MSP430F449 (by TI LPT MSP430 debugger) and the static variables can also be seen.
Does anyone have any idea why this happens and possible solutions? THANKS IN ADVANCE!
This bug has been fixed in the newest version of IAR Embedded Workbench
Here are a few suggestions:
Are you using the RAM disable functionality at all (RAMCTL)?
The RAM memory is made up of n sectors. Each sector can be completely powered down to save leakage,
however, all data is lost. Features of the RAM memory include:
• RAM memory has n sectors of 2k bytes each.
• Each sector 0 to n can be complete disabled, however data retention is lost.
• Each sector 0 to n automatically enters low power retention mode when possible.
Any sleep mode in use? I am thinking about LPMx.5.
When LPMx.5 (LPM3.5 or LPM4.5) is entered, the voltage regulator of the Power Management Module
(PMM) is disabled. All RAM and register contents are lost.
You wrote:
When I use a local variable to copy the value from that static var, it is however not 0x3F.
Do you mean that the local copy is not 0x3F while the global static is, or that copying makes both the copy and the global static work?
If copying makes both work, I suppose that the static global is never referenced. Make it volatile.
Generate a map file (Project->Options->Linker->List, Generate linker listing, Segment map, Module map). Find the map file in Debug/List and search for your global static variable. It should be in the DATA16_Z section (or possibly in the DATA16_I section, with a related slot in DATA16_ID for the initializing value).
If it is not then you have a problem, I believe you can find interesting info in EW430_CompilerReference.pdf, chapter "Descriptions of segments".
Can you try to write to address 0x1CED from within the debugger? You can use the Memory window to test that, and read back. That would confirm that the memory slot itself is working.
If it does not work, the RAM of your chip maybe damaged. That would explain why you have a pattern within 0x1C00-0x1CFF but not outside of that.