As we know, local variables is located on stack. However, what is their order? Are they arranged as the order of their declaration? This means that the first declared variable is arrange on the higher address of stack (stack grows to lower address) ? As an example:
void foo(){
int iArray[4];
int iVar;
}
On stack, the local variable-- iArray and iVar are arranged as followed?
Only if you have optimisation turned off!
Once the optimiser gets hold of your code all bets are off. Common strategies for aggressive optimisations are:
Drop the variable if its never used or is just a copy of another variable.
Reorder varaibles in the order they are used. This helps greatly if your app is using swap space and also helps cache utilisation (on some machines).
Move often used variables into registers. Common on risk machinces with 32 lovely genreral purpose registers. Not so common on Intel with its measly eight single purpose registers.
Change the data type. e.g. casting small ints to intgers often speeds up register loading and caching.
reorder storage to minimise slack bytes. eg char a, double b, char c, int d could be reordered to double b, int d, char a, char c thus saving 10 bytes.
There is no rule you can depend on. Most compilers will use the declaration order unless you start to optimize the code.
Enabling optimizations can cause reuse of stack space, reordering of local variables or even move the variables to CPU registers, so they don't show up on the stack anymore.
[EDIT] On some systems, the stack grows to bigger addresses. So it starts with 0x1000 and the next address is 0x1001 instead of starting with 0xffff and the next address is 0xfffe.
The simplest implementations make it very easy to predict where various variables will end up on the stack. However, those implementations also allow certain security problems (mainly, overflowing a buffer and predicting what the extra data will overwrite, allowing the injection of shellcode).
Since the layout of the stack is implementation defined in most stack-based languages (technically, many such languages don't mandate the use of a stack, but instead have semantics that are easy to implement with a stack), compiler writers have gone to great lengths to make it hard to predict the stack layout at runtime.
Related
I have been recently working on learning embedded systems programming on my own. I have observed a fairly high usage of the keyword volatile qualifier when declaring variables?
What is the significance of volatile when declaring a variable in Embedded System programming?
Basically when the should the key word be used. I did read something about compiler optimization and use of the keyword. Also something related to memory mapping registers.
For example, I read this StackOverflow post but I didn't understand how it applied in an embedded environment. More specifically, I don't understand when the key word should be used. I did read something about compiler optimization and use of the keyword. Also something related to memory mapping registers, but I don't understand when to use it.
Let's have a look at an example. When you look at C header files for PIC microcontrollers, you will see that many elements are declared volatile:
extern volatile unsigned char PORTB # 0x006;
As you have read, the volatile keyword disables compiler optimization. Suppose you write a program that does the following:
PORTB = 0x00; // set all of port B low
while (PORTB == 0x00); // wait for any pin to get high
// do something else
When the compiler optimises this code, it will recognise the second line as an infinite loop: the condition is true and never gets false within its body. Therefore, everything after the infinite loop does not need to be compiled as it will never be ran. Hence, the compiler may decide to not include that part of the code in the generated assembly code.
However, this PORTB is actually linked to a physical port. It is a hardware port whose value may be altered by the external circuitry. This means that although the loop seems to be infinite, it doesn't have to be. The compiler can't possibly know this.
That's where volatile comes in. When PORTB is declared volatile, the compiler won't do any optimisation based on reasoning about PORTB. It will assume that its value may be changed at any time by external factors.
In the embedded systems world, one of the key aspects of the volatile key-word is that it denotes a variable that may change at any time (eg an external/hardware data input - eg an ADC) and therefore the compiler must not optimise use.
But specifically, when used with a control register, it indicates that a read access may in fact change the data!
As a general rule of thumb, I would recommend the use of the volatile qualifier in all of the following:
All hardware register accesses(read and write)
All variables that are accessible in multiple threads (especially interrupt handlers)
Note: accessing a volatile is not necessarily atomic, so it is imperative that you know your hardware and your code structure.
The volatile keyword is primarily used tell the compiler the value of the variable may change anytime. It also tells the compiler the not to apply optimization on the variable. I am not too much of an expert on this but below is good reference that I have referred in the past.
volatile is a qualifier that is applied to a variable when it is declared. It tells the compiler that the value of the variable may change at any time-without any action being taken by the code the compiler finds nearby. The implications of this are quite serious. However, before we examine them, let's take a look at the syntax.
Reference:
Introduction to the volatile keyword
Let me put it in other perspective it is exactly opposite of const keyword.
When compiler encounters const qualifier for any variable it checks if any function or statement is modified it once initialized. Hence flag error.
Volatile is exactly opposite, this variable can be changed by any function. Hence compiler does not apply optimization.
You see this mostly in embedded system programming due to use of interrupts and some programming logic constructs seems redundant.
While the statements about optimization are correct, they seem a little unclear to me. Here is what is really going on.
If you don't use the volatile keyword C may optimize that variable into a register it isn't using at the time. This will make for fewer assembly instructions and the code will execute faster.
For example, consider the following...
extern int my_port; // my_port is defined in a different module somewhere
// presumably a memory mapped hardware port
while (my_port > 0) {so stuff}
The compiler may decide to read my_port into a register only once before the actual while statement, then each time to test my_port it will look at the register not the memory location.
If, however, my_port is a hardware port, the port may change but register won't and the while conditional will not change.
The loop variable (the register) will be "out of phase" with the actual variable (my_port).
Thus the need for the keyword volatile.
Volatile tells C, "Don't optimize this variable into a reg, but read it each and every time you need it."
More instructions are generated, code is a bit slower, but it is always accurate.
For a while now I have been a little confused about the role of constant members within a language, such as Java or C. I understand that at the source code level, they prevent certain critical members from being mutated and changed, but when compiled or interpreted, is there any difference between them and variable members at all or are they all just pointers to memory addresses?
I thought that perhaps the compiler/interpreter has to implement something special to allow a variable to be mutable, something it wouldn't have to when handling a constant member (perhaps making execution faster or making it use less memory?), is this true or am I completely up the wrong tree?
The const variable and the variable are not stored in the same place once your code is executed. The constant values will go in the flash memory with your program. The variables will go in the flash too but will then be copied in the ram to be modified as your program runs. Making a variable const makes your computer save time and space by not pushing everything in the ram. When you need to modify it, you will push it into the Ram anyway, but most of the time const variables will not be modified.
This is in addition to the software fact that you might want to prevent your code to modify a value by mistake.
Where are the local, global, static, auto, register, extern, const, and volatile variables stored?
local variables can be stored either on the stack or in a data segment depending on whether they are auto or static. (if neither auto or static is explicitly specified, auto is assumed)
global variables are stored in a data segment (unless the compiler can optimize them away, see const) and have visibility from the point of declaration to the end of the compilation unit.
static variables are stored in a data segment (again, unless the compiler can optimize them away) and have visibility from the point of declaration to the end of the enclosing scope. Global variables which are not static are also visible in other compilation units (see extern).
auto variables are always local and are stored on the stack.
the register modifier tells the compiler to do its best to keep the variable in a register if at all possible. Otherwise it is stored on the stack.
extern variables are stored in the data segment. The extern modifier tells the compiler that a different compilation unit is actually declaring the variable, so don't create another instance of it or there will be a name collision at link time.
const variables can be stored either on the stack or a readonly data segment depending on whether they are auto or static. However, if the compiler can determine that they cannot be referenced from a different compilation unit, or that your code is not using the address of the const variable, it is free to optimize it away (each reference can be replaced by the constant value). In that case it's not stored anywhere.
the volatile modifier tells the compiler that the value of a variable may change at anytime from external influences (usually hardware) so it should not try to optimize away any reloads from memory into a register when that variable is referenced. This implies static storage.
BTW all this applies to C & C++ as well as Objective-C.
At what level of abstraction are you looking for an answer?
At the physical level, they're all probably stored in gate capacitances and magnetic domains. (Maybe even photons if your swap disk is Wi-Fi or optical fiber connected.)
At one hardware level, copies of any and all of these variables could exist at several places in the register, data cache (perhaps in multiple levels), main memory, and/or storage hierarchy, everything from completely swapped out to disk or non-volatile memory storage (depending on the existence, implementation, and current state of any demand-paged virtual memory subsystem), to perhaps everything in registers if your application’s size and lifetime is tiny enough.
Given the most familiar compiler and runtime implementations, memory (perhaps virtual) is chopped into things called stacks and heaps. Given the formal language definition, this chopping may or may not be required.
At the compiler optimization level, many of these variable may have been optimized out of existence. They're not stored anywhere except as an abstraction.
Local and auto variables are stored on the stack. Global and static variables are stored in a DATA page. register variables are stored in a register on the CPU if possible, otherwise in the stack. extern, const, and volatile do not specify where the variable is stored; the variable is stored where the other storage specifiers say they are.
Local variables are usually stored on the stack, and global variables in a program's "text" segment (in the case of string constants) or on the heap if they're dynamically allocated. Auto variables are usually used in functions/methods, and are generally passed on the stack (sometimes in registers, too, depending on architecture). Register variables are were once stored in registers, but most compilers nowadays ignore the register keyword and put them wherever they see fit -- on the stack or in a register. Extern, const, and volatile members are modifiers and so have no definitive place where they are stored.
So the short answer is, as usual, "it depends".
Local
Local variables whose scope is within the function. It may be two types, auto or static.
If it is declared simply int var, the compiler treats it as the auto storage class. The auto variables are stored on the stack.
The static variables are stored in the data segment.
The register variables are stored in the CPU. If no registers are available to store variables, then the compiler treats them as auto variable.
The global variables are stored in the data segment area.
The const variables are stored in the read-only area. That is code segment area of memory.
What happens when two threads set a BOOL to YES "at the same time"?
Here is code for solution suggested by Jacko.
Use volatile uint32_t with OSAtomicOr32Barrier and OSAtomicAnd32Barrier
#import <libkern/OSAtomic.h>
volatile uint32_t _IsRunning;
- (BOOL)isRunning {
return _IsRunning != 0;
}
- (void)setIsRunning:(BOOL)allowed {
if (allowed) {
OSAtomicOr32Barrier(1, & _IsRunning); //Atomic bitwise OR of two 32-bit values with barrier
} else {
OSAtomicAnd32Barrier(0, & _IsRunning); //Atomic bitwise AND of two 32-bit values with barrier.
}
}
No. Without a locking construct, reading/writing any type variable is NOT atomic in Objective C.
If two threads write YES at the same time to a BOOL, the result is YES regardless of which one gets in first.
Please see: Synchronizing Thread Execution
I would have to diverge from the accepted answer. Sorry.
While objective c does not guarantee that BOOL properties declared as nonatomic
are in fact atomic I'd have to guess that the hardware you most
care about (all iOS and macos devices) have instructions to perform byte reads and stores atomically. So, unless Apple comes out with Road Light OS running
on an IBM microcontroller that has 5 bit wide bus to send 10 bit bytes over
you could just as well use nonatomic BOOLs in a situation that calls for atomic BOOLs. The code would not be portable to Road Light OS but if you can
sacrifice the futureproofing of your code nonatomic is fine for this use case.
I'm sure there are hardened individuals on s.o. that would raise to the challenge of disassembling synthesized BOOL getter and setter for atomic/nonatomic cases to see what's the difference. At least on ARM.
Your takeaway from this is likely this
you can declare BOOL properties as atomic and it won't cost you a dime
on all HW iOS and macOS intrinsically supports.
memory barriers are orthogonal to atomicity
you most definitely should not use 4 byte properties to store booleans into
unless you are into [very] fuzzy logic.
It's idiotic and wasteful, you don't want to be a clone of a Java programmer,
who can't tell a float from a double, or do you?
BOOL variables (which do not obviously support atomic/nonatomic decorators
would not be atomic on some narrow bus architectures objective C would
not be used on anyway (microcontrollers with or without some [very] micro OS are C & assembly territory I suppose. they don't typically need the luggage
objc runtime would bring)
What happens when two threads set a BOOL to YES "at the same time"?
Then its value will be YES. If to threads write the same value to the same memory location, the memory location will have that value, whether that is atomic or not plays no role. It would only play a role if two threads write a different value to the same memory location or if one thread writes to it while another one is reading from it.
Is BOOL read/write atomic in Objective C?
It is if your hardware is a Macintosh running macOS. BOOL is uint32_t on PPC systems and char on Intel systems and writing these data types is atomic on their respective systems.
The Obj-C language makes no such guarantee, though. On other systems it depends on your compiler used and how BOOL is defined for that platfrom. Most compilers (gcc, clang, ...) guarantee that writing a variable of int-size is always atomic, whether other sizes are atomic depends on the CPU.
Note that atomic is not the same as thread-safe. Writing a BOOL is not a memory barrier. The compiler and the CPU may reorder instructions around a BOOL write:
a = 10;
b = YES;
c = 20;
There is no guarantee that instructions are executed in that order. The fact that b is YES does not mean that a is 10. The compiler and CPU are free to shuffle these three instructions around as desired since they don't depend on each other. Explicit atomic instructions as well as locks, mutexes and semaphores are usually memory barriers, that means they instruct the compiler and CPU to not move instructions located before that operation beyond it and not move instructions located after that operation before it (it's a hard border, that instructions may not pass).
Also cache consistency is not guaranteed. Even after you set a BOOL to YES, some other thread may still see it as NO for a limited amount of time. Memory barrier operations are usually also operations that ensure cache synchronization among all threads/cores/CPUs in the system.
And to add something really useful here as well, this is how you can ensure that setting a boolean value is atomic and acts as a memory barrier in 2020 using C11 which will also work in Obj-C Code:
#import <stdatomic.h>
// ...
volatile atomic_bool b = true;
// ...
atomic_store(&b, true);
// ...
atomic_store(&b, false);
Not only will this code guarantee atomic writes to the bool (for which the system will choose an appropriate type), it will also act as a memory barrier (Sequentially Consistent).
To read the boolean atomically from another thread, you'd use
bool x = atomic_load(&b);
You can also use atomic_load_explicit and atomic_store_explicit and pass an explicit memory order, which allows you to control more fine grained which kind of memory reordering is allowed and which ones is not.
Learn more about your possibilities here:
http://llvm.org/docs/Atomics.html
Always read "Notes for optimizers" to see which memory reordering is allowed. If in doubt, always use Sequentially Consistent (memory_order_seq_cst, which is the default if not specified). It will not result in fastest performance but it's the safest option and you really should only use something else if you know what you are doing.
When I write a program and tell it int c=5, it puts the value 5 into a little bit of it's memory, but how does it remember which one? The only way I could think of would be to have another bit of memory to tell it, but then it would have to remember where it kept that as well, so how does it remember where everything is?
Your code gets compiled before execution, at that step your variable will be replaced by the actual reference of the space where the value will be stored.
This at least is the general principle. In reality it will be way more complecated, but still the same basic idea.
There are lots of good answers here, but they all seem to miss one important point that I think was the main thrust of the OP's question, so here goes. I'm talking about compiled languages like C++, interpreted ones are much more complex.
When compiling your program, the compiler examines your code to find all the variables. Some variables are going to be global (or static), and some are going to be local. For the static variables, it assigns them fixed memory addresses. These addresses are likely to be sequential, and they start at some specific value. Due to the segmentation of memory on most architectures (and the virtual memory mechanisms), every application can (potentially) use the same memory addresses. Thus, if we assume the memory space programs are allowed to use starts at 0 for our example, every program you compile will put the first global variable at location 0. If that variable was 4 bytes, the next one would be at location 4, etc. These won't conflict with other programs running on your system because they're actually being mapped to an arbitrary sequential section of memory at run time. This is why it can assign a fixed address at compile time without worrying about hitting other programs.
For local variables, instead of being assigned a fixed address, they're assigned a fixed address relative to the stack pointer (which is usually a register). When a function is called that allocates variables on the stack, the stack pointer is simply moved by the required number of bytes, creating a gap in the used bytes on the stack. All the local variables are assigned fixed offsets to the stack pointer that put them into that gap. Every time a local variable is used, the real memory address is calculated by adding the stack pointer and the offset (neglecting caching values in registers). When the function returns, the stack pointer is reset to the way it was before the function was called, thus the entire stack frame including local variables is free to be overwritten by the next function call.
read Variable (programming) - Memory allocation:
http://en.wikipedia.org/wiki/Variable_(programming)#Memory_allocation
here is the text from the link (if you don't want to actually go there, but you are missing all the links within the text):
The specifics of variable allocation
and the representation of their values
vary widely, both among programming
languages and among implementations of
a given language. Many language
implementations allocate space for
local variables, whose extent lasts
for a single function call on the call
stack, and whose memory is
automatically reclaimed when the
function returns. (More generally, in
name binding, the name of a variable
is bound to the address of some
particular block (contiguous sequence)
of bytes in memory, and operations on
the variable manipulate that block.
Referencing is more common for
variables whose values have large or
unknown sizes when the code is
compiled. Such variables reference the
location of the value instead of the
storing value itself, which is
allocated from a pool of memory called
the heap.
Bound variables have values. A value,
however, is an abstraction, an idea;
in implementation, a value is
represented by some data object, which
is stored somewhere in computer
memory. The program, or the runtime
environment, must set aside memory for
each data object and, since memory is
finite, ensure that this memory is
yielded for reuse when the object is
no longer needed to represent some
variable's value.
Objects allocated from the heap must
be reclaimed—especially when the
objects are no longer needed. In a
garbage-collected language (such as
C#, Java, and Lisp), the runtime
environment automatically reclaims
objects when extant variables can no
longer refer to them. In
non-garbage-collected languages, such
as C, the program (and the programmer)
must explicitly allocate memory, and
then later free it, to reclaim its
memory. Failure to do so leads to
memory leaks, in which the heap is
depleted as the program runs, risking
eventual failure from exhausting
available memory.
When a variable refers to a data
structure created dynamically, some of
its components may be only indirectly
accessed through the variable. In such
circumstances, garbage collectors (or
analogous program features in
languages that lack garbage
collectors) must deal with a case
where only a portion of the memory
reachable from the variable needs to
be reclaimed
There's a multi-step dance that turns c = 5 into machine instructions to update a location in memory.
The compiler generates code in two parts. There's the instruction part (load a register with the address of C; load a register with the literal 5; store). And there's a data allocation part (leave 4 bytes of room at offset 0 for a variable known as "C").
A "linking loader" has to put this stuff into memory in a way that the OS will be able to run it. The loader requests memory and the OS allocates some blocks of virtual memory. The OS also maps the virtual memory to physical memory through an unrelated set of management mechanisms.
The loader puts the data page into one place and instruction part into another place. Notice that the instructions use relative addresses (an offset of 0 into the data page). The loader provides the actual location of the data page so that the instructions can resolve the real address.
When the actual "store" instruction is executed, the OS has to see if the referenced data page is actually in physical memory. It may be in the swap file and have to get loaded into physical memory. The virtual address being used is translated to a physical address of memory locations.
It's built into the program.
Basically, when a program is compiled into machine language, it becomes a series of instructions. Some instructions have memory addresses built into them, and this is the "end of the chain", so to speak. The compiler decides where each variable will be and burns this information into the executable file. (Remember the compiler is a DIFFERENT program to the program you are writing; just concentrate on how your own program works for the moment.)
For example,
ADD [1A56], 15
might add 15 to the value at location 1A56. (This instruction would be encoded using some code that the processor understands, but I won't explain that.)
Now, other instructions let you use a "variable" memory address - a memory address that was itself loaded from some location. This is the basis of pointers in C. You certainly can't have an infinite chain of these, otherwise you would run out of memory.
I hope that clears things up.
I'm going to phrase my response in very basic terminology. Please don't be insulted, I'm just not sure how proficient you already are and want to provide an answer acceptable to someone who could be a total beginner.
You aren't actually that far off in your assumption. The program you run your code through, usually called a compiler (or interpreter, depending on the language), keeps track of all the variables you use. You can think of your variables as a series of bins, and the individual pieces of data are kept inside these bins. The bins have labels on them, and when you build your source code into a program you can run, all of the labels are carried forward. The compiler takes care of this for you, so when you run the program, the proper things are fetched from their respective bin.
The variables you use are just another layer of labels. This makes things easier for you to keep track of. The way the variables are stored internally may have very complex or cryptic labels on them, but all you need to worry about is how you are referring to them in your code. Stay consistent, use good variable names, and keep track of what you're doing with your variables and the compiler/interpreter takes care of handling the low level tasks associated with that. This is a very simple, basic case of variable usage with memory.
You should study pointers.
http://home.netcom.com/~tjensen/ptr/ch1x.htm
Reduced to the bare metal, a variable lookup either reduces to an address that is some statically known offset to a base pointer held in a register (the stack pointer), or it is a constant address (global variable).
In an interpreted language, one register if often reserved to hold a pointer to a data structure (the "environment") that associates variable names with their current values.
Computers ultimately only undertand on and off - which we conveniently abstract to binary. This language is the basest level and is called machine language. I'm not sure if this is folklore - but some programmers used to (or maybe still do) program directly in machine language. Typing or reading in binary would be very cumbersome, which is why hexadecimal is often used to abbreviate the actual binary.
Because most of us are not savants, machine language is abstracted into assembly language. Assemply is a very primitive language that directly controls memory. There are a very limited number of commands (push/pop/add/goto), but these ultimately accomplish everything that is programmed. Different machine architectures have different versions of assembly, but the gist is that there are a few dozen key memory registers (physically in the CPU) - in a x86 architecture they are EAX, EBX, ECX, EDX, ... These contain data or pointers that the CPU uses to figure out what to do next. The CPU can only do 1 thing at a time and it uses these registers to figure out what to do next. Computers seem to be able to do lots of things simultaneously because the CPU can process these instructions very quickly - (millions/billions instructions per second). Of course, multi-core processors complicate things, but let's not go there...
Because most of us are not smart or accurate enough to program in assembly where you can easily crash the system, assembly is further abstracted into a 3rd generation language (3GL) - this is your C/C++/C#/Java etc... When you tell one of these languages to put the integer value 5 in a variable, your instructions are stored in text; the assembler compiles your text into an assembly file (executable); when the program is executed, the program and its instructions are queued by the CPU, when it is show time for that specific line of code, it gets read in the the CPU register and processed.
The 'not smart enough' comments about the languages are a bit tongue-in-cheek. Theoretically, the further you get away from zeros and ones to plain human language, the more quickly and efficiently you should be able to produce code.
There is an important flaw here that a few people make, which is assuming that all variables are stored in memory. Well, unless you count the CPU registers as memory, then this won't be completely right. Some compilers will optimize the generated code and if they can keep a variable stored in a register then some compilers will make use of this!
Then, of course, there's the complex matter of heap and stack memory. Local variables can be located in both! The preferred location would be in the stack, which is accessed way more often than the heap. This is the case for almost all local variables. Global variables are often part of the data segment of the final executable and tend to become part of the heap, although you can't release these global memory areas. But the heap is often used for on-the-fly allocations of new memory blocks, by allocating memory for them.
But with Global variables, the code will know exactly where they are and thus write their exact location in the code. (Well, their location from the beginning of the data segment anyways.) Register variables are located in the CPU and the compiler knows exactly which register, which is also just told to the code. Stack variables are located at an offset from the current stack pointer. This stack pointer will increase and decrease all the time, depending on the number of levels of procedures calling other procedures.
Only heap values are complex. When the application needs to store data on the heap, it needs a second variable to store it's address, otherwise it could lose track. This second variable is called a pointer and is located as global data or as part of the stack. (Or, on rare occasions, in the CPU registers.)
Oh, it's even a bit more complex than this, but already I can see some eyes rolling due to this information overkill. :-)
Think of memory as a drawer into which you decide how to devide it according to your spontaneous needs.
When you declare a variable of type integer or any other type, the compiler or interpreter (whichever) allocates a memory address in its Data Segment (DS register in assembler) and reserves a certain amount of following addresses depending on your type's length in bit.
As per your question, an integer is 32 bits long, so, from one given address, let's say D003F8AC, the 32 bits following this address will be reserved for your declared integer.
On compile time, whereever you reference your variable, the generated assembler code will replace it with its DS address. So, when you get the value of your variable C, the processor queries the address D003F8AC and retrieves it.
Hope this helps, since you already have much answers. :-)