I came to doubt after doing this.
I had created a DLL in which APIs accepts pointers to some values from application. I am able to access those pointers in DLL.
So i had a doubt how is memory allocated for functions present in DLL and called from application. It should occur in calling proccess memory space.
Am i right?
You are right: DLL and application shares the same address space, so at system level the is no problem of allocated memory usage.
But you must be careful with deallocation of memory, which was allocated by code from another module. It will work correctly only if:
Both modules are built on the same platform (e.g. Visual C++).
Both modules use the same heap manager from the third shared DLL (e.g. msvcrt).
Related
I know this is JVM dependent and every virtual machine would choose to implement it a little bit different yet I want to understand the overall concept.
It has been said that for the Memory Segments that the JVM uses to execute a Java program
Java Stacks
Heap
Method Area
PC Registers
Native Method Stacks
are not necessarily implemented with contiguous memory and may be all actually allocated on some heap memory provided from the OS, this leads me to my question.
JVM's that fully use the JIT mechanism and compiles bytecode methods
into native machinecode methods store these methods somewhere, where
would that be? the execution engine ( that is usually written in C /
C++ ) would have to invoke these JIT compiled functions, yet the kernel shouldn't allow a program to execute code saved on the stack / heap / static memory segment, how could the JVM overcome this?
Another question I have is regarding the Java stacks, when a method ( after JIT compilation ) is executed within the processor it's local variables should be saved within the Java stacks, yet again the Java stacks may be implemented with a non-contiguous memory and perhaps even just some stack data structure allocated on the heap acting as a stack, how and where do the local variables of a method being executed get saved? the kernel shouldn't allow a program to treat a heap allocated memory as a process stack, how does JVM overcome this difficuly as well?
Again, I want to emphasis that I'm asking for an overall concept, I know each JVM would choose to implement this a little bit different...
JVM's that fully use the JIT mechanism and compiles bytecode methods into native machinecode methods store these methods somewhere, where would that be?
It is stored in the "Perm Gen" in Java <= 7 and "meta space" in Java 8. This is another native memory region.
the execution engine ( that is usually written in C / C++ ) would have to invoke these JIT compiled functions, yet the kernel shouldn't allow a program to execute code saved on the stack / heap / static memory segment, how could the JVM overcome this?
The memory region is both writable and executable, though I don't know exactly which system call is required to implement this.
Another question I have is regarding the Java stacks, when a method ( after JIT compilation )
Initially the code is not compiled but it uses the stack in the same way.
is executed within the processor it's local variables should be saved within the Java stacks, yet again the Java stacks may be implemented with a non-contiguous memory
There is a stack per thread which is continuous.
and perhaps even just some stack data structure allocated on the heap acting as a stack, how and where do the local variables of a method being executed get saved?
On the thread stack.
the kernel shouldn't allow a program to treat a heap allocated memory as a process stack, how does JVM overcome this difficuly as well?
It doesn't do this.
Is it possible (in Linux / OSX) to control how shared libraries are mapped in memory after loading?
Is it possible to malloc a contiguous region of memory, and pass that to a routine such as "dlopen"?
I do not insist on precise placement of the various segments (.text, .bss, etc.) within the malloc'd region, but only that all the segments are placed within the malloc'd region.
Is it possible (in Linux / OSX) to control how shared libraries are mapped in memory after loading?
I don't know much about OSX, so I'll answer only the Linux part.
Not without modifying the dynamic loader, no.
Is it possible to malloc a contiguous region of memory, and pass that to a routine such as "dlopen"?
malloc will not give you page-alignment required for that. posix_memalign will, but you will likely be better off just doing direct mmap(..., MAP_FIXED, ...). You are asking for implementation of this feature request (nobody's done it yet).
Is there anyway I can load a shared library into shared memory in a process so that some other process can simply map that shared memory (to the same address) and simply invoke functions? I understand that the external in the shared library need to have an additional jump into process-specific memory locations to call into appropriate functions (like elf plt). But, is such a thing viable with today's tools.
But, is such a thing viable with today's tools.
Not with today's tools, nor ever.
Sure, if your shared library has completely self-contained functions, then it will work. But the moment your library references external data or functions, you will crash and burn.
I understand that the external in the shared library need to have an additional jump into process-specific memory locations to call into appropriate functions
I don't think you understand. Let's consider an example:
void *foo() { return malloc(1); }
When this is built into a shared library on Linux, the result is:
0x00000000000006d0 <+0>: mov $0x1,%edi
0x00000000000006d5 <+5>: jmpq 0x5c0 <malloc#plt>
and
Dump of assembler code for function malloc#plt:
0x00000000000005c0 <+0>: jmpq *0x200a5a(%rip) # 0x201020 <malloc#got.plt>
0x00000000000005c6 <+6>: pushq $0x1
0x00000000000005cb <+11>: jmpq 0x5a0
So the question is: where will jmpq *0x200a5a(%rip) go in the second process. Answer: one of two places.
If the first process has already called malloc (very likely), then the jmpq will go to address of malloc in the first process, which is exceedingly unlikely to be the address of malloc in the second process, and more likely to be unmapped, or be in the middle of some data. Either way, you crash.
If the first process has not yet called malloc, then the jmpq in the second process will jump to address of the runtime loader (ld-linux.so.2 or similar on Linux, ld.so on Solaris) resolver function. Again, that address is very unlikely to also be the address of the resolver in the second process, and if it's not, you crash.
But it gets worse from here. If by some improbable magic you ended up actually calling malloc in the second process, that malloc is itself very likely to crash, because it will try to use data structures it has set up previously, using memory obtained from sbrk or mmap. These data structures are present in the first process, but not in the second, and so you crash again.
Having trouble with memory management of a third party library. I have the source but it's very complex (COM stuff), full of macros and those annoying Microsoft annotations, etc, and interacts with another library the source of which I don't have to boot. Now some quick debug runtimes have shown that it's leaking memory and in a pretty big way. I make extensive use of self-releasing pointers like unique_ptr and know that I released everything that I created. Is it my only option to try and clean up (and understand) the source?
In addition, is it safe to allocate COM objects with operator new, or do they have to go in the COM heap?
COM is quite agnostic about how you allocate your own COM objects. They get created by the class factory and your IUnknown::AddRef and Release methods keep the reference count. Using operator new in the class factory and delete this in Release is fine.
You do have to be careful about any pointers you return in your interface methods. Typical Automation objects like BSTR and SAFEARRAY do indeed need to be allocated in the COM heap so that the client code can release them. Pretty hard to mess that up, the API does the work.
The client code can be responsible for the leak, fumbling the reference count is a pretty standard COM bug. Usually easy to diagnose when you have access to the AddRef/Release implementations. Debugging this in Vista or Win7 is also strongly recommended, they have a much better heap manager that doesn't silently ignore attempts to free memory from the wrong heap.
If you're pretty sure it is the COM server that leaks then isolate the problem with the <crtdbg.h> header and unit tests to exercise the interface methods.
I am using Keil's ARM-MDK 4.11. I have a statically allocated block of memory that is used only at startup. It is used before the scheduler is initialised and due to the way RL-RTX takes control of the heap-management, cannot be dynamically allocated (else subsequent allocations after the scheduler starts cause a hard-fault).
I would like to add this static block as a free-block to the system heap after the scheduler is initialised. It would seem that __Heap_ProvideMemory() might provide the answer, this is called during initialisation to create the initial heap. However that would require knowledge of the heap descriptor address, and I can find no documented method of obtaining that.
Any ideas?
I have raised a support request with ARM/Keil for this, but they are more interested in questioning why I would want to do this, and offering alternative solutions. I am well aware of the alternatives, but in this case if this could be done it would be the cleanest solution.
We use the Rowley Crossworks compiler but had a similar issue - the heap was being set up in the compiler CRT startup code. Unfortunately the SDRAM wasn't initialised till the start of main() and so the heap wasn't set up properly. I worked around it by reinitialising the heap at the start of main(), after the SDRAM was initialised.
I looked at the assembler code that the compiler uses at startup to work out the structure - it wasn't hard. Subsequently I have also obtained the malloc/free source code from Rowley - perhaps you could ask Keil for their version?
One method I've used is to incorporate my own simple heap routines and take over the malloc()/calloc()/free() functions from the library.
The simple, custom heap routines had an interface that allowed adding blocks of memory to the heap.
The drawback to this (at least in my case) was that the custom heap routines were far less sophisticated than the built-in library routines and were probably more prone to fragmentation than the built-in routines. That wasn't a serious issue in that particular application. If you want the capabilities of the built-in library routines, you could probably have your malloc() defer to the built-in heap routines until it returns a failure, then try to allocate from your custom heap.
Another drawback is that I found it much more painful to make sure the custom routines were bug-free than I thought it would be at first glance, even though I wasn't trying to do anything too fancy (just a simple list of free blocks that could be split on allocation and coalesced when freed).
The one benefit to this technique is that it's pretty portable (as long as your custom routines are portable) and doesn't break if the toolchain changes it's internals. The only part that requires porting is taking over the malloc()/free() interface and making sure you get initialized early enough.