Looking for Information on dlltool --add-indirect (-a) - dll

I am looking for more information on the --add-indirect option for dlltool. When do you use this option? What does it do?
Information from binutils help on this option:
-a
--add-indirect
Specifies that when dlltool is creating the exports file it should add
a section which allows the exported functions to be referenced without
using the import library. Whatever the hell that means!

First of all, let's make it clear what is exports file. Exports file is needed for creation of DLL. This file is linked with the object files (produced by compiler) that make up the body of the DLL (i.e. functions, classes, etc.) and it handles the interface between the DLL and the outside world. This is a binary file and it can be created by giving the -e option to dlltool when it is creating or reading in *.def file.
The next term that you have to understand is import library. One of the ways to employ DLL by some consumer application is to link this application against the DLL, so that all the exported functionality from DLL is available to the consumer. Such linking to DLLs is usually done by linking to an import library which is essentially an auxiliary static library that contains an import address table (IAT) which is there to allow consumer to reference all the DLL exported functionality. For example, each referenced DLL function contains its own entry in IAT. At runtime, the IAT is filled with appropriate addresses that point directly to corresponding functions in the separately loaded DLL.
Now let's manually create a DLL with dlltool and gcc to give you the feeling what's going on:
gcc -c library.c
produces library.o,
dlltool -e exports.o -l library.dll.a library.o
produces exports file exports.o and import library library.dll.a (.dll.a is conventional suffix for import libraries produced by GCC which emphasizes that the import library is, in fact, static with .a, but is aimed at DLL with .dll),
gcc library.o exports.o -o library.dll
produces library.dll,
gcc consumer.o library.dll.a -o consumer
produces executable consumer.exe which is linked against library.dll.
NOTE: The above is a manual procedure of creating the DLL, and it's discouraged to do that in production because GCC wraps all of that logic in one optimized call:
gcc -shared -o library.dll library.o -Wl,--out-implib,library.dll.a
Back on track, now that we know the basic terminology and purpose, we can easily interpret what is written in help about --add-indirect:
Specifies that when dlltool is creating the exports file it should add
a section which allows the exported functions to be referenced without
using the import library. Whatever the hell that means!
Let's apply that to the previous example. In this case, exports.o will already contain IAT, and therefore resulting library.dll will also contain that information, so we don't need import library library.dll.a because now we can directly link to library.dll itself:
gcc consumer.o library.dll -o consumer
Whether it's useful or not is quite subjective question to ask. I guess from the point of view of us (programmers/users) it's pretty much useless since DLL creation and linkage shouldn't be done explicitly (i.e. through direct invocation of dlltool) anyway, but should rather be done through GCC front end (as noted above). From the point of view of building development tools such as toolchains (like GCC itself) this might be useful since something similar to the above example may actually be used behind the scenes by GCC itself to perform gcc -shared -o library.dll ... and etc.
Finally, it is generally discouraged to link against DLL directly. Although, it works fine with latest versions of MinGW/MinGW-w64, it has been known to have bugs in the past. Furthermore, if pseudo-relocation is disabled, then direct linkage with DLL might result in certain runtime issues. Also, this is the official way MSVC links consumers against DLLs, i.e. without an import library, MSVC simply can't do the linkage, what could also be the reason why you should prefer to always use import libraries. Remember DLL is not the same as shared object (SO) on Linux: their use cases are the same, but their implementations are based on different technologies.

Related

What does the -specs argument do in arm-none-eabi-gcc?

I was having trouble with the linker for the embedded arm gcc compiler, and I found a tutorial somewhere online saying that I could fix my linker errors in arm-none-eabi-gcc by including the argument -specs=nosys.specs, which worked for me, and it was able to compile my code.
My chip is an ATSAM7SE256 microcontroller, which to my understanding is an arm7tdmi processor using the armv4t and thumb instruction sets, and I've been compiling my code using:
arm-none-eabi-gcc -march=armv4t -mtune=arm7tdmi -specs=nosys.specs -o <exe_name>.elf <input_files>
And the code compiles with no issue, but I have no idea if it's doing what I think it's doing.
What is the significance of a spec file? What other values can you set with -specs=, and in what situations would you want to? Is nosys.specs the value I want for a completely embedded arm microcontroller?
It is documented at: https://gcc.gnu.org/onlinedocs/gcc-11.1.0/gcc/Overall-Options.html#Overall-Options
It is a file containing switches to override standard defaults for various build components such as the compiler, assembler and linker. For example it can be used to replace the default C library.
I have never seen it used; typically bare-metal embedded system builds explicitly specify --nostdlib then explicitly link the required library. It could be used for environment specific build environments to link other default code such as an RTOS I guess. Personally I'd rather make all that explicit on the command line that hiding it in a file somewhere.
Essentially it applies the switches specified in the file as if they were defaults, so can be used to define defaults for specific build and execution environments.
The format of the specs file is documented at https://gcc.gnu.org/onlinedocs/gcc-11.1.0/gcc/Spec-Files.html#Spec-Files
Without seeing both the linker errors and the content of the nosys.specs file in this case it is difficult to say how or why it solved your linker problem. The alternative solution of course would be to apply whatever switches are in the specs file directly.

Making a module work like an intrinsic Fortran module

I have a module module1 in a file called mymodule.f90. What should I do in order to make module1 usable like fortran intrinsic module?, i.e. it need only be called in a use statement (use module1) in any programs, subroutines, or functions that use it but I don't need to link /path/to/mymodule/ when compiling those procedures.
I use gfortran, but possibly in the future I will also have to use the Intel fortran compiler.
So maybe I'm misunderstanding you, but you want to use a module without having to tell the compiler where to find the .mod file (that contains the interface definitions for whatever module1 exports), or the linker where the object code can be found?
If so, for GFortran the solution is to download the GCC source code, add your own module as an intrinsic module, and then build your own custom version of GFortran. As a word of warning, unless you're familiar with the GFortran/GCC internals, while this isn't rocket science, it isn't trivial either.
For Intel Fortran, where you presumably don't have access to the source code of the compiler, I suppose you're out of luck.
My suggestion is to forget about this, and instead tell the compiler/linker where your .mod files and object files can be found. There are tools like make, cmake etc. that can help you automate this.
When you compile mymodule.f90 you will obtain an object file (mymodule.o) and a module file (mymodule1.mod). The compiler needs to have access to the module file when it compiles other files that use mymodule1, and the linker needs to have access to the object file when it generates the binary.
You don't need to specify the location of intrinsic modules because they are built in into the compiler. That will not be the case with your modules: you may be able to set up your environment in a way that the locations of your files allow the compiler to find the files without explicitly specifying their paths in compilation or linking commands, but the fact that you don't see it does not mean it's not happening.
For the Intel compiler, the answer is given by https://software.intel.com/en-us/node/694273 :
Directories are searched for .mod files in this order:
1 Directory of the source file that contains the USE statement.
2 Directories specified by the module path compiler option.
3 Current working directory.
4 Directories specified by the -Idir (Linux* and OS X*) or /include (Windows*) option.
5 Directories specified with the CPATH or INCLUDE environment variable.
6 Standard system directories.
For gfortran I have not found such a clear ordered list, but relevant information can be found in
https://gcc.gnu.org/onlinedocs/gfortran/Directory-Options.html
https://gcc.gnu.org/onlinedocs/gcc/Environment-Variables.html
https://gcc.gnu.org/onlinedocs/gcc/Directory-Options.html#Directory-Options
It should be clear to you that a compiler won't be able to understand module files created by other compilers, or even by different enough versions of the same compiler. Therefore, you would need a copy of your "always available" module for each compiler you use, and if you are using multiple versions of a compiler you may need up to one per version - each of them in a different directory to avoid errors.
As you can see, this is not particularly practical, and it is indeed far from common practice. It is usually easier and more clear to the user to specify the path to the relevant module file in the compilation command. This is quite easy to set up if you compile your code using tools such as make.
Finally, remember that, if you make such arrangements for module files, you will also need to make arrangements for the corresponding object files at the linking stage.

How to reuse Fortran modules without copying source or creating libraries

I'm having trouble understanding if/how to share code among several Fortran projects without building libraries or duplicating source code.
I am using Eclipse/Photran with the Intel compiler (ifort) on a linux system, but I believe I'm having a bigger conceptual problem with modules than with the specific tools.
Here's a simple example: In ~/workspace/cow I have a source directory (src) containing cow.f90 (the PROGRAM) and two modules m_graze and m_moo in m_graze.f90 and m_moo.f90, respectively. This project builds and links properly to create the executable 'cow'. The executable and modules (m_graze.mod and m_moo.mod) are stored in ~/workspace/cow/Debug and object files are stored under ~/workspace/cow/Debug/src
Later, I create ~/workplace/sheep and have src/sheep.f90 as the program and src/m_baa.f90 as the module m_baa. I want to 'use m_graze, only: ruminate' in sheep.f90 to get access to the ruminate() subroutine. I could just copy m_graze.f90 but that could lead to code getting out of sync and doesn't take into account any dependencies m_graze might have. For these reasons, I'd rather leave m_graze in the cow project and compile and link sheep.f90 against it.
If I try to compile the sheep project, I'll get an error like:
error #7002: Error in opening the compiled module file. Check INCLUDE paths. [M_GRAZE]
Under Properties:Project References for sheep, I can select the cow project. Under Properties:Fortran Build:Settings:Intel Compiler:Preprocessor I can add ~/workspace/cow/Debug (location of the module files) to the list of include directories so the compiler now finds the cow modules and compiles sheep.f90. However the linker dies with something like:
Building target: sheep
Invoking: Intel(R) Fortran Linker
ifort -L/home/me/workspace/cow/Debug -o "sheep" ./src/sheep.o
./src/sheep.o: In function `sheep':
/home/me/workspace/sheep/src/sheep.f90:11: undefined reference to `m_graze_mp_ruminate_'
This would normally be solved by adding libraries and library paths to the linker settings except there are no appropriate libraries to link to (this is Fortran, not C.)
The cow project was perfectly capable of compiling and linking together cow.f90, m_graze.f90 and m_moo.f90 into an executable. Yet while the sheep project can compile sheep.f90 and m_baa.f90 and can find the module m_graze.mod, it can't seem to find the symbols for m_graze even though all the requisite information is present on the system for it to do so.
It would seem to be an easy matter of configuration to get the linker portion of ifort to find the missing pieces and put them together but I have no idea what magic words need to be entered where in the Photran UI to make this happen.
I confess an utter lack of interest and competence in C and the C build process and I'd rather avoid the diversion of creating libraries (.a or .so) unless that's the only way to make this work.
Ultimately, I'm looking for a pure Fortran solution to this problem so I can keep a single copy of the source code and don't have to manually maintain a pile of custom Makefiles.
So can this be done?
Apologies if this has already been documented somewhere; Google is only showing me simple build examples, how to create modules, and how to link with existing libraries. There don't seem to be (m)any examples of code reuse with modules that don't involve duplicating source code.
Edit
As respondents have pointed out, the .mod files are necessary but not sufficient; either object code (in the form of m_graze.o) or static or shared libraries must be specified during the linking phase. The .mod files describe the interface to the object code/library but both are necessary to build the final executable.
For an oversimplified toy problem such as this, that's sufficient to answer the question as posed.
In a larger project with more complex dependencies (in my case, 80+KLOC of F90 linking to the MKL version of LAPACK95), the IDE or toolchain may lack sufficient automatic or user-interface facilities to make sharing a single canonical set of source files a viable strategy. The choice seems to be between risking duplicate source files getting out of sync, giving up many of the benefits of an IDE (i.e. avoiding manual creation of make/CMake/SCons files), or, in all likelihood, both. While a revision control system and good code organization can help, it's clear that sharing a single canonical set of source files among projects is far from easy given the current state of Eclipse.
Some background which I suspect you already know: Typically (including ifort) compiling the source code for a Fortran module results in two outputs - a "mod" file that contains a description of the Fortran entities that the module defines that the compiler needs to find whenever it sees a USE statement for the module, and object code for the linker that implements the procedures and variable storage, etc., that the module defines.
Your first error (the one you solved) is because the compiler couldn't find the mod file.
The second error is because the linker hasn't been told about the object code that implements the stuff that was in the source file with the module. I'm not an Eclipse user by any means, but a brute force way of specifying that is just to add the object file (xxxxx/Debug/m_graze.o) as an additional linker option (Fortran Build > Settings, under Intel Fortran Linker > Command Line). (Other tool chains have explicit "additional object file" properties for their link stage - there may well be a better way of doing this for the Intel chain.)
For more involved examples you would typically create a library out of the shared code. That's not really C specific, the only Fortran aspect is that the libraries archive of object code needs to be provided alongside the mod files that the Fortran compiler generates.
Yes the object code must be provided. E.g., when you install libnetcdf-dev in Debian (apt-get install libnetcdf-dev), there is a /usr/include/netcdf.mod file that is included.
You can now use all netcdf routines in your Fortran code. E.g.,
program main
use netcdf
...
end
but you'll have link to the netcdf shared (or static) library, i.e.,
gfortran -I/usr/include/ main.f90 -lnetcdff
However, as user MSB mentioned the mod file can only be used by gfortran that comes with the distribution (apt-get install gfortran). If you want to use any other compiler (even a different version that you may have installed yourself) then you'll have to build netcdf yourself using that particular compiler.
So creating a library is not a bad solution.

How do I statically include a library in g++?

As mentioned in the title, this is a real beginner's question.
I'm realizing that after several years of CS courses and projects and such, I have never actually needed to export an executable that someone else could run without compiling the source manually (which is what most/all professors/TAs do, since they want to see your source code anyway).
So my question is basically this:
When I compile some basic C++ code (e.g. "Hello World" code), I always seem to need some sort of external DLLs to run it.
Visual Studio needs the .NET framework.
Cygwin needs Cygwin.dll.
MinGW needs libgcc_s_dw2-1.dll or something similar.
So how do I simply compile an executable such that I (or someone I give the file to) can just double-click it and have it run? I'm guessing there are some fancy command line flags I can use on g++ to statically link the DLLs; I have simply never needed to do this before.
As I said twice, this is a super beginner question, and yet I could not find (easily, anyway) an answer to this question, StackOverflow or anywhere else. Largely, I think, because the search terms are so commonly used in the descriptions for other problems.
Anyway, all help is appreciated.
EDIT:
I'm literally talking about a Hello World program. e.g.:
HelloWorld.cpp:
#include <iostream>
using namespace std;
int main()
{
cout << "Hello World!" << endl;
return 0;
}
Try g++ -static -static-libgcc
However, you should generally avoid static linking. Almost all the utilities on a usual Linux machine are dynamically linked: you can use file or ldd to know if an executable is statically linked. On my Debian/Sid system /usr/bin/ contains 7303 files, but only one is statically linked (it is /usr/bin/rar).
Using dynamic linking is better, because it uses less disk space, and most importantly because dynamic libraries are updated, and that profit to all the executables dynamically linking them.
If you care about dependencies, use your package manager (or distribute your program as a package).
If you already got the static lib you need to link in, (.lib or .a), just put them together with the object files when you linking your application, like
g++ foo.o libfoo.a -o foo
I think your real problem is how to locate which static libs are required by your application and how to get them, maybe you should post your code here and let us know what library you need to link in.
Here is how:
simply put, you specify to link all the standard stuff into your executable.
g++ -o c:\prog.exe c:\HelloWorld.cpp -static-libgcc -static-libstdc++
hope it helps.

.h generated from .h.in?

There are struct definitions in the .h file that my library creates after I build it.. but I cannot find these in the corresponding .h.in. Can somebody tell me how all this works and where it gets the extra info from?
To be specific: I am building pth, the userspace threading library. It has pth_p.h.in, which doesn't contain the struct definition I am looking for, yet when I build the library, a pth_p.h appears and it has the definition I need.
In fact, I have searched every single file in the library before it is built and cannot find where it is generating the struct definition.
Pth uses GNU Autoconf, Automake, and Libtool. By running ./configure you'll be running a shell script which eventually runs m4 to detect the presence of a whole bunch of different system attributes and make changes to a number of files.
It looks like it boils down to ./configure generating Makefile from Makefile.in and then running something via make that triggers the shtool subcommand scpp:
pth_p.h: $(S)pth_p.h.in
$(SHTOOL) scpp -o pth_p.h -t $(S)pth_p.h.in -Dcpp -Cintern -M '==#==' $(HSRCS)
Obscure link, but here's an shtool-scpp manpage, which describes it as:
This command is an additional ANSI C
source file pre-processor for sharing
cpp(1) code segments, internal
variables and internal functions. The
intention for this comes from writing
libraries in ANSI C. Here a common
shared internal header file is usually
used for sharing information between
the library source files.
The operation is to parse special
constructs in files, generate a few
things out of these constructs and
insert them at position mark in tfile
by writing the output to ofile.
Additionally the files are never
touched or modified. Instead the
constructs are removed later by the
cpp(1) phase of the build process. The
only prerequisite is that every file
has a ``"#include ""ofile"""'' at the
top.
.h.in is probably processed within a configure (generated from configure.ac) script, look out for
AC_CONFIG_FILES([thatfile.h])
It replaces variables of the form #VAR# in the .in file with their values.
Edit: Just noticed if I'm right you should retag your question