Am I missunderstanding the meaning of CMAKE_REQUIRED_LIBRARIES?
Let's say, for building my project, I want to check that certain library exists, e.g. cstring. So, I check it in CMakeLists.txt:
include(CheckCXXSourceCompiles)
set(CMAKE_REQUIRED_LIBRARIES cstring)
check_cxx_source_compiles(
"#include <cstring>
int main(){ return 0; }"
HAVE_CSTRING)
This check fails as it is. If I take out the line set(CMAKE_REQUIRED_LIBRARIES cstring), it passes. As it should, I have the library in the system.
The documentation says CMAKE_REQUIRED_LIBRARIES: "A list of libraries to add to the link command. These can be the name of system libraries or they can be Imported Targets".
I have read that this list of libraries only affect the check, i.e. check_cxx_source_compiles(). And once set, these are the ONLY libraries that check_cxx_source_compiles() will see.
So, if I wanted to include cstring in the check, wouldn't it make sense that I made available the library for linking? Also, I would be isolating only that library for the check because that is the only one that I care about in this specific case.
How come the check fails if I set CMAKE_REQUIRED_LIBRARIES?
Related
I have big project with cmake. It mostly works.
But recently some combination of compilation server vs test server broke. Investigation found that final compile/link command calls gcc (...) -licudata -licui18n -licuuc (...), this introduces dependency on shared library which is not present on test server.
How do I find out what in my project (my library, imported library, found library, whatever) adds those 3 flags to compile command?
I don't add them explicitly, so something is done automagically and I want to find it. compile_commands.json doesn't have them because linking flags don't belong in it. CMakeCache.txt has those flags in some obscure variable PC_LIBXML_STATIC_LIBRARIES:INTERNAL but removing them there doesn't affect compile/link command.
Note that this question is not about dealing with libicu specifically but about a method for investigation in general (though comments about eventual known problems with libicu would be appreciated too).
I found out that dependency graphs created by cmake can have more details that was configured for our project. Here are all options: https://cmake.org/cmake/help/latest/module/CMakeGraphVizOptions.html I expect GRAPHVIZ_EXTERNAL_LIBS, GRAPHVIZ_SHARED_LIBS are most important to set to true.
We enabled everything that was possible to enable, filtered out nothing and resulting graph was massive (to big for xdot - luckily .dot files are human readable), but showed that Boost::regex uses those 3 libraries.
FindProtobuf.cmake for Google Protocol Buffers creates a set of targets, for example protobuf::libprotobuf-lite one.
Now, in my project, I'm using this protobuf::libprotobuf-lite in target_link_libraries. And this works fine.
However, after upgrading to protobuf v3 I noticed that generated files have small blocks guarded by
#if LANG_CXX11
Yet, as it seems, this LANG_CXX11 is not defined by FindProtobuf.cmake. It seems I have to define it on my own if I want to use C++11 features with protobuf-generated code.
Obviously, I could just do it globally and be done with it. However, I would much more prefer to somehow "bind" the LANG_CXX11 with the protobuf::libprotobuf-lite target, so that I will get it whenever I used it and only where I use it.
target_compile_definitions will not work here since it doesn't accept imported targets while targets created by FindProtobuf.cmake are imported ones. And anyway, it seems like not the right thing to do.
The only idea I have for now (although not yet tested) is to create another custom target that would only link (PUBLIC) with protobuf::libprotobuf-lite and add the LANG_CXX11 define (PUBLIC as well). Then I would use this custom target instead of protobuf::libprotobuf-lite. But it doesn't feel good either.
How to do it properly with CMake? (I'm on version 3.8 if it matters.)
I'm having trouble understanding if/how to share code among several Fortran projects without building libraries or duplicating source code.
I am using Eclipse/Photran with the Intel compiler (ifort) on a linux system, but I believe I'm having a bigger conceptual problem with modules than with the specific tools.
Here's a simple example: In ~/workspace/cow I have a source directory (src) containing cow.f90 (the PROGRAM) and two modules m_graze and m_moo in m_graze.f90 and m_moo.f90, respectively. This project builds and links properly to create the executable 'cow'. The executable and modules (m_graze.mod and m_moo.mod) are stored in ~/workspace/cow/Debug and object files are stored under ~/workspace/cow/Debug/src
Later, I create ~/workplace/sheep and have src/sheep.f90 as the program and src/m_baa.f90 as the module m_baa. I want to 'use m_graze, only: ruminate' in sheep.f90 to get access to the ruminate() subroutine. I could just copy m_graze.f90 but that could lead to code getting out of sync and doesn't take into account any dependencies m_graze might have. For these reasons, I'd rather leave m_graze in the cow project and compile and link sheep.f90 against it.
If I try to compile the sheep project, I'll get an error like:
error #7002: Error in opening the compiled module file. Check INCLUDE paths. [M_GRAZE]
Under Properties:Project References for sheep, I can select the cow project. Under Properties:Fortran Build:Settings:Intel Compiler:Preprocessor I can add ~/workspace/cow/Debug (location of the module files) to the list of include directories so the compiler now finds the cow modules and compiles sheep.f90. However the linker dies with something like:
Building target: sheep
Invoking: Intel(R) Fortran Linker
ifort -L/home/me/workspace/cow/Debug -o "sheep" ./src/sheep.o
./src/sheep.o: In function `sheep':
/home/me/workspace/sheep/src/sheep.f90:11: undefined reference to `m_graze_mp_ruminate_'
This would normally be solved by adding libraries and library paths to the linker settings except there are no appropriate libraries to link to (this is Fortran, not C.)
The cow project was perfectly capable of compiling and linking together cow.f90, m_graze.f90 and m_moo.f90 into an executable. Yet while the sheep project can compile sheep.f90 and m_baa.f90 and can find the module m_graze.mod, it can't seem to find the symbols for m_graze even though all the requisite information is present on the system for it to do so.
It would seem to be an easy matter of configuration to get the linker portion of ifort to find the missing pieces and put them together but I have no idea what magic words need to be entered where in the Photran UI to make this happen.
I confess an utter lack of interest and competence in C and the C build process and I'd rather avoid the diversion of creating libraries (.a or .so) unless that's the only way to make this work.
Ultimately, I'm looking for a pure Fortran solution to this problem so I can keep a single copy of the source code and don't have to manually maintain a pile of custom Makefiles.
So can this be done?
Apologies if this has already been documented somewhere; Google is only showing me simple build examples, how to create modules, and how to link with existing libraries. There don't seem to be (m)any examples of code reuse with modules that don't involve duplicating source code.
Edit
As respondents have pointed out, the .mod files are necessary but not sufficient; either object code (in the form of m_graze.o) or static or shared libraries must be specified during the linking phase. The .mod files describe the interface to the object code/library but both are necessary to build the final executable.
For an oversimplified toy problem such as this, that's sufficient to answer the question as posed.
In a larger project with more complex dependencies (in my case, 80+KLOC of F90 linking to the MKL version of LAPACK95), the IDE or toolchain may lack sufficient automatic or user-interface facilities to make sharing a single canonical set of source files a viable strategy. The choice seems to be between risking duplicate source files getting out of sync, giving up many of the benefits of an IDE (i.e. avoiding manual creation of make/CMake/SCons files), or, in all likelihood, both. While a revision control system and good code organization can help, it's clear that sharing a single canonical set of source files among projects is far from easy given the current state of Eclipse.
Some background which I suspect you already know: Typically (including ifort) compiling the source code for a Fortran module results in two outputs - a "mod" file that contains a description of the Fortran entities that the module defines that the compiler needs to find whenever it sees a USE statement for the module, and object code for the linker that implements the procedures and variable storage, etc., that the module defines.
Your first error (the one you solved) is because the compiler couldn't find the mod file.
The second error is because the linker hasn't been told about the object code that implements the stuff that was in the source file with the module. I'm not an Eclipse user by any means, but a brute force way of specifying that is just to add the object file (xxxxx/Debug/m_graze.o) as an additional linker option (Fortran Build > Settings, under Intel Fortran Linker > Command Line). (Other tool chains have explicit "additional object file" properties for their link stage - there may well be a better way of doing this for the Intel chain.)
For more involved examples you would typically create a library out of the shared code. That's not really C specific, the only Fortran aspect is that the libraries archive of object code needs to be provided alongside the mod files that the Fortran compiler generates.
Yes the object code must be provided. E.g., when you install libnetcdf-dev in Debian (apt-get install libnetcdf-dev), there is a /usr/include/netcdf.mod file that is included.
You can now use all netcdf routines in your Fortran code. E.g.,
program main
use netcdf
...
end
but you'll have link to the netcdf shared (or static) library, i.e.,
gfortran -I/usr/include/ main.f90 -lnetcdff
However, as user MSB mentioned the mod file can only be used by gfortran that comes with the distribution (apt-get install gfortran). If you want to use any other compiler (even a different version that you may have installed yourself) then you'll have to build netcdf yourself using that particular compiler.
So creating a library is not a bad solution.
I have created a static library containing all my generic classes. Some of these classes use frameworks.
Now I have two projects, one that uses some classes that use frameworks, and one that doesn't use any of the classes that use frameworks.
Because Static Libraries don't support including frameworks (if I am correct). I have to include the frameworks in the project that uses them. But when I compile the project that doesn't use any of the framework-classes the compiler breaks because it still requires the frameworks. Now I know it tries to compile all the (unused) classes from the library because I use the Linker Flag '-ObjC' to prevent 'unrecognized selector' errors.
Does anyone know how to compile only the required source files per project? And prevent from all frameworks having to be included in all projects that use my static library?
First of all, you are right in that a static library cannot include any framework nor other static libraries, it is just the collection of all object files (*.obj) that make up that specific static library.
Does anyone know how to compile only the required source files per project?
The linker will by default only link in object files from the static library that contain symbols referenced by the application. So, if you have two files a.m and b.m in your static library and you only use symbols from a.m in your main program, then b.o (the object file generated from b.c) will not appear in your final executable. As a sub-case, if b.m uses a function/class c which is only declared (not implemented), then you will not get any linker errors. As soon as you include some symbols from b.m in your program, b.o will also be linked and you will get linker errors due to the missing implementation of c.
If you want this kind of selection to happen at symbol rather than at object level granularity, enable dead code stripping in Xcode. This corresponds to the gcc option -Wl,-dead_strip (= linker option -dead_strip in the Build settings Info pane for your project). This would ensure further optimization.
In your case, though, as you correctly say, it is the use of the "-ObjC" linker flag that defeats this mechanism. So this actually depends on you. If you remove the -Objc flag, you get the behavior you like for free, while losing the stricter check on selectors.
And prevent from all frameworks having to be included in all projects that use my static library?
Xcode/GCC support an linking option which is called "weak linking", which allows to lazily load a framework or static library, i.e., only when one of its symbols is actually used.
"weak linking" can be enabled either through a linker flag (see Apple doc above), or through Xcode UI (Target -> Info -> General -> Linked Libraries).
Anyhow, the framework or library must be available in all cases at compile/link time: the "weak" option only affects the moment when the framework is first loaded at runtime. Thus, I don't think this is useful for you, since you would need anyway to include the framework in all of your projects, which is what you do not want.
As a side note, weak_linking is an option that mostly make sense when using features only available on newer SDK version (say, 4.3.2) while also supporting deployment on older SDK versions (say, 3.1.3). In this case, you rely on the fact that the newer SDK frameworks will be actually available on the newer deployment devices, and you conditionally compile in the features requiring it, so that on older devices they will not be required (and will not produce thus the attempt at loading the newer version of the framework and the crash).
To make things worse, GCC does not support a feature known as "auto-linking" with Microsoft compilers, which allow to specify which library to link by means of a #pragma comment in your source file. This could offer a workaround, but is not there.
So, I am really sorry to have to say that you should use a different approach that could equally satisfy your needs:
remove the -ObjC flag;
split your static library in two or more parts according to their dependencies from external frameworks;
resort to including the source files directly.
Abour second part of your question, you can mark a linked framework as Optional :
About first part, it is not clear to me what you intend to do:
A library being declared in a project
A project declaring which files are compiled (via Target > Build phases > Compile sources)
Unless setting complex build rules to include or not files, which if I remember well can be done using .xcconfig files, I don't see any other solutions than splitting your Library. Which I would recommend, for its ease. You should even do several targets in the same project... You could also just use precompiler MACROS (#ifdef...) but that depends on what you want to do.
It sounds like you have library bloat. To keep things small I think you need to refactor your library into separate libraries with minimal dependencies. You could try turning on "Dead Code Stripping" in the "Linker Flags" section of the build target info (Xcode 3.x) to see if that does what you want (doesn't require frameworks used by classes that are dead-stripped.)
When you link against a framework on iOS I don't think that really adds any bloat since the framework is on the device and not in your application. But your library is still a bit bloated by having entire classes that never get used but are not stripped out of the library.
A static library is built before your app is compiled, and then the whole thing is linked into your app. There's no way to include some parts of the library but not others -- you get the whole enchilada.
Since you have the source code for the library, why not just add the code directly to each application? That way you can control exactly what goes into each app. You can still keep your generic classes together in the same location, and use the same code in both apps, but you avoid the hassle of using a library.
What is standard or "most-popular" naming convention for MSVC library builds.
For example, for following platforms library foo has these conventions:
Linux/gcc:
shared: libfoo.so
import: ---
static: libfoo.a
Cygwin/gcc:
shared: cygfoo.dll
import: libfoo.dll.a
static: libfoo.a
Windows/MinGW:
shared: libfoo.dll
import: libfoo.dll.a
static: libfoo.a
What should be used for MSVC buidls? As far as I know, usually names are foo.dll and foo.lib, but how do you usually distinguish between import library and static one?
Note: I ask because CMake creates quite unpleasant collision between them naming both import and static library as foo.lib. See bug report. The answer would
help me to convince the developers to fix this bug.
You distinguish between a library and a .dll by the extension. But you distinguish between a import library and a static library by the filename, not the extension.
There will be no case where an import library exists for a set of code that was built to be a static library, or where a static library exists for a dll. These are two different things.
There is no single MSVC standard filename convention. As a rule, a library name that ends in "D" is often a debug build of library code, msvcrtd.dll vs msvcrt.dll but other than that, there are no standards.
As mentioned by others, there are no standards, but there are popular conventions. I'm unsure how to unambiguously judge what is the most popular convention. In addition the nomenclature for static vs. import libraries, which you asked about, there is also an analogous distinction between the naming of Release libraries vs. Debug libraries, especially on Windows.
Both cases (i.e. static vs. import, and debug vs. release) can be handled in one of two ways: different names, or different directory locations. I usually choose to use different names, because I feel it minimizes the chance of mistaking the library type later, especially after installation or other file moving activities.
I usually use foo.dll and foo.lib for the shared library on Windows, and foo_static.lib for the static library, when I wish to have both shared and static versions. I have seen others use this convention, so it might be the "most popular".
So I would recommend the following addition to your table:
Windows/MSVC:
shared: foo.dll
import: foo.lib
static: foo_static.lib
Then in cmake, you could either
add_library(foo_static STATIC foo.cpp)
or
add_library(FooStatic STATIC foo.cpp)
set_target_properties(FooStatic PROPERTIES OUTPUT_NAME "foo_static")
if for some reason you don't wish to use "foo_static" as the symbolic library name.
There is no standard naming convention for libraries. Traditional library names are prefixed with lib. Many linkers have options to prepend lib to a library name on the command line.
The static and dynamic libraries are usually identified by their file extension; although this is not required. So libmath.a would be a static library whereas libmath.so or libmath.dll would be a dynamic library.
A common naming convention is to append the category of the library to the name. For example, a debug static math library would be 'libmathd.a' or in Windows, 'lib_math_debug'. Some shops also add Unicode as a filename attribute.
If you want, you can append _msvc to the library name to indicate the library requires or was created by MSVC (to differentiate from GCC and other tools). A popular convention when working with multiple platforms, is to place the objects and libraries in platform specific folders. For example a ./linux/ folder would contain objects and libraries for Linux and similarly ./msw/ for Microsoft Windows platform.
This is a style issue. Style issues are often treated like religious issues: none of them are wrong, there is no universal style, and they are an individual preference. What ever system you choose, just be consistent.
As far as I know, there's no real 'standard', at least no standard most software would conform to.
My convention is to name my dynamic and static .lib equally, but place them in different directories if a project happens to support both static and dynamic linkage. For example:
foo-static
foo.lib
foo
foo.lib
foo.dll
The library to link against depends on the choice of the library directories, so it's almost totally decoupled from the rest of the build process (it won't appear in-source if you use MSVC's #pragma comment(lib,"foo.lib") facility, and it doesn't appear in the list of import libraries for the linker).
I've seen this quite a few times. Also, I think that MSVC/Windows based projects tend to stick more often with a single, official linkage type - either static, or dynamic. But that's just my personal observation.
In short:
Windows/MSVC
shared: foo.dll
import: foo.lib
static: foo.lib
You should be able to use this directory-based pattern with CMAKE (never used it). Also, I don't think it's a 'bug'. It's merely lack of standardization. CMAKE does (imho) the right thing not to establish a pseudo-standard if everyone likes it differently.
As the others have said, there is no single standard to file naming on windows.
For our complete product base which covers 100's of exes, dlls, and static libs we have used the following successfully for many years now and it has saved a lot of confusion. Its basically a mixing of several methods I've seen used throughout the years.
In a nutshell all our files of both a prefix and suffix (not including the extension itself). They all start with "om" (based on our company name), and then have a 1 or 2 character combination that roughly identifies the area of code.
The suffix explains what type of built-file they are and includes up to three letters used in combination depending on the build which includes Unicode, Static, Debug (Dll builds are the default and have no explicit suffix identifier). When we started this system Unicode was not so prevalent and we had to support both Unicode and Non-unicode builds (pre Windows 2000 os), now everything is exclusively built unicode but we still use the same nomenclature.
So a typical .lib "set" of files might look like
omfThreadud.lib (Unicode/Debug/Dll)
omfThreadusd.lib (Unicode/Static/Debug)
omfThreadu.lib (Unicode/Release/Dll)
omfThreadus.lib (Unicode/static)
All files are built-in into a common bin folder, which eliminates a lot of dll-hell issues for developers and also makes it simpler to adjust compiler/linker settings - they all point to the same location using relative paths and there is never any need for manual (or automatic) copying of the libraries a project needs. Having these suffixes also eliminates any confusion as to what type of file you may have, and guarantees you can't have a mixed scenario where you put down the debug dll on a release kit or vice-versa. All exes also use a similar suffix (Unicode/Debug) and build into the same bin folder.
There is likewise one single "include" folder, each library has one header file in the include folder that matches the name of the library/dll (for example omfthread.h) That file itself #includes all the other items that are exposed by that library. This keeps its simpler if you want functionality that is in foo.dll you just #include "foo.h"; our libraries are highly segmented by areas of functionality - effectively we don't have any "swiss-army knife" dlls so including the libraries entire functionality makes sense. (Each of these headers also include other prerequisite headers whether they be our internal libraries or other vendor SDKs)
Each of these include files internally uses macros that use #pramga's to add the appropriate library name to the linker line so individual projects don't need to be concerned with that. Most of of our libraries can be built statically or as a DLL and #define OM_LINK_STATIC (if defined) is used to determine which the individual project wants (we usually use the DLLs but in some cases static libraries built-in into the .exe make more sense for deployment or other reasons)
#if defined(OM_LINK_STATIC)
#pragma comment (lib, OMLIBNAMESTATIC("OMFTHREAD"))
#else
#pragma comment (lib, OMLIBNAME("OMFTHREAD"))
#endif
These macros (OMLIBNAMESTATIC & OMLIBNAME) use _DEBUG determine what type of build it is and generate the proper library name to add to the linker line.
We use a common define in the static & dll versions of a library to control proper exporting of the class/functions in dll builds. Each class or function exported from the library is decorated with this macro (the name of which matches the base name for the library, though that is largely unimportant)
class OMUTHREAD_DECLARE CThread : public CThreadBase
In the DLL version of the project settings we define OMFTHREAD_DECLARE=__declspec(dllexport), in the static library version of the library we define OMFTHREAD_DECLARE as empty.
In the libraries header file we define it based on how the client is trying to link to it
#if defined(OM_LINK_STATIC)
#define OMFTHREAD_DECLARE
#else
#define OMFTHREAD_DECLARE __declspec(dllimport)
#endif
A typical project that wants to use one of our internal libraries would just add the appropriate include to their stdafx.h (typically) and it just works, if they need to link against the static version they just add OM_LINK_STATIC to their compiler settings (or define it in the stdafx.h) and it again it just works.
As far as I know there still aren't any conventions with regards to this. Here's an example of how I do it:
{Project}{SubModule}{Platform}{Architecture}{CompilerRuntime}_{BuildType}.lib/dll
The full filename shall be lowercase only and shall only contain alphanumerics with predesignated underscores. The submodule field, including its leading underscore, is optional.
Project: holds project name/identifier. Preferably as short as possible. ie "dna"
SubModule: optional. holds module name. Preferably as short as possible. ie "dna_audio"
Platform: identifies the platform the binary is compiled for. ie "win32" (Windows), "winrt", "xbox", "android".
Architecture: describes the architecture the binary is compiled for. ie "x86", "x64", "arm". There where architecture names are equal for various bitnesses use its name followed by the bitness. ie. "name16", "name32", "name64"
CompilerRuntime: optional. Not all binaries link to a compiler runtime, but if they do, it's included here. ie "vc90" (Visual Studio 2008), "gcc". Where applicable apartment can be included ie "vc90mt"
BuildType: optional. This can hold letters (in any order desired), each which tell something about the build-specifics. d=debug (omitted if release) t=static (omitted if dynamic) a=ansi (omitted if unicode)
Examples (assuming a project named "DNA"):
dna_win32_x86_vc90.lib/dll
dna_win32_x64_vc90_d.lib/dll
dna_win32_x86_vc90_sd.lib
dna_audio_win32_x64_vc90.lib/dll
dna_audio_winrt_x64_vc110.lib/dll