CMake cannot find my library file - cmake

I want to use CMake to create a simple static library Maths, and then link to that library to create a simple executable Alpha.
Here's my directory structure:
/Maths
/build
/
/CMakeLists.txt
/maths.cpp
/maths.h
/Alpha
/build
/
/CMakeLists.txt
/main.cpp
And here are the contents of those files:
* CMakeLists.txt (for Maths) *
cmake_minimum_required(VERSION 2.6)
project(Maths)
add_library(maths STATIC maths.cpp)
* maths.cpp *
int Double(int x)
{
int y = x * 2;
return y;
}
* maths.h *
int Double(int x);
* CMakeLists.txt (for Alpha) *
cmake_minimum_required(VERSION 2.8)
project(Alpha)
add_executable(alpha main.cpp ../Maths/maths.h)
target_link_libraries(maths ../Maths/build/libmaths.a)
* main.cpp *
#include <iostream>
#include "maths.h"
using namespace std;
int main()
{
int x = 5;
int y = Triple(x);
cout << "The answer = " << y << endl;
return 0;
}
So, I enter the directory /Maths/build, and run cmake ... Then, I run make. This creates the library file libmaths.a in the directory /Maths/build.
Then, I enter the directory /Alpha/build, and run cmake ... Then, I run make. However, this gives me the error message:
/usr/bin/ld: cannot find -lmaths
Why can make not find the library file that I just created?
I also tried copyiing libmaths.a and maths.h into the directory /Alpha, and changing Alpha's CMakeLists.txt to:
cmake_minimum_required(VERSION 2.8)
project(Alpha)
add_executable(alpha main.cpp maths.h)
target_link_libraries(maths libmaths.a)
But this gave the same error.

The essential problem here is that libmaths.a is not placed in one of the default linker directories.
The bigger problem is that you passed libmaths.a to target_link_libraries in the first place.
This is bad for several reasons, but the most obvious is that it's completely non-portable. A different compiler toolchain might use a different ending than .a for static libraries, so you won't be able to build there. Also, should you ever decide to switch to dynamic linking in the future, it means refactoring your existing CMake scripts.
The proper solution is to not pass hardcoded filenames to target_link_libraries. There are two roads that you can take to avoid this.
Use a find script. Essentially, you call find_library(MY_MATHLIB maths [...]) and then do target_link_libraries(alpha ${MY_MATHLIB}) instead. This will determine the absolute path of the .a file in a portable way and pass that to the linker. The biggest disadvantage of this approach is that the .a file must already be there for the find call to succeed. So if you want to build maths after running the CMake for alpha or build it as part of alpha, this won't work.
Since both libraries are under your control, consider using CMake's packaging mechanism. This is by far the most convenient approach for building, but unfortunately also the most difficult to set up. If you are willing to spend some time digging through the CMake docs and experimenting with the build, I encourage you to give it a try, but if you just want something that works quickly, you might want to stick with option 1.

Use a CMakeLists.txt in the root directory
/CMakeLists.txt:
CMAKE_MINIMUM_REQUIRED(VERSION 2.8)
PROJECT( MyProject )
ADD_SUBDIRECTORY( Maths )
ADD_SUBDIRECTORY( Alpha )
in Alpha/CMakeLists.txt:
TARGET_LINK_LIBRARIES( alpha maths )
Run CMake on the top-level CMakeLists.txt instead of the individual subdirectory ones

Related

How to include target include directories transitive through multiple linked libraries

we are working on an embedded project in C/C++ and currently some special needs appeared. Background is there are two compiled libraries which define the same symbols. The compiler allows to create relocatable output modules (with partial linking) and to hide symbols for other compilation units when linking. This also means the output module does not need to have all the symbols defined, this will be done in the final linking. Compiler used is TI LTS1.3.0. I will link directly to the relocatable-section of the manual: https://software-dl.ti.com/codegen/docs/tiarmclang/rel1_3_0_LTS/compiler_manual/linker_description/04_linker_options/linker-output-options.html#stdz0756429
The other part of the project is hardly built on CMake with static libraries which are linked against each other via target_link_libraries.
To get this working I created an "add_executable"-target for each of those both output modules with the same symbols. To those I pass the static-libraries by CMake and get the linked with target_link_libraries.
But now I have a problem. All contents of the static libraries are compiled in each of those output modules. This is unwanted behaviour since as said the final linking does the job of linking the missing stuff - so the static-libraries - to it. This should be done with another add_executable command via CMake as well.
using the target include directories property is not suitable since it only adds the include directories of the given target itself but not of the target the target will include and link against.
So e.g. if you have (pseudo code):
#library A
function( create_libA )
add_library( libA src/A.c )
target_include_directories( libA PUBLIC /inc ) #contains A.h
endfunction()
#library B. different location
function( create_libB LIBA )
add_library( libB src/B.c )
target_link_libraries( libB PUBLIC ${LIBA} )
target_include_directories( libB PUBLIC /inc ) #contains B.h
endfunction()
#target output module with partial linking. Only should link and compile LIBTOBELINKEDIN, not libB. different location.
function( build_part_module LIBB LIBTOBELINKEDIN )
add_executable( outputModuleA src/func.c ) #func.c does include A.h
#following would cause libA and libB also to be compiled and linked in the output due to transitive stuff as I understood, which is unwanted.
target_link_libraries( outputModuleA PUBLIC ${LIBB} ${LIBTOBELINKEDIN} )
#trying this
get_target_property(libBInc ${LIBB} INTERFACE_INCLUDE_DIRECTORIES)
#will only include B.h but not A.h. compilation will fail.
target_include_directories(outputModuleA /inc ${libBInc})
I did not find any solution in Cmake itself to solve this problem. It's confusing me since all the include-directories must be known when the libraries are passed transitive, which is stated in the documentation. But I understand that getting the target include directories of just the passed lib does not include the other ones.
Since target_link_libraries does also not work this way I can only think of a maybe recursive solution? But for that my knowledge is just non-existent.
target_link_libraries with something like HEADERS_ONLY would be helpfull for this job.
Also one can say: if the output module contains all the definitions it won't be a problem, since the linker then knows them and will do its magic.
But this is also unwanted, since we use the generated static-libraries to place them into sections in different regions of the RAM directly. This would then mean to create another linker-script for partial linking which defines sections which then can be again moved. But the more we go this direction, the less we need CMake for it.
Instead of get_target_property use $<TARGET_PROPERTY> generator expression: the property's value, extracted by that expression, already includes transitive propagation:
target_include_directories(outputModuleA PRIVATE
$<TARGET_PROPERTY:libB,INTERFACE_INCLUDE_DIRECTORIES>
)
Note, that generator expressions has limited usage: not all functions expects them. Documentation for target_include_directories clearly states that the command supports generator expressions.

cmake, pass result of external program as preprocessor definitions

I'm new to cmake, so correct me if I've messed things up and this should be solved using something other than cmake.
I have main_program, that requires multiple other subprograms in form of bindata to be specified at build phase. Right now I build it by running
cmake -DBINDATA1="\xde\xad..." -DBINDATA2="\xbe\xef" -DBINDATA3="..."
and in code I use them as:
// main_program.cpp
int main() {
#ifdef BINDATA1
perform_action1(BINDATA1);
#endif
#ifdef BINDATA2
perform_action2(BINDATA2);
#endif
[...]
This is rather unclean method as any time I'm changing one of subprograms I have to generate bindata from it and pass it to cmake command.
What I would like to do, is have a project structure:
/
-> main_program
-> subprograms
-> subprogram1
-> subprogram2
-> subprogram3
and when I run cmake, I would like to
compile each of subprograms
generate shellcode from each of them, by running generate_bindata program on them
build main_program passing bindatas from step 2
and when I run cmake, I would like to
compile each of subprograms
generate shellcode from each of them, by running generate_shellcode program on them
build main_program passing shellcodes from step 2
Then let's do that. Let's first write a short script to generate a header:
#!/bin/sh
# ./custom_script.sh
# TODO: Find out proper quoting and add `"` is necessarily. Ie. details.
# Prefer to use actual real variables like `static const char *shellcode[3]`
# instead of raw macro defines.
cat > "$1" <<EOF
#define SHELLCODE1 $(cat "$2")
#define SHELLCODE2 $(cat "$3")
#define SHELLCODE3 $(cat "$4")
EOF
To be portable, write this script in cmake. This script will be run at build phase to generate the header needed for compilation. Then, "model dependencies" - find out what depends on what exactly. Then write it in cmake:
add_executable(subprogram1 sources.c...)
add_executable(subprogram2 sources.c...)
add_executable(subprogram3 sources.c...)
for(i IN ITEMS 1 2 3)
add_custom_target(
COMMENT Generate shellcode${i}.txt with the content of shellcode
# TODO: redirection in COMMAND should be removed, or the command
# should be wrapped in `sh -c ...`.
COMMAND $<TARGET_FILE:subprogram${i}> | generate_shellcode > ${CMAKE_CURRENT_BINARY_DIR}/shellcode${i}.txt
OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/shellcode${i}.txt
DEPENDS $<TARGET_FILE:subprogram${i}> generate_shellcode
)
endfor()
add_custom_command(
COMMENT Generate shellcodes.h from shellcode1.txt shellcode2.txt and shellcode3.txt
COMMAND sh custom_script.sh
${CMAKE_CURRENT_BINARY_DIR}/shellcodes.h
${CMAKE_CURRENT_BINARY_DIR}/shellcode1.txt
${CMAKE_CURRENT_BINARY_DIR}/shellcode2.txt
${CMAKE_CURRENT_BINARY_DIR}/shellcode3.txt
OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/shellcodes.h
DEPENDS
${CMAKE_CURRENT_BINARY_DIR}/shellcode1.txt
${CMAKE_CURRENT_BINARY_DIR}/shellcode2.txt
${CMAKE_CURRENT_BINARY_DIR}/shellcode3.txt
)
# Then compile the final executable
add_executable(main main.c ${CMAKE_CURRENT_BINARY_DIR}/shellcodes.h)
# Don't forget to add includes!
target_include_directories(main PUBLIC ${CMAKE_CURRENT_BINARY_DIR})
# or you may add dependency to a single file instead of target
# Like below only to a single shellcodeswrapper.c file only
# This should help build parallelization.
set_source_files_properties(main.c OBJECT_DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/shellcodes.h)
# Or you may add a target for shelcodes header file and depend on it
add_custom_target(shellcodes DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/shellcodes.h)
add_executable(main main.c)
target_include_directories(main PUBLIC ${CMAKE_CURRENT_BINARY_DIR})
add_dependencies(main shellcodes)
Then your main file:
#include <shellcodes.h> // compiler will find it in BINARY_DIR
int main() {
perform_action1(SHELLCODE1);
perform_action2(SHELLCODE2);
}
So that all your source files are not recompiled each time, I suggest to write a wrapper:
// shellcodewrapper.c
#include <shellcodes.h>
// preserve memory by not duplicating code in each TU
static const char shellcode1[] = SHELLCODE1;
// only this file will be recompiled when SHELLCODE changes
const char *get_shellcode1(void) {
return shellcode1;
}
// shellcodewrapper.h
const char *get_shellcode1(void);
// main.c
#include <shellcodewrapper.h>
int main() {
perform_action1(get_shellcode1());
perform_action2(get_shellcode2());
}
That way when you change the "SHELLCODE" generators, only shellcodewrapper.c will be compiled, resulting in super fast compilation times.
Note how dependency is transferred and how it works - I used files inside BINARY_DIR to transfer result from one command to another, then these files track what was changed and transfer dependency below in the chain. Track dependencies in DEPENDS and OUTPUT in add_custom_command and cmake will properly compile in proper order.

Using an interface library with check_c_source_runs() or try_run()

Use case: I'm trying to compile a test program that probes for a list of TrueType(tm) fonts using SDL2_ttf (with SDL2, Freetype, PNG and Zlib). The SDL2_ttf::SDL2_ttf interface library exists and links successfully with target executables. My problem is how to get check_c_source_runs() to pick up the definitions, include directories and libraries. I'd rather not have to manually extract everything from properties, as in the following code fragment:
include(CheckCSourceRuns)
get_property(defs TARGET SDL2_ttf::SDL2_ttf PROPERTY INTERFACE_COMPILE_DEFINITIONS)
get_property(incs TARGET SDL2_ttf::SDL2_ttf PROPERTY INTERFACE_INCLUDE_DIRECTORIES)
get_property(libs TARGET SDL2_ttf::SDL2_ttf PROPERTY INTERFACE_LINK_LIBRARIES)
## Transform the definitions with "-D"
if (CMAKE_VERSION VERSION_GREATER_EQUAL "3.12")
list(TRANSFORM defs PREPEND "-D")
list(TRANSFORM incs PREPEND "-I")
else ()
## Code that does what list(TRANSFORM...) does in less capable CMake
## versions.
endif ()
set(CMAKE_REQUIRED_DEFINITIONS ${defs})
set(CMAKE_REQUIRED_INCLUDES ${incs})
set(CMAKE_REQUIRED_LIBRARIES ${libs})
check_c_source_runs("
#include <stdint.h>
#include <SDL.h>
#include <SDL_ttf.h>
int main(int argc, char *argv[])
{
const char *fonts[] = {\"DejaVuSans.ttf\", \"LucidaSansRegular.ttf\", \"FreeSans.ttf\", \"AppleGothic.ttf\", \"tahoma.ttf\"};
size_t i, cnt = 0;
SDL_Init(SDL_INIT_VIDEO);
TTF_Init();
for (i = 0; i < sizeof(fonts)/sizeof(fonts[0]); ++i) {
TTF_Font *ttf = TTF_OpenFont(fonts[i], 10);
if (ttf != NULL) {
fputs(fonts[i], stderr);
if (cnt++ > 0) {
fputc(';', stderr);
}
TTF_CloseFont(ttf);
}
}
TTF_Quit();
SDL_Quit();
return 0;
}" ttfprobe_run)
Link libraries are hairy, since there are interface libraries referenced from within SDL2_ttf::SDL2_ttf, e.g. FreeType::FreeType.
Suggestions?
Functions try_compile and try_run and everything which is based on them (e.g. check_c_source_runs) are actually build some other CMake project. Because you cannot pass targets to the CMake project, you have two ways:
Extract all needed target's properties to the variables and pass them to the newly generated project. As you already do.
Write CMakeLists.txt for other project manually, and use calls to find_package and other package-discovery functions in it.
E.g., you may write CMakeLists.txt for other project like that:
# Source file is in SOURCE_FILE parameter,
# resulted executable is copied into the file pointed by EXE_FILE parameter.
cmake_minimum_required(...)
project(test_project)
# This will create 'SDL2_ttf::SDL2_ttf' target
find_package(SDL2_ttf REQUIRED)
add_executable(test_exe ${SOURCE_FILE})
target_link_libraries(test_exe SDL2_ttf::SDL2_ttf)
add_custom_command(OUTPUT ${EXE_FILE}
COMMAND ${CMAKE_COMMAND} -E copy $<TARGET_FILE:test_exe> ${EXE_FILE}
DEPENDS $<TARGET_FILE:test_exe>
)
add_custom_target(copy_exe ALL DEPENDS ${EXE_FILE})
The main challenge is to pass as many variables to the other project as needed for it to be built in the same "environment" as the main project.
Example below handles only variables which could affect on find_package(SDL2_ttf) call:
# Main project
# Somewhere you have this call too.
find_package(SDL2_ttf REQUIRED)
# List of arguments for the subproject
set(SUBPROJECT_ARGS
# This affects on searching for possible `FindSDL2_ttf.cmake` script
-DCMAKE_MODULE_PATH=${CMAKE_MODULE_PATH}
# This affects on searching for `find_*` calls in find script.
-DCMAKE_PREFIX_PATH=${CMAKE_PREFIX_PATH}
)
if (SDL2_ttf_DIR)
# This is a directory with `SDL2_ttfConfig.cmake` script
list(APPEND SUBPROJECT_ARGS -DSDL2_ttf_DIR=${SDL2_ttf_DIR})
endif()
# build subproject
try_compile(TTF_TEST_RESULT # Variable which will contain result of building the subproject
${CMAKE_CURRENT_BINARY_DIR}/ttf_test # Build directory for the subproject
<src-dir> # Source directory for the subproject, where its `CMakeLists.txt` resides.
test_project # Project name of the subproject
CMAKE_FLAGS
-DSOURCE_FILE=<src-file> # Source file
-DEXE_FILE=<exe-file> # Path to the resulted executable file
${SUBPROJECT_ARGS} # The rest of arguments for subproject
OUTPUT_VAR TTF_TEST_OUTPUT # Variable which will contain output of the build process
)
if (TTF_TEST_RESULT)
# Subproject has been built successfully, now we can try to execute resulted file
...
endif()
Tricky? Yes. But this is how CMake works...

CMake : add executable with unknown genererated source files

I have a tool that generates a set source files whose name I am not able to know beforehand.
How to write a proper CMakeLists.txt script for scenario? This question has been asked before here CMake Compiling Generated Files. But it does not have a proper solution.
For instance, in the first answer (https://stackoverflow.com/a/8748478/2912478), OP can predict which files will generated based on the input .idl files. The second answer (https://stackoverflow.com/a/39258996/2912478) shows three different ways to solve but I really couldn't get his solution working.
Test case
I prepared a simple test case. Suppose I have this static file where the main resides (main.cpp).
# main.cpp
void foo(void);
int main() {
foo();
return 0;
}
Currently, I am using this CMakeLists.txt. The custom command generates the source file under src.
# CMakeLists.txt
add_executable(a.out main.cpp)
add_custom_command(
OUTPUT mylib.cpp
COMMAND ${CMAKE_SOURCE_DIR}/genf.sh ${CMAKE_SOURCE_DIR}/src
DEPENDS ${CMAKE_SOURCE_DIR}/genf.sh
)
add_custom_target(GenFile DEPENDS mylib.cpp)
add_dependencies(a.out GenFile)
Here is the hypothetical code generator genf.sh. I use a random number to mimic the fact that we do NOT know which files the generator will generate.
#!/bin/bash
rm -rf $1/*
fname=$(echo $((1 + RANDOM % 100)))
echo "Generating src$fname.cpp"
echo "void foo(void) {}" > $1/src$fname.cpp
Attempt 1
I tried to use GLOB to find all files generated. So I put the following lines at the end of my CMakeLists.txt. This doesn't work because at the moment of running cmake .. there is no files under src. So this solution never links the generated source files.
file(GLOB GeneratedSourceFiles ${CMAKE_SOURCE_DIR}/src/*.cpp)
target_sources(a.out PUBLIC ${GeneratedSourceFiles})

Using a CMake cache variable "before" it is defined

CMake cache variables can be set from virtually everywhere (see here #Florian's What's the CMake syntax to set and use variables?). I was under the assumption that the set value is visible everywhere, even to CMake lists parsed before, but this isn't the case.
Use case
Module A uses ${CMAKE_MYDEF}.
Module B sets the cache variable CMAKE_MYDEF.
add_subdirectory(A) is called before add_subdirectory(B).
Short example showing the behavior
cmake_minimum_required(VERSION 3.7)
project(test)
add_executable(EXEC test.cpp)
target_compile_definitions(EXEC PRIVATE MYDEF=${CMAKE_MYDEF})
set(CMAKE_MYDEF "MyValue" CACHE STRING "")
Questions
How can I make sure CMAKE_MYDEF has the desired value regardless the order I add module A and module B?
Are there any ways to ensure the CMake configuration step is re-run twice or, if applicable, as long as the cache variables get changed? (This isn't probably a clean solution, but since I'm working with legacy code not everything can be done beautifully.)
Are there alternatives to cache variables to achieve the same result without re-running the CMake configuration by hand?
Is it possible to set compiler definitions in the generation phase (i.e. when all CMake cache variables are known and set)? Using some kind of generator expressions?
Edit: Short example solution
Following #Florian's answer, here the adapted example showing the solution:
cmake_minimum_required(VERSION 3.7)
project(test)
add_executable(EXEC test.cpp)
target_link_libraries(EXEC MyOtherLib)
add_library(MyOtherLib INTERFACE)
set(CMAKE_MYDEF "MyValue" CACHE STRING "")
target_compile_definitions(MyOtherLib INTERFACE MYDEF=${CMAKE_MYDEF})
Yes, I'm fully with #Tsyvarev's answer that CMake's parser works sequentially. So variables - even cached ones - or generator expressions - that can't read variables - are no good here.
I just wanted to add the possibilities you have using target and directory properties depending on the dependencies between A and B:
When A depends on B with e.g.
target_link_libraries(A PUBLIC B)
then a simple
target_compile_definitions(B PUBLIC MYDEF=SOME_DEF)
would propagate the necessary define to A.
When B depends an A and A is already known than it would be
target_link_libraries(B PUBLIC A)
target_compile_definitions(A PUBLIC MYDEF=SOME_OTHER_DEF)
If you're working with sub-directories I would recommend putting the definition in the root CMakeLists.txt globally:
add_definitions(-DMYDEF=GLOBAL_DEF)
Finally the full variant with sub-directories letting B decide what to do:
CMakeLists.txt
cmake_minimum_required(VERSION 3.7)
project(test)
add_subdirectory(A)
add_subdirectory(B)
A\CMakeLists.txt
file(WRITE a.cpp [=[
#include <iostream>
#ifndef MYDEF
#define MYDEF "Hello from A!"
#endif
void a()
{
std::cout << MYDEF << std::endl;
}
]=])
add_library(A a.cpp)
B\CMakeLists.txt
file(WRITE b.cpp [=[
void a();
void main()
{
a();
}
]=])
add_executable(B b.cpp)
target_link_libraries(B A)
if (TARGET "A")
target_compile_definitions(A PRIVATE MYDEF="Hello from B!")
else()
set_property(
DIRECTORY ".."
APPEND PROPERTY
COMPILE_DEFINITIONS "MYDEF=\"Hello from Global!\""
)
endif()
Reference
Is Cmake set variable recursive?
CMake processes scripts sequentially, starting from top-level CMakeLists.txt and executing its lines one by one.
So, if read variable before assigning it, you will get nothing. The only specific of CACHE variable in that scenario is possibility for that variable to be assigned on previous cmake invocation.
Needs for using a variable before its assigning taking a place usually signals about bad design. In many situations (even with legacy code), design can be fixed gracefully.
Forcing CMake to reconfigure the project can be accomplished e.g. by touching current script:
to force a re-configure, one could "cmake -E touch"
the CMAKE_CURRENT_LIST_FILE, somehow during target building
or some such.