CMake, multiple targets (asan tsan..) without having to recompile everything - cmake

Goal
I want to define several targets:
make msan: compiles the code with clang with memory sanitizer
make tsan: compiles the code with clang with thread sanitizer
make : compiles the code with gcc
And be able to easily switch between them.
For example I don't want each time I switch rebuild all my objects, (I will have to do it the first time of course, but later if I modify a file and I do make and then make asan it should recompile only this file for each target)
What I have done so far
I have managed to create these targets and from the root directory, but each time I have to do a make clean and recompile.
option(CLANG_MSAN "Enable Clang memory sanitizer" OFF)
if (CLANG_MSAN)
set (CMAKE_CXX_FLAGS "-g -fsanitize=address -fno-omit-frame-pointer")
endif()
add_custom_target(asan
COMMAND ${CMAKE_COMMAND}
-DCLANG_MSAN=ON
-DCMAKE_CXX_COMPILER=clang++
-DCMAKE_C_COMPILER=clang)
Is it possible to do such a thing with CMake?

Yes, but use multiple build directories:
Create a build directory per configuration.
Configure your project in your build directories with the parameters you need. E.g. cmake -DCMAKE_COMPILER=clang -DCMAKE_C_FLAGS="-fsanitize=thread" .. or the stuff from your question.
If you switch the build directory, you changed your setup.
This implies out-of-source builds, which are encouraged by CMake anyway.

I believe you should be able to achieve this with the ExternalProject module. You could add three external projects, one for msan, one for tsan, and one for the basic GCC build. The two sanitiser builds would be marked as EXCLUDE_FROM_ALL 1.
The CMakeLists for all three of them could share the common part via include().
Something like this:
Root CMakeLists.txt
ExternalProject_Add(msan
EXCLUDE_FROM_ALL 1
SOURCE_DIR msan
CMAKE_GENERATOR ...
)
ExternalProject_Add(tsan
EXCLUDE_FROM_ALL 1
SOURCE_DIR tsan
CMAKE_GENERATOR ...
)
ExternalProject_Add(normal
SOURCE_DIR src
CMAKE_GENERATOR ...
)
src/CMakeLists.txt
include(common.cmake)
src/common.cmake
# Normal CMake code for your project
add_library(...)
msan/CMakeLists.txt
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -g -fsanitize=address -fno-omit-frame-pointer")
include(../src/common.cmake)

Related

cmake not rebuilding a non-download external project after manually editing its sources

I'm working on some modifications to the openEMS project. This project uses cmake to build all of its components. The top level CMakeLists.txt file contains the following:
# ...
ExternalProject_Add( openEMS
DEPENDS fparser CSXCAD
SOURCE_DIR ${PROJECT_SOURCE_DIR}/openEMS
CMAKE_ARGS -DCMAKE_TOOLCHAIN_FILE=${CMAKE_TOOLCHAIN_FILE} -DFPARSER_ROOT_DIR=${CMAKE_INSTALL_PREFIX} -DCSXCAD_ROOT_DIR=${CMAKE_INSTALL_PREFIX} -DWITH_MPI=${WITH_MPI} -DCMAKE_INSTALL_PREFIX=${CMAKE_INSTALL_PREFIX}
)
# ...
Inside the openEMS directory, there's another CMakeLists.txt with the following:
# ...
set(SOURCES
openems.cpp
)
# ...
add_library( openEMS SHARED ${SOURCES})
# ...
After building the project successfully once, make does not rebuild anything when, for example, openems.cpp is modified. Why?
$ mkdir build
$ cd build
$ cmake -DBUILD_APPCSXCAD=NO
$ make
[builds all files]
$ touch ../openEMS/openems.cpp
$ make
[ 33%] Built target fparser
[ 66%] Built target CSXCAD
[100%] Built target openEMS
(noting is built)
I have checked and the modification date of openems.cpp is newer than the target. Even deleting the produced library files and binaries, both in the install directory and in the build directory, does not cause it to rebuild anything. The only way I can get it to rebuild is by deleting everything in the build directory and re-running cmake which, of course, rebuilds everything.
This looks like a case of the following. Quoting from the docs for ExternalProject_Add at the section titled "Build Step Options":
BUILD_ALWAYS <bool>
Enabling this option forces the build step to always be run. This can be the easiest way to robustly ensure that the external project's own build dependencies are evaluated rather than relying on the default success timestamp-based method. This option is not normally needed unless developers are expected to modify something the external project's build depends on in a way that is not detectable via the step target dependencies (e.g. SOURCE_DIR is used without a download method and developers might modify the sources in SOURCE_DIR).
If that's the case, the solution would be to add the BUILD_ALWAYS argument to the ExternalProject_Add call like.
ExternalProject_Add( openEMS
DEPENDS fparser CSXCAD
SOURCE_DIR ${PROJECT_SOURCE_DIR}/openEMS
CMAKE_ARGS -DCMAKE_TOOLCHAIN_FILE=${CMAKE_TOOLCHAIN_FILE} -DFPARSER_ROOT_DIR=${CMAKE_INSTALL_PREFIX} -DCSXCAD_ROOT_DIR=${CMAKE_INSTALL_PREFIX} -DWITH_MPI=${WITH_MPI} -DCMAKE_INSTALL_PREFIX=${CMAKE_INSTALL_PREFIX}
BUILD_ALWAYS TRUE
)
If you confirm that this solves the issue, you might want to raise this as an issue to the maintainers of openEMS.
Also note that since the external project there is using CMake as a buildsystem, you could also add the CONFIGURE_HANDLED_BY_BUILD TRUE to the argument list. See the docs for more info.
Edit: The asker opened a GitHub Pull-Request.

How to compile only some set of targets during make

I am using cmake as build generator. version : 3.18.5
I have 5 targets in my project.
I want to compile only particular set of targets depends upon the option i give during make.
How to do this?
for example, if i run
gmake -j4 foo=set2
I want target app1,app3,app5 should get compiled.
if i run
gmake -j4 foo=3
i want target app2,app4 should get compiled.
i tried the above things and inside cmakelist.txt,
if(foo STREQUAL "set2")
add_subdirectory(../../app1)
add_subdirectory(../../app3)
add_subdirectory(../../app5)
elseif(foo STREQUAL "set3")
add_subdirectory(../../app2)
add_subdirectory(../../app4)
endif()
But i did not got the expected result.
Your if(foo STREQUAL ..) construction doesn't work because it is evaluated at configuration time, not during the build. It won't be possible to make the exact syntax you envisioned work, but we can get close.
If you add two custom targets
add_custom_target(set2)
add_custom_target(set3)
and then manually specify the targets that belong to each set as dependencies
add_dependencies(set2 app1 app3 app5)
add_dependencies(set3 app2 app4)
after including all of the subdirectories, then calling gmake -j4 set2 (or cmake --build . --target set2 -- -j4) should build just the 3 apps.

How do i use add_subdirectory() after ExternalProject_Add() has finished downloading?

I basically ask the same question as has been ask here. The question has however not been answered.
I want to use googletest in my project. For this I'm using ExternalProject_Add() which clones the testsuite with git. After that, I like to use add_subdirectory().
This is also what is described in the official repository. The nice thing about this approach is, that the build scripts in googletest handle the building process themself.
The problem is however, that add_subdirectory() can not find the source folder, since it does not exists from the start. Therefore, add_subdirectory() should depend on the completion of ExternalProject_Add().
Is is possible to make add_subdirectory() dependent of ExternalProject_Add(), like add_dependencies() does for targets?
PS. I can make it all compile if I comment add_subdirectory() out, build it (which ends with an error because the googletest library is missing), uncomment it and build it again (success).
ExternalProject_Add(
googletest
GIT_REPOSITORY https://github.com/google/googletest.git
GIT_TAG master
CONFIGURE_COMMAND ""
BUILD_COMMAND ""
INSTALL_COMMAND ""
TEST_COMMAND ""
)
ExternalProject_Get_Property(googletest source_dir binary_dir)
set(GTEST_INCLUDE_DIR ${source_dir}/googletest/include)
set(GMOCK_INCLUDE_DIR ${source_dir}/googlemock/include)
add_subdirectory(${source_dir}
${binary_dir})
I used this tutorial to accomplish that. Just put your ExternalProject code in a separate file, say "CMakeLists.txt.dependencies" and then launch another cmake with execute_process. I use configure_file first to inject configuration information into the external project and to copy it into the build tree.
configure_file(CMakeLists.txt.dependency.in dependency/CMakeLists.txt)
execute_process(COMMAND "${CMAKE_COMMAND}" -G "${CMAKE_GENERATOR}" .
WORKING_DIRECTORY "${CMAKE_BINARY_DIR}/dependency" )
execute_process(COMMAND "${CMAKE_COMMAND}" --build .
WORKING_DIRECTORY "${CMAKE_BINARY_DIR}/dependency" )
Notice that the tutorial has the ExternalProject code in a separate CMakeLists.txt file. So does gtest official documentation. That is because ExternalProject runs at build-time, not configure time. This is a hack to use ExternalProject at configure time by putting it in a third project and executing a separate CMake+build run in the middle of your main CMake configuration step.
With version 3.11 CMake added a FetchContent command. I haven't used it, but the documentation makes it look like a configuration-time replacement for ExternalProject. That would fetch the content at configuration time, making it available for a later add_subdirectory command.
The ExternalProject and FetchContent commands will work fine for a small number of small and/or obscure dependencies. If your dependencies are more than 2, large, and/or popular, I recommend you look at a dependency manager like Conan. It partners with CMake quite well.
You are not following the method correctly. There should only be ExternalProject_Add() as the final command in the CMakeLists.txt.in file.
The CMakeLists.txt.in file is called by the outer CMakeLists.txt file so that the subproject processing happens at configure time via execute_process(). CMakeLists.txt.in is acting as just a glorified downloader.
So, all other commands like add_subdirectory() are added to the outer CMakeLists.txt file.

cmake ExternalProject cache overwritten

Hello I'm facing a problem regarding CMake and External Projects.
I set a compiler and some flags via CMAKE_CACHE_ARGS and/or CMAKE_ARGS this works the first time I run make but on any subsequent call the CMake Cache of the external project is rebuild (deleted) and the flags are not set accordingly to the flags I specified! So I wonder is there any workaround/way to specify the compiler only once to prevent rebuilding of the cache?
Following is a very basic test project which downloads and compiles GTest, first call to make compiles with Clang++ and the given flags, any following call to make will cause the CMake Cache to be rebuild without the proper flags being set!
cmake_minimum_version_required(VERSION 2.8.6)
project(test)
include(ExternalProject)
ExternalProject_Add(
GTest
SVN_REPOSITORY http://googletest.googlecode.com/svn/tags/release-1.7.0/
CMAKE_ARGS
-DCMAKE_CXX_COMPILER:STRING=clang++
-DCMAKE_CXX_FLAGS:STRING="\"-std=c++1y -stdlib=libc++\""
INSTALL_COMMAND "" # One can not install GTest so dont do anything here
LOG_DOWNLOAD 1
LOG_UPDATE 1
LOG_CONFIGURE 1
LOG_BUILD 1
)
I had the same problem, but with a different setup. Although this answer does not seem to apply to your setup it might be helpful for someone else.
In my case the issue was declaring the project with:
project(test LANGUAGES C)
The external project was a C++ project. Adding CXX to languages (or removing the option altogether, since C CXX is the default) solved the problem for me.

CMake executable location

I have a very simple directory structure:
Project
Project/src
Project/build
Source files are in Project/src, and I do the out-of-src build in Project/build. After running cmake ../ ; make, I can run the executable thusly: Project/build$ src/Executable - that is, the Executable is created in the build/src directory.
How do I set the location of the executable in the CMakeLists.txt file? I've attempted to follow some of the examples found at cmake.org, but the links that work don't seem to show this behaviour.
My Project/src/CMakeLists.txt file is listed here.
include_directories(${SBSProject_SOURCE_DIR}/src)
link_directories(${SBSProject_BINARY_DIR}/src)
set ( SBSProject_SOURCES
main.cpp
)
add_executable( TIOBlobs ${SBSProject_SOURCES})
And the top-level Project/CMakeLists.txt:
cmake_minimum_required (VERSION 2.6)
project (SBSProject)
set (CMAKE_CXX_FLAGS "-g3 -Wall -O0")
add_subdirectory(src)
You have a couple of choices.
To change the default location of executables, set CMAKE_RUNTIME_OUTPUT_DIRECTORY to the desired location. For example, if you add
set (CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR})
to your Project/CMakeLists.txt before the add_subdirectory command, your executable will end up in Project/build for Unix builds or build/<config type> for Win32 builds. For further details, run:
cmake --help-property RUNTIME_OUTPUT_DIRECTORY
Another option for a project of this size is to have just one CMakeLists.txt. You could more or less replace add_subdirectory(src) with the contents of Project/src/CMakeLists.txt to achieve the same output paths.
However, there are a couple of further issues.
You probably want to avoid using link_directories generally. For an explanation, run
cmake --help-command link_directories
Even if you do use link_directories, it's unlikely that any libraries will be found in ${SBSProject_BINARY_DIR}/src
Another issue is that the CMAKE_CXX_FLAGS apply to Unix builds, so should probably be wrapped in an if (UNIX) ... endif() block. Of course, if you're not planning on building on anything other than Unix, this is a non-issue.
Finally, I'd recommend requiring CMake 2.8 as a minimum unless you have to use 2.6 - CMake is an actively-developed project and the current version has many significant improvements over 2.6
So a single replacement for Project/CMakeLists.txt could look like:
cmake_minimum_required (VERSION 2.8)
project (SBSProject)
if (UNIX)
set (CMAKE_CXX_FLAGS "-g3 -Wall -O0")
endif ()
include_directories (${SBSProject_SOURCE_DIR}/src)
set (SBSProject_SOURCES
${SBSProject_SOURCE_DIR}/src/main.cpp
)
add_executable (TIOBlobs ${SBSProject_SOURCES})
Another way of relocating the executable file location is via set(EXECUTABLE_OUTPUT_PATH Dir_where_executable_program_is_located)
build/'config type' for Win32 builds.
For MSVC, to avoid the default "/Debug" created folder
set_target_properties(my_target
PROPERTIES
RUNTIME_OUTPUT_DIRECTORY_DEBUG ${CMAKE_CURRENT_BINARY_DIR})