How do I configure portable parallel builds in CMake? - cmake

Is it somehow possible to be able to have a parallel build no matter which build tool is used?
Under Unix we can add make -jN where N are the number of threads, and under Windows I added to the CXX_FLAG "/MP" which is then used in Visual Studio to parallel build...(?) How can I make my version such that CMAKE_MAKE_PROGRAM is not always extended when I run CMake?
What is a general solution?
I came up with this:
# Add some multithreaded build support
MARK_AS_ADVANCED(MULTITHREADED_BUILD)
set(MULTITHREADED_BUILD 12 CACHE STRING "How many threads are used to build the project")
if(MULTITHREADED_BUILD)
if(${CMAKE_GENERATOR} MATCHES "Unix Makefiles")
message(STATUS ${CMAKE_BUILD_TOOL})
set(CMAKE_MAKE_PROGRAM "${CMAKE_MAKE_PROGRAM} -j${MULTITHREADED_BUILD}")
message(STATUS "Added arguments to CMAKE_BUILD_TOOL: ${CMAKE_MAKE_PROGRAM}")
elseif(MSVC)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /MP")
message(STATUS "Added parallel build arguments to CMAKE_CXX_FLAGS: ${CMAKE_CXX_FLAGS}")
endif()
endif()

With CMake 3.12 this is possible. From the release notes:
The cmake(1) Build a Project (cmake --build) gained --parallel [<jobs>] and -j [<jobs>] options to specify a parallel build level. They map to corresponding options of the native build tool.
As mentioned by dkg, you can also set the environment variable CMAKE_BUILD_PARALLEL_LEVEL.
Links to CMake's documentation:
Build a Project
CMAKE_BUILD_PARALLEL_LEVEL

If you have CMake v2.8.8 or higher, you may use Ninja as an alternative of GNU make:
mkdir build
cd build
cmake -G Ninja ..
ninja # Parallel build (no need -j12)
or
mkdir build
cd build
cmake -G Ninja ..
cmake --build . # Parallel build using Ninja
As you can see, no need to use CMAKE_MAKE_PROGRAM, the build is run in parallel by default, optimizing the number of jobs depending on available CPU cores.
Ninja is based on a low-level JSON configuration to speed up the startup phase. Therefore its JSON configuration is not easy to write by hand, and I always generate it using a high-level tool/IDE:
CMake v2.8.8 (2012)
Qt Creator v2.6 (2012)
KDevelop v4.6 (2013)
Meson on Linux (2013)
... see the generators of Ninja configuration at https://github.com/ninja-build/ninja/wiki/List-of-generators-producing-ninja-build-files
As a C++ build often requires lots of memory, your computer must provide as much memory as the number of CPU cores.
As pointed out by Ruslan, CMake 3.12 (2018) has a new option cmake --build -j <N> to limit build to <N> cores (jobs) thus limiting the memory consumption (see also the documentation). If you use an older CMake version, you can still use cmake --build -- -j <N>. The option -- tells to CMake to pass the rest directly to the underlying builder tool, here it is Ninja.

Now it is very simple to parallel build with cmake. You can add "-j jobs_number" when using "cmake --build". For example:
cmake --build . -j 24
More details can be found in CMAKE manual: https://cmake.org/cmake/help/latest/manual/cmake.1.html#build-tool-mode
--parallel [], -j [] The maximum number of concurrent processes to use when building. If is omitted the native build
tool’s default number is used.
The CMAKE_BUILD_PARALLEL_LEVEL environment variable, if set, specifies
a default parallel level when this option is not given.
Some native build tools always build in parallel. The use of
value of 1 can be used to limit to a single job.

As already mention above one can use --parallel [<jobs>] (or -j [<jobs>]) option of CMake to build solution in parallel.
But I would like to note that in Windows a command
cmake.exe --build --parallel <jobs>
starts a root MSBuild process with the following arguments:
msbuild.exe /m:<jobs> /p:CL_MPCount=1 <another-useful-arguments...>
This means the root MSBuild process will start multiple child MSBuild processes due to /m:<jobs> (or /maxcpucount:<jobs>). But each of this child MSBuild runs at most one compiler process due to /p:CL_MPCount=1 argument (for more details find this link). In other words each MSBuild can compile at most one source file at the same time.
To overcome this limitation one may call CMake in the following way:
cmake.exe --build --parallel <n_msbuild> -- /p:CL_MPcount=<n_cl>
This approach sets /MP option thereby allows multiple compiler processes under each MSBuild instance.
And for the record. To configure builds ran form Visual Studio GUI one need to go to Settings:
Option "Projects and Solutions → Build And Run → maximum number of parallel projects build" affects on maxcpucount.
Option "Projects and Solutions → VC++ Project Settings → Build → Maximum Concurrent C++ Compilations" represents the default boundary to /MP.

You can't do this cross-platform. The -jN option is a parameter to make, and not part of the generated Makefile. However, you could have CMake generate a Bash script that runs make for your project using -jN (where the script looks up the number of cores you have).

I have settled down to writing a parallelmake.sh script for Unix Makefiles-based generators. This is done here: https://github.com/gabyx/ApproxMVBB
And the relevant parts in the the CMake file:
https://github.com/gabyx/ApproxMVBB/blob/master/CMakeLists.txt#L89
# Add some multithreaded build support =====================================================================================================
MARK_AS_ADVANCED(MULTITHREADED_BUILD)
SET(MULTITHREADED_BUILD ON CACHE BOOL "Parallel build with as many threads as possible!")
if(MULTITHREADED_BUILD)
if(${CMAKE_GENERATOR} MATCHES "Unix Makefiles")
file(COPY ${ApproxMVBB_ROOT_DIR}/cmake/parallelmake.sh DESTINATION ${PROJECT_BINARY_DIR}
FILE_PERMISSIONS OWNER_READ OWNER_WRITE OWNER_EXECUTE GROUP_READ GROUP_EXECUTE WORLD_READ WORLD_EXECUTE
NO_SOURCE_PERMISSIONS
)
SET(CMAKE_MAKE_PROGRAM "${PROJECT_BINARY_DIR}/parallelmake.sh")
MESSAGE(STATUS "Set make program to ${PROJECT_BINARY_DIR}/parallelmake.sh")
elseif(MSVC)
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}" "/MP")
MESSAGE(STATUS "Added parallel build arguments to CMAKE_CXX_FLAGS: ${CMAKE_CXX_FLAGS}")
endif()
endif()
# ========================================================================================================================================

Related

How do I make makefile generatedby cmake output the last command when error occur? [duplicate]

I use CMake with GNU Make and would like to see all commands exactly (for example how the compiler is executed, all the flags etc.).
GNU make has --debug, but it does not seem to be that helpful are there any other options? Does CMake provide additional flags in the generated Makefile for debugging purpose?
When you run make, add VERBOSE=1 to see the full command output. For example:
cmake .
make VERBOSE=1
Or you can add -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON to the cmake command for permanent verbose command output from the generated Makefiles.
cmake -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON .
make
To reduce some possibly less-interesting output you might like to use the following options. The option CMAKE_RULE_MESSAGES=OFF removes lines like [ 33%] Building C object..., while --no-print-directory tells make to not print out the current directory filtering out lines like make[1]: Entering directory and make[1]: Leaving directory.
cmake -DCMAKE_RULE_MESSAGES:BOOL=OFF -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON .
make --no-print-directory
It is convenient to set the option in the CMakeLists.txt file as:
set(CMAKE_VERBOSE_MAKEFILE ON)
Or simply export VERBOSE environment variable on the shell like this:
export VERBOSE=1
cmake --build . --verbose
On Linux and with Makefile generation, this is likely just calling make VERBOSE=1 under the hood, but cmake --build can be more portable for your build system, e.g. working across OSes or if you decide to do e.g. Ninja builds later on:
mkdir build
cd build
cmake ..
cmake --build . --verbose
Its documentation also suggests that it is equivalent to VERBOSE=1:
--verbose, -v
Enable verbose output - if supported - including the build commands to be executed.
This option can be omitted if VERBOSE environment variable or CMAKE_VERBOSE_MAKEFILE cached variable is set.
Tested on Cmake 3.22.1, Ubuntu 22.04.
If you use the CMake GUI then swap to the advanced view and then the option is called CMAKE_VERBOSE_MAKEFILE.
I was trying something similar to ensure the -ggdb flag was present.
Call make in a clean directory and grep the flag you are looking for. Looking for debug rather than ggdb I would just write.
make VERBOSE=1 | grep debug
The -ggdb flag was obscure enough that only the compile commands popped up.
CMake 3.14+
CMake now has --verbose to specify verbose build output. This works regardless of your generator.
cd project
cmake -B build/
cmake --build build --verbose
It's worth noting however Xcode may not work with --verbose
Some generators such as Xcode don't support this option currently.
Another option it to use the VERBOSE environment variable.
New in version 3.14.
Activates verbose output from CMake and your build tools of choice when you start to actually build your project.
Note that any given value is ignored. It's just checked for existence.
cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=TRUE will generate a file with all compilation commands.
This file is required by some LSP to know how to compile a source file out of the box, but it could also help for debugging compilation problems.
The output file is named ${CMAKE_BINARY_DIR}/compile_commands.json.

cmake building in source directory, not PWD

The question
Debug vs Release in CMake
indicates that
cd ~/codebase
mkdir Release
cd Release
cmake -DCMAKE_BUILD_TYPE=Release ..
make
Will create the Makefile in release, and build the binary there. The intermediate .o files will be in a subdirectory of this.
However, when I do this with my project, CMake ignores the PWD that it is started from. The final target is always the directory ~/codebase/ which contains CMakeList.txt.
In the cmake-gui tool, I specified the source and build directories to be the same directory, the FQN to codebase
I'm new to CMake, and don't know how to get this to work as I expect. What should I modify to get this work as expected?
If you are using a single configuration generator (Ninja/Unix-Makefiles)
Then you need a build folder for each configuration.
Like this:
# Configure the build
cmake -S . -B build/Debug -D CMAKE_BUILD_TYPE=Release
# Actually build the binaries
cmake --build build/Debug
For multi-configuration generators it's slightly different (Ninja Multi-Config, Visual Studio)
# Configure the build
cmake -S . -B build
# Actually build the binaries
cmake --build build --config Debug
If you are wondering why this is necessary it's because cmake isn't a build system. It's a meta-build system (IE a build system that creates build systems). This is basically the result of handling build systems that support multiple-configurations in 1 build. If you'd like a deeper understanding I'd suggest reading a bit about cmake in Craig Scott's book "Professional CMake: A Practical Guide
Note:
My examples use newer cmake cli practices.
EDIT:
That question you linked to has dangerously out of date answers...

Execute process at install stage, every time

I would like to run program to perform do extra installation tasks from CMake. My attempted solution, based on INSTALL(CODE ...) is (this is a real MWE):
macro(MY_EXTRA_STUFF ARG)
execute_process(...)
endmacro()
install(CODE "MY_EXTRA_STUFF(${SOME_ARG})")
but CMake complains when I run ninja install (or make install, depending on generator in use):
[0/1] Install the project...
-- Install configuration: ""
CMake Error at cmake_install.cmake:41 (MY_EXTRA_STUFF):
Unknown CMake command "MY_EXTRA_STUFF".
FAILED: CMakeFiles/install.util
cd /tmp && /usr/bin/cmake -P cmake_install.cmake
ninja: build stopped: subcommand failed.
Is there a way to smuggle my own code into the install stage? The code is too long to fit inside install(CODE "...") nicely. A bonus to do it without an external file. Thanks!
The code passed to install(CODE) is executed as standalone CMake code, thus it shouldn't use definitions (functions,macros, variables) from the rest of CMakeLists.txt.
That is, install(CODE) behaves similar as install(SCRIPT) with a standalone script containing given code.
The thing is that configuration stage (when you call cmake to configure your project) and installation stage, which, as you can see, calls /usr/bin/cmake -P cmake_install.cmake, are separate cmake invocations. These invocations parse different files, so they unaware about context of each other.

How to setup make options with cmake

I would like to setup commands like make debug, make test, etc... What is the best way to do this with cmake, so that I run cmake .. one time (it takes a while) and then be able to choose the build type with make?
I couldn't find any resources on this.
As #Tsyvarev has commented this needs a little extra work for single-configuration environments (respectively CMake's Makefile generators) since the build type is chosen during CMake's configuration and finalized during the build environment generation step.
So here is what I've done:
First you run - e.g. in a script - CMake for all configurations you want to support and choose respective sub-folders for the output:
> cmake -H"." -B"Debug" -DCMAKE_BUILD_TYPE=Debug
> cmake -H"." -B"Release" -DCMAKE_BUILD_TYPE=Release
Note: -H (for "home directory") and -B (for "binary output directory") are undocumented options, but very useful in those cases. And they work with all CMake releases so far. Just be careful not to put spaces between the option and the their values.
Then you can use again CMake to build from/in those sub-folders:
> cmake --build "Debug"
> cmake --build "Release"
And if you want to run the tests you can:
> cmake --build "Debug" --target "test"
> cmake --build "Release" --target "test"
References
Changing CMake files standard location
Does CMake always generate configurations for all possible project configurations?
cmake build multiple targets in different build directories
CMAKE_BUILD_TYPE not being used in CMakeLists.txt

How do I use CMake ExternalProject_Add or alternatives in a cross-platform way?

I would like to build a third-party project that already has CMake as part of my project's CMake strips. ExternalProject_Add is for this purpose, but I have found it can only be made to work with a specific generator, and I wanted it to work on many platforms easily.
For example, here is my external project with an added script for zlib, which has its own CMakeLists.txt:
set(USE_PROJECT_CMAKE_MODULE_PATH "-DCMAKE_MODULE_PATH=${MAKE_MODULE_PATH}")
ExternalProject_Add(ZLIB
SOURCE_DIR ${CMAKE_SOURCE_DIR}/external/zlib
DOWNLOAD_COMMAND ""
UPDATE_COMMAND ""
CMAKE_ARGS
-DCMAKE_INSTALL_PREFIX=<INSTALL_DIR>
-DCMAKE_CXX_FLAGS=${CMAKE_CXX_FLAGS}
-DCMAKE_C_FLAGS=${CMAKE_C_FLAGS}
-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}
${USE_PROJECT_CMAKE_MODULE_PATH}
INSTALL_COMMAND "")
ExternalProject_Add_Step(ZLIB installInternally
COMMAND cd <BINARY_DIR> && make install
DEPENDEES install
ALWAYS 1)
ExternalProject_Get_Property(ZLIB install_dir)
if(UNIX)
set(ZLIB_NAME libz)
else(UNIX)
set(ZLIB_NAME zlib)
endif(UNIX)
add_library(zlib UNKNOWN IMPORTED)
set_property(TARGET zlib PROPERTY IMPORTED_LOCATION ${install_dir}/lib/${ZLIB_NAME}.a)
set(ZLIB_LIBRARIES zlib)
set(ZLIB_LIBRARIES_OPTIONAL ${ZLIB_LIBRARIES})
set(ZLIB_DIR ${install_dir} CACHE INTERNAL "zlib ROOT dir")
set(ZLIB_INCLUDE_DIRS ${install_dir}/include CACHE INTERNAL "zlib include dirs")
set(ZLIB_DEFINES "-msse2 -mfpmath=sse" CACHE INTERNAL "zlib defines")
The problem with this is that it works with make, but not with Xcode or Visual Studio. Perhaps there is some way to take the CMake build commands passed to my project and forward them to ExternalProject_Add.
How can I write ExternalProject_Add calls in a cross-platform way with minimal code complexity, or is there a better alternative?
Problems
-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}
This is enough for single-configuration projects. But for Xcode and Visual Studio, you need to set CMAKE_CONFIGURATION_TYPES plus call build . --config at the build stage. See my answer.
COMMAND cd <BINARY_DIR> && make install
This will work only for Makefile generators of course. To be cross-platform you can use:
--build . --target install --config inside INSTALL_COMMAND of ExternalProject_Add.
Take a look at this template file, and in particular the following lines:
ExternalProject_Add(
"${current_project}"
URL
#HUNTER_PACKAGE_URL#
URL_HASH
SHA1=#HUNTER_PACKAGE_SHA1#
DOWNLOAD_DIR
"#HUNTER_PACKAGE_DOWNLOAD_DIR#"
SOURCE_DIR
"#HUNTER_PACKAGE_SOURCE_DIR#"
INSTALL_DIR
"#HUNTER_PACKAGE_INSTALL_PREFIX#"
# Not used, just avoid creating Install/<name> empty directory
BUILD_COMMAND ""
# This command is empty because all necessary targets will
# be built on install stage
CMAKE_ARGS
"-G#CMAKE_GENERATOR#"
"-C#HUNTER_CACHE_FILE#"
"-C#HUNTER_ARGS_FILE#"
"-D${postfix_name}=${${postfix_name}}"
"-DCMAKE_BUILD_TYPE=${configuration}"
"-DCMAKE_CONFIGURATION_TYPES=${configuration}"
"-DCMAKE_INSTALL_PREFIX=#HUNTER_PACKAGE_INSTALL_PREFIX#"
"-DCMAKE_TOOLCHAIN_FILE=${CMAKE_TOOLCHAIN_FILE}"
INSTALL_COMMAND
"#CMAKE_COMMAND#"
--build .
--target install
--config ${configuration}
--
${jobs_option}
)
Alternative
or is there a better alternative?
Have you seen Hunter?
You can add zlib just like this:
hunter_add_package(ZLIB)
find_package(ZLIB CONFIG REQUIRED)
target_link_libraries(... ZLIB::zlib)
This code works everywhere. Third party dependencies will be downloaded automatically in the configuration step. Example of building with different generator/toolchains (build.py is just a CMake wrapper that sets CMAKE_TOOLCHAIN_FILE and -G/-B):
build.py --toolchain mingw --config Release # MinGW Makefiles
build.py --toolchain vs-12-2013 --config Debug # Visual Studio 12 2013
build.py --toolchain xcode --config Release # Xcode
build.py --toolchain libcxx --config Release # Makefile with -stdlib=libc++ toolchain
build.py --toolchain ios-8-2 --config Release # Xcode with iOS SDK 8.2 toolchain
You got full control what options, build types or number of jobs you want to have while building third-party packages. For instance, this is how you can build four types, Debug, Release, MinSizeRel, and RelWithDebInfo for zlib and link MinSizeRel to the current project:
> build.py --toolchain xcode --verbose --config MinSizeRel --fwd "HUNTER_CONFIGURATION_TYPES=Release;Debug;MinSizeRel;RelWithDebInfo"
/.../clang /.../lib/libz-MinSizeRel.a ... -o /.../_builds/xcode/MinSizeRel/foo
> ls -la /.../.hunter/_Base/d1232c0/326318e/37e4682/Install/lib/libz*
99056 /.../.hunter/_Base/d1232c0/326318e/37e4682/Install/lib/libz-MinSizeRel.a
307872 /.../.hunter/_Base/d1232c0/326318e/37e4682/Install/lib/libz-RelWithDebInfo.a
109536 /.../.hunter/_Base/d1232c0/326318e/37e4682/Install/lib/libz.a
258904 /.../.hunter/_Base/d1232c0/326318e/37e4682/Install/lib/libzd.a
CMake ExternalProject_Add calls work cross-platform by default and will only fail to do so if one uses particular commands that are only available on a subset of operating systems.
Typically, CMAKE_ARGS is used to pass information to each superbuild unit within an external project build. The CMakeLists.txt files that control each miniature part of the overall build use CMake's declarative syntax (e.g., "add_library(library_name SHARED filename1.hpp filename1.cpp). CMake will convert such syntax to the commands that are specific to the particular build system you wish to use (e.g., make and Ninja).
The sample above re: zlib fails to be cross-platform in part because the ExternalProject_Add_Step contains "COMMAND cd && make install", which necessarily only works in situations where invoking "cd" is actually the correct way to change directories, and where invoking "make" is actually the correct way to build software.
CMake's -E option provides a way to invoke basic operations like changing/copying/making/removing directories without making such assumptions.
(By the way, if you're using IDEs such as Visual Studio or Xcode, you'll likely want to invoke one or more IDE generators when using CMake. For instance, setting
-G "Eclipse CDT4 - Unix Makefiles" -DCMAKE_ECLIPSE_GENERATE_SOURCE_PROJECT=TRUE
will cause Eclipse projects to be generated in each build area, and also in the source code area that is shared for all builds. Of course, if you are using Xcode or Visual Studio, you'll have to substitute the appropriate flag for those IDEs. Alternatively, you could consider using Eclipse with Ninja on all platforms, though at the time of writing, I am not completely certain that Ninja is ready for prime-time on non-Linux, non-Windows operating systems.)