When building a CMake project (e.g. on Windows), I can perform a parallel build by executing the following command:
cmake --build . -- /m
The /m switch is passed to msbuild and I get parallelized builds. However, if I have some external project in my CMakeLists, that switch is not passed through to its build command.
What's the best way to ensure my ExternalProject is built in parallel? I don't want to hardcode a BUILD_COMMAND if I can avoid it.
In CMake 3.12, support was added to kick off parallel jobs from within cmake.
The previous versions only supported this via passthrough arguments after the --, which understandably wouldn't pass through to external builds. Though, I'm not sure if the new support works for ExternalProject, it'd be great to try and report back if it works!
I had the same problem with make based external_projects. I ended up with the solution to pass the -j8 flag via a cmake cached variable.
set(EXTERNAL_BUILD_ARGS "/m" CACHE STRING "flags for external project builds")
Related
Prologue
I have a preset-based plain CMake project so that I can build and test it with cmake --preset $PRESET && cmake --build --preset $PRESET && ctest --preset $PRESET. Note that it nicely interacts with Microsoft's CMake Tools extension for Visual Studio Code, be it for building, testing, debugging and Intellisense.
Since I want to handle multiple presets in parallel, I set CMakePresets.json's binaryDir property to ${sourceDir}/build/${presetName}/.
Issue
I want to also build this plain CMake project with colcon. colcon build --cmake-args "--preset $PRESET" doesn't work, though, as it produces
WARNING:colcon.colcon_cmake.task.cmake.build:Could not build CMake package 'root_project_name' because the CMake cache has no 'CMAKE_PROJECT_NAME' variable
root_project_name being the argument to CMake's project() command in the top CMakeLists.txt.
How can I resolve this warning and the subsequent build failure?
Straightforward solution
Not setting CMakePresets.json's binaryDir property at all works fine with colcon, but doesn't allow for multiple preset builds in parallel.
Solution with multiple preset builds in parallel
The reason for this behavior is colcon-core's build verb's passing the build base directory (default: build) suffixed by the found package's name (here: root_project_name) to the colcon-cmake extension here.
The solution is to pass the correct build base to colcon (i.e. colcon build --build-base ./build/$PRESET/ --cmake-args "--preset $PRESET") and to adapt your CMakePresets.json's binaryDir property to ${sourceDir}/build/${presetName}/root_project_name/.
Note that this then works with colcon test as well, i.e. colcon test --build-base ./build/$PRESET/ --ctest-args "--preset $PRESET".
I've written a module on top of a private fork off of TensorFlow that uses nanomsg.
For my local development server, I used cmake install to install nanomsg (to /usr/local) and accessed the header files from their installed location. The project runs fine locally.
However, I now need to package nanomsg within my TensorFlow workspace. I've tried the following two approaches, and find neither satisfactory:
Similar to this answer for OpenCV, I precompiled nanomsg into a private repository, loaded it within my workspace (within tensorflow/workspace.bzl) using an http_archive directive then included the headers and libraries in the relevant build script. This runs fine, but is not a portable solution.
A more portable solution, I created a genrule to run a specific sequence of cmake commands that can be used to build nanomsg. This approach is neater, but the genrule cannot be reused to cmake other projects. (I referred to this discussion).
Clearly cmake is not supported as a first-class citizen in Bazel builds. Is there anyone who has faced this problem in your own projects created a generic, portable way to include libraries within Bazel projects that are built using cmake? If so, how did you approach it?
As Ulf wrote, I think your suggested option 2 should work fine.
Regarding "can I identify if the cmake fails", yes: cmake should return with an error exit code (!= 0) when it fails. This in turn will cause Bazel to automatically recognize the genrule action as failed and thus fail the build. Because Bazel sets "set -e -o pipefail" before running your command (cf. https://docs.bazel.build/versions/master/be/general.html#genrule-environment), it should also work if you chain multiple cmake commands in your genrule "cmd".
If you call out to a shell script in your "cmd" attribute that then actually runs the cmake commands, make sure to put "set -e -o pipefail" in the first line of your script yourself. Otherwise the script will not fail when cmake fails.
If I misunderstood your question "Can I identify if the cmake fails", please let me know. :)
This new project: https://github.com/bazelbuild/rules_foreign_cc seems like a solution(it build rules for cmake to build your project inside bazel).
I have a simple CMakeLists.txt
add_executable(myexecutable test.cpp)
I don't intend to use CMake to actually indirectly be responsible for building the program. I intend to use it so I can use CLion and have a nice IDE experience for working on my c++ program.
If I hit build in CLion, or cmake --build on the commandline, I want it to invoke my real buildtool. How can I specify a buildtool to run instead of the normal build cmake/clion would do? I have only been able to find add_custom_command which can be run before/after the normal cmake/clion build.
You need some fake target to let CLion understand which files are there in the project. Then you can add add_custom_target with your own build commands. So if you avoid building fake target, but instead run custom targets via run configuration by name (after CMake reload CLion will automatically create run configurations for such targets), they will call build for you.
I am trying to use Ninja + CMake to build a project.
This project has a custom target that takes additional arguments.
E.g. make target option=value
It works fine in make, however I am not sure how to get Ninja to take in additional command line arguments.
Is this possible with a Ninja build?
I don't think it's possible to do directly through Ninja. I just scanned through the Ninja documentation to double check and didn't see anything.
Instead, you could modify CMake cache variables via CMake (see cmake -D and cmake -L). That way you could change your build on the fly, or create a few different build directories with different settings in each one.
In CMake I currently have a simple Python script to generate a header, but if I update the script itself CMake won't re-run the script. Is there a way I can get CMake to do this?
It seems you are directly invoking your code generation script when cmake is run. While it is possible solution but it is definitely not a right way to use code generators with cmake.
I recommend you to use add_custom_command for your case:
add_custom_command(
OUTPUT generated.h
COMMAND ${PYTHON_EXECUTABLE} generator.py
DEPENDS generator.py
)
And next you can simple put your header to the list of source files passed to add_library/add_executable commands. cmake will automatically track all the dependencies and invoke your script.
Term DEPENDS generator.py informs cmake that it should regenerate header if script is changed.
With this approach file generated.h will be generated only at build time (when you run make or execute a build command in IDE). In contrast if you are running your script at cmake time (with execute_process command) then you have to rerun cmake to regenerate your file. Which is possible but you need to use some tricks to introduce a non-standard dependency.