Ninja equivalent of Make's "build from this directory down" feature (with CMake)? - cmake

When building a project using CMake and Make, you can execute make from a subdirectory of your build tree (i.e. from a directory below whatever directory contains your top-level Makefile), and make will (as far as I can tell) build all targets at or below that directory. This is because CMake generates a Makefile for every directory that contains targets, so when you're in a directory with targets, make finds the Makefile for building those targets.
When CMake generates Ninja files, however, it only generates one build.ninja file, which is at the top level of the build tree. So calling ninja from a directory other than the top-level directory fails (even the -f option doesn't work because ninja can't find the rules.ninja file).
Is there any way to emulate the "make-like" behavior of building targets at and below a directory? As far as I can tell, there are no Ninja targets that correspond to "all targets at and below a particular directory." (This could be emulated using phony targets named after each directory that depend on all targets at and below that directory, but CMake does not generate such targets by default.)

ninja <DIR>/all works with recent versions of Ninja (1.7.2). Version 1.3.4 does not allow this.
I could not find a reference to this on the manual. However, CMake has this documented here:
Recent versions of the ninja program can build the project through the “all” target. An “install” target is also provided.
For each subdirectory sub/dir of the project, additional targets are generated:
sub/dir/all
Depends on all targets required by the subdirectory.
sub/dir/install
Runs the install step in the subdirectory, if any.
sub/dir/test
Runs the test step in the subdirectory, if any.
sub/dir/package
Runs the package step in the subdirectory, if any.

This worked for me:
cd <build-root>
DIRECTORY=<path-relative-to-build-root>
ninja -t targets all | egrep "^${DIRECTORY}/" | egrep CXX_EXECUTABLE_LINKER | \
sed -e 's/:.*//g' | xargs ninja
ninja -t targets all - lists all targets (including target type)
egrep "^${DIRECTORY}/" - filters list of targets to only include those in desired directory
egrep CXX_EXECUTABLE_LINKER - limits the targets to just C++ executables. You can remove or tweak this to get the set of targets you're interested in.
sed -e 's/:.*//g' - removes the target type e.g. ": CXX_EXECUTABLE_LINKER"
xargs ninja - invokes ninja to build the targets

Good question. I would like to know the answer if you find it.
I am just in the process of transitioning to cmake+ninja myself.
I found that I could not create targets with the same name at different levels
(if there is a way I would be interested to know).
So I adopted a naming convention for different targets
E.g.
name - builds program or library
test.name - runs tests for the named program or library
doxygen.name - build doxygen for the named program or library
For deeper hierarchies you can do something like:
doxygen.subproject
doxygen.subproject.name
Using this pattern you can control precisely what is built but you have to issue the command from the top-level build directory.
I think after I get used to this I will find it more productive as there is no need to change directory before you build or run something and though there is sometimes a little extra typing required the shell history generally has it covered.
This is implemented under the hood by using add_custom_target() and adding appropriate dependencies. I use a macro to do this automatically so that
a macro "add_doxygen()" will add the doxygen target for the program and make the doxygen target at each higher level depend on it using add_dependencies().

Related

How to deploy a Find*.cmake file for an Autotools library in the correct place for Yocto?

I created a new layer over an existing Yocto git for my company project.
In this layer I added a few external autotools based libraries.
A few applications need to link against this libraries and the application projects are all cmake based.
Taking one of this libraries (e.g. libcoap) I could easily find some FindCoAP.cmake to add to my library recipe.
Now if I was running on PC, it would simply be a matter of placing this FindCoAP.cmake file in cmake's ${CMAKE_ROOT}/Modules dir, but how should I, from inside a bitbake recipe (do_install hook), proceed to make my Find*.cmake modules available to anyone's dependent projects?
Should I try to get Yocto's cmake CMAKE_ROOT variable from system-information like this or is it a safer and more reliable way?
do_install_append() {
cmake --system-information | grep CMAKE_ROOT | cut -d \" -f2
install -d ${D}/$CMAKE_ROOT}/Modules
install ${S}/FindCoAP.cmake ${D}/$CMAKE_ROOT}/Modules
}
Thanks in advance.
To ship FindFoo.cmake with non-yet-cmake project
The ideal way is to update upstream project itself. So you will update your recipe and package FindFoo.cmake appropriately.
If you want to do it right now:
Add FindFoo.cmake to your layer (into the files directory next to your recipe).
Add that cmake file to SRC_URI (i.e. SRC_URI += "file://FindFoo.cmake").
Install it in do_install into the directory ${D}${datadir}/cmake/Modules/ for example.
Package it to the dev package by FILES_${PN}-dev variable (see example recipes below).
To use that cmake by other recipe
The usual way is to package .cmake files into the ${PN}-dev package. In your case, your application (which depends on the libcoap) will just set DEPENDS = "libcoap" and all the needed files (like headers, libraries and cmake file) will be copied (well, hardlinked) to the sysroot of your application.
CMake modules are packaged in various recipes for example:
libeigen
opencv
json-spirit
Your application is cmake based, so you will use inherit cmake in the recipe. Native module search path is set in cmake.bbclass.
(BTW, I do a build test of libcoap recipe from homeassistant layer and it worked, but obviously there is no cmake shipped.)

Creating a library in CMake depending on source files not available when generating build files

I have a CMake configuration file building two libraries:
a third-party library (here called ThirdPartyLib) containing a real-time OS / board support package from a supplier. It is built outside CMake using the autotools toolchain.
an extended version of the former library (here called ExtendedThirdPartyLib)
Unfortunately, some source code that I need (various tools) are not built in the ordinary build script for (1). Since I don't want to mess with the suppliers build script I want to add another library (2), building the missing files and thus extending the library from the supplier.
I want to able to do something like this in CMakeFiles.txt:
cmake_minimum_required(VERSION 3.2)
project(bsp)
include(ExternalProject)
ExternalProject_Add(
ThirdPartyLib
URL <http://some.url/bsp.tar.bz2
BUILD_COMMAND make -C ../external/ThirdPartyLib/src
)
set_target_properties(ThirdPartyLib PROPERTIES EXCLUDE_FROM_ALL TRUE)
add_library(ExtendedThirdPartyLib
${CMAKE_CURRENT_BINARY_DIR}/some/path/missing_file1.c
${CMAKE_CURRENT_BINARY_DIR}/some/path/missing_file2.c
)
add_dependencies(ExtendedThirdPartyLib ThirdPartyLib)
target_include_directories(ExtendedThirdPartyLib PUBLIC
${CMAKE_CURRENT_BINARY_DIR}/some/path/include
)
target_link_libraries(ExtendedThirdPartyLib ThirdPartyLib)
The problem here is that the path to missing_file1.c and missing_file2.c are not valid when CMake is generating the build files (they are extracted from the tarball from the supplier). CMake exits with an error output saying: "Cannot find source file".
Is there a neat way to make this work? I.e. is it possible to convince CMake that certain non-existant input files will exist when building of the library begins? Or is there any other recommended way to solve this issue?
(I have temporary made local copies of the files I need to build from the suppliers tarball, but that is of course not a good solution. If those files are changed in future versions of the suppliers package and I forget to overwrite my local copies it could be a horrible mess...
Another "solution" would be to create a small makefile outside CMake and use another ExternalProject_Add in the CMakeFiles.txt somehow. But that's not a good solution either, e.g. if compile and linker flags are modified I need to remember to change the makefile too.)
Personally, I dislike the ExternalProject_Add command, because it does way too many things for my taste, but I've digressed.
What if you do something like this, where bar is simulating your ExtendedThirdPartyLib target, since it depends on generated files
cmake_minimum_required(VERSION 3.11)
project(lol C)
set(SOURCES lol.c) # only this file exists
add_library(lol ${SOURCES})
set(FOO_FILES "foo1.c" "foo2.c")
add_custom_command(OUTPUT ${FOO_FILES}
COMMAND ${CMAKE_COMMAND} -E touch ${FOO_FILES}
WORKING_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}"
COMMENT "Creating ${FOO_FILES}"
VERBATIM)
add_custom_target(foo DEPENDS ${FOO_FILES})
add_library(bar ${FOO_FILES})
add_dependencies(bar foo)
target_link_libraries(lol bar)
The whole approach hinges on the fact that the method, where produced/generated files are procured, is explicitly defined via the custom command and associated custom target.
You should modify the custom command to extract the required files (e.g. could even call some external script) from the tarball (which might require downloading with curl or something similar).

CMake workflow?

I am learning CMake and I am having problems in understanding its multi-step workflow. So far, my understanding is that you:
write a CMakeLists.txt
run cmake from a subdirectory to generate a build file (a Makefile, in my case)
run make
However, I don't understand how you should handle different targets (Release vs Debug). By running CMake in two different subdirectories?
Also, I don't understand why you would edit CMakeCache.txt (there is also a GUI tool for that). To emulate what you would accomplish with ./configure by passing different options?
You got it pretty much right. The write CMakeLists.txt > cmake > make sequence is correct.
Regarding different configurations (Debug vs. Release), you have to differentiate between multi-config generators (Visual Studio, XCode), and single-config generators (everything else). With the multi-config generators, you generate one buildsystem (e.g. solution file) which contains all configurations, and choosing between them happens at build time.
With single-config generators, different configurations are obtained by generating different buildsystems, that is, by running CMake multiple times in different directories (and with a different value of the CMAKE_BUILD_TYPE CMake variable).
So you'd do something like this:
> cd my_project/bld/debug
> cmake ../../src -DCMAKE_BUILD_TYPE=Debug
> cd ../release
> cmake ../../src -DCMAKE_BUILD_TYPE=Release
Regarding editing the cache (usually through CMake GUI or ccmake): you're right again, this largely corresponds to passing options to ./configure from AutoMake world. This would be the typical workflow with a freshly downloaded project (using CMake GUI):
Run CMake GUI, point it to the source directory (input) and binary directory (output) you want
Configure. This will fill the cache with project-specified defaults. If CMake cannot find some dependencies of the project automatically, this will end with an error.
Inspect the cache, change any values you don't like (such as compilation options), fill in any missing options (paths to libraries CMake couldn't find etc.)
Repeat steps 2 & 3 until you're satisfied with the project's setup.
Generate the buildsystem.
Exit CMake GUI and build using the appropriate build tool.
What #Angew said. Plus here's an image of the cmake-gui:
Also note that you install it (the CMake GUI) on Ubuntu with sudo apt install cmake-qt-gui, and you run it with cmake-gui.
Source: Where is the CMake GUI for Linux?
Here's my cmake-gui image:

Supress automatic messages in cmake custom targets

I want to add custom targets with cmake but, some of them must be "silent", because it isn't neccesary. For example, for clean custom commands:
// In CMakeLists.txt
add_custom_target(clean-temporaries
${CMAKE_COMMAND} -P clean-temporaries.cmake
COMMENT "Deleting temporary files"
)
// clean-temporaries.cmake
file(GLOB_RECURSE temporary_files "*[~#]")
file(REMOVE ${temporary_files})
$ cmake .
$ make clean-temporals
[100%] Deleting temporary files
[100%] Built target clean-temporaries
$ make clean
$
We can see that CMake prepares "make clean" to not show messages, but, how can I say to CMake I don't want messages in a custom target?
Try adding a minus at the beginning of the command you want to hide from the console.
-make clean
To deal with temporary files littering your source tree:
Encourage contributors to configure their editors so that temporary files end up in a common directory under their $HOME (eg: vim, emacs).
Encourage contributors to configure their global version control ignore files to always ignore the temporary files for their own work environment (eg. for git: vim, emacs).
Additionally exclude well known temporary file patterns in the version control's ignore file of each project, to be friendly to contributors who haven't yet implemented the two previous steps.
If you do that, it's likely that you don't have to put an additional 'optional' (ie. highly environment specific) step into your build system and you end up with a more generally applicable solution to the problem.
As an additional comment on your example code, I'd avoid building in the source tree and use out-of-source builds instead.

cmake creates a ton of files

I'm new to cmake and have just started playing around with it. My problem is that when I invoke cmake like in the tutorial it generates a massive amount of files.
Is there a way to invoke cmake so that all is left after is one unix makefile in the root directory of my project?
The files CMake generates are all useful and shouldn't be deleted.
You could probably write a script which deletes these at the end of running CMake, but I'd strongly recommend that you let CMake work the way it's designed to.
If you keep your source tree separate from your build tree (do an "out of source build" as CMake calls it), there should be no problems. So e.g.
- project_root_dir
|
|- CMakeLists.txt
|- src_dir
- build_dir
If you cd build_dir, then do cmake ../project_root_dir, all the files CMake generates will be in build_dir and won't pollute your source tree.