I'm using CMake for a project where I want to bundle Clang. I use ExternalProject_Add to build clang from source. However, since Clang and LLVM is huge, a make with nothing changed takes 45 seconds.
Is there a way to make CMake just build the ExternalProject once, and then not even check if anything has changed on subsequent runs if it has already been built successfully?
The best way to use ExternalProject_Add() is to structure your project as a superbuild. This means that your top-level project (the "superbuild") does not build any actual code and instead consists only of ExternalProject_Add calls. Your "real" project is added as one of these "external" projects. This allows you to set up the superbuild with all dependencies, ordering, etc.
The workflow is then as follows:
Generate the superbuild project.
Build the superbuild project. This will build and install all dependencies, and also generate (and build) your real project.
Switch to the buildsystem generated for your real project and start doing further development using that. Your dependencies are already correctly set up and installed by the build of the superbuild project in the previous step, and will never be checked for out-of-dateness by the build.
If you ever need to change the setup of your dependencies, do it and build the superbuild again.
Related
I'm porting a project to linux and cmake. It is part of a huge project, each subproject in a folder. I want to build only my project and its dependencies.
I can comment the other projects from the main CMakeList.txt file, but that doesn't work because there are a few dependencies to other projects (which may also depend on other projects...)
I don't want to modify all the cmake files from other projects to build only the small pieces that I need because is very error prone and time consuming. I could checkin something by mistake which I was not supposed to, for example.
My main problem is that the other projects break continuisly (almost everybody else is just working in windows, so the linux build is not taken good care of) and is slowing me down a lot.
My question is: Is there a way to tell cmake: run through all the project, but compile only what is needed to compile certain subfolder?
You can do:
add_subdirectory(dir EXCLUDE_FROM_ALL)
and then all targets within that subdirectory (and recursive) are by default marked with EXCLUDE_FROM_ALL, that way they will not build by default.
You may also specify the specific targets to be build with:
cmake --build the_build_dir --target this_target -t another_target
We are converting a large Makefile based project to a CMake based system. I have numerous dependencies that I need to build prior to building our code. The first three dependencies are build using the following:
add_subdirectory(dependencies/libexpat/expat)
add_subdirectory(dependencies/libuuid-1.0.3)
add_subdirectory(dependencies/log4c-1.2.4)
expat has it's own CMakeLists.txt file and build with no problems. I would like to force expat to install to the staging directory before continuing. For libuuid I am using a ExternalProject_Add and as part of that process it does install into the staging directory.
Then when I build log4c, which needs expat, I can point it to the location of expat. Otherwise I would need to someone get access to the absolutely path for the temporary build location of expat.
I've tried to add the following after add_subdirectory:
add_subdirectory(dependencies/libexpat/expat)
add_subdirectory(dependencies/libuuid-1.0.3)
install(TARGETS expat LIBRARY DESTINATION ${CMAKE_INSTALL_PREFIX}/usr/lib)
add_subdirectory(dependencies/log4c-1.2.4)
Unfortunately CMake will not run expat's install code. How do I force expat to install after building but before it builds the rest of the project?
This looks like the primary use case for ExternalProject_Add, which is best used as a superbuild setup. This means that your top-level project (the "superbuild") does not build any actual code and instead consists only of ExternalProject_Add calls. Your "real" project is added as one of these "external" projects. This allows you to set up the superbuild with all dependencies, ordering, etc.
The workflow is then as follows:
Generate the superbuild project.
Build the superbuild project. This will build and install all dependencies, and also generate (and build) your real project.
Switch to the buildsystem generated for your real project and start doing further development using that. Your dependencies are already correctly set up and installed by the build of the superbuild project in the previous step.
With a growing codebase, it makes sense to organize it between separate repositories, each repo being a separate CMake-managed project.
Because of modularity, this usually means you end up in a situation where a CMake-managed project Application depends on another CMake-managed project Library, while both are internal code (i.e., code owned and maintained by your structure).
The automatic dependency recompilation issue
Then, if some sources in Library are modified, it needs to be recompiled in order to build Application. The question being:
Is it possible to have the "build Application" command (a button in an IDE, or a call to make on the command line) to first rebuild Library if Library files changed ?
I'd suggest to use the ExternalProject_Add command.
The documentation has slightly changed for the different versions:
CMake v2.8.9 ExternalProject
CMake v3.0. ExternalProject
CMake v3.3 ExternalProject
In case you encounter problems with getting the dependencies right, this thread might help you.
By looking at how the OpenChemistry parent-project does it, and with the confirmation by normanius's answer, it turns out this can be achieved with relatively few CMake script code.
It turns out that CMake CLI is offering an abstraction over the "build" action of the targeted build systems. See --build option.
ExternalProject_Add can be seen as a wrapper to use this CLI interface directly from CMake scripts.
Imagine there is a CMake-managed repository, building libuseful, and a separate CMake-managed repo, building appawesome with a dependency on libuseful.
find_package(libuseful CONFIG) # The usual way to find a dependency
# appawesome is the executable we are building, it depends on libuseful
add_executable(appawesome main.cpp)
target_link_libraries(appawesome libuseful)
Adding automatic rebuild
Then it is possible to make building appawesome systematically first try to rebuild libuseful with some code looking like:
ExternalProject_Add(EP_libuseful)
SOURCE_DIR <libuseful_sourcedir> # containing libuseful's root CMakeLists.txt
BINARY_DIR <libuseful_binarydir> # containing libuseful's CMakeCache.txt
BUILD_ALWAYS 1 # Always rebuild libuseful
)
add_dependencies(libuseful EP_libuseful)
The last line is quite important: find_package() in config mode should make a libuseful imported targed available. The call to ExternalProject_Add made a build target EP_libuseful available (which is a custom build step, building libuseful). The last line just makes sure that libuseful depends on its build step.
A quote from the official documentation:
"Specify rules to run at install time."
What exactly is install time?
The problem for me: I am on Linux, software is installed from packages that are just dependencies and data. There is no CMake that can do anything here. So installation time of software is out of scope from CMake. So what exactly do they mean?
Building a CMake project can roughly be divided into three phases:
Configure time. This includes everything that happens while running cmake itself. This phase is concerned with inspecting certain properties of the host system and generating the specific build files for that platform under the selected configuration.
Build time. This includes everything that happens while actually building your project from the files generated by CMake (like, when running cmake --build or make). This is where all of the actual compilation and linking happens, so at the end of the build phase, you have a usable binary.
Install time. This includes everything that happens when running the INSTALL target generated by CMake (like, when running cmake --build --target install or make install). This takes care of copying the binaries that were generated into the build tree to a different directory. Note that the build tree contains a lot of stuff that is no longer needed once the build is completed if you are only interested in running the binary. Examples include all intermediate build artifacts, like the build files generated during the configure phase or the intermediate object files created during the build phase. Furthermore, the install phase might include additional steps to ensure that the binaries produced during the build are portable. For instance, on Linux systems you might want to remove the build directory from the shared library search path in the binary and replace it with a portable equivalent. So the install phase might do more than just copy all the important files to a new directory. It could also include additional steps that change the binaries to make them more portable.
Note that the last phase is optional. If you do not want to support calling make install but prefer another deployment mechanism, you simply don't use the install command in your CMake script and no INSTALL target will be generated.
I'd like to expand the answer, which ComicSansMS gave you, a little bit.
As he mentioned - CMake generates an extra target called install for the make tool (when you use a Makefile-based generator).
It may look weird for you as a package system is used for Linux. However the install target is still useful or even necessary:
When you develop your application you may need to install (move binaries and possibly some include files) to a certain location so some of your projects may see each other. For example, you may develop a library and a set of non-related applications which use it. Then this library must be installed somewhere to be visible. It doesn't mean you need to put it to the /usr directory; you may use your /home.
The process of Linux package preparation requires an install step. For example, the RPM packaging system does three main steps when the rpm package file is being built: the project is configured, then is compiled and linked and finally is being installed to a certain location. All files from this location are being packed to the rpm file.
If we have a case of highly decentralized development environment, where there are many repositories and projects, is there an existing functionality in CMake that automatically finds dependencies between targets without a top level CMake file?
The workflow is something like this, you specify a directory and all targets are default-configured in the given tree. Then you can go and build any of the projects. I am looking for a behavior similar to that when you build the Android OS.
There is no build-time dependency tracking in CMake across different projects. For this case you need to have a project on the top-level which adds all the subdirectories, so that the target names are available inside a single CMake project.
I am aware of one helper script around CMake which provides the required inter-project dependencies: https://github.com/aldebaran/qibuild
I would say that is getting close to a mature code base. However, it requires additional descriptor files for each project. Might be worth to have a look at it.