I'm working on some modifications to the openEMS project. This project uses cmake to build all of its components. The top level CMakeLists.txt file contains the following:
# ...
ExternalProject_Add( openEMS
DEPENDS fparser CSXCAD
SOURCE_DIR ${PROJECT_SOURCE_DIR}/openEMS
CMAKE_ARGS -DCMAKE_TOOLCHAIN_FILE=${CMAKE_TOOLCHAIN_FILE} -DFPARSER_ROOT_DIR=${CMAKE_INSTALL_PREFIX} -DCSXCAD_ROOT_DIR=${CMAKE_INSTALL_PREFIX} -DWITH_MPI=${WITH_MPI} -DCMAKE_INSTALL_PREFIX=${CMAKE_INSTALL_PREFIX}
)
# ...
Inside the openEMS directory, there's another CMakeLists.txt with the following:
# ...
set(SOURCES
openems.cpp
)
# ...
add_library( openEMS SHARED ${SOURCES})
# ...
After building the project successfully once, make does not rebuild anything when, for example, openems.cpp is modified. Why?
$ mkdir build
$ cd build
$ cmake -DBUILD_APPCSXCAD=NO
$ make
[builds all files]
$ touch ../openEMS/openems.cpp
$ make
[ 33%] Built target fparser
[ 66%] Built target CSXCAD
[100%] Built target openEMS
(noting is built)
I have checked and the modification date of openems.cpp is newer than the target. Even deleting the produced library files and binaries, both in the install directory and in the build directory, does not cause it to rebuild anything. The only way I can get it to rebuild is by deleting everything in the build directory and re-running cmake which, of course, rebuilds everything.
This looks like a case of the following. Quoting from the docs for ExternalProject_Add at the section titled "Build Step Options":
BUILD_ALWAYS <bool>
Enabling this option forces the build step to always be run. This can be the easiest way to robustly ensure that the external project's own build dependencies are evaluated rather than relying on the default success timestamp-based method. This option is not normally needed unless developers are expected to modify something the external project's build depends on in a way that is not detectable via the step target dependencies (e.g. SOURCE_DIR is used without a download method and developers might modify the sources in SOURCE_DIR).
If that's the case, the solution would be to add the BUILD_ALWAYS argument to the ExternalProject_Add call like.
ExternalProject_Add( openEMS
DEPENDS fparser CSXCAD
SOURCE_DIR ${PROJECT_SOURCE_DIR}/openEMS
CMAKE_ARGS -DCMAKE_TOOLCHAIN_FILE=${CMAKE_TOOLCHAIN_FILE} -DFPARSER_ROOT_DIR=${CMAKE_INSTALL_PREFIX} -DCSXCAD_ROOT_DIR=${CMAKE_INSTALL_PREFIX} -DWITH_MPI=${WITH_MPI} -DCMAKE_INSTALL_PREFIX=${CMAKE_INSTALL_PREFIX}
BUILD_ALWAYS TRUE
)
If you confirm that this solves the issue, you might want to raise this as an issue to the maintainers of openEMS.
Also note that since the external project there is using CMake as a buildsystem, you could also add the CONFIGURE_HANDLED_BY_BUILD TRUE to the argument list. See the docs for more info.
Edit: The asker opened a GitHub Pull-Request.
I am trying to add boost bjam build as an external cmake project.
This is my cmake code to build boost
include(ExternalProject)
ExternalProject_Add(
libboost
PREFIX ${BOOST_BUILD_DIRECTORY}
SOURCE_DIR ${BOOST_SOURCE_DIRECTORY}
BINARY_DIR ${BOOST_SOURCE_DIRECTORY}
CONFIGURE_COMMAND ""
BUILD_COMMAND ${b2Path}
--build-dir=${BOOST_BUILD_DIRECTORY}
--link=static
--variant=debug
--build-type=complete
INSTALL_COMMAND ${b2Path}
--prefix=${BOOST_BUILD_DIRECTORY}/${BoostFolderName}
--link=static
--variant=debug
--build-type=complete
INSTALL_DIR ${BOOST_BUILD_DIRECTORY}/${BoostFolderName}
LOG_BUILD ON
LOG_INSTALL ON
)
message("boostbuild done")
The problem here is, once in a while it fails. the same step for bjam doesn't fail if invoked through command line.
The error is
type "C:\Users\adhal\AppData\Local\Temp\jam6818c95b4.000" > "C:\build-sdk-Desktop_Qt_5_11_3_MSVC2017_64bit-Debug\external\boost\boost\bin.v2\libs\program_options\build\msvc-14.1\debug\address-model-64\link-static\runtime-link-static\threading-multi\libboost_program_options-variant-vc141-mt-sgd-x64-1_73-static.cmake"
...failed text-action C:\build-sdk-Desktop_Qt_5_11_3_MSVC2017_64bit-Debug\external\boost\boost\bin.v2\libs\program_options\build\msvc-14.1\debug\address-model-64\link-static\runtime-link-static\threading-multi\libboost_program_options-variant-vc141-mt-sgd-x64-1_73-static.cmake...
...skipped <pC:\proj\sdk\external\boost-program-options\stage\lib\cmake\boost_program_options-1.73.0>libboost_program_options-variant-vc141-mt-sgd-x64-1_73-static.cmake for lack of <pC:\build-sdk-Desktop_Qt_5_11_3_MSVC2017_64bit-Debug\external\boost\boost\bin.v2\libs\program_options\build\msvc-14.1\debug\address-model-64\link-static\runtime-link-static\threading-multi>libboost_program_options-variant-vc141-mt-sgd-x64-1_73-static.cmake...
compile-c-c++ C:\build-sdk-Desktop_Qt_5_11_3_MSVC2017_64bit-Debug\external\boost\boost\bin.v2\libs\regex\build\msvc-14.1\debug\address-model-64\link-static\runtime-link-static\threading-multi\regex.obj
When I run the exact same step, it doesn't fild the file(C:\Users\adhal\AppData\Local\Temp\jam6818c95b4.000). Since it's a temporary file, it's deleted.
Can someone tell me, if I am doing anything wrong ? Is it possible to specify a separate temporary directory for bjam ?
Got same issue. In my case moving build folder up helped.
Issue check story - tmp .bat file contains
type "C:\Users\amorozov\AppData\Local\Temp\jam43e8891f.000" > "bin.v2\libs\program_options\build\msvc-14.1\debug\address-model-64\link-static\runtime-link-static\threading-multi\libboost_program_options-variant-vc141-mt-sgd-x64-1_73-static.cmake"
Running it from cmake binary dir where build happens fails too. Splitting command to type and dir for target directory succeeds. Writing to file with shorter name succeeds. Checked total filepath length for .cmake file and it's over 256 bytes. It seems as cmd doesn't allow redirecting to really long pathnames. Moving whole project few directories up solved my issue.
If that doesn't help, check your tmp .bat file by running b2 under windbg with breakpoint on kernel32!DeleteFileW. Filename is du poi(#esp+4). Hope that helps.
I basically ask the same question as has been ask here. The question has however not been answered.
I want to use googletest in my project. For this I'm using ExternalProject_Add() which clones the testsuite with git. After that, I like to use add_subdirectory().
This is also what is described in the official repository. The nice thing about this approach is, that the build scripts in googletest handle the building process themself.
The problem is however, that add_subdirectory() can not find the source folder, since it does not exists from the start. Therefore, add_subdirectory() should depend on the completion of ExternalProject_Add().
Is is possible to make add_subdirectory() dependent of ExternalProject_Add(), like add_dependencies() does for targets?
PS. I can make it all compile if I comment add_subdirectory() out, build it (which ends with an error because the googletest library is missing), uncomment it and build it again (success).
ExternalProject_Add(
googletest
GIT_REPOSITORY https://github.com/google/googletest.git
GIT_TAG master
CONFIGURE_COMMAND ""
BUILD_COMMAND ""
INSTALL_COMMAND ""
TEST_COMMAND ""
)
ExternalProject_Get_Property(googletest source_dir binary_dir)
set(GTEST_INCLUDE_DIR ${source_dir}/googletest/include)
set(GMOCK_INCLUDE_DIR ${source_dir}/googlemock/include)
add_subdirectory(${source_dir}
${binary_dir})
I used this tutorial to accomplish that. Just put your ExternalProject code in a separate file, say "CMakeLists.txt.dependencies" and then launch another cmake with execute_process. I use configure_file first to inject configuration information into the external project and to copy it into the build tree.
configure_file(CMakeLists.txt.dependency.in dependency/CMakeLists.txt)
execute_process(COMMAND "${CMAKE_COMMAND}" -G "${CMAKE_GENERATOR}" .
WORKING_DIRECTORY "${CMAKE_BINARY_DIR}/dependency" )
execute_process(COMMAND "${CMAKE_COMMAND}" --build .
WORKING_DIRECTORY "${CMAKE_BINARY_DIR}/dependency" )
Notice that the tutorial has the ExternalProject code in a separate CMakeLists.txt file. So does gtest official documentation. That is because ExternalProject runs at build-time, not configure time. This is a hack to use ExternalProject at configure time by putting it in a third project and executing a separate CMake+build run in the middle of your main CMake configuration step.
With version 3.11 CMake added a FetchContent command. I haven't used it, but the documentation makes it look like a configuration-time replacement for ExternalProject. That would fetch the content at configuration time, making it available for a later add_subdirectory command.
The ExternalProject and FetchContent commands will work fine for a small number of small and/or obscure dependencies. If your dependencies are more than 2, large, and/or popular, I recommend you look at a dependency manager like Conan. It partners with CMake quite well.
You are not following the method correctly. There should only be ExternalProject_Add() as the final command in the CMakeLists.txt.in file.
The CMakeLists.txt.in file is called by the outer CMakeLists.txt file so that the subproject processing happens at configure time via execute_process(). CMakeLists.txt.in is acting as just a glorified downloader.
So, all other commands like add_subdirectory() are added to the outer CMakeLists.txt file.
CMAKE : I only want to build part of a downloadable external project
The external project I want to use has the following structure :
- ExternalProject
- Subproject A <- this i care for
- Subproject B
It's all in the same archive that is available for download. The problem is that Subproject B has more dependencies that I don't want in my project, also it's not relevant for what I'm doing. The subprojects are buildable on their own, so for now I just took Subproject A out of the archive and put it in my main project which is working fine, but I'd like to not deploy the external project with my project but allow the user to download the external project on his own when running cmake. Sadly there is no archive for each of the subprojects on their own so all I can download is the full external project.
What I want is to tell CMAKE to download and unpack the whole archive ExternalProject but then only add Subproject A to my project. I read all the documentation on ExternalProject_add but it mostly allows for detailed configuration of the build parameters of the project. Maybe I'm just looking for the wrong keyword and my question is really simple to answer - or it is just not possible.
If someone could point me towards the right approach here I'd be very thankful.
Actual project :
The mentioned archive has two subfolders "octomap" and "octovis". octomap is the one I want to build while octovis will create a lot of errors if the dependencies are not met.
ExternalProject_Add(octomap-1.6.5
URL https://github.com/OctoMap/octomap/archive/v1.6.5.tar.gz
URL_MD5 de09b1189a03ac8cbe4f813951e86605
SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/octomap/
CMAKE_ARGS "-DCMAKE_INSTALL_PREFIX=${EXTERNAL_INSTALL_DIR}"
)
You can define all commands explicitly. For instance you have a directory structure:
FooBar/
- CMakeLists.txt
- Foo/
- - CMakeLists.txt # You wanna build
- Bar/
- - CMakeLists.txt # You wanna ignore
Example with hardcoded paths:
set(ext_dir "${CMAKE_BINARY_DIR}/third-party-activities/ExternalProject/")
set(foobar_dir "${ext_dir}/FooBar")
ExternalProject_Add(
FooBar
URL "your-url-here"
SOURCE_DIR "${foobar_dir}/Source"
CONFIGURE_COMMAND
"${CMAKE_COMMAND}"
"-H${foobar_dir}/Source/Foo"
"-B${foobar_dir}/Builds"
"-DCMAKE_INSTALL_PREFIX=${EXTERNAL_INSTALL_DIR}"
BUILD_COMMAND
"${CMAKE_COMMAND}" --build "${foobar_dir}/Builds"
INSTALL_COMMAND
"${CMAKE_COMMAND}" --build "${foobar_dir}/Builds" --target install
)
Update
Note that it's much easier to patch parent project like this:
# FooBar/CMakeLists.txt
option(BUILD_BAR_SUBPROJECT "Build targets from subproject Bar" ON)
...
add_subdirectory(Foo)
if(BUILD_BAR_SUBPROJECT)
add_subdirectory(Bar)
endif()
... and now you don't need to hack ExternalProject_Add so much:
ExternalProject_Add(
FooBar
URL "your-url-here"
CMAKE_ARGS -DBUILD_BAR_SUBPROJECT=OFF "-DCMAKE_INSTALL_PREFIX=${EXTERNAL_INSTALL_DIR}"
)
I'm pretty new to CMake, and read a few tutorials on how to use it, and wrote some complicated 50 lines of CMake script in order to make a program for 3 different compilers. This probably concludes all my knowledge in CMake.
Now my problem is that I have some source code, whose folder I don't want to touch/mess with when I make the program. I want that all CMake and make output files and folders to go into ../Compile/, so I changed a few variables in my CMake script for that, and it worked for sometime when I did something like this on my laptop:
Compile$ cmake ../src
Compile$ make
Where with that I had a clean output in the folder I'm in right now, which is exactly what I'm looking for.
Now I moved to another computer, and recompiled CMake 2.8.11.2, and I'm almost back to square one! It always compiles the thing into the src folder where my CMakeLists.txt is located.
The part where I choose the directory in my CMake script is this:
set(dir ${CMAKE_CURRENT_SOURCE_DIR}/../Compile/)
set(EXECUTABLE_OUTPUT_PATH ${dir} CACHE PATH "Build directory" FORCE)
set(LIBRARY_OUTPUT_PATH ${dir} CACHE PATH "Build directory" FORCE)
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${dir})
set(CMAKE_BUILD_FILES_DIRECTORY ${dir})
set(CMAKE_BUILD_DIRECTORY ${dir})
set(CMAKE_BINARY_DIR ${dir})
SET(EXECUTABLE_OUTPUT_PATH ${dir})
SET(LIBRARY_OUTPUT_PATH ${dir}lib)
SET(CMAKE_CACHEFILE_DIR ${dir})
And now it always ends with:
-- Build files have been written to: /.../src
Am I missing something?
It sounds like you want an out of source build. There are a couple of ways you can create an out of source build.
Do what you were doing, run
cd /path/to/my/build/folder
cmake /path/to/my/source/folder
which will cause cmake to generate a build tree in /path/to/my/build/folder for the source tree in /path/to/my/source/folder.
Once you've created it, cmake remembers where the source folder is - so you can rerun
cmake on the build tree with
cmake /path/to/my/build/folder
or even
cmake .
if your current directory is already the build folder.
For CMake 3.13 or later, use these options to set the source and build folders
cmake -B/path/to/my/build/folder -S/path/to/my/source/folder
For older CMake, use some undocumented options to set the source and build folders:
cmake -B/path/to/my/build/folder -H/path/to/my/source/folder
which will do exactly the same thing as (1), but without the reliance on the current working directory.
CMake puts all of its outputs in the build tree by default, so unless you are liberally using ${CMAKE_SOURCE_DIR} or ${CMAKE_CURRENT_SOURCE_DIR} in your cmake files, it shouldn't touch your source tree.
The biggest thing that can go wrong is if you have previously generated a build tree in your source tree (i.e. you have an in source build). Once you've done this the second part of (1) above kicks in, and cmake doesn't make any changes to the source or build locations. Thus, you cannot create an out-of-source build for a source directory with an in-source build. You can fix this fairly easily by removing (at a minimum) CMakeCache.txt from the source directory. There are a few other files (mostly in the CMakeFiles directory) that CMake generates that you should remove as well, but these won't cause cmake to treat the source tree as a build tree.
Since out-of-source builds are often more desirable than in-source builds, you might want to modify your cmake to require out of source builds:
# Ensures that we do an out of source build
MACRO(MACRO_ENSURE_OUT_OF_SOURCE_BUILD MSG)
STRING(COMPARE EQUAL "${CMAKE_SOURCE_DIR}"
"${CMAKE_BINARY_DIR}" insource)
GET_FILENAME_COMPONENT(PARENTDIR ${CMAKE_SOURCE_DIR} PATH)
STRING(COMPARE EQUAL "${CMAKE_SOURCE_DIR}"
"${PARENTDIR}" insourcesubdir)
IF(insource OR insourcesubdir)
MESSAGE(FATAL_ERROR "${MSG}")
ENDIF(insource OR insourcesubdir)
ENDMACRO(MACRO_ENSURE_OUT_OF_SOURCE_BUILD)
MACRO_ENSURE_OUT_OF_SOURCE_BUILD(
"${CMAKE_PROJECT_NAME} requires an out of source build."
)
The above macro comes from a commonly used module called MacroOutOfSourceBuild. There are numerous sources for MacroOutOfSourceBuild.cmake on google but I can't seem to find the original and it's short enough to include here in full.
Unfortunately cmake has usually written a few files by the time the macro is invoked, so although it will stop you from actually performing the build you will still need to delete CMakeCache.txt and CMakeFiles.
You may find it useful to set the paths that binaries, shared and static libraries are written to - in which case see how do I make cmake output into a 'bin' dir? (disclaimer, I have the top voted answer on that question...but that's how I know about it).
There's little need to set all the variables you're setting. CMake sets them to reasonable defaults. You should definitely not modify CMAKE_BINARY_DIR or CMAKE_CACHEFILE_DIR. Treat these as read-only.
First remove the existing problematic cache file from the src directory:
cd src
rm CMakeCache.txt
cd ..
Then remove all the set() commands and do:
cd Compile && rm -rf *
cmake ../src
As long as you're outside of the source directory when running CMake, it will not modify the source directory unless your CMakeList explicitly tells it to do so.
Once you have this working, you can look at where CMake puts things by default, and only if you're not satisfied with the default locations (such as the default value of EXECUTABLE_OUTPUT_PATH), modify only those you need. And try to express them relative to CMAKE_BINARY_DIR, CMAKE_CURRENT_BINARY_DIR, PROJECT_BINARY_DIR etc.
If you look at CMake documentation, you'll see variables partitioned into semantic sections. Except for very special circumstances, you should treat all those listed under "Variables that Provide Information" as read-only inside CMakeLists.
Turning my comment into an answer:
In case anyone did what I did, which was start by putting all the build files in the source directory:
cd src
cmake .
cmake will put a bunch of build files and cache files (CMakeCache.txt, CMakeFiles, cmake_install.cmake, etc) in the src dir.
To change to an out of source build, I had to remove all of those files. Then I could do what #Angew recommended in his answer:
mkdir -p src/build
cd src/build
cmake ..
As of CMake Wiki:
CMAKE_BINARY_DIR
if you are building in-source, this is the same as CMAKE_SOURCE_DIR, otherwise this is the top level directory of your
build tree
Compare these two variables to determine if out-of-source build was started
You should not rely on a hard coded build dir name in your script, so the line with ../Compile must be changed.
It's because it should be up to user where to compile.
Instead of that use one of predefined variables:
http://www.cmake.org/Wiki/CMake_Useful_Variables
(look for CMAKE_BINARY_DIR and CMAKE_CURRENT_BINARY_DIR)
Starting from cmake 3.19 you can use also preset files, where you can specify among other useful things also the output binary dir:
{
"version": 2,
"cmakeMinimumRequired": {
"major": 3,
"minor": 19,
"patch": 0
},
"configurePresets": [
{
"name": "default",
"displayName": "Default",
"description": "Default build cfg",
"generator": "Unix Makefiles",
"binaryDir": "${sourceDir}/Compile",
"cacheVariables": {
},
"environment": {
}
}
]
}
Then just run cmake with --preset arg:
cmake --preset=default
Then just cd to your build dir and run make, in your case:
cd ./Compile
make