I have a project for which I want to apply a patch to a git repo before building the code, then unapply that patch after creating the executable. The reason being that I want the patch applied to the build but I want the repository that is patched to stay clean on its master branch. I noticed add_custom_command seemed to have this capability, with PRE_BUILD and POST_BUILD specifiers. I tried something like this:
add_executable(foo ...)
# foo requires a patch to some library
target_link_libraries(foo ...)
add_custom_command(TARGET foo PRE_BUILD COMMAND git apply somepatch.patch)
add_custom_command(TARGET foo POST_BUILD COMMAND git apply --reverse somepatch.patch)
However, if one looks closer at the documentation of add_custom_command they mention this in the section about the PRE_BUILD specifer:
On Visual Studio Generators, run before any other rules are executed within the target. On other generators, run just before PRE_LINK commands.
And indeed this is what happens. The patch is only getting applied after the compilation phase, and before the linking phase. So the code is being compiled before the patch is applied. How can I ensure the patch is performed before compiling the code?
Related
Let's say I have a Python script which does something with just built executable. And I want CMake to rebuild that executable if the script was modified (actually it is enough to just re-run the script, but rebuild an executable is fine too).
add_executable(App src/main.cpp)
add_custom_command(
TARGET App
POST_BUILD
COMMAND "${Python3_EXECUTABLE}" ARGS "scripts/do_stuff.py" "$<TARGET_FILE:App>"
WORKING_DIRECTORY "${CMAKE_CURRENT_LIST_DIR}"
)
How can I achieve that? add_custom_command with TARGET argument doesn't support DEPENDS argument. add_dependency(App "scripts/do_stuff.py") produces an error, because "scripts/do_stuff.py" is not a target, but just a file.
Running the script is very important for correct working of the executable so I don't want define completely separate target via add_custom_command allowing bypass script execution by building just App target.
actually it is enough to just re-run the script
So the executable does not depend on the script. So re-run the script, not the executable.
add_executable(app src/main.cpp)
add_custom_command(
OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/did_do_stuff
COMMAND "${Python3_EXECUTABLE}" "scripts/do_stuff.py" "$<TARGET_FILE:app>"
COMMAND ${CMAKE_COMMAND} -E touch ${CMAKE_CURRENT_BINARY_DIR}/did_do_stuff
DEPENDS "$<TARGET_FILE:app>"
"${CMAKE_CURRENT_LIST_DIR}/scripts/do_stuff.py"
WORKING_DIRECTORY "${CMAKE_CURRENT_LIST_DIR}"
)
add_custom_target(do_stuff
DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/did_do_stuff
)
And build target do_stuff (or all) to run it.
My question is very similar to cmake: add_custom_command / add_custom_target ignoring dependency
But the answer specified does not solve my issue, and also it is for a newer version of cmake (3.20)
I want some files (shader files) to be copied to the executable directory every time the shader source changes
So I have the following code in cmake:
add_custom_command(
OUTPUT ${CMAKE_BINARY_DIR}/Shaders.txt
COMMAND ${CMAKE_COMMAND} -E echo "Actually Copying shaders"
COMMAND ${CMAKE_COMMAND} -E copy_directory ${CMAKE_CURRENT_SOURCE_DIR}/assets/ $<TARGET_FILE_DIR:Editor>/assets
COMMAND ${CMAKE_COMMAND} -E touch ${CMAKE_BINARY_DIR}/Shaders.txt
DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/assets
)
add_custom_target(CopyShaders ALL DEPENDS ${CMAKE_BINARY_DIR}/Shaders.txt)
Now, as I understand, CopyShaders will always be built (since it is always out of date), but Shaders.txt should only be built once (after the shader source is changed), after which it is up to date
I'm trying to build the target CopyShaders using cmake --build build --target CopyShaders
If I'm using MinGW, then "Actually copying shaders" never gets printed if Shaders.txt is present, even if the assets folder containing the shaders has been modified
If I'm using MSVC, I get the following warning on the terminal:
warning MSB8064: Custom build for item "D:\Acads\Programming\opengl\SummerOfCode\build\CMakeFiles\05fb3856b7a5e1f6ce1ea66ca9091779\Shaders.txt.rule" succeeded, but specified dependency "d:\acads\programming\opengl\summerofcode\editor\assets" does not exist. This may cause incremental build to work incorrectly. [D:\Acads\Programming\opengl\SummerOfCode\build\Editor\CopyShaders.vcxproj]
Again, I do have the folder existing
I figured it out
DEPENDS should only take as input files, not folders
Replacing it with all the files inside /assets works
I am having an issue with CMakes Add_External_Project functionality (more of an annoyance than anything else). Specifically, I do not understand the keys CONFIGURE_COMMAND, BUILD_COMMAND and INSTALL_COMMAND.
In the following (working) example, which downloads Google's test library, the two files at the end of the question will ensure that the third party libraries are downloaded and built (not installed).
However, when I tried to add configure and build commands as "CONFIGURE_COMMAND" and "BUILD_COMMAND" (cmake . and cmake --build) instead of having to do execute_process CMake craps out with the error message:
[ 55%] Performing configure step for 'googletest'
/bin/sh: 1: cmake .: not found
Am I trying to do something that is obviously not within the scope of the Add_External_Project functionality?
Example Files:
CMakeLists.txt
cmake_minimum_required (VERSION 3.0)
project (Test VERSION 0.1.0.0 LANGUAGES CXX)
# Download and unpack googletest at configure time
configure_file("${CMAKE_CURRENT_SOURCE_DIR}/CMakeLists.txt.in" "${CMAKE_BINARY_DIR}/googletest-download/CMakeLists.txt" #ONLY)
execute_process(COMMAND ${CMAKE_COMMAND} -G "${CMAKE_GENERATOR}" . WORKING_DIRECTORY "${CMAKE_BINARY_DIR}/googletest-download" )
execute_process(COMMAND ${CMAKE_COMMAND} --build . WORKING_DIRECTORY "${CMAKE_BINARY_DIR}/googletest-download")
add_subdirectory("${CMAKE_BINARY_DIR}/googletest-src" "${CMAKE_BINARY_DIR}/googletest-build")
CMakeLists.txt.in
cmake_minimum_required(VERSION 3.0)
project(third-party NONE)
include(ExternalProject)
ExternalProject_Add(googletest
GIT_REPOSITORY https://github.com/google/googletest.git
GIT_TAG master
SOURCE_DIR "#CMAKE_BINARY_DIR#/googletest-src"
BINARY_DIR "#CMAKE_BINARY_DIR#/googletest-build"
CONFIGURE_COMMAND ""
BUILD_COMMAND ""
INSTALL_COMMAND ""
TEST_COMMAND ""
)
If you don't specify CONFIGURE_COMMAND at all, it will assume a CMake project and run the appropriate cmake command for you (by appropriate, I mean it will use the same CMake generator as your main build, etc.). Similarly, if you leave out BUILD_COMMAND, it will also assume a CMake project and do cmake --build for you. So in your case, just leave out those two lines and ExternalProject_Add() should do exactly what you want.
The main reason you might specify these two options as empty strings is to prevent those steps from doing anything at all. This can be useful, for example, to use ExternalProject_Add() simply for its download and unpacking functionality. This exact situation is used in a technique described here for downloading the source of GoogleTest so it can be added to your project via add_subdirectory(), making it part of your build (see also this answer and other answers to that question for some related material). I suspect this might be where your code is derived from, as the structure looks similar.
For completeness, if you find yourself in a situation where you do need to specify a CMake command, don't use a bare cmake to refer to the command to run. Instead, always use ${CMAKE_COMMAND}, which is provided by CMake as the location of the CMake executable currently being used to process the file. Using this variable means cmake doesn't have to be on the user's PATH and also ensures that if the developer chooses to run a different version of CMake other than the one on the PATH, that same cmake will still be used for the command you are adding.
You can use PATCH_COMMAND like this:
option(WITH_MBEDTLS "Build with mbedtls" OFF)
if(WITH_MBEDTLS)
ExternalProject_Add(external-mbedtls
URL https://github.com/ARMmbed/mbedtls/archive/mbedtls-2.16.1.tar.gz
UPDATE_COMMAND ""
PATCH_COMMAND ./scripts/config.pl set MBEDTLS_THREADING_C &&
./scripts/config.pl set MBEDTLS_THREADING_PTHREAD
CMAKE_ARGS
-DCMAKE_INSTALL_PREFIX:PATH=${PROJECT_BINARY_DIR}/third_party/mbedtls
-DCMAKE_TOOLCHAIN_FILE:PATH=${TOOLCHAIN_FILE}
-DCMAKE_BUILD_TYPE:STRING=Debug
-DENABLE_TESTING:BOOL=OFF
-DENABLE_PROGRAMS:BOOL=ON
TEST_COMMAND ""
)
set(MBEDTLS_PREFIX ${PROJECT_BINARY_DIR}/third_party/mbedtls PARENT_SCOPE)
endif(WITH_MBEDTLS)
I would like to build a third-party project that already has CMake as part of my project's CMake strips. ExternalProject_Add is for this purpose, but I have found it can only be made to work with a specific generator, and I wanted it to work on many platforms easily.
For example, here is my external project with an added script for zlib, which has its own CMakeLists.txt:
set(USE_PROJECT_CMAKE_MODULE_PATH "-DCMAKE_MODULE_PATH=${MAKE_MODULE_PATH}")
ExternalProject_Add(ZLIB
SOURCE_DIR ${CMAKE_SOURCE_DIR}/external/zlib
DOWNLOAD_COMMAND ""
UPDATE_COMMAND ""
CMAKE_ARGS
-DCMAKE_INSTALL_PREFIX=<INSTALL_DIR>
-DCMAKE_CXX_FLAGS=${CMAKE_CXX_FLAGS}
-DCMAKE_C_FLAGS=${CMAKE_C_FLAGS}
-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}
${USE_PROJECT_CMAKE_MODULE_PATH}
INSTALL_COMMAND "")
ExternalProject_Add_Step(ZLIB installInternally
COMMAND cd <BINARY_DIR> && make install
DEPENDEES install
ALWAYS 1)
ExternalProject_Get_Property(ZLIB install_dir)
if(UNIX)
set(ZLIB_NAME libz)
else(UNIX)
set(ZLIB_NAME zlib)
endif(UNIX)
add_library(zlib UNKNOWN IMPORTED)
set_property(TARGET zlib PROPERTY IMPORTED_LOCATION ${install_dir}/lib/${ZLIB_NAME}.a)
set(ZLIB_LIBRARIES zlib)
set(ZLIB_LIBRARIES_OPTIONAL ${ZLIB_LIBRARIES})
set(ZLIB_DIR ${install_dir} CACHE INTERNAL "zlib ROOT dir")
set(ZLIB_INCLUDE_DIRS ${install_dir}/include CACHE INTERNAL "zlib include dirs")
set(ZLIB_DEFINES "-msse2 -mfpmath=sse" CACHE INTERNAL "zlib defines")
The problem with this is that it works with make, but not with Xcode or Visual Studio. Perhaps there is some way to take the CMake build commands passed to my project and forward them to ExternalProject_Add.
How can I write ExternalProject_Add calls in a cross-platform way with minimal code complexity, or is there a better alternative?
Problems
-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}
This is enough for single-configuration projects. But for Xcode and Visual Studio, you need to set CMAKE_CONFIGURATION_TYPES plus call build . --config at the build stage. See my answer.
COMMAND cd <BINARY_DIR> && make install
This will work only for Makefile generators of course. To be cross-platform you can use:
--build . --target install --config inside INSTALL_COMMAND of ExternalProject_Add.
Take a look at this template file, and in particular the following lines:
ExternalProject_Add(
"${current_project}"
URL
#HUNTER_PACKAGE_URL#
URL_HASH
SHA1=#HUNTER_PACKAGE_SHA1#
DOWNLOAD_DIR
"#HUNTER_PACKAGE_DOWNLOAD_DIR#"
SOURCE_DIR
"#HUNTER_PACKAGE_SOURCE_DIR#"
INSTALL_DIR
"#HUNTER_PACKAGE_INSTALL_PREFIX#"
# Not used, just avoid creating Install/<name> empty directory
BUILD_COMMAND ""
# This command is empty because all necessary targets will
# be built on install stage
CMAKE_ARGS
"-G#CMAKE_GENERATOR#"
"-C#HUNTER_CACHE_FILE#"
"-C#HUNTER_ARGS_FILE#"
"-D${postfix_name}=${${postfix_name}}"
"-DCMAKE_BUILD_TYPE=${configuration}"
"-DCMAKE_CONFIGURATION_TYPES=${configuration}"
"-DCMAKE_INSTALL_PREFIX=#HUNTER_PACKAGE_INSTALL_PREFIX#"
"-DCMAKE_TOOLCHAIN_FILE=${CMAKE_TOOLCHAIN_FILE}"
INSTALL_COMMAND
"#CMAKE_COMMAND#"
--build .
--target install
--config ${configuration}
--
${jobs_option}
)
Alternative
or is there a better alternative?
Have you seen Hunter?
You can add zlib just like this:
hunter_add_package(ZLIB)
find_package(ZLIB CONFIG REQUIRED)
target_link_libraries(... ZLIB::zlib)
This code works everywhere. Third party dependencies will be downloaded automatically in the configuration step. Example of building with different generator/toolchains (build.py is just a CMake wrapper that sets CMAKE_TOOLCHAIN_FILE and -G/-B):
build.py --toolchain mingw --config Release # MinGW Makefiles
build.py --toolchain vs-12-2013 --config Debug # Visual Studio 12 2013
build.py --toolchain xcode --config Release # Xcode
build.py --toolchain libcxx --config Release # Makefile with -stdlib=libc++ toolchain
build.py --toolchain ios-8-2 --config Release # Xcode with iOS SDK 8.2 toolchain
You got full control what options, build types or number of jobs you want to have while building third-party packages. For instance, this is how you can build four types, Debug, Release, MinSizeRel, and RelWithDebInfo for zlib and link MinSizeRel to the current project:
> build.py --toolchain xcode --verbose --config MinSizeRel --fwd "HUNTER_CONFIGURATION_TYPES=Release;Debug;MinSizeRel;RelWithDebInfo"
/.../clang /.../lib/libz-MinSizeRel.a ... -o /.../_builds/xcode/MinSizeRel/foo
> ls -la /.../.hunter/_Base/d1232c0/326318e/37e4682/Install/lib/libz*
99056 /.../.hunter/_Base/d1232c0/326318e/37e4682/Install/lib/libz-MinSizeRel.a
307872 /.../.hunter/_Base/d1232c0/326318e/37e4682/Install/lib/libz-RelWithDebInfo.a
109536 /.../.hunter/_Base/d1232c0/326318e/37e4682/Install/lib/libz.a
258904 /.../.hunter/_Base/d1232c0/326318e/37e4682/Install/lib/libzd.a
CMake ExternalProject_Add calls work cross-platform by default and will only fail to do so if one uses particular commands that are only available on a subset of operating systems.
Typically, CMAKE_ARGS is used to pass information to each superbuild unit within an external project build. The CMakeLists.txt files that control each miniature part of the overall build use CMake's declarative syntax (e.g., "add_library(library_name SHARED filename1.hpp filename1.cpp). CMake will convert such syntax to the commands that are specific to the particular build system you wish to use (e.g., make and Ninja).
The sample above re: zlib fails to be cross-platform in part because the ExternalProject_Add_Step contains "COMMAND cd && make install", which necessarily only works in situations where invoking "cd" is actually the correct way to change directories, and where invoking "make" is actually the correct way to build software.
CMake's -E option provides a way to invoke basic operations like changing/copying/making/removing directories without making such assumptions.
(By the way, if you're using IDEs such as Visual Studio or Xcode, you'll likely want to invoke one or more IDE generators when using CMake. For instance, setting
-G "Eclipse CDT4 - Unix Makefiles" -DCMAKE_ECLIPSE_GENERATE_SOURCE_PROJECT=TRUE
will cause Eclipse projects to be generated in each build area, and also in the source code area that is shared for all builds. Of course, if you are using Xcode or Visual Studio, you'll have to substitute the appropriate flag for those IDEs. Alternatively, you could consider using Eclipse with Ninja on all platforms, though at the time of writing, I am not completely certain that Ninja is ready for prime-time on non-Linux, non-Windows operating systems.)
In order to ensure that my Linux builds are identical regardless of the distribution the build host uses, I have packaged up my compiler and the sysroot files into a relocatable tar file and checked that into source control.
So the first step in any build (or at least, a step that must be invoked before any compile step) must be to extract this tar file.
If I was using a makefile, this would be simple to do. However, the project is using cmake and I can't figure out any way to do it with cmake. It might even be that I need this extract step invoked before cmake starts to detect the compiler: I can hard-code the compiler name but if cmake fails if it can't find the compiler then I need the unpack to happen before that test.
Is this possible with cmake?
You can use execute_process to invoke cmake's cross-platform command mode (cmake -E tar). The command would be something like:
execute_process(COMMAND ${CMAKE_COMMAND} -E tar xvf MyCompiler.bz2)
The command which causes CMake to check for a valid compiler is project, so as long as you have your execute_process call before the project call, the unpacking will be done before the compiler check.