I am trying to find a way to enable incremental compilation with CMake through a toolchain upgrade. Here is the problematic scenario :
Branch main uses g++-9 (using CMAKE_CXX_COMPILER=g++-9)
A new branch uses g++-10 (using CMAKE_CXX_COMPILER=g++-10)
Commits are happening on both branches
Incremental builds on one branch work fine
Switching to the other branch and explicitly invoking CMake fails
My question is the following : I'm looking for the proper way to make the invocation of CMake succeed and rebuild all the project from scratch when a toolchain change happens.
Here is a script that will make it quick and easy to reproduce the problem. This script requires Docker. It will create folders Sources and Build at the location where it is executed to avoid littering your filesystem. It then creates Dockerfiles to build docker containers with both g++ and cmake. It then creates a dummy Hello World C++ CMake project. Finally, it creates a folder for build artifacts and then executes the build with g++-9 and then g++-10. The second build fails because CMake generates an error.
#!/bin/bash
set -e
mkdir -p Sources
mkdir -p Build
# Creates a script that will be executed inside the docker container to perform builds
cat << EOF > Sources/Compile.sh
cd /Build \
&& cmake /Sources \
&& make \
&& ./IncrementalBuild
EOF
# Creates a Dockerfile that will be used to have both gcc-9 and cmake
cat << EOF > Sources/Dockerfile-gcc9
FROM gcc:9
RUN apt-get update && apt-get install -y cmake
RUN ln -s /usr/local/bin/g++ /usr/local/bin/g++-9
ADD Compile.sh /Compile.sh
RUN chmod +x /Compile.sh
ENTRYPOINT /Compile.sh
EOF
# Creates a Dockerfile that will be used to have both gcc-10 and cmake
cat << EOF > Sources/Dockerfile-gcc10
FROM gcc:10
RUN apt-get update && apt-get install -y cmake
RUN ln -s /usr/local/bin/g++ /usr/local/bin/g++-10
ADD Compile.sh /Compile.sh
RUN chmod +x /Compile.sh
ENTRYPOINT /Compile.sh
EOF
# Creates a dummy C++ program that will be compiled
cat << EOF > Sources/main.cpp
#include <iostream>
int main()
{
std::cout << "Hello World!\n";
}
EOF
# Creates CMakeLists.txt that will be used to compile the dummy C++ program
cat << EOF > Sources/CMakeLists.txt
cmake_minimum_required(VERSION 3.9)
project(IncrementalBuild CXX)
add_executable(IncrementalBuild main.cpp)
set_target_properties(IncrementalBuild PROPERTIES CXX_STANDARD 17)
EOF
# Build the docker images with both Dockerfiles created earlier
docker build -t cmake-gcc:9 -f Sources/Dockerfile-gcc9 Sources
docker build -t cmake-gcc:10 -f Sources/Dockerfile-gcc10 Sources
# Run a build with g++-9
echo ""
echo "### Compiling with g++-9 and then running the result..."
docker run --rm --user $(id -u):$(id -g) -v $(pwd)/Sources:/Sources -v $(pwd)/Build:/Build -e CXX=g++-9 cmake-gcc:9
echo ""
# Run a build with g++-10
echo "### Compiling with g++-10 and then running the result..."
docker run --rm --user $(id -u):$(id -g) -v $(pwd)/Sources:/Sources -v $(pwd)/Build:/Build -e CXX=g++-10 cmake-gcc:10
echo ""
# Print success if we reach this point
echo "SUCCESS!"
I'm looking for the proper way to make the invocation of CMake succeed and rebuild all the project from scratch when a toolchain change happens.
The proper way is to use a fresh binary directory. Either remove the binary directory when changing and let it recreate or just use a separate different directory for each toolchain.
Use Build/gcc10 binary directory for gcc10 build and Build/gcc9 for gcc9 builds.
No need to cd Build and mkdir with nowadays cmake - use cmake -S. -BBuild. Also do not use make - prefer cmake --build Build to let you switch generator later.
"If you change the toolchain, you should start with a fresh build. There are too many things that assume the toolchain doesn’t change and while you may be able to find workarounds which appear to work, I recommend you always use a fresh build tree for a different toolchain. This same logic also applies if you update the existing toolchain in-place (e.g. you update to a newer version of GCC on Linux, a newer version of Xcode on macOS, etc.). CMake queries compiler capabilities and caches the results. If you change the toolchain in a way that CMake can’t catch, then you end up with stale cached capabilities being used for the new/updated toolchain. Please don’t do that." - Craig Scott
So essentially I don't think it's possible. You just need to blow away your build. The best thing you can do is alert users if CMake isn't doing it for you.
Perhaps reply on this also:
https://discourse.cmake.org/t/how-to-change-toolchain-without-breaking-developer-workflows/1166
Or start another discourse.
Just as make clean deletes all the files that a makefile has produced, I would like to do the same with CMake. All too often I find myself manually going through directories removing files like cmake_install.cmake and CMakeCache.txt, and the CMakeFiles folders.
Is there a command like cmake clean to remove all these files automatically? Ideally this should follow the recursive structure defined within the current directory's CMakeLists.txt file.
CMake 3.X
CMake 3.X offers a 'clean' target.
cmake --build C:/foo/build/ --target clean
From the CMake docs for 3.0.2:
--clean-first = Build target 'clean' first, then build.
(To clean only, use --target 'clean'.)
CMake 2.X
There is no cmake clean in CMake version 2.X
I usually build the project in a single folder like "build". So if I want to make clean, I can just rm -rf build.
The "build" folder in the same directory as the root "CMakeLists.txt" is usually a good choice. To build your project, you simply give cmake the location of the CMakeLists.txt as an argument. For example: cd <location-of-cmakelists>/build && cmake ... (From #ComicSansMS)
In these days of Git everywhere, you may forget CMake and use git clean -d -f -x, that will remove all files not under source control.
CMake official FAQ states:
Some build trees created with GNU autotools have a "make distclean"
target that cleans the build and also removes Makefiles and other
parts of the generated build system. CMake does not generate a "make
distclean" target because CMakeLists.txt files can run scripts and
arbitrary commands; CMake has no way of tracking exactly which files
are generated as part of running CMake. Providing a distclean target
would give users the false impression that it would work as expected.
(CMake does generate a "make clean" target to remove files generated
by the compiler and linker.)
A "make distclean" target is only
necessary if the user performs an in-source build. CMake supports
in-source builds, but we strongly encourage users to adopt the notion
of an out-of-source build. Using a build tree that is separate from
the source tree will prevent CMake from generating any files in the
source tree. Because CMake does not change the source tree, there is
no need for a distclean target. One can start a fresh build by
deleting the build tree or creating a separate build tree.
I googled it for like half an hour and the only useful thing I came up with was invoking the find utility:
# Find and then delete all files under current directory (.) that:
# 1. contains "cmake" (case-&insensitive) in its path (wholename)
# 2. name is not CMakeLists.txt
find . -iwholename '*cmake*' -not -name CMakeLists.txt -delete
Also, be sure to invoke make clean (or whatever CMake generator you're using) before that.
:)
You can use something like:
add_custom_target(clean-cmake-files
COMMAND ${CMAKE_COMMAND} -P clean-all.cmake
)
// clean-all.cmake
set(cmake_generated ${CMAKE_BINARY_DIR}/CMakeCache.txt
${CMAKE_BINARY_DIR}/cmake_install.cmake
${CMAKE_BINARY_DIR}/Makefile
${CMAKE_BINARY_DIR}/CMakeFiles
)
foreach(file ${cmake_generated})
if (EXISTS ${file})
file(REMOVE_RECURSE ${file})
endif()
endforeach(file)
I usually create a "make clean-all" command adding a call to "make clean" to the previous example:
add_custom_target(clean-all
COMMAND ${CMAKE_BUILD_TOOL} clean
COMMAND ${CMAKE_COMMAND} -P clean-all.cmake
)
Don't try to add the "clean" target as a dependence:
add_custom_target(clean-all
COMMAND ${CMAKE_COMMAND} -P clean-all.cmake
DEPENDS clean
)
Because "clean" isn't a real target in CMake and this doesn't work.
Moreover, you should not use this "clean-cmake-files" as dependence of anything:
add_custom_target(clean-all
COMMAND ${CMAKE_BUILD_TOOL} clean
DEPENDS clean-cmake-files
)
Because, if you do that, all CMake files will be erased before clean-all is complete, and make will throw you an error searching "CMakeFiles/clean-all.dir/build.make". In consequence, you can not use the clean-all command before "anything" in any context:
add_custom_target(clean-all
COMMAND ${CMAKE_BUILD_TOOL} clean
COMMAND ${CMAKE_COMMAND} -P clean-all.cmake
)
That doesn't work either.
Simply issuing rm CMakeCache.txt works for me too.
Maybe it's a little outdated, but since this is the first hit when you google cmake clean, I will add this:
Since you can start a build in the build dir with a specified target with
cmake --build . --target xyz
you can of course run
cmake --build . --target clean
to run the clean target in the generated build files.
Starting with CMake 3.24, there exists the --fresh command line option which rebuilds the entire build tree every time:
--fresh
New in version 3.24.
Perform a fresh configuration of the build tree. This removes any
existing CMakeCache.txt file and associated CMakeFiles/ directory, and
recreates them from scratch.
https://cmake.org/cmake/help/latest/manual/cmake.1.html#options
I agree that the out-of-source build is the best answer. But for the times when you just must do an in-source build, I have written a Python script available here, which:
Runs "make clean"
Removes specific CMake-generated files in the top-level directory such as CMakeCache.txt
For each subdirectory that contains a CMakeFiles directory, it removes CMakeFiles, Makefile, cmake_install.cmake.
Removes all empty subdirectories.
It's funny to see this question gets so many attentions and complicated solutions, which indeed shows a pain to not have a clean method with cmake.
Well, you can definitely cd build_work to do you work, then do a rm -rf * when you need to clean. However, rm -rf * is a dangerous command given that many people are often not aware which dir they are in.
If you cd .., rm -rf build_work and then mkdir build_work and then cd build_work, that's just too much typing.
So a good solution is to just stay out of the build folder and tell cmake the path:
to configure: cmake -B build_work
to build: cmake --build build_work
to install: cmake --install build_work
to clean: rm -rf build_work
to recreate build folder: you don't even need mkdir build_work, just configure it with cmake -B build_work.
In the case where you pass -D parameters into CMake when generating the build files and don't want to delete the entire build/ directory:
Simply delete the CMakeFiles/ directory inside your build directory.
rm -rf CMakeFiles/
cmake --build .
This causes CMake to rerun, and build system files are regenerated. Your build will also start from scratch.
try to use:
cmake --clean-first path-of-CMakeLists.txt-file -B output-dir
--clean-first: Build target clean first, then build.
(To clean only, use --target clean.)
Of course, out-of-source builds are the go-to method for Unix Makefiles, but if you're using another generator such as Eclipse CDT, it prefers you to build in-source. In which case, you'll need to purge the CMake files manually. Try this:
find . -name 'CMakeCache.txt' -o -name '*.cmake' -o -name 'Makefile' -o -name 'CMakeFiles' -exec rm -rf {} +
Or if you've enabled globstar with shopt -s globstar, try this less disgusting approach instead:
rm -rf **/CMakeCache.txt **/*.cmake **/Makefile **/CMakeFiles
A solution that I found recently is to combine the out-of-source build concept with a Makefile wrapper.
In my top-level CMakeLists.txt file, I include the following to prevent in-source builds:
if ( ${CMAKE_SOURCE_DIR} STREQUAL ${CMAKE_BINARY_DIR} )
message( FATAL_ERROR "In-source builds not allowed. Please make a new directory (called a build directory) and run CMake from there. You may need to remove CMakeCache.txt." )
endif()
Then, I create a top-level Makefile, and include the following:
# -----------------------------------------------------------------------------
# CMake project wrapper Makefile ----------------------------------------------
# -----------------------------------------------------------------------------
SHELL := /bin/bash
RM := rm -rf
MKDIR := mkdir -p
all: ./build/Makefile
# $(MAKE) -C build
./build/Makefile:
# ($(MKDIR) build > /dev/null)
# (cd build > /dev/null 2>&1 && cmake ..)
distclean:
# ($(MKDIR) build > /dev/null)
# (cd build > /dev/null 2>&1 && cmake .. > /dev/null 2>&1)
#- $(MAKE) --silent -C build clean || true
#- $(RM) ./build/Makefile
#- $(RM) ./build/src
#- $(RM) ./build/test
#- $(RM) ./build/CMake*
#- $(RM) ./build/cmake.*
#- $(RM) ./build/*.cmake
#- $(RM) ./build/*.txt
ifeq ($(findstring distclean,$(MAKECMDGOALS)),)
$(MAKECMDGOALS): ./build/Makefile
# $(MAKE) -C build $(MAKECMDGOALS)
endif
The default target all is called by typing make, and invokes the target ./build/Makefile.
The first thing the target ./build/Makefile does is to create the build directory using $(MKDIR), which is a variable for mkdir -p. The directory build is where we will perform our out-of-source build. We provide the argument -p to ensure that mkdir does not scream at us for trying to create a directory that may already exist.
The second thing the target ./build/Makefile does is to change directories to the build directory and invoke cmake.
Back to the all target, we invoke $(MAKE) -C build, where $(MAKE) is a Makefile variable automatically generated for make. make -C changes the directory before doing anything. Therefore, using $(MAKE) -C build is equivalent to doing cd build; make.
To summarize, calling this Makefile wrapper with make all or make is equivalent to doing:
mkdir build
cd build
cmake ..
make
The target distclean invokes cmake .., then make -C build clean, and finally, removes all contents from the build directory. I believe this is exactly what you requested in your question.
The last piece of the Makefile evaluates if the user-provided target is or is not distclean. If not, it will change directories to build before invoking it. This is very powerful because the user can type, for example, make clean, and the Makefile will transform that into an equivalent of cd build; make clean.
In conclusion, this Makefile wrapper, in combination with a mandatory out-of-source build CMake configuration, make it so that the user never has to interact with the command cmake. This solution also provides an elegant method to remove all CMake output files from the build directory.
P.S. In the Makefile, we use the prefix # to suppress the output from a shell command, and the prefix #- to ignore errors from a shell command. When using rm as part of the distclean target, the command will return an error if the files do not exist (they may have been deleted already using the command line with rm -rf build, or they were never generated in the first place). This return error will force our Makefile to exit. We use the prefix #- to prevent that. It is acceptable if a file was removed already; we want our Makefile to keep going and remove the rest.
Another thing to note: This Makefile may not work if you use a variable number of CMake variables to build your project, for example, cmake .. -DSOMEBUILDSUSETHIS:STRING="foo" -DSOMEOTHERBUILDSUSETHISTOO:STRING="bar". This Makefile assumes you invoke CMake in a consistent way, either by typing cmake .. or by providing cmake a consistent number of arguments (that you can include in your Makefile).
Finally, credit where credit is due. This Makefile wrapper was adapted from the Makefile provided by the C++ Application Project Template.
I use the following shell script for such purposes:
#!/bin/bash
for fld in $(find -name "CMakeLists.txt" -printf '%h ')
do
for cmakefile in CMakeCache.txt cmake_install.cmake CTestTestfile.cmake CMakeFiles Makefile
do
rm -rfv $fld/$cmakefile
done
done
If you are using Windows then use Cygwin for this script.
Clear the cmake build output:
Command line:
$ rm -rf [folder that you builded the project]/
$ cmake --build .
Cmake:
cmake --build . --target clean
To simplify cleaning when using "out of source" build (i.e. you build in the build directory), I use the following script:
$ cat ~/bin/cmake-clean-build
#!/bin/bash
if [ -d ../build ]; then
cd ..
rm -rf build
mkdir build
cd build
else
echo "build directory DOES NOT exist"
fi
Every time you need to clean up, you should source this script from the build directory:
. cmake-clean-build
CMake 3.X
CMake 3.0 and above offers a 'clean' target. This removes any artifacts like object files, library files, executables, generated files, etc.
cmake --build C:/foo/build/ --target clean
You can also clean the build, then run the build. In 1 command.
cmake --build C:/foo/build --clean-first
However, this WON'T cleanup things like the CMakeCache.txt or the associated CMakeFiles/ directory. Which you may want to do. You just have to delete the build folder.
# Just delete the build folder
rm C:/foo/build -rf
# You can also just let git delete the build folder as well
git clean -d -f -x
CMake 3.24
Now in CMake 3.24 you can perform a fresh configuration of the build tree. This removes any existing CMakeCache.txt file and associated CMakeFiles/ directory, and recreates them from scratch.
Generally you want to do this when:
You want to clear cached variable in CMakeCache.txt
You want to change compilers
Any other operations related to CMake caching
cmake -B C:/foo/build --fresh
If you run
cmake .
it will regenerate the CMake files. Which is necessary if you add a new file to a source folder that is selected by *.cc, for example.
While this isn't a "clean" per se, it does "clean" up the CMake files by regenerating the caches.
This is pretty old, but if you completely remove the cmake-build-debug folder, when you compile using cmake it should automatically create a new cmake-build-debug folder with everything you need. Works especially well in CLion.
If you have custom defines and want to save them before cleaning, run the following in your build directory:
sed -ne '/variable specified on the command line/{n;s/.*/-D \0 \\/;p}' CMakeCache.txt
Then create a new build directory (or remove the old build directory and recreate it) and finally run cmake with the arguments you'll get with the script above.
cmake mostly cooks a Makefile, one could add rm to the clean PHONY.
For example,
[root#localhost hello]# ls
CMakeCache.txt CMakeFiles cmake_install.cmake CMakeLists.txt hello Makefile test
[root#localhost hello]# vi Makefile
clean:
$(MAKE) -f CMakeFiles/Makefile2 clean
rm -rf *.o *~ .depend .*.cmd *.mod *.ko *.mod.c .tmp_versions *.symvers *.d *.markers *.order CMakeFiles cmake_install.cmake CMakeCache.txt Makefile
Here is what I use. It's wrapped in a function, It's cross platform and it demonstrated how to find matching filenames or folder names, in case you wanted to make any easy adjustments. This function runs every time I build my scripts and has worked flawlessly for my needs.
function(DELETE_CACHE)
if(CMAKE_HOST_WIN32)
execute_process(COMMAND cmd /c for /r %%i in (CMakeCache.*) do del "%%i" WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR})
execute_process(COMMAND cmd /c for /d /r %%i in (*CMakeFiles*) do rd /s /q "%%i" WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR})
else()
execute_process(COMMAND find . -name "CMakeCache.*" -delete WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR})
execute_process(COMMAND "rm -rf `find . -type d -name CMakeFiles`" WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR})
endif()
endfunction()
I used zsxwing's answer successfully to solve the following problem:
I have source that I build on multiple hosts (on a Raspberry Pi Linux board, on a VMware Linux virtual machine, etc.)
I have a Bash script that creates temporary directories based on the hostname of the machine like this:
# Get hostname to use as part of directory names
HOST_NAME=`uname -n`
# Create a temporary directory for cmake files so they don't
# end up all mixed up with the source.
TMP_DIR="cmake.tmp.$HOSTNAME"
if [ ! -e $TMP_DIR ] ; then
echo "Creating directory for cmake tmp files : $TMP_DIR"
mkdir $TMP_DIR
else
echo "Reusing cmake tmp dir : $TMP_DIR"
fi
# Create makefiles with CMake
#
# Note: switch to the temporary dir and build parent
# which is a way of making cmake tmp files stay
# out of the way.
#
# Note 2: to clean up cmake files, it is OK to
# "rm -rf" the temporary directories
echo
echo Creating Makefiles with cmake ...
cd $TMP_DIR
cmake ..
# Run makefile (in temporary directory)
echo
echo Starting build ...
make
Create a temporary build directory, for example, build_cmake. Hence all your build files will be inside this folder.
Then in your main CMake file add the below command.
add_custom_target(clean-all
rm -rf *
)
Hence while compiling do
cmake ..
And to clean do:
make clean-all