Build kernel module into a specific directory - module

is there a way to set a output-directory for making kernel-modules inside my makefile?
I want to keep my source-direcory clean from the build-files.

KBUILD_OUTPUT and O= did not work for me and were failing to find the kernel headers when building externally.
My solution is to symlink the source files into the bin directory, and dynamically generate a new MakeFile in the bin directory. This allows all build files to be cleaned up easily since the dynamic Makefile can always just be recreated.
INCLUDE=include
SOURCE=src
TARGET=mymodule
OUTPUT=bin
EXPORT=package
SOURCES=$(wildcard $(SOURCE)/*.c)
# Depends on bin/include bin/*.c and bin/Makefile
all: $(OUTPUT)/$(INCLUDE) $(subst $(SOURCE),$(OUTPUT),$(SOURCES)) $(OUTPUT)/Makefile
make -C /lib/modules/$(shell uname -r)/build M=$(PWD)/$(OUTPUT) modules
# Create a symlink from src to bin
$(OUTPUT)/%: $(SOURCE)/%
ln -s ../$< $#
# Generate a Makefile with the needed obj-m and mymodule-objs set
$(OUTPUT)/Makefile:
echo "obj-m += $(TARGET).o\n$(TARGET)-objs := $(subst $(TARGET).o,, $(subst .c,.o,$(subst $(SOURCE)/,,$(SOURCES))))" > $#
clean:
rm -rf $(OUTPUT)
mkdir $(OUTPUT)

If you are building inside the kernel tree you can use the O variable:
make O=/path/to/mydir
If you are compiling outside the kernel tree (module, or any other kind of program) you need to change your Makefile to output in a different directory. Here a little example of a Makefile rule which output in the MY_DIR directory:
$(MY_DIR)/test: test.c
gcc -o $# $<
and then write:
$ make MY_DIR=/path/to/build/directory

The same here, but I used a workaround that worked for me:
Create a sub-directory with/for every arch name (e.g. "debug_64").
Under "debug_64": create symbolic link of all .c and .h files. Keeping the same structure.
Copy the makefile to "debug_64" and set the right flags for 64 Debug build, e.g.
ccflags-y := -DCRONO_DEBUG_ENABLED
ccflags-y += -I$(src)/../../../lib/include
KBUILD_AFLAGS += -march=x86_64
Remember to set the relative directories paths to one level down, e.g. ../inc will be ../../inc.
Repeat the same for every arch/profile.
Now we have one source code, different folders, and different make files.
By the way, creating profiles inside make files for kernel module build is not an easy job, so, I preferred to create a copy of makefile for every arch.

Related

CMake + Ninja : how to implement incremental compilation under the path of different source directories

Every time I get a different source directory, and I have a fixed build directory. Every time I will run cmake /path/to/project and run ninja. It will report an error : Make Error: The source "/path1/to/project/CMakeLists.txt" does not match the source "/path2/to/project/CMakeLists.txt" used to generate cache. Re-run cmake with a different source directory.
So what I did was manually change the options related to the path of project in the CMakeCache.txt. The result is that it will compile the project from scratch every time instead of incrementally compiling, So is there any feasible way to achieve incremental compilation or in this case it is impossible to achieve incremental compilation ?
Mount the source directory always to a same constant absolute location. On Linux, you could use mount namespaces, for example use proot.
proot -b /path/to/project:/work -w /work cmake -B builddir -S .
proot -b /path/to/project:/work -w /work cmake --build builddir

Clang: Using Cmake to build a compile_commands.json for my project

I'm trying to create the compile_commands.json file by following these instructions: http://clang.llvm.org/docs/HowToSetupToolingForLLVM.html
I don't see any option on the docs that allows be to build the compile_commands.json file against my own code. The command that I use when I follow the doc builds the json file, but the contents of the file contains files/directories in the llvm source directory. I'm assuming that if this file contains contents of my own project, then when I run run-clang-tidy.py (from here), then it will run clang-tidy against my own project.
So the question: How do I get Cmake to build the compile_commands.json to contain my own project?
It is unclear exactly what you want to achieve. In the link you provided, the author is building LLVM from source code, which is why the compilation database (compile_commands.json) refers to those sources.
In general to build the compilation database while building the source tree with CMake, you provide it with the -DCMAKE_EXPORT_COMPILE_COMMANDS=ON flag.
So, where the blog post you linked calls
cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=ON path/to/llvm/sources
...you would exchange the path to the sources for your own source tree:
cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=ON this/is/my/source/tree/path
...which should generate the json-file in the current (build) directory.
As in the article, you will probably want to symlink that file to the root of your source tree and then run your tools from there.
Here's a (very) simple build script that I wrote and sometimes use. Maybe it can be helpful to you (adjust to your own needs obviously). It should be run from the root of your source tree.
#!/usr/bin/env sh
type=Debug
if ! [[ -z "$1" ]]; then
type="$1"
mkdir -p build/$type
shift
fi
# Configure build
cmake -S . -B build/$type -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DCMAKE_BUILD_TYPE=$type .
# Build
cmake --build build/$type --parallel 4 --config $type "$#"
ln -fs build/$type/compile_commands.json .

CMake: How to link (ln) additional names after install?

I need to find a way to link additional names to an installed executable, after installing it.
The below example is close, except for two problems. One, the linking is done after every target, not just the install. Two, the links are created in the build directory, not in the install directory (I can probably add the paths necessary to do that, but it would then error out if done before the install.)
cmake_minimum_required(VERSION 2.8.4)
add_executable(gr gr.c)
install(TARGETS gr DESTINATION bin)
add_custom_command(
TARGET gr
POST_BUILD
COMMAND ln;-f;gr;grm
COMMAND ln;-f;gr;grs
COMMAND ln;-f;gr;grh
)
What's simple, clean way to do what I want?
In case it's not clear, the Makefile equivalent is:
gr: gr.c
cc -o gr gr.c
install:
install gr ${BINDIR}
ln -f ${BINDIR}/gr ${BINDIR}/grm
ln -f ${BINDIR}/gr ${BINDIR}/grs
ln -f ${BINDIR}/gr ${BINDIR}/grh
What I have done in similar situations is use the custom command similar to what you have done, but add an additional install command to install the links in the final bin directory alongside the target. So after your add_custom_command:
install(
FILES
${CMAKE_CURRENT_BINARY_DIR}/grm
${CMAKE_CURRENT_BINARY_DIR}/grs
${CMAKE_CURRENT_BINARY_DIR}/grh
DESTINATION bin
)
Of course, this will probably only do what you expect if you change your links to symbolic links (ln -s).

make does not rebuild target on header file modification

I have a makefile of this kind:
program: \
a/a.o \
b/b.o
$(CXX) $(CXXFLAGS) -o program \
a/a.o \
b/b.o
a.o: \
a/a.cpp \
a/a.h
$(CXX) $(CXXFLAGS) -c a/a.cpp
b.o: \
b/b.cpp \
b/b.h
$(CXX) $(CXXFLAGS) -c b/b.cpp
So in the directory of the makefile I have two subdirectories a and b
that contain respectively a.h, a.cpp and b.h, b.cpp.
The problem is that if I modify a .cpp file, issuing a make rebuilds the target program
but if I modify an .h file make do not rebuilds anything but says
make: `program' is up to date.
I can't understand why, because the .h files are in the prerequisites line
along with the .cpp files.
Interestingly, if I issue a make on an object file target like
$ make a.o
instead, the modifications to a/a.h
are detected and the target a/a.o is rebuild.
Where is the problem?
The subdirectories that you added to the question later are causing the problem indeed. The target program depends on a/a.o and b/b.o, but there are no explicit rules to make those to .o files -- only the targets a.o and b.o are present but those are not in the subdirectories.
Therefore, make will look for implicit rules to build a/a.o and b/b.o. That rule does exist, you will see it being found when you run make -d. That implicit rule depends on a/file_a.cpp only, not on a/file_a.h. Therefore, changing a/file_a.cpp will make a/a.o out of date according to that implicit rule, whereas a/file_a.h will not.
For your reference, the make User's Manual has a section Catalogue of Implicit Rules. That also explains that you can use the argument --no-builtin-rules to avoid that implicit behavior. If you use that, you will see that make can not find any rules to make a/a.o and b/b.o.
Finally, running make a.o will run the recipe for the target a.o as defined in your makefile. That target does have a/a.h as its prerequisite so any change to that file will result in a recompile. But essentially, that has nothing to do with the target program, which has different prerequisites.

Looking for a 'cmake clean' command to clear up CMake output

Just as make clean deletes all the files that a makefile has produced, I would like to do the same with CMake. All too often I find myself manually going through directories removing files like cmake_install.cmake and CMakeCache.txt, and the CMakeFiles folders.
Is there a command like cmake clean to remove all these files automatically? Ideally this should follow the recursive structure defined within the current directory's CMakeLists.txt file.
CMake 3.X
CMake 3.X offers a 'clean' target.
cmake --build C:/foo/build/ --target clean
From the CMake docs for 3.0.2:
--clean-first = Build target 'clean' first, then build.
(To clean only, use --target 'clean'.)
CMake 2.X
There is no cmake clean in CMake version 2.X
I usually build the project in a single folder like "build". So if I want to make clean, I can just rm -rf build.
The "build" folder in the same directory as the root "CMakeLists.txt" is usually a good choice. To build your project, you simply give cmake the location of the CMakeLists.txt as an argument. For example: cd <location-of-cmakelists>/build && cmake ... (From #ComicSansMS)
In these days of Git everywhere, you may forget CMake and use git clean -d -f -x, that will remove all files not under source control.
CMake official FAQ states:
Some build trees created with GNU autotools have a "make distclean"
target that cleans the build and also removes Makefiles and other
parts of the generated build system. CMake does not generate a "make
distclean" target because CMakeLists.txt files can run scripts and
arbitrary commands; CMake has no way of tracking exactly which files
are generated as part of running CMake. Providing a distclean target
would give users the false impression that it would work as expected.
(CMake does generate a "make clean" target to remove files generated
by the compiler and linker.)
A "make distclean" target is only
necessary if the user performs an in-source build. CMake supports
in-source builds, but we strongly encourage users to adopt the notion
of an out-of-source build. Using a build tree that is separate from
the source tree will prevent CMake from generating any files in the
source tree. Because CMake does not change the source tree, there is
no need for a distclean target. One can start a fresh build by
deleting the build tree or creating a separate build tree.
I googled it for like half an hour and the only useful thing I came up with was invoking the find utility:
# Find and then delete all files under current directory (.) that:
# 1. contains "cmake" (case-&insensitive) in its path (wholename)
# 2. name is not CMakeLists.txt
find . -iwholename '*cmake*' -not -name CMakeLists.txt -delete
Also, be sure to invoke make clean (or whatever CMake generator you're using) before that.
:)
You can use something like:
add_custom_target(clean-cmake-files
COMMAND ${CMAKE_COMMAND} -P clean-all.cmake
)
// clean-all.cmake
set(cmake_generated ${CMAKE_BINARY_DIR}/CMakeCache.txt
${CMAKE_BINARY_DIR}/cmake_install.cmake
${CMAKE_BINARY_DIR}/Makefile
${CMAKE_BINARY_DIR}/CMakeFiles
)
foreach(file ${cmake_generated})
if (EXISTS ${file})
file(REMOVE_RECURSE ${file})
endif()
endforeach(file)
I usually create a "make clean-all" command adding a call to "make clean" to the previous example:
add_custom_target(clean-all
COMMAND ${CMAKE_BUILD_TOOL} clean
COMMAND ${CMAKE_COMMAND} -P clean-all.cmake
)
Don't try to add the "clean" target as a dependence:
add_custom_target(clean-all
COMMAND ${CMAKE_COMMAND} -P clean-all.cmake
DEPENDS clean
)
Because "clean" isn't a real target in CMake and this doesn't work.
Moreover, you should not use this "clean-cmake-files" as dependence of anything:
add_custom_target(clean-all
COMMAND ${CMAKE_BUILD_TOOL} clean
DEPENDS clean-cmake-files
)
Because, if you do that, all CMake files will be erased before clean-all is complete, and make will throw you an error searching "CMakeFiles/clean-all.dir/build.make". In consequence, you can not use the clean-all command before "anything" in any context:
add_custom_target(clean-all
COMMAND ${CMAKE_BUILD_TOOL} clean
COMMAND ${CMAKE_COMMAND} -P clean-all.cmake
)
That doesn't work either.
Simply issuing rm CMakeCache.txt works for me too.
Maybe it's a little outdated, but since this is the first hit when you google cmake clean, I will add this:
Since you can start a build in the build dir with a specified target with
cmake --build . --target xyz
you can of course run
cmake --build . --target clean
to run the clean target in the generated build files.
Starting with CMake 3.24, there exists the --fresh command line option which rebuilds the entire build tree every time:
--fresh
New in version 3.24.
Perform a fresh configuration of the build tree. This removes any
existing CMakeCache.txt file and associated CMakeFiles/ directory, and
recreates them from scratch.
https://cmake.org/cmake/help/latest/manual/cmake.1.html#options
I agree that the out-of-source build is the best answer. But for the times when you just must do an in-source build, I have written a Python script available here, which:
Runs "make clean"
Removes specific CMake-generated files in the top-level directory such as CMakeCache.txt
For each subdirectory that contains a CMakeFiles directory, it removes CMakeFiles, Makefile, cmake_install.cmake.
Removes all empty subdirectories.
It's funny to see this question gets so many attentions and complicated solutions, which indeed shows a pain to not have a clean method with cmake.
Well, you can definitely cd build_work to do you work, then do a rm -rf * when you need to clean. However, rm -rf * is a dangerous command given that many people are often not aware which dir they are in.
If you cd .., rm -rf build_work and then mkdir build_work and then cd build_work, that's just too much typing.
So a good solution is to just stay out of the build folder and tell cmake the path:
to configure: cmake -B build_work
to build: cmake --build build_work
to install: cmake --install build_work
to clean: rm -rf build_work
to recreate build folder: you don't even need mkdir build_work, just configure it with cmake -B build_work.
In the case where you pass -D parameters into CMake when generating the build files and don't want to delete the entire build/ directory:
Simply delete the CMakeFiles/ directory inside your build directory.
rm -rf CMakeFiles/
cmake --build .
This causes CMake to rerun, and build system files are regenerated. Your build will also start from scratch.
try to use:
cmake --clean-first path-of-CMakeLists.txt-file -B output-dir
--clean-first: Build target clean first, then build.
(To clean only, use --target clean.)
Of course, out-of-source builds are the go-to method for Unix Makefiles, but if you're using another generator such as Eclipse CDT, it prefers you to build in-source. In which case, you'll need to purge the CMake files manually. Try this:
find . -name 'CMakeCache.txt' -o -name '*.cmake' -o -name 'Makefile' -o -name 'CMakeFiles' -exec rm -rf {} +
Or if you've enabled globstar with shopt -s globstar, try this less disgusting approach instead:
rm -rf **/CMakeCache.txt **/*.cmake **/Makefile **/CMakeFiles
A solution that I found recently is to combine the out-of-source build concept with a Makefile wrapper.
In my top-level CMakeLists.txt file, I include the following to prevent in-source builds:
if ( ${CMAKE_SOURCE_DIR} STREQUAL ${CMAKE_BINARY_DIR} )
message( FATAL_ERROR "In-source builds not allowed. Please make a new directory (called a build directory) and run CMake from there. You may need to remove CMakeCache.txt." )
endif()
Then, I create a top-level Makefile, and include the following:
# -----------------------------------------------------------------------------
# CMake project wrapper Makefile ----------------------------------------------
# -----------------------------------------------------------------------------
SHELL := /bin/bash
RM := rm -rf
MKDIR := mkdir -p
all: ./build/Makefile
# $(MAKE) -C build
./build/Makefile:
# ($(MKDIR) build > /dev/null)
# (cd build > /dev/null 2>&1 && cmake ..)
distclean:
# ($(MKDIR) build > /dev/null)
# (cd build > /dev/null 2>&1 && cmake .. > /dev/null 2>&1)
#- $(MAKE) --silent -C build clean || true
#- $(RM) ./build/Makefile
#- $(RM) ./build/src
#- $(RM) ./build/test
#- $(RM) ./build/CMake*
#- $(RM) ./build/cmake.*
#- $(RM) ./build/*.cmake
#- $(RM) ./build/*.txt
ifeq ($(findstring distclean,$(MAKECMDGOALS)),)
$(MAKECMDGOALS): ./build/Makefile
# $(MAKE) -C build $(MAKECMDGOALS)
endif
The default target all is called by typing make, and invokes the target ./build/Makefile.
The first thing the target ./build/Makefile does is to create the build directory using $(MKDIR), which is a variable for mkdir -p. The directory build is where we will perform our out-of-source build. We provide the argument -p to ensure that mkdir does not scream at us for trying to create a directory that may already exist.
The second thing the target ./build/Makefile does is to change directories to the build directory and invoke cmake.
Back to the all target, we invoke $(MAKE) -C build, where $(MAKE) is a Makefile variable automatically generated for make. make -C changes the directory before doing anything. Therefore, using $(MAKE) -C build is equivalent to doing cd build; make.
To summarize, calling this Makefile wrapper with make all or make is equivalent to doing:
mkdir build
cd build
cmake ..
make
The target distclean invokes cmake .., then make -C build clean, and finally, removes all contents from the build directory. I believe this is exactly what you requested in your question.
The last piece of the Makefile evaluates if the user-provided target is or is not distclean. If not, it will change directories to build before invoking it. This is very powerful because the user can type, for example, make clean, and the Makefile will transform that into an equivalent of cd build; make clean.
In conclusion, this Makefile wrapper, in combination with a mandatory out-of-source build CMake configuration, make it so that the user never has to interact with the command cmake. This solution also provides an elegant method to remove all CMake output files from the build directory.
P.S. In the Makefile, we use the prefix # to suppress the output from a shell command, and the prefix #- to ignore errors from a shell command. When using rm as part of the distclean target, the command will return an error if the files do not exist (they may have been deleted already using the command line with rm -rf build, or they were never generated in the first place). This return error will force our Makefile to exit. We use the prefix #- to prevent that. It is acceptable if a file was removed already; we want our Makefile to keep going and remove the rest.
Another thing to note: This Makefile may not work if you use a variable number of CMake variables to build your project, for example, cmake .. -DSOMEBUILDSUSETHIS:STRING="foo" -DSOMEOTHERBUILDSUSETHISTOO:STRING="bar". This Makefile assumes you invoke CMake in a consistent way, either by typing cmake .. or by providing cmake a consistent number of arguments (that you can include in your Makefile).
Finally, credit where credit is due. This Makefile wrapper was adapted from the Makefile provided by the C++ Application Project Template.
I use the following shell script for such purposes:
#!/bin/bash
for fld in $(find -name "CMakeLists.txt" -printf '%h ')
do
for cmakefile in CMakeCache.txt cmake_install.cmake CTestTestfile.cmake CMakeFiles Makefile
do
rm -rfv $fld/$cmakefile
done
done
If you are using Windows then use Cygwin for this script.
Clear the cmake build output:
Command line:
$ rm -rf [folder that you builded the project]/
$ cmake --build .
Cmake:
cmake --build . --target clean
To simplify cleaning when using "out of source" build (i.e. you build in the build directory), I use the following script:
$ cat ~/bin/cmake-clean-build
#!/bin/bash
if [ -d ../build ]; then
cd ..
rm -rf build
mkdir build
cd build
else
echo "build directory DOES NOT exist"
fi
Every time you need to clean up, you should source this script from the build directory:
. cmake-clean-build
CMake 3.X
CMake 3.0 and above offers a 'clean' target. This removes any artifacts like object files, library files, executables, generated files, etc.
cmake --build C:/foo/build/ --target clean
You can also clean the build, then run the build. In 1 command.
cmake --build C:/foo/build --clean-first
However, this WON'T cleanup things like the CMakeCache.txt or the associated CMakeFiles/ directory. Which you may want to do. You just have to delete the build folder.
# Just delete the build folder
rm C:/foo/build -rf
# You can also just let git delete the build folder as well
git clean -d -f -x
CMake 3.24
Now in CMake 3.24 you can perform a fresh configuration of the build tree. This removes any existing CMakeCache.txt file and associated CMakeFiles/ directory, and recreates them from scratch.
Generally you want to do this when:
You want to clear cached variable in CMakeCache.txt
You want to change compilers
Any other operations related to CMake caching
cmake -B C:/foo/build --fresh
If you run
cmake .
it will regenerate the CMake files. Which is necessary if you add a new file to a source folder that is selected by *.cc, for example.
While this isn't a "clean" per se, it does "clean" up the CMake files by regenerating the caches.
This is pretty old, but if you completely remove the cmake-build-debug folder, when you compile using cmake it should automatically create a new cmake-build-debug folder with everything you need. Works especially well in CLion.
If you have custom defines and want to save them before cleaning, run the following in your build directory:
sed -ne '/variable specified on the command line/{n;s/.*/-D \0 \\/;p}' CMakeCache.txt
Then create a new build directory (or remove the old build directory and recreate it) and finally run cmake with the arguments you'll get with the script above.
cmake mostly cooks a Makefile, one could add rm to the clean PHONY.
For example,
[root#localhost hello]# ls
CMakeCache.txt CMakeFiles cmake_install.cmake CMakeLists.txt hello Makefile test
[root#localhost hello]# vi Makefile
clean:
$(MAKE) -f CMakeFiles/Makefile2 clean
rm -rf *.o *~ .depend .*.cmd *.mod *.ko *.mod.c .tmp_versions *.symvers *.d *.markers *.order CMakeFiles cmake_install.cmake CMakeCache.txt Makefile
Here is what I use. It's wrapped in a function, It's cross platform and it demonstrated how to find matching filenames or folder names, in case you wanted to make any easy adjustments. This function runs every time I build my scripts and has worked flawlessly for my needs.
function(DELETE_CACHE)
if(CMAKE_HOST_WIN32)
execute_process(COMMAND cmd /c for /r %%i in (CMakeCache.*) do del "%%i" WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR})
execute_process(COMMAND cmd /c for /d /r %%i in (*CMakeFiles*) do rd /s /q "%%i" WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR})
else()
execute_process(COMMAND find . -name "CMakeCache.*" -delete WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR})
execute_process(COMMAND "rm -rf `find . -type d -name CMakeFiles`" WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR})
endif()
endfunction()
I used zsxwing's answer successfully to solve the following problem:
I have source that I build on multiple hosts (on a Raspberry Pi Linux board, on a VMware Linux virtual machine, etc.)
I have a Bash script that creates temporary directories based on the hostname of the machine like this:
# Get hostname to use as part of directory names
HOST_NAME=`uname -n`
# Create a temporary directory for cmake files so they don't
# end up all mixed up with the source.
TMP_DIR="cmake.tmp.$HOSTNAME"
if [ ! -e $TMP_DIR ] ; then
echo "Creating directory for cmake tmp files : $TMP_DIR"
mkdir $TMP_DIR
else
echo "Reusing cmake tmp dir : $TMP_DIR"
fi
# Create makefiles with CMake
#
# Note: switch to the temporary dir and build parent
# which is a way of making cmake tmp files stay
# out of the way.
#
# Note 2: to clean up cmake files, it is OK to
# "rm -rf" the temporary directories
echo
echo Creating Makefiles with cmake ...
cd $TMP_DIR
cmake ..
# Run makefile (in temporary directory)
echo
echo Starting build ...
make
Create a temporary build directory, for example, build_cmake. Hence all your build files will be inside this folder.
Then in your main CMake file add the below command.
add_custom_target(clean-all
rm -rf *
)
Hence while compiling do
cmake ..
And to clean do:
make clean-all