Does the main kernel "make" command also make modules internally? - module

I am learning how to write kernel drivers and this is my first attempt to build one. I have created a folder drivers/naveen/ for my module files - hello.c,Kconfig and Makefile. These are the contents of these files :
Kconfig
config HELLO_WORLD
tristate "Hello World support"
default m
---help---
This option enables printing hello world
Makefile
obj-$(CONFIG_HELLO_WORLD) += hello.o
hello.c
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/init.h>
static int __init hello_init(void)
{
printk(KERN_ERR "This is NAVEEN module");
return 0;
}
static int __exit hello_exit(void)
{
printk(KERN_ERR "NAVEEN exiting module");
return 0;
}
module_init(hello_init);
module_exit(hello_exit);
MODULE_AUTHOR("Naveen");
MODULE_LICENSE("GPL");
Also, I have added the following line in drivers/Makefile :
obj-$(CONFIG_HELLO_WORLD) += naveen/
and the following line in drivers/Kconfig :
source "drivers/naveen/Kconfig"
My generated .config contains CONFIG_HELLO_WORLD=m.
I did make ARCH=x86_64 -j16 and I can see hello.ko generated. Why? I was expecting to get it generated only when I had done make modules as its set to be as modular with m inside the .config, and not to be compiled with just make. Can someone please explain the behaviour to me or what wrong I am doing?
Does that mean that make also does make modules. I can see from make help that make actually means make all and hence it should do make modules as well internally, and so there should be no need to do make modules once make is successful.

Linux kernel consists of 2 parts - core kernel and modules. When we do simply make, it means make all which means make vmlinux && make modules. Hence, if we have done make, we need not do make modules again and we can simply run the command make modules_install without doing make modules.

You are doing nothing wrong. The modules target has been a dependency of the all target since kernel version 2.6.0 (actually since kernel version 2.5.60 I think).
The way you are adding your module is to add it to the kernel source tree. It is also possible to build custom modules outside the kernel source tree - so called "out-of-tree" kernel modules. Typically, those don't need a Kconfig file and the obj-$(CONFIG_HELLO_WORLD) would be replaced with obj-m in the Makefile.
Here is a Makefile for an "out-of-tree" version of your "hello" module:
ifneq ($(KERNELRELEASE),)
# Kbuild part of Makefile
obj-m += hello.o
else
# Normal part of Makefile
#
# Kernel build directory specified by KDIR variable
# Default to running kernel's build directory if KDIR not set externally
KDIR ?= "/lib/modules/`uname -r`/build"
all:
$(MAKE) -C "$(KDIR)" M=`pwd` modules
clean:
$(MAKE) -C "$(KDIR)" M=`pwd` clean
endif
This Makefile uses a common trick so that the same Makefile can be invoked as a "normal" Makefile and as a "kbuild" Makefile. The "normal" part between the else and endif lines invokes $(MAKE) on the kernel's Makefile (the location of which is specified by the KDIR variable), telling it to build the modules target in the current directory (specified by M=`pwd`). The "kbuild" part is between the ifneq($(KERNELRELEASE),) and else lines and is in the normal "kbuild" format for building parts of the kernel.
That trick depends on the KERNELRELEASE variable being initially unset or empty. It will be set to a non-empty value by the kernel's Makefile rules.

Related

Modern CMake Cross Compiling to AArch64 with Sysroots

Consider the following example project_(CMakeLists.txt):
cmake_minimum_required(VERSION 3.1)
project(CCL LANGUAGES C CXX)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_C_STANDARD 99)
set(CMAKE_C_STANDARD_REQUIRED ON)
find_package(PkgConfig REQUIRED)
find_package(ZLIB)
find_package(PNG)
find_library(MATH_LIBRARY m)
pkg_search_module(OpenEXR OpenEXR)
add_executable(main main.cpp)
if (MATH_LIBRARY)
target_link_libraries(main PUBLIC ${MATH_LIBRARIES})
endif()
Main.cpp:
#include <iostream>
#include <cmath>
int main(void)
{
std::cout << "Hello, sin()" << std::sin(30) << std::endl;
return 0;
}
I want to compile this project with the following CMake toolchain
(aarch64-toolchain.cmake):
# Cross-compilation system information.
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_PROCESSOR aarch64)
# The sysroot contains all the libraries we might need to link against and
# possibly headers we need for compilation.
set(CMAKE_SYSROOT /var/lib/schroot/chroots/ubuntu-focal-arm64)
set(CMAKE_FIND_ROOT_PATH ${CMAKE_SYSROOT})
set(CMAKE_LIBRARY_ARCHITECTURE aarch64-linux-gnu)
# Install path when SYSROOT is read-only.
# set(CMAKE_STAGING_PREFIX aarch64-staging)
# Set the compilers for C, C++ and Fortran.
set(GCC_TRIPLE "aarch64-linux-gnu")
set(CMAKE_C_COMPILER ${GCC_TRIPLE}-gcc-10 CACHE FILEPATH "C compiler")
set(CMAKE_CXX_COMPILER ${GCC_TRIPLE}-g++-10 CACHE FILEPATH "C++ compiler")
set(CMAKE_Fortran_COMPILER ${GCC_TRIPLE}-gfortran CACHE FILEPATH "Fortran compiler")
# Automatically use the cross-wrapper for pkg-config when available.
set(PKG_CONFIG_EXECUTABLE aarch64-linux-gnu-pkg-config CACHE FILEPATH "pkg-config executable")
# Set the architecture-specific compiler flags.
set(ARCH_FLAGS "-mcpu=cortex-a53+crc+simd")
set(CMAKE_C_FLAGS_INIT ${ARCH_FLAGS})
set(CMAKE_CXX_FLAGS_INIT ${ARCH_FLAGS})
set(CMAKE_Fortran_FLAGS_INIT ${ARCH_FLAGS})
# Don't look for programs in the sysroot (these are ARM programs, they won't run
# on the build machine).
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
# Only look for libraries, headers and packages in the sysroot, don't look on
# the build machine.
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
set(CPACK_DEBIAN_PACKAGE_ARCHITECTURE arm64)
Where the sysroot (/var/lib/schroot/chroots/ubuntu-focal-arm64) was setup using:
name=ubuntu-focal
mk-sbuild --arch=arm64 --skip-proposed --skip-updates --skip-security --name=${name} focal
su - $USER
mk-sbuild --arch=arm64 --skip-proposed --skip-updates --skip-security --name=${name} focal
Configuration-wise, this works fine, however when I try to build this project
find_library 'correctly' finds the wrong libm.so library:
$ cmake -DCMAKE_TOOLCHAIN_FILE=../aarch-toolchain.cmake ..
$ make
Scanning dependencies of target main
[ 50%] Building CXX object CMakeFiles/main.dir/main.cpp.o
make[2]: *** No rule to make target '/var/lib/schroot/chroots/ubuntu-focal-arm64/usr/lib/aarch64-linux-gnu/libm.so', needed by 'main'. Stop.
make[1]: *** [CMakeFiles/Makefile2:76: CMakeFiles/main.dir/all] Error 2
make: *** [Makefile:84: all] Error 2
Looking into the sysroot itself, the library is correctly found, but obviously,
it is a symlink to a file local to the sysroot, not the host:
$ ls -hal /var/lib/schroot/chroots/ubuntu-focal-arm64/usr/lib/aarch64-linux-gnu/
lrwxrwxrwx 1 root root 32 Apr 14 2020 libm.so -> /lib/aarch64-linux-gnu/libm.so.6
It seems to me that CMake is picking up the wrong library (find_library should
really be using the compiler libraries first), so I guess that my toolchain-file
is written incorrectly somehow. How should it be changed to correctly find the
math library, without changing the sysroot or project itself?
(Also, please note that this is an example project to illustrate the problem. I still need to be able to search the sysroot for packages.)
Edits
As requested in the comments, the end state should be a proper aarch64 binary
("main"). Essentially, the build commands below should succeed both with and
without the toolchain, and should yield a functional binary (although, the
aarch64 one naturally only work inside the sysroot).
$ mkdir -p host-build && cd host-build && cmake .. && make
$ mkdir -p device-build && cd device-build && cmake -DCMAKE_TOOLCHAIN_FILE=../aarch64-toolchain.cmake .. && make
TLDR: There is no easy way, as far as I know.
Your toolchain file is fine, it's the symlinks that are bad and they should ideally be relative and not absolute if you wish to use it for building. Below I mention a few options and explain the issue. This is all based on my experience with cross-compiling. Perhaps someone knows more.
There isn't anything wrong with your toolchainfile. From my perspective it is configured correctly. This is an issue as you pointed out with the symlinks being relative to your created rootfs and not absolute to the host machine but they are presented as absolute.
IIRC there is not much you can do here.
I'll give you a few options that I know of from personal experience:
Forget cross-compiling and using QEMU chroot into the environment and build there. This is the easiest option.
Get all the required libraries on your host machine (outside of the sysroot) and compile using them - You just have to make sure that you have the exact same libraries as the target machine. Your toolchain file would be then reconfigured to look for libraries among the ones here.
EDIT: As #josch pointed out in the comments under his answer I should expand on this point to make it more clear to users. However because he already mentioned the most important steps to take when applying this, I will refer the reader to his answer. I will however provide a link to debians official HowTo regarding Multi-arch
The other option would mean fixing your projects CMakeLists to more adapt to the cross-compiling. This would require you to specify the absolute paths to the correct libraries. There is a drawback as you need to always point to the newest library. You could for example create another .cmake file that helps you configure the correct paths and because target_link_libraries() allows you to specify absolute paths, you could use the newly created custom targets.
The last option would be to fix the symlinks (ideally by making them relative). This is in my opinion the preferred way. In the end the rootfs that you will be using for building will be just that, i.e. used for building.
As far as I know (I might be wrong here) but the mks-build is just a debian tool that is originally used for packaging .deb files and as such wasn't made to be complimentary to CMake. And if it was then the debian team probably has the toolchain files you are looking for (i.e. I would look for cross-compiling and package guides regarding .deb packages - if they exist they will be included).
CMAKE_SYSROOT, CMAKE_FIND_ROOT_PATH only add a prefix to specific find_ commands/functions. They pretty much can't change anything about the target system or the linker behavior (which is the issue here). Which means they are correctly configured. It's the fake system that is wrongly configured.
You wrote to the debian-cross mailing list so I assume that your question is Debian specific? If so, maybe tag your question as such. Here is how you cross compile any Debian package for amd64 (aarch64) using sbuild:
sbuild --host=arm64 your_package.dsc
sbuild will take care of setting up your chroot as required and call the build tools with the correct arguments. You do not even need to create a special build chroot for that and can just re-use your existing chroots (sbuild will know what to do with them).
If instead, you just want to cross-compile some software using CMake on Debian, the process is as simple as installing a cross-compiler and pointing CMake to it:
$ sudo apt install g++-aarch64-linux-gnu
$ cmake -DCMAKE_CXX_COMPILER=aarch64-linux-gnu-g++ .
$ make VERBOSE=1
...
/usr/bin/aarch64-linux-gnu-g++ -rdynamic CMakeFiles/main.dir/main.cpp.o -o main
...
$ file main
main: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1 ...
That's it. No need for a sysroot.
If you haven't set up arm64 as a foreign architecture on your system, you need to run this once:
dpkg --add-architecture arm64
apt-get update

Is there a provision to invoke jamfile and jamrule from cmake and vice versa?

Trying to migrate a legecy codebase whose buildsystem jam to CMake.
to divide and conquer it , checking whether there and provisions to Is there a provision to invoke jamfile and jamrule from cmake and vice versa .
one option would be add a custom target invoking jam program.
is it also possible to use a jamrule defined in a jamfile / .jam file
Disclaimer: There are several jam flavors. The answer applies to Perforce Jam and some of its compatible descendants.
As you've already mentioned yourself, invoking jam from cmake can be done with add_custom_target/add_custom_command, so that answers the first part of your question.
Since jam rules (or rather actions) can invoke arbitrary commands, the other direction is certainly possible as well. cmake itself is usually not the tool you invoke for building a target. So, depending on your generator, you would actually want to call make, ninja,...
In your question you're not very concrete regarding you migration approach. Assuming you start out with a jam build system with multiple library and executable targets that span a dependency graph, and you want to migrate the build system component by component. If you start bottom up with a library without dependencies (whose sources hopefully live in their own subdirectory), you would replace the rule invocation that builds the library -- e.g. Library libfoo : foo.c bar.c ; -- by a rule invocation that calls e.g. make -- like Make libfoo$(SUFLIB) ;. The rule could be defined (e.g. in Jamrules) as:
rule Make
{
# tell jam where the target will be created
MakeLocate $(1) : $(LOCATE_TARGET) ;
# always invoke the actions, since we don't let jam check the target's dependencies
Always $(1) ;
# we need the source dir in the actions
SOURCE_DIR on $(1) = $(SUBDIR) ;
}
actions Make
{
# get absolute source dir path
sourceDir=`cd $(SOURCE_DIR) && pwd`
# cd into the output directory
cd $(LOCATE)
# generate Makefile, if not done yet
if [ ! -e Makefile ]; then
cmake -G "Unix Makefiles" $(sourceDir) ;
fi
# make the target
make `basename $(1)`
}
If you need other information from jam to be passed to cmake (like the build type or certain build options), define respective on-target variables (like SOURCE_DIR in the example) to have them available in the actions.

Trying to trigger a minimal rebuild with CMake and set_source_files_properties

I've got my problem isolated to a very small two-sources project here: https://github.com/ennorehling/versioning/tree/v0.1
What I am trying to do is to not have a hard-coded version number in version.c, but to feed the version number into my build process from an external variable. See s/build for my build process: If $VERSION is defined, I want that to be the version number that the program prints. I achieve this by executing the command VERSION=1.2 s/build form the shell (or, if VERSION is undefined, by using the most recent tag from git).
The problem with this is that I run cmake .. -DVERSION=$VERSION every time I build, and that re-creates the Makefile, which causes a full rebuild. I would like to only rebuild version.o, since it is the only source file that depends on the version number. Building main.o (and in a real project, every other object, too) is unnecessary. I originally used add_definitions, which would add the -D compile switch to every source, and I thought set_source_files_properties was how I would be able to solve this, but since both object files are generated out of the same generated Makefile in build/CMakeFiles/version.dir/build.make, which gets touched by the cmake process. It seems that make errs on the safe side and just rebuilds everything?
Maybe I am barking up the wrong tree entirely, and just haven't found the correct CMake command for this, I don't know. Maybe there are other, proven ways to achieve what I'm trying to do? At this point, I've spent so much time on this, I'm not ashamed to ask for help.
I have found a way to work around this. The root problem here is that CMake create a Makefile for each library or executable target, and since my code only has one target, it was touching that Makefile every time. So the answer is to make a library target for version.c!
add_library(version OBJECT version.c)
add_executable(hello main.c $<TARGET_OBJECTS:version>)
this creates separate files in build/CMakeFiles/version.dir and build/CMakeFiles/hello.dir, and changes to the VERSION number only affect one of them, and the single target in it. The executable's dependencies are unchanged and don't get rebuilt, and only the linker step is executed, cutting down my build times as desired.
It's a bit clunky, but it works for me. New version of the project is here:
https://github.com/ennorehling/versioning/tree/v1.0
Edit: it turns out that Ubuntu Precise doesn't have CMake 2.8.8 yet, and the OBJECT option to add_library needs it. So instead, I have to actually create a library from that single object file and link it, like this:
add_library(version version.c)
add_executable(hello main.c)
target_link_libraries(hello version)
Not a huge deal, but a little annoying.
In your case you don't need to reconfigure every time. Once the make/build environment is written the problem can be reduced to a simple "has the file changed" problem that is checked by make itself.
So I probably would just generate the version.c inside your build script (and add this file to your .gitignore or alternatively directly generate it into the build directory):
s/build
#!/bin/sh
if [ -z "$VERSION" ]; then
VERSION=$(git describe --tags --match "v*.*")
if [ -z "$VERSION" ]; then
VERSION=1.0.0
fi
fi
echo "const char *version(void) { return \"${VERSION}\"; }" > version.c~
if cmake -E compare_files version.c~ version.c
then
cmake -E remove version.c~
else
cmake -E rename version.c~ version.c
fi
if ! [ -d build ]; then
cmake -E make_directory build
cmake -H. -Bbuild
fi
cmake --build build
CMakeLists.txt
cmake_minimum_required(VERSION 2.8)
project(version C)
add_executable(version version.c main.c)
version.c
const char *version(void) { return "1.0.0"; }
main.c
#include "stdio.h"
const char *version(void);
void main(void) {
puts(version());
}
This seems to be a bug in the CMake Unix Makefile generator. I tried your (original) code example using the Ninja generator, and ninja manages to avoid rebuilding main.c if you only change the version number.

Only run C preprocessor in cmake?

I'm trying to use cmake to simplify distributing my OpenCL program. I have a kernel file which includes several headers and other source files, and I want to have a single self contained executable.
My plan is to have cmake run the C preprocessor on the kernel source, turning the cl file and its includes into a single unit which is easier to work with.
I can use add_custom_command to do it by calling gcc/clang with -E, but then I don't get the flags to include the right directories to find the various header files in the command, and I don't see an easy way to find all current include directories to use in the custom call to the compiler.
Is there a way to run only the C preprocessor on a file with the current cmake environment?
CMake automatically generates make targets for preprocessing files. For each foo.c in your project there's a foo.i make target that will run only the preprocessor (with all the relevant -D and -I flags etc.). Run make help to see all other potentially useful targets that CMake generates in your makefiles.
BTW, I can't see how this "single unit" will be easier to work with for you.
This worked ok so far:
function(add_c_preprocessor_command)
# Add custom command to run C preprocessor.
#
# Arguments
# OUTPUT output file
# SOURCE input file
# TARGET CMake target to inherit compile definitions, include directories, and compile options
# EXTRA_C_FLAGS extra compiler flags added after all flags inherited from the TARGET
set(one_value_args TARGET SOURCE OUTPUT)
set(multi_value_args EXTRA_C_FLAGS)
cmake_parse_arguments(CPP "" "${one_value_args}" "${multi_value_args}" ${ARGN})
string(TOUPPER ${CMAKE_BUILD_TYPE} build_type)
string(REPLACE " " ";" c_flags "${CMAKE_C_FLAGS} ${CMAKE_C_FLAGS_${build_type}}")
add_custom_command(
OUTPUT ${CPP_OUTPUT}
COMMAND ${CMAKE_C_COMPILER}
"-D$<JOIN:$<TARGET_PROPERTY:${CPP_TARGET},COMPILE_DEFINITIONS>,;-D>"
"-I$<JOIN:$<TARGET_PROPERTY:${CPP_TARGET},INCLUDE_DIRECTORIES>,;-I>"
${c_flags}
$<TARGET_PROPERTY:${CPP_TARGET},COMPILE_OPTIONS>
${CPP_EXTRA_C_FLAGS}
-E ${CPP_SOURCE} -o ${CPP_OUTPUT}
COMMAND_EXPAND_LISTS VERBATIM
IMPLICIT_DEPENDS C ${CPP_SOURCE}
DEPENDS ${CPP_SOURCE})
endfunction()
This will be a crude hack, but you can abuse add_definition for that, as "This command can be used to add any flags, but it was originally intended to add preprocessor definitions."
Alternatively you could just set the COMPILE_FLAGS property of the target to "-E", which will achieve the same effect but be local to that target.
I would suggest a different approach.
Build your kernel into a binary (example for ATI Stream: http://developer.amd.com/support/KnowledgeBase/Lists/KnowledgeBase/DispForm.aspx?ID=115 )
Compile this binary data into your program (as a char[] blob) and load it when your program starts.
With cmake and custom targets this should be quite simple.

Create Custom Builds of an Xcode Project

I am going to build a Mac application written in Obj-C with Xcode. For argument's sake let's say it will have 10 optional features. I need a way to enable or disable those features to create custom builds of the application. These builds would be automated (most likely through the Mac OS X Terminal) so I would need a way to state which of these features are enabled/disabled at build time (a configuration file or CLI arguments would be ideal.)
So what is the best way to accomplish this? I'm trying to plan this out before I start coding so that there is proper separation in my code base to allow for these features to come and go. Ideally the custom build would only contain compiled code for the features it should have. In other words I don't want to always compile all the features and condition them out at runtime.
You can use Xcode configurations for this purpose; for each configuration you could include a different prefix header, for example. Then you can trigger builds form the command line via xcodebuild.
If you'd prefer the config file approach, you can use a .xcconfig file instead to define any of the Xcode build settings.
The Xcode Build System Guide describes both of these approaches.
use #ifdef and the -D flag under the compiler flags to control whether stuff is compiled in or out. You can set up lots of different configs this way if you want, and just have the xcode build configurations work nicely.
#include <stdio.h>
int
main (void)
{
#ifdef TEST
printf ("Test mode\n");
#endif
printf ("Running...\n");
return 0;
}
output 1:
$ gcc -Wall -DTEST dtest.c
$ ./a.out
Test mode
Running...
output 2:
$ gcc -Wall dtest.c
$ ./a.out
Running...
source: http://www.network-theory.co.uk/docs/gccintro/gccintro_34.html