Get the list of CMake configure_file calls - cmake

A bit of context first - I'm working on converting a CMake build system to an internal build system. For this I iterate BUILDSYSTEM_TARGETS and use get_property to get all the properties I need and everything works fine, except some files are missing from the build. After checking the CMakeLists.txt files from the original build system I realized configure_file is used in many and quite random places.
I assume CMake is storing configure_file calls internally. If this is the case, is it possible to access this?

Tsyvarev's answer of redefining configure_file works, but be aware that Craig Scott (one of the maintainers of CMake) has an article recommending against redefining CMake commands. Using the internal underscore-prefixed commands is relying on undocumented behaviour that can change in future versions. Using this trick can also result in infinite recursion.
While for your scenario it works fine, if you want to avoid using that trick, you can use the --trace* arguments to the cmake command.
--trace puts cmake in trace mode, which will print a trace of all calls made and from where.
--trace-expand is like --trace, but with variables expanded.
--trace-format=<format> lets you choose between human (a human readable format (the default value)), or json-v1, which prints JSON.
--trace-redirect=<file> puts cmake in trace mode and redirects trace output to a file instead of stderr.
So you could use the human format and grep for configure_file, or you could use the json-v1 format and write a script in a lanugage of your choice to search the JSON for calls to configure_file. You could also possibly use a comandline tool like jq to do the search.

You could redefine configure_file as a function (or macro) at the beginning of the project's CMakeLists.txt. That way allows you to run arbitrary code every time the function is invoked in the project.
Inside redefining function you could implement the logic which you need. For call original function in the redefining one, use underscore-prefixed name (_configure_file):
function(configure_file input output)
# ...
# Do something with the 'output' file. E.g. add it to the global list.
# ...
# After custom processing call original function with all parameters.
_configure_file(${input} ${output} ${ARGN})
endfunction()

Related

Is it possible to read the value of CMAKE_INSTALL_PREFIX variable in postinst?

Just wanted to ask if it is possible to read the value of CMAKE_INSTALL_PREFIX variable in postinst script? The goal is to have postinst script to read a file in ${CMAKE_INSTALL_PREFIX}/share/myapp folder and, based on the content of the file read, to generate a file in ${CMAKE_INSTALL_PREFIX}/etc/myapp folder. I'd like to avoid relying on absolute paths.
Thank you in advance!
Edit
Calling env from postinst gives good information:
DPKG_MAINTSCRIPT_DEBUG=0
DPKG_COLORS=never
DPKG_MAINTSCRIPT_NAME=postinst
DPKG_MAINTSCRIPT_PACKAGE=myapp
DPKG_RUNNING_VERSION=1.19.0.5
DPKG_FRONTEND_LOCKED=true
DPKG_ROOT=
DPKG_MAINTSCRIPT_ARCH=armhf
DPKG_ADMINDIR=/var/lib/dpkg
DPKG_MAINTSCRIPT_PACKAGE_REFCOUNT=1
DPKG_ROOT is probably what I need. It seems that CMAKE_INSTALL_PREFIX is not related to postinst.
Generally speaking at the moment of execution postinst there is no CMake variables (and they definitely never be in the env output). So, the way to do what you want is to generate your postinst script (from postinst.in file) using configure_file function. In the template file you can refer CMake variables, so being rendered your script would contain "hardcoded" literals instead. And that is quite normal.
But, take in account the following:
cpack uses CPACK_PACKAGING_INSTALL_PREFIX variable

Code generator generating its own CMake files and targets

Let's assume I have a script that generates a set of source files forming a target I want to link against in a CMakeLists.txt. If the file names are known to the latter then the usual add_custom_target() and add_custom_command() commands will make it possible to use the generated files as target sources.
Let's assume, though, that only the generator script knows the file names and locations. How can a target library be generated so that the parent CMakeLists.txt can link against it without its knowing the actual file names?
Note that the dependency topic isn't in this question's scope as the script knows itself when to regenerate or not. It's not the finest use of CMake, but it's sufficient in this use case.
Idea #1
The script also generates a generated.cmake file included by the parent one using include(generated.cmake). Problem: CMake doesn't find generated.cmake as it isn't existing at configuration time.
Idea #2
Similar to idea #1, but the script is called with the execute_process() so that generated.cmake is present at configuration time. Problem: The script is not called anymore at subsequent builds, thus ignoring possible changes to its input.
Idea #3
The script passes back a list of targets and files that is somehow considered by the parent CMakeLists.txt. So far I couldn't find a way to do so.
The solution I came with is eventually a mixture of all three ideas.
Solution to idea #1's problem
execute_process() actually ensures that generated_targets.cmake is present at configure time.
Solution to idea #2's and #3's problems
As stated in this answer to "Add dependency to the CMake-generated build-system itself", the CMAKE_CONFIGURE_DEPENDS directory property can be edited to add files whose touching re-triggers the configure step.
The key success factor is that this property can be set after the initial execute_process() call so that the script can identify and list its input dependencies (in an output file) that are then added to CMAKE_CONFIGURE_DEPENDS, hence also solving the input dependency problem.
Resulting pseudo code
# The script generates:
# - <output_dir>/cmake/input_files
# - <output_dir>/cmake/generated_targets.cmake
execute_process(
COMMAND myScript
--output-dir ${CMAKE_CURRENT_BINARY_DIR}/generated
)
# Mark the input files as configure step dependencies so that the execute_process
# commands are retriggered on input file change.
file(STRINGS ${CMAKE_CURRENT_BINARY_DIR}/generated/cmake/input_files _input_files)
set_property(
DIRECTORY APPEND PROPERTY CMAKE_CONFIGURE_DEPENDS
${_input_files}
)
# Add the generated CMake targets.
include(${CMAKE_CURRENT_BINARY_DIR}/generated/cmake/generated_targets.cmake)

Get build command or all compiler flags that will be used to build a target

Is there a sensible way to get a CMake variable containing the build command or all the compiler flags that CMake will associate with a target?
It doesn't seem practical to try to gather and maintain a list of all properties that could add flags. Besides, CMake must have this info somewhere, since it has to eventually generate a build system.
From the CMake docs it looks like this feature once existed and was provided by calling build_command() but this was replaced:
Note In CMake versions prior to 3.0 this command returned a command
line that directly invokes the native build tool for the current
generator.
Is there a new command that gives the old behavior of build_command()?
Is there a sensible way to get a CMake variable containing the build command or all the compiler flags that CMake will associate with a target?
The answer is no (CMake 3.23 is latest at time of writing), not during the CMake configure step.
In general, such a thing is ill-defined, which is likely why it was removed from CMake and will likely not be re-added. The complications arising from generator expressions, multi-config generators, generators that don't construct command lines (like VS/msbuild), source-file-specific properties, and the simple fact that after the command is called, relevant state might change, all make such efforts quixotic.
Honestly, this is such an odd thing to want at configure time, I wonder if this isn't an XY problem. It's unlikely that one target depends on another in such a way that the entire eventual command line is needed (rather than a particular property) to create it.
I know this is many years later now, but what were you trying to do?
CMake provides many ways post-generation to get information about the compiler command lines.
There's the CMake File API, meant for IDE integration,
The CMAKE_EXPORT_COMPILE_COMMANDS option that creates a Clang-compatible compile_commands.json, and then there's
The CMAKE_<LANG>_COMPILER_LAUNCHER variables that would let you instrument a full command line with a custom script while the build is running.
One of these might be useful. The latter is commonly used with ccache, but can be (ab)used with any arbitrary program as long as the output file is eventually generated.
Note that the latter two only work with the Makefile and Ninja generators.
If you want the final output of how the source files will actually be compiled you will want to look at the generated files. I don't really know a better way currently:
Example:
Here is an example output from Ninja Multi
build\CMakeFiles\impl-Release.ninja
This file will list all of the compile definitions, compiler flags, include directories, object directory, etc.
Under the path "cmake-build-debug/CMakeFiles/" you'll find a folder named as "TopFolderOfYourProject.dir", where the cmake generates all its build system files, including a file "build.make". In this file you can see something like this:
CMakeFiles/somepath/somesourcefile.c
#$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --green --progress-dir=xxx\cmake-build-debug\CMakeFiles --progress-num=$(CMAKE_PROGRESS_1) "Building C object CMakeFiles/somepath/somesourcefile.c.obj"
Besides this, you can find extra info about the flags in the file "flags.make", it contains all extra compiler flags specified by developers.
And in "includes_C.rsp/includes_CXX.rsp" you can see the including path.
Build flags are, actually, associated with source files, because you can have differrent flags for different files. On the other hand, for the most cases these flags are equivalent.
Anyways, to get all build flags for a source file you can use COMPILE_FLAGS property:
get_source_file_property(RESULT file.cpp COMPILE_FLAGS)

CMake and clang_complete

I'm wanting to setup my CMakeLists.txt file so that it can generate the .clang_complete file required by the vim plugin clang_complete.
Ordinarily, you would do this by passing a parameter to the python script it supplies with the compiler and all of the parameters for compilation. Note that I am omitting the actual directory cc_args.py is in to save on space.
cc_args.py gcc test.c -o test -I~/IncludeDirs/
You can also do this during the make phase...
make CC='cc_args.py gcc' CXX='cc_args.py g++'
However, I am unsure of how to (if it is possible to) set this up within a CMakeLists.txt file. It's really annoying to have to type this in every time I want to setup clang_complete. The reason why I want to do it this way, is because I have multiple projects that I use a custom script to build the CMakeLists.txt file, so having to write a script for each one or manually place a generic one is a step I'd like to avoid.
I've tried a couple of things that have so far have come up with errors.
I've tried setting CMAKE_CC_COMPILER and CMAKE_CXX_COMPILER to lines similar to the first i.e. "cc_args.py g++". The errors that come up here say that It can't find the compiler (which is understandable).
The next thing I tried was setting the Compiler variables just to the cc_args.py and adding a flag for the actual compiler: suffice to say, that failed horribly. CMake said that it couldn't compile a test program (considering the script isn't a compiler, and the tests don't use the flags I set, I'm not surprised at this).
So without writing any other external scripts that require moving around, is there anyone that can think of a way that can do this?
The solution is to set the CXX environment variable before executing cmake. Something like that:
CXX="$HOME/.vim/bin/cc_args.py clang++" cmake ..
make
See http://www.guyrutenberg.com/2013/01/29/vim-creating-clang_complete-using-cmake/ for more details.
I know you said "without writing any other external scripts," but it seems like you just need a one-liner:
exec cc_args.py g++
And then set that file as your CMAKE_CXX_COMPILER. You could even use CMake's file() function to write the one-liner at build time if you don't want to have to distribute it.

.h generated from .h.in?

There are struct definitions in the .h file that my library creates after I build it.. but I cannot find these in the corresponding .h.in. Can somebody tell me how all this works and where it gets the extra info from?
To be specific: I am building pth, the userspace threading library. It has pth_p.h.in, which doesn't contain the struct definition I am looking for, yet when I build the library, a pth_p.h appears and it has the definition I need.
In fact, I have searched every single file in the library before it is built and cannot find where it is generating the struct definition.
Pth uses GNU Autoconf, Automake, and Libtool. By running ./configure you'll be running a shell script which eventually runs m4 to detect the presence of a whole bunch of different system attributes and make changes to a number of files.
It looks like it boils down to ./configure generating Makefile from Makefile.in and then running something via make that triggers the shtool subcommand scpp:
pth_p.h: $(S)pth_p.h.in
$(SHTOOL) scpp -o pth_p.h -t $(S)pth_p.h.in -Dcpp -Cintern -M '==#==' $(HSRCS)
Obscure link, but here's an shtool-scpp manpage, which describes it as:
This command is an additional ANSI C
source file pre-processor for sharing
cpp(1) code segments, internal
variables and internal functions. The
intention for this comes from writing
libraries in ANSI C. Here a common
shared internal header file is usually
used for sharing information between
the library source files.
The operation is to parse special
constructs in files, generate a few
things out of these constructs and
insert them at position mark in tfile
by writing the output to ofile.
Additionally the files are never
touched or modified. Instead the
constructs are removed later by the
cpp(1) phase of the build process. The
only prerequisite is that every file
has a ``"#include ""ofile"""'' at the
top.
.h.in is probably processed within a configure (generated from configure.ac) script, look out for
AC_CONFIG_FILES([thatfile.h])
It replaces variables of the form #VAR# in the .in file with their values.
Edit: Just noticed if I'm right you should retag your question