How to configure CMake target or command to preprocess C file? - sql

I originally asked this question on CMake mailing list: How to configure target or command to preprocess C file?
I'm porting build configuration based on GNU Autotools to CMake and I have to deal with C preprocessing to generate a file.
The input for preprocessor is SQL file with C preprocessor directives used, like #include "another.sql", etc.
Currently, Makefile uses the following rule to generate plain SQL file as output:
myfile.sql: myfile.sql.in.c
cpp -I../common $< | grep -v '^#' > $#
So, the myfile.sql is meant to be one of products of the build process, similar to share libraries or executables.
What CMake tools should I use to achieve the same effect?
It's unclear to me if I should use add_custom_command, add_custom_target or combine both.
Obviously, I'm looking for a portable solution that would work at least with GNU GCC and Visual Studio toolsets. I presume I will have to define platform-specific custom commands, one for cpp preprocessor, one for cl.exe /P.
Or, does CMake provide any kind of abstraction for C preprocessor?
I scanned the archives, but I only found preprocessing of fortran files or solutions based on make capabilities: make myfile.i
So, it's not quite what I'm looking for.
UPDATE: Added answer based on solution received from Petr Kmoch on CMake mailing list.

I'm answering the question to myself by copying essential parts of solution received from Petr Kmoch as response to my post in the mailing list.
First, create a custom command using add_custom_command (version with the OUTPUT signature) to actually do the preprocessing.
For example:
add_custom_command(
OUTPUT myfile.sql
COMMAND "${CMAKE_C_COMPILER}" -E myfile.sql.in -I ../common
MAIN_DEPENDENCY myfile.sql.in
COMMENT "Preprocessing myfile.sql.in"
VERBATIM)
Second, configure the command trigger:
If the command output file (myfile.sql) is used in another target, added as a source
file added to add_library or add_executable, it is enough to specify it in these commands. CMake will find it as a dependency and run the custom command as required.
If the command output file is a final output not used as dependency in
any other targets, then add a custom target to drive the command using add_custom_target
For example:
add_custom_target(
ProcessSQL ALL
DEPENDS myfile.sql
COMMENT "Preprocessing SQL files"
VERBATIM)
Credits: Petr Kmoch

To take advantage of CMake' make myfile.i feature, you can do this:
add_library(sql_cpp_target EXCLUDE_FROM_ALL myfile.sql.in.c)
Now running make myfile.sql.in.c.i will produce preprocessed source for you, using defined CMAKE_C_FLAGS. It might be possible to change output name and dir for preprocessed file.
At any rate, you'd need to wrap these make invocations into add_custom_target(ALL ...) to make CMake run them during build.
Use CMAKE_MAKE_PROGRAM variable in targets definitions.
If you want to abstract from build tool, you can call cmake itself to build a target for you. Use ${CMAKE_COMMAND} --build ${CMAKE_BINARY_DIR} --target targetname when defining custom target.
Alternatively, you can just add_custom_command() which runs specified compiler to preprocess files and put them at the appropriate place. But this seems to be less flexible, IMO.

Related

Cmake get/query/print build settings on the command line?

Let's assume I have a Cmake C project, and I have something like this in the project:
project(my_project C CXX ASM)
set(my_executable_sources
main.c
file_one.c
)
set(my_executable_sources ${my_executable_sources} file_two.c)
add_executable(my_executable
${my_executable_sources}
file_three.c
)
Let's assume I'm in the ./build subfolder, and cmake ../ -G "Unix Makefiles" has passed successfully.
Can I somehow query build information from the command line using cmake?
For instance, I'm interested here in the final list of source files for my_executable; is there a command that would easily retrieve them? Say, like the following pseudocode:
$ cmake --pseudo-query-build --project="my_project" --target="my_executable" --query="source_files"
my_executable source files:
main.c
file_one.c
file_two.c
file_three.c
I don't see any cmake-generator-independent way of achieving this and even if you know the generator in use the project files generated.
You could modify your project to include a custom target that prints the sources though:
add_custom_target(print_my_executable_sources COMMAND ${CMAKE_COMMAND} -E echo "$<TARGET_PROPERTY:my_executable,SOURCES>" COMMAND_EXPAND_LISTS)
this allows you to use
cmake --build <binary_dir> [--config <configuration>] --target print_my_executable_sources
to print the sources, even if generator expressions are used in the sources.
Note: This does print all files in a single line; to get all the file names on separate lines, you could instead run cmake with the -P option passing the soures via -D and add logic to print one file name per line in the cmake script file.
Alternatively setting the CMAKE_EXPORT_COMPILE_COMMANDS variable to True during configuration could result in the generation of json files that would allow for the extraction of the information, but you'd need a json parser to extract the info. Furthermore this approach only works for the Makefile and Ninja CMake generators. Also I'm not sure you can tell which target a source file belongs to in all cases.
Start by obtaining the target's sources via this line (after add_executable):
get_target_property(MY_TARGET_SOURCES my_executable SOURCES)
And then proceed with a simple line at the end of the file (after add_executable)
message(STATUS ${MY_TARGET_SOURCES})
EDIT: For a full list of available target properties refer to this link.
EDIT2: As I've noticed now that you probably intend to just use it within the CLI, then for my solution you would also have to encapsulate it with a if/endif that checks for a definition e.g. if(SOURCES_DEBUG_INFO) and then run it with -DSOURCES_DEBUG_INFO=TRUE

CMake add_custom_command runs everytime

I am using a custom command for generating C++ Lexers and Parsers from ANTLR4 grammars. Right now I have the following:
set(MY_PARSER_INC
${PROJECT_SOURCE_DIR}/Headers/MyParser/MyLexer.h
${PROJECT_SOURCE_DIR}/Headers/MyParser/MyParser.h
${PROJECT_SOURCE_DIR}/Headers/MyParser/MyParserBaseVisitor.h
${PROJECT_SOURCE_DIR}/Headers/MyParser/MyParserVisitor.h
)
set(MY_PARSER_SRC
${PROJECT_SOURCE_DIR}/Sources/MyParser/MyLexer.cpp
${PROJECT_SOURCE_DIR}/Sources/MyParser/MyParser.cpp
)
add_custom_command(
OUTPUT ${MY_PARSER_INC} ${MY_PARSER_SRC}
DEPENDS ${PROJECT_SOURCE_DIR}/Grammars/MyGrammar.g4
COMMAND ${CMAKE_COMMAND} -E make_directory ${PROJECT_SOURCE_DIR}/Headers/MyParser/
COMMAND ${CMAKE_COMMAND} -E make_directory ${PROJECT_SOURCE_DIR}/Sources/MyParser/
COMMAND java -cp "${ANTLR_CLASSPATH}" "org.antlr.v4.Tool" -Dlanguage=Cpp -visitor -no-listener -package MY::NESTED::NAMESPACE -encoding iso-8859-1 -o ${PROJECT_SOURCE_DIR}/Sources/MyParser/ ${PROJECT_SOURCE_DIR}/Grammars/MyGrammar.g4
COMMAND ${CMAKE_COMMAND} -E copy ${PROJECT_SOURCE_DIR}/Sources/MyParser/*.h ${PROJECT_SOURCE_DIR}/Headers/MyParser/
COMMAND ${CMAKE_COMMAND} -E remove -f ${PROJECT_SOURCE_DIR}/Sources/MyParser/*.h
COMMENT "generating sources for My Parser"
)
and then I use the output files in my add_library command to maintain the dependency.
This does exactly what I expect it to do. It creates the lexers and parsers correctly. It maintains the dependency between those sources and the target library also correctly. Only one problem: It runs every time! Even when the grammar file has not been modified (I checked the file date on the grammar and generated lexers/parsers to be sure)! I have seen some similar questions online but still can't figure out why this is happening.
Any clue?!
EDIT1:
Adding more info, since it may still be unclear.
After the add_custom_command I have the following:
include_directories(${PROJECT_SOURCE_DIR}/Headers/MyParser/)
add_library(MyLibrary SHARED
${MY_PARSER_INC} ${MY_PARSER_SRC}
other_files.hpp other_files.cpp)
which I assume creates a direct dependency between the generated source files and my target library.
Here's my guess.
You need to make a custom target.
https://cmake.org/cmake/help/latest/command/add_custom_target.html?highlight=custom_target
add_custom_target(custom_target_create_parser_code DEPENDS
${MY_PARSER_INC} ${MY_PARSER_SRC}
)
This will make a target that depends on the custom command you wrote.
Now cmake has something to attach your command to.
Now you need to add the dependency to your static library you mentioned.
https://cmake.org/cmake/help/latest/command/add_dependencies.html
add_dependency(your_static_library custom_target_create_parser_code)
Now cmake shouldn't re-run your custom command code everytime.
EDIT BTW:
Craig Scott's book actually has a subchapter devoted to this topic:
https://crascit.com/professional-cmake/
Chapter 18 Custom Targets
Chapter 18.3 Commands That Generate Files
EDIT #2:
If all else fails try the official cmake discourse:
https://discourse.cmake.org/
Your question is well formed and cmake developers look their for questions.
So you should get an answer.

CMake call add_subdirectory within custom command

I'm working with a code generator that produces C++ and a CMakeLists.txt file, unfortunately I cannot use this in my main CMakeLists.txt file for testing purposes.
For example you have the following CMakeLists.txt file:
project(SomeProject CXX C)
add_custom_command(OUTPUT ${SRCS}
COMMAND ${CODEGEN_CLI_PATH} -i "${INPUT}" -o "${OUT}"
COMMENT "Generating sources"
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
VERBATIM
)
add_custom_target(CODEGEN
DEPENDS
${SRCS}
)
# Needs to be executed after the custom command
add_subdirectory(${GENERATED_CMAKE_LISTS_LOCATION})
Is it possible to use functions such as add_subdirectory only after you execute custom commands for a particular target, such as CODEGEN?
I've already tried to execute it by adding an extra line to the existing custom command:
COMMAND ${CMAKE_COMMAND} -D DIR=${GENERATED_CMAKE_LISTS_LOCATION} -P add_subdirectories.cmake
Unfortuantly this doesn't work because it isn't allowed to execute functions like add_subdirectory in script mode.
Neither I can manage to call custom made functions (that are executing add_subdirectory) from add_custom_command that are located in the same file.
Nope, it is not possible. The add_subdirectory command is run during configuration step, while CODEGEN is a target that runs during build.
You seem to be doing something wrong, so the only advice I can give you is to use execute_process command to run commands you need. The execute_process command is executed during configuration stage, so it will be able to generate files you need before add_subdirectory.
But again, please describe your problem, why do you want CMake to do that.
I have a huge fixed unsigned char array that I compiled into a static library. The way I work around it is by:
if(NOT EXISTS ${PATH_TO_FOLDER}/smeagol.a)
add_subdirectory(smeagol)
endif()
I'm still looking for a nicer kung-fu way to do it using cmake. I feel that its out there, and I will update this answer once i find it.

How to define a variable during build with cmake?

I would like to define a CMake variable BUILD_TIME_VAR in CMakeLists.txt:
computed with a python script during build phase
I can then access its content with ${VAR}
In other words, the equivalent during build phase of:
execute_process(COMMAND bash -c "python $SCRIPT $FILE" OUTPUT_VARIABLE GEN_TIME_VAR)
The variable is then used to generate a file which is a dependency to make binaries.
The goal is to make the code more easy to read since otherwise, the computation occur several times.
Rather than calling n times, the python script, to compute BUILD_TIME_VAR, I would like to use the script once, to factor the code in this way:
if(expression_1)
add_custom_command(OUTPUT foo
COMMAND cmd_1(${BUILD_TIME_VAR}))
...
elseif(expression_2)
# elseif section.
add_custom_command(OUTPUT foo
COMMAND cmd_2(${BUILD_TIME_VAR}))
...
else(expression_n)
# else section.
add_custom_command(OUTPUT foo
COMMAND cmd_n(${BUILD_TIME_VAR}))
...
endif(expression)
add_custom_target(${BINARY} ALL
DEPENDS foo)
Thanks you for your help.
If I understand your question correctly, you're effectively trying to create a "build-time variable." That is something that the build tool (actually all the build tools supported by CMake) would have to support. I know of no such functionality in build tools (make, ninja, VS, ...), and hence of no support for such a thing in CMake either.
You could emulate this by writing the results to a file and reading that file in all subsequent build steps using it.
I believe you've misinterpreted the role of CMake. In short, CMake generates build files for the back-end system of your choice (Make, ninja, etc.). This is concisely reviewed here, where the configure/generate step is briefly documented along with how it precedes the actual build step.
It's not clear what you're trying to achieve exactly, but maybe defining custom commands and using their outputs to link them to custom targets for further dependency chaining (see DEPENDS here) might be helpful, e.g.
cmake_minimum_required(VERSION 3.11)
project(foobar)
set(FOO_FILES "foo.txt")
add_custom_command(OUTPUT ${FOO_FILES}
COMMAND ${CMAKE_COMMAND} -E touch ${FOO_FILES}
COMMAND echo hello >> ${FOO_FILES}
WORKING_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}"
COMMENT "Creating ${FOO_FILES}"
VERBATIM)
add_custom_target(foo DEPENDS ${FOO_FILES})
set(BAR_FILES "bar_dummy.txt") # this is only use to link the custom command to the corresponding custom target
add_custom_command(OUTPUT ${BAR_FILES}
COMMAND ${CMAKE_COMMAND} -E touch ${BAR_FILES}
COMMAND cat ${FOO_FILES}
WORKING_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}"
COMMENT "Displaying ${FOO_FILES}"
VERBATIM)
add_custom_target(bar DEPENDS ${BAR_FILES})
add_dependencies(bar foo)
However, you might need to consider the probability of going against the tool, and thus, it might be clearer to have a required step prior to the configuration of your project.

Retrieve all link flags in CMake

In CMake, is it possible to programmatically retrieve the complete list of linker flags that will be used for a given target? The only way I can see to do this is to inspect the link.txt file in the target's CMakeFiles directory. Not ideal.
The use case that I'm interested in is to collect the data to include in something like a pkg-config file. I'm writing a library, and it includes a couple executable utilities that use the library. Building the executables (especially when the library is build statically) requires a non-trivial link line to link to my library and its dependencies. So I'd like to write out the link line necessary for building these executables to a data file included with the package such that other clients can know how to link.
As #Tsyvarev has commented there is no build-in command or property "to programmatically retrieve the complete list of linker flags" in CMake.
But inspired by your hint "so I'd like to write out the link line necessary for building these executables to a data file" I think I found a feasible solution (at least for makefile generators).
And if I understand your request correctly, we are not talking about simple verbose outputs like you get with e.g. CMAKE_VERBOSE_MAKEFILE, which would still need you to copy things manually.
So taking the following into account:
You need to run the generator first to get the real link line
CMake allows you to invent any linker language by name
You can define the link line with CMAKE_>LANG<_LINK_EXECUTABLE using variables and expansion rules
I came up with adding an LinkLine executable using my ECHO "linker" with the single purpose to create a link line file of my choosing:
set(CMAKE_ECHO_STANDARD_LIBRARIES ${CMAKE_CXX_STANDARD_LIBRARIES})
set(CMAKE_ECHO_FLAGS ${CMAKE_CXX_FLAGS})
set(CMAKE_ECHO_LINK_FLAGS ${CMAKE_CXX_LINK_FLAGS})
set(CMAKE_ECHO_IMPLICIT_LINK_DIRECTORIES ${CMAKE_CXX_IMPLICIT_LINK_DIRECTORIES})
set(
CMAKE_ECHO_LINK_EXECUTABLE
"<CMAKE_COMMAND> -E echo \"<FLAGS> <LINK_FLAGS> <LINK_LIBRARIES>\" > <TARGET>"
)
add_executable(LinkLine "")
target_link_libraries(LinkLine MyLibraryTarget)
set_target_properties(
LinkLine
PROPERTIES
LINKER_LANGUAGE ECHO
SUFFIX ".txt"
)
The nice thing about this approach is, that the output of my LinkLine target can be used as any other "officially generated" executable output (e.g. in install() commands or post-build steps with generator expressions):
add_custom_command(
TARGET LinkLine
POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy $<TARGET_FILE:LinkLine> PackageCfg/$<TARGET_FILE_NAME:LinkLine>
)
References
Recursive list of LINK_LIBRARIES in CMake
add_custom_command is not generating a target