How do I generate source files during the pre-build step if and only if they need to be regenerated?
One of my project's dependencies is a library (libfoo) which expensive to relink (several minutes) and even more expensive to rebuild (less than an hour). Generating the source files for this dependency is inexpensive (several seconds), but using out-of-date sources would render the resultant application suite useless. I have a command check_foo.sh that will exit with a non-zero status when the sources must be regenerated, but I haven't been able to determine how to convince CMake to run check_foo.sh during every build and only rebuild libfoo when check_foo.sh returns nonzero.
In trying to create a simple proof, the closest I have gotten is as follows, although only ever runs generate_foo_if.sh once. The ultimate goal is that generate_foo_if.sh gets run unconditionally, but libfoo is only rebuilt when generate_foo_if.sh modifies foo.cpp.
CMakeLists.txt
cmake_minimum_required(VERSION 3.8 FATAL_ERROR)
project(my_project
VERSION 1.0.0.0
LANGUAGES CXX)
add_custom_command(OUTPUT foo.cpp
COMMAND "${CMAKE_CURRENT_SOURCE_DIR}/generate_foo_if.sh" "${CMAKE_CURRENT_BINARY_DIR}/foo.cpp" "${CMAKE_CURRENT_SOURCE_DIR}/check_foo.sh"
WORKING_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}"
COMMENT "Generating foo.cpp..."
)
add_library(foo STATIC foo.cpp)
add_executable(main main.cpp)
target_link_libraries(main foo)
main.cpp
#include "foo.hpp"
int main(int,char**){
return foo::exit_status;
}
foo.hpp
#pragma once
namespace foo {
extern const int exit_status;
}
check_foo.sh
#!/usr/bin/env bash
exit $(((${RANDOM} % 2 )))
generate_foo_if.sh
#!/usr/bin/bash
CHECK=${2:-./check_foo.sh}
if [ ${CHECK} -eq 0 ]; then
exit 0
fi
msg=$(cat <<__EOF
#include "foo.hpp"
namespace foo {
const int exit_status = 1;
}
__EOF
)
echo "${msg}" >${1:-foo.cpp}
Turns out, I wasn't very far off the mark.
The critical difference seems to be in creating a custom target rather than a custom command, using BYPRODUCTS rather than OUTPUT, and explicitly adding the dependency. Updated CMakeLists.txt is below, and results in the desired behavior (i.e. libfoo is only regenerated if it should be).
cmake_minimum_required(VERSION 3.8 FATAL_ERROR)
project(my_project
VERSION 1.0.0.0
LANGUAGES CXX)
add_custom_target(generate_foo
COMMAND "${CMAKE_CURRENT_SOURCE_DIR}/generate_foo_if.sh" "${CMAKE_CURRENT_BINARY_DIR}/foo.cpp" "${CMAKE_CURRENT_SOURCE_DIR}/check_foo.sh"
WORKING_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}"
BYPRODUCTS foo.cpp
)
add_library(foo STATIC foo.cpp)
add_dependencies(foo generate_foo)
include_directories("${CMAKE_CURRENT_SOURCE_DIR}")
add_executable(main main.cpp)
target_link_libraries(main foo)
While I can't really say I know why it works, this does what I want.
Related
I am working on an embedded project managed with CMake and I would like to do different actions on the same executable.
Consider I have the following project:
/cmake/foo.cmake
/CMakeLists.txt
/main.cpp
Irrelevant main.cpp for this question:
#include <iostream>
int main() {
std::cout << "hello world" << std::endl;
return 0;
}
The foo.cmake file contains a function:
function(print_executable_size TARGET_NAME)
add_custom_command(
TARGET ${TARGET_NAME}
POST_BUILD
COMMAND size ${TARGET_NAME}.exe
)
endfunction()
The main CMakeLists.txt has the following content:
cmake_minimum_required(VERSION 3.15)
project(proj)
set(CMAKE_MODULE_PATH "${PROJECT_SOURCE_DIR}/cmake")
include(foo)
add_executable(pgm-dev main.cpp)
add_executable(pgm-prod ALIAS pgm-dev)
# this line works but this is not needed when compiling the "dev" program.
print_executable_size(pgm-dev)
# Uncomment this line leads to TARGET 'pgm-prod' was not created in this directory.
# print_executable_size(pgm-prod)
My ideal build process would be to have:
a cmake target "pgm-dev" that just build the executable,
a cmake target "pgm-prod" that re-build the "pgm-dev" executable if necessary AND do some POST_BUILD actions.
Why having two targets "pgm-dev" and "pgm-prod"?
Consider the "pgm-prod" does some extra build actions like cipher the produced binary so, no need to do it in everyday development.
Why use a cmake file with a function instead of add_custom_command() right after executable?
I have multiple executables concerned in my whole project and I would like to avoid code duplications.
Why not creating another executable, with a different name?
"pgm-dev" and "pgm-prod" are exactly compiled the same way, only post build actions differ.
I thought using add_executable(.. ALIAS ..) would be great for this but it seems I'm not understanding some key points here. What would be the best CMake approach to do what I want?
Don't create an alias. Instead create a custom target that executes the commands and add a dependency to the executable target
add_executable(pgm-dev main.cpp)
add_custom_target(pgm-prod COMMAND size $<TARGET_FILE:pgm-dev>)
add_dependency(pgm-prod pgm-dev)
This way cmake makes sure that pgm-dev is built before the command is executed.
If you need to more than one of those commands to all be executed, you could introduce intermediate targets that execute the command that depend on the original target and create a target for executing all those commands that depends on all of those:
function(pgm_add_custom_postbuild_command ORIGINAL_TARGET NEW_TARGET SUFFIX)
if(NOT TARGET ${ORIGINAL_TARGET})
add_custom_target(${ORIGINAL_TARGET})
endif()
add_custom_target(pgm_internal_${ORIGINAL_TARGET}_${SUFFIX} ${ARGN})
# hide away target in a dedicated folder, if folders are activated in the IDE
set_target_properties(pgm_internal_${ORIGINAL_TARGET}_${SUFFIX} PROPERTIES FOLDER PgmInternal)
add_dependencies(pgm_internal_${ORIGINAL_TARGET}_${SUFFIX} ${ORIGINAL_TARGET})
add_dependencies(${NEW_TARGET} pgm_internal_${ORIGINAL_TARGET}_${SUFFIX})
endfunction()
function(pgm_print_size ORIGINAL_TARGET NEW_TARGET)
pgm_add_custom_postbuild_command(${ORIGINAL_TARGET} ${NEW_TARGET} size
COMMAND size $<TARGET_FILE:${ORIGINAL_TARGET}>)
endfunction()
function(pgm_print_md5sum ORIGINAL_TARGET NEW_TARGET)
pgm_add_custom_postbuild_command(${ORIGINAL_TARGET} ${NEW_TARGET} md5sum
COMMAND ${CMAKE_COMMAND} -E md5sum $<TARGET_FILE:${ORIGINAL_TARGET}>)
endfunction()
pgm_print_size(pgm-dev pgm-prod)
pgm_print_md5sum(pgm-dev pgm-prod)
From relevant search on the web, I got an impression that
google-benchmark is not easily incorporated into a CMake-project.
One way I could do that is adding as external project,
replicating verbatim the corresponding text for GTest in google-test's readme:
# adding google-benchmark as external git project
#=======================================================
configure_file(extern/googlebenchmark/CMakeLists.txt.in googlebenchmark-download/CMakeLists.txt)
execute_process(COMMAND ${CMAKE_COMMAND} -G "${CMAKE_GENERATOR}" .
RESULT_VARIABLE result
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/googlebenchmark-download)
if(result)
message(FATAL_ERROR "CMake step for googlebenchmark failed: ${result}")
endif()
execute_process(COMMAND ${CMAKE_COMMAND} --build .
RESULT_VARIABLE result
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/googlebenchmark-download )
if(result)
message(FATAL_ERROR "Build step for googlebenchmark failed: ${result}")
endif()
# Prevent overriding the parent project's compiler/linker
# settings on Windows
set(gbenchmark_force_shared_crt ON CACHE BOOL "" FORCE)
add_subdirectory(${CMAKE_CURRENT_BINARY_DIR}/googlebenchmark-src
${CMAKE_CURRENT_BINARY_DIR}/googlebenchmark-build
EXCLUDE_FROM_ALL)
if (CMAKE_VERSION VERSION_LESS 2.8.11)
include_directories("${gbenchmark_SOURCE_DIR}/include")
endif()
and having CMakeLists.txt.in with the contents as in
How to build and link google benchmark using cmake in windows
This however has a huge downside: every time we change something in the CMakeLists.txt -- the topmost one -- that hosts all this, it starts building google-benchmark from scratch,
and running all the "tests", whatever they are. Thus compilation times become longer.
Without a root access to a Linux server, is there any more-less portable way
of having the benchmark code installed in one's home directory, while being
able to link towards it in a CMake-project?
EDIT: I must say I've been able to git clone the benchmark code and successfully built it in my home directory.
EDIT: Answering my own question. I am not sure if this warrants a close, and leave it to the patrons to decide, but I've solved the problem as follows. In the CMakeLists.txt one can have the following contents:
cmake_minimum_required(VERSION 3.15)
set(CMAKE_CXX_STANDARD 17)
add_executable(sample_bench sample_bench.cpp)
target_link_libraries(sample_bench PUBLIC benchmark benchmark_main pthread)
target_link_directories(sample_bench PUBLIC ~/local/benchmark/build/src)
target_include_directories(sample_bench PUBLIC
~/local/benchmark/include)
The key here is target_link_directories, which is specified with -L in the example in https://github.com/google/benchmark
Answering my own question. I've solved the problem as follows. In the CMakeLists.txt one can have the following contents:
cmake_minimum_required(VERSION 3.15)
set(CMAKE_CXX_STANDARD 17)
add_executable(sample_bench sample_bench.cpp)
target_link_libraries(sample_bench PUBLIC benchmark benchmark_main pthread)
target_link_directories(sample_bench PUBLIC ~/local/benchmark/build/src)
target_include_directories(sample_bench PUBLIC
~/local/benchmark/include)
The key here is target_link_directories, which is specified with -L in the example in https://github.com/google/benchmark
This is enough to run the sample benchmark, at least -- haven't tried others.
That is, once you have built benchmark somewhere -- even in your home directory, as in the example -- you can point CMake to the designated locations,
in order for your code to compile.
I want to build a c++ static lib out of some protobuf definitions with cmake/make.
I made a custom COMMAND to compile the protobuf to c++, and I set it as a PRE_BUILD dependency to my static lib.
project(mylib)
set(PROTO_PATH "${CMAKE_CURRENT_SOURCE_DIR}/proto_definitions")
file(GLOB PROTO_FILES "${PROTO_PATH}/*.proto")
foreach(PROTO_FILE in ${PROTO_FILES})
string(REGEX REPLACE "[.]proto$" ".pb.cc" OUTPUT_SOURCE ${PROTO_FILE})
list(APPEND OUTPUT_SOURCES ${OUTPUT_SOURCE})
endforeach()
add_custom_command(TARGET ${PROJECT_NAME}
PRE_BUILD
COMMAND protoc --cpp_out=${CMAKE_CURRENT_SOURCE_DIR}/compiled_proto ${PROTO_FILES}
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
COMMENT "some comment")
add_library(${PROJECT_NAME} STATIC ${OUTPUT_SOURCES})
target_include_directories(${PROJECT_NAME} PUBLIC ${CMAKE_CURRENT_BINARY_DIR})
I get the following error when running cmake:
CMake Error: cannot determine link language for target "mylib"
Regardless of this error, the makefiles are generated, but when I make mylib, it does not trigger any proto compilation
The approach more native to CMake would be to add custom commands with the OUTPUT signature to generate the .cc files, and then use them as sources for the library normally. That way, they CMake will know what they are and how to produce them:
project(mylib)
set(PROTO_PATH "${CMAKE_CURRENT_SOURCE_DIR}/proto_definitions")
file(GLOB PROTO_FILES "${PROTO_PATH}/*.proto")
foreach(PROTO_FILE in ${PROTO_FILES})
string(REGEX REPLACE "[.]proto$" ".pb.cc" OUTPUT_SOURCE ${PROTO_FILE})
list(APPEND OUTPUT_SOURCES ${OUTPUT_SOURCE})
endforeach()
add_custom_command(OUTPUT ${OUTPUT_SOURCES}
COMMAND protoc --cpp_out=${CMAKE_CURRENT_SOURCE_DIR}/compiled_proto ${PROTO_FILES}
DEPENDS ${PROTO_FILES}
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
COMMENT "some comment")
add_library(${PROJECT_NAME} STATIC ${OUTPUT_SOURCES})
target_include_directories(${PROJECT_NAME} PUBLIC ${CMAKE_CURRENT_BINARY_DIR})
Done this way, there will be one command which reads all the .proto files and produces all the .cc files — which means that if any of the .proto file changes, all the .cc files will be re-generated. I am not familiar with Protobuffers so I cannot know whether that's sane or not. If they are independent, it would be better to introduce one add_custom_command for each output file.
Also, given the arguments you're passing to protocc, you might have to modify the paths in OUTPUT_SOURCES to correctly point to the generated files.
Also note that CMake comes with a FindProtobuf module which defines a protobuf_generate_cpp() command, so you might want to use that instead of hand-coding the Protobuf support.
Comment for the answer above: There is no in in cmake foreach, which spend me some time to solve it.
After studying his answer(very thanks!), I summarized a method that works for me, which has the following characteristics:
Proto dir and Output dir can be different
Generate grpc_output
Create a custom_command for each proto
My project strcuture is:
hwpb/ (a library for super project)
cpp/
hwpb/ (store *.pb.cc, *.pb.h)
CMakeLists.txt
proto/ (store protos)
go/ (and other languages, not mentioned here)
And this is my CMakeListx.txt:
set(PROTO_DIR "${CMAKE_CURRENT_SOURCE_DIR}/../proto")
set(OUTPUT_DIR "${CMAKE_CURRENT_SOURCE_DIR}/hwpb")
file(GLOB PROTO_FILES "${PROTO_DIR}/*.proto")
foreach(PROTO_FILE ${PROTO_FILES})
get_filename_component (FILENAME ${PROTO_FILE} NAME_WLE)
set(PROTO_SRC "${OUTPUT_DIR}/${FILENAME}.pb.cc")
set(PROTO_HDR "${OUTPUT_DIR}/${FILENAME}.pb.h")
set(GRPC_SRC "${OUTPUT_DIR}/${FILENAME}.grpc.pb.cc")
set(GRPC_HDR "${OUTPUT_DIR}/${FILENAME}.grpc.pb.h")
add_custom_command(
OUTPUT "${PROTO_SRC}" "${PROTO_HDR}" "${GRPC_SRC}" "${GRPC_HDR}"
COMMAND ${_PROTOBUF_PROTOC}
ARGS --plugin=protoc-gen-grpc=${_GRPC_CPP_PLUGIN_EXECUTABLE}
--cpp_out="${OUTPUT_DIR}" --grpc_out="${OUTPUT_DIR}"
-I"${PROTO_DIR}" "${PROTO_FILE}"
DEPENDS ${PROTO_FILE}
)
list(APPEND OUTPUT_SOURCES ${PROTO_SRC} ${GRPC_HDR})
endforeach()
add_library(hwpb ${OUTPUT_SOURCES})
target_link_libraries(hwpb ${_GRPC_GRPCPP_UNSECURE} ${_PROTOBUF_LIBPROTOBUF})
target_include_directories(hwpb PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})
Suppose you have a very simple CMakeLists.txt
add_executable(silent T.cpp A.asm)
CMake will happily generate a C++ target for building silent, with T.cpp in it, but will silently drop any and all reference to A.asm, because it doesn't know what to do with the suffix.
Is there any way to get CMake to loudly complain about this source file it doesn't understand (to aid in porting a Makefile to CMake).
Ignoring unknown file extensions is - unfortunately for your case - by design.
If I look at the code of cmGeneratorTarget::ComputeKindedSources() anything unknown ends up to be classified as SourceKindExtra (to be added as such to generated IDE files).
So I tested a little and came up with the following script that evaluates your executable target source files for valid file extensions by overwriting add_executable() itself:
cmake_minimum_required(VERSION 3.3)
project(silent CXX)
file(WRITE T.cpp "int main() { return 0; }")
file(WRITE T.h "")
file(WRITE A.asm "")
function(add_executable _target)
_add_executable(${_target} ${ARGN})
get_property(_langs GLOBAL PROPERTY ENABLED_LANGUAGES)
foreach(_lang IN LISTS _langs)
list(APPEND _ignore "${CMAKE_${_lang}_IGNORE_EXTENSIONS}")
endforeach()
get_target_property(_srcs ${_target} SOURCES)
foreach(_src IN LISTS _srcs)
get_source_file_property(_lang "${_src}" LANGUAGE)
get_filename_component(_ext "${_src}" EXT)
string(SUBSTRING "${_ext}" 1 -1 _ext) # remove leading dot
if (NOT _lang AND NOT _ext IN_LIST _ignore)
message(FATAL_ERROR "Target ${_target}: Unknown source file type '${_src}'")
endif()
endforeach()
endfunction()
add_executable(silent T.cpp T.h A.asm)
Since you wanted a rather loudly complain by CMake I declared it an FATAL_ERROR in this example implementation.
CMake doesn't just drop unknown files in add_executable().
If alongside with
add_executable(silent T.cpp A.asm)
you have
add_custom_command(OUTPUT A.asm COMMAND <...>
DEPENDS <dependees>)
then whenever <dependees> changed CMake will rerun command for create A.asm before compiling the executable.
Note, that automatical headers scanning doesn't provide such functionality: if your executable includes foo.h then executable will be rebuilt only when foo.h itself is changed. Any custom command creating this header will be ignored.
However, you may change behavior of add_executable by redefining it. See #Florian's answer for example of such redefinition.
I'm writing build configuration with cmake: besides main project with own code there are some external libraries. For sake of easy updating of these libraries (zlib, libpng, ...) I don't want to modify its cmakelists files, but I need specific libraries targets (to use in target_link_libraries() for example).
Another restrictions is that I can't just say that my code needs external libraries installed, all things must be located in one source code tree and must be built together.
To keep everything libraries provide structured (libs, headers) I want to install (like make install does) libraries to local building folder and then include generated cmake-file to import needed targets to my project.
I suppose flow as follows:
build external library with add_subdirectory()
install external library files into local directory
import targets using generated cmake-file
use imported targets in primary project cmake files
Problem is to automate step 2 (need to trigger install target after add_subdiretory inside main project CMakeLists.txt). I can build and install all libraries and then build own code, but this isn't convenient.
So question is how can I tell cmake to do intermediate install during build?
A little working example here:
file structure:
prj/CMakeLists.txt
prj/src/main.cpp
lib/CMakeLists.txt
lib/include/libheader.h
lib/src/libsource.cpp
prj/CMakeLists.txt
project(TestProject)
cmake_minimum_required(VERSION 2.8)
set(CMAKE_VERBOSE_MAKEFILE on)
set(WORK_DIR ${CMAKE_CURRENT_SOURCE_DIR})
# supposed to use add_subdirectory here (with forced install).
# and then include prespecified include-file as here.
include_directories(${WORK_DIR}/../lib/build/install/include)
include(${WORK_DIR}/../lib/build/install/lib/libtargets.cmake)
add_executable(main ${WORK_DIR}/src/main.cpp)
target_link_libraries(main library_target)
lib/CMakeLists.txt
project(TestLib)
cmake_minimum_required(VERSION 2.8)
set(CMAKE_VERBOSE_MAKEFILE on)
set(WORK_DIR ${CMAKE_CURRENT_SOURCE_DIR})
include_directories(${WORK_DIR}/include)
add_library(library_target STATIC ${WORK_DIR}/src/libsource.cpp)
set(CMAKE_INSTALL_PREFIX ${CMAKE_CURRENT_BINARY_DIR}/install)
install(FILES ${WORK_DIR}/include/libheader.h DESTINATION include)
install(TARGETS library_target DESTINATION lib EXPORT libtargets)
install(EXPORT libtargets DESTINATION lib)
prj/src/main.cpp
#include <iostream>
#include "libheader.h"
using std::cout;
int main()
{
cout << getRandomNumber() << "\n";
return 0;
}
lib/include/libheader.h
#ifndef _LIBHEADER_H_
#define _LIBHEADER_H_
int getRandomNumber();
#endif
lib/src/libsource.cpp
#include <iostream>
#include "libheader.h"
int getRandomNumber()
{
return 4; // guaranteed to be random.
}
You can use following command to build all:
pushd . && mkdir lib/build && cd lib/build && cmake .. && make && make install && popd && pushd . && mkdir prj/build && cd prj/build && cmake .. && make || popd
EDIT: desired cmakelists in main project:
set(CMAKE_INSTALL_PREFIX ${CMAKE_CURRENT_BINARY_DIR}/install)
add_subdirectory(${WORK_DIR}/../lib ${CMAKE_CURRENT_BINARY_DIR}/lib.d)
# force install here, somehow
# because if not we will get "include could not find load file" here.
include(${CMAKE_CURRENT_BINARY_DIR}/install/lib/libtargets.cmake)
include_directories(${CMAKE_CURRENT_BINARY_DIR}/install/include)
sorry, I have no privilege to put comment
So, I put the reference link here
https://cmake.org/Bug/view.php?id=14311
this is also useful when including prebuilt library binaries in the
project.
add_library(prebuilt STATIC IMPORTED) set_property(TARGET prebuilt
PROPERTY IMPORTED_LOCATION ${CMAKE_CURRENT_SOURCE_DIR}/libprebuilt.a)
install(TARGETS other_built_targets prebuilt
EXPORT project-export
RUNTIME DESTINATION bin
LIBRARY DESTINATION lib
ARCHIVE DESTINATION lib )
Making prebuilt an imported target makes in convenient to use in the
project and treats it further like any other target.