I would like to compile OpenSSL automatically on Windows using the ExternalProject features of CMake. In the configuration, I can specify one download URL. But I need multiple additional files which I'd like to be unpacked into the same source directory. So this is the intention:
ExternalProject_Add ( ${CMAKE_PROJECT_NAME}
PREFIX ${CMAKE_COMPILE_ROOT}/3rd/${CMAKE_PROJECT_NAME}
URL //sw-storage/3rd/src/${OPENSSL_NAME}/jom.zip
URL //sw-storage/3rd/src/${OPENSSL_NAME}/nasm-2.14.02-win64.zip
URL //sw-storage/3rd/src/${OPENSSL_NAME}/strawberry-perl-5.30.0.1-64bit.zip
URL //sw-storage/3rd/src/${OPENSSL_NAME}/openssl-1.1.1c.tar.gz
CONFIGURE_COMMAND perl Configure VC-WIN64A no-idea no-mdc2 no-rc5 --prefix=${INSTALL_DIR} --openssldir=${INSTALL_DIR}/ssl
BUILD_IN_SOURCE 1
BUILD_COMMAND jom.exe
INSTALL_COMMAND jom.exe install
This does not seem to be possible. Specifying multiple URL entries leads to the following error message:
CMake Error at C:/build/cmake-3.13.4/share/cmake-3.13/Modules/ExternalProject.cmake:2455 (message):
At least one entry of URL is a path (invalid in a list)
Is there a way to do this in CMake ?
CMake documentation states that when more than one URL is specified, they are fall-backs of each-other.
List of paths and/or URL(s) of the external project’s source. When
more than one URL is given, they are tried in turn until one succeeds.
URL_HASH is specified only once, therefore CMake expects all URLs to point to the same file.
In this case, you would need individual projects, and the DEPENDS option of ExternalProject to control dependencies, or try any of the CMake based OpenSSL builds out there, add it with ExternalProject and check if those take care of the dependencies already.
Related
I am working with a library that nominally stores its internal headers in a directory that is not itself on the include path, although its parent is. Including the intended entry point header ends up failing because it links to other internal headers via quoted include without the directory name.
I do #include <SDL2/SDL.h> which the compiler does find, in /usr/include/SDL2/SDL.h on my system, but it then fails to find "begin_code.h" which is included several layers deeper in SDLs internal header code.
In file included from /usr/include/SDL2/SDL.h:32:
In file included from /usr/include/SDL2/SDL_main.h:25:
In file included from /usr/include/SDL2/SDL_stdinc.h:31:
In file included from /usr/include/SDL2/SDL_config.h:4:
In file included from /usr/include/x86_64-linux-gnu/SDL2/_real_SDL_config.h:33:
/usr/include/x86_64-linux-gnu/SDL2/SDL_platform.h:179:10: fatal error: 'begin_code.h' file not found
#include "begin_code.h"
^~~~~~~~~~~~~~
1 error generated.
Adding -iquote /usr/include/SDL2 manually works in my case, but what about in build environments where the SDL2 headers were downloaded to some local directory? The point of cmake is to work with local configurations that vary, so adding a hard-coded single path based on platform would be dumb. I want some future person who wants to compile my code with the SDL2 headers downloaded to ~/projects/headers/SDL2 to be able to compile after specifying only ~/projects/headers to their include path, for example, so they don't have to deal with SDLs internal issues.
It seems to me that all I need is to iterate on every dir on the -iquote path and, if it contains a directory name SDL2, add that directory to the -iquote. Does cmake make available the (system configuration dependent) -iquote path as an traversable list?
This question is my attempt to rephrase this unasnwered question for clarity.
Edit: I get that cmake is not responsible for fixing the issue, but cmake (or, rather, a CMakeList.txt file in my project) should be capable of working around this SDL bug. Hard-coding the assumed path is only reliable for build systems that install SDL2 headers via some standard package manager. I've never seen a unix dev manually download header files and stick them in the system include path, for fear that they might be overwritten or otherwise conflict with a future install of an official headers package. There are other valid places to put include files, so cmake should be able to search them. Isn't eliminating hard-coded paths half the point of cmake?
CMake doesn't provide a way for a custom iteration over include directories.
Instead, you could reformulate your intentions into the form "find a directory with the given header".
That form is expressed with command find_path, which is a natural way in CMake for search include directories.
E.g. that call:
# Task for CMake: Find a directory with "begin_code.h" header in it.
# Possibly, this is subdirectory 'SDL2' of a "normal" include directory.
find_path(SDL2_INCLUDE_DIR_1 "begin_code.h" PATH_SUFFIXES "SDL2")
will fill the variable SDL2_INCLUDE_DIR_1 with the directory containing the header begin_code.h.
This way works perfectly in case of local installation of SDL2, if that installation is hinted for CMake with CMAKE_PREFIX_PATH variable. For support other hints, e.g. SDL2DIR environment variable, you need to add appropriate PATHS options to your call:
find_path(SDL2_INCLUDE_DIR_1 "begin_code.h" PATH_SUFFIXES "SDL2" PATHS ENV SDL2DIR)
If you feel that SDL2 developers could rename the problematic file, but expect that file to be near the SDL2.h, then you could change the above command to search SDL2.h instead of begin_code.h:
find_path(SDL2_INCLUDE_DIR_1 "SDL2.h" PATH_SUFFIXES "SDL2" PATHS ENV SDL2DIR)
I have a CMake configuration file building two libraries:
a third-party library (here called ThirdPartyLib) containing a real-time OS / board support package from a supplier. It is built outside CMake using the autotools toolchain.
an extended version of the former library (here called ExtendedThirdPartyLib)
Unfortunately, some source code that I need (various tools) are not built in the ordinary build script for (1). Since I don't want to mess with the suppliers build script I want to add another library (2), building the missing files and thus extending the library from the supplier.
I want to able to do something like this in CMakeFiles.txt:
cmake_minimum_required(VERSION 3.2)
project(bsp)
include(ExternalProject)
ExternalProject_Add(
ThirdPartyLib
URL <http://some.url/bsp.tar.bz2
BUILD_COMMAND make -C ../external/ThirdPartyLib/src
)
set_target_properties(ThirdPartyLib PROPERTIES EXCLUDE_FROM_ALL TRUE)
add_library(ExtendedThirdPartyLib
${CMAKE_CURRENT_BINARY_DIR}/some/path/missing_file1.c
${CMAKE_CURRENT_BINARY_DIR}/some/path/missing_file2.c
)
add_dependencies(ExtendedThirdPartyLib ThirdPartyLib)
target_include_directories(ExtendedThirdPartyLib PUBLIC
${CMAKE_CURRENT_BINARY_DIR}/some/path/include
)
target_link_libraries(ExtendedThirdPartyLib ThirdPartyLib)
The problem here is that the path to missing_file1.c and missing_file2.c are not valid when CMake is generating the build files (they are extracted from the tarball from the supplier). CMake exits with an error output saying: "Cannot find source file".
Is there a neat way to make this work? I.e. is it possible to convince CMake that certain non-existant input files will exist when building of the library begins? Or is there any other recommended way to solve this issue?
(I have temporary made local copies of the files I need to build from the suppliers tarball, but that is of course not a good solution. If those files are changed in future versions of the suppliers package and I forget to overwrite my local copies it could be a horrible mess...
Another "solution" would be to create a small makefile outside CMake and use another ExternalProject_Add in the CMakeFiles.txt somehow. But that's not a good solution either, e.g. if compile and linker flags are modified I need to remember to change the makefile too.)
Personally, I dislike the ExternalProject_Add command, because it does way too many things for my taste, but I've digressed.
What if you do something like this, where bar is simulating your ExtendedThirdPartyLib target, since it depends on generated files
cmake_minimum_required(VERSION 3.11)
project(lol C)
set(SOURCES lol.c) # only this file exists
add_library(lol ${SOURCES})
set(FOO_FILES "foo1.c" "foo2.c")
add_custom_command(OUTPUT ${FOO_FILES}
COMMAND ${CMAKE_COMMAND} -E touch ${FOO_FILES}
WORKING_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}"
COMMENT "Creating ${FOO_FILES}"
VERBATIM)
add_custom_target(foo DEPENDS ${FOO_FILES})
add_library(bar ${FOO_FILES})
add_dependencies(bar foo)
target_link_libraries(lol bar)
The whole approach hinges on the fact that the method, where produced/generated files are procured, is explicitly defined via the custom command and associated custom target.
You should modify the custom command to extract the required files (e.g. could even call some external script) from the tarball (which might require downloading with curl or something similar).
I have been trying to build Mozilla RR on a Linux box at work using CMake. We have a slightly eccentric arrangement where shared libraries are stored on network drives in locations like /sw/external/product-name/linux64_g63.dll/. Further, I have built some dependencies for the project in $HOME/sw/. (I am not a sudoer on this box.)
I am rather baffled as how I am supposed to communicate to CMake to look in non-standard directories. So far I have fudged:
PKG_CONFIG_PATH=$HOME/sw/capnproto-0.6.1/lib/pkconfig \
CC=gcc-6.3 CXX=g++-6.3 \
cmake \
-DCMAKE_INSTALL_PREFIX=$HOME/sw/rr-5.1.0 \
-DPYTHON_EXECUTABLE=$HOME/bin/python2 \
-DCMAKE_FIND_ROOT_PATH=$HOME/sw/libseccomp-2.2.3/ \
../src/
Which is obviously not a scalable solution, but it does at least complete the configuration successfully and emit some Makefiles.
If I omit -DCMAKE_FIND_ROOT_PATH=$HOME/sw/libseccomp-2.2.3/, CMake fails, complaining about a missing libseccomp-2.2.3 dependency. But it works if I do have that definition, telling me that CMake understands where the libseccomp-2.2.3 files are and so will properly add the paths to the necessary compiler invocations.
However, make does not succeed, because gcc fails to find a required header file from the libseccomp probject. Examining make VERBOSE=1, I find that CMake hasn't added -I$HOME/sw/libseccomp-2.2.3/include to the gcc invocation.
I feel like this is not the right approach. The other answers I have looked at tell me to modify the CMakeLists.txt file, but surely
that is not going to be scalable across multiple CMake projects, and
for each project, that will need me to maintain a separate CMakeLists.txt file for every platform (Solaris/Linux/Darwin/Cygwin) I build the software on.
Is there a canonical solution to solving this problem? Perhaps a per-site configuration file that will tell CMake how to find libraries and headers, for all projects I build on that site?
Your approach is correct, but cmake is never told to include SECCOMP - see end of this post.
The way cmake can be informed about custom dependency directory depends on how the dependency is searched (i.e. on what is written in CMakeLists.txt).
find_package/find_library/find_path/find_program
If dependency is found with one of above-mentioned commands, custom search directories can be easily added with CMAKE_PREFIX_PATH. There is no need to add full path to include, lib or bin - when package root is added find_-command will check appropriate sub-directories. CMAKE_PREFIX_PATH can be also set with environment variable.
Second option is CMAKE_FIND_ROOT_PATH. Every path added to CMAKE_FIND_ROOT_PATH list treated as separate root directory and is searched before system root directory.
Note that CMAKE_FIND_ROOT_PATH will be ignored by find_-commands with NO_CMAKE_FIND_ROOT_PATH argument.
Following four variables may be used to tune the usage of CMAKE_FIND_ROOT_PATH:
CMAKE_FIND_ROOT_PATH_MODE_PACKAGE
CMAKE_FIND_ROOT_PATH_MODE_INCLUDE
CMAKE_FIND_ROOT_PATH_MODE_LIBRARY
CMAKE_FIND_ROOT_PATH_MODE_PROGRAM
When use of host system default libraries is undesired setting CMAKE_FIND_ROOT_PATH_MODE_INCLUDE and CMAKE_FIND_ROOT_PATH_MODE_LIBRARY to ONLY is a good practice. If dependency library or header is not found in CMAKE_FIND_ROOT_PATH the configuration will fail. If cmake is allowed search system paths too, it is most likely that errors will occur during linking step or even runtime.
See find_package docs for more details.
find_package only
All above applies to find_package command too.
find_package can operate in two modes MODULE and CONFIG.
In MODULE mode cmake uses Find[PackageName].cmake script (module) to find dependent package. CMake comes with large number of modules and custom modules can be added with CMAKE_MODULE_PATH variable. Often find-modules can be informed about custom search paths via environment or cmake variables.
E.g. FindGTest.cmake searches path stored in GTEST_ROOT variable.
If no find module is available, find_package enters CONFIG mode. If a dependency package provides [PackageName]Config.cmake or [LowercasePackageName]-config.cmake cmake can be easily informed about that package with [PackageName]_DIR variable.
Example:
CMakeLists.txt contains:
find_package(Qt5)
FindQt5.cmake is not available, but ~/Qt5/Qt5.8/lib/cmake/Qt5Config.cmake file exists, so add
-DQt5_DIR="${HOME}/Qt5/Qt5.8/lib/cmake"
to cmake call.
pkg-config
CMake can use information provided by external pkg-config tool. It is usually done with pkg_check_modules command. Directory used by pkg-config can be customized with PKG_CONFIG_PATH environment variable. According to cmake documentation instead of setting PKG_CONFIG_PATH, custom .pc-files directories can be added via CMAKE_PREFIX_PATH. If CMake version is pre-3.1, PKG_CONFIG_USE_CMAKE_PREFIX_PATH have to be set to TRUE(ON) to enable this feature.
Methods of customizing dependencies search path is defined by CMakeLists.txt content. There is no universal solution here.
And now back to missing SECCOMP headers...
In CMakeLists.txt SECCOMP header is found with
find_path(SECCOMP NAMES "linux/seccomp.h")
but I cannot find any command telling CMake to use the found header. For example:
target_include_directories(<target_name> ${SECCOMP})
or globally:
include_directories(${SECCOMP})
I belive that CMakeLists.txt should be fixed. It is not a platform dependent solution.
I have VTK6 installed on my Debian machine and it places all its CMake files under
$ ls /usr/lib/cmake/vtk-6.3/
[...]
VTKConfig.cmake
vtkModuleAPI.cmake
[...]
When I do
find_package(VTK)
in another project, it all works out fine. However,
include(vtkModuleAPI)
yields the error
include could not find load file:
vtkModuleAPI
I had always been under the impression that find_package() and include share the same search paths, specifically CMAKE_MODULE_PATH. Apparently that's not correct.
Note that
SET(CMAKE_MODULE_PATH "/usr/lib/cmake/vtk-6.3")
include(vtkModuleAPI)
does work.
Also note that I'm using CMake 3.5, so there no longer is a FindVTK.cmake as it used to be.
What are the default search paths for find_package() and include()? Why is vtkModuleAPI.cmake not found?
There are two modes of find_package, which have many differences:
Module mode tries to locate FindXXX.cmake file. The file is searched under directories listed in CMAKE_MODULE_PATH plus under directory where CMake is installed.
Config mode tries to locate XXXConfig.cmake file. The file is searched under directories listed in CMAKE_PREFIX_PATH and some other, system-specific variables. (Full algorithm see in the documentation, linked at the beginning of the post).
Command include searches modules only under directories in CMAKE_MODULE_PATH and special CMake module directory.
As you can see, command include and command find_package in module mode uses similar search paths. But in your case, VTKConfig.cmake can be searched only in config mode of find_package, which uses completely different search algorithm.
In case of VTK, CMake has shipped FindVTK.cmake file, which is used when you call find_package(VTK). But inside, this script uses find_package(VTK QUIET NO_MODULE).
If this call locates file /usr/lib/cmake/vtk-6.3/VTKConfig.cmake, it executes this script, and the script includes vtkModuleAPI.cmake one.
If your VTKConfig.cmake is not located by CMake, you may help it by setting VTK_DIR variable to /usr/lib/cmake/vtk-6.3/.
[Starting with CMake-3.1, FindVTK.cmake is no longer shipped with CMake, so find_package(VTK) immediately tries to locate VTKConfig.cmake].
In any case, modules in directory /usr/lib/cmake/vtk-6.3/ shouldn't be included directly: this directory is private for VTK.
find_package(VTK) uses FindVTK.cmake (in it's module mode, c.f. docu on find_package()), which is shipped by CMake and (in your case) should be located in /usr/share/cmake/Modules.
After adding /usr/lib/cmake/vtk-6.3 to CMAKE_MODULE_PATH, find_package(VTK) will still use the same FindVTK.cmake module.
In case you want to use another FindVTK.cmake module, prepend the path to that FindVTK.cmake module to CMAKE_MODULE_PATH.
include() will not use a find module and only sees files located in the CMAKE_MODULE_PATH.
I've a set of libraries and executables all with their own CMakeLists.txt. All libraries are building their targets to the same path (for example ../build/bin and ../build/lib)... as well as exporting their header files (../build/inc).
Now I wish to build one build system and let CMake figure out the dependencies (using add_subdirectory and global build settings).
The problem is: all libraries do export their headers to build/inc after they are build (when make install in invoked). When I do a whole system build make install is not invoked until after the end (and everything has build). So for example executable progfoo with target_link_libraries( progfoo onelib ) will fail, because CMake figures out the dependency to onelib (which builds fine), but progfoo fails because it looks for headers in build/inc... which were not exported yet. The same thing in general applies to many libraries.
What is the correct CMake-Way to handle these cases? Thank you!
Install is the final step, the one that should be visible to the user. So when you export binaries and headers you should already have binaries built against headers in their original locations.
From CMake point of view you have only 1 target at a time. For example you can build a Web Server and using as dependencies libcurl and boost::asio. It is very possible (and good) to add dependencies to current target using add_subdirectory, but when you have to add include directories you have to do that on a per dependency basis, it would be convenient in example if each of the dependencies provides already a variable with current absolute path to includes.
In example see this dummy libcurl's CMakeLists.txt that set path to absolute include directory
get_filename_component(_DIR "${CMAKE_CURRENT_LIST_FILE}" PATH)
// export into parent scope libcurl header location
set (LIBCURL_INCLUDES ${_DIR}/include PARENT_SCOPE)
Then you can use it from Web Server's CMakeLists.txt for building and later use the same path again to extract files for installing them where required
add_subdirectory(PATH_TO_LIBCURL_CMAKELISTS)
# add include directories for building web server
include_directories( ${LIBCURL_INCLUDES})
#I assume you are installing headers because final user will need them,
#in case you needed them just for building you are already done here
#without even installing them
#gather headers list
file(GLOB libCurlHeadersList
"${LIBCURL_INCLUDES}/*.h"
"${LIBCURL_INCLUDES}/*.hpp"
)
#install header list
install( FILES
${libCurlHeadersList}
DESTINATION ../build/inc/libcurl
)