I tried SConstruct, Sconstruct, sconstruct, all seems to be OK.
My question:
1. Is it case insensitive that all 'sconstruct' files? I think linux system file name is case sensitive, is it by python that scons is case insensitive regarding with file names?
If there's 'sconstruct' file, scons prints:
$ scons
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
scons: `.' is up to date.
scons: done building targets.
Why it says "Reading SConscript file", not saying "Reading SConstruct file"?
I guess it's in old versions of scons, make file names are "SConscript", right? But I tried to rename "SConstruct" to "SConscript", then it prints out:
$ scons
scons: *** No SConstruct file found.
This is so weird to me, what is a "SConscript" concept?
To answer your question about the file name casing of SConstruct see the man page: http://scons.org/doc/production/HTML/scons-man.html
By default, scons searches for a file named SConstruct, Sconstruct, or
sconstruct (in that order) in the current directory and reads its
configuration from the first file found. An alternate file name may be
specified via the -f option.
For a better understanding of SCons.. reading the users guide is also recommended. In the case of your question about what is SConscript?
http://scons.org/doc/production/HTML/scons-user/ch14.html
14.1. SConscript Files
As we've already seen, the build script at the top of the tree is
called SConstruct. The top-level SConstruct file can use the
SConscript function to include other subsidiary scripts in the build.
These subsidiary scripts can, in turn, use the SConscript function to
include still other scripts in the build. By convention, these
subsidiary scripts are usually named SConscript. For example, a
top-level SConstruct file might arrange for four subsidiary scripts to
be included in the build as follows:
Related
I want a add_custom_target to run a loop over some files to create several symbolic links (and maybe doing more stuff).
My approach (maybe not the best?) has been:
add_custom_target(myTargetNAme-link echo "HERE" # This is printed
COMMAND
for file in /mylocation/*
do
### Create links using $file
done
DEPENDS dependency
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
COMMENT "Create a link to use files in dev env" VERBATIM
)
And I get the following error:
/bin/sh: -c: line 1: syntax error: unexpected end of file
Is this a smart way to do operations over files?
What's the right syntax for a foor loop like this in CMAKE?
EDIT
Following the comment in the question, I'd like to be more precise:
I have a project that produces a .so file containing a python module AND it should come with a couple .py files.
What I really want to do is to create a symbolic link from the .so and .py files to the python installation.
I was reusing some code, these targets were called in a script using:
nice -n 11 ninja myTargetNAme-link
Maybe I could avoid doing this as well?
ALSO
I deleted the VERBATIM keyword and it made it possible to use *.py for ln so now the symbolic link are actually generated.
Is it possible to debug find_library from CMake?
What I want is a list of considered paths. My use case is a call like
find_library (FOO_LIBRARY
NAMES foo foo.so.0)
and there is /lib64/libfoo.so.0 on my system. However CMake does not find it. I checked that FIND_LIBRARY_USE_LIB64_PATHS is set to TRUE.
With CMake 3.17 this got added:
The “CMAKE_FIND_DEBUG_MODE” variable was introduced to print extra
find call information during the cmake run to standard error. Output
is designed for human consumption and not for parsing.
So you pass either -DCMAKE_FIND_DEBUG_MODE=ON or --debug-find to your CMake command.
Here is an example output when searching for libFOO:
find_library considered the following locations:
/usr/local/lib64/(lib)FOO(\.so|\.a)
/usr/local/lib/(lib)FOO(\.so|\.a)
/usr/local/lib64/(lib)FOO(\.so|\.a)
/usr/local/lib/(lib)FOO(\.so|\.a)
/usr/local/lib64/(lib)FOO(\.so|\.a)
/usr/local/(lib)FOO(\.so|\.a)
/usr/lib64/(lib)FOO(\.so|\.a)
/usr/lib/(lib)FOO(\.so|\.a)
/usr/lib64/(lib)FOO(\.so|\.a)
/usr/lib/(lib)FOO(\.so|\.a)
/usr/lib64/(lib)FOO(\.so|\.a)
/usr/(lib)FOO(\.so|\.a)
/lib64/(lib)FOO(\.so|\.a)
/lib/(lib)FOO(\.so|\.a)
/opt/(lib)FOO(\.so|\.a)
The item was not found.
I know this isn't a complete answer, but I had the same problem and found that it was necessary to add debug logging to find_library in the CMake source code. I submitted a pull request (work in progress) to add this to mainstream CMake.
I eliminated some possible sources of errors by logging an error message with some relevant details if find_library fails, like this:
set(libssl_names
ssl${_OPENSSL_MSVC_ARCH_SUFFIX}${_OPENSSL_MSVC_RT_MODE}
ssl${_OPENSSL_MSVC_RT_MODE}
ssl
ssleay32${_OPENSSL_MSVC_RT_MODE}
ssleay32
)
find_library(SSL_EAY_DEBUG
NAMES ${libssl_names}
NAMES_PER_DIR
PATHS ${OPENSSL_ROOT_DIR}
PATH_SUFFIXES ${_OPENSSL_PATH_SUFFIXES}
NO_DEFAULT_PATH
)
if(NOT SSL_EAY_DEBUG)
message(FATAL_ERROR "OPENSSL_ROOT_DIR is set to '${OPENSSL_ROOT_DIR}', but did not find any file matching ${OPENSSL_ROOT_DIR}/{${_OPENSSL_PATH_SUFFIXES}}/${CMAKE_FIND_LIBRARY_PREFIXES}{${libssl_names}}${CMAKE_FIND_LIBRARY_SUFFIXES}")
endif()
Which outputs something like:
CMake Error at CMakeLists.txt:526 (message):
OPENSSL_ROOT_DIR is set to '../../Install/openssl/', but did not find any
file matching
../../Install/openssl//{lib/VC/static;VC/static;lib}/lib{ssl64MT;sslMT;ssl;ssleay32MT;ssleay32}.a
Apart from semicolons (instead of commas) in the {braced} portions of the above pattern, and regexes if multiple CMAKE_FIND_LIBRARY_SUFFIXES are configured (e.g. .lib .a on Windows), this is the correct form for shell expansion to a list of paths, which you can pass to ls to check for their existence:
$ ls ../../Install/openssl//{lib/VC/static,VC/static,lib}/lib{ssl64MT,sslMT,ssl,ssleay32MT,ssleay32}.a
ls: ../../Install/openssl//VC/static/libssl.a: No such file or directory
ls: ../../Install/openssl//VC/static/libssl64MT.a: No such file or directory
ls: ../../Install/openssl//VC/static/libsslMT.a: No such file or directory
ls: ../../Install/openssl//VC/static/libssleay32.a: No such file or directory
ls: ../../Install/openssl//VC/static/libssleay32MT.a: No such file or directory
ls: ../../Install/openssl//lib/VC/static/libssl.a: No such file or directory
ls: ../../Install/openssl//lib/VC/static/libssl64MT.a: No such file or directory
ls: ../../Install/openssl//lib/VC/static/libsslMT.a: No such file or directory
ls: ../../Install/openssl//lib/VC/static/libssleay32.a: No such file or directory
ls: ../../Install/openssl//lib/VC/static/libssleay32MT.a: No such file or directory
ls: ../../Install/openssl//lib/libssl64MT.a: No such file or directory
ls: ../../Install/openssl//lib/libsslMT.a: No such file or directory
ls: ../../Install/openssl//lib/libssleay32.a: No such file or directory
ls: ../../Install/openssl//lib/libssleay32MT.a: No such file or directory
../../Install/openssl//lib/libssl.a
It's not obvious (at least to me) that:
relative paths (like ../../Install above) are actually relative to the original source directory (not the current project build directory, which CMake calls CMAKE_BINARY_DIR. (So you should run ls from there, not your build directory).
FIND_LIBRARY_USE_LIB64_PATHS, which is often ON by default, results in your original paths being replaced by mangled ones (lib -> lib64) (not just supplemented by additional search paths, but completely replaced).
Other, even less well-known properties such as FIND_LIBRARY_USE_LIB32_PATHS result in similar mangling (lib -> lib32).
This mangling can be disabled with set(CMAKE_FIND_LIBRARY_CUSTOM_LIB_SUFFIX "").
While creating a RPM package containing a man page (which installation path matches a list of known man locations), CPack seems to compress it using GZip, but then complains that the original, uncompressed file cannot be found. How should that functionality be used then?
Consider the following CMakeLists.txt project:
install(FILES test.1 DESTINATION /usr/share/man)
install(FILES test.2 DESTINATION /usr/share/man/man1)
set(CPACK_PACKAGE_NAME "CPackRPM_man_test")
set(CPACK_GENERATOR "RPM")
include(CPack)
When the line containing “test.2” is commented out, make package operation succeeds, — that is, simply packaging a file that is not destined to a true man page location does not cause any trouble. However, when the full project is processed, the following error message is output:
error: File not found: …/_CPack_Packages/Linux/RPM/CPackRPM_man_test-0.1.1-Linux/usr/share/man/man1/test.2
Indeed, that file simply does not exist:
$ cd …/_CPack_Packages/Linux/RPM/CPackRPM_man_test-0.1.1-Linux/usr/share/man
$ ls *
test.1
man1:
test.2.gz
Worth to note is that DEB generator does not have that issue, — simply because CPackDEB operates on original files only. While looking into CPackRPM.cmake module, I was able to locate the code that messes with man page files, but not the code responsible for undoing the mess-up; the former code works unconditionally, — I could not spot any variable that may tell it not to compress man pages.
The only similar discussion I could find dates back to 2014 and was resolved by updating to a more current version of CMake. Since I am also using openSUSE Linux as the original reporter did (the actual release, of course), I tried to switch from the system-provided CMake 3.3.2 to a locally built 3.7.0-rc2, but that did not change a thing. Therefore, taking into account the popularity of CMake, RPM and man, I suppose that if some bug really existed, it should have been fixed long ago, so the one to blame is me. What am I missing here?
Update. Regarding the use of wildcards that was recommended, — i. e. specifying “test.2*” instead of just “test.2”. CMake does not seem to support globbing in install(FILES …) form: it literally searches for test.2* and fails before generating RPM spec file. Several kinds of pattern matching are supported through another form however:
install(DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR} DESTINATION /usr/share/man/man1 FILES_MATCHING PATTERN *.1*)
install(DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR} DESTINATION /usr/share/man/man2 FILES_MATCHING REGEX "^.+[.]2([.].+)?$")
Although that succeeds, it still does not answer the question of distributing man pages in uncompressed form. Therefore, it looks ugly and counterintuitive.
Worth to mention is how RPM spec file is generated from the originally posted CMakeLists.txt:
%dir "/usr/share/man"
%dir "/usr/share/man/man1"
"/usr/share/man/man1/test.2*"
%config "/usr/share/man/test.1"
%config "/usr/share/man/man1/test.2"
As can be clearly seen, CPackRPM indeed adds an extra wildcarded version of source file name when it belongs to a known man location; unfortunately, it still retains the original (uncompressed) file name, which leads to failure. Perhaps CMake should either warn the user of consequences or even fail early at configuration stage rather than concealing the problem for some later time.
Using a wild card after a man page entry in %files will package the file independent of compression. All I'm saying is to type
/usr/share/man/man1/bash.1*
instead of
/usr/share/man/man1/bash.1.gz
in the %files section of a spec file.
(apologies: cross-posted from CMake mailing list)
I'm trying to get my head round CMake's regex implementation; I have a folder containing 4 folders and 2 text files as follows:
build/
projectA/
CMakeLists.txt
extrafiles/
README
temp/
One line of CMakeLists.txt is:
set(CPACK_SOURCE_IGNORE_FILES "[^projectA]$")
In my source package that is then subsequently generated, build/, projectA/ and extrafiles are present, but temp/ and the 2 text files are not. I'm trying to get to a stage where the regex will ignore everything in the folder except for projectA/, README and CMakeLists.txt, but can't work out at the moment how the regex I've
supplied is giving those results.
I guess what this boils down to is how to match a whole string using regex. I realise that the docs say Matches any character(s) not inside the brackets which is where I guess I'm going wrong...
Further exploration
In trying to understand CMake's regex implementation, I thought I'd start from 1st principles and do some easy stuff.
If I do
set(CPACK_SOURCE_IGNORE_FILES projectA)
then the folder projectA doesn't appear in my source package (as expected); however, if I do
set(CPACK_SOURCE_IGNORE_FILES ^projectA$)
or
set(CPACK_SOURCE_IGNORE_FILES ^/projectA/$)
then projectA does appear. What is it about the ^ (beginning of line) and $ (end of line) that I'm not understanding?
Even more
As probably obvious, projectA is not actually the name of my project, but everything above holds true when I physically rename my project folder to projectA. But, when I replace
set(CPACK_SOURCE_IGNORE_FILES projectA)
with
set(CPACK_SOURCE_IGNORE_FILES <name of my project>)
and rename my actual project folder from projectA to its actual name, I end up with an empty tarball! Argh! I have absolutely no idea what strange tricks CMake is playing on me, but I just want to cry.
Any insight will be greatly appreciated!
SELF CONTAINED EXAMPLE
As requested by Fraser, a self contained example showing 2 of the 'features' I've described. However, I do know that I'm running CMake in a slightly non-standard way, in order to keep everything to do with individual builds together, so if there's any proof running CMake in a more standard way eliminates these problems I'd be interested to see them.
Step 1: creating files
Create tree:
cd ~
mkdir
cd projectA
mkdir projectA
Create C file, and save it as ~/projectA/projectA/helloworld.c:
#include <stdio.h>
#include <stdlib.h>
int main(void) {
printf("!!!Hello World!!!\n"); /* prints !!!Hello World!!! */
printf("!!!Hello CMake!!!\n"); /* prints !!!Hello CMake!!! */
return 0;
}
create a file that won't need compiling, and save it as ~/projectA/test.sh:
#A non compiled program
echo "Hello world!"
create ~/projectA/CMakeLists.txt:
cmake_minimum_required (VERSION 2.6)
project (HelloWorld)
set(CMAKE_INSTALL_PREFIX "$ENV{HOME}/projectAinstall")
add_executable(helloworld projectA/helloworld.c)
install(TARGETS helloworld DESTINATION .)
include(InstallRequiredSystemLibraries)
set(CPACK_GENERATOR "TGZ")
set(CPACK_SOURCE_GENERATOR "TGZ")
include(CPack)
Step 2: compiling
In ~/projectA, run:
chris#chris:~/projectA$ cmake -H. -Bbuild
then:
make -C build && make -C build package && make -C build package_source
this results in 2 tarballs in the build folder. Moving these somewhere else and untarring them shows helloworld in the binary tarball (as expected), and everything from the ~/projectA/projectA in the source tarball, including test.sh which won't get compiled (which Fraser seemed surprised about)
Step 3: random tests
Modifying CMakeLists.txt to include
set(CPACK_SOURCE_IGNORE_FILES "projectA")
and rerunning the CMake / Make commands above results in an empty source tarball, but with the same binary tarball as above. I have now realised that changing the directory tree so that the top level directory is testproject (and so different to its child folder) doesn't result in an empty source tarball, and does only remove the files listed in CPACK_SOURCE_IGNORE_FILES
I don't think you can achieve what you're after using CPACK_SOURCE_IGNORE_FILES (although I'm not certain). As you rightly noted, CMake's regex handling allows for excluding groups of characters, but I don't think it allows for negating whole patterns. [See updated answer at the end of the edits.]
That being said, I guess you can list all the folders you wish to exclude in your install command. Not as robust as excluding everything except "projectA", but still here's the syntax:
install(DIRECTORY .
DESTINATION the_install_subdir
REGEX "build|extrafiles|temp+" EXCLUDE)
Regarding the empty tarball, I imagine that you maybe have <name of my project> both as your project's root dir and as a subdir? So in your example, if you called your project "projectA", then you'd have "projectA/build", "projectA/projectA", etc.
If so, the regex will work on the full path, and hence all files within your project will contain projectA/ within their paths.
As for the crying... well, I can only advise you to get a grip and pull yourself together! :-)
Edit: In response to the comments, here's a quick example of using the install command to achieve the goal:
install(DIRECTORY projectA
DESTINATION the_install_subdir)
install(FILES CMakeLists.txt README DESTINATION the_install_subdir)
Further Edit:
OK, your example helps a lot - I had indeed misunderstood what you were doing. I hadn't picked up that you were actually making 2 different targets ("package" and "package_source"). I had thought you were creating the binary package by doing something like
cpack -G DEB
and that you were creating the other package by doing
cpack -G TGZ
These both build the binary package. My mistake - I should have paid more attention. Sorry!
As for your specific questions:
Question 1
It seems to me that installing files / directories that aren't compiled but are at the same level as the folder containing all the compiled files (i.e. bin), and then ignoring the bin folder using CPACK_SOURCE_IGNORE_FILES results in an empty tarball - is this correct?
I take this to mean: "Should doing set(CPACK_SOURCE_IGNORE_FILES "${CMAKE_BINARY_DIR}") result in an empty tarball?" The answer is probably not.
Because CPACK_SOURCE_IGNORE_FILES represents a regex, I'm sure there are cases where the resultant regex could match every file in the project, and this would cause an empty tarball. However I imagine it's fairly unlikely.
If, rather than using the full path to your bin dir via the variable ${CMAKE_BINARY_DIR} you were to just give the folder name, there would be a much greater chance of an empty tarball. Say you call your bin dir "build" and have set(CPACK_SOURCE_IGNORE_FILES "build"). If your project lived in say ~/test_builds/projectA, then the regex "build" would match every file in the project since each contains "test_builds"; resulting in an empty tarball.
I think this is the crux of issue each time you've generated an empty tarball. Whatever the regex is trying to achieve, it actually ends up matching and excluding all files.
Question 2
It also seems that files in the CMAKE_SOURCE_DIR which aren't 'installed' don't end up in the binary tarball but do end up in the source tarball
Yes, the "package_source" is indeed a different target to the binary package. It by default contains all files in the ${CMAKE_SOURCE_DIR}, whereas the "package" target contains only items added via install commands. Here, the term "source files" is probably a slight misnomer since it means all files in the source tree - not just .c, .cc, .cxx, etc.
Original Question
I think there's a reasonably safe way to achieve your original aim after all! If you use file(GLOB ...) to generate a non-recursive list of all files/folders in your root, then remove those you wish to keep in the source package, you should be able to use the remaining list as the regex value of CPACK_SOURCE_IGNORE_FILES:
file(GLOB SourceIgnoreFiles "${CMAKE_SOURCE_DIR}/*")
set(SourceKeepFiles "${CMAKE_SOURCE_DIR}/projectA"
"${CMAKE_SOURCE_DIR}/CMakeLists.txt"
"${CMAKE_SOURCE_DIR}/README")
list(REMOVE_ITEM SourceIgnoreFiles ${SourceKeepFiles})
# Escape any '.' characters
string(REPLACE "." "\\\\." SourceIgnoreFiles "${SourceIgnoreFiles}")
set(CPACK_SOURCE_IGNORE_FILES "${SourceIgnoreFiles}")
Hopefully this should now work for you. Sorry again for the misdirections.
CMake tends to use absolute paths except in contexts where there's a strong argument for using relative paths. So I'm pretty sure it's running each regex in CPACK_SOURCE_IGNORE_FILES against absolute paths of files (which should answer your question "What is it about the ^ (beginning of line) and $ (end of line) that I'm not understanding?"). Anything that isn't matched by any regex in CPACK_SOURCE_IGNORE_FILES is not ignored.
What you want is probably something like:
set(CPACK_SOURCE_IGNORE_FILES
/build/
/extrafiles/
/temp/
)
I want to add custom targets with cmake but, some of them must be "silent", because it isn't neccesary. For example, for clean custom commands:
// In CMakeLists.txt
add_custom_target(clean-temporaries
${CMAKE_COMMAND} -P clean-temporaries.cmake
COMMENT "Deleting temporary files"
)
// clean-temporaries.cmake
file(GLOB_RECURSE temporary_files "*[~#]")
file(REMOVE ${temporary_files})
$ cmake .
$ make clean-temporals
[100%] Deleting temporary files
[100%] Built target clean-temporaries
$ make clean
$
We can see that CMake prepares "make clean" to not show messages, but, how can I say to CMake I don't want messages in a custom target?
Try adding a minus at the beginning of the command you want to hide from the console.
-make clean
To deal with temporary files littering your source tree:
Encourage contributors to configure their editors so that temporary files end up in a common directory under their $HOME (eg: vim, emacs).
Encourage contributors to configure their global version control ignore files to always ignore the temporary files for their own work environment (eg. for git: vim, emacs).
Additionally exclude well known temporary file patterns in the version control's ignore file of each project, to be friendly to contributors who haven't yet implemented the two previous steps.
If you do that, it's likely that you don't have to put an additional 'optional' (ie. highly environment specific) step into your build system and you end up with a more generally applicable solution to the problem.
As an additional comment on your example code, I'd avoid building in the source tree and use out-of-source builds instead.