Below is the CMakeLists.txt file, that tries to execute terminal command with wildcard character *.txt. Currently it gives error ls: cannot access '*.txt': No such file or directory. But, when I try to run without the wildcard it runs as expected, ie it lists the files in the current directory.
Is there any way I could use wild cards in execute_process?
cmake_minimum_required(VERSION 3.8)
project(foo)
execute_process(COMMAND ls *.txt
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
RESULT_VARIABLE result
OUTPUT_VARIABLE curr_out
ERROR_VARIABLE curr_out
)
message(STATUS "${result} : ${curr_out}")
Filename expansion one of expansions done by your shell before the command is executed and it causes *.txt to expand to the list of files using special rules.
You have to run the shell to cause *.txt to expand.
COMMAND sh -c "ls *.txt"
Using file(GLOB tmp "*.txt") and passing tmp would be a more cmake-ish way.
Related
I am trying to copy dll files from my bin folder to a different folder. I want to copy files from bin/Debug when building in Debug and from bin/Release when building in Release. This is what I currently use (and which does not work).
file(GLOB library_files_debug ${outputdirectory_root}/Debug/*.dll)
file(GLOB library_files_release ${outputdirectory_root}/Release/*.dll)
add_custom_target(copy_dlls_to_wheel ALL
DEPENDS setup.py
COMMAND ${CMAKE_COMMAND} -E echo "Debug files: $<$<CONFIG:Debug>:${library_files_debug}>"
COMMAND ${CMAKE_COMMAND} -E echo "Release files: $<$<CONFIG:Release>:${library_files_release}>"
COMMAND ${CMAKE_COMMAND} -E echo "Destination dir: ${CMAKE_BINARY_DIR}/python/${PROJECT_NAME}"
COMMAND ${CMAKE_COMMAND} -E copy $<$<CONFIG:Debug>:${library_files_debug}> $<$<CONFIG:Release>:${library_files_release}> ${CMAKE_BINARY_DIR}/python/${PROJECT_NAME}
)
I am running on Windows 10, and use Visual Studio to build. When the above target copy_dlls_to_wheel is built in Debug, the first echo statement prints out the correct dll files, and the second echo is empty. However, no files are copied. Instead I get the error message The system cannot find the path specified.
I have also tried to replace the last line with
COMMAND ${CMAKE_COMMAND} -E copy $<$<CONFIG:Debug>:${library_files_debug}> ${CMAKE_BINARY_DIR}/python/${PROJECT_NAME}
, but I get the same result.
However, when I remove the generator expression, and use
COMMAND ${CMAKE_COMMAND} -E copy ${library_files_debug} ${CMAKE_BINARY_DIR}/python/${PROJECT_NAME}
the files are copied correctly to my output folder. I am pretty confident my generator expression is correct, since I get the expected output from the echo commands. Are generator expressions not supported when using cmake -E copy, or is there something else I am doing wrong?
CMake's command line copy is able to process multiple files when you simply provide a list, which is why this works:
COMMAND ${CMAKE_COMMAND} -E copy ${library_files_debug} ${CMAKE_BINARY_DIR}/python/${PROJECT_NAME}
This is expanded to a space-separated list of files when the copy command is ultimately executed, which is the expected syntax.
cmake.exe -E copy mylib1.dll mylib2.dll /your/binary/dir/python/proj
However, when wrapped in a generator expression, the list will not be interpreted correctly by CMake. While the generator expression will be evaluated correctly, the list will be kept as a semicolon-separated list of files, which is the incorrect syntax:
cmake.exe -E copy "mylib1.dll;mylib2.dll" /your/binary/dir/python/proj
This causes the copy command to fail.
To work-around this issue, you could loop over each DLL file you want to copy, if there aren't too many. Something like this could work:
# Loop through the Debug files.
foreach(cur_file ${library_files_debug})
get_filename_component(file_name ${cur_file} NAME)
add_custom_target(copy_dlls_to_wheel_debug_${file_name} ALL
DEPENDS setup.py
COMMAND ${CMAKE_COMMAND} -E echo "DLL file: ${cur_file}"
COMMAND ${CMAKE_COMMAND} -E echo "Destination dir: ${CMAKE_BINARY_DIR}/python/${PROJECT_NAME}"
COMMAND ${CMAKE_COMMAND} -E copy $<$<CONFIG:Debug>:${cur_file}> ${CMAKE_BINARY_DIR}/python/${PROJECT_NAME}
)
endforeach()
# Loop through the Release files.
foreach(cur_file ${library_files_release})
get_filename_component(file_name ${cur_file} NAME)
add_custom_target(copy_dlls_to_wheel_release_${file_name} ALL
DEPENDS setup.py
COMMAND ${CMAKE_COMMAND} -E echo "DLL file: ${cur_file}"
COMMAND ${CMAKE_COMMAND} -E echo "Destination dir: ${CMAKE_BINARY_DIR}/python/${PROJECT_NAME}"
COMMAND ${CMAKE_COMMAND} -E copy $<$<CONFIG:Release>:${cur_file}> ${CMAKE_BINARY_DIR}/python/${PROJECT_NAME}
)
endforeach()
A quicker solution might be to bundle up your DLLs, using CMake's tar command line utility, copy them, then extract them, as suggested in this answer. CMake's tar command does not seem to accept lists wrapped in generator expressions either, so the list of files to bundle together is written to a file.
file(GLOB library_files_debug ${outputdirectory_root}/Debug/*.dll)
file(GLOB library_files_release ${outputdirectory_root}/Release/*.dll)
# Write the filenames (not full path) of the files to pack to file
set(debug_content "")
set(release_content "")
foreach(lib_file ${library_files_debug})
get_filename_component(file_name ${lib_file} NAME)
set(debug_content "${debug_content}${file_name}\n")
endforeach(lib_file ${library_files_debug})
foreach(lib_file ${library_files_release})
get_filename_component(file_name ${lib_file} NAME)
set(release_content "${release_content}${file_name}\n")
endforeach(lib_file ${library_files_release})
set(filenames_debug ${outputdirectory_root}/debug_files.txt)
set(filenames_release ${outputdirectory_root}/release_files.txt)
file(WRITE ${filenames_debug} ${debug_content})
file(WRITE ${filenames_release} ${release_content})
# Read either the list of debug or release files, and pack files
add_custom_command(
TARGET bdist PRE_BUILD
COMMAND ${CMAKE_COMMAND} -E tar "cfj" ${outputdirectory_root}/temp.tar --files-from="$<IF:$<CONFIG:Debug>,${filenames_debug},${filenames_release}>"
WORKING_DIRECTORY ${outputdirectory_root}/$<CONFIG>)
# Unpack the files in the folder building the python wheel
add_custom_command(
TARGET bdist PRE_BUILD
COMMAND ${CMAKE_COMMAND} -E rename ${outputdirectory_root}/temp.tar temp.tar
COMMAND ${CMAKE_COMMAND} -E tar "xfj" temp.tar
COMMAND ${CMAKE_COMMAND} -E remove temp.tar
WORKING_DIRECTORY ${CMAKE_BINARY_DIR}/python/${PROJECT_NAME})
I have a console application called "foo", which takes a reference text file as input (in.txt) and generates text at standard output (I want to keep this behaviour).
In make (not cmake), I use a test target, which calls foo and redirects the output to a file (out.txt) as follows. Then, I use diff to compare the file out.txt with the expected refernece (ref.txt)
test:
./foo -a test/in.txt > test/out.txt
diff test/out.txt test/ref.txt
This works fine using make. Now my question is; how can I use cmake to create a similar Makefile?
From within a subdrectory called build, I tried
project(foo)
...
add_test(NAME test1 COMMAND ./foo ../test/in.txt > ../test/out.txt)
enable_testing()
Using cmake version 3.5, I get a Makefile without errors, but when I call make test, the test itself fails. It seems the cmake command add_test supports command line arguments, but not the redirection. I tried quotes and escaping witout success. Since I could not pass this part, I didn't try to use diff. I just imagine that I could pack foo and diff in one line using & as you can do with bash. That would be the second step.
Turning my comment into an answer
As #Tsyvarev has stated, CTest commands are not run in a shell's context. But you could just add the shell needed yourself and use e.g. sh as the command to be called with add_test().
I've run some tests with your example code and the following did work successfully:
add_test(NAME test1 COMMAND sh -c "$<TARGET_FILE:foo> ../test/in.txt > ../test/out.txt")
This solution is not platform independent (it depends on sh to be available in the search paths).
So if you want to be more flexible you could do something like:
include(FindUnixCommands)
file(TO_NATIVE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/test/in.txt" _in)
file(TO_NATIVE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/test/out.txt" _out)
if (BASH)
add_test(
NAME test1
COMMAND ${BASH} -c "$<TARGET_FILE:foo> ${_in} > ${_out}"
)
else()
if (WIN32)
add_test(
NAME test1
COMMAND ${CMAKE_COMMAND} -E chdir $<TARGET_FILE_DIR:foo> $ENV{ComSpec} /c "$<TARGET_FILE_NAME:foo> ${_in} > ${_out}"
)
else()
message(FATAL_ERROR "Unknown shell command for ${CMAKE_HOST_SYSTEM_NAME}")
endif()
endif()
Additionally there is the possibility to execute a more platform independent diff with ${CMAKE_COMMAND} -E compare_files <file1> <file2>. So you could simplify your complete makefile based example in CMake with:
add_custom_command(
TARGET foo
POST_BUILD
COMMAND ${CMAKE_COMMAND} -E echo "Running $<TARGET_FILE_NAME:foo> ..."
COMMAND foo in.txt > out.txt
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/test
)
add_test(
NAME test1
COMMAND ${CMAKE_COMMAND} -E compare_files in.txt out.txt
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/test
)
References
Integrate bash test scripts in cmake
CMake: piping commands to executable
cmake: make tests successfully passing part of the build process
They say you cannot:
There is no redirection of output using add_test arguments.
Unlike to commands in add_custom_command, which are executed as a part of makefile receipts (that is, in the context of some shell), tests are executed directly by CTest, without any shell involved. So, shell mechanisms don't work for tests.
You may create wrapper script, which calls program, given as parameter, and performs redirection, futher diff and so on. Then use this script (with appropriate arguments) as a COMMAND for add_test.
I have a console application called "foo", which takes a reference text file as input (in.txt) and generates text at standard output (I want to keep this behaviour).
In make (not cmake), I use a test target, which calls foo and redirects the output to a file (out.txt) as follows. Then, I use diff to compare the file out.txt with the expected refernece (ref.txt)
test:
./foo -a test/in.txt > test/out.txt
diff test/out.txt test/ref.txt
This works fine using make. Now my question is; how can I use cmake to create a similar Makefile?
From within a subdrectory called build, I tried
project(foo)
...
add_test(NAME test1 COMMAND ./foo ../test/in.txt > ../test/out.txt)
enable_testing()
Using cmake version 3.5, I get a Makefile without errors, but when I call make test, the test itself fails. It seems the cmake command add_test supports command line arguments, but not the redirection. I tried quotes and escaping witout success. Since I could not pass this part, I didn't try to use diff. I just imagine that I could pack foo and diff in one line using & as you can do with bash. That would be the second step.
Turning my comment into an answer
As #Tsyvarev has stated, CTest commands are not run in a shell's context. But you could just add the shell needed yourself and use e.g. sh as the command to be called with add_test().
I've run some tests with your example code and the following did work successfully:
add_test(NAME test1 COMMAND sh -c "$<TARGET_FILE:foo> ../test/in.txt > ../test/out.txt")
This solution is not platform independent (it depends on sh to be available in the search paths).
So if you want to be more flexible you could do something like:
include(FindUnixCommands)
file(TO_NATIVE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/test/in.txt" _in)
file(TO_NATIVE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/test/out.txt" _out)
if (BASH)
add_test(
NAME test1
COMMAND ${BASH} -c "$<TARGET_FILE:foo> ${_in} > ${_out}"
)
else()
if (WIN32)
add_test(
NAME test1
COMMAND ${CMAKE_COMMAND} -E chdir $<TARGET_FILE_DIR:foo> $ENV{ComSpec} /c "$<TARGET_FILE_NAME:foo> ${_in} > ${_out}"
)
else()
message(FATAL_ERROR "Unknown shell command for ${CMAKE_HOST_SYSTEM_NAME}")
endif()
endif()
Additionally there is the possibility to execute a more platform independent diff with ${CMAKE_COMMAND} -E compare_files <file1> <file2>. So you could simplify your complete makefile based example in CMake with:
add_custom_command(
TARGET foo
POST_BUILD
COMMAND ${CMAKE_COMMAND} -E echo "Running $<TARGET_FILE_NAME:foo> ..."
COMMAND foo in.txt > out.txt
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/test
)
add_test(
NAME test1
COMMAND ${CMAKE_COMMAND} -E compare_files in.txt out.txt
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/test
)
References
Integrate bash test scripts in cmake
CMake: piping commands to executable
cmake: make tests successfully passing part of the build process
They say you cannot:
There is no redirection of output using add_test arguments.
Unlike to commands in add_custom_command, which are executed as a part of makefile receipts (that is, in the context of some shell), tests are executed directly by CTest, without any shell involved. So, shell mechanisms don't work for tests.
You may create wrapper script, which calls program, given as parameter, and performs redirection, futher diff and so on. Then use this script (with appropriate arguments) as a COMMAND for add_test.
I'm trying to redirect stdout and stderr to the same file using CMake. I'm using the execute_process option in CMake with the ERROR_FILE and OUTPUT_FILE option specified.
I'm successfully capturing the output, but the error is not there. What am I doing wrong?
File CMakeLists.txt
add_test(NAME test${ID}
COMMAND ${CMAKE_COMMAND}
-DEXE=../examples/test${exampleID}
-DID=${ID}
-DARGS=${args}
-P ${CMAKE_CURRENT_SOURCE_DIR}/Tester.cmake
)
File Tester.cmake
separate_arguments( ARGS )
# Run the test
execute_process(
COMMAND "${EXE}" ${ARGS}
ERROR_FILE test${ID}.out
OUTPUT_FILE test${ID}.out
)
Specifying the same file for both OUTPUT_FILE and ERROR_FILE has only recently been added in CMake 3.3. See release notes.
As a work-around for earlier versions, use the options OUTPUT_VARIABLE and ERROR_VARIABLE with the same variable and then write the contents of the variable to the file, e.g.:
execute_process(
COMMAND "${EXE}" ${ARGS}
ERROR_VARIABLE _testOut
OUTPUT_VARIABLE _testOut
)
file (WRITE "test${ID}.out" "${_testOut}")
Imagine the following lines in a CMakeFiles.txt:
add_custom_target( target
cd bin
COMMAND echo "test" > README
)
make target will not work as expected, as it will not modify the file bin/README but rather the file ./README. I found out that, in order to make CMake modify bin/README, I have to write
COMMAND cd bin && echo "test" > README
which is time consuming and blows the CMakeLists up when used multiple times. I want a behavior that is much like the behaviour of shell scripts. How can I achieve this?
Use the WORKING_DIRECTORY directive:
add_custom_target( target
COMMAND echo "test" > README
WORKING_DIRECTORY bin
)
EDIT: Reversed COMMAND and WORKING_DIRECTORY order