About a year ago I asked about header dependencies in CMake.
I realized recently that the issue seemed to be that CMake considered those header files to be external to the project. At least, when generating a Code::Blocks project the header files do not appear within the project (the source files do). It therefore seems to me that CMake consider those headers to be external to the project, and does not track them in the depends.
A quick search in the CMake tutorial only pointed to include_directories which does not seem to do what I wish...
What is the proper way to signal to CMake that a particular directory contains headers to be included, and that those headers should be tracked by the generated Makefile?
Two things must be done.
First add the directory to be included:
target_include_directories(test PRIVATE ${YOUR_DIRECTORY})
In case you are stuck with a very old CMake version (2.8.10 or older) without support for target_include_directories, you can also use the legacy include_directories instead:
include_directories(${YOUR_DIRECTORY})
Then you also must add the header files to the list of your source files for the current target, for instance:
set(SOURCES file.cpp file2.cpp ${YOUR_DIRECTORY}/file1.h ${YOUR_DIRECTORY}/file2.h)
add_executable(test ${SOURCES})
This way, the header files will appear as dependencies in the Makefile, and also for example in the generated Visual Studio project, if you generate one.
How to use those header files for several targets:
set(HEADER_FILES ${YOUR_DIRECTORY}/file1.h ${YOUR_DIRECTORY}/file2.h)
add_library(mylib libsrc.cpp ${HEADER_FILES})
target_include_directories(mylib PRIVATE ${YOUR_DIRECTORY})
add_executable(myexec execfile.cpp ${HEADER_FILES})
target_include_directories(myexec PRIVATE ${YOUR_DIRECTORY})
First, you use include_directories() to tell CMake to add the directory as -I to the compilation command line. Second, you list the headers in your add_executable() or add_library() call.
As an example, if your project's sources are in src, and you need headers from include, you could do it like this:
include_directories(include)
add_executable(MyExec
src/main.c
src/other_source.c
include/header1.h
include/header2.h
)
Structure of project
.
├── CMakeLists.txt
├── external //We simulate that code is provided by an "external" library outside of src
│ ├── CMakeLists.txt
│ ├── conversion.cpp
│ ├── conversion.hpp
│ └── README.md
├── src
│ ├── CMakeLists.txt
│ ├── evolution //propagates the system in a time step
│ │ ├── CMakeLists.txt
│ │ ├── evolution.cpp
│ │ └── evolution.hpp
│ ├── initial //produces the initial state
│ │ ├── CMakeLists.txt
│ │ ├── initial.cpp
│ │ └── initial.hpp
│ ├── io //contains a function to print a row
│ │ ├── CMakeLists.txt
│ │ ├── io.cpp
│ │ └── io.hpp
│ ├── main.cpp //the main function
│ └── parser //parses the command-line input
│ ├── CMakeLists.txt
│ ├── parser.cpp
│ └── parser.hpp
└── tests //contains two unit tests using the Catch2 library
├── catch.hpp
├── CMakeLists.txt
└── test.cpp
How to do it
1. The top-level CMakeLists.txt is very similar to Recipe 1, Code reuse with functions and macros
cmake_minimum_required(VERSION 3.5 FATAL_ERROR)
project(recipe-07 LANGUAGES CXX)
set(CMAKE_CXX_STANDARD 11)
set(CMAKE_CXX_EXTENSIONS OFF)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
include(GNUInstallDirs)
set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY
${CMAKE_BINARY_DIR}/${CMAKE_INSTALL_LIBDIR})
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY
${CMAKE_BINARY_DIR}/${CMAKE_INSTALL_LIBDIR})
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY
${CMAKE_BINARY_DIR}/${CMAKE_INSTALL_BINDIR})
# defines targets and sources
add_subdirectory(src)
# contains an "external" library we will link to
add_subdirectory(external)
# enable testing and define tests
enable_testing()
add_subdirectory(tests)
2.Targets and sources are defined in src/CMakeLists.txt (except the conversion target)
add_executable(automata main.cpp)
add_subdirectory(evolution)
add_subdirectory(initial)
add_subdirectory(io)
add_subdirectory(parser)
target_link_libraries(automata
PRIVATE
conversion
evolution
initial
io
parser
)
3.The conversion library is defined in external/CMakeLists.txt
add_library(conversion "")
target_sources(conversion
PRIVATE
${CMAKE_CURRENT_LIST_DIR}/conversion.cpp
PUBLIC
${CMAKE_CURRENT_LIST_DIR}/conversion.hpp
)
target_include_directories(conversion
PUBLIC
${CMAKE_CURRENT_LIST_DIR}
)
4.The src/CMakeLists.txt file adds further subdirectories, which in turn contain CMakeLists.txt files. They are all similar in structure; src/evolution/CMakeLists.txt contains the following:
add_library(evolution "")
target_sources(evolution
PRIVATE
${CMAKE_CURRENT_LIST_DIR}/evolution.cpp
PUBLIC
${CMAKE_CURRENT_LIST_DIR}/evolution.hpp
)
target_include_directories(evolution
PUBLIC
${CMAKE_CURRENT_LIST_DIR}
)
5.The unit tests are registered in tests/CMakeLists.txt
add_executable(cpp_test test.cpp)
target_link_libraries(cpp_test evolution)
add_test(
NAME
test_evolution
COMMAND
$<TARGET_FILE:cpp_test>
)
How to run it
$ mkdir -p build
$ cd build
$ cmake ..
$ cmake --build .
Refer to: https://github.com/sun1211/cmake_with_add_subdirectory
Add include_directories("/your/path/here").
This will be similar to calling gcc with -I/your/path/here/ option.
Make sure you put double quotes around the path. Other people didn't mention that and it made me stuck for 2 days. So this answer is for people who are very new to CMake and very confused.
CMake is more like a script language if comparing it with other ways to create Makefile (e.g. make or qmake). It is not very cool like Python, but still.
There are no such thing like a "proper way" if looking in various opensource projects how people include directories. But there are two ways to do it.
Crude include_directories will append a directory to the current project and all other descendant projects which you will append via a series of add_subdirectory commands. Sometimes people say that such approach is legacy.
A more elegant way is with target_include_directories. It allows to append a directory for a specific project/target without (maybe) unnecessary inheritance or clashing of various include directories. Also allow to perform even a subtle configuration and append one of the following markers for this command.
PRIVATE - use only for this specified build target
PUBLIC - use it for specified target and for targets which links with this project
INTERFACE -- use it only for targets which links with the current project
PS:
Both commands allow to mark a directory as SYSTEM to give a hint that it is not your business that specified directories will contain warnings.
A similar answer is with other pairs of commands target_compile_definitions/add_definitions, target_compile_options/CMAKE_C_FLAGS
I had the same problem.
My project directory was like this:
--project
---Classes
----Application
-----.h and .c files
----OtherFolders
--main.cpp
And what I used to include the files in all those folders:
file(GLOB source_files CONFIGURE_DEPENDS
"*.h"
"*.cpp"
"Classes/*/*.cpp"
"Classes/*/*.h"
)
add_executable(Server ${source_files})
And it totally worked.
You have two options.
The Old:
include_directories(${PATH_TO_DIRECTORY})
and the new
target_include_directories(executable-name PRIVATE ${PATH_TO_DIRECTORY})
To use target_include_directories, You need to have your executable defined - add_executable(executable-name sourcefiles).
So your code should appear like
add_executable(executable-name sourcefiles)
target_include_directories(executable-name PRIVATE ${PATH_TO_DIRECTORY})
You can read more here https://cmake.org/cmake/help/latest/command/target_include_directories.html
This worked for me:
set(SOURCE main.cpp)
add_executable(${PROJECT_NAME} ${SOURCE})
# target_include_directories must be added AFTER add_executable
target_include_directories(${PROJECT_NAME} PUBLIC ${INTERNAL_INCLUDES})
Don't forget to include ${CMAKE_CURRENT_LIST_DIR}.
That's what was causing problems for me.
Example should be like this:
target_include_directories(projectname
PUBLIC "${CMAKE_CURRENT_LIST_DIR}/include"
)
PUBLIC for dependencies which you want to be included by a parent project.
PRIVATE for ones that you don't.
Note to site curators: This answer is very long. In case you are wondering, no it is not from a blog post. I wrote this specifically tailored to answer this question. If you think the length of the answer and its content warrant closing the question as needing focus, then I have no qualms with that. I personally am not a fan of the question anyway, but wanted to give a good answer because it has gotten so much attention over the years and thought the existing answers were lacking in certain ways.
In all the answers to this questions, there is a whole lot of "how" (to get what you want), and precious little "why" (digging into the problem that motivated the question and what the asker may have misunderstood about the ways in which different types of tools like IDEs and build tools do / do not interact and share information with each other, and what information CMake passes / needs to pass to those tools).
This question is vexxing, as it is motivated by a specific behaviour of a specific IDE- Code::Blocks) and CMake, but then poses a question unrelated to that IDE and instead about Makefiles and CMake, assuming that they have done something wrong with CMake which led to a problem with Makefiles, which led to a problem with their IDE.
TL;DR CMake and Makefiles have their own way of tracking header dependencies given include directories and source files. How CMake configures the Code::Blocks IDE is a completely separate story.
What is an "external" header in CMake?
I realized recently that the issue seemed to be that CMake considered those header files to be external to the project. [...]
It therefore seems to me that CMake consider those headers to be external to the project, and does not track them in the depends
As far as I know, there is no official or useful definition of "external header" when it comes to CMake. I have not seen that phrase used in documentation. Also note that the word "project" is a quite overloaded term. Each buildsystem generated by CMake consists of one top-level project, possibly including other external or subdirectory projects. Each project can contain multiple targets (libraries, executables, etc.). What CMake refers to as a target sometimes translates to what IDEs call projects (Ix. Visual Studio, and possibly Code::Blocks). If you had to given such a phrase a meaning, here's what would make sense to me:
In the case that the question is referring to some IDEs' sense of the word "project", which CMake calls "targets", header files are external to a project would be those that aren't intended to be accessed through any of the include directories of a target (Ex. Include directories that come from targets linked to the target in question).
In the case that the question is referring to CMake's sense of the word "project": Targets are either part of a project (defined/created by a call to the project() command, and built by the generated buildsystem), or IMPORTED, (not built by the generated buildsystem and expected to already exist, or built by some custom step added to the generated buildsystem, such as via ExternalProject_Add). Include directories of IMPORTED targets would be those headers which are external to the CMake project in question, and include directories of non-IMPORTED targets would be those that are "part of" the project.
Does CMake track header dependencies? (It depends!)
[...] CMake consider those headers to be external to the project, and does not track them in the depends
I'm not super familiar with the history of CMake, or with header dependency tracking in build tooling, but here is what I've gathered from the searching I have done on the topic.
CMake itself doesn't have much to do with any information related to header/include dependencies of implmentation files / translation units. The only way in which that information is important to CMake is if CMake needs to be the one to tell the generated buildsystem what those dependencies are. It's the generated buildsystem which wants to track changes in header file dependencies to avoid any unnecessary recompilation. For the Unix Makefiles generator in particular, before CMake 3.20, CMake would do the job of scanning header/include dependencies to tell the Makefiles buildsystem about those dependencies. Since v3.20, where supported by the compiler, CMake delegates that resposibility to the compiler by default. See the option which can be used to revert that behaviour here.
The exact details of how header/include dependency scanning differs for each supported CMake generator. For example, you can find some high-level description about the Ninja capabilities/approach on their manual. Since this question is only about Makefiles, I won't attempt to go into detail about other generators.
Notice how to get the header/include dependency information for the buildsystem, you only need to give CMake a list of a target's include directories, and a list of the implementation source files to compile? You don't need to give it a list of header files because that information can be scanned for (either by CMake or by a compiler).
Do IDEs get information about target headers by scanning?
Each IDE can display information in whatever way it wants. Problems like you are having with the IDE not showing headers usually only happen for IDE display formats of the project layout other than the filesystem layout (project headers files are usually in the same project directory as implementation files). For example, such non-filesystem layout views are available in Visual Studio and Code::Blocks.
Each IDE can get header information in whatever way it chooses. As far as I am aware (but I may be wrong for Visual Studio), both Visual Studio and Code::Blocks expect the list of project headers to be explicitly listed in the IDE project configuration files. There are other possible approaches (Ex. header dependency scanning), but it seems that many IDEs choose the explicit list approach. My guess would be because it is simple implementation-wise.
Why would scanning be burdensome for an IDE to find header files associated with a target?(Note: this is somewhat speculation, since I am not a maintainer of any such tools and have only used a couple of them) An IDE could implement the file scanning (which itself is a complicated task), but to know which headers are "in" the target, they'd either need to get information from the buildsystem about how the translation units of the target will get compiled, and that's assuming that all "not-in-target" header include paths are specified with a "system"-like flag, which doesn't have to be the case. Or, it could try to get that information from the meta-buildsystem, which here is CMake. Or it could try to do what CMake now does and try to invoke the selected compiler to scan dependencies. But in either case, they'd have to make some difficult decision about which buildsystems, meta buildsystems, and/or compilers to support, and then do the difficult work of extracting that information from whatever formats those tools store that information in, possibly without any guarantees that those formats will be the same in future tool versions (supporting a change in the format in a newer tool version could be similar to having to supporting a completely separate tool). The IDE could do all that work, or it could just ask you to give it a list of the headers belonging to each target. As you can see, there are cons to the diversity in tooling that the C/C++ ecosystem has. There are pros too, but that's outside the scope of this question.
On the bright side, CMake actually does have a mechanism to try to take some of that work off your shoulders. For such IDEs that have non-filesystem-views, it does implement a simple-heuristic to try to find header files that are associated with source files...
How does header file discovery work for the Code::Block IDE generator for CMake?
At least, when generating a Code::Blocks project the header files do not appear within the project (the source files do).
Here's something interesting: The CodeBlocks editor has the concept of source files and header files that are part of a project, and since CMake doesn't expect/require its users to tell it about each and every header file in the project (it only needs to know about what include directories should be associated with targets), it tries to use a certain heuristic to discover header files that are associated to implementation files. That heuristic is very basic: take the path of each source file in a project, and try changing the extenstion to be like one that is usually given to header files, and see if any such file exists. See the cmExtraCodeBlocksGenerator::CreateNewProjectFile member function in :/Source/cmExtraCodeBlocksGenerator.cxx.
In "Pitchfork Layout" terminology, it would be said that the heuristic assumes that the project uses "merged-header" placement instead of "split-header" placement, where there are separate src/ and include/ directories. So if you don't use merged-header layout, or otherwise have any target headers that don't meet that heuristic, such as utility header files, you'll need to explicitly tell CMake about those files (Ex. using target_sources) for it to pass that knowledge on to the IDE config it generates.
Further readings:
Here's the CMake documentation on its Code::Blocks generator (not much info related to the topic at hand, but good to link anyway).
Here's Code::Blocks' documentation on its "Project View". Here's the .cpb xml schema documentation (see in particular, the Unit element).
If you want to read the CMake code which does the associated header detection, you can find it in the cmExtraCodeBlocksGenerator::CreateNewProjectFile function in the Source/cmExtraCodeBlocksGenerator.cxx file.
Closing Words
I'm certain there are many people who know these tools better than I do. If you are one of those people and notice that I have made a mistake, please graciously correct me in the comments or in chat, or just to edit this post.
Note that while installation of build artifacts is an important part of many projects' lifecycles and is therefore incorporated into the designs of most C/C++ buildsystems, since the question didn't explicitly ask about the configuring the installation part, I have chosen to leave it out of this answer, since it in itself is not a trivial topic to cover (just see how long the related chapters in the "Mastering CMake" book are: The chapter on installation, and the chapter on importing and exporting).
In newer CMake versions we can limit our include-paths to target, like:
target_include_directories(MyApp PRIVATE "${CMAKE_CURRENT_LIST_DIR}/myFolder")
I mean, if the CMakeLists.txt has multiple targets, else, the include-paths are NOT shared with other CMakeLists.txt scripts, and it's enough to do something like:
include_directories("${CMAKE_CURRENT_LIST_DIR}/myFolder")
However, maybe we can simulate what target_include_directories(...) does for CMake 2.8.10 or older versions, like:
set_property(
TARGET MyApp
APPEND PROPERTY
INCLUDE_DIRECTORIES "${CMAKE_CURRENT_LIST_DIR}/myFolder"
)
All done, but seems if you want source-files to be re-compiled once any header-file they use is changed, all such header-files need to be added to each target as well, like:
set(SOURCES src/main.cpp)
set(HEADERS
${CMAKE_CURRENT_LIST_DIR}/myFolder/myHeaderFile.h
${CMAKE_CURRENT_LIST_DIR}/myFolder/myOtherHeader.h
)
add_executable(MyApp ${SOURCES} ${HEADERS})
Where with "seems" I mean that, CMake could detect such header-files automatically if it wanted, because it parses project's C/C++ files anyway.
I am using CLion also my project structure is the following :
--main.cpp
--Class.cpp
--Class.h
--CMakeLists.txt
The CMakeLists.txt before the change:
add_executable(ProjectName main.cpp)
The CMakeLists.txt after the change:
add_executable(ProjectName main.cpp Class.cpp Class.h)
By doing that the program compiled successfully.
Long story short, I rewrote the Godot build system to cmake (only windows part), mostly because I wanted to learn it, but I have trouble compiling Godot with mingw. When I'm trying to compile it, at first everything goes fine, up until the point of linking final exe, where I get a lot of "undefined reference to" errors. It looks like main libraries (core/scene/editor/..) can't see functions from each other. MSVC build is working fine, and scons version is also compiling under mingw, so I clearly just missed something in my cmake version.. I tried to remove some compile/linking options throughout my cmake scripts as a test, but nothing changed. I don't really know how even debug this problem, so if someone could kick me in the right direction I would be really glad for it.
Ok, I finally got time to came back to this.
The problem
So basically the problem was in circular dependency on godot libs. I didn't thought this was the problem because I'm spoiled by MSVC (which doesn't depend on link order). Also, I tried to replicate circular dependency in mingw on a test project with much smaller scale. The project was a 30 libs with two functions in each, first function printing string, and another calling all first functions from all 30 libs, so there is 30 libs of circular dependency. Strangely enough, the project linked no problems, and printed 30^2 strings..
The solution
The solution is to use -Wl,--start-group/-Wl,--end-group linking flags around all libraries. There is two ways you can do it.
First way, is to add all your libraries to the some list of sorts. This could be global property, or property on some target (not just a simple variable), so it could be accessed from other subdirectories. After you formed your list of libraries you simply link it to the executable as follows
# getting all your libs from the global property..
get_property(__LIBS_LIST GLOBAL PROPERTY EXE_LIBS_LIST)
# linking all libraries to the exe..
target_link_libraries(my-exe PRIVATE -Wl,--start-group ${__LIBS_LIST} -Wl,--end-group)
This is the easiest solution, but be cautious about dependencies on libraries which you link to your exe, because it seems that when CMake creates link line for your exe, it first lists all libraries which are linked directly to your exe, and only after it places libraries which came from dependencies of libraries linked directly. Basically if your target dependency tree looks something like this:
exe // your main exe file
- lib_A // lib A linked directly to the main exe
- lib_AA // lib AA linked to the lib_A
- lib_AAA // lib AAA linked to the lib_AA
- lib_B // lib B linked directly to the main exe
- lib_BB // lib BB linked to the lib_B
- lib_BBB // lib BBB linked to the lib_BB
your link order for the exe will look something like this:
// first libs linked directly to the exe
lib_A
lib_B
// only after recursively initial libs dependencies
lib_AA
lib_AAA
lib_BB
lib_BBB
That's meen, that if you will link your libs like target_link_libraries(my-exe PRIVATE -Wl,--start-group ${__LIBS_LIST} -Wl,--end-group), --start-group and --end-group will guard only the libraries linked directly to your exe. I didn't found this described in documentation, but I found SO question which talks pretty much about same behaviour (CMake library linking order). Also, as I tested this on mingw, it didn't mattered how exactly libs lib_AA/lib_AAA/lib_BB/lib_BBB were linked, via PRIVATE or via INTERFACE, the results were the same.
Second way, is to exploit recursivity of link expanding for dependencies of libraries linked directly to the exe. From my example you can see, that dependencies of lib_A (lib_AA/lib_AAA) weren't mixed with dependencies of lib_B (lib_BB/lib_BBB). So basiacaly what we can do, is to create INTERFACE library and connect to it -Wl,--start-group immediately after that. Then add any number of libraries to it's interface and link global-libs to your exe (order does not matter). And in very end, close the group in your global-libs library
add_library(global-libs INTERFACE)
target_link_libraries(global-libs INTERFACE -Wl,--start-group)
# ...
# linking another libs, and linking global-libs to exe
# ...
target_link_libraries(global-libs INTERFACE -Wl,--end-group)
This will ensure, that all libraries connected to global-libs will be surrounded by -Wl,--start-group/-Wl,--end-group.
Now, theoretically, CMake should handle circular dependency by itself, by placing libraries in link line multiple times (how many times controlled by LINK_INTERFACE_MULTIPLICITY). But this method didn't worked for me (mb I just missed something..). Plus, you need declare dependencies between cmake targets, and with -Wl,--start-group/-Wl,--end-group you can just set one specific interface library as a holder for all libs with circular dependencies..
I have a large project which I have divided into "modules" or "subprojects". The directory structure looks like this (simplified, but the basic idea):
project-root/
CMakeLists.txt <-- Contains some variable definitions
module1/
CMakeLists.txt <-- Receives variable definitions from top-level
src/
module2/
CMakeLists.txt <-- Receives variable definitions from top-level
src/
The root CMakeLists.txt does not have any libraries or executables defined. It uses add_subdirectory(module1) and add_subdirectory(module2) to refer to the lower-level modules, which do have libraries and executables.
Now, all this works fine. I'm using CLion and it is able to build the entire project. My problem is when a developer wants to compile only one of the modules, without having to deal with the other modules.
I have created variables in the root-level CMakeLists.txt which are used by the CMakeLists.txt files of the modules. But if I try to compile only one of the modules, of course the variable is undefined (or blank). Is there a standard way of dealing with problems like this? I want to be able to compile each module independently, but I don't want to redefine the variables in every CMakeLists.txt file.
I believe it is exactly what people do when they add a "BUILD_TESTS" option.
Look at grpc for instance.
If you follow the option here, you'll see that if the option is defined, then they add a library (they do a whole bunch of other stuff, too).
So to summarize:
Create an option: option(BUILD_MODULE_1 "Build module 1" ON)
Call add_subdirectory(module1) only if(BUILD_MODULE_1)
In CLion, you can either edit the cache and reload CMake or add the option (e.g. -DBUILD_MODULE_1=OFF) to the "CMake options" in the settings.
I have Rust project with both integration tests (in the /tests dir) and benchmarks (in the /benches dir). There are a couple of utility functions that I need in tests and benches, but they aren't related to my crate itself, so I can't just put them in the /utils dir.
What is idiomatic way to handle this situation?
Create a shared crate (preferred)
As stated in the comments, create a new crate. You don't have to publish the crate to crates.io. Just keep it as a local unpublished crate inside your project and mark it as a development-only dependency.
This is best used with version 2 of the Cargo resolver. For better performance, consider using a Cargo workspace.
.
├── Cargo.toml
├── src
│ └── lib.rs
├── tests
│ └── integration.rs
└── utilities
├── Cargo.toml
└── src
└── lib.rs
Cargo.toml
# ...
[dev-dependencies]
utilities = { path = "utilities" }
utilities/src/lib.rs
pub fn shared_code() {
println!("I am shared code");
}
tests/integration.rs
extern crate utilities;
#[test]
fn a_test() {
utilities::shared_code();
}
A test-only module
You could place a module inside your crate that is only compiled when a specific feature is passed. This is the same concept used for unit tests. This has the advantage that it can access internals of your library code. It has the disadvantage that you need to pass the flag each time you run the code.
This is best used with version 2 of the Cargo resolver.
Cargo.toml
# ...
[features]
test-utilities = []
src/lib.rs
#[cfg(feature = "test-utilities")]
pub mod test_utilities {
pub fn shared_code() {
println!("I'm inside the library")
}
}
tests/integration.rs
extern crate the_library;
#[test]
fn a_test() {
the_library::test_utilities::shared_code();
}
execution
cargo test --features=test-utilities
This is best used with version 2 of the Cargo resolver.
Use a module from an arbitrary file path
This is just ugly to me, and really goes out of the normal path.
utilities.rs
pub fn shared_code() {
println!("This is just sitting out there");
}
tests/integration.rs
#[path = "../utilities.rs"]
mod utilities;
#[test]
fn a_test() {
utilities::shared_code();
}
See also:
Where should I put test utility functions in Rust?
You could add those utility-functions to a pub-module inside your main crate and use the #[doc(hidden)] or #![doc(hidden)] attribute to hide them from the docs-generator. Extra comments will guide the reader to why they are there.
Whilst this doesn't help for benchmarks, I came here looking for a way to do this with multiple integration tests, and later found that you can do the following for integration tests:
Modules with common code follow the ordinary modules rules, so it's ok to create common module as tests/common/mod.rs.
Source: https://doc.rust-lang.org/rust-by-example/testing/integration_testing.html
I have Rust project with both integration tests (in the /tests dir) and benchmarks (in the /benches dir). There are a couple of utility functions that I need in tests and benches, but they aren't related to my crate itself, so I can't just put them in the /utils dir.
What is idiomatic way to handle this situation?
Create a shared crate (preferred)
As stated in the comments, create a new crate. You don't have to publish the crate to crates.io. Just keep it as a local unpublished crate inside your project and mark it as a development-only dependency.
This is best used with version 2 of the Cargo resolver. For better performance, consider using a Cargo workspace.
.
├── Cargo.toml
├── src
│ └── lib.rs
├── tests
│ └── integration.rs
└── utilities
├── Cargo.toml
└── src
└── lib.rs
Cargo.toml
# ...
[dev-dependencies]
utilities = { path = "utilities" }
utilities/src/lib.rs
pub fn shared_code() {
println!("I am shared code");
}
tests/integration.rs
extern crate utilities;
#[test]
fn a_test() {
utilities::shared_code();
}
A test-only module
You could place a module inside your crate that is only compiled when a specific feature is passed. This is the same concept used for unit tests. This has the advantage that it can access internals of your library code. It has the disadvantage that you need to pass the flag each time you run the code.
This is best used with version 2 of the Cargo resolver.
Cargo.toml
# ...
[features]
test-utilities = []
src/lib.rs
#[cfg(feature = "test-utilities")]
pub mod test_utilities {
pub fn shared_code() {
println!("I'm inside the library")
}
}
tests/integration.rs
extern crate the_library;
#[test]
fn a_test() {
the_library::test_utilities::shared_code();
}
execution
cargo test --features=test-utilities
This is best used with version 2 of the Cargo resolver.
Use a module from an arbitrary file path
This is just ugly to me, and really goes out of the normal path.
utilities.rs
pub fn shared_code() {
println!("This is just sitting out there");
}
tests/integration.rs
#[path = "../utilities.rs"]
mod utilities;
#[test]
fn a_test() {
utilities::shared_code();
}
See also:
Where should I put test utility functions in Rust?
You could add those utility-functions to a pub-module inside your main crate and use the #[doc(hidden)] or #![doc(hidden)] attribute to hide them from the docs-generator. Extra comments will guide the reader to why they are there.
Whilst this doesn't help for benchmarks, I came here looking for a way to do this with multiple integration tests, and later found that you can do the following for integration tests:
Modules with common code follow the ordinary modules rules, so it's ok to create common module as tests/common/mod.rs.
Source: https://doc.rust-lang.org/rust-by-example/testing/integration_testing.html