The documentation of CMake's add_executable gives the following specification of the command:
add_executable(<name> [WIN32] [MACOSX_BUNDLE]
[EXCLUDE_FROM_ALL]
source1 [source2 ...])
I now have a rather large project with a lot of sources and was wondering if it is possible to add a directory as a parameter for add_executable instead of specifying each source file individually? If not, are there any best practices or recommendations on how to approach this situation? I can't imagine the only way this would work is by adding each source file individually? How would this work for (really) large projects then, this doesn't seem like an elegant approach...
The best practice is indeed to list all files manually.
In particular, the CMake docs warn about using GLOB for this purpose:
We do not recommend using GLOB to collect a list of source files from
your source tree. If no CMakeLists.txt file changes when a source is
added or removed then the generated build system cannot know when to
ask CMake to regenerate.
This point is somewhat controversial, as many developers prefer that the build system just adjusts automatically to newly added files. The price for this automation is an increase in fragility of the build scripts.
You will have to remember to manually re-run CMake whenever files were added or removed. You also have to ensure that the physical layout of the files on disk matches the logical layout of the projects that you want to build. The latter point is arguably the bigger problem here. By decoupling the build system from the files on disk you add an additional safety net, but you have to pay for it with increased build script maintenance costs.
The biggest disadvantage of the explicit approach is imho that if you forget to add a new file to the CMakeLists, you might be wondering over weird linker errors for a while before realizing your mistake. I personally find the maintenance overhead for this approach acceptable. Sure, you will have a lengthy filelist in your build script, but you do not have to touch it that often and the changes will usually be trivial.
Since this point is somewhat controversial, I won't blame you if you want to use a GLOB for your project. Just be aware of the consequences and be prepared that all the cool kids will laugh at you if your build breaks one day because of this.
Related
The CMake doc says about the command file GLOB:
We do not recommend using GLOB to collect a list of source files from your source tree. If no CMakeLists.txt file changes when a source is added or removed then the generated build system cannot know when to ask CMake to regenerate.
Several discussion threads in the web second that globbing source files is evil.
However, to make the build system know that a source has been added or removed, it's sufficient to say
touch CMakeLists.txt
Right?
Then that's less effort than editing CMakeLists.txt to insert or delete a source file name. Nor is it more difficult to remember. So I don't see any good reason to advise against file GLOB.
What's wrong with this argument?
The problem is when you're not alone working on a project.
Let's say project has developer A and B.
A adds a new source file x.c. He doesn't changes CMakeLists.txt and commits after he's finished implementing x.c.
Now B does a git pull, and since there have been no modifications to the CMakeLists.txt, CMake isn't run again and B causes linker errors when compiling, because x.c has not been added to its source files list.
2020 Edit: CMake 3.12 introduces the CONFIGURE_DEPENDS argument to file(GLOB which makes globbing scan for new files: https://cmake.org/cmake/help/v3.12/command/file.html#filesystem
This is however not portable (as Visual Studio or Xcode solutions don't support the feature) so please only use that as a first approximation, else other people can have trouble building your CMake files under their IDE of choice!
It's not inherently evil - it has advantanges and disadvantages, covered relatively well in this answer here on StackOverflow. But if you use it carelessly, you could end up ignoring dependency changes and requiring clean rebuilds of large parts of your codebase.
I'm personally in favor of using it - in smaller projects, or on certain subdirectories in larger ones - to avoid having to enter every file manually into the build files. Edit: My preference has changed and I currently tend to avoid it.
On top of the reasons other people here posted, imho the worst issue with glob is that it can yield DIFFERENT file lists on different platforms. As I see it, that's a bug. In OSX glob ignores case sensitivity and in a ubuntu box it doesn't.
Globbing breaks all code inspection in things like CLion that otherwise understand limited subsets of CMakeLists.txt and do not and never will support globbing as it is unsafe.
Write script to dump the globbed list and paste it in, its very simple, and then CLion can actually find the referenced files and infer them as useful. Maybe even put such script into the tree so that the other devs can either run it without being a moron OR set git hooks to make it happen.
In no case should some random file dropped into some directory ever get automatically linked that's how trojans happen.
Also CLion without context jumping to known definitions and what not, is like hiking barefoot /// why bother.
This question is about the project command and, by extension, what the concept of a project means in cmake. I genuinely don't understand what a project is, and how it differs from a target (which I do understand, I think).
I had a look at the cmake documentation for the project command, and it says that the project command does this:
Set a name, version, and enable languages for the entire project.
It should go without saying that using the word project to define project is less than helpful.
Nowhere on the page does it seem to explain what a project actually is (it goes through some of the things the command does, but doesn't say whether that list is exclusive or not). The cmake.org examples take us through a basic build setup, and while it uses the project keyword it also doesn't explain what it does or means, at least not as far as I can tell.
What is a project? And what does the project command do?
A project logically groups a number of targets (that is, libraries, executables and custom build steps) into a self-contained collection that can be built on its own.
In practice that means, if you have a project command in a CMakeLists.txt, you should be able to run CMake from that file and the generator should produce something that is buildable. In most codebases, you will only have a single project per build.
Note however that you may nest multiple projects. A top-level project may include a subdirectory which is in turn another self-contained project. In this case, the project command introduces additional scoping for certain values. For example, the PROJECT_BINARY_DIR variable will always point to the root binary directory of the current project. Compare this with CMAKE_BINARY_DIR, which always points to the binary directory of the top-level project. Also note that certain generators may generate additional files for projects. For example, the Visual Studio generators will create a .sln solution file for each subproject.
Use sub-projects if your codebase is very complex and you need users to be able to build certain components in isolation. This gives you a very powerful mechanism for structuring the build system. Due to the increased coding and maintenance overhead required to make the several sub-projects truly self-contained, I would advise to only go down that road if you have a real use case for it. Splitting the codebase into different targets should always be the preferred mechanism for structuring the build, while sub-projects should be reserved for those rare cases where you really need to make a subset of targets self-contained.
I'm learning/vetting CMake at the moment as I'm thinking of migrating our code to it. One thing we do a lot of with our current make system is to "subproject" common code files. For example, we have a lot of shared, generic headers (plus some c/cpp files) which get included in every project we create. I want to replicate this in CMake but I don't see an easy way of doing it. To be precise, I want to do something like:
Parent CMakeLists.txt
add_subdirectory(shared_folder shared_build_folder)
#Next line should somehow add in the files reference in the shared_folder
add_executable([specific files for this project] build_folder)
Child CMakeLists.txt (shared_folder)
#Somehow have a list of files here that get added to the parent project
So far I've found various "ways" of doing this, but all seem a little hacky. I'm coming to the conclusion that this is in fact the way I have to do things and CMake isn't really geared towards this style of development. For clarity, most of my solutions involve doing something like creating a variable at the parent level which consists of a list of files. This variable (via some shenanigans) can get "passed" to/from any children, filled in and then when I call add_exectuable I use that variable to add the files.
All my solutions involve quite a few macros/functions and seemingly quite a bit of overhead. Is this something other people have tried? Any clues on the best approach for doing this?
Thanks
Andrew
We were facing the exact same problem and after some time of crying we accepted the CMake-way and it resulted in a better structured project even if it meant to change some parts of our structure.
When using sub-directories the targets are automatically exported throughout the whole project (even in subsequent other add_subdirectory-calls) once the add_subdirectory-statement was processed: sub-projects which contain common code are creating libraries.
There is also the PARENT_SCOPE which you can use to export variables to parent CMakeLists.txt
For "other" things we simulated the FindPackage-mechanism by including .cmake-files into the main CMakeLists.txt with include. In doing so we can provide variables easily, change the include_directories and do other fancy things global to the project.
As there are no dependencies between cmake-variables, we don't use cmake to configure the source (features of the project), but only the build (compiler, includes, libraries...). This split was the key element of our build-system-refactoring.
I have developed a large MSBuild project to build a portion of our solution. There's a lot of things going on-- XML parsing/replacing, Windows services, remote copy, etc. As a result, the file has grown really difficult to manage, despite my best efforts to add decorations in comments.
As a goof, I broke out the main chunks of functionality out into separate files, like "XML.targets", "Services.targets", etc and imported them into the main "Build.proj." The build still worked and I immediately found it to be much more manageable.
However, all the info I have read on the Import feature of MSBuild is that it should be used to import reusable targets, ie those than can be consumed by -any- MSBuild project without any modifications. The separate projects I'm creating here are the opposite-- specific to one project and will break by default if use with anything else unless modified.
So I guess what I'm asking is, even though I can... should I? Is there an inherent danger in using Import strictly for the purpose of organizing a large project? Is there a better way to do this?
Thanks
No, there is no inherent danger. I think it's a good decision to split large project into several .targets files specific to certain operation since it reduces overall complexity. The idea of creating reusable targets means that they should have as little dependencies on the other parts as possible. By analogy you can think of separate .targets files as classes. The less coupled they are - the better. Because modification in one targets file will less likely break the whole process. You can take a peace of paper, draw your targets files as points with your main project in the center and draw all the connections between them. Say if one targets file overrides target from another or expects some properties from it or somehow else depends on it then there is a connection. In the perfect scenario you'll get something like a star.
In short: you should if it reduces complexity.
There are often conflicts in the Xcode project file (Project.xcodeproj/project.pbxproj) when merging branches (I'm using git). Sometimes it's easy, but at times I end up with a corrupt project file and have to revert. In the worst case I have to fix up the project file manually in a second commit (which can be squashed with the previous) by dragging in files etc.
Does anyone have tips for how to handle merge conflicts in big and complex files like the Xcode project file?
EDIT-- Some related questions:
Git and pbxproj
Should I merge .pbxproj files with git using merge=union?
RESOURCES:
http://www.alphaworks.ibm.com/tech/xmldiffmerge
http://www2.informatik.hu-berlin.de/~obecker/XSLT/#merge
http://tdm.berlios.de/3dm/doc/thesis.pdf
http://www.cs.hut.fi/~ctl/3dm/
http://el4j.svn.sourceforge.net/viewvc/el4j/trunk/el4j/framework/modules/xml_merge/
Break your projects up into smaller, more logical libraries/packages. Massive projects are regularly the sign of a bad design, like the object that does way too much or is way too large.
Design for easy rebuilding -- this also helps if you're writing programs which must be built by multiple tools or IDEs. Many of my 'projects' can be reconstructed by adding one directory.
Remove extraneous build phases. Example: I've removed the "Copy Headers" build phase from all projects. Explicitly include the specific files via the include directive.
Use xcconfig files wherever possible. This also reduces the number of changes you must make when updating your builds. xcconfig files define a collection of build settings, and support #include. Of course, you then delete the (majority of) user defined settings from each project and target when you define the xcconfig to use.
For target dependencies: create targets which perform logical operations, rather than physical operations. This is usually a shell script target or aggregate target. For example: "build dependencies", "run all unit tests", "build all", "clean all". then you do not have to maintain every dependency change every step of a way - it's like using references.
Define a common "Source Tree" for your code, and a second for 3rd party sources.
There are external build tools available. This may be an option for you (at least, for some of your targets).
At this point, a xcodeproj will be much simpler. It will require fewer changes, and be very easy to reconstruct. You can go much further with these concepts to further reduce the complexity of your projects and builds.
You might want to try https://github.com/simonwagner/mergepbx/
It is a script that will help you to merge Xcode project files correctly. Note that it is still alpha.
Disclaimer: I am the author of mergepbx.
The best way I have found is to instruct Git to treat the .pbxproj file as a binary. This prevents messy merges.
Add this to your .gitatributes file:
*.pbxproj -crlf -diff -merge
To compare two Xcode projects open open FileMerge (open xcode and select Xcode (from the manu pane) --> Open developer tools --> FileMerge).
now click "left" button and open xcode project main directory.
click "right" button and open xcode project main directory to compare.
Now click "merge" button!
Thats it!
Another option to consider which may help to reduce the number of times you experience the problem. To explain, I'll call the branch that team members' branches come from the "develop" branch.
Have a convention in your team that when the project file is modified, the changes (along with any other changes required to ensure the build integrity) are committed in a separate commit. That commit is then cherry picked onto the develop branch. Other team members who plan to modify the project file in their branch can then either cherry pick into their branch or rebase their branch on the latest develop. This approach requires communication across the team and some discipline. As I said, it won't always be possible; on some projects it might help a lot and on some projects it might not.