CMake documantation seems clear: unless NO_DEFAULT_PATH is specified, cmake proceeds to first search a bunch of default paths, and if the library is not found, only then it looks in the directories listed in PATHS. A few other NO_* exist, to only exclude some of cmake default paths.
I know how to overcome this behavior. E.g., I can do what's suggested here, namely, do two searches: the first with NO_DEFAULT_PATH, and the second without it. If the first succeeds, the second will be skipped. Win.
However, this question is not about how to achieve what I want (I know the hack). The question is why do I need to do that? Why doesn't cmake look first in the provided paths? It seems logic to me to use those first: if the user is specifying some hints, perhaps I should use those first, and fall back on defaults if they don't work...
Is there a good reason I am missing for the current implementation? I would assume so, I doubt folks down at cmake do things without a good reason...
Edit: when I said 'the user is specifying some hints', I was not referring to the actual HINTS argument of find_library. It was more of a generic 'hints'. However, as suggested in a comment, HINTS is indeed scanned before system paths. My concern comes from cmake's documentation:
Search the paths specified by the HINTS option. These should be paths
computed by system introspection, such as a hint provided by the
location of another item already found. Hard-coded guesses should be
specified with the PATHS option.
The fact that HINTS was not designed to receive hard coded full paths, makes me thing it may fire back in the future, so I'm not sure it is the solution. But maybe it is?
Related
here is a complex problem that I am struggling to find a clean solution for:
Imagine having a Snakemake workflow with several rules that can be parameterized in some way. Now, we might want to test different parameter settings for some rules, to see how the results differ. However, ideally, if these rules depend on the output of other rules that are not parameterized, we want to re-use these non-changing files, instead of re-computing them for each of our parameter settings. Furthermore, if at all possible, all this should be optional, so that in the default case, a user does not see any of this.
There is inherent complexity in there (to specify which files are re-used, etc). I am also aware that this is not exactly the intended use case of Snakemake ("reproducible workflows"), but is more of a meta-feature for experimentation.
Here are some approaches:
Naive solution: Add wildcards for each possible parameter to the file paths. This gets ugly, hard to maintain, and hard to extend really quickly. Not a solution.
A nice approach might be to name each run, and have an individual config file for that name which contains all settings that we need. Then, we only need a wildcard for such a named set of parameter settings. That would probably require to read some table of meta-config file, and process that. That doesn't solve the re-use issue though. Also, that means we need multiple config files for one snakemake call, and it seems that this is not possible (they would instead update each other, but not considered as individual configs to be run separately).
Somehow use sub-workflows, by specifying individual config files each time, e.g., via a wildcard. Not sure that this can be done (e.g., configfile: path/to/{config_name}.yaml). Still not a solution for file re-using.
Quick-and-dirty: Run all the rules up to the last output file that is shared between different configurations. Then, manually (or with some extra script) create directories with symlinks to this "base" run, with individual config files that specify the parameters for the per-config-runs. This still necessitates to call snakemake individually for each of these directories, making cluster usage harder.
None of these solve all issues though. Any ideas appreciated!
Thanks in advance, all the best
Lucas
Snakemake now offers the Paramspace helper to solve this! https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html?highlight=parameter#parameter-space-exploration
I have not tried it yet, but it seems like the solution to the issue!
I'm using add_custom_command() to generate some files. ninja clean removes them, as it should. One of the files is intended as a default/example implementation, to be modified by the user. It is only generated if it does not already exist. I would like for ninja clean not to remove this file.
I have tried a number of things but without success:
add_custom_target(): CMake complains about the missing file unless I name it in BYPRODUCTS, but doing this also leads to removal on clean
set_file_properties(... GENERATED FALSE) doesn't work because CMake complains about the file missing.
set_directory_properties() failed in a similar way: "folder doesn't exist or not yet processed" (it does exist)
I previously generated the example implementation and just let the user copy it or model their code on it. This works, but isn't entirely satisfactory. Is my use-case so unlikely that CMake doesn't support it?
I am afraid you requirment (conceptually, have make create something which make clean does not remove) is rather unusual. I can think of two potential solutions/workarounds.
One, move the file's generation to CMake time. That is, create it using execute_process() instead of add_custom_command(). This may or may not be possible, based on whether the file-generation process (the current custom command) depends on the rest of the build or not.
Two, totally hide the example file's existence from CMake. That is, have the custom command also generate some other file (maybe just a timestamp file) and have its driving custom target depend on that one instead. Do not list the example file as ither the custom command's dependency, output, or byproduct. That way, nothing will depend on it and neither CMake nor Ninja should not care whether it exists or not, so they will not complain or try to clean it up.
If it is an example for the user, it should not be in your build folder, but in the install folder. I don't see why you would need add_custom_command or the other commands you listed.
Therefore, you have to provide install() instructions.
You can then call make install. Cleaning will not remove those and only installing again will overwrite them if necessary.
For those, who come here a long time after the original question was asked (like me), I'll write my solution:
The tool called in add_custom_command generates two files with identical content:
one that is saved in sources, never mentioned anywhere
and one that's marked as byproduct, and then is depended on
So the first one is the file we wanted in the first place.
And the second one is actually used in build process, and gets deleted on clean.
For me the issue is that I actually want to save generated files in VCS so I can track changes. And this approach gives ne what I need.
Considering all freedom that yocto gives to the developer, I have a question.
I would like to make this my_file.inc available only for recipes in one particular meta-layer. I know, that, for instance, using INHERIT keyword inside the local.conf will make my_class.bbclass file available globally for each recipe.
Is it a good practice to add this:
require my_file.inc
inside layer.conf? Or should I change my_file.inc to the my_file.bbclass, and, add INHERIT = "my_file.bbclass" to the layer.conf?
Any other possibilities?
Even if it seems to work, neither of your approaches is technically completely correct. The key point is that all .conf files are parsed first and everything they contain is globally visible throughout the whole build process. So if you add something through the layer.conf file, itis not being pulled in through an unexpected place, it also is not being limited that layer only and might therefore cause breakage at other places.
While I do not have a really good and clean solution, maybe the following can help you:
You can make your custom recipes react on certain keywords in DISTRO_FEATURES or MACHINE_FEATURES. Then you can create a two-staged approach:
Add the desired keyword in local.conf (or your MACHINE, or DISTRO, or whatever configuration)
Make the recipes react to it. If you need the mechanism in several places, then it might be useful to pour it into a .bbclass that your layer brings along and that you pull in for the respective recipes.
This way the effect is properly contained.
Maybe part 5.1.3.2 from the Yocto Project answers your question:
Avoid duplicating include files. Use append files (.bbappend) for each recipe that uses an include file. Or, if you are introducing a new recipe that requires the included file, use the path relative to the original layer directory to refer to the file. For example, use require recipes-core/package/file.inc instead of require file.inc. If you're finding you have to overlay the include file, it could indicate a deficiency in the include file in the layer to which it originally belongs. If this is the case, you should try to address that deficiency instead of overlaying the include file. For example, you could address this by getting the maintainer of the include file to add a variable or variables to make it easy to override the parts needing to be overridden.
So to avoid duplicate inclusion later, it would be better not to include your .inc file(s) this way.
I'm converting a project from Autotools to CMake.
We have a configure.ac file, with the statement:
AC_CHECK_LIB([gsuffix], [gsuffix_create], [], AC_MSG_ERROR([Can not find gsuffix library]))
I want to replace it to cmake, and not sure how (it doesn't have pkg-config)
What I need is:
check libgsuffix exists and find path.
Check gsuffix_create exists in libgsuffix
Add -lgsuffix to compilation - this I think I know how to do.
Can anyone point me to the right direction?
There is no one to one translation. In general, with CMake you don't check whether every header and every library actually works. If you find the file, you assume it will do the trick.
Maybe find_package is right for you, depending someone already wrote such a test or your library provides an according config file.
Find_library is meant to find libraries, but by name.
If you really have to check that your library works, use CheckLibraryExists.
As mentioned, did not find anyway to do this with one command.
I didn't have enough time to understand how to do something completely generic, so I can give my skeleton for a FindGSUFFIX.cmake:
FIND_LIBRARY(GSUFFIX_LIBRARY NAMES gsuffix)
INCLUDE(CheckLibraryExists)
CHECK_LIBRARY_EXISTS(gsuffix gsuffix_create ${GSUFFIX_LIBRARY} GSUFFIX_VERIFIED)
# Handle the QUIETLY and REQUIRED arguments and set PCRE_FOUND to TRUE if all listed variables are TRUE.
INCLUDE(FindPackageHandleStandardArgs)
FIND_PACKAGE_HANDLE_STANDARD_ARGS(GSUFFIX DEFAULT_MSG GSUFFIX_LIBRARY GSUFFIX_VERIFIED)
MARK_AS_ADVANCED(GSUFFIX_LIBRARY)
I'm using Log4Cxx for logging and I'd like to use the %l directive in order to see where is the log trace (filename and line number). The problem I have is that I'm also using CMake that provides full path names to the compiler. This causes that log columns aren't aligned any more and they are now difficult to read since the log4cxx uses the __FILE__ preprocessor macro for determining that information.
Does anyone know any workaround? I'd like to specify for instance either of getting only the "filename.cpp" or better with a nesting parameter, for instance 2 nesting will be "module/submodule/filename.cpp". It seems that by design CMake works with the full path philosophy so I think that this could be solved either by using an unknown to me preprocessor directive or to trick Log4CXX some way.
Looks like you could try using a "ConversionPattern" to at least specify a maximum width for the filename output of log4cxx:
http://logging.apache.org/log4cxx/apidocs/classlog4cxx_1_1_pattern_layout.html
Or do a "find on page" of ConversionPattern here for a config file example:
http://logging.apache.org/log4cxx/index.html
According to the docs, you should get the rightmost characters when you specify a maximum width. Maybe that's a close enough approximation to what you're seeking here.
CMake always passes full path source file names to the compiler so that debug symbols reference the correct source files (for one thing). It's simply easier to guarantee that things work everywhere on all platforms using the full path names.