Considering all freedom that yocto gives to the developer, I have a question.
I would like to make this my_file.inc available only for recipes in one particular meta-layer. I know, that, for instance, using INHERIT keyword inside the local.conf will make my_class.bbclass file available globally for each recipe.
Is it a good practice to add this:
require my_file.inc
inside layer.conf? Or should I change my_file.inc to the my_file.bbclass, and, add INHERIT = "my_file.bbclass" to the layer.conf?
Any other possibilities?
Even if it seems to work, neither of your approaches is technically completely correct. The key point is that all .conf files are parsed first and everything they contain is globally visible throughout the whole build process. So if you add something through the layer.conf file, itis not being pulled in through an unexpected place, it also is not being limited that layer only and might therefore cause breakage at other places.
While I do not have a really good and clean solution, maybe the following can help you:
You can make your custom recipes react on certain keywords in DISTRO_FEATURES or MACHINE_FEATURES. Then you can create a two-staged approach:
Add the desired keyword in local.conf (or your MACHINE, or DISTRO, or whatever configuration)
Make the recipes react to it. If you need the mechanism in several places, then it might be useful to pour it into a .bbclass that your layer brings along and that you pull in for the respective recipes.
This way the effect is properly contained.
Maybe part 5.1.3.2 from the Yocto Project answers your question:
Avoid duplicating include files. Use append files (.bbappend) for each recipe that uses an include file. Or, if you are introducing a new recipe that requires the included file, use the path relative to the original layer directory to refer to the file. For example, use require recipes-core/package/file.inc instead of require file.inc. If you're finding you have to overlay the include file, it could indicate a deficiency in the include file in the layer to which it originally belongs. If this is the case, you should try to address that deficiency instead of overlaying the include file. For example, you could address this by getting the maintainer of the include file to add a variable or variables to make it easy to override the parts needing to be overridden.
So to avoid duplicate inclusion later, it would be better not to include your .inc file(s) this way.
Related
I have multiple projects that I want to share a similar .clang-format style, but I also want to be able to make minor tweaks between each project so they can be slightly different from one another if needed. Currently each project just has the same .clang-format file copied and pasted into its own repository, but it feels wrong because all of the style options are just duplicated from project to project and if I need to change one option I need to go across all projects and manually change it in all of them.
I would like to create my own style that can be used with the "BasedOnStyle" option (See here for more info). That way I can specify that I want all of these projects to be based on the same custom style that would be kept in a shared location, and then I could easily override any project specific options on a per-project basis.
As far as I can tell, there doesn't appear to be any way to create your own style and save it so other projects can be based on the same style. I feel like this is something a lot of users would need (for example if a company wanted to define their own master style that all projects should follow there doesn't appear to be a good way to do it).
Has anyone else run into this problem and found a good solution?
I reached out to the llvm-dev email list and got a response. A feature request is in the works to do something similar to what I want, but not exactly the same. Either way, this should be suitable for my needs when it becomes available.
https://reviews.llvm.org/D93844
here is a complex problem that I am struggling to find a clean solution for:
Imagine having a Snakemake workflow with several rules that can be parameterized in some way. Now, we might want to test different parameter settings for some rules, to see how the results differ. However, ideally, if these rules depend on the output of other rules that are not parameterized, we want to re-use these non-changing files, instead of re-computing them for each of our parameter settings. Furthermore, if at all possible, all this should be optional, so that in the default case, a user does not see any of this.
There is inherent complexity in there (to specify which files are re-used, etc). I am also aware that this is not exactly the intended use case of Snakemake ("reproducible workflows"), but is more of a meta-feature for experimentation.
Here are some approaches:
Naive solution: Add wildcards for each possible parameter to the file paths. This gets ugly, hard to maintain, and hard to extend really quickly. Not a solution.
A nice approach might be to name each run, and have an individual config file for that name which contains all settings that we need. Then, we only need a wildcard for such a named set of parameter settings. That would probably require to read some table of meta-config file, and process that. That doesn't solve the re-use issue though. Also, that means we need multiple config files for one snakemake call, and it seems that this is not possible (they would instead update each other, but not considered as individual configs to be run separately).
Somehow use sub-workflows, by specifying individual config files each time, e.g., via a wildcard. Not sure that this can be done (e.g., configfile: path/to/{config_name}.yaml). Still not a solution for file re-using.
Quick-and-dirty: Run all the rules up to the last output file that is shared between different configurations. Then, manually (or with some extra script) create directories with symlinks to this "base" run, with individual config files that specify the parameters for the per-config-runs. This still necessitates to call snakemake individually for each of these directories, making cluster usage harder.
None of these solve all issues though. Any ideas appreciated!
Thanks in advance, all the best
Lucas
Snakemake now offers the Paramspace helper to solve this! https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html?highlight=parameter#parameter-space-exploration
I have not tried it yet, but it seems like the solution to the issue!
I'm learning/vetting CMake at the moment as I'm thinking of migrating our code to it. One thing we do a lot of with our current make system is to "subproject" common code files. For example, we have a lot of shared, generic headers (plus some c/cpp files) which get included in every project we create. I want to replicate this in CMake but I don't see an easy way of doing it. To be precise, I want to do something like:
Parent CMakeLists.txt
add_subdirectory(shared_folder shared_build_folder)
#Next line should somehow add in the files reference in the shared_folder
add_executable([specific files for this project] build_folder)
Child CMakeLists.txt (shared_folder)
#Somehow have a list of files here that get added to the parent project
So far I've found various "ways" of doing this, but all seem a little hacky. I'm coming to the conclusion that this is in fact the way I have to do things and CMake isn't really geared towards this style of development. For clarity, most of my solutions involve doing something like creating a variable at the parent level which consists of a list of files. This variable (via some shenanigans) can get "passed" to/from any children, filled in and then when I call add_exectuable I use that variable to add the files.
All my solutions involve quite a few macros/functions and seemingly quite a bit of overhead. Is this something other people have tried? Any clues on the best approach for doing this?
Thanks
Andrew
We were facing the exact same problem and after some time of crying we accepted the CMake-way and it resulted in a better structured project even if it meant to change some parts of our structure.
When using sub-directories the targets are automatically exported throughout the whole project (even in subsequent other add_subdirectory-calls) once the add_subdirectory-statement was processed: sub-projects which contain common code are creating libraries.
There is also the PARENT_SCOPE which you can use to export variables to parent CMakeLists.txt
For "other" things we simulated the FindPackage-mechanism by including .cmake-files into the main CMakeLists.txt with include. In doing so we can provide variables easily, change the include_directories and do other fancy things global to the project.
As there are no dependencies between cmake-variables, we don't use cmake to configure the source (features of the project), but only the build (compiler, includes, libraries...). This split was the key element of our build-system-refactoring.
I am trying to clean up some of my projects, and one of the things that are puzzling me is how to deal with header files in static libraries that I have added as "project dependencies" (by adding the project file itself). The basic structure is like this:
MyProject.xcodeproj
Contrib
thirdPartyLibrary.xcodeproj
Classes
MyClass1.h
MyClass1.m
...
Now, the dependencies are all set up and built correctly, but how can I specify the public headers for "thirdPartyLibrary.xcodeproj" so that they are on the search path when building MyProject.xcodeproj. Right now, I have hard-coded the include directory in the thirdPartyLibrary.xcodeproj, but obviously this is clumsy and non-portable. I assume that, since the headers are public and already built to some temporary location in ~/Library (where the .a file goes as well), there is a neat way to reference this directory. Only.. how? An hour of Googling turned up blank, so any help is greatly appreciated!
If I understand correctly, I believe you want to add a path containing $(BUILT_PRODUCTS_DIR) to the HEADER_SEARCH_PATHS in your projects build settings.
As an example, I took an existing iOS project which contains a static library, which is included just in the way you describe, and set the libraries header files to public. I also noted that the PUBLIC_HEADERS_FOLDER_PATH for this project was set to "/usr/local/include" and these files are copied to $(BUILT_PRODUCTS_DIR)/usr/local/include when the parent project builds the dependent project. So, the solution was to add $(BUILT_PRODUCTS_DIR)/usr/local/include to HEADER_SEARCH_PATHS in my project's build settings.
HEADER_SEARCH_PATHS = $(BUILT_PRODUCTS_DIR)/usr/local/include
Your situation may be slightly different but the exact path your looking for can probably be found in Xcode's build settings. Also you may find it helpful to add a Run Script build phase to your target and note the values of various settings at build time with something like:
echo "BUILT_PRODUCTS_DIR " $BUILT_PRODUCTS_DIR
echo "HEADER_SEARCH_PATHS " $HEADER_SEARCH_PATHS
echo "PUBLIC_HEADERS_FOLDER_PATH " $PUBLIC_HEADERS_FOLDER_PATH
.
.
.
etc.
I think that your solution is sufficient and a generally accepted one. One alternative would be to have all header files located under an umbrella directory that can describe the interface to using the depended-on libraries and put that in your include path. I see this as being similar to /usr/include. Another alternative that I have never personally tried, but I think would work would be to create references to all the headers of thirdPartyLibrary from MyProject so that they appear to be members of the MyProject. You would do this by dragging them from some location into MyProject, and then deselecting the checkbox that says to copy them into the project's top level directory. From one perspective this seems feasible to me because it is as if you are explicitly declaring that your project depends on those specific classes, but it is not directly responsible for compiling them.
One of the things to be wary of when addressing this issue is depending on implementation-specific details of Xcode for locating libraries automatically. Doing so may seem innocuous in the meantime but the workflows that it uses to build projects are subject to change with updates and could potentially break your project in subtle and confusing ways. If they are not well-defined in some documentation, I would take any effect as being coincidental and not worth leveraging in your project when you can enforce the desired behavior by some other means. In the end, you may have to define a convention that you follow or find one that you adopt from someone else. By doing so, you can rest assured that if your solution is documented and reproducible, any developer (including yourself in the future) can pick it up and proceed without tripping over it, and that it will stand the testament of time.
The way we do it is to go into build target settings for the main project and add:
User Header Search Path = "Contrib"
and check that it searches recursively. We don't see performance problems with searching recursively even with many (10-15 in some projects) dependencies.
Supposing I have a merge module that installs a file "MyFile.txt" to a certain location, and that I wish to use that merge module, however I want to supply a different copy of "MyFile.txt" from the one supplied with the merge module.
Is it possible to do this? (And for bonus points how can I do this using Wix)
Update: Roughly speaking MyFile.txt is part of a package up component of installable items that we provide to others, they then comine these components with their own to produce an installer.
In the ideal world they would only need to add new files to the output, however this is a replacement for an existing system where they currently have the ability to modify or even replace items (suce as MyFile.txt) in the end installer, and so without the ability to do the same with the merge module the migration path will be difficult.
The packaged up component doesn't need to be a merge module if there is a better solution, however merge modules seemed like the sensible choice and in all other respects provide a very nice re-usable package of installer logic.
It's possible but every technique that I know is a bit of a hack and doesn't scale very well. Can you tell me more about what type of file MyFile.txt is and what the intent of the different flavors of the file? Usually my goal is to never have the same filename twice ( darn component rules ) and then design variation points to support the needs. Sometimes upstream changes to the application are required to do this correctly.