What is the recommended way to include config files depending on the presence of another package or recipe in an image?
For example, if the python3-supervisor recipe is included in a target image, what is the best way to write a custom recipe so that, in this case, the recipe installs a foo.conf file into /etc/supervisor/conf.d/ directory, and if the python3-supervisor recipe is not installed, then it does not do that?
I was looking at DISTRO_FEATURES as that is sometimes used for installing systemd unit files, but there's also IMAGE_FEATURES which might work too if the .conf files are put in their own packages - e.g. foo-conf installs /etc/supervisor/conf.d/foo.conf but only if, say, IMAGE_FEATURES_append = " supervisor-conf".
I wasn't able to find any recommendations in the bitbake manual on how to do this properly. Is there a good example in any of the OpenEmbedded layers?
In my case I'm writing my own recipes for my own applications, that would install their own .conf files into /etc/supervisor/conf.d/ if the python3-supervisor recipe is installed, and fall back to something else (or nothing) if it isn't, but the problem can be thought of as a more general "how do I gate the installation of certain files based on what's going into the image?"
Since there's no reliable way for one package to know what other packages from other recipes are installed in a particular image (each package is built independently of the packages from other recipes, and may or may not be included in the final image), it seems a good way to do this is with a custom DISTRO_FEATURES string.
For example, in local.conf one can specify an arbitrary custom feature, let's call it supervisord:
DISTRO_FEATURES_append = " supervisord"
Then in individual recipes, the following can be used to selectively install supervisord .conf files, if this feature is selected:
# myrecipe.bb
SRC_URI = " \
${#bb.utils.contains('DISTRO_FEATURES', 'supervisord', 'file://myrecipe.conf', '', d)} \
"
# ...
do_install() {
if ${#bb.utils.contains('DISTRO_FEATURES', 'supervisord', 'true', 'false', d)}; then
install -d ${D}${sysconfdir}/supervisor/conf.d
install -m 0644 ${WORKDIR}/myrecipe.conf ${D}${sysconfdir}/supervisor/conf.d/myrecipe.conf
fi
}
Unlike IMAGE_FEATURES, this does not require the features to be pre-defined in a .bbclass or somesuch. Instead, DISTRO_FEATURES is just treated as space-delimited list of strings, and the check for a particular string in this list is trivial.
Related
I have a package that uses config.json for some settings it uses. I keep the package locally rather than installing it from CPAN. My problem is when I make changes to config.json, the package doesn't recognize the changes since the config file's cached elsewhere, forcing me to run zef install --force-install or delete precomp. How can I ensure that the package always recognizes updates to the config file?
When you install packages using zef, it keeps them in the filesystem, but their names are converted into sha1, something like
/home/jmerelo/.rakudobrew/moar-2018.03/install/share/perl6/site/sources/81436475BD18D66BFD96BBCEE07CCCDC0F368879
zef keeps track of them, however, and you can locate them using zef locate, for instance:
zef locate lib/Zef/CLI.pm6
You can run that from a program, for instance this way:
sub MAIN( Str $file ) {
my $location = qqx/zef locate $file/;
my $sha1 = ($location ~~ /\s+ \=\> \s+ (.+)/);
say "$file → $sha1[0]";
}
which will return pretty much the same, except it will give you the first location of the file you give it at the command line:
lib/Zef/CLI.pm6 → /home/jmerelo/.rakudobrew/moar-2018.03/install/share/perl6/site/sources/81436475BD18D66BFD96BBCEE07CCCDC0F368879
You probably need to install your config.json file in a resources directory (which is the preferred location) and then use something like that.
That said, probably actually installing a module you're testing is not the best strategy. If you're still testing things, it's probably better if you just keep it in the directory you're working with and use perl6 -I<that directory> or else use lib <that directory> is probably a better option. You can just delete that when you release, or keep it, since that only adds another directory to the search path and will not harm the released module.
I am using Chef and this s3cmd cookbook.
As this tutorial says I use knife to download and tar it. I actually made s3cmd work following the tutorial instructions, but I have problems understanding where exactly the installation of s3cmd is happening?
Can anyone explain to me what Chef is doing when using the s3cmd recipe?
When you run chef-solo locally (or chef-client in a server setup) you are telling Chef to compile a set of resources defined in your cookbooks recipes to then be applied to the node you are running the command on.
A resource defines the tasks and settings for something you want to setup.
The run_list set for the node defines what cookbook recipes will be compiled.
s3cmd
In your case, your run_list will have probably been recipe[s3cmd].
This instructs Chef to look in the s3cmd cookbook and as you didn't give a specific recipe, it loads s3cmd/recipes/default.rb.
If you gave a specific recipe, like recipe[s3_cmd::other] then Chef would load the file s3_cmd/recipes/other.rb.
Chef will compile all the resources defined in the recipe(s) into a list and the run through the list applying changes as required to your system.
What s3cmd::default does
First it installs some packages (via your distributions package manager)
python, python-setuptools, python-distutils-extra, python-dateutil, s3cmd
Note: This is entirely different to what the readme says about how s3cmd is installed! Always check!
Figures out where the config should go.
if node['s3cmd']['config_dir']
home_folder = node['s3cmd']['config_dir']
else
home_folder = node['etc']['passwd'][node['s3cmd']['user']]['dir']
end
Creates the .s3cfg config file from a template in the cookbook.
template "#{home_folder}/.s3cfg" do...
What else
Cookbooks can also define their own resources and providers to create more reusable cookbooks. The resource names and attributes will be defined in cookbook/resources/blah.rb. The code for each resource action will be in cookbook/providers/blah.rb
Code can be packaged in cookbook/libraries/blah.rb and included in other Ruby files.
Run chef-solo with the --log-level DEBUG option and step through the output. Try and identify the run list compilation phase, and then where everything is being applied.
Is it possible to have Zope2 buildout unpack python files into their normal directories like how standard python modules do, and not under separate .egg directories? It makes looking for files easier when debugging.
The 'regular' setup doesn't support namespaced packages very well, where multiple eggs share a top-level name (such as plone.app and zope, etc.)
Use the collective.recipe.omelette buildout recipe to build a 'regular' setup, it uses symlinks to give you a searchable structure of all eggs used.
[buildout]
parts =
omelette
...
[omelette]
recipe = collective.recipe.omelette
eggs = ${instance:eggs}
You'll find the result in parts/omelette. Note that this structure uses symlinks, so if you use tools like find or ack make sure to configure them to follow symlinks (e.g. find parts/omelette -L and ack --follow).
The omelette directory structure is not used by Python itself, it is purely intended for presenting a coherent library structure from all eggs used in your buildout.
Note that for Windows, you need to install the junction utility as well for the recipe to work.
I would like to use a rpm to build subpackages for different environments (live,testing,developer) but for the same files, so having a package called name-config-live, one called name-config-testing and one called name-config-developer and in them to have the same paths but each with the configs corresponding to the environment it's named after.
as an example
let's say on all environments I have a file called /etc/name.conf and on testing I want it to contain "1", on development "2" and on live "3". Is it possible to do this in the same spec since the subpackage generation only happens last not in the order I enter it. ( and hopefully not with %post -n )
I tried using BuildRoot but it seems that's a global attribute
I don't think there's a native way; I would do a %post like you had noted.
However, I would do this (similar to something I do with an internal-only package I develop for work):
Three separate files /etc/name.conf-developer, /etc/name.conf-live, etc.
Have all three packages provide a virtual package, e.g. name-config
Have main package require name-config
This will make rpm, yum, or whatever require at least one be installed in the same transaction
Have all three packages conflict with each other
Have each config package's %post (and possibly %verify) symlink /etc/name.conf to the proper config
This also helps show the user what is happening
Cons:
It's a little hackish
rpm --whatprovides /etc/name.conf will say it is not owned by any package
We're having problems when trying to deploy a number of projects which use zc.buildout - specifically we're finding that they want to put their PYTHON_EGG_CACHE directories all over the show. We'd like to somehow set this directory to one at the same level as the built-out project, where eggs can be found.
There is some mention online that this can be done for Plone projects, but is it possible to do this without Plone?
Are there some recipes that can set up an environment variable so we can set the PYTHON_EGG_CACHE executable files in ./bin?
The PYTHON_EGG_CACHE is only used for zipped eggs, your best bet is to have zc.buildout install all required eggs unzipped:
[buildout]
...
unzip = true
If your system python has zipped eggs installed that still require unzipping for resource access, and setting the PYTHON_EGG_CACHE in your scripts is your only option (as opposed to setting the environment variable for your user), you could try to use the initialization option of zc.recipe.egg to add arbitrary Python code to your scripts:
[a-part]
recipe = zc.recipe.egg
...
initialization =
import os
os.environ['PYTHON_EGG_CACHE'] = '/tmp/python_eggs'
I'm not sure what you mean. Three options that you normally have:
Buildout, by default, stores the eggs in a directory called eggs/ inside your buildout directory.
You can set the eggs-dir variable inside your buildout.cfg's [buildout] section to some directory. Just tell it where to place them.
You can also set that very same option in .buildout/defaults.cfg inside your home directory. That way you can set a default for all your projects. Handy for storing all your eggs in one place: that can save a lot of download time, for instance.
Does one of those (especially the last one) accomplish what you want?
And: don't muck around with eggs in the generated bin/* files. Let buldout pick the eggs, that's its purpose.