How can we define specific config files for includes in snakemake workflows?
I have a pipeline with the following structure.
The main entrypoint of the pipeline:
workflow/Snakefile
Other smk files:
workflow/rules/Snakefile_0.smk
workflow/rules/Snakefile_1.smk
workflow/rules/Snakefile_2.smk
Inside Snakefile, I include them using
include: "rules/Snakefile_0.smk"
include: "rules/Snakefile_1.smk"
include: "rules/Snakefile_2.smk"
I access and use the config file within these smk files, too. How can I specify config files for each smk file I include?
Snakemake does not support different configfiles for different rules or rule files if they are within the same main Snakefile. All rules of the include: *.smk files have access to the same configfile and config dict.
If you want to use different config-files for each .smk file, you can use the following workaround:
Instead of using the include directive, import the rules of the .smk files as module. You can specify a different configfile for each module imported.
Related
When using the regular include YML configuration file, an error will be reported, indicating that the file does not exist
include:
- local: .gitlab/staging/*.gitlab-ci.yml
Find the error in your .gitlab-ci.yml :
Local file .gitlab/staging/*.gitlab-ci.yml does not exist!
but，it is normal to import one file
include:
- local: .gitlab/staging/oa.gitlab-ci.yml
it works
Use quotes around the path string, otherwise it will not be interpreted properly.
# This matches all `.yml` files in `configs` and any subfolder in it.
include: 'configs/**.yml'
# This matches all `.yml` files only in subfolders of `configs`.
include: 'configs/**/*.yml'
See, Use include:local with wildcard file paths.
Is it possible to overwrite nested config options with Snakemake's --config command line option? For example: an option accessed internally with config['step_a']['opt_b']. Is there some syntax to specify this when starting the Snakemake workflow?
This works but you would need to supply other elements of the step_a dict:
--config "step_a={'opt_b':'c'}"
I think the proper way is to have a different config file using --configfile.
If there is some config that is relatively static and some that is changeable, use two config files. You can supply multiple config files using --configfile.
So after using the Swagger-CodeGen, I was wondering if there was any way to exclude generating the Client Folder (with Configuration, ApiClient, ApiException, etc) since this code seems to be boilerplate and it would be duplicated when I have a multiple endpoints generated with CodeGen.
Besides from manually going to delete the client folder and remapping the namespaces in the generated Api's, is there currently a solution for this?
You can skip file generation by using the .swagger-codegen-ignore file, which works very similar to .gitignore. Here is an example:
# Swagger Codegen Ignore
# Lines beginning with a # are comments
# This should match build.sh located anywhere.
build.sh
# Matches build.sh in the root
/build.sh
# Exclude all recursively
docs/**
Ref: https://github.com/swagger-api/swagger-codegen#ignore-file-format
According to Apache manual, the Include directive is recursive when a directory path is used. But is it recursive when using a wildcard path?
Include "/usr/local/apache/conf/userdata/std/2/username/domain.com/*.conf"
I checked it and it is not recursive.
As Joyce already said, I can confirm by testing it myself that it is not recursive.
Include uses fnmatch as wildcard engine, which doesn't match a slash by default, unless the FNM_PATHNAME flag is set, so a * doesn't match / so domain.com/*.conf will not look in sub-directories.
However, since httpd 2.3.6 it is possible to also use the wildcard for sub-directories.
Examples
Include /usr/local/apache2/conf.d/ssl.conf
This matches only a specific file.
Include /usr/local/apache2/conf.d
If conf.d is a file, it matches only this file. If conf.d is a directory, all files will be matched recursively, including files in sub-directories and non-conf files (which causes an error).
Include /usr/local/apache2/conf.d/*.conf
This will only match the files with a .conf suffix, directly located in the conf.d directory. Files in sub-directories aren't matches.
Include /usr/local/apache2/conf.d/*/*.conf
This will only match the files with a .conf suffix, directly located in sub-directories of the conf.d directory, but it will NOT match files directly located in the conf.d directory.
So for example, if you need to match all .conf files directly located in conf.d and in first level of sub-directories and second level of sub-directories, you can use this:
Include /usr/local/apache2/conf.d/*.conf
Include /usr/local/apache2/conf.d/*/*.conf
Include /usr/local/apache2/conf.d/*/*/*.conf
If you only have valid configuration files in conf.d and want to match every level of subdirectories, then you can use:
Include /usr/local/apache2/conf.d
Instead of using wildcard, you should use a directory.
It has been supported since as early as 1.3 https://httpd.apache.org/docs/1.3/mod/core.html#include
New in Apache 1.3.13 is the feature that if Include points to a directory, rather than a file, Apache will read all files in that directory, and any subdirectory, and parse those as configuration files.
While creating RPMs, the RPM spec files have a directive %defattr . I know that it defines the default attributes for the files that are installed by that RPM. If I write the %defattr as below, what does it mean?
%defattr(-testuser, testuser)
The mode you specified is invalid. %defattr takes four arguments
From http://ftp.rpm.org/max-rpm/s1-rpm-inside-files-list-directives.html#S3-RPM-INSIDE-FLIST-DEFATTR-DIRECTIVE
The %defattr Directive
The %defattr directive allows setting of default attributes for files and directives. The %defattr has a similar format to the %attr directive:
The default permissions, or "mode" for files.
The default user id.
The default group id.
The default permissions, or "mode" for directories.
The %attr directive has the following format:
%defattr(file mode, user, group, dir mode)
As with %attr if a particular attribute does not need to
be specified (usually because the file is installed with that
attribute set properly), then that attribute may be replaced with a
dash. In addition the directory mode may be ommited. %defattr tends to
be used at the top of %files.
To set permissions and ownerships in a spec file treat the directory like a file thusly...
%defattr will set all files without %attr (in this case rww owner apache group apache and set directories to 755).
%files
#%attr(<mode>, <user>, <group>) file
%defattr(644,apache,apache,755)
%attr(-,apache,apache) /var/www/coolapp
%attr(-,apache,apache) /var/www/coolapp/js
%attr(-,apache,apache) /var/www/coolapp/static
/var/www/coolapp/index.html
/var/www/coolapp/__init__.py
/var/www/coolapp/settings.py
/var/www/coolapp/urls.py
/var/www/coolapp/wsgi.py