I'm wondering how to manage paths in my Snakefiles. Say I have this configuration:
current_dir
current_dir/snakefiles
current_dir/configfiles
and I execute my workflows this way:
current_dir$ snakemake -s snakefiles/my_snakefile --configfile configfiles/my_config.yml
I know I can get the path to my Snakefile using the global variable workflow.snakefile, but I would like to get also:
the path to my configfile
the path where I'm executing my snakefile, e.g. current_dir
How to achieve this? Are there other global variables in Snakemake, that I'm not aware of?
Thank you
The working directory is set via Python. You can get it with os.getcwd(). Also please note that there is a canonical way to organize Snakemake workflows: http://snakemake.readthedocs.io/en/latest/project_info/faq.html#what-is-the-recommended-way-to-distribute-a-snakemake-workflow.
While you can of course use something else, following this scheme helps others to understand your workflow. There might of course be cases where this does not fit.
Related
For my workflow, the scripts & snakefile are in a different directory than the output files. I specify the latter using the workdir directive, which seems to work fine.
Now in some cases there are static input files in other directories, whose paths I want to specify relative to the snakefile. According to the documentation, I should use workflow.get_source("path/relative/to/snakefile"). However this gives me
'Workflow' object has no attribute 'get_source'. Do I need some additional imports?
Looking at the documentation, it looks like what's used in the code example is workflow.source_path("path/relative/to/snakefile") rather than get_source - you can give that a try.
Otherwise, though I don't think it's officially documented, workflow.basedir gives you the path of the directory the Snakefile lives in, and you can build the rest of the path off that.
Does gitlab-ci have a way to download a script and store it on the local file system so it could be run? Looks like other's have asked similar questions (see below).
One way to do it would be to use curl (but curl has to exist in the CI runner):
curl -o ./myscript -k https://example.com/myscript.sh
This was from https://stackoverflow.com/a/22800194/3281336.
If I have a script that I would like to use in multiple CI-pipelines, I'd like to have a way to download the script to the local file system to use in the pipeline. NOTE: I don't have the ability to create a custom runner or docker image in my given situation.
If the script were available via git or an https website, what are my alternatives?
Some search results
https://docs.gitlab.com/ee/ci/yaml/includes.html - Gitlab supports a way to include files, even from GIT repos. This might work I just haven't read how.
How to run a script from file in another project using include in GitLab CI? - Similar but the answer uses a multi-project pipeline and a trigger which is really (I think) a different answer.
.gitlab-ci.yml to include multiple shell functions from multiple yml files - Similar but the question is dealing with scripts in YAML files and I'm dealing with a stand alone script.
How to include a PowerShell script file in a GitLab CI YAML file - So far this is the closest to my question and some might consider it the same even though the question is asking about a powershell script. The answer said this wasn't possible to include a script (so maybe this is not possible using the GitLab CI syntax).
If it is possible, please let me know how to do this.
I have been using singularity with some of my workflows and it works great so far. I have a question about binding directories. I can pass singularity arguments when running the snakemake workflow like:
snakemake --use-singularity --singularity-args "-B /path/outside/container/:/path/inside/container/"
Is there a way to have binds be rule specific?
Thanks.
I try to write a workflow using snakemake, but do not know how to configure PATH environment for all rules in the configure file, and did not find relational questions. Anyone can give a help?
Have a look at this FAQ entry. However, please also consider to simply use Conda integration instead of manupulating PATH yourself.
I want to write a script in a zc.buildout recipe, but the script needs to have access to buildout.cfg.
I know how to create a buildout recipe (which has access to its configuration) and how to create a buildout script, but I don't know how the script could read the config.
Is there a way that the script receive the configuration from buildout.cfg?
Or should it open the file by itself and parse the config?
In the last case, what is the best way to get the path? what's the best way to parse it?
Thanks in advance.