Question
When building Singularity images using definition files, is there a way to specify the path to a file on the host system relative to the definition file (i.e. independent of where the build command is called)?
Example to Illustrate the Problem
I have the following files in the same directory (e.g. a git repository):
foobar.def
some_file.txt
foobar.def looks as follows:
Bootstrap: library
From: ubuntu:20.04
Stage: build
%files
# Add some_file.txt at the root of the image
some_file.txt /some_file.txt
This works fine when I build with the following command in the directory which contains the files:
singularity build --fakeroot foobar.sif foobar.def
However, it fails if I call the build command from anywhere else (e.g. from a dedicated "build" directory) because it searches some_file.txt relative to the current working directory of the build command, not relative to the definition file.
Is there a way to implement the definition file such that the build works independently of where the command is called? I know that I could use absolute paths but this is not a viable solution in my case.
To make it even more complicated: My actual definition file is bootstrapping from another local image, which is located in the build directory. So ideally I would need a solution where some files are found relative the working directory while others are found relative to the location of the definition file.
Short answer: Not really
Longer answer: Not really, but there's a reason why and it shouldn't really matter for most use cases. While Docker went the route of letting you specify what your directory context is, Singularity decided to base all of its commands off the current directory where it is being executed. This also follows with $PWD being auto-mounted into the container, so it makes sense for it to be consistent.
That said, is there a reason you can't run singularity build --fakeroot $build_dir/foobar.sif foobar.def from the repo directory? There isn't any other output written besides the final image and it makes more sense for the directory with the data being used to be the context to work from.
Related
For my workflow, the scripts & snakefile are in a different directory than the output files. I specify the latter using the workdir directive, which seems to work fine.
Now in some cases there are static input files in other directories, whose paths I want to specify relative to the snakefile. According to the documentation, I should use workflow.get_source("path/relative/to/snakefile"). However this gives me
'Workflow' object has no attribute 'get_source'. Do I need some additional imports?
Looking at the documentation, it looks like what's used in the code example is workflow.source_path("path/relative/to/snakefile") rather than get_source - you can give that a try.
Otherwise, though I don't think it's officially documented, workflow.basedir gives you the path of the directory the Snakefile lives in, and you can build the rest of the path off that.
when I run nextflow, I get a .nextflow folder, but I can't find a way to change its location (i.e. it is't -work-dir). How can I change the location of the .nextflow folder?
I have looked at launchDir but it seems that is a read-only implicit variable and cannot be overwritten in the CLI, also, the --launchDir option is only valid for the k8s scope (see original chat in gitter)
I'm using Nextflow 20.10.0 build 5430.
Keeping things neat and tidy is admirable. From this comment, it looks like the only way (without doing crazy things...) is to change to the directory you want your .nextflow cache directory to live and point all other options (i.e. -work-dir, -log etc) away to a separate directory:
If you want .nextflow in dir A and the pipeline work dir in B:
cd A
nextflow run -w B
The .nextflow has to be in the launching
directory to properly maintain the history of the executions.
I use lessc 2.7.3. I generate css files via a makefile and use following paths
the makefile is in themes/bodensee
the css is generated in themes/bodensee/css
the less files are in themes/bodensee/less
the maps are in the same folder as the css files.
My problem is that css files misses the themes/bodensee path, so it raises a file not found on css.map files.
lessc -s less/wlb.less --clean-css="--s0 --advanced" --source-map-rootpath=themes/bodensee/ --source-map="css/wlb.css.map" css/wlb.css
The CSS file now contains `sourceMappingURL=css/wlb.css.map``The rootpath does not have any effect.
I also tried a fantasy rootpath and searched for it in the file - it does not appear anywhere. But the option is correct. When I try to missspell the option, LESS drops an error.
What am I missing?
Description of the --source-map-rootpath option from here
Specifies a rootpath that should be prepended to each of the less file paths inside the sourcemap and also to the path to the map file specified in your output css.
Because the basepath defaults to the directory of the input less file, the rootpath defaults to the path from the sourcemap output file to the base directory of the input less file.
Use this option if for instance you have a css file generated in the root on your web server but have your source less/css/map files in a different folder. So for the option above you might have
The problem was indeed related to the Clean-CSS plugin.
I now call
lessc --source-map --clean-css="--s0 --advanced" -s less/wlb.less css/wlb.css which is working.
There is a standalone clean-css program, but that does not generate sources for the Less files. It's not clear if the lessc plugin and the standalone tool are the same or different implementations but both use node.
The standalone cleancss tool removes the source map URL generated by lessc be default (did not play around with the dozens of options).
These Node tools develop very fast and manual/tutorials often are outdated. That's why my make file stopped working. Developers of that tools should really consider not to touch working parameters or features and to keep their code compatible.
I am installing a package manually on my own system because I need to make some changes to it that aren't available in the basic version in my package manager. I also am trying to keep packages installed locally if possible, so I'm installing it with prefix=$HOME/.local instead of the more common prefix=/usr/local.
When I do this, I have no problem executing the program from my terminal, because I added ~/.local/bin to my PATH and the package was installed with relative paths to its shared libraries (i.e. ~/.local/lib/<package>). Executing from the command line is no problem, but I want to be able to access it from the favorites menu in gnome, and for that I need to make use of the <package>.desktop file.
I could hard-code the path to the executable in the .desktop file itself, but when I pull a later version down and re-install it, I'll have to redo those steps. I was wondering if there's a way to avoid that.
I've tried symlinking the executable to a directory where .desktop files do have included in their path, and the application is correctly treated as a GUI option, but launching the executable results in an error trying to find a shared library. I think this has to do with how cmake handles rpaths, which to my understanding is a way of relatively linking executables with their required libraries.
I think what I want to do is have PATH inside a .desktop file include ~/.local/bin, without changing the .desktop file itself. Can I alter the 'default' path used in accessing a .desktop file?
The answer to my question was found in the Archwiki:
Specifically, I needed to add ~/.local/bin to my path in ~/.xinitrc. Now my graphical programs work as expected.
I have my mind crashing with cmake. After this answer I have tried to make a simple example and put it in github because there are a lot of file inside directories and could be boring copy everything here.
What I'd like to do is to build a simple frameworks for handling my qt/opencv/opengl experiments. What I like to have is a repository with those directories:
root*
|-apps
|---test1*
|
|-build
|-cmake*
|
|-modules
|---foo*
(The * signed directory are the ones with some cmake files like CmakeLists.txt or FindXXModule.cmake)
In modules i can write the modules (for example a module for face recognition, a module for draw a red cube in opengl, a module that contains my personal qt widget extension).
Than I need an easy way for create an application and link some modules on it. For that I thought to create a cmake directory where to put the FindXXModule.cmake and in the apps just say: find_package(XXModule).
Note that for now I don't care about installing this repository and the tree structure must be this one (so if I am in a apps/test2 I know I can refer to the cmake directory as ../../cmake or the module directory is ../../modules)
I have wrote a little example with the app named test1 that uses the module foo and i put it in a github repository.
For now I can compile the application test1 with cmake calling cmake path_to_test1_CmakeLists.txt and I am happy about that. But if I try to launch cmake path_to_root_CmakeLists.txt it does not work because the file Findfoo.cmake is read two time (and i did't be able to use some if for not reading it twice).
Then, if i run the test1 cmake a foo directory with cmake cache etc are created in root/cmake and I don't want it. I want all file cmake has to generate are in root/build directory.
So, those are my 2 question:
How create a CmakeLists.txt that can build all the apps and all the future test i will write in the modules directory
How avoid that launching cmake of a single app will create files in the cmake directory.
Sorry if my english and my idea of how cmake works are not good.. i hope it is clear.
EDIT:
One more thing. In Findfoo.cmake I have a bad hack: for adding the real CMakeLists.txt inside a modules/foo when I call the cmake from test1 I have to add a subdirectory that is not in the tree.. Maybe this kind of hack could be deleted reviewing the enteire structure..
As you say you want to put the whole directory structure into source control. This means these folder structure is same on every location where you do a checkout. So why creating the Findfoo.cmake if you have a relative path the the foo directory?
I suggest to put a CMakelists.txt file in to root that adds all subdirectories. To reduce confusion between files generated by CMake and original files, you should create a folder called ./build (or even ../build) and run CMake in that directory with the root directory as first argument. This creates all CMake generated files in the ./build directory and gives you the possibility to clean it up easily. This way of working is called out-of-source build and its highly recommended to use cmake in this way. See this question for an example.