I have changed an initial rule that adds links to database files. Snakemake typically requires all dependent rules to rerun, even the database change is not really relevant. I tried to avoid this behaviour with
snakemake --touch
Unfortunately it does not work and still wants to rerun all blast processes on top.
How to check the dependence tree ? How to mark individual files as up-to-date ?
snakemake --version 6.4.1
Best, Michael
Related
I build my runner and it works fine. However, when it initializes, it first clones the project to /home/user/builds/xxxx... I never, ever, ever want GitLab to use /home. Never. Not for anything. I was told that it is impossible to change it to a different location. I find it hard to believe.
See in the image below, it gives me a warning about templates not found in some made up directory, then clones the entire project under the user's home directory. I don't give it that command - so it must be a default. Is there a way to choose ANY other mount point? The project is several hundred gigabytes and the /home directory is 50k. I cannot control that. So to a different mount point it must go.
I can provide the yml etc, but this is about core behavior of the runner itself - not anything I created. I'm hoping it is a simple variable I can send when initializing the runner.
Thank you in advance.
I have a main Snakefile and several subworkflows running in independent subdirectories (with paths relative to their own directories). I've noticed that if I modify one of the input of a subworkflow, it will rerun correctly but all the following rules that come afterwards are not rerun.
If I understand correctly what is going on, there's a different DAG for the main Snakefile and for each subworkflow. The main DAG is not aware of any modification in a subworkflow and therefore won't trigger a rerun since the output of the subworkflow hasn't been modified yet.
I'd like that all the rules depending of the output of a subworkflow are rerun if there's a modification in that subworkflow. Isn't that what the default behaviour should be ?
I've also tried the other modularisation techniques. Using includes works but is super annoying because I have to modify all the paths to be relative to the main directory (and therefore I can't run snakemake independently in one subdirectory anymore). I've also tried using the new module system coming with snakemake v.6 that is supposed to be replacing subworkflows. Maybe I don't use it correctly, but it doesn't seem to work for my use case. If I import a rule from a subdirectory it complains that there are missing inputs. It doesn't find the scripts because they are in the subdirectory and not in the main directory. So in that sense it works more like an include than a subworkflow.
Do you have any idea on how to solve my issue ?
Here's a small working example with the module implementation:
MainDirectory
| - Snakefile
rule all:
input: "Subdirectory/file.txt"
module other_workflow:
snakefile: "Subdirectory/Snakefile"
use rule * from other_workflow as other_*
| - Subdirectory
| | - Snakefile
rule rule_a:
input:
script = 'code.py'
output: 'file.txt'
shell: 'python {input.script}'
| | - code.py
with open('file.txt', 'w') as f:
print('This is a test.', file=f)
This doesn't work as the snakefile in the main directory uses all the rules in the same workdir, whereas I would like it to be running the imported rules in their own workdir. I can make it work by modifying all the relative paths in the subdirectory but that's not what I want. I want to be able to run it without modifications.
The issue is that you mix input files with code here. If the script code.py is defined as an input file, Snakemake expects it to be in the workdir. If you'd use the script directive or the Jupyter notebook directive instead, the path will be automatically relative to the Snakefile. If that should be not an option for whatever reason, you can instead build the path relative to the current Snakefile via Path(workflow.snakefile).parent / "code.py".
Note, there is really no reason to register code as input files. If you intend to get a rerun upon changes in the code, it is better to rely on snakemake --list-code-changes. The reason Snakemake does not automatically trigger reruns upon code changes is that they can be just cosmetic (e.g. formatting). Hence, it is up to the dev to trigger the rerun, e.g. via --list-code-changes, or manually.
Can't find anything on this online and might be a non-issue, but I figured I'd ask here to make sure.
We run the Wordfence security plugin on a bunch of WordPress sites and have recently seen this "critical issue" reported:
Filename: wp-content/themes/theme-name/node_modules/webpack-assets-manifest/test/fixtures/client.js
File Type: Not a core, theme, or plugin file from wordpress.org.
Details: This file appears to be installed or modified by a hacker to perform malicious activity.
If you know about this file you can choose to ignore it to exclude it from future scans.
The matched text in this file is: require('./Ginger.jpg');
The issue type is: Backdoor:PHP/req_img.3645
Description: A backdoor known as req_img
Now first of all that doesn't look like a backdoor to me, especially since node_modules contents aren't executed unless I run npm (or yarn), as far as I understand. Is this more serious than I think?
Secondly, when running npm/yarn on the server, the node_modules folder has chmod 775 (drwxrwxr-x) by default. Is it okay to leave it like that or should we take any action?
So here's a question. I'm new to Sylius, and am working on some simple CSS updates. I have a local copy of Sylius running with the built-in webserver: server:run. I also have a development server on Digital Ocean, which runs an (almost) identical copy of Sylius, aside from the configs of course.
Something strange is happening with my CSS update, however. I made a change to .navbar-brand within web/assets/compiled/backend_backend_4.css.
This change showed up immediately on my local. On the development server, however, when pulling down the change (git), and verifying that it now exists in that file, the change doesn't seem to propegate. It's effects aren't shown, inspecting the stylesheet doesn't show them, and furthermore viewing the css file sourcecode directly in the browser does not show the change. But on the filesystem it's definitely there.
I've tried clearing the cache, to no avail.
I also checked the assetic value in both config_dev.yml files, and verified they are both set to use_controller: true
Even still, I tried dumping assetic, to no avail.
So I'm wondering what's going on. Additionally, I realize that I probably shouldn't edit CSS files within a folder called 'compiled'. I'm sure there's a way to do that using a compiler, but I'm not yet familiar with the process and am just making minor changes and learning about caching so far.
Yes you are right you shouldn't be editing the compiled files.
You should edit the source files, then run gulp
or on my system i have to explicitly run npm run gulp
I've documented the solution that worked for me here. It didn't involve Gulp at all, but instead uses Assetic:
Assets need to be installed as hard copies first (I'm not quite sure
what this does exactly, but it seems like an important step because
it copies a lot of assets to places. Documentation was unhelpful but
it was suggested on Stack Overflow somewhere.):
app/console assets:install web
Assets should be edited in web/bundles/[bundle-here]/css or js. This
is frequently within syliusweb if it has to do with page styles /
layouts.
Hint: These assets are referred to in files such as
src/Sylius/Bundle/Resources/views/Backend/layout.html.twig (see the
opening:
(% stylesheets
tag, or search universally for this tag).
Within this tag, you'll see that stylesheets have an output to the compiled folder, but also list the
bundles where they pull their original css from. You should edit one of the source css files, if you'd like your changes to end up in the destination css.
After editing assets, dump assetic:
php app/console assetic:dump
Note - it is also possible to set an assetic watcher on these assets
(google to find out how, think it's a -w flag somewhere), but this is
said to only work in development mode, as it should.
After dumping assetic, the assets from the source bundles compile into their assets/compiled versions, usually combining multiple stylesheets. You should now see your asset refresh!
If CMake is re-executed after the source directory is moved, an error like this is given:
CMake Error: The current CMakeCache.txt directory /new-path/CMakeCache.txt is different than the directory /old-path where CMakeCache.txt was created. This may result in binaries being created in the wrong place. If you are not sure, reedit the CMakeCache.txt
What is the best approach to deal with this? It seems that deleting the cache works. Why might I want to edit CMakeCache.txt, rather than just delete it?
I think your best bet is indeed to delete the cache and re-create it. In fact, I tend to go further and remove the entire build tree if I need to move the source, but this is probably overkill.
The CMakeCache.txt file can be edited via the CMake GUI or by hand if you're sure what you're doing, but I find it's not usually worth the effort compared to re-running cmake from scratch. I guess if running CMake takes a significant amount of time (e.g. if you use ExternalProject_Add which could involve downloading and building a third-party resource), it may be preferable to modify the CMakeCache.txt.
This article provides a bit more detail about the CMakeCache.txt file and editing it. In particular it says:
The main reason for editing the cache would be to give CMake the location of a native library such as JPEG, or to stop it from using a native library and use a version of the library in your source tree.
CMake will not alter an existing entry in the cache file itself. If your CMakeLists.txt files change significantly, you will need to remove the relevant entries from the cache file. If you have not already hand-edited the cache file, you could just delete it before re-running CMake.
I applied below two command and worked for me.
flutter clean
flutter pub get
flutter clean + flutter pub get is the easiest solution
I ran into same error while running a CGAL code (which I had already compiled sometime ago) . All I did was create a new directory and copy the ".cpp" file there and run " CGAL_create_CMakelists " and then run cmake command.It worked this way. I don't exactly know but I think the error comes when you run the code which was already compiled and has its build files already in the folder(due to previous compiling).