Function to determine path from imported module output - snakemake

We have a project in which we combine multiple independent Snakemake workflows using subworkflow. Now I'm trying to convert the project to use module instead but I'm facing difficulties due to either lacking features or documentation.
Previsouly we could in the "parent" Snakefile include a subworkflow like this (from the documentation:
subworkflow otherworkflow:
workdir:
"../path/to/otherworkflow"
snakefile:
"../path/to/otherworkflow/Snakefile"
configfile:
"path/to/custom_configfile.yaml"
rule a:
input:
otherworkflow("test.txt")
output: ...
shell: ...
and then e.g. indicate that files should be found in the workdir of the otherworkflowby specifying otherworkflow(<output_file>).
Problem:
When using Snakemake module an equivalent options seems to be missing.
My current workaround:
I reference the output of the module via rules.<...>.output, e.g.:
Snakefile 1:
rule a:
output: "some_file.txt"
shell: ...
Snakefile 2:
module other_workflow:
snakefile: "<path to Snakefile 1>
workdir: "<some workdir path>"
use rule * from otherworkflow as other_*
rule b:
input: rules.other_a.output[0]
shell: ...
Which works to some extent but is not as nice to read as the previous syntax from subworkflow.
Also: if the output contains a wildcard, then this workaround does not work anymore. (expand-ing the wildcard e.g. expand(rules.other_a.output[0], wildcard_name=<value>) does not work; what does work is format-ing rules.other_a.output[0].format(wildcard_name=<value>))
I cannot find a similar functionality when using a Snakemake module for resolving filenames relative a modules workdir.
Q: Is there any solution for indicating an output is from an imported snakemake module and constructing the output path from workdir as we had with subworkflow?

Related

Create symbolic links after building using CMAKE

I want a add_custom_target to run a loop over some files to create several symbolic links (and maybe doing more stuff).
My approach (maybe not the best?) has been:
add_custom_target(myTargetNAme-link echo "HERE" # This is printed
COMMAND
for file in /mylocation/*
do
### Create links using $file
done
DEPENDS dependency
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
COMMENT "Create a link to use files in dev env" VERBATIM
)
And I get the following error:
/bin/sh: -c: line 1: syntax error: unexpected end of file
Is this a smart way to do operations over files?
What's the right syntax for a foor loop like this in CMAKE?
EDIT
Following the comment in the question, I'd like to be more precise:
I have a project that produces a .so file containing a python module AND it should come with a couple .py files.
What I really want to do is to create a symbolic link from the .so and .py files to the python installation.
I was reusing some code, these targets were called in a script using:
nice -n 11 ninja myTargetNAme-link
Maybe I could avoid doing this as well?
ALSO
I deleted the VERBATIM keyword and it made it possible to use *.py for ln so now the symbolic link are actually generated.

How to get the working directory of the cmake process?

How to get the working directory from which cmake was called?
I know that I can run execute_process() at the beginning of my CMake code but maybe there is some better way like a built-in variable to the working directory of the CMake process.
What I want to achieve is to convert a relative path given in a CMake variable (given with -D...) to an absolute path in CMake.
What I want to achieve is to convert a relative path given in a CMake variable (given with -D...) to an absolute path in CMake.
Then simply create a cache variable of type PATH (for directories) or FILEPATH in your CMakeLists.txt. CMake will convert untyped command line arguments to absolute paths automatically.
$ cat CMakeLists.txt
cmake_minimum_required(VERSION 3.22)
project(test NONE)
set(input_file "" CACHE FILEPATH "some input file")
message(STATUS "${input_file}")
$ cmake -S . -B build -Dinput_file=foo.txt
-- /path/to/foo.txt
-- Configuring done
-- Generating done
-- Build files have been written to: /path/to/build
See the documentation:
It is possible for the cache entry to exist prior to the call but have no type set if it was created on the cmake(1) command line by a user through the -D<var>=<value> option without specifying a type. In this case the set command will add the type. Furthermore, if the <type> is PATH or FILEPATH and the <value> provided on the command line is a relative path, then the set command will treat the path as relative to the current working directory and convert it to an absolute path.
Here: https://cmake.org/cmake/help/latest/command/set.html?highlight=current%20working%20directory
Relatedly, when running in script mode (rather than project mode), several of the usual variables are set to the current working directory:
When run in -P script mode, CMake sets the variables CMAKE_BINARY_DIR, CMAKE_SOURCE_DIR, CMAKE_CURRENT_BINARY_DIR and CMAKE_CURRENT_SOURCE_DIR to the current working directory.
https://cmake.org/cmake/help/latest/variable/CMAKE_SOURCE_DIR.html

how to make Cmake output relative path?

In my last deleted question about CMake, I raised a problem about how to make CMake output relative path because vscode's problem panel only
understanding Windows directories or relative paths under workspace.
This won't happen unless some specific situation. If you use vsocde to edit source files in Windows and use WSL as terminal to compile using CMake, you will find vscode cannot work perfectly.
the problem panel will tell you it could not find the error or warning's source code after compilation completes.
gdb could not find source file when debugging.
(1)Solution to the first problem:Add the following code to your outermost CMakeLists.txt:
set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE "${CMAKE_SOURCE_DIR}/custom_output.sh")
The global RULE_LAUNCH_COMPILE property is set to a custom launch script named custom_output.sh which needs to be added to the root of the CMake source tree:
#!/bin/bash
# shell script invoked with the following arguments
# $(CXX) $(CXX_DEFINES) $(CXX_FLAGS) -o OBJECT_FILE -c SOURCE_FILE
# extract parameters
SOURCE_FILE="${#: -1:1}"
OBJECT_FILE="${#: -3:1}"
# invoke compiler
{ "$#" 2> >(sed 's#^/mnt/d/demo/##'|sed "s/warning/${esc}[32m&${esc}[0m/g"|sed "s/error/${esc}[31m&${esc}[0m/g" >&3); } 3>&2
it would output stderr messages also on stderr and transform it to relative path.Where "/mnt/d/demo/" is the string to be deleted. What to delete depends on the source code relative path.
(2)You can use the following command from gdb for remapping to solve the second problem:
set substitute-path old_path new_path
You need add follow code to you launch.json
"customLaunchSetupCommands": [
{
"text": "set substitute-path /mnt/d d:/",
"description": "change directory to workspace",
"ignoreFailures": false
}
]
If you debug with Cortex Debug you should add follw code to you launch.json
"postLaunchCommands": ["set substitute-path /mnt/d d:/"]

How to compile modules inside directories (aka with dots in its name)

I would like to import a module as
Require Import Foo.Bar.
Given that I have a file Bar.v inside directory Foo.
I am currently compiling this module with:
$ coqc Foo/Bar.v
When I try to Require Import Foo.Bar. I get this error:
Error: The file Foo/Bar.vo contains library Bar
and not library Foo.Bar
You can do this by using the -R option. Compile your file using coqc -R Foo Foo Foo/Bar.v. The -R flag takes two options: (1) the directory you want to add to your include path, and (2) the name you want to give in in the module namespace.
Later on, if you have some other file Baz.v that uses Foo.Bar, compile it using coqc -R Foo Foo Baz.v.
If you have one big project with many subdirectories, you can use coq_makefile, you can also use the -R in the toplevel to make the names consistent for all subdirectories once and for all. Have a look for instance at the Makefile for our project, with its corresponding coq_makefile source.

Linux kernel with include

I writing the following for kernel module building:
KDIR=/lib/modules/$(shell uname -r)/build
obj-m=hello.o
PWD= $(shell pwd)
default:
make -C $(KDIR) SUBDIRS=$(PWD) modules
It's work fine. I'm trying to rewrite this make-file using include directive:
KDIR=/lib/modules/$(shell uname -r)/build
obj-m=hello.o
PWD= $(shell pwd)
SUBDIRS=$(PWD)
include $(KDIR)/Makefile
But the following errors are caused:
/lib/modules/3.5.0-17-generic/build/Makefile:323: /home/toker/bundocode/gettingstart/kernel/superhello/scripts/Kbuild.include: No such file or directory
/lib/modules/3.5.0-17-generic/build/Makefile:573: /home/toker/bundocode/gettingstart/kernel/superhello/arch/x86/Makefile: No such file or directory
/bin/bash: /home/toker/bundocode/gettingstart/kernel/superhello/scripts/gcc-goto.sh: No such file or directory
Why?
Kernel makefiles are intended to be included from a parent makefile which contains all of the necessary information to build and link - including the information you were looking to get from your include. This allows the main kbuild process to decide which compiler, which architecture, etc it is building for, based on what the user configures via menuconfig. Getting these definitions from an out-of-tree makefile would undermine that (and would reduce portability of code).
You should read this for information: https://www.kernel.org/doc/Documentation/kbuild/makefiles.txt
John