publish two separate result directories from nextflow process - nextflow

Question

I have a nextflow process that outputs a result folder and a log as below:
process test {
label "x"
input:
path(somefile)
output:
path "test_results"
path "test_log.log"
path "something_for_next_process", emit: f
shell:
'''
myshellcommand
'''
}
And a config file to publish the results like below.
process
{
withLabel : x
{
publishDir =
[
path: {"$params.outdir/"},
pattern: "*_results",
mode: 'copy',
saveAs: {filename -> "${filename.split('_')[0]}/all_results/${filename.split("_")[1]}"}
]
}
withLabel : x
{
publishDir =
[
path: {"$params.outdir/"},
pattern: "*.log",
mode: 'copy',
saveAs: {filename -> "${filename.split('_')[0]}/logs/${filename}"}
]
}
}
I tried multiple combinations however, I can't get a label to publish its desired contents to two different folders. It always takes whichever is the last one in the publish config (in this case the logs). I know I can just put the publishdir options to the process in the workflow and that also works but I would like to do it through the config file. Is there a way I can get this to work?

Answer 1

You can specify the publishDir directive more than once in your nextflow.config by supplying a list of maps, for example:
process {
withLabel: 'x' {
publishDir = [
[
path: { "$params.outdir/" },
pattern: "*_results",
mode: 'copy',
saveAs: { fn -> "${fn.split('_')[0]}/all_results/${fn.split("_")[1]}" }
],
[
path: { "$params.outdir/" },
pattern: "*.log",
mode: 'copy',
saveAs: { fn -> "${fn.split('_')[0]}/logs/${fn}" }
]
]
}
}
If you don't need the withName or withLabel process selectors, the publishDir directive can just be specified multiple times inside of your process definition, for example:
process test {
publishDir(
path: { "$params.outdir/" },
pattern: "*_results",
mode: 'copy',
saveAs: { fn -> "${fn.split('_')[0]}/all_results/${fn.split("_")[1]}" }
)
publishDir(
path: { "$params.outdir/" },
pattern: "*.log",
mode: 'copy',
saveAs: { fn -> "${fn.split('_')[0]}/logs/${fn}" }
)
...
}

Answer 2

You can have multiple publishDir directives for the same process. No need to have the withLabel twice. If it was in the process block, it would be like the snippet below, for example.
process foo {
publishDir "$params.outdir/$sampleId/counts", pattern: "*_counts.txt"
publishDir "$params.outdir/$sampleId/outlooks", pattern: '*_outlook.txt'
publishDir "$params.outdir/$sampleId/", pattern: '*.fq'
input:
Check an example here.