Skip to content

Add documentation for non-argument module inputs #4210

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
itrujnara wants to merge 12 commits into
base: main
Choose a base branch
from
+43 −0
Open

Add documentation for non-argument module inputs #4210

Changes from 3 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
5091b7a
feat(docs): add non-argument `val` channel inputs section to input/ou…
itrujnara May 13, 2026
a3c89df
[automated] Fix code linting
nf-core-bot May 13, 2026
0b6d36f
fix(docs): clarify usage of `val` inputs and subcommands in input/out…
itrujnara May 13, 2026
93b093c
feat(docs): enhance input/output options documentation with examples …
itrujnara May 13, 2026
443237e
feat(docs): add example section for `val` channel inputs in input/out…
itrujnara May 13, 2026
ee660ec
[automated] Fix code linting
nf-core-bot May 13, 2026
6f977eb
fix(docs): remove redundant example section heading from `val` channe…
itrujnara May 13, 2026
e602cc2
[automated] Fix code linting
nf-core-bot May 13, 2026
7c91833
Apply suggestions from code review
itrujnara May 18, 2026
79ab1ee
fix(docs): clarify output format handling in `val` channel inputs doc…
itrujnara May 18, 2026
ab8e427
Merge branch 'main' into main
itrujnara May 18, 2026
5d1655f
Merge branch 'main' into main
itrujnara May 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions ...docs/src/content/docs/specifications/components/modules/input-output-options.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,18 @@ When one and only one of multiple argument are required:

:::

## Non-argument `val` channel inputs

Input channel `val` declarations MAY be used to control behaviours of the module that cannot be expressed with arguments of the underlying tool.

@jfy133 jfy133 Jun 10, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Input channel `val` declarations MAY be used to control behaviours of the module that cannot be expressed with arguments of the underlying tool.
Input channel `val` declarations MAY be used to control optional but critical behaviours of the module that cannot be expressed with arguments of the underlying tool.

Is this what you mean?

At the moment reading this I'm not sure how this differs from the much more concise point 1 of the previous section (although we could have a an example):

Mandatory non-file inputs are options that the tool MUST have to be able to be run.

@itrujnara itrujnara Jun 10, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not quite what I mean. This section aims to codify existing practice for inputs like val action, val compress, and val out_fmt, which are common in modules, but lack appropriate documentation. These input are not really "an argument to the tool", but are still an important argument to the module.

@jfy133 jfy133 Jun 11, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I think I'm starting to follow now.

I guess then just a single additional sentence of:

'This can include parameters to control execution of the tool, outside of the tool parameters itself - such as an optional post-execution compression step (to adhere to nf-core specification XYZ)' or something like this?


- If a module can output arbitrary or numerous file formats, the output format SHOULD be provided through a channel. The module SHOULD NOT perform bespoke string manipulation on input paths.
Comment thread
itrujnara marked this conversation as resolved.
Outdated
:::info{title="Rationale" collapse}

@jfy133 jfy133 Jun 10, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
:::info{title="Rationale" collapse}
:::info{title="Rationale" collapse}

@jfy133 jfy133 Jun 10, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could be an example moved to the previous section

Modules can encounter numerous input name scenarios. Custom string operations necessarily make assumptions about the name of the file (for example that the name of a compressed file has at least two dots). Providing an explicit format input returns full control to the pipeline developer and reduces the risk of unexpected behaviour.

@jfy133 jfy133 Jun 10, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Modules can encounter numerous input name scenarios. Custom string operations necessarily make assumptions about the name of the file (for example that the name of a compressed file has at least two dots). Providing an explicit format input returns full control to the pipeline developer and reduces the risk of unexpected behaviour.
Modules can encounter numerous input name scenarios.
Custom string operations necessarily make assumptions about the name of the file (for example that the name of a compressed file has at least two dots).
Providing an explicit format input returns full control to the pipeline developer and reduces the risk of unexpected behaviour.

@jfy133 jfy133 Jun 10, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't fully understand this point - what do you mean by 'nuermous input name scenarios'? and 'custom string operations' - for what? Is this sentence meant to be a justification here why you should have an explicit extension val channel to control this (as in the alternative methods cenarios are bad)?

@itrujnara itrujnara Jun 10, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This stems from my observations while reworking the bgzip modules. Some implementations assumed that the input name would always be something like variants.vcf.gz, so the module could "fish out" the output extension by splitting on dots and taking the second item from the end. However, it is not guaranteed that it will always be the case, and files could have names like database.gz, where database is likely not the desired extension. At least in my opinion, the module should be agnostic to the exact file name and use inputs to delegate the details to the pipeline. I hope I have explained it clearly enough in the draft.

@jfy133 jfy133 Jun 11, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HRm OK, I think you've made the explainations a bit too generic...

So does it ultimately boil down to:

If a module can emit mutually exclusive output formats of the same output file (e.g. bam/sam/cram), this SHOULD be explicitly defined using a dedicated input val channel?

:::
- If a module contains an optional pipe (for example: compression, sorting), the pipe SHOULD be controlled with a Boolean input channel.
Comment thread
mahesh-panchal marked this conversation as resolved.
Comment thread
muffato marked this conversation as resolved.
Comment thread
muffato marked this conversation as resolved.
Comment on lines +72 to +75

@jfy133 jfy133 Jun 10, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK the more I read about this, this feels mostly more about how to define output names, in which case I think the title of the section should be changed accordingly and rephrased to be more active of what you SHOULD do (a lot of this seems to be more defensive).

E.g., a seciton on output naming would be more like:

  1. Should use ${prefix}
  2. Avoid a default prefix as far as possible (pipeline developers should control this)
  3. Should not include custom strings except if necessary to define extension (controlled by input channel)

Or something like that

@jfy133 jfy133 Jun 10, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also technically this also overlaps with this section actually: https://nf-co.re/docs/specifications/components/modules/naming-conventions#command-file-output-naming

@itrujnara itrujnara Jun 10, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not meant to be a section on outputs, but rather on inputs. The current documentation largely suggests that every input to the module should be an input to the tool, which is clearly not the case. The involvement of output names is due to the drama of val out_fmt vs. ext.suffix (the former being the preferred practice according to maintainer discussions).

@itrujnara itrujnara Jun 10, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically what I want here is to move ideas from random Slack threads to the documentation

@jfy133 jfy133 Jun 11, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current documentation largely suggests that every input to the module should be an input to the tool, which is clearly not the case.

I don't think this is necessarily true: https://nf-co.re/docs/specifications/components/modules/input-output-options#required-val-channel-inputs ('essential for executuion of the tool' does not necessarily reuqire this to be an input to the tool itself - but we could make this clearer if yo uwanted?)


Non-standard `ext` fields (example: `ext.suffix`) SHOULD NOT be used to control module behaviour. `val` inputs SHOULD NOT be used for subcommands, separate modules SHOULD be used instead.
Comment thread
mahesh-panchal marked this conversation as resolved.

## Output channel emissions

Named file extensions MUST be emitted for ALL output channels.
Expand Down