From 33e71ca2e420c79c4b6eaaa2b6e2d799171df225 Mon Sep 17 00:00:00 2001 From: Jonathan Manning Date: Wed, 13 May 2026 12:39:19 +0100 Subject: [PATCH 1/4] docs: document orthogonal process resource labels Introduce the axis-decomposed labels (process_cpus_*, process_mem_*, process_time_*) as the preferred scheme and mark the bundled labels (process_low, process_medium, process_high) as legacy. Add a migration guide for module authors, pipeline maintainers and site-config maintainers. Refs: https://github.com/nf-core/proposals/issues/139 Refs: https://github.com/nf-core/tools/pull/4265 [skip ci] --- .../contribute-existing-pipelines.md | 12 ++- .../contributing/migrating-resource-labels.md | 98 +++++++++++++++++++ .../configuration/nextflow-for-your-system.md | 42 +++++--- .../modules/resource-requirements.md | 43 ++++++-- 4 files changed, 171 insertions(+), 24 deletions(-) create mode 100644 sites/docs/src/content/docs/contributing/migrating-resource-labels.md diff --git a/sites/docs/src/content/docs/contributing/contribute-existing-pipelines.md b/sites/docs/src/content/docs/contributing/contribute-existing-pipelines.md index 279d84749f..8dd761a81e 100644 --- a/sites/docs/src/content/docs/contributing/contribute-existing-pipelines.md +++ b/sites/docs/src/content/docs/contributing/contribute-existing-pipelines.md @@ -224,16 +224,18 @@ This adds your parameters to `nextflow_schema.json` with descriptions and valida ### Resource requirements -Define process resource requirements (CPUs, memory, time) in `conf/base.config` using `withLabel:` selectors: +Define process resource requirements (CPUs, memory, time) in `conf/base.config` using `withLabel:` selectors. The nf-core template ships axis-decomposed labels - pick at most one label per axis (CPU, memory, time) and combine them on each module: ```groovy -withLabel: process_low { - cpus = 2 - memory = 4.GB +withLabel:process_cpus_low { + cpus = { 2 * task.attempt } +} +withLabel:process_mem_low { + memory = { 1.GB * task.attempt } } ``` -Use standardized labels from the [nf-core pipeline template](https://github.com/nf-core/tools/blob/master/nf_core/pipeline-template/conf/base.config). +Use standardized labels from the [nf-core pipeline template](https://github.com/nf-core/tools/blob/main/nf_core/pipeline-template/conf/base.config). The full label catalogue and the deprecation status of the older bundled labels (`process_low`, `process_medium`, `process_high`) are documented in [Resource requirements](/docs/specifications/components/modules/resource-requirements). Reference resources dynamically in process blocks: ```groovy diff --git a/sites/docs/src/content/docs/contributing/migrating-resource-labels.md b/sites/docs/src/content/docs/contributing/migrating-resource-labels.md new file mode 100644 index 0000000000..ce8c435e78 --- /dev/null +++ b/sites/docs/src/content/docs/contributing/migrating-resource-labels.md @@ -0,0 +1,98 @@ +--- +title: Migrating to axis-decomposed resource labels +subtitle: Move pipelines, modules and site configs from bundled to axis-decomposed labels +shortTitle: Migrating resource labels +--- + +The nf-core pipeline template historically shipped a small set of bundled resource labels (`process_low`, `process_medium`, `process_high`, `process_long`, `process_high_memory`) that set CPU, memory and time together. These are being replaced by axis-decomposed labels (`process_cpus_*`, `process_mem_*`, `process_time_*`) that tune each axis independently. + +The bundled labels remain functional for now. Linting emits a warning when a module uses them, and they will be removed after a two-release deprecation window. + +This guide covers the migration steps for module authors, pipeline maintainers, and institutional site-config maintainers. The canonical definition of the label scheme lives in [Resource requirements](/docs/specifications/components/modules/resource-requirements). + +## Why migrate + +- Many tools scale on only one axis (CPU-bound, memory-bound, or wall-clock-bound). Bundled labels force authors to pick a tier that gives appropriate values on one axis and wildly over-provisions the others. +- Common shapes like "many CPUs, little memory" cannot be expressed cleanly with bundled labels. +- Pipelines have already invented ad-hoc additions (`process_high_cpu`, `process_high_memory`) to work around this, with no community-wide convention. + +## Default mapping + +Use this table as the starting point when migrating a module. The values are the closest equivalents in the current template; per-module judgment may override them, especially for `process_low` whose 12 GB memory default is closer to `process_mem_medium` than `process_mem_low`. + +| Legacy label | CPU | Memory | Time | +| --------------------- | --------------------- | -------------------- | --------------------- | +| `process_single` | `process_cpus_single` | (template default) | (template default) | +| `process_low` | `process_cpus_low` | `process_mem_medium` | (template default) | +| `process_medium` | `process_cpus_medium` | `process_mem_medium` | `process_time_medium` | +| `process_high` | `process_cpus_high` | `process_mem_high` | `process_time_long` | +| `process_long` | (unchanged) | (unchanged) | `process_time_long` | +| `process_high_memory` | (unchanged) | `process_mem_high` | (unchanged) | +| `process_low_memory` | (unchanged) | `process_mem_low` | (unchanged) | + +A module that previously declared `label 'process_high'` becomes: + +```groovy title="main.nf" +process FOO { + label 'process_cpus_high' + label 'process_mem_high' + label 'process_time_long' + ... +} +``` + +A module that previously stacked `label 'process_medium'` with `label 'process_high_memory'` becomes: + +```groovy title="main.nf" +process FOO { + label 'process_cpus_medium' + label 'process_mem_high' + label 'process_time_medium' + ... +} +``` + +## For module authors + +1. Review the tool's actual scaling behaviour. Bundled labels often over-provision on at least two axes; this is the chance to right-size the request. +2. Replace each legacy label on the process with the equivalent axis labels from the mapping table, adjusting per the tool's profile. +3. Drop any axis label that matches the template default - omitting a label is the canonical way to say "this axis does not need to be tuned". +4. Run `nf-core modules lint` to confirm no warnings remain. + +## For pipeline maintainers + +1. Update local modules in `modules/local/` using the same approach as above. +2. Override values in the pipeline's `conf/base.config` only where the tool genuinely needs different resources from the template defaults. +3. Drop pipeline-local label definitions that simply duplicate the template (the template's labels are inherited from the nf-core template sync). +4. Keep the bundled-label `withLabel:` blocks in `conf/base.config` until upstream tools-template sync removes them. + +## For institutional config maintainers + +Sites that ship overrides for `withLabel: process_*` in their config bundle need parallel overrides for the new axis labels. The mapping table above gives the default translation; institutions with custom tiering may want different values per axis. + +A minimal `sed` recipe to seed the new labels from existing overrides (always review the diff before committing): + +```bash +# duplicate each bundled-label override into axis overrides +sed -E ' + s/withLabel: ?process_low\b/withLabel:process_cpus_low/; + s/withLabel: ?process_medium\b/withLabel:process_cpus_medium/; + s/withLabel: ?process_high\b/withLabel:process_cpus_high/; +' conf/site.config +``` + +The recipe deliberately rewrites to CPU axes only; memory and time should be reviewed by hand because the bundled labels conflate values in ways that do not translate one-to-one. + +## Deprecation timeline + +| Stage | Status | +| -------- | ----------------------------------------------------------------------------------------------------- | +| Now | Axis labels and bundled labels both supported. Lint emits a warning on bundled labels. | +| +1 minor | Bundled labels still supported. nf-core/modules migrated. nf-core/configs migrated. | +| +2 minor | Bundled labels removed from the template. Pipelines that still reference them must define them locally or migrate. | + +## See also + +- [Resource requirements](/docs/specifications/components/modules/resource-requirements) +- RFC: [nf-core/proposals#139](https://github.com/nf-core/proposals/issues/139) +- Template change: [nf-core/tools#4265](https://github.com/nf-core/tools/pull/4265) diff --git a/sites/docs/src/content/docs/running/configuration/nextflow-for-your-system.md b/sites/docs/src/content/docs/running/configuration/nextflow-for-your-system.md index 8c23a50347..4da37cf15c 100644 --- a/sites/docs/src/content/docs/running/configuration/nextflow-for-your-system.md +++ b/sites/docs/src/content/docs/running/configuration/nextflow-for-your-system.md @@ -21,7 +21,7 @@ Pipelines are coded to configure tools to use available resources when possible Not all tools support dynamic resource configuration. ::: -Most nf-core pipelines use process labels to define resource requirements for each module, as shown in this base configuration example: +Most nf-core pipelines use process labels to define resource requirements for each module. Newer pipelines (built from the nf-core template that ships axis-decomposed labels) split the requirement across three independent axes - CPU, memory and time - so each can be tuned independently: ```groovy process { @@ -30,24 +30,38 @@ process { memory: 256.GB, time: 24.h ] - withLabel:process_low { - cpus = { 2 * task.attempt } - memory = { 14.GB * task.attempt } - time = { 6.h * task.attempt } + withLabel:process_cpus_low { + cpus = { 2 * task.attempt } } - withLabel:process_medium { + withLabel:process_cpus_medium { cpus = { 6 * task.attempt } - memory = { 42.GB * task.attempt } - time = { 8.h * task.attempt } } - withLabel:process_high { + withLabel:process_cpus_high { cpus = { 12 * task.attempt } - memory = { 84.GB * task.attempt } - time = { 10.h * task.attempt } + } + withLabel:process_mem_low { + memory = { 1.GB * task.attempt } + } + withLabel:process_mem_medium { + memory = { 12.GB * task.attempt } + } + withLabel:process_mem_high { + memory = { 72.GB * task.attempt } + } + withLabel:process_time_short { + time = { 1.h * task.attempt } + } + withLabel:process_time_medium { + time = { 8.h * task.attempt } + } + withLabel:process_time_long { + time = { 20.h * task.attempt } } } ``` +Older pipelines use bundled labels (`process_low`, `process_medium`, `process_high`) that set all three axes at once. The bundled labels remain functional but are scheduled for deprecation; see the [resource labels migration guide](/docs/contributing/migrating-resource-labels). + The `resourceLimits` list sets the absolute maximum resources any pipeline job can request (typically matching your machine's maximum available resources). The label blocks define the initial default resources each pipeline job requests. When a job runs out of memory, most nf-core pipelines will attempt to retry the job and increase the resource request up to the `resourceLimits` maximum. @@ -58,16 +72,18 @@ When a job runs out of memory, most nf-core pipelines will attempt to retry the Copy only the labels you want to change into your custom configuration file, not all labels. ::: -To set a fixed memory allocation for all large tasks across most nf-core pipelines (without increases during retries), add this to a custom Nextflow configuration file: +To set a fixed memory allocation for all memory-intensive tasks across most nf-core pipelines (without increases during retries), add this to a custom Nextflow configuration file: ```groovy process { - withLabel:process_high { + withLabel:process_mem_high { memory = 200.GB } } ``` +For pipelines that still use bundled labels, target `process_high` instead. + :::tip To find the default labels and resources of the pipeline you want to optimise, go to its GitHub repository and look at the file `conf/base.config`. ::: diff --git a/sites/docs/src/content/docs/specifications/components/modules/resource-requirements.md b/sites/docs/src/content/docs/specifications/components/modules/resource-requirements.md index 4443a78d71..34f8d1dd49 100644 --- a/sites/docs/src/content/docs/specifications/components/modules/resource-requirements.md +++ b/sites/docs/src/content/docs/specifications/components/modules/resource-requirements.md @@ -10,22 +10,53 @@ The keywords "MUST", "MUST NOT", "SHOULD", etc. are to be interpreted as describ ## Use of labels in modules -An appropriate resource `label` MUST be provided for the module as listed in the [nf-core pipeline template](https://github.com/nf-core/tools/blob/main/nf_core/pipeline-template/conf/base.config). +An appropriate set of resource `label` directives MUST be provided for the module as listed in the [nf-core pipeline template](https://github.com/nf-core/tools/blob/main/nf_core/pipeline-template/conf/base.config). -The template (since nf-core/tools:4.1.0) defines two kinds of labels: +The template defines two label schemes. New and updated modules SHOULD use the axis-decomposed labels; the bundled labels are retained for backwards compatibility and are scheduled for deprecation. -- **Bundled labels** set CPU, memory and time together. Use exactly one per module: `process_single`, `process_low`, `process_medium`, or `process_high`. -- **Modifier labels** override a single axis and SHOULD be stacked on top of a bundled label: `process_long` (extends time), `process_low_memory` (drops memory), `process_high_memory` (raises memory). +### Axis-decomposed labels (preferred) + +Three independent axes cover CPU, memory and time. A module picks at most one label per axis and combines them to describe its resource shape: + +| Axis | Labels (low to high) | +| ------ | ------------------------------------------------------------------------------------- | +| CPU | `process_cpus_single`, `process_cpus_low`, `process_cpus_medium`, `process_cpus_high` | +| Memory | `process_mem_low`, `process_mem_medium`, `process_mem_high` | +| Time | `process_time_short`, `process_time_medium`, `process_time_long` | + +A CPU-bound but memory-light short job declares: + +```groovy title="main.nf" +process FOO { + label 'process_cpus_high' + label 'process_mem_low' + label 'process_time_short' + ... +} +``` + +A module MAY omit a label on an axis it does not need to tune. Axes without an explicit label receive the per-process defaults defined at the top of `base.config`. + +A module MUST NOT declare more than one label on the same axis. `process_cpus_low` and `process_cpus_high` together is invalid and is rejected by the linter. + +### Bundled labels (legacy) + +The original label set ties CPU, memory and time together. Use exactly one per module: `process_single`, `process_low`, `process_medium`, or `process_high`. Two modifier labels override a single axis and SHOULD be stacked on top of a bundled label: `process_long` (extends time), `process_low_memory` (drops memory), `process_high_memory` (raises memory). + +These labels remain functional but emit a lint warning, and SHOULD be migrated to the axis-decomposed scheme. See the [migration guide](/docs/contributing/migrating-resource-labels) for the legacy-to-axis mapping table and a recipe for site-config maintainers. + +### Label precedence When a process matches more than one `withLabel:` block, Nextflow applies them in the order they appear in the config file. The order of the `label` directives in the process itself does not affect precedence. -The nf-core template defines bundled labels first and modifier labels after, so modifier labels always overwrite the corresponding bundled value. + +The nf-core template defines bundled labels first, axis-decomposed labels next, and the legacy modifier labels (`process_long`, `process_low_memory`, `process_high_memory`) last. Stacking a modifier on top of a bundled label resolves to the modifier's value on the axis it overrides, and the bundled values on the others. ## Source of multiple threads or cores value If the tool supports multi-threading, provide the appropriate parameter using the Nextflow `task` variable. For example, `--threads $task.cpus`. -If the tool does not support multi-threading, consider `process_single` unless large amounts of RAM are required. +If the tool does not support multi-threading, consider `process_cpus_single` unless large amounts of RAM are required. ## Specifying multiple threads for piped commands From 431f14be9518533afbff723aba5464ed725e1794 Mon Sep 17 00:00:00 2001 From: Jonathan Manning Date: Thu, 14 May 2026 10:22:18 +0100 Subject: [PATCH 2/4] docs: address @mashehu review on axis-labels migration guide [skip ci] - Move migrating-resource-labels.md to developing/migration-guides/process-axis-labels.md. - Tighten prose: drop redundant phrasing, merge module-author and pipeline-maintainer steps into a single section, sharpen the "Why migrate" bullets. - Convert the two before/after migration examples to diff groovy blocks matching the strict-syntax migration guide's convention. - Reformat the axis label table in resource-requirements.md so each tier is its own cell, with empty cells aligning the "single" tier on the CPU axis with the absent tier on memory/time. - Update the two internal links (nextflow-for-your-system.md, resource-requirements.md) to the new migration-guide path. Co-Authored-By: Claude Opus 4.7 (1M context) --- .../contributing/migrating-resource-labels.md | 98 ------------------- .../migration-guides/process-axis-labels.md | 95 ++++++++++++++++++ .../configuration/nextflow-for-your-system.md | 2 +- .../modules/resource-requirements.md | 12 +-- 4 files changed, 102 insertions(+), 105 deletions(-) delete mode 100644 sites/docs/src/content/docs/contributing/migrating-resource-labels.md create mode 100644 sites/docs/src/content/docs/developing/migration-guides/process-axis-labels.md diff --git a/sites/docs/src/content/docs/contributing/migrating-resource-labels.md b/sites/docs/src/content/docs/contributing/migrating-resource-labels.md deleted file mode 100644 index ce8c435e78..0000000000 --- a/sites/docs/src/content/docs/contributing/migrating-resource-labels.md +++ /dev/null @@ -1,98 +0,0 @@ ---- -title: Migrating to axis-decomposed resource labels -subtitle: Move pipelines, modules and site configs from bundled to axis-decomposed labels -shortTitle: Migrating resource labels ---- - -The nf-core pipeline template historically shipped a small set of bundled resource labels (`process_low`, `process_medium`, `process_high`, `process_long`, `process_high_memory`) that set CPU, memory and time together. These are being replaced by axis-decomposed labels (`process_cpus_*`, `process_mem_*`, `process_time_*`) that tune each axis independently. - -The bundled labels remain functional for now. Linting emits a warning when a module uses them, and they will be removed after a two-release deprecation window. - -This guide covers the migration steps for module authors, pipeline maintainers, and institutional site-config maintainers. The canonical definition of the label scheme lives in [Resource requirements](/docs/specifications/components/modules/resource-requirements). - -## Why migrate - -- Many tools scale on only one axis (CPU-bound, memory-bound, or wall-clock-bound). Bundled labels force authors to pick a tier that gives appropriate values on one axis and wildly over-provisions the others. -- Common shapes like "many CPUs, little memory" cannot be expressed cleanly with bundled labels. -- Pipelines have already invented ad-hoc additions (`process_high_cpu`, `process_high_memory`) to work around this, with no community-wide convention. - -## Default mapping - -Use this table as the starting point when migrating a module. The values are the closest equivalents in the current template; per-module judgment may override them, especially for `process_low` whose 12 GB memory default is closer to `process_mem_medium` than `process_mem_low`. - -| Legacy label | CPU | Memory | Time | -| --------------------- | --------------------- | -------------------- | --------------------- | -| `process_single` | `process_cpus_single` | (template default) | (template default) | -| `process_low` | `process_cpus_low` | `process_mem_medium` | (template default) | -| `process_medium` | `process_cpus_medium` | `process_mem_medium` | `process_time_medium` | -| `process_high` | `process_cpus_high` | `process_mem_high` | `process_time_long` | -| `process_long` | (unchanged) | (unchanged) | `process_time_long` | -| `process_high_memory` | (unchanged) | `process_mem_high` | (unchanged) | -| `process_low_memory` | (unchanged) | `process_mem_low` | (unchanged) | - -A module that previously declared `label 'process_high'` becomes: - -```groovy title="main.nf" -process FOO { - label 'process_cpus_high' - label 'process_mem_high' - label 'process_time_long' - ... -} -``` - -A module that previously stacked `label 'process_medium'` with `label 'process_high_memory'` becomes: - -```groovy title="main.nf" -process FOO { - label 'process_cpus_medium' - label 'process_mem_high' - label 'process_time_medium' - ... -} -``` - -## For module authors - -1. Review the tool's actual scaling behaviour. Bundled labels often over-provision on at least two axes; this is the chance to right-size the request. -2. Replace each legacy label on the process with the equivalent axis labels from the mapping table, adjusting per the tool's profile. -3. Drop any axis label that matches the template default - omitting a label is the canonical way to say "this axis does not need to be tuned". -4. Run `nf-core modules lint` to confirm no warnings remain. - -## For pipeline maintainers - -1. Update local modules in `modules/local/` using the same approach as above. -2. Override values in the pipeline's `conf/base.config` only where the tool genuinely needs different resources from the template defaults. -3. Drop pipeline-local label definitions that simply duplicate the template (the template's labels are inherited from the nf-core template sync). -4. Keep the bundled-label `withLabel:` blocks in `conf/base.config` until upstream tools-template sync removes them. - -## For institutional config maintainers - -Sites that ship overrides for `withLabel: process_*` in their config bundle need parallel overrides for the new axis labels. The mapping table above gives the default translation; institutions with custom tiering may want different values per axis. - -A minimal `sed` recipe to seed the new labels from existing overrides (always review the diff before committing): - -```bash -# duplicate each bundled-label override into axis overrides -sed -E ' - s/withLabel: ?process_low\b/withLabel:process_cpus_low/; - s/withLabel: ?process_medium\b/withLabel:process_cpus_medium/; - s/withLabel: ?process_high\b/withLabel:process_cpus_high/; -' conf/site.config -``` - -The recipe deliberately rewrites to CPU axes only; memory and time should be reviewed by hand because the bundled labels conflate values in ways that do not translate one-to-one. - -## Deprecation timeline - -| Stage | Status | -| -------- | ----------------------------------------------------------------------------------------------------- | -| Now | Axis labels and bundled labels both supported. Lint emits a warning on bundled labels. | -| +1 minor | Bundled labels still supported. nf-core/modules migrated. nf-core/configs migrated. | -| +2 minor | Bundled labels removed from the template. Pipelines that still reference them must define them locally or migrate. | - -## See also - -- [Resource requirements](/docs/specifications/components/modules/resource-requirements) -- RFC: [nf-core/proposals#139](https://github.com/nf-core/proposals/issues/139) -- Template change: [nf-core/tools#4265](https://github.com/nf-core/tools/pull/4265) diff --git a/sites/docs/src/content/docs/developing/migration-guides/process-axis-labels.md b/sites/docs/src/content/docs/developing/migration-guides/process-axis-labels.md new file mode 100644 index 0000000000..9fe2dfff15 --- /dev/null +++ b/sites/docs/src/content/docs/developing/migration-guides/process-axis-labels.md @@ -0,0 +1,95 @@ +--- +title: Migrating to axis-decomposed resource labels +subtitle: Move pipelines, modules and site configs from bundled to axis-decomposed labels +description: How to update modules, pipelines and site configs from the bundled resource labels (process_low etc.) to the axis-decomposed labels (process_cpus_*, process_mem_*, process_time_*). +shortTitle: Migrating resource labels +--- + +The bundled resource labels (`process_low`, `process_medium`, `process_high`, `process_long`, `process_high_memory`) are being replaced by axis-decomposed labels (`process_cpus_*`, `process_mem_*`, `process_time_*`) that tune CPU, memory and time independently. Bundled labels still work and emit a lint warning; they are removed after a two-release deprecation window. + +The full scheme is defined in [Resource requirements](/docs/specifications/components/modules/resource-requirements); the rationale is in [nf-core/proposals#139](https://github.com/nf-core/proposals/issues/139). + +## Why migrate + +- Most tools scale on only one axis; bundled labels over-provision the others, often by hundreds of times. +- Common shapes like "many CPUs, little memory" or "moderate CPUs, high memory" cannot be expressed cleanly with bundled labels. +- Pipelines have been working around this with locally invented labels (`process_high_cpu` etc.), which diverge across the ecosystem. + +## Default mapping + +Start from this table. The `process_low` row is opinionated: its 12 GB legacy default is closer to `process_mem_medium` than `process_mem_low`, so module authors should sanity-check the choice rather than apply it mechanically. + +| Legacy label | CPU | Memory | Time | +| --------------------- | --------------------- | -------------------- | --------------------- | +| `process_single` | `process_cpus_single` | (template default) | (template default) | +| `process_low` | `process_cpus_low` | `process_mem_medium` | (template default) | +| `process_medium` | `process_cpus_medium` | `process_mem_medium` | `process_time_medium` | +| `process_high` | `process_cpus_high` | `process_mem_high` | `process_time_long` | +| `process_long` | (unchanged) | (unchanged) | `process_time_long` | +| `process_high_memory` | (unchanged) | `process_mem_high` | (unchanged) | +| `process_low_memory` | (unchanged) | `process_mem_low` | (unchanged) | + +A `process_high` module becomes: + +```diff groovy title="main.nf" + process FOO { +- label 'process_high' ++ label 'process_cpus_high' ++ label 'process_mem_high' ++ label 'process_time_long' + ... + } +``` + +A `process_medium` + `process_high_memory` stack becomes: + +```diff groovy title="main.nf" + process FOO { +- label 'process_medium' +- label 'process_high_memory' ++ label 'process_cpus_medium' ++ label 'process_mem_high' ++ label 'process_time_medium' + ... + } +``` + +Omit any axis label that matches the template default; that is the canonical way to say "this axis does not need to be tuned". + +## Modules and pipelines + +1. Replace each legacy label on the process with the axis labels from the table, adjusting per the tool's real scaling profile. +2. Apply the same change to any `conf/base.config` overrides in the pipeline, and drop pipeline-local label definitions that just duplicate the template. +3. Run `nf-core modules lint` to confirm the new labels are recognised. + +The bundled-label `withLabel:` blocks in the template's `base.config` are removed by the nf-core template sync; they should not be edited by hand. + +## Institutional site configs + +Sites that override `withLabel: process_*` need parallel overrides for the new axis labels. The default translation is the mapping table above; institutions with custom tiering may want different values per axis. + +A `sed` recipe to seed the CPU axis from existing overrides (always review the diff before committing): + +```bash +sed -E ' + s/withLabel: ?process_low\b/withLabel:process_cpus_low/; + s/withLabel: ?process_medium\b/withLabel:process_cpus_medium/; + s/withLabel: ?process_high\b/withLabel:process_cpus_high/; +' conf/site.config +``` + +The recipe rewrites the CPU axis only. Memory and time overrides do not translate one-to-one and need review by hand. + +## Deprecation timeline + +| Stage | Status | +| -------- | ----------------------------------------------------------------------------------------------------- | +| Now | Both schemes supported. Lint warns on bundled labels. | +| +1 minor | Both schemes supported. nf-core/modules and nf-core/configs migrated. | +| +2 minor | Bundled labels removed from the template. Pipelines that still use them must define them locally. | + +## See also + +- [Resource requirements](/docs/specifications/components/modules/resource-requirements) +- RFC: [nf-core/proposals#139](https://github.com/nf-core/proposals/issues/139) +- Template change: [nf-core/tools#4265](https://github.com/nf-core/tools/pull/4265) diff --git a/sites/docs/src/content/docs/running/configuration/nextflow-for-your-system.md b/sites/docs/src/content/docs/running/configuration/nextflow-for-your-system.md index 4da37cf15c..0bee7e08b6 100644 --- a/sites/docs/src/content/docs/running/configuration/nextflow-for-your-system.md +++ b/sites/docs/src/content/docs/running/configuration/nextflow-for-your-system.md @@ -60,7 +60,7 @@ process { } ``` -Older pipelines use bundled labels (`process_low`, `process_medium`, `process_high`) that set all three axes at once. The bundled labels remain functional but are scheduled for deprecation; see the [resource labels migration guide](/docs/contributing/migrating-resource-labels). +Older pipelines use bundled labels (`process_low`, `process_medium`, `process_high`) that set all three axes at once. The bundled labels remain functional but are scheduled for deprecation; see the [resource labels migration guide](/docs/developing/migration-guides/process-axis-labels). The `resourceLimits` list sets the absolute maximum resources any pipeline job can request (typically matching your machine's maximum available resources). The label blocks define the initial default resources each pipeline job requests. diff --git a/sites/docs/src/content/docs/specifications/components/modules/resource-requirements.md b/sites/docs/src/content/docs/specifications/components/modules/resource-requirements.md index 34f8d1dd49..9e942d3093 100644 --- a/sites/docs/src/content/docs/specifications/components/modules/resource-requirements.md +++ b/sites/docs/src/content/docs/specifications/components/modules/resource-requirements.md @@ -18,11 +18,11 @@ The template defines two label schemes. New and updated modules SHOULD use the a Three independent axes cover CPU, memory and time. A module picks at most one label per axis and combines them to describe its resource shape: -| Axis | Labels (low to high) | -| ------ | ------------------------------------------------------------------------------------- | -| CPU | `process_cpus_single`, `process_cpus_low`, `process_cpus_medium`, `process_cpus_high` | -| Memory | `process_mem_low`, `process_mem_medium`, `process_mem_high` | -| Time | `process_time_short`, `process_time_medium`, `process_time_long` | +| Axis | Labels (low to high) | | | | +| ------ | --------------------- | -------------------- | --------------------- | ------------------- | +| CPU | `process_cpus_single` | `process_cpus_low` | `process_cpus_medium` | `process_cpus_high` | +| Memory | | `process_mem_low` | `process_mem_medium` | `process_mem_high` | +| Time | | `process_time_short` | `process_time_medium` | `process_time_long` | A CPU-bound but memory-light short job declares: @@ -43,7 +43,7 @@ A module MUST NOT declare more than one label on the same axis. `process_cpus_lo The original label set ties CPU, memory and time together. Use exactly one per module: `process_single`, `process_low`, `process_medium`, or `process_high`. Two modifier labels override a single axis and SHOULD be stacked on top of a bundled label: `process_long` (extends time), `process_low_memory` (drops memory), `process_high_memory` (raises memory). -These labels remain functional but emit a lint warning, and SHOULD be migrated to the axis-decomposed scheme. See the [migration guide](/docs/contributing/migrating-resource-labels) for the legacy-to-axis mapping table and a recipe for site-config maintainers. +These labels remain functional but emit a lint warning, and SHOULD be migrated to the axis-decomposed scheme. See the [migration guide](/docs/developing/migration-guides/process-axis-labels) for the legacy-to-axis mapping table and a recipe for site-config maintainers. ### Label precedence From 940eba0c6544db74c7a138b599833c25fe3b1021 Mon Sep 17 00:00:00 2001 From: Jonathan Manning Date: Thu, 14 May 2026 10:24:04 +0100 Subject: [PATCH 3/4] docs: trigger CI / fresh deploy preview after path rename Co-Authored-By: Claude Opus 4.7 (1M context) From 15614699993393786d5a620eb82e39941b9cc718 Mon Sep 17 00:00:00 2001 From: Jonathan Manning Date: Thu, 14 May 2026 10:28:54 +0100 Subject: [PATCH 4/4] docs: prettier-format the deprecation timeline table prettier wants table cells padded to the longest content + 2 spaces; the previous version had a few stray trailing spaces. Co-Authored-By: Claude Opus 4.7 (1M context) --- .../developing/migration-guides/process-axis-labels.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/sites/docs/src/content/docs/developing/migration-guides/process-axis-labels.md b/sites/docs/src/content/docs/developing/migration-guides/process-axis-labels.md index 9fe2dfff15..2e449b23fa 100644 --- a/sites/docs/src/content/docs/developing/migration-guides/process-axis-labels.md +++ b/sites/docs/src/content/docs/developing/migration-guides/process-axis-labels.md @@ -82,11 +82,11 @@ The recipe rewrites the CPU axis only. Memory and time overrides do not translat ## Deprecation timeline -| Stage | Status | -| -------- | ----------------------------------------------------------------------------------------------------- | -| Now | Both schemes supported. Lint warns on bundled labels. | -| +1 minor | Both schemes supported. nf-core/modules and nf-core/configs migrated. | -| +2 minor | Bundled labels removed from the template. Pipelines that still use them must define them locally. | +| Stage | Status | +| -------- | ------------------------------------------------------------------------------------------------- | +| Now | Both schemes supported. Lint warns on bundled labels. | +| +1 minor | Both schemes supported. nf-core/modules and nf-core/configs migrated. | +| +2 minor | Bundled labels removed from the template. Pipelines that still use them must define them locally. | ## See also