Skip to content

feat: adopt nf-core-utils plugin + Nextflow 26 record types#220

Draft
edmundmiller wants to merge 2 commits into
devfrom
migrate-dev-nf-core-utils-records
Draft

feat: adopt nf-core-utils plugin + Nextflow 26 record types#220
edmundmiller wants to merge 2 commits into
devfrom
migrate-dev-nf-core-utils-records

Conversation

@edmundmiller

@edmundmiller edmundmiller commented Jun 22, 2026

Copy link
Copy Markdown
Collaborator

Summary

Migrates nf-core/nascent to (1) the nf-core-utils plugin and (2) the new Nextflow record types. Because record types require Nextflow 26.04's v2 parser, this necessarily ports the pipeline to Nextflow 26.04 (the v2 parser is stricter pipeline-wide).

Warning

Draft. The core migration is verified locally, but full nf-test/snapshot validation needs x86 CI — see Verification below. Opening as draft so CI (x86) can act as the arbiter.

What changed

nf-core-utils plugin (v0.5.0)

  • Adds id 'nf-core-utils@0.5.0' to plugins{}.
  • Replaces the shared utils_nfcore_pipeline / utils_nextflow_pipeline / utils_nfschema_plugin subworkflows with direct plugin/nf-core-utils function calls (getWorkflowVersion, checkConfigProvided, checkProfileProvided, checkCondaChannels, completionEmail, completionSummary, imNotification, dumpParametersToJSON, paramsSummaryMultiqc, methodsDescriptionText, softwareVersionsToYAML, getGenomeAttribute) and removes those subworkflows.
  • Uses v0.5.0 signatures: dumpParametersToJSON(outdir, params), getWorkflowVersion(manifest.version, commitId).

Nextflow record types

  • Declares record Sample { id; strandedness; single_end }; constructs it via the record() builtin at the samplesheet boundary (Channel.fromList preserved).
  • Enables nextflow.enable.types. Records flow through legacy val(meta) nf-core modules unchanged (a record is an immutable Map).

Nextflow 26 compatibility

  • nextflowVersion '!>=24.04.2''!>=26.04.0'; CI NXF_VER floor → 26.04.0.
  • v2 strict-syntax fixes: def declarations + de-shadowed variables (grohmm, coverage_graphs, conf/modules.config, star/align), path("bwa") in bwa/index, and PINTS_VISUALIZER input arity ([meta, bam]).

Verification

Verified locally on Nextflow 26.04.4:

  • nextflow config parses.
  • ✅ Pipeline executes to completed=27 under NF26 — records, plugin, and samplesheet parsing all functional. (For contrast, the current dev branch does not parse under NF26: nextflow.config: 'manifest' is not defined.)

Deferred to x86 CI (could not be completed on the local arm64/macOS dev machine):

  • ⚠️ Full nf-test + snapshots — local runs hit the pre-existing PINTS_VISUALIZER "no signal" data issue (already documented in bwa.nf.test: "FIXME PINTS Fails because it doesn't find anything"), which is unrelated to this migration and appears arm64-specific here. Snapshots will need regenerating for the NF26 task graph.

Known follow-ups before merge

  • Regenerate modules/nf-core/bwa/index/bwa-index.diffdone (2a32f6e): regenerated by hand against the pinned upstream SHA (local nf-core modules patch CLI crashes with OSError in this sandbox); verified the patch applies cleanly and reproduces the local module.
  • Confirm nf-test/snapshots on x86 CI; regenerate snapshots.
  • Consider extending the test PINTS skip to COVERAGE_GRAPHS:PINTS_VISUALIZER (pre-existing flakiness).

🤖 Generated with Claude Code

Migrates the pipeline off the shared nf-core utils subworkflows to the
nf-core-utils plugin and introduces the new Nextflow record types. Because
record types require Nextflow 26.04's v2 parser, this also ports the pipeline
to NF26 (the v2 parser is stricter pipeline-wide).

nf-core-utils plugin:
- Add `id 'nf-core-utils@0.5.0'` to plugins{}; import getWorkflowVersion,
  checkConfigProvided/checkProfileProvided, checkCondaChannels, completionEmail,
  completionSummary, imNotification, dumpParametersToJSON, paramsSummaryMultiqc,
  methodsDescriptionText, softwareVersionsToYAML, getGenomeAttribute from
  plugin/nf-core-utils.
- Replace UTILS_NFCORE_PIPELINE / UTILS_NEXTFLOW_PIPELINE / UTILS_NFSCHEMA_PLUGIN
  wrapper subworkflows with direct plugin calls; remove those subworkflows.
- Match v0.5.0 signatures: dumpParametersToJSON(outdir, params),
  getWorkflowVersion(manifest.version, commitId).

Record types:
- Declare `record Sample { id; strandedness; single_end }`; construct via the
  record() builtin at the samplesheet boundary (Channel.fromList preserved).
- Enable nextflow.enable.types; records flow through legacy val(meta) modules.

NF26 compatibility:
- nextflowVersion '!>=24.04.2' -> '!>=26.04.0'; CI NXF_VER floor -> 26.04.0.
- v2 strict-syntax fixes: def declarations and de-shadowed vars (grohmm,
  coverage_graphs, conf/modules.config, star/align), path("bwa") in bwa/index,
  PINTS_VISUALIZER input arity ([meta, bam]).

Verified locally (NF 26.04.4): `nextflow config` parses; pipeline executes to
completed=27 (records, plugin, samplesheet all functional). Full nf-test/snapshot
validation deferred to x86 CI (see PR notes).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@nf-core-bot

Copy link
Copy Markdown
Member

Warning

Newer version of the nf-core template is available.

Your pipeline is using an old version of the nf-core template: 3.2.1.
Please update your pipeline to the latest version.

For more documentation on how to update your pipeline, please see the nf-core documentation and Synchronisation documentation.

The committed bwa-index.diff was stale (memory 5.5.B, old format) and did
not capture the NF26 path("bwa") output fix. Regenerated against the pinned
upstream sha; verified the patch applies cleanly and reproduces the local
module. Resolves the nf-core lint follow-up noted in the PR.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants