feat: adopt nf-core-utils plugin + Nextflow 26 record types#220
Draft
edmundmiller wants to merge 2 commits into
Draft
feat: adopt nf-core-utils plugin + Nextflow 26 record types#220edmundmiller wants to merge 2 commits into
edmundmiller wants to merge 2 commits into
Conversation
Migrates the pipeline off the shared nf-core utils subworkflows to the
nf-core-utils plugin and introduces the new Nextflow record types. Because
record types require Nextflow 26.04's v2 parser, this also ports the pipeline
to NF26 (the v2 parser is stricter pipeline-wide).
nf-core-utils plugin:
- Add `id 'nf-core-utils@0.5.0'` to plugins{}; import getWorkflowVersion,
checkConfigProvided/checkProfileProvided, checkCondaChannels, completionEmail,
completionSummary, imNotification, dumpParametersToJSON, paramsSummaryMultiqc,
methodsDescriptionText, softwareVersionsToYAML, getGenomeAttribute from
plugin/nf-core-utils.
- Replace UTILS_NFCORE_PIPELINE / UTILS_NEXTFLOW_PIPELINE / UTILS_NFSCHEMA_PLUGIN
wrapper subworkflows with direct plugin calls; remove those subworkflows.
- Match v0.5.0 signatures: dumpParametersToJSON(outdir, params),
getWorkflowVersion(manifest.version, commitId).
Record types:
- Declare `record Sample { id; strandedness; single_end }`; construct via the
record() builtin at the samplesheet boundary (Channel.fromList preserved).
- Enable nextflow.enable.types; records flow through legacy val(meta) modules.
NF26 compatibility:
- nextflowVersion '!>=24.04.2' -> '!>=26.04.0'; CI NXF_VER floor -> 26.04.0.
- v2 strict-syntax fixes: def declarations and de-shadowed vars (grohmm,
coverage_graphs, conf/modules.config, star/align), path("bwa") in bwa/index,
PINTS_VISUALIZER input arity ([meta, bam]).
Verified locally (NF 26.04.4): `nextflow config` parses; pipeline executes to
completed=27 (records, plugin, samplesheet all functional). Full nf-test/snapshot
validation deferred to x86 CI (see PR notes).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Member
|
Warning Newer version of the nf-core template is available. Your pipeline is using an old version of the nf-core template: 3.2.1. For more documentation on how to update your pipeline, please see the nf-core documentation and Synchronisation documentation. |
The committed bwa-index.diff was stale (memory 5.5.B, old format) and did
not capture the NF26 path("bwa") output fix. Regenerated against the pinned
upstream sha; verified the patch applies cleanly and reproduces the local
module. Resolves the nf-core lint follow-up noted in the PR.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Migrates nf-core/nascent to (1) the
nf-core-utilsplugin and (2) the new Nextflow record types. Because record types require Nextflow 26.04's v2 parser, this necessarily ports the pipeline to Nextflow 26.04 (the v2 parser is stricter pipeline-wide).Warning
Draft. The core migration is verified locally, but full
nf-test/snapshot validation needs x86 CI — see Verification below. Opening as draft so CI (x86) can act as the arbiter.What changed
nf-core-utils plugin (v0.5.0)
id 'nf-core-utils@0.5.0'toplugins{}.utils_nfcore_pipeline/utils_nextflow_pipeline/utils_nfschema_pluginsubworkflows with directplugin/nf-core-utilsfunction calls (getWorkflowVersion,checkConfigProvided,checkProfileProvided,checkCondaChannels,completionEmail,completionSummary,imNotification,dumpParametersToJSON,paramsSummaryMultiqc,methodsDescriptionText,softwareVersionsToYAML,getGenomeAttribute) and removes those subworkflows.dumpParametersToJSON(outdir, params),getWorkflowVersion(manifest.version, commitId).Nextflow record types
record Sample { id; strandedness; single_end }; constructs it via therecord()builtin at the samplesheet boundary (Channel.fromListpreserved).nextflow.enable.types. Records flow through legacyval(meta)nf-core modules unchanged (a record is an immutableMap).Nextflow 26 compatibility
nextflowVersion '!>=24.04.2'→'!>=26.04.0'; CINXF_VERfloor →26.04.0.defdeclarations + de-shadowed variables (grohmm,coverage_graphs,conf/modules.config,star/align),path("bwa")inbwa/index, andPINTS_VISUALIZERinput arity ([meta, bam]).Verification
Verified locally on Nextflow 26.04.4:
nextflow configparses.completed=27under NF26 — records, plugin, and samplesheet parsing all functional. (For contrast, the currentdevbranch does not parse under NF26:nextflow.config: 'manifest' is not defined.)Deferred to x86 CI (could not be completed on the local arm64/macOS dev machine):
nf-test+ snapshots — local runs hit the pre-existingPINTS_VISUALIZER"no signal" data issue (already documented inbwa.nf.test: "FIXME PINTS Fails because it doesn't find anything"), which is unrelated to this migration and appears arm64-specific here. Snapshots will need regenerating for the NF26 task graph.Known follow-ups before merge
Regenerate— done (modules/nf-core/bwa/index/bwa-index.diff2a32f6e): regenerated by hand against the pinned upstream SHA (localnf-core modules patchCLI crashes withOSErrorin this sandbox); verified the patch applies cleanly and reproduces the local module.nf-test/snapshots on x86 CI; regenerate snapshots.COVERAGE_GRAPHS:PINTS_VISUALIZER(pre-existing flakiness).🤖 Generated with Claude Code