Skip to content

OA-only mode: run sash from oncoanalyser outputs only (no DRAGEN required) - #58

Open
qclayssen wants to merge 77 commits into
mainfrom
oa-only
Open

OA-only mode: run sash from oncoanalyser outputs only (no DRAGEN required)#58
qclayssen wants to merge 77 commits into
mainfrom
oa-only

Conversation

@qclayssen

@qclayssen qclayssen commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Adds an OA-only mode: sash can now run entirely from oncoanalyser outputs (SAGE somatic VCF, PAVE germline VCF) without requiring DRAGEN germline/somatic inputs.
  • Removes the DRAGEN-specific samplesheet filetypes (dragen_somatic_dir, dragen_germline_dir, dragen_somatic_vcf, dragen_germline_vcf), the BOLT_SMLV_SOMATIC_RESCUE step, and DRAGEN HRD score plumbing that depended on those inputs. DRAGEN support itself continues to live on release/0.7.0/main; this branch is scoped to the OA-only path.
  • Adds a fallback for oncoanalyser's sage_calling/sage/ output directory rename (OA v2.3.0+).
  • normal_id remains required — tumor-only analysis is out of scope for this mode.

Depends on the companion bolt containers built from oa-only-v2: umccr/bolt#34

Known follow-ups (not blocking)

  • docs/usage.md, README.md, and docs/details.md still document the removed dragen_* filetypes and DRAGEN HRD source — needs a doc pass in a follow-up PR.

Test plan

  • nf-test for subworkflows/local/prepare_input (updated to cover OA-only path resolution)
  • End-to-end run against a real oncoanalyser output directory

Validation results

Both legs done — full tables posted on sash#62.

  • SEQC-II re-confirm: exact match on all 7 metrics (purity/ploidy/gender/MSI/TML/TMB) vs the 2026-07-21 baseline.
  • SBJ04890 real-patient: purity/ploidy/gender/QC/MSI identical to its own full DRAGEN+OA run.

Note on TML/TMB in OA-only mode: OA-only germline calls come from PAVE (~1.9k
driver-focused variants), not DRAGEN's genome-wide germline VCF (~5M variants). This
shifts TML/TMB slightly vs a full DRAGEN+OA run on the same sample (SBJ04890: TML 75 vs
61, TMB/Mb 3.0598 vs 3.1123) — purity/ploidy/gender/QC/MSI are unaffected and match
exactly, since PURPLE's CN/SV calls come from the same OA-derived AMBER/COBALT input in
both modes. Don't read the TML/TMB difference as a regression.

qclayssen and others added 30 commits October 2, 2025 12:58
* add ref data for pcgr 2.2.1

* add vep path pcgr

* edit refdata path for vep pcgr

* update pcgr report name name

* change path VEP ref data

* update container version

* remove VEP version

* change bolt version to test
* add process and config for vcf2maf

* fix path vep data

* remove duplicate

* fix and linting

* remove params from VCF2MAF process

* fix and simplify process

* add meta.key to channel

* add genome build as input

* add dockerfile

* Make sigrap (HRDetect, MutationalPatterns) individual process (#16)

* remove trace of genome build and hardcode in VCF2MAF

* use biocontainers container for vcf2maf

* remove string interpolation

* change to stub no version template

* revert deleted line
* change sigrap container

* change cancer report container

* latest bolt version

* update meta id sigrap

* bump sigrap

* remove print

* restaure metadata sigrap

* change multiqc container

* add param for pcgr chunck size

* add mutpat predefined dbs argument

* update container

* add back formating comment
* fix channel prepare reference

* refactor tarball extraction to use proper channel patterns. Follow oncoanalyser pattern for tarball extraction with .collect() and .map()

* adapte extraction tarball to match pcgr expected vep structure

* fix dir name extract tarball for VEP

* change to adapt strip compoment for pcgr data
qclayssen added 9 commits May 19, 2026 11:02
- Document optional dragen_somatic_vcf/dragen_germline_vcf samplesheet
  fields with example
- Document pcgr_variant_chunk_size parameter for hypermutated samples
- Update citation version from 0.6.0 to 0.7.0
…vent PURPLE receiving mnv_filtered intermediate
Ships the graceful PCGR skip on samples whose per-chunk variant count
exceeds MAX_SOMATIC_VARIANTS, plus disable_estimates on chunked runs
where MSI/TMB on partial VCFs is meaningless. Variant tag suffixes
(-pcgr, -circos, -multiqc, -gpgr, -snpeff) preserved.
Moves hrdetect and mutpat modules off the personal docker.io/qclayssen
dev build to the official umccr ghcr release image.
qclayssen added 11 commits July 16, 2026 10:46
- prepare_input.nf: tumor_id/normal_id from explicit samplesheet columns;
  sage path fallback sage_calling/ → sage/ (OA v2.3.0+); dragen germline
  falls back to pave/{normal_id}.pave.germline.vcf.gz; dragen somatic
  returns [] when absent (OA-only mode)
- sash.nf: branch somatic on dragen presence; sage somatic passes through
  directly when no dragen (rescue skipped); germline steps gated on vcf
  presence; somatic report passes [] for dragen vcf; MultiQC dragen dir
  channels filtered when dirs absent
- bolt report module: container → 0.3.1-dev-oa; --vcf_dragen_fp conditional
- samplesheet.csv: add tumor_id/normal_id columns; add OA-only SBX example
groupByMeta requires >=2-element tuples (data[1..-1] throws on a
single-element list). Passing ch_somatic_branch.oa_only directly
emits [meta, [], []], which groupByMeta's recursive flatten collapses
to [meta, sage_vcf, sage_tbi] — the correct 3-arg signature.
Drop DRAGEN somatic VCF, germline VCF, and HRD score handling entirely.
Germline now always uses OA PAVE output; rescue step removed.
CPSR writes no output files when zero variants remain after classification
filtering. Mark cpsr_report output optional and guard the mv command.
Process was removed from the OA-only workflow earlier; the module file
was left behind. No includes or config selectors reference it.
Drop dragen_hrd from BOLT_OTHER_CANCER_REPORT input tuple and script —
always empty in OA-only mode. Drop smlv_dragen_vcf input and dragen_arg
from BOLT_SMLV_SOMATIC_REPORT — replaced by optional --vcf_dragen_fp
guard already handled in bolt.
…mode

- Build the PAVE germline VCF path from normal_id (matches the file's
  actual naming and the fixture NORMAL.pave.germline.vcf.gz) instead of
  tumor_id, which 404'd on every real run.
- Require normal_id and the PAVE germline VCF like every other OA
  input, instead of silently degrading to tumor-only with no
  germline data. Tumor-only was never intended to be supported here,
  and the empty-VCF fallback caused samples to silently drop out of
  the MultiQC report entirely (fixed groupTuple join never completes).
ffc37ac (2026-07-13) changed this from tumor_id to normal_id, claiming
tumor_id "404'd on every real run" - but that fix's own PR (#58) test
plan left "End-to-end run against a real oncoanalyser output directory"
unchecked, and the commit's justification ("matches ... the fixture
NORMAL.pave.germline.vcf.gz") cites a fixture that same commit did not
change, i.e. it matches its own synthetic fixture, not verified real
output.

Tested live against real production oncoanalyser 2.2.0 SEQC-II output
(s3://test-data-503977275616-ap-southeast-2/testdata/analysis/
production/oncoanalyser-wgts-dna/2.2.0/SEQC-II/20260401d6eff920/):
normal_id 404s, tumor_id resolves. Root-caused in oncoanalyser's own
source (subworkflows/local/pave_annotation/main.nf): PAVE_GERMLINE's
sample_id is set via Utils.getTumorDnaSampleName(meta) - oncoanalyser
keys every per-pair output file, including the germline VCF, by the
tumor sample ID, never the normal ID. Stable since 2024-05-02.

Renamed the fixture (NORMAL -> TUMOR.pave.germline.vcf.gz) and updated
the nf-test assertion to match, so the fixture now reflects real
oncoanalyser output naming instead of the code's assumption.

nf-test subworkflows/local/tests/prepare_input/main.nf.test: PASSED
Conflicts resolved:
- CHANGELOG.md: accept Breaking change note from main
- cancer_report/main.nf: keep OA-only input tuple (no dragen_hrd)
- smlv_somatic/rescue/main.nf: deleted (OA-only removes rescue)
- prepare_input.nf: keep OA-only version (PAVE germline, no DRAGEN)
- prepare_reference.nf: accept main's tarball-only policy
- workflows/sash.nf: keep OA-only version (no rescue, no dragen_hrd)
- tests/prepare_input: keep OA-only test suite
Resolves one real conflict: modules/local/bolt/smlv_somatic/rescue/main.nf
was deliberately dropped as an orphan module on this branch (0cefd2b) while
main independently bumped its container tag (3b9fb48) - kept the deletion
since the module is unreferenced in workflows/sash.nf on this branch.
Everything else auto-merged cleanly (bolt 0.3.0->0.3.1 container bumps,
testing-validation docs/SOP additions from main).
@qclayssen

Copy link
Copy Markdown
Collaborator Author

Rebased against main (merged in #61 docs + #63 bolt-version fix) — was CONFLICTING, now CLEAN/MERGEABLE. Ready to merge once bolt #34 lands (per the handover doc's merge order).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants