Add DeepMAsED for deep learning evaluation of metagenomic de novo assemblies

CHANGELOG.md

@@ -17,6 +17,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - [#1042](https://github.com/nf-core/mag/pull/1042) - Add nf-test snapshot for `test_default` profile (by @dialvarezs)
 - [#1044](https://github.com/nf-core/mag/pull/1044) - Add new `--gtdbtk_place_species` parameter (by @dialvarezs)
 - [#1047](https://github.com/nf-core/mag/issues/1007) - Add `--gtdbtk_single_job` to run GTDB-Tk classification for all bins in a single job (requested by @sarah-shah-bioinf, by @dialvarezs)
+- [#1051](https://github.com/nf-core/mag/pull/1051) - Add DeepMAsED assembly error detection to the MAG workflow as two sequential steps (`features` and `predict`) for short-read assemblies (by @SkyLexS).
 - [#1048](https://github.com/nf-core/mag/pull/1048) - Add optional PyPOLCA polishing for long-read assemblies via `--run_pypolca` (by @Harshita-sriv)
 - [#1059](https://github.com/nf-core/mag/pull/1059) - Add `--filtlong_filtering_by_shortreads` parameter to enable filtlong's short-read-based long read filtering (by @dialvarezs)
 - [#1063](https://github.com/nf-core/mag/pull/1063) - Add new `--ale_per_base_output` parameter to enable ALE per-base output (by @dialvarezs)

CITATIONS.md

@@ -68,6 +68,10 @@
 
   > Sieber, C. M. K., et al. 2018. "Recovery of Genomes from Metagenomes via a Dereplication, Aggregation and Scoring Strategy." Nature Microbiology 3 (7): 836-43. doi: 10.1038/s41564-018-0171-1
 
+- [DeepMAsED](https://doi.org/10.1371/journal.pcbi.1009993)
+
+  > Athreya, A., Diment, J., Poroyko, V., & Treangen, T. J. (2022). DeepMAsED: Evaluating the quality of metagenomic assemblies using deep learning. PLoS Computational Biology, 18(5), e1009993. doi: 10.1371/journal.pcbi.1009993
+
 - [FastP](https://doi.org/10.1093/bioinformatics/bty560)
 
   > Chen, S., Zhou, Y., Chen, Y., & Gu, J. (2018). fastp: an ultra-fast all-in-one FASTQ preprocessor. Bioinformatics , 34(17), i884–i890. doi: 10.1093/bioinformatics/bty560.

README.md

@@ -47,6 +47,7 @@ By default, the pipeline currently performs the following: it supports both shor
 The pipeline then:
 
 - performs assembly using [MEGAHIT](https://github.com/voutcn/megahit) and [SPAdes](http://cab.spbu.ru/software/spades/), and checks their quality using [Quast](http://quast.sourceforge.net/quast) and [ALE](https://github.com/sc932/ALE) (if short read data is used)
+- performs per-contig assembly error detection for short-read assemblies using [DeepMAsED](https://github.com/leylabmpi/DeepMAsED)
 - (optionally) polishes long-read assemblies with short reads using [pypolca](https://github.com/gbouras13/pypolca)
 - (optionally) performs ancient DNA assembly validation using [PyDamage](https://github.com/maxibor/pydamage) and contig consensus sequence recalling with [Freebayes](https://github.com/freebayes/freebayes) and [BCFtools](http://samtools.github.io/bcftools/bcftools.html)
 - predicts protein-coding genes for the assemblies using [Prodigal](https://github.com/hyattpd/Prodigal), and bins with [Prokka](https://github.com/tseemann/prokka) and optionally [MetaEuk](https://www.google.com/search?channel=fs&client=ubuntu-sn&q=MetaEuk)

conf/base.config

@@ -155,6 +155,16 @@ process {
         time          = { 8.h * task.attempt }
         errorStrategy = { task.exitStatus in ((130..145) + [104, 175, 247]) ? 'retry' : 'finish' }
     }
+    withName: DEEPMASED_FEATURES {
+        cpus   = { 12 * task.attempt }
+        memory = { 120.GB * task.attempt }
+        time   = { 96.h * task.attempt }
+    }
+    withName: DEEPMASED_PREDICT {
+        cpus   = { 4 * task.attempt }
+        memory = { 24.GB * task.attempt }
+        time   = { 4.h * task.attempt }
+    }
     withName: METABAT2_METABAT2 {
         cpus   = { 8 * task.attempt }
         memory = { 20.GB * task.attempt }

conf/modules.config

@@ -469,6 +469,26 @@ process {
         ext.args   = { params.ale_per_base_output ? "--metagenome" : "--metagenome --nout" }
     }
 
+    withName: DEEPMASED_FEATURES {
+        ext.args = { params.deepmased_features_gzip ? '--gzip' : '' }
+        publishDir = [
+            path: { "${params.outdir}/Assembly/${meta.assembler}/QC/${meta.id}/DeepMAsED/features" },
+            mode: params.publish_dir_mode,
+            saveAs: { filename -> filename.equals('versions.yml') ? null : filename }
+        ]
+        ext.prefix = { "${meta.id}-${meta.assembler}" }
+    }
+
+    withName: DEEPMASED_PREDICT {
+        ext.args = { [ params.deepmased_cpu_only ? '--cpu-only' : '', "--seed ${params.deepmased_predict_seed}" ].join(' ').trim() }
+        publishDir = [
+            path: { "${params.outdir}/Assembly/${meta.assembler}/QC/${meta.id}/DeepMAsED" },
+            mode: params.publish_dir_mode,
+            saveAs: { filename -> filename.equals('versions.yml') ? null : filename }
+        ]
+        ext.prefix = { "${meta.id}-${meta.assembler}" }
+    }
+
     withName: 'QUAST_BINS|CONCAT_QUAST_SUMMARY' {
         publishDir = [
             path: { "${params.outdir}/GenomeBinning/QC" },

conf/test_alternatives.config

@@ -32,6 +32,8 @@ params {
 
     // Input data
     input                      = params.pipelines_testdata_base_path + 'mag/samplesheets/samplesheet.v4.csv'
+    busco_db                   = params.pipelines_testdata_base_path + 'mag/databases/busco/bacteria_odb10.2024-01-08.tar.gz'
+    busco_db_lineage           = 'bacteria_odb10'
     clip_tool                  = 'trimmomatic'
     coassemble_group           = true
     run_busco                  = true

conf/test_longreadonly.config

@@ -49,6 +49,7 @@ params {
     skip_metabinner            = true
     skip_semibin               = true
     skip_ale                   = true
+    skip_deepmased             = true
 }
 
 singularity.pullTimeout = 30.m

conf/test_longreadonly_alternatives.config

@@ -53,4 +53,6 @@ params {
     skip_comebin                  = true
     skip_metabinner               = true
     skip_semibin                  = true
+    skip_metaeuk                  = true
+    skip_deepmased                = true
 }

docs/output.md

@@ -329,6 +329,27 @@ ALE can run only on assemblies generated from short reads, like SPAdes and MEGAH
 
 </details>
 
+### Assembly Quality Control with DeepMAsED
+
+[DeepMAsED (Deep learning for Metagenome Assembly Error Detection)](https://github.com/leylabmpi/DeepMAsED) uses a pre-trained neural network to predict per-contig misassembly scores from read alignment features. It runs in two sequential steps implemented as separate Nextflow modules:
+
+**`DEEPMASED_FEATURES`** extracts alignment statistics (coverage, mismatches, indels, base quality) from the BAM file for each contig and produces feature tables. This is the computationally intensive step.
+
+**`DEEPMASED_PREDICT`** applies the pre-trained model to the feature tables and outputs a misassembly score per contig (0 = correctly assembled, 1 = likely misassembly).
+
+DeepMAsED only runs on short-read assemblies (MEGAHIT, SPAdes). It cannot be used with long-read or hybrid assemblies.
+
+<details markdown="1">
+<summary>Output files</summary>
+
+- `Assembly/[assembler]/QC/[sample/group]/DeepMAsED/features/`
+  - `[sample]-[assembler]_feature_file_paths.tsv`: Index file listing all generated feature table files
+  - `[sample]-[assembler]*_feats.tsv`: Per-contig feature tables (one per parallel processing bin)
+- `Assembly/[assembler]/QC/[sample/group]/DeepMAsED/`
+  - `[sample]-[assembler]_deepmased_predictions.tsv`: Per-contig misassembly scores. Score of 0 indicates a correctly assembled contig; score of 1 indicates a likely misassembly.
+
+</details>
+
 ## Gene prediction
 
 Protein-coding genes are predicted for each assembly.

docs/usage.md

@@ -581,6 +581,16 @@ This feature was removed in version 5.0.0 to strengthen the pipeline's focus on
 
 If you require taxonomic profiling of raw reads, we recommend using [nf-core/taxprofiler](https://nf-co.re/taxprofiler/), which is specifically designed for taxonomic profiling of raw reads and supports a wide range of tools for this purpose.
 
+## A note on DeepMAsED assembly error detection
+
+When enabled, DeepMAsED performs per-contig assembly error detection on short-read assemblies (MEGAHIT and SPAdes).
+
+Useful parameters:
+
+- `--skip_deepmased` to skip DeepMAsED entirely.
+- `--skip_deepmased_predict` to run only feature extraction.
+- `--deepmased_cpu_only` to force CPU-only inference.
+
 ## BIgMAG compatibility
 
 With the parameter `--generate_bigmag_file` a module will be triggered to generate a file that contains the output from all of the bin-quality tools that can be uploaded to the [BIgMAG](https://github.com/jeffe107/BIgMAG) dashboard for visualising and evaluating MAGs.

modules.json

@@ -141,6 +141,16 @@
                         "git_sha": "e753770db613ce014b3c4bc94f6cba443427b726",
                         "installed_by": ["modules"]
                     },
+                    "deepmased/features": {
+                        "branch": "master",
+                        "git_sha": "8eb801777f42fb9e3e1f6e0402e7f571c59e431c",
+                        "installed_by": ["modules"]
+                    },
+                    "deepmased/predict": {
+                        "branch": "master",
+                        "git_sha": "8eb801777f42fb9e3e1f6e0402e7f571c59e431c",
+                        "installed_by": ["modules"]
+                    },
                     "fastp": {
                         "branch": "master",
                         "git_sha": "d9ec4ef289ad39b8a662a7a12be50409b11df84b",