diff --git a/sites/docs/src/content/docs/developing/testing/advanced.md b/sites/docs/src/content/docs/developing/testing/advanced.md index cd7ec3629e..b9a9f51330 100644 --- a/sites/docs/src/content/docs/developing/testing/advanced.md +++ b/sites/docs/src/content/docs/developing/testing/advanced.md @@ -2,7 +2,7 @@ title: Advanced techniques subtitle: Write complex assertions shortTitle: Advanced assertions -weight: 3 +weight: 4 --- Advanced assertion patterns handle complex testing scenarios where standard snapshots are insufficient. diff --git a/sites/docs/src/content/docs/developing/testing/assertions.md b/sites/docs/src/content/docs/developing/testing/assertions.md index 9196df15b6..31825230b9 100644 --- a/sites/docs/src/content/docs/developing/testing/assertions.md +++ b/sites/docs/src/content/docs/developing/testing/assertions.md @@ -2,7 +2,7 @@ title: nf-core guidelines for assertions subtitle: Write test assertions shortTitle: Assertions -weight: 2 +weight: 3 --- nf-core guidelines for assertions: diff --git a/sites/docs/src/content/docs/developing/testing/ci-runners.md b/sites/docs/src/content/docs/developing/testing/ci-runners.md index 6d69e8cf4b..9dca1030a5 100644 --- a/sites/docs/src/content/docs/developing/testing/ci-runners.md +++ b/sites/docs/src/content/docs/developing/testing/ci-runners.md @@ -2,7 +2,7 @@ title: CI runners subtitle: Configure RunsOn and fix CI disk space issues shortTitle: CI runners -weight: 4 +weight: 5 --- nf-core uses [RunsOn](https://runs-on.com/) for self-hosted AWS runners that run GitHub Actions. diff --git a/sites/docs/src/content/docs/developing/testing/overview.md b/sites/docs/src/content/docs/developing/testing/overview.md index f1c9a471f3..3f3524aa85 100644 --- a/sites/docs/src/content/docs/developing/testing/overview.md +++ b/sites/docs/src/content/docs/developing/testing/overview.md @@ -56,6 +56,11 @@ Both directives work within their defined scope: either for all tests within the nf-test replaces paths in snapshots with a unique fingerprint (MD5 sum by default) to ensure file content consistency. +:::note +nf-core pipelines, modules, and subworkflows should specify input files from nf-core/test-datasets. +See the [test-data](./test-data.md) page for more information. +::: + ### Asserting channel item presence with `contains` Use Groovy's `contains` and `collect` methods to assert the presence of items in channel output. @@ -132,6 +137,7 @@ The nf-test documentation provides additional information on: Learn how to write effective test assertions following nf-core guidelines: +- [Test data](./test-data) covers how to find and prepare test-data files - [Writing assertions](./assertions.md) covers nf-core guidelines and common assertion patterns - [Advanced techniques](./advanced.md) covers complex patterns and troubleshooting - [CI runners](./ci-runners.md) covers RunsOn configuration and CI disk space issues diff --git a/sites/docs/src/content/docs/developing/testing/test-data.md b/sites/docs/src/content/docs/developing/testing/test-data.md new file mode 100644 index 0000000000..aff84b4892 --- /dev/null +++ b/sites/docs/src/content/docs/developing/testing/test-data.md @@ -0,0 +1,111 @@ +--- +title: Test data +subtitle: Finding and adding test data for CI testing +shortTitle: Test data +weight: 2 +--- + +nf-core has a central repository on [nf-core/test-datasets](https://github.com/nf-core/test-datasets) containing small data files for testing of nf-core components and pipelines. +This guide explains how to find and upload new test data. + +## Modules and subworkflows + +### Reusing existing test data + +When setting up tests for a nf-core component, test data files should be reused as far as possible. + +:::tip +If a small test data file for your module can be quickly generated by an upstream module, do not add the input file to the test data. +Run the upstream module in the test for your module using the [nf-test `setup` method](https://www.nf-test.com/docs/testcases/setup/?h=setup). +::: + +Test data for modules and subworkflows are stored on the [`modules`](https://github.com/nf-core/test-datasets/tree/modules) branch of nf-core/test-datasets. +The branch is approximately structured as follows: + +```tree +data/ +├── delete_me/ +│ └── / +│ └── +├── generic/ +│ └── / +│ └── +└── / + └── / + │ └── / + │ └── / + └── / ## Bioinformatics only + └── / + └── / + └── / +``` + +Existing nf-core test data can be explored using nf-core/tools. + +```bash +nf-core test-datasets search --branch modules +``` + +Type in the search bar keywords such as for file formats or extensions to find potentially relevant files. +To get more information about the context of the data, see the [nf-core/test-datasets README](https://github.com/nf-core/test-datasets/tree/modules#data-description) file. + +Selecting a file to print different variants of URLs to the test data file for use in tests. + +### Uploading new data + +If you cannot generate [test data on the fly quickly](https://www.nf-test.com/docs/testcases/setup/?h=setup), or reuse existing data, you can upload new compliant test data to the `modules` branch. + +If you need to create a derivative file, use existing test data as a source. +For example, if you need to test a new bioinformatic short-read aligner that requires a genome file index, use an existing `genome.fasta` file on nf-core/test-datasets to generate the index, rather than using your own reference genome. + +:::danger +Do not clone the nf-core/test-datasets repository: it is extremely large and will take a long time to download. +If a local copy is required, we highly recommend using [single-branch clones](https://github.com/nf-core/test-datasets#downloading-test-data). +::: + +1. Verify your new test-data file complies with the [nf-core test-datasets specifications](../../specifications/test-data/overview.md). +2. Fork the nf-core/test-datasets repository including all branches. +3. Upload to the `modules` branch on your fork in a suitable location. + - (Recommended) Use the GitHub website upload function. + - If you are unsure of the suitable location, ask on the nf-core [#test-data](https://nfcore.slack.com/archives/C02L5UB4Y9G) Slack channel. +4. Edit the README of the `modules` branch to add a short entry describing the new test data file. + - Include as much information as possible to support reconstruction of the file (for example, original source accession numbers or URLs, tool name and version used to generate). + - (Optional) If a description requires more than one or two sentences Additionally add a dedicated README markdown file alongside the test-data file itself. +5. Open a pull request and request a review on the nf-core [#request-review](https://nfcore.slack.com/archives/CQY2U5QU9) Slack channel. + +### Research domain specific guidance + +#### Biology + +Avoid adding test data of new organisms as far as possible, and reuse existing files. + +SARS-CoV-2 and _Homo sapiens_ (Chr 21) are the best supported organisms and should be used where possible. +If a new organism is absolutely required for a tool, first propose the addition on the [#test-data](https://nfcore.slack.com/archives/C02L5UB4Y9G) Slack channel. + +## Pipelines + +Each official nf-core pipeline has a dedicated branch on the test-datasets repository, for example, [nf-core/rnaseq](https://github.com/nf-core/test-datasets/tree/rnaseq). + +Reusability principles are less strict for pipeline test data. +Test data should not be reused across pipelines for stability purposes. +Reusing test data from modules for pipeline tests is possible due to the greater stability of these files. + +A pipeline-specific nf-core/test-datasets branch will generally consist of: + +- Small raw input files (for example, FASTQ, FASTA etc. for bioinformatics) +- Samplesheets + +### Adding new test data + +1. Verify your new test-data file complies with the [nf-core test-datasets specifications](../../specifications/test-data/overview.md). +2. Fork the nf-core/test-datasets repository. +3. Upload to the dedicated pipeline branch in a suitable location. + - (Recommended) Use the GitHub website upload function. +4. Edit the README of the dedicated pipeline branch describing how the file was generated. + - Include as much information as possible that could allow reconstruction of the file (for example, original source accession numbers or URLs, tool name and versions used to generate). +5. Open a pull request and request a review on the nf-core [#request-review](https://nfcore.slack.com/archives/CQY2U5QU9) Slack channel. + +## Related documentation + +- [nf-core/test-datasets](https://github.com/nf-core/test-datasets/) +- [Test data specifications](../../specifications/test-data/overview)