[BEP028] provenance - fMRIPrep preprocessing

.github/workflows/validate_datasets.yml

@@ -139,6 +139,10 @@ jobs:
             exit 1;
         fi
 
+    - name: Skip legacy validation for post-legacy datasets
+      run: for DS in mrs_* dwi_deriv pet006 pheno004 volume_timing provenance_*; do touch $DS/.SKIP_VALIDATION; done
+      if: matrix.bids-validator == 'legacy'
+
     - name: Skip stable validation for datasets with unreleased validator features
       # Replace ${EMPTY} with dataset patterns, when this is needed
       # Reset to "for DS in ${EMPTY}; ..." after a validator release
@@ -148,7 +152,7 @@ jobs:
     - name: Skip main validation for datasets with unreleased spec features
       # Replace ${EMPTY} with dataset patterns, when this is needed
       # Reset to "for DS in ${EMPTY}; ..." after a spec release
-      run: for DS in eyetracking_* atlas-* emg_*; do touch $DS/.SKIP_VALIDATION; done
+      run: for DS in dwi_deriv pheno004 provenance_* eyetracking_* atlas-* emg_*; do touch $DS/.SKIP_VALIDATION; done
       if: matrix.bids-validator != 'dev'
 
     - name: Set BIDS_SCHEMA variable for dev version
@@ -157,7 +161,7 @@ jobs:
       # Update this URL to the schema.json from PRs to the spec, when needed.
       # If this variable is unset, dev will generally track the latest development
       # release of https://jsr.io/@bids/schema
-      run: echo BIDS_SCHEMA=https://bids-specification.readthedocs.io/en/latest/schema.json >> $GITHUB_ENV
+      run: echo BIDS_SCHEMA=https://bids-specification--2099.org.readthedocs.build/en/2099/schema.json >> $GITHUB_ENV
 
     - name: Set BIDS_SCHEMA variable for dev-prs version
       if: matrix.bids-validator == 'dev-prs' && env.SPEC_PR != ''

README.md

@@ -391,5 +391,6 @@ TABLE BELOW IS GENERATED AUTOMATICALLY.
 DO NOT EDIT DIRECTLY.
 -->
 
-| name   | description   | datatypes   | suffixes   | link to full data   | maintained by   |
-|--------|---------------|-------------|------------|---------------------|-----------------|
+| name                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          | description                    | datatypes   | suffixes   | link to full data   | maintained by   |
+|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------|:------------|:-----------|:--------------------|:----------------|
+| [provenance_fmriprep Provenance metadata for a derivative dataset after functional MRI preprocessing performed with [`fMRIPrep`](https://fmriprep.org/en/23.1.3/index.html). This example is based upon [ds001734](https://openneuro.org/datasets/ds001734/versions/1.0.5) data. [@bclenet](https://github.com/bclenet) anat, func, prov](https://github.com/bids-standard/bids-examples/tree/master/provenance_fmriprep Provenance metadata for a derivative dataset after functional MRI preprocessing performed with [`fMRIPrep`](https://fmriprep.org/en/23.1.3/index.html). This example is based upon [ds001734](https://openneuro.org/datasets/ds001734/versions/1.0.5) data. [@bclenet](https://github.com/bclenet) anat, func, prov) | T1w, bold, act, ent, soft, env | n/a         | n/a        | n/a                 | n/a             |

dataset_listing.tsv

@@ -94,3 +94,4 @@ emg_MultiBodyParts	EMG recording from multiple body parts with different electro
 emg_TwoHDsEMG	High-density EMG grid recordings from two body parts demonstrating grid placement documentation	n/a	[@neuromechanist](https://github.com/neuromechanist)	emg	channels, electrodes, coordsystem, emg
 emg_TwoWristbands	EMG recordings using two wristbands with dry electrodes to capture forearm muscle activity	n/a	[@neuromechanist](https://github.com/neuromechanist)	emg	channels, electrodes, coordsystem, emg
 mri_chunk	Example MRI dataset to illustrate BIDS chunk entity. A single subject, two chunks.		[@valosekj](https://github.com/valosekj)	anat	T1w
+provenance_fmriprep Provenance metadata for a derivative dataset after functional MRI preprocessing performed with [`fMRIPrep`](https://fmriprep.org/en/23.1.3/index.html). This example is based upon [ds001734](https://openneuro.org/datasets/ds001734/versions/1.0.5) data. [@bclenet](https://github.com/bclenet) anat, func, prov	T1w, bold, act, ent, soft, env

provenance_fmriprep/.bidsignore

@@ -0,0 +1 @@
+sub-001/*

provenance_fmriprep/README.md

@@ -0,0 +1,97 @@
+# Provenance of fMRI preprocessing with `fMRIPrep`
+
+## Goal
+
+This example aims at showing provenance metadata for a preprocessing with [`fMRIPrep`](https://fmriprep.org/en/23.1.3/index.html). Provenance metadata was generated manually ; this acts as a guideline for minimal provenance description at dataset level.
+
+> [!WARNING]
+> Validation of the `sub-001/` directory is ignored through a `.bidsignore` file, as the aim of the example is to focus on provenance metadata.
+
+## Original dataset
+
+This dataset is a derivative dataset of the [ds001734](https://openneuro.org/datasets/ds001734/versions/1.0.5) dataset. ds001734 contains raw and preprocessed fMRI data of two versions of the mixed gambles task, from the Neuroimaging Analysis Replication and Prediction Study (NARPS). **Only raw data for subject `sub-001` was preprocessed for this example.** All `.nii.gz` files were replaced by empty files.
+
+## Directory tree
+
+After preprocessing, and adding provenance metadata, the directory tree is as follows.
+Files marked with ✍️ were generated manually, other files were generated by the preprocessing step.
+
+> [!NOTE]
+> Note that the `docs/` directory contains explanatory data (see [Provenance as a RDF graph](#provenance-as-a-rdf-graph)) that is not required to encode provenance.
+
+```
+.
+├── ✍️ dataset_description.json
+├── ✍️ docs
+│   ├── ✍️ prov-fmriprep.jsonld
+│   └── ✍️ prov-fmriprep.png
+├── ✍️ prov
+│   └── ✍️ prov-fmriprep
+│       ├── ✍️ prov-fmriprep_act.json
+│       ├── ✍️ prov-fmriprep_ent.json
+│       ├── ✍️ prov-fmriprep_env.json
+│       └── ✍️ prov-fmriprep_soft.json
+├── ✍️ README.md
+└── sub-001
+    ├── anat
+    │   ├── sub-001_T1w_brainmask.nii.gz
+    │   ├── sub-001_T1w_dtissue.nii.gz
+    │   ├── sub-001_T1w_label-aparcaseg_roi.nii.gz
+    │   ├── sub-001_T1w_label-aseg_roi.nii.gz
+    │   ├── sub-001_T1w_preproc.nii.gz
+    │   ├── sub-001_T1w_space-MNI152NLin2009cAsym_brainmask.nii.gz
+    │   ├── sub-001_T1w_space-MNI152NLin2009cAsym_dtissue.nii.gz
+    │   └── sub-001_T1w_space-MNI152NLin2009cAsym_preproc.nii.gz
+    └── func
+        ├── sub-001_task-MGT_run-01_bold_confounds.tsv
+        ├── sub-001_task-MGT_run-01_bold_space-MNI152NLin2009cAsym_brainmask.nii.gz
+        ├── sub-001_task-MGT_run-01_bold_space-MNI152NLin2009cAsym_preproc.nii.gz
+        ├── sub-001_task-MGT_run-01_bold_space-T1w_label-aparcaseg_roi.nii.gz
+        ├── sub-001_task-MGT_run-01_bold_space-T1w_label-aseg_roi.nii.gz
+        ├── sub-001_task-MGT_run-02_bold_confounds.tsv
+        ├── sub-001_task-MGT_run-02_bold_space-MNI152NLin2009cAsym_brainmask.nii.gz
+        ├── sub-001_task-MGT_run-02_bold_space-MNI152NLin2009cAsym_preproc.nii.gz
+        ├── sub-001_task-MGT_run-02_bold_space-T1w_label-aparcaseg_roi.nii.gz
+        ├── sub-001_task-MGT_run-02_bold_space-T1w_label-aseg_roi.nii.gz
+        ├── sub-001_task-MGT_run-03_bold_confounds.tsv
+        ├── sub-001_task-MGT_run-03_bold_space-MNI152NLin2009cAsym_brainmask.nii.gz
+        ├── sub-001_task-MGT_run-03_bold_space-MNI152NLin2009cAsym_preproc.nii.gz
+        ├── sub-001_task-MGT_run-03_bold_space-T1w_label-aparcaseg_roi.nii.gz
+        ├── sub-001_task-MGT_run-03_bold_space-T1w_label-aseg_roi.nii.gz
+        ├── sub-001_task-MGT_run-04_bold_confounds.tsv
+        ├── sub-001_task-MGT_run-04_bold_space-MNI152NLin2009cAsym_brainmask.nii.gz
+        ├── sub-001_task-MGT_run-04_bold_space-MNI152NLin2009cAsym_preproc.nii.gz
+        ├── sub-001_task-MGT_run-04_bold_space-T1w_label-aparcaseg_roi.nii.gz
+        └── sub-001_task-MGT_run-04_bold_space-T1w_label-aseg_roi.nii.gz
+
+```
+
+## Provenance as a RDF graph
+
+Provenance metadata can be aggregated as a JSON-LD RDF graph, which is available in [`docs/prov-fmriprep.jsonld`](docs/prov-fmriprep.jsonld). This is a rendered version of the graph, also available in [`docs/prov-fmriprep.png`](docs/prov-fmriprep.png).
+
+![Rendered version of the RDF graph](docs/prov-fmriprep.png)
+
+## Description of processes or pipelines
+
+Provenance can also be described by providing `GeneratedBy` with an array of objects representing pipelines or processes that generated the dataset.
+
+In this case, the `GeneratedBy` field of the `dataset_description.json` file could have contain an object representing the fMRIPrep pipeline.
+
+```JSON
+{
+    "GeneratedBy": [
+        {
+          "Name": "fMRIPrep",
+          "Version": "1.1.4",
+          "Description": "Preprocessing with fMRIPrep",
+          "CodeURL": "https://github.com/nipreps/fmriprep/releases/tag/1.1.4",
+          "Container": {
+            "Type": "docker",
+            "Tag": "poldracklab/fmriprep:1.1.4",
+            "URL": "https://hub.docker.com/layers/poldracklab/fmriprep/1.1.4"
+          }
+        }
+    ]
+}
+```

provenance_fmriprep/dataset_description.json

@@ -0,0 +1,21 @@
+{
+  "Name": "Outputs from fMRIPrep preprocessing of the NARPS data",
+  "BIDSVersion": "1.10.0",
+  "DatasetType": "derivative",
+  "License": "CC0",
+  "Authors": [
+    "Boris Clénet"
+  ],
+  "DatasetLinks": {
+    "ds001734": "https://openneuro.org/datasets/ds001734/versions/1.0.5"
+  },
+  "SourceDatasets": [
+    {
+      "URL": "ds001734",
+      "Version": "1.0.5"
+    }
+  ],
+  "GeneratedBy": [
+    "bids::prov#preprocessing-xMpFqB5q"
+  ]
+}

provenance_fmriprep/docs/prov-fmriprep.jsonld

@@ -0,0 +1,53 @@
+{
+  "@context": "https://bids-specification--2099.org.readthedocs.build/en/2099/provenance-context.json",
+  "Records": {
+    "Software": [
+      {
+        "Id": "bids::prov#fmriprep-awf6cvk6",
+        "Label": "fMRIPrep",
+        "Version": "1.1.4",
+        "AlternativeIdentifier": [
+          "RRID:SCR_016216"
+        ]
+      }
+    ],
+    "Activities": [
+      {
+        "Id": "bids::prov#preprocessing-xMpFqB5q",
+        "Label": "Preprocessing",
+        "Command": "docker run --rm -it -v /soft/freesurfer/license.txt:/opt/freesurfer/license.txt:ro -v /dev/BEP028_BIDSprov/examples/fmriprep/ds001734:/data:ro -v /data/ds001734_fmriprep:/out -v /data/ds001734_fmriprep/work:/scratch poldracklab/fmriprep:1.1.4 /data /out participant --participant-label=001 --reports-only --write-graph -w /scratch",
+        "AssociatedWith": [
+          "bids::prov#fmriprep-awf6cvk6"
+        ],
+        "Used": [
+          "bids::prov#poldracklab/fmriprep-mHl7Dqa0",
+          "bids:ds001734"
+        ]
+      }
+    ],
+    "Files": [],
+    "Datasets": [
+      {
+        "Id": "bids:ds001734",
+        "Label": "NARPS"
+      },
+      {
+        "Id": "bids:current_dataset",
+        "Label": "Outputs from fMRIPrep preprocessing of the NARPS data",
+        "GeneratedBy": [
+          "bids::prov#preprocessing-xMpFqB5q"
+        ]
+      }
+    ],
+    "prov:Entity": [],
+    "Environments": [
+      {
+        "Id": "bids::prov#poldracklab/fmriprep-mHl7Dqa0",
+        "Label": "poldracklab/fmriprep:1.1.4",
+        "AlternativeIdentifier": [
+          "https://hub.docker.com/layers/poldracklab/fmriprep/1.1.4"
+        ]
+      }
+    ]
+  }
+}

provenance_fmriprep/prov/prov-fmriprep/prov-fmriprep_act.json

@@ -0,0 +1,16 @@
+{
+  "Activities": [
+    {
+      "Id": "bids::prov#preprocessing-xMpFqB5q",
+      "Label": "Preprocessing",
+      "Command": "docker run --rm -it -v /soft/freesurfer/license.txt:/opt/freesurfer/license.txt:ro -v /dev/BEP028_BIDSprov/examples/fmriprep/ds001734:/data:ro -v /data/ds001734_fmriprep:/out -v /data/ds001734_fmriprep/work:/scratch poldracklab/fmriprep:1.1.4 /data /out participant --participant-label=001 --reports-only --write-graph -w /scratch",
+      "AssociatedWith": [
+        "bids::prov#fmriprep-awf6cvk6"
+      ],
+      "Used": [
+        "bids::prov#poldracklab/fmriprep-mHl7Dqa0",
+        "bids:ds001734:."
+      ]
+    }    
+  ]
+}

provenance_fmriprep/prov/prov-fmriprep/prov-fmriprep_ent.json

@@ -0,0 +1,8 @@
+{
+  "Datasets": [
+    {
+      "Id": "bids:ds001734:.",
+      "Label": "NARPS"
+    }
+  ]
+}

provenance_fmriprep/prov/prov-fmriprep/prov-fmriprep_env.json

@@ -0,0 +1,11 @@
+{
+  "Environments": [
+    {
+      "Id": "bids::prov#poldracklab/fmriprep-mHl7Dqa0",
+      "Label": "poldracklab/fmriprep:1.1.4",
+      "AlternativeIdentifier": [
+          "https://hub.docker.com/layers/poldracklab/fmriprep/1.1.4"
+      ]
+    }
+  ]
+}

provenance_fmriprep/prov/prov-fmriprep/prov-fmriprep_soft.json

@@ -0,0 +1,12 @@
+{
+  "Software": [
+    {
+      "Id": "bids::prov#fmriprep-awf6cvk6",
+      "Label": "fMRIPrep",
+      "Version": "1.1.4",
+      "AlternativeIdentifier": [
+        "RRID:SCR_016216"
+      ]
+    } 
+  ]
+}