[ENH] BEP044 - Stim-BIDS

[neuromechanist]

This PR addresses #153 by introducing specifications for handling and standardizing stimulus files and their annotations within the BIDS specifications. The changes focus on improving the organization, referencing, and metadata of stimulus files to enhance consistency, reusability, and efficiency.

Important

HTML render of the stimuli page : https://bids-specification--2022.org.readthedocs.build/en/2022/modality-agnostic-files/stimuli.html

Note

The issue: #153
Google Doc
Example PR: bids-standard/bids-examples#433

Known issues:

• The validator fails because extensions in /src/schema/rules/files/raw require datatype. Stimuli might be a special data type that can only be present at the root of the dataset. So, the datatype field is missing for now.
• There are some style errors by the remark validators

cc: @bids-standard/bep044 and @monique2208

[Remi-Gau]

will do a bit of clean up to get less red in CI and maybe see if we can get the HTML version of the BEP to render

[Remi-Gau]

HTML: stimuli page
https://bids-specification--2022.org.readthedocs.build/en/2022/modality-specific-files/stimuli.html

[yarikoptic]

initial quick pass with little recommendations etc

[dorahermes]

lgtm

[yarikoptic]

here it seems to not render entirely as desired - -<label> (and <index>? for part) is missing

[yarikoptic]

also note that there is no .json sidecar mentioned for all those .<extension> files! "explicit better than implicit" and for consistency might be worth doing like done for BEP047 https://bids-specification--2231.org.readthedocs.build/en/2231/modality-specific-files/behavioral-experiments.html

[yarikoptic]

@neuromechanist I do not want to derail this BEP, but it might as well be included here (or later).

I would like your feedback on our

• Develop a script to "slice" videos to "match" DICOM series timing ReproNim/reprostim#14

use case, where we will have a distinct video record for every MRI data file in the dataset (thus per subject/[session/]). I envisioned placing them nearby the data files:

• [ENH] allow for _stim.{mp[34],mkv,avi} to provide stimuli files for func data #750

and that is reflective somehow of @bendichter 's

• [ENH] BEP047 - Add audio/video recordings to behavioral experiments #2231

where it is a audiovideo record of behavior, instead of audiovideo record of stimuli.

One logical way could be to allow for stimuli/sub-<label>/[ses-<label>/]... hierarchy but that feels suboptimal since if unique per sub/ses -- better just go nearby the data.

I would say, we (with @vmdocua) would just add stim- entity to the file, associated with the data file, so e.g. for every _events.tsv level file it would produce the one with extra _stim-reprostim_audiovideo.{mkv,json} .

So in the long run, I would need to advocate allowing for such files in the hierarchy, and that will use stim entity then, with stimuli.tsv on top level providing description for what that reprostim means.

[neuromechanist]

Very nice @yarikoptic. Indeed recording the stimulus/subject individual presentations could unlock accounting for even more variabilities throughout the session.

1. I agree that individual stimuli presentation files should go into the subject/session files

2. This might be a sidetone, but I'd think that [ENH] allow for _stim.{mp[34],mkv,avi} to provide stimuli files for func data #750 and [ENH] BEP047 - Add audio/video recordings to behavioral experiments #2231 are basically two different things, and that (some form of) both should exist, somehow. We should identify if the video file is a stimulus or behavior (or both?) (through sidecar metadata?).

Hopefully we should not overuse stim (especially if the recording captures both subject and stimulus, then it is not stimulus) and we could use something more common like recording:

sub-01_task-forrest_split-01_recording-reprostim_audiovideo.mp4
Ideally, we should allow:
sub-01_task-forrest_split-01_recording-reprostim_events.tsv
as the audiovideo.mp4 is a data file like any other bold or eeg data file that would deserve an events.tsv file ;D.

But, in the case that only the presented stimuli is recorded, like presenting randomized NSD data, capturing the video feed and sharing the presented video per participant (to make sure that event times are correct captured?):
sub-01_task-nsd_stim.mp4 would be enough
and then we have the usual sub-01_task-nsd_events.tsv to match that, as stimulus file is "assumed" to accompany a data file (Bold or EEG for that matter).

I hope that I could convey this not super twisted. LMK.

Either way, both should not have a bearing on BEP044. One of the reasons not to push BEP044 out now is to make sure that the use-cases are mature enough and that tools can sufficiently use it. Hence, working on https://github.com/Annotation-Garden/HEDit. So, please bring on more use cases :D.

[yarikoptic]

re synergy with #2231 (to be BEP047)
I think it would be great to take out into potentially into a separate PR definition of metadata fields for sidecars which could (or must) be provided for any audio/video file. @bendichter defined some of them (see in schema and in rendered html and I think it would be great to agree on them between the two. WDYT?

[yarikoptic]

Hopefully we should not overuse stim (especially if the recording captures both subject and stimulus, then it is not stimulus) and we could use something more common like recording:

sub-01_task-forrest_split-01_recording-reprostim_audiovideo.mp4

I kinda liked it but it made me pause: if I to take that subject specific stimuli and place it following this BEP under top level stimuli/ folder -- what should be the name for such a file?

If it was stim-reprostim (as sub-01_task-forrest_split-01_stim-reprostim_audiovideo.mp4), then everything would make total sense (although not yet this BEP compatible but very much following flex layout idea #1809 ) -- as I would just reorder entities to match the desired groupping order here and have it stim-reprostim_sub-01_task-forrest_split-01_audiovideo.mp4. (I ❤️ it). I still feel that recording- fits nice here but I cannot come up with a generic name for what to use for stim- in such case if to use _recording-reprostim?! (edit: could be explicit and state "stim-captured"... also space for AI to determine... for dbic/QA recordings could be stim-qrcodes)

[yarikoptic]

The summary of my "mental" excercise detailed in

• Formalization of "entities" flow or new-function-in-life for _mod- bids-2-devel#97

TL;DR summary for this, I think I will go with @vmdocua for

sub-01_task-forrest_stim-reprostim_audiovideo.mp4

ATM, although a potentially more kosher and permitting for mix-in with e.g. {datamodality}/ folder could be

sub-01_task-forrest_mod-stimuli_recording-reprostim_audiovideo.mp4

thus allowing for smth like following to be nearby

sub-01_task-forrest_mod-beh_recording-wallcamera_audiovideo.mp4

thus clearly separating and annotating the "datatype" (in _mod-) to tell stim and beh apart.

Little more detailed mental walk through

I think has good generalization principle I filed in bids-standard/bids-2-devel#97 for this PR would sound like:

• [stim-...] entity, and thus having stimuli.{tsv,gz} should remain optional under stimuli/ as in the cases where there was a single stimuli or there are other entities (e.g. _recording, _annot) are present to tell stimuli files apart. So kinda "better compatibility" with what we have now already (cons: harder to determine if to validate file names under stimuli/)
• files with stimuli under specific sub/[ses/]datatype folders could gain either
  • _mod-stim entity to tell them apart from other files in those folders (e.g. recordings of behavior under beh/ folders from bep047) or just to announce their "function-in-life" as the stimuli files, or
  • _stim-<label> entities if multiple stim-uluses for the same data recording.
• generally, we could have define stim/ folder at sub/[ses/] level, to follow the beh/ and other datatypes, to group stimuli per sub/ses, and then avoid having _mod-stim under those since implied, if so was desired but I think it is not needed hence

Then as a result, for my use-case, since I have only 1 type of stimulus recording, if I were to place stimuli files near data file e.g. sub-01_task-forrest_bold.nii.gz , I think I should name it in shortest form as

sub-01_task-forrest_mod-stim_audiovideo.mp4

and following #2283 we could have modalities.tsv (not to be done in this BEP) which could then be used to describe characteristics of stim here. That would also allow to e.g. carry nearby behavior recording (e.g. video camera of the body in the magnet to e.g. assess movements!) under sub-01_task-forrest_mod-beh_audiovideo.mp4.

or if I wanted to be descriptive for my case, although not needed to tell files apart, could still have either

sub-01_task-forrest_mod-stim_recording-reprostim_audiovideo.mp4

or the "mixin" between the two as using the entity with a label would define the data type (stimuli) and provide characteristic:

sub-01_task-forrest_stim-reprostim_audiovideo.mp4

where in stimuli.tsv I would describe reprostim value.

[yarikoptic]

@neuromechanist can you please update the PR merging the #2367 and also addressing the conflicts . What is your plan towards making it available for review?