Skip to content

Add source_video link field to PoseEstimation#56

Open
h-mayorquin wants to merge 3 commits intorly:mainfrom
h-mayorquin:create_formal_links
Open

Add source_video link field to PoseEstimation#56
h-mayorquin wants to merge 3 commits intorly:mainfrom
h-mayorquin:create_formal_links

Conversation

@h-mayorquin
Copy link
Copy Markdown
Contributor

Adds an optional source_video link to ImageSeries on PoseEstimation, consistent with how TrainingFrame already links to its source video. This provides a formal NWB reference to the source video instead of relying on the string paths in original_videos, which can become stale or break when files are moved (e.g., dandi/dandi-cli#1817).

Closes #12 and supersedes #13 with a simpler approach that requires no custom IO mapper.

To discuss: the current approach keeps original_videos is kept for backwards compatibility but I am unsure if we should remove this and bumpb the schema source. I think we can leep it here and decide this later

@h-mayorquin
Copy link
Copy Markdown
Contributor Author

We are discussing whether this should be source_videos.

The thing is that original_videos is plural because the container is meant for multiple cameras. DANNCE:

https://github.com/spoonsso/dannce

@h-mayorquin
Copy link
Copy Markdown
Contributor Author

Add labeled_video as well for consistency.

@rly
Copy link
Copy Markdown
Owner

rly commented Apr 24, 2026

@h-mayorquin and I discussed this over zoom. PoseEstimation does not actually work well for the multi-camera use case. PoseEstimationSeries stores pose estimate time series for a single keypoint for a single camera (coordinates are noted as pixel-based). There is no defined way to store the pose estimates in a global / experimental space, e.g., 3D coordinates after triangulation from multiple cameras. This should probably be stored in a different way, or at least labeled differently. So we were thinking of keeping PoseEstimation for the single camera case, which is the most common use case that I am aware of. Separately, we can add a MultiCameraPoseEstimation object to handle storage of multiple camera videos, linking between PoseEstimationSeries and camera / camera video, and storage of the 3D pose estimates in a global space. It could also store calibration information and positions of the cameras relative to one another. This should be driven by actual multi-camera use cases like DANNCE.

@h-mayorquin
Copy link
Copy Markdown
Contributor Author

Thanks, @rly I added the requested parameter now and the changelog.

@CodyCBakerPhD
Copy link
Copy Markdown

So we were thinking of keeping PoseEstimation for the single camera case, which is the most common use case that I am aware of. Separately, we can add a MultiCameraPoseEstimation object to handle storage of multiple camera videos, linking between PoseEstimationSeries and camera / camera video, and storage of the 3D pose estimates in a global space.

Sounds good to me!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

original_videos should be a link

3 participants