Skip to content

fix(viewer): support cbor2 6.x deserialization (frozendict + tuple)#2390

Open
Bahtya wants to merge 2 commits intonewton-physics:mainfrom
Bahtya:fix/cbor2-6x-deserialization
Open

fix(viewer): support cbor2 6.x deserialization (frozendict + tuple)#2390
Bahtya wants to merge 2 commits intonewton-physics:mainfrom
Bahtya:fix/cbor2-6x-deserialization

Conversation

@Bahtya
Copy link
Copy Markdown

@Bahtya Bahtya commented Apr 9, 2026

Summary

Fixes #2375

cbor2 6.0 introduces two breaking changes:

  • Nested dicts return as cbor2.frozendict — a Mapping, but not a dict subclass
  • Arrays return as tuple instead of list

The isinstance(data, dict) guards in viewer_file.py silently short-circuit on frozendict, so deserialization does nothing and recorder round-trip tests fail.

Changes

File Change
newton/_src/viewer/viewer_file.py Replace isinstance(x, dict)isinstance(x, Mapping) in deserialize(), transfer_to_model(), the warp callback, and _resolve_cache_refs(). Also update _resolve_cache_refs to handle tuple the same as list (cbor2 6.x arrays), always returning a plain list.
pyproject.toml Remove the <6 upper bound added as a workaround in #2373.

Test plan

  • pytest newton/tests/ -k recorder passes with cbor2>=5.7.0,<6
  • pytest newton/tests/ -k recorder passes with cbor2>=6.0 (once 6.0 is released)
  • pip install 'cbor2>=5.7.0' installs without version conflict

Notes

Mapping is already imported from collections.abc at the top of the file, so no new imports are needed.

Bahtya

Summary by CodeRabbit

  • Bug Fixes

    • Clarified install hint for a binary serialization dependency.
  • Improvements

    • Broadened compatibility with newer versions of the binary serialization library.
    • Improved handling of varied mapping types during binary deserialization and data transfer, increasing robustness across frozen/immutable mappings.

cbor2 6.0 introduces two breaking changes in deserialization:
- Nested dicts return as `cbor2.frozendict` (a Mapping, but no longer a
  `dict` subclass)
- Arrays return as `tuple` instead of `list`

The `isinstance(data, dict)` guards in `deserialize()`,
`transfer_to_model()`, and `_resolve_cache_refs()` silently short-circuit
on `frozendict`, causing recorder round-trip tests to fail.

Fix all three sites by switching to `isinstance(x, Mapping)` (already
imported from `collections.abc`). Additionally update `_resolve_cache_refs`
to treat `tuple` the same as `list` when iterating, and always return a
plain `list` so callers aren't surprised by the cbor2 change.

Also removes the `<6` upper bound on the cbor2 dependency that was added
as a short-term workaround in newton-physics#2373, and updates the install hint in the
ImportError message.

Fixes newton-physics#2375

Bahtya
@linux-foundation-easycla
Copy link
Copy Markdown

linux-foundation-easycla bot commented Apr 9, 2026

CLA Missing ID CLA Not Signed

@Bahtya Bahtya had a problem deploying to external-pr-approval April 9, 2026 10:18 — with GitHub Actions Error
@Bahtya Bahtya had a problem deploying to external-pr-approval April 9, 2026 10:18 — with GitHub Actions Error
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Apr 9, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

Run ID: 7ef22f66-88b9-4bec-9f9b-ed0dd2d5af97

📥 Commits

Reviewing files that changed from the base of the PR and between fd4853c and 21de760.

📒 Files selected for processing (1)
  • newton/_src/viewer/viewer_file.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • newton/_src/viewer/viewer_file.py

📝 Walkthrough

Walkthrough

Updated viewer deserialization to accept generic Mapping types (e.g., frozendict) and tuples from cbor2 6.x; resolver now returns plain dict for resolved mappings. The cbor2 example dependency upper-version bound was removed.

Changes

Cohort / File(s) Summary
Viewer deserialization
newton/_src/viewer/viewer_file.py
Generalized checks from dict to typing.Mapping across transfer_to_model, deserialize, depointer_as_key, and _resolve_cache_refs; treat cache-ref containers as any Mapping, recurse mappings, and return plain dict for resolved mappings. Adjusted missing-cbor2 install hint for .bin to pip install 'cbor2>=5.7.0'.
Dependency constraint
pyproject.toml
Removed the upper-bound <6 from the examples optional dependency for cbor2, changing cbor2>=5.7.0, <6cbor2>=5.7.0.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately and specifically describes the main change: supporting cbor2 6.x deserialization by handling frozendict and tuple returns.
Linked Issues check ✅ Passed The PR successfully addresses all coding requirements from #2375: replaces isinstance(x, dict) with Mapping checks, handles tuple arrays, and removes the cbor2 <6 constraint.
Out of Scope Changes check ✅ Passed All changes are directly scoped to the linked issue requirements: viewer_file.py refactoring for Mapping support and pyproject.toml dependency constraint removal.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@newton/_src/viewer/viewer_file.py`:
- Around line 1014-1016: The current _resolve_cache_refs post-pass
unconditionally converts tuples to lists; change it to preserve real tuples by
branching on type: if obj is a list, return a new list with _resolve_cache_refs
applied to each element; if obj is a tuple, return a tuple constructed from
applying _resolve_cache_refs to each element (so genuine tuple-valued fields
keep their type). Keep the existing behavior for other iterables unchanged and
reference the _resolve_cache_refs function (and the surrounding deserialize
flow) when making the change.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: aa50d953-209f-471d-997c-cdf2f1e10542

📥 Commits

Reviewing files that changed from the base of the PR and between 8d7a6e2 and fd4853c.

📒 Files selected for processing (2)
  • newton/_src/viewer/viewer_file.py
  • pyproject.toml

Comment thread newton/_src/viewer/viewer_file.py Outdated
The previous implementation converted both lists and tuples to lists in
the post-pass, causing genuine tuple-valued fields to round-trip as a
different type. Now lists resolve to lists and tuples resolve to tuples,
preserving the original container type for all non-cache-ref fields.

Bahtya
@Bahtya
Copy link
Copy Markdown
Author

Bahtya commented Apr 9, 2026

Thanks for the review! I've addressed the CodeRabbit feedback:

Preserve real tuples in _resolve_cache_refs: The original code converted both list and tuple to a list, which meant genuine tuple-valued fields would round-trip as a different type. I've now split the check so that list objects resolve to list and tuple objects resolve to tuple, preserving the original container type through the post-pass.

@Bahtya Bahtya requested a deployment to external-pr-approval April 9, 2026 17:28 — with GitHub Actions Waiting
@Bahtya Bahtya requested a deployment to external-pr-approval April 9, 2026 17:28 — with GitHub Actions Waiting
@Bahtya
Copy link
Copy Markdown
Author

Bahtya commented Apr 12, 2026

Friendly ping 👋 This PR adds support for cbor2 6.x deserialization by handling frozendict and other type changes. CodeRabbit review found no issues. Would appreciate a review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support cbor2 6.x deserialization changes

1 participant