Skip to content

Fix BatchRotatingKVCache rotated flag deserializing to True#1251

Open
odysa wants to merge 1 commit into
ml-explore:mainfrom
odysa:fix/batch-rotating-kv-cache-rotated
Open

Fix BatchRotatingKVCache rotated flag deserializing to True#1251
odysa wants to merge 1 commit into
ml-explore:mainfrom
odysa:fix/batch-rotating-kv-cache-rotated

Conversation

@odysa
Copy link
Copy Markdown

@odysa odysa commented May 6, 2026

Fixes #1250.

meta_state round-trips self.rotated through str() on save and bool() on load, but bool("False") is True (any non-empty string is truthy). A cache saved with rotated=False reloads as rotated=True, silently corrupting the rotation mask in subsequent make_mask calls.

Fix: parse the bool by string comparison.

- self.rotated = bool(v[3])
+ self.rotated = v[3] == "True"

Added a regression test (test_save_load_batch_rotating_cache) that round-trips both rotated=False and rotated=True. Verified the test fails on main and passes with the fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BatchRotatingKVCache: rotated=False reloads as True from saved prompt cache

1 participant