Fix AOF persistence and WATCH for collection-emptying RMW ops [dev]#1677
Open
Fix AOF persistence and WATCH for collection-emptying RMW ops [dev]#1677
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes correctness issues around object-store RMW operations that empty collections, ensuring they are persisted to AOF and correctly tracked by WATCH, and addresses multi-DB AOF recovery double-dispose behavior.
Changes:
- Ensure collection-emptying object RMW mutations (e.g., LPOP/ZREM/HDEL/SREM that delete the key) set
NeedAofLogso AOF replay preserves deletions. - Increment WATCH version on the “remove key” path for in-place object updates so WATCH/MULTI/EXEC detects key modifications.
- Move AOF replay session disposal to
AofProcessor.Dispose()to avoid double-dispose during multi-DB recovery; add regression tests for these scenarios.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| test/Garnet.test/TransactionTests.cs | Adds a WATCH regression test for list-emptying LPOP within a watched transaction flow. |
| test/Garnet.test/RespAofTests.cs | Adds AOF recovery tests covering object-store RMW deletes/partial deletes across lists, sets, hashes, and sorted sets. |
| test/Garnet.test/MultiDatabaseTests.cs | Adds multi-DB AOF recovery regression test for collection-emptying mutations across DBs. |
| libs/server/Storage/Functions/ObjectStore/RMWMethods.cs | Fixes AOF logging + WATCH version increments when RMW operations remove the key. |
| libs/server/AOF/AofProcessor.cs | Centralizes disposal in AofProcessor.Dispose() to avoid double-dispose across multi-DB recovery passes. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Bug 1: Collection-emptying RMW operations (LPOP, ZREM, HDEL, SREM that empty a collection) were not logged to AOF. InPlaceUpdaterWorker and PostCopyUpdater returned false before setting NeedAofLog when HasRemoveKey was true, so PostRMWOperation never wrote the AOF entry. Data reappeared after restart. Bug 2: InPlaceUpdaterWorker's HasRemoveKey path did not call IncrementVersion on the watch version map, so WATCH/MULTI/EXEC transactions did not detect the key modification and incorrectly committed. Bug 3: AofProcessor.RecoverReplay disposed respServerSession in its finally block, but in multi-DB recovery it ran once per database, causing double-dispose and NullReferenceException in GarnetLatencyMetricsSession.Return(). Moved dispose to AofProcessor.Dispose() which runs exactly once. Fixes #1675 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2403184 to
12b88a8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug 1: Collection-emptying RMW operations (LPOP, ZREM, HDEL, SREM that empty a collection) were not logged to AOF. InPlaceUpdaterWorker and PostCopyUpdater returned false before setting NeedAofLog when HasRemoveKey was true, so PostRMWOperation never wrote the AOF entry. Data reappeared after restart.
Bug 2: InPlaceUpdaterWorker's HasRemoveKey path did not call IncrementVersion on the watch version map, so WATCH/MULTI/EXEC transactions did not detect the key modification and incorrectly committed.
Bug 3: AofProcessor.RecoverReplay disposed respServerSession in its finally block, but in multi-DB recovery it ran once per database, causing double-dispose and NullReferenceException in GarnetLatencyMetricsSession.Return(). Moved dispose to AofProcessor.Dispose() which runs exactly once.
Fixes #1675