perf: Pause online delete if there any any gaps in recent ledger history#5531
perf: Pause online delete if there any any gaps in recent ledger history#5531ximinez wants to merge 275 commits into
Conversation
b2c4c3d to
54f7f3c
Compare
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
|
I've been running this code locally off and on, and one thing I notice is I think this is ok, because that processing includes writing the ledger to disk, and is extremely important. So it makes sense to wait for it. I was concerned that this would cause rotation to stall indefinitely, and while it does pause a lot, it doesn't pause on every ledger, so it does make progress and eventually finish. I am, however, changing the default |
b1a5dc1 to
e13baa5
Compare
ximinez
added
the
JiraRefresh
bthomee
added
JiraRefresh
This comment has been minimized.
This comment has been minimized.
…te-gaps * XRPLF/develop: test: Add tests for check doxygen style (7795) style: Add pre-commit hook to check doxygen style (7794) style: Unify style for all Doxygen comments (7776)
This comment has been minimized.
This comment has been minimized.
![xrplf-ai-reviewer[bot]](https://avatars.githubusercontent.com/u/67929741?s=60&v=4){kind=small}
High Level Overview of Change
Adds another check to the online deletion process to prevent ledger gaps from persisting if the node goes out of sync.
Context of Change
Some nodes struggle to stay in sync when online delete is running due to the added I/O burden. This has proven difficult to avoid without upgrading hardware. If the process takes a particularly long time, which it can with large datasets, the node can end up with many gaps in the recent ledger history. Those gaps will not be filled until the node is idle enough for low-priority historical ledger requests to run.
Note that this is a trade-off. The online deletion may take longer to complete, but the node will stay more up-to-date.
Type of Change