fix(logfiles): share look-back context across log rotation - #3581
Conversation
The look-back buffer for ignoring mapped errors was reset for every version of a log file. A preceding message at the end of a rotated log file (e.g. "Switched to a fork") could not make a mapped error at the beginning of the next version of the log file ignored, and the error was reported as a false failure. Share the look-back buffer across all versions of the log file. The versions are searched from oldest to newest, so the buffer follows the chronological order of the log lines. Break ties in the modification time sorting with the rotation index, so the order is deterministic on filesystems with coarse timestamps. Also check all look-back map pairs for a line - a line can match the 'error' key of multiple pairs and it is enough when any of the matching pairs has its preceding message in the look-back buffer.
There was a problem hiding this comment.
🟡 Not ready to approve
The newly added from collections import deque import in framework_tests/test_logfiles.py is likely out of Ruff isort order (I001) and will fail lint until imports are reordered.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.
Pull request overview
This PR fixes false-positive log failures by making the “look-back” buffer for mapped/ignored errors span across rotated log files, preserving chronological context when errors and their suppressing “trigger” messages land in different rotation segments.
Changes:
- Share the look-back deque across all rotated versions of a log while scanning oldest → newest.
- Make rotated-log ordering deterministic when mtimes tie by breaking ties with the rotation index.
- Ensure mapped-error suppression checks all matching look-back pairs for a line, not just the first match, and add unit tests for these cases.
File summaries
| File | Description |
|---|---|
| framework_tests/test_logfiles.py | Adds regression tests for look-back behavior across rotation, multi-pair matching, and deterministic rotated-log ordering. |
| cardano_node_tests/utils/logfiles.py | Implements shared look-back across rotated logs, deterministic ordering with a rotation-index tiebreak, and multi-pair look-back matching. |
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 1
- Review effort level: Lite
We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.
The look-back buffer for ignoring mapped errors was reset for every version of a log file. A preceding message at the end of a rotated log file (e.g. "Switched to a fork") could not make a mapped error at the beginning of the next version of the log file ignored, and the error was reported as a false failure.
Share the look-back buffer across all versions of the log file. The versions are searched from oldest to newest, so the buffer follows the chronological order of the log lines. Break ties in the modification time sorting with the rotation index, so the order is deterministic on filesystems with coarse timestamps.
Also check all look-back map pairs for a line - a line can match the 'error' key of multiple pairs and it is enough when any of the matching pairs has its preceding message in the look-back buffer.