bridge: Skip unreadable PCP archives instead of failing the whole history - #23304
Open
officialasishkumar wants to merge 1 commit into
Open
bridge: Skip unreadable PCP archives instead of failing the whole history#23304officialasishkumar wants to merge 1 commit into
officialasishkumar wants to merge 1 commit into
Conversation
…tory
The metrics channel globs every *.index file in the pmlogger archive
directory and opens each one as a PCP archive context. Until now only
PM_ERR_LOGFILE (a missing data volume) was tolerated; any other error
while opening a single archive raised ChannelError('not-found'), which
aborted reading of *all* archives in the directory.
As a result a single damaged archive disabled the entire metrics history
in the web UI even though the remaining archives were perfectly readable.
This happens in practice with an empty *.index file left behind by a
freshly rotated or interrupted pmlogger (PM_ERR_NODATA / PM_ERR_LABEL)
and with a corrupted record (PM_ERR_LOGREC).
Treat an unreadable individual archive the same way as a missing data
volume: log a warning and skip it, so the history from the healthy
archives keeps loading. When no archive in the directory can be read at
all, get_archives() still reports 'not-found', matching an empty or
non-existent source.
Fixes: cockpit-project#23023
Fixes: cockpit-project#23214
jelly
reviewed
May 29, 2026
| # healthy archives in this directory. Skip it and carry on; | ||
| # get_archives() still reports 'not-found' when *no* archive in | ||
| # the directory can be read. | ||
| if exc.errno() != c_api.PM_ERR_LOGFILE: |
Member
There was a problem hiding this comment.
So we already skip unreadable archive files as PM_ERR_LOGFILE is:
#define PM_ERR_LOGFILE (-PM_ERR_BASE-30) /* Missing PCP archive file */
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The metrics channel globs every
*.indexfile in the pmlogger archivedirectory and opens each one as a PCP archive context. Until now only
PM_ERR_LOGFILE(a missing data volume) was tolerated when opening anarchive; any other error raised
ChannelError('not-found'), whichaborted reading of all archives in the directory.
As a result a single damaged archive disabled the entire metrics history
in the web UI, even though the remaining archives were perfectly
readable. This shows up in practice in two reported ways:
*.indexfile left behind by a freshly rotated or interruptedpmlogger (
PM_ERR_NODATA/PM_ERR_LABEL) — Empty metrics history file in pmlogger causes metrics page to fail loading in web UI #23214PM_ERR_LOGREC) — Performance Metrics Archive Logger fails on corrupted record. #23023In both cases the user only sees "Metrics history could not be loaded"
with no indication of which file is at fault.
Fix
Treat an unreadable individual archive the same way as a missing data
volume: log a warning naming the offending file and skip it, so the
history from the healthy archives keeps loading. When no archive in the
directory can be read at all,
get_archives()still reportsnot-found,matching the behaviour of an empty or non-existent source.
Testing
test_pcp_partially_broken_archiveputs a healthy archive next toan unreadable one and asserts the healthy archive's samples are still
returned. It fails before the change and passes after.
test_pcp_broken_archiveis updated to assert that a directory whereevery archive is unreadable still reports a bare
not-found.test/pytest/test_pcp.pysuite passes (17 tests).Fixes: #23023
Fixes: #23214
Metrics history survives a single corrupt PCP archive
A single corrupted or empty PCP archive (for example one left behind by an
interrupted or freshly rotated pmlogger) no longer disables the whole
metrics history page. Cockpit now skips the unreadable archive, logs a
warning naming it, and shows the data from the remaining healthy archives.