Make watchForChanges actually reload the KeyStore - #1842
Open
kevinherron wants to merge 1 commit into
Open
Conversation
The setting could not have worked. Four separate defects, any one of which was enough on its own: The watch key was never reset, and a WatchKey stays signalled and is never queued again until it is. The watcher could therefore fire at most once, after which take() blocked forever. processWatchEvent compared the event context against the KeyStore path, but a directory watch reports a context relative to the watched directory while toAbsolutePath resolves against the working directory. Unless the KeyStore happened to sit in the working directory, the comparison never matched. Only ENTRY_MODIFY was registered. Replacing a file by renaming a temporary one over it arrives as a creation, so the safe way to update a KeyStore produced no event at all. This store now writes its own file that way. The reload itself only called loadEntries(), which re-queries the KeyStore already held in memory. Even if an event had arrived, the file was never re-read, so an externally renewed certificate could not have been seen. Reloading now reads into a new KeyStore and swaps it in only once it has loaded, so a file that is unreadable, or still being written, leaves the one in use untouched. Overflow events are treated as a change, since there is no way to tell what was dropped. Closing the store also threw ClosedWatchServiceException out of the watcher thread rather than ending it, so shutdown left an uncaught exception behind. The end-to-end test fails against the previous implementation, timing out after 30 seconds; it passes in well under a second here.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #1841 — review that one first. The diff shown here is against
fix/keystore-certificate-store-writesand will shrink to just the watcher changes once #1841 merges.KeyStoreCertificateStore.Settings.watchForChangespromises that a keystore replaced on disk gets picked up without restarting the server. It could not have worked, for several independent reasons, any one of which was enough on its own.The watcher never called
reset()on itsWatchKey. A key stays signalled and is never queued again until it's reset, so after the first batch of eventstake()blocked for the life of the process — the watcher could fire at most once. That wouldn't have helped anyway, because the event filter comparedp.toAbsolutePath()against the keystore path, and a directory watch reports a context relative to the watched directory whiletoAbsolutePath()resolves againstuser.dir. Unless the keystore happened to sit in the process working directory, the comparison never matched. On top of that onlyENTRY_MODIFYwas registered, so replacing a keystore the safe way — renaming a temporary file over it, which is exactly what this store does as of #1841 — produced no event at all; on Linux that registration becomes an inotify mask thatIN_MOVED_TOisn't part of.Underneath all of it, the reload didn't reload. It cleared the entry cache and called
loadEntries(), which re-queries theKeyStorealready held in memory sinceinitialize(). The cache was emptied and immediately refilled with identical stale entries. Even with everything above fixed, the file was never re-read, so a renewed certificate could not have been seen.The watcher now resets its key, registers
ENTRY_CREATEalongsideENTRY_MODIFY, and resolves event contexts against the directory the watch was registered on. Reloading reads into a newKeyStoreand swaps it in only once it has loaded, so a file that's unreadable, or still being written, leaves the one in use untouched. Overflow events are treated as a change, since there's no way to tell what was dropped.One more thing surfaced while testing this.
close()closes theWatchServicewhile the watcher thread is parked intake(), which raisesClosedWatchServiceException. Nothing caught it, so it propagated out ofrun()and every shutdown left an uncaught exception behind. The thread now exits cleanly, and an interrupt duringclose()restores the interrupt flag instead of swallowing it.Testing
The end-to-end test enables the setting, has a second store rewrite the file, and waits for the first one to notice. I ran it against the previous implementation to confirm it earns its keep: it times out there after thirty seconds and passes in well under a second against this one. A test that drives a feature purely through the public API is worth very little if it passes either way, and this one didn't.
Narrower tests pin the reload-from-disk behaviour without involving a watch service at all, so a regression there fails deterministically instead of as a timeout, and cover the event path resolution and overflow handling directly.
🤖 Generated with Claude Code