Make where() materialise the CA bundle exactly once - #433
Open
Meet-1010 wants to merge 1 commit into
Open
Conversation
The 'if _CACERT_PATH is None' test and the assignment that publishes the path are separate operations. Concurrent callers could each enter their own resource context and register their own atexit hook, but only the last _CACERT_CTX assignment survived, orphaning the earlier context managers. Under zipimport as_file() extracts a temporary file, so an orphan is a file nothing will clean up, and the duplicate atexit hooks can close a context whose path another thread is still holding. Use double-checked locking and publish _CACERT_PATH last, so no thread can observe a path whose context manager is not yet reachable for cleanup. Both the importlib.resources branch and the fallback branch are covered. The new test fails 5/5 runs without this change and passes with it.
alex
reviewed
Aug 25, 2026
| ) | ||
|
|
||
|
|
||
| class TestWhereThreadSafety(unittest.TestCase): |
Member
There was a problem hiding this comment.
this test involves so much monkeypatching that I'm personally pretty skeptical of its usefulness.
Member
There was a problem hiding this comment.
I think we can probably drop teh test entirely tbh
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
where()materialises the CA bundle lazily:The
is Nonetest and the assignment that publishes the path are separate operations. Concurrent callers can all pass the test; each opens its own context manager and registers its ownatexithook, but only the last assignment to_CACERT_CTXsurvives, so the earlier context managers are orphaned — nothing holds them, andexit_cacert_ctxruns once per extra registration against whichever context happens to be current.In the common filesystem install
as_filereturns the real path and__exit__is a no-op, so the damage is bounded to redundantatexitregistrations. When certifi is imported from a zip or wheel,as_fileextracts a temporary file and__exit__deletes it — then an orphaned context is a temp file nobody cleans up, and the duplicate hooks can delete a path another thread is still holding.This is not free-threading-specific
It reproduces with the GIL enabled too — the GIL just hid it after the first call warmed the cache. Measured with 16 threads over 200 trials on CPython 3.14.4:
PYTHON_GIL=1So this is a general thread-safety bug that free-threading makes reliable rather than occasional.
The change
Double-checked locking, with
_CACERT_PATHpublished last so no thread can observe a path whose context manager is not yet reachable for cleanup. Both thesys.version_info >= (3, 11)branch and the fallback branch are patched.Tests
TestWhereThreadSafetyincertifi/tests/test_certify.pycounts entries into the resource context by temporarily replacingcertifi.core.atexit.registerwith a counter, and asserts exactly one per cold start. It fails 5/5 runs without the change and passes with it.It waits on an
Eventrather than aBarrierand skips if fewer than two threads can be started, so a thread-constrained runner degrades rather than deadlocks. All globals and the patchedregisterare restored in afinally.Full suite: 4 passed, both free-threaded and with
PYTHON_GIL=1.Environment: CPython 3.14.4 free-threaded, macOS 15 / arm64.