Skip to content

Fix "Event loop is closed" race in EventLoopThread.force_stop #727

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
puddly merged 2 commits into from
Jul 21, 2026
Merged

Fix "Event loop is closed" race in EventLoopThread.force_stop #727

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
19e86f5
Fix "Event loop is closed" race in EventLoopThread.force_stop
nmingam Jul 1, 2026
1040175
Use contextlib.suppress for the force_stop race guard
nmingam Jul 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 9 additions & 7 deletions bellows/thread.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import asyncio
from concurrent.futures import ThreadPoolExecutor
import contextlib
import functools
import logging

Expand Down Expand Up @@ -52,21 +53,22 @@
return thread_complete

def force_stop(self):
if self.loop is None:
loop = self.loop
if loop is None or loop.is_closed():
return

def cancel_tasks_and_stop_loop():
tasks = asyncio.all_tasks(loop=self.loop)
tasks = asyncio.all_tasks(loop=loop)

for task in tasks:
self.loop.call_soon_threadsafe(task.cancel)
loop.call_soon_threadsafe(task.cancel)

gather = asyncio.gather(*tasks, return_exceptions=True)
gather.add_done_callback(
lambda _: self.loop.call_soon_threadsafe(self.loop.stop)
)
gather.add_done_callback(lambda _: loop.call_soon_threadsafe(loop.stop))

self.loop.call_soon_threadsafe(cancel_tasks_and_stop_loop)
# The worker thread may close the loop after our is_closed() check.
with contextlib.suppress(RuntimeError):

@zigpy-review-bot zigpy-review-bot Jul 21, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional follow-up, not for this PR: the read side is now safe, but the write side that creates the window is still bellows/thread.py:29-31:

finally:
    self.loop.close()
    self.loop = None

Swapping those to publish None before closing narrows the window in which self.loop names a closed loop:

finally:
    loop, self.loop = self.loop, None
    loop.close()

That doesn't remove the need for this suppress — a caller can still snapshot the loop just before close() runs — but it also protects the two other readers of self.loop that this PR doesn't touch: run_coroutine_threadsafe() (line 19) passes self.loop straight to asyncio.run_coroutine_threadsafe, which raises the same RuntimeError on a closed loop, and start()'s restart guard (line 35). It's safe for start() either way, since a None loop already falls through to creating a new one.

Entirely fine to leave for a separate change — the flake this PR targets is fixed as-is.

@TheJulianJES TheJulianJES Jul 28, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zigpy-review-bot Create an issue or PR to follow up on this.

@zigpy-review-bot zigpy-review-bot Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opened #739 for this.

It does the swap suggested above — loop, self.loop = self.loop, None before loop.close() — so self.loop is either None or an open loop for every reader, not just the one force_stop guards. While writing it up, one extra benefit turned up that the original comment did not mention: the old trailing self.loop = None ran after close(), so a start() racing the shutdown could have its freshly installed loop clobbered back to None by the exiting thread. The swap removes that write entirely.

The suppress(RuntimeError) this PR added stays — it is still reachable, and no ordering in _thread_main can remove it.

test_thread_clears_loop_before_closing pins the ordering itself (it records what self.loop is at the moment close() is called); it fails on dev and passes with the change. Full suite: 441 passed.

loop.call_soon_threadsafe(cancel_tasks_and_stop_loop)


class ThreadsafeProxy:
Expand Down Expand Up @@ -98,7 +100,7 @@
# Disconnected
LOGGER.warning("Attempted to use a closed event loop")
return
if asyncio.iscoroutinefunction(func):

Check warning on line 103 in bellows/thread.py

View workflow job for this annotation

GitHub Actions / shared-ci / Run tests Python 3.14 

'asyncio.iscoroutinefunction' is deprecated and slated for removal in Python 3.16; use inspect.iscoroutinefunction() instead

Check warning on line 103 in bellows/thread.py

View workflow job for this annotation

GitHub Actions / shared-ci / Run tests Python 3.14 

'asyncio.iscoroutinefunction' is deprecated and slated for removal in Python 3.16; use inspect.iscoroutinefunction() instead

Check warning on line 103 in bellows/thread.py

View workflow job for this annotation

GitHub Actions / shared-ci / Run tests Python 3.14 

'asyncio.iscoroutinefunction' is deprecated and slated for removal in Python 3.16; use inspect.iscoroutinefunction() instead

Check warning on line 103 in bellows/thread.py

View workflow job for this annotation

GitHub Actions / shared-ci / Run tests Python 3.14 

'asyncio.iscoroutinefunction' is deprecated and slated for removal in Python 3.16; use inspect.iscoroutinefunction() instead

Check warning on line 103 in bellows/thread.py

View workflow job for this annotation

GitHub Actions / shared-ci / Run tests Python 3.14 

'asyncio.iscoroutinefunction' is deprecated and slated for removal in Python 3.16; use inspect.iscoroutinefunction() instead

Check warning on line 103 in bellows/thread.py

View workflow job for this annotation

GitHub Actions / shared-ci / Run tests Python 3.14 

'asyncio.iscoroutinefunction' is deprecated and slated for removal in Python 3.16; use inspect.iscoroutinefunction() instead

Check warning on line 103 in bellows/thread.py

View workflow job for this annotation

GitHub Actions / shared-ci / Run tests Python 3.14 

'asyncio.iscoroutinefunction' is deprecated and slated for removal in Python 3.16; use inspect.iscoroutinefunction() instead

Check warning on line 103 in bellows/thread.py

View workflow job for this annotation

GitHub Actions / shared-ci / Run tests Python 3.14 

'asyncio.iscoroutinefunction' is deprecated and slated for removal in Python 3.16; use inspect.iscoroutinefunction() instead
future = asyncio.run_coroutine_threadsafe(call(), loop)
return asyncio.wrap_future(future, loop=curr_loop)
else:
Expand Down
Loading