Retain DeepEP high-throughput handle eagerly

[fergusfinn]

Base: main (upstream vLLM v0.22.1)

This extracts the DeepEP high-throughput eager-handle diagnostic from the Isambard/UCCL launch path into a reviewable branch. DeepEPHTAll2AllManager now initializes and retains a buffer handle during construction, before live traffic.

This is intentionally marked experimental: it matches the workaround used during debugging, but it still needs runtime validation and a clearer explanation of the underlying lifetime issue before it should be treated as a final fix.

Validation:

• python3 -m py_compile vllm/distributed/device_communicators/all2all.py
• git diff --check

Runtime validation on Isambard is still pending.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

PRs do not trigger a full CI run by default. Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

Agent Guidelines

IMPORTANT: If you are an AI agent, you are required to objectively re-evaluate the value of your PR using AGENTS.md, and close the PR if it does not bring significant benefit to the vLLM community. Failure to do so may result in an immediate ban.

🚀

[doubleword-code]

Summary

This PR adds an eager handle creation in DeepEPHTAll2AllManager.__init__ to retain DeepEP high-throughput buffers before live traffic. The change is minimal (2 lines) and serves diagnostic purposes for the Isambard/UCCL environment—enabling early error detection and better visibility into buffer allocation.

Verdict: The change is reasonable for its stated diagnostic purpose, but has a minor cleanup hygiene issue that should be addressed.

Research Notes

• Examined the Cache class in base_device_communicator.py: uses WeakValueDictionary which holds weak references to values. Without a strong reference elsewhere, handles could be garbage collected between uses.
• The get_handle() method stores handles in the cache via get_or_create(), so the eager handle IS included in the existing cleanup path.
• No other All2All manager implementations (DeepEPLLAll2AllManager, MoriAll2AllManager, FlashInfer managers) use eager handle creation—this pattern is specific to this diagnostic need.
• DeepEP documentation could not be fetched, but the codebase usage shows handles are deep_ep.Buffer objects that allocate GPU/RDMA resources.

Suggested Next Steps

1. Address the stale reference after destroy(): Clear self._eager_handle in the destroy() method to avoid holding a reference to a destroyed handle object.
2. Consider documenting when this diagnostic pattern might be removed or made conditional (e.g., behind a debug flag) if it's not needed long-term.

General Findings

The implementation correctly leverages the existing caching infrastructure. The eager handle will be properly destroyed via destroy() since it's stored in handle_cache._cache. However, the instance variable self._eager_handle will remain as a dangling reference post-destruction—a minor issue but worth cleaning up for code hygiene.

[doubleword-code]

Non-blocking: Eager handle creation for diagnostics is reasonable, but consider clearing this reference in destroy() to avoid holding a stale reference post-cleanup.

Why it matters: After destroy() clears handle_cache._cache and calls handle.destroy() on cached handles, self._eager_handle remains as a reference to a destroyed handle object. While the manager lifecycle typically ends at destroy() (making this functionally harmless), clearing it maintains cleaner object semantics and avoids potential confusion if the manager is ever inspected post-destruction.

Suggested fix: Add self._eager_handle = None at the end of the destroy() method in DeepEPHTAll2AllManager (or in the parent class if this pattern might be reused).

[fergusfinn]

Closing per Fergus: keep vllm-isambard scoped to PR #19 for now and drop these experimental changes from the integration branch.