[WideEP] Update DeepEP version in Dockerfile by tlrmchlsmth · Pull Request #45321 · vllm-project/vllm

tlrmchlsmth · 2026-06-11T21:20:54Z

Update DeepEP to deepseek-ai/DeepEP@d4f41e4 so that we can pick up DeepEPv2.

Missed this in #41183

Ideally we'd have a single source of truth for the commit across both the install script and the Dockerfile but I don't see a sane way to do it at the moment

PyTorch ships an earlier version of NCCL. Per feedback from the NCCL team, I think this should be safe as

libnccl.so is designed to be backward compatible. This means you can compile your program with an older version (e.g., v1.28) and run it with a newer version at runtime (e.g., v1.30). However, the actual runtime environment should only use one version of NCCL at a time.

For the pypi wheel, no change is needed since we don't ship DeepEP in the wheel.

tlrmchlsmth · 2026-06-11T21:44:05Z

Actually an issue here -- we need to update NCCL to > 2.30.4 during DeepEP compilation as well, not just rely on installing it afterwards like I thought we could

DeepEPv2 (ElasticBuffer API) requires NCCL >= 2.30.4 at both compile and runtime. NCCL backward compatibility is guaranteed by NVIDIA, so installing a newer NCCL after PyTorch is safe. - Pin NCCL 2.30.4-1+cuda13.2 in the base build stage so the extensions-build stage inherits it for DeepEP compilation - Pin the same version in the vllm-base runtime stage, replacing the dynamic apt-cache madison lookup - Update DEEPEP_COMMIT_HASH from 73b6ea4 to d4f41e4e93 to pick up DeepEPv2 Co-authored-by: Claude <noreply@anthropic.com> Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>

claude

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.

_{Tip: disable this comment in your organization's Code Review settings.}

The CUDA base image holds NCCL packages, so installing them in the base stage without --allow-change-held-packages fails. Move NCCL installation to the extensions-build stage where it's actually needed for DeepEP compilation, with the correct apt flag. Co-authored-by: Claude <noreply@anthropic.com> Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>

The apt libnccl-dev package only ships host API headers (nccl.h), not the GIN device headers (ncclGinRequest_t etc.) that DeepEPv2 compiles against. Install nvidia-nccl-cu13 via pip instead, which includes the full header set and is auto-detected by DeepEP's build system via EP_NCCL_ROOT_DIR. Co-authored-by: Claude <noreply@anthropic.com> Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>

- Pin NCCL_VERSION to exactly 2.30.7 - Guard pip install (extensions-build) and apt install (vllm-base) behind CUDA major >= 13 checks - CUDA 12 builds fall back to whatever NCCL the base image ships Co-authored-by: Claude <noreply@anthropic.com> Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>

The previous approach failed because: 1. install_python_libraries.sh runs `uv pip install torch` which re-pins nvidia-nccl-cu13==2.28.9, overwriting our 2.30.7 2. apt libnccl-dev only has host API headers, not GIN device headers Fix both by: - Adding NCCL re-install to install_python_libraries.sh after torch, gated on NCCL_VERSION env var and CUDA >= 13 - Replacing apt NCCL install in vllm-base with pip install after all wheels are installed - Removing separate pre-build NCCL RUN from extensions-build Co-authored-by: Claude <noreply@anthropic.com> Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>

The test stage installs requirements/dev.txt which includes requirements/test/cuda.txt pinning nvidia-nccl-cu13==2.28.9, overwriting the 2.30.7 upgrade from vllm-base. Add another NCCL upgrade after all test dependencies are installed. Co-authored-by: Claude <noreply@anthropic.com> Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>

Replace the three separate `uv pip install nvidia-nccl-cu13==2.30.7 --no-deps` commands (in install script, vllm-base, and test stage) with a single UV_OVERRIDE env var that forces uv to resolve nvidia-nccl-cu13 to our pinned version whenever it appears as a transitive dependency. Co-authored-by: Claude <noreply@anthropic.com> Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>

Add instructions for pinning NCCL >= 2.30.4 when installing vLLM via uv/pip rather than the Docker image, covering both UV_OVERRIDE and pip --no-deps approaches. Co-authored-by: Claude <noreply@anthropic.com> Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>

mergify · 2026-06-26T15:29:34Z

Documentation preview: https://vllm--45321.org.readthedocs.build/en/45321/

tlrmchlsmth · 2026-06-26T21:37:40Z

+        ldflags = ["-lnccl"]
+        nccl_lib_paths = find_nccl_library_paths()
+        if nccl_lib_paths:
+            ldflags = [f"-L{p}" for p in nccl_lib_paths] + ldflags


The NCCL symm mem allocator JIT-compiles a C++ extension that links with -lnccl. Previously the apt libnccl-dev package placed libnccl.so in /usr/lib where the system linker could find it. With pip-only NCCL, the .so is in site-packages/nvidia/nccl/lib/ — invisible to ld.

This adds find_nccl_library_paths() to discover the pip package's lib dir and passes -L<path> to the linker, fixing the cannot find -lnccl error without needing apt.

— via claude

mergify · 2026-06-26T21:43:41Z

Hi @tlrmchlsmth, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

The NCCL symmetric memory allocator JIT-compiles a C++ extension with `-lnccl`, but the system linker can't find `libnccl.so` when NCCL is installed only via pip (in site-packages/nvidia/nccl/lib/). Add `find_nccl_library_paths()` to discover the pip package's lib directory and pass `-L<path>` to the linker. This was a pre-existing issue masked by the apt `libnccl-dev` install; removing apt NCCL in favor of pip-only exposed it. Co-authored-by: Claude <noreply@anthropic.com> Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>

pytest.skip() raises a BaseException that bypasses the except-Exception handler in _worker_parallel_launch, causing torch.multiprocessing.spawn workers to SIGSEGV instead of cleanly skipping. Raise a regular GINNotAvailableError inside the subprocess and convert it to pytest.skip() in parallel_launch on the parent side. Co-authored-by: Claude <noreply@anthropic.com> Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>

mergify · 2026-06-29T16:27:23Z

Hi @tlrmchlsmth, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Temporary diagnostic to determine why ginType == NONE on B200-k8s CI pods: checks NIC model, OFED version, nvidia-peermem, kernel version, RDMA device access, and NCCL env vars. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>

query_nccl_gin_type() returns None when it can't query (e.g. _comm_ptr missing, ncclCommQueryProperties unavailable). The previous check only guarded against ginType==0, allowing None to pass through and hit the ElasticBuffer segfault. Now None is treated as GIN-unavailable too. Also promote debug logs to warning so failures are visible in CI. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>

The nvidia-nccl-cu* pip package ships libnccl.so.2 (runtime) but not the libnccl.so dev symlink. The nccl_allocator JIT extension needs -lnccl at link time. The previous apt-based libnccl-dev install provided this symlink; now that we use the pip package instead, we create it explicitly. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>

Use -l:libnccl.so.2 instead of -lnccl so the JIT extension links against the SONAME directly. This removes the need for the libnccl.so dev symlink that the pip nvidia-nccl-cu* package doesn't ship, and that libnccl-dev (apt) would otherwise need to provide. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>

The lifecycle test now runs because NCCL >= 2.30.4 is present, but on pods without IBGDA-capable hardware it hits the GIN RuntimeError from _check_gin_support. Detect this in _spawn_workers and convert to pytest.skip. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>

- Remove debug logger.info for GIN type (was temporary instrumentation) - Remove NCCL GIN diagnostics echo from B200 CI step - Add comment explaining why GIN check happens post-spawn Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>

NCCL validates magic, version, and size on the ncclCommProperties struct (same pattern as NCCL_CONFIG_INITIALIZER). Without these, ncclCommQueryProperties rejects the struct. Co-authored-by: Ilya Markov <ilmarkov@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com> Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>

mergify · 2026-07-02T14:43:11Z

Hi @tlrmchlsmth, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

The NCCL communicator is lazily created on the first collective. When _check_gin_support runs during process_weights_after_loading, _comm_ptr() returns 0 (NULL) and the query fails, incorrectly blocking startup. Fix: return None from query_nccl_gin_type when comm_ptr is 0, and retry the check on subsequent get_handle calls once the comm exists after real collectives. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>

`pip install tensorizer` resolves torch's transitive dep nvidia-nccl-cu13==2.28.9, downgrading the 2.30.7 installed by the Docker image. This breaks deep_ep import (undefined symbol: ncclCommQueryProperties). Switch to `uv pip install` so the UV_OVERRIDE NCCL pin is respected. Also: remove unused _NCCL_GIN_TYPE_NONE constant, improve GIN check comment in test utils. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>

uv pip install fails on AMD CI where no virtual environment exists. Switch to pip install --no-deps to avoid both the venv requirement and the risk of torch pulling in an older NCCL as a transitive dependency. Co-authored-by: Claude <noreply@anthropic.com> Signed-off-by: Tyler Michael Smith <tyler@vllm.ai> Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>

mergify · 2026-07-06T18:16:19Z

Hi @tlrmchlsmth, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

tlrmchlsmth requested review from Harry-Chen and khluu as code owners June 11, 2026 21:20

mergify Bot added the ci/build label Jun 11, 2026

tlrmchlsmth requested review from LucasWilkinson and WoosukKwon June 11, 2026 21:22

tlrmchlsmth added the ready ONLY add when PR is ready to merge/full CI is needed label Jun 11, 2026

tlrmchlsmth marked this pull request as draft June 11, 2026 21:42

tlrmchlsmth force-pushed the fix/deepep-commit-hash branch 2 times, most recently from a8cb6bd to 49fbfce Compare June 12, 2026 18:30

tlrmchlsmth marked this pull request as ready for review June 12, 2026 19:03

tlrmchlsmth marked this pull request as draft June 16, 2026 19:49

WoosukKwon mentioned this pull request Jun 22, 2026

[DeepEP V2] Bound num_max_tokens_per_rank in do_expand=False #46404

Merged

tlrmchlsmth force-pushed the fix/deepep-commit-hash branch from 2de2b77 to 71604cb Compare June 25, 2026 20:23

tlrmchlsmth marked this pull request as ready for review June 25, 2026 20:26

claude Bot reviewed Jun 25, 2026

View reviewed changes

tlrmchlsmth and others added 7 commits June 25, 2026 16:40

mergify Bot added the documentation Improvements or additions to documentation label Jun 26, 2026

tlrmchlsmth commented Jun 26, 2026

View reviewed changes

tlrmchlsmth force-pushed the fix/deepep-commit-hash branch from 78c38b5 to 11feadc Compare June 26, 2026 21:45

tlrmchlsmth requested a review from AndreasKaratzas as a code owner June 27, 2026 03:30

tlrmchlsmth and others added 2 commits June 29, 2026 08:56

Merge branch 'main' into fix/deepep-commit-hash

1e9e837

tlrmchlsmth and others added 6 commits June 29, 2026 12:37

Fix ruff formatting for GINNotAvailableError raise

4bc4ecf

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>

Merge branch 'main' into fix/deepep-commit-hash

519c554

Merge branch 'main' into fix/deepep-commit-hash

cc6712f

Apply ruff formatting fixes

97dbca7

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>

tlrmchlsmth commented Jun 30, 2026

View reviewed changes

Comment thread vllm/utils/nccl.py Outdated

tlrmchlsmth and others added 3 commits June 30, 2026 11:57

tlrmchlsmth commented Jul 1, 2026

View reviewed changes

Comment thread .buildkite/test_areas/kernels.yaml Outdated

tlrmchlsmth commented Jul 1, 2026

View reviewed changes

Comment thread tests/kernels/moe/parallel_utils.py

tlrmchlsmth and others added 2 commits June 30, 2026 21:25

Merge branch 'main' into fix/deepep-commit-hash

f975515

ilmarkov reviewed Jul 1, 2026

View reviewed changes

Comment thread vllm/utils/nccl.py

tlrmchlsmth force-pushed the fix/deepep-commit-hash branch from c6e3c95 to 0f698a0 Compare July 2, 2026 15:15

tlrmchlsmth and others added 2 commits July 2, 2026 14:34

Merge branch 'main' into fix/deepep-commit-hash

b289f00

aoshen02 mentioned this pull request Jul 5, 2026

[Core] Release NCCL communicator memory in sleep mode #46234

Open

tlrmchlsmth and others added 3 commits July 6, 2026 09:41

Merge branch 'main' into fix/deepep-commit-hash

99e5ea6

Merge branch 'main' into fix/deepep-commit-hash

4f04dd6

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

[WideEP] Update DeepEP version in Dockerfile#45321

[WideEP] Update DeepEP version in Dockerfile#45321
tlrmchlsmth wants to merge 30 commits into
mainfrom
fix/deepep-commit-hash

tlrmchlsmth commented Jun 11, 2026 •

edited

Loading

Uh oh!

tlrmchlsmth commented Jun 11, 2026 •

edited

Loading

Uh oh!

claude Bot left a comment

Uh oh!

mergify Bot commented Jun 26, 2026

Uh oh!

tlrmchlsmth Jun 26, 2026

Uh oh!

mergify Bot commented Jun 26, 2026

Uh oh!

mergify Bot commented Jun 29, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mergify Bot commented Jul 2, 2026

Uh oh!

mergify Bot commented Jul 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Uh oh!

Conversation

tlrmchlsmth commented Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tlrmchlsmth commented Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

claude Bot left a comment

Choose a reason for hiding this comment

Claude Code Review

Uh oh!

mergify Bot commented Jun 26, 2026

Uh oh!

tlrmchlsmth Jun 26, 2026

Choose a reason for hiding this comment

Uh oh!

mergify Bot commented Jun 26, 2026

Uh oh!

mergify Bot commented Jun 29, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mergify Bot commented Jul 2, 2026

Uh oh!

mergify Bot commented Jul 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

tlrmchlsmth commented Jun 11, 2026 •

edited

Loading

tlrmchlsmth commented Jun 11, 2026 •

edited

Loading