Skip to content

Fix Stage 0 + Ulysses crash: make bwc_tensor_model_parallel_rank() resilient to MP API absence #7888

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
nathon-lee wants to merge 8 commits into
base: master
Choose a base branch
from
+5 −1
Open

Fix Stage 0 + Ulysses crash: make bwc_tensor_model_parallel_rank() resilient to MP API absence #7888

Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
001f77c
Initial plan
Copilot Feb 27, 2026
b90aee5
Revert "fix: update 1 file reformatted."
Copilot Feb 27, 2026
b6da9af
Merge pull request #5 from nathon-lee/copilot/git-revert-ff886701
nathon-lee Feb 27, 2026
bb7f64f
Merge branch 'deepspeedai:master' into master
nathon-lee Mar 6, 2026
cbc816c
Initial plan
Copilot Mar 6, 2026
5fcc9a7
Reapply "fix: update 1 file reformatted."
Copilot Mar 6, 2026
f7c5d75
Merge pull request #6 from nathon-lee/copilot/remove-commits-from-master
nathon-lee Mar 6, 2026
d11732f
Enhance tensor model parallel rank retrieval
nathon-lee Mar 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion deepspeed/utils/bwc.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,9 +37,13 @@ def bwc_tensor_model_parallel_rank(mpu=None):
elif hasattr(mpu, 'get_slice_parallel_rank'):
# Some DeepSpeed + pipeline parallelism versions
return mpu.get_slice_parallel_rank()
else:
elif hasattr(mpu, 'get_model_parallel_rank'):
# Deprecated Megatron and DeepSpeed convention
return mpu.get_model_parallel_rank()
else:
# mpu does not provide any known tensor/model-parallel rank API.
# Treat as "no tensor model parallelism".
return 0


def bwc_tensor_model_parallel_world_size(mpu=None):
Expand Down