Skip to content

Support RDMA bandwidth calculation for heterogeneous GPU server configurations#643

Open
alpha-baby wants to merge 3 commits into
deepseek-ai:mainfrom
alpha-baby:fujh/enhance_get_rdma_gbs
Open

Support RDMA bandwidth calculation for heterogeneous GPU server configurations#643
alpha-baby wants to merge 3 commits into
deepseek-ai:mainfrom
alpha-baby:fujh/enhance_get_rdma_gbs

Conversation

@alpha-baby

@alpha-baby alpha-baby commented May 25, 2026

Copy link
Copy Markdown
Contributor

Problem Background

The existing implementation of get_rdma_gbs() returned only the bandwidth of a single RDMA NIC. This leads to inaccurate bandwidth calculations in the following heterogeneous server configurations:

  1. GPU-NIC Mismatch: Some server configurations have multiple GPUs sharing fewer RDMA NICs (e.g., 8 GPUs sharing 4 RDMA NICs instead of 1:1 ratio)
  2. NIC Bonding Configuration: Certain network architectures use 802.3ad LACP bonding to aggregate multiple physical NICs into a single logical interface thx @michaelchen1996 RP:Try to detect 802.3ad LACP bonding slaves and multiply by slave count #618

Key Behavioral Changes

Before:

return rate / 8  # Returns single NIC bandwidth only

After:

nic_count = _get_active_rdma_nic_count(output, nic_name)
# Detects bonding and multiplies by slave count...
gpu_count = _get_local_gpu_count()
return rate * nic_count / gpu_count / 8  # Returns per-GPU allocated bandwidth

Compatibility

  • For standard 1 GPU : 1 NIC configurations, the calculation result remains the same as before
  • For multi-GPU shared NIC scenarios, now correctly reflects the actual RDMA bandwidth available to each GPU
  • Function signature remains unchanged, fully backward compatible

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant