[Question]: Do we have any plan to integrate the nccl ep to megatron?

[gangxie112]

Question

Hello,

Is there any plan or internal discussion about inggetrate nccl ep to megatron? I see a draft to add deepep elastic buffer to megatron NVIDIA/Megatron-LM#4632. So what about the nccl ep?

[xiaofanl-nvidia]

hey @gangxie112 yes, we plan to propose it.

Currently we would like to focus on making the foundation of nccl-ep solid. Specifically:

• On par with SOTA solutions in terms of functionality and performance
• Achieve stable API surface so that nccl-ep can provide the same backward compat guarantees just like NCCL
• Test on various NVIDIA platforms and various scales

We'd like to encourage interested users to reach out to us and 1) tell us your use cases and 2) share new feature ideas.
We are open to RFEs and PRs so that we can learn how to meet your needs before we finalize API surface.

[gangxie112]

hey @gangxie112 yes, we plan to propose it.

Currently we would like to focus on making the foundation of nccl-ep solid. Specifically:

* On par with SOTA solutions in terms of functionality and performance

* Achieve stable API surface so that nccl-ep can provide the same backward compat guarantees just like NCCL

* Test on various NVIDIA platforms and various scales

We'd like to encourage interested users to reach out to us and 1) tell us your use cases and 2) share new feature ideas. We are open to RFEs and PRs so that we can learn how to meet your needs before we finalize API surface.

@xiaofanl-nvidia Thanks for the detailed explaination. Based on your roadmap, roughly how long will it take for NCCL EP to mature and be integrated into Megatron? Just a very rough timeline will be appreciated.