Skip to content

[megatron] fix: return 3-tuple under calculate_per_token_loss to fix MoE aux/z-loss grad blowup at CP>1#6836

Open
EricMarcus-ai wants to merge 4 commits into
verl-project:mainfrom
kaiko-ai:fix/megatron-moe-zloss-per-token-cp
Open

[megatron] fix: return 3-tuple under calculate_per_token_loss to fix MoE aux/z-loss grad blowup at CP>1#6836
EricMarcus-ai wants to merge 4 commits into
verl-project:mainfrom
kaiko-ai:fix/megatron-moe-zloss-per-token-cp

[megatron] fix: correct routed-token count for SFT path and cp_size s…

0154b98
Select commit
Loading
Failed to load commit list.
Sign in for the full log view
model_rmpad_fsdp2_unstable_ascend
Started 2026-07-09 21:27:35 ago
Starting job