Summary

Exposes MoRI's blockwise-FP4 (E2M1) combine via SGLANG_MORI_COMBINE_DTYPE=fp4. This maps to
MoRI's fp4_blockwise combine quant type, which transports each combine element as packed FP4
(2 values/byte) instead of FP8 (1 byte/elem) — ~half the combine payload. Combine is a
transport-bound step in MoE decode, so this improves decode throughput/latency with no accuracy loss.

Requires the companion MoRI change (fp4_blockwise combine kernels). Falls back cleanly: any other
SGLANG_MORI_COMBINE_DTYPE value keeps the existing behavior.

What changed

• moriep.py: add CombineDtype.fp4 ("fp4_blockwise"), parse SGLANG_MORI_COMBINE_DTYPE=fp4,
  and map it to MoRI's fp4_blockwise combine quant type. Existing dtypes
  (bf16 / fp8 / fp8_direct_cast) are unchanged.
• test/manual/ep/test_moriep_combine_dtype.py: no-GPU unit test guarding the CombineDtype.fp4
  enum wiring and the existing dtypes.

Validation (MI355x / gfx950, DeepSeek-R1, TP8 EP8 DP-attention, MoRI-EP)

Accuracy — gsm8k, 2000 questions, --parallel 1200, SGLANG_MORI_COMBINE_DTYPE=fp4 ×3

Profile - With FP4

Command Used for Performance -

export SGLANG_USE_AITER=1 SGLANG_AITER_MOE=1 SGLANG_USE_ROCM700A=1
export SGLANG_ROCM_FUSED_DECODE_MLA=1 SGLANG_AITER_MLA_PERSIST=1
export SGLANG_DSR_OPROJ_MXFP4_ASM=1 TORCH_BLAS_PREFER_HIPBLASLT=1
export SGLANG_AITER_AR=1 ROCM_QUICK_REDUCE_QUANTIZATION=INT4 ROCM_QUICK_REDUCE_MAX_SIZE_BYTES_MB=512
export TRITON_ENABLE_PRELOAD_KERNELS=0 TORCH_NCCL_BLOCKING_WAIT=1
export PYTORCH_ALLOC_CONF=expandable_segments:True
export SGLANG_AITER_FUSED_MLA_CONCAT_CACHE=1 SGLANG_AITER_FUSED_KVB_SPLIT_CAT=1

--- MoRI-EP ---
export MORI_SHMEM_MODE=ISOLATION MORI_SHMEM_HEAP_SIZE=16G SGLANG_MORI_LEC_IN_GRAPH=0
export SGLANG_MORI_DISPATCH_DTYPE=auto
export SGLANG_MORI_COMBINE_DTYPE=fp4 # or fp8 for the baseline
export SGLANG_MORI_NUM_MAX_DISPATCH_TOKENS_PER_RANK=4096 SGLANG_MORI_NO_PAD_MASK=1

python -m sglang.launch_server
--model-path /scratch/Deepseek-R1-MXFP4
--tp-size 8 --ep-size 8 --dp-size 8 --enable-dp-attention --enable-dp-lm-head
--moe-a2a-backend mori --deepep-mode normal --moe-dense-tp-size 1
--enforce-shared-experts-fusion
--trust-remote-code --kv-cache-dtype fp8_e4m3
--disable-radix-cache --mem-fraction-static 0.90
--cuda-graph-max-bs 512 --max-running-requests 2048
--host 0.0.0.0 --port 30000`

python /sgl-workspace/sglang/python/sglang/bench_serving.py
--backend sglang --host 127.0.0.1 --port 30000
--dataset-name random
--random-input-len 1024 --random-output-len 2048 --random-range-ratio 1.0
--num-prompts 1536 --max-concurrency 512

Performance — fp8 vs fp4 combine (decode-heavy: random, input 1024 / output 2048, 1536 prompts,
max-concurrency 512, identical server config)

Test plan

• test/manual/ep/test_moriep_combine_dtype.py — passes.
• End-to-end accuracy + perf validated on DeepSeek-R1 with SGLANG_MORI_COMBINE_DTYPE=fp4
  (see above).
• black / ruff / isort clean.

CI States

Latest PR Test (Base): ❌ Run #29387324713
Latest PR Test (Extra): ❌ Run #29387324571

Mori PR reference - ROCm/mori#461