[WIP] Sglang Rollout Refactor

[xiuhu17]

Refactor SGLang Rollout with the New Design

Overview

This PR refactors the SGLang rollout stack to support the new rollout architecture while preserving backward compatibility with the previous interface.

The new design focuses on improving scalability, reliability, and maintainability of the rollout system. In particular, it introduces support for:

• Redesign the old sglang rollout
• SGLang router
• Multi-node SGLang deployment
• Weight / KV cache / CUDA graph onload and offload
• Weight check: snapshot, reset, compare for weight update check.
• async def generate_async(...) for asynchronous multi-turn rollout
• Refactored colocated weight update
• Fault tolerance for dead rollout server restart

The goal is to make the rollout layer production-ready for larger-scale serving and training integration, while minimizing disruption to existing callers.

---

Tests:

Thorough tests(around 75 tests):

• https://docs.google.com/document/d/1hP4zzJQ5ZaMhEnmyoWNLOx3T2E8Yj20DCmTvbn-XTWw/edit?tab=t.0

---

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.