docs(glm52): pegaflow offload → P/D design record; refresh serving-status#597
Conversation
…tatus Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ca04ae33ee
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| 2. **Per-rank engines**: DP8 = 8 independent pools; `OffloadEngine` is single-rank by construction, so one engine per rank with the pinned pool budget split /8. Non-expert KV is rank-local — no cross-rank coordination needed. | ||
| 3. **Hook**: admission's `match_and_add_prefix` grows the CPU leg exactly like qwen3's executor — GPU match first, then `cpu_query_hashes` against pegaflow, lease + async load into freshly-reserved pool pages, `commit_loaded_blocks`, and only then report `cached_tokens`. Loads must complete before the request's first step (admission is a step boundary; block on the `LoadHandle` like qwen3 does). | ||
| 4. **Save policy**: sealed (content-addressed) blocks flow to the host tier on release, same as qwen3's `save_sealed_blocks`. dspark × prefix-cache exclusivity carries over unchanged — drafter on ⇒ no prefix matching ⇒ offload restore off (#590 owns lifting that). | ||
| 5. **Gates** (jz-38): byte parity — evict-then-restore a 1200-token varied prompt and require the warm output byte-identical to the resident-prefix run; step-bench flat vs the D5/D10 anchors (save is off the step path, restore is admission-side); 17-way mixed load with restores zero-error; `--no-prefix-cache` disables the whole leg. |
There was a problem hiding this comment.
When M1 is exposed through the existing server flags, this gate specifies the opposite of the documented behavior: openinfer-server/src/config.rs and Qwen3Executor::set_no_prefix_cache define --kv-offload --no-prefix-cache as pure-L2 mode (disable HBM retention but still restore from the host tier via prefix matching), not as disabling the offload leg. If GLM5.2 follows this design, the test would skip the restore path instead of forcing it, leaving the host-tier integration unvalidated and diverging from user-visible CLI semantics.
Useful? React with 👍 / 👎.
1 participant
Adds
docs/models/glm52/pegaflow-offload-pd.md— the design record for the next campaign:openinfer-kv-offloadin-process bridge already serves qwen3; GLM5.2 adds two-arena-per-layer registration (MLA fp8_ds_mla + index-K, shared block ids, atomic save/load) and the CPU-tier query/lease/load/commit leg in the admission prefix-match. One engine per rank (DP8).*_inproc/_stridedAPIs).Also refreshes
serving-status.md(scheduler split #594/#596 done, #548 closed, #591 review state) and the index rows.🤖 Generated with Claude Code