docs(glm52): pegaflow offload → P/D design record; refresh serving-status

docs/index.md

@@ -56,7 +56,8 @@ Organized by domain (model line / subsystem / playbook / lesson) instead of by l
 
 | Path | TL;DR |
 | --- | --- |
-| `models/glm52/serving-status.md` | Living status: decode serving feature-complete for its scope; sampling surface frozen at `temperature/top_p/top_k/min_p/seed` (stop/min_tokens/logprobs/penalties/n>1 ruled out 2026-07-06). Ordered remaining work: scheduler.rs split → #548 Python-client retest → pegaflow integration → P/D → #590 DSpark×prefix-cache → perf backlog. #551 shelved as unreproducible. |
+| `models/glm52/pegaflow-offload-pd.md` | Design record: GLM5.2 × pegaflow host-tier KV offload (M1: two-arena registration MLA+index-K, CPU-tier leg in admission prefix-match, per-rank in-process engines) → cross-engine P/D vLLM-prefill→openinfer-decode (M2: #540 hash-compat pattern + byte-level layout parity gate). Pins design to vendored pegaflow-core v0.23.2 rev d46fd16, NOT the local v0.22.6 checkout. |
+| `models/glm52/serving-status.md` | Living status: decode serving feature-complete for its scope; sampling surface frozen at `temperature/top_p/top_k/min_p/seed` (stop/min_tokens/logprobs/penalties/n>1 ruled out 2026-07-06). Ordered remaining work: pegaflow M1 host-tier offload → M2 cross-engine P/D → #590 DSpark×prefix-cache → perf backlog. Done: scheduler split #594/#596, #548 closed. #551 shelved as unreproducible. |
 | `models/glm52/dp16-smr-coordinator.md` | Design sketch (no code): cross-node DP16 via a **replicated deterministic coordinator** (SMR/Calvin shape) instead of hub-and-spoke RPC — sequencer stamps a total order on admissions/cancels, per-step output tokens allgathered (≤8 u32/rank), every replica derives byte-identical step shapes locally. Known determinism violation to fix first: `token_tx.is_closed()` disconnect probes → sequenced cancel events. Fallback: framed-TCP hub-and-spoke (<1% overhead at 20–46 ms steps). Cheapest first step stands alone: (admission, outputs) replay journal for desync forensics. |
 | `models/glm52/dspark-mtp.md` | DSpark speculative decoding (community `RedHatAI/GLM-5.2-speculator.dspark`, not native MTP): qwen3-arch 5-layer draft at hidden 6144 + rank-256 Markov head; verify span rides the decode buckets (span-4 default). M1 span-steps, M2 draft lane, M3 greedy round loop (sharegpt c1 1.52×), M4 sampled verify — non-greedy speculation via prefix-match over sampled tokens (c1 code 2.38× at temp 1; full rejection sampling probe-measured out at ≤ +1.5%). |
 | `models/glm52/paged-kv-prefix-cache.md` | Static per-slot KV partitions → per-rank `BlockPool` of 64-token content-hashed pages (Kimi #239 pattern, zero kernel changes): full-lifetime admission reservation, coordinator-shipped `Glm52StepKv` page rows, prefix caching on by default (suffix-only prefill), DSpark × prefix-cache mutually exclusive, launch-ahead lease breaks at page boundaries. Merged as #588: all jz-38 gates green, warm prefix TTFT 14.12 s → 0.84 s (16.9×) byte-identical, step bench flat vs the D5 anchor. |

docs/models/glm52/pegaflow-offload-pd.md

@@ -0,0 +1,40 @@
+# GLM5.2 × pegaflow: host-tier KV offload → P/D disaggregation
+
+> **TL;DR:** Design record (no code yet). The pegaflow in-process bridge already exists and serves qwen3 (`openinfer-kv-offload` embeds `PegaEngine` directly); GLM5.2 integration is an extension, not greenfield. M1 = single-engine host-tier offload wired into the admission prefix-match; M2 = cross-engine P/D (vLLM prefill → openinfer decode) via the #540 hash-compat pattern. Design pins to pegaflow-core **v0.23.2 rev d46fd16** (the vendored dep), not the local v0.22.6 checkout.
+>
+> **Last touched:** 2026-07
+
+## Why pegaflow first
+
+The standing decision is prefill-by-vLLM (no dedicated GLM5.2 prefill path). P/D disaggregation means the decode engine ingests KV that someone else produced — and the ingestion mechanism (query a shared tier by block hash, lease, load into pool pages, commit as matched prefix) is byte-for-byte the same mechanism as host-tier offload restore. M1 builds and gates that mechanism against ourselves; M2 only changes who wrote the blocks.
+
+## What already exists (seam map, 2026-07-06)
+
+- `openinfer-kv-offload` is an **in-process** bridge: `OffloadEngine::new(config, &KvBuffer, stream)` builds a `PegaEngine` in the same process (`engine.rs:299`); no server, no metaserver unless `P2pConfig` (RDMA) is set. Minimal deployment = host pinned-memory tier only.
+- qwen3 drives the full loop today: save sealed blocks (`executor.rs:1504`), CPU-tier query (`:1665`), lease + load + `commit_loaded_blocks` (`:1724`). GLM5.2 stops at the GPU-only `match_and_add_prefix` (`scheduler/mod.rs` admission); the CPU-tier leg is the missing wiring, and `BlockPool` already exposes the split point (`probe_prefix` → `cpu_query_hashes` → `commit_loaded_blocks`, `openinfer-kv-cache/src/pool.rs:169-253`).
+- Keying: openinfer serializes its xxh3 `PositionalLineageHash` u128 as 16 big-endian bytes (`pool.rs:602`); pegaflow treats `BlockKey { namespace, hash }` as opaque bytes and never re-hashes.
+
+## M1 — single-engine host-tier offload (GLM5.2 ↔ GLM5.2)
+
+1. **Registration geometry**: GLM5.2 has *two* per-layer arenas sharing pool block ids — MLA fp8_ds_mla (64-token page × 656 B/token) and index-K (64 × (128+4) B, full-indexer layers only). pegaflow's `KVCacheRegistration` models one arena per "layer", so register **two pegaflow layers per model layer** (`glm52.L{n}.mla`, `glm52.L{n}.idxk`), both blocks-first single-segment with their own `block_stride_bytes` via `register_context_layer_batch_strided` (v0.23.2, `lib.rs:301`). Save/load must move both or neither — a page whose MLA restored but whose index-K didn't is silent corruption; make the save entry per (block, both-arenas) atomic at our bridge layer.
+2. **Per-rank engines**: DP8 = 8 independent pools; `OffloadEngine` is single-rank by construction, so one engine per rank with the pinned pool budget split /8. Non-expert KV is rank-local — no cross-rank coordination needed.
+3. **Hook**: admission's `match_and_add_prefix` grows the CPU leg exactly like qwen3's executor — GPU match first, then `cpu_query_hashes` against pegaflow, lease + async load into freshly-reserved pool pages, `commit_loaded_blocks`, and only then report `cached_tokens`. Loads must complete before the request's first step (admission is a step boundary; block on the `LoadHandle` like qwen3 does).
+4. **Save policy**: sealed (content-addressed) blocks flow to the host tier on release, same as qwen3's `save_sealed_blocks`. dspark × prefix-cache exclusivity carries over unchanged — drafter on ⇒ no prefix matching ⇒ offload restore off (#590 owns lifting that).
+5. **Gates** (jz-38): byte parity — evict-then-restore a 1200-token varied prompt and require the warm output byte-identical to the resident-prefix run; step-bench flat vs the D5/D10 anchors (save is off the step path, restore is admission-side); 17-way mixed load with restores zero-error; `--no-prefix-cache` disables the whole leg.
+
+## M2 — cross-engine P/D (vLLM prefill → openinfer decode)
+
+Blocked on M1. Two hard problems, both with prior art:
+
+- **Hash compat**: openinfer xxh3-lineage vs vLLM SHA-256 `block_hashes` never collide onto the same keys. PR #540 (qwen3) already built the pattern: compute vLLM-compatible hashes for the prompt at admission and query the vLLM-written namespace. Port that, plus the namespace derivation (vLLM side = SHA-256 over model/dtype/tp/heads/... — `connector/common.py:210`; ours must reproduce it byte-for-byte).
+- **Layout parity**: vLLM's GLM5.2 runs the same FlashMLA-sparse fp8_ds_mla + DeepGEMM indexer contracts, but byte-level identity of both arenas (656 B token layout, index-K packing, scale placement) must be *proven*, not assumed — gate: dump one block from each engine for the same prefix and byte-compare before any e2e. Also reconcile page granularity (our fixed 64-token page vs vLLM scheduler-block × dcp/pcp virtual blocks).
+
+## Pitfalls pinned during the survey
+
+- The local `/data/code/pegaworkspace/pegaflow` checkout is v0.22.6 and **lacks the `*_inproc` and `_strided` APIs** the integration uses; read the vendored `~/.cargo/git/checkouts/pegaflow-*/d46fd16/` instead.
+- The PyO3 package is a gRPC client — reference contract only, not our path.
+- vLLM connector requires `storage_offset()==0` and registers CUDA-IPC handles; our in-process path passes raw device pointers and skips IPC entirely.
+
+## Next action
+
+M1 step 1: extend `openinfer-kv-offload`'s `Registration` to model the two-arena-per-layer geometry (today `Registration::from_buffer` assumes the qwen3 fused single-buffer layout), behind the same `OffloadEngine` API so qwen3 is untouched.

docs/models/glm52/serving-status.md

@@ -1,6 +1,6 @@
 # GLM5.2 Serving Status & Remaining Work
 
-> **TL;DR:** Decode serving is feature-complete for its scope (whole-step graph buckets, DSpark speculation, paged KV + prefix cache, VRAM-derived max_model_len); sampling surface frozen at `temperature/top_p/top_k/min_p/seed`. Next up: split the 2.3k-line `scheduler.rs`.
+> **TL;DR:** Decode serving is feature-complete for its scope (whole-step graph buckets, DSpark speculation, paged KV + prefix cache, VRAM-derived max_model_len); sampling surface frozen at `temperature/top_p/top_k/min_p/seed`. Next up: pegaflow host-tier offload (M1 of `pegaflow-offload-pd.md`), then cross-engine P/D.
 >
 > **Last touched:** 2026-07
 
@@ -17,11 +17,12 @@ Known limitation: HTTP `seed` is stripped to `None` by the shared vLLM frontend
 
 ## Remaining work (ordered)
 
-1. **`scheduler.rs` split** — 2332 lines, past the 1k-line ceiling; debt flagged in the #588 review. Next up.
-2. **#548 bench-client hang** — re-run the Python `vllm bench serve` client against current main; the Rust `vllm-bench` client already completes 8/8 at c8. If Python passes too, close.
-3. **P/D disaggregation** — standing decision is prefill-by-vLLM, no dedicated prefill path. Prerequisite: integrate pegaflow first (in-process KV offload seam: numeric layer id + block hash); KV injection work starts after that.
-4. **#590 DSpark × prefix caching** — currently mutually exclusive (draft aux hidden + draft KV are not cacheable alongside 656 B/token MLA KV). Plan: draft cold-starts at the cache-hit boundary; first probe the suffix-only-context accept loss with the shadow-slot infra before committing.
-5. **Perf backlog** — accept parity with the vLLM production reference is reached, so the first TPOT lever is round cost: #582 draft-round graph (external PR #591 under review), #559 bucket-4/8 step premium, adaptive span (5% of rounds accept all 7 drafts), #542 collective wait structure, #569 PDL weight prefetch, cache-aware placement (admission picks a rank before the prefix match — worst-case hit rate ÷8 under concurrency).
+1. **pegaflow M1: host-tier KV offload** — design in `pegaflow-offload-pd.md`; the in-process bridge already serves qwen3, GLM5.2 adds two-arena registration (MLA + index-K) and the CPU-tier leg in admission. Next up.
+2. **pegaflow M2: cross-engine P/D** (vLLM prefill → openinfer decode) — blocked on M1; hash compat via the #540 pattern plus a byte-level MLA/index-K layout parity gate.
+3. **#590 DSpark × prefix caching** — currently mutually exclusive (draft aux hidden + draft KV are not cacheable alongside 656 B/token MLA KV). Plan: draft cold-starts at the cache-hit boundary; first probe the suffix-only-context accept loss with the shadow-slot infra before committing.
+4. **Perf backlog** — accept parity with the vLLM production reference is reached, so the first TPOT lever is round cost: #582 draft-round graph (external PR #591: −4.9% draft round re-measured on jz-38, Request-Changes for three capture bugs, waiting on the author), #559 bucket-4/8 step premium, adaptive span (5% of rounds accept all 7 drafts), #542 collective wait structure, #569 PDL weight prefetch, cache-aware placement (admission picks a rank before the prefix match — worst-case hit rate ÷8 under concurrency).
+
+Done since the 2026-07-06 ruling: `scheduler.rs` split (#594) and the coordinator phase decomposition (#596); #548 closed — the Python `vllm bench serve` c8 hang no longer reproduces on main (64/64, 216.9 tok/s, TPOT p50 30.8 / p99 34.2 ms).
 
 ## Shelved / background