feat(glm52): pegaflow host-tier KV offload — shared DP8 pool, warm TTFT ~34x

[xiaguan]

GLM5.2 gains the pegaflow host tier behind --kv-offload — M1 of the P/D-disaggregation plan (docs/models/glm52/pegaflow-offload-pd.md). Sealed KV blocks flow to one shared pinned pool on request release; a prompt whose prefix fell out of HBM restores from it at admission and reports as cached_tokens.

Design

• Shared host, 8 rank instances, one namespace. openinfer-kv-offload splits OffloadHost (one PegaEngine pool + runtime + P2P lifecycle) from the per-rank OffloadEngine; with_arenas_on() registers each rank as one more pegaflow instance over the same pool. MLA KV is byte-identical across DP ranks (no TP sharding in the latent, replicated non-expert weights), so any rank restores what any rank saved — the host tier has no per-rank placement problem.
• 99 arenas per rank, atomic per block. 78 MLA fp8_ds_mla arenas (64-token pages × 656 B) + 21 index-K sidecars (full-indexer layers, 64 × 132 B) register in one instance, all indexed by the same pool block ids — one save/load entry per block moves both caches together, so an MLA page can never restore without its index-K.
• Restore at admission (a step boundary; all ranks joined): probe → query → load into reserved pool pages → blocking wait → commit, with the probe held across the prefix match so committed blocks can't evict before the re-match. Save on release, fire-and-forget, block guards pin the pages through the D2H copy. Both legs degrade to a full prefill on failure — the host tier is a cache, never a correctness dependency.
• Guard rails: --kv-offload + --no-prefix-cache/DSpark rejected at launch (offload extends the prefix cache). New --kv-offload-hugepages backs the pool with 2 MiB hugepages (plumbed to qwen3 too, previously hardcoded off).
• Also repairs tests/cpu_roundtrip.rs, broken since feat(qwen3): P/D disaggregation with cross-instance KV via pegaflow metaserver P2P #522 (GPU-gated, never compiled in CI).

vLLM layout alignment

Verified at source level against vLLM cdab28319: the device-side layouts are already byte-identical (our kernels are ports of vLLM's) — 656 B/token field order, ÷448 scales, index-K block-split packing, page 64, layer-outermost arenas. The only cross-engine gap for M2 is block hashing (SHA-256-cbor vs xxh3 lineage), which is the #540 pattern's job. Evidence table in the design doc.

Measured (jz-38, 8×H200, --max-model-len 4096, host pool 32 GiB hugepages)

	TTFT (1466-token prompt, greedy, solo)
no offload, cold	5371 / 5064 ms (two runs)
offload on, cold (miss path)	5064 / 5239 ms — within single-sample noise of the above
offload on, warm host-tier restore	157.6 / 140.5 ms (~35×)

Second column re-measured after the review fixes (leases disabled under offload) — every gate re-passed on the fixed branch. The warm run restored all 22 cacheable prefix blocks from the host tier after 160 diverse ~2700-token prompts (c8) had evicted the GPU prefix (per-rank pool 521 pages, churn ≈ 840 pages/rank). HugePages_Free dropped 17408 → 1024 at launch: the pool really is hugepage-backed.

Gates (all pass, ALL-GATES-PASS)

• Evict-restore byte parity: prompt A's 64-token greedy completion is byte-identical cold → warm-restored → no-offload server (cross-config baseline file).
• Mixed load with restores: 16 evicted ~2700-token prompts re-run at c8 → 15 full-prefix restores (42-43 blocks each), 0 errors, 2.2 s wall (same prompts cold at c8: ~11.8 s each).
• Zero host-tier warns/errors in the server log; 160-prompt churn clean.
• cargo test -p openinfer-glm52 -p openinfer-kv-offload --lib green; clippy clean on touched crates.

🤖 Generated with Claude Code

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 59b5fec415

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Release leases when GLM host loads fail to submit

When engine.load returns Err, the query lease created above may still be pinning the host-tier blocks because the load did not get far enough to consume it; the Qwen prefetch path explicitly releases the lease on this same submit-failure path. Under transient CUDA/pegaflow validation or submission errors, repeated GLM restores will leave blocks unevictable until the lease TTL, shrinking the effective host pool and causing later restores to miss even though the request correctly falls back to prefill.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Reject GLM P2P flags instead of ignoring them

For GLM5.2, --kv-offload --kv-p2p-metaserver-addr ... is accepted by the shared CLI, but this branch constructs Glm52KvOffloadOptions with only the host pool settings and silently drops all kv_p2p_* values. In a P/D or cross-instance experiment the server will start as host-only offload while appearing to have P2P configured, so either reject those flags for GLM until M2 is wired or plumb them through explicitly.

Useful? React with 👍 / 👎.

[xiaguan]

Review findings addressed in 0858c8e + f1297c5:

Internal adversarial review (3 findings, all reproduced by hand before fixing):

1. Critical — restore H2D racing a leftover launch-ahead replay. A leased speculative replay keeps writing KV on the rank stream for ~a step after the coordinator joins its argmax D2H; the restore leg H2Ds into freshly-reallocated pool pages on pegaflow's own stream at the very next admission, unordered against it — a replay row landing after the restore would silently poison a content-addressed block for every later match. Fix: launch-ahead leases are disabled while offload is on (~0.7 ms/step, offload deployments only), so the joined D2H is provably the last thing on the rank stream at admission. Contract test added.
2. Major — release-save guards vs full-lifetime admission math. Save guards pin a released request's pages in the active pool until the D2H lands; admission could promise those pages to a new request and later fail_step the engine. Admission now subtracts each rank's in-flight pinned count (RankOffload), degrading to admit-later.
3. Major — teardown order. The coordinator now flushes and drops the offload engines before the workers drop the models, honoring the with_arenas lifetime contract for in-flight D2H copies.

Codex P2s: lease released on load-submit failure (mirrors the qwen3 path); --kv-p2p-* flags now rejected for GLM5.2 instead of silently dropped.

Re-running the jz-38 gate suite on the fixed branch; will post updated numbers.

[xiaguan]

Gate suite re-run on the fixed branch (jz-38, run 3): ALL PASS — byte parity holds cold → warm-restored → no-offload, warm TTFT 140.5 ms (cold 5239/5064 ms), 15 full-prefix restores under 16-way mixed load, zero errors, hugepage pool verified again. Numbers in the PR body updated.