fix(glm5.2): the remaining fixes for agentic serving — DSA indexer, ROCm hicache, PD/kvd wiring

[dorado269]

What this is

Everything still missing from main that GLM-5.2 needs to run
Optimus-AgenticBench Case A with kvaware + kvd + MTP + PD + DP-attention all
on at once. The fixes were established across several debugging runs on two
clusters; this branch is the formalised subset, cut fresh off main.

Much of that work has already reached main by other routes — PR #58 (the DSA
set, since evolved by 1380228), PR #56, and the individual fixes 786f238,
05841f9, b398268+c178b69, e190d65, 2c31a3f. None of it is
re-carried here. In particular main's NodePort guard and ready-timeout are
stronger than the versions that existed on the experiment branch, and its
args.py fp8 ordering is the correct one — all three were left alone.

Supersedes #59, which carried items 6–9 below against an older main plus
experiment packups. Close that one when this merges.

The commits, by kind

Engine correctness — the image crashes without these (2)

Both patch vendor SGLang inside the image at build time; neither touches infera
runtime code. Both are ROCm-only defects where the CUDA path is correct by
accident of a platform difference, which is why upstream CI cannot see them.

commit	fix	symptom without it
9929c94	GLM52_P1V3 — the DSA indexer row count diverges in both directions	Expected lengths.size(0) == B kills a decode scheduler rank, router drops to active_workers: 1. Reproducible ~13 min into Case A
b1d8103	ROCm hicache host allocator → hipHostMalloc	Memory access fault by GPU node-N on address <host VA> the moment kvd writes back; gfx950 is xnack-, so a hard abort

infera correctness — startup failure or silent degradation (3)

commit	fix	symptom without it
7e9cf0d	don't append the decode radix cache under speculative decoding	SGLang rejects the flag outright: the MTP decode leg dies during argument parsing, before loading a weight
30300ad	randomise the free-port-block scan start	two PD legs on one host deterministically pick the same base; the second dies with ZMQError: Address already in use
f0c328d	strip the bind-mount [subpath] from the findmnt source	lsblk says "not a block device", so kvd's L3 silently degrades to buffered on hardware that qualifies for O_DIRECT

Behavioural optimisation — not a bug fix (1)

4b14403 — skip kvd wiring on a PD decode leg. SGLang issues no storage
prefetch on the decode branch, so L3 there is write-only: measured at 180 sets /
0 gets against a prefill leg's 102/102 on the same run. Saves host memory and
D2H bandwidth.

Tests for fixes already on main (2)

Neither carries production code.

commit	covers
fc11a73	main's 05841f9 Python bigram decode — 175 lines of behavioural coverage
618bfec	main's Rust as_u32_any. main has the in-crate unit test but not the real-socket path, which is where a live deployment meets it — and the failure is silent (view stays empty, cache_hits pinned at 0, nothing errors)

Docs and patch bookkeeping (5)

157d535 kvd is no longer vLLM-only in the feature matrix, plus the SGLang
section · aabcb0c a 276-line operator guide for KV-aware + kvd · 7909361 a
README for deploy/docker/, which had none · 21a860b corrects a stale
portability claim about patch 01 · 28319cf gives the ROCm allocator patch the
upstream-status block and index row that deploy/docker/README.md requires of
every patch · e995e07 compresses over-long inline comments, scrubs cluster
identifiers, and records two upstream PRs that collide with patch 01's anchors
(see Review follow-ups).

Added then dropped (2)

4f9834d added Dockerfile.sglang.kvaware-kvd; 97c2ff5 removed it again
along with its two doc references. It was contents-identical to
Dockerfile.sglang, so it bought a second file to keep in sync and nothing
else. Net effect on the tree is zero — left as two commits rather than a
rewrite, since the branch is already under review.

Size

22 files, +1565/−69. Under infera/ the whole change is 4 files, +87 lines;
the two engine patches plus their README are +428 but live entirely in
deploy/docker/patches/. The remaining ~1000 lines are tests (514) and docs.
No experiment packups.

Verification

Patch 01 applies to the pinned base, and the build script proves it. Run
end-to-end against a clean v0.5.15.post1 dsa_indexer.py exported from a real
SGLang checkout (not --dry-run, not git apply --check):
apply_sglang_dsa_patches.sh with DSA_PATCH_SET=indexer applies the patch,
byte-compiles it, finds both markers in the bytecode, and exits 0. A second
run reports "already present" — idempotent.

The second marker discriminates. 21a860b's companion change (already in
9929c94) added _p1v2_rows beside _p1v2_trim because the latter alone could
not tell the one-directional revision from the fixed one. Confirmed by applying
main's pre-P1V3 patch 01 and running the same verification: _p1v2_trim
pyc=1 — it would have passed — while _p1v2_rows pyc=0 fails the build. The
extra marker is load-bearing, not decorative.

The ke_offset route is a real, precedented API. Read against the pinned
base: ke_offset is an existing parameter of DSAIndexerMetadata.topk_transform
whose sole effect is to override seq_lens_topk (dsa_backend.py:310-311), it
is already used that way at dsa_indexer.py:1217, and DSAIndexerMetadata is
the only concrete implementation — so no third-party subclass can hit a
TypeError on the added kwarg.

Both new infera fixes fail without the fix. Verified with a revert harness
rather than by inspection, since a test that passes on the pre-fix code proves
nothing: removing the decode-leg guard makes
test_decode_leg_gets_no_hicache_flags fail; removing the randomisation makes
10 consecutive free_tcp_port_block(4) calls return the identical base 29996
and fails test_repeated_calls_do_not_all_collide. Restored, 73 tests pass.

Upstream status re-queried 2026-08-03, not taken from the header: #33059,
#32762, #31683 OPEN; #30378, #30427 MERGED — all matching what the patch header
claims. For the ROCm allocator, upstream main's pool_host/common.py was read
directly: ALLOC_MEMORY_FUNCS still overrides only "npu" and "musa", no HIP
entry, so main is affected too.

The two ROCm engine patches were each measured on hardware before being
formalised here — the allocator by a standalone micro-repro of the exact kernel
that faults and un-faults on the allocator alone, and GLM52_P1V3 by two
independent Case A crashes followed by two clean ~4,000 s windows on two
different clusters.

Review follow-ups

Found reviewing this branch against the repo's own conventions. All
documentation-only, all fixed in-branch:

• 21a860b — the build script and the gfx942 recipe still described patch 01 as
  having "two edit sites, byte-identical on both releases". It has five, and only
  the GLM52_P1V2 ones were re-read against v0.5.16. The gfx942 image is the one
  arm that depends on that claim, so the two files a reader consults before
  bumping that base were the two still asserting the stronger version.
• 28319cf — deploy/docker/README.md, added in this branch, requires every
  patch to carry its upstream issue/PR and a row in patch.upstream.status.md.
  The sglang_rocm patch added in this same branch had neither.
• e995e07 — three things at once, since they touched the same text:
  (a) nine inline comment blocks ran over the 3-line limit, up to 29 lines.
  Compressed to what/why/how, with the case analysis and the rejected
  0 < q_offset variant moved to the patch header and README rather than
  deleted — that variant passes every functional test, so losing the note
  invites a retry. (b) two cluster names, three pointers to scripts that
  exist only in a private workspace, and an internal workload name were still
  in files headed for a public repo; replaced with this README's existing
  "internal reproduction kit, ask the patch author" convention, keeping the
  evidence-strength qualifiers. (c) searching for the function rather than
  the symptom surfaced two OPEN upstream PRs that rewrite what patch 01 anchors
  on — #32738 (pads heads at
  the same two aiter call sites) and
  #31480 (extracts the
  paged-MQA backend, restructuring the is_aiter() dispatch). Neither fixes
  this defect; both drift the anchors, which fails the build rather than
  mis-applying. This is precisely the gh search blind spot the header warns
  about, and the P1V3 half had never been searched on its own terms before —
  it now has been: still no upstream issue or PR.

What is NOT established yet

Patch 01's GLM52_P1V3 anchor on the gfx942 v0.5.16 base. Verified on
v0.5.15.post1 (above); on v0.5.16 it has been reasoned about — the added
_mask_init_and_local_tokens call sits before rather than inside the anchored
topk_transform call — but not measured, because there is no v0.5.16 checkout
here. A drift writes nothing and exits 1, so the failure mode is a failed build,
not a mis-applied patch. The patch header, the build script and the README all
now say this rather than claiming both bases are covered.

The Rust wire-level test (618bfec) has not been compiled. The dev
workstation's cargo is 1.75 and cannot read this repo's v4 Cargo.lock
(-Znext-lockfile-bump). It is purely additive and rust/router/src/ is
untouched, but neither its compilation nor its own pre-fix failure has been
demonstrated. CI is the gate.

test_decode_radix_vs_speculative.py did not execute here — it is
importorskip("sglang") and skips outside the engine image. Its third case
depends on SGLang's own ServerArgs.__post_init__ raising, which is external
behaviour.

Cluster acceptance — build, kvd/kvaware feature proofs, Case A — runs next; I
will post results on this PR.

Deliberately out of scope

Prefill --mem-fraction-static 0.88 → 0.80 (an experiment-environment value,
not a defect — it stays in the reproduction kits) and the kvd --long-bytes
512G → 64G value (a single node's disk artefact, verified inapplicable
elsewhere).

[kzjeef]

This really require a formal patch for upstream, it can be produce with SGLANG + HICache + DSA model .

This combo is leak of test case in upstream.

please submit a upstream fix for this bug.

[dorado269]

will do within one week