Skip to content

refactor(recipes): aggregated / disaggregated naming, and the RDMA check on the published page - #76

Merged
jiejingzhangamd merged 4 commits into
mainfrom
refactor/recipe-combination-names
Aug 3, 2026
Merged

refactor(recipes): aggregated / disaggregated naming, and the RDMA check on the published page#76
jiejingzhangamd merged 4 commits into
mainfrom
refactor/recipe-combination-names

Conversation

@jiejingzhangamd

Copy link
Copy Markdown
Collaborator

Two changes. Both were pushed after #75 merged, so neither made it in.

1. mixed / pdaggregated / disaggregated

mixed never said what was mixed, and pd needed you to already know it abbreviates prefill/decode. aggregated / disaggregated is the vocabulary the field uses — what NVIDIA Dynamo's recipe pages use, and what someone searching for this feature will type.

Renamed across all three models, so the combination set stays comparable between them — which is the whole reason it is a fixed set:

mixed        -> aggregated          pd         -> disaggregated
mixed-kvd    -> aggregated-kvd      pd-kvd     -> disaggregated-kvd
mixed-dspark -> aggregated-dspark   pd-dspark  -> disaggregated-dspark
                                    pd-sglang  -> disaggregated-sglang

13 directories, plus every reference in prose, link text, link targets, tab labels and tab sync ids. Renaming only one model would have broken cross-model comparability; renaming the directories and leaving the prose would have left text and links disagreeing.

The API values are deliberately unchanged

role: inside each manifest is consumed by the operator and is still mixed / prefill / decode, as is DisaggMode.MIXED in the Python. Renaming those would break every deployed configuration and anything outside this repo using the CRD — a separate decision, on its own terms.

That leaves aggregated/deploy.yaml legitimately containing role: mixed, which reads like a mistake if you meet it cold. Both the repo README and the manual index now say so explicitly: the directory name is what the combination means, role: is the wire format. A reader who spots the discrepancy finds it already accounted for rather than concluding the docs are stale.

Verified: all 13 manifests still pass kubectl apply --dry-run=server; every deploy.yaml path referenced from any document exists; the manual builds clean with warnings-as-errors and all four tabs on each of the three model pages render with the new labels and sync ids.

2. The RDMA device check is now on the published page

The page told readers that ibv_devices is absent and that ibv_get_device_list is the right question — then sent them to the repo README for the command. Someone reading the docs site to find out whether their NICs are visible could not get an answer from the docs site.

The Disaggregated tab now carries both forms: a standalone probe pod for before you deploy, and the in-pod version for after. Both state why it is worth running: when the container cannot see the RDMA devices, nothing says so. Mooncake falls back to a transport that cannot cross nodes, and the symptom is a request that hangs while every pod stays Ready.

It also says which pod to run it in. The router runs the same image but is not privileged and does not mount /dev/infiniband, so it reports 0 — reproducing the exact false negative this check exists to prevent. That was observed twice during validation.

There is no pre-flight script to point at: the one added earlier was withdrawn in #74 for giving wrong verdicts in both directions, so the fabric check is documented as a command rather than a tool.

Also repaired a sentence in the validation section that an earlier edit had left spliced together mid-clause.

`mixed` never said what was mixed, and `pd` needed you to already know it
abbreviates prefill/decode. `aggregated` / `disaggregated` is the
vocabulary the field uses -- it is what NVIDIA Dynamo's recipe pages use,
and what someone searching for this feature will type.

Renamed across all three models so the combination set stays comparable
between them, which is the whole reason it is a fixed set:

  mixed        -> aggregated          pd         -> disaggregated
  mixed-kvd    -> aggregated-kvd      pd-kvd     -> disaggregated-kvd
  mixed-dspark -> aggregated-dspark   pd-dspark  -> disaggregated-dspark
                                      pd-sglang  -> disaggregated-sglang

13 directories, plus every reference in prose, link text, link targets,
tab labels and tab sync ids. Renaming only one model would have broken
cross-model comparability; renaming only the directories and leaving the
prose would have left text and links disagreeing.

THE API VALUES ARE DELIBERATELY UNCHANGED. `role:` inside each manifest
is consumed by the operator and is still `mixed` / `prefill` / `decode`,
as is DisaggMode.MIXED in the Python. Renaming those would break every
deployed configuration and anything outside this repo using the CRD, and
it is a separate decision for the team to make on its own terms.

That leaves `aggregated/deploy.yaml` legitimately containing `role:
mixed`, which reads like a mistake if you meet it cold -- so both the
repo README and the manual index now say so explicitly: the directory
name is what the combination means, `role:` is the wire format. A reader
who spots the discrepancy finds it already accounted for rather than
concluding the docs are stale.

Verified: all 13 manifests still pass `kubectl apply --dry-run=server`;
every deploy.yaml path referenced from any document exists; the manual
builds clean with warnings-as-errors and all four tabs on each of the
three model pages render with the new labels and sync ids.

Signed-off-by: Zhang, Jiejing <jiejing.zhang@amd.com>
The published page told the reader that `ibv_devices` is absent and that
`ibv_get_device_list` is the right question, then sent them to the repo
README for the actual command. Someone reading the docs site to find out
whether their NICs are visible could not get an answer from the docs
site.

The Disaggregated tab now carries both forms: a standalone probe pod for
before you deploy, and the in-pod version for after. Both state the thing
that makes this check worth running at all -- when the container cannot
see the RDMA devices, nothing says so. Mooncake falls back to a transport
that cannot cross nodes, and the symptom is a request that hangs while
every pod stays Ready.

It also says which pod to run it in. The router runs the same image but
is not privileged and does not mount /dev/infiniband, so it reports 0 --
reproducing the exact false negative this check exists to prevent. That
was observed twice during validation.

There is no pre-flight script to point at: the one added earlier was
withdrawn for giving wrong verdicts in both directions, so the fabric
check is documented as a command rather than a tool.

Also repaired a sentence in the validation section that an earlier edit
of mine had left spliced together mid-clause.

Signed-off-by: Zhang, Jiejing <jiejing.zhang@amd.com>
I was wrong, and the correction matters more than the edit.

When I added a pre-flight script to this recipe two revisions ago, I
justified it with "There was no pre-flight anywhere in this repo" -- in a
commit message that is now merged. I had grepped two directories,
examples/recipes/ and manual/recipes/, and reported the result as if I
had searched the repository.

The repository has two layers of preflight, both better than what I
wrote:

  infera/tools/preflight/          a cluster diagnostic covering RDMA
                                   device and link state, cross-node RoCE
                                   bandwidth, Mooncake KV-transfer
                                   bandwidth measured separately over
                                   rdma and tcp, GPU compute/HBM/P2P,
                                   firmware and cross-node consistency,
                                   and storage -- including "KV not on
                                   local NVMe -> FAIL", which is the
                                   local-versus-network check my script
                                   got wrong in both directions before it
                                   was withdrawn. Renders one HTML report,
                                   with a SLURM path for multi-node.

  infera/common/disagg_preflight.py  config validation at launch, before
                                   the engine subprocess starts, for the
                                   two silent-failure modes of cross-node
                                   PD: advertising a non-routable host to
                                   etcd, and configurations prone to
                                   silent TCP fallback.

Both are already referenced from other recipes in this repo
(examples/deepseek_v4, examples/sglang_1p2d_kimi2.6, bench/pd_mori_1p1d).
This recipe was the one that did not know about them, and I made that
worse by building a replacement instead of looking.

Both documents now send the reader to the real tooling, calling out the
Mooncake rdma-versus-tcp rows as the ones that matter here -- a fabric
that will silently serve at TCP speed shows up as a number rather than as
a deployment that is inexplicably slow. The manual keeps the
ibv_get_device_list one-liner for the narrower "can the container see the
devices at all" question, with the warning about which pod to run it in.

Signed-off-by: Zhang, Jiejing <jiejing.zhang@amd.com>
Asking where the docs explain how to use preflight turned up the real
answer: they do not. The tool is referenced from six places across the
examples and troubleshooting pages, each of which links out to a file on
GitHub -- so a reader on the docs site has to leave it to learn how to
run the thing those pages tell them to run first. It was not in the toc.

The invocation was inconsistent too, depending on which page you landed
on: `infera-preflight --firmware` in troubleshooting, `python -m
infera.tools.preflight --dump-path <dir> --netperf --mooncake` in the
DSv4 example. Both work -- `infera-preflight` is a console script in
pyproject.toml -- but nothing said so.

The new page covers what it checks, how to run it single-node and under
SLURM, what the report contains, and the three things that are easy to
get wrong:

  * run it INSIDE the engine container for the full set. torch,
    ais-check, Mooncake and Mori exist only in the image, and without the
    host's libionic.so mounted, in-container ib_write_bw / Mooncake /
    Mori cannot see the ionic RDMA devices -- which looks identical to
    the node not having any.
  * use a fresh --dump-path. Rank 0 decides everyone has reported by
    counting *.json in the directory, so a stale file makes it render
    early.
  * read the Mooncake rows for PD. They report KV-move bandwidth over
    RDMA and over TCP separately, which is what turns "silently serving
    at TCP speed" from a mystery into a number.

It also leads with why the tool exists at all: nearly everything it
checks fails silently. RDMA the container cannot see does not raise --
Mooncake falls back to a transport that cannot cross nodes and the
request simply never returns. A KV path on network storage does not
raise either; the weight load just takes an order of magnitude longer,
exceeds the ready timeout, and presents as a crash loop.

The separate launch-time config validation in
infera/common/disagg_preflight.py is documented alongside it, with the
distinction stated: it is config-only and cannot tell you the hardware is
healthy.

References in the kimi-k3-optimized recipe and the DSv4 example now point
at this page instead of at GitHub.

Signed-off-by: Zhang, Jiejing <jiejing.zhang@amd.com>
@jiejingzhangamd
jiejingzhangamd merged commit 7326ae5 into main Aug 3, 2026
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant