[NV] Add SpeedBench AL collectors for DSV4-Pro-0813 on B300 - #2526
Conversation
Adds the acceptance-length collectors for deepseek-ai/DeepSeek-V4-Pro-DSpark, following the Kimi-K3 DSpark collector (#2352). DSpark ships the draft inside a separate checkpoint, so unlike K3 there is no external draft head to download and no "model" key in the speculative-config. dsv4dspark_fp4_b300_vllm.sh is a copy of the DSV4 MTP collector with only the speculative-config changed (method dspark + draft_sample_method); every serve flag is left identical so the DSpark and MTP curves stay comparable, and it already matches the published DSpark recipe. Benjamin asked for greedy and probabilistic drafting to be characterized on DSV4-Pro. speedbench-al.yml resolves the collector purely as ${model-prefix}_fp4_b300_vllm.sh, so the second arm needs its own dispatchable file; dsv4dsparkprob_fp4_b300_vllm.sh delegates to the base collector with DRAFT_SAMPLE_METHOD=probabilistic rather than duplicating it, so the two arms cannot drift. rejection_sample_method is left at the vLLM default and exposed as an opt-in env knob: the K3 variant bundled it with probabilistic drafting, but the forced-AL config has to stay on a sampling method TRT-LLM also supports. The golden YAML and its README row follow in a separate PR once the collection run is reviewed.
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
…the pass The first collection run failed the preflight in 50s: DeepSeek-V4-Pro-DSpark is not staged at /scratch/models on the B300 cluster, so listing it in the launcher's STAGED_MODELS resolved MODEL_PATH to a read-only mount that does not hold the weights. Drop it from the list so MODEL_PATH resolves to the writable models dir and the collector's existing download block pulls the ~960 GB checkpoint once, on the first run. Once the weights are staged, adding the basename back is a one-line change that switches to the faster read-only mount. Also raise CONCURRENCY from 1 to 32. AL is a per-draft accept/reject property and is independent of batch size, and nothing here sets speculative_disable_by_batch_size, so drafting stays on and the curve stays comparable to the MTP one while the pass takes far less wall-clock — which matters more now that the first run also pays for the download.
The collection run aborted in the shim: on v0.25.1 serve.py already declares --chat-template-kwargs natively (vllm-project/vllm#44244 has landed), so the marker check short-circuited there, but datasets.py no longer matches the pre-#44244 anchor and the assertion killed the run before the first server start. Replace the shim with the Kimi-K3 collector's version, which probes all three pieces of native support (CLI option, speed_bench forward, CustomDataset unpack) and no-ops when they are present, patching only what an older image is missing. Verified against the current vLLM benchmark source: all three probes match, so the shim reports native support and returns without touching anything. Note dsv4_fp4_b300_vllm.sh still carries the old unconditional shim and would hit the same assertion if the MTP curve were ever re-collected on a v0.25.x image.
vllm-project/vllm#44244 is in the images this collector runs on, so the site-packages monkey-patching the older collectors carry is dead code. Drop it. Keep a preflight assertion rather than assuming, because one direction fails silently: if the CLI option exists but the speed_bench path does not forward it, the flag is accepted and ignored, every thinking_on prompt renders without thinking, and the cell writes a non-thinking AL under the thinking_on key. A missing CLI option would instead fail loudly at argument parsing. The assertion turns the silent case into an abort before the first server start. Verified against the current vLLM benchmark source: all three pieces are present, so the preflight passes and the run proceeds.
…d cell The DSpark=4 cell OOMed during warmup: the rejection sampler copies logits to fp32 at max_num_seqs * (1 + num_speculative_tokens) * vocab, which the default max_num_seqs of 1024 makes 2.47 GiB at 4 speculative tokens and 4.4 GiB at 8. The memory profiler does not account for it, so the KV cache is already sized before the allocation fails. The collection only drives 32 concurrent requests, so pin max_num_seqs to 64 and expose gpu-memory-utilization to trade KV cache for headroom if a higher level still runs out. The run also aborted at that cell instead of recording N/A and continuing: wait_for_server_ready exits rather than returns, so call it in a subshell.
…mega_moe The collector inherited EP_SIZE / DP_ATTENTION knobs from the MTP collector, but speedbench-al.yml exports EP_SIZE=1 and DP_ATTENTION=false for the whole matrix, so both recipe flags were silently dropped and the run was plain TP on the auto MoE backend. TP-sharding the FP4 experts cost 141.53 GiB per GPU against an 831 GiB checkpoint (1132 GiB across 8 ranks), roughly 37 GiB per GPU of sharding overhead on weights expert parallel keeps whole, which is most of why there was nothing left for warmup after the KV cache. Hard-code TEP8 instead of honoring knobs the workflow overrides. TP stays 8 to match the MTP collector; AL does not depend on expert placement either way.
functionstackx
left a comment
functionstackx
left a comment
There was a problem hiding this comment.
|
Should we collect the native AL distribution on AgentX as a validation point and compare it against the SPEED-Bench-derived AL? We are benchmarking on the AgentX dataset anyway. |
|
I think @functionstackx's comment that we should move this to the 0813 checkpoint is valid. DeepSeek describes 0813 as the official release superseding the preview, built on the same model structure and with DSpark attached. Considering InferenceX’s policy of tracking the latest model within the same model/framework family, 0813 should become the benchmark target if framework support remains unchanged. When we shift to DeepSeek-V4-Pro-0813, we will need to update the AL distribution. Should we do it now？ |
0813 is now staged on the B300 shared FS (AIAOPS-853), so add its basename to STAGED_MODELS. MODEL_PATH now resolves to /scratch/models instead of triggering the one-off download to the writable models dir on the first collection.
qiching
changed the title
|
@functionstackx Now could we merge? |
|
Claude finished @qiching's task in 4m 51s —— View job Review
One 🟡 WARNING posted inline: the collector's operator-facing docs (usage block, header, Everything else checks out: I diffed |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| # Usage (inside the vLLM container, on a B300 node): | ||
| # export MODEL=deepseek-ai/DeepSeek-V4-Pro-DSpark | ||
| # bash benchmarks/single_node/speedbench/dsv4dspark_fp4_b300_vllm.sh |
There was a problem hiding this comment.
🟡 WARNING: The collector's operator guidance still targets the superseded DeepSeek-V4-Pro-DSpark preview checkpoint, but this PR (commit 6162a13) moved the run to the staged DeepSeek-V4-Pro-0813.
Why it matters: MODEL is a manual workflow_dispatch input to speedbench-al.yml, and this header is the runbook the dispatcher copies from. Following it as written dispatches deepseek-ai/DeepSeek-V4-Pro-DSpark, whose basename is not in the launcher's STAGED_MODELS — so MODEL_PATH resolves to the writable dir and the ~960 GB preview download fires, collecting the AL curve on the checkpoint the review thread explicitly moved away from.
Fix:
| # Usage (inside the vLLM container, on a B300 node): | |
| # export MODEL=deepseek-ai/DeepSeek-V4-Pro-DSpark | |
| # bash benchmarks/single_node/speedbench/dsv4dspark_fp4_b300_vllm.sh | |
| # Usage (inside the vLLM container, on a B300 node): | |
| # export MODEL=deepseek-ai/DeepSeek-V4-Pro-0813 | |
| # bash benchmarks/single_node/speedbench/dsv4dspark_fp4_b300_vllm.sh |
Same staleness in the other operator-facing spots: the header description (lines 12–15, 19 — 0813 ships the DSpark draft natively, not as a separate 960 GB checkpoint), the MODEL:? example on line 48, and the download-block comment on lines 117–121, which now contradicts this PR's own launcher change (it says the checkpoint "is NOT in the launcher's STAGED_MODELS" while runners/launch_b300-nv.sh adds DeepSeek-V4-Pro-0813 in the same PR). The code itself handles the staged path correctly — only the guidance is stale.
| if [[ -n "${MODEL_PATH:-}" ]]; then | ||
| if [[ ! -d "$MODEL_PATH" || -z "$(ls -A "$MODEL_PATH" 2>/dev/null)" ]]; then | ||
| if [[ ! -w "$(dirname "$MODEL_PATH")" ]]; then | ||
| echo "CRITICAL: $MODEL_PATH is empty and $(dirname "$MODEL_PATH") is not writable." | ||
| echo "This means the basename is listed in the launcher's STAGED_MODELS but the" | ||
| echo "weights were never staged. Either get them staged, or remove it from" | ||
| echo "STAGED_MODELS so MODEL_PATH resolves to the writable models dir instead." | ||
| exit 1 | ||
| fi | ||
| echo "=== $MODEL_PATH is empty; downloading $MODEL (~960 GB, first run only) ===" | ||
| hf download "$MODEL" --local-dir "$MODEL_PATH" | ||
| fi | ||
| else | ||
| if [[ "$SERVE_MODEL" != /* ]]; then hf download "$SERVE_MODEL"; fi | ||
| fi |
There was a problem hiding this comment.
🟡 The staging guard in dsv4dspark_fp4_b300_vllm.sh (lines 122-133) only runs hf download when MODEL_PATH is missing or empty (ls -A), but hf download writes shards incrementally into --local-dir. If a ~960GB download is interrupted partway (SLURM timeout/preemption, node failure, network blip over a multi-hour transfer), the directory is left non-empty but incomplete, and every subsequent run's guard sees non-empty and silently skips re-downloading, launching vllm serve against a partial checkpoint. Since hf download is itself resumable/idempotent, prefer always invoking it (it cheaply verifies and skips complete shards) or gating on a completion sentinel written only after it exits 0, rather than gating on mere non-emptiness. This is a real robustness gap in newly-added code, but it only affects the unstaged /data download path — the primary intended flow serves the pre-staged 0813 checkpoint from the read-only /scratch mount where this branch never runs — and failures surface loudly (vllm errors on missing/truncated safetensors, MEASURED=0 exits the job) rather than silently corrupting results, so it is not merge-blocking.
Extended reasoning...
The bug: In dsv4dspark_fp4_b300_vllm.sh lines 122-133, the block that decides whether to fetch the DSpark checkpoint is:
if [[ ! -d "$MODEL_PATH" || -z "$(ls -A "$MODEL_PATH" 2>/dev/null)" ]]; then
...
hf download "$MODEL" --local-dir "$MODEL_PATH"
fiThis treats any non-empty directory as "fully staged" and skips the download entirely otherwise. hf download --local-dir writes each completed file directly into the target directory as the transfer progresses (and even leaves a .cache/huggingface/download/*.incomplete subdir for in-flight files), so a directory can be non-empty long before the ~960GB transfer actually finishes.
Trigger path: This only fires when the model basename is absent from the launcher's STAGED_MODELS list in runners/launch_b300-nv.sh, so MODEL_PATH resolves under the writable /data/models mount instead of the read-only /scratch mount. A multi-hour, ~960GB transfer over a shared SLURM cluster is realistically subject to job time-limit kills, preemption, node failure, or transient network issues. Any of those leaves /data/models/DeepSeek-V4-Pro-DSpark non-empty but incomplete. Every subsequent invocation of this script re-evaluates the guard, finds the directory non-empty, and skips straight to vllm serve "$SERVE_MODEL" ... against the partial checkpoint — the download is never resumed or retried.
Why nothing else catches it: hf download itself is resumable and idempotent — re-invoking it against a partially-populated --local-dir would verify existing files and fetch only what's missing or truncated. But the guard prevents it from ever being re-invoked once the directory is non-empty, defeating that built-in resume behavior entirely. There's no manifest check, file-count comparison, or completion sentinel — just ls -A.
Step-by-step proof:
- Job 1 starts,
MODEL_PATH=/data/models/DeepSeek-V4-Pro-DSparkis empty, guard triggershf download. - Transfer proceeds for several hours; 700GB of the 960GB have landed on disk when the SLURM job hits its time limit and is killed.
/data/models/DeepSeek-V4-Pro-DSparknow contains partial shards — non-empty.- Job 2 (retry/resubmit) starts. The guard runs
ls -A "$MODEL_PATH", gets non-empty output, skips theifblock entirely. vllm serve $SERVE_MODEL ...launches directly against the incomplete checkpoint directory — no re-download, no resume, no warning.
Impact and fix: vLLM will fail to load the missing/truncated safetensors shards, so in practice this fails loudly rather than silently serving wrong results — every cell in that run records N/A, and the MEASURED -eq 0 check at the bottom of the script exits the job with an error. It won't corrupt an AL curve, but it does prevent the download from ever completing on its own; someone has to manually notice, rm -rf the partial directory, and rerun. The fix is straightforward: either always invoke hf download (it's a cheap no-op when already complete) or gate the skip on a sentinel file written only after hf download exits 0.
Why nit, not normal: the intended production flow for this PR is now the pre-staged DeepSeek-V4-Pro-0813 checkpoint read from the read-only /scratch mount (per the PR's final comments and the STAGED_MODELS addition in runners/launch_b300-nv.sh), where this download branch never executes at all — non-emptiness there correctly means "staged and complete" since nothing ever writes into a read-only mount. This branch only matters for a one-time, unstaged /data download, requires a specific mid-transfer interruption to trigger, and fails the job loudly rather than emitting bad data. It's a genuine robustness gap worth hardening but not a merge blocker.
| DeepSeek-R1-0528-NVFP4-v2 | ||
| DeepSeek-V4-Flash | ||
| DeepSeek-V4-Pro | ||
| DeepSeek-V4-Pro-0813 | ||
| GLM-5-FP8 | ||
| GLM-5-NVFP4 | ||
| GLM-5.1 |
There was a problem hiding this comment.
🟡 AGENTS.md line 7 mandates that every PR title use the format <English title> / <中文标题> and the description include a ## 中文说明 section mirroring the English summary — this PR's title and body are English-only with no Chinese counterpart. Please add the Chinese title suffix and a ## 中文说明 section before merging.
Extended reasoning...
What the rule says
AGENTS.md, line 7, states verbatim: "PR and GitHub-issue titles & descriptions must be bilingual — include a Simplified Chinese version in addition to English. Title format: <English title> / <中文标题>. In the PR/issue body, follow the English content with its Chinese translation (e.g. a ## 中文说明 section mirroring the summary...). ... This applies to every PR and every issue."
Since CLAUDE.md explicitly designates AGENTS.md as the authoritative source of all repository guidance ("All guidance for working with this repository lives in AGENTS.md — read that"), this is not a soft style preference — it is a written, repo-mandated convention that applies unconditionally to "every PR."
How this PR violates it
The PR title is [NV] Add SpeedBench AL collectors for DSV4-Pro-0813 on B300 — entirely in English, with no / <中文标题> suffix. The description is likewise entirely in English (explaining the DSpark collector, the delegation pattern for the probabilistic arm, and the greedy-vs-probabilistic characterization ask from Benjamin), with no ## 中文说明 section anywhere in the body mirroring that summary in Simplified Chinese.
Why nothing else catches this
Nothing in the PR pipeline enforces this automatically — the recipe-reminder bot comment on this PR (2026-08-07) only asks authors to loop in a CODEOWNER for the PR_REVIEW_CHECKLIST.md signoff phrase; it does not check for the bilingual title/description format. So a maintainer manually cross-referencing AGENTS.md is the only enforcement mechanism today, and this PR slipped through that check.
Impact
This is a documentation/process gap, not a functional defect — the collector scripts and launcher change (adding DeepSeek-V4-Pro-0813 to STAGED_MODELS) work regardless of what language the PR metadata is in. It does not affect runtime behavior, the benchmark results, or the correctness of the code being merged.
Fix
Retitle the PR to [NV] Add SpeedBench AL collectors for DSV4-Pro-0813 on B300 / 为 B300 上的 DSV4-Pro-0813 添加 SpeedBench AL 采集脚本 (or equivalent), and append a ## 中文说明 section to the description that mirrors the English summary (DSpark collector added for DeepSeek-V4-Pro-DSpark on B300, following the Kimi-K3 precedent; a second dispatch entry point added for the probabilistic-drafting arm; golden YAML to follow in a separate PR once the run is reviewed).
Step-by-step proof
- Open AGENTS.md and read line 7 — the bilingual requirement is stated in imperative, unconditional language ("must be bilingual", "This applies to every PR and every issue").
- Open this PR's title field in the metadata:
[NV] Add SpeedBench AL collectors for DSV4-Pro-0813 on B300. Search for/followed by CJK characters — none present. - Open the PR description: it is four English paragraphs (DSpark checkpoint background, file-copy rationale, dispatch-file rationale, note about the golden YAML follow-up). Search for
## 中文说明— not present anywhere in the body. - Conclusion: both the title-format rule and the body-mirror rule from AGENTS.md line 7 are violated, confirming the finding.
| # ---- Resolve target weights ---- | ||
| # The DSpark checkpoint is NOT in the launcher's STAGED_MODELS (it is not staged | ||
| # on the B300 cluster yet), so MODEL_PATH resolves to the writable models dir and | ||
| # the ~960 GB download below runs once, on the first collection. Add the basename | ||
| # back to STAGED_MODELS once the weights are staged to read them from the faster | ||
| # read-only mount instead. | ||
| if [[ -n "${MODEL_PATH:-}" ]]; then | ||
| if [[ ! -d "$MODEL_PATH" || -z "$(ls -A "$MODEL_PATH" 2>/dev/null)" ]]; then | ||
| if [[ ! -w "$(dirname "$MODEL_PATH")" ]]; then | ||
| echo "CRITICAL: $MODEL_PATH is empty and $(dirname "$MODEL_PATH") is not writable." | ||
| echo "This means the basename is listed in the launcher's STAGED_MODELS but the" | ||
| echo "weights were never staged. Either get them staged, or remove it from" | ||
| echo "STAGED_MODELS so MODEL_PATH resolves to the writable models dir instead." | ||
| exit 1 | ||
| fi | ||
| echo "=== $MODEL_PATH is empty; downloading $MODEL (~960 GB, first run only) ===" | ||
| hf download "$MODEL" --local-dir "$MODEL_PATH" | ||
| fi |
There was a problem hiding this comment.
🟡 The PR pivots the DSpark AL collector target to DeepSeek-V4-Pro-0813 (staged via runners/launch_b300-nv.sh's STAGED_MODELS), but dsv4dspark_fp4_b300_vllm.sh's own docs/comments (lines 30, 48-52, 116-121) still describe the old standalone DeepSeek-V4-Pro-DSpark checkpoint, including a comment stating it is NOT in STAGED_MODELS and will trigger a ~960GB download. An operator following the script's own documented export MODEL=deepseek-ai/DeepSeek-V4-Pro-DSpark usage example gets a basename that misses STAGED_MODELS and pulls an unnecessary 960GB download instead of reading the pre-staged 0813 weights; these references should be updated to DeepSeek-V4-Pro-0813 to match the launcher change and this PR's stated intent.
Extended reasoning...
What the bug is. This PR's headline change (commit 6162a13, "Read staged DeepSeek-V4-Pro-0813 from the read-only mount") adds DeepSeek-V4-Pro-0813 to STAGED_MODELS in runners/launch_b300-nv.sh, following reviewer feedback (chunfangamd/functionstackx) that the collector should target the 0813 checkpoint since it natively ships DSpark, which the author (qiching) explicitly agreed to ("This PR can include -0813. Now I will update this PR since DeepSeek-V4-Pro-0813 already be staged."). However, the collector script itself, dsv4dspark_fp4_b300_vllm.sh, was never updated to match. It still documents and assumes the old standalone deepseek-ai/DeepSeek-V4-Pro-DSpark checkpoint in several places: the Usage example (export MODEL=deepseek-ai/DeepSeek-V4-Pro-DSpark, line 30-31), the MODEL required-var hint (line 48), the SERVE_MODEL/MODEL_PATH fallback comment (lines 49-52), and most notably the entire "Resolve target weights" comment block (lines 116-121), which flatly states: "The DSpark checkpoint is NOT in the launcher's STAGED_MODELS (it is not staged on the B300 cluster yet), so MODEL_PATH resolves to the writable models dir and the ~960 GB download below runs once."\n\nThe code path that triggers it. In runners/launch_b300-nv.sh, MODEL_PATH is resolved purely from string membership: MODEL_BASENAME="${MODEL##*/}", then if [[ " ${STAGED_MODELS[*]} " == *" ${MODEL_BASENAME} "* ]] sets MODEL_PATH to the read-only HF_HUB_CACHE_MOUNT, else it falls back to the writable WRITABLE_MODELS_DIR. The PR added only DeepSeek-V4-Pro-0813 to that list — DeepSeek-V4-Pro-DSpark was never a member and still isn't. So if an operator follows the collector script's own documented usage (export MODEL=deepseek-ai/DeepSeek-V4-Pro-DSpark), MODEL_BASENAME becomes DeepSeek-V4-Pro-DSpark, misses STAGED_MODELS, and MODEL_PATH resolves to the writable dir. The collector's own empty-dir branch (lines 116-133) then fires and runs hf download "$MODEL" --local-dir "$MODEL_PATH" — an unnecessary ~960GB download — instead of reading the pre-staged 0813 weights that this very PR's launcher change was written to serve from the read-only mount.\n\nWhy nothing currently prevents this. In the automated path, speedbench-al.yml's workflow_dispatch supplies MODEL as an operator-chosen input rather than reading it from the script's own comments, so a knowledgeable operator who inputs DeepSeek-V4-Pro-0813 gets a correct, working run today — the automated CI path does not break. The exposure is specifically for anyone who reads and follows the collector script's own embedded documentation (a very plausible thing to do, since that's exactly what the comments are there for), and for anyone auditing the launcher/collector pairing for correctness — the stale download-block comment is now directly contradicted by the same PR's own launcher diff.\n\nStep-by-step proof:\n1. PR adds DeepSeek-V4-Pro-0813 to STAGED_MODELS in runners/launch_b300-nv.sh (confirmed in diff, line ~386).\n2. dsv4dspark_fp4_b300_vllm.sh line 30 still instructs: export MODEL=deepseek-ai/DeepSeek-V4-Pro-DSpark.\n3. An operator runs that exact command as documented.\n4. Launcher/script computes MODEL_BASENAME=DeepSeek-V4-Pro-DSpark.\n5. [[ " ${STAGED_MODELS[*]} " == *" DeepSeek-V4-Pro-DSpark "* ]] is false (only DeepSeek-V4-Pro-0813 and DeepSeek-V4-Pro are present) → MODEL_PATH = ${WRITABLE_MODELS_DIR}/DeepSeek-V4-Pro-DSpark.\n6. Inside the collector, that path is empty and writable, so the "Resolve target weights" block's empty-dir branch fires: hf download "$MODEL" --local-dir "$MODEL_PATH" — a ~960GB download that the launcher change was specifically meant to avoid.\n\nHow to fix. Update the Usage example (line 30-31), the MODEL required-var hint (line 48), the SERVE_MODEL/MODEL_PATH comment (lines 49-52), and the "Resolve target weights" block's comment (lines 116-121) to reference deepseek-ai/DeepSeek-V4-Pro-0813 and note it is now staged in STAGED_MODELS, consistent with the launcher change and the PR's stated intent.
4 participants
Adds the acceptance-length collectors for deepseek-ai/DeepSeek-V4-Pro-DSpark, following the Kimi-K3 DSpark collector (#2352). DSpark ships the draft inside a separate checkpoint, so unlike K3 there is no external draft head to download and no "model" key in the speculative-config.
dsv4dspark_fp4_b300_vllm.sh is a copy of the DSV4 MTP collector with only the speculative-config changed (method dspark + draft_sample_method); every serve flag is left identical so the DSpark and MTP curves stay comparable, and it already matches the published DSpark recipe.
Benjamin asked for greedy and probabilistic drafting to be characterized on DSV4-Pro. speedbench-al.yml resolves the collector purely as ${model-prefix}_fp4_b300_vllm.sh, so the second arm needs its own dispatchable file; dsv4dsparkprob_fp4_b300_vllm.sh delegates to the base collector with DRAFT_SAMPLE_METHOD=probabilistic rather than duplicating it, so the two arms cannot drift. rejection_sample_method is left at the vLLM default and exposed as an opt-in env knob: the K3 variant bundled it with probabilistic drafting, but the forced-AL config has to stay on a sampling method TRT-LLM also supports.
The golden YAML and its README row follow in a separate PR once the collection run is reviewed.