You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The Rust frontend forwards the per-request capture dict verbatim to the engine-core and does not run the Python OpenAI entrypoint's synchronous _admit_capture / resolve_capture_prefix_flags step. As a result, the worker's per-consumer admission constructs FilesystemCaptureRequest(**spec) directly, and rejects the request unless the client has put request_idandtag into the per-consumer spec itself — fields the Python path arranges during admission.
This is the deferred "capture-spec validation parity" item from rust/docs/features/steering_capture.md (Follow-ups), filed here with a concrete repro.
Adding only tag then reports missing 1 required positional argument: 'request_id'. Supplying both works and writes activations:
curl -s http://127.0.0.1:8000/v1/completions -H 'Content-Type: application/json' -d '{ "model": "<model>", "prompt": "The capital of France is", "max_tokens": 8, "capture": {"filesystem": {"request_id": "demo-req-1", "tag": "demo", "hooks": {"post_mlp": [0, 30]}, "positions": "last_prompt"}}}'# → writes <root>/<tag>/<request_id>/{0,30}_post_mlp.{bin,json}, shape [1,5376] bf16
Impact
Capture works end-to-end, but the client must know to inject request_id/tag for the filesystem consumer — a leak of an internal admission detail that the Python entrypoint hides.
Malformed capture specs surface as an engine-core error mid-request rather than a clean 400 up front.
Suggested fix
Replicate the Python entrypoint's _admit_capture in the Rust frontend (or have the worker-side admission inject request_id from the actual scheduled request id, as the Python path effectively does), so:
The client spec no longer needs request_id (and ideally not tag either, or tag defaults sensibly).
Bad specs are rejected with a 400 before scheduling.
Summary
The Rust frontend forwards the per-request
capturedict verbatim to the engine-core and does not run the Python OpenAI entrypoint's synchronous_admit_capture/resolve_capture_prefix_flagsstep. As a result, the worker's per-consumer admission constructsFilesystemCaptureRequest(**spec)directly, and rejects the request unless the client has putrequest_idandtaginto the per-consumer spec itself — fields the Python path arranges during admission.This is the deferred "capture-spec validation parity" item from
rust/docs/features/steering_capture.md(Follow-ups), filed here with a concrete repro.Repro (gemma4-31B GGUF,
--enable-steering --capture-consumers filesystem:root=...)Spec without
tag/request_id→ admission rejected:Worker log:
Adding only
tagthen reportsmissing 1 required positional argument: 'request_id'. Supplying both works and writes activations:Impact
request_id/tagfor the filesystem consumer — a leak of an internal admission detail that the Python entrypoint hides.400up front.Suggested fix
Replicate the Python entrypoint's
_admit_capturein the Rust frontend (or have the worker-side admission injectrequest_idfrom the actual scheduled request id, as the Python path effectively does), so:request_id(and ideally nottageither, ortagdefaults sensibly).400before scheduling.Pointers
vllm/entrypoints/openai/completion/serving.py_admit_capture→build_capture_context(..., request_id)+resolve_capture_prefix_flags.vllm/v1/capture/consumers/filesystem/types.pyFilesystemCaptureRequest(request_id, tag, hooks, positions).vllm/v1/worker/gpu_model_runner.pycapture admission.feat/rust-steering(PR feat(rust): steering, capture, and named steering modules in the rust frontend #187).