Skip to content

Rust frontend: capture-spec parity — filesystem consumer requires client to supply request_id/tag (no _admit_capture) #190

Description

@RhizoNymph

Summary

The Rust frontend forwards the per-request capture dict verbatim to the engine-core and does not run the Python OpenAI entrypoint's synchronous _admit_capture / resolve_capture_prefix_flags step. As a result, the worker's per-consumer admission constructs FilesystemCaptureRequest(**spec) directly, and rejects the request unless the client has put request_id and tag into the per-consumer spec itself — fields the Python path arranges during admission.

This is the deferred "capture-spec validation parity" item from rust/docs/features/steering_capture.md (Follow-ups), filed here with a concrete repro.

Repro (gemma4-31B GGUF, --enable-steering --capture-consumers filesystem:root=...)

Spec without tag/request_id → admission rejected:

curl -s http://127.0.0.1:8000/v1/completions -H 'Content-Type: application/json' -d '{
  "model": "<model>", "prompt": "The capital of France is", "max_tokens": 8,
  "capture": {"filesystem": {"hooks": {"post_mlp": [0, 30]}, "positions": "last_prompt"}}
}'

Worker log:

capture admission rejected req=... consumer=filesystem:
  FilesystemCaptureRequest.__init__() missing 2 required positional arguments: 'request_id' and 'tag'

Adding only tag then reports missing 1 required positional argument: 'request_id'. Supplying both works and writes activations:

curl -s http://127.0.0.1:8000/v1/completions -H 'Content-Type: application/json' -d '{
  "model": "<model>", "prompt": "The capital of France is", "max_tokens": 8,
  "capture": {"filesystem": {"request_id": "demo-req-1", "tag": "demo",
                              "hooks": {"post_mlp": [0, 30]}, "positions": "last_prompt"}}
}'
# → writes <root>/<tag>/<request_id>/{0,30}_post_mlp.{bin,json}, shape [1,5376] bf16

Impact

  • Capture works end-to-end, but the client must know to inject request_id/tag for the filesystem consumer — a leak of an internal admission detail that the Python entrypoint hides.
  • Malformed capture specs surface as an engine-core error mid-request rather than a clean 400 up front.

Suggested fix

Replicate the Python entrypoint's _admit_capture in the Rust frontend (or have the worker-side admission inject request_id from the actual scheduled request id, as the Python path effectively does), so:

  1. The client spec no longer needs request_id (and ideally not tag either, or tag defaults sensibly).
  2. Bad specs are rejected with a 400 before scheduling.

Pointers

  • vllm/entrypoints/openai/completion/serving.py _admit_capturebuild_capture_context(..., request_id) + resolve_capture_prefix_flags.
  • vllm/v1/capture/consumers/filesystem/types.py FilesystemCaptureRequest(request_id, tag, hooks, positions).
  • Worker rejection site: vllm/v1/worker/gpu_model_runner.py capture admission.
  • Found while validating the Rust frontend on feat/rust-steering (PR feat(rust): steering, capture, and named steering modules in the rust frontend #187).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions