Skip to content

metrics: spec-decode acceptance counters for the DFlash draft path #604

Description

@xiaguan

TL;DR: vllm:spec_decode_num_drafts/draft_tokens/accepted_tokens_total read 0; the DFlash speculative path (see #443) tracks draft/acceptance counts internally but nothing forwards them. Fill SchedulerStats.spec_decoding_stats from the qwen3 speculative executor. Part of #602.

Upstream records these as interval deltas (SpecDecodingStats in protocol/stats.rs), including per-position acceptance (num_accepted_tokens_per_pos) — the standard acceptance-rate dashboard panel.

Scope:

  • Surface per-step draft/accept counts from the DFlash executor to the scheduler loop.
  • Carry them on the published snapshot (delta-safe, same rule as the prefix-cache counters issue) and fill spec_decoding_stats in the bridge.
  • Verify acceptance rate on /metrics matches the rate the DFlash perf test reports.

Blocked on nothing — the counters exist; this is plumbing. Coordinate with #443 so the metric lands with the roadmap line rather than after it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    hw:1-gpuNeeds one consumer GPU to verifyobservabilityMetrics, tracing, dashboards, request observability

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions