Skip to content

[FEAT] add rampup for GuideLLM concurrent benchmarks#31

Merged
VincentG1234 merged 1 commit into
mainfrom
FEAT/guidellm-rampup
Jun 11, 2026
Merged

[FEAT] add rampup for GuideLLM concurrent benchmarks#31
VincentG1234 merged 1 commit into
mainfrom
FEAT/guidellm-rampup

Conversation

@VincentG1234

@VincentG1234 VincentG1234 commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator

Summary

Add optional benchmark.rampup to study configs, forwarding GuideLLM's --rampup flag so concurrent benchmarks can ramp load linearly to target concurrency instead of starting at full rate.

Why

Sudden full-concurrency load can skew benchmark metrics (cold caches, queue buildup, OOM risk). GuideLLM supports a ramp-up period; this fork already exposes warmup and cooldown but not rampup, so users could not control how load increases at the start of a run.

What changed

  • auto_tune_vllm/benchmarks/config.py — add optional rampup field; validate > 0 in __post_init__
  • auto_tune_vllm/benchmarks/providers.py — pass --rampup to GuideLLM CLI when set
  • tests/benchmarks/test_guidellm_command.py — CLI construction and validation tests for rampup
  • docs/configuration.md — document rampup (seconds, included in metrics, unlike warmup)
  • examples/study_config.yaml, examples/study_config_minimal.yaml — commented example
  • README.md — fork changelog entry

How tested

  • ruff check .
  • pytest -v tests/benchmarks/test_guidellm_command.py (11 passed)
  • Manual E2E (maintainer): auto-tune-vllm optimize with benchmark.rampup: 10 and verify GuideLLM receives --rampup 10

Risks / limitations

  • Requires a GuideLLM version that supports --rampup (same pattern as existing warmup/cooldown flags).
  • Ramp-up requests are included in reported metrics; only warmup/cooldown exclude phases from measurement.
  • No interaction with fractional warmup/cooldown sum validation (rampup is always absolute seconds).

Links

  • Follows #24 (warmup/cooldown) and #27 (sample_requests) benchmark config extensions.

Signed-off-by: Vincent Gimenes <vincent.gimenes@gmail.com>
@VincentG1234 VincentG1234 merged commit 43d2d25 into main Jun 11, 2026
7 checks passed
@VincentG1234 VincentG1234 deleted the FEAT/guidellm-rampup branch June 11, 2026 08:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant