Skip to content

[Feat] Add timeouts to VLLM Router configuration#983

Open
VariableDeclared wants to merge 2 commits into
vllm-project:mainfrom
VariableDeclared:feat/vllm-req-timeouts
Open

[Feat] Add timeouts to VLLM Router configuration#983
VariableDeclared wants to merge 2 commits into
vllm-project:mainfrom
VariableDeclared:feat/vllm-req-timeouts

Conversation

@VariableDeclared

@VariableDeclared VariableDeclared commented Jun 30, 2026

Copy link
Copy Markdown

FILL IN THE PR DESCRIPTION HERE

FIX #706

Currently the router configuration for the backend will be created with defaults, for some models and scenarios this means the Gateway is timing out before vLLM responds. This is a quick fix to allow users to create a timeout. Long term a more elegant solution would be appropriate.

  • Make sure the code changes pass the pre-commit checks.
  • Sign-off your commit by using -s when doing git commit
  • Try to classify PRs for easy understanding of the type of changes, such as [Bugfix], [Feat], and [CI].

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for configuring timeouts on route rules in the Helm chart, along with corresponding test cases. The reviewer pointed out a potential YAML parsing issue in values.yaml if a user uncomments the example timeout configuration directly, and suggested a safer way to structure the commented-out example.

Comment thread helm/values.yaml Outdated
Comment on lines +666 to +667
timeouts: {}
# requests: 300s

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If a user uncomments the # requests: 300s line directly, it will result in invalid YAML because timeouts is already defined as an inline empty map (timeouts: {}) on the previous line.

To prevent YAML parsing errors when users try to uncomment the example, it is better to comment out the entire block as an example and keep the default timeouts: {} separate.

      # timeouts:
      #   requests: 300s
      timeouts: {}

Signed-off-by: Peter De Sousa <pd@s-ai.com>
@VariableDeclared VariableDeclared force-pushed the feat/vllm-req-timeouts branch from 01dda24 to ed19ef6 Compare June 30, 2026 13:12
Signed-off-by: Peter De Sousa <pd@s-ai.com>
@VariableDeclared VariableDeclared force-pushed the feat/vllm-req-timeouts branch from fdc551b to 681b620 Compare July 1, 2026 14:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: [lmcache/lmstack-router] Router does not forward request after uncertain amount of time

1 participant