[Feat] Implement OpenAI external provider

[shernshiou]

Summary

Wires the existing external_providers/ infrastructure into the live request flow, enabling the router to forward requests to OpenAI (and any other OpenAI-compatible provider) in addition to local vLLM backends.

FIX #883

Changes

services/request_service/request.py

• Added process_external_provider_request() — handles requests destined for external providers (streaming + non-streaming), tracks metrics, propagates OTel spans.
• Modified route_general_request() — added early-exit check after alias resolution: if external_provider_registry.is_external_model(requested_model), the request is forwarded to the external provider and the vLLM backend path is skipped entirely.

routers/main_router.py

• show_models() now accepts Request (FastAPI-injected) and appends external provider model cards to the /v1/models response, giving clients a unified model list.

service_discovery.py

• Added ServiceDiscoveryType.EXTERNAL_ONLY = "external-only".
• Added ExternalOnlyServiceDiscovery — a no-op implementation that returns empty endpoint lists and is always healthy. Used for deployments with no local vLLM backends.

parsers/parser.py

• Added "external-only" to --service-discovery choices.
• Relaxed validation: --service-discovery is no longer required when --external-providers-config is provided (defaults to "external-only" automatically).

app.py

• Added "external-only" branch in initialize_all() to initialize ExternalOnlyServiceDiscovery.
• lifespan() now calls registry.validate_models() at startup to remove any configured models that the provider does not actually serve.

external_providers/base.py

• Added abstract method fetch_available_model_ids() for live model validation.

external_providers/openai_provider.py

• Implemented fetch_available_model_ids() — queries /v1/models and returns the provider's live model list.

external_providers/registry.py

• Added validate_models() — queries each registered provider at startup and removes model IDs not present in the provider's live model list, preventing misconfigured models from being advertised or routed.

examples/external_providers_config.yaml (new)

• Example config for testing with OpenAI as an external provider.

---

How to test

export OPENAI_API_KEY=sk-...

vllm-router \
  --port 8001 \
  --service-discovery external-only \
  --routing-logic roundrobin \
  --external-providers-config examples/external_providers_config.yaml

# List available models (only models OpenAI actually serves will appear)
curl http://localhost:8001/v1/models | jq .

# Chat request
curl http://localhost:8001/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-4o", "messages": [{"role": "user", "content": "hello"}]}'

---

• Make sure the code changes pass the pre-commit checks.
• Sign-off your commit by using -s when doing git commit
• Try to classify PRs for easy understanding of the type of changes, such as [Bugfix], [Feat], and [CI].

Detailed Checklist (Click to Expand)

Thank you for your contribution to production-stack! Before submitting the pull request, please ensure the PR meets the following criteria. This helps us maintain the code quality and improve the efficiency of the review process.

PR Title and Classification

Please try to classify PRs for easy understanding of the type of changes. The PR title is prefixed appropriately to indicate the type of change. Please use one of the following:

• [Bugfix] for bug fixes.
• [CI/Build] for build or continuous integration improvements.
• [Doc] for documentation fixes and improvements.
• [Feat] for new features in the cluster (e.g., autoscaling, disaggregated prefill, etc.).
• [Router] for changes to the vllm_router (e.g., routing algorithm, router observability, etc.).
• [Misc] for PRs that do not fit the above categories. Please use this sparingly.

Note: If the PR spans more than one category, please include all relevant prefixes.

Code Quality

The PR need to meet the following code quality standards:

• Pass all linter checks. Please use pre-commit to format your code. See README.md for installation.
• The code need to be well-documented to ensure future contributors can easily understand the code.
• Please include sufficient tests to ensure the change is stay correct and robust. This includes both unit tests and integration tests.

DCO and Signed-off-by

When contributing changes to this project, you must agree to the DCO. Commits must include a Signed-off-by: header which certifies agreement with the terms of the DCO.

Using -s with git commit will automatically add this header.

What to Expect for the Reviews

We aim to address all PRs in a timely manner. If no one reviews your PR within 5 days, please @-mention one of YuhanLiu11
, Shaoting-Feng or ApostaC.

[gemini-code-assist]

Code Review

This pull request introduces support for external LLM providers (e.g., OpenAI) by implementing an adapter-based architecture and a provider registry. Key additions include a new 'external-only' service discovery mode, automated model validation against live provider APIs, and the logic to route requests to external endpoints. Feedback highlights a critical issue where abstract methods in the base adapter class are defined as synchronous while their implementations are asynchronous, which would lead to runtime errors. Other recommendations include removing redundant methods in the OpenAI provider and optimizing the request processing flow by avoiding unnecessary JSON re-parsing.

[ruizhang0101]

LGTM :))