feat(proxy): resolve root_path per request from a configured prefix list (SERVER_ROOT_PATHS)

[gym-cmd]

TLDR

Problem this solves:

• SERVER_ROOT_PATH is a single scalar stamped onto the app at startup, so one deployment can serve exactly one client-visible URL path prefix: Starlette strips that one prefix before route matching, and a request under any other prefix 404s before a handler runs
• A pod whose ingress preserves several path prefixes into it (e.g. /tenant-a/* and /tenant-b/* both terminating at the same LiteLLM) is therefore forced into one Deployment per prefix purely to encode it — and MCP OAuth discovery can emit only one prefix's URLs, so every other origin's client aborts on the RFC 9728 §3 exact-match check
• This is the implementation of the direction agreed in the #35226 discussion: "the missing piece is per-request root_path, not the discovery builders" — with the allowed-prefix-list shape proposed in the closing comment

How it solves it:

• New opt-in env SERVER_ROOT_PATHS — comma-separated list of the client-visible prefixes the ingress preserves, e.g. SERVER_ROOT_PATHS=/tenant-a,/tenant-b
• A small outermost ASGI middleware matches the incoming path against the list on a whole-segment boundary (longest prefix first) and sets scope["root_path"] to the matched prefix for that request only
• Everything downstream is stock Starlette: routes stay registered root-relative (route matching strips root_path via get_route_path), and request.base_url re-includes it — so the discovery documents' resource and the 401 challenges' resource_metadata land under the prefix the client actually called, with zero changes to the discovery builders
• A path under no configured prefix passes through untouched: root-relative routes serve exactly as today, anything else 404s (the allowed-list semantics). With the env unset the middleware is not added at all — the default middleware stack is byte-identical

Relevant issues

• Follow-up to fix(mcp): derive OAuth discovery URLs from request path when opt-in #35226 (closed after review): the opt-in discovery-builder rewrite there was rejected because the multi-prefix topology was not routable — with a scalar root_path, the second prefix 404s before the builder runs, and prefixed AS endpoints would 404. Per-request root_path dissolves both objections: the whole app routes under each configured prefix, so prefixed discovery and prefixed /authorize//token//register all resolve
• Composes with (does not depend on) fix(mcp): strip root_path before matching the per-server MCP route spelling #35576: the spelling normalization there strips the scope's root_path from _original_path, which this middleware populates per request — so once both land, the legacy /{server}/mcp spelling selection is also correct under per-request prefixes

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Live proxy, config below, no database:

SERVER_ROOT_PATHS="/tenant-a,/tenant-b" LITELLM_MASTER_KEY=sk-1234 \
  python litellm/proxy/proxy_cli.py --config mcp_multiprefix_config.yaml --port 4102

mcp_servers:
  github:
    url: "https://upstream.example/mcp"
    transport: "http"
    auth_type: "oauth2"
    oauth2_flow: "authorization_code"
    client_id: "gateway-managed-client"
    client_secret: "gateway-managed-secret"
    authorization_url: "https://upstream.example/oauth/authorize"
    token_url: "https://upstream.example/oauth/token"

BEFORE (same build, SERVER_ROOT_PATHS unset — i.e. today's behavior): the prefixes are unroutable, exactly as diagnosed in the #35226 discussion:

$ curl -s -o /dev/null -w '%{http_code}\n' -X POST http://127.0.0.1:4103/tenant-a/mcp/github \
    -H 'Content-Type: application/json' -H 'Accept: application/json, text/event-stream' \
    -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'
404
$ curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:4103/tenant-a/.well-known/oauth-protected-resource/mcp/github
404

AFTER (SERVER_ROOT_PATHS="/tenant-a,/tenant-b"): one pod, both prefixes; each 401 challenge advertises a metadata URL under its own prefix:

$ for p in /tenant-a /tenant-b ""; do
    curl -s -i -X POST "http://127.0.0.1:4102${p}/mcp/github" \
      -H 'Content-Type: application/json' -H 'Accept: application/json, text/event-stream' \
      -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' | grep -iE '^HTTP/|^www-authenticate'
  done

HTTP/1.1 401 Unauthorized
www-authenticate: Bearer resource_metadata="http://127.0.0.1:4102/tenant-a/.well-known/oauth-protected-resource/mcp/github"
HTTP/1.1 401 Unauthorized
www-authenticate: Bearer resource_metadata="http://127.0.0.1:4102/tenant-b/.well-known/oauth-protected-resource/mcp/github"
HTTP/1.1 401 Unauthorized
www-authenticate: Bearer resource_metadata="http://127.0.0.1:4102/.well-known/oauth-protected-resource/mcp/github"

and each advertised document's resource is exactly the URL its client called (RFC 9728 §3), per prefix, from the same pod:

$ curl -s http://127.0.0.1:4102/tenant-a/.well-known/oauth-protected-resource/mcp/github
{"authorization_servers": ["http://127.0.0.1:4102/tenant-a/mcp"],
 "resource": "http://127.0.0.1:4102/tenant-a/mcp/github", "scopes_supported": []}

$ curl -s http://127.0.0.1:4102/tenant-b/.well-known/oauth-protected-resource/mcp/github
{"authorization_servers": ["http://127.0.0.1:4102/tenant-b/mcp"],
 "resource": "http://127.0.0.1:4102/tenant-b/mcp/github", "scopes_supported": []}

The AS metadata chain resolves under the prefix too — the 404 trap that invalidated prefixing discovery URLs in #35226's first review round does not exist here, because the whole app routes per-prefix:

$ curl -s http://127.0.0.1:4102/tenant-a/.well-known/oauth-authorization-server/mcp
{"issuer": "http://127.0.0.1:4102/tenant-a/mcp",
 "authorization_endpoint": "http://127.0.0.1:4102/tenant-a/authorize",
 "token_endpoint": "http://127.0.0.1:4102/tenant-a/token",
 "registration_endpoint": "http://127.0.0.1:4102/tenant-a/register", ...}

$ curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:4102/tenant-a/authorize
422        # routed to the real handler (param validation), not 404

Controls, same instance: unprefixed requests emit today's URLs unchanged (resource: http://127.0.0.1:4102/mcp/github), and an unlisted prefix 404s (/tenant-c/... → 404).

Type

🆕 New Feature

Changes

• litellm/proxy/middleware/per_request_root_path_middleware.py (new): PerRequestRootPathMiddleware + SERVER_ROOT_PATHS parsing/normalization (whitespace/trailing-slash canonicalization, leading-/ validation with a warning, dedupe, longest-first ordering so nested prefixes match most-specific). Sets scope["root_path"] on a segment-boundary match; never rewrites scope["path"] (Starlette strips root_path from the path at match time — stripping here too would double-strip)
• litellm/proxy/proxy_server.py: add the middleware last (Starlette's last-added is outermost) and only when SERVER_ROOT_PATHS is set, so the prefix is resolved before any inner middleware or the router inspects the path and the default stack is untouched. Warns when both SERVER_ROOT_PATH and SERVER_ROOT_PATHS are configured (a matched prefix overrides the scalar for that request)
• litellm/proxy/_lazy_features.py: LazyFeatureMiddleware now strips the scope's root_path (falling back to the cached SERVER_ROOT_PATH scalar) before feature prefix matching — the MCP discovery router is lazily registered, so without this it would never load under a per-request prefix. For scalar deployments the scope value equals the cached env value, so behavior is unchanged
• Tests: middleware unit + normalization + mini-app routing tests (new file); end-to-end discovery tests pinning the two-prefixes-one-app documents, the unprefixed-unchanged guard, the unlisted-prefix 404, the challenge-URL prefix contract, and a control pinning that a prefixed well-known request 404s without the middleware; lazy-feature strip cases parametrized alongside the existing SERVER_ROOT_PATH ones

Notes on shape, per the mechanism question left open in the #35226 closing comment (config-file vs env vs header):

• Env var keeps parity with its scalar sibling SERVER_ROOT_PATH and is available at import time, where the middleware stack is assembled. Happy to move it to a general_settings key if you prefer that surface
• Not header-driven: the prefix list is operator-affirmed static topology; deriving it from request headers would let clients steer emitted URLs
• PROXY_BASE_URL remains a scalar override for discovery origin resolution — a multi-prefix deployment should leave it unset and control scheme/host via trusted X-Forwarded-* (unchanged behavior)
• Known limitation, called out rather than hidden: the host-root RFC-inserted well-known spellings (https://host/.well-known/oauth-protected-resource/tenant-a/mcp/github) are not served — same as today. The operative MCP flow is challenge-driven and the advertised prefix-anchored spelling is served and ingress-routable under the tenant prefix. If you want the inserted spellings too, per-prefix route registration is a natural follow-up

QA runbook

1. uv run --no-sync pytest tests/test_litellm/proxy/middleware/ tests/test_litellm/proxy/_experimental/mcp_server/test_discoverable_endpoints.py tests/test_litellm/proxy/_experimental/mcp_server/auth/ tests/test_litellm/proxy/test_lazy_openapi_snapshot.py -q (804 passed locally) and uv run --no-sync pytest "tests/test_litellm/proxy/test_proxy_server.py::TestLazyFeatureMiddleware" -q
2. Write the mcp_servers config above to mcp_multiprefix_config.yaml
3. SERVER_ROOT_PATHS="/tenant-a,/tenant-b" LITELLM_MASTER_KEY=sk-1234 python litellm/proxy/proxy_cli.py --config mcp_multiprefix_config.yaml --port 4102
4. Run the curl loop from the proof section; confirm each prefix's challenge and document carry that prefix and resource equals the MCP URL called
5. Confirm the unprefixed spellings on the same instance are byte-identical to a run without SERVER_ROOT_PATHS, and /tenant-c/... 404s
6. Restart without SERVER_ROOT_PATHS and confirm behavior is identical to upstream (middleware absent from the stack)

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[codecov]

Codecov Report

❌ Patch coverage is 93.75000% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/proxy/proxy_server.py	50.00%	3 Missing ⚠️

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

The PR adds opt-in per-request ASGI root-path resolution for deployments serving multiple preserved URL prefixes and updates lazy feature loading to normalize paths against each request's resolved prefix.

• Adds SERVER_ROOT_PATHS parsing, normalization, and segment-boundary matching.
• Registers the resolver as the outermost proxy middleware.
• Adds middleware, routing, lazy-loading, and MCP discovery coverage for multiple prefixes.

Confidence Score: 5/5

The PR appears safe to merge, with only non-blocking repository-guideline issues remaining.

The runtime root-path implementation has no established blocking failure; the remaining prior-thread concerns are limited to retained commentary and fixture-based mutation of shared test state.

Important Files Changed

Filename	Overview
litellm/proxy/middleware/per_request_root_path_middleware.py	Adds normalized, longest-prefix-first per-request root_path selection without rewriting the ASGI path.
litellm/proxy/_lazy_features.py	Makes lazy feature prefix matching use the request scope's root_path before falling back to the scalar configuration.
litellm/proxy/proxy_server.py	Conditionally installs per-request root-path resolution as the outermost middleware.
tests/test_litellm/proxy/_experimental/mcp_server/test_discoverable_endpoints.py	Adds MCP discovery and challenge coverage for prefixed and unprefixed requests.
tests/test_litellm/proxy/middleware/test_per_request_root_path_middleware.py	Covers normalization, boundary matching, nested prefixes, scalar overrides, WebSockets, and end-to-end routing.
tests/test_litellm/proxy/test_proxy_server.py	Extends lazy feature tests to cover per-request root-path precedence and boundary handling.

_{Reviews (2): Last reviewed commit: "review(greptile): trim implementation co..." | Re-trigger Greptile}

[gym-cmd]

Both P2s addressed in 548bed2 (commentary trim, net −38 lines; fixture-owned MCP registry state in the new tests — replied on each thread). Local re-validation: 346 tests across the middleware, discovery, and lazy-feature suites green, ruff format/check and the strict-budget gate clean.

@greptileai review

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing gym-cmd:feat/proxy-per-request-root-path (88acad9) with litellm_internal_staging (b735578)}