docs(auto-routing): document default_tier for prompts with no scoring signal - #703
Open
tin-berri wants to merge 7 commits into
Open
docs(auto-routing): document default_tier for prompts with no scoring signal#703tin-berri wants to merge 7 commits into
tin-berri wants to merge 7 commits into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Open
5 tasks
… signal The heuristic scorer has no abstain path today, so a prompt matching none of its keyword lists scores 0.0 and falls under simple_medium into SIMPLE. BerriAI/litellm#35050 adds default_tier (MEDIUM by default) for that case; this documents the knob, the decision-log line it produces, and the reason the check is on the dimensions rather than on the weighted score
…the spend change default_tier resolves to a model the same way a classified tier does, through the tier's own models then default_model then MEDIUM, rather than needing its own non-empty entry or a default_model. Adds the upgrade note the behavior change owes existing deployments: the tier mix and the roughly 53% per-turn price move, and default_tier: SIMPLE as the one-line way to keep the old behavior.
…-pool behavior The default_tier change alters existing default behavior with no config edit, which the release-note convention says belongs in a :::danger admonition rather than a Key Highlights bullet. Also corrects the tier-pool line: an empty pool says the tier names no models, so it resolves like an absent tier and falls through to default_model. Only adaptive: true rejects empty pools at load.
tin-berri
force-pushed
the
docs/complexity-router-default-tier
branch
from
August 1, 2026 06:12
9f78e33 to
3d59e7f
Compare
A default_tier nothing can serve is rejected at load whether you set it or take the default. Partial tiers maps are unaffected because the proxy derives a default_model; the shape this rejects is a plugin config whose tiers omits MEDIUM, which plugins cannot rescue with default_model and which previously raised on the first no-signal request.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Documents the
default_tierknob added in BerriAI/litellm#35050The heuristic scorer's dimensions are a keyword whitelist, so a prompt matching none of them scored 0.0 and fell under
simple_mediuminto SIMPLE; unclassifiable traffic was asserted to be simple.default_tier(MEDIUM by default) now covers that case, anddefault_tier: SIMPLErestores the old behaviorAdds the knob to the full-config block, a "No signal" paragraph under the heuristic scorer explaining why the check is on the dimensions rather than the weighted score, and the matching decision-log line
Note for the release note that ships #35050: this changes spend for every deployment on the heuristic default with no config change on their side (blended per-turn price proxy 2.78 -> 4.24 on a 257-session agent corpus, about +53%), so it belongs in a
:::danger Breaking Changesadmonition withdefault_tier: SIMPLEnamed as the opt-out