docs(proxy): document the mock testing request params opt-in - #730
Open
yuneng-berri wants to merge 1 commit into
Open
docs(proxy): document the mock testing request params opt-in#730yuneng-berri wants to merge 1 commit into
yuneng-berri wants to merge 1 commit into
Conversation
Both fallback testing pages still said the mock_testing_* params were stripped from Proxy requests with no way to force a fallback, which sent readers off to stand up a deliberately broken deployment instead Replace that with the opt-in, the full list of params it covers, a working curl for each page, and what to weigh before turning it on in production
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Companion to BerriAI/litellm#35423, which puts the six mock testing request params behind one opt-in on the Proxy
Both fallback testing pages have been wrong since the params were first gated.
docs/proxy/reliability.mdanddocs/proxy/fallback_management.mdtold readers the flags were stripped from Proxy requests and had no effect, with the only suggested alternative being to stand up a deliberately broken deployment. That is what customers were actually doing, and on a production server it means injecting a failing model into a live config just to exercise routingBoth pages now document the opt-in instead, with a working curl on each. Two details the pages spell out rather than leave to inference: the setting covers
mock_timeoutandmock_delayas well as the threemock_testing_*params, even though the key name only saysmock_testing; and it is read fromconfig.yamlonly, so it cannot be set from the Admin UI or/config/updateThe production guidance is deliberately not a blanket "never enable this". The use case that prompted it is verifying failover on a live server, so the pages frame it as staging plus controlled production drills, state plainly that any caller with a valid key can use it while it is on, and say to turn it off when the drill is done
npm run buildpasses and reports no broken links on either page