Skip to content

Feat: runtime reduction mode API (on/off/deterministic)#3

Merged
OsherElhadad merged 1 commit into
mainfrom
feat/runtime-mode-control
Jul 2, 2026
Merged

Feat: runtime reduction mode API (on/off/deterministic)#3
OsherElhadad merged 1 commit into
mainfrom
feat/runtime-mode-control

Conversation

@itay-nakash

Copy link
Copy Markdown
Collaborator

Summary

Add a process-wide reduction mode settable at runtime via POST /labcx/mode, so an external signal (e.g. a vLLM KV-cache-pressure monitor) can activate/deactivate context engineering live — without restarting the proxy.

  • Mode enum on|off|deterministic; ParseMode validates (unknown -> HTTP 400). off = transparent passthrough; on = reduce with the configured method; deterministic is reserved and currently behaves like on.
  • GET /labcx/mode reports the current mode; POST/PUT sets it from {"mode":"..."} or ?mode=. Held in an atomic.Value.
  • Reduce gate becomes: !bypassed(r) && mode != off. The per-request x-labcx-bypass header still forces passthrough. Default mode = on, so existing behavior is unchanged. Fail-open preserved.
  • --mode initial-state flag on lab-cx proxy.
  • TestRuntimeModeToggle: on/off/deterministic flips reduce<->passthrough, bad value -> 400, GET reports state.

Related issue(s)

(Optional) Testing Instructions

  • make test (full suite, with the race detector) is green; go vet and gofmt are clean.
  • Manual: start lab-cx proxy, then
    • curl -XPOST localhost:8080/labcx/mode -d '{"mode":"off"}' -> next reduce-eligible request is forwarded verbatim;
    • curl -XPOST localhost:8080/labcx/mode -d '{"mode":"on"}' -> reduction resumes;
    • curl -XPOST localhost:8080/labcx/mode -d '{"mode":"bogus"}' -> HTTP 400;
    • curl localhost:8080/labcx/mode -> {"mode":"..."}; /ready unaffected.

Fixes #

Add a process-wide reduction mode settable at runtime via POST /labcx/mode, so an
external signal (e.g. a vLLM KV-cache-pressure monitor) can activate/deactivate
context engineering live — without restarting the proxy.

- Mode enum on|off|deterministic; ParseMode validates (unknown -> HTTP 400).
  off = transparent passthrough; on = reduce with the configured method;
  deterministic is reserved and currently behaves like on.
- GET /labcx/mode reports the current mode; POST/PUT sets it from {"mode":"..."}
  or ?mode=. Held in an atomic.Value.
- Reduce gate becomes: !bypassed(r) && mode != off. The per-request
  x-labcx-bypass header still forces passthrough. Default mode = on, so existing
  behavior is unchanged. Fail-open preserved.
- --mode initial-state flag on `lab-cx proxy`.
- TestRuntimeModeToggle: on/off/deterministic flips reduce<->passthrough, bad
  value -> 400, GET reports state.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Itay-Nakash <itay.nakash@ibm.com>
@OsherElhadad
OsherElhadad merged commit 8248897 into main Jul 2, 2026
3 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants