feat(chat): add BFF chat SSE proxy (Phase 4)#216
Merged
Conversation
philmerrell
force-pushed
the
feature/bff-phase-4-chat-proxy
branch
from
b5d6ad8 to
b0d3fd1
Compare
Adds POST /chat/proxy-stream on app-api, the cookie-authenticated SSE
proxy the SPA will adopt in Phase 6 once it cuts over from Bearer-in-
localStorage to the __Host-bff_session cookie. Dormant in Phase 4 —
no SPA caller exists yet.
Mechanics:
- Authenticated via get_current_user_from_session (Phase 2 dep).
SessionRefreshMiddleware refreshes the stored Cognito access token
if near expiry before the handler runs.
- Forwards request body verbatim to {INFERENCE_API_URL}/invocations
with Authorization: Bearer <current_user.raw_token>. Inference-api
already accepts Cognito Bearer tokens via get_current_user_trusted —
no inference-api changes (architecture decision #4).
- Sets X-Accel-Buffering: no and Cache-Control: no-cache so late SSE
events (notably oauth_required after message_stop) reach the
browser without an intermediary holding them back.
- Guarded by CSRFMiddleware via the existing exempt-list; double-
submit X-CSRF-Token must match the __Host-bff_csrf cookie.
- Maps upstream connect/timeout errors to 502/504 to match the
existing converse proxy.
Lifecycle subtlety: the upstream httpx.AsyncClient is built via a
_build_upstream_client() seam and closed in the streaming generator's
finally block — closing the client via async with while a stream is
in flight makes httpx drain the upstream during __aexit__, defeating
the whole point of the streaming proxy. The seam also lets tests
inject a MockTransport-backed client without mutating global httpx.
Tests (16 new):
- auth gate (no session → 401),
- header/body/URL relay,
- SSE and non-SSE response paths,
- upstream 4xx/5xx propagation,
- ConnectError → 502, TimeoutException → 504,
- CSRF: missing → 403, header/cookie mismatch → 403, valid → 200,
plus a guard that /chat/proxy-stream is not in
CSRFMiddleware._EXEMPT_PATHS,
- TTFB < 200ms integration test backed by a real uvicorn server
(httpx ASGITransport buffers the body before headers, so it can't
measure TTFB) with a slow upstream that yields, sleeps 300ms, then
yields more — asserts headers arrive fast and the late
oauth_required event makes it through.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
philmerrell
force-pushed
the
feature/bff-phase-4-chat-proxy
branch
from
b0d3fd1 to
6f6efc2
Compare
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 4 of the BFF Token Handler migration. Adds the cookie-authenticated SSE proxy the SPA will adopt at cutover (Phase 6) — dormant today since no SPA caller exists yet.
Stacked on #215 (Phase 3). Will rebase onto develop after Phase 3 merges. Review can start in parallel; merge order is Phase 3 → Phase 4.
What it does
`POST /chat/proxy-stream` — cookie-authenticated, forwards the request body verbatim to inference-api `/invocations` with `Authorization: Bearer `. Streams SSE responses straight back. Inference-api's runtime authorizer already accepts Cognito Bearer tokens, so zero inference-api changes were needed (architecture decision #4 in the migration plan).
```
Browser → CloudFront /api/* → app-api → inference-api /invocations
(httpOnly cookie) (Bearer )
```
`SessionRefreshMiddleware` (Phase 2) refreshes the stored token if near expiry before this handler runs, so the proxy always forwards a fresh token.
Notable findings
Real bug fixed in the proxy lifecycle: closing `httpx.AsyncClient` via `async with` while a stream is in flight makes httpx drain the upstream during `aexit` — buffering the entire SSE response before headers can flush downstream. This would have hidden the late `oauth_required` SSE event (emitted after `message_stop`) from the SPA. Fix: own the client lifecycle manually, close it in the streaming generator's `finally`. A `_build_upstream_client()` seam was added so tests can substitute a `MockTransport` client without mutating global `httpx`.
Same bug exists in `converse_routes.py` (the API-key converse proxy). Flagged as a separate follow-up — independent of the BFF migration since that proxy isn't browser-facing.
CSRF guard
The route is POST + cookie-auth, so `CSRFMiddleware` enforces double-submit `X-CSRF-Token` ↔ `__Host-bff_csrf` cookie. There's a guard test asserting `/chat/proxy-stream` is not in `_EXEMPT_PATHS` — exempting it would defeat the entire BFF security model.
Test plan
🤖 Generated with Claude Code