Live session stream can stall open indefinitely without erroring, evading all reconnect logic (Vercel CDN compression)

[vprisma]

Summary

On Vercel production deployments, the GET /eve/v1/session/:sessionId/stream NDJSON response is served with content-encoding: br (Vercel's CDN auto-compression, since application/x-ndjson appears to match its compression MIME allowlist). Compressing a long-lived, low-throughput event stream causes the final bytes of a turn to sit in the compressor's buffer without ever flushing — the connection goes silent permanently, without erroring or closing.

Both eve's own official client (followStreamIterable / readNdjsonStream in eve/client, via open-stream.js + ndjson.js) and the vercel-labs/eve-chat-template's own equivalent hand-rolled stream reader share the same blind spot: reconnect/retry logic only fires when the stream throws (isStreamDisconnectError). A connection that stays open but goes silent forever never throws, so it's invisible to the retry policy (streamIdleReconnectPolicy / maxAttempts etc. never get a chance to run, because the code is permanently blocked inside for await (const event of readNdjsonStream(...)) awaiting a chunk that will never arrive).

Impact

100% reproducible (5/5 tries across two separate rounds) on a real Vercel production deployment (smartfront team, Next.js 16.2.6, eve 0.27.3, upgraded from 0.22.1 mid-investigation — same bug on both versions). Every single chat turn got permanently stuck: the assistant's answer streamed and rendered fully, but the UI's "is responding" state (driven by watching for session.completed / session.failed / session.waiting) never cleared, since that terminal event never arrived on the client even though it was already recorded in the session's durable/authoritative event history (verified by manually replaying the same session from startIndex=0 — turn.completed and session.waiting were both present at the correct timestamps).

What we tried at the application level (neither worked, for architectural reasons)

1. vercel.json header override to change the stream response's Content-Type to something outside Vercel's compression allowlist (e.g. application/octet-stream). Confirmed via a diagnostic header that vercel.json rules do reach the route, but a header the origin function already sets explicitly (eve sets Content-Type: application/x-ndjson; charset=utf-8 itself) always wins over the static routing-layer config for that same key.
2. Next.js Middleware to rewrite the response's Content-Type in the request pipeline instead. Deploy failed outright: Edge Runtime is not supported in services. Service "eve" produced Edge Function output "_middleware". Remove the Edge runtime configuration from this service or deploy it outside of services. — eve's routes are deployed as an isolated Vercel Build Output "service", architecturally unreachable from the host Next.js app's own middleware/header config.

Given both of these are blocked, there's currently no way for an application built with eve to work around this compression/stall interaction from its own code — the fix has to happen inside eve's own stream client and/or its Vercel service output.

Our workaround (app-level, not a real fix)

We patched our own copy of the template's streamSessionEvents (the customer app, not this package) to add a 15-second idle timer per connection attempt: if no new event arrives within 15s, the code now forcibly aborts that connection via AbortController (which the existing isStreamDisconnectError check already classifies as a retryable disconnect) and opens a fresh one from the last known event index. This reliably un-sticks the UI in production. We suspect followStreamIterable in eve/client needs the same kind of active idle-timeout-triggered abort (not just error-triggered retry) to be robust against a silently-stalled connection, regardless of platform.

Suggested fixes (any of these would resolve it)

• Add an active per-connection idle timer to followStreamIterable/readNdjsonStream that force-aborts and reconnects when no event has arrived for some threshold, rather than only reacting to thrown errors.
• Serve the stream response with a Content-Type outside Vercel's default CDN compression allowlist (e.g. text/event-stream or application/octet-stream), or otherwise ensure eve's own Vercel Build Output service configuration disables compression for this route, since neither vercel.json nor Next.js Middleware can reach into it from the consuming app.

Environment

• eve 0.27.3 (also reproduced on 0.22.1)
• Next.js 16.2.6, App Router, deployed on Vercel (Fluid Compute)
• Based on vercel-labs/eve-chat-template
• Reproduced with a fresh, cookie-free browser session each time; not intermittent — every first message triggered it before the workaround.

[cedricahenkorah]

Independent reproduction of the same failure class, with a different trigger: no CDN, local eve dev.

Environment: eve 0.27.13, Next.js 16.2.12, two signed-in browsers on one shared session, and a Next route handler that proxies GET /eve/v1/session/:sessionId/stream.

Symptom: one participant's open stream request receives zero bytes during another participant's turn. A second request for the same session and the same startIndex, opened during that turn, receives all 39 chunks live. The stalled connection stays open and silent for five minutes, then fails with undici's default bodyTimeout (UND_ERR_BODY_TIMEOUT). The client then reconnects from its unchanged cursor and receives the full backlog immediately, with no duplicates.

What we traced: in development one stream read crosses four HTTP hops (browser proxy, nitro host, nitro dev worker, and the internal streams.get relay fetch from the worker back to the parent world-local store in packages/eve/src/internal/workflow/development-world-client.ts). We eliminated the browser read path, followStreamIterable's cursor handling, readNdjsonStream, and the world-local reader itself, which repairs missed notifications with a 100 ms disk poller. The stall sits in the relay hops; we did not pin the exact hop. The trigger correlates with a second concurrent session stream from the same browser.

The observation both reports share: every repair mechanism in the chain fires only on a thrown read error. followStreamIterable retries only through isStreamDisconnectError, and the framed workflow stream consumer reconnects only when read() throws. An open but silent connection is invisible to all of them, at every layer. Only undici's 300-second default bodyTimeout converts silence into an error, which is why Node consumers see multi-minute stalls, and a browser fetch, which has no body timeout, stalls permanently.

Our application-level workaround matches the one above, applied server side: our proxy aborts its upstream fetch after 60 seconds without bytes, and the eve client reconnects from the durable cursor without loss.

Offer: we are happy to submit a PR that adds an opt-in read-idle timeout to the client reconnect policy, for example streamReadIdleTimeoutMs next to streamIdleReconnectPolicy. The timer resets on each received event, aborts the connection when it fires, and the existing disconnect-classified retry path then reconnects from the advanced cursor. The default would keep the current behavior. Would the maintainers take that, or is a server-side heartbeat the preferred direction?

[cedricahenkorah]

Independent reproduction of the same failure class, with a different trigger: no CDN, local eve dev.

Environment: eve 0.27.13, Next.js 16.2.12, two signed-in browsers on one shared session, and a Next route handler that proxies GET /eve/v1/session/:sessionId/stream.

Symptom: one participant's open stream request receives zero bytes during another participant's turn. A second request for the same session and the same startIndex, opened during that turn, receives all 39 chunks live. The stalled connection stays open and silent for five minutes, then fails with undici's default bodyTimeout (UND_ERR_BODY_TIMEOUT). The client then reconnects from its unchanged cursor and receives the full backlog immediately, with no duplicates.

What we traced: in development one stream read crosses four HTTP hops (browser proxy, nitro host, nitro dev worker, and the internal streams.get relay fetch from the worker back to the parent world-local store in packages/eve/src/internal/workflow/development-world-client.ts). We eliminated the browser read path, followStreamIterable's cursor handling, readNdjsonStream, and the world-local reader itself, which repairs missed notifications with a 100 ms disk poller. The stall sits in the relay hops; we did not pin the exact hop. The trigger correlates with a second concurrent session stream from the same browser.

The observation both reports share: every repair mechanism in the chain fires only on a thrown read error. followStreamIterable retries only through isStreamDisconnectError, and the framed workflow stream consumer reconnects only when read() throws. An open but silent connection is invisible to all of them, at every layer. Only undici's 300-second default bodyTimeout converts silence into an error, which is why Node consumers see multi-minute stalls, and a browser fetch, which has no body timeout, stalls permanently.

Our application-level workaround matches the one above, applied server side: our proxy aborts its upstream fetch after 60 seconds without bytes, and the eve client reconnects from the durable cursor without loss.

Offer: we are happy to submit a PR that adds an opt-in read-idle timeout to the client reconnect policy, for example streamReadIdleTimeoutMs next to streamIdleReconnectPolicy. The timer resets on each received event, aborts the connection when it fires, and the existing disconnect-classified retry path then reconnects from the advanced cursor. The default would keep the current behavior. Would the maintainers take that, or is a server-side heartbeat the preferred direction?

quick note of transparency: the detailed comment above was written by claude, that I had investigating this bug in my own app. I want to vouch for the substance though, this is a real issue I hit myself while building a multi-participant chat on eve. One user's stream would silently stop receiving events in dev, recover exactly at the 5-minute undici body timeout, and a fresh connection always got the backlog instantly. The 60s server-side recycle workaround described above is running in my app now and works.