Skip to content

fix(eve): skip span writes after the span has ended - #1233

Open
NicoMoli wants to merge 1 commit into
vercel:mainfrom
NicoMoli:fix/776-ended-span-writes
Open

fix(eve): skip span writes after the span has ended#1233
NicoMoli wants to merge 1 commit into
vercel:mainfrom
NicoMoli:fix/776-ended-span-writes

Conversation

@NicoMoli

Copy link
Copy Markdown
Contributor

Description

Fixes #776.

On long-running self-hosted (non-Vercel) deployments, every turn that makes tool calls emits repeated Operation attempted on ended Span errors at error level. It is non-fatal — the turn finishes and replies send — but it fires several times per turn, buries real errors, and trips log-based alerting.

Root cause: eve writes to an OpenTelemetry span after it has been .end()-ed. harness/tool-loop.ts creates the ai.eve.turn span, installs it as the active span via otelContext.with(...), and ends it in a finally while it is still active. internal/logging.ts writes to the active span on every error log with no guard. Tool-call turns widen the async window (model → tool → model), so an error log emitted on a late microtask continuation can resolve after the finally ended the span.

Fix: guard both span-mutating helpers in internal/logging.ts with isRecording(), which the OTel spec defines as returning false once a span has ended.

  • recordErrorOnSpan — early return when the span is not recording. This is the load-bearing guard; the function is exported and called directly by the harness, so it must be self-protecting.
  • recordOnActiveSpan — tightened from span === undefined to also check !span.isRecording(), short-circuiting all three mutation branches (setStatus / recordException / addEvent) before any write.

isRecording was also added to the vendored @opentelemetry/api Span declaration, which did not previously expose it.

No change to harness/tool-loop.ts: the setAttribute there is synchronous inside the awaited body and provably runs before end(), and the direct recordErrorOnSpan(turnSpan, ...) call sits inside the model-call try where the span is still recording, so the guard is a no-op there.

Behavior while a span is live is unchanged — this only drops writes that would otherwise be errors.

How did you test your changes?

Added a span recording guard block to the existing unit-tier src/internal/logging.test.ts. It mocks #compiled/@opentelemetry/api/index.js (importOriginal, real SpanStatusCode, overridden trace.getActiveSpan) in the style of execution/subagent-usage-span.test.ts, and uses a makeFakeSpan(isRecording) factory whose mutators throw when not recording — mirroring the SDK — so any unguarded write fails the test loudly. Cases cover: ended active span (no throw, no mutation), ended span passed directly to recordErrorOnSpan, and three no-regression cases on a recording span.

pnpm --filter eve exec vitest run --config vitest.unit.config.ts src/internal/logging.test.ts  # 23 passed
pnpm fmt
pnpm lint
pnpm typecheck
pnpm guard:invariants

PR Checklist

  • I linked an issue with prior discussion confirming this change is wanted
  • I ran the relevant checks from CONTRIBUTING.md
  • I added tests and documentation where relevant
  • I added a changeset if this touches the published eve package
  • DCO sign-off passes for every commit (git commit --signoff)

🤖 Generated with Claude Code

https://claude.ai/code/session_013RApsUy4qVAeVCWwmCph2R

Signed-off-by: Nicolas Molina <n_m09@hotmail.com>
@vercel

vercel Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

@NicoMoli is attempting to deploy a commit to the Vercel Team on Vercel.

A member of the Team first needs to authorize it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

setAttribute on an already-ended span throws repeatedly during tool-call turns (non-Vercel, long-running)

1 participant