Reimplement Weave tracer and unify emitter interface #411

ultmaster · 2025-12-14T17:25:26Z

The current Weave implementation bases on registering a handler in finish_call. This suffers from drawbacks that the recorded call is not complete, as Weave will internally creates a different CallSchema object and upload to server. In parallel, Weave also provides (implicitly) a way for customizing TraceServer. If properly substituting the TraceServer with a customized implementation, we would be able to get all the traces.

This PR also refactors the emitter interface. The current approach heavily depends on otel, which is not a must for Weave. By decoupling emitter from tracer implementation (which now divides into Otel and non-Otel), and create_span and operation_context are added to tracer interface, we now have a way to clearly separate the creation of spans (via SpanCoreFields) from the propagation of spans (to tracers).

…weave-test

Copilot

Pull request overview

This PR reimplements the Weave tracer integration and unifies the emitter interface across different tracer backends. Key changes include:

Introduces a common SpanRecordingContext protocol and SpanCoreFields type for tracer-agnostic span creation
Refactors emitters (operation, message, exception, object, annotation) to use the unified interface
Replaces the old Weave tracer implementation with a new callback-based architecture using an in-memory trace server

Reviewed changes

Copilot reviewed 36 out of 38 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
agentlightning/tracer/weave.py	Complete rewrite of Weave tracer using callback pattern and unified span recording interface
agentlightning/tracer/dummy.py	New dummy tracer for testing without backend dependencies
agentlightning/tracer/base.py	Added active tracer management and new abstract methods for span creation
agentlightning/tracer/otel.py	Updated to implement unified span creation interface
agentlightning/emitter/*.py	Refactored all emitters to use get_active_tracer() and SpanRecordingContext
agentlightning/utils/otel.py	Added sanitization, flattening, and exception formatting utilities
agentlightning/types/tracer.py	Added SpanCoreFields, SpanRecordingContext, and StatusCode types
agentlightning/instrumentation/weave.py	Implemented comprehensive InMemoryWeaveTraceServer for testing
tests/*/.py	Updated tests to use new recording tracer patterns and unified interfaces
pyproject.toml	Updated weave dependency version constraint and added setuptools override

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

agentlightning/utils/otel.py

agentlightning/types/tracer.py

agentlightning/tracer/dummy.py

Copilot · 2025-12-14T17:28:16Z

tests/emitter/test_operation.py

+    with pytest.raises(ValueError):
+        complicated(1, "req", 7, 8, 9, kwonly="x", kwdefault="y", tag="value")


The test now expects a ValueError to be raised but doesn't verify the actual result or behavior of the function. The original test verified the output result against expected values. Consider asserting on the specific error message or adding a separate test for the success case to ensure the function still works correctly when valid inputs are provided.

Copilot · 2025-12-14T17:28:16Z

agentlightning/tracer/weave.py

+            try:
+                weave_client.flush()
+                # It's possible that the call end futures are from a dedicated Weave thread pool,
+                await asyncio.gather(*[asyncio.wrap_future(future) for future in self._complete_call_futures])


The asyncio.gather call attempts to wrap futures that may already be from a different event loop or thread pool. If the futures in self._complete_call_futures are from a dedicated Weave thread pool (as mentioned in the comment on line 372), wrapping them with asyncio.wrap_future could fail or cause deadlocks. Consider checking if the futures are already asyncio futures before wrapping, or handling the case where they originate from a different executor.

Suggested change

await asyncio.gather(*[asyncio.wrap_future(future) for future in self._complete_call_futures])

await asyncio.gather(*[

future if isinstance(future, asyncio.Future) else asyncio.wrap_future(future)

for future in self._complete_call_futures

])

I'm not sure about this. Let's test it out later.

Copilot · 2025-12-14T17:28:16Z

tests/emitter/test_operation.py

+    assert op.span().start_time < op.span().end_time  # type: ignore
+    assert op.span().start_time < time.time()  # type: ignore
+    assert op.span().end_time < time.time()  # type: ignore
+    assert op.span().start_time < op.span().end_time  # type: ignore


The assertions on lines 412 and 415 are duplicates - both check that start_time is less than end_time. This appears to be a copy-paste error. Consider removing the duplicate assertion on line 415.

Copilot · 2025-12-14T17:28:17Z

tests/emitter/test_operation.py

+
+
+def test_operation_context_records_exceptions(monkeypatch: pytest.MonkeyPatch) -> None:
+    tracer = _install_recording_tracer(monkeypatch)


Variable tracer is not used.

Copilot · 2025-12-14T17:28:17Z

tests/emitter/test_operation.py

-    assert status.status_code == StatusCode.ERROR
-    assert status.description == "boom"
-    assert use_span.exit_calls[-1][1].args == ("boom",)  # type: ignore
+    recording = tracer.recordings[-1]


This statement is unreachable.

ultmaster · 2025-12-15T03:49:56Z

/ci

github-actions · 2025-12-15T03:50:06Z

🚀 CI Watcher for correlation id-3652764795-mj6m877e triggered by comment 3652764795
🏃‍♀️ Tracking 3 workflow run(s):

🟢 Calc-X - PR #411 - ci-calc-x - id-3652764795-mj6m877e — completed/success
🟢 Spider - PR #411 - ci-spider - id-3652764795-mj6m877e — completed/success
🔴 GPU Test - PR #411 - ci-gpu - id-3652764795-mj6m877e — completed/failure

✅ All runs completed.

ultmaster · 2025-12-15T05:06:08Z

/ci

github-actions · 2025-12-15T05:06:20Z

🚀 CI Watcher for correlation id-3653007475-mj6oy8b8 triggered by comment 3653007475
🏃‍♀️ Tracking 4 workflow run(s):

🟢 Spider - PR #411 - ci-spider - id-3653007475-mj6oy8b8 — completed/success
🟢 APO - PR #411 - ci-apo - id-3653007475-mj6oy8b8 — completed/success
🔴 Calc-X - PR #411 - ci-calc-x - id-3653007475-mj6oy8b8 — completed/failure
🔴 GPU Test - PR #411 - ci-gpu - id-3653007475-mj6oy8b8 — completed/failure

✅ All runs completed.

Copilot

Pull request overview

Copilot reviewed 45 out of 47 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

agentlightning/types/core.py:359

Overridden method signature does not match call, where it is passed an argument named 'store'. Overriding method method OtelTracer.init_worker matches the call.
Overridden method signature does not match call, where it is passed an argument named 'store'. Overriding method method WeaveTracer.init_worker matches the call.
Overridden method signature does not match call, where it is passed an argument named 'store'. Overriding method method WeaveTracer.init_worker matches the call.
Overridden method signature does not match call, where it is passed an argument named 'store'. Overriding method method WeaveTracer.init_worker matches the call.

    def init_worker(self, worker_id: int, *args: Any, **kwargs: Any) -> None:

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-12-15T05:08:41Z

tests/tracer/test_agentops.py

+        with agentops_tracer.operation_context("agentops-op", attributes={"foo": "bar"}) as ctx:
+            raise ValueError("agentops boom")
+
+    recorded_span = ctx.get_recorded_span()  # type: ignore


This statement is unreachable.

Copilot · 2025-12-15T05:08:41Z

tests/tracer/test_dummy.py

+        with tracer.operation_context("dummy-error") as ctx:
+            raise RuntimeError("boom")
+
+    recorded_span = ctx.get_recorded_span()  # type: ignore


This statement is unreachable.

Copilot · 2025-12-15T05:08:41Z

tests/tracer/test_otel.py

+            ctx.record_attributes({"custom": "attr"})
+            raise RuntimeError("otel failure")
+
+    recorded_span = ctx.get_recorded_span()  # type: ignore


This statement is unreachable.

Copilot · 2025-12-15T05:08:42Z

tests/tracer/test_weave.py

+        with tracer.operation_context("weave-op", attributes={"foo": "bar"}) as ctx:
+            raise RuntimeError("failed")

+    recorded_span = ctx.get_recorded_span()  # type: ignore


This statement is unreachable.

ultmaster · 2025-12-15T05:57:53Z

/ci

github-actions · 2025-12-15T05:58:04Z

🚀 CI Watcher for correlation id-3653439454-mj6qsqzi triggered by comment 3653439454
🏃‍♀️ Tracking 6 workflow run(s):

🟢 ChartQA - PR #411 - ci-chartqa - id-3653439454-mj6qsqzi — completed/success
🟢 Spider - PR #411 - ci-spider - id-3653439454-mj6qsqzi — completed/success
🟢 Backward Compatibility - PR #411 - ci-compat - id-3653439454-mj6qsqzi — completed/success
🟢 Calc-X - PR #411 - ci-calc-x - id-3653439454-mj6qsqzi — completed/success
🔴 GPU Test - PR #411 - ci-gpu - id-3653439454-mj6qsqzi — completed/failure
🟢 APO - PR #411 - ci-apo - id-3653439454-mj6qsqzi — completed/success

✅ All runs completed.

ultmaster added 24 commits December 12, 2025 01:10

Fix weave tests

caba665

checkpoint

c407f52

fix trace server

340da88

fix weave

d28c038

checkpoint

33ad2f1

fix weave part

616d191

update otel tests

97002dc

.

298c153

update weave impl

25dc856

.

553b5d2

add weave recording context

0ea2351

update emitter impl

44ff1de

update implementations

a7a17e7

emitter tests passed

b317076

update emitter corner case

ccd113e

fix otel tests

5c7a3a2

fix weave tests

c369700

update write traces

fcca904

debug weave tracer

4193228

update tests

1727ebe

fix typo

ff33393

fix docs

8ffb51e

Merge branch 'main' of github.com:microsoft/agent-lightning into fix/…

6ac7cac

…weave-test

update uv lock

a0d5bd3

Copilot AI review requested due to automatic review settings December 14, 2025 17:25

Copilot started reviewing on behalf of ultmaster December 14, 2025 17:25 View session

Copilot AI reviewed Dec 14, 2025

View reviewed changes

ultmaster added 2 commits December 15, 2025 10:57

resolve comments and lint errors

52812a4

update store openapi and weave init

67eba04

ultmaster added the ci-calc-x label Dec 15, 2025

ultmaster added ci-gpu ci-spider labels Dec 15, 2025

ultmaster added 2 commits December 15, 2025 11:34

debug

30ebefd

revert api.ts

a6c77bf

add more tests

bdbfb5f

ultmaster added the ci-apo label Dec 15, 2025

fix agentops tests

fd90221

ultmaster requested a review from Copilot December 15, 2025 05:06

Copilot started reviewing on behalf of ultmaster December 15, 2025 05:06 View session

Copilot AI reviewed Dec 15, 2025

View reviewed changes

ultmaster added 3 commits December 15, 2025 13:12

remove endline

8b9fbb4

check openapi docs

3d381db

fix legacy

88ec370

ultmaster added ci-compat ci-chartqa labels Dec 15, 2025

ultmaster merged commit 087c7d3 into main Dec 15, 2025
55 of 57 checks passed

		with pytest.raises(ValueError):
		complicated(1, "req", 7, 8, 9, kwonly="x", kwdefault="y", tag="value")



		def test_operation_context_records_exceptions(monkeypatch: pytest.MonkeyPatch) -> None:
		tracer = _install_recording_tracer(monkeypatch)

Reimplement Weave tracer and unify emitter interface #411

Reimplement Weave tracer and unify emitter interface #411

Uh oh!

Conversation

ultmaster commented Dec 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Dec 14, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 14, 2025

Choose a reason for hiding this comment

Uh oh!

ultmaster Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 14, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 14, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 14, 2025

Choose a reason for hiding this comment

Uh oh!

ultmaster commented Dec 15, 2025

Uh oh!

github-actions bot commented Dec 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ultmaster commented Dec 15, 2025

Uh oh!

github-actions bot commented Dec 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

ultmaster commented Dec 15, 2025

Uh oh!

github-actions bot commented Dec 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ultmaster commented Dec 14, 2025 •

edited

Loading

github-actions bot commented Dec 15, 2025 •

edited

Loading

github-actions bot commented Dec 15, 2025 •

edited

Loading

github-actions bot commented Dec 15, 2025 •

edited

Loading