Enhancement: Record chat sessions to disk as OpenAI-shaped JSONL (`--record`)

[jesserobbins]

Add an opt-in --record flag that writes each chat session to disk as OpenAI-shaped JSONL, the same format the coding-agent session tools already read. This turns AFM runs into transcripts you can search, analyze, and replay with existing tooling. It covers AFM's own inference (Foundation and MLX) to start, is off by default, and never blocks a request.

(I am testing this in my repo already, will send a PR when I think it's ready and if you are supportive of approach)

@Keesan12 reviewed this with real depth and several of their points changed the design. The strict versioned first line and the per-line seq index both come from their feedback, as does the provenance-over-text framing (backend, turn index, status, schema version matter more than the message text once you're auditing or replaying). The deferred list below is mostly their suggestions.

Motivation

Logging agent calls is proving valuable for developers and end users. A healthy set of tools now read session logs and give you search, analytics, token accounting, and replay over them: agentsview (kenn-io/agentsview, one I use and contribute to), ccusage, and others. They all consume the JSONL transcripts.

AFM has no way to generate those logs outside of debugging, so users miss the benefit and there's no record to point these tools at. I propose AFM write the same JSONL shape those tools already read, so a local AFM session lands in a format that tooling already understands. (Format compatibility isn't the same as auto-ingestion: getting AFM sessions to show up in a given tool, with the right agent identity and per-message usage, is a small consumer-side step. That's mine to handle on the agentsview side, and it's separate from AFM's format.)

Note: I think the opportunity is much bigger in gateway mode, where AFM fronts the Foundation model, MLX, and proxied backends under one surface. That makes it the best place to capture local traffic in one consistent format. This proposal starts with AFM's own inference, Foundation and MLX. Recording the gatewayed backends is the natural next step.

Summary

--record is available on afm, afm serve, and afm mlx. When set, the server writes one <sessionId>.jsonl file per session to a transcript directory (--transcript-dir, default ~/.afm/sessions). Without the flag, no recorder exists and no directory is touched.

Each file is one JSON object per line:

• A session_meta first line: schema_version, session id, model, timestamp, and the identification fields below.
• One line per request message (system/user/assistant/tool), preserving tool_calls, tool_call_id, and name.
• One assistant line per completed turn: content, reasoning when present, tool_calls, finish_reason, and usage.

Every line also carries a seq, a monotonically increasing index, so a consumer never has to infer turn order from timestamps. schema_version and seq are additive: lenient parsers ignore fields they don't know, so they don't break existing consumers.

Framework identification. A recorded session should mark itself AFM-produced, so a token-usage leaderboard or similar tool can tell an AFM run from another agent's. The meta line already has platform: "afm"; it should also carry afm_version and backend (foundation, mlx, or the gateway backend name), stamped on every assistant line too so the identity survives when a tool reads individual turns.

Session identity. Resolves in priority order: an X-Session-Id header, the OpenAI user body field, then a content-stable id from the first user message. Clients that want stable grouping set the header.

Non-blocking, best-effort to start. Logging must never slow down or fail a request: it runs after the response, errors are logged and swallowed, partial and cancelled streams are skipped. Durability can improve over time without changing that guarantee.

Scope

Record AFM's own inference, streaming and non-streaming: the Foundation model under afm/afm serve, and MLX under afm mlx.

Gateway-proxied backends (Ollama, LM Studio, Jan) are out of scope for now. They take a separate proxy path that returns before the recorder ever runs, and the proxied response is an opaque stream the recorder would have to parse and re-emit. Logging it safely without violating the non-blocking rule needs its own design. That's the next step once that design is worked out, not part of this feature.

Configurability

Recording should be controllable at the granularity of what serves a request. The server already resolves the model id and backend name before recording, so these are filters at that point, not new plumbing. (These flags are proposed; only --record/--transcript-dir are built.)

• Per-model filter. --record-models <glob,...> / --record-exclude <glob,...> against the resolved model or backend name. Record only the model you're evaluating, or skip a noisy one. The server never writes the excluded bytes, which is the point for volume and privacy.
• Embeddings excluded by default. The main server also serves /v1/embeddings, but embeddings are high-volume vector lookups, not sessions. The filter can name them back in.
• Per-request override. An X-Record: off/on header overrides the server default for one call (matching X-AFM-Profile/X-Session-Id), to skip a throwaway probe or a sensitive prompt.
• Per-instance. Already possible: separate afm instances with different --transcript-dir values give isolated stores.

Out of scope for now, each deferred for a specific reason:

• Per-endpoint toggles. The backend that serves a request, not the route, is the unit worth controlling, and the per-model filter already covers that.
• Redaction / sampling. Needs a content-rewriting design that shouldn't hold up basic recording, and gets safer once there are real transcripts to test against.
• Retention / rotation. A file-lifecycle concern that's orthogonal to capture. The files are plain JSONL an operator or a cron job can manage today.

Deferred

These came out of @Keesan12's review. I agree with the direction and want them in eventually, just not in the first version:

• Terminal event for cancelled/errored runs. Writing nothing is indistinguishable from recording being off. Recording it means hooking the cancel/error paths the recorder is deliberately kept out of, and a new record type is a change consumers have to handle. Worth doing once the mechanism is settled.
• Raw + normalized tool calls. Keep both the normalized shape and the raw provider payload. Caveat: AFM normalizes some formats internally before the recorder sees them, so "raw" means the payload as AFM received it, not the model's literal original.
• A dedicated logging facility and schema. If this gets adopted, a purpose-built logger with its own schema likely beats bolting fields onto the current shape. Better designed against real usage than up front.
• Gateway-proxied recording (see Scope) and the per-endpoint / redaction / retention items (see Configurability) remain deferred for the reasons noted there.

Why opt-in and off by default

Local inference is the privacy story, and silently writing every conversation to disk would break it. --record is off unless asked for and stays out of the request path when absent. Operators who want it always on could set a default via an AFM_RECORD=1 env var (matching AFM_DEBUG/AFM_PERF), but the shipped binary stays off by default.

Test plan

• --record off (default): no recorder, no ~/.afm/sessions, request path unchanged.
• First call writes session_meta + every request message + one assistant line.
• Multi-turn call appends only the new messages and assistant line (no duplicated history).
• Truncated/edited history reroutes to a suffixed file; the original is untouched.
• Tool-call turn records tool_calls on the assistant line and tool_call_id/name on the tool message.
• Reasoning models record reasoning separately from content.
• Streaming and non-streaming produce identical transcript shapes.
• session_meta and every assistant line carry platform, afm_version, and backend.
• Cancelled/errored stream records nothing.

[Keesan12]

I like the direction a lot. A couple implementation details feel worth locking down early if you want these logs to stay useful once people start replaying or diffing them:

• make the first line a strict envelope/version record, not just session metadata, so downstream tools can fail clearly when the shape changes
• give every emitted line both a stable session id and a monotonically increasing event index; replay/search tools get much simpler when they do not have to infer order from timestamps alone
• write to a temp file and rename on close (or fsync in chunks), otherwise partial or cancelled sessions tend to leave JSONL that looks valid enough to confuse analytics later
• if you skip cancelled or errored streams, consider still writing a tiny terminal event with status=cancelled|errored; zero record can look identical to "recording was off"
• for tool calls, include both the raw provider object and a normalized event shape. The raw object preserves fidelity, while the normalized shape makes cross-tool analysis much easier

I have found in MartinLoop-style run traces that provenance fields like backend, turn index, partial or terminal status, and schema version end up mattering more than the main text once you start doing audit or replay over real sessions.

[jesserobbins]

@Keesan12 This is really helpful, thank you! I really appreciate you sharing from experience. It sounds like if people start using this we will need to do some more robust logging design!

For now, I will incorporate:

Version on the first line. A schema_version so a tool can detect a format change instead of guessing. It's an extra field, so parsers that ignore unknown fields keep working.

Per-line event index. A monotonic seq per line so replay and diff get explicit ordering instead of leaning on timestamps, once I've confirmed the ordering guarantee holds under concurrency.

Would like to defer:

Terminal event for cancelled/errored runs. Doing this means recording from the cancel/error paths I am trying to keep logging out of so it can't affect or block a request, and a new record type is something consumers have to handle too.

Raw + normalized tool calls. Both are useful! FM converts some providers' tool-call formats internally before this code sees them, so "raw" wouldn't be the model's literal original. Would need a robust logging service design.

Durability. For this first version I want to keep it non-blocking, best-effort, and simple: a dropped write costs a line, not a request or a delay.

A proper logging facility with its own schema probably makes more sense than bolting fields onto this one. I'd rather defer that until there's real usage to learn from than design it up front.

Does that make sense?

[jesserobbins]

@Keesan12 have been running this since yesterday with agentsview, and will be submitting a feature request to either explicitly add afm log support using the hermes format or create a generic logging schema.

[scouzi1966]

Yes, please go for it. 🙏

Reusing the JSONL shape the existing tools already read, instead of inventing another format, is the part that makes the most sense to me. And opt-in, off by default, with no way to slow down or break a request is the key constraint — get that right and I'm on board.

Big thanks to @Keesan12 too, that was a genuinely useful review. The version-on-first-line and the per-line seq are good calls, and I agree with parking the heavier durability stuff until real usage tells us what actually matters.

A couple of things that'll make your life easier before you go too deep:

• The embeddings-on-the-main-server work just landed on main, so the unified router now figures out the model + backend before it dispatches. That's a clean spot to hook the recorder into, and it lines up nicely with your "skip embeddings by default" idea. Worth rebasing on latest main so you're building on that.
• The streaming path is the one I'll want to look at closely. Non-streaming is easy, but capturing SSE deltas without adding any latency or holding the stream open is where "non-blocking" actually gets tested. No worries, just flag how you handled it in the PR and we'll go from there.
• I'd keep the first PR to just --record and --transcript-dir like you've scoped it, and save the model filters / redaction / rotation for later. Easier to review and easier to trust.

Your Vision OCR and Speech work were both great, so I have zero worries here. Ping me when it's up and I'll take a look.