GA launch: comprehensive API, SDK, CLI, and MCP audit and blocker remediation

[jiashuoz]

Objective

Perform a comprehensive GA-readiness audit of e2a's public developer interfaces and prepare focused draft PRs for every confirmed critical launch blocker.

The goal is to verify that the REST API, TypeScript SDK, Python SDK, CLI, and MCP interfaces are correct, internally consistent, documented, and safe to freeze for GA. This is an end-to-end quality assignment: audit the interfaces, reproduce and classify defects, file findings, and prepare fixes for critical blockers.

Start here

Read these before testing:

• GA API/SDK/MCP readiness decisions log
• API reference and GA contract
• Repository README
• Cross-interface PR checklist
• TypeScript and Python SDK READMEs
• CLI and MCP READMEs

Treat the generated OpenAPI 3.1 document and the accepted decisions in the GA readiness log as the source of truth. If they disagree, flag the discrepancy rather than guessing.

Scope

Audit all user-facing behavior in these surfaces:

• REST API: routes, methods, request/response schemas, required and nullable fields, status codes, error codes, authentication and scopes, pagination, filtering, idempotency, rate/quota behavior, WebSocket behavior, and OpenAPI accuracy.
• TypeScript SDK: generated and hand-written layers, method coverage, parameter and return types, error mapping, pagination helpers, webhook verification, WebSocket/listen behavior, and runtime behavior.
• Python SDK: the same coverage as TypeScript, including async behavior and parity with the TypeScript SDK where intended.
• CLI: supported GA commands (login, listen, and config), configuration precedence, hosted/self-hosted behavior, forwarding, error output, exit codes, and graceful handling of invalid credentials or connectivity failures.
• MCP: tool inventory, names/descriptions/input schemas, account- versus agent-scoped auth, tool-to-API mapping, pagination, returned content, validation, error translation, and hosted Streamable HTTP behavior.
• Cross-interface consistency: naming, required fields, enum values, defaults, IDs, pagination, auth semantics, errors, and documented examples must agree across every surface that exposes the capability.

The CLI is intentionally slim and does not need parity with every API operation. MCP also intentionally excludes some operations. Verify intentional exclusions against the GA decisions log; do not add surface area solely for symmetry.

Required testing

1. Establish a clean baseline

• Set up the project from the documented instructions.
• Record the commit SHA, OS/runtime versions, dependency versions, and configuration used.
• Run the full existing test suite and record results before changing code.
• Confirm generated artifacts are current using the repository's generation/freshness checks.

2. Build a coverage matrix

Create a matrix of all GA capabilities and mark whether each is exposed through the API, TypeScript SDK, Python SDK, CLI, and MCP. For every applicable cell, record:

• implementation location;
• documentation/example location;
• automated test coverage;
• manual or end-to-end verification result;
• linked finding, issue, or PR.

Include at least:

• deployment info/version discovery;
• agents and domains;
• API keys and authentication scopes;
• inbound message list/get and attachments;
• send, reply, and forward;
• conversations, labels, and read state;
• webhooks and signature verification;
• reviews/HITL approve, reject, and expiry behavior;
• templates, suppressions, and events;
• account export/deletion;
• pagination and filters;
• rate limits versus quota limits;
• WebSocket connection, reconnect, fatal auth failure, and replacement behavior;
• CLI login/config/listen/forward flows;
• MCP initialization, auth modes, tool discovery, tool invocation, and errors.

3. Test behavior, not only compilation

For each applicable capability, cover:

• happy path;
• missing, malformed, boundary-size, and conflicting inputs;
• unauthenticated, wrong-scope, and cross-account access;
• not-found and conflict cases;
• pagination with more than one page, cursor propagation, empty results, and invalid cursors;
• documented error type/code/status mapping;
• retryable versus non-retryable failures;
• unknown enum/event values where forward compatibility is promised;
• hosted URL defaults and self-hosted base URLs;
• examples copied exactly from current documentation.

Add automated regression tests for every confirmed blocker. Prefer contract or end-to-end tests where an interface mismatch could otherwise compile successfully.

4. Exercise real cross-surface workflows

Run and document complete workflows that combine interfaces, including:

• create/configure through the API, then read and act through both SDKs;
• receive a message and verify it through REST, SDK, and MCP;
• send/reply/forward and verify threading and attachments;
• hold a message for review, list it, then approve/reject it through another supported surface;
• webhook delivery plus TypeScript and Python signature verification;
• WebSocket/listen delivery, reconnect, invalid credentials, and connection replacement;
• CLI forwarding to a local handler;
• MCP over the deployed Streamable HTTP transport, not only direct in-process tool calls.

Use synthetic accounts and non-sensitive test data. Do not run destructive account deletion against a shared or production account.

Finding severity

Classify each confirmed finding:

• Critical / GA blocker: data loss or corruption; auth/security boundary failure; documented core workflow unusable; incompatible public contract across interfaces; release/install failure; crash or infinite retry in a normal flow; or behavior likely to require a breaking post-GA change.
• High: major capability is incorrect or unreliable, but a practical workaround exists and the GA contract need not break.
• Medium: localized functional, consistency, documentation, or usability defect.
• Low: polish, maintainability, or minor documentation issue.

Include impact, exact reproduction steps, expected versus actual behavior, affected interfaces, evidence, suspected source, and severity rationale. Do not label something critical solely because a test fails; connect it to user or launch impact.

Issues and PRs

• Track confirmed findings clearly from this umbrella ticket.
• For each critical blocker, prepare a small, focused draft PR with:
  • the root-cause fix;
  • a regression test that fails before the fix and passes after it;
  • updates to every affected public interface;
  • regenerated OpenAPI/SDK artifacts when required;
  • documentation updates when behavior or examples change;
  • completed cross-interface checklist and a link back to this ticket.
• Separate unrelated blockers into separate PRs. Avoid opportunistic refactors.
• Do not merge or publish releases without maintainer approval.
• Record high/medium/low findings as follow-up issues or a clearly actionable findings section; do not expand blocker PRs to absorb them.

Deliverables

Add a final report to this issue containing:

1. Environment and baseline results.
2. Completed cross-interface coverage matrix.
3. Automated and manual test evidence.
4. Findings grouped by severity.
5. Links to all issues and draft PRs.
6. Remaining known risks, intentional exclusions, and untested areas.
7. A clear recommendation: ready for GA, ready with accepted follow-ups, or not ready, with rationale.

Commit reusable test tooling and regression coverage through the relevant blocker PRs. Do not commit credentials, production data, or throwaway local artifacts.

Acceptance criteria

• Every GA API operation has been checked against OpenAPI, implementation, documentation, and applicable client interfaces.
• TypeScript and Python SDK public methods, error behavior, pagination, webhook verification, and WebSocket behavior have been tested.
• All supported CLI commands and forwarding modes have been tested on documented configuration paths.
• Every MCP tool has been checked for schema, auth, API mapping, output, and error behavior over Streamable HTTP.
• Cross-interface naming, required fields, enums, defaults, pagination, status/error semantics, and examples have been compared.
• Core end-to-end workflows have evidence from a clean environment.
• Every confirmed critical blocker has a focused draft PR with regression coverage and cross-interface updates.
• Non-critical findings are recorded with enough detail to schedule independently.
• Full relevant test suites and generation/freshness checks pass after each blocker fix.
• The final report and GA readiness recommendation are posted on this issue.

[cyj-git-0825]

Re-test supplement — audited/tested on latest main (b43bad44)

Following the audit, I re-ran the full test matrix against the current tip of main (b43bad44 — ~56 commits past the original baseline dd12dc6, incl. the event/webhook/WS contract-freeze, soft-delete/trash, canonical error-contract freeze, async at-least-once outbound, and the Go module rename to github.com/tokencanopy/e2a). Full delta write-up: audits/e2a-ga/retest-latest-b43bad44.md (AgentDrive).

Baseline

• Commit b43bad44; Node 22 / npm 10, Python 3.13, Go via golang:1.26 container (Go not on host PATH); server run via Docker Compose (Postgres 16 + Mailpit).

Suites — all green on latest

Suite	Result
SDK vitest	173 pass
CLI vitest	114 pass
MCP vitest	188 pass
Python pytest	308 pass / 4 skip
Go drift gates (TestSpecGoldenNoDrift, generate-sdk-check)	no real drift (spec + both SDK bases in sync; the only local red is a Windows core.autocrlf CRLF artifact — raw committed blob is LF)
Go webhook/webhookpub/webhookdelivery/ws/eventpayload unit	pass (new contract-freeze tests incl. websocket-close + event-envelope golden fixtures)
Live E2E / WebSocket / webhook fan-out (Dockerized server)	pass (auth 401/200, error envelope + request_id, 422/404, pagination bounds, tamper cursor→400, WS takeover 4000, 401 handshake, tenant isolation, fan-out matched=3, retry via River)

Confirmed critical/blocker findings (re-verified live on b43bad44)

• createApiKey is not idempotent — a retried POST /v1/account/api-keys with the same Idempotency-Key mints a second live credential (verified live: two distinct apk_… ids). No Idempotency-Key param, no runIdempotent, no natural uniqueness. Fix is additive/non-breaking. → I'm preparing a focused draft PR for this (root-cause fix + regression test), per this issue's rules.
• CLI listen --json --forward silently drops the forward (and --once --forward --json double-prints JSON, 0 forwards) — verified live. Already being fixed on branch fix/cli-listen-json-forward-drop, so I'm not duplicating it.

Already fixed since the original baseline (verified)

• MCP version drift — server.json/package.json now both 0.5.0, and buildServer defaults to the real PACKAGE_VERSION (no more 0.1.0 on the initialize handshake).
• Validation errors now carry field locations — live 422 returns details.fields[].location (e.g. body.subject: expected string).
• Soft-delete/restore lifecycle works end-to-end (DELETE …/messages/{id} → 200 {deleted:true}; trashed read → 200; POST …/{id}/restore → 200).

High/Medium/Low follow-ups (recorded as separate issues, not folded into the blocker PR)

• [API] Non-uniform filtering/sorting across collections (listAgents/listDomains/listWebhooks/listTemplates are cursor+limit only).
• [CLI] --once --forward exits 0 on a failed forward POST; top-level error→exit mapping + listen TIMEOUT/premature-close paths lack automated coverage.
• [MCP] Dead agentIds filter type in client.ts (masked by an as cast; latent silent-filter-drop).
• [SDK-Py] No-op TypeGuard narrowing (event.data stays Any); listen() -> Any; HTTP-date Retry-After ignored; possible body/header leak in from_api_exception.
• [Webhooks] Delivery record next_retry_at not advanced as River reschedules (stale for polling clients).
• [Repo/DX] Add .gitattributes eol=lf for api/openapi.yaml + generated SDK dirs (Windows contributors otherwise hit spurious drift-gate failures). CLAUDE.md is stale on the module path — still says github.com/Mnexa-AI/e2a; module is now github.com/tokencanopy/e2a.

Verdict

A− · CONDITIONAL GO. No Critical (data-loss/security-boundary/breaking) blocker found; every runnable suite is green and the contract is internally consistent + drift-gated. The two headline pre-GA items (createApiKey idempotency; CLI listen forward-drop) are both additive/non-breaking to fix — freeze the contract, land those two + the High punch list, then tag v1.

[HongyuanJia8]

Audit update against current main (19934ada): I opened #541 for one concrete GA compatibility-gate gap.

Today a structurally equivalent stable components.schemas rename passes oasdiff, even though it renames public generated SDK models; stable operation tag changes can also move generated methods between classes/modules. #541 freezes those stable SDK names/groupings, keeps beta and account-export interior exemptions, and makes no runtime or wire-contract change.

Validation on the branch is green: spec freshness, policy fixtures, the cumulative freeze anchor (af5d3c7...), and go test ./....

The remaining items I think need an explicit decision or landing before calling /v1 stable are:

• reconcile the approved async-send design with the current contract: the design retires persisted 202 in favor of 200, and also calls for deferred-event/subscription and request-event-enum decisions;
• land or otherwise resolve fix(outbound): guard reply/forward against a not-yet-submitted parent #521 for reply/forward threading when the outbound parent has not yet received a provider message ID;
• decide which runtime-only input constraints should be encoded in OpenAPI before the freeze (webhook fields/filters, RFC3339 query formats, wait values, and message-filter bounds);
• designate the first explicit API GA tag, which then supersedes the temporary commit anchor.

Happy to split the remaining contract decisions into focused follow-ups after maintainers confirm the intended async-send semantics.