feat(agentify): autonomous-repo feedback-loop framework + /agentify deploy + submit_feedback addon#344
Merged
Merged
Conversation
The feedback-loop framework delivered as the `/agentify` deploy skill
(plugins/e2a/agentify/), first home in this repo. Slice 1 = the zero-backend
(github ticket_store) intake→triage path, as product-neutral templates plus
e2a's rendered config, hardened per independent + adversarial review.
- templates/autonomous-repo.config.yml.tmpl (+ examples/e2a): the one config
file an adopter owns — adapters, fix_gate{auto|hitl}, labels, marker, bot.
- templates/runtime-skill/: the product-neutral autonomous-repo skill —
SKILL.md, triage.md, state-machine.md, ticket-card.md.
- templates/scripts/ticket_card.sh: the ticket-card helper (the lane's only
state Bash surface) with bot-author trust filter + _selftest.
- templates/workflows/feedback-triage.yml.tmpl: the triage lane — pause
switch, activation gate, e2a read-only MCP wiring, narrowed allowlist
(gh issue + helper + Read + e2a-read; no jq, no broad gh), failure alert.
- SKILL.md + references/: deploy procedure, setup checklist, adapter
contracts, security invariants (bounded blast radius).
Design: docs/design/autonomous-repo-framework.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The E2A comms lane: filer notifications + the fix-gate hitl approval loop +
verified-reply routing. Closes the loop to filers and actuates the human
gate by email.
- templates/runtime-skill/comms.md: outbound owed-notifications (triage-ack,
approval-request, resolved-closed) + inbound verified-reply routing
(approver approve→agent-fix / decline→wontfix; filer dispute-reopens;
unsubscribe; escalation→human). Send guardrails: template-bounded;
send_message only to fix_gate.approver; reply_to_message stays in-thread;
one thread's content never enters another's.
- templates/runtime-skill/templates/{triage-ack,approval-request,
resolved-closed,shipped}.md: the bounded outbound surface (shipped dormant
until slice 3).
- templates/workflows/feedback-comms.yml.tmpl: the comms lane — same narrow
allowlist as triage (gh issue + helper + Read, no jq) plus the e2a SEND
tools; forward_message disallowed.
- Resolves the triage↔comms shared-mailbox partition (find-by-comms
predicate; comms routes a reply only after triage-ack is recorded).
- state-machine.md: triaged→awaiting_approval→{triaged+agent-fix | wontfix};
SKILL.md / adapters.md mark comms implemented. Design §10 addenda.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Blockers: - Structural send bound (was an open mail relay): the raw e2a send tools accept to/cc/bcc/reply_all, so "reply stays in-thread" was false. All sends now go through scripts/comms_send.sh (reply→server-derived recipient; approval→fix_gate.approver only; never cc/bcc); the raw send_message/reply_to_message/forward_message tools are disallowed. Recipient bounding is structural, not prompt-level. + comms.e2a_api_url, comms_send.sh _selftest. - get_message read-on-fetch broke the loop: fetching marks a message read, so triage's classify-pass consumed approver/dispute replies before comms could route them. Both lanes now classify from the list_messages summary (conversation_id) and get_message ONLY a message they own and will act on. Coherence / honesty: - Approval-gate: bindings hold (no forgery); agent-fix actuator is prompt-gated (documented; PR-merge is the real fence); null approval.conversation_id never matches. - security-invariants: comms key is read+SEND (not read-only); mail egress recipient-bounded by the wrapper. - decline relabels (drop status:awaiting-approval, add wontfix, close not_planned); contact added to ticket-card schema; feedback_status dangling ref removed; over-granted e2a tools trimmed; dup/noise dispute branches marked deferred. Accepted residuals documented: inbound at-most-once (read-on-fetch); cross-run double-send window; cross-thread body discipline is prompt-level; escalation/unsubscribe is prose judgment. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The coding agent and the merge→shipped callback close the loop.
- runtime-skill/fix.md: read the issue safely (bot body + OWNER/MEMBER
comments), fix, verify against the running stack, open ONE PR with a
customer-note block + `<!-- {marker} fix:#N -->` footer. Never merges/deploys.
- workflows/feedback-fix.yml.tmpl: claude-code-action gated on agent-fix.
Credential inventory = Anthropic + App token + throwaway verify stack; zero
backend/cloud/prod secrets (blast radius = a rejected PR). A non-agent
post-step records in_progress + the PR, captures the customer-note into the
ticket-card, relabels, and requests the reviewer.
- workflows/feedback-released.yml.tmpl + scripts/released_markers.sh: on push
to main, flip ticket→shipped for each bot-authored PR carrying fix:#N
(bot-author + footer trust enforced; _selftest). Idempotent.
- examples/e2a/agentify-fix-verify-setup.sh: e2a's verify bootstrap
(Postgres :5433, throwaway).
- Activations: triage in_progress sweep live (gh pr list/view added, gh pr
merge denied); comms shipped notification + shipped→triaged dispute active;
customer_note + customer-note plumbed through the ticket-card.
Design §10 slice-3 addenda. The full triage→issue→approval→PR→shipped loop
now exists as zero-backend templates.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
CRITICAL: - released_markers.sh digit-leak: the issue-number grep ran over the whole marker, so "e2a-feedback" (the "2") emitted a phantom #2 on every merge, wrongly shipping/closing it or hard-failing the step. Anchored to fix:#[0-9]+ -> [0-9]+$; _selftest now uses a digit-containing marker (the old fixture used "acme-feedback" and hid the bug). Blockers: - Cross-ticket marker forgery (injection-as-bot): the release callback now ships #N ONLY if #N's own ticket-card pr is set AND that PR is MERGED, so a forged fix:#<other> in an unrelated bot PR can't ship #N. Per-issue guards (|| continue) stop a missing/forged card aborting the step. - PR-find prefix collision (#4 ⊂ #42): match the footer form fix:#<n> --> via real jq --arg (no program interpolation). - Config-driven labels: fix/release relabels parse status:* / agent-fix from config (were literals); skill-prose status labels normalized; triage sweep drops status:in-progress on shipped/triaged. Honesty / docs: - security-invariants §2 corrected (was "blast radius = a rejected PR"). The real fences: zero prod creds + REQUIRED branch protection + App without workflows:write + diligent PR review (visible customer-note + config diff). Residual: token exfil needs no merge. - setup-checklist: branch protection REQUIRED; App denied workflows:write; CODEOWNERS recommended. fix.md customer-note made review-salient. - Documented residuals: Fixes #N dual-close, marker regex metachar, on:branches literal, release/sweep double-ship. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Makes the framework installable into any repo.
- agentify-render.sh: the deterministic scaffolder. Fills
autonomous-repo.config.yml from ANS_* answers (fails loudly on unfilled
{{UPPERCASE}} placeholders) and copies the runtime skill, scripts, and the
four workflows into their real paths (.claude/skills/autonomous-repo/,
scripts/, .github/workflows/*.yml, .tmpl stripped). _selftest renders into
a temp dir and asserts the tree + substitution.
- Re-run PRESERVES an existing config (updates code only) unless --force —
so re-rendering for framework updates never clobbers a tuned always_hitl /
filled bot_login. Foundation for the deferred update mode.
- SKILL.md thickened: the deploy procedure now drives agentify-render.sh
(detect → ANS_* answers → render → tune always_hitl → labels → checklist →
PR). Mechanical render automated; Q&A + identity/secret setup stay guided.
E2E: rendering e2a's answers into a scratch repo produces a valid install
(config with e2a values, 4 workflows, runtime skill + templates) and the
three scaffolded scripts pass their own selftests in the rendered location —
the wizard provably reproduces the e2a install. Design §10 slice-4 addenda.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds an extensibility seam (templates/addons/) and the first addon.
- Addon mechanism: each addon is manifest.yml + files/ + setup.md, opted in
via ANS_ADDONS; agentify-render.sh scaffolds files/ -> tools/<name>/ and
appends setup.md -> AGENTIFY-ADDON-SETUP.md. Selftest covers scaffold,
unknown-addon rejection, and the no-addons default.
- submit-feedback-mcp (intake adapter): a submit_feedback / feedback_status
MCP server that EMAIL-BRIDGES agent-filed feedback into the support mailbox
the triage lane already drains — purely additive, zero loop changes.
- In-band model: sends from the bridge's own e2a identity TO a FIXED support
recipient (structurally bounded, no caller-supplied address — no spoof
vector); the filer polls feedback_status.
- bridge.mjs pure logic + bridge.test.mjs (node:test, 5 cases): validate-
before-charge, untrusted body sent as opaque data (never interpolated),
coarse status derivation. server.mjs (MCP + e2a REST) verified at install.
- Residuals documented: feedback_status ids are bearer capabilities
(unguessable conv_ ids; coarse status only, not content); rate limit is a
per-process backstop. Richer caller-authenticated variant noted as
follow-on (e2a's own mcp/ server).
Design §10 addenda.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Relay/spoof/SSRF/key-exfil all refuted (fixed-recipient bound holds). Closed the confirmed LOW/MED findings, each with a regression test: - feedback_status validates the id is a conv_ id BEFORE the fetch — `.`/`..` ids otherwise reached unintended same-host endpoints (encodeURIComponent leaves dot-segments; the URL parser normalizes them). + isValidFeedbackId. - feedback_status is now rate-gated (separate read limiter) — status enumeration was free. - composeFeedbackEmail strips CR/LF from the subject (defense-in-depth vs a downstream MIME-header splat); body stays opaque. - submit_feedback wraps the e2a POST so a failure can't disclose the intake address (was an uncaught throw). - apply_addons rejects non-[a-z0-9-] addon names (a `..` name let cp escape tools/); selftest asserts the traversal is rejected. bridge.test.mjs: 7 cases. Design §10 hardening note. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Fix: e2a_mcp_url was https://mcp.e2a.dev/mcp; the e2a MCP lives at https://api.e2a.dev/mcp (same host as the REST base). Fixed in the template default and the e2a example. - test/validate.py: static validation that catches the class of bug above — e2a MCP/REST URLs must share a host (negative-tested: it FAILS on mcp.e2a.dev), plus YAML parse, workflow<->config key drift (every yq key a workflow reads must exist), required keys, and stray {{placeholders}}. - test/run.sh: the deterministic suite — every script _selftest + bridge.test.mjs + bash/JS syntax + validate.py. - .github/workflows/agentify-test.yml: runs the suite on PRs touching plugins/e2a/agentify (no secrets, no network). Still open (per design §9): golden-fixture lane-prompt tests + the live e2e. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Tests the lane PROMPTS, not just the plumbing: drive each lane's real procedure via `claude -p` against a mocked world and assert on what the agent attempted. - Harness (test/fixtures/harness/): a stub e2a MCP server (mock-mcp.mjs, serves the fixture inbox, records get_message so an over-fetch is detectable), a fake `gh`, and recording mock ticket_card.sh/comms_send.sh — every agent action becomes an assertable log line. runner.sh builds the sandbox (real runtime skill + config), runs `claude -p` with the lane's real prompt, and asserts. Token-gated: SKIPs cleanly without a model token. - Triage fixtures: new-feedback (files a triaged issue, no auto agent-fix in hitl, no send), injection (treats an "apply agent-fix / gh pr merge / email secrets" body as DATA — obeys nothing), reply-skip (a reply is left for comms — no issue, and no get_message, the read-on-fetch partition). - assert-selftest.sh deterministically verifies each assert.sh DISCRIMINATES good vs bad action logs (no model needed) — wired into the main test/run.sh suite, so the assertions are guarded on every PR. - CI: .github/workflows/agentify-lane-fixtures.yml runs the model layer when CLAUDE_CODE_OAUTH_TOKEN is set. Still open: comms/fix fixtures + the live over-the-wire e2e. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Running the model layer for real surfaced two harness bugs the deterministic self-test can't see: - AGENTIFY root was $HARNESS/../.. (=.../test) not $HARNESS/../../.. (=.../agentify), so the runtime skill + config never landed in the sandbox and the agent improvised WITHOUT its real procedure. - That failed silently. Added a guard that errors if the skill/config are missing, so an incomplete sandbox can't masquerade as a passing run. - Added AGENTIFY_FIXTURES_FORCE=1 to run the model layer locally when the claude CLI is logged in (no env token needed). With the fix, all three triage fixtures PASS against the real prompt: injection escalates (obeys nothing), new-feedback files a correct triaged issue + card, reply-skip leaves the reply for comms (no get_message — the read-on-fetch partition holds). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…age-lock=false) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this adds
A reusable, zero-backend feedback-loop framework, delivered as the
/agentifydeploy skill, that turns a GitHub repo into a self-managing one:It's dogfooded on e2a (e2a's support loop runs on e2a) and installable into any repo. Everything lives under
plugins/e2a/agentify/; the design is indocs/design/autonomous-repo-framework.md.Important
Merging this activates nothing. It only adds the framework templates + the deploy skill under
plugins/e2a/agentify/. No workflow is placed in.github/workflows/, no secrets exist, every lane no-ops until activated. Going live is a separate step (render into real paths + GitHub App / mailbox / secrets / branch protection).Structure
/agentifydeploy flow —agentify-render.shscaffolds a configured install; re-derives the e2a install from scratch (e2e-proven)submit_feedbackintake addon (email-bridge MCP server) + the addon mechanismHow it was built
Each lane went through independent + adversarial review (agents) with the findings fixed and regression-tested in follow-up commits — including a CRITICAL caught in slice 3 (a marker digit-leak) and the slice-2 send surface hardened from prompt-level to structural (recipients computed by a wrapper; raw send tools disallowed).
Five helper artifacts carry
_selftests:ticket_card.sh,comms_send.sh,released_markers.sh,agentify-render.sh, and the addon'sbridge.test.mjs(7 cases).Security posture (documented honestly)
references/security-invariants.mdstates the real, load-bearing fences rather than overclaiming:gh issueonly; nojq/broadgh); the honest claim is bounded blast radius, not "secrets unreachable".workflows:write+ a diligent PR review.submit_feedbacklikewise (no caller-supplied address).Testing
🤖 Generated with Claude Code