Skip to content

feat(cyber): webapp.secret_leak — NPC comms become the agent's objective#365

Open
westonbrown wants to merge 1 commit into
feat/persona-npcfrom
feat/persona-secret-leak
Open

feat(cyber): webapp.secret_leak — NPC comms become the agent's objective#365
westonbrown wants to merge 1 commit into
feat/persona-npcfrom
feat/persona-secret-leak

Conversation

@westonbrown

Copy link
Copy Markdown
Contributor

Stacked on #364 (feat/persona-npc). Addresses @larstalian's main review point: as shipped, PersonaAgent was pure substrate — persona comms changed nothing observable in an episode. This makes an NPC's message the agent's objective, on a live episode.

What

A persona posts a secret into a shared team-chat board; the agent recovers it over HTTP. The first cyber_webapp family whose outcome depends on an inhabitant having spoken.

How

  • Bridge (the impl gap the review predicted). Personas still write to the in-process, attributed chat store; the runtime now also mirrors each message onto a served GET /team/chat the agent can curl. Two views of one message — the app surface the agent sees, and the attributed store the grader trusts. The agent stays walled off from the store, so it can't forge a sender.
  • webapp.secret_leak. check_success: success = the agent recovers + submits the secret (same rung shape as webapp.pentest); an npc_leaked_secret subgoal reads the attributed store to confirm the leak happened in-world. Offered only when the manifest declares an NPC (no inhabitant → empty board → not a task).
  • examples/secret_leak_eval.py. A runnable, manifest-driven tutorial (the "real usage example" from the review): same world, task, and agent, run once with the NPC speaking and once silent — the secret is recoverable only in the first. No model required.

Honest scope / not done

  • The leaking persona is manifest-declared with the secret (the tutorial does this). Auto-injecting a graph-generated secret into a persona's config is a further seam, not built here.
  • Per-actor HTTP attribution is still deferred, so the coarse read_the_board subgoal (derived from requests_made) counts NPC forwarding traffic too; success and the npc_leaked_secret subgoal are unaffected.

Tests

Gating on NPC presence, the chat→served-board bridge end to end, and the grader needing both the leak and the recovery. ruff + mypy --strict + boundary clean; impacted cyber / persona / admit / curriculum suites green (~480 tests).

… objective

Answers the "pure substrate" review: makes persona cover-traffic observably
change an episode. A persona posts a secret into a shared team-chat board, and
the agent recovers it — the first family whose outcome depends on an inhabitant
having spoken.

- Bridge: personas write to the in-process (attributed) chat store as before, and
  the runtime now mirrors each message onto a served GET /team/chat the agent can
  reach over HTTP. Two views of one message: the app surface the agent sees, and
  the attributed store the grader trusts. The agent stays walled off from the
  store, so it can't forge a sender.
- webapp.secret_leak: success = the agent recovers and submits the secret; an
  npc_leaked_secret subgoal reads the attributed store (unforgeable) to confirm
  the leak actually happened in-world. Offered only when the manifest has an NPC.
- examples/secret_leak_eval.py: a runnable, manifest-driven tutorial — same world,
  task, and agent, run once with the NPC speaking and once silent; the secret is
  recoverable only in the first. No model required.

Tests: gating on NPC presence, the chat->served-board bridge end to end, and the
grader needing both the leak and the recovery. ruff + mypy --strict + boundary
clean; impacted cyber/persona/admit/curriculum suites green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Thank you for your contribution to OpenRange. Before this pull request can be merged, please read the Contributor License Agreement and sign it by posting a comment containing exactly the text below. You only need to do this once.


I have read the CLA Document and I hereby sign the CLA


You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant