feat(cyber): webapp.secret_leak — NPC comms become the agent's objective#365
Open
westonbrown wants to merge 1 commit into
Open
feat(cyber): webapp.secret_leak — NPC comms become the agent's objective#365westonbrown wants to merge 1 commit into
westonbrown wants to merge 1 commit into
Conversation
… objective Answers the "pure substrate" review: makes persona cover-traffic observably change an episode. A persona posts a secret into a shared team-chat board, and the agent recovers it — the first family whose outcome depends on an inhabitant having spoken. - Bridge: personas write to the in-process (attributed) chat store as before, and the runtime now mirrors each message onto a served GET /team/chat the agent can reach over HTTP. Two views of one message: the app surface the agent sees, and the attributed store the grader trusts. The agent stays walled off from the store, so it can't forge a sender. - webapp.secret_leak: success = the agent recovers and submits the secret; an npc_leaked_secret subgoal reads the attributed store (unforgeable) to confirm the leak actually happened in-world. Offered only when the manifest has an NPC. - examples/secret_leak_eval.py: a runnable, manifest-driven tutorial — same world, task, and agent, run once with the NPC speaking and once silent; the secret is recoverable only in the first. No model required. Tests: gating on NPC presence, the chat->served-board bridge end to end, and the grader needing both the leak and the recovery. ruff + mypy --strict + boundary clean; impacted cyber/persona/admit/curriculum suites green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
|
Thank you for your contribution to OpenRange. Before this pull request can be merged, please read the Contributor License Agreement and sign it by posting a comment containing exactly the text below. You only need to do this once. I have read the CLA Document and I hereby sign the CLA You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #364 (
feat/persona-npc). Addresses @larstalian's main review point: as shipped, PersonaAgent was pure substrate — persona comms changed nothing observable in an episode. This makes an NPC's message the agent's objective, on a live episode.What
A persona posts a secret into a shared team-chat board; the agent recovers it over HTTP. The first
cyber_webappfamily whose outcome depends on an inhabitant having spoken.How
GET /team/chatthe agent can curl. Two views of one message — the app surface the agent sees, and the attributed store the grader trusts. The agent stays walled off from the store, so it can't forge a sender.webapp.secret_leak.check_success: success = the agent recovers + submits the secret (same rung shape aswebapp.pentest); annpc_leaked_secretsubgoal reads the attributed store to confirm the leak happened in-world. Offered only when the manifest declares an NPC (no inhabitant → empty board → not a task).examples/secret_leak_eval.py. A runnable, manifest-driven tutorial (the "real usage example" from the review): same world, task, and agent, run once with the NPC speaking and once silent — the secret is recoverable only in the first. No model required.Honest scope / not done
read_the_boardsubgoal (derived fromrequests_made) counts NPC forwarding traffic too; success and thenpc_leaked_secretsubgoal are unaffected.Tests
Gating on NPC presence, the chat→served-board bridge end to end, and the grader needing both the leak and the recovery. ruff + mypy
--strict+ boundary clean; impacted cyber / persona / admit / curriculum suites green (~480 tests).