feat(sdk): generic PersonaAgent NPC — pack-agnostic tools, comms, scoped memory#364
feat(sdk): generic PersonaAgent NPC — pack-agnostic tools, comms, scoped memory#364westonbrown wants to merge 12 commits into
Conversation
…ped memory
One reusable persona-driven AgentNPC that binds per-pack by config: it wraps
whatever callables a pack surfaces into the per-tick interface, so the same
class serves web/network/terminal/enterprise/social packs with no per-domain
NPC code and no model training.
- comms.py: MailboxStore/ChatStore + identity-neutral surface helpers. NPCs
communicate through the world (a shared store), never a direct handle. The
runtime hands one frozen surface to every NPC, so sender is injected NPC-side
(the persona's actor_id) and omitted from the model-facing signature — cover
traffic is attributable by construction and a persona cannot forge identity.
- memory.py: ScopedMemory protocol + dependency-free DictMemory, scoped per
{run}:{actor} (scope injected, not model-chosen) so personas can't read each
other's or the SUT's recall.
- persona_agent.py: render_persona (persona data -> system prompt with an
anti-assistant CONSTRAINTS block + independent goal); data-driven _build_tools
with typed adapters for mail/chat/speak and signature-faithful, single-call
wrapping for any other affordance (multi-arg supported, no double-fire);
incoming mail/chat read with a per-NPC cursor and injected into the prompt;
optional scoped long-term memory tools.
- cyber_webapp wires it end-to-end: surface_extras exposes the shared comms
callables, collect_extras drains them attributed by sender for grading, and
reset_episode clears them so a warm-pooled world doesn't leak traffic.
Register per pack with one openrange.npcs entry point (cyber.persona).
Tests: PersonaAgent unit + integration — shared-surface attribution through the
real seam, read cursor, per-persona/per-run memory isolation, signature-faithful
multi-arg tools, manifest/registry resolution, cyber surface+collect wiring, and
scale. ruff + mypy --strict + boundary clean; full suite green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Thank you for your contribution to OpenRange. Before this pull request can be merged, please read the Contributor License Agreement and sign it by posting a comment containing exactly the text below. You only need to do this once. I have read the CLA Document and I hereby sign the CLA You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot. |
…verage Addresses a second adversarial review of the PersonaAgent NPC. Merge-blocker: - ci.yml ran `pytest tests`, overriding pyproject `testpaths`, so the pack-SDK suite (incl. the persona tests) never gated CI. Drop the positional path so testpaths is honored. Correctness: - Buffer incoming comms and clear it only after a successful tick, so a failed LLM call never drops a message; cap the buffer so it can't grow unbounded. - Deterministically phase-stagger cadence (crc32(scope) % cadence) so a population on one cadence doesn't act in lockstep. - Skip a persona's own posts when reading mail/chat; guard malformed message ids so one bad row can't brick a reader. - Sanitize strands-reserved tool-param names (self/cls/agent); punctuation-aware memory tokenizer. Believability (no training): - render_persona gains social grounding (contacts/channels/example_line) + a "plain and human" constraint; article agreement + the "You You" fix. - Diegetic per-tick prompt (no fourth-wall "it's your turn / use a tool"). Docs/examples/tests: - Move the demo to examples/persona_eval.py (ruff-clean; corrected scale claim). - Add a pack-author guide (config-key table + local-model note) to start_here. - New regression tests for each fix + a PersonaAgent-over-real-WebappRuntime integration test asserting attributed collect_extras. ruff + mypy --strict + boundary clean; full suite green (one pre-existing local docker image-sweep flake, unrelated). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Second-pass review refinements (
|
CI does not install the optional `strands` extra, which surfaced two issues the
local (strands-present) run masked:
- Two `tool_spec`-asserting tests now `importorskip("strands")` — without the
extra, `_as_tool` returns the plain callable (by design), so those schema
assertions only apply when strands is installed. The reserved-name test keeps
its strands-free assertion (the raw tool stays usable) and gates only the
schema check.
- Unnamed personas now get a monotonic `persona-N` actor id instead of the base
class's 16-bit-masked id() fallback, which could (and in CI did) collide and
share a memory scope. Fixes a flaky isolation assertion and the underlying
defect.
Verified under a simulated no-strands import: 76 passed, 2 skipped, 0 failed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
westonbrown
force-pushed
the
feat/persona-npc
branch
from
a42b59c to
8ae44d9
Compare
A dedicated simplicity/YAGNI pass — pure subtractions, no correctness or security property touched, all tests green. - Drop the `ScopedMemory` Protocol + `runtime_checkable` + dead `scopes()`; the memory feature keeps its one `DictMemory` impl (typed directly). Removes a one-impl "seam for later" abstraction. - Fold `behavior_axes` into `traits` (two overlapping style channels → one); `_style_directives` collapses to a few lines. - Delete the `speak` comms adapter — no pack surfaces it, and it falls through to `_wrap_action` unchanged. - Drop unused prompt knobs `contacts`/`example_line` (kept `channels`, which is functional). - Remove `Message`/`ScopedMemory` from the public SDK surface (no importers). - Replace the `_MAX_PENDING` overflow breadcrumb with a plain tail slice. Shipped source ~642 -> ~595 LOC, a leaner public API and config schema. Kept the load-bearing complexity (single-invoke tool wrap, actor_id injection, the failed-tick comms buffer, read cursors, cadence stagger, anon-id isolation). ruff + mypy --strict + boundary clean; full suite green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Simplicity pass (
|
Refresh the pinned floor to the current Strands release (Jun 2026); picks up Ollama correctness fixes (unique toolUseId, latencyMs type), context-window lookup, and OpenAI structured-output fixes. Lock diff is scoped to the two strands packages only. No API changes needed: our usage (`tool(name=...)` + reconstructed schema, `Agent(..., callback_handler=None)`, `StrandsAgentBackend`) is unchanged and still current on 1.45; the sole breaking change in the window (Mistral requiring mistralai>=2.0.0) does not touch the Bedrock/Ollama path. ruff + mypy --strict + boundary clean; full suite green on 1.45. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Dependency bump (
|
…sona
A population built with count=n today is n clones sharing everything but an id.
This adds diverse-by-construction personas with no model and no training:
- Numeric trait intensities: `traits: {"curious": 0.9, "patient": 0.3}` renders
"very curious, a little patient". Bool flags still render plain (backward
compatible), so a sampled trait vector reads with variety.
- `sample_persona(seed, roles=[...])`: deterministically samples a distinct
name, role, and numeric trait vector. `[sample_persona(i, roles=...) for i in
range(n)]` gives a varied population instead of clones; the pack supplies its
own role/name/trait vocabulary so it stays domain-neutral.
Addresses the persona-homogeneity gap flagged by recent NPC-diversity work.
Tests: numeric rendering, determinism, population diversity, and a sampled
config driving a real PersonaAgent. ruff + mypy --strict + boundary clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Fixes verified by running the persona through real runtimes: - `_wrap_action` no longer raises on a keyword-only-after-defaulted signature (`(a="x", *, b)`) — synthesized params are sorted required-first so inspect.Signature is legal (forwarding is by name, so order is schema-only). - `DictMemory` caps note length + per-scope count — the only unbounded path into a prompt, so a long-lived long_term_memory persona can't blow up the context. - cyber_webapp bounds the `collect_extras` NPC-comms dump to the last N, so a chatty episode can't bloat graded state (and the dashboard turn that embeds it). - `sample_persona` suffixes the seed onto sampled names (a small `names` pool no longer collides actor_ids / memory scopes) and guards `num_traits < 0`. - Duplicate declared `tools` are deduped; a declared-but-unsurfaced tool warns once; article-agreement handles an empty role. Tests: keyword-only affordance, small-names uniqueness, negative num_traits, tool dedup, missing-tool warning, bounded memory, presence event, chat_post identity binding. ruff + mypy --strict + boundary clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Pre-existing runtime bugs surfaced while stress-testing the persona NPC; kept separate from the feature so they can be reviewed/split independently. - HIGH: the auto-tick daemon could crash. `_auto_tick_loop` re-read `running.tick_stop` each iteration while `_stop_auto_tick` nulled it after a 5 s join timeout → `None.wait` AttributeError, and stop/grade proceeded under a live tick. Bind the Event once in the loop; don't null it (or the thread) while the daemon is still alive. - `_start_npcs` dedups actor_ids — two NPCs sharing an id would share dashboard rows and any id-scoped state; the duplicate is skipped and recorded. - `resolve_manifest_npcs` no longer lets one bad NPC slot kill the whole episode: a per-slot construction error warns + skips, while an unknown type / bad factory contract (NPCError) still raises loudly. Tests (tests/test_persona_episode.py): auto-tick stop keeps the Event under a live daemon, duplicate-named personas seat once, one bad slot is skipped while an unknown type raises. ruff + mypy --strict + boundary clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Robustness / scale / arch hardening (
|
Makes the feature ready to merge along the two axes it was missing: UI
integration and measurable properties.
UI: a seated persona was mute — it showed up on the dashboard (topology seats by
name) but emitted nothing per tick. Now, when a persona uses a comms tool, it
mirrors the message text to the dashboard (`{actor_kind:"npc", speak, ...}`,
the same event shape office_persona emits), so it animates with a speech bubble
and activity line. The chat/mail text IS the model's speech (the prompt forbids
tool-narration), so no separate utterance channel is needed.
Evals (deterministic, model-free, CI-gated — `openrange_pack_sdk.npcs.metrics`):
- sender is never forgeable across comms verbs + adversarial content (0/N) —
separability-by-construction for the grader;
- a sampled population is diverse (unique actor_ids + role entropy >= 0.9);
- the persona prompt doesn't read like an assistant (tell rate < 0.05).
Live-model believability, a non-degeneracy A/B, and per-actor HTTP tagging are
deferred to a follow-up (noted in docs).
Docs: graded runs use only world-mediated comms tools, not http_get (persona
http traffic is un-attributed and pollutes the request log / reached_endpoint);
every persona needs a name (the seating contract); cyber.persona vs
office_persona. ruff + mypy --strict + boundary clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Dashboard integration + CI-gated evals + docs honesty (
|
Ran the deferred model-gated believability eval against a real local model
(qwen3-4b via Ollama). Adds a runnable harness and one detector fix it surfaced.
`examples/persona_believability_eval.py`: drives N sampled personas through a
live model, extracts each utterance, and reports role entropy + actor-id
uniqueness + assistant-tell rate. Skips cleanly (exit 0) when no model/backend
is present, so it's an artifact, not a dependency.
What the run showed (qwen3-4b, N=5): the extracted utterances are believably in
character (tell rate 0.00 — e.g. "Q3 report done, but numbers still need a little
tweaking"), BUT the raw stream leaks chain-of-thought as prose ("the user wants
me to act as Dana...") and tool-calling is unreliable at 4B. So the detector now
also flags meta-reasoning leaks (`the user`, `act as`, `stay in character`,
`my character/persona`) — a real believability failure the assistant-only
patterns missed; the generated persona prompt still scores < 0.05. Takeaway
(docs already note it): prefer a non-thinking instruct model for clean
tool-driven cover traffic; small thinking models leak reasoning + skip tools.
ruff + mypy --strict + boundary clean; the CI tell-detector test now locks in
the reasoning-leak case.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
larstalian
left a comment
larstalian
left a comment
There was a problem hiding this comment.
Solid config-driven NPC binder that replaces the hand-written persona factories — mechanism is clean and well-tested. My main reservation is that as shipped it's pure substrate: the one binding (cyber_webapp) gives personas read-only + comms tools whose output no grader reads and the agent can't see, so nothing observable changes yet, and the PR body oversells that ("SUT can read the same store" is false and shouldn't be — the agent is correctly walled off).
Strictly, this could land with narrowed claims + a task family that consumes the comms — but I'd rather see a runnable example of the real manifest-driven usage first (the two current "examples" are integration tests in disguise), since writing one against a live episode would likely surface impl gaps and change a fair bit here.
Also, remember to clean up comments, they read as implementation log now. Follow .rules 😄
There was a problem hiding this comment.
this file is not an implementation log
There was a problem hiding this comment.
Pulled the CI-status and "deferred to a follow-up" paragraphs, kept the usage guidance. 11591fb
| # No positional path: honor `testpaths` (pyproject) so the pack-SDK and | ||
| # graphschema suites gate CI too, not just top-level tests/. |
There was a problem hiding this comment.
comments read as implementation log, pls remove
There was a problem hiding this comment.
this folder is for user-facing tutorials, not unit or integration tests.
However, I agree that we should have examples on how to use this new feature, but i would then like to see it implemented inside a new tutorial / example script/notebook where social npcs are relevant
There was a problem hiding this comment.
Good call — removed both eval scripts in 11591fb. I'll add a real manifest-driven tutorial with social NPCs instead of parking tests here.
| # Phrases a helpful-assistant or a model reasoning out loud emits, but an | ||
| # in-character persona should not. The meta-reasoning group (the second block) | ||
| # was added after a small local model leaked its chain of thought as prose | ||
| # ("the user wants me to act as Dana...") — a real believability failure the | ||
| # helpful-assistant patterns alone missed. |
There was a problem hiding this comment.
reads as imp log. follow .rules
There was a problem hiding this comment.
Cut the fix-history narration, kept a one-line why. 11591fb
Adversarial front+backend review confirmed the persona seats + speaks on the dashboard (verified end-to-end through the real DashboardView bridge). Fixes the LOW findings it surfaced: - dashboard.js: derive the callout CSS class from `kind` (email/chat), not the raw channel name — mail now renders with the email style instead of chat, and a channel like "ops" no longer picks a nonexistent class. Empty-cast copy now points authors at `cyber.persona` (was `cyber.office_persona`, contradicting the docs). - persona_agent.py: mail emits `kind:"mail"` for the render above; drop the dead `goal` key from the presence payload (the SPA has no consumer). - realize.py: the npc_mail/npc_chat comment no longer claims a grader consumes it (none does yet) — "available to a grader" is honest. - docs: personas animate on comms (seated-but-silent without a comms tool, don't walk between desks); anonymous personas late-spawn at the chat hub. New test (test_persona_episode.py): drives a real EpisodeService + DashboardView with a tool-calling backend and asserts the persona's present + speak events land in the bridge buffer attributed to its actor_id, with mail carrying kind:"mail" — the exact front-back gap that let the render mismatch hide. ruff + mypy --strict + boundary clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…mples
- metrics.py / ci.yml comments no longer narrate the fix history (follow .rules)
- docs: drop the CI-status / deferred notes; keep the usage guidance
- remove examples/persona_{eval,believability_eval}.py — they were integration
tests living in the examples/ tree, which is for user-facing tutorials
|
Thanks Lars, you're right on the main point. Cleaned up in On substrate: agreed, nothing observable changes yet. It's the seam, not a behavior. I'd rather ship the real thing than land it bare. The next step is a manifest-driven example on a live episode plus a task family that reads the drained comms — say, an insider/phishing objective where a persona's message is what you hunt for. That'll surface impl gaps and move some code, like you expected. Want it in this PR or a follow-up? |
2 participants
What
A generic, config-driven
PersonaAgentNPC in the pack SDK: define one persona as data and bind it to any pack by config. The same class serves web / network / terminal / enterprise / social worlds — it wraps whatever callables the pack surfaces into the per-tickinterface. No per-domain NPC code, no model training.Why
Today every pack hand-writes NPC factories (
curious_employee,office_persona, …). This adds one shared, config-driven persona NPC so a pack gets ambient inhabitants — and NPC-to-NPC communication — from a single entry-point line plus manifest config.How it works
render_personaturns{name, role, goal, traits, tone, channels}into a terse system prompt whose load-bearing levers are an anti-assistant CONSTRAINTS block and an independent goal._build_toolsbinds only the declared keys the pack actually provided (fail-soft). The directed comms verbs (mail_send/chat_post) get typed adapters; any other affordance is wrapped with a signature-faithful, single-call tool (multi-arg supported; a side-effecting tool never double-fires).MailboxStore/ChatStoreare shared world state a pack surfaces (surface_extras) and drains (collect_extras). Each NPC gets one frozen surface, so identity is injected NPC-side (the persona'sactor_id) and omitted from the model-facing signature — every message is attributable and unspoofable.DictMemoryscoped per{run}:{actor}(default off).Current scope — substrate
As shipped this is the substrate, not an end-to-end behavior change. In the one binding (
cyber_webapp) personas get read-only + comms tools, the system-under-test is walled off from the comms store, and no task family consumes the drained comms yet — so an episode's pass/fail is unchanged by adding personas. The value here is the generic, attributable ambient-NPC seam; a comms-consuming task family and a runnable, manifest-driven tutorial are the natural next step.Using it (any pack)
{"type": "myp.persona", "count": 3, "config": { "name": "Dana", "role": "accountant", "goal": "reconcile invoices with a colleague", "tools": ["mail_send", "mail_read"], "cadence_ticks": 5}}A network pack passes
"tools": ["shell"]; enterprise passes["sql", "admin_lock", "mail_send"]— same class, different surfaced tools.Non-goals
Tests
_step_npcssurface, read cursor, per-persona/per-run memory isolation, cadence, broken-state, config permutations, generalization across pack shapes.resolve_manifest_npcs→cyber.persona(entry point + replication);cyber_webappsurface_extras/collect_extras/reset_episodewiring; a realEpisodeService+DashboardViewasserting the persona seats and speaks, attributed to itsactor_id.--strict, boundary firewall, coverage.