fix(cyber): grade the pentest on the breach + make the single-tool loop work with real models

[larstalian]

Driving a real model (NVIDIA Nemotron‑3‑Nano‑30B) through the core run_agent / arun_agent loop surfaced that the single‑tool scaffold couldn't actually be scored for cyber. The headline is the grading change; the rest follows from it.

1. Grade on the breach, not a flag submission (the real fix)

The consequence verifier already decides whether a world is real by whether the guarded flag leaks, and admission guarantees a benign request can't trigger that leak — so leak ⟺ exploit. But the episode grader still gated success on the agent submitting the flag string, so that leak signal was never wired into the reward. Proven against the live 30B: a perfect breach (leaked_secret_ids: ['secret_flag']) scored reward=0.333 / success=False, matched_flag=0 on every rollout — zero GRPO gradient on the objective.

WebappPentest.check_success now grades success on the leak (ok = secret_leaked or matched). An exact submission still counts, so a submit‑tool harness and a pure‑shell agent grade identically; matched_flag stays as the strict‑submission subgoal for eval‑mode comparability.

This follows the XBOW / RL‑pentest convention — validate actual exploitation — over the CTF flag‑submission convention (Cybench, NYU CTF, InterCode‑CTF). The latter fits an eval benchmark; OpenRange is a self‑verifying training gym, where effect‑based grading is both standard and more rigorous (the admission gate removes the "incidental discovery" worry that justifies CTF's print≠submit rule). Proof after the change: an agent that runs the exploit and finishes with prose (no flag) now scores 1.0 / success=True — "flag leaked in a response (breach confirmed)."

2. Honour native tool calls (else rollouts die at turn ~2)

parse_action only matched ```bash fences; a function‑calling model emits its native <function=bash><parameter=command>… form, which fell through to the give‑up finish fallback, killing rollouts at turn ~2 with the model's actions discarded. It now reads that form too; the last recognized action across both forms wins. Fence behaviour is unchanged.

3. Drop the cyber submit tool — the agent is pure shell (CodeAct)

With grading on the breach, the cyber agent needs no submit step. The TRL notebook (examples/trl_grpo_cyber.ipynb) now drops the submit tool everywhere — the env, the reference‑breach demo, the pool/adaptive rounds, the Strands any‑agent cell — so the policy acts with one shell tool. The reward table collapses to the honest two rungs it now has (0.333 reached → 1.0 breached).

A generic submit capability stays on the SDK filesystem‑runtime surface (the write‑side of _read_result), routed from finish — it's the secondary/eval path (the or matched branch, the rLLM run_agent loop, SWE‑style "write your answer" tasks), no longer required for cyber. Cyber grading accepts a bare flag, not only {"flag": ...}.

Tests

• test_agent_harness.py: real exploit + leak passes with no submission; native tool‑call form parsed; last‑action‑wins both orders; bare/JSON/non‑flag/empty submissions; submit() before reset() rejected. openrange.agent at 100% branch coverage.
• test_cyber_staged_generation.py: rewritten to assert the breach grades as success without a submission (was: "leak doesn't change the reward").
• 178 cyber/adapter tests green across the grading‑coupled suites. Notebook: every code cell parses, no dangling submit, stale §5 output cleared (live cells need Docker to re‑run, per the notebook). All integration, no test doubles. ruff + ruff‑format + mypy(strict) clean.

.rules audit

Root‑cause fix (the leak signal existed but was unwired); comments are genuine WHY only; removed a dead if None branch in favour of assert (matching the SDK's _read_result); no WHAT/phase markers.

SOTA sources

Cybench, NYU CTF Bench, InterCode‑CTF (flag‑submission eval); XBOW (exploitation‑validation); RL‑for‑pentest (effect/state‑based reward).

🤖 Generated with Claude Code