[bench] docs: state the dependency-PR governance boundary honestly#16
Conversation
Three documents claimed or implied that every change in this repo is governed. PR #14 disproved that: Dependabot edited requirements.txt on GitHub, which never reaches the PreToolUse hook, so a dependency bump landed on main with no verdict and no ledger entry. The gap is structural, not a lapse. The hook can only see Write/Edit/MultiEdit tool calls. Rather than let the claim stand and quietly be false: - CLAUDE.md rule 1 now scopes the guarantee to the tool path and names the one exception outside it. - README self-governance section names it instead of glossing it. - SECURITY.md states it in the security model, and draws the line for reporters: Bench not governing a Dependabot PR is out of scope, while a way to disguise an ordinary tool-path change as one to skip adjudication is a real vulnerability. Also corrects an over-citation I had been repeating. "C-007 forbids auto-merge" is not accurate: C-007 binds changes to the pipeline files themselves and does not literally reach a GitHub merge setting. Not enabling auto-merge is project policy that serves the same purpose. Bench caught this one on review, and in a project about governance a sloppy citation is worth correcting. An accurate narrow claim beats an aspirational broad one. Closing the gap for real needs a CI job that runs the pipeline against dependency PRs, which is worth doing if this ever gets a real user base. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013QFU7kbe53G7BPUZHprb9G
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b1dc8ced64
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| is a bot-authored dependency PR: Dependabot edits `requirements.txt` on | ||
| GitHub, so it never touches the hook and merges without a verdict or a ledger | ||
| entry. Read those diffs before merging and do not enable auto-merge. A report |
There was a problem hiding this comment.
This carve-out describes Dependabot's ungovened bot PRs as requirements.txt updates, but the repo also configures Dependabot for package-ecosystem: github-actions in .github/dependabot.yml lines 13-16, so it can open PRs that edit .github/workflows/tests.yml through GitHub without the PreToolUse hook or a ledger entry. If the policy/security docs only name requirements updates, reviewers and reporters can miss that Actions-version bumps need the same human/no-auto-merge handling; please include the GitHub Actions ecosystem in this boundary.
Useful? React with 👍 / 👎.
Two receipts from drafting the release notes. Both are scratchpad files, so both are recorded with path and verdict and a redacted body. The v2.0.0 tag points at the #16 merge commit, so these land after it and are not part of the release. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013QFU7kbe53G7BPUZHprb9G
1 participant
Three documents claimed or implied that every change in this repo is governed. PR #14 disproved it: Dependabot edited
requirements.txton GitHub, which never reaches the PreToolUse hook, so a dependency bump landed onmainwith no verdict and no ledger entry.The gap is structural, not a lapse. The hook can only see
Write/Edit/MultiEdittool calls.Changes
CLAUDE.mdrule 1README.mdSECURITY.mdThe SECURITY.md line matters for triage: Bench not governing a Dependabot PR is out of scope, since it was never in scope. A way to disguise an ordinary tool-path change as one in order to skip adjudication is a vulnerability.
Also corrects an over-citation
I had been writing "C-007 forbids auto-merge" in commit messages and PR bodies. That is not accurate. C-007 binds changes to the pipeline files themselves and does not literally reach a GitHub merge setting. Not enabling auto-merge is project policy serving the same purpose.
Bench caught this on review. In a project about governance, a sloppy citation is worth correcting.
Why this and not the CI job
An accurate narrow claim beats an aspirational broad one, and it costs nothing. Closing the gap for real means a CI job that runs the pipeline against dependency PRs, which needs an API key in CI. Worth doing if this gets a real user base; not worth it today.