Skip to content

test(#1062): failing tests for checkup review-loop no-diff verification gap#1071

Draft
prompt-driven-github[bot] wants to merge 1 commit into
mainfrom
fix/issue-1062
Draft

test(#1062): failing tests for checkup review-loop no-diff verification gap#1071
prompt-driven-github[bot] wants to merge 1 commit into
mainfrom
fix/issue-1062

Conversation

@prompt-driven-github
Copy link
Copy Markdown
Contributor

Summary

Adds failing tests that detect the bug reported in #1062: the checkup review loop reports a fixer success even when _commit_and_push_if_changed returned the no-op sentinel "No changes to push." and the PR head never advanced. The verifier then runs against an unchanged worktree with the fixer's claim injected into its prompt and _mark_findings_fixed closes the finding without proof.

Test Files

  • Unit tests: tests/test_checkup_review_loop.py — new class TestPostFixNoDiffPushVerification (4 tests)

What This PR Contains

  • 3 failing reproductions covering the bug across three observable channels:
    • rendered report still says fixed after a no-diff push
    • in-memory finding state is marked closed without evidence
    • verification-gap signal (verification=verified; changed_files=none; fixed) is emitted
  • 1 happy-path regression guard (currently passes) so the eventual fix does not regress the success path
  • All 97 pre-existing tests in the file still pass

Root Cause

pdd/checkup_review_loop.py:1037-1112 — the post-fix push guard treats pushed=True as success even when _commit_and_push_if_changed returned the sentinel "No changes to push.". The fix is a localized guard insertion extending the existing pattern at lines 1042-1047 that already handles the analogous "failed push" case.

See investigation steps in #1062 for full root-cause analysis.

Next Steps

  1. Implement the no-op-push guard in pdd/checkup_review_loop.py
  2. Verify the 3 reproduction tests now pass
  3. Verify the happy-path regression guard still passes
  4. Run full test suite to check for regressions
  5. Mark PR as ready for review

Fixes #1062


Generated by PDD agentic bug workflow

…ification gap

Reproduces the bug in #1062 where the checkup review loop treats a fixer's
self-reported success as a completed fix even when _commit_and_push_if_changed
returns the "No changes to push." no-op sentinel and the PR head never
advances.

Adds class TestPostFixNoDiffPushVerification to tests/test_checkup_review_loop.py
with 3 failing reproductions plus 1 happy-path regression guard.

Refs #1062
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: review-loop report must not present unverified fixer attempts as completed fixes

1 participant