Skip to content

fix(checkup): keep review loop running past cost telemetry#1032

Open
Serhan-Asad wants to merge 1 commit into
promptdriven:mainfrom
Serhan-Asad:fix/checkup-review-loop-cost
Open

fix(checkup): keep review loop running past cost telemetry#1032
Serhan-Asad wants to merge 1 commit into
promptdriven:mainfrom
Serhan-Asad:fix/checkup-review-loop-cost

Conversation

@Serhan-Asad
Copy link
Copy Markdown
Collaborator

Summary

Changes pdd checkup --review-loop so cost is reported as telemetry instead of stopping the codex-review / claude-fix / codex-verify loop before max rounds or merge-ready.

Fixes #1030

Changes

  • Deprecates --max-review-cost as a report-only compatibility flag with default 0.0.
  • Removes cost from the review-loop stop predicate; max rounds, wall-clock duration, hard failures, and clean reviewer verdict remain the normal bounds.
  • Keeps max-cost-reached false for ordinary runs even when reported cost exceeds the deprecated value.
  • Updates prompts and shell completions to match the new contract.
  • Updates regression tests so cost no longer prevents fixer or verifier execution.

Verification

conda run -n pdd python -m pytest tests/test_checkup_review_loop.py tests/commands/test_checkup.py tests/test_agentic_checkup.py -q

Result: 117 passed.

Copy link
Copy Markdown

@greptile-apps greptile-apps Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.

@Serhan-Asad Serhan-Asad force-pushed the fix/checkup-review-loop-cost branch from ff00500 to 40faa4d Compare May 15, 2026 21:33
@Serhan-Asad
Copy link
Copy Markdown
Collaborator Author

Verification completed for this PR.

Local focused checks:

git diff --check upstream/main...HEAD
conda run -n pdd pylint --disable=all --enable=unused-argument pdd/checkup_review_loop.py
conda run -n pdd python -m pytest tests/test_checkup_pr_mode.py tests/test_checkup_review_loop.py tests/commands/test_checkup.py tests/test_agentic_checkup.py -q

Results:

  • git diff --check: clean
  • targeted pylint unused-argument check: clean
  • focused pytest suite: 150 passed

Cloud test:

make cloud-test

Run details:

  • Worktree: /private/tmp/pdd-pr1032-cloud
  • Commit tested: 40faa4d75c0009125f8fe8078f0a8a2768786c37
  • Cloud Batch job: pdd-test-run-20260515-144110-40faa4d75
  • Result: 77 passed, 0 failed, 0 errors
  • Report: /private/tmp/pdd-pr1032-cloud/test-results/cloud-batch-results.md

No fixes were needed after the cloud run. The cloud-test worktree was cleaned after the generated ci/cloud-batch/test-durations.json update.

Copy link
Copy Markdown
Contributor

@gltanaka gltanaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blocking findings. This change is needed for #1030: the previous review-loop cost cap could stop after a reviewer/fixer step before verifier confirmation, leaving unresolved findings and an unknown verdict even though the loop still had rounds/time available. The PR consistently changes the contract so cost is report-only telemetry, updates CLI/help/prompts/completions, and adds regressions for both cost-before-fixer and cost-after-fixer cases.\n\nTradeoff noted: callers that treated --max-review-cost as a hard local cost bound will lose that protection, but the new behavior is documented as deprecated/report-only and the loop remains bounded by max rounds, wall-clock duration, and hard failures.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(checkup): run review loop until merge-ready or max rounds, not cost

2 participants