Add post-script dedup guard for concurrent retro proposals

[guyoron1]

What happened

23 empty workspace-update PRs (from a batch force=true workflow dispatch) were closed on 2026-07-24. This triggered 23 concurrent retro agent runs on redhat-developer/rhdh-plugin-export-overlays. At least 5 of these retros independently identified the same improvement ("add empty-diff guard to PR creation") and filed nearly identical issues within a 2-minute window: #2903, #2905, #2906, #2907, #2908 (all created 14:43–14:45 UTC by fullsend-ai-retro[bot]).

What could go better

Each retro agent performs a GitHub issue search to check for duplicates before proposing — but when many retros run concurrently on similar PRs, this check races: retro A searches, finds nothing, and files issue X; retro B searches at nearly the same time, also finds nothing (X doesn't exist yet or hasn't been indexed), and files the same issue as X'. The dedup check is necessary but not sufficient when concurrent retros target the same root cause.

This is related to fullsend-ai/fullsend#3426 (retro dedup across backports/cherry-picks), but the trigger pattern is different: fullsend-ai#3426 is about related PRs with a traceable lineage (backport links), while this case involves N independent PRs that happen to share the same root cause with no explicit relationship. The fix for fullsend-ai#3426 (backport-awareness) would not prevent this scenario.

Confidence: High — 5 duplicate issues from 23 concurrent retros is direct, observable evidence. The race window (2 minutes) and identical proposal content confirm this is a timing-based dedup failure, not a search quality issue.

Proposed change

Add a post-filing dedup guard in the retro post-script (the script that creates GitHub issues from retro proposals). Before creating an issue, the post-script should search for issues created in the last 10–15 minutes with substantially similar titles or content targeting the same repo. If a match is found, skip issue creation and instead add a comment on the existing issue noting the additional evidence.

Implementation options (in order of preference):

1. Post-script lookback search: Before calling gh issue create, query gh api search/issues for issues created by fullsend-ai-retro[bot] in the target repo within the last 15 minutes. Use title similarity (e.g., Levenshtein distance or keyword overlap) to detect near-duplicates. This is the simplest fix and handles the concurrent-filing race regardless of PR relationship.
2. Advisory lock file: Use a repo-scoped lock (e.g., a lightweight GitHub issue label or a coordination file) so only one retro can file a given proposal class at a time. More complex, but prevents the race entirely.
3. Batch retro dispatching: When multiple PRs close within a short window (e.g., 5 minutes), batch them into a single retro run that analyzes all PRs together. This is the most thorough fix but requires changes to the dispatch layer.

Validation criteria

The next time a batch of similar PRs triggers concurrent retro runs on the same repo, at most 1 issue should be created per unique proposal. Specifically: if 5+ retros run concurrently and identify the same improvement, only 1 issue should be filed (the others should either skip or add a comment to the first). Verify by monitoring the next batch-close event on redhat-developer/rhdh-plugin-export-overlays or any repo where multiple similar PRs close simultaneously.

---

Generated by retro agent from redhat-developer/rhdh-plugin-export-overlays#2842

[guyoron1]

/fs-triage

[fullsend-ai-triage]

🤖 Finished Triage · ✅ Success · Started 8:46 AM UTC · Completed 8:51 AM UTC
Commit: c887fc4 · View workflow run →

[fullsend-ai-triage]

Triage Summary

Severity: Medium · Category: Feature

Problem

Concurrent retro agents racing to file issues produce duplicates. The dedup check runs during analysis (before any issues exist), so when N retros run simultaneously on PRs sharing the same root cause, all N pass the check and file near-identical issues. This is a classic TOCTOU race — the check-then-act sequence has no coordination across concurrent runs.

Evidence

The production incident on 2026-07-24 is strong evidence: 23 PRs closed simultaneously triggered 23 retro runs, and 5 filed the same proposal within ~90 seconds. Notably, #849 in this repo is itself a duplicate of this issue — both were generated by the retro agent from different PRs in the same batch, created 4 seconds apart. This is a live demonstration of the problem.

Recommended Approach

Add a lookback search in the retro post-script before creating an issue:

1. Query the GitHub search API for issues created by fullsend-ai-retro[bot] in the target repo within the last 15–30 minutes
2. Compare candidate proposal title against returned titles using keyword overlap or fuzzy matching
3. If a match is found, skip creation and add a comment on the existing issue with the additional evidence

This is the simplest fix (Option 1 from the issue) and handles the concurrent-filing race regardless of PR relationship. It complements the semantic dedup work tracked in related upstream issues (backport-awareness and cross-branch sibling dedup) without depending on them.

Proposed Test Case

Scenario: Concurrent retro dedup guard
  Given a retro post-script execution that created issue "Add empty-diff guard"
  When a second execution attempts to create "Add empty-diff guard to PR creation"
    within the lookback window (15-30 min)
  Then the second execution skips issue creation
    and adds a comment on the first issue

  Given a third execution with title "Improve error handling in merge logic"
  Then a new issue is created (distinct proposal)

  Edge cases:
  - Minor title variations should still match
  - Proposals targeting different repos should not match
  - Issues outside the lookback window should not block new filings

Housekeeping

#849 should be closed as a duplicate of this issue.

---

Labels: This is a new feature (dedup guard) in the retro agent's dispatch pipeline. The 'enhancement' label matches as a feature improvement, and 'component/dispatch' matches because the fix targets the retro dispatch/post-script layer.

---

Next steps:

• /fs-code — agent creates a PR to implement this issue
• /fs-code <your instruction> — agent implements with your specific guidance

[fullsend-ai-triage]

🤖 Finished Triage · ✅ Success · Started 8:56 AM UTC · Completed 8:59 AM UTC
Commit: c887fc4 · View workflow run →

[fullsend-ai-triage]

An open PR already implements the dedup guard feature described here. That PR adds a fullsend file-issue CLI command with a --dedup-window flag that searches for recently-created issues with similar titles before filing a new one — directly addressing the concurrent-retro race condition documented in this issue.

The PR was created for issue #800, which described the same problem and was closed as "Path A complete." Since this issue (#848) is labeled benchmark-pathB, maintainers should evaluate whether the existing PR satisfies both benchmark lanes or whether an independent implementation is still desired for comparison purposes.

Prior triage findings (preserved):

• The root cause is a TOCTOU race in the retro agent's dedup check — concurrent retros search before any issues exist, so all pass and file duplicates.
• Issue Retro post-script should deduplicate proposals against recently-filed bot issues before creating new ones #849 was correctly closed as a duplicate of this issue.
• The recommended approach (Option 1: post-script lookback search) aligns with what PR feat(#800): add file-issue CLI command with dedup guard #822 implements.
• Related but distinct issues: Retro post-script should retract issues a subsequent run identifies as incorrectly filed #730 (retract incorrectly filed issues), Broaden retro skip criteria to all unmerged PRs with no agent interaction #734 (broaden retro skip criteria), Cap evidence issues per parent: stop filing after N data points for the same tracked issue #526 (cap evidence issues per parent) — these cover adjacent dedup scenarios but not the concurrent-filing race.

Addressed by:

• feat(#800): add file-issue CLI command with dedup guard #822

[guyoron1]

/fs-code

[fullsend-ai-coder]

🤖 Finished Code · ✅ Success · Started 9:29 AM UTC · Completed 9:41 AM UTC
Commit: c887fc4 · View workflow run →