Skip to content

Fix duplicate issue creation in weekly link check#773

Open
underwoo wants to merge 1 commit into
NOAA-RDHPCS:mainfrom
underwoo:772-fix-duplicate-linkcheck-issues
Open

Fix duplicate issue creation in weekly link check#773
underwoo wants to merge 1 commit into
NOAA-RDHPCS:mainfrom
underwoo:772-fix-duplicate-linkcheck-issues

Conversation

@underwoo

Copy link
Copy Markdown
Contributor

Summary

Fixes #772

The weekly link check workflow was creating duplicate issues for the same broken link across multiple weeks (e.g., issues #742, #747, #750, and #755 all referenced the same broken http://www.nedit.org/ URL).

Changes

  • New file: .github/scripts/extract_known_urls.py

    • Parses open GitHub issues to extract broken URLs already being tracked
    • Outputs a JSON mapping of URL to the oldest issue number
  • Modified: .github/scripts/report_linkcheck.py

    • Added --known-urls-file argument to accept known URLs
    • Categorizes broken links as "new" or "known"
    • New exit code 3 when all broken links are already tracked (skips issue creation)
    • Known links show "Related to #NNN" in the Status column
  • Modified: .github/workflows/weekly_check_all_links.yml

    • Added step to gather known broken URLs from open issues before reporting
    • Removed the old (buggy) duplicate-check logic
    • Only creates issues when exit code is 1 (new broken links found)

Behavior

Scenario Result
Mix of new + known broken links Creates issue; new links show HTTP error, known links show "Related to #NNN"
Only known broken links No issue created (exit code 3)
No broken links No issue created (exit code 0)
All new broken links Creates issue with all links

Testing

Tested locally with mock data simulating various scenarios (all new, all known, mixed, none broken).

Add URL-level deduplication to prevent the weekly link check workflow
from creating multiple issues for the same broken link.

Changes:
- Add extract_known_urls.py to parse URLs from open issue bodies
- Update report_linkcheck.py to accept --known-urls-file argument
- Add exit code 3 for when all broken links are already tracked
- Known broken links show 'Related to #NNN' in the Status column
- Only create new issues when genuinely new broken links are found

Fixes NOAA-RDHPCS#772
@underwoo underwoo requested a review from chanwilson as a code owner June 16, 2026 14:36
elisabethpatterson pushed a commit that referenced this pull request Jun 23, 2026
* Add hash-pinned dependencies for supply chain security (#774)

- Create requirements.in with direct dependencies
- Generate requirements.txt with pip-compile --generate-hashes
- Add codeowners package for CODEOWNERS file parsing
- Add CI workflow to verify requirements stay in sync
- Document dependency management process in CONTRIBUTING.md

* Auto-assign broken link issues to code owners (#774)

- Add resolve_assignees.py to parse CODEOWNERS and select assignees
- Update weekly link check workflow to assign issues after creation
- Ensures at least one owner per affected file is assigned
- Respects GitHub's 10-assignee limit
- Uses fallback owner when no specific owner exists

Related to #772 and PR #773

---------

Co-authored-by: Chan Wilson <cwilson@meadow.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Weekly link check creates duplicate issues for the same broken link

1 participant