Skip to content

feat: add --merge-with to merge with remote PEP 503 index (#133) - #135

Draft
aiolibsbot wants to merge 3 commits into
bdraco:mainfrom
aiolibsbot:koan/merge-with-remote-index
Draft

feat: add --merge-with to merge with remote PEP 503 index (#133)#135
aiolibsbot wants to merge 3 commits into
bdraco:mainfrom
aiolibsbot:koan/merge-with-remote-index

Conversation

@aiolibsbot

@aiolibsbot aiolibsbot commented May 16, 2026

Copy link
Copy Markdown

What

Adds a --merge-with URL CLI flag (and make_index(merge_with=...) kwarg) that fetches a remote PEP 503 simple index and merges its entries into the generated index.

Why

Closes #133. The motivating workflow is rsync-publishing a partial wheel set to a host that already serves other wheels. Without merge support, the next index rebuild would silently drop the remote-only wheels and force unnecessary rebuilds.

How

  • New remote_index.py module: stdlib-only HTTP + html.parser, no new deps. Returns dict[canonical_name, list[RemoteEntry]].
  • IndexMaker accepts merge_with; remote entries are filtered against the local wheel filename set so local files always win on collisions.
  • generate_project_page gained extra_entries; remote-only files are emitted with the absolute remote URL so pip can fetch them directly.
  • Failure to reach the remote logs a warning and falls back to local-only (no crash).
  • The data-requires-python, PEP 658 (data-dist-info-metadata) and PEP 714 (data-core-metadata) attributes are preserved from the remote anchors.

Testing

  • Added tests/test_remote_index.py with 7 new tests covering parser, fetcher error path, end-to-end merge, attribute round-trip.
  • Full suite: 21 passed, 95% line+branch coverage.

🤖 Generated with Claude Code


Quality Report

Changes: 6 files changed, 454 insertions(+), 12 deletions(-)

Code scan: clean

Tests: failed ([Errno 13] Permission denied: 'pytest')

Branch hygiene: 1 issue(s)

  • Branch is not pushed to remote

Generated by Kōan post-mission quality pipeline

aiolibsbot and others added 3 commits May 16, 2026 23:46
Fetches the remote simple index (read-only) and appends absolute-URL
anchors for any wheel filenames not present locally, so pip can fetch
remote-only wheels without ever overwriting them. Local wheels win on
filename collisions; remote unreachable falls back to local-only.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
flake8 flagged F401 for an unused `from . import FIXTURES` in
tests/test_remote_index.py, and bandit re-flagged B310 on the urlopen
call despite the existing `# noqa: S310` (which is a flake8-bandit
marker, not a bandit one). Add `# nosec B310` to silence bandit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Merge index over rsync or extra index

1 participant