Skip to content

ci: wait for in-flight mirror runs; consolidate release gating on is_release - #1599

Draft
nebasuke wants to merge 3 commits into
mainfrom
ci/mirror-wait
Draft

ci: wait for in-flight mirror runs; consolidate release gating on is_release#1599
nebasuke wants to merge 3 commits into
mainfrom
ci/mirror-wait

Conversation

@nebasuke

@nebasuke nebasuke commented Aug 3, 2026

Copy link
Copy Markdown
Member

Closes #1593 — both follow-ups from the #1587 review.

Wait for the mirror workflow (@Wodann)

On PRs that touch mirror-docker-images.yml (e.g. adding a node tag), the mirror run and the edr-npm-release.yml docker jobs start in parallel, so the docker jobs could race ahead, fail their pulls with manifest unknown, and need a manual re-run once the mirror job was green.

The select-node-image action now waits for the mirror-docker-images.yml run for the same head SHA to complete — the same technique the HH3 regression benchmark workflow uses to wait for edr-ci.yml (waitForEdrCi in .github/scripts/resolve-regression-trigger.cjs). Two adaptations: no mirror run for the SHA means there is nothing to wait for (the common case — most PRs don't touch the mirror workflow), and the wait is best-effort — it never fails the job. On a non-success conclusion (a fork PR's mirror run skips itself by design and concludes skipped; a failed weekly re-sync can share main's head SHA) or on timeout it proceeds with a warning, because the pull itself is the authoritative loud failure for a genuinely missing tag. Release runs skip the step entirely; they don't use the mirror.

  • Poll every 30s with a 15-minute deadline.
  • actions: read added on the five docker-pulling jobs so the step can list workflow runs.
  • Touching mirror-docker-images.yml (comment update) also triggers a mirror run on this PR, exercising the wait against a concurrent mirror run in real CI.

Consolidate the release/non-release gating (@anaPerezGhiglia)

the trick of bypassing steps with the tag output is growing too much and I fear it because I think it's too easy to make a mistake with it. I think we should re-think this workflow and gate that in a more robust way.

check_commit's stringly tag output gated five different steps and jobs with three different expression shapes (tag == '', tag != '', non-empty release-tag input); a mistake in any of them would silently put a release build on the unsafe path — warm poisonable cache, mirror image in the supply chain — with nothing failing. Now:

  • check_commit exposes an explicit is_release output, derived from tag in one place so the two can't diverge; every consumer gates on the same expression. tag itself remains only as the npm dist-tag (next/latest) in publish, and check_commit now logs its decision on every path instead of falling through silently.
  • select-node-image takes is-release instead of the dist-tag it had no business knowing.
  • A tripwire for future edits to the cache gate: release builds fail if target/ exists after the (supposedly skipped) cache step, so removing or breaking that gate fails the release instead of silently shipping binaries built from a warm cache. (A broken is_release derivation is fail-closed separately: publish gates on the same value, so nothing ships.)

Docs

Book mirror chapter: the "benign, re-run manually" paragraph now describes the wait and its best-effort semantics.

The issue originally recorded digest-pinning as follow-up 2; that was superseded within #1587 itself (release runs pull the official image straight from Docker Hub) and the issue has been updated to record Ana's actual remaining concern.

Validation

  • actionlint + shellcheck clean (the 5 remaining shellcheck infos pre-date this PR), zizmor 1.26.1 (CI's pinned version) clean.
  • Wait-script branches tested locally: completed-success and no-run against the live workflow-runs API; skipped, failure, poll-retry, and timeout via stubbed gh/date (all proceed with at most a warning).
  • This PR's own CI ran the wait concurrently with a real mirror run on the same SHA (the docker jobs reached the step after the mirror had already completed, so the poll loop itself was only exercised locally).

Stacked follow-up

#1600 builds on this branch: it splits the workflow into a reusable build pipeline and thin CI/release callers, removing the per-job is_release guards entirely.

@changeset-bot

changeset-bot Bot commented Aug 3, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: b63ac37

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@nebasuke
nebasuke temporarily deployed to github-action-benchmark August 3, 2026 17:56 — with GitHub Actions Inactive
@codecov

codecov Bot commented Aug 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 79.88%. Comparing base (a69221b) to head (b63ac37).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1599      +/-   ##
==========================================
- Coverage   79.89%   79.88%   -0.01%     
==========================================
  Files         452      452              
  Lines       78956    78956              
  Branches    78956    78956              
==========================================
- Hits        63078    63074       -4     
- Misses      13697    13699       +2     
- Partials     2181     2183       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@nebasuke nebasuke added the no changeset needed This PR doesn't require a changeset label Aug 3, 2026
@nebasuke
nebasuke temporarily deployed to github-action-benchmark August 3, 2026 18:11 — with GitHub Actions Inactive
@nebasuke
nebasuke temporarily deployed to github-action-benchmark August 3, 2026 18:11 — with GitHub Actions Inactive
@nebasuke
nebasuke temporarily deployed to github-action-benchmark August 5, 2026 12:33 — with GitHub Actions Inactive
@nebasuke
nebasuke temporarily deployed to github-action-benchmark August 5, 2026 12:37 — with GitHub Actions Inactive
@nebasuke
nebasuke temporarily deployed to github-action-benchmark August 5, 2026 12:37 — with GitHub Actions Inactive
On PRs that touch mirror-docker-images.yml, the mirror run and the
edr-npm-release.yml docker jobs start in parallel, so the docker jobs
could race ahead and fail their pulls with 'manifest unknown', needing
a manual re-run once the mirror was green. The select-node-image action
now polls the mirror workflow run for the same head SHA (when one
exists) until it completes, same technique the HH3 regression benchmark
uses to wait for edr-ci.yml.

Closes #1593
check_commit's stringly tag output gated five different steps and jobs
with three different expression shapes; a mistake in any of them would
silently put a release build on the unsafe path (warm poisonable cache,
mirror image in the supply chain). Derive an explicit is_release output
from tag in one place, gate every consumer with the same expression,
and prove the cold-build invariant at runtime: release builds now fail
loudly if a cargo cache was restored. tag itself remains only as the
npm dist-tag for publish.

Raised by @anaPerezGhiglia in the #1587 review (#1593).
The wait step could hard-fail all five docker job families on any
non-success mirror conclusion — including fork PRs, whose mirror run
skips itself by design and concludes 'skipped', and a failed weekly
re-sync sharing main's head SHA. The pull itself is the authoritative
loud failure for a genuinely missing tag ('manifest unknown'), so
warn and proceed on non-success or timeout instead of failing.

Also from review: reword the cold-build assert comment (it's a
tripwire against future edits to the cache gate, not a proof of the
is_release derivation), log the non-release decision in check_commit
instead of falling through silently, mention the wait in the action
description, and trim reviewer-facing comments.
@nebasuke
nebasuke temporarily deployed to github-action-benchmark August 6, 2026 08:30 — with GitHub Actions Inactive
@nebasuke
nebasuke temporarily deployed to github-action-benchmark August 6, 2026 08:32 — with GitHub Actions Inactive
@nebasuke
nebasuke temporarily deployed to github-action-benchmark August 6, 2026 08:32 — with GitHub Actions Inactive
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no changeset needed This PR doesn't require a changeset

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI docker mirror follow-ups: wait for mirror workflow, verify mirror tags against upstream digests

1 participant