feat(dev-ci): assign createdAt RG tag policy to e2e subs (AROSLSRE-1543) by raelga · Pull Request #6182 · Azure/ARO-HCP

raelga · 2026-07-21T08:33:42Z

Jira: AROSLSRE-1543 (parent story AROSLSRE-1542)

What

Assigns the createdAt resource-group tag Azure Policy to the DEV e2e/customer subscriptions (.ci.dev.e2eSubscriptions) declaratively via dev-ci:

dev-infrastructure/templates/createdat-rg-tag-policy.bicep fans a subscription-scoped policy definition + assignment out to each e2e subscription, mirroring e2e-subscription-rbac-assignments.bicep.
The per-subscription child template inlines the policy rule (append tags['createdAt'] = [utcNow()] when missing). It intentionally does not loadJsonContent across the tree into tooling/; the rule is kept textually in sync with the imperative create-createdat-policy-assignment.sh / rg-createdat-policy-rule.json, and both use the same policy definition/assignment names (aro-rg-createdat-tag / aro-createdat-rg) so they converge on one definition.
Wired as the createdat-rg-tag-policy step in the privileged e2e-subscription-rbac-grants pipeline.
docs/ci/cleanup.md updated to document the new dev-ci path.

Why

cleanup-sweeper rg-ordered discovery treats the createdAt tag as required for any action: delete rule (tooling/cleanup-sweeper/pkg/policy/discovery.go). Azure never stamps that tag by default, and the e2e/customer subscriptions had no policy applying it. As a result those subscriptions were effectively invisible to rg-ordered, letting orphaned resource groups and soft-deleted Key Vaults accumulate (root cause behind the VaultAlreadyExists INT e2e flakes and the leaked-RG backlog).

The policy step lives in the privileged dev-ci tree because writing policy definitions/assignments needs Microsoft.Authorization/policy*/write, which the unattended Contributor bot deliberately lacks.

Testing

bicep build + bicep lint + bicep format clean on both new templates.
templatize pipeline validate parses the modified pipeline schema successfully (the pre-existing range-loop bicepparam field-access warning is shared with the sibling e2e-subscription-rbac-assignments.tmpl.bicepparam and is not introduced here).
Applied on demand via make dev-ci-privileged-local-run as an OWNERS member.

Special notes for your reviewer

This is the DEV slice (.ci.dev.e2eSubscriptions: EA, EA2, Dev-02, Dev-03). INT/STG/PROD e2e subscriptions are ARM-integrated and tracked as follow-ups under the parent story (AROSLSRE-1544/AROSLSRE-1545/AROSLSRE-1546).
The bicepparam intentionally mirrors the existing RBAC-assignments range-loop idiom so the two stay consistent.

PR Checklist

Copilot

Pull request overview

This PR adds a declarative dev-ci mechanism to ensure DEV e2e/customer subscriptions have the Azure Policy needed to stamp tags['createdAt'] on newly created resource groups, enabling cleanup-sweeper rg-ordered to discover and delete eligible RGs.

Changes:

Added subscription fan-out Bicep templates to deploy a subscription-scoped Policy Definition + Assignment to each .ci.dev.e2eSubscriptions subscription.
Wired the new template into the privileged e2e-subscription-rbac-grants dev-ci pipeline as a new ARM step.
Updated cleanup documentation to describe the declarative path (while keeping the imperative script for one-off/non-dev use).

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
docs/ci/cleanup.md	Documents that DEV e2e/customer subs get the `createdAt` RG tag policy via privileged dev-ci (and when to use the imperative script).
dev-infrastructure/templates/createdat-rg-tag-policy.bicep	Subscription-scope fan-out template that targets each e2e/customer subscription.
dev-infrastructure/templates/createdat-rg-tag-policy-subscription.bicep	Per-subscription policy definition + assignment; reuses the existing JSON rule via `loadJsonContent`.
dev-infrastructure/dev-ci/e2e-subscription-rbac-grants/pipeline.yaml	Adds the `createdat-rg-tag-policy` privileged pipeline step using the new templates/config.
dev-infrastructure/configurations/createdat-rg-tag-policy.tmpl.bicepparam	Supplies `.ci.dev.e2eSubscriptions` IDs to the fan-out template, mirroring the existing RBAC grants idiom.

raelga · 2026-07-21T09:16:04Z

/test e2e-parallel

Retesting: environmental flake unrelated to this PR's diff (dev-ci bicep policy + docs; no component image or e2e-path change). Job pull-ci-Azure-ARO-HCP-main-e2e-parallel/2079485450648555520 failed in the aro-hcp-provision-environment pre step, before any test executed:

[swift-vnet] dns readiness (login.microsoftonline.com): giving up after 49 attempt(s) / 487s (limit 480s)
✗ swift-vnet-aks-net exited with code 1
failed to run ARM step: failed to poll deployment: ... context canceled

This is the SWIFT v2 / AKS container DNS cold-start readiness timeout (AROSLSRE-1465/1469), an environment-provisioning issue in CI, not caused by the diff here. Re-running.

raelga · 2026-07-21T09:22:37Z

Force-pushed 1fcc644 → 7c4caa9: renamed the bicep parameter customerSubscriptionIds → e2eSubscriptionIds (and its loop variable / description) for clarity. The values come from .ci.dev.e2eSubscriptions, i.e. the e2e test (hosted-cluster) subscriptions, so the source-accurate name is e2eSubscriptionIds. The previous name was borrowed from the sibling e2e-subscription-rbac-assignments.bicep, where "customer" fits its RBAC context but is misleading here.

No behavior change. Verified the dev-ci render path (templatize generate, full Go templating, not EV2) still resolves the param to the 4 e2e subs and the rendered bicepparam compiles:

param e2eSubscriptionIds -> [974ebd46…, e8c5a115…, e627aa70…, 6ed122d1…]

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

cleanup-sweeper rg-ordered discovery treats the createdAt resource-group tag as required for any delete rule, but Azure never stamps it and the e2e/customer subscriptions had no policy applying it. Those subscriptions were therefore invisible to rg-ordered, letting orphaned resource groups and Key Vaults accumulate. Assign the createdAt append policy declaratively to the DEV e2e subscriptions (.ci.dev.e2eSubscriptions) via a subscription-scoped bicep fan-out, mirroring e2e-subscription-rbac-assignments.bicep and reusing the existing rg-createdat-policy-rule.json as the single source of truth. Wire it as the createdat-rg-tag-policy step in the privileged dev-ci e2e-subscription-rbac-grants pipeline, since writing policy definitions/assignments needs Microsoft.Authorization/policy*/write that the unattended Contributor bot lacks.

roivaz · 2026-07-21T09:59:04Z

/lgtm
/approve

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

openshift-merge-bot · 2026-07-21T14:14:59Z

/retest-required

Remaining retests: 0 against base HEAD 59c33d5 and 2 for PR HEAD c59a089 in total

The rule body is duplicated between rg-createdat-policy-rule.json (imperative script) and createdat-rg-tag-policy-subscription.bicep (declarative dev-ci path). Document the drift risk so both are kept in sync.

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

weherdh · 2026-07-21T22:49:22Z

/lgtm

openshift-ci · 2026-07-21T22:49:29Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: raelga, roivaz, weherdh

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~dev-infrastructure/OWNERS~~ [raelga,roivaz,weherdh]
~~docs/OWNERS~~ [raelga,roivaz,weherdh]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copilot AI review requested due to automatic review settings July 21, 2026 08:33

openshift-ci Bot requested review from avollmer-redhat and sclarkso July 21, 2026 08:33

openshift-ci Bot added the approved label Jul 21, 2026

Copilot started reviewing on behalf of raelga July 21, 2026 08:34 View session

Copilot AI reviewed Jul 21, 2026

View reviewed changes

raelga force-pushed the raelg/createdat-policy-e2e-subs branch from 1fcc644 to 7c4caa9 Compare July 21, 2026 09:21

Copilot AI review requested due to automatic review settings July 21, 2026 09:21

Copilot started reviewing on behalf of raelga July 21, 2026 09:22 View session

Copilot AI reviewed Jul 21, 2026

View reviewed changes

roivaz reviewed Jul 21, 2026

View reviewed changes

Comment thread dev-infrastructure/templates/createdat-rg-tag-policy-subscription.bicep Outdated

raelga force-pushed the raelg/createdat-policy-e2e-subs branch from 7c4caa9 to c59a089 Compare July 21, 2026 09:54

Copilot AI review requested due to automatic review settings July 21, 2026 09:54

Copilot started reviewing on behalf of raelga July 21, 2026 09:55 View session

openshift-ci Bot assigned roivaz Jul 21, 2026

openshift-ci Bot added the lgtm label Jul 21, 2026

Copilot AI reviewed Jul 21, 2026

View reviewed changes

Comment thread dev-infrastructure/templates/createdat-rg-tag-policy-subscription.bicep

raelga requested a review from Copilot July 21, 2026 13:15

Copilot started reviewing on behalf of raelga July 21, 2026 13:16 View session

Copilot AI reviewed Jul 21, 2026

View reviewed changes

Comment thread docs/ci/cleanup.md

docs(ci): note createdAt policy rule is mirrored in bicep and script

459f527

The rule body is duplicated between rg-createdat-policy-rule.json (imperative script) and createdat-rg-tag-policy-subscription.bicep (declarative dev-ci path). Document the drift risk so both are kept in sync.

Copilot AI review requested due to automatic review settings July 21, 2026 14:15

openshift-ci Bot removed the lgtm label Jul 21, 2026

Copilot started reviewing on behalf of raelga July 21, 2026 14:16 View session

Copilot AI reviewed Jul 21, 2026

View reviewed changes

openshift-ci Bot assigned weherdh Jul 21, 2026

openshift-ci Bot added the lgtm label Jul 21, 2026

openshift-merge-bot Bot merged commit 844b4d7 into Azure:main Jul 22, 2026
17 checks passed

Uh oh!

Conversation

raelga commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Why

Testing

Special notes for your reviewer

PR Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

raelga commented Jul 21, 2026

Uh oh!

raelga commented Jul 21, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

roivaz commented Jul 21, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

openshift-merge-bot Bot commented Jul 21, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

weherdh commented Jul 21, 2026

Uh oh!

openshift-ci Bot commented Jul 21, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

raelga commented Jul 21, 2026 •

edited

Loading