Skip to content

ROSAENG-1331: Add MC e2e testing workflow for SRE operators (RMO, AVO) #82850

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
dustman9000 wants to merge 1 commit into
base: main
Choose a base branch
from
+334 −0
Open

ROSAENG-1331: Add MC e2e testing workflow for SRE operators (RMO, AVO) #82850

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions ci-operator/config/openshift/aws-vpce-operator/openshift-aws-vpce-operator-main.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
base_images:
ocm-cli:
name: ocm-cli
namespace: ci
tag: latest
rosa-aws-cli:
name: rosa-aws-cli
namespace: ci
Expand Down Expand Up @@ -126,6 +130,19 @@ tests:
OPERATOR_DEPLOYMENT_NAME: aws-vpce-operator
OPERATOR_NAME: aws-vpce-operator
workflow: rosa-cluster-lease-e2e-workflow
- as: rosa-hcp-mc-e2e
cron: 0 9 * * 0
steps:
cluster_profile: rosa-e2e-01
env:
CLUSTER_SECTOR: rosa-e2e
GINKGO_FOCUS: VpcEndpointTemplate MC Controller
HOSTED_CP: "true"
OCM_LOGIN_ENV: staging
OPENSHIFT_VERSION: "4.21"
OPERATOR_NAME: aws-vpce-operator
REGION: us-west-2
workflow: rosa-operator-mc-e2e
zz_generated_metadata:
branch: main
org: openshift
Expand Down
17 changes: 17 additions & 0 deletions ...ator/config/openshift/route-monitor-operator/openshift-route-monitor-operator-master.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
base_images:
ocm-cli:
name: ocm-cli
namespace: ci
tag: latest
rosa-aws-cli:
name: rosa-aws-cli
namespace: ci
Expand Down Expand Up @@ -214,6 +218,19 @@ tests:
- ref: rosa-operator-coverage-setup
test:
- ref: rosa-operator-e2e
- as: rosa-hcp-mc-e2e
cron: 0 8 * * 0
steps:
cluster_profile: rosa-e2e-01
env:
CLUSTER_SECTOR: rosa-e2e
GINKGO_FOCUS: RHOBS Synthetic Monitoring
HOSTED_CP: "true"
OCM_LOGIN_ENV: staging
OPENSHIFT_VERSION: "4.21"
OPERATOR_NAME: route-monitor-operator
REGION: us-west-2
workflow: rosa-operator-mc-e2e
zz_generated_metadata:
branch: master
org: openshift
Expand Down
86 changes: 86 additions & 0 deletions ci-operator/jobs/openshift/aws-vpce-operator/openshift-aws-vpce-operator-main-periodics.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,90 @@
periodics:
- agent: kubernetes
cluster: build11
cron: 0 9 * * 0
decorate: true
decoration_config:
sparse_checkout_files:
- .ci-operator.yaml
- build/Dockerfile
extra_refs:
- base_ref: main
org: openshift
repo: aws-vpce-operator
sparse_checkout_files:
- .ci-operator.yaml
- build/Dockerfile
labels:
ci-operator.openshift.io/cloud: aws
ci-operator.openshift.io/cloud-cluster-profile: rosa-e2e-01
ci.openshift.io/generator: prowgen
job-release: "4.22"
pj-rehearse.openshift.io/can-be-rehearsed: "true"
name: periodic-ci-openshift-aws-vpce-operator-main-rosa-hcp-mc-e2e
spec:
containers:
- args:
- --gcs-upload-secret=/secrets/gcs/service-account.json
- --image-import-pull-secret=/etc/pull-secret/.dockerconfigjson
- --lease-server-credentials-file=/etc/boskos/credentials
- --report-credentials-file=/etc/report/credentials
- --secret-dir=/secrets/ci-pull-credentials
- --target=rosa-hcp-mc-e2e
command:
- ci-operator
env:
- name: HTTP_SERVER_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
image: quay-proxy.ci.openshift.org/openshift/ci:ci_ci-operator_latest
imagePullPolicy: Always
name: ""
ports:
- containerPort: 8080
name: http
resources:
requests:
cpu: 10m
volumeMounts:
- mountPath: /etc/boskos
name: boskos
readOnly: true
- mountPath: /secrets/ci-pull-credentials
name: ci-pull-credentials
readOnly: true
- mountPath: /secrets/gcs
name: gcs-credentials
readOnly: true
- mountPath: /secrets/manifest-tool
name: manifest-tool-local-pusher
readOnly: true
- mountPath: /etc/pull-secret
name: pull-secret
readOnly: true
- mountPath: /etc/report
name: result-aggregator
readOnly: true
serviceAccountName: ci-operator
volumes:
- name: boskos
secret:
items:
- key: credentials
path: credentials
secretName: boskos-credentials
- name: ci-pull-credentials
secret:
secretName: ci-pull-credentials
- name: manifest-tool-local-pusher
secret:
secretName: manifest-tool-local-pusher
- name: pull-secret
secret:
secretName: registry-pull-credentials
- name: result-aggregator
secret:
secretName: result-aggregator
- agent: kubernetes
cluster: build11
cron: 0 0 31 2 *
Expand Down
86 changes: 86 additions & 0 deletions ...s/openshift/route-monitor-operator/openshift-route-monitor-operator-master-periodics.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -191,6 +191,92 @@ periodics:
- name: result-aggregator
secret:
secretName: result-aggregator
- agent: kubernetes
cluster: build10
cron: 0 8 * * 0
decorate: true
decoration_config:
sparse_checkout_files:
- .ci-operator.yaml
- Dockerfile
extra_refs:
- base_ref: master
org: openshift
repo: route-monitor-operator
sparse_checkout_files:
- .ci-operator.yaml
- Dockerfile
labels:
ci-operator.openshift.io/cloud: aws
ci-operator.openshift.io/cloud-cluster-profile: rosa-e2e-01
ci.openshift.io/generator: prowgen
job-release: "4.21"
pj-rehearse.openshift.io/can-be-rehearsed: "true"
name: periodic-ci-openshift-route-monitor-operator-master-rosa-hcp-mc-e2e
spec:
containers:
- args:
- --gcs-upload-secret=/secrets/gcs/service-account.json
- --image-import-pull-secret=/etc/pull-secret/.dockerconfigjson
- --lease-server-credentials-file=/etc/boskos/credentials
- --report-credentials-file=/etc/report/credentials
- --secret-dir=/secrets/ci-pull-credentials
- --target=rosa-hcp-mc-e2e
command:
- ci-operator
env:
- name: HTTP_SERVER_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
image: quay-proxy.ci.openshift.org/openshift/ci:ci_ci-operator_latest
imagePullPolicy: Always
name: ""
ports:
- containerPort: 8080
name: http
resources:
requests:
cpu: 10m
volumeMounts:
- mountPath: /etc/boskos
name: boskos
readOnly: true
- mountPath: /secrets/ci-pull-credentials
name: ci-pull-credentials
readOnly: true
- mountPath: /secrets/gcs
name: gcs-credentials
readOnly: true
- mountPath: /secrets/manifest-tool
name: manifest-tool-local-pusher
readOnly: true
- mountPath: /etc/pull-secret
name: pull-secret
readOnly: true
- mountPath: /etc/report
name: result-aggregator
readOnly: true
serviceAccountName: ci-operator
volumes:
- name: boskos
secret:
items:
- key: credentials
path: credentials
secretName: boskos-credentials
- name: ci-pull-credentials
secret:
secretName: ci-pull-credentials
- name: manifest-tool-local-pusher
secret:
secretName: manifest-tool-local-pusher
- name: pull-secret
secret:
secretName: registry-pull-credentials
- name: result-aggregator
secret:
secretName: result-aggregator
- agent: kubernetes
cluster: build10
cron: 0 6 * * 0
Expand Down
5 changes: 5 additions & 0 deletions ci-operator/step-registry/rosa/operator/cleanup/rosa-operator-cleanup-commands.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,11 @@ if [[ -n "${SHARED_DIR:-}" ]]; then
OPERATOR_NAMESPACE=$(cat "${SHARED_DIR}/operator-e2e-namespace" 2>/dev/null || true)
fi

# Fallback for MC mode where install step doesn't run
if [[ -z "${OPERATOR_NAMESPACE}" && -n "${OPERATOR_NAME:-}" ]]; then
OPERATOR_NAMESPACE="openshift-${OPERATOR_NAME}"
fi

# Collect operator logs as artifacts for debugging
if [[ -n "${OPERATOR_NAMESPACE}" && -n "${ARTIFACT_DIR:-}" ]]; then
log "Collecting operator logs from ${OPERATOR_NAMESPACE}"
Expand Down
10 changes: 10 additions & 0 deletions ci-operator/step-registry/rosa/operator/mc-e2e/OWNERS
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
approvers:
- dustman9000
- joshbranham
- bmeng
- ravitri
reviewers:
- dustman9000
- joshbranham
- bmeng
- ravitri
17 changes: 17 additions & 0 deletions ci-operator/step-registry/rosa/operator/mc-e2e/rosa-operator-mc-e2e-workflow.metadata.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"path": "rosa/operator/mc-e2e/rosa-operator-mc-e2e-workflow.yaml",
"owners": {
"approvers": [
"dustman9000",
"joshbranham",
"bmeng",
"ravitri"
],
"reviewers": [
"dustman9000",
"joshbranham",
"bmeng",
"ravitri"
]
}
}
34 changes: 34 additions & 0 deletions ci-operator/step-registry/rosa/operator/mc-e2e/rosa-operator-mc-e2e-workflow.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
workflow:
as: rosa-operator-mc-e2e
steps:
env:
HOSTED_CP: "true"
CLUSTER_SECTOR: rosa-e2e
ENABLE_BILLING_ACCOUNT: "yes"
OCM_LOGIN_ENV: staging
CHANNEL_GROUP: stable
REPLICAS: "3"
pre:
- chain: rosa-aws-sts-hcp-provision
- ref: rosa-cluster-wait-ready-hosted-cp
- ref: rosa-cluster-credentials-hypershift-mgmt
- ref: rosa-operator-mc-e2e-setup
test:
- ref: rosa-operator-e2e
post:
- chain: rosa-cluster-deprovision
best_effort: true
- chain: rosa-sts-oidc-config-delete
best_effort: true
- ref: rosa-e2e-cleanup-vpc
best_effort: true
- ref: aws-deprovision-stacks
best_effort: true
documentation: |-
Provisions an HCP on a CI sector, fetches the Management Cluster
kubeconfig, runs operator e2e tests against the MC, then deprovisions.
Used for testing MC-resident operator code paths (HCP reconcilers,
synthetic monitoring probes, VPC endpoint management).

The operator is NOT installed or cleaned up. It is fleet-managed via
MCC/PKO on the MC. Tests run against the existing deployment.
10 changes: 10 additions & 0 deletions ci-operator/step-registry/rosa/operator/mc-e2e/setup/OWNERS
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
approvers:
- dustman9000
- joshbranham
- bmeng
- ravitri
reviewers:
- dustman9000
- joshbranham
- bmeng
- ravitri
21 changes: 21 additions & 0 deletions ci-operator/step-registry/rosa/operator/mc-e2e/setup/rosa-operator-mc-e2e-setup-commands.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
#!/bin/bash

set -o nounset
set -o errexit
set -o pipefail

log(){
echo -e "\033[1m$(date "+%d-%m-%YT%H:%M:%S") " "${*}\033[0m" >&2
}

MC_KUBECONFIG="${SHARED_DIR}/hs-mc.kubeconfig"
if [[ ! -f "${MC_KUBECONFIG}" ]]; then
log "ERROR: MC kubeconfig not found at ${MC_KUBECONFIG}"
log "rosa-cluster-credentials-hypershift-mgmt may have failed"
exit 1
fi

cp "${MC_KUBECONFIG}" "${SHARED_DIR}/kubeconfig"
log "MC kubeconfig set for operator e2e"
KUBECONFIG="${SHARED_DIR}/kubeconfig" oc whoami
log "Connected to MC: $(KUBECONFIG="${SHARED_DIR}/kubeconfig" oc whoami --show-server)"

@coderabbitai coderabbitai Bot Aug 3, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

Do not print the Management Cluster API URL.

oc whoami --show-server writes the cluster URL to the CI log. Replace this message with a generic connection confirmation.

Proposed fix 
-log "Connected to MC: $(KUBECONFIG="${SHARED_DIR}/kubeconfig" oc whoami --show-server)"
+log "Connected to MC"

As per coding guidelines, step registry command scripts must not echo cluster URLs.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
log "Connected to MC: $(KUBECONFIG="${SHARED_DIR}/kubeconfig" oc whoami --show-server)"
log "Connected to MC"
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/step-registry/rosa/operator/mc-e2e/setup/rosa-operator-mc-e2e-setup-commands.sh`
at line 21, Update the connection log around the oc whoami invocation to stop
printing the Management Cluster API URL; replace the command output with a
generic confirmation that the Management Cluster connection succeeded.

Source: Coding guidelines

17 changes: 17 additions & 0 deletions ...tor/step-registry/rosa/operator/mc-e2e/setup/rosa-operator-mc-e2e-setup-ref.metadata.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"path": "rosa/operator/mc-e2e/setup/rosa-operator-mc-e2e-setup-ref.yaml",
"owners": {
"approvers": [
"dustman9000",
"joshbranham",
"bmeng",
"ravitri"
],
"reviewers": [
"dustman9000",
"joshbranham",
"bmeng",
"ravitri"
]
}
}
14 changes: 14 additions & 0 deletions ci-operator/step-registry/rosa/operator/mc-e2e/setup/rosa-operator-mc-e2e-setup-ref.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
ref:
as: rosa-operator-mc-e2e-setup
from: rosa-aws-cli
commands: rosa-operator-mc-e2e-setup-commands.sh
resources:
requests:
cpu: 100m
memory: 100Mi
timeout: 2m0s
documentation: |-
Bridges MC kubeconfig for operator e2e testing. Copies the MC
kubeconfig produced by rosa-cluster-credentials-hypershift-mgmt
(hs-mc.kubeconfig) to the standard kubeconfig path expected by
rosa-operator-e2e.