Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 33 additions & 4 deletions .github/docs/NIGHTLY-BUILD.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,9 @@ Tracks are specified as a comma-separated string in the `NIGHTLY_TRACKS` repo va
| `test-backend-<branch>` | Run backend tests against `<branch>` |
| `test-frontend-<branch>` | Run frontend tests against `<branch>` |
| `deploy-<branch>` | Deploy full stack from `<branch>` with automatic teardown |
| `e2e-<branch>` | Deploy full stack from `<branch>`, run Playwright E2E tests, then teardown |
| `merge-validation:<base>:<overlay>` | Deploy `<base>`, then overlay `<overlay>` on top (colons delimit to avoid branch name ambiguity) |
| `all` | Run all tracks with defaults: tests + deploy on `develop`, MV `main`→`develop` |
| `all` | Run all tracks with defaults: tests + deploy + e2e on `develop`, MV `main`→`develop` |

### Examples

Expand All @@ -27,9 +28,10 @@ test-backend-develop
test-frontend-main,test-backend-main
deploy-develop
deploy-main,deploy-develop
e2e-develop
merge-validation:main:develop
merge-validation:main:feature/my-branch
test-backend-develop,deploy-develop,merge-validation:main:develop
test-backend-develop,deploy-develop,e2e-develop,merge-validation:main:develop
all
```

Expand All @@ -44,7 +46,7 @@ The `resolve-tracks` job parses the tracks string into boolean flags and branch
## Workflow Jobs

### resolve-tracks
Parses the tracks string and outputs boolean flags (`run_test_backend`, `run_test_frontend`, `run_deploy`, `run_mv`) and branch refs for each enabled track.
Parses the tracks string and outputs boolean flags (`run_test_backend`, `run_test_frontend`, `run_deploy`, `run_e2e`, `run_mv`) and branch refs for each enabled track.

### Test Tracks

Expand All @@ -62,7 +64,23 @@ When `deploy-<branch>` is specified, calls the reusable `nightly-deploy-pipeline
- `alb-subdomain`: `nightly-<branch>-api`
- Automatic teardown (unless `skip_teardown` is set)

The deploy pipeline runs: install-infra → check-stack-deps → deploy-infra → deploy-rag → deploy-inference → deploy-app → deploy-frontend + deploy-gateway → smoke-test → teardown.
The deploy pipeline runs: install-infra → check-stack-deps → deploy-infra → deploy-rag → deploy-inference → deploy-app → deploy-frontend + deploy-gateway → smoke-test → e2e-tests (if enabled) → teardown.

### E2E Test Track

When `e2e-<branch>` is specified, calls the reusable `nightly-deploy-pipeline.yml` with:
- `project-prefix`: `nightly-e2e-<branch>`
- `alb-subdomain`: `nightly-e2e-<branch>-api`
- `run-e2e`: `true`
- Automatic teardown (unless `skip_teardown` is set)

This deploys a full stack and runs Playwright E2E tests against it. The E2E job runs after the smoke test confirms the stack is healthy. It uses a separate CI Playwright config (`playwright.ci.config.ts`) that points at the deployed stack URL instead of starting local servers.

E2E test failures are **informational** — they mark the nightly summary as "partial" rather than "failed". This allows the track to stabilize without blocking other tracks.

**Required secrets** (in the `development` environment):
- `E2E_ADMIN_USERNAME` / `E2E_ADMIN_PASSWORD` — Cognito admin test account
- `E2E_USER_USERNAME` / `E2E_USER_PASSWORD` — Cognito regular user test account

### Merge Validation Track

Expand Down Expand Up @@ -91,6 +109,7 @@ Reusable workflow (`workflow_call`) containing the full deploy pipeline.
| `skip-teardown` | no | Skip teardown (default: `false`) |
| `label` | no | Label for job names |
| `source-project-prefix` | no | If set, Docker jobs try ECR image promotion before building |
| `run-e2e` | no | Run Playwright E2E tests after smoke test (default: `false`) |

### Promote-or-Build Pattern

Expand Down Expand Up @@ -126,6 +145,8 @@ Deploy and MV tracks use the `development` GitHub environment with overrides:
### Required Secrets
- `AWS_ROLE_ARN` or `AWS_ACCESS_KEY_ID` + `AWS_SECRET_ACCESS_KEY`
- `GITHUB_TOKEN` (automatically provided)
- `E2E_ADMIN_USERNAME` / `E2E_ADMIN_PASSWORD` — Cognito admin test account (required for e2e track)
- `E2E_USER_USERNAME` / `E2E_USER_PASSWORD` — Cognito regular user test account (required for e2e track)

### Required Variables
- `AWS_REGION`, `CDK_AWS_ACCOUNT`
Expand All @@ -138,6 +159,7 @@ Deploy and MV tracks use the `development` GitHub environment with overrides:
| `backend-coverage` | 30 days | Backend test track enabled |
| `frontend-coverage` | 30 days | Frontend test track enabled |
| `coverage-comparison` | 30 days | Any test track succeeded |
| `playwright-report-<label>` | 30 days | E2E track enabled (includes HTML report, screenshots, traces) |

## Troubleshooting

Expand All @@ -159,3 +181,10 @@ Check that `NIGHTLY_TRACKS` is set as a repository variable (not a secret). Empt

### Merge validation fails on overlay
This is the intended signal — it means the overlay branch has CDK/infra incompatibilities with the base branch that need to be resolved before merging.

### E2E tests fail
- Download the `playwright-report-<label>` artifact for screenshots and traces
- Check that `E2E_ADMIN_USERNAME`, `E2E_ADMIN_PASSWORD`, `E2E_USER_USERNAME`, `E2E_USER_PASSWORD` secrets are set in the `development` environment
- Verify the Cognito test users exist and have the correct roles
- If auth setup fails, the Cognito managed login UI may have changed — check the `auth-admin.setup.ts` / `auth-user.setup.ts` selectors
- E2E failures are informational and do not block other tracks
26 changes: 13 additions & 13 deletions .github/workflows/app-api.yml
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ jobs:

- name: Restore Python packages cache
id: cache-python
uses: actions/cache/restore@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: |
~/.cache/uv
Expand All @@ -96,7 +96,7 @@ jobs:

- name: Restore node_modules cache
id: cache-infrastructure
uses: actions/cache/restore@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: infrastructure/node_modules
key: infrastructure-node-modules-${{ hashFiles('infrastructure/package-lock.json') }}
Expand All @@ -113,7 +113,7 @@ jobs:

- name: Save Python packages cache
if: steps.cache-python.outputs.cache-hit != 'true'
uses: actions/cache/save@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
uses: actions/cache/save@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: |
~/.cache/uv
Expand All @@ -122,7 +122,7 @@ jobs:

- name: Save node_modules cache
if: steps.cache-infrastructure.outputs.cache-hit != 'true'
uses: actions/cache/save@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
uses: actions/cache/save@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: infrastructure/node_modules
key: infrastructure-node-modules-${{ hashFiles('infrastructure/package-lock.json') }}
Expand Down Expand Up @@ -157,7 +157,7 @@ jobs:
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0

- name: Build and export Docker image
uses: docker/build-push-action@d08e5c354a6adb9ed34480a06d141179aa583294 # v7.0.0
uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
with:
context: .
file: backend/Dockerfile.app-api
Expand All @@ -172,7 +172,7 @@ jobs:
echo "Image size: ${IMAGE_SIZE} bytes"

- name: Upload Docker image artifact
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: app-api-docker-image
path: ${{ runner.temp }}/app-api-image.tar
Expand All @@ -187,7 +187,7 @@ jobs:
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2

- name: Restore node_modules cache
uses: actions/cache/restore@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: infrastructure/node_modules
key: infrastructure-node-modules-${{ hashFiles('infrastructure/package-lock.json') }}
Expand Down Expand Up @@ -239,7 +239,7 @@ jobs:
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2

- name: Restore Python packages cache
uses: actions/cache/restore@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: |
~/.cache/uv
Expand Down Expand Up @@ -295,7 +295,7 @@ jobs:
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2

- name: Restore node_modules cache
uses: actions/cache/restore@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: infrastructure/node_modules
key: infrastructure-node-modules-${{ hashFiles('infrastructure/package-lock.json') }}
Expand All @@ -321,7 +321,7 @@ jobs:
bash scripts/stack-app-api/synth.sh

- name: Upload synthesized templates
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: app-api-cdk-synth
path: infrastructure/cdk.out/
Expand Down Expand Up @@ -358,7 +358,7 @@ jobs:
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2

- name: Restore node_modules cache
uses: actions/cache/restore@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: infrastructure/node_modules
key: infrastructure-node-modules-${{ hashFiles('infrastructure/package-lock.json') }}
Expand Down Expand Up @@ -495,7 +495,7 @@ jobs:
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2

- name: Restore node_modules cache
uses: actions/cache/restore@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: infrastructure/node_modules
key: infrastructure-node-modules-${{ hashFiles('infrastructure/package-lock.json') }}
Expand Down Expand Up @@ -531,7 +531,7 @@ jobs:
bash scripts/stack-app-api/tag-latest.sh

- name: Upload stack outputs
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: always()
with:
name: app-api-deployment-outputs
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/codeql.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ jobs:
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2

- name: Initialize CodeQL
uses: github/codeql-action/init@c10b8064de6f491fea524254123dbe5e09572f13 # v4.35.1
uses: github/codeql-action/init@95e58e9a2cdfd71adc6e0353d5c52f41a045d225 # v4.35.2
with:
languages: ${{ matrix.language }}
build-mode: ${{ matrix.build-mode }}
Expand All @@ -63,7 +63,7 @@ jobs:
queries: security-and-quality

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@c10b8064de6f491fea524254123dbe5e09572f13 # v4.35.1
uses: github/codeql-action/analyze@95e58e9a2cdfd71adc6e0353d5c52f41a045d225 # v4.35.2
with:
category: "/language:${{ matrix.language }}"
# Upload SARIF database for debugging and audit trail
Expand Down
30 changes: 15 additions & 15 deletions .github/workflows/frontend.yml
Original file line number Diff line number Diff line change
Expand Up @@ -81,14 +81,14 @@ jobs:

- name: Restore frontend node_modules cache
id: cache-frontend
uses: actions/cache/restore@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: frontend/ai.client/node_modules
key: frontend-node-modules-${{ hashFiles('frontend/ai.client/package-lock.json') }}

- name: Restore CDK node_modules cache
id: cache-infrastructure
uses: actions/cache/restore@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: infrastructure/node_modules
key: infrastructure-node-modules-${{ hashFiles('infrastructure/package-lock.json') }}
Expand All @@ -105,14 +105,14 @@ jobs:

- name: Save frontend node_modules cache
if: steps.cache-frontend.outputs.cache-hit != 'true'
uses: actions/cache/save@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
uses: actions/cache/save@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: frontend/ai.client/node_modules
key: frontend-node-modules-${{ hashFiles('frontend/ai.client/package-lock.json') }}

- name: Save CDK node_modules cache
if: steps.cache-infrastructure.outputs.cache-hit != 'true'
uses: actions/cache/save@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
uses: actions/cache/save@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: infrastructure/node_modules
key: infrastructure-node-modules-${{ hashFiles('infrastructure/package-lock.json') }}
Expand All @@ -127,7 +127,7 @@ jobs:
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2

- name: Restore frontend node_modules cache
uses: actions/cache/restore@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: frontend/ai.client/node_modules
key: frontend-node-modules-${{ hashFiles('frontend/ai.client/package-lock.json') }}
Expand All @@ -138,14 +138,14 @@ jobs:
bash scripts/stack-frontend/build.sh

- name: Upload Frontend build artifacts
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: frontend-build
path: frontend/ai.client/dist/
retention-days: 7

- name: Save Frontend build cache
uses: actions/cache/save@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
uses: actions/cache/save@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: frontend/ai.client/dist
key: frontend-build-${{ hashFiles('frontend/ai.client/src/**', 'frontend/ai.client/angular.json', 'frontend/ai.client/tsconfig.json') }}
Expand All @@ -169,7 +169,7 @@ jobs:
echo "App version: ${APP_VERSION}"

- name: Restore CDK node_modules cache
uses: actions/cache/restore@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: infrastructure/node_modules
key: infrastructure-node-modules-${{ hashFiles('infrastructure/package-lock.json') }}
Expand All @@ -188,7 +188,7 @@ jobs:
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2

- name: Restore frontend node_modules cache
uses: actions/cache/restore@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: frontend/ai.client/node_modules
key: frontend-node-modules-${{ hashFiles('frontend/ai.client/package-lock.json') }}
Expand All @@ -199,7 +199,7 @@ jobs:
bash scripts/stack-frontend/test.sh

- name: Upload Frontend test results
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: frontend-test-results
path: frontend/ai.client/coverage/
Expand Down Expand Up @@ -244,7 +244,7 @@ jobs:
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2

- name: Restore CDK node_modules cache
uses: actions/cache/restore@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: infrastructure/node_modules
key: infrastructure-node-modules-${{ hashFiles('infrastructure/package-lock.json') }}
Expand All @@ -267,7 +267,7 @@ jobs:
bash scripts/stack-frontend/synth.sh

- name: Upload synthesized templates
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: frontend-cdk-synth
path: infrastructure/cdk.out/
Expand Down Expand Up @@ -302,7 +302,7 @@ jobs:
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2

- name: Restore CDK node_modules cache
uses: actions/cache/restore@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: infrastructure/node_modules
key: infrastructure-node-modules-${{ hashFiles('infrastructure/package-lock.json') }}
Expand Down Expand Up @@ -373,7 +373,7 @@ jobs:
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2

- name: Restore CDK node_modules cache
uses: actions/cache/restore@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: infrastructure/node_modules
key: infrastructure-node-modules-${{ hashFiles('infrastructure/package-lock.json') }}
Expand Down Expand Up @@ -402,7 +402,7 @@ jobs:
bash scripts/stack-frontend/deploy-cdk.sh

- name: Upload stack outputs
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: frontend-cdk-outputs
path: infrastructure/frontend-outputs.json
Expand Down
Loading
Loading