feat: Add smoke tests for copi.owasp.org and cornucopia.owasp.org (fi… #2069

immortal71 · 2026-01-23T13:37:46Z

Copilot

Pull request overview

This PR adds smoke tests for the copi.owasp.org and cornucopia.owasp.org applications to verify basic functionality and availability of deployed services, addressing issue #1265.

Changes:

Added smoke_tests.py with test classes for both applications to verify route accessibility and JavaScript functionality
Added comprehensive README documentation for running and understanding the smoke tests
Created dedicated smoke-tests.yaml workflow for scheduled and on-demand testing
Integrated smoke tests into the existing run-tests.yaml workflow

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.

File	Description
tests/scripts/smoke_tests.py	Implements smoke test classes for copi.owasp.org, cornucopia.owasp.org, and integration tests between both applications
tests/scripts/README.md	Provides documentation on what the smoke tests check, how to run them, and their CI/CD integration
.github/workflows/smoke-tests.yaml	Creates a dedicated workflow for running smoke tests on a schedule, manual trigger, or when application code changes
.github/workflows/run-tests.yaml	Integrates smoke tests into the main test suite that runs on pull requests

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-23T13:42:57Z

.github/workflows/run-tests.yaml

+      - name: Run smoke tests
+        run: pipenv run python -m unittest tests.scripts.smoke_tests -v


The smoke tests added to this workflow will fail because the hardening job (line 8-10) blocks egress traffic to all endpoints except those explicitly allowed in hardening.yaml. The smoke tests need to access copi.owasp.org:443 and cornucopia.owasp.org:443, which are not in the allowed-endpoints list.

Either the hardening.yaml file needs to be updated to include these endpoints, or the smoke tests should be made conditional/skipped when hardening is enabled.

Copilot · 2026-01-23T13:42:57Z

tests/scripts/README.md

+
+### For copi.owasp.org (Elixir/Phoenix)
+1. Homepage loads successfully (HTTP 200)
+2. Game route is accessible


The documentation states "Game route is accessible" but the corresponding test (test_02_game_route_accessible in smoke_tests.py) actually tests the homepage route ("/") again, not a game route. This is inconsistent with the implementation.

Either the documentation should be updated to reflect what is actually being tested, or the test should be fixed to test an actual game route like /cards or /games.

Copilot · 2026-01-23T13:42:58Z

tests/scripts/smoke_tests.py

+    def test_01_homepage_loads(self) -> None:
+        """Test that the Copi homepage loads successfully"""
+        try:
+            response = requests.get(self.BASE_URL, timeout=30)
+            self.assertEqual(response.status_code, 200, f"Homepage returned status {response.status_code}")
+            self.assertIn("copi", response.text.lower(), "Homepage should contain 'copi' text")
+        except requests.exceptions.ConnectionError:
+            self.fail(f"Failed to connect to {self.BASE_URL} - service may be down")
+        except requests.exceptions.Timeout:
+            self.fail(f"Request to {self.BASE_URL} timed out after 30 seconds")
+
+    def test_02_game_route_accessible(self) -> None:
+        """Test that a game-related route is accessible"""
+        url = urljoin(self.BASE_URL, "/")
+        try:
+            response = requests.get(url, timeout=30)
+            self.assertEqual(response.status_code, 200, f"Game route returned status {response.status_code}")
+        except requests.exceptions.ConnectionError:
+            self.fail(f"Failed to connect to {url} - service may be down")
+        except requests.exceptions.Timeout:
+            self.fail(f"Request to {url} timed out after 30 seconds")
+
+    def test_03_javascript_loads(self) -> None:
+        """Test that JavaScript assets are being served"""
+        try:
+            response = requests.get(self.BASE_URL, timeout=30)
+            self.assertEqual(response.status_code, 200)
+            self.assertTrue(
+                '<script' in response.text or 'app.js' in response.text or 'phoenix' in response.text.lower(),
+                "JavaScript should be loaded on the page"
+            )
+        except requests.exceptions.ConnectionError:
+            self.fail(f"Failed to connect to {self.BASE_URL} - service may be down")
+        except requests.exceptions.Timeout:
+            self.fail(f"Request to {self.BASE_URL} timed out after 30 seconds")
+
+    def test_04_health_check(self) -> None:
+        """Test that the application server is healthy and responding"""
+        try:
+            response = requests.get(self.BASE_URL, timeout=30)
+            self.assertEqual(response.status_code, 200)
+            self.assertIn('content-type', [h.lower() for h in response.headers.keys()],
+                         "Response should include content-type header")
+        except requests.exceptions.ConnectionError:
+            self.fail(f"Failed to connect to {self.BASE_URL} - service may be down")
+        except requests.exceptions.Timeout:
+            self.fail(f"Request to {self.BASE_URL} timed out after 30 seconds")


The exception handling pattern (try-except blocks for ConnectionError and Timeout) is duplicated across all test methods in the class. This creates maintainability issues as any change to error handling needs to be replicated in multiple places.

Consider extracting this into a helper method that wraps the request logic, for example:

def _make_request(self, url: str, timeout: int = 30) -> requests.Response: try: return requests.get(url, timeout=timeout) except requests.exceptions.ConnectionError: self.fail(f"Failed to connect to {url} - service may be down") except requests.exceptions.Timeout: self.fail(f"Request to {url} timed out after {timeout} seconds")

Then each test can simply call response = self._make_request(url) and focus on the assertions.

Copilot · 2026-01-23T13:42:58Z

.github/workflows/smoke-tests.yaml

+      - name: Install Python dependencies
+        run: pip install -r requirements.txt --require-hashes
+
+      - name: Run smoke tests for copi.owasp.org
+        run: python -m unittest tests.scripts.smoke_tests.CopiSmokeTests -v
+        continue-on-error: false
+
+      - name: Run smoke tests for cornucopia.owasp.org
+        run: python -m unittest tests.scripts.smoke_tests.CornucopiaSmokeTests -v
+        continue-on-error: false
+
+      - name: Run integration smoke tests
+        run: python -m unittest tests.scripts.smoke_tests.IntegrationSmokeTests -v


The workflow installs dependencies using pip and requirements.txt, but the requests library (which is required by smoke_tests.py) is not in requirements.txt - it's only in the Pipfile. This will cause the smoke tests to fail with an import error.

The workflow should install dependencies using pipenv (similar to run-tests.yaml lines 26-29), or the requests library needs to be added to requirements.txt with its hash.

Suggested change

- name: Install Python dependencies

run: pip install -r requirements.txt --require-hashes

- name: Run smoke tests for copi.owasp.org

run: python -m unittest tests.scripts.smoke_tests.CopiSmokeTests -v

continue-on-error: false

- name: Run smoke tests for cornucopia.owasp.org

run: python -m unittest tests.scripts.smoke_tests.CornucopiaSmokeTests -v

continue-on-error: false

- name: Run integration smoke tests

run: python -m unittest tests.scripts.smoke_tests.IntegrationSmokeTests -v

- name: Install Python dependencies with Pipenv

run: |

python -m pip install --upgrade pip

python -m pip install pipenv

pipenv install

- name: Run smoke tests for copi.owasp.org

run: pipenv run python -m unittest tests.scripts.smoke_tests.CopiSmokeTests -v

continue-on-error: false

- name: Run smoke tests for cornucopia.owasp.org

run: pipenv run python -m unittest tests.scripts.smoke_tests.CornucopiaSmokeTests -v

continue-on-error: false

- name: Run integration smoke tests

run: pipenv run python -m unittest tests.scripts.smoke_tests.IntegrationSmokeTests -v

Copilot · 2026-01-23T13:42:59Z

tests/scripts/smoke_tests.py

+    def test_02_game_route_accessible(self) -> None:
+        """Test that a game-related route is accessible"""
+        url = urljoin(self.BASE_URL, "/")
+        try:
+            response = requests.get(url, timeout=30)
+            self.assertEqual(response.status_code, 200, f"Game route returned status {response.status_code}")
+        except requests.exceptions.ConnectionError:
+            self.fail(f"Failed to connect to {url} - service may be down")
+        except requests.exceptions.Timeout:
+            self.fail(f"Request to {url} timed out after 30 seconds")


The test_02_game_route_accessible method is testing the same route as test_01_homepage_loads. Using urljoin(self.BASE_URL, "/") simply returns the BASE_URL, so this test does not fulfill the requirement of testing "at least 2 routes" on the application.

Based on the copi.owasp.org router (lib/copi_web/router.ex), there are several actual routes available such as /cards or /games that could be tested instead. Consider changing this to test a different route, for example:
url = urljoin(self.BASE_URL, "/cards")

Copilot · 2026-01-23T13:42:59Z

.github/workflows/smoke-tests.yaml

+  hardening:
+    name: Harden runner
+    uses: ./.github/workflows/hardening.yaml
+
+  smoke-tests:
+    name: Run Smoke Tests
+    needs: hardening


The smoke-tests workflow uses the hardening job which blocks egress traffic to all endpoints except those explicitly allowed. However, the smoke tests need to access copi.owasp.org:443 and cornucopia.owasp.org:443, which are not in the allowed-endpoints list in .github/workflows/hardening.yaml. This will cause the smoke tests to fail when they try to make HTTP requests to these domains.

The hardening.yaml file needs to be updated to include these endpoints in the allowed-endpoints list, or the smoke-tests workflow should not use the hardening job.

Suggested change

hardening:

name: Harden runner

uses: ./.github/workflows/hardening.yaml

smoke-tests:

name: Run Smoke Tests

needs: hardening

smoke-tests:

name: Run Smoke Tests

…v, fix duplicate route, add helper methods

sydseter · 2026-01-29T07:54:35Z

Sorry I havent' been able to get back to you sooner. Thank you for your effort.
The smoke test need to run against a container in the pipeline (localhost), not production or stage. This means that the copi.owasp.org pipeline should provision two or possible one docker container with the database and the application, run smoke tests against it, if successful, the smoke test will abort the build.
We do not need smoke tests for cornucopia.owasp.org. The reason for that is that vite comes with built in smoke tests that fires up the server and check that all internal links on the website goes to live pages.

immortal71 · 2026-01-29T09:01:46Z

@sydseter Thanks for the feedback. I’ll update this PR so the smoke tests run only against the Dockerized copi app and database in the CI pipeline (localhost) and not production or stage, and I’ll remove the cornucopia.owasp.org smoke tests because Vite’s checks already validate those routes

feat: Add smoke tests for copi.owasp.org and cornucopia.owasp.org (fixes

71901c7

OWASP#1265)

immortal71 requested review from cw-owasp, rewtd and sydseter as code owners January 23, 2026 13:37

Copilot AI review requested due to automatic review settings January 23, 2026 13:37

Copilot started reviewing on behalf of immortal71 January 23, 2026 13:38 View session

Copilot AI reviewed Jan 23, 2026

View reviewed changes

fixed: Address Copilot review feedback - removed hardening, use pipen…

0c1d182

…v, fix duplicate route, add helper methods

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: Add smoke tests for copi.owasp.org and cornucopia.owasp.org (fi… #2069

feat: Add smoke tests for copi.owasp.org and cornucopia.owasp.org (fi… #2069

Uh oh!

immortal71 commented Jan 23, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Jan 23, 2026

Uh oh!

Copilot AI Jan 23, 2026

Uh oh!

Copilot AI Jan 23, 2026

Uh oh!

Copilot AI Jan 23, 2026

Uh oh!

Copilot AI Jan 23, 2026

Uh oh!

Copilot AI Jan 23, 2026

Uh oh!

sydseter commented Jan 29, 2026

Uh oh!

immortal71 commented Jan 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		- name: Run smoke tests
		run: pipenv run python -m unittest tests.scripts.smoke_tests -v

Uh oh!

feat: Add smoke tests for copi.owasp.org and cornucopia.owasp.org (fi… #2069

Are you sure you want to change the base?

feat: Add smoke tests for copi.owasp.org and cornucopia.owasp.org (fi… #2069

Uh oh!

Conversation

immortal71 commented Jan 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 23, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 23, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 23, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 23, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 23, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 23, 2026

Choose a reason for hiding this comment

Uh oh!

sydseter commented Jan 29, 2026

Uh oh!

immortal71 commented Jan 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

immortal71 commented Jan 23, 2026 •

edited

Loading