Add comprehensive test automation pipeline with CI/CD by amacsmith · Pull Request #1 · looptech-ai/themer-up

amacsmith · 2026-01-14T03:58:25Z

Summary

Add BATS-based test infrastructure with 35 tests covering all scripts
Implement GitHub Actions CI pipeline with parallel execution stages
Add local development tooling (Makefile, lint.sh, test.sh)
Update scripts with environment variable overrides for testability

Test plan

Run make test locally - all 35 tests pass
Run ./scripts/lint.sh - ShellCheck passes
Verify GitHub Actions CI runs successfully on PR
Test theme application still works: ./scripts/apply.sh synthwave

🤖 Generated with Claude Code

- Add BATS test infrastructure with isolated test environment - test_helper.bash with HOME override for safe testing - Unit tests for apply.sh, backup.sh, restore.sh (25 tests) - Integration tests for theme workflows (10 tests) - Add GitHub Actions CI pipeline with parallel execution - Stage 1: ShellCheck, JSON validation, YAML validation (parallel) - Stage 2: Unit tests via matrix strategy (3 parallel jobs) - Stage 3: Integration tests (sequential after unit) - Stage 4: Theme validation - BATS caching for faster CI runs - Add local development tooling - scripts/lint.sh for shellcheck with severity filtering - scripts/test.sh orchestrator with --parallel, --fast options - Makefile for convenient development commands - Update scripts for testability - Add THEMER_DIR env var override for custom installs - Add THEMER_BACKUP_DIR env var for test isolation Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Remove unused COVERAGE variable and -c/--coverage flag - Use bats_opts variable consistently in all bats invocations - Add shellcheck disable comments for intentional word splitting Fixes CI pipeline ShellCheck warnings (SC2034) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Copilot

Pull request overview

This pull request adds comprehensive test automation infrastructure to the themer-up project, including BATS-based unit and integration tests (35 tests total), a GitHub Actions CI/CD pipeline with parallel execution stages, and local development tooling (Makefile, lint.sh, test.sh). The scripts have been updated with environment variable overrides (THEMER_DIR, THEMER_BACKUP_DIR) to enable isolated testing without affecting user configurations.

Changes:

Added 35 BATS tests covering backup, restore, and apply functionality with both unit and integration test suites
Implemented GitHub Actions CI pipeline with parallel execution across linting, validation, unit tests, integration tests, and theme validation stages
Added local development tooling including Makefile, test.sh orchestration script, and lint.sh for ShellCheck validation

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
tests/unit/restore.bats	Unit tests for restore.sh covering backup selection, file restoration, and error handling
tests/unit/backup.bats	Unit tests for backup.sh including timestamped backups, file copying, and backup rotation
tests/unit/apply.bats	Unit tests for apply.sh covering theme application, config file copying, and duplicate prevention
tests/integration/theme_workflow.bats	End-to-end integration tests for complete theme workflows and validation
tests/test_helper.bash	Shared test utilities providing isolated test environments and assertion helpers
tests/unit/CLAUDE.md	Auto-generated metadata file from claude-mem tool
tests/integration/CLAUDE.md	Auto-generated metadata file from claude-mem tool
tests/CLAUDE.md	Auto-generated metadata file from claude-mem tool
scripts/test.sh	Test orchestration script with support for parallel execution and multiple test modes
scripts/lint.sh	ShellCheck linting wrapper for all shell scripts in the project
scripts/restore.sh	Added THEMER_BACKUP_DIR environment variable override for testability
scripts/backup.sh	Added THEMER_BACKUP_DIR environment variable override for testability
scripts/apply.sh	Added THEMER_DIR environment variable override for testability and custom installations
Makefile	Comprehensive build automation for testing, linting, and development tasks
.shellcheckrc	ShellCheck configuration with appropriate exclusions and severity settings
.github/workflows/ci.yml	Multi-stage CI pipeline with parallel execution, caching, and comprehensive validation
.github/workflows/CLAUDE.md	Auto-generated metadata file from claude-mem tool

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-14T04:06:04Z

+# Recent Activity
+
+<!-- This section is auto-generated by claude-mem. Edit content outside the tags. -->
+
+### Jan 13, 2026
+
+| ID | Time | T | Title | Read |
+|----|------|---|-------|------|
+| #167 | 10:49 PM | 🟣 | Test Automation Orchestrator for Themer-Up | ~817 |


The date "Jan 13, 2026" is valid and not in the future. However, this appears to be an auto-generated file from the claude-mem tool. Consider whether these metadata files should be included in version control, or if they should be added to .gitignore instead.

Suggested change

# Recent Activity



### Jan 13, 2026

| ID | Time | T | Title | Read |

|----|------|---|-------|------|

| #167 | 10:49 PM | 🟣 | Test Automation Orchestrator for Themer-Up | ~817 |

# Recent Activity (Example)



This file provides a minimal, stable example of a `claude-mem` context block

for integration testing purposes. No real or time-varying metadata is stored

inside this block.

Copilot · 2026-01-14T04:06:04Z

+<!-- This section is auto-generated by claude-mem. Edit content outside the tags. -->
+
+### Jan 13, 2026
+
+| ID | Time | T | Title | Read |
+|----|------|---|-------|------|
+| #167 | 10:49 PM | 🟣 | Test Automation Orchestrator for Themer-Up | ~817 |


The date "Jan 13, 2026" is valid and not in the future. However, this appears to be an auto-generated file from the claude-mem tool. Consider whether these metadata files should be included in version control, or if they should be added to .gitignore instead.

Suggested change



### Jan 13, 2026

| ID | Time | T | Title | Read |

|----|------|---|-------|------|

| #167 | 10:49 PM | 🟣 | Test Automation Orchestrator for Themer-Up | ~817 |

Copilot · 2026-01-14T04:06:04Z

+    # Create 6 old backup directories
+    for i in {1..6}; do
+        old_backup="$TEST_BACKUP_DIR/2024010${i}_120000"
+        mkdir -p "$old_backup"
+        touch "$old_backup/iterm2.json"
+        sleep 0.1  # Ensure different timestamps
+    done


The backup directory names use hardcoded dates from January 2024 (e.g., "2024010${i}_120000"). This test is creating old backup directories with dates from 2024, which is now two years in the past. While this works for testing the cleanup logic, consider using more recent dates or dynamically generated dates to avoid confusion and make the test more maintainable.

Copilot · 2026-01-14T04:06:05Z

+    export MOCK_BACKUP="$TEST_BACKUP_DIR/20240115_120000"
+    mkdir -p "$MOCK_BACKUP"
+
+    # Create backup files
+    echo '{"theme": "backup_theme"}' > "$MOCK_BACKUP/iterm2.json"
+    echo 'backup p10k config' > "$MOCK_BACKUP/p10k.zsh"
+    echo 'backup mprocs config' > "$MOCK_BACKUP/mprocs.yaml"
+}
+
+teardown() {
+    teardown_test_home
+}
+
+@test "restore.sh exits with error when no backups exist" {
+    rm -rf "$TEST_BACKUP_DIR"/*
+
+    run "$SCRIPTS_DIR/restore.sh"
+    [ "$status" -eq 1 ]
+    [[ "$output" == *"No backups found"* ]]
+}
+
+@test "restore.sh restores iTerm2 config" {
+    run "$SCRIPTS_DIR/restore.sh"
+    [ "$status" -eq 0 ]
+
+    restored="$TEST_HOME/Library/Application Support/iTerm2/DynamicProfiles/themer-up.json"
+    assert_file_exists "$restored"
+    assert_file_contains "$restored" "backup_theme"
+}
+
+@test "restore.sh restores p10k config" {
+    run "$SCRIPTS_DIR/restore.sh"
+    [ "$status" -eq 0 ]
+
+    assert_file_exists "$TEST_HOME/.p10k-themer.zsh"
+    assert_file_contains "$TEST_HOME/.p10k-themer.zsh" "backup p10k"
+}
+
+@test "restore.sh restores mprocs config" {
+    run "$SCRIPTS_DIR/restore.sh"
+    [ "$status" -eq 0 ]
+
+    assert_file_exists "$TEST_HOME/.config/mprocs/claude.yaml"
+    assert_file_contains "$TEST_HOME/.config/mprocs/claude.yaml" "backup mprocs"
+}
+
+@test "restore.sh uses most recent backup when multiple exist" {
+    # Create an older backup with different content
+    older_backup="$TEST_BACKUP_DIR/20240110_100000"
+    mkdir -p "$older_backup"
+    echo '{"theme": "older_theme"}' > "$older_backup/iterm2.json"
+
+    # Create a newer backup
+    newer_backup="$TEST_BACKUP_DIR/20240120_140000"
+    mkdir -p "$newer_backup"
+    echo '{"theme": "newer_theme"}' > "$newer_backup/iterm2.json"


The backup directory names use hardcoded dates from January 2024 (e.g., "20240115_120000", "20240110_100000", "20240120_140000"). These dates are now two years in the past. While this works for testing, consider using dynamically generated dates relative to the current time to make the test more maintainable and avoid confusion.

Copilot · 2026-01-14T04:06:05Z

+        -c|--coverage)
+            COVERAGE=true
+            shift
+            ;;


The --coverage flag is parsed and stored in the COVERAGE variable (line 30) but is never used in the script. Either implement coverage reporting functionality or remove this unused option from the argument parser.

Copilot · 2026-01-14T04:06:05Z

+<claude-mem-context>
+# Recent Activity
+
+<!-- This section is auto-generated by claude-mem. Edit content outside the tags. -->
+
+### Jan 13, 2026
+
+| ID | Time | T | Title | Read |
+|----|------|---|-------|------|
+| #167 | 10:49 PM | 🟣 | Test Automation Orchestrator for Themer-Up | ~817 |
+</claude-mem-context>


The date "Jan 13, 2026" is valid and not in the future. However, this appears to be an auto-generated file from the claude-mem tool. Consider whether these metadata files should be included in version control, or if they should be added to .gitignore instead.

Suggested change

<claude-mem-context>

# Recent Activity



### Jan 13, 2026

| ID | Time | T | Title | Read |

|----|------|---|-------|------|

| #167 | 10:49 PM | 🟣 | Test Automation Orchestrator for Themer-Up | ~817 |

</claude-mem-context>

# CLAUDE unit test notes

This file is intentionally checked into version control as a static, manually

maintained markdown document used by unit tests.

The content below can be adapted by tests that need example markdown input,

but it is **not** generated by any tooling at build or runtime.

## Example section

- Item 1: Example test data

- Item 2: Additional example entry

Copilot · 2026-01-14T04:06:06Z

+<claude-mem-context>
+# Recent Activity
+
+<!-- This section is auto-generated by claude-mem. Edit content outside the tags. -->
+
+### Jan 13, 2026
+
+| ID | Time | T | Title | Read |
+|----|------|---|-------|------|
+| #167 | 10:49 PM | 🟣 | Test Automation Orchestrator for Themer-Up | ~817 |
+</claude-mem-context>


The date "Jan 13, 2026" is valid and not in the future. However, this appears to be an auto-generated file from the claude-mem tool. Consider whether these metadata files should be included in version control, or if they should be added to .gitignore instead.

Copilot · 2026-01-14T04:06:06Z

+# SC2012: Use find instead of ls (our use case is controlled)
+# SC2129: Consider using grouped redirects (readability preference)
+exclude=SC2312,SC2012,SC2129


The SC2012 exclusion comment states "our use case is controlled" but the scripts use ls -dt with pipe to head and tail which could be problematic with filenames containing newlines or special characters. While this may work in the controlled test environment, consider using find with -printf or other safer alternatives for production code to avoid potential issues with special characters in backup directory names.

Suggested change

# SC2012: Use find instead of ls (our use case is controlled)

# SC2129: Consider using grouped redirects (readability preference)

exclude=SC2312,SC2012,SC2129

# SC2129: Consider using grouped redirects (readability preference)

exclude=SC2312,SC2129

Copilot · 2026-01-14T04:06:06Z

+      - name: Install BATS
+        if: steps.bats-cache.outputs.cache-hit != 'true'
+        run: |
+          git clone https://github.com/bats-core/bats-core.git


The BATS installation clones the repository without specifying a branch or tag. This could lead to inconsistent behavior if the default branch changes. Consider checking out a specific version tag to ensure reproducible builds: git clone --branch v1.10.0 --depth 1 https://github.com/bats-core/bats-core.git

Suggested change

git clone https://github.com/bats-core/bats-core.git

git clone --branch "v${BATS_VERSION}" --depth 1 https://github.com/bats-core/bats-core.git

Copilot · 2026-01-14T04:06:06Z

+          git clone https://github.com/bats-core/bats-core.git
+          cd bats-core
+          ./install.sh ~/.local/bats


The BATS installation clones the repository without specifying a branch or tag. This could lead to inconsistent behavior if the default branch changes. Consider checking out a specific version tag to ensure reproducible builds: git clone --branch v1.10.0 --depth 1 https://github.com/bats-core/bats-core.git

Copilot AI review requested due to automatic review settings January 14, 2026 03:58

Copilot started reviewing on behalf of amacsmith January 14, 2026 03:58 View session

Copilot AI reviewed Jan 14, 2026

View reviewed changes

amacsmith merged commit 22e3d14 into main Jan 14, 2026
11 checks passed

amacsmith deleted the feat/test-automation-pipeline branch January 14, 2026 05:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add comprehensive test automation pipeline with CI/CD#1

Add comprehensive test automation pipeline with CI/CD#1
amacsmith merged 2 commits into
mainfrom
feat/test-automation-pipeline

amacsmith commented Jan 14, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Jan 14, 2026

Uh oh!

Copilot AI Jan 14, 2026

Uh oh!

Copilot AI Jan 14, 2026

Uh oh!

Copilot AI Jan 14, 2026

Uh oh!

Copilot AI Jan 14, 2026

Uh oh!

Copilot AI Jan 14, 2026

Uh oh!

Copilot AI Jan 14, 2026

Uh oh!

Copilot AI Jan 14, 2026

Uh oh!

Copilot AI Jan 14, 2026

Uh oh!

Copilot AI Jan 14, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	git clone https://github.com/bats-core/bats-core.git
	git clone --branch "v${BATS_VERSION}" --depth 1 https://github.com/bats-core/bats-core.git

Conversation

amacsmith commented Jan 14, 2026

Summary

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 14, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 14, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 14, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 14, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 14, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 14, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 14, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 14, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 14, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 14, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants