feat(testing): e2e scenario harness — real boot, echo provider, cross-LLM evaluation

.github/workflows/ci.yml

@@ -37,3 +37,29 @@ jobs:
 
       - name: Security wiring audit
         run: bash scripts/security-audit.sh
+
+  e2e:
+    name: E2E Scenarios (${{ matrix.os }})
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest, macos-latest]
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-node@v4
+        with:
+          node-version: '22'
+          cache: 'npm'
+
+      - name: Install dependencies
+        run: npm ci --ignore-scripts
+
+      - name: Build
+        run: npm run build:backend
+
+      - name: E2E scenario tests
+        run: npm run test:e2e
+        env:
+          ZORA_E2E: '1'

docs/testing/e2e-cross-llm-evaluation.md

@@ -0,0 +1,61 @@
+# E2E Cross-LLM Evaluation Pattern
+
+## Overview
+
+The cross-LLM evaluation pattern uses two separate provider calls to catch output quality issues that a single-provider test cannot detect. A **generator** produces output; an **evaluator** checks it. The evaluator can be a different model or provider, enabling independent verification.
+
+## How It Works
+
+1. **Generator step**: Submit a task to the primary provider (e.g., Claude or EchoProvider).
+2. **Capture output**: Record the generator's response text.
+3. **Evaluator step**: Submit `"evaluate: <generator output>"` to the evaluator provider (e.g., Gemini or EchoProvider).
+4. **Assert on evaluator response**: The evaluator response should contain `"EVALUATION:"` and indicate success or flag issues.
+
+## Example (from scenario-harness.test.ts, Scenario 5)
+
+```typescript
+// Step 1: Generate
+const genResult = spawnAsk('Write a function to add two numbers', { configDir, cwd });
+const generatedText = genResult.stdout.trim();
+
+// Step 2: Evaluate
+const evalResult = spawnAsk(`evaluate: ${generatedText}`, { configDir, cwd });
+expect(evalResult.stdout).toContain('EVALUATION:');
+```
+
+## EchoProvider Behavior
+
+In CI (no API keys), both steps use EchoProvider:
+- Generator receives `"write"` keyword → returns a minimal code snippet.
+- Evaluator receives `"evaluate:"` prefix → returns `"EVALUATION: [provider:echo] Task appears correct. No issues found."`.
+
+This validates the wiring (two separate CLI invocations, two session files written) without requiring real LLMs.
+
+## Real Provider Configuration
+
+When `ZORA_REAL_PROVIDERS=1` is set and real API keys are available, use `tests/fixtures/e2e-config-real.toml.example` as a template:
+
+```bash
+cp tests/fixtures/e2e-config-real.toml.example tests/fixtures/e2e-config-real.toml
+# Edit to set real provider credentials/models
+ZORA_E2E=1 ZORA_REAL_PROVIDERS=1 npm run test:e2e:real
+```
+
+## CI vs Local Development
+
+| Mode | Config | Providers | API Keys |
+|------|--------|-----------|----------|
+| CI (`ZORA_E2E=1`) | `e2e-config.toml` | EchoProvider | None needed |
+| Local real (`ZORA_E2E=1 ZORA_REAL_PROVIDERS=1`) | `e2e-config-real.toml` | Claude + Gemini | Required |
+
+## Why Two Session Files?
+
+Each `zora-agent ask` invocation writes its own JSONL session file. Scenario 5 asserts that at least two session files are created — one for the generation step and one for the evaluation step. This confirms that both CLI invocations completed their full boot-and-shutdown cycle, not just that the output strings matched.
+
+## Extending the Pattern
+
+To add a new evaluation scenario:
+1. Choose a generator prompt (triggers specific EchoProvider response rule).
+2. Prefix the evaluator prompt with `"evaluate:"`.
+3. Assert both session files exist and the evaluator response contains `"EVALUATION:"`.
+4. For real providers: assert the evaluator response contains no hallucination markers (domain-specific checks).

examples/routines/content-pipeline.toml

@@ -1,20 +1,67 @@
 [routine]
 name = "content-pipeline"
-schedule = "0 8 * * 2"
-model_preference = "claude-haiku"
-max_cost_tier = "included"
-timeout = "30m"
-
-[task]
-prompt = """
-It's Tuesday — time for the weekly MyMoneyCoach.ai content pipeline.
-
-1. Check the content calendar for this week's blog topic
-2. Write a blog post using the StoryBrand framework with Sophia's voice
-3. Generate a Sophia coaching image for the blog header
-4. Create social media soundbites for Wednesday through Monday
-5. Schedule all content in the content calendar
-
-Use the PEACE framework (Plan, Educate, Act, Coach, Empower) for messaging.
-Write blog to ~/.zora/workspace/content/{date}-blog.md
+schedule = "0 8 * * 2"          # Tuesday 08:00
+model_preference = "claude-sonnet-4-6"
+max_cost_tier = "standard"
+timeout = "90m"
+one_shot = true
+
+[routine.env]
+# Pulled from Doppler at runtime — do NOT hardcode values here
+doppler_project = "sophia-wire"
+doppler_config = "dev"
+secrets = ["SOPHIA_CONTENT_MONGODB_URI", "META_PAGE_ID", "META_PAGE_ACCESS_TOKEN", "INSTAGRAM_USER_ID"]
+env_map = { "SOPHIA_CONTENT_MONGODB_URI" = "MONGODB_URI" }
+
+[team]
+name = "Content-Crew"
+mode = "sequential"
+
+[[team.agents]]
+name = "signal-agent"
+prompt_file = "content-pipeline/signal-agent.md"
+tools = ["bash", "fs_read", "fs_write"]
+abort_on_error = true            # Whole pipeline stops if signals insufficient
+
+[[team.agents]]
+name = "writer-agent"
+prompt_file = "content-pipeline/writer-agent.md"
+tools = ["bash", "fs_read", "fs_write"]
+depends_on = "signal-agent"
+
+[human_gate]
+enabled = true
+after = "writer-agent"
+before = "image-agent"
+channel = "telegram"
+message_template = """
+📝 *Tuesday Blog Draft Ready for Review*
+
+*Topic:* {topic.question}
+*Signals:* {topic.signal_count} expert citations
+*Word count:* {word_count}
+
+*Excerpt:*
+{excerpt}
+
+---
+{preview_400_words}
+
+---
+Reply *approve* to publish, *reject* to cancel.
+Auto-publishes in *2 hours* if no response.
 """
+timeout_action = "approve"       # Auto-approve after timeout (not reject)
+timeout_minutes = 120
+
+[[team.agents]]
+name = "image-agent"
+prompt_file = "content-pipeline/image-agent.md"
+tools = ["bash", "fs_read", "fs_write", "mcp_nanobanana"]
+depends_on = "writer-agent"      # Runs after gate clears
+
+[[team.agents]]
+name = "publisher-agent"
+prompt_file = "content-pipeline/publisher-agent.md"
+tools = ["bash", "fs_read", "fs_write", "http"]
+depends_on = ["writer-agent", "image-agent"]

examples/routines/content-pipeline/image-agent.md

@@ -0,0 +1,74 @@
+# ImageAgent — Content Pipeline
+
+You generate 2 images for the blog post: a hero image and a square social image. You run in parallel with the human approval gate (images generate while Rich reviews the draft).
+
+## Input
+
+Read `~/.zora/workspace/content/{TODAY}-brief.json` for topic and domain context.
+
+## StoryBrand Image Rules
+
+**Blog hero image (16:9):** The CUSTOMER is the hero. Show the customer's aspirational end state or the emotional relief of understanding. Do NOT make Sophia the primary subject of blog hero images. Real-looking people in authentic moments — not stock photo poses.
+
+**Social image (1:1):** Sophia as guide/teacher explaining the concept — this is fine for social because Sophia is presenting the content (like a Donald Miller-style explainer).
+
+## Domain → Emotional Angle Mapping
+
+| Domain | Customer Emotional State to Show |
+|--------|----------------------------------|
+| `nervous_system` | Woman taking a breath, visible release of tension, soft expression, natural light |
+| `money_psychology` | Person looking at phone/laptop with visible relief, shoulders relaxed, slight smile |
+| `behavioral_finance` | Person reviewing documents/finances with calm, considered expression — not stressed |
+| `abundance_mindset` | Woman outdoors or in bright space, open body language, warmth |
+| `financial_anxiety` | Transition moment — from furrowed/tense to open/calm (show the after, not the during) |
+
+## Hero Image Prompt Pattern
+
+```
+[Emotional scene: {domain-specific scenario}]. Real woman, 30s-40s, {emotional state from mapping}.
+Lifestyle photography aesthetic, natural window light, shallow depth of field.
+Authentic expression — not posed. Warm neutral tones. No text overlay.
+Clean, modern interior or soft natural outdoor setting.
+```
+
+Example for `nervous_system` + `money_psychology`:
+```
+Woman in her 30s sitting at a kitchen table, hands wrapped around a coffee mug, eyes closed in a moment of calm relief. Morning light. She's just set down her phone. Shoulders relaxed, expression peaceful. Lifestyle photography, warm neutrals, authentic — not a stock photo pose. No text overlay.
+```
+
+## Social Image Prompt Pattern (Sophia as teacher)
+
+```
+PRESERVE EXACTLY: Sophia's facial features, warm smile, green eyes, wavy brown hair.
+NEW SCENE: Sophia in a bright coaching space, gesturing warmly as if explaining {topic_short} to someone.
+Professional but approachable. Teal blazer or soft professional top. Pixar 3D style, warm lighting.
+Square format. No text overlay.
+```
+
+Reference image: `~/.claude/skills/sophia-image-generator/assets/sophia-avatar.png`
+
+## Steps
+
+1. Determine the dominant domain from `brief.json` → select emotional angle
+2. Generate hero image (16:9) via NanoBanana MCP:
+   - Model: `gemini-2.5-flash` (fast, sufficient for hero)
+   - No reference image (customer-focused, not Sophia)
+   - Aspect ratio: `16:9`
+3. Generate social image (1:1) via NanoBanana MCP:
+   - Model: `gemini-3-pro-image-preview` (character consistency matters)
+   - Reference image: `~/.claude/skills/sophia-image-generator/assets/sophia-avatar.png`
+   - Aspect ratio: `1:1`
+4. Download both images to `~/.zora/workspace/content/images/`
+   - Hero: `{slug}-hero.png`
+   - Social: `{slug}-social.png`
+
+## Output
+
+Print:
+```
+✓ Hero image generated: {slug}-hero.png
+✓ Social image generated: {slug}-social.png
+✓ Saved to ~/.zora/workspace/content/images/
+```
+
+Wait for PublisherAgent to copy these to the correct repo paths.

examples/routines/content-pipeline/publisher-agent.md

@@ -0,0 +1,145 @@
+# PublisherAgent — Content Pipeline
+
+You are the final agent. You publish the approved blog post, deploy to production, and post to social. **You only run after human approval has been received (or the 2-hour timeout has elapsed without rejection).**
+
+## Environment Variables (from Doppler `sophia-wire/dev`)
+
+- `MONGODB_URI` — for sophia-wire commands
+- `META_PAGE_ID` — MyMoneyCoach.ai Facebook Page ID
+- `META_PAGE_ACCESS_TOKEN` — long-lived page token
+- `INSTAGRAM_USER_ID` — connected IG Business account ID
+
+## Step 1: Read workspace artifacts
+
+```bash
+TODAY=$(date +%Y-%m-%d)
+# Find the MDX and images from workspace
+ls ~/.zora/workspace/content/${TODAY}-*.mdx
+ls ~/.zora/workspace/content/images/
+```
+
+Parse the slug from the MDX filename: `{TODAY}-{slug}.mdx` → slug is the part after `{TODAY}-`.
+
+## Step 2: Copy files to repo
+
+```bash
+REPO=~/Dev/abundancecoach.ai
+
+cp ~/.zora/workspace/content/${TODAY}-{slug}.mdx \
+   ${REPO}/content/blog/{slug}.mdx
+
+cp ~/.zora/workspace/content/images/{slug}-hero.png \
+   ${REPO}/public/images/blog/{slug}-hero.png
+```
+
+Verify both files exist before continuing.
+
+## Step 3: Git commit
+
+```bash
+cd ~/Dev/abundancecoach.ai
+git add content/blog/{slug}.mdx public/images/blog/{slug}-hero.png
+git commit -m "feat(blog): {title from frontmatter}"
+git push origin main
+```
+
+## Step 4: Deploy to production
+
+```bash
+cd ~/Dev/abundancecoach.ai
+vercel --prod --yes
+```
+
+Wait for vercel to complete (it will print the deployment URL). Then verify:
+
+```bash
+sleep 120
+curl -s -o /dev/null -w "%{http_code}" https://www.mymoneycoach.ai/blog/{slug}
+```
+
+If not 200: alert Rich with the vercel output and the URL, then continue to social (blog may just be slow to propagate).
+
+## Step 5: Mark topic published in sophia-wire
+
+```bash
+MONGODB_URI="$MONGODB_URI" sophia-wire topics-publish {slug} \
+  --post-url "https://www.mymoneycoach.ai/blog/{slug}"
+```
+
+## Step 6: Generate soundbites
+
+```bash
+MONGODB_URI="$MONGODB_URI" sophia-wire brief "{question}" --format soundbites
+```
+
+Use the first soundbite (Wednesday's post) for social today. Save all 5 to `~/.zora/workspace/content/{TODAY}-soundbites.json`.
+
+## Step 7: Post to Facebook Page
+
+```bash
+SOUNDBITE="{wednesday soundbite text}"
+BLOG_URL="https://www.mymoneycoach.ai/blog/{slug}"
+
+curl -s -X POST "https://graph.facebook.com/v21.0/${META_PAGE_ID}/feed" \
+  --data-urlencode "message=${SOUNDBITE}
+
+Read the full post: ${BLOG_URL}" \
+  -d "access_token=${META_PAGE_ACCESS_TOKEN}"
+```
+
+Check response for `"id"` field — if present, post succeeded. If error, alert Rich and skip Instagram.
+
+## Step 8: Post to Instagram
+
+Instagram requires a 2-step process:
+
+**Step 8a — Create media container:**
+```bash
+curl -s -X POST "https://graph.facebook.com/v21.0/${INSTAGRAM_USER_ID}/media" \
+  -d "image_url=https://www.mymoneycoach.ai/images/blog/{slug}-hero.png" \
+  --data-urlencode "caption=${SOUNDBITE}
+
+Link in bio → mymoneycoach.ai" \
+  -d "access_token=${META_PAGE_ACCESS_TOKEN}"
+```
+
+Save the `creation_id` from the response.
+
+**Step 8b — Publish:**
+```bash
+curl -s -X POST "https://graph.facebook.com/v21.0/${INSTAGRAM_USER_ID}/media_publish" \
+  -d "creation_id={creation_id}" \
+  -d "access_token=${META_PAGE_ACCESS_TOKEN}"
+```
+
+**Note:** The Instagram image URL must be publicly accessible. If the hero image isn't yet publicly reachable (Vercel still deploying), wait 60s and retry once.
+
+## Step 9: Send completion notification
+
+Send Rich a Telegram message via Claude Ops:
+
+```
+✅ Content pipeline complete — {TODAY}
+
+📝 Blog: {title}
+🔗 https://www.mymoneycoach.ai/blog/{slug}
+📊 Signals used: {n}
+📱 Facebook: posted
+📷 Instagram: posted
+
+Wednesday soundbite scheduled. 5 soundbites saved to workspace.
+```
+
+## Error handling
+
+| Failure point | Action |
+|--------------|--------|
+| MDX file missing | Alert, abort — do not push empty commit |
+| Hero image missing | Alert, push MDX without image, set image to placeholder |
+| git push fails | Alert with error, abort (do not deploy without commit) |
+| vercel --prod fails | Alert with full vercel output, do NOT post to social |
+| Facebook API error | Alert with error response, skip Instagram |
+| Instagram container error | Alert, skip publish step |
+| Instagram publish error | Alert (container may be orphaned — note creation_id in alert) |
+
+**Never retry social posts.** Duplicate posts are worse than missing posts. Alert and move on.