chore: gitignore run traces + add startup benchmark harness

.github/workflows/ci.yml

@@ -64,7 +64,15 @@ jobs:
         with:
           key: libkrun   # distinct from the no-default-features `test` job's artifacts
       - name: Install libkrun
-        run: brew tap libkrun/krun && brew install libkrun libkrunfw
+        # Homebrew now gates third-party taps behind HOMEBREW_REQUIRE_TAP_TRUST
+        # (set on the runner image) — `brew install` refuses an untapped-trust
+        # formula ("Refusing to load formula … from untrusted tap"). `brew trust`
+        # the tap between tap+install; non-interactive + forward-compatible for
+        # when tap-trust becomes the Homebrew default.
+        run: |
+          brew tap libkrun/krun
+          brew trust libkrun/krun
+          brew install libkrun libkrunfw
       - name: Clippy (libkrun)
         run: cargo clippy --all-targets --features libkrun -- -D warnings
       - name: Test (libkrun)

.gitignore

@@ -23,3 +23,12 @@ __pycache__/
 *.pyc
 .venv/
 scripts/eval/toolz-tasks/
+
+# Run logs + ghost scratch outputs — noisy, regenerable, not evidence (all just
+# {task,cond,trial,score,cost} scalars + progress lines; no transcripts/memory).
+# The curated σ̂ result jsonls under scripts/eval/segmentation/results/ ARE tracked
+# (the evidence archive — see its README); only the .log transcripts there are ignored.
+scripts/eval/segmentation/results/*.log
+scripts/ghost/*.jsonl
+scripts/ghost/*.json
+scripts/ghost/*.log

scripts/bench/README.md

@@ -0,0 +1,45 @@
+# Startup benchmark
+
+`startup.py` runs pillbox startup cases, reads host-emitted
+`session.started` lifecycle events, and summarizes `startup_ms` plus per-stage
+timings.
+
+## Examples
+
+```sh
+# One Docker PTY case, five measured runs after one warmup.
+scripts/bench/startup.py --case docker-claude --warmup 1 --repeat 5
+
+# Server-mode opencode on Docker and libkrun, with a fixed model.
+# Defaults to the same opencode image used by the smoke/eval scripts:
+# PILLBOX_RUNNER_IMAGE=pillbox-runner:dev.
+scripts/bench/startup.py \
+  --case docker-opencode \
+  --case libkrun-opencode \
+  --model zai-coding-plan/glm-4.5-air \
+  --warmup 1 \
+  --repeat 5
+
+# Show the commands without starting agents.
+scripts/bench/startup.py --all-docker --dry-run
+
+# Emit raw measured runs plus aggregate summaries.
+scripts/bench/startup.py --all-docker --json
+```
+
+## Notes
+
+- Cases require the corresponding agent auth and backend support.
+- opencode cases default to `pillbox-runner:dev`; `libkrun-codex-serve` defaults
+  to `pillbox-runner:dev`, matching `scripts/smoke/`. Override with
+  `--runner-image IMAGE` or `PILLBOX_RUNNER_IMAGE`.
+- Every run uses `--detach --json --ttl` and is removed with
+  `pillbox session rm` after its event is captured.
+- By default, the benchmark creates one temporary empty workspace and reuses it
+  for every run. Use `--workspace PATH` to benchmark a real tree.
+- Commands time out after 300 seconds by default. Override with
+  `--timeout SECONDS` for slow cold-start environments.
+- Use `--json` for raw run records and aggregate summaries, or `--csv` for one
+  row per measured startup stage.
+- `PILLBOX` can point at a specific binary; otherwise the script uses
+  `target/debug/pillbox` when present, then falls back to `pillbox` on `PATH`.