Add opt-in disk persistence for the KV prefix cache

[aidiffuser]

Persists the prefix cache's hot slot to disk and restores it across conversation switches and runner restarts, so returning to an earlier long conversation reuses its KV cache instead of re-prefilling. Off by default — enable with EXO_KV_DISK_PERSISTENCE=1. Supersedes #1830.

How it works

• Slots live under EXO_CACHE_HOME/kv-cache/<sha256(model_id)[:16]>/ (location overridable with EXO_KV_DISK_PATH) as slot_N_cache.safetensors + slot_N_tokens.safetensors + slot_N_meta.json, written atomically (tmp + rename, tokens last so a slot only becomes discoverable once complete).
• The hot slot is flushed on conversation switch, when the runner's generation queue drains, and on shutdown.
• On a cache miss, slots are matched by longest token prefix (≥ 1000 tokens); the reusable prefix is capped at the cache's stored length.
• Eviction: a TTL (EXO_KV_DISK_TTL_HOURS, default 24) plus a global size cap across all models (EXO_KV_DISK_MAX_SIZE_GB, default 500) — when over the cap, the globally oldest slot is evicted first (cross-model LRU).
• A janitor TTL-sweeps all model directories under the kv-cache root — at cache init and after every flush — so models that are never loaded again don't keep their slots forever.
• Fail-safe by construction: any load problem logs a warning and falls back to recompute — a bad slot can never crash a runner.

Architecture coverage

Cache state isn't always a pure array tree: the DSA indexer cache (DeepSeek-V3.2 / GLM MoE DSA) stores zero-width value arrays, and DeepseekV4Cache branch state holds Nones, ints and int-lists — none of which safetensors can represent. Such leaves are recorded as JSON placeholder specs in slot_N_meta.json and re-inserted on load, keeping the .safetensors itself compatible with stock mlx_lm.save_prompt_cache / load_prompt_cache for standard caches.

Loading reconstructs cache classes through an explicit registry, which also covers classes load_prompt_cache cannot build (DeepseekV4Cache needs its sliding-window constructor argument, recovered from meta_state). Partial-prefix hits on non-trimmable caches (DeepseekV4, SSM) are skipped, since disk slots carry no snapshots to restore from.

Testing

Nine unit tests, no model weights needed:

• round-trips for KVCache, CacheList with zero-width arrays, DeepseekV4Cache with mixed branch state, and ArraysCache with None entries
• mlx-lm format compatibility in both directions (stock-written slots load here; slots written here load with stock load_prompt_cache)
• the non-trimmable partial-prefix guard, prefix capping, and the opt-in default

Validated live on Llama 3.2 3B, Kimi K2.6 (2-node), GLM-5.1 (2-node), and DeepSeek-V4-Flash (2-node).

Known limitation

In multi-node instances each rank persists and matches its own shard's slots independently. Identical request streams produce identical slots, so ranks agree in practice; after asymmetric eviction or slot corruption a divergent decision is possible. Gathering the slot decision across ranks is a possible follow-up.

---

The trim_cache bug this work uncovered (V4 entries not restoring from snapshots) is fixed separately in #2158; this PR is independent of it and mergeable in either order.