RFC/draft: opt-in libSQL/Turso storage backend for concurrent multi-writer memory

[Donach]

Summary — opt-in libSQL/Turso storage backend (draft / RFC)

ICM keeps memory in a single local SQLite file, so it can't be shared or written
concurrently from more than one machine/process. This adds an opt-in backend
that runs the existing icm-store SQL over the async libsql client, so the
store can live in a libSQL/Turso database — local file, a remote sqld/Turso
server, or an embedded replica. A remote server is the multi-writer path (it
serialises writes from every ICM process).

The default build is completely unchanged — rusqlite, exactly as today. The
two backends are mutually-exclusive Cargo features. Still a draft: I'd like
your read on whether you want this at all and how you'd shape it before polishing.

What changed

• icm-store gains a small sync-over-async facade (crates/icm-store/src/dbcompat.rs)
  mirroring the slice of the rusqlite API the store uses. Under --features turso
  it's aliased as rusqlite, so store.rs/schema.rs are byte-identical to the
  default build apart from cfg-gated imports and the connection-open path.
• Backends are mutually exclusive:
  • default = backend-rusqlite → upstream rusqlite, untouched.
  • cargo build -p icm-cli --no-default-features --features turso,embeddings,tui
    → libSQL. With rusqlite absent there's no extern/alias clash; compile_error!
    guards reject enabling both/neither.
• Feature plumbed through icm-mcp + icm-cli. Runtime backend chosen by env
  (TURSO_DATABASE_URL/LIBSQL_URL + TURSO_AUTH_TOKEN; ICM_TURSO_REPLICA=1).

Verified

• Default backend: 162/162 icm-store tests pass (unchanged behaviour).
• Turso backend: 161/162 — only perf_fts_search_100 regresses (see gaps).
• Against a self-hosted sqld (sqlite-vec loaded via --extensions-path):
  store/recall, and 16 concurrent icm processes wrote with zero lost rows.

Honest remaining gaps

1. Perf: the sync-over-async bridge does a block_on per call, so
   perf_fts_search_100 regresses on the turso build (worse over the network).
   Wants connection reuse or an async store path — I'd value your steer here.
2. No new libSQL-specific tests yet; the existing suite runs under both
   backends, but CI for the turso feature would need a spun-up sqld (or the
   in-memory path).
3. The use crate::dbcompat as rusqlite; alias keeps the diff tiny but is
   unusual — happy to refactor toward a Storage trait if you'd prefer.
4. sqlite-vec must be loaded server-side for the vector schema; embedded
   replicas can't forward the vec0 DDL (so remote is the vector path).

Questions

1. Would you accept a self-hosted libSQL/Turso backend, given icm cloud is
   already Turso-backed — or is hosted-only intentional?
2. Is this feature-flag shape what you'd want? Should I target develop?
3. Preference on the perf path — make the store async, or keep a sync bridge with
   connection pooling?

Branched off main@804dac2 for a clean diff; glad to rebase onto develop.
Setup + verified results in docs/turso-backend.md.

Disclosure: implemented with AI assistance and human-reviewed; commits carry a
Co-Authored-By trailer. Flagging it up front.