Skip to content

feat(memory): deterministic consolidation health pass (A6)#589

Merged
philmerrell merged 1 commit into
developfrom
feature/memory-a6-consolidation
Jul 7, 2026
Merged

feat(memory): deterministic consolidation health pass (A6)#589
philmerrell merged 1 commit into
developfrom
feature/memory-a6-consolidation

Conversation

@philmerrell

Copy link
Copy Markdown
Contributor

What

Workstream A6 — the safe, non-LLM slice of consolidation, chosen deliberately (see Design note below). Adds MemorySpaceService.consolidate (editor+) and POST /memory/spaces/{id}/consolidate → a ConsolidationReport.

The pass auto-fixes only storage hygiene, and reports everything that needs a judgment call:

Concern Behavior
Orphaned objects (keys under a space's prefix with no manifest/index ref — leaks from crashed or raced writes) Auto-GC'd — safe, the content is invisible to every read path
Duplicate content (>1 slug sharing a content hash) Reported — which slug survives is semantic; never auto-merged
Dead [[slug]] wikilinks in MEMORY.md Reported; opt-in stripDeadLinks unlinks them (they point nowhere) while keeping the prose
Over-cap entry count (MEMORY_SPACE_INDEX_CAP, default 200) Flagged (overCap) — never auto-evicted

New MemorySpaceStore.list_keys(space_id) (paginated list_objects_v2 under the space prefix) drives the GC.

Design note — why deterministic-only, and the reconciliation

A6 as specced also merges duplicates, fixes stale entries, and enforces a hard index cap. Those are all LLM-driven, destructive edits to a user's durable "second brain" — a bad merge loses info, a bad "stale" call deletes a live fact — and doing them well needs real corpus data to tune against. That data doesn't exist yet: nothing writes to a space automatically until Workstream B (agentic writes). So this PR ships only what's provably safe and useful today, and establishes the consolidate() seam that the B-era LLM pass extends.

One reconciliation from the plan: "hard index-cap enforcement" is implemented as detect-and-report, not auto-evict — deciding which entry to drop is a judgment call, and auto-deleting durable memory would violate "no rewrites of durable memory." The cap becomes a health signal (overCap), not a deleter.

Scope boundaries (intentional, called out per the no-silent-caps rule)

  • On-demand only — not wired to the scheduler or a threshold auto-trigger yet.
  • Backend + endpoint only — no SPA surfacing yet (a "Clean up / health" action on the detail page is the natural follow-up).
  • No merge / evict / stale-detection — deferred to the Workstream B LLM pass.

Tests

  • 8 service (moto): healthy report, orphan GC + applyGc=false skip, duplicate-reported-without-merge, dead-link report, stripDeadLinks unlinks-but-keeps-prose, over-cap flag (no eviction), editor gate.
  • 4 route: report shape + GC through the route, empty body, viewer 403, flag-off 404.
  • 3 store: list_keys prefix scoping / empty space / disabled.

104 memory + import-boundary tests green.

Spec

docs/specs/user-markdown-memory.md — A6 marked ✅ (deterministic slice), with the LLM pass explicitly deferred to Workstream B on the same seam.

🤖 Generated with Claude Code

The safe, non-LLM slice of Workstream A6. `MemorySpaceService.consolidate`
(editor+) + `POST /memory/spaces/{id}/consolidate` → a `ConsolidationReport`.

Auto-fixes only storage hygiene: orphaned content-addressed objects — keys
under a space's prefix that no manifest entry or the index pointer references
(leaks from crashed/raced writes) — are GC'd (new `MemorySpaceStore.list_keys`
drives it). Everything that needs a judgment call is *reported, not mutated*:
- duplicate content across slugs (same content hash) — which slug survives is
  semantic, so it's flagged, never auto-merged;
- dead `[[slug]]` wikilinks in MEMORY.md — reported; opt-in `stripDeadLinks`
  unlinks them (they point nowhere) while preserving the surrounding prose;
- over-cap entry counts (`MEMORY_SPACE_INDEX_CAP`, default 200) — flagged, never
  auto-evicted.

This deliberately does not merge/evict/rewrite durable memory — that's deferred
to the LLM consolidation pass (Workstream B era), which extends this exact
`consolidate()` seam once agentic writes create real duplication/staleness to
act on. On-demand only for now; scheduler/threshold auto-run and SPA surfacing
are follow-ups.

Tests: 8 service (healthy report, orphan GC + skip, dup-report-no-merge,
dead-link report + strip-keeps-prose, over-cap flag, editor gate) + 4 route
(report shape, no-body, viewer 403, flag-off 404) + 3 store (list_keys prefix
scoping / empty / disabled). 104 memory + boundary tests green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@philmerrell philmerrell merged commit 2c0cbd4 into develop Jul 7, 2026
4 checks passed
@philmerrell philmerrell deleted the feature/memory-a6-consolidation branch July 7, 2026 22:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant