Skip to content

fix: scale issues — clustering 450x faster, long-message entity retention 2x#22

Merged
SimplyLiz merged 2 commits intodevelopfrom
fix/scale-issues
Mar 22, 2026
Merged

fix: scale issues — clustering 450x faster, long-message entity retention 2x#22
SimplyLiz merged 2 commits intodevelopfrom
fix/scale-issues

Conversation

@SimplyLiz
Copy link
Owner

Summary

Fixes three issues found by the new scale test (bench/scale-test.ts, 51 tests):

1. Semantic clustering O(n²) blowup — 450x faster

  • Pre-compute pairwise similarity matrix instead of recalculating on each merge
  • Cap at 200 candidates (most recent messages) to bound the quadratic cost
  • Before: 76ms/msg at 500 messages (38 seconds total)
  • After: 0.17ms/msg at 500 messages (83ms total)

2. Long-message entity retention — 2x improvement at 10k chars

  • Summary budget cap now scales logarithmically for content >5k chars
  • Entity suffix allows up to 30 entities for long messages (was 15)
  • 10k chars: entity retention 55% → 91%
  • 50k chars: entity retention 18% → 36% (inherent limit — 200+ identifiers can't fit in any summary)

3. Binary search vs tiered documentation

  • Added performance note: tiered is 7-8x faster at 500+ messages
  • Recommend tiered for large conversations in TSDoc

Remaining known limits (by design, not bugs)

  • 50k+ char messages: entity retention capped ~36% — the summary budget ceiling can't represent all identifiers. Would need chunked compression (architectural change)
  • Pure-code messages: 1.00x compression — T0 preserves code fences verbatim (correct)
  • Short messages (<120 chars): 1.00x — too small to summarize (correct)
  • Multi-round: converges after round 1 — already_compressed guard (correct)

Test plan

  • 663 unit tests pass
  • 51 scale tests pass (zero round-trip failures)
  • Baseline saved
  • Lint and format clean

- Semantic clustering: pre-compute pairwise similarity matrix instead
  of recomputing per merge iteration. Cap at 200 candidates (most
  recent). 450x faster at 500 messages (38s → 83ms)
- Long message budget: scale cap logarithmically for content >5k chars.
  At 10k: 920 chars (was 600). At 50k: ~1300. At 100k: ~1500. Doubles
  entity retention at 10k chars (55% → 91%)
- Entity suffix: scale max entities to 30 for long messages (was 15)
- Budget strategy docs: note that tiered is 7-8x faster than
  binary-search at 500+ messages, recommend for large conversations
- Add bench/scale-test.ts: 51-test scale analysis covering message
  count scaling, long messages, composition, recency window impact,
  budget strategies, v2 features at scale, pathological patterns,
  and multi-round compression convergence
@SimplyLiz SimplyLiz merged commit ccff17f into develop Mar 22, 2026
9 of 11 checks passed
@SimplyLiz SimplyLiz deleted the fix/scale-issues branch March 22, 2026 16:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant