Skip to content

feat: add quality filter to mem_save and passive capture#34

Merged
Alan-TheGentleman merged 2 commits intoGentleman-Programming:mainfrom
jmeiracorbal:feat/quality-filter-on-save
Mar 13, 2026
Merged

feat: add quality filter to mem_save and passive capture#34
Alan-TheGentleman merged 2 commits intoGentleman-Programming:mainfrom
jmeiracorbal:feat/quality-filter-on-save

Conversation

@jmeiracorbal
Copy link

@jmeiracorbal jmeiracorbal commented Mar 5, 2026

Summary

Add quality filter to mem_save and passive capture.

Motivation

This idea is extracted from rtk-ai/rtk.

Changes

  • Reject observations with title shorter than 3 chars
  • Reject content with fewer than 5 words
  • Reject known noise/placeholder values (ok, done, todo, etc.)
  • Add word count minimum (4 words) to ExtractLearnings for passive capture
  • Update existing tests to use content that meets the quality threshold

Test plan

  • go test ./... passes

Closes #35

- Reject observations with title shorter than 3 chars
- Reject content with fewer than 5 words
- Reject known noise/placeholder values (ok, done, todo, etc.)
- Add word count minimum (4 words) to ExtractLearnings for passive capture
- Update existing tests to use content that meets the quality threshold

This idea is extracted from rtk-ai/rtk (https://github.com/rtk-ai/rtk)
Copy link
Collaborator

@Alan-TheGentleman Alan-TheGentleman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need changes

Copy link
Collaborator

@Alan-TheGentleman Alan-TheGentleman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, appreciate the contribution! I like part of this but want to scope it down.

The ExtractLearnings word count — keep that. Passive capture parses unstructured text so filtering noise there makes total sense.

The handleSave quality gate — I'd like to remove this part. Here's my thinking:

  • mem_save is called by the agent, not humans. If the agent sends garbage, the real fix is better protocol instructions, not server-side validation on AI-generated content.
  • The 5-word minimum is too low to catch actual quality problems. "decided to use JWT" passes — is that quality? The real noise is verbose observations that say nothing useful, and this filter won't catch those.
  • The noise set is English-only. We have Spanish-speaking users ("listo", "hecho", "dale" all bypass it). It'd need constant maintenance across languages.
  • Failed quality checks mean the agent retries with more text = more tokens burned for something that shouldn't happen if the protocol works.

What to keep: minLearningWords in ExtractLearnings + related test updates.

What to remove: quality.go, the checkObservationQuality call in handleSave, the TestCheckObservationQuality* / TestHandleSaveRejects* tests, and revert the test content padding in TestHandleSaveCreatesProjectScopedSession and TestExplicitSessionIDBypassesDefault back to their originals (those were only padded to pass the gate being removed).

Let me know if you have questions!

@jmeiracorbal
Copy link
Author

Hey, appreciate the contribution! I like part of this but want to scope it down.

The ExtractLearnings word count — keep that. Passive capture parses unstructured text so filtering noise there makes total sense.

The handleSave quality gate — I'd like to remove this part. Here's my thinking:

  • mem_save is called by the agent, not humans. If the agent sends garbage, the real fix is better protocol instructions, not server-side validation on AI-generated content.
  • The 5-word minimum is too low to catch actual quality problems. "decided to use JWT" passes — is that quality? The real noise is verbose observations that say nothing useful, and this filter won't catch those.
  • The noise set is English-only. We have Spanish-speaking users ("listo", "hecho", "dale" all bypass it). It'd need constant maintenance across languages.
  • Failed quality checks mean the agent retries with more text = more tokens burned for something that shouldn't happen if the protocol works.

What to keep: minLearningWords in ExtractLearnings + related test updates.

What to remove: quality.go, the checkObservationQuality call in handleSave, the TestCheckObservationQuality* / TestHandleSaveRejects* tests, and revert the test content padding in TestHandleSaveCreatesProjectScopedSession and TestExplicitSessionIDBypassesDefault back to their originals (those were only padded to pass the gate being removed).

Let me know if you have questions!

Hey, appreciate the contribution! I like part of this but want to scope it down.

The ExtractLearnings word count — keep that. Passive capture parses unstructured text so filtering noise there makes total sense.

The handleSave quality gate — I'd like to remove this part. Here's my thinking:

  • mem_save is called by the agent, not humans. If the agent sends garbage, the real fix is better protocol instructions, not server-side validation on AI-generated content.
  • The 5-word minimum is too low to catch actual quality problems. "decided to use JWT" passes — is that quality? The real noise is verbose observations that say nothing useful, and this filter won't catch those.
  • The noise set is English-only. We have Spanish-speaking users ("listo", "hecho", "dale" all bypass it). It'd need constant maintenance across languages.
  • Failed quality checks mean the agent retries with more text = more tokens burned for something that shouldn't happen if the protocol works.

What to keep: minLearningWords in ExtractLearnings + related test updates.

What to remove: quality.go, the checkObservationQuality call in handleSave, the TestCheckObservationQuality* / TestHandleSaveRejects* tests, and revert the test content padding in TestHandleSaveCreatesProjectScopedSession and TestExplicitSessionIDBypassesDefault back to their originals (those were only padded to pass the gate being removed).

Let me know if you have questions!

Hello Alan!

I removed:

  • quality.go and checkObservationQuality: The server-side validation on AI generated content was on the wrong layer. The quality about the observations will be depend of the protocol and the protocol works if the agent charges the tools (fix of deferred tools fix: load deferred memory tools at session start in Claude Code #43) and reads the SKILL.md instructions. A better protocol instructions are more effective than rejecting saves, it's true.
  • TestCheckObservationQuality* and TestHandleSaveRejects* tests

I revert test content padding in TestHandleSaveCreatesProjectScopedSession and TestExplicitSessionIDBypassesDefault too.

@Alan-TheGentleman Alan-TheGentleman merged commit 89142bf into Gentleman-Programming:main Mar 13, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature: Quality filter for mem_save and passive capture as rtk-ai

2 participants