feat: add FlowRAG workflow memory & retrieval module with ChromaDB ba…

[ardi1s]

Summary

Adds FlowRAG, an intelligent workflow memory and retrieval module to Crush.
It captures successful multi-step tool call sequences, stores them in a
ChromaDB vector database, and retrieves similar past workflows via semantic
search to accelerate future tasks.

How It Works

Trigger Detection (Dual Layer)

Primary — Skill-driven: FlowRAG registers as a built-in Skill
(internal/skills/builtin/flowrag/SKILL.md). The LLM automatically
identifies when to trigger it — when a multi-step task completes
without errors, when the user confirms satisfaction, or when the
user explicitly asks to save a workflow.

Secondary — Keyword matching: CompletionDetector recognizes explicit
markers ("task complete", "save workflow", "remember this") alongside
colloquial confirmation phrases ("ok", "搞定了", "done").

Data Flow

Task completed → prompt "Save workflow? (y/n)"
→ y: Segmenter extracts successful steps, skipping IsError retries
→ Embedding generated via OpenAI-compatible API or local trigram hash
→ Stored in ChromaDB collection "crush_workflows"
→ Next similar task: semantic search retrieves Top-K
→ Retrieved workflows injected as system prompt context

Embedding Strategy

Two embedding backends:

• OpenAIEmbeddingClient: calls any OpenAI-compatible /embeddings endpoint
  (e.g. text-embedding-3-small, DeepSeek, Ollama)
• HashEmbeddingClient: zero-dependency trigram-hash embedding that
  produces content-aware vectors locally — no API key required

Vector Storage

VectorStoreBackend interface with two implementations:

• ChromaDBStore: REST API v2 client, auto-creates "crush_workflows"
  collection on first use
• JSONFileStore: local JSON file persistence, zero external dependencies,
  used as automatic fallback when ChromaDB is unavailable

Changes

internal/flowrag/
── detector.go Completion marker detection (Skill + keyword)
── segmenter.go Workflow segmentation with error-step filtering
── store.go Vector store interface, ChromaDB + JSON backends,
OpenAI + Hash embedding clients
── retriever.go Semantic search + system prompt context builder
── workflow.go Orchestrator with unified public API
── workflow_test.go 17 unit tests covering all components
── cmd/demo/main.go Interactive CLI demo with trigram-hash embeddings
(zero-dependency, runs without any API key)
── cmd/e2e_test/main.go End-to-end semantic search verification script
└── README.md Full English documentation

internal/skills/builtin/flowrag/SKILL.md
Built-in Skill definition — LLM autoloads this to know when FlowRAG
should trigger. Describes save/retrieve protocol, ChromaDB config,
and E2E testing procedure.

.gitignore
Added crush.local.json and flowrag_workflows.json to prevent
accidental credential / local data leaks.

Testing

Unit Tests

$ go test ./internal/flowrag/... -count=1 -v

PASS: TestCompletionDetector_Match (15 sub-cases)
PASS: TestTaskCompleteMarker (6 business keyword cases)
PASS: TestShouldTriggerFlowRAG
PASS: TestSegmenter_SuccessfulFlow
PASS: TestSegmenter_ExcludeErrorSteps
PASS: TestWorkflow_ToText
PASS: TestJSONFileStore_InsertAndSearch
PASS: TestJSONFileStore_SearchEmpty
PASS: TestCosineSimilarity (4 precision checks)
PASS: TestRetriever_BuildContextPrompt
PASS: TestRetriever_BuildContextPromptEmpty
PASS: TestWorkflowManager_Integration
PASS: TestTruncate
PASS: TestMustMarshalSteps

17 tests, all passing. Covers: detector, segmenter, both store
backends, cosine similarity precision, retriever, end-to-end
integration.

E2E Semantic Search

$ go run ./internal/flowrag/cmd/e2e_test/

Inserts 6 cross-domain workflows (auth fix, REST API, Python CSV
parser, DB migration, OAuth refresh, Docker Compose) then runs
semantic queries:

"fix a login authentication bug" → #1 auth-login-fix ✓
"python parse csv data" → #1 python-csv-parser ✓
"user registration API endpoint" → #1 rest-api-register ✓
"database migration SQL" → #1 db-migration ✓
"OAuth token refresh expired" → #1 oauth-token-refresh ✓

Interactive Demo

$ go run ./internal/flowrag/cmd/demo/

Fully interactive CLI demo using the HashEmbeddingClient (zero
API key required). Demonstrates insert/search/list operations
with natural language queries.

All existing tests continue to pass — no modifications to
existing code paths outside the new package.

gofmt / go vet: clean

[charmcli]

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

[ardi1s]

I have read the Contributor License Agreement (CLA) and hereby sign the CLA.