Skip to content

RAG Phase 3: RAG eval in the shared #205 eval package #321

Description

@VforVitorio

Sub of Epic #318. documents/audits/AUDIT_RAG_LAYER.md Phase 3 (size M).

Scope: RAG eval inside the shared src/strategy/eval/ package (#205), not a parallel one.

  • Wire a reproducible RAG retrieval eval: ~30 queries including the 3 production question shapes, computing P@k / MRR + a wrong-year rate + a citation-match rate. Revives the one-shot unwired N30B notebook (ML-eval E-11).

Deliverable: f1-eval rag (via the shared harness) reports retrieval quality + wrong-year + citation-match over a fixed query set.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: codebaseTouches src/, scripts/, or notebooks/auditTracks findings from a site/code auditpriority: P1High priority — fix this sprint

    Projects

    Status
    Todo

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions