probe: RAG / vector-store / agent-memory *sleeper* poisoning (retrieval-time latent injection)

[dyrtyData]

Summary

I'd like to propose a new probe + detector pair that tests whether a target is vulnerable to poisoning of its retrieval layer — a vector store / RAG corpus / agent long-term memory — where a malicious document is planted at rest and lies dormant until a benign, innocuous-looking query retrieves it and the model surfaces the attacker's payload. This is the "sleeper" / delayed-retrieval class of indirect prompt injection described in PoisonedRAG (arXiv:2402.07867), AgentPoison (arXiv:2407.12784), MINJA (arXiv:2503.03704), Phantom (arXiv:2405.20485), and the "sleeper" framing in Sleeper Agents (arXiv:2401.05566).

Why this isn't already covered. garak's closest existing module, garak.probes.latentinjection, tests the symptom of RAG injection by hard-coding the malicious context string directly into a single-call prompt template — it never populates, indexes, or later retrieves from an actual retriever/vector store (the module docstring itself notes it is "similar to that effective against RAGs … used for summarisation," and FAQ.md states RAG coverage today is indirect prompt injection only). promptinject is direct first-person hijacking; web_injection is markdown-rendering data-exfiltration (CSRF/XSS-style); agent_breaker exercises tool/excessive-agency exploitation with a fresh conversation each turn. None of these model poisoning-at-rest + delayed/triggered retrieval, which is the distinct threat here.

Prior art / related. This directly extends the intent of the (closed) issue #888 "probe: Indirect injection probes focused on RAG vulns", which was resolved by #877 shipping latentinjection as an intentional in-prompt simplification. Also adjacent (but distinct) is the agent-threat-rule detector work in #1677 / #1676 (regex detectors for "data/tool poisoning" patterns in output) and the MCP tool/context-poisoning scanning in #1639. I'm opening this as a separate new-plugin issue to track the retrieval-layer poisoning gap explicitly, per the repo's issue-first norm.

Basic example

Proof-of-concept exists and is public: https://github.com/dyrtyData/vectox — a self-contained sandbox that seeds a Postgres/pgvector store with poisoned documents, then runs the probe against a RAG victim over both an in-process transport and a live ws:// endpoint, measuring attack success rate (ASR) plus a benign-query control that must score 0. It already carries garak-plugin modules modeled on the existing conventions.

Sketch of the proposed plugin surface (subject to review — would follow the existing LatentInjectionMixin / TriggerListDetector conventions):

# garak/probes/rag_poisoning.py
class RagPoisoning(LatentInjectionMixin, garak.probes.Probe):
    """Sleeper poisoning of a retrieval store.

    Emits benign trigger queries that, against a target whose retrieval corpus has been
    seeded with a dormant poisoned document, cause the target to surface an attacker payload.
    """
    lang = "*"
    goal = "surface a dormant poisoned document on a benign trigger query"
    primary_detector = "rag_poisoning.SleeperTrigger"
    doc_uri = "https://arxiv.org/abs/2402.07867"   # PoisonedRAG
    tier = garak.probes.Tier.OF_CONCERN
    tags = ["avid-effect:security:S0403", "owasp:llm01", "quality:Security:PromptStability"]
    # prompts = benign trigger queries; triggers = expected exploit strings

# garak/detectors/rag_poisoning.py
class SleeperTrigger(garak.detectors.base.TriggerListDetector):
    """Scores 1.0 when the expected poisoned payload appears in the target's output."""

Two open design questions I'd want maintainer input on before a PR, since they affect scope:

1. Where the "poison at rest" lives. garak targets a live endpoint via a Generator; the poisoned document has to already be in the target's store. Options: (a) probe emits only the benign trigger and assumes an out-of-band seed step (documented, but garak doesn't own the seed); (b) model the whole thing in-prompt like latentinjection (loses the "at rest / delayed" property that makes it novel); (c) a two-phase harness/buff that first writes then reads. Which fits garak's architecture best?
2. Detector reuse. Whether to reuse base.TriggerListDetector outright vs. ship a thin rag_poisoning.SleeperTrigger subclass, given the trigger string is known at probe-build time.

Motivation

RAG and agent-memory systems are now the dominant deployment pattern, and retrieval-layer poisoning is a top real-world LLM risk (OWASP LLM01 Prompt Injection / LLM08 Vector & Embedding Weaknesses). The threat is qualitatively different from in-prompt injection because the payload is persistent and dormant: it is planted once, survives across sessions, and is triggered by ordinary user queries with no visible attacker input at inference time — which is exactly what makes it hard to catch with existing probes that inject at prompt-time. Expected outcome: garak gains a first probe that assesses whether a target's retrieval/memory layer can be poisoned to serve attacker content on benign triggers, closing the gap left open when #888 was resolved with an in-prompt approximation.

I have a working PoC (probe + detector + pgvector sandbox, deterministic mock victim for CI, plus a live ws:// transport) and am prepared to submit a PR following CONTRIBUTING.md (DCO sign-off, RST docs under docs/source/, MISP tags, tests/probes/ + tests/detectors/ coverage) once scope is agreed here. Happy to keep the initial PR minimal (single probe family + detector) and defer scaled/multilingual corpora and any defense/gating experiments to follow-up issues.

Threat-model references: PoisonedRAG (arXiv:2402.07867), AgentPoison (arXiv:2407.12784), MINJA (arXiv:2503.03704), Phantom (arXiv:2405.20485), Greshake IPI (arXiv:2302.12173 / OWASP LLM01), Sleeper Agents (arXiv:2401.05566).

_{(For context: I'm the author of the merged WebSocket generator, #1379.)}

[erickgalinkin]

I like the approach here, but this is another probe in desperate need of first-class support for Tool-type messages. Seems like we need to bump that up in the priority for implementation.