Streaming PII proxy for LLM chat — with cryptographically-anchored reversal.
veilstream sits between your chat UI and the LLM. On the way out it detects PII, mints realistic pseudonyms, and keeps a per-session vault. On the way back it streams the LLM's response with pseudonyms reversed live as chunks arrive — tolerating pseudonyms that straddle chunk boundaries, partial prefixes, and all the edge cases that trip up existing libraries.
It introduces PASP — Provenance-Anchored Streaming Pseudonymization — a new algorithm that closes the false-attribution attack every prior library silently suffers from.
user text user sees
│ ▲
▼ │
┌────────────┐ detect + wrap: │
│ detector │ "I'm Jordan" ──► "I'm ⟦hla4:Allison⟧" │
└─────┬──────┘ │ │
│ ▼ │
│ ┌─────────────────┐ │
│ │ LLM │ │
│ │ (sees fakes │ │
│ │ only, returns │ │
│ │ "Hi ⟦hla4: │ │
│ │ Allison⟧!" │ │
│ └────────┬────────┘ │
│ │ │
▼ ▼ │
┌─────────────┐ stream + unwrap (anchor-verified): │
│ vault + │ "Hi ⟦hla4:Allison⟧!" ──► "Hi Jordan!" ───┘
│ deanon trie │
└─────────────┘
│
⟦a⟧ = HMAC(session_key, pseudonym)[:4] ◄── only reversed if anchor matches
| measured | |
|---|---|
| Streaming-reversal overhead | <0.1 ms per chunk |
| Detector latency (medium prompt, CPU) | 150 ms median |
| False-attribution attack rate — legacy reverser, first-name Faker vault, 6 LLMs | 2–32% of responses (median 12%) |
| False-attribution attack rate — PASP | 0% (cryptographically blocked) |
| PASP anchor retention — 13 frontier LLMs with system-prompt hint | median 100%, mean 96.4% (8 of 13 at exactly 100%) |
| PASP utility cost vs unwrapped baseline (word count, engagement) | within generation noise — LLM-as-judge C-win 46% (neutrality band) |
| Core tests | 85 passing (incl. 500+ Hypothesis property cases) |
Full numbers in docs/BENCHMARKS.md.
pip install veilstream # core
pip install "veilstream[redis]" # optional Redis-backed session storefrom veilstream import PrivacyProxy
proxy = PrivacyProxy(
detector="openai/privacy-filter",
faker_seed=42,
anchored=True, # enable PASP — recommended
)
session = proxy.session(session_id="conv_abc123")
safe_prompt = session.pseudonymize("Hi, I'm Jordan Reed (jordan@example.com).")
# safe_prompt == "Hi, I'm ⟦hla4:Allison Hill⟧ (⟦m3kt:donaldgarcia@example.net⟧)."
messages = [
{"role": "system", "content": proxy.system_prompt_hint()},
*session.pseudonymized_history,
{"role": "user", "content": safe_prompt},
]
resp = await client.chat.completions.create(model="gpt-4o-mini", messages=messages)
final = session.deanonymize(resp.choices[0].message.content)
# Real values restored for properly-anchored mentions;
# bare coincidental pseudonym strings in LLM output are *not* reversed.
session.record_assistant_turn(resp.choices[0].message.content)Streaming works the same way — pass the LLM's async iterable to
session.deanonymize_stream(chunks) and yield the result to the user.
Every existing streaming PII-reversal library (Presidio, LLM Guard,
LiteLLM-Presidio, LangChain PresidioReversibleAnonymizer) has the same
latent bug: false attribution.
If the LLM generates a name coincidentally matching one of your pseudonyms — "…for instance, someone named Alice Smith would…" — the naive reverser substitutes the real value in, mis-attributing generic LLM text to your real user. That's a privacy failure inserted by the gateway.
We measured this empirically across 6 LLMs with first-name-pseudonym vaults: the attack fires in 2–32% of responses to generic creative-writing prompts — median 12%, with Llama 3.3 70B as the worst case and Claude Opus 4.7 as the best. Full-name pseudonyms are accidentally safer (0% on venice-uncensored), but nothing in the library guarantees that — switching Faker's locale or your chosen LLM can re-expose the vulnerability.
PASP fixes it cryptographically. Each reversible pseudonym goes out
as ⟦{anchor}:{pseudonym}⟧ where anchor = HMAC-SHA256(session_key, pseudonym)[:4] in base32. The deanonymizer only reverses
fully-anchored matches. An LLM without the session key cannot forge a
valid anchor — so its coincidental pseudonym mentions stay as fake
names. Forgery probability: ~10⁻⁶ per inspection position under the
HMAC-PRF assumption. See docs/CORRECTNESS.md §2.
When the LLM strips the wrapper (paraphrases, quotes without the
markers) the deanonymizer declines to reverse — the user sees the fake
name. This is a soft UX degradation, not a privacy leak. Adding
proxy.system_prompt_hint() to your system message brings retention
to median 100% / mean 96.4% across the 13 frontier LLMs we tested
(8 at exactly 100%; the Llama 3.3 70B outlier sits at 74.6%).
- Chat UIs that can't send customer PII to a third-party LLM. Your app still sees real names / emails / phones; the LLM sees only fakes; the user sees real values in the streamed response.
- Audit + incident response.
session.vault_snapshot()gives you the full real ↔ pseudonym map, scoped to that conversation. - Per-tenant isolation. Same real entity → different pseudonyms
across different
session_ids. Cross-session leakage is structurally impossible. - Secrets discipline. API keys and account numbers are redacted irreversibly — they cannot round-trip even if the LLM echoes the redaction placeholder.
- Defense against LLM hallucination collisions. With
anchored=True, an LLM producing a pseudonym-shaped string by coincidence cannot cause your gateway to mis-attribute it to a real user.
| Label | Operator | Reversible |
|---|---|---|
private_person |
faker.name() |
✓ |
private_email |
faker.email() |
✓ |
private_phone |
faker.phone_number() |
✓ |
private_address |
coarse City, ST |
✓ |
private_url |
stable hash alias | ✓ |
private_date |
consistent session delta ±30 days | ✓ |
account_number |
[REDACTED_ACCOUNT] |
✗ |
secret |
[REDACTED_SECRET] |
✗ |
Policy.default() ships the above. Policy.strict() redacts everything
irreversibly. In anchored mode, irreversible categories are never
wrapped — they cannot be reversed even syntactically.
The deanonymizer maintains a trie of the session's pseudonym → real map and a frontier of active trie walks. A buffer position is confirmed safe to emit only when every surviving walk is at a leaf node — no future token could extend any pending match. Invariant: streaming output equals batch output for any chunking.
Edge cases covered by tests/test_stream.py:
- Pseudonym split across chunks.
- False-positive prefix (
JordachevsJordan). - Overlapping pseudonyms — longest match wins.
- Pseudonym at the very end of the stream with no trailing whitespace.
- Repeated pseudonyms in one response.
- Unicode and emoji inside pseudonyms.
Exercised with @given over random mappings and random chunk boundaries
(~500+ Hypothesis cases per CI run).
Detector, Policy, operators, and SessionStore are all protocols —
swap any one of them in a line.
# Custom detector
class MyDetector:
def detect(self, text):
return [{"label": "private_person", "start": 0, "end": 3, "text": "..."}]
# Redis-backed store (optional extra)
import redis
from veilstream.stores.redis import RedisStore
store = RedisStore(redis.from_url("redis://localhost"))
proxy = PrivacyProxy(detector=MyDetector(), policy=custom_policy, store=store)examples/openai_proxy.py— OpenAI single-turn + streaming.examples/anthropic_proxy.py— Anthropic Messages streaming.examples/venice_demo.py— Venice end-to-end demo.examples/fastapi_gateway.py— drop-in OpenAI-compatible proxy server.
Each one wires up the full pipeline in ~40 lines. See
examples/README.md.
docs/CORRECTNESS.md— streaming-equivalence proof + HMAC-unforgeability reduction.docs/THREAT_MODEL.md— actors, attacks A1–A7, residuals.docs/BENCHMARKS.md— reproducible latency + retention + attack-fire numbers.docs/RELATED_WORK.md— positioning vs Presidio / LLM Guard / LiteLLM / LangChain + academic anchors.docs/ABSTRACT.md— working paper abstract.CHANGELOG.md— version history.CITATION.cff— for academic use.
Scripts live in benchmarks/ — each reproducible with
nothing but a Venice API key:
retention.py— wrapper / anchor-length matrix (270 calls).retention_hard.py— free-form-task retention on one LLM (50 calls).retention_multi_model.py— 7 Venice models in parallel (350 calls).retention_frontier.py— 13 frontier LLMs (Claude Opus 4.7 & Sonnet 4.6, GPT-5.4 / 5.4-mini / 4o, Gemini 3 Pro & Flash, Grok 4.20 & 4.1-fast, Llama 3.3 70B, Qwen 235B instruct & thinking, GLM 4.7).attack_frequency.py/attack_frequency_firstname.py— attack-fire rate on one LLM (full-name / first-name vaults).attack_frequency_multi.py— attack-fire across 6 LLMs (600 calls).utility.py— A/B/C word-count / engagement / confusion heuristics (450 calls).utility_judge.py— pairwise LLM-as-judge utility comparison (450 calls).
Full results: docs/BENCHMARKS.md.
# Install
pip install veilstream
pip install -e ".[dev]" # for development
# Run unit tests
pytest # 85 passing
# Run live integration (downloads ~3 GB model)
OPF_RUN_INTEGRATION=1 pytest tests/test_integration.pyPython ≥ 3.10. Core deps: transformers, torch, faker.
In scope. Pluggable streaming PII proxy, multi-turn session vault, cryptographic provenance anchoring, per-category operators, Redis backing, property-based correctness tests.
Out of scope. General PII framework, bundled LLM client, web server, image/PDF support, compliance claims. This is a data-minimization tool; it does not certify anonymization.
Apache 2.0. See LICENSE.
If you use veilstream or PASP in academic work, CITATION.cff
has the canonical entry. Issues and PRs welcome.