RAG Pipeline — Retrieval Augmented Generation

Author: Sri Harshini Vemula | Software Engineer @ Palantir
Stack: LangChain · FAISS · OpenAI GPT-4 · HuggingFace · Python

A production-grade Retrieval Augmented Generation (RAG) pipeline that ingests documents, embeds them into a semantic vector store, and answers natural language questions with cited sources.

---

Architecture

Documents (PDF/TXT)
       │
       ▼
  Document Loader
       │
       ▼
  Text Chunker  ←── RecursiveCharacterTextSplitter (800 tokens, 150 overlap)
       │
       ▼
  Embeddings   ←── OpenAI text-embedding-3-small  OR  HuggingFace MiniLM
       │
       ▼
  FAISS Index  ←── Persisted locally, MMR retrieval
       │
       ▼
  GPT-4 Chain  ←── Custom prompt, temperature=0, top_k=5
       │
       ▼
  Answer + Sources

---

Features

• Multi-format ingestion — PDF, TXT, entire directories
• MMR Retrieval — Maximal Marginal Relevance reduces redundant context
• Dual embedding backends — OpenAI (production) or HuggingFace (free/offline)
• Cited answers — Returns source doc + page number for every answer
• Persistent index — FAISS index saved to disk, no re-embedding on restart
• Custom RAG prompt — Tuned to prevent hallucination and encourage source citation

---

Quickstart

# 1. Clone & install
git clone https://github.com/shvemula09/rag-document-qa
cd rag-document-qa
pip install -r requirements.txt

# 2. Set your OpenAI key (or skip to use HuggingFace embeddings)
export OPENAI_API_KEY=sk-...

# 3. Add your documents
mkdir sample_docs && cp your_file.pdf sample_docs/

# 4. Run
python rag_pipeline.py sample_docs/

---

Requirements

langchain>=0.1.0
langchain-openai
faiss-cpu
sentence-transformers
pypdf
openai

---

Key Design Decisions

Decision	Choice	Why
Chunking	Recursive + overlap	Preserves sentence/paragraph boundaries
Retrieval	MMR (not simple top-k)	Reduces redundancy in retrieved context
Embeddings	OpenAI / HuggingFace	Flexible — works with or without API key
Prompt	Zero-shot + citation instruction	Reduces hallucination
LLM temp	0.0	Factual Q&A needs determinism

---

Evaluation

Run the included eval script to measure Faithfulness, Answer Relevancy, and Context Recall using RAGAS:

python eval/ragas_eval.py --questions eval/test_questions.json

---

Project Structure

rag-document-qa/
├── rag_pipeline.py        # Core pipeline
├── eval/
│   └── ragas_eval.py      # RAGAS evaluation harness
├── sample_docs/           # Drop your PDFs here
├── faiss_index/           # Auto-generated vector store
└── README.md