A fully offline, privacy-first Retrieval-Augmented Generation (RAG) command-line application built with .NET and Ollama. This application extracts text from a local PDF, processes it into semantic chunks, generates vectors, and allows you to ask questions about the document—all running 100% locally on your machine.
- Zero Cloud Dependencies: Your data never leaves your machine.
- Provider Agnostic: Uses the standardized
Microsoft.Extensions.AIabstractions. - In-Memory Vector Search: Fast cosine-similarity matching without needing a heavy database setup.
- Streaming Responses: Answers are streamed token-by-token in real-time.
-
Ollama Installed: Download and run Ollama from ollama.com.
-
Download the Models: Open your terminal and pull the required Large Language Model (LLM) and Embedding model:
ollama pull llama3.2 ollama pull all-minilm
-
.NET SDK: Ensure you have the .NET 8.0 SDK (or higher) installed.
📦 Initial Setup & Installation
- Clone the repository:
git clone [https://tinyurl.com/yx7scj62](https://tinyurl.com/yx7scj62)
cd LocalPdfRag- Restore and add the required NuGet packages:
# Core AI interfaces
dotnet add package Microsoft.Extensions.AI.Abstractions --prerelease
# Official Ollama implementation of the abstractions
dotnet add package OllamaSharp
# Open-source library for extracting text from PDFs
dotnet add package UglyToad.PdfPig- Run the Application:
dotnet run💡 Troubleshooting: 404 (Not Found) Error If you encounter an error stating Response status code does not indicate success: 404 (Not Found), it means the Ollama server is running but cannot find the exact model string specified in the code.
How to Fix:
-
Run ollama list in your terminal to see your downloaded models and their exact tags.
-
If your models have a specific tag (e.g., llama3.2:latest), update the constants at the top of your Program.cs file to match exactly:
private const string ChatModel = "llama3.2:latest"; private const string EmbeddingModel = "all-minilm:latest";