Add Vertex AI Ranking API as reranker option

[harsh-kumar-patwa]

Fixes #91

Summary

• Adds Google Vertex AI Ranking API as an alternative reranker alongside the existing HuggingFace CrossEncoder
• Controlled via RERANKER_TYPE env var (HF or VERTEX_AI, defaults to HF)
• Uses langchain-google-community[vertexaisearch] which provides VertexAIRank — a drop-in DocumentCompressor for ContextualCompressionRetriever

Changes

• backend/src/chains/hybrid_retriever_chain.py — conditionally use VertexAIRank or CrossEncoderReranker based on RERANKER_TYPE env var
• backend/.env.example — added RERANKER_TYPE, VERTEX_AI_PROJECT_ID, VERTEX_AI_LOCATION
• backend/pyproject.toml — added langchain-google-community[vertexaisearch] dependency
• backend/tests/test_hybrid_retriever_chain.py — added tests for Vertex AI reranker path and missing project ID validation

Config

# Default (no change needed for existing setups)
RERANKER_TYPE=HF

# To use Vertex AI Ranking API
RERANKER_TYPE=VERTEX_AI
VERTEX_AI_PROJECT_ID=your-gcp-project-id
VERTEX_AI_LOCATION=global

Test plan

• All 15 hybrid retriever chain tests pass (including 2 new Vertex AI tests)
• All 18 unit tests pass
• Existing HF reranker behavior is unchanged when RERANKER_TYPE=HF (default)

[Copilot]

Pull request overview

Adds Google Vertex AI Ranking API as an optional reranker in the backend hybrid retriever chain, controlled via environment configuration, while keeping HuggingFace CrossEncoder as the default.

Changes:

• Add RERANKER_TYPE-based selection between HuggingFace CrossEncoder reranker and Vertex AI Ranking API (VertexAIRank).
• Add configuration knobs for Vertex AI (VERTEX_AI_PROJECT_ID, VERTEX_AI_LOCATION) to .env.example.
• Add dependency on langchain-google-community[vertexaisearch] and tests covering the Vertex AI reranker path + missing project ID validation.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
backend/src/chains/hybrid_retriever_chain.py	Select reranker implementation (HF vs Vertex AI) based on RERANKER_TYPE and configure VertexAIRank.
backend/tests/test_hybrid_retriever_chain.py	Add unit tests for Vertex AI reranker creation and validation error when project ID is missing.
backend/pyproject.toml	Add langchain-google-community[vertexaisearch] dependency for VertexAIRank.
backend/.env.example	Document new reranker-related environment variables.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

ensemble_retriever can be referenced here even when it was never assigned. If self.vector_db is None or processed_docs is empty, bm25_retriever is never set, which prevents the ensemble_retriever = EnsembleRetriever(...) block from running; later this code still uses ensemble_retriever, leading to UnboundLocalError. Initialize bm25_retriever/ensemble_retriever to None and either raise a clear error when the ensemble cannot be constructed or provide a fallback retriever composition before reaching this point.