Sahil Shinde
Resume (opens in new tab)
Back to selected work
I.iiApplied AI / RAG

RagLens Evaluation-First RAG System

A retrieval pipeline that narrows, reranks, and evaluates evidence instead of treating a generated answer as proof that retrieval worked.

Stack
  • FastAPI
  • LangChain
  • Qdrant
  • HuggingFace
  • CrossEncoder
  • RAGAS
  • React
  1. PDF — Source document
  2. Chunking
  3. Embeddings — 384 dimensions
  4. Qdrant retrieval — Up to 60 candidates
  5. Metadata filtering
  6. MMR — Up to 40 candidates
  7. CrossEncoder — Up to 8 final chunks
  8. LLM context — Final context chunks
  9. RAGAS evaluation — Measures retrieval results
Documents become 384-dimensional embeddings, retrieval produces up to 60 candidates, MMR keeps up to 40, and CrossEncoder reranking selects up to 8 context chunks before evaluation.
I

Overview

Many RAG projects stop after generating an answer. RagLens keeps retrieval evaluation inside the engineering process.

II

Architecture

PDF content is chunked into 384-dimensional embeddings, retrieved from Qdrant, filtered, diversified with MMR, reranked with CrossEncoder, passed into LLM context, and evaluated with RAGAS.

III

Decisions

Retrieve up to 60 candidates, use MMR to select up to 40, and rerank to up to 8 final context chunks with CrossEncoder.

IV

Measured outcomes

Context Precision moved from 0.568 to 0.792.

Evidence hit rate moved from 83.3% to 100%.

V

Testing

RAGAS evaluation keeps retrieval quality visible alongside the generated result.

Sahil Shinde