RAG-based Local Knowledge Base QA System
Offline-capable question answering over your own documents, no cloud API required.
Overview
A Retrieval-Augmented Generation system that answers questions from a growing library of ingested PDFs, running entirely on local infrastructure. It's built for multi-turn, streaming chat, with adjustable retrieval settings and an intent-based refusal mechanism that declines out-of-scope questions instead of guessing.
How It Works
- A BERT-based intent classifier routes each incoming question.
- Hybrid retrieval runs vector search (ChromaDB) alongside BM25 keyword search, fusing the two rankings with Reciprocal Rank Fusion (RRF).
- Sibling-chunk expansion pulls in neighbouring context around each match.
- A cross-encoder reranks the expanded candidates for relevance.
- A local LLM (Ollama, running Qwen2.5-7B) generates the final answer, streamed back to the chat.
Features
- Streaming, multi-turn chat
- Adjustable retrieval parameters
- Expandable knowledge base — just ingest more PDFs
- Source-citation display alongside every answer
- Intent-based refusal for out-of-scope questions
Tech Stack
Python, Gradio, ChromaDB, BM25, a cross-encoder reranker, and Ollama (Qwen2.5-7B).