LLMOps_series: MultiDocChat: The Session-Scoped RAG Template That Treats Ingestion, Retrieval, and Evaluation as One System

Why Most RAG Demos Break the Moment Two People Use Them

Most RAG tutorials solve the easy problem: one user, one upload, one chat thread. The minute you add another person, or another document set, the app starts leaking context in ways that are hard to debug and easy to ignore.

That is why this repo matters. `yashprogrammer/LLMOps_series` is really a lesson in how to keep a RAG system sane when the input is not a single PDF, but a stream of sessions, each with its own files, index, and chat history. The UI is just the front door.

The Session Boundary Is the Product

The architecture is built around `session_id`. Uploaded documents land in `data/<session_id>/`, and the corresponding FAISS index lives in `faiss_index/<session_id>/`. That is the difference between a notebook demo and something that can pretend to be a multi-tenant service.

The browser keeps the session key in localStorage, so the client keeps speaking the same identity back to the server. That small decision lets ingestion, retrieval, and chat history stay aligned without forcing every request through a heavyweight auth system.

# Conceptual shape of the isolation layer
session_id = get_or_create_session_id()
raw_dir = f"data/{session_id}"
index_dir = f"faiss_index/{session_id}"

# same session, same data path, same retriever state
ingestor = ChatIngestor(use_session_dirs=True)
vectorstore = FaissManager(index_dir=index_dir)
retriever = vectorstore.load_retriever(search_type="mmr")

That is the quiet win. The code makes the session boundary explicit instead of treating it as accidental state floating around the app.

Retrieval Is Not Just Search, It Is Question Rewrite, Then Search

The most interesting part of the retrieval flow is not the vector store. It is the LCEL chain that contextualizes the latest user message against chat history, then sends the rewritten question to FAISS, then hands the retrieved context back to the model.

That makes the system tolerant of pronouns, omissions, and follow-up questions. Instead of assuming the latest user turn contains enough information, it converts conversational context into a cleaner retrieval query first.

# Simplified retrieval sequence
standalone_question = contextualize(chat_history, user_input)
chunks = retriever.get_relevant_documents(standalone_question)
answer = generate_answer(chunks=chunks, chat_history=chat_history, question=standalone_question)

This is a small design choice with big consequences. It reduces the chance that the retriever anchors on the wrong fragment just because the user said "that one" or "the earlier part".

Why the Repo Prefers MMR

The default retriever choice is Maximal Marginal Relevance. That matters because it optimizes for coverage, not just closeness. The point is to avoid three near-duplicate chunks that all say the same thing in slightly different words.

That tradeoff is honest. MMR usually asks a little more of the system, but it gives the model a more varied evidence set. For RAG, that often beats raw nearest-neighbor purity.

The Adapter Pattern That Makes the Core Portable

The neatest engineering move in the repo is the adapter boundary. `FastAPIFileAdapter` wraps web uploads in a portable interface, while `LocalFileAdapter` lets the evaluation loop reuse the same ingestion path without pretending to be a browser.

That means the heart of the system does not belong to FastAPI. It belongs to the ingestion contract. Swap the interface, and the core stays intact.

# The point is portability, not the framework
adapter = FastAPIFileAdapter(upload_file)
# or
adapter = LocalFileAdapter(path_to_fixture)

result = ChatIngestor().ingest(adapter, session_id=session_id)

This is the kind of separation that makes a project easier to test, easier to extend, and easier to explain to the next engineer who inherits it.

The Ops Loop Is the Real Upgrade

The evaluation path is what pushes this repo beyond a clever demo. `run_evaluations.py` and LangSmith turn the same ingestion and retrieval pipeline into something measurable, not just shippable.

That matters because the best way to test a RAG app is often to run the same code path the user will hit. Separate mock logic is comforting, but it rarely tells you where the product actually breaks.

The repo does not just ask, "Can the model answer?" It asks, "Can this pipeline be run, replayed, and evaluated the same way every time?" That is a more serious question.

What This Stack Gets Right, and What It Still Leaves You to Solve

This repo is strongest where many RAG projects are weakest: state isolation, portable ingestion, and evaluation reuse. It is also honest about what it does not solve for you. Durable auth, long-lived chat persistence, and production-grade tenancy still need real system design around the scaffold.

That is not a flaw. It is the point. MultiDocChat is valuable because it exposes the joints instead of hiding them behind a managed abstraction.

If you want a template for learning how production RAG systems actually stay organized, this is the kind of repo worth studying.