LLM-Powered-Semantic-Code-Search-Explainer: LLM-Powered Semantic Code Search Explainer: Why Code RAG Has to Think in ASTs, Not Chunks

The pitch sounds like search, but the design is really about meaning preservation. Most code RAG systems fail the same way: they chop source into generic chunks, then hope embeddings can recover the lost structure. This repo takes the opposite view. Code is hierarchical, so retrieval should be hierarchical too.

That sounds obvious until you try searching for something like auth_provider_v2 inside a class method buried in a file with three related helpers. If the system loses the function boundary, the file path, or the exact identifier, the answer may be semantically close and practically useless. This project exists to prevent that failure mode.

Why code search breaks when you treat code like text

The first mistake is plain chunking. A token window can split a function in half, separate a docstring from the method it describes, or sever an import from the logic that depends on it. Source code is not natural language with braces on top. Its structure carries meaning.

# Bad chunking can lose meaning
class AuthService:
    def authenticate(self, token):
        if not token:
            raise ValueError("Missing token")
        return self.verify(token)

# If this gets cut mid-function, retrieval loses the boundary

Dense embeddings alone do not fix that. They are good at related concepts. They are not reliably good at exact identifiers, file-level context, or the difference between a helper and a public API. For code, exact tokens still matter.

The AST chunker: the repo’s first big idea

The core move is to use tree-sitter to walk the syntax tree instead of splitting text by size. That means functions, methods, and class definitions survive as whole units. The chunker can descend into a class, capture its methods, and stop at sensible boundaries instead of recursing forever into nested details.

That structure matters because code is already organized around intent. A class is a namespace, a method is a capability, and an import is a clue. If you flatten those into generic slices, you lose the scaffolding that helps a model answer code questions correctly.

Contextual enrichment fixes the lost in space problem

The next layer is enrichment. The repo prepends metadata such as file path, module name, function signature, and imports before embedding the code. That helps the model understand where a snippet lives, not just what words it contains.

There is a second tier too. The system can optionally add an LLM-generated summary, which gives search another shot at semantic alignment without replacing the raw source. The UI still shows the original code. The enrichment is for retrieval, not for display.

def build_context_header(file_path, module_name, signature, imports):
    return (
        f"File: {file_path}\n"
        f"Module: {module_name}\n"
        f"Signature: {signature}\n"
        f"Imports: {', '.join(imports)}\n"
    )

# Header plus code gets embedded.
# Raw code still gets returned to the UI.

That separation is a subtle but important design choice. It lets the search layer become more opinionated without contaminating the user-facing source view. The model gets context. The developer gets the real code.

Hybrid retrieval is the real search engine

This repo does not bet on one retrieval method. It runs dense vector search and BM25 in parallel, then combines them with reciprocal rank fusion. That is the right call because semantic scores and keyword scores solve different problems and live on different scales.

In plain English, reciprocal rank fusion says: if a result ranks well in both systems, lift it. If one method loves it and the other ignores it, do not trust either signal alone. That is especially useful in code, where a query like “how do we verify JWTs?” might point to a semantically related helper, while the real answer is sitting behind a specific identifier like JWT_SECRET_KEY.

Why the system is surprisingly transparent

The frontend exposes the raw ingredients of the final ranking: vector_score, bm25_score, and fusion_score. That is a glass-box move. It makes the system easier to debug and easier to trust.

If a result looks wrong, you can see whether semantics overpowered keywords, whether keyword matching rescued a weak embedding, or whether the fusion logic did exactly what it was supposed to do. For a developer tool, that kind of transparency is not cosmetic. It is the product.

What this repo gets right, and where it stops

As an educational build, the architecture is strong. It uses modern Python, a clean service split, and a realistic stack with FastAPI, ChromaDB, MongoDB, and a React frontend. It shows how a small team can assemble a credible code-intelligence system without pretending the hardest problems have disappeared.

The limit is scale. ChromaDB is a sensible choice for local or small-team usage, but a larger multi-tenant search product would eventually need sturdier infrastructure and more operational discipline. That is not a flaw. It is the boundary between a strong reference implementation and a production platform.

How it compares to full products and frameworks

Against Bloop, Sourcegraph Cody, and framework stacks like LangChain or LlamaIndex, this repo occupies a different category. It is not trying to be the destination. It is trying to explain the route.

That distinction matters. A product hides complexity so users can get value. A framework exposes primitives so builders can compose them. This repo sits in the middle as a teaching artifact that shows why the primitives are arranged the way they are.

Why this matters beyond one repo

The broader lesson is simple. Good code intelligence does not start with bigger embeddings. It starts with preserving structure, adding context, and making ranking explainable. Those three moves turn a vague semantic search demo into something that can actually guide a developer.

That is why the repo is interesting even if you never ship its exact stack. It gives you the shape of the problem. Code is not prose. Retrieval should not pretend otherwise.