RepoExplain Wraps a Small Local Model in a Deterministic Pipeline

Eight4bit's CLI answers questions about unfamiliar codebases on 8 GB of RAM, with no API keys and no cloud. Every design choice assumes the model is the least reliable part.

9 min read • View on GitHub • More from eight4bit

An editorial black-ink illustration of a small night workshop. In the foreground a hand-sized single-cylinder engine on a workbench turns a leather belt that drives a document-shuffling mechanism, while an engineer crouches beside it with a torque wrench. Through a large window behind, a massive jet turbine sits idle on a test stand, its intake facing away. The image contrasts a small, deliberately tuned machine with a large, unused one, standing in for RepoExplain's small local model against cloud-scale alternatives.
The whole thesis in one room: a small, hand-tuned machine doing the work while the giant turbine sits idle. RepoExplain would rather engineer around a 4B model than rent a 100B one.
Key Takeaways

The Model That Has To Leave Before The Next One Arrives

Buried in RepoExplain's indexer is a two-line decision that explains the entire project. Before the tool can answer a question, it unloads the embedding model to make room for the answering model.

if cfg["memory"]["unload_embedder"] and chunks:
    embedder.unload(ecfg["model"], kind="embed")

Two local models rarely fit in consumer RAM at the same time. A 4B parameter answer model and a 0.6B embedding model each hold weights, a context window, and working memory. On an 8 GB laptop the arithmetic does not close. So the code does the honest thing: it puts one model away before it takes the other out.

That single call is why the provider abstraction has an explicit unload method at all. Most model clients assume the model is always resident. RepoExplain assumes memory is scarce and choreographs around it. The rest of this article is a series of variations on that theme. Wherever the code could have handed a decision to the model, it usually hands it to a rule instead.

What "Local" Actually Costs

RepoExplain indexes a repository and then answers questions about it. The intended workflow is two commands: repoexplain index ./my-project, then repoexplain ask "Where is the password checked?". It talks to Ollama by default, or any OpenAI-compatible local server such as LM Studio or llama.cpp.

The default models are small on purpose: qwen3:4b-instruct for answers, qwen3-embedding:0.6b for vectors, and an optional qwen2.5-coder:1.5b for one-line file summaries. The stated floor is 8 GB of RAM. There are no accounts and no API keys, because nothing leaves the machine.

That constraint cascades. There is no database, just files under <repo>/.repoexplain/ and a brute-force cosine similarity search over a NumPy array. There is no tokenizer, so evidence is budgeted in characters rather than tokens. There is no reranker, so retrieval is a single embedding lookup plus a few heuristics.

The packaging betrays that real machines were involved. On macOS the project pins urllib3<2, because system Python links LibreSSL and urllib3 2.x wants OpenSSL 1.1.1 or newer. It is a one-line fix for a problem you only discover by running the thing on a Mac that already has Python installed. The README's audience is exactly that person: someone on consumer hardware who has inherited a codebase and would rather not paste it into a cloud chat box.

Rules Before Models

The router decides how to answer before any model is called. It is an ordered list of regex rules, and the order is the whole design.

def _rule_route(question, has_target):
    if DEPS_RX.search(question) and not has_target:     return "deps", ...
    if OVERVIEW_RX.search(question) and not has_target: return "overview", ...
    if LOCATE_RX.search(question):                       return "locate", ...
    if has_target:                                       return "explain", ...
    return "search", ...

The interesting part is has_target. A small helper scans the question for file paths, backticked tokens, and call-shaped names like name(. If it finds one, the broad deps and overview rules are suppressed. So "what does handle_login depend on?" does not trigger the repo-wide dependency report. It routes to explain, because the user named a symbol and almost certainly wants that symbol explained.

That precedence is a judgment encoded in code. A model asked the same question might reasonably produce the repo-wide report and bury the answer. The rule encodes what the user actually wants, and it does so before spending a single token. Symbol resolution is deliberately lenient: the index registers both the full dotted name (auth.Session.login) and the tail (login), so either form matches.

There is an opt-in escape hatch. Setting routing.mode: llm lets the model classify the question, with the rules as fallback when its reply is unusable. The default is the transparent rules. The model can do the work, but only if you ask it to, and never without a deterministic net underneath.

The router in motion. Notice that only one of five routes ever reaches the model without being asked, and the deps route never reaches it at all.

The Model Can Only Read What The Indexer Already Approved

The most important security property in the codebase lives in a short file called tools.py. When the model wants to read a file, it asks for a path, and the tool checks that path against the set of files the indexer already accepted.

def read_file(self, path, start=1, end=None, max_chars=3000):
    path = path.replace("\\", "/")
    if path not in self.files:   # whitelist: only indexed files
        return None
    full = (self.root / path).resolve()
    try:
        full.relative_to(self.root)
    except ValueError:
        return None

The whitelist check is the primary guard. Because the model can only request paths the indexer already accepted, .env files and paths like ../../etc/passwd are simply unreachable. The relative_to check is defence in depth: even if something slipped past the whitelist, it still could not resolve outside the repository root.

A tight black-ink close-up of two hands at a narrow compartment in a small machine, barely wide enough for one gear. The left hand lifts a small gear stamped E out of the slot while the right hand, waiting, holds a gear stamped A. The slot between them is empty. Crosshatching defines the knuckles and stippling defines the gear teeth. The image makes the model swap physical: one model leaves the slot so the next can enter, and the tightness of the compartment is the RAM constraint.
The unload call, made physical. One gear leaves the only slot wide enough to hold it, and the next one waits. That empty beat is the moment the code deliberately creates.

The front door is a scanner that walks the tree and decides what the indexer is allowed to see. It prunes ignored directories in place, so a node_modules folder is never descended into. Each file is checked in a fixed order: symlinks are skipped, filenames matching secret patterns (.env, *.pem, id_rsa*, credentials*) are skipped, gitignored files are skipped, and binaries and lockfiles are skipped.

There is a small, deliberate exception. .env.example passes through while .env and .env.local do not, because a template file is genuinely useful for explaining how configuration works. The rule is an explicit allowlist, and it is documented in the code.

Here is the honest caveat. That protection is by filename only. A hardcoded API key sitting inside config.py would be indexed, because the scanner never opens the file to look. The exposure is bounded: the index is local and gitignored, and the model is local too. But the phrase "skipping secret-like files" is narrower than it sounds. It means skipping files that look like secrets by name, not files that contain them.

The Grounding Check That Doesn't Call A Second Model

Most systems that try to keep a language model honest do it by asking another language model. RepoExplain does not. After an answer streams out, a small module runs a string comparison against the evidence that was fed in.

The check collects the real names it knows about (the symbols and files from the overview), then scans the answer for backticked names and citation numbers that do not appear in that vocabulary or in the evidence. When it finds one, it prints a "Heads-up" line. It does not rewrite the answer and it does not call the model again.

The limit is exactly where you would expect. It checks names, not claims. A confident, fluent, entirely wrong explanation that happens to use the correct symbol names passes the check without a murmur. The project does not pretend otherwise. It ships a heuristic and labels it as one, which is consistent with everything else here: cheap, explainable, and honest about the boundary of what it can actually verify.

Where It Fits

The clearest expression of the philosophy is a table of the router's own routes and whether each one calls the model. Read the deps row twice.

RouteTriggered byCalls the LLM?Evidence assembled
depsDependency questions with no named targetNorender_deps_report(direct=True)
overview"What is this project?" with no named targetYesrender_overview (facts) + 3 semantic chunks
locate"Where is X?"YesSemantic chunks
explainA named symbol or file in the questionYesSemantic chunks + target
searchDefault fallbackYesSemantic chunks

One entire route never touches the model. A dependency question is answered by rendering the dependency report directly from the index. There is no reason to ask a 4B model to read a list of imports, so the code does not ask. The grounding check is skipped for that route too, because there is nothing to ground.

Zoom out and the project sits in a category of its own, defined by four axes. This is a comparison of categories, not a rivalry: the research did not establish that any specific tool is a direct competitor.

RepoExplainCloud code chatOther local tools
Where your code goesNowhereVendor cloudNowhere
Model size~4B params100B+Varies
DeterminismRules first, model secondModel firstVaries
CostFreeSubscriptionFree
RAM floor8 GBN/AVaries

The row that matters most is determinism. Cloud code chat puts the model first and hopes the model behaves. RepoExplain puts rules first and lets the model handle only the parts that genuinely need language. It is a different bet about where the intelligence in the system should live.

What It Doesn't Claim

The project is more interesting for what it admits than for what it promises. Keyword routers are brittle. The long alternations in OVERVIEW_RX and LOCATE_RX work, but the ordering of checks decides which way ambiguous questions fall. "Where does the program start?" reaches overview through a lucky substring, not through understanding.

Character budgets are a crude proxy for tokens. RepoExplain truncates evidence by counting characters and stops when fewer than 200 remain, which is predictable and needs no tokenizer, but it is a guess. The grounding check is a heuristic, as covered above. The overview data structure is typed as Dict[str, Any] throughout, so its schema lives in convention and tests rather than in types, which is a real maintainability risk.

There are smaller rough edges. In read_file, the expression end = min(end or total, total) treats end=0 as "read to the end", because zero is falsy. No current caller passes zero, so it is harmless, but it is a sloppy idiom that a future caller could trip over.

None of this makes the project worse. It makes it legible. The point is not that RepoExplain is flawless. The point is that it is a tool built by someone who assumed the model would sometimes be wrong, the hardware would sometimes be small, and the honest thing to do was to say so out loud.