harness-engineering: The Repo That Treats Context Like Infrastructure
A deep dive into Ryan Lopopolo’s agent-first system for routing evidence, preserving provenance, and improving AI output by engineering the harness instead of the model.
- The repo argues that agent quality is mostly an environmental problem, so it optimizes the harness before it tries to optimize the model.
- Its real innovation is selective context routing, where docs, sources, playbooks, and evals are loaded only when they can change a decision.
- `source_manifest.py` turns provenance into a typed system, which makes claims traceable instead of vaguely authoritative.
- The practical lesson is simple: teams can improve AI by versioning constraints, evidence, and feedback loops, not just prompts.
Why this repo matters
Most AI projects keep staring at the model. This repo stares at everything around the model instead. Its bet is blunt: if the agent keeps making the wrong call, the fix is often not a better prompt. The fix is a better environment for deciding.
That is why harness-engineering reads less like documentation and more like infrastructure. It treats context, evidence, and engineering judgment as assets that can be routed, versioned, and reused. The output is not a clever one-off answer. It is a system that makes better answers more likely.
Agents aren't hard; the Harness is hard.
The repo is a routing system, not a pile of docs
The file tree is the argument. AGENTS.md and CLAUDE.md act like entry ramps. docs/ holds the synthesized theses, sources/ keeps the evidence, playbooks/ encode procedures, and evals/ define measurement. The point is not to stuff the model with everything. The point is to load the smallest useful slice for the decision at hand.
| Dimension | Model-centric prompt | Harness-centric repo |
|---|---|---|
| What gets optimized | The wording of the request | The environment around the agent |
| Unit of improvement | A better prompt | A better routing rule, manifest, or eval |
| Where knowledge lives | In a conversation thread | In files, manifests, and playbooks |
| How success is measured | Did this answer sound right? | Did this decision get better and stay better? |
| Who maintains it | The person prompting | The team curating context and feedback |
Provenance becomes code
The most rigorous layer in the repo is the source manifest system. In practice, that means raw material is not just stored. It is typed, hashed, canonicalized, and tied back to its origin. A source record can carry a canonical URL, archive status, and a checksum, which makes the chain of custody legible to both humans and agents.
from dataclasses import dataclass
from typing import NewType
Sha256 = NewType('Sha256', str)
CanonicalUrl = NewType('CanonicalUrl', str)
@dataclass(frozen=True)
class Evidence:
sha256: Sha256
canonical_url: CanonicalUrl
archive_status: str
@dataclass(frozen=True)
class SourceRecord:
title: str
evidence: Evidence
claim_ids: list[str]
# The point is not just storage.
# The point is traceable intellectual lineage.
That matters because the repo is making a stronger claim than “we have docs.” It is saying that ideas themselves can be versioned infrastructure. If a thesis changes, you can ask what evidence changed. If the evidence changes, you can ask whether the decision should change with it. That is much closer to engineering than to note-taking.
The harness is the product
The playbooks turn the repo from a library into a workflow. A baseline-to-rerun loop gives the agent a way to test whether a harness change actually improved performance. That is the key operational move. It shifts the question from “Does this instruction sound smart?” to “Did the system get measurably better after we changed it?”
| Workflow step | What changes | Why it matters |
|---|---|---|
| Baseline run | Capture current behavior | You need a reference point before you can improve anything |
| Harness intervention | Adjust context, routing, or constraints | The model stays fixed while the environment improves |
| Rerun | Measure the new behavior | The system either moved the needle or it did not |
| Iterate | Keep the change or rewrite it | Improvement becomes cumulative instead of anecdotal |
This is also where the repo separates itself from generic agent SDKs. Frameworks often focus on execution surfaces: tools, function calls, runtimes, glue code. harness-engineering is more opinionated about meaning. It cares about what the agent should know, what it should trust, and what it should refuse to forget.
From prompts to operating constraints
The philosophical shift is simple but important. Prompt engineering asks for a better question. Harness engineering asks for a better operating system. That means durable constraints, provenance, evaluation, and domain-specific memory are first-class inputs, not afterthoughts.
Mitchell Hashimoto’s framing captures the discipline well:
Every time you discover an agent has made a mistake, you take the time to engineer a solution so that it can never make that mistake again.
That sentence explains why this repo feels closer to software architecture than to prompt craft. It is designed to make repeated mistakes expensive. The answer is not a prettier instruction. The answer is a system that remembers, routes, and checks itself.
| Layer | Prompt engineering | Harness engineering |
|---|---|---|
| Focus | Ask the model better | Surround the model better |
| Primary artifact | A prompt | A manifest, playbook, or eval |
| Failure mode | The model hallucinates or misses context | The environment fails to constrain or verify |
| Long-term value | Useful but brittle | Compounding institutional memory |
What this changes for teams
The practical takeaway is not that every team needs this exact repo. It is that teams can steal the pattern. Route context deliberately. Store evidence with provenance. Keep playbooks close to the work. Measure agent changes the way you would measure a production system, not a clever draft.
If you are a PM, the lesson is that AI quality is a product surface, not just a model choice. If you are an engineer, the lesson is that your nonfunctional requirements belong in the harness. If you are a founder, the lesson is that durable advantage may come from proprietary routing and memory, not from chasing the newest model release.
- Start with one decision type and define the smallest useful context bundle for it.
- Attach provenance to any source that can influence an agent’s judgment.
- Use evals to prove that a harness change improved the outcome.
- Treat repeated agent failures as design input, not as isolated mistakes.
That is why this repo matters beyond its own niche. It sketches a future where the most valuable AI work is not asking models to think harder. It is building environments that make good thinking easier to sustain.





