harness-engineering: The Repo That Treats Context Like Infrastructure

A deep dive into Ryan Lopopolo’s agent-first system for routing evidence, preserving provenance, and improving AI output by engineering the harness instead of the model.

9 min read • View on GitHub • More from lopopolo

A wide editorial scene shows a human engineer beside a cabinet of labeled drawers, with only one drawer lit while the rest stay closed. A small agent terminal on the desk receives a curated stack of documents, which explains how the repository routes context instead of dumping in everything at once.
The core move is selective loading. The harness opens the right drawer, not the whole library.
Key Takeaways

Why this repo matters

Most AI projects keep staring at the model. This repo stares at everything around the model instead. Its bet is blunt: if the agent keeps making the wrong call, the fix is often not a better prompt. The fix is a better environment for deciding.

That is why harness-engineering reads less like documentation and more like infrastructure. It treats context, evidence, and engineering judgment as assets that can be routed, versioned, and reused. The output is not a clever one-off answer. It is a system that makes better answers more likely.

Agents aren't hard; the Harness is hard.

Ryan Lopopolo, Software Engineer at OpenAI · Harness engineering: leveraging Codex in an agent-first world

The repo is a routing system, not a pile of docs

The file tree is the argument. AGENTS.md and CLAUDE.md act like entry ramps. docs/ holds the synthesized theses, sources/ keeps the evidence, playbooks/ encode procedures, and evals/ define measurement. The point is not to stuff the model with everything. The point is to load the smallest useful slice for the decision at hand.

This is the repo’s real interface: a decision enters, the harness chooses what context to open, and the rest stays sealed.

DimensionModel-centric promptHarness-centric repo
What gets optimizedThe wording of the requestThe environment around the agent
Unit of improvementA better promptA better routing rule, manifest, or eval
Where knowledge livesIn a conversation threadIn files, manifests, and playbooks
How success is measuredDid this answer sound right?Did this decision get better and stay better?
Who maintains itThe person promptingThe team curating context and feedback
A close-up workshop assembly line shows a source document passing through a hash stamp, then into a manifest ledger, then into a thesis card, and finally into an agent instruction panel. The image explains how raw evidence becomes typed provenance before it reaches the agent.
Provenance is not an annotation here. It is a pipeline.

Provenance becomes code

The most rigorous layer in the repo is the source manifest system. In practice, that means raw material is not just stored. It is typed, hashed, canonicalized, and tied back to its origin. A source record can carry a canonical URL, archive status, and a checksum, which makes the chain of custody legible to both humans and agents.

from dataclasses import dataclass
from typing import NewType

Sha256 = NewType('Sha256', str)
CanonicalUrl = NewType('CanonicalUrl', str)

@dataclass(frozen=True)
class Evidence:
    sha256: Sha256
    canonical_url: CanonicalUrl
    archive_status: str

@dataclass(frozen=True)
class SourceRecord:
    title: str
    evidence: Evidence
    claim_ids: list[str]

# The point is not just storage.
# The point is traceable intellectual lineage.

That matters because the repo is making a stronger claim than “we have docs.” It is saying that ideas themselves can be versioned infrastructure. If a thesis changes, you can ask what evidence changed. If the evidence changes, you can ask whether the decision should change with it. That is much closer to engineering than to note-taking.

The harness is the product

The playbooks turn the repo from a library into a workflow. A baseline-to-rerun loop gives the agent a way to test whether a harness change actually improved performance. That is the key operational move. It shifts the question from “Does this instruction sound smart?” to “Did the system get measurably better after we changed it?”

Workflow stepWhat changesWhy it matters
Baseline runCapture current behaviorYou need a reference point before you can improve anything
Harness interventionAdjust context, routing, or constraintsThe model stays fixed while the environment improves
RerunMeasure the new behaviorThe system either moved the needle or it did not
IterateKeep the change or rewrite itImprovement becomes cumulative instead of anecdotal

This is also where the repo separates itself from generic agent SDKs. Frameworks often focus on execution surfaces: tools, function calls, runtimes, glue code. harness-engineering is more opinionated about meaning. It cares about what the agent should know, what it should trust, and what it should refuse to forget.

From prompts to operating constraints

The philosophical shift is simple but important. Prompt engineering asks for a better question. Harness engineering asks for a better operating system. That means durable constraints, provenance, evaluation, and domain-specific memory are first-class inputs, not afterthoughts.

Mitchell Hashimoto’s framing captures the discipline well:

Every time you discover an agent has made a mistake, you take the time to engineer a solution so that it can never make that mistake again.

Mitchell Hashimoto, Creator of Terraform and Ghostty · Engineer the Harness

That sentence explains why this repo feels closer to software architecture than to prompt craft. It is designed to make repeated mistakes expensive. The answer is not a prettier instruction. The answer is a system that remembers, routes, and checks itself.

LayerPrompt engineeringHarness engineering
FocusAsk the model betterSurround the model better
Primary artifactA promptA manifest, playbook, or eval
Failure modeThe model hallucinates or misses contextThe environment fails to constrain or verify
Long-term valueUseful but brittleCompounding institutional memory

What this changes for teams

The practical takeaway is not that every team needs this exact repo. It is that teams can steal the pattern. Route context deliberately. Store evidence with provenance. Keep playbooks close to the work. Measure agent changes the way you would measure a production system, not a clever draft.

If you are a PM, the lesson is that AI quality is a product surface, not just a model choice. If you are an engineer, the lesson is that your nonfunctional requirements belong in the harness. If you are a founder, the lesson is that durable advantage may come from proprietary routing and memory, not from chasing the newest model release.

That is why this repo matters beyond its own niche. It sketches a future where the most valuable AI work is not asking models to think harder. It is building environments that make good thinking easier to sustain.