AutoAgent: The Repo That Lets AI Rewrite Its Own Agent Harness
A meta-agent uses benchmark scores, a fixed adapter boundary, and a simplicity rule to evolve prompts, tools, and orchestration without human micromanagement.
The core idea is the same: you're not touching the harness Python files like you normally would as an engineer. Instead, you program `program.md`, the Markdown file that provides context to the meta-agent and defines the agent-engineering loop.
- AutoAgent turns agent development into a search problem where benchmark score decides which harness edits survive.
- The human role shifts from prompt tinkerer to designer of objectives, constraints, and evaluation.
- The fixed adapter boundary keeps the system editable where it matters and stable where reproducibility matters.
- The simplicity rule is not cosmetic, because it blocks the meta-agent from winning by piling on unnecessary machinery.
The Human Writes the Rules, the Agent Writes the Code
AutoAgent is not trying to be another friendly wrapper around an LLM. It flips the usual workflow: the human writes program.md, and the system rewrites the agent harness until the benchmark score improves. That makes the repo less like an app and more like a machine for improving apps.
Kevin Gu describes it plainly in the README: "Like autoresearch but for agent engineering." That is the right mental model. The interesting object is not the worker agent alone. It is the process that keeps changing the worker agent.
Inside the Hill-Climbing Loop
The loop is simple on purpose. Diagnose failures, edit agent.py, run Harbor tasks, compare scores, then keep or discard the change. There is no romantic notion of creativity here. A candidate survives because it does better, or because it does just as well with less complexity.
Why the Fixed Adapter Boundary Matters
The architecture split in agent.py is the heart of the repo. One section is editable. The other section is fixed. That gives the meta-agent room to improve prompts, tools, and orchestration without mutating the plumbing that speaks to Harbor and records trajectories.
# Editable section
SYSTEM_PROMPT = ...
def create_tools(environment):
...
# Fixed adapter boundary
def to_atif(trajectory):
...
# Harbor integration remains stable below this lineThat split matters because it makes the system legible. The agent can search a large space, but the evaluation surface stays stable. In practice, that is what keeps the optimization loop from becoming a pile of one-off hacks.
The Simplicity Rule Is the Real Secret
AutoAgent does not treat equal performance as a tie. If a change gets the same score with a simpler harness, it wins. That is the guardrail that keeps the meta-agent from growing bloated in the name of progress.
| Policy | What wins | What gets rejected |
|---|---|---|
| Higher score | Any harness edit that increases passed tasks | A simpler change that does not improve results |
| Equal score with simplicity | A shorter or cleaner harness | A more complex rewrite that adds no value |
| Same score, more machinery | Nothing | Prompt bloat disguised as improvement |
This is not a footnote. It is the quality function. Without it, the system could keep adding tools, layers, and indirection while looking busy and getting harder to debug.
AutoAgent vs. the Usual Agent Stack
Most agent stacks help you assemble behavior by hand. AutoAgent changes the job description. It optimizes the harness itself, not just the prompt, and it uses benchmark feedback as the only arbiter that matters.
| Approach | Who edits what | What is optimized | How success is measured | Unattended improvement |
|---|---|---|---|---|
| Manual prompt engineering | A human edits prompts and tools | The prompt or workflow | Subjective or ad hoc tests | No |
| Standard agent frameworks | A human assembles the stack | The application behavior | Task success during development | Only if the human keeps tuning |
| AutoAgent | The meta-agent edits the harness within a fixed boundary | The harness and its orchestration | Benchmark score in Harbor tasks | Yes |
The difference is subtle but important. AutoAgent is not simply a better agent. It is a better process for producing agents.
What Harbor and ATIF Unlock
The repo leans on Harbor and ATIF to make failure inspectable. That gives the meta-agent structured evidence, not just a vague feeling that something went wrong. Once failures are serialized well, the system can make informed edits instead of guessing in the dark.
That is what turns optimization from folklore into procedure. A benchmark runner, a trajectory format, and a stable adapter boundary together create a loop where changes can be compared cleanly and repeated reliably.
Why This Is More Than a Benchmark Trick
AutoAgent points at a different division of labor. Humans define the objective, the environment, and the constraints. The model iterates on the implementation. If that pattern holds up, the unit of work changes from writing agents to designing the systems that evolve them.
That is the bigger story here. AutoAgent is not just a clever repo. It is a sketch of a new development primitive, where the codebase is intentionally editable by AI and intentionally fenced where stability matters most.
