SimpleTES: The Search Engine That Lets an LLM Evolve Real Scientific Code
A training-free system for scientific discovery that spends compute on proposals, execution, and selection instead of longer chains of thought.
- SimpleTES makes the verifier, not the prompt, the source of intelligence.
- Its search loop turns test-time compute into an evolutionary process that keeps only the candidates that survive execution.
- Docker isolation and repeated evaluation are not implementation details here. They are the trust model.
- The approach works because it targets domains where truth can be checked, not where the model has to improvise an answer from scratch.
Most LLM articles still treat the model as the main event. SimpleTES flips that script. The model is only the generator. The real intelligence lives in the loop that proposes code, runs it, scores it, and keeps going until a verifier approves something better.
That shift matters because scientific code is not judged by eloquence. It is judged by behavior. If a compiler, simulator, benchmark, or test suite can tell you whether something works, then the system can search instead of merely guess.
The Real Unit of Intelligence Is the Loop
The repo is built around a simple premise: ask the LLM for candidates, then let external evaluation decide what survives. In other words, the system does not spend its effort making a single answer sound convincing. It spends it on generating many attempts and filtering them with reality.
That is why the article around SimpleTES feels more like a lab bench than a chat product. The model is stochastic. The evaluator is deterministic or at least measurable. The loop between them is where discovery happens.
Why Trajectories Beat Single-Shot Prompts
SimpleTES organizes work into trajectories. Each trajectory keeps a history of candidate solutions, scores, and metadata, then uses that history to decide what to try next. The point is not to preserve every thought. The point is to preserve the parts of search that help the next generation improve.
The repo frames this with a compact compute model: N ≈ C × L × K × Φ. Chains expand exploration, length deepens refinement, candidates widen the local search, and policy controls how history is presented back to the model. That makes the system tunable in a way that a single prompt is not.
How the loop stays useful
The selector matters as much as the generator. If you show the model too much, the prompt turns into clutter. If you show too little, it repeats itself. SimpleTES treats that as a policy problem, not a prompt-writing problem.
That is the subtle win. The system is not just generating more text. It is managing which parts of the search history remain visible, which candidates get promoted, and how much room the next round gets to explore.
Inside the Evaluation Loop
The evaluator is where the repo stops feeling like an agent demo and starts feeling like infrastructure. The code path extracts the candidate program, mounts it into Docker, runs it against tests or benchmarks, and records the result. That means the model never gets to grade itself.
def evaluate(program_path):
code = _extract_cpp_code(program_path)
# mount into isolated container
# run benchmark or tests
# collect score and metadata
return score, metadata
The details matter. The evaluator uses isolation controls like --network=none and CPU pinning, which keeps untrusted code from reaching out and keeps timing-sensitive tasks from wobbling under noisy system conditions. It also supports repeated evaluation runs and aggregation, which is especially important when the task is stochastic or performance-based.
The Seed Matters: What EVOLVE-BLOCK-START Buys You
The initialization pattern is easy to miss and hard to overstate. The repo uses markers like # EVOLVE-BLOCK-START to tell the system exactly which region is fair game. Everything else stays fixed.
That matters because good search is usually about shrinking the problem, not expanding it. The boilerplate, interfaces, and scaffolding remain stable. The model focuses its effort on the tiny slice of code that actually determines performance.
# EVOLVE-BLOCK-START
# candidate logic goes here
# EVOLVE-BLOCK-END
This is not just neat engineering. It is a search constraint. By protecting the surrounding code, SimpleTES prevents the LLM from wasting edits on structure that already works and keeps the optimization pressure on the active core.
Why This Is More Than One Domain Demo
SimpleTES is not locked to one benchmark family. The repository spans scientific and algorithmic tasks across domains like quantum, GPU optimization, math, and low-level systems code. It also crosses languages, with discovered solutions and evaluators in C++, Rust, Python, and Fortran.
| Dimension | SimpleTES | Why it matters |
|---|---|---|
| Domain fit | Any task with a verifier | The architecture transfers across scientific niches |
| Language | Python orchestration, diverse target code | Search can optimize code in the language that best fits the problem |
| Truth signal | Compiler, simulator, benchmark, test suite | Selection is grounded in measurable outcome |
| Safety | Docker isolation and resource controls | Untrusted code can be evaluated without trusting it |
That breadth makes the repo feel less like a contest tool and more like a general discovery substrate. If a task can be executed and judged, the same loop can try to improve it.
What SimpleTES Is Really Competing With
The cleanest comparison is not against another agent framework. It is against the default LLM habit of asking harder questions and hoping for better answers. SimpleTES instead asks for many answers, tests them, and keeps the best one.
| Approach | What it optimizes | What it trusts | Where it breaks down |
|---|---|---|---|
| Direct prompting | Fast first-pass answers | Model fluency | Fails when the right answer requires search or validation |
| Chain-of-thought style reasoning | Longer internal deliberation | Hidden reasoning traces | Still depends on the model being right without external proof |
| Generic agent loops | Tool use and autonomy | Tool outputs plus model judgment | Can drift if the loop lacks a strong verifier |
| SimpleTES | Selection under test-time compute | Executor, benchmark, or verifier | Needs objective scoring and a meaningful search space |
That last row is the point. SimpleTES is not trying to replace reasoning. It is trying to surround reasoning with a stronger selection mechanism.
The Limits of the Pattern
The pattern is powerful because it is selective. It works best when there is a reliable evaluator, a real search space, and a task whose quality can be measured by execution or scoring. Without that, the loop loses its anchor.
That means SimpleTES is not a universal answer to LLM use. It is a sharp answer to a narrower class of problems. In scientific discovery, that is a feature, not a flaw. The system is honest about where truth can be checked.
So the repo’s real contribution is conceptual as much as technical. It reframes intelligence as search under verification, not speech under pressure. Once you see that, the rest of the design falls into place.





