SimpleTES: The Search Engine That Lets an LLM Evolve Real Scientific Code

A training-free system for scientific discovery that spends compute on proposals, execution, and selection instead of longer chains of thought.

8 min read • View on GitHub • More from wizard-intelligence-learning-lab

A wide scientific workbench where many candidate designs spill out of a drafting machine, pass through a hard verification chamber, and only one refined result advances through a gate. The scene explains that SimpleTES does not trust the first answer. It spends compute on generation, testing, and selection until a verifier promotes the best candidate.
SimpleTES treats the verifier as the center of gravity. The model proposes. The evaluator decides. The loop does the real work.
Key Takeaways

Most LLM articles still treat the model as the main event. SimpleTES flips that script. The model is only the generator. The real intelligence lives in the loop that proposes code, runs it, scores it, and keeps going until a verifier approves something better.

That shift matters because scientific code is not judged by eloquence. It is judged by behavior. If a compiler, simulator, benchmark, or test suite can tell you whether something works, then the system can search instead of merely guess.

The Real Unit of Intelligence Is the Loop

The repo is built around a simple premise: ask the LLM for candidates, then let external evaluation decide what survives. In other words, the system does not spend its effort making a single answer sound convincing. It spends it on generating many attempts and filtering them with reality.

That is why the article around SimpleTES feels more like a lab bench than a chat product. The model is stochastic. The evaluator is deterministic or at least measurable. The loop between them is where discovery happens.

Why Trajectories Beat Single-Shot Prompts

The key abstraction is not a prompt chain. It is a trajectory: a history of candidates, scores, and metadata that shapes the next round of search.

SimpleTES organizes work into trajectories. Each trajectory keeps a history of candidate solutions, scores, and metadata, then uses that history to decide what to try next. The point is not to preserve every thought. The point is to preserve the parts of search that help the next generation improve.

The repo frames this with a compact compute model: N ≈ C × L × K × Φ. Chains expand exploration, length deepens refinement, candidates widen the local search, and policy controls how history is presented back to the model. That makes the system tunable in a way that a single prompt is not.

How the loop stays useful

The selector matters as much as the generator. If you show the model too much, the prompt turns into clutter. If you show too little, it repeats itself. SimpleTES treats that as a policy problem, not a prompt-writing problem.

That is the subtle win. The system is not just generating more text. It is managing which parts of the search history remain visible, which candidates get promoted, and how much room the next round gets to explore.

Inside the Evaluation Loop

The evaluator is where the repo stops feeling like an agent demo and starts feeling like infrastructure. The code path extracts the candidate program, mounts it into Docker, runs it against tests or benchmarks, and records the result. That means the model never gets to grade itself.

def evaluate(program_path):
    code = _extract_cpp_code(program_path)
    # mount into isolated container
    # run benchmark or tests
    # collect score and metadata
    return score, metadata

The details matter. The evaluator uses isolation controls like --network=none and CPU pinning, which keeps untrusted code from reaching out and keeps timing-sensitive tasks from wobbling under noisy system conditions. It also supports repeated evaluation runs and aggregation, which is especially important when the task is stochastic or performance-based.

A close-up of a source file with a marked evolve window, where only one region of code is editable while surrounding boilerplate stays fixed. Beside the page is a sealed Docker chamber and a scoring stamp. The image explains that SimpleTES constrains search to the part of the program that matters while keeping execution isolated and reproducible.
The smartest trick in the repo is also the smallest. Protect the boilerplate, open only the evolve block, and let evaluation happen in a sealed environment.

The Seed Matters: What EVOLVE-BLOCK-START Buys You

The initialization pattern is easy to miss and hard to overstate. The repo uses markers like # EVOLVE-BLOCK-START to tell the system exactly which region is fair game. Everything else stays fixed.

That matters because good search is usually about shrinking the problem, not expanding it. The boilerplate, interfaces, and scaffolding remain stable. The model focuses its effort on the tiny slice of code that actually determines performance.

# EVOLVE-BLOCK-START
# candidate logic goes here
# EVOLVE-BLOCK-END

This is not just neat engineering. It is a search constraint. By protecting the surrounding code, SimpleTES prevents the LLM from wasting edits on structure that already works and keeps the optimization pressure on the active core.

Why This Is More Than One Domain Demo

SimpleTES is not locked to one benchmark family. The repository spans scientific and algorithmic tasks across domains like quantum, GPU optimization, math, and low-level systems code. It also crosses languages, with discovered solutions and evaluators in C++, Rust, Python, and Fortran.

DimensionSimpleTESWhy it matters
Domain fitAny task with a verifierThe architecture transfers across scientific niches
LanguagePython orchestration, diverse target codeSearch can optimize code in the language that best fits the problem
Truth signalCompiler, simulator, benchmark, test suiteSelection is grounded in measurable outcome
SafetyDocker isolation and resource controlsUntrusted code can be evaluated without trusting it

That breadth makes the repo feel less like a contest tool and more like a general discovery substrate. If a task can be executed and judged, the same loop can try to improve it.

What SimpleTES Is Really Competing With

The cleanest comparison is not against another agent framework. It is against the default LLM habit of asking harder questions and hoping for better answers. SimpleTES instead asks for many answers, tests them, and keeps the best one.

ApproachWhat it optimizesWhat it trustsWhere it breaks down
Direct promptingFast first-pass answersModel fluencyFails when the right answer requires search or validation
Chain-of-thought style reasoningLonger internal deliberationHidden reasoning tracesStill depends on the model being right without external proof
Generic agent loopsTool use and autonomyTool outputs plus model judgmentCan drift if the loop lacks a strong verifier
SimpleTESSelection under test-time computeExecutor, benchmark, or verifierNeeds objective scoring and a meaningful search space

That last row is the point. SimpleTES is not trying to replace reasoning. It is trying to surround reasoning with a stronger selection mechanism.

The Limits of the Pattern

The pattern is powerful because it is selective. It works best when there is a reliable evaluator, a real search space, and a task whose quality can be measured by execution or scoring. Without that, the loop loses its anchor.

That means SimpleTES is not a universal answer to LLM use. It is a sharp answer to a narrower class of problems. In scientific discovery, that is a feature, not a flaw. The system is honest about where truth can be checked.

So the repo’s real contribution is conceptual as much as technical. It reframes intelligence as search under verification, not speech under pressure. Once you see that, the rest of the design falls into place.