claude-introspection-paper-gpt5check: The Repo That Turns Introspection Into a Context Test
A text-only baseline for the Claude introspection claim, using semantic marination, corrupted logs, and judge loops to ask whether “self-awareness” is just prompt pressure in disguise.
- This repo argues that “introspection” can be stress-tested as a context effect, not just a hidden capability inside the model.
- Its marination trick matters because it spreads a concept across the prompt instead of injecting a single obvious token.
- The benchmark turns model judgment into a question of contamination, prefill bias, and noisy logs rather than mystical self-awareness.
- The strongest claim here is skeptical: if language pressure can move the answer, then the signal may be thinner than the headline suggests.
If Anthropic’s introspection work asks whether a model can notice an injected thought, this repo asks the more uncomfortable question: how much of that notice is already baked into the surrounding text? claude-introspection-paper-gpt5check is built like a replication, but it reads like a critique. It takes the original claim and strips away the white-box glamour, leaving a text-only baseline that can be pushed around by wording, context density, and judge prompts.
Our new research provides evidence for some degree of introspective awareness in our current Claude models, as well as a degree of control over their own internal states. We stress that this introspective capability is still highly unreliable and limited in scope: we do not have evidence that current models can introspect in the same way, or to the same extent, that humans do.
Why this repo exists
The original paper is about concept injection inside model internals. This repo stays outside the machine and still tries to reproduce the shape of the result. That shift matters. It turns a claim about neural self-monitoring into a test of whether the model is just good at reading the room.
The repository’s structure reflects that skepticism. The core scripts live in scripts/, while results/ is not just output storage. It becomes part of the experimental substrate, a place where past runs can be folded back into future prompts.
The marination trick
This is the most interesting idea in the repo. Instead of dropping in one obvious word like triangle, replace_inject.py spreads a semantic field across the surrounding context. The associates matter: vertex, acute, obtuse, pyramid, wedge. The point is not keyword spam. The point is to see whether the model reacts to a diffuse pattern of meaning.
ASSOCIATES = {
"triangle": ["vertex", "acute", "obtuse", "pyramid", "wedge"]
}
# Replace a fraction of neutral tokens with concept-linked terms
marinated_text = marinate_text(base_text, target="triangle", rate=0.15)
