cc-thinking-skills: A Library of AI Reasoning Skills That Tries to Prove Itself Wrong

A Claude Code skill system, a multi-model evaluation harness, and a brutally honest scorecard that treats mental models like hypotheses, not gospel.

8 to 10 min read • View on GitHub • More from tjboudreaux

A laboratory scale balances a stack of labeled thinking skill cards against a stack of blank control cards, while a clipboard marks a passing threshold. In the background, a terminal sits behind glass like a specimen under test, explaining that the repo measures reasoning frameworks instead of trusting them on faith. — The repo’s core move is methodological: every skill is a claim that has to survive a controlled test.

Key Takeaways

The repo treats thinking skills as hypotheses, not wisdom, and only promotes them if they survive controlled evaluation.
Its real novelty is the harness around the skills, which tries to isolate length bias, tool access, and judge preference.
The router turns a large skill library into something usable by selecting a skill and confidence score for a task.
This is less a prompt pack than a prototype cognitive operating system for Claude Code.

Most prompt libraries ask you to trust the prompt. This one does the opposite. It starts with the uncomfortable premise that a reasoning skill is worthless until it beats a placebo, clears a statistical gate, and convinces more than one judge model.

The uncomfortable truth: the scorecard comes first

That is what makes tjboudreaux/cc-thinking-skills interesting. The repository is not selling you 39 mental models as if they were finished products. It is trying to find out whether any of them actually help Claude Code do better work.

This repository contains the curriculum for the Critical Thinking course at [Institution Name Redacted in Source].

tjboudreaux, Project Creator/Maintainer · tjboudreaux/cc-thinking-skills README

What this repo actually is

The repo has three jobs. The /skills folder holds the reasoning guides. The /evals folder holds the scoring machinery. The analysis and plugin metadata wrap the whole thing in a usable Claude Code package.

The evaluation pipeline is the real product. It turns a reasoning guide into an evidence-backed verdict.

The skills are the surface. The experiment is the product

Each skill is a markdown guide that Claude Code can ingest as a structured reasoning lens. The catalog includes patterns like first principles, OODA, and Bayesian thinking, but the point is not the labels. The point is that each one is testable.

{
  "skill": "thinking-first-principles",
  "confidence": 0.91
}

That turns the repo into a library with a very specific temperament. It does not assume the best framework is the one that sounds smartest. It assumes the best framework is the one that changes outcomes.

How the repo tries to make prompt engineering behave like science

The evaluation contracts define the rules of the game. A skill has to clear a primary delta threshold and a paired significance test before it can be elevated. That matters because it forces the project to separate genuine gain from model theatrics.

Dimension	Ordinary prompt pack	cc-thinking-skills	Full evaluation harness
Evidence standard	Anecdotes and vibes	Claims must beat a threshold	Paired tests and gatekeeping
Prompt structure	Loose instructions	Markdown skill guides	Controlled inputs and contracts
Failure handling	Ignored or hand-waved	Marked kill or refine	Explicit verdicts and reruns
Scalability	Depends on the author’s memory	Catalog of 39 skills	Automated selection and judging
Human selection burden	High	Medium	Lower with router

The clever part is how the repo tries to keep itself honest. It disables tools during evaluation, uses distractor sets, and runs blind pairwise comparisons. In other words, it keeps asking whether the skill improved reasoning, or just gave the model more words to work with.

The placebo problem

Two nearly identical prompt streams move through parallel black boxes, one carrying a dense skill guide and the other a length-matched neutral filler. Both outputs roll into the same blind judge chamber, showing how the harness tests whether the framework helps beyond extra tokens or formatting effects. — The placebo test is the most elegant control in the repo because it targets length bias directly.

This is the strongest part of the harness. If the skill only works because it bloats context, the neutral filler should do the same job. If the skill still wins, then the content matters. That is a much harder claim, and a much better one.

Why the panel judge matters

A single judge model is easy to bias. A panel of judges is still imperfect, but it is harder to fool. The repo leans into that trade-off by using multiple models to vote on quality rather than letting one model declare victory over its own output.

There is a real methodological tension here. LLMs judging LLMs can amplify shared blind spots. But for a repo that is already trying to control for length bias and tool access, the committee approach is the right kind of practical skepticism.

The router solves the last mile problem

Once you have 39 skills, the bottleneck becomes selection. The router is the repo’s answer. It classifies a task, returns a skill, and attaches confidence so the system can behave more like an operating layer than a static library.

Problem	Manual skill choice	Router-based selection
Find the right lens	User remembers the catalog	Model proposes the skill
Speed	Depends on expertise	Fast enough for routine use
Scalability	Breaks as the catalog grows	Improves as the catalog grows
Operational burden	High	Lower
Risk	Wrong skill chosen silently	Wrong choice can be inspected

What this repo is really pointing toward

The deeper idea here is not that prompts should become cleverer. It is that reasoning systems should become testable. That changes the ambition of the project. It is no longer a list of mental models. It is an attempt to build a cognitive stack that can justify its own existence.