security-audit-skill: Cloudflare Built a Security Auditor That Tries to Disprove Itself

security-audit-skill is not a scanner. It is a set of prompts, two validators, and a rule that every finding must survive a fresh adversary before it counts.

10 min read • View on GitHub • More from cloudflare

A courtroom scene drawn in black ink on white. A single document stands accused at the witness stand, marked only with a small fingerprint glyph. A prosecutor agent cross-examines it with a magnifying glass and a stack of source lines. Behind the prosecutor sits a blindfolded hunter agent, unable to speak. The judge's bench holds a single gavel, a single chair, and a single pen. The illustration renders the project's core idea: an AI finding is put on trial and challenged by an adversary who never saw the original reasoning.
Adversarial verification by process topology: the hunter who found the candidate is structurally silenced while a fresh agent tries to tear the finding down.
Key Takeaways

It Ships Prompts, Not a Scanner

You would not guess it from the language bar. GitHub reports cloudflare/security-audit-skill as 100 percent JavaScript, and that number is technically true and substantively misleading. The JavaScript is two .cjs validators and their tests. Everything else is Markdown.

SKILL.md, RECONNAISSANCE.md, HUNTING.md, VALIDATION-AND-REPORTING.md, ATTACK-CLASSES.md, and ten domain files covering memory safety, AI and LLM handling, web auth, client-side code, supply chain, cloud deployment, RPC, resource exhaustion, data isolation, and desktop and mobile IPC. That is the product. Prose written to be executed by a model.

Install is one line:

npx skills add https://github.com/cloudflare/security-audit-skill --skill security-audit

Loading the skill does not start an audit. Guidance mode is the default, and full audit mode requires an explicit request for a codebase audit, a pen test, or a report artifact. If the request is ambiguous, the skill asks one focused question before it creates any files. That single rule sidesteps the failure mode of agentic tools that quietly write artifacts into your repository.

The architecture has three layers, and the split is the whole idea. The methodology lives in Markdown phase files. The contract lives in report-schema.json. The enforcement lives in the validators. The model supplies judgment. The code supplies the veto.

With the recent wave of vulnerability disclosures, it’s a good time to take a closer look at your own code. Turn your coding agent into a team of security auditors with the open-source skill that seeded Cloudflare’s vulnerability discovery harness. The 6 phase approach: 1. Map the codebase 2. Hunt for vulnerabilities 3. Challenge each candidate 4. Record the findings 5. Verify the findings 6. Report what’s confirmed, rejected, or still needs validation Start your hunt here: github.com/cloudflare/securi…

Cloudflare Developers, Cloudflare developer account · Cloudflare Developers post

A Fresh Agent Tries to Disprove Every Finding

The most interesting mechanic is not the hunting. It is what happens to a candidate after it is found.

Every unique candidate goes to a fresh verifier. That verifier did not hunt it. It must not receive another verifier's conclusion. The instruction is blunt: "You did not write this candidate. Try to refute it from repository source and bounded local evidence."

Independence here is not a personality trait requested of the model. It is a property of the process graph. The hunter and the verifier are separate agents with separate context, and the only thing that crosses between them is the candidate itself. The hunter's reasoning is withheld on purpose, because a conclusion is the easiest thing to anchor on.

The verifier returns one of three verdicts. confirmed requires a complete source trace and a bounded observed result. needs_validation carries an exact unresolved fact and no severity. rejected records a disproved candidate. That third state matters as much as the first: a finding that has been actively refuted is a recorded fact, not a silent deletion.

One more rule holds the whole thing together. Verifiers must preserve the fingerprint for the same source-derived root cause across every state. A finding that starts as needs_validation and later earns confirmed keeps its identity, so a later run can recognize it as the same root cause instead of rediscovering it from scratch.

How a candidate crosses a trust boundary: the hunter cannot see the verdict, the verifier cannot see the hunter's reasoning, and the fingerprint survives every state change.

Only confirm established boundary failures.

Cloudflare security-audit-skill maintainers, Project README · cloudflare/security-audit-skill README

Six Phases, One Writer

The pipeline runs in six phases: reconnaissance, coverage-led hunting, adversarial candidate validation, structured findings, independent verification of final records, and target-neutral reporting. The flow is conventional. The coordination rules are not.

Reconnaissance fans out to four read-only research agents. One maps the product, stack, and build and test commands, with an explicit instruction not to run them. One maps principals, authority, and controls. One maps entry surfaces, copies, and sinks. One checks local execution and deployment visibility, including whether the host can actually enforce a sandbox. Every recon agent returns repository-relative file:line references and writes nothing.

That last part is a rule repeated everywhere: only the parent writes shared state. Agents write into their own scratch/ directories and nowhere else. Even their artifacts/ output is parent-owned and promoted through a race-safe copy. In a field where agents routinely share a common scratch space, this is an unusually strict isolation model.

Each hunter is built from a fixed prompt contract, and the hunting method itself is a numbered procedure that turns a hunch into a falsifiable claim:

  1. Name the lower-trust principal and starting capability.
  2. Name the accepted value, action, state transition, or resource selector.
  3. Locate the control that should reject, bind, isolate, limit, or revoke it.
  4. Trace the exact source path after that decision.
  5. Stop at the smallest affected dummy record, wrong return value, process-integrity effect, or locally observable shared-resource effect.
  6. State a source-level change and regression case that enforce the invariant.

Principal, capability, control, path, bounded effect, fix. The shape is designed to be refuted. It also builds in a depth bound: stop a line of investigation as soon as the invariant is settled either way.

Coverage is tracked in a ledger with 19 required fields per unit and statuses that include planned, in_progress, covered, candidate, blocked, and deferred. A unit the budget cannot reach must be explicitly deferred with a stated reason. When hunters are capped, prioritization is deterministic: unauthenticated entry surfaces first, then boundaries protecting credentials or cross-tenant data, then prior-run gaps, with ties broken lexicographically by coverage ID. A stochastic system gets a deterministic scheduler.

The Audit Tool That Audits Itself

The paranoia is where the project earns trust. The validators treat their own inputs as hostile, because the content they check is attacker-influenced by design.

An extreme close-up in black ink on white of a magnifying glass held over a printed finding document. Between two ordinary printed letters, a row of invisible characters is rendered as faint dotted outlines: a zero-width space, a bidi override mark, and a lone surrogate. A hand at the top-right edge of the frame turns the magnifying glass. The illustration visualizes the validator's visible-content and Unicode-scalar checks, which reject findings whose text is technically non-empty but visually blank.
The visible-content check in action: a title made of zero-width spaces passes a naive length check but fails here.

The promotion procedure is the sharpest example. The audited code runs inside a sandbox and could plant a symlink or a FIFO in a scratch directory to trick the parent into copying or blocking on something it should not touch. The defense is a directory-relative, re-verified copy that never reopens a file by path. It is a time-of-check-to-time-of-use defense, applied to the audit tool's own output.

Where It Sits

This is not a leaderboard. The agent-security harness landscape is a set of different answers to one question: what counts as proof? Cloudflare's answer is a structured record that survives adversarial verification and schema validation. Others answer with a crash, a revalidated match, or an expert checklist.

ProjectApproachWhat counts as proof
cloudflare/security-audit-skillSix-phase agent workflow with coverage ledgerA structured finding that survives adversarial verification and schema validation
Anthropic defending-code-reference-harnessExploit generation for C/C++ memory bugsA reproducible crash
Vercel deepsecSAST prefilter plus LLM analysis and revalidationA revalidated static match
Trail of Bits skillsExpert-oriented skill toolboxExpert review against a specialist checklist
Visa visa-vulnerability-agentic-harness11-stage SAST-plus-LLM pipelineAn adversarially verified static finding
Capital One vulnhunterComposable find, fix, verify skillsA verified find-fix-verify cycle
Google mantisAbout 15 security-focused skillsFlexible, no fixed pipeline
Cisco ai-deep-sastSAST plus LLMA triaged static match

These approaches are complementary, not interchangeable. SAST-plus-LLM systems offer more deterministic scanner inputs. Exploit-generation systems offer concrete runtime evidence for supported targets. Cloudflare foregrounds orchestration and evidence records rather than any particular scanner.

The Honest Limits

The project does not oversell itself, and neither should this article. Several costs are structural, not incidental.

There is also a practical concern raised in public discussion: security-framed audit tasks can trigger refusals from some frontier models, which guard against misuse. A commenter on Hacker News suggested a workaround of running bug-class skills without the security framing and combining results afterward. That is a claim from an individual, not an established finding, but it points at how a model's safety behavior can interact with a security workflow.

Tip for security professionals using LLMs to audit code: audit skills that explicitly frame the task as security research sometimes trigger refusals from the top OpenAI and Anthropic models because they guard against misuse. What works for me: separate skills for bug classes (and bugs in general) without the security framing, plus another skill that combines their findings to spot security bugs.

wslh, Hacker News commenter · Hacker News discussion

Except their validation doesn't seem to work. I've gone through both (both personally and for my company), and.. no response for weeks.

wslh, Hacker News commenter · Hacker News discussion

Taken together, the honest picture is a tool that is careful about what it claims and explicit about what it cannot prove. That is the point. The interesting thing about security-audit-skill is not that it finds bugs. It is that it refuses to trust the agent that found them, including in the small places where trusting would have been easier.