`anydoc`: The Rust Parser That Turns Office Files Into One Clean Markdown Path
Firecrawl’s document engine treats messy formats as a routing problem, not a monolith. That design makes it fast, secure, and unusually elegant for LLM pipelines.
- anydoc treats document ingestion like routing, with format detection deciding the path before conversion begins.
- The Universal Document Model is the core abstraction because it lets many office formats share one Markdown renderer.
- PDFs break the pattern on purpose, which keeps the architecture honest about how positional documents differ from semantic ones.
- Rust, fuzzing, and content-based sniffing make the parser feel built for hostile real-world files, not just clean demos.
A Word file, an Excel sheet, a legacy `.doc`, and a ZIP-based OOXML package are all just bytes at the door. `anydoc` starts by asking a smaller question than most converters: what is this thing, really? That one choice explains most of the design.
A document parser that thinks in routes, not formats
Most document tools start with a format and hope everything else follows. `anydoc` starts with detection, then chooses the smallest viable parser. That is why a mislabeled file, a zipped office package, and a binary legacy blob do not all trigger the same machinery.
No single library reliably converts every common document format to clean markdown. You end up stitching four or five tools together, each with its own dependencies, output shape, and failure modes. So we built two Firecrawl projects to solve it: pdf-inspector for PDFs, and AnyDoc for everything else.
The Universal Document Model is the real product
The key abstraction is not Markdown. It is the internal document model made of Blocks and Inlines. `anydoc` converts many office formats into that shared representation first, then renders once, consistently, at the end.
// Conceptual shape of the pipeline
let bytes = input.read_all()?;
let format = detect::from_bytes(&bytes)?;
match format {
Format::Docx | Format::Xls | Format::Rtf | Format::Odt => {
let doc = parse_to_document_model(&bytes)?;
markdown::render(&doc)
}
Format::Pdf => pdf_inspector::render_markdown(&bytes),
other => Err(ConvertError::Unsupported(other)),
}
That split matters. A table extracted from Excel and a table extracted from Word can land in the same Markdown shape because they were normalized through the same IR. You do not get five different downstream behaviors for five upstream file families.
Why PDFs take a different road
This is the architectural tell. PDFs do not get forced through the same semantic funnel, because they are often positional documents rather than cleanly structured ones. `anydoc` treats that as a feature, not a failure.
| Path | What it optimizes for | Model strategy | PDF handling | Best fit |
|---|---|---|---|---|
| Office formats | Fast Markdown from structured document internals | Normalize into Blocks and Inlines | Separate parser path when needed | LLM ingestion for Word, Excel, RTF, and OOXML |
| PDFs | Text-layer extraction without pretending semantics are perfect | Bypass the shared model when structure is too weak | Direct Markdown path via pdf-inspector | Local parsing of readable PDFs |
| General-purpose converters | Coverage across many file kinds | Often broad and generic | Usually one more case in a large pipeline | Mixed conversion tasks, not one narrow ingestion layer |
That choice keeps the system honest. It is better to admit that PDF is different than to squeeze it into a model that looks tidy but loses meaning.
Built for untrusted documents
Office files are an attack surface. `anydoc` answers that with Rust, package inspection, XML hardening, depth and node limits, and fuzzing across formats. The security story is not an add-on. It is part of the ingestion contract.
The package layer matters here. ZIP and OLE containers are inspected before their contents are trusted, which is exactly what you want when files come from crawls, uploads, or unknown sources.
Why this feels faster than the Python stack
The speed story is not just Rust versus Python. It is fewer conversions, fewer heavyweight abstractions, streaming XML parsing, one serializer, and no model inference by default. If the file already contains text and structure, `anydoc` reads that structure directly.
processed 500 DOCX files in 1.7 seconds; free to use.
What `anydoc` is really competing with
The real comparison is not feature count. It is abstraction choice. `anydoc` is a structural compiler for Markdown generation, while many alternatives are broader, heavier, or optimized for different trade-offs.
| Project | Primary goal | Language/runtime | Model strategy | PDF handling | Best use case |
|---|---|---|---|---|---|
| anydoc | Fast LLM-ready Markdown from office files | Rust with Node and Python bindings | Shared document model plus PDF bypass | Dedicated PDF path via pdf-inspector | Local ingestion pipelines |
| MarkItDown | General document to Markdown conversion | Python | Mostly direct conversion | Depends on surrounding stack | Convenient broad conversion |
| Unstructured | Flexible document partitioning | Python | Heavier partitioning pipeline | Often more complex | Enterprise RAG preprocessing |
| Docling | High-quality document parsing | Python and ML-assisted components | Layout-aware extraction | Stronger compute footprint | Document intelligence workflows |
| Pandoc | General document transformation | Haskell and CLI ecosystem | Format translation toolchain | Broad but generic | Multi-format conversion |
| Mammoth | .docx to HTML or Markdown | JavaScript and browser-friendly | Specialized Word mapping | Not a general PDF tool | Best-in-class Word-only conversion |
That is why `anydoc` lands as a systems tool, not a convenience wrapper. It narrows the problem until the abstraction is sharp enough to be useful.
The right abstraction for the AI ingestion layer
For AI pipelines, document conversion is becoming a narrow systems problem. Detect the file, parse what can be parsed, normalize structure when it is real, and bypass the model when the source format resists it. `anydoc` is interesting because it makes that sequence legible.
That is a better thesis than speed alone. Fast is useful, but a fast wrong abstraction ages badly. A small routing layer, a shared IR, and an explicit PDF exception feel like the kind of design that can survive contact with real documents.