forge_markdown_stream: The Rust Renderer That Makes LLM Markdown Feel Instant
A streaming terminal renderer that repairs broken fences, finalizes immutable blocks, and keeps complex markdown usable while tokens are still arriving.
- forge_markdown_stream turns markdown rendering into a block-state problem, so completed content stays frozen while only the last incomplete block keeps changing.
- Its forgiving repair step matters as much as its parser, because LLM output often arrives with broken fences and awkward line boundaries.
- The hardest engineering is not pretty text but layout correctness, especially for tables, wrapping, and ANSI-styled terminal output.
- The project shows a broader CLI lesson for the AI era: responsiveness comes from preserving state, not from reparsing everything faster.
Streaming markdown is easy until the first broken fence lands in the middle of a table. Then the whole terminal starts wobbling. forge_markdown_stream is interesting because it refuses to treat that wobble as normal.
The problem is not markdown. It is markdown that is still arriving
Traditional renderers assume the document exists first and the screen comes second. That works for blog posts. It breaks for LLM output, where text arrives in fragments, lists grow line by line, and code fences often close late or not at all.
The obvious fix, reparsing the whole document on every token, is the wrong one. It feels simple, but it gets slower as the response gets longer, which is exactly the direction AI transcripts tend to go.
The trick: only the pending block stays mutable
That is the whole mental model. Incoming text is buffered, repaired if needed, parsed into blocks, and then split into two states: finalized blocks and one pending block. The finalized part is immutable. Only the pending block remains exposed to change.
That is why this project feels less like a parser and more like a small state machine with discipline. Once a block is safe, it gets locked away. The terminal stops repainting the past.
```text
> a code fence that is still attached to the last line of code ```
```Repair is the quiet hero
The repair layer exists because LLMs do not reliably respect markdown boundaries. A closing fence can arrive glued to code, or a block can end in a way that looks complete to a human and malformed to a parser.
`repair_line` fixes that before parsing gets involved. The point is not cleverness. The point is forgiveness, which is exactly what streaming AI output needs.
Tables are where the terminal gets serious
This is the part that tells you the project is not a toy. Tables force the renderer to measure visible width, ignore ANSI escape sequences, shrink columns when space gets tight, and keep box-drawing characters aligned even after wrapping.
| Approach | Parsing model | Streaming behavior | Failure mode | Why it matters for LLM output |
|---|---|---|---|---|
| Traditional markdown parser | Whole document | Reparse everything on every update | Jitter and O(n²) cost | Fine for static docs, poor for live streams |
| Naive streaming loop | Repeated full re-render | Still rewrites too much state | Slow as the response grows | Looks responsive at first, then degrades |
| forge_markdown_stream | Block-aware incremental state | Only the pending block changes | Requires repair and careful boundaries | Keeps long LLM replies stable in the terminal |
The neat part is that the table logic cares about what the user can actually see, not the raw string length. That is the right mental model for terminal UI, and it is why ANSI-aware wrapping is so easy to get wrong.
ANSI-aware wrapping is harder than it looks
Colored terminal text is sneaky. The escape sequences should not count toward width, but they still have to survive the wrap. If the renderer gets that wrong, colors bleed, lines misalign, or the output turns visually noisy.
That is where `unicode-width`, `colored`, and the wrapping logic pull their weight. The implementation is not flashy. It just respects the fact that terminals are visual systems, not plain text boxes.
Why Rust fits this job
Rust is a natural fit here because the problem is full of places where correctness matters more than convenience. Width calculation, block state, repair rules, and incremental rendering all need explicit control.
The stack reflects that choice. `unicode-width` helps measure what the terminal actually displays. `syntect` handles syntax highlighting. `insta` gives the project a way to lock down edge cases with snapshots.
pub fn push(&mut self, chunk: &str) {
self.buffer.push_str(chunk);
while let Some(pos) = self.buffer.find('\n') {
let line = self.buffer.drain(..=pos).collect::<String>();
let repaired = repair_line(&line, &self.state);
self.parser.push(&repaired);
}
}How it compares to static markdown libraries
Static libraries like `markdown-it` and `marked` are optimized for finished documents. That makes them great general-purpose tools. It also makes them a mismatch for a live stream that keeps changing under your cursor.
| Library style | Best at | Streaming fit | Typical trade-off |
|---|---|---|---|
| Static parser | Completed documents | Weak | Easy to use, but too much work repeats |
| Naive live re-render | Simple demos | Okay at small scale | Gets slower as content grows |
| forge_markdown_stream | Incremental terminal output | Strong | More stateful, but far more stable |
The architectural point is simple: if the user sees a completed block, the program should stop pretending that block is still provisional. That is the difference between a live system and a perpetual draft.
Forge needed a renderer, and the renderer reveals the product
Forge is a broader CLI for working across Git forges, but the renderer tells you how the project thinks about humans and agents. Output has to stay readable while it is still forming, because the CLI is meant to serve both people and software that think out loud.
I wanted one interface that worked the same way everywhere, for humans on the command line and for AI coding agents that need to interact with forges programmatically.
That makes `forge_markdown_stream` feel like infrastructure, not ornament. It exists so the rest of the CLI can stay calm while the stream is still in flight.
The bigger lesson
The real lesson here is about state management under uncertainty. If something is done, lock it. If something is not done, keep it honest. That idea is useful far beyond markdown.
Terminal UX for AI does not improve just by rendering faster. It improves when the renderer understands the difference between mutable and immutable output, and acts on that difference with discipline.
