ds4: The DeepSeek Engine That Treats Inference Like a Storage Problem
Salvatore Sanfilippo’s narrow C engine turns one model family into a readable, hardware-shaped pipeline for long-context local inference.
The main goal was to see if I could write an efficient inference engine using MPS Graph, which is much higher level than writing kernels in raw Metal, but still allows for good performance. Llama.cpp is much more optimized, but the code is also much more complex. This one is around 2k lines of code and is quite easy to read.
- ds4 argues that long-context inference stops being impressive once memory and persistence are treated as first-class systems problems.
- The repo’s power comes from narrowing the target to one model family and one hardware path, which keeps the hot path legible.
- Its architecture makes the GPU look less like a magical accelerator and more like one stage in a controlled data pipeline.
- Compared with general-purpose runners, ds4 is more of a systems essay than a product, and that is the point.
The surprise in ds4 is not that it runs a model locally. It is that it treats a giant context window like something you can manage. That changes the question from "How do we fit inference on a machine?" to "How do we keep state coherent across memory, disk, and device?"
The real trick is not speed. It is making memory behave.
That framing matters because DeepSeek V4 Flash is not a small prompt toy. Once context stretches into the very large range, the engine’s job is not only compute. It becomes about where weights live, how requests are serialized, and how much work can be skipped on the next turn.
Why antirez built a narrow engine instead of a framework
Salvatore Sanfilippo is not approaching inference like a platform vendor. He is approaching it like a systems programmer with a taste for hard edges. That is why ds4 stays deliberately narrow: one model family, one shape, one hot path.
I just released a minimalist inference engine for DeepSeek-R1 (the newer reasoning model) for macOS / Metal: https://github.com/antirez/ds4 It is written in C++ and uses MPS Graph. It’s not raw Metal shaders, but the higher level API, so the implementation is relatively small. It does 17 tokens/sec with the 7B model on my M1 Max (32GB RAM). It's not Llama.cpp speed, but not bad, and the code is 100% focused on DeepSeek, so it may be useful to check how it is implemented.
One model, one shape, one hot path
ds4 does something many inference projects avoid: it hardcodes the model family. That sounds limiting until you see the upside. The compiler, the runtime, and the author all get to stop guessing.
// Simplified shape of the design
#define DS4_N_LAYER 43
#define DS4_N_EMBD 4096
#define DS4_STATIC_ASSERT(x) _Static_assert(x, #x)
// The engine is built around a fixed DeepSeek shape,
// so the implementation can stay direct and readable.That is the philosophical center of the repo. Instead of becoming a universal runner, ds4 becomes a tight pipeline with fewer branches, fewer abstractions, and fewer places for state to drift.
The pipeline: mmap in, graph out, GPU stays busy
The key pieces are easy to name and easy to miss. The model file is memory mapped, so loading starts as virtual memory bookkeeping rather than a giant eager copy. The graph driver schedules work on Metal or CUDA. The KV cache stays close to the device, which makes follow-up turns cheaper and keeps the pipeline coherent.
Why the server is single-worker on purpose
The concurrency model is not a compromise hidden behind a slogan. It is a guardrail. ds4 gives you multiple request producers, but only one worker owns the live session and device state.
That choice reduces races, keeps session ownership obvious, and makes cache persistence practical. It also matches the repo’s larger theme: fewer moving parts, fewer surprises, more control over the hot path.
How ds4 differs from llama.cpp, Ollama, and MLX
| Project | Scope | Hardware approach | Codebase posture | Best fit |
|---|---|---|---|---|
| ds4 | One model family, one narrow engine | MPS Graph on macOS, CUDA path on Linux | Small, explicit, educational | Studying a model-specific inference pipeline |
| llama.cpp | Broad model support across many architectures | Raw Metal, CUDA, CPU, and more | Large, highly optimized, highly general | General local inference with broad compatibility |
| Ollama | Product layer around local models | Usually built on top of llama.cpp | User-friendly and packaged | Serving models with a smooth developer experience |
| MLX | Apple’s framework for machine learning | Apple Silicon first | Framework oriented | Apple-native model experiments and applications |
This is not a better-or-worse comparison. It is a category comparison. ds4 is trying to prove that a narrower engine can be more readable, more teachable, and more exact-fit than a universal shell around everything.
this is beautiful. a very clean, minimalist implementation of DeepSeek-R1 inference on Metal. very readable and educational. thank you antirez
What this repo suggests about the next wave of local inference
ds4 points toward a world where local inference becomes more specialized, not less. As models grow more idiosyncratic, the winning move may be to stop hiding that shape behind universal abstractions.
That is what makes the repo interesting beyond the benchmark chatter. It reads like a disciplined systems argument: if the model is narrow, the engine can be narrow too. And if the engine is narrow, memory, disk, and GPU can finally be coordinated with less ceremony and more honesty.