Kimi-K3: The Open-Weight Model That Bets on Selective Intelligence
A 2.8-trillion-parameter system with 1M-token memory, native multimodality, and a MoE design built to stay sharp by activating less.
- Kimi-K3 argues that frontier models should get smarter by activating less, not by becoming denser.
- Its real novelty is the combination of 2.8T total parameters, 104B active per token, and a 1M-token context window in one design.
- The model’s architecture turns long-context memory into a routing problem, which is why sparsity and multimodality matter together.
- The license is part of the product, because Moonshot is also defining how an open-weight frontier model should be commercialized.
The weird thing about Kimi-K3 is not its size. It is the argument hidden inside the size: frontier models may not need to become dense, memory-hungry blobs to stay capable. Moonshot AI is trying to prove a different operating point for open weights, one where selectivity does the heavy lifting.
That claim shows up everywhere in the release. Kimi-K3 pairs a 2.8T parameter budget with 104B activated parameters per token, a 1M-token context window, and native multimodality. The project is not just a model dump. It is a technical and economic thesis wrapped in weights.
Kimi K3 is a 2.8T-parameter model built on our Kimi Delta Attention and Attention Residuals, with native vision capabilities and a 1-million-token context window. It is the world's first open 3T-class model, designed for frontier intelligence across long-horizon coding, knowledge work, and reasoning.
The architecture is a bet on sparsity, not brute force
The numbers are the point, but not in the usual benchmark-chasing way. Kimi-K3 reportedly uses 896 experts and activates 16 of them per token. That means the model carries an immense reserve of capacity without paying for all of it on every forward pass.
That architecture matters because 1M-token context is where ordinary transformer habits break down. A giant context window can become an expensive cache problem unless the model has some way to compress, route, or evict information without losing the thread.
Kimi-K3 core stack
- 2.8T total parameters
- 104B activated per token
- 896 experts
- 16 experts selected per token
- 69 KDA layers
- 24 gated MLA layers
- 1M-token context window
The hybrid attention stack is the real tell. Kimi Delta Attention handles the long-range bookkeeping, while gated MLA helps keep the attention footprint from exploding. Read together, the design looks less like a standard dense transformer and more like a selective memory system with a very large backup store.
How Kimi-K3 keeps its memory from exploding
The practical question behind a 1M-token window is simple: what survives, and what gets thrown away? Kimi-K3’s answer appears to be a mix of latent attention and incremental updates, which gives it a way to preserve useful state without replaying the entire past at full cost.
That is why the model’s memory story is inseparable from its sparsity story. If only a small subset of experts activates per token, and the attention path is already compressed, then the system can afford to look back farther without collapsing under its own history.
Why native multimodality matters here
A lot of multimodal models still feel like text models with vision bolted on. Kimi-K3 is positioned differently: text, image, and video live in one native stack. That matters because it suggests a shared latent space, not a chain of adapters stitched together after the fact.
The payoff is not just prettier demos. Native multimodality is what makes the architecture feel like a general-purpose working model for code, documents, screenshots, diagrams, and video fragments all in the same session.
This is built for long-horizon work, not chat
Kimi-K3’s target jobs are the ones that punish shallow context: large codebases, document-heavy analysis, autonomous agent workflows, and tool use that stretches across many steps. In that world, the interesting metric is not how fast it answers a prompt. It is how long it can stay coherent while the task unfolds.
| Model class | Strength | Trade-off | Best fit |
|---|---|---|---|
| Dense frontier model | Simple routing, strong single-pass reliability | Memory and compute rise with every token | General chat and broad reasoning |
| Smaller open-weight MoE | Cheaper activation, easier deployment | Less reserve capacity and shorter reach | Local inference and medium-scale tasks |
| Adapter-based multimodal model | Fast way to add vision | Modality can feel stitched together | Image-first workflows |
| Kimi-K3 | Huge reserve capacity with sparse activation | Harder to host, harder to reason about operationally | Long-horizon coding, documents, and agentic work |
Kimi K3 costs up to $15 per 1M tokens. I found a way to test it for $0. around 50M free tokens, no GPU and a 30-second setup. quick setup: 1/ get your API key at https://t.co/DPk6tnNfrU 2/ choose moonshotai/kimi-k3-free 3/ use the OpenAI-compatible base URL 4/ paste it into
The license is part of the product
Moonshot did not stop at architecture. The Kimi K3 License turns the release into a business statement. It draws a line between research access and commercial use, especially for service providers who want to wrap the model into an API business of their own.
| Use case | What the license implies |
|---|---|
| Research and experimentation | The model is available to inspect, test, and build with. |
| Commercial API service | The license introduces additional obligations and negotiation pressure. |
| Branding and attribution | Kimi-K3 naming can become a required visible part of deployment. |
| Distribution economics | Moonshot keeps control over how frontier capability is monetized. |
That makes the release unusual in a useful way. Kimi-K3 is open-weight, but not ideologically open in the most permissive sense. It is an open frontier model with boundaries, and those boundaries are part of the product design.
Model: moonshotai/kimi-k3 API docs: https://t.co/VVnVqIumNi Official Kimi K3 technical report: https://t.co/Vvh3g4rJ9J
What Kimi-K3 says about the open-weight frontier
Kimi-K3 is not just another giant release. It is a signal that open-weight frontier models may be heading toward a new default: sparse, memory-aware, and commercially opinionated. The point is not to become the biggest dense model in the room. The point is to become the most selective system that can still do frontier work.
That is a sharper ambition than raw scale. If Moonshot is right, the next generation of open models will win by making activation, memory, and modality more disciplined. Kimi-K3 is the proof attempt.





