Kimi-K3: The Open-Weight Model That Bets on Selective Intelligence

A 2.8-trillion-parameter system with 1M-token memory, native multimodality, and a MoE design built to stay sharp by activating less.

9 min read • View on GitHub • More from MoonshotAI

A vast machine fills the frame like a cathedral of mechanisms, with only a narrow set of gears and conduits lit up while most of the structure remains dormant. It explains Kimi-K3’s central idea: frontier-scale capacity does not have to mean dense, always-on computation.
The thesis image: a model that gets bigger by getting more selective.
Key Takeaways

The weird thing about Kimi-K3 is not its size. It is the argument hidden inside the size: frontier models may not need to become dense, memory-hungry blobs to stay capable. Moonshot AI is trying to prove a different operating point for open weights, one where selectivity does the heavy lifting.

That claim shows up everywhere in the release. Kimi-K3 pairs a 2.8T parameter budget with 104B activated parameters per token, a 1M-token context window, and native multimodality. The project is not just a model dump. It is a technical and economic thesis wrapped in weights.

Kimi K3 is a 2.8T-parameter model built on our Kimi Delta Attention and Attention Residuals, with native vision capabilities and a 1-million-token context window. It is the world's first open 3T-class model, designed for frontier intelligence across long-horizon coding, knowledge work, and reasoning.

Moonshot AI, Project Team · Kimi K3: Open Frontier Intelligence

The architecture is a bet on sparsity, not brute force

The numbers are the point, but not in the usual benchmark-chasing way. Kimi-K3 reportedly uses 896 experts and activates 16 of them per token. That means the model carries an immense reserve of capacity without paying for all of it on every forward pass.

A single token travels through a sparse routing stack instead of lighting up the whole model.

That architecture matters because 1M-token context is where ordinary transformer habits break down. A giant context window can become an expensive cache problem unless the model has some way to compress, route, or evict information without losing the thread.

Kimi-K3 core stack
- 2.8T total parameters
- 104B activated per token
- 896 experts
- 16 experts selected per token
- 69 KDA layers
- 24 gated MLA layers
- 1M-token context window

The hybrid attention stack is the real tell. Kimi Delta Attention handles the long-range bookkeeping, while gated MLA helps keep the attention footprint from exploding. Read together, the design looks less like a standard dense transformer and more like a selective memory system with a very large backup store.

A close-up cutaway shows a long ribbon of tokens being compressed into a narrow latent channel while only the essential fragments continue forward into memory. It explains how Kimi-K3 can handle extreme context without turning every token into a full dense attention pass.
Long context works when the model can keep the signal and discard the churn.

How Kimi-K3 keeps its memory from exploding

The practical question behind a 1M-token window is simple: what survives, and what gets thrown away? Kimi-K3’s answer appears to be a mix of latent attention and incremental updates, which gives it a way to preserve useful state without replaying the entire past at full cost.

That is why the model’s memory story is inseparable from its sparsity story. If only a small subset of experts activates per token, and the attention path is already compressed, then the system can afford to look back farther without collapsing under its own history.

Why native multimodality matters here

A lot of multimodal models still feel like text models with vision bolted on. Kimi-K3 is positioned differently: text, image, and video live in one native stack. That matters because it suggests a shared latent space, not a chain of adapters stitched together after the fact.

The payoff is not just prettier demos. Native multimodality is what makes the architecture feel like a general-purpose working model for code, documents, screenshots, diagrams, and video fragments all in the same session.

This is built for long-horizon work, not chat

Kimi-K3’s target jobs are the ones that punish shallow context: large codebases, document-heavy analysis, autonomous agent workflows, and tool use that stretches across many steps. In that world, the interesting metric is not how fast it answers a prompt. It is how long it can stay coherent while the task unfolds.

Model classStrengthTrade-offBest fit
Dense frontier modelSimple routing, strong single-pass reliabilityMemory and compute rise with every tokenGeneral chat and broad reasoning
Smaller open-weight MoECheaper activation, easier deploymentLess reserve capacity and shorter reachLocal inference and medium-scale tasks
Adapter-based multimodal modelFast way to add visionModality can feel stitched togetherImage-first workflows
Kimi-K3Huge reserve capacity with sparse activationHarder to host, harder to reason about operationallyLong-horizon coding, documents, and agentic work

Kimi K3 costs up to $15 per 1M tokens. I found a way to test it for $0. around 50M free tokens, no GPU and a 30-second setup. quick setup: 1/ get your API key at https://t.co/DPk6tnNfrU 2/ choose moonshotai/kimi-k3-free 3/ use the OpenAI-compatible base URL 4/ paste it into

axteriix.eth, axteriix · @axteriix on X

The license is part of the product

Moonshot did not stop at architecture. The Kimi K3 License turns the release into a business statement. It draws a line between research access and commercial use, especially for service providers who want to wrap the model into an API business of their own.

Use caseWhat the license implies
Research and experimentationThe model is available to inspect, test, and build with.
Commercial API serviceThe license introduces additional obligations and negotiation pressure.
Branding and attributionKimi-K3 naming can become a required visible part of deployment.
Distribution economicsMoonshot keeps control over how frontier capability is monetized.

That makes the release unusual in a useful way. Kimi-K3 is open-weight, but not ideologically open in the most permissive sense. It is an open frontier model with boundaries, and those boundaries are part of the product design.

Model: moonshotai/kimi-k3 API docs: https://t.co/VVnVqIumNi Official Kimi K3 technical report: https://t.co/Vvh3g4rJ9J

Phala, PhalaNetwork, 141,758 followers · @PhalaNetwork on X

What Kimi-K3 says about the open-weight frontier

Kimi-K3 is not just another giant release. It is a signal that open-weight frontier models may be heading toward a new default: sparse, memory-aware, and commercially opinionated. The point is not to become the biggest dense model in the room. The point is to become the most selective system that can still do frontier work.

That is a sharper ambition than raw scale. If Moonshot is right, the next generation of open models will win by making activation, memory, and modality more disciplined. Kimi-K3 is the proof attempt.