Kimi-K3 Architecture

Sparse MoE & Long-Context Ingress
STEPS:
01 / MULTIMODAL INGRESS 02 / SPARSE MOE (896 EXPERTS) 03 / ATTENTION & SYNTHESIS EMB TOKENS 16 ACT STATE TEXT TOKENS IMAGE PATCHES VIDEO FRAMES SHARED LATENT PROJECTION UNIFIED EMBED ROUTING GATE TOP-K SELECT EXPERT POOL 896 TOTAL EXPERTS 16 EXPERTS ACTIVATED / TOKEN KDA LANE DELTA ATTN GATED MLA LATENT ATTN COMPRESSED MEMORY LONG CONTEXT OUTPUT SYNTH
KIMI-K3 SYSTEM OVERVIEW Select or hover over components to inspect multimodal stream routing, sparse MoE selection, and compressed context memory.
Interactive Spec