Kimi-K3 Architecture
Sparse MoE & Long-Context Ingress
STEPS:
1
2
3
4
5
Pause
Reset
01 / MULTIMODAL INGRESS
02 / SPARSE MOE (896 EXPERTS)
03 / ATTENTION & SYNTHESIS
EMB
TOKENS
16 ACT
STATE
TEXT
TOKENS
IMAGE
PATCHES
VIDEO
FRAMES
SHARED LATENT
PROJECTION
UNIFIED EMBED
ROUTING
GATE
TOP-K SELECT
EXPERT POOL
896 TOTAL EXPERTS
16 EXPERTS ACTIVATED / TOKEN
KDA LANE
DELTA ATTN
GATED MLA
LATENT ATTN
COMPRESSED
MEMORY
LONG CONTEXT
OUTPUT
SYNTH
KIMI-K3 SYSTEM OVERVIEW
Select or hover over components to inspect multimodal stream routing, sparse MoE selection, and compressed context memory.
Interactive Spec