Gemma Architecture

Gemma 4 Mode
01. Multimodal Preprocessing 02. Shared Transformer Trunk 03. Memory & Logit Controls PROJECT LOGITS KV CACHE TEXT TOKENS VISION PATCHES AUDIO FRAMES TEXT EMBED SIGLIP TOWER AUDIO ENC MODALITY PROJ LOCAL ATTN GLOBAL ATTN LOCAL ATTN GLOBAL ATTN SOFT-CAPPING (ON) SHARED KV CACHE
Click any node to inspect details