Google DeepMind open-sources the Gemma 4 multimodal model family

ME News reports that on April 3 (UTC+8), Google DeepMind recently open-sourced the Gemma 4 family of multimodal models. This series supports text and image inputs (with smaller models also supporting audio) and generates text outputs, offering both pre-trained and instruction-tuned variants. The models feature a context window of up to 256K tokens and support over 140 languages. They are available in two architectures—dense and mixture-of-experts (MoE)—with four sizes: E2B, E4B, 26B A4B, and 31B. Core capabilities include high-performance inference, extended multimodal processing, on-device optimization, expanded context windows, enhanced coding and agent abilities, and native system prompt support. Technically, the models employ a hybrid attention mechanism with unified key-value pairs and scaled RoPE (p-RoPE) across global layers. The E2B and E4B models utilize Per-Layer Embedding (PLE), resulting in fewer effective parameters than total parameters. The 26B A4B MoE model activates only 3.8B parameters during inference, achieving speed comparable to a 4B-parameter model. (Source: InFoQ)