Google has released Gemma 4, a large language model that qualifies as truly free and open source under the Apache 2.0 license. Unlike other models with restrictive licenses, Gemma 4 offers total freedom without research-only limitations or profit-sharing requirements.
Unprecedented Size and Efficiency
Gemma 4's most remarkable feature is its compact size. The big model runs on consumer GPUs, while the Edge version operates on phones or Raspberry Pi devices, yet achieves intelligence levels comparable to models requiring data center GPUs. The 31 billion parameter version scores similarly to Kimmy K2.5 thinking, but can run locally with a 20 GB download at roughly 10 tokens per second on a single RTX 4090. In contrast, Kimmy K2.5 requires a 600 GB download, 256 GB of RAM, aggressive quantization and multiple H100s.
Attacking the Memory Bottleneck
Google achieved this shrinkage by targeting AI's real bottleneck: memory bandwidth. Every token generation requires reading massive model weights in VRAM, making memory access cost more critical than model size. Alongside Gemma 4, Google introduced Turbo Quant, a new quantization approach that compresses model weights more efficiently.
Turbo Quant and Per-Layer Embeddings
Turbo Quant uses two innovative steps: it transforms XYZ Cartesian coordinates into polar coordinates with predictable angles, skipping normalization steps and applies the Johnson-Lindenstrauss transform to compress high-dimensional data into single sign bits while preserving distances. However, the real secret behind Gemma 4's small size is per-layer embeddings, which give each neural network layer its own mini cheat sheet for each token, introducing information exactly when needed rather than all at once.
Practical Applications
The result is a small, smart and efficient model suitable for local running and fine-tuning. While still not replacing high-end models, it represents a significant step forward in accessible AI.