Gemma 4: how to boost your local AI performance by 137 percent

The local AI landscape is evolving rapidly, and staying updated with the latest tools is crucial for maximizing hardware potential. Google's recent addition of Multi-Token Prediction (MTP) for the Gemma 4 model family offers a massive performance boost that many users are currently overlooking. - Using an MTP drafter boosted Gemma 4 12B speed from 75 to 179 tokens per second. - Performance gains reached 137% on an RTX 4070 Ti Super GPU. - Speculative decoding uses a smaller drafter model to guess tokens ahead of time. - Activation is simple via the latest llama.cpp build and four specific command flags.

Understanding MTP technology The secret behind this performance leap is Multi-Token Prediction. Google uses this architecture to enable speculative decoding. Instead of the heavy target model generating every token one by one, a small "drafter" model makes guesses about upcoming tokens. The large Gemma 4 model then validates these guesses in parallel, significantly reducing the time required for generation. Even when the model rejects some guesses, the process remains efficient, correcting errors on the fly. This method is particularly effective for dense models like Gemma 4 12B, where patterns in code or structured data make it easier for the drafter to predict upcoming sequences accurately.

How to implement the speed boost To enable this, users need to download the matching drafter file (such as mtp-gemma-4-12B-it.gguf) from the same repository as their main model. Proper alignment between the model and its drafter is essential. Users then simply restart their llama-server with four additional flags: specifying the drafter path, offloading it to the GPU, and defining the maximum number of speculation rounds. Testing across various workloads, from prose writing to code generation, showed consistent speed improvements. While code tasks hit the 200 tokens per second mark, even complex creative writing tasks saw a 73% speed increase, proving the utility of MTP across diverse use cases.

Why this matters for the ecosystem For developers and power users, this update brings near-cloud performance to local machines without needing high-end enterprise hardware. By making large models like Gemma 4 run efficiently on consumer GPUs, Google is narrowing the gap between local privacy-focused AI and massive commercial APIs, empowering users to build complex, responsive AI tools directly on their own workstations.