DiffusionGemma is an experimental open-weight language model that generates text through discrete diffusion, refining 256-token blocks in parallel rather than producing tokens sequentially. The design is intended to avoid the decoding bottleneck of autoregressive language models. The model was created by fine-tuning the mixture-of-experts Gemma 4 model, which has 3.8 billion activated parameters and 25.2 billion total parameters. Its two-stage training process used fewer than 10% of the original autoregressive model’s total training-token budget. Supervised fine-tuning first taught bidirectional denoising; a second phase combined reinforcement learning and sampler distillation to improve generation quality and inference efficiency. Across its evaluation suite, DiffusionGemma generated about 20 tokens per forward pass and about 1,500 output tokens per second on a single Nvidia H100 GPU. The researchers say these results establish a new speed-capability trade-off frontier and exceed autoregressive models, including those using speculative decoding. DiffusionGemma retains Gemma 4’s thinking mode, multimodal-input, and long-context support. It can also still generate text autoregressively with minor performance degradation, suggesting potential hybrid diffusion-autoregressive decoding systems.

The paper "On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?" argues that large language models generate text by statistically predicting likely sequences of words. The publication gained significant attention following the firing of two authors, Timnit Gebru and Margaret Mitchell, by Google.

GPT-NL is a language model designed specifically for the Netherlands, emphasizing strong governance, transparency, and public values. It aims to integrate AI into various sectors such as the workplace, education, and public services while maintaining control over the technology.