DiffusionGemma is a 26B Mixture of Experts (MoE) model that utilizes text diffusion for text generation. It can generate entire blocks of text simultaneously, achieving up to 4x faster performance on GPUs compared to traditional autoregressive Large Language Models.

OpenAI developed a new text-generation model, GPT-2, capable of producing coherent and versatile prose from prompts. Due to safety and security concerns, OpenAI has opted not to release the full algorithm, instead providing a smaller version and withholding data sets and training codes.