NVIDIA Optimizes Text Generation with DiffusionGemma

A major bottleneck in real-time AI development has been the slow pace of text generation. This limitation affects responsiveness, increases serving costs, and makes interactive experiences difficult to achieve.

Different approaches have been tried to speed up this process, but most rely on generating tokens one at a time. However, DiffusionGemma, developed by Google DeepMind in collaboration with NVIDIA, takes a new approach: it produces 256 tokens in parallel per step using diffusion-based denoising.

This innovative method enables faster and higher-throughput AI applications. On a single NVIDIA H100 Tensor Core GPU, DiffusionGemma can deliver up to 1,000 tokens/sec, while on NVIDIA DGX Spark, the speed reaches up to 150 tokens/sec.

For enterprise developers, this significant boost in performance translates into lower serving costs, increased concurrency, and more responsive user experiences without compromising model quality. The model's architecture is built on Gemma 4 26B A4B MoE and optimized for low-latency, memory-bound inference.

DiffusionGemma can run efficiently not only on NVIDIA data center GPUs but also on various client GPUs and systems. This versatility makes it an attractive option for developers looking to integrate text generation into their applications.

To get started with DiffusionGemma, developers can access the model through Hugging Face Transformers for initial testing and prototyping on NVIDIA GeForce RTX 5090 or DGX Spark. For higher throughput or concurrent multi-user serving on DGX Spark, DGX Station, and RTX PRO, they should use vLLM by following the playbooks in Table 2.

NVIDIA's support extends to all stages of development, from local prototyping to production deployment. The company provides Day 0 support across its hardware and software platforms, making it easy for developers to move their projects from experimentation to real-world applications.

Developers can start building with DiffusionGemma using free access to GPU-accelerated endpoints on build.nvidia.com as part of the NVIDIA Developer Program. This allows them to connect custom data sources directly to the browser experience.

The model is available today on Hugging Face with BF16 checkpoints, and an NVFP4 quantized checkpoint for DiffusionGemma can be accessed using NVIDIA Model Optimizer.

NVIDIA's NIM makes it simple to deploy DiffusionGemma from development into production. This tool packages the model as an optimized, containerized inference microservice that includes performance tuning, standardized APIs, and flexibility to run on-premises, in the cloud, or across hybrid environments.

Fine-tuning guides and recipes are available through the NVIDIA NeMo AutoModel library for developers looking to adapt the model to specific tasks or domains. This enables users to fine-tune models directly on top of HuggingFace checkpoints without conversion.