Diffusion LLMs
---
Research checked: 20 September 2026.
Most familiar language models generate text from left to right, predicting each next token from the tokens already available. Diffusion language models explore a different generative process: start with corrupted or incomplete text and refine it over multiple steps. Several positions can be predicted within one step. This changes the order of computation and opens different ways to trade speed against output quality.
Source: Figure 2 from Nie et al., “Large Language Diffusion Models” (2025).
Text requires a different treatment from a photograph. Tokens are discrete symbols, so adding Gaussian noise directly to token identifiers would not have the ordinary meaning it has for pixel values. Some language diffusion approaches operate in continuous representations; masked discrete diffusion instead corrupts text by replacing tokens with a mask symbol. The model learns to predict missing content from what remains. LLaDA is a prominent example of this masked approach. Nie et al., Large Language Diffusion Models.
At generation time, an illustrative masked sequence might evolve from “[MASK] [MASK] [MASK]” to “The [MASK] opened” and finally “The door opened.” The exact schedule determines which positions are filled, retained, or reconsidered. Unlike strict left-to-right generation, the procedure can use visible information on both sides of a missing span. That makes infilling a natural capability to investigate.
LLaDA demonstrated an eight-billion-parameter diffusion language model trained from scratch, with competitive results against relevant autoregressive baselines across several evaluated tasks. Its significance is that instruction following and useful language-model capabilities need not depend exclusively on autoregressive generation. It does not establish a universal advantage over autoregressive models. Nie et al..
Dream 7B investigates another route, including initialization from an autoregressive language model and training techniques suited to diffusion. Its authors report flexible generation order, infilling, and adjustable quality-speed behavior. Taken together, these projects show that diffusion language modeling is a family of design choices rather than one fixed training and sampling recipe. Ye et al., Dream 7B: Diffusion Large Language Models.
Parallel prediction is attractive, but it does not make generation free. Each denoising step still requires model computation, and many such steps may be needed. My systems-level reading is that the relevant comparison must include sequence length, number of refinement steps, hardware, batching, and output quality. Comparing only the number of tokens produced during one pass can hide the cost of repeated passes.
There are also interaction questions. A writing interface may value an early stream of stable text. An editing tool may value revising a whole span under constraints. A code-completion tool may benefit from using both a prefix and a suffix. These applications should be evaluated separately rather than compressed into one headline generation-speed number.
I would compare diffusion and autoregressive systems at matched quality on tasks that include free generation, infilling, and constrained editing. Measure complete-response latency and the time until usable output appears. Diffusion’s promise is a broader set of choices about how to construct text. The research question is which of those choices produces a worthwhile improvement for a particular workload.