ELF-REG: continuous diffusion language models that reason

ELF-REG (Li et al., 2026) scales fully continuous diffusion language models to math reasoning and code generation. Instead of generating token-by-token, the model starts from pure Gaussian noise in the representation space of a frozen Qwen3 encoder and denoises the entire response in parallel; a final decoder call turns the clean representation into text.

This demo runs the official checkpoints with the official inference code and shows you the process live: every step below decodes the model's predicted-clean response at that point of the trajectory, using the paper's prefix early-stop sampler (ρ=8) — the technique that delivers strong accuracy at very low NFE without any few-step training.

Things to try

  • Watch the answer crystallize out of noise: early steps are gibberish, the numerical answer often stabilizes many steps before the text stops changing (the paper's Figure 2 effect).
  • Drag the NFE slider down to 8–16: accuracy degrades gracefully thanks to early-stop.
  • Compare ELF-REG-B (104M) and ELF-REG-L (652M) on the same GSM8K question.

Verified against the paper: GSM8K pass@1 at NFE 64 = 38.11% (paper, 16 seeds) vs 38.40% ± 0.61 (this stack, full test set, 4 seeds) — see README.
Task / checkpoint
4 128
Examples
Task / checkpoint Prompt NFE (denoiser evals + 1 decode) — lower = faster & rougher Seed
{}

Attribution. Paper: ELF-REG · Code (MIT): lizeyu090312/scaling_dLM, built on lillian039/ELF · Official checkpoints: zl310/elf-reg (EMA-0.9999 inference copies mirrored at blanchon/elf-reg-demo-weights). The checkpoint weights carry no explicit license; the code is MIT. Demo not affiliated with the authors.