2 Shanghai AI Laboratory
💡 DAEDAL is a training-free denoising strategy that unlocks dynamic and variable-length generation for DLLMs, achieving performance comparable, and sometimes superior, to meticulously tuned fixed-length baselines.
📖 Click for the full abstract of DAEDAL
Diffusion Large Language Models (DLLMs) are emerging as a powerful alternative to the dominant Autoregressive Large Language Models, offering efficient parallel generation and capable global context modeling. However, the practical application of DLLMs is hindered by a critical architectural constraint: the need for a statically predefined generation length. This static length allocation leads to a problematic trade-off: insufficient lengths cripple performance on complex tasks, while excessive lengths incur significant computational overhead and sometimes result in performance degradation.
While the inference framework is rigid, we observe that the model itself possesses internal signals that correlate with the optimal response length for a given task. To bridge this gap, we leverage these latent signals and introduce DAEDAL, a novel training-free denoising strategy that enables Dynamic Adaptive Length Expansion for Diffusion Large Language Models. DAEDAL operates in two phases: 1) Before the denoising process, DAEDAL starts from a short initial length and iteratively expands it to a coarse task-appropriate length, guided by a sequence completion metric. 2) During the denoising process, DAEDAL dynamically intervenes by pinpointing and expanding insufficient generation regions through mask token insertion, ensuring the final output is fully developed.
Extensive experiments on DLLMs demonstrate that DAEDAL achieves performance comparable, and in some cases superior, to meticulously tuned fixed-length baselines, while simultaneously enhancing computational efficiency by achieving a higher effective token ratio. By resolving the static length constraint, DAEDAL unlocks new potential for DLLMs, bridging a critical gap with their Autoregressive counterparts and paving the way for more efficient and capable generation.
- [2025/08/04] Code are released!
- [2025/08/04] Paper is available on arXiv!
We introduce DAEDAL, a training-free, two-phase inference strategy designed to listen to these internal signals and empower the DLM with dynamic, variable-length denoising.
-
Initial Length Adjustment: Before the main denoising begins, DAEDAL starts with a short, unified length. It performs a quick check: "Based on this prompt, is the currently allocated length sufficient?" It measures this by checking the EOS confidence at the end of the sequence. If the confidence is low, the model is signaling the budget is insufficient. DAEDAL then expands the length by adding more
[MASK]tokens and repeats this check until the model is confident the budget is sufficient. This sets a reasonable global length for the task. -
Iterative Mask Insertion: During the step-by-step denoising process, DAEDAL continues to monitor the model's confidence. If it finds a specific
[MASK]token that the model is extremely uncertain abou, it marks this as an "expansion point". DAEDAL intervenes by replacing that single[MASK]with a block of new[MASK]s. This dynamically inserts "breathing room" into the sequence exactly where the model needs it to elaborate on a complex thought or bridge a logical gap.
DAEDAL allows the DLM to autonomously decide its own response length on a per-problem basis. By resolving the static length constraint, DAEDAL unlocks new potential for DLLMs, bridging a critical gap with their Autoregressive counterparts and paving the way for more efficient and capable generation.
🧑💻 User Input
You are a math expert. You will be given a question to solve. Solve it step by step. Wrap the final answer in a \boxed{}.
Respond in the following format:
<reasoning>
Your reasoning here
</reasoning>
<answer>
\boxed{...}
</answer>
A juggler can juggle 16 balls. Half of the balls are golf balls, and half of the golf balls are blue. How many blue golf balls are there?🤖 Model Output
- Dark blue background with white text: Tokens selected to be filled.
- Orange background: Tokens for expansion.
- White background: Previously filled tokens.
- Shade of the background color (for denoised tokens): Represents the confidence level.
git clone https://github.com/Li-Jinsong/DAEDAL.git
cd DAEDAL
conda create -n daedal python=3.10
conda activate daedal
pip install -r requirements.txtLLaDA-8B-Instruct: GSAI-ML/LLaDA-8B-Instruct
LLaDA-1.5: GSAI-ML/LLaDA-1.5
Then update the MODEL_PATH in scripts
sh scripts/eval_LLaDA_DAEDAL.shsh scripts/eval_LLaDA_Baseline.shIf you find our work helpful, please consider giving a star ⭐ and citation 📝
@article{li2025beyond,
title={Beyond Fixed: Training-Free Variable-Length Denoising for Diffusion Large Language Models},
author={Li, Jinsong and Dong, Xiaoyi and Zang, Yuhang and Cao, Yuhang and Wang, Jiaqi and Lin, Dahua},
journal={arXiv preprint arXiv:2508.00819},
year={2025}
}The code is built upon the below repositories, we thank all the contributors for open-sourcing.