Paper deep dive
In-Place Instruction Following in Diffusion Language Models
Zheng Nie, Zherui Li, Jiaming Zhang, Kun Wang, Zhenhong Zhou, Yufei Guo
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 9/13/2026, 4:54:20 AM
Summary
This paper introduces In-place Instruction Following (IIF), a task where Diffusion Large Language Models (dLLMs) generate text while adhering to constraints anchored at arbitrary positions. The authors propose IIF-Bench, a hierarchical benchmark for evaluating literal, style, and discourse constraints, and Graft, a post-training framework combining constraint-aware Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO). Experiments show Graft significantly improves IIF scores on four dLLMs by addressing the under-prioritization of constraint spans during denoising.
Entities (9)
Relation Signals (7)
IIF-Bench → evaluates → In-place Instruction Following
confidence 96% · construct IIF-Bench, a hierarchical benchmark spanning literal, style, and discourse-function constraints
Graft → improves → In-place Instruction Following
confidence 95% · On four representative dLLMs, GRAFT raises the average IIF score from 57.75 to 73.10
Graft → consistsof → Direct Preference Optimization
confidence 94% · It then applies constraint-aware preference optimization
Graft → consistsof → Supervised Fine-Tuning
confidence 94% · Graft first establishes basic IIF capability through constraint-aware Supervised Fine-Tuning
Diffusion Large Language Models → supports → In-place Prompting
confidence 93% · dLLMs generate text via bidirectional iterative denoising, naturally supporting user-specified constraints... known as In-place Prompting (IPP)
Graft → testedon → LLaDA-1.5
confidence 90% · On four representative dLLMs, Graft improves... LLaDA-1.5
Graft → testedon → Dream-7B
confidence 90% · On four representative dLLMs, Graft improves... Dream-7B
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Diffusion Large Language Models (dLLMs) generate text via bidirectional iterative denoising, naturally supporting user-specified constraints anchored at arbitrary output positions, a paradigm known as In-place Prompting (IPP). We formalize this as the In-place Instruction Following (IIF) task and construct IIF-Bench, a hierarchical benchmark spanning literal, style, and discourse-function constraints, paired with a rubric-based local-global evaluation protocol. An inference-time attention-bias probe suggests that vanilla dLLMs often under-prioritize constraint spans during denoising. We then propose GRAFT, an IPP-oriented post-training framework combining constraint-aware SFT and preference optimization. On four representative dLLMs, GRAFT raises the average IIF score from 57.75 to 73.10 (+15.35 points), with absolute gains of 15.91 and 15.57 points on literal and discourse-function constraints, while preserving general generation ability.
Tags
Links
- Source: https://arxiv.org/abs/2609.07160v1
- Canonical: https://arxiv.org/abs/2609.07160v1
Trouble viewing inline? Open PDF directly →
Full Text
77,930 characters extracted from source content.
Expand or collapse full text
In-Place Instruction Following in Diffusion Language Models Zheng Nie Affiliation: National University of Singapore Zherui Li Affiliation: Nanyang Technological University Jiaming Zhang Affiliation: Nanyang Technological University Kun Wang Affiliation: Nanyang Technological University Zhenhong Zhou Affiliation: Nanyang Technological University Yufei Guo Affiliation: Peking University Abstract Diffusion Large Language Models (dLLMs) generate text via bidirectional iterative denoising, naturally supporting user-specified constraints anchored at arbitrary output positions, a paradigm known as In-place Prompting (IPP). We formalize this as the In-place Instruction Following (IIF) task and construct IIF-Bench, a hierarchical benchmark spanning literal, style, and discourse-function constraints, paired with a rubric-based local-global evaluation protocol. An inference-time attention-bias probe suggests that vanilla dLLMs often under-prioritize constraint spans during denoising. We then propose Graft, an IPP-oriented post-training framework combining constraint-aware SFT and preference optimization. On four representative dLLMs, Graft raises the average IIF score from 57.75 to 73.10 (↑ 15.35 points), with absolute gains of 15.91 and 15.57 points on literal and discourse-function constraints, while preserving general generation ability. Our code and data are available at: https://anonymous.4open.science/r/Graft-36B5. 1 Introduction The rapid development of Large Language Models (LLMs) has significantly enhanced the instruction-following capabilities of natural language generation systems (Radford et al., 2019; Zhao et al., 2025b; Zhou et al., 2023a; Lou et al., 2024). Currently, mainstream LLMs still primarily rely on prefix-based prompting as their mode of interaction: users specify task requirements (Brown et al., 2020; Son et al., 2024), format constraints (Liu et al., 2024; Zhang et al., 2026), or preference descriptions (Li et al., 2024; Zhao et al., 2025a) in the input prompt, after which the model generates a complete response from left to right. While this paradigm is well-suited for general open-ended generation, many real-world scenarios require models to satisfy local constraints at specific positions in the output, such as preserving a given phrase (Zhou et al., 2023a; Pyatkin et al., 2026), performing localized style transfer (Mai et al., 2023; Chen, 2024), or inserting a transition or rebuttal at a designated location (Lai et al., 2024; Tao et al., 2024). For standard autoregressive (AR) LLMs, such arbitrary-position control typically requires additional constrained decoding (Beurer-Kellner et al., 2024; Banerjee et al., 2025), infilling-oriented training (Bavarian et al., 2022; Roziere et al., 2023; Ren et al., 2024), or multi-round editing processes (Ge et al., 2024; Tian et al., 2025), rather than being a native capability of their prefix-to-suffix generation interface. The recent development of Diffusion Large Language Models (dLLMs) offers a natural opportunity to move beyond prefix-only interaction (Nie et al., 2025; Ye et al., 2025; Bie et al., 2025). Unlike autoregressive (AR) LLMs, dLLMs generate from a masked canvas through iterative bidirectional denoising, allowing fixed tokens or structured constraints to be anchored at arbitrary positions in the output sequence (Arriola et al., 2025; Yang et al., 2025). This paradigm, known as In-place Prompting (IPP) (Lee et al., 2025; Jin et al., 2025), is important not merely as an alternative decoding format, but as a new user-facing control interface: users can specify what must happen where in the output, enabling localized editing, structured generation, precise phrase preservation, and position-specific discourse control without relying on constrained decoding or multi-round rewriting Zhang et al. (2026). Although recent studies have used similar anchoring mechanisms for reasoning, safety, and template infilling (Jin et al., 2025; Horvitz et al., 2025; Zhang et al., 2025; Lee et al., 2025; Li et al., 2026b), they mostly treat in-place spans as internal algorithmic devices rather than as an explicit instruction-following capability. This leaves open a central question: can dLLMs reliably execute user-specified in-place constraints while preserving natural boundary transitions and global semantic consistency? To answer this question, we define the In-place Instruction Following (IIF) task. Given a user instruction and an incomplete output canvas containing a fixed in-place constraint, IIF requires the model to generate the surrounding spans while preserving the constraint, connecting it naturally to the context, and maintaining global consistency with the user intent. We further construct IIF-Bench, a hierarchical IIF benchmark covering explanatory, knowledge-intensive, and creative generation scenarios across three progressive levels: ❶ literal constraints, ❷ style constraints, and ❸ discourse-function constraints. To evaluate this capability, we further design a rubric-based fine-grained local-global evaluation protocol that jointly assesses local constraint execution, boundary coherence, global consistency, and fluency, and cross-validates automatic judgments with LLM-as-a-judge, a specialized reward model, and human evaluation. Building on IIF-Bench, we introduce an inference-time attention-bias probe to diagnose how dLLMs use in-place constraints during denoising. The probe requires no parameter updates and selectively increases attention to the IPP span. We find that moderate attention bias improves in-place constraint following, while overly strong bias harms generation quality, suggesting that IIF failures are partly caused by insufficient prioritization of the constraint span rather than a complete lack of linguistic capability. Motivated by this finding, we propose an IPP-oriented post-training framework called Guided Refinement via Anchor-aware Focus Tuning (Graft). Graft first establishes basic IIF capability through constraint-aware Supervised Fine-Tuning, which treats in-place spans as fixed conditioning variables rather than generation targets. It then applies constraint-aware preference optimization to improve the balance among local execution, boundary coherence, and global consistency. The experimental results validate the effectiveness of Graft framework. On 4 representative dLLMs, Graft improves the average overall IIF score from 57.75 to 73.10 (↑ 15.35 points), with absolute gains of 15.91 and 15.57 points on literal and discourse-function constraints, respectively. Further analysis shows that the model achieves stable improvements in local constraint execution, boundary coherence, and global consistency, while incurring only a minor impact on general generation ability. These results demonstrate that Graft can effectively enhance the in-place instruction-following capability of dLLMs, providing a viable path toward coordinating localized constraints with global semantic consistency. Our contributions are summarized as follows: • Task and benchmark. We formalize the IIF task and construct IIF-Bench, a hierarchical benchmark covering literal, style, and discourse-function constraints. • Fine-grained evaluation. We design a fine-grained evaluation protocol and use an attention bias probe to analyze a key bottleneck of dLLMs in following in-place constraints. • Graft. We propose Graft, an IPP-oriented post-training framework that improves in-place constraint following while mitigating the conflict between local constraints and global consistency. Figure 1: Comparison of inference paradigms between standard dLLM decoding and IPP. 2 Background 2.1 Inference Paradigm of dLLMs To efficiently generate long texts, dLLMs adopt a semi-autoregressive paradigm: sequentially decoding text blocks of length K, while internally performing S-step parallel denoising based on the initial state W(0)=C⊕[[MASK]]KW^(0)=C [[MASK]]^K. At each step s∈1,…,Ss∈\1,…,S\, the model predicts the distribution for the remaining masked set ℳs−1M_s-1 given the intermediate state W(s−1)W^(s-1), dynamically filtering out ⌊K/S⌋ K/S positions ℐsI_s with the highest confidence: ℐs=TopKi∈ℳs−1(maxv∈pθ(wi=v∣W(s−1)),⌊K/S⌋). _s=TopK_i _s-1 ( _v p_θ(w_i=v W^(s-1)),\, K/S ). (1) The sequence state then undergoes an update where greedy decoding is applied to the selected positions, unselected targets are forcefully re-masked to defer their generation, and previously resolved context tokens are preserved unchanged, as formalized below: wi(s)=argmaxv∈pθ(wi=v∣W(s−1)),if i∈ℐs,[MASK],if i∈ℳs−1∖ℐs,wi(s−1),if i∉ℳs−1. w_i^(s)= cases v \,p_θ(w_i=v W^(s-1)),&if i _s,\\ [MASK],&if i _s-1 _s,\\ w_i^(s-1),&if i _s-1. cases (2) This predict-and-remask cycle continuously iterates (ℳs=ℳs−1∖ℐsM_s=M_s-1 _s) until the masked set is empty (ℳS=∅M_S= ), at which point the fully resolved text block is merged into the context C to initiate the AR generation for the next block. This standard inference flow is illustrated in the middle panel of Figure 1. Although dLLMs utilize parallel denoising internally, they typically follow a unidirectional trajectory where the [[MASK]]K[[MASK]]^K block is appended strictly to the end of the existing context C. In this conventional mode, dLLM acts as a semi-autoregressive (AR) generator, extending the sequence linearly as a series of resolved blocks. 2.2 Inference Paradigm of IPP The In-place Prompting (IPP) paradigm reformulates sequential generation into a constraint-aware, bidirectional infilling task within the dLLM framework. It partitions the target sequence y into a prompt prefix yprompty_prompt, a front generation section yfronty_front of length LfL_f, an immutable intermediate anchor yanchory_anchor, and a back generation section ybacky_back of length LbL_b, denoted as y=(yprompt,yfront,yanchor,yback)y=(y_prompt,y_front,y_anchor,y_back). The inference process maps directly onto the dLLM formulation by setting the context C=ypromptC=y_prompt, the target block size K=Lf+LbK=L_f+L_b, and embedding the frozen anchor. The initial canvas state at denoising step s=0s=0 is formulated as: y(0)=(yprompt,[MASK],…,[MASK]⏟Lf tokens,yanchor,[MASK],…,[MASK]⏟Lb tokens). y^(0)= (y_prompt, [MASK],…,[MASK]_L_f tokens,y_anchor, [MASK],…,[MASK]_L_b tokens ). (3) During the iterative refinement from step s=1s=1 to S, the parallel predictor pθp_θ evaluates token distributions strictly within the remaining masked positions ℳs−1⊆(yfront∪yback)M_s-1 (y_front∪ y_back). Crucially, yprompty_prompt and yanchory_anchor are treated as fixed context outside the dynamic masked set (i.e., ∀i∈yanchor,i∉ℳs−1∀ i∈ y_anchor,\,i _s-1), remaining structurally frozen as persistent conditioning variables. This forces the model to bidirectionally converge around the predefined anchor. 3 IIF-Bench To systematically evaluate the local control capabilities of dLLMs, we propose the In-place Instruction Following (IIF) task and construct a corresponding benchmark IIF-Bench. 3.1 Motivation and Task Formulation Motivation. Unlike left-to-right AR LLMs, dLLMs support in-place prompting (IPP), allowing constraints to be inserted at arbitrary output positions. Existing benchmarks usually specify constraints upfront and focus on easily verifiable surface forms (Zhou et al., 2023a; Jiang et al., 2024), leaving mid-generation intervention and its tension with global coherence largely unevaluated (Zhou et al., 2023b; Iso, 2024). Therefore, IIF-Bench achieves constraint-following evaluations across three dimensions: ❶ It evaluates IPP capabilities within outputs, not just prefix constraints; ❷ it spans a three-level difficulty spectrum from literal to discourse constraints; and ❸ it inherently embodies and quantifies the core tension between localized constraints and global consistency (Local-Global Tension), making it an ideal probe for diagnosing the attention allocation mechanisms of dLLMs. Figure 2: Overview of the IIF-Bench benchmark and its evaluation protocol. Task Formulation. Under the generalized IPP framework, we formally define the In-place Instruction Following (IIF) task. Given a user instruction represented by the prompt context yprompty_prompt and a fixed in-place control span yanchory_anchor, wrapped by <LITERAL>, <STYLE>, or <DISCOURSE> tags, the model is required to generate the surrounding spans yfronty_front and ybacky_back. The generated content should realize the requirement specified by yanchory_anchor while remaining locally coherent around the anchor and globally consistent with the user instruction. Formally, letting ygen=(yfront,yback)y_gen=(y_front,y_back), we define: y^gen=argmaxygenpθ(ygen∣yprompt,yanchor)s.t.ℛℓ(y^gen,yanchor)=1. y_gen= *arg\,max_y_genp_θ\! (y_gen y_prompt,y_anchor ) .t. _ \! ( y_gen;y_anchor )=1. (4) Here, ℛℓR_ denotes a level-specific realization criterion. Based on the depth of control imposed by yanchory_anchor on the model’s output, we divide the benchmark into 3 difficulty levels detailed in Section 3.2. 3.2 Dataset Introduction IIF-Bench comprises approximately 5K high-quality instances, with data sources broadly covering complex scenarios such as Alpaca Taori et al. (2023), LFQA, and Poetry. To comprehensively measure the model’s following capabilities under varying depths of intervention, we organize the dataset into three difficulty levels: • L1 Literal Constraints. Focuses on exact replication and seamless embedding, requiring the model to use a specified exact phrase as an anchor and output it verbatim at the designated position without disrupting the syntactic structure. • L2 Style Constraints. Focuses on the balance between forced local style transfer and global semantic tension. It requires the model to manifest one of 12 predefined literary styles within a specific paragraph. • L3 Discourse Constraints. Focuses on advanced rhetoric and logical deduction, requiring the model to execute 8 high-frequency logical actions within local paragraphs. It mandates that this local action form a coherent logical loop with the global core argument. 3.3 Multi-dimensional Evaluation Protocol IIF requires both local constraint execution and global response quality. We therefore evaluate each output with a fine-grained local-global rubric rather than a single holistic score. All four dimensions D1–D4 are defined per level to reflect the distinct focal points of each constraint type, while preserving a consistent evaluation structure across levels: D1 Local Execution, D2 Contextual Integration, D3 Boundary Quality, and D4 Global Requirement. The definition of each dimension is specialized for each benchmark level. For L1, D1 checks exact preservation and positional appropriateness of the literal anchor, and absence of paraphrastic substitution; D2 checks whether the pre-constraint text leads syntactically and semantically into the anchor; D3 checks whether the post-constraint text continues naturally from it; D4 checks instruction relevance, output completeness, and absence of verbatim input copying. For L2, D1 checks whether the target style is realized across three independent failure modes: style onset sharpness, style depth beyond surface markers, and global style contamination; D2 checks semantic relevance of the constrained span to the preceding context, with conflict-aware adjustment described in Appendix E; D3 checks transition quality at the style tag boundary; D4 checks overall narrative integrity and instruction fulfillment. For L3, D1 checks whether the required discourse function is correctly performed using function-specific sub-criteria (e.g., concreteness and challenge strength for Counterexample; conditional framing and testability for Hypothesis); D2 checks consistency between the discourse function segment and the global argument; D3 checks whether the function boundary is clearly demarcated; D4 checks grammatical correctness and clarity of expression within the constrained span. To improve evaluation reliability, we use a rubric-based LLM-as-judge framework with GPT-5 as the primary evaluator and JudgeLM as an additional evaluator to enhance assessment credibility. Automatic results are reported in the main experiments, with human evaluation details provided in the appendix. 4 Method Building on IIF-Bench, we first analyze how dLLMs use in-place constraints during iterative denoising, then improve this capability through post-training. We introduce an inference-time attention-bias probe to test whether failures in IIF stem from insufficient prioritization of the constraint span (Section 4.1). Based on the probe results, we propose an IPP-oriented post-training framework with two stages: Graft-SFT (Section 4.2.1) and Graft-DPO (Section 4.2.2). Figure 3: Comprehensive analysis of Quality and Anchor mass across different attention biases (α). 4.1 Attention Bias Probe To isolate the root cause of IPP failures in vanilla dLLMs, we hypothesize that these models already possess the requisite knowledge, but suffer from insufficient attention priority. To verify this, given a set of token indices IPPS_IPP corresponding to the IPP constraint span within the input sequence, we apply an explicit additive bias to the self-attention logits at each denoising step. For a query vector QiQ_i and a key vector KjK_j, the modified attention weight calculation is formulated as follows: Aij=Softmax(QiKjT/d+Mij+α⋅j∈IPP),A_ij=Softmax(Q_iK_j^T/ d+M_ij+α· 1_j _IPP), (5) where MijM_ij denotes the standard attention mask, and α serves as a scalar hyperparameter obtained via grid search on a small-scale validation set. This intervention is implemented by registering forward pre-hooks within the dLLMs, ensuring a zero-parameter update during the inference phase. To precisely quantify the model’s focus on the provided constraints during this process, we define Anchor Mass as the average attention probability mass empirically allocated to the constraint tokens while generating new text. Figure 4: Overview of the proposed in-place oriented post-training framework Graft. As shown in Figure 3, increasing α consistently raises the Anchor Mass for both LLaDA and Dream, confirming that the probe effectively shifts attention toward the IPP span. However, IIF performance follows a non-monotonic trend: moderate bias improves constraint adherence and overall quality, whereas excessive bias overemphasizes the anchor and degrades fluency and contextual fit. These results suggest that vanilla dLLMs are not inherently unable to follow in-place constraints; rather, they often fail to assign the constraint span an appropriate level of priority during denoising. This motivates our post-training framework, which aims to internalize such constraint-aware behavior into the model parameters. 4.2 In-place Oriented Post-Training To internalize the inference-time attention bias discovered in our preliminary analysis into the model parameters, we propose Graft, an In-place Oriented Post-Training framework. This framework consists of two progressive stages: Graft-SFT and Graft-DPO. The key design of this post-training framework is that its training objective is optimized exclusively for the IPP paradigm: namely, regarding the user-inserted anchor as a fixed contextual constraint, instead of a target to be generated. 4.2.1 Graft-SFT To equip the base dLLM with the fundamental ability to execute in-place instructions and maintain boundary smoothness, we first perform Graft-SFT. Unlike standard SFT that calculates the cross-entropy loss over the entire sequence, our objective avoids predicting the constraint itself. Given the target token sequence y=[yprompt,yfront,yanchor,yback],y=[y_prompt,y_front,y_anchor,y_back], the anchor yanchory_anchor acts as a fixed conditioning variable. Calculating the loss over the anchor tokens would misguide the model into memorizing the specific constraint rather than learning the bidirectional contextual transition. To address this, we apply a localized masking strategy to the loss function. Let genS_gen denote the set of token indices corresponding exclusively to the generated spans yfronty_front and ybacky_back. The Graft-SFT loss is formulated as: ℒGraft-SFT _Graft-SFT =s,W(s)[−∑i∈genℓθ(i)], =E_s,W^(s) [- _i _gen _θ^(i) ], (6) ℓθ(i) _θ^(i) =logpθ(yi∣W(s),yprompt,yanchor), = p_θ\! (y_i W^(s),y_prompt,y_anchor ), where W(s)W^(s) represents the intermediate masked state at diffusion step s. By zeroing out the loss gradients on the anchor positions where i∉geni _gen, the model is forced to dedicate its parameter updates exclusively to harmonizing the surrounding context with the given local constraints. 4.2.2 Graft-DPO While Graft-SFT establishes basic instruction-following behavior, navigating the delicate trade-off between local constraint execution and global semantic consistency remains challenging, especially under extreme stylistic or discourse conflicts such as L2 and L3 constraints in IIF-Bench. To further align the model’s behavior, we introduce Graft-DPO to further align the model toward responses that better balance local anchor adherence and global consistency. We construct preference pairs consisting of a chosen response ywy_w and a rejected response yly_l, where ywy_w better balances the local anchor adherence and global consistency. Similarly to the Graft-SFT phase, the preference optimization is strictly evaluated on the generated spans genS_gen, treating yanchory_anchor as a grounded condition. The modified objective is defined as: ℒGraft-DPO =−x,yw,yl[logσ(Δw−Δl)],L_Graft-DPO =-E_x,y_w,y_l [ σ ( _w- _l ) ], (7) where Δw _w =βlogπθ(yw∣x,yanchor)πref(yw∣x,yanchor), =β _θ(y_w x,y_anchor) _ref(y_w x,y_anchor), (8) Δl _l =βlogπθ(yl∣x,yanchor)πref(yl∣x,yanchor). =β _θ(y_l x,y_anchor) _ref(y_l x,y_anchor). Here x denotes the combined input context including yprompty_prompt. 5 Experiments 5.1 Setup We conduct Graft training and evaluation on IIF-Bench with four diffusion LLM backbones: LLaDA-1.5, LLaDA-8B, Dream-7B, and MMaDA. Baselines include their vanilla diffusion counterparts and three prompt-based AR instruction-tuned models: GPT-4o, Llama-3.1-8B-Instruct, and Qwen3-8B-Instruct. Since AR models cannot natively denoise masked spans around a fixed in-place anchor, we convert each IIF instance into a natural-language prompt specifying the anchor content, constraint type, and intended position, and ask the model to generate a complete response satisfying this positional constraint. Thus, these AR baselines test whether left-to-right instruction following can approximate IIF without native IPP-style denoising. We further compare with our aligned variants after Graft-SFT and the full SFT-to-DPO pipeline. Following prior LLM-as-a-judge protocols, we use GPT-5 and JudgeLM (Zhu et al., 2025b) as evaluators and report scores averaged over three runs; detailed settings are provided in Appendix B. 5.2 Overall Performance As shown in Table 1, Graft-aligned dLLMs consistently outperform their vanilla counterparts across all constraint levels, yielding an average absolute improvement of over 15 points in overall score. Although GPT-4o slightly leads at the highest constraint complexity (L3L_3), our models heavily dominate at the L1L_1 and L2L_2 levels. Ablation results confirm that Graft-SFT establishes the foundational capability, while full Graft alignment yields a universal boost. Furthermore, Fig. 5 shows that Graft-aligned models maintain robust general capabilities, indicating that Graft enhances in-place constraint following at negligible cost. Model & Stage GPT-5 Judge JudgeLM L1 L2 L3 Overall L1 L2 L3 Overall AR GPT-4o 63.74 67.18 70.42 67.11 63.39 64.98 65.57 64.65 Llama-3.1-8B-Instruct 58.74 62.36 65.18 62.09 57.79 59.74 61.58 59.70 Qwen3-8B-Instruct 61.42 65.83 67.56 64.94 60.82 62.98 63.49 62.43 Aligned dLLMs (Ours) LLaDA-1.5 ↪ Vanilla 67.42 64.51 56.59 62.84 66.32 61.81 53.49 60.54 ↪ + only Graft-SFT 75.86↑8.44 \!\!8.44 71.42↑6.91 \!\!6.91 64.70↑8.11 \!\!8.11 70.66↑7.82 \!\!7.82 75.41↑9.09 \!\!9.09 69.50↑7.69 \!\!7.69 59.60↑6.11 \!\!6.11 68.17↑7.63 \!\!7.63 ↪ + Graft 78.93↑11.51 \!\!11.51 75.64↑11.13 \!\!11.13 68.09↑11.50 \!\!11.50 74.22↑11.38 \!\!11.38 78.28↑11.96 \!\!11.96 73.32↑11.51 \!\!11.51 63.89↑10.40 \!\!10.40 71.83↑11.29 \!\!11.29 LLaDA-8B ↪ Vanilla 62.18 58.91 51.38 57.49 61.38 56.40 47.52 55.10 ↪ + only Graft-SFT 76.14↑13.96 \!\!13.96 70.42↑11.51 \!\!11.51 65.12↑13.74 \!\!13.74 70.56↑13.07 \!\!13.07 75.84↑14.46 \!\!14.46 68.37↑11.97 \!\!11.97 59.92↑12.40 \!\!12.40 68.04↑12.94 \!\!12.94 ↪ + Graft 80.17↑17.99 \!\!17.99 75.24↑16.33 \!\!16.33 69.53↑18.15 \!\!18.15 74.98↑17.49 \!\!17.49 79.97↑18.59 \!\!18.59 73.39↑16.99 \!\!16.99 64.08↑16.56 \!\!16.56 72.48↑17.38 \!\!17.38 Dream-7B ↪ Vanilla 59.83 56.94 48.65 55.14 58.98 54.54 44.65 52.72 ↪ + only Graft-SFT 73.84↑14.01 \!\!14.01 68.51↑11.57 \!\!11.57 62.76↑14.11 \!\!14.11 68.37↑13.23 \!\!13.23 73.29↑14.31 \!\!14.31 66.06↑11.52 \!\!11.52 58.51↑13.86 \!\!13.86 65.95↑13.23 \!\!13.23 ↪ + Graft 77.84↑18.01 \!\!18.01 72.41↑15.47 \!\!15.47 66.59↑17.94 \!\!17.94 72.28↑17.14 \!\!17.14 77.44↑18.46 \!\!18.46 70.31↑15.77 \!\!15.77 61.94↑17.29 \!\!17.29 69.90↑17.18 \!\!17.18 MMaDA ↪ Vanilla 60.27 56.12 50.21 55.53 59.22 53.62 46.71 53.18 ↪ + only Graft-SFT 72.18↑11.91 \!\!11.91 66.92↑10.80 \!\!10.80 61.12↑10.91 \!\!10.91 66.74↑11.21 \!\!11.21 71.48↑12.26 \!\!12.26 64.62↑11.00 \!\!11.00 56.82↑10.11 \!\!10.11 64.31↑11.13 \!\!11.13 ↪ + Graft 76.42↑16.15 \!\!16.15 71.43↑15.31 \!\!15.31 64.91↑14.70 \!\!14.70 70.92↑15.39 \!\!15.39 75.87↑16.65 \!\!16.65 69.11↑15.49 \!\!15.49 60.61↑13.90 \!\!13.90 68.53↑15.35 \!\!15.35 Table 1: Comprehensive evaluation on IIF-Bench. We compare our aligned diffusion LLMs against vanilla diffusion baselines and advanced AR models. Bold indicates the best performance for each metric, and underline indicates the second-best performance. Green subscripts denote absolute gains over the corresponding vanilla model. Metrics are averaged over 3 independent runs. Figure 5: Performance comparison of LLaDA (left) and Dream (right) across multiple general benchmarks. 5.3 Attention Diagnosis To understand how Graft affects dLLM denoising, we visualize layer-step difference maps of the anchor-probe score in Fig. 6, where Δ=Graft −Vanilla =Graft -Vanilla. The x-axis denotes six uniformly sampled probe checkpoints along the denoising trajectory. Positive values indicate stronger alignment with the in-place anchor after Graft, while negative values indicate weaker anchor-related similarity or attention. For LLaDA, the largest positive change appears in upper layers, while middle layers show slight decreases. For Dream, the positive change is smoother and extends from upper layers to middle layers. These results suggest that Graft does not overwrite the whole pretrained denoising process. Instead, it selectively adjusts task-relevant upper-layer representations to better use the in-place constraint. 5.4 Ablation Study As shown in Tables 2 and 3, we conduct ablation studies on frequency penalty and structural tag suppression. The frequency penalty lowers the logits of already-generated tokens to reduce repetition, while structural tag suppression blocks tag-related tokens from being generated. A moderate frequency penalty consistently yields optimal performance across both vanilla and Graft-aligned models. Notably, Graft-aligned models exhibit a markedly flatter trajectory in the low-penalty range (fp∈[0.0,0.3]fp∈[0.0,0.3]), indicating that anti-repetition behavior has been internalized through post-training and no longer relies heavily on test-time logit intervention. The comparable degradation observed in both model variants under excessive penalty (fp≥0.7fp≥ 0.7) reflects task-structural damage, where necessary literal phrases and discourse connectors are over-suppressed regardless of alignment stage. For structural tag suppression, test-time blocking improves vanilla baselines by preventing metadata leaks. After Graft alignment, this gain narrows and converges across architectures, indicating that post-training establishes native formatting boundaries and reduces reliance on test-time logit interventions. Figure 6: Layer-step difference maps of the anchor-probe score for LLaDA (left) and Dream (right). Model & Stage fp=0.0 fp=0.1 fp=0.3 fp=0.7 fp=1.0 LLaDA-8B (Vanilla) 57.49 57.88 58.32 57.65 55.42 LLaDA-8B (+Graft) 74.98 75.12 75.23 74.15 72.31 Dream-7B (Vanilla) 55.14 55.42 55.68 53.81 51.24 Dream-7B (+Graft) 72.28 72.39 72.46 70.82 68.54 Table 2: Ablation of frequency penalty fpfp for vanilla and aligned Graft models. Model & Stage Tag L1 L2 L3 Overall LLaDA-8B (Vanilla) Off 62.18 58.91 51.38 57.49 On 62.35 60.41 52.14 58.30 LLaDA-8B (+Graft) Off 80.17 75.24 69.53 74.98 On 80.24 75.68 69.71 75.21 Dream-7B (Vanilla) Off 59.83 56.94 48.65 55.14 On 59.94 58.15 49.10 55.73 Dream-7B (+Graft) Off 77.84 72.41 66.59 72.28 On 77.92 72.84 66.82 72.53 Table 3: Ablation of the anchor tag word suppression across different levels with GPT-5 Score. 6 Related Work Diffusion Language Models. Diffusion language models adapt denoising diffusion to text generation. Early approaches mainly explore continuous denoising in embedding space or task-specific sequence generation (Li et al., 2022; Gong et al., 2023; He et al., 2023), while discrete diffusion models introduce token-level corruption processes such as absorbing masks (Austin et al., 2021; Lou et al., 2023). Recent masked diffusion language models have substantially improved the scalability and generation quality of this paradigm. LLaDA (Nie et al., 2025) trains a large-scale masked diffusion language model from scratch and demonstrates competitive instruction-following and in-context learning ability; Dream (Ye et al., 2025) adapts autoregressive LLMs into diffusion models and improves the quality–efficiency trade-off; LLaDA-1.5 (Zhu et al., 2025a) investigates post-training and preference optimization for diffusion LMs; and LLaDA-2.X series (Bie et al., 2025; Bie et al., 2026) further scale up masked diffusion models to 100B parameters. The paradigm has also been extended to multimodal generation through MMaDA (Yang et al., 2025) and LLaDA-V (You et al., 2025), and recent work studies inference acceleration and block-wise generation for more efficient dLLM deployment (Liu et al., 2025; Arriola et al., 2025; Cheng et al., 2025; Wu et al., 2025). Compared with autoregressive LMs, these models perform iterative masked-token refinement with bidirectional context, which makes them naturally suitable for infilling and arbitrary-position control. This property provides the modeling basis for our study of in-place instruction following. In-place Prompting and Mid-sequence Control. Recent work explores dLLMs beyond prefix-only prompting. ICE inserts chain-of-thought prompts into masked positions with confidence-aware early exit (Jin et al., 2025), while Template Infilling uses structural anchors as global blueprints for masked segment generation (Lee et al., 2025). Reasoning-as-infilling fills structured reasoning and answer templates for posterior reasoning and early exits (Horvitz et al., 2025), and DIP dynamically inserts in-context examples during denoising (Li et al., 2026a). Related safety studies show that the same bidirectional and parallel mechanisms create diffusion-specific attack surfaces (Wen et al., 2025; Zhang et al., 2025; Li et al., 2026b). Unlike these works, we study in-place instruction following as a user-facing mechanism for localized constraint satisfaction under global coherence. 7 Conclusion This paper introduces and formalizes the In-Place Instruction Following task and constructs a benchmark spanning three difficulty levels. Through an attention-bias probe, we find evidence that insufficient prioritization of the constraint span is one source of failure in vanilla dLLMs during parallel denoising. To address this, we propose Graft, a post-training framework combining Graft-SFT and Graft-DPO. Additionally, a fine-grained evaluation reveals a persistent trade-off between local and global consistency in style constraint scenarios. Limitations While Graft improves in-place instruction following in dLLMs, this work has several limitations. First, IIF-Bench mainly focuses on text-only in-place instruction following, and its coverage of multilingual, multimodal, and interactive editing scenarios remains limited. Second, although the proposed evaluation protocol combines automatic, reward-model-based, and human judgments, rubric-based evaluation may still be affected by evaluator bias, especially for style and discourse-function constraints. Third, Graft is evaluated on four representative dLLMs, but its scalability to larger diffusion models and its robustness under more complex or conflicting anchors require further investigation. Future work may extend IIF to broader domains and develop more interpretable mechanisms for local–global constraint coordination. References Arriola et al. (2025) Marianne Arriola, Aaron Gokaslan, Justin T. Chiu, Zhihan Yang, Zhixuan Qi, Jiaqi Han, Subham Sekhar Sahoo, and Volodymyr Kuleshov. 2025. Block diffusion: Interpolating between autoregressive and diffusion language models. arXiv preprint arXiv:2503.09573. Austin et al. (2021) Jacob Austin, Daniel D. Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg. 2021. Structured denoising diffusion models in discrete state-spaces. In Advances in Neural Information Processing Systems. Banerjee et al. (2025) Debangshu Banerjee, Tarun Suresh, Shubham Ugare, Sasa Misailovic, and Gagandeep Singh. 2025. Crane: Reasoning with constrained llm generation. arXiv preprint arXiv:2502.09061. Bavarian et al. (2022) Mohammad Bavarian, Heewoo Jun, Nikolas Tezak, John Schulman, Christine McLeavey, Jerry Tworek, and Mark Chen. 2022. Efficient training of language models to fill in the middle. arXiv preprint arXiv:2207.14255. Beurer-Kellner et al. (2024) Luca Beurer-Kellner, Marc Fischer, and Martin Vechev. 2024. Guiding llms the right way: Fast, non-invasive constrained generation. arXiv preprint arXiv:2403.06988. Bie et al. (2026) Tiwei Bie, Maosong Cao, Xiang Cao, Bingsen Chen, Fuyuan Chen, Kun Chen, Lun Du, Daozhuo Feng, Haibo Feng, Mingliang Gong, and 1 others. 2026. Llada2. 1: Speeding up text diffusion via token editing. arXiv preprint arXiv:2602.08676. Bie et al. (2025) Tiwei Bie, Maosong Cao, Kun Chen, Lun Du, Mingliang Gong, Zhuochen Gong, Yanmei Gu, Jiaqi Hu, Zenan Huang, Zhenzhong Lan, Chengxi Li, Chongxuan Li, Jianguo Li, Zehuan Li, Huabin Liu, Ling Liu, Guoshan Lu, Xiaocheng Lu, Yuxin Ma, and 12 others. 2025. Llada2.0: Scaling up diffusion language models to 100b. arXiv preprint arXiv:2512.15745. Brown et al. (2020) Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, and 12 others. 2020. Language models are few-shot learners. Preprint, arXiv:2005.14165. Chen (2024) Jianlin Chen. 2024. Lmstyle benchmark: Evaluating text style transfer for chatbots. arXiv preprint arXiv:2403.08943. Cheng et al. (2025) Shuang Cheng, Yihan Bian, Dawei Liu, Yuhua Jiang, Yihao Liu, Linfeng Zhang, Wenhai Wang, Qipeng Guo, Kai Chen, Biqing Qi, and Bowen Zhou. 2025. Sdar: A synergistic diffusion-autoregression paradigm for scalable sequence generation. arXiv preprint arXiv:2510.06303. Ge et al. (2024) Suyu Ge, Chunting Zhou, Rui Hou, Madian Khabsa, Yi-Chia Wang, Qifan Wang, Jiawei Han, and Yuning Mao. 2024. Mart: Improving llm safety with multi-round automatic red-teaming. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 1927–1937. Gong et al. (2023) Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, and Lingpeng Kong. 2023. Diffuseq: Sequence to sequence text generation with diffusion models. In International Conference on Learning Representations. He et al. (2023) Zhengfu He, Tianxiang Sun, Kuanning Wang, Xuanjing Huang, and Xipeng Qiu. 2023. Diffusionbert: Improving generative masked language models with diffusion models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics. Horvitz et al. (2025) Zachary Horvitz, Raghav Singhal, Hao Zou, Carles Domingo-Enrich, Zhou Yu, Rajesh Ranganath, and Kathleen McKeown. 2025. No compute left behind: Rethinking reasoning and sampling with masked diffusion models. arXiv preprint arXiv:2510.19990. Iso (2024) Hayate Iso. 2024. Autotemplate: A simple recipe for lexically constrained text generation. In Proceedings of the 17th International Natural Language Generation Conference, pages 1–12. Jiang et al. (2024) Yuxin Jiang, Yufei Wang, Xingshan Zeng, Wanjun Zhong, Liangyou Li, Fei Mi, Lifeng Shang, Xin Jiang, Qun Liu, and Wei Wang. 2024. Followbench: A multi-level fine-grained constraints following benchmark for large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4667–4688. Jin et al. (2025) Xiangqi Jin, Yuxuan Wang, Yifeng Gao, Zichen Wen, Biqing Qi, Dongrui Liu, and Linfeng Zhang. 2025. Thinking inside the mask: In-place prompting in diffusion llms. arXiv preprint arXiv:2508.10736. Lai et al. (2024) Wen Lai, Viktor Hangya, and Alexander Fraser. 2024. Style-specific neurons for steering llms in text style transfer. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 13427–13443. Lee et al. (2025) Junhoo Lee, Seungyeon Kim, and Nojun Kwak. 2025. Unlocking the potential of diffusion language models through template infilling. arXiv preprint arXiv:2510.13870. Li et al. (2024) Junlong Li, Fan Zhou, Shichao Sun, Yikai Zhang, Hai Zhao, and Pengfei Liu. 2024. Dissecting human and llm preferences. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1790–1811. Li et al. (2022) Xiang Lisa Li, John Thickstun, Ishaan Gulrajani, Percy Liang, and Tatsunori B. Hashimoto. 2022. Diffusion-lm improves controllable text generation. In Advances in Neural Information Processing Systems. Li et al. (2026a) Yang Li, Han Meng, Chenan Wang, and Haipeng Chen. 2026a. Dip: Dynamic in-context planner for diffusion language models. arXiv preprint arXiv:2601.03199. Li et al. (2026b) Zherui Li, Zheng Nie, Zhenhong Zhou, Yue Liu, Yitong Zhang, Yu Cheng, Qingsong Wen, Kun Wang, Yufei Guo, and Jiaheng Zhang. 2026b. Diffuguard: How intrinsic safety is lost and found in diffusion large language models. In International Conference on Learning Representations. Liu et al. (2024) Michael Xieyang Liu, Frederick Liu, Alexander J Fiannaca, Terry Koo, Lucas Dixon, Michael Terry, and Carrie J Cai. 2024. " we need structured output": Towards user-centered constraints on large language model output. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, pages 1–9. Liu et al. (2025) Zhiyuan Liu, Yicun Yang, Yaojie Zhang, Junjie Chen, Chang Zou, Qingyuan Wei, Shaobo Wang, and Linfeng Zhang. 2025. dllm-cache: Accelerating diffusion large language models with adaptive caching. arXiv preprint arXiv:2506.06295. Lou et al. (2023) Aaron Lou, Chenlin Meng, and Stefano Ermon. 2023. Discrete diffusion modeling by estimating the ratios of the data distribution. arXiv preprint arXiv:2310.16834. Lou et al. (2024) Renze Lou, Kai Zhang, and Wenpeng Yin. 2024. Large language model instruction following: A survey of progresses and challenges. Computational Linguistics, 50(3):1053–1095. Mai et al. (2023) Huiyu Mai, Wenhao Jiang, and Zhi-Hong Deng. 2023. Prefix-tuning based unsupervised text style transfer. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 14847–14856. Nie et al. (2025) Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. 2025. Large language diffusion models. arXiv preprint arXiv:2502.09992. Pyatkin et al. (2026) Valentina Pyatkin, Saumya Malik, Victoria Graf, Hamish Ivison, Shengyi Huang, Pradeep Dasigi, Nathan Lambert, and Hanna Hajishirzi. 2026. Generalizing verifiable instruction following. Advances in Neural Information Processing Systems, 38. Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, and 1 others. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9. Ren et al. (2024) Houxing Ren, Mingjie Zhan, Zhongyuan Wu, and Hongsheng Li. 2024. Empowering character-level text infilling by eliminating sub-tokens. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3253–3267. Roziere et al. (2023) Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, and 1 others. 2023. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950. Son et al. (2024) Guijin Son, SangWon Baek, Sangdae Nam, Ilgyun Jeong, and Seungone Kim. 2024. Multi-task inference: Can large language models follow multiple instructions at once? In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5606–5627. Tao et al. (2024) Zhen Tao, Dinghao Xi, Zhiyu Li, Liumin Tang, and Wei Xu. 2024. Cat-llm: Style-enhanced large language models with text style definition for chinese article-style transfer. arXiv preprint arXiv:2401.05707. Taori et al. (2023) Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/stanford_alpaca. Tian et al. (2025) Xiaoyu Tian, Sitong Zhao, Haotian Wang, Shuaiting Chen, Yunjie Ji, Yiping Peng, Han Zhao, and Xiangang Li. 2025. Think twice: Enhancing llm reasoning by scaling multi-round test-time thinking. arXiv preprint arXiv:2503.19855. Wen et al. (2025) Zichen Wen, Jiashu Qu, Zhaorun Chen, Xiaoya Lu, Dongrui Liu, Zhiyuan Liu, Ruixi Wu, Yicun Yang, Xiangqi Jin, Haoyun Xu, and 1 others. 2025. The devil behind the mask: An emergent safety vulnerability of diffusion llms. arXiv preprint arXiv:2507.11097. Wu et al. (2025) Chengyue Wu, Hao Zhang, Shuchen Xue, Shizhe Diao, Yonggan Fu, Zhijian Liu, Pavlo Molchanov, Ping Luo, Song Han, and Enze Xie. 2025. Fast-dllm v2: Efficient block-diffusion llm. arXiv preprint arXiv:2509.26328. Yang et al. (2025) Ling Yang, Ye Tian, Bowen Li, Xinchen Zhang, Ke Shen, Yunhai Tong, and Mengdi Wang. 2025. Mmada: Multimodal large diffusion language models. arXiv preprint arXiv:2505.15809. Ye et al. (2025) Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong. 2025. Dream 7b: Diffusion large language models. arXiv preprint arXiv:2508.15487. You et al. (2025) Zebin You, Shen Nie, Xiaolu Zhang, Jun Hu, Jun Zhou, Zhiwu Lu, Ji-Rong Wen, and Chongxuan Li. 2025. Llada-v: Large language diffusion models with visual instruction tuning. arXiv preprint arXiv:2505.16933. Zhang et al. (2026) Yitong Zhang, Yongmin Li, Yuetong Liu, Jia Li, Xiaoran Jia, Zherui Li, and Ge Li. 2026. Lookahead-then-verify: Reliable constrained decoding for diffusion llms under context-free grammars. arXiv preprint arXiv:2602.00612. Zhang et al. (2025) Yuanhe Zhang, Fangzhou Xie, Zhenhong Zhou, Zherui Li, Hao Chen, Kun Wang, and Yufei Guo. 2025. Jailbreaking large language diffusion models: Revealing hidden safety flaws in diffusion-based text generation. arXiv preprint arXiv:2507.19227. Zhao et al. (2025a) Siyan Zhao, Mingyi Hong, Yang Liu, Devamanyu Hazarika, and Kaixiang Lin. 2025a. Do llms recognize your preferences? evaluating personalized preference following in llms. arXiv preprint arXiv:2502.09597. Zhao et al. (2025b) Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, and 3 others. 2025b. A survey of large language models. Preprint, arXiv:2303.18223. Zhou et al. (2023a) Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. 2023a. Instruction-following evaluation for large language models. arXiv preprint arXiv:2311.07911. Zhou et al. (2023b) Wangchunshu Zhou, Yuchen Eleanor Jiang, Ethan Wilcox, Ryan Cotterell, and Mrinmaya Sachan. 2023b. Controlled text generation with natural language instructions. In International Conference on Machine Learning, pages 42602–42613. PMLR. Zhu et al. (2025a) Fengqi Zhu, Rongzhen Wang, Shen Nie, Xiaolu Zhang, Chunwei Wu, Jun Hu, Jun Zhou, Jianfei Chen, Yankai Lin, Ji-Rong Wen, and 1 others. 2025a. Llada 1.5: Variance-reduced preference optimization for large language diffusion models. arXiv preprint arXiv:2505.19223. Zhu et al. (2025b) Lianghui Zhu, Xinggang Wang, and Xinlong Wang. 2025b. Judgelm: Fine-tuned large language models are scalable judges. In International Conference on Learning Representations, volume 2025, pages 51257–51296. Appendix A LLM Usage Statement We utilized Large Language Models to refine and polish our original manuscript. Specifically, its use was focused on improving grammar, clarity, conciseness, and word choice. It is important to note that the model was employed solely as a writing aid and did not contribute to the generation of any new content or ideas. Appendix B Experimental Details Implementation details. All training experiments are conducted on NVIDIA A800 GPUs. We apply the same Graft pipeline to each diffusion backbone, including LLaDA-1.5, LLaDA-8B, Dream-7B, and MMaDA. During training, the inserted constraint is fixed in the output canvas, and the model is optimized to generate the surrounding masked spans. At inference time, all diffusion baselines and Graft-aligned variants follow the same IPP decoding format for fair comparison. Benchmark construction. We construct the IIF benchmark by converting existing instruction–response pairs into partially constrained continuation tasks. Each example consists of a user instruction and an assistant-side sequence in which a constraint tag is inserted between a prefix (front text) and a target continuation (back text). The benchmark contains three levels of increasing constraint abstraction. Level 1 evaluates literal adherence: the continuation must preserve either an exact sentence or a set of verbatim keyword phrases sampled from the original answer. Level 2 evaluates semantic style control: poetry-style responses are split into a prefix and a continuation, and the continuation is rewritten under one of twelve target style constraints while preserving the core content. Level 3 evaluates discourse-level control: long-form QA responses are split into a prefix and continuation, and the continuation is rewritten to perform one of eight specified discourse functions, such as giving a counterexample, forming an analogy, or posing rhetorical questions. We use Alpaca and LFQA for literal constraints, Poetry Instructions for style constraints, and LFQA for discourse-function constraints. Tables 6 and 7 summarize the constraint inventories used for Level 2 and Level 3, respectively. Table 4 reports the number of examples and constraint-length statistics for each level of the IIF benchmark. Level #Examples Avg. constraint length Min. Max. Level 1 Literal 1573 129.5 54 395 Level 2 Semantic 1628 114.5 107 134 Level 3 Discourse 1836 96.2 82 106 Total 5037 112.5 54 395 Table 4: Statistics of constraint lengths across the three levels of the IIF benchmark. Training and preference data construction. We use the constructed IIF instances as the data source for both Graft-SFT and Graft-DPO. For each instance, the constraint-satisfying rewritten response produced by our construction pipeline is used as the target response for Graft-SFT. For Graft-DPO, we form preference pairs by treating the constraint-satisfying rewritten response as the chosen response and its corresponding unconstrained or pre-rewrite response as the rejected response. This construction is applied consistently across L1 literal, L2 style, and L3 discourse-function constraints. Training and evaluation are conducted on disjoint splits of the constructed IIF instances. Baselines. We compare Graft with both diffusion-based and autoregressive language model baselines. For diffusion LLMs, all vanilla models are evaluated under the same IPP format as Graft, where the inserted constraint is fixed in the output canvas and the model generates the surrounding masked spans. For autoregressive models, which generate text from left to right, we prompt them to produce a complete response satisfying the same in-place constraint specification. • LLaDA-1.5 and LLaDA-8B. LLaDA represents the masked-diffusion generation paradigm, where masked response tokens are iteratively recovered through parallel denoising. We include LLaDA-8B and LLaDA-1.5 as representative models from this diffusion LLM family, evaluated without Graft-specific adaptation. • Dream-7B. Dream-7B is another discrete diffusion LLM that supports iterative parallel refinement and infilling-style generation. It allows us to evaluate Graft on a diffusion backbone developed independently of LLaDA. • MMaDA. MMaDA is a unified multimodal diffusion model that formulates text generation through masked-token prediction under a shared diffusion architecture. Its inclusion broadens the evaluation to a diffusion backbone with a different architectural origin from both LLaDA and Dream. • GPT-4o. GPT-4o is included as a strong closed-source autoregressive baseline. Since it does not natively perform masked-span denoising around a fixed output constraint, its performance mainly reflects whether a high-capability autoregressive model can satisfy the in-place constraint through prompt understanding. • Llama-3.1-8B-Instruct. Llama-3.1-8B-Instruct is used as an open-weight instruction-tuned autoregressive baseline at a scale comparable to the 7B–8B diffusion backbones. This comparison helps distinguish the benefit of diffusion-style in-place generation from standard autoregressive instruction following. • Qwen3-8B-Instruct. Qwen3-8B-Instruct is included as another open-weight 8B-scale autoregressive instruction-tuned baseline. Together with Llama-3.1-8B-Instruct, it provides a diverse open-model comparison for evaluating constraint-following performance under the same benchmark. Inference hyperparameters. For diffusion LLMs, the generation length is instance-specific. Given each benchmark example, we construct the output canvas as [prompt | MASK ×Lf× L_f | anchor | MASK ×Lb× L_b], where the anchor span contains the fixed inserted constraint. The effective generation length is therefore Lf+LbL_f+L_b, while the full sequence length is capped at 1024 tokens. If the constructed sequence exceeds this cap, LfL_f and LbL_b are proportionally reduced. The inference hyperparameters are summarized in Table 5. Model Steps Gen. Length Block Length Temp. CFG Scale Top-p Alg. Max Length LLaDA-1.5 128 Lf+LbL_f+L_b 32 0.5 0.0 – – 1024 LLaDA-8B 128 Lf+LbL_f+L_b 32 0.5 0.0 – – 1024 MMaDA 128 Lf+LbL_f+L_b 32 0.5 0.0 – – 1024 Dream-7B 128 Lf+LbL_f+L_b – 0.7 – 0.95 maskgit-plus 1024 Table 5: Inference hyperparameters for diffusion LLMs. The generation length is instance-specific and equals the number of masked tokens surrounding the fixed constraint span. ID Target Style Description 1 haunting Gothic, heavy with dread, ruin, and shadowed decay 2 solemn and elegiac, like a funeral lament for something lost 3 wistful and nostalgic, like a faded letter reread years later 4 stoic and pastoral, the quiet voice of a farmer watching seasons turn 5 mystical and reverent, in the cadence of sacred liturgy or prayer 6 austere and minimalist, in the spirit of a Zen haiku—few words, vast silence 7 tender and amorous, in the warm voice of a Neruda love sonnet 8 serene and epiphanic, a quiet moment of sudden inner light 9 jubilant and celebratory, in the expansive voice of a Whitman ode 10 joyful pastoral, bright with sunlight, birdsong, and dancing streams 11 whimsical and playful, in the nonsense-verse tradition of Lear or Carroll 12 surreal and dreamlike, where images dissolve into one another without logical bridges Table 6: Level-2 semantic style constraint inventory. Each L2 instance samples one target style from this inventory and requires the continuation span to instantiate the assigned style while preserving the core content. ID Function Core Requirement 1 Counterexample Provide a concrete case that contradicts, challenges, or complicates the implication of the preceding context. 2 Hypothesis Introduce a speculative “Suppose / Imagine / What if” scenario that extends or stress-tests the preceding claim. 3 Summary Compress the preceding context into a broader takeaway or general principle without introducing new arguments. 4 Transition Briefly acknowledge the preceding point and then pivot to a contrasting or complementary perspective. 5 Analogy Explain the preceding point through a concrete cross-domain comparison. 6 Rebuttal Identify and contest an implicit assumption in the preceding context, then offer a counterclaim with support. 7 Parallelism Continue the discourse using three or more structurally parallel clauses or sentences. 8 Rhetorical Question End with one or more unanswered questions that deepen or reinforce the preceding point. Table 7: Level-3 discourse function inventory. Each L3 instance samples one discourse function and requires the continuation span to perform the specified rhetorical or logical role with respect to the preceding context. Appendix C Extended Experimental Analysis Ablation on Anchor Positions As shown in Table 8, we analyze the spatial generalization of Graft by partitioning single anchor placement into Early (10%–30%), Middle (40%–60%), and Late (70%–90%) segments. Both LLaDA-8B and Dream-7B exhibit a clear middle anchor advantage, where the overall scores peak at the centrally located configuration. This stems from the bidirectional mechanism of In-place Prompting, as a centralized anchor provides a balanced semantic bridge to restrict both flanking spans. Conversely, extreme positioning leaves one side disproportionately long, causing the task to partially degenerate into unconstrained sequential generation. Interestingly, the two architectures diverge at the Late position for stylized constraints (L2L_2). LLaDA shows a performance upturn because its block-wise semi-autoregressive decoding allows longer preceding context blocks to undergo sufficient refinement iterations for style manifestation. In contrast, Dream experiences degradation because its continuous diffusion hook mechanism suffers from attenuated control signals over an expanded attention radius, hindering global stylistic consistency. Model & Stage Position L1 L2 L3 Overall LLaDA-8B (+Graft) Early 79.62 74.21 68.95 74.26 Middle 80.54 75.12 70.43 75.36 Late 79.88 75.81 68.74 74.81 Dream-7B (+Graft) Early 77.12 71.15 66.18 71.48 Middle 78.11 73.24 66.92 72.76 Late 77.93 72.08 65.74 71.92 Table 8: Ablation of single anchor positions. Scaling of Multiple Anchors Table 9 illustrates the impact of scaling the number of uniformly distributed anchors from one to three. Both models follow a sub-linear saturation degradation trajectory, where the transition from one to two anchors induces a severe performance fracture, especially in discourse-function constraints (L3L_3). This sharp decline highlights a critical generalization boundary for Graft post-training. Since the training objective focuses on single-anchor instances, the internalized attention priorities are calibrated for a single focal point. When multiple anchors are introduced, the limited attention resources become fragmented across competing constraints. The model consequently struggles to maintain local execution for each individual anchor while simultaneously ensuring global coherence across fragmented generation segments. Literal constraints (L1L_1) remain the most resilient due to the lack of combinatorial logic or style collisions, suggesting that while Graft effectively internalizes single-point attention bias, multi-focal coordination remains a structural hurdle. Model & Stage # Anchors L1 L2 L3 Overall LLaDA-8B (+Graft) 1 80.23 75.11 69.64 75.03 2 77.86 70.58 63.91 70.78 3 76.04 66.73 58.42 67.06 Dream-7B (+Graft) 1 77.69 72.36 66.71 72.25 2 74.92 67.18 60.46 67.52 3 72.81 63.27 55.04 63.71 Table 9: Ablation of anchor quantity scaling with uniform distribution. Dim Description Max pts C1 Register distance (formality gap between global and local style) 35 C2 Emotional valence conflict (opposing affective directions) 35 C3 Structural incompatibility (local style conflicts with global genre form) 30 Total ConflictScore 100 Table 10: Dimensions used to compute the L2 ConflictScore. Table 11 breaks down the scores by dimension for the double-blind human evaluation subset introduced in Section 3. Scores are decomposed into the four rubric dimensions D1–D4 defined in Appendix E and complement the holistic scores reported in Table 1. Appendix D Human Evaluation Details To empirically validate the reliability of our automated evaluation pipeline, we conduct a double-blind human study on a randomly sampled subset of IIF-Bench. Table 11 provides the fine-grained, per-dimension breakdown across all three difficulty levels, extending the holistic analysis provided in the main text. Model Archetype Model & Stage L1 (40/20/20/20) L2 (40/25∗/20/15) L3 (40/30/20/10) Overall D1 D2 D3 D4 D1 D2∗ D3 D4 D1 D2 D3 D4 Diffusion LLMs Vanilla Baselines LLaDA-1.5 23.33 15.00 14.67 14.00 22.67 18.55 13.00 10.33 19.00 18.67 11.00 7.00 62.41 LLaDA-8B 20.67 14.00 13.33 13.00 20.67 17.43 11.33 9.33 16.67 17.00 10.00 6.67 56.70 Dream-7B 19.33 13.67 13.00 13.00 19.33 17.35 10.67 9.33 15.33 16.00 9.33 7.00 54.45 MMaDA 19.67 13.67 13.33 13.33 19.00 16.98 11.00 9.00 16.00 16.67 9.67 7.00 55.11 Autoregressive Baselines GPT-4o 22.67 14.33 13.67 13.33 27.33 18.87 11.67 10.67 27.67 19.67 13.67 8.33 67.29 Llama-3.1-8B-Instruct 21.33 13.00 12.67 11.00 25.00 18.31 10.67 9.67 25.33 18.33 12.67 8.00 61.99 Qwen3-8B-Instruct 22.33 13.67 12.67 12.33 26.33 19.00 11.33 10.33 26.33 19.00 13.00 8.33 64.88 Aligned Diffusion LLMs (Ours) LLaDA-1.5 ↪ + only Graft-SFT 29.33 16.00 15.33 14.67 28.67 20.10 13.67 11.00 23.33 20.00 13.33 7.00 70.81 ↪ + Graft 31.00 16.33 15.67 15.00 30.33 21.14 14.33 11.33 24.33 21.00 13.67 8.00 74.04 LLaDA-8B ↪ + only Graft-SFT 29.67 15.67 15.33 14.33 27.67 19.89 13.33 10.67 23.67 19.67 13.33 7.33 70.19 ↪ + Graft 31.67 16.33 16.00 16.00 30.33 21.07 14.00 11.00 25.67 21.33 13.67 8.00 75.02 Dream-7B ↪ + only Graft-SFT 28.00 15.67 15.00 14.33 26.67 19.79 13.00 10.67 22.33 19.33 12.67 7.33 68.26 ↪ + Graft 30.33 16.00 15.67 15.00 29.00 20.56 13.67 11.33 24.00 20.33 13.33 7.67 72.30 MMaDA ↪ + only Graft-SFT 27.00 15.33 14.67 14.00 25.67 19.24 12.67 10.33 21.33 18.67 12.67 7.33 66.30 ↪ + Graft 29.67 15.67 15.33 14.67 28.33 20.32 13.33 11.00 23.00 19.67 13.00 7.67 70.55 Table 11: Fine-grained per-dimension breakdown of the Human Evaluation subset. Appendix E Evaluation Rubrics The IIF evaluation rubric is organized around four structural dimensions: local constraint execution (D1), contextual integration (D2), boundary quality (D3), and global requirement (D4). These dimensions are applied consistently across all three benchmark levels. Rather than collapsing output quality into a single holistic score, this design separates the model’s ability to fulfill in-place constraints from its ability to maintain coherence at the local transition and global response levels. Each dimension is instantiated with level-specific sub-criteria that reflect the distinct focal points of L1 (literal fidelity), L2 (semantic realization), and L3 (discourse function execution). The dimension labels serve as a shared structural scaffold rather than shared definitions. Table 12 summarizes the per-sub-item point allocation across all levels. Total scores are computed by direct summation (S=D1+D2+D3+D4S=D1+D2+D3+D4, max 100); the decreasing weight of D4 across levels reflects the increasing complexity of local execution at L2 and L3. For L3-D1, the general point structure is further instantiated by discourse function in Table 13. The score descriptions in Table 13 provide approximate reference points to help annotators map real cases to the corresponding level of criterion fulfillment. Dim / Sub L1 Criterion Pts L2 Criterion Pts L3 Criterion Pts D1 a Verbatim anchor retention 25 Style onset sharpness 15 Function-specific sub-item a† 15 b Positional appropriateness 10 Style depth: beyond surface markers 15 Function-specific sub-item b† 15 c No paraphrastic substitution 5 Residual style interference 10 Function-specific sub-item c† 10 Total 40 40 40 D2 a Syntactic continuity (Pre) 8 Entity consistency (within span) 10 Argumentative consistency 12 b Semantic flow (Pre) 8 Thematic continuity 10 Logical placement 10 c No pre-constraint redundancy 4 No internal hallucination 5 Logical extensibility 8 Total 20 25 30 D3 a Syntactic continuation (Post) 8 Stylistic boundary shift 10 Clear function onset signal 10 b Semantic continuation (Post) 8 No meta-commentary (cap rule) 10 No function anticipation 10 c No immediate repetition 4 — — — — Total 20 20 20 D4 a Instruction relevance 8 Inter-section narrative unity 8 Grammatical correctness 5 b No truncation (output completeness) 6 Global task fulfillment 7 Clarity of expression 5 c No verbatim input copying 6 — — — — Total 20 15 10 Grand Total 100 100 100 Table 12: IIF evaluation rubric: per-sub-item point allocation. D1–D4 serve as structural labels for level-specific criteria. † L3-D1 criteria vary by function (e.g., challenge strength for Counterexample; testability for Hypothesis), see Table 13. Function D1a (15 pts) D1b (15 pts) D1c (10 pts) Counterexample A specific, tangible scenario or case is presented, rather than an abstract statement. The example genuinely complicates or contradicts the main argument in the pre-constraint text. The example satisfies any source-domain condition specified in the constraint. For instance, if “from everyday life” is specified, the example is drawn from common experience rather than a technical or academic domain. If no such condition is specified, the example should remain contextually appropriate and concrete. Hypothesis Conditional or speculative framing is present, such as “if”, “suppose”, “imagine”, or “were it the case”. The hypothesis makes a claim that could in principle be examined or argued for/against. The hypothesis is introduced as an open question to be examined, not as a settled conclusion. Summary The summary covers the key points made in the pre-constraint text without omitting critical arguments. The summary is noticeably more concise than the content it summarizes. The summary does not introduce new claims. The related L3-D2c criterion, logical extensibility, is evaluated using the same evidence, but its 8 points remain counted under D2 to preserve the 100-point total. Transition The content of the two ideas being bridged is genuinely connected, not merely juxtaposed. The transition explains why or how the second idea follows from the first, not just that it does. Score 15 if the logical link is explicit and substantive; 8 if implicit but traceable; 0 if the two ideas remain essentially unrelated despite surface connectives. Surface connective language is present, such as “however”, “building on this”, “turning now to”, or “this leads us to”. Score 15 if explicit connectives are present; 8 if implicit but detectable; 0 if absent. The transition is appropriately brief, not a paragraph-long digression. Score 10 if concise and functional; 5 if slightly extended; 0 if excessively long. Analogy The analogy draws from a domain clearly different from the main topic. The structural mapping between the analogy domain and the main argument is logically valid, such that the shared structure genuinely holds. The analogy is self-explanatory to a general reader without additional context. Score 10 if immediately clear; 5 if it requires minor inferential effort; 0 if opaque. Rebuttal An opposing viewpoint or objection is clearly identified. The refutation provides a reason or evidence against the opposing view, not merely an assertion. The opposing view is presented fairly and is not a strawman. Parallelism Two or more clauses or sentences share the same syntactic pattern. Score 15 for ≥3≥3 parallel structures; 10 for exactly 2; 0 for none. Each parallel element introduces distinct semantic content rather than synonymous repetition. The parallel structure creates a sense of emphasis, rhythm, or cumulative rhetorical force. Rhetorical Question At least one grammatically correct question is present in the constrained section. Score 15 if present; 0 if absent. Conditional on D1a >0>0: the question is not immediately answered and is posed for rhetorical effect. If D1a =0=0, D1b is automatically scored 0. Otherwise, score 15 if fully rhetorical; 8 if partially answered; 0 if directly and completely answered. The question advances or emphasizes the main argument; it is not tangential or unrelated to the discourse. Table 13: Function-specific instantiations of L3-D1. All L3 discourse functions share the same 15/15/10 point structure, but the concrete execution criteria are defined separately for each function. For L2 specifically, the forced insertion of a local style may introduce an inherent tension with the surrounding global context. A model output can therefore appear less contextually smooth not because it fails to follow the instruction, but because the assigned local style itself conflicts with the preceding context. To account for this local–global tension, we apply a conflict-aware adjustment to the L2-D2 score. For each L2 instance, annotators first assign a raw contextual-integration score Sraw,iS_raw,i for D2. They also independently estimate a ConflictScore, which measures the intrinsic difficulty of harmonizing the assigned local style with the surrounding context. The ConflictScore aggregates three sources of tension: register distance, emotional-valence conflict, and structural incompatibility. Table 10 summarizes the three components. Formally, ConflictScorei=C1,i+C2,i+C3,i,ConflictScore_i=C_1,i+C_2,i+C_3,i, with a maximum value of 100. We then compute a conflict-adjusted D2 score for each L2 instance: Sadj,i=Sraw,i+(25−Sraw,i)⋅ConflictScorei100⋅β,S_adj,i=S_raw,i+(25-S_raw,i)· ConflictScore_i100·β, where SD2-raw,iS_D2-raw,i is the raw D2 score of instance i and 25 is the maximum possible score for the L2-D2 dimension and β∈[0,1]β∈[0,1] controls the strength of the adjustment. The adjustment is compensatory rather than punitive: a larger ConflictScore indicates a stronger intrinsic mismatch between the assigned local style and the surrounding context, and therefore reduces the penalty imposed by a low raw D2 score. In other words, high-conflict L2 instances are not treated as equally easy contextual-integration cases as low-conflict ones. The L2-D2 value reported in our main results is the mean adjusted score S¯adj S_adj over all L2 instances. Consequently, unlike the raw rubric scores, the reported L2-D2 value is not restricted to the discrete annotation grid. The overall L2 score is computed as SL2=D1raw+S¯adj+D3raw+D4raw.S_L2=D1_raw+ S_adj+D3_raw+D4_raw. Appendix F Case Study Case: Style Constraint This case illustrates a style-control failure. The requested continuation should maintain a joyful pastoral style, bright with sunlight, birdsong, and dancing streams. The Graft answer follows this style more consistently from beginning to end, using images such as green fields, warm sunlight, birdsong, and streams. The Vanilla answer partially recovers in the second half, but its opening relies on twilight, shadows, and moonlight, which conflicts with the requested brightness and pastoral joy. Example Case Tag: style Instruction: Can you write me a poem about love and reed in the style of JOHN DONNE? Constraint: Write the following continuation in this style: joyful pastoral, bright with sunlight, birdsong, and dancing streams. Reference Continuation Gold Front: Sure, here’s a poem about love and reed in the style of JOHN DONNE: Come live with me, and be my love, And we will some new pleasures prove Of golden sands, and crystal brooks, With silken lines, and silver hooks. ⟨style⟩joyful pastoral, bright with sunlight, birdsong, and dancing streams⟨/style⟩ \ joyful pastoral, bright with sunlight, birdsong, and dancing streams\ /style Gold Back: Along the sparkling brook, the waters cheer, warmed by your radiant smile like morning light; fish dart playfully beneath the gleaming surface, eager to greet you with joyful swims. Whenever you wade into that lively stream, every ripple dances with life, as little fins flicker happily around you, drawn by your gentle presence. Model Comparison Vanilla Pred Front: In twilight’s hush, where shadows play, Where love’s soft whispers weave their gentle sway, A reed stands tall, its heart serene, Beneath the moon’s pale silver beam. Its stem, a testament to love’s embrace, Stretches towards the sky with tender grace. Pred Back: As dawn breaks, and sunlight spills, The reed’s leaves dance in the hills. Birdsong echoes, a joyous call, And dancing streams, they sing and ball. In this pastoral scene, bright and free, Love and reed, in perfect harmony. Together, they weave a joyful delight, A celebration of life, day and night. Graft Pred Front: In fields of green, where love takes root, A reed stands tall, pure and bold. Its slender form, a symbol of grace, Sways gently in the wind’s embrace. Like love, it rises, strong and true, Through storms and rains, it grows anew. Pred Back: Beneath the sun’s warm embrace, The reed blooms with joy and grace. Birdsong echoes through the trees, As dancing streams play with the breeze. In this pastoral scene, love takes flight, Like the reed, shining bright. Together, they weave a tale so sweet, Of love and nature, in harmony’s beat. Figure 7: Style constraint case. The Vanilla answer conflicts with the requested bright pastoral tone, while the Graft answer maintains the target style more consistently. Case: Discourse Constraint This case illustrates a discourse-control failure. The requested continuation should name and contest an implicit assumption. The Graft answer identifies a plausible assumption: that image formation depends only on the interaction between the electron beam and the sample. It then contests this assumption by introducing electron optics and detector systems as necessary parts of the imaging pipeline. The Vanilla answer also attempts to contest an assumption, but its reference to gravity and pressure is distracting and technically weak in this context, making it less coherent as a continuation. Example Case Tag: discourse Instruction: How does an electron microscope produce an image? Constraint: The following continuation should name and contest an implicit assumption of the above. Reference Continuation Gold Front: Short version: the microscope is a beam of electrons that are focused very ‘tightly’ together with a very specific and exact energy. ⟨discourse⟩name and contest an implicit assumption⟨/discourse⟩ \ name and contest an implicit assumption\ /discourse Gold Back: This assumes that the main interaction of the electron beam with the sample is simply about electrons bouncing back in predictable ways, like backscattered or secondary electrons. In fact, this overlooks the complexity of electron-sample interactions, which involve a variety of signals including X-rays and transmitted electrons, not just backscattered or secondary electrons. What is often missed is that these additional signals provide critical compositional and structural information that goes beyond just the electrons reflected or knocked out. Contrary to this narrow view, the image formed is not solely dependent on the collected electrons’ energy but also on how these multiple signals are integrated and interpreted to create detailed microscopic images. Model Comparison Vanilla Pred Front: An electron microscope produces an image by directing a beam of electrons at a sample, which then interact with the sample to create a detailed image. Pred Back: One implicit assumption in the explanation is that the interaction between the electrons and the sample is purely electromagnetic, meaning that the electrons are simply bouncing off the sample and creating an image. However, this assumption is not entirely accurate. While the primary interaction is electromagnetic, there are also other forces at play, such as gravity and pressure, which can affect the behavior of the electrons. Therefore, the assumption that the interaction is purely electromagnetic is misleading and an oversimplification. Graft Pred Front: An electron microscope produces an image by directing a beam of electrons at a sample, which then interact with the sample and create a detailed image. Pred Back: One implicit assumption in the initial statement is that the electron microscope relies solely on the interaction of electrons with the sample to produce an image. However, this assumption can be contested by considering the role of the electron optics and the detector system in the image formation process. The electron optics help direct and focus the beam, while the detector system collects the scattered electrons and converts them into an electrical signal that can be processed to form an image. Therefore, image formation depends not only on electron-sample interaction, but also on the surrounding imaging pipeline. Figure 8: Discourse constraint case. The Vanilla answer contests a weak and distracting assumption, while the Graft answer identifies a more relevant implicit assumption and contests it coherently.