Paper deep dive
Improving the Diversity of LLM Outputs without a Trade-off
Ryoma Sato
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:We propose DAST (Diversifying Arithmetic Sampling with TokenTour), a method that increases the diversity of LLM outputs without any change to the marginal distribution and with negligible generation-time overhead (a few microseconds). We observe that token IDs are often arranged in a meaningless order and reassign them so that tokens with similar meanings appear consecutively. This can be done in advance in a few hundred seconds per model, and the resulting order can be reused for all subsequent generations. By combining this order with arithmetic sampling (or quasi-Monte Carlo methods), we make similar tokens less likely to be generated across runs while preserving the distribution. Our method not only produces qualitatively good ideas but also significantly improves performance on the downstream task of ProtoQA.
Tags
Links
- Source: https://arxiv.org/abs/2609.33038v1
- Canonical: https://arxiv.org/abs/2609.33038v1
Trouble viewing inline? Open PDF directly →
Full Text
35,770 characters extracted from source content.
Expand or collapse full text
Improving the Diversity of LLM Outputs without a Trade-off Ryoma Sato rsato@nii.ac.jp Affiliation: National Institute of Informatics Abstract We propose DAST (Diversifying Arithmetic Sampling with TokenTour), a method that increases the diversity of LLM outputs without any change to the marginal distribution and with negligible generation-time overhead (a few microseconds). We observe that token IDs are often arranged in a meaningless order and reassign them so that tokens with similar meanings appear consecutively. This can be done in advance in a few hundred seconds per model, and the resulting order can be reused for all subsequent generations. By combining this order with arithmetic sampling (or quasi-Monte Carlo methods), we make similar tokens less likely to be generated across runs while preserving the distribution. Our method not only produces qualitatively good ideas but also significantly improves performance on the downstream task of ProtoQA. 1 Introduction Real-world tasks do not always have a unique answer. Many LLM benchmarks evaluate accuracy by having the model produce a single answer, but in practice, it is desirable for an LLM to generate multiple ideas for the user to consider. Simply changing the random seed and generating again, however, may produce similar ideas. Another setting in which diversity matters is solving problems with verifiable solutions [3, 32, 11]. When it is possible to verify whether a problem has been solved, an effective strategy is to have an LLM generate multiple candidate solutions and adopt one that solves the problem, if any. Generating candidates that are as diverse as possible increases the likelihood that at least one is correct. Even in verifiable settings, as discussed by Lee et al. [14], obtaining a single output that passes the tests is not always sufficient: diversity among the passing outputs may itself be important. Diverse outputs are also useful for reinforcement learning with verifiable rewards [37, 16, 18, 15, 27]. In reinforcement learning with verifiable rewards, training can stall when the outputs are either all correct or all incorrect [38, 13, 36, 9], and diversifying LLM outputs alleviates this problem. Even when solutions are not necessarily verifiable, selecting from diverse outputs using an automatic metric can improve performance [6]. Many methods have been proposed for generating diverse ideas. Approaches include changing the sampling temperature, perturbing the prompt [31], sampling without replacement [10, 24], and modifying the beam search criterion [28, 19]. These approaches, however, generally have drawbacks such as changing the original distribution or increasing generation time. Arithmetic Sampling [29] is one of the rare methods that can increase diversity without changing the marginal distribution or increasing generation time, while allowing parallelization just as in i.i.d. generation. The method constructs an arithmetic code from the LLM’s generation distribution. To generate K samples, it first samples a reference position p∼Unif([0,1])p ([0,1]) and then sets the position of sample i=1,2,…,Ki=1,2,…,K to pi=(p+i−1K)mod1p_i=(p+ i-1K) 1. This guarantees that the K samples are evenly spaced over [0,1][0,1]. Figure 1 illustrates next-token generation with K=3K=3. When the next-token distribution is represented on a circle, the K random numbers are placed exactly 120 degrees (360K 360K) apart. With i.i.d. sampling, all random numbers may fall in the same region, whereas arithmetic sampling always spaces them evenly, making repeated generation of the same token less likely. Even with this arrangement, each individual arm is sampled uniformly over 360 degrees, so the marginal distribution is unaffected. The generation overhead is also negligible. Figure 1: Comparison of i.i.d. and arithmetic sampling when generating K=3K=3 samples. With i.i.d. sampling, all three may fall on the same token. Arithmetic sampling places the three random numbers exactly 120 degrees (13 13 of a turn) apart, making them less likely to fall on the same token. Note that even with arithmetic sampling, each arm is sampled uniformly over 360 degrees. Arithmetic sampling, however, does not account for the ordering of the vocabulary around the circle. An LLM’s vocabulary can contain tokens with the same meaning but different token IDs. This can arise for various reasons, including singular and plural forms such as gift and gifts, differences in capitalization such as tour and Tour, and synonyms such as tale and story. Thus, even when arms are 120 degrees apart and fall on different token IDs, those IDs may correspond to tokens with the same meaning (Figure 2, left). In such cases, arithmetic sampling can still generate tokens with the same meaning. In particular, in the next-token distribution, implausible tokens have probabilities close to 00 and occupy arcs too short to see, while a set of similar, plausible tokens occupies a large area. Empirically, several kinds of similar tokens are often scattered around the circle, making this outcome more likely. We observe that vocabulary ordering offers an opportunity for optimization “for free.” The order of vocabulary IDs has no meaning to an LLM and has therefore been overlooked. It is only when arithmetic sampling is used that the order becomes meaningful because of the relationships between the arms. Prior work recognized this opportunity but did not act on it. Indeed, Parashar et al. [20] state that “we limited ourselves to randomizing the vocabulary ordering and then performing arithmetic sampling,” but did not optimize the vocabulary order. We are the first to consider how to assign vocabulary IDs for arithmetic sampling. Finding an optimal vocabulary order is not straightforward. There are n!n! permutations, and optimizing a permutation is often nontrivial, as in the traveling salesman problem. We consider obtaining an ordering of an LLM’s vocabulary using WordTour [23], which optimizes the order of word embeddings to arrange them in one dimension. This approach has the advantages of requiring no calibration data and allowing the order to be reused once constructed. We show that even at the scale of an LLM’s vocabulary, a well-organized ordering can be obtained in a practical amount of time. We also show that using this vocabulary order increases the diversity of LLM outputs “for free” compared with the standard vocabulary order. In particular, our method significantly improves performance on the downstream task of ProtoQA, which measures the ability to generate diverse and valid outputs. Figure 2: Comparison of arithmetic sampling and our proposed method, DAST. LLM vocabularies generally have a disordered arrangement. Arms separated by 120 degrees are less likely to fall on exactly the same token ID, but may fall on similar tokens with different IDs. This is particularly likely in the next-token probability distribution, where irrelevant tokens have probabilities close to 0, while similar tokens that are plausible as the next token have large probabilities (areas). DAST uses WordTour to place tokens with similar meanings next to one another, making arms separated by 120 degrees more likely to fall on tokens with different meanings. Since this simply reassigns vocabulary IDs in advance, exactly the same inference program can produce exactly the same marginal distribution. When K>1K>1 samples are generated simultaneously, the only difference is that the samples are more diverse. 2 Proposed Method (DAST) Simply put, our proposed method, DAST, reorders the vocabulary using WordTour and applies arithmetic sampling. Specifically, let =1,2,…,nV=\1,2,…,n\ be the vocabulary of an LLM, and let i v_i be the (input) embedding of token i. We reorder the vocabulary so that similar tokens are adjacent by solving the following optimization problem: minπ:→∑i=1n∥π(i)−π(i+1)∥ _π _i=1^n\| v_π(i)- v_π(i+1)\| (1) Here, π(n+1)=π(1)π(n+1)=π(1). That is, we arrange the vocabulary around a circle, corresponding to the placement of samples using mod1mod1 in arithmetic sampling. The original WordTour solves optimization problem (1) using the LKH solver [7], whereas DAST constructs an initial tour with the nearest-neighbor heuristic and refines it with 2-OPT. We found that 2-OPT is sufficient for arithmetic sampling. Although the LKH solver can slightly reduce the objective value of problem (1), the difference is small and does not translate into improvements in the diversity of arithmetic sampling or downstream task performance. Since 2-OPT closes almost the entire gap between the LLM’s standard vocabulary order and the optimal order, is simple, and runs quickly, it is well suited to DAST’s purpose. This reordering requires only the model parameters. The absence of a need for calibration data is an advantage. For example, one could also consider reordering tokens based on the model’s activation patterns, but this would require calibration data and would be more expensive. The input embeddings used by DAST come with the model “for free,” making reordering inexpensive. Note that this reordering must be performed separately for each model because models may have different vocabularies. Once it has been performed for a model, however, the order can be reused for all of that model’s outputs. For typical vocabularies of tens of thousands of tokens, reordering takes a few minutes. We also release the vocabulary orders used in our experiments, so users of these models do not need to perform the reordering themselves. Once the reordered vocabulary is available, DAST performs arithmetic sampling as usual. It is extremely simple, but simplicity is a virtue. The reordered vocabulary can be plugged in without even changing the generation program or incurring generation overhead, yet DAST does increase the diversity of LLM outputs. DAST inherits the desirable properties of arithmetic sampling. In particular, it guarantees that the desired result will be obtained if the number of samples K is sufficiently large. Specifically, suppose that a desirable output y exists for a prompt x. Assume that the user does not know y in advance but can recognize it as desirable when the LLM presents y. Such settings range from casual situations, such as brainstorming gifts for a friend, to rigorous ones, such as RLVR. In these settings, the LLM should generate outputs that are as diverse as possible so that at least one matches y. The following theorem holds. Theorem 2.1. Suppose that all token probabilities of the LLM are positive, as is the case for typical LLMs with softmax outputs. For any token sequence x and any token sequence y, there exists a number of samples K such that y is guaranteed to be obtained. That is, Pry1,…,yK∼DAST(x)[∃i∈[K],yi=y]=1 _y_1,…,y_K (x) [∃ i∈[K],y_i=y ]=1 (2) holds. In contrast, no such K exists for i.i.d. sampling. Proof. Let K=⌈1P[y]⌉K= 1P[y] . In the arithmetic code, y is assigned a contiguous interval of length P[y]P[y]. Since the distance between the arms in arithmetic sampling is 1K≤P[y] 1K [y], at least one arm falls in the interval for y, regardless of the initial position of the arms. Thus, there exists an i∈[K]i∈[K] such that yi=y_i=y. In contrast, with i.i.d. sampling, Pr[yi=y]=P[y] [y_i=y]=P[y] and Pr[∀i∈[K],yi≠y]=(1−P[y])K>0 [∀ i∈[K],y_i≠ y]=(1-P[y])^K>0, so for any K, there is always a possibility that y is not generated. ∎ 3 Experiments We demonstrate DAST’s speed, its qualitative effectiveness, and its quantitative effectiveness on ProtoQA. 3.1 Experimental Setup Table 1: Vocabulary sizes of the models. Model SmolLM2-1.7B Qwen2.5-1.5B Qwen2.5-3B Qwen2.5-7B |V||V| 49,152 151,936 151,936 152,064 We conduct experiments using SmolLM2-1.7B-Instruct [1], Qwen2.5-1.5B-Instruct, Qwen2.5-3B-Instruct, and Qwen2.5-7B-Instruct [21]; in the tables and figures we omit the -Instruct suffix for brevity. These models span two model families and a range of sizes, providing a diverse testbed. Their vocabulary sizes are listed in Table 1. The vocabularies are relatively large, making scalability important. We compare i.i.d. sampling, arithmetic sampling with the standard vocabulary order, and DAST. All of these preserve the marginal distribution; they are among the rare approaches that allow diversity to be increased without side effects, in a sense. ProtoQA [2] is a dataset of commonsense questions without a unique answer. For example, 100 people are asked a question such as “Name something that people usually do before they leave for work?” Of these, 43 answer Shower, 30 answer Breakfast, and 7 answer Lock door. The dataset measures the ability to generate diverse ideas by evaluating answer coverage without counting repeated answers more than once. For example, if an LLM is asked to generate K=3K=3 answers and responds with Shower, Shower, Shower, it receives 43 points, the score for the unique answer. If it instead responds with Shower, Breakfast, Lock door, it receives 43 + 30 + 7 = 80 points. Thus, generating more diverse answers can yield a higher score. We use this dataset to evaluate the diversity of DAST. 3.2 Speed Comparison Table 2: The two types of overhead in DAST. “One-time” is the time required to construct the vocabulary order (nearest-neighbor search, tour construction, and 2-OPT combined), which can then be reused for all subsequent generations. The remaining columns show the time required to generate one token. “Common” denotes operations independent of the sampling method, such as the forward pass. DAST’s per-generation overhead relative to i.i.d. sampling is about 2 microseconds, less than 0.4% of the forward-pass time. Measurements were taken on an NVIDIA RTX PRO 6000 Blackwell GPU and an AMD EPYC 9534 CPU. One-time Common Sampling overhead Model (Order construction) (Forward pass, etc.) i.i.d. token ID DAST DAST −- i.i.d. SmolLM2-1.7B 21.3 s 503 μ 8.6 μ 10.1 μ 10.5 μ ++1.9 μ Qwen2.5-1.5B 83.5 s 530 μ 8.5 μ 9.9 μ 10.3 μ ++1.8 μ Qwen2.5-3B 105.6 s 764 μ 8.6 μ 10.0 μ 10.6 μ ++2.0 μ Qwen2.5-7B 228.0 s 1,215 μ 8.9 μ 10.4 μ 10.9 μ ++2.0 μ We measure the speed of DAST. DAST has two types of overhead: the one-time construction of the order and arithmetic sampling at each generation. Table 2 reports the corresponding runtimes. Even the one-time computation can be completed in a few minutes, and the generation overhead is only a few microseconds, making it negligible. 3.3 Vocabulary Ordering Table 3: Objective values (1) of the vocabulary orders (lower is better). The “Held–Karp lower bound” is a rigorous lower bound obtained from the dual problem; the true optimum is at least this value. Thus, the gap between DAST and the true optimum is bounded above by the “DAST / lower bound” column. Model Standard vocabulary IDs DAST Held–Karp lower bound DAST / lower bound SmolLM2-1.7B 212,363 151,905 148,736 ++2.13% Qwen2.5-1.5B 203,768 148,881 145,729 ++2.16% Qwen2.5-3B 223,011 163,290 159,730 ++2.23% Qwen2.5-7B 169,399 146,950 145,173 ++1.22% We analyze the vocabulary orders obtained by optimization. Table 3 reports the objective values for the models’ standard vocabulary orders and the orders obtained by DAST. A lower bound for optimization problem (1) can also be obtained from the dual problem. We report this value as a reference for assessing optimality. The models’ standard vocabulary orders leave considerable room for optimization, and DAST successfully brings the objective values close to the lower bound. Table 4: The models’ standard token ID orders and the token orders produced by DAST. Only selected segments are shown. The standard orders are completely disordered, whereas WordTour places similar tokens in contiguous segments. Model Standard ID DAST SmolLM2-1.7B ␣Rim ␣equates ␣streaks ␣pharmacists "... Luck dialog jas ␣REG ␣Ngu ␣mixer ␣Jesuits ␣exceedingly ␣extremely ␣incredibly ␣exceptionally ␣extraordinarily ␣extraordinary ␣remarkable ␣incredible ␣amazing ␣astounding ␣astonishing ␣startling Qwen2.5-1.5B .DB ␣popularity ␣gew ␣impr setValue FLAG →|→ -7.0mu|max ␣bake wy ␣Economic ␣encontr ␣fname ␣huge ␣enormous ␣immense ␣tremendous ␣tremendously ␣immensely ␣enormously ␣hugely ␣greatly ␣vastly ␣radically ␣drastically Table 4 presents representative segments selected from the orders for SmolLM2-1.7B and Qwen2.5-1.5B. While the models’ standard orders are disordered, DAST places tokens with the same or similar meanings in contiguous segments. The complete orders produced by DAST can be viewed at the following links: SmolLM2-1.7B https://raw.githubusercontent.com/joisino/dast/refs/heads/main/HuggingFaceTB_SmolLM2-1.7B-Instruct.txt Qwen2.5-1.5B https://raw.githubusercontent.com/joisino/dast/refs/heads/main/Qwen_Qwen2.5-1.5B-Instruct.txt Qwen2.5-3B https://raw.githubusercontent.com/joisino/dast/refs/heads/main/Qwen_Qwen2.5-3B-Instruct.txt Qwen2.5-7B https://raw.githubusercontent.com/joisino/dast/refs/heads/main/Qwen_Qwen2.5-7B-Instruct.txt For all models, DAST successfully places tokens with the same meaning close to one another. These lists save users the effort of ordering the vocabulary themselves. 3.4 ProtoQA Figure 3: Quantitative results on ProtoQA. A positive orange bar on the left indicates that DAST outperforms i.i.d. sampling. A positive green bar in the middle indicates that arithmetic sampling outperforms i.i.d. sampling. An orange bar on the left that is higher than the green bar in the middle, as well as a positive blue bar on the right, indicates that DAST outperforms arithmetic sampling. Error bars show 95 percent confidence intervals. Figure 3 shows the experimental results on ProtoQA. We sample 33 answers from the LLM using each method and evaluate their coverage of human answers. Following ProtoQA’s Max Answers@3, the score is normalized so that it is 1 when the three largest answer clusters are covered and 0 when none are covered. To examine the differences between methods in detail, the figure shows score differences and their confidence intervals. DAST and arithmetic sampling achieve approximately 2 to 4 percentage points greater coverage than i.i.d. sampling, and DAST further improves coverage over arithmetic sampling by up to 0.33 percentage points. For Qwen2.5-1.5B and Qwen2.5-3B, the differences exceed the uncertainty indicated by the confidence intervals. This is a substantial gain given that DAST can be introduced with almost no overhead. 3.5 Qualitative Example Figure 4: Example answers from Qwen2.5-0.5B-Instruct, a smaller model used here for illustration, to the ProtoQA question “name a day of the year when most people call home.” The figure uses the actual token orders and arc lengths proportional to the output probabilities. In the standard token order, christmas and Christmas are far apart, and two arms of arithmetic sampling hit these two tokens, making the first tokens of the answers Christmas, christmas, Halloween. In DAST, the Christmas and christmas tokens are close together and are therefore not selected simultaneously; the first tokens of the outputs are christmas, Thanksgiving, Mother. Figure 4 presents an end-to-end qualitative example using the answers of Qwen2.5-0.5B-Instruct, a smaller model used here for illustration, to the ProtoQA question “name a day of the year when most people call home.” In the standard token order, christmas and Christmas are far apart, and two arms of arithmetic sampling hit these two tokens, making the first tokens of the answers Christmas, christmas, Halloween. In DAST, the Christmas and christmas tokens are adjacent and are therefore not selected simultaneously; the first tokens of the outputs are christmas, Thanksgiving, Mother. In this way, DAST is less likely to select distinct tokens with the same meaning, allowing it to produce more diverse outputs. 4 Related Work This section discusses the diversity of LLM outputs. The simplest way to increase diversity is to raise the sampling temperature [8, 4]. Although this increases diversity, it generally reduces quality [4]. This contrasts with our focus on improving diversity without any loss of quality. This trade-off is common to most diversification methods. Wang et al. [31] showed that, when obtaining multiple answers to the same question, perturbing the prompt produces more diverse outputs than generating repeatedly from exactly the same prompt. This method also exhibits a trade-off between diversity and quality depending on the strength of the perturbation. In a similar direction, Wang et al. [30] improve diversity by constructing prompts in multiple languages and cultural contexts. Methods that intervene in representations, such as G2 [22], STARS [39], and SemDiD [25], exhibit a similar trade-off. A related but distinct line of work is sampling without replacement from language models [10, 24]. Kool et al. [10] proposed an efficient method for sampling without replacement from sequence models using the Gumbel-Top-k trick and lazy evaluation of Gumbel samples on the sequence tree. Shi et al. [24] proposed a more efficient method for sampling without replacement using a trie. Although sampling without replacement may slightly improve diversity over sampling with replacement, it does not directly prevent duplicates among semantically equivalent sequences. Another popular heuristic is to modify the beam search criterion [28, 19]. Vijayakumar et al. [28] encourage diversity by adding a scoring term that favors candidates less similar to previously selected samples. Determinantal Beam Search [19] uses ideas from determinantal point processes to select beam search candidates that have high likelihoods while being dissimilar to one another. Although these methods empirically increase diversity, they generally provide no guarantee of diversity in the final outputs and change the original distribution. Simpler heuristics include prompting an LLM to produce different answers [26], but these also generally offer no guarantee of diversity in the final outputs and change the original distribution. They also make generation difficult to parallelize. Wong et al. [34] improve diversity through step-by-step stratified sampling, but this can also change the distribution and generally increases processing time. Other studies seek to improve diversity through post-training, such as Chung et al. [5] and Li et al. [17], but these require a certain amount of computation and, naturally, change the distribution. The same holds for methods such as BACo [33], which use more diverse base models as appropriate. A parallel line of work studies diversification in diffusion language models [35, 12]. Note that our paper assumes autoregressive LLMs and thus differs in scope from these studies. 5 Conclusion We proposed DAST, a method that can increase the diversity of LLM outputs without changing the marginal distribution or increasing generation time, while allowing parallelization. DAST addresses the overlooked role of LLM vocabulary ordering in the literature and improves diversity by explicitly constructing an arrangement. DAST improved the arrangement and increased diversity both qualitatively and quantitatively. References [1] L. B. Allal, A. Lozhkov, E. Bakouch, G. M. Blázquez, G. Penedo, L. Tunstall, A. Marafioti, H. Kydlícek, A. P. Lajarín, V. Srivastav, J. Lochner, C. Fahlgren, X.-S. Nguyen, C. Fourrier, B. Burtenshaw, H. Larcher, H. Zhao, C. Zakka, M. Morlon, C. Raffel, L. von Werra, and T. Wolf. SmolLM2: When smol goes big - Data-Centric training of a small language model. arXiv, 2025. doi: 10.48550/arXiv.2502.02737. URL https://arxiv.org/abs/2502.02737. [2] M. Boratko, X. Li, T. O’Gorman, R. Das, D. Le, and A. McCallum. ProtoQA: A question answering dataset for prototypical Common-Sense reasoning. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP, pages 1122–1136, 2020. doi: 10.18653/v1/2020.emnlp-main.85. URL https://w.aclweb.org/anthology/2020.emnlp-main.85. [3] B. C. A. Brown, J. Juravsky, R. Ehrlich, R. Clark, Q. V. Le, C. Ré, and A. Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling. arXiv, 2024. doi: 10.48550/arXiv.2407.21787. URL https://arxiv.org/abs/2407.21787. [4] M. Caccia, L. Caccia, W. Fedus, H. Larochelle, J. Pineau, and L. Charlin. Language GANs falling short. In Proceedings of the 8th International Conference on Learning Representations, ICLR, 2020. doi: 10.48550/arxiv.1811.02549. URL https://arxiv.org/abs/1811.02549. [5] J. J. Y. Chung, V. Padmakumar, M. Roemmele, Y. Sun, and M. Kreminski. Modifying large language model Post-Training for diverse creative writing. In Proceedings of the 2nd Conference on Language Modeling, COLM, 2025. doi: 10.48550/arXiv.2503.17126. URL https://arxiv.org/abs/2503.17126. [6] M. Freitag, D. Grangier, Q. Tan, and B. Liang. High quality rather than high model probability: Minimum bayes risk decoding with neural metrics. Transactions of the Association for Computational Linguistics, 10:811–825, 2022. doi: 10.1162/tacl_a_00491. URL https://arxiv.org/abs/2111.09388. [7] K. Helsgaun. LKH (Keld Helsgaun), 2018. URL http://webhotel4.ruc.dk/~keld/research/LKH/. [8] D. Ippolito, R. Kriz, J. Sedoc, M. Kustikova, and C. Callison-Burch. Comparison of diverse decoding methods from conditional language models. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, ACL, pages 3752–3762, 2019. doi: 10.18653/v1/p19-1365. URL https://aclanthology.org/P19-1365. [9] M. Jiang, Z. Wang, Q. Wang, P. Búš, M. Cheng, Y. Wang, Q. Liu, R. Li, P. Zeng, R. Liu, A. Liang, Y. Xu, Y. Hu, C. Zhang, and Z. Dong. TAPO: Dynamic teacher and perturbed answer injection for policy optimization. In Proceedings of the 40th AAAI Conference on Artificial Intelligence, AAAI, pages 37462–37471, 2026. doi: 10.1609/aaai.v40i44.41079. URL https://doi.org/10.1609/aaai.v40i44.41079. [10] W. Kool, H. van Hoof, and M. Welling. Stochastic beams and where to find them: The Gumbel-Top-k trick for sampling sequences without replacement. In Proceedings of the 36th International Conference on Machine Learning, ICML, pages 3499–3508, 2019. doi: 10.48550/arxiv.1903.06059. URL https://arxiv.org/abs/1903.06059. [11] S. Lamont, C. Walder, A. Dezfouli, P. Montague, and M. Norrish. 3D-Prover: Diversity driven theorem proving with determinantal point processes. In Advances in Neural Information Processing Systems, NeurIPS, pages 64741–64764, 2025. doi: 10.52202/085713-2169. URL https://arxiv.org/abs/2410.11133. [12] S. Lamont, C. Walder, P. Montague, A. Dezfouli, and M. Norrish. Free lunch for pass@k? low cost diverse sampling for diffusion language models. arXiv, 2026. doi: 10.48550/arXiv.2603.04893. URL https://arxiv.org/abs/2603.04893. [13] T.-L. V. Le, M. Jeon, K. Vu, V. D. Lai, and E. Yang. No prompt left behind: Exploiting Zero-Variance prompts in LLM reinforcement learning via Entropy-Guided advantage shaping. In Proceedings of the 14th International Conference on Learning Representations, ICLR, 2026. doi: 10.48550/arXiv.2509.21880. URL https://arxiv.org/abs/2509.21880. [14] S. Lee, H. Chon, J. Jang, D. Lee, and H. Yu. How diversely can language models solve problems? exploring the algorithmic diversity of Model-Generated code. In Findings of the Association for Computational Linguistics: EMNLP 2025, EMNLP, pages 152–167, 2025. doi: 10.18653/v1/2025.findings-emnlp.10. URL https://arxiv.org/abs/2503.00691. [15] C. Li, Y. Zhang, B. Wang, G. Ma, W. Tang, H. Huang, and N. Duan. SetPO: Set-Level policy optimization for Diversity-Preserving LLM reasoning. In Proceedings of the 43rd International Conference on Machine Learning, ICML, 2026a. doi: 10.48550/arXiv.2602.01062. URL https://arxiv.org/abs/2602.01062. [16] M. Y. Li, A. Zhan, K. Gandhi, N. D. Goodman, and E. B. Fox. QuasiMoTTo: Quasi-Monte carlo Test-Time scaling. arXiv, 2026b. doi: 10.48550/arXiv.2607.01179. URL https://arxiv.org/abs/2607.01179. [17] T. Li, Y. Zhang, P. Yu, S. Saha, D. Khashabi, J. Weston, J. Lanchantin, and T. Wang. Jointly reinforcing diversity and quality in language model generations. arXiv, 2025. doi: 10.48550/arXiv.2509.02534. URL https://arxiv.org/abs/2509.02534. [18] X. Li, Y. Li, X. Fang, S. Ding, P. Li, Y. Chen, Y. Ma, T. Lyu, L. Li, D. Lin, Q. Guo, Q. Liu, and K. Chen. Beyond mode collapse: Distribution matching for diverse reasoning. In Proceedings of the 43rd International Conference on Machine Learning, ICML, 2026c. doi: 10.48550/arXiv.2605.19461. URL https://arxiv.org/abs/2605.19461. [19] C. Meister, M. Forster, and R. Cotterell. Determinantal beam search. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), ACL-IJCNLP, pages 6551–6562, 2021. doi: 10.18653/v1/2021.acl-long.512. URL https://aclanthology.org/2021.acl-long.512. [20] A. Parashar, A. V. Singh, A. Amballa, J. Lai, and B. Rozonoyer. Quasi-random Multi-Sample inference for large language models. In Proceedings of the ICLR 2025 Workshop on Frontiers in Probabilistic Inference: Learning Meets Sampling, FIP, 2025. URL https://arxiv.org/abs/2411.06251. [21] Qwen, :, Y. An, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Wei, H. Lin, J. Yang, J. Tu, J. Zhang, J. Yang, J. Yang, J. Zhou, J. Lin, K. Dang, K. Lu, K. Bao, K. Yang, L. Yu, M. Li, M. Xue, P. Zhang, Q. Zhu, R. Men, R. Lin, T. Li, T. Tang, T. Xia, X. Ren, X. Ren, Y. Fan, S. Yang, Y. Zhang, Y. Wan, Y. Liu, Z. Cui, Z. Zhang, and Z. Qiu. Qwen2.5 technical report. arXiv, 2024. doi: 10.48550/arXiv.2412.15115. URL https://arxiv.org/abs/2412.15115. [22] Z. Ruan, Y. Li, Y. Liu, Y. Chen, W. Luo, P. Li, Y. Liu, and G. Chen. G2: Guided generation for enhanced output diversity in LLMs. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, EMNLP, pages 14116–14134, 2025. doi: 10.18653/v1/2025.emnlp-main.713. URL https://arxiv.org/abs/2511.00432. [23] R. Sato. Word tour: One-dimensional word embeddings via the traveling salesman problem. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT, pages 2166–2172, 2022. doi: 10.18653/v1/2022.naacl-main.157. URL https://arxiv.org/abs/2205.01954. [24] K. Shi, D. Bieber, and C. Sutton. Incremental sampling without replacement for sequence models. In Proceedings of the 37th International Conference on Machine Learning, ICML, pages 8785–8795, 2020. doi: 10.48550/arxiv.2002.09067. URL https://arxiv.org/abs/2002.09067. [25] W. Shi, Y. Cui, Y. Wu, J. Fang, S. Zhang, M. Li, S. Han, J. Zhu, J. Xu, and X. Zhou. Semantic-guided diverse decoding for large language model. In Advances in Neural Information Processing Systems, NeurIPS, pages 148182–148229, 2025. doi: 10.52202/085713-4953. URL https://arxiv.org/abs/2506.23601. [26] S. Troshin, I. Saparina, A. Fokkens, and V. Niculae. Asking a language model for diverse responses. In Proceedings of the 2nd Workshop on Uncertainty-Aware NLP, UncertaiNLP, pages 66–72, 2025. doi: 10.18653/v1/2025.uncertainlp-main.8. URL https://arxiv.org/abs/2509.17570. [27] J. Tuyls, D. J. Foster, A. Krishnamurthy, and J. T. Ash. Representation-Based exploration for language models: From Test-Time to Post-Training. In Proceedings of the 14th International Conference on Learning Representations, ICLR, 2026. doi: 10.48550/arXiv.2510.11686. URL https://arxiv.org/abs/2510.11686. [28] A. K. Vijayakumar, M. Cogswell, R. R. Selvaraju, Q. Sun, S. Lee, D. J. Crandall, and D. Batra. Diverse beam search for improved description of complex scenes. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence, AAAI, pages 7371–7379, 2018. doi: 10.1609/aaai.v32i1.12340. URL https://doi.org/10.1609/aaai.v32i1.12340. [29] L. Vilnis, Y. Zemlyanskiy, P. Murray, A. T. Passos, and S. Sanghai. Arithmetic sampling: Parallel diverse decoding for large language models. In Proceedings of the 40th International Conference on Machine Learning, ICML, pages 35120–35136, 2023. doi: 10.48550/arXiv.2210.15458. URL https://arxiv.org/abs/2210.15458. [30] Q. Wang, S. Pan, T. Linzen, and E. Black. Multilingual prompting for improving LLM generation diversity. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, EMNLP, pages 6367–6389, 2025a. doi: 10.18653/v1/2025.emnlp-main.324. URL https://arxiv.org/abs/2505.15229. [31] T. Wang, Z. Liu, Y. Chen, J. Light, W. Liu, H. Chen, X. Zhang, and W. Cheng. Diversified sampling improves scaling LLM inference. arXiv, 2025b. doi: 10.48550/arXiv.2502.11027. URL https://arxiv.org/abs/2502.11027. [32] T. Wang, Z. Liu, Y. Chen, J. Light, W. Liu, H. Chen, X. Zhang, and W. Cheng. On the effect of sampling diversity in scaling llm inference. In Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, UAI, 2026a. URL https://arxiv.org/abs/2502.11027. [33] Y. Wang, C. Yang, T. Huang, M. Chen, J. May, and M. Lee. Optimizing diversity and quality through Base-Aligned model collaboration. In Proceedings of the 43rd International Conference on Machine Learning, ICML, 2026b. doi: 10.48550/arXiv.2511.05650. URL https://arxiv.org/abs/2511.05650. [34] J. Wong, Y. Orlovskiy, A. Shypula, M. Luo, S. A. Seshia, and J. E. Gonzalez. SimpleStrat: Diversifying language model generation with stratification. In Advances in Neural Information Processing Systems, NeurIPS, pages 116906–116935, 2025. doi: 10.52202/085713-3897. URL https://arxiv.org/abs/2410.09038. [35] J. Wu, Z. Wan, X. Yu, Y. Yang, Y. Huang, I. W. Tsang, and Y. You. Time-Annealed perturbation sampling: Diverse generation for diffusion language models. arXiv, 2026. doi: 10.48550/arXiv.2601.22629. URL https://arxiv.org/abs/2601.22629. [36] H. Xu, S. Chen, R. Qiu, Y. Yan, C. Luo, M. X. Cheng, J. He, and H. Tong. Prune as you generate: Online rollout pruning for faster and better RLVR. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL, pages 13876–13893, 2026. doi: 10.18653/v1/2026.acl-long.632. URL https://arxiv.org/abs/2603.24840. [37] J. Yao, R. Cheng, X. Wu, J. Wu, and K. Tan. Diversity-Aware policy optimization for large language model reasoning. In Advances in Neural Information Processing Systems, NeurIPS, pages 94801–94826, 2025. doi: 10.52202/085713-3169. URL https://arxiv.org/abs/2505.23433. [38] Q. Yu, Z. Zhang, R. Zhu, Y. Yuan, X. Zuo, Y. Yue, W. Dai, T. Fan, G. Liu, J. Liu, L. Liu, X. Liu, H. Lin, Z. Lin, B. Ma, G. Sheng, Y. Tong, C. Zhang, M. Zhang, R. Zhang, W. Zhang, H. Zhu, J. Zhu, J. Chen, J. Chen, C. Wang, H. Yu, Y. Song, X. Wei, H. Zhou, J. Liu, W.-Y. Ma, Y.-Q. Zhang, L. Yan, Y. Wu, and M. Wang. DAPO: An Open-Source LLM reinforcement learning system at scale. In Advances in Neural Information Processing Systems, NeurIPS, pages 113222–113244, 2025. doi: 10.52202/085713-3775. URL https://arxiv.org/abs/2503.14476. [39] D. Zhu, L. T. H. Khanh, A. Y.-M. Cheung, M.-C. Yue, and V. A. Nguyen. Exploring diverse generation paths via inference-time stiefel activation steering. In Proceedings of the 14th International Conference on Learning Representations, ICLR, 2026. doi: 10.48550/arXiv.2601.22010. URL https://arxiv.org/abs/2601.22010.