Paper deep dive
A Shared Taste for Model-Written Text: The Generator-by-Selector Matrices of "AI-AI Bias" Show No Detectable Own-Model Premium
Dmitrij Żatuchin
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 10/2/2026, 8:52:41 AM
Summary
This paper re-analyzes the 'AI-AI bias' matrices from Laurito et al. (PNAS 2025) to test for an 'own-model premium'—whether language models prefer text generated by their own architecture. Using a two-way fixed-effects model and exact permutation tests on 21,828 trials across products, papers, and films, the study finds no statistically significant own-model preference (pooled p=0.14). While models generally prefer model-written text over human-written text, they do not show a detectable bias toward their own specific output. The analysis also controls for position bias and compares results with Tan et al. (ACL 2024), noting differences in experimental design.
Entities (11)
Relation Signals (7)
Zatuchin → authored → A Shared Taste for Model-Written Text
confidence 99% · Title and author list: 'A Shared Taste for Model-Written Text... Dmitrij ̇ Zatuchin'
Laurito et al. → published → PNAS 2025 study
confidence 95% · Laurito et al. (PNAS 2025) showed that large language models... prefer the description written by a language model
GPT-3.5 → samevendoras → GPT-4
confidence 95% · The same-vendor term for the GPT-3.5 and GPT-4 pair is negative in all three datasets.
Own-Model Premium → notdetectedin → Laurito et al. data
confidence 92% · The absence is informative down to about 0.04 share points... What these data do not show is a model recognising and favouring its own prose.
GPT-4 → preferredby → All Selectors
confidence 90% · GPT-4's descriptions chosen 77% to 95% of the time by every selector on products.
position bias → affects → Cell Selection
confidence 88% · Position bias moves single cells by up to 0.42 share points in either direction
Tan et al. → found → Own-Model Premium
confidence 85% · The 4x4 matrix of Tan et al. (ACL 2024) gives gamma = +0.148... The authors report this as models preferring self-generated contexts
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Laurito et al. (PNAS 2025) showed that large language models choosing between two descriptions of the same product, paper or film prefer the description written by a language model over the one written by a person, by a wide margin over what human judges do. Their design crosses five generators with the same five models as selectors, which permits a second question the paper does not headline: does a selector prefer text from its own model beyond what the generator and selector main effects predict? We rebuild the three 5x5 matrices from the per-item counts in the authors' public repository (21,828 valid trials; every cell matches the published value) and fit a two-way fixed-effects model with an own-model term gamma, tested by the exact permutation test over the 120 relabellings of the selectors. The premium is +0.013 on products (exact one-sided p = 0.24), -0.010 on paper abstracts (p = 0.74), +0.054 on films (p = 0.07) and +0.019 pooled (p = 0.14; 95% interval -0.008 to 0.046). The same-vendor term for the GPT-3.5 and GPT-4 pair is negative in all three datasets. Position bias moves single cells by up to 0.42 share points in either direction, and the own-model contrast is unchanged once order-driven items are removed. The design would have detected a premium of 0.05 with 82% (products), 88% (papers), 42% (films) and 97% (pooled) power; the minimum detectable effect at 80% power is 0.034 pooled. The absence is informative down to about 0.04 share points and silent below that. The 4x4 matrix of Tan et al. (ACL 2024) gives gamma = +0.148 at the smallest p its 24 relabellings allow, with a same-family term of the same size. The main result of Laurito et al. stands: models share a taste for model-written text, with GPT-4's descriptions chosen 77% to 95% of the time by every selector on products. What these data do not show is a model recognising and favouring its own prose.
Tags
Links
- Source: https://arxiv.org/abs/2610.00369v1
- Canonical: https://arxiv.org/abs/2610.00369v1
Trouble viewing inline? Open PDF directly →
Full Text
35,742 characters extracted from source content.
Expand or collapse full text
A Shared Taste for Model-Written Text: The Generator-by-Selector Matrices of “AI–AI Bias” Show No Detectable Own-Model Premium Dmitrij ̇ Zatuchin 1,2* 1* Department of Information Technologies, Estonian Entrepreneurship University of Applied Sciences (EUAS), Tallinn, Estonia. 2 Rankfor.AI, Wroc law, Poland; Tallinn, Estonia. Corresponding author(s). E-mail(s): dmitrij.zatuchin@eek.e; Abstract Laurito et al. (PNAS 2025) showed that large language models choosing between two descriptions of the same product, paper or film prefer the description written by a language model over the one written by a person, by a wide margin over what human judges do. Their design crosses five generators with the same five models as selectors, which permits a second question the paper does not headline: does a selector prefer text from its own model beyond what the generator and selector main effects predict? We rebuild the three 5×5 matrices from the per-item counts in the authors’ public repository (21,828 valid trials; every cell matches the published value) and fit a two-way fixed-effects model with an own-model term γ, tested by the exact permutation test over the 120 relabellings of the selectors. The premium is +0.013 on products (exact one-sided p = 0.24),−0.010 on paper abstracts (p = 0.74), +0.054 on films (p = 0.07) and +0.019 pooled (p = 0.14; 95% interval −0.008 to 0.046). The same-vendor term for the GPT-3.5 and GPT-4 pair is negative in all three datasets. Position bias moves single cells by up to 0.42 share points in either direction, and the own- model contrast is unchanged once order-driven items are removed. The design would have detected a premium of 0.05 with 82% (products), 88% (papers), 42% (films) and 97% (pooled) power; the minimum detectable effect at 80% power is 0.034 pooled. The absence is informative down to about 0.04 share points and silent below that. The 4×4 matrix of Tan et al. (ACL 2024) gives γ = +0.148 at the smallest p its 24 relabellings allow, with a same-family term of the same size. The main result of Laurito et al. stands: models share a taste for model-written text, with GPT-4’s descriptions chosen 77% to 95% of the time by every selector on products. What these data do not show is a model recognising and favouring its own prose. Keywords: AI–AI bias, self-preference, LLM-as-a-judge, permutation test, power analysis, re-analysis 1 Introduction Laurito et al. asked whether large language mod- els, used as decision assistants, favour goods pitched in language-model prose over the same goods pitched by a person [1]. For 109 products, 100 paper abstracts and 250 film plot summaries, each item was described once by a human and once by each of five models, and each of the same five models then chose between the two descriptions 1 arXiv:2610.00369v1 [cs.IR] 30 Sep 2026 in a binary prompt shown in both orders. Every selector chose the model-written product descrip- tion more often than the human judges did, by 26 to 59 share points over the ten cells with a human comparison. The finding, which the authors call AI–AI bias, has since drawn a published letter and reply [2, 3]. The design also answers a question the paper leaves in its figures. Because the generators and the selectors are the same five models, the results form three 5×5 matrices, and a matrix can be decomposed: how much of a cell is the genera- tor’s text being attractive to every selector, how much is the selector leaning toward model text in general, and how much is left on the diagonal, where a model judges its own output. That diag- onal is the quantity the self-preference literature is about [4, 5], and it matters in practice. If a selector favours its own model’s prose, then a text tuned for one engine is tuned against the others; if the preference is shared, one model-friendly text serves them all. This note rebuilds the matrices from the authors’ repository and tests the diagonal. It adds nothing to the original finding and takes nothing from it. The contributions are: (i) the three matri- ces rebuilt from per-item counts, with every cell matching the published value; (i) the own-model premium per dataset and pooled, with an exact permutation test, a standard error, and the same- vendor effect separated from it; (i) the effect of position bias on the diagonal, from the runs in which the repository records which position the selector chose; (iv) the minimum detectable effect of this design and a power table for a larger one; and (v) the same test on the one comparable pub- lished matrix, Tan et al. [6], and the placement of both results in the 2024 to 2026 literature on self-preference. 2 Data Source and licence. The article is published under C BY 4.0 and states that all study data are included in the article and its SI Appendix. The code repository named in the article [7] car- ries no licence file; under GitHub’s terms the code is therefore readable and forkable and no further right is granted. We ran no code from it. The repository stores, for every run, one JSON record per item with the number of orderings in which the selector chose the model text, the human text, or gave no valid choice, and, for the runs of January 18 to 30, 2025, the number of times the first-listed text was chosen. No per-trial rows with position exist for the other runs. All quantities in this note are therefore aggregates over items or over cells, and the derived tables we release contain no text from either source. Design as recorded. For each dataset the paper uses one generation prompt and one selec- tion prompt; the repository names them, and 2025 runs of that pair exist for all 75 cells. The paper names GPT-3.5-turbo-0125 for products and films and GPT-3.5-turbo-1106 for papers, for both roles; the runs follow the same split. In the final design each item has one model-written description shown against the human one in both orders, so a cell holds two trials per item: 218 or 220 on products, 186 on papers, 500 on films, less the invalid trials. Papers are the exception in one cell: the GPT-4 text judged by GPT-3.5 was last run in January 2024 with four descriptions per item (744 trials, 124 invalid), and that is the run the published value 0.69 comes from; the January 2025 rerun of the same cell gives 0.71. Cell selection. For each cell we took the lat- est run whose LLM-win share, wins over wins plus losses, rounds to the published value. All 75 cells matched; the largest deviation between a rebuilt share and its published rounding is 0.005. The 75 runs hold 21,828 valid trials and 1,280 invalid ones. Where the repository holds several distinct 2025 runs of a cell (29 cells, 122 distinct runs in all), the others enter a sensitivity analysis in Section 4.3. The paper’s dataset of 100 papers appears as 93 items in every paper run. Human selectors judged only the GPT-3.5 and GPT-4 texts; we take those six values from the article’s Figures 2 to 4. 3 Method Model. Let y ij be the share of valid trials in which selector j chose the text written by generator i over the human text. We fit y ij = μ + a i + b j + γ [i = j] + e ij ,(1) with generator effects a i , selector effects b j and the own-model premium γ on the diagonal. In a full k × k matrix with one self cell per row and column, the least-squares γ equals the mean of the 2 k diagonal cells minus the mean of the k(k − 1) others, so it has the same units as the cells: share points of the binary choice. Inference. The null hypothesis is that model identity carries no information beyond the row and column effects, so the labelling of which selec- tor counts as “self” for each generator is exchange- able. We recompute γ under every relabelling of the five selector columns, 120 in all, and report the one-sided exact p, the share of relabellings with a premium at least as large as the observed one, together with the two-sided p. The smallest one- sided p this design can return is 1/120 = 0.008. The pooled test double-centres each matrix and fits one γ to the three stacked residual matri- ces under a common relabelling. We also report the least-squares standard error, σ/ √ k− 1 per matrix on 15 residual degrees of freedom and σ/ p 3(k− 1) pooled on 45, where σ is the residual standard deviation after all terms. The permuta- tion test makes no distributional assumption; the standard error gives an interval. Same-vendor effect. GPT-3.5 and GPT-4 are the only two models from one vendor, so the two cells where one judges the other’s text carry a family dummy, fitted with the self dummy and permuted with it. Trial-level model. To show what a trial-level analysis would claim, we fit a binomial logis- tic regression on the per-item counts with item, generator and selector fixed effects and the two dummies, with standard errors clustered by item, and we compute the cell-level Pearson dispersion of that fit: the ratio of the observed cell variance to what binomial sampling of the fitted shares predicts. Position. For the 27 runs that record first- listed choices, the two trials of an item with no invalid response fall into four patterns: the model text chosen in both orders, the human text cho- sen in both, the first-listed text chosen in both, or the second-listed in both. The last two are decided by position, the first two by content. We report each run’s first-choice rate, the share of items decided by position, and the LLM-win share among content-decided items, and we compare the diagonal to the off-diagonal cells of the same row on both measures. Items with more than two tri- als whose position split is not determined by the recorded counts (62 of 3,645) are left out of this analysis. Power. Residual variance in a matrix has two parts: binomial sampling of the cell share, p ij (1 − p ij )/n ij , which more trials shrink, and cell-level interaction noise, which they do not. We estimate the second as the residual variance of the fitted model less the mean binomial variance. The minimum detectable effect is then found by simu- lation: to the additive fit of each matrix we add a premium γ on the diagonal, Gaussian interaction noise with the estimated standard deviation, and binomial sampling at the observed trial counts, and we run the exact test on each draw. The reported value is the γ at which the one-sided test at α = 0.05 rejects in 80% of draws. The same machinery, with the pooled interaction estimate, a base share of 0.7 and random row and column effects, gives the power table for larger designs; for k > 6 the null uses 600 random relabellings. All simulation is seeded; the analysis script and the extracted counts are in the package. 4 Results 4.1 The own-model premium Table 1 gives the premium, and Figure 1 shows the residual matrices with the diagonal outlined. On products the five diagonal cells average 0.788 against 0.775 for the twenty others, a premium of +0.013; 29 of the 120 relabellings do at least as well, p = 0.24. On paper abstracts the diago- nal is below the rest, γ = −0.010, p = 0.74. On films the premium is +0.054 with p = 0.07: eight relabellings of 120 reach it, and the 95% interval runs from −0.013 to 0.121. Pooled over the three matrices γ = +0.019, p = 0.14, interval −0.008 to 0.046. In only 4 of the 15 generator rows is the generator’s own model the selector that likes its text most. The same test on the figure values as printed to two decimals gave +0.0125, −0.010, +0.050 and +0.018 with p of 0.27, 0.74, 0.08 and 0.18; the rebuilt counts move nothing of substance. On the logit scale, which is the natural scale if position bias compresses shares multiplicatively toward 0.5 (Section 4.4), the pooled premium is +0.049 log-odds with p = 0.27. 3 Table 1 Own-model premium in the three matrices of Laurito et al. and in Tan et al. γ is the mean of the diagonal cells minus the mean of the others after row and column effects; p is the exact one-sided permutation p over all relabellings of the selectors (120 for k = 5, 24 for k = 4); the interval is±t 0.975 times the least-squares standard error. “Self is row max” counts generators whose own selector gave their text its highest share. Family is the same-vendor term fitted alongside γ (GPT-3.5 and GPT-4; Llama 2 7B and 13B for Tan et al.). MatrixTrials Diag. Off-diag.γ Exact p 95% intervalRow maxFamily (p) Products, 5× 55,419 0.7880.775 +0.0130.242−0.026 to 0.0531 of 5−0.012 (0.75) Paper abstracts, 5× 54,779 0.6050.615−0.0100.742−0.046 to 0.0271 of 5−0.016 (0.68) Movie summaries, 5× 5 11,630 0.6740.620 +0.0540.067−0.013 to 0.1212 of 5−0.039 (0.71) Pooled, three matrices21,828+0.0190.142−0.008 to 0.0464 of 15−0.022 (0.72) Tan et al. 2024, 4× 40.5950.447 +0.1480.042 +0.001 to +0.2963 of 4 +0.163 (0.17) GPT-3.5 GPT-4 Llama 3.1 70B Mixtral 8x22B Qwen2.5 72B GPT-3.5 GPT-4 Llama 3.1 70B Mixtral 8x22B Qwen2.5 72B Generator (text written by) +0.01-0.02+0.01+0.01-0.01 -0.00+0.01-0.00-0.03+0.03 +0.05+0.00-0.03+0.02-0.04 -0.06-0.01+0.06+0.02-0.02 -0.00+0.02-0.04-0.02+0.04 Products γ = +0.013, exact p = 0.24 GPT-3.5 GPT-4 Llama 3.1 70B Mixtral 8x22B Qwen2.5 72B GPT-3.5 GPT-4 Llama 3.1 70B Mixtral 8x22B Qwen2.5 72B -0.03-0.00+0.01+0.08-0.06 -0.01-0.01+0.01-0.02+0.03 +0.04-0.02-0.02-0.01+0.00 -0.00+0.02-0.01-0.01+0.00 -0.00+0.01+0.01-0.04+0.02 Paper abstracts γ = -0.010, exact p = 0.74 GPT-3.5 GPT-4 Llama 3.1 70B Mixtral 8x22B Qwen2.5 72B GPT-3.5 GPT-4 Llama 3.1 70B Mixtral 8x22B Qwen2.5 72B +0.08-0.04-0.02+0.08-0.11 -0.04+0.05+0.03-0.04-0.01 +0.06-0.02-0.01+0.03-0.06 -0.02+0.02-0.02-0.03+0.05 -0.08-0.01+0.01-0.04+0.13 Movie summaries γ = +0.054, exact p = 0.07 GPT-4 GPT-3.5 Llama 2 13B Llama 2 7B GPT-4 GPT-3.5 Llama 2 13B Llama 2 7B +0.17+0.01-0.09-0.09 +0.11+0.07-0.07-0.11 -0.19-0.02+0.10+0.10 -0.09-0.06+0.06+0.10 Tan et al. 2024, NQ-AIR γ = +0.148, exact p = 0.04 −0.1 0.0 0.1 residual share Selector (model choosing); outlined cells: selector judging its own model's text Fig. 1 Residual share after removing generator and selector effects, for the three matrices of Laurito et al. and the NQ- AIR matrix of Tan et al. (bias toward the generated over the retrieved context). Outlined cells are the diagonal, where the selector judges its own model’s text. Blue is above the additive prediction, orange below. 4.2 Same vendor The one pair of models from a single vendor gives two off-diagonal cells per matrix: GPT-3.5 judg- ing GPT-4’s text and the reverse. Fitted alongside the self term, the family coefficient is −0.012 on products, −0.016 on papers and −0.039 on films, −0.022 pooled (p = 0.72), so those two cells sit below what the row and column effects predict, in all three datasets. With one pair the test has no power to speak of, and the design cannot separate a vendor effect from a size effect within a ven- dor; the direction does not favour a same-vendor premium. 4.3 Trial-level inference and why it overstates A binomial regression on the 11,300 item-by-cell counts, with item, generator and selector fixed effects and standard errors clustered by item, returns a self coefficient of −0.035 log-odds on products (SE 0.079), −0.077 on papers (0.064) and +0.256 on films (0.045), +0.135 pooled (0.034, z = 3.9). At face value the film and pooled esti- mates are highly significant, and face value is the wrong reading. The Pearson dispersion of the fit- ted cell shares is 3.7 on products, 1.1 on papers, 7.4 on films and 4.0 pooled: cells vary three to seven times more than binomial sampling of their trials predicts, so the trial is the wrong unit and the cell is the right one. The residual standard deviation per cell is 0.037 on products, 0.034 on papers and 0.063 on films; the part binomial sam- pling explains is 0.027, 0.036 and 0.022, leaving cell-level noise of 0.026, nil, and 0.059. Films are the noisy dataset, and papers are the one where more trials would help. The repository shows where the film noise comes from. For 29 cells it holds more than one distinct 2025 run of the same prompt pair; the median spread between runs of one cell is 0.012 share points, and the largest is 0.161, for GPT-4’s film summaries judged by GPT-3.5. That selector returned 22 invalid choices in one run of that cell, 94 in a second and 142 in the published one, and the share moved from 0.54 to 0.66 to 0.70. Recom- puting γ with the mean over distinct runs per cell 4 −0.10.00.1 own-model premium γ relabellings observed +0.013 MDE 0.045 Products p = 0.242 (29 of 120) −0.10.00.1 own-model premium γ observed -0.010 MDE 0.046 Paper abstracts p = 0.742 (89 of 120) −0.10.00.1 own-model premium γ observed +0.054 MDE 0.085 Movie summaries p = 0.067 (8 of 120) −0.10.00.1 own-model premium γ observed +0.019 MDE 0.034 Pooled, three datasets p = 0.142 (17 of 120) Fig. 2 Exact permutation null of the own-model premium: γ under each of the 120 relabellings of the selectors, with the observed value (solid) and the minimum detectable effect at 80% power (dashed). gives +0.011, −0.010 and +0.054 (p = 0.30, 0.72 and 0.08); with the latest run per cell, +0.005, −0.010 and +0.052 (p = 0.40, 0.69 and 0.09). The conclusion does not depend on which run stands for a cell. 4.4 Position bias Table 2 Position and the diagonal, in the rows where the repository records first-listed choices. “Raw” is the LLM-win share over all valid trials; “content” is the share among items decided the same way in both orders; “by position” is the share of items decided by order. Contrast is the own-model cell minus the mean of the other cells in the row. Contrast, self− others By position Rowrawcontent self others Products, Llama 3.1 70B−0.195+0.019 0.720.26 Products, Mixtral 8x22B +0.121+0.002 0.260.58 Papers, GPT-4−0.024−0.035 0.280.26 Films, GPT-4−0.006−0.088 0.460.53 Mean−0.026−0.025 0.420.41 The authors report first-item rates per selector and note that a strong position habit can mask a preference, since a selector that always takes the first text scores 0.5 on any cell. The recorded runs bear this out and add that the habit runs both ways. Across 27 runs the first-choice rate spans 0.109 (GPT-3.5 judging GPT-4’s film summaries, which took the second text nine times in ten) to 0.923 (Llama 3.1 70B judging Mixtral’s product text). The share of items decided by position runs from 0.08 to 0.84 with a median of 0.40, and in 18 of the 27 runs the model text is chosen at a different rate when listed first than when listed second (Fisher exact p < 0.05). Removing the position-decided items changes a cell’s share by 0.145 on average and by 0.424 at most (Figure 3). The GPT-3.5 film selector’s two runs of one cell on one day had first-choice rates of 0.11 and 0.49, the instability behind Section 4.3. So position bias can move any single cell, including a diagonal one, by a lot. Whether it can manufacture or hide a diagonal effect depends on whether it differs on the diagonal, and Table 2 says it does not. Llama 3.1 70B decides 72% of its own product descriptions by position against 26% of the others, which pulls its own cell down from 0.83 to 0.63 raw, a contrast of −0.195; on content-decided items the contrast is +0.019. Mix- tral shows the mirror image, +0.121 raw and +0.002 on content. GPT-4 on papers and on films is close to zero either way. Over the four rows the mean contrast is −0.026 raw and −0.025 on con- tent, and the share of position-decided items is 0.42 on the diagonal against 0.41 off it. Position bias is large, selector specific, and in these rows blind to whose text is being judged. Since every pair was shown in both orders, its net effect on a cell is compression toward 0.5, and the logit-scale result above is the corresponding check on the full matrices. 4.5 What the design could have seen Figure 4a gives the power of the exact test against the true premium. At γ = 0.05 the products matrix rejects in 82% of draws, papers in 88%, films in 42% and the pooled test in 97%; at γ = 0.10 all four reach at least 89%. The minimum detectable effect at 80% power is 0.045 on prod- ucts, 0.046 on papers, 0.085 on films and 0.034 pooled. The one-sided 95% upper bound on the pooled premium is 0.042. Read together with the discrete floor of p = 1/120: this design was well 5 0.30.40.50.60.70.80.91.0 LLM-win share, all valid trials (as published) 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 LLM-win share, order-consistent items only marker size: share of items decided by position selector judging another model's text selector judging its own model's text Fig. 3 LLM-win share with and without the items decided by position, for the 27 runs that record first-listed choices. Points on the diagonal line are unaffected by position; the vertical distance is the compression toward 0.5 that order habits impose. Own-model cells (diamonds) sit among the others. placed to see an own-model premium of five share points, and would almost surely have seen ten; it says nothing about three or less. For scale, the gap between LLM and human selectors on GPT- 4’s product text is 41 to 59 share points (0.77 to 0.95 against 0.36), and the generator effects span 0.18 to 0.26 share points within a matrix. An own- model effect, if one exists in this setting, is an order of magnitude smaller than the shared effect. Table 3 Power of the one-sided exact test at α = 0.05 for a future k× k study, one dataset, cell-level noise 0.037 and base share 0.7. Analytic value; the simulated value agrees within 0.05 in every cell shown. γ = 0.05γ = 0.10 k n = 200 n = 1,000 n = 200 n = 1,000 50.660.810.991.00 80.860.961.001.00 100.920.981.001.00 150.991.001.001.00 201.001.001.001.00 Table 3 and Figure 4b give the same calcu- lation for a future study. Because the cell-level noise does not shrink with trials, adding trials to a 5×5 design buys little: 3,000 trials per cell reach 85% power for γ = 0.05 where 200 reach 66%. Adding models buys more: eight models at 200 trials per cell give 86%, ten give 92%. A pre- mium of 0.10 is detectable with three models and any reasonable trial count. The standard error of γ is p (σ 2 int + p(1− p)/n)/(k− 1), and the first term dominates once n passes a few hundred. A study designed to settle the own-model question in this paradigm should therefore add generators and selectors, ideally several per vendor so that the vendor and self terms separate, and should record position per trial so the compression can be modelled, which beats removing it. 4.6 Tan et al. beside it The one other published matrix of this shape that we know of is Figure 6 of Tan et al. [6]: four read- ers (GPT-4, GPT-3.5, Llama 2 13B and 7B) each given a question with two contexts, one retrieved and one written by each of the four models as generator, and the cell is the bias toward the gen- erated context on NQ-AIR. Here γ = +0.148 with p = 0.042, which is 1/24, the smallest value 24 relabellings allow, and the self cell is the row max- imum for three generators of four. The authors report this as models preferring self-generated contexts, with Llama 2 7B the exception because it prefers 13B’s text. Two readings temper the num- ber. First, fitted with a family term, the self and same-family coefficients are +0.203 and +0.163: the same-family off-diagonal cells (mean 0.555) are elevated almost as much as the diagonal over the cross-family cells (0.393), which is a family effect as much as a self effect, on two families. Second, a reader’s own generated context restates the reader’s parametric belief, so “own text” and “agrees with what I already think” are one vari- able there, whereas in Laurito et al. the model text and the human text describe the same item from the same facts. The two matrices measure different things and do not conflict. 5 Related work Self-preference in judging has been measured many ways since 2024, and the picture is one of a real but small, heterogeneous and largely style-carried effect. Panickssery et al. tied self- preference to self-recognition in summarisation with three models [4]; Wataoka et al. showed that judges score low-perplexity text higher who- ever wrote it, which makes own-output preference 6 0.000.020.040.060.080.100.120.14 true own-model premium γ (share) 0.0 0.2 0.4 0.6 0.8 1.0 power, one-sided exact test, α = 0.05 80% (a) The published 5 x 5 designs Products (n = 180 to 220) Papers (n = 143 to 620) Movies (n = 358 to 500) Pooled, three datasets 2005001,0003,000 valid trials per cell 5 8 10 15 20 models k (k x k matrix) 0.660.770.810.85 0.860.930.960.97 0.920.970.980.99 0.991.001.001.00 1.001.001.001.00 (b) Power for γ = 0.05, one dataset Fig. 4 (a) Power of the one-sided exact test at α = 0.05 against the true own-model premium, simulated on each matrix’s additive fit with its estimated cell-level noise and its trial counts. (b) Analytic power for a premium of 0.05 in a k×k design with n valid trials per cell, one dataset, cell-level noise 0.037 as estimated here. a special case of familiarity [5]; Davidson et al. and Bai et al. found no consistent self-recognition across ten models [8, 9]. Quality-controlled esti- mates are smaller than raw ones: Chen et al. measure self-preference against gold judgments and find that rewriting both answers into one style cuts it by a third to a half [10]; Chen, Wei et al. find that most of the self-preference of strong judges on verifiable tasks is warranted, because their answers are right [11]; Roytburg et al. show that only about half of previously reported self- preference cases survive a null built from the judge’s own voting distribution [12]; Chae et al. find that it vanishes on three of four rubric dimen- sions once selection quality and judge severity are controlled, while a bare “self” label shifts scores on its own [13]; and Yang et al., across 20 judges on equal-quality pairs, find eight with a positive and nine with a negative self-preference [14]. Guey and Bougault, with a deterministic checker decid- ing validity, find no self-preference in four mid-tier families, with effects under 13 points not excluded [15]. Spiliopoulou et al. and Xu et al. report model- specific and heterogeneous self-bias, strongest for a few frontier judges [16, 17]. Family effects with the self cells removed are the closest analogue to Section 4.2. Awuni et al. cross four open-weight families with the judge never scoring its own output and report a same- family lift of 3.4 to 8.4 points after holding candidate family fixed [18]; Philipp et al. run nine judges over five generators on clinical items with physicians as anchor and find sibling lifts of 6.6 and 11.5 points inside the GPT and Gemma lineages [19]; Li et al. define inheritance and same-family relatedness and find judges favouring related models [20]; Pombal et al. carry a family variant of their rubric metric [21]. Those designs measure positive family effects of 3 to 12 points in their settings; the Laurito matrices, with one ven- dor pair and a negative coefficient, cannot confirm or deny an effect of that size. On the retrieval side, where the question is whether a reader prefers model-written passages and its own in particular, Chen et al. find self- preference under style-based prompts and none under fact-centric RAG prompts [22], and Ross et al. find that answer retention decides which rep- resentation a reader benefits from, with no family alignment [23]. Wu et al. learn per-engine prefer- ence rules for web content and find 79% to 84% of the rules shared across Gemini, GPT and Claude, with an engine-specific increment of 1% to 8% [24]. The present note fits that pattern: a large shared effect and no detectable model-specific one. 6 Limitations The re-analysis inherits the design. Five models, one pair from a single vendor, one prompt per role, one model text per item, and binary choices with some selectors returning many invalid answers: GPT-3.5 on films gave no valid choice in 108 to 142 trials of 500 in each of its five cells. The test is on cell shares, and the noise model behind the power figures treats cell-level interaction as Gaus- sian with a standard deviation estimated on 15 degrees of freedom per matrix; those power num- bers are indicative to perhaps 10 points either way. Position is recorded for 27 of the 75 cells’ worth 7 of runs, and the diagonal comparison in Table 2 rests on four rows. The same-vendor test has two cells per matrix. Nothing here speaks to whether a model can recognise its own text when asked, to judges scoring open-ended outputs, or to models released after 2024. The five models are GPT- 3.5-turbo, GPT-4-1106-preview, Llama 3.1 70B, Mixtral 8x22B and Qwen2.5 72B, and the finding is about them. 7 Conclusion Laurito et al. established that language models prefer language-model prose when a person would not, and that stands. Their matrices add one more fact. Decomposed into what a generator’s text does for every selector and what a selector does for every text, the residue on the diagonal is +0.019 share points pooled, with 17 of 120 relabellings doing as well, an upper bound near 0.04, and no dataset in which the generator’s own model is reli- ably its keenest selector. The same-vendor pair sits below its additive prediction in all three datasets, position bias moves cells by up to 0.42 and leaves the diagonal contrast where it was, and the design had 97% power to see a premium of 0.05. The preference these models show is a shared taste: GPT-4’s product descriptions are chosen 77% to 95% of the time by all five selectors, GPT-3.5’s 55% to 76%, and a person chooses them 36% and 29% of the time. A model preferring its own voice would be a different and narrower phenomenon, and one these data do not contain. Data availability. The package contains the extraction script, the per-cell and per-item counts it produced from commit 73475012 of the authors’ repository, the analysis and figure scripts, the number-check script that recomputes every figure in this note from the analysis output, and the reference-resolution log. No text from the source study is included. Competing interests. The author runs Rankfor.AI, a company that measures how AI engines describe brands and sells that measure- ment. Whether language models share one taste for text or each prefer their own bears on that business, and the reader should weigh the analy- sis with that in mind. The data are the original authors’ and every number here recomputes from their public counts. No funding was received for this work. Use of AI tools. The author used an AI assistant (Claude, Anthropic) for grammar and language editing, and to draw the figures from the analysis output by script. No data in this note were labelled or generated by AI tools. The author reviewed all of this output and takes full responsibility for the content. References [1] Laurito, W., Davis, B., Grietzer, P., Gavenˇciak, T., B ̈ohm, A., Kulveit, J.: AI–AI bias: Large language models favor communi- cations generated by large language models. Proceedings of the National Academy of Sciences 122(31), 2415697122 (2025) https://doi.org/10.1073/pnas.2415697122 . arXiv:2407.12856 [2] Yu, Z.: Addressing the broader implications of AI–AI bias in decision-making systems. Proceedings of the National Academy of Sci- ences 122(40), 2521127122 (2025) https://do i.org/10.1073/pnas.2521127122 [3] Laurito, W., Davis, B., Grietzer, P., Gavenˇciak, T., B ̈ohm, A., Kulveit, J.: Reply to Yu et al.: Datasets, human judges, and future directions for evaluating AI–AI bias. Proceedings of the National Academy of Sciences 122(40), 2522651122 (2025) https://doi.org/10.1073/pnas.2522651122 [4] Panickssery, A., Bowman, S.R., Feng, S.: LLM evaluators recognize and favor their own generations. In: Advances in Neural Informa- tion Processing Systems 37 (NeurIPS 2024) (2024). arXiv:2404.13076 [5] Wataoka,K.,Takahashi,T.,Ri,R.: Self-Preference Bias in LLM-as-a-Judge. arXiv:2410.21819;NeurIPS2024Safe Generative AI Workshop (2024) [6] Tan, H., Sun, F., Yang, W., Wang, Y., Cao, Q., Cheng, X.: Blinded by generated con- texts: How language models merge generated and retrieved contexts when knowledge con- flicts? In: Proceedings of the 62nd Annual Meeting of the Association for Computa- tional Linguistics (Volume 1: Long Papers), 8 p. 6207–6227 (2024). https://doi.org/10.1 8653/v1/2024.acl-long.337 [7] Laurito, W., Davis, B., Gavenˇciak, T.: ai- ai-bias: code and run outputs for “AI–AI bias”. GitHub repository, commit 73475012, https://github.com/lauritowal/ai- ai- bias. No licence file; accessed 18 September 2026 (2025) [8] Davidson, T.R., Surkov, V., Veselovsky, V., Russo, G., West, R., Gulcehre, C.: Self-RecognitioninLanguageModels. arXiv:2407.06946 (2024) [9] Bai, J., Shrivastava, A., Holtzman, A., Tan, C.: Know Thyself? On the Incapabil- ity and Implications of AI Self-Recognition. arXiv:2510.03399 (2025) [10] Chen, Z.-Y., Wang, H., Zhang, X., Hu, E., Lin, Y.: Beyond the surface: Measuring self- preference in LLM judgments. In: Proceed- ings of the 2025 Conference on Empirical Methods in Natural Language Processing (2025). https://doi.org/10.18653/v1/2025.e mnlp-main.86 [11] Chen, W.-L., Wei, Z., Zhu, X., Feng, S., Meng, Y.: Do LLM Evaluators Prefer Them- selves for a Reason? arXiv:2504.03846 (2025) [12] Roytburg, D., Bozoukov, M., Nguyen, M., Barzdukas, J., Puig-Hall, M., Oozeer, N.: Are LLM evaluators really narcissists? sanity checking self-preference evaluations. In: Pro- ceedings of the 43rd International Conference on Machine Learning (ICML 2026) (2026). arXiv:2601.22548 [13] Chae, S., Kim, M., Jung, D., Choi, S., Jung, S.: Self- and other-labels induce bidirectional bias in LLM judges. In: Findings of the Association for Computational Linguistics: EMNLP 2026 (2026). arXiv:2608.18091 [14] Yang, J., Hu, Z., Qiu, C., Deng, Z., Jiao, X., Zhou, T.: Quantifying and Mitigat- ing Self-Preference Bias of LLM Judges. arXiv:2604.22891 (2026) [15] Guey, W., Bougault, P.: Self-Preference IsWeakorAbsentinVerifiable Instruction-Following Revision: A Four- Model Test Under Genuine Authorship. arXiv:2606.20093 (2026) [16] Spiliopoulou, E., Fogliato, R., Burnsky, J., Soliman, T., Ma, J., Horwood, G., Balles- teros, M.: Play Favorites: A Statistical Method to Measure Self-Bias in LLM-as-a- Judge. arXiv:2508.06709 (2025) [17] Xu, J., Li, G., Jiang, J.: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights. arXiv:2509.00462 (2025) [18] Awuni, D.A., Achenie, L.E.K., Partey, B.T.,Owusu,E.G.,Sowah,N.-N.D.: Who Judges Matters: Measuring Family- Conditioned Preference in LLM-as-Judge Panels. arXiv:2609.17857 (2026) [19] Philipp, W., Fassbender, F., Fister, D., Langer, T., Pauly, M.G., Herzog, R., et al.: Clinician-Level Agreement Without Clinical Caution: LLM Evaluator Limits in Medical AI Benchmarking. arXiv:2607.01103 (2026) [20] Li, D., Sun, R., Huang, Y., Zhong, M., Jiang, B., Han, J., Zhang, X., Wang, W., Liu, H.: Preference leakage: A contamination problem in LLM-as-a-judge. In: International Con- ference on Learning Representations (ICLR 2026) (2026). arXiv:2502.01534 [21] Pombal, J., Rei, R., Martins, A.F.T.: Self- Preference Bias in Rubric-Based Evaluation of Large Language Models. arXiv:2604.06996 (2026) [22] Chen, Y.-S., Jin, J., Kuo, P.-T., Huang, C.- W., Chen, Y.-N.: LLMs are biased evaluators but not biased for retrieval augmented gen- eration. In: Findings of the Association for Computational Linguistics: ACL 2025 (2025). https://doi.org/10.18653/v1/2025.findings-a cl.1369 [23] Ross, S., Koopman, B., Vegt, A., Zuccon, G.: On the impact of retrieved content represen- tations in RAG pipelines. arXiv:2605.30790 (2026) 9 [24] Wu, Y., Zhong, X., Kim, J., Xiong, C.: What generative search engines like and how to optimize web content cooperatively. In: Inter- national Conference on Learning Representa- tions (ICLR 2026) (2026). arXiv:2510.11438 10