Paper deep dive
Before You Poll with LLMs: A Deliberative Diagnostic Framework
Ahmed Wali, Hassaan Tayyab
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Can LLMs reason through new information like humans, or do they merely retrieve cached opinions? This is critical for silicon sampling, where LLM personas simulate public opinion at scale. Current evaluations test only whether personas hold the right opinions -- a static snapshot. But opinion research increasingly depends on dynamic fidelity: whether personas update beliefs in response to new arguments, as humans do during deliberation. No existing benchmark tests this. We introduce the Deliberative Polling Diagnostic Framework, which compares human and LLM belief shifts after identical informational interventions. Grounded in deliberative polling, it surfaces failures invisible to static evaluation: models that produce plausible partisan opinions can still misrepresent how those opinions change. Applying the framework to five frontier models using data from America in One Room (526 personas, 72 questions), we find that every model fails, each in a unique manner. GPT-5.1 exhibits reversal: its personas become more hostile toward the opposing party after balanced information, while humans become less so. This reversal is selective (80% on outgroup vs. 26% on policy questions) and symmetric across partisan identities. Gemini 2.0 Flash, Claude Sonnet 4.5, and Llama 3.3 70B exhibit overshoot, shifting correctly but at 5-7x human magnitude. DeepSeek V3 exhibits rigidity with near-zero change. Targeted ablations reveal that policy content triggers these failures and that they are identity-specific: GPT-5.1 reverses on outgroup questions but overshoots on ingroup; Gemini shows the inverse. We term this signature self-sycophancy: conformity to the model's internal stereotype of the persona rather than reasoning from the information provided. Our framework offers a concrete protocol: run the deliberative diagnostic before trusting LLM personas to mimic revised beliefs.
Tags
Links
- Source: https://arxiv.org/abs/2609.15849v1
- Canonical: https://arxiv.org/abs/2609.15849v1
Trouble viewing inline? Open PDF directly →
Full Text
63,795 characters extracted from source content.
Expand or collapse full text
Before You Poll with LLMs: A Deliberative Diagnostic Framework Ahmed Wali Hassaan Tayyab Affiliation: Lahore University of Management Sciences Abstract Can LLMs reason through new information like humans, or do they merely retrieve cached opinions? This question is critical for the growing practice of silicon sampling, where LLM personas simulate public opinion at scale. Current evaluations test only whether these personas hold the right opinions — a static snapshot. But opinion research increasingly depends on dynamic fidelity: whether personas update beliefs in response to new arguments, as humans do during deliberation. No existing benchmark tests this capacity. We introduce the Deliberative Polling Diagnostic Framework, which fills this gap by comparing human and LLM belief shifts after identical informational interventions. Grounded in the methodology of deliberative polling, the framework surfaces failures invisible to static evaluation: models that produce plausible partisan opinions can still misrepresent how those opinions change. Applying the framework to five frontier models using data from America in One Room (526 personas, 72 questions), we observe that every model fails, and in a unique manner. GPT-5.1 exhibits reversal: its personas become more hostile toward the opposing party after balanced information, while humans become less so. This reversal is selective (80% on outgroup vs. 26% on policy questions) and symmetric across partisan identities. Gemini 2.0 Flash, Claude Sonnet 4.5, and Llama 3.3 70B exhibit overshoot, shifting correctly but at 5–7× human magnitude. DeepSeek V3 exhibits rigidity with near-zero change. Targeted ablations reveal that policy content triggers these failures and that they are identity-specific: GPT-5.1 reverses on outgroup questions but overshoots on ingroup; Gemini shows the inverse. We term this behavioral signature self-sycophancy: conformity to the model’s internal stereotype of the persona rather than reasoning from the information provided. Our framework offers practitioners a concrete protocol: run the deliberative diagnostic before trusting LLM personas to accurately mimic revised beliefs. Figure 1: Deliberative Polling Diagnostic and Key Findings. The framework (left) compares human and LLM responses to balanced information. Results reveal self-sycophancy (right), manifesting as outgroup hostility or ingroup favoritism, indicating stereotype retrieval rather than reasoning. 1 Introduction Consider a government agency contemplating a new taxation policy. Rather than convening citizen panels at considerable expense, it queries an LLM: assign 10,000 demographic personas, present them with balanced policy arguments, and measure how opinions shift. This is not merely a hypothetical. LLM-simulated deliberation has been proposed for democratic consultation (Jarrett et al., 2025), and the practice of querying LLM personas as stand-ins for survey respondents, termed silicon sampling (Argyle et al., 2023), is entering academic and commercial use. The appeal is clear: speed, scale, and cost. The risk is subtler. Current evaluations ask only a static question: does the persona hold the right opinion? Studies demonstrate moderate success (Argyle et al., 2023), but also persistent misalignment across demographic groups (Santurkar et al., 2023; Bisbee et al., 2024; Kang et al., 2025) and failures on motivated reasoning tasks (Pate et al., 2026). Yet every application involving deliberation or persuasion depends on a dynamic question no existing benchmark addresses: does the persona update beliefs akin to a human when confronted with new information? Consider two tasks: (1) What does a Republican believe about immigration? and (2) How does a Republican update that belief after reading balanced arguments? The first requires only declarative retrieval; the second requires procedural simulation, which involves computing how a reasoning agent integrates new information. An LLM can excel at the first and catastrophically fail the second. A model that produces plausible Republican opinions might, after balanced arguments, become more hostile toward Democrats, which is the opposite of what deliberative polling shows in humans (Fishkin et al., 2021). No static evaluation would catch this. We present the Deliberative Polling Diagnostic Framework, which tests this dynamic capacity directly. The framework repurposes deliberative polling methodology (Fishkin, 1991; Fishkin et al., 2021) (surveying participants before and after exposure to balanced information) as a diagnostic for LLM evaluation. By administering identical interventions to humans and LLM personas and comparing belief shifts, the framework detects failures invisible to static evaluation. Our work tackles two research questions: RQ1 What is the impact of deliberative polling on belief revision in LLMs compared to their human counterparts and does this vary by model, question type, and partisan identity? RQ2 What mechanisms drive the observed failures, are they triggered by policy content, are they selective to outgroup versus ingroup identity targets, and to what extent can they be attributed to stereotype-driven reasoning rather than alternative explanations? RQ1 establishes the phenomenon across five frontier models; RQ2 isolates its mechanism through targeted ablation experiments. Applying our diagnostic to data from America in One Room (526 personas, 72 questions), we find that no model matches human belief revision patterns. GPT-5.1 exhibits reversal, becoming more hostile toward the opposing party after balanced information while humans become less so. Gemini, Claude, and Llama exhibit overshoot at 5–7× human magnitude. DeepSeek exhibits rigidity. Targeted ablations reveal that these failures are content-triggered and identity-specific: the same persona produces opposite failure modes depending on whether it is asked about the opposing party or its own. We term this self-sycophancy: conformity to the model’s internal stereotype rather than reasoning from the provided information. Our contributions are: (1) the Deliberative Polling Diagnostic Framework for evaluating dynamic fidelity in LLM personas; (2) documentation that all five frontier models fail the diagnostic through distinct failure modes concentrated on identity-relevant content; and (3) behavioral evidence for self-sycophancy as a failure mode distinct from user-directed sycophancy (Sharma et al., 2024), characterized through content trigger and ingroup/outgroup ablation experiments. 2 Related Work 2.1 LLMs as Opinion Proxies Argyle et al. (2023) introduced silicon sampling, demonstrating moderate correlations between GPT-3 personas and human vote predictions. Subsequent work has exposed persistent limits: misalignment across demographic groups (Santurkar et al., 2023; Bisbee et al., 2024), superficial stereotype-driven opinion distributions rather than genuine belief network alignment (Chuang et al., 2024b), and shallow persona binding on partisan meta-perceptions (Kang et al., 2025). Critically, all prior work evaluates static fidelity; whether the model reproduces opinions at a single juncture. We evaluate dynamic fidelity: whether the model reproduces how those opinions change after new information. A model that passes static diagnostics may still fail ours, as we demonstrate with GPT-5.1, which reproduces pre-deliberation opinions while reversing the direction of post-deliberation belief revision. 2.2 Motivated Reasoning and Belief Revision Kunda (1990) established that directional goals bias how people evaluate evidence, yet the empirical record on backfire effects is far more equivocal than commonly assumed: Wood and Porter (2019) found essentially no backfire across 52 issues and 10,100 participants. Deliberative polling produces a parallel finding, balanced information reduces outgroup hostility (Fishkin et al., 2021), establishing the human baseline that GPT-5.1 violates. Concurrently, Pate et al. (2026) show that even base LLMs fail to replicate human motivated reasoning patterns, providing independent evidence that the procedural simulation failure we document may reflect a limitation beyond instruction-tuning. 2.3 Sycophancy in LLMs User-directed sycophancy (deference to explicit user feedback) is well-characterized (Sharma et al., 2024), and model outputs more generally can be modulated through activation steering (Turner et al., 2024). Our notion of self-sycophancy is structurally distinct: the deference target is the model’s own internal stereotype of the assigned persona, not an external user. The behavioral signature is selectivity: the same model conforms to outgroup questions while overshooting on ingroup questions within the same conversation, a pattern inconsistent with uniform training bias or general agreeability. 2.4 Stereotype, Bias, and Intergroup Contact Social identity theory (Tajfel and Turner, 1979) predicts that outgroup derogation and ingroup favoritism are core to intergroup cognition, which maps directly onto the two forms of self-sycophancy we identify. LLM stereotyping research shows that models encode these dynamics in amplified form (Kotek et al., 2023; Hofmann et al., 2024); our results extend this to a dynamic setting where stereotypes actively redirect belief revision. We ground our diagnostic in deliberative polling (Fishkin, 1991; Fishkin et al., 2021) and the contact hypothesis (Allport, 1954; Pettigrew and Tropp, 2006), which together establish that structured deliberation and intergroup contact reliably reduce intergroup hostility, providing a well-characterized human baseline against which we measure LLM fidelity. The affective polarization literature (Iyengar and Westwood, 2015; Iyengar et al., 2019) confirms that partisan hostility operates through social identity rather than policy disagreement, making it precisely the kind of attitude deliberation moderates. Complementary work on multi-agent LLM simulations (Taubenfeld et al., 2024; Chuang et al., 2024a) documents systematic distortions in simulated opinion dynamics, providing independent evidence that LLMs struggle with belief revision in social contexts. 3 The Deliberative Polling Diagnostic Framework 3.1 Framework Design The framework compares human and LLM belief revision after identical informational interventions, testing whether models that reproduce plausible initial opinions can also reproduce how those opinions change. The procedure consists of four phases: (1) pre-poll, in which questions are administered to humans and LLM personas under matched conditions; (2) briefing, in which identical balanced information is provided to both; (3) post-poll, in which the same questions are re-administered; and (4) comparison, in which belief shifts Δ=Post−Pre =Post-Pre are computed and human–LLM patterns are compared. We classify LLM behavior on each question into four categories: match (same direction, comparable magnitude), overshoot (same direction, exaggerated magnitude), reversal (opposite direction from humans), and rigidity (near-zero change despite information exposure). Figure 1 illustrates the diagnostic pipeline and previews the key empirical finding: GPT-5.1 reverses on outgroup questions where humans moderate. 3.2 Why Deliberative Polling? Deliberative polling combines three properties that make it uniquely suited as a diagnostic for dynamic fidelity. First, it provides ground-truth belief revision with known directionality: decades of research confirm that balanced deliberation reduces intergroup hostility (Fishkin et al., 2021), allowing us to distinguish models that fail by reversing from those that fail by overshooting; a distinction invisible in undirected opinion-change studies. Second, the intervention is fully standardized: briefing materials are textual and self-contained, eliminating confounds from interaction dynamics or social pressure. Third, the participant pool is demographically rich: each A1R participant has 13 demographic attributes, enabling matched LLM persona construction and differential fidelity testing across partisan groups and question types. Our primary diagnostic uses the briefing-based protocol (RQ1 and RQ2). We also conduct exploratory analysis of multi-agent deliberation (§7), but scope the main framework to the informational channel where the intervention is maximally controlled. 4 Experimental Setup 4.1 Human Baseline: America in One Room America in One Room (A1R) convened 526 registered voters for structured deliberation in 2019 (Fishkin et al., 2021). Participants were recruited by NORC at the University of Chicago using address-based probability sampling. They completed identical surveys before and after receiving balanced briefing materials covering five policy domains: immigration, healthcare, the economy, the environment, and foreign policy. Each participant profile includes 13 demographic attributes: party affiliation, gender, age, race, education, income, state, region, metropolitan status, housing, marital status, employment, and LGBT identification. The complete dataset is publicly available through Harvard Dataverse under a C0 license; our response data and experiment registry are released at https://doi.org/10.5281/zenodo.22184355. 4.2 Models and Scale We evaluate five frontier instruction-tuned models across 21 experimental conditions (full registry in Appendix B). GPT-5.1 serves as the primary evaluation target at full scale (526 personas × 72 questions). Gemini 2.0 Flash is tested at full scale on a 20-question subset for mechanistic ablations (§6). Claude Sonnet 4.5, Llama 3.3 70B, and DeepSeek V3 are tested on representative subsamples (100 personas × 20 questions) for cross-model generalization. All models are queried at temperature = 0 with outputs restricted to integer responses on the 0–10 scale. Prompt templates are provided in Appendix A. We scope our evaluation to instruction-tuned models, the variants deployed in practice for opinion simulation; recent work suggests base models exhibit parallel failures (Pate et al., 2026; Kang et al., 2025). We administer the pre- and post-deliberation questions independently, that is, the model is never shown its own prior response before answering post-briefing. This mirrors the A1R protocol, in which participants completed the post-deliberation survey without seeing their earlier answers. Because of this, Δ=Post−Pre =Post-Pre is computed under matched conditions on both sides of the comparison, which is what the diagnostic rests on. A conditioned design, in which the model is shown its prior answer and asked whether it revises, measures a different capacity. It would break the human-LLM comparability that motivates our setup, since no corresponding anchor exists in the human data. We therefore treat conditioned belief revision as a complementary evaluation axis and leave it to future work. The multi-agent condition (§7) partially approximates it by injecting peer responses between the briefing and the question, and yields results consistent with the main findings (Table 5). 4.3 Question Categories The A1R instrument contains 107 items. We exclude 35 that lack valid pre/post pairs or do not measure substantive opinion: political knowledge items (7), post-only items without a pre-deliberation baseline (8), process evaluation items (10), party identification and ideological placement measures that would be circular for persona construction (7), and thermometer/framing items (3). The remaining 72 questions span five categories: policy (47 items) covering substantive positions across all five domains; outgroup (5 items) measuring perceptions of the opposing party; factual (9 items) on empirical claims related to the policy domains; values (7 items) on abstract principles; and efficacy (4 items) on political self-efficacy. The distinction between policy and outgroup categories proves empirically central: as shown in §5.2, GPT-5.1’s failure concentrates overwhelmingly on outgroup questions. 4.4 Outgroup Question Format The five outgroup items asked participants to rate agreement with statements about those people where the referent was contextually understood as the opposing political party. The statements target meta-perceptions of the outgroup’s competence, honesty, and reasoning quality (full item text and per-question polarity analysis in Appendix C). Four items express negative attributions (don’t know enough, not thinking clearly) so higher scores indicate greater hostility and a positive Δ means increasing hostility after deliberation. The human baseline (Δ=−0.21 =-0.21) confirms that humans moderate, consistent with the A1R deliberative finding. We replicate this format exactly for LLM personas using the implicit those people phrasing, with explicit party-labeling validation experiments described in §6.2. 4.5 Measurement and Statistical Methods All questions use a 0–10 Likert scale administered identically to humans and LLM personas. Our primary quantity is the belief shift aggregated to the group level (by model, partisan group or question category). To classify LLM behavior, we compute the magnitude ratio, which captures both direction and proportionality. For human–LLM comparisons, we fit mixed-effects models with crossed random effects for persona and question, reporting fixed-effect coefficients with 95% confidence intervals and for reversal rate comparisons across question categories, we deploy Fisher’s exact test. Full statistical model specification appears in Appendix C. One comparison (Gemini on policy questions) produced a singular fit under this specification and is reported using an unpaired fallback, flagged as M3 in Table 11. 5 RQ1: Impact of LLMs on Belief Revision 5.1 Aggregate Results: Every Model Fails Differently Table 1 presents the diagnostic results on outgroup questions across all five models. The central finding is that no model matches human belief revision, and each fails in a distinct way (GPT-5.1 and Gemini at full scale, the remaining three on 100-persona subsamples; see §4.2). Model Outgroup Δ 95% CI d Human (A1R) −0.21-0.21 [-0.332, -0.093] - GPT-5.1 +0.50+0.50 [+0.413, +0.587] 0.27 Gemini 2.0 −1.36-1.36 [-1.426, -1.288] 0.48 Claude 4.5 −1.04-1.04 [-1.187, -0.887] 0.33 Llama 70B −1.42-1.42 [-1.556, -1.292] 0.53 DeepSeek V3 +0.02+0.02 [-0.011, +0.047] 0.09 Table 1: Multi-model diagnostic results on outgroup questions. 95% CIs and Cohen’s d from mixed-effects models with crossed random effects for CASEID and question (Model 2). All model–human comparisons survive Bonferroni correction (α=0.0038α=0.0038) except DeepSeek (p=0.051p=0.051). Non-response codes and invalid outputs excluded; N=523N=523 matched personas. GPT-5.1 and Gemini were run on 526 personas, the other three on 100-persona subsamples (§4.2); cross-model comparisons should be read accordingly. GPT-5.1 exhibits reversal: after balanced information, its personas become more hostile toward the opposing party while humans become less hostile; a systematic backfire effect that is narrow and contested in human populations (Wood and Porter, 2019) but robust in this model. Gemini, Claude, and Llama exhibit overshoot: correct direction but 5–7× human magnitude, encoding an excessively optimistic theory of persuadability. Overshoot magnitudes warrant more caution than the directional findings, since the 13 attributes bound how well personas capture the heterogeneity governing human persuadability; reversal, rigidity, and selectivity do not depend on magnitude calibration. DeepSeek exhibits rigidity: near-zero belief revision and indicates non-formation of opinion (σ≈0σ≈ 0). That each model fails differently is itself significant. Uniform failure would implicate architecture; the diversity of failures instead implicates training-specific factors such as RLHF reward models, data composition, and alignment procedures. All models also produce substantially less response variance than humans, consistent with stereotype retrieval producing a narrower distribution than individual reasoning. Per-persona analysis (Appendix C) confirms that reversal is broad-based. 73.5% of personas reverse on at least one outgroup question, and 42.2% reverse on a majority. The mean per-persona reversal rate is 52.6%. This distribution rules out a subgroup-driven explanation. The resulting mean pairwise ϕ≈φ≈ 0.08 indicates weak cross-question consistency: whether a persona reverses on one outgroup question is only marginally predictive of whether it reverses on another. 5.2 Selectivity: Failures Concentrate on Identity-Relevant Content GPT-5.1’s reversal is not uniform across question types. Table 2 shows reversal rates by category. Category N Reversal Rate Outgroup 5 80% (4/5) Policy 47 26% (12/47) Table 2: GPT-5.1 reversal rates by question category on 526 personas. Fisher’s exact test yields OROR = 11.67, p = 0.027. We note that this does not survive Bonferroni correction; the selectivity claim rests on three convergent experiments rather than this test alone: in §6.2, we directly manipulate question targeting and observe opposite failure modes in the same model (p<0.0001p<0.0001), providing convergent evidence. The pattern suggests that policy questions can be answered through declarative retrieval while outgroup questions require procedural simulation of how information exposure affects in-group attitudes. When procedural simulation is required, the model defaults to stereotype retrieval. 5.3 Symmetry: All Partisan Groups Reverse Party N Human Δ GPT Δ Reversed Democrat 194 −0.07-0.07 +0.37+0.37 Yes Republican 132 −0.37-0.37 +0.41+0.41 Yes Independent 200 −0.26-0.26 +0.69+0.69 Yes Table 3: GPT-5.1 reversal by partisan group. p<0.001p<0.001 for all groups. If reversal reflected one-sided training bias, we would expect asymmetric failure. Table 3 shows the opposite: all three partisan groups reverse. The symmetry reveals an implicit theory encoded in GPT-5.1 applied uniformly regardless of which party the persona belongs to. This mirrors the structure of affective polarization (Iyengar and Westwood, 2015) but contradicts the deliberative polling finding that balanced information reduces hostility (Fishkin et al., 2021). Figure 2: Policy content triggers divergence, and with irrelevant briefings, both models produce identical behavior 6 RQ2: Mechanisms Driving Failure §5 establishes what failures occur; we now investigate why through three targeted experiments that progressively isolate the mechanism, converging on a single behavioral account: self-sycophancy (§6.4). 6.1 Content Trigger Experiment If reversal reflects stereotype retrieval triggered by policy content, removing that content should eliminate it. We replaced balanced policy briefings with Wikipedia articles on irrelevant topics (pasta history, gardening, chess) while maintaining identical prompt structure and question wording for 526 personas on 20 questions. With irrelevant briefings, the divergence disappears: both models produce Δ=−0.75 =-0.75, identical overshoot (Table 4, Figure 2). Policy content specifically triggers GPT-5.1’s reversal; without it, the model defaults to the same overshoot pattern as Gemini. This rules out the hypothesis that reversal is a general property of GPT-5.1’s architecture wherein the model can produce moderation, it simply does not when presented with politically relevant material. Model Policy Briefing Irrelevant Briefing GPT-5.1 +0.50+0.50 (Reversal) −0.75-0.75 (Overshoot) Gemini −1.36-1.36 (Overshoot) −0.75-0.75 (Overshoot) Table 4: Content trigger results. 6.2 Outgroup vs. Ingroup Control If GPT-5.1 encodes an outgroup-specific stereotype, reversal should occur only when personas are asked about the opposing party. We modified the five outgroup questions to explicitly name the target party (“Democrats just don’t know enough” for Republican personas), with reversed mapping for Democrats. Independents were excluded (n=200n=200), yielding N=326N=326. The same model, persona, and briefing material produce opposite failure modes depending solely on the identity target. GPT-5.1 reverses on outgroup (Δ=+0.24 =+0.24) but overshoots on ingroup (Δ=−1.09 =-1.09); the difference is large (t=12.29t=12.29, d=0.43d=0.43). Gemini shows the inverse: near-human on outgroup (Δ=−0.34 =-0.34) but reversing on ingroup (Δ=+0.42 =+0.42, t=−12.60t=-12.60, d=−0.44d=-0.44). GPT-5.1 conforms to a stereotype of outgroup hostility; Gemini conforms to a stereotype of ingroup favoritism. Different stereotypes, same mechanism: cached partisan attitudes overriding reasoning from the briefing. This cross-model symmetry is the strongest evidence that we observe a family of stereotype-driven failures rather than a model-specific defect. 6.3 Ruling Out Alternative Explanations We consider six alternatives. Briefing format: policy questions with the same briefing show only 26% reversal vs. 80% for outgroup (§5.2). Mode collapse: re-running Gemini at T=0.7T=0.7 preserves the direction and structure of the failure; restricted to the n=2,373n=2,373 persona–question cells valid in both conditions, overshoot persists and slightly strengthens (T=0T=0: Δ=−1.36 =-1.36; T=0.7T=0.7: Δ=−1.55 =-1.55), with a paired-observation correlation of r=0.77r=0.77. Partisan training bias: reversal is symmetric across all three groups (§5.3). Persona construction artifact: the same personas produce opposite failure modes depending on question targeting (§6.2). Demographic confound: GPT-5.1’s outgroup Δ remains positive within every stratum of ten of eleven non-party demographic attributes, the exception being two small marital-status strata (Appendix C, Table 13). Label-swap: swapping party labels while holding the other twelve demographic attributes constant produces Δ shifts in which both the assigned label and demographics influence belief revision (Appendix C, Table 12). Three of four cell comparisons reach significance. The label effect is asymmetric across demographic groups: within Democrat-demographic personas, changing the label shifts Δ by 0.380.38 points, while within Republican-demographic personas the corresponding shift is only 0.130.13 points. The demographic effect is consistent across labels (0.230.23 and 0.280.28 points). Label is therefore a strong retrieval cue for Democrat-demographic personas specifically, with demographics retaining a parallel influence throughout. A note on pre-poll asymmetries. Table 8 shows that LLM pre-poll means diverge from human pre-poll means before any briefing is shown. On the four negatively-coded outgroup items, GPT-5.1 starts 1.2 points lower than humans on Q16A (4.75 vs. 5.97) and 3.2 points lower on Q16B (4.47 vs. 7.67), while Gemini starts higher on Q16A (6.22) and lower on Q16B (5.73). These pre-poll divergences could indicate that the persona token plus question alone activates a partisan stereotype baseline that differs from the human distribution even without informational input. This is consistent with prior findings on demographic-only persona construction (Santurkar et al., 2023; Chuang et al., 2024b; Kang et al., 2025). Importantly, our diagnostic is robust to this. We measure the within-persona briefing-conditional shift Δ=Post−Pre =Post-Pre, which absorbs each persona’s baseline and isolates the effect of the briefing on belief revision. Self-sycophancy as we define it concerns this conditional shift (dynamic fidelity), distinct from whether the persona reproduces human starting opinions (pre-poll fidelity). The content-trigger experiment (§6.1) confirms the framing: when policy briefings are replaced with irrelevant content, the divergence disappears even though persona priming and pre-poll asymmetry remain unchanged. We note that behavioral evidence alone cannot fully distinguish self-sycophancy from learned conditional associations in training data that produce the same signature. We characterize this remaining ambiguity in §9. 6.4 Self-Sycophancy: The Best-Supported Behavioral Account The preceding experiments converge on a single account, which we advance as the best-supported behavioral explanation of the observed pattern. We define self-sycophancy as conformity to the model’s internal stereotype of the assigned persona rather than reasoning from provided information, paralleling user-directed sycophancy (Sharma et al., 2024) but with a structural distinction: the deference target is the model’s own cached representation, not an external user. Self-sycophancy manifests in two forms corresponding to social identity theory’s (Tajfel and Turner, 1979) predicted dimensions of intergroup bias. GPT-5.1 exhibits outgroup hostility sycophancy: conforming to the stereotype that partisans are hostile toward the opposing party. Gemini exhibits ingroup favoritism sycophancy: conforming to the stereotype that partisans favor their own party. Three features distinguish self-sycophancy from user-directed sycophancy: (1) the trigger is internal, not external; (2) the deference target is the model’s persona representation, not user preferences; and (3) it is selective since the same model produces opposite failure modes on outgroup vs. ingroup questions within the same conversation. These findings expose a declarative-procedural gap: LLMs retrieve plausible partisan opinions (declarative) but cannot simulate how those opinions update in response to new information (procedural). We stress the epistemological status of this account. Our evidence is behavioral, and cannot distinguish a structurally distinct retrieval process from stereotype-driven heuristics implemented as learned conditional associations in training data. Both classes of explanation involve cached identity-conditional associations overriding the briefing content, which is what self-sycophancy names at the behavioral level; adjudicating between them requires interpretability methods we do not employ here. The empirical signature is the selectivity in §5.2: policy questions (26% reversal) are answerable declaratively, while outgroup questions (80% reversal) require procedural simulation that the model cannot perform, defaulting instead to stereotype retrieval. 7 Exploratory Analysis: Can Interventions Mitigate Self-Sycophancy? Having established self-sycophancy as a systematic failure mode, we briefly examine whether common interventions can mitigate it. Table 5 summarizes results across four interventions; we emphasize these analyses are exploratory. Multi-agent deliberation reduces GPT-5.1’s reversal rate (80% → 20%) but amplifies Gemini’s overshoot (6.5× → 18×). Chain-of-thought and persona-first ordering both eliminate Gemini’s reversal but leave overshoot intact at ∼ 5.7×. Label removal produces the most balanced improvement (reversal 40% → 20%, overshoot 6.5× → 4.3×) but the model still substantially overshoots. The central finding is that no intervention simultaneously corrects direction and calibrates magnitude, and the same intervention can help one model while hurting another. This method–model interaction means that practitioners cannot choose a prompting strategy without first diagnosing which failure mode their model exhibits, further reinforcing the practical value of the diagnostic framework. Intervention Model Reversal Effect Magnitude Effect Baseline GPT-5.1 80% +0.50 (Reversal) Multi-Agent GPT-5.1 80% → 20% +0.50 → +0.17 Baseline Gemini 40% -1.36 (6.5×) Multi-Agent Gemini 40% → 20% 6.5×→×→ 18× Chain-of-Thought Gemini 40% → 0% 5.7× overshoot Label removal Gemini 40% → 20% 6.5×→×→ 4.3× Persona-first Gemini 40% → 0% 5.6× overshoot Table 5: Exploratory intervention results. No intervention simultaneously eliminates reversal and calibrates magnitude. 8 Implications 8.1 Validation Protocol Our framework provides a four-step protocol for any application relying on LLM personas to simulate opinion change: (1) construct a calibration sample from a small deliberative poll in the target domain; (2) simulate matched LLM personas with identical briefings; (3) apply the diagnostic taxonomy: if the model reverses on identity-relevant questions it should not be deployed, if it overshoots substantially, apply calibration scaling estimated from the sample; (4) test for selectivity, since a model matching humans on policy questions but reversing on outgroup questions would pass aggregate metrics while producing systematically misleading results on precisely the questions that matter most. The protocol is domain-agnostic: the same steps apply to political deliberation, healthcare, or consumer research. 8.2 Model Selection Each failure mode produces a distinct policy distortion. GPT-5.1 would lead practitioners to conclude that deliberation increases hostility, thereby recommending against public deliberation based on model artifacts. Gemini would inflate effect sizes by 5–7× overstating the power of informational interventions. DeepSeek would suggest deliberation has no effect, undermining the evidence base for public consultation. 8.3 LLM Cognition Self-sycophancy represents a form of misalignment: stereotype retrieval satisfies the surface objective (produce partisan-like outputs) while failing the underlying goal (simulate partisan reasoning). The finding that persona-first ordering eliminates reversal (§7) is suggestive since when the persona precedes the briefing, the model may process information through the persona’s perspective rather than applying the persona as a post-hoc retrieval cue, paralleling primacy effects in human cognition (Asch, 1946). A broader concern is self-reinforcement: if self-sycophantic outputs are used as synthetic training data, the stereotypes they encode may become self-perpetuating, amplifying polarization beyond its prevalence in human populations. We do not attempt to explain why different models encode different stereotypes; the asymmetry itself demonstrates that self-sycophancy is a family of biases whose content varies across models. 9 Conclusion LLMs know what groups believe but not how beliefs shift. Our framework tests a capacity no existing benchmark evaluates: whether LLM personas revise beliefs like humans after encountering new information. Across five frontier models, we observed failure through distinct modes concentrated on identity-relevant content. The best-supported account of these failures, self-sycophancy, describes not a single bias but a family: GPT-5.1 retrieves outgroup hostility, Gemini retrieves ingroup favoritism, and both override the capacity to reason from new information. These failures are invisible to static evaluation wherein a model producing plausible pre-deliberation opinions can still produce post-deliberation shifts that are directionally wrong, magnitude-inflated, or entirely absent. So before you poll with LLMs, run the Deliberative Diagnostic Framework. Limitations Several limitations constrain our findings and suggest directions for future research. (1) We evaluate a single deliberative poll (America in One Room, 2019, US politics); cross-cultural replication using deliberative polls from other countries and non-political domains would better test the generalizability of self-sycophancy beyond American partisan identity. (2) We test only instruction-tuned models; concurrent work shows base models exhibit parallel failures (Pate et al., 2026; Kang et al., 2025), but confirming whether self-sycophancy manifests pre-RLHF would clarify its origins. (3) Our evidence is behavioral: distinguishing self-sycophancy from learned conditional associations in training data requires interpretability methods (probing classifiers or activation patching), which we leave to future work. (4) Prompt sensitivity is not exhaustively explored; systematic investigation of persona construction strategies, including belief-network conditioning (Chuang et al., 2024b), may reveal whether richer persona representations mitigate stereotype retrieval. (5) The ingroup–outgroup control excludes Independents. (6) The A1R data was collected in 2019; longitudinal replication with updated baselines would assess whether model failures track shifts in human polarization dynamics. (7) Our diagnostic isolates dynamic fidelity by measuring the within-persona briefing conditional shift Δ=Post−Pre =Post-Pre, which is robust to baseline differences between LLM and human pre-poll responses (§6.3). However, pre-poll fidelity itself, whether LLM personas reproduce human starting opinions, is a separable axis we do not directly evaluate. Methods for joint calibration of pre-poll fidelity and dynamic fidelity, including belief-network conditioning (Chuang et al., 2024b), are an important direction for future work. (8) Our stratification and label-swap analyses constrain attribution to partisan identity as the operative variable, while allowing that other demographic attributes may modulate how failure modes manifest; marital status is the one attribute containing strata that track the human direction (Appendix C, Table 13). Disentangling that modulation is left to future work. (9) We do not establish a floor level of output variability against which the observed shifts can be scaled; a repeated-prompting baseline at T=0T=0 would provide this directly, and our temperature ablation (§6.3) offers only indirect evidence that the failure modes are not sampling artifacts. (10) We query proprietary APIs, which are subject to provider-side moderation and post-processing that are not transparent to end users. Our results therefore characterize these systems as deployed rather than their underlying weights, and cross-provider comparisons inherit this ambiguity. That the five models fail in three distinct directions (Table 1) is difficult to attribute to provider-side filtering alone, but we cannot exclude a contribution, and disentangling model behavior from deployment layer would require access to unmoderated endpoints. We note that practitioners using these APIs for opinion simulation are necessarily querying the deployed system. (11) The human and LLM trajectories share survey items and briefing content but do not arise from equivalent treatments. A1R participants experienced a multicomponent deliberative process that included moderated small-group discussion and plenary sessions with competing experts, whereas our primary LLM condition isolates the informational component. Divergence may therefore reflect treatment context as well as model behavior. The multi-agent condition (§7) partially approximates this channel. Ethics Statement This research uses publicly available data from America in One Room. No human subjects were recruited. We caution against deploying LLMs for opinion simulation without validation, as our findings show systematic biases that could mislead policy decisions. Acknowledgments We are deeply grateful to Dr. Ihsan Ayyub Qazi, whose supervision shaped this work from its earliest stages; his guidance and generosity with his time were invaluable throughout. We also thank our colleagues Hamza Sherjeel, Saleh Farooq, Abdullah Ghani and Abdul Rafay Bajwa at IDSL for their contributions and support throughout this work. References Allport (1954) G. W. Allport The nature of prejudice. Addison-Wesley, Cambridge, MA. Cited by: §2.4. Argyle et al. (2023) L. P. Argyle, E. C. Busby, N. Fulda, J. R. Gubler, C. Rytting, and D. Wingate Out of one, many: Using language models to simulate human samples. Political Analysis 31 (3), p. 337–351. External Links: Document Cited by: §1, §2.1. Asch (1946) S. E. Asch Forming impressions of personality. The Journal of Abnormal and Social Psychology 41 (3), p. 258–290. External Links: Document Cited by: §8.3. Bisbee et al. (2024) J. Bisbee, J. D. Clinton, C. Dorff, B. Kenkel, and J. M. Larson Synthetic replacements for human survey data? The perils of large language models. Political Analysis 32 (4), p. 401–416. External Links: Document Cited by: §1, §2.1. Chuang et al. (2024a) Y. Chuang, A. Goyal, N. Harlalka, S. Suresh, R. Hawkins, S. Yang, D. Shah, J. Hu, and T. T. Rogers Simulating opinion dynamics with networks of LLM-based agents. In Findings of the Association for Computational Linguistics: NAACL 2024, Mexico City, Mexico, p. 3326–3346. External Links: Document, Link Cited by: §2.4. Chuang et al. (2024b) Y. Chuang, K. Nirunwiroj, Z. Studdiford, A. Goyal, V. V. Frigo, S. Yang, D. V. Shah, J. Hu, and T. T. Rogers Beyond demographics: Aligning role-playing LLM-based agents using human belief networks. In Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, p. 14010–14026. External Links: Document, Link Cited by: §2.1, §6.3, Limitations. Fishkin et al. (2021) J. S. Fishkin, A. Siu, L. Diamond, and N. Bradburn Is deliberation an antidote to extreme partisan polarization? Reflections on “America in One Room”. American Political Science Review 115 (4), p. 1464–1481. External Links: Document Cited by: §1, §2.2, §2.4, §3.2, §4.1, §5.3. Fishkin (1991) J. S. Fishkin Democracy and deliberation: new directions for democratic reform. Yale University Press, New Haven, CT. Cited by: §1, §2.4. Hofmann et al. (2024) V. Hofmann, P. R. Kalluri, D. Jurafsky, and S. King AI generates covertly racist decisions about people based on their dialect. Nature 633 (8028), p. 147–154. External Links: Document Cited by: §2.4. Iyengar et al. (2019) S. Iyengar, Y. Lelkes, M. Levendusky, N. Malhotra, and S. J. Westwood The origins and consequences of affective polarization in the United States. Annual Review of Political Science 22, p. 129–146. External Links: Document Cited by: §2.4. Iyengar and Westwood (2015) S. Iyengar and S. J. Westwood Fear and loathing across party lines: New evidence on group polarization. American Journal of Political Science 59 (3), p. 690–707. External Links: Document Cited by: §2.4, §5.3. Jarrett et al. (2025) D. Jarrett, M. Pîslar, M. A. Bakker, M. H. Tessler, R. Köster, J. Balaguer, R. Elie, C. Summerfield, and A. Tacchetti Language agents as digital representatives in collective decision-making. External Links: 2502.09369, Document, Link Cited by: §1. Kang et al. (2025) M. Kang, S. Moon, S. H. Lee, A. Raj, J. Suh, and D. Chan Deep binding of language model virtual personas: a study on approximating political partisan misperceptions. In Second Conference on Language Modeling, External Links: Link Cited by: §1, §2.1, §4.2, §6.3, Limitations. Kotek et al. (2023) H. Kotek, R. Dockum, and D. Q. Sun Gender bias and stereotypes in large language models. In Proceedings of the ACM Collective Intelligence Conference (CI ’23), Delft, Netherlands, p. 12–24. External Links: Document Cited by: §2.4. Kunda (1990) Z. Kunda The case for motivated reasoning. Psychological Bulletin 108 (3), p. 480–498. External Links: Document Cited by: §2.2. Pate et al. (2026) N. Pate, A. M. Proma, H. He, J. N. Druckman, D. C. Molden, G. Ghoshal, and E. Hoque Replicating human motivated reasoning studies with LLMs. External Links: 2601.16130, Document, Link Cited by: §1, §2.2, §4.2, Limitations. Pettigrew and Tropp (2006) T. F. Pettigrew and L. R. Tropp A meta-analytic test of intergroup contact theory. Journal of Personality and Social Psychology 90 (5), p. 751–783. External Links: Document Cited by: §2.4. Santurkar et al. (2023) S. Santurkar, E. Durmus, F. Ladhak, C. Lee, P. Liang, and T. Hashimoto Whose opinions do language models reflect?. In Proceedings of the 40th International Conference on Machine Learning (ICML), Proceedings of Machine Learning Research, Vol. 202, p. 29971–30004. External Links: Link Cited by: §1, §2.1, §6.3. Sharma et al. (2024) M. Sharma, M. Tong, T. Korbak, D. Duvenaud, A. Askell, S. R. Bowman, E. Durmus, Z. Hatfield-Dodds, S. R. Johnston, S. Kravec, T. Maxwell, S. McCandlish, K. Ndousse, O. Rausch, N. Schiefer, D. Yan, M. Zhang, and E. Perez Towards understanding sycophancy in language models. In Proceedings of the Twelfth International Conference on Learning Representations (ICLR), External Links: Link Cited by: §1, §2.3, §6.4. Tajfel and Turner (1979) H. Tajfel and J. C. Turner An integrative theory of intergroup conflict. In The Social Psychology of Intergroup Relations, W. G. Austin and S. Worchel (Eds.), p. 33–47. Cited by: §2.4, §6.4. Taubenfeld et al. (2024) A. Taubenfeld, Y. Dover, R. Reichart, and A. Goldstein Systematic biases in LLM simulations of debates. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Miami, Florida, USA, p. 251–267. External Links: Document, Link Cited by: §2.4. Turner et al. (2024) A. M. Turner, L. Thiergart, G. Leech, D. Udell, J. J. Vazquez, U. Mini, and M. MacDiarmid Steering language models with activation engineering. External Links: 2308.10248, Document, Link Cited by: §2.3. Wood and Porter (2019) T. Wood and E. Porter The elusive backfire effect: Mass attitudes’ steadfast factual adherence. Political Behavior 41, p. 135–163. External Links: Document Cited by: §2.2, §5.1. Appendix A Prompt Templates Exact prompt templates for all experimental conditions. A. Standard Pre-Poll Prompt Question: [Question Text] [Scale Instruction] (e.g., "Scale: 0 = Strongly oppose, 10 = Strongly favor") You are a [Party]. You are a [Age] year old [Race] [Gender] with [Education] education and [Income] household income. You live in [Metro] in [State], United States ([Region] Region). You are [Marital Status], [Employment], [Housing] housing, and identify as [LGBT]. Your response (ONLY THE NUMBER): B. Standard Post-Poll Prompt INFORMATIONAL MATERIALS [Briefing Content] (Domain-specific policy arguments OR All-themes summary) END MATERIALS Question: [Question Text] [Scale Instruction] [Same Persona Description as Pre-Poll] Your response (ONLY THE NUMBER): Figure 3: Main Diagnostic Prompts. The standard templates used for all 526 personas in the primary evaluation. C. Content Trigger Experiment (Irrelevant Content) Replaces the [Briefing Content] block in the Post-Poll Prompt. INFORMATIONAL MATERIALS [Wikipedia Article Text] (e.g., History of Pasta, Rules of Chess, Gardening Tips) END MATERIALS ... [Rest of Prompt Unchanged] D. Ingroup/Outgroup Control (Explicit Targeting) Modifies the [Question Text] to explicitly name the target group. Original (Implicit): "How well does this describe ’those people’? ’They just don’t know enough.’" Explicit Outgroup (e.g., for Republican Persona): "How well does this describe Democrats? ’Democrats just don’t know enough.’" Explicit Ingroup (e.g., for Republican Persona): "How well does this describe Republicans? ’Republicans just don’t know enough.’" E. Party Label Ablation (No-Party Condition) Modifies the Persona Description by removing the first sentence. Standard: You are a Republican. You are a 45 year old White Male... No-Party: You are a 45 year old White Male... Figure 4: Mechanistic Variation Prompts. Modifications used to test the Content Trigger (Section 6.1) and Self-Sycophancy (Section 6.2) hypotheses. F. Multi-Agent Deliberation Prompt (Rounds 2–3) Injects peer responses between the Briefing and the Question. INFORMATIONAL MATERIALS [Briefing Content] END MATERIALS GROUP DISCUSSION The following are responses from other participants in your discussion group: [Participant 1 (Democrat)]: "I believe that..." [Participant 2 (Republican)]: "I disagree because..." ... END GROUP DISCUSSION Question: [Question Text] [Scale Instruction] [Standard Persona Description] Your response (ONLY THE NUMBER): Figure 5: Multi-Agent Deliberation Prompt. Used in the exploratory analysis experiments (Section 7) to simulate group discussion. Appendix B Detailed Experimental Conditions Table 6 details the configurations, sample sizes, and purposes of the 21 experimental runs (over 340,000 queries). Table 7 records the API model string, provider endpoint, and inference settings for each model, including provider-specific options where exposed. Prompt construction is identical across models (Appendix A); the differences recorded here concern transport, caching, and provider-side reasoning controls. ID Model Personas Qs Condition Experimental Purpose Tier 0: Human Baseline 0 Human (A1R) 526 72 — Ground truth belief revision data Tier 1: Phenomenon Establishment (GPT-5.1) 1 GPT-5.1 526 72 Standard Main Study: Establish reversal failure mode 2 GPT-5.1 526 20 All-Themes Test if briefing format causes failure 3 GPT-5.1 526 20 Multi-Agent Test if social pressure mitigates reversal 4 GPT-5.1 526 5 Ingroup Control Test behavior toward own party (Symmetry) 5 GPT-5.1 526 5 Outgroup Val. Validation check for outgroup targeting 6 GPT-5.1 526 20 Irrelevant Content Trigger: Test with non-political text Tier 2: Mechanistic Investigation (Gemini 2.0 Flash) 7 Gemini 2.0 526 20 Standard Baseline comparison for mechanism tests 8 Gemini 2.0 526 20 CoT Test if reasoning fixes stereotype retrieval 9 Gemini 2.0 526 20 3-Round Test multi-turn deliberation dynamics 10 Gemini 2.0 526 20 No-Party Test label removal 11 Gemini 2.0 526 20 Label-Swap Counterfactual test (Label vs. Demographics) 12 Gemini 2.0 526 20 Persona-First Test prompt ordering/anchoring effects 13 Gemini 2.0 526 20 Temp=0.7 Test for mode collapse 14 Gemini 2.0 99 1 No Persona Control: Default model behavior 15 Gemini 2.0 99 3 Party Only Control: Stereotype dependence on label 16 Gemini 2.0 526 20 Irrelevant Content trigger check for Gemini 17 Gemini 2.0 526 5 Ingroup Control Ingroup favoritism verification 18 Gemini 2.0 526 5 Outgroup Val. Outgroup moderation verification Tier 3: Generalization (Cross-Model) 19 Claude 4.5 100 20 Standard Test for "Dampening" vs Reversal 20 Llama 3.3 70B 100 20 Standard Open-weights frontier check 21 DeepSeek V3 100 20 Standard Test for "Rigidity" / Refusal Table 6: Complete Registry of Experiments. We systematically isolated variables (briefing content, identity labels, social context) across 21 distinct experimental configurations. Model API model string Provider / endpoint T Max tok. Model-specific settings GPT-5.1 gpt-5.1 OpenAI, chat.completions 0 10 Reasoning disabled (Flex processing tier) (reasoning_effort="none"); 900 s timeout, 5 retries Gemini 2.0 Flash gemini-2.0-flash Google Generative Language API, 0 10 No reasoning control exposed; REST v1beta:generateContent T=0.7T=0.7 in mode-collapse ablation Claude Sonnet 4.5 claude-sonnet-4-5 Anthropic, Message Batches API 0 10 Extended thinking not enabled; no system prompt (batch constraint); caching on briefing block Llama 3.3 70B meta-llama/ DeepInfra, 0 10 No reasoning control exposed Llama-3.3-70B-Instruct api.deepinfra.com/v1/openai DeepSeek V3 deepseek-ai/DeepSeek-V3 DeepInfra, 0 10 Non-reasoning variant; api.deepinfra.com/v1/openai no reasoning control exposed Table 7: Model Configuration Details. All five models were queried through their providers’ public APIs at temperature 00, with output token limits set to force a bare integer on the 0–10 response scale; the exception is the mode-collapse ablation (Table 6), which used T=0.7T=0.7. Reasoning behaviour was controlled where the provider exposed a setting: GPT-5.1 was run with reasoning_effort set to "none", disabling reasoning tokens, on OpenAI’s Flex processing tier with a 900-second timeout and exponential-backoff retries; Claude Sonnet 4.5 was run without extended thinking enabled; the remaining models were queried in their standard non-reasoning configurations, which expose no comparable control. Claude Sonnet 4.5 was run through Anthropic’s Message Batches API, which does not accept a separate system prompt, so persona, briefing, and question were supplied in a single user message with prompt caching applied to the briefing block. Llama 3.3 70B and DeepSeek V3 were served through DeepInfra’s OpenAI-compatible endpoint. Multi-agent conditions raised the output limit to 500 tokens to permit sentence-length responses. Prompt construction was identical across models (Appendix A); the differences recorded here concern transport, caching, and provider-side controls. All runs were executed in December 2025 and January 2026. Appendix C Detailed Outgroup Analysis Human (A1R) GPT-5.1 Gemini 2.0 Question Pre Post Δ Pre Post Δ Pre Post Δ Q16A↓: “They* just don’t know enough.” 5.97 5.96 −0.01-0.01 4.75 6.03 +1.28+1.28 6.22 5.03 −1.18-1.18 Q16B↓: “They* believe some things that aren’t true.” 7.67 7.37 −0.30-0.30 4.47 4.93 +0.46+0.46 5.73 3.37 −2.37-2.37 Q16C↓: “They* are not thinking clearly.” 6.03 4.61 −1.41-1.41 5.54 5.71 +0.17+0.17 6.87 3.63 −3.25-3.25 Q16D↑: “They* have good reasons; there just are better ones on the other side.” 5.02 6.02 +1.01+1.01 7.59 7.51 −0.08-0.08 5.54 5.43 −0.12-0.12 Q16E↓: “They* are looking out for their own interests.” 7.52 7.17 −0.35-0.35 7.16 7.83 +0.67+0.67 7.69 7.82 +0.13+0.13 Mean −0.21-0.21 +0.50+0.50 −1.36-1.36 Table 8: Pre- and post-deliberation means on outgroup questions (disagree–agree scale, 0–10). Human values are computed over personas responding to each item in both waves (n=431n=431–464); survey non-response reduces n below the full sample. *“Those people” refers implicitly to the opposing party. ↓ Negative attributions: higher scores indicate greater hostility; Δ>0 >0 means increased hostility after deliberation. ↑ Positive attribution (Q16D): higher scores indicate greater charity toward the outgroup; reversed polarity. Humans do not increase hostility on any negative item, and become markedly more charitable on Q16D. GPT-5.1 reverses on all four negative items (Δ>0 >0) but not on Q16D, consistent with hostility-coded stereotype retrieval. In the ingroup–outgroup control (§6.2), “They” is replaced with the explicit opposing party label (outgroup) or own party label (ingroup). Of the 523 matched personas, 415 produced active responses on outgroup questions (both human and GPT-5.1 shifted away from zero on the same question). Table 9 reports the distribution of per-persona reversal rates. Per-Persona Reversal Rate N Personas % of Active (N=415N=415) 0% (no reversals) 110 26.5 1–25% 8 1.9 26–50% 122 29.4 51–75% 43 10.4 76–99% 1 0.2 100% (all reversed) 131 31.6 Mean rate 52.6% Median rate 50.0% Table 9: Distribution of per-persona outgroup reversal rates for GPT-5.1 (N=415N=415 personas with active responses on ≥1≥ 1 outgroup question). A persona is counted as reversing if it shifts in the opposite direction from its matched human on the same question; zero-shift observations (human or LLM) are excluded. 73.5% of personas reverse on at least one question; 42.2% reverse on a majority (>50%>50\%). The bimodal distribution (modes at 0% and 100%) rules out a single-outlier explanation while showing that a substantial minority of personas do not reverse, consistent with the content-triggered rather than globally-applied mechanism. Cross-Question Consistency (Phi Correlation Matrix) To test whether reversal reflects a stable persona-level trait or question-specific content activation, Table 10 reports pairwise phi correlations of reversal (yes/no) across outgroup questions within GPT-5.1 personas. Q16A Q16B Q16C Q16D Q16E Q16A — +0.07+0.07 +0.34∗+0.34^* −0.02-0.02 −0.01-0.01 Q16B +0.07+0.07 — +0.10+0.10 +0.01+0.01 +0.33∗+0.33^* Q16C +0.34∗+0.34^* +0.10+0.10 — +0.11+0.11 −0.11-0.11 Q16D −0.02-0.02 +0.01+0.01 +0.11+0.11 — −0.02-0.02 Q16E −0.01-0.01 +0.33∗+0.33^* −0.11-0.11 −0.02-0.02 — Table 10: Pairwise phi correlations of reversal across outgroup questions within GPT-5.1 personas. ∗p<0.05^*p<0.05. Mean ϕ=0.08φ=0.08 (range: −0.11-0.11 to +0.34+0.34). The weak and inconsistent cross-question correlations indicate that reversal is driven by the interaction between persona identity and specific question content rather than a stable persona-level trait. Two pairs reach significance (Q16A–Q16C and Q16B–Q16E), but the majority of pairs show correlations near zero or negative, inconsistent with a persona-level trait explanation. Mixed-Effects Model Specification For each human–LLM comparison we fit Δi,q∼Model+(1|Persona)+(1|Question) _i,q +(1|Persona)+(1|Question), where Δi,q _i,q is the belief shift for persona i on question q, Model is a fixed effect (human vs. LLM), and Persona and Question are crossed random intercepts. Fit with statsmodels. We report the fixed-effect coefficient for Model, its 95% CI, and Cohen’s d; ICC <0.01<0.01, confirming negligible persona-level clustering. Comparison Category β SE z p Note GPT-5.1 vs Human Outgroup +0.717+0.717 0.0550.055 +12.97+12.97 1.86×10−381.86× 10^-38 M2 Gemini vs Human Outgroup −1.133-1.133 0.0480.048 −23.83-23.83 1.83×10−1251.83× 10^-125 M2 Claude vs Human Outgroup −0.817-0.817 0.1090.109 −7.48-7.48 7.63×10−147.63× 10^-14 M2 Llama vs Human Outgroup −1.187-1.187 0.1080.108 −11.02-11.02 3.01×10−283.01× 10^-28 M2 DeepSeek vs Human Outgroup +0.247+0.247 0.1000.100 +2.47+2.47 0.0510.051 (n.s.) M2 GPT-5.1 vs Human Policy −0.036-0.036 0.0130.013 −2.79-2.79 5.20×10−35.20× 10^-3 M2 GPT-5.1 vs Human Efficacy −0.016-0.016 0.0270.027 −0.60-0.60 0.5510.551 (n.s.) M2 Gemini vs Human Policy −0.283-0.283 0.0410.041 −6.86-6.86 6.65×10−126.65× 10^-12 M3† Gemini vs Human Efficacy +0.082+0.082 0.0450.045 +1.83+1.83 0.0670.067 (n.s.) M2 GPT outgroup vs ingroup — — — t=12.29t=12.29 <0.0001<0.0001 paired t Gemini outgroup vs ingroup — — — t=−12.60t=-12.60 <0.0001<0.0001 paired t GPT policy vs irrelevant — — — t=9.31t=9.31 <0.0001<0.0001 indep. t Outgroup vs policy reversal Rate — — OR=11.67OR=11.67 0.027∗0.027^* Fisher’s exact Table 11: Complete statistical test results. M2 = mixed-effects with crossed CASEID and question random effects (preferred specification); M3† = unpaired fallback (Gemini policy: M2 singular). All primary model–human outgroup comparisons survive Bonferroni correction (α=0.05/13=0.0038α=0.05/13=0.0038). ∗Fisher’s exact does not survive correction; the selectivity claim rests on three convergent experiments (content trigger, ingroup/outgroup control, Q16D pattern) rather than this test alone. Effect size for Fisher’s reported as OR; all others as Cohen’s d. Demographics Label Npers Pre Post Δ 95% CI Democrat Democrat [matched] 194 6.16 4.67 −1.49-1.49 [−1.63,−1.35][-1.63,-1.35] Republican [swapped] 194 6.48 4.61 −1.87-1.87 [−1.99,−1.75][-1.99,-1.75] Republican Democrat [swapped] 132 6.26 4.53 −1.72-1.72 [−1.89,−1.56][-1.89,-1.56] Republican [matched] 132 6.61 5.02 −1.59-1.59 [−1.75,−1.44][-1.75,-1.44] Same-label (test of label dominance): Rep-demos vs. Dem-demos || Dem-label Δ diff =−0.23=-0.23, t=−2.06t=-2.06, p=0.040p=0.040 Dem-demos vs. Rep-demos || Rep-label Δ diff =−0.28=-0.28, t=−2.76t=-2.76, p=0.006p=0.006 Same-demographics (test of demographic dominance): Dem-label vs. Rep-label || Rep-demos Δ diff =−0.13=-0.13, t=−1.11t=-1.11, p=0.268p=0.268 Rep-label vs. Dem-label || Dem-demos Δ diff =−0.38=-0.38, t=−3.97t=-3.97, p<0.001p<0.001 Table 12: Label-swap 2×2 (Gemini 2.0 Flash, outgroup questions, Dem and Rep personas only; N=326N=326 personas, 1,630 paired observations). Matched cells use the original persona-label assignment; swapped cells assign the opposite party label while holding all twelve other demographic attributes constant. Three of four cell comparisons reach significance. The label effect is asymmetric across demographics: changing the label shifts Δ by 0.380.38 points within Democrat-demographic personas (p<0.001p<0.001) but only 0.130.13 points within Republican-demographic personas (p=0.27p=0.27). The demographic effect is consistent across labels (0.230.23 and 0.280.28 points, both p<0.05p<0.05). Both label and demographics influence belief revision. Attribute Strata Range of Δ All >0>0 Gender 2 +0.45+0.45 to +0.55+0.55 Yes Age 7 +0.30+0.30 to +0.75+0.75 Yes Race 6 +0.23+0.23 to +0.62+0.62 Yes Education 4 +0.34+0.34 to +0.55+0.55 Yes Income 18 +0.17+0.17 to +0.84+0.84 Yes Region 4 +0.28+0.28 to +0.61+0.61 Yes Metro 2 +0.28+0.28 to +0.53+0.53 Yes Housing 3 +0.11+0.11 to +0.63+0.63 Yes Marital 6 −0.33-0.33 to +0.94+0.94 No Employment 7 +0.01+0.01 to +0.96+0.96 Yes LGBT 5 +0.48+0.48 to +1.04+1.04 Yes Table 13: Demographic stratification of GPT-5.1’s outgroup belief shift across all non-party demographic attributes except state (50 levels, cells too small to stratify). Each row reports the range of Δ over all strata of one attribute; the aggregate GPT-5.1 outgroup Δ=+0.50 =+0.50, while humans shift negatively (Δ=−0.21 =-0.21). The shift is positive in every stratum of ten of the eleven attributes. The exception is marital status, where separated (Δ=−0.33 =-0.33, n=11n=11) and divorced (Δ=−0.11 =-0.11, n=80n=80) personas shift in the human direction. No other attribute contains a negative stratum, which is inconsistent with any single non-party demographic acting as the operative driver of reversal, though marital status may modulate the effect. Note the stratification uses all 526 GPT-5.1 personas, since it doesn’t require human matching. Appendix D Threshold Sensitivity Analysis Refer to Table 14. Category Threshold |Δ|<| |< Human GPT-5.1 Gemini Claude Llama DeepSeek Outgroup 1 (48% of mean |Δ|| |) 24.5 48.0 25.9 59.6 36.6 97.2 2 (96%) 47.3 77.0 56.6 75.9 44.2 97.6 3 (144%) 66.4 85.2 72.5 78.6 62.6 99.8 5 (239%) 86.8 86.9 95.1 92.7 99.8 100.0 Policy 1 (51% of mean |Δ|| |) 33.4 59.2 49.7 66.6 75.7 55.8 2 (101%) 52.2 92.2 86.3 93.7 83.7 85.2 3 (152%) 68.1 96.9 91.6 96.5 96.5 97.2 5 (253%) 85.5 100.0 99.5 100.0 99.9 97.8 Efficacy 1 (58% of mean |Δ|| |) 34.2 55.2 64.6 70.7 56.0 40.7 2 (115%) 56.6 90.5 94.1 99.3 74.0 88.3 3 (173%) 73.2 95.9 99.6 99.7 93.7 98.7 5 (288%) 89.4 100.0 100.0 100.0 99.3 100.0 Table 14: Percentage of responses with |Δ|<| |< 0.1*, by category and threshold (0–10 scale). Values represent % of observations showing no meaningful change. Human mean |Δ|| | for outgroup = 2.09 (95% CI [2.00, 2.17]); policy = 1.98 [1.95, 2.01]; efficacy = 1.73 [1.68, 1.78]. DeepSeek outgroup rigidity (bolded) is 4.0× the human rate at |Δ|<1| |<1 and holds at ≥97%≥ 97\% across all thresholds, confirming the rigidity classification robustly. All other models’ classification as overshoot or reversal is confirmed by directional analysis independently of threshold. Appendix E Per-Partisan Breakdown Refer to Table 15 Democrats Republicans Independents Question Human GPT Human GPT Human GPT Q16A: Don’t know enough +0.32+0.32 +1.16+1.16 −0.40-0.40 +1.19+1.19 −0.12-0.12 +1.45+1.45 Q16B: Believe untrue things −0.22-0.22 +0.44+0.44 −0.28-0.28 +1.19+1.19 −0.38-0.38 +0.00+0.00 Q16C: Not thinking clearly −1.36-1.36 −0.60-0.60 −1.67-1.67 −0.20-0.20 −1.31-1.31 +1.16+1.16 Q16D: Have good reasons +1.14+1.14 −0.05-0.05 +0.94+0.94 −0.55-0.55 +0.91+0.91 +0.21+0.21 Q16E: Own interests −0.20-0.20 +0.88+0.88 −0.50-0.50 +0.43+0.43 −0.42-0.42 +0.61+0.61 Table 15: Reversal Symmetry by Party (GPT-5.1). Belief shifts (Δ ) separated by partisan group. Note that both Democrats and Republicans shift positively (more hostile) on Q16A and Q16B, whereas humans shift negatively or consistently across groups. This confirms the reversal is not driven by one specific party’s data.