Paper deep dive

Probable but Not Valid: A Psychometric Audit of Decision Models as Synthetic Survey Respondents

Mantas Lukauskas, Viktorija Šarkauskaitė

Year: 2026Venue: arXiv preprintArea: cs.AIType: PreprintEmbeddings: 135

Intelligence

Status: not_run | Model: - | Prompt: - | Confidence: 0%

Entities (0)

No extracted entities yet.

Relation Signals (0)

No relation signals yet.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Language models are increasingly used to create synthetic survey samples, also called silicon samples. Decision models are a new class of language models that return a probability for each point of a rating scale instead of text. We ask whether these models, or the newest generative LLMs, can stand in for human respondents on a psychometric survey. We test fourteen decision models from eleven providers and 47 generative LLMs on an open Lithuanian organisational-psychology survey (n=263, 68 items), with 100 real respondent profiles, the Psychometric Similarity Score (PSS) and statistical baselines. The best generative model, MiniMax M3, reaches PSS 0.813. This is level with a Gaussian copula fit only on other respondents (0.824) and below a copula fit in sample (0.927). A second sample of 100 real people scores below the best models, so at this size PSS cannot rank them against real respondents. In 5 of 10 product lines the newest generative release scores below an earlier release of the same line beyond resampling noise, and none gains beyond it. With the default rounded readout, 13 of 14 decision models score below the generative reference gpt-5.4-mini. Of the 47 generative models, 33 reproduce the pattern of item correlations better than any decision model. Averaged over respondents, the probability vectors are much closer to the human item marginals than rounded answers. They also rank respondents about as well as a profile regression fit on 163 real people. As forecasts of each person's own answers, however, every decision model is worse than the answer frequencies of other respondents, mainly because its average vector differs from them. The profile regression improves on these frequencies by only 1.0%, so the profile fields say little about how a person answers. Synthetic respondents built on such profiles should therefore aim at sample-level fidelity.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

131,534 characters extracted from source content.

Expand or collapse full text

Probable but Not Valid: A Psychometric Audit of Decision Models as Synthetic Survey Respondents Mantas Lukauskas Hostinger; Kaunas University of Technology; AI Insight Lab mantas.lukauskas@ktu.lt Viktorija Šarkauskait ̇ e AI Insight Lab; Independent Researcher viktorija@aiinsightlab.ai Abstract Language models are increasingly used to create synthetic survey samples, also called silicon samples. Decision models are a new class of language models that return a probability for each point of a rating scale instead of text. We ask whether these models, or the newest generative LLMs, can stand in for human respondents on a psychometric survey. We test fourteen decision models from eleven providers and 47 generative LLMs on an open Lithuanian organisational-psychology survey (n=263, 68 items), with 100 real respondent profiles, the Psychometric Similarity Score (PSS) and statistical baselines. The best generative model, MiniMax M3, reaches PSS 0.813. This is level with a Gaussian copula fit only on other respondents (0.824) and below a copula fit in sample (0.927). A second sample of 100 real people scores below the best models, so at this size PSS cannot rank them against real respondents. In 5 of 10 product lines the newest generative release scores below an earlier release of the same line beyond resampling noise, and none gains beyond it. With the default rounded readout, 13 of 14 decision models score below the generative reference gpt-5.4-mini. Of the 47 generative models, 33 reproduce the pattern of item correlations better than any decision model. Averaged over respondents, the probability vectors are much closer to the human item marginals than rounded answers. They also rank respondents about as well as a profile regression fit on 163 real people. As forecasts of each person’s own answers, however, every decision model is worse than the answer frequencies of other respondents, mainly because its average vector differs from them. The profile regression improves on these frequencies by only 1.0%, so the profile fields say little about how a person answers. Synthetic respondents built on such profiles should therefore aim at sample-level fidelity. Preprint. arXiv:2610.14912v1 [cs.AI] 10 Oct 2026 1 Introduction Large language models (LLMs) are now used to simulate survey respondents [4,26,1,15], and their role in social science research is widely discussed [22,5,53]. Synthetic samples can match human averages and still differ from human data in variance and in regression estimates [9,10]. A psychometric audit of 37 LLMs found a similar gap on an open Lithuanian organisational-psychology survey [33]. The models were conditioned on real respondent profiles, and the synthetic samples were compared with the human sample on item distributions, item correlations, reliability, mediation and demographic effects. The answers looked plausible. However, the samples did not behave like the human sample, and a Gaussian copula fit to the human data did better on the components that describe the sample as a whole. We call this study the original audit below. In September and October 2026 a new class of models became available through a common API [37]. We call them decision models, which is the name under which the providers list them. The term is unrelated to decision theory. A decision model reads a state (text or JSON) and answers a set of named questions in one request. For a score question with an ordered rubric it returns a probability for every level and the expected level. No text is generated, output tokens are free, and one request can carry 64 to 200 questions. For a survey this looks close to ideal. Each Likert item becomes one score question, the respondent profile becomes the state, and the model returns a full answer distribution for every item. Several failure sources of generated answers cannot occur, such as parsing errors, refusals and long reasoning traces. The generative models have changed as well. Since the original audit, OpenAI, Anthropic, Google, xAI and several open-weight developers have released new versions. Model behaviour can change between versions in ways that are not announced [12], so it is an open question whether these newer models close the gap. We ran eleven of them on the same benchmark through vendor APIs and Fireworks. One of them, GPT-6 Luna, is also offered as a decision model. This allows a comparison of the two interfaces for one model name. This paper aims to test whether decision models are better synthetic respondents than generative LLMs, and whether newer generative LLMs are better than earlier releases. We raised four hypotheses. First, a model that returns a distribution over scale points reproduces the item marginals better than a generative LLM. Second, it does not by itself reproduce the pattern of correlations between items, because the vector for each item is produced without an explicit shared respondent state. Third, it uses the respondent profile only weakly. Fourth, newer generative models score higher than their predecessors. We also ask whether the probability vectors forecast each person’s answers better than the answer frequencies of other respondents. All models are compared on the same 100 respondents, profile fields, scoring and baselines. We use the word valid in the sense of the testing standards [2]. Synthetic answers are valid for a use if they support the same interpretations as human answers. We examine two sources of evidence. The first is internal structure: item distributions, item correlations, reliability and factor congruence. The second is relations to other variables: mediation, group contrasts and the prediction of human outcomes. We do not examine content validity. Our main findings are as follows. Scored against all 263 respondents, the best language models are level with a Gaussian copula fit only on other respondents, and none is above it. Newer generative releases do not score higher. In 5 of 10 product lines the newest release scores below an earlier release of the same line by more than the resampling noise. Averaged over respondents, the vectors of decision models come much closer to the human item marginals than rounded answers do. However, they reproduce the pattern of item correlations and the factor structure less well than most generative LLMs. Their vectors carry about as much information about the person as a regression on the profile, but they are poorly calibrated. As forecasts of each person’s own answers they are worse than the answer frequencies of other respondents, and the profile regression improves on these frequencies by only 1.0%. We score every sample with a revised version of the original scoring, which reverses model answers to reverse-keyed items before comparison. Under it the main conclusion of the 2 original audit holds, and the order of the LLMs changes. Appendix B is a self-contained record of these corrections. The contributions of this paper are as follows. 1. A psychometric evaluation of decision models as synthetic survey respondents: fourteen models from eleven providers, 100 respondents, 68 items and full probability vectors, at a total cost of $1.27. 2. An updated generative leaderboard of 47 LLMs under the revised scoring. It includes 11 models released after the original audit and a comparison of successive releases within 10 product lines. In 5 of these lines the newest release scores lower by more than the resampling noise. 3. A comparison of the generative and the decision interface of GPT-6 Luna. 4. An analysis of readouts, of the internal structure of the samples and of what the probability vectors contain. It uses control grids without a profile and with repeated requests, swaps of education, role and gender, and a profile regression that is read out like a decision model. 5.A test of the probability vectors as forecasts of each person’s own answers with a proper scoring rule. For every decision model the vectors are worse than the answer frequencies of other respondents. 6.A check of the reference points. Besides the real answers of the same 100 people and a copula fit only on other respondents, it uses a second sample of 100 real people and a reference that excludes the training data of the copula. 7. Nine recommendations for evaluating synthetic survey respondents (Section 6). The rest of this paper is organised as follows. Section 2 reviews related work. Section 3 summarises the benchmark and data. Section 4 describes how the models were queried and scored. Section 5 presents the results, and Section 6 discusses them. Section 7 lists the limitations and directions for future work. Finally, Section 8 concludes. 2 Related work LLMs as survey respondents. Persona-conditioned LLMs have been used to reproduce public-opinion splits [4], economic choices [26] and classic experiments [1]. Other studies found that LLM opinions reflect a narrow part of the population [43,18], that synthetic samples can match human averages while showing too little variance and different regression estimates [9], and that LLMs flatten identity groups [50]. Answers of LLMs to World Values Survey items show low variance and a bias that changes from topic to topic [10]. LLM simulations have also predicted the treatment effects of survey experiments about as well as human forecasters [25]. Recent position papers discuss both the promise of LLM social simulations and their limits [3,31]. In HCI and marketing research, silicon samples have been recommended mainly for ideation and pilot studies [23,45]. Lukauskas and Šarkauskait ̇e[33] moved the comparison to the level of a full psychometric instrument and added statistical baselines and a human ceiling. Predicting one person. Park et al.[38]built agents for 1,052 Americans and scored their answers to held-out General Social Survey items against each person’s own test-retest consistency. Agents built from interviews came close to this consistency, and agents given only demographic attributes did worse. Kim and Lee[28]fine-tuned a language model on General Social Survey waves. It predicted missing individual answers well, but answers to questions that had never been asked less well. A model fine-tuned on trial-level data from many psychology experiments predicts the choices of held-out participants [8]. We use a reference that needs no information about the person, namely the answer frequencies of other respondents. How the answer is elicited. The format of elicitation changes what an LLM appears to believe. Survey answers of LLMs depend strongly on answer order and labelling, and adjusting for these effects moves the answers toward uniform [16]. Opinion distributions 3 have been read from token probabilities [43], and distributional alignment differs between log-probabilities, sampled answers and stated distributions [35]. For instruction-tuned models, the first-token probabilities over answer options often do not match the answer the model gives in text [51]. Fine-tuning on survey data brings the predicted answer distribution closer to the answer distributions of human groups [11,48]. Direct numeric ratings give unrealistic rating distributions, and mapping free-text answers onto the scale anchors by semantic similarity gives more realistic ones [34]. All of these studies compare distributions for groups of people. We ask what the distribution says about one person. Decision models make the probability readout the default interface, and the analyst still has to turn each distribution into one answer per respondent. Model versions. The behaviour of a deployed model can change between versions in ways that are not announced and not monotone. Chen et al.[12]found that GPT-4 and GPT-3.5 changed markedly on several tasks within a few months. Synthetic-respondent studies usually report one version per model. We compare successive releases on the same benchmark. Psychometrics of LLMs. Methods from cognitive psychology have been used to study how LLMs decide and reason [7], and psychometric inventories have been applied to the models themselves [40,47]. When models answer such inventories with personas built from the demographics of real people, the psychometric properties of the answers are poor [41]. Response styles and baselines. Tjuatja et al.[49]found that LLMs do not show human-like response biases in survey questions, including acquiescence, and that they react to wording changes that do not move human answers. LLMs also shift Big Five answers toward the socially desirable end when they can infer that they are being assessed [42]. We measure acquiescence with the balanced index used in cross-national survey research [6]. The statistical anchor is a Gaussian copula with empirical marginals [36], a standard model for synthetic tabular data [39]. What this paper adds. Earlier work compares model and human answer distributions for groups of people, mostly on opinion items, and reads probabilities from the next token of a generative model. We make three changes. First, we evaluate models whose only output is a probability vector and compare them with 47 generative LLMs on the item distributions, item correlations, factor structure, reliability, mediation and group contrasts of four survey scales. Second, we score the vectors as forecasts of each person’s own answers with a proper scoring rule, against the answer frequencies of other respondents and a regression on the same profile. Third, we compare successive releases within 10 product lines on one fixed benchmark. We are not aware of a published evaluation of decision models on survey data. The model descriptions on the provider pages are the only documentation [37]. 3 Benchmark and data This section summarises the benchmark, which comes from the original audit [33]. Data. The human reference is an open sample ofn=263 Lithuanian employees [44]. They answered the Individual Work Performance Questionnaire (18 items, 1–5) [30], the Dunham attitudes toward change scale (18 items, 1–5) [17], a change-engagement scale built for the original study (15 items, 1–5) and the UWES-17 work engagement scale (17 items, 1–7) [46]. The change-engagement scale is less well validated than the other three. Together this gives 68 items in 12 subscales. Thirteen items are reverse-keyed: five in the Dunham scale and eight in the change-engagement scale. Profiles and sample. The original audit varied how much of the profile the model sees, from no profile (condition C0) to the full structured profile (C3). We use C3, in which the model sees eleven profile fields: country, gender, age, education, position, total tenure, tenure in the current organisation, sector, organisation size, and two ratings of how intense and how relevant recent organisational change was. The headline sample is 100 respondents stratified 4 by gender and position with a fixed seed. We use exactly the 100 respondent IDs of its gpt-5.4-mini C3 cell. PSS. Every synthetic sample is scored against all 263 humans with five components, PSS = 0.25D + 0.25C + 0.20R + 0.20M + 0.10G.(1) D is one minus the mean Jensen-Shannon divergence (in bits) between the human and synthetic item distributions.Cis the Pearson correlation between the upper triangles of the two 68×68 item correlation matrices, floored at zero. It compares the pattern of the two matrices and does not penalise a uniform rise of all correlations, so Section 5.6 also reports their level and the factor structure.Ris one minus the absolute difference in Cronbach’s α[13] on the Dunham scale.Mis one minus the absolute difference in the standardised indirect effect of attitudes on performance through engagement [24], divided by 0.5.Gis the share of 96 demographic contrasts (8 contrasts by 12 outcomes) with the same sign of Cohen’sdas in humans. It is multiplied by one minus the absolute mean amplification, where the amplification of a contrast is|d|in the synthetic sample minus|d|in humans. Each component lies in [0, 1]. The weights are those of the original audit. References. The original audit compared LLMs with five statistical baselines and a held- out human ceiling, in which 53 humans were scored against the other 210 (PSS 0.825). Its conclusion was that LLM samples are not a drop-in replacement for human survey data. Scoring. We use a revised version of the original scoring. It reverses model answers to the 13 reverse-keyed items before they are compared with the reversed human answers, and it includes the distribution component in the totals of the statistical baselines. Appendix B lists the numbers that change relative to the published version of the original audit. 4 Querying and scoring the models Decision models. We listed all models with the decisions output modality on OpenRouter on 7 October 2026 and again on 10 October 2026 [37]. Appendix Table 13 gives the pinned versions and costs. We kept every model that accepted five- and seven-level rubrics and at least 64 score questions per request. Three candidates were left out. The Span-01 family accepts only a chat-message state, so the profile cannot be passed in the same form. jev-latestis only an alias that points to Jev 1.13. Solar Decide takes about two minutes per respondent and returned server errors for many requests. After eight hours 46 of 100 respondents were complete, below our threshold of 90, so we left it out. The second listing added five models, and all of them met the criteria. Decider v1.1 27B is described by its provider as the successor of Decider v1 27B, and Solar Decide Flash is a low-latency variant of Solar Decide. Clef Omni, Microsoft-Decision-1 and Drex v1.5 are new. This leaves fourteen models from eleven providers: Jev 1.13, Decider v1 27B, Decider v1.1 27B, Liquid d1, Kev 4B, Tev1 4B, Mercury Decide, Solar Decide Flash, Clef, Clef Flash, Clef Omni, GPT-6 Luna Decisions, Microsoft-Decision-1 and Drex v1.5. Requests. The state is the C3 profile as a JSON object with English field names and values. Each item becomes one score question. Its instruction is the Lithuanian sentence “Answer the statement as this employee would answer it” followed by the Lithuanian item text, and its criteria are the numbered Lithuanian scale anchors (Appendix A). All 68 questions go in one request, in instrument order. The three Clef models and Drex v1.5 reject more than 64 questions, so they receive the same profile twice, first with 64 items and then with the last 4. The endpoint exposes no temperature or seed. Failed requests were retried with exponential backoff, and all 1,400 respondent-model cells completed. Readouts. For itemjthe model returns a vectorp j overKlevels. We store the full vector and derive three answers. The rounded expected score (R) isround( P k k p jk ) on the original scale. The most probable category (A) isarg max k p jk , and the sampled category (S) is one draw fromp j , repeated five times with a fixed seed and averaged. R is the default readout because it is closest to a single generated answer. The other two readouts show how much the conclusions depend on this choice. 5 Generative models. The generative reference set has two parts. First, it contains the C3 cells of the original audit (single call, minimal reasoning) with at least 90 complete respondents among the same 100 IDs. This gives 36 cells. Qwen 3.7 Plus has 75 complete respondents and is left out of the rankings. Second, it contains 11 models released after the original audit. They are the new releases of the vendors in that audit that we could reach through a vendor API or Fireworks in October 2026. Mistral Large 4 is left out because it is offered only through OpenRouter. We ran these models with the generator, prompt and parser of the original audit and requested a temperature of 0.7. As in that audit, the GPT-5 and GPT-6 families and models whose API rejects the parameter run at the provider default. Each respondent gets one call with all 68 items. We use the same minimal reasoning setting. It means no extended thinking for Anthropic, a thinking budget of zero for Google, andreasoning_effort=minimalfor the OpenAI-compatible APIs. We drop this parameter when an API rejects it. OpenAI, Anthropic and Google models are called through their vendor APIs: GPT-6 Luna and GPT-6.1 Sol, Claude Opus 5.5 and Claude Sonnet 5.5, Gemini 3.6 Flash and Gemini 3.8 Flash. Grok 4.7 is called through the xAI API, and the open-weight Kimi K3, Qwen 3.8 Max, GLM 5.3 and DeepSeek V4.1 Flash run on Fireworks. The original audit reached Grok and the open-weight models through the Nexos gateway. Decision models are only offered through OpenRouter. All new models completed 100 respondents except Grok 4.7 (99), and the run cost $15.05 at list prices. What differs between the two classes. The comparison of decision and generative models changes more than the output format. A generative model answers all 68 items in one generated text, so each answer can depend on the earlier ones. A decision model returns a separate distribution for each item. The decision request uses an English JSON state with Lithuanian items, while the generative prompt is fully Lithuanian. Decision models are reached through OpenRouter, and generative models through vendor APIs and Fireworks. Finally, most decision endpoints are deterministic, while generative models sample their answers. Our design cannot separate these factors. Section 5.3 compares the two interfaces for one model name, and Section 7 discusses the request design. Scoring and references. All synthetic samples are scored with the revised scoring. The gpt-5.4-mini answers and the other cells of the original audit are rescored from the stored answers with the same code. Besides the held-out human ceiling of the original audit we use four references: the real answers of the same 100 people (oracle), the in-sample copula, an out-of-sample copula fit on the 163 people outside the headline sample, and independent draws from the human marginals. The out-of-sample copula is out of sample for the 100 respondents only. Its 163 training people are part of the 263-person reference, which favours it on the distribution and correlation components. For this reason Section 5.1 also scores every source against the 100 headline people alone and compares it with a second sample of 100 real people. We also read the profile regression of the forecast analysis (below) as a synthetic respondent, in the same three ways as a decision vector. Uncertainty. We use a paired bootstrap [19] with 200 draws. Every draw resamples the 100 respondent IDs and the 263 humans with replacement, and the same draw is applied to every source. Resampling with replacement duplicates rows, which lowers the absolute level of every similarity component, by an amount that differs between sources. We therefore report point estimates on the original samples and use the draws only for standard errors. Intervals for paired differences are the point estimate±1.96 bootstrap standard errors of the difference. Percentile intervals give the same conclusion for 68 of the 77 differences to the best generative cell and for every decision model. Paired differences are reported against two references. The first is gpt-5.4-mini, the headline model of the original audit, which was fixed as a reference before this study. The second is the best generative cell, MiniMax M3. This reference is the highest of 47 noisy estimates, so its point estimate is biased upward and the differences to it are biased away from zero. A source that is within noise of it is close to the top of the generative field. Analysis plan. This study was not preregistered. The benchmark, the 100 respondents, the profile fields, the PSS weights and the scoring code come from the original audit and were not changed. We added the control grids, swaps, forecast scores, reference checks, the 6 structure analysis and the comparison of release waves after the first results and report them as exploratory. The four hypotheses were not registered either, and the tests of the third and fourth hypotheses use these later analyses. Theirp-values are not corrected for multiple testing unless we say so. Further analyses. Response style uses raw answers for humans and models. Predictive validity fits an OLS regression of performance on attitudes and engagement on each synthetic sample and tests it on a fixed 30% split of the humans. The null-mediation test draws 10 triples of composite scores whose human indirect effect is not significant and below 0.10. We leave out the 3 triples in which a total score contains another variable of the triple, such as attitudes in total and cognitive attitudes, because they do not test mediation. The remaining 7 triples are the same for every source. A synthetic sample fabricates a path when its bootstrap interval excludes zero and the effect exceeds 0.10. This criterion does not test whether the synthetic effect differs from the small human effect. The published version of the original audit pooled all LLM cells for this test, while we run it per model atn=100. Internal structure is described by the level of the item correlations, the share of the first eigenvalue, the separation of the subscales, the subscaleαand Tucker’sφ[32] per instrument with an item-permutation null. Weight sensitivity uses 500 Dirichlet draws around the PSS weights (concentration 30). Control grids. For the decision models we ran further grids with the same request format. One grid sends the 100 requests with an empty state, which corresponds to the condition without a profile (C0) in the original audit. Another sends the first 20 respondents again without any change, which measures run-to-run noise. Three more grids swap one field of all 100 profiles. Gender changes between woman and man, and position between managerial and non-managerial. Education follows the mapping of the original audit: university and other higher education become secondary education, and secondary and post-secondary education become university. One respondent without a gender value is left out of the gender swap. We compare the swaps with the swap grids of the original audit, which cover 37 generative models on the same respondents. These grids include Qwen 3.7 Plus. Effects are computed on the 12 composite scores in two ways: as the paireddof the original audit, and as the mean shift toward the first group of the human contrast (university, managerial, woman) in units of the human SD. Both are signed by the direction of each swap. Decider v1 27B had no available endpoint when the role and gender swaps were run. Tev1 4B had by then lowered its limit to 32 questions per request, so we sent its swaps in three parts and compared them with its unchanged profiles, sent again in the same way. All control grids cost $4.33. Forecast scores. A decision model returns a forecast of how a person will answer each item. We score these forecasts against the answers that the same 100 people gave. The main score is the ranked probability score (RPS) [20]. It compares the cumulative forecast with the cumulative observed answer and so respects the order of the scale points. We divide it byK−1 so that 5- and 7-point items can be pooled, and we also report the log score. Both scores are strictly proper [21], so a forecaster gets the best expected score by reporting its true distribution. Before scoring, every forecast is mixed with 1% of the uniform distribution so that the log score stays finite. We use three references. The population distribution gives every respondent the answer frequencies of the 163 humans outside the headline sample. The profile regression is a multinomial logistic regression of each item on the profile fields, fit on the same 163 people with an L2 penalty chosen by 5-fold cross-validation. The uniform distribution uses no knowledge of the survey. Skill is the relative improvement in RPS over the population distribution, with a 95% paired bootstrap interval over respondents. We also give every respondent the model’s own average vector. This tests whether the part of a vector that depends on the person improves the forecast. To separate information from calibration, we mix each vector with the population distribution and choose the weight by leave-one-respondent-out cross-fitting. Generative models return one answer per call. For them we use the test-retest panel of the original audit, in which 30 of the 100 respondents were sent 20 times to each of 31 generative models. The repeated answers form an empirical forecast for each person. We score each model on the panel members with at least 10 valid repeats, which leaves 26 to 30 people per model. To see how much of the result comes from 7 this estimator, we also draw 20 answers from each decision vector and score them in the same way. 5 Results 5.1 Where the language models stand Figure 1 and Table 1 place all language models against the reference points. The oracle, the real answers of the same 100 people, reaches 0.886. A copula fit on other respondents reaches 0.824, and the in-sample copula reaches 0.927. The in-sample copula is above the oracle because it is fit on the 263 people it is scored against, while the 100 real respondents are a subsample with their own sampling noise. Generative LLMs range from 0.510 to 0.813, with a median of 0.695. The best cell is MiniMax M3 (0.813), followed by MiniMax M2.7 (0.809) and Kimi K2.7 Code (0.802). Its lead rests on the mediation component (0.998). Without this component, with the other weights rescaled, MiniMax M2.7 leads (0.821). The top of the field is dense. For 12 generative cells the paired interval to MiniMax M3 contains zero. The difference between the out-of-sample copula and the best cell, +0.01 [−0.07, +0.09], is within noise. Against the oracle the interval, +0.07 [0.00, +0.15], only just contains zero. Given the selection bias of the best cell, we do not read this as parity with real respondents. On the correlation component, the highest value of any generative cell is 0.807 (MiniMax M2.7). The out-of-sample copula reaches a similar value (0.805), and the oracle is far higher (0.923). The fixed reference gpt-5.4-mini sits in the upper middle. It reaches 0.743, with a paired difference of−0.07 [−0.13,−0.01] to MiniMax M3. Its correlation component is clearly lower (−0.16 [−0.24,−0.09]), and its total is held up by a mediation component of 0.929. The demographic component separates the models least. Independent draws from the human marginals carry no profile information and still reach 0.49 on it, above the median generative LLM. A second human sample as reference. The 263-person reference contains the 163 people on which the out-of-sample copula is fit. Appendix Table 16 also scores every source against the 100 headline people alone. On this reference the reported copula sample falls to 0.71, and the mean of 20 copula samples is 0.75. A second sample of 100 real people from outside the headline sample reaches 0.79 on average (0.76–0.84 over 20 draws). The best generative cell, MiniMax M2.7, reaches 0.84, and 8 generative cells score above the average second human sample. Which source leads depends on the component. On the distribution component the second human sample (0.99) and the copula (0.98 without its own sampling noise) are far ahead of every language model (at most 0.90). For correlation, the best generative cells (0.78) are level with the copula without sampling noise (0.77) and above the second human sample (0.73). The mediation and demographic components differ a lot even between two real samples of 100 people (0.56 and 0.53), so a language model can score above real people there. Atn=100 the top of the PSS scale therefore does not separate the best language models from real respondents. 5.2 Decision models against the generative field Table 2 gives the paired differences to MiniMax M3. Under the default rounded readout every decision model is below it, and no interval contains zero. Jev 1.13 is the best decision model with 0.681, and its difference to the best generative cell is−0.13 [−0.21,−0.05]. Read as the most probable category, Jev reaches 0.786, and the difference of−0.03 [−0.10, +0.04] is within noise. This is the best of 42 decision cells (fourteen models and three readouts), and the total rests on the mediation component (Section 5.4). On the correlation component Jev stays below MiniMax M3 (−0.10 [−0.17, −0.03]). The comparison with gpt-5.4-mini does not depend on a selected reference. Under the rounded readout 13 of 14 decision models are below gpt-5.4-mini, with intervals that exclude zero. Jev read as the most probable category is +0.04 [−0.02, +0.10] above it, which is 8 Table 1: Aligned PSS components at C3 on the same 100 respondents, scored against all 263 humans. Generative rows show the three best of 47 generative cells, the best and the worst of the 11 models released after the original audit (marked ∗ ), GPT-6 Luna and the fixed reference gpt-5.4-mini; Appendix Table 15 lists all of them. The median row takes the median of each column separately. Decision models use the rounded expected score. SE is the bootstrap standard error of PSS over 200 draws that resample respondents and humans. The held-out split row is the ceiling reported in the original audit (53 humans scored against 210). The out-of-sample copula and the profile regression are fit on the 163 humans outside the headline sample; the profile regression predicts each item from the profile fields and is read out like a decision model. SourceDist. Corr. Rel. Med. Demo. PSSSE Reference points Same 100 humans (oracle)1.00 0.92 0.99 0.710.67 0.886 0.033 Held-out human split, original audit 0.98 0.76 0.99 0.720.48 0.825 Gaussian copula, in-sample0.99 0.89 0.99 0.970.66 0.927 0.019 Gaussian copula, out-of-sample0.98 0.81 0.97 0.720.38 0.824 0.028 Independent marginals0.99 0.00 0.00 0.670.49 0.430 Profile regression, rounded0.64 0.23 0.78 0.390.36 0.486 Generative LLMs MiniMax M30.71 0.76 0.97 1.000.51 0.813 0.023 MiniMax M2.70.83 0.81 0.98 0.760.52 0.809 0.039 Kimi K2.7 Code0.80 0.81 0.95 0.800.52 0.802 0.030 Claude Sonnet 5.5 ∗ 0.69 0.70 0.98 0.730.44 0.735 0.034 GPT-6 Luna ∗ 0.69 0.57 0.92 0.460.32 0.621 0.035 gpt-5.4-mini0.66 0.60 0.96 0.930.49 0.743 0.020 GPT-6.1 Sol ∗ 0.69 0.45 0.85 0.110.33 0.510 0.030 Median of 47 generative LLMs0.75 0.73 0.96 0.460.43 0.695 Decision models Jev 1.130.77 0.66 0.95 0.500.34 0.681 0.036 Microsoft-Decision-10.68 0.40 1.00 0.790.38 0.667 0.031 Decider v1.1 27B0.63 0.34 0.93 0.950.41 0.660 0.022 Drex v1.50.70 0.46 0.97 0.720.30 0.659 0.035 Decider v1 27B0.63 0.35 0.93 0.930.42 0.659 0.029 Mercury Decide0.78 0.53 0.95 0.340.48 0.636 0.035 Liquid d10.75 0.53 0.99 0.310.39 0.618 0.032 Clef0.69 0.54 0.99 0.090.40 0.563 0.029 Clef Flash0.70 0.62 0.97 0.000.30 0.554 0.018 Clef Omni0.72 0.50 0.99 0.220.06 0.551 0.030 GPT-6 Luna Decisions0.68 0.34 0.89 0.400.33 0.547 0.036 Kev 4B0.61 0.23 0.88 0.270.40 0.477 0.038 Tev1 4B0.70 0.24 0.95 0.000.33 0.460 0.021 Solar Decide Flash0.59 0.15 0.74 0.290.33 0.423 0.029 within noise. In the generative ranking, this readout of Jev would place 5th of the 47 cells (0.786), and its rounded readout would place 27th (0.681). The mediation component is the least stable. Its bootstrap standard error is 0.06 for gpt- 5.4-mini and 0.17 for Jev. Single mediation values atn=100 should not be read as model properties. Differences in PSS that come mainly from this component, such as the high rounded totals of the two Decider versions, deserve the same caution. 5.3 GPT-6 Luna as a generator and as a decision model GPT-6 Luna is the only model in our set that is offered through both interfaces. We cannot verify that both endpoints serve the same weights. The two runs also differ in route (the OpenAI API against OpenRouter), in the prompt (Lithuanian text against an English JSON state with Lithuanian items) and in sampling (the provider-default temperature against a fixed endpoint). As a generator, GPT-6 Luna reaches PSS 0.621 with a correlation component 9 0.40.50.60.70.80.9 PSS Decision, most probable Decision, rounded Generative, new (11) Generative, older (36) MiniMax M3gpt-5.4-mini GPT-6 LunaGPT-6.1 Sol Jev 1.13 Luna Decisions Jev 1.13 Luna Decisions 0.20.40.60.81.0 Correlation component copula, out-of-sample copula, in-sample same 100 humans Figure 1: The best language models are level with a copula fit only on other respondents, and none reaches the in-sample copula or the oracle (the real answers of the same 100 people). Left: PSS of every language-model cell at C3. Right: the correlation component. Each point is one model, and vertical jitter only separates points. Vertical lines mark the out-of-sample copula, the in-sample copula and the oracle. of 0.566. Served as a decision model, it reaches 0.547 with the rounded readout and 0.568 with the most probable category, and its correlation component falls to 0.339. The paired difference in correlation is +0.23 [+0.12, +0.33] in favour of the generator for the rounded readout and +0.18 [+0.07, +0.29] for the most probable category. In PSS the difference, +0.07 [0.00, +0.15], only just includes zero, because the other components move less. Both interfaces give similar distribution components (0.686 and 0.676). Other properties also differ. The decision answers show more agreement, with an acquiescence shift of 1.48 human SD against 0.53 for the generator. They also individualise slightly less (person share 0.36 against 0.40). On the downstream regression the order reverses. A regression trained on the decision sample reachesR 2 =0.18 on held-out humans, while the generated sample gives−0.26. For this model name, the decision interface keeps the marginals and loses much of the correlation pattern. It also adds agreement and keeps the relation between the composite scales better. 5.4 The readout decides which component is sacrificed Table 3 and Figures 2 and 3 show the same vectors read three ways. The rounded expected score and the most probable category give similar correlation because both are deterministic functions of the same vector. They differ on the distribution and mediation components. Jev’s gain from 0.681 to 0.786 comes mostly from the mediation component (0.498 to 0.972), which is the noisiest component. Sampling behaves differently. It moves the distribution component of every model to 0.83– 0.96, close to the copula, because each answer is now a draw from a full distribution. Sampling also lowers the correlation component of every model, by more than half for seven of them, because each item is drawn independently given the profile. Mercury Decide loses the least. It keeps a correlation component of 0.463 when sampled and reaches the highest sampled PSS (0.761), above the generative median. Part of this loss follows from the format. When the item vectors are independent given the profile, the item correlations of a sample can only come from the profile. Tev1 4B gives a check of this independence. When its unchanged profiles are sent again in three separate requests, 98% of the rounded answers stay the same (Section 7). The vector for one item therefore depends little on the other items in the request. The profile regression shows how 10 Table 2: Paired differences to the best generative cell, MiniMax M3 (aligned). Brackets give the point estimate±1.96 bootstrap standard errors of the paired difference over 200 draws, clipped to [−1,1]. Every draw resamples the 100 respondents and the 263 humans once and applies the same draw to both sources. R = rounded expected score, A = most probable category. An interval that contains zero means the difference is within resampling noise. The reference was chosen as the highest point estimate among 47 cells, so the differences to it are biased away from zero. SourceReadout∆ Correlation∆ Mediation∆ PSS Same 100 humans (oracle)+0.16 [+0.08, +0.23] −0.29 [−0.62, +0.04]+0.07 [0.00, +0.15] Gaussian copula (in-sample)+0.12 [+0.05, +0.20] −0.03 [−0.24, +0.18] +0.11 [+0.06, +0.17] Gaussian copula (out-of-sample)+0.04 [−0.06, +0.15] −0.28 [−0.55, −0.01] +0.01 [−0.07, +0.09] MiniMax M2.7+0.04 [−0.01, +0.10] −0.24 [−0.62, +0.14]0.00 [−0.09, +0.08] Kimi K2.7 Code+0.04 [−0.01, +0.09] −0.20 [−0.53, +0.12] −0.01 [−0.08, +0.06] gpt-5.4-mini−0.16 [−0.24, −0.09] −0.07 [−0.29, +0.15] −0.07 [−0.13, −0.01] GPT-6 Luna−0.20 [−0.28, −0.12] −0.54 [−0.86, −0.22] −0.19 [−0.27, −0.12] Jev 1.13R−0.10 [−0.19, −0.02] −0.50 [−0.90, −0.10] −0.13 [−0.21, −0.05] Jev 1.13A−0.10 [−0.17, −0.03] −0.03 [−0.33, +0.28] −0.03 [−0.10, +0.04] Decider v1 27BR−0.42 [−0.49, −0.34] −0.06 [−0.34, +0.21] −0.15 [−0.22, −0.09] Decider v1 27BA−0.42 [−0.50, −0.34] −0.55 [−0.91, −0.20] −0.25 [−0.33, −0.17] Decider v1.1 27BR−0.42 [−0.49, −0.35] −0.04 [−0.29, +0.20] −0.15 [−0.21, −0.09] Decider v1.1 27BA−0.42 [−0.51, −0.33] −0.45 [−0.87, −0.03] −0.23 [−0.32, −0.15] Liquid d1R−0.23 [−0.33, −0.13] −0.69 [−1.00, −0.36] −0.19 [−0.27, −0.12] Liquid d1A−0.24 [−0.32, −0.17] −0.48 [−0.84, −0.11] −0.14 [−0.22, −0.06] Kev 4BR−0.54 [−0.61, −0.47] −0.73 [−1.00, −0.34] −0.34 [−0.42, −0.25] Kev 4BA−0.57 [−0.64, −0.50] −0.29 [−0.72, +0.14] −0.27 [−0.36, −0.17] Tev1 4BR−0.52 [−0.61, −0.43] −1.00 [−1.00, −0.77] −0.35 [−0.41, −0.29] Tev1 4BA−0.59 [−0.68, −0.50] −0.64 [−0.92, −0.35] −0.29 [−0.35, −0.22] Mercury DecideR−0.23 [−0.30, −0.16] −0.66 [−1.00, −0.30] −0.18 [−0.26, −0.10] Mercury DecideA−0.29 [−0.36, −0.22] −0.69 [−1.00, −0.29] −0.19 [−0.27, −0.11] Solar Decide FlashR−0.62 [−0.71, −0.53] −0.70 [−1.00, −0.40] −0.39 [−0.46, −0.32] Solar Decide FlashA−0.53 [−0.63, −0.43] −0.13 [−0.56, +0.31] −0.36 [−0.48, −0.23] ClefR−0.22 [−0.30, −0.14] −0.91 [−1.00, −0.57] −0.25 [−0.32, −0.18] ClefA−0.22 [−0.29, −0.15] −1.00 [−1.00, −0.75] −0.24 [−0.30, −0.19] Clef FlashR−0.14 [−0.20, −0.08] −1.00 [−1.00, −0.79] −0.26 [−0.31, −0.21] Clef FlashA−0.12 [−0.18, −0.06] −0.76 [−1.00, −0.41] −0.19 [−0.27, −0.12] Clef OmniR−0.26 [−0.34, −0.19] −0.78 [−1.00, −0.46] −0.26 [−0.33, −0.20] Clef OmniA−0.33 [−0.40, −0.25] −0.82 [−1.00, −0.45] −0.27 [−0.35, −0.19] GPT-6 Luna DecisionsR−0.42 [−0.53, −0.32] −0.59 [−0.97, −0.22] −0.27 [−0.35, −0.19] GPT-6 Luna DecisionsA−0.37 [−0.47, −0.28] −0.51 [−0.88, −0.14] −0.25 [−0.33, −0.16] Microsoft-Decision-1R−0.36 [−0.44, −0.28] −0.21 [−0.54, +0.12] −0.15 [−0.22, −0.07] Microsoft-Decision-1A−0.40 [−0.48, −0.31] −0.35 [−0.69, −0.01] −0.19 [−0.27, −0.11] Drex v1.5R−0.30 [−0.37, −0.23] −0.28 [−0.63, +0.07] −0.15 [−0.23, −0.07] Drex v1.5A−0.44 [−0.52, −0.36]0.00 [−0.20, +0.20] −0.13 [−0.18, −0.08] much of the correlation pattern the profile alone supports. Read out in the same three ways, it reaches correlation components of 0.23, 0.14 and 0.03 (last row of Table 3). Under the rounded readout, 12 of the 14 decision models are above it. Decision models thus reproduce more of the correlation pattern than the profile alone supports, but less than most generative models. The weighted total hides these trade-offs. An analyst who picks the readout after seeing PSS can choose which property of the sample to give up. This is one form of the analytic flexibility that Cummins[14]found for silicon samples, where configurations that did well on one criterion often did poorly on another. For this reason the readout should be fixed and reported before scoring. 5.5 What the probability vectors contain Table 4 shows what the probability vectors contain. First, they carry population information. Averaged over respondents, they are close to the human item marginals (Jensen-Shannon divergence 0.037–0.156), much closer than the rounded answers (0.218–0.411). Second, only 0.024–0.304 of the answer variance depends on the respondent. In rounded answers, the share of variance that is not an item main effect is 0.145–0.495, against 0.805 for the same 100 humans and a median of 0.53 for the generative LLMs. Third, twelve of the fourteen models are less certain on reverse-keyed items, and for every model the reverse-keyed items 11 Table 3: The same probability vectors read out three ways (aligned, C3,n=100): R is the expected score rounded to the nearest category, A is the most probable category, S is one category drawn from the vector per item (mean of five draws; the SD of PSS across draws is at most 0.040). Bold marks the best readout per model and metric. For reference, the median of 47 generative LLMs is 0.695 for PSS, 0.73 for correlation, 0.75 for distribution and 0.46 for mediation; the best generative cell (MiniMax M3) has PSS 0.813. The last row reads the forecasts of the profile regression, fit on the 163 humans outside the sample, in the same three ways. PSSCorrelationDistributionMediation ModelRASRASRASRAS Jev 1.130.681 0.786 0.700 0.66 0.67 0.35 0.77 0.78 0.91 0.50 0.97 0.83 Microsoft-Decision-10.667 0.622 0.665 0.40 0.37 0.27 0.68 0.68 0.85 0.79 0.65 0.86 Decider v1.1 27B0.660 0.579 0.517 0.34 0.34 0.07 0.63 0.66 0.95 0.95 0.55 0.73 Drex v1.50.659 0.682 0.654 0.46 0.32 0.23 0.70 0.72 0.92 0.72 1.00 0.85 Decider v1 27B0.659 0.560 0.519 0.35 0.35 0.07 0.63 0.66 0.95 0.93 0.44 0.73 Mercury Decide0.636 0.623 0.761 0.53 0.48 0.46 0.78 0.79 0.91 0.34 0.31 0.87 Liquid d10.618 0.671 0.661 0.53 0.52 0.23 0.75 0.81 0.96 0.31 0.52 0.89 Clef0.563 0.571 0.615 0.54 0.54 0.13 0.69 0.75 0.95 0.09 0.00 0.89 Clef Flash0.554 0.622 0.688 0.62 0.65 0.35 0.70 0.75 0.92 0.00 0.24 0.83 Clef Omni0.551 0.539 0.654 0.50 0.44 0.24 0.72 0.74 0.92 0.22 0.18 0.87 GPT-6 Luna Decisions 0.547 0.568 0.609 0.34 0.39 0.21 0.68 0.69 0.86 0.40 0.49 0.84 Kev 4B0.477 0.548 0.513 0.23 0.19 0.06 0.61 0.64 0.91 0.27 0.70 0.74 Tev1 4B0.460 0.527 0.614 0.24 0.17 0.16 0.70 0.70 0.91 0.00 0.36 0.90 Solar Decide Flash0.423 0.458 0.451 0.15 0.24 0.05 0.59 0.54 0.83 0.29 0.87 0.76 Profile regression0.486 0.616 0.484 0.23 0.14 0.03 0.64 0.74 0.99 0.39 0.98 0.64 0.40.50.60.70.80.9 PSS (aligned, C3, n = 100) Solar Decide Flash Tev1 4B Kev 4B GPT-6 Luna Decisions Clef Omni Clef Flash Clef Liquid d1 Mercury Decide Decider v1 27B Drex v1.5 Decider v1.1 27B Microsoft-Decision-1 Jev 1.13 most probable category sampled category (mean of 5) median generative LLM best generative (MiniMax M3) copula, out-of-sample same 100 humans rounded expected score Figure 2: PSS of each decision model under three readouts of the same probability vectors. Bars show the rounded expected score with±1.96 bootstrap standard errors. Vertical lines mark the median and the best of the 47 generative LLMs, the out-of-sample copula and the real answers of the same 100 people. keep the larger divergence after alignment (Appendix Table 14). For Mercury Decide and Microsoft-Decision-1 the difference in entropy is below 0.01, and Clef Flash is more certain on reverse-keyed items. A small share does not mean that the person signal is missing. The profile itself explains little of how people answer. A regression on the profile, fit on 163 real people, gives a between-respondent share of only 0.012, and every decision model is above it (2 to 25 times as large). The vectors also rank respondents about as well as this regression. For the decision models the mean item-level Spearman correlation between the expected score and the person’s answer is 0.10–0.19, 0.15 [0.08, 0.22] for the profile regression and 0.10–0.20 12 0.20.40.60.8 Correlation component 0.6 0.7 0.8 0.9 1.0 Distribution component gpt-5.4-mini copula (out-of-sample) same 100 humans rounded most probable sampled generative LLMs Clef Clef Flash Clef Omni Mercury Decide Kev 4B Liquid d1 Microsoft-Decision-1 Drex v1.5 GPT-6 Luna Decisions Decider v1 27B Decider v1.1 27B Tev1 4B Jev 1.13 Solar Decide Flash Figure 3: Sampling raises the distribution component and lowers the correlation component for every decision model. Each colour is one decision model, and lines connect its three readouts. Grey points are the 47 generative LLMs, the star is gpt-5.4-mini, and the crosses mark the out-of-sample copula and the real answers of the same 100 people. for the single answers of the generative models. Composites built from the expected scores correlate with the same people’s real composites at a medianrof 0.21–0.36, against 0.24 for the regression. Taken together, decision models behave like a population model of item responses with a person signal on top. The signal ranks people about as well as the profile regression does, but it moves them further apart than the profile supports. Rounding turns this signal into whole scale steps, and sampling adds item-level noise that is larger than the signal. These two effects explain the readout pattern in Section 5.4. 5.6 Correlation level and factor structure The correlation component compares the pattern of two correlation matrices. Table 5 adds their level and the factor structure. Generative LLMs make the items correlate too strongly. Their median mean inter-item correlation is 0.36, against 0.22 for the 263 humans and 0.25 for the same 100 people, and 42 of the 47 cells are above the value of these 100 people. The first eigenvalue carries a median share of 0.49 of the variance, against 0.28 and 0.33, and the median subscaleαis 0.88 against 0.81. The original audit described the same over-coherence. Most decision models stay closer to the human level of correlation (median 0.28, range 0.13–0.44). They separate the subscales less well, however. In humans the mean correlation within a subscale exceeds the mean correlation between subscales by 0.16. For 13 of the 14 decision models this margin is smaller, down to 0.02. Tucker’sφgives the same picture. Its median over the four instruments is 0.39–0.79 for the decision models, 0.82 for the median generative LLM and 0.97 for the same 100 people. Every decision model is below 0.85, the usual threshold for a fair similarity of factor structures [32]. The expected scores show why. In them the first eigenvalue carries 0.52–0.85 of the variance, so the person signal of the vectors moves nearly all items in one direction. The profile sets a general level for each respondent, and the vectors add little that is specific to a subscale. This gives a more precise reading of the second hypothesis. In most decision samples the item correlations are close to the human level, but they follow one general direction instead of the subscale structure. 13 Table 4: What the probability vectors contain (aligned, C3,n=100). Entropy is normalised by the log of the number of categories, for forward and reverse-keyed items. Between is the share of the model’s answer variance (between-respondent variance of the expected score plus mean within-respondent variance of the vector) that depends on the respondent. Person is the share of respondent-by-item variance in rounded answers that is not an item main effect. SD ratio is the median item SD relative to all 263 humans. Unique counts distinct 68-item answer vectors. The last two columns give the mean item Jensen-Shannon divergence to the human marginal for the rounded answers and for the average probability vector. EntropyJS to human ModelFwd. Rev. Max p Between Person SD ratio Unique Rounded Mixture Mercury Decide0.61 0.610.580.300.490.59990.2180.085 Jev 1.130.67 0.740.530.160.490.461000.2310.078 Tev1 4B0.81 0.870.400.100.400.441000.2960.082 Clef Flash0.82 0.790.420.120.390.46940.3000.068 Drex v1.50.82 0.840.390.120.380.45990.2980.076 Liquid d10.77 0.880.400.090.380.41990.2510.037 GPT-6 Luna Decisions0.47 0.660.650.130.360.32990.3240.130 Clef Omni0.75 0.780.430.120.320.44980.2830.070 Microsoft-Decision-10.72 0.720.470.180.300.481000.3160.137 Clef0.82 0.840.400.050.280.28800.3120.038 Kev 4B0.88 0.910.360.030.250.26930.3930.080 Decider v1 27B0.81 0.900.400.020.200.24910.3690.040 Decider v1.1 27B0.81 0.900.400.020.200.24930.3700.040 Solar Decide Flash0.75 0.770.470.040.150.351000.4110.156 gpt-5.4-mini (LLM)0.200.2895 GPT-6 Luna (LLM)0.40 Median of 47 generative LLMs0.53 Same 100 humans0.811.03100 5.7 The person signal is stable and comes from the profile Table 6 reports the three control grids. The repeated requests show that most decision models are deterministic. Both Decider versions, the three Clef models and GPT-6 Luna Decisions returned the same vectors, and Kev 4B, Tev1 4B, Mercury Decide and Drex v1.5 changed them only in the second decimal. Four models vary between runs. Of their rounded answers, 95% repeat for Jev 1.13, 92% for Liquid d1, 95% for Microsoft-Decision-1 and 89% for Solar Decide Flash. Even for these four models, run-to-run noise is small next to the differences between respondents. At least 0.82 of the between-respondent variance of the expected score repeats, and the composite scores of the two runs correlate at 0.95 or more. The person signal of Section 5.5 is a stable property of the answers. Without a profile, the deterministic models give every respondent the same 68 answers. Their rounded and most probable readouts then have no item variance, and the rounded PSS falls to 0.11–0.37. Jev 1.13 returned 99 different rounded vectors to 100 identical requests, but this variation equals only 0.02 of its between-respondent variance at C3. Solar Decide Flash is noisier. It returned 100 different rounded vectors, and their variation equals 0.52 of its between-respondent variance at C3. The sampled readout varies at C0 as well. Its PSS rises from 0.39–0.43 at C0 to the C3 values, a median gain of 0.22 (range 0.06 to 0.33), so decision models do read the profile. The generative models of the original audit gain a median of 0.16 between the same two conditions, but the two gains are not directly comparable. Sampling at C0 removes all item correlation from a decision model, while generative models keep some at C0. On average the profile moves the expected score by 0.04–0.18 of the scale range. Decision and generative models differ in the size of this shift relative to the spread inside each vector. The profile also carries part of the population information. For 12 of the 13 models with a C0 grid, the C0 vector is further from the human marginals than the vector averaged over the 100 C3 profiles. The close match of the averaged vectors in Table 4 depends on averaging over real profiles as well as on the item text. For both Decider versions, Clef, Clef Omni and Kev 4B the gap is small, and for Mercury Decide the C0 vector is slightly closer. 14 Table 5: Internal structure of each sample (aligned answers, rounded readout for language models, the same 100 profiles). Meanris the mean inter-item correlation over all 68 items. λ 1 is the share of the largest eigenvalue of the item correlation matrix. W−B is the mean within-subscale minus the mean between-subscale item correlation inside an instrument, averaged over the four instruments.αis the median Cronbach’sαof the 12 subscales. φis the median over the four instruments of Tucker’sφagainst the 263 humans, with three varimax factors per instrument. Null counts the instruments on whichφis above an item-permutation null (K=500,p<0.05). Items without variance in a sample are left out of itsφ.λ 1 (E) is the same eigenvalue share for the expected scores of a decision model. The oracle is part of the 263-person reference, which favours it. SourceMean r λ 1 W−B α φ Null λ 1 (E) Humans (n=263, reference)0.22 0.28 0.16 0.81 1.004 Same 100 humans (oracle)0.25 0.33 0.15 0.81 0.974 Copula, in-sample0.21 0.27 0.12 0.79 0.964 Copula, out-of-sample0.15 0.20 0.14 0.79 0.913 MiniMax M3 (best LLM)0.37 0.44 0.16 0.89 0.873 gpt-5.4-mini0.23 0.29 0.19 0.72 0.692 GPT-6 Luna (LLM)0.24 0.32 0.16 0.77 0.783 Median of 47 generative LLMs0.36 0.49 0.17 0.88 0.823 Jev 1.130.41 0.47 0.13 0.88 0.783 0.68 Decider v1 27B0.23 0.25 0.12 0.58 0.580 0.60 Decider v1.1 27B0.23 0.24 0.11 0.60 0.601 0.60 Liquid d10.31 0.38 0.10 0.79 0.721 0.65 Kev 4B0.21 0.26 0.07 0.53 0.661 0.58 Tev1 4B0.32 0.42 0.02 0.78 0.581 0.71 Mercury Decide0.30 0.48 0.24 0.88 0.721 0.62 Solar Decide Flash0.13 0.32 0.02 0.68 0.390 0.65 Clef0.32 0.36 0.11 0.79 0.721 0.76 Clef Flash0.42 0.49 0.14 0.85 0.762 0.83 Clef Omni0.44 0.47 0.10 0.81 0.790 0.85 GPT-6 Luna Decisions0.23 0.29 0.04 0.66 0.683 0.61 Microsoft-Decision-10.25 0.37 0.16 0.75 0.642 0.52 Drex v1.50.24 0.36 0.14 0.75 0.633 0.56 5.8 Counterfactual swaps on three axes Table 7 reports the swaps. For role and education the decision models move the composites in the direction of the human contrast in 96% and 93% of model-composite pairs (100% and 100% for the generative models). In humans and in models, managers and university graduates report more positive attitudes to change, more engagement and higher contextual performance. The size of the reaction differs between the two classes. Measured in human SD units, the decision models shift by 0.12–0.62 after a role swap (median 0.22, 95% bootstrap interval 0.20–0.25) and by 0.12–0.37 after an education swap (median 0.22). The 37 generative models shift by a median of 0.47 after a role swap and 0.32 after an education swap. The observed human difference is 0.46 [0.31, 0.62] for role, where 7 of 12 contrasts survive a Holm correction. For education it is 0.44 [0.20, 1.04], but it rests on 15 people with secondary education, and none of its contrasts survives the correction. Generative models reproduce the role difference at about its observed size. The interval for the median decision model lies below the interval of the human role contrast. Of the 13 decision models, 7 reach less than half of it, and 5 shift less than the weakest generative model. Role moves the decision models more than education in only 6 of 13 models, while the original audit found role to be the stronger axis in 34 of 37 generative models. The gender swap moves the composites by 0.01–0.16 human SD in the decision models (median 0.03) and by a median of 0.06 in the generative models. In humans the gender contrast is null, and the direction of the model shifts agrees with the sign of the null human estimates in about half of the composites (47% and 50%). Neither class adds a gender effect of practical size. 15 Table 6: Control grids for the decision models. Same is the percentage of rounded answers that are identical when 20 respondents are sent again. Repl. is the share of between- respondent variance of the expected score that is not run-to-run noise. Shift is the mean absolute difference between the C3 expected score and the no-profile (C0) expected score on a 0–1 scale. PSS uses the sampled readout, which is the only readout that gives varying answers at C0, averaged over the same five draws as in Table 3. The JS columns give the mean item Jensen-Shannon divergence between the human marginal and the average probability vector. The generative row uses the C0 and C3 cells of the original audit. GPT-6 Luna Decisions returned HTTP 502 for every request without a profile. RetestProfileJS to human ModelSame Repl. Shift PSS C0 PSS C3C0C3 Jev 1.1395 0.99 0.140.400.70 0.154 0.078 Microsoft-Decision-195 0.98 0.160.390.66 0.206 0.137 Decider v1.1 27B100 1.00 0.040.430.52 0.046 0.040 Drex v1.599 1.00 0.180.400.65 0.156 0.076 Decider v1 27B100 1.00 0.040.430.52 0.046 0.040 Mercury Decide99 1.00 0.130.430.76 0.081 0.085 Liquid d192 0.94 0.080.420.66 0.055 0.037 Clef100 1.00 0.060.430.61 0.047 0.038 Clef Flash100 1.00 0.180.400.69 0.199 0.068 Clef Omni100 1.00 0.100.430.65 0.075 0.070 GPT-6 Luna Decisions100 1.000.610.130 Kev 4B100 1.00 0.090.420.51 0.091 0.080 Tev1 4B98 1.00 0.140.400.61 0.166 0.082 Solar Decide Flash89 0.82 0.140.390.45 0.251 0.156 Median of 37 generative LLMsgain +0.16 The paireddof the original audit, with each difference signed by the direction of the swap, can reverse the order of the two classes. For education it is 1.06 for the median decision model against 0.68 for the median generative model, and for gender it is 0.31 against 0.17. The pairedddivides the mean shift by the spread of the shifts across respondents. A nearly deterministic model shifts every respondent in almost the same way, so this spread is small anddis large. The paireddthus measures how uniform a reaction is rather than how large it is. It should not be compared across models with different levels of answer noise. 5.9 The vectors as forecasts of each person’s answers Table 8 scores the probability vectors against the answers of the same people. The profile itself carries little information about a person’s answers. A regression on the profile fields beats the population distribution of other respondents by only 1.0% [0.1, 1.8]. No decision model reaches the population distribution. Their vectors are 6–55% worse (median 20%), and the interval lies below zero for all 14 models. They are still much better than the uniform distribution, which is 42% worse. Rounding a vector to one answer loses most of this advantage. The rounded answers are 48–105% worse than the population distribution. Most of this loss comes from the average vector. Given to every respondent, the model’s own average vector is 7–51% worse than the population distribution (column Own avg.), about as bad as the individual vectors. For some models the person-specific part adds a further loss, because the vectors spread respondents further apart than the profile supports (Section 5.5). The log score gives the same verdict. All 14 decision models are worse than the population distribution under it, by 7–62%, and GPT-6 Luna Decisions, Solar Decide Flash, Microsoft-Decision-1 and Mercury Decide lose most. All of them except Solar Decide Flash are among the four most confident models. We mixed each vector with the population distribution. Cross-fitting chose a weight of 0.05–0.25 on the vector. This brings the skill to −0.5 to 0.9%, close to the profile regression. No shrunk vector has a skill interval that lies above zero. 16 Table 7: Counterfactual swaps of one profile field, averaged over the 12 composites.|Shift|is the mean absolute shift toward the first group of the human contrast (university, managerial, woman) in human SD units.|d|is the mean absolute paireddof the original audit, with each difference signed by the direction of the swap. Decision models use the rounded readout. Generative rows use the swap grids of the original audit. The human row gives the mean|d| of the observed contrast between the two groups among all 263 humans, which is on the same scale as|Shift|. The education contrast rests on 15 respondents with secondary education, and the gender contrast is null. Section 5.8 gives bootstrap intervals for the medians and the human contrasts. Decider v1 27B had no available endpoint when the role and gender swaps were run. EducationRoleGender Model|Shift| |d| |Shift| |d| |Shift| |d| Jev 1.130.33 1.110.22 0.920.01 0.11 Microsoft-Decision-10.30 1.360.33 1.540.03 0.17 Decider v1.1 27B0.15 1.000.18 1.430.03 0.34 Drex v1.50.37 1.390.28 1.280.08 0.54 Decider v1 27B0.15 1.01 Mercury Decide0.21 0.920.19 0.500.16 0.46 Liquid d10.36 1.290.32 1.170.04 0.19 Clef0.15 0.890.14 0.830.02 0.31 Clef Flash0.23 1.190.62 2.780.04 0.40 Clef Omni0.15 0.830.49 2.330.03 0.24 GPT-6 Luna Decisions0.12 0.620.16 0.850.02 0.16 Kev 4B0.25 1.100.17 0.880.08 0.58 Tev1 4B0.31 1.440.31 1.700.10 0.72 Solar Decide Flash0.20 0.940.12 0.670.03 0.17 Median of decision models0.22 1.060.22 1.170.03 0.31 Median of 37 generative LLMs0.32 0.680.47 1.050.06 0.17 Humans, observed contrast0.440.460.11 The profile improves the forecast for 11 of the 13 models with a C0 grid. The individual vectors beat the model’s own average vector in 7 of 14 models, by a median of 0.3 percentage points. Most of the benefit of the profile thus comes from moving the average vector closer to the population, which agrees with the C0 result of Section 5.7. Differences between the vectors of different respondents do carry information about the person, as the rank correlations show. Without recalibration this information adds little to the forecast, because the vectors spread respondents too far apart. Figure 4 adds the generative models on the panel. Their repeat distributions are 41–104% worse than the population distribution (median 57%). Only a small part of this gap comes from the estimator. Twenty draws from a decision vector, scored in the same way, are 12–64% worse, a median loss of 5 points against the full vector. Most of the gap comes from the repeat distributions themselves, which are too narrow. Their median entropy is 0.32 nats, against 1.36 for the population distribution and 1.37 for the decision vectors. A single answer, which is what a generative survey run uses, is worse still (median 78%). In all 31 generative models, the pooled answers of the panel forecast each person better than that person’s own repeats, by a median of 27 points. Shrinking each person’s repeats toward the model’s pooled answers, with the weight again chosen by cross-fitting, gives−0.3 to 1.8% (median 0.7%). Decision vectors are the better forecasts before recalibration. Once both classes are recalibrated, neither does much better than the profile regression, which uses the same fields. 5.10 Newer generative models do not score higher The data do not support the fourth hypothesis. The 11 generative models released after the original audit have a median PSS of 0.641, against 0.701 for the 36 older cells (Mann-Whitney p=0.06). Without the mediation component the medians are 0.734 and 0.765 (p=0.09). The 17 Table 8: Probability vectors scored as forecasts of the answers of the same 100 people. Skill is 100 (1− RPS/RPS pop ), where RPS is the ranked probability score averaged over items and respondents andRPS pop = 0.128 is the score of the population distribution of the 163 humans outside the sample. Negative values are worse than this distribution. Brackets give 95% paired bootstrap intervals over respondents. Shrunk mixes each vector with the population distribution, with the weight chosen by leave-one-respondent-out cross-fitting. Own average gives every respondent the model’s mean vector over all respondents. C0 uses the vectors returned without a profile, and Rounded scores the rounded expected score as a certain forecast.ρis the mean item-level Spearman correlation between the expected score and the person’s answer. Entropy is the mean entropy of the forecast in nats. The profile regression is a multinomial logistic regression of each item on the profile fields, fit on the same 163 humans. ForecasterSkill (%) Shrunk Own avg. C0 Rounded ρ Entropy Clef−6 [−10, −2]0.6−7 −9 −48 0.151.40 Liquid d1−8 [−12, −3]0.9−9 −16 −51 0.161.35 Decider v1.1 27B−8 [−12, −4]0.1 −10 −10 −55 0.141.40 Decider v1 27B−8 [−12, −4]0.1 −10 −10 −55 0.141.41 Clef Omni−16 [−23, −10]0.8 −17 −16 −58 0.181.29 Clef Flash−18 [−26, −11]0.7 −18 −75 −65 0.191.39 Jev 1.13−18 [−25, −12]0.7 −15 −40 −57 0.171.17 GPT-6 Luna Decisions −22 [−26, −17]0.0 −21−55 0.160.88 Kev 4B−22 [−29, −15] −0.4 −23 −29 −73 0.111.50 Tev1 4B−23 [−30, −17]0.1 −25 −51 −70 0.181.40 Drex v1.5−24 [−32, −16] −0.1 −22 −49 −72 0.131.41 Mercury Decide−32 [−40, −25]0.4 −21 −24 −75 0.131.05 Solar Decide Flash −49 [−58, −41] −0.5 −49 −69 −104 0.111.29 Microsoft-Decision-1 −55 [−67, −43]0.0 −51 −81 −105 0.101.23 Profile regression1.0 [0.1, 1.8]0.151.35 Population distribution01.36 Uniform−42 [−53, −32]1.69 best new model, Claude Sonnet 5.5 (0.735), ranks 15th of 47. GPT-6.1 Sol (0.510) is last of all 47 cells, and GPT-6 Luna (0.621) ranks 40th. Table 9 compares releases within one product line, which holds the vendor and the training lineage roughly fixed. For each line we compare the newest release with the newest release of the same line in the original audit. In 8 of 10 lines the newest release scores lower, 1 is level and 1 gains (sign testp=0.04). The median change is−0.08. For 5 lines the paired interval lies below zero, and for none it lies above zero. Three lines also change size tier (Table 9). Without them, the newest release is lower in 5 of the 6 lines that change (sign testp=0.22). Claude Opus 5.5 (0.606) is below Claude Opus 4.8 (0.740), which was already below Claude Opus 4.7 (0.789). Kimi K3 (0.641) is below Kimi K2.7 Code (0.802). Without the mediation component the newest release is lower in 9 of 10 lines, so the direction of the result does not depend on the noisiest component. The size of single drops, such as those in the Claude Opus and Kimi lines, comes mostly from the mediation component. One decision model also has a successor in our set. Its provider describes Decider v1.1 27B as the successor of Decider v1 27B. The two versions return nearly the same vectors, and 98% of their rounded answers agree. Under the rounded readout their PSS is 0.659 and 0.660, a paired difference of 0.00 [−0.05, +0.05]. The correlation component changes by −0.01 [−0.03, +0.02]. This release therefore changed the answers very little. The new models differ from the old ones mainly in response style. Their median item SD relative to humans is 0.46 against 0.56 (p=0.01). The share of extreme answers is also lower (0.02 against 0.04,p=0.09), but this difference is within noise. New models may individualise less (person share 0.47 against 0.55,p=0.05), and their demographic component falls as well (0.37 against 0.45,p=0.02). The differences in the correlation (0.70 against 0.73) and distribution components (0.72 against 0.76) point the same way but are smaller than the noise. On predictive validity the new models do slightly better (medianR 2 0.15 against −0.06,p=0.11). These are exploratory comparisons of two small groups. None of the 10 18 −120−100−80−60−40−200 Forecast skill relative to the population distribution (%) Generative, single answer Generative, 20 repeats Generative, own average Decision, rounded answer Decision, no profile (C0) Decision, 20 draws from vector Decision, vector population distribution (+0.0%) profile regression (+0.5%) uniform (-43.1%) Figure 4: No language model forecasts a person’s answers better than the answer frequencies of other respondents. Forecast skill on the 26 to 30 respondents of the test-retest panel, relative to the population distribution of the 163 humans outside the headline sample. Each point is one model. Vertical lines mark the population distribution (zero), the profile regression and the uniform distribution. The second row draws 20 answers from each decision vector and scores them like the generative repeats. comparisons of release waves in this section survives a Holm correction (smallest adjusted p=0.14). The pattern suggests that newer models answer in a narrower and more uniform way, which costs fidelity on most components. 5.11 Response style and downstream use All language models compress the scale (Table 10). The share of extreme answers is 0.270 for humans, 0.024 for gpt-5.4-mini and at most 0.196 for the decision models. Every model uses the midpoint at least as often as humans, and most use it clearly more. On the balanced acquiescence index the median generative LLM shifts by 0.21 human SD and decision models by−0.27 to +3.54. All decision models except Clef Flash and Clef Omni shift more than gpt-5.4-mini (+0.21). For Kev 4B, both Decider versions, Drex v1.5 and Microsoft-Decision-1 almost all of this shift comes from the reverse-keyed items, and for Tev1 4B and GPT-6 Luna Decisions most of it does (columns Fwd and Rev). Their answers to reverse-keyed statements agree more than human answers do, while their forward-item means are closer to the human means. This pattern fits misresponse to reversed and negated items better than a general tendency to agree [52]. Jev 1.13, Liquid d1, Mercury Decide and Solar Decide Flash agree more than humans on both kinds of item, Solar Decide Flash by more than three human SD. Microsoft-Decision-1 is an extreme case of misresponse. Its reverse-item mean lies 6.2 human SD above the human value, and its keying gap is 0.21 against 2.00 in humans. On DUN_02 (“I usually resist new ideas”) its mean expected score is 3.86, while the human mean is 1.86. It thus answers most negated statements as if they were worded positively. It is also the only model whose answers to reverse-keyed items are closer to the human answers before alignment than after (Appendix Table 14). Every model has a smaller keying gap than humans, which means that models separate forward and reverse-keyed wording less sharply than people do. Table 11 reports two tests of practical use. A regression trained on the copula sample predicts held-out humans as well as one trained on humans (R 2 0.29 against 0.28). Among decision models only Jev matches this level (0.30). Decider v1 (0.19), GPT-6 Luna Decisions (0.18) and Decider v1.1 (0.15) are next. Generative LLMs vary widely. Only 25 of 47 beat predicting the mean, the median is 0.10, and 6 reach 0.25 or more, with a maximum of 0.30. The aligned gpt-5.4-mini sample givesR 2 =-1.79. In this sample attitudes and performance correlate at 0.748, against 0.372 in humans. Attitudes also predict performance more strongly than engagement does (0.567 for engagement), which is the reverse of the human order 19 Table 9: Aligned PSS of successive releases in each product line with a release after the original audit, oldest first. ∆ compares the two releases in bold: the newest release with the newest release of the same line in the original audit. Brackets give the 95% paired bootstrap interval. In 8 of 10 lines the newest release scores lower, 1 is level and 1 gains. In 5 lines the interval lies below zero. No M is ∆ for PSS without the mediation component, with the other weights rescaled; it is negative in 9 lines. The DeepSeek, Moonshot Kimi and OpenAI small lines change tier as well as version. MiniMax has no release after the original audit, and Qwen 3.7 Plus has too few complete respondents to be scored. LineReleases (PSS)∆95% interval No M Anthropic OpusClaudeOpus4.70.79→ Claude Opus 4.8 0.74→ Claude Opus 5.5 0.61 −0.13 [−0.20, −0.07] −0.02 Anthropic SonnetClaude Sonnet 4.6 0.59→Claude Son- net 5 0.74 → Claude Sonnet 5.5 0.73 0.00 [−0.08, +0.07]0.00 OpenAI flagshipGPT-5.4 0.61→GPT-5.5 0.61→GPT- 6.1 Sol 0.51 −0.10 [−0.17, −0.03] −0.10 OpenAI smallgpt-5.4-mini 0.74→GPT-6 Luna 0.62−0.12 [−0.19, −0.05] −0.03 Google FlashGemini 3 Flash 0.70→Gem- ini 3.5 Flash 0.74→Gem- ini 3.6 Flash 0.64→Gem- ini 3.8 Flash 0.71 −0.02 [−0.10, +0.05] −0.01 xAI GrokGrok 4.3 0.71 → Grok 4.7 0.61−0.10[−0.19, 0.00] −0.04 DeepSeekDeepSeek3.20.66→ DeepSeek V4 Pro 0.75→ DeepSeek V4.1 Flash 0.70 −0.05 [−0.15, +0.06] −0.06 Moonshot Kimi Kimi K2.6 0.75→Kimi K2.7 Code 0.80 → Kimi K3 0.64 −0.16 [−0.23, −0.09] −0.07 Alibaba Qwen Qwen3.5397B0.78→ Qwen 3.8 Max 0.72 −0.06 [−0.15, +0.03] −0.03 Zhipu GLMGLM 5.1 0.71→GLM 5.2 0.64→ GLM 5.3 0.70 +0.06[0.00, +0.12] −0.04 (0.475). A regression fit on this sample gives attitudes far too much weight for real people. This test uses a single human split and 100 training rows, so differences of less than about 0.1 should not be over-read. Fabricated structure is common in the null-mediation test. The copula produces no false paths. Of the 7 triples, generative LLMs fabricate between 0 and 6 (median 2), gpt-5.4-mini 6 and decision models between 0 and 6. No false path appears for 6 generative models and for both Decider versions. Clef and GPT-6 Luna Decisions show one each. The intervals of these four models are as narrow as those of the other models, so their low counts are not a matter of low power. Atn=100 a single triple can change the count, and the ranking between models should be read with this in mind. Appendix F reports the agreement between models and the sensitivity of the ranking to the PSS weights. 5.12 The four hypotheses The probability vectors support the first hypothesis. Averaged vectors come much closer to the human marginals than any rounded answers. Sampled answers reach distribution components of 0.83–0.96, above most generative models (0.618–0.903), and this is the fair comparison because both sides are then samples of answers. With the rounded readout, only 2 of 14 decision models exceed the generative median (0.749). The second hypothesis also holds. No decision model under a deterministic readout exceeds a correlation component of 0.667, and 33 of the 47 generative cells are above this value. This follows partly from the format, as the profile regression shows (Section 5.4). Decision models also reproduce the factor structure less well, because their person signal has nearly one direction (Section 5.6). 20 Table 10: Response style on a common raw orientation (answers to the original wording for humans and models). ARS is the balanced acquiescence index: the mean of the forward- item and reverse-item means on the 0–1 scale, minus 0.5, over the two instruments with reverse-keyed items. ∆ ARS is the difference to humans in units of the human ARS standard deviation. Fwd and Rev are the differences of the forward-item mean and of the reverse-item mean in the same units, so that ∆ ARS is their average. The keying gap is the “acquiescence” index of the original audit (forward mean minus reverse mean), which grows with content consistency. Extreme and midpoint rates are shares of answers at the scale ends and the centre. SD ratio is the median item standard deviation relative to humans. Source∆ ARSFwdRev Keying gap Extreme Midpoint SD ratio Humans (n=263)0.000.000.002.000.270.211.00 gpt-5.4-mini (LLM)+0.21 +0.35 +0.071.510.020.330.28 GPT-6 Luna (LLM)+0.53 −0.66 +1.711.120.000.450.39 Median of 47 generative LLMs +0.21 −1.20 +1.401.200.030.380.54 Solar Decide Flash+3.54 +3.41 +3.681.050.200.350.35 Microsoft-Decision-1+2.68 −0.85 +6.200.210.060.270.48 Kev 4B+1.57 +0.18 +2.950.960.000.430.26 GPT-6 Luna Decisions+1.48 +0.78 +2.181.450.020.270.32 Tev1 4B+1.40 +0.72 +2.081.380.050.330.44 Liquid d1+1.11 +0.86 +1.351.740.060.220.41 Decider v1.1 27B+1.09 +0.20 +1.981.310.000.340.24 Decider v1 27B+1.08 +0.18 +1.991.310.000.340.24 Jev 1.13+1.01 +1.10 +0.921.720.080.290.46 Mercury Decide+0.78 +0.88 +0.681.630.130.330.59 Drex v1.5+0.78 −0.35 +1.911.080.010.300.45 Clef+0.32 +0.23 +0.401.720.000.220.28 Clef Flash−0.23 −0.99 +0.531.280.010.420.46 Clef Omni−0.27 −0.96 +0.411.440.020.310.44 Only part of the third hypothesis holds. A small share of each vector depends on the respondent, but this share is larger than the profile supports, and the vectors rank respondents about as well as a regression on the profile. Without a profile, the sampled PSS of decision models falls by 0.06 to 0.33. The problem is calibration rather than a weak use of the profile. Finally, the comparison of release waves does not support the fourth hypothesis (Section 5.10). 21 Table 11: Downstream use (C3,n=100, rounded readout).R 2 : an OLS regression of performance on attitudes and engagement is fit on the synthetic sample and tested on a fixed 30% split of the humans; the same regression fit on the other 70% of humans reaches R 2 =0.28. The raw column repeats the test without aligning reverse-keyed items, which is how the original audit scored it. Null paths: number of the 7 mediation triples with no human indirect effect for which the synthetic sample shows a significant indirect effect above 0.10. Of the 10 triples drawn, 3 are left out because a total score in them contains another of its variables. SourceR 2 (aligned)∆R 2 R 2 (raw) Null paths Gaussian copula0.29 +0.010/7 gpt-5.4-mini (LLM)−1.79 −2.07 −0.616/7 MiniMax M3 (best LLM)−0.24 −0.511/7 GPT-6 Luna (LLM)−0.26 −0.542/7 Median of 47 generative LLMs0.10 −0.182/7 Jev 1.130.30 +0.020.033/7 Decider v1 27B0.19 −0.09 −0.050/7 Decider v1.1 27B0.15 −0.12 −0.060/7 Liquid d1−0.61 −0.89 −1.315/7 Kev 4B−0.14 −0.42 −0.192/7 Tev1 4B−0.13 −0.40 −0.394/7 Mercury Decide−0.06 −0.330.072/7 Solar Decide Flash−0.81 −1.09 −0.454/7 Clef−0.42 −0.70 −0.181/7 Clef Flash−0.47 −0.75 −0.516/7 Clef Omni−0.95 −1.22 −1.166/7 GPT-6 Luna Decisions0.18 −0.100.081/7 Microsoft-Decision-1−1.44 −1.72 −0.673/7 Drex v1.5−0.01 −0.29 −0.184/7 6 Discussion Marginals and cost. Decision models remove the practical problems of free-text generation and cost a fraction of a cent per respondent. They also return a full answer distribution. Averaged over a sample, this distribution estimates item marginals better than the rounded answers of any language model in our set. On one fixed human split, a regression trained on Jev’s synthetic sample predicts held-out humans about as well as one trained on humans. Only 6 of 47 generative models come close to this. Correlation structure and calibration. Decision models reproduce the pattern of item correlations and the factor structure less well than most generative LLMs. Their person signal moves nearly all items in one direction, so the subscales are less distinct than in human data. They do read the profile, and their vectors rank people about as well as a regression on the profile. However, their average vector differs from the answer frequencies of real respondents, and they move each respondent further from the population than the profile supports. For these reasons their forecasts of single people are worse than the answer frequencies of other people until they are shrunk toward these frequencies. The same loss of correlation structure appears within one model name, as GPT-6 Luna loses much of it when served as a decision model. In most decision models acquiescence is also higher than in generative models, mainly on reverse-keyed items, and all of them compress the scale. All decision models except the two Decider versions still show at least one indirect effect that humans do not show. Therefore, a cleaner output format does not fix the problems that the original audit found in generative samples. These problems come from the way the model represents a respondent, and the output format changes only which of them are visible. How much a profile can tell. A regression that sees the same profile fields and the answers of 163 other people improves the forecast of a person’s answers by only 1.0%. Most of the variation in how people answer these items is not explained by the profile. Hu and Collier[27]found the same for annotation tasks, where persona variables explain less than 22 10% of the variance in most datasets. A model that individualises strongly from these fields will be wrong more often, as the narrow repeat distributions of the generative models and the over-spread decision vectors show. One reading of the correlation results follows from this. The dependence between items in human data comes from differences between people that the profile does not record. A synthetic respondent can reproduce it only by inventing such differences. Generative models do so and gain on the correlation component, but they overdo it. Their items correlate more strongly than in human data, and one general factor carries more of the variance. Decision models stay closer to the population and lose on the correlation component. For profile-based synthetic respondents, the realistic goal is therefore fidelity at the level of the sample. Newer releases. We expected newer models to do at least as well as their predecessors. Instead, the newest release scores lower in 8 of 10 product lines, and in 5 of them the drop is larger than the noise. The new models also answer with less spread, although the comparisons of release waves are exploratory. The one decision-model successor, Decider v1.1 27B, gives almost the same answers as its predecessor. We did not test why the generative releases lose fidelity. One possible reason is post-training that favours moderate answers in assistant use, which would push a model away from the spread of real survey responses. Reinforcement learning from human feedback is known to reduce the diversity of model outputs [29], which fits this explanation. A model upgrade therefore cannot be assumed to preserve the fidelity of a synthetic sample, and a study that switches to the newest version may lose fidelity without noticing. Recommendations for evaluating synthetic respondents. The results give nine recommendations. First, check the orientation of reverse-keyed items on both sides of every comparison. One item with a known answer, such as DUN_02, is enough to catch a mismatch. Second, check that the analysis code selects the right rows for every kind of source, including baselines, because a missing key can drop a component without an error message. Third, report the components with standard errors next to any weighted total. Atn=100 the mediation component alone can move PSS by 0.1. Fourth, treat the readout as part of the method and fix it before scoring, because it decides which property of the sample is kept. Fifth, validate each model version separately, because a newer release can score lower. Sixth, express counterfactual effects in units of the human SD. The paireddalso depends on how noisy a model is, so it can rank a deterministic model above a noisier one with a larger shift. Seventh, when a model returns probabilities, score them against each person’s own answers with a proper scoring rule. Compare them with the answer frequencies of other respondents, with the model’s own average forecast and with a recalibrated version. Eighth, compare synthetic samples with a second human sample of the same size. It shows how much of a similarity score is sampling noise. Ninth, report the level of the item correlations and the factor structure next to any similarity of correlation patterns, because a pattern can match while all correlations are too strong. Advice for survey practice. If an analyst needs rough item marginals before data collection, the vector of a decision model averaged over real profiles of the target population is a cheap first estimate. Without such profiles it is further from the human marginals for most models (Section 5.7). Even with them, it forecasts answers worse than the answer frequencies of other respondents, so it replaces neither a pilot study nor a pretest. In the rounded answers of decision models the median item SD is 0.24–0.59 of the human value. Their answers should therefore not be used to plan sample sizes or to check reliability. In terms of total survey error, the error of the model is a bias. It does not shrink when more synthetic respondents are added, and standard errors computed from a synthetic sample do not include it. Our human reference is a non-probability online sample, so all fidelity results refer to this sample. Where real data from a similar population exist, a copula fit to them is better. For analyses of relations between constructs, neither class of language model is a substitute for a human sample. No model in our set predicts how a particular person will answer better than the answer frequencies of other people. 23 7 Limitations and future work Dataset and conditions. All results come from one instrument set, in one language and one country, and from self-reports. We use one Lithuanian sample, 100 respondents, the C3 profile and, for the decision models, a control without a profile. The intermediate conditions of the profile ladder in the original audit were not repeated. GPT-6 Luna Decisions returned HTTP 502 for every request with an empty state, so it has no C0 cell. Decider v1 27B has no role and gender swaps because its endpoint was unavailable when they were run. The observed human education contrast rests on 15 secondary-educated respondents and is too noisy to say whether any model exaggerates it. Selection among many cells. With 47 generative cells, the best one is partly selected by chance. We report paired intervals against it and against the fixed reference gpt-5.4-mini, but the ranking inside the top group is not stable. A second sample of respondents would be needed to confirm the order of the leading models. Noise atn=100. The mediation and demographic components are noisy at this sample size. A second sample of 100 real people scores below the best language models on PSS when both are compared with the headline sample (Section 5.1). At this size PSS can separate weak from strong synthetic samples, but it cannot rank the best models against real respondents. A run on all 263 profiles would narrow the intervals for the models that matter most. With 200 bootstrap draws the interval ends carry some Monte Carlo error, which matters only for differences close to zero. Reasoning setting and timing. All generative cells use minimal reasoning. For 6 models the original audit also ran high reasoning effort. With it, 3 of them score higher and the others lower, so the setting can move a model in either direction. Newer models may also be tuned for longer reasoning than the minimal setting allows. The new models were queried in October 2026, months after the older cells, and we did not re-query the older models. Vendor-side changes to them since then are not captured. The new open-weight models and Grok 4.7 were also reached through different providers from those in the original audit (Fireworks and the xAI API instead of the Nexos gateway). Request design. The profile is passed to decision models in English JSON while the items and anchors are Lithuanian. Some models may handle the mixed language worse than a fully Lithuanian request. The three Clef models and Drex v1.5 answer the last four items in a separate request that does not see the first 64. Tev1 4B gives a check of this split. Its unchanged profiles, sent again in three parts, repeat 98% of its rounded answers (total variation 0.012), which is the size of its run-to-run noise. Moving targets. The decisions endpoint is an alpha API. We record the resolved model versions, but the endpoint exposes no seed. Our retest repeats 20 respondents once on the same day, so it does not show drift over weeks. Solar Decide is missing because of its latency and server errors, but its low-latency variant Solar Decide Flash is included. Structure tests. Tucker’sφuses an exploratory solution with greedy factor alignment. We did not test measurement invariance with multi-group confirmatory models. Several decision samples contain items without any variance (13 for each Decider version), and at n=100 such models are unstable. Memorisation. The human data have been public since 2020. We did not test the decision models for memorisation, and their close match to the item marginals could partly reflect it. However, for 12 of the 13 models with a C0 grid the vector without a profile is further from the human marginals than the vector averaged over real profiles. This argues against a simple recall of the marginals. Forecast references. The population distribution and the profile regression are fit on 163 people from the same survey, which no language model sees. Generative forecasts come from 24 20 repeats of 26 to 30 respondents at a requested temperature of 0.7 (the provider default where the API fixes it), and a higher temperature would widen them. These results thus compare the models with what can be done with data from the same population. They do not show how far each model is from the best possible forecast. Scope of the scoring update. This paper recomputes the headline numbers, the ranking, response style, predictive validity and the null-mediation test at C3. The other analyses of the original audit, such as the profile ladder and the generative swap grids, are recomputed in its revised version. Future work. The results point to several directions. First, the profile fields improve the forecast of a person’s answers by only 1.0%. Future studies should give the model richer information about each person, such as answers to a few anchor items, answers from an earlier wave or an interview. They can then test whether the forecasts beat the answer frequencies of other respondents. Second, mixing each vector with the population distribution removed most of the forecast loss of every decision model. Recalibration of this kind, fit on a small human sample, should become a standard step before the vectors are used. It remains open how many real respondents such a calibration needs, and at which pilot size a copula fit on the same pilot is better. Third, a decision request scores each item without a shared respondent state, and the person signal of the vectors has nearly one direction. Requests for the joint distribution of item pairs, or sequential requests that condition each item on earlier answers, would show whether the weak factor structure comes from the interface or from the model. Fourth, the two classes fail on different components. A hybrid that combines sampled decision marginals with the correlation pattern of generative samples could be tested without any human answers. Fifth, all results come from one Lithuanian sample and four instruments. Replications in other languages, with other instruments and with probability samples are needed, in particular for the handling of reverse-keyed items. Sixth, in 5 of 10 product lines the newest release scored lower beyond noise. A fixed benchmark of this kind can serve as a regression test that is run again whenever a provider changes a model version. Finally, atn=100 PSS does not separate the best language models from a second sample of real people. Runs on all 263 profiles, or on a larger survey, are needed before models can be ranked against real respondents. 8 Conclusion This paper aimed to test whether decision models, or newer generative LLMs, are better synthetic survey respondents than earlier generative LLMs. On an open psychometric benchmark, neither class does better. Decision models are a cheap and clean interface, and their probability vectors match the answers of the population closely. On the other hand, they recover less item dependence than most generative LLMs and react to role less strongly than generative models and people (median 0.22 against 0.47 and 0.46 human SD). They also spread respondents too far apart. For this reason they forecast single people worse than the answer frequencies of other people, although they rank people about as well as a regression on the profile. The readout that turns a vector into one answer decides which property of the sample is kept. Newer generative models do not close the gap, and in most product lines the newest release scores below the one before it. A statistical model of real responses fit on other people matches the best of 47 language models, and the same model fit in sample beats all of them. The revised scoring changes the order of the LLMs and makes the main conclusion of the original audit stronger. On the full reference the real answers of the same people also score higher than any language model. A second human sample of the same size shows, however, that atn=100 PSS cannot separate the best language models from real respondents. Finally, the profile fields explain little of how a person answers. Work on profile-based synthetic respondents should therefore aim at fidelity at the level of the sample. Richer information about each person would be needed before predicting a single person becomes a realistic goal. Ethics statement. The human data were collected in 2020 for a master’s thesis by the second author [44], under the informed-consent protocol of the awarding university, which 25 covered academic re-use of de-identified data. No direct identifiers were collected, and no new human data were collected for this study. Each model request contained only the eleven coarse profile fields and no identifiers or free text from respondents. Mercury Decide was called through a free endpoint whose provider may retain prompts. Data and code availability. The de-identified human data are those of the original audit (C BY-NC 4.0). The code, the raw model outputs with the full probability vectors, and the analysis tables are available from the corresponding author on reasonable request. Appendix G describes the pipeline. References [1]Gati V. Aher, Rosa I. Arriaga, and Adam Tauman Kalai. Using large language models to simulate multiple humans and replicate human subject studies. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 337–371, 2023. [2]American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. Standards for Educational and Psy- chological Testing. American Educational Research Association, Washington, DC, 2014. [3]Jacy Reese Anthis, Ryan Liu, Sean M. Richardson, Austin C. Kozlowski, Bernard Koch, Erik Brynjolfsson, James Evans, and Michael S. Bernstein. Position: LLM social simulations are a promising research method. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 81005–81034, 2025. [4]Lisa P. Argyle, Ethan C. Busby, Nancy Fulda, Joshua R. Gubler, Christopher Rytting, and David Wingate. Out of one, many: Using language models to simulate human samples. Political Analysis, 31(3):337–351, 2023. doi: 10.1017/pan.2023.2. [5]Christopher A. Bail. Can generative AI improve social science? Proceedings of the National Academy of Sciences, 121(21):e2314021121, 2024. doi: 10.1073/pnas. 2314021121. [6]Hans Baumgartner and Jan-Benedict E. M. Steenkamp. Response styles in marketing research: A cross-national investigation. Journal of Marketing Research, 38(2):143–156, 2001. doi: 10.1509/jmkr.38.2.143.18840. [7] Marcel Binz and Eric Schulz. Using cognitive psychology to understand GPT-3. Proceedings of the National Academy of Sciences, 120(6):e2218523120, 2023. doi: 10.1073/pnas.2218523120. [8] Marcel Binz et al. A foundation model to predict and capture human cognition. Nature, 644:1002–1009, 2025. doi: 10.1038/s41586-025-09215-4. [9]James Bisbee, Joshua D. Clinton, Cassy Dorff, Brenton Kenkel, and Jennifer M. Larson. Synthetic replacements for human survey data? The perils of large language models. Political Analysis, 32(4):401–416, 2024. doi: 10.1017/pan.2024.5. [10] Julien Boelaert, Samuel Coavoux, Étienne Ollion, Ivaylo D. Petev, and Patrick Präg. Machine bias. how do generative language models answer opinion polls? Sociological Methods & Research, 54(3):1156–1196, 2025. doi: 10.1177/00491241251330582. [11]Yong Cao, Haijiang Liu, Arnav Arora, Isabelle Augenstein, Paul Röttger, and Daniel Hershcovich. Specializing large language models to simulate survey response distributions for global populations. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 3141–3154, 2025. doi: 10.18653/v1/2025. naacl-long.162. [12] Lingjiao Chen, Matei Zaharia, and James Zou. How is ChatGPT’s behavior changing over time? Harvard Data Science Review, 6(2), 2024. doi: 10.1162/99608f92.5317da47. 26 [13]Lee J. Cronbach. Coefficient alpha and the internal structure of tests. Psychometrika, 16(3):297–334, 1951. doi: 10.1007/BF02310555. [14] Jamie Cummins. The threat of analytic flexibility in using large language models to simulate human data. Advances in Methods and Practices in Psychological Science, 9 (3), 2026. doi: 10.1177/25152459261461505. [15]Danica Dillion, Niket Tandon, Yuling Gu, and Kurt Gray. Can AI language models replace human participants? Trends in Cognitive Sciences, 27(7):597–600, 2023. doi: 10.1016/j.tics.2023.04.008. [16]Ricardo Dominguez-Olmedo, Moritz Hardt, and Celestine Mendler-Dünner. Questioning the survey responses of large language models. In Advances in Neural Information Processing Systems, volume 37, pages 45850–45878, 2024. doi: 10.52202/079017-1458. [17]Randall B. Dunham, Jean A. Grube, Donald G. Gardner, Larry L. Cummings, and Jon L. Pierce. The development of an attitude toward change instrument. Paper presented at the Academy of Management Annual Meeting, Washington, DC, 1989. [18]Esin Durmus, Karina Nguyen, Thomas I. Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, et al. Towards measuring the representation of subjective global opinions in language models. In First Conference on Language Modeling (COLM), 2024. URLhttps: //openreview.net/forum?id=zl16jLb91v. [19]Bradley Efron and Robert J. Tibshirani. An Introduction to the Bootstrap. Chapman and Hall, 1993. [20] Edward S. Epstein. A scoring system for probability forecasts of ranked categories. Journal of Applied Meteorology, 8(6):985–987, 1969. doi: 10.1175/1520-0450(1969) 008<0985:ASSFPF>2.0.CO;2. [21] Tilmann Gneiting and Adrian E. Raftery. Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association, 102(477):359–378, 2007. doi: 10.1198/016214506000001437. [22]Igor Grossmann, Matthew Feinberg, Dawn C. Parker, Nicholas A. Christakis, Philip E. Tetlock, and William A. Cunningham. AI and the transformation of social science research. Science, 380(6650):1108–1109, 2023. doi: 10.1126/science.adi1778. [23]Perttu Hämäläinen, Mikke Tavast, and Anton Kunnari. Evaluating large language models in generating synthetic HCI research data: A case study. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1–19, 2023. doi: 10.1145/3544548.3580688. [24]Andrew F. Hayes. Introduction to Mediation, Moderation, and Conditional Process Analysis: A Regression-Based Approach. Guilford Press, second edition, 2017. [25]Luke Hewitt, Ashwini Ashokkumar, Isaias Ghezae, and Robb Willer. Predicting results of social science experiments using large language models. Working paper, 2024. [26]John J. Horton, Apostolos Filippas, and Benjamin S. Manning. Large language models as simulated economic agents: What can we learn from Homo silicus? NBER Working Paper 31122, National Bureau of Economic Research, 2023. [27] Tiancheng Hu and Nigel Collier. Quantifying the persona effect in LLM simulations. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10289–10307, 2024. doi: 10.18653/v1/2024.acl-long.554. [28]Junsol Kim and Byungkyu Lee. AI-augmented surveys: Leveraging large language models and surveys for opinion prediction. arXiv:2305.09620, 2023. 27 [29]Robert Kirk, Ishita Mediratta, Christoforos Nalmpantis, Jelena Luketina, Eric Hambro, Edward Grefenstette, and Roberta Raileanu. Understanding the effects of RLHF on LLM generalisation and diversity. In International Conference on Learning Representations, 2024. [30]Linda Koopmans, Claire M. Bernaards, Vincent H. Hildebrandt, Henrica C. W. de Vet, and Allard J. van der Beek. Construct validity of the Individual Work Performance Questionnaire. Journal of Occupational and Environmental Medicine, 56(3):331–337, 2014. doi: 10.1097/JOM.0000000000000113. [31]Austin C. Kozlowski and James Evans. Simulating subjects: The promise and peril of artificial intelligence stand-ins for social agents and interactions. Sociological Methods & Research, 54(3):1017–1073, 2025. doi: 10.1177/00491241251337316. [32]Urbano Lorenzo-Seva and Jos M. F. ten Berge. Tucker’s congruence coefficient as a meaningful index of factor similarity. Methodology, 2(2):57–64, 2006. doi: 10.1027/ 1614-2241.2.2.57. [33]Mantas Lukauskas and Viktorija Šarkauskait ̇e. Plausible but not valid: A psychometric audit of LLMs as synthetic survey respondents, 2026. URLhttps://arxiv.org/abs/ 2608.14606. arXiv:2608.14606. [34]Benjamin F. Maier, Ulf Aslak, Luca Fiaschi, Nina Rismal, Kemble Fletcher, Christian C. Luhmann, Robbie Dow, Kli Pappas, and Thomas V. Wiecki. LLMs reproduce human pur- chase intent via semantic similarity elicitation of Likert ratings, 2025. arXiv:2510.08338. [35]Nicole Meister, Carlos Guestrin, and Tatsunori Hashimoto. Benchmarking distributional alignment of large language models. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 24–49, 2025. doi: 10.18653/v1/2025.naacl-long.2. [36] Roger B. Nelsen. An Introduction to Copulas. Springer, second edition, 2006. doi: 10.1007/0-387-28678-0. [37]OpenRouter. Decision models on OpenRouter.https://openrouter.ai/models? output_modalities=decisions, 2026. Accessed 7 October 2026. [38]Joon Sung Park, Carolyn Q. Zou, Jonne Kamphorst, Niles Egan, Aaron Shaw, Ben- jamin Mako Hill, Carrie Cai, Meredith Ringel Morris, Percy Liang, Robb Willer, and Michael S. Bernstein. LLM agents grounded in self-reports enable general-purpose simu- lation of individuals, 2026. arXiv:2411.10109v3. Version 1 (2024) was titled “Generative Agent Simulations of 1,000 People”. [39]Neha Patki, Roy Wedge, and Kalyan Veeramachaneni. The synthetic data vault. In 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA), pages 399–410, 2016. doi: 10.1109/DSAA.2016.49. [40]Max Pellert, Clemens M. Lechner, Claudia Wagner, Beatrice Rammstedt, and Markus Strohmaier. AI psychometrics: Assessing the psychological profiles of large language models through psychometric inventories. Perspectives on Psychological Science, 19(5): 808–826, 2024. doi: 10.1177/17456916231214460. [41]Nikolay B. Petrov, Gregory Serapio-García, and Jason Rentfrow. Limited ability of LLMs to simulate human psychological behaviours: A psychometric analysis, 2024. arXiv:2405.07248. [42]Aadesh Salecha, Molly E. Ireland, Shashanka Subrahmanya, João Sedoc, Lyle H. Ungar, and Johannes C. Eichstaedt. Large language models display human-like social desirability biases in Big Five personality surveys. PNAS Nexus, 3(12):pgae533, 2024. doi: 10.1093/pnasnexus/pgae533. 28 [43]Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. Whose opinions do language models reflect? In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 29971–30004, 2023. [44] Viktorija Šarkauskait ̇e. Darbuotoju ̨nuostatu ̨ i ̨pokyčius organizacijoje,i ̨sitraukimo i ̨darbą ir subjektyviai suvokiamos darbo atlikimo kokyb ̇es ryšys. Master’s thesis, Vytautas Magnus University, 2020. English title: “The relationship between employees’ attitudes towards organisational change, work engagement, and subjectively perceived work performance”. [45]Marko Sarstedt, Susanne J. Adler, Lea Rau, and Bernd Schmitt. Using large language models to generate silicon samples in consumer and marketing research: Challenges, opportunities, and guidelines. Psychology & Marketing, 41(6):1254–1270, 2024. doi: 10.1002/mar.21982. [46]Wilmar B. Schaufeli and Arnold B. Bakker. UWES: Utrecht Work Engagement Scale. Test Manual. Department of Psychology, Utrecht University, 2003. [47]Gregory Serapio-García, Mustafa Safdari, Clément Crepy, Luning Sun, Stephen Fitz, Peter Romero, Marwa Abdulhai, Aleksandra Faust, and Maja Matarić. A psychometric framework for evaluating and shaping personality traits in large language models. Nature Machine Intelligence, 7(12):1954–1968, 2025. doi: 10.1038/s42256-025-01115-6. [48]Joseph Suh, Erfan Jahanparast, Suhong Moon, Minwoo Kang, and Serina Chang. Language model fine-tuning on scaled survey data for predicting distributions of public opinions. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 21147–21170, 2025. doi: 10.18653/v1/2025. acl-long.1028. [49]Lindia Tjuatja, Valerie Chen, Tongshuang Wu, Ameet Talwalkar, and Graham Neu- big. Do LLMs exhibit human-like response biases? A case study in survey design. Transactions of the Association for Computational Linguistics, 12:1011–1026, 2024. doi: 10.1162/tacl_a_00685. [50]Angelina Wang, Jamie Morgenstern, and John P. Dickerson. Large language models that replace human participants can harmfully misportray and flatten identity groups. Nature Machine Intelligence, 7(3):400–411, 2025. doi: 10.1038/s42256-025-00986-z. [51] Xinpeng Wang, Bolei Ma, Chengzhi Hu, Leon Weber-Genzel, Paul Röttger, Frauke Kreuter, Dirk Hovy, and Barbara Plank. “My answer is C”: First-token probabilities do not match text answers in instruction-tuned language models. In Findings of the Association for Computational Linguistics: ACL 2024, pages 7407–7416, 2024. doi: 10.18653/v1/2024.findings-acl.441. [52] Bert Weijters and Hans Baumgartner. Misresponse to reversed and negated items in surveys: A review. Journal of Marketing Research, 49(5):737–747, 2012. doi: 10.1509/jmr.11.0368. [53]Caleb Ziems, William Held, Omar Shaikh, Jiaao Chen, Zhehao Zhang, and Diyi Yang. Can large language models transform computational social science? Computational Linguistics, 50(1):237–291, 2024. doi: 10.1162/coli_a_00502. 29 A Example request and response The state for one respondent (field values as stored in the processed profile file): "country": "Lithuania", "gender": "woman", "age": "22", "education": "university (Bachelor/Master/Doctorate)", "position": "non-managerial", "tenure_total": "less than 1 year", "tenure_current_org": "less than 6 months", "org_sector": "services", "org_size": "small (up to 50 employees)", "change_intensity": "9", "change_relevance": "6" One of the 68 score questions (item IWPQ_01). The instruction is “Atsakyk taip, kaip šis darbuotojas atsakytu ̨ i ̨teigini ̨.” followed by the item text “Aš gebu suplanuoti savo darbą taip, kad ji ̨baigčiau laiku” (“I can plan my work so that I finish it on time”). The criteria are “1 Niekada”, “2 Retai”, “3 Kartais”, “4 Dažnai” and “5 Visuomet” (never, rarely, sometimes, often, always). Jev 1.13 returned the probabilities (0.01,0.15,0.53,0.30,0.01) for levels 1–5. The expected score is 3.15, so both the rounded readout and the most probable category give 3. B Scoring update relative to the published version of the original audit While re-running the benchmark we revised two steps of the original scoring and one style index. The revised arXiv version of the original audit [33] uses the same code. Table 12 lists the numbers that change relative to the published version, and Figure 5 shows the change in the ranking. Here every sample is scored on the same 100 respondents. For this reason some values differ from the revised version of the original audit in the third decimal. The in-sample copula differs more, because that paper scores it on all 263 draws (PSS 0.936). Table 12: Numbers from the first version (v1) of the original audit [33] that change under the revised scoring, computed on the same 100 respondents. The revised version (v2) of that paper uses the same code. Causes: (A) model answers to reverse-keyed items were not reversed before scoring; (B) the distribution component was missing from the baseline totals; (C) the published index measured keying consistency rather than agreement; (D) the published test pooled all LLM cells; (E) null triples in which a total score contains another variable of the triple are left out. † Qwen 3.7 Plus has fewer than 90 complete respondents at C3 and is not ranked. ∗ Pooled over 37 models and all conditions (n=24,267). QuantityOriginal audit, v1 RevisedCause PSS, gpt-5.4-mini (C3)0.7140.743A Rank of gpt-5.4-mini among the models of the original audit 1 of 3710 of 36 † A Best LLM of the original auditgpt-5.4-mini, 0.714 MiniMax M3, 0.813A Correlation component, gpt-5.4-mini0.5220.602A PSS, Gaussian copula0.6880.927 (in-sample), 0.824 (out-of-sample)B In-sample copula minus best LLM−0.026+0.114A, B Acquiescence shift, gpt-5.4-mini+1.29+0.21 SDA, C Synthetic-trained R 2 on humans, gpt-5.4-mini−0.62−1.79A Null paths with a fabricated indirect effect3/10 ∗ 6/7 (gpt-5.4-mini)A, D, E Reverse-keyed items. Human answers to the 13 reverse-keyed items are stored reversed, so that a high value always means a positive attitude. Model answers to these items were stored as given to the original wording and were not reversed before scoring. We now reverse them withx ′ =min+max−xand call the result the aligned score. The published scores are raw. A consistent respondent with a positive attitude therefore scored high on a reverse-keyed item in the human file and low in the model file. Item DUN_02 (“I usually resist new ideas”) shows the size of the effect. The human mean is 4.14 after reversal, while gpt-5.4-mini answers 1.97 on average, which is close to the human mean before reversal (1.86). The change raises the PSS of every one of the 36 C3 cells of the original audit, by a median of 0.093 and at least 0.016. For gpt-5.4-mini the aligned PSS is 0.743. The change affects all five components. It removes divergence on 13 items, restores the sign of their correlations with the other 55 items, changes the Dunhamα, and enters the attitude 30 0.40.50.60.70.8 PSS, original scoring (raw) 0.55 0.60 0.65 0.70 0.75 0.80 0.85 PSS, revised scoring (aligned) MiniMax M3 MiniMax M2.7 Kimi K2.7 Code Claude Opus 4.7 gpt-5.4-mini Grok 4.20 Llama 4 Maverick Figure 5: PSS of the 36 C3 cells of the original audit under the published scoring (raw) and the revised scoring (aligned). The dashed line marks equal scores. gpt-5.4-mini, the published leader, is shown in red. composites used for mediation and demographic contrasts. Table 14 shows that the raw comparison roughly doubles the divergence on reverse-keyed items. Our raw recomputation of gpt-5.4-mini gives 0.703, against the published 0.714. The gap comes from pooling. For most models the published version averaged the 20 test-retest repeats into the C3 cell, while we use the single run of the main grid for every model. We also checked whether the gain of each model follows its keying gap. The rank correlation is 0.21 (p=0.23), so the gain depends on more than this one property. Baseline totals. The distribution component was missing from the totals of the statistical baselines and counted as zero. With it included, the copula of the original audit reaches 0.927 on the same 100 IDs. This copula is fit on the 263 people it is scored against, so we also report an out-of-sample copula, fit on the 163 people outside the headline sample, which reaches 0.824. The analysis script computes item distances for every cell and selects the rows of one cell by matching model, condition, presentation mode and reasoning effort. Baseline samples have no reasoning-effort value, and a missing value never matches itself. The selection was empty, and the total treated the missing component as zero. The published copula total of 0.688 is the weighted sum of the other four components alone. Acquiescence. The index of the original audit, the forward-item mean minus the reverse- item mean, grows with answers that follow the content of the items. We keep it under the name keying gap and add the balanced acquiescence index (ARS) [6]. The keying gap gives zero to a respondent who agrees with every statement and a large positive value to a respondent who follows the content of the items. ARS is the average of the forward-item mean and the reverse-item mean on a 0–1 scale, minus 0.5, computed on raw answers. With ARS computed in this way for humans and models, gpt-5.4-mini shifts by +0.21 human SD. Humans have a larger keying gap (2.001) than gpt-5.4-mini (1.512). Applying the keying gap to reversed human answers and unreversed model answers reproduces the published shift for gpt-5.4-mini (+1.28 against the published +1.29). Ranking. The first change raises some models more than others, so the order of the LLMs changes (Figure 5). The Spearman correlation between the published and the revised order 31 is 0.79. gpt-5.4-mini moves from first to 10th place, and MiniMax M3 (0.813), MiniMax M2.7 (0.809) and Kimi K2.7 Code (0.802) now lead. The correlation component gains most and rises for every cell (median +0.37). The reliability component also rises for every cell (median +0.11), because the unreversed items had moved the Dunhamαof some models far from the human value. The mediation component falls for 30 of 36 cells (median−0.18). The order changes mainly through the reliability and mediation components, whose changes differ most between models (SD 0.18 and 0.19, against 0.07 for correlation). What holds and what changes. The main conclusion of the original audit holds and becomes stronger. LLM samples are not a drop-in replacement for human data, and the lead of the statistical baseline grows. In its published version the best LLM was slightly above the copula. With the revised scoring the in-sample copula is above every language model, by +0.11 [+0.06, +0.17] over the best one (Table 2), and the out-of-sample copula is level with the best (+0.01 [−0.07, +0.09]). Range restriction holds as well, because it does not depend on item direction. The share of extreme answers is 0.270 for humans and 0.024 for gpt-5.4-mini. LLM samples still produce mediation paths that humans do not show (Section 5.11). The revision changes three findings. First, the best LLM is a different model. Second, the acquiescence finding becomes a pattern of fewer extreme answers, more midpoint answers and a smaller keying gap. Third, the advantage of LLMs on mediation and demographic effects shrinks, as the next paragraph shows. Mediation and demographic effects. The original audit argued that these components are where language models add value over statistical baselines. Against the in-sample copula this does not hold. Its mediation component is 0.968 against 0.929 for gpt-5.4-mini, and its demographic component is 0.663 against 0.489 (Table 1). Against the out-of-sample copula the mediation component is 0.722, and the paired difference to gpt-5.4-mini is−0.21 [−0.39, −0.03]. Its demographic component is 0.382, with a difference of−0.11 [−0.35, +0.13], which is within noise. Neither copula sees the profile, so its demographic component reflects chance agreement only, and the gap between the two copulas shows how noisy this component is at n=100. PSS scores only one indirect effect, and gpt-5.4-mini reproduces it better than the real answers of the same 100 people do (mediation component 0.929 against 0.707). The same sample produces 6 of the 7 cross-construct mediation paths that are absent in the human data. Analyses not affected. Results that compare models only with each other, such as inter- LLM similarity and test-retest reliability, do not depend on the direction of reverse-keyed items. The revised version of the original audit recomputes the analyses that compare models with humans through these items. It also signs the paired differences of its counterfactual swaps by the direction of the swap and reports the swap effects in units of the human SD. Without the sign, swaps in opposite directions cancel, which made the role effect look small. C Models, costs and item keying Table 13 lists the decision models with their pinned versions, context limits, prices and the cost of the C3 grid. Table 14 compares the item divergence of reverse-keyed and forward-keyed items under the raw and the aligned scoring. 32 Table 13: Decision models in the study. All were called through the OpenRouter decisions endpoint with one score question per item. Calls is the number of requests per respondent: the three Clef models and Drex accept at most 64 questions, so they receive the profile twice (64 items, then 4). Price is the listed input price in USD per million tokens; output tokens are free for this endpoint. Cost is the observed mean cost per synthetic respondent in thousandths of a dollar. The C3 grid of the scored models cost $1.27, and the incomplete Solar Decide run another $0.08. ModelResolved versionCalls n Input tokens Price Cost (10 −3 $) Jev 1.13 typesafe/jev-1.13-202609171 1007,800 0.0420.33 Decider v1 27B perplexity/pplx-decider-v1-27b-202610011 10017,817 0.0400.71 Decider v1.1 27B perplexity/pplx-decider-v1.1-27b-202610061 10017,817 0.0200.36 Liquid d1 liquid/d1-202609301 10015,953 0.0400.64 Kev 4B jaredpalmer/kev-4b-202609241 1005,607 0.0420.24 Tev1 4B togethercomputer/tev1-4b-experimental-202609231 10018,264 0.0420.77 Mercury Decide inception/mercury-decide-202609301 10013,230 0.0000.00 Solar Decide † upstage/solar-decide-202609281 4635,091 0.0501.75 Solar Decide Flash upstage/solar-decide-flash-202610081 10035,044 0.0501.75 Clef cloudflare/clef2 10012,635 0.2403.03 Clef Flash cloudflare/clef-flash2 10012,635 0.0901.14 Clef Omni cloudflare/clef-omni-202610092 10013,505 0.1502.03 GPT-6 Luna Decisions openai/gpt-6-luna-decisions-202610061 10012,025 0.1001.20 Microsoft-Decision-1 microsoft/microsoft-decision-1-202610091 1006,140 0.0420.26 Drex v1.5 nace-ai/drex-v1.5-202610092 1005,693 0.0400.23 † Incomplete at the time of writing; not scored. Table 14: Mean item Jensen-Shannon divergence to the human marginal on the two instru- ments with reverse-keyed items (rounded readout). The raw column compares unflipped model answers with reverse-coded human answers, which is what the original audit did for every reverse-keyed item. SourceForward items Reverse items (aligned) Reverse items (raw) gpt-5.4-mini (LLM)0.2430.3560.728 Jev 1.130.1320.1730.524 Decider v1 27B0.2630.4110.602 Decider v1.1 27B0.2650.4130.604 Liquid d10.1680.2880.561 Kev 4B0.2480.4700.555 Tev1 4B0.1940.3670.548 Mercury Decide0.0910.1390.467 Solar Decide Flash0.3510.4490.504 Clef0.2230.3520.681 Clef Flash0.1840.2820.606 Clef Omni0.2060.2750.615 GPT-6 Luna Decisions0.2250.3630.553 Microsoft-Decision-10.1450.5510.291 Drex v1.50.1700.3060.508 D All generative cells Table 15 lists the PSS components of all 47 generative cells under the revised scoring, with the raw PSS for the cells of the original audit. 33 Table 15: All 47 generative cells at C3 (single call, minimal reasoning, same 100 respondents), ordered by aligned PSS. ∗ Released after the original audit and run for this study (11 models). Raw PSS is the original scoring path and Raw rank its order among the models of the original audit. The aligned columns are the components, the total and its bootstrap SE. Null is the number of the 7 null mediation triples with a fabricated path (Table 11). Qwen 3.7 Plus (75 complete respondents) and the six cells run with high reasoning effort are not shown. RawAligned # ModelVendorn PSS Rank Dist. Corr. Rel. Med. Demo. PSS SE Null 1 MiniMax M3Open weights 100 0.6797 0.71 0.76 0.97 1.00 0.51 0.813 0.0231 2 MiniMax M2.7Open weights 100 0.6953 0.83 0.81 0.98 0.76 0.52 0.809 0.0392 3 Kimi K2.7 CodeOpen weights 100 0.6698 0.80 0.81 0.95 0.80 0.52 0.802 0.0301 4 Claude Opus 4.7Anthropic100 0.61418 0.72 0.77 0.96 0.96 0.34 0.789 0.0332 5 Gemini 3.1 Flash-LiteGoogle100 0.6874 0.86 0.72 0.95 0.75 0.42 0.779 0.0475 6 Qwen3.5 397BOpen weights 100 0.63016 0.76 0.65 0.97 0.82 0.65 0.777 0.0331 7 Gemini 2.5 ProGoogle100 0.6826 0.87 0.76 0.94 0.57 0.48 0.758 0.0420 8 DeepSeek V4 ProOpen weights 100 0.7012 0.80 0.80 0.95 0.49 0.61 0.750 0.0511 9 Kimi K2.6Open weights 100 0.63813 0.76 0.70 0.95 0.71 0.48 0.747 0.0341 10 gpt-5.4-miniOpenAI100 0.7031 0.66 0.60 0.96 0.93 0.49 0.743 0.0206 11 Claude Opus 4.8Anthropic100 0.64112 0.70 0.74 0.96 0.79 0.29 0.740 0.0382 12 Claude Sonnet 5Anthropic100 0.6589 0.70 0.70 0.96 0.77 0.41 0.739 0.0393 13 gpt-oss-120bOpen weights 98 0.58724 0.73 0.80 0.94 0.60 0.46 0.737 0.0331 14 Gemini 3.5 FlashGoogle100 0.63015 0.82 0.77 0.93 0.54 0.41 0.735 0.0454 15 Claude Sonnet 5.5 ∗ Anthropic100 0.6390.69 0.70 0.98 0.73 0.44 0.735 0.0341 16 Claude Haiku 4.5Anthropic100 0.64410 0.75 0.73 0.95 0.59 0.43 0.720 0.0342 17 Qwen 3.8 Max ∗ Open weights 100 0.6410.71 0.66 0.99 0.66 0.48 0.720 0.0310 18 GLM 5.1Open weights 100 0.6855 0.82 0.78 0.95 0.39 0.46 0.712 0.0531 19 Gemini 3.8 Flash ∗ Google100 0.6130.80 0.78 0.95 0.45 0.35 0.710 0.0452 20 Grok 4.3xAI100 0.59421 0.72 0.76 0.96 0.48 0.50 0.707 0.0483 21 Nemotron 3 SuperOpen weights 99 0.59920 0.83 0.76 0.96 0.31 0.53 0.704 0.0352 22 DeepSeek V4.1 Flash ∗ Open weights 100 0.6160.77 0.72 0.98 0.48 0.37 0.703 0.0382 23 Gemini 3 FlashGoogle100 0.60919 0.85 0.79 0.93 0.30 0.43 0.699 0.0462 24 GLM 5.3 ∗ Open weights 100 0.6060.74 0.74 1.00 0.44 0.37 0.695 0.0331 25 Grok 4.20 (reasoning)xAI100 0.64111 0.75 0.74 0.96 0.41 0.36 0.683 0.0491 26 Claude Fable 5Anthropic100 0.59322 0.75 0.72 0.96 0.46 0.33 0.682 0.0342 27 Grok 4.20xAI100 0.36136 0.71 0.65 0.97 0.50 0.46 0.679 0.0522 28 DeepSeek 3.2Open weights 100 0.53928 0.87 0.75 0.94 0.12 0.39 0.655 0.0400 29 Gemini 3.1 ProGoogle100 0.57425 0.83 0.78 0.95 0.10 0.39 0.654 0.0353 30 Gemma 4 31BOpen weights 100 0.63314 0.79 0.74 0.96 0.16 0.42 0.649 0.0400 31 GPT-5.4 nanoOpenAI100 0.62817 0.66 0.55 0.92 0.56 0.46 0.644 0.0390 32 Grok 4.1 Fast (reasoning) xAI100 0.53829 0.90 0.73 0.93 0.00 0.50 0.644 0.0273 33 Llama 4 MaverickOpen weights 100 0.41635 0.70 0.57 0.97 0.43 0.45 0.643 0.0381 34 Kimi K3 ∗ Open weights 100 0.5650.72 0.72 0.98 0.27 0.30 0.641 0.0332 35 Llama 4 ScoutOpen weights 100 0.52232 0.73 0.63 0.98 0.48 0.08 0.640 0.0305 36 Grok 4.1 FastxAI100 0.51334 0.90 0.73 0.93 0.00 0.45 0.639 0.0183 37 GLM 5.2Open weights 100 0.54227 0.82 0.79 0.94 0.00 0.46 0.637 0.0204 38 Gemini 3.6 Flash ∗ Google100 0.5930.84 0.77 0.93 0.00 0.48 0.637 0.0304 39 Mistral Large 3Open weights 100 0.53730 0.62 0.70 0.96 0.34 0.38 0.626 0.0505 40 GPT-6 Luna ∗ OpenAI100 0.5660.69 0.57 0.92 0.46 0.32 0.621 0.0352 41 Grok 4.7 ∗ xAI99 0.5560.72 0.65 0.99 0.16 0.37 0.610 0.0310 42 GPT-5.5OpenAI100 0.59023 0.72 0.64 0.97 0.22 0.30 0.609 0.0431 43 Claude Opus 5.5 ∗ Anthropic100 0.5150.70 0.66 0.99 0.20 0.29 0.606 0.0362 44 GPT-5.4OpenAI100 0.53031 0.79 0.71 0.96 0.00 0.40 0.606 0.0161 45 Nova 2 LiteAmazon100 0.55826 0.71 0.54 0.96 0.27 0.45 0.605 0.0362 46 Claude Sonnet 4.6Anthropic100 0.52133 0.75 0.67 0.96 0.00 0.41 0.586 0.0152 47 GPT-6.1 Sol ∗ OpenAI100 0.4790.69 0.45 0.85 0.11 0.33 0.510 0.0302 34 E Reference sample Table 16 scores the main sources against the full reference of 263 people and against the 100 headline people alone. Section 5.1 discusses it. Table 16: PSS against two reference samples. All 263 is the reference of the main text. Headline 100 scores every source against the real answers of the 100 people whose profiles were simulated. The 163 training people of the out-of-sample copula are not part of this reference, while the in-sample copula is fit on all 263 people and still overlaps with it. The second human sample draws 100 of the 163 people outside the headline sample. The 2000-row copula repeats each headline profile 20 times and shows the copula without its own sampling noise. D, C, M and G are the distribution, correlation, mediation and demographic components against the headline 100. All 263Headline 100 SourcePSSDCMG PSS Second human sample (mean of 20)0.92 0.99 0.73 0.56 0.53 0.79 Copula, out-of-sample (reported draw)0.82 0.97 0.61 0.43 0.32 0.71 Copula, out-of-sample (mean of 20)0.86 0.97 0.67 0.56 0.35 0.75 Copula, out-of-sample (2000 rows)0.89 0.98 0.77 0.53 0.30 0.77 Copula, in-sample0.93 0.99 0.83 0.67 0.49 0.83 Best generative (MiniMax M2.7)0.81 0.82 0.74 0.95 0.65 0.84 gpt-5.4-mini0.74 0.65 0.52 0.64 0.62 0.67 Jev 1.13, R0.68 0.76 0.64 0.79 0.51 0.75 Jev 1.13, A0.79 0.77 0.63 0.68 0.54 0.73 Median generative LLM0.70 0.74 0.70 0.71 0.55 0.75 Median decision model, R0.59 0.68 0.41 0.57 0.50 0.61 F Agreement between models and sensitivity to the weights The published version of the original audit found that LLMs agree with each other more than with humans. On the correlation structure, decision models agree with each other (0.384) slightly less than with humans (0.422, Figure 6). Their errors thus show little shared bias on this measure. The ranking of decision models and baselines is stable under changes of the PSS weights. Over 500 Dirichlet draws the mean Spearman correlation with the canonical ranking is 0.95 (minimum 0.71; 0.60 of draws above 0.95). The in-sample copula is first in every draw. Independent marginals, Tev1 4B and Solar Decide Flash form the bottom three in a share of 0.48 of the draws. The changes happen inside the close middle group. 35 Human Gaussian copula (in-sample) gpt-5.4-mini Jev 1.13 Decider v1 27B Decider v1.1 27B Liquid d1 Kev 4B Tev1 4B Mercury Decide Solar Decide Flash Clef Clef Flash Clef Omni GPT-6 Luna Decisions Microsoft-Decision-1 Drex v1.5 Human Gaussian copula (in-sample) gpt-5.4-mini Jev 1.13 Decider v1 27B Decider v1.1 27B Liquid d1 Kev 4B Tev1 4B Mercury Decide Solar Decide Flash Clef Clef Flash Clef Omni GPT-6 Luna Decisions Microsoft-Decision-1 Drex v1.5 1.000.890.600.660.350.340.530.230.240.530.150.540.620.500.340.400.46 0.891.000.560.600.340.340.480.230.260.480.140.510.570.450.310.380.46 0.600.561.000.450.290.280.340.270.330.330.300.550.470.480.450.390.41 0.660.600.451.000.500.500.670.250.260.710.030.560.640.530.360.510.53 0.350.340.290.501.000.960.470.260.210.550.000.330.270.260.350.360.45 0.340.340.280.500.961.000.470.250.200.550.000.320.270.240.350.370.44 0.530.480.340.670.470.471.000.350.360.570.030.460.530.400.400.540.58 0.230.230.270.250.260.250.351.000.550.220.360.370.250.240.350.480.47 0.240.260.330.260.210.200.360.551.000.150.520.370.230.200.470.550.51 0.530.480.330.710.550.550.570.220.151.00-0.020.430.490.380.290.440.46 0.150.140.300.030.000.000.030.360.52-0.021.000.260.100.070.300.280.17 0.540.510.550.560.330.320.460.370.370.430.261.000.600.540.470.480.46 0.620.570.470.640.270.270.530.250.230.490.100.601.000.620.280.390.43 0.500.450.480.530.260.240.400.240.200.380.070.540.621.000.300.310.31 0.340.310.450.360.350.350.400.350.470.290.300.470.280.301.000.490.46 0.400.380.390.510.360.370.540.480.550.440.280.480.390.310.491.000.68 0.460.460.410.530.450.440.580.470.510.460.170.460.430.310.460.681.00 0.0 0.2 0.4 0.6 0.8 1.0 Figure 6: Pearson correlation between the upper triangles of the 68×68 item correlation matrices (aligned, rounded readout). The human matrix uses all 263 respondents, and the copula is the in-sample copula. Every other source uses the same 100 profiles. G Reproducibility All scripts are inscripts/psychometry/.run_decision_c3.pysends the decision requests and stores one JSON file per respondent and model with the full probability vectors, the re- solved model version, the token count and the cost. It skips cells that are already complete, so an interrupted run can be resumed.01_generate_synthetic.pyis the generator of the orig- inal audit; the new generative models were run with–model-set update_2026_10_models –conditions C3_full –reasoning-effort minimal.score_decision_c3.pycomputes raw and aligned PSS and recomputes gpt-5.4-mini from the data of the original au- dit as a check.analyze_decision_c3.pyruns the readout, response-style, predictive- validity, null-mediation, probability and similarity analyses.analyze_generative_c3.py rescores every generative cell with both scoring paths and computes its style and down- stream measures.bootstrap_decision_c3.pyadds the oracle and the out-of-sample copula and runs the paired bootstrap over all sources.run_decision_controls.py sends the C0, retest and swap grids, andanalyze_decision_controls.pyanalyses them. analyze_decision_forecasts.pyscores the probability vectors, the references and the generative repeat distributions as forecasts.analyze_decision_references.pyreads the profile regression as a synthetic respondent and runs the checks against the 100 headline people.analyze_decision_structure.pycomputes the correlation level, the eigenvalue shares, the subscale separation and Tucker’sφ.make_decision_paper_assets.pybuilds every table, figure and number in this paper from the resulting CSV files. No API call is needed after the generation scripts. 36

Instant research discovery