Paper deep dive
No PUN Intended: Plausible Unknown Names for Person-Centred LLM Evaluation
Dimitri Staufer, David Hartmann, Ibrahim Baroud
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Person names are widely used as prompt variables in LLM evaluations of factuality, privacy leakage, bias and abstention, but when a name's evidential status is uncontrolled, measurements may conflate memorisation, retrieval, name priors and wrong-person attribution. We operationalise an unknown name as one with plausible First-Last form, no indexed full-name evidence, and no ambiguity signals under a documented validation run, and introduce PUN (Plausible Unknown Names), a protocol for constructing and validating such names, combining Wikidata-derived components, web-enabled LLM screening, and controlled search revalidation. We report acceptance rate, reproducibility, ablations, and a 204-participant human study, finding accepted names are more name-like than controls while participants recover person evidence in only 3% of cases. We release 300 names with comparison controls.
Tags
Links
- Source: https://arxiv.org/abs/2608.21206v1
- Canonical: https://arxiv.org/abs/2608.21206v1
Trouble viewing inline? Open PDF directly â
Full Text
86,663 characters extracted from source content.
Expand or collapse full text
No PUN Intended: Plausible Unknown Names for Person-Centred LLM Evaluation Dimitri Staufer 1 David Hartmann 1,2 Ibrahim Baroud 3,4 1 Technische UniversitĂ€t Berlin, Berlin, Germany 2 Weizenbaum Institute for the Networked Society, Berlin, Germany 3 Quality & Usability Lab, Technische UniversitĂ€t Berlin, Berlin, Germany 4 German Research Center for Artificial Intelligence (DFKI), Berlin, Germany Abstract Person names are widely used as prompt vari- ables in LLM evaluations of factuality, privacy leakage, bias and abstention, but when a nameâs evidential status is uncontrolled, measurements may conflate memorisation, retrieval, name priors and wrong-person attribution. We op- erationalise an unknown name as one with plausible First-Last form, no indexed full- name evidence, and no ambiguity signals un- der a documented validation run, and introduce PUN (Plausible Unknown Names), a protocol for constructing and validating such names, combining Wikidata-derived components, web- enabled LLM screening, and controlled search revalidation. We report acceptance rate, repro- ducibility, ablations, and a 204-participant hu- man study, finding accepted names are more name-like than controls while participants re- cover person evidence in only 3% of cases. We release 300 names with comparison controls. 1 Introduction Person names appear across many large language model (LLM) evaluation settings. They are used as targets for biography generation (Min et al., 2023), prompts for personal-data auditing (Staufer et al., 2026; Tan et al., 2025), cues in hiring-decision audits (An et al., 2024), possible memorised identi- fiers in privacy leakage tests (Carlini et al., 2021; Huang et al., 2022; Keum et al., 2025), spans to be replaced in anonymisation and pseudonymisa- tion (Lison et al., 2021; Eder et al., 2019; Yermilov et al., 2023), inputs for evaluating abstention when evidence is insufficient (Madhusudhan et al., 2025; Wen et al., 2025), and elements of LLM-generated synthetic examples (Wang et al., 2023; Li et al., 2023; Long et al., 2024). Across these settings, a name is often treated as a convenient prompt variable: something that can be inserted, replaced, perturbed, or compared across conditions. LLM prompt that includes a person name e.g., âWhat is First Last known for?â Attractors e.g., famous person Ambiguity e.g., namesakes, variants Associations e.g., culture, gender, religion This paper contributes 1. Operationalisation ofunknown name: (i)plausiblefull name; (i) no exact-name evidence; (i) no ambiguity signals; with mixed associations 2. Protocol (PUN): Wikidata name poolsâgenerate + filterâLLM + Searchâtime-stamped verdict 3. Showcase: 300 accepted names + comparisons of varying fame, prevalence, famous-name proximity Example visualisation of famous-name proximity using gpt-image-2. âBarack Obamaâ ordered by length-normalised Levenshtein edit distance (A.7). âBaracc OpamaââBar ObamaotsvikââBarnt Obamaotsvikâ Figure 1: Paper motivation and contribution overview. Yet person names are not neutral placeholders. A full-name string can simultaneously act as a key to public evidence about a referent, as in âBarack Obamaâ; an ambiguous identifier shared by multi- ple people, as in âJohn Williamsâ; a form linked to nearby spellings or reordered variants, as in âStephen Kingâ versus âSteven Kingâ; and a sig- nal of culture, gender, ethnicity, religion, or class. These roles have been studied separately in work on web-based person-name disambiguation (Ar- tiles et al., 2007; Balog et al., 2009; Delgado et al., 2018) and name-conditioned social or demographic inference (Jeoung et al., 2023; Gautam et al., 2024; Pawar et al., 2025). As Figure 1 illustrates, several signals can attach to the same prompted full-name string, making measurements in person-centred LLM evaluations difficult to interpret when the stringâs evidential status is uncontrolled. 1 arXiv:2608.21206v1 [cs.CL] 21 Aug 2026 For example, when an LLM answers âWhat is First Last known for?â in a biography evaluation, recommends whether to interview First Last in a hiring audit, produces personal details for First Last in a privacy test, or refuses to answer a ques- tion such as âHas First Last been arrested?â, the behaviour may have several causes. It may reflect indexed public evidence, memorised training data, retrieval at inference time, name-conditioned social priors, wrong-person conflation, or unsupported generation. Conversely, when a model abstains, it may be appropriately uncertain, may have failed to retrieve available evidence, or may be following a generic safety pattern unrelated to the name itself. Using names with different amounts and kinds of public evidence can therefore help tease apart these possible explanations. Without such control, mea- surements of factuality, privacy leakage, bias, and abstention can conflate distinct sources of model behaviour. Approach.We address this problem by develop- ing a flexible protocol for generating operationally unknown names. We define operational unknown- ness for person-name prompts using a protocol we call PUN (Plausible Unknown Names). It con- structs, screens, and revalidates plausible person- name prompts. Here, âplausibleâ means a two- component First Last string built from attested given-name and surname components and passing explicit form constraints. It does not mean cultur- ally universal, naturally acceptable, or legally valid as a name. The protocol relies on Wikidata for candidate generation and uses a web-enabled LLM and search engine for verification. The term âopera- tionalâ is deliberate. We do not claim non-existence of a person or that no private record contains the name. Nor do we claim that future search results will remain empty. We claim only that the name satisfies a conservative, time-bounded evidential condition that can be inspected, challenged, and revalidated. Contributions.This paper makes three contribu- tions: (1) a practical operationalisation of name unknownness, specifying both the acceptance crite- rion and diagnostics for borderline cases; (2) PUN, an auditable protocol for generating and validat- ing plausible unknown names; and (3) a showcase resource of 300 accepted names with comparison controls, validation traces, reproducibility diagnos- tics, protocol ablations, and a 204-participant hu- man agreement audit. 2 Operational Unknownness We use operationally unknown for the binary ev- idential status of a plausible full-name string un- der a specified validation run. Letnbe the full- name string used in an evaluation prompt, and let PUN t denote the validation protocol run at timet. The protocolPUN t specifies the retrieval systems, settings, query forms, and decision rules used to search for indexed public evidence. Section 4 de- scribes the concrete instantiation used in this paper. A candidate is accepted as operationally un- known underPUN t if (i) it is a plausible two- component First Last string, (i)PUN t finds no indexed public evidence that the exact full name, or a configured equivalent, refers to a person, and (i) PUN t finds no full-name-level ambiguity signal, such as a nearby spelling, reordered form, search correction, famous-name attractor, or unclear non- person referent. For example, Leora Vandeskel would be rejected if the run found a trace for that exact name, a reordered form such as Vandeskel Leora, a similar spelling such as Liora Vandeskel, or a non-person entity such as âVandeskel Corpâ. 3 Background and Related Work 3.1 Name Form, Resources, and Plausibility PUN uses plausible name in a narrow operational sense: a candidate has First Last form, is built from attested given-name and surname components, and passes explicit form checks. This is less ambi- tious than sociolinguistic, legal, demographic, or community-specific plausibility. Personal names vary in morphology, orthography, frequency, con- ventionality, cultural distribution, and social mean- ing (Van Langendonck, 2007; Motschenbacher, 2020). Naming systems also vary in order, number of components, inherited surnames, patronymics, mononyms, particles, multiple surnames, and script conventions (Lawson, 2016; Ndlovu and Völkel, 2023). Existing name resources and pseudonymi- sation work usually start from observed spans: ParaNames derives Wikidata entity names for mul- tilingual NER, linking, translation, and translitera- tion (SĂ€levĂ€ and Lignos, 2024), while anonymisa- tion and pseudonymisation systems replace sensi- tive person-name spans with masks or surrogates (Lison et al., 2021; Eder et al., 2019; Yermilov et al., 2023; Szawerna et al., 2024; Szawerna and Suchardt, 2026). PUN instead constructs full-name prompts that retain name-like form while making their indexed full-name evidence auditable. 2 3.2 Full Names as Ambiguous Evidence Keys A full name is not a reliable identifier. The same string can refer to multiple people, and one per- son can appear under multiple spellings, scripts, transliterations, aliases, abbreviations, reordered forms, or other variants. Web-based person-name disambiguation is built around this problem: sys- tems must separate namesakes while reconciling variant forms (Artiles et al., 2007; Balog et al., 2009; Delgado et al., 2018; Zhagorina et al., 2018). Absence of search evidence is also not absolute, since search-engine coverage and retrievability can change over time, and hit-count estimates can be ap- proximate, unstable, and provider-dependent (Bar- Ilan, 1999; Uyar, 2009; SĂĄnchez et al., 2018). 3.3 Names in LLM Evaluation LLM evaluations use names in biography and fac- tuality evaluation (Min et al., 2023), privacy and memorisation audits (Carlini et al., 2021; Huang et al., 2022; Keum et al., 2025; Staufer et al., 2026; Tan et al., 2025), anonymisation and pseudonymi- sation settings (Lison et al., 2021; Eder et al., 2019; Yermilov et al., 2023), hiring and employment au- dits (An et al., 2024; Wang et al., 2024), bias and social-inference studies (De-Arteaga et al., 2019; Romanov et al., 2019; Jeoung et al., 2023; An et al., 2025; Pawar et al., 2025), and abstention or unan- swerability tests (Rajpurkar et al., 2018; Madhusud- han et al., 2025; Wen et al., 2025). Nonword or malformed-input probes avoid public-person evi- dence by moving away from plausible names (Ra- makrishna et al., 2023; Vitevitch, 2025). PUN occu- pies the middle ground: the input remains a plau- sible full-name prompt, but indexed full-name ev- idence and full-name-level ambiguity signals are controlled by construction. When a model pro- duces person-specific facts for such a name, those facts are unsupported relative to the documented validation run of PUN rather than attributable to indexed full-name evidence found by the protocol. 4 The PUN Protocol Given candidate two-component name strings, PUN checks whether they satisfy the operational- unknownness criterion from Section 2: no indexed public full-name evidence and no full-name-level ambiguity signals under a documented validation run. The released showcase in Section 5 is one run of this protocol,PUN t , with fixed sources, providers, prompts, query templates, thresholds, and timestamps. Appendix A.2 gives the imple- mentation details needed for replication. As Figure 2 shows, the protocol has three stages: (1) candidate generation, (2) local format checks, and (3) web-facing validation. Candidate gener- ation builds plausible First Last strings from at- tested Wikidata name components. Local checks remove malformed or out-of-scope strings before web-facing validation. The web-facing stages then assign the evidential verdict, with a web-enabled LLM for broad triage and controlled web searches as the auditable basis for acceptance or rejection. The two web-facing stages are deliberately sep- arated. The web-enabled LLM can surface exact referents, spelling or order variants, fictional char- acters, organisations, famous-name attractors, and other nearby matches that fixed search templates may miss. It is not the evidential basis for accep- tance, because its retrieval path, query expansion, and ranking decisions are not fully observable. 4.1 Name Sources and Candidate Generation We use full names from Wikidata human entities as a source of attested name surface forms, rather than generating pronounceable but possibly fictional or parodic strings. We parse entity labels into given- name and surname positions and derive a country key by mapping the personâs recorded birthplace to a country 1 . Candidates are sampled by combin- ing one given-name component and one surname component into a First Last string (Figure 2.1). Sampled candidates are filtered by form, script, length, frequency, and duplicate checks. Candi- dates must have exactly two components, no ti- tles or suffixes, Latin-script form with diacritics retained, and a bounded total length. We also exclude overly common components and require each retained component to appear in multiple source records and to have native-language meta- data. These filters reduce malformed strings, pars- ing artefacts, and combinations that are unlikely to survive evidence screening (Figure 2.2). They make plausibility a documented form condition rather than a claim that a name is culturally repre- sentative, legally valid, or natural in every naming system. 1 The source-country key is used only for sampling and au- dit. It is not assigned to generated names and is not interpreted as nationality, ethnicity, language, or demographic status. 3 1. Candidate generation 2. Local checks 3. Web-facing validation Wikidata name pools birthplace (country) Generation sample one given and one surname Format Validation First Last; no titles; no suffixes; Latin script Web-enabled LLM queries Who is name? Controlled Web Search exact name; variants Operationally unknown Examples Givens: Adeline (France); AfrĂąnio (Brazil); Anjali (India); Akifumi (Japan) Surnames: Söllig (Germany); AbadĂa (Spain); Adekunle (Nigeria); Antonenko (Ukraine) given (Brazil) + surname (Ukraine)â âAfrĂąnio Antonenkoâ âDidier-Jules Sölligâ âDr. Zoe van Delkâ âhas title âMoffat Bourdeâ âMehmet Novickâ âfound, simi- lar, or attractor âPinsach Nibbrigâ âIronside Birlikâ âfull-name hit, near trace, or search hint Figure 2: Overview of PUNâs three stages: candidate generation, local format checks, and web-facing validation. 4.2 Web-facing Validation After generation and local checks, candidates are first screened with a web-enabled LLM using a person-identification query (âWho is name?â). The response is adjudicated into one of four labels: âą found: the exact full name is tied to concrete biographical detail. âą ambiguous: the response points to a similar spelling, reordered form, partial-token person, fictional character, organisation, place, work, product, or other nearby referent. âą fail : the response is policy-like, unusable, inconclusive, or otherwise failed. âą no_info: the response reports no information about the exact full-name string and mentions no alternate person or entity. Only candidates labelledno_infoproceed to controlled web search, where we check the exact full name and configured variants, including quoted and unquoted forms, reversed order, ASCII-folded forms for names with diacritics, and bounded spelling or phonetic variants. These variants are intended to catch cases where a bare exact query misses evidence but nearby forms, search correc- tions, or reordered strings do. A web search candidate is rejected asfoundif the exact or configured full-name form appears in a person-like context in a search-result title, snippet, URL, knowledge-panel field, or page con- tent. We operationalise person-like context with fixed biographical terms, person-oriented domains, and knowledge-panel type cues. If the name ap- pears only in a non-person or unclear context, or if search corrections and configured variants point to a nearby full-name form, the candidate is re- jected asambiguous. A name is accepted as oper- ationally unknown only when controlled search finds neither indexed public full-name evidence nor full-name-level ambiguity signals. 4.3 Comparison-Name Construction Alongside the operationally unknown names, we construct comparison names for boundary cases around operational unknownness. They let us (i) compare accepted unknown names with names of public people whose fame and exact-name widespreadness varies, and (i) use strings that are close to public-person names at controlled sur- face distances (e.g., Mike MyersâMice Myers âMikeuelaOlou) to measure how strongly mod- els map unfamiliar name-like strings onto existing names as surface similarity decreases. Public-person controls are sampled from Wiki- data human entities. We use page views, article length, language count, and sitelink count as a re- producible famousness proxy, and combine this with quoted exact-name search-result counts as a proxy for full-name widespreadness. This sepa- rates a personâs public prominence from the web footprint of the literal full-name string. Famous-name distance controls are build by per- turbing Wikidata-linked public-person names in 12 increasingly distant layers, from close orthographic edits to suffix attachments and fragmentary recom- binations. Appendix A.7 reports the release strata and construction details. 5 The PUN Resource The released resource is a showcase set from one documented run of PUN, not a dataset of perma- nently unknown people. It contains 300 full names accepted as operationally unknown under PUN, to- gether with 300 public-person comparison names of varying fame and full-name prevalence, and 12 x 300 = 3,600 famous-name distance controls. The entire source code and showcase resource are pub- licly available under C BY-NC 4.0. 2 2 https://anonymous.4open.science/r/PUN 4 Other 17 12 1310 9 6 69 Brazil 3 12 0 11 9 Germany 0 42214 6 Japan 533 2 3 2 10 Turkey 65 2 3 11 10 U.S. 05 124 318 France 53360 2 20 U.S. Italy RussiaBrazil France Hungary Other surname-source country given-source country Figure 3: Source-country pair distribution for the 300 accepted operationally unknown names. 5.1 Name Candidate Generation We use Wikidata human labels as an auditable source of attested name components, not as a rep- resentative sample of global naming practices. For this run, we processed the Wikidata dump from May 2026. It yielded 13.5M human entities, of which 4.17M entered the source pool after name parsing and birthplace-country mapping. Tokenis- ing this pool produced 356,558 distinct given-name components and 989,563 surname components. Af- ter local filters, 8,404 given names and 30,576 sur- names remained eligible (details in Appendix A.1). We sample names components by source- country rather than by raw pool size to reduce geographical skew, and disallow same-country re- combinations to lower the chance of reconstruct- ing attested within-country full names. The gen- erated evaluation-scope candidates span 545 ob- served source-country pairs. Appendix Figures 9 and 10 show the full filtered-slot and generated- candidate country-pair distributions. 5.2 Name Validation Yield Figure 4 summarises the validation yield from 52,726 generated candidates. Local checks re- move 2,849 malformed or out-of-scope strings be- fore web-facing validation. Among the remaining candidates, the web-enabled LLM screen is domi- nated by ambiguity: 36,097 candidates are linked to nearby names, variants, famous-name attractors, or other referents, while 2,449 are tied to exact-person evidence. A further 9,550 produce failed outputs, largely because of policy-like refusals. Only 1,781 candidates receive ano_infolabel and proceed to controlled search. Controlled search rejects most of these web- Format validation 52,726 First Last candidates pass 49,877 (94.6%) reject 2,849 (5.4%) Web-enabled LLM queries ambiguous 36,097 (72.4%) fail 9,550 (19.1%) found 2,449 (4.9%) no_info 1,781 (3.6%) Controlled web search ambiguous 1,372 (77.0%) found 108 (6.1%) operationally unknown 300 (16.8%) Figure 4: Validation yield for our showcase run of PUN. LLMno_infocandidates. It finds 108 exact- person traces and 1,372 ambiguity signals, leaving 300 accepted names. The accepted names therefore make up 0.6% of the generated candidates (roughly one name per 176 candidates). Figure 3 shows the source-country pair distribution for these names. The rejected cases are informative because they show why exact quoted search alone is insufficient. Figure 5 groups common web-LLM ambiguity mo- tifs, including similar names, partial-token matches, reordered forms, and nearby public referents. Fig- ure 6 groups failed responses, such as policy-like refusals. These patterns motivate the controlled- search ablations reported in Section 7.2. 0100200300400500600700 0 5 10 15 20 25 30 35 historical/family reference corrected person match non-person entity similar public figure no exact name in response historical/family reference corrected person match non-person entity similar public figure no exact name in response LLM response motif frequency LLM response length (sentences) Figure 5: LLM response motifs for ambiguous. 6 Experimental Setup 6.1 Protocol Reproducibility The validation stack contains two hosted model components:the web-enabled LLM (gpt-4.1-mini) that retrieves or synthesises evidence, and the adjudicator LLM (gpt-5.2) that maps the response to a discrete label. Even with fixed prompts and zero temperature, these are not fully deterministic. Following common practice in recent LLM evaluations, we therefore 5 05001,0001,5002,000 0 5 10 15 policy refusal partial refusal blank response policy refusal partial refusal blank response LLM response motif frequency LLM response length (sentences) Figure 6: LLM response motifs for fail. estimate stochastic variation with three repeated reruns on pre-specified cohorts (20% of the 52,726 candidates). We use the same model versions, prompts, hyperparameters, and database snapshot, and report agreement over the paper-facing labels found, ambiguous, no_info, and fail. 6.2 Protocol Ablations We ablate only the controlled-search validator, us- ing the 1,781 candidates that passed web-LLM screening withno_info. Each ablation removes one search guardrail and reports how many addi- tional candidates would be accepted relative to the full validator. Generation-policy ablations are re- ported in Appendix A.10. 6.3 Name-Likeness We evaluate whether accepted PUN names are tech- nically name-like under two string-level diagnos- tics: character-level predictability and tokenizer fragmentation. For character-level predictability, we train an LSTM language model on ParaNames, a multilingual corpus of 140 million names across 400+ languages (SĂ€levĂ€ and Lignos, 2024), and score each evaluation string by bits per character (BPC). Lower BPC indicates that a string is more predictable under the name model. We compare accepted operationally unknown names against public-person controls, famous-name perturbation controls, and three non-name control sets. For tokenizer fragmentation, we use the fact that subword tokenizers tend to keep frequent strings or fragments as larger units and split rarer forms into more pieces. For each namenand tokenizerT, we compute|T (n)|/|n|, and use the median across 14 tokenizers 3 as the nameâs fragmentation score. 3 OpenAIr50k_base,p50k_base,cl100k_base, and o200k_base; GPT-2, RoBERTa, OPT, BERT, multilingual BERT, XLM-R, mT5, mBART-50, Qwen2.5, and Mistral. Appendix A.8 gives additional LSTM details, and Appendix A.9 gives the tokenizer-footprint diag- nostics. 6.4 Human Agreement We conducted a Prolific study withn= 204 partici- pants from 27 self-reported language backgrounds and 31 countries of residence. The study asked whether people make the same distinctions as the protocol. Participants first judged whether a string looks like a plausible full name, covering (i) ac- cepted unknown names, (i) real-person names that should be findable, and (i) strings that should clearly not be names. Then, they inspected ev- idence by opening prepared Google, Bing, and DuckDuckGo queries in their own browsers, as well as by using web-enabled ChatGPT, Gemini, or Perplexity to search for selected unknown and ambiguous names, and relabelling saved web-LLM outputs. The study covers 495 unique person- name strings: 300 released unknown names, 108 controlled-search positives, 12 controlled-search ambiguous cases added for live-chatbot lookup, and 75 additional names unique to saved web- LLM outputs for classifier-agreement labelling. This allows us to separate name form plausibil- ity from subjective human judgement, automated web-LLM screening from human-driven chatbot lookups, controlled-search agreement, and humanâ LLM classifier-label agreement. Detailed study design and setup are in Appendix A.11. 7 Results 7.1 Protocol Reproducibility End-to-end agreement is 75.0% with very small between-run variation (standard deviation 0.1 per- centage points), while classifier-only agreement on the saved original web responses is 98.2%. This in- dicates that most instability comes from rerunning the web-enabled model and its search context, not from the discrete classifier. Theno_infolabel is substantially less stable than the classifier itself (31.6% end-to-end agree- ment versus 92.4% classifier-only agreement in the ten-replicateno_inforerun). In ten full reruns of the sourceno_infocohort, only around 31% of labels remainno_infoin any single rerun. Most switches are toambiguous(57.7%), while switches tofoundare rare (1.4%). This supports the design choice that web-LLMno_infois only a triage sig- nal, where acceptance as operationally unknown 6 0246810 BPC (lower = more name-like) Public figure PUN d = 2 d = 4 d = 6 d = 8 Word-like non-name Structured gibberish Random string Public Op. unknown Perturbation Non-name Mean Perturbation levels d = 3, 5, 7 omitted. Figure 7: LSTM name-likeness by group, measured as bits per character (BPC; lower = more name-like). requires the subsequent controlled-search stage. If we impose the stricter name-level criterion that a released name must remainno_infoin every web- LLM rerun, 49 of the 300 released names survive three reruns and 27 survive ten reruns. We refer to these stricter subsets asstable_no_info_3and stable_no_info_10. 7.2 Protocol Ablations Figure 4 shows the main validation attrition: only 300 of the 1,781 web-LLMno_infocandidates survive controlled search. Table 1 asks which controlled-search guardrails account for this reduc- tion. Exact quoted search alone would accept 1,115 names, i.e., 815 additional names rejected by the full validator. Removing reordered, ASCII-folded, vowel, and phonetic variants would accept 942 names, making variant search the largest search- side guardrail. ValidatorAcceptedâ Full validator300â Exact quoted only1,115+815 No fuzzy/reordered variants942+642 No unquoted queries356+56 Table 1: Controlled-search ablations on the 1,781 web- LLMno_infocandidates (ârelative to full validator). 7.3 Name-Likeness Our LSTM and tokenizer experiments show that ac- cepted PUN names are more name-like than famous- name perturbations and non-name controls, but less 01k1M1B Exact-name Google results (log scale) 2 4 6 8 10 12 BPC (lower = more name-like) Public figure PUN Perturbation Non-name Group centroid Rcio Sekkwis Jaiust Ave FYKJT SXPICCBZTH Archiviato Basketball Merrick Garland John Terry Akshaya Boerwinkel Abdulaziz Kolesnichenko Figure 8: Web footprint versus LSTM name-likeness. predictable than public-person names. In Figure 7, lower bits per character (BPC) means that a string is more predictable under the ParaNames-trained name model. Accepted PUN names have median BPC 3.16, above public-person names (2.45) but below all displayed famous-name perturbations and non-name controls, including word-like non- names (4.24), structured gibberish (7.75), and ran- dom strings (8.57). Figure 8 visualises how public- person names are highly name-like but have a large web footprint, while random strings have little foot- print but are not name-like. Accepted PUN names instead cluster close to the ideal lower-left region with few exact-name Google results and low BPC. Tokenizer fragmentation supports the same in- terpretation at the level of name parts. Accepted PUN names have median fragmentation 0.36 pieces per character, close to public-person names and word-like non-names (both 0.33), and far below random strings (0.59). Fragmentation correlates with LSTM BPC across all scored rows (r = 0.74, SpearmanÏ = 0.63), but only weakly with exact- name search prevalence (SpearmanÏ = â0.16). Appendix Figures 12 and 13 report the tokenizer plots. 7.4 Human Agreement The human agreement audit tests whether the pro- tocolâs evidential distinctions are recoverable by non-expert participants. We separate four ques- tions: whether accepted names look like plausi- ble full names; whether participants can find per- 7 son evidence for names accepted as operationally unknown; whether they can recover evidence for names rejected by controlled search asfound; and whether they assign the same labels as the auto- mated classifier when shown saved web-LLM out- puts. Participants were given definitions offound, ambiguous,no_info, andfail, and had to pass a comprehension check before beginning the study. Name plausibility. Participants judged the 300 released unknown names as plausible full names in 63.0% of ratings. By comparison, names rejected by PUN, were judged plausible in 77.2% of rat- ings. The non-name controls were rarely rated as plausible: 9.8% for word-like no-names, 0.0% for structured gibberish, and 0.5% for random charac- ter strings. This shows that operational plausibility, as validated via LSTM, and human-perceived plau- sibility are related but distinct properties. Controlled-searchagreement. Participants searched the 300 released unknown names using Google, Bing, and DuckDuckGo. They selectedno_infoin 75.8% of judgments and foundin 3.0%. Most remaining disagreement wasambiguous(17.2%), typically reflecting nearby names, partial-token matches, spelling variants, or search-provider corrections.The stricterstable_no_info_10subset behaves more strongly under the same audit: participants selected no_infoin 84.0% of judgments,ambiguousin 12.3%, andfoundin 2.5%. Thus, human search mostly supports the released verdicts and confirms that ambiguity is the main boundary case. Web-enabled chatbots.When participants used an assigned web-enabled chatbot to search for the 300 operationally unknown names, they labelled the chatbot response asno_infoin 82.8% of cases, ambiguousin 12.1%,foundin 2.0%, andfailin 3.0%. Thefoundrate was concentrated in Chat- GPT responses (5.4%), with 0.0% for Gemini and 0.0% for Perplexity. For names where the PUN web- enabled LLM returnedno_info, participants using chatbots also usually failed to find evidence. The no_inforate was 88.9% with ChatGPT, 100.0% with Gemini, and 88.2% with Perplexity. Finally, the saved-response labelling task checks whether participants can recognise exact-person evidence when it is present in a web-LLM output. They separate exact-person evidence from all other outcomes in 88.7% of cases. Fine-grained four- way agreement is lower, especially forambiguous andfail, because these labels require distinguish- ing near-match evidence, absence statements, and unusable responses. Overall, the human audit sup- ports the main protocol decision: accepted names are generally perceived as names and yield hardly any recoverable person evidence, with disagree- ment being concentrated in ambiguous boundary cases that our reproducibility tests surfaced too. 8 Discussion PUN is best understood as a measurement device, not as a catalogue of non-existent people. Our experiments confirm that operationally unknown names remain name-like, and human study partic- ipants are unable to recover person evidence in 97% of cases. At the same time, the claim remains operational, as verdicts can change based on demo- graphics and social associations. Our protocol enables controlled follow-on stud- ies. For example, accepted names can be paired with synthetic facts to test whether models repro- duce, distort, protect, or forget introduced person records. They can serve as negative controls in privacy audits, since a method that extracts claims for operationally unknown names may not be mea- suring leakage alone. They can support name-bias studies with fewer confounds from known individ- uals, and machine-unlearning benchmarks where the inserted records and deletion targets are known even when original pretraining data are opaque. Revalidating the same unknown names over time can also measure how search systems and retrieval- augmented models begin attaching information to a previously unsupported full-name string. 9 Conclusion We introduced PUN, a protocol for constructing plausible person-name prompts that are opera- tionally unknown under a documented validation run. We demonstrated that person names should not be treated as interchangeable prompt strings. Their evidential status is an experimental variable. PUN supplies one missing baseline for measuring what LLMs do when a prompt has person-name form but lacks validated public full-name evidence. Fu- ture work should extend the protocol beyond Latin- script two-component names, quantify provider and locale sensitivity, and integrate operationally unknown names into downstream benchmarks. 8 Limitations Operationalunknownnessisprovider-, configuration-,locale-,and time-dependent. Search engines may miss evidence, change indexes, alter spell correction, or report unstable result counts. Hosted LLMs used for screening and adjudication may also change over time, even when prompts and decoding settings are fixed. The released run should therefore be read as a documented validation snapshot rather than as a permanent property of the accepted strings. The present resource is restricted to Latin-script two-component First Last strings. It does not cover non-Latin scripts, transliteration, patronymics, mononyms, multi-part surnames, name particles, culturally variable name order, or many other nam- ing practices. The Wikidata-derived source pools are not demographically representative, and the source-country key is only a sampling and audit proxy, not a claim about nationality, ethnicity, lan- guage, or cultural identity. The 300-name seed set is intentionally small and conservative. It is intended as a showcase resource and evaluation control, not as a comprehensive cat- alogue of unknown names. Before high-stakes or time-sensitive use, users should revalidate names under a newly documentedPUN t . Absence of indexed full-name evidence underPUN t is not ev- idence that no person has the name, that no private or offline record contains it, or that no model has encountered the string during training. Finally, the protocol controls full-name evidence and ambiguity, but does not remove all social mean- ing from names. Given-name and surname com- ponents may still carry associations with language, region, gender, religion, class, ethnicity, or other perceived attributes. Evaluations using PUN should therefore interpret model behaviour as conditioned on plausible name strings with controlled full-name evidence, not demographically neutral identifiers. Ethical Considerations This work constructs realistic-looking person-name prompts for evaluation. Such names may coin- cide with real but non-indexed individuals, or with people whose public evidence was not found by the validation run. In addition, the public-person comparison controls intentionally contain names of real public figures sampled from Wikidata. The resource must therefore not be used to claim that accepted names prove non-existence, to fabricate biographies, to impersonate real people, or to tar- get people with the same or similar names. Public- person controls should likewise be used only as aggregate controls, not for individual-level claims about the named persons. The intended use of the resource is aggregate evaluation of LLM behaviour, including factuality, abstention, privacy-related be- haviour, and name-conditioned conflation. The protocol uses public web evidence and Wikidata-derived name components. Where raw search traces contain personal information, we min- imise disclosure by releasing hashes, labels, query metadata, and summary evidence rather than unnec- essary personal snippets. Repository maintainers should avoid redistributing sensitive personal de- tails that are not needed for reproducibility. We conducted a human agreement audit with Prolific participants. Participants consented, used a desktop or laptop browser, and completed compre- hension and attention checks. The study collected task responses and platform metadata for assign- ment, exclusion, and aggregate reporting. Partic- ipants were paid USD 8.09/h for a median com- pletion time of 8.2 minutes. No participant-level identifying information is released. LLM-based Tools.We used LLM-based tools in a limited way during manuscript preparation and implementation. GitHub Copilot was used for code completion and minor refactoring, while GPT 5.5 was used to suggest alternative phrasings, and pol- ish Matplotlib visualisations and L A T E X formatting, e.g., table layouts in the appendix sections. All algorithmic design decisions, experimental imple- mentation and execution, data analysis, and sub- stantive writing were carried out by the authors, and we verified all AI-assisted edits for correctness. References Haozhe An, Christabel Acquaye, Colin Wang, Zongxia Li, and Rachel Rudinger. 2024. Do large language models discriminate in hiring decisions on the ba- sis of race, ethnicity, and gender? In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 386â397, Bangkok, Thailand. Association for Computational Linguistics. Haozhe An, Connor Baumler, Abhilasha Sancheti, and Rachel Rudinger. 2025. On the mutual influence of gender and occupation in LLM representations. In Proceedings of the 63rd Annual Meeting of the Asso- ciation for Computational Linguistics, pages 1663â 1680. Association for Computational Linguistics. 9 Javier Artiles, Julio Gonzalo, and Satoshi Sekine. 2007. The SemEval-2007 WePS evaluation: Establishing a benchmark for the web people search task. In Pro- ceedings of the Fourth International Workshop on Semantic Evaluations (SemEval-2007), pages 64â69, Prague, Czech Republic. Association for Computa- tional Linguistics. Krisztian Balog, Leif Azzopardi, and Maarten de Rijke. 2009. Resolving person names in web people search. In Weaving Services and People on the World Wide Web, pages 301â323. Springer. Judit Bar-Ilan. 1999. Search engine results over time: A case study on search engine stability. Cybermetrics, 2/3(1):1. Nicholas Carlini, Florian TramĂšr, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom B. Brown, Dawn Song, Ăl- far Erlingsson, Alina Oprea, and Colin Raffel. 2021. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Secu- rity 21), pages 2633â2650. USENIX Association. Maria De-Arteaga, Alexey Romanov, Hanna Wal- lach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, and Adam Tauman Kalai. 2019. Bias in bios: A case study of semantic representation bias in a high-stakes setting. In Proceedings of the Conference on Fair- ness, Accountability, and Transparency, FAT* â19, pages 120â128, New York, NY, USA. Association for Computing Machinery. AgustĂn D. Delgado, Soto Montalvo, Raquel MartĂnez- Unanue, and VĂctor Fresno. 2018. A survey of per- son name disambiguation on the web. IEEE Access, 6:59496â59514. Elisabeth Eder, Ulrike Krieg-Holz, and Udo Hahn. 2019. De-identification of emails: Pseudonymizing privacy- sensitive data in a German email corpus. In Pro- ceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2019), pages 259â269, Varna, Bulgaria. INCOMA Ltd. Vagrant Gautam, Arjun Subramonian, Anne Lauscher, and Os Keyes. 2024. Stop! in the name of flaws: Disentangling personal names and sociodemographic attributes in NLP. In Proceedings of the 5th Work- shop on Gender Bias in Natural Language Process- ing (GeBNLP), pages 323â337, Bangkok, Thailand. Association for Computational Linguistics. Jie Huang, Hanyin Shao, and Kevin Chen-Chuan Chang. 2022. Are large pre-trained language models leaking your personal information? In Findings of the Asso- ciation for Computational Linguistics: EMNLP 2022, pages 2038â2047, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. Sullam Jeoung, Jana Diesner, and Halil Kilicoglu. 2023. Examining the causal impact of first names on lan- guage models: The case of social commonsense rea- soning. In Proceedings of the 3rd Workshop on Trust- worthy Natural Language Processing (TrustNLP 2023), pages 61â72, Toronto, Canada. Association for Computational Linguistics. Seongho Keum, Dongwon Shin, Leo Marchyok, Sanghyun Hong, and Sooel Son. 2025.Private investigator: Extracting personally identifiable in- formation from large language models using opti- mized prompts. In 34th USENIX Security Symposium (USENIX Security 25), pages 8175â8194. USENIX Association. Edwin D. Lawson. 2016. Personal naming systems. In Carole Hough, editor, The Oxford Handbook of Names and Naming, pages 169â198. Oxford Univer- sity Press, Oxford. Zhuoyan Li, Hangxiao Zhu, Zhuoran Lu, and Ming Yin. 2023. Synthetic data generation with large lan- guage models for text classification: Potential and limitations. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Process- ing, pages 10443â10461, Singapore. Association for Computational Linguistics. Pierre Lison, IldikĂł PilĂĄn, David Sanchez, Montser- rat Batet, and Lilja Ăvrelid. 2021. Anonymisation models for text data: State of the art, challenges and future directions. In Proceedings of the 59th Annual Meeting of the Association for Computational Lin- guistics and the 11th International Joint Conference on Natural Language Processing, pages 4188â4203, Online. Association for Computational Linguistics. Lin Long, Rui Wang, Ruixuan Xiao, Junbo Zhao, Xiao Ding, Gang Chen, and Haobo Wang. 2024. On LLMs-driven synthetic data generation, curation, and evaluation: A survey. In Findings of the Associa- tion for Computational Linguistics: ACL 2024, pages 11065â11082, Bangkok, Thailand. Association for Computational Linguistics. Nishanth Madhusudhan, Sathwik Tejaswi Madhusud- han, Vikas Yadav, and Masoud Hashemi. 2025. Do LLMs know when to NOT answer? investigating abstention abilities of large language models. In Proceedings of the 31st International Conference on Computational Linguistics, pages 9329â9345, Abu Dhabi, UAE. Association for Computational Linguis- tics. Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettle- moyer, and Hannaneh Hajishirzi. 2023. FActScore: Fine-grained atomic evaluation of factual precision in long form text generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12076â12100, Singa- pore. Association for Computational Linguistics. Heiko Motschenbacher. 2020. Corpus linguistic ono- mastics: A plea for a corpus-based investigation of names. Names, 68(2):88â103. 10 Sambulo Ndlovu and Svenja Völkel. 2023. Introduction: The cultural aspect of personal names. In Sambulo Ndlovu, editor, Personal Names and Naming from an Anthropological-Linguistic Perspective, pages 1â11. De Gruyter Mouton, Berlin and Boston. Siddhesh Milind Pawar, Arnav Arora, Lucie-AimĂ©e Kaf- fee, and Isabelle Augenstein. 2025. Presumed cul- tural identity: How names shape LLM responses. In Findings of the Association for Computational Lin- guistics: EMNLP 2025, pages 22147â22172, Suzhou, China. Association for Computational Linguistics. Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. Know what you donât know: Unanswerable ques- tions for SQuAD. In Proceedings of the 56th An- nual Meeting of the Association for Computational Linguistics, pages 784â789, Melbourne, Australia. Association for Computational Linguistics. Anil Ramakrishna, Rahul Gupta, Jens Lehmann, and Morteza Ziyadi. 2023. INVITE: a testbed of au- tomatically generated invalid questions to evaluate large language models for hallucinations. In Find- ings of the Association for Computational Linguis- tics: EMNLP 2023, pages 5422â5429, Singapore. Association for Computational Linguistics. Alexey Romanov, Maria De-Arteaga, Hanna Wal- lach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, Anna Rumshisky, and Adam Tauman Kalai. 2019. Whatâs in a name? reducing bias in bios without access to protected attributes. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pages 4187â4195. Association for Computational Linguis- tics. JonneSĂ€levĂ€andConstantineLignos.2024. ParaNames 1.0:Creating an entity name cor- pus for 400+ languages using Wikidata.In Proceedings of the 2024 Joint International Con- ference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 12599â12610, Torino, Italia. ELRA and ICCL. David SĂĄnchez, Laura MartĂnez-Sanahuja, and Montser- rat Batet. 2018. Survey and evaluation of web search engine hit counts as research tools in computational linguistics. Information Systems, 73:50â60. Dimitri Staufer, Kirsten Morehouse, David Hartmann, and Bettina Berendt. 2026. Human-centred LLM privacy audits: Findings and frictions. In Proceed- ings of the CHI Conference on Human Factors in Computing Systems (CHI â26), Workshop on Human- centered Evaluation and Auditing of Language Mod- els (HEAL). ArXiv:2603.12094. Maria Irena Szawerna, Simon Dobnik, Therese Lind- ström Tiedemann, Ricardo Muñoz SĂĄnchez, Xuan- Son Vu, and Elena Volodina. 2024. Pseudonymiza- tion categories across domain boundaries. In Pro- ceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 13303â 13314, Torino, Italia. ELRA and ICCL. Maria Irena Szawerna and Jacob Lee Suchardt. 2026. Fill-in-the-blanks: Automatic generation and evalua- tion of language modelsâ pseudonyms for english and swedish texts. In Proceedings of the Fifteenth Lan- guage Resources and Evaluation Conference (LREC 2026), pages 1155â1169, Palma, Mallorca, Spain. European Language Resources Association (ELRA). Juntao Tan, Liangwei Yang, Zuxin Liu, Zhiwei Liu, Rithesh R N, Tulika Manoj Awalgaonkar, Jianguo Zhang, Weiran Yao, Ming Zhu, Shirley Kokane, Silvio Savarese, Huan Wang, Caiming Xiong, and Shelby Heinecke. 2025. PersonaBench: Evaluating AI models on understanding personal information through accessing (synthetic) private user data. In Findings of the Association for Computational Lin- guistics: ACL 2025, pages 878â893, Vienna, Austria. Association for Computational Linguistics. Ahmet Uyar. 2009. Investigation of the accuracy of search engine hit counts. Journal of Information Science, 35(4):469â480. Willy Van Langendonck. 2007. Theory and Typology of Proper Names, volume 168 of Trends in Linguistics. Studies and Monographs. Mouton de Gruyter, Berlin and New York. Michael S Vitevitch. 2025. Examining chat gpt with nonwords and machine psycholinguistic techniques. Plos one, 20(6):e0325612. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. Self-instruct: Aligning language models with self-generated instructions. In Proceed- ings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13484â13508, Toronto, Canada. Association for Computational Linguistics. Ze Wang, Zekun Wu, Xin Guan, Michael Thaler, Adri- ano Koshiyama, Skylar Lu, Sachin Beepath, Ediz Ertekin, and Maria Perez-Ortiz. 2024. JobFair: A framework for benchmarking gender hiring bias in large language models. In Findings of the Associa- tion for Computational Linguistics: EMNLP 2024, pages 3227â3246, Miami, Florida, USA. Association for Computational Linguistics. Bingbing Wen, Jihan Yao, Shangbin Feng, Chenjun Xu, Yulia Tsvetkov, Bill Howe, and Lucy Lu Wang. 2025. Know your limits: A survey of abstention in large language models. Transactions of the Association for Computational Linguistics, 13:529â556. Oleksandr Yermilov, Vipul Raheja, and Artem Chern- odub. 2023. Privacy- and utility-preserving NLP with anonymized data: A case study of pseudonymization. In Proceedings of the 3rd Workshop on Trustwor- thy Natural Language Processing (TrustNLP 2023), pages 232â241, Toronto, Canada. Association for Computational Linguistics. 11 Ksenia Zhagorina, Pavel Braslavski, and Vladimir Gu- sev. 2018. Personal names popularity estimation and its application to record linkage. In European Con- ference on Advances in Databases and Information Systems, pages 71â79. Springer. A Appendix This appendix provides supplementary details for the main analyses:candidate generation (Appendix A.1), protocol implementation (Ap- pendix A.2), comparison-name construction (Ap- pendix A.7), ablations (Appendix A.10), name- likeness diagnostics (Appendices A.8 and A.9), and the human agreement study (Appendix A.11). A.1 Name Candidate Generation Details This appendix records the construction choices behind Section 5.1. We store the source entity, the original label, the component role, and a source-country key derived from the source per- sonâs recorded birthplace. The source-country key is used only for sampling and audit. It is not a claim about the generated name or about the source personâs nationality, ethnicity, language, or demo- graphic status. Coverage and script mix. The May 2026 Wiki- data dump yielded 13.5M human entities. The selected labels are already heavily Latin-script, and the mapped source pool remains so after name pars- ing and birthplace-country mapping. Table 2 sum- marises this skew. These counts are extraction di- agnostics, not demographic estimates. ScriptSelected lbsPct.Mapped lbsPct. Latin13,039,63996.354,032,78796.76 Cyrillic238,4161.76111,8722.68 Han/CJK118,1790.872100.01 Arabic40,5380.308,0480.19 Other115,7580.8514,8140.36 Table 2: Compact script distribution for the Wikidata human labels retained by the pipeline. âSelected lbsâ are the one-label-per-human extraction output. âmapped lbsâ are the subset with a parsable name and mapped birthplace-country key. Source-country-key skew. The mapped source pool contains 4,167,732 humans across 667 source- country keys. Before filtering, Germany and the United States are the largest keys, and the top ten keys account for 56.3% of the mapped pool. After filtering, 40,715 eligible component slots remain across 26 country keys, with U.S., French, German, Italian, and Russian source pools accounting for 70.5% combined. Table 3 gives a compact sum- mary. Source-country keyMSHPct.ECSPct. Germany618,95414.854,66211.45 United States433,64910.408,86921.78 France219,7345.278,17720.08 United Kingdom193,4654.641,4053.45 Italy182,2484.374,55911.20 Russia139,5413.352,4456.01 Other keys2,380,34157.1210,59826.03 Table 3: Compact source-country-key skew before and after component filtering. MSH = mapped source hu- mans; ECS = eligible component slots. Component filters. Tokenising the mapped source humans gives 356,558 distinct given-name components and 989,563 distinct surname compo- nents. Components are retained only if (a) they are sufficiently rare within their source-country pool, (b) appear in at least two source records, (c) have length at least six, (d) use Latin-script form, and (e) have non-empty native-language metadata. These filters reduce common-name collisions, parsing artefacts, very short tokens, and components with weak provenance. Table 4 gives the cumulative attrition. Cumulative filterGivenSurname Raw pool356,558989,563 Rarity p†5Ă10 â5 259,192756,262 Pool occurrencesâ„ 264,692221,573 Lengthâ„ 645,266184,410 Latin-script form38,841182,578 Native-language metadata present8,40430,576 Table 4: Cumulative reduction of distinct Wikidata- derived components under the release filters. Candidate sampling and local validation.Can- didate names are sampled by combining one eli- gible given-name component and one eligible sur- name component into a two-token First Last string. We sample across source-country-key pairs rather than directly from raw component counts, and re- ject same-country recombinations. This prevents the largest Wikidata pools from dominating candi- date generation and reduces the chance of recon- structing attested same-country full names. Local validation is applied before any web- facing call. It rejects strings with invalid token count, unsupported script, malformed punctuation, 12 duplicate form, titles, suffixes, excessive length, or components that fail the rarity, support, length, script, or provenance constraints. These checks define operational form plausibility for this paper run. They do not guarantee sociolinguistic nat- uralness, legal validity, demographic balance, or cross-cultural representativeness. A.2 Protocol Implementation Details This appendix subsection records implementation details for the protocol summarised in Section 4. These details are part of the documented validation runPUN t and should be fixed when reproducing or revalidating the released resource. Wikidata extraction and component pools.We extract labels from Wikidata human entities and parse retained labels into candidate given-name and surname positions. For each component, we store the source entity, the original full-name label, the component role, and a source-country key de- rived from the source personâs recorded birthplace. Components are retained only when they pass the local form constraints used by the release pipeline. In the paper run, candidates had to use Latin-script components, retain diacritics, have components of at least six characters, have no more than 60 charac- ters in total, contain no titles or suffixes, and avoid unsupported punctuation. Components also had to have within-country-pool probability at most 5Ă 10 â5 , appear in at least two source records, and have a non-empty Wikidata native-language entry. Wikidata coverage, script mix, and country skew.Tables 5 and 6 describe the Wikidata slice used by the pipeline. They do not describe ev- ery alias or every language-specific label stored in Wikidata. For each human entity, the extraction keeps one label under the paper runâs language- priority policy. We then assign that retained la- bel to a Unicode script bucket. These buckets are script diagnostics, not language, nationality, ethnic- ity, or demographic labels. The retained labels are mostly English-priority labels (86.2%) or multilin- gual labels (7.8%); the next largest label-language shares are Russian (1.1%), Chinese (0.6%), French (0.4%), Japanese (0.2%), German (0.2%), Spanish (0.2%), and Arabic (0.2%). Candidate sampling and local validation.Can- didate names are sampled by combining one eli- gible given-name component and one eligible sur- name component into a two-token First Last string. Local validation is applied before any web-facing call. It rejects strings with invalid token count, unsupported scripts, malformed punctuation, dupli- cate forms, titles, suffixes, excessive length, or com- ponents that fail the frequency and provenance con- straints. These checks are intended to reduce pars- ing artefacts and extremely common name compo- nents, not to guarantee sociolinguistic naturalness. Web-enabled LLM screening. For the paper run, locally valid candidates were queried with the prompt âWho is name?â. We used a web- enabled model for broad evidence triage and a sep- arate adjudicator model to map the response to a discrete label. The screen usedgpt-4.1-miniwith web_search_previewandtemperature: 0.0; the adjudicator usedgpt-5.2withtemperature: 0.0. The adjudicator assigned one of four labels: found,ambiguous,no_info, orfail. The full ad- judicator prompt and label rubric are reported in Appendix A.3. Only candidates labelledno_info were passed to controlled search. Controlled search queries. Controlled search was issued through Google Search viaserper.dev. The verifier checked the exact full name and a bounded set of configured equivalents. Query forms included quoted and unquoted full-name queries, reversed-order variants, ASCII-folded vari- ants for names with diacritics, exchanged-vowel variants, and phonetic variants. For example, a name such as JosĂ© NĂșñez is also checked as Jose Nunez, NĂșñez JosĂ©, vowel-edited forms, and pho- netic edits such as surname-final consonant sub- stitutions. These variants should catch full-name evidence, search corrections, and near-name ambi- guity that exact quoted search alone may miss. Evidence and ambiguity rules. A candidate is rejected asfoundwhen a search-result title, snip- pet, URL, knowledge-panel field, or inspected page content contains the exact or configured full- name form in a person-like context. Person-like context is operationalised with a fixed list of bi- ographical terms, person-oriented domains, and knowledge-panel type cues. A candidate is rejected asambiguouswhen the evidence does not estab- lish exact full-name person evidence but points to a nearby full-name form, spelling correction, re- ordered form, famous-person attractor, fictional character, organisation, place, work, product, or other unclear referent. Search corrections and con- figured variants that point to a different nearby full- 13 Script bucketSelected Wikidata-human labelsPct.Mapped source-human labelsPct. Latin13,039,63996.354,032,78796.76 Cyrillic238,4161.76111,8722.68 Han/CJK118,1790.872100.01 Arabic40,5380.308,0480.19 Hangul/Korean20,7000.1542 < 0.01 Hebrew14,3480.114,0970.10 Devanagari6,6330.05140 < 0.01 Japanese kana/mixed4,7710.04142 < 0.01 Armenian4,7660.042,8560.07 Thai4,5100.033140.01 Greek3,8270.031,3940.03 Bengali1,7530.01198 < 0.01 Myanmar/Burmese1,0350.01174 < 0.01 Georgian338 < 0.0119 < 0.01 Ethiopic/Amharic181 < 0.012 < 0.01 Thaana40 < 0.015 < 0.01 Tifinagh6 < 0.0100.00 Canadian Aboriginal syllabics/Inuktitut00.0000.00 Mongolian00.0000.00 Other or mixed script33,2900.255,4310.13 Table 5: Script distribution of the Wikidata human labels retained by the pipeline. âSelected labelsâ are the one-label-per-human extraction output under the paper runâs language-priority policy; âmapped source-human labelsâ are the subset with a parsable name and mapped birthplace-country key. Han/CJK and Japanese-kana buckets are Unicode script buckets, not language identifiers. Source-country keyMSHPct.ECSPct. Germany618,95414.854,66211.45 United States433,64910.408,86921.78 France219,7345.278,17720.08 United Kingdom193,4654.641,4053.45 Italy182,2484.374,55911.20 Czech Republic156,9213.775611.38 Spain140,6703.381,2002.95 Japan140,2293.361,7444.28 Russia139,5413.352,4456.01 Poland121,7432.929892.43 Austria74,6941.791240.30 Ukraine70,0731.683640.89 Table 6: Largest birthplace-derived source-country shares before and after release filters. The mapped source pool contains 4,167,732 humans across 667 coun- try keys; the top ten account for 56.3%. Eligible com- ponent slots are the remaining country-specific given- name and surname options after rarity, support, length, Latin-script, and native-language-metadata filters. The eligible set contains 40,715 slots across 26 country keys, with the top five accounting for 70.5%. MSH = Mapped source humans. ECS = Eligible component slots. name form are treated as ambiguity signals rather than as acceptance evidence. Stored audit trace. For each candidate, the run stores provenance and validation metadata suffi- cient for later inspection: sampled components, source records, local-validation outcome, web- enabled LLM response hashes, adjudicated labels, controlled-search query forms, provider metadata, timestamps, evidence labels, verdict reasons, and final acceptance status. Because operational un- knownness is time-bounded, later uses of the re- source should either cite the original validation run or rerun the controlled-search stage under a newly documented PUN t . A.3 Adjudicator Prompt and Label Rubric The web-enabled LLM screen uses a retrieval-style prompt, âWho is name?â. The returned text is then classified by a separate adjudicator. The pa- per run used the following rubric, with JSON-only output: Choose exactly one label:found,similar, attractor,no_info, orfail. Apply the deci- sion order below and stop at the first match.fail: refusal or policy text, garbled output, error text, or insufficient information to classify.found: the exact full name is explicitly identified as a specific person with concrete biographical detail; multi- ple people with the exact name are notfound, but an explicit alias or alternate-spelling equiva- lence to a specific person isfound.attractor: the response treats the queried name as a mis- spelling, typo, or close variant of a known public person.similar: any other person or entity is mentioned that is not the exact full name, includ- ing partial-token matches, alternate spellings with- out explicit alias equivalence, fictional characters, organisations, places, works, products, and pub- lic, social, or genealogy records.no_info: the response explicitly reports no information about the exact name and mentions no other people or entities; name meaning, etymology, or origin only also counts asno_info. Exact-name matching is case- and diacritics-insensitive; all tokens must 14 be present and token spelling must match, with surrounding titles allowed. Forpaper-facingverdicts,foundmaps toweb_evidence_found;similarand attractormap toambiguous;failmaps tovalidation_incomplete; andno_infois only a triage label.A name is accepted as no_indexed_full_name_evidence_foundonly after controlled search also completes without exact-person evidence or ambiguity signals. A.4 Controlled-Search Query Inventory Table 7 gives the controlled-search query inventory used for the paper run. Query variants are gen- erated deterministically from the normalised full- name string. The verifier records every executed query, its query type, provider metadata, search- result traces, and whether any trace is classified asexact_person_trace,ambiguity_signal, unrelated, orinconclusive. Acceptance re- quires a clean web-LLMno_infolabel and com- pletion of all configured mandatory query types. Query typeFormPurpose exact_quotedâ"First Last"âexact full-name evidence unquoted First Lastranked evidence and corrections reversed "Last First"; unquotedorder variants ascii_folddiacritics removedaccent-insensitive evidence vowel_variantone vowel editedspelling-near variants phonetic_variantone conservative editsound-near variants page_contenttop-result page textevidence beyond snippets Table 7: Controlled-search query inventory. The paper run used Google Search viaserper.dev, locale meta- data stored with the run, a two-pass policy, page-content fetching for top organic results, and configured caps of 200 vowel variants, 50 phonetic variants, and 500 total variants per name. The phonetic variant rules are deliberately conservative single-token substitutions:phâf, ckâk,câk,qâk,zâs,vâw,yâi,jâg, andshâsch.If an exact or configured full- name form appears in a person-like context, the verdict isweb_evidence_found; if search hints, spelling corrections, variants, reordered forms, or nearby entities point to another plausible ref- erent, the verdict isambiguous; if mandatory queries fail or traces are inconclusive, the verdict is validation_incomplete. A.5 Release Schema, Manifests, and Revalidation The public release separates accepted unknown names from comparison and rejected-control strata. Table 8 summarises the paper-facing files and the key audit columns. Raw provider responses and search traces may contain personal data from in- dexed web pages; the release therefore prioritises names, labels, timestamps, query metadata, hashes, counts, verdict reasons, and compact trace sum- maries over unnecessary raw snippets. For later use, the recommended procedure is to treat the released verdict as tied to its original PUN t , then run a new controlled-search valida- tion before using the names in a fresh experiment. The implementation records a validation-stack hash from the local-filter, web-screen, and controlled- search configurations, so regenerated releases can be compared against the paper run. The linked repository uses: python -m pun_pipeline.run paper_release \ âdump /path/to/latest-all.json.bz2 \ âtarget-unknowns 300 \ âcomparison-per-set 100 \ âexact-perturbation-counts Because publishing the accepted strings can it- self make them indexed, later validation should record the release timestamp and ignore self- indexing traces only when they clearly originate from the dataset release rather than from indepen- dent person evidence. A.6 Reproducibility Details The reproducibility study was last run on 21 May 2026 using a source database snapshot with 49,880 latest web-screened candidates and 300 released operationally unknown names. For consistency with the release-yield analyses, theno_info-only rows below are restricted to the 1,781 web-LLM no_infonames in the frozen 52,726-candidate release-yield cohort. The web-enabled model was gpt-4.1-miniwithweb_search_preview; the classifier wasgpt-5.2. Both model calls used tem- perature 0.0. The runs reused the released prompts and classifier rubric, sample seed 13, and the same pre-specified sample across replicates. The study started at 2026-05-21T09:55:34Z and finished at 2026-05-21T13:52:28Z. We ran three evaluations on a 20% sample of the latest web-screened cohort (9,976 names), fol- lowed by three and ten full reruns of the source no_infocohort (1,781 names per replicate). Ta- ble 9 gives the detailedno_info-cohort transition counts. Across the tenno_info-only reruns, only 1.4% of all rerun labels and 2.5% of released-name rerun labels switch tofound. Most non-no_info 15 ArtifactRows in paper runKey columns release/release_unknown_names.csv300 name, status, checked_at, run_id, validation_stack_hash, max_query_type, expected_query_count, executed_query_count release/comparison_names.csv4,276accepted unknowns, public-person controls, proximity controls, stable-no_info duplicate strata, comparison metadata release/known_person_controls.csv300Wikidata QID, Wikipedia metadata, fame bin, exact-name prevalence, comparison level release/ambiguous_name_controls.csv100rejected ambiguity controls with source status, decisive query type, and trace metadata release/famous_name_perturbation_controls.csv3,600source public person, generated variant, distance level, edit metadata, exact quoted prevalence release/release_manifest.jsonâpublic manifest with release file paths, row counts, byte sizes, hashes, generation timestamp, and omitted-artifact notes release/reproducibility_study_20260521T095532Z_noinfo3_stable_no_info.csv49accepted unknowns stable as no_info across the three-replicate rerun panel release/reproducibility_study_20260521T095532Z_noinfo10_stable_no_info.csv27accepted unknowns stable as no_info across the ten-replicate rerun panel release/reproducibility_study_20260521T095532Z_summary.jsonâaggregate reproducibility-study summary for the rerun panels Table 8: Release-schema summary. Stable-no_inforows duplicate accepted unknown names for analysis strata and are not additional unique names. Rerun setCohortNamesLabel calls found ambiguous no_info failAlways no_info 3 repsall source no_info1,7815,34376 (1.4%)3,115 (58.3%)1,678 (31.4%)474 (8.9%)326 (18.3%) 3 repsreleased unknowns30090025 (2.8%)546 (60.7%)260 (28.9%)69 (7.7%)49 (16.3%) 10 repsall source no_info1,78117,810244 (1.4%)10,272 (57.7%)5,635 (31.6%)1,659 (9.3%)189 (10.6%) 10 repsreleased unknowns3003,00074 (2.5%)1,823 (60.8%)857 (28.6%)246 (8.2%)27 (9.0%) Table 9: Detailedno_info-only reproducibility outcomes. Counts are over rerun labels for names whose source web-screen label wasno_info. Theambiguouscolumn combines rerunsimilarandattractorlabels. âAlways no_infoâ is a name-level criterion: the name must receive no_info in every replicate. rerun labels are ambiguity signals (57.7% of all rerun labels; 60.8% for released names). A.7 Comparison-Name Construction Details Table 10 summarises the comparison-name re- lease produced by the paper run.The com- binedrelease/comparison_names.csvfile con- tains the accepted unknown names, known- public-person controls, famous-name perturbations, and stable-no_infosubset rows. The ambigu- ous/similar controls are written to a separate file because they are rejected validation cases rather than accepted unknowns or known-public-person controls. Known-public-person controls are built from a Latin-script subset of enriched Wikidata humans. The paper run used a 2,000-record candidate pool with complete required Wikipedia metadata af- ter merging/fetchingwikipedia_pageviewsand wikipedia_article_length. Exact-name preva- lence is measured with the quoted query â"Full Name" â and reused from stored checks when the same backend/query record already exists. Records are rank-binned into five fame bins and five exact- prevalence bins. In diagonal mode, a record with fame-bin positionfand prevalence-bin positionp is assigned to one of five ordered comparison lev- els by averaging the normalized bin positions and clipping to the range1â5. The final known-person control file contains60rows per comparison level. LetV,L,G, andKdenote page views, article length, Wikipedia language count, and Wikidata sitelink count. The fame score is S = 0.45 log(1 + V ) + 0.25 log(1 + L) + 0.20 log(1 + G) + 0.10 log(1 + K). It is only a sampling proxy. Exact-name prevalence is stored separately so analyses can distinguish public-person prominence from the web footprint of the literal name string. Famous-name perturbations are generated from public-person source names with deter- ministic seeded edits.Levelsd1âd2con- tain close orthographic, abbreviation, or double- edit variants; levelsd3âd7use increasing suf- fix and recombination changes; levelsd8âd12 are fragmentary or near-jibberish recombina- tions. Each row storessource_name,source_qid, proximity_distance_level, a textual distance label, raw and normalized edit distance, genera- tion strategy, donor-part provenance, and the exact quoted prevalence count for the generated variant string. The release was generated with exact pertur- bation counts enabled, yielding300rows at every distance level. A.8 LSTM Name-Likeness Details The LSTM audit is a string-form diagnostic, not ev- idence of operational unknownness. It asks whether accepted PUN names occupy a plausible full-name region under a character-level model trained on ParaNames. The score is bits per character (BPC), so lower values indicate higher predictability under the name model. We compare accepted unknown names with public-person controls, famous-name perturbation controls, and fixed non-name controls. The main plots in Figures 7 and 8 show that ac- cepted PUN names remain closer to public-person names than to non-name controls, while preserving the low exact-name search footprint required by the protocol. 16 File / stratumRoleRowsConstruction note release/comparison_names.csv: unknownaccepted operationally unknown names300fixed-size release from controlled-search verdicts release/comparison_names.csv: comparison_1âcomparison_5public-person controls300 60 rows per diagonal fame/prevalence level release/comparison_names.csv: famous_name_proximity_d1âd12famous-name perturbations3,600 300 unique variants per proximity distance level release/comparison_names.csv: stable_no_info_3stricter unknown subset49duplicate rows for names always no_info in the three-replicate rerun panel release/comparison_names.csv: stable_no_info_10stricter unknown subset27duplicate rows for names always no_info in the ten-replicate rerun panel release/ambiguous_name_controls.csvambiguity controls100sampled candidates rejected through similar-name or attractor evidence Table 10: Paper-facing comparison and control files. The twostable_no_infostrata duplicate accepted unknown- name rows for analysis; they are not additional unique names. The two-dimensional interpretation matters. Public-person controls are highly name-like but in- tentionally have indexed evidence; random strings are evidence-sparse but not name-like. Accepted PUN names are useful because they combine low BPC with zero or near-zero exact-name prevalence under the documented validation run. BPC there- fore supports the form-plausibility claim, while web-facing validation remains the evidential basis for operational unknownness. A.9 Tokenizer Footprint Details Tokenizer fragmentation is an auxiliary audit of familiar-looking subword structure. For each string, we compute token pieces per character under 14 tokenizers and use the median score across tokeniz- ers. Higher scores mean that the string is split into more pieces per character. This audit targets a dif- ferent signal from the LSTM: tokenizers encode model- and corpus-specific subword inventories, while BPC measures character-level predictability under a name-trained model. Figures 12 and 13 show that tokenizer fragmen- tation broadly agrees with the LSTM diagnostic. Accepted PUN names have low median fragmenta- tion, similar to public-person names and word-like non-names, and much lower than random strings. The correlation with BPC indicates shared sensitiv- ity to string regularity, while the weak correlation with exact-name search prevalence confirms that tokenizer footprint is not a proxy for indexed full- name evidence. A.10 Ablation Details The ablations were last run on 21 May 2026 from the state snapshot used for the paper re- lease. Validation-stack ablations replay stored web-enabled LLM and controlled web search audit traces and therefore make no new web-facing val- idation calls. Table 11 reports stage attrition over the 52,726 generated candidates. Table 12 replays counterfactual controlled web search query policies for the 1,781 local-pass names that received a web- LLMno_infolabel. Table 13 reports 500-name StageCategoryCountPct. generated_candidatesall52726100.0 local_filterpass4987794.6 local_filterrejected_local28495.4 azure_web_screensimilar2796853.04 azure_web_screenfail955018.11 azure_web_screenattractor812915.42 azure_web_screenfound24494.64 azure_web_screenno_info17813.38 serper_verificationambiguous13722.6 serper_verification no_indexed_full_name evidence_found3000.57 serper_verificationweb_evidence_found1080.2 serper_verificationvalidation_incomplete10.0 Table 11: Stage attrition in the validation stack. Percent- ages are relative to all generated candidates. diagnostic regeneration runs for generation-policy variants; these are not alternate releases, but checks on which generation constraints affect diversity and efficiency before validation. A.11 Human Agreement Study Details The human-audit study is run as a Prolific external study with a custom web application and database- backed packet assignment. Prolific passes partici- pant, study, and session IDs through URL parame- ters. The application assigns the next packet with the fewest completed sessions and restores prior responses for returning participants. Stale started sessions are marked separately from completed ses- sions and are not counted toward packet comple- tion. Participants must consent, use a desktop/laptop browser, and pass a comprehension item about ex- act versus variant evidence. The Prolific screeners require English and access to at least one of the tar- get chatbot providers. The study intentionally uses broad geographic distribution rather than country quotas; country-level demographics are obtained from Prolific metadata for reporting rather than for balancing. Each packet contains six plausibility items, three fixed attention checks, six search tasks, one live chatbot task, and one saved-response classification task. The attention checks are drawn from three fixed control categories: a word-like non-name, structured gibberish, and a random character string. 17 ExperimentEvaluable NamesAcceptedAccepted PctWeb Evidence FoundAmbiguousValidation IncompleteNewly Accepted Vs FullLost Full Stack Accepts full_stack_recorded_verdict178130016.841081372100 exact_quoted_only1781111562.619257318150 no_unquoted_queries178135619.9910713171560 no_fuzzy_variants178194252.89936121346420 no_page_content178129916.795713962901 Table 12: Controlled-search query ablations on the web-LLMno_infosubset.newly_accepted_vs_fullcounts names that would be accepted by the ablated query policy but are rejected by the full protocol. ConditionGeneratedLocal PassLocal Pass PctSame CountryDistinct Country PairsAttempts paper_baseline50046893.6024322732 allow_same_country50047294.43124120273 pool_size_country_sampling50047194.2010710052 no_rare_token_cap50048396.6049710590 no_native_language_requirement50047394.60331500 allow_source_name_match50048296.4023422398 Table 13: Generation-policy ablations over 500 generated candidates per condition, followed by the same local validation profile. We exclude submissions with no consent, failed comprehension, duplicate participant IDs, comple- tion under one third of the pilot median time, two or more failed attention checks, more than two unex- plained âcould not checkâ responses, or systemati- cally unrelated pasted evidence. These exclusions are pre-specified and logged before final analysis. Figures 14 and 15 show the controlled-search and live-chatbot screens used in the human study. The full export contains 224 completed submis- sions. After review exclusions, 205 completed sub- missions remain usable; for packet-level analysis we select the earliest non-flagged completed sub- mission for each packet, yielding 204 selected par- ticipants and complete coverage of all 204 packets. Median completion time for the selected submis- sions is 8.2 minutes. The selected analysis set con- tains 1,224 plausibility judgments, 612 attention- check judgments, 1,224 controlled-search judg- ments, 204 live-chatbot judgments, and 204 saved- output classification judgments. We inspect divergent evidence strings before interpreting raw disagreement as protocol error. Common failure modes include blank pasted evi- dence, text explicitly saying âno relevant evidenceâ, search-engine result-page URLs, and snippets that contain only component-name or near-match evi- dence rather than the exact full-name string. The clearest informative divergences are ambiguity cases repeated across packets, e.g., Ămile-Joseph Toshir Ì o triggering Ămile Joseph results, Karl-Peter Clarendon triggering Carl Peters, Kaitlyn VaËckĂĄËr triggering Katie Vackar, and Heinz-JĂŒrgen Freder- ick triggering Heinz-JĂŒrgen Friedrich. Reporting therefore separates raw participant labels, attention- filtered labels, and a conservative audited exact- evidence subset. A.12 Human-Subjects and Data Handling The Prolific study was designed as a low-risk evidence-validation task. Participants did not make decisions about real people; they judged whether a displayed string looked like a full name and whether public search results or chatbot responses contained exact-person evidence, ambiguity sig- nals, or no relevant evidence. The entry screen required consent, desktop/laptop access, and a com- prehension check about exact-name versus variant evidence before assignment to a packet. Partici- pants could report that a search could not be com- pleted and could withdraw from participation at any time. Participants were compensated through Prolific under the posted study terms. Payment review and analysis inclusion were kept separate: completed good-faith submissions were approved for payment even when excluded from analysis for pre-specified data-quality reasons. The exclusion full rules used for analysis were: no consent, failed comprehen- sion, duplicate participant ID, completion under one third of the pilot median time, two or more failed attention checks, more than two unexplained âcould not checkâ responses, or systematically un- related pasted evidence. 18 Norway 82k81k 49k 48k26k13k 12k11k11k 10k8k8k6k 4k4k2k2k1k1k1k 7680 Australia 109k 108k65k65k35k17k16k 14k14k 13k 11k 10k8k5k5k 2k2k1k1k1k 0544 Argentina 116k115k 69k69k 37k18k17k15k15k 14k12k11k 9k 6k5k3k 2k2k2k1k1k 578 Mexico 129k129k 77k77k 41k 20k 19k 17k17k15k13k 12k 10k6k6k3k 2k2k2k2k1k 646 Canada 218k217k130k 129k69k 33k32k 29k29k 26k 22k21k 17k10k10k5k 4k 3k03k 2k1k Sweden 232k230k138k137k74k35k34k31k31k27k 24k22k 18k 11k 10k0 4k 3k3k3k 2k1k Switzerland 245k 244k 146k145k78k38k36k32k32k 29k 25k23k 19k 12k11k 5k5k03k3k 2k1k Austria 259k 257k154k154k82k40k38k34k34k30k26k25k20k 12k12k 6k5k 4k 3k0 2k1k Indonesia 286k284k170k170k 91k 44k 43k38k38k34k 29k 27k 22k14k 13k6k5k 4k4k4k 3k 1k Czechia 300k 298k179k 178k 96k 46k45k40k40k35k31k 29k 0 14k14k 7k6k 4k4k4k 3k 1k Hungary 327k325k 195k194k 104k50k 49k 43k43k38k33k31k25k015k7k6k 4k4k4k 3k 2k Ukraine 382k 379k 227k226k 122k 58k57k50k50k45k 39k 36k 29k 18k08k7k5k5k5k 4k2k Poland 620k616k 369k 368k 198k95k92k 082k73k63k 59k 47k30k28k 14k12k 8k8k8k6k3k Belgium 804k 799k479k 477k256k123k 119k 106k106k082k77k61k 39k 36k18k15k 11k11k 10k8k 4k Turkey 1.4M1.4M812k808k434k 209k 202k180k180k160k 139k 130k103k65k62k30k26k 19k 18k17k13k7k Russia 1.9M1.9M 1.1M1.1M0286k277k246k246k 219k191k 178k 142k 90k 84k 41k 36k25k25k 24k 18k 9k Neth. 1.9M1.9M 1.2M1.1M617k0287k255k255k227k 198k 185k147k 93k 87k43k37k26k26k 24k 18k10k U.K. 2.7M2.7M1.6M1.6M853k410k0353k353k315k274k255k203k 129k 121k 59k 51k37k36k34k25k13k India 2.8M2.8M1.7M1.7M 905k 435k 422k 374k374k334k 290k 271k216k136k128k63k0 39k 38k36k27k 14k Italy 3.4M3.4M02.0M1.1M522k506k 449k449k 400k348k325k258k164k154k76k65k46k46k43k32k17k Spain 3.7M3.7M2.2M2.2M1.2M574k557k 494k494k 441k 383k0284k180k 169k 83k72k51k51k47k35k 19k Germany 4.2M4.2M2.5M01.3M648k628k558k558k 497k 432k404k321k203k 191k94k 81k58k57k53k40k 21k Japan 5.8M5.7M3.4M3.4M1.8M882k856k760k0678k 589k 550k437k277k261k128k110k 79k 78k73k54k 29k Brazil 6.2M6.2M3.7M3.7M2.0M 949k921k 817k817k 729k 0 592k 470k 298k 280k137k118k85k84k78k58k31k France 0 9.2M 5.5M5.5M3.0M1.4M1.4M1.2M1.2M1.1M 949k 887k705k446k420k206k177k127k125k117k87k46k U.S. 14.3M08.5M8.5M4.6M2.2M2.1M 1.9M1.9M 1.7M1.5M1.4M1.1M686k646k317k273k 195k193k 181k134k71k France U.S. ItalyGermany RussiaNeth. U.K. Poland Japan Belgium Brazil Spain Czechia Hungary UkraineSwedenIndiaSwitzerlandCanadaAustriaAustralia Norway surname-source country given-source country Figure 9: Every eligible filtered given-source/surname-source country pair. 19 Norway 172316 9 101010635 9 8 9 8 421 3 141 Canada 4852453035 2122 251618 11 131613 12 5 7 6 42 Mexico 5346 42 43 41 3125 22 3120 11 1315 9 7 1211 5 241 Australia 675748464538 19 32 24 32 19 1813 1211 7 9 663 Indonesia 7969 7253484048313220 2114 25 14 19 4 10 4 9 41 Switzerland 856866 49 554038 39 2124 26202613 14 5 99 85 Czechia 725571664740 41 4038313023 19 17255 7 610 41 Austria 788671555560313132 22 27251718 9 11 9 536 1 Argentina 8573725356434835332620 2421 181516 9 11 665 4 Sweden 9193 716363 69 4536 39 31 22 20 22 27 21 178 7 8 4 3 Ukraine 106102 9169 77 61464336433223 22 29 16 12 13683 1 Poland 175127117 999993 746453 49 563840 42 23131720 11 6 2 Spain 204 192 152147 119 10010088737273 49 42 362828161717 11 6 Hungary 241 196 1751681601381181027185 59 42 52 49 2035 2121 166 4 Belgium 2752042201471511251201017474725251622830 212111 9 12 U.K. 285246238164 189 17713812710410681 77 556638 22 29 2326186 Russia 305237262217180165130131110 91 77 6873753328311325136 Italy 402282 293 258220186154143 129 118118758372 44 38272623 144 India 367332308 209 243 195 176147126134 129 78606450463826 19 188 Neth. 349 327278255271183 191 121 146117 112 81 7989 5248313233 199 Germany 400344314284264 195 186 222 126128 9496 686647 42 5028261810 Japan 578473437383 389 258232215164164 149 120103 96 616743 41 38 22 10 France 623503410 414 329 284257 221 22820815014813110685856540 41 2516 U.S. 594 610526483370354273282 249 251163166133162 91 736352533120 Turkey 748576576426463347313264 239 228163172145131115 92935959 462517 Brazil 826 649 553460 398379 361276261 224 169 181180166102 77 63515123 12 U.S. France BrazilRussia Italy Japan Neth. Hungary BelgiumGermany U.K. India Spain PolandUkraineCzechiaSwedenAustraliaSwitzerlandCanadaAustria Norway surname-source country given-source country Figure 10: Full source-country pair distribution for the 52,726 evaluation-scope generated candidates. 20 Switzerland 1 Norway 1 Mexico 11 Sweden 11 Austria 11 Australia 11 Spain 21 Argentina 121 Indonesia 1111 Canada 111111 Neth. 111111 Russia 11111111 Ukraine 22112 Czechia 3 111111 Belgium 3 11111111 Hungary 111121211 Italy 21 33 111 U.K. 3 1 3 1 3 21 Poland 3 1 3 1111121 India 412121112 Brazil 3 121112 3 12 Germany 4221411121 Japan 533 2 3 22212111 Turkey 65 2 3 11111111121 U.S. 5 124 3 422114112 France 5336 2 3 22 3 11 33 11 U.S. Italy RussiaBrazil France HungaryJapan PolandNeth. Germany India Belgium U.K. Spain CzechiaUkraineSweden Norway SwitzerlandAustriaCanadaAustralia surname-source country given-source country Figure 11: Full source-country pair distribution for the 300 accepted operationally unknown names. Unlike Figure 3, this appendix heatmap includes every observed country row and column without an Other aggregation. 21 0.20.40.60.8 Median tokenizer fragments per character Public figure PUN d = 2 d = 4 d = 6 d = 8 Word-like non-name Structured gibberish Random string Public Op. unknown Perturbation Non-name Mean Figure 12: Median subword-tokenizer fragmentation across 14 tokenizers. Higher values indicate that tok- enizers split the string into more pieces per character. 0.20.40.60.8 Median tokenizer fragments per character 2 4 6 8 10 12 BPC (lower = more name-like) Public figure PUN Perturbation Non-name Centroid Figure 13: Tokenizer fragmentation compared with LSTM BPC. Figure 14: Controlled-search task screen with a pre- pared query link, exact-evidence labels, evidence field, and confidence item. Figure 15: Live-chatbot task screen with a prepared prompt, exact-evidence labels, citation field, and confi- dence item. 22