Paper deep dive

Agentic Framework for Political Biography Extraction

Yifei Zhu, Songpo Yang, Jiangnan Zhu, Junyan Jiang

Year: 2026Venue: arXiv preprintArea: cs.CLType: PreprintEmbeddings: 155

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%

Last extracted: 7/20/2026, 3:13:14 PM

Summary

The paper introduces a two-stage 'Synthesis-Coding' framework using recursive agentic Large Language Models (LLMs) to automate the extraction of structured political elite biographies from unstructured web sources. The upstream synthesis stage curates and filters information to create signal-dense contexts, while the downstream coding stage maps this data into structured dataframes. The authors demonstrate that this approach outperforms human collective intelligence (Wikipedia) in information synthesis and matches or exceeds human expert accuracy in extraction, while mitigating biases associated with long, multi-language corpora.

Entities (7)

Large Language Models · technology · 99%Synthesis-Coding Framework · methodology · 98%Elite Biographies · data-type · 95%Political Science · field · 95%Recursive Agentic LLMs · technology · 95%Wikipedia · data-source · 92%Paths to Power · dataset · 90%

Relation Signals (5)

Recursive Agentic LLMs → automates → Elite Biographies

confidence 95% · leverage Large Language Models (LLMs) to automate the extraction of multi-dimensional elite biographies

Synthesis-Coding Framework → uses → Recursive Agentic LLMs

confidence 95% · an upstream synthesis stage that uses recursive agentic LLMs to search, filter, and curate biography

Synthesis-Coding Framework → matchesoroutperforms → Human Experts

confidence 92% · LLM coders match or outperform human experts in extraction accuracy

Synthesis-Coding Framework → appliedto → Political Science

confidence 90% · providing a generalizable template for extracting structured narratives from unstructured text... in political science

Synthesis-Coding Framework → outperforms → Wikipedia

confidence 90% · the agentic system synthesizes more information from web resources than human collective intelligence (Wikipedia)

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:The production of large-scale political datasets typically demands extracting structured facts from vast piles of unstructured documents or web sources, a task that traditionally relies on expensive human experts and remains prohibitively difficult to automate at scale. In this paper, we leverage Large Language Models (LLMs) to automate the extraction of multi-dimensional elite biographies, addressing a long-standing bottleneck in political science research. We propose a two-stage ``Synthesis-Coding'' framework for complex extraction task: an upstream synthesis stage that uses recursive agentic LLMs to search, filter, and curate biography from heterogeneous web sources, followed by a downstream coding stage that maps curated biography into structured dataframes. We validate this framework through three primary results. First, we demonstrate that, when given curated contexts, LLM coders match or outperform human experts in extraction accuracy. Second, we show that in web environments, the agentic system synthesizes more information from web resources than human collective intelligence (Wikipedia). Finally, we diagnosed that directly coding from long and multi-language corpora introduces bias that the synthesis stage can alleviate by curating evidence into signal-dense representations. By comprehensive evaluation, We provide a generalizable, scalable framework for building transparent and expansible large scale database in political science.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

153,429 characters extracted from source content.

Expand or collapse full text

Agentic Framework for Political Biography Extraction Yifei Zhu * Songpo Yang † Jiangnan Zhu ‡ Junyan Jiang § Abstract The production of large-scale political datasets typically demands extracting structured facts from vast piles of unstructured documents or web sources, a task that traditionally re- lies on expensive human experts and remains prohibitively difficult to automate at scale. In this paper, we leverage Large Language Models (LLMs) to automate the extraction of multi- dimensional elite biographies, addressing a long-standing bottleneck in political science re- search. We propose a two-stage “Synthesis-Coding” framework for complex extraction task: an upstream synthesis stage that uses recursive agentic LLMs to search, filter, and curate bi- ography from heterogeneous web sources, followed by a downstream coding stage that maps curated biography into structured dataframes. We validate this framework through three pri- mary results. First, we demonstrate that, when given curated contexts, LLM coders match or outperform human experts in extraction accuracy. Second, we show that in web environments, the agentic system synthesizes more information from web resources than human collective intelligence (Wikipedia). Finally, we diagnosed that directly coding from long and multi- language corpora introduces bias that the synthesis stage can alleviate by curating evidence into signal-dense representations. By comprehensive evaluation, We provide a generalizable, scalable framework for building transparent and expansible large scale database in political science. * Ph.D candidate, Department of Politics and Public Administration, The University of Hong Kong. Email: frankyifei@connect.hku.hk. † Boya Postdoctoral Fellow, School of International Studies, Peking University. Email: yangsp21@mails. tsinghua.edu.cn. ‡ Associate Professor, Department of Politics and Public Administration, The University of Hong Kong. Email: zhujn@hku.hk. § Assistant Professor, Department of Political Science, Columbia University. Email: j3160@columbia.edu. arXiv:2603.18010v1 [cs.CL] 23 Feb 2026 1 Introduction The empirical revolution in political science has been fueled by the mass analysis of digi- tized political records (Grimmer and Stewart 2013; Gentzkow et al. 2019). Disclosed government documents, digitized news reports, and crawled web pages enable large-scale research using po- litical facts 1 , facilitating theory building on representation, state capacity, and regime durability (Binderkrantz et al. 2024; Fisman et al. 2020; Jiang 2018; Jiang and Zhang 2020; Nyrup et al. 2025). Yet transforming unstructured document stacks into structured, analyzable datasets re- mains prohibitively labor-intensive. The core bottleneck is fact extraction: researchers must gather evidence from sources, extract specific information, and structure verifiable information into data suitable for downstream analysis, a process that currently requires extensive trained manual labor. Scaling political data production beyond its current bottleneck demands automated solutions that can replicate, and potentially exceed, the validity of human coding while dramatically reducing labor costs. This paper develops and evaluates automated LLM-based solutions for the extraction of po- litical facts from unstructured documents at scale, focusing on one consequential class of polit- ical facts: structured elite biography. Elite behaviour, shaped by backgrounds, incentives, and networks, systematically influences policy making, public opinion, and regime stability (Putnam 1976; Svolik 2012; King et al. 2013; Alexiadou 2015; Reuter and Szakonyi 2019; Jiang 2018; Woldense and Kroeger 2024). Structured elite biographies are analytically rich for downstream analysis, but computationally prohibitive under traditional manual methods. Manually construct- ing a political biography requires gathering related sources, distinguishing the right sources, ex- tracting biographical facts, including entities, events, and relations from unstructured text into temporally organized lists. 2 In a landmark study, Nyrup et al. (2025) mobilized over 30 research 1 By “political facts,” we refer to verifiable, descriptive attributes of political actors and institutions, including office holding, educational backgrounds, career paths, and institutional affiliations, that can be reliably documented and cross-checked across sources. 2 While we focus on elite biographies, the challenges identified here, such as information dispersion, temporal inconsistency, source conflict, and extraction from unstructured text, apply broadly to other “narrative” political data, such as tracking policy evolution, coding event data from news reports, or reconstructing negotiation processes from diplomatic cables. 1 assistants across three years to manually assemble the “Paths to Power” (PtP) dataset on cabinet members worldwide from diverse web resources. 3 While these achievements, together with many other similar efforts (Armstrong et al. 2024; Bäck et al. 2021; Jiang 2018; Lee and McClean 2022; Raleigh and Wigmore-Shepherd 2022; Vittori et al. 2023), have proven indispensable for answer- ing theoretical questions across diverse political contexts, the need for manual extraction imposes severe limitations, such as discontinued or out of date datasets, costly coverage expansion and new variable addition. Recent advances in Large Language Models (LLMs) offer a potential path toward scalable political text processing (Benoit et al. 2025; Ornstein et al. 2025; Gilardi et al. 2023; Palmer et al. 2024). However, existing political-science applications focus on classification tasks, where output is one label from a predefined finite set (Benoit et al. 2025; Halterman and Keith 2024; Ziems et al. 2024). The harder question is whether LLM-based systems can perform valid extraction tasks, with multiple fields codebooks and undefined output spaces. Extraction in political science also features long, noisy document collections and even web resources, where naive zero shot or few shot fails to address (Liu et al. 2024). To address automation of extraction tasks without sacrificing validity, we propose and evalu- ate an agentic framework for political facts extraction from web sources. The agentic framework consists of two stages: an upstream synthesis stage that utilizes agentic recursive LLM calls to search and refine evidence from web sources, and a downstream coding stage that maps that re- fined evidence into structured facts. We evaluate this framework against human extraction using a validated ground truth dataset of political elites biographies from China, the United States, and a comparative sample of OECD countries. We first validate the coding ability of LLMs when the in- put is human curated Wikipedia biography, showing that LLM coders can match and exceed human coding quality using curated short corpora. We then prove that the agentic workflow outperforms human collective synthesis (Wikipedia) in producing curated biography corpora for global political 3 The sheer scale of such efforts is often understated. Coding a comprehensive cabinet dataset typically requires tens of thousands of RA hours. For instance, the WhoGov dataset (Nyrup and Bramwell 2020) took nearly a decade of intermittent work to finalize. 2 elites. Finally, utilizing two coding corpora containing same information but different composition, we test the generalizability of the two stage (synthesis and coding) framework for extraction task, showing that long and multi-language corpora can introduce coding quality degradation, which proper synthesis can mitigate. Our contribution is fourfold. First, we formalize political fact extraction as distinct from clas- sification. Whereas classification typically assigns texts to a fixed set of labels, extraction requires identifying and structuring facts from open-domain sources and assembling them into coherent career histories. Second, we propose a Synthesis-then-Coding framework that treats information synthesis as a prerequisite for valid extraction and clarifies why skipping synthesis (e.g., naive context-window stuffing) induces a quantity–quality trade-off that degrades performance. Third, we develop and open-source a scalable agentic package that operationalizes synthesis through it- erative, tool-using retrieval and refinement, and we show that it can outperform human collective synthesis (Wikipedia) in producing high-signal corpora for global political elites. Fourth, we ap- ply this framework to generate a large cross-national dataset of political elite biographies, lowering the barrier to producing and maintaining high-quality data in information-poor environments and providing a generalizable template for extracting structured narratives from unstructured text. 2 The Challenge of Extracting Political Biographies at Scale To understand why automatic solutions are necessary, it is useful to begin with the type of political facts that pose the greatest challenges for large-scale data production. Among the many categories of political facts, elite biographical information (e.g., who political elites are, where they come from, and how they advance through institutions), constitutes a particularly demanding case. Such data form the backbone of comparative political research. Granular information on educational backgrounds, career trajectories, and kinship networks has been central to theories of political representation (Carnes 2024; Lee and McClean 2022), authoritarian power-sharing (Svolik 2012; Raleigh and Wigmore-Shepherd 2022), and technocratic governance (Lin 2020; 3 Vittori et al. 2023). In the study of Chinese politics, for example, detailed career histories have enabled scholars to uncover the logic of factional patronage (Jiang 2018; Shih et al. 2012) and to assess the regime’s claims of meritocratic selection (Liu 2024). 4 Despite their importance, elite biographical data remain exceptionally difficult to produce at scale. While digitization has expanded access to political texts, 5 extraction continues to rely overwhelmingly on manual coding by experts or research assistants. Crowdsourcing platforms offer an alternative labor model, but they have generally proven unsuitable for complex elite data extraction, which requires substantial domain knowledge to resolve ambiguities in names, titles, and political affiliations (Benoit et al. 2016). These constraints translate into extraordinary labor requirements in practice. The “Paths to Power” dataset (Nyrup et al. 2025), covering cabinet ministers in 141 countries over 55 years, required more than five years of coordinated work by over 30 research assistants. The LEAD dataset (Ellis et al. 2015) assigned multiple coders to each leader and still took three years to complete. Other prominent efforts, including Funke et al. (2023) and Braun and Raddatz (2010), likewise required years of intensive manual verification. As summarized in Table 1, high-quality elite datasets typically mobilize large teams over extended periods and depend on sustained institutional funding from agencies such as the European Research Council or U.S. National Science Foundation (Alexiadou 2022; Ellis et al. 2015). Even with these investments, manual production has systematic limitations. First of all, inter- coder reliability remains imperfect. For example, Nyrup et al. (2025) reports intercoder reliability of around 0.80 for cabinet-level biographical attributes, with agreement falling below 0.70 for certain variables. 6 More importantly, most datasets remain static snapshots. Among the datasets 4 While our discussion focuses on national-level elites (e.g., cabinet ministers) due to data availability, the theo- retical importance of biographical data extends to local officials, bureaucrats, and party cadres (Landry 2008). The scalability constraints we identify are arguably even more severe for these lower-tier populations, where N is larger and data are noisier. 5 We distinguish between digitization (converting physical records into digital text) and extraction (converting un- structured text into structured databases). The latter remains the primary bottleneck for narrative political data. 6 Similar levels of coder disagreement are reported in other elite datasets, including LEAD (Ellis et al. 2015), Archigos (Goemans et al. 2009), and WhoGov (Nyrup and Bramwell 2020), where resolving inconsistencies often requires multiple coding rounds or adjudication by senior researchers. More broadly, methodological surveys of text and manual data construction highlight that human hand-coding can introduce measurable errors and biases, motivating audits, cross-validation, and supervised approaches (Grimmer and Stewart 2013; Gentzkow et al. 2019). 4 Table 1: Available Datasets on Political Elites Since 2007 ∗ DatasetNYears † RegionVariables ‡ Production Details Countries(D/C/I/P) National Leaders Baturo (2016)–1960–2010Global✓/✓/–/✓2 RAs, 2009–14 Baturo and Elkink (2022)–1950–2017Global–/✓/–/–1 RA Baturo and Tolstrup (2023)1321918–2019Global–/–/–/✓Built on 11 datasets Bomprezzi et al. (2025)1771989–2018Global✓/–/–/✓27 RAs, 1.6M entities De Luca et al. (2018)1401992–2013Global✓/–/–/–Augmented Archigos Dreher et al. (2009)721970–2002Global✓/✓/–/– Ellis et al. (2015)1881875–2004Global✓/✓/–/–2 RAs/leader, 3 yrs Eschenauer-Engler and Herre (2023)–1950–2020Global–/–/–/✓– Fearon et al. (2007)1611945–1999Global✓/–/–/– Funke et al. (2023)601900–2020Global–/–/✓/–9 RAs, 20k+ pages Gerring et al. (2019)1622010–2013Global✓/✓/–/– Goemans et al. (2009)1881875–2015Global–/–/–/✓– Herre (2023)1821945–2020Global–/–/✓/–15 RAs Licht (2022)–1960–2015Global–/–/–/✓5 grad students Mattes et al. (2016)1691919–2018Global–/–/–/✓9 RAs+Experts Yu and Jong-A-Pin (2020)1771946–2011Global✓/✓/–/– Sub-National& Ministerial-Level Elites Alexiadou (2015)181945–2013OECD–/✓/–/– Alexiadou (2022)181945–2015OECD✓/✓/–/–Multiple coders, 6 yrs Alexiadou et al. (2022)131980–2014W. Europe–/✓/–/–Multiple experts Armstrong et al. (2024)1911972–2017Global✓/✓/–/–6 RAs Bäck et al. (2021)131789–2021Great Powers✓/✓/–/– Braun and Raddatz (2010)1541996–2005Global–/–/–/–72,769 names checked Carozzi and Repetto (2016)11994–2006Italy✓/–/–/– Fuchs and Richert (2018)231967–2012OECD✓/✓/✓/–10 RAs Hallerberg and Wehner (2012)271973–2010OECD✓/–/–/–6 RAs Jiang (2018)11997–2015China✓/✓/–/✓20+ RAs Lee and McClean (2022)41983–2017Asia✓/✓/–/– Nyrup and Bramwell (2020)1771966–2023Global–/–/✓/–9+ coders Nyrup et al. (2025)1411966–2021Global✓/✓/–/–30+ RAs, multi-year Raleigh and Wigmore-Shepherd (2022)231996–2017Africa–/–/–/– Ricart-Huguet (2021)161960–2010Africa✓/–/–/– Vittori et al. (2023)312000–2020EU+4✓/✓/–/–Country experts ∗ This list is illustrative rather than exhaustive. We prioritize datasets that (1) focus on individual-level attributes of political elites, (2) have been widely cited in top political science journals, and (3) involve substantial manual coding efforts. Datasets focused solely on voting records (e.g., roll-call data) are excluded as they represent a different class of “atomic” facts. † Years reflect the temporal coverage of the most recent available version. The end year indicates when the dataset was last updated, which may postdate the cited foundational paper (e.g., Archigos 4.1, initially published by Goemans et al. (2009), was subsequently updated to cover leaders through 2015). Most datasets have not been updated for several years, with many remaining frozen a decade or more behind current events. ‡ Variables: D = Demographics (education, ethnicity, birthplace, family background); C = Career (pre-office occupa- tion/political experience); I = Ideology/party affiliation; P = Power dynamics (entry/exit manner, tenure, transitions). Production scale indicates the reported labor intensity of manual data collection; – = Not reported or insufficient detail. 5 surveyed, only a small fraction have been updated in the past five years, while widely used re- sources such as Archigos (Goemans et al. 2009) and LEAD (Ellis et al. 2015) have remained unchanged for a decade or more. Updating comprehensive elite datasets often requires thousands of additional labor hours, making continuous maintenance prohibitively costly. These production constraints shape the substantive scope of political inquiry. Existing datasets disproportionately fo- cus on actors at the apex of political power, while mid-level bureaucrats, local officials, and other actors central to policy implementation remain largely absent from comparative data. Even among top-tier elites, coverage is uneven: while finance ministers (Armstrong et al. 2024) and foreign ministers (Bäck et al. 2021) are relatively well documented, systematic data on portfolios such as education, health, or infrastructure remain scarce. The structure of available information further constrains what can be collected. When com- prehensive Wikipedia biographies exist, researchers can extract structured facts from consolidated text. Such cases, however, are unevenly distributed across countries and levels of government and often omit early careers, family ties, or post-tenure activities. In their absence, researchers must reconstruct careers by manually searching across government websites, news archives, and organi- zational announcements. Information is then fragmented across heterogeneous sources, frequently in local languages and embedded in noise. As a result, empirical research tends to concentrate where information is easiest to obtain rather than where theoretical questions are most consequen- tial, producing an “information structure bias” (Wilson and Knutsen 2022). Taken together, these realities expose two fundamental bottlenecks that any automated solu- tion must address. The first is cost and scalability: manual coding scales linearly with dataset size, limiting expansion beyond narrow elite populations and hindering timely updates. The second is transparency and replicability: manually assembled datasets typically release only final records, with limited documentation of sources, conflicts, or adjudication rules, complicating verification and reuse. These constraints restrict not only the scale of political data production but also its verifiability, a growing concern as comparative political science increasingly relies on large-N observational evidence. 6 The challenges discussed above extend beyond elite studies. Scholars studying contentious politics increasingly rely on real-time event data scraped from news sites and social media (King et al. 2013; Muthiah et al. 2015; Zhang and Pan 2019). Legal and regulatory research requires track- ing policy evolution across fragmented official gazettes, court databases, and agency announce- ments (Baturo et al. 2017; Fang et al. 2025; Liebman et al. 2020). Trade and investment research depends on synthesizing information from corporate filings, diplomatic cables, and industry publi- cations (Hassan et al. 2019; Thrall 2025). In each case, the core challenge is identical: transform- ing vast, unstructured, and often conflicting information into valid, structured datasets at scales that manual coding cannot sustain. Political fact extraction, therefore, represents less a niche technical problem than a fundamental bottleneck constraining the empirical scope of comparative political science. 3 From Classification to Extraction: The Context Challenge If manual coding is the bottleneck, recent advances in generative language models offer a theoretical solution. A rapidly growing body of work demonstrates that Large Language Models (LLMs) can replicate human judgments on classification tasks such as ideology scaling (Wu et al. 2023), stance detection (Benoit et al. 2025; Gilardi et al. 2023), and topic classification (Ornstein et al. 2025) with high reliability. 7 These classification tasks share a common structure: they map bounded, pre-selected texts into finite label sets, holding the input document fixed. The model receives a well-defined text—a speech, a manifesto, a social media post—and must interpret its content according to a predefined codebook. 7 Empirical validations demonstrate that LLMs not only match but often exceed human performance on political text classification. Gilardi et al. (2023) find that ChatGPT’s zero-shot accuracy surpasses crowdworkers by approximately 25 percentage points on tasks involving stance, topics, and frame detection, while also achieving higher intercoder agreement than trained human coders. Benoit et al. (2025) show that LLM ratings of party manifestos correlate with expert judgments at 0.87–0.92, reaching the upper bound of human expert agreement, with intra-LLM consistency typically exceeding 0.90 compared to human intercoder reliability of 0.3–0.5. Likewise, Wu et al. (2023) demonstrate that LLM-generated ideology scores achieve test-retest correlations of 0.997 and better predict human perceptions of politician ideology than traditional behavioral measures. These studies establish that modern LLMs already deliver both superior accuracy and consistency on classification tasks. 7 Political data extraction, however, represents a fundamentally different computational prob- lem. 8 Unlike classification, which assigns labels to fixed texts, extraction entails actively searching for and reconstructing structured facts from vast, dispersed information environments where no single document contains complete information. This shift introduces four compounding chal- lenges that classification benchmarks do not address. 9 First, extraction is open-domain: relevant entities, organizations, and position titles are not exhaustively enumerated ex ante, so the system must recognize and standardize an effectively unbounded set of possible answers rather than se- lecting from a fixed menu. Second, extraction exhibits high task complexity: a single record (one official’s career) requires answering many heterogeneous sub-questions (e.g., identity resolution, appointment dates, organizational affiliations, position titles, status flags), and reconciling contra- dictions across sources, rather than producing a single label. Third, extraction involves context dependency: information retrieval is inherently path-dependent. Discovering one fact (e.g., an offi- cial served as “Assistant Secretary at Commerce”) provides the contextual cue necessary to locate subsequent facts (e.g., searching for “Commerce Assistant Secretary 2015” rather than the initial broad query “John Smith government”). 10 Unlike independent classification labels, career events form temporal sequences that require stateful reasoning to reconstruct. These three challenges are formidable, and current evidence does not establish that LLMs can reliably address them at scale in real-world extraction scenarios. In principle, the same generative capabilities that enable classification could extend to these problems, yet translating this theoretical potential into validated extraction systems remains an open question. Even perfect solutions to the first three challenges would not resolve a fourth constraint that is orthogonal to model capability: the long-context problem. In open-ended environments such as the open web, potentially relevant information for a single individual is dispersed across hundreds or thousands of documents (far 8 We provide a formal mathematical distinction between classification and extraction in Online Appendix A1. 9 In Natural Language Processing (NLP) literature, this class of problems is commonly described as open-domain slot filling or complex information extraction (Angeli et al. 2015). Unlike classification tasks with predefined label sets, these approaches aim to recover canonical attribute values from unstructured text under a specified codebook. 10 This dependency mirrors the challenge of multi-hop question answering (Yang et al. 2018), where answering a query requires aggregating evidence from disjoint text segments. For biographical data, a resignation date in one document may resolve the tenure end date for a position mentioned in another, but only if both documents have been located and linked. 8 exceeding the token budgets of even extended-context models), and is submerged in vast quantities of irrelevant content. This challenge operates on two dimensions, neither of which can be resolved through straightforward technical improvements. First, models may fail to locate related information or generate false information to reconcile contradictory or misleading signals in too long contexts (Liu et al. 2024; Mallen et al. 2023; Shi et al. 2023). 11 Second, and more fundamentally, there exists a capacity constraint that no architec- tural refinement can overcome: even models with 100K+ token windows cannot accommodate the full universe of potentially relevant documents for a given extraction target. A mid-level bureau- crat in a large country may be mentioned across thousands of government websites, news articles, and policy documents spanning decades; web searches routinely return result sets that, if naively concatenated, would exceed 1 million tokens. These findings reveal that the binding constraint in open-ended extraction is not merely reading long text, but deciding which sources to read and how to condense them into high-signal inputs before structured coding. Valid extraction at scale there- fore requires an architectural solution that governs information selection and condensation before LLM-based coding can be meaningfully applied. 4 An Agentic Solution to the Extraction Challenge Building on the preceding analysis, we introduce an agentic architecture that directly targets the upstream bottleneck of evidence acquisition. The core idea is to decompose extraction into two analytically distinct stages: synthesis, which locates, evaluates, and consolidates relevant evidence from open-ended sources, and coding, which extracts structured facts from curated inputs. We operationalize this design through a recursive retrieval-and-synthesis loop that mirrors the iterative logic of human research and enables valid extraction from noisy web environments. 11 Liu et al. (2024) demonstrate a U-shaped performance curve in multi-document question answering: accuracy peaks when relevant information appears at the beginning or end of long contexts but degrades sharply (often by over 20 percentage points), when the same information is positioned in the middle. This “lost-in-the-middle” phenomenon persists across model families (GPT-3.5, Claude, open-source alternatives) and is not resolved by simply extending context windows: models with 16K-token capacity perform identically to their 4K counterparts when processing inputs that fit in both, indicating that raw capacity does not translate into robust information use (Liu et al. 2024, 162) 9 The central difficulty of open-web extraction lies in deciding which sources to consult and how to compress dispersed evidence into inputs suitable for downstream coding. Standard Retrieval- Augmented Generation (RAG) systems address this by retrieving document chunks based on se- mantic similarity to a query, then passing concatenated results to the model (Lewis et al. 2020). In a web search scenario, this is equivalent to issuing a single keyword query, retrieving the top-ranked pages, and conducting extraction on the aggregated results. This one-shot approach is fundamen- tally brittle under the context dependency challenge identified earlier. The relevance of a document often cannot be determined ex ante but depends on information uncovered in prior retrieval steps. Key entities, affiliations, and career transitions are frequently discoverable only after intermediate facts have been established, rendering fixed retrieval strategies systematically incomplete. Consider the biographical reconstruction task shown in Figure 1. When Wikipedia contains a comprehensive biography (left panel, green boxes), a single LLM pass suffices to extract structured facts. But when Wikipedia is absent or incomplete (the common case for non-elite officials), the system must search across heterogeneous web sources (right panel, blue boxes). Crucially, informative follow-up queries are endogenous to what has been learned from earlier documents. Discovering that an official served as “UNEP Executive Director” provides the contextual anchor needed to locate subsequent positions (“Climate Council member”), professional affiliations (“Belt & Road Coalition Vice-President”), or organizational roles (“Plastic REV Foundation CEO”) that would be invisible to an initial broad search. Static RAG, by committing to a fixed retrieval strategy before any evidence has been examined, cannot exploit these path-dependent cues. RAG has emphasized the model’s capacity to observe: given curated context, LLMs can reliably produce structured outputs in zero-shot or few-shot settings (Ornstein et al. 2025; Gilardi et al. 2023; Benoit et al. 2025). What has received less attention is that modern LLMs can also act. Specifically, they can generate executable commands that interact with external retrieval systems, enabling autonomous information gathering. In a recursive setting, the interleaving of action and observation allows the model to search iteratively, examine retrieved documents, reason about gaps in current knowledge, and decide what sources to consult next (Yao et al. 2023). Each retrieval 10 Figure 1: Two coding strategies for elite biographies. Left: when a Wikipedia page exists, we code directly from the curated page with a single LLM pass. Right: when Wikipedia is missing or incomplete, we search across web sources and iteratively synthesize a synthetic report, then code from that report. The lower panel illustrates the structured output as an ordered biography (career, education, and affiliations) anchored on a timeline. This contrast highlights why extraction from open-web sources requires adaptive synthesis rather than one-shot retrieval. 11 action is thus conditioned on information accumulated in previous steps, allowing the system to resolve the path-dependent nature of open-domain retrieval while progressively compressing a noisy information universe into a condensed corpus that avoids long-context constraints. We operationalize this capability through an agentic framework that repositions the LLM from passive reader to active research assistant. Rather than treating retrieval as preprocessing, the architecture implements a recursive reasoning–action loop: the model iteratively (i) reasons about current knowledge gaps (e.g., “I have identified the UNEP directorship but lack information on prior ministerial roles”), (i) acts to acquire missing evidence via targeted search queries or docu- ment inspection, and (i) updates a running synthetic report consolidating verified findings. Each step is executed through a minimal set of deterministic retrieval tools, which carry out machine- readable commands (e.g., search(“Erik Solheim OECD DAC”) or o pen u rl(url_5)) and return text for inspection. The agent iteratively incorporates evidence, decides whether further retrieval is needed, and maintains only the task description, recent interaction history, and current report in context to ground search decisions while avoiding context overflow. The final synthetic report, a compressed, wiki-like summary of curated evidence, serves as the sole input to the downstream coding step that produces the structured biography. The exact prompt templates used for the su- pervisor, searcher, and coder agents are listed in Online Appendix A6. The agentic framework combines scalability, transparency, and validity in automated data production. 12 It can process thousands of targets in parallel without task-specific model training or human supervision, substantially reducing time and labor costs. At the same time, every re- trieved source is archived and linked to the generated synthetic report, allowing researchers to inspect intermediate evidence, trace how claims were verified or adjudicated, and identify poten- tial errors or biases (Bail 2024). 13 To evaluate whether these architectural advantages translate into valid and scalable data production, we design a multi-stage empirical evaluation. Experiment 1 tests whether large language models can accurately code structured facts from curated biograph- 12 For a practical guide to applying this framework to new extraction tasks, including step-by-step recommendations on codebook design, synthesis configuration, and evaluation, see Online Appendix A7. 13 For a complete step-by-step trace of an agentic extraction run for a specific official, see the case study of Erik Solheim in Online Appendix A5. 12 ical texts. Experiment 2 examines whether agentic synthesis from open-web sources can recover reliable biographical information in fragmented and noisy information environments. The last Experiment then assesses the architectural mechanisms underlying these results by holding the re- trieved evidence fixed and varying how that evidence is represented to the coder. This diagnostic experiment clarifies why synthesis is essential for reliable extraction at scale. 5 Experiment 1: The Coding Challenge This section addresses the first research question by isolating the coding component of au- tomated biography extraction. The objective is to assess whether, given identical and curated evidence, large language models can extract structured biographical facts with accuracy compa- rable to human coders. By holding the information environment constant and varying only the coder, this design directly tests whether coding itself constitutes a binding constraint in automated political data production. We implement this test in a setting where authoritative human-coded benchmarks exist, enabling direct validation of event-level extraction accuracy. 5.1 Data Human-Coded Benchmark: The CPED The Chinese case provides a uniquely suitable bench- mark for evaluating coding validity because it offers high-quality, human-coded biographical data at scale. Our analysis relies on the Chinese Political Elite Database (CPED), a comprehensive biographical database covering more than 4,000 key city-, provincial-, and national-level lead- ers since the late 1990s (Jiang 2018). Outside of the Central Organization Department’s internal archives of the Chinese Communist Party, CPED is widely recognized as the most authoritative digital repository of Chinese political curricula vitae. 14 Importantly, all biographies in CPED are manually coded by trained research assistants following standardized rules, producing structured career histories that serve as a ground-truth benchmark for validation. 14 CPED provides detailed information on career trajectories, educational backgrounds, native place, birth year, ethnicity, and records of corruption investigations. 13 Sample Construction From the CPED population, we constructed a stratified random sample of 197 officials, balanced across three administrative ranks to capture variation in career com- plexity: (i) bureau-director level (equivalent to city mayors or provincial department heads), (i) vice-ministerial level (provincial governors or vice-ministers), and (i) ministerial level (provin- cial party secretaries or national ministers). This stratification ensures representation across the hierarchy of Chinese bureaucratic advancement, where career paths differ systematically by rank. These officials exhibit complex, longitudinal career histories typical of Chinese bureaucratic ad- vancement, with multiple concurrent and sequential positions across party, government, and state- owned enterprise sectors. Decomposing these complex career histories into discrete positional observations yields over 4,000 structured biographical entries in the CPED benchmark. Each entry records a specific position with standardized fields: organization, location, role/title, start date, end date, and the administrative rank. 5.2 Evaluation Design To isolate coding performance, human and LLM coders are provided with an identical infor- mation environment. For each official, the input consists of the full Baidu Baike profile associated with that individual. 15 Under this setup, we generate two structured biographies per official. The baseline biography corresponds to the existing CPED record produced by trained research assis- tants. The treatment biography is generated by applying a long-context LLM coder to the identical Baidu Baike text in a single pass. Because both biographies draw on the same evidence source and follow the same codebook, any performance differences can be attributed to the coder rather than to variation in information availability or task definition. We evaluate multiple LLM architectures (Grok-4.1-Fast, Gemini-2.5-Flash, and Qwen-2.5) to assess robustness across models. 15 In the Chinese context, official biographical information is highly standardized due to the party-state’s institu- tionalized nomenklatura system. Baidu Baike, the dominant Chinese equivalent of Wikipedia, serves as the primary repository for official profiles. All sampled officials have Baidu Baike entries, which consolidate career narratives drawn from official announcements, government websites, and authoritative media sources. The human-coded ground truth in CPED was originally derived primarily from these same Baidu Baike profiles. 14 CGT Construction While CPED provides a high-quality human-coded benchmark, human an- notation is not error-free. To establish a more reliable reference standard, we construct a Consol- idated Ground Truth (CGT) through a three-step validation pipeline. First, for each official, we pool all claims produced by both Human_wiki and LLM_wiki into a unified candidate set. 16 These claims are then normalized into a standardized codebook (entity, role, organization, start_date, end_date, status), enabling direct comparison across coders. Second, each normalized claim is subjected to evidence-based validation using an LLM-as-judge protocol (Gu et al. 2024; Li et al. 2024), which evaluates supporting evidence from Baidu Baike and supple- mentary authoritative Chinese sources and classifies claims as verified, contradicted, or uncertain. Verified claims enter the CGT; contradicted claims are excluded; uncertain cases are flagged for review. Third, to assess the reliability of automated validation, we conducted a manual audit of 500 randomly sampled claims (50 officials× 10 claims). Two independent Chinese-speaking research assistants reviewed the underlying evidence and judge classifications, achieving 94% agreement. Identified systematic error patterns were corrected by refining validation prompts and re-running affected cases. Full CGT construction procedures, judge prompts, and audit results are documented in Online Appendix A3. Performance Metrics We evaluate coding performance at the official level by comparing system- generated claims against the Consolidated Ground Truth (CGT). Let b C i denote claims produced by a candidate system for official i, andC ∗ i denote CGT claims. We define true positives (TP i ), false positives (FP i ), and false negatives (FN i ) as: TP i = b C i ∩C ∗ i ,FP i = b C i ∗ i ,FN i = C ∗ i \ b C i . 16 We use the subscript “wiki” as a generic shorthand to denote open-access, collaborative encyclopedia sources. For the Chinese sample, this refers specifically to Baidu Baike data; for the U.S. and OECD samples discussed later, it refers to Wikipedia. 15 From these quantities, we compute Precision, Recall, and F1 score: Precision i = TP i TP i + FP i ,Recall i = TP i TP i + FN i ,F1 i = 2· Precision i · Recall i Precision i + Recall i . Precision captures the accuracy of extracted facts (the share of claims supported by verified evidence), while Recall measures coverage (the share of true career events successfully recov- ered). F1 is the harmonic mean balancing both dimensions. High precision but low recall yields incomplete biographies; high recall but low precision contaminates datasets with hallucinations. Estimation Strategy To estimate differences in coding performance between human and LLM coders, we fit additive fixed-effect models of the form: Y i = α + β· 1(Coder i = LLM) + γ· Model i + ε i ,(1) where Y i ∈F1, Precision, Recall denotes the performance metric for official i. The indicator 1(Coder i = LLM) captures whether the biography was produced by an LLM or by human coders, while Model i includes fixed effects for LLM architecture (Grok, Gemini, Qwen). The coefficient β therefore identifies the average difference in coding performance between LLMs and humans, holding constant both the evidence source (Baidu Baike) and the extraction codebook (CPED). Standard errors are clustered at the official level, and 95% confidence intervals are obtained via nonparametric bootstrap (1,000 iterations). 5.3 Results Figure 2 reports estimated differences in extraction performance relative to the human base- line, with coefficients from Equation 1 and 95% confidence intervals. Across all metrics, contem- porary LLM coders match or exceed the human baseline when applied to the same curated Baidu Baike corpus. Grok-4.1-Fast increases F1 by 0.109 significantly, driven by improvements in both precision and recall. Gemini-2.5-Flash exhibits similar, though more moderate gains with balanced 16 improvements in precision and recall. Even smaller open-source models such as Qwen-3 achieve near-human capability. Figure 2: Experiment 1 Results: LLM coding performance relative to the human baseline (China sample, N=197). Points indicate coefficient estimates with 95% confidence intervals. The human- coded baseline (Human_wiki) is normalized to zero. Positive values indicate that LLMs outper- form human coders on the corresponding metric. These results establish that coding accuracy is not a binding constraint in automated political data production. When provided with curated inputs, LLMs can reliably map unstructured text to a complex, multi-field biographical codebook, achieving performance that matches or exceeds trained human coders. The most pronounced advantage lies in recall: leading models recover 10–16 percentage points more true career events than human coders working from the same corpus. This pattern is consistent with known limitations of manual annotation (principle agent problems, attention fatigue, selective reading, and time pressure), which lead human coders to systematically omit valid but less salient information, especially for officials with long and overlapping career histories. Importantly, these recall gains come with only modest changes in precision. Greater difference exists in marginal production costs. For human coding, we assume a skilled coder paid $25 per hour. Given an average coding time of approximately 15 minutes per offi- cial, this yields a per-unit cost of $6.25. For LLM coding, costs are computed based on token- level pricing for long-context inference. A typical Baidu Baike biography contains approximately 5,000–9,000 input and output tokens combined. Under the price of gemini-2.5-flash, the best- performing model, each official costs on average $0.13 to process. 17 6 Experiment 2: The Synthesis Challenge Having established that modern LLMs can validly and efficiently code curated biographical inputs, we now turn to the central challenge addressed by this paper: whether automated systems can match or exceed human collective curation (Wikipedia) in the upstream task of consolidating noisy and fragmented web sources into codeable evidence. We evaluate this challenge using two complementary settings: contemporary U.S. political elites and ministerial officials from OECD countries. Unlike the Chinese case examined in Experiment 1, the U.S. and OECD contexts lack comprehensive human-coded biographical benchmarks covering the full range of elite career at- tributes. 17 Constructing such benchmarks manually would require thousands of research assistant hours and would reproduce precisely the scalability bottleneck that automated synthesis is de- signed to overcome. Rather than attempting to recreate human-coded benchmarks at prohibitive cost, Experiment 2 therefore evaluates synthesis performance in settings where only elite rosters are available ex ante and biographical information must be recovered from the open web. These contexts feature relatively high Wikipedia coverage, but with systematically varying degrees of in- formation completeness and fragmentation beyond what encyclopedic curation captures. Together, they allow us to assess whether automated synthesis can recover biographical facts that are omitted, unevenly documented, or dispersed outside Wikipedia’s curated summaries. 6.1 Data U.S. Political Elites The U.S. sample comprises 198 contemporary political elites drawn from a comprehensive roster compiled by the authors. 18 We employed stratified random sampling focused 17 Surprisingly, despite the size and maturity of the U.S. political science literature, there is, to the best of our knowl- edge, no publicly available dataset that provides CPED-style, career-long biographical coding for the full population of American political elites across offices and career stages. Existing resources typically focus on specific institutions (e.g., Congress or the presidency) or a narrow subset of attributes (Bonica 2016). For OECD countries, several cross- national databases document cabinet composition and tenure, but as shown in Table 1, the scope of coded attributes remains substantially narrower than CPED, with limited coverage of education, pre-political careers, concurrent posi- tions, and post-tenure trajectories. 18 The full roster covers all state governors, members of Congress, Cabinet members, and Supreme Court justices from 1776 to 2025. 18 on the post-2000 period to ensure high data density on the open web, selecting three equal cohorts of 66 officials: Cabinet members, state governors, and members of the 119th Congress. Unlike the centralized personnel records found in authoritarian hierarchies, American political data is structurally decentralized. Information is dispersed across federal databases, state archives, and local media, requiring the synthesis agent to navigate a highly heterogeneous source landscape to reconstruct coherent biographical narratives. OECD Political Elites The OECD sample consists of 200 ministerial officials from 36 member countries serving between 2011 and 2019, selected from the WhoGov 2.0 database (Nyrup and Bramwell 2020) via simple random sampling. 19 This sample tests the system’s ability to han- dle breadth across linguistic and institutional contexts. The primary challenge here is not merely depth, but the unevenness of digital curation: officials from smaller member states or minor port- folios often lack English-language Wikipedia entries, forcing the agent to retrieve and synthesize information from native-language government websites, party manifestos, and local press. 6.2 Evaluation Design Experiment 2 isolates the contribution of upstream synthesis by holding the downstream LLM coder fixed and varying only the method used to construct evidentiary corpora. All biography types are ultimately coded by the same LLM using an identical extraction codebook; differences in performance therefore reflect variation in how evidence is located, consolidated, and curated prior to coding. Synthesis Conditions We compare three synthesis conditions that differ in how biographical evidence is assembled from the open web. Human collective synthesis (Wikipedia baseline). Wikipedia represents the outcome of large- scale human collective curation: volunteer editors identify sources, resolve contradictions through 19 Unlike the China and U.S. samples, which use stratified sampling to capture vertical hierarchies, the OECD sample focuses on horizontally comparable cabinet-level officials to maximize cross-national coverage. 19 citation norms, and compress verified information into narrative biographies. For this baseline condition, we use the existing Wikipedia page for each official 20 as the sole input corpus. This setting reflects a best-case benchmark for human synthesis, benefiting from years of accumulated editorial effort. Agentic synthesis (full web). The full-web agent implements the iterative retrieval–reasoning loop described in Section 4. Starting from a broad query (official name and approximate role), the agent issues successive searches conditioned on information discovered in earlier steps, inspects retrieved documents, and consolidates verified claims into a running synthetic report. Wikipedia is treated as one source among many rather than an authoritative endpoint. Each claim in the report is explicitly linked to archived sources, and the agent terminates once sufficient evidence has been gathered to populate the extraction codebook. On average, the agent conducts 15–25 searches and 12–20 document inspections per official, with synthesis costs of approximately $0.20 per case (search APIs), in addition to downstream coding costs. Full architectural details in Online Appendix A2. Agentic synthesis without Wikipedia. To assess whether agentic gains depend on access to cu- rated encyclopedic content, we implement a non-Wikipedia variant in which all wiki-domain URLs are blocked during retrieval and document inspection. The agent must reconstruct a Wikipedia- equivalent evidentiary base entirely from non-Wikipedia sources, such as government websites, parliamentary records, party materials, and news archives. This condition directly tests whether automated synthesis can mitigate information-structure bias in contexts where encyclopedic cura- tion is sparse or absent. Biography Types Combining these synthesis conditions with a fixed downstream LLM coder yields three biography types. The baseline biography (LLM_wiki) is generated by applying the LLM coder to the Wikipedia page. The two treatment biographies are generated by applying the same coder to synthetic reports produced by the full-web agent (LLM_agent) and the non- 20 For wiki resources we used all sources with “wiki” domain; we also excluded grokipedia resources in all agent experiments. 20 Wikipedia agent (LLM_nowiki), respectively. Because the coder and extraction codebook are identical across conditions, performance differences isolate the contribution of upstream synthesis. Table 2 summarizes the experimental contrasts and associated costs. Table 2: Experiment 2: Synthesis Method Comparison Biography TypeSynthesis MethodDownstream CoderCorpus TypeLengthCost LLM_wiki (baseline)Human (Wikipedia)LLM (fixed)Wiki page ∼8k$0.01 LLM_agent (treatment 1)Agent (full-web)LLM (fixed)Synthetic report ∼10k$0.21 LLM_nowiki (treatment 2)Agent (non-wiki)LLM (fixed)Synthetic report ∼10k$0.21 Notes. All three biography types use the same downstream LLM coder (Grok-4.1-Fast). The experimental contrast isolates the synthesis contribution by holding the coder constant and varying only the upstream evidence construction method. Cost includes both synthesis (search API) and coding (LLM API) expenses per official. Ground Truth Construction As in Experiment 1, evaluation in Experiment 2 relies on a similar design by constructing Consolidated Ground Truth (CGT) through evidence-based validation. For each official, we pool all claims extracted across synthesis conditions into a unified candidate set, normalize them into a common codebook, and validate each claim against archived source evidence using an automated judge. Verified claims constitute the CGT used for evaluation. To assess reliability, we conduct targeted manual audits on a random subset of claims across both the U.S. and OECD samples, including multilingual cases. Agreement between automated judgments and human review exceeds 90%. Detailed consistency metrics for a randomly selected sample of officials across diverse linguistic contexts (OECD) are reported in Online Appendix Table A3.1. Performance Metrics Performance is evaluated using the same individual-level Precision, Re- call, and F1 metrics as in Experiment 1, computed by comparing system-generated claim sets to the CGT. Estimation Strategy We estimate the effect of agentic synthesis using the following specifica- tion: Y i = α + β 1 · 1(Synthesis i = Agent) + β 2 · 1(Synthesis i = NoWiki) + γ· Controls i + ε i ,(2) 21 where Y i ∈ F1, Precision, Recall denotes performance for official i. The baseline category is Wikipedia-based synthesis. Controls include downstream model indicators and sample fixed ef- fects (U.S. vs. OECD). Coefficients β 1 and β 2 capture the average performance difference between agentic and human collective synthesis, holding the coder constant. Standard errors are clustered at the official level, with 95% confidence intervals obtained via bootstrap. 6.3 Results To rigorously quantify the synthesis challenge in these contexts, we first present the compo- sition of URLs retrieved by our agentic framework across all three regions (including the China sample from Experiment 1 for comparison). Table 3 summarizes the distribution of these sources (see also Figure A4.4 in the Online Appendix for a visual breakdown). The data reveals distinct structural divergences across regions. While the average volume of retrieved URLs is consistent across samples (≈ 21–22 URLs per official), the composition of evidentiary sources differs funda- mentally. The China baseline exhibits a high concentration of state-sanctioned information, with 78.4% of evidence derived from journalism (43.4%) and official government sources (35.0%). In contrast, the U.S. sample demonstrates a highly fragmented distribution. Reliance on official gov- ernment sources drops to 26.9%, while civil society sources, including non-wiki databases (10.4%) and NGO groups (8.9%), comprise a substantial portion of the evidence base, compared to neg- ligible levels in China. The OECD sample occupies an intermediate position, balancing journal- ism (25.8%) and government sources (22.1%) with a significant reliance on wiki-based references (17.4%). This contrast confirms that for democratic elites, valid extraction requires synthesizing evidence from a broad, diverse spectrum of non-official sources. Figure 3 reports the aggregate effects of agentic synthesis relative to the Wikipedia baseline, pooling the U.S. and OECD samples (N = 398). 21 Across both contexts, agentic synthesis sub- stantially improves coverage while maintaining acceptable levels of accuracy. The full-web agent 21 For completeness, Appendix A4.2 reports parallel results for the China sample, where the encyclopedic baseline is already highly curated. 22 Table 3: Retrieved URL Category Composition by Sample (Top 6 Categories) RegionAvg URLsGovtWikiJournalismDatabasesNGOSocial China21.27.44 (35.0%)2.07 (9.8%)9.22 (43.4%)0.05 (0.2%)0.41 (1.9%)0.13 (0.6%) US22.46.02 (26.9%)2.58 (11.5%)3.10 (13.8%)3.18 (14.2%)1.99 (8.9%)0.63 (2.8%) OECD22.14.87 (22.1%)3.02 (13.7%)5.70 (25.9%)1.63 (7.4%)1.71 (7.8%)1.15 (5.2%) Notes. Values represent the average number of unique URLs retrieved per official by the agentic framework, with the category’s share of total regional volume in parentheses. Source categories are classified as follows: “Govt” includes official government sources; “Wiki” includes Wikipedia and wiki-derived encyclopedias; “Journalism” covers news media outlets; “Databases” refers to non-wiki structured reference databases (e.g., VoteSmart, Ballotpedia); “NGO” includes advocacy groups and NGOs; “Social” includes personal or professional social media platforms. (LLM_agent) increases F1 by 14.7 percentage points (95% CI: [12.1, 17.3]), driven primarily by large recall gains of 31.4 points (95% CI: [27.8, 35.0]). Precision declines modestly by 5.2 points (95% CI: [−7.8, −2.6]). Even when prohibited from accessing Wikipedia, the non-wiki agent (LLM_nowiki) achieves sizable improvements, increasing F1 by 11.7 points (95% CI: [9.3, 14.1]) and recall by 24.3 points (95% CI: [21.1, 27.5]), with a comparable precision reduction of 4.1 points (95% CI: [−6.5, −1.7]). Crucially, the dominance of recall gains underscores that agentic synthesis primarily serves to bridge information gaps left by human curation, significantly expand- ing the scope of biographical records beyond encyclopedic baselines. Figure 3: Agentic synthesis versus Wikipedia baseline (pooled U.S. and OECD samples, N = 398). Points indicate coefficient estimates from Equation 2 with 95% confidence intervals. The Wikipedia baseline (LLM_wiki) is normalized to zero. Positive values indicate that agentic syn- thesis outperforms Wikipedia-based extraction. The regional decomposition in Figure 4 further clarifies the magnitude and generality of these gains. Across regions, agentic synthesis raises absolute F1 from approximately 0.76–0.77 under the 23 Wikipedia baseline to roughly 0.87–0.89, a performance level that is well within the range required for downstream empirical applications. Importantly, these improvements are not driven by a single context. For the full-web agent, F1 increases by 15.1 percentage points in the U.S. sample and by 14.3 points in the non-U.S. sample, yielding a pooled gain of 14.7 points. Even when Wikipedia access is prohibited, the non-wiki agent delivers substantial improvements: F1 rises by 12.5 points in the U.S., 11.0 points in the non-U.S. sample, and 11.7 points overall. Two implications follow. First, the scale of improvement is remarkably stable across regions, despite large differences in baseline coverage and information structure. Second, the slightly larger gains outside the U.S. are consistent with the intuition that agentic synthesis is most valuable where curated coverage is weakest. Together, these results indicate that agentic workflows do not merely refine already well- documented cases, but systematically elevate extraction quality across heterogeneous information environments. Figure 4: Agentic synthesis effects by sample. Points report F1 estimates with 95% confidence intervals. Baselines are normalized to sample-specific Wikipedia means (U.S.: 0.82; OECD: 0.73). Collectively, these results demonstrate that agentic synthesis systematically expands the infor- mational scope of elite biographies. The significant gains in recall reflect a structural characteristic of human curation, which prioritizes high-visibility and prominent roles, even though existing Wikipedia data remains highly precise. Consequently, granular details such as early-career po- sitions, officials from smaller nations, non-political affiliations, and concurrent appointments are frequently compressed or omitted. By actively querying local news archives, government records, and organizational filings, agentic workflows successfully recover this “long tail” of politically 24 relevant information. The resulting trade-off between precision and recall is both modest and favorable. While ab- solute precision remains robust at 0.82–0.85, recall improves substantially by 24–31 percentage points. For most empirical research, capturing a significantly larger volume of verified facts jus- tifies a marginal increase in noise, especially given the transparency of source archives and the potential for downstream validation. Across both samples, agentic synthesis achieves F1 scores between 0.87 and 0.94. These levels exceed reported human intercoder reliability in comparable elite datasets and are achieved at a fraction of the cost required for manual data collection. 7 Experiment 3: Why Synthesis Matters? The first two experiments establish two results: large language models can accurately code structured biographies when provided with curated evidence, and agentic retrieval can substantially expand coverage beyond Wikipedia. A natural follow-up question is whether explicit synthesis is still necessary once long-context models can ingest very large inputs. If a model can process hun- dreds of thousands of tokens, one might expect that simply concatenating all retrieved documents would suffice for accurate extraction. This section evaluates that assumption. Holding the underly- ing web evidence fixed, we show that extraction performance depends materially on how evidence is represented to the coder. In particular, long context alone does not eliminate omission and degra- dation errors. Instead, a synthesis step that compresses and organizes evidence into a signal-dense representation is critical for reliable extraction. 7.1 Evaluation Design To isolate representational effects, we hold the downstream coding procedure fixed and com- pare alternative representations of the same retrieved web evidence. Using a fixed Grok-based agent retrieval trajectory, we construct two long-context corpora from identical underlying sources (Table ) . The first is a raw internet corpus (LLM_raw), defined as the direct concatenation of 25 all retrieved documents in full. The second is a refined internet corpus (LLM_refined), defined as a compressed, signal-dense representation composed of selected passages produced during the agentic reasoning loop. For comparison, we retain the Wikipedia-based long-context baseline (LLM_wiki). This design holds the evidence universe constant and varies only the representation presented to the coder. Table 4: Representations used in the diagnostic comparison ConditionUpstream evidence constructionDownstream coderRepresentationTypical context length LLM_wikiHuman (Wikipedia)LLMWiki narrative∼8k LLM_rawAgent (fixed trajectory)LLMRaw concatenation∼300k LLM_refinedAgent (fixed trajectory)LLMRefined passages∼30k Notes. For LLM_raw and LLM_refined, the underlying retrieval trajectory is identical; only the representation supplied to the coder differs. We estimate biography-specific associations between recall and two mechanism proxies. The first captures a quantity channel: context length, operationalized using token-length bins. The second captures a quality channel: language composition, measured as the share of non-English tokens in the coding input. Formally, we estimate specifications of the form: Recall i = α + X a∈A 1(Bio i = a)· (ψ a M i ) + γ· Controls i + ε i , where M i denotes the mechanism proxy and ψ a captures biography-specific slopes. This design allows us to diagnose whether long-context failures arise from scale effects, representation quality, or both. 7.2 Results Figure 5 reports a clear refinement premium. Relative to the Wikipedia long-context baseline (LLM_wiki), the refined representation (LLM_refined) improves F1 by 10.4 percentage points and recall by 17.2 points, with only a modest precision change (-0.9 points). By contrast, raw concatenation (LLM_raw) yields substantially smaller gains: F1 increases by 4.5 points and re- call by 8.8 points, accompanied by a larger precision decline (-2.8 points). Because LLM_raw 26 and LLM_refined are constructed from the same retrieved evidence, this contrast isolates repre- sentation as the binding factor. Long-context extraction failures therefore stem not from missing evidence, but from how evidence is organized and presented to the coder. Figure 5: Refined versus raw corpora: the refinement premium. Figure 6 provides further mechanism evidence. Panel B shows a monotonic quantity penalty: recall declines as context length increases beyond moderate ranges, with the largest losses in the longest bins, consistent with long-context omission errors. Panel A shows a complementary qual- ity channel: higher non-English token shares are associated with lower recall in several bins, in- dicating that heterogeneous or weakly structured inputs further strain extraction. Together, these patterns explain why synthesis-then-coding outperforms raw concatenation even when both draw on the same evidence universe. These results clarify why explicit synthesis remains essential. Long context increases ac- cess to evidence, but it does not guarantee effective use of that evidence. Without refinement, large inputs dilute signal, exacerbate attention limits, and amplify representational noise. Agentic synthesis mitigates these failures by compressing evidence into structured, signal-dense represen- tations that align with the extraction task. In short, synthesis is not a substitute for long context, nor is long context a substitute for synthesis. Reliable large-scale extraction requires both. Model heterogeneity under long-context conditions and descriptive comparisons of corpus composition, are reported in appendix A4. 27 Figure 6: Mechanism evidence for long-context extraction failures. Panel A plots associations between non-English token share and recall; Panel B plots associations between context length and recall. Points denote estimated contrasts relative to the baseline bin; bars indicate 95% confidence intervals. 8 Conclusion “The historian,” as Carr (1961) famously observed, “is necessarily selective. The belief in a core of historical facts existing objectively and independently of the interpretation of the historian is a preposterous fallacy.” A less noticed corollary applies with equal force to the political scientist: the structured datasets that undergird comparative inference are not neutral recordings of political reality, but artifacts of particular production processes—constrained by labor costs, source avail- ability, and the cognitive limits of those doing the coding. This paper takes that constraint seriously and asks whether modern language technology can relax it without sacrificing validity. Our answer, based on three experiments spanning Chinese, American, and OECD political elites, is cautiously affirmative—with important qualifications about how automated systems must be designed to earn that optimism. When given curated biographical inputs identical to those used by trained research assistants, contemporary large language models match or exceed human coding quality, with leading models recovering ten to sixteen percentage points more verified career events than their human counterparts working from the same source. In open-web environments, where no curated Wikipedia biography exists, an agentic synthesis workflow raises absolute F1 scores from 28 the mid-seventies to the high eighties—performance levels that meet or exceed reported human intercoder reliability in comparable elite datasets—at roughly three percent of the per-unit cost of manual collection. And when we hold the evidence universe fixed and vary only how that evidence is represented to the coder, we show that long-context concatenation is not a substitute for synthesis: raw document aggregation yields substantially smaller and noisier gains than an explicitly refined, signal-dense representation derived from the same retrieved sources. These results carry complicated normative implications for a field that has long treated human coding as the unquestioned benchmark for complex extraction. On one hand, the demonstration that LLMs can match and often exceed trained coders on identical inputs should prompt reflection about whether the costs of manual annotation have been buying the validity they were assumed to guarantee. Human coding is subject to well-documented failure modes—principal-agent attrition, selective reading, attention fatigue, and inconsistent adjudication across coders—that systematic LLM evaluation does not share to the same degree. On the other hand, automated pipelines in- troduce their own form of bias, one that is less visible precisely because it is technical rather than human. Our mechanism evidence shows that extraction performance degrades predictably under long inputs, multilingual corpora, and poorly structured source collections—biases that researchers may not notice if they do not inspect intermediate representations. The implication is that validity depends on upstream design choices (e.g., retrieval strategy, evidence compression, source credi- bility weighting) that are no less consequential than the coder’s codebook. The broader significance of our framework lies in what it makes newly possible rather than merely what it does more cheaply. As Table 1 documents, the most analytically important datasets in comparative elite research have remained static for years or decades, updated only when sus- tained institutional funding and coordinated RA labor can be mobilized. This structural rigidity shapes inquiry: researchers tend to study actors at the apex of formal institutions, in countries with dense English-language documentation, using attributes that require the fewest contextual infer- ences. The result is a systematic information structure bias in which the distribution of available data drives theoretical attention rather than the reverse (Wilson and Knutsen 2022). An agentic 29 synthesis workflow does not eliminate this bias, but it substantially attenuates two of its main sources—the cost of expanding coverage to lower-visibility elites and the difficulty of maintaining data currency as political environments evolve. That the non-Wikipedia agent delivers F1 gains nearly as large as the full-web agent suggests that the technology is most valuable precisely where human curation is thinnest: officials from smaller states, minor portfolios, or less-documented institutional contexts where the conventional approach would simply leave the field blank. There is also an interesting methodological parallel worth noting. The precision–recall trade- off we document in agentic synthesis mirrors a well-known dilemma in human coding between breadth and accuracy. Human coders working under time pressure tend to record the most salient positions and omit earlier or concurrent roles that require more inferential effort to recover. The agentic workflow reverses this asymmetry: it excels at recovering the “long tail” of biographical facts precisely because its search behavior is not anchored by salience but by evidential complete- ness. The modest precision declines we observe are consistent with false positives generated at the margin, but they are interpretable, source-traceable, and addressable through downstream valida- tion in a way that human omission errors typically are not. Several directions follow naturally from these findings. The relationship between extraction and classification in political science remains underexplored. Many constructs typically treated as classification targets—regime type, populism, democratic backsliding, policy diffusion—could in principle be reconceptualized as aggregations of extracted sub-claims: time-stamped events, actor attributions, institutional changes. Testing whether a synthesis-then-coding architecture im- proves both the accuracy and the auditability of such labels relative to direct prediction represents a promising frontier. A second direction concerns the contested end of the extraction spectrum. Our evaluations focus on relatively verifiable biographical facts where ground truth is unambigu- ous. Extending the framework to more interpretive attributes—rhetorical frames, policy positions, soft-power signals—raises harder questions about what a “ground truth” even means and how human and machine judgment should be combined when the target concept is itself contested. Fi- nally, the biographical data produced by our framework create new empirical leverage for network 30 approaches to elite politics. By standardizing career events, organizational affiliations, and tempo- ral sequences across tens of thousands of officials, the resulting database permits reconstruction of fine-grained elite networks—co-service ties, overlapping tenures, shared educational institutions— at a scale and cross-national scope that hand-coded datasets have never been able to sustain. How these networks shape recruitment, policy coalitions, and regime stability remains largely unex- plored in comparative work, not because the theoretical questions are unimportant, but because the data have not existed to answer them. They now can. References Alexiadou, Despina. “Cabinet ministers and inequality.” European Journal of Political Research 61, no. 2 (2022): 326–350. — . “Ideologues, partisans, and loyalists: Cabinet ministers and social welfare reform in parlia- mentary democracies.” Comparative Political Studies 48, no. 8 (2015): 1051–1086. Alexiadou, Despina, William Spaniel, and Hakan Gunaydin. “When technocratic appointments signal credibility.” Comparative Political Studies 55, no. 3 (2022): 386–419. Angeli, Gabor, Melvin Jose Johnson Premkumar, and Christopher D Manning. “Leveraging lin- guistic structure for open domain information extraction.” In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Con- ference on Natural Language Processing (Volume 1: Long Papers), 344–354. 2015. Armstrong, Brenna, Tiffany D Barnes, Daina Chiba, and Diana Z O’brien. “Financial crises and the selection and survival of women finance ministers.” American Political Science Review 118, no. 3 (2024): 1305–1323. Bäck, Hanna, Jan Teorell, Alexander Von Hagen-Jamar, and Alejandro Quiroz Flores. “War, per- formance, and the survival of foreign ministers.” Foreign Policy Analysis 17, no. 2 (2021): oraa024. Bail, Christopher A. “Can Generative AI improve social science?” Proceedings of the National Academy of Sciences 121, no. 21 (2024): e2314021121. Baturo, Alexander. “Cursus Honorum: Personal background, careers and experience of political leaders in democracy and dictatorship—New data and analyses.” Politics and Governance 4, no. 2 (2016): 138–157. Baturo, Alexander, Niheer Dasandi, and Slava J Mikhaylov. “Understanding state preferences with text as data: Introducing the UN General Debate corpus.” Research& Politics 4, no. 2 (2017): 2053168017712821. Baturo, Alexander, and Johan A. Elkink. “What countries select more experienced leaders? The PolEx measure of political experience.” British Journal of Political Science 52, no. 3 (2022): 1455–1464. 31 Baturo, Alexander, and Jakob Tolstrup. “Incumbent takeovers.” Journal of Peace Research 60, no. 2 (2023): 373–386. Benoit, Kenneth, Drew Conway, Benjamin E Lauderdale, Michael Laver, and Slava Mikhaylov. “Crowd-sourced text analysis: Reproducible and agile production of political data.” American Political Science Review 110, no. 2 (2016): 278–295. Benoit, Kenneth, Scott De Marchi, Conor Laver, Michael Laver, and Jinshuai Ma. “Using large language models to analyze political texts through natural language understanding.” American Journal of Political Science (2025). Binderkrantz, Anne Skorkjær, Jørgen Grønnegård Christensen, Peter Munk Christiansen, Marie Kaldahl Nielsen, and Helene Helboe Pedersen. “Closed shutters or revolving doors? Elite career track similarity and elite sector transfers in Denmark.” European Journal of Political Research 63, no. 3 (2024): 1022–1041. Bomprezzi, Pietro, Axel Dreher, Andreas Fuchs, Teresa Hailer, Andreas Kammerlander, Lennart C Kaplan, Silvia Marchesi, Tania Masi, Charlotte Robert, and Kerstin Unfried. Wedded to Prosperity? Informal Influence and Regional Favoritism. CEPR Discussion Paper 18878 (v.2), 2025. Bonica, Adam. “Database on ideology, money in politics, and elections: Public version 2.0 [com- puter file].” URL: https://data. stanford. edu/dime (2016). Braun, Matías, and Claudio Raddatz. “Banking on politics: When former high-ranking politicians become bank directors.” The World Bank Economic Review 24, no. 2 (2010): 234–279. Carnes, Nicholas. White-collar government: The hidden role of class in economic policy making. University of Chicago Press, 2024. Carozzi, Felipe, and Luca Repetto. “Sending the pork home: Birth town bias in transfers to Italian municipalities.” Journal of public economics 134 (2016): 42–52. Carr, Edward Hallett. What is History? Cambridge, UK: Cambridge University Press, 1961. De Luca, Giacomo, Roland Hodler, Paul A. Raschky, and Michele Valsecchi. “Ethnic favoritism: An axiom of politics?” Journal of Development Economics 132 (2018): 115–129. Dreher, Axel, Michael J. Lamla, Sarah M. Lein, and Frank Somogyi. “The impact of political leaders’ profession and education on reforms.” Journal of Comparative Economics 37, no. 1 (2009): 169–193. Ellis, Cali Mortenson, Michael C Horowitz, and Allan C Stam. “Introducing the LEAD data set.” International Interactions 41, no. 4 (2015): 718–741. Eschenauer-Engler, Tanja, and Bastian Herre. “Coup leaders: A new comprehensive dataset, 1950– 2020.” Forthcoming, European Political Science (2023). Fang, Hanming, Ming Li, and Guangli Lu. Decoding China’s Industrial Policies. Tech. rep. Na- tional Bureau of Economic Research, 2025. Fearon, James D., Kimuli Kasara, and David D. Laitin. “Ethnic minority rule and civil war onset.” American Political Science Review 101, no. 1 (2007): 187–193. Fisman, Raymond, Jing Shi, Yongxiang Wang, and Weixing Wu. “Social ties and the selection of China’s political elite.” American Economic Review 110, no. 6 (2020): 1752–1781. 32 Fuchs, Andreas, and Katharina Richert. “Development minister characteristics and aid giving.” European Journal of Political Economy 53 (2018): 186–204. Funke, Manuel, Moritz Schularick, and Christoph Trebesch. “Populist leaders and the economy.” American Economic Review 113, no. 12 (2023): 3249–3288. Gentzkow, Matthew, Bryan Kelly, and Matt Taddy. “Text as data.” Journal of Economic Literature 57, no. 3 (2019): 535–574. Gerring, John, Erzen Oncel, Kevin Morrison, and Daniel Pemstein. “Who rules the world? A por- trait of the global leadership class.” Perspectives on politics 17, no. 4 (2019): 1079–1097. Gilardi, Fabrizio, Meysam Alizadeh, and Maël Kubli. “ChatGPT outperforms crowd workers for text-annotation tasks.” Proceedings of the National Academy of Sciences 120, no. 30 (2023): e2305016120. Goemans, Henk E, Kristian Skrede Gleditsch, and Giacomo Chiozza. “Introducing Archigos: A dataset of political leaders.” Journal of Peace research 46, no. 2 (2009): 269–283. Grimmer, Justin, and Brandon M Stewart. “Text as data: The promise and pitfalls of automatic content analysis methods for political texts.” Political analysis 21, no. 3 (2013): 267–297. Gu, Jiawei, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, et al. “A survey on llm-as-a-judge.” The Innovation (2024). Hallerberg, Mark, and Joachim Wehner. “The educational competence of economic policymakers in the EU.” Global Policy 3 (2012): 9–15. Halterman, Andrew, and Katherine A Keith. “Codebook LLMs: Evaluating LLMs as Measurement Tools for Political Science Concepts.” arXiv preprint arXiv:2407.10747 (2024). Hassan, Tarek A, Stephan Hollander, Laurence Van Lent, and Ahmed Tahoun. “Firm-level political risk: Measurement and effects.” The quarterly journal of economics 134, no. 4 (2019): 2135– 2202. Herre, Bastian. “Identifying ideologues: A global dataset on political leaders, 1945–2020.” British Journal of Political Science 53, no. 2 (2023): 740–748. Jiang, Junyan. “Making bureaucracy work: Patronage networks, performance incentives, and eco- nomic development in China.” American Journal of Political Science 62, no. 4 (2018): 982– 999. Jiang, Junyan, and Muyang Zhang. “Friends with benefits: Patronage networks and distributive politics in China.” Journal of Public Economics 184 (2020): 104143. King, Gary, Jennifer Pan, and Margaret E Roberts. “How censorship in China allows government criticism but silences collective expression.” American political science Review 107, no. 2 (2013): 326–343. Landry, Pierre F. Decentralized Authoritarianism in China: the Communist Party’s control of local elites in the post-Mao era. Cambridge University Press, 2008. Lee, Don S, and Charles T McClean. “Breaking the cabinet’s glass ceiling: the gendered effect of political experience in presidential democracies.” Comparative Political Studies 55, no. 6 (2022): 992–1020. 33 Lewis, Patrick, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.” In NeurIPS. 2020. Li, Haitao, Qian Dong, Junjie Chen, Huixue Su, Yujia Zhou, Qingyao Ai, Ziyi Ye, and Yiqun Liu. “Llms-as-judges: a comprehensive survey on llm-based evaluation methods.” arXiv preprint arXiv:2412.05579 (2024). Licht, Amanda A. “Introducing Regular Turnover Details, 1960–2015: A dataset on world leaders’ legal removal from office.” Journal of Peace Research 59, no. 2 (2022): 277–285. Liebman, Benjamin L, Margaret E Roberts, Rachel E Stern, and Alice Z Wang. “Mass digitization of Chinese court decisions: How to use text as data in the field of Chinese law.” Journal of Law and Courts 8, no. 2 (2020): 177–201. Lin, Rongrong. “The rise of technocratic leadership in the 1990s in the People’s Republic of China.” Politics and Governance 8, no. 4 (2020): 157–167. Liu, Hanzhang. “Meritocracy as authoritarian co-optation: Political selection and upward mobility in China.” American Political Science Review 118, no. 4 (2024): 1856–1872. Liu, Nelson F, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. “Lost in the middle: How language models use long contexts.” Transactions of the Association for Computational Linguistics 12 (2024): 157–173. Mallen, Alex, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. “When not to trust language models: Investigating effectiveness of parametric and non-parametric memories.” In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 9802–9822. 2023. Mattes, Michaela, Brett Ashley Leeds, and Naoko Matsumura. “Measuring change in source of leader support: The CHISOLS dataset.” Journal of Peace Research 53, no. 2 (2016): 259– 267. Muthiah, Sathappan, Bert Huang, Jaime Arredondo, David Mares, Lise Getoor, Graham Katz, and Naren Ramakrishnan. “Planned protest modeling in news and social media.” In Proceedings of the AAAI Conference on Artificial Intelligence, 29:3920–3927. 2. 2015. Nyrup, Jacob, and Stuart Bramwell. “Who governs? A new global dataset on members of cabinets.” American Political Science Review 114, no. 4 (2020): 1366–1374. Nyrup, Jacob, Carl Henrik Knutsen, Peter Egge Langsæther, and Ina Lyftingsmo Kristiansen. “Paths to power: A new dataset on the social profile of governments.” British Journal of Political Science 55 (2025): e129. Ornstein, Joseph T, Elise N Blasingame, and Jake S Truscott. “How to train your stochastic parrot: Large language models for political texts.” Political Science Research and Methods 13, no. 2 (2025): 264–281. Palmer, Alexis, Noah A Smith, and Arthur Spirling. “Using proprietary language models in aca- demic research requires explicit justification.” Nature Computational Science 4, no. 1 (2024): 2–3. 34 Putnam, Robert D. The Comparative Study of Political Elites. Englewood Cliffs, NJ: Prentice-Hall, 1976. https://archive.org/details/comparativestudy00putn. Raleigh, Clionadh, and Daniel Wigmore-Shepherd. “Elite coalitions and power balance across African regimes: introducing the African cabinet and political elite data project (ACPED).” Ethnopolitics 21, no. 1 (2022): 22–47. Reuter, Ora John, and David Szakonyi. “Elite Defection under Autocracy: Evidence from Russia.” American Political Science Review 113, no. 2 (2019): 552–568. Ricart-Huguet, Joan. “Colonial education, political elites, and regional political inequality in Africa.” Comparative Political Studies 54, no. 14 (2021): 2546–2580. Shi, Freda, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H Chi, Nathanael Schärli, and Denny Zhou. “Large language models can be easily distracted by irrelevant con- text.” In International Conference on Machine Learning, 31210–31227. PMLR, 2023. Shih, Victor, Christopher Adolph, and Mingxing Liu. “Getting ahead in the communist party: ex- plaining the advancement of central committee members in China.” American political sci- ence review 106, no. 1 (2012): 166–187. Svolik, Milan W. The politics of authoritarian rule. Cambridge University Press, 2012. Thrall, Calvin. “Informational lobbying and commercial diplomacy.” American Journal of Political Science 69, no. 3 (2025): 1147–1162. Vittori, Davide, Jean-Beniot Pilet, Sebastien Rojon, and Emilien Paulis. “Technocratic ministers in office in European countries (2000–2020): What’s new?” Political Studies Review 21, no. 4 (2023): 867–886. Wilson, Matthew Charles, and Carl Henrik Knutsen. “Geographical coverage in political science research.” Perspectives on Politics 20, no. 3 (2022): 1024–1039. Woldense, Josef, and Alex Kroeger. “Elite Change without Regime Change: Authoritarian Persis- tence in Africa and the End of the Cold War.” American Political Science Review 118, no. 1 (2024): 178–194. Wu, Patrick Y, Jonathan Nagler, Joshua A Tucker, and Solomon Messing. “Large language models can be used to estimate the latent positions of politicians.” arXiv preprint arXiv:2303.12057 (2023). Yang, Zhilin, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D Manning. “HotpotQA: A dataset for diverse, explainable multi-hop ques- tion answering.” In Proceedings of the 2018 conference on empirical methods in natural lan- guage processing, 2369–2380. 2018. Yao, Shunyu, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. “ReAct: Synergizing reasoning and acting in language models.” In International Conference on Learning Representations. 2023. https://openreview.net/forum?id=WE_vluYUL- X. Yu, Shu, and Richard Jong-A-Pin. “Rich or alive? Political (in)stability, political leader selection and economic growth.” Journal of Comparative Economics 48, no. 3 (2020): 561–577. 35 Zhang, Han, and Jennifer Pan. “Casm: A deep-learning approach for identifying collective ac- tion events with text and image data from social media.” Sociological Methodology 49, no. 1 (2019): 1–57. Ziems, Caleb, William Held, Omar Shaikh, Jiaao Chen, Zhehao Zhang, and Diyi Yang. “Can large language models transform computational social science?” Computational Linguistics 50, no. 1 (2024): 237–291. 36 Online Appendix Table of Contents A1 Formalizing Classification versus ExtractionA-1 A1.1 Definitions and notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-1 A1.2 Extraction as synthesis→ coding . . . . . . . . . . . . . . . . . . . . . . . . . . . A-1 A1.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-2 A2 Architecture and Model DetailsA-3 A2.1 Architecture and information flow . . . . . . . . . . . . . . . . . . . . . . . . . . A-3 A2.2 Models and operational constraints . . . . . . . . . . . . . . . . . . . . . . . . . . A-4 A2.2.1 Other infrastructure costs . . . . . . . . . . . . . . . . . . . . . . . . . . . A-6 A3 Consolidated Ground Truth (CGT) ConstructionA-9 A3.1 Protocol (pooling, consensus, and verification) . . . . . . . . . . . . . . . . . . . . A-9 A3.2 Audit checks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-10 A4 Supplementary ResultsA-12 A4.1 Model performance without external resources . . . . . . . . . . . . . . . . . . . A-12 A4.2 Agentic Synthesis in a High-Curation Setting: China . . . . . . . . . . . . . . . . A-12 A4.3 Model heterogeneity under long-context conditions . . . . . . . . . . . . . . . . . A-13 A4.4 Cross-national heterogeneity in retrieved corpora composition . . . . . . . . . . . A-14 A4.5 Disaggregated mechanism plots . . . . . . . . . . . . . . . . . . . . . . . . . . . A-14 A5 Case Study: Erik Solheim Agent RunA-15 A5.1 Agent Execution Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-15 A5.2 Three-Phase Search Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-16 A5.2.1 Phase 1: Initial Skeleton Construction (Messages 0–7) . . . . . . . . . . . A-16 A5.2.2 Phase 2: Gap Filling – Family and Mid-Career (Messages 8–20) . . . . . . A-17 A5.2.3 Phase 3: Deep Dive – Early Life and Education (Messages 21–40) . . . . . A-17 A5.3 Source Diversity and Language Composition . . . . . . . . . . . . . . . . . . . . A-18 A5.4 Retrieved Evidence and Archive Structure . . . . . . . . . . . . . . . . . . . . . . A-18 A5.5 Ground Truth Comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-18 A5.6 Key Insights and Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-19 A5.6.1 Discovery Successes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-19 A5.6.2 Coverage Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-19 A5.6.3 Efficiency Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-20 A6 PromptsA-21 A6.1 Architecture prompts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-21 A6.1.1 Supervisor prompt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-21 A6.1.2 Searcher prompt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-23 A6.2 Experiment prompts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-25 A6.2.1 Query template (EN) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-25 A-0 A6.2.2 Research plan template (EN) . . . . . . . . . . . . . . . . . . . . . . . . . A-26 A6.3 Evaluation prompts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-26 A6.3.1 Fact-checking (related-content judge) prompt . . . . . . . . . . . . . . . . A-26 A6.3.2 Entrywise evaluation prompt . . . . . . . . . . . . . . . . . . . . . . . . . A-27 A7 A Practical Guide to Information Extraction with LLMsA-31 A7.1 A minimal workflow for reliable extraction . . . . . . . . . . . . . . . . . . . . . A-31 A-1 A1 Formalizing Classification versus Extraction To formalize the distinction emphasized in the main text, we represent (i) classification as a closed-set mapping and (i) political fact extraction as a two-stage pipeline that couples evidence synthesis with structured record construction. The key implication is methodological: larger or “more capable” language models need not yield reliable gains for extraction if the binding con- straint is open-domain discovery, long-context integration, and multi-stage task execution under explicit budgets. A1.1 Definitions and notation We use extraction to denote the end-to-end task of producing structured political facts from unstructured corpora (a single document, a fixed document collection, or an open-ended docu- ment universe), following a predefined codebook. Within extraction, we distinguish synthesis as evidence acquisition and refinement (search/browse/source selection, filtering, and condensation) from coding as mapping a fixed, refined corpus into codebook-conformant structured records. For classification, let x denote the input text (a document or fixed set of documents) from an input spaceX, and letY denote a pre-defined, finite set of disjoint labels (e.g.,Y =Left, Right). The classification task is a mapping f classification :X→Y, often implemented by selecting the most likely label, ˆy = arg max y∈Y Pr(y| x). Classification is therefore discriminative: the output space is fixed and known ex ante, and errors are primarily mislabeling. A1.2 Extraction as synthesis→ coding Synthesis corresponds to evidence retrieval and refinement over a large, heterogeneous source universe (e.g., the open web). Let D denote the (implicit) universe of candidate documents and let q denote a query (e.g., an entity name plus accumulated context from prior steps). Evidence retrieval can be written as a retrieval mapping that selects a bounded subset of evidence: f retrieval : (q, h)→D k ⊆D, where h is interaction history and k is a number of retrieved corpus (or retrieved corpus). In practice, synthesis also includes filtering and condensation of D k into a curated corpus that is feasible for downstream coding. Let g denote a condensation operator that maps retrieved evidence into a curated corpus x (e.g., a synthetic report) in an input spaceX: g :D k →X. A-1 We therefore write synthesis as the composition f syn (q, h) = g ( f ret (q, h) ) ∈X. Coding then maps a fixed, refined corpus into structured records under an explicit codebook. Here, syn means synthesis. LetB denote a target codebook consisting of fieldsb 1 ,..., b K (e.g., Organization, Role, Start Date, End Date). Let V denote the vocabulary of the language model and let V ≤L denote the set of token sequences up to length L (a convenient representation for LLM outputs). A codebook-conformant record is a tuple z = (v 1 ,..., v K ) where each field value v k ∈ V ≤L . Let Z ⊆ (V ≤L ) K denote the set of such records, and let T denote the set of finite sequences of records (trajectories). We write the coding step as f code :X×B→T. The key difference from classification is that the output space is effectively open and the task is not separable: values are not drawn from a small closed set, and field-level decisions depend on other fields and on evidence scattered across documents. For elite biographies, the target is an ordered career trajectory rather than a single record. Let the extracted trajectory be ˆτ = (z 1 ,..., z T ) ∈ T , where each z t ∈ Z is a codebook-conformant record. End-to-end extraction is the two-stage composition: ˆτ = f ext (q, h,B) := f code f syn (q, h),B . Accordingly, beyond mislabeling, extraction failures include hallucination, span/value errors, miss- ing events (recall loss), and codebook violations. A1.3 Summary Table A1.1: Conceptual distinction between classification and extraction tasks. DimensionClassification ( f cls )Extraction (Synthesis→ Coding) PipelineSingle-step mappingTwo-stage: synthesis f syn (retrieval f ret + condensation g) then coding f code ; end-to- end f ext Output space Y (finite, closed)Intermediate evidenceD k ⊆ D, refined cor- pus x ∈ X, and trajectory ˆτ ∈ T (open, ef- fectively unbounded) ObjectiveLabel selectionEvidence synthesis + reconstruction (re- cover structured values) Typical errorsMislabelingMissed facts (recall loss); Unsupported facts (precision loss) A-2 A2 Architecture and Model Details This appendix documents the system architecture and model configuration used to produce the agentic biographies and the derived long-context corpora used in the experiments. In the main text, we emphasize the core conceptual feature—a tool-using, ReAct-style workflow for iterative evidence gathering and synthesis (Yao et al. 2023). Here we provide additional implementation detail to make the design auditable and to clarify what is held constant across comparisons. Figure A2.1: Architecture for agentic synthesis. A Supervisor maintains global state and delegates bounded retrieval tasks to specialized Searcher workers. Retrieved evidence is stored in an Archive and then mapped by a Coder into a structured biography. A2.1 Architecture and information flow We implement a Supervisor–Worker architecture to manage the cognitive overhead of open- web synthesis. The central division of labor is between (i) strategic, long-horizon reasoning about what is missing and what to search next and (i) tactical, short-horizon retrieval and reading of specific sources. Core components. The Supervisor interprets the extraction objective and codebook, orches- trates the workflow over multiple cycles, and produces the final consolidated record. The Su- pervisor does not process the full raw web corpus directly; instead, it operates on structured evi- dence packets returned by specialized Searcher workers. The Archive stores retrieved content and provenance metadata (e.g., URLs and retrieval time), enabling deduplication, backtracking when contradictions arise, and transparent linkage between claims and supporting passages. The Coder A-3 converts the Supervisor’s stabilized draft and the archived evidence bundle into the structured bi- ography output used for evaluation. Two-stage pipeline. Our pipeline separates the extraction of political facts into (i) synthesis, which transforms a large, noisy document set into a higher-signal evidentiary representation, and (i) coding, which maps that representation into a structured biography under a fixed codebook. This separation is crucial for interpretation: it allows us to hold the downstream coder constant while varying the upstream synthesizer (RQ2), and to isolate representation effects while holding the underlying web evidence fixed (Section 7). Intermediate data structures. To keep multi-cycle synthesis auditable, we use explicit interme- diate objects: (i) a Structured Input (objective, codebook, and constraints), (i) a System State (running plan, partial biography, unresolved gaps, and bookkeeping), (i) Information Batch Overviews produced by Searchers (source details, task-specific summaries, and extracted pas- sages), and (iv) a Structured Final Output (codebook-conformant biography with evidence point- ers via the Archive). Prompts are role-specific and enforce these interfaces; full prompt templates and tool specifications are provided in the replication materials. A2.2 Models and operational constraints We evaluate multiple model families as downstream coders (e.g., LLM_wiki) and as agentic components (e.g., LLM_agent). The primary model families used in this version are Grok-4.1- Fast, Gemini-2.5-Flash, Qwen-3-80B, and Qwen-3-225B. Across experiments, we hold prompts and codebooks fixed within each role and keep system budgets (e.g., maximum steps and termina- tion rules) constant within comparison Bios. Model choice is consequential in agentic synthesis because end-to-end performance depends not only on reading comprehension, but also on tool-use reliability and the ability to sustain long- horizon interaction without drifting. We therefore prioritize models that jointly satisfy three prac- tical constraints: strong reasoning in a fixed corpus, robust multi-turn planning and tool use, and affordability at scale. These constraints motivate the inclusion of “Fast/Flash” variants where avail- able and efficient open-weight alternatives. Finally, we design synthesis to remain operationally bounded. The Supervisor maintains a running search summary and a gap list, decomposes the task into Searcher instructions, and ter- minates when a step budget is reached or when the marginal value of additional retrieval declines. Because the workflow logs its actions and preserves archived evidence, we can audit intermediate representations and support claim verification during CGT construction (Appendix A3). Table A2.1 documents the search behavior of different agent models during the upstream retrieval phase. For each model family (Gemini 2.5 Flash, Grok, Qwen, Qwen 225B), we re- port metrics across multiple configurations (non_wiki and wiki variants) and geographic regions (Overall, US, OECD, China). The “Searched” column captures the total volume of search results processed, while “Search Times” indicates how many search queries each model issued. “Used URLs” counts the distinct web sources each model successfully retrieved, providing a proxy for retrieval breadth. These metrics reveal substantial variation in search strategies across models: A-4 Table A2.1: Agent Search Metrics by Model and Region ModelVersionRegion Officials Searched (Avg) Search Times (Avg) Used URLs (Avg) Gemini 2.5 Flash Gemini 2.5 Flash non_wikiOverall398 62,389 (156.75)8,935 (22.44)9,299 (23.37) Gemini 2.5 Flash non_wikiUS198 30,672 (154.91)4,220 (21.31)4,917 (24.83) Gemini 2.5 Flash non_wikiOECD200 31,717 (158.59)4,715 (23.57)4,382 (21.91) Gemini 2.5 Flash model_wiki Overall398 61,046 (153.35)8,035 (20.18)7,075 (17.78) Gemini 2.5 Flash model_wikiUS198 28,917 (146.05)3,651 (18.44)3,348 (16.91) Gemini 2.5 Flash model_wiki OECD200 32,129 (160.65)4,384 (21.92)3,727 (18.64) Grok-4.1-Fast Grok-4.1-Fastnon_wikiOverall398 68,677 (172.60)5,706 (14.34)10,020 (25.18) Grok-4.1-Fastnon_wikiUS198 36,218 (182.92)2,992 (15.11)5,020 (25.35) Grok-4.1-Fastnon_wikiOECD200 32,459 (162.29)2,714 (13.57)5,000 (25.00) Grok-4.1-Fastmodel_wiki Overall598 95,260 (159.28)5,938 (9.94)13,045 (21.82) Grok-4.1-Fastmodel_wikiUS198 30,230 (152.68)2,249 (11.36)4,433 (22.39) Grok-4.1-Fastmodel_wikiChina200 36,735 (183.68)1,522 (7.61)4,224 (21.12) Grok-4.1-Fastmodel_wiki OECD200 28,295 (141.47)2,167 (10.84)4,388 (21.94) Qwen Qwenmodel_wiki Overall398 41,411 (104.06)5,747 (14.43)8,640 (21.71) Qwenmodel_wikiUS198 21,225 (107.20)2,788 (14.08)4,646 (23.46) Qwenmodel_wiki OECD200 20,186 (100.93)2,959 (14.79)3,994 (19.97) Qwen 225B Qwen 225Bmodel_wiki Overall39835,302 (88.67)4,521 (11.36)7,121 (17.89) Qwen 225Bmodel_wikiUS19816,626 (83.97)2,204 (11.13)3,285 (16.59) Qwen 225Bmodel_wiki OECD20018,676 (93.38)2,317 (11.59)3,836 (19.18) Notes. This table presents detailed search metrics for the agent search process across different models, versions, and regions. Each metric column reports both the total count and the per-official average in parentheses. “Searched (Avg)” shows total search results retrieved (average per official). “Search Times (Avg)” shows the number of search operations performed (average per official). “Used URLs (Avg)” counts the total number of useful URLs retrieved by agent from search results (average per official). Overall region represents aggregated totals across all sub-regions (e.g., US + OECD, or US + China + OECD). A-5 some models (e.g., Gemini) issue more queries and retrieve more results, while others (e.g., Grok) converge more efficiently on relevant sources. Notably, Grok demonstrates superior search efficiency across all models. As shown in Ta- ble A2.1, Grok achieves comparable or better coverage with substantially fewer search opera- tions: it requires only 11.7 search times per official on average (across both versions), compared to 21.3 for Gemini 2.5 Flash and 14.4 for Qwen models. This efficiency translates directly into lower resource consumption and cost—Grok’s per-official cost ($0.15–$0.18) is approximately 4–5× lower than Gemini 2.5 Flash ($0.65–$0.79) while maintaining competitive retrieval quality (88–102 unique URLs per official versus 123 for Gemini). Grok’s ability to converge on relevant sources with fewer queries makes it particularly well-suited for large-scale agentic synthesis where cost and efficiency are critical constraints. Larger models (Qwen 225B) tend to generate more output tokens than smaller variants (Qwen 80B), driving up costs despite similar input prices. The 225B variant v2 generates 5.2× more output tokens than the 80B variant (29.6M vs. 5.7M), contributing to its 2.9× higher total cost. Table A2.2 reports the token usage and costs for each model configuration across different modes. Agent costs represent the full agentic synthesis pipeline (Searcher + Supervisor + Coder). Wiki LC (long-context) costs represent single-pass coding from Wikipedia pages. LC Raw and LC Synth are long-context variants using retrieved documents. A2.2.1 Other infrastructure costs We relied on external providers for web research and robust web content retrieval. In particu- lar, we used Jina and Exa as retrieval services capable of extracting full page contents from URLs. For web search, we used serp.dev to obtain programmatic access to Google’s search results. Beyond model inference costs, production deployments should reserve budget for several API-related expenses. For search, about $400 covers roughly 400,000 queries (around $0.001 per request). Large-scale document fetching via Jina and exa services is on the order of $200. Overall, our full experimental effort—including development, testing, experimentation, and eval- uation—amounted to approximately $5,000 in total spending across all LLM APIs, search, and retrieval services, as well as coders’ hiring cost. Table A2.3 breaks down the average number of URLs retrieved per official by type, model, and region. We categorize URLs into eight types using a politician-centric reliability hierarchy: official government sources (primary/authoritative), wiki pages, news/journalism media (tertiary/interpre- tive), non-wiki reference databases (secondary/structured), social media platforms, NGO/advocacy sources, commercial sources, and other sources. The distribution varies substantially across models and regions. Notably, wiki variants consistently retrieve more wiki pages (2.16–4.17 per official) compared to non-wiki variants (1.36–2.04), while non-wiki variants rely more heavily on non-wiki reference databases (e.g., Grok non-wiki: 2.74 per official in US vs Grok v7 wiki: 2.34). Grok v7 (wiki) shows particularly high government source usage in China (7.44 per official, representing 35% of all URLs) compared to US (26.9%) and OECD (22.1%). China data reflects only the final Grok v7 experiment due to processing log loss for earlier China search results. A-6 Table A2.2: Model Token Usage and Costs ModelModeInput (M) Output (M) Input ($ ) Output ($ ) Total ($ ) Per Official ($ ) Agent Models Grok 4 FastAgent Wiki271.713.1$54.35$6.54$60.88$0.153 Grok 4 FastNon-Wiki323.415.0$64.68$7.49$72.17$0.181 Gemini 2.5 Flash Agent Wiki754.512.3$226.36$30.79$257.15$0.646 Gemini 2.5 Flash Non-Wiki938.212.7$281.46$31.69$313.15$0.787 Qwen3 225BAgent Wiki517.429.6$93.12$15.99$109.11$0.274 Qwen3 80BAgent Wiki341.45.7$30.72$6.25$36.97$0.093 Long-Context (LC) Modes Grok 4 FastWiki9.01.3$1.79$0.64$2.43$0.006 Grok 4 FastLC Raw51.21.4$10.23$0.69$10.92$0.027 Grok 4 FastLC Synth23.41.8$4.68$0.88$5.56$0.014 Gemini 2.5 Flash Wiki13.00.6$3.90$1.59$5.49$0.014 Gemini 2.5 Flash LC Raw45.30.8$13.59$1.98$15.57$0.039 Gemini 2.5 Flash LC Synth25.30.8$7.60$2.04$9.64$0.024 Qwen3 225BWiki13.02.5$2.34$1.37$3.71$0.009 Qwen3 225BLC Raw29.52.0$5.31$1.06$6.37$0.016 Qwen3 225BLC Synth24.32.4$4.37$1.28$5.65$0.014 Qwen3 80BWiki12.90.5$1.16$0.51$1.67$0.004 Qwen3 80BLC Raw29.53.3$2.65$3.61$6.26$0.016 Qwen3 80BLC Synth24.44.1$2.19$4.48$6.67$0.017 Notes. Sample size: N=398 officials for Agent modes; N varies for LC modes. Agent Wiki includes models with Wikipedia access; Non-Wiki blocks Wikipedia during synthesis. Wiki represents single-pass long-context coding from Wikipedia pages only. LC Raw uses retrieved documents; LC Synth uses supervisor-enhanced retrieved documents. Input/Output in millions (M) of tokens. Price based on openrouter model prices. A-7 Table A2.3: URL Types per Official by Model and Region ModelRegion Total Govt News Wiki Reference Platforms NGO Commercial Other Grok Grok non-wikiUS25.61 9.023.80 0.184.040.712.280.784.81 Grok non-wikiOECD 25.00 7.567.27 0.341.941.252.121.283.24 Grok wikiUS22.39 6.023.10 2.583.180.631.990.534.35 Grok wikiChina21.23 7.449.22 2.070.050.130.410.001.92 Grok wikiOECD 22.05 4.875.70 3.021.631.151.711.232.73 Gemini Gemini non-wiki US25.09 7.043.37 0.024.951.212.771.024.72 Gemini non-wiki OECD 22.02 4.625.43 0.032.782.202.111.862.99 Gemini wikiUS16.99 3.841.79 2.582.860.661.400.743.12 Gemini wikiOECD 18.92 3.433.79 3.031.911.671.501.482.11 Qwen 225B wikiUS16.85 4.592.24 2.652.350.831.580.192.41 wikiOECD 19.18 5.184.76 2.481.371.691.250.461.99 Qwen 80B wikiUS23.46 6.713.85 2.682.741.511.760.493.73 wikiOECD 19.97 4.592.81 2.772.062.981.390.762.60 Notes. This table shows the average number of URLs retrieved per official by type, model, and region. Grok v7 = model_wiki variant; Grok non-wiki = non_wiki variant; Gemini v2 = model_wiki variant; Gemini non-wiki = non_wiki variant; Qwen 225B v2 and Qwen 80B = model_wiki variants. China data reflects only the final Grok v7 experiment due to processing log loss for earlier China search results. Govt = official government sources; News = journalism and media; Wiki = Wikipedia and wiki-style pages; Reference = non-wiki reference databases (e.g., VoteSmart, Ballotpedia); Platforms = social media platforms; NGO = advocacy/NGO sources; Commercial = com- mercial/business sources; Other = uncategorized sources including entertainment, media, search engines, and miscel- laneous. A-8 A3 Consolidated Ground Truth (CGT) Construction This appendix provides the full claim-level CGT protocol summarized in the main text. Our goal is to produce a defensible, auditable reference set of claims for scoring. A3.1 Protocol (pooling, consensus, and verification) Inputs (fixed pool per individual). For each individual i, we construct a fixed pool of 9 bi- ographies: four agent biographies (two agent model families × two variants), four LLM_wiki biographies (four coder models applied to the same Wiki corpus), and one human-written Wiki biography (Human_wiki). We construct the CGT from this pool and score all candidate systems against it. Long-context baselines (LLM_raw and LLM_refined) are scored against the CGT but are not included in the CGT pool to avoid mechanically altering the consensus set. Step 1 (claim extraction and normalization). We decompose each biography into a set of atomic, comparable claims (e.g., education events; offices held with dates; party membership). We then normalize claims to reduce superficial disagreement: we canonicalize entity names and common aliases when available; standardize role and organization strings (e.g., ministry/agency names); and harmonize date formats, resolving partial dates into comparable intervals when possi- ble. After normalization, paraphrases that express the same event are treated as the same claim. Step 2 (consensus filter for high-confidence claims). For each normalized claim, we compute its presence rate in the 9-biography pool: presence(claim) = #bios containing claim 9 . Claims with presence ≥ 5/9 enter the CGT as high-confidence claims. Claims with presence ≤ 4/9 are treated as disputed/low-confidence and proceed to evidence verification. Step 3 (evidence-conditional verification for low-confidence claims). Low-confidence claims are evaluated against a pooled evidence bundle: the union of archived passages and sources col- lected across all agent runs and variants for the same individual. We add a low-confidence claim to the CGT only if it is supported by explicit evidence in this pooled archive. Pooling across agent runs mitigates dependence on any single retrieval trajectory and improves robustness to id- iosyncratic search failures. In our implementation, we operationalize this step with an evidence- conditional verifier (GPT-5-mini), which receives the candidate claim and the pooled archive text and returns a supported/unsupported judgment. In practice, we used the soft label to let LLMs label the level of support by 1-5, and treat claims scored above 3 as supported claims. Step 4 (CGT definition and scoring). Let C High i denote the set of high-confidence claims and C Validated i the set of evidence-validated low-confidence claims. We define the claim-level CGT as: C ∗ i =C High i ∪C Validated i . A-9 Each candidate system output is converted into a normalized claim set b C i and scored against C ∗ i using precision, recall, and F1 as defined in the main text. A3.2 Audit checks We validate the reliability of the automated CGT construction through two distinct audit stud- ies. To assess whether evidence-conditional verification aligns with expert judgment and external search verification, we drew a random sample of 20 officials from the OECD dataset. Table A3.1 summarizes the consistency rates, measured as the percentage of exact matches between the auto- mated CGT verdicts and the alternative verification methods. Human–machine alignment. Two graduate research assistants independently verified extracted claims for the sampled officials. Auditors were provided with the claim text and access to open- web search but were blinded to the model’s verdict. For non-English sources, auditors utilized translation tools alongside original source inspection. Agreement rates were calculated by com- paring human judgments against the automated judge’s outputs. As shown in Table A3.1, the average agreement rate is 91.3%, indicating strong alignment between the automated verifier and human judgment in adjudicating low-consensus discoveries. External validation (Exa). To rule out model-specific artifacts in the retrieval process, we cross- validated claims using Exa deepsearch, a neural search engine optimized for semantic retrieval and fact checking (Exa 2025). We queried Exa to retrieve high-quality, independent evidence for each claim and compared its verification results with our pipeline’s verdicts. The analysis reveals a negligible discrepancy rate (average agreement 98.7%), confirming that the synthesized ground truth is factually grounded and robust to retrieval method variations. A-10 Table A3.1: Audit results: Consistency checks across verification methods for sampled OECD officials. Agreement Rate (%) Country Official NamePosition (Abbreviated)HumanExa CZEPavel BlazekMin. of Justice93.598.1 CZEJaromir DrabekMin. of Labor & Social Affairs89.098.8 JPNAiko ShimajiriMin. in Charge of “Cool Japan” Strategy91.499.6 JPNKenichiro SasaeAmbassador to the US95.097.7 SVKFrantisek RuzickaPermanent Rep. to the UN (NY)92.199.4 SVKPavol PavlisMin. of Economy92.498.4 DNKRasmus PrehnMin. for Development Cooperation94.999.7 DNKKirsten BrosbolMin. of Environment90.097.9 KORJu Chul-KiSenior Sec. for Foreign Affairs & Security88.398.4 KORLee Byung-HoDir., National Intelligence Service90.599.6 COLAlfonso Gomez MendezMin. of Justice & Law88.298.0 COLLuis Felipe Henao Cardona Min. of Housing & Territorial Dev.91.697.7 FINJan VapaavuoriMin. of Economic Affairs89.899.2 FINJari LindstromMin. of Justice & Employment94.099.3 SVNBostjan ZeksMin. w/o Portfolio (Slovenians Abroad)92.098.9 SVNGorazd ZmavcMin. w/o Portfolio (Slovenians Abroad)91.298.8 IRLAnne Colette AndersonPermanent Rep. to the UN (NY)90.799.7 IRLKatherine ZapponeMin. for Children & Youth Affairs88.497.8 Note: Agreement Rate indicates the percentage of claims where the auditor (Human or Exa) reached the same verification verdict (Supported/Unsupported) as the automated CGT pipeline. A-11 A4 Supplementary Results This section presents additional figures and analyses referenced in the main text to support key empirical claims. We focus on: (1) model performance without external resources, (2) comparison of agent-synthesized and Wiki-based corpora, (3) diagnostic checks of model heterogeneity under long-context constraints, (4) corpus composition and compression, (5) cross-national heterogeneity in retrieved corpora composition, and (6) granular mechanism plots illustrating recall dynamics and language effects. A4.1 Model performance without external resources Figure A4.1 illustrates that models exhibit poor performance on both precision and recall when operating without external resource access (e.g., without web search or retrieved documents). This three-panel comparison highlights the substantial performance gap between models with and without access to external information sources. Figure A4.1: Three-panel comparison showing poor model performance without external resources on both precision and recall. Models with access to external resources (web search, retrieved documents) significantly outperform those operating without such access. A4.2 Agentic Synthesis in a High-Curation Setting: China As a supplementary analysis, we examine agentic synthesis performance in the China setting, where the encyclopedic baseline is unusually strong. Unlike the U.S. and OECD samples discussed in the main text, Chinese political elites benefit from highly standardized and centrally curated biographical documentation, and Baidu Baike entries are typically comprehensive. As a result, any gains from synthesis are expected to be modest if the agent primarily reproduces already well-documented information. Figure A4.2 reports the comparison between agent-synthesized biographies and the Wiki long-context baseline in this setting. Consistent with expectations, the magnitude of improvements is substantially smaller than in the U.S. and OECD samples. The agent increases F1 by 2.7 percentage points and recall by 3.8 points, accompanied by a modest precision gain of 1.4 points. These results serve two purposes. First, they confirm that agentic synthesis does not degrade performance in environments where curated encyclopedic coverage is already strong. Second, the A-12 presence of small but detectable recall gains indicates that even in highly curated contexts, synthe- sis can recover incremental information omitted from baseline entries, such as minor concurrent appointments or short transitional roles. Together, the China results reinforce the interpretation of the main findings: the value of agentic synthesis scales with gaps in existing curation, yield- ing large gains where coverage is incomplete and converging toward parity where high-quality encyclopedic resources already exist. Figure A4.2: China setting: Comparison of agent-synthesized biographies and encyclopedic long- context baseline. A4.3 Model heterogeneity under long-context conditions We visualize performance disparities between coder models under uniform, long-context sce- narios. Figure A4.3 documents how both raw and refined pipelines (LLM_raw and LLM_refined) are affected. Figure A4.3: Performance differences across coder models (experiment-controlled) under long- context conditions, including both LLM_raw and LLM_refined variants. A-13 Figure A4.4: Token length and language composition for raw vs. refined corpora: refinement compresses token count while retaining high compositional correspondence. A4.4 Cross-national heterogeneity in retrieved corpora composition Figures A4.4, and A4.5 illustrate cross-national heterogeneity in retrieved corpora compo- sition. Figure A4.4 shows that refinement compresses token count while retaining high compo- sitional correspondence between raw and refined corpora. Figure A4.5 shows the percentage of non-English resources in each country’s retrieved corpus, illustrating substantial variation in lan- guage diversity across the OECD sample. Figure A4.5: Cross-national heterogeneity in retrieved corpora language composition (descriptive analysis). Each bar represents the percentage of non-English resources in the retrieved corpus for each country, illustrating substantial variation in language diversity across the OECD sample. Higher non-English shares are associated with lower extraction performance. A4.5 Disaggregated mechanism plots The following plots provide granular diagnostics on recall as a function of context length and language. Figure A4.6 shows the language composition effects on recall with country fixed effects, A-14 demonstrating that higher non-English token shares are associated with lower recall. Figure A4.7 shows the token length effects on recall accross models , demonstrating that higher token length are associated with lower recall. Figure A4.6: Language composition effects on recall (country fixed effects). Higher non-English token shares are associated with lower recall, indicating a quality channel in cross-national re- trieval. A5 Case Study: Erik Solheim Agent Run This case study details the complete agent execution process for retrieving and synthesizing information about Erik Solheim, former Norwegian Minister of the Environment (2007–2012) and Executive Director of UN Environment Programme (2016–2018). We illustrate how the Supervisor–Searcher architecture (Section A2) operates in practice for a Non-US political figure requiring multilingual evidence synthesis. A5.1 Agent Execution Overview The agent processed this case through 31 API calls across 3 systematic batches, conducting 12 web searches that returned 156 results, from which 14 documents were retrieved and archived. Table A5.1 summarizes the key metrics. A-15 Figure A4.7: Binned recall by context length, disaggregated by model. A5.2 Three-Phase Search Strategy The agent execution followed an iterative refinement pattern, transitioning from broad infor- mation gathering to targeted gap-filling: A5.2.1 Phase 1: Initial Skeleton Construction (Messages 0–7) Supervisor’s Goal: Create a comprehensive initial sweep prioritizing official Norwegian gov- ernment sources and Wikipedia to establish baseline biographical details. Searcher Execution: • Query: Erik Solheim AND (biografi OR miljøminister OR SV OR født) • Target Sites: no.wikipedia.org, regjeringen.no, stortinget.no • Language: Norwegian (NO) and English (EN) Key Evidence Retrieved: • no.wikipedia.org: “Erik Solheim (født 18. januar 1955)... Sjef for FNs miljøprogram... SV... MDG (2019–)” • regjeringen.no (Government CV): Confirmed unique combined portfolio of Minister of the Environment and International Development; SV leadership 1987–1997 • stortinget.no (Parliament): “Solheim, Erik (1955-)... Foreldre: Bjørn Elling Solheim og Sophie Grindstad... Cand.mag. 1980” A-16 Table A5.1: Erik Solheim Agent Run: Key Metrics MetricValue Total API Calls31 messages Total Input Tokens474,595 Total Output Tokens35,583 Total Tokens Used510,178 Web Searches12 searches (9 rounds) Search Results Returned156 results URLs Retrieved14 documents Retrieved Chunks18 chunks Processing Batches3 batches Final Report Length4,371 characters Archive Update: Identity confirmed (born 1955); baseline timeline established for SV lead- ership (1987–97), ministerial roles (2005–12), and UNEP (2016–18). Gaps identified: spouse names, early schooling details, and 1985–1987 career gap. A5.2.2 Phase 2: Gap Filling – Family and Mid-Career (Messages 8–20) Supervisor’s Goal: Target Phase 2 expansion for precise details on early education, spouse names, career gaps (specifically 1985–1987 at Norges Handikapforbund), and post-2018 positions. Searcher Execution: • Query 1: Erik Solheim AND (kone OR Sissel Bugge OR Gry Ulverud) • Query 2: Erik Solheim AND (Norges Handikapforbund OR konsulent) • Query 3: Erik Solheim AND (post-2018 OR MDG OR Green Belt) Key Evidence Retrieved: • snl.no (Store norske leksikon): “Gift 1) 1989 med Sissel Bugge... Gift 2) 2004 med Gry Høeg Ulverud... Konsulent i Norges Handikapforbund 1985–1987” • linkedin.com: Post-2018 roles including “President Europe-Asia Center (Jul 2023–)” and “Champion Global Renewables Alliance (Nov 2024–)” Archive Update: Validated marriages (Sissel Bugge 1989–1998, Gry Ulverud 2004–present) and children; confirmed Consultant role at Norges Handikapforbund (1985–87); mapped extensive post-2018 advisory roles. A5.2.3 Phase 3: Deep Dive – Early Life and Education (Messages 21–40) Supervisor’s Goal: Identify primary and secondary schools attended before Oslo katedral- skole; check memoirs or interviews for early life details in the Bislett area. Searcher Execution: A-17 • Query 1: Erik Solheim AND (skole OR Bolteløkka OR Ila OR Bislett) • Query 2: Erik Solheim AND (oppvekst OR barndom OR tidlig liv) • Language: Norwegian (NO) only Key Evidence Retrieved: • vg.no (Factbox): “...Oppvokst på Bislett, gikk på Bolteløkka skole, Ila skole og Oslo kate- dralskole” • nettavisen.no: “...Vokst opp: Bislett... Bor nå: St. Hanshaugen” Archive Update: Success (“needle in the haystack”); identified primary school Bolteløkka skole and lower secondary Ila skole. Timeline now continuous from childhood through 2025. A5.3 Source Diversity and Language Composition The agent successfully navigated multilingual evidence, retrieving documents across multiple source types: Table A5.2: Source Distribution for Erik Solheim Retrieval Source TypeCountExamples Government/Official6regjeringen.no CVs, stortinget.no Reference/Encyclopedia2no.wikipedia.org, snl.no News Media5vg.no, nettavisen.no Professional2linkedin.com, hydropower.org Other1oslobyleksikon.no, geni.com The search strategy demonstrates adaptive multilingual retrieval: initial queries combined Norwegian terms (“født”, “kone”, “skole”) with English disambiguation, prioritizing high-credibility Norwegian government sources while using English for cross-verification. This reflects the lan- guage composition patterns shown in Figure A4.5. A5.4 Retrieved Evidence and Archive Structure The agent archived 18 content chunks across 14 documents. Table A5.3 shows the archived entries with their evidence support categories under the FactNet protocol. A5.5 Ground Truth Comparison Table A5.4 presents the consolidated ground truth (CGT) biography entries and their match categories against the agent output. This comparison reveals both the strengths and limitations of the agentic retrieval process. A-18 Table A5.3: Case Study (Grok Candidates): Archived Entries and Evidence Support TypeCandidate EntrySupport Education1961.01–1969.12 | Bolteløkka skole | Primary schoolFULLY_SUPPORTED Education1969.01–1972.12 | Ila skole | Lower secondaryFULLY_SUPPORTED Education1970.01–1974.12 | Oslo katedralskole | High schoolFULLY_SUPPORTED Education1974.01–1980.12 | Universitetet i Oslo (UiO) | MasterFULLY_SUPPORTED Party1977.01–1997.05 | Sosialistisk Venstreparti (SV) | Member/LeaderFULLY_SUPPORTED Party1977.01–1980.12 | Sosialistisk Ungdom (SU) | LeaderFULLY_SUPPORTED Party1981.01–1985.12 | Sosialistisk Venstreparti (SV) | PartisekretærFULLY_SUPPORTED Party1987.01–1997.05 | Sosialistisk Venstreparti (SV) | Party LeaderFULLY_SUPPORTED Party2019.01–Present | Miljøpartiet De Grønne (MDG) | Member/AdvisorFULLY_SUPPORTED Career1985.01–1987.12 | Norges Handikapforbund | KonsulentFULLY_SUPPORTED Career1989.10–1993.09 | Stortinget | Stortingsrepresentant Sør-TrøndelagFULLY_SUPPORTED Career1993.10–2001.09 | Stortinget | Stortingsrepresentant OsloFULLY_SUPPORTED Career2000.03–2005.12 | Utenriksdepartementet (UD) | SpesialrådgiverFULLY_SUPPORTED Career2005.10–2007.10 | Utenriksdepartementet (UD) | UtviklingsministerFULLY_SUPPORTED Career2007.10–2012.03 | Miljøverndepartementet / UD | Miljøvernminister + UtviklingsministerFULLY_SUPPORTED Career2013.01–2016.12 | OECD | Chair DACFULLY_SUPPORTED Career2016.01–2018.11 | UN Environment Programme | Executive DirectorFULLY_SUPPORTED Career2017.01–Present | BRIGC / Green Belt and Road Institute | PresidentFULLY_SUPPORTED Career2019.01–2023.12 | APRIL / WRI / TREELION | AdvisorFULLY_SUPPORTED Career2023.07–Present | Europe-Asia Center | PresidentFULLY_SUPPORTED Career2024.11–Present | Global Renewables Alliance | ChampionFULLY_SUPPORTED RelativesfatherFULLY_SUPPORTED RelativesmotherFULLY_SUPPORTED Relativesex-spouseFULLY_SUPPORTED RelativesspouseFULLY_SUPPORTED RelativeschildFULLY_SUPPORTED A5.6 Key Insights and Analysis A5.6.1 Discovery Successes The agent demonstrated strong performance on several fronts: 1. Iterative Refinement: Successfully transitioned from broad queries (“biografi OR miljømin- ister”) to targeted searches (“Bolteløkka OR Ila”), demonstrating adaptive query reformula- tion. 2. Long-Tail Recovery: Recovered specific primary and secondary school names (Bolteløkka skole, Ila skole) that represent “needle in the haystack” information requiring precise Norwegian- language queries. 3. Cross-Source Synthesis: Integrated information across Wikipedia, government CVs, par- liamentary records, encyclopedia entries, and contemporary news sources to build a compre- hensive timeline. 4. Recent Activity Tracking: Successfully identified post-2018 positions including 2024 ap- pointments (Global Renewables Alliance) through LinkedIn and news sources. A5.6.2 Coverage Limitations The comparison with CGT reveals systematic gaps: A-19 Table A5.4: Case Study (Ground Truth): CGT Entries and Match Categories TypeCGT EntryMatch Education1961.01–1969.12 | Bolteløkka skole | Primary schoolFULL_MATCH Education1969.01–1972.12 | Ila skole | Lower secondaryFULL_MATCH EducationNA–1974.01 | Oslo Cathedral School | High schoolFULL_MATCH Education1975.01–1980.01 | University of Oslo | cand.mag.FULL_MATCH Party1977.01–1981.01 | Socialist Youth | LeaderFULL_MATCH Party1981.01–1985.01 | Socialist Left Party | Party SecretaryFULL_MATCH Party1985.01–1987.12 | Socialist Left Party | Central Exec.NO_MATCH Party1987.04–1997.05 | Socialist Left Party | Party LeaderPARTIAL_MATCH Party1989.10–2019.01 | Socialist Left Party | MemberPARTIAL_MATCH Party2019.01–Present | Green Party | MemberFULL_MATCH Career1974.01–1975.01 | Norwegian Air Force | ConscriptNO_MATCH Career1985.01–1987.12 | Norges Handikapforbund | ConsultantFULL_MATCH Career1989.10–2001.09 | Parliament of Norway | Member of ParliamentFULL_MATCH Career2000.03–2005.10 | Ministry of Foreign Affairs | Special AdviserFULL_MATCH Career2005.10–2012.03 | Government of Norway | Minister of International DevelopmentFULL_MATCH Career2007.10–2012.03 | Government of Norway | Minister of the EnvironmentFULL_MATCH Career2012.03–2013.01 | Ministry of Foreign Affairs | Special AdviserNO_MATCH Career2013.01–2016.06 | OECD | Chair of DACFULL_MATCH Career2016.06–2018.11 | UN Environment Programme | Executive DirectorPARTIAL_MATCH Career2018.11–Present | Belt and Road Green Development Coalition | Vice PresidentPARTIAL_MATCH Career2018.11–Present | Climate Council of Chief Minister MK Stalin | MemberNO_MATCH Career2018.11–Present | Global Solar Council | Global AmbassadorNO_MATCH Career2018.11–Present | Global Wind Energy Council | AdviserNO_MATCH Career2018.11–Present | Green Hydrogen Organization | ChairmanPARTIAL_MATCH Career2018.11–Present | International Hydropower Association | Board MemberNO_MATCH Career2019–Present | Green Belt and Road Institute | PresidentFULL_MATCH Career2019–Present | World Resources Institute | Senior AdviserFULL_MATCH Career2019.05–Present | Plastic REVolution Foundation | CEONO_MATCH RelativesfatherFULL_MATCH RelativesmotherFULL_MATCH Relativesformer spouseFULL_MATCH RelativesspouseFULL_MATCH RelativeschildFULL_MATCH 1. Weakly Connected Nodes: Several concurrent advisory roles (Global Solar Council, Global Wind Energy Council, International Hydropower Association) were missed, suggesting the agent did not exhaustively traverse all post-2018 organizational affiliations. 2. Minor Positions: Shorter-term roles (Norwegian Air Force conscript 1974–75, Special Ad- viser 2012–13, Plastic REVolution Foundation CEO) were not discovered, indicating chal- lenges with brief or less-documented career phases. 3. Granularity Gaps: Party membership continuity (1989–2019) was captured as a consoli- dated period rather than the granular breakdown in CGT, reflecting codebook representation choices. A5.6.3 Efficiency Analysis Token usage breakdown reveals the cost structure of agentic synthesis: The Searcher consumed 89% of total tokens, reflecting the computational cost of processing retrieved documents. The average of 14,534 input tokens per searcher call indicates substantial context accumulation across the multi-turn conversation. A-20 Table A5.5: Token Usage Breakdown by Component ComponentInputOutputTotal Searcher Agent421,48332,723454,206 Coder Agent53,1122,86055,972 Total474,59535,583510,178 A6 Prompts We list the exact prompt templates used in our pipeline, grouped by stage. Table A6.1: Prompt templates used in the research. StagePrompt ArchitectureSupervisor prompt ArchitectureSearcher prompt (Archive on) ExperimentQuery template (EN) ExperimentResearch plan template (EN) EvaluationFact-checking (related-content judge) prompt EvaluationEntrywise evaluation prompt A6.1 Architecture prompts A6.1.1 Supervisor prompt You are the Supervisor for a multi-step deep web research agent. You reason based on the structured state: - Research request (user query, constraints, codebook) - Search batch history (each batch_overview with supervisor_task_instruction, research_summary, detailed_analysis) - todo_list (remaining search gaps with [k] counters) - global_summary (running summary of findings so far) Each turn you must: 1) Update ‘global_summary‘ so it is a readable, self-contained summary of all solid facts found so far. 2) Update ‘todo_list‘ so it reflects the remaining important gaps. 3) Decide to either CONTINUE (delegate one focused next task) or FINISH (no more search). OUTPUT FORMAT (JSON ONLY, no extra text, no markdown fences): "todo_list": "...", "next_task_instruction": "... or null", "global_summary": "..." A-21 Field rules: - ‘global_summary‘: - Treat as the single evolving research summary. - Start from the previous global_summary, integrate new reliable facts from the latest batch_overview. - Keep it coherent and self-contained; someone reading only this should understand the main findings. - ‘todo_list‘: - Text block listing remaining gaps, typically as lines like ‘[k] <gap description>‘ (plus optional headings). - When a gap is fully answered, remove it. - When partially answered, rewrite to express only what is still missing. - When a gap was clearly targeted by the last Searcher task and remains unresolved, increment its k (e.g. ‘[1]‘->‘[2]‘->‘[3]‘). - If k would exceed 3, keep the gap for transparency but do NOT target it again with new tasks. - ‘next_task_instruction‘: - Non-empty string => CONTINUE mode. - null => FINISH mode. - Must be a single, focused, self-contained instruction for the Searcher: * Briefly restate the overall goal. * Clearly state WHAT new information is needed (never HOW to search; no tool names or keyword syntax). CONTINUE mode (non-empty ‘next_task_instruction‘): - Use when there are still important gaps in todo_list that are plausibly answerable by web research (prefer k = 1 or 2). - Decompose broad gaps into concrete questions when possible (e.g. "exact dates for role X" instead of "complete career history"). - Focus each instruction on 1 main sub-task (or 1-2 very closely related gaps). FINISH mode (‘next_task_instruction‘ = null): - Use when remaining gaps are minor, low-value, or have high counters (>3), or the user’s request is sufficiently answered. - In this case, produce a comprehensive final_report based on global_summary and batch history: * Summarize all the information that was found as detailed as possible, include the source of the information. * Note any major remaining uncertainties or unsolved gaps. * Make it self-contained and directly address the original research request. Today is current_date. A-22 A6.1.2 Searcher prompt You are a professional Search Agent executing a research task to search, browse, and retrieve as broad relevant information as possible. You are capable of creatively and strategically design keywords to search for related and diverse information. The final goal is to complete the task and handoff to the supervisor with a comprehensive research_summary, and archive every relevant piece of information found during the process. ### Understand the Task - You receive a **self-contained task instruction** from the Supervisor that includes: - The overall research goal - A summary of what has been found so far - The specific objective for this search batch - Any relevant constraints - Read the provided ‘current_task_instruction‘ carefully - The instruction should contain all context you need (goal, prior findings, current objective) - Focus on the **specific objective** stated in the instruction ## Your Core Action Loop You search, retrieve, and archive to complete the task: 1. Search web for relevant information, Retrieve for detailed review, Archive relevant information. 2. Handoff to the supervisor if collected enough information. ### Execute Search - Call ‘web_search(search_intent=...)‘ with a structured search plan - ‘any_of‘ means at least one of the terms in the list should appear in results. - ‘must_include‘ means all of the terms in the list must appear in results. - ‘must_not_include‘ means none of the terms in the list may appear in results. - Start broad, then narrow based on results - Adjust the terms in ‘must_include‘ and ‘any_of‘ to make the search more specific or more broad based on observed results. - Avoid overly restrictive ‘must_include‘ terms - Mention generic meta-words like biography, bio, profile in ‘any_of‘ instead of ‘must_include‘ - Only use site restrictions when REALLY necessary - Flexibly use keywords in different languages as appropriate - You have *max_search_attempts* search attempts, use wisely. ### Retrieve URLs Content for Browsing A-23 - After each ‘web_search‘ call, call ‘retrieve_documents(urls=[...])‘ for the **promising** URLs from the latest results. - Select up to 10 promising URLs per retrieve call. - Skip retrieving if no results appear relevant. ### Archive Relevant Documents - Archived information will be reviewed by the supervisor for reference - For each relevant document found during browsing, call ‘archive_document(detailed_analysis=[...])‘: - ‘url‘: Document URL - ‘title‘: Document title - ‘task_summary‘: Summary of how this document addresses the task - ‘relevant_chunk_labels‘: List of chunk labels for relevant paragraphs (e.g., ["[CHUNK:abc12345:001]", "[CHUNK:abc12345:002]"]) - Archive every piece of information that is relevant to the task. - Should have archived all relevant documents by the time you handoff. ## Handoff to Supervisor When the task is complete, call ‘handoff_to_supervisor_with_overview‘: - ‘research_summary‘: Comprehensive narrative including: - **What was found**: Specific information with concrete details - **What is lacking**: Information not found or uncertain - ‘search_intent_summary‘: Feedback on search effectiveness: - ‘bad_must_include‘: Terms that performed poorly - ‘good_any_of‘: Terms that worked well - ‘search_languages‘: Languages used in searches ## Tools (USE ONLY THESE) - web_search(search_intent: object) - execute search - retrieve_documents(urls: list[string]) - fetch and chunk document content from URLs - archive_document(detailed_analysis: list[object]) - archive every relevant chunk found during browsing to storage for future reference - handoff_to_supervisor_with_overview(research_summary: string, search_intent_summary: object) - final handoff ## Important: # Maintain loops of search, retrieve, and archive to complete the task incrementally. # Handoff when the task is complete. # Reflect and reason with the context, accompanied with each tool call, affix a brief reflection paragraph. ## Context Today is current_date. A-24 A6.2 Experiment prompts A6.2.1 Query template (EN) Find comprehensive public information about current_name, a political or public figurecountry_clauseoccupation_clauseyear_clause. REQUIRED INFORMATION: - Basic biographical details: birth year, place of birth (province/state, city/county), gender - Party affiliation history with year ranges, if applicable - For each party affiliation: year range, party name, position title (if any) - Education history (primary, secondary, tertiary, and post-secondary) and highest education attainment - For each education entry: year range, organization name, education level (e.g., Below high school/High school/Bachelor/Master/Doctorate/Diploma/Certificate), major/field - Occupation/career timeline with organizations, positions, and year ranges - For each role: year range, organization name, position title, employed/unemployed - Family/relatives (if available): relation (spouse/grandparents/parents/children/siblings) and name only - Death status and year range, if applicable - If there is no definitive information on death, assume the individual is still alive. SEARCH REQUIREMENTS: - Confirm all information is about current_nameoccupation_clause_short - Summarize in English; prioritize official government sources, newsletter, pedia, organization and personal websites - Use strategic keyword variations; capture precise year ranges to build a detailed chronological position list - wiki pages are not available due to technical reasons, so it’s not strange if searcher returns no urls for wiki pages. QUALITY REQUIREMENTS: - Ensure objectivity, completeness, and accuracy - Politicians may have multiple roles in different careers/fields/positions, which should be filled as ’Concurrent’. - Present a clear, chronological timeline that integrates both education and full career history. Diligently identify and fill any gaps, especially throughout the typical workforce age (18-65), ensuring minimal periods of missing information. - Career together with education history should be completely filled, with no gaps (unemployed years should be filled as ’Unemployed’). A-25 OUTPUT FORMAT: - Include a comprehensive narrative biography (>=600 characters) integrating all details. - Include the source of the information, credible or not, ensure reproducibility. A6.2.2 Research plan template (EN) # Phase 1: Comprehensive Initial Sweep 1. Execute broad searches for "current_name" to gather a holistic view: basic biographical details (birth/death, family), main career milestones, education, and political affiliations simultaneously. 2. Construct an initial timeline skeleton from the broad results, capturing all immediately available years, roles, and organizations. 3. Identify unique identifiers (e.g., specific keywords, middle names, known associations) to disambiguate from homonyms. # Phase 2: Targeted Expansion & Detail Enrichment 1. Leverage specific entities found in Phase 1 (e.g., "Party X", "University Y", "Ministry Z") to perform targeted searches for precise dates, specific position titles, and missing details. 2. Specifically expand on known entities to get granular details: - Education: Verify degrees, majors, and institutions. - Party History: Clarify roles and affiliation periods. - Career: Flesh out concurrent roles and specific job titles using organization-specific keywords. # Phase 3: Gap Analysis & Narrative Synthesis 1. Analyze the timeline for chronological gaps (especially within age 18-65). Perform specific queries to fill these gaps (e.g., check for private sector work or unlisted periods). 2. Re-verify any ambiguous data points (e.g., relatives, death date if unclear) and finalize the dataset. 3. Synthesize all verified data into a cohesive narrative biography (>=600 characters). A6.3 Evaluation prompts A6.3.1 Fact-checking (related-content judge) prompt You are a careful fact-checking assistant. Your task is to evaluate **one biographical fact** about a person using ONLY the A-26 provided related content (snippets aggregated from multiple URLs). Person identifier: official_id Person name: official_name Biographical fact to check: ‘text entry ‘ Related content (this is your ONLY evidence source; do not use outside knowledge): ‘text related_content ‘ Instructions: - Decide whether the fact is fully supported, partially supported, unclear, or contradicted by the related content. - Treat faithful translations between languages as equivalent evidence. Output JSON with exactly these fields: - entry_text: the original fact text (string) - verdict: 1-5 from not true to totally true, no option as unsure - rationale: 1-3 sentences explaining your verdict, citing key phrases from the content (but do NOT invent new facts). Do NOT include any commentary outside the JSON object. A6.3.2 Entrywise evaluation prompt ## Task: Entrywise Biography Evaluation You are an expert evaluator of biographical data extraction quality. Your task is to perform a detailed, entry-by-entry evaluation comparing a **candidate biography** against a **CGT (Consolidated Ground Truth) biography**. --- ## Person Information - **Official ID**: official_id A-27 - **Official Name**: official_name - **Experiment Type**: experiment_type --- ## Core Evaluation Principle: Content Accuracy Over Structure This is the most important guiding principle: - When there are structural differences (e.g., CGT has one merged entry vs candidate has multiple split entries), prioritize judging whether the **total information content** is equivalent. - If multiple candidate entries together accurately express the information in one CGT entry, this should be scored as a strong match (8-10). - Do NOT penalize for splitting/merging differences alone; only penalize for actual information gaps or conflicts. --- ## Scoring Rubric (1-5 Scale) ### For CGT Entry Evaluation (How well is each CGT fact captured by the candidate?) | Score | Category | Description | |-------|----------|-------------| | **5** | FULL_MATCH | Perfect or near-perfect match. All key details (time, organization, position) are correct; only trivial wording differences allowed. | | **4** | PARTIAL_MATCH | Good match with small gaps or simplifications (e.g., missing end date, simplified organization name) but the core fact is accurate. | | **3** | PARTIAL_MATCH | Partial match. The same event is referenced but with significant gaps or minor errors. | | **2** | WEAK_MATCH | Very weak/unclear match. Only loosely related content; most details are missing or wrong. | | **1** | NO_MATCH | No match at all. The CGT fact is completely absent from the candidate biography. | ### For Candidate Entry Evaluation (How well is each candidate fact supported by CGT?) | Score | Category | Description | |-------|----------|-------------| A-28 | **5** | FULLY_SUPPORTED | Fully or almost fully supported by CGT. Clear matching CGT entry with at most trivial differences. | | **4** | PARTIALLY_SUPPORTED | Mostly supported. Core fact is in CGT, with small additions or wording differences. | | **3** | PARTIALLY_SUPPORTED | Partially supported. Related CGT entry exists but there are notable differences or missing details. | | **2** | WEAKLY_SUPPORTED | Weakly supported. Only loosely related CGT content; candidate may contain errors. | | **1** | NOT_SUPPORTED | No support (hallucination). This candidate entry has no real basis in the CGT. | --- ## Difference Codes (for CGT evaluations with score < 5) When a CGT entry is not perfectly matched, select applicable codes from: | Code | Meaning | |------|---------| | ‘TIME_YEAR‘ | Year is incorrect | | ‘TIME_MISSING‘ | Time information is missing from candidate | | ‘ORG_WRONG‘ | Organization name is incorrect (not just abbreviation) | | ‘POSITION_WRONG‘ | Position/title is incorrect | | ‘POSITION_INCOMPLETE‘ | Missing concurrent positions or partial title | | ‘EXTRA_INFO‘ | Candidate has extra information not in CGT (neutral/positive) | --- ## Flexible Alignment Rules ### 1-to-N Matching (CGT merged entry vs Candidate split entries) - If the candidate splits one CGT entry into multiple lines, list all matching candidate entries separated by " || " in the ‘matched_candidate_entries‘ field. - Score based on whether the combined information is complete and accurate. ### N-to-1 Matching (Multiple CGT entries vs one Candidate entry) - In candidate evaluation, reference multiple CGT entries like "CGT#3,#4,#5". - This is acceptable if the candidate correctly aggregates the information. ### Semantic Equivalence - Different phrasings of the same fact should match (e.g., "Mayor" = "City Mayor"). - Abbreviations vs full names are acceptable (e.g., "EPA" = "Environmental Protection Agency"). - Cross-language translations are equivalent if semantically the same. A-29 --- ## Important Notes 1. **Be consistent**: Apply the same standards across all entries. 2. **Section tags**: Lines like "[party]", "[occupation]", "[education]", "[relatives]" are structural markers, not facts. Skip them when counting entries. 3. **Empty lines**: Ignore empty lines when counting and evaluating. 4. current date is 2025-11-25 --- ## Input Data ### CGT BIOGRAPHY (Ground Truth): ‘text cgt_biography ‘ ### CANDIDATE BIOGRAPHY (Experiment: experiment_type): ‘text experiment_biography ‘ --- ## Output Format Produce a JSON object with exactly these fields: - ‘official_id‘: string (copy from input: "official_id") - ‘official_name‘: string (copy from input: "official_name") - ‘experiment_type‘: string (copy from input: "experiment_type") - ‘cgt_entry_count‘: integer (number of non-empty, non-tag lines in CGT) - ‘candidate_entry_count‘: integer (number of non-empty, non-tag lines in candidate) - ‘cgt_evaluations‘: array of objects, one per CGT entry, each with: - ‘index‘: integer (1-based) - ‘cgt_entry_text‘: string (the CGT line) - ‘matched_candidate_entries‘: string (matching candidate text or "NO_MATCH") - ‘match_score‘: integer (1-5) - ‘match_category‘: string ("FULL_MATCH", "PARTIAL_MATCH", "WEAK_MATCH", or "NO_MATCH") - ‘difference_codes‘: array of strings (codes from the table above, or empty) A-30 - ‘notes‘: string (brief explanation) - ‘candidate_evaluations‘: array of objects, one per candidate entry, each with: - ‘index‘: integer (1-based) - ‘candidate_entry_text‘: string (the candidate line) - ‘matched_cgt_entries‘: string (e.g., "CGT#3" or "CGT#1,#2" or "NO_SUPPORT") - ‘support_score‘: integer (1-5) - ‘support_category‘: string ("FULLY_SUPPORTED", "PARTIALLY_SUPPORTED", "WEAKLY_SUPPORTED", or "NOT_SUPPORTED") - ‘notes‘: string (brief explanation) - ‘qualitative_summary‘: string (2-4 sentences on overall quality) Do not include markdown fences or any text outside the JSON object. A7 A Practical Guide to Information Extraction with LLMs This appendix provides a practical guide for applying Large Language Models (LLMs) to information extraction tasks in the social sciences. The guiding principle is to treat extraction as an end-to-end data-production task and, when necessary, to separate it into two modular stages: synthesis (evidence acquisition and refinement) and coding (mapping a refined corpus into a struc- tured codebook). This modular design improves auditability and helps us diagnose whether errors arise from missing evidence (a synthesis failure) or incorrect mapping (a coding failure). A7.1 A minimal workflow for reliable extraction Step 0 (define the record and the evidence rule). Extraction is only well-defined relative to a codebook. Before using an LLM, we specify (i) a field-level codebook (variables, types, allowed formats), (i) normalization rules (names, organizations, dates), and (i) a groundedness require- ment (what constitutes sufficient evidence for a claim). In political fact extraction, a small number of ambiguous fields (e.g., office titles, start/end dates) can drive large downstream measurement error, so explicit formatting and disambiguation rules are essential. Step 1 (diagnose whether synthesis is necessary). The most important practical decision is whether the available sources are effectively curated or open-ended and noisy. When a short, high-signal source exists (e.g., Wikipedia, an official CV, a curated archive), we can often run coding-only: we provide the curated text and ask the model to output the structured record. 22 When relevant evidence is dispersed across many documents (e.g., the open web) or the total context exceeds any fixed window, synthesis is necessary: we must decide which sources to read and how to condense them into a signal-dense representation before coding can be valid. 22 Even in curated settings, long contexts can degrade reliability when relevant facts are buried deep in lengthy inputs (Liu et al. 2024). A-31 Step 2 (implement coding with constraints and groundedness). The coding stage maps a fixed input corpus into a structured record. In practice, we recommend three safeguards: constrain outputs to be strictly codebook-conformant (e.g., JSON with fixed keys and date formats); require evidence pointers (quotes/snippets) for each claim to reduce hallucination risk (Mallen et al. 2023); and, when feasible, separate generation from validation (a second pass or second model that checks codebook compliance and evidence support). Modern LLMs can often perform this stage in a zero- shot or few-shot manner when the input is curated and the codebook is explicit (Ornstein et al. 2025; Ziems et al. 2024). Step 3 (implement synthesis as bounded, credibility-aware refinement). Synthesis is an evidence- refinement process: retrieve, filter, cross-check, and compress information into a corpus that is feasible for coding. While there are multiple valid implementations (keyword search, embedding retrieval, human-in-the-loop), open-ended political fact extraction often requires an interactive process because early discoveries change what should be searched next. Agentic workflows oper- ationalize this by alternating between reasoning and tool use (ReAct) (Yao et al. 2023), enabling adaptive query refinement and iterative evidence accumulation. Operationally, we recommend explicit retrieval budgets (steps/tokens/sources), credibility-aware filtering (prioritize authoritative sources; deduplicate near-identical content), and compression with traceability (store a refined corpus plus source-linked snippets so claims remain auditable). Step 4 (evaluate and interpret trade-offs). For extraction, both false positives and false neg- atives are substantively costly. Precision captures whether extracted claims are correct; recall captures whether the system recovers relevant claims; and F1 summarizes the trade-off. When precision is low, the coding stage is often hallucinating or mis-mapping (tighten groundedness and codebook constraints; improve synthesis filtering). When recall is low, the system is missing evidence (increase synthesis coverage or improve the refined representation). A-32

Instant research discovery