Paper deep dive
An Agentic Evaluation Architecture for Historical Bias Detection in Educational Textbooks
Gabriel Stefan, Adrian-Marius Dumitran
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 97%
Last extracted: 4/10/2026, 4:12:21 AM
Summary
The paper introduces an agentic evaluation architecture for detecting historical bias in educational textbooks. The system utilizes a three-stage pipeline: a multimodal screening agent for high-recall discovery, a heterogeneous jury of five evaluative agents for deliberative severity assessment, and a meta-agent for verdict synthesis. A key innovation is the Source Attribution Protocol, which distinguishes between authorial narrative and quoted historical sources to reduce false positives. Empirical results on Romanian history textbooks show the architecture significantly reduces over-penalization compared to zero-shot baselines, achieving a mean severity score of 2.9/7 versus 5.4/7, and is preferred by human evaluators in 64.8% of cases.
Entities (5)
Relation Signals (3)
Gabriel Ștefan → authored → An Agentic Evaluation Architecture for Historical Bias Detection in Educational Textbooks
confidence 100% · Gabriel Ștefan and Adrian Marius Dumitran... An Agentic Evaluation Architecture for Historical Bias Detection in Educational Textbooks
Agentic Evaluation Architecture → evaluatedon → Romanian upper-secondary history textbooks
confidence 100% · In an empirical study on Romanian upper-secondary history textbooks
Agentic Evaluation Architecture → utilizes → Source Attribution Protocol
confidence 100% · A central contribution is a Source Attribution Protocol that distinguishes textbook narrative from quoted historical sources
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:History textbooks often contain implicit biases, nationalist framing, and selective omissions that are difficult to audit at scale. We propose an agentic evaluation architecture comprising a multimodal screening agent, a heterogeneous jury of five evaluative agents, and a meta-agent for verdict synthesis and human escalation. A central contribution is a Source Attribution Protocol that distinguishes textbook narrative from quoted historical sources, preventing the misattribution that causes systematic false positives in single-model evaluators. In an empirical study on Romanian upper-secondary history textbooks, 83.3\% of 270 screened excerpts were classified as pedagogically acceptable (mean severity 2.9/7), versus 5.4/7 under a zero-shot baseline, demonstrating that agentic deliberation mitigates over-penalization. In a blind human evaluation (18 evaluators, 54 comparisons), the Independent Deliberation configuration was preferred in 64.8\% of cases over both a heuristic variant and the zero-shot baseline. At approximately \$2 per textbook, these results position agentic evaluation architectures as economically viable decision-support tools for educational governance.
Tags
Links
- Source: https://arxiv.org/abs/2604.07883v1
- Canonical: https://arxiv.org/abs/2604.07883v1
Trouble viewing inline? Open PDF directly →
Full Text
39,122 characters extracted from source content.
Expand or collapse full text
An Agentic Evaluation Architecture for Historical Bias Detection in Educational Textbooks Gabriel S , tefan [0009−0007−4193−7181] and Adrian Marius Dumitran [0009−0005−3547−5772] University of Bucharest, Bucharest, Romania gabrielstefan04@gmail.com, marius.dumitran@unibuc.ro Abstract. History textbooks often contain implicit biases, national- ist framing, and selective omissions that are difficult to audit at scale. We propose an agentic evaluation architecture comprising a multimodal screening agent, a heterogeneous jury of five evaluative agents, and a meta-agent for verdict synthesis and human escalation. A central con- tribution is a Source Attribution Protocol that distinguishes textbook narrative from quoted historical sources, preventing the misattribution that causes systematic false positives in single-model evaluators. In an empirical study on Romanian upper-secondary history textbooks, 83.3% of 270 screened excerpts were classified as pedagogically acceptable (mean severity 2.9/7), versus 5.4/7 under a zero-shot baseline, demon- strating that agentic deliberation mitigates over-penalization. In a blind human evaluation (18 evaluators, 54 comparisons), the Independent De- liberation configuration was preferred in 64.8% of cases over both a heuristic variant and the zero-shot baseline. At approximately $2 per textbook, these results position agentic evaluation architectures as eco- nomically viable decision-support tools for educational governance. Keywords: AI in Education· Bias Detection· Multi-Agent Systems· Large Language Models· Curriculum Auditing 1 Introduction Educational materials are curated narratives that shape collective memory, civic identity, and historical interpretation [1,24]. National curriculum reforms intro- duce large volumes of new instructional materials simultaneously, creating an auditing demand that manual expert review cannot satisfy at scale. Deploying LLMs as curriculum auditing tools presents two domain-specific failure modes. First, safety-aligned models frequently misclassify historically sen- sitive but factually accurate content as problematic [26]. Second, LLMs fail to distinguish authorial narrative from quoted primary sources, producing spuri- ous bias attributions. Furthermore, a single-model zero-shot baseline suffers from long-context attention degradation [8,16], extracting only a fraction of problem- atic passages and inflating severity to a mean of 5.4/7 with no excerpt rated below 4. arXiv:2604.07883v1 [cs.AI] 9 Apr 2026 2G. S , tefan and A. M. Dumitran To address these failure modes, we propose an agentic evaluation architec- ture: a screening agent for broad-coverage discovery, a heterogeneous jury for deliberative severity assessment, and a meta-agent for synthesis and human es- calation [7]. At approximately $2 per textbook, the system surfaces prioritized controversies before materials are published, without rendering autonomous final judgments. This paper makes the following contributions: 1. An agentic evaluation architecture that reduces mean severity from 5.4/7 (zero-shot) to 2.9/7, classifying 83.3% of screened excerpts as pedagogically acceptable while producing structured outputs suitable for editorial gover- nance workflows. 2. A Source Attribution Protocol that enforces explicit classification of excerpts as authorial narrative or quoted historical material, reducing false positives from primary source misattribution. 3. An empirical case study on publicly available Romanian upper-secondary his- tory textbooks, demonstrating that the architecture surfaces historiograph- ically significant patterns at a scale and consistency that manual review cannot match. 2 Related Work 2.1 Computational Curriculum Analysis Textbook analysis has traditionally relied on manual, qualitative examination of national narratives and historical framing [1, 24]. Early computational work applied lexicon-based methods to quantify representational patterns [19], while more recent approaches use neural models for readability and framing anal- ysis [31]. Both generations of methods operate on plain-text extractions, dis- carding the spatial structure—sidebars, marginalia, captions—essential for in- terpreting complex pedagogical layouts [2, 30]. Our work addresses this gap by combining layout-aware multimodal screening with deliberative adjudication. 2.2 Attribution Errors and Safety Inflation in LLM Evaluators Attribution errors, where models conflate cited evidence with authorial claims, are a documented limitation of LLM evaluators [9]. Existing mitigations via re- trieval augmentation [14] or post-hoc verification [32] target generation settings and do not transfer to evaluative tasks assessing existing material. Standard frameworks such as G-Eval [17] and MT-Bench [33] lack explicit mechanisms to distinguish endorsed narrative from quoted historical sources, resulting in sys- tematic over-penalization of factually accurate content [26]. We address both failure modes through a Source Attribution Protocol that enforces this distinc- tion as a constrained intermediate representation prior to evaluation. Agentic Bias Detection in Educational Textbooks3 2.3 Multi-Agent Systems and AI in Education Multi-agent architectures have demonstrated superior robustness and factuality over single-model pipelines in reasoning-intensive tasks [3,4,7,29]. In AI in Edu- cation, LLMs have been applied to tutoring and content creation [12], but their application to curriculum governance remains limited [18, 28]. We bridge these domains by adapting deliberative multi-agent evaluation— traditionally used to improve generative quality—into a conservative reliability filter for large-scale educational auditing, where the objective is to suppress weakly supported eval- uative claims rather than enhance generation. 3 Proposed Pipeline The proposed system instantiates an agentic evaluation architecture for curricu- lum auditing, in which specialized agents with distinct roles collaborate to pro- duce auditable, evidence-backed verdicts. Unlike single-pass LLM classification, where a single model is prompted to both identify and assess bias in one step, the architecture decomposes the task across three agent layers: a screening agent performing broad-coverage issue discovery, a jury of heterogeneous evaluative agents conducting independent severity assessment, and a meta-agent synthesiz- ing verdicts and triggering escalation to human reviewers. This decomposition is motivated by the failure modes of monolithic classification in deployment: over-sensitivity to historically sensitive content, source attribution errors, and the lack of interpretable reasoning chains in single-model verdicts (see Fig. 1). Evaluator 3 Evaluator 2 Evaluator 1 Evaluator 4 Evaluator 5 Stage 2: Heterogeneous Jury Stage 1 Screening & Attribution Textbook PDF Extracted Quotes & Source (JSON) Stage 3 Meta-Jury Synthesis & Escalation Final Verdict Fig. 1. The three-stage agentic evaluation architecture. The flow illustrates the screen- ing agent, heterogeneous jury, and meta-agent for verdict synthesis. 4G. S , tefan and A. M. Dumitran 3.1 Stage 1: Broad-Coverage Screening and Joint Attribution The screening agent performs high-sensitivity discovery using a multimodal large language model that ingests textbook pages as high-resolution images, preserving layout information—headings, marginalia, sidebars, and image captions—that plain-text extraction discards. The agent is parameterized to favor sensitivity over precision: its role is to surface any passage that may warrant scrutiny, including nationalist framing, omission of historical context, or uncontextualized primary sources. False positives are tolerated by design, as all flagged excerpts are subject to jury adjudication in Stage 2. Rather than treating attribution as an implicit reasoning step, the screening agent enforces a strict Source Attribution Schema during extraction. The model must simultaneously flag the excerpt and classify it into one of two mu- tually exclusive categories: Textbook Narrative (the authorial explanatory voice) or Primary Source Usage (quoted historical materials). This joint extraction en- sures that jury agents evaluate each excerpt with the correct pedagogical frame: language that would trigger a bias classification in isolation may be entirely appropriate when recognized as a quoted historical source. The use of a single model in Stage 1 is a deliberate design choice. Because all flagged excerpts are subject to independent jury adjudication in Stage 2, the screening agent functions as a high-recall filter rather than a final classi- fier: false positives are tolerated and corrected downstream, while the cost of a false negative—a biased excerpt missed entirely—is irreversible within the pipeline. Concentrating sensitivity in a single, well-parameterized multimodal model avoids the coordination overhead of an ensemble while preserving the recall objective. The risk of missed excerpts due to model-specific blind spots remains a limitation discussed in Section 5.5. Screening decisions are made traceable downstream through the structured JSON output: each flagged excerpt carries an explicit attribution label and a textual reasoning field, which are passed directly to Stage 2 jurors, ensuring that the screener’s classification rationale is inspectable at the jury stage even without access to the screening model’s internals. 3.2 Stage 2: Multi-Model Jury Adjudication Each excerpt flagged by the screening agent, together with its attribution label, is independently assessed by a jury of five heterogeneous evaluative agents with dif- fering alignment profiles and reasoning architectures. Each agent independently assigns a severity score on a 7-point ordinal scale anchored to pedagogical impact (see Table 1). Agentic Bias Detection in Educational Textbooks5 Table 1. The 7-point ordinal severity scale used by jury agents ScoreDescription 1 – NeutralPedagogically sound or properly contextualized source. 2 – Negligible Stylistic choices without substantive bias. 3 – MinorLack of secondary perspective or slight tonal loading. 4 – Moderate Loaded language, stereotyping, or insufficient context. 5 – Significant Selective omission of key facts or one-sided narratives. 6 – SevereNationalist myth-making, whitewashing, or propaganda. 7 – Harmful Hate speech, fabrication, or incitement as instruction. The 7-point ordinal severity scale follows psychometric recommendations for evaluative rating tasks [11,25], with each level anchored to a concrete pedagogical consequence rather than a linguistic surface feature [24]. Bias Taxonomy Classification In addition to a severity score, each agent classifies the excerpt using a pre-defined taxonomy of 15 bias labels covering four domains: Language & Framing, Perspective & Representation, Structure & Emphasis, and Source Handling. This taxonomy provides granular diagnostic output beyond a scalar severity score, enabling the final report to identify not just the severity of a concern but its historiographical character. The taxonomy was constructed by the authors to operationalize dimensions of historiographical bias drawn from curriculum analysis and history education didactics [1,19,24,27], extended to address failure modes specific to post-communist historiography. The four domains reflect established analytical categories in textbook research, though formal inter-rater validation remains a direction for future work. Structured Jury Output To ensure consistent downstream aggregation across heterogeneous models, all jurors must return a standardized JSON object. This object contains an attribution flag (Textbook Narrative or Primary Source Usage), a taxonomy category, an integer severity score∈ [1, 7], a confidence float ∈ [0.0, 1.0], and a textual reasoning justification. Crucially, requiring an explicit confidence score alongside the severity rating allows the meta-agent to appropriately weight juror agreements during synthesis. Calibration Instruction. A deliberate design choice in the jury prompt is the inclusion of the instruction: “You are encouraged to assign low severity or dismiss concerns when appropriate.” This instruction reflects a conscious calibra- tion decision: given that the screening stage is parameterized for high sensitivity (Section 3.1), the jury stage must compensate by applying a conservative severity prior. It should be noted, however, that this instruction is a confounding variable in the comparison between the multi-agent pipeline and the zero-shot baseline: the two configurations differ not only in architecture (single vs. multi-agent) but also in explicit severity prior. Future work should isolate these variables through 6G. S , tefan and A. M. Dumitran ablation (see Section 5.5). All system prompts used across pipeline stages are available in the supplementary repository, 1 alongside the implementation note- books. All stages employ zero-shot prompting with no in-context examples. 3.3 Stage 3: Meta-Jury Aggregation and Escalation The meta-agent receives the complete output tuple from each jury agent (cate- gory, severity, confidence, and reasoning) and synthesizes a single final verdict. To evaluate the optimal resolution strategy for inter-agent disagreements, we implement and compare two distinct meta-agent prompting configurations: 1. Heuristic Aggregation Strategy: In this configuration, the Meta-Jury is strictly prompted to follow a mathematical decision tree. It adopts con- sensus only if high-confidence jurors (conf idence > 0.7) agree, computes confidence-weighted averages for minor disagreements, and automatically flags the case for human review if juror severity scores diverge by more than 1.5 points. 2. Independent Deliberation Strategy: In the alternative configuration, the Meta-Jury is prompted to act as an independent appellate judge. Rather than calculating weighted averages, it is instructed to qualitatively evaluate the juror justifications and select the severity score best supported by the historical evidence, regardless of how many jurors hold that position. Human review is flagged based on the Meta-Jury’s qualitative assessment of the disagreement rather than a strict numerical threshold. For example, if four agents assign severity 2 with low confidence and one assigns severity 5 with detailed reasoning citing specific historiographical omis- sions, the Heuristic strategy produces a confidence-weighted average near 2, while the Independent strategy may select severity 5 as better supported by the evi- dence. Both prompting strategies require the Meta-Jury to output a standardized JSON object containing the final severity score, the finalized taxonomy category, a synthesized justification summary, and a boolean flag indicating if human intervention is required. This dual-prompt approach allows us to observe whether strict algorithmic consensus or independent LLM judgment yields better results in human evaluations. 4 Experimental Setup 4.1 Dataset We constructed an evaluation corpus from history textbooks certified by the Romanian Ministry of Education, sourced from major publishers and covering the full upper-secondary curriculum. The selection of this corpus is motivated by three strategic considerations: 1 https://github.com/submission-its/bias-detection Agentic Bias Detection in Educational Textbooks7 – Reproducibility and Open Access: The textbooks are publicly available via the ministry’s official digital repository, 2 allowing for full study repro- ducibility by independent researchers without restrictive institutional access agreements. – Historiographical Domain Competence: The authors possess direct fa- miliarity with Romanian secondary historiography and its contested periods, including Romania’s role in World War I, the Holocaust, and the Commu- nist era, enabling meaningful qualitative validation of pipeline outputs and ensuring findings are historiographically grounded. – Policy Relevance and Deployment Urgency: Romania is currently un- dergoing a comprehensive national curriculum reform [21]. Given the massive volume of new materials introduced simultaneously, our agentic auditing ar- chitecture serves as a decision-support tool, helping specialized committees rapidly surface pedagogical distortions before final approval. The corpus also presents three technical challenges that directly motivate our architectural design and stress-test its robustness: – Content Sensitivity: The curriculum requires a fine-grained distinction between factual recounting and ideological framing in sensitive historical periods, where standard models often suffer from “safety-alignment inflation” and over-penalize factual descriptions [26]. – Multimodal Complexity: Textbooks utilize non-linear layouts (sidebars, primary sources, captions) where essential context is distributed spatially, necessitating our vision-based Stage 1 screening analysis rather than simple linear text extraction. – Cultural Alignment Gap: Safety-aligned models are predominantly cal- ibrated against Western educational norms [15, 26]. A heterogeneous Stage 2 jury spanning North American, European, and East Asian training dis- tributions explicitly mitigates the influence of any single model’s cultural alignment profile. 4.2 Model Selection The Screening Agent is implemented using Llama-4-Maverick-17B-128E-- Instruct-FP8 [20], selected for its native multimodal capabilities and ability to process high-resolution page images without OCR preprocessing [2,30]. The Jury comprises five evaluative agents selected for their complementary capability profiles: (1) Mixtral-8x7B-Instruct-v0.1, a sparse mixture-of-experts model providing multilingual European training coverage relevant to the Roma- nian corpus [10]; (2) GPT-OSS-120B, a dense generalist model utilized for its broad historical knowledge and stable baseline performance [23]; (3) DeepSeek- V3.1, a reasoning-optimized model chosen for high-fidelity structured attribu- tion and framing analysis [6]; (4) Cogito-v2-1-671B, an ultra-large-scale model (671B) included to capture historiographical nuances that may be lost at smaller 2 https://w.manuale.edu.ro 8G. S , tefan and A. M. Dumitran parameter counts [5]; and (5) Kimi-K2-Thinking, an extended-reasoning model that externalizes deliberation via inspectable reasoning traces, ensuring the at- tribution logic is auditable [13]. Verdict Synthesis (Stage 3, Section 3.3) is performed by a GPT-5.2 meta- agent [22]. This model adjudicates the jury’s independent outputs—including category, severity, confidence, and reasoning—to produce a final verdict under the two experimental aggregation configurations. 4.3 Implementation Details and Report Generation Data Processing and Parsing. The input documents are segmented into non- overlapping five-page batches, rendered at 200 DPI (max 1280px). All outputs are serialized as structured JSON, with a three-attempt retry logic for schema validation. Failed parses are discarded to maintain pipeline integrity. Token Allocation and Cost. Extended-reasoning models (Kimi-K2-Thinking, DeepSeek-V3.1) are allocated 16,000 output tokens; standard models are capped at 4,096. Processing a full textbook costs approximately $2, with the jury stage dominating (∼70%, of which Kimi-K2-Thinking alone accounts for ∼50%), fol- lowed by the meta-agent (∼20%) and screening (∼10%). All models were ac- cessed via third-party inference APIs, 3 with the exception of the meta-agent, which was accessed directly via the official OpenAI API. Stochastic Variation. Due to the non-deterministic nature of the screening agent, the two evaluated configurations operated on slightly different candidate sets (Heuristic: 281; Independent: 270). While qualitative patterns remained consis- tent, cross-configuration severity statistics are presented as indicative, as the meta-agent strategies were evaluated on these naturally occurring variations rather than an identical frozen set. 4.4 Human Evaluation Methodology To assess practical utility, we conducted a blind comparison of anonymized re- ports from the three configurations: Zero-shot, Heuristic, and Independent De- liberation. For each textbook, evaluators selected the most accurate and ped- agogically sound report via a public interface, 4 justifying their choice through criteria such as analysis depth, taxonomy application, and tone objectivity. 3 https://w.together.ai/ 4 https://submission-its.github.io/bias-detection/ Agentic Bias Detection in Educational Textbooks9 Fig. 2. A generated HTML report highlighting extracted historical bias, taxonomy categorization, and assigned severity scores Report Format. Pipeline outputs are serialized as static HTML reports (see Fig. 2), presenting each flagged excerpt alongside its attribution label, taxonomy category, severity score, and the synthesized jury reasoning. The full corpus of reports and underlying JSON data was anonymized and published to a public GitHub repository to enable evaluator access and independent reproducibility. 5 Results and Discussion 5.1 Severity Calibration: Agentic Pipeline vs. Zero-Shot Baseline The screening agent flagged 270 candidate excerpts, of which 225 (83.3%) re- ceived a final severity of 3 or below following jury adjudication— classified as pedagogically acceptable. Of the remaining 45 cases with severity ≥ 4, the most common categories were Selection Bias and Omission / Underdevelopment. Ta- ble 2 presents the full distribution. 10G. S , tefan and A. M. Dumitran Table 2. Distribution of final severity scores (N = 270, Independent Deliberation configuration) Severity LabelCount % 1Neutral / Pedagogically Sound6 2.2 2Negligible Framing63 23.3 3Minor Imbalance156 57.8 4Moderate Bias43 15.9 5Significant Distortion2 0.7 6–7Severe / Educational Harm0 0.0 A zero-shot, single-model baseline auditing the identical full textbooks exhib- ited severe recall failure, extracting only 39 candidate excerpts compared to the pipeline’s 270. Furthermore, it systematically inflated severity scores for these 39 excerpts, assigning a mean of 5.4/7 (with no score below 4). In contrast, the agentic pipeline averaged 2.9/7 across its comprehensive set of 270 candidates (Fig. 3). While this extraction failure precludes a strict per-excerpt comparison, the yield discrepancy validates the need for a dedicated screening agent, just as the inflated severity highlights the baseline’s lack of deliberative nuance. To isolate the impact of the multi-agent architecture from context window and prompting variables, we conducted an ablation study evaluating the zero- shot baseline on identical 5-page chunks with the explicit leniency instruction. While chunking resolved the baseline’s initial extraction failure, it transformed the single model into an over-sensitive filter, yielding 192 total excerpts com- pared to the deliberative pipeline’s 109. Crucially, the multi-agent jury acts as a vital consensus filter: it suppressed negligible stylistic noise (reducing flags of severity 3 and below from 91.1% to 81.7% of total findings) while doubling the concentration of actionable, higher-severity issues (severity 4 and above: 17.4% vs. 8.9%). Furthermore, the deliberative architecture generated more confident assessments (mean confidence 0.84 vs. 0.78) with substantially deeper histori- ographical justifications (mean 268 vs. 207 characters per explanation). This confirms that while chunking enables basic extraction, multi-agent deliberation is strictly necessary to prevent alert fatigue and curate a confident, high-signal report for editorial governance. Agentic Bias Detection in Educational Textbooks11 1234567 0 20 40 60 2 . 2 23 . 3 57 . 8 15 . 9 0 . 7 15 . 4 41 33 . 3 10 . 3 Severity Score Percentage of Excerpts (%) Agentic Pipeline (μ = 2.9N = 270) Fig. 3. Severity distributions for the agentic pipeline (dark gray, μ = 2.9) and zero-shot baseline (light gray, μ = 5.4). Samples differ in size and configuration. The distribution is strongly concentrated around severity 3, with a thin right tail (2 excerpts at severity 5; none at 6 or 7). This indicates that the agentic architecture is well-calibrated for deployment: the jury layer avoids both under- penalization and the over-penalization that would render an automated auditing tool untrustworthy in an editorial governance workflow. Inter-Agent Agreement. All five jury agents produced valid evaluations for 96.3% of excerpts (260/270); the remaining 10 cases retained four valid agents after retry exhaustion. The mean inter-agent severity range was 1.28 points on the 7-point scale, with 69.6% of excerpts showing ranges of ≤ 1 point—indicating strong consensus on the majority of cases. Only 3.3% exhibited ranges of ≥ 3 points. The meta-agent escalation mechanism flagged 18 cases (6.7%) for human review, demonstrating selective rather than indiscriminate escalation. 5.2 Role of Source Attribution in Severity Calibration To assess the effect of attribution classification on downstream jury behavior, we compared severity outcomes across the two categories assigned during screen- ing. Excerpts classified as Primary Source Usage (N = 56) received a mean final severity of 2.68/7, compared to 2.95/7 for Textbook Narrative excerpts (N = 214). While the absolute difference is modest (0.27 points), the direction is consistent with the intended design: jury agents apply more lenient evaluation to historical quotations than to authorial exposition. Qualitative examination of jury agent reasoning for primary source excerpts showed a consistent shift from normative condemnation of archaic language to- ward pedagogical evaluation of contextual framing—a pattern observed across 12G. S , tefan and A. M. Dumitran multiple agents in the jury. These findings suggest that explicit attribution en- forcement meaningfully conditions jury evaluation, reducing the risk of misclas- sifying historically situated language as authorial bias. 5.3 Key Historiographical Bias Patterns While overt hate speech was absent from the evaluated corpus, the pipeline identified prevalent structural biases of direct relevance to pre-publication gov- ernance review. The most frequent and severe failure mode was historical saniti- zation through selective omission: accounts of Romania’s entry into World War I frequently framed military participation strictly as territorial recovery, omitting ideological alignment with Nazi Germany and state-sponsored mass violence. Jury agents assigned the highest severity scores (up to 5/7) to these excerpts, classifying them as substantive distortions rather than neutral summarizations. Beyond omission, the architecture consistently detected ethno-nationalist es- sentialism and Eurocentric framing. Textbooks frequently presented ethnogene- sis narratives as biological or metaphysical continuities, suppressing alternative historiographical debates and framing national identity as a fixed, inherent at- tribute. Excerpts classified under Perspective Limitation produced the highest category-level mean severity at 3.4/7 (N = 15), followed by Omission / Underde- velopment at 3.08/7 (N = 26). Table 3 summarizes the most frequent taxonomy categories. Table 3. Most frequent bias taxonomy categories with mean severity scores (Indepen- dent Deliberation, N = 270) CategoryCount Mean Severity Narrative Framing682.88 Primary Source Framing472.62 Selection Bias383.05 Omission / Underdevelopment263.08 Moral Loading232.83 National or Cultural Centering162.88 Perspective Limitation153.40 Source Selection Bias122.92 Teleological Narrative83.12 Other (4 categories)172.71 Notably, while the surface-level sentiment of cultural descriptions often ap- peared positive or neutral, the reasoning-oriented agents within the jury exposed unexamined normative assumptions beneath the text. These patterns, structural omissions and essentialist framing rather than overt hate speech, are precisely the type of subtle historiographical distortion that is difficult to detect manually at scale and that a pre-publication governance tool must surface reliably. Agentic Bias Detection in Educational Textbooks13 5.4 Human Evaluation of Report Quality We conducted a blind comparative study with 18 evaluators (history and com- puter science students, University of Bucharest) performing 54 paired compar- isons across three textbooks. Evaluators were provided with the source material alongside three anonymized HTML reports, one per configuration, and selected the most accurate, comprehensive, and pedagogically sound relative to the orig- inal text. Report assignments were randomized to prevent positional bias. The Independent Deliberation configuration was preferred in 64.8% of cases (35/54), followed by Heuristic Aggregation (25.9%) and the Zero-Shot Base- line (9.3%), with consistent rankings across all three textbooks and publishers (Fig. 4). The preference for Independent Deliberation suggests that qualitative jury synthesis—particularly surfacing well-supported minority positions— yields more actionable reports than fixed numerical thresholds. The near-total rejec- tion of the Zero-Shot Baseline confirms that single-pass auditing is insufficient for educational governance. The most cited selection criteria were Clarity and structure (50/54 eval- uations), Correct identification of issues (41/54), Depth of analysis (35/54), and Comprehensiveness (33/54), confirming that readability is a primary de- terminant of usability. As this evaluation measured report quality rather than per-excerpt detection accuracy, establishing formal precision and recall against expert-annotated ground truth remains a priority for future work. Corvinul (11th)Niculescu (12th)Sigma (11th) 0 5 10 15 11 1212 6 44 1 22 Number of Votes (out of 18) Independent Deliberation Heuristic Aggregation Zero-Shot Baseline Fig. 4. Evaluator preferences (N = 54 comparisons). Independent Deliberation was consistently preferred in randomized blind testing across all textbooks. 14G. S , tefan and A. M. Dumitran 5.5 Limitations and Future Work Several limitations constrain our findings. First, our comparative analysis be- tween the agentic pipeline and the zero-shot baseline illustrates the delta between a specialized architecture and a naive deployment scenario, which introduces sig- nificant confounding variables. The baseline processed entire textbooks within a single context window, precipitating severe recall failure (N = 39 vs. 270), whereas the screening agent processed localized five-page batches. This discrep- ancy precludes a controlled per-excerpt comparison. Additionally, the reliance on a single model in Stage 1 introduces a recall risk: excerpts missed by the screener are never surfaced for jury evaluation. Exploring a multi-model ensem- ble for screening is a priority for future work. Second, the jury’s explicit leniency calibration instruction is absent from the baseline. Consequently, no current ablation isolates whether the observed severity reduction stems independently from the multi-agent architecture, the Source Attribution Schema, or simply the prompt-level calibration and narrowed context window. Finally, our human evaluation assessed report usability rather than detection accuracy; establishing formal precision and recall requires expert-annotated ground truth. Future work must address these gaps through controlled ablation studies, such as evaluating a single-model baseline on identical five-page chunks with the exact leniency prompt, and rigorous expert annotation. Beyond governance, the structured controversy reports suggest a student-facing direction: a tool allowing students to interrogate contested passages in real time, extending the system from pre-publication auditing toward supporting critical historical thinking in the classroom. 6 Conclusions This paper introduces an automated auditing architecture for detecting subtle historiographical biases in high-stakes educational materials. By decomposing the task into broad-coverage discovery and deliberative adjudication, we address two primary failure modes of previous approaches: the inability to detect struc- tural omission and the tendency to hallucinate bias in primary sources. The system identified significant patterns of ethno-nationalist essentialism and sanitization by omission across the evaluated corpus, validating its capac- ity to surface meaningful historiographical concerns rather than superficial flags. The Meta-Jury selectively escalated only 6.7% of cases for human review, demon- strating that the architecture can prioritize expert attention without overwhelm- ing reviewers—a property essential for practical deployment. Beyond methodological contributions, this work addresses a real-world gover- nance need. Romania’s ongoing curriculum reform requires verifying a massive volume of new history textbooks simultaneously—making exhaustive manual bias analysis infeasible. At approximately $2 per textbook, the architecture offers a scalable decision-support tool for review committees, surfacing controversies before materials reach classrooms without replacing expert judgment. Future Agentic Bias Detection in Educational Textbooks15 work will focus on cross-national corpus replication, controlled ablation studies, expert-annotated ground truth, and a student-facing interface for in-classroom critical inquiry. References 1. Apple, M.: Official knowledge: Democratic education in a conservative age, third edition. Routledge (2014). https://doi.org/10.4324/9780203814383 2. Bai, J., Bai, S., Yang, S., et al.: Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond (2023). https://doi.org/ 10.48550/arXiv.2308.12966 3. Chan, C.M., Chen, W., Su, Y., et al.: Chateval: Towards better llm evaluations via multi-agent debate. In: ICLR (2024). https://doi.org/10.48550/arXiv.2308. 07201 4. Chen, W., Su, Y., Zuo, et al.: Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors. In: ICLR (2024). https://doi.org/10.48550/ arXiv.2308.10848 5. Deep Cogito: Cogito v2.1 671b model card. https://huggingface.co/deepcogito (2025) 6. DeepSeek-AI, et al.: Deepseek-v3 technical report (2024). https://doi.org/10. 48550/arXiv.2412.19437 7. Du, Y., Li, S., Torralba, A., Tenenbaum, et al.: Improving factuality and reasoning in language models through multiagent debate. arXiv preprint arXiv:2305.14325 (2023), https://arxiv.org/abs/2305.14325 8. Hsieh, C.P., Simig, D., et al.: Ruler: What’s the real context size of your long- context language models? In: Proceedings of EMNLP (2024), https://arxiv.org/ abs/2404.06654 9. Ji, Z., Lee, N., Frieske, R., et al.: Survey of hallucination in natural language generation. ACM Computing Surveys 55(12), 1–38 (2023). https://doi.org/10. 1145/3571730 10. Jiang, A.Q., Sablayrolles, A., Roux, A., et al.: Mixtral of experts (2024). https: //doi.org/10.48550/arXiv.2401.04088 11. Joshi, A., Kale, S., Chandel, et al.: Likert scale: Explored and explained. British Journal of Applied Science & Technology 7(4), 396–403 (2015). https://doi.org/ 10.9734/BJAST/2015/14975 12. Kasneci, E., Sessler, K., Küchemann, S., et al.: ChatGPT for good? on opportuni- ties and challenges of large language models for education. Learning and Individ- ual Differences 103, 102274 (2023). https://doi.org/10.1016/j.lindif.2023. 102274 13. Kimi Team, Bai, Y., et al.: Kimi k2: Open agentic intelligence (2025). https: //doi.org/10.48550/arXiv.2507.20534 14. Lewis, P., Perez, E., Piktus, A., et al.: Retrieval-augmented generation for knowledge-intensive nlp tasks. In: NeurIPS (2020). https://doi.org/10.48550/ arXiv.2005.11401 15. Liang, P., Bommasani, R., Lee, T., et al.: Holistic evaluation of language models. In: TMLR (2023). https://doi.org/10.48550/arXiv.2211.09110 16. Liu, N.F., Lin, K., Chen, J., et al.: Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics 12, 157–173 (2024). https://doi.org/10.1162/tacl_a_00638 16G. S , tefan and A. M. Dumitran 17. Liu, Y., Iter, D., Xu, Y., et al.: G-Eval: NLG evaluation using GPT-4 with better human alignment. In: Proceedings of EMNLP. p. 2511–2522 (2023). https:// doi.org/10.18653/v1/2023.emnlp-main.153 18. Luckin, R., Holmes, W., Griffiths, M., Forcier, L.B.: Intelligence Unleashed: An Argument for AI in Education. Pearson London (2016) 19. Lucy, L., Demszky, D., Bromley, P., Jurafsky, D.: Content analysis of text- books via natural language processing: Findings on gender, race, and ethnicity in texas us history textbooks. In: AERA Open (2020). https://doi.org/10.1177/ 2332858420940312 20. Meta AI, et al.: Llama 4 maverick. https://ai.meta.com/llama/ (2025) 21. Ministerul Educat , iei: Comunicat de presă nr. 109/2025 privind constituirea grupurilor de lucru pentru elaborarea programelor s , colare. https://w.edu.ro/ press_rel_109_2025_grupuri_lucru_programe_scolare_inv_liceal (2025), ac- cessed: 2025-02-24 22. OpenAI: Gpt-5.2 technical specifications. https://developers.openai.com/api/ docs/models/gpt-5.2 (2026) 23. OpenAI, Agarwal, S., Ahmad, L., Ai, J., Altman, S., et al.: gpt-oss-120b & gpt- oss-20b model card (2025). https://doi.org/10.48550/arXiv.2508.10925 24. Pingel, F.: UNESCO Guide on Textbook Research and Textbook Revision. UN- ESCO, Paris (2010) 25. Preston, C.C., Colman, A.M.: Optimal number of response categories in rating scales: Reliability, validity, discriminating power, and respondent preferences. Acta Psychologica 104(1), 1–15 (2000). https://doi.org/10.1016/S0001-6918(99) 00050-5 26. Röttger, P., Kirk, H.R., Vidgen, B., et al.: Xstest: A test suite for identifying exaggerated safety behaviours in large language models. In: NAACL (2024). https: //doi.org/10.48550/arXiv.2308.01263 27. Stradling, R.: Multiperspectivity in History Teaching: A Guide for Teachers. Coun- cil of Europe Publishing (2003) 28. UNESCO: Guidance for generative AI in education and research. UNESCO Pub- lishing, Paris (2023) 29. Wu, Q., Bansal, G., Zhang, et al.: Autogen: Enabling next-gen llm applications via multi-agent conversation. arXiv preprint arXiv:2308.08155 (2023). https://doi. org/10.48550/arXiv.2308.08155 30. Xu, Y., Li, M., Cui, L., et al.: Layoutlm: Pre-training of text and layout for docu- ment image understanding. In: ACM SIGKDD (2020). https://doi.org/10.1145/ 3394486.3403172 31. Zhai, X., Chu, X., Chai, C.S., et al.: A review of artificial intelligence (ai) in education from 2010 to 2020. Complexity 2021, 1–18 (2021). https://doi.org/ 10.1155/2021/8812542 32. Zhang, Y., Li, Y., Cui, L., et al.: Siren’s song in the ai ocean: A survey on hal- lucination in large language models. Computational Linguistics (2025). https: //doi.org/10.1162/COLI.a.16 33. Zheng, L., Chiang, W.L., Sheng, Y., Zhuang, et al.: Judging LLM-as-a-judge with MT-Bench and chatbot arena. In: Advances in Neural Information Processing Sys- tems (NeurIPS). vol. 36 (2023), https://arxiv.org/abs/2306.05685