Paper deep dive
AutoXRD: Autonomous LLM Agents and Comprehensive Evaluation for Powder Diffraction Analysis
Yuetong Wu, Maojun Sun
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 9/2/2026, 4:03:15 AM
Summary
The paper introduces AutoXRD, an autonomous Large Language Model (LLM) agent framework designed for powder X-ray diffraction (XRD) analysis. It addresses the challenge of end-to-end automation by organizing analysis as stepwise refinement with deterministic crystallographic and physical checks. The authors propose XRDBench, a comprehensive evaluation benchmark consisting of two tracks: XRDBench-QA (100 diagnostic tasks for scientific reasoning) and XRDBench-E2E (34 executable workflows for complete analysis). The study evaluates ten recent LLMs, finding that while models perform reasonably on diagnostic tasks, they struggle with autonomous execution, particularly in refinement-action selection and phase quantification. GPT-5.6 Sol achieved the highest overall score.
Entities (11)
Relation Signals (8)
XRDBench → hascomponent → XRDBench-QA
confidence 95% · XRDBench with two complementary tracks. XRDBench-QA contains 100 bounded diagnostic tasks
XRDBench → hascomponent → XRDBench-E2E
confidence 95% · XRDBench-E2E contains 34 executable workflows
AutoXRD → uses → XRDBench
confidence 95% · We further introduce XRDBench... We evaluate ten recent LLMs... using XRDBench
GPT-5.6 Terra → achievedscore → 81.0
confidence 90% · GPT-5.6 Terra the highest XRDBench-E2E point estimate of 81.0
GPT 5.6 Sol → achievedscore → 81.1
confidence 90% · GPT-5.6 Sol achieves the highest overall score of 81.1
AutoXRD → addresses → Powder X-ray Diffraction
confidence 90% · AutoXRD... organizes powder-XRD analysis as stepwise refinement
AutoXRD → executes → FullProf
confidence 85% · executes FullProf, GSAS-II, or task-appropriate numerical routines
AutoXRD → executes → GSAS-II
confidence 85% · executes FullProf, GSAS-II, or task-appropriate numerical routines
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Powder X-ray diffraction (XRD) is central to materials characterization, yet reliable end-to-end automation remains challenging. An XRD agent must interpret diffraction evidence, operate refinement software, manage coupled parameters in a defensible order, and distinguish numerical improvement from physical validity. In this paper, we propose AutoXRD, an autonomous large language model (LLM) agent framework that organizes powder-XRD analysis as stepwise refinement, grounds actions in observed evidence, and applies deterministic crystallographic and physical checks before accepting results. We further introduce XRDBench with two complementary tracks. XRDBench-QA contains 100 bounded diagnostic tasks that isolate scientific reasoning and decision-making, whereas XRDBench-E2E contains 34 executable workflows that test whether agents can compose these capabilities into complete analyses requiring file inspection, crystallographic-software execution, iterative refinement, evidence preservation, and reporting. We evaluate ten recent LLMs across 1,340 model--task runs. Models average only 57.8 out of 100, falling from 61.9 on XRDBench-QA to 53.7 on XRDBench-E2E. They perform best on refinement-history assessment and result acceptance, but remain substantially weaker on refinement-action selection, phase quantification, indexing, and Rietveld refinement. GPT-5.6 Sol achieves the highest overall score of 81.1, GPT-5.6 Terra the highest XRDBench-E2E point estimate of 81.0, and GPT-5.6 Luna the best score--cost trade-off. Ablations show that all six AutoXRD components consistently improve performance, supporting the framework design. Finally, execution-trace analysis reveals recurring failures in coupled-parameter control, quantitative reasoning, evidence preservation, and workflow termination, motivating stronger scientific constraints, uncertainty-aware decisions, and more efficient planning.
Tags
Links
- Source: https://arxiv.org/abs/2609.00070v1
- Canonical: https://arxiv.org/abs/2609.00070v1
Trouble viewing inline? Open PDF directly →
Full Text
53,738 characters extracted from source content.
Expand or collapse full text
AutoXRD: Autonomous LLM Agents and Comprehensive Evaluation for Powder Diffraction Analysis CCS: Computing methodologies Intelligent agentsCCS: Computing methodologies Natural language processingCCS: Applied computing Physical sciences and engineering Yuetong Wu† Affiliation: The Chinese University of Hong Kong, Hong Kong, China email: yuetongwu@link.cuhk.edu.hk and Maojun Sun†* Affiliation: The Hong Kong Polytechnic University, Hong Kong, China email: mj.sun@connect.polyu.hk © none Abstract. Powder X-ray diffraction (XRD) is central to materials characterization, yet reliable end-to-end automation remains challenging. An XRD agent must interpret diffraction evidence, operate refinement software, manage coupled parameters in a defensible order, and distinguish numerical improvement from physical validity. In this paper, we propose AutoXRD, an autonomous large language model (LLM) agent framework that organizes powder-XRD analysis as stepwise refinement, grounds actions in observed evidence, and applies deterministic crystallographic and physical checks before accepting results. We further introduce XRDBench with two complementary tracks. XRDBench-QA contains 100 bounded diagnostic tasks that isolate scientific reasoning and decision-making, whereas XRDBench-E2E contains 34 executable workflows that test whether agents can compose these capabilities into complete analyses requiring file inspection, crystallographic-software execution, iterative refinement, evidence preservation, and reporting. We evaluate ten recent LLMs across 1,340 model–task runs. Models average only 57.8 out of 100, falling from 61.9 on XRDBench-QA to 53.7 on XRDBench-E2E. They perform best on refinement-history assessment and result acceptance, but remain substantially weaker on refinement-action selection, phase quantification, indexing, and Rietveld refinement. GPT-5.6 Sol achieves the highest overall score of 81.1, GPT-5.6 Terra the highest XRDBench-E2E point estimate of 81.0, and GPT-5.6 Luna the best score–cost trade-off. Ablations show that all six AutoXRD components consistently improve performance, supporting the framework design. Finally, execution-trace analysis reveals recurring failures in coupled-parameter control, quantitative reasoning, evidence preservation, and workflow termination, motivating stronger scientific constraints, uncertainty-aware decisions, and more efficient planning. https://stephen-smj.github.io/AutoXRD/ Keywords: scientific agents, benchmark, powder diffraction, Rietveld refinement, AI for science 1. Introduction Figure 1. Overview of AutoXRD and the XRDBench evaluation protocol. AutoXRD combines an autonomous agent runtime, skill-guided powder-diffraction workflows, executable crystallographic backends, and evidence-based scientific validation. XRDBench evaluates diagnostic reasoning through XRDBench-QA and autonomous end-to-end XRD analysis through XRDBench-E2E. Powder X-ray diffraction (XRD) compresses three-dimensional reciprocal-space information into a one-dimensional intensity profile. Peak overlap, background, instrumental aberrations, preferred orientation, finite crystallite size, strain, and phase mixtures can therefore produce similar observations. Rietveld refinement addresses overlap by fitting the complete profile (Rietveld, 1969; Young, 1993), but it does not turn analysis into an unconstrained curve-fitting problem. The parameters governing the crystal structure, specimen, and instrument are strongly coupled; successful practice depends on releasing them in a defensible order and inspecting the observed, calculated, and difference patterns after each stage (Toby, 2006; Toby, 2024). These requirements expose two related gaps in current language-agent research. First, tool-using language models can interleave reasoning and external actions (Yao et al., 2023; Schick et al., 2023), and domain agents have automated parts of chemistry and experimentation (Bran et al., 2024; Boiko et al., 2023), but a general tool loop provides no scientific guarantee. A syntactically valid action may jointly release non-identifiable parameters, a lower RwpR_wp may arise from an incorrect structural or peak-profile hypothesis, and a fluent report may conceal an impossible occupancy or an unresolved phase. Second, current XRD-agent evaluation is not yet systematic. Existing studies demonstrate the feasibility of LLM-assisted Rietveld refinement (Li et al., 2026; Shin et al., 2026), but evaluations are often based on selected cases, isolated questions, or final fit statistics. No single criterion is sufficient: exact matching cannot accommodate valid open-ended analyses, fit metrics do not establish physical correctness, and judging only the final report cannot reveal whether a failure arose from scientific reasoning, software execution, missing artifacts, or poor termination. The field therefore lacks a unified protocol that tests both individual XRD decisions and complete executable workflows while measuring scientific quality, execution validity, efficiency, and failure behavior across models. Our abstraction treats refinement as a stepwise graph of scientific decisions. A node stores a complete fit state; an edge stores one structured action, the evidence that supports it, and predictions that can falsify its rationale. Numerical software computes the candidate child state, but deterministic checks—not the language model—decide whether the step is accepted. Rejected edges remain first-class evidence, enabling failure analysis and future policy distillation. Figure 1 summarizes AutoXRD. At runtime, the language model maintains session context and memory, plans with ten XRD-specific skills, and invokes permissioned tools within a sandbox. The skills translate domain procedures into executable scientific plans that specify the refinement stage, parameters or code, numerical bounds, required evidence, and expected outcomes. The scientific action layer parses diffraction patterns and structures, enforces parameter constraints, executes FullProf, GSAS-I, or task-appropriate numerical routines, and stores the resulting artifacts. Pattern-quality, residual, structural, and uncertainty evidence is then used to accept or reject candidate transitions in an evidence-gated refinement graph. The right panel of Figure 1 presents XRDBench, designed to address the evaluation gap through two complementary tracks. XRDBench-QA contains 100 bounded diagnostic tasks that isolate scientific reasoning and decisions, enabling reliable exact, objective, and reference-guided assessment. XRDBench-E2E contains 34 executable workflows over measured patterns, requiring agents to inspect inputs, implement and run numerical analyses, validate intermediate results, and deliver auditable artifacts. Neither track is sufficient alone: the former diagnoses what an agent understands, whereas the latter tests whether those capabilities can be composed into a functioning analysis. Together they evaluate the same XRD process at different degrees of integration and report performance, efficiency, and failure behavior under a common protocol. Using XRDBench, we evaluate ten recent LLMs in 1,340 agent runs. Average performance decreases from 61.9 on XRDBench-QA to 53.7 on XRDBench-E2E, and the two track scores are only moderately correlated (r=0.56r=0.56, ρ=0.65ρ=0.65), showing that diagnostic competence does not reliably translate into autonomous execution. Models perform best on refinement-history assessment and result acceptance, but remain substantially weaker on refinement-action selection, phase quantification, indexing, and Rietveld refinement. To summarize, our main contributions are: • A scientifically grounded autonomous XRD agent. We introduce AutoXRD, which combines XRD skill-guided planning, executable crystallographic backends, and deterministic scientific validation. Rather than accepting a step merely because the software runs or the fit residual decreases, AutoXRD requires the predicted evidence change and crystallographic checks to pass, while preserving both accepted and rejected actions in an auditable refinement trajectory. • A two-track benchmark for XRD reasoning and execution. We introduce XRDBench, comprising 100 bounded XRDBench-QA diagnostic tasks and 34 executable XRDBench-E2E workflows across eleven powder-diffraction task families. Its complementary evaluation protocols measure scientific decisions, numerical results, physical validity, artifact completeness, efficiency, and failure behavior. • A large-scale empirical study of LLMs for XRD. We evaluate ten recent LLMs across 1,340 agent runs, 22,135 model turns, and 625.6M selected-trajectory tokens. Our analysis shows that diagnostic performance does not reliably transfer to autonomous execution and identifies refinement-action selection, phase quantification, indexing, and Rietveld refinement as major remaining bottlenecks. 2. Background and Related Work Table 1. Comparison of language agents for X-ray and neutron diffraction. ✓/✗ indicate explicit support or evaluation/absence or no report. The five module columns cover phase or structure analysis, executable refinement, fit diagnosis, scientific validation, and structured reporting. Skills counts named procedures; Tools counts scientific callable interfaces, including skill-linked Python programs but excluding general file and shell tools. E2E requires agent-controlled execution of a material-specific task; Mtls. gives the number and source type of test materials, where sim. and real denote simulated and measured data; – denotes not reported or not applicable. Runs count model–task executions; LLM counts exclude judges and meta-optimizers. Supported XRD modules Agent resources Agent evaluation scope System Phase/Struct. Refine Fit Diag. Valid. Report Skills Tools Task QA E2E Mtls. Tasks Runs LLMs LLM-CSP Agent (Cao et al., 2025b) ✓ ✗ ✗ ✓ ✗ 0 5 PXRD structure pred. ✗ ✓ ∼ 6.4K sim. ∼ 6.4K ∼ 6.4K 1 AI X-ray Scientist (Chen et al., 2026) ✗ ✗ ✗ ✓ ✓ 0 3 SCXRD alignment ✗ ✓ 2 real 1 22 3 Rongzai Agent (Li et al., 2026) ✗ ✓ ✓ ✓ ✓ – 4 NPD refinement ✗ ✓ 5 real 5 20 4 AgentBuild-Rietveld (Shin et al., 2026) ✗ ✓ ✓ ✓ ✓ 0 6 PXRD refinement ✗ ✓ 3 real 8 34 1 NeuDiff Agent (Xiao et al., 2026) ✓ ✓ ✓ ✓ ✓ 0 4 SCND structure det. ✗ ✓ 1 real 1 10 2 AutoXRD (ours) ✓ ✓ ✓ ✓ ✓ 10 9 PXRD/NPD, 11 families ✓ ✓ 34 real 134 1,340 10 2.1. Rietveld Refinement and Its Failure Modes Let xix_i denote the diffraction coordinate (e.g., 2θ2θ, time of flight, or d-spacing), yiy_i the observed intensity, σi _i its standard uncertainty, and y^i(θ) y_i(θ) the profile calculated from parameters θ. Classical Rietveld refinement minimizes weighted least squares (Rietveld, 1969): (1) θ∗=argminθ∈Θχ2(θ),χ2(θ)=∑i=1nwi[yi−y^i(θ)]2,θ^*= _θ∈ χ^2(θ), χ^2(θ)= _i=1^nw_i [y_i- y_i(θ) ]^2, where wi=σi−2w_i= _i^-2 when uncertainties are available. The vector θ includes scale, background, zero shift, lattice, profile, atomic coordinates, displacement factors, occupancies, preferred orientation, and microstructure terms. Le Bail profile matching can align the cell, background, and peak profile before atomic parameters are exposed (Bail et al., 1988). For a fully crystalline mixture with all relevant phases included in the refinement model, phase weight fractions can be derived from refined scale factors and phase-specific ZMVZMV terms, where Z, M, and V denote formula units per cell, formula mass, and unit-cell volume (Hill and Howard, 1987). Preferred orientation and line shape require specialized models (Dollase, 1986; Thompson et al., 1987). No scalar fit index establishes structural correctness. For example, the weighted-profile factor (2) Rwp=100∑iwi(yi−y^i)2∑iwiyi2R_wp=100 _iw_i(y_i- y_i)^2 _iw_iy_i^2 depends on data quality and uncertainty conventions; a wrong model on weak data may achieve a lower value than an excellent model on precise data (Toby, 2006). This motivates explicit checks of residual structure, parameter correlations, chemistry, bounds, and unresolved peaks. 2.2. Automation of XRD Analysis Crystallographic engines such as FullProf (Rodríguez-Carvajal, 1993) and GSAS-I (Toby and Dreele, 2013) provide mature numerical capabilities. Automation has progressed from high-throughput wrappers (SrRietveld) (Tian et al., 2013), to expert systems (AutoFP) (Cui et al., 2015), reinforcement learning (PowderBot) (Feng et al., 2019), black-box optimization (BBO-Rietveld) (Ozaki et al., 2020), derivative-guided recipe selection (Toby, 2024), and learned parameter prediction (Mun et al., 2026). Robotic PXRD systems have integrated sample preparation, measurement, and automated analysis (Yotsumoto et al., 2024). Phase identification is related but distinct. Deep models can learn multi-phase labels from synthetic patterns (Lee et al., 2020; Schuetzke et al., 2021); XCA performs autonomous probabilistic phase identification without a language agent (Maffettone et al., 2021); CrystalShift uses symmetry-constrained pseudo-refinement and Bayesian comparison (Chang et al., 2025); Dara searches and refines multiple phase combinations (Fei et al., 2026); and RADAR-PD couples mismatch-tolerant prediction with Rietveld verification (Yadav et al., 2026). Recent end-to-end pipelines broaden this scope: AGAPI-XRD combines database matching, a fine-tuned structure generator, and GSAS-I/BGMN refinement (Campbell et al., 2026), while MatDiffract couples hierarchical retrieval with full-pattern refinement and quantitative phase fitting (Wang et al., 2026). COD (Gražulis et al., 2012), opXRD (Hollarek et al., 2026), SimXRD-4M (Cao et al., 2025a), and SIMPOD (Rincón et al., 2025) improve data availability. These advances automate engines, prediction, or search. Our focus is complementary: constraining and evaluating the sequence of decisions made by a language agent operating scientific software. Table 1 compares five language agents by their supported XRD modules, callable scientific resources, and evaluation scope. The multimodal LLM-CSP agent combines pattern predictors, a knowledge graph, and search for structure and space-group classification, but does not execute iterative refinement (Cao et al., 2025b). The AI X-ray Scientist closes the loop over detector observations and motor commands for single-crystal beamline alignment (Chen et al., 2026). Rongzai and AgentBuild-Rietveld operate GSAS-I for powder refinement and fit diagnosis, whereas NeuDiff connects neutron-data reduction, structure refinement, validation, and provenance (Li et al., 2026; Shin et al., 2026; Xiao et al., 2026). Their evaluations remain specialized: five neutron-powder tasks for Rongzai, eight X-ray cases for AgentBuild, and one repeated reference workflow for NeuDiff. OPENXRD provides a complementary non-agent benchmark of 217 crystallography questions over 74 language and multimodal models, but does not test scientific software execution (Vosoughi et al., 2026). XRDBench unifies these two evaluation needs through 100 diagnostic tasks and 34 executable powder-diffraction workflows across eleven families, evaluated with ten LLM agents under one protocol. 2.3. Tool-Using and Scientific Agents ReAct interleaves language reasoning with actions (Yao et al., 2023), while Toolformer learns when and how to call APIs (Schick et al., 2023). AgentBench (Liu et al., 2024), τ-bench (Yao et al., 2025), MLAgentBench (Huang et al., 2024), and DSAEval (Sun et al., 2026e) show that endpoint success alone can hide long-horizon failures and inconsistency. Scientific-agent work includes chemistry agents (Bran et al., 2024), autonomous laboratory orchestration (Boiko et al., 2023), and broad research automation (Lu et al., 2024; Sun et al., 2026a; Sun et al., 2026b; Sun et al., 2026d; Sun et al., 2026c). ScienceAgentBench argues for authentic, task-level assessment before claims of end-to-end discovery (Chen et al., 2025). AutoXRD applies this lesson to a narrow scientific domain with unusually sharp validity constraints. It differs from generic agents by making action schemas, falsifiable predictions, stage order, coupled-parameter exclusions, mandatory physical checks, and rejected steps part of the executable protocol. 3. Problem Formulation 3.1. Refinement States and Structured Actions An input instance is (3) ℐ=(,ℳ,),=(xi,yi,σi)i=1n,I=(D,M,C), =\(x_i,y_i, _i)\_i=1^n, where ℳM is instrument metadata and C is a set of candidate structures. At step t, the agent observes (4) st=(θt,qt,rt,vt,ct,t),s_t=( _t,q_t,r_t,v_t,c_t,O_t), where qt=(Rwp,t,Rexp,t,GoFt)q_t=(R_wp,t,R_exp,t,GoF_t) contains fit scalars, rtr_t contains residual features, vtv_t is a set of physical violations, ctc_t describes complexity and correlations, and tO_t indexes backend output files. Here RwpR_wp is the weighted-profile residual, RexpR_exp is the statistically expected residual, and GoF=Rwp/RexpGoF=R_wp/R_exp denotes goodness of fit. AutoXRD supports two complementary execution modes. In the validated refinement path, the agent proposes a structured action (5) at=(kt,gt,Pt,Bt,t,Πt),a_t=(k_t,g_t,P_t,B_t,Z_t, _t), where ktk_t is an action kind, gtg_t a refinement stage, PtP_t a parameter subset, BtB_t optional bounds, tZ_t cited evidence, and Πt _t falsifiable predictions. Each evidence item is (f,ν,τ,src)(f,ν,τ,src): feature name, observed value, decision threshold, and data source. A prediction π=(m,d,δ)π=(m,d,δ) specifies metric m, required direction d∈−1,+1d∈\-1,+1\, and minimum effect δ≥0δ≥ 0. In the native code-agent path, actions may instead read, create, edit, or execute task files. These broader actions are retained verbatim in the trajectory and are evaluated through execution, artifact, and physical gates. The deterministic backend produces a candidate (6) (st+1′,t+1)=F(st,at,),(s_t+1 ,O_t+1)=F(s_t,a_t;D), but the transition is not automatically accepted. This distinguishes three questions: (i) did the optimizer find a numerically better point; (i) did the intervention change the feature predicted by its stated mechanism; and (i) is the child state physically and statistically admissible? 3.2. Refinement Decision Graph A refinement decision graph is G=(V,ℰ+,ℰ−)G=(V,E^+,E^-). Each s∈Vs∈ V is a fit state, ℰ+E^+ contains accepted actions, and ℰ−E^- contains rejected actions. Let H(s,a,s′)∈0,1H(s,a,s )∈\0,1\ denote hard admissibility and (7) M(s,a,s′)=∏π∈Π(a)[dπ(mπ(s′)−mπ(s))≥δπ]M(s,a,s )= _π∈ (a)I\! [d_π (m_π(s )-m_π(s) )≥ _π ] denote support for the action’s expected outcome. The candidate child is accepted only if (8) Accept(s,a,s′)=H(s,a,s′)M(s,a,s′)[U(s′)−U(s)>0]=1.Accept(s,a,s )=H(s,a,s )\,M(s,a,s )I\! [U(s )-U(s)>0 ]=1. Crucially, utility cannot compensate for H=0H=0 or M=0M=0. 4. AutoXRD 4.1. Layered Agent Architecture As shown in Figure 1, AutoXRD separates five responsibilities. The agent controller manages the model, tools, sessions, and budgets. An XRD procedure library provides ten skills: pattern quality control, structure validity checking, executable GSAS-I workflows, Le Bail initialization, validated PCR generation, staged refinement, residual extraction, residual diagnosis, result acceptance, and final scientific review. The XRD execution layer parses patterns and structures, invokes GSAS-I, FullProf, or task-appropriate numerical crystallography code, and extracts reproducible measurements. Pre-execution and post-execution checks enforce parameter, fit, residual, and physical constraints. Finally, a run-history store and report generator expose the complete sequence of accepted and rejected steps. The language model is therefore a planner, implementer, and explainer, not the numerical optimizer or final scientific authority. This division follows a capability principle: use language models for context-dependent hypothesis selection, mature crystallographic engines for nonlinear optimization, and deterministic code for invariants that can be expressed exactly. 4.2. Deterministic Residual Evidence For residuals ei=yi−y^ie_i=y_i- y_i, the feature extractor estimates robust scale (9) σ^e=mediani|ei−median(e)|0.67448975 σ_e= median_i|e_i-median(e)|0.67448975 and standardized residuals zi=(ei−median(e))/σ^ez_i=(e_i-median(e))/ σ_e. Let e~i=ei−e¯ e_i=e_i- e, where e¯=n−1∑iei e=n^-1 _ie_i. The extractor reports normalized absolute residual, lag-one autocorrelation (10) ρ1=∑i=1n−1e~ie~i+1∑i=1ne~i2, _1= _i=1^n-1 e_i e_i+1 _i=1^n e_i^2, low/high-angle signed bias, the fraction of points in contiguous regions with |zi|≥5|z_i|≥ 5, and an unexplained-peak ratio. These features make diagnoses testable. For example, a nearly angle-independent peak-position offset can motivate zero-shift refinement. Geometry-dependent specimen-displacement and calibration effects must first be excluded, and the action must predict a minimum decrease in a position-bias feature. If RwpR_wp decreases but the predicted feature does not, Equation 7 rejects the causal explanation. 4.3. Validated PCR Generation FullProf PCR files are positional and contain codewords that can couple multiple parameters. The PCR validator and generator parses comment-anchored semantic slots into a catalog, freezes catalogued codeword groups, and releases only selectors permitted by ktk_t. It validates stage membership, selector coverage, constraints, phase count, pattern count, and mode before writing a new file. The source is never overwritten. Let Γ(k) (k) be the allowed semantic selector set for action kind k and cl(P)cl(P) the closure of P under existing constraint groups. Compilation is permitted only when (11) P⊆Γ(k),cl(P)∖P=∅,P (k), (P) P= , unless the expansion is explicitly declared. This prevents a seemingly local edit from silently releasing unrelated parameters. 4.4. Scientific Validity Checks and Candidate Ranking The pre-execution check rejects actions outside their stage and high-risk actions without bounds. It also rejects or requires explicit constraints for strongly correlated parameter combinations, such as simultaneous refinement of occupancy and displacement parameters or poorly identifiable combinations of zero shift, lattice parameters, and wavelength. After execution, H(st,at,st+1′)=0H(s_t,a_t,s _t+1)=0 if a new physical violation appears, RwpR_wp regresses by more than 2%2\%, the unexplained-peak ratio increases by more than 0.020.02, the maximum absolute correlation exceeds 0.980.98, or any declared prediction fails. Among states that pass all mandatory checks, the current implementation ranks candidates with U(s)=−[ U(s)=- [ Rwp(s)max(Rexp(s),ϵ)+2R(s)+3Q(s)+0.002K(s) R_wp(s) (R_exp(s),ε)+2R(s)+3Q(s)+0.002K(s) (12) +10[Cmax(s)−0.90]++10|v(s)|], +10[\,C_ (s)-0.90\,]_++10|v(s)| ], where R is residual score, Q unexplained-peak ratio, K parameter count, and CmaxC_ maximum absolute correlation. Here ϵ>0ε>0 is a numerical stabilizer. Weights are transparent policy constants rather than learned scientific truth. Equation 8 uses this score only after all mandatory checks pass. 4.5. Complete Refinement History Each run record contains (at,st,st+1′,dect,t+1)(a_t,s_t,s_t+1 ,dec_t,O_t+1), where dectdec_t is the acceptance decision and t+1O_t+1 lists the corresponding code, backend inputs, outputs, and evidence artifacts. The history records the proposed action, supporting observations, before/after states, validation results, and output-file paths for every accepted or rejected step. Retaining rejected attempts makes the final result reproducible and allows scientists to review which hypotheses were tried, why they failed, and how the accepted refinement path was obtained. A three-ring sunburst summarizes XRDBench by evaluation track, capability stratum, and task family, with separate difficulty counts. Figure 2. Composition of XRDBench. The inner ring separates the 100 diagnostic tasks from the 34 executable end-to-end workflows; the middle ring groups related capabilities; and the outer ring shows nine XRDBench-QA capability categories and eleven XRDBench-E2E task families.A three-ring sunburst summarizes XRDBench by evaluation track, capability stratum, and task family, with separate difficulty counts. 5. XRDBench 5.1. From Bounded Questions to Executable Workflows An autonomous diffraction analysis combines local scientific judgments with implementation, backend use, evidence preservation, and termination. Endpoint-only evaluation reveals whether a workflow succeeded but obscures why; bounded questions isolate the failure but cannot establish autonomy. XRDBench therefore contains two complementary parts that evaluate the same process at different levels of integration: (13) ℬQ⏟bounded XRD questions⟶ℬE⏟state-changing workflows, B_Q_bounded XRD questions B_E_state-changing workflows, where the arrow denotes increasing task integration, not information flow. The first part is XRDBench-QA. Here QA denotes question answering under bounded evidence and output requirements, not only multiple choice. Thirty cases require exact option sets, 40 require structured evidence-grounded reports, and 30 require quantitative outcomes plus reports. XRDBench-E2E contains 34 open workflows in which the agent must inspect files, implement and execute an analysis, revise failures, and package the evidence supporting its answer. Figure 2 summarizes the two tracks, their capability strata, and their task families; the complete task inventory is provided in the supplementary material. The QA track uses three evaluation forms: exact matching for 30 scientific decisions, reference-guided assessment for 40 structured diagnoses, and recomputed objective metrics plus report assessment for 30 quantitative outcomes. The end-to-end track instead evaluates executed reports and artifacts through execution and physical gates, task-specific numerical metrics, and scientific judging. The complete protocol table is included in the supplementary material. 5.2. XRDBench-QA XRDBench-QA evaluates whether an agent can make scientifically grounded decisions from bounded diffraction evidence. Its 100 tasks cover nine capabilities spanning refinement decisions, evidence interpretation, residual diagnosis, and quantitative recovery. Seventy tasks use deterministically generated inputs and ground truth, while the remaining 30 are constructed from experimental quantitative-phase-analysis data reported by the IUCr and phase-identification cases adapted from Dara (Madsen et al., 2001; Fei et al., 2026). Each public task provides the relevant pattern or refinement-trajectory evidence together with a required response format, whereas ground-truth answers, numerical targets, and scoring rubrics are retained by the evaluator. For an objective metric m, baseline b, and oracle o, higher- and lower-is-better utilities are (14) u↑(m)=clip(m−bo−b,0,1),u↓(m)=clip(b−mb−o,0,1).u_ (m)=clip\! ( m-bo-b,0,1 ), u_ (m)=clip\! ( b-mb-o,0,1 ). Parameter recovery combines mechanism detection and normalized errors; QPA combines phase-set F1, fraction error/closure, amorphous accuracy, and report quality; phase identification combines phase-set F1, indexed coverage, unknown handling, and report quality. With normalized case scores uj(Q)∈[0,1]u_j^(Q)∈[0,1], the QA track score is SQ=∑j=1100uj(Q)S_Q= _j=1^100u_j^(Q). 5.3. XRDBench-E2E The end-to-end track evaluates whether an agent can transform diffraction inputs into a complete, executable, and scientifically supported analysis. It contains 34 workflows spanning eleven powder-diffraction task families, including phase identification and quantification, pattern prediction, indexing, peak and microstructure analysis, structure determination, database retrieval, and Rietveld refinement. The workflows use 30 Cu Kα patterns derived from two opXRD sources (Hollarek et al., 2026) and four neutron-diffraction examples distributed with FullProf, with 11 categorized as Medium and 23 as Hard. Each task provides the diffraction data, instrument information, and executable workspace required by the agent, while target results and scoring criteria remain evaluator-side. For detailed analysis, we report all eleven task families separately; Each fresh, networkless workspace provides public inputs, native file/code tools, task-relevant XRD procedures, GSAS-I, FullProf, and numerical libraries. The agent must deliver final_report.md plus task-specific executed code and evidence. An execution gate checks input inspection, state change, successful execution, evidence, deliverables, and oracle isolation; an independent physical gate rejects numerical or crystallographic contradictions. Let gj=gjexecgjphys∈0,1g_j=g_j^execg_j^phys∈\0,1\ be their joint outcome. Let ϕA,j,ϕM,j,ϕJ,j∈[0,1] _A,j, _M,j, _J,j∈[0,1] denote artifact integrity, task-specific numerical correctness, and evidence-grounded scientific judging, respectively. The case and track scores are (15) SE,j S_E,j =100gj(0.15ϕA,j+0.55ϕM,j+0.30ϕJ,j), =100g_j (0.15 _A,j+0.55 _M,j+0.30 _J,j ), SE S_E =134∑j=134SE,j. = 134 _j=1^34S_E,j. 5.4. Aggregation and Reproducibility We primarily report (SQ,SE)(S_Q,S_E) and use (16) Soverall=0.5SQ+0.5SES_overall=0.5S_Q+0.5S_E All runs record trajectories, commands, artifacts, time, model turns, tool calls, and tokens; public cases enable audit, while future claims require sequestered extensions (Sainz et al., 2023). Figure 3. Overall performance, ordered by the overall score. The wider, hatched bar denotes Overall, followed by XRDBench-QA and XRDBench-E2E.A grouped bar chart compares overall, QA, and end-to-end scores for ten models. 6. Experiments 6.1. Experimental Setup We evaluate GPT-5.6 Sol, Terra, and Luna; MiMo-V2.5-Pro and MiMo-V2.5; DeepSeek-V4-Pro and Flash; MiniMax-M2.7; Claude Haiku 4.5; and Gemini 3.1 Pro Low under the AutoXRD agent framework. On XRDBench-QA, each case permits at most 20 tool calls and 8,192 output tokens per turn; On XRDBench-E2E, each workflow permits at most 50 agent steps and 32,768 output tokens per turn; GPT-5.6 Sol supplies the scientific-judgment component after execution and physical gates and objective metrics are evaluated. The complete protocol is provided in the supplementary material. 6.2. Research Questions and Statistical Analysis We ask: RQ1, which XRD capabilities current models master; RQ2, whether they compose into reliable workflows; RQ3, how ranks and failure modes change with integration; and RQ4, how efficiency and component choices affect performance. Our evaluation comprises 1,340 runs across ten models, involving 22,135 model turns. All intervals use 20,000 stratified task-bootstrap replicates; model differences use paired resampling within track. Thus intervals describe task variation, not decoding stochasticity. Ten radar charts compare nine bounded XRD capabilities and nine end-to-end reporting domains. Figure 4. Fine-grained performance across 18 dimensions: nine XRDBench-QA capabilities and nine end-to-end reporting domains. Each model has a separate 0–100 radar; numeric radial labels are omitted.Ten radar charts compare nine bounded XRD capabilities and nine end-to-end reporting domains. A heatmap compares final score, scoring-gate passage, deliverable completion, artifact validity, objective metrics, and scientific judging across eleven end-to-end powder-diffraction task families. Figure 5. XRDBench-E2E bottlenecks across eleven task families, averaged over ten models. Rows are ordered by final score; gate and completion columns are pass rates, while the remaining columns are effective 0–100 components. A single-column horizontal bar chart shows positive score contributions for six AutoXRD components. A single-column horizontal bar chart shows positive score contributions for six AutoXRD components. Figure 6. Component ablations on XRDBench-QA. Bars show the full-system score minus component removal. 7. Results Four logo scatter plots compare overall score with duration, model turns, tokens, and logarithmic estimated inference cost. Figure 7. Full-benchmark efficiency and cost-effectiveness. Resource means pool all 134 task runs per model; the vertical axis is the overall score.Four logo scatter plots compare overall score with duration, model turns, tokens, and logarithmic estimated inference cost. 7.1. Scientific Capability and Workflow Integration Figure 3 compares model performance across the two tracks and the overall benchmark. GPT-5.6 Sol and GPT-5.6 Terra lead overall with 81.09 and 79.41, followed by GPT-5.6 Luna at 68.87. The ten-model mean falls by 8.22 points, from 61.89 on XRDBench-QA to 53.67 on XRDBench-E2E, and seven models score lower on the end-to-end track. Although the tracks contain different tasks, the consistent decrease and the gate failures below show that composing scientific decisions into executable analyses remains substantially more difficult than answering bounded questions. On XRDBench-QA, models are strongest at refinement-history assessment (88.0) and result acceptance (81.0), but weak at next-action selection (44.0) and quantitative phase analysis (33.0). They therefore judge completed evidence more reliably than they control coupled parameters or recover phase fractions. All models miss the high-risk occupancy/zero–cell coupling cases; nine also accept a lower-RwpR_wp state that introduces an unexplained peak, allowing a familiar scalar metric to override contradictory residual evidence. End-to-end workflows expose different bottlenecks. Figure 5 contrasts workflow validity and completion with result quality across the eleven task families. Indexing is weakest (37.72) despite 90% scoring-gate passage, 80% complete deliverables, and 86.2 artifact validity: its objective-metric score is only 26.9 and its scientific-judge score is 33.2, showing that agents often execute and document a workflow but recover inaccurate cells or HKL assignments. Rietveld refinement instead combines a low final score (48.58) with the lowest scoring-gate rate (71.25%) and only 50% complete deliverables, indicating that workflow validity and evidence completion are major bottlenecks in addition to result quality. Crystallite and defect/disorder analyses score only 48.30 and 47.52, although their small family sizes make these comparisons descriptive. Peak analysis passes the scoring gate in 96.7% of runs and produces complete deliverables in 90.0%, yet scores only 57.60, further separating successful execution from scientific correctness. Figure 4 places the nine XRDBench-QA capabilities and nine end-to-end reporting domains on one profile per model. 7.2. Rank Transfer, Completion, and Efficiency XRDBench-QA and XRDBench-E2E scores are moderately correlated (Pearson r=0.557r=0.557; Spearman ρ=0.648ρ=0.648), but rankings change materially. DeepSeek-V4-Pro and Gemini 3.1 Pro Low gain 16.46 and 20.58 points and rise to third and fifth on XRDBench-E2E, whereas MiMo-V2.5-Pro, MiMo-V2.5, and DeepSeek-V4-Flash lose 21.8–27.6 points. These reversals show that diagnostic scores do not fully predict autonomous workflow performance. The overall estimates of Sol (81.09) and Terra (79.41) differ by 1.68 points with paired 95% interval [−1.46,4.80][-1.46,4.80]; neither is significantly ahead. Figure 7 compares performance with resource use across all 134 tasks per model. Terra approaches Sol’s score while using 298.7K rather than 349.2K tokens and costing $0.68 rather than $1.97 per task. Luna offers the strongest score–cost trade-off (68.87 at $0.07), but its 435-second duration exceeds Terra’s 239 seconds, showing that monetary and latency efficiency need not coincide. 7.3. Failure Analysis Trace inspection exposes three recurring failure types in end-to-end workflows. First, scientific outcome failures occur after genuine execution: indexing often produces inaccurate cells or HKL assignments despite passing the gate, while refinement may improve fit statistics without a valid structural result. Second, execution and evidence failures prevent scoring or completion: 46 of 340 runs (13.5%) fail at least one execution or physical gate, while 117 (34.4%) fail at least one full-completion requirement, commonly because code, backend outputs, or required evidence is missing or invalid. Third, termination failures consume almost the full 50-step budget without producing a valid deliverable. These traces explain why aggregate QA competence transfers only partially to autonomous workflows and separate runnable-but-wrong science from incomplete execution. 7.4. Ablation and Component Analysis Figure 6 summarizes the component effects on GPT-5.6 Luna. Each of the six tested components improves downstream performance. Result-acceptance checks and final scientific review yield the largest gains (17.5 and 17.1 points). Domain procedures add 2.5 points, while pattern quality control, residual reasoning, and staged refinement add 1.1–1.6 points. The consistent gains support combining procedural guidance with explicit verification. 8. Discussion XRDBench-QA and XRDBench-E2E expose complementary failure modes. The former diagnoses whether a model recognizes a valid action or scientific outcome; the latter adds implementation, backend operation, evidence preservation, physical validation, and delivery. Low Rietveld joint gate passage indicates incomplete or physically invalid workflows, whereas low Indexing accuracy despite high joint gate passage indicates incorrect science after genuine execution. The former calls for stronger execution contracts and stopping controls; the latter for better candidate generation and uncertainty-aware algorithms. Even the strongest models occasionally accept an invalid refinement because a single fit statistic improves, despite clear deterioration in the residual pattern. In end-to-end workflows, models may also produce convincing reports without retaining the output files and intermediate results needed to verify their conclusions. These findings show that reliable XRD agents must evaluate numerical fit, physical validity, and reproducible execution evidence together. Accordingly, XRD-agent evaluation should report not only final performance, but also execution success, physical-check passage, artifact completeness, and computational cost. 9. Conclusion AutoXRD combines LLM planning, crystallographic tools, and deterministic validation; XRDBench evaluates diagnostic reasoning and end-to-end workflows. Across ten models and 1,340 runs, diagnostic competence only moderately predicts autonomous performance: indexing is executable but inaccurate, while Rietveld refinement is limited by completion and validity. Reliable diffraction agents must expose the evidence and invariants supporting each conclusion, not merely produce plausible reports. Ethical Considerations Incorrect autonomous diffraction conclusions can affect materials decisions and downstream experiments. AutoXRD is decision support rather than expert certification: it exposes uncertainty and validation failures, while users remain responsible for provenance, safety, calibration, licensing, and independent scientific review. References Bail et al. (1988) A. L. Bail, H. Duroy, and J.L. Fourquet Ab-initio structure determination of lisbwo6 by x-ray powder diffraction. Materials Research Bulletin 23 (3), p. 447–452. External Links: Document Cited by: §2.1. Boiko et al. (2023) D. A. Boiko, R. MacKnight, B. Kline, and G. Gomes Autonomous chemical research with large language models. Nature 624, p. 570–578. External Links: Document Cited by: §1, §2.3. Bran et al. (2024) A. M. Bran, S. Cox, O. Schilter, C. Baldassari, A. D. White, and P. Schwaller Augmenting large language models with chemistry tools. Nature Machine Intelligence 6, p. 525–535. External Links: Document Cited by: §1, §2.3. Campbell et al. (2026) C. R. Campbell, J. Ely, J. Lee, F. M. Abel, and K. Choudhary Hybrid DiffractGPT–rietveld refinement framework for automated X-ray diffraction analysis. External Links: 2607.08890 Cited by: §2.2. Cao et al. (2025a) B. Cao, Y. Liu, Z. Zheng, R. Tan, J. Li, and T. Zhang SimXRD-4M: big simulated X-ray diffraction data and crystal symmetry classification benchmark. In International Conference on Learning Representations, External Links: Link Cited by: §2.2. Cao et al. (2025b) Q. Cao, Y. Xu, and M. Xiao LLM-agent framework for multimodal crystal structure prediction. Computer Systems Applications 34 (8), p. 33–42. External Links: Document Cited by: §2.2, Table 1. Chang et al. (2025) M. Chang, S. Ament, M. Amsler, D. R. Sutherland, L. Zhou, J. M. Gregoire, C. P. Gomes, R. B. van Dover, and M. O. Thompson Probabilistic phase labeling and lattice refinement for autonomous materials research. npj Computational Materials 11. External Links: Document Cited by: §2.2. Chen et al. (2026) Z. Chen, A. N. Petsch, A. J. Israelski, R. Plumley, L. Shen, C. Wang, C. Peng, Y. Ni, A. Bansil, S. Chowdhury, M. Li, J. B. Thayer, V. Thampy, and J. J. Turner An agentic artificially intelligent x-ray scientist. Nature Machine Intelligence 8, p. 1075–1086. External Links: Document Cited by: §2.2, Table 1. Chen et al. (2025) Z. Chen, S. Chen, Y. Ning, Q. Zhang, B. Wang, B. Yu, Y. Li, Z. Liao, C. Wei, Z. Lu, V. Dey, M. Xue, F. N. Baker, B. Burns, D. Adu-Ampratwum, X. Huang, X. Ning, S. Gao, Y. Su, and H. Sun ScienceAgentBench: toward rigorous assessment of language agents for data-driven scientific discovery. In International Conference on Learning Representations, External Links: Link Cited by: §2.3. Cui et al. (2015) X. Cui, Z. Feng, Y. Jin, Y. Cao, D. Deng, H. Chu, S. Cao, C. Dong, and J. Zhang AutoFP: a gui for highly automated rietveld refinement using an expert system algorithm based on FullProf. Journal of Applied Crystallography 48, p. 1581–1586. External Links: Document Cited by: §2.2. Dollase (1986) W. A. Dollase Correction of intensities for preferred orientation in powder diffractometry: application of the march model. Journal of Applied Crystallography 19, p. 267–272. External Links: Document Cited by: §2.1. Fei et al. (2026) Y. Fei, M. J. McDermott, C. L. Rom, S. Wang, and G. Ceder Dara: automated multiple-hypothesis phase identification and refinement from powder X-ray diffraction. Chemistry of Materials 38 (3), p. 1364–1376. External Links: Document Cited by: §2.2, §5.2. Feng et al. (2019) Z. Feng, Q. Hou, Y. Zheng, W. Ren, J. Ge, T. Li, C. Cheng, W. Lu, S. Cao, J. Zhang, and T. Zhang Method of artificial intelligence algorithm to improve the automation level of rietveld refinement. Computational Materials Science 156, p. 310–314. External Links: Document Cited by: §2.2. Gražulis et al. (2012) S. Gražulis, A. Daškevič, A. Merkys, D. Chateigner, L. Lutterotti, M. Quirós, N. R. Serebryanaya, P. Moeck, R. T. Downs, and A. L. Bail Crystallography open database (cod): an open-access collection of crystal structures and platform for world-wide collaboration. Nucleic Acids Research 40 (D1), p. D420–D427. External Links: Document Cited by: §2.2. Hill and Howard (1987) R. J. Hill and C. J. Howard Quantitative phase analysis from neutron powder diffraction data using the rietveld method. Journal of Applied Crystallography 20 (6), p. 467–474. External Links: Document Cited by: §2.1. Hollarek et al. (2026) D. Hollarek, H. Schopmans, J. Östreicher, J. Teufel, B. Cao, A. Alwen, S. Schweidler, M. Singh, T. Kodalle, H. Hu, G. Heymans, M. Abdelsamie, A. Hardiagon, A. Wieczorek, S. Zhuk, R. Schwaiger, S. Siol, F. Coudert, M. Wolf, C. M. Sutter‐Fella, B. Breitung, A. M. Hodge, T. Zhang, and P. Friederich OpXRD: open experimental powder x‐ray diffraction database. Advanced Intelligent Discovery. External Links: Document Cited by: §2.2, §5.3. Huang et al. (2024) Q. Huang, J. Vora, P. Liang, and J. Leskovec MLAgentBench: evaluating language agents on machine learning experimentation. In Proceedings of the 41st International Conference on Machine Learning, Vol. 235, p. 20271–20309. Cited by: §2.3. Lee et al. (2020) J. Lee, W. B. Park, J. H. Lee, S. P. Singh, and K. Sohn A deep-learning technique for phase identification in multiphase inorganic compounds using synthetic xrd powder patterns. Nature Communications 11, p. 86. External Links: Document Cited by: §2.2. Li et al. (2026) Q. Li, H. Wang, D. Xiong, J. Zhong, W. Ji, H. Hu, Y. Zhang, B. Zhang, H. Wang, Y. Zhu, R. Du, Z. Zhang, F. Qi, and J. Zhang Rongzai agent: a large language model-based autonomous assistant for rietveld refinement of neutron diffraction data. External Links: 2605.13911 Cited by: §1, §2.2, Table 1. Liu et al. (2024) X. Liu, H. Yu, H. Zhang, Y. Xu, X. Lei, H. Lai, Y. Gu, H. Ding, K. Men, K. Yang, S. Zhang, X. Deng, A. Zeng, Z. Du, C. Zhang, S. Shen, T. Zhang, Y. Su, H. Sun, M. Huang, Y. Dong, and J. Tang AgentBench: evaluating llms as agents. In International Conference on Learning Representations, External Links: Link Cited by: §2.3. Lu et al. (2024) C. Lu, C. Lu, R. T. Lange, J. Foerster, J. Clune, and D. Ha The AI scientist: towards fully automated open-ended scientific discovery. External Links: 2408.06292 Cited by: §2.3. Madsen et al. (2001) I. C. Madsen, N. V. Y. Scarlett, L. M. D. Cranswick, and T. Lwin Outcomes of the international union of crystallography commission on powder diffraction round robin on quantitative phase analysis: samples 1a to 1h. Journal of Applied Crystallography 34 (4), p. 409–426. External Links: Document Cited by: §5.2. Maffettone et al. (2021) P. M. Maffettone, L. Banko, P. Cui, Y. Lysogorskiy, M. A. Little, D. Olds, A. Ludwig, and A. I. Cooper Crystallography companion agent for high-throughput materials discovery. Nature Computational Science 1 (4), p. 290–297. External Links: Document Cited by: §2.2. Mun et al. (2026) S. J. Mun, Y. Nam, and S. Choi Automation of rietveld refinement through machine learning. Journal of Applied Crystallography 59 (2), p. 564–577. External Links: Document Cited by: §2.2. Ozaki et al. (2020) Y. Ozaki, Y. Suzuki, T. Hawai, K. Saito, M. Onishi, and K. Ono Automated crystal structure analysis based on blackbox optimisation. npj Computational Materials 6, p. 75. External Links: Document Cited by: §2.2. Rietveld (1969) H. M. Rietveld A profile refinement method for nuclear and magnetic structures. Journal of Applied Crystallography 2 (2), p. 65–71. External Links: Document Cited by: §1, §2.1. Rincón et al. (2025) S. Rincón, G. González, M. A. Macías, and P. Arbeláez A new benchmark for machine learning applied to powder x-ray diffraction. Scientific Data 12, p. 1186. External Links: Document Cited by: §2.2. Rodríguez-Carvajal (1993) J. Rodríguez-Carvajal Recent advances in magnetic structure determination by neutron powder diffraction. Physica B: Condensed Matter 192 (1–2), p. 55–69. External Links: Document Cited by: §2.2. Sainz et al. (2023) O. Sainz, J. A. Campos, I. García-Ferrero, J. Etxaniz, O. L. de Lacalle, and E. Agirre NLP evaluation in trouble: on the need to measure llm data contamination for each benchmark. In Findings of the Association for Computational Linguistics: EMNLP 2023, p. 10776–10787. External Links: Document, 2310.18018 Cited by: §5.4. Schick et al. (2023) T. Schick, J. Dwivedi-Yu, R. Dessi, R. Raileanu, M. Lomeli, E. Hambro, L. Zettlemoyer, N. Cancedda, and T. Scialom Toolformer: language models can teach themselves to use tools. In Advances in Neural Information Processing Systems, Vol. 36, p. 68539–68551. Cited by: §1, §2.3. Schuetzke et al. (2021) J. Schuetzke, A. Benedix, R. Mikut, and M. Reischl Enhancing deep-learning training for phase identification in powder x-ray diffractograms. IUCrJ 8, p. 408–420. External Links: Document Cited by: §2.2. Shin et al. (2026) W. Shin, C. A. Bridges, M. T. McDonnell, and R. Ferreira da Silva Fantastic scientific agents and how to build them: AgentBuild for rietveld refinement. External Links: 2606.12834 Cited by: §1, §2.2, Table 1. Sun et al. (2026a) M. Sun, R. Han, B. Jiang, H. Qi, D. Sun, Y. Yuan, and J. Huang A survey on large language model-based agents for statistics and data science. The American Statistician 80 (3), p. 331–344. External Links: Document Cited by: §2.3. Sun et al. (2026b) M. Sun, R. Han, B. Jiang, H. Qi, D. Sun, Y. Yuan, and J. Huang LAMBDA: a large model based data agent. Journal of the American Statistical Association 121 (553), p. 1–13. External Links: Document Cited by: §2.3. Sun et al. (2026c) M. Sun, R. Han, B. Jiang, H. Qi, D. Sun, Y. Yuan, and J. Huang Rejoinder to the discussions on “a survey on large language model-based agents for statistics and data science” a statistical view of LLM-based data science agents: reasoning, uncertainty, and reliability. The American Statistician 80 (3), p. 352–359. Cited by: §2.3. Sun et al. (2026d) M. Sun, R. Han, B. Jiang, H. Qi, D. Sun, Y. Yuan, and J. Huang Rejoinder to the discussions on “LAMBDA: a large model based data agent”. Journal of the American Statistical Association 121 (553), p. 36–43. External Links: Document Cited by: §2.3. Sun et al. (2026e) M. Sun, Y. Xie, Y. Wu, R. Han, B. Jiang, D. Sun, Y. Yuan, and J. Huang DSAEval: evaluating data science agents on a wide range of real-world data science problems. External Links: 2601.13591 Cited by: §2.3. Thompson et al. (1987) P. Thompson, D. E. Cox, and J. B. Hastings Rietveld refinement of Debye–Scherrer synchrotron X-ray data from Al2O3Al_2O_3. Journal of Applied Crystallography 20, p. 79–83. External Links: Document Cited by: §2.1. Tian et al. (2013) P. Tian, W. Zhou, J. Liu, Y. Shang, C. L. Farrow, P. Juhás, and S. J. L. Billinge SrRietveld: a program for automating rietveld refinements for high-throughput powder diffraction studies. Journal of Applied Crystallography 46, p. 255–258. External Links: Document Cited by: §2.2. Toby (2024) B. H. Toby A simple solution to the rietveld refinement recipe problem. Journal of Applied Crystallography 57, p. 175–180. External Links: Document Cited by: §1, §2.2. Toby and Dreele (2013) B. H. Toby and R. B. V. Dreele GSAS-I: the genesis of a modern open-source all-purpose crystallography software package. Journal of Applied Crystallography 46 (2), p. 544–549. External Links: Document Cited by: §2.2. Toby (2006) B. H. Toby R factors in rietveld analysis: how good is good enough?. Powder Diffraction 21 (1), p. 67–70. External Links: Document Cited by: §1, §2.1. Vosoughi et al. (2026) A. Vosoughi, A. Shahnazari, Y. Xi, Z. Zhang, G. Hess, C. Xu, and N. Abdolrahim OPENXRD: a comprehensive benchmark framework for LLM/MLLM XRD question answering. Digital Discovery 5, p. 1991–2015. External Links: Document Cited by: §2.2. Wang et al. (2026) H. Wang, M. Chen, H. Luo, Y. Wen, F. Qi, X. Chai, M. Liu, and F. Hou MatDiffract: a material-informed automated analysis platform for X-ray powder diffraction. External Links: 2607.20880 Cited by: §2.2. Xiao et al. (2026) Z. Xiao, L. Zhang, G. Zhang, and X. Wang NeuDiff Agent: a governed AI workflow for single-crystal neutron crystallography. Journal of Applied Crystallography 59 (4), p. 1102–1111. External Links: Document Cited by: §2.2, Table 1. Yadav et al. (2026) L. Yadav, Y. Cheng, and M. Doucet Automated multiphase identification and refinement in powder diffraction using mismatch-tolerant machine learning. External Links: 2605.12478 Cited by: §2.2. Yao et al. (2025) S. Yao, N. Shinn, P. Razavi, and K. Narasimhan τ-bench: a benchmark for tool-agent-user interaction in real-world domains. In International Conference on Learning Representations, External Links: Link Cited by: §2.3. Yao et al. (2023) S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao ReAct: synergizing reasoning and acting in language models. In International Conference on Learning Representations, External Links: Link Cited by: §1, §2.3. Yotsumoto et al. (2024) Y. Yotsumoto, Y. Nakajima, R. Takamoto, Y. Takeichi, and K. Ono Autonomous robotic experimentation system for powder x-ray diffraction. Digital Discovery 3, p. 2523–2532. External Links: Document Cited by: §2.2. R. A. Young (Ed.) (1993) R. A. Young (Ed.) The rietveld method. Oxford University Press. External Links: Document Cited by: §1.