Paper deep dive
RAIL: An Automatic Classifier of the Artificial Intelligence Readiness Level
Juan Irving Vasquez, Juan Terven, Laura-Ivoone Garay-Jimenez
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 95%
Last extracted: 8/16/2026, 2:27:07 AM
Summary
This paper introduces the Unified AI Readiness Level (AIRL), a nine-level ordinal scale that integrates Technology Readiness Levels (TRL), Machine Learning TRLs (MLTRL), and dimensional AI readiness models. To automate assessment, the authors propose RAIL (Readiness Assessment via Independent LLM-experts), a panel-of-experts classifier using multiple specialized Large Language Models to evaluate evidence and apply dimensional caps, ensuring conservative and auditable maturity estimates.
Entities (6)
Relation Signals (5)
RAIL â operationalizes â AIRL
confidence 98% · we propose RAIL... a panel-of-experts classifier that operationalizes the scale
AIRL â integrates â TRL
confidence 95% · The Unified AI Readiness Level (AIRL) proposed here integrates the three into a single ordinal scale... The adaptation of the NASA Technology Readiness Levels to AI
AIRL â integrates â MLTRL
confidence 95% · The Unified AI Readiness Level (AIRL) proposed here integrates the three... The Machine Learning Technology Readiness Levels of Lavin et al.
RAIL â uses â LLM
confidence 95% · RAIL... a panel-of-experts classifier... each a large language model with a narrowly scoped mandate
RAIL â avoids â overestimation
confidence 90% · showing consistency and avoiding overestimation from monolithic LLM classifiers.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Assessing the maturity of artificial intelligence technologies is essential for investment decisions, project management, and policy monitoring, yet the available readiness frameworks are heterogeneous and difficult to apply automatically: the adaptation of Technology Readiness Levels to AI lacks AI-specific gating criteria, the Machine Learning Technology Readiness Levels presuppose access to internal process artifacts, and AI/data readiness dimension models employ scales that resist direct comparison. This paper makes two contributions. First, we unify these three frameworks into the Unified AI Readiness Level (AIRL), a nine-level ordinal scale built on an environmental evidence ladder and complemented by dimensional caps (covering specification, data existence, data quality, data legality, expert knowledge, and algorithmic maturity) together with a generality-anchoring rule and explicit assignment disciplines, so that a readiness level becomes decidable from a natural-language description of the work alone. Second, we propose RAIL (Readiness Assessment via Independent LLM-experts), a panel-of-experts classifier that operationalizes the scale: one evidence agent and six independent dimension agents, each a large language model with a narrowly scoped mandate, deliver verdicts that a deterministic minimum rule aggregates and a chief expert reviews under asymmetric authority, confirming or lowering the panel's recommendation but never raising it above the caps. The method was tested in the analysis of several research works showing consistency and avoiding overestimation from monolithic LLM classifiers.
Tags
Links
- Source: https://arxiv.org/abs/2608.13428v1
- Canonical: https://arxiv.org/abs/2608.13428v1
Trouble viewing inline? Open PDF directly â
Full Text
56,160 characters extracted from source content.
Expand or collapse full text
RAIL: An Automatic Classifier of the Artificial Intelligence Readiness Level Juan Irving Vasquez â CIETEC-IPN Instituto Polit Ìecnico Nacional M Ìexico City, M Ìexico Juan Terven CICATA-QRO Instituto Polit Ìecnico Nacional Quer Ìetaro, M Ìexico Laura- Ivoone Garay-Jim Ìenez, Instituto Polit Ìecnico Nacional, UPIITA, M Ìexico City, M Ìexico August, 2026 Abstract Assessing the maturity of artificial intelligence technologies is essential for investment decisions, project management, and policy monitoring, yet the available readiness frame- works are heterogeneous and difficult to apply automatically: the adaptation of Technology Readiness Levels to AI lacks AI-specific gating criteria, the Machine Learning Technology Readiness Levels presuppose access to internal process artifacts, and AI/data readiness di- mension models employ scales that resist direct comparison. This paper makes two contri- butions. First, we unify these three frameworks into the Unified AI Readiness Level (AIRL), a nine-level ordinal scale built on an environmental evidence ladder and complemented by dimensional caps (covering specification, data existence, data quality, data legality, expert knowledge, and algorithmic maturity) together with a generality-anchoring rule and explicit assignment disciplines, so that a readiness level becomes decidable from a natural-language description of the work alone. Second, we propose RAIL (Readiness Assessment via In- dependent LLM-experts), a panel-of-experts classifier that operationalizes the scale: one evidence agent and six independent dimension agents, each a large language model with a narrowly scoped mandate, deliver verdicts that a deterministic minimum rule aggregates and a chief expert reviews under asymmetric authority, confirming or lowering the panelâs recommendation but never raising it above the caps. The method was tested in the analysis of several research works showing consistency and avoiding overestimation from monolithic LLM classifiers. Code available at https://github.com/irvingvasquez/RAIL 1 Introduction Artificial intelligence has moved from laboratory research to a strategic asset in industry, government, and science[16], and with this transition the question of maturity has become as consequential as the question of performance. Investors must decide which prototypes are close enough to deployment to fund, project managers must plan the transition from research artifact to product, and policy bodies must monitor the state of national and institutional AI portfolios. All of these decisions presuppose the ability to place a given piece of work on a common maturity scale. For conventional engineered systems this role has long been played by the Technology Readiness Levels (TRL) introduced by NASA [24, 18], a nine-level ordinal scale ranging from the observation of basic principles to a system proven in operation. The TRL scale, however, was conceived for deterministic hardware and software, and a growing body of literature shows that it transfers poorly to artificial intelligence: AI systems â Corresponding author: jvasquezg@ipn.mx 1 arXiv:2608.13428v1 [cs.AI] 13 Aug 2026 Actor Title, Abstract, Experiments summary, conclusions RAIL Classifier Chief expert AIRL Number, Argumentation Experts panel Figure 1: The RAIL classifier estimates a level of readiness for an AI application. It receives a natural language description of the work and estimates the level and provides an argumentation of the decisions. The decision is taken by a chief who valorates the comments of a panel of experts. are data-dependent, stochastic, and sensitive to distribution shift, so that a system validated in the laboratory may degrade unpredictably once deployed [13, 5, 4]. In response, sev- eral AI-specific readiness frameworks have been proposed, from the contextualization of the nine TRLs for AI by the European Commissionâs AI Watch initiative [21], through the Ma- chine Learning Technology Readiness Levels (MLTRL) that recast the progression as a gated systems-engineering process [13], to dimensional models that decompose readiness into AI and data specific facets such as data quality and data legality [9]. While conceptually mature, these frameworks are heterogeneous and difficult to apply automatically: the AI adaptation of the TRL lacks AI-specific gating criteria, MLTRL presupposes access to internal process artifacts that are rarely visible in a textual description of a work, and the dimensional models employ scales of differing lengths that resist direct comparison. In practice, therefore, readi- ness assignment remains a manual exercise performed by panels of subject-matter experts [7, 12, 3]. Automatic estimation of readiness levels has been explored, but existing approaches fall short of the problem just described. Bibliometric and clustering methods estimate matu- rity only at the level of entire technology fields and at coarse granularity [7, 12]; supervised ensembles depend on small, domain-specific labeled datasets [6]; and recent single-LLM as- sistants, while promising, inherit the biases and hallucination risks of a single model and still deviate from expert judgment in a non-negligible fraction of cases [3]. Moreover, virtually all of this work targets the legacy unidimensional TRL rather than the richer, multidimensional AI-readiness frameworks, whose dimensions each require specialized expertise. The literature thus exhibits a twofold gap: AI-specific frameworks lack automated, reproducible estimation mechanisms, and existing estimators fail to reproduce the the property that makes manual assessment reliable. This paper closes that gap with two contributions. First, we propose the Unified AI Readiness Level (AIRL), an adapted nine-level ordinal scale that integrates the EU inter- pretation of the TRL [21], the MLTRL of Lavin et al. [13], and the dimensional model of Eljasik-Swoboda et al. [9] into a single framework built on an environmental evidence ladder, complemented by dimensional caps over six AI-specific readiness dimensions, a generality- anchoring rule, and explicit assignment disciplines, so that a readiness level becomes de- cidable from a natural-language description of the work alone. Second, we propose RAIL (Readiness Assessment via Independent LLM-experts), a panel-of-experts classifier that op- erationalizes the scale (See Figure 1): one evidence expert and six independent dimension experts, each LLM with a narrowly scoped mandate, deliver verdicts that a deterministic minimum rule aggregates and a chief expert reviews under asymmetric authority, confirming or lowering the panelâs recommendation but never raising it above the caps. By construction, the resulting classifier is sound with respect to the framework, conservative under ambigu- ity, neutral under silence (lack of evidence), and auditable. We evaluate the approach on a 2 corpus of graduate theses, comparing monolithic and panel-based classification under three readiness protocols, and show that the panel architecture avoids both the maturity inflation and the missed dimensional gaps exhibited by monolithic LLM classifiers. The remainder of the paper is organized as follows. Section 2 reviews readiness frame- works and prior work on automatic readiness estimation. Section 3 presents the Unified AI Readiness Level and its assignment rules. Section 4 describes the RAIL panel architecture. Section 5 reports the experimental evaluation, and Section 6 concludes. 2 Related Work The Technology Readiness Level (TRL) scale was introduced by NASA as a nine-level met- ric to assess the maturity of a technology, ranging from the observation of basic principles (TRL 1) to a system proven in an operational environment (TRL 9) [24, 17]. Owing to its simplicity and domain independence, the scale was subsequently adopted by the European Commission, defense agencies, and innovation-management bodies as a standard instrument for acquisition and funding decisions. However, the original TRL was conceived for deter- ministic hardware and software systems, and a growing body of literature shows that it is ill-suited for artificial intelligence (AI): AI systems are data-dependent, stochastic, and sen- sitive to distribution shifts, so a system validated in a laboratory may degrade unpredictably once deployed [14, 5, 4]. 2.1 From TRL to AI Readiness Level Several adaptations of the TRL scale have therefore been proposed. Eljasik-Swoboda et al. [9] extended the READINESSnavigator innovation-assessment tool with six AI- and data- specific readiness dimensions (e.g., algorithmic, data-quality, and data-legal readiness), re- placing manufacturing-oriented criteria with data-centric ones. Mart Ìınez-Plumed et al. [20], within the European Commissionâs AI Watch initiative, contextualized the nine TRLs for AI and introduced bidimensional readiness-versus-generality charts, showing that high readiness is only attained by narrow, low-generality AI applications. Lavin and Renard [15] and later Lavin et al. [14] formalized the Machine Learning Technology Readiness Levels (MLTRL), a ten-level (0â9) systems-engineering framework featuring non-monotonic âswitchbacks,â stan- dardized TRL Cards, and gated multidisciplinary reviews spanning research to continuous post-deployment monitoring. More recent work has broadened readiness assessment beyond purely technical maturity. In the national-security domain, Browne et al. [5] interviewed defense experts who deemed the classical TRL an ineffective metric for AI and proposed the AI Readiness Level (AIRL) framework, which gates TRL progression behind minimum thresholds in five dimensions: alignment, justified confidence, governance, human readiness, and data readiness; a compan- ion study elaborates this multidimensional framework for military combat systems [4]. Their case studies show that commercially mature systems (TRL 9) can score as low as AIRL 1 once AI-specific risks are considered. M Ìuller et al. [22] proposed the Use Case-Centered AI Readiness Level (UCAIRL), an eleven-level scale that makes domain-specific problem framing, data audits, and compliance mandatory prerequisites of technical implementation, addressing the paradox of technically mature AI projects that fail operationally. Domain- and organization-oriented adaptations follow the same trend: a unified nine-level TRL lad- der for clinical AI derived through systematic review and Delphi synthesis [25], a TRL-based mapping of AI maturity in maternal health interventions [19], an organizational AI-readiness index for the public sector built on digital-transformation and data-management maturity [1], and an enterprise-level AIRL scale grounded in diffusion-of-innovations and technologyâ organizationâenvironment theory [10]. Collectively, this evolution reflects a shift from a unidimensional, technology-centric scale toward multidimensional, socio-technical readiness models in which data, governance, human factors, and use-case context are first-class evalu- ation criteria. 3 2.2 Automatic Estimation of Readiness Levels Regardless of the framework adopted, assigning a readiness level remains a predominantly manual exercise performed by panels of subject-matter experts, which is slow, costly, poorly scalable, and prone to subjectivity and inter-rater inconsistency [7, 12, 3]. A second research stream has therefore explored the automatic estimation of readiness levels. Early work framed the problem with soft-computing techniquesâartificial neural net- works, genetic algorithms, and fuzzy logicâto recognize readiness levels of R&D projects under uncertainty [27]. Chukhray et al. [6] trained a stacking ensemble of weak regressors, with a random-forest meta-learner, on 56 university R&D projects to estimate product readi- ness and commercialization cost, explicitly positioning the ensemble as a low-cost surrogate for a panel of human meta-experts. A complementary line of work exploits bibliometric signals as observable proxies of maturity: Dastoor et al. [7] fitted S-curves to publications, patents, grants, and NASA Spinoff data for 31 technologies and applied ordinal regression to predict TRLs, while Jain and Kumar [12] clustered 136 technology trends with unsuper- vised methods (MDBSCAN), classifying maturity into coarse watch/prepare/act bands with 84.9% accuracy and without labeled data. Most recently, Betancourt et al. [3] leveraged large language models (LLMs), fine-tuning LLaMA 2 and GPT-3.5-Turbo on approximately 2,500 TRL-specific samples to build a conversational virtual assistant that estimates TRLs through natural-language dialogue; on eight real prototypes, the assistant matched expert assessments exactly in 50% of cases and within one level in a further 37.5%. Despite this progress, existing automatic approaches exhibit important limitations. Bib- liometric and clustering methods estimate maturity only at the level of entire technology fields, operate at a coarse granularity (often three bands rather than nine levels), and cannot assess an individual project from its own evidence [7, 12]. Supervised ensembles depend on small, domain-specific labeled datasets that are expensive to curate and generalize poorly [6]. Single-LLM assistants, while promising, inherit the biases and hallucination risks of a single model, rely on limited fine-tuning corpora, and still deviate from expert judgment in a non- negligible fraction of cases [3]. Moreover, virtually all estimation work targets the classical TRL scale, and does not yet address the richer, multidimensional AI-readiness frameworks reviewed, whose dimensions (e.g., data, governance, and human readiness) each require spe- cialized expertise. The literature thus reveals a twofold gap. First, while AI-specific readiness frameworks have matured conceptually, they lack automated, scalable, and reproducible estimation mech- anisms; conversely, existing automatic estimators remain tied to the legacy unidimensional TRL. Second, the very property that makes manual assessment reliableâthe deliberation of a multidisciplinary panel of experts, as institutionalized in gated reviews [14] and Delphi syntheses [25]âis precisely what current single-model estimators fail to reproduce. This motivates the model proposed in this work: a panel of LLM-based experts in which multiple specialized agents, each responsible for a distinct readiness dimension, independently assess the available evidence and aggregate their judgments. 3 The Unified AI Readiness Level The frameworks reviewed in the preceding section approach the maturity of artificial intelli- gence from several perspectives. However, we have restricted our research to three proposals. The adaptation of the NASA Technology Readiness Levels to AI proposed by Mart Ìınez- Plumed et al. [21] preserves the canonical nine-level environmental progression, from basic principles observed to an actual system proven in operation, and demonstrates that readi- ness in AI cannot be stated meaningfully without fixing the level of generality at which a technology is expected to perform. The Machine Learning Technology Readiness Levels of Lavin et al. [13] translate the same progression into an operational systems-engineering process for machine learning, contributing precise definitions of the intermediate transitions (proof of principle, proof of concept, capability, integration) as well as mechanisms, such as switchbacks and gated reviews, that acknowledge the non-monotonic character of real ML development. Finally, Eljasik-Swoboda et al. [9] decompose readiness into AI-specific dimensions: specification, algorithmic maturity, and four facets of data readiness (existence, format and quality, legality, and expert knowledge). Then, they establish the principle that 4 the overall readiness of an innovation is the minimum across its constituent dimensions. None of the three frameworks alone suffices for the classification task addressed in this work: the TRL scale lacks AI-specific gating criteria, ML-TRL presupposes access to internal process artifacts that are rarely visible in a textual description of a work, and the dimensional model of Eljasik-Swoboda et al. employs heterogeneous scales of five to nine levels that resist direct comparison. The Unified AI Readiness Level (AIRL) proposed here integrates the three into a single ordinal scale with explicit, evidence-based assignment rules, designed so that a human an- notator or a large language model can assign a level to a natural-language description of an AI endeavour in a reproducible manner. 3.1 Structure of the AIRL scale The AIRL scale comprises nine ordinal levels whose primary discriminating variable is the evaluation environment in which evidence of functioning has been produced. This choice fol- lows directly from the environmental grouping implicit in the EU adaptation of the TRLs [21], where levels one through four correspond to laboratory conditions, five and six to relevant or simulated operational conditions, and seven through nine to the operational environment it- self. The environment is preferred over alternative discriminators, such as reported predictive performance, because it is observable in textual descriptions, is largely domain-independent, and correlates with the residual risk that separates a demonstrated result from a dependable product: as both [21] and [13] observe, benchmark performance does not directly translate into a technology that is ready for use in real-world environments. At the lower end of the scale, AIRL 1 denotes work in which basic principles have been formulated but no experiment has been executed; it absorbs both TRL 1 [18, 23] and the theoretical âfirst principlesâ stage that MLTRL [9] designates as Level 0. AIRL 2 requires that a concrete application concept exists and that exploratory experiments have been per- formed on sample, toy, or synthetic data, in correspondence with MLTRLâs goal-oriented research stage, in which experiments probe specific model properties rather than end-to-end performance. AIRL 3 constitutes the experimental proof of principle: the approach is val- idated in a testbed, typically against public benchmarks or simulated data, with defined metrics and baseline comparisons; this level unifies TRL 3 with MLTRL Level 2 and marks the first point at which quantitative claims become admissible evidence. AIRL 4 requires that the constituent components (model, data pipeline, and interfaces) have been integrated and shown to work together in a controlled environment, with the codebase raised to what Lavin et al. term prototype caliber. The middle of the scale captures the transition from research artifact to application. AIRL 5 is reached when the technology has been validated on real, representative data of the target use case in a relevant environment, and when the evaluation includes application- oriented measures in addition to conventional ML metrics; this level merges TRL 5 with the proof-of-concept stage of MLTRL, whose defining requirement is precisely the substitution of curated research data by noisy, real-world data. AIRL 6 corresponds to the demonstration of the technology as a capability, in the sense given to that term by Lavin et al.: the model no longer operates in isolation but as a module of a larger workflow, demonstrated in a relevant environment before stakeholders beyond the research team, with product requirements and their verification and validation measures explicitly drafted. This level deliberately marks the research-to-product handoff that MLTRL identifies as the âvalley of deathâ of applied machine learning [2], and that the READINESS navigator observations of Eljasik-Swoboda et al. associate with the structural bias of technically oriented organizations toward technology readiness at the expense of market-facing activities. The upper levels concern operation. AIRL 7 requires an actual system prototype func- tioning in the operational environment, typically as a pilot, field trial, beta program, or shadow deployment on live data. AIRL 8 denotes the system complete and qualified: ver- ified against the full set of requirements, subjected to deployment-oriented testing regimes such as canary or shadow tests, and, in regulated domains, certified by the competent au- thority (a condition that the EU TRL [21] rubric makes explicit and that we adopt as a hard requirement). AIRL 9, finally, is reserved for systems proven in sustained operation. Following Lavin et al., we require at this level not merely the fact of deployment but evidence 5 of the maintenance apparatus that deployment of learning systems demands: monitoring for data drift, concept drift, and performance degradation, together with defined retraining or feedback processes. A system described as launched but lacking any monitoring provision is, under the unified scale, qualified rather than proven. 3.2 Dimensional gating The environmental ladder alone would misclassify a well-known failure mode of AI projects: work that exhibits advanced experimental evidence while resting on unresolved foundations, most commonly of a legal or data-related nature. To capture this, the AIRL incorporates the dimensional model of Eljasik-Swoboda et al. [9] not as a set of parallel scales but as a system of caps on the environmentally derived level. Six dimensions are monitored: the specification of the use case and its success criteria; the existence of and access to the required data; the understanding of data format and the measurement of data quality, including bias and class imbalance; the legality of data use, encompassing personal-data protection and freedom-to-operate considerations; the availability of captured expert knowledge where the task demands it; and the algorithmic dimension, covering the selection, evaluation on real data, and tuning of the learning method. When a description provides explicit evidence that one of these dimensions is deficient (for instance, that the legality of processing the underlying data remains unclear, or that a concrete learning target has not been defined) the assigned level is bounded above by the level compatible with that deficiency, irrespective of the sophistication of the reported experiments. Formally, the final level is the minimum of the environmentally evidenced level and the caps induced by explicitly evidenced dimensional gaps. This construction preserves the minimum principle of the source framework, in which an innovation cannot outrank its weakest readiness field, while adapting it to the epistemic situation of a classifier that observes only a textual description: silence with respect to a dimension is treated as neutral, and only affirmative evidence of a gap triggers a cap. The same minimum principle is applied compositionally, following the system-level rule of Lavin et al. whereby the readiness of a system equals the lowest readiness of its essential constituent parts; a pipeline that couples a mature, deployed component with a newly prototyped one is classified at the level of the latter. 3.3 Generality anchoring and assignment discipline A distinctive contribution of Mart Ìınez-Plumed et al. [21] is the observation that readiness and generality trade off against one another: a technology specialized to a narrow, controlled domain may attain the highest levels while its more general formulation remains at the stage of basic research, so that a readiness statement is ill-defined unless the scope of capability is fixed. The unified scale internalizes this observation as an anchoring rule rather than as a second axis. Each description is classified at the level of generality that it itself claims: when the demonstrated evidence pertains to a narrower scope than the claimed one, the level assigned is that which the evidence supports for the broad claim, which is in general lower; conversely, a description that explicitly restricts its own scope is evaluated within that restriction. This preserves the analytical content of the readiness-versus-generality charts while yielding the single ordinal label that a classification setting requires. Three further disciplines govern assignment, all motivated by properties of the source frameworks. First, evidence takes precedence over intention: aspirational formulations con- tribute nothing to the level, which is determined solely by what has demonstrably been done, a direct consequence of the gated-review logic of MLTRL [13], in which graduation from a level requires satisfied criteria rather than declared plans. Second, ambiguity is resolved conservatively toward the lower of two adjacent candidate levels, reflecting the risk-averse posture that all three frameworks inherit from their systems-engineering lineage. Third, maturity is not inherited: the incorporation of a pretrained or off-the-shelf component does not transfer that componentâs readiness to the work under evaluation, and, symmetrically, a previously deployed system undergoing rework is classified at the current state of the re- worked component, in accordance with the switchback mechanisms through which MLTRL [13]formalizes regressions in maturity. Taken together, the environmental ladder, the dimen- sional caps, the generality anchoring, and the assignment disciplines define a classification function from textual descriptions to the ordinal set 1,..., 9 that is faithful to the three 6 Table 1: Correspondence between the Unified AI Readiness Level (AIRL) and related frame- works. TRL levels follow the EU adaptation [21]; MLTRL levels follow Lavin et al. [13]. The dimensions of Eljasik-Swoboda et al. [9] act transversally as caps on all levels. AIRL Defining evidenceTRL MLTRL 1Principles formulated; no experiment executed10 2Concept with exploratory experiments on sample or synthetic data21 3Proof of principle in testbeds or benchmarks with defined metrics32 4Components integrated and validated in a controlled environment43 5Validation on real, representative data in a relevant environment54 6Capability demonstrated within a larger workflow; R&Dâproduct handoff 65 7System prototype operating in the operational environment (pilot)76â7 8System complete, verified, and qualified or certified88 9Sustained operation with monitoring and maintenance processes99 NLP description Evidence Expert environmental ladder, rules R1âR3, R5âR7 D 1 Specification D 2 Data existence D 3 Data quality D 4 Data legality D 5 Expert knowledge D 6 Algorithmic S1: independent, parallel experts Minimum rule â â = min(â,c 1 ,...,c 6 ,Ï) S2: deterministic (no LLM) Chief Expert reviews full panel report, proposesv, records dissent Clamp AIRL = min(v,â â ) S3: verdict Final AIRL â 1,..., 9 + justification, binding caps, quoted evidence, dissent â , flags r,z c 1 c 2 c 3 c 4 c 5 c 6 â â Full panel report v hard bound â â confirm or lower â never raise above the caps Figure 2: Workflow of the RAIL classifier. The description is evaluated by several experts. Then, a chief expert uses the of the experts an provides a final level. source frameworks while remaining decidable from the information a description actually contains. 4 RAIL Classifier Assigning an AIRL to a natural-language description is, in principle, a task a single LLM could attempt in one pass [8, 26]: the rulebook of Section 3 could be supplied as an in- struction and the model asked for an integer. In preliminary experimentation, however, such monolithic prompting conflates three judgments that the framework deliberately keeps dis- tinct (the placement of the work on the environmental ladder, the detection of explicit gaps in the six readiness dimensions, and the application of the minimum rule that binds them) and it exhibits the two failure modes this conflation predicts. First, the model tends to hal- lucinate gaps [11], penalizing descriptions for dimensions about which they are merely silent, in violation of the neutrality-of-silence principle of Section 3.2. Second, and symmetrically, it tends to let strong experimental evidence argue past an explicitly stated deficiency, allow- ing, for example, an impressive benchmark result to outweigh an unresolved data-legality question that the framework treats as a hard cap. Both failure modes stem from performing gating and evidence assessment within a single, holistic judgment. We therefore operational- ize the classifier as a panel of experts: a set of specialized agents, each instantiated as a LLM with a narrowly scoped mandate, whose individual verdicts are combined by a deterministic 7 aggregation operator and reviewed by a presiding agent that issues the final verdict. The architecture mirrors the structure of the framework itself, so that each formal element of Section 3 is the explicit responsibility of exactly one component of the system. 4.1 Panel composition The panel comprises eight assessing agents and one deterministic operator, organized in three stages. In the first stage, a single evidence expert and six dimension experts examine the description independently and in parallel; none of them sees the output of any other. In the second stage, a deterministic aggregator, implemented in ordinary program logic rather than as a language model, combines their verdicts through the minimum rule. In the third stage, a chief expert receives the complete panel reportâevery individual verdict together with the aggregated recommendationâand pronounces the final classification under an asymmetric authority constraint described below. The independence of the first stage is essential: because each dimension expert evaluates the description in isolation, a gap detected in one dimension cannot contaminate the judgment of another, and the evidence expertâs placement on the ladder cannot be biased by knowledge of pending caps. This design corresponds to the epistemic situation the framework assumes, in which each readiness dimension is a property of the described work that either is or is not explicitly evidenced, independently of the others. 4.2 The evidence expert The evidence expert is responsible solely for the environmental ladder of Section 3.1. Given the description, it identifies the most advanced evaluation environment for which demon- strated evidence exists (from paper studies through synthetic experimentation, benchmark testbeds, integrated laboratory pipelines, real-data validation, capability demonstration, op- erational piloting, and qualification, to monitored production) and returns a provisional level in 1,..., 9, together with the decisive evidence, quoted or closely paraphrased from the description. Because several of the assignment disciplines of Section 3.3 require a holistic reading of the text rather than a dimensional one, they are assigned to this agent: the compo- sitional minimum over essential components (R1), the exclusion of aspirational claims from the evidence base (R2), the non-inheritance of maturity from pretrained or off-the-shelf con- stituents (R5), the anchoring of the level to the claimed generality (R6), and the treatment of reworked, previously deployed systems at the current state of the reworked component (R7). Ambiguity between adjacent levels is resolved downward by instruction (R3). The evidence expert additionally emits two Boolean flags consumed later by the aggregator: whether the described application belongs to a regulated domain, and whether regulatory certification or approval is explicitly stated. It does not, by construction, evaluate any of the six readiness dimensions. 4.3 The dimension experts Each of the six dimension experts is the specialist for exactly one dimension of Section 3.2: the specification of the use case and its success criteria (D 1 ), the existence of and access to the required data (D 2 ), the understanding of data format and the measurement of data quality including bias and imbalance (D 3 ), the legality of data use (D 4 ), the capture of required expert knowledge (D 5 ), and the algorithmic dimension covering selection, evaluation on real data, and tuning (D 6 ). A dimension expert does not output a level; it outputs a cap c i â 9, 6, 4, where 9 denotes the absence of any cap, and 6 and 4 denote the two severities of explicitly evidenced deficiency defined by the framework. Two disciplines govern these agents and are enforced by both instruction and interface. The first is the neutrality of silence: an expert may impose a cap only upon affirmative textual evidence of a gap, and a description that does not mention the expertâs dimension must receive c i = 9; the criteria that qualify as affirmative evidence at each severity are enumerated verbatim in the agentâs mandate, taken from the dimensional table of the framework. The second is evidentiary traceability: any cap must be accompanied by a verbatim quotation of the passage that triggered it, which is preserved in the classification record. At the interface level, the system adopts a fail-neutral policy: a malformed or unparsable expert response is interpreted as c i = 9 rather than as a cap, so that infrastructure faults can never manifest as spurious readiness penalties. 8 4.4 Deterministic aggregation The verdicts of the first stage are combined without recourse to a language model. Let â â 1,..., 9 denote the provisional level returned by the evidence expert, c 1 ,...,c 6 the dimensional caps, r â 0, 1 the regulated-domain flag, and z â 0, 1 the certification flag. The aggregated recommendation is â â = min â, c 1 , ..., c 6 , Ï , Ï = ( 7 if r = 1 and z = 0, 9 otherwise, (1) where Ï implements the regulated-domain discipline (R9), under which deployment-level language in a regulated domain cannot yield a level above seven in the absence of stated certification. Equation (1) is the direct formalization of the minimum principle inherited from Eljasik-Swoboda et al. [9] and of the system-level composition rule of Lavin et al. [13]. Implementing it as ordinary program logic rather than as a prompted judgment yields three properties that a monolithic classifier cannot offer. The rule is exact, in that the binding constraint is computed rather than estimated; it is auditable, in that the output records which caps were binding, each with its quoted evidence; and it is immune to persuasion, in that no rhetorical property of the input text can influence the arithmetic that combines the panelâs findings. 4.5 The chief expert and the final verdict The final stage submits the complete deliberation to a presiding agent, the chief expert, which receives the description, the evidence expertâs placement with its rationale and applied rules, every dimensional verdict with its evidence or its declared silence, and the aggregated recommendation â â . The chief expert fulfils two functions. The first is quality control over the panel itself: reviewing whether the evidence expert credited aspirational claims, overlooked a compositional-minimum or generality-anchoring situation, or over-read weak evidence, and whether any dimensional cap was granted too leniently. The second is the production of the final justification: a concise statement, suitable as a dataset label rationale, citing the decisive evidence and any binding cap. Crucially, the chief expertâs authority is asymmetric. Denoting its proposed level by v, the final classification is AIRL = min v, â â ,(2) so that the presiding agent may confirm the panelâs recommendation or lower it, but may never raise it above the deterministic bound. The asymmetry is a soundness requirement rather than a heuristic. The dimensional caps of Equation (1) are hard constraints of the framework: a description whose data legality is explicitly unresolved is at most AIRL 4 re- gardless of any other merit, and permitting a final holistic judgment to override that bound would reintroduce, at the last stage, precisely the argue-past failure mode that the panel architecture was designed to eliminate. The constraint is therefore enforced twice, once as instruction within the chief expertâs mandate and once as the clamp of Equation (2) in pro- gram logic, so that soundness does not depend on instruction-following alone. Disagreement is nonetheless informative: when the chief expert dissentsâlowering the verdict, or objecting to a cap it must nevertheless respectâthe dissent is recorded verbatim in the classification output. Over a corpus, the dissent record functions as a diagnostic of the panel, indicating which agentsâ mandates require refinement, in a manner analogous to the review switchbacks through which MLTRL feeds gate-level disagreement back into the development process [13]. 4.6 Properties of the architecture The resulting classifier is a function from descriptions to 1,..., 9 with properties that follow from its construction rather than from empirical tuning. It is sound with respect to the framework, in the sense that no output can exceed the bound implied by the evidenced environmental level and the explicitly evidenced dimensional gaps, by Equations (1) and (2). It is conservative, since ambiguity is resolved downward at the evidence stage and the final stage can only confirm or lower. It is neutral under silence, because caps require affirmative quoted evidence and interface faults default to the absence of a cap. It is auditable, since every 9 classification carries the provisional level, the set of binding caps with their quotations, the applied assignment rules, and any dissent, which together constitute a complete justification trail; when the classifier is used to produce training data for a distilled single-model classifier, this trail supplies rationale annotations at no additional cost. Finally, the architecture is modular : the first-stage agents are mutually independent and may be executed concurrently, individual mandates may be revised without retraining or re-prompting the remainder of the panel, and the deterministic core guarantees that such revisions alter only the judgments delegated to language models, never the logic that binds them. 5 Experiments For the evaluation of the proposed AIRL protocol and the RAIL classifier, we conducted a combinatorial experiment in which each configuration is defined by a readiness-level protocol and a classification strategy. âą Protocols â TRL. The original Technology Readiness Level (TRL) protocol proposed by Mank- ins [18], encoded as a single prompt. â AI TRL. The AI-specific adaptation proposed by Martinez-Plumed et al. [21], encoded as a single prompt. â AIRL. The readiness-level protocol proposed in this work. âą Classification strategies â Monolithic LLM (Mono). A single large language model assigns the readiness level in a single inference using the corresponding protocol description as a prompt. â RAIL. The proposed multi-agent architecture, in which a panel of expert agents collaboratively determines the readiness level. This strategy is only applicable to AIRL. The evaluation corpus consists of masterâs and doctoral theses developed at CIDETEC. Since no publicly available dataset pairs technical descriptions of AI systems with expert- assigned readiness levels, we constructed our own corpus from institutional documents for which we have the appropriate usage rights. For each thesis, we extracted the title, abstract, experimental summary, and conclusions, as these sections contain the most relevant evidence for readiness assessment. The experi- mental sections were summarized using Gemini Pro to reduce their length while preserving the essential technical information. All prompts, source data, and implementation details are available in our repository https://github.com/irvingvasquez/RAIL. Experiments were executed locally on a work- station equipped with an Intel Core i9 processor, an NVIDIA RTX 4090 GPU, and the Ollama implementation of Qwen3:32B. Since no benchmark dataset with ground-truth readiness labels exists, conventional ac- curacy metrics cannot be computed. Instead, the results in Table 2 are analyzed from three complementary perspectives: (i) the agreement among protocols and classification strate- gies, (i) the internal deliberation of the RAIL architecture through the justification traces generated by its expert panel, and (i) the computational cost of the multi-agent approach. The selected theses also provide face validity for the evaluation, as academic research projects typically conclude at the proof-of-concept or prototype stage rather than operational deployment. Consequently, readiness levels concentrated in the lower-to-middle range are consistent with the expected maturity of the evaluated works. 5.1 Agreement structure across configurations The four configurations produce markedly different distributions over the same ten docu- ments. The original TRL protocol under a monolithic classifier yields a mean of 4.7 (range 4â6), the AI-adapted TRL of Mart Ìınez-Plumed et al. [21] a mean of 6.2 (range 4â7, with seven as the modal value, assigned to six of the ten theses), the AIRL under a monolithic classifier a mean of 4.7 (range 3â5, with five assigned to eight documents), and the full RAIL configuration a mean of 4.5 (range 3â5). Three regularities stand out. 10 WorkTRL [18] (Mono) IA TRL [21] (Mono) AIRL (Mono) AIRL (RAIL) Alvarez 20234754 Gante 20234533 Brito 20246744 Vasquez 20095755 Mendoza 20184454 Vazquez 20195655 Rodriguez 20196755 Silva 20234555 Olguin 20234755 Lopez 20185755 Average4.76.24.74.5 Table 2: Experimentation results. Several master and PhD thesis weree analyzed by automatic technoloy readiness classifiers. Mono: monolitic LLM. First, the AI-adapted TRL under monolithic prompting is systematically the most gen- erous configuration: it exceeds the original TRL on nine of ten documents, by +1.5 levels on average, and it places the majority of the corpus at TRL 7, the operational-prototype level. The per-document justifications explain the mechanism. For the earliest thesis in the corpus (Vasquez 2009), the classifier assigns TRL 7 on the grounds that the algorithm reconstructed a single physical object with a real stereoscopic camera, while simultaneously acknowledg- ing that the real-world test âwas limited to a single object, introducing uncertainty about broader generality.â A single-instance laboratory demonstration is thereby promoted to the operational environment, and the generality reservation, although correctly noticed, is ex- pressed as commentary rather than as a level reduction. This is precisely the argue-past failure mode anticipated in Section 4: within a single holistic judgment, salient positive evidence (a real camera, a real object) outweighs a constraint (generality anchoring) that the prompt states but nothing enforces. That masterâs theses are not, as a population, one step from deployment suggests that these assignments overestimate maturity; the systematic +1.7-level gap between this configuration and RAIL quantifies the inflation. Second, the AIRL protocol under a monolithic classifier exhibits the opposite pathology in attenuated form: compression toward the center of the scale. Eight of ten documents receive level 5, and the dimensional apparatus almost never fires. The monolithic classifier, asked to scan six dimensions and apply the minimum rule within one generation, tends to report all caps as neutral. The comparison with RAIL on the same protocol is instructive: the panel lowers two of the ten labels (from 5 to 4 in both cases) and equals the monolithic label on the remaining eight, so the two configurations agree within one level on the entire corpus, yet the disagreements are exactly the documents where an explicit dimensional gap exists in the text. For the tomato detection thesis (Alvarez 2023), the monolithic classifier assigns level 5 with no caps, whereas RAILâs independent D 3 expert locates and quotes the passage âel conjunto de validaci Ìon real carec Ìıa de balance en sus clasesâ (an explicitly stated, unmitigated class imbalance in the real validation set) and caps the label at 4. Decomposition thus increases the recall of explicitly stated gaps: the same model that overlooks the passage when judging everything at once retrieves it verbatim when its only mandate is data quality. See It is worth noting that the quotation is returned in Spanish from an English-language prompt, indicating that the evidentiary-traceability mechanism operates across the corpusâs bilingual composition. Third, RAIL is empirically conservative with respect to its monolithic counterpart: it never exceeds the monolithic AIRL label on any document. This is consistent with, though not implied by, the construction and it corroborates the design intent that the panel archi- tecture removes upward pressure rather than adding downward noise. 11 D1: Specification D2: Data Existence D3: Data Quality (Class Imbalance) D4: Data Legality D5: Expert KnowledgeD6: Algorithmic Maturity 12345 Provisional AIRL Final AIRL Figure 3: AIRL Multi-Agent Evaluation: The Alvarez thesis initially received a readiness score of 5. However, the Data Quality Expert identified unhandled class imbalance within the dataset. As a result, the final AIRL score was capped at 4. 5.2 Panel-internal audit The per-agent records of the RAIL classifications permit an audit that no monolithic config- uration supports, and the audit yields both confirmations and a finding of genuine method- ological value. On the confirmation side, the simulation-only thesis (Gante 2023) is the panelâs cleanest case: the evidence expert places it at level 3, correctly registering R2 and R5, all six dimen- sion experts return neutral caps, and the chief expert confirms with an empty dissent. The cloud-deployed segmentation thesis (Brito 2024) exercises the opposite path: the evidence expert reads the public deployment of the model on a serverless platform as an operational prototype and proposes level 7 (itself a debatable over-read, since a publicly reachable infer- ence endpoint is not a pilot with real users) but the deterministic layer caps the label at 4, and the chief expertâs dissent field records exactly the tension the architecture is designed to surface: the deployment suggests operational proximity, yet the binding cap and the gen- erality anchoring override it. The dissent record thus functions as intended, preserving a disagreement that a single scalar label would erase. Across the audited classifications the chief expert confirmed the deterministic recommendation in every case and never lowered it, so the over-eager-chief failure mode did not materialize in this sample. 5.3 Computational cost Decomposed deliberation multiplies inference. On the experimental platform (a single RTX 4090 running Qwen3-32B through Ollama, sequential execution, fixed seed), the monolithic base- line classifies a document in approximately 22 seconds at a mean of roughly 4,000 prompt- plus-completion tokens, while a full RAIL classification requires approximately 131 seconds per document, a factor of six. For the intended use of the classifier, the production of audited training labels in batch, the cost is incurred once per document and buys the justification trail analyzed above; for interactive use, the monolithic AIRL configuration with the deter- ministic post-check of Section 5.2 offers a defensible low-cost operating point, at the price of the reduced cap recall documented in Section 5.1. 5.4 Threats to validity The principal limitations of this evaluation are its scale and the absence of ground truth. Ten documents from a single institution, in a single genre (graduate theses in robotics and applied computer vision, in Spanish and English), classified by a single model at a single seed, support an analysis of agreement structure and mechanism, not a claim of accuracy; 12 the systematic inflation of the AI-adapted TRL baseline, for example, is established relative to the face validity of the corpus rather than against expert labels. The summarization of the experimental sections by a separate large language model introduces a further mediation between the theses and the classifiers, common to all configurations but unquantified. 6 Conclusions In this study we have addressed the problem of assessing the maturity of artificial intelligence technologies from natural-language descriptions. We introduced the Unified AI Readiness Level (AIRL), a nine-level ordinal scale that integrates previous protocols. Second, we pro- posed RAIL, a panel-of-experts classifier that operationalizes the scale. The experimental evaluation over a corpus of graduate theses confirmed the failure modes that motivated the architecture. A monolithic classifier prompted with an AI-adapted TRL rubric systematically inflated maturity, exceeding the original TRL baseline; a monolithic classifier prompted with the full AIRL rulebook compressed its outputs toward the center of the scale and almost never activated the dimensional caps. In contrast, the proposed RAIL never exceeded its monolithic counterpart, and its independent dimension experts recovered explicitly stated gap that the same underlying model overlooked when judging holistically. The dissent mechanism preserved informative disagreements between the chief expert and the deterministic bound, providing a diagnostic signal that a single scalar label would erase. These benefits come at a computational cost of roughly six times that of a single inference, a price that is justified for the batch production of audited training labels, while the monolithic AIRL configuration with a deterministic post-check remains a defensible low-cost alternative for interactive use. The evaluation also delimits the scope of our claims. The corpus comprises ten docu- ments from a single institution and genre, classified by a single model at a single seed, and no expert-assigned ground truth is available; consequently, our results establish agreement structure and mechanism rather than accuracy. Future work will proceed along three lines: the construction of a larger, multi-institutional corpus with expert-labeled readiness levels to enable proper accuracy and inter-rater agreement studies. References [1] Muna Alfadhli, Nuri C. Onat, Murat Kucukvar, and Somaya Al-Maadeed. Analyzing AI readiness through digital transformation and data management: A case study of Qatarâs government sector. Applied Mathematics & Information Sciences, 19(3):497â507, 2025. [2] Jeremy Balch, Gilbert R Upchurch, Azra Bihorac, and Tyler J Loftus. Bridging the artificial intelligence valley of death in surgical decision-making. Surgery, 169(4):746â 748, 2021. [3] Juan Betancourt, Andr Ìes Coral, Anabel Fraga, Cristhian Figueroa, and Gustavo Ramirez-Gonzalez. Intelligent virtual assistant for calculating technology readiness lev- els using large language models (LLM). IEEE Access, 13, 2025. [4] S. Tucker Browne and Mark M. Bailey. Rethinking technological readiness in the era of AI uncertainty. arXiv preprint arXiv:2506.11001, 2025. [5] Samuel T. Browne, Thomas D. Pike, and Mark M. Bailey. A proposed framework for artificial intelligence safety and technology readiness assessments for national security applications. OSF Preprints, 2024. [6] Nataliya Chukhray, Nataliya Shakhovska, Oleksandra Mrykhina, Lidiya Lisovska, and Ivan Izonin. Stacking machine learning model for the assessment of R&D productâs readiness and method for its cost estimation. Mathematics, 10(9), 2022. [7] Jehan Dastoor, Heying Zhang, Michael G. Balchanos, and Dimitri N. Mavris. A biblio- metric approach to characterizing technology readiness levels using machine learning. In AIAA SCITECH 2023 Forum, 2023. [8] Ning Ding, Yujia Qin, Guang Yang, Fuchao Wei, Zonghan Yang, Yusheng Su, Shengding Hu, Yulin Chen, Chi-Min Chan, Weize Chen, et al. Parameter-efficient fine-tuning of 13 large-scale pre-trained language models. Nature machine intelligence, 5(3):220â235, 2023. [9] Tobias Eljasik-Swoboda, Christian Rathgeber, and Rainer Hasenauer. Assessing tech- nology readiness for artificial intelligence and machine learning based innovations. In Proceedings of the 8th International Conference on Data Science, Technology and Ap- plications (DATA), pages 281â288, 2019. [10] Grazia Garlatti Costa, Roberto Pugliese, and Francesco Venier. Exploring artificial intelligence adoption among Italian firms: the AI readiness level. International Journal of Business Information Systems, 51(7), 2026. [11] Yixiao Huang, Hanlin Zhu, Tianyu Guo, Jiantao Jiao, Somayeh Sojoudi, Michael Jor- dan, Stuart J Russell, and Song Mei. Generalization or hallucination? understanding out-of-context reasoning in transformers. Advances in Neural Information Processing Systems, 38:139807â139855, 2026. [12] Bhavesh Mahender Jain and Deepak Kumar. Predicting tech readiness through bib- liometric analysis using unsupervised machine learning. In Proceedings of the XXXVI ISPIM Innovation Conference, 2025. [13] Alexander Lavin, Ciar Ìan M Gilligan-Lee, Alessya Visnjic, Siddha Ganju, Dava Newman, Sujoy Ganguly, Danny Lange, At Ìıl Ìım G Ìune ̧s Baydin, Amit Sharma, Adam Gibson, et al. Technology readiness levels for machine learning systems. Nature Communications, 13(1):6039, 2022. [14] Alexander Lavin, Ciar Ìan M. Gilligan-Lee, Alessya Visnjic, Siddha Ganju, Dava New- man, Sujoy Ganguly, Danny Lange, Atılım G Ìune ̧s Baydin, Amit Sharma, Adam Gibson, Stephan Zheng, Eric P. Xing, Chris Mattmann, James Parr, and Yarin Gal. Technol- ogy readiness levels for machine learning systems. Nature Communications, 13(1):6039, 2022. [15] Alexander Lavin and Gregory Renard. Technology readiness levels for AI & ML. arXiv preprint arXiv:2006.12497, 2020. [16] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton.Deep learning.nature, 521(7553):436â444, 2015. [17] John C. Mankins. Technology readiness levels: A white paper. Technical report, NASA, Office of Space Access and Technology, 1995. [18] John C Mankins et al. Technology readiness levels, 1995. [19] Nico Marquardt, Vladimir Choi, Charles Martyn-Dickens, Marelize Gorgens, Sam Math- ewlynn, Tobias Kurth, Philipp Bouteiller, and Lothar H Wieler. From pilot to practice: a scoping review protocol mapping the development of ai-enabled solutions for maternal health using technology readiness levels. BMJ open, 15(8):e105622, 2025. [20] Fernando Mart Ìınez-Plumed, Emilia G Ìomez, and Jos Ìe Hern Ìandez-Orallo. AI Watch: Assessing technology readiness levels for artificial intelligence. Technical report, Publi- cations Office of the European Union, Joint Research Centre, 2020. [21] Fernando Mart Ìınez-Plumed, Emilio G Ìomez-Guti Ìerrez, and Jos Ìe Hern Ìandez-Orallo. Ai watch: Assessing technology readiness levels for artificial intelligence. EUR 30401 EN, Publications Office of the European Union, Luxembourg, 2020. [22] Benedikt M Ìuller, Daniel Roth, and Matthias Kreimeyer. Reframing AI readiness: a multi-dimensional use case-centered AI readiness framework. In Proceedings of the De- sign Society, Volume 6: DESIGN 2026, 2026. [23] International Standar Organization. Space systems â definition of the technology readi- ness levels (trls) and their criteria of assessment, 2013. [24] Stanley R. Sadin, Frederick P. Povinelli, and Robert Rosen. The NASA technology push towards future space mission systems. Technical report, NASA, 1989. [25] Mahsa Sayyari, Hasti Karimi, Sina Baghi Keshtan, Atousa Saleknezhad, Reza Bemana, and Iraj Rezaie. Toward a unified technology readiness ladder for clinical artificial intelligence: A systematic review and Delphi synthesis. InfoScience Trends, 2025. 14 [26] Xiaofei Sun, Xiaoya Li, Jiwei Li, Fei Wu, Shangwei Guo, Tianwei Zhang, and Guoyin Wang. Text classification via large language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 8990â9005, 2023. [27] O. M. Yusufova and A. R. Nevredinov. An approach to applying soft computing models to recognize technology readiness levels in research and development (R&D) projects. In AIP Conference Proceedings, volume 2383, page 070006, 2022. 15