Paper deep dive
Schema-Guided Hierarchical Information Extraction and Semantic Evaluation Using Generative AI
Modhurita Mitra, Jan-Willem Versteeg, Maarten D. Schermer, Shiva Nadi Najafabadi, Marie L. De Bruin, Lourens T. Bloem
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 8/9/2026, 1:29:04 AM
Summary
The paper presents a schema-based framework for hierarchical information extraction from unstructured Health Technology Assessment (HTA) documents using generative AI (specifically Claude Opus 3). The method uses a JSON schema as an information model to guide zero-shot extraction in a single API call. It introduces a path-based semantic matching algorithm to evaluate extracted results against a human-generated gold standard, classifying matches as exact, semantic, useful, or non-match. The framework achieved an F1 score >90% for 12 out of 14 attributes, reducing extraction time by ~30 times compared to human experts.
Entities (7)
Relation Signals (6)
Schema-Guided Hierarchical Information Extraction → achievesmetric → F1 Score
confidence 95% · extract 12 out of 14 attributes with an F1 score of >90%
Schema-Guided Hierarchical Information Extraction → appliesto → HTA
confidence 95% · extract a number of attributes of interest from documents from health technology assessment (HTA) organisations
Schema-Guided Hierarchical Information Extraction → uses → Claude Opus 3
confidence 95% · using the generative AI model Claude Opus 3
Schema-Guided Hierarchical Information Extraction → operatesin → Zero-shot Mode
confidence 90% · performed in a single call to the model, in zero-shot mode
NICE → providesdatafor → Schema-Guided Hierarchical Information Extraction
confidence 90% · extract ... from documents published by the health technology assessment organisation NICE
Schema-Guided Hierarchical Information Extraction → utilizesalgorithm → Path-based Semantic Matching
confidence 90% · introduce a path-based semantic matching algorithm to align the nested, variable-cardinality attributes
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:We present a schema-based framework for extracting complex, structured information from unstructured text documents using generative AI, followed by automated semantic evaluation of the extracted information against a gold standard. The schema, serving as an information model encoding domain knowledge, provides a unified, systematic, and consistent framework for extraction of hierarchical, nested information, with attributes of variable cardinality, and subsequent evaluation of the results. Information extraction from a document is performed in a single call to the model, in zero-shot mode. In the evaluation step, we introduce a path-based semantic matching algorithm to align the nested, variable-cardinality attributes in the extracted results with those in the gold standard. We use generative AI for semantic comparison of the extracted and gold standard values of an attribute, and introduce a rubric to classify the result of the comparison, according to domain-specific considerations, as an exact, semantic, useful, or non-match. We were able to extract 12 out of 14 attributes with an F1 score of $>$90\% from documents published by the health technology assessment organisation NICE, using the generative AI model Claude Opus 3. The time needed to extract the attributes from a document was $\sim$30 times lower than the time taken by a human domain expert. We further demonstrate generalisability of this framework across different generative AI models and transferability across different HTA organisations and languages.
Tags
Links
- Source: https://arxiv.org/abs/2608.06167v1
- Canonical: https://arxiv.org/abs/2608.06167v1
Trouble viewing inline? Open PDF directly →
Full Text
55,707 characters extracted from source content.
Expand or collapse full text
Schema-Guided Hierarchical Information Extraction and Semantic Evaluation Using Generative AI Modhurita Mitra a,1 , Jan-Willem Versteeg b,2 , Maarten D. Schermer a,3 , Shiva Nadi Najafabadi a,4 , Marie L. De Bruin b,5 , Lourens T. Bloem b,6 a Research Engineering Team, Information and Technology Services, Utrecht University, Utrecht, The Netherlands b Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, Utrecht University, Utrecht, The Netherlands 1 m.mitra@u.nl, 2 j.versteeg@u.nl, 3 m.d.schermer@u.nl, 4 s.nadinajafabadi@u.nl, 5 m.l.debruin@u.nl, 6 l.t.bloem@u.nl Abstract—We present a schema-based framework for extract- ing complex, structured information from unstructured text documents using generative AI, followed by automated semantic evaluation of the extracted information against a gold standard. The schema, serving as an information model encoding domain knowledge, provides a unified, systematic, and consistent frame- work for extraction of hierarchical, nested information, with attributes of variable cardinality, and subsequent evaluation of the results. Information extraction from a document is performed in a single call to the model, in zero-shot mode. In the evaluation step, we introduce a path-based semantic matching algorithm to align the nested, variable-cardinality attributes in the extracted results with those in the gold standard. We use generative AI for semantic comparison of the extracted and gold standard values of an attribute, and introduce a rubric to classify the result of the comparison, according to domain- specific considerations, as an exact, semantic, useful, or non- match. We were able to extract 12 out of 14 attributes with an F1 score of>90% from documents published by the health technology assessment organisation NICE, using the generative AI model Claude Opus 3. The time needed to extract the attributes from a document was∼30 times lower than the time taken by a human domain expert. We further demonstrate generalisability of this framework across different generative AI models and transferability across different HTA organisations and languages. Index Terms—Generative AI, LLMs, information extraction, natural language processing, NLP, schema-based extraction, in- formation model, health technology assessment, HTA, evaluation of structured outputs I. INTRODUCTION The schema-based information extraction method that is the subject of this paper emerged from our efforts to extract a number of attributes of interest from documents from health technology assessment (HTA) organisations. The content of these documents is highly specialised and deep domain knowl- edge is required to interpret them, with the content and the structure of the documents differing from document to document, organisation to organisation, and over time. The structure of the desired attributes is complex and involves hier- archical relationships between attributes of variable cardinality (an attribute of variable cardinality is a list whose length is not known in advance). Extracting this kind of intricate, nuanced data from unstructured text is a difficult information extraction problem. In this proof-of-concept study carried out as a collaboration between HTA domain researchers and Research Engineers, we extracted these attributes using generative AI. We developed a unified schema-based framework to guide a generative AI model to perform information extraction as well as sub- sequent evaluation of the results. The schema serves as a single, shared information model [1]–[3] that encapsulates the domain knowledge needed for the information extraction, output formatting, and evaluation steps – thus orchestrating the entire end-to-end information extraction and evaluation pipeline. Figure 1 shows a schematic representation of the pipeline. Document Generative AI model Extracted attributes Generative AI model Evaluation results Schema Extraction instructions Evaluation instructions + Gold standard Fig. 1. A high-level overview of the end-to-end information extraction and evaluation pipeline, highlighting the central role of the schema. A human- generated gold standard is needed for the evaluation step. I. HEALTH TECHNOLOGY ASSESSMENT (HTA) Health Technology Assessment is the process of systemat- ically reviewing a health technology (e.g., a drug, a medical device, or a medical therapy) based on clinical evidence sum- marised in reports that include conclusions regarding various attributes (e.g. clinical effectiveness and cost-effectiveness of drugs) that affect the adoption of that health technology in patient care [4]. In the European Union, this process is largely carried out independently by each country, resulting in heterogeneous data in different languages and formats. To facilitate comparative HTA research, which is the comparison and analysis of HTA processes, practices, and recommendations across different countries and HTA organisations [5], one needs to extract, arXiv:2608.06167v1 [cs.AI] 6 Aug 2026 for a given health technology, the same attributes from these different documents in a standardised format. Due to the complexity of HTA documents and the deep domain expertise required to interpret them, the current standard practice for extracting attributes from HTA documents is to employ domain experts to perform this information extraction task manually. The scientific goal of this project is to create an Open Science database of a number of attributes of interest extracted from these documents in order to facilitate their downstream use by different stakeholders. We identified 14 research- relevant attributes, listed in Table I, that we wanted to extract from HTA documents for the purpose of comparative HTA research. The number of instances of each attribute can vary from document to document – for example, multiple drugs might be assessed in a document, or one drug might be assessed for treating multiple indications (medical conditions). These attributes have a nested structure – for example, each drug in the document has its own corresponding brand name, relative effectiveness assessment, and final recommendation. The nested structure of the attributes is illustrated in Figure 2. Our task was to extract these attributes from a set of HTA documents. TABLE I ATTRIBUTES TO BE EXTRACTED FROM HTA DOCUMENTS AttributeDescription HTA IDName of HTA organisation performing the assessment Treatment typeIs the technology being assessed a medicine (drug), device, or therapy? Assessment typeIs this the first assessment, a reassessment, or an indication broadening? Assessment dateWhen was the assessment finalised/published? Internal identifierCode or label identifying the document IndicationMedical condition for which the drug is assessed INNInternational nonproprietary name of assessed drug Brand nameBrand name of assessed drug ComparatorsDrug(s) with which the performance of the assessed drug is compared Relative effectiveness assessment outcome Outcome of the relative effectiveness assessment for this drug-indication combination Cost-effectiveness assessment outcome Outcome of the cost-effectiveness assessment for this drug-indication combination Final recommendationWhat is the final reimbursement recommendation for this drug-indication combination? Managed entry agreement Was any managed entry agreement proposed? Clinical restrictionsClinical restrictions stated in the recommendation I. RELATED WORK Information extraction is the extraction of structured data in machine-readable format from unstructured text data in digital format (e.g., [6]–[8]). The main approaches to perform information extraction tasks have traditionally fallen into one of two categories: rule-based methods, and machine learning- based methods [3], [9]. With the advent of large language models (LLMs), these models are increasingly being used to perform information extraction tasks. To improve performance for certain tasks in specialised fields, fine-tuning BERT [10] or BART [11] models and, more recently, open-weights generative AI models, with domain-specific data has become popular practice [12]. These fine-tuning methods require labelled training data, the techni- cal know-how for fine-tuning LLMs, and the infrastructure to schema hta_id treatment_type assessment_type assessment_date internal_identifier indications [] indication_name technologies [] inn brand_name comparators outcome_rea outcome_cea final_recommendation managed_entry_agreement clinical_restrictions Fig. 2. Schema structure for attributes to be extracted from HTA documents, illustrating the hierarchical nature of the desired attributes. Here denotes a dictionary and [] denotes a list. fine-tune and deploy the models. Thus this method is not easily accessible to domain experts with limited technical knowledge and infrastructural resources. On the other hand, there has been increasing interest in the development of novel prompting techniques to harness the language capabilities and knowledge inherent in gener- ative AI models. These techniques make it possible to use general-purpose generative AI models for a wide range of natural language processing tasks, including information ex- traction. Some such prompt engineering techniques are chain- of-thought prompting [13], LLM-as-a-judge [14], and zero-, few-, and many-shot prompting [15]–[17]. Structured information extraction via constrained decoding has become common practice in the past couple of years [18]– [21]. In constrained decoding, a response schema is passed to a generative AI API along with but separate from the prompt, and the AI produces a response consistent with the schema. LLMs fine-tuned on domain-specific materials science data have been used in a study to extract hierarchical attributes of variable cardinality [22]. The nesting was only one level deep in this study, and only an exact string match was performed during automated evaluation. In a study on populating knowledge bases, a schema for representing nested, hierarchical information was populated by recursively traversing it, and extracting the relevant informa- tion at each level by issuing a new prompt to a generative AI model [23]. An ontology was used for grounding and normalisation of the extracted information. However, this kind of grounding is not an evaluation of the inherent correctness of the extracted information – it is only validation of the existence of the extracted information in the ontology. In a study on information extraction from histopathology reports, a JSON schema was provided in the prompt to guide the extraction [24]. However, this JSON schema was flat – hierarchical, nested data with attributes of variable cardinality were not extracted in this study. IV. DATA The data consists of a set of HTA documents downloaded from the website of the National Institute for Health and Care Excellence (NICE) [25] in the United Kingdom. We chose 8 representative documents as our development set (also known as validation set in machine learning). In addi- tion, we manually created a gold standard for this development set. The development set consisted of the documents we used to craft and refine our prompts. The representative development set covered a diverse range of values of the attributes – some documents were initial assessments while others were reassessments, some values of the relative effectiveness, cost- effectiveness, and final recommendation were positive while others were negative, etc. In some documents, multiple drugs were assessed for the treatment of one indication, while in others, one drug was assessed for the treatment of multiple indications. The test set consisted of 50 randomly chosen HTA docu- ments, for which we again manually created a gold standard. We evaluated the performance of our method on this test set, against the gold standard. The gold standard for the test set contained a total of 57 indications, 60 technologies, and 837 instances of leaf-level attributes. A leaf-level attribute is one which does not have any further sub-attributes in the schema in Figure 2. V. METHOD Using our schema-based framework, we extract complex data – hierarchical, with attributes of variable cardinality – in a single model call, in zero-shot mode, from entire documents (20-40 pages long), and subsequently perform automated se- mantic evaluation of the extracted attributes. By single model call we mean that we make only one API call per document to the generative AI model in the extraction step; we send a single prompt to the model and it returns a single output containing the entire information to be extracted from the document, all at once. By zero-shot we mean that we do not provide examples in the prompt for in-context learning [15], [16]. We simply provide a schema which serves as an information model, as well as accompanying instructions, to a state-of-the-art generative AI model, Claude Opus 3 from Anthropic [26], [27], accessed via its public API [28]. A. Extraction 1) Methodology development: We initially used a Q&A method with the generative AI model Claude Opus 3 from Anthropic to extract the attributes of interest, instructing the model to return the extracted values in JSON object format [29]. This simple, flat JSON object structure turned out to not be applicable in a general manner to all documents. Some documents assessed one drug for treating multiple indica- tions, while others assessed multiple drugs for treating one indication. It became evident that we needed a nested JSON structure to represent this kind of hierarchical data structure with attributes of variable cardinality. We observed that a model could interpret a well-designed JSON schema functioning as an information model, and faithfully extract information that was consistent with this schema. We provided the schema in the prompt – providing a response schema separately to the API for structured extraction was not available for Claude models till 2025 [20], while we had started on schema-based extraction for this work in early 2024. A schema in the prompt provides a soft guideline for the output structure, whereas a response schema enforces the structure as a hard constraint. Nevertheless, every JSON output we obtained was schema-compliant, though we provided the JSON schema only in the prompt. 2) Model choice and settings: We chose to use the gen- erative AI model Claude Opus 3 from Anthropic because it produced the most accurate and consistent results [29]–[32] among the models we tried at the time we performed the extraction part of this study (2024-2025). The other models we tried were the GPT 3.5 [33] and GPT 4 [34] series of models from OpenAI. The temperature parameter in a generative AI model con- trols the randomness and variability of the outputs [35]. To make the results as deterministic and reproducible as possible, we set the temperature parameter to zero. The code, prompts, schema, and results can be found in the GitHub repository 1 for this project. 3) Prompt: The prompt consisted of three parts. The first part was the data – the text of the HTA document, which we obtained by using the PyPDF package [36] to convert the PDF documents to text format. The second part consisted of the instructions, which included setting the AI’s role, description of the task to be performed, and requirements regarding the output format. The third part was the schema, which specified the attributes to be extracted and the format for the output. B. Schema as information model The schema, 2 expressed in JSON Schema format, plays a central role in this work. It serves as an information model [1]–[3] encoding domain knowledge, and comes into play multiple times and in multiple ways during the extraction and evaluation processes: 1) Encoding domain information: The leaf-level keys in the schema specify the names of the attributes to be ex- tracted. The description field provides a description of the attribute to be extracted, and any other relevant domain knowledge about the attribute. Where applicable, the type, pattern, and enum fields provide constraints on the value corresponding to a given key in the output. The type field constrains the type of the output – for example, to “string” or “null”. The pattern field forces the value to follow a certain format, for example Y-M-D for dates. The enum field restricts the value of the output to one of a limited number of pre-specified values – for 1 https://github.com/UtrechtUniversity/hta-genai 2 The schema can be found here: https://github.com/UtrechtUniversity/ hta-genai/blob/main/config/schema.json example to “positive”, “negative”, or null. The enum field thus provides a value set [3], [37] accompanying the information model defined by the schema. The type field is also used to specify the nested, hierarchical, variable cardinality structure of the data – if the type of an attribute is set to “array”, that means that the attribute is a list of unknown length; if type is “object”, that indicates that the attribute is a dictionary with further sub-attributes. 2) Specifying output format: The schema specifies the desired output format – the format at the level of the individual key-value pairs for the attributes, as well as the format of the entire JSON object with the nested structure and attributes of variable cardinality. 3) Providing context during evaluation: During evaluation, we check if the extracted values match the gold standard values. If the values are not an exact string match, we use generative AI to check if they match semantically. For this semantic comparison, we provide the attribute name and the associated description field from the schema to provide context about the values being compared. Figure 3 shows an excerpt from the full schema. It shows the entry for one of the attributes to be extracted – assessment_type. "assessment_type": "type": ["string", "null"], "enum": ["initial assessment", "reassessment", " indication broadening", "non-submission", " unknown", null], "description": "Type of assessment being performed " Fig. 3. Entry for the field assessment_type in the schema For creating the schema, the HTA domain experts specified the attributes to extract, and their hierarchical, nested structure – they knew which attributes had variable cardinality and thus needed to have a list structure, and which attributes had further sub-attributes and thus needed to have a dictionary structure. They communicated this information to the Research Engineers who then designed the schema in JSON Schema format. The domain experts also provided the domain knowl- edge encapsulated in the schema, such as the information in the description, pattern, type, and enum fields. C. Evaluation The results are evaluated against a human-generated gold standard. Both the extracted results and the gold standard are generated in the format prescribed by the schema. The fact that the schema has a nested structure complicates the process of finding the corresponding elements in the extracted JSON object and the gold standard JSON object. Some non-leaf-level attributes such as indications and technologies are lists containing multiple elements. The number of elements in such a list in the extracted JSON object might be different from the number of elements in the corresponding list in the gold standard JSON object. Even if the same number of elements are present in such a list in both the extracted and gold standard JSON objects, the elements might be in a different order in each of these lists. Therefore, direct comparison of the extracted JSON object with the gold standard is a non-trivial task. Manually aligning the extracted output with the gold stan- dard and then comparing the corresponding attributes was extremely tedious, especially during the prompt engineering phase when we had to check the complicated nested JSON output manually each time we refined our prompt. This motivated us to develop an automated evaluation method. 1) Methodology: The evaluation process is illustrated in Figure 4. We approach the problem of identifying the corre- sponding elements in the extracted and gold standard JSON objects by finding the path to each leaf-level element in one JSON object, and then finding the matching path in the other JSON object. To test if two paths are matching, we have to compare them at each nesting level. Since the values of the corresponding attributes at some level of the two paths might not be an exact lexical match, we also need to check for a semantic match. For non-leaf attributes which are lists of dictionaries, such as indications and technologies, matching a dictionary list element (a list element which is a dictionary) in the extracted results to the corresponding dictionary list element in the gold standard requires us to choose an identifier attribute (or anchor attribute) in the dictionary. An identifier attribute is a dictionary item that defines or identifies a dictionary list element in a list of dictionaries. It must be a defining characteristic of the parent attribute. At the indications level in the schema in Figure 2, the identifier attribute is indication_name. At the technologies level, the pri- mary identifier attribute is inn which corresponds to the INN (international nonproprietary name) of the drug. If no INN is present, brand_name is used as the secondary identifier attribute. At each comparison step for a leaf-level attribute in Figure 4, we first check if the attribute is present in both the extracted results and the gold standard. One of the situations listed in Table I arises, and the match is classified as True Positive (TP), True Negative (TN), False Positive (FP), or False Negative (FN) according to this table. TABLE I CLASSIFICATION OF MATCH BETWEEN GOLD STANDARD AND EXTRACTED ATTRIBUTES Gold standardExtractedValuesClassification attribute present?attribute present?match? NoNo–TN NoYes–FP YesNo–FN YesYes YesTP NoFP When both the extracted and gold standard attributes are present, we first check if there is an exact match between the Gold standard, extracted results Compare top-level attributes Form best-matching 1:1 (gold standard, extracted results) indication pairs Select one indication pair Both indications non-null? Form best-matching 1:1 (gold standard, extracted results) technology pairs Select one technology pair Both tech- nologies non-null? INNs/ brand names match? Compare technology- level attributes More technology pairs? More indication pairs? Evaluation results Yes No Yes No Yes No Yes No Yes No Fig. 4. Flowchart illustrating the comparison of extracted attributes with the gold standard two values, using a simple string comparison. If this returns an exact match, we classify the result as TP. If the comparison does not result in an exact string match, we compare the two values using generative AI. We instruct the generative AI model to classify the result of the comparison into one of the following four categories: 1) Exact match: The string comparison does not return an exact match, but the values are almost exactly the same, with very minor differences – for example, an em dash is used instead of an en dash, or “alpha” is used instead of α. We classify this as TP. 2) Semantic match: The values are lexically different but semantically identical, i.e., they convey the same mean- ing using different words, in the natural language sense of paraphrase [38] – for example, if one of the values of indication_name is “pyrexia in children” and the other is “fever in kids”. We classify this as TP. 3) Useful match: The values are neither lexically nor se- mantically identical, but the information extracted is still useful for the purposes of comparative HTA research – for example, if one value of indication_name is “fever in children under 12 years of age” and the other is simply “fever”. We classify this as TP. Since HTA deals with reimbursement decisions and not clinical decisions, this classification is appropriate. 4) Non-match: The values are neither lexically identical nor semantically similar. We classify this as FP. Fortheattributeswhicharelistsofdictionaries (indications and technologies), in order to identify the corresponding dictionary list elements in the extracted data and the gold standard, we need to compare the corresponding identifier attributes. Multiple values of an identifier attribute in the two lists might match in some way (exact, semantic, or useful). We pick the best one-to-one match by ranking the matches by match type (exact > semantic > useful), and then choosing the match corresponding to the highest-ranked match type. In case there are multiple instances of the highest-ranked match type, we perform a greedy search [39] and choose the first match that we encounter. 2) Model choice: We chose the generative AI model Claude Opus 4.6 [40] for comparing the values when the comparison between the extracted results and the gold standard did not result in an exact string match. We chose this model because it was the most advanced model from Anthropic that was available at the time we performed the evaluation part of this study (March 2026). All the evaluations for the analyses listed in subsection V-D were performed using Claude Opus 4.6, for the sake of uniformity of evaluation over different models, runs, and organisations. D. Analyses performed We performed the following analyses: 1) Performance metrics: Computing performance metrics (precision, recall, accuracy, F1 score) for the extracted values, against the gold standard. 2) Consistency analysis: Testing the consistency of results in item 1 in this list over two runs with the same model, model parameters, prompt, and data. 3) Time, costs, scalability: Computing the time needed for extraction, the associated generative AI API costs, and how these quantities scale with number of documents. 4) Comparison with other methods: Comparing with rule- based and traditional machine learning methods. 5) Generalisability: Testing generalisability across different models. 6) Transferability: Testing transferability across different HTA organisations and languages. Table I shows an overview of the analyses performed, and the results of these analyses are reported in Section VI. TABLE I OVERVIEW OF ANALYSES, WITH MODELS USED FOR EXTRACTION AND EVALUATION, AND THE DATES ON WHICH THESE WERE PERFORMED AnalysisHTA organisationExtraction model (date)Evaluation model (date) Performance metricsNICEClaude Opus 3 (July 2025)Claude Opus 4.6 (March 2026) Consistency analysisNICEClaude Opus 3 (July 2025)Claude Opus 4.6 (March 2026) Time, costs, scalabilityNICEClaude Opus 3 (July 2025)N/A (only extraction time and costs compared) Comparison with other methods NICERule-based, machine learning (2024-2025)Evaluation performed manually (2024-2025) GeneralisabilityNICEClaude Opus 3, Claude Opus 4.6, Gemini 3.1 Pro, GPT OSS 120B (March 2026) Claude Opus 4.6 (March 2026) TransferabilityNICE, ZIN, HASClaude Opus 4.6 (March 2026)Claude Opus 4.6 (March 2026) VI. RESULTS A. Performance metrics Figure 5 shows the performance metrics (precision, recall, accuracy, F1 score) per attribute, and also over all attributes. We see that for 12 of the 14 attributes extracted from HTA doc- uments, all the metrics have a value > 90%. For two attributes, outcome_rea and clinical_restrictions, the per- formance is poorer – the F1 score is 0.88 for outcome_rea and 0.83 for clinical_restrictions. Extracting the attribute outcome_rea is a difficult task, even for a human. The relative effectiveness discussion in HTA documents is often long, nuanced, and ambiguous. When the document contains sentences along the lines of “This drug seems to be clinically effective, but the evidence presented is insufficient”, generative AI usually classified outcome_rea as positive, while the human expert usually classified it as negative in the gold standard. For the attribute clinical_restrictions, the reason for the relatively poor performance is the fact that this attribute is reported along with the attribute indication_name, in the same section of the HTA document – the “Rec- ommendations” section, making it difficult to decouple clinical_restrictions from indication_name. As a toy example, suppose that a drug is being as- sessed for treating fever in children under 12 years of age. The human expert might extract the two at- tributes as indication_name: “fever in children under 12 years of age”, clinical_restrictions: null, which will be the gold standard. Generative AI might extract the attributes as indication_name: “fever”, clinical_restrictions: “only for children under 12 years of age”. Since we also accept useful matches during the evaluation process, the extracted indication_name will be considered as matching the gold standard and the match will be classified as TP, but the extracted clinical_restrictions will not match the gold stan- dard and the match will be classified as FP. During the evaluation process, most (634 out of 796) of our comparisons against the gold standard yielded an exact match during the initial check for an exact string match, which is a rule-based, deterministic check. For the comparisons that did not yield an exact string match, we used the generative AI model Claude Opus 4.6 to classify the result of the comparison between the extracted and gold standard values as an exact match, semantic match, useful match, or non-match. We followed up the comparisons that were performed by generative AI with a manual check to ascertain if the AI had classified the result of each comparison correctly. Claude Opus 4.6 found 20 exact matches, 19 semantic matches, 84 useful matches, and 39 non-matches. The human and the AI agreed on all the non-matches. For the other matches that were classified as exact, semantic, or useful matches by the AI, the human sometimes classified them differently, but still as one of these three kinds of matches – not as a non-match. Since a non-match is classified as FP and exact, semantic, and useful matches are all classified as TP, this discrepancy is immaterial for the purposes of calculating the metrics. Thus we verified that the human and the AI agreed on the performance metrics, even though there was some discrepancy between the two regarding exact/semantic/useful match classification. B. Consistency analysis Due to the generative variability of generative AI models, the results from two runs with the same model, model parame- ters, prompt, and data may not necessarily be the same [41]. To examine the consistency (in the sense of test-retest reliability [42]) of our results over different runs, we performed two runs, and used the results from the first run as the gold standard for the second run. We calculated the performance metrics, and the values of all the metrics for all the attributes were > 96%. C. Time, costs, scalability On average, it took Claude Opus 3 ∼0.5 minute to extract the attributes from one HTA document, and the API cost was ∼0.5 euro per document. For comparison, it takes a domain expert ∼15 minutes to extract the attributes from one document. Thus there is a∼30-fold increase in time-efficiency when generative AI is employed instead of a human for this task. The time and API costs scale linearly with the number of documents. D. Comparison with other methods We compared our schema-based generative AI method to two traditional methods – rule-based and machine learning approaches. The details of the comparison are presented in a companion paper focused on the analysis and implications of these three approaches for comparative HTA research [43]. Here we present a summary of the results. hta id treatment type assessment type assessment date internal identifier indication name inn brand name comparators outcome rea outcome cea final recommendation managed entry agreement clinical restrictions OVERALL 0 0.2 0.4 0.6 0.8 0.9 1 Value PrecisionRecall Accuracy F1 score Fig. 5. Evaluation metrics per attribute, and over all attributes Since no existing traditional method that we are aware of can extract data which has the complex structure illustrated in Figure 2, in a single call, we do not have other methods to directly compare with, in terms of the entire output containing all the attributes. However, we could compare our method with existing traditional methods a few attributes at a time. Using a rule-based method, we were able to extract fourattributes– hta_id, assessment_date, internal_identifier,and indication_name with 90% accuracy, and assessment_type, inn, and brand_name with 70% accuracy. We were not able to extract the other attributes with significant accuracy. We tried several machine learning algorithms to classify the three attributes whose values could be broadly described as binary (positive/negative) – outcome_rea, outcome_cea, and final_recommendation. The methods we tried were Gradient Boosting, Support Vector Machine, Logistic Re- gression, Random Forest, Naive Bayes, and BERT. For the outcome_rea attribute, the best-performing models were Gradient Boosting and Random Forest with an F1 score of 0.76 for both models. For the outcome_cea and final_ recommendation attributes, the best-performing model was Gradient Boosting with F1 scores of 0.98 and 0.93 respectively. Comparing these metrics to those in Figure 5, we see that the generative AI approach outperforms both rule-based and machine learning approaches. In addition, this method can extract all the desired attributes from a document via a single call to the model. This makes the implementation simpler as compared to both the rule-based method, which needs the creation of a separate rule for each attribute, and the machine- learning methods, which need labelled training data for each attribute to be classified. E. Generalisability across models We used this schema-based framework to extract results using three other models – two proprietary models, Claude Opus 4.6 and Gemini 3.1 Pro [44], and the open-weights model GPT OSS 120B [45] accessed through the AI-Hub [46] provided by SURF [47], the Dutch IT cooperative for research and education. Claude Opus 4.6 was used as the model for the automated evaluation in all these cases. Figure 6 shows the F1 scores for these four models. It can be seen that we get comparable results from all four models, demonstrating the generalisability of this framework across different generative AI models. F. Transferability across HTA organisations and languages In Figure 7 we compare the performance of our pipeline designed for NICE documents, as-is, without modification, on 25 documents each from Zorginstituut Nederland (ZIN) [48], the Dutch HTA organisation, and Haute Autorit ́ e de Sant ́ e (HAS) [49], the French HTA organisation. Claude Opus 4.6 was used for both the extraction and evaluation steps. We see that our approach performs reasonably well for several attributes for ZIN and HAS too, thus illustrating the broad transferability of our schema-based framework across different HTA organisations and languages. VII. CONCLUSIONS AND DISCUSSION Using a schema-based framework, we have successfully ex- tracted data with a hierarchical structure, containing attributes of variable cardinality, from HTA documents. We have also evaluated the results in an automated manner. We used a single, unified schema and generative AI in both the extraction and evaluation steps. While we have integrated the extraction and evaluation steps together in the single end-to-end pipeline shown in Figure 1, the evaluation step is independent of the extraction step. The evaluation algorithm can be applied to any outputs that follow a prescribed schema, regardless of the extraction method, provided the schema and a gold standard which follows the same schema are available. Since the evaluation step needs a gold standard, the evaluation algorithm can only be used on the labelled development and test datasets, and not in production, on unseen data. Since we developed our prompt on a development set for NICE documents, using the generative AI model Claude Opus 3, we get the best performance for this combination of HTA organisation and generative AI model. While still broadly performing well, differences in the way different HTA organisations report assessments, as well as model differences between model providers and model drift between generations of models from the same provider, might cause the perfor- mance to degrade. Refining the prompt based on documents published by a particular HTA organisation, using a particular generative AI model, might improve performance. The apparently poorer performance of Claude Opus 4.6, a more advanced model than Claude Opus 3, is in some cases hta id treatment type assessment type assessment date internal identifier indication name inn brand name comparators outcome rea outcome cea final recommendation managed entry agreement clinical restrictions OVERALL 0 0.2 0.4 0.6 0.8 0.9 1 F1 score Claude Opus 3Claude Opus 4.6Gemini 3.1 ProGPT OSS 120B Fig. 6. F1 scores per attribute, and over all attributes, for the generative AI models Claude Opus 3, Claude Opus 4.6, Gemini 3.1 Pro, and GPT OSS 120B hta id treatment type assessment type assessment date internal identifier indication name inn brand name comparators outcome rea outcome cea final recommendation managed entry agreement clinical restrictions OVERALL 0 0.2 0.4 0.6 0.8 0.9 1 F1 score NICE ZIN HAS Fig. 7.F1 scores per attribute, and over all attributes, for the HTA organisations NICE, ZIN, and HAS. HAS does not report the attributes internal_identifier and outcome_cea. For managed_entry_agreement for HAS, either both the AI-extracted value and the gold standard value were null, or the AI did not classify the requirement of future assessments as an outcome-based managed entry agreement like the human expert did. due to the fact that Claude Opus 4.6 extracts the attributes at a higher degree of granularity than either the gold standard or the data extracted by Claude Opus 3. Thus it might be necessary to make the prompt more specific and explicitly specify the level of granularity desired in the output. We use a proprietary generative AI model, Claude Opus 3, for this information extraction task. We do not have access to the model itself, and only use it via its public API. Thus the availability, latency, and costs depend on the proprietary model provider and these parameters can change in the future. Of particular concern is the fact that proprietary model providers routinely deprecate older models in favour of new ones, which endangers not only the reproducibility of the results but also the long-term functionality and sustainability of the information extraction and evaluation pipeline. With open-weights models increasing in size and capability, the problems associated with the use of proprietary models can in principle be avoided by downloading and locally deploying state-of-the-art open-weights models. Figure 6 shows that the performance of the open-weights model GPT OSS 120B is comparable to that of state-of-the-art proprietary models. Thus this method can be used for information extraction from private and/or sensitive documents, provided an organisation possesses the technical expertise and infrastructure for hosting such open-weights models locally. The gold standard dataset that we used was created by a single domain expert. Manual extraction of attributes by humans is itself prone to subjectivity and error [50], thus, for a more robust assessment of the performance of this method, one should obtain gold standard datasets created by multiple domain experts for the same set of HTA documents, and perform inter-rater reliability analysis [29], [51] to compare the agreement between multiple human raters, and that between human raters and AI. While we have used generative AI to automate the te- dious extraction and evaluation steps, human expertise was indispensable in this study. HTA domain experts provided the domain knowledge about the attributes to extract and their structure, and they prepared the gold standards. The Research Engineers designed the schema and wrote the scaffolding code in Python to programmatically extract the attributes via the generative AI API. The match classifications performed by generative AI during the evaluation step were manually verified by humans. Thus human expertise and involvement remain essential when the results need to be highly valid [31], [52], like in this use case – comparative HTA research. VIII. FUTURE WORK We are using this schema-based framework for extracting complex information in two domains very different from HTA – a scoping review of young people’s geographies, which is from the human geography domain, and extracting sociode- mographic attributes from the Oxford Dictionary of National Biography [53], which is from the historical sociology domain. The preliminary results from these two projects are promising. ACKNOWLEDGEMENT We thank Martine de Vos and Ingo Schr ̈ oder for reading the manuscript and providing feedback that improved the paper. Generative AI use disclosure: Generative AI was used for coding assistance to write the pipeline, and to generate the L A T E X code for the tables, figures, and references in this paper. The generative AI tools used were OpenAI’s GPT series of models, Anthropic’s Claude series of models, and Google’s Gemini series of models. All the content was reviewed care- fully by the authors and the authors take full responsibility for the final content of the manuscript. REFERENCES [1] D.Sutherland,“Amodelofinformation,”inProceedings ofthe9thNationalComputerSecurityConference,vol. 247.Washington,DC,1986,p.175–183.[Online]. Available:https://csrc.nist.gov/files/pubs/conference/1986/09/ 15/proceedings-9th-national-computer-security-confere/final/docs/ 1986-9th-ncsc-proceedings.pdf [2] International Organization for Standardization, “Health informatics — clinical information models — characteristics, structures and requirements,” ISO, International Standard ISO 13972:2022, February 2022, standard number: 79498. [Online]. Available: https://w.iso. org/standard/79498.html [3] Y. Wang, L. Wang, M. Rastegar-Mojarad, S. Moon, F. Shen, N. Afzal, S. Liu, Y. Zeng, S. Mehrabi, S. Sohn, and H. Liu, “Clinical information extraction applications: A literature review,” Journal of Biomedical Informatics, vol. 77, p. 34–49, 2018. [Online]. Available: https://w.sciencedirect.com/science/article/pii/S1532046417302563 [4] B. O’Rourke, W. Oortwijn, T. Schuller, and International Joint Task Group, “The new definition of health technology assessment: A milestone in international collaboration,” International Journal of Technology Assessment in Health Care, vol. 36, no. 3, p. 187–190, 2020, epub 2020 May 13. [Online]. Available: https: //pubmed.ncbi.nlm.nih.gov/32398176/ [5] R. A. Vreman, A. K. Mantel-Teeuwisse, A. M. H ̈ ovels, H. G. Leufkens, and W. G. Goettsch, “Differences in Health Technology Assessment Recommendations Among European Jurisdictions: The Role of Practice Variations,” Value in Health, vol. 23, no. 1, p. 10–16, 2020. [Online]. Available: https://w.sciencedirect.com/ science/article/pii/S1098301519323411 [6] J. Cowie and W. Lehnert, “Information extraction,” Commun. ACM, vol. 39, no. 1, p. 80–91, Jan. 1996. [Online]. Available: https://doi.org/10.1145/234173.234209 [7] S. Sarawagi, “Information extraction,” Found. Trends Databases, vol. 1, no. 3, p. 261–377, Mar. 2008. [Online]. Available: https: //doi.org/10.1561/1900000003 [8] J. Piskorski and R. Yangarber, Information Extraction: Past, Present and Future. Berlin, Heidelberg: Springer Berlin Heidelberg, 2013, p. 23– 49. [Online]. Available: https://doi.org/10.1007/978-3-642-28569-12 [9] D. Xu, W. Chen, W. Peng, C. Zhang, T. Xu, X. Zhao, X. Wu, Y. Zheng, Y. Wang, and E. Chen, “Large language models for generative information extraction: a survey,” Frontiers of Computer Science, vol. 18, no. 6, p. 186357, 2024. [Online]. Available: https://doi.org/10.1007/s11704-024-40555-y [10] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), J. Burstein, C. Doran, and T. Solorio, Eds.Minneapolis, Minnesota: Association for Computational Linguistics, Jun. 2019, p. 4171–4186. [Online]. Available: https://aclanthology.org/N19-1423/ [11] M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, and L. Zettlemoyer, “BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, D. Jurafsky, J. Chai, N. Schluter, and J. Tetreault, Eds.Online: Association for Computational Linguistics, Jul. 2020, p. 7871–7880. [Online]. Available: https://aclanthology.org/2020.acl-main.703/ [12] Q. Chen, Y. Hu, X. Peng, Q. Xie, Q. Jin, A. Gilson, M. B. Singer, X. Ai, P.-T. Lai, Z. Wang, V. K. Keloth, K. Raja, J. Huang, H. He, F. Lin, J. Du, R. Zhang, W. J. Zheng, R. A. Adelman, Z. Lu, and H. Xu, “Benchmarking large language models for biomedical natural language processing applications and recommendations,” Nature Communications, vol. 16, no. 1, p. 3280, Apr 2025. [Online]. Available: https://doi.org/10.1038/s41467-025-56989-2 [13] J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. H. Chi, Q. V. Le, and D. Zhou, “Chain-of-thought prompting elicits reasoning in large language models,” in Proceedings of the 36th International Conference on Neural Information Processing Systems, ser. NIPS ’22.Red Hook, NY, USA: Curran Associates Inc., 2022. [Online]. Available: https://dl.acm.org/doi/10.5555/3600270.3602070 [14] L. Zheng, W.-L. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. P. Xing, H. Zhang, J. E. Gonzalez, and I. Stoica, “Judging LLM-as-a-judge with MT-bench and Chatbot Arena,” in Proceedings of the 37th International Conference on Neural Information Processing Systems, ser. NIPS ’23.Red Hook, NY, USA: Curran Associates Inc., 2023. [Online]. Available: https://dl.acm.org/doi/10.5555/3666122.3668142 [15] T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa, “Large language models are zero-shot reasoners,” in Proceedings of the 36th International Conference on Neural Information Processing Systems, ser. NIPS ’22.Red Hook, NY, USA: Curran Associates Inc., 2022. [Online]. Available: https://dl.acm.org/doi/10.5555/3600270.3601883 [16] T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei, “Language models are few-shot learners,” in Proceedings of the 34th International Conference on Neural Information Processing Systems, ser. NIPS ’20.Red Hook, NY, USA: Curran Associates Inc., 2020. [Online]. Available: https://dl.acm.org/doi/abs/10.5555/3495724.3495883 [17] R. Agarwal, A. Singh, L. Zhang, B. Bohnet, L. Rosias, S. Chan, B. Zhang, A. Anand, Z. Abbas, A. Nova, J. D. Co-Reyes, E. Chu, F. Behbahani, A. Faust, and H. Larochelle, “Many-shot in-context learning,” in Proceedings of the 38th International Conference on Neural Information Processing Systems, ser. NIPS ’24.Red Hook, NY, USA: Curran Associates Inc., 2024. [Online]. Available: https://dl.acm.org/doi/10.5555/3737916.3740363 [18] B. T. Willard and R. Louf, “Efficient guided generation for large language models,” 2023. [Online]. Available: https://arxiv.org/abs/2307. 09702 [19] M. Pokrass, “Introducing structured outputs in the API,” OpenAI Blog, Aug. 2024. [Online]. Available: https://openai.com/index/ introducing-structured-outputs-in-the-api/ [20] Anthropic, “Structured outputs on the Claude Developer Platform,” Claude Blog, Nov. 2025. [Online]. Available: https://claude.com/blog/ structured-outputs-on-the-claude-developer-platform [21] S. Geng, H. Cooper, M. Moskal, S. Jenkins, J. Berman, N. Ranchin, R. West, E. Horvitz, and H. Nori, “JSONSchemaBench: A Rigorous Benchmark of Structured Outputs for Language Models,” 2025. [Online]. Available: https://arxiv.org/abs/2501.10868 [22] J. Dagdelen, A. Dunn, S. Lee, N. Walker, A. S. Rosen, G. Ceder, K. A. Persson, and A. Jain, “Structured information extraction from scientific text with large language models,” Nature Communications, vol. 15, no. 1, p. 1418, Feb 2024. [Online]. Available: https: //doi.org/10.1038/s41467-024-45563-x [23] J. H. Caufield, H. Hegde, V. Emonet, N. L. Harris, M. P. Joachimiak, N. Matentzoglu, H. Kim, S. Moxon, J. T. Reese, M. A. Haendel, P. N. Robinson, and C. J. Mungall, “Structured Prompt Interrogation and Recursive Extraction of Semantics (SPIRES): a method for populating knowledge bases using zero-shot learning,” Bioinformatics, vol. 40, no. 3, p. btae104, Feb 2024. [Online]. Available: https://doi.org/10.1093/bioinformatics/btae104 [24] J. B. Balasubramanian, D. Adams, I. Roxanis, A. B. de Gonzalez, P. Coulson, J. S. Almeida, and M. Garc ́ ıa-Closas, “Leveraging large language models for structured information extraction from pathology reports,” Journal of Pathology Informatics, vol. 19, p. 100521, 2025. [Online]. Available: https://w.sciencedirect.com/science/article/pii/ S2153353925001075 [25] NationalInstituteforHealthandCareExcellence,“Tech- nologyappraisal:Guidance,qualitystandardsandadvice,” https://w.nice.org.uk/guidance/published?ngt=Technology% 20appraisal%20guidance&ndt=Guidance&ps=9999,NICE,London, UK, 2025, accessed: August 7, 2026. [26] AnthropicAI,“Claude3modelcard,”2024,accessed: 2024-04-17.[Online].Available:https://w-cdn.anthropic.com/ de8ba9b01c9ab7cbabf5c33b80b7bbc618857627/Model CardClaude 3.pdf [27] Anthropic, “Introducing the Next Generation of Claude,” Mar. 2024, Claude 3 model family. [Online]. Available: https://w.anthropic. com/news/claude-3-family [28] Anthropic AI, “API reference,” May 2024, accessed: 2024-05-21. [Online]. Available: https://docs.anthropic.com/en/api/complete [29] M. Mitra, M. G. de Vos, N. Cortinovis, and D. Ometto, “Generative AI for Research Data Processing: Lessons Learnt From Three Use Cases,” in 2024 IEEE 20th International Conference on e- Science (e-Science), 2024, p. 1–10. [Online]. Available: https: //ieeexplore.ieee.org/document/10678704 [30] G. Cong, W. Fan, F. Geerts, X. Jia, and S. Ma, “Improving data quality: consistency and accuracy,” in Proceedings of the 33rd International Conference on Very Large Data Bases, ser. VLDB ’07.VLDB Endowment, 2007, p. 315–326. [Online]. Available: https://dl.acm.org/doi/10.5555/1325851.1325890 [31] R. F. Boisvert, R. Cools, and B. Einarsson, “Assessment of Accuracy and Reliability,” in Accuracy and Reliability in Scientific Computing.SIAM, 2005, p. 13–32. [Online]. Available: https: //epubs.siam.org/doi/abs/10.1137/1.9780898718157.ch2 [32] NationalAcademiesofSciences,Engineering,andMedicine, “Understanding Reproducibility and Replicability,” in Reproducibility and Replicability in Science.Washington, DC: National Academies Press,May2019,accessed:2024-04-17.[Online].Available: https://w.ncbi.nlm.nih.gov/books/NBK547546/ [33] OpenAI, “GPT-3 model card,” https://github.com/openai/gpt-3/blob/ master/model-card.md, 2020, accessed: 2024-04-17. [34] —, “GPT-4 system card,” 2023, accessed: 2024-04-17. [Online]. Available: https://cdn.openai.com/papers/gpt-4-system-card.pdf [35] Anthropic AI, “Glossary: Temperature,” May 2024, accessed: 2024- 05-21. [Online]. Available: https://docs.anthropic.com/en/docs/glossary# temperature [36] M. Fenniak and PyPDF contributors, PyPDF Documentation, 2026, version 6.14.2. [Online]. Available: https://pypdf.readthedocs.io/en/ stable/ [37] eCQI Center, “Value set,” https://ecqi.healthit.gov/glossary/value-set, Office of the National Coordinator for Health Information Technology, 2025, accessed: July 2, 2025. [38] R. Bhagat and E. Hovy, “What Is a Paraphrase?” Computational Linguistics, vol. 39, no. 3, p. 463–472, 09 2013. [Online]. Available: https://doi.org/10.1162/COLI a00166 [39] P. E. Black, “Greedy algorithm,” Dictionary of Algorithms and Data Structures [online], February 2005. [Online]. Available: https: //xlinux.nist.gov/dads//HTML/greedyalgo.html [40] Anthropic, “Introducing Claude Opus 4.6,” Feb. 2026, Claude Opus 4.6. [Online]. Available: https://w.anthropic.com/news/claude-opus-4-6 [41] Y. Lee, K. Son, T. S. Kim, J. Kim, J. J. Y. Chung, E. Adar, and J. Kim, “One vs. Many: Comprehending Accurate Information from Multiple Erroneous and Inconsistent AI Generations,” in Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency, ser. FAccT ’24.New York, NY, USA: Association for Computing Machinery, 2024, p. 2518–2531. [Online]. Available: https://doi.org/10.1145/3630106.3662681 [42] G. Vilagut, Test-Retest Reliability.Dordrecht: Springer Netherlands, 2014, p. 6622–6625. [Online]. Available: https://doi.org/10.1007/ 978-94-007-0753-5 3001 [43] J.-W. Versteeg, M. L. De Bruin, M. Schermer, S. Nadi Najafabadi, M. Mitra, C. Leopold, A. K. Mantel-Teeuwisse, W. G. Goettsch, and L. T. Bloem, “Text mining methods for automated data extraction from health technology assessment reports of medicines using classical natural language processing and generative artificial intelligence,” JAMIA Open, vol. 9, no. 2, p. ooag051, 04 2026. [Online]. Available: https://doi.org/10.1093/jamiaopen/ooag051 [44] Google DeepMind, “Gemini 3.1 Pro,” Feb. 2026. [Online]. Available: https://deepmind.google/models/gemini/pro/ [45] OpenAI, “gpt-oss-120b & gpt-oss-20b Model Card,” 2025. [Online]. Available: https://arxiv.org/abs/2508.10925 [46] SURF, “AI-Hub,” 2025. [Online]. Available: https://w.surf.nl/en/ themes/artificial-intelligence/projects-and-collaborations/ai-hub [47] SURF, “SURF: The Dutch IT cooperative for research and education,” Moreelsepark 48, 3511 EP, Utrecht, The Netherlands. [Online]. Available: https://w.surf.nl [48] Zorginstituut Nederland, “Zorginstituut Nederland (ZIN),” https://w. zorginstituutnederland.nl/, Diemen, The Netherlands, 2026, accessed: 2026-04-17. [49] Haute Autorit ́ e de Sant ́ e, “Haute Autorit ́ e de Sant ́ e (HAS),” https://w. has-sante.fr/, Saint-Denis, France, 2026, accessed: 2026-04-17. [50] Z. Wang, T. Nayfeh, J. Tetzlaff, P. O’Blenis, and M. H. Murad, “Error rates of human reviewers during abstract screening in systematic reviews,” PLOS ONE, vol. 15, no. 1, p. 1–8, 01 2020. [Online]. Available: https://doi.org/10.1371/journal.pone.0227742 [51] J. R. Landis and G. G. Koch, “The measurement of observer agreement for categorical data,” Biometrics, vol. 33, no. 1, p. 159–174, 1977. [Online]. Available: http://w.jstor.org/stable/2529310 [52] J. W. Creswell and J. D. Creswell, Research Design: Qualitative, Quantitative, and Mixed Methods Approaches, 6th ed.Thousand Oaks, CA: SAGE Publications, Inc., 2023. [Online]. Available: https://uk.sagepub.com/en-gb/eur/research-design/book270550 [53] “Oxford Dictionary of National Biography,” 2004, online ed. Continuously updated; originally published in 60 volumes, edited by H. C. G. Matthew and Brian Harrison. [Online]. Available: https://w.oxforddnb.com/