Paper deep dive
CRITICS - Critical Science Without Borders: Language Models to Promote Critical Thinking in Science Education
Rodrigo Agerri, Itziar Aldabe, Elena Cabrio, Mark Cieliebak, Jan Deriu, Mariana Flores, Jurgita Kapociute-Dzikiene, Dovile Kuiziniene, Arantza Rico, Aritz Ruiz-Gonzalez, Aitor Soroa, Mantas Vaskevicius, Serena Villata
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 9/20/2026, 4:08:35 AM
Summary
The CRITICS project is a three-year CHIST-ERA IV Cofund initiative aimed at democratizing scientific knowledge by breaking down language barriers. It converges Large Language Model (LLM)-based Machine Translation (MT) with educational technology to translate scientific content into low-resource languages (Basque, Lithuanian, French, German). The project develops a three-component system: an LLM-based scientific document translator, a Teaching and Learning Sequence (TLS) generator that extracts argumentation graphs, and an LLM-based automatic assessor that generates critical questions and feedback to foster critical thinking and scientific argumentation skills in students.
Entities (19)
Relation Signals (17)
HiTZ Center â affiliatedwith â University of the Basque Country
confidence 95% · The project is coordinated by the HiTZ Center from the University of the Basque Country EHU
CRITICS â aimstopromote â Critical thinking
confidence 95% · fostering critical thinking in science education.
CRITICS â coordinatesby â HiTZ Center
confidence 95% · The project is coordinated by the HiTZ Center from the University of the Basque Country EHU
CRITICS â fundedby â CHIST-ERA
confidence 95% · CRITICS is a three-year CHIST-ERA IV Cofund 2025 project funded within the topic ââScience in your own languageâ'.
CRITICS â targetslanguage â Basque
confidence 95% · Our focus extends beyond fluency and general translation quality... for our target language pairs: EnglishâLithuanian, EnglishâBasque, EnglishâFrench and English-German.
CRITICS â targetslanguage â German
confidence 95% · Our focus extends beyond fluency and general translation quality... for our target language pairs: EnglishâLithuanian, EnglishâBasque, EnglishâFrench and English-German.
CRITICS â targetslanguage â Lithuanian
confidence 95% · Our focus extends beyond fluency and general translation quality... for our target language pairs: EnglishâLithuanian, EnglishâBasque, EnglishâFrench and English-German.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:The CRITICS project addresses science accessibility and literacy by converging advanced Machine Translation (MT) based on Large Language Models (LLMs) with educational technology. By leveraging MT systems specifically optimized for scientific content, educational institutions can provide accurate, culturally relevant translations of scientific materials in students' native languages, ensuring that complex scientific concepts are comprehensible while maintaining technical accuracy. Building on these translations, the project explores the design and evaluation of innovative science teaching-learning proposals grounded in curriculum-aligned teaching-learning. Thus, CRITICS will investigate key components of scientific argumentation and critical thinking practices together with textual feedback aligned with learning objectives and assessment criteria inspired by competence-based evaluation frameworks. CRITICS aims to break down language barriers to accessing cutting-edge research and educational materials currently available only in high-resourced languages, thereby facilitating the democratization of scientific knowledge and fostering critical thinking in science education.
Tags
Links
- Source: https://arxiv.org/abs/2609.13942v1
- Canonical: https://arxiv.org/abs/2609.13942v1
Trouble viewing inline? Open PDF directly â
Full Text
37,278 characters extracted from source content.
Expand or collapse full text
*1 CRITICS - Critical Science Without Borders: Language Models to Promote Critical Thinking in Science Education Rodrigo Agerri Itziar Aldabe Elena Cabrio Mark Cieliebak Jan Deriu Mariana Flores Jurgita KapoÄiĆ«tÄ-DzikienÄ DovilÄ KuizinienÄ Arantza Rico Aritz Ruiz-GonzĂĄlez Aitor Soroa Mantas VaĆĄkeviÄius Serena Villata Abstract The CRITICS project addresses science accessibility and literacy through the convergence of advanced Machine Translation (MT) based on Large Language Models (LLMs) and educational technology. By leveraging MT systems specifically optimized for scientific content, educational institutions can provide accurate, culturally relevant translations of scientific materials in studentsâ native languages, ensuring that complex scientific concepts are comprehensible while maintaining technical accuracy. Building on these translations, the project explores the design and evaluation of innovative science teachingâlearning proposals grounded in curriculum-aligned teachingâlearning. Thus, CRITICS will investigate key components of scientific argumentation and critical thinking practices together with textual feedback aligned with learning objectives and assessment criteria inspired by competence-based evaluation frameworks. CRITICS aims to break down language barriers in accessing cutting-edge research and educational materials currently only available in high-resourced languages, thereby facilitating the democratization of scientific knowledge while fostering critical thinking in science education. keywords Science Education ,Scientific Terminology ,Argumentation ,Machine Translation ,Natural Language Processing â copyrightyear: 2026â copyright: Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (C BY 4.0).â venue: SEPLN 2026: 42nd International Conference of the Spanish Society for Natural Language Processing, LeĂłn, Spain, 22-25 September 2026.â email: rodrigo.agerri@ehu.eusâ email: itziar.aldabe@ehu.eusâ email: elena.cabrio@univ-cotedazur.frâ email: mark.cieliebak@zhaw.châ email: janmilan.deriu@zhaw.châ email: mariana.flores@ehu.eusâ email: jurgita.kapociute-dzikiene@vdu.ltâ email: dovile.kuiziniene@vdu.ltâ email: arantza.rico@ehu.eusâ email: aritz.ruiz@ehu.eusâ email: a.soroa@ehu.eusâ email: mantas.vaskevicius@vdu.ltâ email: serena.villata@univ-cotedazur.frâ address: HiTZ Center - Ixa, University of the Basque Country EHUâ address: UniversitĂ© CĂŽte dâAzur, Inria, CNRS, I3S, Franceâ address: Centre for Artificial Intelligence, ZHAW School of Engineeringâ address: Faculty of Informatics, Vytautas Magnus University, Lithuaniaâ address: Department of Mathematics, Experimental and Social Sciences Education, University of the Basque Country EHUâ corresp: Corresponding author. 1 Introduction CRITICS11 1 https://w.hitz.eus/critics is a three-year CHIST-ERA IV Cofund 2025 project funded within the topic âScience in your own languageâ.22 2 https://w.chistera.eu/projects-call-2025 This call addresses the translation of scientific knowledge to bridge linguistic and cultural gaps for those who must disseminate and access scientific knowledge beyond their linguistic scope. The project is coordinated by the HiTZ Center from the University of the Basque Country EHU, funded by MICIU/AEI /10.13039/501100011033 and by the European Union (PCI2025-167239-2). In addition to HiTZ, the EHU team includes researchers from the Department of Mathematics, Experimental and Social Sciences Education. The consortium is also formed by the CNRS/UniversitĂ© CĂŽte dâAzur in France (grant no ANR-25-CHR4-0002-02), Vytautas Magnus University (VMU) (funded by the Research Council of Lithuania, agreement No. S-CHIST-ERA-26-1), and the ZHAW Center for Artificial Intelligence in Switzerland (grant no. 20CH-1_238349). Scientific literacy and the development of critical thinking are essential components for a democratic society. However, access to scientific knowledge continues to be conditioned by language, since a large proportion of research articles and advanced educational resources are published almost exclusively in English or other high-resource languages such as Spanish. This situation creates linguistic inequality that limits learning opportunities for both students and teachers of science, particularly in low-resource languages such as Basque, and hinders studentsâ ability to learn how to construct evidence-based arguments in their own language. The CRITICS project addresses these challenges through the convergence of advanced MT systems based on language models (LLMs) and educational technologies. The accessibility to scientific content in our own languages through advanced MT naturally connects to the automated generation of science education materials, where LLMs can be applied to synthesize and adapt complex scientific concepts into level-appropriate and pedagogically sound resources. Thus, CRITICS will develop and adapt LLMs to facilitate the generation of customized science education materials MĂ©heut and Psillos (2004). CRITICS will mostly follow recent science education research focused on Design-Based Research Ruiz-GonzĂĄlez et al. (2025) and consider education materials to specify the driving problem/questions, the learning objectives focused on competency acquisition, the scientific practices including scientific argumentation and critical thinking, and the activities to be made by the science students. The availability of machine-translated scientific knowledge will be crucial to investigating and developing LLMs for the automatic generation of appropriate education materials in the studentsâ native languages that relate to local studentsâ experiences Ruiz-GonzĂĄlez et al. (2025). Figure 1: Three main components in the CRITICS approach. Overall Approach: Figure 1 illustrates the integrated approach provided by CRITICS, which directly addresses the core objectives of the call by creating a comprehensive ecosystem that not only machine translates scientific content but transforms it into educationally meaningful resources with built-in assessment capabilities. CRITICS will develop a three-component system as in Figure 1 for cross-lingual scientific knowledge transfer. The first component is an LLM-based scientific document translation system that surpasses conventional MT approaches, which often struggle with domain-specific terminology and notation Kleidermacher and Zou (2026); Kim et al. (2024), using adapters fine-tuned on specialized scientific corpora to preserve structural elements and field-specific terminology. The second component, a Teaching and Learning Sequence (TLS) generator, will investigate how to transform translated scientific literature into structured educational resources by analyzing the corpus to identify key concepts and pedagogical approaches, extracting argumentation graphs to generate TLSs. The third component, an LLM-based automatic assessor, will focus on how to evaluate student responses to TLS-derived activities by analyzing evidence-based argumentation and applying automatically generated scientific critical questions, with Argument Mining capabilities to detect misconceptions and fallacies. The CRITICS LLMs adapted to the tasks will be placed in a continuous improvement loop where assessments feed back into translation quality, teaching resource development, and assessment criteria refinement. 2 Related Work and Novelty CRITICS will comply with TRL 5 â technology validated in relevant environments (industrially relevant environment in the case of key enabling technologies),33 3 https://horizoneuropencpportal.eu/sites/default/files/2022-12/trl-assessment-tool-guide-final.pdf where the qualitative evaluation will be done involving domain-experts (science education teachers and researchers on the two main disciplines targeted in the project, natural science and AI) in both the supervision and the actual evaluation. 2.1 LLM-based MT of Scientific Documents LLMs have substantially advanced the field of MT, particularly for high-resource languages and general-purpose domains. Proprietary models deliver strong fluency and document-level coherence, benefiting from long-context modeling and multilingual pretraining, though notable gaps remain in low-resource scenarios Wang et al. (2023); Zhu et al. (2024). Prompt-based strategies (especially zero- and few-shot prompting) have emerged as effective methods for improving LLM translation performance Zhang et al. (2023). Terminology-aware prompting and term injection have received growing attention as key techniques for addressing a persistent challenge in domain-specific MT: terminological inconsistency. This is especially critical in scholarly translation, where inaccurate terminology can distort meaning or undermine academic precision Kleidermacher and Zou (2026). Recent work highlights diverse strategies for integrating terminology into LLM-based MT Sabo et al. (2024); Lu et al. (2024); Kim et al. (2024). Despite these advances, less-resourced languages have received limited attention in LLM-based MT research. Recent work KapoÄiĆ«tÄ-DzikienÄ et al. (2025) demonstrates promising results for EnglishâLithuanian translation using open-weight LLMs. However, these findings are based on benchmark data rather than real scholarly texts, thus highlighting the need for more work in domain-specific, low-resource scenarios. Progress beyond the state-of-the-art: This project builds on recent advances in LLM-based MT but goes further by addressing the core challenges of scholarly translation (particularly for low-resource languages such as Lithuanian and Basque, with French included for comparative purposes). Our focus extends beyond fluency and general translation quality to include terminological precision, structural accuracy, and document-level coherence â i.e., key requirements in scholarly texts. We will benchmark a range of available LLMs, prioritizing open-weight models with good performance on the target languages such as Gemma Team et al. (2024), Llama Grattafiori et al. (2024) or language-specific models such as Latxa Etxaniz et al. (2024), to identify the most effective options for our target language pairs: EnglishâLithuanian, EnglishâBasque, EnglishâFrench and English-German. A central focus of the project is document-level translation, where maintaining consistency of terminology and coherence across sections is essential. To support this, we will develop domain-specific terminology glossaries. These glossaries will be integrated into the translation workflow through prompt engineering or as part of the fine-tuning datasets. Beyond linguistic accuracy, we will also develop tools to preserve the layout and structure of scholarly documents, ensuring that translations retain citations, equations, code, and section formatting. 2.2 Argument(ation) Mining (AM) Existing research on mining, assessing, reasoning over, and generating arguments Stede et al. (2019); Lawrence and Reed (2019) involves the automatic extraction of arguments and their relationships from text. Starting from the identification of claims, premises, and evidence in a piece of text, as well as the relationships between them, such as support, attack, or rebuttal, computational argumentation methods can be used to analyze the strengths and weaknesses of arguments, their quality, and to identify common patterns and themes. Usually, supervised learning methods are used to address AM tasks, leading to the need to define beforehand annotated datasets for the specific tasks and application scenarios, even if recently some empirical approaches have been proposed to synthesize and generate arguments. Among the recent challenges of the AM field (Dagstuhl Seminar 22432 âTowards a Unified Model of Scholarly Argumentationâ), there is the task of detecting manipulation in scientific publications through the identification of duplicate arguments and the computation of argument similarity Glockner et al. (2024), which strictly relates to the fallacy detection task. Moreover, a few works have focused on highlighting distortions in scientific communication Augenstein (2021), as well as in detecting quantified information mismatches between reported and actual scientific findings Wright et al. (2022). Progress beyond the state-of-the-art: While scientific texts aim to present verifiable evidence for a series of stated claims, the interpretation of such objective sources of evidence is often ambiguous and subjective. In this project, we will first enhance and tailor Argument mining approaches to address domain-specific features of scientific argumentation in science education texts. Then, we will explore the notion of similarity between natural language arguments, which is currently unexplored, but it would be crucial to check the validity and coherence of argumentative scientific content across languages, both for human-generated and machine-translated texts. The result of this task is the automatic extraction of the full argument graph, on which further inferences can be applied. Such argumentation graphs will also be exploited for critical questions generation, to evaluate critical thinking in scientific contexts across languages. Applications of argument mining in scientific discourse have been so far limited, in particular, because current methods are insensitive to the factual content of scientific arguments, an issue that will be carefully addressed in the context of CRITICS, following our work in the medical education domain Molinet et al. (2024). 2.3 Critical Questions Generation and Automatic Assessment Critical questions are vital for competency-based science assessments because they address deeper understanding beyond fact memorization to apply scientific knowledge and skills in practical contexts, which is the central aim of competency-based assessment. Unlike simple recall questions, critical questions help to evaluate essential higher-order thinking skills, measure studentsâ ability to transfer knowledge to new topic-specific contexts and provide diagnostic insights into reasoning patterns and misconceptions Lemons and Lemons (2013); Nussbaum (2021). There have been approaches to automated question and feedback generation Gao et al. (2024); Fu et al. (2024), but automatic Critical Questions Generation is a largely unexplored task Calvo Figueras and Agerri (2025), although some works have argued about their utility in identifying fallacious reasoning, and also for assessing the quality of argumentative essays Song et al. (2014). Progress beyond the state-of-the-art: Text-based automated assessment systems have been developed to automatically grade, classify or to provide feedback or guidance Gao et al. (2024). However, CRITICS will provide for the first time textual feedback for students by automatically generating Critical Questions, argument feedback and grades based on the TLSs developed for topic specific competency-based assessments in natural sciences and AI. More specifically, the generation of Critical Questions will be aligned with specific learning goals and assessment criteria specified in the TLSs, ensuring that the feedback is relevant and focused on the intended learning outcomes. Note that the generation will be multilingual, namely, in Basque, French, German and Lithuanian. These questions often apply models such as the Bloomâs taxonomy, CER (Claim, Evidence, Reasoning) or CQMAA (Critical Questions Model of Argument Assessment) to structure studentsâ responses and evaluations Nussbaum et al. (2020); Nussbaum (2021). 2.4 Evaluation and Benchmarking MT evaluation typically relies on automated metrics like BLEU, ChrF++, TER, or COMET. Though WMT includes some biomedical translation resources, comprehensive scientific text datasets remain notably absent Kleidermacher and Zou (2026). However, automatic evaluation of text generation tasks remains an open challenge. In CRITICS, these tasks will be focused (apart from MT, covered above) in the evaluation of Critical Questions and TLS, and Automatic Feedback Generation. Although the task of automatic Critical Questions Generation is rather new Calvo Figueras and Agerri (2025), automatic metrics for question generation in general have been investigated. Novel approaches like QGEval offer multi-dimensional assessment Fu et al. (2024). Still, there is a strong research interest in devising more comprehensive LLM-based evaluation methods Gu et al. (2024); Li et al. (2025), also for question generation in particular Fu et al. (2024). Despite these new automatic methods, human evaluation is still considered to be essential. Thus, expert assessments would focus on grammatical correctness, relevance, semantic soundness, quality of the evidence-based argumentation, and (in some cases) alignment with cognitive frameworks like Bloomâs Taxonomy Lawrence and Reed (2019). The consortium has ample research experience regarding evaluation and benchmarking relevant to the CRITICS project. EHU has developed a state-of-the-art open-source LLM for Basque, Latxa Etxaniz et al. (2024). EHU, in collaboration with CNRS, has released the first multilingual LLM for the medical domain, evaluating it on medical and scientific benchmarks. EHU has collected the largest corpora available for Basque, which contains circa 1.4 billion tokens Etxaniz et al. (2024), and has experience in the development of new benchmarks for multilingual evaluation of LLMs Alonso et al. (2024); Etxaniz et al. (2024); GarcĂa-Ferrero et al. (2024). Regarding Argument Mining, CNRS and EHU have released the first multilingual dataset for the medical domain annotated with argument structures Sviridova et al. (2024) and the first publicly available dataset of Critical Questions Calvo Figueras and Agerri (2025) linked to Argument Schemes. EHU has already published various results with similarity and LLM-based evaluation methods in pairwise ranking- and reference-based settings Calvo Figueras and Agerri (2025); Yeginbergen et al. (2025); Zubiaga et al. (2024). ZHAW has developed theoretical foundations for the applications of automated evaluation for generative models (with a focus on Machine Translation) Von DĂ€niken et al. (2024); Deriu et al. (2023). This provides a strong foundation for correctly applying automated evaluation and calibrating the metrics to avoid biased evaluation results von DĂ€niken et al. (2024). They also published work on the robustness and reliability of trained metrics Deriu et al. (2023). Progress beyond the state-of-the-art: The use of LLM-based evaluation for text generation tasks related to MT and Critical Questions and Feedback generation in the scientific domain remains an open research problem. CRITICS will provide new LLM-based evaluation methods specifically tailored to the relevant features of translating MT of scientific documents and to the science teaching-related criteria of the automatically generated Critical Questions and Assessment/Feedback. 3 Methodology and Work Plan Although CRITICSâ vision applies across disciplines, the project focuses on two areas: (i) natural sciences (biology, chemistry, and physics) and (i) Artificial Intelligence. The former is key to competency-based assessments, while the latter poses specific translation challenges due to the continuous introduction of new terminology that may lack established equivalents in less-resourced languages Kleidermacher and Zou (2026); Zhang et al. (2024). CRITICS will target a diverse spectrum of languages, namely, Basque (agglutinative language isolate), Lithuanian (East Baltic, inflectional), German (West Germanic, inflectional), and French (Romance, synthetic-fusional). Every objective will be addressed in a multilingual setting, namely, including the four target languages of the project. 1. Objective 1. LLM-based Machine Translation of Scientific Documents: Focused on developing and adapting open-weight LLMs for high-quality, document-level MT of scientific texts, particularly in low- and medium-resource language pairs (e.g., EnglishâLithuanian/Basque/French/German). 2. Objective 2. Argumentation serves as a fundamental mechanism in scientific discourse, facilitating the process of reaching conclusions and facilitating science literacy and critical thinking. This objective will focus on training LLMs to recognize evidence-based argumentation and identify fallacies and scientific misconceptions. 3. Objective 3. Automatic Assessment and Critical Thinking: Rather than merely identifying gaps in the scientific discourse, automatic assessment will also involve the generation of Critical Questions and Detailed Feedback in competency-based assessment settings (PISA style - Programme for International Student Assessment) where cross-linguistic comparability is essential. 4. Objective 4. Evaluation: qualitatively evaluate the generation of critical questions and automatic assessment feedback using LLMs-as-a-Judge paradigm in a way that can be compared with human-generated judgments Calvo Figueras and Agerri (2025). 3.1 Work Plan The project work plan consists of three Development-Evaluation cycles that will be replicated for each of the technical work packages (WP2-WP5). The overall structure of the work plan structured in work packages is as follows. In a first step, in WP2 the consortium will define the final requirements for the use cases (Parallel Translator of Scientific Documents and Teacher Evaluator) of the project. Also in WP2 the data collection required to address the objectives and research questions related to each use case will be performed. Regarding the MT use case, most of the relevant data sources for natural sciences (biology, chemistry, and physics) and Artificial Intelligence are already identified following open science criteria. MT of scientific data. The choice of English data sources aims to ensure a massive amount of scientific content to develop the LLM-based MT of scientific articles in WP3 and WP5 while being able to distribute the pre-processed corpus and the scientific outcomes. Data sources will include Biorxiv44 4 https://api.biorxiv.org/ (biology and biochemistry), PubMed Central55 5 https://pmc.ncbi.nlm.nih.gov/ (biomedical and life sciences, some data also available in French), chemrxiv,66 6 https://chemrxiv.org/ arxiv for Physics and Artificial Intelligence,77 7 https://arxiv.org/ ACL Anthology Corpus (AI and NLP),88 8 https://github.com/shauryr/ACL-anthology-corpus and other sources relevant for the scientific disciplines targeted in CRITICS, for example, evidence-based scientific educational resources such as Scientix,99 9 https://w.scientix.eu/ STEM learning centre,1010 10 https://w.stem.org.uk/ e-bug.1111 11 https://w.e-bug.eu/ Note that pre-print papers usually include information about their peer-reviewed version, which will prioritize to automatically translate into the targeted languages. Furthermore, most of these resources provide API access to their content in some structured format to avoid any OCR of PDF articles. A final selection and analysis of the most appropriate data sources will be done in WP2. Competency-based Assessments data. We will also compile and annotate (WP2) 500 competency-based assessments in higher education (questionnaires with student responses and corresponding teacherâs assessments) are available in-house at UPV/EHU and will be the basis for developing and evaluating the Teacher Evaluator tool and research in WP4. This data will be fully anonymized and released following privacy and open science policies. Once both types of data are compiled and pre-processed, WP2 will annotate argumentation graphs, critical questions, and feedback/assessment for benchmarking in WP5. Using the data for evaluation and development, WP3 will primarily focus on the advancement of SOTA by training and experimenting with LLMs for MT of scientific data. In WP4 novel research will address the quality assessment of argumentation in scientific and educational texts. WP5 will investigate new techniques to evaluate challenging text generation tasks such as MT of scientific documents, Critical Questions and Automatic Feedback Generation. Finally, WP6 will apply every result obtained in WP2, WP3 and WP4 to develop the two tools and prototypes for a Parallel Translator of Scientific Documents and a Teacher Evaluator. 1. WP2. Data Collection: In this WP we will address the data collection, processing, and manual annotation and evaluation for benchmarking of the various modules in WP5. The correct description of the data guarantees its traceability, contributing to the explainability and transparency of the models. We also plan to obtain new high-quality instruction and alignment datasets derived from already instructed models Zhang et al. (2024) using the magpie approach Xu et al. (2024) to reduce the need for expensive human-annotated instruction data. 2. WP3. LLM-based MT: Adapt and optimize MT techniques for scholarly documents, with a focus on low-resource language pairs (English-Lithuanian, English-Basque) and medium-resource pairs (English-French, English-German). Improve translation quality using open-weight LLMs Bai et al. (2023); Team et al. (2024); Grattafiori et al. (2024), domain-specific data Zhang et al. (2024), and strategies like prompt engineering, terminology integration Kim et al. (2024), and lightweight fine-tuning. 3. WP4. Argumentation:. Tailor Argument mining approaches to address domain-specific features of scientific argumentation in multilingual educational texts. Assess the coherence of argumentative scientific content across languages. Critical questions generation Calvo Figueras and Agerri (2025), to enhance and evaluate studentsâ ability to analyze complex texts, evaluate claims and evidence, and construct well-reasoned arguments. 4. WP5. Evaluation and Benchmarking: First, this WP evaluates the translated scientific texts. We investigate two approaches. First, the human evaluation, and second automated evaluation procedures. For human evaluation, the main challenge is reproducibility, as human ratings suffer from high variance; thus, we will develop protocols and guidelines to reduce this issue. The main challenges for automated evaluation are that they disagree with human feedback and tend to prefer certain systems disproportionately. Thus, the second focus of this WP is developing and calibrating automated metrics to evaluate MT and assessment generation Calvo Figueras and Agerri (2025); Yeginbergen et al. (2025); Zubiaga et al. (2024). 4 Concluding Remarks CRITICS aims to contribute to the democratization of scientific knowledge by integrating LLM-based MT with critical thinking in science education, overcoming language barriers for low- and medium-resource languages like Basque, Lithuanian, French, and German. Through its six work packages, the project will strive to deliver optimized MT for scientific documents, argumentation mining tailored to scientific texts, automated critical feedback generation, and robust evaluation methods, all validated via applications such as the Parallel Reader and Teacher Evaluator. By addressing key challenges in terminology consistency, document structure preservation, and multilingual argumentation assessment, CRITICS advances beyond the current state-of-the-art in domain-specific NLP and educational tech. The projectâs emphasis on open-weight models, synthetic data generation, and LLM-as-a-judge metrics will investigate how to ensure reproducible, transparent results that foster scientific literacy across diverse linguistic contexts. Future extensions could scale to additional low-resource languages and integrate emerging multimodal LLMs for visual scientific content, amplifying the projectâs impact on global science education. Acknowledgements. EHUâs researchers acknowledge the following MCIN/AEI/10.13039/501100011033 projects: (i) CRITICS (PCI2025-167239-2) funded by MICIU/AEI /10.13039/501100011033 and co-funded by the European Union; (i) DeepThought (PID2024-159202OB-C21) funded by ERDF, EU, and (i) Xixare (PID2024-159566NB-I00). CNRS/UniCA research is supported by the French National Research Agency (grant ANR-25-CHR4-0002-02). References MĂ©heut and Psillos (2004) M. MĂ©heut, D. Psillos, Teachingâlearning sequences: aims and tools for science education research, International Journal of Science Education 26 (2004) 515â535. Ruiz-GonzĂĄlez et al. (2025) A. Ruiz-GonzĂĄlez, A. Rico, J. Guisasola, Learning About Sound in Initial Teacher Training: Evaluation and Redesign of a Teaching-Learning Sequence, in: Connecting Science Education with Cultural Heritage: Selected Papers from the ESERA 2023 Conference, Springer, 2025, p. 157â171. Kleidermacher and Zou (2026) H. C. Kleidermacher, J. Zou, Science across languages: assessing LLM multilingual translation of scientific papers, in: Findings of the EACL 2026, 2026, p. 3932â3947. Kim et al. (2024) S. Kim, M. Sung, J. Lee, H. Lim, J. G. Perez, Efficient terminology integration for LLM-based translation in specialized domains, in: Proceedings of the Ninth Conference on Machine Translation, 2024, p. 636â642. Wang et al. (2023) L. Wang, C. Lyu, T. Ji, Z. Zhang, D. Yu, S. Shi, Z. Tu, Document-level machine translation with large language models, in: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023, p. 16646â16661. Zhu et al. (2024) W. Zhu, H. Liu, Q. Dong, J. Xu, S. Huang, L. Kong, J. Chen, L. Li, Multilingual machine translation with large language models: Empirical results and analysis, in: Findings of NAACL 2024, 2024, p. 2765â2781. Zhang et al. (2023) B. Zhang, B. Haddow, A. Birch, Prompting large language model for machine translation: A case study, in: International conference on machine learning, PMLR, 2023, p. 41092â41110. Sabo et al. (2024) M. Sabo, J. Klein, G. Bernardinello, Boosting machine translation with AI-powered terminology features, in: EAMT, 2024, p. 25â26. Lu et al. (2024) H. Lu, H. Yang, H. Huang, D. Zhang, W. Lam, F. Wei, Chain-of-dictionary prompting elicits translation in large language models, in: EMNLP, 2024, p. 958â976. KapoÄiĆ«tÄ-DzikienÄ et al. (2025) J. KapoÄiĆ«tÄ-DzikienÄ, T. Bergmanis, M. Pinnis, Localizing AI: Evaluating Open-Weight Language Models for Languages of Baltic States, in: NoDaLiDa/Baltic-HLT, 2025, p. 287â295. Team et al. (2024) G. Team, T. Mesnard, C. Hardin, R. Dadashi, S. Bhupatiraju, S. Pathak, L. Sifre, M. RiviĂšre, M. S. Kale, J. Love, et al., Gemma: Open models based on gemini research and technology, arXiv preprint arXiv:2403.08295 (2024). Grattafiori et al. (2024) A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughan, et al., The llama 3 herd of models, arXiv preprint arXiv:2407.21783 (2024). Etxaniz et al. (2024) J. Etxaniz, O. Sainz, N. Miguel, I. Aldabe, G. Rigau, E. Agirre, A. Ormazabal, M. Artetxe, A. Soroa, Latxa: An open language model and evaluation suite for basque, in: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2024, p. 14952â14972. Stede et al. (2019) M. Stede, J. Schneider, G. Hirst, Argumentation mining, Springer, 2019. Lawrence and Reed (2019) J. Lawrence, C. Reed, Argument mining: A survey, Computational linguistics 45 (2019) 765â818. Glockner et al. (2024) M. Glockner, Y. Hou, P. Nakov, I. Gurevych, Missci: Reconstructing fallacies in misrepresented science, in: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2024, p. 4372â4405. Augenstein (2021) I. Augenstein, Determining the credibility of science communication, in: Proceedings of the Second Workshop on Scholarly Document Processing, 2021, p. 1â6. Wright et al. (2022) D. Wright, J. Pei, D. Jurgens, I. Augenstein, Modeling information change in science communication with semantically matched paraphrases, in: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022, p. 1783â1807. Molinet et al. (2024) B. Molinet, S. Marro, E. Cabrio, S. Villata, Explanatory argumentation in natural language for correct and incorrect medical diagnoses, Journal of Biomedical Semantics 15 (2024) 8. Lemons and Lemons (2013) P. P. Lemons, J. D. Lemons, Questions for assessing higher-order cognitive skills: Itâs not just bloomâs, CBEâLife Sciences Education 12 (2013) 47â58. Nussbaum (2021) E. M. Nussbaum, Critical integrative argumentation: Toward complexity in studentsâ thinking, Educational Psychologist 56 (2021) 1â17. Gao et al. (2024) R. Gao, H. E. Merzdorf, S. Anwar, M. C. Hipwell, A. R. Srinivasa, Automatic assessment of text-based responses in post-secondary education: A systematic review, Computers and Education: Artificial Intelligence 6 (2024) 100206. Fu et al. (2024) W. Fu, B. Wei, J. Hu, Z. Cai, J. Liu, Qgeval: benchmarking multi-dimensional evaluation for question generation, in: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024, p. 11783â11803. Calvo Figueras and Agerri (2025) B. Calvo Figueras, R. Agerri, Benchmarking Critical Questions Generation: A Challenging Reasoning Task for Large Language Models, in: Findings of the EMNLP 2025, 2025, p. 5635â5652. Song et al. (2014) Y. Song, M. Heilman, B. B. Klebanov, P. Deane, Applying argumentation schemes for essay scoring, in: Proceedings of the first workshop on argumentation mining, 2014, p. 69â78. Nussbaum et al. (2020) M. Nussbaum, L. Tian, M. Van Winkle, H. Perera, L. Putney, I. Dove, K. Carroll, Using the critical questions model of argumentation for science teacher professional learning and student outcomes, International Society of the Learning Sciences (ISLS) (2020). Gu et al. (2024) J. Gu, X. Jiang, Z. Shi, H. Tan, X. Zhai, C. Xu, W. Li, Y. Shen, S. Ma, H. Liu, et al., A survey on llm-as-a-judge, The Innovation (2024). Li et al. (2025) D. Li, B. Jiang, L. Huang, A. Beigi, C. Zhao, Z. Tan, A. Bhattacharjee, Y. Jiang, C. Chen, T. Wu, et al., From generation to judgment: Opportunities and challenges of llm-as-a-judge, in: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 2025, p. 2757â2791. Alonso et al. (2024) I. Alonso, M. Oronoz, R. Agerri, Medexpqa: Multilingual benchmarking of large language models for medical question answering, Artificial intelligence in medicine 155 (2024) 102938. GarcĂa-Ferrero et al. (2024) I. GarcĂa-Ferrero, R. Agerri, A. A. Salazar, E. Cabrio, I. De la Iglesia, A. Lavelli, B. Magnini, B. Molinet, J. Ramirez-Romero, G. Rigau, et al., Medmt5: An open-source multilingual text-to-text llm for the medical domain, in: Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), 2024, p. 11165â11177. Sviridova et al. (2024) E. Sviridova, A. Yeginbergen, A. Estarrona, E. Cabrio, S. Villata, R. Agerri, Casimedicos-arg: A medical question answering dataset annotated with explanatory argumentative structures, in: Proceedings of the 2024 conference on empirical methods in natural language processing, 2024, p. 18463â18475. Yeginbergen et al. (2025) A. Yeginbergen, M. Oronoz, R. Agerri, Dynamic knowledge integration for evidence-driven counter-argument generation with large language models, in: Findings of the Association for Computational Linguistics: ACL 2025, 2025, p. 22568â22584. Zubiaga et al. (2024) I. Zubiaga, A. Soroa, R. Agerri, A llm-based ranking method for the evaluation of automatic counter-narrative generation, in: Findings of the Association for Computational Linguistics: EMNLP 2024, 2024, p. 9572â9585. Von DĂ€niken et al. (2024) P. Von DĂ€niken, J. M. Deriu, D. Tuggener, M. Cieliebak, Favi-score: A measure for favoritism in automated preference ratings for generative ai evaluation, in: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2024, p. 4437â4454. Deriu et al. (2023) J. M. Deriu, P. Von DĂ€niken, D. Tuggener, M. Cieliebak, Correction of errors in preference ratings from automated metrics for text generation, in: Findings of the Association for Computational Linguistics: ACL 2023, 2023, p. 6456â6474. von DĂ€niken et al. (2024) P. von DĂ€niken, J. M. Deriu, A. Rodrigo, M. Cieliebak, Improving quantification with minimal in-domain annotations: Beyond classify and count, in: Proceedings of the International AAAI Conference on Web and Social Media, volume 18, 2024, p. 1585â1598. Zhang et al. (2024) D. Zhang, Z. Hu, S. Zhoubian, Z. Du, K. Yang, Z. Wang, Y. Yue, Y. Dong, J. Tang, Sciinstruct: a self-reflective instruction annotated dataset for training scientific language models, Advances in Neural Information Processing Systems 37 (2024) 1443â1473. Xu et al. (2024) Z. Xu, F. Jiang, L. Niu, Y. Deng, R. Poovendran, Y. Choi, B. Y. Lin, Magpie: Alignment data synthesis from scratch by prompting aligned llms with nothing, arXiv preprint arXiv:2406.08464 (2024). Bai et al. (2023) J. Bai, S. Bai, Y. Chu, Z. Cui, K. Dang, X. Deng, Y. Fan, W. Ge, Y. Han, F. Huang, et al., Qwen technical report, arXiv preprint arXiv:2309.16609 (2023).