Paper deep dive
Cross-Lingual Clinical Annotation Projection as Constrained Text Generation: A Six-Language Study
Ălvaro Rey-Blanes, Francisco J. Moreno-Barea, Francisco J. Veredas
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 95%
Last extracted: 9/11/2026, 5:19:44 AM
Summary
This study evaluates cross-lingual clinical annotation projection as a constrained text generation task using Large Language Models (LLMs). The authors developed a workflow that inserts entity tags directly into immutable target-language text, followed by deterministic validation. They compared this direct LLM approach against supervised machine learning (ML) and hybrid ML-LLM refinement methods for transferring Spanish Disease, Symptom, and Procedure annotations into six languages (English, Czech, Italian, Dutch, Romanian, Swedish). Results showed that direct LLM projection, particularly using GLM 5.2 and Gemma4:31B, achieved the highest Strict F1 scores (0.9201 and 0.9133 respectively), outperforming previous state-of-the-art methods and significantly reducing the need for expert annotation time.
Entities (18)
Relation Signals (16)
gemma4:31b â achievedperformance â 0.9133 Strict F1
confidence 95% · locally deployable Gemma4:31B achieved 0.9133
GLM-5.2 â achievedperformance â 0.9201 Strict F1
confidence 95% · GLM 5.2 obtained a mean Strict F1 of 0.9201 across 18 languageâentity combinations
MultiClinCorpus â containsannotationsfor â Procedure
confidence 95% · MultiClinCorpus, which projects Disease, Symptom, and Procedure annotations
MultiClinCorpus â containsannotationsfor â Disease
confidence 95% · MultiClinCorpus, which projects Disease, Symptom, and Procedure annotations
MultiClinCorpus â containsannotationsfor â Symptom
confidence 95% · MultiClinCorpus, which projects Disease, Symptom, and Procedure annotations
MultiClinCorpus â transfersfrom â Spanish
confidence 95% · projects Disease, Symptom, and Procedure annotations from Spanish into English, Dutch, Italian, Romanian, Swedish, and Czech.
MultiClinCorpus â transfersto â English
confidence 95% · projects Disease, Symptom, and Procedure annotations from Spanish into English, Dutch, Italian, Romanian, Swedish, and Czech.
MultiClinCorpus â transfersto â Italian
confidence 95% · projects Disease, Symptom, and Procedure annotations from Spanish into English, Dutch, Italian, Romanian, Swedish, and Czech.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Background: To determine whether cross-lingual clinical annotation projection can be formulated as a text-preserving, document-level generative task that produces verifiable character-level annotations for multilingual clinical corpus construction, and to characterize its robustness and computational trade-offs relative to candidate-based projection pipelines. Methods: We developed a constrained LLM projection workflow that inserts entity tags directly into immutable target-language text, followed by deterministic validation and character-offset reconstruction. We evaluated it alongside supervised candidate-span projection and hybrid ML-LLM refinement for transferring Spanish Disease, Symptom, and Procedure annotations into six languages. Evaluation used MultiClinAI gold standard with strict span matching and character-overlap F1 Results: Direct LLM projection achieved the strongest and most consistent performance. GLM 5.2 obtained a mean Strict F1 of 0.9201 across 18 language-entity combinations, while locally deployable Gemma4:31B achieved 0.9133. The best LLM configuration improved Strict F1 over the previous state of the art in all 18 settings, by 0.0564-0.1512, yielding 55,416 grounded mentions with reconstructed offsets. Conclusions: Direct LLM-based projection enables high-quality multilingual clinical annotation transfer and provides a practical approach for extending clinical NLP resources to languages with fewer annotated datasets and language-specific tools. Combined with local inference and deterministic validation, it can substantially reduce expert time and cost for multilingual clinical corpus construction.
Tags
Links
- Source: https://arxiv.org/abs/2609.11450v1
- Canonical: https://arxiv.org/abs/2609.11450v1
Trouble viewing inline? Open PDF directly â
Full Text
54,739 characters extracted from source content.
Expand or collapse full text
Cross-Lingual Clinical Entity Projection: Comparing Supervised, Hybrid, and Large Language Model Approaches Abstract Background: To determine whether cross-lingual clinical annotation projection can be formulated as a text-preserving, document-level generative task that produces verifiable character-level annotations for multilingual clinical corpus construction, and to characterize its robustness and computational trade-offs relative to candidate-based projection pipelines. Methods: We developed a constrained LLM projection workflow that inserts entity tags directly into immutable target-language text, followed by deterministic validation and character-offset reconstruction. We evaluated it alongside supervised candidate-span projection and hybrid MLâLLM refinement for transferring Spanish Disease, Symptom, and Procedure annotations into six languages. Evaluation used MultiClinAI gold standard with strict span matching and character-overlap F1 Results: Direct LLM projection achieved the strongest and most consistent performance. GLM 5.2 obtained a mean Strict F1 of 0.9201 across 18 languageâentity combinations, while locally deployable Gemma4:31B achieved 0.9133. The best LLM configuration improved Strict F1 over the previous state of the art in all 18 settings, by 0.0564â0.1512, yielding 55,416 grounded mentions with reconstructed offsets. Conclusions: Direct LLM-based projection enables high-quality multilingual clinical annotation transfer and provides a practical approach for extending clinical NLP resources to languages with fewer annotated datasets and language-specific tools. Combined with local inference and deterministic validation, it can substantially reduce expert time and cost for multilingual clinical corpus construction. Cross-Lingual Clinical Annotation Projection as Constrained Text Generation: A Six-Language Study Ălvaro Rey-Blanes1,2,3,* Francisco J. Moreno-Barea1,2,3 Francisco J. Veredas1,2,3 1Department of Programming Languages and Computer Sciences, Universidad de MĂĄlaga, MĂĄlaga, Spain 2Research Institute of Multilingual Language Technologies, Universidad de MĂĄlaga, MĂĄlaga, Spain 3IBIMA Plataforma BIONAND, Instituto de InvestigaciĂłn BiomĂ©dica de MĂĄlaga, MĂĄlaga, Spain *Corresponding author: alvaroreyb@uma.es Keywords: cross-lingual annotation projection, clinical named entity recognition, multilingual clinical NLP, large language models, clinical corpora 1 Background and Significance Clinical narratives contain essential information on clinical conditions, signs and symptoms, therapeutic interventions, and diagnostic procedures. Automatically identifying these concepts is therefore a central requirement for the secondary use of clinical text in applications such as cohort identification, clinical decision support, epidemiological surveillance, and retrospective research. Named Entity Recognition (NER) provides the span-level representations required by many of these applications, but its development remains highly dependent on manually annotated corpora produced or validated by domain experts [18]. Such resources are unevenly distributed across languages. English benefits from numerous annotated datasets, pretrained models, and evaluation benchmarks, whereas many other languages have substantially fewer reusable clinical corpora. This imbalance is particularly relevant in the clinical domain, where expert annotation is costly and time-consuming. Cross-lingual annotation projection offers an alternative to independently annotating equivalent resources in every language. Given an annotated source document and its translation, projection methods identify the corresponding target-language span while preserving its semantic label and associated metadata [30, 7]. A single expert-annotated source corpus can therefore support multiple multilingual resources or provide supervision for target-language extraction systems. Annotation projection is not equivalent to copying character offsets between texts. Translation may expand, contract, reorder, or reformulate entity mentions, and sentence restructuring can alter their positions. The task therefore requires not only identifying semantic equivalence but also recovering the exact target boundaries required for span-based evaluation and downstream processing. Clinical language further complicates projection through specialised terminology, abbreviations, numerical expressions, and variable descriptive formulations. Diseases, symptoms, and procedures may also differ in lexical form and annotation granularity across translations. Projection must therefore preserve both conceptual equivalence and exact span boundaries. Building on these efforts, MultiClinAI provides a common benchmark for multilingual clinical information extraction and cross-lingual corpus construction [5]. It comprises MultiClinNER, which evaluates clinical entity recognition across seven languages, and MultiClinCorpus, which projects Disease, Symptom, and Procedure annotations from Spanish into English, Dutch, Italian, Romanian, Swedish, and Czech. 1.1 Related Work Cross-lingual annotation projection transfers annotations from a source document to a semantically equivalent target-language text and has been used to create resources for languages with limited manually labelled data [30]. Unlike NER, the source entity, boundaries, and semantic category are known; the task is to locate the equivalent target expression and recover its character offsets. Early approaches relied on statistical word alignment, bilingual dictionaries, machine translation, and string similarity [7]. but are less reliable under paraphrasing, reordering, expansion, or contraction. More recent methods use multilingual contextual representations to align tokens or rank candidate spans. Projected annotations can serve directly as multilingual corpus labels or as weak supervision for target-language NER models [19]. Direct corpus construction requires each projection to match an exact target substring while preserving entity type and annotation scope. 1.1.1 Word and Span Alignment Word alignment identifies translational correspondences between source and target tokens. Contextual multilingual encoders enable this without relying solely on lexical overlap: SimAlign extracts links without task-specific training [14]. whereas AWESoME Align fine-tunes multilingual encoders on parallel data [6]. Token-level correspondences must still be converted into valid entity spans because translations may change the number or continuity of aligned tokens. Proposed solutions include propagating source-span evidence through token alignments [23], ranking explicit target-span candidates [21]. and span-oriented alignment methods designed to preserve annotation boundaries [13]. Boundary sensitivity motivates both exact and relaxed biomedical NER evaluation [29]. MultiClinAI combines strict entity-type and character-offset matching with character-overlap metrics, distinguishing localisation errors from boundary disagreements. 1.1.2 Clinical Annotation Projection and MultiClinCorpus Clinical annotation projection has been applied through several paradigms. FRASIMED generated French clinical annotations using BERT-based alignment [31]; RodrĂguez-Miret et al. generated Catalan resources from translated Spanish corpora with expert validation [26]; and E3C-3.0 used an LLM-based semi-automatic procedure followed by human revision across several European languages [17, 11]. These studies demonstrate the feasibility of multilingual clinical corpus construction using encoder-based, translation-based, and generative projection methods. Within MultiClinAI, the MultiClinCorpus shared task provides a common benchmark for cross-lingual clinical annotation projection [5]. Submitted systems explored markedly different projection paradigms. The systems submitted to MultiClinCorpus explored three complementary projection paradigms. Team blue adopted a lightweight lexical approach based on cognate detection and fuzzy string matching [28]. ICB-UMA formulated projection as target-span candidate ranking: candidates were scored using heuristic constraints and ranked by an XGBoost model combining surface, positional, and semantic features, with uncertain predictions optionally revised through LLM-based correction [25] .ClinicalAligner26AM used a domain-adapted cross-lingual token aligner to project source-span evidence and decode target spans, with variants incorporating MultiClinNER predictions and task-specific supervision [24]. These approaches represent lexical matching, supervised span selection with generative refinement, and neural token alignment, and provide the closest methodological comparators for this work. Despite these advances, conventional projection pipelines remain dependent on explicit alignment, candidate generation, or ranking mechanisms. LLMs offer a document-level alternative that can exploit broader semantic and contextual information without explicit candidate enumeration. However, clinical corpus construction requires projected annotations to remain exactly grounded in the target text, without paraphrasing, normalization, or surface-form alteration, and to yield deterministic character offsets. The key question is therefore whether generative semantic matching can be constrained to preserve textual integrity and produce verifiable, corpus-ready annotations. This study makes three contributions. First, we formulate projection as constrained document-level text tagging, treating the target document as an immutable textual surface rather than relying on explicit alignment or pre-generated candidates. Second, we couple LLM inference with deterministic validation and character-offset reconstruction to verify tag structure, entity counts, and text preservation and produce reproducible character-level annotations. Third, we evaluate this formulation across six target languages and three entity types against supervised candidate-based and hybrid MLâLLM comparators, characterising robustness, exact-boundary accuracy, and computational trade-offs. 2 Objective The objective of this study was to determine whether cross-lingual clinical annotation projection can be formulated as a constrained, text-preserving, document-level generative task capable of producing accurate and verifiable character-level annotations for multilingual clinical corpus construction. We further assessed the robustness of this formulation across languages and entity types and characterised its accuracy and computational trade-offs relative to candidate-based supervised and hybrid projection approaches. 3 Materials and Methods We evaluated three projection strategies: supervised machine learning (ML), hybrid MLâLLM correction, and direct LLM-based projection. All were evaluated on the same official test set using Strict F1 and character-overlap F1. Strict evaluation required exact agreement in entity type, and start and end offsets; unmatched predictions and gold entities were counted as false positives and false negatives, respectively. Precision, recall, and F1 were computed as P=TâP/(TâP+FâP)P=TP/(TP+FP), R=TâP/(TâP+FâN)R=TP/(TP+FN), and F1=2ĂPĂR/(P+R)F_1=2Ă PĂ R/(P+R). To quantify partial boundary agreement, overlap similarity between a gold span g and prediction p was defined as ÏâĄ(g,p)=2Ă|gâ©p|/(|g|+|p|)Ï(g,p)=2Ă|gâ© p|/(|g|+|p|). Character precision (PcP_c) and recall (RcR_c) were obtained from the best overlaps, with Fâ1c=2ĂPcĂRc/(Pc+Rc)F1_c=2Ă P_cĂ R_c/(P_c+R_c) when Pc+Rc>0P_c+R_c>0, and 0 otherwise. The gold-standard test annotations were hidden from participants; predictions were submitted to the official MultiClinAI evaluation server11 1 http://temu.bsc.es:8080/, which returned the corresponding evaluation scores. 3.1 Data The dataset consists of parallel clinical documents released for the MultiClinCorpus shared task [16, 5]. Spanish source documents are paired with translations into six target languages: English, Czech, Italian, Dutch, Romanian, and Swedish. The annotations cover three entity types: Disease, Procedure, and Symptom. The corpus is divided into independent training and test partitions, summarised in Table 1. The complete test partition contains 3,260 parallel documents, with between 59,552 and 81,510 annotations depending on the target language and entity type. Official evaluation was performed on the gold-standard subset reported in Table 1. All test results reported in this work refer exclusively to this evaluated subset. Table 1: Training and gold-standard test data. Training values indicate the number of annotated entities; test values indicate the number of annotated entities used for gold-standard evaluation. Each target language contains 1,258 training documents. Training partition Gold-standard test subset Target language Disease Procedure Symptom Disease Procedure Symptom English 25,118 26,733 27,465 2,567 3,568 3,080 Czech 25,793 27,501 27,806 2,560 3,603 3,094 Italian 26,159 27,394 27,929 2,569 3,553 3,087 Dutch 25,733 27,445 27,675 2,584 3,654 3,096 Romanian 25,561 27,313 27,015 2,576 3,572 3,080 Swedish 25,580 27,079 27,531 2,555 3,587 3,095 To investigate complementary approaches to cross-lingual clinical entity projection, we evaluated three strategies that differ in the level at which the target span is identified and in the role assigned to the LLM. Figure 1 provides an overview of the task to be performed by the three pipelines, which are described in detail in the following subsections. Figure 1: Overview of the main clinical entity cross-lingual projection task. 3.2 Method 1: Window-based Projection with Machine Learning We formulated cross-lingual clinical entity projection as a binary classification problem over candidate spans in the target document. Given an annotated source entity and its parallel target document, candidate windows with lengths close to the source entity were generated. During training, the aligned target entity constituted the positive instance, while nearby and sampled windows constituted negative instances (Figure 2). Figure 2: Overview of the window-based machine-learning approach for cross-lingual clinical entity projection. Candidate spans are generated in the target document, represented through surface, positional, structural, and semantic features, and ranked by a supervised binary classifier to select the final projected span. Candidate generation combined local hard negatives surrounding the aligned target entity with soft negatives sampled from the target document. Each source entityâcandidate pair was represented by 20 features covering length, positional differences, token and character similarity, word and character n-gram overlap, term frequencyâinverse document frequency (TFâIDF) cosine similarity, Levenshtein and semantic similarity, JensenâShannon and Hellinger distances, BLEU, and METEOR [27, 20, 1]. Absolute character positions, document identifiers, and raw text were excluded from the feature matrix; class labels were used only as supervision. For each sourceâtarget language pair and entity type, source mentions were partitioned into training (80%) and validation (20%) subsets. All candidate windows derived from the same mention were assigned to the same subset to prevent information leakage. Each classifier was trained on the training subset, with its decision threshold selected by maximising validation F1. We compared seven classifiers, sharing training and evaluation protocol: Random Forest, Extra Trees, Histogram Gradient Boosting, class-balanced logistic regression, a support vector machine (SVM) with a radial basis function (RBF) kernel, a multilayer perceptron, and XGBoost [2, 10, 8, 3, 4, 15]. Random Forest and Extra Trees used 500 trees with a minimum leaf size of two; Histogram Gradient Boosting used depth 6, a 0.05 learning rate, and 300 iterations; and the RBF SVM used standardised features with C=1.0C=1.0. The multilayer perceptron used 128- and 64-unit hidden layers, ReLU activations, Adam optimisation, and early stopping. XGBoost used 500 trees, depth 4, a 0.05 learning rate, and row and column subsampling of 0.9. 3.3 Method 2: Hybrid MLâLLM Refinement The hybrid strategy used the ML projection as input and applied an LLM-based refinement step only to potentially incorrect projections. Figure 3 summarises this selective correction pipeline, including LLM-based assessment, span correction, and grounding of the resulting expression in the target document. For each target language and entity type, the classifier achieving the highest Strict F1 returned by the official MultiClinAI evaluation 22 2 http://temu.bsc.es:8080/ was selected as the input to the LLM correction stage. Gold-standard test annotations were not accessible to the participants. Figure 3: Overview of the hybrid MLâLLM projection approach. The best machine-learning projection is first assessed by an LLM; exact matches are retained, whereas uncertain or incorrect projections are corrected and subsequently grounded in the original target document to obtain valid character offsets. Following the prompt-defined decision criteria, the LLM classified each projection as an exact match, mid match, or no match. Exact matches were retained, whereas mid match and no match cases were refined using the complete target document and affected projections (see Supplementary Appendix A for prompts). The proposed expression was grounded to character offsets by exact string matching. Unique occurrences were accepted directly; when multiple matches existed, the occurrence with the start offset closest to the corresponding Spanish annotation was selected, exploiting the approximate preservation of document structure across translations. Unresolved suggestions were discarded. 3.4 Method 3: LLM Entity Projection The third approach formulated annotation transfer as a constrained LLM-based span-projection task. Figure 4 summarises the pipeline from tagged Spanish source and unannotated target documents to generation, validation, and character-offset reconstruction. The model inserted DISEASE, SYMPTOM, and PROCEDURE tags into the target text while preserving every original character. Source tags were normalised to this label set, and the source annotations determined which entity types were projected for each document pair. For each document pair, the model received the tagged Spanish source and untagged target document. The prompt (Supplementary Appendix A) specified the active labels and required the model to return only the tagged target text, prohibiting translation, paraphrasing, commentary, or any modification beyond XML tag insertion. Tag counts for each active label were required to match the source annotations. Generation and validation were treated separately. Outputs were checked for balanced tags, matching sourceâtarget entity counts, and preservation of the target text after tag removal, using canonicalised line endings and HTML entities. Invalid outputs were retained as .invalid.txt sidecars for auditability, while any recoverable mentions resolvable to valid target spans remained included in the quantitative evaluation. Predictions were converted from inline XML to character offsets by sequentially parsing the output, removing clinical tags, and tracking cumulative character position. This directly yielded start and end offsets, including for repeated expressions, without separate occurrence selection. Strict evaluation required exact agreement in document, entity type, and offsets, while character-overlap F1 captured partial boundary agreement. Figure 4: Overview of the direct LLM-based entity projection approach. The tagged Spanish source document and its unannotated parallel translation are provided to the LLM under constrained generation instructions. The resulting tagged target text is validated and converted into character-offset entity spans for evaluation. 3.5 Models and Inference Settings We evaluated three models: gemma4:31b, qwen3.6:35b, and glm-5.2 [9, 22, 12]. The Gemma and Qwen models were served locally through an internal server, whereas GLM-5.2 was accessed through the Ollama cloud service. All models received the same sourceâtarget document pairs, prompt template, entity-label constraints, and post-processing procedure. For the local runs, maximum generation length was set to num_predict=4096 tokens, and the context window was set to num_ctx=8192. Streaming was enabled, think was set to false, and keep_alive was set to 30m. The connection timeout was set to 30 s and entity_count_max_attempts was set to 5. Temperature was set to zero. For the GLM-5.2 cloud runs, streaming was disabled and temperature was set to zero. The request timeout was set to 600 s, up to three API-level attempts were permitted per document, and a 0.2 s delay was applied between requests. No explicit num_ctx, num_predict, keep_alive, or think parameter was supplied to the cloud endpoint; these settings therefore remained under the provider defaults. 4 Results Results compare candidate-based projection with generative refinement, direct document-level projection across languages and entity types, exact-boundary performance against previous methods, and the associated computational trade-offs. Previous state-of-the-art results are used as an external reference for assessing the magnitude and consistency of the observed improvements. 4.1 Candidate-based Projection and the Effect of Generative Refinement Table 2: Best ML model and LLM refinement by target language and entity type. The ML model is selected by the highest test Strict F1 among the evaluated classifiers. Bold indicates the larger Strict F1 within each MLâLLM pair. Dashes indicate that LLM refinement was not completed. Lang Entity ML Model System Strict Char F1 P R F1 TP FP FN EN Disease Extra Trees ML model 0.7222 0.7242 0.7232 1859 715 708 0.8529 LLM Ref. 0.8899 0.8878 0.8888 2279 282 288 0.9522 Procedure Random Forest ML model 0.6561 0.6508 0.6534 2322 1217 1246 0.7966 LLM Ref. 0.8343 0.8299 0.8321 2961 588 607 0.9259 Symptom Extra Trees ML model 0.6367 0.6321 0.6344 1947 1111 1133 0.8007 LLM Ref. 0.8759 0.8734 0.8747 2690 381 390 0.9561 CS Disease Extra Trees ML model 0.4480 0.4461 0.4471 1142 1407 1418 0.5984 LLM Ref. 0.7982 0.7836 0.7909 2006 507 554 0.8912 Procedure Extra Trees ML model 0.4539 0.4294 0.4413 1547 1861 2056 0.5749 LLM Ref. 0.7587 0.7286 0.7433 2625 835 978 0.8477 Symptom Extra Trees ML model 0.3736 0.3681 0.3708 1139 1910 1955 0.5605 LLM Ref. 0.7521 0.7414 0.7467 2294 756 800 0.8754 IT Disease Extra Trees ML model 0.7086 0.7088 0.7087 1821 749 748 0.7571 LLM Ref. 0.8993 0.8898 0.8945 2286 256 283 0.9503 Procedure Extra Trees ML model 0.6630 0.6583 0.6606 2339 1189 1214 0.7271 LLM Ref. 0.8573 0.8486 0.8529 3015 502 538 0.9249 Symptom Logistic Regression ML model 0.6310 0.6093 0.6200 1881 1100 1206 0.7980 LLM Ref. 0.8259 0.8024 0.8140 2477 522 610 0.9319 NL Disease Extra Trees ML model 0.5748 0.5712 0.5730 1476 1092 1108 0.6671 LLM Ref. 0.8008 0.7810 0.7908 2018 502 566 0.8853 Procedure Extra Trees ML model 0.5618 0.5290 0.5449 1933 1508 1721 0.6431 LLM Ref. 0.7681 0.7351 0.7512 2686 811 968 0.8592 Symptom Extra Trees ML model 0.4839 0.4761 0.4800 1473 1571 1621 0.6194 LLM Ref. 0.7567 0.7489 0.7528 2317 745 777 0.8911 RO Disease Extra Trees ML model 0.6527 0.6522 0.6524 1680 894 896 0.7313 LLM Ref. 0.8659 0.8521 0.8589 2195 340 381 0.9375 Procedure Extra Trees ML model 0.6119 0.5971 0.6044 2133 1353 1439 0.7072 LLM Ref. 0.8472 0.8287 0.8378 2960 534 612 0.9274 Symptom Extra Trees ML model 0.5084 0.5026 0.5055 1548 1497 1532 0.6155 LLM Ref. 0.8021 0.7987 0.8004 2460 607 620 0.9150 SV Disease Random Forest ML model 0.5439 0.5436 0.5437 1389 1165 1166 0.6635 LLM Ref. 0.8002 0.7933 0.7968 2027 506 528 0.8918 Procedure Extra Trees ML model 0.5343 0.5040 0.5187 1808 1576 1779 0.6362 LLM Ref. 0.7807 0.7513 0.7657 2695 757 892 0.8749 Symptom Extra Trees ML model 0.4672 0.4582 0.4626 1418 1617 1677 0.6309 LLM Ref. 0.7900 0.7780 0.7840 2408 640 687 0.9184 Candidate-based projection showed substantial variation across languages and entity types (Table 2). Tree-based ensembles provided the strongest ML baselines, yielding the highest Strict F1 in 17 of the 18 languageâentity settings: Extra Trees was selected in 15 cases, Random Forest in two, and logistic regression in one. Performance was highest for English and Italian and lowest for Czech, while Disease consistently achieved higher Strict F1 than Procedure and Symptom. Mean Strict F1 for Symptom was 0.5122, compared with 0.5706 for Procedure. These differences indicate that candidate-based projection is sensitive to the linguistic and boundary characteristics of the target expression despite the inclusion of multilingual semantic features. Generative refinement substantially reduced this dependency on the initial ML decision. In every completed comparison, LLM refinement increased Strict F1, with a mean absolute gain of 0.2462, while reducing both false-positive and false-negative projections. Character-overlap F1 increased from a mean of 0.6878 for the ML baselines to 0.9087 after refinement, corresponding to a mean absolute gain of 0.2209. The larger character-overlap scores relative to Strict F1 indicate that a substantial proportion of the residual errors involved inaccurate span boundaries rather than complete failure to localise the target entity. Nevertheless, because refinement operates on an upstream candidate-based projection, its ability to recover an entity remains dependent on the information supplied by the preceding projection stage. 4.2 Robustness of Document-level Projection across Languages and Entity Types Table 3: Strict span-level results and character-overlap F1 by target language and entity type. Bold indicates the highest Strict F1 among the available models within each pair. Lang Entity Model Strict Char P R F1 TP FP FN F1 EN Disease Gemma4:31B 0.9590 0.9657 0.9623 2479 106 88 0.9868 Qwen3.6:35B 0.8915 0.8703 0.8807 2234 272 333 0.9493 GLM 5.2 0.9684 0.9677 0.9680 2484 81 83 0.9851 Procedure Gemma4:31B 0.9294 0.9302 0.9298 3319 252 249 0.9780 Qwen3.6:35B 0.8947 0.8307 0.8615 2964 349 604 0.9303 GLM 5.2 0.9401 0.9372 0.9387 3344 213 224 0.9786 Symptom Gemma4:31B 0.9445 0.9445 0.9445 2909 171 171 0.9826 Qwen3.6:35B 0.8683 0.8390 0.8534 2584 392 496 0.9396 GLM 5.2 0.9599 0.9471 0.9534 2917 122 163 0.9795 CS Disease Gemma4:31B 0.9135 0.9199 0.9167 2355 223 205 0.9719 Qwen3.6:35B 0.8571 0.8270 0.8417 2117 353 443 0.9275 GLM 5.2 0.9250 0.9305 0.9278 2382 193 178 0.9740 Procedure Gemma4:31B 0.8896 0.8837 0.8867 3184 395 419 0.9645 Qwen3.6:35B 0.8627 0.7880 0.8236 2839 452 764 0.9135 GLM 5.2 0.8979 0.8912 0.8946 3211 365 392 0.9646 Symptom Gemma4:31B 0.9038 0.8992 0.9015 2782 296 312 0.9710 Qwen3.6:35B 0.8427 0.7999 0.8208 2475 462 619 0.9283 GLM 5.2 0.9196 0.9131 0.9163 2825 247 269 0.9752 IT Disease Gemma4:31B 0.9656 0.9720 0.9688 2497 89 72 0.9874 Qwen3.6:35B 0.9025 0.8505 0.8758 2185 236 384 0.9397 GLM 5.2 0.9670 0.9685 0.9677 2488 85 81 0.9851 Procedure Gemma4:31B 0.9451 0.9552 0.9502 3394 197 159 0.9833 Qwen3.6:35B 0.9208 0.8283 0.8721 2943 253 610 0.9214 GLM 5.2 0.9430 0.9538 0.9484 3389 205 164 0.9851 Symptom Gemma4:31B 0.9203 0.9206 0.9205 2842 246 245 0.9860 Qwen3.6:35B 0.8044 0.7752 0.7895 2393 582 694 0.9354 GLM 5.2 0.9391 0.9397 0.9394 2901 188 186 0.9900 NL Disease Gemma4:31B 0.8787 0.8777 0.8782 2268 313 316 0.9417 Qwen3.6:35B 0.8282 0.7910 0.8092 2044 424 540 0.9018 GLM 5.2 0.8949 0.8932 0.8941 2308 271 276 0.9463 Procedure Gemma4:31B 0.8579 0.8407 0.8492 3072 509 582 0.9340 Qwen3.6:35B 0.8216 0.7411 0.7793 2708 588 946 0.8805 GLM 5.2 0.8640 0.8487 0.8563 3101 488 553 0.9371 Symptom Gemma4:31B 0.8343 0.8311 0.8327 2573 511 523 0.9335 Qwen3.6:35B 0.7664 0.7258 0.7455 2247 685 849 0.8862 GLM 5.2 0.8315 0.8224 0.8269 2546 516 550 0.9318 RO Disease Gemma4:31B 0.9532 0.9557 0.9544 2462 121 114 0.9854 Qwen3.6:35B 0.9019 0.8498 0.8751 2189 238 387 0.9406 GLM 5.2 0.9560 0.9604 0.9582 2474 114 102 0.9877 Procedure Gemma4:31B 0.9420 0.9418 0.9419 3364 207 208 0.9827 Qwen3.6:35B 0.9152 0.8099 0.8593 2893 268 679 0.9120 GLM 5.2 0.9448 0.9490 0.9469 3390 198 182 0.9859 Symptom Gemma4:31B 0.9240 0.9234 0.9237 2844 234 236 0.9831 Qwen3.6:35B 0.8324 0.8016 0.8167 2469 497 611 0.9369 GLM 5.2 0.9487 0.9484 0.9485 2921 158 159 0.9868 SV Disease Gemma4:31B 0.8913 0.9014 0.8963 2303 281 252 0.9626 Qwen3.6:35B 0.8283 0.8082 0.8181 2065 428 490 0.9228 GLM 5.2 0.9031 0.9151 0.9090 2338 251 217 0.9676 Procedure Gemma4:31B 0.8935 0.8938 0.8937 3206 382 381 0.9697 Qwen3.6:35B 0.8539 0.7775 0.8140 2789 477 798 0.9124 GLM 5.2 0.8948 0.8918 0.8933 3199 376 388 0.9694 Symptom Gemma4:31B 0.8903 0.8866 0.8885 2744 338 351 0.9730 Qwen3.6:35B 0.8262 0.7819 0.8035 2420 509 675 0.9206 GLM 5.2 0.8801 0.8682 0.8741 2687 366 408 0.9650 Direct document-level projection achieved high Strict F1 across all six target languages and three entity types (Table 3). GLM 5.2 obtained the highest overall mean Strict F1 (0.9201), followed closely by Gemma4:31B (0.9133), while Qwen3.6:35B obtained 0.8300; GLM 5.2 led in 13 of 18 languageâentity combinations and Gemma4:31B in five. English, Italian, and Romanian produced the strongest overall results, whereas Dutch was the most challenging target language. Disease was the highest-performing entity type for all three models, followed by Procedure and Symptom. Character-overlap F1 was consistently higher than Strict F1, with macro-averages of 0.9719 for GLM 5.2, 0.9710 for Gemma4:31B, and 0.9222 for Qwen3.6:35B. This indicates that many residual errors involved entity-boundary mismatches rather than localisation of an unrelated span. 4.3 Exact-boundary Recovery and Improvement over Previous Projection Methods Table 4: Exact-span projection performance relative to the strongest previously reported MultiClinCorpus method. Results are reported using Strict F1, requiring exact agreement of entity type and character offsets. Î 1 denotes the absolute improvement over the previous best method. Lang Entity Previous method Previous F1 Best model Strict F1 Î 1 EN Disease CA26AM+MCAI 0.8960 GLM 5.2 0.9680 +0.0720 Procedure CA26AM+MCAI 0.8410 GLM 5.2 0.9387 +0.0977 Symptom CA26AM+MCAI 0.8790 GLM 5.2 0.9534 +0.0744 CS Disease CA26AM+MCAI 0.8510 GLM 5.2 0.9278 +0.0768 Procedure CA26AM+MCAI 0.8190 GLM 5.2 0.8946 +0.0756 Symptom CA26AM+MCAI 0.8110 GLM 5.2 0.9163 +0.1053 IT Disease CA26AM+MCAI 0.8820 Gemma4:31B 0.9688 +0.0868 Procedure CA26AM+MCAI 0.7990 Gemma4:31B 0.9502 +0.1512 Symptom CA26AM+MCAI 0.8310 GLM 5.2 0.9394 +0.1084 NL Disease CA26AM+MCAI 0.8200 GLM 5.2 0.8941 +0.0741 Procedure CA26AM+MCAI 0.7700 GLM 5.2 0.8563 +0.0863 Symptom CA26AM+MCAI 0.7680 Gemma4:31B 0.8327 +0.0647 RO Disease CA26AM+MCAI 0.8980 GLM 5.2 0.9582 +0.0602 Procedure CA26AM+MCAI 0.8560 GLM 5.2 0.9469 +0.0909 Symptom CA26AM+MCAI 0.8610 GLM 5.2 0.9485 +0.0875 SV Disease CA26AM+MCAI 0.8280 GLM 5.2 0.9090 +0.0810 Procedure CA26AM+MCAI 0.8060 Gemma4:31B 0.8937 +0.0877 Symptom CA26AM+MCAI 0.8110 Gemma4:31B 0.8885 +0.0775 Average 0.8348 â 0.9214 +0.0866 The gains observed with document-level projection were preserved under the strictest evaluation criterion, which requires exact agreement of entity type and character boundaries. As shown in Table 4, the best document-level configuration exceeded the strongest previously reported MultiClinCorpus method, CA26AM+MCAI [24], in all 18 languageâentity combinations. Mean Strict F1 increased from 0.8348 to 0.9214, an absolute improvement of 0.0866. Gains were observed for every language and entity type and ranged from +0.0602 for Romanian Disease to +0.1512 for Italian Procedure. The consistency of these improvements under exact-span evaluation is particularly relevant for corpus construction, because successful projection requires not only localisation of the corresponding clinical concept but also recovery of the precise textual boundaries needed to generate reproducible character-level annotations. 4.4 Accuracy-Efficiency Trade-offs The projection paradigms exhibited substantially different computational profiles. GLM 5.2 required an average of 6.54 s per document through cloud inference, whereas locally deployed Gemma4:31B required 27.01 s per document, with a mean generation rate of 33.32 tokens/s. In comparison, our local open-source implementation of the ClinicalAligner33 3 BiomedicalAligner approach required approximately 0.1 s per document after training. ClinicalAligner additionally required a dedicated training stage, estimated at approximately 2â4 h on an RTX 3090 in our implementation, whereas the LLM approaches can be applied directly without task-specific training. Under these measurements, the approximate point at which the initial alignment-model training cost is amortised is 1,118â2,236 documents relative to GLM 5.2 and 268â535 documents relative to Gemma4:31B. These values should be interpreted as implementation- and hardware-dependent estimates rather than intrinsic properties of the models. 5 Discussion Our findings indicate that cross-lingual clinical annotation projection can be effectively reformulated as a constrained document-level generation problem, provided that generative inference is coupled with deterministic text-preservation validation and character-level grounding. This formulation substantially outperformed the current state of the art for cross-lingual clinical annotation projection. Rather than relying on explicit alignment or candidate generation and ranking, the LLM operates on the complete parallel documents and directly reconstructs the annotated target text. This formulation appears particularly well suited to clinical projection, where translation often preserves the underlying clinical information while modifying lexical form, word order, or entity boundaries. Gemma4:31B is particularly relevant because, like CA26AM+MCAI, it can be deployed entirely locally. Its consistent advantage over the previous state of the art despite requiring no specialised alignment architecture indicates that document-level generation can improve both semantic localisation and exact span reconstruction. The strict improvement ranged from +0.0564 to +0.1512 F1, with the largest gains observed for Procedure entities. These gains indicate not only successful semantic localisation but also more accurate reconstruction of the complete annotated span. The results across languages also show that projection difficulty cannot be explained solely by the expected availability of language resources. English, Italian, and Romanian generally produce the strongest LLM results, whereas Dutch is the most challenging target language. Czech and Swedish, despite being comparatively less represented in many multilingual NLP resources, remain competitive under direct LLM projection. This suggests that document-level contextual reasoning can partially compensate for weaker lexical correspondence or lower representation during multilingual pretraining, while also indicating that linguistic resource availability alone does not determine projection performance. A clearer linguistic pattern emerges for the ML approach. Among the non-English targets, Italian and Romanian benefit most from candidate-based projection, while performance decreases for more distant languages and is particularly limited for Czech. This behaviour is consistent with the feature representation used by the classifier, which combines lexical overlap, edit similarity, character and word n-grams, positional information, and multilingual semantic similarity. Romance languages provide stronger surface and structural correspondence with Spanish, making the candidate-ranking problem easier. In contrast, direct LLM projection exhibits substantially less dependence on these explicit form-level similarities. The results obtained with the hybrid strategy further suggest that the main advantage of LLM-based projection arises when the model can reason over the complete projection problem rather than being restricted to revising candidates produced by an upstream system. Candidate-based methods remain attractive because they substantially constrain the search space and reduce inference requirements, but their attainable performance is limited by candidate generation: if the correct target span is absent or poorly represented among the candidates, subsequent classification or generative refinement cannot fully recover it. Direct projection removes this dependency by locating and annotating the corresponding entity directly within the complete target document. These findings are particularly relevant for multilingual corpus construction. A single LLM can be applied across languages and entity types without training separate projection classifiers or maintaining a specialised alignment model for each setting. In particular, locally deployable models make it possible to retain the complete projection workflow within institutional infrastructure, an important consideration in clinical NLP settings in which governance, privacy, or data-sharing restrictions may preclude externally hosted inference services. Nevertheless, direct LLM projection introduces failure modes that differ from those of conventional alignment systems. Generative models may alter the target text, omit annotations, duplicate entities, or produce malformed tags. Deterministic validation and offset reconstruction are therefore not merely post-processing conveniences but integral components of the proposed formulation: they ensure that accepted projections preserve the original target text and can be converted into reproducible character-level annotations before incorporation into the resulting corpus. The different projection paradigms also involve a clear accuracyâefficiency trade-off. Once training has been amortised, alignment-based projection remains considerably more computationally efficient for large or repeatedly processed collections. Direct LLM projection instead exchanges higher per-document inference cost for substantially higher exact-span accuracy, the absence of task-specific training, and a simpler projection pipeline. This trade-off is particularly relevant for local deployment, where both LLM-based and alignment-based approaches can operate without external inference services and the preferred strategy can therefore be selected according to corpus size, computational resources, and required annotation quality. 5.1 Limitations This study has several limitations. First, all experiments were conducted within the MultiClinCorpus benchmark, using Spanish as the single source language and six European target languages. Although these languages include Romance, Germanic, and Slavic families, the extent to which the observed robustness generalises to other source languages, more typologically distant target languages, or independently authored multilingual clinical documents remains unknown. The evaluation was also restricted to three entity typesâDisease, Symptom, and Procedureâand therefore does not establish performance for other clinical concepts or more complex annotation structures. External validation on additional clinical corpora and language pairs will be necessary to assess the generalisability of the proposed formulation. Second, evaluation relied on the official gold-standard span annotations and quantified exact and partial boundary agreement, but we did not conduct an additional bilingual or clinical-expert error analysis of the residual projections. Deterministic validation verifies tag structure, entity counts, text preservation, and character-offset reconstruction, but cannot by itself establish the semantic correctness of a structurally valid projected span. Similarly, the present study evaluates annotation projection directly rather than the downstream utility of the resulting projected corpora; future work should determine whether models trained on automatically projected annotations retain performance when applied to independently annotated, native-language clinical text. Finally, only three LLMs and specific inference configurations were evaluated, and computational measurements depend on model implementation, serving infrastructure, and hardware. Accordingly, absolute efficiency estimates and the relative advantage of particular models should not be interpreted as invariant across deployment environments or future model versions. 6 Conclusion This study demonstrates that cross-lingual clinical annotation projection can be formulated as a constrained, text-preserving document-level generative task capable of producing accurate and verifiable character-level annotations across multiple languages. Direct LLM projection consistently outperformed candidate-based supervised, hybrid, and previous state-of-the-art projection approaches across six target languages and three clinical entity types, while avoiding the need for explicit alignment, candidate generation, or task-specific training. Across the evaluated corpus, the best LLM configuration for each languageâentity combination produced 55,416 projected clinical mentions across six target languages and three entity types. By transferring expert annotations from a single source language while preserving exact textual grounding and reproducible character offsets, the proposed formulation provides a practical mechanism for extending annotated clinical resources, particularly where manually annotated corpora and language-specific NLP resources remain limited. Although direct LLM projection entails higher inference costs than specialised alignment-based approaches, it combines strong exact-span accuracy with a simpler task-specific workflow and, when locally deployed, can operate entirely within institutional infrastructure. Coupled with deterministic validation and offset reconstruction, this makes text-preserving document-level projection a practical and auditable strategy for multilingual clinical corpus construction, with the potential to reduce the amount of expert annotation required when extending existing resources across languages. Funding This work was supported by the Ministerio de Ciencia e InnovaciĂłn (MICINN) under project PID2024-155334OB-I00. Funding for the open access charge was provided by Universidad de MĂĄlaga / CBUA. References [1] S. Banerjee and A. Lavie (2005) METEOR: an automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for MT and/or Summarization, p. 65â72. Cited by: §3.2. [2] L. Breiman (2001) Random forests. Machine Learning 45 (1), p. 5â32. Cited by: §3.2. [3] T. Chen and C. Guestrin (2016) XGBoost: a scalable tree boosting system. In Proceedings of KDD, p. 785â794. Cited by: §3.2. [4] C. Cortes and V. Vapnik (1995) Support-vector networks. Machine Learning 20 (3), p. 273â297. Cited by: §3.2. [5] F. G. Donoso, S. Lima-Lopez, J. Rosell, E. FarrĂ©-Maduel, and M. Krallinger (2026) The MultiClinAI shared task on multilingual clinical corpus construction and concept extraction: systems, evaluation, and datasets. In Proceedings of the 11th Social Media Mining for Health Research and Applications (SMM4H-HeaRD 2026) Workshop and Shared Tasks, G. Lopez-Garcia and G. Gonzalez-Hernandez (Eds.), San Diego, United States, p. 309â331. External Links: Document, Link, ISBN 979-8-89176-432-3 Cited by: §1.1.2, §1, §3.1. [6] Z. Dou and G. Neubig (2021) Word alignment by fine-tuning embeddings on parallel corpora. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, P. Merlo, J. Tiedemann, and R. Tsarfaty (Eds.), Online, p. 2112â2128. External Links: Document, Link Cited by: §1.1.1. [7] M. Ehrmann, M. Turchi, and R. Steinberger (2011) Building a multilingual named entity-annotated corpus using annotation projection. In Proceedings of the International Conference Recent Advances in Natural Language Processing 2011, R. Mitkov and G. Angelova (Eds.), Hissar, Bulgaria, p. 118â124. External Links: Link Cited by: §1.1, §1. [8] J. H. Friedman (2001) Greedy function approximation: a gradient boosting machine. The Annals of Statistics 29 (5), p. 1189â1232. Cited by: §3.2. [9] Gemma-Team (2026) Gemma 4 technical report. External Links: 2607.02770, Link Cited by: §3.5. [10] P. Geurts, D. Ernst, and L. Wehenkel (2006) Extremely randomized trees. Machine Learning 63 (1), p. 3â42. Cited by: §3.2. [11] S. Ghosh, B. Altuna, S. Farzi, P. Ferrazzi, A. Lavelli, G. Mezzanotte, M. Speranza, and B. Magnini (2025) Low-resource information extraction with the european clinical case corpus. External Links: 2503.20568, Link Cited by: §1.1.2. [12] GLM-5-Team (2026) GLM-5: from vibe coding to agentic engineering. External Links: 2602.15763, Link Cited by: §3.5. [13] L. Jacqmin, G. Marzinotto, J. Gromada, E. Szczekocka, R. KoĆodyĆski, and G. Damnati (2021) SpanAlign: efficient sequence tagging annotation projection into translated data applied to cross-lingual opinion mining. In Proceedings of the Seventh Workshop on Noisy User-generated Text (W-NUT 2021), W. Xu, A. Ritter, T. Baldwin, and A. Rahimi (Eds.), Online, p. 238â248. External Links: Link, Document Cited by: §1.1.1. [14] M. Jalili Sabet, P. Dufter, F. Yvon, and H. SchĂŒtze (2020) SimAlign: high quality word alignments without parallel training data using static and contextualized embeddings. In Findings of the Association for Computational Linguistics: EMNLP 2020, T. Cohn, Y. He, and Y. Liu (Eds.), Online, p. 1627â1643. External Links: Document, Link Cited by: §1.1.1. [15] D. P. Kingma and J. Ba (2015) Adam: a method for stochastic optimization. In Proceedings of ICLR, Cited by: §3.2. [16] S. Lima LĂłpez, J. Rosell, J. RodrĂguez Miret, F. Gallego-Donoso, and M. Krallinger (2026) MultiClinAI corpus: multilingual clinical entity annotation projection and extraction. Zenodo. External Links: Document, Link Cited by: §3.1. [17] B. Magnini, B. Altuna, A. Lavelli, A. Minard, M. Speranza, and R. Zanoli (2023) European clinical case corpus. In European Language Grid: A Language Technology Platform for Multilingual Europe, G. Rehm (Ed.), p. 283â288. External Links: ISBN 978-3-031-17258-8, Document, Link Cited by: §1.1.2. [18] A. NĂ©vĂ©ol, H. Dalianis, S. Velupillai, G. Savova, and P. Zweigenbaum (2018) Clinical natural language processing in languages other than english: opportunities and challenges. Journal of Biomedical Semantics 9 (1), p. 12. External Links: Document, Link Cited by: §1. [19] J. Ni, G. Dinu, and R. Florian (2017) Weakly supervised cross-lingual named entity recognition via effective annotation and representation projection. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), R. Barzilay and M. Kan (Eds.), Vancouver, Canada, p. 1470â1480. External Links: Document, Link Cited by: §1.1. [20] K. Papineni, S. Roukos, T. Ward, and W. Zhu (2002) BLEU: a method for automatic evaluation of machine translation. In Proceedings of ACL, p. 311â318. Cited by: §3.2. [21] A. Politov, O. Shkalikov, R. JĂ€kel, and M. FĂ€rber (2025) Revisiting projection-based data transfer for cross-lingual named entity recognition in low-resource languages. In Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025), R. Johansson and S. Stymne (Eds.), Tallinn, Estonia, p. 499â507. External Links: Link, ISBN 978-9908-53-109-0 Cited by: §1.1.1. [22] Qwen Team, Alibaba Group (2026) Qwen3.6-35b. Note: https://qwen.ai/blog?id=qwen3.6-35b-a3b Cited by: §3.5. [23] F. Remy (2026) ClinicalAligner26AM: a cross-lingual aligner for dataset translation; evidences from the multiclincorpus shared task. External Links: 2606.08673, Link Cited by: §1.1.1. [24] F. Remy (2026) Parallia at #SMM4H-HeaRD 2026: ClinicalAligner26AM: a cross-lingual aligner for dataset translation; evidences from the MultiClinCorpus shared task. In Proceedings of the 11th Social Media Mining for Health Research and Applications (SMM4H-HeaRD 2026) Workshop and Shared Tasks, G. Lopez-Garcia and G. Gonzalez-Hernandez (Eds.), San Diego, United States, p. 165â172. External Links: Document, Link, ISBN 979-8-89176-432-3 Cited by: §1.1.2, §4.3. [25] Ă. Rey-Blanes, S. GimĂ©nez-GĂłmez, F. J. Veredas, and F. J. Moreno-Barea (2026) ICB-UMA at #SMM4HâHeaRD 2026: hybrid clinical entity projection for MultiClinAI: adaptive candidate windows, XGBoost, and LLM refinement. In Proceedings of the 11th Social Media Mining for Health Research and Applications (SMM4H-HeaRD 2026) Workshop and Shared Tasks, G. Lopez-Garcia and G. Gonzalez-Hernandez (Eds.), San Diego, United States, p. 127â132. External Links: Document, Link, ISBN 979-8-89176-432-3 Cited by: §1.1.2. [26] J. RodrĂguez-Miret, E. FarrĂ©-Maduell, S. Lima-LĂłpez, L. Vigil, V. Briva-Iglesias, and M. Krallinger (2024) Exploring the potential of neural machine translation for cross-language clinical natural language processing (nlp) resource generation through annotation projection. Information 15 (10). External Links: Link, ISSN 2078-2489, Document Cited by: §1.1.2. [27] G. Salton and C. Buckley (1988) Term-weighting approaches in automatic text retrieval. Information Processing & Management 24 (5), p. 513â523. Cited by: §3.2. [28] K. Sharma, R. Singhal, and J. Bedi (2026) Blue at SMM4H-HeaRD 2026: class-weighted transformer ensembles with structured decoding and chain-of-thought blending across six health NLP shared tasks. In Proceedings of the 11th Social Media Mining for Health Research and Applications (SMM4H-HeaRD 2026) Workshop and Shared Tasks, G. Lopez-Garcia and G. Gonzalez-Hernandez (Eds.), San Diego, United States, p. 72â81. External Links: Link, Document, ISBN 979-8-89176-432-3 Cited by: §1.1.2. [29] R. T. Tsai, S. Wu, W. Chou, Y. Lin, D. He, J. Hsiang, T. Sung, and W. Hsu (2006) Various criteria in the evaluation of biomedical named entity recognition. BMC Bioinformatics 7 (1), p. 92. External Links: Document, Link, ISSN 1471-2105 Cited by: §1.1.1. [30] D. Yarowsky, G. Ngai, and R. Wicentowski (2001) Inducing multilingual text analysis tools via robust projection across aligned corpora. In Proceedings of the First International Conference on Human Language Technology Research, External Links: Link Cited by: §1.1, §1. [31] J. Zaghir, M. Bjelogrlic, J. Goldman, S. Aananou, C. Gaudet-Blavignac, and C. Lovis (2024) FRASIMED: a clinical French annotated resource produced through crosslingual BERT-based annotation projection. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), N. Calzolari, M. Kan, V. Hoste, A. Lenci, S. Sakti, and N. Xue (Eds.), Torino, Italia, p. 7450â7460. External Links: Link Cited by: §1.1.2. Appendix A Prompts Listing A1: Match-classification prompt in English ⏠You are a strict bilingual clinical text matcher. Your task is to determine whether a <SOURCE_LANGUAGE> entity mention is represented in a <TARGET_LANGUAGE> text window. If there is any noise, consider it a no match. Source entity: <SOURCE_ENTITY> Target window: <TARGET_WINDOW> Return a JSON object with this exact schema: "match_label": "exact match | mid match | no match", "justification": "string" Label definitions: - exact match: the target window clearly contains the same concept or a direct translation of the source entity - mid match: partial, approximate, broader, narrower, abbreviated, or contextually related match - no match: the target window does not represent the same concept Rules: - Compare meaning, not literal wording - Consider translations, abbreviations, paraphrases, and morphological variants - If uncertain between exact match and mid match, choose mid match - If match_label is "exact match", justification must be an empty string - If match_label is "mid match" or "no match", justification must contain a brief explanation - Return JSON only Listing A2: Span correction prompt in English ⏠You are a biomedical NER expert. Below is a clinical document in <TARGET_LANGUAGE>, followed by entity spans that were not good matches. For each entity, use the document context and the <SOURCE_LANGUAGE> reference to suggest the most accurate <TARGET_LANGUAGE> entity string from the document. DOCUMENT: <TARGET_DOCUMENT> PROBLEMATIC ENTITIES: <PROBLEMATIC_ENTITIES> Respond ONLY with a JSON array, no explanation: ["id": 1, "suggestion": "...", ...] Listing A3: Cross-lingual projection prompt in Spanish ⏠Eres especialista en anotaciĂłn de corpus clĂnicos multilingĂŒes. Proyecta Ășnicamente las siguientes etiquetas XML desde el documento fuente en espa~nol al documento traducido. Conserva exactamente el texto traducido y devuelve Ășnicamente el documento final en el idioma que corresponda con las etiquetas insertadas. Proyecta al documento traducido las etiquetas XML clĂnicas presentes en el documento espa~nol. Reglas obligatorias: Usa Ășnicamente estas etiquetas: ACTIVE_LABELS. Conserva exactamente el texto traducido; solo puedes insertar etiquetas XML. MantĂ©n el mismo nĂșmero de etiquetas de apertura y cierre para cada etiqueta que en el documento espa~nol. No a~nadas explicaciones, markdown, comillas ni ningĂșn texto adicional. Devuelve Ășnicamente el documento traducido etiquetado. Documento espa~nol etiquetado: <documento_espa~nol_etiquetado_xml> SPANISH_XML_DOCUMENT </documento_espa~nol_etiquetado_xml> Documento traducido sin etiquetas: <documento_traducido> TARGET_DOCUMENT </documento_traducido> Listing A4: Cross-lingual projection prompt in English ⏠You are a specialist in multilingual clinical corpus annotation. Project only the following XML tags from the Spanish source document into the translated document. Preserve the translated text exactly and return only the final document in the appropriate language with the tags inserted. Project the clinical XML tags present in the Spanish document onto the translated document. Mandatory rules: Use only these tags: ACTIVE_LABELS. Preserve the translated text exactly; you may only insert XML tags. Maintain the same number of opening and closing tags for each label as in the Spanish document. Do not add explanations, markdown, quotation marks, or any additional text. Return only the tagged translated document. Tagged Spanish document: <spanish_doc_tagged_xml> SPANISH_XML_DOCUMENT </spanish_doc_tagged_xml> Untagged translated document: <translated_document> TARGET_DOCUMENT </translated_document>