Paper deep dive
PolERo: Studying Political Evasion in Romanian
Gabriel Stefan, Sergiu Nisioi
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 9/3/2026, 4:34:18 AM
Summary
This paper introduces PolERo, a dataset of 3,574 human-annotated question-answer pairs from Romanian presidential transcripts, designed to study political evasion and response clarity. The authors evaluate various NLP models, including TF-IDF baselines, fine-tuned encoders (RoBERTa, XLM-R), a proposed multi-head sliding-window encoder, and large language models (LLMs) via zero/few-shot prompting. The study highlights that fine-tuned encoders are competitive, cross-lingual transfer from English is asymmetric, and ambivalent evasion categories remain challenging. PolERo is noted as the first resource for response clarity classification in Romanian and a non-English language.
Entities (18)
Relation Signals (18)
PolERo → isinlanguage → Romanian
confidence 98% · PolERo: Studying Political Evasion in Romanian... dataset... extracted from official transcripts of five Romanian presidents.
PolERo → containsexamplesof → Political Evasion
confidence 95% · We introduce PolERo, a dataset of 3,574 human-annotated question-answer pairs... studying political evasion
PolERo → coverspresidents → Traian Băsescu
confidence 95% · official transcripts covering five Romanian presidents: ... Traian Băsescu ...
PolERo → coverspresidents → Ion Iliescu
confidence 95% · official transcripts covering five Romanian presidents: Ion Iliescu...
PolERo → coverspresidents → Klaus Iohannis
confidence 95% · official transcripts covering five Romanian presidents: ... Klaus Iohannis ...
PolERo → coverspresidents → Ilie Bolojan
confidence 95% · official transcripts covering five Romanian presidents: ... Ilie Bolojan ...
PolERo → coverspresidents → Nicușor Dan
confidence 95% · official transcripts covering five Romanian presidents: ... and Nicușor Dan.
PolERo → isevaluatedby → RoBERT-large
confidence 95% · The same five architectures are fine-tuned on PolERo, with the addition of RoBERT-large
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Political evasion refers to responses that engage with a question while withholding the requested information. Recent NLP work frames political evasion as a classification task using a two-level taxonomy of response clarity and fine-grained evasion strategies. Existing work on response clarity and evasion classification is limited to English, leaving open whether the taxonomy and model behavior transfer across languages and political contexts. We introduce PolERo, a dataset of 3,574 human-annotated question-answer pairs extracted from official transcripts of five Romanian presidents. We evaluate multiple classification approaches on both datasets under matched conditions, including TF-IDF baselines, fine-tuned encoder models, a proposed sliding-window encoder, and zero/few-shot LLM prompting. We study cross-lingual transfer through joint bilingual training and machine-translation-based data augmentation. Our results indicate that fine-tuned encoders are competitive, cross-lingual transfer is asymmetric, and ambivalent evasion categories involving pragmatic cues remain the main challenge across all model families.
Tags
Links
- Source: https://arxiv.org/abs/2609.02391v1
- Canonical: https://arxiv.org/abs/2609.02391v1
Trouble viewing inline? Open PDF directly →
Full Text
95,178 characters extracted from source content.
Expand or collapse full text
PolERo: Studying Political Evasion in Romanian Gabriel S , tefan and Sergiu Nisioi * Laboratory for Cybernetic Research and Good Life Human Language Technologies Research Center Faculty of Mathematics and Computer Science University of Bucharest gabrielstefan04@gmail.com sergiu.nisioi@unibuc.ro Abstract Political evasion refers to responses that en- gage with a question while withholding the re- quested information. Recent NLP work frames political evasion as a classification task using a two-level taxonomy of response clarity and fine-grained evasion strategies. Existing work on response clarity and evasion classification is limited to English, leaving open whether the taxonomy and model behavior transfer across languages and political contexts. We introduce PolERo, a dataset of3,574human-annotated question-answer pairs extracted from official transcripts of five Romanian presidents. We evaluate multiple classification approaches on both datasets under matched conditions, includ- ing TF-IDF baselines, fine-tuned encoder mod- els, a proposed sliding-window encoder, and zero/few-shot LLM prompting. We study cross- lingual transfer through joint bilingual training and machine-translation-based data augmenta- tion. Our results indicate that fine-tuned en- coders are competitive, cross-lingual transfer is asymmetric, and ambivalent evasion categories involving pragmatic cues remain the main chal- lenge across all model families. 1 Introduction Political interviews often contain responses that are relevant to the interviewer’s questions while at the same time remaining non-responsive. Such responses may involve implicit answers, partial replies, topic shifts (Msagalla and Visser, 2021), deflections, or strategic ambiguity (Bach et al., 2025), making it difficult to determine whether the requested information was actually provided. In political science and discourse analysis, these phe- nomena are commonly studied under the notions of equivocation and evasion (Bavelas et al., 1988; Bull, 1994; Clayman, 2001; Bull, 2003; Rasiah, 2010; Romaniuk, 2013; Bull and Strawson, 2020). * Corresponding authors. Beyond political communication, response clar- ity is also relevant to broader NLP problems in- cluding dialogue understanding, question answer- ing, and implicitly for creating better user-facing technologies. Despite recent progress in polit- ical discourse processing (Glavaš et al., 2019; Huguet Cabot et al., 2020; Afli et al., 2024), re- sponse clarity remains largely unexplored in NLP. Existing work has focused primarily on question answerability (Rajpurkar et al., 2018; Min et al., 2020), responder intent (Ferracane et al., 2021), or broader discourse phenomena in political speech (Subramanian et al., 2019; Reinig et al., 2024; Verma et al., 2026), while leaving open the problem of determining whether a response adequately ad- dresses a question. Unlike traditional QA settings (Rajpurkar et al., 2018; Min et al., 2020; Clark et al., 2020a), response clarity requires reasoning about pragmatic alignment, modeling information suffi- ciency (Jain and Garimella, 2026), and discourse structure (Mann, 1984), extending beyond standard answerability formulations. Recently, Thomas et al. (2024) introduced the first NLP dataset and taxonomy of 3,400 question- answer (QA) pairs from the U.S. presidential in- terviews. The dataset served as a reference for SemEval-2026 Task 6: CLARITY (Thomas et al., 2026) a task that aims to detect and classify re- sponse ambiguity in political discourse. Evidence for response clarity classification currently remains restricted to U.S. English political interviews. Fur- thermore, response clarity is not a purely linguistic phenomenon; it is shaped by institutional context, interaction format, and political communication practices. Currently, no resource targets response clarity or evasion classification in Romanian polit- ical speech. To the best of our knowledge, such resources are also absent for non-English languages more broadly. To address this, we introduce PolERo (Political Evasion in Romanian), a dataset of 3,574 human- arXiv:2609.02391v1 [cs.CL] 2 Sep 2026 annotated question-answer pairs extracted from official transcripts covering five Romanian presi- dents. PolERo adopts the exact annotation protocol and two-level taxonomy proposed by Thomas et al. (2024), enabling controlled comparison between English and Romanian political interviews. Romanian is an Eastern Romance language with historically limited NLP resources. Recent efforts have significantly expanded this landscape, intro- ducing general evaluation benchmarks and founda- tional corpora (Dumitrescu and Avram, 2020; Du- mitrescu et al., 2021; Rogoz et al., 2026), alongside Romanian-adapted language models (Masala et al., 2020; Dumitrescu et al., 2020; Dima et al., 2024; Masala et al., 2024). Furthermore, Romanian polit- ical communication has attracted sustained interest in discourse studies and political linguistics with a focus on political opinion mining (Gîfu and Cristea, 2013; Vasilescu et al., 2024) and hate-speech dis- course (Gheorghiu and Praisler, 2022; Dinu et al., 2025). Nevertheless, despite this progress, no prior datasets or analyses target response clarity or eva- sion classification in any type of speech. This makes PolERo, to the best of our knowledge, the first such resource for a non-English language. This paper makes several contributions: •The release of PolERo, 1 a human-annotated dataset for response clarity and evasion classi- fication in a Romanian political corpus. •Multiple classification results for both PolERo and the English dataset of Thomas et al. (2024) under matched conditions, covering en- coder models, a multi-head encoder for long text classification, and different LLM prompt- ing strategies. •A cross-lingual transfer analysis examining the consistency of evasion patterns and model behavior across the two corpora, considering low-data scenarios and machine-translation- based data augmentation. 2 Dataset and Annotation 2.1 Data Collection and Preprocessing We collect official transcripts from the Romanian presidential websites, 2 covering official press con- 1 The dataset, code, and prompts are released under the C BY 4.0 License and are available athttps://github.com/ gabriel-stefan/polero. 2 https://w.presidency.roandhttps://archive. presidency.ro ferences, interviews, and declarations from five Romanian presidents: Ion Iliescu, Traian B ̆ asescu, Klaus Iohannis, Ilie Bolojan, and Nicus , or Dan. The corpus spans 25 years between 2001–2026. In such contexts, it is common for a sin- gle interviewer question to contain multiple sub- questions. The initial pool of extracted data con- tained 6,234 raw question-response candidates, comprising 8,233 sub-questions in total. For the current task, we enforce a one-to-one question- response mapping, and we use GPT-5.4 to flag the set of instances where more than one question is asked in a single instance. As such, we remove 2,053 multi-part entries and 607 entries contain- ing no extractable question. During the annotation process, evaluators were instructed to flag any re- maining double-barreled questions in the labeling platform (see the appendix B.1). No such instances were reported, indicating that no remaining multi- part questions were present in the retained corpus. The retained corpus consists of 3,574 single-part QA pairs. We divide the dataset into a training set of 3,278 and a test set of 296 examples. Response Clear Reply Ambivalent Clear Non-Reply ExplicitImplicit General Partial/half-answer Dodging Deflection Declining Ignorance Clarification clarity evasion Figure 1: Clarity-evasion two-level taxonomy mapping. 2.2 Taxonomy PolERo adopts the two-level response clarity tax- onomy of Thomas et al. (2024), focusing on the distinction between clarity and ambiguity in politi- cal responses (see Figure 1). The first level assigns each response one of three clarity labels: Clear Reply — the information is given in the requested form, Ambivalent — a valid answer is given but ad- mits multiple interpretations, or Clear Non-Reply — the speaker openly refuses to share the informa- tion. The second level divides these classes into nine fine-grained evasion categories. Clear Reply contains only Explicit information stated in the re- quested form. The Ambivalent category includes cases where: 1. Implicit information is conveyed without being explicitly stated; 2. General information is pro- vided, lacking the requested specificity; 3. Partial half-answers are provided; 4. Dodging or ignoring the question altogether; and 5. Deflection of the topic by shifting focus to a different point. The final category, named Clear Non-Reply, includes Declining to Answer in an explicit manner while acknowledging the question, Ignorance when re- spondents admit they do not know the answer, and Clarification when respondents ask for more de- tails. An expert in linguistics, working professionally as an authorized translator, adapted the initial En- glish taxonomy for Romanian, fixing category def- initions and boundary cases. Appendix A repro- duces the Romanian annotation guidelines. 2.3 Annotation Protocol We build a screening quiz of 15 QA pairs spanning ambiguous evasion categories, where each item is labeled by an expert to serve as a benchmark ref- erence. The quiz is used as a screening step to ensure that the three annotators understand the tax- onomy and can consistently handle borderline and difficult cases. All annotators are native Romanian women aged 22–24 with undergraduate training in psychology. Annotators complete a training phase that covers all nine categories of taxonomy (subsection 2.2) using worked examples. Each instance requires a single annotation decision: assigning one of the nine evasion labels to the response. The three-way clarity label is then inferred by traversing the tax- onomy upward. The annotation process spanned five weeks. During this period, we held three group review sessions with the expert to resolve disagree- ments and clarify boundary cases. Mean annotation time was 69 seconds per instance, totaling approxi- mately 27 hours per annotator. All annotators were compensated for their work (see section 9). A single annotator labels each training instance, with the workload evenly distributed between the three. All three independently label the test split to support inter-annotator agreement measurement and evaluation using multiple annotations. For 2.4% of test instances where all three annotators as- signed different evasion labels, the expert assigned the final label. 2.4 Worked Example Î: “Domnule Pres , edinte, p ̆adurea este o chestiune de sigurant , ̆a nat , ional ̆a?” R: “Eu cred c ̆a p ̆adurile sunt o chestiune de suflet nat , ionals , i de aceea sunt foarte încrez ̆ator c ̆a vom reus , i s ̆a schimb ̆am lucrurile în bine.” English: Q: “Mr. President, are forests a matter of national security?” A: “I believe forests are a matter of national spirit, and for that reason I am very confident that we will succeed in changing things for the better.” The response receives clarity label Ambivalent and evasion label Deflection. The answer engages with the topic introduced in the question (forests as a national concern), but does not address whether forests constitute a national security issue. It re- frames the discussion and shifts the attention away from the original premise without explicitly reject- ing it. Figure 2: Overall clarity distribution for PolERo com- pared to the English CLARITY dataset. 2.5 Dataset Statistics Label distributions are similar across splits. In the training set, Clear Reply accounts for 54.9% of instances, Ambivalent for 37.7%, and Clear Non- Reply for 7.4%. The test set shows a comparable distribution (56.8%, 36.1%, 7.1%). Figure 2 compares the overall clarity label dis- tribution of PolERo with the English CLARITY dataset (Thomas et al., 2024). Clear Reply is the majority class in the Romanian dataset (55.09%), while the English corpus is primarily composed of Ambivalent responses (59.80%). Another point of departure between the two datasets is the way in which multi-part questions have been treated: in English, the interviews are decomposed into sep- arate sub-questions using ChatGPT, while in Ro- manian, we discard such instances as we consider AI models insufficient for creating gold-standard data. An exploratory analysis is provided in the Appendix C. The analysis reveals variation in re- sponse clarity across speakers, communicative set- tings, and topics. Romanian presidential discourse is dominated by Explicit answers (55.09%), al- though this proportion decreases in more formal contexts such as declarations and press conferences. Evasion strategies also vary across presidents and topics: earlier speakers rely more heavily on Dodg- ing, whereas post-2014 presidents exhibit more balanced distributions with higher rates of Gen- eral, Deflection, and Declining to answer. Finally, the length of the answer aligns with established theories of verbose evasion (Bavelas et al., 1988; Rasiah, 2010; Bull and Strawson, 2020), with in- direct strategies such as General, Deflection, and Partial responses tending to be substantially longer than direct responses. 2.6 Annotation Quality To assess the performance of the three annotators, the expert independently labeled a benchmark set of 85 instances. Against this gold standard, the annotators achieved accuracies of 95.3%, 95.3%, and 94.1% on the clarity task, and 89.4%, 88.2%, and 84.7% on the evasion task (Appendix B.3). Inter-annotator agreement on the test set, mea- sured using Fleiss’sκ(Fleiss, 1971), is0.843 for clarity and0.678for evasion, corresponding to near-perfect and substantial agreement on the scale of Landis and Koch (1977). At the cate- gory level, agreement is highest for Clarification (κ = 1.000), Claims Ignorance (κ = 0.881), and Explicit (κ = 0.858), and lowest for Deflection (κ = 0.364) and General (κ = 0.464). A de- tailed analysis of the annotation is provided in Ap- pendix B. 3 Experimental Setup 3.1 Task Formulation We report macro-averaged F1 on the test split of each dataset as the primary metric across all ex- periments. We evaluate the models on two clas- sification tasks derived from the two-level taxon- omy (subsection 2.2): (1) clarity classification, a three-class task over the coarse-grained labels; and (2) evasion classification, a nine-class task over the fine-grained labels. For the three-class clarity task, we also compute a macro-F1 score by deterministi- Tokenized sequence T = tok(Q⊕ A) Chunk1Chunk0 Chunk M−1 Shared Encoder h 1 h 0 h M−1 Max-Pooling v = max M−1 k=0 h k Dropout Clarity (3 classes) Evasion (9 classes) Figure 3: Multi-head architecture. The tokenized con- catenated input (Q⊕A) is split into overlapping chunks, each chunk is encoded by a shared encoder, chunk repre- sentations are aggregated via element-wise max-pooling, and two task-specific heads predict clarity (3 classes) and evasion (9 classes). cally (3) mapping each predicted evasion label to its parent clarity class. Thus, we can directly com- pare models trained specifically for the three-class task against those whose coarse-grained labels are inferred through the fine-grained objective. 3.2 Models We establish baselines across four model fami- lies on both the English CLARITY dataset and PolERo, training and evaluating each language in- dependently. Input formats, hyperparameters, and prompt templates are reported in Appendix D.2. A TF-IDF logistic regression classifier serves as a non-neural lower bound. The model compari- son covers single-head (SH) encoder fine-tuning, a custom multi-head (MH) encoder rendered in Figure 3, and LLMs under zero-shot and few-shot prompting without fine-tuning: GPT-5.4, Llama- 3.3-70B-Instruct, Qwen3.6-35B-A3B, Gemma-4- 31B-it, DeepSeek-V4-Pro, and gpt-oss-120b. Standard encoder configurations are finetuned in- dependently for each task. In English, we evaluate RoBERTa-large (Liu et al., 2019), ModernBERT- large (Warner et al., 2025), ELECTRA-large (Clark et al., 2020b), BERT-large-cased (Devlin et al., 2019), and XLM-RoBERTa-large (Conneau et al., 2020). The same five architectures are fine-tuned on PolERo, with the addition of RoBERT-large (Masala et al., 2020), 3 and Romanian BERT (Du- mitrescu et al., 2020), both pretrained on Roma- nian corpora, and multilingual mBERT-base (De- vlin et al., 2019). The Multi-head (MH) encoder architecture con- tains a shared encoder and two linear heads that pre- dict clarity and evasion jointly (Figure 3). To pro- cess long question-answer pairs without truncation, sequences are segmented into overlapping chunks and encoded independently. The chunk representa- tions are aggregated via element-wise max-pooling into a single response vector, which is passed to a 3-class clarity head and a 9-class evasion head. The training objective is the unweighted sum of the two cross-entropy losses. Models are trained with 7-fold stratified cross-validation, and fold pre- dictions are ensembled by probability averaging at inference time. Matching the single-head configu- rations, we train RoBERTa-large, XLM-RoBERTa- large (added to support the cross-lingual experi- ments below), and RoBERT-large, a Romanian- only encoder model. Full architectural details and mathematical formulations together with compo- nent ablations isolating the contribution of chunked pooling, joint supervision, and ensembling are re- ported in Appendix E. 4 Results and Discussion The results in this work are reported on the En- glish dataset using the development set from the SemEval 2026 CLARITY Shared Task, since the official test of the task was not publicly available at the time of writing. The multi-head RoBERTa- large achieves one of the strongest scores (0.8) on the SemEval 2026 CLARITY task (Stefan and Ni- sioi, 2026; Thomas et al., 2026) among encoder- only models, especially given the size of under 400 million parameters. 4 However, on the development set, the same model obtains a smaller.713score for the 3-way clarity task prediction. Table 1 reports the macro-F1 scores for the strongest models from each approach. A com- plete set of results for all models, cross-lingual setups, and prompting strategies is provided in Ap- pendix F. 3 We emphasize the first two letters of Romanian BERT large: RoBERT-large to increase readability and distinguish it more easily from the RoBERTa-large model, both used extensively in this work. 4 According to the official report by Thomas et al. (2026), all top-scoring systems in the range of0.82–0.89relied on proprietary LLMs. Traditional classifiers. TF-IDF + logistic re- gression reaches a macro-F1.422/0.453on En- glish/Romanian 3-class clarity detection, confirm- ing that surface-level lexical features alone are in- sufficient. Additional per-class error patterns are reported in Appendix D. Monolingual encoder fine-tuning. RoBERTa- large is the strongest single-head English encoder model for the 3-class clarity (C) identification task (.580) and the 9-class fine-grained evasion (E) de- tection task (.481). The mapped (M) fine-grained task significantly outperforms (.656) the simple clarity identification task, showing that it is bet- ter to train on a more fine-grained taxonomy and merge than to train directly on fewer un-balanced classes. Unsurprisingly, for Romanian, this model per- forms significantly worse than encoder models pre-trained on Romanian, such as RoBERT-large (Masala et al., 2020). The latter is the strongest single-head model on PolERo (.504evasion,.701 mapped). More detailed encoder comparisons are available in Appendix F. EN (CLARITY)RO (PolERo) ModelCEMCEM TF-IDF + LR.422.301.441.453.312.472 Best Encoders (Fine-Tuned) RoBERTa (SH).580.481.656.646.428.613 RoBERTa (MH).713.568.731.696.501.652 XLM-R. (SH).579.419.564.680.365.686 XLM-R. (MH).665.475.644.694.508.706 RoBERT (SH)–.688.504.701 RoBERT (MH)–.729.545.763 Best LLMs (Few-Shot Prompting) Llama-3.3-70B (few-shot).535.382.564.555.312.411 GPT-OSS-120B (few-shot).542.554.729.553.580.729 DeepSeek-V4-Pro (few-shot).601.545.670.627.594.751 Gemma-4-31B-it.721.583.662.672.619.738 Qwen3.6-35B-A3B.669.544.747.746.682.760 GPT-5.4.703.626.761.763.666.763 Table 1: Summary of test-set macro-F1 for the highest- performing models across all tested approaches. C: Clarity (direct 3-class prediction). E: Evasion (direct 9- class prediction). M: Mapped (3-class clarity prediction obtained by mapping the predicted 9-class evasion label upward through the taxonomy). SH: Single-head, MH: Multi-head. Best scores are highlighted in bold. LLMs obtain the strongest results for the fine-grained Evasion prediction, while smaller-sized multi-head encoders re- main competitive for predicting response clarity. Multi-head fine-tuning.The custom multi-head model architecture improves performance across all metrics compared to the corresponding single- head model on both datasets.Multi-head RoBERTa-large is the strongest English encoder configuration, outperforming its single-head coun- terpart by13macro-F1 points on direct clarity and 9 on evasion, and surpassing GPT-5.4 on direct clarity prediction. The multi-head RoBERT-large improves over its single-head Romanian-specific counterpart by4points on clarity,4on evasion, and6on mapped clarity. These gains are consis- tent with the dual clarity+evasion tasks acting as a regularizer: gradients from the coarser clarity su- pervision help stabilize predictions for rare evasion categories, and vice-versa. Multi-headXLM-RoBERTa-largeobtains smaller gains in both languages than the language- specialized backbones, indicating that multilingual pretraining sacrifices some within-language perfor- mance for broader cross-language coverage. As expected, models pretrained specifically for a given language consistently outperform multilingual models on that language: RoBERT-large achieves higher scores than XLM-RoBERTa-large on all Romanian evaluation metrics, while RoBERTa- large similarly outperforms XLM-RoBERTa-large on the English benchmarks. StepComponentClarityEvasion English (CLARITY), RoBERTa-large S0Single-head.597 ±.006 .474 ±.024 S1Multi-head.638 ±.007 .478 ±.013 S2Sliding-window.648 ±.024 .481 ±.040 S3Cross-validation.662 ±.017 .513 ±.016 S4Ensembling.694 ±.009 .523 ±.013 Gain over S0+.097+.049 Romanian (PolERo), RoBERT-large S0Single-head.678 ±.022 .511 ±.015 S1Multi-head.701 ±.022 .531 ±.030 S2Sliding-window.705 ±.017 .515 ±.019 S3Cross-validation.707 ±.015 .516 ±.046 S4Ensembling.721 ±.011 .537 ±.021 Gain over S0+.043+.026 Table 2: Cumulative ablation of the multi-head architec- ture. Table 2 adds each component cumulatively to a single-head baseline, repeating the full encoder pipeline for both languages, with RoBERTa-large as the English backbone and RoBERT-large as the Romanian one. Each configuration is run over three random seeds (7, 42, 123); we report mean and stan- dard deviation of test-set macro-F1 together with the increment over the preceding step. Relative to the single-head baseline, the complete architecture gains+0.097macro-F1 on English clarity,+0.049 on English evasion,+0.043on Romanian clarity and+0.026on Romanian evasion. The joint multi- head objective provides the largest single gain on clarity (+0.041English,+0.023Romanian), while cross-validation and ensembling account for most of the improvement on the evasion task (+0.032, +0.010 on English). Predicting mapped vs. direct clarity. The two- step strategy of predicting the nine fine-grained eva- sion categories and then projecting upward through the taxonomy outperforms the direct clarity head for RoBERTa-large on English, for RoBERT-large on Romanian, and for nearly every LLM configu- ration on both datasets. This is consistent with the results reported at the Shared Task (Thomas et al., 2026), where the winning reverse derivation strat- egy proved to be the single most effective approach across the entire competition. Four encoders do not benefit from this approach: mBERT-base, BERT-large-cased, ELECTRA-large, and ModernBERT-large. In each case, the eva- sion head assigns most predictions to Explicit and General, causing the projection step to overpre- dict Clear Reply at the expense of Ambivalent and Clear Non-Reply. ELECTRA-large on the Roma- nian split is the clearest example: the model pre- dicts Explicit for every instance, yielding a macro- F1 of.084, and the mapped clarity score drops to.24because Ambivalent and Clear Non-Reply are never predicted. ModernBERT-large shows the same pattern, though less severely, with two categories still receiving zero recall on Romanian (Claims ignorance and Partial/half-answer). The two-step strategy therefore depends on the back- bone being able to assign a meaningful probability mass to rare categories. Otherwise, the projection step amplifies the model’s bias toward the majority class rather than correcting it. LLM prompting. LLMs achieve some of the best results across all metrics on both datasets, though their ranking varies by task, and there are many cases when models perform significantly weaker than encoders (Table 1). Here we report only the strongest candidates; for a full set of re- sults, see Appendix F.3 and Table 11. Gemma-4- 31B-it obtains the best few-shot scores on direct clarity prediction for English, Qwen3.6-35B-A3B achieves the best score on few-shot evasion for Ro- manian, and GPT-5.4 attains strong mapped clarity scores on both, although it is on-par with the multi- head RoBERT model. DeepSeek-V4-Pro and gpt-oss-120B fall be- tween these models. Few-shot prompting improves the mapped metric for most models because the ex- amples help clarify the boundaries between similar categories, especially Dodging-Deflection and Im- plicit-General. Under zero-shot prompting, these are the category pairs that models commonly con- fuse, and even one example per category substan- tially improves alignment. Llama-3.3-70B is the exception. Its mapped clarity score on English drops from.624to.564under few-shot prompting. An analysis of the outputs reveals that the model be- comes heavily biased toward Deflection, predicting it for 79.5% of all English test instances, regardless of the gold label. The same holds for Romanian, where Deflection accounts for 44% of all predic- tions. In our experiments, the 70B Llama model obtained relatively weak results, which was unex- pected given that it is one of the strongest baselines at the SemEval Task 6 CLARITY (Thomas et al., 2026). Data contamination. The performance differ- ences between LLMs and fine-tuned encoders likely stem from multiple factors. Firstly, LLMs benefit from broad parametric world knowledge stored in large billion-parameter-sized networks, which help to resolve entity-dependent evasion. Secondly, all evaluated LLMs (except Llama-3.3- 70B) are prompted with reasoning enabled, allow- ing for potentially better deliberation over prag- matic intent. Notably, Llama-3.3-70B, which lacks reasoning, is also the weakest LLM in both lan- guages. Thirdly, we cannot rule out that these mod- els might have been exposed to the data (which is publicly available) during their pre-training phase. To assess whether the advantage of LLMs is in- fluenced by data contamination, we explore the CoDeC framework, Contamination Detection via Completion (Zawalski et al., 2026), on three open- source LLMs: Qwen3.6-35B-A3B, DeepSeek-V4- Pro, and Gemma-4-31B-it. CoDeC scores are ex- tracted solely from the text, yielding contamination percentages on the English test set of37%,38%, and45%; while on the Romanian test set, the cor- responding scores are43%,41%, and39%. All six values fall below the80%threshold that would in- dicate a clear case of data contamination, according to the Zawalski et al. (2026). Even if the models could have been exposed to the same texts dur- ing their pre-training (this includes the encoders), the associated labels were not openly available; therefore, we could not find sufficient evidence for contamination. 5 Cross-Lingual Transfer We assess cross-lingual generalization using the multi-head (MH) architecture and XLM-RoBERTa- large and RoBERTa-large, although the latter is reserved for Appendix, Table 11 since it is not designed to be a multilingual encoder. Each model is trained in multiple cross-lingual configurations: •EN→RO: trained on English, evaluated on Romanian. • RO→EN: trained on Romanian, evaluated on English. •EN + RO→RO and→EN: joint training on the concatenation of both datasets, evaluated only on Romanian or English. The first two configurations explore the ability to generalize across the two languages and political spheres (U.S. vs. Romania), while the remaining configurations explore the possibility of increas- ing the predictive power using multilingual data augmentation. Results are available in Table 3. Training on English data and predicting on Romanian yields a significant 6 to 7 point drop (.694 → .635) using both XLM- and RoBERTa- large models. Training in the opposite direction, in Romanian→English, results in a worse perfor- mance degradation of 11 to 14 points. In both sce- narios, the performance degradation makes the MH models similar to other single-head models such as mBERT-base, BERT-large-cased, ELECTRA- large, or ModernBERT (see Table 11 containing complete experimental results in the Appendix). The model trained on Romanian strongly over- predicts Clear Reply, misclassifying 54% of En- glish Ambivalent answers as Clear Reply, and sim- ilarly maps most English Implicit and Deflection responses to Explicit. These results indicate that cross-lingual predictions do not lead to catastrophic losses and that the task may benefit from additional multilingual data. Multilingual augmentation. As shown in Ta- ble 3, combining the two datasets and predicting on each test set leads to improvements over indi- vidual training of the same model. One of the best configurations for Romanian is joint EN+RO train- ing with XLM-RoBERTa-large, which achieves the highest Romanian evasion score among fine-tuned models (0.570). This may be the case because joint training regularizes rare categories: Romanian ex- amples such as Clarification or Claims ignorance are reinforced by similar English instances, reduc- ing the data scarcity that causes failures. For exam- ple, Declining to answer reaches perfect recall on the Romanian test split under joint training. Fur- thermore, joint training encourages the shared en- coder to model evasion in a cross-lingual way rather than relying on language-specific lexical cues. The English dataset contains more Ambivalent evasive answers, where evasion is expressed through topic shifts and intent rather than specific words. When trained jointly, the model may transfer this ability to Romanian, improving its ability to detect evasion without clear lexical signals. As a result, Roma- nian evasion scores increase by0.062over the in- language XLM-RoBERTa-large baseline, whereas the English scores show an insignificant improve- ment (+0.008). RoBERTa-large does not benefit in the same way (Table 11). Its tokenizer and pretraining are opti- mized for English, so adding Romanian data in- troduces more noise than useful signal, causing English evasion performance to drop from0.568 to0.503. Overall, the English performance drops consistently across all criteria when doing joint training. 5.1 Machine Translation As a secondary analysis, we extend the cross- lingual setup with machine-translated training and test data in order to reduce the biases of the mul- tilingual encoders. All translations are generated with NLLB-200-distilled-600M (NLLB Team et al., 2022). Appendix F.5 reports a semi-automatic anal- ysis of the quality estimation of the translations using XCOMET-XL (Guerreiro et al., 2024). The model indicates higher MT quality for RO→EN than EN→RO, although manual inspection shows that many EN→RO translations flagged as errors are acceptable, with pragmatic categories such as Deflection remaining the most error-prone in both translation directions. Translating the English data into Romanian par- tially recovers cross-lingual clarity performance but provides limited gains for evasion detection, according to the results presented in Table 3. Cat- egories such as Deflection and Dodging suffer the largest degradation under MT augmentation and also receive the lowest translation quality scores, EN (CLARITY)RO (PolERo) ModelCEMCEM Best encoders fine-tuned on the training set RoBERT-large (MH)–.729.545.763 RoBERTa (MH).713.568.731.–.–.– Cross-lingual XLM-R multi-head models: Individual Training.665.475.644.694.508.706 Joint Training (EN+RO).679.483.642.723.570.717 Low-data scenario; only test set available EN→ RO–.635.399.655 EN2RO→ RO–.665.412.706 RO→ EN.551.398.527– RO2EN→ EN.477.358.439– MT augmentation RO + EN2RO→ RO–.718.489.684 EN + RO2EN→ EN.694.477.602– Best LLMs (Few-Shot Prompting) Gemma-4-31B-it.721.583.662.672.619.738 Qwen3.6-35B-A3B.669.544.747.746.682.760 GPT-5.4.703.626.761.763.666.763 Table 3: The first two and the last three rows contain the best models for each language and task. The ar- row→shows the training and testing direction of the XLM-R, multi-head models. The labels EN2RO and RO2EN indicate machine translated training sets. In low-data scenarios it is best to use few shot prompting and LLMs. When data is available, joint multi-lingual training achieves better results than monolingual train- ing using machine translated data. whereas more formulaic categories such as Clarifi- cation and Declining to answer translate reliably. A transfer asymmetry between English and Roma- nian is visible — machine translating texts from Romanian to English degrades the results more than the other way around. We believe that differ- ences in class distributions and different discourse structures, shaped by historical practices, geopo- litical contexts, and topics of interest, make the Romanian data less useful for detecting evasion in U.S. English. MT-based training set augmen- tation provides inconclusive results compared to individual or joint multilingual training. 6 Remarks Based on our Romanian case study, we propose the following takeaways to guide future assessments of political evasion in new languages: 1.In low-resource settings, when creating a dataset for a new language such as Roma- nian, it is advisable to first benchmark the dataset using few-shot prompting with avail- able LLMs, preferably open-weight models. 2. If LLMs perform poorly for a particular language, translating existing datasets with NLLB-200 or similar tools and training cross- lingual encoders may be a viable alternative, although factors related to data representa- tiveness (Do ̆ gruöz et al., 2023), geopolitical factors, and local histories must be carefully taken into account. 3.If a sufficiently large training set can be con- structed for the target language (i.e., at least 3,000 samples) and existing monolingual pre- trained encoders are available, then a strong strategy is to use the multi-head approach de- scribed in Section 3; otherwise, if monolin- gual encoders are not available, then joint mul- tilingual training may be more beneficial than augmenting the data with machine-translated examples. 4. The strong performance of LLMs must be weighed against cost and efficiency; our pro- posed multi-head encoder proved competitive with LLMs on the clarity classification task while using fewer than 500M parameters; at the same time, the encoders have limitations in terms of fine-grained evasion detection. 5.The distribution of ambiguous responses can have a significant effect on the generalizability of cross-lingual models. 7 Conclusions In this work, we introduce PolERo, a novel Roma- nian dataset of3,574question-answer pairs from presidential transcripts annotated using a two-level taxonomy of evasion. We compare multiple models on an existing English dataset and on our proposed Romanian dataset under matched conditions, cov- ering traditional classifiers, standard encoder fine- tuning, a custom multi-head architecture for joint clarity and evasion prediction, and large language models under zero-shot and few-shot prompting. Our results only partially corroborate previous findings (Thomas et al., 2026) that prompting modern LLMs strongly outperforms fine-tuned en- coders. We show that highly competitive results can be obtained on a language such as Romanian using encoder models only. We propose a sliding- window and max-pooling approach with two clas- sification heads that significantly increases the pre- dictive power of encoder models, making them competitive with LLMs for the three-class clarity classification task while using only a fraction of the memory footprint. However, for the nine-class fine-grained evasion task, larger models achieve better predictive performance. Finally, we investigate cross-lingual transfer from the perspective of multilingual training versus monolingual training with machine-translated texts. Our results reveal an asymmetric relation: English- trained models generalize better to Romanian than the other way around. Joint multilingual training yields good cross-lingual performance for Roma- nian, while ambivalent categories remain the main challenge across all model families, as they depend on pragmatic intent rather than surface form. 8 Limitations PolERo covers a single non-English language. Whether the taxonomy, annotation difficulty, and model behavior observed here generalize to other languages and political contexts remains an open question. The annotation protocol assigns a single anno- tator per training instance, with triple annotation reserved for the test split. Evasion classification requires interpreting communicative intent, which can vary across annotators. Our annotators have undergraduate training in psychology rather than political science or linguistics, which may affect interpretation of borderline cases. Full triple anno- tation with domain experts would likely reduce this uncertainty, but at substantially higher cost. 9 Ethical Considerations Source data consists of official transcripts pub- lished on the public archives of the Romanian pres- idency. The speakers are public officials whose statements were issued in their official capacity; no private communications, personal data, or non- public records are included. The three annotators were each paid the equiv- alent of 80 EUR for approximately 27 hours of labeling work distributed over five weeks. The linguistic expert contributed voluntarily. Before participation, annotators were informed of the task purpose, the political nature of the content, and the option to withdraw at any point without penalty. Clarity and evasion classification of political speech can be used to selectively frame or attack in- dividual speakers. We release PolERo for research on response clarity in political discourse and dis- courage its deployment as a standalone instrument of public accountability. Model predictions are not ground truth: as the inter-annotator agreement statistics show, even trained human annotators dis- agree on a fraction of instances. Automated labels should not be treated as definitive characterizations of any individual speaker’s behavior. Finally, the computational cost associated with the experimental pipeline presented in this work, covering fine-tuning and inference experiments, was conducted on a single NVIDIA H100 80GB GPU, with a total estimated budget of approxi- mately 180 GPU hours. AI was used to check grammar and to generate boilerplate code. Acknowledgments We would like to thank the reviewers for their help- ful comments, and in particular, reviewer NvEV for their creative suggestions on how this work can be used as a political tool. This research is partially supported by the project “Romanian Hub for Artificial Intelligence - HRIA”, Smart Growth, Digitization and Finan- cial Instruments Program, 2021-2027, MySMIS no. 351416, and by InstRead: Research Instruments for Text Complexity, Simplification, and Readabil- ity Assessment CNCS - UEFISCDI project number PN-IV-P2-2.1-TE-2023-2007. References Haithem Afli, Houda Bouamor, Cristina Blasi Casagran, and Sahar Ghannay, editors. 2024. Proceedings of the Second Workshop on Natural Language Process- ing for Political Sciences @ LREC-COLING 2024. ELRA and ICCL, Torino, Italia. Parker Bach, Carolyn E Schmitt, and Shannon C Mc- Gregor. 2025. Let me be perfectly unclear: strategic ambiguity in political communication. Communica- tion Theory, 35(2):96–106. Janet Beavin Bavelas, Alex Black, Lisa Bryson, and Jennifer Mullett. 1988. Political equivocation: A situational explanation. Journal of Language and Social psychology, 7(2):137–145. Peter Bull. 1994. On identifying questions, replies, and non-replies in political interviews. Journal of language and social psychology, 13(2):115–131. Peter Bull. 2003. The Microanalysis of Political Com- munication: Claptrap and Ambiguity. Routledge, London and New York. Peter Bull and Will Strawson. 2020. Can’t answer? Won’t answer? An analysis of equivocal responses by Theresa May in Prime Minister’s Questions. Par- liamentary Affairs, 73(2):429–449. Jonathan H. Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski, Vitaly Nikolaev, and Jennimaria Palomaki. 2020a. TyDi QA: A bench- mark for information-seeking question answering in typologically diverse languages. Transactions of the Association for Computational Linguistics, 8:454– 470. Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. 2020b.Electra: Pre- training text encoders as discriminators rather than generators. Preprint, arXiv:2003.10555. Steven E Clayman. 2001. Answers and evasions. Lan- guage in society, 30(3):403–442. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettle- moyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Pro- ceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 8440– 8451, Online. Association for Computational Lin- guistics. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. George-Andrei Dima, Andrei-Marius Avram, Cristian- George Craciun, and Dumitru-Clementin Cercel. 2024. RoQLlama: A lightweight Romanian adapted language model. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 4531–4541, Miami, Florida, USA. Association for Computational Linguistics. Anca Dinu, Andreea C. Moldovan, and Adina Mar- incea. 2025. AntiSemRO: Studying the Romanian expression of antisemitism. In Proceedings of the 15th International Conference on Recent Advances in Natural Language Processing - Natural Language Processing in the Generative AI Era, pages 291–298, Varna, Bulgaria. INCOMA Ltd., Shoumen, Bulgaria. A. Seza Do ̆ gruöz, Sunayana Sitaram, and Zheng Xin Yong. 2023. Representativeness as a forgotten les- son for multilingual and code-switched data collec- tion and preparation. In Findings of the Associa- tion for Computational Linguistics: EMNLP 2023, pages 5751–5767, Singapore. Association for Com- putational Linguistics. Stefan Dumitrescu, Andrei-Marius Avram, and Sampo Pyysalo. 2020. The birth of Romanian BERT. In Findings of the Association for Computational Lin- guistics: EMNLP 2020, pages 4324–4328, Online. Association for Computational Linguistics. Stefan Daniel Dumitrescu and Andrei-Marius Avram. 2020. Introducing RONEC - the Romanian named entity corpus. In Proceedings of the Twelfth Lan- guage Resources and Evaluation Conference, pages 4436–4443, Marseille, France. European Language Resources Association. Stefan Daniel Dumitrescu, Petru Rebeja, Beata Lorincz, Mihaela Gaman, Andrei Avram, Mihai Ilie, Andrei Pruteanu, Adriana Stan, Lorena Rosia, Cristina Ia- cobescu, Luciana Morogan, George Dima, Gabriel Marchidan, Traian Rebedea, Madalina Chitez, Dani Yogatama, Sebastian Ruder, Radu Tudor Ionescu, Razvan Pascanu, and Viorica Patraucean. 2021. Liro: Benchmark and leaderboard for romanian language tasks. In Thirty-fifth Conference on Neural Informa- tion Processing Systems Datasets and Benchmarks Track (Round 1). Elisa Ferracane, Greg Durrett, Junyi Jessy Li, and Ka- trin Erk. 2021. Did they answer? subjective acts and intents in conversational discourse. In Proceedings of the 2021 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, pages 1626–1644, Online. Association for Computational Linguistics. Joseph L Fleiss. 1971. Measuring nominal scale agree- ment among many raters. Psychological bulletin, 76(5):378–382. Oana Celia Gheorghiu and Alexandru Praisler. 2022. Hate speech revisited in Romanian political dis- course: from the Legion of the Archangel Michael (1927–1941) to AUR (2020–present day). Humani- ties and social sciences communications, 9(1):235. Daniela Gîfu and Dan Cristea. 2013. Towards an au- tomated semiotic analysis of the romanian political discourse. Computer Science Journal of Moldova, 61(1):36–64. Goran Glavaš, Federico Nanni, and Simone Paolo Ponzetto. 2019. Computational analysis of political texts: Bridging research efforts across communities. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts, pages 18–23, Florence, Italy. Association for Computational Linguistics. Nuno M. Guerreiro, Ricardo Rei, Daan van Stigt, Luisa Coheur, Pierre Colombo, and André F. T. Mar- tins. 2024. xCOMET: Transparent machine trans- lation evaluation through fine-grained error detection. Transactions of the Association for Computational Linguistics, 12:979–995. Pere-Lluís Huguet Cabot, Verna Dankers, David Abadi, Agneta Fischer, and Ekaterina Shutova. 2020. The Pragmatics behind Politics: Modelling Metaphor, Framing and Emotion in Political Discourse. In Find- ings of the Association for Computational Linguistics: EMNLP 2020, pages 4479–4488, Online. Association for Computational Linguistics. Akriti Jain and Aparna Garimella. 2026. Knowing what’s missing: Assessing information sufficiency in question answering. In Findings of the Associ- ation for Computational Linguistics: EACL 2026, pages 4163–4174, Rabat, Morocco. Association for Computational Linguistics. J. Richard Landis and Gary G. Koch. 1977. The mea- surement of observer agreement for categorical data. Biometrics, 33(1):159–174. Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2020. Focal loss for dense object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(2):318–327. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretrain- ing Approach. Preprint, arXiv:1907.11692. William C. Mann. 1984. Discourse structures for text generation. In 10th International Conference on Computational Linguistics and 22nd Annual Meet- ing of the Association for Computational Linguistics, pages 367–375, Stanford, California, USA. Associa- tion for Computational Linguistics. Mihai Masala, Denis C. Ilie-Ablachim, Dragos Cor- latescu, Miruna Zavelca, Marius Leordeanu, Ho- ria Velicu, Marius Popescu, Mihai Dascalu, and Traian Rebedea. 2024.Openllm-ro – technical report on open-source romanian llms.Preprint, arXiv:2405.07703. Mihai Masala, Stefan Ruseti, and Mihai Dascalu. 2020. RoBERT – a Romanian BERT model. In Proceed- ings of the 28th International Conference on Com- putational Linguistics, pages 6626–6637, Barcelona, Spain (Online). International Committee on Compu- tational Linguistics. Sewon Min, Julian Michael, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2020. AmbigQA: Answering am- biguous open-domain questions. In Proceedings of the 2020 Conference on Empirical Methods in Nat- ural Language Processing (EMNLP), pages 5783– 5797, Online. Association for Computational Lin- guistics. Brighton Msagalla and Marianna Visser. 2021. Agenda- setting through topic shift in Tanzanian parliamentary debate: The derailment of strategic manoeuvring. Southern African Linguistics and Applied Language Studies, 39(4):327–337. NLLB Team, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Hef- fernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, and 20 others. 2022. No language left behind: Scal- ing human-centered machine translation. Preprint, arXiv:2207.04672. Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. Know what you don’t know: Unanswerable ques- tions for SQuAD. In Proceedings of the 56th Annual Meeting of the Association for Computational Lin- guistics (Volume 2: Short Papers), pages 784–789, Melbourne, Australia. Association for Computational Linguistics. Parameswary Rasiah. 2010. A framework for the sys- tematic analysis of evasion in parliamentary dis- course. Journal of Pragmatics, 42(3):664–680. Ines Reinig, Ines Rehbein, and Simone Paolo Ponzetto. 2024. How to do politics with words: Investigat- ing speech acts in parliamentary debates. In Pro- ceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 8287– 8300, Torino, Italia. ELRA and ICCL. Ana-Cristina Rogoz, Radu Tudor Ionescu, Alexandra- Valentina Anghel, Ionut-Lucian Antone-Iordache, Si- mona Coniac, and Andreea Iuliana Ionescu. 2026. A large-scale benchmark for evaluating large language models on medical question answering in romanian. Preprint, arXiv:2508.16390. Tanya Romaniuk. 2013. Pursuing answers to questions in broadcast journalism. Research on Language & Social Interaction, 46(2):144–164. Gabriel Stefan and Sergiu Nisioi. 2026. SG-UniBuc- NLP at SemEval-2026 task 6: Multi-head RoBERTa with chunking for long-context evasion detection. In Proceedings of the 20th International Workshop on Semantic Evaluation (2026), pages 964–972, San Diego, California, USA. Association for Computa- tional Linguistics. Shivashankar Subramanian, Trevor Cohn, and Timothy Baldwin. 2019. Target based speech act classifica- tion in political campaign text. In Proceedings of the Eighth Joint Conference on Lexical and Com- putational Semantics (*SEM 2019), pages 273–282, Minneapolis, Minnesota. Association for Computa- tional Linguistics. Konstantinos Thomas, Giorgos Filandrianos, Maria Lymperaiou, Chrysoula Zerva, and Giorgos Stamou. 2024. “I Never Said That”: A dataset, taxonomy and baselines on response clarity classification. In Findings of the Association for Computational Lin- guistics: EMNLP 2024, pages 5204–5233, Miami, Florida, USA. Association for Computational Lin- guistics. Konstantinos Thomas, Giorgos Filandrianos, Maria Lymperaiou, Chrysoula Zerva, and Giorgos Stamou. 2026. SemEval-2026 task 6: CLARITY – unmask- ing political question evasions. In Proceedings of the 20th International Workshop on Semantic Evalua- tion (2026), pages 3704–3715, San Diego, California, USA. Association for Computational Linguistics. Andra Vasilescu, Mihaela-Viorica Constantinescu, Ari- adnaS , tef ̆ anescu, andS , erban Hartular. 2024. Insights into Romanian political discourse. Cambridge Schol- ars Publishing. Bhuvanesh Verma, Mounika Marreddy, and Alexander Mehler. 2026. Predicting convincingness in politi- cal speech: How emotional tone shapes persuasive strength. In The Proceedings for the 15th Workshop on Computational Approaches to Subjectivity, Sen- timent Social Media Analysis (WASSA 2026), pages 37–51, Rabat, Morocco. Association for Computa- tional Linguistics. Benjamin Warner, Antoine Chaffin, Benjamin Clavié, Orion Weller, Oskar Hallström, Said Taghadouini, Alexis Gallagher, Raja Biswas, Faisal Ladhak, Tom Aarsen, Griffin Thomas Adams, Jeremy Howard, and Iacopo Poli. 2025. Smarter, better, faster, longer: A modern bidirectional encoder for fast, memory efficient, and long context finetuning and inference. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2526–2547, Vienna, Austria. Association for Computational Linguistics. Michal Zawalski, Meriem Boubdir, Klaudia Bał azy, Be- smira Nushi, and Pablo Ribalta. 2026. Detecting data contamination in llms via in-context learning. In In- ternational Conference on Learning Representations, volume 2026, pages 1337–1371. A Taxonomy and Annotation Guidelines A.1 Romanian Annotation Guidelines Label names. We retained the nine original En- glish label names (Explicit, Implicit, General, Partial/half-answer, Dodging, Deflection, Declin- ing to answer, Claims ignorance, Clarification) to ensure our dataset remains directly comparable with the English corpus. Descriptions and examples. We translated all category descriptions into Romanian and replaced the English examples of Thomas et al. (2024) with Romanian examples drawn from political discourse data in our dataset. Each example pairs an answer (R) with a question (Î) and a short rationale identi- fying the features that trigger the label. A Romanian linguistic expert assisted with exam- ple selection to ensure that each category is illus- trated by a clear and representative instance. Ta- ble 10 reproduces the guidelines, with English de- scriptions and bilingual examples (Romanian as supplied to annotators; English translation in italics for readability). A.2 Summaries and Quality Control Annotators were required to classify the evasion strategy based on the original text. In addition, the annotators received GPT-5.4-generated summaries and were asked to verify the validity of the sum- maries. To ensure that the annotators were reading the source text, we inserted 60 intentionally incor- rect summaries. The annotators correctly flagged 58of these (96.67%), producing only two false neg- atives. None of the other summaries were flagged as inconsistent. B Annotation Protocol B.1 Labeling Platform We used Labelbox 5 to manage the annotation work- flow. Each item displayed the original interview question and answer verbatim, with the GPT-5.4- generated question and answer summaries attached as side panels for reference (Figure 4). The anno- tator completed two tasks per item: (i) assign one of the nine evasion labels from the Level-2 taxon- omy, and (i) verify whether the answer summary correctly reflects the original. The platform provided the complete annotation guidelines, which included the taxonomy table (Ta- ble 10) and an extended set of examples for border- case examples covering categories with closely re- lated semantics (e.g. Dodging vs. Deflection, Gen- eral vs. Partial/half-answer). Annotators could consult the guidelines at any point during a label- ing session. Annotators could additionally tag any instance with one of five issue categories: •Parsing Error — the QA pair was malformed during extraction. •Not a Question — the journalist’s statement does not solicit any information. •Multiple Questions — a single journalist’s question contains several sub-questions. •Insufficient Context — the response cannot be accurately classified without additional back- ground information. •Borderline / Hard to Decide — the response is difficult to classify and may fall between two evasion categories. Annotators could also create custom issue cate- gories to report any additional problems not cov- ered by the predefined taxonomy. Items tagged with the first three issues were filtered from the released corpus. Items tagged Insufficient 5 https://labelbox.com/ Context or Borderline / Hard to Decide were esca- lated to the expert, who either adjudicated a final label or removed the instance. Figure 4: Labelbox annotation interface. Left panel: evasion classification and summary validity check. Cen- ter: original question and answer. Right panel: GPT- 5.4-generated question and answer summaries as attach- ments. B.2 Inter-Annotator Agreement Raw annotator agreement on the test set is high, especially for clarity. All three annotators agree on 87.2% of clarity items, while the remaining 12.8% receive a two-thirds majority, leaving no items with- out consensus. Agreement on evasion is slightly lower but still substantial: 69.6% of items receive full three-way agreement, 28.0% receive a two- thirds majority, and only 2.4% result in complete disagreement between all three annotators. Figure 5 shows pairwise Fleiss’κacross the three clarity labels. All pairwise values are at least 0.85, with the lowest agreement at the Clear Re- ply-Ambivalent boundary (κ = 0.85). Disagree- ment is mainly found between Clear Reply and Ambivalent answers rather than between the two non-ambiguous labels (Clear Reply vs. Clear Non- Reply: κ = 0.98). Figure 5: Pairwise Fleiss’κbetween clarity categories. Figure 6 shows the same matrix for the nine eva- sion categories. The lowest pairwiseκvalues occur for Deflection, in particular Deflection-Dodging (κ = 0.26) and Deflection-General (κ = 0.38). These confusions reflect how close the categories are in the taxonomy. Dodging and Deflection both involve leaving the question unanswered. They dif- fer in whether the answer acknowledges the ques- tion before shifting away. General and Deflection differ in whether the answer remains on topic at all, which depends on how broadly the topic is interpreted. Figure 6: Pairwise Fleiss’κbetween evasion categories. Figure 7 shows the per-category one-vs-rest Fleiss’κ. Six categories cross the substantial- agreement threshold ofκ = 0.6(Clarification, Claims ignorance, Explicit, Declining to answer, Partial/half-answer, Dodging); two fall between fair and substantial (Implicit, General); one falls in the fair range (Deflection). Figure 7: One-vs-rest Fleiss’ κ per evasion category. B.3 Expert Benchmark Evaluation The linguistic expert independently labeled a benchmark of 85 instances drawn from the test split. Figure 8 reports per-annotator accuracy against the expert: 95.3%, 95.3%, 94.1% on clarity and 89.4%, 88.2%, 84.7% on evasion. Clarity accuracy is 5–10 percentage points higher than evasion accuracy for each annotator, which is consistent with the greater difficulty of the nine-class task compared to the three-class task. Figure 8: Per-annotator clarity and evasion accuracy against expert benchmark. C Exploratory Data Analysis This appendix reports descriptive statistics over the 3,574question-answer pairs in the dataset (3,278 train;296test). The statistics are computed over the full dataset and use the majority-vote labels on the test set for both clarity and evasion categories. C.1 Label Distribution Figures 9 and 10 show the label distributions for clarity and evasion. Explicit answers make up the majority of the dataset (55.09%). The other eight evasion strategies are much less common, with Clarification occurring in only 1.09% of the question-answer pairs. Figure 9: Overall label distribution for Level 1 (Clarity). Figure 10: Overall label distribution for Level 2 (Eva- sion). CLARITY splits multi-part interviewer turns into one labeled row per sub-question using Chat- GPT (Thomas et al., 2024), whereas PolERo re- moves them entirely. In CLARITY,67.4%of in- stances are such fragments, and they are labeled Ambivalent far more often than single-question in- stances (63.7%vs.51.7%). When limiting CLAR- ITY to examples with a single question (N = 1,225): Ambivalent falls from59.8%to51.7% and Clear Reply rises from30.1%to36.1%(Fig- ure 11). The difference between the two distribu- tions shrinks minimally. Figure 11: Clarity distribution of the English CLARITY corpus before and after restricting it to single-question instances, compared with PolERo. C.2 Distribution by Speaker The corpus covers five Romanian presidents (Fig- ure 12). Coverage reflects data availability: B ̆ as- escu (1,214) and Iohannis (1,100) account for 64.8% of the corpus; Bolojan (56) is much less frequent, as he has been an interim president for 103 days. Figure 13 shows the clarity rates for each speaker. Iliescu and B ̆ asescu give the most direct answers, with Clear Reply rates of 62.06% and 62.60%. In contrast, Bolojan gives the most Ambivalent an- swers (53.57%), and Iohannis has the highest rate of Clear Non-Replies (11.55%). Figure 12: Total number of question-answer pairs per speaker. Figure 13: Clarity label distribution per speaker. Evasion distributions per speaker are shown in Figure 14. Dodging is most frequent for Iliescu (11.87%) and B ̆ asescu (9.06%). Deflection is most common for Bolojan (12.50%) and Dan (10.08%). Declining to answer is highest for Iohannis (9.09%) and Bolojan (8.93%). The post-2014 speakers (Io- hannis, Bolojan, and Dan) show a more balanced distribution of evasion types, with higher propor- tions of General, Deflection, and Declining to an- swer responses. In contrast, Iliescu and B ̆ asescu rely more heavily on Dodging, while still remain- ing largely dominated by Explicit answers. C.3 Distribution by Communicative Context We assigned a context label to each source tran- script from event titles using keywords associated with each category. Interviews make up the major- ity of the dataset (1,904 pairs), followed by dec- larations (1,213 pairs) and press conferences (457 pairs). Figure 15 reports clarity rates across the three resulting contexts. Interviews have the highest rate of Clear Replies (63.03%) and the lowest rate of Figure 14: Evasion label distribution per speaker. Clear Non-Replies (3.62%). Declarations and press conferences show very similar clarity distributions. Figure 15: Clarity distribution per communicative con- text. Part of the higher clarity rate in interviews is linked to differences in topic distribution. Per- sonal & Others makes up 38.39% of interview questions, compared to only 12.94% in declara- tions and 8.75% in press conferences (see Section C.4). This category also has the highest Clear Re- ply rate of any topic (65.62%). Press conferences contain more Justice & Anti-corruption and Domes- tic Policy questions, which have the lowest Clear Reply rates. Evasion distributions per context are shown in Figure 16. Interviews show a higher percentage of Dodging (10.45%), but low levels of the other evasion types. Declarations and press conferences have roughly four to six times the rate of Declining to answer (8.41% and 6.13%) and higher Deflec- tion (7.50% and 10.72%) compared to interviews (1.42% and 3.31%). Press conferences have the highest Deflection rate overall. If we exclude inter- views and consider only the more formal settings of declarations and press conferences (n=1,670), the overall proportion of direct answers drops sig- nificantly: the Explicit rate falls to 46.05%, while evasive categories such as Deflection (8.38%) and Declining to answer (7.79%) account for a larger share of responses. Figure 16: Evasion label distribution per communicative context. Figure 17 crosses speaker with context. Ili- escu and B ̆ asescu interviews (n=754,n=1,150) explain the high interview Explicit rate (62.47%, 63.39%) and also account for most Dodging (12.47%, 9.13%) in interviews. Iohannis decla- rations (n=852) account for the bulk of Declining to answer in the declaration row (9.74%). B ̆ as- escu declarations show the highest General share (23.80%) and Bolojan press conferences (n=36) the highest combined non-Explicit share (72.22%). Figure 17: Evasion distribution per speaker×context cell. C.4 Distribution by Topic Category Categories were defined by first inspecting the ques- tion content and identifying nine main topics. GPT- 5.4 then assigned a single category label to each question. Figure 18 shows the overall distribution of these topics. Figure 19 shows the clarity rates for each topic. Figure 18: Total number of question-answer pairs per topic category. Even though Justice & Anti-corruption questions make up only 7.81% of the data, they trigger the highest rate of Clear Non-Replies (15.05%). Figure 19: Clarity label distribution per topic category. Per-category evasion distributions are shown in Figure 20. Personal & Others has the highest Dodg- ing share (11.21%), although most responses in this category are still Explicit. Justice & Anti- corruption has the highest rates of Declining to answer (9.32%) and Claims ignorance (5.02%). Education & Research shows the highest share of Implicit (11.76%), followed by Society & Social Issues (11.40%). General responses are most fre- quent for Society & Social Issues (18.42%) and Public Health (18.10%). C.5 Temporal Coverage QA pairs span2001–2026in two main periods: 2001–2007(Iliescu and B ̆ asescu,1,935pairs) and 2015–2026(Iohannis, Bolojan, and Dan,1,543 pairs). The intervening2008–2014window con- tains96pairs, due to limited archival availability. Per-year volumes and per-speaker timelines are shown in Figures 21 and 22. Year-level clarity composition is shown in Fig- Figure 20: Evasion label distribution per topic category. Figure 21: Yearly volume of QA pairs, 2001–2026. ure 23. Clear Non-Reply rates remain below 6% in 2001–2007, and increase to around 10–20% in 2016–2026, with Ambivalent responses increasing in parallel. C.6 Length and Verbosity Mean question length is30.7words, while the median is21words; mean answer length is76.9 words, while the median is45words. Overall length distributions are shown in Figure 24. Answer length differs substantially by label (Fig- ures 25 and 26). Clear Replies average 88.4 words, Ambivalent answers 67.4, and Clear Non-Replies 39.7. Looking at the specific evasion strategies, Clari- fication answers are the shortest (mean6.4, median 4) followed by Dodging (mean27.6, median13). In contrast, General, Deflection, and Partial/half- answer responses are the longest non-Explicit cate- gories (averaging89.7,88.6, and87.3words). This aligns with verbose evasion tactics, where politi- cians surround the question with extra information instead of answering it directly. Per-speaker mean answer length ranges from 52.9words (Dan) to110.6(Bolojan,n=56), with Iohannis at73.1, B ̆ asescu at69.3and Iliescu at 102.6. The full per-speaker answer-length distri- Figure 22: Yearly QA volume per speaker. Figure 23: Clarity label composition per year, 2001– 2026. bution is shown in Figure 27. Speaker verbosity does not align with evasion: Iliescu is among the most verbose speakers and also one of the most Explicit, while Dan is the most concise and is also predominantly Explicit. D Experiment Details This appendix expands the experimental setup and analysis summarised in Sections 3 and 4. D.1 Evaluation Across all experiments, we evaluate model perfor- mance using macro-averaged F1 as the primary metric. The macro-F1 is calculated as the un- weighted mean of the class-specific F1 scores: F1 macro = 1 |C| |C| X i=1 F1 i (1) where|C|is the total number of classes (3 for clar- ity, 9 for evasion), andF1 i is the harmonic mean of precision (P i ) and recall (R i ) for class i: F1 i = 2· P i · R i P i + R i (2) To compute precision and recall, model predic- tions are compared against human annotations. For the 3-class clarity task, predictions are eval- uated strictly against a single ground-truth label Figure 24: Question and answer word-count distribu- tions. Figure 25: Answer word-count distribution per clarity label. obtained via majority vote. A prediction is consid- ered correct only if it exactly matches this label. For the 9-class evasion task, a prediction is con- sidered correct if it matches at least one annota- tor’s label, which reflects valid differences in hu- man interpretation. If it matches none, we use the majority-vote label as the ground truth for an incor- rect prediction. D.2 Methodology TF-IDF baseline. We extract word unigram TF- IDF features and tunemin _df ∈ 1, 2, 3and max _df ∈ 0.9, 0.95, 1.0.We use an L2- regularized logistic regression classifier with class- balanced weights. We perform a grid search over C ∈ 0.25, 0.5, 1, 2, 3, 5, 10and select the best model based on development set macro-F1. For CLARITY, the best configuration is min _df=2,max _df=0.95,C=0.5for clarity, andmin _df=2,max _df=0.9,C=2for evasion. For PolERo, the best parameters aremin _df=3, max _df=0.9,C=5for clarity andmin _df=1, max _df=0.9, C=2 for evasion. Encoder Fine-Tuning Input formatting. Encoder models use the input format “Question: q : a”. On CLARITY,qis the GPT-decomposed atomicquestionfieldpairedwiththe interview_answer.On PolERo,qis the interview_questionfield paired with the same interview_answer, since the Romanian dataset Figure 26: Answer word-count distribution per evasion label. Figure 27: Answer word-count distribution per speaker. already contains single-question inputs. Single-head encoders. Each of the eight single- head encoder configurations is fine-tuned sepa- rately for each task for up to 20 epochs using AdamW, with a batch size of 64. The maximum sequence length is set to 512 for all models ex- cept ModernBERT-large, which uses a maximum sequence length of 8192. The best checkpoint is selected on an85/15train/validation split of the official training set and evaluated once on the test set. Learning rates vary by architecture: 5× 10 −6 for ModernBERT-large and ELECTRA- large;1× 10 −5 for RoBERTa-large and XLM- RoBERTa-large; and2× 10 −5 for BERT-large- cased, mBERT-base, RoBERT-large, and Roma- nian BERT. Multi-head architecture. We apply this archi- tecture to three encoder configurations: RoBERTa- large, XLM-RoBERTa-large, and RoBERT-large. All multi-head models use 7-fold cross-validation stratified by the clarity label. Models are trained for 20 epochs using AdamW with a learning rate of 1×10 −5 and a batch size of 8. We use early stop- ping based on the combined validation macro-F1 of both heads, with a patience of 5. At inference time, we ensemble all 7 folds by averaging their predicted class probabilities and selecting the most probable class for each classification head. Large Language Models Open-source LLMs were run locally, while propri- etary models were accessed via the OpenAI API platform 6 . All models are queried with default settings and reasoning configurations, using a re- stricted output schema. Zero-shot prompts include the task description, the full set of category names, and one-line definitions. Few-shot prompts add one labeled example per category. Prompts are run separately for the clarity and evasion tasks and use the dataset’s language. E Multi-Head Architecture Details This appendix expands the multi-head architecture introduced in Section 3 and isolates the contribu- tion of each component on the English dataset. All ablation numbers are 7-fold cross-validation means and standard deviations of validation macro-F1, computed on the training split with folds stratified by the clarity label. Test-set scores for this configu- ration are reported in Table 11. E.1 Hierarchical Input Processing Responses in both datasets are often longer than the 512-token limit of standard pretrained encoders. Because evasion cues can appear anywhere in a re- sponse, simple truncation may remove the relevant span. To avoid this, we segment each tokenized question-answer sequenceTinto overlapping win- dows of length L=512 with stride S=256: C k = T[kS : kS + L] for k = 0,...,M − 1, (3) whereM =⌈max(|T|− L, 0)/S⌉ + 1. The final chunk extends to|T| and is zero-padded if shorter. The window sizeLis fixed by the encoder’s posi- tional embedding capacity. The 50% overlap en- sures that every interior token is included in at least two consecutive windows, reducing the chance that important evasion cues are separated across chunks. Each chunk is encoded independently by the shared encoder. We extract the hidden state at po- sition 0 of each chunk,h k = H k [0, :] ∈R d , and aggregate theMchunk vectors via element-wise max-pooling: v j = M−1 max k=0 h k,j for j = 1,...,d.(4) 6 https://platform.openai.com The position-0 token corresponds to the encoder’s start-of-sequence token only for the first chunk; for subsequent chunks, it is the first content token of the window. We use this position uniformly because the encoder distributes contextual informa- tion across all tokens. The pooled vectorvis passed through dropout (p=0.1) and fed into two linear classification heads: ˆy c = softmax(W c · Dropout(v) + b c ),(5) ˆy e = softmax(W e · Dropout(v) + b e ),(6) withW c ∈R 3×d andW e ∈R 9×d . The training ob- jective is the unweighted sum of two cross-entropy losses,L = L c + L e . At inference time, the 7 models are ensembled by averaging predicted class probabilities and taking the argmax per head. E.2 Component Ablations The ablations below isolate the impact of each ar- chitectural choice over single-head encoder fine- tuning. Pooling strategy. Table 4 compares element- wise max-pooling with mean-pooling and a base- line that keeps only the first chunk representation. Max-pooling performs best. Mean-pooling per- forms worse than max-pooling, indicating that sim- ple averaging is less effective for aggregating chunk representations. The first-chunk baseline misses any signal beyond the first 512 tokens, which af- fects 28.8% of English responses. PoolingClarityEvasion First chunk only0.67± 0.010.42± 0.01 Mean-pooling0.68± 0.020.43± 0.02 Max-pooling 0.70± 0.02 0.45± 0.02 Table 4: Chunk aggregation strategy. 7-fold CV valida- tion macro-F1 (mean± std). Multi-task versus single-task. Table 5 shows that joint training matches single-task performance on clarity and improves single-task evasion by +0.03macro-F1. The coarser clarity supervision acts as a regularizer for rare evasion categories: each evasion label maps deterministically to a clar- ity label, so gradients fromL c help stabilize repre- sentations of the parent class, even when the eva- sion head provides limited signal. ObjectiveClarityEvasion Single-task (clarity) 0.70± 0.02– Single-task (evasion)–0.42± 0.01 Multi-task0.70± 0.02 0.45± 0.02 Table 5: Effect of joint training. 7-fold CV validation macro-F1 (mean± std). Ensemble size. Table 6 reports performance for k ∈3, 5, 7folds. Increasingkreduces variance and increases the amount of training data per fold model. Improvements are consistent across both subtasks and larger for the more difficult evasion task. FoldsClarityEvasion 30.66± 0.010.42± 0.02 50.68± 0.020.43± 0.03 7 0.70± 0.02 0.45± 0.02 Table 6: Effect of ensemble size.k-fold CV validation macro-F1 (mean± std). Loss function. Table 7 compares unweighted cross-entropywithinverse-frequencyclass- weighted cross-entropy and focal loss (γ=2) (Lin et al., 2020). Neither alternative improves macro-F1. Class weighting increases minority- class recall (Clear Non-Reply from 0.62 to 0.67, Partial/half-answer from 0.00 to 0.10), but this is offset by lower precision on majority classes. Focal loss yields intermediate performance between the two.Most evasion errors occur between pragmatically similar classes (Implicit, Deflection, General, Dodging) that share surface features. Reweighting increases the training signal for rare classes but does not reduce overlap between similar evasion categories. LossClarityEvasion Cross-entropy (unweighted) 0.70± 0.02 0.45± 0.02 Class-weighted CE0.66± 0.02 0.41± 0.02 Focal (γ=2)0.69± 0.02 0.44± 0.02 Table 7: Effect of loss function. 7-fold CV validation macro-F1 (mean± std). E.3 Seed Variation We repeat the full multi-head pipeline for both monolingual configurations under three random seeds (7,42,123). The seed controls the cross- validation split, the per-fold training seed, and weight initialization. Table 8 reports mean and standard deviation of macro-F1. Cross-validation scores are stable: combined macro-F1 varies by±.002on English and±.008 on Romanian, as each figure averages over seven folds. Test-set scores vary more, as they come from a single evaluation: clarity is.676 ±.019 on English and.743 ±.023 on Romanian, evasion.535 ±.024 and .558 ±.037 . This variation is smaller than the dif- ferences we report between model families, so the effects cannot be attributed to seed choice. MetricEnglishRomanian 7-fold cross-validation (validation folds) Clarity.700 ±.001 .676 ±.006 Evasion.450 ±.004 .388 ±.011 Combined.575 ±.002 .532 ±.008 Test set (7-fold ensemble) Clarity.676 ±.019 .743 ±.023 Evasion (any-of).535 ±.024 .558 ±.037 Mapped-clarity.682 ±.021 .739 ±.027 Table 8: Seed variation of the multi-head architecture: macro-F1 mean±standard deviation over three seeds (7,42,123), with RoBERTa-large as the English back- bone and RoBERT-large as the Romanian one. F Extended Model Comparison and Error Analysis Table 11 presents the complete test-set macro-F1 re- sults for all baselines, architectural variants, cross- lingual transfer setups, and LLM prompting strate- gies discussed in this work. F.1 Traditional classifier The TF-IDF logistic regression baseline achieves similar macro-F1 on both languages (0.42on En- glish,0.45on Romanian), but its error patterns show which parts of the task rely on lexical cues. The model correctly identifies most Clear Reply instances in Romanian (64% recall), but performs much worse in English (33% recall). In contrast, the Ambivalent class is often con- fused with the other two labels. In Romanian, this misclassification goes both ways: 48% of true Ambivalent instances are predicted as Clear Re- ply, while 33% of true Clear Reply instances are predicted as Ambivalent. This suggests that any answer that reuses question vocabulary is often treated as a reply by the baseline, even when it fails to provide a concrete answer. Clear Non-Reply performs worst, with an F1 of 0.25 on English and 0.26 on Romanian (and recall as low as 24%). As reported in Appendix C.6, these are typically very short non-replies (e.g., Clarifica- tion, Declining to answer). Because this classifier relies on term frequency, these brief answers gen- erate weak feature signals, causing the model to underpredict these minority classes. The slightly higher performance on Romanian is consistent with the higher share of Explicit replies in its training distribution (Section C.1). More cases can be solved through lexical overlap. F.2 Encoder Models Best single-head models. Among single-head encoders, RoBERTa-large is the strongest English model on evasion. RoBERT-large is the strongest model on PolERo. It is pretrained on Romanian text and uses a tok- enizer trained on Romanian data, unlike the shared multilingual vocabulary used by XLM-RoBERTa. Although XLM-RoBERTa-large is the strongest multilingual model among those evaluated and is competitive with monolingual encoders on Roma- nian clarity, it underperforms on the fine-grained evasion task. Failuremodes. ELECTRA-largeand ModernBERT-large are the two main failure cases. ELECTRA uses a replaced-token-detection (RTD) objective, which trains the model at the token level to distinguish real from replaced tokens. On small and imbalanced datasets like those tested, we hypothesize that RTD may make the model more susceptible to overfitting on frequent lexical cues rather than learning broader evasion features. As a result, on the Romanian dataset, the model assigns all inputs to the Explicit class. ModernBERT-large shows the same issue, though less severely. Its local-global attention mechanism improves efficiency for longer input sequences compared to standard BERT encoders, which may offer limited benefit for short question- answer inputs. In our experiments, three rare cat- egories (Claims ignorance, Partial/half-answer, Deflection) receive zero recall on Romanian, and the mapped clarity predictions inherit the same majority-class bias. Multi-head architecture. Multi-head models consistently outperform single-head models across all reported metrics. The strongest performance is observed for RoBERTa-large (English) and RoBERT-large (Romanian), both of which are pre- trained on in-language corpora and appear to ben- efit most from the joint signal in our experiments. XLM-RoBERTa-large shows larger relative gains, but still underperforms monolingual encoders, sug- gesting that multilingual representations may not fully capture fine-grained signals within a given language. Figure 28: Out-of-fold (OOF) confusion matrix for the 9-class evasion task on CLARITY (EN) using multi- head RoBERTa-large. Figure 29: Out-of-fold (OOF) confusion matrix for the 9-class evasion task on PolERo (RO) using multi-head RoBERT-large. Error Analysis. While the boundary between Clear Reply and Ambivalent remains the largest source of errors, the out-of-fold confusion matri- ces for our strongest models (Figure 28, Figure 29) show that the fine-grained errors between evasion strategies concentrate along two main semantic axes. The most prominent axis of confusion is Implicit– Explicit. Both represent valid, on-topic answers. The difference depends on whether the requested information is stated directly or must be inferred from context. In English, this axis accounts for285 errors, and it remains a major source of error in Romanian with 215 misclassifications. The second major axis is General-Explicit. Both classes provide relevant information in response to the requested information. The distinction depends on whether the response directly addresses the spe- cific details of the question or remains more general. This axis accounts for146errors in English, but increases substantially in Romanian, becoming the largest source of error with 315 misclassifications. Another distinct source of errors is Dodging- Deflection. Both involve topic shifts. The differ- ence depends on whether the model first attempts to answer the question before shifting topic or shifts topic without addressing the question. In English, this axis accounts for157errors, although decreas- ing to 22 errors in Romanian. Partial/half-answer is the most difficult category for all encoders, with an F1 of0.066on English and0.024on Romanian. Correctly identifying it requires determining whether only part of a multi- part question is answered while another part is ig- nored, a relational judgment that standard encoder representations do not capture well. Error structure.We inspect where the errors of the strongest fine-tuned encoders happen relative to annotator agreement on the test split. For multi- head RoBERTa-large on the English evasion task, 43.7%of errors occur on items with full3/3an- notator agreement and50.4%on items with2/3 agreement, while only5.9%involve perfect dis- agreement. The pattern is more evident on multi- head RoBERT-large for Romanian, where54.7% of evasion errors and56.6%of clarity errors fall on items with unanimous labels. Errors occur on items with the cleanest annotation signal rather than on the genuinely ambiguous ones, largely because unanimous and high-agreement items make up the vast majority of the dataset. The higher apparent accuracy on the perfect-disagreement subset (.758 on English evasion vs..528on3/3items) is an expected result of the evasion evaluation rule (Sec- tion D.1), which marks a prediction as correct if it matches any of the three distinct annotator labels. Calibration.The difference between top-1 prob- ability on correct and incorrect predictions is nar- Single-headMulti-head∆ (MH− SH) ConfigurationCEMCEMCEM EN→ RO.506.282.512.635.399.655+.129+.117+.143 EN2RO→ RO.565.314.616.665.412.706+.100+.098+.090 EN + RO→ RO.695.403.686.723.570.717+.028+.167+.031 EN→ EN2RO.535.334.574.616.448.612+.081+.114+.038 EN + RO→ EN.628.459.617.679.483.642+.051+.024+.025 EN + RO→ EN + RO.676.508.688.737.529.705+.061+.021+.017 Mean gain+.075+.090+.057 Table 9: Single-head vs. multi-head XLM-RoBERTa-large across cross-lingual configurations. C: clarity, E: evasion, M: mapped clarity; test-set macro-F1. row across both tasks. The English RoBERTa-large multi-head averages.605on correct evasion predic- tions and.483on incorrect ones. The clarity head averages.835vs..726. The Romanian RoBERT- large multi-head is more confident in both cases (.923vs..822for clarity;.859vs..637for eva- sion), consistent with the more imbalanced Roma- nian label distribution. Confident errors occur in both languages: the Romanian clarity head assigns probability1.00to Clear Reply on single-word re- sponses unanimously labeled Ambivalent. The En- glish evasion head assigns probability above.85 to Explicit on3/3-agreement Implicit and Deflec- tion responses. Top-1 probability is therefore not a reliable rejection signal at deployment. Single-head vs. multi-head.Table 9 isolates the architectural contribution from the training-data configuration by running each cross-lingual set- ting with both a standard single-head encoder and the multi-head architecture, using XLM-RoBERTa- large as a common backbone with identical data, in- put format and evaluation protocol. The multi-head architecture outperforms the single-head baseline in every configuration and on every metric, by a mean of+.075macro-F1 on clarity,+.090on eva- sion and+.057on mapped clarity. The gains are therefore attributable to the architecture rather than to the training-data configuration. The advantage is largest on the fine-grained evasion task, consistent with the joint objective stabilising rare categories. It is also largest where training data is scarcest: zero-shot transfer (EN→RO) gains+.129on clar- ity, whereas the joint bilingual settings gain+.028 and +.051. F.3 LLM Prompting Analysis The complete LLM results in Table 11 show that GPT-5.4 achieves the best mapped scores on both languages and the strongest English evasion perfor- mance. Qwen3.6-35B-A3B performs best on Ro- manian evasion under few-shot prompting, while Gemma-4-31B-it is best on English direct clarity. For both languages, the performance difference between the strongest LLMs and encoder ensem- bles is larger on the evasion task than on the clarity task. The clarity task mainly requires checking whether the answer matches the question, which strong encoders handle well. In contrast, the nine- class evasion task requires distinguishing between closely related rhetorical strategies and benefits from broader discourse and pragmatic knowledge. Multi-head fine-tuning reduces this difference, but does not eliminate it. Performance on the am- bivalent evasion categories remains the main chal- lenge for both model families. F.4 Machine Translation Translate-Train recovers part of the mapped clarity gap but yields only small improvements on the evasion task. Machine translation preserves enough propositional content to recover the coarse three- class distinction, but distorts pragmatic cues that separate ambivalent categories. Deflection and Dodging are most affected, as both rely more on discourse structure than lexical cues. As a result, both show substantial recall drops under translation compared to in-language single- head models. On the Romanian test set, recall for Deflection decreases from 41% to 10%, while Dodging drops from 50% to 30%. F.5 Machine Translation Quality We evaluate the quality of the NLLB-200-distilled- 600M translations used in the cross-lingual experi- ments with XCOMET-XL, a model that estimates translation quality without reference translations. Results are reported for both translation directions on the training and test splits. For RO→EN, translating the Romanian data re- sulted in mean QE scores of0.650(train,n=3,278) and0.655(test,n=296), with clear variation across categories. Formulaic categories translate well, including Clarification (0.781train,0.941test) and Declining to answer (0.763train,0.763test), which obtain the highest scores. Categories driven by pragmatics score lower. Deflection has the low- est scores in both splits (0.582train,0.473test). This drop matches the recall degradation for De- flection and Dodging under MT augmentation. For EN→RO, translating the English data re- sulted in mean QE scores of0.218(train,n=3,448) and0.171(test,n=308). Although these scores are notably low, a manual inspection of the transla- tions shows that many spans flagged as errors are acceptable translations. Despite this noise, the rela- tive ranking across categories is consistent with the RO→EN direction: Deflection and General score lowest, while Clarification and Claims ignorance score highest. Clarity (L1)Evasion (L2)DescriptionExample (Romanian; English in italics) Clear ReplyExplicitThe requested information is given directly, in the form re- quested, without ambiguity or required inference. Î: Când va fi gata autostrada? R: Lucr ̆ arile vor fi finalizate în luna martie 2025. Q: When will the highway be ready? A: Works will be completed in March 2025. Ambivalent Implicit The requested information is conveyed but not stated in the expected form; the listener must infer the conclusion. Î: V ̆ a vet , i da demisia din funct , ia de ministru? R: Nu mai pot r ̆ amâne o secund ̆ a în acest guvern. Q: Will you resign as minister? A: I cannot stay in this government a second longer. GeneralThe response stays on topic but remains too broad, abstract, or vague to deliver the requested specifics. Î: În ce lun ̆ a va fi gata autostrada? R: Dezvoltarea infrastructurii e o prioritate zero pen- tru noi s , i muncim zi de zi pe s , antiere. Q: In which month will the highway be ready? A: Infrastructure development is our top priority and we work every day on site. Partial/half- answer Covers concretely only one com- ponent of the requested informa- tion and omits the rest. Î: Ce companii de stat vor fi privatizate anul acesta? R: V ̆ a pot spune cu certitudine c ̆ a Pos , ta Român ̆ a se afl ̆ a pe aceast ̆ a list ̆ a. Q: Which state-owned companies will be privatised this year? A: I can tell you with certainty that the Romanian Postal Service is on the list. DodgingDoes not engage with the sub- stance of the question; the topic is neither acknowledged nor touched. Î: De ce at , i pierdut alegerile locale? R: Presa ar trebui s ̆ a se ocupe de problemele reale ale cet ̆ at , enilor. Q: Why did you lose the local elections? A: The press should focus on citizens’ real problems. Deflection Acknowledges the topic of the question but shifts the argument and focus to something other than what is asked. Î: Cât , i bani at , i alocat pentru acest program? R: Programul este un proiect la caret , inem foarte mult, îns ̆ a marea realizare de anul acesta este c ̆ a am reus , it s ̆ a moderniz ̆ am peste 50 de s , coli. Q: How much money did you allocate to this pro- gramme? A: The programme is a project we care deeply about, but this year’s big achievement is that we modernised over 50 schools. Clear Non-Reply Declining to an- swer A verbal, assumed and explicit refusal to provide the requested information. Î: Când vet , i publica raportul financiar? R: Nu voi face niciun comentariu pe acest subiect pân ̆ a la finalizarea auditului. Q: When will you publish the financial report? A: I will not comment on this matter until the audit is complete. Claimsigno- rance The speaker explicitly admits not holding the requested infor- mation at the moment. Î: La ce dat ̆ a exact ̆ a a ordonat guvernul reamenajarea navei HMAS Kanimbla? R: Nus , tiu acea dat ̆ a. Voi aflas , i voi informa Camera. Q: On what exact date did the government order the refit of HMAS Kanimbla? A: I do not know that date. I will find out and inform the Chamber. ClarificationDoes not provide the requested information and instead asks for clarification of what the ques- tion refers to. Î: A fost decizia dumneavoastr ̆ a s ̆ a eliberat , i fondul? R: V ̆ a referit , i la fondul public sau cel privat? Q: Was it your decision to release the fund? A: Do you mean the public fund or the private one? Table 10: PolERo taxonomy. Descriptions and examples are taken from the Romanian annotation guidelines used by annotators; English translations are provided in italics for readability. CLARITY (EN, n=308)PolERo (RO, n=296) ModelClarityEvasionMappedClarityEvasionMapped TF-IDF + Logistic Regression.422.301.441.453.312.472 Single-head encoder fine-tuning mBERT-base.529.347.517.601.328.502 BERT-large-cased.587.361.581.475.282.501 ELECTRA-large.607.231.524.500.084.241 ModernBERT-large.575.360.524.440.246.443 RoBERTa-large.580.481.656.646.428.613 XLM-RoBERTa-large.579.419.564.680.365.686 Romanian BERT (RO)–.684.470.634 RoBERT-large (RO)–.688.504.701 Multi-head encoder fine-tuning RoBERTa-large.713 .568.731.696.501.652 XLM-RoBERTa-large.665.475.644.694.508.706 RoBERT-large (RO)–.729.545.763 Cross-lingual transfer (XLM-RoBERTa-large, multi-head) EN→ RO–.635.399.655 EN2RO→ RO–.665.412.706 RO→ RO2EN–.681.494.702 RO + RO2EN→ RO–.708.492.681 RO + EN2RO→ RO–.718.489.684 EN + RO→ RO–.723.570.717 RO→ EN.551.398.527– RO2EN→ EN.477.358.439– EN→ EN2RO.616.448.612– EN + EN2RO→ EN.621.460.647– EN + RO2EN→ EN.694.477.602– EN + RO→ EN.679.483.642– EN + RO→ EN + RO † Clarity .737Evasion .529Mapped .705 Cross-lingual transfer (RoBERTa-large, multi-head) EN→ RO–.624.409.639 EN2RO→ RO–.594.421.584 RO→ RO2EN–.625.427.635 RO + RO2EN→ RO–.664.512.652 RO + EN2RO→ RO–.677.491.674 EN + RO→ RO–.712.463.681 RO→ EN.572.337.499– RO2EN→ EN.590.409.543– EN→ EN2RO.479.357.502– EN + EN2RO→ EN.672.513.654– EN + RO2EN→ EN.675.505.641– EN + RO→ EN.671.503.683– EN + RO→ EN + RO † Clarity .715Evasion .528Mapped .718 LLM prompting (no fine-tuning) Llama-3.3-70B (zero-shot).474.377.624.479.354.446 Llama-3.3-70B (few-shot).535.382.564.555.312.411 Gemma-4-31B-it (zero-shot).655.546.619.657.560.713 Gemma-4-31B-it (few-shot).721.583.662.672.619.738 GPT-OSS-120B (zero-shot).548.559.683.575.544.740 GPT-OSS-120B (few-shot).542.554.729.553.580.729 DeepSeek-V4-Pro (zero-shot).599.512.693.604.556.696 DeepSeek-V4-Pro (few-shot).601.545.670.627.594.751 Qwen3.6-35B-A3B (zero-shot).673.536.719.684.631.738 Qwen3.6-35B-A3B (few-shot).669.544.747 .746.682.760 GPT-5.4 (zero-shot).687.613.729.687.649.744 GPT-5.4 (few-shot).703.626.761.763.666 .763 Table 11: Full test-set macro-F1 across all models. Clarity: direct 3-class prediction. Evasion: direct 9-class prediction. Mapped: 3-class clarity prediction obtained by mapping the predicted 9-class evasion label upward through the taxonomy. For cross-lingual setups, EN2RO denotes English data machine-translated into Romanian, and RO2EN denotes Romanian data machine-translated into English. Best score per column is bolded; second-best is underlined. † Evaluated on the combined EN+RO test set (n=604).