Paper deep dive
ProBel: Propaganda Detection with Techniques, Spans, and Explanations
Mohamed Bayan Kmainasi, Ali Ezzat Shahroor, Elisa Sartori, Giovanni Da San Martino, Firoj Alam
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Propaganda detection includes several related prediction levels, ranging from sentence-level decisions to technique classification and span identification. However, it remains unclear how supervision at these levels interacts when learned jointly across Arabic and English. We present ProBel, an Arabic and English resource that aligns binary labels, multi-label annotations over 23 propaganda techniques grouped into six coarse categories, technique-labeled spans, and reference explanations for the same news sentences. It includes a substantially larger English collection and supports matched binary, coarse-grained, multi-label, and span-level tasks in both languages. We evaluate zero-shot prompting, task-specific fine-tuning, and joint training under a shared setup. A single bilingual multi-task model achieves the best overall performance and remains competitive across tasks and languages. Cross-task analysis shows that transfer depends on the supervision level. Joint classification training preserves binary performance, whereas span-only training can weaken sentence-level prediction. Joint bilingual training yields the most stable results, while monolingual fine-tuning can reduce transfer to the other language. We will release the data, code, and evaluation scripts.
Tags
Links
- Source: https://arxiv.org/abs/2608.22388v1
- Canonical: https://arxiv.org/abs/2608.22388v1
Trouble viewing inline? Open PDF directly â
Full Text
77,202 characters extracted from source content.
Expand or collapse full text
ProBel: Propaganda Detection with Techniques, Spans, and Explanations Mohamed Bayan Kmainasi 1 , Ali Ezzat Shahroor 1 , Elisa Sartori 2 , Giovanni Da San Martino 2 , Firoj Alam 1 1 Qatar Computing Research Institute, Qatar, 2 University of Padova, Italy mkmainasi, alsh34060,fialam@hbku.edu.qa, elisa.sartori.7@studenti.unipd.it, giovanni.dasanmartino@unipd.it https://huggingface.co/collections/QCRI/media-integrity-intelligence Abstract Propaganda detection includes several related prediction levels, ranging from sentence-level decisions to technique classification and span identification. However, it remains unclear how supervision at these levels interacts when learned jointly across Arabic and English. We present ProBel an Arabic and English resource that aligns binary labels, multi-label annota- tions over 23 propaganda techniques grouped into six coarse categories, technique-labeled spans, and reference explanations for the same news sentences. It includes a substantially larger English collection and supports matched binary, coarse-grained, multi-label, and span- level tasks in both languages. We evaluate zero- shot prompting, task-specific fine-tuning, and joint training under a shared setup. A single bilingual multi-task model achieves the best overall performance and remains competitive across tasks and languages. Cross-task analysis shows that transfer depends on the supervision level. Joint classification training preserves bi- nary performance, whereas span-only training can weaken sentence-level prediction. Joint bilingual training yields the most stable results, while monolingual fine-tuning can reduce trans- fer to the other language. We will release the data, code, and evaluation scripts. 1 1 Introduction Protecting public information has become an ur- gent global concern. The World Economic Forum ranks misinformation and disinformation as the sec- ond most severe global risk over the next two years (World Economic Forum, 2026). Propaganda con- tributes to this challenge by shaping public opinion through selective framing, emotional appeals, and other manipulative strategies (BarrĂłn-Cedeño et al., 2019; Da San Martino et al., 2019). Timely detec- tion can help readers, journalists, fact-checkers, 1 Resources: Code | Dataset | Model and moderators examine such content before it spreads widely. Propaganda detection has been studied through sentence-level decisions, fine-grained technique classification, and span identification (Da San Mar- tino et al., 2019, 2020; Piskorski et al., 2023; Hasanain et al., 2024). These tasks provide com- plementary views of the same content. However, they are often developed and evaluated separately. It remains unclear how supervision transfers across these tasks and whether the effects are consistent across Arabic and English. As shown in Figure 1, propaganda analysis can address four interrelated questions. Is propaganda present, which technique is used, where does it appear, and why does the identified text support the decision? Binary labels, multi-label technique annotations, technique-labeled spans, and natural language explanations address these questions, re- spectively. These tasks are related because a posi- tive decision should be supported by an identified technique and corresponding textual evidence. Existing resources cover parts of this struc- ture. SemEval-2020 studies propaganda spans and techniques in English news (Da San Mar- tino et al., 2020), SemEval-2023 studies persua- sion techniques in multilingual news (Piskorski et al., 2023), and ArPro provides fine-grained Arabic span annotations (Hasanain et al., 2024). PropXplain pairs Arabic and English binary labels with reference explanations (Hasanain et al., 2025). These resources separate annotation tasks across datasets, languages, or evaluation settings, limiting controlled analysis of cross-task supervision. We present PROBEL, an ArabicâEnglish re- source aligning binary labels, multi-label anno- tations for 23 propaganda techniques, technique- labeled spans, and reference explanations for news sentences and social media posts. It substantially expands the English collection while supporting the same tasks and taxonomy in both languages. 1 arXiv:2608.22388v1 [cs.CL] 23 Aug 2026 Figure 1: Overview of the aligned tasks and learning setup. Each Arabic and English sentence includes a binary label, fine-grained techniques grouped into six categories, labeled evidence spans, and a reference explanation, supporting task-specific and joint learning. This design enables controlled comparisons across task-specific, multi-task, monolingual, and bilin- gual settings, and allows us to assess whether tasks reinforce one another or cause negative transfer during joint training. We organize the study around three research questions. RQ1 How do zero-shot prompting, in-context learning, task-specific fine-tuning, and multi- task learning perform across binary detection, technique classification, and span identifica- tion? RQ2How does supervision transfer across binary, technique-level, and span-level tasks during joint training? RQ3How do the expanded English data and joint Arabic and English training affect perfor- mance across languages? RQ4 Can distillation, from a stronger teacher or from the modelâs own privileged predictions, replace direct fine-tuning? Our results show that a bilingual multi-task model performs best overall and remains com- petitive with specialized models across tasks and languages. Transfer varies by supervision level: joint classification preserves binary performance, whereas span-only training can weaken sentence- level prediction. Joint ArabicâEnglish training yields the most stable results and avoids the degra- dation observed with monolingual fine-tuning. Our contributions are as follows. âą A unified bilingual resource. PROBELaligns binary labels, 23-technique multi-label annota- tions, technique-labeled spans, and reference explanations for Arabic and English content. âą An expanded English collection. The En- glish data are substantially enlarged and sup- port the same annotation levels as the Arabic data, while the Arabic splits remain compara- ble with earlier results. âąA controlled cross-task study. Task-specific and multi-task models are evaluated under a shared setup to measure transfer across anno- tation levels. âąA bilingual transfer analysis. Monolingual and joint Arabic and English training are com- pared across all core tasks, together with anal- yses of cross-language transfer and long-tailed techniques. âąA distillation analysis. We show that neither teacher traces nor privileged self-distillation matches direct fine-tuning on PROBEL, diag- nose why the published on-policy recipe stalls on our task, and repair it with a saturating clip. 2 Related Work Research on propaganda detection spans sentence- level classification, technique identification, and span detection (Da San Martino et al., 2019, 2020). These tasks have also been extended to multilin- gual and multimodal settings (Dimitrov et al., 2021; Piskorski et al., 2023; Dimitrov et al., 2024). Re- cent work also investigates extractive rationales and natural language explanations to make model 2 decisions easier to inspect (Camburu et al., 2018; Mathew et al., 2021; Wiegreffe and Marasovi Ì c, 2021; Hase and Bansal, 2021). 2.1 Propaganda detection across tasks Early work examined article-level propaganda iden- tification and organization (BarrĂłn-Cedeño et al., 2019). Da San Martino et al. introduced 18 pro- paganda techniques for sentence-level binary de- tection and fragment-level technique identification (Da San Martino et al., 2019). SemEval-2020 later evaluated span identification and technique clas- sification in English news using 14 consolidated techniques (Da San Martino et al., 2020). Subsequent shared tasks expanded language, do- main, and modality coverage. SemEval-2021 stud- ied propaganda techniques in English memes (Dim- itrov et al., 2021), SemEval-2023 evaluated 23 per- suasion techniques in multilingual news (Piskorski et al., 2023), and SemEval-2024 addressed multi- lingual memes using a hierarchy of 22 techniques (Dimitrov et al., 2024). For Arabic, ArPro provides news paragraphs annotated with 23 techniques at paragraph and span levels (Hasanain et al., 2024). These resources offer complementary binary, technique, and span annotations. However, dif- ferences in task coverage, language, annotation design, and evaluation settings hinder controlled analysis of supervision transfer across tasks and between Arabic and English. 2.2 Span-Level Rationales Span-level rationales highlight input segments that support a model prediction and improve inter- pretability in tasks such as fact-checking, hate speech, and deception detection (Mathew et al., 2021; Yu et al., 2021; Russo et al., 2023). In pro- paganda detection, annotated spans identify text expressing a technique, while natural language ra- tionales justify the assigned label. Recent work has explored generating such explanations (Zavolok- ina et al., 2024; Atanasova et al., 2020; Hasanain et al., 2025). However, few studies jointly examine propaganda labels, technique spans, and natural language explanations in a unified setting. 2.3 Comparison with Prior Resources PropXplain (Hasanain et al., 2025) is the clos- est resource to PROBEL, providing Arabic and English propaganda instances with binary labels and reference explanations. However, PROBELhas broader task coverage, aligning binary labels, 23 fine-grained techniques, technique-labeled spans, and natural language rationales. It also expands the English collection and supports controlled analysis of cross-task supervision, multilingual transfer, and joint Arabic-English training. In Table 9, we com- pare the task coverage of related resources. Earlier English datasets introduced binary, technique, and span annotations, while later shared tasks expanded their linguistic and multimodal scope. ArPro pro- vides binary, technique, and span annotations for Arabic, whereas PropXplain adds explanations for Arabic and English binary classification. PRO- BELuniquely combines bilingual data with all four annotation levels under a shared 23-technique tax- onomy. 3PROBEL Resource 3.1 Overview PROBEL is an Arabic and English resource for explainable propaganda detection. Each instance includes four complementary forms of supervision. These consist of a binary propaganda label, one or more labels from a shared inventory of 23 tech- niques, text spans annotated with their correspond- ing techniques, and a reference natural language explanation. The resource contains news sentences and social media posts and supports binary clas- sification, fine-grained multi-label classification, and span identification under a common annotation structure. Figure 2 summarizes the construction process. Figure 2: Overview of the construction of PROBEL. Arabic and English content is represented with a binary label, six coarse categories, 23 fine-grained techniques, annotated text spans, and a reference explanation. 3 Lang. PropXplain Train/Dev/Test PROBEL Train/Dev/Test Addedinthis work AR 18,453/1,318/ 1,326 18,453/1,318/ 1,326 Technique labels andannotated spans EN 4,472/621/ 922 18,775/2,567/ 3,993 Additionaldata, technique labels, annotatedspans, and explanations Table 1: Comparison PROBEL with PropXplain. 3.2 Source collections and splits Arabic data. The Arabic collection is collected from PropXplain consisting of news and social me- dia content (Hasanain et al., 2025). The news por- tion contains paragraphs from articles published by 300 news agencies and covers 14 topics, including politics, human rights, and science and technology. The social media portion focuses on the Israeliâ Palestinian war. It was collected using 14 manually selected keywords and phrases that reflected top- ics discussed during October and early November 2023. The Twitter search API was used to retrieve posts published during the second week of Novem- ber 2023, resulting in approximately 5.7K posts. English data. The English collection extends pre- viously released data (Hasanain et al., 2025) with 97 additional articles, resulting in 347 articles from 42 news sources. The articles collection covers topics discussed during late 2023 and early 2024, with substantial coverage of politics and the Israeliâ Palestinian war. In Table 1, we summarize the relation between PROBEL and PropXplain. The Arabic splits are retained unchanged, while the English collection is substantially expanded. The present resource also supports fine-grained technique classification and span identification in both languages. 3.3 Annotation structure For the PROBEL, as shown in Figure 2, we use a common structure. Each instancexis represented asA(x) =âšb,Y,S,eâ©,wherebis the binary propa- ganda label,Yis the set of fine-grained technique labels,Scontains the annotated spans and their corresponding techniques, andeis the reference natural language explanation. We use the same representation for Arabic and English. Binary propaganda label.Each instance is labeled as either propagandistic or non- propagandistic. An instance is considered propa- gandistic when at least one propaganda technique is annotated and non-propagandistic otherwise. We use this rule to align the binary and fine-grained annotations (Hasanain et al., 2024, 2025). Fine-grained technique labels. We use the taxon- omy adopted in prior multilingual and Arabic pro- paganda resources (Piskorski et al., 2023; Hasanain et al., 2024). It organizes 23 fine-grained tech- niques. An instance may receive several labels when it contains more than one technique. Annotated text spans. For each occurrence of a technique, annotators identify the text span that expresses it and assign the corresponding technique label. An instance may contain several spans, and the same technique may appear more than once. We retain the character boundaries and technique label for every span. Reference explanations. Each instance includes a natural language explanation in the same lan- guage as the input. For propagandistic content, the explanation describes how the relevant word- ing expresses the annotated techniques. For non- propagandistic content, it explains why the text does not contain a propaganda technique. Follow- ing PropXplain, the explanations for the spans are generated using the gold labels, techniques and annotated spans. 3.4 Annotation Technique and span annotation. We retain the fine-grained annotations from the source collec- tions and apply the 23 techniques to the added English data. Each article was annotated indepen- dently by at least two annotators. Annotators se- lected all applicable techniques and marked the spans that expressed them. An expert annotator reviewed the annotations and resolved disagree- ments. Explanation generation. Following Hasanain et al. (2025), we use GPT-o1 to generate explanations from the gold binary label, technique labels, and annotated spans. Each explanation is produced in the input language, including for the newly added English data. Annotation agreement. For Arabic, three an- notators independently annotated each paragraph, and two expert consolidators reviewed the anno- tations (Hasanain et al., 2024). The agreement between the individual annotations and the consoli- dated labels usingÎłis 0.546 for span annotations. For English, the corresponding agreement score was 0.535. We generate explanations following the proce- dure of Hasanain et al. (2025). In that study, three 4 annotators evaluated each explanation in the Ara- bic and English test sets for faithfulness, clarity, plausibility, and informativeness using a five-point scale. Agreement, measured withr â wg(j) , ranged from 0.89 to 0.92 for Arabic and from 0.94 to 0.95 for English. To limit additional annotation cost, we do not repeat the human evaluation for the addi- tional English set. 3.5 Dataset statistics In Table 2, we report the split-wise data distribution. The dataset containsâŒ48K instances, including âŒ21 Arabic andâŒ27K English instances. Propa- gandistic instances account for 63.7% of Arabic and 28.0% of English. Lang. TrainDevTestTotal Pos. (%) AR18,453 1,318 1,326 21,09763.7 EN20,077 2,567 3,993 26,63728.0 Total 38,530 3,885 5,319 47,73443.8 Table 2: Dataset distribution by language. Pos.: the percentage of propagandistic instances. In Table 10, we summarize text length and the number of techniques and spans per propagandistic instance. Arabic shows a more skewed distribution, with its three most frequent techniques covering 94.8% of propagandistic instances, compared with 67.0% in English. The full 23-technique distribu- tion appears in the supplementary material. The mean explanation lengths for Arabic and English are 50.0 and 56.4 words, respectively. The technique distribution is heavily skewed: the three most frequent techniques cover 94.8% of Arabic and 67.0% of English propagandistic instances (Figure 7, Appendix C); per-technique test support is in Table 18. 4 Tasks and Experimental Setup 4.1 Task formulations We study four core prediction tasks as presented in Figure 1. Binary propaganda detection. Given an input text, the model predicts whether it contains at least one propaganda technique. We compute macro-F 1 for this task. Multilabel technique classification. Given an in- put text, the model predicts all applicable tech- niques from the 23-label taxonomy. We compute micro-F 1 to account for multiple labels. Coarse-grained technique classification. The 23- technique taxonomy is highly imbalanced, with a 1. Input sentence The results have been unmitigated disaster after disaster. Technique: Loaded Language 2. LLM response Span-tag ... <span type="Loaded_Language"> disaster after disaster</span>... â cannot represent overlapping spansâ marks spans in context Span-occ "text": "disaster after disaster", "label": "Loaded_Language", "occurrence": 1 â inexact copies are lostâ keeps overlaps and repeats Figure 3: Two LLM span representations. Span-tag inserts inline tags, while span-occ outputs one object per span and supports repeated or overlapping spans. few frequent techniques and limited support for many others, a common pattern in propaganda datasets (Hasanain et al., 2024). To reduce label sparsity and support more reliable learning and evaluation, we map the 23 fine-grained techniques to six broader categories, as shown in Table 11. For the multilibel classification, the task is formu- lated as follows. Given an input text, the model predicts all applicable coarse-grained categories. We compute micro-F 1 for task as well. Span detection with technique labels Given an input text, the model extracts each propagandis- tic span and assigns the corresponding technique. We evaluate two output representations. Span-tag reproduces the input with inline technique tags, whereas Span-occ returns a JSON list of (technique, span, occurrence) triples. An example is provided in Figure 3. The occurrence index distinguishes repeated surface forms. We evaluate both repre- sentations using the overlap-adjusted micro-F 1 of Da San Martino et al. (2020). Explanation generation For each classification formulation, generative models also produce an explanation justifying the predicted label. We com- pute BERTScore-F 1 (Zhang et al., 2020) to evaluate the generated explanations. 4.2 Comparison systems Baselines. We compute majority-label and random baselines for the classification formulations. For span detection, we use a whole-text baseline that assigns the most frequent technique to the complete input. Prompting baselines We evaluate open-weight and proprietary LLMs under zero-shot prompting using only the task instructions, label definitions, and required output format. For in-context learning (ICL), we retrievekdemonstrations from the train- 5 ing pool with BGE-M3 embeddings (Chen et al., 2024); the retriever, pool, andkare selected on development data (Appendix F). Fine-tuned models. We compare task-specific models trained separately for each task and lan- guage with a bilingual multi-task model trained across all formulations. All generative models use Qwen2.5-7B-Instruct with LoRA (Hu et al., 2022), with checkpoints selected by development loss. We provide Hyperparameters in Appendix D. We also evaluate discriminative baselines using classi- fication heads on Llama-3.1-8B and Qwen2.5-7B, along with AraBERT-v2 for Arabic and BERT-base for English (Appendix H). Decoding and parsing. We use fixed output tem- plates, a shared parser, and greedy decoding with zero temperature. In Appendix D, we provide the templates, parsing details, and hyperparameters. 4.3 Controlled comparisons Cross-task transfer. We compare four training setups. The first uses binary supervision only. The second uses the three classification formulations covering binary, coarse-grained, and fine-grained prediction. The third uses the two span formula- tions. The fourth uses the five formulations. We evaluate each setup on every formulation, includ- ing those not observed during training, to measure transfer across tasks. Cross-language transfer. We compare Arabic- only, English-only, and joint Arabic and English training under the same multi-task configuration. This analysis examines whether supervision in one language improves performance in the other or weakens its existing capabilities. Distillation.We also test whether the aligned an- notations can be replaced by cheaper supervision: off-policy distillation from GPT-5 traces (Hsieh et al., 2023), and on-policy self-distillation from a frozen teacher that sees the gold annotation in con- text (Zhao et al., 2026). Beyond this comparison, we diagnose why the published on-policy objective stalls and propose a saturating clip that repairs it. 5 Experimental Results 5.1 Baseline Results In Table 3, we show that zero-shot performance re- mains limited and varies substantially across mod- els and languages. The strongest open model out- performs GPT-5 in each language, while several SystemAREN No learning Majority label0.380 0.419 Random, fair coin0.499 0.470 Zero-shot, open weights Fanar-2-27B (TEAM et al., 2026)0.409 0.431 Llama-3.1-8B (Grattafiori et al., 2024) 0.672 0.437 Qwen2.5-7B (Team, 2024)0.439 0.683 Qwen3-VL-8B (Yang et al., 2025)0.433 0.669 Qwen3-VL-8B (think)0.577 0.625 Gemma-4-E4B (Team, 2026)0.442 0.671 Zero-shot, proprietary GPT-5 (CoT) (Singh et al., 2025)0.646 0.657 Gemini-3.1-Pro (CoT) 2 0.505 0.673 MT-SFT (fine-tuned)0.7630.735 Table 3: Binary macro-F 1 without task-specific training; best open model per language in bold. 0.0 0.2 0.4 0.6 0.8 Arabic F 1 0.36 0.41 0.58 0.68 0.76 BinaryCoarseTechniqueSpan-tagSpan-occ Zero-shotICL (k=5) Task-specific fine-tuning Multi-task bilingual 0.0 0.2 0.4 0.6 0.8 English F 1 0.19 0.24 0.27 0.41 0.74 Figure 4: Performance of Qwen2.5-7B-Instruct across five tasks and four training settings. Full configurations and ablations appear in Appendices D and F. models approach or fall below the random base- line on Arabic. Fine-tuning yields clear gains over all prompted systems in both languages. Exact McNemar tests confirm the main improvements as statistically significant. We provide further details about the statistical significance in Appendix L. 5.2 Results across Tasks & Settings Difficulty increases with output granularity. In Figure 4, we show that performance declines as out- puts become more fine-grained. Models perform best on binary classification, followed by coarse and technique classification, and generally perform worst on span prediction. English task-specific span-occ provides the only exception. In-context learning. Few-shot ICL improves per- formance across all Arabic tasks, including binary detection. In English, it provides limited gains and reduces binary performance, where zero-shot 2 https://deepmind.google/models/gemini/pro/ 6 prompting is already strong. It also fails to close the gap on span detection, which remains the most challenging task in both languages. A single bilingual multi-task model performs best overall. MT-SFT matches or outperforms task-specific models on all classification tasks and remains within 1.4 points on span prediction (See Fig 4). It handles both languages and all tasks. En- glish remains more challenging as its test set con- tains fewer propagandistic instances, which limits positive examples for fine-grained techniques. Span representations vary by language. In Fig- ure 4, we find that span-tag performs better in Ara- bic, while span-occ performs better in English. We therefore report both formats. In Appendix E, we provide detail results and comparisons. 5.3 Transfer across Tasks In Figure 5, we show that joint supervision gener- ally outperforms single-task training. Classification tasks reinforce one another, preserving binary per- formance while improving coarse- and technique- level prediction. Span-only training weakens bi- nary detection, whereas classification-only training fails to recover accurate span boundaries, indicat- ing that sentence- and token-level supervision pro- vide complementary signals. Training on all five tasks combines these benefits. MT-SFT matches or exceeds task-specific models in most settings, with small losses on span prediction and the largest gains on English classification tasks. Overall, multi-task learning yields the strongest and most balanced performance across languages and tasks. Binary CoarseTechnique Span-tag Span-occ ZS ICL Bin Cls Spn All All bi prompt fine-tuned 0.440.440.240.060.09 0.670.550.380.130.20 0.750.560.250.180.20 0.760.670.570.210.22 0.480.510.220.420.37 0.770.670.560.410.36 0.760.680.580.410.36 Arabic test Binary CoarseTechnique Span-tag Span-occ ZS ICL Bin Cls Spn All All bi prompt fine-tuned 0.680.290.120.090.11 0.630.280.170.140.12 0.710.270.050.070.06 0.700.280.180.100.09 0.640.320.150.170.25 0.710.350.270.140.20 0.740.410.270.190.24 = task in the training mix (bold) colour ranks scores within each column English test Figure 5: Supervision transfer with Qwen2.5-7B- Instruct across prompting and fine-tuning settings. Out- lined cells mark trained tasks, and colors are normalized by column. Table 15 reports the full results. 0.00.10.20.30.40.50.60.7 five-task average F 1 AR EN AR+EN training 0.55 0.32 0.11 0.34 0.56 0.37 zero-shot Arabic test English test Figure 6: Five-task average by training language, Qwen2.5-7B-Instruct. English-only fine-tuning falls below the zero-shot level on Arabic. 5.4 Data Expansion & Cross-Lingual Transfer English data expansion improves performance. As shown in Table 4, expanding the English train- ing set improves results on the new benchmark. Binary-only training overfits and degrades on the original test set, whereas multi-task training im- proves both benchmarks. Cross-lingual transfer is asymmetric. Figure 6 shows that transfer across languages is highly asym- metric. Arabic fine-tuning transfers effectively to English, especially for structured prediction tasks, while English-only fine-tuning transfers poorly to Arabic and substantially degrades technique and span prediction. Joint bilingual training is most robust. Joint training achieves the strongest overall performance across both languages and avoids the severe Arabic degradation caused by English-only fine-tuning. It therefore provides the best balance between task performance and cross-lingual generalization. PROBELOriginal English training dataEN testEN test Original, binary only0.5820.641 Expanded, binary only0.7110.593 Expanded, multi-task bilingual0.7350.685 Table 4: English binary macro-F 1 under a fixed training recipe. Data expansion improves PROBEL, while only multi-task training generalizes to the original PropX- plain test set. 5.5 Distillation and Direct Supervision We test whether distillation can replace direct task supervision and we evaluate two approaches. (i) Off-policy distillation trains Qwen2.5-7B to imitate GPT-5 reasoning traces generated from the refer- ence annotations (Hsieh et al., 2023). (i) On-policy self-distillation (OPSD) instead matches a student conditioned on the input to a teacher conditioned on both the input and gold annotation (Zhao et al., 2026), as illustrated in Figure 9. Following (Zhao et al., 2026), OPSD clips each token-level forward- KL contribution 7 L = 1 |Ëy| |Ëy| X n=1 X vâV min(â n,v ,Ï),(1) whereâ n,v = p T (v)[logp T (v)â logp S (v)], and the released configuration sets Ï = 0.06. Starting point TrainingBinAvg. Qwen2.5-7B none (zero-shot)0.561 0.256 off-policy, GPT-5 traces0.697 0.441 OPSD, best configuration 0.623 0.288 Qwen3-8B none (zero-shot)0.490 0.295 OPSD, hard Ï = 0.060.499 0.298 OPSD, hard Ï = 0.50.485 0.295 OPSD, soft Ï = 0.50.588 0.314 MT-SFT none0.749 0.464 OPSD, hard Ï = 0.50.742 0.429 OPSD, soft Ï = 0.50.751 0.464 Table 5: Distillation results across starting points. Bin denotes binary macro-F 1 , and Avg. the five-task average across languages. Appendix K reports full results. Direct supervision remains strongest. In Table 5, off-policy distillation substantially improves the zero-shot model, whereas OPSD provides limited gains. Neither approach matches MT-SFT, showing that distillation complements rather than replaces direct supervision. Hard clipping suppresses decision-token gradi- ents. In Figure 9, we find that teacherâstudent disagreement concentrates on a small number of decision tokens with KL contributions above the clipping threshold. Hard clipping removes their gradients, while increasingÏadds limited useful supervision. Saturating clipping improves OPSD. We replace hard clipping with f Ï (â) = Ï log 1 + â Ï ,(2) which reduces large contributions without elimi- nating their gradients. This consistently improves OPSD, recovering Qwen3-8Bâs English binary and increasing the average performance, with similar trends on HateXplain (Mathew et al., 2021) and GSM8K (Cobbe et al., 2021). Nevertheless, even with the proposed clipping function, distillation remains complementary to rather than a replace- ment for direct supervision. Additional analyses are provided in Appendix K. 6 Comparison with Published Results In Table 7, we compare our bilingual multi-task model with the PropXplain systems that, like ours, predict the label and generate an explanation (Hasanain et al., 2025), on identical test splits. MT- SFT obtains the best macro-F 1 in both languages Testbed (metric)ÏZero-shot HardSoft PROBEL EN binary (macro-F 1 )0.50.4490.448 0.692 HateXplain (macro-F 1 )0.50.6040.666 0.687 GSM8K (accuracy)0.060.9180.900 0.928 AIME/HMMT (Avg@12)0.0534.241.939.8 Table 6: Hard versus saturating (soft) clipping on four testbeds. Zero-shot is the model before OPSD training; each row uses its own metric and thresholdÏ(the pub- lished value on GSM8K and the original AIME/HMMT benchmarks,Ï=0.5elsewhere). Best per row in bold. Appendix K reports the full grids. while additionally producing techniques and spans from the same checkpoint. Table 8 compares span detection against the pub- lished GPT-4 results of Hasanain et al. (2024). The Arabic comparison is on the identical test split. For English no published result exists on our test set, which is introduced in this work; the closest published number is on the SemEval- 2023 development set, and is marked as a different split. SemEval-2020 span identification omits tech- nique labels, and SemEval-2023 techniques (Pisko- rski et al., 2023; Lepekhin and Sharoff, 2023) are paragraph-level, so neither is directly comparable. SystemMacro-F 1 F 1 BERT ⥠Arabic, same test split Llama-3.1-8B (base) (Hasanain et al., 2025)0.5880.507 Llama-3.1-8B (FT) (Hasanain et al., 2025)0.7600.706 MT-SFT (ours; five tasks, two languages)0.7630.664 English, original PropXplain test set Llama-3.1-8B (base) (Hasanain et al., 2025)0.5620.596 Llama-3.1-8B (FT) (Hasanain et al., 2025)0.6750.751 MT-SFT (ours; five tasks, two languages)0.6850.718 Table 7: Comparison with PropXplain on matched test splits. Bold marks the best macro-F 1 . ⥠Explanation BERTScores use different backbones across studies and are not directly comparable. SystemAREN GPT-4, zero-shot (Hasanain et al., 2024)0.1170.111 â Task-specific SFT (ours)0.4210.255 MT-SFT (ours)0.4110.241 Table 8:Technique-labeled span detection using overlap-adjusted micro-F 1 . Arabic uses the same test split. â The English GPT-4 score uses the SemEval-2023 development set and is not directly comparable. 7 Conclusions and Future Work We frame propaganda analysis around four interre- lated questions. Is propaganda present, which tech- niques appear, where do they occur, and why do the identified spans support the prediction? To study these questions jointly, we introduce PROBEL, an ArabicâEnglish resource that extends PropXplain 8 with aligned binary, multi-label, span-level, and ex- planation annotations. Our experiments show that bilingual multi-task training provides the strongest and most balanced performance across tasks and languages, while simplifying deployment through a single model. Future work will extend the resource to additional languages and domains, improve span and explanation generation, and investigate more effective transfer and distillation methods for fine- grained propaganda analysis. Limitations Our study focuses on Arabic and English propa- ganda in news sentences and social media posts. We use a shared taxonomy of 23 techniques to enable controlled comparisons across tasks and lan- guages. As span annotation requires careful judg- ment, particularly at ambiguous boundaries, we use multiple annotators and expert review to improve consistency. Reference explanations provide valu- able supervision and evaluation targets. Our exper- iments cover several representative model families and training settings. Future work can extend this analysis to additional architectures, domains, lan- guages, and methods for evaluating explanation faithfulness. Ethics and Broader Impact Propaganda detection can support media analysis, fact-checking, and research on harmful or manip- ulative communication. The same systems may also produce incorrect labels or be used to suppress legitimate criticism, satire, or political expression. We therefore recommend using these models to support trained human reviewers, particularly in high-stakes moderation and policy settings. The dataset may contain political, sensitive, or offen- sive content from news and social media sources. Users should handle the data carefully and consider the cultural and political context of each instance. We will release data, code, and evaluation scripts to improve transparency and reproducibility. We also encourage users to report results separately across languages and tasks to avoid masking uneven per- formance. Acknowledgments The work of F. Alam, M. Hasanain, and F. Ahmed is supported by the NPRP grant 14C-0916-210015 from the Qatar National Research Fund part of Qatar Research Development and Innovation Coun- cil (QRDI). The findings achieved herein are solely the responsibility of the authors. References Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Pi- otr Stanczyk, Sabela Ramos Garea, Matthieu Geist, and Olivier Bachem. 2024. On-policy distillation of language models: Learning from self-generated mistakes. In The Twelfth International Conference on Learning Representations (ICLR). Pepa Atanasova, Jakob Grue Simonsen, Christina Li- oma, and Isabelle Augenstein. 2020. Generating fact checking explanations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7352â7364, Online. Association for Computational Linguistics. Alberto BarrĂłn-Cedeño, Giovanni Da San Martino, Is- raa Jaradat, and Preslav Nakov. 2019. Proppy: A system to unmask propaganda in online news. In Pro- ceedings of the The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Con- ference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 - February 1, 2019, pages 9847â9848. AAAI Press. Oana-Maria Camburu, Tim RocktĂ€schel, Thomas Lukasiewicz, and Phil Blunsom. 2018. e-snli: Natu- ral language inference with natural language expla- nations. In Proceedings of the Advances in Neural Information Processing Systems 31: Annual Con- ference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, MontrĂ©al, Canada, pages 9560â9572. Jianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024.M3- embedding: Multi-linguality, multi-functionality, multi-granularity text embeddings through self- knowledge distillation. In Findings of the Asso- ciation for Computational Linguistics: ACL 2024, pages 2318â2335, Bangkok, Thailand. Association for Computational Linguistics. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word prob- lems. ArXiv preprint, arXiv:2110.14168. Giovanni Da San Martino, Alberto BarrĂłn-Cedeño, Henning Wachsmuth, Rostislav Petrov, and Preslav Nakov. 2020. SemEval-2020 task 11: Detection of propaganda techniques in news articles. In Proceed- ings of the Fourteenth Workshop on Semantic Evalu- ation, pages 1377â1414, Barcelona (online). Interna- tional Committee for Computational Linguistics. 9 Giovanni Da San Martino, Seunghak Yu, Alberto BarrĂłn-Cedeño, Rostislav Petrov, and Preslav Nakov. 2019. Fine-grained analysis of propaganda in news article. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natu- ral Language Processing (EMNLP-IJCNLP), pages 5636â5646, Hong Kong, China. Association for Com- putational Linguistics. Dimitar Dimitrov, Firoj Alam, Maram Hasanain, Abul Hasnat, Fabrizio Silvestri, Preslav Nakov, and Gio- vanni Da San Martino. 2024. Semeval-2024 task 4: Multilingual detection of persuasion techniques in memes. In Proceedings of the 2024 Annual Confer- ence of the North American Chapter of the Associa- tion for Computational Linguistics. Dimitar Dimitrov, Bishr Bin Ali, Shaden Shaar, Firoj Alam, Fabrizio Silvestri, Hamed Firooz, Preslav Nakov, and Giovanni Da San Martino. 2021. Task 6 at SemEval-2021: Detection of persuasion tech- niques in texts and images. In Proceedings of the 15th International Workshop on Semantic Evalua- tion, SemEval â21, Bangkok, Thailand. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al- Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. 2024. The llama 3 herd of mod- els. arXiv preprint arXiv:2407.21783. Maram Hasanain, Fatema Ahmad, and Firoj Alam. 2024. Can GPT-4 identify propaganda? annotation and de- tection of propaganda spans in news articles. In Pro- ceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC-COLING â24, pages 2724â 2744, Torino, Italia. ELRA and ICCL. Maram Hasanain, Md Arid Hasan, Mohamed Bayan Kmainasi, Elisa Sartori, Ali Ezzat Shahroor, Gio- vanni Da San Martino, and Firoj Alam. 2025. PropX- plain: Can LLMs enable explainable propaganda detection? In Findings of the Association for Compu- tational Linguistics: EMNLP 2025, pages 23855â 23863, Suzhou, China. Association for Computa- tional Linguistics. Peter Hase and Mohit Bansal. 2021. When can models learn from explanations? a formal framework for understanding the roles of explanation data. ArXiv preprint, arXiv:2102.02201. Cheng-Yu Hsieh, Chun-Liang Li, Chih-Kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alexander Ratner, Ranjay Krishna, Chen-Yu Lee, and Tomas Pfister. 2023. Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes. In Findings of the Association for Com- putational Linguistics: ACL 2023, Toronto, Canada. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In Proceedings of the 10th In- ternational Conference on Learning Representations, ICLR â22. Jonas HĂŒbotter, Frederike LĂŒbeck, Lejs Behric, An- ton Baumann, Marco Bagatella, Daniel Marta, Ido Hakimi, Idan Shenfeld, Thomas Kleine Buening, Car- los Guestrin, and Andreas Krause. 2026. Reinforce- ment learning via self-distillation. ArXiv preprint. Simran Kaur, Narutatsu Ri, Yinghui He, Liam Fowl, and Sanjeev Arora. 2026. Rethinking on-policy self-distillation for thinking models. ArXiv preprint, arXiv:2607.05184. Mikhail Lepekhin and Serge Sharoff. 2023. FTD at SemEval-2023 task 3: News genre and propaganda detection by comparing mono- and multilingual mod- els with fine-tuning on additional data. In Proceed- ings of the 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 549â555, Toronto, Canada. Association for Computational Linguistics. Binny Mathew, Punyajoy Saha, Seid Muhie Yimam, Chris Biemann, Pawan Goyal, and Animesh Mukher- jee. 2021. Hatexplain: A benchmark dataset for ex- plainable hate speech detection. In Proceedings of the AAAI conference on artificial intelligence, vol- ume 35, pages 14867â14875. Jakub Piskorski, Nicolas Stefanovitch, Giovanni Da San Martino, and Preslav Nakov. 2023. SemEval- 2023 task 3: Detecting the category, the framing, and the persuasion techniques in online news in a multi- lingual setup. In Proceedings of the 17th Interna- tional Workshop on Semantic Evaluation (SemEval- 2023), pages 2343â2361, Toronto, Canada. Associa- tion for Computational Linguistics. Daniel Russo, Serra Sinem Tekiro Ì glu, and Marco Guerini. 2023. Benchmarking the generation of fact checking explanations. Transactions of the Associa- tion for Computational Linguistics, 11:1250â1264. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms.ArXiv preprint, arXiv:1707.06347. Danil Semin, Ond Ë rej DuĆĄek, and Zden Ë ek Kasner. 2026. Strategies for span labeling with large language mod- els. ArXiv preprint, arXiv:2601.16946. Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, Adi Ganesh, Ahmed El-Kishky, Aidan McLaughlin, Aiden Low, AJ Ostrow, Akhila Ananthram, et al. 2025. Openai gpt-5 system card. arXiv preprint arXiv:2601.03267. FANAR TEAM, Ummar Abbas, Mohammad Shah- meer Ahmad, Minhaj Ahmad, Abdulaziz Al-Homaid, Anas Al-Nuaimi, Enes Altinisik, Ehsaneddin As- gari, Sanjay Chawla, Shammur Chowdhury, Fahim Dalvi, Kareem Darwish, Nadir Durrani, Mohamed Elfeky, Ahmed Elmagarmid, Mohamed Eltabakh, Asim Ersoy, Masoomali Fatehkia, Mohammed Qusay 10 Hashim, Majd Hawasly, Mohamed Hefeeda, Musâab Husaini, Keivin Isufaj, Soon-Gyo Jung, Houssam Lachemat, Ji Kim Lucas, Abubakr Mohamed, Tas- nim Mohiuddin, Basel Mousi, Hamdy Mubarak, Ah- mad Musleh, Mourad Ouzzani, Amin Sadeghi, Hus- rev Taha Sencar, Mohammed Shinoy, Omar Sinan, and Yifan Zhang. 2026. Fanar 2.0: Arabic generative ai stack. ArXiv preprint, arXiv:2603.16397. Gemma Team. 2026. Gemma 4 technical report. ArXiv preprint, arXiv:2607.02770. Qwen Team. 2024. Qwen2.5: A party of foundation models. Sarah Wiegreffe and Ana Marasovi Ì c. 2021. Teach me to explain: A review of datasets for explainable natural language processing. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks. World Economic Forum. 2026. The global risks report 2026. Technical report, World Economic Forum. Published: 14 January 2026. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayi- heng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xu- ancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. 2025. Qwen3 technical report. ArXiv preprint, arXiv:2505.09388. Shunyuan Yu, Giovanni Da San Martino, Mitra Mo- htarami, James Glass, and Preslav Nakov. 2021. In- terpretable propaganda detection in news articles. ArXiv preprint, arXiv:2108.12802. Liudmila Zavolokina, Kilian Sprenkamp, Zoya Katashinskaya, Daniel Gordon Jones, and Gerhard Schwabe. 2024. Think fast, think slow, think criti- cal: Designing an automated propaganda detection tool. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, CHI â24, Honolulu, HI, USA. Association for Computing Ma- chinery. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. BERTScore: Evaluating text generation with BERT. In Proceed- ings of the 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net. Siyan Zhao, Zhihui Xie, Mengchen Liu, Jing Huang, Guan Pang, Feiyu Chen, and Aditya Grover. 2026. Self-distilled reasoner: On-policy self-distillation for large language models. In Forty-third International Conference on Machine Learning. A Comparison with Prior Work In Table 9, we compare the task coverage of closely related propaganda resources. Prior datasets typi- cally support only a subset of binary classification, technique classification, span identification, and natural language explanation generation. In con- trast, PROBELcombines Arabic and English data with binary labels, a shared taxonomy of 23 tech- niques, technique-annotated spans, and reference explanations. ResourceLang. Bin. Tech. Span NLE Da San Martino et al. (Da San Martino et al., 2019)ENâ18ââ SemEval-2020 (Da San Martino et al., 2020)ENâ14ââ SemEval-2023 (Piskorski et al., 2023)9Lâ23â ArPro (Hasanain et al., 2024)ARâ23ââ SemEval-2024 (Dimitrov et al., 2024)4Lâ22â PropXplain (Hasanain et al., 2025)AR/ENâââ PROBELAR/ENâ23â Table 9: Task coverage in prior propaganda resources. Bin.: binary sentence- or paragraph-level classification; Tech.: the number of fine-grained techniques; Span: technique-annotated text spans; NLE: natural language explanations. 9L/4L: nine/four languages. BPROBEL Additional Details Table 10 summarizes the annotation statistics by language. Arabic instances are longer on average and contain more techniques and annotated spans than English instances. They also show substan- tially higher proportions of multi-technique and multi-span annotations, indicating greater annota- tion complexity. C Label Inventory and Task Derivation Both languages share a unified inventory of 23 pro- paganda techniques grouped into six coarse cate- gories (Table 11), following the SemEval-2023 per- suasion taxonomy (Piskorski et al., 2023), which was adopted for Arabic by ArPro (Hasanain et al., 2024). Per-technique test support is given in Ta- ble 18. Hierarchical annotation. Span annotations are the primary labels. A sentence is propagandistic if it contains at least one annotated span; fine- and coarse-grained labels are derived from the span techniques. We verified this hierarchy for every sentence in both languages, ensuring that all anno- tation levels are consistent by construction. 11 Lang. Text Tech. Span Multi-T Multi-S AR32.61.83.053.370.7 EN23.81.31.524.930.6 Table 10: Annotation statistics by language. Text: the mean input length in words. Tech. and Span represent mean counts over propagandistic instances. Multi-T and Multi-S represent the percentages of propagandistic instances with more than one technique or span. Figure 7: Frequency of the ten most common techniques among propagandistic instances. Annotation characteristics.Arabic annotations are denser than English, with more spans, distinct techniques, and repeated techniques per sentence, as well as shorter spans. This increases structured prediction complexity and motivates the span-occ representation. D Experimental Setup Table 12 lists every configuration used in the paper. We fix one recipe before running the ablations and reuse it for every fine-tuned model, so no compar- ison mixes recipes. All generative systems share Qwen2.5-7B-Instruct, emit one fixed textual format per task, and are scored by a single parser, so the differences we report come from supervision rather than from decoding or evaluation choices. E Span Representations: Details and Results Span-tag preserves sentence context and is gener- ally more robust, whereas span-occ naturally sup- ports overlapping spans and repeated strings. The two formats exhibit complementary failure modes: Coarse categoryFine-grained techniques Manipulative WordingLoaded Language, Exaggerationâ Minimisation,Obfuscationâ VaguenessâConfusion, Repetition ReputationName CallingâLabeling, Ques- tioning the Reputation, Doubt, Guilt by Association, Appeal to Hypocrisy JustificationAppeal to Authority, Appeal to FearâPrejudice, Appeal to Popu- larity, Appeal to Values, Flag Wav- ing SimplificationCausal Oversimplification, Conse- quential Oversimplification, False DilemmaâNo Choice CallSlogans, Conversation Killer, Ap- peal to Time DistractionRedHerring,StrawMan, Whataboutism Table 11: The shared label inventory: 23 techniques in six coarse categories, identical for both languages. span-tag can break alignment when the model edits the copied sentence, while span-occ fails when gen- erated spans do not exactly match the input, particu- larly in Arabic, where spans are shorter and denser. Encoder baselines instead use token-level indexing, which is naturally aligned with BIO tagging; we did not use character-offset indexing with LLMs, whose tokenized view makes predicted offsets un- reliable (Semin et al., 2026). Table 13 compares all systems on the span task under both representa- tions. Both span formats are built from the same gold spans (Algorithm 1), parsed back to character off- sets by one routine per format (Algorithm 2), and scored by the same metric, so the representation is the only variable between them. Scoring. Parsed predictionsSand gold spansT are evaluated using the overlap-adjusted measure of Da San Martino et al. (2020), where only label- matching pairs contribute their character overlap normalized by the span in focus. P = 1 |S| X sâS X tâT |sâ© t| |s| ÎŽ(â(s),â(t)), R = 1 |T| X sâS X tâT |sâ© t| |t| ÎŽ(â(s),â(t)),(3) with F 1 their harmonic mean; sums and span counts are aggregated over the whole test set (micro) after removing duplicate (start, end, label) triples. 12 ParameterValue Fine-tuning (identical for every regime) BackboneQwen2.5-7B-Instruct AdapterLoRA r=16, α=32 Learning rate1Ă 10 â5 , warmup ratio 0.05 Epochs / batch4 / 4 per device (4 GPUs) Max sequence length 4,096 Checkpoint selection minimum validation loss Training regimes Binary onlybinary Classificationbinary, coarse, technique Spans onlyspan-tag, span-occ All fiveall tasks above LanguagesArabic, English, and both In-context learning Retrieversrandom, BM25, LaBSE, mE5, BGE-M3, hybrid Poolssame-language, cross-lingual, mixed Demonstrationsk â1, 3, 5, most similar last Selection18 combinations, ranked on dev at k=3 Discriminative baselines HeadsLlama-3.1-8B, Qwen2.5-7B + linear head EncodersAraBERT-v2 (AR), BERT-base (EN) ParameterValue Off-policy distillation (Appendix K) Teacher tracesGPT-5, rationalize gold Recipeas fine-tuning; 8 epochs; dev-selected OPSD (original recipe unless noted) AdapterLoRA r=64, α=128, all proj. Learning rate5Ă 10 â6 ; max gradient norm 0.1 Effective batch32 (2/device, accum. 4, 4 GPUs) Divergenceforward KL, full vocabulary Rolloutstemp. 1.1, top-p 0.95, top-k 20, budget 1,024 tokens Steps150, ckpt. every 10 (ext.: 450) Selectiondevelopment subset (360 items) ClipÏ â0,.06,.1,.2,.5, 1, hard or saturating Seeds42; replicates 13, 77 on the pivotal contrast Evaluation Decodinggreedy Binarymacro-F 1 Coarse, technique micro-F 1 Spansoverlap-adjusted micro-F 1 ExplanationsBERTScore-F 1 , BLEU, METEOR Table 12: Configurations used throughout the paper. ArabicEnglish SystemtagoccBIOtagoccBIO Whole sentence as span0.082 0.082 0.082 0.038 0.038 0.038 AraBERT-v2 / BERT-baseâ0.249â0.227 Qwen2.5-7B zero-shot0.058 0.093â0.087 0.108â Qwen2.5-7B ICL, k=50.128 0.200â0.142 0.122â GPT-5 (CoT)â0.218â0.180â Task-specific SFT0.421 0.374â0.173 0.255â MT-SFT0.411 0.362â0.189 0.241â Table 13: Technique-labeled span detection (overlap- adjusted micro-F 1 ; best per column in bold). Tag and occ are the generative formats; BIO is the encodersâ token-level tagging. GPT-5 emits one span list, scored under the occ protocol. F In-Context Learning Ablations We select the in-context configuration in two stages so that the main comparison uses one justified set- ting rather than a tuned-per-cell best case. Stage 1: which retriever and which pool. We rank all 18 combinations of six retrievers and three demonstration pools on development data atk=3, scoring the mean of the primary metric over Ara- bic and English binary and technique classification (Figure 8). Dense retrieval with BGE-M3 over a same-language pool ranks first (0.497). Every cross-lingual configuration scores within 0.004 of the random same-language baseline (0.426), with the best reaching only 0.430, and sparse retrieval with BM25 trails all dense retrievers. same lang. mixed cross-ling. Random BM25 LaBSE mE5 BGE-M3 Hybrid 0.4260.4190.419 0.4610.4500.409 0.4810.4940.418 0.4880.4810.407 0.4970.4720.430 0.4860.4650.415 Figure 8: Development ranking of retrieverâpool com- binations at k=3; the outlined cell is the winner. Stage 2: how many demonstrations. We run the winner, its cross-lingual counterpart, and a ran- dom control on the full test set atk â 1, 3, 5 (Table 14). Retrieved same-language demonstra- tions improve monotonically withkon Arabic for every task except binary, wherek=3is best. On English they do not help binary detection at any k. Cross-lingual demonstrations track the random baseline throughout, even though the English pool 13 Algorithm 1 Target construction Span-tag 1: sort gold spans by start offset, descending 2: for each span (s,e,â) do 3: insert<span type="â">ats,</span>at e 4: end for right-to-left insertion keeps offsets valid for disjoint spans; overlapping spans cannot be represented as well-formed inline markup and are carried only by span-occ Span-occ 5: for each span (s,e,â) do 6: w â sentence[s:e];k âwhich occur- rence of w starts at s (1st, 2nd, . . . ) 7: emit"text": w, "label": â, "occurrence": k 8: end for is 4.2 times larger than the Arabic one. When a same-language pool exists, even a small one, it should be used. ConfigurationBinCoaTech S-tag S-occ Arabic Random, k=50.598 0.483 0.265 0.065 0.162 BGE-M3 cross, k=5 0.622 0.501 0.268 0.051 0.107 BGE-M3 same, k=1 0.613 0.512 0.286 0.085 0.170 BGE-M3 same, k=3 0.688 0.536 0.349 0.114 0.190 BGE-M3 same, k=5 0.669 0.550 0.376 0.128 0.200 English Random, k=50.682 0.262 0.134 0.094 0.105 BGE-M3 cross, k=5 0.628 0.238 0.140 0.115 0.106 BGE-M3 same, k=1 0.695 0.286 0.151 0.109 0.105 BGE-M3 same, k=3 0.655 0.276 0.158 0.125 0.119 BGE-M3 same, k=5 0.632 0.281 0.172 0.142 0.122 Table 14: In-context learning on the full test sets; same/cross is the demonstration-pool language. G Full Transfer Results Table 15 gives the absolute values behind Fig- ure 5 for both test languages, including every cross- lingual cell: rows named by one language are fine- tuned on that language only, so their columns under the other language measure cross-lingual transfer. H Complete Baseline Results Table 16 extends the binary comparison of §5.1 to all five tasks. Three notes. First, the proprietary models answer one multi-task prompt that produces Algorithm 2 Output parsing Span-tag 1:for each<span type="â">c</span>, left to right do 2:Ësâ untagged output characters consumed so far 3:if sentence[Ës : Ës+|c|] = cthen accept (Ës, Ës+|c|,â) 4: else accept the unused occurrence ofcclos- est to Ës, if any 5:end if 6: end for Span-occ 7: recover the JSON array (strip fences, repair commas, close truncation) 8: for each object (w,â,k) do 9: locate thek-th occurrence ofw: exact match, then Arabic-normalized,thencase- insensitive 10: end for both parsers drop unmatched, zero-length, and duplicate spans all five outputs at once, while the open models answer one prompt per task; the two Qwen2.5- 7B multi-task rows quantify this protocol differ- ence, lifting its Arabic binary score from 0.439 to 0.642, within 0.004 of GPT-5 under the identi- cal prompt. Second, the majority baseline exceeds every training-free system on Arabic coarse and technique classification (0.558 and 0.466), because one category and one technique dominate the Ara- bic label distribution; scores on these two tasks should be read against that floor rather than against zero. Third, classification heads stay level with MT-SFT on Arabic and lead on English technique classification, and the encoders stay competitive on Arabic classification (0.654 coarse) while dropping sharply on English; both families produce no expla- nations, and neither produces occurrence-indexed spans. Random baselines average five seeds; the single span list of the multi-task prompt is scored under the occurrence protocol (hence span-tag â), and encoder spans come from token-level tagging under the same overlap-adjusted metric. We in- clude previously reported results only when they use directly comparable data splits and evaluation metrics. 14 Arabic testEnglish test TrainingBinCoaTechS-tagS-occBinCoaTechS-tagS-occ none (zero-shot)0.439 0.437 0.238 0.058 0.093 0.683 0.290 0.125 0.087 0.108 AR, all five tasks0.767 0.666 0.560 0.408 0.364 0.607 0.361 0.272 0.198 0.177 EN, all five tasks0.400 0.061 0.047 0.004 0.041 0.712 0.355 0.268 0.139 0.203 AR+EN, all five tasks 0.763 0.682 0.575 0.411 0.362 0.735 0.410 0.272 0.189 0.241 AR, binary only0.754 0.564 0.254 0.177 0.203 0.638 0.259 0.084 0.108 0.106 EN, binary only0.544 0.514 0.044 0.043 0.059 0.711 0.272 0.051 0.069 0.059 AR, classification (3)0.759 0.667 0.566 0.207 0.216 0.608 0.350 0.252 0.110 0.112 EN, classification (3)0.500 0.097 0.037 0.053 0.055 0.702 0.276 0.182 0.097 0.090 AR, spans only0.485 0.515 0.222 0.421 0.374 0.649 0.366 0.121 0.204 0.174 EN, spans only0.313 0.112 0.095 0.019 0.066 0.639 0.323 0.153 0.173 0.255 Table 15: Every training regime evaluated on every task in both languages; best per column in bold. I Explanation Quality We score generated explanations against the refer- ence explanations of the binary cell (Table 17), with AraBERT-v2 as the BERTScore backbone for Ara- bic and BERT-base for English. Raw BERTScore is therefore not comparable across languages: pair- ing each reference with a random same-language reference already scores 0.58 on Arabic and 0.63 on English. The near-floor Arabic zero-shot score has a simple cause: the zero-shot model answers in English for every Arabic input (100% of test items), and such cross-language pairs fall below even the random same-language floor. Retrieved demonstrations largely fix the output language (2% remain in English) and fine-tuning removes the is- sue entirely. The fine-tuned row uses the model trained on each languageâs own data; the bilingual MT-SFT matches it within 0.002 on every measure. Against the floors, the fine-tuned models clear their languageâs baseline by the same margin in both, +0.09Arabic and+0.09English, while the seem- ingly strong English zero-shot score is only+0.03 above chance pairing. Reference-based measures do not assess whether an explanation reflects the modelâs own decision process, which we leave to future work. J Long-Tailed Techniques Technique performance follows the frequency of the label (Table 18). MT-SFT handles the two most frequent techniques well in Arabic, degrades on mid-frequency ones, and predicts almost none of the rare tail: of the 14 Arabic techniques with fewer than 20 test occurrences, 11 score zero, as do 10 of the 11 such techniques in English. Improving the tail is the clearest remaining target for this bench- mark. K Distillation This appendix details the two distillation routes of Section 5.5 and reports every trained configuration. In brief, off-policy teacher imitation loses to direct references; OPSD stalls because hard clipping re- moves gradients on decision tokens; saturating clip- ping repairs it on PROBEL; and replication on the original benchmarks confirms that the published clip remains effective there. Off-policy route. We generate chain-of-thought traces for the full training set with GPT-5, condi- tioned on the gold annotations so the traces ratio- nalize the correct answer (Hsieh et al., 2023), and fine-tune the student to imitate them (the API ex- poses text, not distributions). The best bilingual model reaches 0.727 Arabic and 0.668 English bi- nary macro-F 1 (Table 5), below the no-reasoning MT-SFT at 0.763 and 0.735, and 0.441 against 0.505 on the four tasks both output formats support: a stronger teacherâs reasoning, imitated off-policy, loses to plain references (Zhao et al., 2026). On-policy route (OPSD). OPSD (Zhao et al., 2026) uses one set of weights in two roles (Fig- ure 9): the student sees the task promptxand sam- ples a rollout, while the teacher, the same model frozen at initialization, additionally sees the ver- ified gold blockg(labels, span quotes, and the reference explanation). Training pulls the student toward the teacherâs better-informed next-token dis- tributions via the clipped divergence of Equation 1, computed over the full vocabulary at every rollout position; gradients flow only through the student (Agarwal et al., 2024). The clipÏexists because stylistic tokens occasionally carry huge divergence; the released configuration fixesÏ=0.06, which the original work reports as untuned. We adopt the 15 Arabic testEnglish test SystemBinCoaTechS-tag S-occBinCoaTechS-tag S-occ No learning Majority label0.380 0.558 0.466 0.000 0.000 0.419 0.196 0.162 0.000 0.000 Random, fair coin0.499 0.244 0.093 0.000 0.000 0.470 0.101 0.031 0.000 0.000 Whole sentence as spanâ0.082 0.082â0.038 0.038 Zero-shot, open weights (one prompt per task) Fanar-2-27B0.409 0.513 0.320 0.081 0.103 0.431 0.290 0.203 0.116 0.121 Llama-3.1-8B0.672 0.482 0.274 0.103 0.097 0.437 0.218 0.098 0.074 0.080 Qwen2.5-7B0.439 0.437 0.238 0.058 0.093 0.683 0.290 0.125 0.087 0.108 Qwen3-VL-8B0.433 0.448 0.196 0.021 0.029 0.669 0.281 0.149 0.046 0.075 Qwen3-VL-8B (think)0.577 0.464 0.239 0.138 0.137 0.625 0.278 0.166 0.156 0.145 Gemma-4-E4B0.442 0.391 0.343 0.080 0.117 0.671 0.280 0.187 0.138 0.150 Zero-shot, proprietary (one multi-task prompt, single span list) GPT-50.595 0.420 0.309â0.184 0.663 0.374 0.276â0.196 GPT-5 (CoT)0.646 0.467 0.358â0.218 0.657 0.356 0.252â0.180 Gemini-3.1-Pro0.553 0.392 0.297â0.200 0.661 0.369 0.284â0.218 Gemini-3.1-Pro (CoT)0.505 0.305 0.218â0.148 0.673 0.364 0.282â0.213 Protocol bridge: open model under the identical multi-task prompt Qwen2.5-7B (multi-task)0.516 0.237 0.103â0.035 0.616 0.241 0.121â0.072 Qwen2.5-7B (multi-task, CoT) 0.642 0.248 0.188â0.058 0.598 0.218 0.150â0.084 Fine-tuned discriminative (one model per task and language) Llama-3.1-8B + head0.778 0.683 0.582n/an/a0.732 0.449 0.392n/an/a Qwen2.5-7B + head0.765 0.690 0.581n/an/a0.741 0.420 0.353n/an/a AraBERT-v2 / BERT-base0.746 0.654 0.529 0.249n/a0.719 0.402 0.293 0.227n/a MT-SFT (ours)0.763 0.682 0.575 0.411 0.362 0.735 0.410 0.272 0.189 0.241 Table 16: All baselines on all five tasks (binary macro-F 1 ; coarse and technique micro-F 1 ; spans overlap-adjusted micro-F 1 ). BERTScoreBLEUMETEOR SystemARENARENAREN Zero-shot0.381 0.657 0.002 0.035 0.024 0.212 ICL, k=50.626 0.672 0.057 0.050 0.205 0.254 SFT (same lang.) 0.663 0.718 0.102 0.103 0.254 0.327 Table 17: Explanation quality against the reference ex- planations. ArabicEnglish TechniqueSup.F 1 Sup.F 1 Loaded Language6750.7984410.440 Name CallingâLabeling2060.5622910.291 ExaggerationâMinimisation1590.2721060.053 Questioning the Reputation990.521730.115 Causal Oversimplification580.033400.000 Doubt460.0801190.135 Appeal to FearâPrejudice190.080840.121 Tail (sup. < 20)14 tech. 0.036 11 tech. 0.009 Table 18: Per-technique F 1 of MT-SFT with test support. The last row averages the techniques with fewer than 20 test occurrences. authorsâ released implementation and recipe (Ta- ble 12), changing only the data interface. Why the hard clip can delete the signal. The clip acts per vocabulary element, and â min(â,Ï)/ââ = 1[â < Ï]almost everywhere: an element above threshold is not attenuated, it is removed from the gradient (Schulman et al., 2017). When the student emits a wrong label to- ken, the teacher concentrates its mass on the single correct entry, and that one element carries nearly the whole per-token divergence: label-token di- vergence reaches 5.5 nats at the 90th percentile, about90Ăthe published threshold. The saturating clip of Equation 2 (soft in tables, versus hard for min(â,Ï)) grows only logarithmically on such out- liers, but its gradientf âČ Ï (â) = 1/(1 + â/Ï)never vanishes. Reading the grid. Table 19 lists all 21 trained OPSD configurations. For Qwen2.5 rows, think/no- think are prompt variants; for Qwen3 rows, sX/tY are the chat templateâs native thinking flags for student and teacher (default sOFF/tON); label-only privilege drops the reference explanation; LoRA rank is r=64 unless noted. 16 Figure 9: Privileged self-distillation on PROBEL. Top: one LLM in two roles: a student given only the news textx, and the same weights frozen with the gold annota- tiony â in context. The teacher scores the studentâs own rollout, and only the student is updated. Bottom: dis- agreement concentrates on the wrong label token (the depicted token carries 5.6 nats,â90Ăthe published clipÏ = 0.06), exactly where the hard clip is flat and contributes zero gradient, while our saturating clip con- tinues to provide a learning signal. The clipâs shape decides the outcome. At the published settings OPSD is flat everywhere: the best of seven Qwen2.5-7B configurations gains +0.032AVG over zero-shot, and it is one that removes the clip; every configuration that keeps the published clip stays within+0.001. On MT- SFT, the strongest initialization, the trained model agrees with its own starting point on 99.1% of Arabic binary test predictions (p=1.0; Table 21). The saturating clip is the only intervention that repairs the model consistently, and the effect repli- cates across three seeds: 0.314±0.003 AVG against 0.296±0.001 for theÏ-matched hard clip, driven by repairing Qwen3-8Bâs collapsed English bi- nary head from 0.449 to 0.692±0.001 macro-F 1 (p<0.001), with no damage on MT-SFT. The re- pair needs both the gradient-preserving shape and an adequate threshold: removing the clip entirely does not fix the English head (0.460), the saturat- ing shape at the publishedÏ=0.06does not either (0.445), the repair holds acrossÏ â0.1, 0.2, 0.5, and atÏ=1, where the loss approaches the un- clipped objective, it vanishes again (0.449). Across runs, the saturating clip repaired the head in all five runs (Ï â 0.1, 0.2, 0.5), whereas the hard clip left it collapsed in five of six; oneÏ=0.06seed partially escaped (0.668). InitConfigurationBinAVG Qwen2.5 (ZS 0.561/0.256) think, hard Ï=.060.519 0.253 think, no clip0.514 0.245 no-think, hard Ï=.060.574 0.257 no-think, no clip0.556 0.277 no-think, no clip, r=16 0.623 0.288 think, no clip, r=160.465 0.232 label-only privilege0.458 0.246 Qwen3-8B (ZS 0.490/0.295) sOFF/tON, hard Ï=.06 0.499 0.298 sON/tON, hard Ï=.060.521 0.302 sOFF/tOFF, hard Ï=.06 0.501 0.296 hard Ï=.06, top-200.508 0.297 sOFF/tON, hard Ï=.50.485 0.295 sOFF/tON, soft Ï=.10.586 0.317 sOFF/tON, soft Ï=.20.605 0.323 sOFF/tON, soft Ï=.50.588 0.314 sOFF/tON, soft Ï=10.494 0.296 soft Ï=.5, 450 steps0.579 0.307 MT-SFT (0.749/0.464) hard Ï=.060.748 0.462 hard Ï=.50.742 0.429 soft Ï=.50.751 0.464 soft Ï=.5, 450 steps0.741 0.449 Table 19: All 21 trained OPSD configurations (mean of Arabic and English; each blockâs initialization in parentheses). The premise holds, and the signal lives in the la- bel.Could OPSD fail because the privileged con- text does not help the teacher? No. The gold block lifts label accuracy (exact match, 300 training- pool items over the three classification tasks) from 42.0% to 93.7% for the base Qwen2.5-7B and from 70.7% to 94.3% for MT-SFT (Table 20). Label lines alone match or exceed the full block (98.3% and 92.7%), so the label itself is the carrier, and a mismatched gold block collapses the base teacher to 20.3%, repeating the asserted wrong label on 73% of items. The teacherâs distributions carry far more task information than the studentâs; whatever fails, it is not an absent signal. The signal is tail-concentrated, and the clip deletes the tail. We compute the per-position divergence on student rollouts and bucket tokens 17 10 â3 10 â2 10 â1 10 0 10 1 clip threshold Ï (nats) 0 20 40 60 80 100 KL mass retained S ( Ï ) (%) Ï =0.06 Ï =0.5 Qwen3 sOFF/tON, label tokens Qwen3 sOFF/tON, all tokens Qwen2.5 no-think, label tokens Qwen2.5 no-think, all tokens 10 â4 10 â3 10 â2 10 â1 10 0 10 1 10 2 per-element divergence â (nats) 0.0 0.2 0.4 0.6 0.8 1.0 gradient factor â f /â â label-token p90 hard, Ï=0.06 hard, Ï=0.5 saturating, Ï=0.5 Figure 10: Signal survival under the clip. Left: share of divergence a hard clip keeps,S(Ï), for label tokens (solid) and all tokens (dashed). Right: each clipâs gradient factor over the empirical label-token divergence (gray histogram); hard clips drop to zero at their thresholds. At the publishedÏ=0.06, only 1.8% of the total divergence mass and 0.8% of the label-token mass survive. Teacher contextQwen2.5-7BMT-SFT none (= student)42.070.7 full gold block93.794.3 label lines only98.392.7 explanation only71.083.0 mismatched block20.353.0 copies the mismatched label73.038.7 Table 20: Teacher label accuracy (%) under different privileged contexts, on 300 training-pool items; the italic row is how often the teacher repeats a wrong as- serted label. into label, quoted span, style, other. The median is tiny, 0.001 to 0.05 nats: the privileged teacher agrees with the student almost everywhere, and the task signal concentrates in a thin tail on de- cision tokens, whose 90th-percentile divergence reaches 5.5 nats on Qwen3-8B. Figure 10 quan- tifies the consequence with the survival measure S(Ï) = P min(â + ,Ï)/ P â + : atÏ=0.06the hard clip retains 1.8% of this modelâs total divergence mass and 0.8% of its label-token mass; evenÏ=0.5 retains only 10.2% and 4.7%. Survival is below 1% in every content bucket, and the heaviest tails sit on the two most frequent techniques, so no reweight- ing across techniques or classes can rescue the hard clip. The freeze is measurable end to end: the adapters reach a maximum entry of4.4Ă10 â4 after 150 steps, development scores stay flat throughout training (checkpoint selection:0.5Ăbinary accu- racy+ 0.25Ăcoarse+ 0.25Ăfine-grained micro- F 1 on the held-out 360-item development subset), and trained students are 98.1 to 99.1% prediction- identical to their initializations. The mechanism generalizes, and the clip choice follows the domain. On HateXplain (Mathew et al., 2021), saturating clipping outperforms hard clipping at both thresholds (Figure 11): 0.640 vs. 0.628 at the publishedÏand 0.687 vs. 0.666 at Ï=0.5(untrained: 0.604). On GSM8K (Cobbe et al., 2021), the published hard clip degrades accuracy (0.918 to 0.900), while saturating clip- ping reaches 0.928 and both shapes reach 0.940 atÏ=0.5. On Llama-3.1-8B, saturating clipping also improves AVG by+0.060over hard clipping (single run). The oracle also bounds what repair can achieve: extended runs plateau at+0.02AVG over the base model, far below MT-SFT, and on top of MT-SFT the method is safe but not additive. Privileged self-distillation, even repaired, is a tar- geted correction tool in this regime, not a substitute for direct supervision (HĂŒbotter et al., 2026; Kaur et al., 2026). Replication on the original benchmarks. We replicate the published setup with the authorsâ code and data, Qwen3-1.7B, and evaluation protocol (Avg@12 on AIME24, AIME25, and HMMT25; best checkpoint within 100 steps). The recipe im- proves our base model from 34.2 to 41.9, matching the published gain (37.1 to 43.4); absolute scores are uniformly lower, including zero-shot, so we compare changes relative to our baseline. Remov- ing the clip reproduces the published degradation (36.4 at best), while saturating clipping reaches 39.8, below hard clipping (Figure 12). The gap is stable across three seeds (41.5±0.6 vs. 39.6±0.6), and loweringÏdoes not close it. The divergence profiles explain the contrast (Figure 13): on math- 18 =0.05 (published) =0.5 34 36 38 40 42 Avg@12 untrained AIME/HMMT (theirs) Qwen3-1.7B =0.06 (published) =0.5 0.90 0.91 0.92 0.93 0.94 accuracy untrained GSM8K Qwen3-8B =0.06 (published) =0.5 0.60 0.62 0.64 0.66 0.68 0.70 macro-F1 untrained HateXplain Qwen2.5-7B =0.06 (published) =0.5 0.5 0.6 0.7 macro-F1 untrained ProBel EN binary Qwen3-8B hard clipsaturating clipuntrained model Figure 11: Hard and saturating clips at the publishedÏ(left) andÏ=0.5(right); each panel uses its own metric, and dotted lines mark the untrained model. Points show medians when several seeds exist. On the original OPSD benchmark, hard clipping is better; on GSM8K, a larger threshold suffices for either shape; on HateXplain, saturating clipping is better at both thresholds; on PROBEL, hard clipping remains near the untrained floor (one of three seeds at the published Ï partially escaped; see text), while saturating clipping reaches 0.69 at Ï=0.5. ematics, the above-Ïtail is dominated by stylistic tokens (81% exceedÏ), so restoring their gradi- ents hurts; on PROBEL, it is dominated by deci- sion tokens (48% aboveÏ, mean divergence68Ï), so hard clipping removes task supervision. Thus, the published clip is appropriate when the tail is mostly stylistic, whereas a gradient-preserving clip is needed when it carries the task signal. 20406080100120140 Training step 34 36 38 40 42 44 Avg@12 (3-benchmark mean) published base published +OPSD hard clip =0.05 (3 seeds) saturating clip =0.05 (3 seeds) no clip Figure 12: Replication of OPSD on the original bench- marks (Qwen3-1.7B, authorsâ code and data). Bands show three seeds; reference lines mark the published base and best results. L Significance Tests Since all systems are evaluated on identical test items, paired significance testing is possible wher- ever prediction files exist. Table 21 reports exact (binomial) McNemar tests for central binary com- parisons; âbest promptedâ is Llama-3.1-8B on Ara- bic and Qwen2.5-7B zero-shot on English, and the OPSD rows use Qwen3-8B with the saturating clip and the MT-SFT initialization with the hard clip. Every fine-tuning gain is significant atp<0.001, and the open-versus-GPT-5 gap atp<0.05on Ara- bic andp<0.001on English. The final row quan- tifies the OPSD freeze more sharply than any ag- gregate metric: after 150 steps of training at the published settings on top of MT-SFT, exactly 6 predictions improved and 6 degraded out of 1,326. Comparison (binary task)b c p Llama-3.1-8B vs. GPT-5 CoT (AR)251 197 0.012 Qwen2.5-7B vs. GPT-5 CoT (EN)746 417 <0.001 MT-SFT vs. best prompted (AR)242 116 <0.001 MT-SFT vs. best prompted (EN)396 225 <0.001 MT-SFT vs. zero-shot (AR)579 179 <0.001 ICL retrieved vs. random, k=3 (AR)298 151 <0.001 Extended vs. original EN training (EN) 368 250 <0.001 OPSD saturating vs. zero-shot (EN)311 180 <0.001 OPSD hard Ï=.06 vs. MT-SFT (AR)66 1.0 Table 21: Exact McNemar tests on paired binary predic- tions (b/c: items only the first/second system classifies correctly). The last row shows the OPSD freeze: only 6+6of 1,326 predictions change, so the trained model is statistically indistinguishable from its initialization. M Data Release Our proposed dataset will be released under the C BY-NC-SA 4.0 (Creative Commons Attribution-NonCommercial-ShareAlike 4.0 Inter- national) License:https://creativecommons.org/ licenses/by-nc-sa/4.0/. 19 64202 log10 per-token KL(p T p S ) 0.0 0.2 0.4 0.6 Density =0.05 zero gradient above (hard clip) Mathematics (their training data, Qwen3-1.7B) task tokens 15% above style tokens 81% above 64202 log10 per-token KL(p T p S ) =0.06 zero gradient above (hard clip) ProBel (our training data, Qwen3-8B) decision tokens 48% above style tokens 57% above Figure 13: Per-token forward KL between the privileged teacher and the student at initialization, on each domainâs OPSD training data (mathematics: Qwen3-1.7B; PROBEL: Qwen3-8B). The shaded region above the publishedÏ receives no gradient under the hard clip. The above-Ïtail is mainly stylistic on mathematics but consists of decision tokens on PROBEL. 20