Paper deep dive
Polarization Detection: A Hybrid Approach with AfroXLMR-Social and DeBERTa for Low- and High-Resource Settings
Muhammad Abdullahi Said
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 95%
Last extracted: 7/14/2026, 6:30:11 AM
Summary
This paper presents a hybrid modeling strategy for the POLAR Shared Task 2026 to detect and characterize online polarization in English and Hausa. The approach uses DeBERTa for English binary detection and AfroXLMR-Social for fine-grained subtasks (Types and Manifestations) across both languages. To address computational constraints and data scarcity, the authors employ Low-Rank Adaptation (LoRA) for efficient fine-tuning and nlpaug for textual data augmentation. The system achieves competitive F1-scores across all subtasks, demonstrating that task-specific model selection and domain-adaptive pre-training are critical for robust polarization detection, especially in low-resource settings.
Entities (12)
Relation Signals (9)
POLAR Shared Task 2026 → focuseson → English
confidence 98% · focusing on the detection and characterization of polarized discourse in English and Hausa
POLAR Shared Task 2026 → focuseson → Hausa
confidence 98% · focusing on the detection and characterization of polarized discourse in English and Hausa
Muhammad Abdullahi Said → affiliatedwith → African Institute for Mathematical Sciences
confidence 97% · Muhammad Abdullahi Said African Institute for Mathematical Sciences (AIMS)
DeBERTa → usedfor → Subtask 1: Detection
confidence 97% · for English binary detection, we leverage the monolingual strength of DeBERTa
AfroXLMR-Social → usedfor → Subtask 3: Manifestation
confidence 96% · for Hausa and all fine-grained subtasks (Types and Manifestations), we utilize AfroXLMR-Social
AfroXLMR-Social → usedfor → Subtask 2: Type
confidence 96% · for Hausa and all fine-grained subtasks (Types and Manifestations), we utilize AfroXLMR-Social
nlpaug → usedfor → Data Augmentation
confidence 95% · We utilized nlpaug (Ma, 2019) to robustly handle class imbalance
Low-Rank Adaptation → appliedto → AfroXLMR-Social
confidence 94% · To adapt these models efficiently, we freeze the pre-trained backbone and inject trainable low-rank matrices
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:The rapid proliferation of online polarization threatens social cohesion, necessitating robust automated detection systems that operate effectively across diverse linguistic contexts. This paper presents our system description for the POLAR Shared Task 2026, focusing on the detection and characterization of polarized discourse in English and Hausa. We propose a hybrid modeling strategy: for English binary detection, we leverage the monolingual strength of \textbf{DeBERTa}, while for Hausa and all fine-grained subtasks (Types and Manifestations), we utilize \textbf{AfroXLMR-Social}. This domain-adapted multilingual model proved critical for capturing the nuances of polarization in social media text. To further address computational constraints and data scarcity, we implement Low-Rank Adaptation (LoRA) and textual data augmentation via \texttt{nlpaug}. We report competitive results across all three subtasks, demonstrating that model selection tailored to specific subtask requirements yields the best balance of performance.
Tags
Links
- Source: https://arxiv.org/abs/2607.10312v1
- Canonical: https://arxiv.org/abs/2607.10312v1
Trouble viewing inline? Open PDF directly →
Full Text
12,187 characters extracted from source content.
Expand or collapse full text
Polarization Detection: A Hybrid Approach with AfroXLMR-Social and DeBERTa for Low- and High-Resource Settings Muhammad Abdullahi Said African Institute for Mathematical Sciences (AIMS) mohdasaid@aims.ac.za Abstract The rapid proliferation of online polarization threatens social cohesion, necessitating robust automated detection systems that operate effectively across diverse linguistic contexts. This paper presents our system description for the POLAR Shared Task 2026, focusing on the detection and characterization of polarized discourse in English and Hausa. We propose a hybrid modeling strategy: for English binary detection, we leverage the monolingual strength of DeBERTa, while for Hausa and all fine-grained subtasks (Types and Manifestations), we utilize AfroXLMR-Social. This domain-adapted multilingual model proved critical for capturing the nuances of polarization in social media text. To further address computational constraints and data scarcity, we implement Low-Rank Adaptation (LoRA) and textual data augmentation via nlpaug. We report competitive results across all three subtasks, demonstrating that model selection tailored to specific subtask requirements yields the best balance of performance. https://github.com/mohdasaid/NLP-LLM/blob/main/Polarization.ipynb Polarization Detection: A Hybrid Approach with AfroXLMR-Social and DeBERTa for Low- and High-Resource Settings Muhammad Abdullahi Said African Institute for Mathematical Sciences (AIMS) mohdasaid@aims.ac.za 1 Introduction Online polarization defined as the sharp division of public opinion into opposing factions characterized by hostility and lack of empathy has become a pervasive issue in digital discourse. The POLAR Shared Task 2026 aims to benchmark Natural Language Processing (NLP) systems on their ability to detect this phenomenon across a wide range of languages and cultures (Naseem et al., 2025). Detecting polarization extends beyond traditional sentiment analysis. It requires capturing subtle rhetorical strategies such as "us vs. them" framing, invalidation, and dehumanization. This complexity is exacerbated in low-resource languages like Hausa, where labeled data is scarce and standard multilingual models often lack sufficient exposure to dialectal variations. In this work, we propose a strategic, hybrid training pipeline. Our key contributions are: 1. Task-Specific Model Selection: We observe that while English benefits from specialized monolingual models for binary detection (Subtask 1), the fine-grained tasks (Subtasks 2 & 3) in both languages require the robust social-media understanding provided by AfroXLMR-Social(Belay et al., 2025). 2. Domain-Adaptive Efficacy: We demonstrate that AfroXLMR-Social, a model continued-pre-trained on African social media content, is indispensable for the Hausa subtasks, significantly outperforming generic multilingual baselines. 3. Parameter Efficiency & Augmentation: We employ LoRA (Low-Rank Adaptation) (Hu et al., 2021) to fine-tune effectively on consumer hardware and use nlpaug (Ma, 2019) to generate synthetic examples, addressing severe class imbalance in the manifestation labels. 2 Related Work 2.1 Polarization Detection Early computational approaches to polarization relied on network topology, inferring ideological stance from retweet or follower graphs. However, content-based approaches using Transformers have recently gained traction. Studies have shown that models like BERT and RoBERTa can detect stance and toxicity, which are proxies for polarization. The POLAR benchmark advances this by explicitly taxonomizing polarization into types (political, religious) and rhetorical manifestations. 2.2 Multilingual Models for African NLP Massively multilingual models like mBERT and XLM-R often underperform on African languages due to the "curse of multilinguality," where capacity is diluted across too many languages. AfroXLMR-Social (Belay et al., 2025) builds upon the success of the original AfroXLMR (Alabi et al., 2022) by incorporating "Domain-Adaptive Pre-training" (DAPT) on social media corpora. This adaptation is crucial for Hausa, as it aligns the model’s internal representations with the noisy, informal, and code-mixed text styles found in the POLAR dataset. 2.3 Efficient Fine-Tuning As model sizes grow, full fine-tuning becomes prohibitively expensive. Parameter-Efficient Fine-Tuning (PEFT) methods like Adapters and LoRA have emerged as standard solutions. LoRA, in particular, has been shown to match or exceed full fine-tuning performance in low-data regimes by reducing the risk of catastrophic forgetting. By freezing the pre-trained backbone, we maintain the linguistic knowledge acquired during the extensive DAPT phase. 3 Task Description We address three hierarchical subtasks provided by the organizers: 1. Subtask 1 (Detection): A binary classification task to determine if a text x contains polarized content (y∈0,1y∈\0,1\. 2. Subtask 2 (Type): A multi-label classification task identifying the topic of polarization. Labels include Political, Religious, Racial/Ethnic, Gender/Sexual Identity, and Other. 3. Subtask 3 (Manifestation): A multi-label task identifying rhetorical devices used to express polarization. Labels include Vilification, Stereotyping, Dehumanization, Extreme Language, Lack of Empathy, and Invalidation. 4 Methodology 4.1 Model Architecture Strategy 4.1.1 English Subtask 1: DeBERTa For the binary detection task in English, we utilized DeBERTa-v3-base. DeBERTa improves upon BERT and RoBERTa by using disentangled attention and an enhanced mask decoder, making it highly effective for subtle classification tasks in high-resource languages where standard grammar prevails. 4.1.2 Hausa & Subtasks 2/3: AfroXLMR-Social For all Hausa tasks and the fine-grained English tasks (Types and Manifestations), we employed AfroXLMR-Social. The complexity of Subtasks 2 and 3 requires a model that understands the social context of language slang, hashtags, and informal phrasing which is the core strength of this domain-adapted checkpoint. 4.2 Low-Rank Adaptation (LoRA) To adapt these models efficiently, we freeze the pre-trained backbone and inject trainable low-rank matrices (A,BA,B) into the Query (Q) and Value (V) attention projections. h=W0x+αrBAxh=W_0x+ αrBAx (1) We used rank r=8r=8 and scaling factor α=16α=16. This approach allowed us to fine-tune distinct models for each subtask without exceeding GPU memory limits. 4.3 Data Augmentation We utilized nlpaug (Ma, 2019) to robustly handle class imbalance. • Synonym Replacement: We replaced up to 10% of words with synonyms from WordNet. • Random Insertion: We inserted contextually relevant words to vary sentence structure. This was crucial for Subtask 3, where classes like Dehumanization had very few positive examples. 4.4 Training Strategy 4.4.1 Loss Function For Subtask 1, we minimize Binary Cross-Entropy (BCE). For Subtasks 2 and 3, we use BCEWithLogitsLoss, which applies a sigmoid activation to each class logit independently: ℒ=−1C∑c=1C[yclog(σ(zc))+(1−yc)log(1−σ(zc))]L=- 1C _c=1^C[y_c (σ(z_c))+(1-y_c) (1-σ(z_c))] (2) 4.4.2 Dynamic Learning Rate We implemented a linear warmup for the first 10% of steps, followed by a cosine decay. This schedule proved essential for stabilizing the training of the DeBERTa model, which can be sensitive to initialization. 5 Experimental Setup 5.1 Data and Preprocessing The data was provided in CSV format. We cleaned the text by removing URL artifacts but retained emojis and hashtags, as they are strong indicators of sentiment in social media. We used the XLM-R tokenizer with a maximum sequence length of 128. 5.2 K-Fold Cross-Validation We employed 5-Fold Cross-Validation. For each language, the training data was split into 5 stratified folds. We trained 5 independent models, each using 4 folds for training and 1 for validation. The final test predictions were generated using soft voting from the ensemble of 5 models. 5.3 Hyperparameters Table 1 details the configuration used. Parameter Value Batch Size 16 Learning Rate 2e−52e^-5 Epochs 5 LoRA Rank (r) 8 LoRA Alpha 16 Dropout 0.1 Optimizer AdamW Weight Decay 0.01 Table 1: Hyperparameters for both English and Hausa. 6 Results 6.1 Subtask 1: Polarization Detection Table 2 presents the detection results. The switch to DeBERTa for English yielded a noticeable improvement in F1-Score over our initial multilingual baselines. For Hausa, AfroXLMR-Social remained superior. Lang Model F1-Score English DeBERTa-v3 0.7917 Hausa AfroXLMR-Social 0.8133 Table 2: Subtask 1 Results (Validation Average). 6.2 Subtasks 2 & 3: Fine-grained Tasks Tables 3 and 4 summarize the multi-label results. AfroXLMR-Social performed exceptionally well here for both languages. Its pre-training on social media data likely allowed it to better recognize the rhetorical "manifestations" (Subtask 3) which are often signaled by informal social cues rather than formal vocabulary. Language F1-Score English 0.3976 Hausa 0.3276 Table 3: Subtask 2 (Type) using AfroXLMR-Social. Language F1-Score English 0.4979 Hausa 0.2367 Table 4: Subtask 3 (Manifestation) using AfroXLMR-Social. 7 Discussion 7.1 The Dominance of AfroXLMR-Social A key finding of our experiments is the robustness of AfroXLMR-Social for the complex tasks (2 and 3). Even for English, where monolingual models usually dominate, AfroXLMR-Social provided competitive and stable results for determining polarization types and manifestations. We hypothesize this is because the "social" pre-training exposes the model to the exact kind of toxic and polarized discourse patterns that these subtasks aim to classify, regardless of the language. 7.2 Challenges with Augmentation While nlpaug improved recall for minority classes, it occasionally introduced semantic drift. For instance, replacing "regime" with "government" in a political post might subtly alter the polarized tone (negative to neutral). Future work should explore embedding-based augmentation (BERT-based insertion) to preserve semantic consistency better. 8 Conclusion Our system demonstrates that combining a domain-adapted multilingual backbone with efficient fine-tuning (LoRA) is a highly effective strategy for polarization detection in low-resource languages. References Alabi et al. (2022) Jesujoba O Alabi, David I Adelani, Marius Mosbach, and Dietrich Klakow. 2022. Adapting pre-trained language models to african languages via multilingual adaptive fine-tuning. In Proceedings of COLING, pages 4336–4349. Belay et al. (2025) Tadesse Destaw Belay, Israel Abebe Azime, and Ibrahim Said Ahmad. 2025. Afroxlmr-social: Adapting pre-trained language models for african languages social media text. arXiv preprint arXiv:2503.18247. Hu et al. (2021) Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685. Ma (2019) Edward Ma. 2019. Nlp augmentation. PyPI. Naseem et al. (2025) Usman Naseem, Juan Ren, Saba Anwar, and Sarah Kohail. 2025. Polar: A benchmark for multilingual, multicultural, and multi-event online polarization. arXiv preprint arXiv:2505.20624. Appendix A Appendix: Detailed Experimental Settings A.1 Hardware and Compute All models were trained on a single NVIDIA T4 GPU (16GB VRAM) using the Google Colab environment. The average training time per fold was approximately 20 minutes for Hausa and 15 minutes for English. A.2 Augmentation Details For the nlpaug implementation, we used the SynonymAug and RandomWordAug classes. • Synonym Augmentation: aug_p=0.1aug\_p=0.1 (probability of augmenting a token). • Stop Words: We utilized the NLTK stop word list to prevent the augmentation of structurally important words. A.3 Label Distribution The POLAR dataset exhibits significant class imbalance. In Subtask 3, categories like Dehumanization were significantly rarer than Vilification. The use of weighted loss functions was considered but ultimately the augmentation strategy proved more effective in preliminary trials.