Paper deep dive
AHA-Memes: A Fine-Grained Multimodal Benchmark for Understanding Hate in Arabic Memes
Mohamed Bayan Kmainasi, Ali Ezzat Shahroor, Abul Hasnat, Md. Rafiul Biswas, Wajdi Zaghouani, Firoj Alam
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 8/1/2026, 1:30:56 AM
Summary
The paper introduces AHA-Memes, the first large-scale fine-grained multimodal benchmark for understanding hate in Arabic memes. It addresses the gap in Arabic multimodal hate detection by providing 5K manually annotated memes with binary and fine-grained multi-label annotations, alongside ~66K silver-labeled memes. The study benchmarks various text-only, image-only, and multimodal models, highlighting challenges in culturally grounded hate detection.
Entities (26)
Relation Signals (26)
AHA-Memes → targets → Arabic
confidence 98% · We introduce AHA-Memes, which is, to our knowledge, the first large-scale Arabic hateful meme benchmark...
AHA-Memes → annotates → Hate Types
confidence 95% · The dataset includes 5K manually annotated memes using a taxonomy that captures hate types, i.e., attack strategies.
AHA-Memes → contains → 5K manually annotated memes
confidence 95% · The dataset includes 5K manually annotated memes using a taxonomy that captures hate types, i.e., attack strategies.
AHA-Memes → contains → ~66K silver-labeled memes
confidence 95% · We further provide ~66K silver-labeled memes to support future studies.
Hate Types → includes → Mocking
confidence 95% · hateful_type captures the attack strategy as a multi-label annotation over Mocking...
Hate Types → includes → Incitement
confidence 95% · hateful_type captures the attack strategy as a multi-label annotation over ... Incitement...
Hate Types → includes → Dehumanization
confidence 95% · hateful_type captures the attack strategy as a multi-label annotation over ... Dehumanization...
Hate Types → includes → Slurs
confidence 95% · hateful_type captures the attack strategy as a multi-label annotation over ... Slurs...
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Hateful memes are a growing form of multimodal online harm, where hostile intent is often conveyed through the joint interpretation of images, text, cultural references, and implicit targets. While hateful meme detection has advanced in high-resource languages, Arabic remains underexplored, with existing meme resources focusing mainly on propaganda or coarse harmful-content labels. We introduce AHA-Memes (Arabic HAteful Memes), which is, to our knowledge, the first large-scale Arabic hateful meme benchmark with fine-grained, multi-label annotations. The dataset includes 5K manually annotated memes using a taxonomy that captures hate types, i.e., attack strategies. We further provide ~66K silver-labeled memes to support future studies. We benchmark text-only, image-only, and late-fusion multimodal models, as well as few-shot in-context learning (ICL) and open- and closed-weight Vision-Language Models (VLMs) under zero-shot and fine-tuning settings. Our results establish strong baselines and highlight key challenges in culturally grounded Arabic hateful meme detection. We release the dataset, annotation guidelines, and evaluation scripts to support future research. WARNING: This paper contains examples that may be disturbing to readers.
Tags
Links
- Source: https://arxiv.org/abs/2607.27393v1
- Canonical: https://arxiv.org/abs/2607.27393v1
Trouble viewing inline? Open PDF directly →
Full Text
87,730 characters extracted from source content.
Expand or collapse full text
AHA-Memes: A Fine-Grained Multimodal Benchmark for Understanding Hate in Arabic Memes Warning: This paper contains examples that may be disturbing to readers. Mohamed Bayan Kmainasi1, Ali Ezzat Shahroor1, Abul Hasnat2, Md. Rafiul Biswas3, Wajdi Zaghouani4, Firoj Alam1 1Qatar Computing Research Institute, Qatar, 2APAVI.AI, France 3Hamad Bin Khalifa University, Qatar, 4Northwestern University in Qatar, Qatar mkmainasi, alsh34060, mbiswas, fialam@hbku.edu.qa mhasnat@gmail.com, wajdi.zaghouani@northwestern.edu Abstract Hateful memes are a growing form of multimodal online harm, where hostile intent is often conveyed through the joint interpretation of images, text, cultural references, and implicit targets. While hateful meme detection has advanced in high-resource languages, Arabic remains underexplored, with existing meme resources focusing mainly on propaganda or coarse harmful-content labels. We introduce AHA-Memes,111AHA-Memes: Arabic HAteful Memes which is, to our knowledge, the first large-scale Arabic hateful meme benchmark with fine-grained, multi-label annotations. The dataset includes 5K manually annotated memes using a taxonomy that captures hate types, i.e., attack strategies. We further provide ∼ 66K silver-labeled memes to support future studies. We benchmark text-only, image-only, and late-fusion multimodal models, as well as few-shot in-context learning (ICL) and open- and closed-weight Vision-Language Models (VLMs) under zero-shot and fine-tuning settings. Our results establish strong baselines and highlight key challenges in culturally grounded Arabic hateful meme detection. We release the dataset, annotation guidelines, and evaluation scripts to support future research. 222Project resources: GitHub — Hugging Face AHA-Memes: A Fine-Grained Multimodal Benchmark for Understanding Hate in Arabic Memes Warning: This paper contains examples that may be disturbing to readers. Mohamed Bayan Kmainasi1, Ali Ezzat Shahroor1, Abul Hasnat2, Md. Rafiul Biswas3, Wajdi Zaghouani4, Firoj Alam1 1Qatar Computing Research Institute, Qatar, 2APAVI.AI, France 3Hamad Bin Khalifa University, Qatar, 4Northwestern University in Qatar, Qatar mkmainasi, alsh34060, mbiswas, fialam@hbku.edu.qa mhasnat@gmail.com, wajdi.zaghouani@northwestern.edu 1 Introduction A smiling family photograph may appear harmless on its own, yet the same image can communicate hate when combined with a targeted caption. Likewise, a national flag paired with an apparently benign proverb may become sectarian or xenophobic depending on the communities, histories, and stereotypes that viewers bring to the meme. Memes therefore pose a distinctly multimodal and culturally grounded hate detection problem Shahroor et al. (2026). Their meaning is often produced through the joint interpretation of image, text, and shared background knowledge, rather than by either modality in isolation Sharma et al. (2022). This dependence on cultural context also complicates human judgment, reduces label consistency, and makes automated detection more challenging. In a cross-country study, Bui et al. (2025) found that annotators from five countries agreed on meme labels only 74% of the time, with agreement falling to 67% between U.S. and Indian annotators. Such disagreement among human annotators suggests that models trained mainly on culturally homogeneous and English-centric resources are likely to struggle when hate is expressed through culturally specific visual and linguistic cues. Figure 1 illustrates this with two examples from AHA-Memes: a doctored portrait that mocks a woman’s appearance (Hateful), and a television still repurposed as everyday sarcasm (Not-Hateful), where the hateful or benign reading emerges only from the image–text pairing. Figure 1: Examples from the AHA-Memes dataset across hateful and non-hateful categories. Despite growing interest in multimodal hate detection, existing meme benchmarks remain focused in English and often rely on binary or coarse labels Kiela et al. (2020). This limits their ability to capture who is targeted, what form of hate is expressed, and how hateful meaning is constructed through the interaction of image, text, and cultural context. Recent multilingual efforts have begun to broaden coverage, but available resources remain limited in scale and granularity. For example, Multi3Hate provides parallel multilingual annotations but only 300 samples per language Bui et al. (2025), while GuardHarMem introduces fine-grained meme hate labels only for English El-amrany et al. (2025). Arabic is especially underserved in this space. Although Arabic text-based hate-speech detection has advanced through shared tasks such as OSACT Mubarak et al. (2022), and Arabic meme research has studied propaganda in ArMeme Alam et al. (2024a); Kmainasi et al. (2025a), there is still no dedicated benchmark for fine-grained hate detection in Arabic memes. We address this gap with AHA-Memes, a manually annotated fine-grained multimodal dataset for Arabic hateful memes. The dataset includes hateful and non-hateful memes and provides multi-label annotations for hate types. Figure 2 gives an overview of the end-to-end construction pipeline. We conduct detailed experiments on broad set of systems, including text encoders, image encoders, multimodal fusion models, open-weight VLMs, and closed-weight VLMs and LLMs. Our experiments show that fine-tuned open VLMs perform best for binary detection, that embedded text carries much of the signal while visual information remains complementary, and that fine-grained hate categorisation remains challenging. We also evaluate retrieval-augmented few-shot in-context learning as a training-free alternative. Our contributions are as follows. • We introduce AHA-Memes, the first large-scale Arabic hateful-meme dataset, consisting of 5K manually annotated memes with binary hate labels and fine-grained multi-label hate types. In addition, we provide silver labels333By silver labels, we refer to labels generated by an LLM. For this task we used Gemini-3.1-Pro. for targeted protected categories and features such as dialect, emotion, and sentiment. • We develop an Arabic-contextual bilingual annotation scheme for binary hate labels, fine-grained hate types, and targeted protected categories, which can support future dataset development for Arabic hateful-meme analysis. • We provide a comprehensive benchmark across five model families, comparing text-only, image-only, multimodal fusion, open-weight VLM, and closed-weight VLM/LLM systems under zero-shot, few-shot, and fine-tuning settings. • We release an additional ∼ 66K Arabic meme corpus with silver labels for binary hate, fine-grained hate type, target category, dialect, emotion, and sentiment, enabling future work on large-scale Arabic multimodal harmful-content modelling. 2 Related Work Hateful Meme Detection and Resources. The Facebook Hateful Memes Challenge Kiela et al. (2020) established hateful-meme detection as a multimodal task that requires joint reasoning over image and text. Its use of benign confounders showed that neither modality alone is sufficient, motivating later work on cross-modal fusion, visual-to-text prompting, and LLM-based reasoning for meme interpretation Kumar and Nandakumar (2022); Cao et al. (2022); Lin et al. (2024). More recent studies further show that external knowledge, cross-modal alignment, and cultural grounding are important for reliable meme understanding Sharma et al. (2022); Ren et al. (2026); Kmainasi et al. (2025a). However, most research in this area remains English-centric, and recent evidence suggests that VLMs may inherit culturally narrow assumptions when applied to non-English memes Wang et al. (2026). Recent efforts have begun to address under-resourced languages, including Bengali, Tamil, Malayalam Shahroor et al. (2026). However, these resources remain fragmented in scale, language coverage, and annotation depth Das and Mukherjee (2023); Ponnusamy et al. (2024); Lu et al. (2024). In addition, most resources use binary or coarse labels, which obscure who is targeted, what type of hate is expressed, and how hate is rhetorically framed. Figure 2: Overview of the AHA-Memes construction pipeline. Arabic Resources. Arabic multimodal disinformation or harmful-content research has been driven mainly by text-based datasets and shared tasks. OSACT advanced offensive-language and hate-speech detection in Arabic social media Mubarak et al. (2022), while related shared tasks and datasets have addressed propaganda detection and explanation in Arabic and multilingual settings Hasanain et al. (2025). Recent multilingual and multimodal research on Arabic has addressed fact-checking Alam et al. (2023) and the understanding of news and social-media content Kmainasi et al. (2025b). Arabic multimodal social-media analysis has focused primarily on propaganda rather than hate. The ArMeme dataset Alam et al. (2024b) and ArAIEval shared tasks Hasanain et al. (2024b) introduced Arabic meme propaganda-technique detection, followed by work on coarse hate labels, explanation-enhanced modelling, and explainable Arabic-English propaganda detection Alam et al. (2024a); Kmainasi et al. (2025a); Alam et al. (2022). These resources are valuable for studying persuasion and propaganda, but they do not provide systematic fine-grained annotations for hateful memes, including hate, and fine-grained hate type, as presented in Table 1. We address this gap by developing AHA-Memes, a dedicated fine-grained benchmark for Arabic hateful-meme detection. Dataset Lang. Size Multi-label Task granularity Source ArMeme (Alam et al., 2024b) AR 5,725 No Propaganda status Social media ArAIEval (Hasanain et al., 2024b)† AR 3,062 No Binary propaganda ArMeme Propaganda-to-Hate (Alam et al., 2024a)† AR ∼ 3.1K Yes Hate + hate types ArAIEval MAHED Task 3 (Zaghouani et al., 2025)† AR 3,562 No Binary hate Social media MemeXplain (Kmainasi et al., 2025a)† AR/EN 5,725 AR + 10K EN No Label + explanations ArMeme + Hateful Memes AraHarMeme (El-Amrany et al., 2026) AR 5,313 No Three-way harmfulness Social media ArPoMeme (Zaghouani et al., 2026) AR ∼ 7.3K Yes Ideology and polarization Political memes AHA-MEMES (ours) AR 5K + 66K‡ Yes Hate + 8 hate / 3 non-hate types four platforms Table 1: Comparison of Arabic multimodal meme datasets. †The resource contains data derived from or overlapping with an earlier Arabic meme dataset. ‡AHA-Memes contains 5K human-annotated gold memes and approximately 66K auxiliary silver-labeled memes. In-Context Learning for Multimodality. In-context learning offers a training-free alternative to fine-tuning, however, its effectiveness depends strongly on which demonstrations are included in the prompt Brown et al. (2020); Abdelali et al. (2024). Similarity-based selection is a common strategy for selecting useful examples in text classification Liu et al. (2022). In multimodal learning, most ICL work has focused on captioning and visual question answering, often using retrieval from a single modality. It remains underexplored for multimodal classification, especially in culturally grounded and low-resource settings. We address this gap by selecting demonstrations through reciprocal-rank fusion of text and image similarities Cormack et al. (2009). 3 Dataset 3.1 Data Curation We sourced memes from Facebook, Instagram, Pinterest, and Twitter (X). For Facebook, Instagram, and Pinterest, we manually selected public groups and pages focused on public figures, politics, and social commentary, contexts in which hateful memes are likely to circulate. We then collected images from these public sources using a semi-manual procedure following prior work Alam et al. (2024b). For Twitter (X), we collected memes through keyword-based search. In Figure 2, we provide the data curation pipeline. 3.2 Filtering and OCR Text Duplicate Removal. Social-media collections are highly redundant as users often repost identical or slightly modified content. We therefore removed exact and near-duplicate images after initial data curation. For near-duplicate filtering, we extracted image embeddings using a model fine-tuned on social-media images Alam et al. (2020) following the approach discussed in Alam et al. (2024b). We then computed pairwise Euclidean distances between image embeddings across the full dataset. Image pairs with a distance of 3.63.6 or lower were treated as near-duplicates, and only one image from each duplicate cluster was retained. OCR Text. We extracted embedded text using EasyOCR,444https://github.com/JaidedAI/EasyOCR which we selected for its broad script coverage and scene-text detection capabilities Liao et al. (2022). Because our study focuses on multimodal image-text analysis, we discarded memes with no detectable embedded text. As a result, every retained meme contains both visual and textual modalities. 3.3 Image Selection for Annotation. Harmful, hateful, and misleading content represents a relatively small proportion of online content, however, its societal impact can be substantial, as reported in prior studies Sharma et al. (2022). To increase the proportion of relevant examples while preserving coverage of non-hateful memes, we used a weak pre-selection strategy before manual annotation. Specifically, we applied Gemma-3-12B to assign preliminary binary hateful labels to 71K memes. We then selected 5K memes for manual annotation from this pre-selected pool. This sampling strategy produced a balanced and representative annotation set, consisting of 1,850 hateful memes and 3,150 non-hateful memes. 3.4 Annotation Manual Annotation Tasks. We manually annotate each meme for binary hatefulness and, fine-grained hierarchical labels. The annotation scheme is organized as follows. • is_hateful is a binary judgment. Following established definitions Kiela et al. (2020), a meme is labeled as Hateful if it directly or indirectly attacks people on the basis of a protected characteristic, including ethnicity, race, nationality, immigration status, religion, caste, sex, gender identity, sexual orientation, disability, or disease. We make two boundary cases explicit. First, attacks against groups that themselves promote or perpetrate hate, such as terrorist organizations, are not treated as hate speech. Second, content that is offensive, rude, or insulting but does not target a protected category is labeled Not-Hateful. This separates hatefulness from generic offensiveness. • For memes labeled Hateful, hateful_type captures the attack strategy as a multi-label annotation over Mocking, Incitement, Dehumanization, Slurs, Contempt, Inferiority, and Exclusion. A meme may receive multiple hate-type labels (e.g., Slurs and Dehumanization). • For memes labeled Not-Hateful, non_hateful_type assigns one or more labels from Humor, Sarcasm, and Other. This hierarchical structure supports binary hate detection at the top level, enables fine-grained categorization for hateful content, and ensures that fine-grained labels are consistent with the binary decision by construction. We provide category definitions and examples in Appendix B. Annotation Team. Annotation was conducted by a third-party company. Three trained native Arabic-speaking annotators annotated the dataset using bilingual guidelines (See Appendix B) in English and Arabic, which we developed to capture dialectal and culturally specific cues Hasanain et al. (2024a). Before annotation, annotators were informed that the dataset may contain offensive, hateful, or otherwise harmful content. In addition, they were informed about the use of the data. We also conducted multiple rounds of training and guideline refinement to improve annotation consistency and quality. In total, AHA-Memes contains 5K manually annotated memes, with an annotation cost of ∼$4K 4K. Annotator Agreement We measured agreement with Cohen’s κ agreement on the annotated dataset. For each multi-label task, every subtype was treated as a separate binary label. Cohen’s κ was computed independently for each subtype and each of the three annotator pairs, then macro-averaged across subtypes and annotator pairs. Annotation agreement are 0.91, 0.75, and 0.67 for is_hateful, hateful_type, and non_hateful_type, respectively, indicating substantial to near-perfect agreement for all annotation tasks Landis and Koch (1977). 3.5 Statistics and Data Splits Distribution. AHA-Memes is moderately imbalanced, with 1,850 memes (37.0%) labeled as Hateful. The fine-grained label distribution is long-tailed (Table 2). On the non-hateful side, Sarcasm (1,393) and Humor (1,331) are the most frequent labels. Among hateful memes, Mocking (1,007) is the most common attack strategy, whereas Exclusion (17) is the rarest. Since fine-grained labels are multi-label, the total label count exceeds the number of memes. The proportion of multi-label memes also increases from 9.9% in the train split to 21.4% in the test split, reflecting the richer co-occurring strategies captured in the triple-annotated gold subset. Per-split label cardinality is reported in Appendix C.3. Splits. We partition AHA-Memes into train, development, and test splits containing 3,500 (70%), 500 (10%), and 1,000 (20%) memes, respectively. The splits are stratified by the binary hate label, and we ensure that no meme appears in more than one split. The label distribution across the train, development, and test sets is reported in Table 2. Hate Fine-grained Train Dev Test Total Hateful Mocking 706 90 211 1,007 Hateful Incitement 320 51 85 456 Hateful Dehumanization 247 42 58 347 Hateful Slurs 252 42 47 341 Hateful Contempt 107 18 50 175 Hateful Inferiority 57 14 32 103 Hateful Exclusion 10 4 3 17 Hateful Other (H) 18 2 7 27 Not Hateful Other (NH) 380 50 96 526 Not Hateful Sarcasm 934 126 333 1,393 Not Hateful Humor 863 136 332 1,331 Table 2: Fine-grained sub-type distribution across splits. Counts sum to more than the number of memes because the label is multi-label. Humor, Sarcasm, and Other are non-hateful subtypes; the rest are hateful attack types. H: Hateful; NH: Not-Hateful. 3.6 Auxiliary Silver Dataset In addition to the 5K human-verified memes that constitute the core AHA-Memes, we release an auxiliary dataset of ∼ 66K Arabic memes with LLM-generated annotations from Gemini 3.1 Pro.555https://deepmind.google/models/model-cards/gemini-3-1-pro/ We refer to this resource as a silver dataset Wang et al. (2024); Du et al. (2023) as its labels are not human-verified. The dataset is intended to support weakly supervised learning, semi-supervised learning, active learning, retrieval-based analysis, and future annotation expansion. The motivation for releasing this auxiliary set is twofold. First, even without labels, a large in-domain collection of Arabic memes is valuable for the community. It captures meme templates, OCR noise, dialectal and code-mixed text, culturally specific references, visual stereotypes, and rapidly evolving political and social topics. Such data can support meme-understanding tasks that require joint reasoning over image content, embedded text, social context, and target entities Pramanick et al. (2021). Second, the silver labels provide a scalable starting point for methods that explicitly model noisy supervision. Weak-supervision frameworks have shown that noisy heuristic labels can be useful when denoised or combined with small amounts of gold data (Ratner et al., 2016, 2017). Similarly, semi-supervised and self-training methods use pseudo-labels on large unlabeled collections to improve model learning, especially when combined with confidence filtering and consistency-based training (Sohn et al., 2020). Recent work on LLM-assisted annotation further suggests that LLMs can reduce annotation cost and help bootstrap labeled resources, while also emphasizing the importance of distinguishing model-generated annotations from human-verified labels (Tan et al., 2024; He et al., 2024). We therefore position the ∼ 66K set as an auxiliary community resource rather than as a core contribution. All primary results in this paper are reported on the human-verified 5K set. Silver annotations are produced with Gemini 3.1 Pro using two prompts: for the unlabelled ∼ 66K memes the model predicts the label and subtype together with rich metadata (topic, dialect, sentiment, target categories, propaganda techniques, etc.), whereas for the 5K gold memes it generates the same metadata conditioned on the human label, so it never relabels them. The silver labels indicate that hateful content is relatively rare while the collection remains dialectally and topically diverse. Full distributions and the prompts are provided in Appendix E. 4 Experiments We evaluate models on the two tasks defined by our annotation scheme. The first is binary hateful-meme detection, where each meme is classified as Hateful or Not-Hateful. The second is fine-grained multi-label hate-type classification. 4.1 Evaluation Measures For the binary task, we use macro-F1 as the primary evaluation measure and additionally report accuracy and recall. For the fine-grained hate-type task, we use macro-F1 averaged over all labels as the primary measure, with micro-F1 reported as a complementary measure to account for overall multi-label performance. 4.2 Models We conduct experiments with five model groups covering unimodal encoders, multimodal fusion models, and vision-language models (VLMs). The text-only baselines use OCR text with AraBERTv2 (Antoun et al., 2020), MARBERTv2 (Abdul-Mageed et al., 2021), CAMeLBERT-mix (Inoue et al., 2021), XLM-R-base (Conneau et al., 2020), and mBERT (Devlin et al., 2019). The image-only baselines use the meme image with ViT-B/16 (Dosovitskiy et al., 2020), BEiT-B/16 (Bao et al., 2022), Swin-B (Liu et al., 2021), ConvNeXt-V2-tiny (Woo et al., 2023), and DINOv2-B (Oquab et al., 2024). Each unimodal encoder is followed by a linear classification head. The late-fusion models concatenate pooled representations from the two strongest text encoders and the two strongest vision encoders. The concatenated vector is passed to a two-layer MLP with GELU activation, dropout of 0.1, and hidden size 512. We further benchmark both open- and closed-weight vision-language models (VLMs), including Qwen3-VL-8B-Instruct (Qwen3-VL-8B) Xu et al. (2025), Fanar-2-Oryx-IVU FANAR TEAM et al. (2026), Phi-3.5-Vision Abdin et al. (2024), SmolVLM-Instruct, Qwen3-VL-8B-Thinking, InternVL3.5-8B, Gemini-2.5-Pro, Gemini-2.5-Flash Comanici et al. (2025), and GPT-5 Singh et al. (2026). Unless otherwise stated, all VLMs are evaluated in the zero-shot setting. In addition, we fine-tune Qwen3-VL-8B and evaluate its performance under both fine-tuning and few-shot in-context learning settings. We conduct few-shot experiments only with Qwen3-VL-8B to examine the effect of few-shot examples under a controlled setting, rather than exhaustively compare prompting across all VLMs. 4.3 Training and Model Selection All fine-tuned encoders and fusion models are trained with AdamW using a weight decay of 0.01, a linear learning-rate schedule, and 6% warmup. For binary hate detection, we use a softmax classification head with cross-entropy loss. For fine-grained hate-type prediction, we use a sigmoid output layer with binary cross-entropy loss to support multi-label classification. For each model family, we tune the learning rate, number of epochs, and batch size, and select the best checkpoint according to development macro-F1. The implementation details are provided in Appendix C.2. Open-weight VLMs (Qwen3-VL-8B and Qwen3-VL-2B) are fine-tuned with low-rank adaptation using LoRA with rank 16 and α=32α=32, applied to all linear layers while keeping the vision tower frozen. These models are trained for 3 epochs with a learning rate of 1×10−41×10^-4, a cosine schedule, and bfloat16 precision. For all supervised settings, models are trained on train, selected on dev by macro-F1, and evaluated once on the held-out test split using the single best configuration. For fine-grained hate-type prediction, the sigmoid decision threshold is tuned on the dev set. Zero-shot models are evaluated directly on test. 5 Results and Discussion 5.1 Results Across Model Families In Table 3, we report performance across all experimental settings including random and majority-class baselines. Binary Fine-grained Family Model macro-F1 Acc RecHate macro-F1 micro-F1 Baseline Majority 0.399 0.663 0.000 0.050 0.295 Random 0.500 0.553 0.337 0.125 0.233 Text (FT) MARBERTv2 0.709 0.743 0.596 0.263 0.390 CAMeLBERT-mix 0.689 0.726 0.567 0.252 0.375 AraBERTv2 0.678 0.706 0.611 0.295 0.411 XLM-R-base 0.650 0.716 0.418 0.259 0.415 mBERT 0.637 0.667 0.564 0.245 0.364 Image (FT) ConvNeXt-V2-tiny 0.657 0.700 0.513 0.246 0.418 Swin-base 0.657 0.686 0.585 0.258 0.365 BEiT-base 0.651 0.708 0.451 0.255 0.385 ViT-base 0.645 0.708 0.427 0.268 0.424 DINOv2-base 0.589 0.681 0.309 0.233 0.349 Fusion (FT) MARBERTv2++ViT 0.730 0.757 0.650 0.274 0.428 MARBERTv2++BEiT 0.724 0.750 0.656 0.318 0.444 AraBERTv2++ViT 0.706 0.741 0.591 0.279 0.416 AraBERTv2++BEiT 0.698 0.739 0.549 0.298 0.426 Open VLM (zero-shot) Qwen3-VL-8B-Instruct 0.643 0.743 0.318 0.176 0.352 Fanar-2-Oryx-IVU 0.535 0.700 0.154 0.115 0.288 Phi-3.5-vision 0.486 0.682 0.095 0.044 0.159 SmolVLM-Instruct 0.473 0.559 0.229 0.152 0.341 Qwen3-VL-8B-Thinking 0.469 0.676 0.077 0.191 0.382 InternVL3.5-8B 0.458 0.676 0.062 0.164 0.369 Open VLM (fine-tuned) Qwen3-VL-8B 0.768 0.797 0.656 0.334 0.475 Qwen3-VL-2B 0.739 0.782 0.558 0.284 0.437 Open VLM (few-shot) Qwen3-VL-8B 0.708 0.771 0.454 0.241 0.405 Closed VLM (zero-shot) Gemini-2.5-pro 0.711 0.774 0.457 0.340 0.465 GPT-5 0.628 0.741 0.282 0.301 0.464 Gemini-3.5-flash 0.499 0.694 0.104 0.271 0.486 Table 3: Results on the AHA-Memes test split. FT: Fine-tuned Bold marks the best value per column. RecHate is recall of the Hateful class, and fine-grained hate-type macro-F1 is averaged over all ten categories. Fine-tuned VLMs are strongest for binary task. Qwen3-VL-8B with LoRA achieves the best binary macro-F1 of 0.768. It outperforms the best late-fusion model, the best zero-shot closed VLM, and the strongest text encoder. Qwen3-VL-2B also performs well, reaching 0.739 macro-F1, which is close to the best fusion models. This suggests that supervised multimodal model is the most effective approach when labeled data are available. Zero-shot VLMs show low hateful-class recall. Qwen3-VL-8B-Instruct is the strongest zero-shot open VLM, reaching 0.643 binary macro-F1, followed by Fanar-2-Oryx-IVU. The remaining open VLMs perform close to the majority baseline on binary and show very low Hateful recall. The best zero-shot open VLM falls behind Gemini-2.5-pro by about 7 macro-F1 and the fine-tuned Qwen3-VL-8B by about 13 points. Fine-tuning substantially changes this pattern, improving Qwen3-VL-8B from 0.643 to 0.768 on binary and from 0.176 to 0.334 on fine-grained hate-type prediction. Fine-grained hate-type classification remains difficult. The best fine-grained hate-type macro-F1 is 0.340, achieved by Gemini-2.5-pro, with fine-tuned Qwen3-VL-8B close behind at 0.334. All model groups remain below 0.35 macro-F1. The gap between macro-F1 and micro-F1 suggests that models perform better on frequent labels such as Humor and Sarcasm than on rare hate types. Fine-tuning improves the performance, however, it does not fully mitigate the effect of label imbalance. Fusion helps, but text provides the strongest unimodal signal. Late fusion improves over the best text-only encoder on binary and increases Hateful recall, with MARBERTv2++BEiT matching the highest recall of 0.656. Image-only encoders are competitive, however, they are less reliable as standalone models. This pattern suggests that embedded Arabic text carries much of the task signal, while visual content provides complementary context that is most useful when combined with text. Arabic-specific encoders provide strong text baselines. Among text-only models, MARBERTv2 achieves the best binary macro-F1, while AraBERTv2 achieves the best fine-grained hate-type macro-F1. Both Arabic-specific encoders outperform mBERT and XLM-R-base on binary, highlighting the value of Arabic-centric pre-training for dialect-rich meme text. 5.2 Effect of Prompting We compare direct prompting with rationale-style prompting in English and Arabic for closed VLMs. Full results are reported in Appendix Table 8. Two trends are consistent. First, model choice has a larger effect than prompt format. Gemini-2.5-pro performs best across all prompting settings, followed by GPT-5 and Gemini-3.5-flash. Second, rationale-style prompting yields limited and model-dependent gains. It has little effect on binary performance, improves fine-grained hate-type prediction for the larger models, and hurts Gemini-3.5-flash when rationales are requested in Arabic. This suggests that Arabic rationale generation can introduce additional errors for lower-capacity models. 5.3 Retrieval-Augmented Few-Shot ICL Figure 3: Few-shot learning for Qwen3-VL-8B. Binary performance improves with more samples, however, it falls below zero-shot at K=1K=1. Fine-grained hate-type performance improves modestly and saturates early. RRF is the strongest retrieval strategy. We evaluate retrieval-augmented few-shot ICL. For each test meme, we retrieve K labeled examples from the train split and place them before the query as image-text demonstrations. We compare random demonstrations with text-based retrieval, image-based retrieval, and reciprocal rank fusion (RRF) over jina-clip-v2 embeddings (Koukounas et al., 2024; Cormack et al., 2009). We evaluate K∈1,3,5K∈1,3,5 using Qwen3-VL-8B and Qwen3-VL-2B. Details results are reported in Appendix D. Few-shot ICL gives moderate gains over zero-shot prompting, but remains below fine-tuning. As shown in Figure 3, RRF is the strongest retrieval strategy. The best setting is Qwen3-VL-8B with RRF and K=5K=5, which improves binary macro-F1 from 0.643 to 0.708 and fine-grained hate-type macro-F1 from 0.176 to 0.241 over the zero-shot Qwen3-VL-8B baseline. However, this remains below full fine-tuning, which reaches 0.768 binary macro-F1 and 0.334 fine-grained hate-type macro-F1. These results show that retrieval selects more informative demonstrations but still falls short of supervised adaptation. Retrieval mainly improves hateful-class recall. RRF with K=5K=5 raises Hateful recall from 0.318 to 0.454, reducing the zero-shot model’s bias toward non-hateful predictions. Gains for fine-grained hate-type prediction are smaller and are driven mostly by frequent labels such as Humor and Mocking, while rare hateful types remain difficult. A single demonstration is unstable and hurts binary performance, whereas K=3K=3 and K=5K=5 provide more reliable context. 5.4 Error Analysis Low recall is the main zero-shot failure. Zero-shot VLMs show low Hateful recall on binary. They under-predict the Hateful class and therefore miss many hateful memes, making this the most important safety failure for moderation. Fine-tuning mitigates this problem, Qwen3-VL-8B and the best fusion model recover roughly two-thirds of hateful memes. This suggests that the low-recall pattern is mainly a limitation of the zero-shot setting rather than a limitation of the model family. Fine-grained hate-type predictions are biased toward frequent labels. The zero-shot closed VLMs predict fine-grained hate-type distributions that differ considerably from the gold label distribution, as shown in Appendix Table 9. Gemini-3.5-flash over-predicts frequent labels such as Humor and Sarcasm, which improves micro-F1, however, weakens macro-F1. Gemini-2.5-pro over-predicts hateful types such as Mocking, Contempt, and Slurs, increasing recall at the expense of precision. GPT-5 is more cautious. Exclusion has only three test instances and is almost never predicted correctly. Longer meme text increases difficulty. Binary errors increase with the length of the extracted Arabic text for all models, as shown in Appendix Table 10. The trend is steepest for Gemini-3.5-flash. Longer text often require more context, pragmatic inference, and target identification, whereas memes with short text are more likely to contain direct lexical cues. Larger models are more robust, however, the trend remains visible across systems. Implicit hate is harder to classify. Qualitative inspection of Gemini-2.5-pro with Arabic rationale prompting shows that false negatives are not caused by OCR or image-understanding failures. The model identifies the visual template and reads dialectal Arabic correctly but treats implicit sectarian, religious, or misogynistic attacks as humor or satire and predicts Not-Hateful. False positives are offensive dialectal jokes that annotators judged not to be hateful. These cases show that the hardest boundary is the normative judgment of implicit hate rather than multimodal prediction. This motivates the explicit taxonomy and annotation guidelines described in §3.4. 6 Conclusion and Future Work We presented AHA-Memes, a fine-grained Arabic hateful-meme benchmark comprising 5K human-annotated memes with a hierarchical annotation scheme covering hatefulness, attack subtype, and target category. We also release a separate auxiliary corpus of ∼ 66K silver-labeled memes with model-generated labels and metadata for weak supervision rather than evaluation. Experiments show that fine-tuning an open-weight VLM gives the best binary performance, while late fusion and Arabic text encoders remain strong baselines. Text provides the strongest unimodal signal, while visual context offers complementary information. Fine-grained classification remains challenging because models struggle to recover rare and overlapping hate categories. Zero-shot VLMs make conservative predictions and often miss hateful memes, while few-shot prompting provides only modest gains. Future work will focus on rare and co-occurring hate categories, target-aware modeling, clearer separation between offensive humor and protected-group hate, and broader coverage of dialects and emerging meme formats. Limitations AHA-Memes is designed as a focused benchmark for Arabic hateful-meme understanding rather than a comprehensive representation of Arabic meme culture. We collected data four public platforms, Facebook, Instagram, Pinterest, and Twitter/X, which gives broad public coverage, however, our collection may not capture all dialects, regions, platform communities, or private and ephemeral content. Some annotation boundaries are inherently nuanced, especially between offensive humor, satire, political criticism, and protected-group hate. We address this detailed annotation guideline, and training. The auxiliary silver corpus broadens the resource and supports weakly supervised research, but its labels are model-generated and not human-verified. Ethics and Broader Impact AHA-Memes supports research on Arabic multimodal hate detection and safer content moderation. It is developed from publicly available memes and includes hateful, offensive, and sensitive material solely for research and evaluation. We do not collect or release personal information, which helps minimize privacy risks. As models can miss harmful content or over-flag legitimate content, they should be used with human oversight, clear documentation, and ongoing monitoring. Annotation was performed by trained, fairly compensated native-speaker annotators using guidelines that distinguish protected-group hate from generic offensiveness and legitimate criticism. To further reduce privacy and misuse risks, we release OCR text, labels, metadata, and image references. Acknowledgments The work was supported by NPRP grant 14C-0916-210015 from the Qatar National Research Fund, part of the Qatar Research Development and Innovation Council (QRDI). The findings reported herein are solely the responsibility of the authors. References A. Abdelali, H. Mubarak, S. A. Chowdhury, M. Hasanain, B. Mousi, S. Boughorbel, S. Abdaljalil, Y. El Kheir, D. Izham, F. Dalvi, M. Hawasly, N. Nazar, Y. Elshahawy, A. Ali, N. Durrani, N. Milic-Frayling, and F. Alam (2024) LAraBench: benchmarking Arabic AI with large language models. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), Y. Graham and M. Purver (Eds.), St. Julian’s, Malta, p. 487–520. External Links: Link, Document Cited by: §2. M. Abdin, J. Aneja, H. Awadalla, A. Awadallah, A. A. Awan, N. Bach, A. Bahree, A. Bakhtiari, J. Bao, H. Behl, et al. (2024) Phi-3 technical report: a highly capable language model locally on your phone. ArXiv preprint arXiv:2404.14219. External Links: Link Cited by: §4.2. M. Abdul-Mageed, A. Elmadany, and E. M. B. Nagoudi (2021) ARBERT & MARBERT: deep bidirectional transformers for Arabic. Online, p. 7088–7105. External Links: Link, Document Cited by: §4.2. F. Alam, A. Barrón-Cedeño, G. S. Cheema, G. K. Shahi, S. Hakimov, M. Hasanain, C. Li, R. Míguez, H. Mubarak, W. Zaghouani, et al. (2023) Overview of the clef-2023 checkthat! lab task 1 on check-worthiness in multimodal and multigenre content.. In CLEF (Working Notes), p. 219–235. Cited by: §2. F. Alam, M. R. Biswas, U. Shah, W. Zaghouani, and G. Mikros (2024a) Propaganda to hate: a multimodal analysis of arabic memes with multi-agent llms. In International Conference on Web Information Systems Engineering, p. 380–390. Cited by: §1, §2, Table 1. F. Alam, A. Hasnat, F. Ahmad, Md. A. Hasan, and M. Hasanain (2024b) ArMeme: propagandistic content in Arabic memes. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Y. Al-Onaizan, M. Bansal, and Y. Chen (Eds.), Miami, Florida, USA, p. 21071–21090. External Links: Link, Document Cited by: §2, Table 1, §3.1, §3.2. F. Alam, H. Mubarak, W. Zaghouani, G. Da San Martino, and P. Nakov (2022) Overview of the wanlp 2022 shared task on propaganda detection in arabic. In Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP), p. 108–118. Cited by: §2. F. Alam, F. Ofli, M. Imran, T. Alam, and U. Qazi (2020) Deep learning benchmarks and datasets for social media image classification for disaster response. p. 151–158. Cited by: §3.2. W. Antoun, F. Baly, and H. Hajj (2020) AraBERT: transformer-based model for arabic language understanding. p. 9. Cited by: §4.2. H. Bao, L. Dong, S. Piao, and F. Wei (2022) BEiT: BERT pre-training of image transformers. External Links: Link Cited by: §4.2. T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei (2020) Language models are few-shot learners. p. 1877–1901. Cited by: §2. M. D. Bui, K. von der Wense, and A. Lauscher (2025) Multi3Hate: multimodal, multilingual, and multicultural hate speech detection with vision-language models. In Proceedings of the 2025 Annual Conference of the Nations of the Americas Chapter of the Association for Computational LinguisticsProceedings of the 2022 Conference on Empirical Methods in Natural Language ProcessingProceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021)Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)Proceedings of the 2024 Conference on Empirical Methods in Natural Language ProcessingAdvances in Neural Information Processing SystemsProceedings of the Second Workshop on NLP for Positive Impact (NLP4PI)Findings of the Association for Computational Linguistics: EMNLP 2023Proceedings of the ACM Web Conference 2024Proceedings of the 2022 Conference on Empirical Methods in Natural Language ProcessingProceedings of the 63rd Annual Meeting of the Association for Computational LinguisticsAdvances in Neural Information Processing SystemsProceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT)Proceedings of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT)Proceedings of the ACM Web Conference 2026Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and EvaluationFindings of the Association for Computational Linguistics: EMNLP 2021Proceedings of the 2025 IEEE International Conference on Big Data (BigData)Proceedings of the Fourteenth Workshop on Semantic EvaluationCompanion Proceedings of the ACM Web Conference 2025Proceedings of the Second Workshop on Trolling, Aggression and Cyberbullying (TRAC)Proceedings of the ACM Web Conference 2026 (W)Proceedings of the IEEE conference on computer vision and pattern recognition2020 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM)Proceedings of the International AAAI Conference on Web and Social MediaProceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING)Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP)Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP)Proceedings of Deep Learning Inside Out (DeeLIO): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning ArchitecturesProceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information RetrievalProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)Advances in Neural Information Processing Systems 33 (NeurIPS)Advances in Neural Information Processing Systems 35 (NeurIPS)Proceedings of the AAAI Conference on Artificial IntelligenceProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP)Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)LREC 2020 Workshop Language Resources and Evaluation Conference 11–16 May 2020Proceedings of the Sixth Arabic Natural Language Processing WorkshopProceedings of the 58th Annual Meeting of the Association for Computational LinguisticsProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)International Conference on Learning RepresentationsWeb Information Systems Engineering – WISE 2024Findings of the Association for Computational Linguistics: EMNLP 2025Findings of the Association for Computational Linguistics: NAACL 2025Companion Proceedings of the ACM Web Conference 2026, L. Biester, D. Demszky, Z. Jin, M. Sachan, J. Tetreault, S. Wilson, L. Xiao, J. Zhao, A. Herbelot, X. Zhu, A. Palmer, N. Schneider, J. May, E. Shutova, D. Jurafsky, J. Chai, N. Schluter, J. Tetreault, J. Burstein, C. Doran, T. Solorio, M. Barhamgi, H. Wang, X. Wang, C. Christodoulopoulos, T. Chakraborty, C. Rose, V. Peng, L. Chiruzzo, A. Ritter, and L. Wang (Eds.), NAACL ’25EMNLP ’22EMNLP ’24NeurIPS ’20EMNLP ’23W ’24EMNLP ’22ACL ’25NeurIPS ’24W ’26LREC-COLING ’24W ’25CVPR ’16W Companion ’26, Vol. 12020-December1371536, p. 9714 – 9731. External Links: Document Cited by: §1, §1. R. Cao, R. K. Lee, W. Chong, and J. Jiang (2022) Prompting for multimodal hateful meme classification. Conference paper p. 321 – 332. External Links: Document Cited by: §2. G. Comanici, E. Bieber, M. Schaekermann, I. Pasupat, N. Sachdeva, I. Dhillon, M. Blistein, O. Ram, D. Zhang, E. Rosen, et al. (2025) Gemini 2.5: pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. ArXiv preprint arXiv:2507.06261. External Links: Link Cited by: §4.2. A. Conneau, K. Khandelwal, N. Goyal, V. Chaudhary, G. Wenzek, F. Guzmán, E. Grave, M. Ott, L. Zettlemoyer, and V. Stoyanov (2020) Unsupervised cross-lingual representation learning at scale. Online, p. 8440–8451. External Links: Link, Document Cited by: §4.2. G. V. Cormack, C. L. A. Clarke, and S. Buettcher (2009) Reciprocal rank fusion outperforms condorcet and individual rank learning methods. p. 758–759. Cited by: §D.1, §2, §5.3. M. Das and A. Mukherjee (2023) BanglaAbuseMeme: a dataset for Bengali abusive meme classification. Singapore, p. 15498–15512. External Links: Link, Document Cited by: §2. J. Devlin, M. Chang, K. Lee, and K. Toutanova (2019) BERT: pre-training of deep bidirectional transformers for language understanding. Minneapolis, Minnesota, p. 4171–4186. External Links: Link, Document Cited by: §4.2. A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby (2020) An image is worth 16x16 words: transformers for image recognition at scale. CoRR abs/2010.11929. External Links: Link, 2010.11929 Cited by: §4.2. W. Du, L. Advani, Y. Gambhir, D. Perry, P. Shiralkar, Z. Xing, and A. Colak (2023) Effective proxy for human labeling: ensemble disagreement scores in large language models for industrial NLP. In Proceedings of the Third Workshop on Natural Language Generation, Evaluation, and Metrics (GEM), S. Gehrmann, A. Wang, J. Sedoc, E. Clark, K. Dhole, K. R. Chandu, E. Santus, and H. Sedghamiz (Eds.), Singapore, p. 53–61. External Links: Link Cited by: §3.6. S. El-Amrany, A. El Khaiari, A. El Bekkali, S. Lamsiyah, M. R. Brust, K. Lavangnananda, J. E. Pecero, and P. Bouvry (2026) AraHarMeme: a multimodal benchmark for harmful meme detection in arabic. In 2026 23rd International Joint Conference on Computer Science and Software Engineering (JCSSE), p. 564–569. External Links: Document Cited by: Table 1. S. El-amrany, S. Lamsiyah, M. R. Brust, and P. Bouvry (2025) GuardHarMem and HarMDetect: a multimodal dataset and benchmark model for fine-grained harmful meme classification. Social Network Analysis and Mining 15 (1), p. 63. External Links: Document Cited by: §1. FANAR TEAM, U. Abbas, M. S. Ahmad, M. Ahmad, A. Al-Homaid, A. Al-Nuaimi, E. Altinisik, E. Asgari, S. Chawla, S. Chowdhury, F. Dalvi, K. Darwish, N. Durrani, M. Elfeky, A. Elmagarmid, M. Eltabakh, A. Ersoy, M. Fatehkia, M. Q. Hashim, M. Hawasly, M. Hefeeda, M. Husaini, K. Isufaj, S. Jung, H. Lachemat, J. K. Lucas, A. Mohamed, T. Mohiuddin, B. Mousi, H. Mubarak, A. Musleh, M. Ouzzani, A. Sadeghi, H. T. Sencar, M. Shinoy, O. Sinan, and Y. Zhang (2026) Fanar 2.0: Arabic generative AI stack. ArXiv preprint arXiv:2603.16397. External Links: Link Cited by: §4.2. M. Hasanain, F. Ahmad, and F. Alam (2024a) Can GPT-4 identify propaganda? annotation and detection of propaganda spans in news articles. Torino, Italy, p. 2724–2744. External Links: Link Cited by: §3.4. M. Hasanain, M. A. Hasan, M. B. Kmainasi, E. Sartori, A. E. Shahroor, G. Da San Martino, and F. Alam (2025) PropXplain: can LLMs enable explainable propaganda detection?. Suzhou, China, p. 23855–23863. External Links: Link, Document, ISBN 979-8-89176-335-7 Cited by: §2. M. Hasanain, Md. A. Hasan, F. Ahmad, R. Suwaileh, Md. R. Biswas, W. Zaghouani, and F. Alam (2024b) ArAIEval shared task: propagandistic techniques detection in unimodal and multimodal Arabic content. In Proceedings of the Second Arabic Natural Language Processing Conference, N. Habash, H. Bouamor, R. Eskander, N. Tomeh, I. Abu Farha, A. Abdelali, S. Touileb, I. Hamed, Y. Onaizan, B. Alhafni, W. Antoun, S. Khalifa, H. Haddad, I. Zitouni, B. AlKhamissi, R. Almatham, and K. Mrini (Eds.), Bangkok, Thailand, p. 456–466. External Links: Link, Document Cited by: §2, Table 1. X. He, Z. Lin, Y. Gong, A. Jin, H. Zhang, C. Lin, J. Jiao, S. M. Yiu, N. Duan, and W. Chen (2024) AnnoLLM: making large language models to be better crowdsourced annotators. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 6: Industry Track), Mexico City, Mexico, p. 165–190. External Links: Link, Document Cited by: §3.6. G. Inoue, B. Alhafni, N. Baimukan, H. Bouamor, and N. Habash (2021) The interplay of variant, size, and task type in Arabic pre-trained language models. Kyiv, Ukraine (Online). Cited by: §4.2. D. Kiela, H. Firooz, A. Mohan, V. Goswami, A. Singh, P. Ringshia, and D. Testuggine (2020) The hateful memes challenge: detecting hate speech in multimodal memes. Conference paper p. . External Links: Link Cited by: §1, §2, 1st item. M. B. Kmainasi, A. Hasnat, M. A. Hasan, A. E. Shahroor, and F. Alam (2025a) MemeIntel: explainable detection of propagandistic and hateful memes. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, C. Christodoulopoulos, T. Chakraborty, C. Rose, and V. Peng (Eds.), Suzhou, China, p. 30263–30279. External Links: Link, Document, ISBN 979-8-89176-332-6 Cited by: §1, §2, §2, Table 1. M. B. Kmainasi, A. E. Shahroor, M. Hasanain, S. R. Laskar, N. Hassan, and F. Alam (2025b) LlamaLens: specialized multilingual LLM for analyzing news and social media content. Albuquerque, New Mexico, p. 5642–5664. External Links: Link, Document, ISBN 979-8-89176-195-7 Cited by: §2. A. Koukounas, G. Mastrapas, S. Eslami, B. Wang, M. K. Akram, M. Günther, I. Mohr, S. Sturua, N. Wang, and H. Xiao (2024) jina-clip-v2: multilingual multimodal embeddings for text and images. ArXiv preprint arXiv:2412.08802. Cited by: §D.1, §5.3. G. K. Kumar and K. Nandakumar (2022) Hate-CLIPper: multimodal hateful meme classification based on cross-modal interaction of CLIP features. Abu Dhabi, United Arab Emirates (Hybrid), p. 171–183. External Links: Link, Document Cited by: §2. J. R. Landis and G. G. Koch (1977) The measurement of observer agreement for categorical data. Biometrics 33 (1), p. 159–174. Cited by: §3.4. M. Liao, Z. Zou, Z. Wan, C. Yao, and X. Bai (2022) Real-time scene text detection with differentiable binarization and adaptive scale fusion. IEEE transactions on pattern analysis and machine intelligence 45 (1), p. 919–931. Cited by: §3.2. H. Lin, Z. Luo, W. Gao, J. Ma, B. Wang, and R. Yang (2024) Towards explainable harmful meme detection through multimodal debate between large language models. Conference paper p. 2359 – 2370. External Links: Document Cited by: §2. J. Liu, D. Shen, Y. Zhang, B. Dolan, L. Carin, and W. Chen (2022) What makes good in-context examples for GPT-3?. p. 100–114. Cited by: §2. Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo (2021) Swin transformer: hierarchical vision transformer using shifted windows. External Links: 2103.14030, Link Cited by: §4.2. J. Lu, B. Xu, X. Zhang, H. Wang, H. Zhu, D. Zhang, L. Yang, and H. Lin (2024) Towards comprehensive detection of Chinese harmful memes. Conference paper p. . External Links: Link Cited by: §2. H. Mubarak, H. Al-Khalifa, and A. Al-Thubaity (2022) Overview of OSACT5 Shared Task on Arabic Offensive Language and Hate Speech Detection. Marseille, France, p. 162–166. External Links: Link Cited by: §1, §2. M. Oquab, T. Darcet, T. Moutakanni, H. Vo, M. Szafraniec, V. Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby, M. Assran, N. Ballas, W. Galuba, R. Howes, P. Huang, S. Li, I. Misra, M. Rabbat, V. Sharma, G. Synnaeve, H. Xu, H. Jegou, J. Mairal, P. Labatut, A. Joulin, and P. Bojanowski (2024) DINOv2: learning robust visual features without supervision. External Links: 2304.07193, Link Cited by: §4.2. R. Ponnusamy, K. Pannerselvam, S. Rajiakodi, P. K. Kumaresan, S. Thavareesan, B. Sivagnanam, K.A. Anshid, S. S. Kumar, P. Buitelaar, and B. R. Chakravarthi (2024) From laughter to inequality: annotated dataset for misogyny detection in Tamil and Malayalam memes. Conference paper p. 7480 – 7488. External Links: Link Cited by: §2. S. Pramanick, S. Sharma, D. Dimitrov, Md. S. Akhtar, P. Nakov, and T. Chakraborty (2021) MOMENTA: a multimodal framework for detecting harmful memes and their targets. In Findings of the Association for Computational Linguistics: EMNLP 2021, Punta Cana, Dominican Republic, p. 4439–4455. External Links: Link, Document Cited by: §3.6. A. Ratner, S. H. Bach, H. Ehrenberg, J. Fries, S. Wu, and C. Ré (2017) Snorkel: rapid training data creation with weak supervision. Proceedings of the VLDB Endowment 11 (3), p. 269–282. External Links: Document Cited by: §3.6. A. J. Ratner, C. M. De Sa, S. Wu, D. Selsam, and C. Ré (2016) Data programming: creating large training sets, quickly. In Advances in Neural Information Processing Systems, Vol. 29, p. 3567–3575. Cited by: §3.6. C. Ren, D. Jeong, M. Wu, Y. Huang, Y. Gao, and Y. Li (2026) A survey of multimodal hate meme detection. Expert Systems with Applications 323, p. 132507. External Links: Document Cited by: §2. A. E. Shahroor, M. B. Kmainasi, A. Hasnat, D. Dimitrov, G. Da San Martino, P. Nakov, and F. Alam (2026) MemeLens: multilingual multitask VLMs for memes. arXiv: 2601.12539. External Links: Link Cited by: §1, §2. S. Sharma, F. Alam, Md. S. Akhtar, D. Dimitrov, G. Da San Martino, H. Firooz, A. Halevy, F. Silvestri, P. Nakov, and T. Chakraborty (2022) Detecting and understanding harmful memes: a survey. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22, L. D. Raedt (Ed.), p. 5597–5606. Note: Survey Track External Links: Link Cited by: §1, §2, §3.3. A. Singh, A. Fry, A. Perelman, A. Tart, A. Ganesh, A. El-Kishky, A. McLaughlin, A. Low, A. Ostrow, A. Ananthram, et al. (2026) OpenAI GPT-5 system card. ArXiv preprint arXiv:2601.03267. External Links: Link Cited by: §4.2. K. Sohn, D. Berthelot, N. Carlini, Z. Zhang, H. Zhang, C. A. Raffel, E. D. Cubuk, A. Kurakin, and C. Li (2020) FixMatch: simplifying semi-supervised learning with consistency and confidence. In Advances in Neural Information Processing Systems, Vol. 33, p. 596–608. Cited by: §3.6. Z. Tan, D. Li, S. Wang, A. Beigi, B. Jiang, A. Bhattacharjee, M. Karami, J. Li, L. Cheng, and H. Liu (2024) Large language models for data annotation and synthesis: a survey. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Miami, Florida, USA, p. 930–957. External Links: Link, Document Cited by: §3.6. J. Wang, T. Wang, and Z. Zeng (2024) On the use of silver standard data for zero-shot classification tasks in information extraction. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), N. Calzolari, M. Kan, V. Hoste, A. Lenci, S. Sakti, and N. Xue (Eds.), Torino, Italia, p. 12423–12434. External Links: Link Cited by: §3.6. M. Wang, K. Ren, P. Jalan, A. Ashraf, T. V. Vu, R. Seetharaman, S. Nawaz, and U. Naseem (2026) From native memes to global moderation: cross-cultural evaluation of vision–language models for hateful meme detection. Dubai, United Arab Emirates, p. 9788–9799. External Links: Document Cited by: §2. S. Woo, S. Debnath, R. Hu, X. Chen, Z. Liu, I. S. Kweon, and S. Xie (2023) ConvNeXt V2: co-designing and scaling convnets with masked autoencoders. CoRR abs/2301.00808. External Links: Link, Document, 2301.00808 Cited by: §4.2. J. Xu, Z. Guo, H. Hu, Y. Chu, X. Wang, J. He, Y. Wang, X. Shi, T. He, X. Zhu, et al. (2025) Qwen3-Omni technical report. ArXiv preprint arXiv:2509.17765. External Links: Link Cited by: §4.2. W. Zaghouani, K. Attia, Md. R. Biswas, and F. Eryani (2026) ArPoMeme: an annotated arabic multimodal dataset for political ideology and polarization. In Proceedings of the Fifteenth Language Resources and Evaluation Conference (LREC 2026), Palma, Mallorca, Spain, p. 6597–6608. External Links: Link, Document, ISBN 978-2-493814-49-4 Cited by: Table 1. W. Zaghouani, Md. R. Biswas, M. Bessghaier, S. Ibrahim, G. Mikros, A. Hasnat, and F. Alam (2025) MAHED shared task: multimodal detection of hope and hate emotions in Arabic content. In Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks, K. Darwish, A. Ali, I. Abu Farha, S. Touileb, I. Zitouni, A. Abdelali, S. Al-Ghamdi, S. Alkhereyf, W. Zaghouani, S. Khalifa, B. AlKhamissi, R. Almatham, I. Hamed, Z. Alyafeai, A. Alowisheq, G. Inoue, K. Mrini, and W. Alshammari (Eds.), Suzhou, China, p. 560–574. External Links: Link, Document, ISBN 979-8-89176-356-2 Cited by: Table 1. X. Zhai, B. Mustafa, A. Kolesnikov, and L. Beyer (2023) Sigmoid loss for language image pre-training. p. 11975–11986. Cited by: §D.1. Appendix A Data and Resource Release We will release666anonymous.com the supporting code, documentation, and dataset under the C BY-NC-SA 4.0 license to facilitate adoption, reproducibility, and future extensions. Appendix B Annotation Guidelines B.1 English Annotation Guideline Purpose. The purpose of this annotation is: (i) to identify whether a meme is Hateful or Not-Hateful; (i) for hateful memes, to identify the attack type(s) (multi-label) and the targeted protected category(ies) (multi-label); and (i) for non-hateful memes, to assign one or more non-hateful subtypes (multi-label). The annotation design is hierarchical and conditional: the fine-grained tasks are completed based on the binary label decision, discussed below. Definitions. We define hateful content as a direct or indirect attack on people on the basis of a protected characteristic, including ethnicity, race, nationality, immigration status, religion, caste, sex, gender identity, sexual orientation, and disability or disease. An attack includes violent or dehumanizing speech, such as comparing people to non-human entities, statements of inferiority, calls for exclusion or segregation, or mockery of a hate crime. We apply two boundary rules: • Attacking groups that themselves promote or carry out hate, such as terrorist organisations, is not considered hate speech. • A meme that is insulting or offensive but does not target a protected category is labelled Not-Hateful under this scheme. B.1.1 Task 1: Binary Classification (is_hateful; Hateful vs. Not-Hateful) The task is to select exactly one of Hateful or Not-Hateful. The choice determines the subsequent annotation task: if Hateful, complete Tasks 2 if Not-Hateful, complete Task 3. B.1.2 Task 2: Attack Type (hateful_type; Hateful only; multi-label) The task is to select all attack types that apply; a meme may exhibit several at once (e.g., Slurs ++ Dehumanization, or Mocking ++ Inferiority). • Dehumanization: explicitly or implicitly describing a group as subhuman, such as by comparing an ethnic group to animals or using “fish season” metaphors (Figure 4). • Inferiority: claiming that a group is inferior, less worthy, or less important than society or another group, such as through class-based belittling of “peasants” (Figure 5). • Incitement: explicitly or implicitly calling for harm to be inflicted on a group, including physical attacks, such as through imagery of forced physical discipline (Figure 6). • Mocking: joking about, undermining, belittling, or disparaging a group, such as by portraying its members as buffoonish or body-shaming a gender (Figure 7). • Contempt: expressing intensely negative feelings toward a group, such as through statements like “Men are trash” or expressions of ideological contempt (Figure 8). • Slurs: using prejudicial terms to refer to or characterize a group, including racial, religious, or gender-based slurs in Arabic dialects (Figure 9). • Exclusion: advocating, planning, or justifying the exclusion or segregation of a group, such as through statements like “Go back to your country” (Figure 10). • Other: a hateful attack that does not clearly fall under any of the types above. This option is selected only when no other type applies and cannot be combined with another attack type. B.1.3 Task 3: Not-Hateful Subtype (non_hateful_type; Not-Hateful only; multi-label) Non-hateful memes contain humorous, neutral, or positive content that does not target or harm a protected group, are intended for entertainment, and do not promote violence, hatred, or discrimination. The task is to select one or more subtypes. • Humor: content intended to entertain or amuse through jokes, puns, exaggeration, or playful commentary on everyday situations (work, family, food, travel), with no negative implication toward a protected group (Figure 11). • Sarcasm: irony in which the intended meaning is the opposite of what is stated, used for humorous effect and directed at general behaviour or institutions (e.g., government, bureaucracy, inflation) rather than a protected group (Figure 12). • Other: neutral, positive, or informational memes that are not clearly humour or sarcasm (e.g., motivational quotes, announcements, generic reactions) and do not target a protected group. Figure 4: Example memes annotated as Hateful / Dehumanization. Figure 5: Example memes annotated as Hateful / Inferiority. Figure 6: Example memes annotated as Hateful / Incitement. Figure 7: Example memes annotated as Hateful / Mocking. Figure 8: Example memes annotated as Hateful / Contempt. Figure 9: Example memes annotated as Hateful / Slurs. Figure 10: Example annotated as Hateful / Exclusion. Figure 11: Example memes annotated as Not-Hateful / Humor. Figure 12: Example memes annotated as Not-Hateful / Sarcasm. B.2 Arabic Annotation Guideline The following is the Arabic version of the guidelines used during annotation. Purpose and definitions. Task 1: Meme classification. Task 2: Attack types (multi-label). Appendix C Experimental Details C.1 Prompting the Zero-shot VLMs Each zero-shot query combines a system message with an instruction message. The system message frames the model as an expert Arabic multimodal content-moderation annotator covering MSA and the Egyptian, Levantine, Gulf, and Maghrebi dialects.. The instruction provides the meme image and OCR-extracted text and restricts the output to the predefined label space. For binary classification, the model predicts Hateful or Not-Hateful from the combined image–text content, guided by definitions that distinguish identity-based hate from general offensiveness. It returns the prediction as "label": …. For fine-grained classification, the prompt lists the ten taxonomy labels with brief definitions and asks for all applicable labels as a JSON list. The cot-en and cot-ar variants add a free-text reasoning field in English or Arabic, respectively. We fine-tune the open-weight VLMs with the same binary and fine-grained templates to generate the target label set directly. C.2 Training and Compute We fine-tune the text, image, and fusion models with the HuggingFace Transformers Trainer. We use AdamW with weight decay 0.01, a linear learning-rate schedule, 6% warmup, and 128-token text inputs. Hyperparameters are selected from the search space as reported in Table 4, and the best checkpoint is chosen based on macro-F1 on the development set. For fine-grained hate-type prediction, we tune a global sigmoid threshold on the development set over 0.05,0.10,…,0.90\0.05,0.10,…,0.90\. Open-weight Qwen3-VL models are fine-tuned with ms-swift using LoRA with rank 16 and α=32α=32, applied to all linear layers while freezing the vision encoder. We train for 3 epochs with learning rate 1×10−41×10^-4, a cosine schedule, and bfloat16. All experiments use seed 42 on NVIDIA H200 GPUs. Hyper-param. Text Image Fusion VLM Learning rate 2,3,5 ×10−5× 10^-5 3,5 ×10−5× 10^-5 2,3 ×10−5× 10^-5 1 ×10−4× 10^-4 Epochs 5, 10 10, 15 5, 10 3 Batch size 16, 32 32 16 4 Table 4: Grid-search space for the fine-tuned families (best by dev macro-F1). The open VLMs use fixed LoRA settings rather than a grid. The fusion-head learning rate is fixed at 1×10−41×10^-4. All runs use seed 42. C.3 Additional Dataset Analysis OCR-text length. In Table 5, we report the distribution of OCR-extracted text length. The splits are well matched (mean ≈ 86-88 characters), and Hateful memes carry noticeably more text than Not-Hateful ones on the test split (96.2 vs. 83.1 characters), consistent with the finding (Section 5.4) that hateful intent is often conveyed through longer, more contextual captions. Multi-label distribution. In Table 6, we report the number of fine-grained categories assigned to each meme. The test split shows the greatest category overlap, with an average of 1.25 labels per meme; 21.4% of test memes receive at least two categories, compared with 9.9% in the training split. Label co-occurrence. Table 7 shows the most frequent fine-grained pairs. Mocking acts as a hub, co-occurring with Dehumanization, Incitement, Contempt, and Slurs, while among non-hateful memes Humor and Sarcasm frequently appear together. These patterns motivate modelling hate as co-occurring strategies rather than a single label per meme. Split Mean Median Max Train 85.5 76 641 Dev 87.8 80 619 Test 87.5 77 644 Table 5: OCR-text length (characters) per split. Test set class-wise: Hateful 96.2; Not-Hateful 83.1 (mean) Split Avg 1 2 3 4 Train 1.11 3,154 298 48 0 Dev 1.15 439 47 14 0 Test 1.25 786 178 32 4 Table 6: Fine-grained label cardinality: number of memes with 1-4 fine-grained labels, and the mean number of labels per meme. Fine-grained pair Count Dehumanization ++ Mocking 145 Incitement ++ Mocking 114 Humor ++ Sarcasm 81 Contempt ++ Mocking 78 Dehumanization ++ Incitement 66 Mocking ++ Slurs 60 Inferiority ++ Mocking 55 Incitement ++ Slurs 44 Contempt ++ Incitement 39 Dehumanization ++ Slurs 31 Table 7: Most frequent fine-grained hate-type co-occurrence pairs (all splits). C.4 Effect of Prompting on the Closed Models Table 8 reports the full prompting comparison summarised in §5.2. For each closed model it gives macro-F1 under the direct prompt and under chain-of-thought reasoning in English and in Arabic. The model ordering is stable across prompts, and chain-of-thought helps the larger models on fine-grained categories while hurting the small Gemini-3.5-flash when the reasoning is in Arabic. Model Prompt Binary Fine-grained Gemini-2.5-pro default 0.711 0.340 cot-en 0.709 0.358 cot-ar 0.726 0.333 GPT-5 default 0.628 0.301 cot-en 0.642 0.313 cot-ar 0.647 0.315 Gemini-3.5-flash default 0.499 0.271 cot-en 0.499 0.272 cot-ar 0.493 0.265 Table 8: Macro-F1 of the zero-shot closed VLMs under direct (default) and chain-of-thought (English or Arabic) prompting. The best value per column is in bold. C.5 Additional Error Analysis We provide two supporting tables for the zero-shot error analysis in §5.4. Table 9 compares, for each label, the predicted count from each closed VLM with the corresponding gold label count. This highlights the over-prediction of frequent non-hateful labels such as Humor and Sarcasm, as well as several hateful fine-grained categories. Table 10 reports binary error rates across three OCR-text length groups, showing that longer captions are harder for all models, with the largest degradation observed for the smallest model. Label Gold Pro GPT-5 Flash Humor 332 728 878 885 Sarcasm 333 349 404 869 Mocking 211 465 164 120 Contempt 50 264 64 45 Slurs 47 166 31 9 Table 9: Subtype label over- and under-prediction (predicted count vs. gold support) for the zero-shot closed models under the default prompt. Model (default) 21–60 61–120 120+ Gemini-2.5-pro 0.222 0.221 0.255 GPT-5 0.239 0.260 0.299 Gemini-3.5-flash 0.258 0.312 0.382 Table 10: Binary error rate by OCR-text length (characters). Longer, more contextual captions are harder, and the smallest model degrades most. Appendix D In-Context Learning D.1 Few-Shot Retrieval Setup For each test meme, we retrieve demonstrations from the 3,500-meme training pool using a CLIP-style dual encoder. Our main retriever is jina-clip-v2 (Koukounas et al., 2024), which embeds images and text into a shared multilingual space; we use SigLIP-2 (Zhai et al., 2023) as an ablation. We construct each prompt as a multi-turn chat with a task system prompt, K demonstration turns, and a final query turn. Each demonstration includes the meme image, its OCR text, and its gold label. We compare text-only retrieval, image-only retrieval, and reciprocal rank fusion (RRF). Text and image retrieval rank candidates by cosine similarity in the corresponding modality. RRF combines the two rank lists as s(d)=∑m∈t,v160+rankm(d)s(d)= _m∈t,v 160+rank_m(d) (Cormack et al., 2009), making the fusion independent of modality-specific score scales. Retrieval and prompt construction are performed using only the training dataset. We use RRF with K=3K=3, cosine similarity, jina-clip-v2 embeddings, and similar-last ordering. D.2 Few-Shot Results In Tables 11 and 12, we report the strategy-by-K results for Qwen3-VL-8B and Qwen3-VL-2B, respectively. For Qwen3-VL-8B, RRF achieves the best performance on both subtasks and improves consistently up to K=5K=5. On binary task, however, all strategies perform below zero-shot at K=1K=1. Qwen3-VL-2B follows a different pattern, peaking at K=3K=3 and dropping at K=5K=5, while image-only retrieval matches RRF. These results suggest that the smaller model benefits from a small number of demonstrations, although it uses longer or fused demonstration contexts less effectively. Binary (macro-F1) Fine-grained (macro-F1) Strategy K0K0 K1K1 K3K3 K5K5 K1K1 K3K3 K5K5 zero-shot 0.643 — — — — — — random — 0.547 0.637 0.648 0.191 0.197 0.196 text — 0.579 0.659 0.688 0.225 0.215 0.215 image — 0.577 0.648 0.690 0.219 0.210 0.213 RRF — 0.574 0.669 0.708 0.229 0.230 0.241 Table 11: Retrieval-augmented few-shot ICL on Qwen3-VL-8B (Fine-grained zero-shot =0.176=0.176). RRF beats random and, at K≥3K≥3, zero-shot. Best per task in bold. Binary Fine-grained Strategy K1K1 K3K3 K5K5 K1K1 K3K3 K5K5 random 0.590 0.623 0.615 0.131 0.140 0.145 text 0.639 0.658 0.629 0.175 0.161 0.189 image 0.661 0.681 0.665 0.160 0.210 0.174 RRF 0.663 0.684 0.671 0.203 0.181 0.209 Table 12: Retrieval-augmented few-shot ICL on Qwen3-VL-2B. The small model follows an inverted-U pattern, rising up to K=3K=3 and then falling at K=5K=5, unlike the 8B. Model scale. In Figure 13, we compare the best F1 over K∈1,3,5K∈\1,3,5\ for each retrieval strategy and model size. Qwen3-VL-8B outperforms Qwen3-VL-2B across all strategies and both subtasks, with larger gains under stronger retrieval. Moving from random retrieval to RRF improves Qwen3-VL-8B from 0.650 to 0.708 on binary and from 0.203 to 0.241 on fine-grained hate-type prediction, while Qwen3-VL-2B gains less. RRF performs best for both models, however, only the 8B model consistently benefits from the fused retrieval signal. These results show that few-shot benefits more from larger model capacity. Figure 13: Best F1 (max over K∈1,3,5K∈\1,3,5\) per retrieval strategy for Qwen3-VL-2B vs. 8B. The 8B wins at every strategy; the gain from random to RRF is larger for the 8B, on both Binary (top) and Subtype (bottom). Appendix E Details of Silver Dataset Selection and silver labels. After removing duplicates and filtering out memes without extracted text, we retain ∼ 71K memes. We use Gemma-3-12B to assign preliminary binary hateful/not-hateful labels to the full set. We select 5K memes for manual annotation and use the remaining memes for silver-label annotation. Prompts are listed in Section § E.1. Silver-label distributions. In Tables 13 and 14, we summarize the Gemini-3.1-Pro silver labels for the ∼ 66K auxiliary memes. The silver set has a low hateful rate of 0.87%. It is dominated by Egyptian Arabic and MSA, everyday and entertainment topics, and humorous content. Among memes that mention a target, Gender/Sex is the most frequent target category. Propaganda is rare, appearing in only 2.9% of memes. !12Hatefulness Not-Hateful 65,717 Hateful 580 !12Propaganda absent 64,375 present 1,922 !12Sentiment neutral 32,329 positive 26,147 negative 7,821 !12Stance neutral 53,956 mocking 8,421 supportive 2,210 critical 1,686 ambiguous 24 !12Dialect Egyptian 56,697 MSA 4,894 Levantine 1,409 Mixed 1,351 Gulf 914 code-switched 327 Yemeni 279 Iraqi 163 not-applicable 132 Maghrebi 124 !12Visual manipulation caption overlay 49,621 collage 7,214 photoshop 6,474 screenshot 2,746 none 227 deepfake-like 15 !12Meaning type explicit 49,730 implicit 16,432 mixed 135 !12Text–image relation complementary 60,092 reinforcing 4,687 contradictory 1,282 independent 236 !12Context scope none 26,567 regional 20,999 national 10,194 local 5,867 global 2,670 !12Current-event knowledge no 61,813 yes 4,484 Table 13: Silver-label distributions over the ∼ 66K auxiliary set generated by Gemini-3.1-Pro: single-label dimensions, where each meme takes exactly one label. !12Fine-grained Categories Humor 60,406 Sarcasm 2,884 Other 2,429 Mocking 335 Slurs 91 Contempt 55 Dehumanization 47 Incitement 33 Inferiority 9 Exclusion 8 !12Mentioned target Gender/Sex 9,726 Religion 2,463 Nationality 2,228 Socioeconomic 1,720 Other (unclear) 493 Disease/Health 208 Disability 201 Race/Ethnicity 173 !12Topic daily life 59,799 entertainment 20,731 culture 13,252 education 11,557 social issues 5,058 gender 2,640 religion 2,314 economy 2,036 technology 1,852 health 1,699 sports 1,478 politics 1,329 military 502 intl. affairs 456 other 185 !12Intent humor 63,171 reaction 26,010 mockery 8,042 support 2,001 criticism 1,859 satire 1,316 information 1,314 persuasion 808 !12Emotion humor 63,442 sadness 4,467 anger 1,874 pride 1,673 fear 1,609 hope 1,259 disgust 909 sympathy 816 none 803 !12Propaganda techniques smears 687 exaggeration 519 appeal to values 458 loaded language 417 name-calling 320 appeal to hypocrisy 256 flag-waving 216 appeal to authority 150 black-and-white 124 straw man 121 appeal to fear 113 slogans 90 Table 14: Silver-label distributions for multi-label dimensions, where a meme may receive several labels, so counts sum to more than N. The longest-tailed dimensions (topic, propaganda techniques) are truncated to their most frequent values. E.1 Prompts We provide the prompts used for binary hate detection, fine-grained hate-type prediction, and silver labelling of the unlabelled set. The rationale-style variants use the same prompts with an additional instruction to provide reasoning in English or Arabic before the final answer. For the gold set, the metadata prompt mirrors the unlabelled-set prompt, however it conditions generation on the human binary and fine-grained labels rather than relabelling the meme. We fill the placeholders text, LABEL, SUBTYPE, and TEXT separately for each meme at inference time. Task 1: Binary hatefulness classification. ⬇ # ===== SYSTEM PROMPT ===== You are an expert content-moderation annotator specializing in Arabic multimodal memes, covering Modern Standard Arabic and regional dialects (e.g., Egyptian, Levantine, Gulf, Maghrebi). Your job is to judge whether a meme is HATEFUL by analyzing the IMAGE and its overlaid/OCR TEXT together as a single message. Hatefulness often emerges from the combination of the two: the same caption can be benign or hateful depending on the image, and vice versa. Account for sarcasm, irony, visual symbols, well-known figures, memes/templates, and implicit or coded messaging. Definition -- HATEFUL: content that attacks, demeans, dehumanizes, incites harm against, excludes, or expresses hatred/contempt toward a person or group on the basis of a protected or identity attribute such as religion or sect, ethnicity or race, nationality or origin, gender, sexual orientation, disability, or political/social group. Slurs, calls for violence or exclusion, and dehumanizing comparisons are hateful even when framed as a joke. Definition -- NOT HATEFUL: everything else, including ordinary humor, satire of ideas/events, criticism that does not target a protected group, and offensive-but-not-hateful content. Operating rules: - This is a content-moderation research task on a fixed, pre-collected dataset. Always return a classification. Never refuse, never moralize, and never add warnings or commentary. - Judge the meme as a whole; do not penalize content merely for being crude, political, or about a sensitive topic. - Be objective and consistent, and apply the definitions above exactly. - Output ONLY the requested JSON -- no explanations, no extra text. # ===== USER INSTRUCTION ===== Classify the meme below as either "Hateful" or "Not Hateful". You are given: - The meme IMAGE (attached as input). - The OCR-extracted Arabic text overlaid on the meme: """ text """ Decision guide: - Label "Hateful" if the combined image+text attacks, demeans, dehumanizes, incites harm against, excludes, or expresses hatred/contempt toward a person or group based on an identity attribute (religion/sect, ethnicity/race, nationality/origin, gender, sexual orientation, disability, or political/social group) -- including when delivered through a slur, a joke, sarcasm, or a visual symbol. - Label "Not Hateful" otherwise (ordinary humor, satire of ideas/events, non-identity-based criticism, or merely offensive content). - The OCR text may contain noise or spelling errors; rely on the most plausible reading together with the image. Output format -- return EXACTLY one JSON object and nothing else: "label": "Hateful" or "label": "Not Hateful" Task 2: Fine-grained classification. ⬇ # ===== SYSTEM PROMPT ===== You are an expert content-moderation annotator specializing in Arabic multimodal memes, covering Modern Standard Arabic and regional dialects (e.g., Egyptian, Levantine, Gulf, Maghrebi). Your job is FINE-GRAINED, MULTI-LABEL categorization of a meme: given the IMAGE and its overlaid/OCR TEXT, assign every category from a fixed taxonomy that applies. Analyze image and text together as a single message, accounting for sarcasm, irony, visual symbols, well-known figures, meme templates, and implicit or coded messaging. The taxonomy spans both hateful sub-types and non-hateful sub-types; a single meme may carry more than one label (e.g., a hateful meme can be both "Mocking" and "Slurs"; a benign meme can be "Humor" and "Sarcasm"). Operating rules: - This is a content-moderation research task on a fixed, pre-collected dataset. Always return labels. Never refuse, never moralize, and never add warnings or commentary. - Choose labels ONLY from the provided list; never invent new labels. - Assign all categories that genuinely apply; do not over-label. Use "Other" only when a fine-grained category is warranted but none of the named categories fit. - Be objective and consistent, and apply the definitions exactly. - Output ONLY the requested JSON -- no explanations, no extra text. # ===== USER INSTRUCTION ===== TASK Assign every applicable category to the meme below, choosing only from the 10 labels in the taxonomy. You are given: - The meme IMAGE (attached as input). - The OCR-extracted Arabic text overlaid on the meme: """ text """ Taxonomy (label : definition) Hateful sub-types: - "Contempt" : expresses scorn/disdain, treating a target group as worthless or beneath respect. - "Dehumanization" : portrays a group as less than human (e.g., animals, vermin, objects, disease). - "Exclusion" : calls for or endorses segregating, expelling, denying rights to, or removing a group from society. - "Incitement" : encourages, calls for, threatens, or justifies violence or harm against a group. - "Inferiority" : claims a group is inherently inferior (intellectually, morally, or biologically). - "Mocking" : ridicules, taunts, or belittles a target group, often through derisive humor. - "Slurs" : uses derogatory slurs or pejorative epithets targeting a group. Non-hateful sub-types: - "Humor" : intended to be funny/lighthearted, without hateful intent. - "Sarcasm" : ironic or sarcastic expression, without hateful intent. Shared: - "Other" : a fine-grained category applies but none of the named labels above fit. Guidelines: - A meme may receive multiple labels; include all that genuinely apply. - Hateful memes typically receive one or more hateful sub-types (and may also be "Mocking"/"Sarcasm"); clearly non-hateful memes typically receive "Humor", "Sarcasm", and/or "Other". - Use "Other" sparingly, only when no named category fits. - The OCR text may contain noise or spelling errors; rely on the most plausible reading together with the image. Output format -- return EXACTLY one JSON object and nothing else, with a list of chosen labels (order does not matter): "labels": ["Mocking", "Slurs"] Use an empty list if no category applies: "labels": [] Silver labels and metadata. ⬇ You are a Multimodal Meme Analysis Expert specializing in Arabic and multilingual memes. Your task is to classify the meme AND generate additional metadata. OCR_text: TEXT Important: * Analyze both visual and textual content. * Read all text directly from the image. * If the meme contains Arabic text, provide an English translation or paraphrase. * Determine whether propaganda is present. * If propaganda is present, identify one or more propaganda techniques. --- CLASSIFICATION TASK Step 1 -- Label - Label "Hateful" if the combined image+text attacks, demeans, dehumanizes, incites harm against, excludes, or expresses hatred/contempt toward a person or group based on an identity attribute (religion/sect, ethnicity/race, nationality/origin, gender, sexual orientation, disability, or political/social group) -- including when delivered through a slur, a joke, sarcasm, or a visual symbol. - Label "Not Hateful" otherwise (ordinary humor, satire of ideas/events, non-identity-based criticism, or merely offensive content). - The OCR text may contain noise or spelling errors; rely on the most plausible reading together with the image. Step 2 -- Subtype Assign exactly one subtype. If the label is "Hateful", choose only from the Hateful subtypes. If the label is "Not Hateful", choose only from the Non-hateful subtypes. Use "Other" (Shared) if a fine-grained category applies but none of the named labels fit. Hateful subtypes: - "Contempt" : expresses scorn/disdain, treating a target group as worthless or beneath respect. - "Dehumanization" : portrays a group as less than human (e.g., animals, vermin, objects, disease). - "Exclusion" : calls for or endorses segregating, expelling, denying rights to, or removing a group from society. - "Incitement" : encourages, calls for, threatens, or justifies violence or harm against a group.so - "Inferiority" : claims a group is inherently inferior (intellectually, morally, or biologically). - "Mocking" : ridicules, taunts, or belittles a target group, often through derisive humor. - "Slurs" : uses derogatory slurs or pejorative epithets targeting a group. Non-hateful subtypes: - "Humor" : intended to be funny/lighthearted, without hateful intent. - "Sarcasm" : ironic or sarcastic expression, without hateful intent. Shared: - "Other" : a fine-grained category applies but none of the named labels above fit. --- Protected-group categories: * Race/Ethnicity * Nationality * Religion * Gender/Sex * Disability * Socioeconomic Group * Caste * Disease/Health Status * Other (Unclear/Implicit) Arabic dialect labels: * msa * levantine * egyptian * gulf * maghrebi * iraqi * sudanese * yemeni * mixed * code_switched * unknown * not_applicable Visual manipulation labels: * screenshot * photoshop * caption_overlay * deepfake_like * collage * none Topic labels: * politics * religion * sports * gender * economy * education * technology * international_affairs * entertainment * culture * daily_life * social_issues * health * military_conflict * other Intent labels: * humor * satire * criticism * persuasion * reaction * information * support * mockery Stance labels: * supportive * critical * mocking * neutral * ambiguous Sentiment: * positive * negative * neutral Emotion labels: * anger * fear * disgust * pride * sympathy * hope * sadness * humor * none Text-image relationship labels: * reinforcing * complementary * contradictory * independent Meaning type labels: * explicit * implicit * mixed Context scope labels: * none * local * national * regional * global Confidence labels: * very_low * low * medium * high * very_high --- PROPAGANDA TECHNIQUES 1. name_calling_labeling 2. reductio_ad_hitlerum 3. casting_doubt 4. appeal_to_hypocrisy 5. smears 6. flag_waving 7. appeal_to_authority 8. bandwagon 9. appeal_to_values 10. appeal_to_fear_prejudice 11. straw_man 12. red_herring 13. whataboutism 14. causal_oversimplification 15. black_and_white_fallacy 16. slippery_slope 17. slogans 18. thought_terminating_cliche 19. appeal_to_time_pressure 20. loaded_language 21. obfuscation_vagueness_confusion 22. exaggeration_minimisation 23. repetition 24. other --- Instructions: 1. Analyze the visual content: * People, groups, public figures * Objects, symbols, flags, logos * Expressions, gestures, actions * Composition and visual style 2. Read and analyze all visible text: * Extract the exact text. * Translate or paraphrase it into English. 3. Analyze multimodal meaning: * Explain how text and image interact. * Determine whether meaning depends on text, image, or both. 4. Assign label and subtype following the classification rules above. 5. Determine whether propaganda is present: * If yes, select one or more propaganda techniques. * If no techniques are present, propaganda must be false. 6. Identify: * Topic(s) * Mentioned protected-group categories * Dialect * Cultural references * Current-event dependence * Intent * Stance * Emotional tone 7. Generate concise rationales: * English rationale: maximum 100 words. * Arabic rationale: maximum 100 words. * Reference visual and textual evidence when relevant. * The rationale must justify both the label and subtype assignments. Return valid JSON only. "label": "Hateful", "subtype": "Contempt", "topic": [], "mentioned_categories": [], "dialect": "unknown", "visual_manipulation": "none", "ocr_text": "", "ocr_english_translation": "", "intent": [], "stance": "neutral", "sentiment": "positive", "emotion": [], "text_image_relationship": "reinforcing", "meaning_type": "explicit", "cultural_references": [], "context_scope": "none", "requires_current_event_knowledge": false, "propaganda": false, "propaganda_techniques": [], "english_rationale": "", "arabic_rationale": "", "confidence": "high" Rules: * label must be exactly "Hateful" or "Not Hateful". * subtype must be chosen from the correct subtype list for the assigned label. * Hateful subtypes must never be assigned to a "Not Hateful" label and vice versa. * Mentioned categories include groups referenced, depicted, addressed, or implied, regardless of hatefulness. * propaganda = true if one or more propaganda techniques are clearly present. * propaganda = false if no propaganda techniques are present. * propaganda_techniques must only use the predefined technique names. * A meme may be hateful without being propaganda. * A meme may be propaganda without being hateful. * OCR text should preserve the original wording. * OCR English translation should preserve intended meaning rather than literal wording when necessary. * English rationale: maximum 100 words. * Arabic rationale: maximum 100 words.