Paper deep dive
Adapting Reinforcement Learning with Chain-of-Thought Supervision for Explainable Detection of Hateful and Propagandistic Memes
Mohamed Bayan Kmainasi, Mucahid Kutlu, Ali Ezzat Shahroor, Abul Hasnat, Firoj Alam
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 95%
Last extracted: 6/20/2026, 5:20:11 AM
Summary
The paper proposes a reinforcement learning-based post-training methodology using Group Relative Policy Optimization (GRPO) to improve the explainable detection of hateful and propagandistic memes in Multimodal Large Language Models (MLLMs). The approach utilizes Chain-of-Thought (CoT) distillation from strong models (like GPT-4o) to provide reasoning traces and employs a multi-stage pipeline involving Supervised Fine-Tuning (SFT) and GRPO with task-specific rewards (classification accuracy, format compliance, and reasoning-length regularization). The method was evaluated on the Hateful Memes (English) and ArMeme (Arabic) benchmarks, demonstrating significant improvements in both classification accuracy and the quality of natural-language explanations.
Entities (9)
Relation Signals (4)
GPT-4o → generates → Chain-of-Thought
confidence 100% · The explanations were generated using GPT-4o, motivated by evidence that advanced generative models produce high-quality rationales
ArMeme → isusedfor → Propaganda Detection
confidence 100% · ArMeme is an Arabic meme dataset designed for propaganda detection.
Chain-of-Thought → providessupervisionfor → Multimodal Large Language Models
confidence 95% · The training of such models emphasizes CoT supervision and reinforcement learning (RL) techniques.
Group Relative Policy Optimization → optimizes → Multimodal Large Language Models
confidence 90% · improves classification performance and reference-based explanation quality in thinking-based MLLMs via task-specific rewards and Group Relative Policy Optimization (GRPO).
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Hateful and propagandistic memes exploit the interplay between images and text to convey harmful intent that neither modality reveals alone. Although thinking-based multimodal large language models (MLLMs) have advanced vision-language understanding, their application to meme content moderation remains underexplored. We propose a reinforcement learning-based post-training method that improves classification performance and reference-based explanation quality in thinking-based MLLMs via task-specific rewards and Group Relative Policy Optimization (GRPO). Concretely, we (i) conduct a systematic empirical study of off-the-shelf MLLMs for hateful and propagandistic meme understanding across English and Arabic benchmarks, (ii) extend existing meme datasets with weakly supervised chain-of-thought (CoT) rationales via distillation and multi-LLM fine-grained propaganda annotations, (iii) introduce a GRPO-based objective with thinking-length regularization that jointly optimizes classification accuracy and explanation quality, and (iv) investigate self-supervised GRPO on unlabeled memes using consensus-based pseudo-labels. Experiments on the Hateful Memes and ArMeme benchmarks show that our approach improves over previously reported results on FHM accuracy (up to +2.1%, from 79.9% to 82.0%) and on ArMeme macro-F1 (up to +7.6 points, from 0.536 to 0.612 with explanations; +6.1 compared to the original ArMeme benchmark), while also generating natural-language explanations. On ArMeme, sequence-classification baselines remain stronger in terms of raw accuracy, whereas our approach provides more balanced per-class performance along with explanations. We publicly release our code, data extensions, and evaluation resources.
Tags
Links
- Source: https://arxiv.org/abs/2606.15307v1
- Canonical: https://arxiv.org/abs/2606.15307v1
Trouble viewing inline? Open PDF directly →
Full Text
112,597 characters extracted from source content.
Expand or collapse full text
Adapting Reinforcement Learning with Chain-of-Thought Supervision for Explainable Detection of Hateful and Propagandistic Memes Mohamed Bayan Kmainasi a , Mucahid Kutlu b , Ali Ezzat Shahroor a , Abul Hasnat c,d , Firoj Alam a a Qatar Computing Research Institute, Doha, Qatar b Qatar University, Doha, Qatar c APAVI.AI, France d Blackbird.AI, New York, NY, USA Abstract Hateful and propagandistic memes exploit the interplay between images and text to convey harmful intent that neither modality reveals alone. Although thinking-based multimodal large language models (MLLMs) have advanced vision-language understanding, their application to meme content moderation remains underexplored. We propose a reinforcement learning-based post-training method that improves classification performance and reference-based explanation quality in thinking-based MLLMs via task-specific rewards and Group Relative Policy Optimization (GRPO). Concretely, we (i) conduct a systematic empirical study of off-the- shelf MLLMs for hateful and propagandistic meme understanding across English and Arabic benchmarks, (i) extend existing meme datasets with weakly supervised chain-of-thought (CoT) rationales via distillation and multi-LLM fine-grained propaganda annotations, (i) introduce a GRPO-based objective with thinking-length regularization that jointly optimizes classification accuracy and explanation quality, and (iv) investigate self-supervised GRPO on unlabeled memes using consensus-based pseudo-labels. Experiments on the Hateful Memes and ArMeme benchmarks show that Our approach improves over previously reported results on FHM accuracy (up to +2.1%, from 79.9% to 82.0%) and on ArMeme macro-F1 (up to +7.6 points, from 0.536 to 0.612 with explanations; +6.1 compared to the original ArMeme benchmark), while also generating natural-language explanations. On ArMeme, sequence-classification baselines remain stronger in terms of raw accuracy, whereas our approach provides more balanced per-class performance along with explanations. We publicly release our code, data extensions, and evaluation resources. WARNING: This paper contains examples which may be disturbing to the reader Keywords: hateful meme detection, propaganda detection, multimodal large language models, reinforcement learning, chain-of-thought reasoning 1. Introduction Memes blend images and text with humor and cultural refer- ences, forming a predominant mode of communication on social media [66,3]. While often harmless, they can be exploited to spread hate speech, disinformation, and propaganda [77]. The use of humor and irony may trivialize toxic content, potentially normalizing hostile views [72], while memes also serve as pow- erful tools for political persuasion and manipulation [3,61]. Given the pervasive role of social media, timely detection and moderation of such multimodal content are crucial for safer online environments. This has motivated growing interest in automated approaches that leverage multimodal understanding for content moderation at scale. Recently, multimodal large language models (MLLMs) have significantly advanced vision–language understanding [82,90]. Thinking-based MLLMs represent the recent paradigm in mul- timodal reasoning, generating explicit intermediate reasoning steps before producing a final answer [93]. This paradigm is particularly relevant for memes, where meaning is often implicit ∗ Corresponding author Email addresses: mkmainasi@hbku.edu.qa (Mohamed Bayan Kmainasi), mucahidkutlu@qu.edu.qa (Mucahid Kutlu), alsh34060@hbku.edu.qa (Ali Ezzat Shahroor), mhasnat@gmail.com (Abul Hasnat), fialam@hbku.edu.qa (Firoj Alam) and depends on image–text interaction rather than unimodal cues. The training of such models emphasizes CoT supervision and reinforcement learning (RL) techniques [19]. However, whether such reasoning capabilities can be effectively transferred to sub- jective, culturally situated tasks like meme moderation remains an open question. Significant progress has been achieved through agentic rea- soning frameworks [55,52], few-shot adaptation [14,37], and supervised fine-tuning (SFT) on multimodal datasets [43]. How- ever, only very recent or concurrent work [60] has begun to explore CoT, SFT, and RL for meme analysis. In particular, thinking-based MLLMs remain underexplored for hateful and propagandistic meme analysis, and the interaction between CoT supervision, fine-grained labeling, and RL-based post-training has not yet been systematically studied. Recent RL methods such as GRPO [19] have demonstrated strong results for structured reasoning tasks including mathemat- ical problem solving and code generation, yet their application to subjective multimodal classification remains largely unexplored. Meme moderation requires jointly optimizing classification per- formance and explanation quality through a structured output format, a setting where standard SFT alone provides limited control over the balance between prediction correctness and ra- tionale faithfulness. This motivates us to investigate RL-based post-training with composite reward functions tailored for ex- arXiv:2606.15307v1 [cs.CL] 13 Jun 2026 Figure 1: Overview of the proposed methodology. Starting from raw memes, we derive binary and fine-grained supervision, OCR text, and guidelines to construct instruction-following datasets. Weak supervision from a strong MLLM distills step-by-step CoT rationales. The model is trained via a two-stage pipeline: SFT warm- up followed by GRPO-based RL, jointly optimizing classification correctness, output-format compliance, reference-based explanation similarity, and reasoning-length regularization. Acc. = accuracy; MET = METEOR; Len = explanation length; Fmt = format compliance. plainable meme understanding. To address this research gap, and as illustrated in Figure 1, we propose a reasoning-centric training methodology (lower part) for explainable meme understanding grounded in CoT su- pervision and RL. The methodology employs a multi-stage post- training pipeline. First, SFT warm-up aligns the model with gold labels, explanations, and distilled reasoning traces. This is fol- lowed by supervised GRPO that jointly optimizes classification correctness, output-format compliance, reference-based expla- nation similarity, and reasoning-length regularization. Finally, self-supervised GRPO leverages consensus-based pseudo-labels from unlabeled memes. To assess generalizability, we evaluate the methodology across two distinct tasks, hate and propaganda detection, and two languages, English and Arabic, using the associated datasets Facebook Hateful Memes (FHM) [42] and Arabic Propagandistic Memes (ArMeme) [5]. This work builds on our prior research on multimodal meme analysis [43,45]. In particular, Kmainasi et al.[45]introduced GRPO-based post-training for thinking-based MLLMs, showing that reinforcement learning with task-specific rewards improves both classification and explanation quality on the English Hate- ful memes benchmark. However, that study was limited to a single dataset (i.e., hateful meme) with binary labels under a fully supervised setting, leaving its generalizability across tasks, languages, and training regimes largely unexplored. More broadly, this work represents, to the best of our knowledge, the first systematic study of GRPO in multimodal reasoning under cross-lingual, fine-grained, and self-supervised settings. This motivates the following research questions: RQ1: Does incorporating fine-grained labels (e.g., protected categories, at- tack types, propaganda techniques) during SFT improve down- stream performance when optimized with GRPO? RQ2: To what extent do synthetic CoT rationales distilled from stronger MLLMs enhance classification accuracy and explanation qual- ity? RQ3: Do synthetic fine-grained annotations produced by multiple LLMs outperform single-source labeling? RQ4: How effective is GRPO compared to SFT alone, and what roles do SFT warm-up and thinking-length regularization play in training stability? RQ5: Can GRPO be extended to a self-supervised setting using consensus-based pseudo-labels in place of human annotations? To address these questions, we substantially extended our prior work Kmainasi et al.[45]with the following contributions: •Cross-task and cross-lingual generalization: We extend the methodology, proposed by Kmainasi et al.[45], to Ara- bic propagandistic meme classification on the ArMeme dataset, introducing a dual-annotator pipeline with LLM- based consolidation to construct fine-grained propaganda labels in a previously unlabeled setting (RQ1, RQ3). 2 •Comprehensive empirical analysis: We provide exten- sive ablations, unimodal baselines, and cross-dataset eval- uations, systematically analyzing the interplay between CoT supervision, fine-grained labeling, and RL-based op- timization (RQ2). •Stabilized RL optimization: We introduce a threshold- based thinking-length rewardR think during GRPO that penalizes overly short CoT completions but saturates be- yond a minimum length. This discourages reward hack- ing, such as empty thinking traces, without promoting verbosity (RQ4). •Self-supervised GRPO: We explore consensus-based pseudo-labeling from multiple stochastically sampled re- sponses, enabling RL training without human annotations (RQ5). Our findings are summarized as follows: (RQ1) Fine-grained supervision consistently improves the SFT warm-up stage. On FHM, adding fine-grained labels and explanations (Cls+FG+Exp: Table 9) improves macro-F1 from 0.75 to 0.77. On ArMeme, the gain is larger, with macro-F1 increasing from 0.43 to 0.51 (Table 10). (RQ2) Distilled CoT provides additional gains over fine- grained SFT. Macro-F1 increases from 0.77 to 0.78 on FHM and from 0.51 to 0.54 on ArMeme when CoT distillation is added to the fine-grained SFT setting (Cls+FG+Exp,CoTD vs. SFT Cls+FG+Exp: Tables 9, 10). (RQ3) Multi-LLM fine-grained annotation is effective for ArMeme improves SFT macro-F1 from 0.43 to 0.51, an absolute gain of 8.0 points (Table 10). (RQ4) GRPO further improves performance when initial- ized from CoTD warm-up and trained withR think . This setting reaches 0.80 macro-F1 on FHM (Table 9), and 0.597 macro-F1 on ArMeme (Table 10). (RQ5) Self-supervised GRPO shows mixed effects. It im- proves ArMeme macro-F1 by 1.5 points, from 0.597 to 0.612, but slightly reduces FHM performance, with a 1.0-point drop in macro-F1 (Table 11). This may reflect data distribution mis- match in the FHM unlabeled pool, which is aggregated from different English meme datasets. We release all code, data extensions, prompting templates, and evaluation resources. 1 The remainder of this paper is or- ganized as follows. Section 2 reviews related work on meme moderation, RL for MLLMs, and self-supervised training. Sec- tion 3 describes the datasets and annotation pipelines. Section 4 presents the proposed methodology. Section 5 details the ex- perimental setup. Section 6 reports results and ablation studies. Section 7 discusses broader implications and limitations, and Section 8 concludes with directions for future work. 2. Related Work 2.1. Multimodal meme content moderation Early meme detection systems relied on unimodal cues such as OCR-based text features with traditional ML [70,6,11,4,77] 1 https://github.com/MohamedBayan/MemeReason or CNN-based image classifiers [78]. Most existing research has focused on binary classification [13,33], supported by bench- marks such as hateful and propagandistic memes [42,22] and HarMeme [67]. A limited number of works have explored fine- grained annotation [58, 27]. More recently, several studies have moved beyond detec- tion toward explainability and enhanced multimodal reasoning. Grasso et al.[29]augmented classification with knowledge- graph reasoning, Cao et al.[14]addressed few-shot detection by modularizing LLM components, and Lin et al.[50]proposed a multi-agent debate framework. Kmainasi et al.[43]extended Hateful Memes with human-verified rationales for explainable MLLM training. Despite these advances, systematic investigations to enhance CoT reasoning for meme understanding remain limited. Existing work has explored CoT in specific settings such as multi-hop reasoning for misogynistic memes [46] and distilled abductive reasoning [51]. At the same time, recent work has highlighted the fragmentation of meme understanding across tasks and lan- guages [74], motivating unified modeling approaches. However, the use of RL/GRPO for multimodal meme detection remains largely underexplored, with only concurrent work [60] investi- gating GRPO for meme moderation. Furthermore, propagan- distic meme analysis has received comparatively less attention, especially in Arabic contexts [77, 5]. 2.2. RL for MLLMs RL-based post-training has emerged as a key technique for aligning MLLMs with task-specific objectives [91]. Methods range from RLHF with PPO [65,73] and preference-based DPO [68] to the more recent GRPO [19], which has shown strong results in mathematical reasoning [75], self-training [69], and code generation [16]. A detailed comparison of these algorithms is provided in Section 4.1. In this work, we expand GRPO to hateful and propagandistic meme analysis and study how CoT supervision interacts with RL-based post-training. 2.3. Self-supervised Training for LLMs Recent work has explored self-supervised strategies to lever- age unlabeled data during fine-tuning. SemiEvol [57] propa- gates supervision from small labeled sets, while SPO [48] con- structs synthetic preference data. M-UPT [88] and V-Zero [86] introduce fully self-supervised RL using majority voting and co-evolutionary training, respectively. We build on this di- rection in Section 4.5, investigating whether consensus-based pseudo-labeling can provide reliable supervision for meme clas- sification. 3. Datasets This section presents the datasets used in this study, includ- ing the benchmark descriptions, annotation resources, and data construction procedures. We describe the two evaluation bench- marks (Section 3.1 and 3.2), the associated natural language rationales (Section 3.3), the fine-grained annotation pipeline 3 Table 1: Distribution of labels in the Hateful Memes dataset. Class Label Train DevTestTotal Non-hateful5,481340 1,2507,071 Hateful3,0192007503,969 Total8,500540 2,000 11,040 Table 2: Distribution of labels in the ArMeme dataset. Class LabelTrain DevTest Total Not Propaganda2,634384746 3,764 Propaganda972141275 1,388 Not-meme1993057286 Other2022956287 Total4,007584 1,134 5,725 developed for ArMeme (Section 3.4), the chain-of-thought dis- tillation process (Section 3.5), and the unlabeled data collected for self-supervised training (Section 3.6). 3.1. English Hateful Memes Hateful Memes [42] is a widely used benchmark for multi- modal hate speech detection, containing≈11K memes curated to ensure that effective classification requires joint multimodal understanding. The dataset includes a mixture of synthetically generated and real-world memes with a balanced distribution between hateful and non-hateful content. We use the explanation augmented version proposed by [43], which includes human- verified explanations for the binary labels. 2 Table 1 summarizes the dataset statistics. 3.2. Arabic Propagandistic Memes (ArMeme) ArMeme [5] is an Arabic meme dataset designed for pro- paganda detection. It includes four labels: propaganda, not- propaganda, not-meme, and other. Similarly, for this work, we use the explanation augmented version of the dataset proposed by [43], which provides English explanations. We adopt this ver- sion as prior studies have shown that LLMs often generate more reliable outputs when prompted in English than in non-English languages [44]. In addition, the explanations in the test set were manually verified. The label distribution of the dataset is shown in Table 2. 3.3. Natural Language Rationales As mentioned earlier, both datasets are accompanied by natural language rationales released by Kmainasi et al.[43]. 3 An example explanation is presented in Figure 1. The expla- nations were generated using GPT-4o, motivated by evidence that advanced generative models produce high-quality rationales closely aligned with human reasoning [34]. To ensure reliabil- ity, the generated rationales were manually validated by human annotators along multiple dimensions, including faithfulness, clarity, plausibility, and informativeness, achieving an average score of approximately 4 on a 5-point Likert scale. Furthermore, 2 We focus on the unseen splits, as these were augmented with explanations. 3 https://huggingface.co/datasets/QCRI/MemeXplain inter-annotator agreement was strong, with average agreement scores above 0.83 for ArMeme and 0.92 for FHM, indicating a high level of consistency in the annotations. 3.4. Fine-Grained Annotation Hateful memes dataset. For the hateful memes, we use publicly available annotations from Mathias et al.[58]categorizing each hateful instance by protected category (religion, race, sex, dis- ability, and nationality) and attack type (dehumanizing, inferior- ity, inciting violence, mocking, contempt, slurs, and exclusion). Propagandistic memes dataset. For the Arabic propagandistic memes, no publicly available fine-grained propaganda anno- tations exist. We therefore develop a multi-LLM annotation pipeline following Hasanain et al.[31]. The pipeline operates in two stages: Stage 1: Dual independent annotations. Each meme is anno- tated independently by two models: GPT-4.1 and Llama-4- Scout. Both models receive the same system prompt defining 23 propaganda techniques and produce a structured list of pro- paganda techniques along with short rationales (1–4 sentences) grounded in the meme’s image, text, binary label, and explana- tion. Stage 2: Consolidation via Gemini-3-Pro. We employ Gemini- 3-Pro as a consolidator to merge the two independent outputs into a single final annotation. The consolidator evaluates over- laps, resolves disagreements, validates unique techniques from either annotator, and adds missing techniques when justified. Human and LLM agreement. To assess the reliability of the LLM-generated annotations, we manually annotated a subset of approximately 150 memes. Each meme was independently labeled by two annotators with prior experience in the task. Fol- lowing Hasanain et al.[32], we unified the annotated labels before computing agreement between human and LLM annota- tions. Using Gwet’s AC1 [30], we obtain an agreement score of 0.77. Overall, the results indicate strong consistency between human and LLM annotations, despite the subjective nature of propaganda meme annotation and the non-trivial human dis- agreement reported in prior work [32]. 3.5. Chain-of-Thought Distillation (CoTD) Training thinking-based MLLMs requires step-by-step rea- soning traces, which are expensive to collect from humans. Meanwhile, recent advances in commercial reasoning-oriented models have demonstrated strong reasoning capabilities in prac- tice [47,17]. Motivated by these observations, we adopt a distillation-based approach, referred to as chain-of-thought dis- tillation. We use GPT-4.1 to generate intermediate step-by-step rea- soning traces, as presented in Figure 1. The generation is condi- tioned on the meme image, extracted text, annotation guidelines, and both binary and fine-grained labels. These private reasoning sequences are used exclusively during training and are not ex- posed during inference. To prevent label leakage, the model is explicitly prompted to reason independently, without copying or 4 paraphrasing the reference explanations. Our approach follows prior work on CoTD for improving model reasoning [99]. CoT evaluation with LLM-as-judge. We evaluate CoT quality using an LLM-as-judge protocol [43]. Two vision–language models, InternVL3.5 [87] and Phi-3.5 [2], independently score each sample on a 5-point Likert scale. We select these models as a cost-effective alternative to proprietary judges. The evaluation uses four dimensions. Informativeness measures whether the CoT uses salient visual and textual evidence. Clarity assesses the logical structure of the reasoning. Plausibility evaluates con- sistency with human judgment. Faithfulness measures whether the reasoning is grounded in observable content. These scores assess the judged quality of the distilled traces as training supervision, not the faithfulness of the model’s latent reasoning process at inference time. We measured inter-judge agreement using the r ∗ wg( j) index [39]: r ∗ wg( j) = 1− S 2 X σ 2 mv , whereS 2 X is the variance between judges andσ 2 mv is the maxi- mum possible variance for a 5-point scale. As shown in Table 3, both judges assign average scores above 4.0 with agreement>0.93 across all dimensions and datasets, suggesting that the distilled CoT traces are of accept- able quality as training supervision. Table 3: LLM-as-a-judge evaluation of distilled CoT quality (1–5 Likert). IVL = InternVL3.5-8B; Phi = Phi-3.5-Vision;r ∗ wg( j) denotes inter-judge agreement. Metrics are grouped into correctness (faithfulness, plausibility) and communica- tion quality (clarity, informativeness). ArMemeHateful Memes MetricIVL Phi r ∗ w g( j) IVL Phi r ∗ w g( j) Faithfulness4.82 4.75 .9644.78 4.56 .944 Plausibility4.68 4.40 .9484.61 4.24 .934 Clarity4.72 4.72 .9574.73 4.54 .943 Informativeness 4.40 3.96 .9344.40 4.10 .942 Average4.66 4.46 .9514.63 4.36 .940 Human annotation. In addition, we manually verified a ran- dom subset of 200 samples from each dataset. Table 4 reports human agreement and average annotation scores for ArMeme and Hateful Memes across four dimensions: informativeness, clarity, plausibility, and faithfulness. The highr ∗ wg( j) values sug- gest substantial agreement among annotators, indicating that the evaluation criteria were generally interpreted consistently. The average scores are relatively higher for both datasets, suggest- ing that the LLM-based annotation and consolidation pipeline produced outputs that human evaluators generally judged to be informative, clear, plausible, and faithful. Human and LLM judge agreement. Cross-evaluator agreement between mean human and mean LLM scores was also high across both datasets, withr ∗ wg( j) ≥0.93 across the four dimen- sions. This suggests that the LLM-judge ratings were broadly aligned with human annotations. Human annotators and compensation. We recruited nine an- notators through a third-party company and compensated them at standard local rates. All annotators received detailed annota- tion guidelines and signed non-disclosure agreements in accor- dance with institutional policy. The annotator pool included both men and women, with educational backgrounds ranging from Bachelor’s to Master’s degrees. Each meme was independently annotated by three annotators. Table 4: Human evaluation scores on ArMeme and Hateful Memes datasets. Metrics are grouped into correctness (faithfulness, plausibility) and communica- tion quality (clarity, informativeness). ArMemeHateful r ∗ w g( j) Avg.r ∗ w g( j) Avg. Faithfulness0.89 4.28± 0.33 0.91 4.63± 0.28 Plausibility0.88 4.29± 0.33 0.91 4.65± 0.26 Clarity0.87 4.55± 0.35 0.92 4.63± 0.27 Informativeness 0.87 4.54± 0.34 0.91 4.67± 0.28 3.6. Unlabeled Data Construction To study self-supervised GRPO, we construct unlabeled training sets for both tasks. For the hateful meme task, we leverage several publicly available English meme datasets: MAMI (Multimedia Auto- matic Misogyny Identification; 11,000 memes) [27], Memotion (6,992 samples for offensiveness classification) [76], and MET- Meme (3,994 memes annotated for metaphor occurrence) [92]. Although these datasets are originally labeled, we discard all la- bels and treat the data as unlabeled to simulate a self-supervised setting. To avoid potential data overlap, we remove any dupli- cate or near-duplicate memes that overlap with the evaluation test sets, yielding a combined pool of 21,986 memes across the three datasets. We then apply the disagreement-based sampling strategy, described in Section 4.5, to select 2,000 memes (≈9.1% of the total pool). For Arabic propaganda memes, publicly available datasets are extremely limited. We therefore collect memes from social media platforms such as Facebook, Instagram, Pinterest, and X (Twitter), following the ArMeme dataset collection methodol- ogy [5]. Since direct API access is unavailable for Facebook, Instagram, and Pinterest, we adopt a semi-automatic approach in which images are manually loaded in a browser and auto- matically crawled. For X, we employ keyword-based crawl- ing targeting content related to public figures, celebrities, and political discussions. The collected images are then passed through a multi-stage filtering pipeline: (i) duplicate and near- duplicate removal using feature vectors extracted by a fine-tuned ResNet18 [35] with nearest-neighbor search (Euclidean distance ≤3.6); (i) OCR-based filtering via EasyOCR 4 to discard im- agess with no detectable text; and (i) an in-house meme-vs.- non-meme classifier to retain only valid memes. After removing all samples overlapping with the test sets, we curate a final pool of 15,652 memes. From this pool, we sample 2,000 memes 4 https://github.com/jaidedai/easyocr 5 (≈12.8% of the total) using the disagreement-based sampling strategy described in Section 4.5. 4. Methodology In this section, we present our RL-based methodology for hateful and propagandistic meme analysis using thinking-based MLLMs (Figure 1). The approach jointly trains MLLMs for classification, explanation generation, and step-by-step reason- ing through a structured output format and multi-stage training. We first provide the necessary background on RL-based post- training, then formulate the task and define the structured output format. We then describe the three training stages: (i) SFT warm- up, (i) supervised GRPO optimization with a composite reward function, and (i) self-supervised GRPO using consensus-based pseudo-labels. 4.1. Background 4.1.1. RL for language model post-training. RL has been leveraged for post-training MLLMs with dy- namic feedback [91]. In this paradigm, the language model is treated as a policyπ θ that generates a sequence of tokens given an input prompt. At each decoding step, the model selects a token (action) conditioned on the preceding context (state), and a scalar reward is assigned to the complete generation [40]. The training objective is to maximize expected reward while con- straining the updated policy to remain close to a reference policy, typically an SFT-initialized checkpoint, to prevent catastrophic forgetting [65]. Methods such as RLHF use PPO [73] or DPO [68] to align model behavior [65,96]. PPO trains a separate value network to estimate expected future reward and uses it as a variance-reduction baseline, which is effective but approximately doubles the memory cost during training. DPO circumvents this by reparameterizing the reward into a preference-based loss over paired examples, eliminating the need for both a value net- work and online rollouts, but limiting exploration to the offline preference data [68]. 4.1.2. Group Relative Policy Optimization (GRPO). More recently, GRPO [19] employs rule-based rewards to encourage high-quality CoT, achieving strong performance in mathematical reasoning [75], self-training [69], and code gen- eration [16]. Unlike PPO, GRPO does not require a learned value network. Instead, for each input, it samples a group of Kcandidate completions, computes a scalar reward for each, and uses the group mean reward as a baseline. The advantage of each completion is computed relative to this group average, and the policy is updated using a clipped surrogate objective with KL regularization against the reference policy [19]. This group-relative normalization avoids the overhead of a value net- work while retaining the benefits of online exploration through fresh rollouts at each training step. Several recent extensions have been proposed, including DAPO [95], which removes the KL penalty and uses dynamic sampling, and GTPO [83], which introduces group-level token-wise optimization. In this work, we adopt the standard GRPO formulation as our RL algorithm. 4.1.3. CoT supervision and thinking-based MLLMs. Thinking-based MLLMs represent the recent paradigm in multimodal reasoning, generating explicit intermediate reason- ing steps before producing a final answer [93]. This paradigm is particularly relevant for memes, where meaning is often implicit and depends on image–text interaction rather than unimodal cues. The training of such models emphasizes CoT supervision and reinforcement learning techniques [19]. In practice, CoT supervi- sion is often obtained through knowledge distillation: a stronger teacher model generates step-by-step reasoning traces for each training example, which are then used as supervision targets during fine-tuning of the student model [79,99]. Recent open- source thinking-based MLLMs, such as Qwen3-VL-Thinking [49] and OpenVL-Thinker [20], demonstrate that explicit CoT reasoning can substantially improve performance on complex multimodal benchmarks. In variants that include distilled CoT, the reasoning trace is placed inside<think></think>tags; in variants without CoT, empty tags are included to preserve the output format while excluding the reasoning content from loss computation. 4.2. Task Formulation Given a memexconsisting of an image and extracted text, the task is to predict a label and generate a rationale that justifies the prediction, as presented in Figure 1. To enable explicit reasoning while maintaining a structured and parseable output, we adopt a three-part generation format. Given an instruction- style prompt c(x), the model produces: y = <think> t </think> Label: ˆ ℓ Explanation: ˆe , wheretis a private CoT reasoning trace, ˆ ℓthe predicted label, andˆea natural language explanation. The reasoning tracet serves as an internal workspace that encourages the model to decompose the problem before committing to a prediction, while the explanationˆeprovides a human-readable justification that can be evaluated for faithfulness. LetD=(x i , y ⋆ i ) N i=1 denote the labeled dataset, wherey ⋆ i contains the gold label, explanation, and distilled teacher reasoning. 4.3. SFT Warm-Up Before applying reinforcement learning, the model must first be aligned with the target output format and task-specific pat- terns. Without proper warm-up, GRPO yields limited benefits, as RL alone has difficulty simultaneously learning the required output structure and the task-specific decision boundary. To provide a stable foundation for subsequent RL optimization, we initialize the MLLM via SFT onD. Letπ θ denote the autore- gressive model parameterized byθ. Parameters are optimized by minimizing: L SFT (θ) = E (c,y ⋆ )∼D h − logπ θ (y ⋆ | c) i (1) This stage aligns the model with gold labels, explanations, and distilled reasoning traces, providing initialization for subsequent GRPO. We consider three SFT variants: (i) SFT (Cls+Exp) with binary labels and explanations only; (i) SFT 6 (Cls+FG+Exp), which adds fine-grained annotations; and (i) SFT (Cls+FG+Exp, CoTD), which additionally includes distilled CoT reasoning in the <think> block. We refer to applying GRPO without this SFT warm-up, ini- tializing GRPO directly from the pretrained MLLM backbone, as the cold-start setting, which we use as a baseline in our ex- periments (Section 5). 4.4. GRPO Optimization Building on the SFT-initialized model, we apply reinforce- ment learning to further optimize the policy beyond what maximum-likelihood training can achieve. As described in Sec- tion 4.1, GRPO replaces the learned value network of PPO with a group-relative baseline. This design is particularly suited to our setting for three reasons: (i) it avoids the memory overhead of a separate value network, which is critical when fine-tuning large multimodal models under constrained GPU resources; (i) it operates online with fresh rollouts, enabling diverse reasoning exploration; and (i) its group-relative advantage normalization naturally accommodates our composite reward function, where multiple reward components with different scales must be balanced. The MLLM is treated as a stochastic policyπ θ . For each input, we sampleKcandidate outputsy 1 ,..., y K and compute a scalar reward for each. Given the structured output format defined in Section 4.2, the training must balance multiple objec- tives: predicting the correct label, generating a faithful explana- tion, maintaining format compliance, and producing sufficient reasoning traces. Standard SFT provides limited control over this balance, as the cross-entropy loss treats all output tokens equally regardless of their functional role [65]. To address this, we design a composite reward function that assigns distinct reward signals to each objective: R(y) = α fmt R fmt (y) + α lbl R lbl (y) + α exp R exp (y) + α think R think (y) + α met R met (y)(2) The individual reward components are: • R fmt (y) (format): encourages structurally consistent out- puts with reasoning, prediction, and explanation. • R lbl (y) (label correctness): rewards the correct class. • R exp (y) (explanation length): softly regularizes rationale length toward∼100 words via R exp = exp(−(L− 100) 2 /2σ 2 ) whereσ= 20, set empirically to balance conciseness and informativeness. • R think (y) (thinking length): discourages reward-hacking behavior such as empty or uninformative CoT traces by enforcing minimum reasoning sufficiency: R think (y) = 1,L think ≥ L min , exp − (L think −L min ) 2 2σ 2 think ,L think < L min , (3) whereL min = 150 andσ think = 50 are set empirically based on observed reasoning lengths during SFT. This one-sided formulation penalizes only reasoning traces shorter than the minimum threshold, thereby mitigating re- ward hacking [28] by discouraging overly short reasoning without incentivizing verbosity. Without this regulariza- tion, training shows a consistent reward-hacking pattern (Appendix Appendix C). The model learns to produce shorter or semantically empty CoT traces, compressing or omitting the<think>segment while still receiving high immediate reward. • R met (y) (semantic similarity): rewards METEOR similar- ity [9] to the gold rationale, encouraging explanations that are both relevant and faithful to the reference annotations. Weights for supervised GRPO are set empirically on the devel- opment set that maximized macro-F1 toα fmt =0.35,α lbl =0.35, α exp =0.08,α met =0.12, andα think =0.10, giving the highest prior- ity to classification correctness and output format compliance, which are prerequisites for downstream usability. Objective function. GRPO uses the group-average reward ̄ Ras baseline: ̄ R = 1 K K X k=1 R k ,A k = R k − ̄ R std(R j ) + ε Here,A k is the normalized advantage of thek-th completion relative to the group. Completions with above-average reward re- ceive positive advantage and are reinforced, while below-average completions are suppressed. The GRPO objective maximizes the clipped surrogate with KL regularization: J G (θ) = E " K X k=1 X t min r k,t A k , clip(r k,t , 1−ε, 1+ε)A k − β KL π θ (·| h k,t ) π ref (·| h k,t ) # (4) wherer k,t =π θ (a k,t | h k,t )/π θ old (a k,t | h k,t ) is the importance ratio, εthe clipping threshold that prevents excessively large policy updates,βthe KL coefficient that controls deviation from the reference policy, andπ ref a fixed reference policy from the SFT checkpoint. The clipping mechanism and KL penalty jointly ensure that each optimization step makes bounded updates to the policy, preventing training instability. 4.5. Self-Supervised GRPO While the supervised GRPO pipeline described above relies on human-annotated labels, obtaining such annotations at scale is expensive and often impractical, particularly when extending to new domains or languages where labeled data is scarce. Recent work has explored self-supervised strategies to leverage unla- beled data during fine-tuning. M-UPT [88] introduces fully self-supervised RL using majority voting over stochastic roll- outs, and V-Zero [86] extends this to a co-evolutionary setting. However, these approaches have primarily targeted structured reasoning domains. Inspired by this line of work, we extend our GRPO method to a self-supervised setting using consensus- based pseudo-supervision, and investigate whether this strategy 7 can provide reliable supervision for meme classification, where semantic interpretation depends on subtle image–text interac- tions. Hard-sample selection. Not all unlabeled samples are equally informative for RL training. Samples on which the model is already confident provide limited gradient signal, while gen- uinely ambiguous instances can expose decision boundaries that benefit from further optimization [94]. To identify such informative instances, we run inference at multiple temperatures 0,0.2,0.4,0.8,1.0and retain only samples where predictions are not in full agreement, ensuring each instance exposes gen- uine model uncertainty. Only samples with partial agreement (3/5 or 4/5) are retained, ensuring each training instance repre- sents a case of genuine model uncertainty rather than random noise. From each dataset, we sample 2,000 such disagreement examples. Consensus pseudo-reward. In the absence of gold labels, we derive supervision from the model’s own predictions through majority voting. For each input, the model generatesKcandi- dates. Letn c = P K j=1 I [ˆc(y j ) =c] andc maj =arg max c n c . If a strict majority exists (n c maj >⌊K/2⌋), completions predicting this class receive reward 1; otherwise all receive 0: R mv (y i ) = 1, n c maj >⌊K/2⌋ and ˆc(y i ) = c maj , 0, otherwise. The strict majority requirement ensures that pseudo-labels are assigned only when the model exhibits sufficient agreement, reducing the risk of reinforcing incorrect predictions. The final self-supervised reward combines consensus with structural rewards: R(y) = α fmt R fmt + α mv R mv + α exp R exp + α think R think (5) withα fmt =0.30,α mv =0.20,α exp =0.20,α think =0.30, set empiri- cally. Compared to the supervised setting, the thinking-length reward receives a higher weight to compensate for the absence of gold-label supervision, as reasoning collapse is more likely without explicit correctness feedback. The consensus-based pseudo-reward provides useful learning signal particularly for underrepresented categories when unlabeled data is in-domain. 5. Experimental Setup This section describes the models, baselines, training con- figurations, and evaluation protocol used across all experiments, covering both pretrained evaluation and the multi-stage post- training pipeline. 5.1. Models We use both open- and closed-weight MLLMs. Open-weight baselines include Llama-3.2-11B, Llama-4-Scout-17B [25], Qwen3-VL-8B [8], Gemma-3-12B [84], and Kimi-VL-A3B [85]. To assess whether an Arabic-centric model behaves differently on Arabic benchmark, we additionally evaluate Fanar-2-Oryx-IVU [26]. As a closed-weight model, we use GPT-4.1 [62]. For training, we use Qwen3-VL-8B-Thinking, a reasoning-enhanced checkpoint reported in [49] to perform strongly on benchmarks such as MMMU [97] and MathVista [56]. Experiments include both thinking and instruction-tuned variants. 5.2. Zero-Shot and CoT Baselines We evaluate all models under both zero-shot and CoT prompting. To ensure a fair comparison, we disable the internal thinking mode of thinking-enabled models in the non-CoT setting. This setup is intended to measure pretrained reasoning ability without task-specific fine-tuning. All prompts, including those used for Arabic memes, are written in English, following prior evidence that LLMs often produce more reliable and stable outputs when prompted in English [44]. For ArMeme, this is an evaluation design choice to improve consistency under the tested setup, not a deployment recommendation. 5.3. Unimodal Baselines To isolate the individual contribution of each modality, we fine-tune unimodal classifiers on both benchmarks. For vision, we fine-tune BEiT [10], ConvNeXt [54], DINOv2 [64], ResNet- 101 [36], Swin Transformer [53], and ViT [23]. For text, we fine-tune mBERT [21], QARiB [1], AraBERTv2 [7], Distil- BERT [71], and XLM-RoBERTa [18]. All models are trained under a unified setup to ensure fair comparison. 5.4. Multimodal Sequence Classification Baselines To contextualize the performance of our generative RL-based approach, we fine-tune Gemma-3-12B-IT [41] and Qwen3-VL- 8B-Instruct [93] on both training splits of both benchmarks, and additionally Qwen3-VL-8B-Thinking [93] on ArMeme, using standard cross-entropy classification heads. Unlike our genera- tive models, these baselines are trained solely on the classifica- tion objective and do not generate explanations. 5.5. Fine-Tuning Configurations This section describes the configurations for the three post- training stages introduced in Section 4 and also presented in Figure 1: (i) SFT warm-up, (i) supervised GRPO, and (i) self-supervised GRPO. SFT warm-up The three SFT variants are described in Sec- tion 4.3. In variants without CoT, empty<think></think> tags are included to preserve the thinking format while exclud- ing them from loss computation. Supervised GRPO We apply supervised GRPO from multiple initialization checkpoints to analyze the effect of SFT warm- up. We compare GRPO initialized directly from the pretrained backbone, referred ascold-start, with GRPO initialized from task-specific SFT checkpoints, thereby measuring the contribu- tion of supervised warm-up before reinforcement learning. We also evaluate the effect of the thinking-length reward R think . 8 Self-supervised GRPO. Self-supervised GRPO is initialized from the best supervised checkpoint and trained on 2,000 disagreement-based unlabeled samples per dataset (Section 4.5). 5.6. Training Setup We describe the key training hyperparameters below; full configurations for all stages, including unimodal baselines, are provided in Appendix Appendix A. All multimodal models are trained with partial parameter fine-tuning, where only the lan- guage model is updated while visual components remain frozen. Optimization was done using DeepSpeed ZeRO-3 in bfloat16 on 4 NVIDIA H200 GPUs, with gradient checkpointing and the vision encoder frozen. SFT uses a per-device batch size of 4, learning rate 1×10 −5 , cosine scheduler with warm-up ratio 0.05, and AdamW with weight decay 0.1 for 5 epochs. GRPO uses learning rate 1×10 −6 , batch size 4, KL coefficient 0.05, clip range 0.2, temperature 1.0, top-p0.85, andK=16 candidates per input with max 4096 tokens for 5 epochs. Self-supervised GRPO uses learning rate 2×10 −7 , gradient accumulation of 4, and trains for 4 epochs. Unless otherwise noted, the reported results are from a single random seed (42). 5.7. Evaluation Metrics We follow the evaluation protocol of [43]. For the clas- sification task, we report Accuracy (Acc), Weighted F1 (W- F1), and Macro F1 (M-F1). For explanation quality, we report BERTScore (BS) [98] and METEOR (MET) [9]. We use these as reference-based measures of explanation similarity, rather than as direct indicators of reasoning faithfulness or logical validity. We use macro-F1 as the primary comparison metric be- cause it is more robust to class imbalance, while also reporting accuracy for comparability with prior work. 6. Results and Analysis This section presents our experimental results. We begin with a comparison against prior work, followed by unimodal and zero-shot baselines, a detailed analysis of RL post-training and self-supervised GRPO, cross-dataset analysis, statistical sig- nificance tests, and a summary of findings organized by research question. 6.1. Comparison with State-of-the-Art Tables 5 and 6 compare our approach with published base- lines. 5 On FHM (Table 5), our approach achieves 0.80 macro-F1, outperforming the multimodal sequence-classification baselines Gemma-3-12B-IT and Qwen3-VL-8B-Instruct, which obtain lower macro-F1 and do not generate explanations. This in- dicates that our reasoning-based approach improves balanced classification performance while also producing interpretable rationales. 5 All baseline results are reported from the original papers and are not re- implemented. Table 5: Comparison with SOTA on Hateful Memes. Seq. cls. baselines use classification heads without explanations. Sup.: Supervised GRPO. ModelAcc W-F1 M-F1 BS MET Prior work [42]69.5– [15]73.0– [89]76.4– [94]77.2– [12]77.7– [59]78.8– [43]79.9 0.800.79 0.78 0.49 Seq. cls. baselines Gemma-3-12B-IT78.0 0.770.74– Qwen3-VL-8B-Instruct 78.0 0.780.77– Proposed Proposed (Sup.)82.0 0.820.80 0.78 0.52 + Self-Sup.81.8 0.810.79 0.77 0.51 Table 6: Comparison with SOTA on ArMeme. Seq. cls. baselines use clas- sification heads without explanations. Sup.: Supervised GRPO. Self-Sup.: Self-supervised GRPO. ModelAcc W-F1 M-F1 BS MET Prior work [5]69.7 0.69.551– [43]72.1 .699.536.70.35 Seq. cls. baselines Gemma-3-12B-IT74.8 .724.597– Qwen3-VL-8B-Instruct 76.6 .753.594– Proposed Proposed (Sup.)72.6 .711.597 .740 .441 + Self-Sup.72.8 .715.612 .741 .442 On ArMeme (Table 6), self-supervised GRPO achieves 0.612 macro-F1, improving over [43] by +7.6 points and over the original ArMeme benchmark of [5] by +6.1 points. It also outperforms the strongest sequence-classification baseline, Qwen3-VL-8B-Instruct, in macro-F1 (0.612 vs. 0.594), while additionally generating explanations. These results highlight the advantage of reasoning-based optimization for balanced per-class performance, particularly in the more challenging four-class ArMeme setting. 6.2. Unimodal Baselines As unimodal baselines, we fine-tune image-only and text- only classifiers on both benchmarks (Table 7). Across both datasets, text-only classifiers consistently outperform image- only classifiers in macro-F1. The gap is especially large on ArMeme, where mean macro-F1 increases from 0.267 for image- only models to 0.482 for text-only models, suggesting that tex- tual cues provide the strongest signal for propaganda detection. However, unimodal models still lag behind our multimodal reasoning approach. On FHM, the best unimodal baseline reaches 0.53 macro-F1, compared with 0.80 for our approach. On ArMeme, the gap is 0.509 vs. 0.612 macro-F1. These re- sults show that cross-modal reasoning captures complementary 9 Table 7: Unimodal fine-tuned baselines on Hateful Memes and ArMeme. Best per column in bold; “–” indicates the model was not evaluated on that dataset. FHMArMeme ModelAcc W-F1 M-F1Acc W-F1 M-F1 Image-only BEiT-base61.0 .560.50065.1 .578.289 ConvNeXt-large 63.0 .510.42065.5 .527.203 DINOv2-large58.0 .560.510 65.8 .618.402 ResNet-10163.0 .480.38065.8 .522.198 Swin-large63.0 .570.51067.2 .585.267 ViT-base61.0 .560.500 64.0 .556.243 Average61.5 .540.47065.6 .564.267 Text-only AraBERTv2–67.9 .668.503 DistilBERT64.0 .590.530– mBERT64.0 .550.480 71.3 .676.446 Qarib63.0 .570.52070.5 .670.469 XLM-RoBERTa 64.0 .570.510 71.5 .691.509 Average63.8 .570.51070.3 .676.482 signals that unimodal models often miss. This is especially im- portant for memes, where harmful content often emerges from the interaction between the image and text. 6.3. Zero-Shot and CoT Table 8 reports the zero-shot and CoT results across all models on both benchmarks. For clarity, we use abbreviated model names: Llama (Llama-3.2-11B) [25], L4-Scout (Llama- 4-Scout-17B), 6 Qwen-I/T (Qwen3-VL-8B Instruct/Thinking), Gemma (Gemma-3-12B), and Kimi-I/T (Kimi-VL-A3B In- struct/Thinking). On FHM, no single model dominates: GPT-4.1 [63] with CoT the best macro-F1, and L4-Scout obtains best BS score. On ArMeme, all models exhibit substantially lower performance compared to binary FHM dataset, reflecting the added difficulty of four-class Arabic propaganda classification. Overall, most models underperform without CoT on both tasks. This trend is clearer on ArMeme. For example, Llama without CoT pre- dicts the not-meme class for roughly 95% of test instances and achieves only 7.0% accuracy. 6.3.1. Impact of CoT Averaging across all nine models, CoT prompting improves mean macro-F1 by +0.022 on FHM and +0.079 on ArMeme, while consistently degrading explanation quality (mean BS drops of−5.91 and−2.7 points, respectively). On FHM, CoT exhibits a systematic and asymmetric effect: it improves classification for most models (up to +0.063 Macro- F1 points for Kimi-I), primarily by reducing false negatives, but consistently degrades explanation quality as measured by BS (drops of 1.6 to 12.5 points), suggesting that CoT shifts 6 https://ai.meta.com/blog/llama-4-multimodal-intelligenc e/ explanations away from the concise annotation style used in the references. On ArMeme, CoT produces dramatically larger Macro-F1 gains (up to +0.19 for Gemma), demonstrating that explicit rea- soning is essential when models face a multilingual, multi-class task far from their pretraining distribution. The sole exception is GPT-4.1, which performs slightly better without CoT, sug- gesting its internal representations already encode sufficient task knowledge for Arabic propaganda detection. Across both bench- marks, CoT benefits vary substantially across models rather than being confined to a specific model family, while explanation quality consistently degrades, highlighting a trade-off between classification performance and explanation calibration. Model selection. These observations demonstrate the need for further investigation into how to effectively exploit open-weight thinking-based MLLMs that are trainable under limited com- putational resource. Based on overall performance across both benchmarks and ease of implementation, we select the Qwen-T model as the base MLLM for our subsequent experiments. 6.4. Analysis of RL Post-Training Tables 9 and 10 report the effect of SFT warm-up and GRPO on both benchmarks. We organize the analysis around four key findings. On FHM, SFT with fine-grained supervision (Cls + FG + Exp) improves macro-F1 from 0.75 to 0.77 over the variant without fine-grained labels. The gain is substantially larger on ArMeme, where macro-F1 increases by +8.0%. This shows that incorporating fine-grained labels improves decision consistency even without explicit reasoning supervision. The effect is par- ticularly clear on ArMeme, where our multi-LLM annotation pipeline provides effective fine-grained supervision for a dataset that previously lacked such annotations. CoT distillation improves explanation quality. Adding distilled CoT supervision during SFT yields further gains (Tables 9, 10). SFT (Cls+FG+Exp, CoTD) achieves the strongest warm-up per- formance on FHM, reaching 0.78 macro-F1, a +1 point gain over SFT (Cls+FG+Exp), with a 0.01 increase in BS. On ArMeme, it improves macro-F1 from 0.51 to 0.54, while BS remains un- changed. Thus, the effect of CoTD on explanation quality is more visible on FHM during SFT. For ArMeme, improvements in BS emerge only after GRPO. Overall, explicit reasoning su- pervision improves classification performance on both datasets and provides a stronger initialization for GRPO. SFT warm-up is critical for GRPO. Cold-start GRPO achieves only 0.75 and 0.33 macro-F1 on FHM and ArMeme, respectively, showing that GRPO provides limited benefits without proper warm-up. This variant also produces weaker explanations, with a BS of 0.73 on FHM, further highlighting the importance of SFT initialization. In contrast, all SFT-initialized GRPO models substantially outperform both SFT-only and cold-start variants. For example, initializing GRPO from SFT-Cls+FG+Exp im- proves macro-F1 from 0.77 to 0.79 on FHM and from 0.51 to 0.58 on ArMeme. These results suggest that once the model 10 Table 8: Performance of pretrained MLLMs on both benchmarks. Llama = Llama-3.2-11B; L4-Scout = Llama-4-Scout-17B; Qwen-I/T = Qwen3-VL-8B (Instruct/Thinking); Gemma = Gemma-3-12B; Kimi-I/T = Kimi-VL-A3B (Instruct/Thinking). Acc: Accuracy; BS: BERTScore; Met: METEOR. FHMArMeme ModelAcc W-F1 M-F1 BS MetAcc W-F1 M-F1 BS Met Llama +CoT60.0 .606.592 .600 .16535.3 .401.216 .563 .131 Llama64.2 .599.547 .660 .234 7.0.048.061 .591 .157 Qwen-I +CoT65.3 .658.652 .646 .26364.6 .626.388 .627 .210 Qwen-I63.0 .630.630 .662 .216 30.2 .373.259 .616 .199 Qwen-T +CoT70.2 .706.694 .581 .20263.1 .561.281 .598 .173 Qwen-T65.6 .661.652 .659 .21522.1 .108.168 .601 .207 Gemma +CoT65.5 .659.655 .621 .13567.5 .625.345 .597 .103 Gemma65.2 .654.652 .664 .21528.5 .197.155 .627 .213 L4-Scout +CoT75.4 .687.545 .596 .11167.9 .582.284 .589 .093 L4-Scout77.0 .710.580 .653 .25254.1 .569.340 .649 .276 GPT-4.1 +CoT74.1 .739.719 .470 .07268.7 .599.290 .577 .083 GPT-4.171.3 .715.700 .595 .15969.7 .619.350 .637 .180 Kimi-I +CoT67.7 .667.636 .621 .18639.9 .437.235 .602 .134 Kimi-I67.0 .625.573 .678 .25721.8 .263.176 .618 .200 Kimi-T +CoT70.5 .704.685 .614 .15557.2 .562.310 .586 .115 Kimi-T67.5 .668.639 .660 .22228.3 .336.217 .611 .223 Fanar-2-Oryx-IVU +CoT 68.0 .680.660 .600 .12064.4 .593.330 .582 .091 Fanar-2-Oryx-IVU69.0 .690.670 .650 .20057.3 .504.240 .614 .161 Table 9: SFT and GRPO on FHM. Cls = class labels; FG = fine-grained; Exp = explanation; CoTD = CoT distillation;R think = thinking-length reward. † / ‡ denote Holm-corrected paired-bootstrap improvements in macro-F1 over the SFT warm-up the row was initialised from, atα= 0.05/0.01 (one-sided); see Section 6.7. ModelAcc W-F1 M-F1 BS Met SFT warm-up SFT (Cls+Exp)77.0 0.770.75 0.77 0.48 SFT (Cls+FG+Exp)78.1 0.780.77 0.77 0.48 SFT (Cls+FG+Exp, CoTD)79.2 0.790.78 0.78 0.50 RL / GRPO GRPO (Cold Start)76.8 0.770.75 0.73 0.47 SFT-Cls+Exp→ GRPO ‡ 80.4 0.800.78 0.76 0.50 SFT-Cls+FG+Exp→ GRPO † 81.1 0.810.79 0.77 0.52 SFT-CoTD→ GRPO81.2 0.810.79 0.78 0.52 RL / GRPO + R think SFT-CoTD→ GRPO + R think † 82.0 0.820.80 0.78 0.52 is aligned with the target task through SFT, GRPO can more effectively refine label correctness and rationale quality. Thinking-length regularization. AddingR think to the CoTD- initialized variant improves macro-F1 on both benchmarks (Ta- bles 9, 10). On FHM, macro-F1 increases from 0.79 to 0.80, yielding the strongest supervised result. On ArMeme, the gain is larger, with macro-F1 improving from 0.57 to 0.59 (+2.0%). This suggests thatR think mainly improves balanced per-class performance, especially on the harder four-class ArMeme task. This is consistent with the training-dynamics analysis below: R think stabilizes reasoning length and mitigates length-shortening Table 10: SFT and GRPO on ArMeme. Same notation as Table 9; † / ‡ denote Holm-corrected paired-bootstrap macro-F1 gains over the corresponding SFT warm-up (Section 6.7). ModelAcc W-F1 M-F1 BS Met SFT warm-up SFT (Cls+Exp)64.2 0.630.43 0.61 0.35 SFT (Cls+FG+Exp)69.3 0.670.51 0.73 0.43 SFT (Cls+FG+Exp, CoTD)70.3 0.690.54 0.73 0.43 RL / GRPO GRPO (Cold Start)70.9 .651.332 .711 .421 SFT-Cls+FG+Exp→ GRPO † 72.4 .703.557 .741 .439 SFT-Cls+Exp→ GRPO ‡ 72.8 .706.579 .742 .443 SFT-CoTD→ GRPO † 72.9 .706.577 .741 .441 RL / GRPO + R think SFT-CoTD→ GRPO + R think † 72.6 .711.597 .740 .441 bias in rationale generation. GRPO training dynamics. An analysis of reward trajectories and completion lengths across initialization regimes (see Fig- ure C.5 in Appendix Appendix C) reveals a consistent reward hacking phenomenon, particularly on ArMeme. The mean com- pletion length decreases rapidly while the reward continues to increase, indicating that the policy exploits a shortcut by produc- ing shorter generations rather than improving reasoning quality. Initialization with CoTD partially mitigates this behavior by in- troducing a structural prior over reasoning generation, while the explicitR think reward stabilizes training dynamics across both datasets. These observations further support the importance of 11 Table 11: Effect of self-supervised GRPO The supervised baseline is the best GRPO +R think checkpoint. Sup.: Supervised. † denotes a Holm-corrected paired-bootstrap improvement of the self-supervised row over the supervised baseline at α = 0.05 (one-sided); see Section 6.7. ModelAcc W-F1 M-F1 BS Met FHM Sup. GRPO + R think 82.0 0.820.80 0.78 0.52 + Self-Sup. GRPO81.8 0.810.79 0.77 0.51 ArMeme Sup. GRPO + R think 72.6 .711.597 .740 .441 + Self-Sup. GRPO † 72.8 .715.612 .741 .442 SFT warm-up and thinking-length regularization reported above. 6.5. Self-Supervised GRPO We evaluate self-supervised GRPO (Section 4.5) by fur- ther training the best supervised GRPO +R think checkpoint on 2,000 unlabeled memes per dataset. The memes are selected using disagreement-based sampling under multi-temperature in- ference. We retain only samples with partial agreement (3/5 or 4/5), so each training instance contains measurable prediction uncertainty. Table 11 reports the effect of this self-supervised stage. ArMeme. Self-supervised GRPO improves macro-F1 by 1.5 points, from 0.597 to 0.612, and yields a small gain in METEOR, from .441 to .442 (Table 11). A per-class recall analysis shows that the gains are concentrated in minority classes. Propaganda improves from 40.0% to 41.8%, not-meme from 54.4% to 57.9%, and other from 50.0% to 51.8%. This comes at a small cost to the majority class, not-propaganda, which decreases by 0.8 points. The prediction distribution shifts slightly toward minority classes and moves closer to the true label distribution. These results suggest that the consensus-based pseudo-reward provides a useful learning signal for underrepresented categories when unlabeled data is in-domain (see Figure B.3 for an example). FHM. On FHM, self-supervised GRPO reduces macro-F1 by 1.0 point (Table 11). Per-class analysis shows an asymmetric effect. Not-hateful recall improves from 87.5% to 92.2%, while hateful recall drops from 72.9% to 64.4%. The model develops a bias toward the majority class, with 80.5% of the case model predicted the meme as not-hateful. The different outcomes on the two benchmarks are not contradictory. On the four-class ArMeme task, majority consensus is harder to reach, so the pseudo-reward fires more often on borderline cases, including minority-class examples. This provides gradient signal for un- derrepresented categories. On the binary FHM task, small policy biases can more easily produce unanimous agreement on the majority class. As a result, the pseudo-reward becomes self- reinforcing for “not-hateful” and amplifies, rather than corrects, the existing class skew. Analysis of dataset-dependent behavior. We attribute the diver- gent outcomes to three factors. (i) Distribution alignment: The ArMeme unlabeled set is collected from the same Arabic social media sources as the Table 12: Cross-dataset comparison. FHM = Hateful Memes (binary); ArMeme = Arabic propaganda (four-class). FHMArMeme SettingAcc M-F1 BSAcc M-F1 BS Best unimodal (image) 63.0 .510–67.2 .267– Best unimodal (text)64.0 .530– 71.5 .509– Best zero-shot77.0 .719 .67869.7 .350 .649 Best seq. cls.78.0 .770– 76.6 .594– Best SFT79.2 .780 .78070.3.54.73 Best GRPO81.2 .790 .780 72.9 .577 .741 GRPO+R think 82.0 .800 .78072.6 .597 .740 + Self-Sup.81.8 .790 .770 72.8 .612 .741 Prior SOTA79.9 .790 .78072.1 .536.70 labeled set, ensuring distributional consistency, whereas the FHM unlabeled set is curated from different English meme datasets, introducing a potential distribution shift relative to the FHM benchmark. (i) Majority-vote bias amplification: In binary classifica- tion, even small prediction biases can produce near-unanimous agreement among sampled outputs, leading to uniform rewards and weak gradient signals. In contrast, the four-class setting makes majority consensus harder to achieve, preserving reward variation and gradient diversity. (i) Task complexity: Binary tasks rely on a single deci- sion boundary, which can be quickly shifted by biased pseudo- supervision, whereas multi-class tasks involve multiple decision boundaries and provide richer gradient signals. 6.6. Cross-Dataset Analysis Table 12 provides a side-by-side comparison of the best results under each experimental setting. Several cross-dataset patterns emerge from this analysis. Task difficulty. ArMeme is consistently more challenging than FHM across all experimental settings. In the zero-shot setting, macro-F1 drops from 0.719 on FHM to 0.350 on ArMeme. This gap reflects the combined effect of four-class classification (vs. binary classification), Arabic-language difficulties, and the inher- ent subtlety of propaganda detection compared with hatefulness detection. Effectiveness of RL post-training. On both benchmarks, perfor- mance improves from SFT warm-up to GRPO and GRPO+R think (Tables 9, 10). Compared with the best SFT checkpoint, SFT- CoTD, the best supervised GRPO+R think checkpoint improves macro-F1 from .78 to .80 on FHM and from .54 to .597 on ArMeme. The gain is therefore larger on ArMeme, the more imbalanced four-class task. This suggests that RL-based post- training is useful for improving balanced per-class performance in our setting. RL without supervised warm-up. When GRPO is applied di- rectly to the base model, without an initial SFT stage, perfor- mance on ArMeme collapses to a degenerate macro-F1 of 0.332, while remaining reasonable on FHM (0.75), see Tables 9 and 10. This shows that starting RL from an unadapted base model 12 is much more fragile as task complexity increases, making SFT initialization particularly important for challenging multilingual and multi-class settings. Generative reasoning vs. classification. On ArMeme, the best seq. cls. baseline has lower macro-F1 than our generative model (0.594 vs. 0.612) and produces no explanations. On FHM, our approach also outperforms seq. cls. baselines in macro-F1. This contrast suggests that generative reasoning models can provide more balanced per-class performance while also producing in- terpretable outputs. 6.7. Statistical Significance We assess statistical significance using paired bootstrap re- sampling over test examples [24] withB= 10,000 samples. Since all models are evaluated on the same test sets, each boot- strap sample preserves the paired predictions by example index and recomputes macro-F1 for both systems. We use a one-sided alternative hypothesis, testing whether the post-training model improves over its comparison checkpoint. Each post-training row in Tables 9, 10, and 11 is compared against the checkpoint indicated by the row label, such as GRPO vs. its SFT warm-up, or self-supervised GRPO vs. the preceding supervised GRPO. Cold-start GRPO is excluded. We control the family-wise error rate within each table using Holm–Bonferroni correction [38]. For ArMeme and FHM, supervised GRPO improvements are statistically significant atp ≤0.05 after correction, ex- ceptSFT-CoTD→ GRPOon FHM (Holm-adjustedp= 0.12). In Table 11, the FHM self-supervised result is not significant (p= 0.94), while the ArMeme result is marginally significant (p= 0.047). Overall, supervised RL provides stronger ev- idence of improvement over SFT initialization, whereas the self-supervised stage shows mixed evidence. 6.8. Summary of Findings by Research Question We revisit the research questions posed in the introduction and summarize our findings. RQ1. Fine-grained supervision. On FHM (Table 9), fine- grained labels provide consistent gains. SFT-Cls+FG+Exp →GRPO reaches .79 macro-F1, compared with .78 for SFT- Cls+Exp→GRPO without fine-grained supervision.On ArMeme (Table 10), the effect is largest during SFT. SFT (Cls+FG+Exp) improves over SFT (Cls+Exp) from .43 to .51 macro-F1, an +8.0 point improvement. RQ2: Synthetic CoT supervision. CoTD provides consistent gains during SFT warm-up on both benchmarks: +1 point macro-F1 and +2% METEOR on FHM; +3 points macro-F1 and comparable METEOR score on ArMeme. When used as GRPO initialization, CoTD-based checkpoints achieve the high- est BERTScore (0.78 on FHM), suggesting that explicit reason- ing supervision can improve reference-based explanation fidelity and provides a stronger starting point for RL. RQ3. Multi-LLM synthetic annotation. Our annotation pipeline, using GPT-4.1 and Llama-4-Scout as annotators and Gemini-3- Pro as consolidator, provides effective propaganda supervision on ArMeme, where no prior fine-grained annotations were avail- able. At the SFT stage, it improves macro-F1 from .43 to .51, an +8.0 point improvement. This suggests that multi-LLM pipelines can provide scalable proxy fine-grained supervision in our setting. RQ4. Supervised GRPO post-training. GRPO consistently im- proves over SFT baselines on both datasets (Tables 9, 10). On FHM, the best SFT checkpoint, CoTD, reaches .78 macro-F1, while the best supervised GRPO+R think checkpoint reaches .80. On ArMeme, macro-F1 improves from .54 with the best SFT checkpoint to .597 with supervised GRPO+R think , a +5.7 point improvement. SFT warm-up is critical, as cold-start GRPO drops to a low macro-F1 of .332 on ArMeme. Thinking-length regularization yields the best supervised configuration, confirm- ing that both SFT warm-up andR think are important for stable GRPO in our setup. RQ5. Self-supervised RL. Self-supervised GRPO improves macro-F1 by +1.5 points on ArMeme, from .597 to .612 (Ta- ble 11). Per-class recall improves across all three minority classes, with gains of +1.8 points for propaganda, +3.5 points for not-meme, and +1.8 points for other. On FHM, self-supervised GRPO reduces macro-F1 by +1.0 point (Table 11). This may be due to mismatch in data distribution, as the FHM unlabeled pool is curated from public datasets rather than the FHM distribution. These results suggest that self-supervised GRPO is more effec- tive when the unlabeled and target distributions are aligned and the label space is sufficiently diverse to reduce consensus-based majority-class bias. 7. Discussion In this section, we reflect on the broader implications of our experimental findings and discuss the main limitations of the current work. Reward shaping for reasoning stability. Our training dynamics analysis (Section 6.4, Appendix Appendix C) shows that stan- dard GRPO exhibits a length-shortening bias, particularly on ArMeme. The thresholded lower-bound thinking-length reward R think mitigates this behavior by penalizing rationales that fall be- low a minimum reasoning-length threshold, while assigning full reward once the threshold is reached. Thus,R think encourages sufficient reasoning without incentivizing unnecessary verbosity. This finding suggests that explicit reasoning-length constraints can support more stable RL-based post-training in this setting, as rule-based rewards alone may not be sufficient to maintain rationale quality. More broadly, reward exploitation is a well- documented concern in RL-tuned language models when the reward signal is purely rule-based [80,81,28]. Our results ex- tend this observation to multimodal reasoning tasks and show that even a simple thresholded length penalty can mitigate length- shortening behavior, offering a lightweight alternative to more complex reward-model-based approaches. 13 Scalability of synthetic supervision. Fine-grained labels consis- tently improve SFT warm-up on both benchmarks, with particu- larly large gains on ArMeme (Section 6.4). Beyond the immedi- ate performance gains, the success of our multi-LLM annotation pipeline (GPT-4.1 + Llama-4-Scout annotators, Gemini-3-Pro as a consolidator) raises a broader question about the scalability of synthetic supervision for content moderation. Our results suggest that multi-source LLM agreement can serve as a use- ful proxy for human annotation quality in structured labeling tasks, though the extent to which this generalizes to more subjec- tive or culturally nuanced annotation schemes remains an open question. Conditions for effective self-supervised RL. Self-supervised GRPO succeeds on ArMeme (+1.5% macro-F1), where the unlabeled data is in-domain and the four-class setting preserves gradient diversity, but degrades on FHM due to distributional mismatch and majority-vote bias in the binary setting. This suggests that consensus-based pseudo-labeling requires both distribution alignment and sufficient label-space diversity to provide reliable supervision. This finding has practical implica- tions for deploying RL-based moderation systems. In-domain unlabeled data is abundant on social media platforms, making self-supervised GRPO a viable strategy for continuous model adaptation, provided the label space is sufficiently diverse to avoid consensus-induced label homogenization. However, self- supervised GRPO is sensitive to distribution alignment between unlabeled and target data, and can amplify class biases when dis- tributions diverge, as observed on FHM. These sensitivities sug- gest that careful data curation and class-aware pseudo-labeling strategies would be needed for robust deployment. Generative reasoning vs. classification. On ArMeme, our gen- erative reasoning model outperforms the strongest sequence- classification baseline in macro-F1 (0.612 vs. 0.594) while also producing explanations. On FHM, our approach also achieves stronger macro-F1 than the sequence-classification baselines. This consistent pattern suggests that generative reasoning mod- els provide more balanced per-class performance than sequence- classification models, while additionally producing interpretable outputs that can support human review. For content moderation in practice, both balanced class performance and actionable ex- planations are desirable, which favors the generative reasoning approach. Limitations. We outline several aspects of the present study that point toward natural extensions in future work. Our evaluation focuses on two representative benchmarks, FHM (English) and ArMeme (Arabic), which together span binary and fine-grained labeling across two languages; broader coverage of additional languages, dialects, and culturally specific meme traditions re- mains an interesting direction. Memes are also a fast-evolving medium, with new templates, visual styles, and cultural ref- erences appearing continuously, so periodic re-evaluation on freshly collected data would help track how detection systems generalize over time. We focus on static image-with-text memes; extending the framework to animated GIFs and short-form video memes is left for future work. The thinking-based generation format introduces additional inference-time tokens compared to direct classifiers, which is an inherent trade-off when produc- ing explanations alongside predictions, and deployments with strict latency budgets may benefit from distilled or shorter-CoT variants. Finally, although our self-supervised GRPO reduces reliance on human-labeled data, scaling unlabeled meme col- lection across many languages still involves non-trivial filtering and curation effort. 8. Conclusion We studied the extent to which GRPO-based training can improve classification performance and reference-based expla- nation quality in thinking-based MLLMs for meme modera- tion. Our RL-based post-training method with a GRPO ob- jective jointly rewards correct classification, structurally valid outputs, and semantically aligned explanations. We addition- ally constructed extended datasets with distilled CoT rationales and multi-LLM fine-grained propaganda annotations. Across experiments on both the English Hateful Memes and Arabic ArMeme benchmarks, our approach improves over prior re- ported results: +2.1% accuracy on FHM and +7.6% macro-F1 on ArMeme over prior work. Key findings include: (i) fine- grained supervision and distilled CoT complement RL-based optimization; (i) multi-LLM annotation pipelines scale fine- grained annotation effectively; (i) thinking-length regulariza- tion mitigates length-shortening bias in rationale generation; and (iv) self-supervised GRPO extracts useful signal from unlabeled in-domain data. Acknowledgements The work of M. B. Kmainasi, A. E. Shahroor and F. Alam was supported by NPRP grant 14C-0916-210015 from the Qatar National Research Fund, part of the Qatar Research Develop- ment and Innovation Council (QRDI). The findings reported herein are solely the responsibility of the authors. Declaration of generative AI and AI-assisted technologies in the manuscript preparation process During the preparation of this work the authors used Claude Opus 4 in order to refine grammar and stylistic phrasing. After using this tool, the authors reviewed and edited the content as needed and take full responsibility for the content of the publication. Appendix A. Training Configurations Tables A.13–A.14 report the full hyperparameter configura- tions for all training stages. 14 Table A.13: Unimodal baseline training configurations. ParameterTextImage Batch Size32512 Learning Rate3×10 −5 3×10 −4 Max Seq. Length512– OptimizerAdamW Weight Decay0.010.01 Epochs1010 Model SelectionValidation Acc Hardware1× A100 Table A.14: Multimodal training configurations across stages. ParameterSFT Sup. GRPOSS-GRPO Learning Rate1×10 −5 1×10 −6 2×10 −7 Batch Size444 Grad. Accumulation114 Epochs554 SchedulerCosine (warmup ratio 0.05) OptimizerAdamW (weight decay 0.1) Precisionbfloat16 ParallelismZeRO Stage 3 Frozen ModulesVision Encoder Hardware4× H200 GRPO-specific Candidates (K)–1616 Temperature–1.01.0 Top- p–0.850.85 Top-k–5050 KL Coefficient–0.050.05 Clip Range (ε)–0.20.2 Max Length–40964096 Pseudo-labeling– Majority Vote Appendix B. Qualitative Examples We present three representative examples (see Figure B.2, B.3 and B.4) illustrating how model predictions improve across training stages. For each example, we show the ground-truth label and explanation alongside the model’s prediction before and after a given training stage. Appendix C. GRPO Training Dynamics Figure C.5 compares GRPO reward trajectories and mean completion length across initialization regimes and datasets. When initialized directly from the pretrained backbone (cold- start), GRPO exhibits noisier reward curves with only modest gains and largely stable completion length, indicating that RL alone has difficulty simultaneously learning the required output structure and the task-specific decision boundary. Initializing from SFT (Cls+Exp) yields substantially smoother trajectories and larger gains, suggesting that once the model is aligned to the target schema and supervision distribution, GRPO can focus on improving label correctness and rationale quality. However, on ArMeme, all configurations without thinking-length regular- ization exhibit a progressive decline in completion length—a symptom of reward hacking through shorter generations. Ex- plicit thinking-length regularization (R think ) stabilizes reasoning length and prevents this collapse, as detailed in Section 6.4. References [1]Abdelali, A., Hassan, S., Mubarak, H., Darwish, K., Samih, Y., 2021. Pre-training BERT on Arabic tweets: Practical considerations. ArXiv preprint arXiv:2102.10684. URL: https://arxiv.org/abs/2102.10684. [2]Abdin, M., Aneja, J., Awadalla, H., Awadallah, A., Awan, A.A., Bach, N., Bahree, A., Bakhtiari, A., Bao, J., Behl, H., et al., 2024. Phi-3 technical report: A highly capable language model locally on your phone. ArXiv preprint arXiv:2404.14219. URL:https://arxiv.org/abs/24 04.14219. [3]AlAfnan, M.A., 2025. The role of memes in shaping political discourse on social media. Studies in Media and Communication 13, 1–10. URL:https://doi.org/10 .11114/smc.v13i2.7482, doi:10.11114/smc.v13i2 .7482. [4]Alam, F., Cresci, S., Chakraborty, T., Silvestri, F., Dim- itrov, D., Martino, G.D.S., Shaar, S., Firooz, H., Nakov, P., 2022. A survey on multimodal disinformation de- tection, in: Calzolari, N., Huang, C.R., Kim, H., Puste- jovsky, J., Wanner, L., Choi, K.S., Ryu, P.M., Chen, H.H., Donatelli, L., Ji, H., Kurohashi, S., Paggio, P., Xue, N., Kim, S., Hahm, Y., He, Z., Lee, T.K., Santus, E., Bond, F., Na, S.H. (Eds.), Proceedings of the 29th International Conference on Computational Linguistics, International Committee on Computational Linguistics, Gyeongju, Republic of Korea. p. 6625–6643. URL: https://aclanthology.org/2022.coling-1.576. [5]Alam, F., Hasnat, A., Ahmad, F., Hasan, M.A., Hasanain, M., 2024. ArMeme: Propagandistic content in Arabic memes, in: Al-Onaizan, Y., Bansal, M., Chen, Y.N. (Eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Association for Compu- tational Linguistics, Miami, FL, USA. p. 21071–21090. URL:https://aclanthology.org/2024.emnlp-mai n.1173/, doi:10.18653/v1/2024.emnlp-main.1173. [6]Amalia, A., Sharif, A., Haisar, F., Gunawan, D., Nasution, B.B., 2018. Meme opinion categorization by using optical character recognition (OCR) and Naïve Bayes algorithm, in: Proceedings of the Third International Conference on Informatics and Computing, IEEE, Palembang, Indonesia. p. 1–5. URL:https://doi.org/10.1109/IAC.2018 .8780410, doi:10.1109/IAC.2018.8780410. [7] Antoun, W., Baly, F., Hajj, H., 2020.AraBERT: Transformer-based model for Arabic language understand- ing, in: Al-Khalifa, H., Magdy, W., Darwish, K., Elsayed, T., Mubarak, H. (Eds.), Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, 15 Figure B.2: Example 1 — ArMeme: SFT→GRPO. The SFT model incorrectly labels a humorous school comparison meme as propaganda, misinterpreting satire as manipulation. After GRPO, the model correctly identifies the lighthearted tone and classifies it as not-propaganda. Figure B.3: Example 2 — ArMeme: GRPO→Self-Supervised GRPO. The supervised model misclassifies a meme that uses cultural identity and emotional language as not-propaganda. After self-supervised GRPO on unlabeled data, the model correctly detects the propaganda technique. 16 Figure B.4: Example 3 — Hateful Memes: SFT→GRPO. The SFT model misses implicit hate, interpreting a meme mocking disability through an incest joke as “dark humor.” After GRPO, the model’s reasoning correctly identifies the dehumanizing stereotype and classifies it as hateful. with a Shared Task on Offensive Language Detection, Eu- ropean Language Resource Association, Marseille, France. p. 9–15. URL:https://aclanthology.org/2020. osact-1.2/. [8]Bai, S., Cai, Y., Chen, R., Chen, K., Chen, X., Cheng, Z., Deng, L., Ding, W., Gao, C., Ge, C., Ge, W., Guo, Z., Huang, Q., Huang, J., Huang, F., Hui, B., Jiang, S., Li, Z., Li, M., Li, M., Li, K., Lin, Z., Lin, J., Liu, X., Liu, J., Liu, C., Liu, Y., Liu, D., Liu, S., Lu, D., Luo, R., Lv, C., Men, R., Meng, L., Ren, X., Ren, X., Song, S., Sun, Y., Tang, J., Tu, J., Wan, J., Wang, P., Wang, P., Wang, Q., Wang, Y., Xie, T., Xu, Y., Xu, H., Xu, J., Yang, Z., Yang, M., Yang, J., Yang, A., Yu, B., Zhang, F., Zhang, H., Zhang, X., Zheng, B., Zhong, H., Zhou, J., Zhou, F., Zhou, J., Zhu, Y., Zhu, K., 2025. Qwen3-VL technical report. ArXiv preprint arXiv:2511.21631. URL: https://arxiv.org/abs/2511.21631. [9] Banerjee, S., Lavie, A., 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments, in: Goldstein, J., Lavie, A., Lin, C.Y., Voss, C. (Eds.), Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, Association for Com- putational Linguistics, Ann Arbor, MI, USA. p. 65–72. URL: https://aclanthology.org/W05-0909. [10]Bao, H., Dong, L., Piao, S., Wei, F., 2022. BEiT: BERT pre-training of image transformers, in: Proceedings of the International Conference on Learning Representations, OpenReview.net, Virtual. URL:https://openreview .net/forum?id=p-BhZSz59o4. [11]Boinepelli, S., Shrivastava, M., Varma, V., 2020. SIS@IIITH at SemEval-2020 task 8: An overview of sim- ple text classification methods for meme analysis, in: Her- belot, A., Zhu, X., Palmer, A., Schneider, N., May, J., Shutova, E. (Eds.), Proceedings of the Fourteenth Work- shop on Semantic Evaluation, International Committee for Computational Linguistics, Barcelona (online). p. 1190– 1194. URL:https://aclanthology.org/2020.seme val-1.157, doi:10.18653/v1/2020.semeval-1.157. [12] Burbi, G., Baldrati, A., Agnolucci, L., Bertini, M., Del Bimbo, A., 2023. Mapping memes to words for multi- modal hateful meme classification, in: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, IEEE, Paris, France. p. 2832–2836. URL:ht tps://doi.org/10.1109/ICCVW60793.2023.00303 , doi:10.1109/ICCVW60793.2023.00303. [13]Cao, R., Hee, M.S., Kuek, A., Chong, W.H., Lee, R.K.W., Jiang, J., 2023. Pro-Cap: Leveraging a frozen vision- language model for hateful meme detection, in: Proceed- ings of the 31st ACM International Conference on Multi- media, ACM, Ottawa, ON, Canada. p. 5244–5252. URL: https://doi.org/10.1145/3581783.3612498, doi:10.1145/3581783.3612498. 17 Figure C.5: GRPO training dynamics across initialization regimes and datasets, showing mean reward (blue) and completion length (magenta). ArMeme exhibits strong reasoning-length collapse under GRPO, indicating reward hacking through shorter generations. CoT warm-up partially mitigates this effect, while explicit thinking-length regularization (R think ) stabilizes reasoning length and prevents collapse. Curves smoothed with a moving average of 1000 steps. [14] Cao, R., Lee, R.K., Jiang, J., 2024. Modularized networks for few-shot hateful meme detection, in: Chua, T., Ngo, C., Kumar, R., Lauw, H.W., Lee, R.K. (Eds.), Proceedings of the ACM Web Conference 2024, ACM, Singapore. p. 4575–4584. URL:https://doi.org/10.1145/3589 334.3648145, doi:10.1145/3589334.3648145. [15]Cao, R., Lee, R.K.W., Chong, W.H., Jiang, J., 2022. Prompting for multimodal hateful meme classification, in: Goldberg, Y., Kozareva, Z., Zhang, Y. (Eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, Association for Computational Lin- guistics, Abu Dhabi, United Arab Emirates. p. 321–332. URL:https://aclanthology.org/2022.emnlp-mai n.22, doi:10.18653/v1/2022.emnlp-main.22. [16] Chen, J., Tam, Y.C., 2025. Predicate-guided generation for mathematical reasoning, in: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Suzhou, China. p. 9097–9110. URL:https://doi.or g/10.18653/v1/2025.emnlp-main.462 , doi:10.186 53/v1/2025.emnlp-main.462. [17]Comanici, G., Bieber, E., Schaekermann, M., Pasupat, I., Sachdeva, N., Dhillon, I., Blistein, M., Ram, O., Zhang, D., Rosen, E., et al., 2025. Gemini 2.5: Pushing the fron- tier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. ArXiv preprint 18 arXiv:2507.06261. URL:https://arxiv.org/abs/25 07.06261. [18] Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzmán, F., Grave, E., Ott, M., Zettlemoyer, L., Stoyanov, V., 2020. Unsupervised cross-lingual rep- resentation learning at scale, in: Jurafsky, D., Chai, J., Schluter, N., Tetreault, J. (Eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Lin- guistics, Association for Computational Linguistics, On- line. p. 8440–8451. URL:https://aclanthology.o rg/2020.acl-main.747/, doi:10.18653/v1/2020.a cl-main.747. [19] DeepSeek-AI, Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang, P., Bi, X., Zhang, X., Yu, X., Wu, Y., Wu, Z.F., Gou, Z., Shao, Z., Li, Z., Gao, Z., Liu, A., Xue, B., Wang, B., Wu, B., Feng, B., Lu, C., Zhao, C., Deng, C., Zhang, C., Ruan, C., Dai, D., Chen, D., Ji, D., Li, E., Lin, F., Dai, F., Luo, F., Hao, G., Chen, G., Li, G., Zhang, H., Bao, H., Xu, H., Wang, H., Ding, H., Xin, H., Gao, H., Qu, H., Li, H., Guo, J., Li, J., Wang, J., Chen, J., Yuan, J., Qiu, J., Li, J., Cai, J.L., Ni, J., Liang, J., Chen, J., Dong, K., Hu, K., Gao, K., Guan, K., Huang, K., Yu, K., Wang, L., Zhang, L., Zhao, L., Wang, L., Zhang, L., Xu, L., Xia, L., Zhang, M., Zhang, M., Tang, M., Li, M., Wang, M., Li, M., Tian, N., Huang, P., Zhang, P., Wang, Q., Chen, Q., Du, Q., Ge, R., Zhang, R., Pan, R., Wang, R., Chen, R.J., Jin, R.L., Chen, R., Lu, S., Zhou, S., Chen, S., Ye, S., Wang, S., Yu, S., Zhou, S., Pan, S., Li, S.S., Zhou, S., Wu, S., Yun, T., Pei, T., Sun, T., Wang, T., Zeng, W., Zhao, W., Liu, W., Liang, W., Gao, W., Yu, W., Zhang, W., Xiao, W.L., An, W., Liu, X., Wang, X., Chen, X., Nie, X., Cheng, X., Liu, X., Xie, X., Liu, X., Yang, X., Li, X., Su, X., Lin, X., Li, X.Q., Jin, X., Shen, X., Chen, X., Sun, X., Wang, X., Song, X., Zhou, X., Wang, X., Shan, X., Li, Y.K., Wang, Y.Q., Wei, Y.X., Zhang, Y., Xu, Y., Li, Y., Zhao, Y., Sun, Y., Wang, Y., Yu, Y., Zhang, Y., Shi, Y., Xiong, Y., He, Y., Piao, Y., Wang, Y., Tan, Y., Ma, Y., Liu, Y., Guo, Y., Ou, Y., Wang, Y., Gong, Y., Zou, Y., He, Y., Xiong, Y., Luo, Y., You, Y., Liu, Y., Zhou, Y., Zhu, Y.X., Huang, Y., Li, Y., Zheng, Y., Zhu, Y., Ma, Y., Tang, Y., Zha, Y., Yan, Y., Ren, Z.Z., Ren, Z., Sha, Z., Fu, Z., Xu, Z., Xie, Z., Zhang, Z., Hao, Z., Ma, Z., Yan, Z., Wu, Z., Gu, Z., Zhu, Z., Liu, Z., Li, Z., Xie, Z., Song, Z., Pan, Z., Huang, Z., Xu, Z., Zhang, Z., Zhang, Z., 2025. DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning. ArXiv preprint arXiv:2501.12948. URL:https://arxi v.org/abs/2501.12948. [20]Deng, Y., Bansal, H., Yin, F., Peng, N., Wang, W., Chang, K.W., 2025.OpenVLThinker: An early exploration to complex vision-language reasoning via iterative self- improvement. ArXiv preprint arXiv:2503.17352. URL: https://arxiv.org/abs/2503.17352. [21]Devlin, J., Chang, M.W., Lee, K., Toutanova, K., 2019. BERT: Pre-training of deep bidirectional transformers for language understanding, in: Burstein, J., Doran, C., Solorio, T. (Eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Association for Com- putational Linguistics, Minneapolis, MN, USA. p. 4171– 4186. URL:https://aclanthology.org/N19-1423/, doi:10.18653/v1/N19-1423. [22]Dimitrov, D., Alam, F., Hasanain, M., Hasnat, A., Silvestri, F., Nakov, P., Da San Martino, G., 2024. SemEval-2024 task 4: Multilingual detection of persuasion techniques in memes, in: Ojha, A.K., Do ̆ gruöz, A.S., Tayyar Mad- abushi, H., Da San Martino, G., Rosenthal, S., Rosá, A. (Eds.), Proceedings of the 18th International Workshop on Semantic Evaluation, Association for Computational Linguistics, Mexico City, Mexico. p. 2009–2026. URL: https://aclanthology.org/2024.semeval-1.275. [23]Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N., 2021. An image is worth 16x16 words: Transformers for image recognition at scale, in: Proceedings of the International Conference on Learning Representations, OpenReview.net, Virtual. URL:https://openreview.net/forum?id= YicbFdNTTy. [24]Dror, R., Baumer, G., Shlomov, S., Reichart, R., 2018. The hitchhiker’s guide to testing statistical significance in natural language processing, in: Proceedings of the 56th Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers), Association for Compu- tational Linguistics, Melbourne, Australia. p. 1383–1392. [25]Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al., 2024. The Llama 3 herd of models. arXiv preprint arXiv:2407.21783 . [26]FANAR TEAM, Abbas, U., Ahmad, M.S., Ahmad, M., Al-Homaid, A., Al-Nuaimi, A., Altinisik, E., Asgari, E., Chawla, S., Chowdhury, S., Dalvi, F., Darwish, K., Durrani, N., Elfeky, M., Elmagarmid, A., Eltabakh, M., Ersoy, A., Fatehkia, M., Hashim, M.Q., Hawasly, M., Hefeeda, M., Husaini, M., Isufaj, K., Jung, S.G., Lachemat, H., Lucas, J.K., Mohamed, A., Mohiuddin, T., Mousi, B., Mubarak, H., Musleh, A., Ouzzani, M., Sadeghi, A., Sencar, H.T., Shinoy, M., Sinan, O., Zhang, Y., 2026. Fanar 2.0: Arabic generative AI stack. ArXiv preprint arXiv:2603.16397. URL: https://arxiv.org/abs/2603.16397. [27]Fersini, E., Gasparini, F., Rizzi, G., Saibene, A., Chulvi, B., Rosso, P., Lees, A., Sorensen, J., 2022. SemEval- 2022 task 5: Multimedia automatic misogyny identifica- tion, in: Emerson, G., Schluter, N., Stanovsky, G., Ku- mar, R., Palmer, A., Schneider, N., Singh, S., Ratan, S. (Eds.), Proceedings of the 16th International Work- shop on Semantic Evaluation, Association for Computa- tional Linguistics, Seattle, WA, USA. p. 533–549. URL: 19 https://aclanthology.org/2022.semeval-1.74, doi:10.18653/v1/2022.semeval-1.74. [28]Firooz, H., Liu, R., Lu, Y., Hou, Z., Xiong, F., Zhang, X., Jian, C., Zhu, Z., Ma, J., Tao, J., et al., 2025. Scaling reinforcement learning for content moderation with large language models. ArXiv preprint arXiv:2512.20061. URL: https://arxiv.org/abs/2512.20061. [29] Grasso, B., La Gatta, V., Moscato, V., Sperlì, G., 2024. KERMIT: Knowledge-empowered model in harmful meme detection. Information Fusion 106, 102269. URL:https: //doi.org/10.1016/j.inffus.2024.102269 , doi:10.1016/j.inffus.2024.102269. [30]Gwet, K.L., 2008. Computing inter-rater reliability and its variance in the presence of high agreement. British Journal of Mathematical and Statistical Psychology 61, 29–48. [31] Hasanain, M., Ahmad, F., Alam, F., 2024a. Can GPT-4 identify propaganda? Annotation and detection of propa- ganda spans in news articles, in: Calzolari, N., Kan, M.Y., Hoste, V., Lenci, A., Sakti, S., Xue, N. (Eds.), Proceedings of the 2024 Joint International Conference on Computa- tional Linguistics, Language Resources and Evaluation, ELRA and ICCL, Torino, Italy. p. 2724–2744. URL: https://aclanthology.org/2024.lrec-main.244. [32]Hasanain, M., Ahmad, F., Alam, F., 2024b. Large language models for propaganda span annotation, in: Al-Onaizan, Y., Bansal, M., Chen, Y.N. (Eds.), Findings of the Association for Computational Linguistics: EMNLP 2024, Association for Computational Linguistics, Miami, Florida, USA. p. 14522–14532. URL:https://aclanthology.org/2 024.findings-emnlp.850/, doi:10.18653/v1/2024 .findings-emnlp.850. [33] Hasanain, M., Hasan, M.A., Ahmad, F., Suwaileh, R., Biswas, M.R., Zaghouani, W., Alam, F., 2024c. ArAIEval shared task: Propagandistic techniques detection in uni- modal and multimodal Arabic content, in: Habash, N., Bouamor, H., Eskander, R., Tomeh, N., Abu Farha, I., Abdelali, A., Touileb, S., Hamed, I., Onaizan, Y., Al- hafni, B., Antoun, W., Khalifa, S., Haddad, H., Zitouni, I., AlKhamissi, B., Almatham, R., Mrini, K. (Eds.), Pro- ceedings of the Second Arabic Natural Language Pro- cessing Conference, Association for Computational Lin- guistics, Miami, FL, USA. p. 456–466. URL:https: //aclanthology.org/2024.arabicnlp-1.44. [34] Hasanain, M., Hasan, M.A., Kmainasi, M.B., Sartori, E., Shahroor, A.E., Da San Martino, G., Alam, F., 2025. PropXplain: Can LLMs enable explainable propaganda detection?, in: Christodoulopoulos, C., Chakraborty, T., Rose, C., Peng, V. (Eds.), Findings of the Association for Computational Linguistics: EMNLP 2025, Association for Computational Linguistics, Suzhou, China. p. 23855– 23863. URL:https://aclanthology.org/2025.fi ndings-emnlp.1296/, doi:10.18653/v1/2025.fin dings-emnlp.1296. [35] He, K., Zhang, X., Ren, S., Sun, J., 2016a. Deep resid- ual learning for image recognition, in: Proceedings of the IEEE conference on computer vision and pattern recogni- tion, IEEE. p. 770–778. [36] He, K., Zhang, X., Ren, S., Sun, J., 2016b. Deep resid- ual learning for image recognition, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Las Vegas, NV, USA. p. 770–778. URL:https://doi.org/10.1109/CVPR.2016.90, doi:10.1109/CVPR.2016.90. [37]Hee, M.S., Kumaresan, A., Lee, R.K.W., 2024. Bridg- ing modalities: Enhancing cross-modality hate speech detection with few-shot in-context learning, in: Proceed- ings of the 2024 Conference on Empirical Methods in Natural Language Processing, Association for Compu- tational Linguistics, Miami, FL, USA. URL:https: //doi.org/10.18653/v1/2024.emnlp-main.445, doi:10.18653/v1/2024.emnlp-main.445. [38] Holm, S., 1979. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics 6, 65–70. [39]James, L.R., Demaree, R.G., Wolf, G., 1984. Estimat- ing within-group interrater reliability with and without response bias. Journal of Applied Psychology 69, 85–98. URL:https://doi.org/10.1037/0021-9010.69.1 .85, doi:10.1037/0021-9010.69.1.85. [40]Kaelbling, L.P., Littman, M.L., Moore, A.W., 1996. Rein- forcement learning: A survey. Journal of Artificial Intelli- gence Research 4, 237–285. [41]Kamath, A., Ferret, J., Pathak, S., Vieillard, N., Merhej, R., Perrin, S., Matejovicova, T., Ramé, A., Rivière, M., Rouillard, L., et al., 2025. Gemma 3 technical report. arXiv preprint arXiv:2503.19786 URL:https://arxiv.org/ abs/2503.19786. [42]Kiela, D., Firooz, H., Mohan, A., Goswami, V., Singh, A., Ringshia, P., Testuggine, D., 2020. The hateful memes challenge: Detecting hate speech in multimodal memes, in: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., Lin, H. (Eds.), Advances in Neural Information Processing Systems, Virtual. URL:https://proceedings.neur ips.c/paper/2020/hash/1b84c4cee2b8b3d823b 30e2d604b1878-Abstract.html. [43]Kmainasi, M.B., Hasnat, A., Hasan, M.A., Shahroor, A.E., Alam, F., 2025. MemeIntel: Explainable detection of pro- pagandistic and hateful memes, in: Christodoulopoulos, C., Chakraborty, T., Rose, C., Peng, V. (Eds.), Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, Association for Computational Lin- guistics, Suzhou, China. p. 30251–30267. URL:https: //aclanthology.org/2025.emnlp-main.1539/, doi:10.18653/v1/2025.emnlp-main.1539. 20 [44]Kmainasi, M.B., Khan, R., Shahroor, A.E., Bendou, B., Hasanain, M., Alam, F., 2024. Native vs non-native lan- guage prompting: A comparative analysis, in: Proceedings of the International Conference on Web Information Sys- tems Engineering, Springer, Doha, Qatar. p. 406–420. URL:https://doi.org/10.1007/978-981-96-057 6-7_30, doi:10.1007/978-981-96-0576-7_30. [45]Kmainasi, M.B., Kutlu, M., Shahroor, A.E., Hasnat, A., Alam, F., 2026. Can thinking models think to detect hateful memes?, in: Companion Proceedings of the ACM Web Conference, ACM, Dubai, United Arab Emirates. p. 1–10. URL:https://doi.org/10.1145/3774905.3795465, doi:10.1145/3774905.3795465. [46]Kumari, G., Jain, K., Ekbal, A., 2024. M3Hop-CoT: Misogynous meme identification with multimodal multi- hop chain-of-thought. ArXiv preprint arXiv:2410.09220. URL: https://arxiv.org/abs/2410.09220. [47] Latif, E., Zhou, Y., Guo, S., Gao, Y., Shi, L., Nyaaba, M., Bewerdorff, A., Yang, X., Zhai, X., et al., 2025. Comparative evaluation of OpenAI o1 and human perfor- mance in higher order cognition. Scientific Reports URL: https://doi.org/10.1038/s41598-025-33629-9, doi:10.1038/s41598-025-33629-9. [48]Li, J., Huang, H., Zhang, Y., Xu, P., Chen, X., Song, R., Shi, L., Wang, J., Xu, H., 2024. Self-supervised preference optimization: Enhance your language model with prefer- ence degree awareness, in: Findings of the Association for Computational Linguistics: EMNLP 2024, Association for Computational Linguistics, Miami, FL, USA. p. 14452– 14466. URL:https://aclanthology.org/2024.fi ndings-emnlp.845/. [49] Li, M., Zhang, Y., Long, D., Chen, K., Song, S., Bai, S., Yang, Z., Xie, P., Yang, A., Liu, D., et al., 2026. Qwen3-VL-embedding and Qwen3-VL-reranker: A uni- fied framework for state-of-the-art multimodal retrieval and ranking. ArXiv preprint arXiv:2601.04720. URL: https://arxiv.org/abs/2601.04720. [50]Lin, H., Luo, Z., Gao, W., Ma, J., Wang, B., Yang, R., 2024.Towards explainable harmful meme detec- tion through multimodal debate between large language models, in: Chua, T., Ngo, C., Kumar, R., Lauw, H.W., Lee, R.K. (Eds.), Proceedings of the ACM Web Con- ference 2024, ACM, Singapore. p. 2359–2370. URL: https://doi.org/10.1145/3589334.3645381, doi:10.1145/3589334.3645381. [51]Lin, H., Luo, Z., Ma, J., Chen, L., 2023. Beneath the surface: Unveiling harmful memes with multimodal rea- soning distilled from large language models, in: Bouamor, H., Pino, J., Bali, K. (Eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, Association for Computational Linguistics, Singapore. p. 9114–9128. URL:https://aclanthology.org/2023.findings -emnlp.611, doi:10.18653/v1/2023.findings-emn lp.611. [52]Liu, Z., Fan, C., Lou, H., Wu, Y., Deng, K., 2025. MIND: A multi-agent framework for zero-shot harmful meme detection. ArXiv preprint arXiv:2507.06908. URL: https://arxiv.org/abs/2507.06908. [53]Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., Guo, B., 2021. Swin Transformer: Hierarchical vision transformer using shifted windows, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, IEEE, Montreal, QC, Canada. p. 9992–10002. URL:ht tps://doi.org/10.1109/ICCV48922.2021.00986, doi:10.1109/ICCV48922.2021.00986. [54] Liu, Z., Mao, H., Wu, C.Y., Feichtenhofer, C., Darrell, T., Xie, S., 2022. A ConvNet for the 2020s, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, New Orleans, LA, USA. p. 11966– 11976. URL:https://doi.org/10.1109/CVPR52688. 2022.01167, doi:10.1109/CVPR52688.2022.01167. [55] Lu, J., Xu, B., Zhang, X., Zhu, H., Wang, K., Yang, L., Lin, H., 2025. Is having rationales enough? Rethinking knowledge enhancement for multimodal hateful meme de- tection, in: Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Infor- mation Retrieval, ACM, Padua, Italy. p. 559–569. URL: https://doi.org/10.1145/3726302.3730014, doi:10.1145/3726302.3730014. [56]Lu, P., Bansal, H., Xia, T., Liu, J., Li, C., Hajishirzi, H., Cheng, H., Chang, K., Galley, M., Gao, J., 2024. Math- Vista: Evaluating mathematical reasoning of foundation models in visual contexts, in: Proceedings of the Interna- tional Conference on Learning Representations, OpenRe- view.net, Vienna, Austria. URL:https://openreview .net/forum?id=KUNzEQMWU7. [57]Luo, J., Luo, X., Chen, X., Xiao, Z., Ju, W., Zhang, M., 2025. Semi-supervised fine-tuning for large language models, in: Findings of the Association for Computa- tional Linguistics: NAACL 2025, Association for Compu- tational Linguistics, Albuquerque, NM, USA. p. 2795– 2808. URL:https://aclanthology.org/2025.find ings-naacl.151/. [58] Mathias, L., Nie, S., Mostafazadeh Davani, A., Kiela, D., Prabhakaran, V., Vidgen, B., Waseem, Z., 2021. Findings of the WOAH 5 shared task on fine grained hateful memes detection, in: Mostafazadeh Davani, A., Kiela, D., Lambert, M., Vidgen, B., Prabhakaran, V., Waseem, Z. (Eds.), Proceedings of the 5th Workshop on Online Abuse and Harms (WOAH 2021), Association for Computational Linguistics, Online. p. 201–206. URL: https://aclanthology.org/2021.woah-1.21, doi:10.18653/v1/2021.woah-1.21. 21 [59]Mei, J., Chen, J., Lin, W., Byrne, B., Tomalin, M., 2024. Improving hateful meme detection through retrieval- guided contrastive learning, in: Ku, L.W., Martins, A., Srikumar, V. (Eds.), Proceedings of the 62nd Annual Meet- ing of the Association for Computational Linguistics (Vol- ume 1: Long Papers), Association for Computational Linguistics, Bangkok, Thailand. p. 5333–5347. URL: https://aclanthology.org/2024.acl-long.291/, doi:10.18653/v1/2024.acl-long.291. [60]Mei, J., Sun, M., Chen, J., Qin, P., Li, Y., Chen, D., Byrne, B., 2025. ExPO-HM: Learning to explain-then-detect for hateful meme detection. ArXiv preprint arXiv:2510.08630. URL: https://arxiv.org/abs/2510.08630. [61]Mih ̆ ailescu, M.G., 2024. Never mess with the “memers”: How meme creators are redefining contemporary politics. Social Media + Society 10, 20563051241296256. URL: https://doi.org/10.1177/20563051241296256, doi:10.1177/20563051241296256. [62] OpenAI, 2023. GPT-4 technical report. ArXiv preprint arXiv:2303.08774. URL:https://arxiv.org/abs/23 03.08774. [63]OpenAI, 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 . [64] Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., Assran, M., Ballas, N., Galuba, W., Howes, R., Huang, P.Y., Li, S.W., Misra, I., Rabbat, M., Sharma, V., Synnaeve, G., Xu, H., Jégou, H., Mairal, J., Labatut, P., Joulin, A., Bojanowski, P., 2024. DINOv2: Learning robust visual features without supervision. Transactions on Machine Learning Research URL:https://openre view.net/forum?id=a68SUt6zFt. [65]Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C.L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P.F., Leike, J., Lowe, R., 2022. Training language models to follow in- structions with human feedback, in: Advances in Neural Information Processing Systems, New Orleans, LA, USA. URL:http://papers.nips.c/paper_files/paper /2022/hash/b1efde53be364a73914f58805a00173 1-Abstract-Conference.html. [66]Pandiani, D.S.M., Sang, E.T.K., Ceolin, D., 2025. ‘Toxic’ memes: A survey of computational perspectives on the detection and explanation of meme toxicities. Online Social Networks and Media 47, 100317. URL:http s://doi.org/10.1016/j.osnem.2025.100317 , doi:10.1016/j.osnem.2025.100317. [67]Pramanick, S., Sharma, S., Dimitrov, D., Akhtar, M.S., Nakov, P., Chakraborty, T., 2021. MOMENTA: A multi- modal framework for detecting harmful memes and their targets, in: Moens, M.F., Huang, X., Specia, L., Yih, S.W.t. (Eds.), Findings of the Association for Computational Lin- guistics: EMNLP 2021, Association for Computational Linguistics, Punta Cana, Dominican Republic. p. 4439– 4455. URL:https://aclanthology.org/2021.find ings-emnlp.379 , doi:10.18653/v1/2021.finding s-emnlp.379. [68] Rafailov, R., Sharma, A., Mitchell, E., Manning, C.D., Ermon, S., Finn, C., 2023. Direct preference optimization: Your language model is secretly a reward model, in: Ad- vances in Neural Information Processing Systems, New Orleans, LA, USA. URL:http://papers.nips.c/p aper_files/paper/2023/hash/a85b405ed65c647 7a4fe8302b5e06ce7-Abstract-Conference.html. [69]Ranaldi, L., Pucci, G., 2025. Multilingual reasoning via self-training, in: Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technolo- gies (Volume 1: Long Papers), Association for Compu- tational Linguistics, Albuquerque, NM, USA. p. 11566– 11582. URL:https://doi.org/10.18653/v1/2025 .naacl-long.577, doi:10.18653/v1/2025.naacl-l ong.577. [70] Ren, C., Jeong, D., Wu, M., Huang, Y., Gao, Y., Li, Y., 2026. A survey of multimodal hate meme detection. Expert Systems with Applications , 132507. [71]Sanh, V., Debut, L., Chaumond, J., Wolf, T., 2019. Distil- BERT, a distilled version of BERT: Smaller, faster, cheaper and lighter. ArXiv preprint arXiv:1910.01108. URL: https://arxiv.org/abs/1910.01108. [72]Schmid, U.K., 2025. Humorous hate speech on social me- dia: A mixed-methods investigation of users’ perceptions and processing of hateful memes. New Media & Society 27, 1588–1606. URL:https://doi.org/10.1177/14 614448231198169, doi:10.1177/1461444823119816 9. [73]Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O., 2017. Proximal policy optimization algo- rithms. ArXiv preprint arXiv:1707.06347. URL:https: //arxiv.org/abs/1707.06347. [74] Shahroor, A.E., Kmainasi, M.B., Hasnat, A., Dimitrov, D., Martino, G.D.S., Nakov, P., Alam, F., 2026. Memelens: Multilingual multitask vlms for memes. arXiv preprint arXiv:2601.12539 . [75]Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y.K., Wu, Y., Guo, D., 2024. DeepSeek- Math: Pushing the limits of mathematical reasoning in open language models. ArXiv preprint arXiv:2402.03300. URL: https://arxiv.org/abs/2402.03300. [76]Sharma, C., Bhageria, D., Scott, W., PYKL, S., Das, A., Chakraborty, T., Pulabaigari, V., Gambäck, B., 2020. 22 SemEval-2020 task 8: Memotion analysis — the visuo- lingual metaphor!, in: Herbelot, A., Zhu, X., Palmer, A., Schneider, N., May, J., Shutova, E. (Eds.), Proceedings of the Fourteenth Workshop on Semantic Evaluation, Asso- ciation for Computational Linguistics, Barcelona (online). p. 759–773. URL:https://aclanthology.org/202 0.semeval-1.99, doi:10.18653/v1/2020.semeval -1.99. [77]Sharma, S., Alam, F., Akhtar, M.S., Dimitrov, D., Martino, G.D.S., Firooz, H., Halevy, A.Y., Silvestri, F., Nakov, P., Chakraborty, T., 2022. Detecting and understanding harm- ful memes: A survey, in: Raedt, L.D. (Ed.), Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, International Joint Conferences on Artificial Intelligence Organization, Vienna, Austria. p. 5597–5606. URL:https://doi.org/10.24963/ijcai.2022/781, doi:10.24963/IJCAI.2022/781. [78] Shrestha, I., Rusert, J., 2020. NLP_UIOWA at SemEval- 2020 task 8: You’re not the only one cursed with knowl- edge — multi branch model memotion analysis, in: Her- belot, A., Zhu, X., Palmer, A., Schneider, N., May, J., Shutova, E. (Eds.), Proceedings of the Fourteenth Work- shop on Semantic Evaluation, International Committee for Computational Linguistics, Barcelona (online). p. 891– 900. URL:https://aclanthology.org/2020.seme val-1.113, doi:10.18653/v1/2020.semeval-1.113. [79]Shridhar, K., Stolfo, A., Sachan, M., 2023. Distilling reasoning capabilities into smaller language models, in: Rogers, A., Boyd-Graber, J., Okazaki, N. (Eds.), Findings of the Association for Computational Linguistics: ACL 2023, Association for Computational Linguistics, Toronto, Canada. p. 7059–7073. URL:https://aclantholo gy.org/2023.findings-acl.441, doi:10.18653/v1/ 2023.findings-acl.441. [80]Singhal, P., Goyal, T., Xu, J., Durrett, G., 2024. A long way to go: Investigating length correlations in RLHF, in: Proceedings of the First Conference on Language Model- ing, Philadelphia, PA, USA. URL:https://openrevi ew.net/forum?id=G8LaO1P0xv. [81]Skalse, J., Howe, N.H.R., Krasheninnikov, D., Krueger, D., 2022. Defining and characterizing reward hacking, in: Proceedings of the 36th International Conference on Neural Information Processing Systems, New Orleans, LA, USA. [82] Sun, L., Ahuja, C., Chen, P., D’Zmura, M., Batmanghe- lich, K., Bontrager, P., 2025. Multi-modal large language models are effective vision learners, in: Proceedings of the IEEE/CVF Winter Conference on Applications of Com- puter Vision, IEEE, Tucson, AZ, USA. p. 8617–8626. URL:https://openaccess.thecvf.com/content/ WACV2025/html/Sun_Multi-Modal_Large_Languag e_Models_are_Effective_Vision_Learners_WAC V_2025_paper.html. [83] Tan, H., Pan, J., 2025. GTPO and GRPO-S: Token- and sequence-level reward shaping with policy entropy. ArXiv preprint arXiv:2508.04349. URL:https://arxiv.org/ abs/2508.04349. [84] Team, G., Kamath, A., Ferret, J., Pathak, S., Vieillard, N., Merhej, R., Perrin, S., Matejovicova, T., Ramé, A., Rivière, M., et al., 2025a. Gemma 3 technical report. ArXiv preprint arXiv:2503.19786. URL:https://arxiv.org/abs/25 03.19786. [85] Team, K., Du, A., Yin, B., Xing, B., Qu, B., Wang, B., Chen, C., Zhang, C., Du, C., Wei, C., et al., 2025b. Kimi- VL technical report. ArXiv preprint arXiv:2504.07491. URL: https://arxiv.org/abs/2504.07491. [86]Wang, H., Yang, Y., Hu, J., Zhu, M., Chen, W., 2026. V-Zero: Self-improving multimodal reasoning with zero annotation. ArXiv preprint arXiv:2601.10094. URL:ht tps://arxiv.org/abs/2601.10094. [87] Wang, W., Gao, Z., Gu, L., Pu, H., Cui, L., Wei, X., Liu, Z., Jing, L., Ye, S., Shao, J., Wang, Z., Chen, Z., Zhang, H., Yang, G., Wang, H., Wei, Q., Yin, J., Li, W., Cui, E., Chen, G., Ding, Z., Tian, C., Wu, Z., Xie, J., Li, Z., Yang, B., Duan, Y., Wang, X., Hou, Z., Hao, H., Zhang, T., Li, S., Zhao, X., Duan, H., Deng, N., Fu, B., He, Y., Wang, Y., He, C., Shi, B., He, J., Xiong, Y., Lv, H., Wu, L., Shao, W., Zhang, K., Deng, H., Qi, B., Ge, J., Guo, Q., Zhang, W., Zhang, S., Cao, M., Lin, J., Tang, K., Gao, J., Huang, H., Gu, Y., Lyu, C., Tang, H., Wang, R., Lv, H., Ouyang, W., Wang, L., Dou, M., Zhu, X., Lu, T., Lin, D., Dai, J., Su, W., Zhou, B., Chen, K., Qiao, Y., Wang, W., Luo, G., 2025. InternVL3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency. ArXiv preprint arXiv:2508.18265. URL: https://arxiv.org/abs/2508.18265. [88] Wei, L., Li, Y., Wang, C., Wang, Y., Kong, L., Huang, W., Sun, L., 2025. First SFT, second RL, third UPT: Continual improving multi-modal LLM reasoning via unsupervised post-training. ArXiv preprint arXiv:2505.22453. URL: https://arxiv.org/abs/2505.22453. [89]Wu, F., Chen, G., Cao, J., Yan, Y., Li, Z., 2024a. Multi- modal hateful meme classification based on transfer learn- ing and a cross-mask mechanism. Electronics 13, 2780. URL:https://doi.org/10.3390/electronics131 42780, doi:10.3390/electronics13142780. [90] Wu, J., Zhong, M., Xing, S., Lai, Z., Liu, Z., Chen, Z., Wang, W., Zhu, X., Lu, L., Lu, T., Luo, P., Qiao, Y., Dai, J., 2024b. VisionLLM v2: An end-to-end generalist multimodal large language model for hundreds of vision- language tasks, in: Globersons, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J.M., Zhang, C. (Eds.), Advances in Neural Information Processing Systems, Van- couver, BC, Canada. URL:http://papers.nips.c /paper_files/paper/2024/hash/81a60d18e010b27 b36cd465c6604b915-Abstract-Conference.html. 23 [91]Wu, X., 2025. Sailing by the stars: A survey on reward models and learning strategies for learning from rewards. ArXiv preprint arXiv:2505.02686. URL:https://arxi v.org/abs/2505.02686. [92] Xu, Y., Hua, Y., Li, S., Wang, Z., 2024. Exploring chain-of- thought for multi-modal metaphor detection, in: Proceed- ings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Asso- ciation for Computational Linguistics, Bangkok, Thailand. p. 91–101. URL:https://aclanthology.org/2024. acl-long.6/, doi:10.18653/v1/2024.acl-long.6. [93]Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al., 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388 . [94]Yang, C., Liu, Y., Zhu, F., Han, J., Hu, S., 2024. Uncertainty-guided modal rebalance for hateful memes detection, in: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Vol- ume 1: Long Papers), Association for Computational Linguistics, Bangkok, Thailand. p. 4361–4371. URL: https://aclanthology.org/2024.acl-long.239/. [95]Yu, Q., Zhang, Z., Zhu, R., Yuan, Y., Zuo, X., Yue, Y., Dai, W., Fan, T., Liu, G., Liu, L., et al., 2025. DAPO: An open-source LLM reinforcement learning system at scale. ArXiv preprint arXiv:2503.14476. URL:https: //arxiv.org/abs/2503.14476. [96]Yu, T., Yao, Y., Zhang, H., He, T., Han, Y., Cui, G., Hu, J., Liu, Z., Zheng, H., Sun, M., 2024. RLHF-V: Towards trustworthy MLLMs via behavior alignment from fine- grained correctional human feedback, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, Seattle, WA, USA. p. 13807–13816. URL:https://doi.org/10.1109/CVPR52733.2024 .01310, doi:10.1109/CVPR52733.2024.01310. [97]Yue, X., Ni, Y., Zheng, T., Zhang, K., Liu, R., Zhang, G., Stevens, S., Jiang, D., Ren, W., Sun, Y., Wei, C., Yu, B., Yuan, R., Sun, R., Yin, M., Zheng, B., Yang, Z., Liu, Y., Huang, W., Sun, H., Su, Y., Chen, W., 2024. MMMU: A massive multi-discipline multimodal understanding and reasoning benchmark for expert AGI, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, Seattle, WA, USA. p. 9556–9567. URL:https://doi.org/10.1109/CVPR52733.2024 .00913, doi:10.1109/CVPR52733.2024.00913. [98] Zhang, T., Kishore, V., Wu, F., Weinberger, K.Q., Artzi, Y., 2020. BERTScore: Evaluating text generation with BERT, in: Proceedings of the International Conference on Learning Representations, OpenReview.net, Addis Ababa, Ethiopia. URL:https://openreview.net/forum?i d=SkeHuCVFDr. [99]Zhu, X., Li, J., Liu, Y., Ma, C., Wang, W., 2024. Distilling mathematical reasoning capabilities into small language models. Neural Networks 179, 106594. URL:https: //doi.org/10.1016/j.neunet.2024.106594 , doi:10 .1016/j.neunet.2024.106594. 24