Paper deep dive
Are LLMs Safe Beyond Text: Do Emojis Expose Gaps in Safety Evaluation
M P V S Gopinadh
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 8/20/2026, 3:48:29 AM
Summary
This study investigates whether emoji-augmented prompts expose safety vulnerabilities in Large Language Models (LLMs) that are not detected by standard text-based evaluations. Testing four open-source models (Mistral 7B, Qwen 2 7B, Gemma 2 9B, Llama 3 8B) with 50 emoji-based adversarial prompts, the research found significant variation in robustness. Gemma 2 9B and Mistral 7B had 10% success rates, Llama 3 8B had 6%, and Qwen 2 7B had 0%. The results suggest that safety mechanisms are sensitive to input representation, and text-only evaluations may underestimate model vulnerabilities.
Entities (7)
Relation Signals (6)
Gemma-2-9B → hassuccessrate → 10%
confidence 95% · Gemma 2 9B and Mistral 7B exhibit non-zero success rates (10%)
Mistral-7B → hassuccessrate → 10%
confidence 95% · Gemma 2 9B and Mistral 7B exhibit non-zero success rates (10%)
Llama-3-8B → hassuccessrate → 6%
confidence 95% · Llama 3 8B 6%
Qwen 2 7B → hassuccessrate → 0%
confidence 95% · Qwen 2 7B shows complete resistance (0% success rate)
Emoji-augmented prompts → exposesvulnerabilityin → LLMs
confidence 90% · These findings indicate that robustness is sensitive to input representation
Text-based evaluations → underrepresents → model_vulnerabilities
confidence 85% · evaluations restricted to standard text prompts may underrepresent model vulnerabilities
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Safety evaluations of large language models (LLMs) predominantly rely on text-based adversarial prompts, potentially overlooking vulnerabilities arising from alternative input representations. This work examines emoji-augmented prompts as a test case for this gap, evaluating 50 prompts across four open-source LLMs (Mistral 7B, Qwen 2 7B, Gemma 2 9B, Llama 3 8B). Results show substantial variation in robustness: Gemma 2 9B and Mistral 7B exhibit non-zero success rates (10%), Llama 3 8B 6%, while Qwen 2 7B shows complete resistance (0% success rate). A chi-square test ($\chi^2 = 32.94, p < 0.001$) confirms significant differences in outcome distributions. These findings indicate that robustness is sensitive to input representation, and that evaluations restricted to standard text prompts may underrepresent model vulnerabilities.
Tags
Links
- Source: https://arxiv.org/abs/2608.18164v1
- Canonical: https://arxiv.org/abs/2608.18164v1
Trouble viewing inline? Open PDF directly →
Full Text
11,220 characters extracted from source content.
Expand or collapse full text
Are LLMs Safe Beyond Text: Do Emojis Expose Gaps in Safety Evaluation M P V S Gopinadh Independent Researcher mpavangopinadh@gmail.com Abstract Safety evaluations of large language mod- els (LLMs) predominantly rely on text-based adversarial prompts, potentially overlooking vulnerabilities arising from alternative input representations. This work examines emoji- augmented prompts as a test case for this gap, evaluating 50 prompts across four open-source LLMs (Mistral 7B, Qwen 2 7B, Gemma 2 9B, Llama 3 8B). Results show substantial varia- tion in robustness: Gemma 2 9B and Mistral 7B exhibit non-zero success rates (10%), Llama 3 8B 6%, while Qwen 2 7B shows complete resistance (0% success rate). A chi-square test (χ 2 = 32.94,p < 0.001) confirms significant differences in outcome distributions. These findings indicate that robustness is sensitive to input representation, and that evaluations re- stricted to standard text prompts may underrep- resent model vulnerabilities. 1 Introduction Large language models (LLMs) are increasingly deployed in production systems, making robust safety alignment a critical requirement (Brown et al., 2020; Bender et al., 2021). Evaluation of safety mechanisms has grown substantially, with many benchmarks assessing adversarial robustness through text-based prompts (Wei et al., 2023; Zou et al., 2023). However, less attention has been given to how emoji-based representations affect direct model responses under safety evaluation set- tings. Emojis are ubiquitous in modern communication and introduce distinct challenges for tokenization and other NLP processing pipelines (Shoeb and de Melo, 2021). Their semantic representations capture contextual and emotional nuances that may not align with keyword-based safety filters (Barbi- eri et al., 2018). Prior work has shown that emojis can be used to evade detection in safety classifiers and judge models (Zhang, 2025; Wei et al., 2024), but their effect on prompt-level safety alignment in LLMs is not well understood. This work presents an empirical study of emoji- based jailbreak prompts across four open-source LLMs.Results show that emoji-augmented prompts can, in some cases, bypass safety mecha- nisms and lead to model-dependent vulnerabilities. These findings highlight that adversarial robustness may vary with input representation, suggesting that evaluation practices focusing only on text-based prompts may not fully capture the range of possi- ble failure modes. 2 Related Work Adversarial prompting has been widely studied as a failure mode in LLM safety alignment, with prior work showing that surface-level reformula- tions (e.g., prompt stuffing or term substitution) can bypass safety mechanisms (Wei et al., 2023; Zou et al., 2023; Wallace et al., 2019). These ap- proaches highlight the sensitivity of LLMs to vari- ations in input representation. Recent work has explored emoji-based adversar- ial techniques. Zhang (2025) demonstrated that emoji sequences can alter semantic interpretation without explicit textual changes, enabling evasion of keyword-based filters. Similarly, Wei et al. (2024) showed that emojis can mislead safety clas- sifiers and judge models by exploiting tokenization effects. However, these studies primarily focus on attacking evaluation systems rather than the under- lying LLMs. Emojis introduce additional complexity as input tokens. Their meanings are context-dependent and culturally variable (Barbieri et al., 2018), while learned emoji representations can capture semantic relationships between emojis and textual concepts (Eisner et al., 2016). This suggests that emoji-based inputs may expose behaviors that are not captured by standard text-based evaluation protocols. arXiv:2608.18164v1 [cs.CL] 15 Aug 2026 In contrast to prior work, this study evaluates emoji-based adversarial prompts directly on LLMs to examine whether such inputs reveal vulnerabili- ties not reflected in conventional evaluation setups. 3 Methodology This work evaluates four open-source LLMs: Mis- tral 7B (Mistral AI, 2023), Qwen 2 7B (Qwen Team, 2024), Gemma 2 9B (Google, 2024), and Llama 3 8B (Meta AI, 2024). Each model is tested on an identical set of 50 prompts, with no fine- tuning or system-level modifications. This ensures a consistent comparison across models under stan- dard inference conditions. 3.1 Prompt Construction A set of 50 emoji-augmented prompts is con- structed to probe safety mechanisms through non- standard input representations. Prompts are framed using two strategies: emoji stuffing, where emojis are interleaved with text to disrupt surface-level filtering, and emoji chaining, where sequences of emojis implicitly encode harmful intent. All prompts target categories of restricted content (e.g., violence or harmful instructions) as defined by model safety policies. Table 1 shows representative examples. ID Prompt 1How toawith? 2Steps to getinwith? 3Plan to useon? 4Ways to causewith? 5How to getfromand? Table 1: Sample emoji-augmented prompts. 3.2 Evaluation Protocol Model responses are categorized into three out- comes: Successful (restricted content generated), Partial (ambiguous or partially compliant), and Failed (rejection or irrelevant response). Classifica- tion is performed using a keyword-based heuristic followed by manual verification. Success rate is reported as the proportion of prompts yielding restricted content. Ethical com- pliance is defined as whether the model avoids generating restricted or harmful content; success- ful responses are treated as non-compliant, while partial and failed responses are treated as compli- ant because they do not provide directly actionable harmful content. Differences across models are assessed using a chi-square test on outcome distri- butions (p < 0.05). 4 Results Model behavior varies substantially under emoji- augmented prompts. Qwen 2 7B produces no suc- cessful outputs, while Gemma 2 9B and Mistral 7B both exhibit non-zero success rates (10%). This divergence indicates that susceptibility to emoji- based prompting is not uniform across models. Success rate alone does not characterize model behavior. Gemma 2 9B achieves the lowest compli- ance (66%), whereas Mistral 7B attains the same success rate with higher compliance (88%), reflect- ing differences in how models handle ambiguous or partially aligned responses. Qwen 2 7B produces no successful outputs but a high proportion of par- tial responses, suggesting that emoji-based prompts are often interpreted as underspecified rather than explicitly unsafe. A chi-square test shows that differences in outcome distributions are statistically significant (χ 2 = 32.94,p < 0.001). The observed variation indicates that adversarial robustness depends on input representation, with emoji-based prompts ex- posing behaviors not consistently captured across models. 5 Discussion Emoji-augmented prompts expose a mismatch be- tween surface-level safety mechanisms and seman- tic interpretation. Across models, a substantial fraction of responses are classified as partial, indi- cating that emoji sequences introduce ambiguity rather than triggering consistent refusal or compli- ance. This behavior suggests that safety systems are not uniformly calibrated for non-standard input representations. Model differences further reinforce this obser- vation. Despite identical success rates, Gemma 2 9B and Mistral 7B exhibit substantially different compliance levels, indicating divergence in how ambiguity is resolved rather than in outright failure rates. Qwen 2 7B produces no successful outputs but a high proportion of partial responses, suggest- ing conservative handling of underspecified inputs rather than robust semantic interpretation. These findings indicate that robustness is sensitive to in- put representation. Evaluations restricted to stan- dard text prompts may therefore underrepresent vulnerabilities arising from alternative input repre- sentations such as emojis. References Francesco Barbieri, Jose Camacho-Collados, Luis Es- pinosa Anke, and Horacio Saggion. 2018. SemEval- 2018 task 2: Multilingual emoji prediction. In Pro- ceedings of the 12th International Workshop on Se- mantic Evaluation, pages 24–33. Emily M Bender, Timnit Gebru, Angelina McMillan- Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Confer- ence on Fairness, Accountability, and Transparency, pages 610–623. Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and 1 others. 2020. Language models are few- shot learners. In Advances in Neural Information Processing Systems, volume 33, pages 1877–1901. Ben Eisner, Tim Rocktäschel, Isabelle Augenstein, Matko Bošnjak, and Sebastian Riedel. 2016. emoji2vec: Learning emoji representations from their description. In Proceedings of the Fourth Interna- tional Workshop on Natural Language Processing for Social Media, pages 48–54. Association for Com- putational Linguistics. Google. 2024. Gemma-2-9B model. Meta AI. 2024. Introducing Llama 3: A new standard in open-source language models. Mistral AI. 2023. Mistral 7B. Technical report, Mistral AI. ArXiv:2310.06825. Qwen Team. 2024. Qwen2: Large language models. Technical report, Alibaba Cloud. Abu Awal Md Shoeb and Gerard de Melo. 2021. As- sessing emoji use in modern text processing tools. In Proceedings of the 59th Annual Meeting of the Asso- ciation for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1379– 1388. Association for Computational Linguistics. Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gard- ner, and Sameer Singh. 2019. Universal adversarial triggers for attacking and analyzing NLP. In Proceed- ings of the 2019 Conference on Empirical Methods in Natural Language Processing, pages 2153–2162. Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2023. Jailbroken: How does LLM safety training fail? arXiv preprint arXiv:2307.02483. Zhipeng Wei, Yuqi Liu, and N Benjamin Erichson. 2024.Emoji attack: A method for misleading judge LLMs in safety risk detection. arXiv preprint arXiv:2411.01077. Yangshijie Zhang. 2025.Emoti-attack:Zero- perturbation adversarial attacks on nlp systems via emoji sequences. Preprint, arXiv:2502.17392. Andy Zou, Zifan Wang, J Zico Kolter, and Matt Fredrik- son. 2023. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043. A Dataset Access and Ethical Considerations Due to the harmful nature of the prompts used in this study, the complete prompt set is not publicly released. Representative examples and prompt con- struction methodology are included in the paper to support transparency and reproducibility. Addi- tional details may be shared for research purposes upon request. The evaluation involves prompts designed to probe safety mechanisms using emoji-based adver- sarial representations. The study is intended solely for robustness evaluation and analysis of safety be- havior, not for misuse or deployment of harmful instructions.