Paper deep dive
Adversarial Robustness of Vision in Open Foundation Models
Jonathan Fox, William J Buchanan, Pavlos Papadopoulos
Models: Llama 3.2 Vision-8B-2, LLaVA-1.5-13B
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 97%
Last extracted: 3/11/2026, 12:39:03 AM
Summary
This paper investigates the adversarial robustness of open-weight Vision-Language Models (VLMs), specifically LLaVA-1.5-13B and Meta's Llama 3.2 Vision-8B-2, against untargeted Projected Gradient Descent (PGD) attacks. The study evaluates these models on the VQA v2 dataset and finds that while Llama 3.2 Vision has a lower baseline accuracy, it exhibits greater robustness (smaller performance drop) under adversarial perturbation compared to LLaVA. The findings suggest that adversarial robustness does not necessarily correlate with standard benchmark performance and is influenced by architectural and training factors.
Entities (4)
Relation Signals (3)
LLaVA-1.5-13B ā evaluatedon ā VQA v2
confidence 100% Ā· These are tested for untargeted PGD... and empirically evaluated on the Visual Question Answering (VQA) v2 dataset subset.
Llama 3.2 Vision-8B-2 ā evaluatedon ā VQA v2
confidence 100% Ā· These are tested for untargeted PGD... and empirically evaluated on the Visual Question Answering (VQA) v2 dataset subset.
Projected Gradient Descent ā attacks ā Vision-Language Model
confidence 95% Ā· The vision modality represents a viable attack vector for degrading the performance of contemporary open-weight VLMs
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:With the increase in deep learning, it becomes increasingly difficult to understand the model in which AI systems can identify objects. Thus, an adversary could aim to modify an image by adding unseen elements, which will confuse the AI in its recognition of an entity. This paper thus investigates the adversarial robustness of LLaVA-1.5-13B and Meta's Llama 3.2 Vision-8B-2. These are tested for untargeted PGD (Projected Gradient Descent) against the visual input modality, and empirically evaluated on the Visual Question Answering (VQA) v2 dataset subset. The results of these adversarial attacks are then quantified using the standard VQA accuracy metric. This evaluation is then compared with the accuracy degradation (accuracy drop) of LLaVA and Llama 3.2 Vision. A key finding is that Llama 3.2 Vision, despite a lower baseline accuracy in this setup, exhibited a smaller drop in performance under attack compared to LLaVA, particularly at higher perturbation levels. Overall, the findings confirm that the vision modality represents a viable attack vector for degrading the performance of contemporary open-weight VLMs, including Meta's Llama 3.2 Vision. Furthermore, they highlight that adversarial robustness does not necessarily correlate directly with standard benchmark performance and may be influenced by underlying architectural and training factors.
Tags
Links
- Source: https://arxiv.org/abs/2512.17902
- Canonical: https://arxiv.org/abs/2512.17902
Trouble viewing inline? Open PDF directly ā
Full Text
97,005 characters extracted from source content.
Expand or collapse full text
Adversarial Robustness of Vision in Open Foundation Models Jonathan Fox 1* , William J Buchanan 1 and Pavlos Papadopoulos 1 1* Blockpass ID Lab, Edinburgh Napier University, Colinton Road, Edinburgh, EH10 5DT, UK. Abstract With the increase in deep learning, it becomes increasingly difficult to under- stand the model in which AI systems can identify objects. Thus, an adversary could aim to modify an image by adding unseen elements, which will confuse the AI in its recognition of an entity. This paper thus investigates the adversar- ial robustness of LLaVA-1.5-13B and Metaās Llama 3.2 Vision-8B-2. These are tested for untargeted PGD (Projected Gradient Descent) against the visual input modality, and empirically evaluated on the Visual Question Answering (VQA) v2 dataset subset. The results of these adversarial attacks are then quantified using the standard VQA accuracy metric. This evaluation is then compared with the accuracy degradation (accuracy drop) of LLaVA and Llama 3.2 Vision. A key finding is that Llama 3.2 Vision, despite a lower baseline accuracy in this setup, exhibited a smaller drop in performance under attack compared to LLaVA, particularly at higher perturbation levels. Overall, the findings confirm that the vision modality represents a viable attack vector for degrading the perfor- mance of contemporary open-weight VLMs, including Metaās Llama 3.2 Vision. Furthermore, they highlight that adversarial robustness does not necessarily cor- relate directly with standard benchmark performance and may be influenced by underlying architectural and training factors. Keywords:Llama 3.2 Vision, Multimodal Foundation Models, Projected Gradient Descent, Vision-Language Model Safety, Adversarial Examples 1 arXiv:2512.17902v1 [cs.CV] 19 Dec 2025 1 Introduction Recent advances in artificial intelligence have led to the development offounda- tion modelsā large-scale models pre-trained on vast, diverse datasets, exhibiting remarkable capabilities across a range of downstream tasks (Bommasani et al. 2021). Prominent examples include Large Language Models (LLMs), which process text, and more recently, Vision-Language Models (VLMs), which integrate visual understanding alongside language capabilities. The proliferation of these models, particularly powerful open-source variants like Metaās Llama series (Touvron et al. 2023a,b; Grattafiori et al. 2024), has democra- tised access but simultaneously introduced significant security considerations (Kapoor et al. 2024). While the security of foundation models is an active research area, much focus has centred on the textual vulnerabilities of LLMs, such as prompt injection or jailbreaking. The integration of vision in VLMs, however, creates an expanded attack surface, introducing vulnerabilities specific to the visual modality that remain comparatively underexplored. Specifically, VLMs are susceptible toadversarial image attacksā carefully crafted, often imperceptible, perturbations applied to input images that can deceive the model and cause erroneous or unintended textual outputs. Understanding this vulnerabil- ity is critical, as these multimodal models are increasingly integrated into diverse applications, from content generation to assistive technologies. This paper addresses the pressing need to assess the adversarial robustness of the vision componentwithin state-of-the-art open-weight VLMs. It specifically investi- gates models related to the influential Llama family, namely LLaVA and Llama 3.2 Vision, and offers one of the first systematic comparisons of visual adversarial robust- ness in popular open models. This shows that robustness does not always align with standard accuracy metrics. 1.1 Aim and objectives This paper aims to investigate the security implications arising from the visual modal- ity in these open-source foundation models. The primary focus is on evaluating the adversarial robustness of two prominent VLMs related to Metaās Llama family: LLaVA and Llama 3.2 Vision, using adversarial image attacks. Additionally, the research seeks to analyse how differences in model architecture and training methodologies might influence their robustness against such visual perturbations. The objectives are as follows: ā¢ Present the state-of-the-art in the technical evolution of language and com- puter vision models; integration of multimodality with pre-trained models; and adversarial machine learning attacks & defences. ā¢ Implement LLaVA and Llama 3.2 Vision and test robustness against adversarial examples using Projected Gradient Descent. 2 ā¢ Quantify, compare and evaluate the results considering the modelsā architecture and training. 2 Background Recent advancements in artificial intelligence, such as the Transformer (Vaswani et al. 2017), have given rise to large language models but also underpin the development of vision-language models. This section lays the groundwork for this evolution, pro- viding the background necessary for language and vision model development, as well as adversarial robustness, to understand the research questions, related work, and methodology of this study. 2.1 Language Models Over the past decade, the field of natural language processing (NLP) has been marked by a shift away from traditional statistical methods towards deep learning models. Prior approaches such as n-gram models (Chen and Goodman 1996), Hid- den Markov Models (HMMs) (Rabiner 1989), and phrase-based machine translation (Koehn et al. 2003) have now been replaced as state of the art (SOTA) by neural network approaches. 2.1.1 Neural Network Approaches to NLP Although early neural network research, including backpropagation, dates back to the 1980s (Rumelhart et al. 1986), these models did not achieve widespread use in NLP due to data and hardware limitations. Indeed, it was not until 2012 that large-scale deep learning gained mainstream attention in AI research. This breakthrough occurred when AlexNet ā a convolutional neural network (CNN) ā achieved remarkable results on the ImageNet Large-Scale Visual Recognition Challenge (ILSVRC-2012). This annual challenge, considered a benchmark for SOTA computer vision, involved the classification of over a million images across 1,000 object categories. AlexNet achieved a top-5 error rate of only 15.3%, significantly outperforming the second-best method, which had an error rate of 26.2% (Krizhevsky et al. 2012). Although this result occurred in the field of computer vision, it demonstrated the effectiveness of large-scale, GPU-accelerated deep learning and reignited interest in applying neural networks to NLP tasks. Following the breakthrough, Recurrent Neural Networks (RNNs) or, specifically, a type known as Long Short-Term Memory (LSTM) (Hochreiter and Schmidhuber 1997), originally developed in 1997 to address the vanishing gradient issue, emerged as a promising candidate for applying deep learning to NLP due to their architecture for sequence modelling that captures temporal dependencies. Then, the development of Recurrent Continuous Translation Models (Kalchbrenner and Blunsom 2013) led to a significant shift away from phrase-based statistical methods towards neural machine translation (NMT). The subsequent introduction of the Sequence-to-Sequence (Seq2Seq) framework (Sutskever et al. 2014) formalised the encoder-decoder architecture for NMT tasks, 3 significantly improving performance over previous statistical approaches. Neverthe- less, the sequential processing characteristic of these models continued to limit their ability to capture long-range dependencies effectively. 2.1.2 Attention & The Transformer Architecture A significant advancement occurred with the introduction of the attention mechanism in the paper āNeural Machine Translation by Jointly Learning to Align and Trans- lateā (Bahdanau et al. 2016). Attention allows models to selectively focus on relevant parts of the input sequence when generating output, overcoming the limitations of fixed-length context vectors used in earlier Seq2Seq models. This dynamic weight- ing mechanism proved crucial for handling long sequences and improving translation quality. Building upon the concept of attention, the Transformer architecture, introduced in Attention Is All You Need(Vaswani et al. 2017), revolutionised sequence modelling by entirely replacing recurrent and convolutional layers with self-attention mechanisms. The Transformer consists of an encoder and a decoder, each composed of multiple identical layers. The encoder maps an input sequence of token embeddings (x 1 ,...,x n ) to a sequence of continuous representationsz= (z 1 ,...,z n ). Each encoder layer contains two sub- layers: a multi-head self-attention mechanism and a position-wise fully connected feed- forward network. Each sub-layer is wrapped with a residual connection followed by layer normalisation. The decoder, conditioned on the encoder outputsz, generates an output sequence (y 1 ,...,y m ) autoregressively, one token at a time. Each decoder layer consists of three sub-layers: a masked multi-head self-attention mechanism (to preserve the autoregres- sive property), a multi-head cross-attention mechanism over the encoder outputs, and a feed-forward network. As with the encoder, each sub-layer is surrounded by residual connections, followed by layer normalisation. The core component, self-attention, enables the model to dynamically weigh the relevance of different tokens in a sequence when computing contextualised represen- tations. Multi-head attention extends this mechanism by projecting the input into multiple subspaces, allowing the model to jointly attend to information from different representation subspaces at different positions. 2.1.3 Foundation Models Leveraging the Transformer architecture, the era of foundation models began with OpenAIās Generative Pre-trained Transformer (GPT) (Radford and Narasimhan 2018). GPT specifically utilised a unidirectional transformer decoder architecture that processed text from left to right. It demonstrated that unsupervised pre-training on vast text corpora followed by supervised fine-tuning could dramatically improve per- formance across diverse NLP tasks. This model established the present paradigm that dominates the field, allowing models to acquire general language understand- ing from unlabelled data before adapting to specific downstream applications. With 4 Fig. 1: The Transformer architecture illustrates the encoder (left) and decoder (right) stacks with multi-head attention and feed-forward layers. Reproduced from Vaswani et al. (2017) (Vaswani et al. 2017). 117 million parameters, GPT achieved state-of-the-art results on numerous language understanding benchmarks. This pre-training approach was significantly scaled in subsequent models like GPT- 2 (Radford et al. 2019) and GPT-3 (Brown et al. 2020)āthe latter incorporating approximately 100 times the parameters of GPT-2 (175 billion). This trend was formalised by the āScaling Laws for Neural Language Modelsā paper (Kaplan et al. 2020), which showed that performance consistently improves with model size and data volume, leading to the pursuit of ever-larger models. However, simply scaling models introduced challenges related to safety, alignment with human values, and instruction following. To address this, techniques like Rein- forcement Learning from Human Feedback (RLHF) were developed (Ouyang et al. 2022). RLHF fine-tunes models based on human preferences, significantly improving their ability to generate helpful, honest, and harmless responses, becoming a standard practice for state-of-the-art models. 5 As the rapid pace of development has continued, fierce competition along with safety concerns among leading AI companies has led to the current landscape where power- ful closed-source models dominate top benchmarks (Wang et al. 2024; Chiang et al. 2024; White et al. 2024). Examples include OpenAIās GPT-4 series (OpenAI et al. 2024), Googleās Gemini family, and Anthropicās Claude 3 models. These models exhibit advanced reasoning, multimodal understanding, and near-human performance on various complex tasks. However, their closed nature limits transparency and broad research access. In parallel, the development of open-weight foundation models has gained significant traction, aiming to democratize access to state-of-the-art AI. Notable examples include Metaās Llama series (Touvron et al. 2023a,b; Grattafiori et al. 2024) and DeepSeekās R1 (DeepSeek-AI et al. 2025). These models offer competitive performance while fostering wider research, innovation, reproducibility, and community scrutiny. 2.2 Vision-Language Models Vision-language models have developed significantly in the last few years, mirroring the evolution of language models in NLP. 2.2.1 Early Multimodal Learning The integration of vision and language has a long history, with early attempts driven by the need for effective content-based image retrieval (CBIR), where users could search for images using natural language queries. Early approaches associated images with textual descriptors by relying on hand- crafted features and simple linguistic cues. These methods typically divided images into regions and attempted to map these segments to corresponding wordsāoften nouns and adjectivesāto facilitate retrieval tasks (Mori et al. 1999). While effective in limited domains, such approaches were constrained by their inability to capture the high-level semantics in visual and textual modalities. The deep learning revolution started by AlexNet (Krizhevsky et al. 2012) in 2012 also had a profound impact on CBIR. The automated extraction of features by Convolutional Neural Networks (CNNs) enabled models to learn complex visual rep- resentations directly from raw pixel data and eliminated the need for manual feature engineering. A significant step was the introduction of DeViSE (Deep Visual-Semantic Embedding) (Frome et al. 2013), a model designed to align visual features with word embeddings within a shared semantic space. DeViSE employed a CNN to extract image features and a separate neural network for word embeddings, projecting both into a common vector space. The training utilised a contrastive loss function, which encouraged the alignment of related image-text pairs while separating unrelated ones. This enabled zero-shot learning capabilities, allowing the model to recognise object categories not seen during training by leveraging learned semantic relationships. The success of 6 DeViSE highlighted the potential of deep learning to bridge vision and language, influencing subsequent multimodal model development. As with NLP, a further leap in integrating vision and language was achieved by incor- porating attention mechanisms. The paper Show, Attend and Tell (Xu et al. 2015) introduced soft and hard attention strategies, enabling models to dynamically focus on regions within images during the caption generation process. This method signifi- cantly improved the descriptive accuracy and interpretability of the resulting captions by explicitly aligning image regions with segments of generated text. The paper, Deep visual-semantic Alignments for generating image descriptions (Karpathy and Fei-Fei 2015), introduced a multimodal recurrent neural network architecture that established strong alignments between specific image regions and corresponding sentence fragments. This approach combined a CNN for visual feature extraction with an RNN for text generation but crucially incorporated a novel align- ment objective that matched specific regions in images with words in captions through a common embedding space. By decomposing both images and sentences into more granular components (regions and words) and explicitly learning the correspondence between them, the model achieved more descriptive and accurate caption generation. This alignment approach established a fundamental technique for grounding language in visual content that influenced subsequent work in vision-language modelling. Following the success of the Transformer architecture in NLP, researchers recognised the potential to extend these models to vision-language tasks. ViLBERT (Vision-and- Language BERT) (Lu et al. 2019) pioneered this extension by building upon the BERT (Bidirectional Encoder Representations from Transformers), the first bidirectional Transformer model (Devlin et al. 2019). ViLBERT introduced a two-stream architecture that allowed for the simultaneous processing of visual and textual information, enabling the model to learn joint representations of both modalities. This was achieved by pre-training on large- scale image-text pairs, allowing the model to learn rich contextual embeddings for both images and text. The modelās architecture consisted of two parallel streams: one for visual features extracted from images using CNNs and another for text tokens. The model employed co-attentional transformer layers to facilitate cross-modal interactions, allowing it to learn joint representations that effectively captured the relationships between visual and textual modalities. It demonstrated state-of-the-art performance across multiple vision-language tasks, including visual question answering and visual commonsense reasoning. Following ViLBERTās dual-stream approach, architectural innovation led to more efficient designs. VisualBERT (Li et al. 2019) introduced a unified single-stream archi- tecture that processed visual and textual data within a single Transformer encoder. Rather than maintaining separate streams, it directly combined region-based visual features (extracted via Faster R-CNN) with word embeddings, allowing self-attention to model cross-modal relationships without explicit interaction layers. 7 UNITER (Chen et al. 2020) further advanced the single-stream approach through four pre-training tasks: masked language modelling, masked region modelling, image- text matching, and optimal transport-based word-region alignment. These objectives enhanced the modelās ability to learn fine-grained cross-modal relationships. Experi- mental results showed that these single-stream architectures consistently outperformed dual-stream counterparts like ViLBERT and LXMERT while using fewer parameters, validating the benefits of unrestricted attention flows between modalities. This archi- tectural shift established a new standard for vision-language integration, particularly for tasks requiring detailed visual reasoning like VQA and image captioning. Despite these advancements, a common bottleneck persisted: the reliance on CNN- based visual feature extraction. CNNs, while powerful, imposed significant compu- tational overhead and limited scalability when integrated into larger multimodal frameworks. This limitation became increasingly apparent as researchers sought to train on larger datasets, with computational costs becoming a major constraint on further progress. 2.2.2 Scaling Vision-Language Learning with Transformers This limitation was addressed by the Vision Transformer (ViT) introduced in (Doso- vitskiy et al. 2020). By partitioning images into fixed-size patches and processing them using self-attention mechanisms, ViT eliminated the need for convolutional opera- tions, thereby aligning the processing paradigms of vision and language under a single Transformer-based framework. The authors demonstrated that when trained on suffi- cient data, this approach could outperform CNNs while offering better computational scaling properties. Building upon the architectural innovations of ViT, contrastive learning emerged as a powerful method for training robust vision-language models. CLIP (Contrastive Language-Image Pre-training) (Radford et al. 2021) paired a vision transformer with a GPT-based text encoder in a dual-encoder architecture. By leveraging contrastive objectives trained on 400 million image-text pairs gathered from the internet, CLIP achieved remarkable zero-shot generalisation across a diverse range of tasks. This approach focused on embedding-based retrieval rather than text generation, learning a shared representation space where related image-text pairs were positioned closer together than unrelated ones. The effectiveness of scale in vision-language learning was further demonstrated by ALIGN (Jia et al. 2021). Unlike CLIP, ALIGN employed a CNN for visual processing paired with a BERT-based text encoder but similarly used a dual-encoder contrastive approach. Its key innovation was training on an even larger dataset of 1.8 billion image-text pairs with significantly noisier annotations. This work empirically validated that model performance continued to improve with data scale even when using nois- ier training signals, establishing that data quantity could effectively compensate for reduced annotation quality in multimodal representation learning. 8 While CLIP and ALIGN demonstrated powerful visual-semantic understanding, they remained limited to retrieval and classification tasks and lacked generative capabil- ities. SimVLM (Wang et al. 2022) addressed this limitation by introducing the first unified encoder-decoder visual language model capable of open-ended text generation from visual inputs. SimVLM employed a novel prefix language modelling objective that allowed for more efficient training with weakly supervised image-text pairs. This closed-source model represented a significant milestone in vision-language integration, enabling tasks like image captioning and visual question answering with a single gen- erative framework rather than requiring task-specific fine-tuning as earlier models did. Building upon the generative capabilities of SimVLM, BLIP (Bootstrapping Language- Image Pre-training) (Li et al. 2022) further advanced open-source vision-language modelling. BLIP introduced a novel architecture combining a Vit encoder, a BERT encoder, and a causal language model decoder in a unified framework. It employed a unique bootstrapping approach with two key innovations: a captioning-and-filtering pipeline to generate synthetic captions for web images and a multimodal mixture of encoder-decoder pre-training objectives. This architecture enabled strong performance across image-text retrieval, visual question answering, and image captioning, making it the first open-source model to excel at both discriminative and generative vision- language tasks. 2.2.3 Multimodal Foundation Models While previous vision-language models demonstrated impressive capabilities in task- specific scenarios, they lacked the flexibility to engage in open-ended, multi-turn conversations grounded in a visual context. Flamingo (Alayrac et al. 2022) addressed this limitation by pioneering a fully conversational vision-language model. By integrat- ing a frozen large language model (Chinchilla) with visual inputs through learnable multimodal adapters, Flamingo enabled seamless interactions between text and images within the same conversation flow. The modelās perceiver resampler archi- tecture efficiently transformed visual features from a frozen vision encoder into a format compatible with the language modelās token representations. Despite being closed-source, Flamingo demonstrated remarkably few-shot learning capabilities across diverse vision-language tasks, setting a new paradigm for multimodal foundation models. While Flamingo achieved impressive results, its approach remained closed-source. BLIP-2 (Li et al. 2023) introduced an open-source alternative through its innova- tive Q-Former architecture. This bootstrapping approach connected a frozen ViT encoder to a frozen large language model via a lightweight query transformer module that served as a multimodal adapter. The Q-Former took as input a set of learn- able query tokens along with key-value pairs from the vision encoderās embeddings, transforming them into query-aligned image representations compatible with the lan- guage modelās input space. This design eliminated the need for end-to-end training of massive multimodal models while still achieving state-of-the-art performance on vari- ous vision-language tasks, including visual question answering, where it outperformed 9 Flamingo on the VQAv2 benchmark. Building upon BLIP-2ās architecture, LLaVA (Large Language and Vision Assistant) (Liu et al. 2023) marked a significant milestone as the first multimodal instruction-tuned model for vision-language tasks. LLaVAās architecture connected a frozen CLIP Vision Encoder (ViT-L/14) with Vicunaāa fine- tuned version of Metaās Llama language modelāthrough a trainable projection layer that mapped visual features into the LLMās word embedding space. This simple yet effective design enabled end-to-end fine-tuning for multimodal instruction-following capabilities. LLaVA employed a two-stage instruction tuning approach: first, a feature alignment pre-training phase using 595K image-text pairs from C3M to establish visual- language connections, followed by instruction fine-tuning on 158K GPT-4-generated multimodal instruction-following samples. These samples included conversational QA (58K samples), detailed descriptions (23K samples), and complex reasoning (77K samples). This comprehensive training enabled LLaVA to achieve 85.1% of GPT-4ās performance on multimodal instruction-following tasks while outperforming contem- poraries like OpenFlamingo and BLIP-2 in instruction adherence and reasoning. When combined with GPT-4 as an ensemble, LLaVA achieved state-of-the-art performance in Science QA (92.53% A significant evolution in vision-language models arrived with OpenAIās GPT-4V (GPT 2024), also known as GPT-4 Vision. While its architecture remains a closed source, it demonstrates impressive multimodal performance. The model has shown human-competitive performance on visual benchmarks while also demonstrating emergent capabilities like optical character recognition, multi- image reasoning, and diagram interpretation without explicit training for these tasks. Despite its impressive capabilities, GPT-4Vās closed-source nature limits aca- demic scrutiny and understanding of its internal mechanisms, creating challenges for researchers seeking to build upon its innovations. LLaMA 3.2 Vision (Grattafiori et al. 2024) represents Metaās entry into multimodal foundation models with an open-weights approach, in contrast to closed systems like GPT-4V. Rather than developing a native multimodal architecture from scratch, Meta implemented an adapter-based approach that preserves the text capabilities of the pre-trained Llama 3.1 language models. The architecture consists of three main components: (1) an image encoder based on Vision Transformer (ViT-H/14) with 850M parameters (compared to LLaVAās ViT-L/14), (2) an image adapter with cross-attention layers that integrate visual infor- mation into the language model, and (3) a video adapter for temporal reasoning. The cross-attention layers, which add approximately 100B parameters for the Llama 3.1 405B model, are strategically inserted after every fourth self-attention layer in the core language model. This represents a significant architectural difference from LLaVA, which used a simple projection layer to map visual features into the LLMās embedding space. 10 While LLaVA was trained on 595K image-text pairs from C3M and 158K GPT- 4-generated instruction samples, Metaās training pipeline for Llama 3.2 Vision is substantially more extensive, using approximately 6B image-text pairs for initial pre-training, followed by 500M higher-quality images for adapter annealing. During adapter training, they update the parameters of the image encoder but intention- ally preserve the language model parameters, ensuring that the text-only capabilities remain intact. This approach creates a drop-in replacement for Llama 3.1 models that can process both visual and textual inputs while maintaining strong language capabilities. The post-training phase follows a similar recipe to the text-only models, with sev- eral rounds of alignment, including supervised fine-tuning, rejection sampling, and direct preference optimisation, along with safety mitigations to produce a model that balances helpfulness with responsible behaviour. 2.3 Adversarial Robustness Adversarial robustness refers to a modelās ability to maintain performance and security when confronted with inputs specifically crafted to deceive it. This property is critical as deep learning systems are increasingly deployed in safety-sensitive domains like autonomous driving, medical diagnostics, and content moderation. For vision-language models (VLMs), robustness is particularly crucial. These models often operate in user- facing applications where vulnerabilities in either modality could lead to significant security risks or unreliable behaviour. The vulnerability of neural networks to adversarial examples was first systemati- cally demonstrated in the seminal paper āIntriguing Properties of Neural Networksā (Szegedy et al. 2014). The authors discovered that imperceptible, non-random per- turbations applied to input images could cause state-of-the-art neural networks to misclassify them with high confidence. These āadversarial examplesā were generated by optimising the input image to maximise the modelās prediction error, revealing a fundamental weakness. This finding presented a paradox: how could models that gen- eralise well on standard test data be so easily fooled by inputs indistinguishable to humans? Subsequent research sought to understand the root causes of this vulnerability. Nguyen et al. (Nguyen et al. 2015) further highlighted the gap between human and machine perception by showing that deep neural networks could confidently clas- sify completely unrecognisable, abstract patterns as familiar objects. This suggested that networks might rely on superficial statistical correlations rather than robust, human-like semantic understanding. Goodfellow et al. (Goodfellow et al. 2015) proposed the ālinear explanation,ā arguing that the high dimensionality of input spaces makes even models with predominantly linear behaviour susceptible. Small perturbations across many dimensions can accu- mulate to cause large changes in the output, even if the model behaves linearly locally. 11 These discoveries have spurred the growth of Adversarial Machine Learning (AML) as a field dedicated to studying attacks on machine learning systems and developing defences. A particularly challenging property, identified early on in one study (Szegedy et al. 2014), is that adversarial examples crafted to fool one model often successfully deceive other models. This is often even for those with different architectures or trained on dif- ferent data subsets. This cross-model vulnerability suggests that adversarial examples exploit fundamental characteristics of the data distribution or the learning process itself rather than just overfitting to specific model parameters. The existence of such adversarial examples has profound implications for real-world systems beyond the theory of visual perturbations, causing incorrect image interpreta- tions. Understanding these vulnerabilities is the first step towards building more secure and reliable AI systems, and this necessitates a structured approach to classifying and analysing these threats. 2.3.1 Taxonomy of Adversarial Machine Learning One of the earliest structured taxonomies to categorise attack methodologies system- atically (Pitropakis et al. 2019) defined adversarial attacks based on two primary phases: preparation and manifestation. Preparation includes access to the model, leading to the distinction between White-box attacks, where the attacker has com- plete knowledge of the modelās architecture, weights, and gradients, and Black-box attacks, where no internal model details are accessible. The manifestation categorises adversarial attacks by their specificity (targeted vs untargeted), mode (colluding vs non-colluding), and type (poisoning/training time attacks vs evasion/run time attacks). This taxonomy established a foundation for categorising threats against machine learning systems. More recently, the National Institute of Standards and Technology (NIST) introduced a comprehensive classification formalising an industry-wide standard for evaluating both attacks and defences (Vassilev et al. 2024). This taxonomy provides a structured approach to understanding the entire adversarial machine learning landscape, covering attack vectors, defence mechanisms, and evaluation criteria across different domains and modalities. 2.3.2 Formalisation of Threat Models A formalised approach to threat models is essential for the rigorous evaluation of adversarial robustness. Carlini and Wagner (Carlini et al. 2019) provided a landmark contribution by outlining comprehensive best practices and establishing a framework for evaluating defences. They formalised the adversarial robustness problem as finding the minimal perturbationĪ“that causes misclassification: minimiseā„Ī“ā„ p (1) 12 such thatf(x+Ī“)Ģø=f(x)(2) x+Ī“ā[0,1] n (3) Wherefis the model,xis the input,Ī“is the perturbation,ā„Ī“ā„ p represents theL p norm measuring perturbation magnitude, and the final constraint ensures the perturbed input remains valid (e.g., pixel values within range). 2.3.3 Defence Evaluation Principles Their work established three core principles for rigorous evaluation: ā¢ Threat Model Specification: Clearly defining adversarial goals (e.g., misclas- sification), knowledge (white-box, black-box, grey-box), and capabilities (e.g., perturbation budget specified byL p norm). ā¢ Adaptive Adversary Attacks: Testing against optimised attacks specifically designed to overcome the proposed defence, rather than relying solely on stan- dard, pre-existing attacks. An adaptive adversary modifies attack strategies based on knowledge of the defence mechanism. ā¢ White-Box vs Black-Box Testing: Ensuring defences are robust under both scenarios. Defences that only work in black-box settings might rely on āsecurity through obscurityā (e.g., gradient masking) rather than true robustness. 2.3.4 Common Evaluation Pitfalls Carlini and Wagner identified critical pitfalls that can lead to overestimating a defenceās effectiveness: ā¢ Gradient Masking/Obfuscation: Some defences inadvertently (or deliber- ately) make it harder to find useful gradients, hindering gradient-based attacks. This creates an illusion of security that often fails against gradient-free methods, transfer attacks, or adaptive white-box attacks. ā¢ Weak Attack Configuration: Using suboptimal attack parameters (e.g., insuf- ficient iterations, inappropriate step sizes, or incorrect loss functions for the attack goal) can make a defence appear stronger than it is. ā¢ Inadequate Testing Scope: Failing to test against a diverse range of attack types, particularly adaptive attacks tailored to the defence mechanism. 3 Literature Review Building upon the foundational concepts introduced in the previous section, this section examines specific research directly relevant to adversarial examples against vision components in multimodal systems, detailing attack methodologies that form the basis for the experimental approach. 13 3.1 Types of Adversarial Examples in Computer Vision Adversarial examples in computer vision can be generated through various techniques, each with different computational requirements and effectiveness. 3.1.1 Fast Gradient Sign Method (FGSM) One of the earliest and most influential methods for generating adversarial examples is the Fast Gradient Sign Method (FGSM), introduced by Goodfellow et al. (Goodfellow et al. 2015). FGSM operates on the principle that neural networks, particularly in high-dimensional spaces, behave in a surprisingly linear fashion. This linearity allows for the efficient generation of adversarial perturbations by taking a single step in the direction that maximises the modelās loss. The perturbationĪ·is calculated as: Ī·=ĪµĀ·sign ā x J(Īø,x,y true ) wherexis the original input image,y true is the true label,Īørepresents the model parameters,Jis the loss function (e.g., cross-entropy),ā x Jis the gradient of the loss with respect to the input imagex, sign(Ā·) is the sign function, andĪµis a small scalar controlling the magnitude of the perturbation (often constrained by theL ā norm). The adversarial examplex adv is then created by adding this perturbation to the original image: x adv =x+Ī· Despite its simplicity and computational efficiency (requiring only one gradient com- putation), FGSM proved remarkably effective at fooling networks, demonstrating that adversarial vulnerability was a fundamental property linked to linearity in high dimensions (Goodfellow et al. 2015). Figure 2 illustrates how a small, visually imper- ceptible FGSM perturbation can cause a model to misclassify an image with high confidence. 3.1.2 Carlini & Wagner Attacks While FGSM provides a fast way to generate adversarial examples, Carlini & Wagner (Carlini and Wagner 2017) treat the generation process as an optimization problem, aiming to find the minimal perturbation required to cause misclassification according to a specific distance metric (e.g.,L 0 ,L 2 ,L ā ). The authorās family of attacks have come to be known as Carlini & Wagner (CW) attacks (Carlini and Wagner 2017) and are designed to be particularly effective against defensive techniques like defensive distillation, which FGSM might fail to overcome. The CW attacks, especially theL 2 version, formulate the problem as minimising the perturbation normā„Ī“ā„ 2 subject to the constraint that the perturbed inputx+Ī“is misclassified. This is often achieved by minimising a loss function that encourages misclassification while simultaneously penalising the perturbation size: min Ī“ ā„Ī“ā„ 2 2 +cĀ·L(x+Ī“) 14 (a) Original Image: āPandaā (57.7% confidence) (b) Adversarial Noise (Īµ= 0.007) (c) Adversarial Image: āGib- bonā (99.3% confidence) Fig. 2: Demonstration of adversarial perturbation using FGSM. An imperceptible perturbationĪ·=ĪµĀ·sign(ā x J(Īø,x,y)) is added to the original image (a), resulting in an adversarial image (c) that causes misclassification with high confidence. The noise (b) is scaled for visibility. Reproduced from Goodfellow et al. (2015) (Goodfellow et al. 2015). whereLis a loss function designed such thatL(x ā² )ā¤0 if and only if the model classifiesx ā² incorrectly (for targeted attacks, incorrectly as the target class), andc is a constant balancing the two terms. CW attacks are generally more powerful but computationally more expensive than FGSM, often serving as a stronger benchmark for evaluating defences (Carlini and Wagner 2017). 3.1.3 Projected Gradient Descent (PGD) Projected Gradient Descent (PGD) (Madry et al. 2019) is arguably the most widely used and powerful first-order adversarial attack, building upon the ideas of FGSM but employing an iterative approach. Instead of taking one large step, PGD takes multiple smaller steps in the direction of the gradient sign, projecting the result back onto the allowed perturbation region (e.g., anL ā ball of radiusĪµaround the original inputx) after each step. The update rule for each iterationtis: x t+1 = Ī B Īµ (x) x t +Ī±Ā·sign ā x t J(Īø,x t ,y true ) Here,x 0 is typically initialised randomly within the perturbation ballB Īµ (x),Ī±is the step size (usually smaller thanĪµ), and Ī B Īµ (x) denotes the projection operator that ensures the perturbed imagex t+1 remains within theĪµ-ball around the original image x. By using multiple iterations and random starts, PGD is much more effective at finding adversarial examples than FGSM, especially for models trained to be robust. Madry et al. (Madry et al. 2019) framed adversarial robustness within a min-max optimisation perspective, arguing that PGD acts as a universal first-order adversary capable of find- ing the worst-case perturbation within the specified threat model. Consequently, PGD 15 has become the standard benchmark attack for evaluating adversarial defences and is commonly used in adversarial training regimes to improve model robustness. 3.2 Multimodal Foundation Model Robustness 3.2.1 Expanded Attack Surface Multimodal Foundation Models inherently present a more complex security landscape than their unimodal counterparts. By processing both visual and textual inputs simul- taneously, these models effectively incorporate additional attack vectors for potential exploitation, analogous to adding more doors or windows to a house, resulting in additional points of entry. This expanded attack surface creates unique vulnerabili- ties at the intersection of modalities, where adversarial manipulations in one domain can propagate to and influence the other and potentially bypass unimodal specific defences. For example, in one paper (Liu et al. 2025), the authors demonstrated that query- relevant images could bypass text-based safety measures and produce unsafe output in multimodal models without even requiring adversarial perturbations. Using three image generation methods, they found an average increase in Attack Success Rate (ASR), exceeding 30% in LLaVA-1.5-7B, and that text-only alignment proves insuffi- cient for securing these systems. Furthermore, despite the growing prevalence of multimodal systems, research on foun- dation model robustness has primarily focused on text-based attacks, such as prompt injection and jailbreaking (Shin et al. 2020; Wallace et al. 2021; Wei et al. 2023; Zou et al. 2023). This emphasis on textual vulnerabilities has left visual attack vectors comparatively underexplored, creating a potential blind spot in security evaluations. The integration of vision capabilities potentially introduces adversarial examples, as discussed in relation to computer vision systems as an attack vector to foundation models. 3.2.2 Cross-Modal Adversarial Attacks The integration of vision and language creates vulnerabilities where adversarial manip- ulations in one modality can not only influence but potentially jailbreak the other. The authors of Image Hijacks: Adversarial Images Can Control Generative Models at Runtime (Luke Bailey et al. 2023) established that carefully crafted pixel perturba- tions could effectively hijack the behaviour of Vision-Language Models (VLMs). The authors conducted white-box attacks against LLaVA (specifically, the LLaMA-2-13B- Chat language model combined with a CLIP ViT-L/14 vision encoder), employing their āBehaviour Matchingā algorithmāa targeted approach based on Projected Gra- dient Descent (PGD)āto generate adversarial images. These images contained subtle perturbations, constrained withinL ā norms ranging from 1/255 to 64/255, yet were sufficient to induce specific, targeted behaviours in the model. Their experiments demonstrated high success rates (exceeding 80%) across four distinct attack types: forcing the VLM to output exact predefined adversarial messages (Specific String Attack), compelling the model to expose private information within API calls (Context 16 Leak Attack), bypassing RLHF-based safety guardrails to generate harmful content (Jailbreak Attack), and embedding factual distortions into images to modify the modelās perceived world knowledge (Disinformation Attack). This work highlighted the significant risk posed by adversarial images, showing they could reliably control the textual output of generative VLMs at runtime. Concurrently with this paper, a study (Schlarmann and Hein 2023) further investi- gated similar vulnerabilities through an evaluation of the adversarial robustness of OpenFlamingo, an open-source implementation of the Flamingo architecture. Their work demonstrated that even imperceptible adversarial perturbations (as small as Īµ= 1/255) could fully manipulate generated captions. They employed both targeted attacks, designed to force specific adversary-defined outputs like fake news, and untar- geted attacks aimed at degrading output quality. Achieving high success rates in image captioning and Visual Question Answering (VQA) tasks, their findings underscore the significant real-world security risks posed by such attacks, including misinformation propagation and user manipulation. The paper Jailbreak in Pieces: Compositional Adversarial Attacks on Multimodal Language models (Shayegani et al. 2023) introduced ācompositional adversarial attacksā that exploit cross-modal alignment vulnerabilities in Vision-Language Mod- els (VLMS). Their approach paired benign textual prompts with adversarial images, effectively bypassing text-only safety measures without requiring white-box access to the language model component. By targeting the embedding space through four distinct strategies (textual triggers, OCR textual triggers, visual triggers, and com- bined OCR+visual triggers), they achieved attack success rates of up to 91% on LLaVA. Notably, these attacks demonstrated that the same adversarial image could jailbreak multiple textual prompts, and once a model was compromised, subse- quent prompts continued generating harmful outputsāa phenomenon termed ācontext contamination. Expanding investigation to commercial systems, researchers evaluated the black-box adversarial robustness of Googleās Bard, demonstrating that adversarial image per- turbations could manipulate model responses with a 22% attack success rate on image descriptions (Dong et al. 2023). Their study revealed vulnerabilities in Bardās face detection and toxicity detection defences and demonstrated attack transferability across other commercial systems, including GPT-4V (45% success), Bing Chat (26%), and ERNIE Bot (86%). This work highlighted that even sophisticated closed-source models remain vulnerable to adversarial manipulation through their visual components despite employing robust text-based safety filters. Carlini et al. (Carlini et al. 2023) investigated whether alignment techniques like RLHF confer robustness specifically against text-based adversarial attacks. They demon- strated that current text-based attacks often fail against aligned language models, not due to inherent robustness from alignment but because the existing NLP attack methods themselves lack sufficient power. In contrast, the research highlighted that multimodal vision-language models could still be controlled via subtle image pertur- bations, even when aligned. These findings question whether alignment truly hardens 17 models against sophisticated textual manipulation or merely addresses superficial behaviours, suggesting stronger text-based attacks might eventually bypass align- ment in text-only models. Understanding precisely where this increased vulnerability from multimodality arises is an important area for future work, as it is likely that future models incorporating additional modalities, such as audio, will introduce new vulnerabilities and expand the attack surface (Carlini et al. 2023). A different study (Bhagwatkar et al. 2024) investigated architectural and prompt design factors affecting adversarial robustness in vision-language models. Applying PGD attacks against several open foundation models, including LLaVA but not Llama 3.2 vision, they made two key discoveries: first, contrary to intuition, neither increasing the input resolution of vision encoders nor scaling up the language model component enhanced robustness against adversarial examples. Second, they found that certain prompt design choices, particularly the inclusion of task-specific instructions and contextual information, could marginally improve robustness in some settings. In response to concerns about visual input safety, Meta released their own multimodal content moderation model, Llama 3 Guard Vision (Chi et al. 2024), designed to clas- sify and prevent harmful interactions with their Llama 3.2 vision model, similar to how their LLM-based model, Llama Guard (Inan et al. 2023) safeguards their conver- sational models. Llama Guad Vision was fine-tuned on 22,500 multimodal prompts and 40,034 response examples, outperforming GPT-4o in safety classification with higher F1 scores and lower false positive rates. The authors evaluated Llama guard visionās robustness in this paper (Chi et al. 2024) against adversarial attacks, show- ing that PGD image perturbations (Īµ= 8/255) successfully increased harmful content misclassification rates from 21% to 70%. The paper concludes that Meta recommends adversarial training to improve robustness before deploying and limiting model access to images in safety-critical applications - interesting for a pre-trained model designed to be a drop-in security enhancement for Llama 3.2 vision. 4 Methodology This paper investigates the security of vision in state-of-the-art open-source foundation models by evaluating the adversarial robustness of popular open-source models related to Metaās Llama family. Specifically, we employ adversarial image attacks against LLaVa and Llama 3.2 Vision. Additionally, the research seeks to explore how model architecture and training influence robustness by analysing their relationship with adversarial performance. 4.1 Problem setting The objectives of the methodology are defined in this section. 4.1.1 Formal Definition Letfrepresent a VLM, which takes an imagexāR HĆWĆC (where H, W, C are height, width, channels) and a text promptqas input, and generates a textual output 18 sequencey=f(x,q). The goal of an adversarial attack in this context was to find a small perturbationĪ“, such that when added to the original imagex, the resulting adversarial imagex adv =x+Ī“caused the model to produce an undesirable or incorrect outputy adv =f(x adv ,q), wherey adv significantly differed from the expected output y=f(x,q). The magnitude of the perturbationĪ“was constrained, typically using an L p norm, to ensure it remained imperceptible or near-imperceptible to humans. 4.1.2 Models Under Investigation This study focused on two prominent open-weight Foundation Models from the Llama family: ā¢ LLaVA (Large Language and Vision Assistant) v1.5-13B: An established VLM that connected a pre-trained CLIP ViT-L/14 visual encoder to a Vicuna- 13B language model (a Llama variant) via a simple projection layer. It underwent instruction tuning on multimodal datasets (Liu et al. 2023). ā¢ Llama 3.2 Vision-8B-2: Metaās first open multimodal foundation model, which integrated a ViT-H/14 image encoder with a Llama 3.1 8B language model using a more complex cross-attention adapter mechanism. It was pre-trained and aligned on significantly larger datasets (Grattafiori et al. 2024). These models represented different architectural approaches (simple projection vs. cross-attention adapter) and training scales, allowing for comparative analysis of robustness. 4.1.3 Notation ā¢ x: Original input image. ā¢ q: Input text prompt. ā¢ y: Generated text output from the VLM for the original input,y=f(x,q). ā¢ Īø: Parameters of the VLMf. ā¢ Ī“: Adversarial perturbation added to the image. ā¢ x adv : Adversarial image,x adv =x+Ī“. ā¢ y adv : Generated text output for the adversarial input,y adv =f(x adv ,q). ā¢ Īµ: Maximum perturbation budget, constrainingā„Ī“ā„ p ā¤Īµ. ā¢ J(Īø,x,q,y target ): Loss function used to guide the attack, measuring the discrep- ancy between the modelās prediction and a target (or lack thereof for untargeted attacks). 4.1.4 Threat Model Following the best practices outlined by Carlini and Wagner (Carlini et al. 2019), the threat model was defined as follows: 19 ā¢ Goal: Untargeted attack. The objective was not to force the VLM to output a specific incorrect string but rather to degrade the quality, relevance, or correctness of the generated texty adv compared to the expected outputyfor the clean image xand promptq. This was often achieved by maximising the modelās internal loss function with respect to the input image. ā¢ Knowledge: White-box access. The attacker was assumed to have complete knowledge of the VLMās architecture, parameters (Īø), and gradients (ā x J). This allowed for the use of powerful gradient-based attack methods like PGD. ā¢ Capabilities: The attacker could only modify the input imagexby adding a perturbationĪ“. The text promptqremained unchanged. The perturbation was constrained by theL ā norm, such thatā„Ī“ā„ ā ā¤Īµ. This ensured that the changes to individual pixel values were bounded, maintaining the visual similarity between xandx adv . The value ofĪµdefined the strength of the attack. 4.1.5 Assumptions ā¢ The attack vector was solely the visual input (x). The textual prompt (q) was assumed to be benign and fixed for each image-question pair during the attack generation and evaluation. ā¢ The Visual Question Answering (VQA) v2 dataset (Goyal et al. 2017) served as the benchmark task. Performance degradation on this task was considered indicative of reduced model robustness, as VQA required integrated visual and language understanding. ā¢ The effectiveness of the attack was measured by the degradation in the modelās ability to correctly answer the questions associated with the images after perturbation. 4.2 Method This study employed Projected Gradient Descent (PGD) as the primary method for generating adversarial examples targeting the visual input of the selected VLMs. 4.2.1 Attack Algorithm: Projected Gradient Descent (PGD) PGD was chosen due to its established effectiveness as a powerful, first-order white- box adversarial attack (Madry et al. 2019). As discussed in the Background section, PGD built upon the Fast Gradient Sign Method (FGSM) (Goodfellow et al. 2015) but utilised an iterative approach with smaller steps and projection, which made it significantly more effective at finding adversarial examples, particularly against mod- els potentially incorporating defensive measures. Madry et al. (Madry et al. 2019) demonstrated that PGD served as a universal first-order adversary, capable of find- ing approximate worst-case perturbations within a given threat model (L ā ball in 20 this case). Its widespread adoption as a standard benchmark for evaluating adversar- ial robustness made it suitable for rigorously assessing and comparing the resilience of LLaVA and Llama 3.2 Vision. 4.2.2 PGD Implementation for Generative VLMs The standard PGD algorithm was adapted here for the context of generative VLMs performing Visual Question Answering (VQA). 4.2.3 Objective Consistent with the untargeted attack goal defined in the Threat Model, the objective was to find a perturbationĪ“that, when added to the original imagex, maximised the modelās internal loss function for the given image-prompt pair (x,q). The aim was not to force a specific incorrect output string but to generate an adversarial image x adv =x+Ī“that caused the VLMfto produce a degraded, irrelevant, or incorrect textual outputy adv =f(x adv ,q) compared to the expected outputy=f(x,q). 4.2.4 Loss Function The attack leveraged the modelās internal loss function, typically related to the neg- ative log-likelihood or cross-entropy of generating the output sequence given the inputs. For an untargeted attack on the image modality, the PGD algorithm aimed to maximise this lossJ(Īø,x adv ,q) with respect to the input imagex adv . The gradient ā x adv Jindicated the direction in the image space that most increased the modelās loss (i.e., made the current prediction less likely or increased model uncertainty/error signal). 4.2.5 Optimisation Process The PGD attack iteratively refined the perturbationĪ“. Starting with an initial per- turbed imagex 0 (oftenxorxplus small random noise within theĪµ-ball), each iteration updated the image as follows: x t+1 = Ī B Īµ (x) x t +Ī±Ā·sign ā x t J(Īø,x t ,q) Where: ā¢ x t was the adversarial image at iterationt. ā¢ Ī±was the step size, determining how large a step was taken in the gradientās sign direction. ā¢ ā x t J(Īø,x t ,q) was the gradient of the loss function with respect to the image inputx t , computed using the model parametersĪøand the fixed promptq. ā¢ sign(Ā·) extracted the direction of steepest ascent for the loss. ā¢ Ī B Īµ (x) was the projection operator. After each step, it ensured the resulting image x t+1 remained within theL ā ball of radiusĪµcentred around the original image 21 x. This meant for every pixel channeli,|x t+1 i āx i |ā¤Īµ. It also clipped pixel values to the valid range (e.g., [0, 1] for normalised images). This process was repeated for a predefined number of iterations. 4.2.6 PGD Hyperparameters The effectiveness and characteristics of the PGD attack were controlled by several hyperparameters: ā¢ Perturbation Budget (Īµ): Defined the maximum allowedL ā distance between the original imagexand the adversarial imagex adv . A largerĪµallowed for stronger attacks but potentially more perceptible perturbations. ā¢ Step Size (Ī±): Controlled the magnitude of the update at each iteration. It was typically set to a value smaller thanĪµ(e.g.,Ī±=Īµ/kfor somek). ā¢ Number of Iterations: Determined how many gradient ascent steps were performed. More iterations generally led to stronger attacks but increased computational cost. The specific values chosen for these hyperparameters in the experiments were detailed in the following Experimental Setup section. 4.3 Experimental setup 4.3.1 Models The experiments utilised the two open-weight VLMs defined in the Problem Set- ting section, loaded via the Hugging Face Transformers library with the following identifiers: ā¢ LLaVA-1.5-13B, Identifier: llava-hf/llava-1.5-13b-hf. ā¢ Llama 3.2 Vision-8B-2, Identifier: meta-llama/Llama-3.2-Vision-8B-2. Both models were loaded usingfloat16precision for computational efficiency, as specified in the projectās configuration file (config.json). The choice of these models allowed for a direct comparison between an established VLM architecture (LLaVA) and Metaās newer, adapter-based approach (Llama 3.2 Vision). 4.3.2 Dataset The study was conducted using the Visual Question Answering (VQA) v2 dataset (Goyal et al. 2017), specifically the validation split. This large-scale dataset con- tained open-ended questions about images from the Microsoft COCO collection (Lin et al. 2015), which required an understanding of vision, language, and common- sense knowledge to answer. The complete dataset included over 200,000 images from Microsoft COCO, more than 1 million questions, and over 10 million answers across diverse question types requiring reasoning, recognition, and understanding of visual content. 22 Due to the computational demands of generating adversarial examples for the entire dataset, a subsetting approach was employed, as implemented in createvqav2subset.py. This script randomly sampled a specified number of image- question-answer triplets from the full validation set, which created reproducible subsets across runs. The size of this subset was controlled by thesubsetsizeparameter in the config.jsonfile. For the experiments reported in this study, a subset size of 500 sam- ples was used, providing a balance between computational feasibility and statistical significance with margins of error aroundĀ±3-4%. Each sample consisted of an image, a question, and a list of ground-truth answers provided by human annotators. 4.3.3 Evaluation Metrics The primary metric used to evaluate model performance and robustness was the stan- dard VQA accuracy, consistent with the VQA v2 challenge evaluation protocol (Goyal et al. 2017). This metric measured the agreement between the modelās generated answer and the set of human-provided ground-truth answers for each question. Calculating the VQA accuracy involved normalising both the predicted answer and the ground-truth answers to account for variations in phrasing, case sensitivity, and punctuation. The evaluation process was implemented in thecomputevqaaccuracy function withinvqautils.py, which was adapted from the original VQA evaluation code (Goyal et al. 2017). An answer was considered correct if the normalised predicted answer matched at least one of the normalised ground-truth answers exactly or if either the prediction was a substring of a ground truth or vice-versa, allowing for minor variations. An empty prediction was always marked as incorrect. This VQA accuracy was calculated for each model on both the original (clean) images and the corresponding adversarial images generated using PGD. The differ- ence between the average clean accuracy and the average adversarial accuracy, termed theaccuracy drop, served as the key indicator of the modelās susceptibility to the adversarial attack under a specific perturbation budget (Īµ). A larger accuracy drop indicated lower adversarial robustness. The evaluation process was orchestrated by the run eval.pyscript, which computed and recorded these metrics for each parameter set defined inconfig.json. 4.3.4 Implementation details The experiments were implemented in Python 3, leveraging several key libraries from the scientific computing and machine learning ecosystem. The core deep learning framework used was PyTorch, along with the Hugging Facetransformerslibrary for model loading (modelfactory.py) and processing, andacceleratefor efficient hardware utilisation. Image processing relied on the Pillow (PIL) library. The PGD attack logic was implemented inpgd utils.py, following the iterative pro- cess described in the Method section. The attack utilised the modelās internal loss function during backpropagation to compute gradients with respect to the input image 23 pixels. Mixed-precision training (float16, specified inconfig.json) and gradient checkpointing (modelfactory.py) were employed to manage the significant memory requirements of the large VLMs. The main evaluation loop was orchestrated byruneval.py. This script handled loading the VQA subset, iterating through samples, generating both clean and adver- sarial predictions using themodelinfer.pymodule, computing VQA accuracy via vqautils.py, and saving results. Model-specific prompting and processing logic were handled withinmodelinfer.pyandruneval.pyto ensure correct input formatting for both LLaVA and Llama 3.2 Vision. The PGD attack hyperparameters were defined inconfig.jsonunderpgd parameterssets. Experiments were conducted across a range of perturbation budgets (Īµ), specifically: Īµā2/255,4/255,8/255,16/255,128/255,255/255 The step size (Ī±) and number of iterations were adjusted proportionally toĪµ, ranging fromĪ±= 0.00196, iterations=5 forĪµ= 2/255 up toĪ±= 0.06274, iterations=30 forĪµ= 255/255, ensuring a consistent attack strength relative to the budget. All experiments were executed on an NVIDIA A100 GPU possessing 80GB of VRAM, as required by the memory demands of the 13B and 8B parameter models under evaluation, particularly during the gradient computations needed for the PGD attack. 5 Evaluation This section focuses on quantifying the impact of Projected Gradient Descent (PGD) adversarial attacks on the performance of LLaVA-1.5-13B and Llama 3.2 Vision-8 B- 2 on the Visual Question Answering (VQA) task. The section begins by presenting the baseline performance of each model on the clean VQA v2 subset, followed by their performance under adversarial conditions with varying perturbation budgets (Īµ). A comparative analysis then examines the relative robustness of the two models based on the observed accuracy degradation. Finally, a critical analysis discusses the implications of these findings in the context of the research questions, relates them to existing literature, and acknowledges the limitations of the study. 5.1 Results The adversarial robustness of LLaVA-1.5-13B and Llama 3.2 Vision-8B-2 was evalu- ated on a subset of 500 samples from the VQA v2 validation dataset. Performance was measured using the standard VQA accuracy metric on both original (clean) images and adversarial images generated via Projected Gradient Descent (PGD) with vary- ingL ā perturbation budgets (Īµ). The accuracy drop (Clean Accuracy - Adversarial Accuracy) quantifies the impact of the attack. On the clean dataset subset, LLaVA-1.5-13B achieved a baseline VQA accuracy of 87.4%. Llama 3.2 Vision-8B-2 achieved a baseline VQA accuracy of 42.8% on the 24 Table 1: VQA Accuracy (%) under Subtle Adversarial Perturbations (Īµā¤16/255) CleanAdversarial Accuracy (Drop) ModelAcc. (%)Īµ= 2/255Īµ= 4/255Īµ= 8/255Īµ= 16/255 LLaVA-1.5-13B87.480.4 (ā7.0)80.6 (ā6.8)79.0 (ā8.4)76.8 (ā10.6) Llama 3.2 Vision-8B-242.836.2 (ā6.6)32.4 (ā10.4)33.0 (ā9.8)36.2 (ā6.6) same subset. It was noted that the observed baseline accuracy for Llama 3.2 Vision (42.8%) was substantially lower than the 75.2% reported by Meta on the full VQA v2 dataset (Grattafiori et al. 2024). In contrast, the observed accuracy for LLaVA (87.4%) was closer to benchmarks reported for similar models (around 84% (Liu et al. 2023)). This significant discrepancy for Llama 3.2 Vision likely stems primarily from implementation-specific factors, such as deviations from the precise, prompt format- ting, image preprocessing, or generation parameters required by the model for optimal VQA performance rather than solely subset sampling effects (which typically yield margins of error aroundĀ±3-4% for this size). Additionally, LLaVAās explicit instruc- tion tuning on QA datasets and its larger 13B language model backbone (compared to Llama 3.2 Visionās 8B) might have contributed to its higher baseline performance in this specific experimental setup. Nevertheless, as the primary goal of this study was to assess the relative change in accuracy under adversarial attack, the observed base- line served as the reference point for evaluating robustness, mitigating the impact of this discrepancy on the comparative findings regarding accuracy drop. Furthermore, a minor variation was observed in Llama 3.2 Visionās baseline accuracy during the run with the largest perturbation budgetĪµ= 255/255. While the baseline was consistently 42.8% across other runs, it was recorded as 41.6% for this specific run. This slight difference arose because theĪµ= 255/255 evaluation batch was interrupted and had to be re-run. Since each run, including the restarted one, operated on a randomly drawn subset of 500 samples, minor variations in baseline accuracy due to sampling variability are statistically expected. The small difference observed (1.2 percentage points) falls well within the anticipated margin of error for this subset size, demonstrating the statistical consistency of the baseline measurements across runs despite the interruption. Consequently, the accuracy drop reported for theĪµ= 255/255 case is calculated relative to its specific 41.6% baseline, while 42.8% serves as the reference for all other perturbation levels. The models were subjected to untargeted PGD attacks withĪµvalues ranging from 2/255 (subtle) to 255/255 (maximum). The resulting VQA accuracies and accuracy drops are presented in two tables, separating subtle perturbations (Īµā¤16/255) from larger ones (Īµā„128/255). Table 1 shows the performance under subtle perturbations, which are generally near-imperceptible. Table 2 shows the performance under larger, more perceptible perturbations. 25 Table 2: VQA Accuracy (%) under Large Adversarial Perturbations (Īµā„128/255) CleanAdversarial Accuracy (Drop) ModelAcc. (%)Īµ= 128/255Īµ= 255/255 LLaVA-1.5-13B87.467.4 (ā20.0)51.4 (ā36.0) Llama 3.2 Vision-8B-242.837.4 (ā5.4)31.4 ā” (ā10.2 ā” ) ā” Clean accuracy was 41.6% in theĪµ= 255/255 run. Fig. 3: VQA Accuracy vs. Adversarial Perturbation Strength (Īµ). Compares LLaVA- 1.5-13B and Llama 3.2 Vision-8B-2 performance under PGD attack with varyingL ā budgets. The tables show that for LLaVA, the accuracy drop increases steadily withĪµ, reaching 36.0% points atĪµ= 255/255. For Llama 3.2 Vision, the drop is significant at lowĪµ (peaking at 10.4 points atĪµ= 4/255) but appears less sensitive to further increases in perturbation magnitude, with a drop of only 10.2 points atĪµ= 255/255 (relative to its 41.6% baseline in that run). Figure 3 visually represents the VQA accuracy for both models as the perturbation budgetĪµincreases. Qualitative analysis of individual samples revealed instances where adversarial per- turbations caused models to generate incorrect, irrelevant, or nonsensical answers compared to their outputs on the corresponding clean images, particularly at higher perturbation levels. 26 5.2 Comparative Analysis On the clean VQA v2 subset, LLaVA-1.5-13B demonstrated significantly higher base- line performance (87.4% accuracy) compared to Llama 3.2 Vision-8B-2 (42.8% accu- racy). This substantial difference in clean accuracy highlighted the varying capabilities of the models on the VQA task prior to any adversarial manipulation. When subjected to PGD attacks, the models exhibited distinct responses. LLaVAās accuracy degraded progressively as the perturbation budget (Īµ) increased. The accu- racy drop started at 7.0 percentage points forĪµ= 2/255 and reached a substantial 36.0 percentage points at the maximum perturbation ofĪµ= 255/255. In contrast, Llama 3.2 Vision showed a notable initial drop in accuracy even at low perturbation levels (e.g., 10.4 points atĪµ= 4/255), but its performance degradation appeared less sensitive to further increases inĪµ. The maximum accuracy drop observed for Llama 3.2 Vision was 10.4 points (atĪµ= 4/255), and even atĪµ= 255/255, the drop was only 10.2 points relative to its baseline in that specific run. Based purely on the magnitude of the accuracy drop under attack, Llama 3.2 Vision appeared quantitatively more robust, particularly at higher perturbation levels (Īµā„ 128/255). Its accuracy, while lower overall, degraded less severely in percentage points compared to LLaVA when strong adversarial noise was introduced. However, LLaVA maintained a higher absolute accuracy score even under the strongest attack (Īµ= 255/255, LLaVA accuracy 51.4% vs Llama 3.2 Vision accuracy 31.4%). These differing robustness profiles might stem from several factors. LLaVAās simpler projection layer architecture could potentially be more susceptible to perturbations that directly affect the mapped visual features. Llama 3.2 Visionās more complex cross-attention adapter mechanism, combined with its significantly larger pre-training dataset (billions vs millions of image-text pairs) and potentially more sophisticated alignment process, might contribute to its relatively stable performance under increas- ing perturbation strengths, even though its baseline VQA performance was lower on this specific task and dataset subset. The larger vision encoder (ViT-H vs ViT-L) in Llama 3.2 Vision could also play a role, although previous work suggested encoder size alone might not guarantee robustness (Bhagwatkar et al. 2024). 5.3 Critical Analysis The results provided insights into the adversarial robustness of Metaās Llama 3.2 Vision and its comparison with LLaVA. Addressing the first research question, Llama 3.2 Vision demonstrated vulnerability to PGD-based adversarial examples targeting its visual input. Its VQA accuracy dropped by up to 10.4 percentage points even with relatively subtle perturbations (Īµ= 4/255). This indicated that, despite its advanced architecture and extensive training, the model was not inherently immune to visually grounded adversarial attacks degrading its performance. 27 5.3.1 Nuanced picture of robustness Regarding the second research question, comparing Llama 3.2 Vision to LLaVA revealed a nuanced picture of robustness. While LLaVA achieved significantly higher baseline VQA accuracy, it suffered a much larger degradation under attack, with accu- racy dropping by up to 36.0% points. Llama 3.2 Vision, despite its lower baseline, exhibited a smaller maximum accuracy drop (10.4 points) and greater stability against increasing perturbation strengths. Therefore, based on the relative decrease in perfor- mance under attack, Llama 3.2 Vision could be considered more robust than LLaVA in this specific experimental setup. The architectural differences (cross-attention adapter vs. simple projection) and the vastly larger scale of pre-training data used for Llama 3.2 Vision likely contributed to this greater resilience against stronger perturbations, potentially fostering more stable internal representations, even if not translating to superior baseline performance on this VQA subset. These findings carry significant implications for the security of deployed VLMs. The observed vulnerability of both models, particularly Llama 3.2 Vision as a state-of-the- art open-weight model, underscores the risks associated with visual input channels. Even perturbations designed simply to maximise internal loss, without a specific mali- cious target, were sufficient to degrade performance noticeably. This suggests that real-world applications relying on these models could be susceptible to performance degradation or manipulation through adversarial images. The results also highlighted a potential trade-off: LLaVAās higher baseline performance came at the cost of lower robustness, while Llama 3.2 Visionās greater robustness was accompanied by lower baseline accuracy on this task. This suggests that optimising for standard benchmark performance might not automatically confer adversarial resilience. 5.3.2 Limitations Several limitations should be acknowledged. Firstly, the evaluation was conducted on a subset of 500 samples from the VQA v2 dataset due to computational con- straints; results on the full dataset or other datasets might differ. Secondly, the study focused exclusively on the PGD attack under anL ā threat model. Other attack algorithms (e.g., CW attacks, AutoAttack) or norm constraints (L 2 ,L 0 ) could reveal different vulnerabilities. The attack was also untargeted; targeted attacks aiming for specific incorrect outputs might pose different challenges. Thirdly, robustness was evaluated solely on the VQA task. Performance degradation might vary on other mul- timodal tasks like image captioning or complex reasoning. Finally, while efforts were made to use appropriate hyperparameters, the computational cost limited exhaustive exploration of the PGD parameter space (iterations, step size). 5.3.3 Adversarial attacks The findings align with broader research demonstrating the susceptibility of VLMs to adversarial attacks via their vision component (Luke Bailey et al. 2023; Schlar- mann and Hein 2023; Shayegani et al. 2023; Dong et al. 2023). The vulnerability observed in Llama 3.2 Vision echoes the results from Metaās own evaluation of Llama Guard 3 Vision (Chi et al. 2024), where PGD attacks significantly impacted 28 the safety classifierās performance, reinforcing the notion that visual perturbations pose a genuine threat even to models designed with safety considerations. While Bhagwatkar et al. (Bhagwatkar et al. 2024) found a limited correlation between model/encoder scale and robustness in their study (which included LLaVA but not Llama 3.2 Vision), our results suggest that architectural choices (adapter vs. pro- jection) and training scale/methodology might indeed influence relative robustness, warranting further investigation. The lower baseline but higher relative robustness of Llama 3.2 Vision compared to LLaVA presents an interesting data point in this ongoing discussion. 6 Conclusions This paper has investigated the security of the vision component in contemporary open-source foundation models, specifically by evaluating the adversarial robustness of LLaVA-1.5-13B and Metaās Llama 3.2 Vision-8B-2. A further aim was to explore potential links between model architecture, training, and observed robustness. These aims were pursued through several objectives, which have been successfully met. Both models were implemented, and their adversarial robustness was tested using untargeted PGD against the visual input modality. The experiments were conducted within an industry practice, formalised threat model, and empirically evaluated on the Visual Question Answering (VQA) v2 dataset subset. The results of these adversarial attacks were quantified using the standard VQA accu- racy metric, allowing for a direct comparison between the modelsā performance on clean versus perturbed images across various attack strengths (Īµ). The evaluation compared the accuracy degradation (accuracy drop) of LLaVA and Llama 3.2 Vision, revealing distinct robustness profiles. Llama 3.2 Vision, despite a lower baseline accu- racy in this setup, exhibited a smaller drop in performance under attack compared to LLaVA, particularly at higher perturbation levels. These quantitative findings were then evaluated in the context of the modelsā differing architectures (simple projection layer in LLaVA vs. cross-attention adapter in Llama 3.2 Vision) and their respective training scales and methodologies. Through the successful completion of these objectives, the primary aim of evaluating and comparing the adversarial robustness of these two key open-weight VLMs was achieved, providing insights into the security of their visual components. 6.1 Significant findings The empirical evaluation yielded several significant findings regarding the adversarial robustness of the vision component in the selected foundation models: ā¢ Universal Vulnerability:Both models demonstrated clear vulnerability to untargeted Projected Gradient Descent (PGD) attacks applied to the visual input. Measurable degradation in VQA accuracy occurred even with subtle, near-imperceptible perturbations (Īµā¤16/255). 29 ā¢ Distinct Robustness Profiles:The models exhibited markedly different responses to increasing perturbation strengths. LLaVAās accuracy degraded pro- gressively and substantially with largerĪµ, suffering a maximum drop of 36.0 percentage points. Conversely, Llama 3.2 Vision showed a significant initial drop but greater stability against stronger perturbations, with its accuracy drop plateauing around 10 percentage points. However, LLaVA maintained higher absolute accuracy scores across all perturbation levels due to its superior baseline performance - but this was likely due to Llama 3.2 implementation. ā¢ Architectural and Training Implications:The observed difference in rela- tive robustness tentatively suggests that architectural choices (Llama 3.2 Visionās cross-attention adapter vs. LLaVAās simpler projection) and the scale of pre- training data may influence a modelās resilience to visual adversarial examples, potentially favouring more complex integration mechanisms and larger datasets for stability, even if not for peak task performance in all settings. Collectively, these findings confirm that the vision modality represents a viable attack vector for degrading the performance of contemporary open-weight VLMs, including Metaās Llama 3.2 Vision. Furthermore, they highlight that adversarial robustness does not necessarily correlate directly with standard benchmark performance and may be influenced by underlying architectural and training factors. Overall, we believe that this paper is one of the first to provide systematic comparisons of visual adversarial robustness in popular open models, and outlines that robustness does not always align with standard accuracy metrics. 6.2 Future Work Building upon the findings and acknowledging the limitations of this study, several avenues for future research emerge. This includes the replicating the experiments on either the full VQA v2 dataset or a larger subset, and other multimodal benchmarks would provide a more comprehensive assessment of robustness across different tasks and data distributions. Along with this, there is the opportunity for diverse adversar- ial attacks, such as investigating vulnerability to a wider range of attack algorithms beyond untargeted PGD, like Carlini & Wagner (CW) attacks or targeted attacks aiming to induce specific incorrect outputs (e.g., Image Hijacks), would offer a more complete security assessment. In relation to investigating Performance Discrepancies, further work is needed to pin- point the exact reasons for the observed lower baseline performance of Llama 3.2 Vision on the VQA subset compared to reported benchmarks, potentially involving different prompt configurations or preprocessing steps. With architecture and training Analysis, a deeper analysis into how specific components (e.g., the design of multi- modal adapters) and training phases (pre-training data scale, alignment techniques like RLHF) quantitatively impact adversarial robustness would be highly valuable. With defence mechanisms, research into developing and evaluating effective defence strate- gies specifically tailored for VLMs like Llama 3.2 Vision is essential for mitigating the identified risks. 30 Finally, there is a need to further investigate native multimodality vs. adapters. This would involve investigating newer architectures, such as Metaās Llama 4, which report- edly employs early fusion to jointly pre-train text and image data, and could reveal differences in robustness compared to the adapter-based approach of Llama 3.2 Vision. Understanding how native multimodal training impacts vulnerability is highly relevant for the field. References (2024) GPT-4V(ision) system card. URL https://openai.com/index/gpt-4v-system-c ard/ Alayrac JB, Donahue J, Luc P, et al (2022) Flamingo: A Visual Language Model for Few-Shot Learning. https://doi.org/10.48550/arXiv.2204.14198, URL http: //arxiv.org/abs/2204.14198, arXiv:2204.14198 Bahdanau D, Cho K, Bengio Y (2016) Neural Machine Translation by Jointly Learning to Align and Translate. https://doi.org/10.48550/arXiv.1409.0473, URL http: //arxiv.org/abs/1409.0473, arXiv:1409.0473 Bhagwatkar R, Nayak S, Bashivan P, et al (2024) Improving Adversarial Robustness in Vision-Language Models with Architecture and Prompt Design. In: Al-Onaizan Y, Bansal M, Chen YN (eds) Findings of the Association for Computational Lin- guistics: EMNLP 2024. Association for Computational Linguistics, Miami, Florida, USA, p 17003ā17020, https://doi.org/10.18653/v1/2024.findings-emnlp.990, URL https://aclanthology.org/2024.findings-emnlp.990/ Bommasani R, Hudson DA, Adeli E, et al (2021) On the Opportunities and Risks of Foundation Models. ArXiv URL https://w.semanticscholar.org/paper/On-the -Opportunities-and-Risks-of-Foundation-Models-Bommasani-Hudson/76e9e2ec3d e437ffb30d8b7b629f7fe3e61de5c2 Brown T, Mann B, Ryder N, et al (2020) Language Models are Few-Shot Learners. In: Advances in Neural Information Processing Systems, vol 33. Curran Associates, Inc., p 1877ā1901, URL https://proceedings.neurips.c/paper f iles/paper/2020/ hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html Carlini N, Wagner D (2017) Towards Evaluating the Robustness of Neural Networks. https://doi.org/10.48550/arXiv.1608.04644, URL http://arxiv.org/abs/1608.046 44, arXiv:1608.04644 Carlini N, Athalye A, Papernot N, et al (2019) On Evaluating Adversarial Robustness. https://doi.org/10.48550/arXiv.1902.06705, URL http://arxiv.org/abs/1902.067 05, arXiv:1902.06705 Carlini N, Nasr M, Choquette-Choo CA, et al (2023) Are aligned neural networks adversarially aligned? In: Advances in Neural Information Processing Systems, p 31 61478ā61500, URL https://proceedings.neurips.c/paperf iles/paper/2023/hash/c 1f0b856a35986348ab3414177266f75-Abstract-Conference.html Chen SF, Goodman JT (1996) An Empirical Study of Smoothing Techniques for Language Modeling. https://doi.org/10.48550/arXiv.cmp-lg/9606011, URL http: //arxiv.org/abs/cmp-lg/9606011, arXiv:cmp-lg/9606011 Chen YC, Li L, Yu L, et al (2020) UNITER: UNiversal Image-TExt Representation Learning. In: Vedaldi A, Bischof H, Brox T, et al (eds) Computer Vision ā ECCV 2020. Springer International Publishing, Cham, p 104ā120, https://doi.org/10.100 7/978-3-030-58577-87 Chi J, Karn U, Zhan H, et al (2024) Llama Guard 3 Vision: Safeguarding Human-AI Image Understanding Conversations. https://doi.org/10.48550/arXiv.2411.10414, URL http://arxiv.org/abs/2411.10414, arXiv:2411.10414 Chiang WL, Zheng L, Sheng Y, et al (2024) Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference. https://doi.org/10.48550/arXiv.2403.0413 2, URL http://arxiv.org/abs/2403.04132, arXiv:2403.04132 DeepSeek-AI, Guo D, Yang D, et al (2025) DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. URL https://w.semanticscho lar.org/paper/DeepSeek-R1%3A-Incentivizing-Reasoning-Capability-in-DeepSee k-AI-Guo/34471a2fa18ea22efad5287cf4aeb18542c98a9b Devlin J, Chang MW, Lee K, et al (2019) BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In: Proceedings of the 2019 Conference of the North. Association for Computational Linguistics, Minneapolis, Minnesota, p 4171ā4186, https://doi.org/10.18653/v1/N19-1423, URL http://aclweb.org/ant hology/N19-1423 Dong Y, Chen H, Chen J, et al (2023) How Robust is Googleās Bard to Adversarial Image Attacks? https://doi.org/10.48550/arXiv.2309.11751, URL http://arxiv.or g/abs/2309.11751, arXiv:2309.11751 Dosovitskiy A, Beyer L, Kolesnikov A, et al (2020) An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. ArXiv URL https://w.semanticsc holar.org/paper/An-Image-is-Worth-16x16-Words%3A-Transformers-f or-at-Dos ovitskiy-Beyer/268d347e8a55b5eb82fb5e7d2f800e33c75ab18a Frome A, Corrado G, Shlens J, et al (2013) DeViSE: A Deep Visual-Semantic Embed- ding Model. In: Neural Information Processing Systems, URL https://w.semant icscholar.org/paper/DeViSE%3A-A-Deep-Visual-Semantic-Embedding-Model-Fro me-Corrado/4a4069693bee00d1b0759ca3df35e59284e9845?sort=total-citations Goodfellow IJ, Shlens J, Szegedy C (2015) Explaining and Harnessing Adversarial Examples. https://doi.org/10.48550/arXiv.1412.6572, URL http://arxiv.org/abs/ 32 1412.6572, arXiv:1412.6572 Goyal Y, Khot T, Summers-Stay D, et al (2017) Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering. https: //doi.org/10.48550/arXiv.1612.00837, URL http://arxiv.org/abs/1612.00837, arXiv:1612.00837 Grattafiori A, Dubey A, Jauhri A, et al (2024) The Llama 3 Herd of Models. https: //doi.org/10.48550/arXiv.2407.21783, URL http://arxiv.org/abs/2407.21783, arXiv:2407.21783 Hochreiter S, Schmidhuber J (1997) Long Short-Term Memory. Neural Computation 9(8):1735ā1780. https://doi.org/10.1162/neco.1997.9.8.1735, URL https://direct.m it.edu/neco/article/9/8/1735-1780/6109 Inan H, Upasani K, Chi J, et al (2023) Llama Guard: LLM-based Input-Output Safe- guard for Human-AI Conversations. https://doi.org/10.48550/arXiv.2312.06674, URL http://arxiv.org/abs/2312.06674, arXiv:2312.06674 Jia C, Yang Y, Xia Y, et al (2021) Scaling Up Visual and Vision-Language Repre- sentation Learning With Noisy Text Supervision. In: International Conference on Machine Learning, URL https://w.semanticscholar.org/paper/Scaling-Up-Vis ual-and-Vision-Language-Learning-With-Jia-Yang/141a5033d9994242b18b3b217 e79582f1e9306 Kalchbrenner N, Blunsom P (2013) Recurrent Continuous Translation Models. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Seattle, Washington, USA, p 1700ā1709, https://doi.org/10.18653/v1/D13-1176, URL https://aclanthology .org/D13-1176 Kaplan J, McCandlish S, Henighan T, et al (2020) Scaling Laws for Neural Language Models. ArXiv URL https://w.semanticscholar.org/paper/Scaling-Laws-f or-N eural-Language-Models-Kaplan-McCandlish/e6c561d02500b2596a230b341a8eb8b9 21ca5bf2 Kapoor S, Bommasani R, Klyman K, et al (2024) On the Societal Impact of Open Foundation Models. https://doi.org/10.48550/arXiv.2403.07918, URL http://arxi v.org/abs/2403.07918, arXiv:2403.07918 Karpathy A, Fei-Fei L (2015) Deep visual-semantic alignments for generating image descriptions. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) p 3128ā3137. https://doi.org/10.1109/CVPR.2015.7298932, URL http: //ieeexplore.ieee.org/document/7298932/ Koehn P, Och FJ, Marcu D (2003) Statistical phrase-based translation. In: Proceed- ings of the 2003 Conference of the North American Chapter of the Association 33 for Computational Linguistics on Human Language Technology - NAACL ā03, vol 1. Association for Computational Linguistics, Edmonton, Canada, p 48ā54, https://doi.org/10.3115/1073445.1073462, URL http://portal.acm.org/citation.cf m?doid=1073445.1073462 Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet Classification with Deep Convolutional Neural Networks. In: Advances in Neural Information Processing Sys- tems, vol 25. Curran Associates, Inc., URL https://papers.nips.c/paperfiles/pap er/2012/hash/c399862d3b9d6b76c8436e924a68c45b-Abstract.html Li J, Li D, Xiong C, et al (2022) BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation. https://doi.org/10.4 8550/arXiv.2201.12086, URL http://arxiv.org/abs/2201.12086, arXiv:2201.12086 Li J, Li D, Savarese S, et al (2023) BLIP-2: Bootstrapping Language-Image Pre- training with Frozen Image Encoders and Large Language Models. https://do i.org/10.48550/arXiv.2301.12597, URL http://arxiv.org/abs/2301.12597, arXiv:2301.12597 Li LH, Yatskar M, Yin D, et al (2019) VisualBERT: A Simple and Performant Baseline for Vision and Language. ArXiv URL https://w.semanticscholar.org/paper/Vi sualBERT%3A-A-Simple-and-Performant-Baseline-for-Li-Yatskar/5aec474c31a2f 4b74703c6f786c0a8f85c450da Lin TY, Maire M, Belongie S, et al (2015) Microsoft COCO: Common Objects in Context. https://doi.org/10.48550/arXiv.1405.0312, URL http://arxiv.org/abs/14 05.0312, arXiv:1405.0312 Liu H, Li C, Wu Q, et al (2023) Visual Instruction Tuning (LLaVA). Advances in Neural Information Processing Systems 36:34892ā34916. URL https://papers.nips. c/paper files/paper/2023/hash/6dcf277ea32ce3288914faf369fe6de0-Abstract-Con ference.html Liu X, Zhu Y, Gu J, et al (2025) M-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models. In: Leonardis A, Ricci E, Roth S, et al (eds) Computer Vision ā ECCV 2024. Springer Nature Switzerland, Cham, p 386ā403, https://doi.org/10.1007/978-3-031-72992-822 Lu J, Batra D, Parikh D, et al (2019) ViLBERT: Pretraining Task-Agnostic Visi- olinguistic Representations for Vision-and-Language Tasks. In: Neural Information Processing Systems, URL https://w.semanticscholar.org/paper/ViLBERT%3A -Pretraining-Task-Agnostic-Visiolinguistic-Lu-Batra/65a9c7b0800c86a196bc14e76 21f895c6ab287 Luke Bailey, E. L. Ong, Stuart D. Russell, et al (2023) Image Hijacks: Adversarial Images can Control Generative Models at Runtime. https://doi.org/10.48550/arx iv.2309.00236 34 Madry A, Makelov A, Schmidt L, et al (2019) Towards Deep Learning Models Resis- tant to Adversarial Attacks. https://doi.org/10.48550/arXiv.1706.06083, URL http://arxiv.org/abs/1706.06083, arXiv:1706.06083 Mori Y, Takahashi H, Oka R (1999) Image-to-word transformation based on dividing and vector quantizing images with words. URL https://w.semanticscholar.org/ paper/Image-to-word-transformation-based-on-dividing-and-Mori-Takahashi/8b2 9ffb4207435540ddecf4b14a8a32106b33830?citedSort=relevance&sort=total-citatio ns Nguyen A, Yosinski J, Clune J (2015) Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), p 427ā436, https://doi.org/ 10.1109/CVPR.2015.7298640, URL https://ieeexplore.ieee.org/document/7298640 OpenAI, Achiam J, Adler S, et al (2024) GPT-4 Technical Report. https://doi.org/10 .48550/arXiv.2303.08774, URL http://arxiv.org/abs/2303.08774, arXiv:2303.08774 Ouyang L, Wu J, Jiang X, et al (2022) Training language models to follow instructions with human feedback. https://doi.org/10.48550/arXiv.2203.02155, URL http: //arxiv.org/abs/2203.02155, arXiv:2203.02155 Pitropakis N, Panaousis E, Giannetsos T, et al (2019) A taxonomy and survey of attacks against machine learning. Computer Science Review 34:100199. https://do i.org/10.1016/j.cosrev.2019.100199, URL https://w.sciencedirect.com/science/ article/pii/S1574013718303289 Rabiner L (1989) A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE 77(2):257ā286. https://doi.org/10.110 9/5.18626, URL https://ieeexplore.ieee.org/document/18626 Radford A, Narasimhan K (2018) Improving Language Understanding by Generative Pre-Training. URL https://w.semanticscholar.org/paper/Improving-Languag e-Understanding-by-Generative-Radford-Narasimhan/cd18800a0f e0b668a1c19f2 ec95b5003d0a5035 Radford A, Wu J, Child R, et al (2019) Language Models are Unsupervised Multitask Learners. URL https://cdn.openai.com/better-language-models/language model sareunsupervisedmultitasklearners.pdf Radford A, Kim JW, Hallacy C, et al (2021) Learning Transferable Visual Models From Natural Language Supervision (CLIP). https://doi.org/10.48550/arXiv.210 3.00020, URL http://arxiv.org/abs/2103.00020, arXiv:2103.00020 Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representations by back- propagating errors. Nature 323(6088):533ā536. https://doi.org/10.1038/323533a0, URL https://w.nature.com/articles/323533a0 35 Schlarmann C, Hein M (2023) On the Adversarial Robustness of Multi-Modal Foun- dation Models. In: 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), p 3679ā3687, https://doi.org/10.1109/ICCVW60793.2023. 00395, URL https://ieeexplore.ieee.org/document/10350690 Shayegani E, Dong Y, Abu-Ghazaleh N (2023) Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models. In: The Twelfth Interna- tional Conference on Learning Representations, URL https://openreview.net/f or um?id=plmBsXHxgR&trk=publicpostcomment-text Shin T, Razeghi Y, IV RLL, et al (2020) AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts. https://doi.org/10.485 50/arXiv.2010.15980, URL http://arxiv.org/abs/2010.15980, arXiv:2010.15980 Sutskever I, Vinyals O, Le QV (2014) Sequence to Sequence Learning with Neural Networks. ArXiv URL https://w.semanticscholar.org/paper/Sequence-to-Seq uence-Learning-with-Neural-Networks-Sutskever-Vinyals/cea967b59209c6be22829 699f05b8b1ac4dc092d Szegedy C, Zaremba W, Sutskever I, et al (2014) Intriguing properties of neural net- works. In: 2nd International Conference on Learning Representations (ICLR 2014). arXiv, https://doi.org/10.48550/arXiv.1312.6199, URL http://arxiv.org/abs/1312 .6199, arXiv:1312.6199 Touvron H, Lavril T, Izacard G, et al (2023a) LLaMA: Open and Efficient Foundation Language Models. https://doi.org/10.48550/arXiv.2302.13971, URL http://arxiv. org/abs/2302.13971, arXiv:2302.13971 Touvron H, Martin L, Stone KR, et al (2023b) Llama 2: Open Foundation and Fine- Tuned Chat Models. ArXiv URL https://w.semanticscholar.org/paper/Llama-2 %3A-Open-Foundation-and-Fine-Tuned-Chat-Models-Touvron-Martin/104b0b1 da562d53cbda87aec79ef6a2827d191a Vassilev A, Oprea A, Fordyce A, et al (2024) Adversarial Machine Learning: A Tax- onomy and Terminology of Attacks and Mitigations. Tech. Rep. NIST Artificial Intelligence (AI) 100-2 E2023, National Institute of Standards and Technology, https://doi.org/10.6028/NIST.AI.100-2e2023, URL https://csrc.nist.gov/pubs/ai/ 100/2/e2023/final Vaswani A, Shazeer N, Parmar N, et al (2017) Attention is All you Need. In: Advances in Neural Information Processing Systems, vol 30. Curran Associates, Inc., URL https://proceedings.neurips.c/paper files/paper/2017/hash/3f5e243547dee91fbd 053c1c4a845a-Abstract.html Wallace E, Feng S, Kandpal N, et al (2021) Universal Adversarial Triggers for Attack- ing and Analyzing NLP. https://doi.org/10.48550/arXiv.1908.07125, URL http://arxiv.org/abs/1908.07125, arXiv:1908.07125 36 Wang Y, Ma X, Zhang G, et al (2024) MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark. https://doi.org/10.48550/arXiv .2406.01574, URL http://arxiv.org/abs/2406.01574, arXiv:2406.01574 Wang Z, Yu J, Yu AW, et al (2022) SimVLM: Simple Visual Language Model Pre- training with Weak Supervision. https://doi.org/10.48550/arXiv.2108.10904, URL http://arxiv.org/abs/2108.10904, arXiv:2108.10904 Wei A, Haghtalab N, Steinhardt J (2023) Jailbroken: How Does LLM Safety Training Fail? https://doi.org/10.48550/arXiv.2307.02483, URL http://arxiv.org/abs/2307 .02483, arXiv:2307.02483 White C, Dooley S, Roberts M, et al (2024) LiveBench: A Challenging, Contamination- Free LLM Benchmark. https://doi.org/10.48550/arXiv.2406.19314, URL http: //arxiv.org/abs/2406.19314, arXiv:2406.19314 Xu K, Ba J, Kiros R, et al (2015) Show, Attend and Tell: Neural Image Caption Gen- eration with Visual Attention. In: International Conference on Machine Learning, URL https://w.semanticscholar.org/paper/Show%2C-Attend-and-Tell%3A-N eural-Image-Caption-with-Xu-Ba/4d8f2d14af5991d4f0d050d22216825cac3157bd Zou A, Wang Z, Carlini N, et al (2023) Universal and Transferable Adversarial Attacks on Aligned Language Models. https://doi.org/10.48550/arXiv.2307.15043, URL http://arxiv.org/abs/2307.15043, arXiv:2307.15043 37