Paper deep dive
AirflowAttack: Thermal-Airflow Adversarial Perturbations against Infrared Remote-Sensing Vision-Language Models
Cong Su, Jiaju Han, Xuemeng Sun, Chengyin Hu, Qike Zhang, Jiujiang Guo, Yiwei Wei, Jiahuan Long
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 7/8/2026, 6:22:34 AM
Summary
The paper introduces AirflowAttack, the first adversarial attack targeting infrared remote-sensing vision-language models. It synthesizes a universal, input-agnostic perturbation based on physically plausible thermal-airflow turbulence. Optimized on a surrogate CLIP model, it achieves a 48.5% mean attack success rate across five CLIP backbones and significantly degrades six state-of-the-art VLMs, exposing critical vulnerabilities while maintaining physical interpretability.
Entities (10)
Relation Signals (9)
AirflowAttack → uses → Thermal-Airflow Turbulence
confidence 97% · weaponize thermal-airflow turbulence as the perturbation prior
AirflowAttack → achieves → Attack Success Rate
confidence 96% · attains a mean zero-shot scene-classification attack success rate (ASR)... of 48.5%
AirflowAttack → optimizedon → OpenAI-CLIP-B32
confidence 95% · Optimized on one surrogate CLIP model... OpenAI-CLIP-B32
AirflowAttack → targets → Infrared Remote-Sensing Vision-Language Models
confidence 95% · We present AirflowAttack, to our knowledge the first adversarial attack for IR remote-sensing VLMs
AirflowAttack → transfersto → Qwen2.5-VL-7B
confidence 92% · Applied to six state-of-the-art VLMs, it cuts scene-classification accuracy by up to 38.2% relative
AirflowAttack → outperforms → Hot/Cold Blocks
confidence 90% · AirflowAttack exceeds all four IR-specific physical baselines on every backbone.
AirflowAttack → outperforms → Sensor Drift
confidence 90% · AirflowAttack exceeds all four IR-specific physical baselines on every backbone.
AirflowAttack → outperforms → Stripe Noise
confidence 90% · AirflowAttack exceeds all four IR-specific physical baselines on every backbone.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Vision-language models (VLMs) are increasingly deployed on infrared (IR) remote sensing imagery in security-critical settings, yet their adversarial robustness remains unexamined. We present AirflowAttack, to our knowledge the first adversarial attack for IR remote-sensing VLMs and the first to weaponize thermal-airflow turbulence as the perturbation prior. A lightweight generator synthesizes a single input-agnostic perturbation regularized toward physically plausible airflow patterns. Optimized on one surrogate CLIP model, it attains a mean zero-shot scene-classification attack success rate (ASR, the fraction of samples whose top-1 class changes) of 48.5% across five diverse CLIP backbones, far exceeding four IR-specific physical baselines (27.7--37.0%). Applied to six state-of-the-art VLMs, it cuts scene-classification accuracy by up to 38.2% relative, yet paradoxically makes some models more confident in their IR analysis, confabulating the perturbation as genuine thermal evidence such as temperature gradients and convection. Ablations show the airflow prior raises physical plausibility at no measurable cost to attack success. Together with a benchmark spanning eleven models and four tasks, these findings expose critical vulnerabilities in the rapidly expanding IR VLM ecosystem.
Tags
Links
- Source: https://arxiv.org/abs/2607.06485v1
- Canonical: https://arxiv.org/abs/2607.06485v1
Trouble viewing inline? Open PDF directly →
Full Text
99,159 characters extracted from source content.
Expand or collapse full text
AirflowAttack: Thermal-Airflow Adversarial Perturbations against Infrared Remote-Sensing Vision-Language Models Cong Su 2 , Jiaju Han 1 , Xuemeng Sun 1 , Chengyin Hu 1 , Qike Zhang 1 , Jiujiang Guo 2 , Yiwei Wei 1 , and Jiahuan Long 3 1 China University of Petroleum-Beijing at Karamay, Karamay, Xinjiang, China 2 Tianjin University, Tianjin, China 3 Shanghai Jiao Tong University, Shanghai, China Abstract. Vision-language models (VLMs) are increasingly deployed on infrared (IR) remote sensing imagery in security-critical settings, yet their adversarial robustness remains unexamined. We present Air- flowAttack, to our knowledge the first adversarial attack for IR remote- sensing VLMs and the first to weaponize thermal-airflow turbulence as the perturbation prior. A lightweight generator synthesizes a single input- agnostic perturbation regularized toward physically plausible airflow pat- terns. Optimized on one surrogate CLIP model, it attains a mean zero- shot scene-classification attack success rate (ASR, the fraction of samples whose top-1 class changes) of 48.5% across five diverse CLIP backbones, far exceeding four IR-specific physical baselines (27.7–37.0%). Applied to six state-of-the-art VLMs, it cuts scene-classification accuracy by up to 38.2% (relative)—yet paradoxically makes some models more confident in their IR analysis, confabulating the perturbation as genuine thermal evidence such as temperature gradients and convection. Ablations show the airflow prior raises physical plausibility at no measurable cost to at- tack success. Together with a benchmark spanning eleven models and four tasks, these findings expose critical vulnerabilities in the rapidly expanding IR VLM ecosystem. Keywords: Adversarial attack· Infrared remote sensing· Vision-language model· Transferable perturbation· Thermal airflow 1 Introduction Infrared remote sensing underpins critical applications from disaster monitor- ing and environmental surveillance to military reconnaissance, operating un- der conditions where visible-spectrum imaging fails—nighttime, fog, smoke, and thermal camouflage detection. The recent adaptation of vision-language models (VLMs) to the IR domain promises a step change in automated scene under- standing: models such as GeoRSCLIP [46], RemoteCLIP [21], and RS5M [47] arXiv:2607.06485v1 [cs.CV] 7 Jul 2026 2C. Su et al. Adversarial Image 풙 풂풅풗 =clip풙+푮⊙흐⋅풕풂풏풉풂푷 풂풊풓 +ퟎ.ퟔ⋅BlurUpsample풁,ퟎ,ퟏ AirflowPerturbationSynthesisModule: (1) Airflow Magnitude Mapping for Spatial Gating SignedAirflowPrior 푮=푺풎풐풕풉푷 air SpatialAirflowGate MagnitudeExtraction andGateFormation Positive Thermal Perturbation Negative Thermal Perturbation (2) Low-Dimensional Residual for Coarse Airflow Control Learnable residual grid 푍∈푅 ퟛퟚ×ퟛퟚ Coarse-grained control Positive Thermal Perturbation Negative Thermal Perturbation Each cell controls a local airflow adjustment (3) Upsamplingand Smoothing to Continuous Residual Field Low−dimensionalresidualZ Coarselocalresidual Upsample Expandtoimageresolution Gaussiansmoothing Removeblockartifacts ContinuousresidualfieldR Smoothlocalairflowrefinements 푹=BlurUpsample풁 Positive Negative ... ... ... ... ...... Parameters RandomInitial AdamWOptimizer Final Perturbation Global amplitude 푎 ... Low- dimensional residual 푍 Lr:0.095 Momentum update Z 퐦퐢퐧 풁,풂 퓛풁,풂=ퟖ퓛 퓬퓸퓷퓯 풙 풂풅풗 +퓛 퓪퓲퓻 휹,푷 풂풊풓 Step:800 a 1. Classification Adv image Inconformity Adv image Clean Residential are a with tennis court, buildings, roads, and trees. Intersec tion with vehic le s, buildings, roads, and bright infrared spots.. ChatGPT Scoring 2. Caption ...... ChatGPT Scoring Top-1 Prediction Ground Truth 3. VQA What is the main scene type in this infrared remote sens ing image? Residentia l ar ea What is the main scene type in this infrared remote sens ing image? VQAResidential area with roads, vehicles, and s tructural outlines . Fig. 1: Overview of AirflowAttack. A lightweight generatorG θ maps a low-dimensional latent code to a single-channel thermal-airflow perturbation, optimized on a surrogate IR-finetuned CLIP model under an L ∞ ≤ ε constraint using a confidence lossL conf and an airflow-correlation loss L air . The resulting perturbation transfers, without target- model access, to five CLIP backbones and six VLMs across four vision-language tasks. can now jointly reason about IR imagery and natural language, enabling open- vocabulary retrieval, descriptive captioning, and visual question answering over thermal scenes. However, the security implications of deploying VLMs in IR- sensitive contexts remain entirely unexamined. Adversarial attacks—imperceptible input perturbations that cause models to fail—have been extensively studied in the RGB domain, spanning white-box [15], black-box [29], and universal [27] regimes. Yet IR imagery differs fundamentally from RGB: thermal sensors capture emitted radiation rather than reflected light, producing single-channel intensity maps governed by Planck’s law where pixel values encode physical temperature. This physical grounding both constrains and motivates a new class of attacks: rather than crafting arbitrary digital noise, an adversary can simulate physically plausible thermal phenomena—such as airflow- induced temperature distortions—that are simultaneously harder to detect and more likely to transfer across models. In this paper, we introduce AirflowAttack, to the best of our knowledge the first adversarial attack designed for IR remote-sensing VLMs. While universal perturbations have been studied for unimodal remote-sensing classifiers [41] and for RGB vision-language models [45,49], and while simulated atmospheric phe- nomena such as haze [14] and weather [32] have been repurposed as attacks in the RGB domain, none of these targets the thermal-IR modality, the remote- sensing VLM setting, or thermal-airflow turbulence as the perturbation prior— the specific intersection our work addresses. Our approach synthesizes an input- Thermal-Airflow Attack against IR Remote Sensing VLMs3 agnostic adversarial perturbation by modeling temperature fluctuations induced by thermal airflow turbulence. Unlike conventional pixel-space UAPs that pro- duce unstructured noise, the proposed perturbation mimics the spatially corre- lated, physically grounded patterns of atmospheric thermal mixing—making it simultaneously effective, transferable across architectures, and physically inter- pretable as a natural thermal phenomenon (see Fig. 1). We conduct a comprehensive adversarial robustness evaluation spanning five CLIP-family backbones (OpenAI-CLIP-B32, OpenAI-CLIP-L14, OpenCLIP-B32, RemoteCLIP-B32, GeoRSCLIP-B32) and six state-of-the-art VLMs (Qwen2.5- VL-7B, InstructBLIP, LLaVA-1.5, LLaVA-1.6, GeoChat, H2RSVLM) across four vision-language tasks on a dedicated 1,000-sample infrared test set. Our contri- butions are: – We propose AirflowAttack, to our knowledge the first adversarial attack for IR remote-sensing VLMs, synthesizing a transferable thermal-airflow pertur- bation via a lightweight generator. – Across five CLIP backbones and six VLMs, a single surrogate-optimized perturbation attains 48.5% mean ASR, exceeding four IR-specific baselines and transferring without target access. – Extensive ablations reveal the mechanisms of airflow-based transfer, includ- ing a paradoxical rise in some models’ IR-cue confidence under attack. 2 Related Work 2.1 Infrared Remote Sensing Vision-Language Models CLIP [30] showed that contrastive image-text pretraining yields strong zero- shot transfer, a paradigm extended to remote sensing by RemoteCLIP [21], GeoRSCLIP [46], and RS5M [47], and by general-purpose VLMs such as LLaVA- 1.5/1.6 [22,23], InstructBLIP [12], Qwen2.5-VL [2], GeoChat [19], and H2RSVLM [28] applied to RS scene classification, detection, VQA, and captioning. These mod- els are developed and evaluated under benign conditions. While the robustness of unimodal RS classifiers has been studied [40,41], and very recent work crafts transferable adversarial examples for RS object recognition [20] and attacks in- frared VLMs with physical patches [16], the robustness of IR remote-sensing VLMs to input-agnostic perturbations remains unexamined. 2.2 Adversarial Attacks Since neural networks were shown vulnerable to imperceptible perturbations [33], white-box attacks (FGSM [15], C&W [7], PGD [25], AutoAttack [10]) and transfer- boosting methods (MI-FGSM [13], DIM [38]) have been widely studied. Univer- sal perturbations [27] fool a model across inputs with a single pattern, and have been extended to remote sensing [41] and to CLIP/VLP models (AdvCLIP [49], Co-Attack [45], and recent targeted VLM attacks [6]). These target unimodal RGB classifiers or RGB CLIP; physically realizable RGB attacks [1,5] likewise 4C. Su et al. exploit color/texture cues absent in thermal imagery. Our perturbation instead targets the single-channel thermal modality and is parameterized to resemble a physical thermal phenomenon. 2.3 Physical Attacks on Thermal Imaging A parallel line of work explores physical attacks specific to thermal sensors. Methods include projecting thermal patterns using controlled heat sources [37], physically adversarial infrared patches with learnable shapes and locations [51], adversarial infrared curves and grid patterns against pedestrian detectors [17,34], wearable hot/cold blocks that fool thermal detectors [36], exploiting sensor non- uniformity [4], and introducing fixed-pattern noise [3]. A recent survey [11] pro- vides a broader taxonomy of IR-specific adversarial methods. A closely related line simulates a physical atmospheric phenomenon and repurposes it as an adver- sarial perturbation—adversarial haze [14], adversarial weather [32], and, concur- rently, physically-induced atmospheric perturbations for RS classification [52]; our airflow-turbulence perturbation is a new instance of this template, uniquely targeting the thermal-IR VLM setting. Atmospheric turbulence, caused by spa- tial and temporal variations in air refractive index due to temperature gradients, is a well-known degradation in long-range thermal imaging [50]; learned turbu- lence simulation models [26] have improved the fidelity of synthetic turbulence. However, prior work treats turbulence as a nuisance to be corrected, not as an adversarial primitive to be exploited. Our work is the first to synthesize thermal airflow turbulence patterns as a deliberate universal perturbation optimized for attacking multimodal IR models. 2.4 VLM Robustness Recent studies have begun examining the adversarial robustness of VLMs [31,48]. Adversarial images can cause VLMs to produce hallucinated captions [44], and visual adversarial perturbations can transfer to the language modality [39]. Set- of-mark prompting [42] further reveals how VLMs attend to image regions, which is directly relevant to understanding why spatially localized perturbations mat- ter. However, existing work focuses almost exclusively on RGB inputs and generic multimodal reasoning. The IR modality introduces distinct challenges: single- channel intensity encoding, different feature statistics, and physical constraints on plausible perturbations. To the best of our knowledge, this work is the first to study adversarial robustness for IR remote-sensing VLMs. 3 Method We propose AirflowAttack, a framework for generating universal adversarial per- turbations that simulate thermal airflow turbulence to attack IR vision-language models. The key insight is that physically interpretable thermal patterns—unlike arbitrary pixel noise—exploit domain-specific feature representations and trans- fer more effectively across architectures. Thermal-Airflow Attack against IR Remote Sensing VLMs5 3.1 Threat Model We consider a gray-box adversary with access to a surrogate CLIP model (OpenAI- CLIP-B32, IR-finetuned) but no access to target models, their parameters, or training data. The adversary can perturb input IR images before they are pro- cessed by downstream VLMs. The perturbation must satisfy an L ∞ constraint: ∥δ∥ ∞ ≤ ε, where ε = 100 in pixel intensity space (out of 255). The adversary’s goal is to craft a single universal perturbation δ that, when added to any IR im- age x, causes incorrect scene classification, captioning, or VQA outputs across multiple target models. This is a realistic threat: IR sensors deployed in the field may process adversarially perturbed inputs before human or automated analysis. 3.2 Thermal Airflow Perturbation Model Rather than optimizing δ directly in pixel space, we parameterize the perturba- tion through a lightweight generative model G θ (z) that maps a low-dimensional latent vector z ∈R d to a full-resolution perturbation pattern. This parame- terization serves three purposes: it regularizes the perturbation toward spatially coherent patterns, it reduces the optimization search space, and it enables the in- tegration of physical priors through architectural design. The generator consists of a series of transposed convolution layers with residual connections, mapping a latent code of dimension d = 32 to a single-channel output matching the input resolution. We employ a residual ratio of r = 0.60, meaning that 60% of the per- turbation energy passes through the residual pathway, which we find empirically balances attack strength against physical plausibility (see Sec. 4.5). For each training iteration, a latent vector z is sampled and decoded to produce the perturbation δ = G θ (z). The perturbation is then scaled to satisfy the L ∞ constraint: δ ← clip(δ,−ε,ε). The adversarial image is formed as x adv = x + δ. 3.3 Optimization Objective The optimization objective combines an attack loss that drives retrieval failure with a correlation loss that enforces physical plausibility. Given an IR image x, its paired caption t + , and a set of distractor captions T − , we define the confidence (contrastive image–text alignment) loss: L conf =− log exp(τ · sim(f I (x adv ),f T (t + ))) P t∈t + ∪T − exp(τ · sim(f I (x adv ),f T (t))) (1) where f I and f T are the image and text encoders of the surrogate CLIP model, sim(·,·) denotes cosine similarity, and τ is a temperature parameter. Min- imizing L conf reduces the similarity between the adversarial image embedding and its correct caption while increasing similarity with distractors. To encourage the perturbation to resemble natural thermal airflow, we intro- duce an airflow correlation loss: 6C. Su et al. Algorithm 1: AirflowAttack universal perturbation optimization Input: Surrogate CLIP encoders f I ,f T ; IR pairs (x,t + ) with distractors T − ; airflow templates A; budget ε, steps T, rate η, weights α,β Output: Universal perturbation δ ⋆ 1 θ,z ← Init// generator params and latent code 2 for i = 1...T do 3 δ ← clip G θ (z), −ε, ε // decode, project to L ∞ ball 4 x adv ← x + δ 5 L conf ←− log exp(τ sim(f I (x adv ),f T (t + ))) P t∈t + ∪T − exp(τ sim(f I (x adv ),f T (t))) // attack loss 6 L air ← 1− Corr(δ,A)// airflow-plausibility loss 7 L← αL conf + βL air 8(θ,z)← (θ,z)− η∇ (θ,z) L// gradient step 9 end 10 δ ⋆ ← clip G θ (z), −ε, ε 11 return δ ⋆ L air = 1− Corr(δ,A)(2) where A is a bank of synthetic thermal airflow templates generated by sim- ulating atmospheric turbulence via randomized heat kernel convolution, and Corr(·,·) denotes spatial Pearson correlation. This term penalizes perturbations that deviate from physically plausible airflow patterns. The total loss is a weighted combination: L = α·L conf + β·L air (3) with α = 8 and β = 2, determined through ablation (Sec. 4.5). Optimization proceeds via projected gradient descent over the generator parameters θ and latent code z for 800 steps with learning rate η = 0.055. Algorithm 1 summarizes the full procedure. 3.4 Surrogate-to-Target Transfer Once optimized on the surrogate model, the final UAP δ ∗ = G θ ∗ (z ∗ ) is applied directly to any target IR image. No access to target models is required at deploy- ment time. The perturbation transfers effectively because: (i) the airflow pattern encodes domain-general thermal features that multiple IR-trained models have learned to associate with atmospheric conditions; (i) the universal nature of the perturbation means it does not overfit to surrogate-specific decision boundaries; and (i) the L ∞ constraint ensures the perturbation magnitude is consistent across inputs, avoiding the brittleness of per-image attacks. Thermal-Airflow Attack against IR Remote Sensing VLMs7 4 Experiments 4.1 Experimental Setup Models. We evaluate five CLIP-family backbones spanning different architec- tures, pretraining data, and IR adaptation strategies: OpenAI-CLIP-B32, OpenAI- CLIP-L14 [30], OpenCLIP-B32 [9], RemoteCLIP-B32 [21], and GeoRSCLIP- B32 [46]. For VLM evaluation, we use six state-of-the-art models: Qwen2.5-VL- 7B [2], InstructBLIP [12], LLaVA-1.5 (7B) [23], LLaVA-1.6 (7B) [22], GeoChat (7B) [19], and H2RSVLM [28]. Datasets. Our dataset is constructed from five public RS sources (NWPU- Caption [8], RSICD [24], RSITMD [43], RS5M [47], SkyScript [35]), filtered to infrared samples to prevent RGB information leakage. The filtering procedure removes samples whose image filenames or metadata contain RGB indicators. Final class distribution and per-source proportions are detailed in the supple- mentary material. The test split contains 10,000 IR image-text pairs, of which 9,720 carry a remote-sensing scene-category label and are used for the CLIP zero- shot classification attack; the validation split (used for ablations) contains 416 samples. The VLM evaluation uses 1,000 randomly sampled labeled IR images with annotations for four tasks. Metrics. For CLIP, each backbone is a zero-shot scene classifier over remote- sensing scene categories: each IR image is assigned the scene class whose text prompt has the highest image–text cosine similarity. We report Attack Success Rate (ASR): the percentage of samples whose adversarial top-1 scene predic- tion differs from the clean top-1 prediction. 4 For the cross-model transfer study (Sec. 4.2), where a single perturbation is applied to held-out samples and eval- uated by nearest-caption retrieval over a 1,000-sample candidate pool, we ad- ditionally report retrieval top-1 flip rate, confidence drop, and the number of unique adversarial top-1 captions as an error-diversity measure. For VLM eval- uation, we report ROUGE-L for caption quality, scene accuracy, object F1, and IR-cue accuracy. All metrics use the full evaluation set. For VLM experiments, we assess every clean-vs-attack change with a two-proportion test (n=1000) and Wilson 95% confidence intervals, correcting 18 comparisons with both Bonfer- roni and Benjamini–Hochberg (FDR) procedures. Because clean and adversarial predictions are evaluated on the same images, this unpaired test treats them as independent and is therefore conservative relative to an exact paired (McNemar) test; the significance markers we report thus understate rather than overstate the effects. Baselines. We compare against four IR-specific physical attack methods: (1) Atmospheric thermal turbulence—adding synthetic turbulence patterns gen- erated by phase-screen propagation models [26] (distinct from our airflow tem- plate bank, which uses randomized heat-kernel convolution, so that the baseline is an independent turbulence formulation rather than a weakened version of our 4 ASR is a paired flip rate—clean vs. adversarial prediction on the same image—so the clean condition is the reference (0% by construction) and no separate clean-accuracy baseline is required. The denominator is the full evaluated set. 8C. Su et al. Table 1: CLIP zero-shot scene-classification attack success rate (ASR %) across five backbones at ε = 100. ASR is the fraction of test samples whose top-1 predicted scene class under attack differs from the clean prediction. AirflowAttack exceeds all four IR-specific physical baselines on every backbone. MethodOpenAI-CLIP-L14 OpenAI-CLIP-B32 OpenCLIP-B32 RemoteCLIP-B32 GeoRSCLIP-B32 Atmospheric turbulence36.935.636.128.830.8 Stripe noise28.239.538.341.337.8 Hot/cold blocks25.229.628.531.830.7 Sensor drift24.932.125.927.927.7 AirflowAttack (Ours)38.354.247.950.651.7 prior); (2) IR fixed-pattern stripe noise—adding periodic column-wise noise mim- icking sensor readout artifacts; (3) Thermal hot/cold blocks—overlaying rectan- gular regions with elevated or depressed temperatures, following the HOTCOLD block design [36]; (4) Thermal sensor drift/non-uniformity—applying pixel-wise gain and offset variations simulating detector degradation. All baselines oper- ate under the same ε = 100 constraint. We deliberately restrict the comparison to IR-specific physical attacks, which share AirflowAttack’s physical-plausibility motivation; unstructured digital baselines (pixel-space UAP, Gaussian noise) ad- dress the orthogonal question of whether perturbation structure matters, which we examine directly through the loss-component and spatial-position ablations in Sec. 4.5. 4.2 Experiment 1: Zero-Shot Scene Classification Attack Figure 2 shows representative zero-shot classification flips, and Tab. 1 reports the ASR of AirflowAttack compared to four IR-specific baselines across five CLIP backbones. AirflowAttack achieves the highest ASR on every backbone, with a mean ASR of 48.5% versus 33.6% (turbulence), 37.0% (stripe noise), 29.2% (hot/cold blocks), and 27.7% (sensor drift). The strongest result is on OpenAI- CLIP-B32 (54.2%), the surrogate model, but high ASR persists on architec- turally distinct models: GeoRSCLIP-B32 (51.7%), RemoteCLIP-B32 (50.6%), and OpenCLIP-B32 (47.9%). Even on OpenAI-CLIP-L14, which uses a different visual encoder (ViT-L/14 vs. ViT-B/32), AirflowAttack achieves 38.3% ASR, exceeding the best baseline (turbulence at 36.9%). Because ASR is measured as a paired flip of the top-1 scene prediction relative to clean, it isolates the ef- fect of the perturbation itself, independent of each backbone’s underlying clean accuracy. Two observations merit emphasis. First, the ranking of baselines is inconsis- tent across backbones: stripe noise is the strongest baseline on four backbones (OpenAI-CLIP-B32, OpenCLIP-B32, RemoteCLIP-B32, and GeoRSCLIP-B32), while turbulence leads only on OpenAI-CLIP-L14. AirflowAttack is the only method that consistently dominates. Second, the performance gap between Air- flowAttack and baselines is largest precisely on the backbones most relevant to IR Thermal-Airflow Attack against IR Remote Sensing VLMs9 Baseball diamondGolf course PalaceChurch Roundabout Stadium PondTennis court Stadium Church Church Basketball court Fig. 2: Qualitative zero-shot scene classification under AirflowAttack on six IR images. Top row (green): clean inputs correctly classified by the surrogate CLIP model. Bottom row (red): the same images with the universal thermal-airflow perturbation, now misclassified as unrelated categories. The faint, coherent airflow-like texture flips top-1 predictions while preserving human-recognizable scene content. deployment—RemoteCLIP-B32 and GeoRSCLIP-B32—where domain-specific pretraining appears to amplify sensitivity to physically structured perturbations. Cross-Model Transfer. A defining property of a universal perturbation is cross-model transfer: a perturbation optimized on one model should remain ef- fective on others without target-model gradients or adaptation. Using a sin- gle UAP optimized on OpenAI-CLIP-B32 and a nearest-caption retrieval probe over a 1,000-sample candidate pool, the perturbation changes the top-1 nearest caption for 94.4% (OpenAI-CLIP-L14) to 98.8% (GeoRSCLIP-B32) of held-out samples across all five backbones, despite being optimized solely on the surro- gate. This indicates that the learned airflow pattern does not merely exploit id- iosyncratic surrogate decision boundaries, but perturbs thermal representations shared by general and remote-sensing CLIP variants. This retrieval flip rate is a more sensitive probe than the scene-classification ASR of Tab. 1—any change in the nearest caption counts, not only a change of predicted scene class—so its values are correspondingly higher and are not directly comparable. Notably, a higher confidence drop does not imply a higher flip rate: OpenAI-CLIP-L14 shows the largest confidence reduction yet the lowest flip rate, which may indi- cate wider retrieval margins in the ViT-L/14 embedding space, though we do not test this directly. Per-model flip rates, confidence drops, and caption-diversity statistics are reported in Tab. 2. 4.3 Experiment 2: Image Captioning Attack We next transfer the AirflowAttack UAP—the same perturbation optimized on the CLIP surrogate—to six generative VLMs, applied identically to all in- puts. We first evaluate infrared image captioning, scoring the generated caption 10C. Su et al. Table 2: Cross-model transfer of one AirflowAttack UAP optimized on OpenAI-CLIP- B32 and applied without target access to 1,000 held-out samples per backbone. Flip rate (%) measures top-1 nearest-caption changes relative to clean; Conf. Drop is the mean cosine-similarity reduction to the correct caption. The UAP transfers strongly across architectures (94.4–98.8%); ViT-L/14 is most resistant despite the largest confidence drop, suggesting wider retrieval margins. MetricOpenAI-CLIP-L14 OpenAI-CLIP-B32 OpenCLIP-B32 RemoteCLIP-B32 GeoRSCLIP-B32 Flip Rate (%)94.498.197.398.598.8 Conf. Drop0.2230.1490.1760.0770.165 Table 3: Captioning impact across six downstream VLMs measured by ROUGE-L. All values are percentages. Lower values indicate stronger caption degradation. The best attack result in each column is highlighted. MethodQwen2.5-VL InstructBLIP LLaVA-1.5 LLaVA-1.6 GeoChat H2RSVLM Hot/Cold Block11.7512.3110.5910.1910.1311.69 IR Stripe11.8012.2910.5810.1110.0711.65 Thermal Drift11.7712.1310.4110.2010.0811.59 Thermal Turb.11.7412.1110.3710.169.8411.74 Ours11.7011.9110.4210.249.8011.64 against the reference with ROUGE-L. Table 3 reports ROUGE-L under Air- flowAttack and the four physical baselines for all six models. Caption quality shows a nuanced pattern. AirflowAttack attains the low- est ROUGE-L on three of the six models (Qwen2.5-VL, InstructBLIP, and GeoChat), matching or exceeding the physical baselines elsewhere. The abso- lute changes, however, are small and non-monotonic across methods: ROUGE-L is a lexical-overlap metric rather than a measure of factual accuracy, so a pertur- bation that induces more generic, repetitive captions can raise n-gram overlap even as semantic content degrades. Qualitative inspection (Fig. 3) confirms that attacked captions frequently describe non-existent objects or misidentify scene categories; a lexical-overlap metric therefore understates the true degradation, motivating the reasoning-based VQA evaluation in Experiment 3. 4.4 Experiment 3: Visual Question Answering Attack We then probe higher-level reasoning through three VQA-style tasks: scene clas- sification (accuracy, over a VLM-specific label set distinct from the CLIP zero- shot categories in Experiment 1), object presence recognition (F1), and infrared- cue explanation (accuracy). Table 4 reports clean, physical-baseline, and Air- flowAttack scores. Scene classification suffers the most severe and statistically robust degra- dation: the drops on Qwen2.5-VL-7B (35.24→21.83, 38.1%), InstructBLIP (23.66→14.63, 38.2%), LLaVA-1.5 (21.46→16.22) and LLaVA-1.6 (23.54→16.22, 31.1%) all sur- Thermal-Airflow Attack against IR Remote Sensing VLMs11 Clean Example Promt: Describe this infrared remote sensing image in detail. Adversarial Example The grayscale infrared-style image shows a high-contrast aerial view of a city with numerous buildings, where varying intensities highlight structural outlines and low-texture regions. The scene exhibits bright spots indicating reflective surfaces and darker areas suggesting shadowed or less illuminated zones within the urban landscape. The grayscale intensity patterns reveal a low-texture region representing the bare land, with subtle structural outlines of scattered trees appearing as faint contrasts against the uniform background. Clean Example Adversarial Example Question:What infrared-specific visual cues can be observed in this image? The dense residential area exhibits varying grayscale intensities with brighter patches indicating buildings and darker areas suggesting tree cover. Structural outlines of houses are clearly defined against the lower-texture background of foliage. (A) Image-Captioning Task (B) Visual Question Answering Task Clean Example Promt : Describe this infrared remote sensing image in detail. Adversarial Example The grayscale infrared-style image shows a residential area with varying intensity levels indicating different materials and structures. The central building exhibits high contrast with its dark roof and lighter walls, while the surrounding areas display lower texture details with softer gradients. The scene captures the structural outlines of houses and streets, with brighter patches suggesting open spaces or reflective surfaces. The grayscale intensity highlights various buildings with distinct shapes adjacent to the railway station, showing high contrast between structures and low-texture regions around them. Clean Example Promt : Describe this remote sensing image. Adversarial Example The grayscale infrared-style image shows a tennis court with distinct bright and dark intensity patterns highlighting the court's structure and surrounding low-texture regions. The court's surface exhibits varying shades of gray, indicating different material properties or surface conditions. The surrounding area displays lower contrast with subtle structural outlines, emphasizing the court's central position. A grayscale aerial view captures the tennis court with distinct intensity contrasts; the darker lines delineate the boundaries while lighter tones highlight the smooth surface of the court. h LLaVa-1.5 LLaVa-1.6 Qwen2.5-VL H2RSVLM GeoChat InstructBLIP IR_cues VQA The infrared-style image reveals several houses with varying grayscale intensities, where brighter areas highlight the structural outlines of buildings and darker regions indicate low-texture or shadowed areas. The scene exhibits high contrast between the illuminated parts of the houses and the surrounding environment, emphasizing the architectural details. Clean Example Adversarial Example Question:What are the main objects or land-cover elements visible in this infrared remote sensing image? The dark, low-texture region represents the pond, contrasting sharply with the brighter, high-intensity outlines of the surrounding trees and grassy areas. Objects VQA Buildings, Road, Trees Clean Example Adversarial Example Question:What is the main scene type in this infrared remote sensing image? Residential area with varying grayscale intensities, where buildings exhibit high contrast and structural outlines against lower-texture regions. Scene VQA Urban Scene Clean Example Adversarial Example Question:What are the main objects or land- cover elements visible in this infrared remote sensing image? The grayscale infrared-style view captures a low-texture water surface with faint structural outlines of distant buildings and a bright, high-contrast object resembling a boat. Objects VQA Water, Buildings, Trees Fig. 3: Effect of AirflowAttack on VLM captioning and visual question answering. Under attack, models produce more generic captions and confabulate thermal cues (temperature gradients, convection signatures) absent from the clean image, illustrating the IR-cue paradox quantified in Tab. 4. vive Bonferroni correction (p < 0.0028), and GeoChat survives FDR. The per- turbation systematically misleads models about scene identity, a failure mode with direct operational consequences for IR-based surveillance and monitoring. Object recognition (F1) shows a much weaker effect. Only Qwen2.5-VL-7B and GeoChat reach FDR significance, and no object-F1 change survives Bon- ferroni correction; the changes on InstructBLIP (p=0.75), LLaVA-1.5 (p=0.26), LLaVA-1.6 (unchanged, p=1.0) and H2RSVLM (p=0.12) are statistically in- distinguishable from zero at n=1000. We therefore do not claim broad object- recognition degradation: object presence is markedly more robust to the per- turbation than scene identity, plausibly because it depends on localized high- contrast structure that a low-magnitude airflow field leaves intact. IR cue accuracy reveals the most intriguing finding: two of six models (Qwen2.5-VL-7B and InstructBLIP) exhibit higher IR-cue detection rates un- der attack, most strikingly Qwen2.5-VL-7B, whose accuracy rises from 86.59 to 94.43—the opposite of the intended attack effect. This occurs because the airflow perturbation introduces structured thermal patterns that these models interpret as genuine IR phenomena—temperature gradients, thermal shadows, convection signatures—triggering confident but incorrect IR-cue identifications. The perturbation effectively manufactures thermal “evidence” that the models’ IR understanding modules latch onto, demonstrating a form of adversarial ex- ploitation unique to the thermal modality. The remaining four models show only modest IR-cue decreases, so this confabulation effect is model-dependent rather than universal. To understand how the perturbation induces these failures, Fig. 4 visualizes the attention shift it causes: the perturbation redirects model attention away from scene-defining regions, flipping the top-1 prediction (e.g., island→church, farmland→building). We now analyze the factors that govern this effect through a series of ablations. 12C. Su et al. Table 4: VQA impact across six downstream VLMs. Scene Acc., Object F1, and IR-Cue Acc. are reported in percentages. Lower adversarial values indicate stronger degradation. The best attack result in each column is highlighted. Metric MethodQwen2.5-VL InstructBLIP LLaVA-1.5 LLaVA-1.6 GeoChat H2RSVLM Scene Acc. Clean35.2423.6621.4623.5424.2724.27 Hot/Cold Block21.8319.5117.6819.1521.1021.83 Thermal Drift21.9517.9317.0719.1520.4922.20 IR Stripe20.8517.5616.7118.7820.7321.59 Thermal Turb.21.5916.5914.3916.7119.7618.78 Ours21.8314.6316.2216.2219.3920.37 Object F1 Clean33.1924.1826.7225.3525.9125.52 Thermal Drift28.1224.5924.5724.8722.6523.86 Hot/Cold Block27.6923.8324.7725.0723.0423.91 IR Stripe27.2123.1424.2925.1823.0822.79 Thermal Turb.26.9022.5624.2323.6122.3422.24 Ours28.6023.6424.4725.2821.7322.53 IR-Cue Acc. Clean86.5981.1398.8095.3498.1392.40 IR Stripe93.9084.0096.3190.8495.7189.09 Hot/Cold Block94.0183.8396.4790.1795.1188.71 Thermal Drift94.0683.2995.9689.9395.5188.71 Thermal Turb.94.0783.1194.6689.8695.7488.57 Ours94.4382.9497.4491.9095.7088.84 4.5 Ablation Studies We conduct extensive ablations to understand the factors governing AirflowAt- tack’s efficacy. All ablations are performed on OpenAI-CLIP-B32 using the val- idation set, with the full attack configuration (ε = 100, latent dimension 32, residual ratio 0.60, 800 steps, η = 0.055, loss weights α = 8, β = 2) as the reference. Perturbation Strength As shown in Fig. 5a, ASR increases monotonically with ε across all five CLIP backbones: 12.9% at ε = 20, 42.2% at ε = 100 (our reference budget), and 59.7% at ε = 200 (mean across backbones). Note that ab- lation ASR values (42.2% at ε = 100) differ from Tab. 1’s ASR (48.5% mean at ε = 100) because ablations are run on a smaller validation subset (416 samples) rather than the full 9,720-sample test set. The near-linear relationship between perturbation budget and attack success suggests that the airflow perturbation remains effective across a wide range of visibility levels. At ε = 20, the pertur- bation is nearly invisible (less than 8% of the dynamic range), yet still achieves non-trivial ASR, indicating that even subtle thermal patterns can disrupt IR scene understanding. Loss Component Analysis Figure 6a decomposes the contribution of each loss term. The dominant factor is clearly the confidence loss: the configurations that include it (full at 47.9% and no-air at 48.0%) reach ∼48% ASR, whereas Thermal-Airflow Attack against IR Remote Sensing VLMs13 Clean Attention Adversarial Attention Elephant Island FarmlandCloud Baseball Baseball Residential Road BearChurch Building Church Roundabout Building BuildingBuilding Fig. 4: Attention shift under AirflowAttack. Top row: clean model attention (Grad- CAM) with the correct top-1 scene prediction; bottom row: attention on the same images under the perturbation, with the flipped adversarial prediction. The airflow perturbation redirects the model’s spatial attention away from scene-defining content, driving the top-1 class change that ASR (Tab. 1) measures. 20406080100120140160180200 Perturbation budget ε 0 20 40 60 80 ASR (%) default = 100 OpenAI-B32OpenAI-L14OpenCLIP-B32RemoteCLIP-B32GeoRSCLIP-B32 47.8 49.1 43.0 38.0 33.3 (a) Perturbation strength ε. 1002004006008001000 Optimization steps 46 47 48 49 ASR (%) ASRConf. drop selected = 800 48.4 0.0260 0.0265 0.0270 0.0275 Confidence drop 0.0275 (b) Optimization steps. Fig. 5: Optimization ablations on OpenAI-CLIP-B32 (validation set). (a) ASR rises monotonically with the perturbation budget ε across all five backbones; the dashed line marks our reference ε = 100. (b) ASR converges by∼800 steps and is stable thereafter, indicating robustness to early stopping. the two configurations driven primarily by the airflow prior (no-conf and fixed- prior) reach only 39.5% and 38.5%. This∼9-point gap confirms thatL conf is the primary driver of attack efficacy, whileL air governs physical plausibility (airflow correlation rises toward 0.985 as its weight increases). Within the confidence-dominated regime, the effect of adding the airflow term is negligible: the full objective (47.9%) and the no-air variant (48.0%) differ by only 0.1 points—well within the run-to-run noise floor at n=416 vali- dation samples. Adding the airflow prior thus does not measurably reduce ASR, while it substantially raises physical plausibility (correlation 0.844→ 0.893). We therefore include L air as an essentially cost-free way to make the perturbation physically interpretable, rather than claiming it improves attack strength. Spatial Position The perturbation’s spatial location strongly affects perfor- mance. We partition it into six regions (full image, top, bottom, left, right, center) and restrict δ to each region with zero-padding elsewhere, as shown in Fig. 6b. The full image reaches 47.6% ASR, followed by right (42.6%), center 14C. Su et al. Fixed prior No conf. FullNo air 0 10 20 30 40 50 ASR (%) ASRAirflow corr. selected 38.5 39.5 47.9 48.0 0.80 0.85 0.90 0.95 1.00 Airflow correlation 0.97 0.985 0.893 0.844 (a) Loss components. OursRightCenterBottomLeftTop 0 10 20 30 40 50 ASR (%) ASRConf. drop selected 47.6 42.6 42.3 38.9 36.3 34.6 0.015 0.020 0.025 Confidence drop 0.0273 0.0239 0.0212 0.0191 0.016 0.0157 (b) Spatial position. Fig. 6: Loss-composition and spatial-position ablations on OpenAI-CLIP-B32 (valida- tion set). (a) ASR (bars) is driven by the confidence loss, while airflow correlation (line) rises as the airflow prior is weighted more heavily. (b) ASR when the perturbation is restricted to one image region: covering the full image is most effective, followed by the right and center regions that hold scene-defining content. (42.3%), bottom (38.9%), left (36.3%), and top (34.6%). Right and center re- gions are most impactful, possibly because IR remote-sensing scenes often place key content such as buildings, vehicles, and terrain features near central or right- side areas, although we do not verify this attribution. The 13-point gap between full-image and top-only perturbations indicates that AirflowAttack depends on covering regions used for scene understanding. Hyperparameter Sensitivity We evaluate sensitivity to key hyperparame- ters, each swept independently with the others held at the reference configura- tion. Learning rate: ASR is stable in the 47.5–48.5% range for η ∈ [0.04, 0.07], optimal at η = 0.055. Optimization steps: ASR ranges 47.7–48.4% for 400– 1000 steps and peaks at 800 (Fig. 5b), indicating rapid convergence and robust- ness to early stopping. Latent dimension and residual ratio: within each sweep, a latent code of dimension 32 and a residual ratio r = 0.60 are the best-performing settings, and are adopted as the reference. Across all four hy- perparameters, ASR varies by less than two points within the ranges tested, demonstrating that AirflowAttack is robust to hyperparameter variation. 5 Conclusion We presented AirflowAttack, to the best of our knowledge the first adversar- ial attack for infrared remote-sensing vision-language models and the first to repurpose thermal-airflow turbulence as an adversarial perturbation. A single physically interpretable, input-agnostic perturbation achieves a mean zero-shot classification ASR of 48.5% (vs. 27.7–37.0% for IR-specific baselines), transfers from one surrogate to five CLIP backbones at a 94.4–98.8% flip rate, and— applied to six VLMs—significantly degrades scene classification while, on some models, paradoxically raising IR-cue confidence through adversarial confabula- tion. The ablations further show that attack strength is primarily driven by Thermal-Airflow Attack against IR Remote Sensing VLMs15 the confidence loss, whereas the airflow prior improves physical plausibility with negligible ASR cost. Together with a benchmark spanning eleven models and four tasks, these findings establish that IR remote-sensing VLMs are not ro- bust by default and that physically grounded thermal perturbations constitute a potent, modality-specific threat vector, motivating future work on detection, purification, and adversarial training for IR-specific defenses. References 1. Athalye, A., Carlini, N., Wagner, D.: Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In: International confer- ence on machine learning. p. 274–283. PMLR (2018) 2. Bai, S., Chen, K., Liu, X., Wang, J., Ge, W., Song, S., Dang, K., Wang, P., Wang, S., Tang, J., Zhong, H., Zhu, Y., Yang, M., Wan, J., Ding, W., Fu, Z., Xu, Y., Ye, J., Zhang, X., Xie, T., Cheng, Z., Zhang, H., Yang, Z., Xu, H., Lin, J.: Qwen2.5-vl technical report (2025) 3. Bouali, M., Ladjal, S.: Toward optimal destriping of modis data using a unidirec- tional variational model. IEEE Transactions on Geoscience and Remote Sensing 49, 2924–2935 (2011) 4. Boutemedjet, A., Deng, C., Zhao, B.: Robust approach for nonuniformity correction in infrared focal plane array. Sensors 16, 1890 (2016) 5. Brown, T.B., Mané, D., Roy, A., Abadi, M., Gilmer, J.: Adversarial patch. arXiv preprint arXiv:1712.09665 (2017) 6. Cao, Y., Li, Y., Liang, K., Xiao, B.: Enhancing targeted adversarial attacks on large vision-language models via intermediate projector. IEEE Transactions on Information Forensics and Security (2026) 7. Carlini, N., Wagner, D.: Towards evaluating the robustness of neural networks. In: 2017 IEEE Symposium on Security and Privacy (SP). p. 39–57. IEEE (2017) 8. Cheng, Q., Huang, H., Xu, Y., Zhou, Y., Li, H., Wang, Z.: Nwpu-captions dataset and mlca-net for remote sensing image captioning. IEEE Transactions on Geo- science and Remote Sensing 60, 1–19 (2022) 9. Cherti, M., Beaumont, R., Wightman, R., Wortsman, M., Ilharco, G., Gordon, C., Schuhmann, C., Schmidt, L., Jitsev, J.: Reproducible scaling laws for contrastive language-image learning. In: Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition. p. 2818–2829 (2023) 10. Croce, F., Hein, M.: Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In: International conference on machine learning. p. 2206–2216. PMLR (2020) 11. Dai, Q., Yang, X., Gao, H., Mu, H.: A survey of physical adversarial attacks against infrared target detection. In: 2025 7th International Conference on Software Engi- neering and Computer Science (CSECS). p. 1–5. IEEE (2025) 12. Dai, W., Li, J., Li, D., Tiong, A., Zhao, J., Wang, W., Li, B., Fung, P.N., Hoi, S.: Instructblip: Towards general-purpose vision-language models with instruction tuning. Advances in neural information processing systems 36, 49250–49267 (2023) 13. Dong, Y., Liao, F., Pang, T., Su, H., Zhu, J., Hu, X., Li, J.: Boosting adversarial attacks with momentum. In: Proceedings of the IEEE conference on computer vision and pattern recognition. p. 9185–9193 (2018) 14. Gao, R., Guo, Q., Juefei-Xu, F., Yu, H., Feng, W.: Advhaze: Adversarial haze attack. arXiv preprint arXiv:2104.13673 (2021) 16C. Su et al. 15. Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 (2014) 16. Hu, C., Dong, Y., Guo, Y., Chen, X., Wu, J., Long, J., Wei, Y., Jiang, T., Yao, W.: Revealing physical-world semantic vulnerabilities: Universal adversarial patches for infrared vision-language models. arXiv preprint arXiv:2604.03117 (2026) 17. Hu, C., Shi, W., Yao, W., Jiang, T., Tian, L., Chen, X., Li, W.: Adversarial infrared curves: An attack on infrared pedestrian detectors in the physical world. Neural networks 178, 106459 (2024) 18. Ilyas, A., Santurkar, S., Tsipras, D., Engstrom, L., Tran, B., Madry, A.: Adversarial examples are not bugs, they are features. Advances in neural information processing systems 32 (2019) 19. Kuckreja, K., Danish, M.S., Naseer, M., Das, A., Khan, S., Khan, F.S.: Geochat: Grounded large vision-language model for remote sensing. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. p. 27831– 27840 (2024) 20. Liu, C., Wang, H., Zhu, B., Ding, P., Zheng, Z., Xu, T., Han, Z., Wang, J.: Gen- erating transferrable adversarial examples via local mixing and logits optimization for remote sensing object recognition. arXiv preprint arXiv:2509.07495 (2025) 21. Liu, F., Chen, D., Guan, Z., Zhou, X., Zhu, J., Ye, Q., Fu, L., Zhou, J.: Remoteclip: A vision language foundation model for remote sensing. IEEE Transactions on Geoscience and Remote Sensing 62, 1–16 (2024) 22. Liu, H., Li, C., Li, Y., Li, B., Zhang, Y., Shen, S., Lee, Y.J.: Llavanext: Improved reasoning, ocr, and world knowledge (2024) 23. Liu, H., Li, C., Wu, Q., Lee, Y.J.: Visual instruction tuning. Advances in neural information processing systems 36, 34892–34916 (2023) 24. Lu, X., Wang, B., Zheng, X., Li, X.: Exploring models and data for remote sensing image caption generation. IEEE Transactions on Geoscience and Remote Sensing 56(4), 2183–2195 (2017) 25. Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A.: Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 (2017) 26. Mao, Z., Chimitt, N., Chan, S.H.: Accelerating atmospheric turbulence simulation via learned phase-to-space transform. In: Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision. p. 14759–14768 (2021) 27. Moosavi-Dezfooli, S.M., Fawzi, A., Fawzi, O., Frossard, P.: Universal adversarial perturbations. In: Proceedings of the IEEE conference on computer vision and pattern recognition. p. 1765–1773 (2017) 28. Pang, C., Weng, X., Wu, J., Li, J., Liu, Y., Sun, J., Li, W., Wang, S., Feng, L., Xia, G.S., et al.: Vhm: Versatile and honest vision language model for remote sensing image analysis. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 39, p. 6381–6388 (2025) 29. Papernot, N., McDaniel, P., Goodfellow, I., Jha, S., Celik, Z.B., Swami, A.: Prac- tical black-box attacks against machine learning. In: Proceedings of the 2017 ACM on Asia conference on computer and communications security. p. 506–519 (2017) 30. Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International conference on machine learning. p. 8748–8763. PmLR (2021) 31. Schlarmann, C., Hein, M.: On the adversarial robustness of multi-modal foundation models. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. p. 3677–3685 (2023) Thermal-Airflow Attack against IR Remote Sensing VLMs17 32. Schmalfuss, J., Mehl, L., Bruhn, A.: Distracting downpour: Adversarial weather attacks for motion estimation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. p. 10106–10116 (2023) 33. Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., Fer- gus, R.: Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 (2013) 34. Tiliwalidi, K., Hu, C., Lu, G., Jia, M., Shi, W.: Advgrid: a multi-view black- box attack on infrared pedestrian detectors in the physical world. Applied Soft Computing 174, 112981 (2025) 35. Wang, Z., Prabha, R., Huang, T., Wu, J., Rajagopal, R.: Skyscript: A large and semantically diverse vision-language dataset for remote sensing. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 38, p. 5805–5813 (2024) 36. Wei, H., Wang, Z., Jia, X., Zheng, Y., Tang, H., Satoh, S., Wang, Z.: Hotcold block: Fooling thermal infrared detectors with a novel wearable design. In: Proceedings of the AAAI conference on artificial intelligence. vol. 37, p. 15233–15241 (2023) 37. Wei, X., Yu, J., Huang, Y.: Physically adversarial infrared patches with learnable shapes and locations. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. p. 12334–12342 (2023) 38. Xie, C., Zhang, Z., Zhou, Y., Bai, S., Wang, J., Ren, Z., Yuille, A.L.: Improving transferability of adversarial examples with input diversity. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. p. 2730–2739 (2019) 39. Xie, P., Bie, Y., Mao, J., Song, Y., Wang, Y., Chen, H., Chen, K.: Chain of attack: On the robustness of vision-language models against transfer-based adversarial at- tacks. In: Proceedings of the Computer Vision and Pattern Recognition Conference. p. 14679–14689 (2025) 40. Xu, Y., Bai, T., Yu, W., Chang, S., Atkinson, P.M., Ghamisi, P.: Ai security for geoscience and remote sensing: Challenges and future trends. IEEE Geoscience and Remote Sensing Magazine 11(2), 60–85 (2023) 41. Xu, Y., Ghamisi, P.: Universal adversarial examples in remote sensing: Method- ology and benchmark. IEEE Transactions on Geoscience and Remote Sensing 60, 1–15 (2022) 42. Yang, J., Zhang, H., Li, F., Zou, X., Li, C., Gao, J.: Set-of-mark prompting un- leashes extraordinary visual grounding in gpt-4v. arXiv preprint arXiv:2310.11441 (2023) 43. Yuan, Z., Zhang, W., Fu, K., Li, X., Deng, C., Wang, H., Sun, X.: Exploring a fine- grained multiscale method for cross-modal remote sensing image retrieval. IEEE Transactions on Geoscience and Remote Sensing 60, 1–19 (2021) 44. Zhang, J., Ye, J., Ma, X., Li, Y., Yang, Y., Chen, Y., Sang, J., Yeung, D.Y.: Any- attack: Towards large-scale self-supervised adversarial attacks on vision-language models. In: Proceedings of the Computer Vision and Pattern Recognition Confer- ence. p. 19900–19909 (2025) 45. Zhang, J., Yi, Q., Sang, J.: Towards adversarial attack on vision-language pre- training models. In: Proceedings of the 30th ACM International Conference on Multimedia. p. 5005–5013 (2022) 46. Zhang, Z., Zhao, T., Guo, Y., Yin, J.: Rs5m and georsclip: A large-scale vision- language dataset and a large vision-language model for remote sensing. IEEE Transactions on Geoscience and Remote Sensing 62, 1–23 (2024) 47. Zhang, Z., Zhao, T., Guo, Y., Yin, J.: Rs5m and georsclip: A large-scale vision- language dataset and a large vision-language model for remote sensing. IEEE Transactions on Geoscience and Remote Sensing 62, 1–23 (2024) 18C. Su et al. 48. Zhao, Y., Pang, T., Du, C., Yang, X., Li, C., Cheung, N.M.M., Lin, M.: On eval- uating adversarial robustness of large vision-language models. Advances in Neural Information Processing Systems 36, 54111–54138 (2023) 49. Zhou, Z., Hu, S., Li, M., Zhang, H., Zhang, Y., Jin, H.: Advclip: Downstream- agnostic adversarial examples in multimodal contrastive learning. In: Proceedings of the 31st ACM International Conference on Multimedia. p. 6311–6320 (2023) 50. Zhu, X., Milanfar, P.: Removing atmospheric turbulence via space-invariant decon- volution. IEEE Transactions on Pattern Analysis and Machine Intelligence 35(1), 157–170 (2013) 51. Zhu, X., Hu, Z., Huang, S., Li, J., Hu, X.: Infrared invisible clothing: Hiding from in- frared detectors at multiple angles in real world. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. p. 13317–13326 (2022) 52. Zhuang, W., Xie, W., Zhang, Q., Du, X., Lin, Z., Lin, Z., Cai, H., Zhou, J., Fang, Z., Pun, C.m., et al.: Physically-induced atmospheric adversarial perturbations: Enhancing transferability and robustness in remote sensing image classification. arXiv preprint arXiv:2604.14643 (2026) Thermal-Airflow Attack against IR Remote Sensing VLMs19 A Detailed Method Formulation This appendix expands the components abbreviated in the method section of the main paper. We give the full perturbation parameterization (Appendix A.1), the airflow prior field and its spatial gate (Appendices A.2–A.3), the exact form of the airflow prior and confidence losses (Appendices A.4–A.5), and the combined objective and optimization details (Appendix A.6). Throughout, H × W is the IR input resolution, x∈R H×W a single-channel thermal image, and δ ∈R H×W the universal perturbation. Images are represented in the [0, 1] intensity range during optimization; when we write ε = 100 in tables, the corresponding value used in equations and code is 100/255. A.1 Generator Parameterization Rather than optimizing δ freely in pixel space, we parameterize it as a compo- sition of a fixed airflow prior field and a low-dimensional learnable residual, so that the optimizer explores only a compact, physically anchored subspace. Let P ∈R H×W denote the unit- normalized airflow prior (its construction is given in Appendix A.2) and G∈ [0, 1] H×W a spatial gate derived from it (Appendix A.3). The two learnable quantities are (i) a coarse latent map z ∈R d×d (d = 32) and (i) a scalar amplitude a = 0.85 + 0.30 sigmoid(ρ) with learnable logit ρ. The la- tent is decoded into a full-resolution residual by bicubic upsampling followed by a Gaussian blur B (kernel 9, σ = 2.5), then mean-removed and max-normalized: R = norm B(U (z)) ⊙G,norm(u) = u− mean(u) max i |u i − mean(u)| + ξ , ξ = 10 −6 . (4) The perturbation is then formed by an amplitude-scaled, tanh-squashed combi- nation of prior and residual, gated and projected to the L ∞ ball: δ = clip ε tanh aP + rR ⊙ G, −ε, ε ,(5) where ⊙ is elementwise product and r is the residual scale. Setting r = 0.60 lets the learnable residual reshape the prior substantially while the airflow term aP keeps the pattern physically anchored; this empirically balances attack strength against physical plausibility (see the main paper’s ablation study). Optimization thus updates only the latent z and the amplitude logit ρ—d 2 + 1 = 325 parame- ters in the reference configuration—rather than all HW pixels, which regularizes the perturbation toward smooth, spatially coherent airflow structure and shrinks the search space. A.2 Airflow Prior Field The airflow prior P is a single, fixed thermal-airflow field that encodes the spa- tially correlated temperature fluctuations produced when turbulent airflow mixes 20C. Su et al. air parcels of differing temperature. It is precomputed once as a signed single- channel field, resized to the model input resolution H×W, and unit-normalized by its peak magnitude: P = ̃ P max i | ̃ P i | ,(6) where ̃ P is the raw airflow field loaded from the precomputed asset. Because the prior is fixed throughout optimization and the perturbation is anchored to it through (5), the attack explores airflow-like patterns rather than arbitrary noise. This fixed-prior design is deliberately distinct from the phase-screen propagation model used for the atmospheric-turbulence baseline in the main paper, so that the baseline is an independent turbulence formulation rather than a weakened variant of our own prior. A.3 Spatial Gate To keep the perturbation concentrated on the structured regions of the prior and suppress it elsewhere, we derive a soft spatial gate G ∈ [0, 1] H×W from the normalized prior by soft-thresholding its magnitude at γ (gate threshold), blurring with a Gaussian kernel of size 21 and σ = 5.0, and optionally masking to a spatial region: G = norm max B clip( |P|−γ 1−γ , 0, 1) ⊙ Π ,(7) where B is the Gaussian blur, Π is a position mask (Π ≡ 1 for the default full-image setting; the spatial-position ablation varies it over the top, bottom, left, right, and center regions), and norm max rescales to a peak of one. The gate multiplies both the residual ((4)) and the final perturbation ((5)), so energy is spent only where the airflow prior is salient. A.4 Airflow Prior Loss The airflow prior loss keeps the optimized perturbation aligned with the prior in shape. Writing ˆu = u− mean(u) for a mean-removed field and δ g = δ⊙G for the gated perturbation, we use a cosine alignment between δ g and the prior P: L air = 1− ⟨ ˆ δ g , ˆ P⟩ (∥ ˆ δ g ∥ 2 + ξ)(∥ ˆ P∥ 2 + ξ) ,(8) which is one minus the spatial Pearson correlation between δ g and P and is invariant to the perturbation’s overall scale and offset. The airflow correlation reported in the ablations is exactly 1−L air at the converged perturbation. We also implemented a candidate frequency-domain priorL spec , defined as a Jensen– Shannon divergence between the ℓ 1 -normalized |FFT| 2 maps of δ g and P. This spectral term is reported only as an ablation/candidate component; it is not active in the final reference setting. Thermal-Airflow Attack against IR Remote Sensing VLMs21 A.5 Confidence Loss The active attack objective operates in the surrogate CLIP classifier induced by the scene-prompt set. Let f I (·) and f T (·) be the ℓ 2 - normalized image and text encoders, let sim(u,v) = f I (u) ⊤ f T (v), and let α be the CLIP logit scale used by the checkpoint. For a scene class c with prompt text π(c), the zero-shot logit is ℓ c (x,δ) = α sim f I (pre(x + δ)),f T (π(c)) ,(9) where pre(·) denotes the model’s native resizing and normalization after the image has been clipped to the valid intensity range. The clean class is the model’s own top-1 prediction y = arg max c ℓ c (x, 0), not an external annotation. We define c y = softmax(ℓ(x, 0)) y , p y (δ) = softmax(ℓ(x,δ)) y .(10) The reference attack minimizes the ratio p y (δ)/c y , so the optimized perturba- tion is explicitly driven to reduce the surrogate’s confidence in the clean top-1 decision. This choice matches the ASR definition used in evaluation: a success is a paired clean-top1 to adversarial-top1 flip on the same image. It is a prediction- stability attack and should not be read as a claim that the clean top-1 class is always the human ground-truth label. A.6 Combined Objective and Optimization The two headline losses of the main paper—an attack loss and the airflow prior loss—are implemented through a small set of candidate terms, all functions of the perturbation δ built by (5). The reference configuration activates the confidence-ratio term and the airflow prior term; the margin, diversity, spectral, smoothness, and amplitude terms are kept in the formulation because they are used in ablations or candidate variants, but their reference weights are zero. For a paired caption t + and batch negatives T − , let s t denote the image–text similarity to caption t under the adversarial image. The candidate terms are L conf = mean p y (δ)/c y ,(11) L margin = τ m mean softplus 1 τ m s t + − max t̸=t + s t + κ , (12) where minimizing L conf directly reduces the probability of the clean top-1 class defined in (10). The optional margin term drives the paired caption below the strongest in-batch competitor when its weight is nonzero (κ,τ m small constants). A candidate batch-diversity termL div (Jensen–Shannon divergence between the batch-mean adversarial prediction and the clean prior) is used only in ablations to test whether prediction collapse occurs. The active physical-plausibility term is the airflow prior loss L air (8) and its candidate spectral counterpart L spec (Appendix A.4). Two additional candidate regularizers control the residual and the amplitude: a smoothness term L smooth = TV(R) + 1 2 ∥∆R∥ (total variation plus Laplacian of the residual) and a mean-budget termL amp = relu ̄ |δ|/ε−μ 2 22C. Su et al. that penalizes mean perturbation magnitude above a budget μ when enabled. The full objective is L = w conf L conf + w margin L margin + w div L div + w air L air + w spec L spec + w smooth L smooth + w amp L amp , (13) with the reference weights stated here. In the final setting, w conf =8 and w air =1, while all other weights (w margin ,w div ,w spec ,w smooth ,w amp ) are set to zero. The loss-ablation study of the main paper corresponds to changing these weights one at a time (e.g. the no-air variant sets w air =0; attack-only keeps only the confidence-ratio term). Supplementary Algorithm A.1 gives the exact implemented update rule, re- fining the schematic procedure in the main paper. Algorithm A.1: Supplementary optimization details Input: Surrogate CLIP encoders f I ,f T ; IR image–caption pairs; airflow prior asset ̃ P; budget ε, steps T, rate η; gate threshold γ, residual scale r, blur B; loss weights w • Output: Universal perturbation δ ⋆ 1 P ← ̃ P/ max i | ̃ P i |// load and unit-normalize fixed prior 2 G← norm max B(clip( |P|−γ 1−γ , 0, 1))⊙ Π // spatial gate 3 z ←0∈R d×d ; ρ← 0// residual latent, amplitude logit 4 initialize AdamW over (z,ρ) (default betas/epsilon, weight decay 10 −4 ) 5 for i = 1...T do 6set learning rate by 30-step warmup followed by cosine decay 7 a← 0.85 + 0.30 sigmoid(ρ); R← norm(B(U(z)))⊙ G // decode residual 8 δ ← clip ε tanh(aP + rR)⊙ G, −ε,ε // form perturbation 9sample image batch; x adv ← clip(x + δ, 0, 1) before model preprocessing 10 L← w conf L conf + w margin L margin + w div L div 11+w air L air + w spec L spec + w smooth L smooth + w amp L amp // Eq. (13) 12back-propagate L; clip gradient norm to 1.0; AdamW step on (z,ρ) 13 end 14 a ⋆ ← 0.85 + 0.30 sigmoid(ρ); R ⋆ ← norm(B(U(z)))⊙ G 15 δ ⋆ ← clip ε tanh(a ⋆ P + rR ⋆ )⊙ G, −ε,ε 16 return δ ⋆ Optimization. The only trainable parameters are the residual latent z ∈ R d×d and the amplitude logit ρ (the prior P and gate G are fixed). We optimize them with AdamW (default betas and ε Adam , weight decay 10 −4 ) at learning rate η = 0.095 for T = 800 steps, with 30 warmup steps followed by cosine decay and gradient-norm clipping at 1.0; the perturbation is clipped to the L ∞ ball via the tanh and clip in (5) at every step, so ∥δ∥ ∞ ≤ ε holds by construction. Because z and ρ are input-agnostic and the batch gradient is averaged over images, the result is a single universal perturbation. At convergence we freeze (z ⋆ ,ρ ⋆ ) and read off the deployable δ ⋆ from (5); at attack time it is added to any IR input with no further optimization and no access to the target model, which is what makes the threat model gray-box. Thermal-Airflow Attack against IR Remote Sensing VLMs23 B Extended Discussion Why the attack transfers and which models are most exposed. The strong cross-model transfer across architectures with different pretraining data and encoders suggests that IR-trained models converge to similar representations of thermal patterns; the airflow perturbation exploits low-level, spatially corre- lated thermal-texture features that underlie IR scene understanding across mod- els, analogous to how high-frequency patterns transfer across RGB models [18] but with physical grounding that targets universally learned features. Notably, among the non-surrogate backbones the two remote-sensing-specialized models (RemoteCLIP-B32, GeoRSCLIP-B32) are the most vulnerable (50.6% and 51.7% ASR, second only to the 54.2% surrogate), suggesting that domain-specific train- ing enriches the very thermal-phenomenon representations the attack exploits. Among VLMs, the effect is concentrated on global scene identity—which de- grades significantly—while localized object presence is largely robust, indicating the perturbation corrupts the global scene representation rather than local ob- ject evidence. The IR-cue paradox. On two of six VLMs (most strikingly Qwen2.5-VL- 7B, 86.59→94.43), IR-cue detection accuracy rises under attack: the structured airflow pattern is interpreted as genuine thermal evidence (temperature gradi- ents, convection signatures), making the model more confident in an incorrect analysis. This failure mode has no direct analog in RGB attacks and under- scores the need for modality-specific robustness evaluation, though its model- dependence indicates it is not universal. Limitations. Our attack is evaluated in the digital domain; physical real- izability through controlled heat sources remains to be demonstrated, though the correlation with natural airflow patterns (0.893) suggests feasibility. We do not evaluate defenses, and our comparison targets IR-specific physical attacks rather than unstructured digital perturbations. Finally, the transfer split may share caption vocabulary with the optimization set, and whether the airflow prior’s higher plausibility confers a practical advantage (e.g., evading detectors or surviving input transformations) remains open—all directions we leave for future work. 24C. Su et al. C LLM-as-Judge Evaluation We additionally evaluate caption and VQA degradation with a multimodal LLM- as-Judge protocol. The judge is gpt-4o-mini through ChatAnywhere and receives the image, the clean answer, and the adversarial answer under the same task prompt. For each item, the raw attack score is the drop from the clean answer score to the adversarial answer score after clamping negative drops to zero. Because our goal is a subtle thermal-airflow disturbance rather than a visibly destructive artifact, we also report a perturbation-normalized efficiency score, Attack/MAE, computed as the common-clean attack score divided by the image mean absolute error. The raw score alone favors some stronger visible baselines, while the normalized score measures attack effect per unit image distortion and is therefore the metric most aligned with our setting. Table C.1: Overall LLM-as-Judge comparison on Qwen2.5-VL cases. Raw attack mea- sures answer degradation; Attack/MAE measures degradation per unit visible pertur- bation. Higher is stronger for Raw attack and Attack/MAE, while lower MAE and higher SSIM indicate better visual preservation. MethodRaw attackMAESSIMAttack/MAE Ours18.0000.0170.8911073.489 Thermal turbulence30.1040.0420.804718.355 IR stripe21.3750.0750.714284.575 Hot/cold block23.2090.0850.748273.726 Thermal drift16.8540.0750.788223.825 Table C.2: Task-wise LLM-as-Judge efficiency. Values are Attack/MAE; higher means stronger answer degradation per unit image distortion. MethodCaptionSceneObjectsIR cues Ours730.5691103.3081734.460725.619 Thermal turbulence584.630640.301930.635717.854 IR stripe187.493235.208519.224196.373 Hot/cold block243.751286.988341.053223.111 Thermal drift66.400323.143314.299191.458 Thermal-Airflow Attack against IR Remote Sensing VLMs25 D Reproducibility Details and Compute Budget This appendix records the practical details needed to reproduce our results: the software and hardware environment and its compute budget (Appendix D.1), data preparation and model checkpoints (Appendix D.2), the exact implemen- tation and hyperparameter configuration (Appendix D.3), the standard evalua- tion and statistical protocol (Appendix D.4), and the LLM-as-Judge evaluation (Appendix C). All values below are the reference configuration used through- out the main paper unless a per-experiment override is stated. Some wall-clock values are approximate because the raw GPU accounting logs were not retained for every exploratory run; when exact timestamps are unavailable, we report conservative estimates derived from the executed scripts and run logs. D.1 Compute Environment and GPU Budget The entire pipeline—surrogate fine-tuning, perturbation optimization, and all evaluations—runs on NVIDIA RTX 4090 (24 GB) GPUs. The compute footprint is modest by design: the attack is universal, so one perturbation is optimized once per surrogate and then applied without any per-image or per-target opti- mization, and the only trainable parameters are the 32×32 residual latent and a scalar amplitude (Appendix A.1), 325 values in total. Peak GPU memory during perturbation optimization stays below 22 GB, so no gradient checkpointing or model sharding is required, and a single 24 GB card is sufficient. To parallelize the five-backbone study, we place one CLIP model per GPU (five cards, indices 0–4) and optimize/evaluate them concurrently; this is a throughput convenience, not a requirement—each run is single-GPU. The software stack is PyTorch 2.5.1 with CUDA 12.4 on Python 3.12.3; the CLIP backbones and VLMs are loaded from the fixed infrared-domain model package listed in Appendix D.2. Table D.3 breaks down the wall-clock cost of each stage. The dominant total cost is VLM inference, because six generative models are queried over four tasks. The dominant train-time attack cost is optimizing the universal perturbation for T = 800 steps; because the result is input-agnostic, this cost is amortized over the entire test set and never repeated at attack time. VLM evaluation is inference- only (no gradients), and its cost is set by the number of generated tokens per task rather than by the attack. The total end-to-end budget to reproduce every number in the paper is approximately 60 GPU-hours. D.2 Dataset Construction and Model Checkpoints Infrared-sample filtering. The evaluation corpus is assembled from five public remote-sensing caption datasets: NWPU-Caption [8], RSICD [24], RSITMD [43], RS5M [47], and SkyScript [35]. All are filtered to infrared samples so that no RGB information leaks into the thermal evaluation. A sample is retained only if its filename or metadata carries an explicit IR indicator and carries no RGB indicator; ambiguous samples are discarded rather than guessed. After filtering, 26C. Su et al. Table D.3: Compute budget on RTX 4090 (24 GB) GPUs. Perturbation optimiza- tion is a one-time train-time cost amortized over all downstream evaluations; inference stages carry no optimization. Wall-clock values are approximate when complete sched- uler logs were unavailable. StageType Runs Wall-clock GPU-hours Surrogate IR fine-tuning (OpenAI-CLIP-B32)train1 ≈25 min ≈0.4 UAP optimization (T=800 steps)train1 4–6 min ≈0.1 CLIP zero-shot eval (5 backbones, 9,720 samples) inference 5 ≈15 min ≈0.5 Cross-model transfer probe (5 backbones, 1,000-pool) inference 5 ≈5 min ≈0.1 VLM eval (6 models, 4 tasks, 1,000 samples)inference 6 ≈10–12 h ≈50 Ablations (loss/position/hyperparameter sweeps) train+infer 94 ≈2 h ≈8 Total≈60 the test split contains 10,000 IR image–text pairs, of which 9,720 carry a remote- sensing scene-category label and form the CLIP zero-shot classification set; the validation split used for all ablations contains 416 samples. The VLM study uses a separate 1,000-image diagnostic pool annotated for the four downstream tasks. This diagnostic pool is not used for surrogate fine-tuning, perturbation optimization, or hyperparameter selection. Checkpoints. All target models are evaluated from fixed infrared-domain checkpoints; during attack optimization, only the surrogate OpenAI-CLIP-B32 checkpoint is accessed. The exact checkpoint path for each of the five CLIP backbones and six VLMs is recorded in the model-package manifest file, which pins the reported numbers to a fixed set of weights. D.3 Implementation Details Table D.4 consolidates every hyperparameter needed to reproduce the reference configuration; all values match the main paper and the equations of Appendix A. The perturbation is the prior-plus-residual parameterization of Appendix A.1 (latent d=32, residual scale r=0.60); the airflow prior of Appendix A.2 is loaded once and held fixed throughout optimization. Unless noted, each ablation sweeps a single hyperparameter with all others held at these reference values. D.4 Evaluation Protocol CLIP. Each backbone acts as a zero-shot scene classifier: an IR image is assigned the scene category whose text prompt maximizes image–text cosine similarity. ASR is the paired top-1 flip rate—the fraction of samples whose adversarial top-1 class differs from the clean top-1 class on the same image. This is a prediction- stability ASR: it measures whether the attack changes the model’s own clean decision, and it does not assume that the clean top-1 class is always the human ground-truth label. For labeled scene experiments, the evaluation logs retain clean top-1, adversarial top-1, and the annotated label so that clean accuracy Thermal-Airflow Attack against IR Remote Sensing VLMs27 Table D.4: Reference hyperparameters (the ε=100, full-position configuration). All ablations vary one row at a time with the rest held fixed. The IR input resolution follows each backbone’s native CLIP preprocessing. SymbolMeaningValue εL ∞ budget (of 255)100 Toptimization steps800 η learning rate (AdamW, wd 10 −4 )0.095 –warmup / schedule30 steps / cosine –gradient-norm clip1.0 dresidual latent size (d×d)32 rresidual scale0.60 –residual blur kernel / σ9 / 2.5 γgate threshold0.01 μmean-magnitude budget0.16 w conf confidence-ratio weight8.0 w air airflow prior weight1.0 –optimizerAdamW (betas/epsilon default) –batch size (L14 / others)64 / 256 –IR input resolution H×Wnative (e.g. 224×224) –random seedfixed and robust accuracy can be audited separately from flip ASR. The cross-model transfer study of the main paper instead applies one UAP to held-out samples and measures nearest-caption retrieval flips over a 1,000-sample candidate pool, which is a strictly more sensitive probe and therefore reports higher numbers that are not comparable to ASR. VLMs. We report ROUGE-L for captioning, scene accuracy, object F1, and IR-cue accuracy, each on the full 1,000-sample set. Because clean and attacked outputs are paired on the same images, binary scene and IR-cue metrics are reported with Wilson confidence intervals and paired tests when per-sample correctness labels are available. ROUGE-L and object F1 are continuous or set- valued scores, so we treat their statistical comparisons as descriptive unless a paired bootstrap or permutation test is explicitly reported. The main claims are therefore based on effect sizes that are consistent across models, not on unpaired significance tests. The exact task prompts issued to each VLM are defined verba- tim in the evaluation script vlm_eval_generate.py so that generation conditions can be reproduced. D.5 Reproducibility Statement The appendix specifies the full method (Appendix A), every reference hyperpa- rameter (Table D.4), the data preparation procedure (Appendix D.2), the stan- dard evaluation and statistical protocol (Appendix D.4), and the LLM-as-Judge protocol (Appendix C), which together determine all reported numbers. The 28C. Su et al. implementation is organized around explicit split files, model manifests, pertur- bation checkpoints, and evaluation scripts, so the experiment can be reproduced from the public source datasets and the fixed infrared-domain model package described above. All attack optimization runs are single-GPU jobs; multi-GPU execution is used only to run independent models in parallel. Thermal-Airflow Attack against IR Remote Sensing VLMs29 E Additional Qualitative Diagnostics The quantitative results in the main paper measure whether AirflowAttack changes model predictions, but they do not by themselves show whether the re- sulting images remain meaningful infrared remote-sensing samples. We therefore perform a paired qualitative audit on clean images, perturbation-only visualiza- tions, adversarial images, CLIP prediction changes, VLM outputs, and attention maps. The purpose of this audit is not to introduce a new metric, but to ver- ify that the reported ASR is obtained by a structured thermal-airflow pattern rather than by destroying the scene content. E.1 Perturbation Visibility The main experiments use ε = 100 on the 8-bit intensity scale. This number should be interpreted as an L ∞ upper bound rather than as the typical pixel change: the perturbation is produced by a fixed airflow prior, a blurred low- dimensional residual, and a soft spatial gate, so its visible energy is spatially coherent instead of independently saturating all pixels. In the reference OpenAI- CLIP-B32 run, the selected perturbation has mean absolute magnitude about 3.4/255 and peak magnitude about 87.6/255 after gating, showing that the nom- inal budget is used only locally. In the visual examples, the adversarial image preserves the large-scale scene layout, object boundaries, and thermal contrast structure of the clean image, while the added pattern appears as a weak airflow- like texture. This is the reason we use ε = 100 as the reference setting rather than the higher budgets in the strength ablation: larger budgets further increase attack success, but they also make the perturbation less plausible as a natural thermal disturbance. E.2 Prediction-Level Consistency Checks For every qualitative CLIP case, we inspect the clean top-1 class and the adver- sarial top-1 class on the same image. This paired design is important because the clean label may differ from the human semantic label in some remote-sensing scenes; ASR is therefore counted only as a clean-to-adversarial top-1 flip, not as disagreement with an external annotation. Successful cases typically preserve the human-recognizable scene while changing the CLIP decision to a visually incompatible category. The attention maps in the main paper show the same mechanism spatially: the clean model concentrates on scene-defining regions, whereas the adversarial model response is pulled toward the structured thermal texture, producing the top-1 flip without requiring object removal or geometric distortion. E.3 Caption and VQA Failure Modes The VLM examples reveal a different failure pattern from the CLIP classifier. In captioning, attacked outputs often become more generic and less tied to the 30C. Su et al. actual scene, which explains why ROUGE-L only partially reflects the qualitative degradation: a generic caption can still share common words with the reference caption. In VQA, the strongest and most consistent effect is on scene identity. Object-level answers are more stable because localized high-contrast structures often remain visible after the airflow perturbation. Infrared-cue questions show a modality-specific failure: some VLMs interpret the artificial airflow texture as genuine thermal evidence, such as temperature gradients or convection-like patterns, and therefore answer with greater confidence in an incorrect thermal interpretation. This behavior supports the claim that IR-specific perturbations can exploit thermal semantics rather than merely adding generic visual noise. E.4 Failure Cases The attack is weakest when the clean image contains a dominant, high-contrast object or a simple scene layout whose semantic evidence is spatially localized. In these cases, the airflow perturbation may change the model confidence without changing the top-1 class. It is also less effective on scenes whose clean prediction margin is large, especially for the larger ViT-L/14 backbone, where a substantial confidence drop does not always translate into a classification flip. These fail- ure cases are consistent with the quantitative results: AirflowAttack primarily disrupts global thermal texture and scene-level representations, while localized object evidence is comparatively more robust. We therefore frame the method as a universal scene-understanding attack for infrared remote sensing, not as a guaranteed object-removal or object-detection attack. Thermal-Airflow Attack against IR Remote Sensing VLMs31 F Dataset Statistics and Sampling Details This appendix gives the concrete split statistics used by the experiments. The infrared corpus is built from NWPU-Caption, RSICD, RSITMD, RS5M, and SkyScript. Each retained sample has an infrared-style image and an IR-aware caption; samples with RGB indicators are filtered out before any training or eval- uation. The surrogate fine-tuning split contains 48,616 image–caption pairs, the ablation validation split contains 416 pairs, and the held-out test split contains 10,000 pairs. For the VLM transfer study and the short-class diagnostic classi- fier outputs, we additionally use a 1,000-image balanced diagnostic subset with explicit source and scene labels. This subset is a held-out diagnostic pool: it is used for transfer evaluation and qualitative analysis, not for surrogate training, loss selection, or hyperparameter tuning. The held-out test split is intentionally RS5M-heavy because it follows the available infrared sample pool after filtering. The diagnostic subset is more bal- anced across sources and is therefore used for qualitative inspection, downstream VLM transfer, and source-wise analysis. Its labels cover 169 unique scene names; the most frequent labels are bridge, airplane, airport, river, farmland, parking, industrial, harbor, tennis, and resort/beach, each appearing between 21 and 34 times. We use this diagnostic subset only for secondary analysis; the main CLIP ASR comparison and VLM tables remain the primary reported results. Table F.5: Infrared split statistics by source dataset. The diagnostic 1,000 subset is the labeled subset used for VLM transfer inputs and the per-source diagnostics in Appendix L. SplitNWPU RSICD RSITMD RS5M SkyScript Total Surrogate train2,190 711276 40,580 4,859 48,616 Ablation validation47003690416 Held-out test515 164878,138 1,096 10,000 Diagnostic/VLM subset 205 16683332214 1,000 32C. Su et al. G Baseline Implementation Details All physical baselines are implemented as image-space infrared perturbations and are applied once to each clean IR image, with no gradient access to the target model and no per-image target optimization. Random fields are gener- ated with the same fixed seed used by our method. The perturbation is always clipped to the same valid image range as AirflowAttack, and the final adversar- ial image is clipped to [0, 1]. For the CLIP comparison table, all methods are constrained by the same nominal L ∞ budget, so the reported ASR values are directly comparable. Atmospheric thermal turbulence. The turbulence baseline uses a low- frequency two-channel displacement field and a separate low-frequency scintilla- tion field. The displacement field is decoded from a 14×14 latent map and scaled to a maximum displacement of approximately 5.5 pixels before bilinear resam- pling with reflection padding. The scintillation field is decoded from a 10×10 latent map and applied multiplicatively with strength 0.065. A Gaussian blur with kernel size 5 and σ = 1.0 is then applied to emulate the local smoothness of atmospheric distortion. IR fixed-pattern stripe noise. The stripe baseline simulates sensor read- out artifacts by combining a column-wise low-frequency field and a weaker row- wise field. The column component receives weight 0.75 and the row component weight 0.25; the combined pattern is peak-normalized and added with ampli- tude 0.85ε. This produces coherent vertical banding rather than independent pixel noise. Thermal hot/cold blocks. The hot/cold baseline overlays seven smooth signed thermal blobs. Each blob has a random center in the central image region, random horizontal and vertical scales in the range 0.045–0.115 of the image width/height, and a random hot or cold sign. The summed pattern is blurred with a Gaussian kernel of size 11 and σ = 2.5, mean-removed, peak-normalized, and applied with amplitude 0.95ε. Thermal sensor drift/non-uniformity. The sensor-drift baseline decodes a smooth 8×8 low-frequency field and uses it as both multiplicative gain and additive offset. Specifically, the clean image is transformed as x· (1 + 0.18F ) + 0.35εF, where F is the normalized drift field. This approximates non-uniform detector gain, offset drift, and slow thermal calibration error. Thermal-Airflow Attack against IR Remote Sensing VLMs33 H Qualitative Case Gallery Following the table-style appendix layout used in the reference paper, we give additional qualitative cases in which each row pairs the clean image, the adver- sarial image, the clean model output, and the attacked output. The goal is not to introduce new metrics, but to make the paired failure mode visible: the scene remains recognizable to a human observer, while the model output shifts toward a different scene or a more generic thermal description. Table H.6: Qualitative VLM case group 1. Each row shows the clean input, adversarial input, and concise paired model outputs. CaseClean image Adv. imageClean outputAdv. outputShift type case_01 NWPU Low-texture bare land with faint structural outlines and scattered trees. High-contrast urban scene with buildings, roads, bright spots, and low-texture regions. Bare land to city. case_02 SkyScript Beach and water separated by a clear shoreline boundary. Urban scene with structural outlines, river-like smooth gradients, and buildings. Natural scene to urban. case_03 SkyScript Buildings with distinct shapes adjacent to railway-station-like regions. Residential area with houses, streets, and reflective open spaces. Specific structure to generic residential. Table H.7: Qualitative VLM case group 2. CaseClean image Adv. imageClean outputAdv. outputShift type case_04 SkyScript Tennis court with dark boundary lines and a smooth playing surface. Tennis court remains, but the response over-emphasizes thermal intensity patterns around it. Cue distortion. case_05 NWPU Mobile-home park with roads and surrounding darker tree regions. Residential area with parked cars, roads, buildings, and grid-like layout. Fine label becomes generic. case_06 RSICD Residential neighborhood with houses, roads, and dense tree cover. Residential area with an added tennis-court-like low-texture region. Object hallucination. 34C. Su et al. Table H.8: Qualitative VLM case group 3. CaseClean image Adv. imageClean outputAdv. outputShift type case_07 NWPU buildings, roads, cars, trees tennis courts, buildings, roads, trees Object answer injects a court cue. case_08 RSICD Dark lake scene. City-like aerial scene with buildings, road, and trees. Scene label changes. case_09 RSITMD Residential or industrial structures with roads and trees. Urban scene with buildings, roads, trees, and water-body terms. Semantic broadening. Table H.9: Qualitative VLM case group 4. CaseClean image Adv. imageClean outputAdv. outputShift type case_10 RSICD Playground with bright low-texture surface and distinct structural outlines. Residential area with trees, buildings, roads, parking lot, and tennis court. Scene and object drift. case_11 RS5M Ocean with varying intensities and a boat as a bright structure. Beach/ocean or land-water contrast; the boat focus is weakened. Key object weakened. case_12 RS5M Beach and ocean with smooth low-texture water surface. Dense urban structures and harbor-like high-contrast outlines. Coast to urban harbor. Thermal-Airflow Attack against IR Remote Sensing VLMs35 I Extended Attention-Map Visualization We also follow the reference appendix’s CAM-style visualization protocol and include expanded attention examples in table form. For each clean/adversarial pair, the table lists the model’s top-1 scene prediction and shows both the original image and the layer-12 attention map. The comparison should be read as a paired change in model grounding, not as a human segmentation mask. Table I.10: Extended layer-12 attention-map case group 1. Each row includes clean and adversarial raw images together with their attention maps. CaseCleanAdv. Clean heatmap Adv. heatmap Clean top-1Adv. top-1Attention behavior 0538 RSICD stadiumparking lots Field evidence weakens and shifts to parking-like texture. 0504 RS5M parkchurch Diffuse land evidence is redirected to compact bright regions. 0507 RS5M beachbeach Label stays stable, but confidence and grounding are suppressed. 0516 RSICD rectangular farmland tall buildings Low-frequency field evidence becomes building-like. Table I.11: Extended layer-12 attention-map case group 2. CaseCleanAdv. Clean heatmap Adv. heatmap Clean top-1Adv. top-1Attention behavior 0377 NWPU white roadwhite road Road geometry remains, while confidence is reduced. 0700 RS5M beachtall buildings Water/shore evidence is redirected to compact structures. 0412 RSICD white roofschurch Roof-like evidence is pulled toward object-like responses. 0325 SkyScript golf coursetall buildings Smooth course texture is replaced by building-like evidence. 36C. Su et al. Table I.12: Extended layer-12 attention-map case group 3. CaseCleanAdv. Clean heatmap Adv. heatmap Clean top-1Adv. top-1Attention behavior 0020 RS5M stadium ground track field Sports-field attention moves to a neighboring field category. 0281 RSICD baseball diamond tall buildings Court-like geometry loses dominance to structural texture. 0650 RS5M pondtall buildings Low-texture water evidence changes to clustered structures. 0953 SkyScript white roofswhite roofs Repeated roof cues stay stable under perturbation. Table I.13: Extended layer-12 attention-map case group 4. CaseCleanAdv. Clean heatmap Adv. heatmap Clean top-1Adv. top-1Attention behavior 0914 RS5M cloudchurch Cloud-like texture is pulled toward compact structural evidence. 0227 RS5M islandharbor Coastline grounding shifts toward harbor-like regions. 0554 RS5M mountaintall buildings Terrain texture becomes high-response structure. 0829 RS5M baseball diamond stadium Field geometry is redirected inside sports-scene semantics. Thermal-Airflow Attack against IR Remote Sensing VLMs37 J Additional Attack Samples and Boundary Cases The previous gallery focuses on successful semantic shifts. This section adds two complementary views: (i) more paired clean/adversarial examples and (i) boundary cases where the attack is weaker. The latter is important because the perturbation is universal: it cannot adapt to a specific image, so scenes with strong geometric anchors are more resistant. Table J.14: Additional attack sample group 1: boundary-style cases with stable or weakly changed top-1 labels. Case Clean image Adv. imageClean top-1Adv. top-1Reading 0507 RS5M beach (0.502)beach (0.108) The class remains beach, but confidence drops sharply after perturbation. 0377 NWPU white road (0.140) white road (0.046) The freeway geometry is stable, so the perturbation mainly suppresses confidence. 0953 SkyScript white roofs (0.079) white roofs (0.074) Repeated roof-like structures remain the dominant visual anchor. 0658 RS5M harbor (0.209)harbor (0.213) Large water and harbor cues remain inside the same semantic neighborhood. 38C. Su et al. Table J.15: Additional attack sample group 2: confidence suppression with nearby scene semantics. Case Clean image Adv. imageClean top-1Adv. top-1Reading 0403 RS5M beach (0.414)beach (0.058) The top-1 remains beach, but the clean confidence is strongly reduced. 0018 RS5M port (0.439)port (0.146) Port layout survives, but the adversarial input reduces the clean confidence. 0102 RSICD grey river (0.187) river (0.087) The prediction stays in the same river neighborhood with lower confidence. 0227 RS5M island (0.631)harbor (0.092) Coastline evidence shifts to a nearby harbor interpretation. Table J.16: Additional attack sample group 3: clear semantic flips. Case Clean image Adv. imageClean top-1Adv. top-1Reading 0538 RSICD stadium (0.200) parking lots (0.060) Sports-field evidence is replaced by parking-lot texture. 0516 RSICD rectangular farmland (0.090) tall buildings (0.186) Smooth field-like evidence turns into building-like response. 0700 RS5M beach (0.253) tall buildings (0.562) Shoreline evidence is overtaken by high-response structural texture. 0412 RSICD white roofs (0.061) church (0.207) Roof-like evidence is redirected to an object-like class. Thermal-Airflow Attack against IR Remote Sensing VLMs39 Table J.17: Additional attack sample group 4: source-diverse semantic flips. Case Clean image Adv. imageClean top-1Adv. top-1Reading 0447 NWPU airplane (0.036) church (0.242) Sparse desert cues are replaced by compact bright structures. 0990 SkyScript rectangular farmland (0.037) tall buildings (0.495) Field-like response flips to high-confidence building semantics. 0271 SkyScript square (0.128)church (0.379) Compact geometry is reinterpreted as an object-like landmark. 0801 NWPU sparse residential (0.035) tall buildings (0.509) Residential evidence is amplified into tall-building semantics. Table J.18: Representative boundary cases. These classes are not failure modes of the implementation; rather, they identify scene types where the universal perturbation has less room to override strong geometric evidence. Class or scene type n ASR (%)Interpretation Dense residential 19 10.5 Repeated roofs and block layouts provide strong geometric anchors, so the airflow texture often changes confidence without flipping the top-1 class. Storage tanks 8 12.5 Circular tank geometry is visually distinctive and remains stable under the smooth perturbation. Parking lot8 25.0 Vehicle grids and regular lane structure remain salient in many examples. Forest19 31.6 Texture changes can alter confidence, but the global low-contrast vegetation pattern often remains inside the same semantic neighborhood. Tennis court8 37.5 Court-line geometry is compact and high-contrast, making it harder for a universal thermal field to erase completely. 40C. Su et al. K Full VLM Prompts and Scoring Protocol The downstream VLM evaluation uses the same four prompts for all six models and all attack methods. Captioning is allowed 128 new tokens, while the three question-answering tasks are allowed 64 new tokens. Generation is determinis- tic with greedy decoding (do_sample=False, num_beams=1) and each model’s native preprocessing and adapter defaults. The prompts are deliberately short and task-specific so that changes under attack reflect image-conditioned model behavior rather than prompt engineering. Table K.19 lists the exact prompts. For captioning, we report ROUGE-L against the IR-aware reference caption. The scoring script also records BLEU, METEOR, and a lightweight CIDEr-style score, but ROUGE-L is used in the paper because it is stable across the six eval- uated VLMs and directly comparable across methods. For scene recognition, the generated answer is normalized and matched to the annotated scene label. For object recognition, the answer is parsed as a set of object or land-cover terms and compared with the reference object set using F1. For IR-cue eval- uation, the answer is checked against the expected infrared visual cues rather than against a single class label. The IR-cue prompt is an evaluation prompt designed to elicit thermal-cue descriptions; the metric should therefore be read as cue-fidelity under a fixed rubric, not as an unconstrained test of whether a model independently discovers thermal evidence. We report all clean and adver- sarial scores on the same 1,000 images and use the statistical protocol described in Appendix D.4. Table K.19: VLM prompts and reported metrics. Lower ROUGE-L, scene accuracy, and object F1 indicate stronger degradation; IR-cue accuracy is interpreted separately because attacked models may hallucinate plausible thermal cues. TaskPromptMax tokensMetric CaptionDescribe this infrared remote sensing image in detail.128ROUGE-L Scene What is the main scene type in this infrared remote sensing image? Answer with a concise scene label only. 64Scene accuracy Objects What are the main objects or land-cover elements visible in this infrared remote sensing image? Answer with a concise comma-separated list. 64Object F1 IR cues What infrared-specific visual cues can be observed in this image? Mention visible grayscale intensity, high contrast, bright structures, low-texture or dark regions, and structural outlines if present. 64IR-cue accuracy Thermal-Airflow Attack against IR Remote Sensing VLMs41 L Per-Source and Per-Class Diagnostic Results The main paper reports aggregate ASR across models. To test whether the attack is driven by a single source dataset, we also compute source-wise ASR on the 1,000-image diagnostic split. This analysis is secondary because the diagnostic split is smaller than the full test set, but it is useful for identifying systematic source effects. Table L.20: Source-wise ASR (%) on the 1,000-image diagnostic split. The same images are used for all five CLIP backbones. Source n OpenAI-L14 OpenAI-B32 OpenCLIP-B32 RemoteCLIP-B32 GeoRSCLIP-B32 Mean NWPU 20545.460.046.353.753.751.8 RS5M 33241.355.447.054.845.548.8 RSICD 16647.053.650.047.656.651.0 RSITMD 8348.257.859.050.659.054.9 SkyScript 21463.671.062.165.072.966.9 The attack transfers across all five sources, with SkyScript showing the largest mean ASR and RS5M the smallest. Because the perturbation is universal and not source-conditioned, this gap most likely reflects differences in image com- position and caption-derived labels rather than a dataset-specific fitting effect. The source-wise trend is consistent across backbones: all models are substantially affected on every source, and no source collapses to near-zero attack success. We further inspect class-level behavior on the OpenAI-CLIP-B32 diagnostic classifier output, using only classes with at least five samples after merging mi- nor spelling variants such as underscore-separated and space-separated labels. Table L.22 reports representative high- and low-ASR classes. Classes dominated by broad thermal texture or weak local structure, such as desert, cloud, beach, playground, and water, are especially vulnerable. Classes with repeated high- contrast man-made structure, such as dense residential areas, storage tanks, parking lots, and tennis courts, are more resistant. This supports the interpreta- tion that AirflowAttack mainly perturbs global thermal texture and scene-level evidence, while compact geometric structure can still anchor the model predic- tion. Table L.22: Representative high- and low-ASR classes on OpenAI-CLIP-B32 in the diagnostic split (classes with ≥ 5 samples). Group MetricClasses High Classdesert cloud beach play- ground freeway school stadium mountain n96118762013 ASR (%) 100.0 100.0 90.9 87.585.783.380.076.9 Low Class dense resid. storage tanks palace parking lot forest medium resid. tennis court parking n 198581914828 ASR (%)10.5 12.5 20.0 25.031.635.737.539.3