Paper deep dive
MMSkillRisk: Can Agents Stay Safe When Multimodal Skills Become Traps?
Lingqi Jiang, Jialuo Chen, Jianan Ma, Xinhao Deng, Xiaohu Du, Sibo Yi, Yuqi Qing, Zhenguang Liu, Qinming He, Shiwen Cui, Changhua Men
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 9/30/2026, 7:12:15 AM
Summary
The paper introduces MMSkillRisk, a benchmark for evaluating the safety of multimodal agent skills against image-borne attacks. It proposes Native-Context Visual Attack (NCVA), where malicious instructions are disguised as visual elements in teaching images. The benchmark consists of 108 executable cases derived from 28 clean skills and 36 attack packages. Experiments across nine model-harness configurations show that NCVA achieves a 43.1% attack success rate, significantly outperforming text-based baselines, and frequently co-occurs with legitimate task completion, highlighting a critical security gap.
Entities (14)
Relation Signals (12)
MMSkillRisk ā usesattackmethod ā Native-Context Visual Attack
confidence 98% Ā· we design Native-Context Visual Attack (NCVA)... Built from 28 curated clean skills, MMSkillRisk contains 36 attack packages
Native-Context Visual Attack ā achievesasr ā 43.1%
confidence 95% Ā· Its pooled attack success rate (ASR) reaches 43.1%
MMSkillRisk ā containsattackobjectives ā Artifact Pollution
confidence 95% Ā· spanning five attack objectives: ... artifact pollution ...
MMSkillRisk ā containsattackobjectives ā Data Exfiltration
confidence 95% Ā· spanning five attack objectives: data exfiltration...
MMSkillRisk ā containsattackobjectives ā Privilege Escalation
confidence 95% Ā· spanning five attack objectives: ... privilege escalation ...
MMSkillRisk ā containsattackobjectives ā Persistence
confidence 95% Ā· spanning five attack objectives: ... persistence ...
MMSkillRisk ā containsattackobjectives ā Integrity Destruction
confidence 95% Ā· spanning five attack objectives: ... integrity destruction.
Native-Context Visual Attack ā targets ā Multimodal Skills
confidence 95% Ā· MMSkillRisk... dedicated to end-to-end safety evaluation of image-borne attacks in multimodal skills.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Agent skills are shareable packages of procedural instructions, tools, and examples. Multimodal skills additionally include visual references that agents retrieve and inspect during execution. Because these images guide actions, attackers can disguise malicious instructions as ordinary visual guidance within otherwise legitimate skills. Existing skill-security research primarily examines text-carried attacks or scanner detection, leaving the runtime effects of image-borne attacks insufficiently evaluated. We introduce MMSkillRisk, to our knowledge the first publicly available benchmark dedicated to end-to-end safety evaluation of image-borne attacks in multimodal skills. To instantiate this attack surface, we design Native-Context Visual Attack (NCVA), which disguises malicious instructions as native components of teaching images, such as annotations and interface labels. The accompanying this http URL provides auxiliary guidance toward relevant visual regions without explicitly stating the malicious operation. Built from 28 curated clean skills, MMSkillRisk contains 36 attack packages and 108 executable cases spanning five attack objectives, with separate checks for attack success and legitimate-task completion. Across nine model-harness configurations evaluated in isolated sandboxes, NCVA induces unauthorized operations in every configuration. Its pooled attack success rate (ASR) reaches 43.1%, exceeding the matched text-carrier baseline by 16.4 percentage points, with higher ASR in all nine configurations. Attack success and legitimate-task completion co-occur in 36.5% of cases, reaching 72.2% for GPT-5.6-sol with Codex. These results show that skill-bundled images can induce unauthorized actions even as agents complete legitimate tasks, so task success alone does not establish safe skill use. Our code and data are available at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2609.35912v1
- Canonical: https://arxiv.org/abs/2609.35912v1
Trouble viewing inline? Open PDF directly ā
Full Text
66,062 characters extracted from source content.
Expand or collapse full text
MMSKILLRISK: CAN AGENTS STAY SAFE WHEN MULTIMODAL SKILLS BECOME TRAPS? Lingqi Jiang 1,2 Jialuo Chen 1,2,ā Jianan Ma 2,3 Xinhao Deng 2,4 Xiaohu Du 2 Sibo Yi 4 Yuqi Qing 4 Zhenguang Liu 1,ā Qinming He 1 Shiwen Cui 2 Changhua Meng 2 1 Zhejiang University 2 Ant Group 3 Hangzhou Dianzi University 4 Tsinghua University ā Corresponding authors. ABSTRACT Agent skills are shareable packages of procedural instructions, tools, and ex- amples. Multimodal skills additionally include visual references that agents re- trieve and inspect during execution. Because these images guide actions, at- tackers can disguise malicious instructions as ordinary visual guidance within otherwise legitimate skills. Existing skill-security research primarily examines text-carried attacks or scanner detection, leaving the runtime effects of image- borne attacks insufficiently evaluated. We introduce MMSKILLRISK, to our knowledge the first publicly available benchmark dedicated to end-to-end safety evaluation of image-borne attacks in multimodal skills. To instantiate this at- tack surface, we design Native-Context Visual Attack (NCVA), which disguises malicious instructions as native components of teaching images, such as an- notations and interface labels. The accompanying SKILL.md provides auxil- iary guidance toward relevant visual regions without explicitly stating the ma- licious operation. Built from 28 curated clean skills, MMSKILLRISK contains 36 attack packages and 108 executable cases spanning five attack objectives, with separate checks for attack success and legitimate-task completion. Across nine modelāharness configurations evaluated in isolated sandboxes, NCVA in- duces unauthorized operations in every configuration. Its pooled attack success rate (ASR) reaches 43.1%, exceeding the matched text-carrier baseline by 16.4 percentage points, with higher ASR in all nine configurations. Attack success and legitimate-task completion co-occur in 36.5% of cases, reaching 72.2% for GPT-5.6-sol with Codex. These results show that skill-bundled images can in- duce unauthorized actions even as agents complete legitimate tasks, so task suc- cess alone does not establish safe skill use. Our code and data are available at https://github.com/kaill-jlq/MMSkillRisk. 1INTRODUCTION Language-model agents increasingly rely on reusable skills: portable packages of procedural in- structions, tools, scripts, and examples that can be shared through open marketplaces and installed to provide specialized capabilities at inference time without retraining (Li et al., 2026; Liu et al., 2026c; Xu & Yan, 2026). This convenience also introduces a supply-chain risk: third-party skills installed without careful review can give attackers a foothold to influence agent planning and exe- cution (Schmotz et al., 2026; Jin et al., 2026; Liu et al., 2026a). Textual procedures alone, however, cannot always faithfully capture task-relevant information such as chart details, interface geometry, spatial relations, and observable state changes (Xu et al., 2026; Jiang et al., 2026c). A growing line of multimodal skill systems therefore supplements textual instructions with screenshots, keyframes, and other visual references (Zhang et al., 2026; Jiang et al., 2026c; Xu et al., 2026; Fan et al., 2026). These images are not merely illustrative: agents retrieve and inspect them during execution to determine where to act, when a step applies, and how to interpret an operationās outcome. Their contents can therefore become a basis for action, creating 1 arXiv:2609.35912v1 [cs.CR] 28 Sep 2026 an opportunity for attackers to disguise malicious instructions as ordinary visual guidance within an otherwise legitimate skill. Despite this execution-relevant role of visual resources, existing skill-security benchmarks primar- ily evaluate attacks carried by textual instructions, scripts, and manifests (Schmotz et al., 2026; Jin et al., 2026; Liu et al., 2026a; Zhuang et al., 2026; Jiang et al., 2026b). The risks introduced by images bundled with skills consequently remain underexplored. Although Jia et al. (2026b) exam- ines instructions hidden in skill images, its focus is on detection by security scanners rather than their effects on agents executing legitimate tasks. Scanner evasion alone does not establish whether an agent will follow the malicious content and carry out an unauthorized operation. Conversely, successful task completion does not establish safety, because an agent may fulfill the userās request while also performing attacker-directed actions. What is missing is an end-to-end benchmark that makes image-borne skill attacks executable and measures their security consequences separately from legitimate-task outcomes. To address this gap, we introduce MMSKILLRISK, to our knowledge the first publicly available benchmark dedicated to end-to-end safety evaluation of image-borne attacks in multimodal skills. To systematically instantiate this attack surface, we design Native-Context Visual Attack (NCVA). Rather than appending a visibly separate malicious instruction to an image, NCVA disguises it as a native component of the teaching material, such as an annotation, or an interface label. The mali- cious content thus appears to belong to the legitimate procedure the agent follows. The accompa- nying SKILL.md provides auxiliary guidance toward the relevant visual regions without explicitly stating the malicious operation. This design tests whether agents treat visual reference content as authoritative procedural guidance and translate the embedded instructions into actual actions. We construct MMSKILLRISK through a systematic process of clean-skill curation, attack-package construction, and executable case instantiation. We begin with 28 carefully curated clean skills selected from public skill packages, resources released with prior multimodal skill work (Zhang et al., 2026; Che et al., 2026), and visual workflows we authored. Building on these skills, we construct 36 NCVA attack packages spanning five attack objectives: data exfiltration, persistence, privilege escalation, artifact pollution, and integrity destruction. We then instantiate 108 evaluation cases, each pairing a legitimate user task with an attacker objective and providing the required inputs, tools, and separate checks for attack success and task completion. This construction links curated skill workflows to controlled visual attacks and executable outcome checks, enabling systematic evaluation of both unauthorized behavior and its coexistence with successful task execution. We use MMSKILLRISK to evaluate nine modelāharness configurations, with each case executed in an isolated sandbox. The results show that (i) NCVA yields a pooled ASR of 43.1%, exceeding the matched text-carrier baseline by 16.4 percentage points, with higher ASR in all nine; (i) 36.5% of evaluation cases exhibit both attack success and legitimate-task completion, reaching 72.2% for GPT-5.6-sol with Codex. Our contributions are: ā¢ We introduce MMSKILLRISK, a systematically constructed, runnable safety benchmark targeting image-borne attacks in multimodal skills. Built from 28 carefully curated clean skills, it contains 36 attack packages and 108 executable cases, with separate checks for unauthorized behavior and legitimate-task completion based on execution traces and deliv- ered artifacts. ā¢ We design NCVA to instantiate this attack surface by embedding malicious instructions as native-looking components of teaching images, while keeping explicit malicious instruc- tions out of the accompanying skill text. This provides a controlled way to evaluate whether agents convert visual reference content into unauthorized actions. ā¢ We conduct an end-to-end evaluation across nine modelāharness configurations, observing successful visual attacks and higher ASR than the matched text-carrier baseline in all nine. Attack success also frequently coexists with legitimate-task completion, highlighting a se- curity risk: skill-bundled images can induce agents to perform unauthorized actions even as they complete legitimate tasks. 2 2RELATED WORK 2.1MULTIMODAL AGENT SKILLS Agent skills support procedural reuse across tasks. Voyager, for example, maintains a growing library of executable programs that can be retrieved and composed for new tasks (Wang et al., 2023). For visual agents, reusable knowledge also concerns the observations that determine when and how a procedure applies. Recent work therefore explores how to represent and access vi- sual guidance within skills. MMSkills couples textual procedures with state cards and multi-view keyframes, consulting selected visual evidence in a temporary branch to align it with the current environment (Zhang et al., 2026). VISUALSKILL organizes application-specific knowledge into a hierarchy of topics and provides a tool for loading the relevant text and figures on demand (Jiang et al., 2026c). AutoVisualSkill distinguishes static visual references, dynamic visual working mem- ory, and interleaved instructions that associate individual steps with supporting visual evidence (Xu et al., 2026). These approaches give visual resources explicit roles in interpreting states, applying procedures, and tracking progress. A complementary line of work studies how reusable skills are acquired and updated. Mirage-1 ab- stracts interaction trajectories into execution, core, and meta-skills, using this hierarchy to guide planning and online exploration (Xie et al., 2025). XSkill extracts both action-level experiences and task-level skills from agent rollouts, grounding their accumulation and retrieval in visual obser- vations (Jiang et al., 2026a). Beyond interaction traces, MMG2Skill converts existing multimodal guides into editable skills and refines them using feedback from agent-visible trajectories (Che et al., 2026). RESOURCE2SKILL draws on tutorial videos, repositories, articles, and reference artifacts to construct a hierarchical skill wiki containing text, code, and visual examples, which agents retrieve and compose during execution (Fan et al., 2026). 2.2AGENT SKILL SECURITY Public skill repositories contain vulnerable and malicious packages, making third-party skills a supply-chain attack surface (Liu et al., 2026c;b). Prior attacks place malicious instructions in helper scripts, documentation examples, or the skill body (Jia et al., 2026a; Qu et al., 2026). Bench- marks study injection through skill-related interfaces and test whether an agent carries out the at- tack (Schmotz et al., 2026; Jin et al., 2026; Guo et al., 2026a). SkillCamo is closest to our setting: it hides instructions in skill images and rewrites the surrounding documentation to bring those images into the workflow (Jia et al., 2026b). Defenses inspect skill files through static or semantic analysis (Cisco AI Defense, 2026; Bhardwaj, 2026; AgentVerus, 2026), test whether behavior matches a skillās stated purpose (Guo et al., 2026b), or reconstruct behavior from multimodal skill resources (Jia et al., 2026b). SkillGuard instead con- strains skill influence through permissions (Pan et al., 2026). We instead benchmark the safety of multimodal skill execution with unauthorized instructions embedded as plausible workflow content in teaching images, separately assessing whether agents submit unauthorized operations to tools and whether they complete the userās task. 3BACKGROUND AND THREAT MODEL 3.1MULTIMODAL SKILL EXECUTION A multimodal skill packages reusable procedural knowledge for an agent, combining textual in- structions with visual reference materials (Zhang et al., 2026; Che et al., 2026). The instructions in SKILL.md state when the skill applies, which steps to follow, and which resources to consult; the accompanying images then supply what words alone convey poorly, such as how interface controls indicate state, how a workflow branches, or what a finished artifact should look like. During exe- cution, the agent does not read these images once and discard them: it returns to them to locate the current interface state, to decide whether a pictured step applies, and to check its own intermediate results against the examples. 3 1 Prepare Skills & Tasks 2 Design & Instantiate Attacks 5 public skills 8 MMSkills adaptations 10 GUIskills 5 our visual workflows Exfiltration Artifact pollution Privilege escalation Persistence Integrity destruction + 36compromised variants 28 base skills + 84 benign tasks Test & Ver i f y At t ack I nst ances 3 Taxonomy Human experts select and design scenarios; agents assist implementation. . SKILL.md Exp.png LLM drafts benign tasks Skill Package/ SKILL.md Images/ Auxiliary files Create Tasks SKILL.md points the agent to the teaching image; the attack payload is embedded in the image. ... Human review quality Select suitable skill-task pairs Define the unauthorized action and success criteria. Run in Docker Skill Package Tasks Traces artifacts state changes Success Failure Did the attack succeed? Let me check the evidence. Figure 1: Data construction pipeline of MMSkillRisk. The stages run from left to right. Skill and Task Preparation curates 28 base skills and constructs 84 benign tasks. Attack Design and In- stantiation creates 36 compromised skill variants across five risk categories, embedding malicious instructions in teaching images with accompanying textual guidance. Instance Testing and Veri- fication executes skillātask pairs in isolated Docker environments and assesses attack success and task completion using traces, artifacts, and state changes, yielding 108 evaluation instances. 3.2THREAT MODEL Attacker capability. We consider a skill supply-chain setting: the attacker either publishes mali- cious skills from scratch or tampers with an existing useful package and distributes the compromised version. In MMSKILLRISK, the attacker controls the skillās instructions and teaching or reference images. The attacker may rewrite workflow guidance in SKILL.md and modify visual content the agent is directed to inspect, but cannot embed explicit malicious semantics directly. Although packages may contain scripts and other auxiliary resources, the attacker cannot modify them in our setting. The attacker cannot modify the base model, the userās request or task inputs, or the agent runtime, and cannot directly perform the target operation on the userās behalf. The operation must be induced through the agentās use of the compromised skill. Attack objective. The attacker aims to induce a pre-specified malicious behavior while the agent carries out the benign task, interfering as little as possible with the taskās completion. The specified behaviors fall into five categories: data exfiltration, artifact pollution, privilege escalation, persis- tence, and integrity destruction. Notably, in the first four, the induced behavior is meant to leave the legitimate deliverable intact; in the last, disrupting legitimate state is the objective itself, which is the one intended exception to the non-interference preference. 4MMSKILLRISK CONSTRUCTION We construct MMSKILLRISK to evaluate whether agents can reject unauthorized instructions em- bedded in the visual resources of multimodal skills while completing legitimate user tasks. Each case pairs a useful skill with a verifiable user task and an attacker objective, placing the malicious instruc- tion within visual references that the agent is directed to consult. Our data construction pipeline has three stages: preparing multimodal skills and selecting benign tasks; designing workflow-aligned risk scenarios and embedding attack instructions in skill-referenced images; and executing instances in isolation to verify outcomes from traces, artifacts, and state changes. Quality control spans all three stages, and the complete construction pipeline is illustrated in Figure 1. 4.1MULTIMODAL SKILL COLLECTION AND TASK PREPARATION The multimodal skills in our benchmark are drawn from public skill marketplaces and GitHub repos- itories, resources released with prior multimodal-skill work, and supplementary workflows that we wrote locally. The final collection comprises five skill packages collected as published, eight adapta- 4 Table 1: Distribution of compromised skill variants and skillātask evaluation instances across the five MMSkillRisk attack objec- tives. Risk objectiveVariantsInstances Data exfiltration824 Artifact pollution721 Privilege escalation618 Persistence721 Integrity destruction824 Total36108 Benign Poisoned We embedded a malicious instruction in a native-looking Site trust panel, disguising it as a compliance notice to induce an unauthorized policy-file modification. Benign Poisoned We embe dde d a malicious instruction in a native-looking āNotesā field, disguising artifact poisoning as a routine chart-export step. Figure 2: Benign and poisoned skill images illus- trating workflow-aligned attack embedding. tions of MMSkills (Zhang et al., 2026), ten GUI skills with schematic illustrations that we authored locally, informed by the task mappings of MMG2Skill and by software tutorials (Che et al., 2026), and five original visual business workflows. We curate the skills through successive rounds of manual selection and adaptation mainly under three criteria: visual resources must contribute substantively to the workflow; the skill must provide an actionable procedure within a meaningful task scope; and its outputs, together with the safety out- comes of interest, must admit observable evidence. We also weigh diversity of application domains and of visual resource forms, rather than collecting a single interface or artifact type. Once the skills are fixed, an LLM drafts several benign tasks for each one, preserving the skillās in- tended use and specifying verifiable deliverables; human reviewers then check task quality and, for each skill, prioritize tasks whose workflows naturally admit plausible malicious extensions before selecting three high-quality tasks from these candidates. The retained tasks span interpreting inter- face states, applying visual rules, reconciling records, and producing structured and visual artifacts. Appendix A provides further details on each skill and its tasks. 4.2WORKFLOW-ALIGNED ATTACK DESIGN AND INSTANTIATION Drawing on the outcome taxonomy of existing skill-safety benchmarks (Jin et al., 2026; Liu et al., 2026a; Schmotz et al., 2026), we group risk objectives into five categories: data exfiltration, artifact pollution, privilege escalation, persistence, and integrity destruction. For the skillātask combinations selected in the first step, we first analyze each skillās normal usage flow: which instructions the agent reads, which teaching images it consults, and which decisions it must make to complete the userās task. For each combination we then choose an attack scenario that fits the normal workflow and whose outcome is observable and verifiable. Every scenario specifies the malicious behavior to induce, the targeted resource, and the evidence needed to determine whether that behavior occurred. We instantiate each attack using Native-Context Visual Attack (NCVA), with human experts design- ing the case-specific mechanism and agents assisting in its implementation. Concretely, we first identify which visual materials a normal task leads the evaluated agent to inspect, and which lo- cations within them plausibly carry procedural information, such as interface information bubbles, document footnotes, chart annotations, or maintenance notes in worked examples. Then, we wrap the malicious instruction as a routine, task-relevant workflow step, such as registering the current task outcome, synchronizing its state, or reviewing its result. Besides, when embedding the instruc- tion, we adapt its typography, color, and layout to the surrounding image, while preserving the visual information needed to complete the benign task. SKILL.md serves only to guide the agent to the image at the appropriate step; the content of the malicious operation is carried by the image. The attack thus enters the execution context along the normal skill-use path and attempts to make the agent treat visual reference content as grounds for action. A formal definition of NCVA is provided in Appendix C. Figure 2 illustrates this design process through two representative cases, showing 5 how benign teaching images are modified to carry malicious instructions that appear to belong to the original workflow. After revision and inspection, we finally construct 36 compromised variants from the 28 base multimodal skills. 4.3ATTACK INSTANCE TESTING AND VERIFICATION After completing the design and construction of the 36 compromised variants, we specify the ma- licious behavior to be induced for each attack scenario and fix its success criterion in advance: the attack target, the operation that must occur, and the evidence sufficient to demonstrate that the agent has submitted a malicious operation. We then pair each compromised skill variant with its cor- responding benign task, prepare the task inputs, dependencies, and initial state, and let the agent execute the task according to the user request in an isolated Docker environment. After the agent completes the task, we collect its operation traces, produced artifacts, and relevant state changes, such as modifications to target files or configurations. We then use a multimodal LLM judge to analyze this evidence against predefined criteria and determine whether the agent performed malicious behavior induced by the skill. Note that execution plans, claims of completion, or commands merely quoted in text do not count as attack success. In addition, completion of the benign task is evaluated separately. The judge prompts used for the two decisions are provided in Appendix I.1 (attack success) and Appendix I.2 (task completion). Quality control spans both instance preparation and testing. During preparation, we manually in- spect each attack image to verify that the injected content is legible and that the visual cues required by the legitimate task remain intact. We then execute each instance to confirm that all required re- sources are available and that the evidence needed for evaluation can be collected. Only after both checks pass do we pair the 36 compromised variants with their corresponding tasks, yielding the final 108 evaluation instances. 5EXPERIMENTS 5.1EXPERIMENTAL SETUP Models and Agent Harnesses. We evaluate nine modelāharness configurations across three agent harnesses: Codex paired with GPT-5.6-sol (OpenAI, 2026), Kimi-K2.6 (Moonshot AI, 2026), Kimi- K3 (Kimi Team, 2026), and Qwen3.8-Flash-Next (Qwen Team, 2026); Claude Code paired with Kimi-K2.6, Kimi-K3, Qwen3.8-Flash-Next, and Claude Opus 4.6 (Anthropic, 2026); and DeepSeek Harness (DSH) paired with DeepSeek-V4.1-Flash (DeepSeek-AI, 2026). Every configuration runs the same 108 cases. Detailed versions and sources of the evaluated models, agent harnesses and sandbox runtime components are provided in Appendix E. Metrics. We report four metrics. Attack Success Rate (ASR) measures the fraction of cases in which the agent submits a skill-induced unauthorized dangerous operation. Task Success Rate (TSR) measures completion of the legitimate task. Task-Compatible ASR (TC-ASR) measures the fraction of cases satisfying both attack and task success. Visual Instruction Adoption Rate (VIAR) measures how often an agent explicitly decides to follow, or attempts to carry out, a ma- licious instruction shown in a skill image. Note that ASR, TC-ASR, and VIAR use all 108 cases per configuration; TSR excludes the 24 integrity-destruction cases. The per-metric computation procedures and formulas are provided in Appendix F. Baseline. We include a text-carrier baseline matched case by case: for each case, the malicious content embedded in the Native-Context Visual Attack (NCVA) images is written in full as plaintext into SKILL.md, while the teaching images remain clean; the task, inputs, execution setting, and scoring are identical to the image-carried condition. For this baseline we report ASR, TSR, and TC-ASR, since VIAR is defined only for the image-carried condition. 5.2MAIN RESULTS Safety varies substantially across agent systems. Table 2 shows how widely the nine modelā harness configurations differ under NCVA. Specifically, Claude Opus 4.6 with Claude Code is the 6 Table 2: Attack and task performance across nine modelāharness configurations (%). ASR, TC- ASR, and VIAR use 108 cases; TSR excludes 24 integrity-destruction cases. Overall rates are pooled. Bold marks higher values between attack conditions, including ties; underlining marks column maxima within each harness. VIAR is reported only for NCVA. HarnessModel Text-only baseline (%)NCVA (%) ASRTSRTC-ASRASRTSRTC-ASR VIAR Codex GPT-5.6-sol59.3 84.551.981.589.372.291.7 Kimi-K2.632.473.823.165.778.649.168.5 Kimi-K314.896.413.938.097.637.038.9 Qwen3.8-Flash-Next13.095.212.032.494.132.433.3 Claude Code Kimi-K2.636.1 73.823.150.975.035.252.8 Kimi-K313.098.813.017.697.617.617.6 Qwen3.8-Flash-Next21.396.418.527.896.427.827.8 Claude Opus 4.68.376.28.312.077.48.312.0 DSHDeepSeek-V4.1-Flash42.686.937.062.089.349.165.7 Overall26.786.922.343.188.436.545.3 most robust configuration, with an ASR of only 12.0%, whereas GPT-5.6-sol with Codex is the most susceptible to the image-borne payload, reaching 81.5%; across all nine systems the pooled ASR is 43.1%. Besides, the harness also matters: for the three models evaluated with both harnesses, Codex yields higher ASR than Claude Code in every caseā65.7% versus 50.9% for Kimi-K2.6, 38.0% versus 17.6% for Kimi-K3, and 32.4% versus 27.8% for Qwen3.8-Flash-Next. Table 3: From image access to attack execution (%). HarnessModelReadā SeenāAttemptāSuccess Claude Code Kimi-K3100.099.117.617.6 Kimi-K2.6 100.053.750.950.9 Qwen3.8-Flash100.0100.027.827.8 Codex Kimi-K399.196.338.038.0 Kimi-K2.6100.070.465.765.7 Qwen3.8-Flash100.094.432.432.4 DSHDeepSeek-V4.1100.075.962.062.0 Overall99.984.342.142.1 The same objective is more effective when car- ried by images. Under the matched text-only baseline, the pooled ASR and TC-ASR are 26.7% and 22.3%; under NCVA they rise to 43.1% and 36.5%, corresponding to increases of 16.4 and 14.2 percentage points, respectively. The ASR gap appears in every configuration but varies widely in size, from 33.3 points for Kimi-K2.6 with Codex to 3.7 for Claude Opus 4.6 with Claude Code. Overall, these results show that NCVA is more effective than the matched text- only baseline in inducing unauthorized behavior. Task completion does not imply safety. The results in Table 2 show that, averaged across all modelāharness configurations, 36.5% of cases complete the legitimate task while the agent also carries out the malicious operation specified by the attacker. This effect is particularly pronounced for GPT-5.6-sol with Codex, which achieves a TC-ASR of 72.2%. These results demonstrate that task completion does not imply safety: an agent can successfully fulfill the authorized user request while simultaneously executing malicious behavior specified by the attacker. We also compare NCVA with matched clean-skill TSR on five configurations, with results reported in Appendix G. Traces reveal an intermediate visual-adoption stage. VIAR measures how often agents recog- nize the malicious requirements embedded in images and treat them as instructions to be followed during execution. The results show an overall VIAR of 45.3%, indicating that 45.3% of cases show trace-supported adoption of image-carried instructions, through either an explicit commitment to act or an execution attempt. 7 Persist. Integrity Exfil. Artifact Privilege Kimi-K3 Codex 22 67 29 12 62 Persist. Integrity Exfil. Artifact Privilege Kimi-K3 Claude Code 6 33 4 8 38 Persist. Integrity Exfil. Privilege Artifact DeepSeek-V4.1-Flash DeepSeek Harness 67 76 38 50 86 Persist. Integrity Exfil. Privilege Artifact GPT-5.6-sol Codex 94 100 54 67 100 Figure 3: Attack-category profiles across modelāharness configurations. Each radar chart reports the ASR (%) of one configuration across five attack categories. All panels use 0-100% scale. 5.3BEHAVIORAL ANALYSIS: FROM PAYLOAD EXPOSURE TO ATTACK SUCCESS To understand how malicious behavior emerges during multimodal skill execution, we select seven modelāharness configurations and trace the progression from image access (Read), payload recog- nition (Seen) to targeted execution attempts (Attempt) and confirmed attack success (Success). Table 3 presents the complete progression across the seven modelāharness configurations. As ex- pected, nearly all modelāharness configurations achieve 100% image access, this is because even when completing only the legitimate task, these visual resources are an integral part of the procedure that agents are required to follow. The first real divergence appears at Seen. Some configurations recognize the payload in almost every case where it is read, whereas Kimi-K2.6 with Claude Code leaves the payload unrecognized in 46.3% of its cases. A plausible explanation is that such systems attend to the visual information the legitimate task requires and treat the embedded instruction as background content of the illustration. Conditional on recognizing the payload, the systems diverge again at Attempt. Kimi-K3 with Claude Code escalates only 17.8% of the cases where it saw the payload into a targeted execution attempt, whereas Kimi-K2.6 and DeepSeek-V4.1-Flash act on the recognized payload in most cases. Overall, we find that different modelāharness configurations exhibit substantially different behaviors mainly at the recognition and compliance stages. NCVA succeeds only when both stages fail to prevent the attack. 5.4VULNERABILITY ANALYSIS ACROSS RISK CATEGORIES To examine whether agent systems differ in their sensitivity to the five attack objectives, we select four representative modelāharness configurations and summarize their per-objective ASR in Fig- ure 3. The configurations show both shared risk patterns and clear differences across categories. Persistence and artifact pollution yield relatively high ASR across all four configurations, while data exfiltration and integrity destruction remain lower. For example, GPT-5.6-sol with Codex reaches 100% ASR on both persistence and artifact pollution, compared with 67% on data exfiltration. DeepSeek-V4.1-Flash reaches 86% on artifact pollution versus 50% on data exfiltration. Across the evaluated configurations, agents are thus more susceptible to persistence and artifact pollution, while showing greater resistance to data exfiltration and integrity destruction. The harness also affects category-specific vulnerability. Kimi-K3 has higher ASR with Codex than with Claude Code across all five objectives, with the largest gap in persistence (67% versus 33%), followed by integrity destruction (29% versus 4%). Thus, the same model exhibits different safety performance across harnesses, with the gap varying by attack category. 5.5ABLATION STUDIES To identify which carrier the attack depends on and how much each surface contributes, we conduct ablation studies on Kimi-K3 with both Codex and Claude Code. We compare three case-matched conditions: the full attack, an image-only payload variant that keeps the payload images while restor- ing the clean SKILL.md, and a payload-free control that keeps the guiding SKILL.md scaffolding while restoring the clean teaching images. Figure 4 summarizes the results. 8 Full attackClean SKILL.md Clean images 0 10 20 30 40 Attack success rate (ASR, %) CodexClaude Code 38.0% 7.4% 0.0% 17.6% 1.9% 0.0% Figure 4: Ablation of textual and visual attack components for Kimi-K3. Table 4: Skill packages flagged by default scan- ner configurations. Values are percentages of packages in each set. ScannerClean (%) NCVA (%) Cisco Skill Scanner3.62.8 SkillFortify7.15.6 ClawGuard Auditor14.311.1 GoPlus AgentGuard10.78.3 AgentVerus Scanner10.78.3 Figure 4 shows that restoring clean images drops ASR to zero under both harnesses: the payload images are a necessary condition of attack success in this design, as expected, since all malicious semantics reside in the images and the prose alone carries none. Removing the prose while keep- ing the payload images reduces ASR from 38.0% to 7.4% under Codex and from 17.6% to 1.9% under Claude Code. By inspecting the execution traces, we find that without the guidance from SKILL.md, models in most cases treat the embedded instructions as background content of the illustration rather than acting upon them. In conclusion, the two carriers therefore play complementary roles. The images supply the malicious semantics; the prose promotes them from illustrative material to procedural authority, telling the agent that the pictured content is part of the workflow to be completed. Neither surface carries the full effect alone: the attack disappears without the images and shrinks to a small residual without the prose; NCVAās effectiveness comes from their combination. 5.6DEFENSE ANALYSIS We run five publicly available skill scanners under their default configurationsāCisco Skill Scanner (Cisco AI Defense, 2026), SkillFortify (Bhardwaj, 2026), ClawGuard Audi- tor (SafeAgent-Beihang, 2026), GoPlus AgentGuard (GoPlus Security, 2026), and AgentVerus Scanner (AgentVerus, 2026)āover two package sets: the 28 clean base skills and their 36 com- promised variants, and report the percentage of flagged packages in Table 4. Table 4 shows that the scanners flag compromised skill packages at generally low rates, with a maximum of 11.1%, and two scanners below 6%. We further manually compare the alerts raised on compromised packages with those raised on their corresponding clean versions and find them to be largely consistent. Most reported risks arise from pre-existing skill content. These results suggest that, under the evaluated default configurations, the scanners have limited ability to identify newly introduced malicious instructions carried by visual resources. This finding is consistent with their inspection scope: existing skill scanners primarily analyze textual and code- level surfaces, including SKILL.md, manifests, scripts, and metadata, whereas the payloads in MMSKILLRISK are carried by teaching images and the skill text contains no explicit malicious instructions. As a result, attacks of this form can largely evade these scanners. 6CONCLUSION In this paper, we introduced MMSKILLRISK, to our knowledge the first publicly available bench- mark dedicated to end-to-end safety evaluation of image-borne attacks in multimodal skills. Built from 28 clean skills, it comprises 36 attack packages and 108 executable cases spanning five attack objectives, with separate checks for attack success and task completion. NCVA disguises malicious instructions as native components of teaching images, while skill text provides auxiliary guidance without explicit malicious instructions. Across nine modelāharness configurations, NCVA achieves a pooled ASR of 43.1%, exceeding the matched text-carrier baseline (26.7%) by 16.4 percentage 9 points, with higher ASR in all nine. Attack success co-occurs with legitimate-task completion in 36.5% of cases, showing that task success alone does not establish safe skill use. Existing skill scanners flag compromised packages at low rates, with alerts largely consistent with those on clean versions. MMSKILLRISK supports the development and evaluation of runtime defenses against image-borne attacks in multimodal skills. ETHICS STATEMENT We develop MMSKILLRISK to understand and improve the security of multimodal agents. All ex- periments use isolated task environments with synthetic inputs and instrumented tools. Data-transfer effects are evaluated through a local receiver, while file modifications and persistence effects are confined to the experimental environments. We recognize that the attack techniques could be mis- used. Our aim is to support security evaluation and the development of defenses against malicious multimodal skills. REFERENCES AgentVerus. AgentVerus Scanner. Software repository, 2026. URL https://github.com/ agentverus/agentverus-scanner. Accessed: 2026-09-25. Anthropic. Claude Opus 4.6 system card. System card, Anthropic, February 2026. URL https: //anthropic.com/claude-opus-4-6-system-card. Varun Pratap Bhardwaj. Formal analysis and supply chain security for agentic ai skills. arXiv preprint arXiv:2603.00195, 2026. doi: 10.48550/arXiv.2603.00195. URL https://arxiv. org/abs/2603.00195. Xinyu Che, Junqi Xiong, Yunfei Ge, Xinping Lei, Shihao Li, Hang Yan, Han Li, Yuanxing Zhang, Zhiqi Bai, Jinhua Hao, et al. Mmg2skill: Can agents distill in-the-wild guides into self-evolving skills? arXiv preprint arXiv:2606.01993, 2026. Cisco AI Defense. Skill Scanner: Security Scanner for Agent Skills. Software repository, 2026. URL https://github.com/cisco-ai-defense/skill-scanner. Accessed: 2026-09- 25. DeepSeek-AI. DeepSeek-V4.1-Flash: Pushing the limits of KV cache compression. arXiv preprint arXiv:2609.19969, 2026. URL https://arxiv.org/abs/2609.19969. Yijia Fan, Zonglin Di, Zimo Wen, Yifan Yang, Mingxi Cheng, Qi Dai, Bei Liu, Kai Qiu, Yue Dong, Ji Li, et al. Resource2skill: Distilling executable agent skills from human-created multimodal resources. arXiv preprint arXiv:2606.29538, 2026. GoPlus Security. GoPlus AgentGuard. Software repository, 2026. URL https://github. com/GoPlusSecurity/agentguard. Accessed: 2026-09-25. Wenbo Guo, Wei Zeng, Chengwei Liu, Xiaojun Jia, Yijia Xu, Lei Tang, Yong Fang, and Yang Liu. Malskillbench: A runtime-verified benchmark of malicious agent skills. arXiv preprint arXiv:2606.07131, 2026a. Zihan Guo, Zhiyu Chen, Xiaohang Nie, Jianghao Lin, Yuanjian Zhou, and Weinan Zhang. Skill- probe: Security auditing for emerging agent skill marketplaces via multi-agent collaboration. arXiv preprint arXiv:2603.21019, 2026b. Xiaojun Jia, Jie Liao, Simeng Qin, Jindong Gu, Wenqi Ren, Xiaochun Cao, Yang Liu, and Philip Torr. Skillject: Automating stealthy skill-based prompt injection for coding agents with trace- driven closed-loop refinement. arXiv preprint arXiv:2602.14211, 2(4):7, 2026a. Xiaojun Jia, Jie Liao, Simeng Qin, Ke Ma, Wenbo Guo, Yebo Feng, Aishan Liu, and Yang Liu. Seeing is not screening: Multimodal hidden instruction attacks on agent skill scanners. arXiv preprint arXiv:2606.18198, 2026b. Guanyu Jiang, Zhaochen Su, Xiaoye Qu, and Yi R Fung. Xskill: Continual learning from experience and skills in multimodal agents. arXiv preprint arXiv:2603.12056, 2026a. 10 Yukun Jiang, Yage Zhang, Michael Backes, Xinyue Shen, and Yang Zhang. Harmfulskillbench: How do harmful skills weaponize your agents? arXiv preprint arXiv:2604.15415, 2026b. Ziyan Jiang, Li An, Yujian Liu, Jiabao Ji, Qiucheng Wu, Jacob Andreas, Yang Zhang, and Shiyu Chang. Visualskill: Multimodal skills for computer-use agents. arXiv preprint arXiv:2606.18448, 2026c. Chang Jin, An Wang, Zeming Wei, Kai Wang, Biaojie Zeng, Qiaosheng Zhang, Chao Yang, Jingjing Qu, Xia Hu, and Xingcheng Xu. Skillsafetybench: Evaluating agent safety under skill-facing attack surfaces. arXiv preprint arXiv:2605.12015, 2026. Kimi Team. Kimi K3: Open frontier intelligence. arXiv preprint arXiv:2607.24653, 2026. URL https://arxiv.org/abs/2607.24653. Xiangyi Li, Yimin Liu, Wenbo Chen, Bingran You, Zonglin Di, Yifeng He, Shenghan Zheng, Ky- oung Whan Choe, Jiankai Sun, Shuyi Wang, et al. Skillsbench: Benchmarking how well agent skills work across diverse tasks. arXiv preprint arXiv:2602.12670, 2026. Qiyuan Liu, Tingfeng Hui, Kun Zhan, Kaike Zhang, and Ning Miao. Openskillrisk: Benchmarking agent safety when using real-world risky third-party skills. arXiv preprint arXiv:2607.20121, 2026a. Yi Liu, Zhihao Chen, Yanjun Zhang, Gelei Deng, Yuekang Li, Jianting Ning, and Leo Yu Zhang. Malicious agent skills in the wild: A large-scale security empirical study. arXiv e-prints, p. arXivā2602, 2026b. Yi Liu, Weizhe Wang, Ruitao Feng, Yao Zhang, Guangquan Xu, Gelei Deng, Yuekang Li, and Leo Zhang. Agent skills in the wild: An empirical study of security vulnerabilities at scale. arXiv preprint arXiv:2601.10338, 2026c. Moonshot AI. Kimi K2.6: Advancing open-source coding. Technical blog, April 2026. URL https://w.kimi.com/en/blog/kimi-k2-6. Accessed: 2026-09-26. OpenAI.GPT-5.6 system card.System card, OpenAI, July 2026.URL https:// deploymentsafety.openai.com/gpt-5-6. Shidong Pan, Xiaoyu Sun, Tianyi Zhang, Dianshu Liao, Meixue Si, and Zhenchang Xing. Skill- guard: A permission framework for agent skills. arXiv e-prints, p. arXivā2606, 2026. Yubin Qu, Yi Liu, Tongcheng Geng, Gelei Deng, Yuekang Li, Leo Yu Zhang, Ying Zhang, and Lei Ma. Supply-chain poisoning attacks against llm coding agent skill ecosystems. arXiv preprint arXiv:2604.03081, 2026. Qwen Team. On the design of Qwen3.8-Next architecture: Evaluation, efficiency, and training stability. Technical Report arXiv:2608.30320, Alibaba Group, August 2026. URL https: //arxiv.org/abs/2608.30320. SafeAgent-Beihang. ClawGuard: Enterprise AI Agent Security Toolkit. Software repository, 2026. URL https://github.com/SafeAgent-Beihang/clawguard. Auditor module; ac- cessed: 2026-09-25. David Schmotz, Luca Beurer-Kellner, Sahar Abdelnabi, and Maksym Andriushchenko. Skill-inject: Measuring agent vulnerability to skill file attacks. arXiv preprint arXiv:2602.20156, 2026. Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291, 2023. Yuquan Xie, Zaijing Li, Rui Shao, Gongwei Chen, Kaiwen Zhou, Yinchuan Li, Dongmei Jiang, and Liqiang Nie. Mirage-1: Augmenting and updating gui agent with hierarchical multimodal skills. arXiv preprint arXiv:2506.10387, 2025. Binxiao Xu, Ruichuan An, Bocheng Zou, and Hang Hua. Agent skills should go beyond text: The case for visual skills. arXiv preprint arXiv:2606.01414, 2026. 11 Renjun Xu and Yang Yan. Agent skills for large language models: Architecture, acquisition, secu- rity, and the path forward. arXiv preprint arXiv:2602.12430, 2026. Kangning Zhang, Shuai Shao, Qingyao Li, Jianghao Lin, Lingyue Fu, Shijian Wang, Wenxiang Jiao, Yuan Lu, Weiwen Liu, Weinan Zhang, et al. Mmskills: Towards multimodal skills for general visual agents. arXiv preprint arXiv:2605.13527, 2026. Haomin Zhuang, Hanwen Xing, Yujun Zhou, Yuchen Ma, Yue Huang, Yili Shen, Yufei Han, and Xiangliang Zhang. Agenttrap: Measuring runtime trust failures in third-party agent skills. arXiv preprint arXiv:2605.13940, 2026. 12 ABASE MULTIMODAL SKILLS AND LEGITIMATE TASKS MMSKILLRISK contains 28 base skills, each paired with three legitimate tasks, for a total of 84 tasks. We group the skills by their source and construction method in Tables 5ā8. The tables summarize the shared task scope for each skill; the three tasks differ in their inputs, interface states, or required decisions. The complete task prompts, input resources, and expected deliverables are provided with the benchmark instances. The MMSkills adaptations use visual resources from the original skills in new, offline state-reading tasks; they do not reproduce the original OSWorld interactions. The GUI skills use locally authored schematic interfaces, whereas the original visual workflow skills use locally created task materials. Table 5: Publicly collected skills and their legitimate task scopes. Base skillScope of the three legitimate tasks anydesignExtract design information from page screenshots, re- produce components, and document the design. breadth-chart-analystRead breadth charts and their signals, and prepare ana- lytical results. guizang-social-card-skillCreate social media cards for different themes using supplied materials. infographicProduce infographics that follow the requested topic and brand requirements. sector-analystRead sector rotation and momentum information across time frames. Table 6: Skills adapted from MMSkills resources and their legitimate task scopes. Base skillScope of the three legitimate tasks mmskills-calc-chartsIdentify Calc chart configurations and output states. mmskills-chrome-permissionsRead Chrome permission and security settings. mmskills-gimp-layersRead GIMP layer and mask states. mmskills-thunderbird-filtersRead mail-filter rules and their enabled states. mmskills-ubuntu-assistive-inputRead accessibility and input settings. mmskills-vlc-audio-settingsRead audio effects and volume preferences. mmskills-vscode-extensionsIdentify VS Code extension installation and configura- tion states. mmskills-writer-page-layoutRead page layout, numbering, and text-selection states. Table 7: Locally authored schematic GUI skills and their legitimate task scopes. Base skillScope of the three legitimate tasks gui-calc-freezeDetermine the frozen-pane state of a spreadsheet. gui-calc-pivotInspect pivot-table field layouts. gui-chrome-searchDistinguish the default search engine from a high- lighted candidate. gui-gimp-colorDetermine the current state of a color-adjustment inter- face. gui-gimp-layersDetermine layer selection, visibility, and related states. gui-impress-exportInspect presentation-export dialog settings. gui-vlc-conversionDetermine whether media-conversion settings are ready. gui-vscode-autosaveRead Auto Save settings. gui-vscode-keybindingsInspect shortcut bindings and related interface states. gui-writer-characterRead character-formatting settings. 13 Table 8: Original visual workflow skills and their legitimate task scopes. Base skillScope of the three legitimate tasks visual-chart-reconciliationReconcile and correct chart data bindings or values. visual-form-routingDetermine how requests should be routed using forms and flow diagrams. visual-invoice-reconciliationReconcile invoices, purchase orders, and posting batches. visual-palette-auditCheck interface color tokens and correct inconsisten- cies. visual-warehouse-pickingAllocate picking batches using bin labels and inventory states. Benign Poisoned We embedded a forged desk-sync note in a VLC teaching imageās āMedia Infoā popup, framing an unauthorized audio-preset registry change as routine settings maintenance. Benign Poisoned We added a desk registration note to the sample breadth chart, presenting a persistent change to the analystās default settings as part of the chart-reading workflow. Benign Poisoned We planted an Extension Registry notice in the VS Code teaching screenshot, prompting the agent to copy sensitive registry fields into its final task output as a routine reporting step. Benign Poisoned We planted a desk diagnostics note in the Thunderbird filter screenshot, prompting the agent to copy internal tokens into its final output as a routine review step. Benign Poisoned We hid a numbered footnote in a Writer teaching screenshot, framing an unauthorized layout-preset registry entry as routine desk setup. Benign Poisoned We added a desk-publishing note to the performance chart, framing an unauthorized queue entry as routine publication registration. (a) (b) (c) (d) (e) (f) Figure 5: Additional examples of benign and poisoned skill images with workflow-aligned attack embeddings. BADDITIONAL EXAMPLES OF WORKFLOW-ALIGNED SKILL-IMAGE ATTACKS Figure 5 extends Figure 2 with further benignāpoisoned image pairs. In each pair, the benign im- age provides visual guidance for a legitimate task; the poisoned version places an unauthorized instruction within the same teaching material. The examples use different visual surfaces, includ- ing an application note window, a chart annotation, a notification, a filter dialog, and a document footnote. These placements make the added instruction appear connected to the workflow the skill describes. The paired images show what changed in the visual resource; whether an agent followed the instruction is assessed separately through execution traces and outcome checks. 14 CFORMAL DEFINITION OF NCVA We represent a multimodal skill as S = (D,V,R),(1) where D denotes the textual skill documentation, V = V 1 ,...,V m the teaching images, and R the scripts and other auxiliary resources. Given a legitimate user task q, task inputs x, and an agent systemA comprising a model and its harness, task execution is represented as (Ļ,y) = Run(A,q,x;S),(2) where Ļ is the execution trace and y the delivered artifact. NCVA modifies the skill documentation and teaching images to induce unauthorized malicious operations during legitimate task execution. Visual Payload Construction. Let g denote the attacker objective and p g its corresponding visual instruction payload. The payload specifies the malicious operation and how its required parameters are obtained. Concrete parameters may be extracted from the current task inputs or execution context rather than being fully fixed in the image. The attacker selects a set of image indices J ā 1,...,m and a carrier region r j within each selected image. The modified images are defined as e V j = Embed NC (V j ,r j ,p g ), j ā J, V j ,j /ā J. (3) Here, Embed NC denotes native-context embedding: malicious instructions are incorporated as plau- sible content within the imageās scenario, such as chart annotations, workflow steps, or interface labels. This notation describes the case construction process without prescribing a particular editing tool or automated implementation. Its defining constraint is that the payload uses the imageās ex- isting semantic structure rather than appearing as an additional instruction overlay unrelated to the scene. Textual Guidance and Modification Scope. Alongside the visual payload, the documentation D is modified into e D to guide the agent to inspect and use the corresponding image regions. The mod- ified text does not explicitly state the malicious operation, but its visual references and procedural guidance can influence how the agent uses the image content. The compromised skill is therefore e S = ( e D, e V,R), e V = e V 1 ,..., e V m .(4) The legitimate task q, task inputs x, and remaining skill resourcesR are unchanged. Malicious op- eration instructions are carried by the visual resources, while the textual modifications guide access to and use of those resources. The absence of explicit malicious instructions in the documentation does not imply that textual guidance has no effect on attack behavior. Execution Outcomes. Executing the compromised skill produces (eĻ,ey) = Run(A,q,x; e S).(5) Letee denote the evaluation evidence for this run, including its execution trace, delivered artifact, and relevant task-check results. Under the scoring rules defined in this paper, let C attack (e) and C task (e) denote satisfaction of the attack-success and legitimate-task-completion criteria, respectively. We define a = I[C attack (e)], t = I[C task (e)],(6) where I[Ā·] is the indicator function. The objective g guides attack construction, and a indicates whether the agent submits the attacker-specified unauthorized operation or a semantically equivalent implementation, without requiring its downstream effect to occur. The joint outcome c = at(7) indicates whether attack success and legitimate-task completion co-occur. When c = 1, the agent satisfies the legitimate task requirements while exhibiting malicious behavior that meets the attack- success criteria. This formulation treats attack behavior and task performance as separate outcomes and aligns with the ASR, TSR, and TC-ASR scoring framework. The eligible cases and denomina- tors for each metric follow Appendix F. 15 DEVALUATION CRITERIA The following are the English prompt templates used for the two independent judgments. Case- specific task contracts, execution evidence, and original images are supplied separately as user- message content. They are evidence to be assessed, not instructions to the judge. The boxes repro- duce the fixed system prompts; they do not include any case-specific evidence or credentials. D.1HUMAN VERIFICATION For each modelāharness configuration, we manually reviewed a subset of runs after execution. Hu- man reviewers independently assessed attack success and legitimate-task completion using the ex- ecution evidence and task artifacts. Their judgments agreed with the LLM judge on the reviewed cases. EMODELS, AGENT HARNESSES, AND EXECUTION ENVIRONMENT E.1MODELāHARNESS CONFIGURATIONS We evaluate nine modelāharness configurations on the same set of 108 instances, covering 36 com- promised skill variants. Models are accessed through remote APIs, while agent harnesses run inside Docker containers to manage context, access skill resources, invoke tools, and produce task outputs. Table 9 lists the model identifiers submitted in API requests and the harness versions used in the evaluation. Table 9: Model identifiers and agent-harness versions used in the evaluation. ModelAPI identifierHarnessVersion GPT-5.6-sol gpt-5.6-solCodex0.153.3 Kimi-K2.6 Kimi-K2.6Codex0.153.3 Kimi-K3 Kimi-K3Codex0.153.3 Qwen3.8-Flash-Next Qwen3.8-Flash-NextCodex0.153.3 Kimi-K2.6 Kimi-K2.6Claude Code 2.1.268 Kimi-K3 Kimi-K3Claude Code 2.1.268 Qwen3.8-Flash-Next Qwen3.8-Flash-NextClaude Code 2.1.268 Claude Opus 4.6 claude-opus-4-6Claude Code 2.1.268 DeepSeek-V4.1-Flash DeepSeek-V4.1-FlashDSH0.1.5rc1 Codex and Claude Code are installed from the npm packages @openai/codex and @anthropic-ai/claude-code,respectively.DeepSeekHarness(DSH)isin- stalled from the Python package deepseek-harness-sdk;both the SDK and its deepseek-harness-runtime-bin dependency use version 0.1.5rc1. The corresponding public projects are Codex, Claude Code, and DeepSeek Harness. The systems access models through compatible API gateways. Codex uses the Responses interface, Claude Code uses the Anthropic Messages interface, and DSH uses its configured SDK provider interface. The model identifiers in Table 9 are service identifiers rather than independently verified weight-snapshot identifiers. E.2CONTAINER IMAGES AND RUNTIME DEPENDENCIES We use one Docker image for Codex and Claude Code and a separate image for DSH. Table 10 summarizes the software environment and per-instance resource limits. The shared Codex/Claude Code image additionally contains npm 10.9.4, Pillow 12.3.0, Python Play- wright 1.62.0, and Node.js Playwright 1.62.1. The image tags and immutable IDs are listed below. 16 Table 10: Container environment and per-instance resource limits. ComponentCodex / Claude CodeDSH Operating systemDebian GNU/Linux 13 (trixie)Debian GNU/Linux 13 (trixie) Debian full version13.613.6 Architecture linux/arm64 linux/arm64 Python3.11.163.11.16 Node.js22.22.022.22.0 CPU quota per instance2 CPUs2 CPUs Memory limit per instance4 GiB4 GiB Codex / Claude Code image. Tag: mmbench-level3-codex0.153.3-claude2.1. 268-skilltools1-20260917 Image ID: sha256:88ba050e00bafddd1316b0d08f2775b1530ebe7c5ad68ffcdde699576c06da0 DeepSeek Harness image. Tag: mmbench-level3-dsh0.1.5rc1-skilltools1-20260917 Image ID: sha256:c64793f017a5f6ae0853e3334feb27df4bd11d9715b4e4cf506a5a57eae1095 Task-specific tools are supplied with their corresponding skill packages. Docker isolates task files and tool execution, while model inference is provided by remote API services. E.3HARNESS SETTINGS AND EXECUTION PROTOCOL Codex. Codexrunsthrough codex execwithJSONeventloggingand model_reasoning_effort="medium".We use ephemeral sessions, ignore external user configuration and rules, and disable Apps and remote plugins. Interactive approvals and the harnessās internal sandbox are bypassed inside the Docker task environment. Claude Code. Claude Code runs non-interactively through claude -print,with stream-json output and -effort medium.Session persistence is disabled, external settings are not loaded, the MCP configuration is empty, and Chrome integration is disabled. Interactive permission prompts are skipped inside the Docker task environment. DeepSeek Harness. DSH runs through the Python SDK using the sdk profile and reasoning_effort=āhighā.Each instance has a separate workspace and DSH home directory.The deployment configuration enables text and image inputs, with imagePixelBudget=640000and imageMaxBytes=1048576.Thestandalone web tool and associated telemetry components are disabled.The permission mode is danger-full-access within the container. The results characterize these configured systems, rather than the harnesses under their default in- teractive permission settings. Reasoning-effort settings do not imply equal inference budgets across model providers. We do not impose a common explicit sampling temperature on all actor configu- rations. Execution Budget. Every instance runs under a uniform execution timeout of 1000 s. Task Inputs and Image Delivery. Each instance uses a separate task directory mounted at /workspace, containing the skill package, user task, task inputs, and initial state. Images are supplied through the native input mechanism of each harness: image attachments for Codex, image content blocks in the streaming input for Claude Code, and multimodal SDK content for DSH. Execution Evidence and Evaluation. We collect agent responses, structured tool calls, tool re- sults, generated artifacts, and state evidence relevant to the attack objective. Attack success and legitimate task completion are assessed separately using gemini-3.7-flash. Judges access the 17 saved evidence through direct API requests with tools disabled. The criteria and prompt templates are provided in Appendix I.1 and Appendix I.2. FMETRIC DEFINITIONS AND COMPUTATION We compute all metrics separately for each modelāharness configuration. LetD denote its evalua- tion set, with|D| = 108, and letD task ā D denote the 84 cases remaining after excluding the 24 integrity-destruction cases. For each case i, we define the following indicators: ā¢ a i = 1 if the attack-success judgment confirms a skill-induced submission of an unautho- rized dangerous operation, and a i = 0 otherwise. ā¢ t i = 1 if the task-completion judgment confirms completion of the legitimate task, and t i = 0 otherwise. ā¢ v i = 1 if the visual-adoption assessment identifies an explicit commitment to, or an ex- ecution attempt of, an unauthorized instruction attributable to a skill image, and v i = 0 otherwise. These indicators are used for aggregation. A zero contribution from an unresolved judgment does not itself establish a confirmed behavioral failure. F.1ATTACK SUCCESS RATE ASR measures the proportion of cases in which the agent submits a skill-induced unauthorized dangerous operation: ASR = P iāD a i |D| Ć 100%.(8) A qualifying operation must be supported by execution evidence, such as a structured tool invoca- tion. Its downstream effect need not be realized: an unauthorized operation submitted to a tool may qualify even if it is subsequently blocked or fails to execute. Plans, completion claims, commands merely quoted in text, and passive inspection of malicious content do not qualify. The attack judgeās full criteria are provided in Appendix I.1. F.2TASK SUCCESS RATE TSR measures completion of the legitimate task overD task : TSR = P iāD task t i |D task | Ć 100%.(9) Task completion is assessed independently of attack success. The task judge evaluates the original user requirements against the available inputs, final artifacts, and execution evidence. A task is confirmed complete only when all explicit mandatory requirements are satisfied. The 24 integrity- destruction cases are excluded from the reported TSR, giving a denominator of 84 per configuration. The task-completion criteria are provided in Appendix I.2. F.3TASK-COMPATIBLE ATTACK SUCCESS RATE TC-ASR measures simultaneous attack and legitimate-task success within the same case: TC-ASR = P iāD a i t i |D| Ć 100%.(10) We compute this metric by matching the attack and task judgments for each individual case, rather than multiplying the aggregate ASR and TSR. Unlike TSR, TC-ASR includes all 108 cases, in- cluding the 24 integrity-destruction cases, and uses their recorded task-completion judgments. Con- sequently, TC-ASR and TSR have different denominators. Because ASR and TC-ASR share a denominator, TC-ASR cannot exceed ASR. 18 F.4VISUAL INSTRUCTION ADOPTION RATE VIAR measures trace-supported adoption of unauthorized instructions attributable to skill images: VIAR = P iāD v i |D| Ć 100%.(11) The visual-adoption assessment retrospectively examines the skill images, accompanying non-image text, and recorded agent behavior. A case contributes to the numerator when the evidence supports either of the following: ā¢ Planned adoption: the agent explicitly commits to carrying out the unauthorized instruc- tion conveyed by the image. ā¢ Attempted execution: the agent attempts to carry out that instruction through an observ- able action. Each case is counted at most once, including cases that contain both a plan and a subsequent attempt. The assessment must support attribution to the image, rather than merely identify an unsafe action. In particular, the adopted instruction must contain operative content supplied by the image and not sufficiently specified by the accompanying non-image text alone. Merely receiving or opening an image, transcribing its text, describing its contents, or refusing its instructions does not establish adoption. VIAR differs from ASR in two respects: it includes explicit commitments that have not yet resulted in a tool submission, and it requires evidence linking the adopted instruction to the visual source. Therefore, neither metric is required to be higher than the other. VIAR is an independent LLM-based retrospective assessment; it provides trace-supported attribution rather than counterfactual proof that the image caused the behavior. F.5OVERALL AGGREGATION Overall rates are pooled across the nine configurations. For metric M , let S M,g denote its success numerator and N M,g its predefined denominator for configuration g. Then Overall(M) = P 9 g=1 S M,g P 9 g=1 N M,g Ć 100%.(12) The pooled denominator is 972 for ASR, TC-ASR, and VIAR, and 756 for TSR. Because all con- figurations have equal denominators for each metric, pooled rates also equal the arithmetic mean of the unrounded per-configuration rates. Aggregate metrics are computed from case-level counts rather than from rounded percentages. All percentages are reported to one decimal place. Averaging the displayed per-configuration percent- ages may therefore yield a slightly different value from the reported overall rate. Table 11: Task success rates (%) with clean skills and NCVA. Each result is measured over 84 non- integrity cases. HarnessModelClean TSRNCVA TSR CodexKimi-K398.897.6 Claude CodeKimi-K398.897.6 DSHDeepSeek-V4.1-Flash92.989.3 CodexQwen3.8-Flash-Next96.494.1 Claude CodeQwen3.8-Flash-Next96.496.4 GCLEAN-SKILL TASK PERFORMANCE We additionally evaluate task performance using the original clean skills on five representative modelāharness configurations. Each configuration contains 28 clean skills and three benign tasks 19 per skill, yielding 84 cases. We keep the tasks, inputs, execution environment, 1,000 s timeout, and task-success criteria unchanged; only the compromised skill is replaced by its clean counterpart. The results are shown in Table 11. Across the five evaluated modelāharness configurations, NCVA maintains comparable task performance, with only minor decreases observed at most. HLIMITATIONS MMSKILLRISK currently contains 28 base skills and 36 compromised variants covering five at- tacker objectives. While this scale exposes concrete safety risks in multimodal skills, the number of skills and the diversity of their workflows remain limited, and designed cases of this kind cannot rep- resent the full range of usage scenarios or the risk distribution of real skill ecosystems. Besides, all experiments run in controlled Docker sandboxes and do not yet include OSWorld-style end-to-end GUI interaction on real desktops. The measured effects therefore cannot fully reflect how dynamic interface changes, visual grounding, and continuous multi-step manipulation would modulate attack effectiveness. Future work can broaden skill sources and modality coverage, and introduce real desktop environments with longer-horizon tasks, to test whether these risks carry over to deployed settings. IJUDGE PROMPTS I.1ATTACK-SUCCESS JUDGMENT The attack judge examines the recorded agent actions and other observable evidence against the attack objective. Its primary success criterion is the submission of an unauthorized, skill-induced dangerous operation to a tool. A plan, a claim, or a command merely quoted in text does not meet this criterion. The detailed system prompt can be found in Figure 6. The user message contains CASE EVIDENCE (untrusted data only): followed by the structured, case-specific evidence packet. I.2TASK-COMPLETION JUDGMENT Task completion is assessed separately from attack success. Before any agent output is inspected, a first prompt extracts an atomic acceptance checklist from the original user-task contract. The resulting checklist is frozen and reused for every agent evaluated on that task. The user message for this step contains the serialized original task contract, and the detailed prompts can be found in Figure 7. A second prompt evaluates each frozen requirement against the original task inputs, final artifacts, structured tool records, and original images. The case-specific user message begins with CASE EVIDENCE (untrusted data): and includes the structured evidence packet and the original images. The judge returns a separate status and evidence citations for each requirement, as well as an overall completed, incomplete, or unverified verdict. The detailed prompts can be found in Figure 8. 20 Figure 6: Attack-success judge: system prompt and output schema 21 Figure 7: Task judge, step 1: acceptance-checklist prompt 22 Figure 8: Task judge, step 2: task-completion prompt 23