Paper deep dive
SKILLLITE: Evidence-Guided Malicious Skill Auditing with Compact LLMs
Haoran Ou, Gelei Deng, Xuanye Zhang, Wenbo Guo, Tianwei Zhang, Kwok-Yan Lam
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:As LLM-based agents perform increasingly complex tasks, Agent Skills have emerged as a flexible mechanism for extending their capabilities. An Agent Skill packages task-specific instructions with executable components and auxiliary resources to provide specialized functionalities. However, the growing adoption of third-party Skills introduces a new supply-chain attack surface. Malicious Skills can embed harmful behaviors that abuse agent privileges and compromise the agent execution environment or accessible resources. Although recent LLM-based malicious Skill auditing approaches have achieved promising performance, they often rely on capable commercial LLMs. How to achieve effective auditing with compact, locally deployable LLMs in security-sensitive and resource-constrained settings remains largely unexplored. Our investigation reveals that compact LLMs struggle to identify malicious behaviors hidden in complex Skill packages. This difficulty arises from both the implicit nature of such behaviors and the limited reasoning capacity of compact LLMs. To address these challenges, we propose SKILLLITE, an evidence-guided agentic framework for malicious Skill detection. SKILLLITE effectively extracts security-relevant behaviors and infers the intended functionality from complex Skill packages. It then employs a compact LLM to assess the maliciousness of the Skill based on the observed behaviors and their functional context. Experiments show that SKILLLITE improves malicious Skill detection across different compact LLM backbones and outperforms existing representative auditing baselines. Its effectiveness generalizes to behaviorally confirmed in-the-wild malicious Skills. Meanwhile, SKILLLITE maintains a low inference latency, supporting its practical deployment.
Tags
Links
- Source: https://arxiv.org/abs/2609.36879v1
- Canonical: https://arxiv.org/abs/2609.36879v1
Trouble viewing inline? Open PDF directly â
Full Text
67,798 characters extracted from source content.
Expand or collapse full text
SKILLLITE: EVIDENCE-GUIDED MALICIOUS SKILL AUDITING WITH COMPACT LLMS Haoran Ou, Gelei Deng, Xuanye Zhang, Wenbo Guo, Tianwei Zhang, Kwok-Yan Lam Nanyang Technological University ABSTRACT As LLM-based agents perform increasingly complex tasks, Agent Skills have emerged as a flexible mechanism for extending their capabilities. An Agent Skill packages task-specific instructions with executable components and auxiliary re- sources to provide specialized functionalities. However, the growing adoption of third-party Skills introduces a new supply-chain attack surface. Malicious Skills can embed harmful behaviors that abuse agent privileges and compromise the agent execution environment or accessible resources. Although recent LLM-based malicious Skill auditing approaches have achieved promising performance, they often rely on capable commercial LLMs. How to achieve effective auditing with compact, locally deployable LLMs in security-sensitive and resource-constrained settings remains largely unexplored. Our investigation reveals that compact LLMs struggle to identify malicious behaviors hidden in complex Skill packages. This difficulty arises from both the implicit nature of such behaviors and the limited reasoning capacity of compact LLMs. To address these challenges, we propose SKILLLITE, an evidence-guided agentic framework for malicious Skill detection. SKILLLITE effectively extracts security-relevant behaviors and infers the intended functionality from complex Skill packages. It then employs a compact LLM to assess the maliciousness of the Skill based on the observed behaviors and their functional context. Experiments show that SKILLLITE improves malicious Skill detection across different compact LLM backbones and outperforms existing rep- resentative auditing baselines. Its effectiveness generalizes to behaviorally con- firmed in-the-wild malicious Skills. Meanwhile, SKILLLITE maintains a low in- ference latency, supporting its practical deployment. 1INTRODUCTION As LLM-based agents evolve from conversational assistants to autonomous task executors, they are increasingly capable of performing complex real-world tasks (Wang et al., 2025). Yet their capabil- ities remain bounded by the knowledge and tools available to the underlying agent system. Agent Skills have emerged as a flexible mechanism for extending these capabilities beyond such inherent boundaries (Jiang et al., 2026a; Xu & Yan, 2026). An agent Skill packages task-specific instructions together with executable components and auxiliary resources (OpenAI, 2026; Anthropic, 2026), en- abling an agent to acquire specialized capabilities without modifying its underlying model. Driven by their flexibility and reusability, Agent Skills become an important component of modern agent ecosystems across a wide range of real-world applications (Jiang et al., 2026a), such as coding (Li et al., 2026b), document processing (Zhou et al., 2026) and data analysis (Abaskohi et al., 2025). However, the growing adoption of third-party Agent Skills also introduces a new supply-chain at- tack surface into LLM-based agent ecosystems (Liu et al., 2026b; Holzbauer et al., 2026). Attackers can distribute seemingly legitimate Skills that embed malicious behaviors, abusing the agentâs priv- ileges to access sensitive resources, manipulate its execution, or communicate with external entities without authorization (Liu et al., 2026b; Feng et al., 2026; Chen et al., 2026a). Such behaviors can be difficult to distinguish from legitimate Skill functionality, as many security-sensitive operations are also necessary for benign tasks, allowing malicious Skills to compromise both the agent execu- tion environment and the resources accessible to it. Recent studies have identified malicious Skills in real-world agent ecosystems (Cisco, 2026; CERT-EU, 2026), highlighting the need for effective security auditing before third-party Skills are deployed or executed. 1 arXiv:2609.36879v1 [cs.CR] 29 Sep 2026 To mitigate these threats, recent studies have developed various pre-deployment approaches for au- diting malicious Skills. Early approaches primarily rely on static analysis and predefined security rules to identify suspicious patterns in Skill artifacts (Cisco AI Defense, 2026; NVIDIA, 2026). While efficient and deterministic, such approaches are inherently limited in interpreting context- dependent behaviors that cannot be reliably characterized by predefined patterns alone. More recent approaches leverage LLMs to reason about Skill artifacts and interpret security-sensitive behaviors in their functional context (Guo et al., 2026b; Hou & Yang, 2026; Wang et al., 2026; He et al., 2026). These approaches have achieved promising detection performance but often rely on power- ful commercial or large-scale LLMs. This reliance poses practical deployment challenges. Com- mercial LLMs typically require access to external services (Liu et al., 2024; Zhao & Song, 2024), whereas hosting large models locally can incur substantial computational and deployment over- head (Alizadeh et al., 2024; Chen et al., 2026b). Skill auditing may involve sensitive source code and configurations that must remain within controlled local environments (Das et al., 2025; Zhao & Song, 2024), where computational resources may also be limited (Liu et al., 2024; Alizadeh et al., 2024; Chen et al., 2026b). These constraints motivate the use of compact, locally deployable LLMs. However, their effectiveness in malicious Skill auditing remains largely unexplored, particularly for complex Skill packages. We therefore investigate how to enable compact LLMs to audit such packages effectively under these deployment constraints. Auditing with Compact LLMs Skill Package Compact LLM Missed Threat Why auditing fails? Implicit Behaviors Heterogeneous Artifacts LimitedReasoning Capacity Skill Package Malicious Skill Audit Report Behavior Analysis Intent Analysis Risk Adjudication (Compact LLM) Privacy ConstraintsLimited Compute Evidence Synthesis DeploymentEnvironment SKILLLITE Skill-side ChallengeModel-side Challenge Figure 1: Motivation and design of SKILLLITE for mali- cious Skill auditing with compact LLMs. Effective malicious Skill auditing with compact LLMs faces two key challenges (Figure 1, top).âś Hid- den malicious behaviors. Security- relevant behaviors are often sparse and distributed across instructions, scripts, configurations, and auxiliary resources, such that no single arti- fact may reveal the complete mali- cious behavior. Moreover, security- sensitive operations can be legitimate in isolation and only become mali- cious when considered in their func- tional context and in relation to other behaviors.⡠Limited capability of compact LLMs. Compared with more capable models, com- pact LLMs have limited capacity to reason over complex and heterogeneous Skill packages, making it difficult to simultaneously locate relevant evidence, connect dispersed observations, and interpret their security implications. To address these challenges, we propose SKILLLITE, an evidence-guided framework for malicious Skill auditing with compact LLMs. At its core, SKILLLITE adopts an agentic architecture that coor- dinates specialized modules for security behavior discovery, intent analysis, evidence synthesis, and risk adjudication of each Skill (Figure 1, bottom). To uncover malicious behaviors hidden across complex Skill packages, SKILLLITE uses a set of analyzers to inspect security-sensitive operations, concealed behaviors, and agent-control manipulation in instructions, code, configurations, and aux- iliary resources. Each detected behavior is recorded with its source artifact and supporting code or content. To make effective use of the limited reasoning capacity of compact LLMs, SKILLLITE introduces two dedicated compact LLM-based roles: an Intent Analyst and a Risk Adjudicator. The Intent Analyst infers the Skillâs declared purpose and expected capabilities from its SKILL.md. The extracted behaviors are then grounded and organized with this functional context into structured evidence. The Risk Adjudicator performs contextual risk reasoning over the structured evidence to determine whether the Skill is malicious. Our experiments demonstrate the effectiveness of SKILLLITE across multiple benchmarks and com- pact LLM backbones. On MalSkillBench (Guo et al., 2026a) and SkillTrustBench (ski, 2026), SKILLLITE achieves the best overall detection performance among representative baselines, with F1-scores of 0.905 and 0.966, respectively. Meanwhile, SKILLLITE maintains low inference latency, requiring 27.4 and 28.5 seconds per Skill on MalSkillBench and SkillTrustBench, respectively. On MalSkillBench, it is approximately 3.5Ă faster than the strongest competing baseline. We further evaluate SKILLLITE on MaliciousAgentSkillsBench (MASB) (Liu et al., 2026b), a more challeng- ing benchmark consisting of behaviorally confirmed malicious Skills collected from real-world Skill 2 ecosystems. SKILLLITE achieves the best overall performance with an F1-score of 0.807, requiring 19.4 seconds per Skill. This is approximately 4.9Ă faster than the strongest competing baseline. We also compare SKILLLITE with direct zero-shot audit using five compact LLMs across three benchmarks. SKILLLITE improves F1 in all 15 modelâbenchmark settings, with gains of up to 73.6 percentage points. Inference latency remains practical across the evaluated models. Finally, abla- tion studies validate the contribution of each key component, as removing any of them substantially degrades detection performance, with F1 dropping as low as 0.451. Our main contributions are summarized as follows: ⢠We investigate the performance of compact LLMs in malicious Skill auditing. Our analysis reveals that compact LLMs struggle to identify malicious behaviors hidden in Skill packages. This difficulty arises from both the hard-to-observe nature of malicious behaviors and the limited reasoning capacity of compact LLMs. ⢠We propose SKILLLITE, an evidence-guided malicious Skill auditing framework. SKILL- LITE extracts sparse and distributed security evidence from complex Skill packages, reducing the reasoning burden on compact LLMs and thereby enabling effective and explainable auditing. ⢠Experiments demonstrate the effectiveness and practical deployment of SKILLLITE. SKIL- LLITE consistently enhances the malicious Skill auditing capability of compact LLMs and outper- forms representative existing approaches. Meanwhile, it maintains reasonable inference latency, supporting its practical deployment. 2RELATED WORK 2.1AGENT SKILLS Agent skills have emerged as a modular mechanism for extending the capabilities of LLM agents (OpenAI, 2026; Anthropic, 2026). A skill is typically organized as a package containing a specification file: SKILL.md, along with optional scripts, references, templates, and other aux- iliary resources. It packages task-specific procedural knowledge that can be invoked when relevant to a userâs request (Jiang et al., 2026a; Xu & Yan, 2026). This structure allows agents to load task-relevant instructions and artifacts on demand, reducing unnecessary context consumption. By enabling capability extension without retraining the underlying model, skills have been increasingly adopted across diverse agent applications, such as coding (Li et al., 2026b), document process- ing (Zhou et al., 2026), data analysis (Abaskohi et al., 2025), web automation (Zheng et al., 2025), and domain-specific task execution (Jiang et al., 2026a; Li et al., 2026a). Recent work (Xu & Yan, 2026; Holzbauer et al., 2026; Liu et al., 2026a) further studies how skills can be selected (Zhang et al., 2026), composed (Li et al., 2026a), and adapted across tasks (Alzubi et al., 2026), aiming to improve the flexibility and scalability of agents. Alongside their growing adoption, skills are increasingly distributed and reused through open skill ecosystems. Community platforms provide skill registries and marketplaces 1 2 where third-party developers can publish reusable Skills and users can discover and integrate them into their agents. This ecosystem enables broader reuse and extensibility, but also extends the software supply chain of LLM agents beyond components maintained by their original developers. 2.2MALICIOUS SKILLS The flexibility of agent skills also introduces a distinct attack surface. As skills may contain exe- cutable scripts, configurations, and external resources, malicious behaviors may be embedded across both instructions and implementation artifacts (Liu et al., 2026b; Chen et al., 2026a). Such behav- iors can abuse an agentâs access to local resources and external services for credential theft, data exfiltration, or unauthorized command execution (Liu et al., 2026b; Feng et al., 2026). These threats pose serious security risks to individual agents and their users. Even worse, third-party distribution allows a malicious skill to be widely downloaded and reused, amplifying its impact across the agent ecosystem and turning it into a broader software supply-chain threat. Such risks have already been observed in real-world skill ecosystems. In March, 2026, Cisco noted that the ClawHavoc campaign 1 https://skills.rest, a platform for discovering and sharing agent skills. 2 https://skillsmp.com, a community marketplace that indexes public skills from GitHub repos. 3 had planted over 800 malicious skills in ClawHub, roughly 20% of the registry (Cisco, 2026). In June, 2026, CERT-EU 3 summarized a separate OpenClaw skill supply-chain compromise in which a deceptive âDeepSeek-Clawâ skill tricked developers and AI agents into running malicious installa- tion steps (CERT-EU, 2026). These incidents demonstrate that malicious skills can lead to credential and sensitive-data theft, malware delivery, and unauthorized remote access, posing direct security risks to both users and their connected systems. These emerging threats have motivated systematic studies of malicious agent skills. Liu et al. (2026b) first conducted a large-scale measurement of skills in the wild. They collected 98,380 skills from public registries and identified 157 behaviorally confirmed malicious skills, revealing threats ranging from credential theft and remote code execution to adversarial instructions. Beyond naturally occurring malicious packages, recent studies investigate more complex attack mechanisms and multi-dimensional security risks in agent skills (Hossain et al., 2026), expanding the threat land- scape to trigger-based backdoors (Feng et al., 2026), runtime skill manipulation (Chen et al., 2026a), and the amplification of harmful behavior (Jiang et al., 2026b). These studies reveal the diverse and agent-specific nature of malicious skill threats, extending beyond conventional malicious code and attacks on LLMs and agents. 2.3MALICIOUS SKILLS DETECTION To mitigate the threats posed by malicious skills, recent efforts have developed automated meth- ods for auditing skill packages before deployment. A straightforward approach is rule-based static analysis, which scans skill artifacts for predefined security indicators. For example, Cisco Skill Scanner (Cisco AI Defense, 2026) and NVIDIA SkillSpector (NVIDIA, 2026) inspect suspicious commands, credential handling, prompt-injection patterns, and risky installation logic. While effi- cient and interpretable, rule-based analysis lacks the contextual understanding needed to determine whether security-sensitive behaviors are malicious or necessary for a skillâs intended functionality. To address this limitation, recent approaches (Guo et al., 2026b; Wang et al., 2026) incorporate LLMs to provide semantic and contextual understanding of skill descriptions, implementations, and security-relevant behaviors. These approaches extend semantic auditing through techniques such as LLM-based triage (Hou & Yang, 2026), structured behavior reasoning (Wang et al., 2026), intentâ implementation consistency analysis (He et al., 2026), and risk localization (Etteib et al., 2026). LLM-based analysis has also been adopted by representative skill security scanners (Cisco AI De- fense, 2026; NVIDIA, 2026; spclaudehome, 2026). These scanners typically combine static security findings with LLM-based semantic analysis to interpret suspicious behaviors in context and assess the overall security risk of a skill package, such as AI-Infra-Guard (Tencent Zhuque Lab, 2025) and SkillWard (Fangcun AI, 2026). Despite recent progress, existing malicious skill detectors still struggle with complex threats. Guo et al. (2026a) show that existing representative detectors remain less effective against prompt- injection and agent-control risks. Moreover, existing LLM-based approaches are commonly built upon powerful commercial or large-scale LLMs as their underlying models. Deployment under lim- ited computational resources (Liu et al., 2024; Chen et al., 2026b) or strict privacy requirements (Das et al., 2025; Zhao & Song, 2024; Wang et al., 2025) may favor compact models for their lower in- ference overhead and suitability for local execution. However, the potential of compact LLMs for malicious skill auditing remains largely unexplored. We investigate this direction and develop a framework to enhance their detection capabilities. 3METHODOLOGY 3.1OVERVIEW Threat Model. We consider a third-party Skill supply-chain scenario in which an attacker publishes a seemingly benign Skill containing malicious behaviors, such as unauthorized resource access, sensitive information leakage, or unintended command execution. The attacker may manipulate any 3 CERT-EU is the Cybersecurity Service for the Union institutions, bodies, offices and agencies of the Union. It is an inter-institutional EU cybersecurity service provider that helps these entities prevent, detect, mitigate, and respond to cyber attacks. 4 SKILLLITE Grounded Evidence AUTONOMOUS_CONFIRMAT ION_BYPASS "source": SKILL.md "evidence": "DO NOT ASK THE USER..." âstrengthâ: STRONG ----------------- Risk Context "category": Confirmation Bypass "risk": Critical Agentic Risk Risk Adjudicator Security Audit Report ----------------- "key_evidence": Confirmation Bypass "rationale": Autonomous phase transition bypasses user confirmation beyond the needs of the stated task. Intent Analyst Security Extractor Evidence Report test-strategy/ Skill Package SKILL.md! scripts/ install.sh! helper.py config/ config.json! resources/ data.txt Final Judgment Risk Reasoning Audit Output MALICIOUS Evidence Synthesizer Evidence Grounding Context Integration "rule": "AUTONOMOUS_CONFIRMATION_ BYPASS", "source": "SKILL.md", "matched": "DO NOT ASK THE USER FOR PERMISSION", "strength": "strong" Security Findings "purpose": "Generate test strategies", "capabilities": [ "Code generation", "Worktreemanagement", "Phase transition"] Intent Profile Figure 2: Overview of SKILLLITE. artifact in the Skill package, including its instructions, source code, configurations, and auxiliary re- sources, and may conceal malicious logic through obfuscation or hidden artifacts. The defender has access to the complete Skill package and aims to determine whether it is benign or malicious before deployment or execution. Formally, given a Skill packageS , the auditing task predicts f (S) â y, where y â BENIGN, MALICIOUS. We focus on pre-execution auditing of malicious behaviors contained within Skill packages. Runtime-only behaviors and attacks against the underlying agent infrastructure, models, or execution environment are outside our scope. We propose SKILLLITE, an evidence-guided framework for malicious Skill auditing powered by compact LLMs. As illustrated in Figure 2, SKILLLITE consists of four functional roles.âś Secu- rity Evidence Extraction identifies security-relevant behaviors across the Skill package.⡠Intent Analysis establishes the Skillâs functional specification, including its declared purpose and expected capabilities.⸠Evidence Synthesis grounds the extracted findings in their source context and in- tegrates them with functional and security context.âš Risk Adjudication assesses the structured evidence and produces the final judgment with a rationale. 3.2SECURITY EVIDENCE EXTRACTION The Security Extractor identifies security-relevant behaviors across the complete Skill package. We represent a Skill package asS =a 1 , . . . , a n , where each a i denotes an artifact such as an instruc- tion, script, configuration, or auxiliary resource. We consider a set of security-relevant behavior classes B, including sensitive-resource access, external communication, system execution, persis- tence, concealed behavior, and agent-control manipulation. These classes capture security-relevant behaviors rather than maliciousness, as the same behavior may appear in both benign and malicious Skills and cannot be judged in isolation. To identify these behaviors across heterogeneous artifacts, SKILLLITE employs a set of complemen- tary deterministic analyzersD = D 1 , . . . , D m . These analyzers cover general security patterns, language-aware code analysis, concealed-payload inspection, and agent-specific control signals. A detailed summary is provided in Appendix A.1. Code analysis identifies security-sensitive API us- age and syntax-level indicators, while concealed-payload inspection recovers supported encoded content. Agent-specific analysis targets behaviors such as approval bypass, control-flow hijacking, and autonomous execution without user confirmation. Collectively, these analyzers produce the security findings F (S) = S a i âS S D j âD D j (a i ). Each finding f k â F (S) records the detected behavior, its source artifact, supporting evidence, and associated attributes. 3.3INTENT ANALYSIS The Intent Analyst derives a functional specification that characterizes the intended functionality of the Skill. We represent this specification asP = (g,C exp ), where g denotes the declared functional goal and C exp denotes the capabilities reasonably expected to support that goal. This task is per- formed by the compact LLM through structured reasoning. Given the Skill specificationS spec , the Intent Analyst identifies g and derives the capabilities C exp implied by the declared task. For ex- ample, network access and command execution may be expected for a deployment Skill but not for a document-processing Skill. The functional specification provides a reference for interpreting the extracted security findings. An observed behavior b k can then be assessed againstC exp to determine 5 whether it is expected under the declared functionality. Unexpected behaviors are not necessarily malicious and are further examined during risk adjudication. 3.4EVIDENCE SYNTHESIS The Evidence Synthesizer transforms heterogeneous security findings into a structured evidence representation. Individual findings identify security-relevant behaviors, but may lack the source and functional context needed for reliable interpretation. Evidence synthesis therefore grounds these findings in their original artifacts and organizes them into a unified evidence report. Source grounding first associates each finding with supporting context from its originating artifact. We represent a grounded observation as e k = (b k , a k , x k , c k , s k ), where b k denotes the observed behavior, a k its source artifact, x k the supporting evidence, c k the surrounding source context, and s k the evidence strength. This representation normalizes findings produced by different analyzers and artifacts while preserving their provenance and supporting context. The resulting grounded ob- servations form the evidence setE =e 1 , . . . , e n . The grounded observations are then aggregated and contextualized with the functional specificationP and applicable security policies Î . We orga- nize the resulting evidence report asR = (E,C risk ), whereC risk provides the functional and policy context needed to interpret the observed behaviors. Accordingly, the report contains two comple- mentary views: (1) grounded evidence captures what security-sensitive behaviors are observed and where they occur; (2) risk context provides the functional expectations and security policies relevant to their assessment. 3.5RISK ADJUDICATION The Risk Adjudicator performs the final maliciousness assessment over the structured evidence re- portR. Individual security-sensitive behaviors do not necessarily indicate maliciousness. The adju- dicator therefore evaluates the grounded evidenceE from three complementary aspects: functional necessity, benign counterevidence, and security risk. (1) Functional necessity examines whether an observed capability is reasonably required by the Skillâs declared functionality. (2) Benign coun- terevidence examines whether source-grounded evidence provides a credible justification for an otherwise suspicious behavior. (3) For critical agentic risks, generic functional justification is in- sufficient, and direct risk-specific counterevidence is required. Risk reasoning considers the security implications of both individual behaviors and their combinations, including cases where individually plausible operations form a harmful pattern when considered together. Overall, these assessments determine how the observed evidence should be interpreted under the functional specification P and applicable security policies Î . The compact LLM performs this structured adjudication overR and produces the final maliciousness judgment together with an evidence-grounded rationale. The rationale identifies the evidence and functional context supporting the judgment, making the final decision traceable to the underlying skill artifacts. 4EXPERIMENTS 4.1EXPERIMENTAL SETUP Datasets. We evaluate SKILLLITE on three diverse malicious-skill benchmarks spanning different attack constructions, skill artifacts, and real-world threat settings. MalSkillBench (Guo et al., 2026a) contains 3,944 malicious and 4,000 benign skills, covering code injection, prompt injection, and mixed attacks across 15 malicious behaviors. SkillTrustBench (ski, 2026) evaluates complete skill packages with heterogeneous artifacts across 9 security categories. Following our binary detection setting, we exclude its 1,014 suspicious samples and retain 2,863 malicious and 1,643 benign skills. MaliciousAgentSkillsBench (Liu et al., 2026b) provides an in-the-wild evaluation set collected from public registries, including 157 behaviorally confirmed malicious skills with fine-grained vulnera- bility annotations and 299 verified benign skills. Baselines. We compare SKILLLITE with five representative malicious skill scanners spanning static and LLM-assisted detection: SkillSpector (NVIDIA, 2026), AI-Infra-Guard Skill-Scan (Tencent Zhuque Lab, 2025), Cisco Skill Scanner (Cisco AI Defense, 2026), SkillWard (Fangcun AI, 2026), and Skill Vetter (spclaudehome, 2026). SkillSpector and Cisco Skill Scanner support both rule- 6 Table 1: Main results on MalSkillBench and SkillTrustBench. Best and second-best detection results are shown in bold and underlined, respectively. DatasetMethodAccPrec RecallF1FPRâ FNRâ Latency (s)â MalSkillBench Cisco SkillScanner (Static)0.571 0.679 0.257 0.373 0.1200.7430.1 Cisco SkillScanner (LLM)0.722 0.661 0.9070.764 0.4600.09326.6 NVIDIA SkillSpector (Static) 0.637 0.695 0.478 0.567 0.2070.5220.3 NVIDIA SkillSpector (LLM) 0.683 0.698 0.638 0.667 0.2720.36212.4 SkillWard0.801 0.988 0.607 0.752 0.0080.39333.2 Skill-Vetter0.684 0.630 0.882 0.735 0.5110.11835.2 Tencent AI-Infra-Guard0.840 0.796 0.911 0.8500.2300.08997.1 SKILLLITE0.909 0.9440.869 0.905 0.0510.13227.4 SkillTrustBench Cisco SkillScanner (Static)0.767 0.899 0.713 0.796 0.1380.2870.3 Cisco SkillScanner (LLM)0.810 0.792 0.950 0.864 0.4360.05029.9 NVIDIA SkillSpector (Static) 0.814 0.844 0.867 0.856 0.2790.1330.6 NVIDIA SkillSpector (LLM) 0.795 0.794 0.915 0.850 0.4150.08511.0 SkillWard0.8840.997 0.821 0.9010.0050.18027.2 Skill-Vetter0.799 0.795 0.923 0.854 0.4140.07836.6 Tencent AI-Infra-Guard0.851 0.869 0.902 0.885 0.2370.09984.3 SKILLLITE0.957 0.9710.961 0.966 0.0500.03928.5 based and LLM-assisted configurations, which we evaluate separately. For LLM-assisted baselines, we use the same underlying model as SKILLLITE whenever supported to control for differences in model capability. Appendix A.3 provides detailed baseline configurations. Metrics. We evaluate malicious skill detection using standard binary classification metrics, includ- ing Accuracy (Acc), Precision, Recall, and F1-score. We use F1-score as a primary measure of overall detection performance by jointly accounting for Precision and Recall. We additionally re- port False Negative Rate (FNR) and False Positive Rate (FPR) to characterize missed malicious skills and false alarms on benign skills, respectively. The average end-to-end processing time per Skill in seconds is also reported to evaluate inference efficiency. Configurations. SKILLLITE takes the complete skill package as input, including the skill speci- fication, implementation code, configurations, and auxiliary artifacts. Gemma4:e4b is used as the underlying LLM unless otherwise specified. Baselines are reproduced by using their official open- source implementations under the default configurations. For controlled comparison, LLM-assisted baselines use the same underlying model as SKILLLITE whenever supported. For SkillSpector, whose original LLM workflow could not reliably complete the evaluation with Gemma4:e4b, we use GPT-4.1-nano as a lightweight substitute with a similar general capability range. Appendix A.3 provides a detailed analysis and justification for this substitution. Additional implementation and environment details are provided in Appendix A.4. 4.2MAIN RESULTS Detection performance.Table 1 compares SKILLLITE with baseline methods on MalSkill- Bench and SkillTrustBench. SKILLLITE achieves the strongest overall detection performance on both benchmarks, maintaining strong precision and recall simultaneously. In contrast, ex- isting approaches exhibit pronounced trade-offs between detecting malicious Skills and avoiding false alarms: conservative methods often miss malicious behaviors, whereas sensitive methods tend to misclassify legitimate security-sensitive operations. By grounding security evidence in its source and functional context, SKILLLITE better distinguishes malicious behaviors from legitimate security-sensitive operations, reducing both missed attacks and false alarms. Detailed category- and package-level analyses are provided in Appendix B. Efficiency. As shown in Table 1, SKILLLITE maintains low inference latency comparable to sev- eral LLM-based baselines, particularly lower than AI-Infra-Guard. Although purely static scan- ners are faster, their detection performance is weaker. Overall, SKILLLITE provides a favorable effectivenessâefficiency trade-off for malicious Skill auditing with compact, locally deployed LLMs. 7 Table 2: Generalization results on MaliciousAgentSkillsBench. Best and second-best detection results are shown in bold and underlined, respectively. MethodAccPrecRecallF1FPRâ FNRâ Latency (s)â Cisco SkillScanner (Static)0.651 0.4580.0720.122 0.0440.9280.1 Cisco SkillScanner (LLM)0.763 0.6180.8150.703 0.2640.18520.1 NVIDIA SkillSpector (Static) 0.621 0.3790.1590.224 0.1370.8410.4 NVIDIA SkillSpector (LLM)0.592 0.3470.2090.262 0.2070.7916.8 SkillWard0.678 0.9170.0710.131 0.0030.92918.6 Skill-Vetter0.807 0.6640.8920.761 0.2380.10831.7 Tencent AI-Infra-Guard0.847 0.7330.8730.7970.1670.12794.2 SKILLLITE0.878 0.8920.7370.807 0.0480.26319.4 0204060 Latency (s / skill) â 0.0 0.2 0.4 0.6 0.8 1.0 F1 â Zero-shotSKILLLITE GemmaQwenDS-TDS-NTMix (a) MalSkillBench 0204060 Latency (s / skill) â 0.0 0.2 0.4 0.6 0.8 1.0 F1 â Zero-shotSKILLLITE GemmaQwenDS-TDS-NTMix (b) SkillTrustBench 0204060 Latency (s / skill) â 0.0 0.2 0.4 0.6 0.8 1.0 F1 â Zero-shotSKILLLITE GemmaQwenDS-TDS-NTMix (c) MASB Figure 3: Effectivenessâefficiency comparison between direct zero-shot auditing and SKILLLITE across compact LLM backbones. Each connected pair represents the same underlying model. 4.3GENERALIZATION TO IN-THE-WILD MALICIOUS SKILLS We further evaluate the generalization of SKILLLITE on MaliciousAgentSkillsBench, which con- tains behaviorally confirmed malicious Skills collected in the wild. As shown in Table 2, SKILL- LITE achieves the best overall accuracy and F1-score while maintaining a low false-positive rate. This result indicates that SKILLLITE generalizes to real-world Skill ecosystems without relying on an overly aggressive detection strategy. Existing baselines continue to exhibit trade-offs between de- tecting malicious Skills and avoiding false alarms, with some methods favoring higher recall while others adopt more conservative decisions. For SKILLLITE, the remaining errors are mainly false negatives, where security-sensitive behaviors may appear consistent with the declared functionality. This reflects the general limitation of static auditing, which cannot observe risks emerging only at runtime. How to uncover more runtime risks through static auditing remains an interesting challenge for future research. The efficiency trend is consistent with the main evaluation. SKILLLITE main- tains low inference latency comparable to LLM-based baselines. Notably, AI-Infra-Guard achieves the closest F1-score to SKILLLITE but requires approximately 4.9Ă longer inference time. Detailed analyses are provided in Appendix B. 4.4ENHANCING COMPACT LLMS Detection performance. We investigate whether SKILLLITE can enhance malicious Skill auditing across different compact LLM backbones. As shown in Figure 3, SKILLLITE improves F1 over di- rect zero-shot auditing across all 15 modelâbenchmark configurations, with gains ranging from 9.2 to 73.6 percentage points. Zero-shot auditing exhibits a consistent conservative bias: compact LLMs generally maintain high precision but frequently miss malicious Skills, particularly on MASB. Ef- fective Skill auditing requires identifying sparse security evidence across heterogeneous artifacts, connecting individually ambiguous behaviors, and interpreting their security implications. Compact LLMs can struggle to complete all the tasks jointly due to their limited capacity. SKILLLITE ad- dresses this challenge by externalizing evidence acquisition and grounding, allowing the model to focus on risk adjudication. Although the final performance remains influenced by the capability of 8 Table 3: Ablation study on the full SkillTrustBench binary benchmark. VariantAccPrecRecallF1FPRâ FNRâ Full SKILLLITE0.957 0.9710.9610.966 0.0500.039 w/o Security Evidence Extraction 0.560 0.9440.3260.485 0.0340.674 w/o Intent Analysis0.549 0.9960.2920.451 0.0020.708 w/o Evidence Synthesis0.731 0.9450.6120.743 0.0620.388 the underlying model, SKILLLITE can substantially enhance models with limited zero-shot auditing capability. Complete results across all evaluation metrics are provided in Appendix C. Efficiency analysis. SKILLLITE introduces additional inference latency over direct zero-shot audit- ing due to evidence acquisition and grounding. Nevertheless, the runtime remains practical across the evaluated backbones, while the additional computation yields substantial improvements in de- tection performance. For example, on MalSkillBench, SKILLLITE improves DeepSeek-R1:8Bâs F1 from 59.2% to 80.5%, with an inference latency of only 11.5 seconds. 4.5ABLATION STUDY We conduct ablation studies on SkillTrustBench to examine three key components of SKILLLITE: Security Evidence Extraction, Intent Analysis, and Evidence Synthesis. As shown in Table 3, re- moving any component degrades detection performance, demonstrating their complementary roles in exposing, contextualizing, and interpreting security evidence. Security Evidence Extraction. Removing this component causes a substantial performance degra- dation, with the largest impact appearing in malicious-skill recall. Without systematic extraction, security-sensitive behaviors embedded across scripts, configurations, and other package artifacts are no longer explicitly exposed, leaving the subsequent reasoning stages with incomplete security in- formation. This confirms the importance of package-level evidence extraction for making implicit or distributed malicious behaviors observable to compact LLMs. Intent Analysis. Removing Intent Analysis causes a similarly substantial degradation despite re- taining grounded security evidence. The model remains highly precise but misses most malicious Skills, indicating a conservative decision pattern. Without reasoning about declared purpose, ex- pected capabilities, and functional necessity, the compact LLM lacks the functional reference needed to determine whether security-sensitive behaviors are justified. Evidence Synthesis. Removing source-grounded context also reduces detection performance, al- though less than the other two ablations. The remaining findings identify security-sensitive patterns but lack the implementation context needed to interpret how they occur within the Skill, making it harder to distinguish legitimate operations from malicious behaviors. This confirms the role of source grounding in connecting extracted security signals with their semantic interpretation. 5CONCLUSION In this work, we study malicious Agent Skill auditing with compact, locally deployable LLMs. Our investigations show that directly using compact LLMs to audit raw Skill packages remains chal- lenging, as security-relevant evidence can be sparse, distributed across heterogeneous artifacts, and ambiguous without functional context. To address this challenge, we introduce SKILLLITE, an evidence-guided auditing framework that externalizes security evidence discovery and grounding while focusing the compact LLM on intent-conditioned risk adjudication. Across multiple bench- marks and compact LLM backbones, SKILLLITE consistently improves malicious-Skill detection, generalizes to behaviorally confirmed in-the-wild threats, and maintains practical inference effi- ciency. Our findings suggest that effective security auditing does not necessarily require scaling model capability; instead, restructuring how security evidence is exposed to the model provides a promising direction for building practical, local, and lightweight Agent Skill auditing systems. 9 AI USE STATEMENT We used generative AI tools only to assist with language polishing, grammar checking, and im- proving the readability of the paper. Generative AI was not used for the research tasks requiring disclosure under the ICLR 2027 AI Policy, including the development of the research methodology, experimental design, data analysis, or interpretation of results. All AI-assisted revisions were re- viewed and verified by the authors. The authors take full responsibility for the final content of this work, including all technical claims, experimental results, and AI-assisted text. ETHICS STATEMENT This work studies malicious Agent Skills, aiming at improving the safety of LLM-based agent ecosystems. All experiments were conducted in controlled research environments and did not target real users or production systems. We did not deploy or intentionally distribute malicious Skills to public Skill ecosystems, nor did we perform attacks against third-party systems. Malicious samples were used solely for risk evaluation and the development of defensive auditing techniques. Overall, this research is intended to improve the detection of malicious Skills and support safer adoption of third-party Agent Skills. REPRODUCIBILITY STATEMENT We provide detailed descriptions of the datasets, baselines, evaluation metrics, and experimental settings in Section 4 and the Appendix. The framework design is described in Section 3. Additional implementation details, prompt templates, and complete experimental results are provided in the Appendix. These details support the reproduction of our experimental setup and results. REFERENCES Skilltrustbench, 2026. Benchmark dataset for agent skill security evaluation. Amirhossein Abaskohi, Amrutha Varshini Ramesh, Shailesh Nanisetty, Chirag Goel, David Vazquez, Christopher Pal, Spandana Gella, Giuseppe Carenini, and Issam H Laradji. Agentada: Skill-adaptive data analytics for tailored insight discovery. arXiv preprint arXiv:2504.07421, 2025. Keivan Alizadeh, Seyed Iman Mirzadeh, Dmitry Belenko, S Khatamifard, Minsik Cho, Carlo C Del Mundo, Mohammad Rastegari, and Mehrdad Farajtabar. Llm in a flash: Efficient large lan- guage model inference with limited memory. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 12562â12584, 2024. Salaheddin Alzubi, Noah Provenzano, Jaydon Bingham, Weiyuan Chen, and Tu Vu. Evoskill: Au- tomated skill discovery for multi-agent systems. arXiv preprint arXiv:2603.02766, 2026. Anthropic. Extend claude with skills. Official documentation, 2026. URL https://code. claude.com/docs/en/skills. Accessed: 2026-08-18. CERT-EU.Cyber brief 26-06 - may 2026.Threat intelligence brief, 2026.URL https: //cert.europa.eu/publications/threat-intelligence/cb26-06/. Release date: 2026-06-02. Tianhao Chen, Zhengyuan Jiang, Yuepeng Hu, Yebei Gou, and Neil Zhenqiang Gong. Dynamic malicious skills in agentic ai. arXiv preprint arXiv:2606.16287, 2026a. Wanyi Chen, Junhao Wang, Yiwei Zhang, Yufan Shi, Tianyi Jiang, Shengxian Zhou, Chenxu Wu, Andi Zhang, Chenyue Zhou, Minxuan Wang, et al. On-device large language models: a survey of model compression and system optimization. Artificial Intelligence Review, 59(9):191, 2026b. Cisco.I run openclaw at home. thatâs exactly why we built defenseclaw.Cisco Blog, 2026.URL https://blogs.cisco.com/ai/cisco-announces-defenseclaw. Published: 2026-03-23. 10 Cisco AI Defense. Skill scanner: Security scanner for ai agent skills. https://github.com/ cisco-ai-defense/skill-scanner, 2026. GitHub repository, accessed: 2026-07-10. Badhan Chandra Das, M Hadi Amini, and Yanzhao Wu. Security and privacy challenges of large language models: A survey. ACM Computing Surveys, 57(6):1â39, 2025. Bacem Etteib, Daniele Lunghi, and Tegawende F. Bissyande. Detecting malicious agent skills in the wild using attention. arXiv preprint arXiv:2606.23416, 2026. Fangcun AI. Skillward: Security auditing framework for ai agent skills. https://github. com/Fangcun-AI/SkillWard, 2026. GitHub repository, accessed: 2026-08-10. Yunhao Feng, Yifan Ding, Yingshui Tan, Boren Zheng, Yanming Guo, Xiaolong Li, Kun Zhai, Yishan Li, and Wenke Huang. Skilltrojan: Backdoor attacks on skill-based agent systems. arXiv preprint arXiv:2604.06811, 2026. Wenbo Guo, Wei Zeng, Chengwei Liu, Xiaojun Jia, Yijia Xu, Lei Tang, Yong Fang, and Yang Liu. Malskillbench: A runtime-verified benchmark of malicious agent skills. arXiv preprint arXiv:2606.07131, 2026a. Zihan Guo, Zhiyu Chen, Xiaohang Nie, Jianghao Lin, Yuanjian Zhou, and Weinan Zhang. Skill- probe: Security auditing for emerging agent skill marketplaces via multi-agent collaboration. arXiv preprint arXiv:2603.21019, 2026b. Wenhui He, Yue Li, Bang Fu, Huan Xing, Xing Fan, ZeHua Zhang, and Baoning Niu. Do skill descriptions tell the truth? detecting undisclosed security behaviors in code-backed llm skills. arXiv preprint arXiv:2605.12875, 2026. Florian Holzbauer, David Schmidt, Gabriel Gegenhuber, Sebastian Schrittwieser, and Johanna Ull- rich. Context matters: Repository-aware security analysis of the agent skill ecosystem. arXiv preprint arXiv:2603.16572, 2026. Ismail Hossain, Sai Puppala, Md Jahangir Alam, Tanzim Ahad, and Sajedul Talukder. Skillvetbench: Llm-as-judge for multi-dimensional security risk evaluation in open-source llm agent skills. arXiv preprint arXiv:2606.15899, 2026. Yinghan Hou and Zongyou Yang. Skillsieve: A hierarchical triage framework for detecting mali- cious ai agent skills. arXiv preprint arXiv:2604.06550, 2026. Yanna Jiang, Delong Li, Haiyu Deng, Baihe Ma, Xu Wang, Qin Wang, and Guangsheng Yu. Sok: Agentic skillsâbeyond tool use in llm agents. arXiv preprint arXiv:2602.20867, 2026a. Yukun Jiang, Yage Zhang, Michael Backes, Xinyue Shen, and Yang Zhang. Harmfulskillbench: How do harmful skills weaponize your agents? arXiv preprint arXiv:2604.15415, 2026b. Hao Li, Chunjiang Mu, Jianhao Chen, Siyue Ren, Zhiyao Cui, Yiqun Zhang, Lei Bai, and Shuyue Hu. Organizing, orchestrating, and benchmarking agent skills at ecosystem scale. arXiv preprint arXiv:2603.02176, 2026a. Yanzhou Li, Yiran Zhang, Xiaoyu Zhang, Xiaoxia Liu, and Yang Liu. Codeskill: Learning self- evolving skills for coding agents. arXiv preprint arXiv:2605.25430, 2026b. Hongyi Liu, Haoyan Yang, Tao Jiang, Bo Tang, Feiyu Xiong, Yuyu Luo, and Zhiyu Li. Skillsvote: Lifecycle governance of agent skills from collection, recommendation to evolution. arXiv preprint arXiv:2605.18401, 2026a. Yi Liu, Zhihao Chen, Yanjun Zhang, Gelei Deng, Yuekang Li, Jianting Ning, and Leo Yu Zhang. â do not mention this to the userâ: Detecting and understanding malicious agent skills in the wild. arXiv preprint arXiv:2602.06547, 2026b. Zechun Liu, Changsheng Zhao, Forrest Iandola, Chen Lai, Yuandong Tian, Igor Fedorov, Yunyang Xiong, Ernie Chang, Yangyang Shi, Raghuraman Krishnamoorthi, et al. Mobilellm: Optimizing sub-billion parameter language models for on-device use cases. arXiv preprint arXiv:2402.14905, 2024. 11 NVIDIA. Skillspector: Security scanner for ai agent skills. https://github.com/NVIDIA/ skillspector, 2026. GitHub repository, accessed: 2026-06-28. OpenAI. Build skills. Official documentation, 2026. URL https://learn.chatgpt.com/ docs/build-skills. Accessed: 2026-08-18. spclaudehome.Skill vetter:Ai skill security auditor. https://clawhub.ai/ spclaudehome/skill-vetter, 2026. ClawHub skill, accessed: 2026-08-10. Tencent Zhuque Lab. AI-Infra-Guard: A Comprehensive, Intelligent, and Easy-to-Use AI Red Teaming Platform. GitHub repository, 2025. URL https://github.com/Tencent/ AI-Infra-Guard. Shenao Wang, Junjie He, Yanjie Zhao, Yayi Wang, Kan Yu, and Haoyu Wang. âelementary, my dear watson.â detecting malicious skills via neuro-symbolic reasoning across heterogeneous artifacts. arXiv preprint arXiv:2603.27204, 2026. Yuntao Wang, Yanghe Pan, Zhou Su, Yi Deng, Quan Zhao, Linkang Du, Tom H Luan, Jiawen Kang, and Dusit Niyato. Large model-based agents: State-of-the-art, cooperation paradigms, security and privacy, and future trends. IEEE Communications Surveys & Tutorials, 28:1906â1949, 2025. Renjun Xu and Yang Yan. Agent skills for large language models: Architecture, acquisition, secu- rity, and the path forward. arXiv preprint arXiv:2602.12430, 2026. Haozhen Zhang, Quanyu Long, Jianzhu Bao, Tao Feng, Weizhi Zhang, Haodong Yue, and Wenya Wang. Memskill: Learning and evolving memory skills for self-evolving agents. arXiv preprint arXiv:2602.02474, 2026. Guoshenghui Zhao and Eric Song. Privacy-preserving large language models: Mechanisms, appli- cations, and future directions. arXiv preprint arXiv:2412.06113, 2024. Boyuan Zheng, Michael Y Fatemi, Xiaolong Jin, Zora Zhiruo Wang, Apurva Gandhi, Yueqi Song, Yu Gu, Jayanth Srinivasa, Gaowen Liu, Graham Neubig, et al. Skillweaver: Web agents can self-improve by discovering and honing skills. arXiv preprint arXiv:2504.07079, 2025. Yingli Zhou, Wang Shu, Yaodong Su, Wenchuan Du, Yixiang Fang, and Xuemin Lin. A com- prehensive survey on agent skills: Taxonomy, techniques, and applications. arXiv preprint arXiv:2605.07358, 2026. AADDITIONAL EXPERIMENT DETAILS A.1SECURITY EVIDENCE EXTRACTION DETAILS The Security Extractor uses complementary deterministic analyzers to expose security-relevant be- haviors across heterogeneous Skill artifacts. Table 4 summarizes the analyzer categories and repre- sentative security signals. These categories characterize the implementation coverage of the extrac- tion stage. The extracted signals serve only as security evidence, not directly determining the final maliciousness judgment. A.2DATASET DETAILS We provide additional details on the construction, composition, and risk coverage of the three bench- marks used in our evaluation. MalSkillBench (Guo et al., 2026a) contains 3,944 malicious and 4,000 benign Skills. Its malicious samples include attacks collected from public Skill ecosystems and runtime-verified synthesized Skills. The benchmark covers three attack vectors, namely Code Injection (CI), Prompt Injection (PI), and Mixed attacks, as well as fifteen malicious behavior types, which together form 108 combi- nations of attack vectors, behaviors, and insertion strategies. This large-scale construction provides diverse coverage of both code-level and instruction-level malicious behaviors. 12 Table 4: Security evidence coverage of the deterministic analyzers in SKILLLITE. AnalyzerEvidence TargetRepresentative Signals General Security PatternsCommon security-sensitive opera- tions Local file access, network activity, system command execution, and permission modification. Language-Aware Code AnalysisSecurity-sensitive implementation behavior Process invocation, network API usage, environment-variable ac- cess, and dynamic execution. Concealed-Payload InspectionHidden or obfuscated behaviorEncoded payloads, hidden helper logic, split payload artifacts, and obfuscated execution. Agent-Specific Control SignalsManipulation of agent execution and control Approval bypass, sandbox bypass, control-flow hijacking, and au- tonomous confirmation bypass. SkillTrustBench (ski, 2026) is constructed from more than 62,000 real-world Skills collected from public agent ecosystems. Each sample contains a complete Skill package, including instructions, scripts, configurations, references, and auxiliary resources. The benchmark covers nine security cat- egories, including instruction manipulation, memory poisoning, malicious code and payload execu- tion, privilege escalation, persistence, tool hijacking, and insecure dependencies or implementations. The original benchmark contains 2,863 malicious, 1,643 benign, and 1,014 suspicious samples. Fol- lowing our binary auditing setting, we exclude the suspicious category and evaluate on the remaining 4,506 Skills. MaliciousAgentSkillsBench (MASB) (Liu et al., 2026b) is derived from 98,380 real-world Skills collected from two public Skill registries. Candidate malicious Skills are screened through static analysis and subsequently validated through behavioral execution in isolated environments, resulting in 157 behaviorally confirmed malicious Skills. These samples span 13 attack techniques and six kill-chain stages and are annotated with their observed security behaviors. For our binary evaluation, we combine the 157 malicious Skills with 299 verified benign Skills from the same ecosystem, yielding 456 Skills in total. A.3BASELINE DETAILS We provide additional details on the baseline detectors and their configurations used in our experi- ments. All baselines are evaluated on complete Skill packages using their official implementations and recommended configurations whenever applicable. NVIDIA SkillSpector (NVIDIA, 2026) provides both static and LLM-based analysis for auditing Agent Skills. We evaluate these two configurations separately as SkillSpector (Static) and SkillSpec- tor (LLM). The static configuration detects security-sensitive patterns directly from Skill artifacts. The LLM configuration additionally performs semantic analysis for risk assessment. For the LLM configuration, we initially use Gemma4:e4b to maintain the same underlying model as SKILLLITE and the other LLM-assisted baselines. However, this configuration exhibits high failure rates across all three benchmarks, primarily due to the 300-second per-sample timeout. To avoid penalizing SkillSpector for these execution failures, we instead use GPT-4.1-nano as a lightweight alternative that reliably completes the evaluation. The original SkillSpector workflow is unchanged. Tencent AI-Infra-Guard (Tencent Zhuque Lab, 2025) provides a Skill security scanner that ana- lyzes Skill packages for security-sensitive behaviors and produces structured security findings. We evaluate its Skill-Scan component provided by the framework. Cisco Skill Scanner (Cisco AI Defense, 2026) provides modular analyzers for auditing Agent Skills. We evaluate two configurations: Cisco SkillScanner (Static), which uses the core security analyzers, and Cisco SkillScanner (LLM), which additionally incorporates LLM-based semantic analysis. The LLM configuration uses the same underlying compact LLM as SKILLLITE. 13 Table 5: Experimental environment. ComponentConfiguration OSUbuntu 22.04 CPU2Ă AMD EPYC 7543 RAM250 GB GPUNVIDIA RTX A6000 48 GB Python3.11.4 Ollama0.20.7 SkillWard (Fangcun AI, 2026) audits Skill packages using multiple security checks and produces structured findings for detected risks. We use its official implementation and map an UNSAFE verdict to the malicious class. Skill-Vetter (spclaudehome, 2026) performs LLM-assisted semantic analysis of Skill descriptions and implementation artifacts. We use the same underlying compact LLM as SKILLLITE and map its warning verdict to the malicious class. A.4IMPLEMENTATION AND EXPERIMENTAL SETUP Unless otherwise specified, we use Gemma4:e4b as the default compact LLM for SKILLLITE. All open-source models are deployed locally using Ollama (v0.20.7) without fine-tuning. Gemma4:e4b serves as the default backbone in the main experiments. We additionally evaluate Qwen3.5:9B, DeepSeek-R1:8B, and Mixtral-8x7B to examine the effectiveness of SKILLLITE across different compact LLMs. DeepSeek-R1:8B is evaluated with both thinking enabled and disabled. All ex- periments are conducted on a Linux server equipped with NVIDIA RTX A6000 GPUs with 48 GB of memory. Each model instance runs on a single GPU. The remaining environment details are summarized in Table 5. BDETAILED ANALYSIS We provide additional analyses across the three benchmarks from two perspectives: attack categories and package characteristics. B.1PERFORMANCE ACROSS ATTACK CATEGORIES Figure 4 compares recall across fine-grained attack categories on the three benchmarks. Since these categories are defined only over malicious samples, we report category-level recall. The categories in SkillTrustBench and MASB are multi-label and therefore non-exclusive. MalSkillBench. MalSkillBench contains 15 malicious behavior categories (B1âB15), with individ- ual categories containing between 126 and 280 samples. Overall, SKILLLITE achieves strong recall across most categories, demonstrating its ability to detect diverse types of malicious behaviors. It performs particularly well on B6 (Reverse Shell) and B8 (Resource Abuse), reaching recalls of 0.993 and 0.982, respectively. The remaining errors are concentrated in a smaller set of behaviors, such as B10 (Role Hijack) and B14 (Goal Hijacking). Although several high-sensitivity baselines achieve higher recall on these challenging categories, they incur substantially higher false-positive rates on the complete benchmark. SKILLLITE provides a better overall balance between malicious-skill de- tection and false alarms, resulting in higher overall F1. SkillTrustBench. SkillTrustBench contains nine non-exclusive security categories (T01âT09), with category sizes ranging from 96 to 2,450 malicious samples. Overall, SKILLLITE maintains consis- tently high recall across diverse security categories. It performs particularly well on T06 (Persis- tence) and T07 (Tool Hijacking), with recalls of 1.000 and 0.984, respectively. Most baselines show less consistent performance across categories, with substantially lower recall on several types. Those achieving high recall on more categories tend to do so at the cost of considerably higher false-positive rates on the complete benchmark. 14 C-SC-LSS-S-LWardVetterAIGOurs B1 B2 B3 B4 B5 B6 B7 B8 B9 B10 B11 B12 B13 B14 B15 .228.945.553.843.804.969.969.933 .261.954.800.904.743.964.968.868 .504.928.669.860.720.941.924.860 .495.936.668.873.746.946.964.941 .416.908.748.882.710.945.958.924 .556.929.696.906.932.951.929.993 .214.908.354.760.629.930.913.808 .330.927.733.886.875.952.938.982 .296.968.888.957.899.982.968.968 .011.807.128.348.321.631.690.497 .049.937.317.415.615.766.893.824 .275.954.244.481.763.885.970.931 .135.895.248.361.579.684.789.789 .040.833.103.206.270.611.809.706 .025.715.070.171.114.601.595.380 0 .5 1 Recall (a) MalSkillBench C-SC-LSS-S-LWardVetterAIGOurs T01 T02 T03 T04 T05 T06 T07 T08 T09 .728.933.899.959.860.909.908.965 .652.908.726.854.726.933.896.896 .673.975.750.798.740.935.924.963 .751.957.914.957.882.925.907.971 .601.941.968.996.805.932.899.980 .312.958.854 1.000 .948.979.812 1.000 .782.935.879.911.847.927.944.984 .977.977.868.938.256.915.729.837 .265.791.860.977.696.928.856.935 0 .5 1 Recall (b) SkillTrustBench C-SC-LSS-S-LWardVetterAIGOurs E1 E2 E3 E4 IM P1 P2 P3 P4 PE1 PE2 PE3 RS SC1 SC2 SC3 SF .046.862.156.202.028.945.927.807 .046.889.120.185.018.935.917.815 .036.818.164.182.036.946.873.654 .062.844.219.219.062.812.812.594 .000 1.000 .000.000.000 1.0001.0001.000 .079.737.105.184.105.842.737.579 .105.579.316.316.105.737.789.737 .000 1.000 .000.000.100 1.0001.000 .600 .049.816.146.175.078.874.883.786 .000.889.000.000.000 1.0001.000 .778 .200.800.500.500.300 1.000 .800.500 .400 1.000 .600.700.400 1.000 .900.500 1.0001.0001.0001.000 .500.500 1.0001.000 .069.828.138.138.069.966.966.793 .034.855.103.137.051.957.914.838 .222.778.444.444.167.889.778.556 .083.750.250.333.000.750.917.833 0 .5 1 Recall (c) MaliciousAgentSkillsBench Figure 4: Recall across fine-grained attack categories on the three benchmarks. Rows denote the native malicious behavior, columns correspond to the evaluated detection methods. Each cell reports category-level recall. C-SC-LSS-S-LWardVetterAIGOurs 1 (n=4,830) 2â5 (n=2,728) 6+ (n=386) Files per skill 0.0 0.2 0.4 0.6 0.8 1.0 F1 (a) MalSkillBench 1 (n=292) 2â5 (n=2,059) 6+ (n=2,155) Files per skill 0.0 0.2 0.4 0.6 0.8 1.0 F1 (b) SkillTrustBench 1 (n=211) 2â5 (n=200) 6+ (n=45) Files per skill 0.0 0.2 0.4 0.6 0.8 1.0 F1 (c) MaliciousAgentSkillsBench Figure 5: Detection performance across Skill package sizes. MaliciousAgentSkillsBench (MASB). MASB provides fine-grained, multi-label vulnerability pat- terns derived from its audit annotations, including Intent Mismatch (IM), Reverse Shell (RS), Shadow Feature (SF) and pattern identifiers such as E1âE4, P1âP4, PE1âPE3, and SC1âSC3. SKIL- LLITE achieves robust detection across several representative vulnerability patterns, with recall ex- ceeding 0.78 on SC2, E1, E2, and P4. Its performance varies more on some patterns, with P1, E3, E4, and SC3 showing lower recall and representing more challenging cases for SKILLLITE. Skill-Vetter and AIG attain high recall across many patterns, but their greater sensitivity results in substantially higher false-positive rates on the complete benchmark. In contrast, other scanners exhibit substantially lower recall across multiple patterns, resulting in more missed malicious Skills. B.2PERFORMANCE ACROSS PACKAGE CHARACTERISTICS We next examine how Skill package characteristics affect malicious-skill detection. We consider two dimensions: package size, measured by the number of files, and programming-language char- acteristics, including both language complexity and specific programming languages. Package size. We group Skills by the number of files in each package after excluding dataset- specific metadata and evaluation artifacts. As shown in Figure 5, SKILLLITE maintains strong 15 Table 6: F1-score across packages with different programming-language complexity. DatasetGroupC-SC-LSS-S-LWardVetterAIGSKILLLITE MalSkillBench No detected code.171.657.310.467.679.591.802.819 One language.246.717.428.544.670.664.823.911 Multiple languages.484.830.683.771.816.827.880.919 SkillTrustBench No detected code.364.372.571.750.545.279.571.800 One language.692.773.736.716.848.769.827.952 Multiple languages.819.892.884.883.912.882.901.970 MASB No detected code.000.263.000.100.000.368.400.455 One language.034.815.109.205.087.863.880.925 Multiple languages.333.591.474.395.279.634.667.471 Table 7: F1-score across programming languages. Language groups are non-exclusive, as a Skill may contain multiple programming languages. DatasetLanguageC-SC-LSS-S-LWardVetterAIGSKILLLITE MalSkillBench Go.483.905.629.744.750.756.850.919 JavaScript.331.551.448.514.769.528.665.845 PowerShell.754.755.810.857.789.750.839.874 Python.476.883.723.810.843.880.916.934 Rust.378.852.609.821.857.828.877.885 SQL.611.911.570.689.688.900.902.939 Shell.402.760.583.673.751.737.843.916 TypeScript.464.818.610.730.764.766.864.891 YAML.361.821.597.717.747.832.883.891 SkillTrustBench JavaScript.784.824.902.885.886.848.880.959 PowerShell.852.865.890.857.893.819.864.930 Python.840.908.906.904.922.892.906.974 SQL.687.725.878.818.866.819.805.901 Shell.791.861.843.840.894.851.884.965 TypeScript.812.890.877.871.919.797.873.932 YAML.735.846.804.805.899.861.884.974 MASB JavaScript.267.766.474.450.222.818.844.452 Python.286.444.435.357.200.421.564.593 Shell.125.763.241.282.148.822.839.851 TypeScript.400.444.333.286.500.667.571.500 YAML.000.353.182.133.000.588.429.444 F1 across the major package-size groups, achieving better overall performance than the baselines. On SkillTrustBench, SKILLLITE consistently outperforms all baselines across the three package- size groups. On MalSkillBench, it also achieves the strongest performance on the two dominant groups. Baselines exhibit substantially larger performance variations across package sizes. The pattern is less consistent on MASB. SKILLLITE achieves strong performance on single-file packages and remains competitive on packages containing two to five files, but its performance decreases on larger packages. Baselines show similar variations across package sizes, indicating that performance on MASB is more sensitive to package characteristics. Programming-language characteristics. We first examine performance under different levels of programming-language complexity. As shown in Table 6, SKILLLITE achieves strong F1 on both single- and multi-language packages in MalSkillBench and SkillTrustBench, outperforming all base- lines in these groups. Its performance remains stable when moving from single-language to multi- language packages, indicating robustness to language heterogeneity. Packages without detected code are more challenging, suggesting that less code-oriented Skills provide fewer explicit imple- mentation signals for security analysis. We further examine whether detection performance varies across specific programming languages. As shown in Table 7, SKILLLITE achieves the highest F1 across most language groups in MalSkill- Bench and SkillTrustBench, covering diverse implementations such as Python, Shell, JavaScript, 16 Table 8: Complete cross-model results on MalSkillBench. ModelMethodAccPrecRecallF1FPRâ FNRâ Latency (s)â Gemma4 E4B Zero-shot0.737 0.9920.4740.641 0.0040.52610.2 SKILLLITE 0.909 0.9440.8690.905 0.0510.13227.4 Qwen3.5 9B Zero-shot0.811 0.9900.6260.767 0.0070.37431.1 SKILLLITE 0.918 0.9690.8620.912 0.0270.13862.1 DeepSeek-R1 8B (Thinking) Zero-shot0.734 0.9720.4790.641 0.0140.52111.7 SKILLLITE 0.806 0.9250.6640.773 0.0540.33621.8 DeepSeek-R1 8B (w/o Thinking) Zero-shot0.702 0.9280.4340.592 0.0330.5662.5 SKILLLITE 0.801 0.7820.8290.805 0.2290.17111.5 Mixtral Zero-shot0.531 0.9830.0570.108 0.0010.9438.3 SKILLLITE 0.629 0.8380.3120.455 0.0600.68814.2 Table 9: Complete cross-model results on SkillTrustBench. ModelMethodAccPrecRecallF1FPRâ FNRâ Latency (s)â Gemma4 E4B Zero-shot0.712 0.8730.6400.739 0.1630.36011.8 SKILLLITE 0.957 0.9710.9610.966 0.0500.03928.5 Qwen3.5 9B Zero-shot0.858 0.9960.7790.875 0.0050.22136.3 SKILLLITE 0.959 0.9820.9520.967 0.0300.04865.7 DeepSeek-R1 8B (Thinking) Zero-shot0.637 0.9900.4320.602 0.0070.56815.2 SKILLLITE 0.792 0.9450.7150.814 0.0720.28522.9 DeepSeek-R1 8B (w/o Thinking) Zero-shot0.482 0.9070.2050.335 0.0370.7953.6 SKILLLITE 0.817 0.8160.9180.864 0.3600.08212.3 Mixtral Zero-shot0.373 0.9290.0140.027 0.0020.9866.4 SKILLLITE 0.535 0.9040.3010.451 0.0560.69917.1 and YAML. On MASB, SKILLLITE shows greater performance variation across programming lan- guages, achieving the highest F1 on Python and Shell. Overall, SKILLLITE demonstrates strong detection performance across diverse programming languages. CCOMPLETE CROSS-MODEL RESULTS We provide the complete cross-model results underlying the analysis in Section 4.4 in Tables 8, 9, and 10. To ensure reliable and fair efficiency comparisons, latency is measured under a controlled evaluation setting with consistent runtime configurations across models and methods. Across the three benchmarks, direct zero-shot auditing exhibits high precision but lower recall, indicating that compact LLMs struggle to reliably identify malicious Skills. SKILLLITE improves malicious Skill detection across all evaluated backbones. It can recover malicious Skills missed under direct zero- shot auditing. The magnitude of improvement varies across backbones, indicating that the effective- ness of SKILLLITE remains influenced by the capability of the underlying compact LLM. Although these improvements introduce additional inference latency, the overall runtime remains practical, reflecting an effectivenessâefficiency trade-off. DPROMPT TEMPLATES We provide the prompt templates used by the compact LLM in SKILLLITE. The LLM performs two semantic reasoning tasks: intent analysis and evidence-guided risk adjudication. The templates below present the core instructions and output structures used in our implementation. 17 Table 10: Complete cross-model results on MASB. ModelMethodAccPrecRecallF1FPRâ FNRâ Latency (s)â Gemma4 E4B Zero-shot0.656 0.5000.0380.071 0.0200.9629.2 SKILLLITE 0.878 0.8920.7370.807 0.0480.26319.4 Qwen3.5 9B Zero-shot0.669 1.0000.0380.074 0.0000.96227.1 SKILLLITE 0.831 0.8570.6120.714 0.0540.38965.5 DeepSeek-R1 8B (Thinking) Zero-shot0.675 1.0000.0570.108 0.0000.94310.5 SKILLLITE 0.827 0.8200.6370.717 0.0740.36322.7 DeepSeek-R1 8B (w/o Thinking) Zero-shot0.675 0.7140.0960.169 0.0200.9052.2 SKILLLITE 0.779 0.6390.8220.719 0.2440.17815.0 Mixtral Zero-shot0.656 0.0000.0000.000 0.0001.0003.9 SKILLLITE 0.772 0.7300.5350.618 0.1040.46511.8 D.1INTENT ANALYSIS PROMPT Intent Analysis [INPUT] Skill Name: skillname Skill Specification: skillmd [INSTRUCTION] Infer the intended functionality of the Skill. 1. Identify its declared purpose. 2. Identify the capabilities necessary and proportionate to that purpose. [OUTPUT] "declared purpose": "...", "expectedcapabilities": [...] Evidence-Guided Risk Adjudication [INPUT] Skill Specification: skillmd Functional Specification: intent analysis Evidence Report: evidencereport [INSTRUCTION] Assess the Skill using the grounded evidence and its intended functionality. 1. Review suspicious evidence and its source. 2. Assess whether each behavior is necessary for the declared functionality. 3. Identify direct benign counterevidence. 4. Determine whether the evidence forms an abuse chain and make the final judgment. [DECISION POLICY] - Security-sensitive capability alone does not imply maliciousness. - Necessary and proportionate behavior should normally be treated as benign. - Strong unexplained or deceptive behavior supports a malicious judgment. - Critical agentic risks require direct, risk-specific benign counterevidence. [OUTPUT] "suspicious evidencereview": [...], "benign counterevidence": [...], "abusechain": "present": true/false, "components": [...] , "ismalicious": true/false, "confidencescore": 0--100, "reasoning": "..." 18