[Submitted on 13 Oct 2026 (v1), last revised 14 Oct 2026 (this version, v2)]
Abstract:Agent skills turn natural-language instructions into actions, creating a security challenge: legitimate automation and malicious behaviour can look deceptively similar. A practical screen must catch nearly every attack while keeping the review load manageable. We present SkillScreen, a compact decision model trained to score skill maliciousness directly from SKILL$.$md in a single forward pass. On MaliciousSkillBench, SkillScreen flags 20-25% of benign skills at 99% recall on unseen attack families, substantially fewer than the word-level classifier and prompted LLM judges tested. Under the same training protocol, it also generalizes better than a fully fine-tuned encoder to unseen sources. Yet unfamiliar benign sources sharply increase false positives, showing that recognizing new attacks does not ensure reliable screening of new skill collections. Our results demonstrate the value of task-trained decision models for high-recall skill screening. Project page: this https URL
Submission history
From: Jerrold Huang [view email]
[v1] Tue, 13 Oct 2026 05:03:43 UTC (639 KB)
[v2] Wed, 14 Oct 2026 04:56:03 UTC (639 KB)
