SkillSpector

# SkillSpector **Security scanner for AI agent skills.** Detect vulnerabilities, malicious patterns, and security risks before installing agent skills. [](https://www.python.org/downloads/) [](https://www.apache.org/licenses/LICENSE-2.0) ## Overview AI agent skills (used by Claude Code, Codex CLI, Gemini CLI, etc.) execute with implicit trust and minimal vetting. In the 31,132-skill analyzed subset of the research dataset, **26.1% of skills contain vulnerabilities** and **5.2% show likely malicious intent**. SkillSpector helps you answer: **"Is this skill safe to install?"** SkillSpector is part of the [NVIDIA Verified Skills pipeline](https://docs.nvidia.com/skills/), which scans, evaluates, and signs agent skills before publication. Skills that pass are published to the [NVIDIA skills catalog](https://github.com/NVIDIA/skills). ## Documentation - **[Scan agent skills before installation](https://docs.nvidia.com/skills/scanning-agent-skills)** — Hosted guide: when to scan, how to read a report, and how to gate installs. - **[Development guide](docs/DEVELOPMENT.md)** — Architecture, package layout, and how to extend the analyzer pipeline. - **[Analysis resource bounds](docs/ANALYSIS_RESOURCE_BOUNDS.md)** — Fail-closed bundle, parser, nested-artifact, ledger, and finding ceilings. - **[Pi extension](docs/PI_EXTENSION.md)** — Install SkillSpector as a Pi tool for scanning skills from inside agent sessions. - **[OpenCode extension](docs/OPENCODE_EXTENSION.md)** — Install SkillSpector as an OpenCode tool and `/skillspector` command for scanning skills from inside agent sessions. ## Features - **Multi-format input**: Scan Git repos, URLs, zip files, directories, or single files - **71 vulnerability patterns** across 17 categories: prompt injection, data exfiltration, privilege escalation, supply chain, excessive agency, output handling, system prompt leakage, memory poisoning, tool misuse, rogue agent, anti-refusal, trigger abuse, dangerous code (AST), taint tracking, YARA signatures, MCP least privilege, and MCP tool poisoning - **Two-stage analysis**: Fast static analysis + optional LLM semantic evaluation - **Live vulnerability lookups**: SC4 queries [OSV.dev](https://osv.dev) for real-time CVE data with automatic offline fallback - **Multiple output formats**: Terminal, JSON, Markdown, and SARIF reports - **Risk scoring**: 0-100 score with severity labels and clear recommendations - **Baseline / false-positive suppression**: Accept known findings via a glob-rule or fingerprint baseline so re-scans surface only *new* issues ([docs](docs/SUPPRESSION.md)) ## Quick Start ### Installation > **Open-source software notice:** This project will download and install additional third-party open source software projects. Review the license terms of these open source projects before use. Create and activate a virtual environment first (all `make` targets assume the venv is active). Use **uv** or **pip**; the Makefile uses `uv` if available, otherwise `pip`. **Quick install with uv (CLI-only):** ```bash uv tool install git+https://github.com/NVIDIA/skillspector.git # Update later: uv tool update skillspector ``` If you plan to run `skillspector mcp`, install the MCP extra at install time: ```bash uv tool install 'skillspector[mcp] @ git+https://github.com/NVIDIA/skillspector.git' ``` **From source:** ```bash # Clone the repository git clone https://github.com/NVIDIA/skillspector.git cd skillspector # Create and activate virtual environment uv venv .venv && source .venv/bin/activate # or: python3 -m venv .venv && source .venv/bin/activate # Install for production use make install # Or install with development dependencies make install-dev ``` ### Docker (no Python required) Run SkillSpector without installing Python by building it locally from the included [Dockerfile](Dockerfile). The image is based on the Docker Official Python `3.12-slim-bookworm` image. **Build the image:** ```bash make docker-build # or: docker build -t skillspector . ``` **Scan a local directory** by mounting your current directory into `/scan`, the container's working directory: ```bash docker run --rm -v "$PWD:/scan" skillspector scan ./my-skill/ --no-llm ``` **Scan with LLM analysis** by passing credentials with a local `.env` file: ```bash cat > .env <<'EOF' SKILLSPECTOR_PROVIDER=anthropic ANTHROPIC_API_KEY=sk-ant-... EOF ``` ```bash docker run --rm \ -v "$PWD:/scan" \ --env-file .env \ skillspector scan ./my-skill/ ``` Or pass credentials directly from your shell environment: ```bash docker run --rm \ -v "$PWD:/scan" \ -e SKILLSPECTOR_PROVIDER=anthropic \ -e ANTHROPIC_API_KEY="$ANTHROPIC_API_KEY" \ skillspector scan ./my-skill/ ``` **Write a report to the host filesystem** by writing to the mounted directory: ```bash docker run --rm \ -v "$PWD:/scan" \ skillspector scan ./my-skill/ --no-llm --format json --output report.json ``` **Optional alias** for repeated static scans: ```bash alias skillspector-docker='docker run --rm -v "$PWD:/scan" skillspector' skillspector-docker scan ./my-skill/ --no-llm ``` ### Basic Usage ```bash # Scan a local skill directory skillspector scan ./my-skill/ # Scan a single SKILL.md file skillspector scan ./SKILL.md # Scan a Git repository skillspector scan https://github.com/user/my-skill # Scan a zip file skillspector scan ./my-skill.zip ``` #### Size limits SkillSpector enforces two independent caps on remote and archive inputs to bound the impact of oversized downloads and zip bombs: - **Per-ingest cap**: `INGEST_MAX_BYTES` (100 MiB) — applied to streamed URL downloads, total uncompressed size of zip archives, and post-clone disk usage of Git repos. - **Zip member cap**: `INGEST_MAX_ZIP_MEMBERS` (10,000) — caps the number of entries in a single zip. Note that the per-file 1 MB analysis cap (`MAX_FILE_BYTES`) is a separate, downstream limit: it bounds what individual analyzers will read out of an already-ingested directory. The ingest caps above bound how much content can land on disk in the first place. A breach of either ingest cap fails closed with an `IngestLimitExceededError`. ### Output Formats ```bash # Terminal output (default) - pretty formatted skillspector scan ./my-skill/ # JSON output - machine readable skillspector scan ./my-skill/ --format json --output report.json # Markdown output - for documentation skillspector scan ./my-skill/ --format markdown --output report.md # SARIF output - for CI/CD integration and IDE tooling skillspector scan ./my-skill/ --format sarif --output report.sarif ``` ### Batch Scanning Scan entire directories of skills in parallel from `contrib/batch_scan/`: ```bash python -m contrib.batch_scan.batch_scan ./my-skills/ --no-llm python -m contrib.batch_scan.batch_scan ./my-skills/ --workers 20 -f json -o report.json python -m contrib.batch_scan.batch_scan ./tests/fixtures/ -f terminal --workers 20 ``` Supports multilingual detection (zh/ja/ko) and terminal/JSON/Markdown output. For LLM scans with higher concurrency, configure multiple API keys following [`.env.example`](contrib/batch_scan/.env.example) — the pool improves throughput and resilience, provided the keys don't share an account-level rate limit. See the [contrib guide](contrib/batch_scan/docs/) for details. > **Note on LLM support:** The default configuration targets DeepSeek as the > cheapest public option. DeepSeek-Chat is > [expected to sunset](https://api-docs.deepseek.com/), and the contributor > does not have hardware to test against local models. The batch scanner was > originally tested with OpenAI-compatible endpoints — DeepSeek's lack of > structured-output support required manual JSON-parsing patches. If you can > contribute a more universal backend (Ollama, vLLM, or a different provider), > PRs are very welcome. ### Suppressing False Positives (baseline) Suppress known/accepted findings so the risk score reflects only un-triaged issues and re-scans surface only *new* findings. See the [suppression guide](docs/SUPPRESSION.md) for the full reference. ```bash # Accept all current findings into a baseline (run once), then commit it. skillspector baseline ./my-skill/ -o .skillspector-baseline.yaml # Scan against the baseline — only NEW findings are reported and scored. skillspector scan ./my-skill/ --baseline .skillspector-baseline.yaml # Review what was suppressed (still excluded from the score). skillspector scan ./my-skill/ --baseline .skillspector-baseline.yaml --show-suppressed ``` A baseline can also use drift-tolerant glob rules (by rule id, file path, or message) — see [`.skillspector-baseline.example.yaml`](.skillspector-baseline.example.yaml). Exact fingerprint baselines are evidence-bound: changing the scanned source or SkillSpector version keeps the finding active until it is reviewed again. When a selected baseline or baseline output is stored inside the skill directory, SkillSpector excludes that exact file from content analysis so its suppression text cannot create findings or enter regenerated fingerprints; sibling files remain in normal scan scope. ### LLM Analysis For the best results, configure an OpenAI-compatible LLM endpoint for semantic analysis. Pick a provider with `SKILLSPECTOR_PROVIDER`; hosted providers ship bundled default models, while CLI providers fall back to the local runtime's default model unless `SKILLSPECTOR_MODEL` is set. SkillSpector also works against local OpenAI-compatible servers (Ollama, vLLM, llama.cpp) and managed inference gateways. | Provider (`SKILLSPECTOR_PROVIDER`) | Credential env var | Endpoint | Default model | | ---------- | ---- | ---- | ---- | | `openai` | `OPENAI_API_KEY` (+ optional `OPENAI_BASE_URL`) | api.openai.com (or any OpenAI-compatible URL) | `gpt-5.4` | | `anthropic` | `ANTHROPIC_API_KEY` | api.anthropic.com | `claude-opus-4-6` | | `anthropic_proxy` | `ANTHROPIC_PROXY_API_KEY` + `ANTHROPIC_PROXY_ENDPOINT_URL` | Any Vertex-style raw-predict proxy | `claude-sonnet-4-6` | | `bedrock` | `AWS_PROFILE` (optional) + `AWS_REGION` — SigV4 via boto3 | AWS Bedrock Runtime | `us.anthropic.claude-sonnet-4-6-20250915-v1:0` | | `nv_build` | `NVIDIA_INFERENCE_KEY` | build.nvidia.com | `z-ai/glm-5.2` | | `ollama` | _(none)_ | `OLLAMA_BASE_URL` (default `http://localhost:11434/v1`) | `llama3.1:8b` | | `azure_openai` | `AZURE_OPENAI_API_KEY` + `AZURE_OPENAI_ENDPOINT` | Azure OpenAI Service | `gpt-4o` (deployment defaults to the model label) | | `openai_compatible` | `SKILLSPECTOR_COMPAT_API_KEY` + `SKILLSPECTOR_COMPAT_BASE_URL` | Any OpenAI-compatible endpoint | `llama-3.1-70b-versatile` | | `claude_cli` | _(none — uses local CLI auth)_ | local `claude` binary | local Claude runtime fallback, or `SKILLSPECTOR_MODEL` | | `codex_cli` | _(none — uses local CLI auth)_ | local `codex` binary | local Codex runtime fallback, or `SKILLSPECTOR_MODEL` | | `gemini_cli` | _(none — uses local CLI auth)_ | local `gemini` binary | local Gemini runtime fallback, or `SKILLSPECTOR_MODEL` | | `opencode_cli` | _(none — uses local CLI auth)_ | local `opencode` 1.18.33 binary | local OpenCode runtime fallback, or `SKILLSPECTOR_MODEL` | Structured output is requested through LangChain's `with_structured_output`, whose default forces a tool call. Some models reject a forced tool call with HTTP 400 (`tool_choice: type "tool" and "any" are not supported for this model`). The `anthropic` and `anthropic_proxy` providers route those models (`claude-fable-5-1`, `claude-mythos-5-1`, or any registry entry with `structured_output: json_schema`) to the native JSON-schema response format. Bedrock has no JSON-schema output for them, so the `bedrock` provider leaves `toolChoice` at `auto`, asks for the tool call in the prompt, and retries a prose answer; it recognises the model from the model ID, a geo/global inference-profile ID, or a foundation-model / inference-profile ARN. An application-inference-profile ARN hides the model, so add that ARN to the registry (`SKILLSPECTOR_MODEL_REGISTRY`) with `tool_choice: auto`. `SKILLSPECTOR_STRUCTURED_OUTPUT_METHOD=json_schema|function_calling` overrides the method for any provider. ```bash # Stock OpenAI export SKILLSPECTOR_PROVIDER=openai export OPENAI_API_KEY=sk-... skillspector scan ./my-skill/ # Anthropic export SKILLSPECTOR_PROVIDER=anthropic export ANTHROPIC_API_KEY=sk-ant-... skillspector scan ./my-skill/ # Anthropic via Vertex-style proxy (corporate gateways, GCP Vertex AI) export SKILLSPECTOR_PROVIDER=anthropic_proxy export ANTHROPIC_PROXY_ENDPOINT_URL=https://my-gateway.example.com/models/claude-sonnet-4-6:streamRawPredict export ANTHROPIC_PROXY_API_KEY=your-bearer-token export SKILLSPECTOR_MODEL=claude-sonnet-4-6 skillspector scan ./my-skill/ # AWS Bedrock (Claude via SigV4) export SKILLSPECTOR_PROVIDER=bedrock # Optional: select an AWS named profile. When unset, the standard # boto3 credential chain (env vars, instance metadata, SSO, etc.) resolves. # export AWS_PROFILE=my-profile export AWS_REGION=us-west-2 # default if unset # Default model: us.anthropic.claude-sonnet-4-6-20250915-v1:0 # Override with any Bedrock model ID, cross-region inference-profile # ID, or your own application-inference-profile ARN: # export SKILLSPECTOR_MODEL=us.anthropic.claude-opus-4-6-20250915-v1:0 skillspector scan ./my-skill/ # NVIDIA build.nvidia.com export SKILLSPECTOR_PROVIDER=nv_build export NVIDIA_INFERENCE_KEY=nvapi-... skillspector scan ./my-skill/ # Local Claude CLI — no API key; uses your existing `claude auth login` session # Requires: claude CLI installed and authenticated (claude auth login) export SKILLSPECTOR_PROVIDER=claude_cli # Uses the local Claude CLI runtime fallback unless SKILLSPECTOR_MODEL is set. # export SKILLSPECTOR_MODEL=claude-sonnet-4-6 skillspector scan ./my-skill/ # Local Codex CLI — no API key; uses your existing `codex login` session # Requires: codex CLI installed and authenticated export SKILLSPECTOR_PROVIDER=codex_cli skillspector scan ./my-skill/ # Gemini (via OpenAI compatibility layer) export SKILLSPECTOR_PROVIDER=openai export OPENAI_API_KEY="YOUR_GEMINI_API_KEY" export OPENAI_BASE_URL="https://generativelanguage.googleapis.com/v1beta/openai/" export SKILLSPECTOR_MODEL=gemini-3.5-flash skillspector scan ./my-skill/ # Local Ollama — no API key export SKILLSPECTOR_PROVIDER=ollama # export OLLAMA_BASE_URL=http://localhost:11434/v1 # shown default export SKILLSPECTOR_MODEL=llama3.1:8b skillspector scan ./my-skill/ # Azure OpenAI export SKILLSPECTOR_PROVIDER=azure_openai export AZURE_OPENAI_API_KEY=... export AZURE_OPENAI_ENDPOINT=https://example.openai.azure.com/ export AZURE_OPENAI_DEPLOYMENT=my-deployment skillspector scan ./my-skill/ # Any other OpenAI-compatible endpoint export SKILLSPECTOR_PROVIDER=openai_compatible export SKILLSPECTOR_COMPAT_API_KEY=... export SKILLSPECTOR_COMPAT_BASE_URL=https://api.groq.com/openai/v1 export SKILLSPECTOR_MODEL=llama-3.1-70b-versatile skillspector scan ./my-skill/ # Override the provider's default model export SKILLSPECTOR_MODEL=gpt-5.2 skillspector scan ./my-skill/ # Skip LLM analysis (faster, static analysis only) skillspector scan ./my-skill/ --no-llm ``` ### MCP Server Run SkillSpector as a [Model Context Protocol](https://modelcontextprotocol.io) server so any MCP-capable agent (Claude Code, Codex CLI, Gemini CLI) or remote runtime can call scanning as a tool and **gate skill/MCP installs on the result** — turning SkillSpector into a runtime guardrail instead of an out-of-band audit step. `skillspector mcp` requires `skillspector[mcp]`. ```bash # Install, or reinstall if you already used the CLI-only path uv tool install --force 'skillspector[mcp] @ git+https://github.com/NVIDIA/skillspector.git' # FastMCP stdio transport for local CLI agents skillspector mcp # streamable HTTP/SSE transport for remote / A2A callers skillspector mcp --transport http --host 127.0.0.1 --port 8000 ``` The stdio transport is the current FastMCP path for local CLI agents, and the initialize hang reported in issue #199 still applies there. The server exposes a single tool: - **`scan_skill(target, use_llm=true, output_format="json")`** — scans a Git URL, file URL, `.zip`, `.md` file, or directory and returns a structured verdict: `risk_score` (0-100), `severity`, `recommendation`, `safe_to_install`, and `findings`. It also reports `llm_used` / `scan_mode` so a low score from a static-only scan is never mistaken for a clean full scan. Register it with Claude Code via: ```bash claude mcp add skillspector -- skillspector mcp ``` > **Security — HTTP transport trust model** > > The HTTP transport ships **without authentication**. Any caller that can > reach the port can invoke `scan_skill`. Over stdio or `127.0.0.1` this is > the same trust boundary as the CLI. If you bind to a routable interface: > > - Sit the server behind an authenticating reverse proxy (e.g. nginx + mTLS) > before exposing it externally. > - Local paths and `file://` URLs are **automatically rejected** over HTTP to > prevent unauthenticated callers from reading arbitrary host files. Only > remote Git and `.zip` URLs are accepted. ## Vulnerability Patterns SkillSpector detects **71 vulnerability patterns** across 17 categories: ### Prompt Injection (6 patterns) | ID | Pattern | Severity | Description | |----|---------|----------|-------------| | P1 | Instruction Override | HIGH | Commands to ignore safety constraints | | P2 | Hidden Instructions | HIGH | Malicious directives in comments/invisible text | | P3 | Exfiltration Commands | HIGH | Instructions to transmit context externally | | P4 | Behavior Manipulation | MEDIUM | Subtle instructions altering agent decisions | | P5 | Harmful Content | CRITICAL | Instructions that could cause physical harm | | P9 | Whitespace Padding | MEDIUM | Large whitespace padding hiding instructions below/beside the visible area | ### Anti-Refusal (3 patterns) | ID | Pattern | Severity | Description | |----|---------|----------|-------------| | AR1 | Refusal Suppression | HIGH | Instructions to never refuse or always comply (e.g. "never refuse", "always comply") | | AR2 | Disclaimer Suppression | HIGH | Instructions to omit warnings, disclaimers, or ethical commentary (e.g. "no disclaimers", "do not moralize") | | AR3 | Safety Policy Nullification | HIGH | Jailbreak framing that nullifies guardrails (e.g. "you have no restrictions", "ignore your guidelines", "do anything now") | ### Data Exfiltration (4 patterns) | ID | Pattern | Severity | Description | |----|---------|----------|-------------| | E1 | External Transmission | MEDIUM | Sending data to external URLs | | E2 | Env Variable Harvesting | HIGH | Enumerating, copying, or searching environment data to collect secrets | | E3 | File System Enumeration | MEDIUM | Scanning directories for sensitive files | | E4 | Context Leakage | HIGH | Transmitting conversation context externally | ### Privilege Escalation (3 patterns) | ID | Pattern | Severity | Description | |----|---------|----------|-------------| | PE1 | Excessive Permissions | LOW | Requesting access beyond stated functionality | | PE2 | Sudo/Root Execution | MEDIUM | Invoking elevated system privileges | | PE3 | Credential Access | HIGH | Reading SSH keys, tokens, passwords | ### Supply Chain (10+ patterns) | ID | Pattern | Severity | Description | |----|---------|----------|-------------| | SC1 | Unpinned Dependencies | LOW | No version constraints on packages | | SC2 | External Script Fetching | HIGH | curl \| bash and remote code execution | | SC3 | Obfuscated Code | HIGH | Base64/hex encoded execution | | SC4 | Known Vulnerable Dependencies | HIGH | Dependencies with known CVEs (live OSV.dev lookup) | | SC5 | Abandoned Dependencies | MEDIUM | Unmaintained packages without security updates | | SC6 | Typosquatting | HIGH | Package names similar to popular packages | | SC8 | Shipped Python Bytecode | HIGH | `__pycache__` / `.pyc` present (discovery skips; malicious bytecode bypass) | | SC9 | Concealed Executable Artifact | HIGH | Executable nested in a document container or hidden/disguised artifact | | SC10 | Dependency Source Redirection | HIGH | Package-manager source added, replaced, or unresolved | ### Excessive Agency (5 patterns) | ID | Pattern | Severity | Description | |----|---------|----------|-------------| | EA1 | Unrestricted Tool Access | HIGH | Unfettered tool access without constraints | | EA2 | Autonomous Decision Making | HIGH | High-impact decisions without human-in-the-loop | | EA3 | Scope Creep | MEDIUM | Capabilities extending beyond stated purpose | | EA4 | Unbounded Resource Access | MEDIUM | No rate limits or quotas on resource consumption | | EA5 | External Model or Provider Selection | MEDIUM/HIGH | Model/provider pins or coding-CLI shell-outs that can switch billing accounts | ### Output Handling (3 patterns) | ID | Pattern | Severity | Description | |----|---------|----------|-------------| | OH1 | Unvalidated Output Injection | HIGH | Model output used without sanitization | | OH2 | Cross-Context Output | MEDIUM | Output flows across trust boundaries without validation | | OH3 | Unbounded Output | MEDIUM | No limits on output size or generation rate | ### System Prompt Leakage (3 patterns) | ID | Pattern | Severity | Description | |----|---------|----------|-------------| | P6 | Direct Leakage | HIGH | Instructions that expose system prompts or internal rules | | P7 | Indirect Extraction | MEDIUM | Extraction via rephrasing, translation, or side-channels | | P8 | Tool-Based Exfiltration | HIGH | System prompts exfiltrated via file writes or network requests | ### Memory Poisoning (3 patterns) | ID | Pattern | Severity | Description | |----|---------|----------|-------------| | MP1 | Persistent Context Injection | HIGH | Content designed to persist across interactions | | MP2 | Context Window Stuffing | MEDIUM | Filler content displacing safety constraints | | MP3 | Memory Manipulation | HIGH | Tampering with agent memory or stored state | ### Tool Misuse (3 patterns) | ID | Pattern | Severity | Description | |----|---------|----------|-------------| | TM1 | Tool Parameter Abuse | HIGH | Crafted parameters for unintended behavior (shell=True, --force) | | TM2 | Chaining Abuse | HIGH | Tool chains that bypass individual safety checks | | TM3 | Unsafe Defaults | MEDIUM | Overly permissive defaults (disabled TLS, no auth) | ### Rogue Agent (2 patterns) | ID | Pattern | Severity | Description | |----|---------|----------|-------------| | RA1 | Self-Modification | CRITICAL | Modifying own code or configuration at runtime | | RA2 | Session Persistence | HIGH | Unauthorized persistence via cron jobs or startup scripts | ### Trigger Abuse (3 patterns) | ID | Pattern | Severity | Description | |----|---------|----------|-------------| | TR1 | Overly Broad Trigger | MEDIUM | Trigger patterns matching common words | | TR2 | Shadow Command Trigger | HIGH | Triggers that shadow built-in commands or other skills | | TR3 | Keyword Baiting Trigger | MEDIUM | Generic triggers designed to maximize activation | ### Behavioral AST (9 patterns) | ID | Pattern | Severity | Description | |----|---------|----------|-------------| | AST1 | exec() Call | CRITICAL | Direct exec() enabling arbitrary code execution | | AST2 | eval() Call | HIGH | Direct eval() evaluating arbitrary expressions | | AST3 | Dynamic Import | HIGH | \_\_import\_\_() loading arbitrary modules at runtime | | AST4 | subprocess Call | HIGH | External command execution via subprocess | | AST5 | os.system / exec-family | HIGH | Shell commands via os module | | AST6 | compile() Call | MEDIUM | Code object creation from strings | | AST7 | Dynamic getattr() | MEDIUM | Arbitrary attribute access with non-literal names | | AST8 | Dangerous Execution Chain | CRITICAL | exec/eval combined with dynamic source (network, encoded data) | | AST9 | Reflective getattr() Sink | HIGH | Reflective exec via `getattr(os,'system')` / `getattr(builtins,'exec')` that evades AST1/AST5 | ### Taint Tracking (5 patterns) | ID | Pattern | Severity | Description | |----|---------|----------|-------------| | TT1 | Direct Taint Flow | HIGH | Data flows directly from a source to a sink without sanitization | | TT2 | Variable-Mediated Taint Flow | MEDIUM | Data flows from source to sink through intermediate variables | | TT3 | Credential Exfiltration Chain | CRITICAL | Credentials (env vars, secrets) flow to network output sinks | | TT4 | File Read to Network Exfiltration | HIGH | File contents flow to network output sinks | | TT5 | External Input to Code Execution | CRITICAL | Network or user input flows to exec/eval/subprocess sinks | ### YARA Signatures (4 patterns) | ID | Pattern | Severity | Description | |----|---------|----------|-------------| | YR1 | Malware Match | CRITICAL | YARA rule match for known malware signatures | | YR2 | Webshell Match | CRITICAL | YARA rule match for webshell patterns | | YR3 | Cryptominer Match | HIGH | YARA rule match for crypto mining indicators | | YR4 | Hack Tool / Exploit Match | HIGH | YARA rule match for hack tools or exploit code | ### MCP Least Privilege (4 patterns) | ID | Pattern | Severity | Description | |----|---------|----------|-------------| | LP1 | Underdeclared Capability | HIGH | Code uses capabilities not listed in declared permissions | | LP2 | Wildcard Permission | MEDIUM | Permission list contains wildcards (\*, all, full, any) | | LP3 | Missing Permission Declaration | MEDIUM | No permissions field but code has detectable capabilities | | LP4 | Overdeclared Permission | LOW | Permission declared but no corresponding code capability found | ### MCP Tool Poisoning (4 patterns) | ID | Pattern | Severity | Description | |----|---------|----------|-------------| | TP1 | Hidden Instructions | HIGH | Hidden directives in metadata (HTML comments, zero-width chars, base64, data URIs) | | TP2 | Unicode Deception | HIGH | Homoglyphs, RTL overrides, mixed-script identifiers in tool metadata | | TP3 | Parameter Description Injection | MEDIUM | Injection patterns in parameter definitions (overrides, system tokens, malicious defaults) | | TP4 | Description-Behavior Mismatch | MEDIUM | Declared tool description does not match actual code behavior (LLM-powered) | All detected patterns are listed in the tables above. ## Risk Scoring ### Score Calculation - **CRITICAL issues**: +50 points - **HIGH issues**: +25 points - **MEDIUM issues**: +10 points - **LOW issues**: +5 points - **Executable scripts**: 1.3x multiplier ### Severity Levels | Score | Severity | Recommendation | |-------|----------|----------------| | 0-20 | LOW | SAFE | | 21-50 | MEDIUM | CAUTION | | 51-80 | HIGH | DO NOT INSTALL | | 81-100 | CRITICAL | DO NOT INSTALL | ## Example Output ### Terminal Output ``` SkillSpector Security Report v2.0.0 Skill: suspicious-skill Source: ./suspicious-skill/ Scanned: 2026-01-29 10:30:00 UTC Risk Assessment Metric Value Score 78/100 Severity HIGH Recommendation DO NOT INSTALL Components (3) File Type Lines Executable SKILL.md markdown 142 No scripts/sync.py python 87 Yes requirements.txt text 3 No Issues (2) HIGH: Env Variable Harvesting (E2) Location: scripts/sync.py:23 Finding: for key, val in os.environ.items():... Confidence: 94% Explanation: This code collects environment variables containing API keys and secrets, then sends them to an external server. HIGH: External Transmission (E1) Location: scripts/sync.py:45 Finding: requests.post("https://api.skill.io/env"... Confidence: 89% Explanation: Data is being sent to an external server. Combined with env harvesting above, this indicates credential exfiltration. ``` ## Configuration ### Environment Variables | Variable | Description | Required | |----------|-------------|----------| | `SKILLSPECTOR_PROVIDER` | Active LLM provider: `openai`, `anthropic`, `anthropic_proxy`, `bedrock`, `nv_build`, `ollama`, `azure_openai`, `openai_compatible`, `claude_cli`, `codex_cli`, `gemini_cli`, or `opencode_cli`. Hosted providers use bundled `model_registry.yaml` defaults; CLI providers fall back to the local runtime's default model unless `SKILLSPECTOR_MODEL` is set. Defaults to `nv_build`. | Optional | | `NVIDIA_INFERENCE_KEY` | Credential for the `nv_build` provider (build.nvidia.com). | Required for LLM analysis when `SKILLSPECTOR_PROVIDER=nv_build` | | `OPENAI_API_KEY` | Credential for the OpenAI provider (`SKILLSPECTOR_PROVIDER=openai`). Also serves as the tier-2 fallback in the credential waterfall when the active provider returns no credentials. | Required for LLM analysis when `SKILLSPECTOR_PROVIDER=openai` | | `OPENAI_BASE_URL` | Override the OpenAI endpoint (e.g. point at Ollama). | Optional | | `SKILLSPECTOR_REASONING_EFFORT` | Optional provider- and model-dependent reasoning-effort setting. Non-empty values are trimmed and passed through unchanged; unset or blank preserves provider-default behavior. | Optional | | `SKILLSPECTOR_OUTPUT_LANGUAGE` | Short, single-line language label (letters, numbers, spaces, `_`, or `-`; maximum 64 characters) for human-readable LLM finding text such as messages, explanations, and remediation. Rule IDs, severity values, paths, code, and other machine-readable values remain unchanged. Unset, blank, or invalid values preserve the default output language. | Optional | | `SKILLSPECTOR_TEMPERATURE` | Optional sampling temperature from `0` to `1` for hosted providers. Unset or blank preserves the provider default. Lower values can reduce run-to-run variation but do not guarantee identical output. | Optional | | `SKILLSPECTOR_SEED` | Optional integer sampling seed for OpenAI-compatible and Azure OpenAI providers. Other hosted providers and CLI providers do not receive it. Provider support remains model-dependent. | Optional | | `SKILLSPECTOR_COMPACT_PROMPTS` | Opt-in compact line numbering in LLM prompts: numbered lines render as `L1:`, `L2:` instead of zero-padded `L01:`, `L02:`. Accepted truthy values are `1`, `true`, and `yes` (case-insensitive; surrounding whitespace is trimmed). Unset or any other value keeps the default zero-padded format. | Optional | | `ANTHROPIC_API_KEY` | Credential for the Anthropic provider (`SKILLSPECTOR_PROVIDER=anthropic`). | Required for LLM analysis when `SKILLSPECTOR_PROVIDER=anthropic` | | `ANTHROPIC_BASE_URL` | Override the native Anthropic endpoint (default: `https://api.anthropic.com`). | Optional | | `ANTHROPIC_PROXY_ENDPOINT_URL` | Full endpoint URL for the Anthropic proxy provider (Vertex-style raw-predict). | Required when `SKILLSPECTOR_PROVIDER=anthropic_proxy` | | `ANTHROPIC_PROXY_API_KEY` | Bearer token for the Anthropic proxy provider. | Required when `SKILLSPECTOR_PROVIDER=anthropic_proxy` | | `ANTHROPIC_PROXY_API_VERSION` | `anthropic_version` value sent in the request body (default: `vertex-2023-10-16`). | Optional | | `AWS_PROFILE` | Named AWS profile for the Bedrock provider — authenticates via SigV4 through boto3. When unset, the standard boto3 credential chain (env vars, instance metadata, SSO, etc.) resolves. | Optional (used when `SKILLSPECTOR_PROVIDER=bedrock`) | | `AWS_REGION` | AWS region for the Bedrock Runtime endpoint. Defaults to `us-west-2`. | Optional (used when `SKILLSPECTOR_PROVIDER=bedrock`) | | `OLLAMA_BASE_URL` | Ollama OpenAI-compatible endpoint. Defaults to `http://localhost:11434/v1`. | Optional (used when `SKILLSPECTOR_PROVIDER=ollama`) | | `AZURE_OPENAI_API_KEY` | API key for the Azure OpenAI provider. | Required when `SKILLSPECTOR_PROVIDER=azure_openai` | | `AZURE_OPENAI_ENDPOINT` | Azure resource endpoint for the Azure OpenAI provider. | Required when `SKILLSPECTOR_PROVIDER=azure_openai` | | `AZURE_OPENAI_DEPLOYMENT` | Azure deployment name. Defaults to the selected model label. | Optional | | `AZURE_OPENAI_API_VERSION` | Azure OpenAI API version. Defaults to `2024-06-01`. | Optional | | `SKILLSPECTOR_COMPAT_API_KEY` | API key for a generic OpenAI-compatible provider. | Required when `SKILLSPECTOR_PROVIDER=openai_compatible` | | `SKILLSPECTOR_COMPAT_BASE_URL` | Base URL for a generic OpenAI-compatible provider. | Required when `SKILLSPECTOR_PROVIDER=openai_compatible` | | `SKILLSPECTOR_MODEL` | Override the active provider model. For hosted providers, this replaces the bundled default from the LLM Analysis table. For CLI providers, this is forwarded as `--model` instead of using the local runtime fallback. | Optional | | `SKILLSPECTOR_MODEL_REGISTRY` | Override the bundled per-provider YAML registry (`src/skillspector/providers/<provider>/model_registry.yaml`) with a custom path. | Optional | | `SKILLSPECTOR_LOG_LEVEL` | Log level: `DEBUG`, `INFO`, `WARNING`, `ERROR` (default: `WARNING`). | Optional | > **CLI providers** (`claude_cli`, `codex_cli`, `gemini_cli`, `opencode_cli`): No API key is needed. Authentication is managed entirely by the agent CLI's own login session. SkillSpector never reads or forwards API keys when these providers are active. The subprocess is run with capabilities restricted, and untrusted skill content is delivered only via stdin. > > `opencode_cli` currently fails closed unless the installed OpenCode version is exactly `1.18.33`, the version whose configuration precedence and deny-all semantics are verified by this release. ### CLI Options ```bash skillspector scan --help Options: -f, --format [terminal|json|markdown|sarif] Output format [default: terminal] -o, --output PATH Output file path --no-llm Skip LLM analysis (static only) --yara-rules-dir PATH Extra YARA rules directory -b, --baseline PATH Suppress findings listed in a baseline --show-suppressed List baseline-suppressed findings -V, --verbose Show detailed progress --help Show this message and exit # Generate a baseline of all current findings (see docs/SUPPRESSION.md) skillspector baseline <path> [-o FILE] [--no-llm] [--reason TEXT] ``` ## Integrating SkillSpector SkillSpector is built to be driven by other tools (CI pipelines, install gates, editor integrations). Its exit code and JSON output are a stable contract. ### Exit codes `skillspector scan` exits with: | Code | Meaning | |------|---------| | `0` | Scan completed, `risk_score` ≤ 50 (recommendation `SAFE` or `CAUTION`), and no enabled strict gate fired | | `1` | Scan completed and either `risk_score` > 50, `--fail-on-findings` found an active finding, or `--fail-on-incomplete` found partial/incomplete analysis | | `2` | Error (bad input, unreadable source, internal failure) | > By default, the exit code collapses `SAFE` and `CAUTION` into `0`. Use `--fail-on-findings` to gate on any active finding, `--fail-on-incomplete` to gate on incomplete coverage, or read the JSON `recommendation` field for custom policy. ### Machine-readable output `--format json` produces a JSON report; with no `--output`/`-o` it is written to stdout: ```bash skillspector scan ./my-skill/ --format json ``` The top-level shape is (this example shows a full LLM-backed scan; with `--no-llm`, `metadata.llm_requested` is `false`): ```json { "skill": { "name": "...", "source": "...", "scanned_at": "<ISO 8601>" }, "risk_assessment": { "score": 0, "severity": "LOW", "recommendation": "SAFE" }, "components": [ { "path": "...", "type": "...", "lines": 0, "executable": false, "size_bytes": 0 } ], "issues": [ { "id": "...", "category": "...", "severity": "...", "confidence": 0.0, "location": { "file": "...", "start_line": 0 } } ], "metadata": { "has_executable_scripts": false, "skillspector_version": "...", "llm_requested": true, "llm_available": true, "inference_usage": [ { "node": "semantic_security_discovery", "request_kind": "structured_output", "provider": "nv_inference", "model": "azure/anthropic/claude-opus-4-6", "model_source": "provider_response", "usage_source": "provider_response", "prompt_tokens": 1000, "completion_tokens": 100, "cached_tokens": 400, "cache_write_tokens": 50, "total_tokens": 1100 } ] } } ``` - `risk_assessment.severity` ∈ `LOW | MEDIUM | HIGH | CRITICAL`. - `risk_assessment.recommendation` ∈ `SAFE | CAUTION | DO_NOT_INSTALL`, mapped from severity: `LOW → SAFE`, `MEDIUM → CAUTION`, `HIGH`/`CRITICAL → DO_NOT_INSTALL`. - `metadata.llm_error` appears only when LLM analysis was requested but unavailable. - AE1 findings use **Incomplete referenced artifact analysis**. Their source location identifies the reference; `evidence` identifies the affected target, analyzer reasons, and available bounds. Review the target's completeness ledger when `reasons_truncated` is true. See [referenced-artifact diagnostics and Perl help text](docs/ANALYSIS_RESOURCE_BOUNDS.md#diagnosing-incomplete-referenced-artifacts) for interpretation and corrective actions. - `metadata.inference_usage` contains one sanitized record per LLM response when the provider exposes token counters. It is an empty list when usage is unavailable; SkillSpector never estimates missing tokens. Prompt totals are inclusive of cache reads and writes so downstream pricing can separate those partitions safely. `model_source` distinguishes an independently identified provider model from the exact requested model used when response identity is absent or ambiguous. SkillSpector does not currently send Anthropic prompt-cache controls, so its scan requests cannot select the separate 5-minute or 1-hour cache-write tiers; TTL-specific response fields are normalized defensively into the aggregate cache-write counter. - See [Inference usage telemetry](docs/INFERENCE_USAGE.md) for the complete provenance, cache-accounting, privacy, fail-closed ingestion, and downstream pricing contract. - The full per-issue shape is defined by `Finding.to_dict()` in [models.py](src/skillspector/models.py); rely on the fields above and treat any additional fields as best-effort. For CI/IDE tooling, `--format sarif` emits SARIF 2.1.0. ### Recommended gate mapping When using SkillSpector as an install gate, map the recommendation to an action: | `recommendation` | Suggested action | |------------------|------------------| | `SAFE` | allow | | `CAUTION` | prompt / warn the user | | `DO_NOT_INSTALL` | block | SkillSpector computes the score band and recommendation; how strict the gate is (e.g. whether `CAUTION` blocks in CI) is a policy decision for the integrating tool. ## Development ### Setup All `make` targets assume a virtual environment is already created and activated. The Makefile uses **uv** if available, else **pip**. ```bash # Clone, create venv, activate, install dev dependencies git clone https://github.com/NVIDIA/skillspector.git cd skillspector uv venv .venv && source .venv/bin/activate # or: python3 -m venv .venv && source .venv/bin/activate make install-dev # Run tests make test # Run tests with coverage make test-cov # Run linting make lint # Format code make format ``` ## How It Works SkillSpector uses a two-stage detection pipeline: ### Stage 1: Static Analysis - Fast regex-based pattern matching across 11 static analyzers - AST-based behavioral analysis detecting dangerous calls (exec, eval, subprocess, etc.) - Live vulnerability lookups via OSV.dev for known CVEs in dependencies - Scans all analyzer-eligible files in the skill - High recall (catches most issues) - Moderate precision (some false positives) A valid, root-level OpenSSF Model Signing signature (`skill.oms.sig`) is retained in the component inventory as type `oms_signature`, but excluded from static and LLM content analysis. OMS bundles necessarily contain long base64-encoded payload, signature, and certificate fields; generic obfuscated-code checks can otherwise misclassify those fields as hidden executable content. The recognizer checks the minimal OMS DSSE/in-toto structure; it does not verify the signature, certificate chain, transparency-log entry, or signer identity. Invalid or unrecognized signature files are scanned normally. ### Stage 2: LLM Semantic Analysis (Optional) - Evaluates context and intent - Filters false positives - Provides human-readable explanations - Improves precision to ~87% The LLM prompt includes anti-jailbreak protections to prevent malicious skills from manipulating the analysis. ## Live Vulnerability Lookups (SC4) SC4 uses the [OSV.dev](https://osv.dev) API to check dependencies against the full Open Source Vulnerabilities database — covering tens of thousands of advisories across PyPI and npm. - **No API key required** — OSV.dev is free and unauthenticated. - **Batch queries** — all dependencies are checked in a single HTTP call. - **Automatic fallback** — if OSV.dev is unreachable (air-gapped/offline), a small built-in fallback list is used. - **Caching** — results are cached in-memory for 1 hour to avoid redundant API calls during a session. The tool requires outbound HTTPS access to `api.osv.dev` for live vulnerability data. When that is not available, findings are limited to the static fallback list. ## Trust model and data egress SkillSpector is defense-in-depth, not a sandbox. Know what it does and does not do before relying on it: - **It never executes the scanned skill.** All analysis is static (regex, Python AST, YARA) plus optional LLM evaluation of file *contents* — the skill's code is never run. - **LLM analysis sends analyzer-eligible file contents to the configured provider.** When LLM analysis is enabled (the default), file contents are sent to the active `SKILLSPECTOR_PROVIDER` endpoint. Recognized OMS signature files are excluded. Use `--no-llm` to keep contents local (static analysis only). - **SC4 sends dependency names to OSV.dev.** The supply-chain check queries [OSV.dev](https://osv.dev) with the package names and versions the skill declares, to look up known CVEs. This is fundamental to the check and runs even with `--no-llm`. It sends dependency coordinates (not file contents), requires no API key, and falls back to a bundled list when OSV.dev is unreachable. - **It does not sandbox the host.** SkillSpector flags risky patterns *before* you install a skill; it does not contain or isolate a skill you choose to install anyway. ## Limitations - **Non-English content**: May miss patterns in other languages - **Image-based attacks**: Cannot analyze text in images - **Encrypted/binary code**: Cannot analyze compiled or encrypted content - **Runtime behavior**: Static analysis only, no dynamic execution - **Offline SC4**: Without network access to `api.osv.dev`, SC4 uses a small static fallback list ## Research Background Based on research from "Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale" (Liu et al., 2026): - **Dataset**: 42,447 skills from major marketplaces; 31,132 were analyzed for the following rates - **Vulnerable**: 26.1% of the analyzed subset contain at least one vulnerability - **High-severity**: 5.2% of the analyzed subset show likely malicious intent - **Key finding**: Skills with executable scripts are 2.12x more likely to be vulnerable ## Python API Integration ```python from skillspector import graph # Invoke the LangGraph workflow result = graph.invoke({ "input_path": "/path/to/skill", "output_format": "json", # terminal, json, markdown, or sarif "use_llm": True, # False for static-only analysis }) # Access results print(f"Risk Score: {result['risk_score']}/100") print(f"Severity: {result['risk_severity']}") print(f"Recommendation: {result['risk_recommendation']}") for finding in result["filtered_findings"]: print(f"[{finding['severity']}] {finding['rule_id']}: {finding['message']}") ``` ## License Apache License 2.0 - see [LICENSE](LICENSE) for details. ## Contributing Contributions are welcome! Please read our contributing guidelines and submit pull requests. ## Support - **Issues**: [GitHub Issues](https://github.com/NVIDIA/skillspector/issues)