skill-checker 0.1.2 → 0.1.4

package/README.md

@@ -4,49 +4,179 @@ Security checker for Claude Code skills — detect injection, malicious code, an
 
 ## Features
 
-- **48 security rules** across 6 categories: structural validity, content quality, injection detection, code safety, supply chain, and resource abuse
+- **52 security rules** across 6 categories: structural validity, content quality, injection detection, code safety, supply chain, and resource abuse
 - **Scoring system**: Grade A–F with 0–100 score
 - **Dual entry**: CLI tool + PreToolUse hook for automatic interception
 - **Configurable policies**: strict / balanced / permissive approval strategies
+- **Context-aware detection**: severity reduction in code blocks and documentation sections, with zero reduction for injection rules
+- **IOC threat intelligence**: built-in seed data for known malicious hashes, C2 IPs, and typosquat names
 - **Multiple output formats**: terminal (color), JSON, hook response
 
+## Security Standard & Benchmark
+
+Skill Checker's 52 rules are aligned with established security frameworks including OWASP Top 10 for LLM Applications (2025), MITRE CWE, and MITRE ATT&CK. The tool ships with a reproducible benchmark dataset of six fixture skills covering all rule categories. This alignment is an internal mapping exercise — Skill Checker does not claim third-party certification or external audit status.
+
+See [docs/SECURITY_BENCHMARK.md](docs/SECURITY_BENCHMARK.md) for the full rule mapping matrix, benchmark methodology, scoring model, and known limitations.
+
 ## Quick Start
 
 ```bash
+# Install globally
+npm install -g skill-checker
+
 # Scan a skill directory
+skill-checker scan ./path/to/skill/
+
+# Or run without installing
 npx skill-checker scan ./path/to/skill/
+```
 
-# Scan with JSON output
-npx skill-checker scan ./path/to/skill/ --format json
+## Usage
 
-# Scan with strict policy
-npx skill-checker scan ./path/to/skill/ --policy strict
+```bash
+skill-checker scan <path> [options]
 ```
 
-## Installation
+| Option | Description |
+|--------|-------------|
+| `-f, --format <format>` | Output format: `terminal` (default), `json`, `hook` |
+| `-p, --policy <policy>` | Approval policy: `strict`, `balanced` (default), `permissive` |
+| `-c, --config <path>` | Path to config file |
 
 ```bash
-npm install -g skill-checker
+# Colored terminal report
+skill-checker scan ./my-skill
+
+# JSON output for CI/programmatic use
+skill-checker scan ./my-skill --format json
+
+# Hook response format (for PreToolUse integration)
+skill-checker scan ./my-skill --format hook
+
+# Strict policy — deny on HIGH and above
+skill-checker scan ./my-skill --policy strict
+```
+
+Exit code `0` = no critical issues, `1` = critical issues detected.
+
+## Hook Integration
+
+Skill Checker can run automatically as a Claude Code [PreToolUse hook](https://docs.anthropic.com/en/docs/claude-code/hooks), intercepting skill file writes before they happen.
+
+### Setup
+
+```bash
+npx tsx hook/install.ts
+```
+
+This adds a hook entry to `~/.claude/settings.json`:
+
+```json
+{
+  "hooks": {
+    "PreToolUse": [
+      {
+        "matcher": "Write|Edit",
+        "hook": "/path/to/skill-gate.sh"
+      }
+    ]
+  }
+}
 ```
 
+### How It Works
+
+1. Claude Code intercepts Write/Edit operations targeting SKILL.md files
+2. `skill-gate.sh` receives the file content via stdin (JSON)
+3. Runs `skill-checker scan --format hook` on the content
+4. Returns a permission decision: `allow`, `ask`, or `deny`
+
+The hook is fail-closed — if the scanner is unavailable, JSON parsing fails, or any unexpected error occurs, it returns `ask` (never silently allows).
+
+### Requirements
+
+- `jq` must be installed for JSON parsing
+- `skill-checker` must be globally installed or available via `npx`
+
+## Scoring
+
+Base score starts at **100**. Each finding deducts points by severity:
+
+| Severity | Deduction |
+|----------|-----------|
+| CRITICAL | -25 |
+| HIGH | -10 |
+| MEDIUM | -3 |
+| LOW | -1 |
+
+| Grade | Score | Meaning |
+|-------|-------|---------|
+| A | 90–100 | Safe to install |
+| B | 75–89 | Minor issues |
+| C | 60–74 | Review advised |
+| D | 40–59 | Significant risk |
+| F | 0–39 | Not recommended |
+
 ## Configuration
 
-Create a `.skillcheckerrc.yaml` in your project root or home directory:
+Create `.skillcheckerrc.yaml` in your project root or home directory:
 
 ```yaml
-policy: balanced
+# Approval policy
+policy: balanced    # strict / balanced / permissive
 
+# Override severity for specific rules
 overrides:
-  CODE-006: LOW
+  CODE-006: LOW     # env var access is expected in my skills
+  SUPPLY-002: LOW   # I trust npx -y in my workflow
 
+# Ignore rules entirely
 ignore:
-  - CONT-006
+  - CONT-006        # reference-heavy skills are fine
+```
+
+Config is resolved in order: CLI `--config` flag → project directory (walks up) → home directory → defaults.
+
+### Policy Matrix
+
+| Severity | strict | balanced | permissive |
+|----------|--------|----------|------------|
+| CRITICAL | deny | deny | ask |
+| HIGH | deny | ask | report |
+| MEDIUM | ask | report | report |
+| LOW | report | report | report |
+
+## Rule Categories
+
+| Category | Rules | Examples |
+|----------|-------|---------|
+| Structural (STRUCT) | 8 | Missing SKILL.md, invalid frontmatter, binary files |
+| Content (CONT) | 7 | Placeholder text, lorem ipsum, promotional content |
+| Injection (INJ) | 9 | Zero-width chars, prompt override, tag injection, encoded payloads |
+| Code Safety (CODE) | 12 | eval/exec, shell execution, rm -rf, obfuscation |
+| Supply Chain (SUPPLY) | 10 | Unknown MCP servers, suspicious domains, malicious hashes, typosquat |
+| Resource Abuse (RES) | 6 | Unrestricted tool access, disable safety checks, ignore project rules |
+
+See [docs/SECURITY_BENCHMARK.md](docs/SECURITY_BENCHMARK.md) for the complete rule mapping with OWASP/CWE/ATT&CK references.
+
+## Programmatic API
+
+```typescript
+import { scanSkillDirectory } from 'skill-checker';
+
+const report = scanSkillDirectory('./my-skill', {
+  policy: 'strict',
+  overrides: { 'CODE-006': 'LOW' },
+  ignore: ['CONT-001'],
+});
+
+console.log(report.grade, report.score, report.results.length);
 ```
 
 ## License
 
-This project is licensed under the **GNU Affero General Public License v3.0 (AGPLv3)** - see the [LICENSE](LICENSE) file for details.
+This project is licensed under the **GNU Affero General Public License v3.0 (AGPLv3)** — see the [LICENSE](LICENSE) file for details.
 
-**商业授权 (Commercial License)**
+**Commercial License (商业授权)**
 
-如果您希望将本工具集成到闭源的商业产品、SaaS 服务中，或者由于公司合规原因无法遵守 AGPLv3 协议，请通过 Alexander.kinging@gmail.com 联系作者购买商业授权。
+If you want to integrate this tool into a closed-source commercial product or SaaS, or cannot comply with AGPLv3 due to company policy, contact Alexander.kinging@gmail.com for a commercial license.

package/dist/cli.js

@@ -1,16 +1,9 @@
-var __require = /* @__PURE__ */ ((x) => typeof require !== "undefined" ? require : typeof Proxy !== "undefined" ? new Proxy(x, {
-  get: (a, b) => (typeof require !== "undefined" ? require : a)[b]
-}) : x)(function(x) {
-  if (typeof require !== "undefined") return require.apply(this, arguments);
-  throw Error('Dynamic require of "' + x + '" is not supported');
-});
-
 // src/cli.ts
 import { createRequire } from "module";
 import { Command } from "commander";
 
 // src/parser.ts
-import { readFileSync, readdirSync, statSync, existsSync } from "fs";
+import { readFileSync, readdirSync, lstatSync, existsSync, openSync, readSync, closeSync } from "fs";
 import { join, extname, basename, resolve } from "path";
 import { parse as parseYaml } from "yaml";
 var BINARY_EXTENSIONS = /* @__PURE__ */ new Set([
@@ -124,52 +117,66 @@ function enumerateFiles(dirPath, warnings) {
     }
     for (const entry of entries) {
       const fullPath = join(currentDir, entry.name);
-      if (entry.isDirectory()) {
+      const relativePath = fullPath.slice(dirPath.length + 1);
+      let lstats;
+      try {
+        lstats = lstatSync(fullPath);
+      } catch {
+        continue;
+      }
+      if (lstats.isSymbolicLink()) {
+        warnings.push(`Skipped symlink: ${relativePath}`);
+        continue;
+      }
+      if (lstats.isDirectory()) {
         if (SKIP_DIRS.has(entry.name)) continue;
         if (WARN_SKIP_DIRS.has(entry.name)) {
-          const rel = fullPath.slice(dirPath.length + 1);
-          warnings.push(`Skipped directory: ${rel}. May contain unscanned files.`);
+          warnings.push(`Skipped directory: ${relativePath}. May contain unscanned files.`);
           continue;
         }
         walk(fullPath, depth + 1);
         continue;
       }
-      const ext = extname(entry.name).toLowerCase();
-      let stats;
-      try {
-        stats = statSync(fullPath);
-      } catch {
+      if (!lstats.isFile()) {
+        warnings.push(`Skipped special file: ${relativePath}`);
         continue;
       }
+      const ext = extname(entry.name).toLowerCase();
       const isBinary = BINARY_EXTENSIONS.has(ext);
-      const relativePath = fullPath.slice(dirPath.length + 1);
       let content;
       if (!isBinary) {
-        if (stats.size <= FULL_READ_LIMIT) {
+        if (lstats.size <= FULL_READ_LIMIT) {
           try {
             content = readFileSync(fullPath, "utf-8");
           } catch {
           }
-        } else if (stats.size <= 5e7) {
+        } else if (lstats.size <= 5e7) {
+          let fd;
           try {
-            const fd = __require("fs").openSync(fullPath, "r");
+            fd = openSync(fullPath, "r");
             const buf = Buffer.alloc(PARTIAL_READ_LIMIT);
-            const bytesRead = __require("fs").readSync(fd, buf, 0, PARTIAL_READ_LIMIT, 0);
-            __require("fs").closeSync(fd);
+            const bytesRead = readSync(fd, buf, 0, PARTIAL_READ_LIMIT, 0);
             content = buf.slice(0, bytesRead).toString("utf-8");
-            warnings.push(`Large file partially scanned (first ${PARTIAL_READ_LIMIT} bytes): ${relativePath} (${stats.size} bytes total)`);
+            warnings.push(`Large file partially scanned (first ${PARTIAL_READ_LIMIT} bytes): ${relativePath} (${lstats.size} bytes total)`);
           } catch {
-            warnings.push(`Large file could not be read: ${relativePath} (${stats.size} bytes)`);
+            warnings.push(`Large file could not be read: ${relativePath} (${lstats.size} bytes)`);
+          } finally {
+            if (fd !== void 0) {
+              try {
+                closeSync(fd);
+              } catch {
+              }
+            }
           }
         } else {
-          warnings.push(`File too large to scan: ${relativePath} (${stats.size} bytes). Content not checked.`);
+          warnings.push(`File too large to scan: ${relativePath} (${lstats.size} bytes). Content not checked.`);
         }
       }
       files.push({
         path: relativePath,
         name: basename(entry.name, ext),
         extension: ext,
-        sizeBytes: stats.size,
+        sizeBytes: lstats.size,
         isBinary,
         content
       });
@@ -261,7 +268,8 @@ var structuralChecks = {
           category: "STRUCT",
           severity: "HIGH",
           title: "Unexpected binary/executable file",
-          message: `Found unexpected file: ${file.path} (${ext})`
+          message: `Found unexpected file: ${file.path} (${ext})`,
+          source: file.path
         });
       }
     }
@@ -287,12 +295,14 @@ var structuralChecks = {
       }
     }
     for (const warning of skill.warnings) {
+      const pathMatch = warning.match(/:\s*(.+?)(?:\s*\(|$)/);
       results.push({
         id: "STRUCT-008",
         category: "STRUCT",
         severity: "MEDIUM",
         title: "Scan coverage warning",
-        message: warning
+        message: warning,
+        source: pathMatch?.[1]?.trim()
       });
     }
     return results;
@@ -1004,7 +1014,11 @@ function getHookAction(policy, severity) {
       LOW: "report"
     }
   };
-  return matrix[policy][severity];
+  const row = matrix[policy];
+  if (!row) {
+    return matrix.balanced[severity];
+  }
+  return row[severity];
 }
 
 // src/checks/code-safety.ts
@@ -1096,7 +1110,8 @@ var codeSafetyChecks = {
           severity: "CRITICAL",
           title: "eval/exec/Function constructor",
           loc,
-          lineNum
+          lineNum,
+          source
         });
         if (!SHELL_EXEC_FALSE_POSITIVES.some((p) => p.test(line))) {
           checkPatterns(results, line, SHELL_EXEC_PATTERNS, {
@@ -1104,7 +1119,8 @@ var codeSafetyChecks = {
             severity: "CRITICAL",
             title: "Shell/subprocess execution",
             loc,
-            lineNum
+            lineNum,
+            source
           });
         }
         checkPatterns(results, line, DESTRUCTIVE_PATTERNS, {
@@ -1113,6 +1129,7 @@ var codeSafetyChecks = {
           title: "Destructive file operation",
           loc,
           lineNum,
+          source,
           codeBlockCtx: cbCtx
         });
         checkPatterns(results, line, NETWORK_PATTERNS, {
@@ -1121,6 +1138,7 @@ var codeSafetyChecks = {
           title: "Hardcoded external URL/network request",
           loc,
           lineNum,
+          source,
           codeBlockCtx: cbCtx
         });
         checkPatterns(results, line, FILE_WRITE_PATTERNS, {
@@ -1128,7 +1146,8 @@ var codeSafetyChecks = {
           severity: "HIGH",
           title: "File write outside expected directory",
           loc,
-          lineNum
+          lineNum,
+          source
         });
         checkPatterns(results, line, ENV_ACCESS_PATTERNS, {
           id: "CODE-006",
@@ -1136,6 +1155,7 @@ var codeSafetyChecks = {
           title: "Environment variable access",
           loc,
           lineNum,
+          source,
           codeBlockCtx: cbCtx
         });
         checkPatterns(results, line, DYNAMIC_CODE_PATTERNS, {
@@ -1143,7 +1163,8 @@ var codeSafetyChecks = {
           severity: "HIGH",
           title: "Dynamic code generation pattern",
           loc,
-          lineNum
+          lineNum,
+          source
         });
         {
           const isDoc = isInDocumentationContext(lines, i);
@@ -1153,7 +1174,8 @@ var codeSafetyChecks = {
               severity: "HIGH",
               title: "Permission escalation",
               loc,
-              lineNum
+              lineNum,
+              source
             });
           }
         }
@@ -1184,6 +1206,7 @@ function checkPatterns(results, line, patterns, opts) {
         message: `At ${opts.loc}: ${line.trim().slice(0, 120)}${msgSuffix}`,
         line: opts.lineNum,
         snippet: line.trim().slice(0, 120),
+        source: opts.source,
         reducedFrom
       });
       return;
@@ -1215,7 +1238,8 @@ function scanEncodedStrings(results, text, source) {
         title: "Long encoded string",
         message: `${source}:${lineNum}: Found ${str.length}-char encoded string.`,
         line: lineNum,
-        snippet: str.slice(0, 80) + "..."
+        snippet: str.slice(0, 80) + "...",
+        source
       });
     }
   }
@@ -1230,7 +1254,8 @@ function scanEncodedStrings(results, text, source) {
         severity: "MEDIUM",
         title: "High entropy string",
         message: `${source}:${lineNum}: String "${match[0].slice(0, 30)}..." has entropy ${entropy.toFixed(2)} bits/char.`,
-        line: lineNum
+        line: lineNum,
+        source
       });
     }
   }
@@ -1247,7 +1272,8 @@ function scanEncodedStrings(results, text, source) {
         category: "CODE",
         severity: "CRITICAL",
         title: "Multi-layer encoding detected",
-        message: `${source}: Contains nested encoding/decoding operations.`
+        message: `${source}: Contains nested encoding/decoding operations.`,
+        source
       });
       break;
     }
@@ -1262,7 +1288,8 @@ function scanObfuscation(results, text, source) {
       category: "CODE",
       severity: "MEDIUM",
       title: "Obfuscated variable names",
-      message: `${source}: Found ${obfMatches.length} hex-style variable names (e.g. ${obfMatches[0]}). May indicate obfuscated code.`
+      message: `${source}: Found ${obfMatches.length} hex-style variable names (e.g. ${obfMatches[0]}). May indicate obfuscated code.`,
+      source
     });
   }
 }
@@ -1453,6 +1480,7 @@ var supplyChainChecks = {
           message: `${source}:${lineNum}: References an MCP server. Verify it is from a trusted source.${msgSuffix}`,
           line: lineNum,
           snippet: line.trim().slice(0, 120),
+          source,
           reducedFrom
         });
       }
@@ -1464,7 +1492,8 @@ var supplyChainChecks = {
           title: "npx -y auto-install",
           message: `${source}:${lineNum}: Uses npx -y which auto-installs packages without confirmation.`,
           line: lineNum,
-          snippet: line.trim().slice(0, 120)
+          snippet: line.trim().slice(0, 120),
+          source
         });
       }
       if (NPM_INSTALL_PATTERN.test(line) || PIP_INSTALL_PATTERN.test(line)) {
@@ -1494,6 +1523,7 @@ var supplyChainChecks = {
             message: `${source}:${lineNum}: Installs packages. Verify package names are legitimate.${msgSuffix}`,
             line: lineNum,
             snippet: line.trim().slice(0, 120),
+            source,
             reducedFrom
           });
         }
@@ -1506,7 +1536,8 @@ var supplyChainChecks = {
           title: "git clone command",
           message: `${source}:${lineNum}: Clones a git repository. Verify the source.`,
           line: lineNum,
-          snippet: line.trim().slice(0, 120)
+          snippet: line.trim().slice(0, 120),
+          source
         });
       }
       const urls = line.match(URL_PATTERN) || [];
@@ -1535,6 +1566,7 @@ var supplyChainChecks = {
               message: `${source}:${lineNum}: Uses insecure HTTP: ${url}${msgSuffix}`,
               line: lineNum,
               snippet: url,
+              source,
               reducedFrom
             });
           }
@@ -1548,7 +1580,8 @@ var supplyChainChecks = {
               title: "IP address used instead of domain",
               message: `${source}:${lineNum}: Uses raw IP address: ${url}. This may bypass DNS-based security.`,
               line: lineNum,
-              snippet: url
+              snippet: url,
+              source
             });
           }
         }
@@ -1562,7 +1595,8 @@ var supplyChainChecks = {
               title: "Suspicious domain detected",
               message: `${source}:${lineNum}: References suspicious domain "${domain}".`,
               line: lineNum,
-              snippet: url
+              snippet: url,
+              source
             });
             break;
           }
@@ -1762,7 +1796,7 @@ var resourceChecks = {
 
 // src/ioc/matcher.ts
 import { createHash } from "crypto";
-import { openSync, readSync, closeSync, statSync as statSync2 } from "fs";
+import { openSync as openSync2, readSync as readSync2, closeSync as closeSync2, statSync } from "fs";
 import { join as join3 } from "path";
 
 // src/utils/levenshtein.ts
@@ -1811,18 +1845,18 @@ function isPrivateIP(ip) {
 var HASH_CHUNK_SIZE = 64 * 1024;
 function computeFileHash(filePath) {
   const hash = createHash("sha256");
-  const fd = openSync(filePath, "r");
+  const fd = openSync2(filePath, "r");
   try {
     const buf = Buffer.alloc(HASH_CHUNK_SIZE);
     let bytesRead;
     do {
-      bytesRead = readSync(fd, buf, 0, HASH_CHUNK_SIZE, null);
+      bytesRead = readSync2(fd, buf, 0, HASH_CHUNK_SIZE, null);
       if (bytesRead > 0) {
         hash.update(buf.subarray(0, bytesRead));
       }
     } while (bytesRead > 0);
   } finally {
-    closeSync(fd);
+    closeSync2(fd);
   }
   return hash.digest("hex");
 }
@@ -1833,7 +1867,7 @@ function matchMaliciousHashes(skill, ioc) {
   for (const file of skill.files) {
     const filePath = join3(skill.dirPath, file.path);
     try {
-      const stat = statSync2(filePath);
+      const stat = statSync(filePath);
       if (stat.size === 0) continue;
       const hash = computeFileHash(filePath);
       if (hash === EMPTY_FILE_HASH) continue;
@@ -1924,7 +1958,8 @@ var iocChecks = {
         severity: "CRITICAL",
         title: "Known malicious file hash",
         message: `File "${match.file}" matches known malicious hash: ${match.description}`,
-        snippet: match.hash
+        snippet: match.hash,
+        source: match.file
       });
     }
     const ipMatches = matchC2IPs(skill, ioc);
@@ -1936,7 +1971,8 @@ var iocChecks = {
         title: "Known C2 IP address",
         message: `${match.source}:${match.line}: Contains known C2 server IP: ${match.ip}`,
         line: match.line,
-        snippet: match.snippet
+        snippet: match.snippet,
+        source: match.source
       });
     }
     const skillName = skill.frontmatter.name;
@@ -2029,8 +2065,8 @@ function deduplicateResults(results) {
     LOW: 1
   };
   for (const r of results) {
-    const sourceFile = extractSourceFile(r.message);
-    const key = `${r.id}::${sourceFile}`;
+    const sourceKey = r.source ?? `_no_source_:${r.category}:${r.line ?? ""}`;
+    const key = `${r.id}::${sourceKey}`;
     const group = groups.get(key);
     if (group) {
       group.push(r);
@@ -2044,16 +2080,13 @@ function deduplicateResults(results) {
     const best = { ...group[0] };
     if (group.length > 1) {
       best.occurrences = group.length;
-      best.message += ` (${group.length} occurrences in this file)`;
+      const suffix = best.source ? ` (${group.length} occurrences in this file)` : ` (${group.length} occurrences)`;
+      best.message += suffix;
     }
     deduped.push(best);
   }
   return deduped;
 }
-function extractSourceFile(message) {
-  const m = message.match(/^(?:At\s+)?([^:]+):\d+:/);
-  return m?.[1] ?? "unknown";
-}
 function calculateScore(results) {
   let score = 100;
   for (const r of results) {
@@ -2328,8 +2361,13 @@ var program = new Command();
 program.name("skill-checker").description(
   "Security checker for Claude Code skills - detect injection, malicious code, and supply chain risks"
 ).version(pkg.version);
+var VALID_POLICIES = ["strict", "balanced", "permissive"];
 program.command("scan").description("Scan a skill directory for security issues").argument("<path>", "Path to the skill directory").option("-f, --format <format>", "Output format: terminal, json, hook", "terminal").option("-p, --policy <policy>", "Policy: strict, balanced, permissive").option("-c, --config <path>", "Path to config file").action(
   (path, opts) => {
+    if (opts.policy && !VALID_POLICIES.includes(opts.policy)) {
+      console.error(`Error: invalid policy "${opts.policy}". Valid values: ${VALID_POLICIES.join(", ")}`);
+      process.exit(1);
+    }
     const config = loadConfig(path, opts.config);
     if (opts.policy) {
       config.policy = opts.policy;