@agentsid/scanner 0.1.0

package/README.md

@@ -0,0 +1,205 @@
+<p align="center">
+  <h1 align="center">AgentsID Scanner</h1>
+  <p align="center">
+    <strong>The Lighthouse of agent security.</strong><br/>
+    Scan any MCP server. Get a security report card.
+  </p>
+</p>
+
+<p align="center">
+  <a href="https://agentsid.dev"><img src="https://img.shields.io/badge/by-AgentsID-f59e0b?style=flat-square" alt="AgentsID" /></a>
+  <a href="https://github.com/stevenkozeniesky02/agentsid-scanner/blob/master/LICENSE"><img src="https://img.shields.io/badge/license-MIT-f59e0b?style=flat-square" alt="License" /></a>
+</p>
+
+---
+
+Your MCP server exposes tools to AI agents. How secure is it?
+
+Most MCP servers ship with no authentication, no per-tool permissions, no input validation, and tool descriptions vulnerable to prompt injection. **You just don't know it yet.**
+
+AgentsID Scanner tells you.
+
+## Quick Start
+
+```bash
+npx @agentsid/scanner -- npx @some/mcp-server
+```
+
+That's it. You get a letter grade and detailed findings.
+
+## What It Scans
+
+| Category | What It Checks | Why It Matters |
+|----------|---------------|----------------|
+| **Injection** | Tool descriptions for 11 prompt injection patterns | Malicious tool descriptions can hijack agent behavior |
+| **Permissions** | Tool names classified by risk (destructive, execution, financial, credential) | 50 tools with no access control is a 50-surface attack |
+| **Validation** | Input schemas for missing constraints, unbounded strings, optional-only params | No validation = arbitrary input to your tool handlers |
+| **Auth** | Authentication indicators in tool surface | No auth tools = unauthenticated agents calling your tools |
+| **Secrets** | Tools that may expose credentials in output | API keys, tokens, passwords leaked in responses |
+| **Output** | Unfiltered file/data output | Sensitive file contents returned without redaction |
+
+## The Report
+
+```
+╔══════════════════════════════════════════════════════════════╗
+║          AgentsID Security Scanner — Report                  ║
+╚══════════════════════════════════════════════════════════════╝
+
+Server: my-mcp-server v1.0.0
+Tools:  23
+Scanned: 2026-03-29T12:00:00.000Z
+
+Overall Grade: D (42/100)
+
+Category Grades:
+  injection       A
+  permissions     F
+  validation      D
+  auth            F
+  output          B
+
+Tool Risk Profile:
+  destructive          ████ 4
+  execution            ██ 2
+  credential_access    █ 1
+
+Findings: 31
+  CRITICAL: 2
+  HIGH: 8
+  MEDIUM: 15
+  LOW: 6
+
+Recommendations:
+  1. Address CRITICAL and HIGH findings immediately
+  2. Add per-tool permission controls (agentsid.dev/docs)
+  3. Implement input validation on all tool parameters
+  4. Add authentication to server endpoints
+```
+
+## Usage
+
+### Scan a local MCP server (stdio)
+
+```bash
+# Scan any npx-installable MCP server
+agentsid-scan -- npx @modelcontextprotocol/server-filesystem ./
+
+# Scan a local server file
+agentsid-scan -- node my-server.mjs
+
+# Scan a Python MCP server
+agentsid-scan -- python -m my_mcp_server
+```
+
+### Scan a remote MCP server (HTTP)
+
+```bash
+agentsid-scan --url https://mcp.example.com/mcp
+```
+
+### JSON output
+
+```bash
+agentsid-scan --json -- npx @some/mcp-server > report.json
+```
+
+### Pass environment variables
+
+```bash
+agentsid-scan --env API_KEY=xxx --env DB_URL=postgres://... -- node server.mjs
+```
+
+## Grading
+
+Starts at 100 points. Deductions per finding:
+
+| Severity | Deduction | Example |
+|----------|-----------|---------|
+| CRITICAL | -25 | Shell execution tool with no auth |
+| HIGH | -15 | Tool exposes credentials in output |
+| MEDIUM | -8 | String params without length limits |
+| LOW | -3 | Optional-only input parameters |
+| INFO | 0 | Read-only tool detected |
+
+| Grade | Score | Meaning |
+|-------|-------|---------|
+| A | 90-100 | Excellent security posture |
+| B | 75-89 | Good — minor issues |
+| C | 60-74 | Acceptable — needs improvement |
+| D | 40-59 | Poor — significant risks |
+| F | 0-39 | Failing — critical vulnerabilities |
+
+## Injection Detection
+
+The scanner checks tool descriptions for 11 prompt injection patterns:
+
+- **Instruction override** — "ignore previous instructions", "disregard all rules"
+- **Role hijacking** — "you are now a..."
+- **Memory wipe** — "forget everything"
+- **Tool redirection** — "instead of X, call Y"
+- **Hidden actions** — "also execute..."
+- **Concealment** — "do not tell the user"
+- **Stealth operations** — "secretly", "covertly"
+- **Security bypass** — "override auth", "skip validation"
+- **Encoded payloads** — base64, eval(), template injections
+- **Unicode obfuscation** — escaped characters hiding instructions
+
+## Risk Classification
+
+Every tool is classified by name pattern:
+
+| Risk Level | Patterns | Example Tools |
+|------------|----------|---------------|
+| **Critical** | execute, shell, admin, sudo, payment | `shell_run`, `admin_reset`, `process_payment` |
+| **High** | delete, remove, drop, deploy, credential | `delete_user`, `deploy_prod`, `get_api_key` |
+| **Medium** | create, update, send, write | `create_issue`, `send_email`, `write_file` |
+| **Info** | read, get, list, search, describe | `get_status`, `list_users`, `search_docs` |
+
+## Fix Your Grade
+
+The scanner tells you what's wrong. Here's how to fix it:
+
+### Add per-tool permissions
+
+```bash
+npm install @agentsid/guard
+```
+
+[AgentsID Guard](https://github.com/stevenkozeniesky02/shell-guard) validates every tool call against permission rules before execution. 50 tools, 16 categories, all protected.
+
+### Or add the SDK to your existing server
+
+```bash
+npm install @agentsid/sdk
+```
+
+Three lines of middleware in your MCP server. Full docs at [agentsid.dev/docs](https://agentsid.dev/docs).
+
+## Programmatic Usage
+
+```javascript
+import { scanStdio, scanHttp, scanToolDefinitions } from "@agentsid/scanner";
+
+// Scan a local server
+const report = await scanStdio("npx @some/server", { json: true });
+
+// Scan a remote server
+const report = await scanHttp("https://mcp.example.com", { json: true });
+
+// Scan tool definitions directly (no server needed)
+const report = scanToolDefinitions(myToolArray, { json: true });
+```
+
+## Contributing
+
+Found a pattern we're not detecting? Open an issue or PR. The rule engine is in `src/rules.mjs` — adding a new pattern is one regex.
+
+## Links
+
+- [AgentsID](https://agentsid.dev) — Identity & auth for AI agents
+- [AgentsID Guard](https://github.com/stevenkozeniesky02/shell-guard) — 50-tool protected MCP server
+- [Documentation](https://agentsid.dev/docs)
+
+## License
+
+MIT

package/action/action.yml

@@ -0,0 +1,42 @@
+name: 'AgentsID Security Scanner'
+description: 'Scan MCP server tool definitions for security vulnerabilities. Grades auth, permissions, injection risks, and tool safety.'
+author: 'AgentsID'
+
+branding:
+  icon: 'shield'
+  color: 'orange'
+
+inputs:
+  server-command:
+    description: 'Command to start the MCP server (stdio transport)'
+    required: false
+  server-url:
+    description: 'URL of a remote MCP server (HTTP transport)'
+    required: false
+  tool-definitions:
+    description: 'Path to a JSON file containing tool definitions (static analysis, no server needed)'
+    required: false
+  fail-on:
+    description: 'Fail the check if grade is below this (A, B, C, D, F). Default: no failure.'
+    required: false
+    default: ''
+  comment:
+    description: 'Post results as a PR comment (true/false). Default: true'
+    required: false
+    default: 'true'
+
+outputs:
+  grade:
+    description: 'Overall letter grade (A-F)'
+  score:
+    description: 'Numeric score (0-100)'
+  findings:
+    description: 'Number of findings'
+  critical:
+    description: 'Number of critical findings'
+  report:
+    description: 'Full JSON report'
+
+runs:
+  using: 'node20'
+  main: 'index.mjs'

package/action/index.mjs

@@ -0,0 +1,179 @@
+/**
+ * AgentsID Security Scanner — GitHub Action
+ *
+ * Runs on every PR to an MCP server repo. Scans tool definitions,
+ * grades security posture, and comments on the PR with findings.
+ *
+ * Three scan modes:
+ * 1. server-command: Start an MCP server via stdio and enumerate tools
+ * 2. server-url: Connect to a remote MCP server via HTTP
+ * 3. tool-definitions: Static analysis of a JSON file with tool definitions
+ */
+
+import * as core from "@actions/core";
+import * as github from "@actions/github";
+import { scanStdio, scanHttp, scanToolDefinitions } from "../src/scanner.mjs";
+import { readFileSync } from "fs";
+
+async function run() {
+  try {
+    const serverCommand = core.getInput("server-command");
+    const serverUrl = core.getInput("server-url");
+    const toolDefsPath = core.getInput("tool-definitions");
+    const failOn = core.getInput("fail-on").toUpperCase();
+    const shouldComment = core.getInput("comment") === "true";
+
+    let reportJson;
+
+    // ── Scan ──
+    if (serverCommand) {
+      core.info(`Scanning MCP server: ${serverCommand}`);
+      reportJson = await scanStdio(serverCommand, { json: true, timeout: 60000 });
+    } else if (serverUrl) {
+      core.info(`Scanning remote MCP server: ${serverUrl}`);
+      reportJson = await scanHttp(serverUrl, { json: true, timeout: 30000 });
+    } else if (toolDefsPath) {
+      core.info(`Scanning tool definitions: ${toolDefsPath}`);
+      const tools = JSON.parse(readFileSync(toolDefsPath, "utf-8"));
+      const toolArray = Array.isArray(tools) ? tools : tools.tools || [];
+      reportJson = scanToolDefinitions(toolArray, { json: true, serverName: toolDefsPath });
+    } else {
+      // Auto-detect: look for common MCP tool definition files
+      const autoFiles = [
+        "tools.json",
+        "mcp-tools.json",
+        "src/tools.json",
+        ".mcp/tools.json",
+      ];
+
+      let found = false;
+      for (const f of autoFiles) {
+        try {
+          const content = readFileSync(f, "utf-8");
+          const tools = JSON.parse(content);
+          const toolArray = Array.isArray(tools) ? tools : tools.tools || [];
+          if (toolArray.length > 0) {
+            core.info(`Auto-detected tool definitions: ${f}`);
+            reportJson = scanToolDefinitions(toolArray, { json: true, serverName: f });
+            found = true;
+            break;
+          }
+        } catch {}
+      }
+
+      if (!found) {
+        core.warning("No scan target specified. Use server-command, server-url, or tool-definitions input.");
+        return;
+      }
+    }
+
+    const report = JSON.parse(reportJson);
+
+    // ── Set Outputs ──
+    core.setOutput("grade", report.grade.overall);
+    core.setOutput("score", report.grade.score);
+    core.setOutput("findings", report.findings.length);
+    core.setOutput("critical", report.summary.CRITICAL || 0);
+    core.setOutput("report", reportJson);
+
+    // ── Log Summary ──
+    core.info(`Grade: ${report.grade.overall} (${report.grade.score}/100)`);
+    core.info(`Tools: ${report.toolCount}`);
+    core.info(`Findings: ${report.findings.length} (${report.summary.CRITICAL || 0} critical, ${report.summary.HIGH || 0} high)`);
+
+    // ── PR Comment ──
+    if (shouldComment && github.context.payload.pull_request) {
+      const token = process.env.GITHUB_TOKEN;
+      if (token) {
+        const octokit = github.getOctokit(token);
+        const { owner, repo } = github.context.repo;
+        const prNumber = github.context.payload.pull_request.number;
+
+        const gradeEmoji = {
+          A: "🟢", B: "🟢", C: "🟡", D: "🔴", F: "🔴"
+        }[report.grade.overall] || "⚪";
+
+        // Build category grades table
+        const catRows = Object.entries(report.grade.categories || {})
+          .map(([cat, g]) => `| ${cat} | ${g} |`)
+          .join("\n");
+
+        // Build critical/high findings list
+        const criticalFindings = report.findings
+          .filter((f) => f.severity === "CRITICAL" || f.severity === "HIGH")
+          .slice(0, 10)
+          .map((f) => `- **${f.severity}**: ${f.detail}${f.tool && f.tool !== "*" ? ` (\`${f.tool}\`)` : ""}`)
+          .join("\n");
+
+        const commentBody = `## ${gradeEmoji} AgentsID Security Scan: **${report.grade.overall}** (${report.grade.score}/100)
+
+| Metric | Value |
+|--------|-------|
+| Tools scanned | ${report.toolCount} |
+| Total findings | ${report.findings.length} |
+| Critical | ${report.summary.CRITICAL || 0} |
+| High | ${report.summary.HIGH || 0} |
+| Medium | ${report.summary.MEDIUM || 0} |
+| Low | ${report.summary.LOW || 0} |
+
+### Category Grades
+
+| Category | Grade |
+|----------|-------|
+${catRows}
+
+${criticalFindings ? `### Critical & High Findings\n\n${criticalFindings}` : "### No critical or high findings 🎉"}
+
+${report.grade.score < 60 ? `### Recommendations
+
+1. Add per-tool permission controls — [AgentsID Docs](https://agentsid.dev/docs)
+2. Implement input validation on tool parameters
+3. Add authentication to server endpoints
+4. Use [AgentsID Guard](https://github.com/stevenkozeniesky02/shell-guard) for built-in protection` : ""}
+
+---
+<sub>Scanned by [AgentsID Security Scanner](https://github.com/stevenkozeniesky02/agentsid-scanner) · [Fix your grade](https://agentsid.dev/docs)</sub>`;
+
+        // Check for existing comment and update it
+        const { data: comments } = await octokit.rest.issues.listComments({
+          owner, repo, issue_number: prNumber,
+        });
+
+        const existingComment = comments.find(
+          (c) => c.body?.includes("AgentsID Security Scan")
+        );
+
+        if (existingComment) {
+          await octokit.rest.issues.updateComment({
+            owner, repo, comment_id: existingComment.id, body: commentBody,
+          });
+          core.info("Updated existing PR comment");
+        } else {
+          await octokit.rest.issues.createComment({
+            owner, repo, issue_number: prNumber, body: commentBody,
+          });
+          core.info("Posted PR comment");
+        }
+      } else {
+        core.warning("GITHUB_TOKEN not set — cannot post PR comment");
+      }
+    }
+
+    // ── Fail Check ──
+    if (failOn) {
+      const gradeOrder = { A: 5, B: 4, C: 3, D: 2, F: 1 };
+      const currentGrade = gradeOrder[report.grade.overall] || 0;
+      const requiredGrade = gradeOrder[failOn] || 0;
+
+      if (currentGrade < requiredGrade) {
+        core.setFailed(
+          `Security grade ${report.grade.overall} (${report.grade.score}/100) is below required grade ${failOn}`
+        );
+      }
+    }
+  } catch (err) {
+    core.setFailed(`Scanner failed: ${err.message}`);
+  }
+}
+
+run();