@apitella/scan 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +146 -0
- package/dist/analyze.js +134 -0
- package/dist/baseline.js +0 -0
- package/dist/fetch.js +162 -0
- package/dist/index.js +167 -0
- package/dist/report.js +126 -0
- package/dist/schema-types.js +4 -0
- package/package.json +52 -0
package/README.md
ADDED
|
@@ -0,0 +1,146 @@
|
|
|
1
|
+
# @apitella/scan
|
|
2
|
+
|
|
3
|
+
Check an MCP server's tool surface for the gaps that bite agents in production —
|
|
4
|
+
missing safety annotations, descriptions that contradict their own hints, credential
|
|
5
|
+
strings and prompt-injection phrasing in tool text — at **build time**, in **CI**, or
|
|
6
|
+
against a **live URL**.
|
|
7
|
+
|
|
8
|
+
Same checks as the free scanner at [apitella.com/scan](https://apitella.com/scan),
|
|
9
|
+
runnable where you build the server.
|
|
10
|
+
|
|
11
|
+
```
|
|
12
|
+
npx @apitella/scan http://localhost:3000/mcp
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
```
|
|
16
|
+
Apitella scan my-warehouse-mcp v1.4.0
|
|
17
|
+
6 tools · 2 resources
|
|
18
|
+
|
|
19
|
+
▲ create_shipment [no-safety-hints]
|
|
20
|
+
Neither readOnlyHint nor destructiveHint is declared — an agent can't tell
|
|
21
|
+
whether this is safe to auto-approve.
|
|
22
|
+
▲ delete_record [destructive-hint-mismatch]
|
|
23
|
+
Description reads as destructive, but destructiveHint isn't set to true — an
|
|
24
|
+
agent skipping confirmation has no signal this is dangerous.
|
|
25
|
+
· server [no-version-signal]
|
|
26
|
+
No changelog or version signal on this server — if any of this changes,
|
|
27
|
+
nothing tells you unless it's being monitored.
|
|
28
|
+
|
|
29
|
+
2 warnings — see above.
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
## Usage
|
|
33
|
+
|
|
34
|
+
| | |
|
|
35
|
+
| --------------------------------------- | ----------------------------------------------------------------------------------- |
|
|
36
|
+
| `npx @apitella/scan <url>` | Connect (Streamable HTTP, falling back to SSE) and scan. Works against `localhost`. |
|
|
37
|
+
| `npx @apitella/scan --input tools.json` | Scan a saved `tools/list` dump — no running server needed. |
|
|
38
|
+
| `-H, --header "X-API-Key: ..."` | Extra request header. Repeatable. |
|
|
39
|
+
| `--token <t>` | Bearer token shorthand. |
|
|
40
|
+
| `--baseline [file]` | Diff the scan against a committed snapshot (default `.apitella/baseline.json`). |
|
|
41
|
+
| `--update-baseline` | Write the current surface to the baseline file and exit 0. |
|
|
42
|
+
| `--json` | Machine-readable output (includes the baseline diff when `--baseline` is set). |
|
|
43
|
+
| `--fail-on <error\|warning>` | Non-zero exit threshold (default: `warning`). Ignored with `--baseline`. |
|
|
44
|
+
| `--no-fail` | Always exit 0 unless the scan itself failed. |
|
|
45
|
+
|
|
46
|
+
### Exit codes
|
|
47
|
+
|
|
48
|
+
| Code | Meaning |
|
|
49
|
+
| ---- | ------------------------------------------------------------------------ |
|
|
50
|
+
| `0` | Clean, below `--fail-on`, no regression vs. `--baseline`, or `--no-fail` |
|
|
51
|
+
| `1` | Findings at or above `--fail-on`, or a regression vs. `--baseline` |
|
|
52
|
+
| `2` | Usage error, or couldn't reach / parse the target |
|
|
53
|
+
|
|
54
|
+
## Baseline: make schema changes show up in code review
|
|
55
|
+
|
|
56
|
+
Commit a snapshot of your server's surface, then fail a PR only when the change makes
|
|
57
|
+
it **worse** — a tool removed, a tool that lost its `readOnlyHint` / `destructiveHint`,
|
|
58
|
+
or a new error/warning. Pre-existing findings you've decided to live with don't fail
|
|
59
|
+
the build.
|
|
60
|
+
|
|
61
|
+
```
|
|
62
|
+
# once, checked in:
|
|
63
|
+
npx @apitella/scan http://localhost:3000/mcp --update-baseline
|
|
64
|
+
git add .apitella/baseline.json
|
|
65
|
+
|
|
66
|
+
# on every PR:
|
|
67
|
+
npx @apitella/scan http://localhost:3000/mcp --baseline
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
```
|
|
71
|
+
vs. baseline .apitella/baseline.json
|
|
72
|
+
− removed get_user
|
|
73
|
+
+ added delete_user
|
|
74
|
+
! delete_user lost its safety hint — it declared destructiveHint in the baseline and now declares neither.
|
|
75
|
+
▲ new delete_user [no-safety-hints]
|
|
76
|
+
|
|
77
|
+
This change makes the MCP surface worse than the baseline.
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
When a change is intentional, re-run `--update-baseline` and commit the new snapshot —
|
|
81
|
+
the diff lands in the PR alongside the code.
|
|
82
|
+
|
|
83
|
+
## In CI
|
|
84
|
+
|
|
85
|
+
### GitHub Action
|
|
86
|
+
|
|
87
|
+
```yaml
|
|
88
|
+
# .github/workflows/mcp-scan.yml
|
|
89
|
+
- uses: actions/checkout@v4
|
|
90
|
+
- uses: actions/setup-node@v4
|
|
91
|
+
with: { node-version: 20 }
|
|
92
|
+
- run: |
|
|
93
|
+
npm ci && npm run build
|
|
94
|
+
node dist/server.js &
|
|
95
|
+
npx --yes wait-on http://localhost:3000/mcp
|
|
96
|
+
- uses: dodogeny/apitella-saas/cli@v1
|
|
97
|
+
with:
|
|
98
|
+
url: http://localhost:3000/mcp
|
|
99
|
+
baseline: .apitella/baseline.json
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
Action inputs: `url` **or** `input` (a `tools/list` dump), `baseline`, `fail-on`,
|
|
103
|
+
`headers` (newline-separated `Name: value`), `package-version`. A full example is in
|
|
104
|
+
[`examples/mcp-scan.yml`](./examples/mcp-scan.yml).
|
|
105
|
+
|
|
106
|
+
### Plain npx
|
|
107
|
+
|
|
108
|
+
```yaml
|
|
109
|
+
- run: npx @apitella/scan --input tools.json --fail-on error
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
## What it checks
|
|
113
|
+
|
|
114
|
+
- **`no-safety-hints`** — a tool declares neither `readOnlyHint` nor `destructiveHint`.
|
|
115
|
+
- **`destructive-hint-mismatch`** — the description reads as destructive ("deletes…",
|
|
116
|
+
"wipes…") but `destructiveHint` isn't `true`.
|
|
117
|
+
- **`readonly-hint-contradicted`** — `readOnlyHint: true` on a tool that takes a
|
|
118
|
+
`content` / `body` / `payload` / `data` parameter.
|
|
119
|
+
- **`instruction-injection`** — tool or parameter text contains instruction-override
|
|
120
|
+
phrasing or invisible characters aimed at the model reading it.
|
|
121
|
+
- **`credential-exposed`** — an AWS key, GitHub/Slack token, or private-key block in
|
|
122
|
+
tool text.
|
|
123
|
+
- **`insecure-transport`** — the server URL is plain `http://`.
|
|
124
|
+
|
|
125
|
+
The CVE lookup and the LLM-backed semantic injection scan that
|
|
126
|
+
[apitella.com](https://apitella.com) runs are not in this CLI — it stays offline and
|
|
127
|
+
fast. Continuous monitoring of a deployed server (and of servers you depend on) is the
|
|
128
|
+
hosted product.
|
|
129
|
+
|
|
130
|
+
## What it does not do
|
|
131
|
+
|
|
132
|
+
It does not send your schema anywhere. Everything runs locally.
|
|
133
|
+
|
|
134
|
+
## Development
|
|
135
|
+
|
|
136
|
+
```
|
|
137
|
+
npm install
|
|
138
|
+
npm run build
|
|
139
|
+
npm test
|
|
140
|
+
npm run dev -- http://localhost:3000/mcp # run from source
|
|
141
|
+
```
|
|
142
|
+
|
|
143
|
+
The rule logic in `src/analyze.ts` is a port of the transport-free checks in the
|
|
144
|
+
Apitella API (`api/src/lib/scanAnalysis.ts` and the synchronous half of
|
|
145
|
+
`api/src/lib/securityChecks.ts`). The two are kept in sync by hand for now; the tests
|
|
146
|
+
in `test/` lock the CLI's behaviour so a divergence is caught.
|
package/dist/analyze.js
ADDED
|
@@ -0,0 +1,134 @@
|
|
|
1
|
+
// Narrow, low-false-positive credential shapes — same approach gitleaks/trufflehog take.
|
|
2
|
+
const CREDENTIAL_PATTERNS = [
|
|
3
|
+
{ name: "AWS access key ID", pattern: /\bAKIA[0-9A-Z]{16}\b/ },
|
|
4
|
+
{ name: "GitHub token", pattern: /\bgh[pousr]_[A-Za-z0-9]{36,}\b/ },
|
|
5
|
+
{ name: "Slack token", pattern: /\bxox[baprs]-[A-Za-z0-9-]{10,}\b/ },
|
|
6
|
+
{
|
|
7
|
+
name: "private key block",
|
|
8
|
+
pattern: /-----BEGIN (RSA |EC |OPENSSH )?PRIVATE KEY-----/,
|
|
9
|
+
},
|
|
10
|
+
];
|
|
11
|
+
// Override / injection phrasing aimed at whatever model reads the description, not at a
|
|
12
|
+
// human skimming a tool list. Clear imperative overrides only — "ignore" alone shows up in
|
|
13
|
+
// plenty of legitimate text ("ignores trailing whitespace").
|
|
14
|
+
const INJECTION_PATTERNS = [
|
|
15
|
+
/ignore (all |any |previous |prior |the above )+instructions?/i,
|
|
16
|
+
/disregard (all |any |previous |prior |the above )+instructions?/i,
|
|
17
|
+
/you (must|should|will) always\b/i,
|
|
18
|
+
/do not (tell|inform|mention|notify) the user/i,
|
|
19
|
+
/\bsystem prompt\s*:/i,
|
|
20
|
+
/\bnew instructions?\s*:/i,
|
|
21
|
+
// Zero-width / invisible-formatting characters — hides text from a human reviewer while
|
|
22
|
+
// a model still reads it.
|
|
23
|
+
/[]/,
|
|
24
|
+
];
|
|
25
|
+
const DESTRUCTIVE_VERBS = /\b(delete|deletes|deleted|deleting|remove|removes|removed|removing|drop|drops|dropped|dropping|wipe|wipes|wiped|wiping|purge|purges|purged|purging|destroy|destroys|destroyed|destroying|erase|erases|erased|erasing)\b/i;
|
|
26
|
+
// Exact param-name match — catch a payload param named exactly "content" without
|
|
27
|
+
// false-positiving on "contentType" or "dataSource".
|
|
28
|
+
const WRITE_SHAPED_PARAM_NAMES = new Set([
|
|
29
|
+
"content",
|
|
30
|
+
"body",
|
|
31
|
+
"payload",
|
|
32
|
+
"data",
|
|
33
|
+
]);
|
|
34
|
+
function scanText(text, operationId, location, findings) {
|
|
35
|
+
if (!text)
|
|
36
|
+
return;
|
|
37
|
+
const where = location ? `Parameter "${location}"` : "Description";
|
|
38
|
+
for (const { name, pattern } of CREDENTIAL_PATTERNS) {
|
|
39
|
+
if (pattern.test(text)) {
|
|
40
|
+
findings.push({
|
|
41
|
+
severity: "error",
|
|
42
|
+
rule: "credential-exposed",
|
|
43
|
+
operation: operationId,
|
|
44
|
+
message: `${where} contains what looks like a ${name}.`,
|
|
45
|
+
});
|
|
46
|
+
}
|
|
47
|
+
}
|
|
48
|
+
for (const pattern of INJECTION_PATTERNS) {
|
|
49
|
+
if (pattern.test(text)) {
|
|
50
|
+
findings.push({
|
|
51
|
+
severity: "error",
|
|
52
|
+
rule: "instruction-injection",
|
|
53
|
+
operation: operationId,
|
|
54
|
+
message: `${where} contains text that reads as an instruction override rather than documentation.`,
|
|
55
|
+
});
|
|
56
|
+
break;
|
|
57
|
+
}
|
|
58
|
+
}
|
|
59
|
+
}
|
|
60
|
+
export function analyze(schema, url) {
|
|
61
|
+
const findings = [];
|
|
62
|
+
const toolOps = schema.operations.filter((op) => !op.category);
|
|
63
|
+
const resourceCount = schema.operations.filter((op) => op.category === "resource").length;
|
|
64
|
+
const resourceTemplateCount = schema.operations.filter((op) => op.category === "resource_template").length;
|
|
65
|
+
const promptCount = schema.operations.filter((op) => op.category === "prompt").length;
|
|
66
|
+
if (url && url.startsWith("http://")) {
|
|
67
|
+
findings.push({
|
|
68
|
+
severity: "warning",
|
|
69
|
+
rule: "insecure-transport",
|
|
70
|
+
operation: null,
|
|
71
|
+
message: `${url} is served over plain HTTP, not HTTPS — everything in the exchange crosses the wire readable.`,
|
|
72
|
+
});
|
|
73
|
+
}
|
|
74
|
+
for (const op of schema.operations) {
|
|
75
|
+
scanText(op.description, op.id, null, findings);
|
|
76
|
+
for (const param of op.params) {
|
|
77
|
+
scanText(param.description, op.id, param.name, findings);
|
|
78
|
+
}
|
|
79
|
+
// Safety hints are a tool-only concept — resources/prompts carry a category, tools
|
|
80
|
+
// never do.
|
|
81
|
+
if (op.category)
|
|
82
|
+
continue;
|
|
83
|
+
const readOnlyHint = op.annotations?.readOnlyHint;
|
|
84
|
+
const destructiveHint = op.annotations?.destructiveHint;
|
|
85
|
+
if (op.description &&
|
|
86
|
+
DESTRUCTIVE_VERBS.test(op.description) &&
|
|
87
|
+
destructiveHint !== true) {
|
|
88
|
+
findings.push({
|
|
89
|
+
severity: "warning",
|
|
90
|
+
rule: "destructive-hint-mismatch",
|
|
91
|
+
operation: op.id,
|
|
92
|
+
message: "Description reads as destructive, but destructiveHint isn't set to true — an agent skipping confirmation has no signal this is dangerous.",
|
|
93
|
+
});
|
|
94
|
+
}
|
|
95
|
+
if (readOnlyHint === true) {
|
|
96
|
+
const writeParam = op.params.find((p) => WRITE_SHAPED_PARAM_NAMES.has(p.name.toLowerCase()));
|
|
97
|
+
if (writeParam) {
|
|
98
|
+
findings.push({
|
|
99
|
+
severity: "warning",
|
|
100
|
+
rule: "readonly-hint-contradicted",
|
|
101
|
+
operation: op.id,
|
|
102
|
+
message: `Declared readOnlyHint, but takes a "${writeParam.name}" parameter — that's what a write payload looks like, not a read.`,
|
|
103
|
+
});
|
|
104
|
+
}
|
|
105
|
+
}
|
|
106
|
+
if (readOnlyHint === undefined && destructiveHint === undefined) {
|
|
107
|
+
findings.push({
|
|
108
|
+
severity: "warning",
|
|
109
|
+
rule: "no-safety-hints",
|
|
110
|
+
operation: op.id,
|
|
111
|
+
message: "Neither readOnlyHint nor destructiveHint is declared — an agent can't tell whether this is safe to auto-approve.",
|
|
112
|
+
});
|
|
113
|
+
}
|
|
114
|
+
}
|
|
115
|
+
findings.push({
|
|
116
|
+
severity: "note",
|
|
117
|
+
rule: "no-version-signal",
|
|
118
|
+
operation: null,
|
|
119
|
+
message: "No changelog or version signal on this server — if any of this changes, nothing tells you unless it's being monitored.",
|
|
120
|
+
});
|
|
121
|
+
const counts = {
|
|
122
|
+
error: findings.filter((f) => f.severity === "error").length,
|
|
123
|
+
warning: findings.filter((f) => f.severity === "warning").length,
|
|
124
|
+
note: findings.filter((f) => f.severity === "note").length,
|
|
125
|
+
};
|
|
126
|
+
return {
|
|
127
|
+
toolCount: toolOps.length,
|
|
128
|
+
resourceCount,
|
|
129
|
+
resourceTemplateCount,
|
|
130
|
+
promptCount,
|
|
131
|
+
findings,
|
|
132
|
+
counts,
|
|
133
|
+
};
|
|
134
|
+
}
|
package/dist/baseline.js
ADDED
|
Binary file
|
package/dist/fetch.js
ADDED
|
@@ -0,0 +1,162 @@
|
|
|
1
|
+
import { readFile } from "node:fs/promises";
|
|
2
|
+
import { Client } from "@modelcontextprotocol/sdk/client/index.js";
|
|
3
|
+
import { StreamableHTTPClientTransport } from "@modelcontextprotocol/sdk/client/streamableHttp.js";
|
|
4
|
+
import { SSEClientTransport } from "@modelcontextprotocol/sdk/client/sse.js";
|
|
5
|
+
const CONNECT_TIMEOUT_MS = 15_000;
|
|
6
|
+
function resolveType(prop) {
|
|
7
|
+
if (prop.type)
|
|
8
|
+
return prop.type;
|
|
9
|
+
const variants = prop.anyOf ?? prop.oneOf;
|
|
10
|
+
if (variants && variants.length > 0) {
|
|
11
|
+
return [...new Set(variants.map(resolveType))].join(" | ");
|
|
12
|
+
}
|
|
13
|
+
return "unknown";
|
|
14
|
+
}
|
|
15
|
+
function paramsFromInputSchema(inputSchema) {
|
|
16
|
+
if (!inputSchema || typeof inputSchema !== "object")
|
|
17
|
+
return [];
|
|
18
|
+
const schema = inputSchema;
|
|
19
|
+
const properties = schema.properties ?? {};
|
|
20
|
+
const required = new Set(schema.required ?? []);
|
|
21
|
+
return Object.entries(properties).map(([name, prop]) => ({
|
|
22
|
+
name,
|
|
23
|
+
required: required.has(name),
|
|
24
|
+
type: resolveType(prop),
|
|
25
|
+
enumValues: prop.enum,
|
|
26
|
+
description: prop.description,
|
|
27
|
+
}));
|
|
28
|
+
}
|
|
29
|
+
function normalize(input) {
|
|
30
|
+
const toolOps = input.tools.map((tool) => ({
|
|
31
|
+
id: tool.name,
|
|
32
|
+
description: tool.description,
|
|
33
|
+
params: paramsFromInputSchema(tool.inputSchema),
|
|
34
|
+
annotations: tool.annotations,
|
|
35
|
+
}));
|
|
36
|
+
const resourceOps = (input.resources ?? []).map((r) => ({
|
|
37
|
+
id: `resource:${r.uri}`,
|
|
38
|
+
description: r.description,
|
|
39
|
+
params: [],
|
|
40
|
+
category: "resource",
|
|
41
|
+
mimeType: r.mimeType,
|
|
42
|
+
}));
|
|
43
|
+
const resourceTemplateOps = (input.resourceTemplates ?? []).map((t) => ({
|
|
44
|
+
id: `resource_template:${t.uriTemplate}`,
|
|
45
|
+
description: t.description,
|
|
46
|
+
params: [],
|
|
47
|
+
category: "resource_template",
|
|
48
|
+
mimeType: t.mimeType,
|
|
49
|
+
}));
|
|
50
|
+
const promptOps = (input.prompts ?? []).map((p) => ({
|
|
51
|
+
id: `prompt:${p.name}`,
|
|
52
|
+
description: p.description,
|
|
53
|
+
params: (p.arguments ?? []).map((arg) => ({
|
|
54
|
+
name: arg.name,
|
|
55
|
+
required: arg.required ?? false,
|
|
56
|
+
type: "string",
|
|
57
|
+
description: arg.description,
|
|
58
|
+
})),
|
|
59
|
+
category: "prompt",
|
|
60
|
+
}));
|
|
61
|
+
return {
|
|
62
|
+
operations: [
|
|
63
|
+
...toolOps,
|
|
64
|
+
...resourceOps,
|
|
65
|
+
...resourceTemplateOps,
|
|
66
|
+
...promptOps,
|
|
67
|
+
],
|
|
68
|
+
};
|
|
69
|
+
}
|
|
70
|
+
async function connectWithTimeout(client, transport) {
|
|
71
|
+
let timer;
|
|
72
|
+
const timeout = new Promise((_, reject) => {
|
|
73
|
+
timer = setTimeout(() => reject(new Error("Connection timed out")), CONNECT_TIMEOUT_MS);
|
|
74
|
+
});
|
|
75
|
+
try {
|
|
76
|
+
await Promise.race([client.connect(transport), timeout]);
|
|
77
|
+
}
|
|
78
|
+
catch (err) {
|
|
79
|
+
await client.close().catch(() => { });
|
|
80
|
+
throw err;
|
|
81
|
+
}
|
|
82
|
+
finally {
|
|
83
|
+
clearTimeout(timer);
|
|
84
|
+
}
|
|
85
|
+
}
|
|
86
|
+
// Try Streamable HTTP first, fall back to SSE — same order the MCP SDK docs recommend for
|
|
87
|
+
// the migration period. A fresh Client per attempt.
|
|
88
|
+
async function connect(url, headers) {
|
|
89
|
+
const requestInit = Object.keys(headers).length > 0 ? { headers } : undefined;
|
|
90
|
+
try {
|
|
91
|
+
const client = new Client({ name: "apitella-scan", version: "0.1.0" });
|
|
92
|
+
await connectWithTimeout(client, new StreamableHTTPClientTransport(new URL(url), { requestInit }));
|
|
93
|
+
return client;
|
|
94
|
+
}
|
|
95
|
+
catch {
|
|
96
|
+
const client = new Client({ name: "apitella-scan", version: "0.1.0" });
|
|
97
|
+
await connectWithTimeout(client, new SSEClientTransport(new URL(url), { requestInit }));
|
|
98
|
+
return client;
|
|
99
|
+
}
|
|
100
|
+
}
|
|
101
|
+
export async function fetchFromUrl(url, headers) {
|
|
102
|
+
const client = await connect(url, headers);
|
|
103
|
+
try {
|
|
104
|
+
const info = client.getServerVersion();
|
|
105
|
+
const capabilities = client.getServerCapabilities() ?? {};
|
|
106
|
+
const [tools, resources, resourceTemplates, prompts] = await Promise.all([
|
|
107
|
+
client.listTools().then((r) => r.tools),
|
|
108
|
+
capabilities.resources
|
|
109
|
+
? client.listResources().then((r) => r.resources)
|
|
110
|
+
: Promise.resolve([]),
|
|
111
|
+
capabilities.resources
|
|
112
|
+
? client.listResourceTemplates().then((r) => r.resourceTemplates)
|
|
113
|
+
: Promise.resolve([]),
|
|
114
|
+
capabilities.prompts
|
|
115
|
+
? client.listPrompts().then((r) => r.prompts)
|
|
116
|
+
: Promise.resolve([]),
|
|
117
|
+
]);
|
|
118
|
+
return {
|
|
119
|
+
schema: normalize({
|
|
120
|
+
tools: tools,
|
|
121
|
+
resources: resources,
|
|
122
|
+
resourceTemplates: resourceTemplates,
|
|
123
|
+
prompts: prompts,
|
|
124
|
+
}),
|
|
125
|
+
meta: {
|
|
126
|
+
serverName: info?.name ?? null,
|
|
127
|
+
serverVersion: info?.version ?? null,
|
|
128
|
+
},
|
|
129
|
+
};
|
|
130
|
+
}
|
|
131
|
+
finally {
|
|
132
|
+
await client.close().catch(() => { });
|
|
133
|
+
}
|
|
134
|
+
}
|
|
135
|
+
// Accepts a saved `tools/list` result, an object with tools/resources/prompts keys, or a
|
|
136
|
+
// bare array of tools — whatever a build step can dump without booting the server.
|
|
137
|
+
export async function fetchFromFile(path) {
|
|
138
|
+
let parsed;
|
|
139
|
+
try {
|
|
140
|
+
parsed = JSON.parse(await readFile(path, "utf8"));
|
|
141
|
+
}
|
|
142
|
+
catch (err) {
|
|
143
|
+
throw new Error(`Couldn't read ${path} as JSON: ${err.message}`);
|
|
144
|
+
}
|
|
145
|
+
if (Array.isArray(parsed)) {
|
|
146
|
+
return normalize({ tools: parsed });
|
|
147
|
+
}
|
|
148
|
+
if (parsed && typeof parsed === "object") {
|
|
149
|
+
const obj = parsed;
|
|
150
|
+
// Unwrap a JSON-RPC envelope ({ jsonrpc, id, result: { tools } }) if that's what got saved.
|
|
151
|
+
const source = (obj.result && typeof obj.result === "object" ? obj.result : obj);
|
|
152
|
+
if (Array.isArray(source.tools)) {
|
|
153
|
+
return normalize({
|
|
154
|
+
tools: source.tools,
|
|
155
|
+
resources: source.resources,
|
|
156
|
+
resourceTemplates: source.resourceTemplates,
|
|
157
|
+
prompts: source.prompts,
|
|
158
|
+
});
|
|
159
|
+
}
|
|
160
|
+
}
|
|
161
|
+
throw new Error(`${path} doesn't look like a tools/list dump — expected an array of tools or an object with a "tools" array.`);
|
|
162
|
+
}
|
package/dist/index.js
ADDED
|
@@ -0,0 +1,167 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
import { parseArgs } from "node:util";
|
|
3
|
+
import { analyze } from "./analyze.js";
|
|
4
|
+
import { diffSnapshots, makeSnapshot, readSnapshot, writeSnapshot, } from "./baseline.js";
|
|
5
|
+
import { fetchFromFile, fetchFromUrl } from "./fetch.js";
|
|
6
|
+
import { renderBaselineDiff, renderJson, renderText } from "./report.js";
|
|
7
|
+
const VERSION = "0.1.0";
|
|
8
|
+
const DEFAULT_BASELINE = ".apitella/baseline.json";
|
|
9
|
+
const HELP = `
|
|
10
|
+
apitella-scan — check an MCP server's tool surface for safety-annotation gaps
|
|
11
|
+
and prompt-injection risks.
|
|
12
|
+
|
|
13
|
+
USAGE
|
|
14
|
+
npx @apitella/scan <url> [options]
|
|
15
|
+
npx @apitella/scan --input tools.json [options]
|
|
16
|
+
|
|
17
|
+
ARGUMENTS
|
|
18
|
+
<url> MCP server URL (Streamable HTTP or SSE). Works against
|
|
19
|
+
localhost — point it at your dev server.
|
|
20
|
+
|
|
21
|
+
OPTIONS
|
|
22
|
+
-i, --input <file> Scan a saved tools/list dump instead of connecting.
|
|
23
|
+
Accepts a bare array of tools, a { "tools": [...] }
|
|
24
|
+
object, or a JSON-RPC response envelope.
|
|
25
|
+
-H, --header <h> Extra request header, "Name: value". Repeatable.
|
|
26
|
+
--token <t> Bearer token — shorthand for -H "Authorization: Bearer <t>".
|
|
27
|
+
--baseline [file] Compare the scan to a committed snapshot and report what
|
|
28
|
+
changed (default file: ${DEFAULT_BASELINE}). Exits non-zero
|
|
29
|
+
only on a regression — a removed tool, a tool that lost its
|
|
30
|
+
safety hint, or a new error/warning.
|
|
31
|
+
--update-baseline Write the current surface to the baseline file and exit 0.
|
|
32
|
+
Run this to accept the current state as the new reference.
|
|
33
|
+
--json Emit JSON instead of the formatted report.
|
|
34
|
+
--fail-on <lvl> Without --baseline: exit non-zero when findings reach this
|
|
35
|
+
level, "error" or "warning" (default: warning).
|
|
36
|
+
--no-fail Always exit 0 unless the scan itself failed.
|
|
37
|
+
-h, --help Show this help.
|
|
38
|
+
-v, --version Show version.
|
|
39
|
+
|
|
40
|
+
EXIT CODES
|
|
41
|
+
0 clean / below threshold / no regression / --no-fail
|
|
42
|
+
1 findings at or above --fail-on, or a regression vs. --baseline
|
|
43
|
+
2 usage error, or couldn't reach / parse the target
|
|
44
|
+
|
|
45
|
+
Part of Apitella — https://apitella.com/scan
|
|
46
|
+
`;
|
|
47
|
+
class UsageError extends Error {
|
|
48
|
+
}
|
|
49
|
+
function parseHeaders(raw, token) {
|
|
50
|
+
const headers = {};
|
|
51
|
+
for (const entry of raw ?? []) {
|
|
52
|
+
const idx = entry.indexOf(":");
|
|
53
|
+
if (idx === -1) {
|
|
54
|
+
throw new UsageError(`Bad --header "${entry}" — expected "Name: value".`);
|
|
55
|
+
}
|
|
56
|
+
headers[entry.slice(0, idx).trim()] = entry.slice(idx + 1).trim();
|
|
57
|
+
}
|
|
58
|
+
if (token)
|
|
59
|
+
headers["Authorization"] = `Bearer ${token}`;
|
|
60
|
+
return headers;
|
|
61
|
+
}
|
|
62
|
+
async function main() {
|
|
63
|
+
let parsed;
|
|
64
|
+
try {
|
|
65
|
+
parsed = parseArgs({
|
|
66
|
+
allowPositionals: true,
|
|
67
|
+
options: {
|
|
68
|
+
input: { type: "string", short: "i" },
|
|
69
|
+
header: { type: "string", short: "H", multiple: true },
|
|
70
|
+
token: { type: "string" },
|
|
71
|
+
baseline: { type: "string" },
|
|
72
|
+
"update-baseline": { type: "boolean", default: false },
|
|
73
|
+
json: { type: "boolean", default: false },
|
|
74
|
+
"fail-on": { type: "string", default: "warning" },
|
|
75
|
+
"no-fail": { type: "boolean", default: false },
|
|
76
|
+
help: { type: "boolean", short: "h", default: false },
|
|
77
|
+
version: { type: "boolean", short: "v", default: false },
|
|
78
|
+
},
|
|
79
|
+
});
|
|
80
|
+
}
|
|
81
|
+
catch (err) {
|
|
82
|
+
throw new UsageError(err.message);
|
|
83
|
+
}
|
|
84
|
+
const { values, positionals } = parsed;
|
|
85
|
+
if (values.help) {
|
|
86
|
+
process.stdout.write(HELP);
|
|
87
|
+
return 0;
|
|
88
|
+
}
|
|
89
|
+
if (values.version) {
|
|
90
|
+
process.stdout.write(`${VERSION}\n`);
|
|
91
|
+
return 0;
|
|
92
|
+
}
|
|
93
|
+
const failOn = values["fail-on"];
|
|
94
|
+
if (failOn !== "error" && failOn !== "warning") {
|
|
95
|
+
throw new UsageError(`--fail-on must be "error" or "warning", got "${failOn}".`);
|
|
96
|
+
}
|
|
97
|
+
const url = positionals[0];
|
|
98
|
+
if (!url && !values.input) {
|
|
99
|
+
throw new UsageError("Pass an MCP server URL or --input <file>. See --help.");
|
|
100
|
+
}
|
|
101
|
+
if (url && values.input) {
|
|
102
|
+
throw new UsageError("Pass a URL or --input, not both.");
|
|
103
|
+
}
|
|
104
|
+
// `--baseline` with no value falls through parseArgs as undefined; treat its presence via
|
|
105
|
+
// the raw argv so `--baseline` alone means "use the default path".
|
|
106
|
+
const baselineFlagPresent = process.argv.includes("--baseline");
|
|
107
|
+
const baselinePath = values.baseline ??
|
|
108
|
+
(baselineFlagPresent || values["update-baseline"]
|
|
109
|
+
? DEFAULT_BASELINE
|
|
110
|
+
: null);
|
|
111
|
+
const target = values.input ? values.input : url;
|
|
112
|
+
let meta = null;
|
|
113
|
+
let schema;
|
|
114
|
+
if (values.input) {
|
|
115
|
+
schema = await fetchFromFile(values.input);
|
|
116
|
+
}
|
|
117
|
+
else {
|
|
118
|
+
try {
|
|
119
|
+
const res = await fetchFromUrl(url, parseHeaders(values.header, values.token));
|
|
120
|
+
meta = res.meta;
|
|
121
|
+
schema = res.schema;
|
|
122
|
+
}
|
|
123
|
+
catch (err) {
|
|
124
|
+
throw new UsageError(`Couldn't scan ${url}: ${err.message}`);
|
|
125
|
+
}
|
|
126
|
+
}
|
|
127
|
+
const report = analyze(schema, values.input ? null : url);
|
|
128
|
+
if (values["update-baseline"]) {
|
|
129
|
+
const path = baselinePath ?? DEFAULT_BASELINE;
|
|
130
|
+
await writeSnapshot(path, makeSnapshot(schema, target, report.findings));
|
|
131
|
+
process.stdout.write(`Baseline written to ${path} — ${report.toolCount} tool${report.toolCount === 1 ? "" : "s"}, ${report.findings.length} finding${report.findings.length === 1 ? "" : "s"} recorded.\n`);
|
|
132
|
+
return 0;
|
|
133
|
+
}
|
|
134
|
+
const diff = baselinePath
|
|
135
|
+
? diffSnapshots(await readSnapshot(baselinePath), {
|
|
136
|
+
schema,
|
|
137
|
+
findings: report.findings,
|
|
138
|
+
})
|
|
139
|
+
: null;
|
|
140
|
+
if (values.json) {
|
|
141
|
+
process.stdout.write(`${renderJson(report, target, meta, diff)}\n`);
|
|
142
|
+
}
|
|
143
|
+
else {
|
|
144
|
+
process.stdout.write(renderText(report, target, meta));
|
|
145
|
+
if (diff && baselinePath) {
|
|
146
|
+
process.stdout.write(renderBaselineDiff(diff, baselinePath));
|
|
147
|
+
}
|
|
148
|
+
}
|
|
149
|
+
if (values["no-fail"])
|
|
150
|
+
return 0;
|
|
151
|
+
if (diff)
|
|
152
|
+
return diff.regressed ? 1 : 0;
|
|
153
|
+
const threshold = failOn;
|
|
154
|
+
const hit = report.counts.error > 0 ||
|
|
155
|
+
(threshold === "warning" && report.counts.warning > 0);
|
|
156
|
+
return hit ? 1 : 0;
|
|
157
|
+
}
|
|
158
|
+
main()
|
|
159
|
+
.then((code) => process.exit(code))
|
|
160
|
+
.catch((err) => {
|
|
161
|
+
if (err instanceof UsageError) {
|
|
162
|
+
process.stderr.write(`${err.message}\n`);
|
|
163
|
+
process.exit(2);
|
|
164
|
+
}
|
|
165
|
+
process.stderr.write(`Unexpected error: ${err.stack ?? err}\n`);
|
|
166
|
+
process.exit(2);
|
|
167
|
+
});
|
package/dist/report.js
ADDED
|
@@ -0,0 +1,126 @@
|
|
|
1
|
+
// No color dependency — a tiny ANSI helper that no-ops when stdout isn't a TTY or NO_COLOR
|
|
2
|
+
// is set, so CI logs stay clean.
|
|
3
|
+
const useColor = process.stdout.isTTY && !process.env.NO_COLOR;
|
|
4
|
+
const paint = (code, s) => useColor ? `\x1b[${code}m${s}\x1b[0m` : s;
|
|
5
|
+
const c = {
|
|
6
|
+
bold: (s) => paint("1", s),
|
|
7
|
+
dim: (s) => paint("2", s),
|
|
8
|
+
red: (s) => paint("31", s),
|
|
9
|
+
yellow: (s) => paint("33", s),
|
|
10
|
+
green: (s) => paint("32", s),
|
|
11
|
+
cyan: (s) => paint("36", s),
|
|
12
|
+
};
|
|
13
|
+
const MARK = {
|
|
14
|
+
error: c.red("✖"),
|
|
15
|
+
warning: c.yellow("▲"),
|
|
16
|
+
note: c.dim("·"),
|
|
17
|
+
};
|
|
18
|
+
const LABEL = {
|
|
19
|
+
error: c.red,
|
|
20
|
+
warning: c.yellow,
|
|
21
|
+
note: c.dim,
|
|
22
|
+
};
|
|
23
|
+
function countsLine(report) {
|
|
24
|
+
const parts = [
|
|
25
|
+
`${report.toolCount} tool${report.toolCount === 1 ? "" : "s"}`,
|
|
26
|
+
];
|
|
27
|
+
if (report.resourceCount) {
|
|
28
|
+
parts.push(`${report.resourceCount} resource${report.resourceCount === 1 ? "" : "s"}`);
|
|
29
|
+
}
|
|
30
|
+
if (report.resourceTemplateCount) {
|
|
31
|
+
parts.push(`${report.resourceTemplateCount} resource templates`);
|
|
32
|
+
}
|
|
33
|
+
if (report.promptCount) {
|
|
34
|
+
parts.push(`${report.promptCount} prompt${report.promptCount === 1 ? "" : "s"}`);
|
|
35
|
+
}
|
|
36
|
+
return parts.join(" · ");
|
|
37
|
+
}
|
|
38
|
+
export function renderText(report, target, meta) {
|
|
39
|
+
const lines = [];
|
|
40
|
+
const server = meta?.serverName
|
|
41
|
+
? `${meta.serverName}${meta.serverVersion ? ` v${meta.serverVersion}` : ""}`
|
|
42
|
+
: target;
|
|
43
|
+
lines.push("");
|
|
44
|
+
lines.push(`${c.bold("Apitella scan")} ${c.dim(server)}`);
|
|
45
|
+
lines.push(c.dim(countsLine(report)));
|
|
46
|
+
lines.push("");
|
|
47
|
+
const ordered = [
|
|
48
|
+
...report.findings.filter((f) => f.severity === "error"),
|
|
49
|
+
...report.findings.filter((f) => f.severity === "warning"),
|
|
50
|
+
...report.findings.filter((f) => f.severity === "note"),
|
|
51
|
+
];
|
|
52
|
+
for (const f of ordered) {
|
|
53
|
+
const scope = f.operation ? c.cyan(f.operation) : c.dim("server");
|
|
54
|
+
lines.push(` ${MARK[f.severity]} ${scope} ${LABEL[f.severity](`[${f.rule}]`)}`);
|
|
55
|
+
lines.push(` ${f.message}`);
|
|
56
|
+
}
|
|
57
|
+
lines.push("");
|
|
58
|
+
const { error, warning } = report.counts;
|
|
59
|
+
if (error === 0 && warning === 0) {
|
|
60
|
+
lines.push(c.green(" ✔ Nothing flagged — every tool declares a safety hint and nothing contradicts itself."));
|
|
61
|
+
}
|
|
62
|
+
else {
|
|
63
|
+
const bits = [];
|
|
64
|
+
if (error)
|
|
65
|
+
bits.push(c.red(`${error} error${error === 1 ? "" : "s"}`));
|
|
66
|
+
if (warning)
|
|
67
|
+
bits.push(c.yellow(`${warning} warning${warning === 1 ? "" : "s"}`));
|
|
68
|
+
lines.push(` ${bits.join(", ")} — see above.`);
|
|
69
|
+
lines.push(c.dim(" These are point-in-time. To catch changes like these after you ship, monitor the server free at apitella.com."));
|
|
70
|
+
}
|
|
71
|
+
lines.push("");
|
|
72
|
+
return lines.join("\n");
|
|
73
|
+
}
|
|
74
|
+
export function renderBaselineDiff(diff, baselinePath) {
|
|
75
|
+
const lines = [];
|
|
76
|
+
lines.push(`${c.bold("vs. baseline")} ${c.dim(baselinePath)}`);
|
|
77
|
+
const nothing = diff.addedOps.length === 0 &&
|
|
78
|
+
diff.removedOps.length === 0 &&
|
|
79
|
+
diff.lostSafetyHint.length === 0 &&
|
|
80
|
+
diff.newFindings.length === 0 &&
|
|
81
|
+
diff.resolvedFindings.length === 0;
|
|
82
|
+
if (nothing) {
|
|
83
|
+
lines.push(c.green(" ✔ No change to the surface since the baseline."));
|
|
84
|
+
lines.push("");
|
|
85
|
+
return lines.join("\n");
|
|
86
|
+
}
|
|
87
|
+
for (const id of diff.removedOps) {
|
|
88
|
+
lines.push(` ${c.red("−")} removed ${c.cyan(id)}`);
|
|
89
|
+
}
|
|
90
|
+
for (const id of diff.addedOps) {
|
|
91
|
+
lines.push(` ${c.green("+")} added ${c.cyan(id)}`);
|
|
92
|
+
}
|
|
93
|
+
for (const id of diff.lostSafetyHint) {
|
|
94
|
+
lines.push(` ${c.red("!")} ${c.cyan(id)} lost its safety hint — it declared readOnlyHint/destructiveHint in the baseline and now declares neither.`);
|
|
95
|
+
}
|
|
96
|
+
for (const f of diff.newFindings) {
|
|
97
|
+
const scope = f.operation ? c.cyan(f.operation) : c.dim("server");
|
|
98
|
+
lines.push(` ${MARK[f.severity]} new ${scope} ${LABEL[f.severity](`[${f.rule}]`)}`);
|
|
99
|
+
lines.push(` ${f.message}`);
|
|
100
|
+
}
|
|
101
|
+
for (const f of diff.resolvedFindings) {
|
|
102
|
+
const scope = f.operation ? c.cyan(f.operation) : c.dim("server");
|
|
103
|
+
lines.push(` ${c.green("✓")} resolved ${scope} ${c.dim(`[${f.rule}]`)}`);
|
|
104
|
+
}
|
|
105
|
+
lines.push("");
|
|
106
|
+
lines.push(diff.regressed
|
|
107
|
+
? c.red(" This change makes the MCP surface worse than the baseline.")
|
|
108
|
+
: c.dim(" No regressions — update the baseline to accept these changes (--update-baseline)."));
|
|
109
|
+
lines.push("");
|
|
110
|
+
return lines.join("\n");
|
|
111
|
+
}
|
|
112
|
+
export function renderJson(report, target, meta, diff) {
|
|
113
|
+
return JSON.stringify({
|
|
114
|
+
target,
|
|
115
|
+
server: meta ?? null,
|
|
116
|
+
summary: {
|
|
117
|
+
toolCount: report.toolCount,
|
|
118
|
+
resourceCount: report.resourceCount,
|
|
119
|
+
resourceTemplateCount: report.resourceTemplateCount,
|
|
120
|
+
promptCount: report.promptCount,
|
|
121
|
+
...report.counts,
|
|
122
|
+
},
|
|
123
|
+
findings: report.findings,
|
|
124
|
+
baseline: diff ?? null,
|
|
125
|
+
}, null, 2);
|
|
126
|
+
}
|
package/package.json
ADDED
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "@apitella/scan",
|
|
3
|
+
"version": "0.1.0",
|
|
4
|
+
"description": "Check an MCP server's tool surface for safety-annotation gaps and prompt-injection risks — at build time, in CI, or against a live URL.",
|
|
5
|
+
"type": "module",
|
|
6
|
+
"bin": {
|
|
7
|
+
"apitella-scan": "dist/index.js"
|
|
8
|
+
},
|
|
9
|
+
"files": [
|
|
10
|
+
"dist",
|
|
11
|
+
"README.md"
|
|
12
|
+
],
|
|
13
|
+
"engines": {
|
|
14
|
+
"node": ">=20"
|
|
15
|
+
},
|
|
16
|
+
"scripts": {
|
|
17
|
+
"build": "tsc -p tsconfig.json",
|
|
18
|
+
"dev": "tsx src/index.ts",
|
|
19
|
+
"test": "node --test --import tsx test/analyze.test.ts test/baseline.test.ts",
|
|
20
|
+
"prepublishOnly": "npm run build"
|
|
21
|
+
},
|
|
22
|
+
"keywords": [
|
|
23
|
+
"mcp",
|
|
24
|
+
"model-context-protocol",
|
|
25
|
+
"security",
|
|
26
|
+
"linter",
|
|
27
|
+
"ci",
|
|
28
|
+
"agents",
|
|
29
|
+
"llm"
|
|
30
|
+
],
|
|
31
|
+
"license": "MIT",
|
|
32
|
+
"homepage": "https://apitella.com/scan",
|
|
33
|
+
"repository": {
|
|
34
|
+
"type": "git",
|
|
35
|
+
"url": "git+https://github.com/dodogeny/apitella-saas.git",
|
|
36
|
+
"directory": "cli"
|
|
37
|
+
},
|
|
38
|
+
"bugs": {
|
|
39
|
+
"url": "https://github.com/dodogeny/apitella-saas/issues"
|
|
40
|
+
},
|
|
41
|
+
"publishConfig": {
|
|
42
|
+
"access": "public"
|
|
43
|
+
},
|
|
44
|
+
"dependencies": {
|
|
45
|
+
"@modelcontextprotocol/sdk": "^1.11.0"
|
|
46
|
+
},
|
|
47
|
+
"devDependencies": {
|
|
48
|
+
"@types/node": "^22.10.2",
|
|
49
|
+
"tsx": "^4.19.2",
|
|
50
|
+
"typescript": "^5.7.2"
|
|
51
|
+
}
|
|
52
|
+
}
|