efaimo 0.4.0 → 0.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -4,6 +4,124 @@ All notable changes to efaimo are documented here. Format follows
4
4
  [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this project uses
5
5
  [Semantic Versioning](https://semver.org/).
6
6
 
7
+ ## [0.6.0] - 2026-11-02
8
+
9
+ ### Added
10
+
11
+ - **S108: a reference file nothing points at**. A file under
12
+ `references/` that no markdown link or backticked path resolves to, whose
13
+ path never appears in `SKILL.md`, whose directory is never named there, and
14
+ that no reference file which IS reached mentions either. It ships in the
15
+ package, installs byte-perfect, and an agent has no route to it.
16
+
17
+ Found in this project's own skills on 2026-09-04: `read-back` had shipped a
18
+ 4.6 KB failure gallery that grew for a month with nothing linking to it, and
19
+ `unreleased-guard` had shipped `the-gap.md` the same way. The installer
20
+ tests, the house style guard and an A(100) were all green, because none of
21
+ them is about whether a shipped file is reachable. The workspace that
22
+ publishes those skills got a checker the same day; this is the rule for
23
+ everyone else's `references/`.
24
+
25
+ Reported and **not scored**, the way S105 is. Measured before it was
26
+ written, on the public corpus at the commits `research/skills-index/manifest.json`
27
+ pins: 2 of 36 skills carry a `references/` directory at all, 4 files between
28
+ them, no orphan, so there is no population to calibrate a penalty on and no
29
+ published grade moves. Scoring it is a separate decision. It fires under
30
+ `--strict`, which is where our own seven repositories run it.
31
+
32
+ Transitive on purpose: a reader who opens `references/api.md` can follow a
33
+ link inside it, so a file reached that way is not an orphan. Scoped to
34
+ `references/` on purpose: `scripts/` is executed rather than read and
35
+ `assets/` is consumed by scripts, so an unmentioned file there is a
36
+ different claim.
37
+
38
+ ### Changed
39
+
40
+ - **`rulesVersion` is `3`**. No grade moves with S108, but a
41
+ report from this ruleset carries a rule id that ruleset `2` could not have
42
+ produced, and `check --diff` across the two would attribute that to the
43
+ subject rather than to the ruler.
44
+
45
+ ### Security
46
+
47
+ - **Seven advisories against the MCP SDK and zod, fixed by upgrading**.
48
+ `@modelcontextprotocol/client` and
49
+ `@modelcontextprotocol/server` move from 2.0.0 to 2.3.1, the first release
50
+ that carries every patch, and still pinned exactly. zod's floor moves from
51
+ `^4.5.4` to `^4.6.6`. `pnpm audit --prod` went from 7 findings (5 high, 2
52
+ moderate) to none.
53
+
54
+ What they reach in this tool, read from the call sites: `efaimo mcp` serves
55
+ over stdio only, so the four server advisories about the HTTP transport
56
+ (GHSA-f897-62hf-6gxq body size, GHSA-jm63-cq47-j5jp shared transport,
57
+ GHSA-46jr-4wx8-v453 memory, GHSA-rvq5-wwqv-78pq bearer auth) describe code it
58
+ never runs. The OAuth advisory on the client (GHSA-6qxp-vccf-f47h) needs an
59
+ auth provider, and efaimo passes none. The one with a plausible path is
60
+ GHSA-6prh-2h8m-c8cw, headers re-sent across an origin redirect: `--header`
61
+ values go to Streamable HTTP and legacy SSE targets, and `requestInit` sets
62
+ `redirect: "error"`, but nobody has shown that every request the SSE
63
+ fallback makes inherits it. zod (GHSA-mrhx-c7xc-m3w5) has no known path here.
64
+
65
+ Who gets it when: zod's range floats, so a fresh `npx efaimo@0.5.0` already
66
+ resolves a patched zod. The SDK pins are exact, so that half reaches users
67
+ with this release and not before.
68
+
69
+ Measured after the upgrade with the same command as 0.5.0's entry: `check
70
+ --mcp "node dist/cli.js mcp"` still reports `grade A (100)` and `clean,
71
+ nothing to migrate`.
72
+
73
+ - **Ten dev-only advisories, fixed by moving vitest from 4.1.10 to 4.1.12**,
74
+ which re-resolves vite from 8.2.0 to 8.3.5 and source-map-js
75
+ to 1.2.2. None of them ships in the package; `pnpm audit` over every
76
+ dependency now finds nothing.
77
+
78
+ ## [0.5.0] - 2026-09-04
79
+
80
+ ### Changed
81
+
82
+ - **`efaimo mcp` is built on the 2.x SDK line and now satisfies the
83
+ specification this tool audits other servers against.** The 1.x line speaks
84
+ 2025-11-25; it cannot answer `server/discover` and cannot carry the SEP-2549
85
+ cache fields, so the one MCP server this project ships failed two MUST-level
86
+ items of the 2026-07-28 revision, and `check --mcp` said so about it on every
87
+ run. The skill this project publishes about that migration opens with "upgrade
88
+ the SDK first". This is that.
89
+
90
+ Measured against the built server, before and after, with the same command:
91
+
92
+ ```
93
+ before 2026-07-28 readiness 3 items to migrate E106, E118, E107
94
+ after 2026-07-28 readiness clean, nothing to migrate
95
+ ```
96
+
97
+ Read off the wire rather than from our own tool as well: a modern opening now
98
+ answers `server/discover` with `supportedVersions`, `capabilities`,
99
+ `resultType: "complete"`, `ttlMs` and `cacheScope`, and `tools/list` carries
100
+ the same three fields. A 2025-era client that opens with `initialize` is still
101
+ served, unchanged, because `serveStdio` pins the era per connection rather
102
+ than making the server pick one.
103
+
104
+ - **The client half moved with it**, from `@modelcontextprotocol/sdk` to
105
+ `@modelcontextprotocol/client`. Doing only the server half would have left
106
+ both generations installed, and E101 stops firing the moment any 2.x package
107
+ is present, so the half-migration would have silenced the rule that was
108
+ telling the truth.
109
+
110
+ ### Fixed
111
+
112
+ - **Two moderate advisories, by subtraction.** `qs` arrived through
113
+ `body-parser` through `express` through the 1.x SDK, which pulls a whole HTTP
114
+ and OAuth stack that a stdio server never speaks. `pnpm audit --prod` now
115
+ reports no known vulnerabilities, and production dependencies went from 30 to
116
+ 13.
117
+
118
+ ### Notes
119
+
120
+ - No rule changed, so no grade moved. `rulesVersion` is untouched and the
121
+ published skills index does not need regenerating.
122
+ - `check --skill`, `weigh`, `find` and `test` are byte-identical in behaviour;
123
+ the only thing that moved is what `efaimo mcp` puts on the wire.
124
+
7
125
  ## [0.4.0] - 2026-09-03
8
126
 
9
127
  ### Fixed
@@ -408,6 +526,8 @@ First release.
408
526
  Documented rule set (`docs/RULES.md`), token methodology (`docs/METHODOLOGY.md`),
409
527
  and integration guide (`docs/INTEGRATIONS.md`).
410
528
 
529
+ [0.6.0]: https://github.com/efaimo-ai/efaimo/releases/tag/v0.6.0
530
+ [0.5.0]: https://github.com/efaimo-ai/efaimo/releases/tag/v0.5.0
411
531
  [0.4.0]: https://github.com/efaimo-ai/efaimo/releases/tag/v0.4.0
412
532
  [0.3.0]: https://github.com/efaimo-ai/efaimo/releases/tag/v0.3.0
413
533
  [0.2.0]: https://github.com/efaimo-ai/efaimo/releases/tag/v0.2.0
package/README.md CHANGED
@@ -1,5 +1,5 @@
1
1
  <p align="center">
2
- <img src="assets/banner.png" alt="efaimo. The audit CLI for Agent Skills and MCP servers." width="820">
2
+ <img src="https://raw.githubusercontent.com/efaimo-ai/efaimo/main/assets/banner.png" alt="efaimo. The audit CLI for Agent Skills and MCP servers." width="820">
3
3
  </p>
4
4
 
5
5
  <p align="center">
@@ -87,7 +87,7 @@ csv-cleanup skill that a capable model does not need, and a skill that encodes a
87
87
  convention the model cannot guess.
88
88
 
89
89
  <p align="center">
90
- <img src="assets/test-ab.png" alt="efaimo test A/B results: csv-cleanup measures +0 points (no measurable effect) while contoso-crm-import measures +100 points (helps); both skills lint clean at grade A" width="880">
90
+ <img src="https://raw.githubusercontent.com/efaimo-ai/efaimo/main/assets/test-ab.png" alt="efaimo test A/B results: csv-cleanup measures +0 points (no measurable effect) while contoso-crm-import measures +100 points (helps); both skills lint clean at grade A" width="880">
91
91
  </p>
92
92
 
93
93
  <details>
@@ -137,7 +137,7 @@ npx efaimo check --mcp "npx -y my-server" # quality grade + 2026-07-28 migrat
137
137
  per tool, with an optional `--anthropic` exact Claude count.
138
138
 
139
139
  <p align="center">
140
- <img src="assets/demo.png" alt="efaimo weigh example output" width="720">
140
+ <img src="https://raw.githubusercontent.com/efaimo-ai/efaimo/main/assets/demo.png" alt="efaimo weigh example output" width="720">
141
141
  </p>
142
142
 
143
143
  `check --mcp` connects live (it speaks both the legacy handshake and the new
@@ -155,7 +155,7 @@ surface that is an A. Pass `--strict-readiness` when you want CI to fail on
155
155
  the diff. Full list: [docs/RULES.md](./docs/RULES.md).
156
156
 
157
157
  <p align="center">
158
- <img src="assets/check.png" alt="efaimo check example output: quality grade A (95) plus a four-item 2026-07-28 migration diff for the reference server" width="800">
158
+ <img src="https://raw.githubusercontent.com/efaimo-ai/efaimo/main/assets/check.png" alt="efaimo check example output: quality grade A (95) plus a four-item 2026-07-28 migration diff for the reference server" width="800">
159
159
  </p>
160
160
 
161
161
  *(That is the official reference server: solid quality, four things left to
@@ -253,7 +253,7 @@ your host actually serializes it, and grades the quality, before any of it reach
253
253
  your context window.
254
254
 
255
255
  <p align="center">
256
- <img src="assets/architecture.png" alt="efaimo weighs, checks, and tests what your agent loads" width="880">
256
+ <img src="https://raw.githubusercontent.com/efaimo-ai/efaimo/main/assets/architecture.png" alt="efaimo weighs, checks, and tests what your agent loads" width="880">
257
257
  </p>
258
258
 
259
259
  ## How it compares
@@ -349,7 +349,7 @@ npx efaimo weigh "npx -y my-server" --diff base.json --allow-increase 10
349
349
  `--badge badge.svg` writes an SVG plus a shields.io endpoint JSON for your
350
350
  README. These two are real efaimo output for the reference server:
351
351
 
352
- <img src="assets/badge-weigh.svg" alt="context cost: 1120 tok (o200k)"> <img src="assets/badge-grade.svg" alt="efaimo: A (95)">
352
+ <img src="https://raw.githubusercontent.com/efaimo-ai/efaimo/main/assets/badge-weigh.svg" alt="context cost: 1120 tok (o200k)"> <img src="https://raw.githubusercontent.com/efaimo-ai/efaimo/main/assets/badge-grade.svg" alt="efaimo: A (95)">
353
353
 
354
354
  More recipes (pre-commit, GitLab, editor audit, programmatic use):
355
355
  [docs/INTEGRATIONS.md](./docs/INTEGRATIONS.md).
@@ -358,7 +358,7 @@ More recipes (pre-commit, GitLab, editor audit, programmatic use):
358
358
 
359
359
  | family | covers |
360
360
  |---|---|
361
- | **S101 to S106** | skills: frontmatter and trigger quality, trigger collisions, context budget, reference integrity, injection hygiene |
361
+ | **S101 to S108** | skills: frontmatter and trigger quality, trigger collisions, context budget, reference integrity, injection hygiene, a filename one capitalisation away from `SKILL.md`, and a reference file nothing points at |
362
362
  | **E101 to E118** | MCP 2026-07-28 readiness: deprecated primitives, statelessness, `server/discover`, `resultType`, cache fields, transport |
363
363
  | **E121 to E128, E130** | MCP quality: description quality, annotations, schema hygiene, tool-count and token-cost budgets |
364
364
  | **E141 to E146** | MCP findability: exclusive vocabulary, whether a search returns the tool, indistinguishable pairs, names that carry no domain word |
package/SKILL.md CHANGED
@@ -3,8 +3,8 @@ name: efaimo
3
3
  description: Audit MCP servers and Agent Skills for quality and context-window cost. Use when the user wants to check an MCP server's token cost, lint a skill, verify 2026-07-28 MCP spec readiness, find out whether a search can single out a server's tools under deferred tool loading, or add an efaimo badge or CI gate.
4
4
  license: Apache-2.0
5
5
  metadata:
6
- version: "0.4.0"
7
- homepage: "https://efaimo.ai"
6
+ version: "0.6.0"
7
+ homepage: "https://efaimo.com"
8
8
  ---
9
9
 
10
10
  # efaimo
@@ -74,3 +74,14 @@ methodology doc for the full method.
74
74
  efaimo is a linter and cost profiler, not a security scanner: its injection checks
75
75
  are surface heuristics. For supply-chain security use a dedicated scanner. Full
76
76
  rule reference: https://github.com/efaimo-ai/efaimo/blob/main/docs/RULES.md
77
+
78
+ <!-- generated:siblings -->
79
+
80
+ ## Siblings
81
+
82
+ Every skill in this set is about a report that was true about the wrong thing. The set: https://efaimo.com/skills
83
+
84
+ - `denominator` - the counts efaimo prints are only worth as much as the sets they were taken over.
85
+ - `mcp-stateless-migration` - when `check --mcp` prints a readiness list and you want to work it.
86
+
87
+ <!-- /generated:siblings -->
@@ -1,8 +1,8 @@
1
1
  import crossSpawn from "cross-spawn";
2
- import { Client } from "@modelcontextprotocol/sdk/client/index.js";
3
- import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js";
4
- import { StreamableHTTPClientTransport } from "@modelcontextprotocol/sdk/client/streamableHttp.js";
5
- import { SSEClientTransport } from "@modelcontextprotocol/sdk/client/sse.js";
2
+ import { Client } from "@modelcontextprotocol/client";
3
+ import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
4
+ import { StreamableHTTPClientTransport } from "@modelcontextprotocol/client";
5
+ import { SSEClientTransport } from "@modelcontextprotocol/client";
6
6
  import { introspectStateless } from "./stateless.js";
7
7
  import { minimalChildEnv } from "../util/childEnv.js";
8
8
  import { truncate, withTimeout } from "../util/misc.js";
@@ -1,3 +1,3 @@
1
- import { Server } from "@modelcontextprotocol/sdk/server/index.js";
2
- export declare function buildMcpServer(): Server;
1
+ import { McpServer } from "@modelcontextprotocol/server";
2
+ export declare function buildMcpServer(): McpServer;
3
3
  export declare function runMcpServer(): Promise<void>;
@@ -1,88 +1,94 @@
1
- import { Server } from "@modelcontextprotocol/sdk/server/index.js";
2
- import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
3
- import { CallToolRequestSchema, ListToolsRequestSchema } from "@modelcontextprotocol/sdk/types.js";
1
+ import { McpServer } from "@modelcontextprotocol/server";
2
+ import { serveStdio } from "@modelcontextprotocol/server/stdio";
3
+ import { z } from "zod";
4
4
  import { checkSkillSet } from "../check/check.js";
5
5
  import { weighSkills } from "../weigh/weigh.js";
6
6
  import { findSkills } from "../skills/parse.js";
7
7
  import { renderCheckMarkdown, renderSkillSetMarkdown, renderWeighMarkdown } from "../reporters/markdown.js";
8
8
  import { VERSION } from "../version.js";
9
+ /**
10
+ * `efaimo mcp`: a small, read-only MCP server that exposes the skill checks to an
11
+ * agent, so it can lint or weigh a skill mid-session before committing it to
12
+ * context. Deliberately narrow: the tools only read files. They spawn no process
13
+ * and open no socket (unlike `check --mcp`, which connects to a live server), and
14
+ * `test` is not exposed at all because it spends tokens. That keeps the surface
15
+ * safe for an agent to call unattended.
16
+ *
17
+ * Built on the 2.x SDK line since 0.5.0. The 1.x line speaks 2025-11-25 and
18
+ * cannot answer `server/discover` or carry the SEP-2549 cache fields, so this
19
+ * server failed two MUST-level items of the specification the rest of this tool
20
+ * audits other servers against, and the skill this project publishes about that
21
+ * migration opens with "upgrade the SDK first". It was the only server we ship
22
+ * and the only one we had not migrated.
23
+ *
24
+ * `serveStdio` owns the era decision rather than this file: the opening exchange
25
+ * selects it, one instance is pinned for the connection, and a 2025-era client
26
+ * that opens with `initialize` is still served. Registering the tools once, on a
27
+ * factory that serves both eras, is the whole reason to use it instead of
28
+ * wiring a transport by hand.
29
+ */
9
30
  // Both tools only read local files, so the same hints apply: safe to call without
10
31
  // a confirmation, safe to repeat, and not reaching out to any external service.
11
32
  const READ_ONLY = { readOnlyHint: true, idempotentHint: true, openWorldHint: false };
12
- function requireString(args, key) {
13
- const v = args[key];
14
- if (typeof v !== "string" || v.trim() === "") {
15
- throw new Error(`"${key}" is required and must be a non-empty string`);
16
- }
17
- return v;
18
- }
19
- const pathSchema = (what) => ({
20
- type: "object",
21
- properties: { path: { type: "string", description: what } },
22
- required: ["path"],
23
- additionalProperties: false,
33
+ // One shared shape. `.min(1)` is the schema saying what `requireString` used to
34
+ // say at runtime: an empty path is not a path. Under the 2.x registration the
35
+ // SDK validates against this before the handler runs, so the check moved from
36
+ // our code into the contract, which is where a client can also see it.
37
+ const pathInput = (what) => z.object({ path: z.string().min(1).describe(what) });
38
+ const CHECK_DESCRIPTION = "Lint an Agent Skill, or a folder of skills, for spec compliance, trigger quality, " +
39
+ "context-window cost, reference integrity, and injection hygiene. Returns a grade from " +
40
+ "A to F and the findings. Read-only: reads files only.";
41
+ const WEIGH_DESCRIPTION = "Measure the context-window token cost of an Agent Skill: the metadata loaded into every " +
42
+ "session and the body loaded when the skill triggers. Returns token counts per skill. " +
43
+ "Read-only: reads files only.";
44
+ const text = (s) => ({ content: [{ type: "text", text: s }] });
45
+ const failure = (e) => ({
46
+ content: [{ type: "text", text: `error: ${e instanceof Error ? e.message : String(e)}` }],
47
+ isError: true,
24
48
  });
25
- const TOOLS = [
26
- {
27
- name: "efaimo_check_skill",
28
- description: "Lint an Agent Skill, or a folder of skills, for spec compliance, trigger quality, " +
29
- "context-window cost, reference integrity, and injection hygiene. Returns a grade from " +
30
- "A to F and the findings. Read-only: reads files only.",
31
- inputSchema: pathSchema("Path to a SKILL.md file or a directory that contains skills."),
49
+ export function buildMcpServer() {
50
+ const server = new McpServer({ name: "efaimo", version: VERSION }, { capabilities: { tools: {} } });
51
+ server.registerTool("efaimo_check_skill", {
52
+ description: CHECK_DESCRIPTION,
53
+ inputSchema: pathInput("Path to a SKILL.md file or a directory that contains skills."),
32
54
  annotations: READ_ONLY,
33
- async run(args) {
34
- const p = requireString(args, "path");
35
- const res = await checkSkillSet(p, p);
55
+ }, async ({ path }) => {
56
+ try {
57
+ const res = await checkSkillSet(path, path);
36
58
  if (res.perSkill.length === 1 && res.setFindings.length === 0) {
37
- return renderCheckMarkdown(res.perSkill[0].report);
59
+ return text(renderCheckMarkdown(res.perSkill[0].report));
38
60
  }
39
- return renderSkillSetMarkdown(res);
40
- },
41
- },
42
- {
43
- name: "efaimo_weigh_skill",
44
- description: "Measure the context-window token cost of an Agent Skill: the metadata loaded into every " +
45
- "session and the body loaded when the skill triggers. Returns token counts per skill. " +
46
- "Read-only: reads files only.",
47
- inputSchema: pathSchema("Path to a SKILL.md file or a directory that contains skills."),
48
- annotations: READ_ONLY,
49
- async run(args) {
50
- const p = requireString(args, "path");
51
- const set = findSkills(p);
52
- if (!set.skills.length)
53
- throw new Error(`no SKILL.md found under "${p}"`);
54
- return renderWeighMarkdown(await weighSkills(set));
55
- },
56
- },
57
- ];
58
- export function buildMcpServer() {
59
- const server = new Server({ name: "efaimo", version: VERSION }, { capabilities: { tools: {} } });
60
- server.setRequestHandler(ListToolsRequestSchema, () => ({
61
- tools: TOOLS.map((t) => ({
62
- name: t.name,
63
- description: t.description,
64
- inputSchema: t.inputSchema,
65
- annotations: t.annotations,
66
- })),
67
- }));
68
- server.setRequestHandler(CallToolRequestSchema, async (req) => {
69
- const tool = TOOLS.find((t) => t.name === req.params.name);
70
- if (!tool) {
71
- return { content: [{ type: "text", text: `unknown tool: ${req.params.name}` }], isError: true };
61
+ return text(renderSkillSetMarkdown(res));
72
62
  }
63
+ catch (e) {
64
+ return failure(e);
65
+ }
66
+ });
67
+ server.registerTool("efaimo_weigh_skill", {
68
+ description: WEIGH_DESCRIPTION,
69
+ inputSchema: pathInput("Path to a SKILL.md file or a directory that contains skills."),
70
+ annotations: READ_ONLY,
71
+ }, async ({ path }) => {
73
72
  try {
74
- const text = await tool.run(req.params.arguments ?? {});
75
- return { content: [{ type: "text", text }] };
73
+ const set = findSkills(path);
74
+ if (!set.skills.length)
75
+ throw new Error(`no SKILL.md found under "${path}"`);
76
+ return text(renderWeighMarkdown(await weighSkills(set)));
76
77
  }
77
78
  catch (e) {
78
- const message = e instanceof Error ? e.message : String(e);
79
- return { content: [{ type: "text", text: `error: ${message}` }], isError: true };
79
+ return failure(e);
80
80
  }
81
81
  });
82
82
  return server;
83
83
  }
84
84
  export async function runMcpServer() {
85
- const server = buildMcpServer();
86
- await server.connect(new StdioServerTransport());
85
+ // A factory, not an instance: `serveStdio` pins one per connection after the
86
+ // opening exchange tells it which era it is talking to.
87
+ serveStdio(() => buildMcpServer(), {
88
+ onerror: (e) => console.error(`efaimo mcp: ${e.message}`),
89
+ });
87
90
  console.error(`efaimo mcp v${VERSION}: read-only skill tools ready on stdio`);
91
+ // stdin holds the process open; there is nothing to await. Returning a promise
92
+ // that never settles would only make the shutdown path harder to read.
93
+ await new Promise(() => { });
88
94
  }
@@ -1,7 +1,7 @@
1
1
  import path from "node:path";
2
2
  import { scanTextForInjection } from "../injection.js";
3
3
  import { tokenize, QUERY_STOPWORDS } from "../../find/tokenize.js";
4
- import { formatTokens } from "../../util/misc.js";
4
+ import { formatTokens, readTextSafe, isBinaryPath } from "../../util/misc.js";
5
5
  const NAME_RE = /^[a-z0-9]+(?:-[a-z0-9]+)*$/;
6
6
  const STANDARD_FIELDS = new Set(["name", "description", "license", "compatibility", "metadata", "allowed-tools"]);
7
7
  function skillLabel(s) {
@@ -309,4 +309,116 @@ const s106 = {
309
309
  return findings;
310
310
  },
311
311
  };
312
- export const SKILL_RULES = [s101, s102, s103, s104, s105, s106];
312
+ // S108. A file under references/ that nothing points at.
313
+ //
314
+ // Found on 2026-09-04 in two of this project's own seven published skills:
315
+ // read-back shipped a 4.6 KB failure gallery that had been growing for a
316
+ // month, unreleased-guard shipped the-gap.md, and no SKILL.md linked to
317
+ // either. Both were packaged, both installed byte-perfect into a user's skills
318
+ // directory, and an agent had no route to them. The agentskills.io layout puts
319
+ // material "loaded into context as needed" under references/, and the only
320
+ // thing that can cause a load is a path in the instructions. Installer tests,
321
+ // house style and an A(100) were all green about it, because none of them is
322
+ // about whether a shipped file is reachable.
323
+ //
324
+ // Reachable means: a reference the parser already resolved (a markdown link
325
+ // or a backticked path), the file's skill-relative path appearing anywhere in
326
+ // the body, or an ancestor directory named with its trailing slash ("see
327
+ // references/"); and transitively the same from any reference file that is
328
+ // itself reached, because a reader who opens references/api.md can follow a
329
+ // link inside it. The lenient direction is deliberate: a false red here
330
+ // teaches the reader to skip the rule.
331
+ //
332
+ // Scoped to references/ on purpose. scripts/ is executed rather than read and
333
+ // assets/ is consumed by scripts, so an unmentioned file there is a different
334
+ // claim. Measured on the public corpus at the pinned commits before this was
335
+ // written (research/skills-index/manifest.json): 2 of 36 skills carry a
336
+ // references/ directory at all, 4 files between them, no orphan. Our own set
337
+ // had two on 2026-09-04 and has none since. There is no population to
338
+ // calibrate a penalty on, so this is reported without being scored, the way
339
+ // S105 is; scoring it is a separate decision that has to move grades with
340
+ // evidence behind it.
341
+ const S108_MAX = 10;
342
+ const s108 = {
343
+ id: "S108",
344
+ title: "reference file nothing points at",
345
+ surface: "skill",
346
+ check({ skill }) {
347
+ const refDir = path.join(skill.dir, "references");
348
+ const prefix = refDir + path.sep;
349
+ const candidates = skill.files.filter((f) => f.path.startsWith(prefix));
350
+ if (!candidates.length)
351
+ return [];
352
+ const rel = (abs) => path.relative(skill.dir, abs).split(path.sep).join("/");
353
+ // The file's own path, or any ancestor directory of it named on its own
354
+ // with its trailing slash, appearing in the text.
355
+ //
356
+ // "On its own" is load-bearing and the first version did not have it: it
357
+ // asked whether the text contained "references/", and every link to a
358
+ // sibling file under references/ contains exactly that, so linking ONE
359
+ // reference file marked every other file in the directory as reached and
360
+ // the rule could not fire on the fixture written to make it fire. The
361
+ // slash has to be followed by something that cannot continue a path.
362
+ const dirNamed = (text, dir) => new RegExp(dir.replace(/[.*+?^${}()|[\]\\]/g, "\\$&") + "/(?![A-Za-z0-9_.-])").test(text);
363
+ const mentions = (text, r) => {
364
+ if (text.includes(r))
365
+ return true;
366
+ const parts = r.split("/");
367
+ for (let i = 1; i < parts.length; i++) {
368
+ if (dirNamed(text, parts.slice(0, i).join("/")))
369
+ return true;
370
+ }
371
+ return false;
372
+ };
373
+ const reached = new Set();
374
+ const resolvedRefs = new Set(skill.referencedPaths.filter((x) => x.exists && !x.escapes).map((x) => path.resolve(x.resolved)));
375
+ for (const c of candidates) {
376
+ if (resolvedRefs.has(path.resolve(c.path)) || mentions(skill.body, rel(c.path)))
377
+ reached.add(c.path);
378
+ }
379
+ // Transitive: a reached text file can point at its siblings, by the
380
+ // skill-relative path or by a path relative to itself.
381
+ const scanned = new Set();
382
+ let grew = true;
383
+ while (grew) {
384
+ grew = false;
385
+ for (const r of [...reached]) {
386
+ if (scanned.has(r) || isBinaryPath(r))
387
+ continue;
388
+ scanned.add(r);
389
+ const text = readTextSafe(r);
390
+ if (text === undefined)
391
+ continue;
392
+ const from = path.dirname(r);
393
+ for (const c of candidates) {
394
+ if (reached.has(c.path))
395
+ continue;
396
+ const own = path.relative(from, c.path).split(path.sep).join("/");
397
+ if (mentions(text, rel(c.path)) || text.includes(own)) {
398
+ reached.add(c.path);
399
+ grew = true;
400
+ }
401
+ }
402
+ }
403
+ }
404
+ const label = skillLabel(skill);
405
+ const findings = [];
406
+ for (const c of candidates) {
407
+ if (reached.has(c.path))
408
+ continue;
409
+ if (findings.length >= S108_MAX)
410
+ break;
411
+ findings.push({
412
+ ruleId: "S108",
413
+ severity: "warn",
414
+ title: "reference file nothing points at",
415
+ message: `"${rel(c.path)}" ships with the skill and nothing in SKILL.md, or in any reference it does reach, names it; an agent has no path to it, so it costs the reader disk and the publisher maintenance and is never loaded`,
416
+ target: label,
417
+ fixHint: "link it from SKILL.md at the point where it applies, or delete it",
418
+ graded: false,
419
+ });
420
+ }
421
+ return findings;
422
+ },
423
+ };
424
+ export const SKILL_RULES = [s101, s102, s103, s104, s105, s106, s108];
@@ -12,6 +12,11 @@
12
12
  * no `rulesVersion` field at all; the absence IS the value.
13
13
  * "2" the first ruleset that declares itself. Adds the findability family
14
14
  * (E141-E145), which no earlier version had.
15
+ * "3" adds S108, a reference file nothing points at. Reported and not
16
+ * scored, so no grade moves with it; the number moves anyway, because a
17
+ * report from this ruleset carries a rule id that "2" could not have
18
+ * produced, and a diff across the two would attribute that to the
19
+ * subject.
15
20
  *
16
21
  * Bump it when a rule is added, removed, renumbered, or changed in what it
17
22
  * fires on. `test/meta.test.ts` pins the rule inventory, so an added or
@@ -23,4 +28,4 @@
23
28
  * would make it move on every release whether or not a rule moved, which is
24
29
  * the thing it exists to distinguish.
25
30
  */
26
- export declare const RULES_VERSION = "2";
31
+ export declare const RULES_VERSION = "3";
@@ -12,6 +12,11 @@
12
12
  * no `rulesVersion` field at all; the absence IS the value.
13
13
  * "2" the first ruleset that declares itself. Adds the findability family
14
14
  * (E141-E145), which no earlier version had.
15
+ * "3" adds S108, a reference file nothing points at. Reported and not
16
+ * scored, so no grade moves with it; the number moves anyway, because a
17
+ * report from this ruleset carries a rule id that "2" could not have
18
+ * produced, and a diff across the two would attribute that to the
19
+ * subject.
15
20
  *
16
21
  * Bump it when a rule is added, removed, renumbered, or changed in what it
17
22
  * fires on. `test/meta.test.ts` pins the rule inventory, so an added or
@@ -23,4 +28,4 @@
23
28
  * would make it move on every release whether or not a rule moved, which is
24
29
  * the thing it exists to distinguish.
25
30
  */
26
- export const RULES_VERSION = "2";
31
+ export const RULES_VERSION = "3";
package/dist/version.d.ts CHANGED
@@ -1,2 +1,2 @@
1
1
  /** Kept in sync with package.json by test/meta.test.ts. */
2
- export declare const VERSION = "0.4.0";
2
+ export declare const VERSION = "0.6.0";
package/dist/version.js CHANGED
@@ -1,2 +1,2 @@
1
1
  /** Kept in sync with package.json by test/meta.test.ts. */
2
- export const VERSION = "0.4.0";
2
+ export const VERSION = "0.6.0";
package/docs/RULES.md CHANGED
@@ -65,7 +65,7 @@ never reused, so a gap is permanent and is better stated than smoothed over.
65
65
  | E128 | warn | total tool-definition tax over ~10k / ~25k tokens, estimated (needs `weigh` data); hard CI failure is left to the `--max-tokens`/`--diff` gate |
66
66
  | E130 | info | instruction-injection patterns in a tool description or server instructions (shallow heuristic, never a security verdict; use a dedicated scanner for depth). **Reported but never scored** |
67
67
 
68
- ## Skills (S101-S107)
68
+ ## Skills (S101-S108)
69
69
 
70
70
  | id | sev | what it catches | spec |
71
71
  |---|---|---|---|
@@ -76,6 +76,7 @@ never reused, so a gap is permanent and is better stated than smoothed over.
76
76
  | S105 | info | instruction-injection patterns in the skill body or description (shallow heuristic, never a security verdict). **Reported but never scored** | internal |
77
77
  | S106 | error/warn/info | referenced file missing, escaping the skill dir (`..`), or nested more than one level deep | agentskills.io |
78
78
  | S107 | warn | a file one capitalisation away from a skill, such as `skill.md`. Set-level and never scored: the spec names `SKILL.md` exactly, so it is not a skill here on any platform, but a case-insensitive filesystem (the macOS and Windows default) may hand it to a host, which makes a repository work on a laptop and have no skill at all in Linux CI | internal |
79
+ | S108 | warn | a file under `references/` that nothing points at: no markdown link or backticked path resolves to it, its path does not appear in `SKILL.md`, no ancestor directory of it is named there, and no reference file that IS reached mentions it either. It ships and installs perfectly and an agent has no route to it. **Reported but never scored** _(ruleset 3)_: at the pinned public corpus only 2 of 36 skills carry a `references/` directory and neither has an orphan, so there is no population to calibrate a penalty on; the two real instances were in this project's own skills | internal |
79
80
 
80
81
  ## Findability (E141-E145)
81
82
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "efaimo",
3
- "version": "0.4.0",
3
+ "version": "0.6.0",
4
4
  "description": "Audit what your agent loads: quality and context cost for MCP servers and Agent Skills",
5
5
  "keywords": [
6
6
  "mcp",
@@ -65,17 +65,19 @@
65
65
  "prepublishOnly": "npm run typecheck && npm run build && npm run test"
66
66
  },
67
67
  "dependencies": {
68
- "@modelcontextprotocol/sdk": "^1.30.0",
68
+ "@modelcontextprotocol/client": "2.3.1",
69
+ "@modelcontextprotocol/server": "2.3.1",
69
70
  "commander": "^15.0.0",
70
71
  "cross-spawn": "^7.0.6",
71
72
  "gpt-tokenizer": "^3.4.0",
72
73
  "picocolors": "^1.1.1",
73
- "yaml": "^2.9.0"
74
+ "yaml": "^2.9.0",
75
+ "zod": "^4.6.6"
74
76
  },
75
77
  "devDependencies": {
76
78
  "@types/cross-spawn": "^6.0.6",
77
79
  "@types/node": "^26.1.1",
78
80
  "typescript": "^7.0.2",
79
- "vitest": "^4.1.10"
81
+ "vitest": "^4.1.12"
80
82
  }
81
83
  }