@ssheleg/make-skill 0.27.1 → 0.29.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,3 +1,47 @@
1
+ ## v0.29.0 — the standard now reads a hook key the host ignores
2
+
3
+ Claude Code 2.1.270 began printing `hooks.json: unknown key "if" in
4
+ hooks.PreToolUse[1] ignored` at every session start. The key had been there since
5
+ agent-sync 0.1.0, and the same shape sat in the hooks template `task-pipeline`
6
+ exports into consumer projects — where the documentation gate then ran on **every**
7
+ Bash call rather than on commits. Nothing caught it: not nine member validators, not
8
+ this auditor, and not `claude plugin validate --strict`, which accepts the file.
9
+
10
+ - **`HOOKS_SCHEMA`** reads the plugin's `hooks/hooks.json` from a skill at
11
+ `<plugin>/skills/<name>` and refuses any key outside the set its level accepts —
12
+ a matcher group takes `matcher` + `hooks`; a `command` handler takes `type`,
13
+ `command`, `args`, `if`, `shell`, `timeout`, `statusMessage`, `once`, `async`,
14
+ `asyncRewake` and three `@internal` keys. Both sets are read out of the 2.1.270
15
+ binary's schema rather than from prose. A manifest that does not parse is a GAP
16
+ (the host then loads no hooks at all); a plugin with no manifest passes and says
17
+ so; a `prompt`/`agent`/`http`/`mcp_tool` handler is **declared unchecked**,
18
+ because only the command handler's key set was measured.
19
+ - **`BUNDLE_NESTED` stops reporting `__pycache__`.** A byte-compile cache appears
20
+ the moment a test imports a shipped module; it is gitignored and npm-excluded in
21
+ every member, and three of them were carrying the gap for a directory that never
22
+ ships.
23
+ - `references/host-capabilities.md` states both key sets, where `if` lives, the
24
+ loader's exact warning, and that `validate --strict` is not the gate for this.
25
+
26
+ Every shipped skill in the family passes: 28 of 28 at `0 GAP`, now `19 PASS`.
27
+
28
+ ## v0.28.0 — the house audit measures the token budget instead of estimating it
29
+
30
+ Sherlock external-v3 (13 findings) plus the enterprise handoff (PR #18) and the
31
+ CI correction the audit forced across the family.
32
+
33
+ - **The house auditor was issuing a token verdict from an estimate.** With no
34
+ tokenizer on the runner it fell back to chars/3.9 and gapped seven family
35
+ skills that are all inside the limit when actually measured. That is the defect
36
+ this script's own doctrine names (FIX-MS-01.01): the estimate overshoots prose
37
+ carrying Russian and code, and a verdict from the wrong instrument wearing the
38
+ right instrument's name is worse than no verdict. Every member's CI now
39
+ installs tiktoken and pins this version, so the budget is MEASURED.
40
+ - `references/enterprise.md` gains the dependency-closure and provenance
41
+ material for reviewing or adapting someone else's skill.
42
+ - The reworded row that pointed at it hit exactly the 4750 working limit this
43
+ repo's own validator refuses — tightened rather than waived.
44
+
1
45
  ## v0.27.1 — the tarball stops shipping compiled Python, and the audit vocabulary the routing block promised becomes advertised
2
46
 
3
47
  Family audit 2026-09-06, wave `AUDIT-WAVE-0906`. Two findings, both a gap between what
package/README.md CHANGED
@@ -164,6 +164,7 @@ agent opens only when the situation calls for them:
164
164
  | `surfaces.md` | Claude Code vs the Claude API vs claude.ai — the Skills API (upload, versions, 8 per request), and the no-network / no-package-install limits that break scripts moved between surfaces |
165
165
  | `enterprise.md` | installing a skill you didn't write, and running a fleet — risk tiers, the review checklist, the five approval gates, lifecycle, recall limits, rollback |
166
166
  | `retrofit.md` | the audit procedure — the 14-item checklist, what counts as evidence for a PASS, and the short form for a personal skill |
167
+ | `outcome-evaluation.md` | the outcome-eval method — frozen inputs, baseline-vs-current arms, artifact checks, the routing/correctness/visual split, and PASS/FAIL/ERROR/NOT_RUN verdicts with no host-locked actor |
167
168
  | `host-capabilities.md` | hooks, subagents, commands, scripts and MCP dependencies — what each buys, what it costs in always-on tokens, hook events and exit-code semantics, and the degradation clauses that keep a skill working where none of them exist |
168
169
  | `claude-code-plugin.md` | the [Claude Code layer](https://code.claude.com/docs/en/plugins-reference) — `plugin.json` / `marketplace.json` schemas, plugin sources, component locations, host-only front-matter, path variables, cache and symlink rules, the `claude plugin` CLI |
169
170
  | `distribution.md` | the repo layout, every install channel, exact CLI flags, npm publishing traps, the release checklist |
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@ssheleg/make-skill",
3
- "version": "0.27.1",
3
+ "version": "0.29.0",
4
4
  "description": "Create, retrofit, audit, and ship agent skills & Claude Code plugins the proven ssheleg way \u2014 conformance to the Agent Skills open standard AND Anthropic's platform rules (front-matter limits, disclosure budgets, per-surface runtime limits, the Skills API, evals) plus the Claude Code plugin reference (manifest schemas, component layout, claude plugin validate --strict), marketplace repo layout, version sync, validator + CI, multi-channel distribution (plugin, vercel skills CLI, npx, Cursor), npm gotchas, the review checklist for third-party skills, and MCP / A2A rules for protocol-connected skills. This package is the installer CLI.",
5
5
  "keywords": [
6
6
  "skill",
@@ -47,6 +47,7 @@
47
47
  "node": ">=16"
48
48
  },
49
49
  "scripts": {
50
- "test": "python3 test/validate.py && python3 test/plant_guard_test.py && python3 test/checker_parity_test.py && python3 test/residue_test.py && node test/installer_test.js"
50
+ "test": "python3 test/validate.py && python3 test/plant_guard_test.py && python3 test/checker_parity_test.py && python3 test/residue_test.py && node test/installer_test.js && npm run test:audit",
51
+ "test:audit": "for t in test/audit_regressions/*.py; do python3 \"$t\" || exit 1; done"
51
52
  }
52
53
  }
@@ -3,7 +3,7 @@
3
3
  "name": "make-skill",
4
4
  "displayName": "Make Skill",
5
5
  "description": "Create, retrofit, audit, and ship agent skills & Claude Code plugins the proven ssheleg way: conformance to the Agent Skills open standard, Anthropic's platform rules (surfaces, Skills API, evals) and the Claude Code plugin reference, marketplace repo layout, version sync, validator + CI, multi-channel distribution (plugin, vercel skills CLI, npx, Cursor), npm gotchas, end-to-end first publish, the review checklist for third-party skills, plus MCP / A2A references for protocol-connected skills.",
6
- "version": "0.27.1",
6
+ "version": "0.29.0",
7
7
  "author": {
8
8
  "name": "ssheleg",
9
9
  "url": "https://x.com/sshlg93"
@@ -5,7 +5,7 @@ license: MIT
5
5
  compatibility: Authoring works on any agent. The bundled scripts/ need python3. Publishing steps need git, gh, node and npm; the plugin gates need the claude CLI. Not usable on the Claude API surface, which has no network and no runtime package install.
6
6
  metadata:
7
7
  author: ssheleg
8
- version: "0.27.1"
8
+ version: "0.29.0"
9
9
  homepage: https://github.com/ssheleg/make-skill
10
10
  ---
11
11
 
@@ -22,8 +22,9 @@ orchestrator, release automation). **make-skill itself** is built to this canon.
22
22
  | `references/agent-skills-spec.md` | authoring or auditing ANY `SKILL.md` — hard limits from both authorities, optional fields, budgets, who rejects what |
23
23
  | `references/authoring.md` | writing or tuning a body/description — naming, third person, degrees of freedom, script rules, eval loops |
24
24
  | `references/surfaces.md` | shipping anywhere but Claude Code — Skills API upload/versions/8-per-request, claude.ai zip, the no-network limits |
25
- | `references/enterprise.md` | installing someone else's skill, or governing a fleet — risk tiers, review checklist, approval gates, lifecycle |
25
+ | `references/enterprise.md` | reviewing, adapting or installing an external skill — dependency closure, provenance, risk tiers, lifecycle |
26
26
  | `references/retrofit.md` | auditing an existing skill/repo — the 14-item checklist, the evidence rules, the personal-skill short form |
27
+ | `references/outcome-evaluation.md` | proving a skill changed real outcomes — frozen inputs, baseline vs current, artifact checks, the routing/correctness/visual split, PASS/FAIL/ERROR/NOT_RUN |
27
28
  | `references/host-capabilities.md` | shipping a **hook, subagent, command, script or MCP dependency** — what each buys and costs, hook events and exit codes, the degradation clauses |
28
29
  | `references/claude-code-plugin.md` | anything shipping as a **Claude Code plugin/marketplace** — manifest schemas, component layout, path variables, `validate` failures |
29
30
  | `references/distribution.md` | the repo layout, releases, and all five channels — plugin, skills CLI, npx, Cursor, umbrella family repo |
@@ -45,9 +46,8 @@ Detect from the request and any path in `$ARGUMENTS`; announce the choice.
45
46
  | Personal skill should become installable | Promote |
46
47
 
47
48
  With no argument, **detect instead of asking**: a `SKILL.md`, `.claude-plugin/`
48
- or `plugins/*/skills/*/` in the current directory → run the Retrofit audit and
49
- report the gap table plus ONE next action. Nothing to detect → ask in one line
50
- what to create.
49
+ or `plugins/*/skills/*/` here → run the Retrofit audit, report the gap table
50
+ plus ONE next action. Nothing to detect → ask in one line what to create.
51
51
 
52
52
  Distributable work is a real project: spec (`docs/evidence/specs/`) before
53
53
  code, and the spec locks target-project file contracts FIRST — skills are
@@ -129,20 +129,19 @@ House additions on top of the spec:
129
129
 
130
130
  ### Degradation contract (every skill that touches a host capability)
131
131
 
132
- Hooks, subagents, `/commands`, plugin path variables and MCP servers exist only
133
- inside Claude Code — a minority of where skills run. **Each is an accelerator
134
- with a written fallback; the skill still finishes its job without it, more
135
- slowly.** Write the three cases into the body, in the agent's words, at the
136
- point it will need them (shapes: `references/host-capabilities.md`):
137
-
138
- - **Not Claude Code** (Cursor, Codex, skills CLI, API): no hooks, no subagents,
139
- no `/command`. Name the inline procedure; bundled `scripts/` still travel, so
140
- give the path per channel.
141
- - **Recommended plugin/skill absent**: say once what is degraded, continue on
142
- the manual path. A stage that refuses to start because an optional companion
143
- is missing is broken, not strict.
144
- - **Tool, interpreter or MCP server absent**: state it once, fall back to the
145
- by-hand procedure, never retry in a loop. Interactive auth is a human step.
132
+ Hooks, subagents, `/commands`, plugin path variables and MCP servers are HOST
133
+ capabilities that vary by host AND version — subagents and MCP are native to
134
+ some non-Claude runtimes, so DETECT them, never assume "Claude Code only".
135
+ **Each is an accelerator with a written fallback; the skill finishes its job
136
+ without it, more slowly** — a portable body names the inline procedure, not one
137
+ host's exact tool spelling. Write the three fallback cases into the body, in
138
+ the agent's words, at the point it will need them — a HOST lacking a capability
139
+ (the set differs per host: not every non-Claude runtime lacks subagents/MCP), a
140
+ recommended companion absent, and a tool/interpreter/MCP server absent (state it
141
+ once, fall back by hand, never loop; interactive auth is a human step). The
142
+ fallback shapes are in `references/host-capabilities.md`; the per-host
143
+ capability matrix (with each norm's owner and check date) in
144
+ `references/agent-skills-spec.md`.
146
145
 
147
146
  A fallback you know but did not write is not a fallback.
148
147
 
@@ -198,28 +197,25 @@ Done = the five VERIFIED facts in that sequence's step 10 — nothing assumed.
198
197
 
199
198
  ## Retrofit (bring an existing skill/repo up to standard)
200
199
 
201
- Audit first, fix second, in the same session. Verdict per item: PASS / GAP /
202
- NOT-RUN with evidence — a `file:line` or the output of the command you actually
203
- ran. "Looks fine" is not a verdict, and neither is a PASS on a check that was
204
- reasoned about instead of executed; a check whose tool is absent is **NOT-RUN
205
- with the reason**, never a PASS.
206
-
207
- **Run the bundled auditor first** — it does the mechanical half
208
- deterministically, and it never depends on an unset variable:
209
-
210
- ```bash
211
- make-skill-audit <skill-dir> --house # Claude Code: the plugin's bin/ is on PATH
212
- ```
213
-
214
- Anywhere else, run `scripts/audit_skill.py` from the make-skill directory you
215
- just read this from. **Then work the 14-item checklist in
216
- `references/retrofit.md`** — spec floor, plugin floor, surfaces, one-job, entry
217
- point, layout and version sync, validator and CI, evaluations, README,
218
- distribution, repo meta, gotchas, protocols, host capabilities. For a PERSONAL
219
- skill only three items apply; that file says which.
220
-
221
- **Then:** report the gap table, fix everything fixable now, bump a minor/patch
222
- version, run the release checklist.
200
+ **Three modes, three effect contracts (MS-03): `audit` reads (evidence + plan
201
+ only); `retrofit` writes only what the request scoped; `release` publishes.**
202
+ The move between them is decided by INTENT and prior authorization, never by
203
+ the skill invoked — a compliance QUESTION stays an audit («аудит скилов» asks
204
+ for a verdict, not a diff). Verdict per item: PASS / GAP / NOT-RUN with
205
+ evidence — a `file:line` or the command's actual output. "Looks fine" is not a
206
+ verdict, nor is a PASS reasoned about instead of executed; a check whose tool
207
+ is absent is **NOT-RUN with the reason**, never a PASS.
208
+
209
+ **Run the bundled auditor first** (the deterministic mechanical half), then
210
+ work the 14-item checklist — both the `make-skill-audit --house` invocation
211
+ and the checklist live in `references/retrofit.md`; a PERSONAL skill owes only
212
+ three of the items.
213
+
214
+ **Then: report the gap table — and stop there in `audit` mode.** Only with
215
+ `retrofit` granted: fix what the report names; only with `release`: bump
216
+ minor/patch and run the release checklist. Load
217
+ `references/outcome-evaluation.md` only when the work CHANGES behaviour — a
218
+ conformance audit stops at its report, no outcome arms.
223
219
 
224
220
  ## Promote (personal → distributable)
225
221
 
@@ -134,9 +134,40 @@ Rules:
134
134
  |---|---|---|
135
135
  | `skills-ref validate ./<skill dir>` | the open standard's frontmatter rules | Python, installed from source out of `github.com/agentskills/agentskills`; not on npm or PyPI |
136
136
  | Skills API upload | Anthropic's extra rules (reserved words, XML tags, dir name, 30 MB) | the only place they are enforced — see `references/surfaces.md` |
137
- | `claude plugin validate … --strict` | plugin/marketplace **manifests only**, not SKILL.md frontmatter | `references/claude-code-plugin.md` |
137
+ | `claude plugin validate … --strict` | plugin/marketplace manifests — AND, since Claude Code v2.1.233, `SKILL.md` frontmatter in skill directories too (version-gate the claim; older builds validated manifests only) | `references/claude-code-plugin.md` |
138
138
  | your `test/validate.py` | house rules + everything the three above miss | the only one that runs on every commit |
139
139
 
140
+ ## Host capability matrix — presence varies by host AND version (MS-04)
141
+
142
+ "Claude Code only" is wrong for several of these: some non-Claude runtimes
143
+ (Codex) provide subagents and MCP natively. DETECT the capability; do not
144
+ assume its absence. Presence is per host AND per version.
145
+
146
+ | Capability | Claude Code | Codex | Cursor | skills CLI / API | Detect by |
147
+ |---|---|---|---|---|---|
148
+ | Hooks | yes | no | no | no | host docs / config |
149
+ | Subagents | yes | **yes (native)** | no | no | runtime probe |
150
+ | MCP servers | yes | **yes (native)** | varies | no | runtime probe |
151
+ | `/commands` | yes | no | no | no | host docs |
152
+ | Plugin path vars | yes | no | no | no | env presence |
153
+
154
+ And every NORM this skill enforces carries its **owner** (spec = the Agent
155
+ Skills standard, host = a runtime's own rule, house = this family), whether it
156
+ is **required vs recommended**, and the **date last verified against source** —
157
+ a number with no owner reads as physics, and an unversioned norm has no expiry:
158
+
159
+ | Norm | Owner | Required? | Last checked |
160
+ |---|---|---|---|
161
+ | `name` ≤ 64 chars, `description` ≤ 1024 | spec | required | 2026-09-09 |
162
+ | body < 5000 tokens | spec | recommended | 2026-09-09 |
163
+ | body < 4750 tokens (headroom) | house | recommended | 2026-09-09 |
164
+ | Skills API: 8 skills/request, 30 MB | host | required (that surface) | 2026-09-09 |
165
+ | `--strict` validates SKILL.md frontmatter | host (Claude Code ≥ v2.1.233) | n/a | 2026-09-09 |
166
+
167
+ A recommendation (500 lines, 5000 tokens, the house limits) is a QUALITY norm,
168
+ not a universal reason a host refuses to LOAD the skill — a body over budget is
169
+ worse authoring, not an install error everywhere.
170
+
140
171
  ## Conformance checklist
141
172
 
142
173
  - [ ] `name`: matches dir, ≤64 chars, `[a-z0-9-]`, no leading/trailing/double
@@ -20,6 +20,7 @@ and [agentskills.io](https://agentskills.io/skill-creation/best-practices).
20
20
  - Body patterns worth copying
21
21
  - Workflows and feedback loops
22
22
  - Content guidelines (terminology, time, paths, table of contents)
23
+ - Selective loading — every reference earns its tokens
23
24
  - Scripts — the rules that separate a script from a liability
24
25
  - A gate's self-test runs in both directions
25
26
  - Evaluation and iteration — evals before prose
@@ -157,6 +158,10 @@ field makes the agent worse than it was without the skill.
157
158
  - **Add what the agent lacks**; cut anything it already knows. Test: "would the
158
159
  agent get this wrong without this line?" No → delete it.
159
160
  - **Procedures over answers** — teach the method, not one instance's result.
161
+ - **Copying from OUTSIDE** — when a body pattern is borrowed from an external
162
+ source rather than written fresh, route the intake through the source/runtime
163
+ contract in `references/enterprise.md` (§ Selective knowledge adoption): a
164
+ pinned permalink and an attribution receipt, and no-key is not no-dependency.
160
165
 
161
166
  ## Workflows and feedback loops
162
167
 
@@ -201,6 +206,46 @@ the body.
201
206
  rewriting the eight ssheleg routers into English cut them **3408 → 1885
202
207
  tokens** with no loss of meaning.
203
208
 
209
+ ## Selective loading — every reference earns its tokens
210
+
211
+ A reference is a loan against the context window, and the load condition is the
212
+ contract that says when the loan is worth taking. Three rules, each of which has
213
+ been paid for:
214
+
215
+ - **Every reference states its own load condition** — the `**Load this when:**`
216
+ opener this file itself uses, or a "read X when Y" sentence at the link site in
217
+ `SKILL.md`. "See `references/`" is a pointer at a directory: it loads either
218
+ everything (the budget bleeds on every turn) or nothing (dead doctrine), and
219
+ `scripts/audit_skill.py` flags it. A condition names the situation, not the
220
+ file: "read `references/mcp.md` when the skill must reach an MCP server", never
221
+ "additional details in mcp.md".
222
+ - **Split primary from appendix, and say which is which.** Primary material is
223
+ what a decision REQUIRES: the contract, the acceptance criteria, the failure
224
+ modes. Appendix material is depth — worked examples, history, fixtures — that
225
+ can stay unloaded without changing any decision. The split decides what a
226
+ budget cut may touch: **mandatory decisions and acceptance are never truncated
227
+ to fit a budget; the appendix is.** A body that trimmed its acceptance to make
228
+ a line count fit has traded the contract for the heuristic that was supposed
229
+ to protect it. Variants follow the same rule: where a reference exists in more
230
+ than one language or platform flavour, the load condition names WHICH variant a
231
+ given task requires, and only that one is loaded — a multilingual bundle that
232
+ loads every variant pays for the same knowledge twice.
233
+ - **A required reference that does not resolve blocks the work.** If the body
234
+ says a decision depends on `references/x.md` and the file is absent, the audit
235
+ gap is a stop, not a warning — proceeding without the contract is how a skill
236
+ ships with half its acceptance. `audit_skill.py` exits non-zero on exactly
237
+ this.
238
+
239
+ **The budget is tokens, and lines are only a heuristic.** "500 lines / 5000
240
+ tokens" pairs a length heuristic with the real limit; the 500 never proves the
241
+ 5000. A multilingual body makes the gap concrete: Russian encodes at 1.9–2.3
242
+ chars/token against English's 5.0 (`cl100k`), so 300 Cyrillic lines can out-cost
243
+ 500 English ones. A token figure is therefore either **measured by a named
244
+ tokenizer** or **an explicitly labeled estimate** — `audit_skill.py` reports
245
+ `~N tokens (chars / 3.9)`, and the divisor is printed because an estimate that
246
+ hides its basis reads as a measurement. Never state a bare token count nothing
247
+ computed.
248
+
204
249
  ## Scripts — the rules that separate a script from a liability
205
250
 
206
251
  A bundled script is more reliable than generated code, costs no context (only its
@@ -273,7 +318,10 @@ gate's own output.
273
318
  ## Evaluation and iteration — evals before prose
274
319
 
275
320
  **Build the evaluations before writing extensive documentation.** Otherwise the
276
- skill documents imagined problems.
321
+ skill documents imagined problems. The judging method — frozen inputs, the
322
+ baseline arm, artifact checks, the routing/correctness/visual split and the
323
+ PASS/FAIL/ERROR/NOT_RUN verdicts — is `references/outcome-evaluation.md`; read
324
+ it when a verdict is about to be written down.
277
325
 
278
326
  1. **Identify gaps** — run the agent on representative tasks with NO skill.
279
327
  Record the specific failures.
@@ -340,6 +388,8 @@ quality — are in `references/enterprise.md`.
340
388
  - [ ] Name follows one pattern, is not vague, contains no reserved word
341
389
  - [ ] Body under 500 lines / 5000 tokens; detail in one-level-deep files
342
390
  - [ ] Every reference has a stated load condition; >100-line ones have a TOC
391
+ - [ ] Primary (decisions, acceptance) split from appendix; cuts only ever hit the appendix
392
+ - [ ] Token figures are measured by a named tokenizer or labeled as estimates with their basis
343
393
  - [ ] Degrees of freedom match task fragility
344
394
  - [ ] No time-sensitive statements outside an "Old patterns" section
345
395
  - [ ] Consistent terminology; forward slashes everywhere
@@ -397,6 +397,24 @@ npx <name> # from a NON-repo cwd
397
397
  claude plugin update <name>@<name> # full id required
398
398
  ```
399
399
 
400
+ ## The publishable payload — what must NOT ride along
401
+
402
+ The auditor's `DIST_*` checks (`scripts/audit_skill.py`) and this section are
403
+ one rule: a payload is what a consumer receives, and two things leak into it by
404
+ accident.
405
+
406
+ - **An outward symlink.** A skill installed by symlink is correct; a symlink
407
+ INSIDE the payload pointing OUTSIDE it is not — it resolves on the author's
408
+ machine and dangles (or leaks a path) everywhere else. `npm pack` follows the
409
+ files allowlist, but a symlink caught by a glob ships as a link, so keep the
410
+ package's own files real and let installation create the links.
411
+ - **An undeclared secret.** A `.env`, a `*token*` fixture, an `id_rsa`,
412
+ anything whose name reads as a credential must be excluded — `.npmignore` or
413
+ the `files` allowlist, and the auditor names any that remain. A closure
414
+ validated against the WHOLE checkout passes while the PACKAGED copy is
415
+ missing a required reference or carrying a secret; validate the payload, not
416
+ the checkout — `npm pack --dry-run` prints exactly what ships.
417
+
400
418
  ## Release checklist (every version)
401
419
 
402
420
  1. Bump the four versions together (`package.json` only if npm-distributed — else
@@ -20,6 +20,7 @@ plus the security section of the
20
20
  - Registry — what to record per skill
21
21
  - Recall limits and consolidation
22
22
  - Versioning, rollback, integrity
23
+ - Selective knowledge adoption
23
24
 
24
25
  ## Why this is a security boundary at all
25
26
 
@@ -94,6 +95,31 @@ Reading the results: declining trigger accuracy → fix the description; coexist
94
95
  conflicts → narrow descriptions or merge the skills; persistently low output
95
96
  quality → rewrite or add validation; persistent failure across updates → deprecate.
96
97
 
98
+ ## Selective knowledge adoption — the source/runtime contract
99
+
100
+ When a skill BORROWS from an external source — a snippet, a method, a whole
101
+ reference — the intake is ONE canonical procedure, and it reaches a verdict
102
+ WITHOUT calling any setup, login or provider: classification is reading, not
103
+ running. Four verdicts, keyed by what the source actually needs:
104
+
105
+ | The source... | Verdict | Why |
106
+ |---|---|---|
107
+ | authenticates by **OAuth but ships no key** | ADOPT, **declare the dependency** | no key is not "no dependency" — the runtime still needs the OAuth service; declare it, do not conclude it is free-standing |
108
+ | runs an **`npx <tool>` with the package undeclared** | REJECT the undeclared runtime fetch, or **declare the exact package+version** | an `npx` at runtime is a supply-chain edge; an undeclared one is a fetch of whatever the registry serves that day |
109
+ | ships **no license** | DO NOT COPY the bytes | reference the METHOD (procedures over answers), never vendor unlicensed source; adopt only what you may |
110
+ | names a **deprecated source** | adopt the **method**, not the bytes; note the deprecation | pinning to a dead/moving ref imports a liability with an expiry |
111
+
112
+ Two receipts every adoption carries:
113
+
114
+ - **A pinned permalink** to the exact source revision (a commit-addressed URL,
115
+ never a branch tip) for anything copied OR adapted — the bytes must be
116
+ traceable to what they came from.
117
+ - **An attribution receipt** — who wrote it, under what license, adapted how —
118
+ beside the borrowed content, so the next reader can re-verify the chain.
119
+
120
+ `references/authoring.md`'s body-pattern list routes here whenever a pattern is
121
+ copied from outside rather than written fresh.
122
+
97
123
  ## Lifecycle
98
124
 
99
125
  1. **Plan** — pick workflows that are repetitive, error-prone, or need specialist
@@ -139,3 +165,44 @@ user's active set stays focused.
139
165
  signed commits in the skill repo so provenance is more than a claim.
140
166
  - **Git is the source of truth**, one directory per skill, changes through pull
141
167
  requests — which also gives you the rollback and the audit trail for free.
168
+
169
+ ## Selective knowledge adoption
170
+
171
+ Reviewing a collection does not mean installing it. When the operator wants
172
+ methods without new dependencies, distinguish a transferable method from the
173
+ upstream workflow, runtime, assets and services that happen to deliver it.
174
+
175
+ 1. Freeze the repository commit and inventory skills, commands, referenced files,
176
+ scripts, bundled binaries, submodules and remote marketplace entries. Mark
177
+ what was read semantically and what was only scanned. A directory listing or
178
+ search hit is not a complete audit of its contents.
179
+ 2. Record the applicable license/NOTICE at the file or subtree level. Do not
180
+ infer every subtree's terms from the root or a front-matter label. An unclear
181
+ or restricted copying grant is a reference-only outcome, not a reason to
182
+ remove attribution. Preserve required notices for any permitted adaptation.
183
+ 3. Trace the proposed transfer's dependency closure: required local references,
184
+ interpreters/packages, commands, host APIs, MCP servers, credentials, runtime
185
+ fetches, fonts/images, installer and update behavior. Repository build tools
186
+ and optional test tooling are distinct from runtime requirements.
187
+ 4. Choose explicitly: adapt a bounded method, retain as a reference, offer an
188
+ optional adapter, defer for missing evidence, or reject. Do not install an
189
+ upstream orchestrator merely to obtain its typography or review checklist.
190
+ 5. Author the smallest local procedure that fills a measured gap. Name its
191
+ family owner and router. Keep existing paths and shared artifact contracts;
192
+ avoid a second brand folder, plan format or always-on workflow. Reuse an
193
+ existing skill/mode unless a separate invocation has a distinct user job.
194
+ 6. For dependency-free core, require no new credential, service, automatic
195
+ download or mandatory external package. Use the host's already available
196
+ capabilities or a documented inline path. Optional tooling must remain
197
+ optional in the exit criteria; lack of a renderer means NOT_RUN for visual
198
+ inspection, never an invented screenshot verdict.
199
+ 7. Test the isolated transfer's reference closure and absence of unintended
200
+ network/credential requirements. Compare actual outputs on representative
201
+ tasks with the existing setup, including no-op, ambiguity and coexistence.
202
+ An improved instruction or structural PASS is not an observed quality gain.
203
+
204
+ The transfer record contains source commit/path/digest, scope read, relevant
205
+ terms, method retained, material excluded, destination, dependency decisions,
206
+ acceptance cases and the future update owner. No external text acquires higher
207
+ instruction priority because it was fetched during this review. Do not execute
208
+ foreign installers or scripts simply to inventory them.
@@ -83,7 +83,28 @@ an unanchored regex. Tool names are what you match (`Bash`, `Write`,
83
83
  `mcp__server__tool`). A plugin's OWN MCP server is matched as
84
84
  `mcp__plugin_<plugin>_<server>__<tool>`, and an `mcp_tool` hook names the server
85
85
  as `plugin:<plugin>:<server>` — the bare key never fires. Narrow further with
86
- `if`, a permission rule: `"if": "Bash(git commit *)"`.
86
+ `if`, a permission rule: `"if": "Bash(git commit *)"` — **inside the handler
87
+ object**, beside `type` and `command`.
88
+
89
+ **Two key sets, and putting a key in the wrong one is silent.** Read out of the
90
+ 2.1.270 binary's schema: a matcher group takes `matcher` and `hooks`, nothing
91
+ else; a `command` handler takes `type`, `command`, `args`, `if`, `shell`,
92
+ `timeout`, `statusMessage`, `once`, `async`, `asyncRewake` (plus three
93
+ `@internal` keys). Anything outside its set is **ignored** — the hook still runs,
94
+ with the key doing nothing. Before 2.1.270 nothing said so at all; from it the
95
+ loader prints once per session:
96
+
97
+ ```
98
+ Plugin <name>: hooks.json: unknown key "if" in hooks.PreToolUse[1] ignored
99
+ ```
100
+
101
+ `claude plugin validate --strict` accepts the file either way, so a skill audit
102
+ is the only gate that catches this before a session starts — `HOOKS_SCHEMA` in
103
+ `scripts/audit_skill.py` reads both sets. The family paid for that check: `if`
104
+ sat beside `matcher` in `agent-sync` from 0.1.0 to 1.20.0 (harmless — its guard
105
+ narrowed to a commit with its own parser), and in the hooks template
106
+ `task-pipeline` tells projects to paste into `settings.json`, where the
107
+ documentation gate then ran on **every** Bash call instead of on commits.
87
108
 
88
109
  **Exit codes are the contract:**
89
110
 
@@ -245,6 +266,8 @@ what the agent reads at the exact moment something is missing.
245
266
  - [ ] Hook scripts exit 0 silently on "not mine" and on a missing interpreter
246
267
  - [ ] `PostToolUse` advises (`systemMessage`), `PreToolUse` blocks — not the reverse
247
268
  - [ ] Hook commands quote `"${CLAUDE_PLUGIN_ROOT}"` and set a `timeout`
269
+ - [ ] Every key is in the set its level accepts — a filter goes on the handler,
270
+ never beside `matcher`, where the host ignores it without failing
248
271
  - [ ] Hook scripts are executable, have a shebang, and need no `jq`
249
272
  - [ ] No command named after a skill — or the collision is on a recorded, dated
250
273
  exception list (see *Commands*); every `argument-hint` quoted
@@ -0,0 +1,58 @@
1
+ # Outcome evaluation — judging what the skill produced, not what it said
2
+
3
+ **Load this when:** proving a skill changed real outcomes — before a release,
4
+ after a retrofit, or when two versions disagree. The eval loop in
5
+ `references/authoring.md` says WHEN to eval (before prose, against a baseline);
6
+ this file is the METHOD: what is frozen, what is compared, and what a verdict
7
+ is allowed to mean.
8
+
9
+ ## The method, in one table
10
+
11
+ | Element | Rule |
12
+ |---|---|
13
+ | Inputs | **frozen** — the same prompts, files and fixtures for every arm; an input that drifts between arms measures the drift, not the skill |
14
+ | Arms | **baseline without the skill** and **the current skill version** — two runs, same inputs; a delta with no baseline arm is a number with no zero |
15
+ | What is judged | **actual artifacts** — the file written, the diff produced, the page rendered; never the transcript's claim that it did so ("wrote the file" in prose is not the file) |
16
+ | Actors | the host supplies the tool names — a CLI, a browser, a subagent are ways to RUN an arm, and **no specific one is mandatory**; an eval hard-wired to one host's tool cannot run anywhere else, which is a hostlock wearing a harness |
17
+
18
+ ## Three judgments, never blended
19
+
20
+ One eval row answers ONE of these, and the report keeps the axes apart:
21
+
22
+ - **Routing** — did the skill fire when it should, and stay quiet when it
23
+ should not? Judged on trigger behaviour alone; a perfect output from a skill
24
+ that fired on the wrong prompt is a routing failure with good manners.
25
+ - **Output correctness** — is the artifact right? Judged mechanically where
26
+ possible (a validator, a diff against an expected shape, an exit code), by
27
+ rubric where not.
28
+ - **Visual judgment** — does the rendered thing read well? A human-or-judge
29
+ call on the artifact's presentation. It never stands in for correctness: a
30
+ beautiful wrong answer fails, an ugly right one passes and files a note.
31
+
32
+ Blending them is how a skill "improves" on paper: one blended score lets a
33
+ routing regression hide behind a correctness win.
34
+
35
+ ## Verdicts: PASS, FAIL, ERROR, NOT_RUN
36
+
37
+ - **PASS / FAIL** — the judgment ran against the artifact and answered.
38
+ - **ERROR** — the process broke: the arm crashed, the fixture was malformed,
39
+ the judge threw. An error is its own status; folding it into FAIL blames the
40
+ skill for the harness, and folding it into PASS is fiction.
41
+ - **NOT_RUN** — the tool the case needs is absent on this host (no browser, no
42
+ subagent, no network where one is required). NOT_RUN is never PASS, and it
43
+ names what it would take to run — the same contract the accessibility and
44
+ audit doctrines hold.
45
+
46
+ An aggregate over rows inherits the weakest honesty: any ERROR or NOT_RUN in a
47
+ set means the set is not "all green", however many PASSes surround it.
48
+
49
+ ## Worked examples, one per axis
50
+
51
+ ```
52
+ axis: routing input: "подключи оплату картой" expect: stripe-billing fires
53
+ axis: routing input: "explain what a webhook is" expect: no skill fires
54
+ axis: correctness artifact: the generated SKILL.md check: audit_skill.py exits 0
55
+ axis: visual artifact: the rendered landing hero check: judge rubric §type-rhythm
56
+ ```
57
+
58
+ Four rows, three axes, and no row's verdict can move another's.
@@ -92,7 +92,139 @@ BODY_TARGET_TOKENS = 4750
92
92
  # No tokenizer in the stdlib. 3.9 chars/token is measured, not assumed: tokenizing
93
93
  # this skill's own bundle gives 3.78-4.47. `claude plugin details` is far more
94
94
  # pessimistic (~2.8) and will always show a bigger number than this estimate.
95
+ # AND the estimate is an ESTIMATE (FIX-MS-01.01): a 16 000-hieroglyph body is
96
+ # 16 000 cl100k tokens and estimates ~4 102 — so the estimate never grants a
97
+ # token PASS and is never CALLED tokens. A real, NAMED tokenizer measures;
98
+ # without one the token budget is UNMEASURED and the estimate rides beside it
99
+ # as its own field.
95
100
  CHARS_PER_TOKEN = 3.9
101
+
102
+ # Optional tokenizer adapter. `None` = unresolved; tests may inject
103
+ # `(callable, "name")` or `(None, None)` directly to pin either path.
104
+ # `MAKE_SKILL_TOKENIZER` selects a tiktoken encoding by name; an UNSUPPORTED
105
+ # name refuses to measure (a warning + UNMEASURED) — it never silently falls
106
+ # back to another encoding or to the estimate, because a verdict from the
107
+ # wrong instrument wearing the right instrument's name is worse than no
108
+ # verdict (FIX-MS-01.02).
109
+ TOKENIZER = None
110
+ DEFAULT_ENCODING = "cl100k_base"
111
+
112
+ # Who owns each threshold — `spec` is the Agent Skills standard / Anthropic's
113
+ # platform rules, `house` is this family's working rule, `host` would be a
114
+ # per-host runtime limit. A number without its authority reads as physics;
115
+ # these are policies, each negotiable only with its owner.
116
+ THRESHOLDS = {
117
+ "BODY_MAX_LINES": ("spec", 500),
118
+ "BODY_MAX_TOKENS": ("spec", 5000),
119
+ "BODY_TARGET_TOKENS": ("house", 4750),
120
+ "DESC_MAX_CHARS": ("spec", 1024),
121
+ }
122
+
123
+ # The differential corpus: pinned counts for DEFAULT_ENCODING, measured with
124
+ # tiktoken 0.14.0 (2026-09-09). Same string + same tokenizer revision must give
125
+ # the same measured count on every machine — a drift here means the adapter
126
+ # or the encoding changed, and either is a finding, never a rounding error.
127
+ TOKEN_CORPUS = {
128
+ "english": ("the quick brown fox jumps over the lazy dog", 9),
129
+ "code": ("def verify(sig, key):\n return hmac.compare_digest(sig, key)\n", 15),
130
+ "russian": ("проверка бюджета токенов выполняется настоящим токенизатором", 28),
131
+ "mixed": ("body budget: бюджет тела — 5000 tokens, не оценка", 22),
132
+ "cjk": ("字符预算不是估计值", 9),
133
+ }
134
+
135
+
136
+ def resolve_tokenizer():
137
+ global TOKENIZER
138
+ if TOKENIZER is None:
139
+ name = os.environ.get("MAKE_SKILL_TOKENIZER") or DEFAULT_ENCODING
140
+ # tiktoken caches encoding data in TMPDIR by default — residue a test
141
+ # run must not leave. A stable per-user cache, unless the operator
142
+ # already chose one.
143
+ os.environ.setdefault("TIKTOKEN_CACHE_DIR", os.path.join(
144
+ os.path.expanduser("~"), ".cache", "make-skill", "tiktoken"))
145
+ try:
146
+ import tiktoken
147
+ enc = tiktoken.get_encoding(name)
148
+ TOKENIZER = (lambda text: len(enc.encode(text)), f"tiktoken:{name}")
149
+ except Exception as exc:
150
+ if os.environ.get("MAKE_SKILL_TOKENIZER"):
151
+ print(f"audit: tokenizer {name!r} is unsupported ({exc}) — token "
152
+ "budgets are UNMEASURED, not silently re-measured with a "
153
+ "different encoding", file=sys.stderr)
154
+ TOKENIZER = (None, None)
155
+ return TOKENIZER
156
+
157
+
158
+ def corpus_check():
159
+ """The adapter against the pinned oracle. Returns (verdict, detail):
160
+ 'agree' when every sample matches its pinned count, 'NOT_RUN' without a
161
+ tokenizer, 'DISAGREE' naming the first divergent sample."""
162
+ count_fn, tok_name = resolve_tokenizer()
163
+ if count_fn is None:
164
+ return "NOT_RUN", "no tokenizer installed — the differential did not run"
165
+ if tok_name != f"tiktoken:{DEFAULT_ENCODING}":
166
+ return "NOT_RUN", (f"corpus counts are pinned for {DEFAULT_ENCODING}; "
167
+ f"{tok_name} is a different instrument")
168
+ for name, (sample, pinned) in sorted(TOKEN_CORPUS.items()):
169
+ got = count_fn(sample)
170
+ if got != pinned:
171
+ return "DISAGREE", (f"{name}: adapter says {got}, the pinned oracle "
172
+ f"says {pinned} — same string, same revision, "
173
+ "different count")
174
+ return "agree", f"{len(TOKEN_CORPUS)} samples agree with {tok_name}"
175
+
176
+
177
+ # Optional FULL-YAML conformance adapter (FIX-MS-02.02). The strict subset
178
+ # parser is the always-available precheck; where a real YAML parser is ALSO
179
+ # installed, it is used as an ORACLE — the subset parse is compared against it
180
+ # on quoted scalars, escapes and multiline forms, and a divergence is a
181
+ # reported finding. Without the parser there is NO PASS of a full-YAML check:
182
+ # the verdict is NOT_RUN, never a green tick, and malformed input is always an
183
+ # error rather than a silent pass.
184
+ YAML_PARSER = None
185
+
186
+
187
+ def resolve_yaml_parser():
188
+ global YAML_PARSER
189
+ if YAML_PARSER is None:
190
+ try:
191
+ import yaml
192
+ YAML_PARSER = (lambda s: yaml.safe_load(s), "pyyaml")
193
+ except Exception:
194
+ YAML_PARSER = (None, None)
195
+ return YAML_PARSER
196
+
197
+
198
+ def yaml_conformance(frontmatter_text):
199
+ """Compare the strict-subset parse against a full YAML parser, where one is
200
+ installed. Returns (verdict, detail): 'agree' | 'DIVERGES: …' |
201
+ 'MALFORMED: …' | 'NOT_RUN: …' (no parser). The subset parse never scores
202
+ a full-YAML PASS on its own."""
203
+ parse, name = resolve_yaml_parser()
204
+ if parse is None:
205
+ return "NOT_RUN", ("no full YAML parser installed — the strict subset "
206
+ "precheck ran, but full-YAML conformance is unproven")
207
+ try:
208
+ real = parse(frontmatter_text)
209
+ except Exception as exc: # noqa: BLE001 — any parse error
210
+ return "MALFORMED", f"the real parser rejects this frontmatter: {exc}"
211
+ if not isinstance(real, dict):
212
+ return "MALFORMED", "frontmatter is not a mapping"
213
+ subset, _lines = parse_frontmatter(frontmatter_text)
214
+ diffs = []
215
+ for k in set(real) | set(subset):
216
+ rv, sv = real.get(k), subset.get(k)
217
+ if isinstance(rv, dict) and isinstance(sv, dict):
218
+ for kk in set(rv) | set(sv):
219
+ if rv.get(kk) != sv.get(kk):
220
+ diffs.append(f"{k}.{kk}: subset {sv.get(kk)!r} vs {name} {rv.get(kk)!r}")
221
+ elif rv != sv:
222
+ diffs.append(f"{k}: subset {sv!r} vs {name} {rv!r}")
223
+ if diffs:
224
+ return "DIVERGES", "; ".join(sorted(diffs))
225
+ return "agree", f"the subset parse matches {name}"
226
+
227
+
96
228
  TOC_MIN_LINES = 100 # Anthropic: longer reference files need a table of contents
97
229
 
98
230
  SPEC_KEYS = {"name", "description", "license", "compatibility", "metadata", "allowed-tools"}
@@ -115,6 +247,32 @@ TIME_BRANCH_RE = re.compile(
115
247
  r"(?:january|february|march|april|may|june|july|august|september|october|"
116
248
  r"november|december|\d{4})\b", re.I)
117
249
  BUNDLE_DIRS = ("references", "scripts", "assets")
250
+ # A byte-compile cache is not a bundled directory: `__pycache__` appears the moment a
251
+ # test imports a shipped module, is gitignored and npm-excluded in every member, and
252
+ # three of them were reporting BUNDLE_NESTED for a directory that never ships.
253
+ BUNDLE_IGNORE = ("__pycache__",)
254
+
255
+ # Claude Code's hooks schema, read out of the 2.1.270 binary rather than from the
256
+ # documentation: `Rt()` is the matcher GROUP and `Td()` the handler union. A key outside
257
+ # these sets is IGNORED — silently before 2.1.270, and from it announced once per session
258
+ # as `Plugin <name>: hooks.json: unknown key "<k>" in hooks.<Event>[<i>] ignored`. It is
259
+ # not a syntax error, `claude plugin validate --strict` passes the file, and the hook goes
260
+ # on running with the filter absent. agent-sync shipped `"if"` beside `"matcher"` from
261
+ # 0.1.0 to 1.20.0 and task-pipeline exported the same shape in the template it tells
262
+ # projects to paste into `settings.json`, where the gate then ran on every Bash call.
263
+ HOOK_GROUP_KEYS = {"matcher", "hooks"}
264
+ # The `command` handler. `if` is HERE — beside `type` and `command`, never beside
265
+ # `matcher`. The last three are marked `@internal` in the schema and are accepted.
266
+ HOOK_COMMAND_KEYS = {
267
+ "type", "command", "args", "if", "shell", "timeout", "statusMessage", "once",
268
+ "async", "asyncRewake", "rewakeMessage", "rewakeSummary", "cloud",
269
+ }
270
+
271
+ # A file in a publishable payload whose name reads as a credential. Fixtures and
272
+ # test data are the common carriers; the auditor names it rather than shipping it.
273
+ DIST_SECRET_RE = re.compile(
274
+ r"(?i)(?:^|[._/-])(?:secret|secrets|token|password|passwd|api[_-]?key|apikey|"
275
+ r"credential|credentials|\.env|id_rsa|id_ed25519|private[_-]?key)(?:$|[._/-])")
118
276
 
119
277
 
120
278
  class Audit:
@@ -142,11 +300,23 @@ class Audit:
142
300
 
143
301
 
144
302
  def parse_frontmatter(text):
145
- """YAML subset: top-level scalars, block scalars, one nested map.
146
-
147
- Returns (data, line_of_key). A full YAML parser is not in the stdlib and a
148
- skill's frontmatter is a flat map by specification, so this is enough — and
149
- it keeps the script dependency-free, which is the point of shipping it.
303
+ """A STRICT stdlib precheck of an explicitly bounded YAML subset — NOT full
304
+ YAML, and it does not pretend to be (FIX-MS-02.01).
305
+
306
+ The supported subset: top-level scalars (quoted or plain), block scalars
307
+ (`>`/`|` and their chomping variants), inline flow sequences (`[a, b]`),
308
+ and ONE nested map. Everything else is out of subset and must be caught,
309
+ never waved through — a precheck that silently accepts what the real
310
+ uploader rejects is worse than no precheck.
311
+
312
+ Returns (data, line_of_key). A bare scalar that a real YAML parser would
313
+ COERCE keeps its type here too: `version: 1.0` becomes the float 1.0, not
314
+ the string "1.0", so a field the contract requires to be a string
315
+ (metadata values, name, description) is caught by its own type check
316
+ instead of passing as a stringified number. Quote it and it stays a
317
+ string. A full YAML parser is not in the stdlib and a skill's frontmatter
318
+ is a flat map by specification, so this is enough — and it keeps the
319
+ script dependency-free, which is the point of shipping it.
150
320
 
151
321
  A plain scalar may continue on indented lines and YAML folds them into one
152
322
  value with a space. Dropping those lines is how a description whose real
@@ -154,8 +324,11 @@ def parse_frontmatter(text):
154
324
  and the 970 working limit — a clean bill from the family's standard-keeper
155
325
  for a skill the Skills API rejects on upload (2026-08-16, B-63).
156
326
  """
327
+ parse_frontmatter.last_duplicates = []
157
328
  data, lines, key, mode = {}, {}, None, None
158
329
  scalars = set()
330
+ duplicates = [] # (scope, key) pairs seen more than once
331
+ nested_seen = set()
159
332
  for i, raw in enumerate(text.split("\n"), start=2): # +2: the opening '---'
160
333
  if not raw.strip():
161
334
  continue
@@ -165,6 +338,9 @@ def parse_frontmatter(text):
165
338
  key, mode = None, None
166
339
  continue
167
340
  key, val = m.group(1), m.group(2).strip()
341
+ if key in data:
342
+ duplicates.append(("top", key))
343
+ nested_seen = set()
168
344
  lines[key] = i
169
345
  if val in (">", "|", ">-", "|-", ">+", "|+"):
170
346
  data[key], mode = "", "block"
@@ -184,12 +360,38 @@ def parse_frontmatter(text):
184
360
  elif mode == "map":
185
361
  m = re.match(r"^\s+([A-Za-z0-9_-]+):\s*(.*)$", raw)
186
362
  if m:
187
- data[key][m.group(1)] = _unquote(m.group(2).strip())
363
+ if m.group(1) in nested_seen:
364
+ duplicates.append((key, m.group(1)))
365
+ nested_seen.add(m.group(1))
366
+ data[key][m.group(1)] = _typed_scalar(m.group(2).strip())
188
367
  for k in scalars:
189
368
  data[k] = _finish_scalar(data[k])
369
+ parse_frontmatter.last_duplicates = duplicates
190
370
  return data, lines
191
371
 
192
372
 
373
+ def _typed_scalar(v):
374
+ """A bare scalar keeps the TYPE a real YAML parser would give it; a quoted
375
+ one stays a string. This is what lets a string-required field notice that
376
+ `1.0` is a float and `true` is a bool (FIX-MS-02.01)."""
377
+ v = v.strip()
378
+ if len(v) >= 2 and v[0] == v[-1] and v[0] in "\"'":
379
+ return v[1:-1] # explicitly quoted → string
380
+ if v in ("true", "false", "True", "False"):
381
+ return v.lower() == "true"
382
+ if v in ("null", "~", "Null", "NULL", ""):
383
+ return None if v != "" else ""
384
+ if re.match(r"^[+-]?\d+$", v):
385
+ return int(v)
386
+ if re.match(r"^[+-]?(\d+\.\d*|\.\d+|\d+)([eE][+-]?\d+)?$", v) and \
387
+ re.search(r"[.eE]", v):
388
+ try:
389
+ return float(v)
390
+ except ValueError:
391
+ return v
392
+ return v
393
+
394
+
193
395
  def _finish_scalar(v):
194
396
  """A flow sequence is a LIST, not a string that happens to look like one.
195
397
 
@@ -236,13 +438,38 @@ def audit(skill_dir, house=False):
236
438
  fm, fm_lines = parse_frontmatter(m.group(1))
237
439
  body = text[m.end():]
238
440
 
441
+ dups = getattr(parse_frontmatter, "last_duplicates", [])
442
+ if dups:
443
+ named = ", ".join(k if scope == "top" else f"{scope}.{k}" for scope, k in dups)
444
+ a.gap("FM_DUPLICATE_KEY", "duplicate frontmatter key(s): %s — a real YAML "
445
+ "parser rejects or last-wins them; either way the precheck must not "
446
+ "pass two values for one key" % named, rel)
447
+ else:
448
+ a.ok("FM_DUPLICATE_KEY", "no duplicate frontmatter keys", rel)
449
+
450
+ yv, yd = yaml_conformance(m.group(1))
451
+ if yv == "DIVERGES":
452
+ a.gap("FM_YAML_CONFORMANCE", "the strict-subset parse diverges from the "
453
+ "installed YAML parser: %s — the uploader will read what the real "
454
+ "parser reads, not the subset" % yd, rel)
455
+ elif yv == "MALFORMED":
456
+ a.gap("FM_YAML_CONFORMANCE", "malformed frontmatter: %s" % yd, rel)
457
+ elif yv == "NOT_RUN":
458
+ a.ok("FM_YAML_CONFORMANCE", "full-YAML conformance NOT_RUN (%s) — the "
459
+ "strict subset precheck ran; install PyYAML to prove conformance" % yd, rel)
460
+ else:
461
+ a.ok("FM_YAML_CONFORMANCE", "the subset parse matches the installed "
462
+ "YAML parser", rel)
463
+
239
464
  _check_name(a, fm, fm_lines, name_on_disk, rel)
240
465
  _check_description(a, fm, fm_lines, rel, house)
241
466
  _check_optional_fields(a, fm, fm_lines, rel)
242
467
  _check_keys(a, fm, fm_lines, rel)
243
468
  _check_body_budget(a, body, rel, house)
244
469
  _check_bundle(a, skill_dir, text, name_on_disk)
470
+ _check_plugin_hooks(a, skill_dir)
245
471
  _check_links(a, skill_dir, text, rel)
472
+ _check_distribution(a, skill_dir, name_on_disk)
246
473
  _check_prose(a, body, rel)
247
474
  return a
248
475
 
@@ -368,32 +595,53 @@ def _check_keys(a, fm, lines, rel):
368
595
 
369
596
  def _check_body_budget(a, body, rel, house=False):
370
597
  n_lines = body.count("\n") + 1
598
+ count_fn, tok_name = resolve_tokenizer()
371
599
  est = int(len(body) / CHARS_PER_TOKEN)
372
600
  # Both, not either: a body over the line budget still has to report its
373
- # token count, or the second fix arrives only after the first one ships.
601
+ # token verdict, or the second fix arrives only after the first one ships.
374
602
  over = False
375
603
  if n_lines >= BODY_MAX_LINES:
376
604
  a.gap("BODY_LINES", "body is %d lines, the budget is < %d — move detail into "
377
605
  "references/" % (n_lines, BODY_MAX_LINES), rel)
378
606
  over = True
379
- if est >= BODY_MAX_TOKENS:
380
- a.gap("BODY_TOKENS", "body is ~%d tokens (%d chars / %s), the budget is < %d"
381
- % (est, len(body), CHARS_PER_TOKEN, BODY_MAX_TOKENS), rel)
607
+ if count_fn is None:
608
+ # No adapter, no token verdict: the budget is UNMEASURED, and the
609
+ # byte/char estimate is reported as an ESTIMATE — it is not tokens,
610
+ # it cannot PASS the budget, and it cannot fail it either (a
611
+ # 16k-hieroglyph body estimates ~4k and measures 16k).
612
+ a.ok("BODY_TOKENS_UNMEASURED",
613
+ "token budget UNMEASURED — no tokenizer installed; the estimate "
614
+ "~%d (%d chars / %s) is an estimate, NOT tokens: install tiktoken "
615
+ "to measure" % (est, len(body), CHARS_PER_TOKEN), rel)
616
+ if not over:
617
+ a.ok("BODY_BUDGET", "body is %d lines (budget %d); token budget "
618
+ "unmeasured" % (n_lines, BODY_MAX_LINES), rel)
619
+ if house:
620
+ a.ok("BODY_HEADROOM_UNMEASURED",
621
+ "the %d-token working limit needs a measurement — unmeasured, "
622
+ "not passed" % BODY_TARGET_TOKENS, rel)
623
+ return
624
+ measured = count_fn(body)
625
+ if measured >= BODY_MAX_TOKENS:
626
+ a.gap("BODY_TOKENS", "body is %d tokens (%s), the budget is < %d"
627
+ % (measured, tok_name, BODY_MAX_TOKENS), rel)
382
628
  over = True
383
629
  if not over:
384
- a.ok("BODY_BUDGET", "body is %d lines / ~%d tokens (budget %d / %d)"
385
- % (n_lines, est, BODY_MAX_LINES, BODY_MAX_TOKENS), rel)
630
+ a.ok("BODY_BUDGET", "body is %d lines / %d tokens (%s; budget %d / %d)"
631
+ % (n_lines, measured, tok_name, BODY_MAX_LINES, BODY_MAX_TOKENS), rel)
386
632
  # The house half, and it is the same rule DESC_HEADROOM applies to the other
387
633
  # field: a body at the ceiling cannot absorb the next paragraph, so it gets
388
- # absorbed into a reference that should have been split instead.
389
- if house and not over and est >= BODY_TARGET_TOKENS:
390
- a.gap("BODY_HEADROOM", "body is ~%d tokens — inside the %d budget but past the "
391
- "%d working limit (house rule): the next section will breach it, and the "
392
- "answer then is a split, not a trim"
393
- % (est, BODY_MAX_TOKENS, BODY_TARGET_TOKENS), rel)
634
+ # absorbed into a reference that should have been split instead. House
635
+ # thresholds ride the MEASURED count only — a house rule on an estimate is
636
+ # a verdict on the instrument.
637
+ if house and not over and measured >= BODY_TARGET_TOKENS:
638
+ a.gap("BODY_HEADROOM", "body is %d tokens (%s) — inside the %d budget but "
639
+ "past the %d working limit (house rule): the next section will "
640
+ "breach it, and the answer then is a split, not a trim"
641
+ % (measured, tok_name, BODY_MAX_TOKENS, BODY_TARGET_TOKENS), rel)
394
642
  elif house and not over:
395
- a.ok("BODY_HEADROOM", "body is ~%d/%d tokens, inside the working limit"
396
- % (est, BODY_TARGET_TOKENS), rel)
643
+ a.ok("BODY_HEADROOM", "body is %d/%d tokens (%s), inside the working limit"
644
+ % (measured, BODY_TARGET_TOKENS, tok_name), rel)
397
645
 
398
646
 
399
647
  def _bundle_closure(skill_dir, skill_text):
@@ -454,6 +702,56 @@ def _bundle_closure(skill_dir, skill_text):
454
702
  return seen
455
703
 
456
704
 
705
+ def package_closure(payload_files, required, optional):
706
+ """Whether a PACKAGED payload resolves its references (PXS-06 / ED-01.02).
707
+
708
+ `payload_files` is the set of paths actually present in what would ship —
709
+ the packaged copy, not the whole checkout, because a checkout resolves a
710
+ ref through a neighbour the package leaves behind. A REQUIRED reference
711
+ missing from the payload FAILS closure; an OPTIONAL one missing is reported
712
+ as optional, never promoted to required (an external optional reference is
713
+ not the package's to carry).
714
+ """
715
+ present = set(payload_files)
716
+ missing_required = sorted(r for r in required if r not in present)
717
+ missing_optional = sorted(o for o in optional if o not in present)
718
+ return {"ok": not missing_required,
719
+ "missing_required": missing_required,
720
+ "optional_unavailable": missing_optional}
721
+
722
+
723
+ def _check_distribution(a, skill_dir, dir_name):
724
+ """The publishable payload carries no outward symlink and no undeclared secret.
725
+
726
+ A symlink pointing outside the skill directory resolves on the author's
727
+ machine and dangles (or leaks) everywhere else; a file whose name reads as
728
+ a credential is a secret nobody declared. Neither belongs in what ships.
729
+ """
730
+ root = os.path.realpath(skill_dir)
731
+ found_symlink = found_secret = False
732
+ for base, dirs, files in os.walk(skill_dir):
733
+ for entry in list(dirs) + files:
734
+ full = os.path.join(base, entry)
735
+ rel = os.path.relpath(full, skill_dir)
736
+ if os.path.islink(full):
737
+ target = os.path.realpath(full)
738
+ if not (target == root or target.startswith(root + os.sep)):
739
+ found_symlink = True
740
+ a.gap("DIST_SYMLINK_ESCAPE",
741
+ "%s is a symlink pointing outside the skill directory — it "
742
+ "dangles or leaks when the payload ships without its target"
743
+ % rel, os.path.join(dir_name, rel))
744
+ if os.path.isfile(full) and DIST_SECRET_RE.search(rel):
745
+ found_secret = True
746
+ a.gap("DIST_UNDECLARED_SECRET",
747
+ "%s reads as a credential — a secret must not ride in a "
748
+ "publishable payload; exclude it (.npmignore / files allowlist)"
749
+ % rel, os.path.join(dir_name, rel))
750
+ if not found_symlink and not found_secret:
751
+ a.ok("DIST_PAYLOAD", "publishable payload carries no outward symlink or "
752
+ "undeclared secret", dir_name)
753
+
754
+
457
755
  def _check_bundle(a, skill_dir, skill_text, dir_name):
458
756
  """references/ scripts/ assets/: one level deep, reachable, navigable."""
459
757
  reachable = _bundle_closure(skill_dir, skill_text)
@@ -464,6 +762,8 @@ def _check_bundle(a, skill_dir, skill_text, dir_name):
464
762
  for entry in sorted(os.listdir(d)):
465
763
  full = os.path.join(d, entry)
466
764
  rel = os.path.join(dir_name, sub, entry)
765
+ if entry in BUNDLE_IGNORE:
766
+ continue
467
767
  if os.path.isdir(full):
468
768
  a.gap("BUNDLE_NESTED", "%s/%s/ is nested — keep bundled files one level "
469
769
  "deep, or the agent previews them instead of reading them"
@@ -485,6 +785,84 @@ def _check_bundle(a, skill_dir, skill_text, dir_name):
485
785
  a.ok("BUNDLE_LAYOUT", "bundled directories present and checked")
486
786
 
487
787
 
788
+ def _check_plugin_hooks(a, skill_dir):
789
+ """The plugin's `hooks.json` carries only keys the host evaluates.
790
+
791
+ A skill inside a Claude Code plugin sits at `<plugin>/skills/<name>`, so the hooks
792
+ manifest is two levels up. This runs once per skill, which means a defective manifest
793
+ is reported once per skill of that plugin — a repetition, never a miss.
794
+
795
+ Reported as a GAP rather than a note because the failure is silent in both
796
+ directions: the key does nothing, and nothing said so for six weeks across two
797
+ repositories. `claude plugin validate --strict` accepts the file, so a skill audit is
798
+ the only gate this class can be caught by before a session start.
799
+ """
800
+ d = os.path.abspath(skill_dir.rstrip("/"))
801
+ parent = os.path.dirname(d)
802
+ if os.path.basename(parent) != "skills":
803
+ a.ok("HOOKS_SCHEMA", "not a plugin skill layout (<plugin>/skills/<name>) — no "
804
+ "hooks manifest to read from here")
805
+ return
806
+ plugin = os.path.dirname(parent)
807
+ path = os.path.join(plugin, "hooks", "hooks.json")
808
+ rel = os.path.join(os.path.basename(plugin), "hooks", "hooks.json")
809
+ if not os.path.isfile(path):
810
+ a.ok("HOOKS_SCHEMA", "the plugin ships no hooks/hooks.json — nothing to read", rel)
811
+ return
812
+ try:
813
+ data = json.load(open(path, encoding="utf-8"))
814
+ except Exception as e:
815
+ a.gap("HOOKS_SCHEMA", "hooks.json does not parse (%s) — Claude Code loads no "
816
+ "hooks at all from a manifest it cannot read" % e, rel)
817
+ return
818
+ events = data.get("hooks")
819
+ if not isinstance(events, dict):
820
+ a.gap("HOOKS_SCHEMA", "hooks.json has no `hooks` object — the manifest declares "
821
+ "nothing the host can run", rel)
822
+ return
823
+ bad = []
824
+ unchecked = set()
825
+ for event, groups in sorted(events.items()):
826
+ if not isinstance(groups, list):
827
+ bad.append("hooks.%s is not a list of matcher groups" % event)
828
+ continue
829
+ for i, group in enumerate(groups):
830
+ if not isinstance(group, dict):
831
+ bad.append("hooks.%s[%d] is not an object" % (event, i))
832
+ continue
833
+ for k in sorted(set(group) - HOOK_GROUP_KEYS):
834
+ bad.append('hooks.%s[%d]: %r sits beside `matcher`, where a matcher group '
835
+ 'takes only matcher+hooks — the host ignores it (a filter '
836
+ 'belongs inside the handler)' % (event, i, k))
837
+ handlers = group.get("hooks")
838
+ if not isinstance(handlers, list):
839
+ bad.append("hooks.%s[%d].hooks is not a list of handlers" % (event, i))
840
+ continue
841
+ for j, h in enumerate(handlers):
842
+ if not isinstance(h, dict):
843
+ bad.append("hooks.%s[%d].hooks[%d] is not an object" % (event, i, j))
844
+ continue
845
+ if h.get("type") != "command":
846
+ unchecked.add(str(h.get("type")))
847
+ continue
848
+ for k in sorted(set(h) - HOOK_COMMAND_KEYS):
849
+ bad.append("hooks.%s[%d].hooks[%d]: %r is not a key the command-hook "
850
+ "schema knows — the host ignores it" % (event, i, j, k))
851
+ for msg in bad:
852
+ a.gap("HOOKS_SCHEMA", msg, rel)
853
+ if not bad:
854
+ note = ""
855
+ if unchecked:
856
+ # Say what was NOT read. The command handler's key set was measured; the
857
+ # prompt/agent/http/mcp_tool handlers were not, and claiming them clean would
858
+ # be a verdict about something this check never looked at.
859
+ note = " (handler type(s) %s not checked — only the command handler's key "
860
+ note %= ", ".join(sorted(unchecked))
861
+ note += "set is measured here)"
862
+ a.ok("HOOKS_SCHEMA", "every hooks.json key is one Claude Code evaluates%s" % note,
863
+ rel)
864
+
865
+
488
866
  def _check_links(a, skill_dir, skill_text, rel):
489
867
  """A relative link that escapes the skill directory arrives broken everywhere.
490
868