@ssheleg/make-skill 0.25.0 → 0.25.2

package/CHANGELOG.md

@@ -1,5 +1,88 @@
 # Changelog
 
+## v0.25.2 — the evals have been run, and the discovery check runs somewhere
+
+- **MSK-01 closed: the evaluation suite was executed against two models, and the
+  numbers live in `test/evals/RESULTS.md` as a dated run (2026-08-31), not a
+  claim.** All 22 trigger queries probed blind per model — haiku 22/22, sonnet
+  20/22 (q01 and q10 answered "none": under-triggering on positives, ZERO false
+  positives on the near-miss negatives across both models) — and all four
+  behavioural scenarios executed and scored per `expected_behavior` line (sonnet
+  31/35, haiku 24/35; the s01 repositories were re-audited with the bundled
+  auditor rather than trusted from the agent's report). The Method section
+  states the harness and its limits: one probe per query instead of the
+  README's fired/3, Agent-tool subagents on the author's machine whose global
+  config carries the family routing map (a bias that makes the clean negatives
+  the strongest result), model aliases rather than exact ids, and two probes
+  counted with a stated non-compliance caveat. Headline finding: the standing
+  worry (turn-stealing) did not materialize; the observed failure is sonnet
+  under-triggering on two positive categories, and s03-haiku answered a
+  surfaces question without consulting the installed skill — selection and
+  consultation measured apart, on purpose.
+- **MSK-03 closed: the discovery check the body demands now runs in CI.**
+  SKILL.md's stray-SKILL.md gotcha ends "verify with the skills CLI's `--list`"
+  — and nothing ran it anywhere: the operator machine's hygiene guard refuses
+  the bare skills CLI on family members by design, and CI never asked. The new
+  `skills-cli-discovery` job in `validate.yml` runs the listing against
+  `ssheleg/make-skill` and asserts it serves exactly ONE skill, named
+  make-skill, with ANSI stripped before asserting. Its own job, like
+  `claude-plugin-validate`, so a registry outage cannot mask an
+  offline-validator failure; it reads the repository's default branch, so the
+  post-merge push run and the release's `workflow_call` on the tag assert the
+  published tree.
+- **MSK-04 closed: body headroom restored by displacement — the words moved,
+  none were lost.** The Cyrillic budget measurement (1.9–2.3 chars/token,
+  3408→1885) moved to `authoring.md` → *Content guidelines*; the six skeleton
+  filenames moved into `distribution.md`'s layout tree; the five verified facts
+  and the machine-refresh command sequence are now referenced at their single
+  home in `distribution.md`'s checklists instead of being restated; the
+  long-file rule already lived in `authoring.md` and the body now points there.
+  Re-measured with the bundled auditor: body ~4742/4750 tokens (8 spare) →
+  ~4677/4750 (73 spare), after also paying for the next bullet. The
+  description is untouched at 965/970 — its headroom is spent only on trigger
+  changes, and this release makes none.
+- **XF-10: the MCP/A2A carve-out now runs both directions at the layer an agent
+  reads first.** The References-table rows for `mcp.md` and `a2a.md` carry "the
+  protocol wire itself → `agent-interop` (agent-stack)" — the two reference
+  files' own headers already said it, and now the router-level table says it
+  too, so a reader deciding what to load learns the boundary before opening
+  either file.
+- Also: `test/evals/README.md`'s file table said "20 realistic queries" over a
+  `triggers.json` holding 22 — the same drift class as MSK-02, in a wording no
+  guard could parse. Reworded to the checkable "22 trigger queries" form, which
+  puts the sentence under `test/validate.py`'s counted-claims sweep instead of
+  under anyone's memory. `SKILL-CARD.md`'s evaluation status and posture
+  sections now describe the executed run and the replication still owed,
+  instead of "never executed".
+
+## v0.25.1 — the rule keeps its default, and the family's exception is written down
+
+- **The same-name command rule now carries its recorded exception** (operator
+  decision, 2026-08-30: no renames anywhere in the family). The default in
+  `references/host-capabilities.md` is unchanged — never name a command after a
+  skill in the same plugin — and the one recorded exception is enumerated beside
+  it: the ssheleg family ships same-named commands deliberately (`task-pipeline`,
+  `project-audit`, `seo-aeo-audit`, `sheleg-design`, `agent-sync`, and
+  `super-ux`'s `vision` / `ux-foundation` / `ux-flows` / `ux-audit`), with the
+  cost stated rather than waived: the skill wins the trigger, and each command
+  stays an always-on token cost that may be unreachable in the picker.
+  `retrofit.md` item 14 and the Cursor rule teach the auditor the same path, so
+  family audits report ASY-02 / SEO-02 / TPA-02 / SHD-06 / SUX-03 as
+  *deliberate, recorded — no change needed* instead of open gaps. A collision
+  NOT on a dated, recorded exception list is still the gap the rule names.
+- **A hand-typed eval count drifted, exactly as this skill's own gotcha predicts**
+  (MSK-02). `test/evals/RESULTS.md` said "20 queries" and `SKILL-CARD.md` said
+  "20 trigger queries" over a `triggers.json` holding 22. Both corrected — and
+  both statements are now compared, not typed: `test/evals_validate.py` parses
+  RESULTS.md's stated counts against the artifacts and refuses a claim-free
+  RESULTS.md rather than passing vacuously, with negative self-tests planting an
+  off-by-one count and a claim-free file; `test/validate.py`'s counted-claims
+  sweep gains the `N trigger queries` / `N behavioural scenarios` patterns for
+  every other document. Both guards were watched failing against the real
+  20-vs-22 defect before the numbers were corrected.
+- The `SKILL.md` body is untouched on purpose: at ~4742/5000 tokens it has 8
+  tokens of headroom, so the whole amendment lives in the references.
+
 ## v0.25.0 — the installer refuses the shadow it documents, loudly
 
 - **The family audit of 2026-08-29 reproduced the shadow live:** a bare

package/cursor/rules/make-skill.mdc

@@ -140,7 +140,10 @@ through a quoted `"${CLAUDE_PLUGIN_ROOT}/…"` with a `timeout`; hook scripts ex
 0 silently when the event is not theirs and when the interpreter is missing;
 `PreToolUse` blocks (exit 2 sends stderr to the model), `PostToolUse` advises via
 `systemMessage`; plugin agents may not carry `hooks`, `mcpServers` or
-`permissionMode`; a command is never named after a skill in the same plugin.
+`permissionMode`; a command is never named after a skill in the same plugin —
+unless the collision is on the recorded, dated exception list in the skill's
+`references/host-capabilities.md` (the ssheleg family, operator decision
+2026-08-30), which an audit reports as deliberate and recorded, not as a gap.
 
 ## Hard rules
 

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@ssheleg/make-skill",
-  "version": "0.25.0",
-  "description": "Create, retrofit, audit, and ship agent skills & Claude Code plugins the proven ssheleg way — conformance to the Agent Skills open standard AND Anthropic's platform rules (front-matter limits, disclosure budgets, per-surface runtime limits, the Skills API, evals) plus the Claude Code plugin reference (manifest schemas, component layout, claude plugin validate --strict), marketplace repo layout, version sync, validator + CI, multi-channel distribution (plugin, vercel skills CLI, npx, Cursor), npm gotchas, the review checklist for third-party skills, and MCP / A2A rules for protocol-connected skills. This package is the installer CLI.",
+  "version": "0.25.2",
+  "description": "Create, retrofit, audit, and ship agent skills & Claude Code plugins the proven ssheleg way \u2014 conformance to the Agent Skills open standard AND Anthropic's platform rules (front-matter limits, disclosure budgets, per-surface runtime limits, the Skills API, evals) plus the Claude Code plugin reference (manifest schemas, component layout, claude plugin validate --strict), marketplace repo layout, version sync, validator + CI, multi-channel distribution (plugin, vercel skills CLI, npx, Cursor), npm gotchas, the review checklist for third-party skills, and MCP / A2A rules for protocol-connected skills. This package is the installer CLI.",
   "keywords": [
     "skill",
     "plugin",

package/plugins/make-skill/.claude-plugin/plugin.json

@@ -3,7 +3,7 @@
   "name": "make-skill",
   "displayName": "Make Skill",
   "description": "Create, retrofit, audit, and ship agent skills & Claude Code plugins the proven ssheleg way: conformance to the Agent Skills open standard, Anthropic's platform rules (surfaces, Skills API, evals) and the Claude Code plugin reference, marketplace repo layout, version sync, validator + CI, multi-channel distribution (plugin, vercel skills CLI, npx, Cursor), npm gotchas, end-to-end first publish, the review checklist for third-party skills, plus MCP / A2A references for protocol-connected skills.",
-  "version": "0.25.0",
+  "version": "0.25.2",
   "author": {
     "name": "ssheleg",
     "url": "https://x.com/sshlg93"

package/plugins/make-skill/skills/make-skill/SKILL.md

@@ -5,7 +5,7 @@ license: MIT
 compatibility: Authoring works on any agent. The bundled scripts/ need python3. Publishing steps need git, gh, node and npm; the plugin gates need the claude CLI. Not usable on the Claude API surface, which has no network and no runtime package install.
 metadata:
   author: ssheleg
-  version: "0.25.0"
+  version: "0.25.2"
   homepage: https://github.com/ssheleg/make-skill
 ---
 
@@ -27,8 +27,8 @@ orchestrator, release automation). **make-skill itself** is built to this canon.
 | `references/host-capabilities.md` | shipping a **hook, subagent, command, script or MCP dependency** — what each buys and costs, hook events and exit codes, the degradation clauses |
 | `references/claude-code-plugin.md` | anything shipping as a **Claude Code plugin/marketplace** — manifest schemas, component layout, path variables, `validate` failures |
 | `references/distribution.md` | the repo layout, releases, and all five channels — plugin, skills CLI, npx, Cursor, umbrella family repo |
-| `references/mcp.md` | skill vs **MCP** server, declaring the dependency, consent and untrusted-output rules |
-| `references/a2a.md` | the skill spans two autonomous agents (**A2A**) — choosing it, the two meanings of "skill", driving a peer safely |
+| `references/mcp.md` | skill vs **MCP** server, declaring the dependency, consent and untrusted-output rules; the protocol wire itself → `agent-interop` (agent-stack) |
+| `references/a2a.md` | the skill spans two autonomous agents (**A2A**) — choosing it, the two meanings of "skill", driving a peer safely; the wire itself → `agent-interop` (agent-stack) |
 
 Missing from this copy? Raw fallback:
 `raw.githubusercontent.com/ssheleg/make-skill/main/plugins/make-skill/skills/make-skill/references/<file>`
@@ -81,8 +81,8 @@ differences and the checklist: `references/agent-skills-spec.md`.
   99% of budget turns the next correction into a fight with the validator.
   Heavier material goes to `references/`, `scripts/`, `assets/` INSIDE the skill
   dir, one level deep, each linked from the body with a stated load trigger
-  ("read X when Y") — never a bare "see references/". Reference files >100 lines
-  open with a `## Contents` list: a partial `head` read is what agents get.
+  ("read X when Y") — never a bare "see references/". Long-file and layout rules:
+  `references/authoring.md` → *Content guidelines*.
 - Gotchas stay in `SKILL.md`: the agent can't know to open a file about a trap
   it doesn't know exists.
 - **Write for the weakest surface you claim** (`references/surfaces.md`): the
@@ -97,9 +97,8 @@ House additions on top of the spec:
   survives in four places only, where the string itself is the point: a
   **trigger phrase**, a **refusal phrase** (the operator types both — translated,
   they no longer match what was said), a **proper noun**, a **language example**
-  (`«вы»/«ты»`). A budget rule before a style one: Russian encodes at 1.9–2.3
-  chars/token against English's 5.0 (`cl100k`), and rewriting the eight ssheleg
-  routers into English cut them **3408 → 1885 tokens** with no loss of meaning.
+  (`«вы»/«ты»`). A budget rule before a style one — the measured cost of
+  Cyrillic prose is in `references/authoring.md` → *Content guidelines*.
 - `description` starts "Use when …" and lists concrete trigger phrases — English
   AND Russian (user works in both). A skill nobody triggers is dead weight. Hold
   **5% headroom here too** (≤970 of 1024): a near-miss neighbour forces a "NOT
@@ -153,10 +152,9 @@ A fallback you know but did not write is not a fallback.
 Retrofit), wrapped as `bin/make-skill-audit` for Claude Code. Beside them:
 `hooks/` (PostToolUse, silent unless a `SKILL.md` was written),
 `commands/skill-audit.md` (deliberately NOT the skill's name),
-`agents/skill-auditor.md`, and six skeletons: `assets/SKILL.template.md`,
-`assets/plugin.template.json`, `assets/marketplace.template.json`,
-`assets/hooks.template.json`, `assets/agent.template.md`,
-`assets/command.template.md`.
+`agents/skill-auditor.md`, and six `assets/*.template.*` skeletons — one per
+component, filenames in `references/distribution.md` → *The distributable repo
+layout*.
 
 ## Create (personal)
 
@@ -196,8 +194,7 @@ regardless:
 Take it ALL the way, no half-done handoffs. Only the first publish needs a human
 (npm 2FA); **arming CI publishing is part of shipping**, so the second does not.
 **The 11-step sequence is in `references/distribution.md` → *First publish*.**
-Done = five VERIFIED facts: repo + CI green, npm resolvable via npx, plugin
-installed, skills-CLI discovery working, next tag publishing without a human.
+Done = the five VERIFIED facts in that sequence's step 10 — nothing assumed.
 
 ## Retrofit (bring an existing skill/repo up to standard)
 
@@ -310,9 +307,8 @@ session:
 - everything green BEFORE the tag: `python3 test/validate.py` plus BOTH
   `claude plugin validate … --strict` runs;
 - **refresh THIS machine's global installs as Definition of Done** (per global
-  `~/.claude/CLAUDE.md`): `claude plugin marketplace update <name>` →
-  `claude plugin update <name>@<name>` → `npx skills update <name> --global
-  --yes && rm -f ~/.claude/skills/<name>`, then remind about the restart;
+  `~/.claude/CLAUDE.md`) — the exact three-command sequence is checklist step 6;
+  then remind about the restart;
 - **move the family pin in the SAME session.** A member released without its
   umbrella pin bumped is invisible: `list` advertises the old version and
   `update` installs it (seen here 2026-08-10);

package/plugins/make-skill/skills/make-skill/references/authoring.md

@@ -155,6 +155,11 @@ the body.
   first 100 lines and never learn the rest exists.
 - **One level deep from `SKILL.md`.** A file reachable only through another file
   gets partially read or missed entirely. Every reference links from the body.
+- **English prose is a budget rule before a style one** (the house rule in
+  `SKILL.md` states which four literals stay Cyrillic). The measured cost:
+  Russian encodes at 1.9–2.3 chars/token against English's 5.0 (`cl100k`), and
+  rewriting the eight ssheleg routers into English cut them **3408 → 1885
+  tokens** with no loss of meaning.
 
 ## Scripts — the rules that separate a script from a liability
 

package/plugins/make-skill/skills/make-skill/references/distribution.md

@@ -51,7 +51,11 @@ the shape from `ssheleg/super-ux`:
 │   ├── .claude-plugin/plugin.json      # ONLY the manifest lives in .claude-plugin/
 │   ├── skills/<skill>/SKILL.md  +  references/*.md  +  scripts/  +  assets/
 │   │                                   # skeletons live HERE, not at the repo root:
-│   │                                   # only the skill dir travels on every channel
+│   │                                   # only the skill dir travels on every channel.
+│   │                                   # make-skill ships six: SKILL.template.md,
+│   │                                   # plugin.template.json, marketplace.template.json,
+│   │                                   # hooks.template.json, agent.template.md,
+│   │                                   # command.template.md
 │   ├── bin/<exe>                       # Claude Code puts this on the Bash PATH:
 │   │                                   # the only reliable way to hand the agent
 │   │                                   # a runnable command (no path variable)

package/plugins/make-skill/skills/make-skill/references/host-capabilities.md

@@ -143,7 +143,17 @@ body receives `$ARGUMENTS`. Two rules cost a debugging round each:
 
 - **Never name a command after a skill in the same plugin.** Commands are skills
   now: both claim `/<name>`, the skill wins, the command is unreachable always-on
-  cost visible only in `claude plugin details`.
+  cost visible only in `claude plugin details`. That is the default for every new
+  plugin. **One recorded exception (operator decision, 2026-08-30): the ssheleg
+  family ships same-named commands deliberately** — `task-pipeline`,
+  `project-audit`, `seo-aeo-audit`, `sheleg-design`, `agent-sync`, and
+  `super-ux`'s `vision`, `ux-foundation`, `ux-flows`, `ux-audit`. The cost is
+  accepted, not waived: the skill wins the trigger, and each command stays an
+  always-on token cost that may be unreachable in the picker. An audit that finds
+  a collision on this list reports it as *deliberate, recorded — no change
+  needed*; a collision NOT on a dated, recorded exception list is still the gap
+  this rule names. A rule's exception is enumerated, dated, and carries its cost
+  — never implied.
 - **Quote `argument-hint`.** Bare `[a | b]` is a YAML flow sequence; a comma
   inside it drops the entire frontmatter block, leaving a command with no
   description and no warning.
@@ -236,7 +246,8 @@ what the agent reads at the exact moment something is missing.
 - [ ] `PostToolUse` advises (`systemMessage`), `PreToolUse` blocks — not the reverse
 - [ ] Hook commands quote `"${CLAUDE_PLUGIN_ROOT}"` and set a `timeout`
 - [ ] Hook scripts are executable, have a shebang, and need no `jq`
-- [ ] No command named after a skill; every `argument-hint` quoted
+- [ ] No command named after a skill — or the collision is on a recorded, dated
+      exception list (see *Commands*); every `argument-hint` quoted
 - [ ] Plugin agents carry no `hooks` / `mcpServers` / `permissionMode`
 - [ ] Scripts are stdlib-only, inside the skill dir, invoked by a resolvable path
 - [ ] No command the agent is told to RUN contains `${CLAUDE_PLUGIN_ROOT}` — it is

package/plugins/make-skill/skills/make-skill/references/retrofit.md

@@ -125,8 +125,11 @@ Report the table before changing anything, then fix.
     MCP server: the degradation contract written in the body for all three axes
     (not Claude Code / recommended plugin absent / tool absent); hooks that
     exit 0 silently when the event is not theirs; `PostToolUse` advising rather
-    than blocking; commands quoted and never named after a skill; plugin agents
-    free of `hooks`, `mcpServers` and `permissionMode`.
+    than blocking; commands quoted and never named after a skill (a collision on
+    the recorded, dated exception list in `references/host-capabilities.md` —
+    the ssheleg family's same-named commands, operator decision 2026-08-30 — is
+    reported as *deliberate, recorded — no change needed*, not as a gap); plugin
+    agents free of `hooks`, `mcpServers` and `permissionMode`.
 
 ## Personal skills — the short form