@ssheleg/make-skill 0.7.0 → 0.8.0

package/CHANGELOG.md

@@ -1,5 +1,139 @@
 # Changelog
 
+## v0.8.0 — 2026-08-03
+
+Read Anthropic's four Agent Skills pages end to end (overview, authoring best
+practices, enterprise, the API skills guide) and audited the canon against them.
+The canon had been written against the open standard and the Claude Code plugin
+reference only — so it was correct about the format and blind to everything
+Anthropic's own surfaces enforce, recommend, or forbid. Three new references,
+six new validator rules, and one self-violation fixed.
+
+### Added
+- **`references/surfaces.md`** — the surface a skill runs on is a portability
+  contract, and it was undocumented here. Claude Code has full network and local
+  installs; the **Claude API container has neither**; claude.ai varies. A skill
+  whose script `pip install`s or curls works on one surface and silently fails
+  on the next. Also the whole Skills API — beta headers, `container.skills[]`,
+  **max 8 skills per request**, upload rules (top-level dir must match `name`,
+  `display_title` uniqueness, **< 30 MB**), epoch-timestamp versions,
+  delete-all-versions-before-delete — the claude.ai zip channel (per user, no
+  admin push), and the fact that **nothing syncs between surfaces**.
+- **`references/enterprise.md`** — installing a skill you did not write is
+  installing software, and the canon said so without saying how to check.
+  Anthropic's risk-tier table, the 8-step review checklist, the five approval
+  gates (triggering accuracy, isolation, **coexistence**, instruction following,
+  output quality), lifecycle with separation of duties, registry fields, recall
+  limits, and the production versioning/rollback/integrity rules.
+- **`references/authoring.md`** — the craft that the spec reference had only
+  hinted at: naming conventions (gerund; never `helper`/`utils`/`tools`),
+  **degrees of freedom** matched to task fragility, workflow checklists and
+  feedback loops, the script rules (solve-don't-defer, no voodoo constants,
+  execute-vs-read intent, declare packages), content guidelines, and
+  **evaluation-driven development** — evals before prose, ≥3 scenarios, a
+  no-skill baseline, the Claude-A/Claude-B loop, testing on every target model.
+  The trigger-eval loop moved here from the spec reference.
+- Six validator rules with negative self-tests that assert the *reason* for the
+  failure, not just a non-zero exit: reserved words in `name`, XML tags in
+  `name`/`description`, first/second person in `description`, the body token
+  budget, a `## Contents` list on references past 100 lines, and angle-bracket
+  placeholders in the SKILL template.
+- **`test/evals/`** — make-skill's own evaluation suite, because the canon now
+  requires one from every skill and shipping the rule without obeying it is how
+  a standard becomes decoration. `triggers.json`: 20 queries, 10 that must fire
+  (EN + RU) and 10 near-misses that share the keywords — "add a skills section
+  to my CV", "publish this React library to npm", "implement an MCP server for
+  Postgres". `scenarios.json`: four behavioral evaluations in Anthropic's shape
+  (create, retrofit-with-evidence, surface portability, third-party review),
+  scored line by line, with fixtures under `test/evals/fixtures/` — deliberately
+  broken and deliberately not named `SKILL.md`. The validator proves they exist
+  and are well-formed and cannot run them: that part is a human with an agent
+  (`test/evals/README.md`).
+
+### Fixed
+- **`name` may not contain `anthropic` or `claude`, and neither field may
+  contain XML tags.** The open standard is silent on both; Anthropic's platform
+  enforces them at upload. So `claude-helper` loads happily in Claude Code and
+  is rejected the day someone ships it to the API — a failure that only ever
+  appears on someone else's machine. Canon, spec reference, Cursor rule, README
+  and validator now carry the rule.
+- **Descriptions must be third person.** They are injected into the system
+  prompt, where "I can help you…" / "You can use this to…" measurably degrades
+  selection. Stated in the canon and checked by the validator.
+- **This skill was over its own token budget.** `SKILL.md` was ~5.2k estimated
+  tokens against a stated `< 5000` — a rule the repo enforced on everyone but
+  itself, because only the 500-line half was ever checked. The body budget is
+  now measured (chars/4, frontmatter excluded as level-1 metadata) and the body
+  was cut to ~4.6k by moving the repo layout, the two Claude Code gates, the
+  validator spec and the release checklist into `references/distribution.md`.
+- **Reference files longer than 100 lines now open with a `## Contents` list.**
+  Agents preview long files with `head`; without a table of contents they act on
+  the first hundred lines and never learn the rest exists. All eight references
+  gained one — five were in violation.
+- **The SKILL template seeded an invalid skill.** Its `name: <skill-name>` and
+  `<angle-bracket>` description placeholders are XML tags in exactly the two
+  fields where Anthropic rejects them. Replaced with plain placeholders, and the
+  validator now checks the skeleton it ships.
+- **MCP tool names: two conventions, both real.** Anthropic's authoring guidance
+  writes `ServerName:tool_name`; Claude Code's runtime names are
+  `mcp__<server>__<tool>`. The canon documented only the second, which reads as
+  wrong against the docs — both are recorded now, with "list and match" still
+  the rule.
+- Retrofit audit gained surface honesty (item 3) and evaluations (item 8);
+  authoring rules gained naming style, prescriptiveness-to-fragility, and the
+  no-time-branching rule with the "Old patterns" escape hatch.
+
+## v0.7.1 — 2026-07-31
+
+Re-read the plugin reference against what the canon actually says, and against
+what Claude Code 2.1.212 actually does. Six corrections, two of them places
+where the docs and the binary disagree.
+
+### Fixed
+- **`claude plugin validate` validates the manifest, not front matter.** The
+  docs' troubleshooting table says it checks "`plugin.json`, skill/agent/command
+  frontmatter, and `hooks/hooks.json`"; a `SKILL.md` carrying an invented
+  front-matter key passes `--strict` untouched. The canon said this gate
+  "catches a class your validator cannot see" — true for manifests, misleading
+  for front matter, which is exactly where the house validator earns its place.
+- **`claude plugin update <bare-name>` does not work**, though the docs list a
+  bare name as accepted: it answers `Plugin "make-skill" not found` **and exits
+  0**, so a release script sees success and ships nothing. The reference now
+  records both the documented form and the observed one.
+- **`skills` adds to the default scan — except at a marketplace root.** When a
+  plugin entry's `source` resolves to the marketplace root, the listed
+  subdirectories replace the default `skills/` scan (and if none exist, the
+  default runs after all). The reference stated the rule without its exception.
+- **"Every component directory sits at the plugin root" was too absolute.** That
+  is the default; a manifest path field may point anywhere inside the plugin,
+  and since 2.1.140 Claude Code warns when a manifest key leaves a default
+  folder unscanned. Only `.claude-plugin/` is genuinely off-limits.
+- **`allowed-tools`: Claude Code is looser than the spec.** It accepts a
+  space- or comma-separated string or a YAML list; the open standard accepts
+  only the space-separated string. Canon, Cursor rule and the validator's own
+  error message now say which form travels and why, instead of calling a legal
+  Claude Code skill invalid.
+- **npm plugin sources** resolve to `unknown` only when no `version` is set
+  anywhere — the entry or manifest version still wins.
+
+### Changed
+- The stray-`SKILL.md` rule is scoped honestly: a plugin-root `SKILL.md` is a
+  legal single-skill plugin in Claude Code (2.1.142+). It is multi-channel
+  distribution — the skills CLI shipping every `SKILL.md` in the tree — that
+  makes the rule, and the validator says so.
+- Version pinning: omitting `version` from both manifests is legal and hands
+  updates to the git SHA. This canon pins and bumps by choice, which the gotcha
+  now states rather than implies.
+
+### Added
+- **LSP and monitor fields in `references/claude-code-plugin.md`.** Its own load
+  condition promised "adding agents / hooks / MCP / LSP / monitors", and the
+  file had a table row for each and nothing else: now the required and optional
+  LSP keys with the 2.1.205 `restartOnCrash` / `shutdownTimeout` trap and the
+  same-extension race, and the monitor entry schema with its trust,
+  `${user_config.*}` rejection and session-lifetime rules. Plus the missing
+  `themes/` row.
+
 ## v0.7.0 — 2026-07-30
 
 Conformance to Anthropic's own plugin reference

package/README.md

@@ -53,11 +53,15 @@ resolvable install — not "should work now".
 
 ## What you get
 
-**Conformance to the open standard, not just a house style.** The
+**Conformance to both rulebooks, not just a house style.** The
 [Agent Skills specification](https://agentskills.io/specification) sets hard
 limits — `name` is 1–64 chars of `a-z0-9` and single hyphens and must match its
 directory, `description` caps at 1024 characters, the body should stay under 500
-lines and 5000 tokens. Local conventions that merely *feel* right drift from those
+lines and 5000 tokens. [Anthropic's platform docs](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview)
+add rules the open standard is silent about and only the Skills API enforces —
+no XML tags in `name`/`description`, and no `anthropic`/`claude` anywhere in the
+name, so `claude-helper` loads fine in Claude Code and is rejected the day
+someone uploads it. Local conventions that merely *feel* right drift from both
 without anyone noticing; here they're checked.
 
 **Plugins Anthropic's own tooling accepts.** The
@@ -70,16 +74,26 @@ both manifests, in CI as its own job. That check is what caught this repo's own
 `marketplace.json` shipping `homepage` and `repository` at a level where Claude
 Code ignores them.
 
+**Evaluations, not vibes.** The canon requires every skill to carry at least
+three behavioral scenarios and a trigger set whose negatives are *near-misses* —
+queries that share the keywords and need something else. make-skill ships its
+own in [`test/evals/`](test/evals/), because a description that also fires on
+"review my pull request" steals turns from every other skill you have installed.
+
 **A validator that can fail.** `test/validate.py` (Python stdlib only, no deps)
-checks spec rules, version sync across every manifest, front-matter on commands
-and Cursor rules, link integrity, and the traps below. CI runs it plus **negative
-self-tests** — deliberately broken copies that must make it exit non-zero. A
-validator nobody has seen fail is decoration.
+checks both rulebooks' front-matter rules, the 500-line **and** 5000-token body
+budgets, a `## Contents` list on every reference past 100 lines, version sync
+across every manifest, front-matter on commands and Cursor rules, link
+integrity, and the traps below. CI runs it plus **negative self-tests** —
+deliberately broken copies that must make it exit non-zero, each asserted to
+fail for the right reason. A validator nobody has seen fail is decoration.
 
 **Every distribution channel, with the flags that actually work.** Claude Code
 plugin, the [vercel `skills` CLI](https://github.com/vercel-labs/skills) (70+
 agents), `npx`, Cursor rules, and umbrella-repo families — including the
-one-channel-per-agent rule that stops them from shadowing each other.
+one-channel-per-agent rule that stops them from shadowing each other — plus
+Anthropic's own surfaces (Skills API upload, claude.ai zip), which sync with
+none of the above and impose their own limits on what a skill may do at runtime.
 
 **The gotchas that each cost a debugging round**, so your first publish works
 instead of your fourth:
@@ -104,9 +118,12 @@ agent opens only when the situation calls for them:
 
 | Reference | Covers |
 |---|---|
-| `agent-skills-spec.md` | the open standard — field limits, optional front-matter, token budgets, the description trigger-eval loop |
+| `agent-skills-spec.md` | the hard rules from both authorities — field limits, where the open standard and Anthropic's platform differ, budgets, which checker says no |
+| `authoring.md` | the craft — naming, third-person descriptions, degrees of freedom, workflows and feedback loops, script rules, evaluation-driven development |
+| `surfaces.md` | Claude Code vs the Claude API vs claude.ai — the Skills API (upload, versions, 8 per request), and the no-network / no-package-install limits that break scripts moved between surfaces |
+| `enterprise.md` | installing a skill you didn't write, and running a fleet — risk tiers, the review checklist, the five approval gates, lifecycle, recall limits, rollback |
 | `claude-code-plugin.md` | the [Claude Code layer](https://code.claude.com/docs/en/plugins-reference) — `plugin.json` / `marketplace.json` schemas, plugin sources, component locations, host-only front-matter, path variables, cache and symlink rules, the `claude plugin` CLI |
-| `distribution.md` | every install channel, exact CLI flags, npm publishing traps |
+| `distribution.md` | the repo layout, every install channel, exact CLI flags, npm publishing traps, the release checklist |
 | `mcp.md` | [MCP](https://modelcontextprotocol.io) — skill vs server, primitives and methods, transports, consent and untrusted-output rules |
 | `a2a.md` | [A2A](https://a2a-protocol.org) — Agent Cards, task lifecycle, method mapping, v0.x→1.0 wire drift |
 
@@ -197,6 +214,7 @@ cursor/rules/make-skill.mdc           # self-contained Cursor rule
 templates/                            # SKILL.template.md + the two manifest skeletons
 bin/make-skill.js + package.json      # zero-dep npx installer
 test/validate.py                      # structural validator
+test/evals/                           # trigger set + behavioral scenarios (data)
 .github/workflows/{validate,release}.yml
 install.sh  README.md  CHANGELOG.md  CONTRIBUTING.md  SECURITY.md  LICENSE
 docs/superpowers/{specs,plans}/       # historical design records
@@ -212,7 +230,7 @@ install). Security reports: [SECURITY.md](SECURITY.md).
 
 Built by ssheleg — [sshlg.me](https://sshlg.me)
 
-- X / Twitter — [@fuck_this_year](https://x.com/fuck_this_year)
+- X / Twitter — [@sshlg93](https://x.com/sshlg93)
 - Telegram — [@sshlg](https://t.me/sshlg)
 
 Part of the [ssheleg skill family](https://github.com/ssheleg/sshlg-skills):

package/cursor/rules/make-skill.mdc

@@ -1,5 +1,5 @@
 ---
-description: "make-skill: create, retrofit, audit, or publish an agent skill / plugin the proven ssheleg way — the Agent Skills open standard (front-matter limits, progressive-disclosure budgets), the Claude Code plugin reference (manifest schemas, component layout, claude plugin validate --strict), marketplace repo layout, four-way version sync, validator + CI, multi-channel distribution (Claude Code plugin, skills CLI, npx, Cursor), npm gotchas, and the MCP / A2A rules for protocol-connected skills. Apply when creating a new skill, upgrading one to standard, checking one against the spec, or wrapping one into a plugin."
+description: "make-skill: create, retrofit, audit, or publish an agent skill / plugin the proven ssheleg way — the Agent Skills open standard AND Anthropic's platform rules (front-matter limits, disclosure budgets, per-surface runtime limits, evals), the Claude Code plugin reference (manifest schemas, component layout, claude plugin validate --strict), marketplace repo layout, four-way version sync, validator + CI, multi-channel distribution (Claude Code plugin, skills CLI, npx, Cursor, Skills API), npm gotchas, the review checklist for installing someone else's skill, and the MCP / A2A rules for protocol-connected skills. Apply when creating a new skill, upgrading one to standard, checking one against the spec, or wrapping one into a plugin."
 alwaysApply: false
 ---
 
@@ -9,22 +9,38 @@ Self-contained rule: Cursor copies this file into foreign projects, so it carrie
 every contract inline and uses **absolute URLs only — never a relative link**
 (a relative link dangles the moment the file is copied).
 
-## Spec floor — the Agent Skills open standard (non-negotiable)
+## Spec floor — two authorities, both non-negotiable
 
-<https://agentskills.io/specification> — the house canon below only ADDS to it.
+<https://agentskills.io/specification> (the portable format) and
+<https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview>
+(what Anthropic's own surfaces enforce). Write to the intersection; the house
+canon below only ADDS to it.
 
-- `name`: 1–64 chars, `a-z0-9` + single internal hyphens, == directory name.
-- `description`: ≤1024 chars (that cap is on the field, not the whole front-matter).
+- `name`: 1–64 chars, `a-z0-9` + single internal hyphens, == directory name,
+  **no angle brackets and never containing `anthropic`/`claude`** — reserved
+  substrings Claude Code loads happily and the Skills API rejects on upload.
+- `description`: ≤1024 chars (the cap is on the field, not the whole
+  front-matter), no angle brackets, **third person** (it goes into the system
+  prompt; "I can help you…" degrades selection), stating WHAT it does and WHEN
+  to use it.
 - Portable front-matter keys: `name`, `description`, `license`,
   `compatibility` (≤500), `metadata` (string→string map), `allowed-tools`
-  (space-separated string, experimental). Claude Code additionally reads
+  (space-separated string — Claude Code also takes a comma-list or YAML list, but
+  only the space form is portable). Claude Code additionally reads
   `when_to_use`, `argument-hint`, `arguments`, `disable-model-invocation`,
   `user-invocable`, `disallowed-tools`, `model`, `effort`, `context`, `agent`,
   `background`, `hooks`, `paths`, `shell` — legal, ignored elsewhere, never
   load-bearing. Anything outside both sets is a typo.
 - Body < 500 lines and < 5000 tokens. Overflow goes to `references/` (docs),
   `scripts/` (code), `assets/` (templates) INSIDE the skill dir, one level deep,
-  each with a stated load trigger ("read X when Y"). Gotchas stay in SKILL.md.
+  each linked from the body with a stated load trigger ("read X when Y"), and
+  any reference over 100 lines opens with a `## Contents` list (a partial `head`
+  read is what the agent often gets). Gotchas stay in SKILL.md.
+- **Surfaces differ and nothing syncs between them:** the Claude API container
+  has NO network and NO runtime package install, claude.ai varies, only Claude
+  Code has both. A script that `pip install`s or curls is a Claude Code skill —
+  declare it in `compatibility` or drop the dependency. Skills API also caps a
+  request at 8 skills and an upload at 30 MB.
 - Upstream checker: `skills-ref validate <skill dir>` (Python, from the
   `agentskills/agentskills` repo).
 
@@ -32,8 +48,20 @@ every contract inline and uses **absolute URLs only — never a relative link**
 
 - `description` starts "Use when …" with concrete trigger phrases in English AND
   Russian. A skill nobody triggers is dead weight.
+- One naming pattern, preferably gerund (`processing-pdfs`); never `helper`,
+  `utils`, `tools`, `data`.
 - One skill = one job. Multiple concerns → multiple skills + a shared contract file.
-- Body: imperative, procedural, checklists over prose. State non-negotiables as such.
+- Body: imperative, procedural, checklists over prose. State non-negotiables as
+  such. Match prescriptiveness to fragility — exact commands for destructive or
+  order-dependent work, direction only where context decides.
+- **Evals before prose:** run the task with NO skill, record the failures, write
+  ≥3 evaluations (including should-NOT-trigger cases), then write the minimum
+  that passes; re-run on every model you claim. Adding to an existing set → test
+  coexistence, a broad description steals other skills' triggers.
+- Scripts: handle their own errors, justify every constant, declare packages,
+  and say whether the agent should RUN or READ each one.
+- No time-branching text ("before August, use the old API") — superseded
+  material goes under `## Old patterns`; forward slashes in every path.
 - Commands are thin wrappers: front-matter `description` (+ `argument-hint`), body =
   "invoke skill X in mode Y", pass `$ARGUMENTS`.
 - Never overwrite user data: seed only when absent; overwrite only behind `--force`.
@@ -104,6 +132,11 @@ checker, not a house opinion:
 - **Cursor .mdc:** embed contracts inline (files get copied into foreign projects).
 - **e2e npx from a NON-repo cwd** (inside the repo it resolves the local package →
   false "command not found").
+- **Installing a skill you did not write is installing software.** Read every
+  file, run scripts sandboxed, grep for network calls and credentials, and check
+  the instructions for "ignore previous rules", hidden actions, or data routed
+  outward. A skill that fetches its instructions from a URL can change after the
+  review that approved it.
 
 ## Protocol-connected skills (MCP / A2A)
 
@@ -131,3 +164,8 @@ Claude Code plugin (`claude plugin marketplace add` → `install <name>@<name>`)
 vercel skills CLI (`npx skills add <owner>/<repo>`, discovers via marketplace.json),
 npx (`npx github:<owner>/<repo>`, or the registry after publish), Cursor (global via
 `npx skills add … --agent cursor --global`, or per-project `.cursor/rules/*.mdc`).
+Anthropic's own surfaces are separate deployments that sync with nothing: the
+Skills API (`POST /v1/skills`, beta headers `skills-2025-10-02` +
+`code-execution-2025-08-25`, workspace-wide, ≤8 skills per request, <30 MB, the
+uploaded top-level directory must match `name`) and claude.ai (zip upload in
+Settings → Features, per user, no admin push).

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@ssheleg/make-skill",
-  "version": "0.7.0",
-  "description": "Create, retrofit, audit, and ship agent skills & Claude Code plugins the proven ssheleg way — conformance to the Agent Skills open standard (front-matter limits, progressive-disclosure budgets) and the Claude Code plugin reference (manifest schemas, component layout, claude plugin validate --strict), marketplace repo layout, version sync, validator + CI, multi-channel distribution (plugin, vercel skills CLI, npx, Cursor), npm gotchas, and MCP / A2A rules for protocol-connected skills. This package is the installer CLI.",
+  "version": "0.8.0",
+  "description": "Create, retrofit, audit, and ship agent skills & Claude Code plugins the proven ssheleg way — conformance to the Agent Skills open standard AND Anthropic's platform rules (front-matter limits, disclosure budgets, per-surface runtime limits, the Skills API, evals) plus the Claude Code plugin reference (manifest schemas, component layout, claude plugin validate --strict), marketplace repo layout, version sync, validator + CI, multi-channel distribution (plugin, vercel skills CLI, npx, Cursor), npm gotchas, the review checklist for third-party skills, and MCP / A2A rules for protocol-connected skills. This package is the installer CLI.",
   "keywords": [
     "skill",
     "plugin",
@@ -22,7 +22,10 @@
   },
   "repository": "github:ssheleg/make-skill",
   "license": "MIT",
-  "author": "ssheleg",
+  "author": {
+    "name": "ssheleg",
+    "url": "https://x.com/sshlg93"
+  },
   "bin": {
     "make-skill": "bin/make-skill.js"
   },

package/plugins/make-skill/.claude-plugin/plugin.json

@@ -2,10 +2,11 @@
   "$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json",
   "name": "make-skill",
   "displayName": "Make Skill",
-  "description": "Create, retrofit, audit, and ship agent skills & Claude Code plugins the proven ssheleg way: conformance to the Agent Skills open standard and the Claude Code plugin reference, marketplace repo layout, version sync, validator + CI, multi-channel distribution (plugin, vercel skills CLI, npx, Cursor), npm gotchas, end-to-end first publish, plus MCP / A2A references for protocol-connected skills.",
-  "version": "0.7.0",
+  "description": "Create, retrofit, audit, and ship agent skills & Claude Code plugins the proven ssheleg way: conformance to the Agent Skills open standard, Anthropic's platform rules (surfaces, Skills API, evals) and the Claude Code plugin reference, marketplace repo layout, version sync, validator + CI, multi-channel distribution (plugin, vercel skills CLI, npx, Cursor), npm gotchas, end-to-end first publish, the review checklist for third-party skills, plus MCP / A2A references for protocol-connected skills.",
+  "version": "0.8.0",
   "author": {
-    "name": "ssheleg"
+    "name": "ssheleg",
+    "url": "https://x.com/sshlg93"
   },
   "homepage": "https://github.com/ssheleg/make-skill",
   "repository": "https://github.com/ssheleg/make-skill",