skilldrop-cli 1.3.0 → 1.5.0

package/README.md

@@ -15,22 +15,22 @@ npx skilldrop-cli install --pack dev-team     # or product-manager, solution-arc
 
 From a held-out eval, written blind from the skill's description:
 
-> *"We're taking ClaimLens to the investment committee on Thursday… Can you build a lean canvas for it and set out our riskiest assumptions and how we'd test them?"*
+> *"Our design agency sent the Figma file for Ostwick Museum's new signage as a JSON export… Can you (1) list the pages and the frames on each, (2) check the Final page against what the Brief page asks for, and (3) draft the comments I should leave for the agency on any frames with problems — one line each."*
 
-| Plain Claude Code | With [`lean-canvas`](https://sananthanarayan.github.io/skilldrop/skills/lean-canvas/) |
+| Plain Claude Code | With [`figma-diagrams`](https://sananthanarayan.github.io/skilldrop/skills/figma-diagrams/) |
 |---|---|
-| Finds the corrected break-even (~225 seats), then recommends *"scope the ask to Claims-only Phase 1"*. | *"Corrected, break-even is 225 seats — more than all three departments combined (215), so full rollout still doesn't cover running costs at £40/seat."* |
+| Finds the same three problems, then drafts comments of two sentences or more, and adds a fourth one about the Archive page that the file doesn't support. | *"The cover email is wrong on both counts. The Sign Types page has 11 frames, not 12, and Type 07 is missing."* Then one line per frame with a problem. |
 
-The blind judge picked the skill's answer: *"A misses this and recommends a Claims-only phase that cannot break even."*
+Both blind judges picked the skill's answer: *"A keeps each comment to a single sentence as asked, while B's main comments run to two sentences and its fourth comment is speculative."*
 
 ## The receipts
 
-Every skill runs as a real agent session twice: once with the skill, once without. The results are [published either way](https://sananthanarayan.github.io/skilldrop/evals/). In the last full run (23 October 2026: 117 skills, 520 evals, `claude-sonnet-5`):
+Every skill runs as a real agent session twice: once with the skill, once without. The results are [published either way](https://sananthanarayan.github.io/skilldrop/evals/). In the latest run (5 November 2026: 111 skills, 300 held-out evals written blind from each skill's description, `claude-sonnet-5-5`, scripts free to run):
 
-- **With a skill, answers met 73% of their checks; without one, 53%.** The checks were written by each skill's author, so the fairer numbers are the next ones.
-- **Two blind judges preferred the skill's answer in 62% and 61% of pairs.** Each judge saw only the request and both answers, in shuffled order. On the 176 held-out evals that no revision or admission was based on, it was 49% and 48%: no preference.
-- **On Sonnet 5.5, now the standard tier, the judges preferred the plain agent.** On all 308 held-out evals the skill won 40% and 43% of pairs ([what the benchmark showed](guides/explanation/what-the-benchmark-showed.md#standard-on-sonnet-55)).
-- **4 skills that didn't beat the plain agent were [retired](https://sananthanarayan.github.io/skilldrop/evals/#retired)**, and they're listed. Each new skill has to win on held-out evals before it ships.
+- **With a skill, answers met 84% of their checks; without one, 76%.**
+- **Two blind judges preferred the skill's answer in 44% and 42% of pairs.** Each judge saw only the request and both answers, in shuffled order. Skills that ship a script came out slightly ahead of the plain agent (52%); skills that are instructions alone lost about two pairs in three ([what the benchmark showed](guides/explanation/what-the-benchmark-showed.md#a-new-baseline-with-scripts-free-to-run)).
+- **Loops are measured against their own skills, used without the loop.** Three are clearly ahead of both their skills and the plain agent: [`release`](packs/dev-team/loops/release/LOOP.md), [`modernize`](packs/dev-team/loops/modernize/LOOP.md) and [`ship-agent`](packs/ai-engineering/loops/ship-agent/LOOP.md). Start there: `npx skilldrop-cli install --loop release`. Four loops that lost to their own skills were retired ([RFC-0074](docs/rfcs/0074-loops-measured-four-retired.md)).
+- **Ten skills and four loops that didn't beat the alternative were [retired](https://sananthanarayan.github.io/skilldrop/evals/#retired)**, and they're listed. Each new skill has to win on held-out evals before it ships.
 
 [Get started](https://sananthanarayan.github.io/skilldrop/start/) · [Browse the packs](https://sananthanarayan.github.io/skilldrop/packs/) · [Search all skills](https://sananthanarayan.github.io/skilldrop/catalogue/) · [Docs](https://sananthanarayan.github.io/skilldrop/docs/) · [What stays stable in 1.x](guides/reference/versioning-and-stability.md) · [Contribute](CONTRIBUTING.md)
 
@@ -80,7 +80,7 @@ Install one pack. Every role pack also brings `core` (intake, critique, review c
 ## How it works
 
 - **A skill produces one file you own.** Each ships a quality bar, named anti-patterns and acceptance evals, so the output is an artifact, not a conversation. See [every skill, by category](guides/reference/skill-catalogue.md).
-- **Loops order skills, and a gate decides when work moves on.** Five loops cover the lifecycle (`discover`, `design`, `build`, `release`, `operate`), plus the `ship-a-draft` wrapper for any document and loops for jobs beside it, such as `research`, `comply`, `adopt` for a team starting with AI, `ship-agent` for an AI agent and `provision` for a Terraform change. Each gate is a script, a review panel or a person, chosen by how expensive the mistake is to undo. See [the operating model](guides/explanation/operating-model.md) for every stage and gate on one page, [the loops, stage by stage](guides/reference/loops.md), and [why loops](guides/explanation/loops.md).
+- **Loops order skills, and a gate decides when work moves on.** Loops cover three lifecycle stages (`design`, `build`, `release`), plus the `ship-a-draft` wrapper for any document and loops for jobs beside it, such as `research`, `comply`, `ship-agent` for an AI agent and `provision` for a Terraform change. Every loop is measured against its own skills used without it; four that lost were retired. Each gate is a script, a review panel or a person, chosen by how expensive the mistake is to undo. See [the operating model](guides/explanation/operating-model.md) for every stage and gate on one page, [the loops, stage by stage](guides/reference/loops.md), and [why loops](guides/explanation/loops.md).
 - **Measured, not asserted.** `run_bench.py` runs each acceptance eval with the skill and without it, in a sandbox, and reports the lift, a blind preference from two judges, and the cost per run. The results are published whether or not they flatter the skills. See [how skills are checked](https://sananthanarayan.github.io/skilldrop/evals/) and [RFC-0040](docs/rfcs/0040-skill-benchmark.md).
 - **Copy, never transform.** A skill is a plain `SKILL.md` folder in the [Agent Skills](https://agentskills.io) format, at `packs/<pack>/skills/<name>/`. What runs in your agent is byte-identical to what is reviewed here. There is no runtime, and the tooling has zero dependencies.
 - **Updates keep your edits.** `npx skilldrop-cli update` replaces files you haven't touched. For any file you changed, it leaves the new version beside it as `<file>.upstream`. See [upgrade installed skills](guides/how-to/upgrade-skills.md).

package/bin/skilldrop.js

@@ -264,11 +264,16 @@ function readPacksCatalog(dir, source) {
 }
 
 /* RFC-0046: a skill that left the catalogue keeps a record in catalogue.json `retired`, so an
-   installed copy can say why it stopped updating instead of looking like an unreachable source. */
-function retiredNote(cat, s) {
+   installed copy can say why it stopped updating instead of looking like an unreachable source.
+   RFC-0074: a loop can be retired too; its record says "kind": "loop". `isLoop` picks the kind
+   the caller is asking about; left out, a record of either kind answers (search). */
+function retiredNote(cat, s, isLoop) {
   const r = cat && cat.retired && cat.retired[s];
   if (!r) return null;
-  return `'${s}' was retired in ${r.in}: ${r.why}` + (r.instead ? ` Use ${r.instead} instead.` : "");
+  const kind = r.kind || "skill";
+  if (isLoop !== undefined && kind !== (isLoop ? "loop" : "skill")) return null;
+  const what = kind === "loop" ? `The loop '${s}'` : `'${s}'`;
+  return `${what} was retired in ${r.in}: ${r.why}` + (r.instead ? ` Use ${r.instead} instead.` : "");
 }
 
 /* Minimal TOML read — the [pack] table's basic-string scalars (name, version, description).
@@ -1225,7 +1230,7 @@ function expandNames(args, cat) {
     return packMembers(ps, args.flags.pack, "skills");
   }
   if (!args._.length) die("nothing to install — pass skill names, --pack <name>, or --all");
-  for (const s of args._) if (!skillExists(cat, s)) die(retiredNote(cat, s) || `unknown skill '${s}' in catalog '${cat.source}'`);
+  for (const s of args._) if (!skillExists(cat, s)) die(retiredNote(cat, s, false) || `unknown skill '${s}' in catalog '${cat.source}'`);
   return args._.slice();
 }
 
@@ -1567,7 +1572,7 @@ function installedRows(flags) {
     const loop = isLoopEntry(entry);
     try {
       cat = resolveCatalog(src === BUNDLED ? undefined : src);
-      if (!loop && !unitExists(cat, s, loop)) retired = retiredNote(cat, s);
+      if (!unitExists(cat, s, loop)) retired = retiredNote(cat, s, loop);
       if (unitExists(cat, s, loop)) {
         current = unitMeta(cat, s, loop).version || null;
         // Same version, different files: the catalog changed the skill without bumping it.
@@ -1711,7 +1716,7 @@ function outdated(args) {
     console.log(`\n${drift.length} skill(s) changed upstream with the same version. Read the change first (skilldrop diff <skill>),` +
                 `\nthen take it with: skilldrop update --changed`);
   if (gone.length)
-    console.log(`\n${gone.length} retired skill(s) will not update. Keep them as they are, or remove them: skilldrop uninstall ${gone.map((r) => r.s).join(" ")}${targetHint(args.flags)}`);
+    console.log(`\n${gone.length} retired skill(s) or loop(s) will not update. Keep them as they are, or remove them: skilldrop uninstall ${gone.map((r) => r.s).join(" ")}${targetHint(args.flags)}`);
   console.log(stale.length ? `\n${stale.length} outdated — run: skilldrop update` : (drift.length || gone.length ? "" : "everything up to date."));
 }
 
@@ -1767,7 +1772,7 @@ function diff(args) {
   const loop = isLoopEntry(entry);
   const src = args.flags.from || (entry ? lsrc(entry) : BUNDLED);
   const cat = resolveCatalog(src === BUNDLED ? undefined : src);
-  if (!unitExists(cat, s, loop)) die(loop ? `'${s}' is no longer in catalog '${cat.source}'` : (retiredNote(cat, s) || `'${s}' is no longer in catalog '${cat.source}'`));
+  if (!unitExists(cat, s, loop)) die(retiredNote(cat, s, loop) || `'${s}' is no longer in catalog '${cat.source}'`);
   const theirs = unitHashes(cat, s, loop), mine = fileHashes(path.join(dest, s));
   const base = (entry && entry.files) || {};
   const changed = Object.keys(theirs).filter((f) => f in mine && mine[f] !== theirs[f]).sort();
@@ -1811,7 +1816,7 @@ function doctor(args) {
   else try { data = JSON.parse(raw); } catch (e) { add("error", `${lp} is not valid JSON`, "fix or delete it; skilldrop can't track updates until then"); }
   const names = Object.keys(data);
   for (const n of names) {
-    const note = !isLoopEntry(data[n]) && lsrc(data[n]) === BUNDLED && retiredNote(resolveCatalog(), n);
+    const note = lsrc(data[n]) === BUNDLED && retiredNote(resolveCatalog(), n, isLoopEntry(data[n]));
     if (note) add("warn", note, `skilldrop uninstall ${n}${targetHint(args.flags)}, or keep it as it is; it will not update`);
     if (!fs.existsSync(path.join(dest, n))) add("error", `${n} is in the ledger but its folder is gone`, `skilldrop install ${n} (or uninstall ${n} to drop the record)`);
     else if (!data[n].file) {
@@ -2293,7 +2298,7 @@ function installLoops(args) {
   }
   if (!names.length && args.flags.all) names = available.slice();
   if (!names.length) die("nothing to install — pass loop names, --pack <name>, or --all");
-  for (const n of names) if (!available.includes(n)) die(`unknown loop '${n}' in catalog '${cat.source}'`);
+  for (const n of names) if (!available.includes(n)) die(retiredNote(cat, n, true) || `unknown loop '${n}' in catalog '${cat.source}'`);
 
   let bad = 0;
   for (const n of names) {
@@ -2569,7 +2574,7 @@ function info(args) {
   if (args.flags.pack) return infoPack(args);
   const cat = resolveCatalog(args.flags.from);
   const s = args._[0] || die("pass a skill name, or --pack <name>");
-  if (!skillExists(cat, s)) die(retiredNote(cat, s) || `unknown skill '${s}' in catalog '${cat.source}'`);
+  if (!skillExists(cat, s)) die(retiredNote(cat, s, false) || `unknown skill '${s}' in catalog '${cat.source}'`);
   const m = manifestOf(cat, s);
   const ps = packsOf(cat) || {};
   const inPacks = Object.entries(ps).filter(([, p]) => p.skills.includes(s)).map(([n]) => n);

package/catalogue.json

@@ -18,8 +18,7 @@
         "success-metrics",
         "user-story-splitter",
         "user-journey-map",
-        "feedback-synthesis",
-        "lean-canvas"
+        "feedback-synthesis"
       ]
     },
     "design-the-system": {
@@ -124,7 +123,6 @@
         "ai-adoption-rollout",
         "ai-usage-policy",
         "agent-adoption-stage",
-        "ai-usage-report",
         "agent-config-review",
         "agent-incident-response"
       ]
@@ -217,6 +215,12 @@
     "md-to-html": { "in": "1.2.0", "why": "On its held-out evals, in each of two trials and with both blind judges, the agent without it made the better page: the skill's script showed raw HTML as text and its security policy blocked embeds, and it adds no rule the agent breaks." },
     "mermaid-render": { "in": "1.2.0", "why": "On its held-out evals, in each of two trials and with both blind judges, the agent without it fixed the same diagrams, ran the Mermaid CLI itself and said plainly what it had not rendered; the skill adds no rule it breaks." },
     "marketing-flyer": { "in": "1.2.0", "why": "On its held-out evals, in each of two trials and with both blind judges, the agent without it built the whole flyer with placeholders where the skill stopped or used a generic template; it adds no rule the agent breaks." },
-    "deck-builder": { "in": "1.2.0", "why": "On its held-out evals the agent without it built the better deck in each of two trials; rebuilt rule-first, with its script able to run, it still lost on fresh evals (judges 2 and 1 of 4; 69% of checks against 81%).", "instead": "md-to-pptx" }
+    "deck-builder": { "in": "1.2.0", "why": "On its held-out evals the agent without it built the better deck in each of two trials; rebuilt rule-first, with its script able to run, it still lost on fresh evals (judges 2 and 1 of 4; 69% of checks against 81%).", "instead": "md-to-pptx" },
+    "ai-usage-report": { "in": "1.4.0", "why": "On its held-out evals, over three trials and with both blind judges, the agent without it made the better report: it kept every rule the skill teaches (counts show activity, not quality; missing data is not zero; no leaderboard), while the skill refused to rank when asked, hid names the manager had supplied, and its script read neither eval's export." },
+    "lean-canvas": { "in": "1.4.0", "why": "On its held-out evals, in each of three trials and with both blind judges, the agent without it made the better canvas: it caught the same errors and went further, while the skill's lead on checks came from its own format and its script's break-even left out the cost of replacing lost customers.", "instead": "business-case" },
+    "adopt": { "in": "1.5.0", "kind": "loop", "why": "On its held-out evals its own four skills, installed without the loop, were preferred in every pair by both blind judges: the loop narrowed the policy to the first use case when the team had no rules at all, its gate swapped out the use case the sponsor had chosen, and its reply carried gate jargon. Install ai-use-case-triage, ai-readiness-assessment, ai-usage-policy and ai-adoption-rollout and run them in the pack's order." },
+    "apply": { "in": "1.5.0", "kind": "loop", "why": "On its held-out evals its own skills, installed without the loop, were preferred in every pair by both blind judges: its fit gate stopped before writing anything even when the candidate had already decided to apply, and its confirm gate left form answers blank. Use job-fit-analysis to decide, then resume-tailor, cover-letter and application-form-answers." },
+    "discover": { "in": "1.5.0", "kind": "loop", "why": "On its held-out evals the plain agent was preferred in every pair and its own skills in three of four: its ratification gate refused to record an approval the requester had already given, and its reply carried gate jargon. Use feedback-synthesis on raw signal, then prd-draft." },
+    "operate": { "in": "1.5.0", "kind": "loop", "why": "On its held-out evals its own skills, installed without the loop, were preferred in every pair by both blind judges: the skills already link instrumentation, runbook, comms and postmortem, and the loop added gate jargon and broke its own one rule. Use observability-plan, runbook-generator, incident-comms and postmortem-generator in that order." }
   }
 }

package/contracts/catalogue.schema.json

@@ -31,13 +31,14 @@
     },
     "retired": {
       "type": "object",
-      "description": "RFC-0046. Skills removed from the catalogue, so an installed copy can say why it stopped updating. A retired name has no folder and nothing references it; validate.py checks both.",
+      "description": "RFC-0046. Skills (and, RFC-0074, loops) removed from the catalogue, so an installed copy can say why it stopped updating. A retired name has no folder and nothing references it; validate.py checks both.",
       "additionalProperties": {
         "type": "object",
         "additionalProperties": false,
         "required": ["in", "why"],
         "properties": {
           "in": { "type": "string", "pattern": "^\\d+\\.\\d+\\.\\d+$", "description": "The skilldrop-cli version that removed it." },
+          "kind": { "enum": ["skill", "loop"], "description": "RFC-0074. What was retired; a record without it is a skill." },
           "why": { "type": "string", "description": "One sentence, shown by outdated, update, doctor, diff, info and install." },
           "instead": { "type": "string", "description": "A skill that covers the same job, when one does. Omit it when the agent without a skill does as well." }
         }

package/model-routing.json

@@ -107,8 +107,8 @@
       "rationale": "Structures a decision into MADR/Nygard. Some judgment on title/consequences; not adversarial."
     },
     "agent-adoption-stage": {
-      "tier": "standard",
-      "rationale": "Places a team on a fixed 5-stage ladder from observables and routes one unlock. Bounded diagnosis against a rubric; escalate via ambiguous-input when the observables are contradictory or the distribution is heavily skewed."
+      "tier": "light",
+      "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 79% and 86% of checks in two tier trials, against standard's 82% and heavy's 79% and 79%, within 5 points of the best both times."
     },
     "agent-budget": {
       "tier": "standard",
@@ -135,8 +135,8 @@
       "rationale": "Repo inspection plus synthesis against a fixed section order and line budget — the same shape as reverse-architecture. The judgment is the counterfactual test on each convention line, which is bounded; escalate via large-input on monorepos."
     },
     "ai-adoption-rollout": {
-      "tier": "standard",
-      "rationale": "Generates a phased people-rollout with gates and enablement from a stated tool and population. Structured planning against a fixed shape; escalate via ambiguous-input for large or politically contested rollouts."
+      "tier": "light",
+      "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 79% and 71% of checks in two tier trials, against standard's 75% and heavy's 64% and 64%, within 5 points of the best both times."
     },
     "ai-readiness-assessment": {
       "tier": "standard",
@@ -146,25 +146,21 @@
       "tier": "standard",
       "rationale": "Drafts a policy document against a fixed structure (tiers, behaviours, review matrix, exception route). Bounded generation; escalate via ambiguous-input where a regulatory regime is named or contested."
     },
-    "ai-usage-report": {
-      "tier": "standard",
-      "rationale": "Tabulates telemetry into a report. Mostly mechanical, but the 'AI theater' effectiveness call needs judgment."
-    },
     "ai-use-case-triage": {
-      "tier": "standard",
-      "rationale": "Scores candidates on weighted axes and forces a rejection list. Structured comparison with stated weights; escalate via ambiguous-input when candidates are numerous or the value data is contested."
+      "tier": "light",
+      "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 83% and 92% of checks in two tier trials, against standard's 81% and heavy's 85% and 92%, within 5 points of the best both times."
     },
     "api-contract-draft": {
-      "tier": "standard",
-      "rationale": "Schema synthesis against fixed conventions. The resource modeling needs judgment; the conventions bound it. Escalate via ambiguous-input on contested multi-team contracts."
+      "tier": "light",
+      "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 76% and 92% of checks in two tier trials, against standard's 69% and heavy's 69% and 76%, within 5 points of the best both times."
     },
     "application-form-answers": {
-      "tier": "standard",
-      "rationale": "Sorts questions into three kinds and drafts short answers from the resume. The hard rule is mechanical: a declaration is never answered for the person."
+      "tier": "light",
+      "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 88% and 94% of checks in two tier trials, against standard's 88% and heavy's 81% and 81%, within 5 points of the best both times."
     },
     "architecture-diagrams": {
-      "tier": "standard",
-      "rationale": "Translates prose to diagram syntax. Correctness matters; reasoning is bounded."
+      "tier": "light",
+      "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 93% and 100% of checks in two tier trials, against standard's 93% and heavy's 93% and 93%, within 5 points of the best both times."
     },
     "audience-profile": {
       "tier": "light",
@@ -187,8 +183,8 @@
       "rationale": "Extraction + tagging [explicit]/[implied]/[inferred]. The tagging needs care — above light, below heavy."
     },
     "bug-triage": {
-      "tier": "standard",
-      "rationale": "Extraction plus judgment calls (severity vs priority, hypothesis ranking) — same shape as brief-intake: above light, below heavy."
+      "tier": "heavy",
+      "rationale": "Measured (RFC-0073): on its held-out evals the heavy tier met 92% and 92% of checks in two tier trials, against standard's 76% and light's 70% and 86%, more than 5 points ahead both times; moved once its examples/ oracle shipped (RFC-0074)."
     },
     "bundle-size-review": {
       "tier": "standard",
@@ -219,8 +215,8 @@
       "rationale": "Multi-perspective deliberation + weighted reconciliation across distinct expert seats. Cheaper models collapse the seats into one voice and either manufacture or miss the real cruxes. Never downgrade."
     },
     "cover-letter": {
-      "tier": "standard",
-      "rationale": "A short letter from supplied facts, with a script listing every figure and name the person's material doesn't contain."
+      "tier": "light",
+      "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 100% and 100% of checks in two tier trials, against standard's 100% and heavy's 100% and 100%, within 5 points of the best both times."
     },
     "dashboard-spec": {
       "tier": "standard",
@@ -231,8 +227,8 @@
       "rationale": "Schema + SLA synthesis against fixed quality dimensions and an evolution matrix. Semantics capture needs care; bounded by the catalog. Escalate via large-input on wide schemas."
     },
     "decision-log": {
-      "tier": "light",
-      "rationale": "Extracts decisions/owners/dates from notes. Pattern extraction; escalate on long transcripts via large-input rule."
+      "tier": "standard",
+      "rationale": "Measured (RFC-0073): on its held-out evals the light tier fell more than 5 points behind the best in both tier trials (79% and 61% against standard's 79%); one step up, to standard."
     },
     "delivery-metrics-report": {
       "tier": "standard",
@@ -243,8 +239,8 @@
       "rationale": "Reads a version diff, changelogs and CI evidence against a fixed checklist; the merge verdict follows from what is verified, not open-ended judgment."
     },
     "design-doc": {
-      "tier": "standard",
-      "rationale": "Synthesizes a design doc from a brief. Escalate to heavy via large-input or ambiguous-input when alternatives are genuinely contested."
+      "tier": "light",
+      "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 88% and 88% of checks in two tier trials, against standard's 92% and heavy's 91% and 91%, within 5 points of the best both times."
     },
     "design-system-spec": {
       "tier": "standard",
@@ -276,16 +272,16 @@
       "rationale": "Compression to one page. Needs to keep the right signal; escalate on very long source docs."
     },
     "feature-implement-loop": {
-      "tier": "standard",
-      "rationale": "Code + test generation against acceptance criteria. The adversarial pass delegates to devils-advocate (heavy) where available; the loop itself is generation."
+      "tier": "light",
+      "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 85% and 85% of checks in two tier trials, against standard's 85% and heavy's 76% and 85%, within 5 points of the best both times."
     },
     "feedback-synthesis": {
       "tier": "standard",
       "rationale": "Coding feedback into themes is bounded judgment; the counts by distinct person and segment come from its script."
     },
     "figma-diagrams": {
-      "tier": "standard",
-      "rationale": "API calls + spec generation. Bounded transformation."
+      "tier": "light",
+      "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 92% and 92% of checks in two tier trials, against standard's 92% and heavy's 75% and 92%, within 5 points of the best both times."
     },
     "file-to-markdown": {
       "tier": "light",
@@ -300,8 +296,8 @@
       "rationale": "Judgment-heavy: generating real rivals, rating every evidence-hypothesis cell honestly and resisting the favourite theory is where a fast pass fails; the script only does the arithmetic."
     },
     "incident-comms": {
-      "tier": "standard",
-      "rationale": "Audience-segmented message generation under a strict don't-retract discipline. Same shape as exec-summary/audience-profile — translation and altitude, not adversarial reasoning."
+      "tier": "light",
+      "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 93% and 100% of checks in two tier trials, against standard's 85% and heavy's 92% and 93%, within 5 points of the best both times."
     },
     "information-architecture": {
       "tier": "standard",
@@ -323,13 +319,9 @@
       "tier": "standard",
       "rationale": "Checks evidence against a fixed checklist. Bounded synthesis, not adversarial review."
     },
-    "lean-canvas": {
-      "tier": "standard",
-      "rationale": "Filling the canvas is structured; the value is in ranking assumptions by risk and naming the cheapest test that could kill the riskiest one."
-    },
     "llm-eval-harness": {
-      "tier": "standard",
-      "rationale": "Methodology synthesis against a fixed decision tree and pitfall catalog. Grading-method and metric choices need judgment but are bounded; escalate via ambiguous-input on novel task types."
+      "tier": "light",
+      "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 71% and 71% of checks in two tier trials, against standard's 71% and heavy's 71% and 71%, within 5 points of the best both times."
     },
     "mcp-pairing": {
       "tier": "standard",
@@ -340,8 +332,8 @@
       "rationale": "Mechanical conversion to Confluence storage format done by its script, with an optional publish; the model checks what was dropped and what will be overwritten."
     },
     "md-to-docx": {
-      "tier": "standard",
-      "rationale": "A stdlib script does the conversion; the model restructures the Markdown so it converts cleanly (heading levels, footnotes, HTML), reads each warning and explains the losses. Bounded editing, no open-ended generation."
+      "tier": "heavy",
+      "rationale": "Measured (RFC-0073): on its held-out evals the heavy tier met 79% and 93% of checks in two tier trials, against standard's 64% and light's 71% and 79%, more than 5 points ahead both times."
     },
     "md-to-pptx": {
       "tier": "light",
@@ -368,40 +360,40 @@
       "rationale": "Structured derivation against a fixed roll-up discipline; the causal-sentence and gap judgments need care but the template bounds them. Escalate via ambiguous-input when the company objectives themselves are contested."
     },
     "output-hygiene": {
-      "tier": "standard",
-      "rationale": "The mechanical pass is a deterministic script; the model does bounded rewriting of flagged prose against a fixed tell list, not adversarial review."
+      "tier": "light",
+      "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 100% and 100% of checks in two tier trials, against standard's 92% and heavy's 100% and 92%, within 5 points of the best both times."
     },
     "postmortem-generator": {
-      "tier": "standard",
-      "rationale": "Synthesizes a structured doc from incident notes. Contributing-factor chains need judgment but are bounded; escalate via large-input on huge scrollbacks."
+      "tier": "light",
+      "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 93% and 91% of checks in two tier trials, against standard's 92% and heavy's 90% and 87%, within 5 points of the best both times."
     },
     "pr-description-writer": {
-      "tier": "standard",
-      "rationale": "Reads a diff and rewrites it for a reviewer. Bounded by the diff; the judgment is in picking what a reviewer needs first."
+      "tier": "light",
+      "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 85% and 85% of checks in two tier trials, against standard's 85% and heavy's 77% and 85%, within 5 points of the best both times."
     },
     "prd-draft": {
-      "tier": "standard",
-      "rationale": "Synthesizes a structured requirements doc from a brief. The solution-language discipline needs care; bounded by the template. Escalate via ambiguous-input on contested scope."
+      "tier": "heavy",
+      "rationale": "Measured (RFC-0073): on its held-out evals the heavy tier met 85% and 76% of checks in two tier trials, against standard's 68% and light's 68% and 68%, more than 5 points ahead both times; moved once its examples/ oracle shipped (RFC-0074)."
     },
     "pre-merge-review": {
-      "tier": "standard",
-      "rationale": "Orchestration: runs a deterministic gate script, dispatches the (heavy) reviewer subagents, and renders a rule-based go/no-go. The adversarial reasoning is delegated to the panel, so the skill itself is coordination."
+      "tier": "light",
+      "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 93% and 93% of checks in two tier trials, against standard's 86% and heavy's 93% and 93%, within 5 points of the best both times."
     },
     "prfaq": {
       "tier": "standard",
       "rationale": "Synthesizes a press release and FAQs from a product concept against a fixed format. The specificity discipline needs care but the template bounds it; escalate via ambiguous-input when the concept itself is contested."
     },
     "prompt-caching-advisor": {
-      "tier": "standard",
-      "rationale": "Structural analysis of prompt with threshold arithmetic and pattern matching — above light, no need for heavy."
+      "tier": "heavy",
+      "rationale": "Measured (RFC-0073): on its held-out evals the heavy tier met 90% and 90% of checks in two tier trials, against standard's 63% and light's 83% and 73%, more than 5 points ahead both times; moved once its examples/ oracle shipped (RFC-0074)."
     },
     "refactor-plan": {
       "tier": "standard",
       "rationale": "Sequencing behaviour-preserving steps follows known patterns (characterise, parallel change, strangle); escalate via ambiguous-input for contested rewrites."
     },
     "release-notes": {
-      "tier": "standard",
-      "rationale": "Classifies commits and rewrites them for a reader. The audience translation needs judgment; escalate via large-input on very long ranges."
+      "tier": "light",
+      "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 96% and 96% of checks in two tier trials, against standard's 91% and heavy's 100% and 98%, within 5 points of the best both times."
     },
     "rendering-strategy": {
       "tier": "standard",
@@ -412,12 +404,12 @@
       "rationale": "Question banks are rubric-shaped, but ranking by design impact and deriving falsification conditions for the specific feature is synthesis — above light."
     },
     "research-plan": {
-      "tier": "standard",
-      "rationale": "Structured planning against a fixed shape (decision, sub-questions, methods, criteria, stopping rule); needs judgment to cut scope but no adversarial reasoning."
+      "tier": "light",
+      "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 79% and 79% of checks in two tier trials, against standard's 79% and heavy's 79% and 79%, within 5 points of the best both times."
     },
     "resume-tailor": {
-      "tier": "standard",
-      "rationale": "Reorders and rewords existing content against a posting, with a script listing every figure and name the original doesn't contain."
+      "tier": "light",
+      "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 100% and 100% of checks in two tier trials, against standard's 100% and heavy's 100% and 100%, within 5 points of the best both times."
     },
     "reverse-architecture": {
       "tier": "standard",
@@ -444,20 +436,20 @@
       "rationale": "Matches questions to evidence and writes short answers. The hard rule is mechanical: no evidence, no answer."
     },
     "service-blueprint": {
-      "tier": "standard",
-      "rationale": "Structured mapping of a described service into fixed lanes, with fail points, waits and a ranking. Tracing causes needs judgment, but the lane structure and the input bound it."
+      "tier": "light",
+      "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 100% and 92% of checks in two tier trials, against standard's 92% and heavy's 85% and 92%, within 5 points of the best both times."
     },
     "skill-author": {
-      "tier": "standard",
-      "rationale": "Structured authoring against a fixed anatomy. The judgment is in the description and sibling boundaries, which the interview anchors; skill-review is the adversarial pass."
+      "tier": "light",
+      "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 100% and 93% of checks in two tier trials, against standard's 93% and heavy's 100% and 92%, within 5 points of the best both times."
     },
     "skill-review": {
       "tier": "heavy",
       "rationale": "Adversarial audit with a verdict: routing collisions and paraphrased safety issues are what a fast pass misses, and the lint script covers only the mechanical half."
     },
     "slide-outliner": {
-      "tier": "standard",
-      "rationale": "Outlines a deck to a time budget. Structuring, not deep reasoning."
+      "tier": "light",
+      "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 86% and 90% of checks in two tier trials, against standard's 83% and heavy's 83% and 93%, within 5 points of the best both times."
     },
     "slo-spec": {
       "tier": "standard",
@@ -476,8 +468,8 @@
       "rationale": "Formats Sonar API findings into a report. The hard analysis is Sonar's; this is synthesis. Escalate on very large diffs."
     },
     "source-synthesis": {
-      "tier": "standard",
-      "rationale": "Bounded synthesis over supplied text with a fixed citation format and strength criteria; the citation checker catches the mechanical failures, so mid-tier judgment is enough."
+      "tier": "light",
+      "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 83% and 78% of checks in two tier trials, against standard's 76% and heavy's 85% and 82%, within 5 points of the best both times."
     },
     "sql-review": {
       "tier": "heavy",
@@ -488,32 +480,32 @@
       "rationale": "Framework selection plus evidenced multi-factor judgment ending in a ranked, committal verdict. Cheaper models fill cells with generic filler, skip the TOWS crossings, and refuse to take a side. Never downgrade."
     },
     "subagent-design": {
-      "tier": "standard",
-      "rationale": "Design synthesis against a fixed topology catalog and role-card template. The fan-out-or-refuse judgment is bounded by the three-reasons rule; escalate via ambiguous-input on genuinely novel workflow shapes."
+      "tier": "heavy",
+      "rationale": "Measured (RFC-0073): on its held-out evals the heavy tier met 100% and 100% of checks in two tier trials, against standard's 86% and light's 85% and 86%, more than 5 points ahead both times; moved once its examples/ oracle shipped (RFC-0074)."
     },
     "success-metrics": {
       "tier": "standard",
       "rationale": "Structured measurement design; the primary-metric and counter-metric choices need judgment but the template bounds it. Escalate via ambiguous-input when the goal itself is contested."
     },
     "team-status-report": {
-      "tier": "standard",
-      "rationale": "A script computes every count and the suggested RAG; the model writes plain-language summaries, attributes outside facts and frames risks and asks. Bounded writing over computed numbers."
+      "tier": "light",
+      "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 93% and 93% of checks in two tier trials, against standard's 93% and heavy's 93% and 93%, within 5 points of the best both times."
     },
     "tech-debt-register": {
       "tier": "standard",
       "rationale": "Structures and ranks items against stated evidence. Judgment is bounded: a cost with no evidence is recorded as unknown, not estimated."
     },
     "terraform-module": {
-      "tier": "standard",
-      "rationale": "Structured generation against fixed module conventions and a secure-defaults checklist; the judgment is bounded to choosing defaults and validation rules for a described need."
+      "tier": "light",
+      "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 100% and 100% of checks in two tier trials, against standard's 92% and heavy's 83% and 100%, within 5 points of the best both times."
     },
     "terraform-plan-review": {
       "tier": "heavy",
       "rationale": "Adversarial review of a destructive change: judging which replacements lose data, which access changes widen exposure, and what is out of scope needs careful reasoning; a missed destroy costs far more than the model."
     },
     "test-plan-generator": {
-      "tier": "standard",
-      "rationale": "Synthesizes a structured plan from a change description. Risk ranking needs judgment but is bounded; escalate via ambiguous-input on contested or high-stakes releases."
+      "tier": "light",
+      "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 76% and 92% of checks in two tier trials, against standard's 76% and heavy's 61% and 54%, within 5 points of the best both times."
     },
     "threat-model": {
       "tier": "heavy",
@@ -524,24 +516,24 @@
       "rationale": "Arithmetic breakdown plus heuristic judgment on dominant cost drivers — standard fits."
     },
     "tool-use-schema-writer": {
-      "tier": "standard",
-      "rationale": "Schema generation from natural-language description — structured output task that fits standard tier."
+      "tier": "light",
+      "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 94% and 94% of checks in two tier trials, against standard's 96% and heavy's 97% and 97%, within 5 points of the best both times."
     },
     "tracker-brief-sync": {
       "tier": "standard",
       "rationale": "Maps doc sections to items and writes acceptance criteria that restate requirements; a script checks the spec, writes the import file and computes drift. Bounded transformation with a deterministic check."
     },
     "user-journey-map": {
-      "tier": "standard",
-      "rationale": "Structured synthesis into a fixed four-row stage model plus bounded ranking judgment. Escalate via ambiguous-input when the persona or outcome is contested or research inputs conflict."
+      "tier": "heavy",
+      "rationale": "Measured (RFC-0073): on its held-out evals the heavy tier met 83% and 83% of checks in two tier trials, against standard's 75% and light's 67% and 75%, more than 5 points ahead both times."
     },
     "user-story-splitter": {
       "tier": "standard",
       "rationale": "Decomposes an epic into INVEST slices with testable ACs. Structured synthesis; escalation rules cover large or contradictory PRDs."
     },
     "ux-writing": {
-      "tier": "standard",
-      "rationale": "Short-form writing to fixed slot rules and character budgets, plus a terminology consistency pass. Bounded generation with clear checks."
+      "tier": "light",
+      "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 92% and 92% of checks in two tier trials, against standard's 83% and heavy's 75% and 83%, within 5 points of the best both times."
     },
     "web-performance-review": {
       "tier": "standard",

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "skilldrop-cli",
-  "version": "1.3.0",
+  "version": "1.5.0",
   "description": "Portable AI-agent skills, measured against the agent without them. ADRs, PRDs, runbooks, decks and reviews, installable into Claude Code, Cursor, Kiro, and any IDE that takes custom instructions. One command per skill or per role-based pack.",
   "author": {
     "name": "Sanjay Ananthanarayan",

package/packs/ai-engineering/skills/agent-adoption-stage/SKILL.md

@@ -82,7 +82,6 @@ The load-bearing idea: **the bottleneck moves.** Each stage is limited by someth
 - ❌ Assessing organisational preconditions — data, policy ownership, incentives — that's [`ai-readiness-assessment`](../ai-readiness-assessment/SKILL.md), a different axis (a team can be governance-ready and stuck at stage 1).
 - ❌ Planning how a tool reaches people — cohorts, enablement, comms — that's [`ai-adoption-rollout`](../ai-adoption-rollout/SKILL.md).
 - ❌ Choosing which use cases to build — that's [`ai-use-case-triage`](../ai-use-case-triage/SKILL.md).
-- ❌ Reporting measured usage from telemetry — that's [`ai-usage-report`](../ai-usage-report/SKILL.md).
 - ❌ Designing one specific loop or fleet — that's `agent-loop-design` / `subagent-design`, which this skill routes *to*.
 
 ## Anti-patterns to avoid