@supersuit/superskill 0.2.2 → 0.3.1

package/CHANGELOG.md

@@ -1,5 +1,30 @@
 # Changelog
 
+## 0.3.1 (2026-09-29)
+
+The 0.3.0 changes below, released. The v0.3.0 tag's publish refused on a red test (SPEC.md still
+named 0.2.2), so nothing was published under 0.3.0.
+
+## 0.3.0 (2026-09-29, not published)
+
+**A golden's approval says why, and what it rests on.** Before, an approval recorded who and when,
+with an optional note, so "a person approved it" read the same whether they liked it or it had
+landed a client. (Gary Sheng, on essays he and his co-founder had approved: *"golden approval also
+needs to carry rationale and weight. Right now the approval is based on Wilson and Gary being happy,
+not proven results of landing clients."*)
+
+- `superskill approve` now requires a rationale and asks for a basis: `judgment` (you read it and it
+  is right, the default) or `outcome` (it produced a result someone can check; `--evidence` is
+  required). New flags `--rationale` (`--note` still works), `--basis`, `--evidence`.
+- Approvals accumulate in `APPROVAL.json` under `approvals`, so two people approving, and a later
+  outcome, are all kept. The newest is mirrored at the top level for older readers; a pre-0.3.0 file
+  reads as one judgment approval.
+- `doctor` reports each golden's weight (`g1: 2 judgment, 1 outcome`) and says plainly when every
+  approval is judgment only. Informational: the level still needs one approved golden, since many
+  skills have no measurable outcome.
+- Tests: rationale and evidence refusals, legacy normalization, accumulation, and the doctor's
+  judgment-only finding appearing and clearing. Both new guards were mutated and went red.
+
 ## 0.2.2 (2026-09-29)
 
 - An inline flow map (`scope: { form: essay, audience: builders, purpose: persuade }`, `check: {

package/README.md

@@ -54,7 +54,7 @@ Every rule and threshold is in [SPEC.md](SPEC.md).
 | `superskill init <skill> --from-session <transcript>` | Turn the session where you did the job by hand into the first eval and a golden candidate (Claude Code `.jsonl`, or any text file as the request). |
 | `superskill miss <skill> "<what happened>" [--expected "..."]` | Log a time the skill got it wrong. |
 | `superskill fix <skill> <miss-id> --eval <id> [--commit <sha>]` | Close a miss. Refuses without an eval that exists. |
-| `superskill approve <skill> <golden>` | A person signs off on a golden. Terminal only, asks for your name, so an agent cannot approve its own output. |
+| `superskill approve <skill> <golden> [--basis judgment\|outcome] [--rationale ...] [--evidence ...]` | A person signs off on a golden, saying why and what it rests on: `judgment` (it reads right) or `outcome` (it produced a checkable result, with evidence). Terminal only, asks for your name, so an agent cannot approve its own output. Approvals accumulate. |
 | `superskill collection <folder...> [--budget <chars>] [--overlap 0.5]` | Listing budget used, descriptions that get cut off, pairs of skills an agent could confuse (with near-miss triggers to add). |
 | `superskill miss import <skill> --freedom-ledger [--ledger <file>]` | Import runs that needed correcting from Freedom's run ledger. |
 | `superskill snippet` | Print a block for `AGENTS.md` / `CLAUDE.md` that teaches any agent these habits. |

package/SPEC.md

@@ -1,6 +1,6 @@
 # The superskill standard
 
-**Version 0.2.2** (2026-09-29). The reference checker is `@supersuit/superskill`; where this
+**Version 0.3.1** (2026-09-29). The reference checker is `@supersuit/superskill`; where this
 document and the checker disagree, the checker has a bug.
 
 A **superskill** runs on frontier intelligence, is checked against examples a person approved,
@@ -97,7 +97,7 @@ A line in a bundled file containing `superskill-ignore` is skipped by `no-absolu
 
 | Rule | Severity | Threshold |
 |---|---|---|
-| `golden-approved` | fail | at least one golden has an `APPROVAL.json` with non-empty `approved_by` and a valid `approved_at`; info when it was approved against an earlier `SKILL.md` |
+| `golden-approved` | fail | at least one golden has an approval with non-empty `approved_by` and a valid `approved_at`; info when it was approved against an earlier `SKILL.md`; info naming the weight (judgment and outcome approvals per golden), and saying so plainly when no golden has an outcome yet |
 | `misses-log-present` | fail | `MISSES.md` exists (it may have no entries) |
 | `no-stale-open-miss` | fail | no miss has been open more than 14 days |
 | `fixed-miss-has-eval` | fail | every fixed miss names an eval id present in `evals.json` or `goldens/` |
@@ -160,9 +160,24 @@ A bare array of cases is accepted on read, as is `assertions` for `expectations`
 - `APPROVAL.json`, written only by `superskill approve` at an interactive terminal:
 
 ```json
-{ "approved_by": "Ann Example", "approved_at": "2026-09-10T15:00:00.000Z", "skill_sha": "<sha256 of SKILL.md>", "note": "Exactly the shape I send my manager." }
+{ "approvals": [
+  { "approved_by": "Ann Example", "approved_at": "2026-09-10T15:00:00.000Z", "skill_sha": "<sha256 of SKILL.md>",
+    "rationale": "Exactly the shape I send my manager.", "basis": "judgment" },
+  { "approved_by": "Ann Example", "approved_at": "2026-09-20T15:00:00.000Z", "skill_sha": "<sha256 of SKILL.md>",
+    "rationale": "My manager adopted it as the team template.", "basis": "outcome",
+    "evidence": "Sent 2026-09-19; adopted as the template in the team wiki" }
+] }
 ```
 
+**Every approval carries a rationale and a basis, because being liked and being proven are
+different weights.** `judgment`: a person read the output and says it is right. `outcome`: the
+output produced a result in the world that someone can check (a client landed, a call booked, a
+template adopted), and `evidence` says what happened and where to check it. Approvals accumulate:
+two people approving, and later an outcome, all stay on the record. The newest approval is also
+mirrored at the top level (`approved_by`, `approved_at`, `skill_sha`, `note`), so a reader written
+before 0.3.0 still sees it. A pre-0.3.0 file with a single approval reads as one `judgment`
+approval whose `note` is its rationale.
+
 A golden is also an eval: `--run` judges the skill's output for `input.md` against the approved
 output.
 

package/package.json

@@ -1,19 +1,42 @@
 {
   "name": "@supersuit/superskill",
-  "version": "0.2.2",
+  "version": "0.3.1",
   "description": "Score any agent skill folder as skill, tested, or superskill. An open standard and a zero-dependency CLI.",
   "type": "module",
-  "bin": { "superskill": "bin/superskill.mjs" },
+  "bin": {
+    "superskill": "bin/superskill.mjs"
+  },
   "exports": {
     "./yaml": "./src/frontmatter.mjs",
     "./package.json": "./package.json"
   },
-  "files": ["bin/", "src/", "SPEC.md", "README.md", "CHANGELOG.md", "LICENSE"],
-  "scripts": { "test": "node --test test/*.test.mjs" },
-  "engines": { "node": ">=20" },
+  "files": [
+    "bin/",
+    "src/",
+    "SPEC.md",
+    "README.md",
+    "CHANGELOG.md",
+    "LICENSE"
+  ],
+  "scripts": {
+    "test": "node --test test/*.test.mjs"
+  },
+  "engines": {
+    "node": ">=20"
+  },
   "license": "MIT",
-  "repository": { "type": "git", "url": "git+https://github.com/SupersuitUp/superskill.git" },
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/SupersuitUp/superskill.git"
+  },
   "homepage": "https://supersuit.wiki/concepts/superskill",
-  "keywords": ["agent-skills", "skills", "claude-code", "codex", "evals", "superskill"],
+  "keywords": [
+    "agent-skills",
+    "skills",
+    "claude-code",
+    "codex",
+    "evals",
+    "superskill"
+  ],
   "dependencies": {}
 }

package/src/commands/approve.mjs

@@ -4,13 +4,18 @@ import { join } from "node:path";
 import { createInterface } from "node:readline/promises";
 import { parseArgs, clock, UsageError } from "../args.mjs";
 import { skillDir, refuse } from "./common.mjs";
-import { readGoldens } from "../goldens.mjs";
+import { readGoldens, approvalEntry, withApproval, BASES } from "../goldens.mjs";
 
-export const help = `superskill approve <skill> <golden-id> [--note "<why it is right>"]
+export const help = `superskill approve <skill> <golden-id> [--rationale "<why it is right>"] [--basis judgment|outcome] [--evidence "<what happened, where to check>"]
 
 Record that a person checked goldens/<id>/ and signs off on its output. Works only at an
 interactive terminal and asks for your name, so an agent cannot approve its own output.
-Writes goldens/<id>/APPROVAL.json with the SKILL.md hash it was approved against.
+
+Every approval says WHY (--rationale, or asked) and WHAT IT RESTS ON (--basis, or asked):
+  judgment  you read it and it is right (the default)
+  outcome   it produced a result someone can check; --evidence is required
+Approvals accumulate: two people approving, or a judgment approval later backed by an outcome,
+all stay on the record. Writes goldens/<id>/APPROVAL.json with the SKILL.md hash.
 `;
 
 export async function run(argv) {
@@ -28,12 +33,18 @@ export async function run(argv) {
     process.stdout.write(`\n--- goldens/${id}/${g.outputFile} ---\n${g.output.slice(0, 2000)}${g.output.length > 2000 ? "\n[...]" : ""}\n---\n`);
     const name = (await rl.question("Your name (blank to cancel): ")).trim();
     if (!name) return refuse("not approved");
-    const note = a.flags.note || (await rl.question("Why is this right? (optional): ")).trim();
+    const rationale = String(a.flags.rationale || a.flags.note || (await rl.question("Why is this right? ")).trim());
+    let basis = a.flags.basis;
+    if (!basis) basis = /^o/i.test((await rl.question("What does this rest on: (j)udgment, you read it and it is right, or (o)utcome, it produced a result someone can check? [j] ")).trim()) ? "outcome" : "judgment";
+    const evidence = basis === "outcome" ? String(a.flags.evidence || (await rl.question("What happened, and where can someone check it? ")).trim()) : "";
     const sha = createHash("sha256").update(readFileSync(join(dir, "SKILL.md"))).digest("hex");
+    const made = approvalEntry({ name, rationale, basis, evidence, at: clock(a.flags).toISOString(), sha });
+    if (made.error) return refuse(made.error);
     const p = join(dir, "goldens", id, "APPROVAL.json");
     const existed = existsSync(p);
-    writeFileSync(p, JSON.stringify({ approved_by: name, approved_at: clock(a.flags).toISOString(), skill_sha: sha, note }, null, 2) + "\n");
-    process.stdout.write(`${existed ? "re-approved" : "approved"} goldens/${id}/ by ${name}\n`);
+    const prior = existed ? g.approval : null;
+    writeFileSync(p, JSON.stringify(withApproval(prior, made.entry), null, 2) + "\n");
+    process.stdout.write(`${existed ? "added an approval to" : "approved"} goldens/${id}/ by ${name} (${basis})\n`);
     return 0;
   } finally {
     rl.close();

package/src/goldens.mjs

@@ -1,5 +1,6 @@
 // goldens/<id>/: input.md, the approved output (output.md, or any other non-input file),
-// and APPROVAL.json {approved_by, approved_at, skill_sha, note} written by a person.
+// and APPROVAL.json written by a person: { approvals: [{approved_by, approved_at, skill_sha,
+// rationale, basis, evidence}] }, or the single-approval shape from before 0.3.0.
 import { readdirSync, readFileSync, existsSync, statSync } from "node:fs";
 import { join } from "node:path";
 
@@ -25,10 +26,58 @@ export function readGoldens(dir) {
       outputFile: outputName || null,
       output: outputName ? readFileSync(join(gdir, outputName), "utf8") : null,
       approval,
+      approvals: approvalsOf(approval),
       approvalError,
     });
   }
   return out;
 }
 
-export const isApproved = (g) => Boolean(g.approval && typeof g.approval.approved_by === "string" && g.approval.approved_by.trim() && g.approval.approved_at && !Number.isNaN(new Date(g.approval.approved_at).getTime()));
+/** What an approval rests on. `judgment`: the people who approved it read it and said it is right.
+ *  `outcome`: it produced a result in the world someone can check (a client landed, a call booked).
+ *  Being liked and being proven are different weights, and a golden says which it carries. */
+export const BASES = ["judgment", "outcome"];
+
+const validDate = (d) => Boolean(d) && !Number.isNaN(new Date(d).getTime());
+
+/** Every approval on a golden, normalized. Reads the 0.3.0 `{ approvals: [...] }` shape and the
+ *  single-approval shape before it (its `note` becomes the rationale, its basis judgment). */
+export function approvalsOf(approval) {
+  if (!approval || typeof approval !== "object") return [];
+  const list = Array.isArray(approval.approvals) ? approval.approvals : [approval];
+  return list
+    .filter((a) => a && typeof a.approved_by === "string" && a.approved_by.trim() && validDate(a.approved_at))
+    .map((a) => ({
+      approved_by: a.approved_by.trim(),
+      approved_at: a.approved_at,
+      skill_sha: a.skill_sha || null,
+      rationale: String(a.rationale ?? a.note ?? "").trim(),
+      basis: a.basis === "outcome" ? "outcome" : "judgment",
+      evidence: String(a.evidence ?? "").trim(),
+    }));
+}
+
+/** How much a golden's approval weighs: how many people vouched for it, and how many results it produced. */
+export function weightOf(g) {
+  const list = g.approvals || approvalsOf(g.approval);
+  return { judgment: list.filter((a) => a.basis === "judgment").length, outcome: list.filter((a) => a.basis === "outcome").length };
+}
+
+/** A new approval, or the reason it is refused. An approval says why, and an outcome says what happened. */
+export function approvalEntry({ name, rationale, basis = "judgment", evidence = "", at, sha }) {
+  if (!String(name || "").trim()) return { error: "not approved: no name" };
+  if (!String(rationale || "").trim()) return { error: "an approval says why the output is right; give a rationale" };
+  if (!BASES.includes(basis)) return { error: `basis must be one of ${BASES.join(", ")}` };
+  if (basis === "outcome" && !String(evidence || "").trim()) return { error: "an outcome approval needs evidence: what happened, and where someone can check it" };
+  return { entry: { approved_by: name.trim(), approved_at: at, skill_sha: sha, rationale: rationale.trim(), basis, ...(basis === "outcome" ? { evidence: evidence.trim() } : {}) } };
+}
+
+/** The APPROVAL.json to write: every earlier approval kept, the new one appended, and the newest
+ *  mirrored at the top level so a reader written before 0.3.0 still sees an approval. */
+export function withApproval(existing, entry) {
+  const approvals = [...approvalsOf(existing), entry];
+  const { approved_by, approved_at, skill_sha, rationale } = entry;
+  return { approved_by, approved_at, skill_sha, note: rationale, approvals };
+}
+
+export const isApproved = (g) => approvalsOf(g.approval).length > 0;

package/src/rules/superskill.mjs

@@ -2,7 +2,7 @@
 // got something wrong, and proven recently on a current model against the no-skill baseline.
 import { createHash } from "node:crypto";
 import { defineRules } from "./define.mjs";
-import { readGoldens, isApproved } from "../goldens.mjs";
+import { readGoldens, isApproved, weightOf } from "../goldens.mjs";
 import { readMisses } from "../misses.mjs";
 import { readEvals, readLatestRun } from "../evals.mjs";
 
@@ -33,6 +33,13 @@ export const superskillRules = defineRules([
       const sha = createHash("sha256").update(ctx.raw).digest("hex");
       const stale = approved.filter((g) => g.approval.skill_sha && g.approval.skill_sha !== sha).map((g) => g.id);
       if (stale.length) out.push(f("info", `golden${stale.length === 1 ? "" : "s"} ${stale.join(", ")} approved against an earlier SKILL.md`, "Re-run the golden and re-approve if the output still holds."));
+      // Liked is not proven. Say which weight the approvals carry, so "a person approved it" is
+      // never read as "it worked in the world".
+      const w = approved.map((g) => ({ id: g.id, ...weightOf(g) }));
+      const proven = w.filter((x) => x.outcome > 0);
+      const summary = w.map((x) => `${x.id}: ${x.judgment} judgment, ${x.outcome} outcome`).join("; ");
+      if (!proven.length) out.push(f("info", `approved on judgment only, no outcome recorded yet (${summary})`, "When a golden produces a real result, record it: `superskill approve <skill> <golden> --basis outcome --evidence \"<what happened, where to check>\"`."));
+      else out.push(f("info", `approval weight: ${summary}`, ""));
       return out;
     },
   },