@renzrollon/interlock 1.0.0

package/skills/evals/SKILL.md

@@ -0,0 +1,160 @@
+---
+name: evals
+description: Author an Interlock eval case from an observed model-behaviour failure, and read a completed eval results run. Use when you have a reproduced failure — an archived proposal, a changelog entry, a run artifact, or a transcript — worth regression-testing against a real model, or when an eval run has finished and you need to know whether it is a regression, run-to-run variance, or no signal. Authoring needs cited evidence; the verdict comes from `interlock evals triage`, never from your own reading.
+license: MIT
+compatibility: Requires the early-access eval harness (`claude plugin eval`, enabled with CLAUDE_CODE_WALNUT_SPIRE=1) to RUN a suite. Authoring a case and triaging an existing results file need only Node.js >= 18 and the bundled interlock CLI.
+allowed-tools: Bash(interlock *) Bash(claude plugin eval *) Read Grep Glob Write
+metadata:
+  type: authoring
+  outputs:
+    - evals/<case>/case.yaml
+    - evals/<case>/graders/*.md
+---
+
+This skill covers the two jobs the eval harness does not do for you: turning an
+observed failure into a case, and reading a finished run. Everything with a
+correct answer — is this a regression? — belongs to `interlock evals triage`, not
+to you. Your job is evidence, grader choice, and explanation.
+
+## 1. Authoring a case — only from an observed failure
+
+A case regression-tests a **reproduced** failure: a model read correct
+instructions and did the wrong thing anyway. Before you write anything, require
+the evidence:
+
+- an archived proposal (`openspec/changes/archive/*/proposal.md` `## Why`),
+- a `CHANGELOG.md` entry,
+- a run artifact or a transcript,
+- a **captured skeleton** — `interlock evals capture --run <runId> [--task <id>]
+  --out <dir>` turns a recorded ship-run trajectory into a case draft whose
+  provenance names the trajectory and the events it was derived from, or
+- a `file:line` span in the prompt or skill under test that shows the surface
+  the model talked past.
+
+**A captured skeleton is a draft, not a finished case.** It is admissible
+evidence because it is a run artifact and a reader can follow its citation back
+to the trajectory. But every value capture derived rather than read carries a
+`CONFIRM` marker — the grader pattern when no value could be quoted, and the
+prompt when the trajectory recorded none. **Resolve every `CONFIRM` marker
+against the cited trajectory, or ask the user, before authoring the case. Never
+author a case that still carries one**, and never move a skeleton into `evals/`
+unresolved: an unconfirmed pattern is a regression test for a failure nobody has
+shown happened.
+
+**If no observed failure is offered, say so and stop.** Do not author a
+speculative case. A suite of hypotheticals teaches the team to ignore it, which
+is the one thing an advisory gate cannot survive. A case that only duplicates an
+existing `node:test` assertion is refused for the same reason — the unit suite
+already covers it deterministically.
+
+Record the citation in the case (`provenance:` in `case.yaml`, plus a comment
+explaining what went wrong). The structural gate `test/evals.test.mjs` fails a
+case with no provenance.
+
+### The case files
+
+Every case pins `schema_version` and lives in its own directory under `evals/` —
+never under `skills/`, `agents/`, `commands/` or `workflows/`, which would shadow
+a component. Follow `evals/handoff-status-enum/` as the format exemplar; it was
+written to establish these conventions. Where a case needs a fixture workspace,
+use `case.yaml` with `context.scaffold_script` and run with `--scaffold`;
+otherwise the same `case.yaml` shape without it.
+
+## 2. Choosing a grader — cheapest that expresses the assertion
+
+The harness has six grader types. Four are free and deterministic — `regex`,
+`tool_used`, `tool_order`, `file_exists` — and two call a judge — `llm`,
+`baseline`.
+
+**Pick the cheapest grader that expresses the assertion.** If the assertion is
+set-membership, tool-invocation, file-existence, or ordering shaped, it is
+deterministic and a judged grader is the wrong tool — more expensive, and prone
+to run-to-run variance triage will (correctly) refuse to call a regression. The
+strongest seed case, `handoff-status-enum`, needs no judge at all: the status is
+in a fixed set, so a `regex` decides it.
+
+**When you do reach for a judged grader, state in the grader body which part of
+the assertion could not be expressed deterministically.** If you cannot name
+that part, the assertion was deterministic and you chose the wrong grader.
+
+A case you intend to run per-pull-request tag `smoke`, and a smoke case uses
+**only** deterministic graders — the gate enforces this. A grader that asserts a
+skill *fired* (`tool_used: Skill`) is marked `arm: with-only`: it is a
+plugin-fired indicator, reported against the baseline arm, not part of the score.
+
+## 3. Reading a run — triage owns the verdict
+
+Given a finished results file, **get the classification from
+`interlock evals triage --results <file>`, do not derive it yourself.** Its exit
+code is the verdict: `0` pass, `1` regression, `2` no signal, `3` configuration.
+This is deliberate — the one gate whose rule a model could re-argue on each run
+would be exactly the wrong gate to ship in a suite built to catch models
+re-arguing rules.
+
+Your contribution is **explanation, not classification**:
+
+- When triage reports a **regression**, explain which recent change to the
+  surface under test plausibly caused it — the prompt sentence, the skill edit,
+  the contract change. Do not reclassify it as variance because it looks flaky;
+  triage already separated a single judged dip (variance) from a judged majority
+  (regression).
+- When triage reports **variance**, report it as variance and say what would
+  confirm it (further runs failing the same judged grader).
+- When triage reports **no signal** — a partial run, an interrupted run, a
+  rejected credential — report that evals could not run. Do not present any case
+  as passing or failing.
+
+Never restate, override, or soften a verdict triage gave you.
+
+### Read a transcript before you explain a judged case
+
+**For every case whose score depends on a judged grader, read at least one
+transcript for that case before you write your explanation of it.** A score is
+the outcome; the trace is the only place the cause is visible. The failure this
+closes is a model that satisfied a judge by narrowing the task rather than by
+doing it — the score moves, every other step in the loop sees a number, and
+nobody sees why.
+
+Reading is for explanation and **never for reclassification**. If the transcript
+suggests a different classification than triage produced, report what you saw in
+the trace and leave the verdict alone — the rule above still holds without
+exception.
+
+A run whose graders are all deterministic needs no transcript: the tool trace the
+grader already read is the evidence.
+
+Your report carries a `transcripts read:` line, always present:
+
+```
+transcripts read: <case>/<id>, <case>/<id>
+transcripts read: none
+transcripts read: unavailable, because <what stopped you reading one>
+```
+
+Use the **none** form when you read no transcript — never omit the line, because
+a reader cannot otherwise tell an unexamined result from an examined one. Use the
+**unavailable, because …** form when the run produced no readable transcript, and
+in that case do not present your explanation as transcript-grounded. Degradation
+is spoken.
+
+## 4. A failing case is a report, never a prompt edit
+
+When a case fails, your output is a report: the case, the grader that failed, and
+the sentence in the surface under test that plausibly caused it. **Do not edit
+the skill, shared contract, agent definition, or prompt under test to make a case
+pass.** Changing the surface under test is a separate, explicit decision the user
+makes — silently editing a prompt to turn a case green is exactly the "edit the
+test until it passes" failure the suite exists to prevent, one layer up.
+
+## 5. When the harness is gated
+
+`claude plugin eval` prints `` `plugin eval` is currently in early access `` and
+does nothing until `CLAUDE_CODE_WALNUT_SPIRE=1` is set in the environment — never
+in committed `.claude/settings.json`, which would produce a suite that looks
+configured and does not run.
+
+When the harness refuses because early access is not enabled, **report the
+prerequisite and where to set it** (the environment, or the CI job's `env:`
+block). Do not report an empty run as a passing suite. Authoring a case and
+triaging an existing results file both work without the harness enabled — only
+running a suite needs it.

package/skills/explain-code/SKILL.md

@@ -0,0 +1,42 @@
+---
+name: explain-code
+description: Explain code to a beginner in that stack — either a module walkthrough (what each function does, how calls link, where data flows, which patterns recur) or a commit teach-in (why each file changed, what changed, blast radius, what would break if it were left out). Use when asked to explain code, walk through a directory, or explain the last commit.
+license: MIT
+compatibility: Requires git for commit teach-in mode. Optional openspec CLI for change scoping.
+argument-hint: "[path | --commit | --commits N]"
+allowed-tools: Bash(git *) Bash(interlock-graph *) Bash(openspec *) Read Write Grep Glob
+metadata:
+  type: teaching
+---
+
+Teach code to someone new to this stack. **Not reviewing, not judging — teaching.**
+
+Calibrate to a reader who knows programming fundamentals but is new to this project's language and framework. Explain the idioms that stack takes for granted, one short aside each. Do not explain what a function is.
+
+## Modes
+
+| Mode | When | Load |
+|------|------|------|
+| **Module walkthrough** (default) | A file or directory is named, or the user asks how an area works | `${CLAUDE_SKILL_DIR}/module-walkthrough.md` |
+| **Commit teach-in** | Last commit, a hash or range, "what changed", `--commit` / `--commits N` | `${CLAUDE_SKILL_DIR}/commit-teachin.md` |
+
+When ambiguous, prefer the module walkthrough unless the user mentioned a commit or "what changed".
+
+**After choosing a mode, read the matching sibling file and follow it** for steps, structure, and output format. This file only routes.
+
+## Output path
+
+Resolve the destination before writing. Both modes use the same rule:
+
+| Situation | Write to |
+|---|---|
+| Scoped to an OpenSpec change — a change name was passed, or one resolves via `openspec status --json` | `openspec/changes/<change-name>/code-explanation.md` |
+| No change context (ad-hoc walkthrough) | `docs/CODE_EXPLANATION.md` |
+
+**Do not "simplify" this back to a single path.** A change-scoped explanation belongs beside `manual-test-plan.md` in its own change directory. Writing every run to `docs/CODE_EXPLANATION.md` overwrites the previous change's explanation — and because `docs/` is the corpus `/interlock:docs-digest` indexes, each rewrite permanently marks `.claude/graph/DOCS_DIGEST.md` stale through its `source_hashes`.
+
+## Shared rules
+
+- **Single agent, no fan-out.** Understanding builds sequentially; parallel investigators produce four disconnected fragments, which is the opposite of a walkthrough.
+- Write the full explanation to the resolved path (overwrite), then tell the user the file is ready with a 5–10 line summary.
+- Pull domain context on demand: `.claude/graph/DOCS_DIGEST.md` first, then `interlock-graph context` / `docs`. Never preload all of `docs/` — see `${CLAUDE_PLUGIN_ROOT}/shared/TOOL-ECONOMY.md` Rule 0.5.

package/skills/explain-code/commit-teachin.md

@@ -0,0 +1,136 @@
+# Commit teach-in — explain-code
+
+Teach what changed and why — not a code review.
+
+## Scope
+
+Every source file touched in the commit range (skip binaries/lockfiles; cap ~25). Skip tests unless they are the only files and the user asked about them.
+
+## Steps
+
+1. Retrieve domain context on demand — Read `.claude/graph/DOCS_DIGEST.md` if present; then `interlock-graph context` / `docs` as needed (`${CLAUDE_PLUGIN_ROOT}/shared/TOOL-ECONOMY.md` Rule 0.5).
+2. Resolve range: default `HEAD~1..HEAD`; `--commits N` → `HEAD~N..HEAD`; or an explicit hash/range from the user.
+3. Gather:
+```bash
+git log -1 --format='%H%n%s%n%b'   # or log for the range
+git diff --stat "$FROM" "$TO"
+git diff --name-status "$FROM" "$TO"
+git diff "$FROM" "$TO" -- <each source file>   # cap hunks ~300 lines/file
+```
+4. **Blast radius per file** — files that reference the import-specifier stem (basename without extension). For generic stems (`index`, `route`, `page`, `layout`, `config`, `types`, `utils`, `constants`, `main`, `mod`, `SKILL`, `README`, `PROMPT`) qualify with parent directory. Prefer `rg -l` / `git grep -l`; exclude self. Optional: `interlock-graph consumers <symbol>` when the graph exists. Name-match may false-positive — note incidental hits.
+5. For each changed source file, fill the **Change Context** block (required). Optionally add function breakdown only for complex new/changed logic a beginner needs.
+6. Write Commit Summary, Per-File Explanation, Change Flow, and Beginner Takeaway; write to the output path resolved in `SKILL.md`:
+   - Scoped to an OpenSpec change (name passed, or an active change resolves via `openspec status --json`) → `openspec/changes/<change-name>/code-explanation.md`, alongside `manual-test-plan.md`
+   - No change context → `docs/CODE_EXPLANATION.md`
+   - Per-change output is deliberate: one shared `docs/` file would overwrite the previous change's explanation and permanently mark `.claude/graph/DOCS_DIGEST.md` stale via its `source_hashes`.
+
+## Change Context (required)
+
+### Change Context (required in commit teach-in)
+
+Use this block for every changed source file in commit mode. Status = Added / Modified / Deleted / Renamed from `git diff --name-status`.
+
+```markdown
+#### `path/to/file.ext` — <Added | Modified | Deleted | Renamed>
+
+**Why this file needed to change:**
+Plain-language reason this file was part of the commit. Tie it back to the commit's
+goal — what would be incomplete without touching this file?
+
+**What changed:**
+- Bullet the concrete edits from the diff (new function, changed signature, added
+  field, deleted branch…). Skip boilerplate (import reordering, whitespace).
+- Explain any TypeScript / React / Next.js concept a beginner wouldn't know,
+  in one short aside each.
+
+**Blast radius:**
+- Who is affected downstream, based on the referencing files gathered for this path.
+- Rate it: **Isolated** (self-contained), **Local** (a few callers in one area),
+  or **Wide** (shared util/type/config touched by many files).
+- Name the specific files most likely to feel the change and how.
+
+**If this file had been left out:**
+- What would break, fail to compile, or behave wrong if this file's change were
+  missing from the commit but every other file's change shipped?
+- Be concrete: name the type error, runtime failure, broken flow, or silent bug.
+- If the file is genuinely independent (nothing else depends on the change),
+  say so plainly — "nothing would break; this change is self-contained."
+```
+
+Rules for Change Context:
+- Teach, don't judge. No code-review verdicts, no "should have" critiques.
+- Ground every claim in the diff and blast-radius data — do **not** invent callers, functions, or failure modes you can't see.
+- Be concise but complete — every changed source file gets its own block.
+
+## Output Format
+
+### Commit teach-in
+
+Match the explain-code loop structure (no review verdicts):
+
+```markdown
+# Code Explanation: commit <short-hash> — <subject>
+
+## Commit Summary
+
+One or two sentences, in plain language: what does this commit accomplish and why?
+Reference the commit subject but translate any jargon for a beginner.
+
+---
+
+## Per-File Explanation
+
+[One Change Context block per changed source file — see above]
+
+---
+
+## Change Flow
+
+If the commit spans multiple layers (e.g. type → data layer → component → route),
+draw the dependency order as a short chain:
+
+`types/batch.ts (new field)` → `lib/db.ts (reads field)` → `app/api/.../route.ts` → `components/BatchCard.tsx`
+
+One line per chain. Write `(single-layer — no cross-layer flow)` if N/A.
+
+---
+
+## Beginner Takeaway
+
+One sentence naming the single most important concept a beginner should learn
+from studying this commit (e.g. "shared types are a coupling point — changing one
+field ripples through every consumer").
+```
+
+If there are no commits to explain: write only `No commits to explain yet.` and stop.
+
+## Tone & Style
+
+## Tone & Style
+
+- **Friendly, encouraging, zero condescension** — "we" language, like a senior sitting next to you
+- **Skip the obvious** — don't explain what `const x = 5` does
+- **Explain the non-obvious** — why this pattern? why server vs client? why does this call SQS here?
+- **Name every external service call** — never skip an infra interaction, these are critical for understanding the system
+- **Use analogies** — "Think of SQS like a to-do list for a background worker — we drop a note saying 'process this batch', and a separate service picks it up later"
+- **Show the full journey** — from user action → through the code → to the database/service → back to the UI
+- **Keep it scannable** — headers, bullet points, tables, code blocks. No walls of text.
+
+---
+
+## What NOT to Do
+
+- Don't review or critique the code
+- Don't suggest improvements (unless asked)
+- Don't explain import statements, simple variable declarations, or trivial types
+- Don't explain test files
+- Don't use jargon without defining it on first use
+- Don't be condescending ("as you probably know" / "this is simple")
+- Don't explain node_modules or third-party library internals (just say what the library does)
+- Don't skip infrastructure calls — every DB query, S3 operation, SQS message, and auth check must appear in the flow
+- Don't use subagents or workflows — this skill requires sequential understanding across files
+- In commit teach-in, don't skip Change Context fields — every changed source file needs Why / What / Blast radius / If left out
+- Don't invent blast-radius callers or breakage modes not evidenced by the diff or reference search
+
+- In commit teach-in, don't skip Change Context fields — every changed source file needs Why / What / Blast radius / If left out
+- Don't invent blast-radius callers or breakage modes not evidenced by the diff or reference search