@zalom/plastic 1.0.0-alpha.2 → 1.0.0-alpha.21

package/skills/creating-intent/references/lifecycle.md

@@ -0,0 +1,74 @@
+# Building an Intent — Full Lifecycle Detail
+
+## What → `## Intent` section
+
+The desire. One paragraph. What the human or agent wants.
+This exists from the moment the intent is created.
+
+**Deliverable:** `{ID}--{slug}.md`
+
+## Why → `## Context` + `### Decisions` sections
+
+Why this intent exists. Grows over time through brainstorming and exploration.
+
+- **Context** — what we knew going in + what we decided along the way
+- **Decisions** — main premises derived from Context plus decisions from brainstorming/grilling
+- Decisions are Why-level: "status belongs on actions because multiple workstreams", not How-level: "use ACTION_N.md files"
+
+**Deliverable:** `spec.md` (consolidated specification from Context + Decisions + brainstorming)
+
+## How → Planning and preparation
+
+Research decisions, create the implementation plan, define actions.
+
+**Deliverable:** `plan.md` + `actions/` + `checklist.md` (execution registry with checkboxes covering all actions)
+
+## Exec → Execute actions
+
+Execute actions from the plan, track progress via checklist.
+
+**Deliverable:** `outcome.md` (detailed result). `## Outcome` in intent.md = short summary written as last step.
+
+## `## Insights` — Append-only work log
+
+Captured throughout ALL stages. One-liner bullet points.
+Never modified, only appended.
+
+Tracks: stage transitions, decisions, shifts, blocks, cancellations, material for future intents.
+This is how execution is tracked. When this intent completes, Insights
+is where to look for what comes next. New intents spawned from Insights
+appear in the `chain` field.
+
+## `## Links`
+
+Wikilinks for Obsidian graph navigation. Human-facing counterpart to the
+frontmatter knowledge graph.
+
+## Conventions — Filesystem as Schema
+
+State is derived from what exists, not from what's declared.
+
+| Convention | Signal |
+|---|---|
+| No `## Context` | Intent is fleeting (quick capture, non-actionable) |
+| `## Context` has content | Intent is permanent (developed, actionable) |
+| `## Outcome` has content | Intent is done |
+| `## Insights` has `(autonomous)` entries | Intent is/was being delivered autonomously |
+
+### Transitions
+
+- Fleeting → permanent: add `## Context` (one-way, also makes it actionable)
+- There is no separate "non-actionable → actionable" transition — permanence implies actionability
+- Even research intents are actionable: the research itself is the action, the conclusion is the outcome
+
+## Creating an Intent — Full Steps
+
+1. Determine the target store: `~/.plastic/store/` for global intents (default), `~/.plastic/projects/{slug}/store/` for project intents
+2. Determine the Folgezettel ID: If root (no parent), find highest root number +1. If branch, run `"${CLAUDE_PLUGIN_ROOT}/scripts/folgezettel-id" <parent_id> <store_path>`
+3. Create the intent directory in the chosen store (e.g., `ID--three-to-five-words`)
+4. Create `{ID}--{slug}.md` with frontmatter (id, intent, sources, chain, created, author, tags)
+5. Write `## Intent` — the What
+6. Add remaining sections: `## Context`, `## Outcome`, `## Insights`, `## Links`
+7. Update the appropriate `INDEX.md` — add to Active section and appropriate cluster
+
+A fleeting intent can skip `## Context` — just `## Intent` and empty sections.

package/skills/creating-intent/references/wikilinks.md

@@ -0,0 +1,8 @@
+# Wikilink Conventions
+
+| Syntax | Meaning |
+|--------|---------|
+| `[[ID]]` | Link to intent in same store (e.g., `[[1a1]]`) |
+| `[[ID\|display text]]` | Link with human-readable label |
+| `[[global:ID]]` | Link to intent in `~/.plastic/store/` |
+| `[[project-slug:ID]]` | Link to intent in `~/.plastic/projects/{slug}/store/` |

package/skills/creating-project/SKILL.md

@@ -1,8 +1,8 @@
 ---
-name: plastic:creating-project
+name: plastic-creating-project
 description: >-
   Create a new project from an implementation intent. Sets up project directory,
-  git init, AGENTS.md with founding intent decisions, plastic:install --local,
+  git init, AGENTS.md with founding intent decisions, plastic-install --local,
   tactical mirror, projects.yml registration, and framework scaffolding.
   Use when an implementation intent spawns a project, or manually by user.
 ---
@@ -31,9 +31,9 @@ cd <project_root>/<slug>
 git init
 ```
 
-### 3. Run `plastic:install --local`
+### 3. Run `plastic-install --local`
 
-Invoke `plastic:install --local` in the project directory. This creates:
+Invoke `plastic-install --local` in the project directory. This creates:
 ```
 .plastic/
 ├── store/
@@ -164,3 +164,7 @@ Log in `## Insights` of each founding intent:
 
 Announce to user:
 > "Project `<slug>` created at `<path>`. AGENTS.md populated with [N] decisions from [founding intent IDs]. Tactical mirror `1` is now the active intent in the project store."
+
+## References
+
+- Read `references/hubs-projects.md` for the full hub/project relationship model, project creation flow, and cross-linking conventions

package/skills/creating-project/references/hubs-projects.md

@@ -0,0 +1,55 @@
+# Hubs and Projects
+
+## Hubs
+
+A Hub is a cloud of intents around related topics. Hubs emerge naturally from
+Folgezettel branching — intents that spawn in the same direction cluster.
+
+- A Hub can spawn a Project. The Hub holds the founding ideas.
+- A single intent can also spawn a Project.
+- Hub-spawned projects revolve around different ideas around related topics.
+- Intent-spawned projects revolve around the single founding intent.
+- A Project is the deliverable outcome of one or more intents.
+
+Hubs are represented as clusters in INDEX.md.
+
+## Projects — Full Detail
+
+A Project is a deliverable grouping of intents. Projects have two stores:
+
+- **Global store** (`~/.plastic/store/`): strategic intents
+- **Project store** (`~/.plastic/projects/{slug}/store/`): tactical intents
+
+`projects.yml` maps project slugs to codebase paths:
+```yaml
+projects:
+  plastic:
+    path: "/path/to/plastic"
+    remote: "git@github.com:org/plastic.git"
+    registered: '2026-05-26'
+    status: active
+```
+
+Config resolution: `~/.plastic/projects/{slug}/config.yml` overrides `~/.plastic/config.yml`.
+
+Cross-linking: project intents reference global intents via `[[global:ID]]`. Global intents reference project intents via `[[project-slug:ID]]`.
+
+## Privacy and Collaboration
+
+**Plastic is personal.** All intent data lives under `~/.plastic/` — one location,
+one git repo, never pushed. Each person has their own intent store.
+
+Collaboration happens through pull requests and project conventions, not shared intents.
+When an intent delivers something that changes how a project works, the decision gets
+written into the project's shared files (README, docs, config). The intents themselves
+are private working memory.
+
+## Project Creation Flow
+
+When an implementation intent spawns a project:
+1. Determine project path from config `project_roots` or intent context
+2. `gh repo create --private` (agent-created repos are always private by default)
+3. Set up project directory, git init, AGENTS.md with founding intent decisions
+4. Register in `projects.yml`
+5. Create tactical mirror in project store
+6. The global intent completes; the tactical mirror becomes the active intent

package/skills/dashboard/SKILL.md

@@ -0,0 +1,92 @@
+---
+name: plastic-dashboard
+description: Use when the user wants an overview of intents, asks "where are we", "what's next", "what should I work on", "show the dashboard", or invokes /plastic-dashboard. Renders a deterministic Value×Effort work cockpit across the global store and all projects, and emits a machine-readable queue that auto mode consumes.
+---
+
+# Dashboard — Plastic Work Cockpit
+
+A deterministic, template-driven overview of the intent store(s). It answers three
+questions at a glance — **where we are** (active + last touched), **where we go next**
+(a Value×Effort matrix), and **how to conduct it** (a disposition verb per intent) — and
+emits a JSON manifest that `plastic-auto` reads to pick the next dispatchable intent.
+
+The script does the rendering. The LLM is **never** in the rendering path: same store
+state → byte-identical output, regardless of model. Do not hand-summarize intents when
+this skill applies — run the script and show its output verbatim.
+
+## When to Use
+
+- User invokes `/plastic-dashboard`
+- User asks "where are we", "what's next", "what should I work on", "show me the intents"
+- `plastic-continuing` embeds the `continue` view on resume
+- `plastic-auto` reads `--json` to choose the next dispatchable intent
+
+## Procedure
+
+### Step 1 — Run the script
+
+```bash
+ruby ~/.plastic/scripts/dashboard.rb [continue|project <slug>|all] [--json]
+```
+
+| Mode | Shows |
+|------|-------|
+| `continue` (default) | Cross-scope: active + last touched, then the Value×Effort matrix for all scopes |
+| `project <slug>` | One project: active line + its Value×Effort matrix |
+| `all` | Per-scope roll-up summary |
+| `--json` | The auto-mode manifest (any mode); machine-readable, not for humans |
+
+The script is **read-only**. Print its stdout verbatim — do not reformat, re-sort, or
+re-summarize. That is what keeps the output uniform.
+
+### Step 2 — Read the matrix
+
+```
+                 small effort          big effort
+   high value    QUICK WIN             ★ NEXT BIG THING
+   low value     DEFER → agent         TRIAGE / question
+   (type=research/exploration → RESEARCH band, regardless of quadrant)
+```
+
+Disposition verbs: `▸ drive` (human leads), `⇢ defer` (agent knocks off),
+`⊙ research` (research/explore agent), `⚑ triage` (human review / maybe abandon).
+Flags: `⇡ unblocked` (a dependency just completed), `(Nd)` stale age.
+
+### Step 3 — Act on it
+
+- **★ Next big thing** and `▸ drive` / `⚑ triage` items → the human leads (brainstorm → plan → exec).
+- `⇢ defer` and `⊙ research` items → dispatchable to agents.
+- In auto mode, read `--json` and work `dispatchable_queue` in `rank` order; leave
+  `human_only` for the user.
+
+## JSON contract
+
+```json
+{ "generated_for": "auto-mode", "scope": "<scope|all>",
+  "next_big_thing": "<id|null>",
+  "dispatchable_queue": [ {"id","scope","disposition","type","value","effort","flags","rank"} ],
+  "human_only": ["<id>", "..."] }
+```
+
+## How classification works (deterministic)
+
+- **Effort** — small for `research`/`exploration`/`bugfix`, for already-scoped intents
+  (plan/checklist exists), or deep refinement branches; big otherwise.
+- **Value** — high only for an explicit `value: high` frontmatter field, or a
+  human-authored root intent that has spawned follow-on work (`chain` non-empty); low otherwise.
+- **Override** — a `value: high|normal|low` field in an intent's frontmatter wins. This is
+  the only place model judgment enters, and only as pre-stamped data (never at render time).
+
+## Eval
+
+This skill's eval is **render the template**: run the engine against the test fixture
+store and assert byte-identical text + JSON output against the golden snapshots in
+`test/fixtures/dashboard/`. See `test/dashboard_test.rb`. If output drifts from the
+golden files without an intentional template change, the skill is broken.
+
+## Notes
+
+- Clusters (Zettelkasten grouping in INDEX.md) are intentionally **not** rendered — they
+  are orthogonal to "what to work on next."
+- Glyphs are monochrome Unicode (no emoji); they render in standard terminal emulators.
+- This is additive: it changes no core lifecycle, gate, or cycle logic.

package/skills/doctor/SKILL.md

@@ -0,0 +1,116 @@
+---
+name: plastic-doctor
+description: Use when diagnosing Plastic installation health, after updates, or when something seems broken. Runs checks and reports findings with fix options.
+---
+
+# Doctor — Plastic Health Check
+
+## When to Use
+
+- User invokes `/plastic-doctor`
+- After `plastic-update` completes (automatically)
+- When hooks aren't firing, skills aren't loading, or something seems broken
+- When the user says "check plastic", "diagnose", "what's wrong with plastic"
+
+## Procedure
+
+### Step 1: Run the diagnostic script
+
+```bash
+ruby ~/.plastic/scripts/doctor.rb --agent claude
+```
+
+Replace `claude` with the current agent type if known (`codex`, `hermes`).
+
+Parse the JSON output from stdout. The script is read-only and never modifies
+files. Errors go to stderr.
+
+Exit codes indicate check results, not script failure:
+- `0` — all checks passed
+- `1` — warnings found
+- `2` — failures found
+
+All three exit codes mean the script ran successfully. Do not treat non-zero
+as an error.
+
+### Step 2: Determine overall status
+
+Read the `status` field from the JSON root:
+
+| Status | Meaning |
+|--------|---------|
+| `pass` | Everything is healthy |
+| `warn` | Warnings found but Plastic works normally |
+| `fail` | Blocking issues that prevent Plastic from operating |
+
+### Step 3: Fill the report template
+
+Read `report.md` from the same directory as this SKILL.md
+(`~/.plastic/skills/doctor/report.md` at runtime, or the plugin source
+`skills/doctor/report.md` during development).
+
+Group checks by category. For each category, list the checks with their
+status icon and message. If a check has `details`, list them as sub-items.
+
+Present the filled template to the user.
+
+### Step 4: Offer fixes (if applicable)
+
+If any checks have `fixable: true` AND status is not `pass`:
+
+1. Group fixable items by category
+2. Show what each fix would do (from `fix_hint`)
+3. Ask the user: **"Fix all / Select individually / Skip"**
+
+If no fixable issues exist, skip this step.
+
+### Step 5: Apply fixes
+
+Use the `fix_hint` value to determine the correct action:
+
+| Fix hint pattern | Agent action |
+|---|---|
+| "chmod +x on the listed files" | Run `chmod +x` on each file listed in `details` |
+| "Create missing directory" | Run `mkdir -p` on the path |
+| "Create INDEX.md with required sections" | Write INDEX.md with the 5 sections: Active, Future, Clusters, Abandoned, Completed |
+| "Add missing entries to INDEX.md" | Add orphaned intents to the appropriate INDEX.md section |
+| "Remove stale references from INDEX.md" | Edit INDEX.md to remove ghost references |
+| "Re-run installer" | Run `npx @zalom/plastic@latest --agent` |
+
+For fixes the agent cannot handle automatically, explain what the user needs
+to do manually.
+
+### Step 6: Verify
+
+After applying fixes, re-run the diagnostic script:
+
+```bash
+ruby ~/.plastic/scripts/doctor.rb --agent claude
+```
+
+Show the updated results.
+
+- If all checks pass: announce success.
+- If issues remain: explain what is still wrong and what the user can do.
+
+## Post-Update Mode
+
+When invoked from `plastic-update` (not directly by the user):
+
+1. Run the diagnostic script as in Step 1.
+2. If all checks pass: show a single line — **"Health check: all clear."**
+3. If issues are found: show the full report (Steps 3-6).
+
+This keeps the update flow clean when nothing is wrong.
+
+## Important Notes
+
+- The script is **read-only**. It inspects but never modifies files.
+  All fixes are performed by the agent using standard tools.
+- The script outputs JSON to stdout. Any diagnostic errors go to stderr.
+- Non-zero exit codes mean "issues found", not "script crashed".
+  Always parse stdout regardless of exit code.
+
+## References
+
+- Read `references/gates-stuck-detection.md` for the full gate enforcement table, bridge file pattern, and stuck detection thresholds when diagnosing gate failures or stuck agents

package/skills/doctor/references/gates-stuck-detection.md

@@ -0,0 +1,38 @@
+# Gate Enforcement and Stuck Detection
+
+## Bridge File Pattern
+
+`/tmp/plastic-{session}.json` is the hot cache; the filesystem is the authority.
+SessionStart rebuilds the bridge file from the filesystem — the bridge is always disposable.
+
+## Gate Taxonomy
+
+| Gate type | Purpose |
+|---|---|
+| **Pre-flight** | Prerequisites exist? |
+| **Revision** | New info invalidated prior work? |
+| **Escalation** | Blocked items surface to user |
+| **Abort** | Inconsistent state detected |
+
+## Full Gate Enforcement
+
+| Gate | Blocked action | Required prerequisite |
+|---|---|---|
+| Pre-flight | Cannot write `plan.md` | `spec.md` must exist |
+| Pre-flight | Cannot create `actions/` | `spec.md` must exist |
+| Pre-flight | Cannot write `outcome.md` | `checklist.md` must exist with all items checked |
+| Revision | Cannot proceed to Exec | Context changed after spec.md — re-derive spec |
+| Abort | Cannot complete intent | `outcome.md` exists but checklist has unchecked items |
+
+Hard blocking — hooks exit with code 2 when gates fail.
+
+## Stuck Detection
+
+| Condition | Threshold | Action |
+|---|---|---|
+| Consecutive gate failures | 3+ | Warning |
+| Consecutive gate failures | 5+ | Force savepoint + escalate |
+| No activity | 5+ min | Warning |
+| No activity | 10+ min | Force savepoint + escalate |
+| Context pressure | 80% | Warning |
+| Context pressure | 90% | Force savepoint |

package/skills/doctor/report.md

@@ -0,0 +1,96 @@
+# Plastic Doctor Report
+
+<!-- =======================================================================
+  AGENT INSTRUCTIONS -- How to fill this template
+  =========================================================================
+  1. Run the doctor script. It outputs JSON with check results.
+  2. Replace every {{placeholder}} below with the corresponding JSON value.
+  3. For the category sections: the template shows ONE example section.
+     Repeat that pattern for each unique category in the checks array.
+     The six known categories and their display names are:
+       global_store       -> "Global Store"
+       conventions        -> "Conventions"
+       agent_registration -> "Agent Registration"
+       core_files         -> "Core Files"
+       project_stores     -> "Project Stores"
+       deprecations       -> "Deprecations"
+  4. For each check within a category, emit one line with the status icon
+     and the check message. If the check has non-empty details, list them
+     as indented sub-items.
+  5. The "Fixable Issues" section should ONLY appear if at least one check
+     has fixable=true AND status is not "pass". Omit the entire section
+     otherwise.
+  6. Status icons (plain text, no emoji):
+       pass -> [PASS]
+       warn -> [WARN]
+       fail -> [FAIL]
+  7. The overall status icon in the header uses the same mapping.
+  8. After filling, remove all HTML comments -- they are instructions only.
+  ======================================================================= -->
+
+## {{overall_status_icon}} Overall: {{status}} -- Plastic v{{version}}
+
+Checked at: {{timestamp}}
+
+### Summary
+
+{{pass}} passed, {{warn}} warnings, {{fail}} failed -- {{total}} checks total
+
+---
+
+<!-- =====================================================================
+  CATEGORY SECTIONS
+  ======================================================================
+  Repeat the block below ONCE PER CATEGORY present in the checks array.
+  Group checks by their "category" field. Use the display name mapping
+  above for the heading. Within each category, list every check as a
+  single line: status icon + message. If a check has non-empty "details",
+  list each detail as an indented bullet beneath.
+
+  Example category section (for agent_registration with two checks):
+  ===================================================================== -->
+
+### Agent Registration
+
+- [PASS] Claude Code adapter registered
+- [FAIL] 2 hook scripts not executable
+  - ~/.claude/hooks/plastic-session-start
+  - ~/.claude/hooks/plastic-gate-check
+
+<!-- =====================================================================
+  Repeat the above pattern for each category found in the checks array.
+  Only include categories that have at least one check.
+  Order categories as they appear in the checks array.
+  ===================================================================== -->
+
+---
+
+<!-- =====================================================================
+  FIXABLE ISSUES SECTION
+  ======================================================================
+  Include this section ONLY if one or more checks have fixable=true AND
+  status is "warn" or "fail". If no fixable issues exist, omit everything
+  from the "Fixable Issues" heading through the end of the horizontal
+  rule that follows the table.
+
+  For each fixable check that is not "pass", emit one table row:
+    | status_icon | check_message | fix_hint |
+  ===================================================================== -->
+
+### Fixable Issues
+
+| Status | Issue | Fix |
+|--------|-------|-----|
+| [FAIL] | 2 hook scripts not executable | chmod +x on the listed files |
+
+<!-- =====================================================================
+  Repeat one row per fixable non-pass check.
+  ===================================================================== -->
+
+---
+
+<!-- =====================================================================
+  FOOTER -- always include this line exactly as written.
+  ===================================================================== -->
+
+Run `plastic doctor --fix` to auto-fix all fixable issues, or ask me to fix them now.

package/skills/evaluating-skills/SKILL.md

@@ -0,0 +1,140 @@
+---
+name: plastic-evaluating-skills
+description: >
+  Evaluate Plastic skills for correctness, convention compliance, and
+  progressive disclosure. Use when testing whether a skill produces good
+  outputs, verifying convention compliance after changes, running evals
+  against skills or instructions, creating evals for a new or updated
+  skill, checking if a description triggers correctly, or assessing
+  whether a skill is still needed. Also use when the user says "evaluate",
+  "test the skill", "run evals", "check conventions", or "write evals".
+---
+
+# Evaluating Skills
+
+Eval methodology for Plastic skills, based on
+[agentskills.io](https://agentskills.io/skill-creation/evaluating-skills)
+and [Anthropic's eval guide](https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents).
+
+## Gotchas
+
+- Assertions written before observing output are almost always wrong — run
+  the eval first, observe actual output, THEN write assertions
+- Near-miss negative test cases are the most valuable — prompts that share
+  keywords with should-trigger cases but need a different skill entirely
+- Select the best skill iteration by validation pass rate, not the last one
+- Grade outcomes, not execution paths — if the agent solved the task via an
+  unexpected route but produced correct output, that is a pass
+- Same skill can behave differently across agent frameworks — test on each
+  target agent (Claude Code, Hermes, OpenClaw, Codex)
+
+## Procedure
+
+### Step 1: Choose eval scope
+
+Determine what you are evaluating:
+
+- **Description triggering** — does the agent activate the right skill for
+  a given prompt? Tests the description field effectiveness.
+- **Output quality** — does the skill produce correct results when activated?
+  Tests the skill body and references.
+- **Convention compliance** — does the output follow Plastic conventions?
+  Read `references/convention-checks.md` for the full assertion library.
+
+Multiple scopes can apply to the same skill. Start with the scope that
+addresses your immediate concern, add others as needed.
+
+### Step 2: Design test cases
+
+Create `evals/evals.json` in the skill being evaluated. Copy the starter
+template from `assets/eval-template.json` in this skill.
+
+**For description triggering:**
+- Write ~20 queries: 8-10 should-trigger, 8-10 should-not-trigger
+- Split 60/40 into train and validation sets (proportional mix in each)
+- Include near-miss negatives that share keywords but need a different skill
+- In `expected_output`, describe whether the skill should or should not activate
+  and why
+
+**For output quality:**
+- Start with 2-3 test cases, expand after first results
+- Use realistic user prompts with varied phrasing, detail level, and formality
+- In `expected_output`, describe what correct output looks like — not exact text
+- Use `files` array for any input files the test needs
+
+**For convention compliance:**
+- Start with 2-3 test cases targeting specific convention areas
+- In `expected_output`, describe which conventions must be met
+- Read `references/convention-checks.md` for the full assertion library
+
+Leave `assertions` arrays empty. They are populated after Step 4.
+
+### Step 3: Run paired evals
+
+Dispatch a subagent per test case to ensure clean context — no leakage
+between test runs. Run each case twice:
+
+1. **With skill** — the skill is available and loaded
+2. **Without skill** — the skill is not available (baseline comparison)
+
+The delta between with-skill and without-skill measures what the skill adds.
+If the delta is negligible, the skill may not be adding value for that case.
+
+Grade outcomes, not paths. An unexpected tool-call sequence that produces
+correct output is still a pass.
+
+### Step 4: Write assertions after observing
+
+Review actual outputs from Step 3. Write specific, verifiable assertions
+based on what you observed — not what you expected beforehand.
+
+Choose the grader type that fits each assertion:
+- **Code-based** — structure, file existence, format validity, counts
+- **LLM-as-judge** — quality, completeness, tone, semantic correctness
+- **Human** — edge cases, calibration, judgment calls
+
+Read `references/eval-methodology.md` for the full grader taxonomy and
+LLM-judge calibration protocol.
+
+Good assertions are specific, verifiable, and countable:
+"Output includes at least 3 concrete recommendations."
+
+Weak assertions are vague: "Output is good."
+Brittle assertions use exact phrase matching.
+
+Require concrete evidence for PASS. No benefit of the doubt.
+
+### Step 5: Grade and iterate
+
+Compute pass rates per test case and aggregate across the eval suite.
+
+Track two metrics separately:
+- **pass@k** — succeeded at least once in k trials (capability)
+- **pass^k** — succeeded every time in k trials (reliability)
+
+Read `references/eval-methodology.md` for formulas and interpretation.
+
+Three signal sources for skill improvement:
+1. **Failed assertions** — specific gaps in the skill
+2. **Human feedback** — broader quality issues not captured by assertions
+3. **Execution transcripts** — reveals WHY things went wrong
+
+Feed all three plus the current SKILL.md to propose targeted changes.
+Iterate on the train set only. Check the validation set for generalization.
+5 iterations is usually enough. If not improving after 5, the test cases
+themselves may be the problem — revisit Step 2.
+
+### Step 6: Graduate and monitor
+
+Once a skill hits ~100% on capability evals (pass@k = 1.0 for 3+
+consecutive runs):
+
+1. **Graduate** capability evals into regression tests — run them on every
+   skill change to protect against backsliding
+2. **Monitor** the with/without delta over time — if it shrinks to zero
+   across 3+ runs, the model may have internalized the skill
+3. **Retire** cautiously — archive the skill, do not delete. Re-test after
+   model updates in case capabilities regress.
+
+Read `references/eval-methodology.md` for graduation criteria, retirement
+detection, and the three-layer eval taxonomy.

package/skills/evaluating-skills/assets/eval-template.json

@@ -0,0 +1,12 @@
+{
+  "skill_name": "",
+  "evals": [
+    {
+      "id": 1,
+      "prompt": "",
+      "expected_output": "",
+      "files": [],
+      "assertions": []
+    }
+  ]
+}