@vegastack/skills 0.1.0 → 0.2.0

package/skill/skillify/SKILL.md

@@ -0,0 +1,154 @@
+---
+name: skillify
+description: Repo-local skill factory and auditor for the vegastack-skills monorepo. Use when asked to "skillify this", "make this a skill", "turn this workflow into a skill", or scaffold a new skill under skills/; when asked to "audit this skill", "is this skill complete", "check skill completeness", or score a skill against the repo contract; when deciding whether a workflow, script, or prompt should become a skill at all; when a skill description under- or over-triggers and needs tuning; or when a skill in this repo is missing tests, a README, a refresh contract, behavioral eval evidence, or repo wiring. Only for skills inside this repository, not for authoring skills in other projects or installing third-party skills.
+---
+
+# Skillify
+
+Turn a raw workflow into a properly-skilled unit of this monorepo, or audit an existing `skills/<name>/` tree against the full repo contract (CONTRIBUTING.md, "Adding a new skill"). Skillify is repo-only: it creates and scores skills in this repository, not anywhere else. Deep wiring/release mechanics belong to `skill-maintainer`; cross-reference it rather than restating it.
+
+## The completeness checklist
+
+Score every item pass / fail / N/A. N/A requires a one-line rationale; N/A without one is a fail.
+
+1. **SKILL.md spec-compliant** — frontmatter is exactly `name` + `description`; name matches the directory and the grammar (starts with a lowercase letter, `[a-z0-9-]`, no consecutive hyphens, max 64); description max 1024 chars with no angle brackets; body under 500 lines; no harness-specific body syntax. `bun run validate:skill` agrees.
+2. **Description triggers well** — states triggering conditions only (never the workflow), third person, trigger words front-loaded, per [authoring](references/authoring.md); a should/should-not-trigger query set with near-miss negatives exists at `tests/fixtures/trigger-queries.json`.
+3. **README walkthrough** — repo-side `README.md`: install, contents table, behavior contract.
+4. **References routed** — `references/` for on-demand detail with a routing table in SKILL.md; or N/A for a self-contained skill.
+5. **Scripts deterministic** — dependency-free Node under `scripts/`, `--json` for machine output, documented exit codes, atomic writes and symlink refusal for anything mutating; or N/A.
+6. **Assets** — templates/schemas/examples under `assets/`; or N/A.
+7. **Unit tests + fixtures** — `tests/*.test.ts` (bun test) covering every deterministic branch; fixtures under `tests/fixtures/`.
+8. **Consistency test** — a test asserting prose, scripts, templates, and registry agree: relative links resolve, documented flags exist, template inventory matches the scaffolder.
+9. **Behavioral eval evidence** — with-skill vs baseline subagent runs on 2–3 realistic prompts per the [eval playbook](references/eval-playbook.md); summary in the PR body; unresolved gaps in a KNOWN_GAPS section of the skill's README.
+10. **Refresh contract** — `refresh/sources.json` + `refresh/REFRESH.md` tracking every volatile claim; or an explicit evergreen waiver in REFRESH.md stating why nothing in the skill decays.
+11. **Codex metadata** — `agents/openai.yaml`.
+12. **Repo wiring** — sync-skill allowlist entry, root README table row, CHANGELOG entry (`skill-maintainer` owns the detail).
+13. **Green check** — `bun run check` passes.
+
+**Verdict:** all pass → `properly skilled`. At most three misses, none of items 1, 7, or 13 → `close — create: <missing items>`. Otherwise → `needs skillify — run skillify on <target>`. Always report the score as `<passed>/13` plus the verdict.
+
+## Phase 0 — Should this be a skill?
+
+Before anything else, check:
+
+- Will it be invoked 2+ times? One-off work is not a skill.
+- Is there more than ~20 lines of logic or judgment? Trivial helpers do not need the full contract.
+- Is there a trigger phrase a user would actually say?
+
+If any answer is no: it is a script or a doc, not a skill. Stop — do not scaffold, do not write a SKILL.md. Say why and move on.
+
+Scope upper bound: **one skill = one capability = one coherent trigger family.** If the target spans distinct intents users would invoke separately ("run the build" / "roll back the deploy" / "notify the team" are three intents), do not build one skill covering them all. Propose the split and ask which target to skillify first.
+
+## Phase 1 — Audit
+
+For an existing skill, score the checklist against the actual tree and stop with the verdict:
+
+```
+Skill: <name>            Path: skills/<name>/
+Score: <passed>/13       Verdict: <verdict>
+Missing: <item>: <one-line evidence> ...
+```
+
+For a new skill the audit is trivially 0/13 — proceed. When the request was only "audit" / "is this complete", deliver the verdict and the shortest path to `properly skilled`; do not start editing unasked.
+
+## Phase 2 — Elicit requirements
+
+Interview before writing (skip questions the conversation already answers):
+
+- **Triggers:** what exact phrases should invoke this? What near-miss requests should NOT? (These become the query set.)
+- **Output:** what does done look like — files, report shape, side effects?
+- **Edge cases:** empty input, missing config, offline, partial state?
+- **Existing behavior:** is there code/prose to absorb? What did its author correct over time?
+
+Then classify every factual claim the skill will make:
+
+| Class | Meaning | Where it lives |
+|---|---|---|
+| Durable | versionless principle | references prose; refresh never edits |
+| Mechanism-coupled | durable intent through a vendor-named mechanism | marked `<!-- source: SOURCE-ID -->`; edited only with registry evidence |
+| Volatile | version pins, limits, model names, URLs | `refresh/sources.json` entries or refresh-tracked files only |
+
+Every mechanism-coupled or volatile claim needs a registry source — or the skill declares an evergreen waiver. No unclassified claims ship.
+
+## Phase 3 — Scaffold and write
+
+Scaffold the contract tree (dry-run first; `--write` to create; `--json` for machine output):
+
+```sh
+node <skill-dir>/scripts/scaffold-skill.mjs <name> --dir <repo-root>
+node <skill-dir>/scripts/scaffold-skill.mjs <name> --dir <repo-root> --write
+```
+
+The scaffolder validates the name grammar, refuses existing directories and symlinks, stages in a temp sibling then renames, and prints the manual wiring steps. Then write, in this order:
+
+1. *Description + trigger query set* — engineer the description per [authoring](references/authoring.md); write 8–10 should-trigger and 8–10 near-miss should-not-trigger queries into `tests/fixtures/trigger-queries.json`.
+2. *SKILL.md body* — workflow, one excellent example, routing table. Respect the token budgets in [authoring](references/authoring.md).
+3. *References* — detail that only some invocations need.
+4. *Scripts* — only for work that is deterministic and repeated (criteria in [authoring](references/authoring.md)).
+5. *Draft tests* — but do not treat them as the quality bar yet; that is Phase 4's job.
+
+## Phase 4 — Behavioral eval — the quality gate
+
+**Tests lock in behavior. If the behavior is mediocre, tests lock in mediocrity.** Prove quality first, then let tests cement it.
+
+Follow the [eval playbook](references/eval-playbook.md): for each of 2–3 realistic prompts, launch two subagents in the same turn — one told to follow the new SKILL.md, one baseline without it — and compare outputs against the skill's claimed value. This is a procedure you execute with your own subagents, never a custom eval harness or tooling.
+
+Iterate at most 3 cycles: eval → apply the top improvements to the skill → re-eval. Pass, or ship with a KNOWN_GAPS section (format in the playbook). If the baseline already matches the with-skill output, the skill is not earning its tokens — cut it or narrow it.
+
+## Phase 5 — Lock in
+
+Now that quality is proven:
+
+1. Finalize unit tests and fixtures locking in the eval-proven behavior; add the consistency test (item 8).
+2. Write `refresh/sources.json` + `refresh/REFRESH.md` from the Phase 2 claim classification — or the evergreen waiver.
+3. Finish `README.md` and `agents/openai.yaml`.
+4. Wire the repo: sync-skill allowlist, root README row, CHANGELOG entry — as printed by the scaffolder; `skill-maintainer` documents each step in depth.
+
+## Phase 6 — Verify
+
+```sh
+node packages/cli/scripts/validate-skill.mjs skills/<name>
+bun test skills/<name>
+bun run check
+```
+
+Re-score the checklist and report `<passed>/13` with the verdict. Anything below `properly skilled` ships only with named gaps.
+
+## Worked example: skillifying a "release-notes" workflow
+
+```
+Phase 0: yes — run at every release, ~80 lines of conventions, trigger "draft the release notes"
+Phase 1: 0/13 (new)
+Phase 2: triggers "draft/write the release notes", NOT "write a changelog entry for this PR";
+  output = CHANGELOG section + npm summary; claims: changeset flow = durable,
+  npm publish mechanics = mechanism-coupled (source NPM-PUBLISH), CLI version pin = volatile
+Phase 3: scaffold-skill.mjs release-notes --dir . --write; description + 18 trigger queries;
+  body + references/format.md; no scripts (judgment-heavy, N/A with rationale)
+Phase 4: eval cycle 1 — baseline subagent invents section headings, with-skill misses breaking-change
+  callouts → add callout contract to SKILL.md; cycle 2 — with-skill clearly better, baseline still
+  wrong shape → pass
+Phase 5: tests lock the section contract; sources.json gets NPM-PUBLISH; wiring done
+Phase 6: bun run check green; 13/13 → properly skilled
+```
+
+## Anti-patterns
+
+- Writing lock-in tests before the behavioral eval — locks in mediocrity.
+- A description that summarizes the workflow — agents follow the description and skip the body.
+- Skipping the baseline run because "the output looks fine" — that is not evidence.
+- Eval without a fix cycle — vanity metrics.
+- Obviously-irrelevant negatives in the trigger query set — near-misses or nothing.
+- Version pins or model names in SKILL.md — volatile facts live in refresh-tracked locations.
+- Multi-intent skills spanning unrelated triggers — split them.
+- Two skills answering the same trigger — merge or kill one.
+- Prose and scripts disagreeing about flags or paths — that is what the consistency test catches.
+- Building eval tooling — evals are instructions you run with subagents, not code you write.
+
+## Routing
+
+| Need | Read |
+|---|---|
+| description engineering, numeric limits, token budgets, script-vs-instructions, claim classes | [authoring](references/authoring.md) |
+| eval method, pass criteria, trigger-query doctrine, cycle protocol, KNOWN_GAPS, model guidance | [eval playbook](references/eval-playbook.md) |
+| the scaffolded starting points | `assets/templates/` |
+| skillify's own freshness stance | [refresh/REFRESH.md](refresh/REFRESH.md) |

package/skill/skillify/agents/openai.yaml

@@ -0,0 +1,4 @@
+interface:
+  display_name: "VegaStack Skillify"
+  short_description: "Create and audit skills for the vegastack-skills repo"
+  default_prompt: "Use $skillify to audit or scaffold a skill in skills/ against the full repo contract."

package/skill/skillify/assets/templates/README.md.template

@@ -0,0 +1,26 @@
+# {{name}}
+
+TODO: one-paragraph summary for humans and agents browsing the repo. The agent entry point is [SKILL.md](SKILL.md); everything else loads progressively from there.
+
+## Install
+
+```sh
+npx @vegastack/skills add {{name}}
+```
+
+## What's in this skill
+
+| Path | Purpose |
+|---|---|
+| [SKILL.md](SKILL.md) | Agent entry point |
+| `references/` | TODO: on-demand detail, or delete this row |
+| `scripts/` | TODO: deterministic dependency-free Node scripts, or delete this row |
+| `assets/` | TODO: templates/schemas/examples, or delete this row |
+| [refresh/sources.json](refresh/sources.json) | Source registry for volatile claims |
+| [refresh/REFRESH.md](refresh/REFRESH.md) | Freshness contract |
+| [agents/openai.yaml](agents/openai.yaml) | Codex interface metadata |
+| `tests/` | Bun tests and fixtures (never packaged) |
+
+## Behavior
+
+TODO: what the skill does when invoked, its output contract, and its guardrails.

package/skill/skillify/assets/templates/REFRESH.md.template

@@ -0,0 +1,21 @@
+# Refresh contract — {{name}}
+
+Instructions for the scheduled refresh agent (and any human running a manual refresh). This file plus `sources.json` is the complete freshness contract for this skill.
+
+## What this skill claims
+
+TODO: classify before shipping (see skillify references/authoring.md):
+
+- **Durable rules**: versionless principles in this skill's prose. The refresh agent NEVER edits these; if a source change invalidates one, open an issue quoting the evidence.
+- **Mechanism-coupled claims**: sentences marked with a source comment naming a SOURCE-ID from `sources.json`. The agent may propose edits to these marked sentences only, in the same PR as the registry update that evidences the change.
+- **Volatile layer**: `refresh/sources.json` itself (checksums, versions, retrieval times) — the only file the agent edits freely.
+
+## Evergreen waiver
+
+TODO: if this skill has NO mechanism-coupled or volatile claims, replace the sources TODO in `sources.json` with this waiver, filled in: "This skill's content is evergreen: it asserts no version pins, vendor mechanism names, numeric vendor limits, or dated facts. Rationale: ... Revisit if a future edit introduces any." Then delete the refresh steps below. Otherwise delete this section.
+
+## How to refresh
+
+1. TODO: deterministic pass (which command, which registry).
+2. TODO: what may be auto-accepted vs what needs human review.
+3. One PR per refresh, evidence-linked; refresh branches touch only `refresh/` metadata.

package/skill/skillify/assets/templates/SKILL.md.template

@@ -0,0 +1,26 @@
+---
+name: {{name}}
+description: TODO - state only the conditions that should load this skill. Third person, trigger phrases in quotes first, symptoms and situations next, one clause naming what this skill is NOT for. Never summarize the workflow. Max 1024 characters, no angle brackets.
+---
+
+# {{name}}
+
+TODO: one-paragraph overview - what this skill guarantees and the single trigger family it serves.
+
+## When to use
+
+- TODO: concrete situations and user phrasings
+- TODO: when NOT to use - the nearest adjacent intent and where it goes instead
+
+## Workflow
+
+1. TODO
+2. TODO
+
+## Example
+
+TODO: one excellent, realistic example (input and output). One great example beats five mediocre ones.
+
+## Routing
+
+TODO: if this skill has references/, add a routing table (Need | Read) here; otherwise delete this section and record item 4 as N-A in the checklist with a rationale.

package/skill/skillify/assets/templates/openai.yaml.template

@@ -0,0 +1,4 @@
+interface:
+  display_name: "{{name}}"
+  short_description: "TODO: imperative, under 60 characters"
+  default_prompt: "Use ${{name}} to TODO."

package/skill/skillify/assets/templates/skill.test.ts.template

@@ -0,0 +1,40 @@
+import { describe, expect, test } from 'bun:test'
+import { existsSync, readFileSync } from 'node:fs'
+import { join, resolve } from 'node:path'
+import { validateSkill } from '../../../packages/cli/scripts/validate-skill.mjs'
+
+const skillRoot = resolve(import.meta.dir, '..')
+
+describe('{{name}} contract', () => {
+  test('SKILL.md passes repo validation', () => {
+    const result = validateSkill(skillRoot)
+    expect(result.message).toBe('Skill is valid!')
+    expect(result.ok).toBe(true)
+  })
+
+  test('frontmatter carries only name and description', () => {
+    const content = readFileSync(join(skillRoot, 'SKILL.md'), 'utf8')
+    const frontmatter = /^---\n([\s\S]*?)\n---/.exec(content)
+    expect(frontmatter).not.toBeNull()
+    const keys = (frontmatter as RegExpExecArray)[1]
+      .split('\n')
+      .filter(line => /^[A-Za-z]/.test(line))
+      .map(line => line.split(':')[0])
+    expect(keys.sort()).toEqual(['description', 'name'])
+  })
+
+  test('relative markdown links resolve', () => {
+    for (const file of ['SKILL.md', 'README.md']) {
+      const body = readFileSync(join(skillRoot, file), 'utf8')
+      for (const match of body.matchAll(/\]\(([^)#\s]+)\)/g)) {
+        const target = match[1]
+        if (/^[a-z][a-z0-9+.-]*:/.test(target)) continue
+        expect(`${file} -> ${target}: ${existsSync(join(skillRoot, target))}`).toBe(`${file} -> ${target}: true`)
+      }
+    }
+  })
+
+  // TODO: add unit tests for every deterministic branch of scripts/, plus the
+  // consistency test (prose, scripts, templates, and registry agree). Lock in
+  // eval-proven behavior only AFTER the behavioral eval passes (skillify Phase 4).
+})

package/skill/skillify/assets/templates/sources.json.template

@@ -0,0 +1,6 @@
+{
+  "schemaVersion": 1,
+  "retrievalBaseline": "{{date}}",
+  "note": "TODO: add one entry per volatile or mechanism-coupled external claim (see skillify references/authoring.md, claim classification), or record an evergreen waiver in REFRESH.md and keep sources empty.",
+  "sources": []
+}

package/skill/skillify/references/authoring.md

@@ -0,0 +1,72 @@
+# Authoring reference
+
+Rules for writing the parts of a skill that determine whether it triggers, how much it costs, and whether it stays true over time.
+
+## Description engineering
+
+The description is the primary triggering mechanism: it is always in context, and the agent decides from it alone whether to load the body.
+
+- **Triggering conditions only — never the workflow.** A description that summarizes the process becomes a shortcut: agents follow the summary and skip the body. State when to load the skill, not what the skill will do step by step.
+- **Third person.** The text is injected into a system prompt ("Creates and audits...", "Use when..."), never "I can help you...".
+- **Front-load trigger words.** Harness skill lists truncate long descriptions; the first clause must carry the strongest triggers.
+- **Cover the ways users actually ask.** Exact phrases in quotes, symptoms, file types, adjacent phrasings, casual variants. Include the situations where the skill competes with a neighbor and should win.
+- **Name the boundary.** One clause on what the skill is NOT for prevents over-triggering ("Only for skills inside this repository...").
+- **No angle brackets** — the repo validator rejects them.
+
+Bad: `Helps with skills.` (no triggers, no boundary)
+Bad: `Audits a skill by scoring 13 checklist items, then scaffolds, evals, and wires it.` (workflow summary — the body will be skipped)
+Good: `Use when asked to "skillify this", "make this a skill", audit a skill's completeness, or decide whether a workflow should become a skill. Only for skills inside this repository.`
+
+## Numeric limits
+
+<!-- mirrored: these numbers are volatile and mirrored from the standards sources tracked by the skill-maintainer registry; on drift, fix them there first, then here. -->
+
+| Thing | Limit |
+|---|---|
+| `name` | 1–64 chars, starts with a lowercase letter, `[a-z0-9-]`, no consecutive hyphens, no leading/trailing hyphen, must equal the directory name |
+| `description` | 1–1024 chars |
+| Frontmatter keys | `name` + `description` only in this repo (spec also allows `license`, `compatibility`, `metadata`, `allowed-tools`; unknown keys hard-error on claude.ai packaging) |
+| SKILL.md body | under 500 lines / ~5k tokens |
+| Listing budgets | Claude truncates listed name+description around 1,536 chars; Codex caps the whole skill list at ~2% of context / 8,000 chars — front-load triggers |
+| Relative references | one level deep, plain relative paths, no harness-specific syntax in the body |
+
+## Trigger query sets
+
+Every skill ships `tests/fixtures/trigger-queries.json`: an array of `{"query": "...", "should_trigger": true|false}`.
+
+- 8–10 should-trigger: different phrasings of the same intent — formal, casual, typo-ridden; cases that never name the skill but clearly need it; cases where a neighboring skill competes and this one should win.
+- 8–10 should-NOT-trigger: **near-misses only.** Queries sharing keywords or domain with the skill but needing something else — adjacent intents, ambiguous phrasing a naive keyword match would catch. "Write a fibonacci function" as a negative for a PDF skill tests nothing.
+- Queries must be realistic: concrete detail, file paths, a little backstory — what a user would actually type, not abstract category labels.
+
+The set is both a design artifact (it forces the description to draw a real boundary) and eval input (see the [eval playbook](eval-playbook.md)).
+
+## Token economy
+
+Three loading levels; spend accordingly:
+
+1. **Metadata (name + description)** — in every conversation, always. ~100 tokens. Every word must earn its place.
+2. **SKILL.md body** — loaded on every trigger. Keep the workflow, one excellent example, and a routing table; push everything else down a level. One great example beats five mediocre ones.
+3. **references/ / scripts/ / assets/** — loaded or executed on demand; effectively unlimited, but give each reference a clear routing condition so agents read only what the task needs.
+
+Cross-reference other skills by name (`skill-maintainer`) instead of restating their content. Don't document a script's flags in prose beyond what routing needs — the script's own `--help`/usage error is the source of truth, and the consistency test keeps the two honest.
+
+## Scripts vs instructions
+
+**Deterministic and repeated → script. Judgment → instructions.**
+
+- If two invocations with the same input must produce the same bytes, that is a script.
+- Watch eval/test transcripts for agents independently rewriting the same helper — three subagents each writing their own `parse_frontmatter` is a strong signal the skill should bundle that script once.
+- Repo conventions for bundled scripts: dependency-free Node (`.mjs`), runnable from the skill dir with plain relative paths, `--json` for machine-readable output, documented exit codes (0 ok / 1 finding-or-refusal / 2 usage), atomic writes (stage then rename), symlink refusal, and an explicit `--write` gate for anything mutating — dry-run is the default.
+- Do NOT script judgment: descriptions, verdicts, review prose. A script that fakes judgment produces confident garbage.
+
+## Claim classification for refresh
+
+Classify every factual claim while writing, not after:
+
+| Class | Test | Treatment |
+|---|---|---|
+| **Durable** | still true if every vendor renames everything | plain prose; refresh automation never touches it |
+| **Mechanism-coupled** | durable intent expressed through a vendor-named mechanism (an option name, an API shape) | mark the sentence `<!-- source: SOURCE-ID -->`; editable only alongside registry evidence for that source |
+| **Volatile** | version pins, numeric limits, model names, URLs, anything with a date | lives in `refresh/sources.json` entries or other refresh-tracked files — never inline in SKILL.md |
+
+The refresh contract falls out of this table: each distinct SOURCE-ID becomes a registry entry with URLs, checksum, threshold; REFRESH.md states which files the refresh agent may edit (the volatile layer), which sentences it may propose edits to (marked ones), and which it must only flag (durable). A skill with no mechanism-coupled or volatile claims declares an **evergreen waiver** in REFRESH.md — one paragraph stating why nothing decays — and keeps `sources: []`.

package/skill/skillify/references/eval-playbook.md

@@ -0,0 +1,62 @@
+# Behavioral eval playbook
+
+How to prove a skill actually improves agent behavior before tests cement it. This is a procedure you execute with your own subagents (Task tool or equivalent) — never a custom eval harness, script, or viewer. If you find yourself writing eval tooling, stop; write instructions instead.
+
+## Why eval before tests
+
+Tests lock in behavior. If the behavior is mediocre, tests lock in mediocrity. The eval proves the quality bar first; only then do unit tests cement the proven-good behavior. The corollary: if you didn't watch an agent fail *without* the skill, you don't know the skill teaches anything.
+
+## Method: with-skill vs baseline
+
+1. **Pick 2–3 realistic prompts.** The kind of thing a user would actually type — concrete, with file paths and context, exercising the skill's hardest documented use case, not its happy path. Reuse the strongest entries from `tests/fixtures/trigger-queries.json` where they fit.
+2. **Launch both runs per prompt in the same turn.** One subagent instructed to read and follow the new SKILL.md at its path; one baseline subagent given the identical prompt with no mention of the skill. (When improving an existing skill, the baseline is a snapshot of the old version instead.) Same-turn launch keeps the comparison honest and fast.
+3. **Capture outputs to a scratch workspace** (never committed): `<scratch>/eval/<prompt-id>/{with_skill,baseline}/`.
+4. **Compare against the skill's claimed value**, reading transcripts, not just final outputs.
+
+## Pass criteria
+
+The with-skill run must be *materially* better on the dimensions the skill exists for:
+
+- Output has the contract shape the skill defines (sections, verdicts, file layout) where the baseline's does not.
+- The with-skill agent uses bundled scripts/templates instead of reinventing them; note any helper both runs wrote independently — that helper should become a bundled script.
+- No contract violations the checklist would catch (frontmatter keys, name grammar, missing pieces).
+- The transcript shows the skill's guidance being *used*, not just loaded — sections nobody used are bloat to cut.
+
+**Failure that matters most:** the baseline matches the with-skill output. Then the skill is not earning its tokens — cut it, narrow it, or fold it into a reference.
+
+## Trigger-query eval (description level)
+
+The body eval above assumes the skill got loaded. Separately check the description triggers correctly: walk `tests/fixtures/trigger-queries.json` and judge, for each query, whether an agent seeing only the skill list (name + description) would load this skill. Near-miss negatives are the valuable half — a description that survives them draws a real boundary. Fix failures by editing the *description* (triggers, boundary clause), never by stuffing workflow summary into it.
+
+## Cycle protocol (at most 3)
+
+```
+CYCLE 1: run evals → list concrete improvements → apply them to the skill files
+CYCLE 2: re-run the SAME prompts → compare before/after → apply what remains
+CYCLE 3: re-run → pass, or ship with KNOWN_GAPS
+```
+
+Hard stop after 3 cycles. Endless polishing is worse than an honest gap list. Generalize from feedback — fix the pattern that caused the failure, don't overfit wording to one prompt.
+
+## KNOWN_GAPS format
+
+Ships at the bottom of the skill's README.md when the eval did not fully pass:
+
+```markdown
+## KNOWN_GAPS
+
+- gap: baseline-quality section ordering under multi-file inputs
+  evidence: eval cycle 3, prompt "draft release notes for the 0.4 tag"
+  why unresolved: would require restructuring the routing table
+  next: revisit when references/format.md is split by artifact type
+```
+
+One entry per gap: `gap` / `evidence` (cycle + prompt) / `why unresolved` / `next`. The eval summary itself (prompts used, cycles run, verdict per prompt) goes in the PR body, not the repo.
+
+## Model guidance
+
+<!-- volatile: everything in this section decays; keep model facts out of SKILL.md and update here only. -->
+
+- Run both arms of every comparison on the **same model** — otherwise you measure the model gap, not the skill.
+- Use the model that will actually run the skill in production (typically the one powering the current session); a weaker grader cannot judge a stronger writer.
+- If a separate grading pass is used, grade blind: give the grader both outputs unlabeled and ask which better satisfies the skill's stated contract.

package/skill/skillify/refresh/REFRESH.md

@@ -0,0 +1,24 @@
+# Refresh contract — skillify
+
+Instructions for the scheduled refresh agent (and any human running a manual refresh). This file plus `sources.json` is the complete freshness contract for this skill.
+
+## What this skill claims
+
+- **Durable rules** (SKILL.md, most of `references/`): the phase model, checklist structure, eval-before-tests ordering, description doctrine, claim-classification taxonomy. Versionless; the refresh agent NEVER edits these. If external evidence invalidates one, open an issue quoting the evidence — do not edit.
+- **Mirrored volatile claims**: skillify does not track its own external sources. Its two volatile spots are marked in place:
+  - the numeric limits and harness listing budgets table in `references/authoring.md` (marked `<!-- mirrored -->`);
+  - the model guidance section in `references/eval-playbook.md` (marked `<!-- volatile -->`).
+
+  Both mirror facts whose sources of truth are the agent-skills standards sources tracked in the **skill-maintainer** registry (`skills/skill-maintainer/refresh/sources.json`). `sources` here is therefore deliberately empty, and the refresh runner treats this registry as valid with nothing to fetch.
+
+## How to refresh
+
+1. There is no deterministic fetch pass for this skill — an automated run against this registry selects zero sources and exits 0. Do not add sources here that duplicate skill-maintainer's; one registry per fact.
+2. When skill-maintainer's standards sources drift (spec limits, harness listing budgets, model catalogs), that skill's refresh PR is the evidence. Update skillify's two marked sections to match **on a normal branch** (refresh branches are CI-restricted to `refresh/` metadata), referencing the skill-maintainer refresh PR as evidence.
+3. If a future edit gives skillify a directly-owned external claim (its own vendor URL, pin, or protocol fact), stop mirroring for that claim: add a real entry to `sources.json` and mark the sentence `<!-- source: SOURCE-ID -->`.
+
+## Never
+
+- Never edit the phase model, checklist items, or verdict grammar in a refresh.
+- Never hand-write checksum/version/timestamp values anywhere in this repo.
+- Never duplicate a skill-maintainer source entry into this registry.

package/skill/skillify/refresh/sources.json

@@ -0,0 +1,6 @@
+{
+  "schemaVersion": 1,
+  "retrievalBaseline": "2026-08-08T00:00:00Z",
+  "note": "Deliberately empty. Skillify's only time-decaying claims (agent-skill numeric limits and harness listing budgets in references/authoring.md; model guidance in references/eval-playbook.md) are mirrored from the standards sources tracked in the skill-maintainer registry and are refreshed through that skill's registry, not duplicated here. See REFRESH.md.",
+  "sources": []
+}

package/skill/skillify/scripts/scaffold-skill.mjs

@@ -0,0 +1,138 @@
+#!/usr/bin/env node
+// Scaffold a new skill tree at skills/<name>/ from skillify's templates.
+//
+//   node scripts/scaffold-skill.mjs <skill-name> --dir <repo-root> [--write] [--json]
+//
+// Dry-run by default: prints the plan (files that would be created plus the
+// remaining manual wiring steps) and creates nothing. --write stages the tree
+// in a temporary sibling inside skills/ and renames it into place, refusing
+// existing directories and symlinks. Exit codes: 0 ok, 1 refusal or failure,
+// 2 usage error.
+import { lstat, mkdir, mkdtemp, readFile, rename, rm, writeFile } from 'node:fs/promises'
+import { dirname, join, resolve } from 'node:path'
+import { fileURLToPath } from 'node:url'
+
+const here = dirname(fileURLToPath(import.meta.url))
+const templatesRoot = resolve(here, '..', 'assets', 'templates')
+
+// Template file -> output path inside skills/<name>/ (null = tests/<name>.test.ts).
+export const templateFiles = [
+  ['SKILL.md.template', 'SKILL.md'],
+  ['README.md.template', 'README.md'],
+  ['sources.json.template', 'refresh/sources.json'],
+  ['REFRESH.md.template', 'refresh/REFRESH.md'],
+  ['openai.yaml.template', 'agents/openai.yaml'],
+  ['skill.test.ts.template', null],
+]
+
+// Full grammar shared by the repo validator and every target harness: starts
+// with a lowercase letter, then lowercase letters/digits/hyphens, no
+// consecutive hyphens, no trailing hyphen, at most 64 characters.
+export function validateName(name) {
+  if (typeof name !== 'string' || name.length === 0) return 'skill name is required'
+  if (name.length > 64) return `name is ${name.length} characters; the maximum is 64`
+  if (!/^[a-z]/.test(name)) return 'name must start with a lowercase letter'
+  if (!/^[a-z0-9-]+$/.test(name)) return 'name may contain only lowercase letters, digits, and hyphens'
+  if (name.includes('--')) return 'name must not contain consecutive hyphens'
+  if (name.endsWith('-')) return 'name must not end with a hyphen'
+  return null
+}
+
+export function wiringSteps(name) {
+  return [
+    `Add skills/${name}/ packaged files to the allowlist in packages/cli/scripts/sync-skill.mjs (the build fails loudly on unlisted files)`,
+    `Add a ${name} row to the Skills table in the root README.md`,
+    `Add a CHANGELOG entry (changeset) introducing ${name}`,
+  ]
+}
+
+async function entryAt(path) {
+  try {
+    return await lstat(path)
+  } catch {
+    return null
+  }
+}
+
+export async function scaffoldSkill({ name, dir, write = false, now = new Date() }) {
+  const nameError = validateName(name)
+  if (nameError) throw new Error(`Invalid skill name ${JSON.stringify(name ?? null)}: ${nameError}`)
+  if (!dir) throw new Error('--dir <repo-root> is required')
+  const repoRoot = resolve(dir)
+  const skillsRoot = join(repoRoot, 'skills')
+  const skillsEntry = await entryAt(skillsRoot)
+  // lstat does not follow symlinks, so a symlinked skills/ fails isDirectory().
+  if (!skillsEntry || !skillsEntry.isDirectory()) {
+    throw new Error(`${skillsRoot} is not a real directory - point --dir at the vegastack-skills repo root`)
+  }
+  const target = join(skillsRoot, name)
+  if (await entryAt(target)) throw new Error(`Refusing to scaffold: ${target} already exists`)
+
+  const outputs = templateFiles.map(([source, output]) => [source, output ?? `tests/${name}.test.ts`])
+  const plan = { name, target, files: outputs.map(([, output]) => output), wiring: wiringSteps(name), wrote: false }
+  if (!write) return plan
+
+  const date = now.toISOString().slice(0, 10)
+  const staging = await mkdtemp(join(skillsRoot, `.${name}.scaffold-`))
+  try {
+    for (const [source, output] of outputs) {
+      const body = await readFile(join(templatesRoot, source), 'utf8')
+      const rendered = body.replaceAll('{{name}}', name).replaceAll('{{date}}', date)
+      const destination = join(staging, output)
+      await mkdir(dirname(destination), { recursive: true })
+      await writeFile(destination, rendered)
+    }
+    if (await entryAt(target)) throw new Error(`Refusing to scaffold: ${target} already exists`)
+    await rename(staging, target)
+  } catch (error) {
+    await rm(staging, { recursive: true, force: true })
+    throw error
+  }
+  return { ...plan, wrote: true }
+}
+
+function parseArguments(argv) {
+  const options = { name: undefined, dir: undefined, write: false, json: false }
+  const rest = [...argv]
+  while (rest.length) {
+    const flag = rest.shift()
+    if (flag === '--dir') {
+      const value = rest.shift()
+      if (value === undefined || value.startsWith('-')) throw new Error('--dir requires a value')
+      options.dir = value
+    } else if (flag === '--write') options.write = true
+    else if (flag === '--json') options.json = true
+    else if (flag.startsWith('-')) throw new Error(`Unknown option: ${flag}`)
+    else if (options.name === undefined) options.name = flag
+    else throw new Error(`Unexpected argument: ${flag}`)
+  }
+  if (!options.name || !options.dir) {
+    throw new Error('Usage: node scripts/scaffold-skill.mjs <skill-name> --dir <repo-root> [--write] [--json]')
+  }
+  return options
+}
+
+const invokedDirectly = process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)
+if (invokedDirectly) {
+  let options
+  try {
+    options = parseArguments(process.argv.slice(2))
+  } catch (error) {
+    console.error(String(error.message ?? error))
+    process.exit(2)
+  }
+  try {
+    const result = await scaffoldSkill(options)
+    if (options.json) {
+      console.log(JSON.stringify(result, null, 2))
+    } else {
+      console.log(result.wrote ? `Created ${result.target}` : `Dry run - pass --write to create ${result.target}`)
+      for (const file of result.files) console.log(`  ${file}`)
+      console.log('Remaining manual wiring:')
+      for (const step of result.wiring) console.log(`  - ${step}`)
+    }
+  } catch (error) {
+    console.error(String(error.message ?? error))
+    process.exit(1)
+  }
+}