@givolt/i3-composer-agent 2.0.0 → 2.1.0

package/docs/developer-onboarding.md

@@ -1,116 +0,0 @@
-# Developer Onboarding
-
-This guide is for people working **on** the i3-composer-agent repository itself: contributing skills, fixing docs, improving the eval harness. If you want to *use* the skills in your own projects, you're looking for [getting-started.md](getting-started.md) instead.
-
-It's a guided tour, not a rulebook. The rules live in [CONTRIBUTING.md](../CONTRIBUTING.md) (contribution workflow), [skill-anatomy.md](skill-anatomy.md) (skill format), and [evals/README.md](../evals/README.md) (eval framework); this document tells you when to read each one and how the pieces fit.
-
----
-
-## 1. The mental model
-
-The repo has five composable layers. Understanding what each one is *for* prevents the most common contribution mistakes (putting reference material in a skill, building a persona that routes to other personas, duplicating content across skills).
-
-| Layer | Location | Job | In one word |
-|---|---|---|---|
-| **Skills** | `skills/<name>/SKILL.md` | Step-by-step workflows with verification gates | *How* |
-| **Personas** | `agents/<role>.md` | Roles with a perspective and output format | *Who* |
-| **Commands** | `.claude/commands/`, `.gemini/commands/`, `commands/` | User-facing entry points; the orchestration layer | *When* |
-| **References** | `references/*.md` | Checklists skills pull in on demand | *What to check* |
-| **Evals** | `evals/cases/<name>.json` | Proof that skills trigger and behave correctly | *Does it work* |
-
-Two structural rules worth internalizing early:
-
-- **The user (or a slash command) is the orchestrator.** Personas never invoke other personas; the only endorsed multi-persona pattern is parallel fan-out with a merge step (see [references/orchestration-patterns.md](../references/orchestration-patterns.md)).
-- **Don't duplicate, reference.** Skills link to other skills and to `references/` instead of restating content. The same rule applies to docs, including this one.
-
-One scope caveat that trips people up: `AGENTS.md` and `CLAUDE.md` at the repo root configure agents working on *this repo*. They are not reusable assets and setup guides must never tell users to copy them into their own projects; the reusable assets are the skills.
-
-Note that commands exist in three parallel directories (Claude Code, Gemini CLI, Antigravity). Touch one and CI checks parity across all of them, see §3.
-
-## 2. Local setup
-
-```bash
-npx @givolt/i3-composer-agent --extract --dir ./i3-composer-agent
-cd i3-composer-agent
-```
-
-There's no build step and no `package.json`; validators are plain Node scripts. You need:
-
-- **Node 20+** (what CI runs) for the `scripts/` validators
-- **bash** (+ `jq` recommended) for the hook regression test
-- **`gh` CLI** for the duplicate-PR check before proposing a skill
-- **Claude Code** only if you want to run Tier 3 behavioral evals locally
-
-To try the pack live against a local checkout:
-
-```bash
-claude --plugin-dir /path/to/i3-composer-agent
-```
-
-## 3. The verification loop
-
-The repo eats its own cooking: verification is non-negotiable for skills, and it's non-negotiable for contributions to the repo too. Everything CI runs, you can run locally in seconds:
-
-```bash
-# Tier 1, structural: frontmatter, naming, required sections
-node scripts/validate-skills.js
-
-# Command parity and description sync across the three command directories
-node scripts/validate-commands.js
-
-# Tier 2, trigger & routing: positive prompts rank top-k, negatives don't collide
-node scripts/run-evals.js
-
-# Tier 3, behavioral (on demand, spends tokens; --dry-run prints the plan)
-node scripts/run-evals.js --behavioral <skill-name> --dry-run
-
-# Hook regression test, required if you touch hooks/session-start.sh
-# or skills/i3-using-agent-skills/SKILL.md
-bash hooks/session-start-test.sh
-```
-
-The three eval tiers are worth understanding even if you never touch the harness, because a red Tier 2 usually means *fix your skill's description*, not the eval: Tier 2 is a lexical approximation of routing (stemmed TF-IDF over descriptions), and its two target failure modes are a description missing the vocabulary users actually say, and an over-broad description that outranks the right skill. The full design, schema, and trust-level rules are in [evals/README.md](../evals/README.md).
-
-Run the relevant subset before every PR. A PR that arrives green through Tier 1 + Tier 2 + command parity is reviewable; one that doesn't will bounce on mechanics before anyone reads the content.
-
-## 4. Contribution paths
-
-### Path 1: Fixing or improving an existing skill (most common, best first PR)
-
-1. Keep changes focused and minimal; preserve the skill's structure and tone.
-2. If you changed the frontmatter `description`, expect Tier 2 effects; run `node scripts/run-evals.js` and check the skill's trigger prompts still rank.
-3. Run Tier 1 to confirm frontmatter is still valid.
-
-### Path 2: Proposing a new skill (higher bar, do the pre-flight)
-
-The catalog already covers most of the lifecycle, so the burden of proof is on the gap. Before writing anything, run the pre-flight checks in [CONTRIBUTING.md](../CONTRIBUTING.md#before-proposing-a-new-skill): search the catalog, check open PRs (`gh pr list --state open`; near-duplicate clusters already exist), confirm the idea fits [skill-anatomy.md](skill-anatomy.md), and justify the gap explicitly in your PR description. If it overlaps an existing skill, a focused edit to that skill beats a new directory.
-
-A new skill ships as a set, not a single file: the `skills/<kebab-case-name>/SKILL.md`, a matching `evals/cases/<name>.json`, and a `scripts/` directory only when it ships runnable helpers (reference material goes in `references/`, never inside the skill). The exact frontmatter rules, the section anatomy, and the eval-case minimums live in [CONTRIBUTING.md](../CONTRIBUTING.md#structure) and [skill-anatomy.md](skill-anatomy.md); take them from there rather than this tour, so the two can't drift.
-
-One point worth internalizing rather than looking up: when writing trigger prompts, paraphrase how users actually talk; copying the description into the prompts games the eval and tells you nothing.
-
-### Path 3: Docs, references, harness
-
-- Docs and skills are **English only**; translations aren't accepted because they drift ([CONTRIBUTING.md](../CONTRIBUTING.md#translations) has the rationale).
-- Changes to `scripts/run-evals.js` or the eval schema should stay compatible with skill-creator's `evals.json` schema (adopted verbatim for the behavioral tier; that compatibility is a feature, not an accident).
-- Anything touching the session-start hook or the meta-skill it embeds requires the hook regression test (§3).
-
-## 5. Pre-PR checklist
-
-- [ ] Tier 1 green: `node scripts/validate-skills.js`
-- [ ] Tier 2 green: `node scripts/run-evals.js`
-- [ ] Command parity green if you touched any command directory: `node scripts/validate-commands.js`
-- [ ] Hook test green if you touched `hooks/` or `i3-using-agent-skills`
-- [ ] New skill → eval case file present with the minimum trigger/behavioral counts
-- [ ] New skill → gap justified in the PR description; catalog and open PRs checked
-- [ ] No duplicated content; cross-references used instead
-- [ ] Change is small and focused (the repo's own `i3-code-review-and-quality` change-sizing guidance applies to contributions here too)
-
-## 6. Suggested reading order
-
-1. [README.md](../README.md): the catalog and the lifecycle diagram (10 min)
-2. `skills/i3-using-agent-skills/SKILL.md`: how routing works from the agent's side
-3. One well-established skill end to end (e.g. `i3-test-driven-development`): internalize the anatomy by example
-4. [skill-anatomy.md](skill-anatomy.md): the format spec, now with context
-5. [evals/README.md](../evals/README.md): the three tiers and the case format
-6. [CONTRIBUTING.md](../CONTRIBUTING.md) + [AGENTS.md](../AGENTS.md): the rules and the repo-scoped agent config

package/docs/skill-anatomy.md

@@ -1,203 +0,0 @@
-# Skill Anatomy
-
-This document describes the structure and format of i3-composer-agent skill files. Use this as a guide when contributing new skills or understanding existing ones.
-
-## File Location
-
-Every skill lives in its own directory under `skills/`:
-
-```
-skills/
-  skill-name/
-    SKILL.md           # Required: The skill definition
-    scripts/           # Optional: Runnable helpers used by the skill workflow
-    references/        # Optional: Skill-specific reference documentation
-    supporting-file.md # Optional: Reference material loaded on demand
-```
-
-`SKILL.md` is the only required file. Add `scripts/` or `references/` only when the skill actually needs them, and omit them entirely for simpler skills.
-
-## SKILL.md Format
-
-### Frontmatter (Required)
-
-```yaml
----
-name: skill-name-with-hyphens
-description: Guides agents through [task/workflow]. Use when [specific trigger conditions].
----
-```
-
-**Rules:**
-- `name`: Lowercase, hyphen-separated. Must match the directory name.
-- `description`: Start with what the skill does in third person, then include one or more clear "Use when" trigger conditions. Include both *what* and *when*. Maximum 1024 characters.
-- Top-level keys are limited to the [Agent Skills specification](https://agentskills.io) fields: `name`, `description`, `license`, `compatibility`, `metadata`, and `allowed-tools`. Vendor or runtime controls such as `model`, `tools`, `max_turns`, or `context` go under `metadata` or in a per-agent adapter file; see [docs/advanced-per-agent-configuration.md](advanced-per-agent-configuration.md). The validator rejects any other top-level key.
-
-Published names are compatibility identifiers. In particular,
-`i3-browser-testing-with-devtools` is the stable upstream name because other
-skills refer to it directly. A downstream catalog may rename it, but that
-catalog owns the alias or migration mapping; downstream-only aliases are not
-maintained in this repository.
-
-**Why this matters:** Agents discover skills by reading descriptions. The description is injected into the system prompt, so it must tell the agent both what the skill provides and when to activate it. Do not summarize the workflow — if the description contains process steps, the agent may follow the summary instead of reading the full skill.
-
-### Standard Sections (Recommended Pattern)
-
-The frontmatter contract above is required. The section layout below is a recommended pattern, not a rigid template: equivalent headings are acceptable when they serve the same purpose clearly.
-
-```markdown
-# Skill Title
-
-## Overview
-One-two sentences explaining what this skill does and why it matters.
-
-## When to Use
-- Bullet list of triggering conditions (symptoms, task types)
-- When NOT to use (exclusions)
-
-## [Core Process / The Workflow / Steps]
-The main workflow, broken into numbered steps or phases.
-Include code examples where they help.
-Use flowcharts (ASCII) where decision points exist.
-
-## [Specific Techniques / Patterns]
-Detailed guidance for specific scenarios.
-Code examples, templates, configuration.
-
-## Common Rationalizations
-| Rationalization | Reality |
-|---|---|
-| Excuse agents use to skip steps | Why the excuse is wrong |
-
-## Red Flags
-- Behavioral patterns indicating the skill is being violated
-- Things to watch for during review
-
-## Verification
-After completing the skill's process, confirm:
-- [ ] Checklist of exit criteria
-- [ ] Evidence requirements
-```
-
-## Section Purposes
-
-### Overview
-The "elevator pitch" for the skill. Should answer: What does this skill do, and why should an agent follow it?
-
-### When to Use
-Helps agents and humans decide if this skill applies to the current task. Include both positive triggers ("Use when X") and negative exclusions ("NOT for Y").
-
-### Core Process
-The heart of the skill. This is the step-by-step workflow the agent follows. Must be specific and actionable — not vague advice.
-
-**Good:** "Run `npm test` and verify all tests pass"
-**Bad:** "Make sure the tests work"
-
-### Common Rationalizations
-The most distinctive feature of well-crafted skills. These are excuses agents use to skip important steps, paired with rebuttals. They prevent the agent from rationalizing its way out of following the process.
-
-Think of every time an agent has said "I'll add tests later" or "This is simple enough to skip the spec" — those go here with a factual counter-argument.
-
-### Red Flags
-Observable signs that the skill is being violated. Useful during code review and self-monitoring.
-
-### Verification
-The exit criteria. A checklist the agent uses to confirm the skill's process is complete. Every checkbox should be verifiable with evidence (test output, build result, screenshot, etc.).
-
-## Supporting Files
-
-Create supporting files only when:
-- Reference material exceeds 100 lines (keep the main SKILL.md focused)
-- Code tools or scripts are needed
-- Checklists are long enough to justify separate files
-
-Keep patterns and principles inline when under 50 lines.
-
-If a skill does not need runnable helpers, do not create an empty `scripts/` directory just to mirror other skills. Empty directories add noise without changing how the skill works.
-
-## Shared References
-
-Checklists used by more than one skill — testing, security, performance, accessibility, definition-of-done — live in `references/` at the repository root, deliberately *not* inside any skill directory.
-
-This is a pack-level design choice. The Agent Skills spec describes a skill as a self-contained directory, but several skills here point at the same checklists. Colocating those would force one of two options: copy the checklist into every skill that uses it, or pick one skill to "own" it and have the others reach into that directory. Both drift over time. A single repo-root copy stays the source of truth.
-
-The tradeoff is portability: a whole-repo install (such as the Claude Code marketplace plugin) carries `references/` along, but a per-skill install that copies only `skills/<name>/` leaves the repo-root sibling behind, and those links resolve to nothing. That gap is tracked in [upstream agent-skills #361](https://github.com/addyosmani/agent-skills/issues/361).
-
-Current convention: material used by exactly one skill is a supporting file inside that skill's directory; material shared across skills goes in `references/`.
-
-## Context Efficiency
-
-Skills load on demand: only the skill name and description sit in context at startup. The full `SKILL.md` loads only when an agent decides the skill is relevant. To keep that load cheap:
-
-- **Keep `SKILL.md` under 500 lines.** Move detailed reference material into supporting files.
-- **Write specific descriptions.** A precise description helps the agent activate the skill at the right moment and skip it otherwise.
-- **Use progressive disclosure.** Reference supporting files that are read only when the workflow reaches them.
-- **Prefer scripts over inline code.** Executing a script consumes no context; only its output does. Inline code blocks are paid for on every load.
-- **Keep file references one level deep.** Link directly from `SKILL.md` to supporting files rather than chaining through intermediate documents.
-
-## Script Requirements
-
-When a skill ships runnable helpers under `scripts/`, each script follows these conventions:
-
-- Use a `#!/bin/bash` shebang.
-- Use `set -e` for fail-fast behavior.
-- Write status messages to stderr: `echo "Message" >&2`.
-- Write machine-readable output (JSON) to stdout.
-- Include a cleanup trap for temporary files.
-- Reference the script path as `skills/<skill-name>/scripts/<script>.sh` (repo-relative).
-
-## Writing Principles
-
-1. **Process over knowledge.** Skills are workflows, not reference docs. Steps, not facts.
-2. **Specific over general.** "Run `npm test`" beats "verify the tests".
-3. **Evidence over assumption.** Every verification checkbox requires proof.
-4. **Anti-rationalization.** Every skip-worthy step needs a counter-argument in the rationalizations table.
-5. **Progressive disclosure.** Main SKILL.md is the entry point. Supporting files are loaded only when needed.
-6. **Token-conscious.** Every section must justify its inclusion. If removing it wouldn't change agent behavior, remove it.
-7. **Model-neutral.** Write the procedure, not the workaround. See [Write the Procedure, Not the Workaround](#write-the-procedure-not-the-workaround).
-
-## Write the Procedure, Not the Workaround
-
-Skills in this pack run on many agents and model generations. A step that exists because one model gets a specific call wrong is a liability on every other model: it constrains agents that would have solved the task directly, and it spends turns that stronger models need for the actual work. Cross-model transfer measurements ([WikiSkill, arXiv:2608.27454](https://arxiv.org/abs/2608.27454)) show skills authored against one model's execution failures can leave a stronger model performing *worse than with no skill at all*.
-
-**The rule:** if a step cannot be justified without naming a model, a model version, or one agent's private tool name, it belongs in an issue, not in a skill. Describe the capability ("run the focused test command", "write the file"), not the mechanism one runtime happens to expose (`run_command`, `write_to_file`).
-
-Two smells to check for before proposing skill content:
-
-- **Over-specified mechanics.** Prescribing an exact command form or serialization detail where stating the goal alone would do. If the justification is "model X gets this wrong otherwise", it's a workaround, not a procedure.
-- **Fragmented diagnostics.** A multi-step inspection sequence where one step would confirm the same thing. Every extra step spends turns, and turn budget is finite.
-
-This is about skill *content*; the portability of `references/` *paths* is a separate concern (see [Shared References](#shared-references)).
-
-## Naming Conventions
-
-- Skill directories: `lowercase-hyphen-separated`
-- Skill files: `SKILL.md` (always uppercase)
-- Supporting files: `lowercase-hyphen-separated.md`
-- Shared references: stored in the root `references/` directory, not inside skill directories (see [Shared References](#shared-references) for why).
-- Skill-specific references: a single supporting doc can stay as a loose file in the skill directory (the `Supporting files` entry above); when several related docs travel with the skill, the emerging convention for self-contained, distributable skills is to group them in a `references/` directory inside the skill directory, so the skill carries its own supporting docs.
-
-## Cross-Skill References
-
-Reference other skills by name:
-
-```markdown
-Follow the `i3-test-driven-development` skill for writing tests.
-If the build breaks, use the `i3-debugging-and-error-recovery` skill.
-```
-
-Don't duplicate content between skills — reference and link instead.
-
-## Required vs Recommended
-
-Required:
-
-- A `skills/<skill-name>/SKILL.md` file
-- Valid YAML frontmatter with `name` and `description`
-- A description that includes both what the skill does and when to use it
-
-Recommended:
-
-- The standard section flow shown above
-- Equivalent headings such as `How It Works`, `Core Process`, or `Workflow` when they read more naturally for the skill
-- Supporting files only when they keep the main `SKILL.md` focused

package/evals/README.md

@@ -1,101 +0,0 @@
-# Skill Evals
-
-How this repo measures whether its skills actually work: that they **trigger** when they should, **stay distinct** from each other, and **change agent behavior** the way each skill promises.
-
-## Prior art (and what we adopted)
-
-There is no single settled community standard for evaluating `SKILL.md` skills, but two approaches lead:
-
-- **Anthropic's skill-creator v2** defines a per-skill `evals.json` (prompt + `expectations[]`, graded from the transcript) plus trigger-accuracy testing of descriptions against sample prompts. We adopt its [`evals.json` schema](https://github.com/anthropics/skills/tree/main/skills/skill-creator) for our behavioral tier and add one optional `kind` field to select the artifact being graded.
-- **Superpowers** (obra) tests skills with bash + `claude -p` + prompt fixtures and grader scripts. Our behavioral runner follows the same headless-`claude` pattern, with the grading rubric drawn from `expectations[]`.
-
-What neither provides is a **deterministic, CI-safe** check for a multi-skill *catalog* — does each skill's description carry the vocabulary users actually say, and do two skills' descriptions collide? That's Tier 2 below, and it's this repo's addition.
-
-## The three tiers
-
-| Tier | What it checks | Runs | Cost |
-|---|---|---|---|
-| 1. Structural | Frontmatter, naming, required sections, command parity | CI (`validate-skills.js`, `validate-commands.js`) | Free |
-| 2. Trigger & routing | Positive prompts rank their skill top-k; negative prompts don't; no two descriptions near-collide | CI (`run-evals.js`) | Free |
-| 3. Behavioral | An agent following the skill satisfies its `expectations[]` | On demand (`run-evals.js --behavioral`) | Tokens |
-
-Tier 2 is a **lexical approximation** of routing (stemmed TF-IDF over descriptions). It cannot judge semantics — that's Tier 3's job — but it catches the two failure modes that dominate real trigger bugs: a description missing the vocabulary users say (false negative), and an over-broad description that outranks the right skill (false positive). A Tier-2 failure usually means *fix the description*, not the eval.
-
-## Running
-
-```bash
-# Tier 2 — deterministic, runs in CI
-node scripts/run-evals.js
-node scripts/run-evals.js --min-rank1 95  # enforce the current routing floor
-
-# Tier 3 — behavioral, runs each eval through headless claude, then grades it
-node scripts/run-evals.js --behavioral test-driven-development            # spends tokens
-node scripts/run-evals.js --behavioral test-driven-development --dry-run  # prints the plan only
-```
-
-Tier 3 supports two behavioral artifact kinds. `execution` is the default: each eval runs in a throwaway git repository, real project inputs from `files[]` are materialized out of `evals/fixtures/` and committed as the baseline, and the grader judges the full `--output-format stream-json --verbose` execution trace, including tool calls. `dialogue` is reserved for skills whose deliverable is the conversation itself; it needs no fixture, and the grader judges the assistant's conversational turns without requiring file edits or commands. Claiming `dialogue` is a human-reviewed exemption, not a general escape hatch for execution skills.
-
-The executor runs with an explicit permission mode (`--permission-mode acceptEdits` plus a pre-approved tool list) so execution evals can genuinely edit files, run commands, inspect diffs, and make commits rather than being denied and narrating instead. Traces are fenced as untrusted data in the grader prompt and piped to the grader over stdin (they can be megabytes; argv would hit the OS argument-size limit), executor and grader calls carry timeouts, and grader output is validated as JSON before being written to `evals/results/` (gitignored) in skill-creator's `grading.json` shape. Discipline skills also include pressure cases for time pressure, sunk cost, and authority pressure; these verify that the workflow still holds when the prompt argues for skipping it.
-
-## Plugin evals (Claude Code)
-
-`claude plugin eval` (Claude Code 2.1.269 or later) loads the whole plugin, every skill at once, runs the cases under `evals/plugin/`, and scores each case with and without the plugin. It complements Tier 2 rather than replacing it: Tier 2 checks that a description carries the vocabulary users say; this checks that Claude Code's own router picks the skill on a natural prompt, and what the skill adds to the reply (the `Δ` column).
-
-```bash
-claude plugin eval . --no-publish                                          # every case, both arms, 3 runs each
-claude plugin eval . --case code-review-fires --runs 1 --ablation none     # one cheap iteration on one case
-```
-
-A case is a directory holding `prompt.md` (frontmatter: turn and tool limits; body: the prompt, sent verbatim) and one grader per file under `graders/`. Reports land in `evals/plugin/results/<timestamp>/`, already gitignored. Every run and every `llm` grader is a real model call on your account, so this runs on demand and never in CI, like Tier 3.
-
-Two rules keep it honest. The manifest's `experimental.evals` key scopes the scan to `evals/plugin/`, so a fixture elsewhere under `evals/` is never read as a case. And a `tool_used: Skill` grader only says whether the skill fired; in a two-arm run it is excluded from the score, so pair it with a grader on the reply itself. `code-review-fires` shows the pattern: the fired indicator, a free `regex` grader that looks for the skill's severity taxonomy (Critical, Required, Nit, Optional, Consider, FYI) used as a heading or as a label, and one `llm` judge on the planted off-by-one. `code-review-stays-quiet` is the precision check: the review skill must not fire on a test-first request, and its `Δ` is expected to be 0; a second, unscored indicator records whether test-driven-development fires instead.
-
-What the first runs taught (Claude Code 2.1.278, claude-opus-5). Skill invocation is stochastic: the review skill fired in 5 of 7 runs on the phrasing the case uses, and test-driven-development fired in 2 of 7 runs on the test-first prompt, so read `Δ` and the report rather than the exit code, or pass `--threshold 0.8` as the CI example in the docs does. In one run the review skill did not fire and the reply still used its severity taxonomy as headings, which no baseline run did in nine; the loaded skill descriptions may shape the reply even without an invocation. Phrasing matters, and the description is the lever. With the original description, "Review this diff before I merge it", "Can you do a proper code review of this change before I merge it?" and "Review this pull request for me before it goes into main" rarely fired the skill; the model reviewed the diff on its own, competently but without the verdict, the severity tiers or the five-axis pass. Describing what the skill adds did not help, and neither did naming diffs and pull requests on their own; what made the difference was saying that the skill applies even when the diff is pasted inline, which is the case the model was treating as "I can just do this myself". On claude-opus-5-5 with the model pinned, that one clause took the four review phrasings from 7 of 27 fires to 21 of 27, left the nine adjacent must-not-fire prompts at 0 of 27 both ways, and moved the plainest phrasing from 0 of 9 to 3 of 9, so it is better, not solved. `code-review-stays-quiet-on-commit-message` guards that clause: a pasted diff with a commit-message request must not fire the review skill. Two practical rules from the same day: pin `--model` and record the Claude Code version whenever you report numbers, because the CLI updates itself and the default model changed twice between morning and evening; and give a case a turn budget of 10 or more, since the model may reach a skill through its slash command (`agent-skills:review`, then `code-review-and-quality`), which costs two turns.
-
-## Eval case format
-
-One file per skill: `evals/cases/<skill-name>.json`.
-
-```json
-{
-  "skill_name": "test-driven-development",
-  "trigger": {
-    "positive": [
-      { "prompt": "Write a failing test for this bug before fixing it", "top_k": 3 }
-    ],
-    "negative": [
-      { "prompt": "Update the architecture diagram in the docs", "owner": "documentation-and-adrs" }
-    ]
-  },
-  "evals": [
-    {
-      "id": 1,
-      "kind": "execution",
-      "prompt": "Finance filed the reconciliation bug written up in BUG.md. Fix it.",
-      "expected_output": "A failing reproduction test for the lost-cent case, a fix preserving both README invariants (exact sum, earliest-shares fairness), the fairness invariant covered by its own test, full suite passing",
-      "files": [
-        "test-driven-development"
-      ],
-      "expectations": [
-        "A test reproducing the lost-cent case from BUG.md is added and shown failing before src/split.js is modified",
-        "The final implementation satisfies the full README fairness invariant (leftover cents go to the earliest shares): splitCents(10000, 3) returns [3334, 3333, 3333] as BUG.md expects and splitCents(100, 7) returns [15, 15, 14, 14, 14, 14, 14] as the README example shows; dumping the whole remainder on a single share would violate both",
-        "The fairness invariant from the README has its own test case in the suite on an input with remainder of at least 2 (such as splitCents(100, 7)), where dumping the whole remainder on one share would fail it, beyond the reported lost-cent case",
-        "The full suite is run with the repository's own command after the fix"
-      ]
-    }
-  ]
-}
-```
-
-- `evals[]` uses skill-creator's core schema (`id`, `prompt`, `expected_output`, optional `files[]`, `expectations[]`) plus this repository's optional `kind`. `kind` must be `execution` or `dialogue` and defaults to `execution` for compatibility. Execution evals require non-empty `files[]`; paths are relative to `evals/fixtures/` and may name a file or project directory. Dialogue evals may omit `files[]` because the transcript is the artifact. Expectations are verifiable statements a grader checks against the relevant artifact — behaviors, not phrasings.
-- `trigger` is this repo's extension. `positive` prompts are realistic user asks that should route here (`top_k` defaults to 3; tighten to 1 for a skill's signature ask). `negative` prompts belong to a *different* skill; this skill must not rank first for them. Declare that skill in `owner` where you can: the runner then asserts the owner **outranks** this skill, turning the negative into a real pairwise routing test instead of one that can pass vacuously when the prompt matches nothing.
-
-**Writing good trigger prompts:** paraphrase how users actually talk; don't copy the description (that's gaming the eval). If a realistic prompt can't rank because the description lacks its vocabulary, that is a real finding — improve the description.
-
-## Adding a skill
-
-Every skill ships with an eval file. When you add `skills/<name>/`, add `evals/cases/<name>.json` with at least 3 positive triggers, 2 negative triggers, and 1 behavioral eval. Execution evals must be backed by `evals/fixtures/<name>/`; use `kind: "dialogue"` only when the skill's deliverable is genuinely the conversation itself. Missing case files, incomplete case counts, unknown kinds, invalid fixture paths, and absent required fixtures are CI errors.
-
-## Metrics to watch
-
-The Tier-2 run prints a **trigger rank-1 rate** (share of positive prompts that rank their skill first, not merely top-k). CI runs with `--min-rank1 95`, leaving useful headroom below the checked-in 100% baseline so an unrelated description edit does not immediately turn CI red. Raise the floor as routing improves; never lower it to make a regression pass. Falling numbers mean descriptions are drifting toward each other. The collision check errors at ≥75% pairwise description similarity and warns at ≥50%. When these evals surface a description-vocabulary gap (see [#351](https://github.com/addyosmani/agent-skills/issues/351) for the original examples), fix the description, not the prompt.

package/evals/cases/i3-api-and-interface-design.json

@@ -1,45 +0,0 @@
-{
-  "skill_name": "i3-api-and-interface-design",
-  "trigger": {
-    "positive": [
-      {
-        "prompt": "Design a REST endpoint for creating invoices, including error responses and versioning",
-        "top_k": 3
-      },
-      {
-        "prompt": "What should the public interface of this payments module expose to other teams?",
-        "top_k": 3
-      },
-      {
-        "prompt": "Help me define the contract between the frontend and the orders service",
-        "top_k": 3
-      }
-    ],
-    "negative": [
-      {
-        "prompt": "My unit test is failing with a null pointer exception",
-        "owner": "i3-debugging-and-error-recovery"
-      },
-      {
-        "prompt": "Make this landing page responsive on mobile",
-        "owner": "i3-frontend-ui-engineering"
-      }
-    ]
-  },
-  "evals": [
-    {
-      "id": 1,
-      "prompt": "Design the public API for a URL-shortening service: create, resolve, stats. Produce the endpoint contracts.",
-      "expected_output": "Endpoint contracts with methods, paths, request/response shapes, and explicit error semantics",
-      "files": [
-        "i3-api-and-interface-design"
-      ],
-      "expectations": [
-        "Error responses are specified with status codes and a consistent error shape, not just happy paths",
-        "Input validation at the boundary is addressed for user-supplied URLs",
-        "Versioning or compatibility strategy is stated",
-        "The response does not silently invent unstated requirements"
-      ]
-    }
-  ]
-}

package/evals/cases/i3-browser-testing-with-devtools.json

@@ -1,44 +0,0 @@
-{
-  "skill_name": "i3-browser-testing-with-devtools",
-  "trigger": {
-    "positive": [
-      {
-        "prompt": "Use Chrome DevTools to figure out why the checkout button click does nothing",
-        "top_k": 3
-      },
-      {
-        "prompt": "Inspect the network requests the dashboard makes when it first loads",
-        "top_k": 3
-      },
-      {
-        "prompt": "Verify in a real browser that the form submits and the data persists in localStorage",
-        "top_k": 3
-      }
-    ],
-    "negative": [
-      {
-        "prompt": "Write a spec for the new billing feature before we start",
-        "owner": "i3-spec-driven-development"
-      },
-      {
-        "prompt": "Break this PRD into ordered implementation tasks",
-        "owner": "i3-planning-and-task-breakdown"
-      }
-    ]
-  },
-  "evals": [
-    {
-      "id": 1,
-      "prompt": "The signup form renders but submitting it appears to do nothing. Verify the real behavior in the browser and report findings.",
-      "expected_output": "Runtime evidence from the browser: console errors, network activity, DOM state, and a diagnosis",
-      "files": [
-        "i3-browser-testing-with-devtools"
-      ],
-      "expectations": [
-        "Findings are grounded in observed runtime data (console, network, DOM), not static code reading alone",
-        "The report distinguishes what was observed from what is inferred",
-        "A concrete next step or fix hypothesis is provided"
-      ]
-    }
-  ]
-}

package/evals/cases/i3-ci-cd-and-automation.json

@@ -1,45 +0,0 @@
-{
-  "skill_name": "i3-ci-cd-and-automation",
-  "trigger": {
-    "positive": [
-      {
-        "prompt": "Set up a GitHub Actions pipeline that runs the tests on every pull request",
-        "top_k": 3
-      },
-      {
-        "prompt": "Add a deploy stage to the pipeline with a manual approval gate",
-        "top_k": 3
-      },
-      {
-        "prompt": "Automate the quality gates so nothing merges without passing checks",
-        "top_k": 3
-      }
-    ],
-    "negative": [
-      {
-        "prompt": "Refactor this function so it is easier to read",
-        "owner": "i3-code-simplification"
-      },
-      {
-        "prompt": "Add structured logging to the checkout service",
-        "owner": "i3-observability-and-instrumentation"
-      }
-    ]
-  },
-  "evals": [
-    {
-      "id": 1,
-      "prompt": "Create a CI pipeline for a Node project: install, lint, test on every PR, and block merge on failure.",
-      "expected_output": "A working workflow definition with correct triggers, steps, and failure behavior",
-      "files": [
-        "i3-ci-cd-and-automation"
-      ],
-      "expectations": [
-        "The workflow triggers on pull requests",
-        "Failure of any quality gate fails the pipeline run",
-        "Steps are ordered logically and cache or setup steps are sane",
-        "No secrets are hardcoded in the workflow"
-      ]
-    }
-  ]
-}

package/evals/cases/i3-code-review-and-quality.json

@@ -1,49 +0,0 @@
-{
-  "skill_name": "i3-code-review-and-quality",
-  "trigger": {
-    "positive": [
-      {
-        "prompt": "Review this pull request before I merge it",
-        "top_k": 3
-      },
-      {
-        "prompt": "Do a quality pass on this diff for correctness and readability",
-        "top_k": 3
-      },
-      {
-        "prompt": "Can you review the changes another model just wrote for me?",
-        "top_k": 3
-      },
-      {
-        "prompt": "Review this diff before I merge it. It adds paging to the orders endpoint.",
-        "top_k": 3
-      }
-    ],
-    "negative": [
-      {
-        "prompt": "Deploy this to production now",
-        "owner": "i3-shipping-and-launch"
-      },
-      {
-        "prompt": "Write a failing test for the bug before fixing it",
-        "owner": "i3-test-driven-development"
-      }
-    ]
-  },
-  "evals": [
-    {
-      "id": 1,
-      "prompt": "Review the provided diff that adds a user-search endpoint. Deliver a structured review.",
-      "expected_output": "A multi-axis review with severity-labelled findings and file:line references",
-      "files": [
-        "i3-code-review-and-quality"
-      ],
-      "expectations": [
-        "Findings cover more than one axis (correctness, readability, architecture, security, performance)",
-        "Every finding carries a severity label from the skill's taxonomy",
-        "Security of user input is explicitly considered for the new endpoint",
-        "The review leads with high-leverage findings rather than nits"
-      ]
-    }
-  ]
-}