@kolisachint/hoocode-agent 0.5.30 → 0.5.32

package/CHANGELOG.md

@@ -1,5 +1,239 @@
 # Changelog
 
+## [0.5.32] - 2026-08-23
+
+### Breaking Changes
+
+- The tool-output setting is renamed and its values with it: `toolOutputDisplay`
+  (`collapsed` / `peek` / `standard`) is now `toolOutputView` (`radar` /
+  `glance` / `full`), and the default moves from `standard` to `glance`. Old
+  settings files still load — `collapsed` reads as `radar`, `peek` as `glance`,
+  `standard` as `full` — and the old key is rewritten the first time the view is
+  changed. Nothing is lost, but a session that used to open with every tool
+  result on screen now opens with call lines and folds the bodies away.
+
+- Keybindings moved off keys the terminal, the shell, or the prompt editor
+  already owned. The bindings are configurable as before, so a
+  `~/.hoocode/keybindings.json` that pins any of these keeps working:
+
+  | Action | Was | Now |
+  |---|---|---|
+  | Open model selector | `ctrl+l` | `alt+m` |
+  | Open external editor | `ctrl+g` | `alt+e` |
+  | Voice record | `ctrl+r` | `alt+r` |
+  | Session picker: toggle path | `ctrl+p` | `alt+p` |
+  | Session picker: sort order | `ctrl+s` | `alt+o` |
+  | Session picker: rename | `ctrl+r` | `alt+r` |
+  | Session picker: delete | `ctrl+d` | `alt+x` |
+  | Session picker: named filter | `ctrl+n` | `alt+n` |
+  | Model picker: save | `ctrl+s` | `alt+s` |
+  | Model picker: enable all | `ctrl+a` | `alt+a` |
+  | Model picker: clear all | `ctrl+x` | `alt+x` |
+  | Model picker: toggle provider | `ctrl+p` | `alt+g` |
+  | Tree filters | `ctrl+d/t/u/l/a` | `alt+1`…`alt+5` |
+  | Tree filter cycle | `ctrl+o` / `shift+ctrl+o` | `alt+c` / `shift+alt+c` |
+  | Team roster focus | `alt+n` | unchanged |
+
+### Added
+
+- A **view dial** for tool output, on `alt+o` (`shift+alt+o` cycles back), with
+  three stops from least to most:
+
+  - **radar** — one line per *chain*: a run of consecutive tool calls. While the
+    run is working the line shows its shape, in order, with the failures marked
+    — `◐ grep › read › bash✗ › edit › bash…  4 done · 1 failed · running`. Once
+    the agent moves on it becomes what the run amounted to —
+    `● Edited packages/tui/src/keys.ts  5 calls · 1 failed · 453 lines`.
+    Consecutive repeats collapse (`read ×4`) and long chains elide their middle,
+    but never a failure. `alt+u` turns a chain back into its individual calls.
+  - **glance** — the tool's own call line, one per call, body folded away. The
+    new default.
+  - **full** — call line plus the result body, as before.
+
+- **A failure always shows why it failed, in every view.** Previously a failed
+  tool in the folded views was a red dot and nothing else — the one thing you
+  always want to see was the one thing they hid.
+
+  The footer shows where the dial sits, with a glyph that fills up as the view
+  widens.
+
+- `alt+u` opens one thing, newest first, repeating to peel backwards;
+  `shift+alt+u` re-folds. In radar it turns a chain back into its calls;
+  elsewhere it opens a single tool body. `ctrl+o` is unchanged and still expands
+  everything at once.
+
+  It works from the newest backwards rather than through a cursor because the
+  transcript is bottom-anchored with no app-level scrolling — anything far
+  enough up is in the terminal's own scrollback, where this process can neither
+  scroll nor place a selection, so a cursor would move somewhere you cannot see.
+
+### Removed
+
+- The `▸` disclosure caret on folded tool rows. It is a click-target idiom in a
+  TUI with no pointer, it cost two columns on every row, and it advertised a
+  per-row action that could only ever be reached from the keyboard. Opening is
+  now `ctrl+o` (everything) or `alt+u` (one thing, newest first).
+
+- `/cd <path>` moves the whole session to another directory without leaving the
+  process — provider auth, the warmed model list and the terminal all survive.
+  Everything cwd-bound is rebuilt for the new root (tools, context files,
+  project settings, extensions, skills, agents, MCP servers), and because
+  sessions are stored per project, a fresh session starts there; the one you
+  left is still on disk and `/resume` in the old directory reopens it. Bare
+  `/cd` goes home, `/cd -` returns to the previous directory, and the argument
+  completes against real subdirectories. Bound to `alt+w`.
+
+- Keys for the cockpit moves that had none: `alt+g` cycles the agent mode
+  (ask → plan → build → debug, taken from the mode command's own list rather
+  than a copy), `alt+s` opens settings, `alt+k` opens the shortcut list,
+  `alt+t` opens the session tree, and `alt+h` resumes from history. `/new` and
+  `/fork` stay unbound by default — one replaces the transcript, the other needs
+  a message picked out of it.
+
+### Changed
+
+- The keybinding set is organized into three rings, and which ring a key is in
+  is decided by its modifier. `ctrl` is the view: what is on screen right now.
+  `alt` is the cockpit: what the agent is and where it works. Inside a picker,
+  every `ctrl` key belongs to the query you are typing, so a picker's own verbs
+  are all on `alt`. That last rule is what most of the moves above are: the
+  session picker took `ctrl+a` (start of line), `ctrl+u` (kill to start) and
+  `ctrl+d` (delete character) out of its own search field, and the model picker
+  put "save" on `ctrl+s`, which is XOFF on a terminal with flow control on.
+
+  Every default is checked against what a terminal without the Kitty keyboard
+  protocol actually sends. Only `alt+<letter>` and `alt+<digit>` survive there,
+  and legacy `alt+p`/`alt+n` double as `alt+up`/`alt+down`, so no scope binds
+  both halves of either pair — that is why the model picker's provider toggle is
+  `alt+g` rather than the obvious `alt+p`, which would have fired the reorder
+  key sitting next to it. `test/keybinding-layout.test.ts` holds all of this.
+
+- `ctrl+n` no longer means two things. It was both "cycle the task panel" and
+  the session picker's named filter; the filter moves to `alt+n`.
+
+- `/hotkeys` is grouped by ring — Flow, View, Cockpit — instead of by
+  Navigation / Editing / Other.
+
+### Fixed
+
+- A self-rendering tool's call line (`edit`) sat one column left of every other
+  row while its body was folded away, because it skipped the padded shell its
+  own diff frame needs. Harmless while `peek` was opt-in, visible on every edit
+  now that `glance` is the default.
+
+- The startup banner names the working directory and never re-read it, so after
+  `/cd` it kept advertising the directory you had left.
+
+- `edit`'s call line sat one column right of every other tool's in the folded
+  views. Its render component pads horizontally to carry the diff's header band,
+  and with no band to draw that padding was just an indent. It now pads only
+  when the band is there.
+
+- A failed call's reason now hangs off its radar row instead of starting back at
+  the left margin.
+## [0.5.31] - 2026-08-23
+
+### Added
+
+- `/settings` has an **External tools** category. hoocode ships five optional
+  Rust binaries — `rg`, `fd`, `embsearch`, `webtools`, `voicetools` — that
+  expand what it can do, and nothing in the product ever said so. hoocode works
+  without every one of them, which is exactly why they stayed invisible: search
+  silently got slower, semantic ranking silently never happened, and web and
+  voice were features nobody knew were there. Each row shows live status
+  (installed / found on `PATH` / env override / not installed), and opens a
+  detail view naming what it enables, what hoocode does instead when it is
+  missing, where the release comes from, and the env vars that steer it.
+
+- hoocode can ship skills of its own. It read them from `~/.agents/skills`,
+  `.hoocode/skills`, `.claude/skills` and installed packages — every source
+  except itself — which is why it shipped three subagents and zero skills while
+  telling users skills are the extension unit. Built-ins are catalogued in
+  `core/builtin-skills.ts` and load at lowest precedence, so a skill of the same
+  name from anywhere else wins and the collision is reported.
+
+  A skill costs its description on every turn, so each built-in can be gated on
+  the feature it serves rather than on everyone's token budget.
+
+- The first one: `plugin-authoring`, the craft half of `ProposePlugin`/
+  `UpdatePlugin` — when a capability is worth extracting, naming and describing
+  it so it triggers again, portability rules, and the hook trap where a changed
+  command adds a second hook instead of replacing one. Gated on
+  `enablePluginTools`, which is off by default, so a default session pays
+  nothing for it.
+
+  Built-ins are materialized to a content-addressed cache under
+  `~/.hoocode/cache/builtin-skills/`. A skill is loaded by reading its file, so
+  its location has to be a real path, and the compiled standalone binary has no
+  install directory to read from; materializing the same embedded copy
+  everywhere keeps the skill set identical across npm, pnpm, source and the
+  binary. If the cache cannot be written the built-ins are absent and everything
+  else runs normally. `--no-skills` and `--light` suppress them.
+
+- An agent-selection eval: `bun run agent-eval` scores the built-in agent roster
+  against a gold set of real tasks, reporting how often each agent is chosen
+  when it should be, how often the parent correctly keeps work inline, and a
+  confusion matrix naming which agent loses to which. It reuses the plugin G4
+  trigger harness rather than adding a second one, and describes each agent with
+  the summarized text `<available_agents>` actually emits.
+
+  This exists to settle whether `plan` and `explore` are two agents or one:
+  they ship with the same tools, the same isolation and the same background
+  flag, and `complexity` on the Task tool already expresses the only other
+  difference. That was being argued from intuition; it is now measurable.
+
+### Fixed
+
+- The G4 plugin trigger gate never had a judge. `trigger-eval.ts` takes its
+  model call as a parameter so scoring stays testable, and nothing in the tree
+  ever passed one — so every G4 run has reported `not-run` since it was written.
+  `createLlmTriggerJudge` is that judge, shared with the agent-selection eval.
+
+- `/new-skill`, `/new-agent` and `/new-command` scaffolded different content
+  depending on whether `--platform` was set. Each command has two write paths —
+  the per-vendor emitters and the plain `.hoocode/` writer — and each carried
+  its own copy of the body, which had drifted in both directions: the
+  `.hoocode/` command body documented the `${@:N}` / `${@:N:L}` slice
+  placeholders that the platform path silently omitted, and the `.hoocode/`
+  agent body identified the agent as running inside hoocode where the platform
+  one did not. Both paths read one definition now, and the richer text won in
+  each case.
+
+### Changed
+
+- The four built-in mode prompts have one home. `templates/modes/<mode>/system.md`
+  is now the only copy: `core/mode-prompts.ts` re-exports the embedded copy as
+  `DEFAULT_MODE_PROMPTS` instead of carrying a second, hand-written set. The two
+  had already drifted — `/init` scaffolds the template text into a project, so a
+  user who ran it and a user who did not were getting differently worded mode
+  rules from the same version.
+
+- The `/grill` phases moved out of TypeScript into `templates/prompts/*.md`.
+  They are prose the runtime injects verbatim — no interpolation, no branching —
+  so they are edited as prose now. The message `/grill` produces is unchanged.
+
+- The Task delegation appendix moved to `templates/prompts/task-main.md` and its
+  two `task-background-*` variants. At ~600 tok/turn it is the largest block of
+  always-on text hoocode emits once the Task tool is on, and it had exactly one
+  interpolation slot in otherwise static prose, so a string constant bought
+  nothing. The rendered prompt is byte-identical on both the with- and
+  without-background-agents branches.
+
+- `ProposePlugin` and `UpdatePlugin` shed ~55% of their always-on prompt
+  guidance (~626 to ~301 tok/turn when the plugin system is enabled). What was
+  removed was how-to-author-well guidance, which is not a tool contract and now
+  lives in the `plugin-authoring` skill; what stayed is the trigger, the
+  transparency rule for autonomous authoring, the hook trap, and the two hard
+  prohibitions. Several removed lines also restated the tools' own descriptions
+  or each other.
+
+- Settings rows that are inert without one of those binaries now say so. The
+  `web` and `semantic search` tool-group switches, the web tools timeout and the
+  voice silence window carry a `needs <binary>` marker and explain the fallback
+  and whether hoocode will fetch the binary. The rows stay settable — the
+  setting is what makes hoocode fetch the binary in the first place, and hiding
+  a row when its dependency is missing would recreate the same silence.
+
 ## [0.5.30] - 2026-08-23
 
 ### Added

package/dist/core/agent-selection-eval.d.ts

@@ -0,0 +1,91 @@
+/**
+ * Does the agent roster steer a dispatch to the right agent?
+ *
+ * The `Task` tool asks the model to pick a `subagent_type` from
+ * `<available_agents>`, and that block contains nothing but each agent's
+ * summarized description. So agent selection is the same question G4 asks about
+ * skills — "given these descriptions and this situation, which one fires?" —
+ * and it reuses the same harness rather than growing a second one.
+ *
+ * What this exists to decide: `plan` and `explore` ship with the same tools, the
+ * same isolation and the same `background` flag, differing only in model tier
+ * and output contract — and `complexity` on the Task tool already expresses the
+ * tier. Whether they are two agents or one is a question about whether the model
+ * can actually tell them apart from their descriptions, which is measurable and
+ * was previously being argued from intuition.
+ *
+ * The candidates use `summarizeAgentDescription`, not the raw frontmatter: the
+ * summary is what the system prompt actually emits, and evaluating the full
+ * description would score text the model never sees.
+ */
+import { type TriggerCandidate, type TriggerCase, type TriggerEvalOutcome, type TriggerJudge } from "./extensions/plugins/trigger-eval.js";
+/** How often one agent was chosen where another was expected. */
+export interface ConfusionEntry {
+    expected: string;
+    actual: string;
+    count: number;
+}
+export interface AgentSelectionReport {
+    corpusHash: string;
+    agents: string[];
+    caseCount: number;
+    /** Cases where the expected agent was chosen, over cases expecting any agent. */
+    accuracy?: number;
+    /** Cases correctly left to the parent, over cases expecting no delegation. */
+    inlineAccuracy?: number;
+    /** Every wrong pick, most frequent first. The pairs here are the finding. */
+    confusion: ConfusionEntry[];
+    /** Per-agent recall: chosen / expected. An agent nobody picks is dead weight. */
+    perAgent: Array<{
+        agent: string;
+        expected: number;
+        chosen: number;
+        recall: number;
+    }>;
+}
+/**
+ * The built-in agents as judge candidates, described exactly as the system
+ * prompt describes them.
+ *
+ * `own: true` for all of them: unlike a plugin eval there is no foreign roster
+ * to discriminate against, so every case is scored against the same closed set.
+ * The `expect: null` cases carry the discriminative half instead — they ask
+ * whether the model declines to delegate work it should keep.
+ */
+export declare function agentCandidates(cwd?: string): TriggerCandidate[];
+/** Load a gold set: `{ "cases": [{ "prompt": "...", "expect": "explore" | null }] }`. */
+export declare function loadAgentCases(file: string): TriggerCase[] | undefined;
+/**
+ * Every expected agent in the gold set must exist in the roster.
+ *
+ * A typo'd or removed agent name would otherwise score as a permanent miss and
+ * read as a description problem, which is the most expensive way to be wrong
+ * about an eval.
+ */
+export declare function validateAgentCases(candidates: readonly TriggerCandidate[], cases: readonly TriggerCase[]): string[];
+/**
+ * Turn a scored run into the report that answers the design question.
+ *
+ * `runTriggerEval`'s recall/specificity are the right numbers for a plugin
+ * defending itself against a foreign roster. Here the roster is closed, so the
+ * useful shape is a confusion matrix: which agent loses to which, and how often.
+ * "explore and plan are interchangeable" is a claim about one cell.
+ */
+export declare function summarizeAgentSelection(outcome: Extract<TriggerEvalOutcome, {
+    status: "ran";
+}>, candidates: readonly TriggerCandidate[]): AgentSelectionReport;
+export type AgentSelectionOutcome = {
+    status: "not-run";
+    reason: string;
+} | {
+    status: "ran";
+    report: AgentSelectionReport;
+    outcome: Extract<TriggerEvalOutcome, {
+        status: "ran";
+    }>;
+};
+/** Score the roster against a gold set. Never throws; a missing model is `not-run`. */
+export declare function runAgentSelectionEval(candidates: readonly TriggerCandidate[], cases: readonly TriggerCase[] | undefined, judge: TriggerJudge | undefined): Promise<AgentSelectionOutcome>;
+/** Human-readable report for the CLI. */
+export declare function formatAgentSelectionReport(report: AgentSelectionReport): string;
+//# sourceMappingURL=agent-selection-eval.d.ts.map

package/dist/core/agent-selection-eval.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"agent-selection-eval.d.ts","sourceRoot":"","sources":["../../src/core/agent-selection-eval.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;GAmBG;AAKH,OAAO,EAEN,KAAK,gBAAgB,EACrB,KAAK,WAAW,EAChB,KAAK,kBAAkB,EACvB,KAAK,YAAY,EACjB,MAAM,sCAAsC,CAAC;AAE9C,iEAAiE;AACjE,MAAM,WAAW,cAAc;IAC9B,QAAQ,EAAE,MAAM,CAAC;IACjB,MAAM,EAAE,MAAM,CAAC;IACf,KAAK,EAAE,MAAM,CAAC;CACd;AAED,MAAM,WAAW,oBAAoB;IACpC,UAAU,EAAE,MAAM,CAAC;IACnB,MAAM,EAAE,MAAM,EAAE,CAAC;IACjB,SAAS,EAAE,MAAM,CAAC;IAClB,iFAAiF;IACjF,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,8EAA8E;IAC9E,cAAc,CAAC,EAAE,MAAM,CAAC;IACxB,6EAA6E;IAC7E,SAAS,EAAE,cAAc,EAAE,CAAC;IAC5B,iFAAiF;IACjF,QAAQ,EAAE,KAAK,CAAC;QAAE,KAAK,EAAE,MAAM,CAAC;QAAC,QAAQ,EAAE,MAAM,CAAC;QAAC,MAAM,EAAE,MAAM,CAAC;QAAC,MAAM,EAAE,MAAM,CAAA;KAAE,CAAC,CAAC;CACrF;AAED;;;;;;;;GAQG;AACH,wBAAgB,eAAe,CAAC,GAAG,GAAE,MAAsB,GAAG,gBAAgB,EAAE,CAU/E;AAED,yFAAyF;AACzF,wBAAgB,cAAc,CAAC,IAAI,EAAE,MAAM,GAAG,WAAW,EAAE,GAAG,SAAS,CAiBtE;AAED;;;;;;GAMG;AACH,wBAAgB,kBAAkB,CAAC,UAAU,EAAE,SAAS,gBAAgB,EAAE,EAAE,KAAK,EAAE,SAAS,WAAW,EAAE,GAAG,MAAM,EAAE,CASnH;AASD;;;;;;;GAOG;AACH,wBAAgB,uBAAuB,CACtC,OAAO,EAAE,OAAO,CAAC,kBAAkB,EAAE;IAAE,MAAM,EAAE,KAAK,CAAA;CAAE,CAAC,EACvD,UAAU,EAAE,SAAS,gBAAgB,EAAE,GACrC,oBAAoB,CAiDtB;AAED,MAAM,MAAM,qBAAqB,GAC9B;IAAE,MAAM,EAAE,SAAS,CAAC;IAAC,MAAM,EAAE,MAAM,CAAA;CAAE,GACrC;IAAE,MAAM,EAAE,KAAK,CAAC;IAAC,MAAM,EAAE,oBAAoB,CAAC;IAAC,OAAO,EAAE,OAAO,CAAC,kBAAkB,EAAE;QAAE,MAAM,EAAE,KAAK,CAAA;KAAE,CAAC,CAAA;CAAE,CAAC;AAE5G,uFAAuF;AACvF,wBAAsB,qBAAqB,CAC1C,UAAU,EAAE,SAAS,gBAAgB,EAAE,EACvC,KAAK,EAAE,SAAS,WAAW,EAAE,GAAG,SAAS,EACzC,KAAK,EAAE,YAAY,GAAG,SAAS,GAC7B,OAAO,CAAC,qBAAqB,CAAC,CAmBhC;AAED,yCAAyC;AACzC,wBAAgB,0BAA0B,CAAC,MAAM,EAAE,oBAAoB,GAAG,MAAM,CAkB/E","sourcesContent":["/**\n * Does the agent roster steer a dispatch to the right agent?\n *\n * The `Task` tool asks the model to pick a `subagent_type` from\n * `<available_agents>`, and that block contains nothing but each agent's\n * summarized description. So agent selection is the same question G4 asks about\n * skills — \"given these descriptions and this situation, which one fires?\" —\n * and it reuses the same harness rather than growing a second one.\n *\n * What this exists to decide: `plan` and `explore` ship with the same tools, the\n * same isolation and the same `background` flag, differing only in model tier\n * and output contract — and `complexity` on the Task tool already expresses the\n * tier. Whether they are two agents or one is a question about whether the model\n * can actually tell them apart from their descriptions, which is measurable and\n * was previously being argued from intuition.\n *\n * The candidates use `summarizeAgentDescription`, not the raw frontmatter: the\n * summary is what the system prompt actually emits, and evaluating the full\n * description would score text the model never sees.\n */\n\nimport { createHash } from \"node:crypto\";\nimport { existsSync, readFileSync } from \"node:fs\";\nimport { loadAgentRegistry, summarizeAgentDescription } from \"./agent-registry.js\";\nimport {\n\trunTriggerEval,\n\ttype TriggerCandidate,\n\ttype TriggerCase,\n\ttype TriggerEvalOutcome,\n\ttype TriggerJudge,\n} from \"./extensions/plugins/trigger-eval.js\";\n\n/** How often one agent was chosen where another was expected. */\nexport interface ConfusionEntry {\n\texpected: string;\n\tactual: string;\n\tcount: number;\n}\n\nexport interface AgentSelectionReport {\n\tcorpusHash: string;\n\tagents: string[];\n\tcaseCount: number;\n\t/** Cases where the expected agent was chosen, over cases expecting any agent. */\n\taccuracy?: number;\n\t/** Cases correctly left to the parent, over cases expecting no delegation. */\n\tinlineAccuracy?: number;\n\t/** Every wrong pick, most frequent first. The pairs here are the finding. */\n\tconfusion: ConfusionEntry[];\n\t/** Per-agent recall: chosen / expected. An agent nobody picks is dead weight. */\n\tperAgent: Array<{ agent: string; expected: number; chosen: number; recall: number }>;\n}\n\n/**\n * The built-in agents as judge candidates, described exactly as the system\n * prompt describes them.\n *\n * `own: true` for all of them: unlike a plugin eval there is no foreign roster\n * to discriminate against, so every case is scored against the same closed set.\n * The `expect: null` cases carry the discriminative half instead — they ask\n * whether the model declines to delegate work it should keep.\n */\nexport function agentCandidates(cwd: string = process.cwd()): TriggerCandidate[] {\n\tconst registry = loadAgentRegistry({ cwd, includeBuiltins: true, includeClaude: false });\n\treturn registry\n\t\t.list()\n\t\t.map((agent) => ({\n\t\t\tname: agent.name,\n\t\t\tdescription: summarizeAgentDescription(agent.description ?? \"\"),\n\t\t\town: true,\n\t\t}))\n\t\t.sort((a, b) => a.name.localeCompare(b.name));\n}\n\n/** Load a gold set: `{ \"cases\": [{ \"prompt\": \"...\", \"expect\": \"explore\" | null }] }`. */\nexport function loadAgentCases(file: string): TriggerCase[] | undefined {\n\tif (!existsSync(file)) return undefined;\n\ttry {\n\t\tconst raw = JSON.parse(readFileSync(file, \"utf-8\")) as { cases?: unknown };\n\t\tif (!Array.isArray(raw.cases)) return undefined;\n\t\tconst cases = raw.cases.filter(\n\t\t\t(c): c is TriggerCase =>\n\t\t\t\t!!c &&\n\t\t\t\ttypeof c === \"object\" &&\n\t\t\t\ttypeof (c as TriggerCase).prompt === \"string\" &&\n\t\t\t\t(c as TriggerCase).prompt.trim().length > 0 &&\n\t\t\t\t((c as TriggerCase).expect === null || typeof (c as TriggerCase).expect === \"string\"),\n\t\t);\n\t\treturn cases.length > 0 ? cases : undefined;\n\t} catch {\n\t\treturn undefined;\n\t}\n}\n\n/**\n * Every expected agent in the gold set must exist in the roster.\n *\n * A typo'd or removed agent name would otherwise score as a permanent miss and\n * read as a description problem, which is the most expensive way to be wrong\n * about an eval.\n */\nexport function validateAgentCases(candidates: readonly TriggerCandidate[], cases: readonly TriggerCase[]): string[] {\n\tconst known = new Set(candidates.map((c) => c.name));\n\tconst problems: string[] = [];\n\tfor (const [i, testCase] of cases.entries()) {\n\t\tif (testCase.expect !== null && !known.has(testCase.expect)) {\n\t\t\tproblems.push(`case ${i} expects \"${testCase.expect}\", which is not in the roster`);\n\t\t}\n\t}\n\treturn problems;\n}\n\nfunction hashCorpus(candidates: readonly TriggerCandidate[], cases: readonly TriggerCase[]): string {\n\tconst h = createHash(\"sha256\");\n\tfor (const c of candidates) h.update(`${c.name} ${c.description} `);\n\tfor (const c of cases) h.update(`${c.prompt} ${c.expect ?? \"\"} `);\n\treturn h.digest(\"hex\").slice(0, 16);\n}\n\n/**\n * Turn a scored run into the report that answers the design question.\n *\n * `runTriggerEval`'s recall/specificity are the right numbers for a plugin\n * defending itself against a foreign roster. Here the roster is closed, so the\n * useful shape is a confusion matrix: which agent loses to which, and how often.\n * \"explore and plan are interchangeable\" is a claim about one cell.\n */\nexport function summarizeAgentSelection(\n\toutcome: Extract<TriggerEvalOutcome, { status: \"ran\" }>,\n\tcandidates: readonly TriggerCandidate[],\n): AgentSelectionReport {\n\tconst confusion = new Map<string, ConfusionEntry>();\n\tconst expectedCounts = new Map<string, number>();\n\tconst chosenCounts = new Map<string, number>();\n\tlet delegated = 0;\n\tlet delegatedCorrect = 0;\n\tlet inline = 0;\n\tlet inlineCorrect = 0;\n\n\tfor (const result of outcome.record.results) {\n\t\tif (result.expected === null) {\n\t\t\tinline++;\n\t\t\tif (result.actual === null) inlineCorrect++;\n\t\t\tcontinue;\n\t\t}\n\t\tdelegated++;\n\t\texpectedCounts.set(result.expected, (expectedCounts.get(result.expected) ?? 0) + 1);\n\t\tif (result.correct) {\n\t\t\tdelegatedCorrect++;\n\t\t\tchosenCounts.set(result.expected, (chosenCounts.get(result.expected) ?? 0) + 1);\n\t\t\tcontinue;\n\t\t}\n\t\t// \"(none)\" is a real outcome, not a missing value: declining to delegate\n\t\t// work that should have been delegated is a different failure from picking\n\t\t// the wrong agent, and collapsing them would hide which one is happening.\n\t\tconst actual = result.actual ?? \"(none)\";\n\t\tconst key = `${result.expected} ${actual}`;\n\t\tconst entry = confusion.get(key) ?? { expected: result.expected, actual, count: 0 };\n\t\tentry.count++;\n\t\tconfusion.set(key, entry);\n\t}\n\n\tconst perAgent = candidates\n\t\t.map((candidate) => {\n\t\t\tconst expected = expectedCounts.get(candidate.name) ?? 0;\n\t\t\tconst chosen = chosenCounts.get(candidate.name) ?? 0;\n\t\t\treturn { agent: candidate.name, expected, chosen, recall: expected > 0 ? chosen / expected : 0 };\n\t\t})\n\t\t.sort((a, b) => a.recall - b.recall);\n\n\treturn {\n\t\tcorpusHash: outcome.record.corpusHash,\n\t\tagents: candidates.map((c) => c.name),\n\t\tcaseCount: outcome.record.caseCount,\n\t\taccuracy: delegated > 0 ? delegatedCorrect / delegated : undefined,\n\t\tinlineAccuracy: inline > 0 ? inlineCorrect / inline : undefined,\n\t\tconfusion: [...confusion.values()].sort((a, b) => b.count - a.count),\n\t\tperAgent,\n\t};\n}\n\nexport type AgentSelectionOutcome =\n\t| { status: \"not-run\"; reason: string }\n\t| { status: \"ran\"; report: AgentSelectionReport; outcome: Extract<TriggerEvalOutcome, { status: \"ran\" }> };\n\n/** Score the roster against a gold set. Never throws; a missing model is `not-run`. */\nexport async function runAgentSelectionEval(\n\tcandidates: readonly TriggerCandidate[],\n\tcases: readonly TriggerCase[] | undefined,\n\tjudge: TriggerJudge | undefined,\n): Promise<AgentSelectionOutcome> {\n\tif (cases && cases.length > 0) {\n\t\tconst problems = validateAgentCases(candidates, cases);\n\t\tif (problems.length > 0) return { status: \"not-run\", reason: `invalid gold set: ${problems.join(\"; \")}` };\n\t}\n\n\tconst outcome = await runTriggerEval(\"agent-selection\", candidates, cases, judge);\n\tif (outcome.status === \"not-run\") return outcome;\n\n\treturn {\n\t\tstatus: \"ran\",\n\t\treport: {\n\t\t\t...summarizeAgentSelection(outcome, candidates),\n\t\t\t// The shared harness hashes its own way; restate it over exactly what\n\t\t\t// this eval judged so two reports are comparable on their own terms.\n\t\t\tcorpusHash: hashCorpus(candidates, cases ?? []),\n\t\t},\n\t\toutcome,\n\t};\n}\n\n/** Human-readable report for the CLI. */\nexport function formatAgentSelectionReport(report: AgentSelectionReport): string {\n\tconst pct = (v: number | undefined) => (v === undefined ? \"n/a\" : `${Math.round(v * 100)}%`);\n\tconst lines = [\n\t\t`corpus ${report.corpusHash} - ${report.caseCount} case(s) over ${report.agents.length} agent(s): ${report.agents.join(\", \")}`,\n\t\t`delegation accuracy ${pct(report.accuracy)}   inline accuracy ${pct(report.inlineAccuracy)}`,\n\t\t\"\",\n\t\t\"per agent (lowest recall first):\",\n\t];\n\tfor (const row of report.perAgent) {\n\t\tlines.push(`  ${row.agent.padEnd(18)} ${row.chosen}/${row.expected} chosen  (${pct(row.recall)})`);\n\t}\n\tif (report.confusion.length > 0) {\n\t\tlines.push(\"\", \"confusions (expected then actual):\");\n\t\tfor (const entry of report.confusion) {\n\t\t\tlines.push(`  ${entry.expected} => ${entry.actual}   x${entry.count}`);\n\t\t}\n\t}\n\treturn lines.join(\"\\n\");\n}\n"]}

package/dist/core/agent-selection-eval.js

@@ -0,0 +1,186 @@
+/**
+ * Does the agent roster steer a dispatch to the right agent?
+ *
+ * The `Task` tool asks the model to pick a `subagent_type` from
+ * `<available_agents>`, and that block contains nothing but each agent's
+ * summarized description. So agent selection is the same question G4 asks about
+ * skills — "given these descriptions and this situation, which one fires?" —
+ * and it reuses the same harness rather than growing a second one.
+ *
+ * What this exists to decide: `plan` and `explore` ship with the same tools, the
+ * same isolation and the same `background` flag, differing only in model tier
+ * and output contract — and `complexity` on the Task tool already expresses the
+ * tier. Whether they are two agents or one is a question about whether the model
+ * can actually tell them apart from their descriptions, which is measurable and
+ * was previously being argued from intuition.
+ *
+ * The candidates use `summarizeAgentDescription`, not the raw frontmatter: the
+ * summary is what the system prompt actually emits, and evaluating the full
+ * description would score text the model never sees.
+ */
+import { createHash } from "node:crypto";
+import { existsSync, readFileSync } from "node:fs";
+import { loadAgentRegistry, summarizeAgentDescription } from "./agent-registry.js";
+import { runTriggerEval, } from "./extensions/plugins/trigger-eval.js";
+/**
+ * The built-in agents as judge candidates, described exactly as the system
+ * prompt describes them.
+ *
+ * `own: true` for all of them: unlike a plugin eval there is no foreign roster
+ * to discriminate against, so every case is scored against the same closed set.
+ * The `expect: null` cases carry the discriminative half instead — they ask
+ * whether the model declines to delegate work it should keep.
+ */
+export function agentCandidates(cwd = process.cwd()) {
+    const registry = loadAgentRegistry({ cwd, includeBuiltins: true, includeClaude: false });
+    return registry
+        .list()
+        .map((agent) => ({
+        name: agent.name,
+        description: summarizeAgentDescription(agent.description ?? ""),
+        own: true,
+    }))
+        .sort((a, b) => a.name.localeCompare(b.name));
+}
+/** Load a gold set: `{ "cases": [{ "prompt": "...", "expect": "explore" | null }] }`. */
+export function loadAgentCases(file) {
+    if (!existsSync(file))
+        return undefined;
+    try {
+        const raw = JSON.parse(readFileSync(file, "utf-8"));
+        if (!Array.isArray(raw.cases))
+            return undefined;
+        const cases = raw.cases.filter((c) => !!c &&
+            typeof c === "object" &&
+            typeof c.prompt === "string" &&
+            c.prompt.trim().length > 0 &&
+            (c.expect === null || typeof c.expect === "string"));
+        return cases.length > 0 ? cases : undefined;
+    }
+    catch {
+        return undefined;
+    }
+}
+/**
+ * Every expected agent in the gold set must exist in the roster.
+ *
+ * A typo'd or removed agent name would otherwise score as a permanent miss and
+ * read as a description problem, which is the most expensive way to be wrong
+ * about an eval.
+ */
+export function validateAgentCases(candidates, cases) {
+    const known = new Set(candidates.map((c) => c.name));
+    const problems = [];
+    for (const [i, testCase] of cases.entries()) {
+        if (testCase.expect !== null && !known.has(testCase.expect)) {
+            problems.push(`case ${i} expects "${testCase.expect}", which is not in the roster`);
+        }
+    }
+    return problems;
+}
+function hashCorpus(candidates, cases) {
+    const h = createHash("sha256");
+    for (const c of candidates)
+        h.update(`${c.name} ${c.description} `);
+    for (const c of cases)
+        h.update(`${c.prompt} ${c.expect ?? ""} `);
+    return h.digest("hex").slice(0, 16);
+}
+/**
+ * Turn a scored run into the report that answers the design question.
+ *
+ * `runTriggerEval`'s recall/specificity are the right numbers for a plugin
+ * defending itself against a foreign roster. Here the roster is closed, so the
+ * useful shape is a confusion matrix: which agent loses to which, and how often.
+ * "explore and plan are interchangeable" is a claim about one cell.
+ */
+export function summarizeAgentSelection(outcome, candidates) {
+    const confusion = new Map();
+    const expectedCounts = new Map();
+    const chosenCounts = new Map();
+    let delegated = 0;
+    let delegatedCorrect = 0;
+    let inline = 0;
+    let inlineCorrect = 0;
+    for (const result of outcome.record.results) {
+        if (result.expected === null) {
+            inline++;
+            if (result.actual === null)
+                inlineCorrect++;
+            continue;
+        }
+        delegated++;
+        expectedCounts.set(result.expected, (expectedCounts.get(result.expected) ?? 0) + 1);
+        if (result.correct) {
+            delegatedCorrect++;
+            chosenCounts.set(result.expected, (chosenCounts.get(result.expected) ?? 0) + 1);
+            continue;
+        }
+        // "(none)" is a real outcome, not a missing value: declining to delegate
+        // work that should have been delegated is a different failure from picking
+        // the wrong agent, and collapsing them would hide which one is happening.
+        const actual = result.actual ?? "(none)";
+        const key = `${result.expected} ${actual}`;
+        const entry = confusion.get(key) ?? { expected: result.expected, actual, count: 0 };
+        entry.count++;
+        confusion.set(key, entry);
+    }
+    const perAgent = candidates
+        .map((candidate) => {
+        const expected = expectedCounts.get(candidate.name) ?? 0;
+        const chosen = chosenCounts.get(candidate.name) ?? 0;
+        return { agent: candidate.name, expected, chosen, recall: expected > 0 ? chosen / expected : 0 };
+    })
+        .sort((a, b) => a.recall - b.recall);
+    return {
+        corpusHash: outcome.record.corpusHash,
+        agents: candidates.map((c) => c.name),
+        caseCount: outcome.record.caseCount,
+        accuracy: delegated > 0 ? delegatedCorrect / delegated : undefined,
+        inlineAccuracy: inline > 0 ? inlineCorrect / inline : undefined,
+        confusion: [...confusion.values()].sort((a, b) => b.count - a.count),
+        perAgent,
+    };
+}
+/** Score the roster against a gold set. Never throws; a missing model is `not-run`. */
+export async function runAgentSelectionEval(candidates, cases, judge) {
+    if (cases && cases.length > 0) {
+        const problems = validateAgentCases(candidates, cases);
+        if (problems.length > 0)
+            return { status: "not-run", reason: `invalid gold set: ${problems.join("; ")}` };
+    }
+    const outcome = await runTriggerEval("agent-selection", candidates, cases, judge);
+    if (outcome.status === "not-run")
+        return outcome;
+    return {
+        status: "ran",
+        report: {
+            ...summarizeAgentSelection(outcome, candidates),
+            // The shared harness hashes its own way; restate it over exactly what
+            // this eval judged so two reports are comparable on their own terms.
+            corpusHash: hashCorpus(candidates, cases ?? []),
+        },
+        outcome,
+    };
+}
+/** Human-readable report for the CLI. */
+export function formatAgentSelectionReport(report) {
+    const pct = (v) => (v === undefined ? "n/a" : `${Math.round(v * 100)}%`);
+    const lines = [
+        `corpus ${report.corpusHash} - ${report.caseCount} case(s) over ${report.agents.length} agent(s): ${report.agents.join(", ")}`,
+        `delegation accuracy ${pct(report.accuracy)}   inline accuracy ${pct(report.inlineAccuracy)}`,
+        "",
+        "per agent (lowest recall first):",
+    ];
+    for (const row of report.perAgent) {
+        lines.push(`  ${row.agent.padEnd(18)} ${row.chosen}/${row.expected} chosen  (${pct(row.recall)})`);
+    }
+    if (report.confusion.length > 0) {
+        lines.push("", "confusions (expected then actual):");
+        for (const entry of report.confusion) {
+            lines.push(`  ${entry.expected} => ${entry.actual}   x${entry.count}`);
+        }
+    }
+    return lines.join("\n");
+}
+//# sourceMappingURL=agent-selection-eval.js.map

package/dist/core/agent-selection-eval.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"agent-selection-eval.js","sourceRoot":"","sources":["../../src/core/agent-selection-eval.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;GAmBG;AAEH,OAAO,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AACzC,OAAO,EAAE,UAAU,EAAE,YAAY,EAAE,MAAM,SAAS,CAAC;AACnD,OAAO,EAAE,iBAAiB,EAAE,yBAAyB,EAAE,MAAM,qBAAqB,CAAC;AACnF,OAAO,EACN,cAAc,GAKd,MAAM,sCAAsC,CAAC;AAuB9C;;;;;;;;GAQG;AACH,MAAM,UAAU,eAAe,CAAC,GAAG,GAAW,OAAO,CAAC,GAAG,EAAE,EAAsB;IAChF,MAAM,QAAQ,GAAG,iBAAiB,CAAC,EAAE,GAAG,EAAE,eAAe,EAAE,IAAI,EAAE,aAAa,EAAE,KAAK,EAAE,CAAC,CAAC;IACzF,OAAO,QAAQ;SACb,IAAI,EAAE;SACN,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,CAAC;QAChB,IAAI,EAAE,KAAK,CAAC,IAAI;QAChB,WAAW,EAAE,yBAAyB,CAAC,KAAK,CAAC,WAAW,IAAI,EAAE,CAAC;QAC/D,GAAG,EAAE,IAAI;KACT,CAAC,CAAC;SACF,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,aAAa,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC;AAAA,CAC/C;AAED,yFAAyF;AACzF,MAAM,UAAU,cAAc,CAAC,IAAY,EAA6B;IACvE,IAAI,CAAC,UAAU,CAAC,IAAI,CAAC;QAAE,OAAO,SAAS,CAAC;IACxC,IAAI,CAAC;QACJ,MAAM,GAAG,GAAG,IAAI,CAAC,KAAK,CAAC,YAAY,CAAC,IAAI,EAAE,OAAO,CAAC,CAAwB,CAAC;QAC3E,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,GAAG,CAAC,KAAK,CAAC;YAAE,OAAO,SAAS,CAAC;QAChD,MAAM,KAAK,GAAG,GAAG,CAAC,KAAK,CAAC,MAAM,CAC7B,CAAC,CAAC,EAAoB,EAAE,CACvB,CAAC,CAAC,CAAC;YACH,OAAO,CAAC,KAAK,QAAQ;YACrB,OAAQ,CAAiB,CAAC,MAAM,KAAK,QAAQ;YAC5C,CAAiB,CAAC,MAAM,CAAC,IAAI,EAAE,CAAC,MAAM,GAAG,CAAC;YAC3C,CAAE,CAAiB,CAAC,MAAM,KAAK,IAAI,IAAI,OAAQ,CAAiB,CAAC,MAAM,KAAK,QAAQ,CAAC,CACtF,CAAC;QACF,OAAO,KAAK,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,SAAS,CAAC;IAC7C,CAAC;IAAC,MAAM,CAAC;QACR,OAAO,SAAS,CAAC;IAClB,CAAC;AAAA,CACD;AAED;;;;;;GAMG;AACH,MAAM,UAAU,kBAAkB,CAAC,UAAuC,EAAE,KAA6B,EAAY;IACpH,MAAM,KAAK,GAAG,IAAI,GAAG,CAAC,UAAU,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC;IACrD,MAAM,QAAQ,GAAa,EAAE,CAAC;IAC9B,KAAK,MAAM,CAAC,CAAC,EAAE,QAAQ,CAAC,IAAI,KAAK,CAAC,OAAO,EAAE,EAAE,CAAC;QAC7C,IAAI,QAAQ,CAAC,MAAM,KAAK,IAAI,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,QAAQ,CAAC,MAAM,CAAC,EAAE,CAAC;YAC7D,QAAQ,CAAC,IAAI,CAAC,QAAQ,CAAC,aAAa,QAAQ,CAAC,MAAM,+BAA+B,CAAC,CAAC;QACrF,CAAC;IACF,CAAC;IACD,OAAO,QAAQ,CAAC;AAAA,CAChB;AAED,SAAS,UAAU,CAAC,UAAuC,EAAE,KAA6B,EAAU;IACnG,MAAM,CAAC,GAAG,UAAU,CAAC,QAAQ,CAAC,CAAC;IAC/B,KAAK,MAAM,CAAC,IAAI,UAAU;QAAE,CAAC,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,IAAI,IAAI,CAAC,CAAC,WAAW,GAAG,CAAC,CAAC;IACpE,KAAK,MAAM,CAAC,IAAI,KAAK;QAAE,CAAC,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,MAAM,IAAI,CAAC,CAAC,MAAM,IAAI,EAAE,GAAG,CAAC,CAAC;IAClE,OAAO,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,KAAK,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;AAAA,CACpC;AAED;;;;;;;GAOG;AACH,MAAM,UAAU,uBAAuB,CACtC,OAAuD,EACvD,UAAuC,EAChB;IACvB,MAAM,SAAS,GAAG,IAAI,GAAG,EAA0B,CAAC;IACpD,MAAM,cAAc,GAAG,IAAI,GAAG,EAAkB,CAAC;IACjD,MAAM,YAAY,GAAG,IAAI,GAAG,EAAkB,CAAC;IAC/C,IAAI,SAAS,GAAG,CAAC,CAAC;IAClB,IAAI,gBAAgB,GAAG,CAAC,CAAC;IACzB,IAAI,MAAM,GAAG,CAAC,CAAC;IACf,IAAI,aAAa,GAAG,CAAC,CAAC;IAEtB,KAAK,MAAM,MAAM,IAAI,OAAO,CAAC,MAAM,CAAC,OAAO,EAAE,CAAC;QAC7C,IAAI,MAAM,CAAC,QAAQ,KAAK,IAAI,EAAE,CAAC;YAC9B,MAAM,EAAE,CAAC;YACT,IAAI,MAAM,CAAC,MAAM,KAAK,IAAI;gBAAE,aAAa,EAAE,CAAC;YAC5C,SAAS;QACV,CAAC;QACD,SAAS,EAAE,CAAC;QACZ,cAAc,CAAC,GAAG,CAAC,MAAM,CAAC,QAAQ,EAAE,CAAC,cAAc,CAAC,GAAG,CAAC,MAAM,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;QACpF,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC;YACpB,gBAAgB,EAAE,CAAC;YACnB,YAAY,CAAC,GAAG,CAAC,MAAM,CAAC,QAAQ,EAAE,CAAC,YAAY,CAAC,GAAG,CAAC,MAAM,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;YAChF,SAAS;QACV,CAAC;QACD,yEAAyE;QACzE,2EAA2E;QAC3E,0EAA0E;QAC1E,MAAM,MAAM,GAAG,MAAM,CAAC,MAAM,IAAI,QAAQ,CAAC;QACzC,MAAM,GAAG,GAAG,GAAG,MAAM,CAAC,QAAQ,IAAI,MAAM,EAAE,CAAC;QAC3C,MAAM,KAAK,GAAG,SAAS,CAAC,GAAG,CAAC,GAAG,CAAC,IAAI,EAAE,QAAQ,EAAE,MAAM,CAAC,QAAQ,EAAE,MAAM,EAAE,KAAK,EAAE,CAAC,EAAE,CAAC;QACpF,KAAK,CAAC,KAAK,EAAE,CAAC;QACd,SAAS,CAAC,GAAG,CAAC,GAAG,EAAE,KAAK,CAAC,CAAC;IAC3B,CAAC;IAED,MAAM,QAAQ,GAAG,UAAU;SACzB,GAAG,CAAC,CAAC,SAAS,EAAE,EAAE,CAAC;QACnB,MAAM,QAAQ,GAAG,cAAc,CAAC,GAAG,CAAC,SAAS,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;QACzD,MAAM,MAAM,GAAG,YAAY,CAAC,GAAG,CAAC,SAAS,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;QACrD,OAAO,EAAE,KAAK,EAAE,SAAS,CAAC,IAAI,EAAE,QAAQ,EAAE,MAAM,EAAE,MAAM,EAAE,QAAQ,GAAG,CAAC,CAAC,CAAC,CAAC,MAAM,GAAG,QAAQ,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC;IAAA,CACjG,CAAC;SACD,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,MAAM,GAAG,CAAC,CAAC,MAAM,CAAC,CAAC;IAEtC,OAAO;QACN,UAAU,EAAE,OAAO,CAAC,MAAM,CAAC,UAAU;QACrC,MAAM,EAAE,UAAU,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC;QACrC,SAAS,EAAE,OAAO,CAAC,MAAM,CAAC,SAAS;QACnC,QAAQ,EAAE,SAAS,GAAG,CAAC,CAAC,CAAC,CAAC,gBAAgB,GAAG,SAAS,CAAC,CAAC,CAAC,SAAS;QAClE,cAAc,EAAE,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,aAAa,GAAG,MAAM,CAAC,CAAC,CAAC,SAAS;QAC/D,SAAS,EAAE,CAAC,GAAG,SAAS,CAAC,MAAM,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,KAAK,GAAG,CAAC,CAAC,KAAK,CAAC;QACpE,QAAQ;KACR,CAAC;AAAA,CACF;AAMD,uFAAuF;AACvF,MAAM,CAAC,KAAK,UAAU,qBAAqB,CAC1C,UAAuC,EACvC,KAAyC,EACzC,KAA+B,EACE;IACjC,IAAI,KAAK,IAAI,KAAK,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QAC/B,MAAM,QAAQ,GAAG,kBAAkB,CAAC,UAAU,EAAE,KAAK,CAAC,CAAC;QACvD,IAAI,QAAQ,CAAC,MAAM,GAAG,CAAC;YAAE,OAAO,EAAE,MAAM,EAAE,SAAS,EAAE,MAAM,EAAE,qBAAqB,QAAQ,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,EAAE,CAAC;IAC3G,CAAC;IAED,MAAM,OAAO,GAAG,MAAM,cAAc,CAAC,iBAAiB,EAAE,UAAU,EAAE,KAAK,EAAE,KAAK,CAAC,CAAC;IAClF,IAAI,OAAO,CAAC,MAAM,KAAK,SAAS;QAAE,OAAO,OAAO,CAAC;IAEjD,OAAO;QACN,MAAM,EAAE,KAAK;QACb,MAAM,EAAE;YACP,GAAG,uBAAuB,CAAC,OAAO,EAAE,UAAU,CAAC;YAC/C,sEAAsE;YACtE,qEAAqE;YACrE,UAAU,EAAE,UAAU,CAAC,UAAU,EAAE,KAAK,IAAI,EAAE,CAAC;SAC/C;QACD,OAAO;KACP,CAAC;AAAA,CACF;AAED,yCAAyC;AACzC,MAAM,UAAU,0BAA0B,CAAC,MAA4B,EAAU;IAChF,MAAM,GAAG,GAAG,CAAC,CAAqB,EAAE,EAAE,CAAC,CAAC,CAAC,KAAK,SAAS,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC,GAAG,GAAG,CAAC,GAAG,CAAC,CAAC;IAC7F,MAAM,KAAK,GAAG;QACb,UAAU,MAAM,CAAC,UAAU,MAAM,MAAM,CAAC,SAAS,iBAAiB,MAAM,CAAC,MAAM,CAAC,MAAM,cAAc,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE;QAC9H,uBAAuB,GAAG,CAAC,MAAM,CAAC,QAAQ,CAAC,sBAAsB,GAAG,CAAC,MAAM,CAAC,cAAc,CAAC,EAAE;QAC7F,EAAE;QACF,kCAAkC;KAClC,CAAC;IACF,KAAK,MAAM,GAAG,IAAI,MAAM,CAAC,QAAQ,EAAE,CAAC;QACnC,KAAK,CAAC,IAAI,CAAC,KAAK,GAAG,CAAC,KAAK,CAAC,MAAM,CAAC,EAAE,CAAC,IAAI,GAAG,CAAC,MAAM,IAAI,GAAG,CAAC,QAAQ,aAAa,GAAG,CAAC,GAAG,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC;IACpG,CAAC;IACD,IAAI,MAAM,CAAC,SAAS,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QACjC,KAAK,CAAC,IAAI,CAAC,EAAE,EAAE,oCAAoC,CAAC,CAAC;QACrD,KAAK,MAAM,KAAK,IAAI,MAAM,CAAC,SAAS,EAAE,CAAC;YACtC,KAAK,CAAC,IAAI,CAAC,KAAK,KAAK,CAAC,QAAQ,OAAO,KAAK,CAAC,MAAM,OAAO,KAAK,CAAC,KAAK,EAAE,CAAC,CAAC;QACxE,CAAC;IACF,CAAC;IACD,OAAO,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;AAAA,CACxB","sourcesContent":["/**\n * Does the agent roster steer a dispatch to the right agent?\n *\n * The `Task` tool asks the model to pick a `subagent_type` from\n * `<available_agents>`, and that block contains nothing but each agent's\n * summarized description. So agent selection is the same question G4 asks about\n * skills — \"given these descriptions and this situation, which one fires?\" —\n * and it reuses the same harness rather than growing a second one.\n *\n * What this exists to decide: `plan` and `explore` ship with the same tools, the\n * same isolation and the same `background` flag, differing only in model tier\n * and output contract — and `complexity` on the Task tool already expresses the\n * tier. Whether they are two agents or one is a question about whether the model\n * can actually tell them apart from their descriptions, which is measurable and\n * was previously being argued from intuition.\n *\n * The candidates use `summarizeAgentDescription`, not the raw frontmatter: the\n * summary is what the system prompt actually emits, and evaluating the full\n * description would score text the model never sees.\n */\n\nimport { createHash } from \"node:crypto\";\nimport { existsSync, readFileSync } from \"node:fs\";\nimport { loadAgentRegistry, summarizeAgentDescription } from \"./agent-registry.js\";\nimport {\n\trunTriggerEval,\n\ttype TriggerCandidate,\n\ttype TriggerCase,\n\ttype TriggerEvalOutcome,\n\ttype TriggerJudge,\n} from \"./extensions/plugins/trigger-eval.js\";\n\n/** How often one agent was chosen where another was expected. */\nexport interface ConfusionEntry {\n\texpected: string;\n\tactual: string;\n\tcount: number;\n}\n\nexport interface AgentSelectionReport {\n\tcorpusHash: string;\n\tagents: string[];\n\tcaseCount: number;\n\t/** Cases where the expected agent was chosen, over cases expecting any agent. */\n\taccuracy?: number;\n\t/** Cases correctly left to the parent, over cases expecting no delegation. */\n\tinlineAccuracy?: number;\n\t/** Every wrong pick, most frequent first. The pairs here are the finding. */\n\tconfusion: ConfusionEntry[];\n\t/** Per-agent recall: chosen / expected. An agent nobody picks is dead weight. */\n\tperAgent: Array<{ agent: string; expected: number; chosen: number; recall: number }>;\n}\n\n/**\n * The built-in agents as judge candidates, described exactly as the system\n * prompt describes them.\n *\n * `own: true` for all of them: unlike a plugin eval there is no foreign roster\n * to discriminate against, so every case is scored against the same closed set.\n * The `expect: null` cases carry the discriminative half instead — they ask\n * whether the model declines to delegate work it should keep.\n */\nexport function agentCandidates(cwd: string = process.cwd()): TriggerCandidate[] {\n\tconst registry = loadAgentRegistry({ cwd, includeBuiltins: true, includeClaude: false });\n\treturn registry\n\t\t.list()\n\t\t.map((agent) => ({\n\t\t\tname: agent.name,\n\t\t\tdescription: summarizeAgentDescription(agent.description ?? \"\"),\n\t\t\town: true,\n\t\t}))\n\t\t.sort((a, b) => a.name.localeCompare(b.name));\n}\n\n/** Load a gold set: `{ \"cases\": [{ \"prompt\": \"...\", \"expect\": \"explore\" | null }] }`. */\nexport function loadAgentCases(file: string): TriggerCase[] | undefined {\n\tif (!existsSync(file)) return undefined;\n\ttry {\n\t\tconst raw = JSON.parse(readFileSync(file, \"utf-8\")) as { cases?: unknown };\n\t\tif (!Array.isArray(raw.cases)) return undefined;\n\t\tconst cases = raw.cases.filter(\n\t\t\t(c): c is TriggerCase =>\n\t\t\t\t!!c &&\n\t\t\t\ttypeof c === \"object\" &&\n\t\t\t\ttypeof (c as TriggerCase).prompt === \"string\" &&\n\t\t\t\t(c as TriggerCase).prompt.trim().length > 0 &&\n\t\t\t\t((c as TriggerCase).expect === null || typeof (c as TriggerCase).expect === \"string\"),\n\t\t);\n\t\treturn cases.length > 0 ? cases : undefined;\n\t} catch {\n\t\treturn undefined;\n\t}\n}\n\n/**\n * Every expected agent in the gold set must exist in the roster.\n *\n * A typo'd or removed agent name would otherwise score as a permanent miss and\n * read as a description problem, which is the most expensive way to be wrong\n * about an eval.\n */\nexport function validateAgentCases(candidates: readonly TriggerCandidate[], cases: readonly TriggerCase[]): string[] {\n\tconst known = new Set(candidates.map((c) => c.name));\n\tconst problems: string[] = [];\n\tfor (const [i, testCase] of cases.entries()) {\n\t\tif (testCase.expect !== null && !known.has(testCase.expect)) {\n\t\t\tproblems.push(`case ${i} expects \"${testCase.expect}\", which is not in the roster`);\n\t\t}\n\t}\n\treturn problems;\n}\n\nfunction hashCorpus(candidates: readonly TriggerCandidate[], cases: readonly TriggerCase[]): string {\n\tconst h = createHash(\"sha256\");\n\tfor (const c of candidates) h.update(`${c.name} ${c.description} `);\n\tfor (const c of cases) h.update(`${c.prompt} ${c.expect ?? \"\"} `);\n\treturn h.digest(\"hex\").slice(0, 16);\n}\n\n/**\n * Turn a scored run into the report that answers the design question.\n *\n * `runTriggerEval`'s recall/specificity are the right numbers for a plugin\n * defending itself against a foreign roster. Here the roster is closed, so the\n * useful shape is a confusion matrix: which agent loses to which, and how often.\n * \"explore and plan are interchangeable\" is a claim about one cell.\n */\nexport function summarizeAgentSelection(\n\toutcome: Extract<TriggerEvalOutcome, { status: \"ran\" }>,\n\tcandidates: readonly TriggerCandidate[],\n): AgentSelectionReport {\n\tconst confusion = new Map<string, ConfusionEntry>();\n\tconst expectedCounts = new Map<string, number>();\n\tconst chosenCounts = new Map<string, number>();\n\tlet delegated = 0;\n\tlet delegatedCorrect = 0;\n\tlet inline = 0;\n\tlet inlineCorrect = 0;\n\n\tfor (const result of outcome.record.results) {\n\t\tif (result.expected === null) {\n\t\t\tinline++;\n\t\t\tif (result.actual === null) inlineCorrect++;\n\t\t\tcontinue;\n\t\t}\n\t\tdelegated++;\n\t\texpectedCounts.set(result.expected, (expectedCounts.get(result.expected) ?? 0) + 1);\n\t\tif (result.correct) {\n\t\t\tdelegatedCorrect++;\n\t\t\tchosenCounts.set(result.expected, (chosenCounts.get(result.expected) ?? 0) + 1);\n\t\t\tcontinue;\n\t\t}\n\t\t// \"(none)\" is a real outcome, not a missing value: declining to delegate\n\t\t// work that should have been delegated is a different failure from picking\n\t\t// the wrong agent, and collapsing them would hide which one is happening.\n\t\tconst actual = result.actual ?? \"(none)\";\n\t\tconst key = `${result.expected} ${actual}`;\n\t\tconst entry = confusion.get(key) ?? { expected: result.expected, actual, count: 0 };\n\t\tentry.count++;\n\t\tconfusion.set(key, entry);\n\t}\n\n\tconst perAgent = candidates\n\t\t.map((candidate) => {\n\t\t\tconst expected = expectedCounts.get(candidate.name) ?? 0;\n\t\t\tconst chosen = chosenCounts.get(candidate.name) ?? 0;\n\t\t\treturn { agent: candidate.name, expected, chosen, recall: expected > 0 ? chosen / expected : 0 };\n\t\t})\n\t\t.sort((a, b) => a.recall - b.recall);\n\n\treturn {\n\t\tcorpusHash: outcome.record.corpusHash,\n\t\tagents: candidates.map((c) => c.name),\n\t\tcaseCount: outcome.record.caseCount,\n\t\taccuracy: delegated > 0 ? delegatedCorrect / delegated : undefined,\n\t\tinlineAccuracy: inline > 0 ? inlineCorrect / inline : undefined,\n\t\tconfusion: [...confusion.values()].sort((a, b) => b.count - a.count),\n\t\tperAgent,\n\t};\n}\n\nexport type AgentSelectionOutcome =\n\t| { status: \"not-run\"; reason: string }\n\t| { status: \"ran\"; report: AgentSelectionReport; outcome: Extract<TriggerEvalOutcome, { status: \"ran\" }> };\n\n/** Score the roster against a gold set. Never throws; a missing model is `not-run`. */\nexport async function runAgentSelectionEval(\n\tcandidates: readonly TriggerCandidate[],\n\tcases: readonly TriggerCase[] | undefined,\n\tjudge: TriggerJudge | undefined,\n): Promise<AgentSelectionOutcome> {\n\tif (cases && cases.length > 0) {\n\t\tconst problems = validateAgentCases(candidates, cases);\n\t\tif (problems.length > 0) return { status: \"not-run\", reason: `invalid gold set: ${problems.join(\"; \")}` };\n\t}\n\n\tconst outcome = await runTriggerEval(\"agent-selection\", candidates, cases, judge);\n\tif (outcome.status === \"not-run\") return outcome;\n\n\treturn {\n\t\tstatus: \"ran\",\n\t\treport: {\n\t\t\t...summarizeAgentSelection(outcome, candidates),\n\t\t\t// The shared harness hashes its own way; restate it over exactly what\n\t\t\t// this eval judged so two reports are comparable on their own terms.\n\t\t\tcorpusHash: hashCorpus(candidates, cases ?? []),\n\t\t},\n\t\toutcome,\n\t};\n}\n\n/** Human-readable report for the CLI. */\nexport function formatAgentSelectionReport(report: AgentSelectionReport): string {\n\tconst pct = (v: number | undefined) => (v === undefined ? \"n/a\" : `${Math.round(v * 100)}%`);\n\tconst lines = [\n\t\t`corpus ${report.corpusHash} - ${report.caseCount} case(s) over ${report.agents.length} agent(s): ${report.agents.join(\", \")}`,\n\t\t`delegation accuracy ${pct(report.accuracy)}   inline accuracy ${pct(report.inlineAccuracy)}`,\n\t\t\"\",\n\t\t\"per agent (lowest recall first):\",\n\t];\n\tfor (const row of report.perAgent) {\n\t\tlines.push(`  ${row.agent.padEnd(18)} ${row.chosen}/${row.expected} chosen  (${pct(row.recall)})`);\n\t}\n\tif (report.confusion.length > 0) {\n\t\tlines.push(\"\", \"confusions (expected then actual):\");\n\t\tfor (const entry of report.confusion) {\n\t\t\tlines.push(`  ${entry.expected} => ${entry.actual}   x${entry.count}`);\n\t\t}\n\t}\n\treturn lines.join(\"\\n\");\n}\n"]}

package/dist/core/agent-session-runtime.d.ts

@@ -26,6 +26,13 @@ export type CreateAgentSessionRuntimeFactory = (options: {
     sessionManager: SessionManager;
     sessionStartEvent?: SessionStartEvent;
 }) => Promise<CreateAgentSessionRuntimeResult>;
+/**
+ * Thrown when `/cd` is pointed at something that is not a usable directory.
+ */
+export declare class ChangeDirectoryError extends Error {
+    readonly path: string;
+    constructor(message: string, path: string);
+}
 /**
  * Thrown when /import references a JSONL file path that does not exist.
  */
@@ -82,6 +89,27 @@ export declare class AgentSessionRuntime {
     }): Promise<{
         cancelled: boolean;
     }>;
+    /**
+     * Move the whole runtime to another directory without leaving the process.
+     *
+     * Everything cwd-bound is rebuilt for the new root — tools, context files
+     * (AGENTS.md and friends), project settings, extensions, skills, agents, the
+     * MCP server set — which is exactly why this cannot just reassign a string:
+     * those services are constructed per cwd and a half-moved runtime would run
+     * the new repo's code against the old repo's rules.
+     *
+     * Sessions are stored per project, so the move starts a fresh session in the
+     * target directory rather than dragging the current transcript across. The
+     * old session is already persisted; `/resume` in the old directory reopens
+     * it. A session directory the user pinned explicitly (`--session-dir`, the
+     * env var, or the setting) is *not* cwd-derived and is carried over as-is.
+     *
+     * @throws {ChangeDirectoryError} When the target is missing or not a directory.
+     */
+    changeDirectory(targetCwd: string): Promise<{
+        cancelled: boolean;
+        cwd: string;
+    }>;
     fork(entryId: string, options?: {
         position?: "before" | "at";
         withSession?: (ctx: ReplacedSessionContext) => Promise<void>;