omp-llm-role 1.0.0 → 1.2.0

package/docs/dev/testing.md

@@ -0,0 +1,253 @@
+# Testing
+
+How the plugin is verified: the unit suite (`node --test tests/`, run by the
+pre-commit hook) and the live checks that need a real omp session and a real
+OpenRouter key. Node type-stripping does not typecheck, so a green suite plus a
+real run is the gate — see [`quirks.md`](quirks.md).
+
+## Unit seams
+
+Each seam names the test file that covers it and the scenarios it must keep
+passing.
+
+1. **Engine metrics** (`engine-metrics.test.ts`) — `buildModels` maps every
+   leaderboard field into `metrics`; an absent field stays null (0-filled, not
+   imputed); every metric has a `METRIC_META` entry with the right kind; the
+   affine/identity transforms; a role weighting a new metric ranks it.
+2. **Tier gate** (`tier-gate.test.ts`) — paid+credit → billed-only; free-tier
+   key → `:free`-only; zero budget both branches → abort; non-openrouter
+   providers filtered.
+3. **Variant resolution** (`variant-resolution.test.ts`) — exact/dated/bare/alias
+   ordering; `:batch` never emitted; `:free` only on the free branch; org
+   prefixes (incl. `~`) ignored; ties lexicographic.
+4. **Surgical edit** (`config-edit.test.ts`) — commented config, unknown keys,
+   missing roles upsert, chain-key replace-in-place (no duplicate YAML keys),
+   unchanged file → zero-byte diff and no mtime change; structural surprises
+   rejected; the empty-collection spellings omp's own writer emits (a null token
+   on the key line, a flow `[]`/`{}` on its own indented line — and an empty
+   *parent* block, `task:`/`retry:` + `  {}`) read as empty and are normalized
+   into the block form on write, with a prune-only patch creating nothing (issue
+   #54); a sibling `retry.*` key neither aborts the chain scan nor swallows an
+   appended chain key (issue #55); the patched output parses as real YAML.
+5. **Hysteresis** (`hysteresis.test.ts`) — no-current adopt; ineligible current
+   switch; margin-below keep; margin-above switch; `switchMargin: 0` always takes
+   best; cost override (`switched-cost`); `priceSwitchFraction: 0` keeps the
+   margin in charge; an inside-margin challenger short of the fraction stays
+   kept; the day gate.
+6. **Chain pruning** (`chain-pruning.test.ts`) — plugin-written stale keys
+   removed; owner-written keys preserved; `writeFallbackChains: false`; a role
+   that left the managed set is pruned; the chain prefers candidates at or below
+   the chosen price.
+7. **Chain suffixes** (`chain-suffix.test.ts`) — chain values carry the role's
+   suffix, the key stays bare; an entry without thinking support gets no suffix;
+   two roles on one model share a level-free chain.
+8. **Probe gate** (`probe-gate.test.ts`) — 200 → ok; the no-allowed-providers 404
+   → blocked; any other failure → usable; blocked best → next adopted; blocked
+   current → replaced; chains contain only probe-clean candidates; verdicts
+   cached across roles; all-blocked → role untouched.
+9. **Keyed catalog** (`keyed-catalog.test.ts`) — key-blocked candidates pruned
+   from the walk; only candidates in neither catalog are probed; key-blocked
+   never in a chain; no-filter/unavailable degrade to the probe walk.
+10. **Key availability** (`key-availability.test.ts`) — proper subset → active;
+    equal sets → no-filter; empty → unavailable; a keyed id absent from public →
+    no-filter; aliases collapse; two parallel GETs, one authenticated; never
+    throws.
+11. **Allowed-providers whitelist** (`allowed-providers.test.ts`,
+    `whitelist.test.ts`) — `fetchAllowedProviders` parses the 404 body's
+    "permits only:" list into a set and returns `null` for a 200, a 404 without
+    the clause, or a transport throw, with the canary request shape (POST,
+    bearer, `provider.only`, `max_tokens: 1`); the updater prunes every model's
+    route pool to allowed providers before ranking, so a blocked best route
+    falls to the next-best allowed route as the auto-pin, the chain carries only
+    allowed routes, a bare entry matches a tiered slug by prefix, the notify line
+    reports the active whitelist and the pruned count (or "no whitelist"), and a
+    `null` whitelist reproduces today's output byte-for-byte.
+12. **Endpoint ceilings** (`endpoint-ceilings.test.ts`) — the narrower reads
+    context/output/tools, a missing field is null; a failing route drops, a null
+    field is kept; a model with no capable route is ineligible; a filtered role
+    is priced on the surviving pool; `maxPriceUsdPerM` caps the thinking-adjusted
+    price.
+13. **Route pricing (provider pin)** (`route-pricing.test.ts`) — a pinned role
+    prices the matching route, not the blend; a tiered pin matches verbatim;
+    `maxPriceUsdPerM` caps the route price; throughput is the route p50 with a
+    blend fallback; no matching route → ineligible; the `@<slug>` selector and
+    chain carry the pin; a pin matching no route leaves the role unchanged. The
+    route-ordering score (spec #71): `rankedRoutes` orders by the value-for-money
+    score `vfm = wT·tNorm − wP·pNorm` (a throughput-heavy role picks a faster
+    route than a price-heavy role); the soft `filters.preferredMinThroughput`
+    floor sorts a below-floor route last **without dropping it**; a route with no
+    p50 uses the model's blended throughput (for both the score and the floor
+    check); equal scores break by `providerSlug` then `id`; `explainModel`'s
+    `autoPin` is the score-max route.
+14. **Thinking price** (`thinking-price.test.ts`) — bare/off unadjusted; a level
+    scales only thinking-capable models; the frontier uses the effective price;
+    `resolveSettings` validates thinking and rejects the legacy `suffixes` map;
+    catalog `thinking[]` wins over the OR flag; meta levels never gate.
+15. **OpenRouter blend** (`openrouter-blend.test.ts`) — price/throughput are the
+    `1/price²`-weighted means over the page routes; flex/priority/degraded/free/
+    `:batch` stay out; throughput renormalizes; the find-row fallback; the weight
+    follows the input price.
+16. **Writing metric** (`writing.test.ts`) — `parseWritingEvidence` keys finite
+    scores and rejects junk; `applyWritingScores` fills uncovered models; a
+    covered model ranks above an uncovered one.
+17. **Benchmark sources** (`benchmark-sources.test.ts`) — `resolveBenchmarkSource`
+    maps an llm-stats benchmark page, a bare id, the writing leaderboard and a
+    Design Arena link, `null` for an unknown host; `normalizeMetricKey` is
+    dot-free; `parseBenchmarkPayload` reads the llm-stats `entries[]` and writing
+    evidence shapes and rejects junk; `parseBenchmarkPayloadMeta` reads the
+    per-entry `self_reported`/`verified` flags (19/20 → `trust`, no flag → `null`,
+    no entries shape → `null`); `parseBenchmarkCatalog` carries `isCommunity`,
+    `modality` and the maintenance fields (`updatedAt`/`versionCount`/
+    `latestVersionRowCount`/`starCount`, each `null` when missing or non-finite);
+    the catalog cache rejects a pre-`isCommunity`/`modality`/maintenance cache
+    (and an empty one); the scores cache rejects a cache lacking the `meta` key
+    (and accepts `meta: null`);
+    `applyBenchmarkScores` fills uncovered
+    models; `loadBenchmarkScores` follows fresh → live → stale with a temp cache
+    dir and an injected fetch; a declaration executes (payload path, id/score
+    fields, `scoreMax`, join) and a metric colliding with a shipped key is
+    rejected; `dryRunDeclaration` reports coverage/leader and rejects a zero-join;
+    `extractBenchmarkLinks` pulls URLs from prose.
+18. **Agent creation** (`create-agent.test.ts`) — a purpose fits the expected
+    archetype; a `--weights` set violating Σ(non-price) = 1 − w_price is refused;
+    an existing agent file without `--force` is refused and writes no role;
+    `--dry-run` writes neither file; an architect `spec` replaces the
+    description/body; `applyFocusBenchmarks` gives a named benchmark a decisive
+    share, is a no-op for the archetype's own specialist set, and keeps both
+    invariants; the nine-axis gate (the maintenance, provenance and cross-source
+    axes are annotations, never a drop).
+19. **Agent removal** (`remove-agent.test.ts`) — the role's lock-file keys and the
+    agent file are deleted (backup written); a shipped default role, a reserved
+    name and an invalid name are refused; nothing-to-remove errors; `--dry-run`
+    writes nothing; the updater drops `modelRoles.<n>` and the plugin-managed
+    `task.disabledAgents` entry.
+20. **Agent pins / disable** (`agent-pins.test.ts`, `agent-disable.test.ts`) —
+    `parseAgentPin` takes the first `@role`; `discoverAgentPins` scans the three
+    scopes, most specific wins; the sync is not day-gated; an unrelated entry
+    keeps its position; an unknown role is never added.
+21. **Role enable / lock** (`role-enable.test.ts`, `role-lock.test.ts`) —
+    disabling drops the role from the resolved set and its `modelRoles` line; a
+    locked role is not switched and its chain survives the prune; unlocking
+    restores the switch.
+22. **Settings** (`settings.test.ts`, `settings-schema.test.ts`) — opt-in roles;
+    `priceSwitchFraction` range; flat/nested merge; `required` as string or
+    array; an external metric key is weightable, a dotted one rejected;
+    `package.json` `omp.settings` deep-equals `deriveSettingsSchema()`. The
+    `providerPinning` preset sets `preferOwnProvider: true` +
+    `filters.preferredMinThroughput: 50` (the registry row and the expanded def
+    agree), an explicit `filters.preferredMinThroughput` beats the preset, and
+    the schema carries the `roles.<role>.filters.preferredMinThroughput` row
+    (number, min 0, default 0).
+23. **Explorer** (`explorer.test.ts`, `explorer-scopes.test.ts`) — rank deltas;
+    contributions sum to `q`; `inverseCardinal` round-trips; Export preserves
+    siblings and normalizes nested roles; the scope switcher; the availability
+    overlay.
+24. **Project scope** (`project-updater.test.ts`, `project-setup.test.ts`,
+    `project-roles-command.test.ts`) — project mode scopes config/state/history/
+    lock to `<cwd>/.omp`; the day gate is per scope; the registry write;
+    `setupProject` add/drop/keep, `--dry-run`, `--force`; the command is
+    registered.
+25. **Session model** (`session-model.test.ts`) — the hook fires only after a real
+    write on a real `default` change; a dry run and a kept default skip it; a
+    throwing hook is contained.
+26. **Docs** (`docs.test.ts`) — every relative markdown link resolves; no
+    reference to a moved doc; the CHANGELOG shape; `package.json` `files` entries
+    exist; the README has no dev-only headings.
+27. **Architect provenance** (`architect-provenance.test.ts`) — the vendored
+    prompt header matches `ARCHITECT_PROMPT_VERSION` and the version named in
+    [`agent-authoring.md`](agent-authoring.md).
+28. **Architect retry** (`agent-architect.test.ts`) — a malformed architect
+    response is retried once and the parsed spec returned; an always-malformed
+    run throws carrying the raw output; a `run()` failure propagates with no
+    retry.
+29. **Error reporting** (`error-report.test.ts`) — `reportUnexpectedError`
+    through its one entry point with fully injected deps: `off` → no fetch, no
+    ledger write, `action: "off"`; `auto` + token → exactly one create whose body
+    carries the fingerprint marker, plugin version, runtime, platform, label and
+    stack, with the `bug`/`needs-triage` labels; redaction (home dir → `~`,
+    `node_modules` prefix → `<pkg>/`, `sk-…`/`ghp_…`/`Bearer …` → `<redacted>`)
+    leaves nothing token-shaped in the title, body or URL; dedupe (an open match
+    is commented on, never a second create; a closed match on a later day
+    creates a fresh issue linking the old one); caps (the 4th distinct
+    fingerprint in one UTC day, and a fingerprint already created earlier the
+    same day even when its issue is closed, are both `suppressed` with no
+    request; a later UTC day re-creates the regression); permission fall-through
+    (`401`/`403`/`404` from the search or the create → `offered`, the creation
+    counters untouched); no token → no HTTP at all, an `issues/new` URL whose
+    query carries the title, body and labels opened exactly once; token
+    resolution (`GITHUB_TOKEN` → `GH_TOKEN` → the injected `gh` executor, with a
+    throwing or empty `gh` falling through to the URL path); containment (a
+    rejected fetch, a 500, a timeout and a malformed JSON body each → `failed`,
+    one notify line, no throw); fingerprint stability (a varying id inside the
+    message with the same first stack frame dedupes, a different label does not);
+    `ask` (a `confirm(true)` surfaces the payload and files exactly once, a
+    `confirm(false)` or a headless session sends nothing and offers the URL — it
+    never silently falls back to `auto`); an unrecognized `errorReporting` value
+    fails closed (no fetch, no ledger, `action: "off"`); a defect **string** (as
+    `reportDefect` passes it) is fingerprinted by its message, so two distinct
+    aborts under one label differ; the in-process guard; and a ledger that
+    tolerates junk and truncates a pathological stack — including a multi-byte
+    one — to the 64 KB **byte** budget.
+30. **CI/CD** (`ci.test.ts`) — the workflow parses and declares both jobs with
+    `release` needing `test`; the `test` job's command is identical to the
+    pre-commit hook's; the tag trigger covers every CHANGELOG tag link and the
+    package version; `.node-version` is the file the workflow reads; the release
+    job declares `id-token: write` and references no npm token; the release job
+    runs the `notes` guard before `npm publish` and both steps name the same
+    notes file; the four lockstep version fields agree; no tracked file lives
+    under `cache/`; and
+    `scripts/release.ts` against fixture repos — `due`'s exit codes, `notes`'s
+    five refusals and its exact section body, `cut --dry-run` writing nothing,
+    and `cut` bumping the four fields, cutting the changelog, committing and
+    tagging while refusing a dirty tree, an existing tag, an empty
+    `## [Unreleased]` and a HEAD that is not `origin/main`.
+31. **Auto-pin** (`auto-pin.test.ts`) — the selector gains the best route's
+    `@<slug>`; the chain carries every gate-passing route of the primary (all but
+    the pinned best) then of each fallback model, in score order (the `vfm` route
+    score desc, spec #71), deduped by
+    provider slug; a degraded / over-`maxPriceUsdPerM` / endpoint-filter-failing /
+    whitelist-blocked route is excluded; the manual pin wins and the no-pin path
+    is bare; a re-run is idempotent and a pin change prunes the old key.
+32. **Provider display** (`provider-display.test.ts`,
+    `provider-display-wiring.test.ts`) — `providerPinOf` reads the single
+    `openRouterRouting.only[0]` (else `vercelGatewayRouting`), preferring
+    OpenRouter, and returns null for a missing/empty/multi-entry block or a
+    non-string entry; `providerPinOfSelector` parses the trailing `@<slug>` with
+    and without a thinking suffix and returns null without one;
+    `withProviderSuffix` appends, strips a previous suffix, is idempotent, and
+    returns the base name for a null pin; `applyProviderDisplay` mutates a
+    record's `name` and no-ops on a non-record or a non-string name. The wiring
+    (a fake `pi`/`ctx` recording handlers) suffixes the live model on
+    `session_start`, applies the pin parsed from `event.to` on
+    `retry_fallback_applied` (falling back to the live model's routing block),
+    applies the pin from `event.model` on `retry_fallback_succeeded`, re-derives
+    from the live model on `turn_start` (the silent-restore case), strips when
+    neither source carries a pin, never calls `pi.setModel`, and contains a
+    throwing live-model read.
+
+## Live checks
+
+- **E2E refresh** — `omp plugin link .` → new omp session → observe the day-gated
+  run; `/refresh-roles` → verify the `config.yml` diff + notification; a second
+  same-day session → no-op; a forced `/refresh-roles` matches the in-session
+  decisions; `/explore-roles --no-open` → `curl` the notified port for
+  `/api/bootstrap`, a repeat invocation keeps the same port, and the port is
+  released when the session ends.
+- **Agent creation** — `/create-agent` in a session (RPC mode dispatches slash
+  commands) → the architect authors the body, the updater line
+  `@<n>: (unset) -> <selector>` + `modelRoles.<n>` land in `config.yml`, then a
+  headless spawn whose record reads
+  `{"agent":"<n>","agentSource":"user","modelRole":"<n>"}` with `resolvedModel`
+  equal to the role's selector.
+- **Benchmark sources** — a role weighting `bench:<id>` fetches the source and
+  ranks on it; the explorer's `/api/bootstrap` lists the external metric with its
+  derived metadata.
+- **Agent removal** — `/remove-agent --name <n>` in a session deletes both
+  artifacts and the updater pass removes the config entries.
+- **CI/CD** — push a branch and open a PR (or push to `main`) and `gh run watch`
+  the `test` job green; push a tag that disagrees with the package version and
+  confirm the `release` job fails in the `notes` guard *before* any publish;
+  run the release job against an already-cut tag
+  (`gh workflow run ci.yml --ref v<version>`) and confirm npm's `gitHead` equals
+  the tag commit, `dist-tags.latest` moved, and the GitHub Release body is that
+  version's CHANGELOG section. See [`ci.md`](ci.md).

package/docs/dev/writes-and-state.md

@@ -1,22 +1,25 @@
 # Writes and state
 
 Maintainer reference for the plugin's write path and the files it owns.
-Normative decisions: [`spec.md`](spec.md) §6.2, §6.4, §8 and §9. Implementation:
-`src/config-edit.ts` (the edit), `src/state.ts` (the files), `src/updater.ts`
-(orchestration), `src/agent-pins.ts` (the pin-derived sync).
+Implementation: `src/config-edit.ts` (the edit), `src/state.ts` (the files),
+`src/updater.ts` (orchestration), `src/agent-pins.ts` (the pin-derived sync).
 
 ## The surgical config edit
 
 `patchConfig(configText, patch)` (`src/config-edit.ts`) rewrites
 `~/.omp/agent/config.yml` as **text**, line-oriented — no runtime YAML
-dependency (required for marketplace installs). It touches exactly three
-surfaces:
+dependency (required for marketplace installs). omp hot-reloads `config.yml`
+(task/eval preflight re-reads the settings) and writes it under its own
+`config.yml.lock`, so the plugin's mtime guard and refresh lock are what keep a
+concurrent omp write from being clobbered. It touches exactly three surfaces:
 
 - `modelRoles.<role>` values (upsert missing roles at the block end, two-space
   indent; delete the line for a role that left the managed set);
 - the managed keys inside `retry.fallbackChains` (replace-in-place, never
   duplicate a YAML key; prune keys the plugin wrote on a previous run and no
-  longer references);
+  longer references). A pinned role's chain key is the level-free form of its
+  selector and carries the `@<slug>` pin (`openrouter/<id>@<slug>`); the pin
+  rides before the thinking suffix on the values;
 - the plugin-managed names in `task.disabledAgents` (add/remove, other entries
   and their order preserved).
 
@@ -36,16 +39,40 @@ Rules the algorithm enforces:
   documents; `findTopLevel` rejects inline values, indented-only blocks and
   duplicates; `chainEntries` rejects odd indents, duplicate chain keys and
   unexpected lines. Any surprise throws `ConfigEditError` and the caller aborts
-  without writing.
+  without writing. **Exception — an empty managed collection is not a
+  surprise.** omp re-serializes `config.yml` through its own writer (Bun's
+  `YAML.stringify` via `stringifyYamlConfig`), which spells an empty collection
+  either as a null token on the key line (`task.disabledAgents: null`) or as a
+  flow collection on its own indented line (`fallbackChains:` then `    {}`,
+  `task.disabledAgents:` then `    []`). Both read as the empty collection, and
+  the patch path rewrites that placeholder line into the block form
+  (`disabledAgents:` + `- "designer"`) instead of refusing the run — a user's
+  `disabledAgents: null` used to abort every session start with `config edit
+  refused` (issue #54). Any *other* inline value on a managed key line
+  (`disabledAgents: ["x"]`) still throws. The rule applies at both levels: a
+  managed key's own placeholder (`task.disabledAgents:` + `    []`) **and** an
+  empty **parent** block (`task:` + `  {}`, `retry:` + `  {}`), where the new
+  child key replaces the `{}` line — inserting *after* it emits YAML no parser
+  accepts, while the read-back self-check still passes, so that corruption was
+  silent. A prune-only patch creates no key at all.
+- **Blocks are bounded by their own indent.** `blockEnd(lines, key, indent)`
+  stops at the first line at or above the key's indent, so `chainEntries` sees
+  only the `fallbackChains` block. A sibling `retry.<other>` key — omp ships ten
+  (`retry.maxRetries`, `retry.enabled`, `retry.baseDelayMs`, …) and writes them
+  into the same `retry:` block — therefore ends the block instead of throwing
+  `unexpected line in fallbackChains block`, and a new chain key is appended
+  inside the block rather than after it.
 - **Read-back self-check.** After patching, `patchConfig` re-reads the patched
   text through the same line-oriented reader and asserts it equals the intended
   state (every upserted role, every removal, the exact chain-key set, every
   disabled-agent add/remove). On any mismatch it throws — never corrupt.
 - **Atomic write.** `writeConfigAtomic(path, patchedText, mtimeBefore)` re-stats
   the mtime first; a changed mtime means the config moved underneath and returns
-  `"conflict"` (the caller re-reads and recomputes). Otherwise it writes
-  `<path>.llm-role-tmp` and `rename`s over the target. A file absent at read
-  time (`mtimeBefore === 0`) is writable only while still absent.
+  `"conflict"` (the caller re-reads and recomputes). Otherwise it `mkdir -p`s the
+  target's directory, writes `<path>.llm-role-tmp` and `rename`s over the target.
+  A file absent at read time (`mtimeBefore === 0`) is writable only while still
+  absent. The `mkdir -p` is what lets the project scope write into
+  `<cwd>/.omp/plugins/`, which omp does not create until something writes there.
 - **3-attempt mtime retry.** `writePatch` (`src/updater.ts`) loops up to
   `CONFLICT_RETRIES = 3`: read → patch → atomic write; a `"conflict"` retries,
   three conflicts abort with `CONFLICT_ABORT`.
@@ -53,31 +80,191 @@ Rules the algorithm enforces:
   (below); a lock it cannot acquire returns `"locked"` and the caller aborts
   (or no-ops, for the day-gated sync).
 
+## The settings lock file and the feature flags
+
+The plugin's settings live in `~/.omp/plugins/omp-plugins.lock.json` →
+`settings["omp-llm-role"]` (or the project lock in project mode). The write path
+(`src/role-settings.ts`) stores every role key as a **flat dotted key** because
+omp's `/settings` Plugins tab shallow-merges the settings object and does not
+flatten nested objects. The capability flags join that set:
+
+- `features.<id>` — the global flag for capability `<id>`;
+- `roles.<role>.features.<id>` — the per-role flag (wins over the global in
+  either direction);
+- `roles.<role>.preferOwnProvider` — the `providerPinning` policy field.
+
+`mergeExport` writes these as flat dotted keys alongside the existing role keys
+(`roles.<name>.weights.<metric>`, `.description`, `.enabled`, `.locked`,
+`.required`, `.thinking`, `.providerPin`, `.cacheHitRate`, `.lambda`,
+`.filters.image`). `mergeExport` first deletes the role's existing
+`roles.<name>.*` flat dotted keys, so an Export means "make this role's keys
+match this def" — a capability flag set back to inherit (or a cleared knob) is
+removed rather than left stale. `mergeRemove` needs no change: its existing
+`roles.<name>.` prefix already covers the new keys, so `/remove-agent` deletes
+them.
+
+**Validation aborts** (notify, no write), in the existing one-error-string-per-
+violation style (`resolveSettings` in `src/settings.ts`):
+
+- `features.<id> is not a known capability` / `role <name>: features.<id> is not
+  a known capability` — an unknown id, global or per-role;
+- `features.<id>: must be a boolean, got <json>` / `role <name>: features.<id>
+  must be a boolean, got <json>` — a non-boolean flag value;
+- `features: must be an object, got <json>` / `role <name>: features must be an
+  object` — a non-record `features`;
+- `role <name>: preferOwnProvider must be a boolean, got <json>` — a non-boolean
+  policy field.
+
+**`mergeRawSettings` does NOT expand.** `resolveSettings` calls
+`expandFeatures` per role, so its `roles` map is the **effective** def (the
+authored def with the enabled flags' recommended knobs filled in, `features`
+stripped). `mergeRawSettings` — and therefore `roleUniverse(...).def` — keeps the
+**authored** def (shipped ⊕ raw overrides, no preset, `features` retained),
+because that is the def the explorer edits and exports. Expanding inside
+`mergeRawSettings` would make a role whose only authored change is a flag read as
+dirty against its own baseline, and `mergeExport` (which writes every key of a
+dirty def) would persist the expanded knobs as explicit values — silently
+converting the flag into hard values and defeating both "adjust by hand" and
+re-applying a changed recommendation. Export writes the **flag**, never the
+expanded values.
+
+**Near-collision warning.** omp's plugin-level `enabledFeatures` (present in the
+lock file, `null` by default) is a **different gate** — omp's own feature
+gating — that this plugin never reads (zero occurrences in `src/`, `web/`,
+`tests/`). The plugin's `features.<id>` keys are plugin settings, not omp
+feature flags; do not merge the two.
+
+## The provider-pin auto-configuration (spec #58)
+
+When a role's **expanded** def sets `preferOwnProvider: true` (the
+`providerPinning` capability's recommended bundle) and carries **no** manual
+`providerPin`, the updater binds the role's requests to the chosen model's best
+route by writing that route's provider slug as the selector's `@<slug>` suffix:
+
+```
+openrouter/<id>@<slug>:<level>
+```
+
+omp's `splitUpstreamRouting` parses the trailing `@<slug>` and applies
+`compat.openRouterRouting = { only: [slug] }`, so the price the ranking assumed
+is the price the request pays. The slug is `bestRoute(chosenModel, def)`'s
+`providerSlug` — the max-value-for-money candidate route (see scoring.md,
+*Route ordering*). The pin is re-derived on every run, so it follows the ranking when the
+chosen model changes. A manual `roles.<role>.providerPin` **wins** over the
+automatic choice and stays a **hard** gate (a model with no matching route is
+ineligible); clearing it resumes the automatic choice.
+
+**Chain shape.** The primary's chain key is the selector minus the trailing
+`:level` — so it **includes** the pin (`openrouter/<id>@<slug>`). Its values, in
+order:
+
+1. the same model on **every** gate-passing route except the pinned best, in
+   score order (the `vfm` route score desc — see scoring.md, *Route ordering*),
+2. then, for each of the `fallbackChainDepth` fallback models, that model on
+   **every** gate-passing route, in score order.
+
+Length = `(R_primary − 1) + Σ R_fallback`, where `R` is the number of distinct
+gate-passing provider slugs for that model — data-driven, not a fixed cap.
+Entries dedup by chain value, so two routes sharing a provider slug collapse to
+one. Every entry carries the same `@<slug>` suffix (and the role's thinking level
+where the target advertises it). A model with no gate-passing route contributes
+nothing. The gates are the engine's `rankedRoutes` set: non-degraded
+(`status === 0`), the endpoint capability filters (`tools` / `minContextTokens` /
+`minOutputTokens`), a usable billed price, and the `maxPriceUsdPerM` cap
+(thinking-adjusted). The soft `filters.preferredMinThroughput` floor is a
+**preference, not a gate** — it sorts below-floor routes last but never drops
+them, so a below-floor route still appears in the chain. The allowed-providers
+whitelist is not a `rankedRoutes` filter: the updater prunes each model's route
+pool to allowed providers before ranking, which is what makes `rankedRoutes`
+whitelist-aware. The probe walk target is unchanged (`1 + fallbackChainDepth` —
+it counts models, not routes).
+
+**Prune on a pin change.** Because the chain key includes the pin, a pin change
+changes the key; the updater's chain prune (`state.pluginWrittenChainKeys`) drops
+the old key so a stale pinned chain does not linger.
+
+**No-ops.** The auto-pin writes nothing when: the capability is off
+(`preferOwnProvider` not `true`); a manual `providerPin` is set; the role is
+locked or disabled; the selector is not an OpenRouter selector; the model has no
+route data; no route passes the role's endpoint filters; or the best route has no
+usable billed price. With the capability off the written config is byte-identical
+to before.
+
 ## State, history, lock
 
 All under the agent dir (`agentDir()` in `src/state.ts`), next to the config
 they describe. Agent-dir resolution: `OMP_LLM_ROLE_AGENT_DIR` (test hook) →
 `PI_CODING_AGENT_DIR` → non-default `OMP_PROFILE` → `~/.omp/agent`.
 
+**Project scope (issue #27).** `runUpdater` resolves the project dir as
+`<cwd>/.omp` (omp's project dir, no walk-up). When
+`<cwd>/.omp/plugins/omp-plugins.lock.json` carries a non-empty `omp-llm-role`
+settings entry, the whole run is scoped to the project: the config
+(`<cwd>/.omp/config.yml`), the settings read (project lock over the user-level
+one), and the state/history/lock files all live under `<cwd>/.omp`. The global
+`~/.omp/agent/config.yml` is **never** written in project mode. Without a project
+settings entry the global behavior is unchanged.
+
 | File | Shape | Purpose |
 |---|---|---|
 | `llm-role-state.json` | `{ lastRunDay, managedRoles, roleLastSelector, pluginWrittenChainKeys, previousModelRoles, managedDisabledAgents }` | day gate, managed-role set, last selectors, plugin-owned chain keys, pre-write `modelRoles` snapshot, **managed `task.disabledAgents` names** |
 | `llm-role-history.jsonl` | one JSON row per completed run: `{ ts, trigger, keyMeta{isFreeTier, limitRemaining, creditsRemaining}, decisions[] }` | append-only decision log |
+| `llm-role-projects.json` | `{ projects: [{ root, lastUsed }] }` | the plugin's memory of the projects where `/project-roles` was used (the explorer's scope list) |
 | `.llm-role-refresh.lock` | `O_EXCL` create, holds `"<pid> <iso>"` | serializes concurrent session starts; stale (> 60 s) locks are unlinked and retried once |
+| `llm-role-error-reports.json` | `{ reports: [{ fingerprint, label, title, firstSeen, lastSeen, createdDay?, offeredDay?, count, action, issueNumber?, issueUrl? }], dayCount: { day, count } }` | what the opt-in `errorReporting` path filed, offered or suppressed |
+
+- `llm-role-projects.json` (issue #28) is **global knowledge** and lives in the
+  agent dir, never a project's `.omp`. `registerProject(root)` (`src/project-registry.ts`)
+  upserts the root (stamped now) and prunes entries whose project lock file is
+  gone, then replaces the file atomically (temp + rename) so two sessions in
+  different repos cannot interleave into a corrupt file. Reading is pure and
+  tolerant: a missing or unparseable file yields `[]`. The write point is the
+  project-aware `runUpdater`, **immediately after `resolveProjectDir()` and
+  before any early return**, guarded only by `dryRun` — so a day-stamped no-op
+  session start still registers the project (the whole point of "projects
+  configured before this feature appear on their next session start"), while a
+  `--dry-run`/cancelled `/project-roles` registers nothing. A write failure is
+  logged, never thrown: the registry is a convenience index, not a run
+  prerequisite.
+
+- `llm-role-error-reports.json` is the ledger of the opt-in error-reporting path
+  (`src/error-report.ts`, `ERROR_REPORT_LEDGER_FILE`) — the plugin's memory of
+  what it filed, offered or suppressed. Like `llm-role-projects.json` it is
+  **global knowledge** and lives in the agent dir, never a project's `.omp`; the
+  write point is `writeLedger` (temp file + `rename`, so two sessions cannot
+  interleave into a corrupt file) and reading is pure and tolerant — missing,
+  unparseable or unexpected-shape content yields an empty ledger, so a corrupt
+  ledger never blocks reporting or the run. Each `reports[]` row remembers one
+  fingerprint's `action` (`created`/`commented`/`offered`/`suppressed`), its
+  `issueNumber`/`issueUrl`, the `createdDay`/`offeredDay` that gate re-creation
+  and re-offering, and a `count`; `dayCount` tracks the UTC-day creation total
+  that feeds the 3-creations-per-day cap. Nothing is written when
+  `errorReporting` is `off`.
 
 - `previousModelRoles` is the full snapshot of the last `modelRoles` block
   **before** the plugin changed it — a manual rollback aid; there is no rollback
-  command (spec §12).
+  command.
 - `pluginWrittenChainKeys` is what makes chain pruning safe: only keys the
   plugin wrote on a previous run are candidates for pruning; an owner-written,
   unreferenced key is left alone.
 - `managedRoles` drives `roleRemovals`: a role the plugin managed on a previous
   run but no longer does (disabled via `enabled: false` / `weights: null`, or
   removed from settings) has its `modelRoles.<role>` line deleted, so a stale
-  pin cannot keep routing `@<role>`.
+  pin cannot keep routing `@<role>`. A role whose provider pin matched no route
+  joins `managedRoles` without a rewrite, so its current chain survives the
+  prune.
 - `managedDisabledAgents` is the set of agent names the plugin added to
   `task.disabledAgents`; a later run removes a name whose agent file is gone
   (e.g. `/remove-agent`) or whose role is no longer disabled.
+- Each `decisions[]` entry carries `availabilitySource` (`"keyed-catalog"` when the
+  keyed catalog was active for that role, else `"probe"`) and `keyBlockedCount` (the
+  candidates pruned as key-blocked). The keyed-blocked set itself is deliberately
+  **not** enumerated — it can be hundreds of ids; `blocked[]` keeps its old meaning,
+  the probe-blocked ids examined in the walk. `endpointBlocked[]` records the ranking
+  ids the role's endpoint filters dropped (no capable standard-tier route), rendered
+  `; endpoint-blocked: …` on the decision line; `pinBlocked[]` records the ranking ids
+  the role's provider pin dropped (no route matching the pin), rendered
+  `; pin-blocked: …`.
 - Role removal (`/remove-agent`) goes through `removeRoleSettings` in
   `src/role-settings.ts` — the same validate/backup/atomic-write path as
   `writeRoleSettings`, deleting the role's flat dotted keys and any nested entry.
@@ -92,7 +279,8 @@ no-op. `/refresh-roles` passes `force` and always runs.
 A concurrent second starter loses the lock and finds the day already stamped →
 no-op. Session-start runs are **awaited** before the first prompt is dispatched
 (a deferred timer would be cleared when a short-lived session exits before it
-fires).
+fires). The gate is **per scope**: a project run loads the project's own
+`llm-role-state.json`, so a project and the global config each refresh once a day.
 
 ## `task.disabledAgents` sync (pin-derived, not day-gated)
 
@@ -127,27 +315,43 @@ session, not the next day. Two consequences a maintainer must know:
 ## Error handling
 
 `runUpdater` wraps the whole run; `ConfigEditError` and any other throw become
-`omp-llm-role: aborted, no write: <message>`. Abort paths and what each leaves
-untouched:
+`omp-llm-role: aborted, no write: <message>`. The aborts split into two classes,
+and the `defect` column below marks which ones `RunResult` flags: an **exception
+abort** (a bug this plugin should fix) carries `defect` and, when the opt-in
+`errorReporting` setting is on, is reported through `src/error-report.ts`; an
+**enumerated environment abort** (an expected condition) leaves `defect` unset
+and is notify-only. Abort paths and what each leaves untouched:
 
-| Failure | Behavior | State / config |
-|---|---|---|
-| Settings validation fails (spec §7) | abort, notify the offending role/key | no write, no state change |
-| Ranking data unavailable (fetch fails, no fresh cache) | abort, notify | no write, no state change |
-| Key fetch (`omp token`/registry) or `/api/v1/key`, `/api/v1/credits` fails | abort, notify | no write, no state change |
-| Catalog fetch fails | abort, notify | no write, no state change |
-| Both tier branches false (no budget, no free quota) | abort, notify | no write, no state change |
-| Lock not acquired | abort (`LOCK_ABORT`) | no write, no state change |
-| Write conflict after 3 retries | abort (`CONFLICT_ABORT`) | no write, state file untouched |
-| `patchConfig` structural surprise / self-check mismatch | abort (`config edit refused: …`) | no write, no state change |
-| Zero decisions changed | no write at all (config mtime untouched) | history row appended (full run) |
-| Dry run | decisions reported, nothing written | no day-gate stamp, no history row |
+| Failure | Behavior | State / config | `defect` |
+|---|---|---|---|
+| Settings validation fails | abort, notify the offending role/key | no write, no state change | — |
+| Ranking data unavailable (fetch fails, no fresh cache) | abort, notify | no write, no state change | — |
+| Key fetch (`omp token`/registry) or `/api/v1/key`, `/api/v1/credits` fails | abort, notify | no write, no state change | — |
+| Catalog fetch fails | abort, notify | no write, no state change | yes |
+| Both tier branches false (no budget, no free quota) | abort, notify | no write, no state change | — |
+| Lock not acquired | abort (`LOCK_ABORT`) | no write, no state change | — |
+| Write conflict after 3 retries | abort (`CONFLICT_ABORT`) | no write, state file untouched | — |
+| `patchConfig` structural surprise / self-check mismatch | abort (`config edit refused: …`) | no write, no state change | yes |
+| Any other throw inside the run (the catch-all) | abort, notify | no write, no state change | yes |
+| Zero decisions changed | no write at all (config mtime untouched) | history row appended (full run) | — |
+| Dry run | decisions reported, nothing written | no day-gate stamp, no history row | — |
+
+- **`defect` vs the enumerated aborts.** The enumerated environment aborts —
+  settings invalid, ranking data unavailable, tier `none`, lock, write conflict
+  and the key-fetch failure — are conditions of the user's machine or account,
+  not plugin defects, so `runUpdater` returns them without `defect` and the
+  extension does not report them. Only the exception aborts carry `defect` (the
+  message), and the extension's `reportDefect` helper routes it to
+  `reportUnexpectedError`. A command handler that throws *outside* `runUpdater`
+  (the six catch sites) is reported the same way via `reportCommandError`.
 
 - The day-gated agent sync's lock/conflict failures are handled separately: a
   lock it cannot acquire is benign (`{ decisions: [], wrote: false }`); a
   conflict aborts with `CONFLICT_ABORT`.
 - A role whose probed candidates are all blocked is left untouched with a notify
-  note (not an abort).
+  note (not an abort). A role whose provider pin matches no route is likewise
+  left untouched — no selector or chain upsert — with a notify note naming the
+  role and the pin.
 - The live-session model hook (`applySessionModel`) runs only after a real write
   and only on a real `default` change; a hook failure is notified but never
   fails the run (the config write already succeeded).