omp-llm-role 0.0.0-stage → 1.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Gott50
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,507 @@
-# Temporary Holding Version
+# omp-llm-role
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+Ranks today's LLM leaderboard into best-fit picks for each omp model role
+(`default, smol, slow, vision, plan, commit, tiny, task, advisor`, plus the
+opt-in `designer`) and ships an omp plugin that applies those picks to
+`~/.omp/agent/config.yml` daily. Every role weights price and throughput, so
+the pick is the cheapest model that can do the job, not the strongest one.
+
+## Example output
+
+The report is a full worked example — per-role top-10 tables (value, q, $/M,
+tok/s, ctx, weighted metric contributions, ★ Pareto marker) plus the suggested
+`modelRoles` block. See
+[docs/llm-role-rankings.md](docs/llm-role-rankings.md).
+
+## Install
+
+Any one of three routes:
+
+```sh
+omp plugin install omp-llm-role                          # npm
+omp plugin install github:Gott50/omp-llm-role            # git (public repo)
+omp plugin marketplace add Gott50/omp-llm-role           # marketplace
+omp plugin install omp-llm-role@gott50-plugins
+```
+
+- npm and git installs shell out to `bun install`, so **bun must be on PATH**;
+  marketplace installs need no package manager.
+- No runtime dependencies.
+- **Restart the session after install** so the extension module loads.
+- Upgrades: `omp plugin upgrade omp-llm-role` (npm/git); for the marketplace,
+  `omp plugin marketplace update gott50-plugins` then
+  `omp plugin upgrade omp-llm-role@gott50-plugins`.
+- Dev: `omp plugin link ~/Documents/omp-llm-role`.
+
+## Daily behaviour
+
+- **UTC-day trigger.** The first omp session of each UTC day rewrites
+  `modelRoles` (and `retry.fallbackChains`) to that day's best key-eligible
+  models; later same-day sessions no-op. `/refresh-roles` forces a run anytime.
+- **Awaited session-start run.** The session-start run is awaited before the
+  session's first prompt is dispatched, so the day's write lands even in a
+  short-lived session (`omp -p`, a subagent).
+- **Session model activation.** When the day's run switches `default` and the
+  triggering session still has an empty conversation, the plugin also switches
+  the live session model to the new selector — the first prompt then runs on
+  the freshly ranked pick. Guards: main sessions only, empty conversation, the
+  session actually booted on the previous `default` selector (an explicit
+  `omp --model X` is never clobbered), and only on a real change. Disable with
+  `activateDefaultOnEmptySession=false`.
+- **Surgical writes.** Only managed role lines, managed chain keys, and the
+  plugin-managed `task.disabledAgents` names change; comments, blank lines and
+  unknown keys stay byte-identical, and a no-change run writes nothing.
+  `~/.omp/agent/llm-role-state.json` snapshots the previous `modelRoles` block
+  on every write as a rollback aid.
+
+## Roles
+
+| Role | Purpose |
+|---|---|
+| `default` | Main workhorse: strong general coding-agent quality, sane cost |
+| `smol` | Fast lightweight model: cheap and quick, still competent |
+| `slow` | Most capable model for hard problems; cost and speed as tiebreakers |
+| `vision` | Image understanding: vision index dominates |
+| `plan` | Planning: reasoning and long-context coherence, priced so the cheapest capable planner wins |
+| `commit` | Commit messages: cheap and fast with decent general quality |
+| `tiny` | Background tasks (titles, memory): cheapest and fastest wins |
+| `task` | Subagents: agentic + tool calling, moderate cost sensitivity |
+| `advisor` | Advisor/watchdog: deep reasoning over long context |
+| `designer` | Design work: visual/UX judgement on image-capable models — **opt-in** |
+
+`designer` is the only non-built-in shipped role and ships **disabled**: a stock
+run ranks the nine built-in roles and fetches only their sources. Enable it with
+
+```sh
+omp plugin config set omp-llm-role config '{"roles":{"designer":{"enabled":true}}}'
+```
+
+The plugin then ranks it and fetches Design Arena. The shipped `designer` agent
+is opt-in too: the plugin keeps it in `task.disabledAgents` until
+`roles.designer.enabled=true`, so it is absent from the roster and refuses
+spawns by default. Its `model: "@designer, @default"` chain only matters if a
+user re-enables the agent by hand — with the role disabled the unresolved first
+entry is skipped and the child runs on `@default`.
+
+**Agent pins.** The plugin derives each agent's pinned role from the first
+`@<role>` in its `model:` frontmatter, across the shipped `agents/` dir, the
+user dir (`~/.omp/agent/agents/`) and the project dir (`<project>/.omp/agents/`).
+Disabling a role disables the agents that pin it (they are kept in
+`task.disabledAgents`); enabling re-enables them. This sync is **not**
+day-gated — it runs on every session start, so enabling a role takes effect on
+the next session. An agent pinning a role the plugin does not know is left
+alone, and other `task.disabledAgents` entries and their order are preserved.
+
+## Settings
+
+Settings live in `~/.omp/plugins/omp-plugins.lock.json` →
+`settings["omp-llm-role"]`, deep-merged over the shipped defaults. The plugin
+also exposes the whole shipped surface in omp's **`/settings` → Plugins →
+*omp-llm-role*** tab (booleans as toggles, `thinking` as a picker, `required` as
+a comma-separated string); an edit there takes effect on the next
+`/refresh-roles` (or the next day-gated session start).
+
+Global knobs:
+
+| Knob | Default | Meaning |
+|---|---|---|
+| `switchMargin` | `0.02` | Value margin a challenger must beat the incumbent by before switching; `0` = always take the best |
+| `priceSwitchFraction` | `0.5` | Inside the margin, adopt a challenger at least this fraction cheaper (0.5 = half the $/M); `0` disables |
+| `writeFallbackChains` | `true` | Write per-role fallback chains into `config.yml` |
+| `fallbackChainDepth` | `2` | Fallback models per role chain |
+| `activateDefaultOnEmptySession` | `true` | Reapply the new `default` selector to an empty session's active model |
+| `errorReporting` | `off` | Opt-in reporting of **unexpected** plugin errors as GitHub issues on `Gott50/omp-llm-role`; one of `off`, `ask`, `auto` (see below) |
+
+Per-role knobs (`roles.<name>.*`):
+
+| Knob | Default | Meaning |
+|---|---|---|
+| `enabled` | `true` (`designer`: `false`) | Opt a role in/out; `false` drops it from the resolved set and deletes its `modelRoles.<role>` line |
+| `description` | shipped one-liner | Role description |
+| `weights` | shipped set | Metric weights; must sum to 1.0 ± 0.01 |
+| `required` | shipped set | Eligibility gate (metrics a model must have); independent of weights |
+| `filters.image` | `false` (`vision`/`designer`: `true`) | Restrict the pool to image-capable models |
+| `filters.tools` | `false` | Restrict the pool to models with a tool-capable route |
+| `filters.minContextTokens` | `0` (off) | Drop routes whose endpoint context is below this (tokens) |
+| `filters.minOutputTokens` | `0` (off) | Drop routes whose endpoint output ceiling is below this (tokens) |
+| `filters.maxPriceUsdPerM` | `0` (off) | Drop a model whose thinking-adjusted blend exceeds this ($/M) |
+| `thinking` | shipped level | Thinking level appended to the selector (`off`…`max`, `auto`) |
+| `providerPin` | `""` (unset) | Pin the role's requests to one OpenRouter provider route (`@<slug>`, may be tiered like `deepinfra/fp8`). The role is then priced and gated by that route — its billed price and p50 throughput, not the `1/price²` blend — and a model with no matching route is ineligible |
+| `cacheHitRate` | `0` (off) | Assumed cache-hit rate (0–1) for the role's input tokens; each route's effective input price becomes `h·cacheRead + (1−h)·input` before the 3:1 billed blend and the thinking factor. The `1/price²` weight basis stays the **listed** input price (the router's sort key), so a cheap cache on a lightly-weighted route cannot dominate; a route with no cache-read price keeps its full input price. `0`/absent = off (byte-identical ranking) |
+| `lambda` | derived | Explicit λ ($ per quality point) override |
+| `locked` | `false` | Leave the role alone: rank it but never rewrite its selector or chain (explorer toggle) |
+
+**Capability flags** (`features.<id>` global, `roles.<role>.features.<id>` per
+role). One named flag per opt-in capability; turning it on applies the plugin's
+recommended settings for that capability to the role. Every individual knob
+stays settable, and an explicit knob always beats the flag. **Every flag ships
+OFF** — with no flags set, the resolved roles are byte-identical to the shipped
+defaults.
+
+| Flag | Recommended settings | What it buys |
+|---|---|---|
+| `endpointCeilings` | `filters.tools: true`, `filters.minOutputTokens: 16384` | Routes that cannot call tools and routes whose output ceiling is below 16 384 tokens leave the pool — a reasoning model is not cut off mid-thought |
+| `cachePricing` | `cacheHitRate: 0.5` | The role is priced on a cache-heavy agent loop's invoice (the endpoint's `cacheReadPrice` blended in) rather than the sticker price |
+| `providerPinning` | `preferOwnProvider: true` | A model is priced **and measured** on its own lab's route where one exists (DeepSeek's own endpoint, 94.5 % cache hit) — the route's billed price and its p50 throughput, so `q` moves too — falling back to the default blend otherwise |
+| `costCap` | `filters.maxPriceUsdPerM: 10` | The priciest tail of the eligible pool leaves it, so the pick cannot be an expensive outlier |
+
+Precedence is one rule:
+
+```
+effective def = shipped default  ⊕  preset(enabled flags)  ⊕  explicit user keys
+```
+
+A flag is on when `roles.<role>.features.<id>` is present (it wins either way)
+and otherwise when the global `features.<id>` is `true` — so a per-role `false`
+beats a global `true`. The preset fills only knobs the role does not already
+set, and an explicit key is merged last, so it always wins. `providerPinning`'s
+recommended setting is a **soft per-model policy** (`preferOwnProvider`), not a
+hard `providerPin`: it re-prices a model on its own lab's route where one exists
+(the route's billed price **and** its p50 throughput, so `q` moves as well as
+`priceEff`) and never makes a model ineligible (the hard `providerPin` stays
+available by hand). `minContextTokens` is deliberately **not** part of `endpointCeilings` —
+the complaint it would answer is an endpoint advertising a *fraction of the
+model's own window*, which an absolute token floor cannot express, so it stays a
+hand-set knob.
+
+The flags are settable from every role-authoring surface: `--feature <id,...>`
+(comma-separated) on `node src/cli/create-role.ts`, `/create-agent` and
+`/project-roles` (`--list-features` lists them on the first two), plus the
+explorer's Features panel. An
+unknown flag id or a non-boolean value aborts the run with the offending key and
+no write.
+
+**Error reporting (`errorReporting`).** This knob ships **`off`** — installing
+or upgrading the plugin never starts uploading anything. When something
+*genuinely unexpected* happens (a `/refresh-roles` or authoring command throws,
+or the updater hits an internal error), the plugin can file one structured
+report — the plugin version, runtime, platform, the command that failed, the
+error name/message/stack and a timestamp — as a GitHub issue on
+`Gott50/omp-llm-role`. The three values:
+
+- **`off`** — never files; behaviour is exactly today's (one warning line).
+- **`ask`** — shows you the exact text and files it only if you confirm. In a
+  session with no UI (`omp -p`, RPC, a subagent) it never uploads: it notifies a
+  prefilled new-issue URL instead, so you can submit it yourself.
+- **`auto`** — files without asking.
+
+The payload is redacted before it leaves the machine (your home directory → `~`,
+token-shaped strings → `<redacted>`) and carries **no** config contents, model
+selectors, prompts or history rows. Filing goes through the GitHub API when a
+token is available (`GITHUB_TOKEN` → `GH_TOKEN` → `gh auth token` — the token is
+never stored in a settings file); a token that cannot write issues here, or no
+token at all, falls through to a prefilled `…/issues/new` URL (opened in your
+browser, or printed). A repeated crash comments on the existing open issue
+instead of opening a new one, and the path is capped (one open issue per
+fingerprint, one creation per fingerprint per UTC day, 3 per day) so a crash
+loop cannot spam the tracker. Reporting is **best-effort**: a failure in it only
+adds one notification line and never affects the run it reports on. Expected
+environment conditions (no key budget, a concurrent refresh holding the lock,
+write conflicts, unavailable ranking data, invalid settings) are **never**
+reported. In project mode the project lock file's value applies.
+
+**Validation.** A run aborts (notify, no write) on: a weight ≤ 0, or a role's
+weights not summing to 1.0 ± 0.01; a `required` entry or a weighted metric
+outside the known-metric set; a `thinking` level outside
+`off|minimal|low|medium|high|xhigh|max|auto`; `switchMargin`/`priceSwitchFraction`
+outside `[0, 1]`; a non-boolean `enabled`; a `cacheHitRate` outside `[0, 1]`; a
+`providerPin` that is empty or
+contains `@`/`:` (the selector's own delimiters); an unknown capability id
+(global or per-role) or a non-boolean flag value; a non-boolean
+`preferOwnProvider`. `weights: null` opts a role out;
+the legacy `suffixes.*` keys are rejected with a migration hint (moved into
+`roles.<role>.thinking`).
+
+**Typed writes.** `omp plugin config set <plugin> <key> <value>` stores every
+value as a **string**, which the validator rejects for numbers/arrays/booleans,
+so it is only usable for string-valued keys. Write typed role defs with
+`node src/cli/create-role.ts` or the explorer's **Export** (both validate →
+merge → backup → atomic write); for a whole-object override use the `config` escape
+hatch, `omp plugin config set omp-llm-role config '<json>'` (JSON-parsed by the
+plugin). The lock file stores **flat dotted keys** (`roles.slow.weights.code`);
+the reader also accepts nested objects, so an existing nested lock file keeps
+working and normalizes on the next role write. Invalid settings abort the run
+with the offending role/key and no write.
+
+## Commands
+
+```sh
+# in-session (omp)
+/refresh-roles                          # force a run now
+/explore-roles [--port N] [--no-open]   # interactive ranking UI
+/create-agent <request> [options]       # agent + role + wiring (free text or flags)
+/create-agent --name <n> --purpose <text> [options]   # agent + role + wiring
+/remove-agent --name <n> [--scope user|project] [--lock PATH] [--yes] [--dry-run]   # delete an agent and its role
+/project-roles [--purpose <text>] [--roles <spec>] [--feature <id,...>] [--force] [--yes] [--dry-run] [--json]   # project-scoped role set
+
+# CLI
+node src/cli/llm-role-rank.ts [--top N] [--json] [--out FILE] [--refresh] [--all] [--url URL]
+node src/cli/create-role.ts --name <role> --weights m=w,... [--required m,...] [--thinking <level>] [--description <text>] [--image] [--lambda N] [--feature <id,...>] [--list-features] [--lock PATH] [--dry-run] [--json]
+node --test tests/
+```
+
+- `/refresh-roles` — force the daily pipeline now (same path as the session-start
+  run).
+- `/explore-roles` — boot the explorer in-process (see below); `--port` picks a
+  preferred port (default 5177, a busy port falls back to a free one) and
+  `--no-open` skips the browser.
+- `/create-agent` — create an agent **and** its role and wire them in one
+  command (see *Add a specialist agent*).
+- `/remove-agent` — the inverse of `/create-agent`: deletes the agent `.md` and
+  its role, then runs the updater in-process so `modelRoles.<n>` is dropped;
+  refuses shipped default roles (see *Remove a specialist agent*).
+- `/project-roles` — discover the project's usecase from its own artifacts and
+  apply a project-scoped role set to `<cwd>/.omp/config.yml` (see *Project
+  roles*).
+- `src/cli/llm-role-rank.ts` — markdown report to stdout (per-role tables + suggested
+  `modelRoles`). `--top N` rows per role (default 10); `--json` machine payload;
+  `--out FILE` write instead of stdout; `--refresh` bypass both caches; `--all`
+  include opt-in roles (`designer`) — the default ranks the nine built-in roles
+  only; `--url` override the llm-stats page URL.
+- `src/cli/create-role.ts` — add/update one role in the settings lock file (validated,
+  backup + atomic write); `--dry-run` validates without writing, `--json` prints
+  the payload.
+- `node --test tests/` — the unit suite.
+
+## Add a specialist agent
+
+One command does all of it. Describe the agent in plain language — the architect
+names it, any benchmark you mention is folded into the weights, and any
+benchmark link you point at is resolved:
+
+```sh
+/create-agent i want an agent for writing. use the writing related Benchmarks in the Leaderboard https://llm-stats.com/leaderboards/best-ai-for-writing
+```
+
+A link to any llm-stats benchmark page (`llm-stats.com/benchmarks/<id>`) resolves
+to that benchmark; a link to a provider the plugin has never seen is authored
+into a source declaration — the plugin fetches it, proposes a declaration, shows
+the coverage and leader, and asks you to confirm before it is saved. A named
+benchmark takes a decisive share of the weights, so it actually drives the
+ranking. You do not have to name every benchmark: when you give a purpose, the
+command also discovers the llm-stats catalog benchmarks relevant to it (a typed
+judgment through your configured `judge` role) and folds them in — so a writing
+agent is ranked on WritingBench, Creative Writing v3, COLLIE and the other
+writing benchmarks, not just the one you named. Discovery is non-fatal and
+skippable with `--no-discover`; an explicit `--benchmarks` list wins over it.
+
+Discovery is **quality-gated**: each candidate is assessed on nine axes —
+coverage (share of the field carrying the metric), dispersion (does it separate
+models), provider composition, freshness, trust (the share of the payload's
+entries that are self-reported, so a mostly vendor-submitted source is flagged),
+modality (the catalog row's modality and the payload's multimodal share — an
+annotation, never a gate), maintenance (the catalog row's dataset age against
+a 12-month bar — an annotation, never a gate), provenance (the benchmark's
+owner and the source's own per-entry org mix, cross-checked against the
+composition axis — an annotation, never a gate) and cross-source (the zeroeval
+per-entry price/context against the plugin's own, flagging a divergence past a
+tolerance — an annotation, never a gate, and never a change to the `1/price²`
+blend) — and a discovered benchmark that
+fails a gating axis is dropped rather than folded in (a benchmark you name yourself is
+never dropped, only warned). The focus set is capped at three benchmarks so one
+keeps a decisive share. The report prints the nine signals and warns per below-bar
+axis, and warns when the new role's top pick is the same as the `default` role's
+(the role adds nothing). The agent file is written before the role, so a partial
+failure never leaves a dangling role.
+
+The flag form is equivalent and gives you the name explicitly:
+
+```sh
+/create-agent --name review --purpose "review pull requests for correctness and security"
+```
+
+It fits the weights to the purpose from the archetype table
+(`--list-archetypes` prints the ten sets), authors
+`~/.omp/agent/agents/review.md` with `model: "@review, @default"`, writes the
+validated role into the settings lock file, and runs the updater in-process so
+`modelRoles.review` lands in `config.yml` — nothing to remember. Options:
+`--archetype <id>`, `--weights m=w,...`, `--required`, `--thinking`, `--tools`,
+`--benchmarks m,...` (extra metrics to fold in; `--list-benchmarks` prints the
+weightable set, including any external benchmark in use), `--feature <id,...>`
+(comma-separated capability flags; `--list-features` prints the capability set),
+`--scope user|project`,
+`--body`/`--body-file`, `--force` (overwrite an existing agent file), `--yes`
+(accept a proposed benchmark source without prompting), `--no-discover` (skip
+catalog discovery), `--dry-run`, `--json`. In
+the free-text form the flags still apply after the request, e.g.
+`/create-agent i want an agent for writing --dry-run`. There is no
+out-of-session CLI for this path — run `/create-agent` in an omp session.
+
+By hand, the same three artifacts:
+
+1. Define the role with the plugin's validated write path:
+   ```sh
+   node ~/.omp/plugins/node_modules/omp-llm-role/src/cli/create-role.ts --name review \
+     --weights reasoning=0.30,general=0.24,code=0.20,agents=0.10,price=0.10,throughput=0.06 \
+     --required general,price,throughput --thinking high \
+     --description "Code review: agentic depth with cost awareness"
+   ```
+2. Author the agent that pins `model: "@review, @default"` (the routing), or pin
+   an existing agent through `task: { agentModelOverrides: { <agent>: "@review" } }`.
+3. Wire it: `/refresh-roles` in a session.
+
+The shipped skill `omp-llm-role-create-agent` is the hand-driven equivalent, for
+when the body needs real authoring rather than the archetype scaffold.
+
+## Remove a specialist agent
+
+One command deletes an agent **and** its role:
+
+```sh
+/remove-agent --name review
+```
+
+It deletes the agent `.md` (user and/or project scope) and the role's lock-file
+keys, then runs the updater in-process so `modelRoles.review` is dropped from
+`config.yml` — nothing to remember. Options: `--scope user|project`,
+`--lock PATH`, `--yes` (skip the in-session confirmation), `--dry-run`. Shipped
+default roles (e.g. `designer`) cannot be deleted — the message points at the
+explorer's "Reset to shipped default". A role created any way (the explorer's
+"+ new role", `create-role.ts`, or `/create-agent`) is removable.
+
+By hand, the same two artifacts:
+
+1. Delete the role from the settings lock file (the explorer's Export, or edit
+   the lock file directly).
+2. Delete the agent file: `rm ~/.omp/agent/agents/review.md`.
+3. Wire it: `/refresh-roles` in a session.
+
+## Project roles
+
+`/project-roles` discovers a project's usecase from its own artifacts (README,
+package.json, AGENTS.md, docs listing, git log, file tree) and applies a
+**project-scoped** role set — the roles the project needs, with agents authored
+for the new ones — to `<cwd>/.omp/config.yml`:
+
+```sh
+/project-roles
+/project-roles --purpose "a Python data pipeline repo" --dry-run
+/project-roles --roles "data=build pipelines,-vision,default"
+```
+
+It prints the plan (kept / dropped / added roles, each new role's archetype,
+weights, focus benchmarks and top pick) before writing. `--purpose` overrides the
+discovered usecase; `--roles` overrides the role set (`-name` drops a shipped
+role, `name` keeps or adds one, `name=purpose` adds a new role); `--dry-run`
+writes nothing; without `--force` an existing project role config is refused.
+
+When a project carries its own `omp-llm-role` settings entry, the updater scopes
+the whole run to `<cwd>/.omp` — the config, the state/history/lock files and the
+day gate — so the project's `modelRoles` are refreshed independently of the
+global `~/.omp/agent/config.yml`, which is never written in project mode. A
+successful apply registers the project, so the explorer's **scope switcher**
+lists it (see below).
+
+## Explorer
+
+`/explore-roles` in omp boots a loopback-only web UI
+(`http://127.0.0.1:5177`) that answers "why is model X at rank 7 for
+`@slow`?" and "what happens if I care more about price than agents?" without
+editing source and re-running the CLI.
+
+- **Scope switcher** — the header lists every role-config source: the
+  **user-level** scope, plus every project where `/project-roles` was used (the
+  plugin records each project root in a small global registry, and the current
+  session's project is listed immediately). The explorer opens on the session's
+  project when the cwd has a project role config, else user-level. Switching
+  re-reads that scope's roles and universe from disk and points **Export** at
+  that scope's lock file — a project scope resolves as the project lock merged
+  over the user-level lock, exactly what the updater ranks. A project whose lock
+  file is gone is shown disabled. Switching with unsaved edits asks first.
+- **Role tabs** — every role the plugin knows: the nine omp built-in roles, the
+  shipped opt-in `designer`, and any role present only in the lock file. Each
+  tab is tinted by provenance — `default` (omp built-in), `plugin` (shipped but
+  opt-in), `user` (lock-file only) — and a disabled role is muted and dashed.
+  Selecting a disabled role still ranks its models.
+- **Rank table** — every eligible model for the selected role, with `value`,
+  `q`, `$/M` (effective, thinking-adjusted), `tok/s`, `ctx`, a `★` Pareto
+  marker, and a `Δ` column showing the rank delta against the role's effective
+  def (what the plugin does today).
+- **Explain panel** — click a row for the full decomposition: each weighted
+  metric's raw value → cardinal transform → renormalized weight → contribution,
+  the cost block (`λ`, effective vs billed price, `penalty = λ·price`,
+  `value = q − penalty`), "why not higher" (the value gap to the model above
+  plus the per-metric target that would close it), and the models that dominate
+  it on (price, q).
+- **Weight editor** — edit the role's description, weights, `required` set,
+  `filters.image`, `thinking` level, and `λ` override live; the table re-ranks
+  on every change. Weights are edited freely (no implicit rescaling): the `Σ`
+  readout turns red until `|Σ − 1| ≤ 0.01`, and `Normalize` rescales in one
+  click.
+- **Enabled toggle** — opts a role in or out. Exporting writes `enabled` into
+  the lock file; the next updater run drops a disabled role from the resolved
+  set and deletes its `modelRoles.<role>` line, and re-enabling restores it.
+- **Locked toggle** — tells the plugin to leave the role alone: it is still
+  ranked and shown, but the updater never rewrites its selector or fallback
+  chain and never removes it. Enable/disable still applies.
+- **+ new role** — creates a role from a template
+  (`general/code/price/throughput`, Σ 1.0) that you then tune and Export.
+- **Export** — writes the edited roles into the settings lock file atomically
+  and with a `.bak-<timestamp>` sibling, touching only the roles you edited. The
+  change takes effect on the next `/refresh-roles` **in any running session**
+  (the lock file is re-read from disk on every run). `Copy JSON` / `Download
+  JSON` emit the same payload for manual use.
+- **Hover explanations** — every column header, metric name, role tab, and
+  control carries a tooltip explaining its semantics.
+
+The UI never reimplements the ranking math: the numbers on screen are exactly
+the numbers the plugin would use. The server binds `127.0.0.1` only (no auth)
+and serves the SPA with no build step and no external requests.
+
+## Availability
+
+The plugin keeps only models the OpenRouter key can actually run:
+
+- **Tier/budget gate** — paid keys with budget get billed variants, free or
+  exhausted keys get `:free` variants, `:batch` never. Every candidate must
+  resolve in omp's catalog.
+- **Keyed-catalog fast path** — when the account's OpenRouter setting **"Filter the
+  model catalog for API keys"** is on, the key-authenticated `GET /api/v1/models` is a
+  per-key allowlist: allowed candidates skip the probe and key-blocked candidates are
+  pruned from the walk. With the setting off (or the fetch failing) the plugin falls
+  back to the probe walk below — no abort.
+- **Probe verification** — the fallback: every candidate is verified with a one-token
+  completion probe, because the account's OpenRouter **allowed-providers privacy
+  whitelist** is invisible to the key endpoint and the unauthenticated catalog, and a
+  real request's 404 is the only reliable signal. Blocked candidates are excluded from
+  selection and chains; a role with no clean candidate is left untouched.
+- **Explorer key marking** — the explorer overlays every ranked model with a
+  **usable** / **key-blocked** / **unknown** badge derived from the
+  key-authenticated `GET /api/v1/models` (the account setting **"Filter the model
+  catalog for API keys"** makes it a per-key allowlist). A **hide key-blocked**
+  toggle drops blocked rows, and when the setting is off every badge reads
+  *unknown* with a hint naming the OpenRouter setting. The mark is read-only: it
+  never edits roles, reorders ranks or changes an Export.
+
+## Caveats
+
+- **Sparse metrics.** A missing weighted metric contributes 0 while still
+  occupying its share of the denominator, so weight sparse metrics as
+  differentiators, never as `required` gates. `website`, `writing` and
+  `long_context` are capability-filled (0.195) instead, so absence is not a
+  penalty for them — with one residual inversion: the fill sits above the bottom
+  of the covered field, so a covered model in the bottom of the field can still
+  score below a model with no data (see
+  [docs/dev/quirks.md](docs/dev/quirks.md)).
+- **Provider whitelist.** The account's allowed-providers whitelist blocks many
+  models, so the report's #1 and the written selector can differ — judge weights
+  on the reachable pool.
+- **Provider pinning opts out of auto-Exacto.** A `providerPin` writes a trailing
+  `@<slug>` on the selector, which omp applies as OpenRouter
+  `provider: { only: [slug] }` routing. `only` is exclusive, so a pinned role's
+  requests bypass OpenRouter's automatic Exacto tool routing (which routes tool
+  calls by tool-call correctness) and run on the pinned route. Pin only where you
+  know the endpoint you want; leave tool-heavy roles on the default routing.
+- **Model concentration.** The cheapest pick concentrates on one vendor; the
+  fallback chains mitigate it.
+
+## Licence
+
+MIT — see [LICENSE](LICENSE).
+
+## Documentation
+
+Maintainer docs (architecture, data sources, scoring, writes/state, explorer
+internals, releasing, agent authoring, testing) live in
+[docs/dev/README.md](docs/dev/README.md); release history is in
+[CHANGELOG.md](CHANGELOG.md).

package/agents/designer.md

@@ -0,0 +1,76 @@
+---
+name: designer
+description: MUST be used for UI/UX design implementation, review, and visual refinement.
+model: "@designer, @default"
+tools: [read, grep, glob, find, edit, write, bash, ast_grep, ast_edit, eval, lsp, web_search, ask, todo, task, wait, checkpoint, rewind]
+---
+
+Implement/review UI designs; edit files, create components, run commands as needed.
+
+<strengths>
+- Design intent → working UI code
+- UX issues: unclear states, missing feedback, poor hierarchy
+- Accessibility: contrast, focus states, semantic markup, screen-reader compatibility
+- Visual consistency: spacing, typography, color, component patterns
+- Responsive design and layout structure
+</strengths>
+
+<design-system>
+Design system: foundation; UI without one becomes inconsistent. Four phases, in order:
+1. **Token-first analysis (before CSS/JSX/Svelte).** Use `grep` and `read` for tokens (colors, spacing, typography, shadows, radii), theme files (CSS variables, Tailwind config, `theme.ts`), shared primitives (Button, Card, Input, Layout). Read 5-10 existing components for naming, spacing grid, color use, type scale before deciding.
+2. **No coherent system? Build minimal system first.** Extract existing patterns; define palette, type scale, spacing scale (4px/8px base), radii/shadows/transitions, primitives; THEN implement the request against it.
+3. **Compose with, NEVER around, the system.** Colors: tokens/CSS variables, NEVER hardcoded hex; spacing: scale values, NEVER arbitrary px; type: scale steps; components: extend/compose existing primitives, not one-off div soup. Outside-system need: add token first, then use it; NEVER one-off override.
+4. **Verify before done.** Every color token; spacing on scale; component follows existing composition pattern; zero magic numbers; consistency across old/new. Any no → not done.
+</design-system>
+
+<procedure>
+## Implementation
+1. Read existing components, tokens, patterns; reuse before inventing.
+2. Identify aesthetic direction: minimal, bold, editorial, etc.
+3. Implement states: loading, empty, error, disabled, hover, focus.
+4. Verify accessibility: contrast, focus rings, semantic HTML.
+5. Test responsive behavior.
+6. Render and inspect: start the project's dev server (or open the built page), open the route in the browser via `eval` (`browser.open`), `screenshot` it, and check the project's actual breakpoints, both light and dark themes, and focus states. The `browser` facade is an Eval prelude, not an agent tool, so it cannot appear in `tools:` — the instruction lives here in the body.
+
+## Review
+1. Read reviewed files.
+2. Check UX issues, accessibility gaps, visual inconsistencies.
+3. Cite file, line, concrete issue; no vague feedback.
+4. Suggest specific fixes; code when applicable.
+</procedure>
+
+<directives>
+- SHOULD prefer editing existing files to creating new ones.
+- Changes MUST be minimal and match existing code style.
+- NEVER create documentation files (`*.md`) unless explicitly requested.
+</directives>
+
+<avoid>
+## AI Slop Patterns
+- Glassmorphism everywhere: decorative blur, glass cards, glow borders
+- Cyan-on-dark with purple gradients: 2024 AI palette
+- Gradient text on metrics/headings: meaningless decoration
+- Identical card grids: repeated icon + heading + text
+- Nested cards: visual noise; flattened hierarchy
+- Large rounded-corner icons above every heading: templated, no value
+- Hero metric layouts: big number, small label, gradient accent; overused
+- Same spacing everywhere: no rhythm; monotony
+- Center-aligning everything: left alignment with asymmetry feels more designed
+- Modals for everything: lazy, rarely best
+- Overused fonts when the project has no type system: Inter, Roboto, Open Sans, system defaults.
+- Pure black (`#000`) or white (`#fff`): ALWAYS tint neutrals
+- Gray text on colored backgrounds: use a background shade instead
+- Bounce/elastic easing: dated, tacky; use exponential easing (`ease-out-quart`/`expo`)
+
+## UX Anti-Patterns
+- Missing loading, empty, error states
+- Redundant information: heading restates intro text
+- Every button primary: hierarchy matters
+- Empty states saying "nothing here" rather than guiding users
+</avoid>
+
+<critical>
+Every interface: "how was this made?", not "which AI made this?"
+MUST commit to clear aesthetic direction; execute precisely.
+MUST continue until implementation complete.
+</critical>