@lmctl-ai/lmctl 0.2.19 → 0.2.21

package/durable-memory/model_selection.md

@@ -1,381 +0,0 @@
-# Model Selection
-
-`model=` on a `_MEMBER_` line is a provider-specific model request.
-
-For `.lmctl` teamfile compatibility mode:
-
-- `lmctl lint <teamfile>` resolves `model=` for `claude`, `gemini`, `qwen`,
-  and `hermes` against the lmprice model catalog (see
-  `## lmprice Model Catalog` below) and warns when the model is not found.
-  This REPLACED the static tested-model lists that used to live in this
-  document (2026-09-24) — hand-copied lists contradicted the catalog.
-
-**Best-effort contract (operator, 2026-09-25, hardened 2026-09-29): lint is
-advisory and is NOT claimed to be 100% up to date.** Checks can lag provider
-reality in BOTH directions — a model lint warns about may have just been
-added upstream, and a model lint accepts may have just been retired. Every
-model check is a warning, never an error, full stop: lint has exactly two
-outcomes (ok / warning) and always exits 0 on warnings-only. There used to be
-a `--strict` flag that could promote warnings to a failing exit code; the
-operator removed it entirely (2026-09-29: "no need to give users the choice
-with no obvious value") — lmctl decides the scoping, not the caller. Exit 1
-is reserved for genuine validation errors (a malformed teamfile, an
-unparseable member line), never for a warning. The runtime stance for
-anything unrecognized or out of date is always: "not recognized — could
-still be valid; no cost information," never a failure. Cost is equally
-best-effort (n/a with a reason, never a guessed price).
-- `codex` and `agy` model validation queries each provider's own live
-  catalog command (`codex debug models`, `agy models`) instead of a
-  hand-copied list — see their `## Codex` / `## Antigravity` sections
-  below.
-- `opencode` model validation reads configured models from
-  `~/.config/opencode/opencode.json` and WARNS (never injects) when a
-  `model=` isn't declared there and isn't a config-free github-copilot
-  GPT/Gemini/o-series id. When `model=` is present, `lmctl lint` also
-  opens a short-lived ACP session and asks opencode itself to validate
-  `model=` and `effort=`. It never sends a prompt. See the `## OpenCode`
-  section below.
-- `kimi` model validation reads configured aliases from
-  `$KIMI_CODE_HOME/config.toml` (default `~/.kimi-code/config.toml`) and
-  WARNS (never injects) when `model=` isn't one of the declared
-  `[models.<alias>]` table names, or when the config file itself is
-  missing/unreadable. Same rationale as opencode: kimi model aliases are
-  user-defined, not a fixed vendor catalog, so a hardcoded list in this
-  doc would be wrong for every user whose aliases differ. See the
-  `## Kimi` section below.
-- `hermes` model ids are the ENCODED `<provider>:<model>` form Hermes's
-  ACP server advertises (`acp_adapter/model_catalog.py`). lmctl sends the
-  value through ACP `session/set_model` and does NOT maintain a Hermes
-  model list to validate against: Hermes resolves models at runtime from
-  its own provider config + auth, so any hardcoded list here would be
-  wrong for most users. Lint warns only when `model=` has no `provider:`
-  prefix. Hermes only treats a leading `<x>:` as a provider delimiter when
-  `x` is a known provider name (`hermes_cli/models.py:parse_model_input`);
-  a bare id stays the model name and Hermes then has to infer the provider
-  by detection, which fails for models it cannot place. After the shape
-  check, lint ALSO resolves the id against the lmprice catalog (the hermes
-  adapter splits the `provider:` prefix) and warns on a catalog miss —
-  warning-only, since Hermes may still resolve it from user config. See the
-  `## Hermes` section below.
-- `copilot` accepts `model=` syntactically, but model selection is not
-  supported and lint warns that the value is ignored.
-- `lmctl seed <teamfile>` passes `model=` through to the provider seed
-  call. It does not contact provider model catalogs.
-
-The `claude`/`gemini`/`qwen` tables in the sections below are HISTORICAL
-evidence records of the retired tested-model lists (pre-2026-09-24); lint no
-longer consults them — it resolves against the lmprice catalog.
-
-## lmprice Model Catalog
-
-Since 2026-09-24 (0.2.1), model validation and costing share one catalog
-layer fed by the public lmprice dataset
-(`https://lmctl.com/dataset/model_list.json` + `aliases.json`):
-
-- **Loading**: ETag-conditional refresh into `~/.lmctl/cache/lmprice/` with a
-  6h TTL (`LMCTL_PRICE_TTL=0` forces; `LMCTL_PRICE_DATA=<dir>` pins a
-  snapshot and disables network); bundled snapshot from build time is the
-  offline fallback; the loader never throws.
-- **Resolution order** (provider-aware): provider-qualified alias hit
-  (evidence `established`/`asserted` → standards chain → pinned row;
-  `unresolved` is terminal, never guessed) → provider adapter family match →
-  miss. States: priced / estimated / known-unpriced / unresolved / unknown /
-  no-catalog.
-- **Invariant (operator, 2026-09-25): if we can lint, then we can cost.**
-  lmprice build validation rejects half-bindings (asserted alias ending in a
-  null record pin); lmctl surfaces any legacy known-unpriced as the lint
-  warning "recognized but no price information."
-- **Freshness**: the catalog is only as current as the last lmprice publish
-  plus the client TTL — see the best-effort contract at the top of this
-  document. `cost --debug` shows the dataset's `generated_at`.
-
-## Claude
-
-| Configured `model=` | Evidence |
-| --- | --- |
-| `claude-opus-4-6` | Ported from lmctl0 tested-model catalog. |
-| `claude-opus-4-7` | Ported from lmctl0 tested-model catalog. |
-| `claude-opus-4-8` | Added from operator-confirmed availability on 2026-06-13; `lmctl lint` compatibility smoke accepts it. |
-| `claude-opus-5` | Operator-confirmed valid 2026-07-25; next-gen Opus release following the `-5` naming (cf. `claude-sonnet-5`/`claude-fable-5`, dropping the `-4-` infix). Added to the lint tested-model catalog. |
-| `claude-sonnet-4-6` | Ported from lmctl0 tested-model catalog. |
-| `claude-sonnet-5` | Operator-confirmed 2026-07-05 (via GitHub Copilot: `opencode --model github-copilot/claude-sonnet-5 run …` → "build · claude-sonnet-5"). Also a valid native Claude Code model id (`claude --model claude-sonnet-5`). |
-| `claude-haiku-4-5` | Ported from lmctl0 tested-model catalog. |
-| `claude-fable-5` | Session observed locally via `lmctl0 ls` on 2026-06-11; `lmctl lint` compatibility smoke accepts it. Anthropic announced on 2026-06-12 that access to Fable 5 was suspended for customers; keep the id valid for existing teamfiles, but do not use it for new work while unavailable. |
-| `claude-fable-5-1` | Fable 5.1, math.lmctl's designated review model as of 2026-09-02; requested addition to the lint tested-model catalog to silence the "not in the tested model list" warning on seed. |
-
-## Codex
-
-**2026-09-15: this section is no longer read by `lmctl lint`.** It used to
-be a hand-copied list that went stale in BOTH directions within the same
-week: it kept `gpt-5.4-mini` listed as valid after OpenAI withdrew it from
-the subscription (194 codex members with no `model=` pinned were silently
-falling through to it as the ambient default), and separately omitted
-`gpt-5.6-terra`/`gpt-5.6-luna` after an operator quota downshift moved 260
-members onto them — both real, working models (`lmctl prompt` confirmed
-live), producing 45 fleet-wide false-positive warnings. `codex debug
-models` ("Render the raw model catalog as JSON") shipped since the first
-incident and gives codex the same live-catalog treatment `agy` already
-has — lint now queries it directly (`model-selection-lint.ts`'s
-`loadCodexModels`), filtering to entries with `"visibility": "list"` (the
-catalog's own "offered to users" flag — `gpt-reserve`/`codex-auto-review`
-come back `"hide"` and are excluded the same way a UI picker would skip
-them). Left as historical reference only, not maintained going forward.
-
-| Configured `model=` | Evidence |
-| --- | --- |
-| `gpt-5.5` | Observed from `codex debug models` on 2026-09-15. |
-| `gpt-5.6-sol` | Observed from `codex debug models` on 2026-09-15. |
-| `gpt-5.6-terra` | Observed from `codex debug models` on 2026-09-15. |
-| `gpt-5.6-luna` | Observed from `codex debug models` on 2026-09-15. |
-| `gpt-6-astra` | Observed from `codex debug models` on 2026-09-15. |
-
-**Banned as a build model (operator, 2026-09-06):** `gpt-5.3-codex-spark`
-— plenty of regular-GPT quota headroom now, so there's no cost reason left
-to reach for the cheap/fast classification model on build work. `lmctl
-lint` still warns on explicit `model=gpt-5.3-codex-spark` for this reason
-regardless of live-catalog membership (it has since also dropped out of
-the live catalog entirely).
-
-**Retired:** `gpt-5.4-mini` -- removed from the OpenAI subscription
-2026-09-15 while 194 codex members across ~90 teamfiles had no `model=`
-pinned and were falling through to it as the ambient default. Unlike
-`claude-fable-5` above (suspended, may return, kept valid), this id was
-withdrawn outright and is not expected back.
-
-## Gemini
-
-**2026-09-24: lint no longer reads a tested-model table here** — gemini
-models resolve against the lmprice catalog (chat rows only; embedding /
-image / tts / realtime rows are excluded by the adapter). Note the
-`gemini` provider requires an API/enterprise Google account; personal
-subscriptions should use `agy`.
-
-## Qwen
-
-**2026-09-24: lint no longer reads a tested-model table here** — qwen
-models resolve against the lmprice catalog (bare, `dashscope/*`, and
-`qwen/` rows). `qwen/qwen3.5-plus` is an established identity whose
-current catalog price is null (known-unpriced — not a warning);
-`qwen/qwen3.6-plus` is deliberately unresolved in the alias table.
-
-## Antigravity
-
-Antigravity (`agy`) is a distinct provider from Gemini even though it stores
-state under `~/.gemini/antigravity-cli`. `lmctl` supports it through native
-terminal resume plus one-shot `agy -p` turns.
-
-**2026-09-05: this section is no longer read by `lmctl lint`.** It used to
-be a hand-copied snapshot of `agy models`' output (literally captioned
-"Observed from `agy models` on <date>" below) — which went stale on every
-new model release (fable-5.1 once, then Gemini 3.6/3.7/3.8 all landing the
-same day). `agy` is the one provider with its own live model-catalog
-command (`agy models`, tab-separated `<id>\t<Display Name>` lines), so lint
-now queries it directly (`model-selection-lint.ts`'s `loadAgyModels`)
-instead of maintaining a copy of it here. Left as historical reference only,
-not maintained going forward.
-
-| Configured `model=` | Evidence |
-| --- | --- |
-| `Gemini 3.5 Flash (Medium)` | Observed from `agy models` on 2026-07-20. |
-| `Gemini 3.5 Flash (High)` | Observed from `agy models` on 2026-07-20. |
-| `Gemini 3.5 Flash (Low)` | Observed from `agy models` on 2026-07-20. |
-| `Gemini 3.1 Pro (Low)` | Observed from `agy models` on 2026-07-20. |
-| `Gemini 3.1 Pro (High)` | Observed from `agy models` on 2026-07-20. |
-| `Claude Sonnet 4.6 (Thinking)` | Observed from `agy models` on 2026-07-20. |
-| `Claude Opus 4.6 (Thinking)` | Observed from `agy models` on 2026-07-20. |
-| `GPT-OSS 120B (Medium)` | Observed from `agy models` on 2026-07-20. |
-
-**Pricing (2026-09-29, operator decision, math.lmctl findings 002/003):** separate from the lint
-question above ("is this a real agy model," answered live) is "does this resolve in the lmprice
-catalog for cost purposes" — the fleet's largest unpriced population (86 members, 4 base models,
-11 effort-tier declared strings). Root cause: agy exposes Google's Gemini models under its own
-display-label strings with a reasoning-effort tier baked into the name (`"Gemini 3.8 Flash (High)"`),
-and no upstream catalog source (LiteLLM or OpenRouter) will ever carry that exact string —
-normalizing punctuation can't fix a naming SYSTEM difference, only an explicit, evidenced row can
-(see `backlog/`'s finding-002/003 threads for the full reasoning).
-
-**Two layers, discovered while fixing this, not designed up front:** lmprice's own `aliases.json`
-already independently covers 4 of the 11 strings (established evidence, sourced from
-antigravity.google/docs/models, added 2026-09-24 — predates this fix): `gemini 3.8 flash (high)`,
-`gemini 3.8 flash (medium)`, `gemini 3.1 pro (high)`. Those resolve via lmctl's existing alias-table
-step and never touch new code. `src/providers/model-catalog.ts`'s `agyAdapter`/
-`AGY_DISPLAY_TO_IDENTITY` (added here) covers the remaining 7 lmprice hasn't published yet — all of
-3.7-flash and 3.6-flash had zero coverage from either mechanism before this. Confirmed live via
-`resolveOfferForCost`'s `provenance` field: `"alias-pin"` for the 4 lmprice already has,
-`"route-preference"` (this new table) for the other 7 — no conflict, no double-resolution. Raw
-`agy models` output this table was built from (verified live, 2026-09-29, agy binary on this
-machine):
-
-```
-gemini-3.8-flash-high	Gemini 3.8 Flash (High)
-gemini-3.8-flash-medium	Gemini 3.8 Flash (Medium)
-gemini-3.8-flash-low	Gemini 3.8 Flash (Low)
-gemini-3.7-flash-high	Gemini 3.7 Flash (High)
-gemini-3.7-flash-medium	Gemini 3.7 Flash (Medium)
-gemini-3.7-flash-low	Gemini 3.7 Flash (Low)
-gemini-3.6-flash-high	Gemini 3.6 Flash (High)
-gemini-3.6-flash-medium	Gemini 3.6 Flash (Medium)
-gemini-3.6-flash-low	Gemini 3.6 Flash (Low)
-gemini-3.1-pro-high	Gemini 3.1 Pro (High)
-gemini-3.1-pro-low	Gemini 3.1 Pro (Low)
-claude-sonnet-4-6	Claude Sonnet 4.6 (Thinking)
-claude-opus-4-6-thinking	Claude Opus 4.6 (Thinking)
-gpt-oss-120b-medium	GPT-OSS 120B (Medium)
-```
-
-Every effort-tier variant of one base model maps to that base model's own catalog identity —
-reasoning effort/thinking budget changes how many tokens a turn generates, not the per-token rate,
-true of every major vendor's reasoning-effort parameter (OpenAI, Claude, Gemini alike).
-
-**agy fronts Claude too, not just Gemini (math.lmctl, 2026-09-29):** 3 members declare `Claude Opus
-4.6 (Thinking)` on `provider=agy`. `coversRecord`/`AGY_FIRST_PARTY_PROVIDERS` were Gemini-only when
-this was first written -- a real gap for Claude-on-agy, caught before it silently priced at zero.
-Verified per-string, live:
-
-- `claude opus 4.6 (thinking)` already had an established display-label alias in lmprice's table
-  (like the 4 Gemini ones above, added 2026-09-24) -- resolves via `alias-pin`, never touches
-  `AGY_MODEL_TO_IDENTITY` at all. No code change needed for this one specifically.
-- `claude sonnet 4.6 (thinking)` had NO such alias and was a genuine `unknown` -- fixed by adding
-  `"claude sonnet 4.6": "claude-sonnet-4-6"` to `AGY_MODEL_TO_IDENTITY`, broadening `coversRecord` to
-  `/gemini|claude/`, and adding `"anthropic"` to `AGY_FIRST_PARTY_PROVIDERS` so the bare
-  `litellm:claude-sonnet-4-6` (provider `anthropic`) row wins the tie-break over routed variants
-  (`azure_ai/`, `deepinfra/`, `vertex_ai/`, `snowflake/`, `perplexity/`, `openrouter/`).
-- `gpt-oss 120b (medium)` remains **deliberately unmapped**: unlike Claude/Gemini, OpenAI does not
-  host this open-weight model itself, so there is no single first-party row to prefer among a dozen
-  equally-plausible third-party inference providers (groq, fireworks_ai, cerebras, deepinfra, ...).
-  Picking one without evidence of which route agy actually uses would be exactly the kind of guess
-  this whole explicit-mapping design exists to avoid -- stays `unknown` until there's real evidence
-  (e.g. an antigravity.google/docs/models entry, same provenance as the existing aliases).
-
-## OpenCode
-
-opencode model ids use `<provider>/<model>` shape. **lmctl does NOT manage opencode
-model config.** When lmctl spawns an opencode member it writes only
-`permission: "allow"` (the YOLO contract — opencode ACP has no
-`--dangerously-skip-permissions` flag) plus its own `mcp.lmctl` bridge entry into
-the member's `.opencode/opencode.json`. It never injects a provider/model catalog.
-Models resolve from **opencode's own default models.dev catalog + your
-`~/.config/opencode/opencode.json`** — you own the model config.
-
-What needs declaring:
-- **github-copilot GPT / Gemini / o-series** → served **config-free** from opencode's
-  default catalog. Just `opencode auth login github-copilot`, then
-  `model=github-copilot/gpt-5.5`, etc.
-- **github-copilot Claude** (`claude-opus-4.8`, `claude-sonnet-5`, …) → **must be
-  declared** in your `opencode.json`. Copilot proxies Claude through its own API and
-  opencode's default catalog does not route them (→ "404 page not found"); declaring
-  them under a `github-copilot` provider (`@ai-sdk/openai-compatible`,
-  `baseURL https://api.githubcopilot.com`) fixes it.
-- **Custom providers** (`deepseek/*`, `z-ai-coding/*`, `openrouter/*`,
-  `ollama-local/*`, …) → **must be declared** (not in models.dev).
-
-Copy the published sample and adjust for your account:
-
-```sh
-mkdir -p ~/.config/opencode
-cp examples/opencode.json ~/.config/opencode/opencode.json   # or: curl -sL https://lmctl.com/examples/opencode.json -o ~/.config/opencode/opencode.json
-```
-
-**Lint = detect + warn (never inject).** `lmctl lint`/`seed` warns when a
-`provider=opencode model=` is **not declared** in your `opencode.json` and isn't a
-config-free github-copilot GPT/Gemini/o id — pointing you at the sample. It also
-warns when `model=` is absent (opencode would silently pick its own default). It
-does not edit your config.
-
-`effort=`/variants: an opencode member's `effort=` maps to opencode's `--variant`
-(reasoning effort). Declare the variants on the model in your `opencode.json` (see
-the sample's Claude/GPT entries with `variants: { high, xhigh, max }`); on the ACP
-prompt path lmctl sends `session/set_config_option` with `configId: "model"` and
-`configId: "effort"` before `session/prompt`.
-
-For each opencode member with `model=`, `lmctl lint` runs the same ACP
-initialization and config-option calls, then exits before `session/prompt`. A
-provider rejection text is preserved inside an `ACP RPC error:` diagnostic so
-lint reflects opencode's live catalog and validation rules. If the local binary
-cannot start, complete its handshake, or respond before the probe timeout, lint
-skips this live check without treating the environment problem as a teamfile
-error.
-
-Example:
-
-```md
-_MEMBER_ alias=Reviewer provider=opencode model=github-copilot/gpt-5.4 effort=xhigh sessionid=<seeded-id>
-```
-
-## Kimi
-
-Kimi (Moonshot AI's Kimi Code CLI) speaks ACP, same transport family as opencode/
-gemini/copilot. `model=` values are **user-defined aliases** declared in
-`$KIMI_CODE_HOME/config.toml` under `[models.<alias>]` table headers (e.g.
-`[models."kimi-code/k3"]`), not a fixed vendor catalog — same shape as opencode,
-not claude/codex. `lmctl` does not manage this file; it only reads it to validate
-`model=` at lint/seed time and warns (never injects) when the config is missing
-or the alias isn't declared.
-
-Kimi's ACP mode defaults to permission-**prompting** (unlike every argv-YOLO or
-config-file-YOLO provider). lmctl forces it into full-auto via an ACP protocol
-call (`session/set_config_option`, `configId: "mode"`, `value: "auto"`) sent
-after `session/new`/`session/load` and before the first `session/prompt` —
-fail-closed: if that call is rejected, lmctl throws rather than proceeding, on
-the one-shot transport (`src/providers/transports/acp.ts`). See
-`src/providers/yolo-contract.ts`'s `PROTOCOL_LEVEL_YOLO` set for the contract.
-
-Kimi 0.28.x exposes `thinking` through ACP only as an `on`/`off` toggle.
-Turning it on selects the current model alias's `default_effort` from
-`config.toml`; it does not accept a session-level effort value. Choose or
-define a different model alias when a different default effort is required.
-Accordingly, lmctl warns that `_MEMBER_ effort=` is ignored for Kimi.
-
-Example:
-
-```md
-_MEMBER_ alias=Reviewer provider=kimi model="kimi-code/k3" sessionid=<seeded-id>
-```
-
-`effort=` support by provider:
-
-| Provider | Status |
-| --- | --- |
-| `opencode` | Supported (via the model's declared `variants` in opencode.json). |
-| `kimi` | Not supported on the Kimi 0.28.x baseline; effort is fixed by the selected model alias's `default_effort`. |
-| `claude` | Claude Code has native `--effort`, but lmctl has not wired `_MEMBER_ effort=` to Claude yet. |
-| `codex` | Native effort surfaces exist; lmctl has not wired `_MEMBER_ effort=` to Codex yet. |
-| `agy` | No verified effort flag; only `--model` supported. |
-| `hermes` | Not supported. Hermes exposes reasoning effort through its own config (`--reasoning`) but not through its ACP surface, so lmctl ignores `effort=` for a Hermes member and warns. |
-
-## Hermes
-
-Hermes is driven over ACP (`hermes acp`), and its ACP model ids are the
-encoded `provider:model` strings it advertises in `session/new`
-(`acp_adapter/model_catalog.py`, `encode_model_choice`). lmctl applies
-`model=` with ACP `session/set_model` when it differs from the loaded model.
-
-```md
-_MEMBER_ alias=Reviewer provider=hermes model="deepseek:deepseek-v4.1-flash" sessionid=<uuid>
-```
-
-There is deliberately **no** tested-model table here. Hermes resolves its
-model inventory at runtime from the user's own provider configuration and
-credentials, so a list in this repository would be wrong for most users —
-the same reasoning as `kimi` and `opencode`. Lint checks the SHAPE only:
-the value should carry a `provider:` prefix, because without one Hermes
-has to infer the provider by model-name detection and fails for models it
-cannot place.
-
-V1 supports ACP seeding and structured prompt/dispatch against one effective
-`HERMES_HOME`. Existing session IDs default to strict resume. Yolo mode enables
-both the shell approval bypass and the per-process ACP edit mode; plan mode is
-rejected. Native deletion, interactive resume, fork, and persistent daemon
-sessions are not supported by this integration. The reader is read-only and
-shows active transcript rows; token totals do not establish last-turn usage.
-
-Release prerequisites: Hermes must return JSON-RPC errors for missing load/resume
-IDs. Task 140 also uncovered an ACP compression-head restore defect: restart-safe
-resume after compression requires a separate Hermes fix before this provider lands.
-
-The source fixes are reviewed and committed locally in the Hermes checkout as
-`989fd87461e23493fe62250b1671f9fc368e9117`. This is not an upstream release or an
-installed-runtime update: the current GitHub identity has read-only access to
-NousResearch/hermes-agent, and validation used the fixed checkout explicitly.
-Provider publication remains gated on an approved, available patched Hermes
-runtime and final owner landing approval (task 141 notes 548-549).