@ljwei-stak/dsh-model-router 0.14.0 → 0.16.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.dsh-plugin/client.js +790 -228
- package/.dsh-plugin/index.mjs +124 -29
- package/.dsh-plugin/shared/adaptive-feedback.mjs +192 -0
- package/.dsh-plugin/shared/assignment-solver.mjs +86 -0
- package/.dsh-plugin/shared/dynamic-data-view.mjs +65 -0
- package/.dsh-plugin/shared/dynamic-data.mjs +294 -0
- package/.dsh-plugin/shared/harness-plan.mjs +5 -2
- package/.dsh-plugin/shared/livebench.mjs +57 -20
- package/.dsh-plugin/shared/model-profiles.mjs +7 -1
- package/.dsh-plugin/shared/official-tools-remote.mjs +3 -1
- package/.dsh-plugin/shared/router-state.mjs +18 -7
- package/.dsh-plugin/shared/router.mjs +115 -152
- package/.dsh-plugin/shared/run-ledger.mjs +93 -24
- package/CHANGELOG.md +29 -0
- package/INSTALLATION_GUIDE.zh.md +41 -27
- package/README.md +158 -363
- package/README.zh.md +157 -353
- package/docs/ROUTING_ADAPTIVE_LEARNING.zh.md +151 -0
- package/docs/ROUTING_DERIVATION_EXPERIMENTS.zh.md +684 -0
- package/docs/ROUTING_LIVEBENCH_DATA_SEARCH.zh.md +74 -0
- package/docs/ROUTING_LIVEBENCH_VALIDATION.zh.md +154 -0
- package/docs/ROUTING_RESEARCH.zh.md +59 -0
- package/docs/ROUTING_SELFTEST_STEPS.zh.md +192 -0
- package/docs/WORKBENCH_USER_GUIDE.zh.md +192 -0
- package/docs/assets/workbench-task-preview.jpg +0 -0
- package/package.json +21 -2
package/README.md
CHANGED
|
@@ -1,450 +1,245 @@
|
|
|
1
1
|
# Model Router · DeepSeek Harness Desktop
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Version: **0.16.0**. See [routing research](docs/ROUTING_RESEARCH.zh.md), [mathematical derivation and experiments](docs/ROUTING_DERIVATION_EXPERIMENTS.zh.md), and the [LiveBench evidence audit and validation protocol](docs/ROUTING_LIVEBENCH_VALIDATION.zh.md). The latest public 2026-06-25 score/cost tables are aggregates; its paired question-quality/cost matrix has not been obtained, and no real routing gain is claimed.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
A cost-aware model-routing workbench for **DeepSeek Harness Desktop**. Describe a task, compare your configured models, inspect the plan, and explicitly choose when to execute it.
|
|
6
6
|
|
|
7
|
-
[
|
|
7
|
+
**0.16.0:** [dynamic data and adaptive feedback](docs/ROUTING_ADAPTIVE_LEARNING.zh.md) adds opt-in public benchmark/curated fixed-USD pricing refresh and task-specific, time-decayed, shrinkage-regularized subjective utility. Objective proxy quality floors remain separate. Public sources default off; preferences are shared within one local DSH_HOME and limited to the latest 200 runs. This is not a universal vendor-price scraper or evidence of real quality–cost gains.
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
Current version: **0.16.0** (`@ljwei-stak/dsh-model-router`). Its declared host compatibility is **DeepSeek Harness Desktop 0.2.0-rc.1 / 0.2.0-rc.2**. The router and [GAL](https://github.com/Alice-Marx/deepseek-harness-galgame) have been independent plugins since 0.12.0.
|
|
10
10
|
|
|
11
|
-
|
|
11
|
+
**0.15.0 changes:** a four-tab workbench with routing advice directly below task planning, protection against stale execution previews, preserved spending totals across history pruning and retries, and a standalone mock UI preview for source development.
|
|
12
12
|
|
|
13
|
-
|
|
13
|
+
[简体中文](README.zh.md) · [Workbench guide (Chinese)](docs/WORKBENCH_USER_GUIDE.zh.md) · [Installation guide (Chinese)](INSTALLATION_GUIDE.zh.md) · [Migration](MIGRATION.md) · [Changelog](CHANGELOG.md)
|
|
14
14
|
|
|
15
|
-
|
|
15
|
+
For self-testing, follow the [desktop acceptance and quality–cost experiment steps (Chinese)](docs/ROUTING_SELFTEST_STEPS.zh.md). Start with an 18-question, three-model pilot and an explicit spending limit. Offline helpers do not collect answers or call models; first-stage selection replay is distinct from end-to-end team execution.
|
|
16
16
|
|
|
17
|
-
|
|
17
|
+
## What is included
|
|
18
18
|
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
| Area | Available behavior |
|
|
19
|
+
| Capability | Current behavior |
|
|
22
20
|
| --- | --- |
|
|
23
|
-
|
|
|
24
|
-
|
|
|
25
|
-
|
|
|
26
|
-
|
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
In **DeepSeek Harness Desktop → Plugins → Add plugin**, enter one exact package name:
|
|
35
|
-
|
|
36
|
-
| What you want | Installation input | Repository |
|
|
37
|
-
| --- | --- | --- |
|
|
38
|
-
| Model routing and official tools | `@ljwei-stak/dsh-model-router@0.14.0` | [Model Router](https://github.com/Alice-Marx/dsh-model-router) |
|
|
39
|
-
| GAL only, or GAL alongside the router | `@ljwei-stak/dsh-galgame@0.1.0` | [DeepSeek Harness GAL](https://github.com/Alice-Marx/deepseek-harness-galgame) |
|
|
40
|
-
|
|
41
|
-
For npm installation, select the official **HTTPS** source `https://registry.npmjs.org/` if a mirror has not synchronized the exact version. Install, enable, and restart when prompted. Router details should show **0.13.0** and its sidebar **Model Router**; the separate GAL package adds **Gal Module**. Either package can be installed without the other. Global `npm install -g` does not register a plugin in your Desktop profile.
|
|
42
|
-
|
|
43
|
-
### Upgrade from `@ljwei-stak/model-router-galgame`
|
|
44
|
-
|
|
45
|
-
The package was renamed in 0.13.0. The old name's last version is also 0.13.0, with identical content, so future updates only come under the new name. The plugin manager cannot update the old entry in place:
|
|
21
|
+
| Task planning | Local task classification and complexity analysis; single-route recommendation, dependent team work packages, or an explicit configured model. |
|
|
22
|
+
| Model configuration | Reads exact `provider/model` routes from Harness; optional user-supplied quality, USD prices, specialties, CLI model names, execution and billing preferences. |
|
|
23
|
+
| Routing presets | Economy, balanced and quality-first presets; quality floors, estimated token costs and bounded assignment search. Missing prices stay unknown. |
|
|
24
|
+
| Read-only execution | Preview and confirm a routed run from the workbench, or invoke it in an official Harness session. Supported headless CLIs and model-directory API calls return recorded results. |
|
|
25
|
+
| Editable CLI execution | One supported official CLI or a sequential CLI team, using approved isolated Git worktrees and conditional patch integration. |
|
|
26
|
+
| Official tools | Detect, install/update, check launch readiness and inspect login status for eight registry tools; installation jobs have logs and cancellation. |
|
|
27
|
+
| Interactive terminal | PowerShell/system shell and seven CLI targets, including fixed login commands, live output and multiple sessions. |
|
|
28
|
+
| Costs and subscriptions | Daily/monthly budget checks, economy downgrade or pause, subscription-first routing, quota cooldown and explicit handling of other subscription failures. |
|
|
29
|
+
| History and quality | Per-package results, dependency views, failure details, rerun/reassignment, ratings and optional stronger-model review. |
|
|
30
|
+
| Execution boundaries | Shows what each route can read/write and whether its process uses the Harness sandbox. |
|
|
46
31
|
|
|
47
|
-
|
|
48
|
-
2. Add `@ljwei-stak/dsh-model-router@0.14.0` and restart when prompted.
|
|
49
|
-
3. Do not keep both installed: they share the profile entry id `model-router-galgame` and the `model_router_*` tool names. Because the entry id is unchanged, router settings saved in the profile and the run history in `~/.dsh/model-router/state.json` carry over.
|
|
32
|
+
Generating a route recommendation is local: it does not start a vendor CLI or make a paid model call. Actual execution can incur charges.
|
|
50
33
|
|
|
51
|
-
|
|
34
|
+

|
|
52
35
|
|
|
53
|
-
|
|
54
|
-
2. Install `@ljwei-stak/dsh-galgame@0.1.0` in the **same profile**.
|
|
55
|
-
3. Remove the old `@ljwei-stak/model-router-galgame` plugin and add `@ljwei-stak/dsh-model-router@0.14.0` (see above). Finish both installations before continuing play; this removes the old combined GAL entry and leaves one entry from the independent GAL plugin.
|
|
56
|
-
4. Open the new GAL entry and verify your progress. The standalone plugin retains the old localStorage keys in the same profile. For a different profile or a missing state, select the matching story and import the JSON backup.
|
|
36
|
+
## Install in DeepSeek Harness Desktop
|
|
57
37
|
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
### Install a versioned release archive
|
|
61
|
-
|
|
62
|
-
Download the `.tgz` attachment from the [router releases](https://github.com/Alice-Marx/dsh-model-router/releases) (or run `npm pack @ljwei-stak/dsh-model-router@0.14.0`) or [GAL v0.1.0 release](https://github.com/Alice-Marx/deepseek-harness-galgame/releases/tag/v0.1.0). Enter the downloaded file's absolute path, for example:
|
|
38
|
+
Open **Plugins → Add plugin** and enter:
|
|
63
39
|
|
|
64
40
|
```text
|
|
65
|
-
|
|
66
|
-
```
|
|
67
|
-
|
|
68
|
-
Where a checksum sidecar is supplied, compare it with:
|
|
69
|
-
|
|
70
|
-
```powershell
|
|
71
|
-
Get-FileHash -Algorithm SHA256 -LiteralPath 'D:\Plugins\ljwei-stak-dsh-model-router-0.14.0.tgz'
|
|
72
|
-
```
|
|
73
|
-
|
|
74
|
-
You can also extract the archive and enter its inner `package` directory, which contains `package.json` and `.dsh-plugin`. The download may be stored on your preferred drive; runtime data location is controlled by the Harness profile. Upgrade through the plugin manager without modifying `app.asar` or bypassing dependency checks.
|
|
75
|
-
|
|
76
|
-
### Version history
|
|
77
|
-
|
|
78
|
-
| Version | Intended host | Scope |
|
|
79
|
-
| --- | --- | --- |
|
|
80
|
-
| **Router 0.14.0** | **0.2.0-rc.1 / 0.2.0-rc.2** | Interactive 官方工具终端 (shell and official CLIs, logins); official tools follow each vendor's latest release with an update hint, keeping signer checks. Restart Harness fully after updating. Exact npm version or `@latest`. |
|
|
81
|
-
| [Router 0.13.3](https://github.com/Alice-Marx/dsh-model-router/releases/tag/v0.13.3) | 0.2.0-rc.1 / 0.2.0-rc.2 | Official CLI runs work on Harness Desktop for Windows (sandbox runner environment; Codex uses its own read-only sandbox). |
|
|
82
|
-
| [Router 0.13.2](https://github.com/Alice-Marx/dsh-model-router/releases/tag/v0.13.2) | 0.2.0-rc.1 / 0.2.0-rc.2 | Codex in non-Git workspaces; sandboxed CLI runs on Desktop for Windows still produced no output. |
|
|
83
|
-
| [Router 0.13.1](https://github.com/Alice-Marx/dsh-model-router/releases/tag/v0.13.1) | 0.2.0-rc.1 / 0.2.0-rc.2 | Fixed `cannot get property "credentials" without inject`; Codex refused non-Git workspaces on Windows. |
|
|
84
|
-
| [Router 0.13.0](https://github.com/Alice-Marx/dsh-model-router/releases/tag/v0.13.0) | 0.2.0-rc.1 / 0.2.0-rc.2 | Health check, cost control, subscription-first billing, run history; runs fail in the Harness (fixed in 0.13.1). |
|
|
85
|
-
| **GAL 0.1.0** | **0.2.0-rc.1 / 0.2.0-rc.2** | First standalone GAL package, two core stories. Exact npm version or `@next`. |
|
|
86
|
-
| [Router 0.12.0](https://github.com/Alice-Marx/dsh-model-router/releases/tag/v0.12.0) | 0.2.0-rc.1 / 0.2.0-rc.2 | First independent router release (GAL split out). |
|
|
87
|
-
| [Combined 0.11.1](https://github.com/Alice-Marx/model-router-galgame/releases/tag/v0.11.1) | 0.2.0-rc.1 / 0.2.0-rc.2 | Historical combined router and GAL package; corrected 0.11.0 release documentation. |
|
|
88
|
-
| [Combined 0.11.0](https://github.com/Alice-Marx/model-router-galgame/releases/tag/v0.11.0) | 0.2.0-rc.1 / 0.2.0-rc.2 | Historical GAL player release. |
|
|
89
|
-
| 0.10.2 | 0.2.0-rc.1 / 0.2.0-rc.2 | Unpublished local compatibility build. |
|
|
90
|
-
| [0.10.1](https://github.com/Alice-Marx/model-router-galgame/releases/tag/v0.10.1) | 0.2.0-rc.1 only | Historical release; rejected on rc.2 by its peer dependency range. |
|
|
91
|
-
| [0.9.0](https://github.com/Alice-Marx/model-router-galgame/releases/tag/v0.9.0) | 0.2.0-rc.1 | Historical official Desktop adaptation. |
|
|
92
|
-
| 0.8.0 | DSH 0.1.7-rc.2 dependencies | Incompatible with Desktop 0.2.0-rc.1 / rc.2. |
|
|
93
|
-
| 0.4.32 | Legacy DSH settings dependency range | Legacy release; do not use it for the current host. |
|
|
94
|
-
|
|
95
|
-
Configure providers, models, and credentials on Harness's official **Models** page. The router only considers registered routes; its profile editor does not accept API keys.
|
|
96
|
-
|
|
97
|
-
## Use the Model Router workbench
|
|
98
|
-
|
|
99
|
-

|
|
100
|
-
|
|
101
|
-
*Operation schematic, not a Desktop screenshot. The generated result appears **below** the model-profile editor, so scroll down after clicking the button.*
|
|
102
|
-
|
|
103
|
-
1. Open **Model Router** in the sidebar. The top status shows how many providers and routes the official **Model directory** returned. If it is empty, configure a provider and model on Harness's official **Models** page, then click **Refresh** in the workbench's directory card. A route appearing in the directory does **not** prove that its credentials or network connection work.
|
|
104
|
-
2. In **Task planning → Task description**, write what you want done and list distinct actions on separate lines for a compound request. Choose **Single task** for one direct recommendation or **Team allocation** for dependent work packages. Team mode decomposes requests that the router classifies as complex; selecting the mode alone does not start a team.
|
|
105
|
-
3. The **Estimated budget for this run (USD)** field may show `10`. That is a **$10 estimate target**, based on your supplied prices and estimated tokens; generating the plan itself is local and free. Set it to `0` to remove the estimate constraint. Neither value caps a provider bill or authorizes a model call.
|
|
106
|
-
4. Click **Generate route recommendation**. Planning runs locally. Scroll down **past Model pricing and capabilities** to **Route recommendation**. Read the recommended `provider/model`, complexity band, estimated cost or “price needs configuration,” and the **execution channel** badge. In team mode, check each package's objective, difficulty, route, dependencies, verification checklist, and warnings. `Official CLI` means a trusted local launch entry is ready; `Model directory API` means the plan routes through Harness's configured model API.
|
|
107
|
-
5. To improve the recommendation, save each exact `provider/model` route's quality score and input/output prices in **USD per million tokens** in the **Model pricing and capabilities** editor, then click **Generate route recommendation** again. Saving a profile invalidates the previous result; a missing price remains unknown. Review the **Official tools** cards below the result for probe, install/repair, and launch-readiness status. ZCode opens its own installer and requires you to complete its directory choice.
|
|
108
|
-
6. To run a read-only task directly from the workbench, use **Run from the workbench** (在工作台执行) below the plan. It reuses the task description and planning mode (single task, team allocation, or one chosen model), lets you pick the preset, and takes an absolute workspace path (left empty, the last run's workspace is reused; the path must exist). **Preview execution plan** asks the Host for the same plan `model_router_execute` would use, including the automatic economy downgrade, plus the cost estimate, the budget check, and every reason that needs confirmation (over budget, unsandboxed headless CLI), listed together. Nothing runs until you press **Confirm and run**; the Host re-checks the reasons and refuses if something changed in the meantime. The result appears in the run history.
|
|
109
|
-
7. For editable runs or a live second opinion, open an **official Harness session** and request `model_router_consult` for a live second opinion, `model_router_tool_run` for one ready vendor CLI, or `model_router_team_execute` for dependent CLI work packages. You can also use `/router` to request a plan and `/tools` to inspect tools in a session. An editable CLI task requires approval and a clean Git repository; check the vendor run record for the actual model and charge.
|
|
110
|
-
|
|
111
|
-
For example, enter: “**Plan a complex, three-minute science-fiction short film.** Extract the premise and constraints; design a beat sheet and shot list; review visual continuity and production risks; synthesize a handoff checklist.” Choose **Team allocation**, generate the plan, and inspect which configured routes it assigns to the writing and review packages. This plugin produces a *plan* and can delegate supported CLI tasks; it does **not** render a film or control nine creative applications.
|
|
112
|
-
|
|
113
|
-
The [full Chinese workbench guide](docs/WORKBENCH_USER_GUIDE.zh.md) walks through the controls, result location, model-profile setup, and session handoff.
|
|
114
|
-
|
|
115
|
-
## Health check, cost control and quality loop
|
|
116
|
-
|
|
117
|
-
The workbench shows these at the top. The matching session tools are in parentheses.
|
|
118
|
-
|
|
119
|
-
1. **Onboarding health check** (`model_router_health`). When you first open the workbench, it checks every official tool in the registry: is it installed, which version is installed (next to the vendor's latest release, with an **update available** hint), and is it logged in. Versions are no longer pinned: the latest version comes from the npm registry (`<package>/latest`) or, for ZCode, the official download page, cached for about 12 hours; offline or on errors the card just says the latest version is unknown. New vendor releases are not tested by the plugin; parsers tolerate output changes where they can, so please report problems. Login checks only run cheap status commands with short timeouts, or check that a credential file exists (its contents are never read):
|
|
120
|
-
|
|
121
|
-
| Tool | How login is detected | Log in with |
|
|
122
|
-
| --- | --- | --- |
|
|
123
|
-
| Claude Code | `claude auth status --json` | `claude auth login` |
|
|
124
|
-
| Codex | `codex login status` | `codex login` |
|
|
125
|
-
| Gemini CLI | `GEMINI_API_KEY`/`GOOGLE_API_KEY`, or `~/.gemini/oauth_creds.json` | `gemini` |
|
|
126
|
-
| Kimi Code | `KIMI_API_KEY`/`MOONSHOT_API_KEY`, or `~/.kimi-code/credentials/kimi-code.json` (`$KIMI_CODE_HOME`) | `kimi login` |
|
|
127
|
-
| MiMo Code | `MIMO_API_KEY`, or `~/.local/share/mimocode/auth.json` (`$XDG_DATA_HOME`; may hold only provider API keys, so billing stays unknown) | `mimo auth login` |
|
|
128
|
-
| Grok Build | `XAI_API_KEY`, or `~/.grok/auth.json` (`$GROK_HOME`). On Windows with the npm prefix off drive C the plugin runs Grok with `GROK_HOME=<npm prefix>\.model-router-grok`; the health card then names that path | `grok login` (`--device-auth` without a browser); for the relocated home run `$env:GROK_HOME='<path>'; grok login` |
|
|
129
|
-
| MiniMax Code | `MINIMAX_API_KEY`, or the `status` field of the non-secret `~/.minimax/auth/prod/<cn\|global>/mcode-public/auth-state.json` (`$MINIMAX_DATA_DIR`; `authenticated` → logged in, `anonymous` → unknown because a key saved with `mcode set-minimax-key` still works) | `mcode login` |
|
|
130
|
-
| ZCode | Desktop app without a status command → unknown. Any release whose `ZCode.exe` carries a valid Authenticode signature from 北京智谱华章科技股份有限公司 is used (trust-on-first-use check of its CLI script per signed build, see below) | Welcome page → *Connect BigModel / Z.ai*; for GLM Coding Plan pick *编程套餐* in Model settings |
|
|
131
|
-
|
|
132
|
-
The file paths come from the vendors' published packages. A missing file is reported as **unknown**, not logged out, because these CLIs can also authenticate through custom providers or (MiMo) a free anonymous channel. The check never starts a login.
|
|
133
|
-
- **一键安装最新版** / **更新到最新版 X** installs `<package>@latest` from the official registry (ZCode: the newest signed installer from its download page). An install newer than the published latest is never downgraded.
|
|
134
|
-
- **去登录** (log in) shows the login command so you can copy it.
|
|
135
|
-
- Results are cached for 10 minutes. Routing **skips logged-out CLIs immediately** and goes straight to the model-catalog API; before this, Codex took about 14 s to fail and fall back. A tool is not skipped when its API key is configured. If a CLI reports a login error at run time, it is also marked as logged out — but because you were running on its subscription, later steps on that route are **paused and ask** (“订阅登录已失效…未自动改用 API Key”) instead of silently using the API key, also after a restart (`authFailures` in `state.json`), until the CLI runs on its subscription again. The same applies when a profile explicitly sets `billing: "subscription-first"` or `"subscription-only"` and the CLI is logged out. Logged-out CLIs without either signal, and CLIs that are not installed, still go straight to the API.
|
|
136
|
-
2. **Visible routing decisions**. Plans show the selected model, preset, difficulty score, estimated cost, and channel (official CLI or API, with a "CLI 未登录" badge when the CLI is logged out). The run history shows each step's actual channel. On fallback it shows the real redacted error: Claude JSON `result`, Codex `turn.failed.error.message`, or a stderr snippet. When `allowManualReassign` is on (the default), you can reassign a step to another configured route and rerun it.
|
|
137
|
-
3. **Cost control**. Set daily and monthly budgets (USD, `0` = unlimited) and see meters for today's and this month's spend.
|
|
138
|
-
- Before a run, the estimate is checked against the budget.
|
|
139
|
-
- `model_router_tool_run` is estimated the same way when you name its `provider`/`model` route (without a route the CLI default model has no price, so only an already-exhausted budget pauses it); over budget it returns `paused-budget` until called with `confirmOverBudget: true`.
|
|
140
|
-
- After a run, the actual cost is recorded. CLI-reported cost is used first (Claude `total_cost_usd`, Grok `total_cost_usd` when the server reported a complete cost, MiMo's per-step `cost`); otherwise token usage × your configured prices. Token usage is read from Claude, Codex, Gemini, Grok (`end` event), MiMo (`step-finish` parts) and MiniMax (`exec.result.usage`). Kimi and ZCode do not report usage in their headless output. Without prices, only token counts are kept.
|
|
141
|
-
- `overBudgetAction` decides what happens over budget. `downgrade` (the default) re-plans with the economy preset and pauses if it still doesn't fit. `pause` asks you right away. Continuing from a session requires `confirmOverBudget=true`, which triggers a host approval. When a run needs several approvals (editing files, API-key retry, over budget, unsandboxed CLI), they are listed together in **one** prompt.
|
|
142
|
-
- **Only API-billed spend counts against budgets**: model-catalog API calls, and official CLI calls that use an API key (injected by the plugin, inherited from the environment, or a CLI whose own login is an API key according to the health check).
|
|
143
|
-
- When an official CLI runs on a **subscription login** with no API key (for example Claude Pro/Max, or Codex signed in with ChatGPT), its reported or usage × price figure is shown separately as **订阅参考费用 (subscription reference cost, priced at API rates)** and is **not** counted toward the daily/monthly budget. The cost card and run history show both the budget-counted spend and the reference cost. Tool rows show whether each CLI is on a subscription login or bills an API key.
|
|
144
|
-
- When the login type cannot be detected (CLIs without a status command, CLIs launched by the team runner), it is treated as a subscription login unless a matching API-key environment variable is set.
|
|
145
|
-
- Routed runs follow the [subscription-first rule](#subscription-first-api-key-only-when-the-quota-runs-out) for every provider: the subscription is used first and only quota/rate-limit exhaustion switches the step to an API key, which is then counted.
|
|
146
|
-
4. **Presets** (`routingPreset`): 省钱优先 (economy) / 均衡 (balanced, the default) / 效果优先 (quality). Presets tilt the quality/cost/latency weights and move the quality floor a cheaper substitute must clear by ±0.04. Balanced is identical to the previous planner.
|
|
147
|
-
5. **Subtask DAG** (`model_router_rerun_step`). Team packages are shown in dependency columns. Every `model_router_execute`, `model_router_team_execute` and `model_router_tool_run` call is recorded with its kind and each step's status (done, fallback, failed, blocked). A team run stops at the first failure, so later steps show as blocked. **重跑此步** (rerun this step) reruns only the failed step and its unfinished downstream steps. Finished steps keep their results and feed the rerun as dependency context.
|
|
148
|
-
- Routed runs: any failed step can be rerun or reassigned.
|
|
149
|
-
- Team runs: **read-only** team runs support single-step rerun through the same signed runner and Harness sandbox. An **editable** (`workspace-write`) team run that **stopped at a failed step** can be continued (**在新工作区续跑**): a fresh isolated worktree is created from the current checkout, the earlier worktree's changes are applied to it first, then the failed step and its unfinished downstream steps run, and the combined patch is integrated as usual. This needs approval, a clean repository still at the run's base commit (otherwise it refuses), and a run recorded by this version (with its base commit). Runs that were already integrated, or wait for manual integration, cannot be rerun step by step, and successful steps of an editable run cannot be reassigned (their changes would be applied twice).
|
|
150
|
-
- Tool runs: a failed `model_router_tool_run` call can be repeated (**重新执行此调用**, packageId `direct`) with the same tool, task, mode and model. It is recorded as a new run linked by `rerunOf`; an editable call starts again from a fresh worktree of the current checkout and needs approval.
|
|
151
|
-
6. **Security boundaries**. The workbench card and the plugin settings list each route's readable and writable scope and its sandbox status:
|
|
152
|
-
- Read-only runs write nothing.
|
|
153
|
-
- File-modifying runs use an isolated Git worktree and need host approval.
|
|
154
|
-
- On Linux/macOS, a headless CLI launched directly has no Harness process sandbox. With `confirmUnsandboxedCli` on (the default), you are asked first.
|
|
155
|
-
7. **Quality loop** (`model_router_rate`). `reviewMode` can be `off`, `sample` (uses `reviewSampleRate`), or `always`. When on, a stronger configured model reviews cheap-model output, and the verdict is stored with the run. You can rate each result 👍/👎. Ratings add a small shrunk bias (at most ±0.04) to that exact `provider/model`'s quality score for future routing.
|
|
156
|
-
|
|
157
|
-
**Local data (on by default)**: run history is **saved locally by default**; there is no switch. It is kept in `~/.dsh/model-router/state.json` (or `$DSH_HOME/model-router/state.json`) and contains:
|
|
158
|
-
- the health report, subscription quota state (which subscriptions are exhausted until when) and CLIs whose subscription login failed at run time;
|
|
159
|
-
- the last 200 runs (routed, team and tool runs), with the **full task text** (up to 20,000 characters) and **each step's answer excerpt** (up to 4,000 characters);
|
|
160
|
-
- workspace paths, routing decisions, costs and ratings.
|
|
161
|
-
|
|
162
|
-
The file never leaves your machine and holds no API keys. If your tasks contain sensitive content, keep this file in mind. Delete it, or its `runs` array, to clear the history.
|
|
163
|
-
|
|
164
|
-
Several Host processes can share one DSH home: each write takes `state.json.lock`, re-reads the file, applies its change and replaces the file atomically, so concurrent runs are not lost. Subscription quota marks and runtime login failures written by another process take effect on the next call without a restart (the file is re-read when its mtime/size/inode changes), including clears. If the file cannot be parsed, it is kept as `state.json.corrupt-<time>` and the workbench and `model_router_health` (`notices`) show a warning for 7 days; recover history from that backup if needed.
|
|
165
|
-
|
|
166
|
-
**Known gaps**:
|
|
167
|
-
- Kimi and ZCode report no token usage in headless output, so their steps show "subscription login, no usage reported" or "cost unknown".
|
|
168
|
-
- ZCode login state stays unknown (no status command); MiniMax is unknown when only an API key or no state file is present. Kimi/MiMo/Grok/MiniMax detection proves only that a credential/state file says so, not that the token is still valid.
|
|
169
|
-
- The workbench starts **read-only** runs only; editable runs still start from a session (`model_router_tool_run`, `model_router_team_execute`).
|
|
170
|
-
- The UI has passed build checks, unit tests and a headless browser pass with the real client bundle and a mock Host bridge. It has not been verified in a real Harness desktop.
|
|
171
|
-
|
|
172
|
-
## Subscription first, API key only when the quota runs out
|
|
173
|
-
|
|
174
|
-
Many vendors sell coding subscriptions besides pay-as-you-go API keys: Claude Pro/Max, ChatGPT plans for Codex, Gemini, Kimi Code, the MiniMax Token Plan, the GLM Coding Plan (Zhipu / Z.ai) and others. The router's rule is:
|
|
175
|
-
|
|
176
|
-
> **If a subscription is available, use it. Only when that subscription hits its quota or rate limit does the same step switch to the API key.**
|
|
177
|
-
|
|
178
|
-
**Two kinds of subscription.**
|
|
179
|
-
|
|
180
|
-
| Kind | How it is recognised | How it runs |
|
|
181
|
-
| --- | --- | --- |
|
|
182
|
-
| **CLI account login** (`cli-login`) | The route's provider maps to an official CLI (Claude Code, Codex, Gemini CLI, Kimi Code, MiniMax Code, ZCode for GLM…) and the health check finds an account login. | The CLI runs with **all API-key variables removed** (`ANTHROPIC_API_KEY`, `OPENAI_API_KEY`/`CODEX_API_KEY`, `GEMINI_API_KEY`/`GOOGLE_API_KEY`, `KIMI_API_KEY`/`MOONSHOT_API_KEY`, `MINIMAX_API_KEY`, `ZAI_API_KEY`, …) and no key is injected, so it can only bill the subscription. |
|
|
183
|
-
| **Coding-plan key route** (`plan-key`) | A Harness provider whose base URL and key are a coding plan, marked with `subscription: "plan-key"` in its model profile. Provider ids such as `glm-coding-plan`, `kimi-code`, `minimax-token-plan` are recognised automatically; the profile setting always wins. | The plan route is called through the model catalog first. Its `apiRoute` names the pay-as-you-go route used when the plan is exhausted. |
|
|
184
|
-
|
|
185
|
-
Configure a plan-key route in Harness's **Models** page like any Anthropic- or OpenAI-compatible provider, using the vendor's documented plan endpoint and plan key (the plugin never reads or stores keys):
|
|
186
|
-
|
|
187
|
-
| Plan | Anthropic-compatible base URL (Claude Code style) | OpenAI-compatible base URL |
|
|
188
|
-
| --- | --- | --- |
|
|
189
|
-
| GLM Coding Plan | `https://open.bigmodel.cn/api/anthropic` (intl `https://api.z.ai/api/anthropic`) | `https://open.bigmodel.cn/api/coding/paas/v4` (intl `https://api.z.ai/api/coding/paas/v4`) — not the general `/api/paas/v4`, which bills account balance |
|
|
190
|
-
| Kimi Code | `https://api.kimi.com/coding/` | `https://api.kimi.com/coding/v1`, model `kimi-for-coding` |
|
|
191
|
-
| MiniMax Token Plan | `https://api.minimaxi.com/anthropic` (intl `https://api.minimax.io/anthropic`) | — |
|
|
192
|
-
|
|
193
|
-
Check each plan's terms first: some vendors (the GLM Coding Plan FAQ, for example) say plan quota applies only inside their supported coding tools, and other API use needs the standard API service. Endpoints and model ids change; verify them in the vendor docs.
|
|
194
|
-
|
|
195
|
-
**Profile fields** (in “逐模型价格与能力” or `modelProfilesJson`):
|
|
196
|
-
|
|
197
|
-
```json
|
|
198
|
-
[
|
|
199
|
-
{ "provider": "glm-coding-plan", "model": "glm-4.6", "subscription": "plan-key",
|
|
200
|
-
"apiRoute": { "provider": "zhipu", "model": "glm-4.6" } },
|
|
201
|
-
{ "provider": "anthropic", "model": "claude-sonnet-4-5", "billing": "subscription-first" },
|
|
202
|
-
{ "provider": "deepseek", "model": "deepseek-chat", "billing": "api-only" }
|
|
203
|
-
]
|
|
204
|
-
```
|
|
205
|
-
|
|
206
|
-
- `billing`: `subscription-first` (default; may be omitted), `api-only` (never use the subscription), or `subscription-only` (never fall back to an API key; the step fails with the reason instead).
|
|
207
|
-
- `subscription`: `plan-key`, `cli-login`, or `none` (always API). Omitted means automatic.
|
|
208
|
-
- `apiRoute`: only for `plan-key` routes; an exact catalog route. When the router picks that API route directly, it still uses the plan first while the plan has quota.
|
|
209
|
-
|
|
210
|
-
**Exhaustion detection and switching.** When the subscription attempt fails, its error text is matched against documented vendor messages:
|
|
211
|
-
|
|
212
|
-
| Vendor | Quota exhausted | Rate limited |
|
|
213
|
-
| --- | --- | --- |
|
|
214
|
-
| Claude Code | “You've hit your session/weekly/Opus limit · resets 3:45pm” | “Server is temporarily limiting requests”, “Request rejected (429)” |
|
|
215
|
-
| Codex | `usage_limit_reached` (`resets_at` / `resets_in_seconds`), “You've hit your usage limit” | `rate_limit_exceeded` |
|
|
216
|
-
| Gemini | `RESOURCE_EXHAUSTED`, “Quota exceeded”, “exhausted your daily quota” (`retry in Xs`) | — |
|
|
217
|
-
| Kimi Code | 403 “You've reached your 5-hour / weekly (7-day) / monthly usage limit” | “concurrent request limit”, 429 “receiving too many requests”, “engine is currently overloaded” |
|
|
218
|
-
| MiniMax | error `2056` “usage limit exceeded” / “Token Plan usage limit reached” | error `2045` |
|
|
219
|
-
| GLM | errors `1308`–`1310`, `1316`–`1321` (“Usage limit reached for … will reset at YYYY-MM-DD HH:MM:SS”, “已达到…使用上限”) | errors `1302`, `1305` |
|
|
220
|
-
| Any | — | HTTP `429`, “Too Many Requests”, “rate limit” |
|
|
221
|
-
|
|
222
|
-
On a match the subscription is marked **exhausted until the reported reset time** (`resets_at`, `resets_in_seconds`, `retry in Xs`, GLM's reset timestamp, Claude's “resets 3:45pm”), otherwise for `subscriptionCooldownMinutes` (default 60; rate limits without a time are skipped for 1 minute). The **same step is retried on the API key at once**, and later steps skip the exhausted subscription until it recovers. The run history records the channel (`billing: subscription | api`) and the reason, for example **“订阅额度已用尽(预计 10-2 18:30 恢复),已切换 API Key。”** The state survives restarts (`quota` in `state.json`).
|
|
223
|
-
|
|
224
|
-
**Other subscription failures do not spend the API key silently.** When the subscription really ran and failed for another reason (timeout, crash, unparsable output, non-zero exit, authentication error, a plan endpoint returning 5xx…), the step is **paused** by default (`onSubscriptionFailure: "ask"`). Run history marks it **等待确认** with the real error, and downstream steps wait (**等待上游确认**). In the run history choose **改用 API 重试** (retry on the API key), **重试订阅** (retry the subscription, e.g. after logging in again) or **取消** (cancel the step and the steps waiting on it). In a session, `model_router_execute` returns `status: "paused-subscription-failure"` with `awaitingConfirmation`; the model asks you and calls `model_router_rerun_step` with `subscriptionChoice: "api" | "subscription" | "cancel"`. `api` raises a host approval prompt. Set `onSubscriptionFailure` (cost card: “订阅调用失败(非额度用尽)时”) to `api` for the old automatic fallback, or `fail` to stop the step without asking. A CLI that is not installed or has no headless adapter was never attempted, so its step still uses the catalog API as before.
|
|
225
|
-
|
|
226
|
-
If a vendor changes its wording, add patterns in `quotaPatternsJson`, keyed by tool id, provider id or `*`:
|
|
227
|
-
|
|
228
|
-
```json
|
|
229
|
-
{ "kimi-code": { "quota": ["额度已用完"], "rateLimit": ["请求过于频繁"] }, "my-glm-plan": { "quota": ["1308"] } }
|
|
41
|
+
@ljwei-stak/dsh-model-router@0.16.0
|
|
230
42
|
```
|
|
231
43
|
|
|
232
|
-
|
|
44
|
+
Use the official HTTPS npm source `https://registry.npmjs.org/` when a mirror lacks that exact version. Enable the plugin, then fully quit Harness (including its tray process) and restart it. Plugin details should show **0.16.0** and the sidebar should contain **模型路由** (Model Router). A global `npm install -g` alone does not register a Desktop plugin.
|
|
233
45
|
|
|
234
|
-
|
|
46
|
+
You can instead install a local `.tgz` archive or its extracted inner `package` directory through the plugin manager. Both must contain `package.json` and `.dsh-plugin`. See [releases](https://github.com/Alice-Marx/dsh-model-router/releases) for archived builds, or build your own checkout below.
|
|
235
47
|
|
|
236
|
-
|
|
48
|
+
### Upgrade from the old package name
|
|
237
49
|
|
|
238
|
-
|
|
50
|
+
The old name `@ljwei-stak/model-router-galgame` ended at 0.13.0. Remove that plugin entry while retaining your profile/application data, then add `@ljwei-stak/dsh-model-router@0.16.0`. Do not keep both entries installed: they share the internal plugin id `model-router-galgame` and tool names. The retained id lets existing router settings and history carry over.
|
|
239
51
|
|
|
240
|
-
|
|
52
|
+
For the combined 0.11.x package, back up the profile and export the GAL saves you want to preserve before upgrading. Story playback, artwork, music and saves now belong to the independent `@ljwei-stak/dsh-galgame` package. The detailed save limitations are in [MIGRATION.md](MIGRATION.md).
|
|
241
53
|
|
|
242
|
-
|
|
54
|
+
## Use the workbench
|
|
243
55
|
|
|
244
|
-
|
|
56
|
+
The 0.15.0 workbench groups the workflow into four navigation tabs. Its header summarizes registered routes, detected tools and today's recorded API spending.
|
|
245
57
|
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
|
|
249
|
-
|
|
250
|
-
|
|
251
|
-
|
|
252
|
-
C = clip(0.10 + 0.30L + 0.18R + 0.28D
|
|
253
|
-
+ 0.22·I_code + 0.20·I_reasoning + 0.12·I_vision)
|
|
254
|
-
```
|
|
255
|
-
|
|
256
|
-
The initial bands are `simple` for `C < 0.34`, `balanced` for `0.34 ≤ C < 0.66`, and `complex` above that. Explicit simple requests such as a short translation, and high-stakes action requests such as designing a security-sensitive architecture, have rule-based overrides. These are text signals, not a model's hidden chain of thought.
|
|
257
|
-
|
|
258
|
-
### 2. Split compound work into a directed acyclic graph
|
|
58
|
+
| Tab | What you do there |
|
|
59
|
+
| --- | --- |
|
|
60
|
+
| **任务与执行** — Task and execution | Describe the task, select a planning mode, generate advice, preview a read-only run and inspect run history. |
|
|
61
|
+
| **模型配置** — Model configuration | Refresh/search the model directory and edit exact-route profiles. |
|
|
62
|
+
| **官方工具** — Official tools | Complete onboarding checks, inspect/install tools and use the interactive terminal. |
|
|
63
|
+
| **预算与安全** — Budget and security | Set routing/cost/review policies and inspect subscription status and execution boundaries. |
|
|
259
64
|
|
|
260
|
-
|
|
65
|
+
Switching tabs keeps the panels mounted, preserving in-progress installation and terminal state.
|
|
261
66
|
|
|
262
|
-
|
|
263
|
-
analysis ──┬── keyword extraction ────────────┐
|
|
264
|
-
├── architecture design ──┐ │
|
|
265
|
-
└── security review ──────┴── verification ── synthesis
|
|
266
|
-
```
|
|
67
|
+

|
|
267
68
|
|
|
268
|
-
|
|
69
|
+
*Screenshot from the local UI preview with fixture models, prices, and health data. No real model or Harness connection was used. Expand a work package to inspect its goal, dependencies, and acceptance checks.*
|
|
269
70
|
|
|
270
|
-
|
|
71
|
+
1. Configure providers, models and credentials on Harness's official **Models** page. In **模型配置**, refresh the directory. A listed route does not prove that its credentials, network or model entitlement work.
|
|
72
|
+
2. Optionally enter quality and prices for the routes you want compared. Prices are **USD per million tokens**; quality is a user-supplied **0–100** estimate. Leave unknown prices blank rather than entering zero.
|
|
73
|
+
3. In **任务与执行**, describe concrete deliverables, constraints and acceptance criteria. Select **单任务** (single recommendation), **团队分工** (dependent packages for sufficiently complex requests), or **指定模型** (one exact route, without comparing others). Team mode alone does not start any team.
|
|
74
|
+
4. Set the per-run estimate target; `0` removes that planning constraint. Click **生成路由建议**. The result appears directly below planning. Check the route, complexity, cost, execution channel and each package's dependencies/checklist.
|
|
75
|
+
5. For a real read-only run, use **在工作台执行**. Enter an existing absolute workspace path, or reuse the last run's path. Click **预览执行计划**, review the current route and all required confirmations, then explicitly confirm execution. The Host rechecks the plan before starting; changed conditions can require confirmation again.
|
|
76
|
+
6. Inspect **执行记录与子任务** for the actual channel, response, failures, cost and ratings. Editable CLI tasks start from an official Harness session using the tools below.
|
|
271
77
|
|
|
272
|
-
|
|
78
|
+
Changing task text, planning mode, profiles, directory or relevant settings invalidates the prior recommendation. Changing execution inputs also discards the pending or completed preview, so execution requires a new preview of the current request. Previewing execution uses current Host state and can differ from a previous local plan.
|
|
273
79
|
|
|
274
|
-
|
|
275
|
-
|
|
276
|
-
For a feasible candidate `m` and package `i`, the ranking combines normalized quality, price, latency, specialty match, reasoning-effort fit, and risk:
|
|
80
|
+
Example task:
|
|
277
81
|
|
|
278
82
|
```text
|
|
279
|
-
|
|
280
|
-
|
|
281
|
-
|
|
282
|
-
|
|
83
|
+
Review this project and prepare an implementation plan.
|
|
84
|
+
1. Map the entry points and identify the main UI workflows.
|
|
85
|
+
2. Review the interface at wide and narrow widths; list layout problems.
|
|
86
|
+
3. Check the execution and billing boundaries.
|
|
87
|
+
4. Combine the findings into a prioritized change list with verification steps.
|
|
283
88
|
```
|
|
284
89
|
|
|
285
|
-
The
|
|
286
|
-
|
|
287
|
-
| Package difficulty | Quality | Cost | Latency | Specialty | Reasoning | Risk penalty |
|
|
288
|
-
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
|
|
289
|
-
| Simple | 0.28 | 0.45 | 0.14 | 0.04 | 0.07 | 0.02 |
|
|
290
|
-
| Balanced | 0.40 | 0.26 | 0.10 | 0.09 | 0.10 | 0.05 |
|
|
291
|
-
| Complex | 0.48 | 0.14 | 0.06 | 0.14 | 0.11 | 0.07 |
|
|
292
|
-
| Synthesis | 0.58 | 0.08 | 0.04 | 0.08 | 0.16 | 0.06 |
|
|
90
|
+
The router plans and delegates text/code tasks. It does not itself render videos, generate artwork or control creative applications.
|
|
293
91
|
|
|
294
|
-
|
|
92
|
+
## Session tools
|
|
295
93
|
|
|
296
|
-
|
|
94
|
+
Ask an official Harness session to invoke these tools. Configure the session workspace before tasks that access local files.
|
|
297
95
|
|
|
298
|
-
|
|
96
|
+
| Tool | Purpose |
|
|
97
|
+
| --- | --- |
|
|
98
|
+
| `model_router_routes` | List configured routes; does not verify account/network availability. |
|
|
99
|
+
| `model_router_plan` | Generate a local `single` or `team` plan. |
|
|
100
|
+
| `model_router_consult` | Ask one configured model for a live second opinion via the model API. |
|
|
101
|
+
| `model_router_execute` | Execute a read-only routed plan, or one explicit `provider`/`model`, with recorded results. |
|
|
102
|
+
| `model_router_tools` / `model_router_health` | Inspect installation/readiness or version/login/billing health. |
|
|
103
|
+
| `model_router_tool_install` | Install one fixed registry tool from its official source. |
|
|
104
|
+
| `model_router_tool_run` | Run one supported CLI in `read-only` or approved `workspace-write` mode. |
|
|
105
|
+
| `model_router_team_execute` | Replan among currently executable CLI routes, then run work packages sequentially. |
|
|
106
|
+
| `model_router_rerun_step` | Rerun a failed step and its unfinished dependents, resume eligible editable teams, or repeat a failed single-tool run. |
|
|
107
|
+
| `model_router_rate` | Record `up`, `down` or `clear` for one stored result. |
|
|
299
108
|
|
|
300
|
-
|
|
109
|
+
Session commands are `/router <task>`, `/tools`, and `/tools install|cancel <tool-id>`.
|
|
301
110
|
|
|
302
|
-
|
|
111
|
+
A local planning request:
|
|
303
112
|
|
|
304
|
-
```
|
|
305
|
-
|
|
306
|
-
uncached_input_tokens × input_price
|
|
307
|
-
+ cache_read_tokens × cache_read_price
|
|
308
|
-
+ cache_write_tokens × cache_write_price
|
|
309
|
-
+ output_tokens × output_price
|
|
310
|
-
) / 1,000,000
|
|
311
|
-
|
|
312
|
-
estimated_saving = (all_strong_baseline − routed_estimate)
|
|
313
|
-
/ all_strong_baseline
|
|
113
|
+
```json
|
|
114
|
+
{"task":"Review the architecture and prepare a verification checklist","mode":"team","budgetUsd":10}
|
|
314
115
|
```
|
|
315
116
|
|
|
316
|
-
|
|
317
|
-
|
|
318
|
-
### Worked example: cheap extraction, strong design
|
|
117
|
+
A read-only single-tool request:
|
|
319
118
|
|
|
320
|
-
|
|
321
|
-
|
|
322
|
-
| Demo route | Quality | Input USD / 1M | Output USD / 1M | Specialty |
|
|
323
|
-
| --- | ---: | ---: | ---: | --- |
|
|
324
|
-
| `cheap-provider/Flash Custom` | 0.79 | $0.10 | $0.20 | Summarization |
|
|
325
|
-
| `strong-provider/Reasoning Custom` | 0.97 | $5.00 | $25.00 | Reasoning, code |
|
|
326
|
-
|
|
327
|
-
For the exact example input below, the complexity score is about **0.705** and the plan has six packages: analysis, three explicit execution packages, verification, and synthesis.
|
|
328
|
-
|
|
329
|
-
```text
|
|
330
|
-
请处理项目。
|
|
331
|
-
- 请提取关键词
|
|
332
|
-
- 请设计复杂系统架构
|
|
333
|
-
- 最后验证架构安全性
|
|
119
|
+
```json
|
|
120
|
+
{"tool":"codex","task":"Inspect the repository and list UI layout issues; do not edit files","mode":"read-only"}
|
|
334
121
|
```
|
|
335
122
|
|
|
336
|
-
|
|
123
|
+
Supply `provider` and `model` together when selecting an exact catalog route. A vendor's CLI model name may differ from its Harness id; use `cliModel` or team `cliModelsJson` only for names verified with that CLI. ZCode uses its own default and cannot switch model per call. Most CLIs do not report a verifiable actual model id; check the vendor run record.
|
|
337
124
|
|
|
338
|
-
|
|
125
|
+
The CLI team runner is the plugin's own sequential dependency executor. It does not control the model selection or lifecycle of Harness's built-in Agent Teams.
|
|
339
126
|
|
|
340
|
-
|
|
341
|
-
| --- | --- | ---: | ---: |
|
|
342
|
-
| Analysis | Reasoning | 80 / 495 | $0.012775 |
|
|
343
|
-
| Extract keywords | Flash | 96 / 900 | $0.000190 |
|
|
344
|
-
| Design architecture | Reasoning | 96 / 1,305 | $0.033105 |
|
|
345
|
-
| Verify security requirement | Reasoning | 96 / 1,305 | $0.033105 |
|
|
346
|
-
| Independent verification | Reasoning | 92 / 630 | $0.016210 |
|
|
347
|
-
| Synthesis | Reasoning | 132 / 1,170 | $0.029910 |
|
|
127
|
+
## Official tools and execution modes
|
|
348
128
|
|
|
349
|
-
|
|
129
|
+
| Registry id | Tool | Managed single-tool/team modes |
|
|
130
|
+
| --- | --- | --- |
|
|
131
|
+
| `kimi-code` | Kimi Code | Approved `workspace-write` |
|
|
132
|
+
| `claude-code` | Claude Code | `read-only`, approved `workspace-write` |
|
|
133
|
+
| `codex` | Codex CLI | `read-only`, approved `workspace-write` |
|
|
134
|
+
| `minimax-code` | MiniMax Code | Approved `workspace-write` |
|
|
135
|
+
| `mimo-code` | MiMo Code | `read-only`, approved `workspace-write` |
|
|
136
|
+
| `grok-build` | Grok Build | `read-only`, approved `workspace-write` |
|
|
137
|
+
| `gemini` | Gemini CLI | Routed headless adapter; not the managed single-tool/team runner |
|
|
138
|
+
| `zcode` | ZCode | Windows; approved `workspace-write` |
|
|
350
139
|
|
|
351
|
-
|
|
352
|
-
($0.148085 − $0.125295) / $0.148085 × 100 = 15.39% estimated saving
|
|
353
|
-
```
|
|
140
|
+
Installers use fixed official npm packages at each vendor's `@latest`, or the official publisher-signed Windows ZCode installer. An installation newer than the reported latest is not downgraded. Latest-version lookups are cached for about 12 hours; unavailable lookups leave the version unknown. New vendor releases are not automatically validated by this project. ZCode's installation window still requires you to choose a directory and finish installation.
|
|
354
141
|
|
|
355
|
-
|
|
142
|
+
Installation, trusted launch readiness, account login and model entitlement are separate checks. An unknown login status is not proof of logout. On Windows, supported managed launch entries are checked against publisher signatures or official npm-tarball attestations; ZCode also records trust for its bundled script per signed build. See [the registry](.dsh-plugin/shared/official-tool-registry.mjs) and [executor](.dsh-plugin/shared/official-tool-executor.mjs) for the exact platform rules.
|
|
356
143
|
|
|
357
|
-
|
|
144
|
+
`model_router_execute` has fixed headless adapters for Claude, Codex and Gemini. Other suppliers use the model-directory API unless a supported subscription route applies. Missing/unusable CLIs can fall back to that API; subscription failures follow the policy below.
|
|
358
145
|
|
|
359
|
-
|
|
146
|
+
Editable execution needs a clean Git repository, Host approval and the Harness process sandbox. Changes are made in an isolated worktree and applied back only after successful execution and a matching clean source workspace. Failed or conflicting integration is reported. Git-ignored outputs need separate review. Read-only CLI flags do not necessarily limit file reads to the workspace; check **安全边界**, especially Codex's broader readable scope.
|
|
360
147
|
|
|
361
|
-
|
|
362
|
-
[
|
|
363
|
-
{
|
|
364
|
-
"provider": "your-exact-provider-id",
|
|
365
|
-
"model": "your-exact-model-id",
|
|
366
|
-
"quality": 80,
|
|
367
|
-
"pricing": { "input": 0.5, "output": 2.0, "currency": "USD" },
|
|
368
|
-
"specialties": ["code", "summarization"],
|
|
369
|
-
"cliModel": "name-configured-in-that-vendor-cli",
|
|
370
|
-
"execution": "auto"
|
|
371
|
-
}
|
|
372
|
-
]
|
|
373
|
-
```
|
|
148
|
+
### Interactive terminal
|
|
374
149
|
|
|
375
|
-
|
|
150
|
+
In **官方工具 → 官方工具终端**, select the shell or an installed CLI, choose interactive/login mode where available, enter an existing absolute directory, and review the startup confirmation. ZCode is a desktop app and is not a terminal target.
|
|
376
151
|
|
|
377
|
-
|
|
152
|
+
These sessions run **outside the Harness sandbox**, with the Host user's login environment, proxies and API-key variables. Commands typed there can read/write files with that user's permissions. Terminal launches are separate from routed execution and its approval/billing policy.
|
|
378
153
|
|
|
379
|
-
|
|
154
|
+
The optional `@lydell/node-pty` dependency supplies a real pseudo terminal. If it cannot load, the card reports pipe mode: full-screen interfaces, resizing and arrow keys may not work. At most four sessions run concurrently, unread sessions expire after about two minutes, and each session lasts at most six hours. Explicitly end sessions when finished; closing the workbench or unloading the plugin also ends them.
|
|
380
155
|
|
|
381
|
-
|
|
156
|
+
The plugin does not persist terminal input/output. It stores session metadata only. With text selected, Ctrl+C copies; otherwise it interrupts. Ctrl+Shift+V pastes.
|
|
382
157
|
|
|
383
|
-
##
|
|
158
|
+
## Costs, subscriptions and quality
|
|
384
159
|
|
|
385
|
-
|
|
160
|
+
- **Budgets:** per-run planning target plus daily/monthly checks. Over-budget routing can try the economy preset or pause for confirmation. Unknown prices remain unknown. These are local accounting/estimation controls, not a hard provider billing cap.
|
|
161
|
+
- **Subscription-first:** the default is to prefer a detected CLI account subscription or a configured coding-plan route. API-key environment variables are removed for CLI subscription attempts. `billing` can be `subscription-first`, `api-only` or `subscription-only`; `subscription` can be `cli-login`, `plan-key` or `none`.
|
|
162
|
+
- **Fallback:** detected quota exhaustion/rate limits mark a subscription in cooldown and can retry the routed step on its configured API route, subject to policy and availability. Other attempted subscription failures pause by default (`onSubscriptionFailure: "ask"`): choose API retry, subscription retry or cancel in history. Managed editable/single-tool/team runs do not automatically retry on the API.
|
|
163
|
+
- **Accounting:** API costs count toward budgets. Subscription runs display an API-price reference cost separately. Actual usage is taken from CLI-reported amounts or token usage and supplied prices when available; provider bills remain authoritative. Version 0.15.0 retains spending from evicted detailed runs and replaced retry results, so trimming history or retrying a step does not erase earlier charges. Costs already discarded by older versions cannot be reconstructed.
|
|
164
|
+
- **Quality and preference:** optional `off`/`sample`/`always` stronger-model review can add paid calls; it is not treated as human feedback. Explicit ratings adjust separate subjective utility for the exact route and task type, using time decay and low-sample shrinkage (default limit ±0.04, configurable up to ±0.1). They do not change objective quality scores or their hard floors; learning can be disabled, ratings withdrawn, or its starting point reset.
|
|
386
165
|
|
|
387
|
-
|
|
166
|
+
Coding-plan keys and endpoints stay in Harness provider settings. A `plan-key` profile may name an exact configured `apiRoute` as its fallback; the profile editor does not accept credentials or arbitrary commands. Vendor plan eligibility and terms must be checked with the vendor.
|
|
388
167
|
|
|
389
|
-
|
|
390
|
-
| --- | --- |
|
|
391
|
-
| Codex, Claude Code (Windows) | Valid Authenticode signature by `OpenAI OpCo, LLC` / `Anthropic, PBC` (Claude also needs ≥ 2.1.259 for its restricted flags) |
|
|
392
|
-
| ZCode (Windows) | `ZCode.exe` signed by 北京智谱华章科技股份有限公司 (any version; version read from the signed file); `resources/glm/.node-bundle-meta.json` names `zcode.cjs`; the unsigned `zcode.cjs` is recorded on first use per signed build and refused if it later changes under the same build (`model-router/zcode-trust.json`) |
|
|
393
|
-
| ZCode installer | Official CDN URL from the official page, HTTPS, size cap, Authenticode publisher |
|
|
394
|
-
| MiMo, Grok, MiniMax | Unsigned binaries/scripts: every code file must equal the file in the official npm tarball for the **installed** version (tarball sha512 from registry.npmjs.org, cached in `model-router/npm-attestations.json`). The first run of a new version needs network once. |
|
|
168
|
+
## How routing works
|
|
395
169
|
|
|
396
|
-
|
|
170
|
+
The planner is a deterministic local heuristic, not a learned guarantee of task success. It:
|
|
397
171
|
|
|
398
|
-
|
|
399
|
-
|
|
400
|
-
|
|
401
|
-
|
|
402
|
-
| `model_router_execute` | Run a routed plan, or one explicit model, through each vendor's headless CLI when enabled. Missing, logged-out, or failed CLIs fall back to the model API (a hung CLI and the processes it started get SIGTERM, then SIGKILL after 5 s; a cancel waits at most 1.5 s; Windows uses `taskkill /T /F`). The task may be up to 64,000 UTF-8 bytes (about 21,000 Chinese or 64,000 English characters) and is checked before any paid call; long tasks and dependency answers are truncated per step so each step's prompt fits. A run that stops on an error after paid steps is still recorded with their cost. Records cost and supports per-step rerun and rating. Read-only. |
|
|
403
|
-
| `model_router_health` | Onboarding health check: installed, version, login state per official tool. Logged-out tools are skipped by routing. |
|
|
404
|
-
| `model_router_rerun_step` / `model_router_rate` | Rerun (optionally reassign) one failed step of a recorded run, continue an editable team run in a fresh seeded worktree, or repeat a failed tool run; rate a result 👍/👎 to nudge future routing. |
|
|
405
|
-
| `model_router_tools` / `model_router_tool_install` | Probe or install a fixed official tool. `/tools` exposes the human command. |
|
|
406
|
-
| `model_router_tool_run` | Run one ready vendor CLI in the approved session workspace. Estimated and budget-checked first (`confirmOverBudget`). |
|
|
407
|
-
| `model_router_team_execute` | Run dependent work packages through ready vendor CLIs in order; stop on failure or a reported model mismatch. |
|
|
172
|
+
1. Estimates task type/complexity from the text and splits suitable compound requests into analysis, execution, verification and synthesis packages.
|
|
173
|
+
2. Uses only configured routes, overlays exact-route profiles, and filters modality and quality constraints where possible.
|
|
174
|
+
3. Compares quality, estimated price, latency, specialties, reasoning fit and risk using the selected preset.
|
|
175
|
+
4. Prunes dominated candidates and searches a bounded set of dependent assignments. Constraint relaxation, missing evidence and unassignable packages are reported.
|
|
408
176
|
|
|
409
|
-
|
|
177
|
+
Estimated cost is token usage × supplied USD-per-million prices, including optional cache prices. Quality can come from user profiles, supplied benchmark data or catalog heuristics; catalog hints are not measured success probabilities. A vision route still needs actual input support, and current routed execution is text-oriented rather than a general multimodal asset pipeline.
|
|
410
178
|
|
|
411
|
-
|
|
179
|
+
The implementations are [router.mjs](.dsh-plugin/shared/router.mjs), [harness-plan.mjs](.dsh-plugin/shared/harness-plan.mjs) and [routing-presets.mjs](.dsh-plugin/shared/routing-presets.mjs).
|
|
412
180
|
|
|
413
|
-
|
|
181
|
+
## Data and privacy
|
|
414
182
|
|
|
415
|
-
|
|
183
|
+
Router profiles/preferences are held in the Harness plugin settings. Runtime state is stored in `<DSH_HOME>/model-router/state.json` (default `~/.dsh/model-router/state.json`): health/onboarding, subscription cooldown, ratings, up to 200 recent runs, 50 terminal metadata records and opt-in public data snapshots. Applied rates and data/feedback policy versions are frozen in run records. Preferences are local to that shared DSH_HOME, not authenticated per-user accounts; public data requests do not upload tasks or feedback.
|
|
416
184
|
|
|
417
|
-
|
|
185
|
+
Version 0.15.0 also retains bounded spending aggregates for up to 90 populated calendar dates and 24 months. These contain amounts/counts, not task or response content.
|
|
418
186
|
|
|
419
|
-
|
|
420
|
-
- **Fixed launch targets.** The client can only name the shell or a registry CLI (plus its fixed login subcommand); it never supplies an executable, arguments, or environment. On Windows, native `.exe` files start directly, npm `.cmd` shims through `cmd.exe /d /s /c`, `.ps1`-only installs through `powershell -File`.
|
|
421
|
-
- **Backend.** A pseudo terminal from the prebuilt, N-API [`@lydell/node-pty`](https://www.npmjs.com/package/@lydell/node-pty) (an optional dependency with per-platform binaries; no build tools or install scripts). It loads in the Harness Desktop Host (Electron 44 running as Node). If it cannot load, sessions fall back to pipes: no TTY, so full-screen TUIs may not render or start, resizing does nothing, arrow keys may not work, and the Host echoes input. Line-based logins and PowerShell work. The card shows which backend is active.
|
|
422
|
-
- **Transport.** Output streams through long-poll reads over the plugin's existing Typert remote (`terminalRead`); keystrokes (`terminalWrite`) and resizes (`terminalResize`) are separate calls.
|
|
423
|
-
- **Lifetime.** Ending the tab, closing the workbench panel, unloading or reloading the plugin, or quitting Harness kills the process. A session that nobody reads for about 2 minutes is killed, and no session runs longer than 6 hours. At most 4 sessions run at once.
|
|
424
|
-
- **Privacy.** Input and output are never logged or saved. The card's history (and `state.json` → `terminalSessions`) stores only the tool, mode, working directory, start/end time, duration, exit code, and end reason.
|
|
425
|
-
- **Restart after installing.** Harness serves the new card immediately, but a running Host keeps the old plugin code; until you quit Harness completely (including the tray icon) and start it again, the card shows “插件后台版本较旧,请完全退出并重启 Harness”.
|
|
426
|
-
- **Keys.** With text selected, Ctrl+C copies; otherwise it interrupts. Ctrl+Shift+V pastes. Drag the terminal's lower-right corner to change its height.
|
|
187
|
+
Run history includes task text, output excerpts, route/workspace, failures and costs. Stored task text is capped at 20,000 characters and each answer at 4,000; reruns can therefore depend on truncated stored context. The plugin does not store API keys in model profiles or terminal transcripts. Task execution still sends the selected task/context to the chosen provider or CLI.
|
|
427
188
|
|
|
428
|
-
|
|
189
|
+
State writes use a lock and atomic replacement. Unparseable state is preserved as a `state.json.corrupt-<timestamp>` backup and surfaced as a notice. Back up state/profile data before migrations and avoid sharing unredacted histories or logs.
|
|
429
190
|
|
|
430
|
-
|
|
191
|
+
## Develop from a cloned checkout
|
|
431
192
|
|
|
432
|
-
|
|
193
|
+
The Git clone is a complete source checkout; the npm package is a runtime distribution and excludes client source, scripts, tests and the experiment project.
|
|
433
194
|
|
|
434
|
-
|
|
195
|
+
Requirements: **Node.js 22.19+** and **pnpm 10.34.6** (pinned in `packageManager`). In PowerShell, from the directory containing your clone:
|
|
435
196
|
|
|
436
197
|
```powershell
|
|
198
|
+
cd .\dsh-model-router
|
|
437
199
|
pnpm install --frozen-lockfile --strict-peer-dependencies
|
|
438
200
|
npm run build:client
|
|
439
201
|
npm test
|
|
440
202
|
npm run check:client
|
|
441
|
-
npm
|
|
203
|
+
npm run pack:local
|
|
442
204
|
```
|
|
443
205
|
|
|
444
|
-
|
|
206
|
+
If already in the repository, omit the first `cd`. The archive appears in `dist`; install that absolute `.tgz` path through Desktop's plugin manager in a test profile. Do not treat successful unit tests as proof of vendor account access, actual model identity or billing.
|
|
445
207
|
|
|
446
|
-
|
|
208
|
+
To inspect the UI without starting Harness or vendor tools:
|
|
447
209
|
|
|
448
|
-
|
|
210
|
+
```powershell
|
|
211
|
+
npm run preview:ui
|
|
212
|
+
```
|
|
213
|
+
|
|
214
|
+
Open [the local preview](http://127.0.0.1:4173). It uses mock models, tools and history, with populated, empty, error, onboarding and delayed-preview scenarios for layout/interaction checks. Simulated execution does not call providers, install tools or start a terminal. This is not Host integration verification. Use `npm run preview:ui -- --port 4174` if the default port is occupied; refreshing picks up client-source edits.
|
|
215
|
+
|
|
216
|
+
| Location | Role |
|
|
217
|
+
| --- | --- |
|
|
218
|
+
| `.dsh-plugin/index.mjs` | Host plugin, session tools, settings, execution and remote handlers. |
|
|
219
|
+
| `.dsh-plugin/client/` | React workbench source and CSS. |
|
|
220
|
+
| `.dsh-plugin/shared/` | Planner, tool registry/runtime, billing, ledger, state and terminal contracts. |
|
|
221
|
+
| `.dsh-plugin/client.js` | Generated Desktop client bundle; rebuild after client changes. |
|
|
222
|
+
| `scripts/` and `tests/` | Build/package/verification helpers and regression tests. |
|
|
223
|
+
| `experiment-plugin/` | Separate research experiment project; not shipped as part of the router plugin. |
|
|
224
|
+
| `docs/` and dated reports | User documentation and historical verification records. |
|
|
225
|
+
|
|
226
|
+
`check:client` rebuilds in memory and compares the generated output with the committed bundle; `prepack` refuses a stale bundle. pnpm 10 has no `pnpm peers check` command: use the strict install above. Unavailable/incompatible host SDK packages may require the installed Harness runtime for local integration testing; [setup-harness-dev.mjs](scripts/setup-harness-dev.mjs) links missing SDK packages from an explicitly supplied Host `node_modules`, and [start-harness-preview.mjs](scripts/start-harness-preview.mjs) starts a separate local web Host profile. These helpers do not replace dependency verification or Desktop acceptance checks.
|
|
227
|
+
|
|
228
|
+
Legacy compatibility/research files and dated reports are preserved for reference; their presence does not make every older integration an active feature of the current packaged plugin. Releases and synchronization follow [AGENTS.md](AGENTS.md); use versioned packages or verified release tags to reproduce a release.
|
|
229
|
+
|
|
230
|
+
## Troubleshooting
|
|
231
|
+
|
|
232
|
+
| Symptom | Check |
|
|
233
|
+
| --- | --- |
|
|
234
|
+
| No routes or planning disabled | Configure Harness Models, refresh the directory and wait for tool detection. |
|
|
235
|
+
| Cost unknown | Supply both input/output USD prices for each compared route. |
|
|
236
|
+
| CLI installed but not executable | Review launch-readiness reason, platform/mode support and installation source. |
|
|
237
|
+
| Login/entitlement still fails | Check the vendor account, CLI login, network and actual model access; installation alone is insufficient. |
|
|
238
|
+
| Frontend says Host is older | Fully quit Harness, including the tray process, then restart. |
|
|
239
|
+
| Preview differs from old advice | Directory, login, quota or budget state changed; review the new preview. |
|
|
240
|
+
| Terminal TUI does not render | Check whether the card reports PTY or pipe fallback. |
|
|
241
|
+
| Editable team stopped | Read the failed package and integration state before rerunning or manually reviewing its worktree. |
|
|
242
|
+
|
|
243
|
+
For issues include host/plugin versions, selected mode, tool status and redacted errors: [issue tracker](https://github.com/Alice-Marx/dsh-model-router/issues).
|
|
449
244
|
|
|
450
245
|
License: [MIT](LICENSE).
|