dsh-llm-fallbacks 0.2.1 → 0.2.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +17 -2
- package/README.zh-CN.md +17 -2
- package/dist/automatch.d.ts +98 -0
- package/dist/client/FallbacksCard.d.ts +4 -3
- package/dist/client/fallbacks-store.d.ts +12 -6
- package/dist/client/index.d.ts +9 -7
- package/dist/client/index.js +138 -108
- package/dist/client/locales.d.ts +24 -11
- package/dist/commands.d.ts +4 -0
- package/dist/config.d.ts +13 -0
- package/dist/events.d.ts +19 -5
- package/dist/index.d.ts +1 -1
- package/dist/index.js +157 -32
- package/dist/role-resolution.d.ts +81 -0
- package/dist/roles.d.ts +8 -0
- package/package.json +20 -19
- package/scripts/repair-fallbacks-switch-logs.ts +374 -0
package/README.md
CHANGED
|
@@ -24,6 +24,20 @@ dsh plugin --profile dsh-tui add dsh-llm-fallbacks # dsh-tui terminal profile
|
|
|
24
24
|
|
|
25
25
|
Same plugin, either front end — the only difference is the `--profile` flag. Pin a version with `@<version>`. A registry install fetches the **built package** (`dist/`), nothing builds on the target machine. Registry / git / local-directory variants, uninstall, and `--dump-config` verification → [docs/install.md](docs/install.md).
|
|
26
26
|
|
|
27
|
+
### Repair existing sessions (versions before 0.2.2)
|
|
28
|
+
|
|
29
|
+
Versions before 0.2.2 wrote durable `fallbacks/switch` session events that newer dsh releases refuse to load (issue #52 — the apply()-time event-type registration is ineffective because plugin and host resolve different module instances). If existing sessions fail to open after an upgrade, clone this repository and repair the logs (stop dsh first):
|
|
30
|
+
|
|
31
|
+
```sh
|
|
32
|
+
git clone https://github.com/omdsh-dev/dsh-llm-fallbacks.git
|
|
33
|
+
cd dsh-llm-fallbacks
|
|
34
|
+
pnpm install
|
|
35
|
+
pnpm repair:fallbacks-switch-logs -- --dry-run # preview which sessions would change
|
|
36
|
+
pnpm repair:fallbacks-switch-logs -- --apply --backup # mark legacy events ignorable
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
The script scans `~/.dsh/sessions` by default (override with `--root <dir>`), marks legacy `fallbacks/switch` events `ignorable: true` so the host read path accepts the session again, and keeps a `<file>.bak` per repaired log. `--apply` requires `--backup` and must run with dsh stopped. From 0.2.2 on, the plugin stops writing durable switch events, so no new sessions need repair.
|
|
40
|
+
|
|
27
41
|
### Minimal configuration
|
|
28
42
|
|
|
29
43
|
Add a `fallbacks:` section to the dsh settings document (default `$DSH_HOME/settings.yaml`):
|
|
@@ -52,14 +66,15 @@ No rule match → the built-in `inherit` → `rootChain`. `enabled` defaults to
|
|
|
52
66
|
|
|
53
67
|
### Verify
|
|
54
68
|
|
|
55
|
-
Save and restart the session, then type `/fallbacks` — the read-only in-session diagnostics (origin, resolved role, chain, recent `fallbacks/switch` events, cooldown status).
|
|
69
|
+
Save and restart the session, then type `/fallbacks` — the read-only in-session diagnostics (origin, resolved role, chain, recent `fallbacks/switch` events, cooldown status). The plugin no longer writes durable `fallbacks/switch` session events (issue #52 — the apply()-time registration was proven ineffective), so new switches show up in the info logs, not in the recent-switch surfaces; sessions written by older plugin versions that contain `fallbacks/switch` events are repaired with `scripts/repair-fallbacks-switch-logs.ts`, which marks legacy events ignorable so those sessions load again (see the Features note below). In a dsh-tui profile, `/fallbacks config` additionally reads back the composed configuration (the TUI has no settings page — config is file-only; see [docs/configuration.md](docs/configuration.md)).
|
|
56
70
|
|
|
57
71
|
## Features
|
|
58
72
|
|
|
59
73
|
- **Automatic fallback for root and subagents**: any agent switches down the chain to the next available provider/model on model failure — no manual model switching.
|
|
60
74
|
- **Two-block config**: `rootChain` for the root agent; declared role entities (`roles.list`) referenced by `roles.rules` (or the built-in `inherit`).
|
|
75
|
+
- **Dispatch-time role resolution**: on a subagent's first request its role is resolved in three stages — explicit (`agentPreset` matches a declared role id) → deterministic rules (unchanged) → LLM auto-match from the declared role taxonomy (`fallbacks.roleAutoMatch`, default `true`). The resolved role's chain-head model is injected into the first request and recorded via an explicit `role → model` log line (no durable `fallbacks/switch` event is written — issue #52 stop-write); set `roleAutoMatch: false` to disable the LLM auto-match stage (the explicit `agentPreset` stage still applies — with no explicit role this reproduces the previous rules-only behavior). The settings card always renders an **Enable role auto-match** switch (default `true`) to toggle it — the schema default applies even to legacy configs that never declared the key.
|
|
61
76
|
- **Cooldown and revert**: failed / switched-away models are not re-selected during cooldown; `revertPolicy: cooldown-expiry` returns to the primary model automatically.
|
|
62
|
-
- **Visible behavior**: every switch
|
|
77
|
+
- **Visible behavior**: every switch is recorded in an info-level log line (from/to/role/reason) — no silent model switching. The plugin deliberately writes **no** durable `fallbacks/switch` session events (issue #52: the apply()-time event-type registration was proven ineffective, and a session containing the event refused to load after a dsh restart). Sessions written by older plugin versions that contain such events are repaired by `scripts/repair-fallbacks-switch-logs.ts`, which marks legacy events ignorable so affected sessions load again.
|
|
63
78
|
- **Safety valves**: `maxSwitchesPerStep` caps switches per step and `alwaysModeRetryCap` caps always-mode retries — chain loops cannot amplify latency.
|
|
64
79
|
- **No-config no-op**: `enabled` defaults to off; with no chains configured the plugin behaves exactly like not being installed.
|
|
65
80
|
|
package/README.zh-CN.md
CHANGED
|
@@ -24,6 +24,20 @@ dsh plugin --profile dsh-tui add dsh-llm-fallbacks # dsh-tui 终端 profile
|
|
|
24
24
|
|
|
25
25
|
同一个插件、两个前端——区别只在 `--profile` 参数。钉版本:加 `@<version>`。registry 安装拉取的是**已构建产物**(`dist/`),目标机无需构建。registry / git / 本地目录变体、卸载与 `--dump-config` 验证 → [docs/install.md](docs/install.md)。
|
|
26
26
|
|
|
27
|
+
### 修复旧会话(0.2.2 之前的版本)
|
|
28
|
+
|
|
29
|
+
0.2.2 之前的版本会把 `fallbacks/switch` 事件写入会话持久化日志,而新版 dsh 拒绝加载这类会话(issue #52——apply() 时的注册因插件与宿主解析到不同模块实例而无效)。如果升级后已有会话打不开,clone 本仓库并修复日志(先停 dsh):
|
|
30
|
+
|
|
31
|
+
```sh
|
|
32
|
+
git clone https://github.com/omdsh-dev/dsh-llm-fallbacks.git
|
|
33
|
+
cd dsh-llm-fallbacks
|
|
34
|
+
pnpm install
|
|
35
|
+
pnpm repair:fallbacks-switch-logs -- --dry-run # 预览哪些会话会被改动
|
|
36
|
+
pnpm repair:fallbacks-switch-logs -- --apply --backup # 给旧事件打 ignorable 标记
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
脚本默认扫描 `~/.dsh/sessions`(可用 `--root <dir>` 覆盖),把遗留 `fallbacks/switch` 事件标记为 `ignorable: true`,宿主读路径即可重新接受该会话;每个被修复的日志保留一份 `<file>.bak`。`--apply` 必须搭配 `--backup`,且须在 dsh 停止时运行。从 0.2.2 起插件不再写 durable 切换事件,新会话无需修复。
|
|
40
|
+
|
|
27
41
|
### 最小配置
|
|
28
42
|
|
|
29
43
|
在 dsh 的设置文档(默认 `$DSH_HOME/settings.yaml`)中添加 `fallbacks:` 分节:
|
|
@@ -52,14 +66,15 @@ fallbacks:
|
|
|
52
66
|
|
|
53
67
|
### 验证
|
|
54
68
|
|
|
55
|
-
保存并重启会话后,键入 `/fallbacks`——只读的会话内诊断(来源、解析角色、链、最近的 `fallbacks/switch`
|
|
69
|
+
保存并重启会话后,键入 `/fallbacks`——只读的会话内诊断(来源、解析角色、链、最近的 `fallbacks/switch` 事件、冷却状态)。插件**不再写入** durable `fallbacks/switch` 会话事件(issue #52——apply() 时的注册被证伪无效),因此新切换只出现在 info 日志中,不再出现在 recent-switch 展示面;由旧版插件写入、含 `fallbacks/switch` 事件的会话,可用 `scripts/repair-fallbacks-switch-logs.ts` 修复——脚本把旧事件标记为 ignorable,会话即可重新加载(见下方「能力一览」说明)。在 dsh-tui profile 中,`/fallbacks config` 额外回读组合配置(TUI 无设置页——配置仅文件,见 [docs/configuration.md](docs/configuration.md))。
|
|
56
70
|
|
|
57
71
|
## 能力一览
|
|
58
72
|
|
|
59
73
|
- **root / subagent 自动降级**:任意 agent 在模型故障下按链切换到下一个可用 provider/model,无需手动换模型。
|
|
60
74
|
- **两块制配置**:`rootChain` 管 root 代理;声明式角色实体(`roles.list`)供 `roles.rules` 引用(或内置 `inherit`)。
|
|
75
|
+
- **派发时角色解析**:在 subagent 的首次请求上,其角色按三个阶段解析——显式(`agentPreset` 匹配已声明角色 id)→ 确定性规则(不变)→ LLM 自动匹配(从已声明角色体系中选择,`fallbacks.roleAutoMatch` 默认 `true`)。解析出的角色的链头模型注入首次请求,并以显式 `role → model` 日志行记录(不写 durable `fallbacks/switch` 事件——issue #52 停写);设 `roleAutoMatch: false` 仅关闭 LLM 自动匹配阶段(显式 `agentPreset` 阶段仍生效——无显式角色时即复现原有仅规则行为)。设置卡总是渲染「启用角色自动匹配」开关(默认 `true`)以切换之——即使是从未声明过该键的旧配置,schema 默认值同样生效。
|
|
61
76
|
- **冷却与回主**:被切离/失败的模型在冷却期内不再入选;`revertPolicy: cooldown-expiry` 冷却到期后自动回主模型。
|
|
62
|
-
-
|
|
77
|
+
- **行为可见**:每次切换以 info 级日志行(from/to/role/reason)记录——无静默换模型。插件**刻意不写** durable `fallbacks/switch` 会话事件(issue #52——apply() 时的事件类型注册被证伪无效,含该事件的会话在 dsh 重启后拒绝加载)。由旧版插件写入、含此类事件的会话由 `scripts/repair-fallbacks-switch-logs.ts` 修复——旧事件被标记 ignorable 后,受影响会话可重新加载。
|
|
63
78
|
- **安全阀**:`maxSwitchesPerStep` 限制每 step 切换次数、`alwaysModeRetryCap` 限制 always 模式重试——链循环不会放大延迟。
|
|
64
79
|
- **无配置回归(no-op)**:`enabled` 默认关闭;未配置任何链时行为与未安装插件完全一致。
|
|
65
80
|
|
|
@@ -0,0 +1,98 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* LLM auto-match role selection (plan fallbacks-role-automatch Task 3).
|
|
3
|
+
*
|
|
4
|
+
* This module is the bounded LLM caller behind the resolver's stage-3
|
|
5
|
+
* auto-match hook (`resolveRoleAtDispatch`, `role-resolution.ts`). When the
|
|
6
|
+
* explicit and rules stages resolve to `'inherit'` and `roleAutoMatch` is
|
|
7
|
+
* enabled, `pickRoleByLlm` asks the model to pick the best-fit declared role
|
|
8
|
+
* from the taxonomy and returns its declared RAW id, or `null` (= `'inherit'`).
|
|
9
|
+
*
|
|
10
|
+
* Contract (architect-locked, plan Global Constraints · Auto-match bounded):
|
|
11
|
+
* - ONE bounded completion per decision: small `maxTokens`
|
|
12
|
+
* ({@link AUTOMATCH_MAX_TOKENS}) and a timeout
|
|
13
|
+
* ({@link AUTOMATCH_TIMEOUT_MS}, default 5s) that abandons the stream via
|
|
14
|
+
* the streaming contract's cancellation (`options.signal`) — no dangling
|
|
15
|
+
* iterator, no unhandled rejection.
|
|
16
|
+
* - Never throws. Every failure mode — throw, timeout, `none`, no valid id,
|
|
17
|
+
* garbage answer, malformed inputs — resolves to `null`, so the "never
|
|
18
|
+
* throws out of `agent/request`" invariant holds end-to-end (the resolver
|
|
19
|
+
* deliberately does NOT catch the hook).
|
|
20
|
+
* - Fast paths that avoid an LLM call entirely: empty taxonomy, absent
|
|
21
|
+
* `ctx.llm` service, and no concrete provider/model route for the judgment
|
|
22
|
+
* call. The plugin does NOT listen on `llm/stream`, so this call cannot
|
|
23
|
+
* recurse into the plugin.
|
|
24
|
+
*
|
|
25
|
+
* Judgment-call route choice (documented): the concrete provider/model for the
|
|
26
|
+
* judgment call is taken from the roles' DECLARED context first — the first
|
|
27
|
+
* exact (non-wildcard) `provider/model` selector across the declared chains,
|
|
28
|
+
* in declaration order (the operator's own taxonomy is the most stable route
|
|
29
|
+
* set). If no declared chain yields an exact selector, the agent's current
|
|
30
|
+
* `options.provider`/`options.model` is used when both halves are present;
|
|
31
|
+
* otherwise the call is skipped (`null`).
|
|
32
|
+
*
|
|
33
|
+
* Pure helpers exported for unit testing:
|
|
34
|
+
* - {@link buildAutomatchPrompt} — the taxonomy prompt (declared ids +
|
|
35
|
+
* personas + agent origin/agentPreset; instructs exactly-one-declared-id-or-
|
|
36
|
+
* `none`).
|
|
37
|
+
* - {@link parseAutomatchAnswer} — parses a declared id out of the model
|
|
38
|
+
* answer with surrounding whitespace/punctuation/quote/backtick tolerance;
|
|
39
|
+
* `none` (case-insensitive) and unknown ids → `null`. `roleIds` is the
|
|
40
|
+
* trimmed-id → declared-raw-id map, the same canonicalization the resolver
|
|
41
|
+
* uses.
|
|
42
|
+
*
|
|
43
|
+
* @module dsh-llm-fallbacks/automatch
|
|
44
|
+
*/
|
|
45
|
+
import { type GenerateOptions, type StreamChunk } from '@deepseek-ai/dsh-llm';
|
|
46
|
+
import type { FallbacksRole, FallbacksRoles } from './config.ts';
|
|
47
|
+
import type { AgentLike } from './roles.ts';
|
|
48
|
+
/** Default auto-match completion timeout in ms — abandons the stream. */
|
|
49
|
+
export declare const AUTOMATCH_TIMEOUT_MS = 5000;
|
|
50
|
+
/** Output bound (tokens) for the judgment call: one id or the literal `none`. */
|
|
51
|
+
export declare const AUTOMATCH_MAX_TOKENS = 32;
|
|
52
|
+
/**
|
|
53
|
+
* Minimal `llm` service surface {@link pickRoleByLlm} reads. The real
|
|
54
|
+
* `LlmRuntime.stream(options)` matches structurally; this face keeps the
|
|
55
|
+
* module free of the full cordis Context merge.
|
|
56
|
+
*/
|
|
57
|
+
export interface AutomatchLlm {
|
|
58
|
+
stream(options: GenerateOptions): AsyncIterable<StreamChunk>;
|
|
59
|
+
}
|
|
60
|
+
/**
|
|
61
|
+
* Minimal context surface: `ctx.get('llm')` (optional service — absent →
|
|
62
|
+
* `null` fast-path). The broad `get(name: string)` face is satisfied by the
|
|
63
|
+
* real cordis `Context` (`ReflectService.get` overloads return `any`/the
|
|
64
|
+
* merged service), so the Task-4 wiring can pass `ctx` directly.
|
|
65
|
+
*/
|
|
66
|
+
export interface AutomatchContext {
|
|
67
|
+
get(name: string): AutomatchLlm | undefined;
|
|
68
|
+
}
|
|
69
|
+
/** Options for {@link pickRoleByLlm}. */
|
|
70
|
+
export interface PickRoleByLlmOptions {
|
|
71
|
+
/** Completion timeout in ms; defaults to {@link AUTOMATCH_TIMEOUT_MS}. */
|
|
72
|
+
timeoutMs?: number;
|
|
73
|
+
/** Warning sink — the decision path injects the plugin logger. */
|
|
74
|
+
warn: (message: string) => void;
|
|
75
|
+
}
|
|
76
|
+
/**
|
|
77
|
+
* Build the auto-match taxonomy prompt: every declared role's trimmed id and
|
|
78
|
+
* persona, the agent's origin (and non-empty `agentPreset` when present), and
|
|
79
|
+
* the exactly-one-declared-id-or-`none` instruction. Pure — no I/O.
|
|
80
|
+
*/
|
|
81
|
+
export declare function buildAutomatchPrompt(roles: readonly FallbacksRole[], agent: AgentLike): string;
|
|
82
|
+
/**
|
|
83
|
+
* Parse the model's answer into a declared role id.
|
|
84
|
+
*
|
|
85
|
+
* The answer is expected to BE an id (possibly wrapped in whitespace, quotes,
|
|
86
|
+
* backticks, or trailing punctuation); surrounding characters outside the id
|
|
87
|
+
* grammar (`[a-zA-Z0-9-]`) are stripped, then the remaining string must match
|
|
88
|
+
* a declared id exactly (case-insensitive) via `roleIds` (trimmed id →
|
|
89
|
+
* declared RAW id, the resolver's canonicalization). `none` (case-insensitive)
|
|
90
|
+
* and any unknown/malformed answer → `null`.
|
|
91
|
+
*/
|
|
92
|
+
export declare function parseAutomatchAnswer(answer: string, roleIds: ReadonlyMap<string, string>): string | null;
|
|
93
|
+
/**
|
|
94
|
+
* Bounded LLM role-selection call (see module header). Resolves the declared
|
|
95
|
+
* RAW id of the best-fit role, or `null` on any fast path or failure. NEVER
|
|
96
|
+
* throws.
|
|
97
|
+
*/
|
|
98
|
+
export declare function pickRoleByLlm(ctx: AutomatchContext, roles: FallbacksRoles, agent: AgentLike, opts: PickRoleByLlmOptions): Promise<string | null>;
|
|
@@ -1,8 +1,9 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* Fallbacks settings card — the `fallbacks` plugin card on the web settings
|
|
3
|
-
* "插件配置" page (spec §4). Registered into the `settings.plugin.item`
|
|
4
|
-
* (
|
|
5
|
-
* cards and the advisor card
|
|
3
|
+
* "插件配置" page (spec §4). Registered into the `settings.plugin.item` keyed
|
|
4
|
+
* slot (key `fallbacks`, the settings namespace the card edits, alongside
|
|
5
|
+
* the upstream bash/agent-loop/web-search cards and the advisor card, in
|
|
6
|
+
* registration order); owner props are empty and all data flows
|
|
6
7
|
* through {@link FallbacksSettingsController}.
|
|
7
8
|
*
|
|
8
9
|
* The card chrome replicates the upstream `PluginCard` contract (self-drawn:
|
|
@@ -180,12 +180,18 @@ export declare function classifyModel(provider: string, raw: string, catalog: Ca
|
|
|
180
180
|
*/
|
|
181
181
|
export declare function extractRecentSwitches(entries: readonly HistoryEntry[], limit?: number): FallbacksSwitchSnapshot[];
|
|
182
182
|
/**
|
|
183
|
-
* Derive the
|
|
184
|
-
*
|
|
185
|
-
*
|
|
186
|
-
*
|
|
187
|
-
*
|
|
188
|
-
*
|
|
183
|
+
* Derive the "current effective model" (spec §2.5 D-6): ① disabled / empty
|
|
184
|
+
* rootChain → unavailable; ② a recent switch exists → the latest one's `to`;
|
|
185
|
+
* ③ otherwise → the config's primary target. A **display value** — never a
|
|
186
|
+
* live route probe.
|
|
187
|
+
*
|
|
188
|
+
* INTENTIONAL D-6 CONTRACT RETENTION: after the AC-2 trim (plan
|
|
189
|
+
* fallbacks-settings-visibility Task 2) the settings card's status block no
|
|
190
|
+
* longer consumes this derivation, and no other production code imports it —
|
|
191
|
+
* it is retained (NOT dead code to delete) as the spec §2.5 D-6 derived-value
|
|
192
|
+
* surface, pinned by `tests/fallbacks-store.spec.ts` (D-6 display-value
|
|
193
|
+
* contract). Keep both exports until the spec derivation is removed or gains
|
|
194
|
+
* a real consumer.
|
|
189
195
|
*/
|
|
190
196
|
export declare function deriveEffectiveModel(config: FallbacksConfig, switches: readonly FallbacksSwitchSnapshot[]): EffectiveModelView;
|
|
191
197
|
/** One chain selector row: provider + model (or wildcard). */
|
package/dist/client/index.d.ts
CHANGED
|
@@ -1,8 +1,9 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* dsh-llm-fallbacks client half: registers the Fallbacks card into the
|
|
3
|
-
* plugin-config page's `settings.plugin.item` slot (the official
|
|
4
|
-
* settings page —
|
|
5
|
-
* agent-loop / web-search cards
|
|
3
|
+
* plugin-config page's `settings.plugin.item` keyed slot (the official
|
|
4
|
+
* "插件配置" settings page — key `fallbacks`, the settings namespace the card
|
|
5
|
+
* edits, appearing after the upstream bash / agent-loop / web-search cards
|
|
6
|
+
* and the advisor card in registration order).
|
|
6
7
|
*
|
|
7
8
|
* Wiring (mirrors dsh-advisor):
|
|
8
9
|
* - Registers the `fallbacks` locale dictionaries (zh/en).
|
|
@@ -11,10 +12,11 @@
|
|
|
11
12
|
* `/api/fallbacks/get|set|reset`), while `settings.describe` (writable +
|
|
12
13
|
* namespace directory) and the provider/model catalog stay on
|
|
13
14
|
* `connection.api` (see `fallbacks-store.ts`).
|
|
14
|
-
* - Registers the `settings.plugin.item` card `
|
|
15
|
-
* with a business-only inject face
|
|
16
|
-
* the snapshot-selector hook); the
|
|
17
|
-
* is removed — deleting the section
|
|
15
|
+
* - Registers the `settings.plugin.item` card `key: 'fallbacks'` (the rc.7
|
|
16
|
+
* keyed slot — no `id`/`order`) with a business-only inject face
|
|
17
|
+
* ({@link FallbacksSettingsController} + the snapshot-selector hook); the
|
|
18
|
+
* old Settings-nav section registration is removed — deleting the section
|
|
19
|
+
* registration deletes the nav entry.
|
|
18
20
|
* - Refreshes the store on pushed invalidations — the forwarded remote
|
|
19
21
|
* events `settings/document-updated` (ns-filtered to the fallbacks
|
|
20
22
|
* namespace; refetches the descriptor + recent-switch summary) and
|