dsh-retry-boost 1.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.en.md +196 -0
- package/README.md +175 -0
- package/cordis.patch.yml +30 -0
- package/lib/index.js +209 -0
- package/lib/policy.js +258 -0
- package/package.json +58 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 dsh-retry-boost contributors
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.en.md
ADDED
|
@@ -0,0 +1,196 @@
|
|
|
1
|
+
# dsh-retry-boost
|
|
2
|
+
|
|
3
|
+
Auto-retry transient gateway failures until the task completes — a DSH plugin
|
|
4
|
+
that hot-applies a resilience `retryPolicy` to every `llm-pi-ai` provider
|
|
5
|
+
(also re-applied automatically when providers are added at runtime).
|
|
6
|
+
No more manual **"continue"** after rate-limit disconnects.
|
|
7
|
+
|
|
8
|
+
[中文说明](./README.md)
|
|
9
|
+
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
## The problem
|
|
13
|
+
|
|
14
|
+
When a gateway answers with **`429 insufficient_quota`** — the exact wording
|
|
15
|
+
SenseNova (and many other LLM gateways) uses — DSH's error classifier maps it
|
|
16
|
+
to the **`QUOTA`** failure code ([`isQuotaExceededError`](https://github.com/deepseek-ai/deepseek-harness))
|
|
17
|
+
*before* the generic 429 → `RATE_LIMIT` mapping.
|
|
18
|
+
|
|
19
|
+
DSH's built-in request retry (`dsh-llm-retry`) only retries failures whose
|
|
20
|
+
code is listed in the provider's `retryableCodes`. The **default** policy is:
|
|
21
|
+
|
|
22
|
+
```text
|
|
23
|
+
retryableCodes: [EMPTY_RESPONSE, RATE_LIMIT, SERVER, TIMEOUT, TRANSPORT] // ← no QUOTA
|
|
24
|
+
maxRetries: 5
|
|
25
|
+
backoff: 500ms → 10s
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
`QUOTA` is **not** in that list, so a transient `429 insufficient_quota` fails
|
|
29
|
+
fast: no retry at all, the turn dies instantly, and every manual "continue"
|
|
30
|
+
hits the same wall. Relevant upstream report:
|
|
31
|
+
[deepseek-ai/deepseek-harness discussion #892](https://github.com/deepseek-ai/deepseek-harness/discussions/892)
|
|
32
|
+
(0 replies so far — the fix is not upstream yet).
|
|
33
|
+
|
|
34
|
+
## The fix
|
|
35
|
+
|
|
36
|
+
This plugin writes a widened `retryPolicy` into the `llm-pi-ai` settings
|
|
37
|
+
namespace on startup. llm-pi-ai's settings `onChange` hot-applies it —
|
|
38
|
+
**no restart needed**. The injected policy:
|
|
39
|
+
|
|
40
|
+
```text
|
|
41
|
+
retryableCodes: [EMPTY_RESPONSE, RATE_LIMIT, QUOTA, SERVER, TIMEOUT, TRANSPORT, PI_AI_ERROR, UNKNOWN]
|
|
42
|
+
maxRetries: 50
|
|
43
|
+
backoff: 1s → 60s, jitter 0.2 (exponential)
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
Result: a transient 429 / stream drop / timeout is retried with a widening
|
|
47
|
+
backoff **until the task completes** — same as the settings.yaml workaround
|
|
48
|
+
used in production against SenseNova, but packaged so anyone can install it.
|
|
49
|
+
|
|
50
|
+
## Coverage and known limits
|
|
51
|
+
|
|
52
|
+
This plugin injects into providers of the `llm-pi-ai` settings namespace —
|
|
53
|
+
**all direct primary routes are covered**. The following are **out of reach**
|
|
54
|
+
(the plugin has no configuration surface to influence them):
|
|
55
|
+
|
|
56
|
+
- **Image-input variant routes** (e.g. `vision-toolkit-<provider>`): separate
|
|
57
|
+
adapters registered by vision plugins (e.g. dsh-vision-toolkit) via
|
|
58
|
+
`registerAdapter`. Their `retryPolicy` is captured once at registration and
|
|
59
|
+
never reads settings. DSH falls back to the built-in default for such routes
|
|
60
|
+
— whose `retryableCodes` do **not** include `QUOTA` — so a SenseNova 429
|
|
61
|
+
still fails fast on that route.
|
|
62
|
+
- The plugin auto-**detects and logs a warning** for such routes
|
|
63
|
+
(`warnUnprotectedVariants`, on by default, re-checked on adapter
|
|
64
|
+
updates). If you see `variant route "..." retries WITHOUT QUOTA`,
|
|
65
|
+
that is this detector.
|
|
66
|
+
- Way out: patch the variant adapter upstream (delegate
|
|
67
|
+
`providerRetryPolicy` to the upstream route), switch to a natively
|
|
68
|
+
image-capable model, or disable paste auto-switching to variants.
|
|
69
|
+
- **Self-registered routes of other third-party LLM adapter plugins**: same
|
|
70
|
+
story — they bypass the llm-pi-ai namespace, so injection cannot reach them.
|
|
71
|
+
|
|
72
|
+
## Why a plugin and not just settings.yaml?
|
|
73
|
+
|
|
74
|
+
- **One command install** — works for every provider, not just the one you
|
|
75
|
+
hand-edited.
|
|
76
|
+
- **Follows new providers** — add a provider tomorrow; the policy is applied
|
|
77
|
+
on the next start.
|
|
78
|
+
- **Configurable** — `fill` / `boost` / `force` strategies, adjustable budget.
|
|
79
|
+
- **Transparent** — the injected policy shows up in `settings.yaml`, fully
|
|
80
|
+
editable and visible; the plugin is idempotent and never fights your edits.
|
|
81
|
+
|
|
82
|
+
## How it differs from `dsh-chat-continue`
|
|
83
|
+
|
|
84
|
+
| | dsh-retry-boost (this) | [dsh-chat-continue](https://github.com/Chu-m/dsh-chat-continue) |
|
|
85
|
+
| --- | --- | --- |
|
|
86
|
+
| Layer | **First line of defense** — makes DSH's own retry actually work | **Second line** — retries *after* the built-in retry is exhausted |
|
|
87
|
+
| Mechanism | Injects `retryPolicy` into every `llm-pi-ai` provider (official retry chain) | Intercepts `agent/request-error` and re-issues the failed request itself |
|
|
88
|
+
| Backoff | Exponential, 1s → 60s, jittered | Fixed interval |
|
|
89
|
+
| Manual confirm | — | Optional notification popup |
|
|
90
|
+
| Install | `github:hhb1028/dsh-retry-boost#main` | `@wuxjs/dsh-chat-continue` |
|
|
91
|
+
|
|
92
|
+
They are **complementary**: run this one alone, or stack it with
|
|
93
|
+
dsh-chat-continue as a last-resort net.
|
|
94
|
+
|
|
95
|
+
## Requirements
|
|
96
|
+
|
|
97
|
+
- DSH `>= 0.1.1-rc.1` (host plugins via `dsh.bundle.patch`)
|
|
98
|
+
- Node.js `>= 20`
|
|
99
|
+
|
|
100
|
+
## Install
|
|
101
|
+
|
|
102
|
+
```bash
|
|
103
|
+
dsh plugin --profile <your-profile> add "github:hhb1028/dsh-retry-boost#main"
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
or from npm once published (see [Roadmap](#roadmap)):
|
|
107
|
+
|
|
108
|
+
```bash
|
|
109
|
+
dsh plugin --profile <your-profile> add dsh-retry-boost
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
Then restart DSH. Check the log for lines like:
|
|
113
|
+
|
|
114
|
+
```
|
|
115
|
+
dsh-retry-boost: injected retry policy for "sensenova" (mode=normal retries=50 backoff=1000ms->60000ms codes=8)
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
The injected policies are then visible in your profile's `settings.yaml`
|
|
119
|
+
under `llm-pi-ai.providers.*.retryPolicy` — edit them freely.
|
|
120
|
+
|
|
121
|
+
## Configuration
|
|
122
|
+
|
|
123
|
+
Tune the plugin in the roster row (`cordis.patch.yml`) merged into your
|
|
124
|
+
profile, or through your DSH plugin-config surface:
|
|
125
|
+
|
|
126
|
+
| Key | Default | Meaning |
|
|
127
|
+
| --- | --- | --- |
|
|
128
|
+
| `strategy` | `fill` | `fill` = only providers **without** an explicit `retryPolicy`; `boost` = also merge missing codes and raise caps on explicit ones; `force` = replace every provider's policy |
|
|
129
|
+
| `mode` | `normal` | `normal` = bounded retries; `always` = unbounded (DANGER: retries forever) |
|
|
130
|
+
| `maxRetries` | `50` | retry budget in `normal` mode |
|
|
131
|
+
| `initialDelayMs` | `1000` | first backoff delay |
|
|
132
|
+
| `maxDelayMs` | `60000` | backoff ceiling |
|
|
133
|
+
| `jitterRatio` | `0.2` | jitter ratio (0–1) |
|
|
134
|
+
| `retryableCodes` | the 8 codes above | failure codes to retry |
|
|
135
|
+
| `warnUnprotectedVariants` | `true` | log a warning when an unprotected variant route is detected (see "Coverage and known limits") |
|
|
136
|
+
|
|
137
|
+
## Uninstall
|
|
138
|
+
|
|
139
|
+
```bash
|
|
140
|
+
dsh plugin --profile <your-profile> remove <name>
|
|
141
|
+
```
|
|
142
|
+
|
|
143
|
+
The injected `retryPolicy` sections stay in `settings.yaml` after removal —
|
|
144
|
+
delete them by hand (or tell the plugin to `force` before uninstalling, then
|
|
145
|
+
restore your own values) to return to DSH defaults.
|
|
146
|
+
|
|
147
|
+
## How it works
|
|
148
|
+
|
|
149
|
+
1. On startup the plugin reads the `llm-pi-ai` settings namespace.
|
|
150
|
+
2. For each provider it plans a `retryPolicy` per `strategy` (pure functions
|
|
151
|
+
in `lib/policy.js`, fully unit-tested).
|
|
152
|
+
3. It merges the patch through DSH's settings provider: schema-validated by
|
|
153
|
+
llm-pi-ai (the same `RetryPolicySchema` from `dsh-llm`), persisted to
|
|
154
|
+
`settings.yaml`, hot-applied by `installSettingsSection`'s `onChange`.
|
|
155
|
+
4. Writes are idempotent: re-running with an already-boosted config produces
|
|
156
|
+
an empty patch.
|
|
157
|
+
5. Variant-route watch: listens for `llm/adapters-updated` and re-checks the
|
|
158
|
+
captured `retryPolicy` of `vision-toolkit-*` style variant routes; warns
|
|
159
|
+
once per route while QUOTA is missing (re-armed after a fix). Warn-only —
|
|
160
|
+
an honest statement of the reach boundary.
|
|
161
|
+
|
|
162
|
+
## Development
|
|
163
|
+
|
|
164
|
+
```bash
|
|
165
|
+
npm test # unit + schema-integration + apply-flow tests (node --test)
|
|
166
|
+
npm run check # syntax check
|
|
167
|
+
```
|
|
168
|
+
|
|
169
|
+
No build step — the plugin is plain ESM. The schema-integration test passes
|
|
170
|
+
generated policies through the *real* `RetryPolicySchema` /
|
|
171
|
+
`resolveRetryPolicy` from your framework checkout, so an injected patch can
|
|
172
|
+
never be rejected by the settings validator.
|
|
173
|
+
|
|
174
|
+
## Roadmap
|
|
175
|
+
|
|
176
|
+
- [ ] Publish to npm (`dsh-retry-boost`)
|
|
177
|
+
- [x] Friendlier reload: re-run the plan when `llm-pi-ai` settings change
|
|
178
|
+
(v1.2.0: listens on `llm/adapters-updated`, debounced 300ms, idempotent —
|
|
179
|
+
providers added at runtime are protected without a restart)
|
|
180
|
+
- [ ] Optional settings page (Settings → Retry Boost) reusing a generic form
|
|
181
|
+
|
|
182
|
+
## Changelog
|
|
183
|
+
|
|
184
|
+
### 1.2.0
|
|
185
|
+
|
|
186
|
+
- Providers added at runtime are now protected automatically: the plugin
|
|
187
|
+
re-sweeps on `llm/adapters-updated` (shared with the variant-route warning),
|
|
188
|
+
debounced 300ms. The plan is idempotent — sweeps triggered by this plugin's
|
|
189
|
+
own writes never write again, so there is no update loop. Startup behavior
|
|
190
|
+
is unchanged from v1.1.0.
|
|
191
|
+
- Useful when you add a new provider to settings.yaml (e.g. a second key as a
|
|
192
|
+
backup channel) or via the settings UI while DSH is running.
|
|
193
|
+
|
|
194
|
+
## License
|
|
195
|
+
|
|
196
|
+
MIT
|
package/README.md
ADDED
|
@@ -0,0 +1,175 @@
|
|
|
1
|
+
# dsh-retry-boost
|
|
2
|
+
|
|
3
|
+
遇到网关瞬时故障(429 限流/断流/超时)时自动退避重试、直到任务完成——一个 DSH
|
|
4
|
+
插件,启动时把防断流 `retryPolicy` 热注入到所有 `llm-pi-ai` provider;
|
|
5
|
+
之后在运行中新增/修改的 provider 也会自动补上(监听 `llm/adapters-updated`)。
|
|
6
|
+
再也不用手动发「继续」了。
|
|
7
|
+
|
|
8
|
+
[English README](./README.en.md)
|
|
9
|
+
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
## 问题背景
|
|
13
|
+
|
|
14
|
+
网关返回 **`429 insufficient_quota`**(商汤 SenseNova 和很多 LLM 网关都这么措辞)时,
|
|
15
|
+
DSH 的错误分类器会先命中 `isQuotaExceededError` 把它归为 **`QUOTA`** 码,
|
|
16
|
+
而不是通用的 429 → `RATE_LIMIT`。
|
|
17
|
+
|
|
18
|
+
DSH 内置重试(`dsh-llm-retry`)只重试 `retryableCodes` 列表里的错误码。而**默认**策略是:
|
|
19
|
+
|
|
20
|
+
```text
|
|
21
|
+
retryableCodes: [EMPTY_RESPONSE, RATE_LIMIT, SERVER, TIMEOUT, TRANSPORT] // ← 没有 QUOTA
|
|
22
|
+
maxRetries: 5
|
|
23
|
+
backoff: 500ms → 10s
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
`QUOTA` 不在列表里 → 瞬时 `429 insufficient_quota` 直接 **fail-fast**:
|
|
27
|
+
一次都不重试、turn 立刻死掉,手动「继续」也会再撞同一堵墙。上游同源问题:
|
|
28
|
+
[deepseek-ai/deepseek-harness discussion #892](https://github.com/deepseek-ai/deepseek-harness/discussions/892)
|
|
29
|
+
(目前 0 回复,官方尚未修复)。
|
|
30
|
+
|
|
31
|
+
## 解决方案
|
|
32
|
+
|
|
33
|
+
插件启动时把一份加宽的 `retryPolicy` 写入 `llm-pi-ai` 设置命名空间,
|
|
34
|
+
llm-pi-ai 的 settings `onChange` 会**热生效,无需重启**。注入的策略:
|
|
35
|
+
|
|
36
|
+
```text
|
|
37
|
+
retryableCodes: [EMPTY_RESPONSE, RATE_LIMIT, QUOTA, SERVER, TIMEOUT, TRANSPORT, PI_AI_ERROR, UNKNOWN]
|
|
38
|
+
maxRetries: 50
|
|
39
|
+
backoff: 1s → 60s,抖动 0.2(指数退避)
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
效果:瞬时 429 / 断流 / 超时会被指数退避**重试直到任务完成**——就是把商汤生产
|
|
43
|
+
环境里用的那个 settings.yaml 补丁产品化,让任何人都能一键装上。
|
|
44
|
+
|
|
45
|
+
## 覆盖范围与已知限制
|
|
46
|
+
|
|
47
|
+
本插件注入的是 `llm-pi-ai` 设置命名空间里的 provider——**直连主路由全覆盖**。
|
|
48
|
+
以下路径**不在射程内**(本插件没有任何配置面能影响它们):
|
|
49
|
+
|
|
50
|
+
- **图片输入变体路由**(如 `vision-toolkit-<provider>`):由视觉类插件
|
|
51
|
+
(如 dsh-vision-toolkit)通过 `registerAdapter` 注册的独立 adapter,
|
|
52
|
+
其 `retryPolicy` 在注册时一次性捕获、不读 settings。DSH 对这类路由
|
|
53
|
+
会落到内置默认值——`retryableCodes` 里**没有 QUOTA**,商汤 429 会在
|
|
54
|
+
这条路线上照样秒断。
|
|
55
|
+
- 插件会自动**检测并在日志里警告**这类路由(`warnUnprotectedVariants`,
|
|
56
|
+
默认开启,路由注册/更新时复检)。看到
|
|
57
|
+
`variant route "..." retries WITHOUT QUOTA` 就是它。
|
|
58
|
+
- 出路:给变体 adapter 的上游提补丁(让 `providerRetryPolicy` 委托
|
|
59
|
+
upstream 主路由)、换原生支持图片输入的模型、或临时关闭粘贴自动切变体。
|
|
60
|
+
- **其他第三方 LLM adapter 插件注册的自有路由**:同样不经过 llm-pi-ai
|
|
61
|
+
命名空间,本插件无法注入。
|
|
62
|
+
|
|
63
|
+
## 为什么做成插件而不是只改 settings.yaml?
|
|
64
|
+
|
|
65
|
+
- **一条命令安装**——对*所有* provider 生效,不只是你手改的那一个。
|
|
66
|
+
- **跟着新 provider 走**——以后新增 provider,下次启动自动套用。
|
|
67
|
+
- **可配置**——`fill` / `boost` / `force` 三种策略,重试预算可调。
|
|
68
|
+
- **透明**——注入的策略会出现在 `settings.yaml` 里,看得见、改得动;
|
|
69
|
+
插件幂等,绝不和你手改的配置打架。
|
|
70
|
+
|
|
71
|
+
## 与 dsh-chat-continue 的区别
|
|
72
|
+
|
|
73
|
+
| | dsh-retry-boost(本项目) | [dsh-chat-continue](https://github.com/Chu-m/dsh-chat-continue) |
|
|
74
|
+
| --- | --- | --- |
|
|
75
|
+
| 定位 | **第一道防线**——让 DSH 内置重试真正生效 | **第二道防线**——内置重试耗尽后兜底重试 |
|
|
76
|
+
| 机制 | 给所有 `llm-pi-ai` provider 注入 `retryPolicy`(走官方重试链路) | 拦截 `agent/request-error` 自己重发失败的请求 |
|
|
77
|
+
| 退避 | 指数退避 1s → 60s + 抖动 | 固定间隔 |
|
|
78
|
+
| 手动确认 | —— | 可选弹窗通知 |
|
|
79
|
+
| 安装 | `github:hhb1028/dsh-retry-boost#main` | `@wuxjs/dsh-chat-continue` |
|
|
80
|
+
|
|
81
|
+
两者**互补**:单独用本项目即可;也可以叠上 chat-continue 作为最后的兜底网。
|
|
82
|
+
|
|
83
|
+
## 环境要求
|
|
84
|
+
|
|
85
|
+
- DSH `>= 0.1.1-rc.1`(支持 `dsh.bundle.patch` 宿主插件)
|
|
86
|
+
- Node.js `>= 20`
|
|
87
|
+
|
|
88
|
+
## 安装
|
|
89
|
+
|
|
90
|
+
```bash
|
|
91
|
+
dsh plugin --profile <你的profile名> add "github:hhb1028/dsh-retry-boost#main"
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
发布到 npm 后(见 [Roadmap](#roadmap)):
|
|
95
|
+
|
|
96
|
+
```bash
|
|
97
|
+
dsh plugin --profile <你的profile名> add dsh-retry-boost
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
然后重启 DSH。日志里会出现类似:
|
|
101
|
+
|
|
102
|
+
```
|
|
103
|
+
dsh-retry-boost: injected retry policy for "sensenova" (mode=normal retries=50 backoff=1000ms->60000ms codes=8)
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
注入后的策略会出现在 profile 的 `settings.yaml` 中
|
|
107
|
+
`llm-pi-ai.providers.*.retryPolicy` 下——可以随意手动修改。
|
|
108
|
+
|
|
109
|
+
## 配置项
|
|
110
|
+
|
|
111
|
+
在合并进 profile 的 `cordis.patch.yml` roster 行(或 DSH 的插件配置界面)里调整:
|
|
112
|
+
|
|
113
|
+
| 键 | 默认值 | 含义 |
|
|
114
|
+
| --- | --- | --- |
|
|
115
|
+
| `strategy` | `fill` | `fill` = 只处理**没有**显式 `retryPolicy` 的 provider;`boost` = 对显式配置的也合并缺失错误码并抬高上限;`force` = 全部替换为插件策略 |
|
|
116
|
+
| `mode` | `normal` | `normal` = 有界重试;`always` = 无限重试(⚠️ 慎用,永远重试下去) |
|
|
117
|
+
| `maxRetries` | `50` | `normal` 模式下的重试次数上限 |
|
|
118
|
+
| `initialDelayMs` | `1000` | 首次退避延迟 |
|
|
119
|
+
| `maxDelayMs` | `60000` | 退避上限 |
|
|
120
|
+
| `jitterRatio` | `0.2` | 抖动比例(0–1) |
|
|
121
|
+
| `retryableCodes` | 上面 8 个码 | 需要重试的失败码 |
|
|
122
|
+
| `warnUnprotectedVariants` | `true` | 检测到不受保护的变体路由(见「覆盖范围与已知限制」)时打日志警告 |
|
|
123
|
+
|
|
124
|
+
## 卸载
|
|
125
|
+
|
|
126
|
+
```bash
|
|
127
|
+
dsh plugin --profile <你的profile名> remove <名字>
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
卸载后注入到 `settings.yaml` 里的 `retryPolicy` 段**不会自动删除**——手动清掉
|
|
131
|
+
(或卸载前先用 `force` 策略并恢复你自己的值),即可回到 DSH 默认行为。
|
|
132
|
+
|
|
133
|
+
## 工作原理
|
|
134
|
+
|
|
135
|
+
1. 启动时读取 `llm-pi-ai` 设置命名空间。
|
|
136
|
+
2. 按 `strategy` 为每个 provider 规划目标 `retryPolicy`(`lib/policy.js` 纯函数,
|
|
137
|
+
全量单元测试)。
|
|
138
|
+
3. 通过 DSH 的 settings provider 合并写入:经 llm-pi-ai 校验(与 `dsh-llm`
|
|
139
|
+
的 `RetryPolicySchema` 完全一致)→ 持久化到 `settings.yaml` →
|
|
140
|
+
`installSettingsSection` 的 `onChange` 热生效。
|
|
141
|
+
4. 写入幂等:已达标时再次运行产生空 patch,绝不反复改写。
|
|
142
|
+
5. 变体路由巡检:监听 `llm/adapters-updated`,对 `vision-toolkit-*` 等变体
|
|
143
|
+
路由的已捕获 `retryPolicy` 复检;缺 QUOTA 时打警告(每条路由一次,
|
|
144
|
+
修复后重新武装),只警告不改动——诚实告知射程边界。
|
|
145
|
+
|
|
146
|
+
## 开发
|
|
147
|
+
|
|
148
|
+
```bash
|
|
149
|
+
npm test # 单元 + 真实 schema 集成 + apply 流程测试(node --test)
|
|
150
|
+
npm run check # 语法检查
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
无构建步骤——纯 ESM。schema 集成测试会把生成的策略跑过框架**真实**的
|
|
154
|
+
`RetryPolicySchema` / `resolveRetryPolicy`,保证注入的 patch 永远不会被设置校验拒绝。
|
|
155
|
+
|
|
156
|
+
## Roadmap
|
|
157
|
+
|
|
158
|
+
- [ ] 发布到 npm(`dsh-retry-boost`)
|
|
159
|
+
- [x] llm-pi-ai 配置变化时自动重跑策略(v1.2.0:监听 `llm/adapters-updated`,
|
|
160
|
+
防抖 300ms,幂等防自触发——运行中新增的 provider 无需重启即受保护)
|
|
161
|
+
- [ ] 可选的设置页面(Settings → Retry Boost)
|
|
162
|
+
|
|
163
|
+
## 更新记录
|
|
164
|
+
|
|
165
|
+
### 1.2.0
|
|
166
|
+
|
|
167
|
+
- 运行中新增的 provider 自动受保护:监听 `llm/adapters-updated`(与变体路由
|
|
168
|
+
警告共用事件),防抖 300ms 后重跑注入计划。计划幂等——本插件自己写入引发
|
|
169
|
+
的扫描不会造成二次写入,更不会死循环;启动行为与 v1.1.0 完全一致。
|
|
170
|
+
- 适用于:先启动 DSH、之后往 settings.yaml 加新 provider(比如第二把 key 的
|
|
171
|
+
备用通道)、或在设置界面新增供应商的场景,全程无需重启。
|
|
172
|
+
|
|
173
|
+
## 许可证
|
|
174
|
+
|
|
175
|
+
MIT
|
package/cordis.patch.yml
ADDED
|
@@ -0,0 +1,30 @@
|
|
|
1
|
+
# dsh-retry-boost bundle patch: inserts the host plugin row into the profile
|
|
2
|
+
# roster. Applied as a profile bundle layer (the `dsh.bundle.patch` manifest
|
|
3
|
+
# field). The node half (exports ".") runs in the host process and hot-applies
|
|
4
|
+
# a resilience retryPolicy to every llm-pi-ai provider on startup. No browser
|
|
5
|
+
# half.
|
|
6
|
+
#
|
|
7
|
+
# Defaults mirror the SenseNova 429 fix: bounded 50-attempt retries with an
|
|
8
|
+
# exponential backoff of 1s -> 60s (jitter 0.2) across the FULL failure-code
|
|
9
|
+
# set (QUOTA included). Tune `strategy`: "fill" (only providers without an
|
|
10
|
+
# explicit retryPolicy), "boost" (also merge codes/raise caps on explicit
|
|
11
|
+
# ones), "force" (replace everything).
|
|
12
|
+
- insert:
|
|
13
|
+
- id: retry-boost
|
|
14
|
+
name: dsh-retry-boost
|
|
15
|
+
config:
|
|
16
|
+
strategy: fill
|
|
17
|
+
mode: normal
|
|
18
|
+
maxRetries: 50
|
|
19
|
+
initialDelayMs: 1000
|
|
20
|
+
maxDelayMs: 60000
|
|
21
|
+
jitterRatio: 0.2
|
|
22
|
+
retryableCodes:
|
|
23
|
+
- EMPTY_RESPONSE
|
|
24
|
+
- RATE_LIMIT
|
|
25
|
+
- QUOTA
|
|
26
|
+
- SERVER
|
|
27
|
+
- TIMEOUT
|
|
28
|
+
- TRANSPORT
|
|
29
|
+
- PI_AI_ERROR
|
|
30
|
+
- UNKNOWN
|
package/lib/index.js
ADDED
|
@@ -0,0 +1,209 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* dsh-retry-boost — DSH plugin entry.
|
|
3
|
+
*
|
|
4
|
+
* On startup it plans and hot-applies a resilience `retryPolicy` for every
|
|
5
|
+
* provider registered under the built-in `llm-pi-ai` settings namespace, so
|
|
6
|
+
* transient gateway failures — most importantly `429 insufficient_quota`
|
|
7
|
+
* (code `QUOTA`, which DSH's default retry list does NOT contain) — are
|
|
8
|
+
* automatically retried with a widened exponential backoff until the task
|
|
9
|
+
* completes, with no manual "continue" needed.
|
|
10
|
+
*
|
|
11
|
+
* Providers added after boot (or whose explicit policy was removed) are picked
|
|
12
|
+
* up automatically: the plugin re-sweeps on `llm/adapters-updated`, so a newly
|
|
13
|
+
* registered provider gets protected without a restart.
|
|
14
|
+
*
|
|
15
|
+
* The write goes through the settings provider (`llm-pi-ai` namespace, merge
|
|
16
|
+
* patch), which is validated by llm-pi-ai and hot-applied by its
|
|
17
|
+
* `installSettingsSection` onChange → no restart, and it is visible and
|
|
18
|
+
* user-editable in settings.yaml afterwards. Writes are idempotent.
|
|
19
|
+
*/
|
|
20
|
+
|
|
21
|
+
import {
|
|
22
|
+
findUnprotectedVariants,
|
|
23
|
+
isPolicyProtected,
|
|
24
|
+
normalizeConfig,
|
|
25
|
+
planInjection,
|
|
26
|
+
} from "./policy.js";
|
|
27
|
+
|
|
28
|
+
/** Plugin identity, matched by the cordis.patch.yml roster row. */
|
|
29
|
+
export const name = "retry-boost";
|
|
30
|
+
/** Wait for the settings service before touching the `llm-pi-ai` namespace. */
|
|
31
|
+
export const inject = ["settings"];
|
|
32
|
+
|
|
33
|
+
/** The settings namespace owned by the built-in llm-pi-ai plugin. */
|
|
34
|
+
export const LLM_PI_AI_NS = "llm-pi-ai";
|
|
35
|
+
|
|
36
|
+
/** Defensive wait upper bound for the namespace to appear (boot ordering). */
|
|
37
|
+
const WAIT_TRIES = 50;
|
|
38
|
+
const WAIT_STEP_MS = 50;
|
|
39
|
+
|
|
40
|
+
/** Coalesce bursts of `llm/adapters-updated` events before a runtime re-sweep. */
|
|
41
|
+
const RESWEEP_DEBOUNCE_MS = 300;
|
|
42
|
+
|
|
43
|
+
/** @param {number} ms */
|
|
44
|
+
function sleep(ms) {
|
|
45
|
+
return new Promise((resolve) => setTimeout(resolve, ms));
|
|
46
|
+
}
|
|
47
|
+
|
|
48
|
+
/**
|
|
49
|
+
* llm-pi-ai is a built-in plugin and registers its namespace before profile
|
|
50
|
+
* plugins apply, but we guard boot-order races with a short bounded wait.
|
|
51
|
+
* @param {import("@deepseek-ai/dsh-settings").SettingsProvider} settings
|
|
52
|
+
*/
|
|
53
|
+
async function waitForNamespace(settings) {
|
|
54
|
+
for (let attempt = 0; attempt < WAIT_TRIES; attempt += 1) {
|
|
55
|
+
if (settings.get?.(LLM_PI_AI_NS) !== undefined) return;
|
|
56
|
+
await sleep(WAIT_STEP_MS);
|
|
57
|
+
}
|
|
58
|
+
throw new Error(
|
|
59
|
+
`dsh-retry-boost: settings namespace "${LLM_PI_AI_NS}" never appeared — ` +
|
|
60
|
+
"is the built-in llm-pi-ai plugin enabled?",
|
|
61
|
+
);
|
|
62
|
+
}
|
|
63
|
+
|
|
64
|
+
/**
|
|
65
|
+
* @param {import("@deepseek-ai/cordis").Context} ctx - cordis plugin context.
|
|
66
|
+
* @param {object} [config] - plugin config from the roster row (see cordis.patch.yml).
|
|
67
|
+
*/
|
|
68
|
+
export function apply(ctx, config = {}) {
|
|
69
|
+
let cfg;
|
|
70
|
+
try {
|
|
71
|
+
cfg = normalizeConfig(config);
|
|
72
|
+
} catch (error) {
|
|
73
|
+
ctx.logger.error("dsh-retry-boost: %s", error.message);
|
|
74
|
+
return;
|
|
75
|
+
}
|
|
76
|
+
ctx.logger.info(
|
|
77
|
+
"dsh-retry-boost: strategy=%s mode=%s maxRetries=%d backoff=%dms->%dms codes=%s",
|
|
78
|
+
cfg.strategy, cfg.mode, cfg.maxRetries, cfg.initialDelayMs, cfg.maxDelayMs,
|
|
79
|
+
cfg.retryableCodes.join(","),
|
|
80
|
+
);
|
|
81
|
+
ctx.inject(["settings"], async (sctx) => {
|
|
82
|
+
try {
|
|
83
|
+
await waitForNamespace(sctx.settings);
|
|
84
|
+
await sweepProviders(ctx, sctx.settings, cfg, "startup");
|
|
85
|
+
} catch (error) {
|
|
86
|
+
ctx.logger.error("dsh-retry-boost: failed to apply retry policies: %o", error);
|
|
87
|
+
}
|
|
88
|
+
});
|
|
89
|
+
watchRuntimeProviders(ctx, cfg);
|
|
90
|
+
if (cfg.warnUnprotectedVariants) {
|
|
91
|
+
watchVariantRoutes(ctx, cfg);
|
|
92
|
+
}
|
|
93
|
+
}
|
|
94
|
+
|
|
95
|
+
/**
|
|
96
|
+
* Re-plan and hot-apply policies for providers that appear AFTER boot (user
|
|
97
|
+
* registers a new provider in settings.yaml or via the UI). Driven by the same
|
|
98
|
+
* `llm/adapters-updated` event the variant watcher listens on, debounced to
|
|
99
|
+
* coalesce bursts. Loop-safe: the plan is idempotent (resolveTargetPolicy
|
|
100
|
+
* returns null for unchanged providers), so the sweep triggered by this
|
|
101
|
+
* plugin's own write ends after one extra no-op pass. If the namespace has not
|
|
102
|
+
* appeared yet the sweep is a no-op; the startup path still covers boot.
|
|
103
|
+
* @param {import("@deepseek-ai/cordis").Context} ctx - cordis plugin context.
|
|
104
|
+
* @param {object} cfg - normalized plugin config.
|
|
105
|
+
*/
|
|
106
|
+
function watchRuntimeProviders(ctx, cfg) {
|
|
107
|
+
if (typeof ctx.on !== "function") return;
|
|
108
|
+
ctx.inject(["settings"], (sctx) => {
|
|
109
|
+
const settings = sctx.settings;
|
|
110
|
+
let timer = null;
|
|
111
|
+
const resweep = () => {
|
|
112
|
+
timer = null;
|
|
113
|
+
Promise.resolve()
|
|
114
|
+
.then(() => sweepProviders(ctx, settings, cfg, "runtime"))
|
|
115
|
+
.catch((error) => ctx.logger.error("dsh-retry-boost: runtime re-sweep failed: %o", error));
|
|
116
|
+
};
|
|
117
|
+
ctx.on("llm/adapters-updated", () => {
|
|
118
|
+
if (timer !== null) clearTimeout(timer);
|
|
119
|
+
timer = setTimeout(resweep, RESWEEP_DEBOUNCE_MS);
|
|
120
|
+
});
|
|
121
|
+
});
|
|
122
|
+
}
|
|
123
|
+
|
|
124
|
+
/**
|
|
125
|
+
* Plan the injection for the currently registered providers and write it when
|
|
126
|
+
* non-empty. Idempotent: writes nothing when every provider is already at its
|
|
127
|
+
* target policy.
|
|
128
|
+
* @param {import("@deepseek-ai/cordis").Context} ctx - cordis plugin context.
|
|
129
|
+
* @param {import("@deepseek-ai/dsh-settings").SettingsProvider} settings
|
|
130
|
+
* @param {object} cfg - normalized plugin config.
|
|
131
|
+
* @param {"startup"|"runtime"} source - sweep origin, for log context.
|
|
132
|
+
* @returns {Promise<boolean>} whether a settings update was issued.
|
|
133
|
+
*/
|
|
134
|
+
async function sweepProviders(ctx, settings, cfg, source) {
|
|
135
|
+
const current = settings.get(LLM_PI_AI_NS);
|
|
136
|
+
const providers = current?.providers ?? {};
|
|
137
|
+
const { patch, report } = planInjection(providers, cfg);
|
|
138
|
+
const touched = Object.keys(patch.providers);
|
|
139
|
+
if (touched.length === 0) {
|
|
140
|
+
if (source === "startup") {
|
|
141
|
+
ctx.logger.info("dsh-retry-boost: nothing to adjust (strategy=%s)", cfg.strategy);
|
|
142
|
+
}
|
|
143
|
+
return false;
|
|
144
|
+
}
|
|
145
|
+
await settings.update(LLM_PI_AI_NS, patch);
|
|
146
|
+
for (const row of report) {
|
|
147
|
+
if (row.action === "skipped") continue;
|
|
148
|
+
ctx.logger.info(
|
|
149
|
+
"dsh-retry-boost: (%s) %s provider \"%s\" (%s)",
|
|
150
|
+
source,
|
|
151
|
+
row.action === "injected" ? "injected retry policy for" : "boosted retry policy for",
|
|
152
|
+
row.provider,
|
|
153
|
+
row.detail,
|
|
154
|
+
);
|
|
155
|
+
}
|
|
156
|
+
return true;
|
|
157
|
+
}
|
|
158
|
+
|
|
159
|
+
/**
|
|
160
|
+
* Warn about registered variant routes (vision-toolkit image-input twins etc.)
|
|
161
|
+
* whose captured retryPolicy still fails QUOTA fast. These routes are registered
|
|
162
|
+
* by OTHER adapters outside the llm-pi-ai settings namespace, so this plugin
|
|
163
|
+
* cannot inject a fix — warning is the honest maximum. Re-checks on every
|
|
164
|
+
* llm/adapters-updated (variant registration is async), warns once per route
|
|
165
|
+
* until its policy is fixed or the route disappears, then re-arms.
|
|
166
|
+
* @param {import("@deepseek-ai/cordis").Context} ctx - cordis plugin context.
|
|
167
|
+
* @param {object} cfg - normalized plugin config.
|
|
168
|
+
*/
|
|
169
|
+
function watchVariantRoutes(ctx, cfg) {
|
|
170
|
+
ctx.inject(["llm"], (lctx) => {
|
|
171
|
+
const llm = lctx.llm;
|
|
172
|
+
/** Routes already warned about; removed when fixed or gone, so a later
|
|
173
|
+
* regression warns again instead of being silenced forever. */
|
|
174
|
+
const warned = new Set();
|
|
175
|
+
const sweep = () => {
|
|
176
|
+
let providers;
|
|
177
|
+
try {
|
|
178
|
+
providers = llm.listProviders();
|
|
179
|
+
} catch {
|
|
180
|
+
return;
|
|
181
|
+
}
|
|
182
|
+
for (const finding of findUnprotectedVariants(providers, (id) => llm.providerRetryPolicy(id))) {
|
|
183
|
+
if (warned.has(finding.provider)) continue;
|
|
184
|
+
warned.add(finding.provider);
|
|
185
|
+
ctx.logger.warn(
|
|
186
|
+
"dsh-retry-boost: variant route \"%s\" (%s) retries WITHOUT QUOTA — 429 insufficient_quota fails fast there. " +
|
|
187
|
+
"Variant routes live outside the llm-pi-ai settings namespace, so this plugin cannot protect them; " +
|
|
188
|
+
"patch the variant adapter upstream or pick a natively image-capable model.",
|
|
189
|
+
finding.provider,
|
|
190
|
+
finding.name ?? finding.provider,
|
|
191
|
+
);
|
|
192
|
+
}
|
|
193
|
+
for (const id of [...warned]) {
|
|
194
|
+
let policy;
|
|
195
|
+
try {
|
|
196
|
+
policy = llm.providerRetryPolicy(id);
|
|
197
|
+
} catch {
|
|
198
|
+
warned.delete(id); // route gone — re-arm
|
|
199
|
+
continue;
|
|
200
|
+
}
|
|
201
|
+
if (isPolicyProtected(policy)) warned.delete(id); // fixed — re-arm for future regressions
|
|
202
|
+
}
|
|
203
|
+
};
|
|
204
|
+
sweep();
|
|
205
|
+
if (typeof ctx.on === "function") {
|
|
206
|
+
ctx.on("llm/adapters-updated", () => sweep());
|
|
207
|
+
}
|
|
208
|
+
});
|
|
209
|
+
}
|
package/lib/policy.js
ADDED
|
@@ -0,0 +1,258 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* dsh-retry-boost — pure retry-policy planning.
|
|
3
|
+
*
|
|
4
|
+
* DSH's built-in request retry (`dsh-llm-retry`) only retries a failure when
|
|
5
|
+
* the failure code is present in the provider's `retryableCodes`. The default
|
|
6
|
+
* policy shipped by `dsh-llm` is:
|
|
7
|
+
*
|
|
8
|
+
* maxRetries 5 · initialDelayMs 500 · maxDelayMs 10_000
|
|
9
|
+
* retryableCodes [EMPTY_RESPONSE, RATE_LIMIT, SERVER, TIMEOUT, TRANSPORT]
|
|
10
|
+
*
|
|
11
|
+
* `QUOTA` is missing, so a gateway answering `429 insufficient_quota` (the
|
|
12
|
+
* wording SenseNova — and many other gateways — actually use) is classified
|
|
13
|
+
* as QUOTA by `classifyPiAiError` and **fails fast**: no retry, the turn dies,
|
|
14
|
+
* the user has to click "continue" and often hits the same 429 again.
|
|
15
|
+
*
|
|
16
|
+
* This module plans a per-provider `retryPolicy` that fixes exactly that:
|
|
17
|
+
* - adds QUOTA (+ PI_AI_ERROR, UNKNOWN) to the retryable set,
|
|
18
|
+
* - raises the retry budget (50 attempts),
|
|
19
|
+
* - widens the exponential backoff window (1s → 60s, jittered).
|
|
20
|
+
*
|
|
21
|
+
* The plugin writes the planned patch into the `llm-pi-ai` settings namespace
|
|
22
|
+
* (hot-applied by llm-pi-ai's settings.onChange — no restart needed).
|
|
23
|
+
*/
|
|
24
|
+
|
|
25
|
+
/** Every failure code the built-in classifier can produce. */
|
|
26
|
+
export const DEFAULT_RETRYABLE_CODES = Object.freeze([
|
|
27
|
+
"EMPTY_RESPONSE",
|
|
28
|
+
"RATE_LIMIT",
|
|
29
|
+
"QUOTA",
|
|
30
|
+
"SERVER",
|
|
31
|
+
"TIMEOUT",
|
|
32
|
+
"TRANSPORT",
|
|
33
|
+
"PI_AI_ERROR",
|
|
34
|
+
"UNKNOWN",
|
|
35
|
+
]);
|
|
36
|
+
|
|
37
|
+
export const STRATEGIES = Object.freeze(["fill", "boost", "force"]);
|
|
38
|
+
export const MODES = Object.freeze(["normal", "always"]);
|
|
39
|
+
|
|
40
|
+
export const DEFAULT_CONFIG = Object.freeze({
|
|
41
|
+
/** fill = only touch providers that have no explicit retryPolicy (default).
|
|
42
|
+
* boost = additionally strengthen providers that do (merge codes, raise caps).
|
|
43
|
+
* force = replace every provider's retryPolicy with the boost policy. */
|
|
44
|
+
strategy: "fill",
|
|
45
|
+
/** "normal" = bounded retries, "always" = unbounded (mode: always, careful). */
|
|
46
|
+
mode: "normal",
|
|
47
|
+
maxRetries: 50,
|
|
48
|
+
initialDelayMs: 1000,
|
|
49
|
+
maxDelayMs: 60000,
|
|
50
|
+
jitterRatio: 0.2,
|
|
51
|
+
retryableCodes: [...DEFAULT_RETRYABLE_CODES],
|
|
52
|
+
/** Warn (log only) when a registered variant route (e.g. vision-toolkit
|
|
53
|
+
* image-input twins) would still fail QUOTA fast — such routes live outside
|
|
54
|
+
* the llm-pi-ai settings namespace, so injection cannot reach them. */
|
|
55
|
+
warnUnprotectedVariants: true,
|
|
56
|
+
});
|
|
57
|
+
|
|
58
|
+
/** Keep only the keys this plugin owns, so it never drags foreign keys into settings. */
|
|
59
|
+
const KNOWN_KEYS = new Set(Object.keys(DEFAULT_CONFIG));
|
|
60
|
+
|
|
61
|
+
/** @returns a copy of `input` restricted to known config keys (undefined-safe). */
|
|
62
|
+
export function pickKnown(input = {}) {
|
|
63
|
+
if (typeof input !== "object" || input === null) return {};
|
|
64
|
+
const out = {};
|
|
65
|
+
for (const key of KNOWN_KEYS) {
|
|
66
|
+
if (key in input && input[key] !== undefined) out[key] = input[key];
|
|
67
|
+
}
|
|
68
|
+
return out;
|
|
69
|
+
}
|
|
70
|
+
|
|
71
|
+
function fail(message) {
|
|
72
|
+
throw new TypeError(`dsh-retry-boost config: ${message}`);
|
|
73
|
+
}
|
|
74
|
+
|
|
75
|
+
/** Validate and normalize a plugin config object. */
|
|
76
|
+
export function normalizeConfig(input = {}) {
|
|
77
|
+
const base = { ...DEFAULT_CONFIG, ...pickKnown(input) };
|
|
78
|
+
if (!STRATEGIES.includes(base.strategy)) fail(`strategy must be one of ${STRATEGIES.join("/")}`);
|
|
79
|
+
if (!MODES.includes(base.mode)) fail(`mode must be one of ${MODES.join("/")}`);
|
|
80
|
+
if (base.warnUnprotectedVariants !== undefined && typeof base.warnUnprotectedVariants !== "boolean") {
|
|
81
|
+
fail("warnUnprotectedVariants must be a boolean");
|
|
82
|
+
}
|
|
83
|
+
const maxRetries = Number(base.maxRetries);
|
|
84
|
+
if (!Number.isSafeInteger(maxRetries) || maxRetries < 0) fail("maxRetries must be a non-negative safe integer");
|
|
85
|
+
const initialDelayMs = Number(base.initialDelayMs);
|
|
86
|
+
const maxDelayMs = Number(base.maxDelayMs);
|
|
87
|
+
const jitterRatio = Number(base.jitterRatio);
|
|
88
|
+
if (!Number.isFinite(initialDelayMs) || initialDelayMs <= 0) fail("initialDelayMs must be a positive number");
|
|
89
|
+
if (!Number.isFinite(maxDelayMs) || maxDelayMs <= 0) fail("maxDelayMs must be a positive number");
|
|
90
|
+
if (initialDelayMs > maxDelayMs) fail("initialDelayMs must be <= maxDelayMs");
|
|
91
|
+
if (!Number.isFinite(jitterRatio) || jitterRatio < 0 || jitterRatio > 1) fail("jitterRatio must be between 0 and 1");
|
|
92
|
+
const codes = Array.isArray(base.retryableCodes)
|
|
93
|
+
? base.retryableCodes
|
|
94
|
+
: [...DEFAULT_RETRYABLE_CODES];
|
|
95
|
+
if (codes.length === 0) fail("retryableCodes must not be empty");
|
|
96
|
+
for (const code of codes) {
|
|
97
|
+
if (typeof code !== "string" || code.length === 0) fail("retryableCodes must contain only non-empty strings");
|
|
98
|
+
}
|
|
99
|
+
if (new Set(codes).size !== codes.length) fail("retryableCodes must not contain duplicates");
|
|
100
|
+
return Object.freeze({
|
|
101
|
+
strategy: base.strategy,
|
|
102
|
+
mode: base.mode,
|
|
103
|
+
maxRetries,
|
|
104
|
+
initialDelayMs,
|
|
105
|
+
maxDelayMs,
|
|
106
|
+
jitterRatio,
|
|
107
|
+
retryableCodes: Object.freeze([...codes]),
|
|
108
|
+
warnUnprotectedVariants: base.warnUnprotectedVariants ?? true,
|
|
109
|
+
});
|
|
110
|
+
}
|
|
111
|
+
|
|
112
|
+
/** Union-preserving code merge: keep `existing` order, append missing `want` codes. */
|
|
113
|
+
export function mergeCodes(existing, want) {
|
|
114
|
+
const seen = new Set(existing);
|
|
115
|
+
return [...existing, ...want.filter((code) => !seen.has(code))];
|
|
116
|
+
}
|
|
117
|
+
|
|
118
|
+
/**
|
|
119
|
+
* Build the policy the plugin would write for one provider.
|
|
120
|
+
* @param {string} strategy - fill | boost | force
|
|
121
|
+
* @param {object|undefined} existing - the provider's current retryPolicy, if any.
|
|
122
|
+
* @param {object} cfg - normalized plugin config.
|
|
123
|
+
* @returns {object|null} the target retryPolicy, or null when nothing should change.
|
|
124
|
+
*/
|
|
125
|
+
export function resolveTargetPolicy(strategy, existing, cfg) {
|
|
126
|
+
const boost = makePolicy(cfg);
|
|
127
|
+
// No explicit policy (or forced): write the boost policy.
|
|
128
|
+
if (strategy === "force" || existing === undefined) return boost;
|
|
129
|
+
if (strategy === "fill") return null; // explicit user policy wins
|
|
130
|
+
// boost: keep the user's mode, raise the caps, complete the codes.
|
|
131
|
+
const merged = { ...existing };
|
|
132
|
+
merged.retryableCodes = mergeCodes(existing.retryableCodes ?? [], cfg.retryableCodes);
|
|
133
|
+
if (existing.mode === "normal") {
|
|
134
|
+
merged.maxRetries = Math.max(existing.maxRetries ?? 0, cfg.maxRetries);
|
|
135
|
+
}
|
|
136
|
+
const backoff = { ...(existing.backoff ?? {}) };
|
|
137
|
+
backoff.initialDelayMs = Math.min(existing.backoff?.initialDelayMs ?? cfg.initialDelayMs, cfg.initialDelayMs);
|
|
138
|
+
backoff.maxDelayMs = Math.max(existing.backoff?.maxDelayMs ?? 0, cfg.maxDelayMs);
|
|
139
|
+
backoff.jitterRatio = existing.backoff?.jitterRatio ?? cfg.jitterRatio;
|
|
140
|
+
merged.backoff = backoff;
|
|
141
|
+
return equalPolicy(merged, existing) ? null : merged;
|
|
142
|
+
}
|
|
143
|
+
|
|
144
|
+
/** The policy this plugin writes for a provider (bounded or unbounded). */
|
|
145
|
+
export function makePolicy(cfg) {
|
|
146
|
+
const backoff = {
|
|
147
|
+
initialDelayMs: cfg.initialDelayMs,
|
|
148
|
+
maxDelayMs: cfg.maxDelayMs,
|
|
149
|
+
jitterRatio: cfg.jitterRatio,
|
|
150
|
+
};
|
|
151
|
+
if (cfg.mode === "always") return { mode: "always", backoff };
|
|
152
|
+
return {
|
|
153
|
+
mode: "normal",
|
|
154
|
+
maxRetries: cfg.maxRetries,
|
|
155
|
+
retryableCodes: [...cfg.retryableCodes],
|
|
156
|
+
backoff,
|
|
157
|
+
};
|
|
158
|
+
}
|
|
159
|
+
|
|
160
|
+
function equalPolicy(a, b) {
|
|
161
|
+
if (a === undefined || b === undefined) return a === b;
|
|
162
|
+
return JSON.stringify(sortPolicy(a)) === JSON.stringify(sortPolicy(b));
|
|
163
|
+
}
|
|
164
|
+
|
|
165
|
+
/** Canonical, order-independent serialization for change detection. */
|
|
166
|
+
function sortPolicy(policy) {
|
|
167
|
+
return {
|
|
168
|
+
mode: policy.mode,
|
|
169
|
+
...(policy.mode === "normal"
|
|
170
|
+
? { maxRetries: policy.maxRetries, retryableCodes: [...policy.retryableCodes].sort() }
|
|
171
|
+
: {}),
|
|
172
|
+
backoff: {
|
|
173
|
+
initialDelayMs: policy.backoff?.initialDelayMs,
|
|
174
|
+
maxDelayMs: policy.backoff?.maxDelayMs,
|
|
175
|
+
jitterRatio: policy.backoff?.jitterRatio,
|
|
176
|
+
},
|
|
177
|
+
};
|
|
178
|
+
}
|
|
179
|
+
|
|
180
|
+
/**
|
|
181
|
+
* Plan the settings patch for `llm-pi-ai`.
|
|
182
|
+
* @param {Record<string, any>} providers - raw providers section (may be {}).
|
|
183
|
+
* @param {object} cfg - normalized plugin config.
|
|
184
|
+
* @returns {{patch: {providers: Record<string, {retryPolicy: object}>}, report: Array<object>}}
|
|
185
|
+
* `report` rows are {provider, action: "injected"|"boosted"|"skipped"} with a short summary.
|
|
186
|
+
*/
|
|
187
|
+
export function planInjection(providers = {}, cfg) {
|
|
188
|
+
const patch = { providers: {} };
|
|
189
|
+
const report = [];
|
|
190
|
+
for (const [id, profile] of Object.entries(providers ?? {})) {
|
|
191
|
+
if (typeof profile !== "object" || profile === null) continue;
|
|
192
|
+
const existing = profile.retryPolicy;
|
|
193
|
+
const target = resolveTargetPolicy(cfg.strategy, existing, cfg);
|
|
194
|
+
if (target === null) {
|
|
195
|
+
report.push({ provider: id, action: "skipped", detail: existing ? "user policy kept" : "no provider" });
|
|
196
|
+
} else if (equalPolicy(target, existing)) {
|
|
197
|
+
report.push({ provider: id, action: "skipped", detail: "already boosted" });
|
|
198
|
+
} else {
|
|
199
|
+
patch.providers[id] = { retryPolicy: target };
|
|
200
|
+
report.push({
|
|
201
|
+
provider: id,
|
|
202
|
+
action: existing === undefined ? "injected" : "boosted",
|
|
203
|
+
detail: summaryOf(target),
|
|
204
|
+
});
|
|
205
|
+
}
|
|
206
|
+
}
|
|
207
|
+
return { patch, report };
|
|
208
|
+
}
|
|
209
|
+
|
|
210
|
+
/** Prefix vision-toolkit-style image-input variant routes register under. */
|
|
211
|
+
export const VARIANT_PROVIDER_PREFIX = "vision-toolkit-";
|
|
212
|
+
|
|
213
|
+
/**
|
|
214
|
+
* Whether one resolved retryPolicy actually covers the codes this plugin
|
|
215
|
+
* exists for. `mode: "always"` retries unconditionally and always qualifies.
|
|
216
|
+
* @param {object|undefined} policy - a resolved (registration-captured) policy.
|
|
217
|
+
* @param {string[]} [requiredCodes] - codes that must be retryable.
|
|
218
|
+
* @returns {boolean}
|
|
219
|
+
*/
|
|
220
|
+
export function isPolicyProtected(policy, requiredCodes = ["QUOTA"]) {
|
|
221
|
+
if (policy === undefined || policy === null) return false;
|
|
222
|
+
if (policy.mode === "always") return true;
|
|
223
|
+
const codes = Array.isArray(policy.retryableCodes) ? policy.retryableCodes : [];
|
|
224
|
+
return requiredCodes.every((code) => codes.includes(code));
|
|
225
|
+
}
|
|
226
|
+
|
|
227
|
+
/**
|
|
228
|
+
* Find registered variant routes (vision-toolkit image-input twins and any
|
|
229
|
+
* future adapter following the same `<prefix><upstream>` naming) whose captured
|
|
230
|
+
* retryPolicy still fails QUOTA fast. These routes live OUTSIDE the llm-pi-ai
|
|
231
|
+
* settings namespace — the plugin cannot inject a fix — so the caller should
|
|
232
|
+
* warn instead. Pure: providers in, findings out.
|
|
233
|
+
* @param {Array<{id: string, name?: string}>} providers - llm.listProviders() output.
|
|
234
|
+
* @param {(providerId: string) => object|undefined} readPolicy - bound llm.providerRetryPolicy.
|
|
235
|
+
* @returns {Array<{provider: string, name?: string}>}
|
|
236
|
+
*/
|
|
237
|
+
export function findUnprotectedVariants(providers, readPolicy) {
|
|
238
|
+
const findings = [];
|
|
239
|
+
for (const provider of providers ?? []) {
|
|
240
|
+
if (typeof provider?.id !== "string" || !provider.id.startsWith(VARIANT_PROVIDER_PREFIX)) continue;
|
|
241
|
+
let policy;
|
|
242
|
+
try {
|
|
243
|
+
policy = readPolicy(provider.id);
|
|
244
|
+
} catch {
|
|
245
|
+
continue; // route vanished mid-scan; the next adapters-updated sweep re-checks
|
|
246
|
+
}
|
|
247
|
+
if (!isPolicyProtected(policy)) {
|
|
248
|
+
findings.push({ provider: provider.id, ...(provider.name === undefined ? {} : { name: provider.name }) });
|
|
249
|
+
}
|
|
250
|
+
}
|
|
251
|
+
return findings;
|
|
252
|
+
}
|
|
253
|
+
|
|
254
|
+
function summaryOf(policy) {
|
|
255
|
+
const backoff = policy.backoff;
|
|
256
|
+
if (policy.mode === "always") return `mode=always backoff=${backoff.initialDelayMs}ms->${backoff.maxDelayMs}ms`;
|
|
257
|
+
return `mode=normal retries=${policy.maxRetries} backoff=${backoff.initialDelayMs}ms->${backoff.maxDelayMs}ms codes=${policy.retryableCodes.length}`;
|
|
258
|
+
}
|
package/package.json
ADDED
|
@@ -0,0 +1,58 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "dsh-retry-boost",
|
|
3
|
+
"version": "1.2.0",
|
|
4
|
+
"description": "让 DeepSeek Harness 自动重试商汤 429/QUOTA 等瞬时网关故障直到任务完成——启动时给所有 llm-pi-ai provider 热注入加固版 retryPolicy,运行中新增的 provider 也会自动补上,再也不用手动发「继续」。",
|
|
5
|
+
"keywords": [
|
|
6
|
+
"deepseek-harness",
|
|
7
|
+
"deepseek-harness-plugin",
|
|
8
|
+
"dsh",
|
|
9
|
+
"dsh-plugin",
|
|
10
|
+
"cordis-plugin",
|
|
11
|
+
"llm",
|
|
12
|
+
"retry",
|
|
13
|
+
"backoff",
|
|
14
|
+
"rate-limit",
|
|
15
|
+
"429",
|
|
16
|
+
"quota",
|
|
17
|
+
"resilience",
|
|
18
|
+
"reconnect",
|
|
19
|
+
"auto-retry"
|
|
20
|
+
],
|
|
21
|
+
"type": "module",
|
|
22
|
+
"engines": {
|
|
23
|
+
"node": ">=20"
|
|
24
|
+
},
|
|
25
|
+
"main": "lib/index.js",
|
|
26
|
+
"exports": {
|
|
27
|
+
".": "./lib/index.js",
|
|
28
|
+
"./package.json": "./package.json"
|
|
29
|
+
},
|
|
30
|
+
"files": [
|
|
31
|
+
"lib",
|
|
32
|
+
"cordis.patch.yml",
|
|
33
|
+
"README.md",
|
|
34
|
+
"README.en.md",
|
|
35
|
+
"LICENSE"
|
|
36
|
+
],
|
|
37
|
+
"dsh": {
|
|
38
|
+
"engines": {
|
|
39
|
+
"dsh": ">=0.1.1-rc.1"
|
|
40
|
+
},
|
|
41
|
+
"bundle": {
|
|
42
|
+
"patch": "./cordis.patch.yml"
|
|
43
|
+
}
|
|
44
|
+
},
|
|
45
|
+
"scripts": {
|
|
46
|
+
"test": "node --test",
|
|
47
|
+
"check": "node --check lib/index.js && node --check lib/policy.js"
|
|
48
|
+
},
|
|
49
|
+
"license": "MIT",
|
|
50
|
+
"repository": {
|
|
51
|
+
"type": "git",
|
|
52
|
+
"url": "git+https://github.com/hhb1028/dsh-retry-boost.git"
|
|
53
|
+
},
|
|
54
|
+
"bugs": {
|
|
55
|
+
"url": "https://github.com/hhb1028/dsh-retry-boost/issues"
|
|
56
|
+
},
|
|
57
|
+
"homepage": "https://github.com/hhb1028/dsh-retry-boost#readme"
|
|
58
|
+
}
|