@naced/llm-multi-rate-limiter 0.1.0 → 0.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +41 -0
- package/README.md +52 -7
- package/README.zh.md +50 -7
- package/lib/client.js +269 -4
- package/lib/config-resolve.js +259 -0
- package/lib/index.d.ts +91 -0
- package/lib/index.js +140 -42
- package/lib/strategies/sliding-window.js +27 -1
- package/lib/strategies/token-bucket.js +30 -2
- package/lib/types/config.js +42 -1
- package/package.json +28 -6
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,46 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## [0.1.2] - 2026-10-28
|
|
4
|
+
|
|
5
|
+
Concurrency accounting fix, provider-wide tiers, wildcard keys, adaptive 429 backoff, and a tested toolchain.
|
|
6
|
+
|
|
7
|
+
### Fixed
|
|
8
|
+
|
|
9
|
+
- **Queue drain over-granted concurrency**: a waiter released by `releaseSlot()` counted against neither `#concurrent` nor any reservation, so one release could admit the entire queue at once. Both strategies now reserve a slot when they grant one, and `#drain()` compares `#concurrent + #reserved` against `maxConcurrent`.
|
|
10
|
+
- **Timeout and drain could settle the same waiter twice**: `acquire()` now guards its settlement with a `settled` flag.
|
|
11
|
+
- **Retry storm after a queue timeout**: a queue timeout now fails with `RATE_LIMIT_QUEUE_TIMEOUT` instead of the retryable `RATE_LIMIT`, so `dsh-llm-retry` no longer replays the same saturated model six times. Reject mode uses `RATE_LIMIT_REJECTED`.
|
|
12
|
+
- **Limiter instances leaked**: idle limiters with no override are swept every minute (10-minute idle window) once they have no concurrency, reservations or waiters.
|
|
13
|
+
|
|
14
|
+
### Added
|
|
15
|
+
|
|
16
|
+
- **Provider tier** (`providers.<provider>`): one cap for every model of a provider, below per-model overrides and above `defaults`.
|
|
17
|
+
- **Wildcard model keys** (`openai/*`, `*/*`): `*` compiles to `[^/]*`, so it stays inside one path segment and cannot silently cover another provider. The most specific pattern wins.
|
|
18
|
+
- **Adaptive 429 backoff** (`cooldown`): when the provider answers 429, that model's limits are scaled by `factor` for `cooldownMs` (or the response's `Retry-After`, if longer) and recover after `recoverAfter` consecutive successes. Counted in the status snapshot as `effectiveMaxRpm` / `cooling`.
|
|
19
|
+
- **Settings page**: model search filter, delete confirmation, and whole-config JSON import/export.
|
|
20
|
+
- **Tests**: `node --test` suites for tier resolution, wildcard matching, cooldown bookkeeping and both strategies — including a regression test that one release admits exactly one waiter.
|
|
21
|
+
- **Type declarations** (`lib/index.d.ts`) and an **ESLint** flat config; `npm run check` runs syntax, lint and tests.
|
|
22
|
+
- CI now runs the full check on both Ubuntu and Windows.
|
|
23
|
+
|
|
24
|
+
### Changed
|
|
25
|
+
|
|
26
|
+
- `lib/config-resolve.js` holds tier resolution and the cooldown tracker as pure functions, shared by the host entry and the tests.
|
|
27
|
+
- The status snapshot reports `reserved` slots per model and `throttledByProvider` in totals.
|
|
28
|
+
|
|
29
|
+
## [0.1.1] - 2026-10-28
|
|
30
|
+
|
|
31
|
+
Repository and release plumbing; no behavioural change to the limiter.
|
|
32
|
+
|
|
33
|
+
### Added
|
|
34
|
+
|
|
35
|
+
- `repository`, `homepage`, `bugs` and `author` metadata, so the npm page links back to GitHub.
|
|
36
|
+
- `publishConfig.access: "public"` — a plain `npm publish` on this scoped package no longer needs `--access public`.
|
|
37
|
+
- GitHub Actions: [`ci.yml`](.github/workflows/ci.yml) runs the syntax check and `npm pack --dry-run` on every push and PR; [`publish.yml`](.github/workflows/publish.yml) publishes on `v*` tags through npm Trusted Publishing (OIDC) with provenance, and creates the GitHub release. No npm token is stored in the repository.
|
|
38
|
+
- `prepublishOnly` runs the same syntax check, so a broken `lib/` can never be published.
|
|
39
|
+
|
|
40
|
+
### Changed
|
|
41
|
+
|
|
42
|
+
- README (both languages) gained npm/CI badges, install-from-registry instructions, and development/release sections.
|
|
43
|
+
|
|
3
44
|
## [0.1.0] - 2026-10-26
|
|
4
45
|
|
|
5
46
|
Fork of `@leaf233/dsh-llm-rate-limiter` 0.3.0, retargeted at the DSH 0.2.1 client contract.
|
package/README.md
CHANGED
|
@@ -1,15 +1,24 @@
|
|
|
1
1
|
# @naced/llm-multi-rate-limiter
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
[](https://www.npmjs.com/package/@naced/llm-multi-rate-limiter)
|
|
4
|
+
[](https://www.npmjs.com/package/@naced/llm-multi-rate-limiter)
|
|
5
|
+
[](LICENSE)
|
|
6
|
+
[](https://github.com/NacedWang/dsh-llm-multi-rate-limiter/actions/workflows/ci.yml)
|
|
7
|
+
|
|
8
|
+
Per-`provider/model` LLM rate limiting for DeepSeek Harness, with an editable **Settings** page (Settings → Multi-model limits), a sidebar entry and a live status panel.
|
|
4
9
|
|
|
5
10
|
Forked from [`@leaf233/dsh-llm-rate-limiter`](https://github.com/Leafyezi233/dsh-llm-rate-limiter) 0.3.0 (MIT, © the upstream contributors). The configuration surface moved from the "built-in plugin config" card to a `settings.section` page and a sidebar entry; the card entry is gone.
|
|
6
11
|
|
|
7
12
|
## Features
|
|
8
13
|
|
|
9
14
|
- **Two strategies**: token bucket (bursts allowed) or sliding window (strict N per minute).
|
|
15
|
+
- **Three configuration tiers**: `models` (exact or wildcard key) > `providers` (one cap per provider) > `defaults`.
|
|
16
|
+
- **Wildcard keys**: `openai/*` covers every model of that provider without its own limit. `*` stays inside one path segment and never crosses `/`.
|
|
10
17
|
- **Per-model overrides**, keyed `provider/model`, e.g. `openai/gpt-5.1`.
|
|
18
|
+
- **Adaptive 429 backoff**: when the provider answers 429, that model slows down by `factor` for `cooldownMs` and recovers after `recoverAfter` consecutive successes.
|
|
11
19
|
- **Queue or reject** throttled calls (`onThrottled`, bounded by `maxQueueWaitMs`).
|
|
12
20
|
- **Live panel**: polls the `/llm-multi-rate-limiter` status channel once per second — per-model token/window usage, granted calls, average wait, recent events.
|
|
21
|
+
- **Settings page**: model search, delete confirmation, and whole-config JSON import/export.
|
|
13
22
|
|
|
14
23
|
## Configuration
|
|
15
24
|
|
|
@@ -26,32 +35,48 @@ Forked from [`@leaf233/dsh-llm-rate-limiter`](https://github.com/Leafyezi233/dsh
|
|
|
26
35
|
maxRpm: 60
|
|
27
36
|
burstSize: 10
|
|
28
37
|
refillRate: 1
|
|
38
|
+
providers:
|
|
39
|
+
openai:
|
|
40
|
+
maxRpm: 30
|
|
41
|
+
maxConcurrent: 3
|
|
29
42
|
models:
|
|
30
43
|
openai/gpt-5.1:
|
|
31
44
|
maxRpm: 20
|
|
32
45
|
maxConcurrent: 2
|
|
33
|
-
|
|
34
|
-
maxRpm:
|
|
35
|
-
|
|
46
|
+
"azure/*":
|
|
47
|
+
maxRpm: 40
|
|
48
|
+
cooldown:
|
|
49
|
+
enabled: true
|
|
50
|
+
cooldownMs: 30000
|
|
51
|
+
factor: 0.5
|
|
52
|
+
recoverAfter: 3
|
|
36
53
|
onThrottled: queue
|
|
37
54
|
maxQueueWaitMs: 240000
|
|
38
55
|
```
|
|
39
56
|
|
|
40
57
|
- The row `id` is also the settings namespace (`dsh-settings` keys forms by `entry.options.id`), so keep it in step with the row.
|
|
41
|
-
- `models
|
|
58
|
+
- Precedence: exact `models` key > most specific wildcard in `models` > `providers[<provider>]` > `defaults`.
|
|
59
|
+
- `models[<key>]` only needs the fields you override; the rest is inherited tier by tier.
|
|
42
60
|
- Under the token bucket, an omitted `refillRate` (tokens/second) is derived from `maxRpm / 60`.
|
|
61
|
+
- A wildcard `*` matches within one path segment only, so `openai/*` cannot accidentally cover another provider; use `*/*` to cover everything.
|
|
43
62
|
|
|
44
63
|
Edits made in the Settings page are written back to `cordis.patch.yml` immediately and hot-reload through `dsh-hmr`.
|
|
45
64
|
|
|
46
65
|
## Install
|
|
47
66
|
|
|
48
|
-
|
|
67
|
+
From the npm registry (recommended):
|
|
68
|
+
|
|
69
|
+
1. Add the dependency to your DSH profile's `package.json` (`~/.dsh/profiles/desktop/package.json`):
|
|
49
70
|
```json
|
|
50
|
-
"@naced/llm-multi-rate-limiter": "
|
|
71
|
+
"@naced/llm-multi-rate-limiter": "^0.1.2"
|
|
51
72
|
```
|
|
73
|
+
then install it in the profile directory (`pnpm install`, or the DSH plugin manager's install action).
|
|
52
74
|
2. Add `"@naced/llm-multi-rate-limiter"` to `dsh.profile.bundles`.
|
|
53
75
|
3. Append the `cordis.patch.yml` row above, then restart DSH.
|
|
54
76
|
|
|
77
|
+
From a local checkout (development): point the dependency at the folder instead of a version —
|
|
78
|
+
`"@naced/llm-multi-rate-limiter": "file:./packages/llm-multi-rate-limiter"` — and keep the same `bundles` entry.
|
|
79
|
+
|
|
55
80
|
## Security
|
|
56
81
|
|
|
57
82
|
This plugin intercepts the `llm/stream` waterfall, so it sits in front of every model call. What it actually does with that position:
|
|
@@ -63,6 +88,26 @@ This plugin intercepts the `llm/stream` waterfall, so it sits in front of every
|
|
|
63
88
|
|
|
64
89
|
If the limiter itself throws, the call is passed through untouched rather than failing, so a bug here cannot take the assistant offline.
|
|
65
90
|
|
|
91
|
+
## Development
|
|
92
|
+
|
|
93
|
+
```bash
|
|
94
|
+
git clone https://github.com/NacedWang/dsh-llm-multi-rate-limiter.git
|
|
95
|
+
cd dsh-llm-multi-rate-limiter
|
|
96
|
+
npm install
|
|
97
|
+
npm run check # syntax check + ESLint + node:test
|
|
98
|
+
npm pack --dry-run
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
There is no build step: `lib/` is hand-written ESM plus a browser bundle, shipped as-is so the published artifact matches what runs.
|
|
102
|
+
|
|
103
|
+
## Releasing
|
|
104
|
+
|
|
105
|
+
1. Bump `version` and add a `CHANGELOG.md` entry.
|
|
106
|
+
2. `git commit -am "chore: release vX.Y.Z" && git tag vX.Y.Z && git push --follow-tags`
|
|
107
|
+
3. The [`publish`](.github/workflows/publish.yml) workflow publishes to npm via **Trusted Publishing (OIDC)** — no `NPM_TOKEN` secret exists — and creates the matching GitHub release.
|
|
108
|
+
|
|
109
|
+
Because publishing is tied to this repository, the workflow filename in the npm Trusted Publisher settings must stay `publish.yml`.
|
|
110
|
+
|
|
66
111
|
## License
|
|
67
112
|
|
|
68
113
|
MIT. Based on `@leaf233/dsh-llm-rate-limiter`; see [LICENSE](LICENSE) for both copyright lines.
|
package/README.zh.md
CHANGED
|
@@ -1,15 +1,24 @@
|
|
|
1
1
|
# @naced/llm-multi-rate-limiter
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
[](https://www.npmjs.com/package/@naced/llm-multi-rate-limiter)
|
|
4
|
+
[](https://www.npmjs.com/package/@naced/llm-multi-rate-limiter)
|
|
5
|
+
[](LICENSE)
|
|
6
|
+
[](https://github.com/NacedWang/dsh-llm-multi-rate-limiter/actions/workflows/ci.yml)
|
|
7
|
+
|
|
8
|
+
按 **provider/model** 限制 LLM 请求频率的 DeepSeek Harness 插件，带一个可编辑的「设置」页（Settings → 多模型限速）、侧边栏入口，以及实时状态面板。
|
|
4
9
|
|
|
5
10
|
本包 fork 自 [`@leaf233/dsh-llm-rate-limiter`](https://github.com/Leafyezi233/dsh-llm-rate-limiter) 0.3.0（MIT，版权归上游作者），主要改动是把配置界面从「内置插件配置」卡片改为 `settings.section` 设置页 + 侧边栏入口，并移除了卡片入口。
|
|
6
11
|
|
|
7
12
|
## 功能
|
|
8
13
|
|
|
9
14
|
- **两种限速策略**：令牌桶 `token-bucket`（允许突发）／滑动窗口 `sliding-window`（严格每分钟 N 次）。
|
|
15
|
+
- **三层配置**：`models`（精确或通配符 key）> `providers`（整个 provider 一个上限）> `defaults`。
|
|
16
|
+
- **通配符 key**：`openai/*` 覆盖该 provider 下所有没有单独配置的模型，`*` 只在单段内匹配、不跨 `/`。
|
|
10
17
|
- **按模型独立配置**：key 为 `provider/model`，例如 `openai/gpt-5.1`。
|
|
18
|
+
- **429 自适应降速**：provider 返回 429 时，该模型自动按 `factor` 降速并保持 `cooldownMs`，连续 `recoverAfter` 次成功后恢复。
|
|
11
19
|
- **排队或拒绝**：`onThrottled: queue` 时请求等待空位（受 `maxQueueWaitMs` 约束），`reject` 时立即返回错误。
|
|
12
20
|
- **实时面板**：每秒拉取 `/llm-multi-rate-limiter` 状态通道，显示各模型当前令牌/窗口占用、累计放行、平均等待，以及最近事件。
|
|
21
|
+
- **设置页**：模型搜索过滤、删除二次确认、整份配置的 JSON 导入/导出。
|
|
13
22
|
|
|
14
23
|
## 配置
|
|
15
24
|
|
|
@@ -26,32 +35,46 @@
|
|
|
26
35
|
maxRpm: 60
|
|
27
36
|
burstSize: 10
|
|
28
37
|
refillRate: 1
|
|
38
|
+
providers:
|
|
39
|
+
openai:
|
|
40
|
+
maxRpm: 30
|
|
41
|
+
maxConcurrent: 3
|
|
29
42
|
models:
|
|
30
43
|
openai/gpt-5.1:
|
|
31
44
|
maxRpm: 20
|
|
32
45
|
maxConcurrent: 2
|
|
33
|
-
|
|
34
|
-
maxRpm:
|
|
35
|
-
|
|
46
|
+
"azure/*":
|
|
47
|
+
maxRpm: 40
|
|
48
|
+
cooldown:
|
|
49
|
+
enabled: true
|
|
50
|
+
cooldownMs: 30000
|
|
51
|
+
factor: 0.5
|
|
52
|
+
recoverAfter: 3
|
|
36
53
|
onThrottled: queue
|
|
37
54
|
maxQueueWaitMs: 240000
|
|
38
55
|
```
|
|
39
56
|
|
|
40
57
|
- 行 `id` 同时是设置命名空间（`dsh-settings` 按 `entry.options.id` 建表单），改名需与 `cordis.patch.yml` 一致。
|
|
41
|
-
- `models[<
|
|
58
|
+
- 优先级：`models` 精确 key > `models` 里最具体的通配符 > `providers[<provider>]` > `defaults`。
|
|
59
|
+
- `models[<key>]` 只写需要覆盖的字段，其余逐层继承。
|
|
42
60
|
- 令牌桶下 `refillRate`（个/秒）若省略，会按 `maxRpm / 60` 推导。
|
|
43
61
|
|
|
44
62
|
设置页里改任何字段都会立即写回 `cordis.patch.yml` 并热生效（`dsh-hmr` 监听 profile patch）。
|
|
45
63
|
|
|
46
64
|
## 安装
|
|
47
65
|
|
|
48
|
-
|
|
66
|
+
走 npm registry（推荐）：
|
|
67
|
+
|
|
68
|
+
1. 在 DSH profile 的 `package.json`（`~/.dsh/profiles/desktop/package.json`）里加依赖：
|
|
49
69
|
```json
|
|
50
|
-
"@naced/llm-multi-rate-limiter": "
|
|
70
|
+
"@naced/llm-multi-rate-limiter": "^0.1.2"
|
|
51
71
|
```
|
|
72
|
+
然后在 profile 目录执行 `pnpm install`（或用 DSH 插件管理器的安装动作）。
|
|
52
73
|
2. `dsh.profile.bundles` 列表中加入 `"@naced/llm-multi-rate-limiter"`。
|
|
53
74
|
3. 在 `cordis.patch.yml` 加上面那段配置，重启 DSH。
|
|
54
75
|
|
|
76
|
+
本地开发时把依赖改成指向目录：`"@naced/llm-multi-rate-limiter": "file:./packages/llm-multi-rate-limiter"`，`bundles` 一项保持不变。
|
|
77
|
+
|
|
55
78
|
## 卸载
|
|
56
79
|
|
|
57
80
|
删除 `cordis.patch.yml` 中该行与 `bundles` 里的包名，再删 `node_modules/@naced/llm-multi-rate-limiter` 即可。
|
|
@@ -67,6 +90,26 @@
|
|
|
67
90
|
|
|
68
91
|
限速器自身抛错时会原样放行请求，而不是让调用失败——这里的 bug 不会把助手整体弄挂。
|
|
69
92
|
|
|
93
|
+
## 开发
|
|
94
|
+
|
|
95
|
+
```bash
|
|
96
|
+
git clone https://github.com/NacedWang/dsh-llm-multi-rate-limiter.git
|
|
97
|
+
cd dsh-llm-multi-rate-limiter
|
|
98
|
+
npm install
|
|
99
|
+
npm run check # 语法检查 + ESLint + node:test
|
|
100
|
+
npm pack --dry-run
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
没有编译步骤：`lib/` 是手写的 ESM 加一份浏览器 bundle，原样发布，所以线上跑的就是这份代码。
|
|
104
|
+
|
|
105
|
+
## 发版
|
|
106
|
+
|
|
107
|
+
1. 改 `version`，在 `CHANGELOG.md` 补一条。
|
|
108
|
+
2. `git commit -am "chore: release vX.Y.Z" && git tag vX.Y.Z && git push --follow-tags`
|
|
109
|
+
3. [`publish`](.github/workflows/publish.yml) 工作流通过 **Trusted Publishing (OIDC)** 发布到 npm —— 仓库里不存在 `NPM_TOKEN` 密钥 —— 并自动创建对应的 GitHub Release。
|
|
110
|
+
|
|
111
|
+
因为发布权限绑定了本仓库，npm 的 Trusted Publisher 设置里工作流文件名必须保持 `publish.yml`。
|
|
112
|
+
|
|
70
113
|
## License
|
|
71
114
|
|
|
72
115
|
MIT。基于 `@leaf233/dsh-llm-rate-limiter`，双版权行见 [LICENSE](LICENSE)。
|
package/lib/client.js
CHANGED
|
@@ -361,6 +361,11 @@ window.__ModuleLoader__.load({
|
|
|
361
361
|
const cfg = snap !== undefined && snap.status === "ready" && snap.value ? snap.value : {};
|
|
362
362
|
const writable = snap === undefined ? true : snap.writable !== false;
|
|
363
363
|
const [draftKey, setDraftKey] = react.useState("");
|
|
364
|
+
const [draftProvider, setDraftProvider] = react.useState("");
|
|
365
|
+
const [query, setQuery] = react.useState("");
|
|
366
|
+
const [confirmKey, setConfirmKey] = react.useState("");
|
|
367
|
+
const [importError, setImportError] = react.useState("");
|
|
368
|
+
const fileInputRef = react.useRef(null);
|
|
364
369
|
|
|
365
370
|
/** Queue one atomic namespace mutation; failures leave the form as-is. */
|
|
366
371
|
function push(ops) {
|
|
@@ -383,15 +388,107 @@ window.__ModuleLoader__.load({
|
|
|
383
388
|
setPath(path, n);
|
|
384
389
|
}
|
|
385
390
|
}
|
|
391
|
+
/** Provider tier: same shape, one slashless key. */
|
|
392
|
+
function setProvider(key, field, raw) {
|
|
393
|
+
const path = ["providers", key, field];
|
|
394
|
+
if (raw === "") unsetPath(path);
|
|
395
|
+
else {
|
|
396
|
+
const n = Number(raw);
|
|
397
|
+
if (!Number.isFinite(n)) return;
|
|
398
|
+
setPath(path, n);
|
|
399
|
+
}
|
|
400
|
+
}
|
|
401
|
+
function addProvider() {
|
|
402
|
+
const key = draftProvider.trim();
|
|
403
|
+
if (!key || /[\s/]/.test(key) || providerKeys.includes(key)) {
|
|
404
|
+
setDraftProvider("");
|
|
405
|
+
return;
|
|
406
|
+
}
|
|
407
|
+
setPath(["providers", key], {});
|
|
408
|
+
setDraftProvider("");
|
|
409
|
+
}
|
|
410
|
+
function removeProvider(key) {
|
|
411
|
+
unsetPath(["providers", key]);
|
|
412
|
+
}
|
|
413
|
+
const setCooldown = (field, value) => setPath(["cooldown", field], value);
|
|
414
|
+
/** Downloads the whole namespace value so it can be re-imported. */
|
|
415
|
+
function exportConfig() {
|
|
416
|
+
const payload = {
|
|
417
|
+
$schema: "llm-multi-rate-limiter/config@1",
|
|
418
|
+
exportedAt: new Date().toISOString(),
|
|
419
|
+
config: cfg,
|
|
420
|
+
};
|
|
421
|
+
const blob = new Blob([JSON.stringify(payload, null, 2)], { type: "application/json" });
|
|
422
|
+
const url = URL.createObjectURL(blob);
|
|
423
|
+
const a = document.createElement("a");
|
|
424
|
+
a.href = url;
|
|
425
|
+
a.download = "llm-multi-rate-limiter.json";
|
|
426
|
+
document.body.appendChild(a);
|
|
427
|
+
a.click();
|
|
428
|
+
setTimeout(() => { a.remove(); URL.revokeObjectURL(url); }, 0);
|
|
429
|
+
setImportError("");
|
|
430
|
+
}
|
|
431
|
+
/** Applies an imported document as one atomic namespace mutation. */
|
|
432
|
+
function applyImport(text) {
|
|
433
|
+
let parsed;
|
|
434
|
+
try {
|
|
435
|
+
parsed = JSON.parse(text);
|
|
436
|
+
} catch {
|
|
437
|
+
setImportError(t("importInvalidJson"));
|
|
438
|
+
return;
|
|
439
|
+
}
|
|
440
|
+
const imported = parsed !== null && typeof parsed === "object"
|
|
441
|
+
&& parsed.config !== null && typeof parsed.config === "object"
|
|
442
|
+
? parsed.config
|
|
443
|
+
: parsed;
|
|
444
|
+
if (imported === null || typeof imported !== "object" || Array.isArray(imported)) {
|
|
445
|
+
setImportError(t("importInvalidShape"));
|
|
446
|
+
return;
|
|
447
|
+
}
|
|
448
|
+
if (!writable) {
|
|
449
|
+
setImportError(t("importReadOnly"));
|
|
450
|
+
return;
|
|
451
|
+
}
|
|
452
|
+
const ops = [];
|
|
453
|
+
for (const field of ["enabled", "strategy", "defaults", "providers", "models", "cooldown", "onThrottled", "maxQueueWaitMs"]) {
|
|
454
|
+
if (Object.prototype.hasOwnProperty.call(imported, field)) {
|
|
455
|
+
ops.push({ op: "set", path: [field], value: imported[field] });
|
|
456
|
+
}
|
|
457
|
+
}
|
|
458
|
+
if (ops.length === 0) {
|
|
459
|
+
setImportError(t("importEmpty"));
|
|
460
|
+
return;
|
|
461
|
+
}
|
|
462
|
+
setImportError("");
|
|
463
|
+
push(ops);
|
|
464
|
+
}
|
|
465
|
+
async function onPickFile(event) {
|
|
466
|
+
const file = event.target.files && event.target.files[0];
|
|
467
|
+
event.target.value = "";
|
|
468
|
+
if (!file) return;
|
|
469
|
+
try {
|
|
470
|
+
applyImport(await file.text());
|
|
471
|
+
} catch {
|
|
472
|
+
setImportError(t("importUnreadable"));
|
|
473
|
+
}
|
|
474
|
+
}
|
|
386
475
|
|
|
387
|
-
const
|
|
476
|
+
const providerKeys = Object.keys(cfg.providers ?? {}).sort();
|
|
477
|
+
const allModelKeys = Object.keys(cfg.models ?? {}).sort();
|
|
478
|
+
const needle = query.trim().toLowerCase();
|
|
479
|
+
// The filter is presentation only: it never hides a stored limit from
|
|
480
|
+
// `mutate`, it just narrows what this render lists.
|
|
481
|
+
const modelKeys = needle === ""
|
|
482
|
+
? allModelKeys
|
|
483
|
+
: allModelKeys.filter((key) => key.toLowerCase().includes(needle));
|
|
484
|
+
const cooldown = cfg.cooldown ?? {};
|
|
388
485
|
const strategy = cfg.strategy ?? "token-bucket";
|
|
389
486
|
const bucket = strategy !== "sliding-window";
|
|
390
487
|
const defaults = cfg.defaults ?? {};
|
|
391
488
|
|
|
392
489
|
function addModel() {
|
|
393
490
|
const key = draftKey.trim();
|
|
394
|
-
if (!key || !/^[^/\s]+\/.+$/.test(key) ||
|
|
491
|
+
if (!key || !/^[^/\s]+\/.+$/.test(key) || allModelKeys.includes(key)) {
|
|
395
492
|
setDraftKey("");
|
|
396
493
|
return;
|
|
397
494
|
}
|
|
@@ -466,8 +563,23 @@ window.__ModuleLoader__.load({
|
|
|
466
563
|
jsxs("div", { className: "rlr-section", children: [
|
|
467
564
|
jsx("div", { className: "rlr-sectHead", children: t("models") }),
|
|
468
565
|
jsx("div", { className: "rlr-status", children: t("modelsHint") }),
|
|
469
|
-
|
|
566
|
+
allModelKeys.length > 6
|
|
567
|
+
? jsxs("div", { className: "rlr-row", children: [
|
|
568
|
+
jsx("input", {
|
|
569
|
+
className: "rlr-input",
|
|
570
|
+
style: { width: 240 },
|
|
571
|
+
type: "search",
|
|
572
|
+
placeholder: t("searchPlaceholder"),
|
|
573
|
+
value: query,
|
|
574
|
+
onChange: (e) => setQuery(e.target.value),
|
|
575
|
+
}),
|
|
576
|
+
jsx("span", { className: "rlr-modelMeta", children: modelKeys.length + " / " + allModelKeys.length }),
|
|
577
|
+
] })
|
|
578
|
+
: null,
|
|
579
|
+
allModelKeys.length === 0
|
|
470
580
|
? jsx("p", { className: "rlr-status", children: t("noModels") })
|
|
581
|
+
: modelKeys.length === 0
|
|
582
|
+
? jsx("p", { className: "rlr-status", children: t("noMatches") })
|
|
471
583
|
: modelKeys.map((key) => {
|
|
472
584
|
const o = cfg.models?.[key] ?? {};
|
|
473
585
|
const off = o.enabled === false;
|
|
@@ -499,7 +611,23 @@ window.__ModuleLoader__.load({
|
|
|
499
611
|
})
|
|
500
612
|
: null,
|
|
501
613
|
jsx("button", { className: "rlr-btnSec", disabled: !writable, onClick: () => toggleModel(key), children: off ? t("enable") : t("disable") }),
|
|
502
|
-
|
|
614
|
+
confirmKey === key
|
|
615
|
+
? jsxs("span", { style: { display: "flex", gap: 4, alignItems: "center" }, children: [
|
|
616
|
+
jsx("button", {
|
|
617
|
+
className: "rlr-danger",
|
|
618
|
+
disabled: !writable,
|
|
619
|
+
onClick: () => { removeModel(key); setConfirmKey(""); },
|
|
620
|
+
children: t("confirmDelete"),
|
|
621
|
+
}),
|
|
622
|
+
jsx("button", { className: "rlr-btnSec", onClick: () => setConfirmKey(""), children: t("cancel") }),
|
|
623
|
+
] })
|
|
624
|
+
: jsx("button", {
|
|
625
|
+
className: "rlr-danger",
|
|
626
|
+
disabled: !writable,
|
|
627
|
+
onClick: () => setConfirmKey(key),
|
|
628
|
+
title: t("remove"),
|
|
629
|
+
children: "✕",
|
|
630
|
+
}),
|
|
503
631
|
] }),
|
|
504
632
|
] });
|
|
505
633
|
}),
|
|
@@ -517,6 +645,93 @@ window.__ModuleLoader__.load({
|
|
|
517
645
|
] }),
|
|
518
646
|
] }),
|
|
519
647
|
|
|
648
|
+
/* ── provider 级配置 ── */
|
|
649
|
+
jsxs("div", { className: "rlr-section", children: [
|
|
650
|
+
jsx("div", { className: "rlr-sectHead", children: t("providers") }),
|
|
651
|
+
jsx("div", { className: "rlr-status", children: t("providersHint") }),
|
|
652
|
+
providerKeys.length === 0
|
|
653
|
+
? jsx("p", { className: "rlr-status", children: t("noProviders") })
|
|
654
|
+
: providerKeys.map((key) => {
|
|
655
|
+
const o = cfg.providers?.[key] ?? {};
|
|
656
|
+
return jsxs("div", { className: "rlr-modelRow", key, children: [
|
|
657
|
+
jsxs("div", { style: { display: "flex", flexDirection: "column", gap: 2, flex: 1, minWidth: 0 }, children: [
|
|
658
|
+
jsx("span", { className: "rlr-modelName", children: key + "/*" }),
|
|
659
|
+
jsx("span", { className: "rlr-modelMeta", children: t("providerScope") }),
|
|
660
|
+
] }),
|
|
661
|
+
jsxs("div", { style: { display: "flex", alignItems: "center", gap: 6 }, children: [
|
|
662
|
+
jsx("input", {
|
|
663
|
+
className: "rlr-input", type: "number", min: 1, style: { width: 64 }, title: t("maxRpm"),
|
|
664
|
+
disabled: !writable, value: o.maxRpm ?? "", placeholder: "rpm",
|
|
665
|
+
onChange: (e) => setProvider(key, "maxRpm", e.target.value),
|
|
666
|
+
}),
|
|
667
|
+
jsx("input", {
|
|
668
|
+
className: "rlr-input", type: "number", min: 1, style: { width: 56 }, title: t("maxConcurrent"),
|
|
669
|
+
disabled: !writable, value: o.maxConcurrent ?? "", placeholder: t("concurrent"),
|
|
670
|
+
onChange: (e) => setProvider(key, "maxConcurrent", e.target.value),
|
|
671
|
+
}),
|
|
672
|
+
jsx("button", { className: "rlr-danger", disabled: !writable, onClick: () => removeProvider(key), title: t("remove"), children: "✕" }),
|
|
673
|
+
] }),
|
|
674
|
+
] });
|
|
675
|
+
}),
|
|
676
|
+
jsxs("div", { className: "rlr-row", children: [
|
|
677
|
+
jsx("input", {
|
|
678
|
+
className: "rlr-input",
|
|
679
|
+
style: { width: 200 },
|
|
680
|
+
placeholder: t("addProviderPlaceholder"),
|
|
681
|
+
value: draftProvider,
|
|
682
|
+
disabled: !writable,
|
|
683
|
+
onChange: (e) => setDraftProvider(e.target.value),
|
|
684
|
+
onKeyDown: (e) => { if (e.key === "Enter") addProvider(); },
|
|
685
|
+
}),
|
|
686
|
+
jsx("button", { className: "rlr-btn", disabled: !writable, onClick: addProvider, children: t("addProvider") }),
|
|
687
|
+
] }),
|
|
688
|
+
] }),
|
|
689
|
+
|
|
690
|
+
/* ── 429 自适应降速 ── */
|
|
691
|
+
jsxs("div", { className: "rlr-section", children: [
|
|
692
|
+
jsx("div", { className: "rlr-sectHead", children: t("cooldown") }),
|
|
693
|
+
jsx("div", { className: "rlr-status", children: t("cooldownHint") }),
|
|
694
|
+
jsxs("div", { className: "rlr-row", children: [
|
|
695
|
+
jsxs("label", { style: { display: "flex", alignItems: "center", gap: 6, cursor: "pointer" }, children: [
|
|
696
|
+
jsx("input", {
|
|
697
|
+
type: "checkbox",
|
|
698
|
+
checked: cooldown.enabled ?? true,
|
|
699
|
+
disabled: !writable,
|
|
700
|
+
onChange: (e) => setCooldown("enabled", e.target.checked),
|
|
701
|
+
}),
|
|
702
|
+
t("cooldownEnabled"),
|
|
703
|
+
] }),
|
|
704
|
+
] }),
|
|
705
|
+
jsx(NumField, { label: t("cooldownMs"), value: cooldown.cooldownMs, placeholder: "30000", min: 1000, step: 1000, disabled: !writable, onChange: (v) => setCooldown("cooldownMs", Number(v) || 30000) }),
|
|
706
|
+
jsx(NumField, { label: t("cooldownFactor"), value: cooldown.factor, placeholder: "0.5", min: 0.05, step: 0.05, disabled: !writable, onChange: (v) => setCooldown("factor", Number(v) || 0.5) }),
|
|
707
|
+
jsx(NumField, { label: t("recoverAfter"), value: cooldown.recoverAfter, placeholder: "3", min: 1, step: 1, disabled: !writable, onChange: (v) => setCooldown("recoverAfter", Number(v) || 3) }),
|
|
708
|
+
] }),
|
|
709
|
+
|
|
710
|
+
/* ── 配置导入 / 导出 ── */
|
|
711
|
+
jsxs("div", { className: "rlr-section", children: [
|
|
712
|
+
jsx("div", { className: "rlr-sectHead", children: t("preset") }),
|
|
713
|
+
jsx("div", { className: "rlr-status", children: t("presetHint") }),
|
|
714
|
+
jsxs("div", { className: "rlr-row", children: [
|
|
715
|
+
jsx("button", { className: "rlr-btn", onClick: exportConfig, children: t("exportConfig") }),
|
|
716
|
+
jsx("button", {
|
|
717
|
+
className: "rlr-btn",
|
|
718
|
+
disabled: !writable,
|
|
719
|
+
onClick: () => { if (fileInputRef.current) fileInputRef.current.click(); },
|
|
720
|
+
children: t("importConfig"),
|
|
721
|
+
}),
|
|
722
|
+
jsx("input", {
|
|
723
|
+
ref: fileInputRef,
|
|
724
|
+
type: "file",
|
|
725
|
+
accept: "application/json,.json",
|
|
726
|
+
style: { display: "none" },
|
|
727
|
+
onChange: onPickFile,
|
|
728
|
+
}),
|
|
729
|
+
] }),
|
|
730
|
+
importError !== ""
|
|
731
|
+
? jsx("div", { className: "rlr-status", style: { color: "var(--fg-danger, #d9534f)" }, children: importError })
|
|
732
|
+
: null,
|
|
733
|
+
] }),
|
|
734
|
+
|
|
520
735
|
/* ── 被限速时的行为 ── */
|
|
521
736
|
jsxs("div", { className: "rlr-section", children: [
|
|
522
737
|
jsx("div", { className: "rlr-sectHead", children: t("behavior") }),
|
|
@@ -613,6 +828,31 @@ window.__ModuleLoader__.load({
|
|
|
613
828
|
reject: "拒绝请求",
|
|
614
829
|
maxQueueWait: "最大排队等待 (ms)",
|
|
615
830
|
saveHint: "改动即写入 profile 的 cordis.patch.yml，无需重启。",
|
|
831
|
+
searchPlaceholder: "搜索模型…",
|
|
832
|
+
noMatches: "没有匹配的模型。",
|
|
833
|
+
confirmDelete: "确认删除",
|
|
834
|
+
cancel: "取消",
|
|
835
|
+
providers: "Provider 级配置",
|
|
836
|
+
providersHint: "作用于该 provider 下所有未单独配置的模型；优先级低于模型专属配置。",
|
|
837
|
+
noProviders: "未配置 provider 级限速。",
|
|
838
|
+
addProvider: "+ 添加 provider",
|
|
839
|
+
addProviderPlaceholder: "例如 openai",
|
|
840
|
+
providerScope: "作用于该 provider 的全部模型",
|
|
841
|
+
cooldown: "429 自适应降速",
|
|
842
|
+
cooldownHint: "provider 返回 429 时临时调低该模型的速率，连续成功后再恢复。",
|
|
843
|
+
cooldownEnabled: "遇到 429 时自动降速",
|
|
844
|
+
cooldownMs: "降速持续时间 (ms)",
|
|
845
|
+
cooldownFactor: "降速系数 (0.05–0.9)",
|
|
846
|
+
recoverAfter: "恢复所需的连续成功次数",
|
|
847
|
+
preset: "配置导入 / 导出",
|
|
848
|
+
presetHint: "导出的 JSON 可在另一台机器导入，覆盖当前全部设置。",
|
|
849
|
+
exportConfig: "导出 JSON",
|
|
850
|
+
importConfig: "导入 JSON",
|
|
851
|
+
importInvalidJson: "导入失败：不是合法 JSON。",
|
|
852
|
+
importInvalidShape: "导入失败：需要限速配置对象。",
|
|
853
|
+
importReadOnly: "导入失败：当前配置只读。",
|
|
854
|
+
importEmpty: "导入失败：文件里没有可识别的限速配置字段。",
|
|
855
|
+
importUnreadable: "导入失败：无法读取该文件。",
|
|
616
856
|
}),
|
|
617
857
|
en: Object.assign({}, en, {
|
|
618
858
|
nav: "Multi-model limits",
|
|
@@ -640,6 +880,31 @@ window.__ModuleLoader__.load({
|
|
|
640
880
|
reject: "Reject the request",
|
|
641
881
|
maxQueueWait: "Max queue wait (ms)",
|
|
642
882
|
saveHint: "Edits are written to the profile's cordis.patch.yml immediately.",
|
|
883
|
+
searchPlaceholder: "Search models…",
|
|
884
|
+
noMatches: "No model matches this filter.",
|
|
885
|
+
confirmDelete: "Delete",
|
|
886
|
+
cancel: "Cancel",
|
|
887
|
+
providers: "Per-provider limits",
|
|
888
|
+
providersHint: "Applies to every model of that provider that has no per-model limit.",
|
|
889
|
+
noProviders: "No per-provider limit yet.",
|
|
890
|
+
addProvider: "+ Add provider",
|
|
891
|
+
addProviderPlaceholder: "e.g. openai",
|
|
892
|
+
providerScope: "covers every model of this provider",
|
|
893
|
+
cooldown: "Adaptive 429 backoff",
|
|
894
|
+
cooldownHint: "After the provider answers 429, this model runs slower until calls succeed again.",
|
|
895
|
+
cooldownEnabled: "Slow down automatically after a 429",
|
|
896
|
+
cooldownMs: "Cooldown duration (ms)",
|
|
897
|
+
cooldownFactor: "Speed factor (0.05–0.9)",
|
|
898
|
+
recoverAfter: "Consecutive successes to recover",
|
|
899
|
+
preset: "Import / export",
|
|
900
|
+
presetHint: "Exported JSON can be imported on another machine; it replaces every setting.",
|
|
901
|
+
exportConfig: "Export JSON",
|
|
902
|
+
importConfig: "Import JSON",
|
|
903
|
+
importInvalidJson: "Import failed: not valid JSON.",
|
|
904
|
+
importInvalidShape: "Import failed: expected a limiter configuration object.",
|
|
905
|
+
importReadOnly: "Import failed: the configuration is read-only.",
|
|
906
|
+
importEmpty: "Import failed: the file has no recognisable limiter settings.",
|
|
907
|
+
importUnreadable: "Import failed: the file could not be read.",
|
|
643
908
|
}),
|
|
644
909
|
}), "llm-multi-rate-limiter: dictionaries");
|
|
645
910
|
|