@leaf233/dsh-llm-rate-limiter 0.1.1 → 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +41 -0
- package/COMPATIBILITY.md +91 -3
- package/README.md +70 -4
- package/lib/client.js +236 -2
- package/lib/index.js +160 -1
- package/lib/status-rpc.js +285 -0
- package/package.json +11 -3
package/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,47 @@ All notable changes to this project will be documented in this file.
|
|
|
4
4
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/).
|
|
6
6
|
|
|
7
|
+
## [0.2.1] - 2026-09-25
|
|
8
|
+
|
|
9
|
+
### Fixed
|
|
10
|
+
- **Live status panel stopped working on DSH 0.1.5-rc.3** — the channel silently failed to mount and the panel sat on "○ 重试中" forever. Root cause is upstream: DSH 0.1.5-rc.3 changed `@deepseek-ai/dsh-client-connection`'s own `inject` from `["webServer", "credentials"]` to `["credentials"]`, while `HostConnectionService.register()` still dereferences `owner.webServer`. Cordis rebinds a cross-fiber service's `ctx` to the *reader's* fiber, so `connection.rpc.handle()` threw `cannot get property "webServer" without inject` — which the plugin's own try/catch swallowed.
|
|
11
|
+
- The plugin now falls back to a **self-registered `kind: "prefix"` route** on its own fiber (which can see `webServer`) and reuses `connection.requestRejection()` for the 403/401 fence. Both carriers speak the identical wire protocol, so **`lib/client.js` is unchanged**.
|
|
12
|
+
|
|
13
|
+
### Added
|
|
14
|
+
- `createChannelRoute({ channel, handler, reject })` and `endpointFromPath()` in `lib/status-rpc.js` — a Connection-RPC-compatible webserver prefix route, mirroring the framework's `rpcFetchHandler` request-shape decisions (404 / 415 / 413 / 400 / `gateway/bad-request` / 500 / 200)
|
|
15
|
+
- `test-status-route.mjs` (73 assertions) in four layers: route grammar, request-shape decisions, a simulated 0.1.5 regression (proves both the fallback *and* the preference for `rpc.handle` when it works), and the real 0.1.5 package (auto-skips when DSH is absent)
|
|
16
|
+
- `tools/verify-live-route.mjs` (19 assertions) — boots the real `dsh-host-webserver` on a real socket plus the real `dsh-client-connection`, then issues real HTTP requests; exits 2 (skipped) without a DSH install
|
|
17
|
+
- `pnpm run verify:live` script
|
|
18
|
+
|
|
19
|
+
### Changed
|
|
20
|
+
- `dsh.compatibility.dshReleases` now declares `0.1.5-rc.3` as compatible alongside `0.1.2-rc.1`
|
|
21
|
+
- The status channel's mount log now names the active carrier (`connection.rpc` vs `self-registered route`), and a failed path-1 registration is reported as a warning instead of being swallowed
|
|
22
|
+
- If `connection.requestRejection()` is unavailable or throws, the plugin **refuses to mount** rather than publishing an unauthenticated route
|
|
23
|
+
|
|
24
|
+
### Notes
|
|
25
|
+
- Path 1 (`connection.rpc.handle`) is still preferred: it keeps route ownership, request validation, and withdrawal with the framework. The fallback exists only for hosts where path 1 is broken.
|
|
26
|
+
- When DSH fixes the upstream `inject` mismatch, path 1 succeeds again and the fallback goes unused — no plugin change required.
|
|
27
|
+
- Verified on: DSH `0.1.5-rc.3` (client-connection 0.1.5-rc.3, host-webserver 0.1.5-rc.3), Cordis 4.0.2, Node 24.21.0. Test totals: 19 + 61 + 70 + 77 + 73 = **300 assertions**, plus 31 in `tools/verify-install.mjs` and 19 in `tools/verify-live-route.mjs`. Producer: **deepseek-v4.1-flash**.
|
|
28
|
+
|
|
29
|
+
## [0.2.0] - 2026-09-13
|
|
30
|
+
|
|
31
|
+
### Added
|
|
32
|
+
- **Live status panel** in the settings card: request/granted/rejected/timeout/aborted counters, average wait, per-model progress bars (token balance or window occupancy), concurrency and queue badges, and a rolling event log
|
|
33
|
+
- **`connection.rpc` status channel** at `/llm-rate-limiter` with `snapshot` and `reset` endpoints — the framework supplies POST+JSON transport, the Host/Origin fence (403) and browser authentication (401), and withdraws the channel with the plugin fiber
|
|
34
|
+
- **"清零" button** that zeroes all statistics and drops retained events
|
|
35
|
+
- `lib/status-rpc.js` — channel handler and envelope helpers (`success` / `failure`)
|
|
36
|
+
- `test-status-rpc.mjs` (61 assertions): envelopes, endpoint routing, JSON-safety, ring-buffer bounds, every counter path
|
|
37
|
+
- `test-client-bundle.mjs` (70 assertions): loads the real client bundle in a VM and mounts the card and panel on a miniature React runtime — covers live rendering, collapse-stops-polling, and every degraded phase
|
|
38
|
+
|
|
39
|
+
### Changed
|
|
40
|
+
- `dsh.client.inject` now also requires `@deepseek-ai/dsh-client-connection`, so the connection service is available before the panel mounts
|
|
41
|
+
- `dsh.compatibility.dshReleases` declares the verified DSH release (aligned with dsh-context's convention)
|
|
42
|
+
- CI runs the two new suites and syntax-checks `lib/status-rpc.js`
|
|
43
|
+
|
|
44
|
+
### Notes
|
|
45
|
+
- The status channel is optional: on a host without a connection service it is never registered and the panel shows "状态通道不可用" instead, leaving configuration untouched
|
|
46
|
+
- Statistics are in-memory only and reset when the host restarts
|
|
47
|
+
|
|
7
48
|
## [0.1.1] - 2026-09-12
|
|
8
49
|
|
|
9
50
|
### Changed
|
package/COMPATIBILITY.md
CHANGED
|
@@ -1,8 +1,73 @@
|
|
|
1
1
|
# dsh-llm-rate-limiter — DSH 版本兼容性检测报告
|
|
2
2
|
|
|
3
|
-
>
|
|
4
|
-
>
|
|
5
|
-
> 测试方式:
|
|
3
|
+
> 初检日期: 2026-07-28 · 复检日期: **2026-09-25**
|
|
4
|
+
> 已验证 DSH 版本: **0.1.2-rc.1**、**0.1.5-rc.3**
|
|
5
|
+
> 测试方式: 源码分析 + 可执行回归测试（`test-status-route.mjs`、`tools/verify-live-route.mjs` 走真实 socket）
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## 〇、0.1.5-rc.3 回归与修复（2026-09-25 复检）
|
|
10
|
+
|
|
11
|
+
### 现象
|
|
12
|
+
升级到 DSH **0.1.5-rc.3** 后，状态面板永久停在「○ 重试中」，实时数据不再刷新；限速本身与设置页配置完全正常。
|
|
13
|
+
|
|
14
|
+
### 根因（已用真实包 + 真实 Cordis 复现，非推断）
|
|
15
|
+
DSH 0.1.5-rc.3 把 **`@deepseek-ai/dsh-client-connection` 这个框架插件自身**的 `inject` 从
|
|
16
|
+
`["webServer", "credentials"]` 改成了 `["credentials"]`（lib/index.js:736），
|
|
17
|
+
但其 `HostConnectionService.register()` 仍然直接解引用 `owner.webServer`：
|
|
18
|
+
|
|
19
|
+
```js
|
|
20
|
+
// dsh-client-connection/lib/index.js:618
|
|
21
|
+
return owner.effect(() => owner.webServer.register(route), `... rpc channel`);
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
而 `connection.rpc` 的 getter 取 `const owner = this.ctx`，Cordis 的 `createTraceable`
|
|
25
|
+
会把跨 fiber 读取的 service 的 `ctx` 重绑定到**读取者**的 fiber
|
|
26
|
+
（实测 `owner.fiber === consumer.fiber` 为 true）。因此 `owner.webServer` 的解析从读取者的
|
|
27
|
+
inject 出发 —— 读取者只 inject 了 `["connection"]`，于是抛：
|
|
28
|
+
|
|
29
|
+
```
|
|
30
|
+
cannot get property "webServer" without inject
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
该异常发生在插件 `ctx.inject(["connection"], ...)` 回调内，被插件自己的 try/catch 吞掉，
|
|
34
|
+
只留一条 `logger.warn`，于是通道**静默未注册**。
|
|
35
|
+
|
|
36
|
+
### 线上证据
|
|
37
|
+
| 探测 | 修复前 | 含义 |
|
|
38
|
+
|---|---|---|
|
|
39
|
+
| `GET /api/xxx` | 401 | `/api` 路由存在，仅认证被拒 |
|
|
40
|
+
| `GET /llm-rate-limiter/snapshot` | **404** | 该前缀**无路由**（走 SPA fallback） |
|
|
41
|
+
| `POST /llm-rate-limiter/snapshot` | **405** | 同上 |
|
|
42
|
+
|
|
43
|
+
而 `dsh --profile web --dump-config` 中插件条目仍存在 → 插件确实被加载，只是通道注册失败。
|
|
44
|
+
|
|
45
|
+
### 交叉验证（隔离变量）
|
|
46
|
+
| 实验 | 结果 |
|
|
47
|
+
|---|---|
|
|
48
|
+
| 0.1.2 代码 + fiber `[webServer,credentials]` | ✅ 注册成功 |
|
|
49
|
+
| 0.1.2 代码 + fiber `[credentials]` | ❌ 失败 |
|
|
50
|
+
| 0.1.5 代码 + fiber `[webServer,credentials]` | ✅ 注册成功 |
|
|
51
|
+
| 0.1.5 代码 + fiber `[credentials]`（真实形态） | ❌ 失败 |
|
|
52
|
+
|
|
53
|
+
→ 差异**只由 inject 列表决定**，与两版 `register` 代码差异无关。另实测：仅给**消费者**外层 fiber 加
|
|
54
|
+
`webServer`（或嵌套 `ctx.inject(["connection","webServer"])`）**均无效** —— 因为 `owner` 绑定的是
|
|
55
|
+
connection service 自身那条 fiber。
|
|
56
|
+
|
|
57
|
+
### 修复（v0.2.1，方案 E）
|
|
58
|
+
保留路径 1 为首选，新增路径 2 兜底，两者**同一套 wire 协议**，浏览器端零改动：
|
|
59
|
+
|
|
60
|
+
| 路径 | 触发条件 | 做法 |
|
|
61
|
+
|---|---|---|
|
|
62
|
+
| 1（首选） | `connection.rpc.handle()` 正常 | 框架持有路由、请求校验与 fiber 级撤销 |
|
|
63
|
+
| 2（兜底） | 路径 1 抛错 | 插件在**自己的 fiber**（能看到 `webServer`）注册 `kind:"prefix"` 路由，并复用 `connection.requestRejection()` 保留 403/401 栅栏 |
|
|
64
|
+
|
|
65
|
+
- 若 `connection.requestRejection()` 缺失或抛错，插件**拒绝挂载**，绝不发布未鉴权路由
|
|
66
|
+
- 上游修复该 inject 不一致后，路径 1 会自动重新生效，无需再改插件
|
|
67
|
+
|
|
68
|
+
### 回归测试
|
|
69
|
+
- `test-status-route.mjs`（73 断言）四层：路由语法、请求形态判定、**模拟 0.1.5 回归**（同时验证「兜底生效」与「路径 1 可用时优先」）、真实 0.1.5 包（无 DSH 时自动跳过）
|
|
70
|
+
- `tools/verify-live-route.mjs`（19 断言）：真实 `dsh-host-webserver` 真 socket + 真实 `dsh-client-connection`，发真实 HTTP 请求；无 DSH 时退出码 2（跳过）
|
|
6
71
|
|
|
7
72
|
---
|
|
8
73
|
|
|
@@ -30,6 +95,20 @@
|
|
|
30
95
|
| 5 | `settingsScope.mutate([{ op, path, value }])` | dsh-client-ui-settings | ✅ 1处 | SettingsScopeController.mutate (嵌套操作) |
|
|
31
96
|
| 6 | `ctx.slots.inject("settings.plugin.item", ...)` | dsh-client-ui-settings-plugins | ✅ 6处 | 插件设置卡片注册 slot |
|
|
32
97
|
|
|
98
|
+
### v0.2.0 新增依赖（状态通道）
|
|
99
|
+
|
|
100
|
+
| # | API | 所属包 | 本地存在 | 稳定性评估 |
|
|
101
|
+
|---|-----|--------|----------|------------|
|
|
102
|
+
| 1 | `ctx.inject(["connection"], cb)` + `conn.rpc.handle(channel, handler)` | dsh-client-connection | ⚠️ 0.1.2 可用 / **0.1.5 抛错** | 首选路径。0.1.5-rc.3 起因该包自身 `inject` 缺 `webServer` 而抛 `cannot get property "webServer" without inject`，故 v0.2.1 增加路径 2 兜底 |
|
|
103
|
+
| 1b | `conn.requestRejection(req)` + `ctx.inject(["webServer"], cb)` + `webServer.register({kind:"prefix",...})` | dsh-client-connection + dsh-host-webserver | ✅ 0.1.2 / 0.1.5 | **v0.2.1 兜底路径**。复用框架的 403/401 栅栏，自行注册前缀路由；wire 协议与路径 1 逐字一致 |
|
|
104
|
+
| 2 | handler 签名 `(endpoint, payload, signal) => envelope` | dsh-client-connection | ✅ `rpcFetchHandler`（605-631） | 框架强制 POST + `application/json`（否则 404/415），并自动套 `requestRejection`（Host/Origin 不信→403，未认证→401） |
|
|
105
|
+
| 3 | envelope `{ok:true,value}` / `{ok:false,error:{code,message,details}}` | dsh-client-connection | ✅ 与 `clientRequestSchema` / `errorResponse` 一致 | 契约字面量，dsh-context 逐字复制同一范式 |
|
|
106
|
+
| 4 | 客户端 `ctx.get("connection")?.rpc.call(channel, endpoint, payload, signal)` | dsh-client-connection | ✅ `createWebConnectionRpc.call`（client.js 4606-4628） | 传输失败 reject；返回 `result`（即 envelope）。经 `rpcCallOf` 反射读取，服务缺席/敌意时降级为 `undefined` |
|
|
107
|
+
| 5 | `dsh.client.inject` 声明 `@deepseek-ai/dsh-client-connection` | DSH web shell | ✅ | bundle 依赖声明，确保连接服务先于本插件可用；**不放进模块级 `inject`**，以便服务缺席时面板降级而非整卡不加载 |
|
|
108
|
+
| 6 | `dsh.compatibility.dshReleases` 声明 | DSH 打包约定 | ✅ 对齐 `dsh-context` | 成熟插件物料约定 |
|
|
109
|
+
|
|
110
|
+
> 证据：dsh-context v0.50.0 使用同一组 API（`watchDetailChannel`，lib/index.js 1943-2012），其 compatibility 矩阵覆盖 DSH `0.1.2-rc.1` → `0.1.5-rc.1`，说明 `connection.rpc` 是**持久公开契约**而非临时接口。
|
|
111
|
+
|
|
33
112
|
### Schema 依赖 — 1 个包
|
|
34
113
|
|
|
35
114
|
| 包 | 本地版本 | 用到的 API |
|
|
@@ -130,4 +209,13 @@
|
|
|
130
209
|
- [ ] `settings.register` 返回值结构是否变化
|
|
131
210
|
- [ ] `settingsScope.set/mutate` 接口是否变化
|
|
132
211
|
- [ ] `settings.plugin.item` slot 是否仍可注入
|
|
212
|
+
- [ ] **状态通道**：`connection.rpc.handle` 是否仍抛 `webServer` 相关错误（0.1.5-rc.3 会）
|
|
213
|
+
- [ ] **状态通道**：`connection.requestRejection` 仍为公开 API（兜底路径的鉴权依赖它）
|
|
214
|
+
- [ ] **状态通道**：`webServer.register({kind:"prefix"})` 仍接受前缀路由
|
|
215
|
+
- [ ] **状态通道**：`snapshot` / `reset` 端点往返成功（面板显示"● 实时"而非"重试中"）
|
|
216
|
+
- [ ] **状态通道**：折叠卡片后 DevTools Network 无 `/llm-rate-limiter/snapshot` 轮询
|
|
217
|
+
- [ ] **状态通道**：无 connection 服务的组合下，卡片配置区仍可用且面板显示降级文案
|
|
218
|
+
- [ ] **状态通道**：直接 `POST /llm-rate-limiter/snapshot` 应返回 401（路由存在且被栅栏保护），而非 404/405（路由缺失）
|
|
219
|
+
|
|
220
|
+
> 上述清单已由 `test-status-route.mjs` + `tools/verify-live-route.mjs` 自动覆盖，升级 DSH 后先跑 `pnpm test && pnpm run verify:live`。
|
|
133
221
|
4. **如果 cordis 升级到 5.0+**：整个 `apply(ctx)` 接口、`ctx.effect()`、`ctx.on()` 签名可能重写，需要全面适配
|
package/README.md
CHANGED
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
[](LICENSE)
|
|
5
5
|
[](COMPATIBILITY.md)
|
|
6
6
|
[](package.json)
|
|
7
|
-
[](test-status-route.mjs)
|
|
8
8
|
|
|
9
9
|
Per-model LLM call rate limiter for [DeepSeek Harness](https://github.com/deepseek-ai/dsh) with queue/reject support and interactive GUI configuration.
|
|
10
10
|
|
|
@@ -17,9 +17,64 @@ Per-model LLM call rate limiter for [DeepSeek Harness](https://github.com/deepse
|
|
|
17
17
|
- **Queue mode** — throttled requests wait in queue and are released when a slot opens
|
|
18
18
|
- **Reject mode** — throttled requests fail immediately (integrates with `dsh-llm-retry` for auto-backoff)
|
|
19
19
|
- **Interactive GUI** — collapsible card in DSH Settings → Plugins → Configurable
|
|
20
|
+
- **Live status panel** — real-time counters, per-model progress bars and an event log (v0.2.0)
|
|
20
21
|
- **Hot-reload** — settings changes take effect immediately, no restart needed
|
|
21
22
|
- **Every request checked** — intercepts `llm/stream` waterfall, covering every LLM call in every agent turn
|
|
22
23
|
|
|
24
|
+
---
|
|
25
|
+
|
|
26
|
+
## Status Panel (v0.2.0)
|
|
27
|
+
|
|
28
|
+
Expanding the card shows a live panel at the top of its body:
|
|
29
|
+
|
|
30
|
+
```
|
|
31
|
+
📊 实时状态 ● 实时 [清零]
|
|
32
|
+
请求 42 · 通过 38 · 拒绝 2 · 超时 1 · 中止 1 · 平均等待 214ms
|
|
33
|
+
deepseek/deepseek-chat [令牌桶] ▓▓▓▓▓▓▓░░░ 7.5/10 并发 2/5 排队 1
|
|
34
|
+
openai/gpt-4o [滑动窗口] ▓▓▓▓▓▓▓▓▓▓ 3/3 rpm 并发 1/5
|
|
35
|
+
12:00:03 timeout openai/gpt-4o 等待 1m
|
|
36
|
+
12:00:01 rejected openai/gpt-4o
|
|
37
|
+
12:00:00 granted deepseek/deepseek-chat 等待 4.2s
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
| Aspect | Behaviour |
|
|
41
|
+
|--------|-----------|
|
|
42
|
+
| Data channel | Channel `/llm-rate-limiter` — authenticated (401/403 fence), POST+JSON, auto-cleaned with the plugin fiber |
|
|
43
|
+
| Carrier | Prefers the framework's `connection.rpc.handle()`; falls back to a self-registered prefix route that reuses `connection.requestRejection()` when the framework path is broken (see below) |
|
|
44
|
+
| Cadence | 1 s polling while the card is expanded; backs off 2 s → 4 s → 8 s after failures |
|
|
45
|
+
| Collapsed card | The panel unmounts, so **no polling runs at all** |
|
|
46
|
+
| Endpoints | `snapshot` (live counters) and `reset` (zero the statistics) |
|
|
47
|
+
| Without a channel | Shows "状态通道不可用" and leaves the rest of the card fully functional |
|
|
48
|
+
| Counters | `requests` / `granted` / `rejected` / `timeouts` / `aborted` / `totalWaitMs`, plus the last 8 events (ring buffer of 64) |
|
|
49
|
+
| Progress bars | Token bucket shows `tokens/burstSize`; sliding window shows `countInWindow/maxRpm`; both turn amber as the limit approaches |
|
|
50
|
+
|
|
51
|
+
> 🧠 **From Hindsight memory (dsh-context-host-client)** — the channel idiom is dsh-context's: `ctx.inject(["connection"])` → `conn.rpc.handle(channel, handler)`, with the browser side resolving `ctx.get("connection")?.rpc.call` defensively so a missing service degrades instead of throwing.
|
|
52
|
+
|
|
53
|
+
### Carrier fallback (DSH 0.1.5-rc.3)
|
|
54
|
+
|
|
55
|
+
DSH 0.1.5-rc.3 changed `@deepseek-ai/dsh-client-connection`'s own `inject` from
|
|
56
|
+
`["webServer", "credentials"]` to `["credentials"]`, but its
|
|
57
|
+
`HostConnectionService.register()` still dereferences `owner.webServer`.
|
|
58
|
+
Cordis rebinds a cross-fiber service's `ctx` to the *reader's* fiber, so
|
|
59
|
+
`connection.rpc.handle()` throws:
|
|
60
|
+
|
|
61
|
+
```
|
|
62
|
+
cannot get property "webServer" without inject
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
The plugin now detects that and mounts the same channel itself:
|
|
66
|
+
|
|
67
|
+
| Path | When | How |
|
|
68
|
+
|------|------|-----|
|
|
69
|
+
| 1 (preferred) | `connection.rpc.handle()` works | The framework owns the route, request validation, and fiber-scoped withdrawal |
|
|
70
|
+
| 2 (fallback) | Path 1 throws | The plugin registers a `kind: "prefix"` route on its own fiber (which *can* see `webServer`) and reuses `connection.requestRejection()` for the 403/401 fence |
|
|
71
|
+
|
|
72
|
+
Both carriers speak the identical wire protocol, so **the browser half is
|
|
73
|
+
unchanged** — the panel cannot tell which one is live. If
|
|
74
|
+
`connection.requestRejection()` is unavailable, the plugin refuses to mount
|
|
75
|
+
rather than publishing an unauthenticated route.
|
|
76
|
+
|
|
77
|
+
|
|
23
78
|
---
|
|
24
79
|
|
|
25
80
|
## Installation
|
|
@@ -159,8 +214,18 @@ cd dsh-llm-rate-limiter
|
|
|
159
214
|
# Install deps
|
|
160
215
|
pnpm install
|
|
161
216
|
|
|
162
|
-
# Run
|
|
163
|
-
|
|
217
|
+
# Run the full suite (300 assertions across 5 files)
|
|
218
|
+
pnpm test
|
|
219
|
+
|
|
220
|
+
# Individual suites
|
|
221
|
+
node test-strategies.mjs # 19 — rate-limit algorithms
|
|
222
|
+
node test-status-rpc.mjs # 61 — channel handler + counters
|
|
223
|
+
node test-client-bundle.mjs # 70 — real client bundle on a miniature React runtime
|
|
224
|
+
node test-host-integration.mjs # 77 — real apply() wiring
|
|
225
|
+
node test-status-route.mjs # 73 — self-registered route + the 0.1.5 regression
|
|
226
|
+
|
|
227
|
+
# Live HTTP proof against a real DSH install (real socket, real webserver)
|
|
228
|
+
pnpm run verify:live # 19 — exits 2 (skipped) when DSH is absent
|
|
164
229
|
|
|
165
230
|
# Run E2E rate-limit test
|
|
166
231
|
node test-3rpm.mjs
|
|
@@ -177,7 +242,8 @@ The plugin uses a live symlink when installed via `link:` — edits to `lib/` ta
|
|
|
177
242
|
|
|
178
243
|
| DSH Version | Status | Notes |
|
|
179
244
|
|-------------|--------|-------|
|
|
180
|
-
| 0.1.
|
|
245
|
+
| 0.1.2-rc.1 | ✅ Tested | Original target; `connection.rpc` carrier |
|
|
246
|
+
| 0.1.5-rc.3 | ✅ Tested | Requires the carrier fallback (see above); verified over real HTTP |
|
|
181
247
|
| 0.2.x | ⚠️ Untested | May need API adjustments |
|
|
182
248
|
| Cordis 5+ | ⚠️ Untested | Major version change likely requires rewrite |
|
|
183
249
|
|
package/lib/client.js
CHANGED
|
@@ -53,6 +53,34 @@ window.__ModuleLoader__.load({
|
|
|
53
53
|
".rlr-modelMeta{display:flex;gap:12px;font-size:12px;color:var(--dsw-alias-label-tertiary)}",
|
|
54
54
|
".rlr-section{display:flex;flex-direction:column;gap:4px;margin-top:2px}",
|
|
55
55
|
".rlr-sectHead{margin:10px 0 0;font-size:14px;font-weight:500}",
|
|
56
|
+
// ── status panel ──
|
|
57
|
+
".rlr-statusHead{display:flex;align-items:center;gap:8px;margin:10px 0 0}",
|
|
58
|
+
".rlr-statusTitle{font-size:14px;font-weight:500;flex:1}",
|
|
59
|
+
".rlr-chips{display:flex;flex-wrap:wrap;gap:6px;margin:8px 0}",
|
|
60
|
+
".rlr-chip{font-size:11px;padding:2px 8px;border-radius:10px;background:var(--dsw-alias-surface-s2);color:var(--dsw-alias-label-secondary);white-space:nowrap;font-variant-numeric:tabular-nums}",
|
|
61
|
+
".rlr-chipOk{background:var(--dsw-alias-state-success-primary,#16a34a);color:#fff}",
|
|
62
|
+
".rlr-chipWarn{background:var(--dsw-alias-state-warning-primary,#d97706);color:#fff}",
|
|
63
|
+
".rlr-chipErr{background:var(--dsw-alias-state-error-primary,#dc2626);color:#fff}",
|
|
64
|
+
".rlr-phase{font-size:11px;color:var(--dsw-alias-label-tertiary)}",
|
|
65
|
+
".rlr-phaseLive{color:var(--dsw-alias-state-success-primary,#16a34a)}",
|
|
66
|
+
".rlr-phaseFailed{color:var(--dsw-alias-state-error-primary,#dc2626)}",
|
|
67
|
+
".rlr-models{display:flex;flex-direction:column;gap:4px;margin:6px 0}",
|
|
68
|
+
".rlr-mRow{display:flex;align-items:center;gap:10px;padding:6px 10px;border-radius:10px;border:1px solid var(--dsw-alias-border-l4)}",
|
|
69
|
+
".rlr-mName{font-size:12px;font-family:monospace;min-width:0;flex:1;overflow:hidden;text-overflow:ellipsis;white-space:nowrap}",
|
|
70
|
+
".rlr-mKind{font-size:10px;padding:1px 6px;border-radius:8px;background:var(--dsw-alias-surface-s2);color:var(--dsw-alias-label-tertiary);white-space:nowrap}",
|
|
71
|
+
".rlr-mMeta{font-size:11px;color:var(--dsw-alias-label-tertiary);white-space:nowrap;font-variant-numeric:tabular-nums}",
|
|
72
|
+
".rlr-mIdle{opacity:.55}",
|
|
73
|
+
".rlr-bar{width:90px;height:6px;border-radius:3px;background:var(--dsw-alias-surface-s2);overflow:hidden;flex:none}",
|
|
74
|
+
".rlr-barFill{height:100%;background:var(--dsw-alias-brand-primary,#4f46e5);transition:width .3s}",
|
|
75
|
+
".rlr-barFillWarn{background:var(--dsw-alias-state-warning-primary,#d97706)}",
|
|
76
|
+
".rlr-log{display:flex;flex-direction:column;gap:1px;margin:6px 0;font-family:monospace;font-size:11px}",
|
|
77
|
+
".rlr-logRow{display:flex;gap:8px;padding:1px 4px;border-radius:4px;color:var(--dsw-alias-label-tertiary)}",
|
|
78
|
+
".rlr-logRowNew{background:var(--dsw-alias-surface-s2)}",
|
|
79
|
+
".rlr-logEv{min-width:66px}",
|
|
80
|
+
".rlr-logEvOk{color:var(--dsw-alias-state-success-primary,#16a34a)}",
|
|
81
|
+
".rlr-logEvWarn{color:var(--dsw-alias-state-warning-primary,#d97706)}",
|
|
82
|
+
".rlr-logEvErr{color:var(--dsw-alias-state-error-primary,#dc2626)}",
|
|
83
|
+
".rlr-logMs{min-width:56px;text-align:right}",
|
|
56
84
|
].join("\n");
|
|
57
85
|
const tagId = "@dsh-llm-rate-limiter/rate-limiter-card.css";
|
|
58
86
|
if (typeof document !== "undefined" && document.querySelector("style[data-plugin-css=" + JSON.stringify(tagId) + "]") === null) {
|
|
@@ -75,8 +103,209 @@ window.__ModuleLoader__.load({
|
|
|
75
103
|
scope.mutate([{ op: "unset", path }]);
|
|
76
104
|
}
|
|
77
105
|
|
|
106
|
+
/* ── status channel access (dsh-context defensive idiom) ─────────── */
|
|
107
|
+
const STATUS_CHANNEL = "/llm-rate-limiter";
|
|
108
|
+
|
|
109
|
+
/**
|
|
110
|
+
* The connection's bound generic-RPC caller, or undefined when the
|
|
111
|
+
* service is absent or hostile. Every read is guarded, so this can
|
|
112
|
+
* never throw — a missing channel degrades the panel, not the card.
|
|
113
|
+
*/
|
|
114
|
+
function rpcCallOf(ctx) {
|
|
115
|
+
try {
|
|
116
|
+
const rpc = ctx.get("connection")?.rpc;
|
|
117
|
+
const fn = rpc?.call;
|
|
118
|
+
if (rpc && typeof fn === "function") return fn.bind(rpc);
|
|
119
|
+
} catch {}
|
|
120
|
+
return undefined;
|
|
121
|
+
}
|
|
122
|
+
|
|
123
|
+
/** Format a duration in ms for the chips/log (compact). */
|
|
124
|
+
function fmtMs(ms) {
|
|
125
|
+
if (!Number.isFinite(ms) || ms <= 0) return "0";
|
|
126
|
+
if (ms < 1000) return Math.round(ms) + "ms";
|
|
127
|
+
if (ms < 60_000) return (ms / 1000).toFixed(1) + "s";
|
|
128
|
+
return Math.round(ms / 60_000) + "m";
|
|
129
|
+
}
|
|
130
|
+
|
|
131
|
+
/** Human label for a strategy id. */
|
|
132
|
+
function strategyLabel(id) {
|
|
133
|
+
return id === "sliding-window" ? "滑动窗口" : "令牌桶";
|
|
134
|
+
}
|
|
135
|
+
|
|
136
|
+
/** Tunable polling cadence: first retry delay doubles up to 8s. */
|
|
137
|
+
const POLL_MS = 1000;
|
|
138
|
+
const POLL_MAX_MULTIPLIER = 8;
|
|
139
|
+
|
|
140
|
+
/* ── React component: live status panel ──────────────────────────── */
|
|
141
|
+
function RateLimiterStatus({ statusCall, cfg }) {
|
|
142
|
+
const [snap, setSnap] = react.useState(null);
|
|
143
|
+
const [phase, setPhase] = react.useState("connecting");
|
|
144
|
+
const prevRev = react.useRef(-1);
|
|
145
|
+
const [highlight, setHighlight] = react.useState(false);
|
|
146
|
+
|
|
147
|
+
react.useEffect(() => {
|
|
148
|
+
const call = statusCall();
|
|
149
|
+
if (call === undefined) {
|
|
150
|
+
setPhase("no-channel");
|
|
151
|
+
return undefined;
|
|
152
|
+
}
|
|
153
|
+
let stop = false;
|
|
154
|
+
let timer = null;
|
|
155
|
+
let failures = 0;
|
|
156
|
+
|
|
157
|
+
const tick = async () => {
|
|
158
|
+
try {
|
|
159
|
+
const r = await call(STATUS_CHANNEL, "snapshot", {});
|
|
160
|
+
if (stop) return;
|
|
161
|
+
if (r?.ok !== true || !r.value) throw new Error("bad envelope");
|
|
162
|
+
setSnap(r.value);
|
|
163
|
+
setPhase("live");
|
|
164
|
+
failures = 0;
|
|
165
|
+
} catch {
|
|
166
|
+
if (stop) return;
|
|
167
|
+
failures += 1;
|
|
168
|
+
setPhase("failed");
|
|
169
|
+
}
|
|
170
|
+
if (stop) return;
|
|
171
|
+
const multiplier = Math.min(2 ** Math.min(failures, 3), POLL_MAX_MULTIPLIER);
|
|
172
|
+
timer = setTimeout(tick, POLL_MS * multiplier);
|
|
173
|
+
};
|
|
174
|
+
|
|
175
|
+
tick();
|
|
176
|
+
return () => {
|
|
177
|
+
stop = true;
|
|
178
|
+
if (timer) clearTimeout(timer);
|
|
179
|
+
};
|
|
180
|
+
}, [statusCall]);
|
|
181
|
+
|
|
182
|
+
// Flash newly arrived events when `rev` advances between polls.
|
|
183
|
+
react.useEffect(() => {
|
|
184
|
+
if (!snap) return undefined;
|
|
185
|
+
if (prevRev.current === -1) { prevRev.current = snap.rev; return undefined; }
|
|
186
|
+
if (snap.rev === prevRev.current) return undefined;
|
|
187
|
+
prevRev.current = snap.rev;
|
|
188
|
+
setHighlight(true);
|
|
189
|
+
const t = setTimeout(() => setHighlight(false), 700);
|
|
190
|
+
return () => clearTimeout(t);
|
|
191
|
+
}, [snap]);
|
|
192
|
+
|
|
193
|
+
// ── non-live phases ──
|
|
194
|
+
if (phase === "no-channel") {
|
|
195
|
+
return jsx("div", { className: "rlr-status", children: "状态通道不可用（需要 DSH web 组合）" });
|
|
196
|
+
}
|
|
197
|
+
if (phase === "connecting" && !snap) {
|
|
198
|
+
return jsx("div", { className: "rlr-status", children: "正在连接状态通道…" });
|
|
199
|
+
}
|
|
200
|
+
if (!snap) {
|
|
201
|
+
return jsx("div", { className: "rlr-status", children: "连接中断，重试中…" });
|
|
202
|
+
}
|
|
203
|
+
|
|
204
|
+
const t = snap.totals ?? {};
|
|
205
|
+
const avgWait = t.granted > 0 ? (t.totalWaitMs ?? 0) / t.granted : 0;
|
|
206
|
+
const live = phase === "live";
|
|
207
|
+
|
|
208
|
+
function onReset() {
|
|
209
|
+
const call = statusCall();
|
|
210
|
+
if (call === undefined) return;
|
|
211
|
+
Promise.resolve(call(STATUS_CHANNEL, "reset", {})).catch(() => undefined);
|
|
212
|
+
}
|
|
213
|
+
|
|
214
|
+
// Union of configured models and models that have actually been used.
|
|
215
|
+
const snapModels = snap.models ?? {};
|
|
216
|
+
const keys = Array.from(new Set([...Object.keys(cfg.models ?? {}), ...Object.keys(snapModels)]));
|
|
217
|
+
|
|
218
|
+
const chips = [
|
|
219
|
+
{ label: "请求 " + (t.requests ?? 0) },
|
|
220
|
+
{ label: "通过 " + (t.granted ?? 0), tone: "ok" },
|
|
221
|
+
(t.rejected ?? 0) > 0 && { label: "拒绝 " + t.rejected, tone: "err" },
|
|
222
|
+
(t.timeouts ?? 0) > 0 && { label: "超时 " + t.timeouts, tone: "warn" },
|
|
223
|
+
(t.aborted ?? 0) > 0 && { label: "中止 " + t.aborted, tone: "warn" },
|
|
224
|
+
{ label: "平均等待 " + fmtMs(avgWait) },
|
|
225
|
+
].filter(Boolean);
|
|
226
|
+
|
|
227
|
+
const events = Array.isArray(snap.events) ? snap.events.slice().reverse() : [];
|
|
228
|
+
|
|
229
|
+
return jsxs("div", { className: "rlr-section", children: [
|
|
230
|
+
jsxs("div", { className: "rlr-statusHead", children: [
|
|
231
|
+
jsx("span", { className: "rlr-statusTitle", children: "📊 实时状态" }),
|
|
232
|
+
jsx("span", {
|
|
233
|
+
className: "rlr-phase" + (live ? " rlr-phaseLive" : " rlr-phaseFailed"),
|
|
234
|
+
children: live ? "● 实时" : "○ 重试中",
|
|
235
|
+
}),
|
|
236
|
+
jsx("button", { type: "button", className: "rlr-btnSec", onClick: onReset, children: "清零" }),
|
|
237
|
+
] }),
|
|
238
|
+
|
|
239
|
+
snap.enabled === false
|
|
240
|
+
? jsx("div", { className: "rlr-status", children: "限速已停用" })
|
|
241
|
+
: null,
|
|
242
|
+
|
|
243
|
+
jsx("div", { className: "rlr-chips", children: chips.map((c, i) =>
|
|
244
|
+
jsx("span", {
|
|
245
|
+
key: i,
|
|
246
|
+
className: "rlr-chip" + (c.tone === "ok" ? " rlr-chipOk" : c.tone === "warn" ? " rlr-chipWarn" : c.tone === "err" ? " rlr-chipErr" : ""),
|
|
247
|
+
children: c.label,
|
|
248
|
+
}),
|
|
249
|
+
) }),
|
|
250
|
+
|
|
251
|
+
keys.length === 0
|
|
252
|
+
? jsx("div", { className: "rlr-status", children: "尚无模型调用记录" })
|
|
253
|
+
: jsx("div", { className: "rlr-models", children: keys.map((key) => {
|
|
254
|
+
const m = snapModels[key];
|
|
255
|
+
if (!m) {
|
|
256
|
+
return jsxs("div", { key, className: "rlr-mRow rlr-mIdle", children: [
|
|
257
|
+
jsx("span", { className: "rlr-mName", children: key }),
|
|
258
|
+
jsx("span", { className: "rlr-mMeta", children: "未调用" }),
|
|
259
|
+
] });
|
|
260
|
+
}
|
|
261
|
+
const isBucket = m.strategy !== "sliding-window";
|
|
262
|
+
const numerator = isBucket ? (m.tokens ?? 0) : (m.countInWindow ?? 0);
|
|
263
|
+
const denominator = isBucket ? (m.burstSize ?? 0) : (m.maxRpm ?? 0);
|
|
264
|
+
const pct = denominator > 0 ? Math.min(100, Math.round((numerator / denominator) * 100)) : 0;
|
|
265
|
+
// A nearly empty bucket / a nearly full window both mean "close to throttling".
|
|
266
|
+
const nearLimit = isBucket ? pct <= 20 : pct >= 80;
|
|
267
|
+
return jsxs("div", { key, className: "rlr-mRow", children: [
|
|
268
|
+
jsx("span", { className: "rlr-mName", title: key, children: key }),
|
|
269
|
+
jsx("span", { className: "rlr-mKind", children: strategyLabel(m.strategy) }),
|
|
270
|
+
jsx("span", { className: "rlr-bar", children:
|
|
271
|
+
jsx("span", { className: "rlr-barFill" + (nearLimit ? " rlr-barFillWarn" : ""), style: { width: pct + "%" } }),
|
|
272
|
+
}),
|
|
273
|
+
jsx("span", { className: "rlr-mMeta", children:
|
|
274
|
+
(isBucket ? numerator + "/" + denominator : numerator + "/" + denominator + " rpm"),
|
|
275
|
+
}),
|
|
276
|
+
jsx("span", { className: "rlr-mMeta", children: "并发 " + (m.concurrent ?? 0) + "/" + (m.maxConcurrent ?? 0) }),
|
|
277
|
+
(m.queued ?? 0) > 0
|
|
278
|
+
? jsx("span", { className: "rlr-chip rlr-chipWarn", children: "排队 " + m.queued })
|
|
279
|
+
: null,
|
|
280
|
+
] });
|
|
281
|
+
}) }),
|
|
282
|
+
|
|
283
|
+
events.length > 0
|
|
284
|
+
? jsx("div", { className: "rlr-log", children: events.map((e, i) => {
|
|
285
|
+
const good = e.event === "granted" || e.event === "completed";
|
|
286
|
+
const bad = e.event === "rejected";
|
|
287
|
+
const d = new Date(e.ts ?? Date.now());
|
|
288
|
+
const hh = String(d.getHours()).padStart(2, "0");
|
|
289
|
+
const mm = String(d.getMinutes()).padStart(2, "0");
|
|
290
|
+
const ss = String(d.getSeconds()).padStart(2, "0");
|
|
291
|
+
return jsxs("div", {
|
|
292
|
+
className: "rlr-logRow" + (highlight && i === 0 ? " rlr-logRowNew" : ""),
|
|
293
|
+
children: [
|
|
294
|
+
jsx("span", { children: hh + ":" + mm + ":" + ss }),
|
|
295
|
+
jsx("span", { className: "rlr-logEv " + (good ? "rlr-logEvOk" : bad ? "rlr-logEvErr" : "rlr-logEvWarn"), children: e.event }),
|
|
296
|
+
jsx("span", { className: "rlr-mName", title: e.model, children: e.model }),
|
|
297
|
+
(e.waitMs ?? 0) > 0
|
|
298
|
+
? jsx("span", { className: "rlr-logMs", children: "等待 " + fmtMs(e.waitMs) })
|
|
299
|
+
: null,
|
|
300
|
+
],
|
|
301
|
+
});
|
|
302
|
+
}) })
|
|
303
|
+
: null,
|
|
304
|
+
] });
|
|
305
|
+
}
|
|
306
|
+
|
|
78
307
|
/* ── React component: collapsible card ───────────────────────────── */
|
|
79
|
-
function RateLimiterCard({ settingsScope }) {
|
|
308
|
+
function RateLimiterCard({ settingsScope, statusCall }) {
|
|
80
309
|
const [cfg, setCfg] = react.useState(() => {
|
|
81
310
|
const snap = settingsScope.getSnapshot?.();
|
|
82
311
|
return snap?.status === "ready" && snap.value ? { ...snap.value } : {};
|
|
@@ -129,6 +358,8 @@ window.__ModuleLoader__.load({
|
|
|
129
358
|
]
|
|
130
359
|
}),
|
|
131
360
|
open && jsxs("div", { className: "rlr-body", children: [
|
|
361
|
+
/* ── 实时状态面板（折叠即卸载 → 停止轮询） ── */
|
|
362
|
+
jsx(RateLimiterStatus, { statusCall, cfg }),
|
|
132
363
|
/* ── 启用开关 ── */
|
|
133
364
|
jsxs("div", { className: "rlr-row", children: [
|
|
134
365
|
jsx("label", { children: [
|
|
@@ -220,7 +451,10 @@ window.__ModuleLoader__.load({
|
|
|
220
451
|
name: "settings.plugin.item",
|
|
221
452
|
key: name,
|
|
222
453
|
locale: name,
|
|
223
|
-
|
|
454
|
+
// `statusCall` is resolved per render (never cached) so a
|
|
455
|
+
// late-arriving or hot-reloaded connection service is picked
|
|
456
|
+
// up without re-registering the card.
|
|
457
|
+
inject: () => ({ settingsScope: scope, statusCall: () => rpcCallOf(ctx) }),
|
|
224
458
|
}, RateLimiterCard);
|
|
225
459
|
});
|
|
226
460
|
}
|
package/lib/index.js
CHANGED
|
@@ -21,10 +21,16 @@
|
|
|
21
21
|
import { TokenBucketStrategy } from "./strategies/token-bucket.js";
|
|
22
22
|
import { SlidingWindowStrategy } from "./strategies/sliding-window.js";
|
|
23
23
|
import { RateLimiterConfig } from "./types/config.js";
|
|
24
|
+
import { CHANNEL, createChannelRoute, createStatusChannel } from "./status-rpc.js";
|
|
24
25
|
|
|
25
26
|
const name = "llm-rate-limiter";
|
|
26
27
|
const SETTINGS_NS = "llm-rate-limiter";
|
|
27
28
|
|
|
29
|
+
/** Maximum retained events in the status ring (bounded memory). */
|
|
30
|
+
const EVENT_RING_LIMIT = 64;
|
|
31
|
+
/** Events carried by one snapshot (the panel shows these). */
|
|
32
|
+
const SNAPSHOT_EVENT_COUNT = 8;
|
|
33
|
+
|
|
28
34
|
/* ── Helpers ─────────────────────────────────────────────────────────── */
|
|
29
35
|
|
|
30
36
|
/**
|
|
@@ -106,6 +112,56 @@ function apply(ctx) {
|
|
|
106
112
|
/** @type {Map<string, TokenBucketStrategy | SlidingWindowStrategy>} */
|
|
107
113
|
const limiters = new Map();
|
|
108
114
|
|
|
115
|
+
// ── Live statistics (read by the status RPC channel) ───────
|
|
116
|
+
// Counters are plain numbers so a snapshot is always JSON-safe.
|
|
117
|
+
const totals = {
|
|
118
|
+
requests: 0,
|
|
119
|
+
granted: 0,
|
|
120
|
+
rejected: 0,
|
|
121
|
+
timeouts: 0,
|
|
122
|
+
aborted: 0,
|
|
123
|
+
totalWaitMs: 0,
|
|
124
|
+
};
|
|
125
|
+
/** Bumped on every recorded event and on reset — the panel's change cursor. */
|
|
126
|
+
let rev = 0;
|
|
127
|
+
/** Bounded ring of recent events (newest last). */
|
|
128
|
+
const eventRing = [];
|
|
129
|
+
|
|
130
|
+
function recordEvent(event) {
|
|
131
|
+
eventRing.push({ ...event, ts: Date.now() });
|
|
132
|
+
if (eventRing.length > EVENT_RING_LIMIT) eventRing.shift();
|
|
133
|
+
rev += 1;
|
|
134
|
+
}
|
|
135
|
+
|
|
136
|
+
/** Build the JSON-safe snapshot served on the "snapshot" endpoint. */
|
|
137
|
+
function buildSnapshot() {
|
|
138
|
+
const models = {};
|
|
139
|
+
for (const [key, limiter] of limiters) {
|
|
140
|
+
try {
|
|
141
|
+
models[key] = limiter.getStatus();
|
|
142
|
+
} catch {
|
|
143
|
+
// A strategy that cannot report status must not break the whole panel.
|
|
144
|
+
}
|
|
145
|
+
}
|
|
146
|
+
return {
|
|
147
|
+
rev,
|
|
148
|
+
ts: Date.now(),
|
|
149
|
+
enabled: cfg.enabled !== false,
|
|
150
|
+
strategy: cfg.strategy,
|
|
151
|
+
onThrottled: cfg.onThrottled,
|
|
152
|
+
totals: { ...totals },
|
|
153
|
+
models,
|
|
154
|
+
events: eventRing.slice(-SNAPSHOT_EVENT_COUNT),
|
|
155
|
+
};
|
|
156
|
+
}
|
|
157
|
+
|
|
158
|
+
/** Zero every counter and drop retained events (the panel's clear button). */
|
|
159
|
+
function resetTotals() {
|
|
160
|
+
for (const key of Object.keys(totals)) totals[key] = 0;
|
|
161
|
+
eventRing.length = 0;
|
|
162
|
+
rev += 1;
|
|
163
|
+
}
|
|
164
|
+
|
|
109
165
|
// ── Settings wiring (dynamic inject) ───────────────────────
|
|
110
166
|
// Wrapped in ctx.inject so the plugin works on older DSH versions that
|
|
111
167
|
// don't have a settings service at all — the callback simply never runs.
|
|
@@ -128,6 +184,92 @@ function apply(ctx) {
|
|
|
128
184
|
}, "rate-limiter: settings scope teardown");
|
|
129
185
|
});
|
|
130
186
|
|
|
187
|
+
// ── Status channel (dynamic inject) ────────────────────────
|
|
188
|
+
// Same defensive shape as the settings wiring: on a host without a
|
|
189
|
+
// connection service the callback never runs and the plugin stays silent.
|
|
190
|
+
// Either path below registers inside an effect owned by this fiber, so
|
|
191
|
+
// unloading the plugin withdraws the channel automatically.
|
|
192
|
+
//
|
|
193
|
+
// Two mounting paths exist, and both speak the same wire protocol, so the
|
|
194
|
+
// browser half never needs to know which one is live:
|
|
195
|
+
//
|
|
196
|
+
// 1. connection.rpc.handle(channel, handler) — the framework's generic
|
|
197
|
+
// channel registry (preferred: the framework owns the route, the
|
|
198
|
+
// request validation, and the withdrawal).
|
|
199
|
+
//
|
|
200
|
+
// 2. a self-registered prefix route reusing connection.requestRejection()
|
|
201
|
+
// — the fallback for hosts where path 1 is broken. DSH 0.1.5-rc.3
|
|
202
|
+
// changed dsh-client-connection's own `inject` from
|
|
203
|
+
// ["webServer", "credentials"] to ["credentials"] while its
|
|
204
|
+
// HostConnectionService.register() still dereferences
|
|
205
|
+
// `owner.webServer`, and Cordis rebinds a cross-fiber service's `ctx`
|
|
206
|
+
// to the *reader's* fiber — so rpc.handle() throws
|
|
207
|
+
// 'cannot get property "webServer" without inject'. Registering the
|
|
208
|
+
// route on our own fiber (which injects webServer below) avoids that
|
|
209
|
+
// while still using the framework's own 403/401 fence.
|
|
210
|
+
ctx.inject(["connection"], (scopedCtx) => {
|
|
211
|
+
const connection = scopedCtx.get("connection");
|
|
212
|
+
const channel = createStatusChannel({ buildSnapshot, resetTotals });
|
|
213
|
+
|
|
214
|
+
// ── Path 1: the framework's generic channel registry ──
|
|
215
|
+
const handle = typeof connection?.rpc?.handle === "function"
|
|
216
|
+
? connection.rpc.handle.bind(connection.rpc)
|
|
217
|
+
: undefined;
|
|
218
|
+
|
|
219
|
+
if (handle !== undefined) {
|
|
220
|
+
try {
|
|
221
|
+
scopedCtx.effect(() => {
|
|
222
|
+
const unregister = handle(CHANNEL, channel);
|
|
223
|
+
return () => { unregister(); };
|
|
224
|
+
}, "rate-limiter: status rpc channel");
|
|
225
|
+
logger?.info("rate limiter status channel mounted via connection.rpc (%s)", CHANNEL);
|
|
226
|
+
return;
|
|
227
|
+
} catch (err) {
|
|
228
|
+
logger?.warn(
|
|
229
|
+
"connection.rpc channel registration failed (%s); falling back to a self-registered route",
|
|
230
|
+
err instanceof Error ? err.message : String(err),
|
|
231
|
+
);
|
|
232
|
+
}
|
|
233
|
+
}
|
|
234
|
+
|
|
235
|
+
// ── Path 2: self-registered route (needs webServer on THIS fiber) ──
|
|
236
|
+
// The fence is mandatory: without connection.requestRejection() we would
|
|
237
|
+
// be publishing an unauthenticated route, so we refuse to mount at all.
|
|
238
|
+
// Probing once here turns a broken fence into a mount-time refusal instead
|
|
239
|
+
// of a per-request surprise.
|
|
240
|
+
try {
|
|
241
|
+
if (typeof connection?.requestRejection !== "function") {
|
|
242
|
+
logger?.warn("connection.requestRejection unavailable; status channel not mounted");
|
|
243
|
+
return;
|
|
244
|
+
}
|
|
245
|
+
connection.requestRejection({ headers: {} });
|
|
246
|
+
} catch (err) {
|
|
247
|
+
logger?.warn(
|
|
248
|
+
"connection.requestRejection unusable (%s); refusing to mount an unfenced status route",
|
|
249
|
+
err instanceof Error ? err.message : String(err),
|
|
250
|
+
);
|
|
251
|
+
return;
|
|
252
|
+
}
|
|
253
|
+
|
|
254
|
+
scopedCtx.inject(["webServer"], (webCtx) => {
|
|
255
|
+
const webServer = webCtx.get("webServer");
|
|
256
|
+
if (typeof webServer?.register !== "function") return; // no web carrier: degrade silently
|
|
257
|
+
|
|
258
|
+
try {
|
|
259
|
+
webCtx.effect(() => webServer.register(createChannelRoute({
|
|
260
|
+
channel: CHANNEL,
|
|
261
|
+
handler: channel,
|
|
262
|
+
reject: (req) => connection.requestRejection(req),
|
|
263
|
+
})), "rate-limiter: status route (self-registered)");
|
|
264
|
+
} catch (err) {
|
|
265
|
+
logger?.warn("status channel registration failed: %s", err instanceof Error ? err.message : String(err));
|
|
266
|
+
return;
|
|
267
|
+
}
|
|
268
|
+
|
|
269
|
+
logger?.info("rate limiter status channel mounted via self-registered route (%s)", CHANNEL);
|
|
270
|
+
});
|
|
271
|
+
});
|
|
272
|
+
|
|
131
273
|
// ── LLM stream interceptor ─────────────────────────────────
|
|
132
274
|
// ctx.on('llm/stream') doesn't need the settings service — it's a plain
|
|
133
275
|
// waterfall listener that reads the shared `cfg` variable (initialised
|
|
@@ -147,6 +289,7 @@ function apply(ctx) {
|
|
|
147
289
|
}
|
|
148
290
|
|
|
149
291
|
const limiter = ensureLimiter(limiters, modelKey, cfg);
|
|
292
|
+
totals.requests += 1;
|
|
150
293
|
|
|
151
294
|
// ── Phase 1: acquire ────────────────────────────────────────
|
|
152
295
|
let slotTaken = false;
|
|
@@ -156,12 +299,16 @@ function apply(ctx) {
|
|
|
156
299
|
// reject mode still enforces the frequency limit, not just concurrency.
|
|
157
300
|
const permit = limiter.acquireNonBlocking();
|
|
158
301
|
if (!permit.granted) {
|
|
302
|
+
totals.rejected += 1;
|
|
303
|
+
recordEvent({ event: "rejected", model: modelKey });
|
|
159
304
|
logger?.warn("model %C rejected — rate limit reached", modelKey);
|
|
160
305
|
yield terminalChunk(modelKey, options.signal);
|
|
161
306
|
return;
|
|
162
307
|
}
|
|
163
308
|
// permit granted → slot already taken by acquireNonBlocking; skip acquireSlot.
|
|
164
309
|
slotTaken = true;
|
|
310
|
+
totals.granted += 1;
|
|
311
|
+
recordEvent({ event: "granted", model: modelKey, waitMs: 0 });
|
|
165
312
|
} else {
|
|
166
313
|
// Queue mode — park until a slot opens.
|
|
167
314
|
const permit = await limiter.acquire({
|
|
@@ -169,9 +316,17 @@ function apply(ctx) {
|
|
|
169
316
|
timeoutMs: cfg.maxQueueWaitMs ?? 60_000,
|
|
170
317
|
});
|
|
171
318
|
if (!permit.granted) {
|
|
319
|
+
const aborted = options.signal?.aborted === true;
|
|
320
|
+
if (aborted) {
|
|
321
|
+
totals.aborted += 1;
|
|
322
|
+
recordEvent({ event: "aborted", model: modelKey, waitMs: permit.waitMs ?? 0 });
|
|
323
|
+
} else {
|
|
324
|
+
totals.timeouts += 1;
|
|
325
|
+
recordEvent({ event: "timeout", model: modelKey, waitMs: permit.waitMs ?? 0 });
|
|
326
|
+
}
|
|
172
327
|
logger?.warn(
|
|
173
328
|
"model %C %s after %d ms",
|
|
174
|
-
modelKey,
|
|
329
|
+
modelKey, aborted ? "aborted" : "queue timeout", permit.waitMs,
|
|
175
330
|
);
|
|
176
331
|
yield terminalChunk(modelKey, options.signal);
|
|
177
332
|
return;
|
|
@@ -179,6 +334,9 @@ function apply(ctx) {
|
|
|
179
334
|
// Queue mode: acquire() does NOT take a concurrent slot.
|
|
180
335
|
// We must call acquireSlot() before the actual call.
|
|
181
336
|
slotTaken = false;
|
|
337
|
+
totals.granted += 1;
|
|
338
|
+
totals.totalWaitMs += permit.waitMs ?? 0;
|
|
339
|
+
recordEvent({ event: "granted", model: modelKey, waitMs: permit.waitMs ?? 0 });
|
|
182
340
|
}
|
|
183
341
|
|
|
184
342
|
// ── Phase 2: run the real LLM call ─────────────────────────
|
|
@@ -187,6 +345,7 @@ function apply(ctx) {
|
|
|
187
345
|
yield* next();
|
|
188
346
|
} finally {
|
|
189
347
|
limiter.releaseSlot();
|
|
348
|
+
recordEvent({ event: "completed", model: modelKey });
|
|
190
349
|
}
|
|
191
350
|
});
|
|
192
351
|
|
|
@@ -0,0 +1,285 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* dsh-llm-rate-limiter — status RPC channel (Host half).
|
|
3
|
+
*
|
|
4
|
+
* Exposes live rate-limiter statistics to the browser over the framework's
|
|
5
|
+
* generic Connection RPC channel. The framework supplies POST+JSON transport,
|
|
6
|
+
* the Host/Origin fence (403) and browser authentication (401); this module
|
|
7
|
+
* only answers endpoints and shapes envelopes.
|
|
8
|
+
*
|
|
9
|
+
* Two mounting paths exist, and both speak the *same* wire protocol so the
|
|
10
|
+
* browser half never needs to know which one is active:
|
|
11
|
+
*
|
|
12
|
+
* 1. `connection.rpc.handle(channel, handler)` — the framework's own generic
|
|
13
|
+
* channel registry. Preferred: the framework owns the route, the request
|
|
14
|
+
* validation, and the fiber-scoped withdrawal.
|
|
15
|
+
*
|
|
16
|
+
* 2. `createChannelRoute({ channel, handler, reject })` — a self-registered
|
|
17
|
+
* prefix route for hosts where path 1 is broken. DSH 0.1.5-rc.3 changed
|
|
18
|
+
* `@deepseek-ai/dsh-client-connection`'s own `inject` from
|
|
19
|
+
* `["webServer", "credentials"]` to `["credentials"]` while
|
|
20
|
+
* `HostConnectionService.register()` still dereferences
|
|
21
|
+
* `owner.webServer`, so `rpc.handle` throws
|
|
22
|
+
* `cannot get property "webServer" without inject`. This route is
|
|
23
|
+
* registered on our own fiber (which *can* see `webServer`) and reuses the
|
|
24
|
+
* connection service's public `requestRejection()` fence, so the 403/401
|
|
25
|
+
* policy is still the framework's.
|
|
26
|
+
*
|
|
27
|
+
* Envelope contract (mirrors the Connection wire schema):
|
|
28
|
+
* success: { ok: true, value: <JSON-safe> }
|
|
29
|
+
* failure: { ok: false, error: { code, message, details } }
|
|
30
|
+
*
|
|
31
|
+
* @module dsh-llm-rate-limiter/status-rpc
|
|
32
|
+
*/
|
|
33
|
+
|
|
34
|
+
/**
|
|
35
|
+
* Channel name. Must satisfy the framework's `assertChannel` pattern
|
|
36
|
+
* `/^\/[A-Za-z0-9._~-]+$/` and must not be the reserved `/api` channel.
|
|
37
|
+
*/
|
|
38
|
+
const CHANNEL = "/llm-rate-limiter";
|
|
39
|
+
|
|
40
|
+
/**
|
|
41
|
+
* Endpoint grammar for one path segment, copied verbatim from the framework's
|
|
42
|
+
* `ENDPOINT_SEGMENT_PATTERN` in dsh-client-connection. Keeping it identical is
|
|
43
|
+
* what makes the self-registered route accept and reject exactly the same URLs
|
|
44
|
+
* as `rpc.handle` would.
|
|
45
|
+
*/
|
|
46
|
+
const ENDPOINT_SEGMENT_PATTERN = /^[A-Za-z0-9_$.-]+$/;
|
|
47
|
+
|
|
48
|
+
/**
|
|
49
|
+
* Buffered-body cap for the self-registered route. Both endpoints are called
|
|
50
|
+
* with an empty payload, so this is a defensive memory bound rather than a
|
|
51
|
+
* real limit (the framework uses its own, much larger, carrier cap).
|
|
52
|
+
*/
|
|
53
|
+
const MAX_REQUEST_BODY_BYTES = 1024 * 1024;
|
|
54
|
+
|
|
55
|
+
/**
|
|
56
|
+
* Correlation id echoed when the request envelope cannot be parsed at all —
|
|
57
|
+
* the same literal the framework uses (`INVALID_REQUEST_RPC_ID`).
|
|
58
|
+
*/
|
|
59
|
+
const INVALID_REQUEST_RPC_ID = "invalid-request";
|
|
60
|
+
|
|
61
|
+
/**
|
|
62
|
+
* Build a failure envelope. Field-for-field the same shape the Connection
|
|
63
|
+
* host accepts, so callers can rely on `ok` discrimination alone.
|
|
64
|
+
* @param {string} code - stable machine-readable error code.
|
|
65
|
+
* @param {string} message - human-readable detail.
|
|
66
|
+
* @param {Record<string, unknown>} [details] - optional structured context.
|
|
67
|
+
* @returns {{ ok: false, error: { code: string, message: string, details: Record<string, unknown> } }}
|
|
68
|
+
*/
|
|
69
|
+
function failure(code, message, details = {}) {
|
|
70
|
+
return { ok: false, error: { code, message, details } };
|
|
71
|
+
}
|
|
72
|
+
|
|
73
|
+
/** Build a success envelope. */
|
|
74
|
+
function success(value) {
|
|
75
|
+
return { ok: true, value };
|
|
76
|
+
}
|
|
77
|
+
|
|
78
|
+
/**
|
|
79
|
+
* Create the channel handler.
|
|
80
|
+
*
|
|
81
|
+
* @param {object} deps
|
|
82
|
+
* @param {() => object} deps.buildSnapshot - live snapshot builder.
|
|
83
|
+
* @param {() => void} deps.resetTotals - zero the counters (and bump `rev`).
|
|
84
|
+
* @returns {(endpoint: string, payload: unknown, signal?: AbortSignal) => Promise<object>}
|
|
85
|
+
*/
|
|
86
|
+
function createStatusChannel({ buildSnapshot, resetTotals }) {
|
|
87
|
+
return async function statusChannel(endpoint) {
|
|
88
|
+
if (endpoint === "snapshot") return success(buildSnapshot());
|
|
89
|
+
if (endpoint === "reset") {
|
|
90
|
+
resetTotals();
|
|
91
|
+
return success({});
|
|
92
|
+
}
|
|
93
|
+
return failure(
|
|
94
|
+
"llm-rate-limiter/unknown-endpoint",
|
|
95
|
+
`unknown endpoint: ${String(endpoint)}`,
|
|
96
|
+
);
|
|
97
|
+
};
|
|
98
|
+
}
|
|
99
|
+
|
|
100
|
+
/* ── self-registered route (the DSH 0.1.5-rc.3 fallback) ───────────────── */
|
|
101
|
+
|
|
102
|
+
/** Narrow an unknown JSON value to a plain object. */
|
|
103
|
+
function isRecord(value) {
|
|
104
|
+
return typeof value === "object" && value !== null && !Array.isArray(value);
|
|
105
|
+
}
|
|
106
|
+
|
|
107
|
+
/**
|
|
108
|
+
* Extract the endpoint from a channel-relative pathname, mirroring the
|
|
109
|
+
* framework's `endpointFromPath`: a single `channel/<endpoint>` segment whose
|
|
110
|
+
* segments all satisfy {@link ENDPOINT_SEGMENT_PATTERN}.
|
|
111
|
+
*
|
|
112
|
+
* @param {string} channel - the channel prefix (e.g. `/llm-rate-limiter`).
|
|
113
|
+
* @param {string} pathname - request pathname.
|
|
114
|
+
* @returns {string | undefined} the endpoint, or undefined when the URL is not
|
|
115
|
+
* a well-formed member of this channel.
|
|
116
|
+
*/
|
|
117
|
+
function endpointFromPath(channel, pathname) {
|
|
118
|
+
if (!pathname.startsWith(`${channel}/`)) return undefined;
|
|
119
|
+
const endpoint = pathname.slice(channel.length + 1);
|
|
120
|
+
const segments = endpoint.split("/");
|
|
121
|
+
if (segments.some((segment) => segment === "" || segment === "." || segment === ".." || !ENDPOINT_SEGMENT_PATTERN.test(segment))) return undefined;
|
|
122
|
+
return endpoint;
|
|
123
|
+
}
|
|
124
|
+
|
|
125
|
+
/**
|
|
126
|
+
* Read and decode the request body, refusing anything above `limit` bytes so a
|
|
127
|
+
* hostile caller cannot make the host buffer without bound.
|
|
128
|
+
*
|
|
129
|
+
* @param {import("node:http").IncomingMessage} req - node request stream.
|
|
130
|
+
* @param {number} limit - maximum accepted byte count.
|
|
131
|
+
* @returns {Promise<string>} the decoded UTF-8 body.
|
|
132
|
+
* @throws {Error & { code: "TOO_LARGE" }} when the body exceeds `limit`.
|
|
133
|
+
*/
|
|
134
|
+
async function readBody(req, limit) {
|
|
135
|
+
const chunks = [];
|
|
136
|
+
let received = 0;
|
|
137
|
+
for await (const chunk of req) {
|
|
138
|
+
const buffer = typeof chunk === "string" ? Buffer.from(chunk) : chunk;
|
|
139
|
+
received += buffer.byteLength;
|
|
140
|
+
if (received > limit) {
|
|
141
|
+
const error = new Error("request body too large");
|
|
142
|
+
error.code = "TOO_LARGE";
|
|
143
|
+
throw error;
|
|
144
|
+
}
|
|
145
|
+
chunks.push(buffer);
|
|
146
|
+
}
|
|
147
|
+
return Buffer.concat(chunks).toString("utf8");
|
|
148
|
+
}
|
|
149
|
+
|
|
150
|
+
/**
|
|
151
|
+
* Write one `server-response` envelope as JSON — byte-for-byte the shape
|
|
152
|
+
* `Response.json(...)` produces on the framework's own route.
|
|
153
|
+
*
|
|
154
|
+
* @param {import("node:http").ServerResponse} res - response to own.
|
|
155
|
+
* @param {string} rpcId - correlation id to echo.
|
|
156
|
+
* @param {object} result - the `{ ok, value }` / `{ ok, error }` envelope.
|
|
157
|
+
*/
|
|
158
|
+
function sendEnvelope(res, rpcId, result) {
|
|
159
|
+
res.writeHead(200, { "content-type": "application/json" });
|
|
160
|
+
res.end(JSON.stringify({ type: "server-response", rpcId, result }));
|
|
161
|
+
}
|
|
162
|
+
|
|
163
|
+
/**
|
|
164
|
+
* Build a webserver prefix route that speaks the Connection RPC wire protocol.
|
|
165
|
+
*
|
|
166
|
+
* This is the fallback carrier for {@link createStatusChannel} on hosts where
|
|
167
|
+
* `connection.rpc.handle` throws. Every request-shape decision mirrors the
|
|
168
|
+
* framework's `rpcFetchHandler` so the browser half is byte-compatible with the
|
|
169
|
+
* preferred path:
|
|
170
|
+
*
|
|
171
|
+
* | condition | response |
|
|
172
|
+
* |---|---|
|
|
173
|
+
* | fence/auth rejected by `reject` | `403` (or `401`) + `forbidden`/`unauthorized` |
|
|
174
|
+
* | non-POST, or URL outside the channel | `404 not found` |
|
|
175
|
+
* | `content-type` not `application/json` | `415` |
|
|
176
|
+
* | body over {@link MAX_REQUEST_BODY_BYTES} | `413` |
|
|
177
|
+
* | body not JSON | `400 body is not JSON` |
|
|
178
|
+
* | envelope not a `client-request` | `gateway/bad-request` envelope |
|
|
179
|
+
* | `method` ≠ endpoint | `gateway/bad-request` envelope |
|
|
180
|
+
* | handler threw | `500 handler failure: ...` |
|
|
181
|
+
* | otherwise | `200` + `server-response` envelope |
|
|
182
|
+
*
|
|
183
|
+
* @param {object} deps
|
|
184
|
+
* @param {string} deps.channel - channel prefix to own (e.g. `/llm-rate-limiter`).
|
|
185
|
+
* @param {(endpoint: string, payload: unknown, signal?: AbortSignal) => Promise<object>} deps.handler - endpoint dispatcher.
|
|
186
|
+
* @param {(req: import("node:http").IncomingMessage) => number | undefined} deps.reject - the framework's `connection.requestRejection` fence; returns an HTTP status to refuse with, or undefined to allow.
|
|
187
|
+
* @returns {{ kind: "prefix", path: string, handler: (req: import("node:http").IncomingMessage, res: import("node:http").ServerResponse) => Promise<void> }}
|
|
188
|
+
*/
|
|
189
|
+
function createChannelRoute({ channel, handler, reject }) {
|
|
190
|
+
return {
|
|
191
|
+
kind: "prefix",
|
|
192
|
+
path: channel,
|
|
193
|
+
handler: async (req, res) => {
|
|
194
|
+
// 1. The framework's own fence: Host/Origin trust (403) then browser
|
|
195
|
+
// authentication (401). Reusing it keeps the security policy identical.
|
|
196
|
+
const rejection = reject(req);
|
|
197
|
+
if (rejection !== undefined) {
|
|
198
|
+
res.writeHead(rejection);
|
|
199
|
+
res.end(rejection === 401 ? "unauthorized" : "forbidden");
|
|
200
|
+
return;
|
|
201
|
+
}
|
|
202
|
+
|
|
203
|
+
// 2. POST-only, and only inside this channel (framework: 404 otherwise).
|
|
204
|
+
const url = new URL(req.url ?? "/", "http://dsh.internal");
|
|
205
|
+
const endpoint = endpointFromPath(channel, url.pathname);
|
|
206
|
+
if (req.method !== "POST" || endpoint === undefined) {
|
|
207
|
+
res.writeHead(404);
|
|
208
|
+
res.end("not found");
|
|
209
|
+
return;
|
|
210
|
+
}
|
|
211
|
+
|
|
212
|
+
// 3. JSON only (framework: 415).
|
|
213
|
+
const mediaType = String(req.headers["content-type"] ?? "").split(";", 1)[0].trim().toLowerCase();
|
|
214
|
+
if (mediaType !== "application/json") {
|
|
215
|
+
res.writeHead(415);
|
|
216
|
+
res.end("content type must be application/json");
|
|
217
|
+
return;
|
|
218
|
+
}
|
|
219
|
+
|
|
220
|
+
// 4. Buffer the body under a defensive cap, then parse it.
|
|
221
|
+
let raw;
|
|
222
|
+
try {
|
|
223
|
+
raw = await readBody(req, MAX_REQUEST_BODY_BYTES);
|
|
224
|
+
} catch (error) {
|
|
225
|
+
if (error?.code === "TOO_LARGE") {
|
|
226
|
+
res.writeHead(413, { connection: "close" });
|
|
227
|
+
res.end();
|
|
228
|
+
req.destroy?.();
|
|
229
|
+
return;
|
|
230
|
+
}
|
|
231
|
+
throw error;
|
|
232
|
+
}
|
|
233
|
+
let body;
|
|
234
|
+
try {
|
|
235
|
+
body = JSON.parse(raw);
|
|
236
|
+
} catch {
|
|
237
|
+
res.writeHead(400);
|
|
238
|
+
res.end("body is not JSON");
|
|
239
|
+
return;
|
|
240
|
+
}
|
|
241
|
+
|
|
242
|
+
// 5. Validate the client-request envelope; echo the correlation id
|
|
243
|
+
// whenever it is a string, exactly like the framework does.
|
|
244
|
+
const rpcId = typeof body?.rpcId === "string" ? body.rpcId : INVALID_REQUEST_RPC_ID;
|
|
245
|
+
if (!isRecord(body) || body.type !== "client-request" || typeof body.rpcId !== "string" || typeof body.method !== "string") {
|
|
246
|
+
sendEnvelope(res, rpcId, failure("gateway/bad-request", "invalid client-request message", { issues: [] }));
|
|
247
|
+
return;
|
|
248
|
+
}
|
|
249
|
+
if (body.method !== endpoint) {
|
|
250
|
+
sendEnvelope(res, rpcId, failure(
|
|
251
|
+
"gateway/bad-request",
|
|
252
|
+
`method ${JSON.stringify(body.method)} does not match endpoint ${JSON.stringify(endpoint)}`,
|
|
253
|
+
{ issues: [] },
|
|
254
|
+
));
|
|
255
|
+
return;
|
|
256
|
+
}
|
|
257
|
+
|
|
258
|
+
// 6. Dispatch. Tie the signal to the response lifetime, as the framework's
|
|
259
|
+
// http bridge does, so a dropped client can cancel the work.
|
|
260
|
+
const abort = new AbortController();
|
|
261
|
+
res.on?.("close", () => {
|
|
262
|
+
if (!res.writableEnded) abort.abort();
|
|
263
|
+
});
|
|
264
|
+
let result;
|
|
265
|
+
try {
|
|
266
|
+
result = await handler(endpoint, body.payload, abort.signal);
|
|
267
|
+
} catch (error) {
|
|
268
|
+
res.writeHead(500);
|
|
269
|
+
res.end(`handler failure: ${String(error)}`);
|
|
270
|
+
return;
|
|
271
|
+
}
|
|
272
|
+
sendEnvelope(res, rpcId, result);
|
|
273
|
+
},
|
|
274
|
+
};
|
|
275
|
+
}
|
|
276
|
+
|
|
277
|
+
export {
|
|
278
|
+
CHANNEL,
|
|
279
|
+
MAX_REQUEST_BODY_BYTES,
|
|
280
|
+
createChannelRoute,
|
|
281
|
+
createStatusChannel,
|
|
282
|
+
endpointFromPath,
|
|
283
|
+
failure,
|
|
284
|
+
success,
|
|
285
|
+
};
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@leaf233/dsh-llm-rate-limiter",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"description": "Per-model LLM call rate limiter for DeepSeek Harness with queue support",
|
|
3
|
+
"version": "0.2.1",
|
|
4
|
+
"description": "Per-model LLM call rate limiter for DeepSeek Harness with queue support and a live status panel",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "lib/index.js",
|
|
7
7
|
"exports": {
|
|
@@ -14,14 +14,22 @@
|
|
|
14
14
|
},
|
|
15
15
|
"client": {
|
|
16
16
|
"inject": [
|
|
17
|
+
"@deepseek-ai/dsh-client-connection",
|
|
17
18
|
"@deepseek-ai/dsh-client-ui-settings"
|
|
18
19
|
],
|
|
19
20
|
"platform": "web"
|
|
21
|
+
},
|
|
22
|
+
"compatibility": {
|
|
23
|
+
"dshReleases": {
|
|
24
|
+
"0.1.2-rc.1": "compatible",
|
|
25
|
+
"0.1.5-rc.3": "compatible"
|
|
26
|
+
}
|
|
20
27
|
}
|
|
21
28
|
},
|
|
22
29
|
"scripts": {
|
|
23
30
|
"build": "echo 'no build needed'",
|
|
24
|
-
"test": "node test-strategies.mjs"
|
|
31
|
+
"test": "node test-strategies.mjs && node test-status-rpc.mjs && node test-client-bundle.mjs && node test-host-integration.mjs && node test-status-route.mjs",
|
|
32
|
+
"verify:live": "node tools/verify-live-route.mjs"
|
|
25
33
|
},
|
|
26
34
|
"repository": {
|
|
27
35
|
"type": "git",
|