@leaf233/dsh-llm-rate-limiter 0.1.1 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -4,6 +4,25 @@ All notable changes to this project will be documented in this file.
4
4
 
5
5
  The format is based on [Keep a Changelog](https://keepachangelog.com/).
6
6
 
7
+ ## [0.2.0] - 2026-09-13
8
+
9
+ ### Added
10
+ - **Live status panel** in the settings card: request/granted/rejected/timeout/aborted counters, average wait, per-model progress bars (token balance or window occupancy), concurrency and queue badges, and a rolling event log
11
+ - **`connection.rpc` status channel** at `/llm-rate-limiter` with `snapshot` and `reset` endpoints — the framework supplies POST+JSON transport, the Host/Origin fence (403) and browser authentication (401), and withdraws the channel with the plugin fiber
12
+ - **"清零" button** that zeroes all statistics and drops retained events
13
+ - `lib/status-rpc.js` — channel handler and envelope helpers (`success` / `failure`)
14
+ - `test-status-rpc.mjs` (61 assertions): envelopes, endpoint routing, JSON-safety, ring-buffer bounds, every counter path
15
+ - `test-client-bundle.mjs` (70 assertions): loads the real client bundle in a VM and mounts the card and panel on a miniature React runtime — covers live rendering, collapse-stops-polling, and every degraded phase
16
+
17
+ ### Changed
18
+ - `dsh.client.inject` now also requires `@deepseek-ai/dsh-client-connection`, so the connection service is available before the panel mounts
19
+ - `dsh.compatibility.dshReleases` declares the verified DSH release (aligned with dsh-context's convention)
20
+ - CI runs the two new suites and syntax-checks `lib/status-rpc.js`
21
+
22
+ ### Notes
23
+ - The status channel is optional: on a host without a connection service it is never registered and the panel shows "状态通道不可用" instead, leaving configuration untouched
24
+ - Statistics are in-memory only and reset when the host restarts
25
+
7
26
  ## [0.1.1] - 2026-09-12
8
27
 
9
28
  ### Changed
package/COMPATIBILITY.md CHANGED
@@ -30,6 +30,19 @@
30
30
  | 5 | `settingsScope.mutate([{ op, path, value }])` | dsh-client-ui-settings | ✅ 1处 | SettingsScopeController.mutate (嵌套操作) |
31
31
  | 6 | `ctx.slots.inject("settings.plugin.item", ...)` | dsh-client-ui-settings-plugins | ✅ 6处 | 插件设置卡片注册 slot |
32
32
 
33
+ ### v0.2.0 新增依赖(状态通道)
34
+
35
+ | # | API | 所属包 | 本地存在 | 稳定性评估 |
36
+ |---|-----|--------|----------|------------|
37
+ | 1 | `ctx.inject(["connection"], cb)` + `conn.rpc.handle(channel, handler)` | dsh-client-connection | ✅ `HostConnectionService.register`(lib/index.js 572-589) | **公开通用 API**;注册包在 `owner.effect` 中,卸载自动撤通道。通道名须匹配 `/^\/[A-Za-z0-9._~-]+$/` 且非 `/api` |
38
+ | 2 | handler 签名 `(endpoint, payload, signal) => envelope` | dsh-client-connection | ✅ `rpcFetchHandler`(605-631) | 框架强制 POST + `application/json`(否则 404/415),并自动套 `requestRejection`(Host/Origin 不信→403,未认证→401) |
39
+ | 3 | envelope `{ok:true,value}` / `{ok:false,error:{code,message,details}}` | dsh-client-connection | ✅ 与 `clientRequestSchema` / `errorResponse` 一致 | 契约字面量,dsh-context 逐字复制同一范式 |
40
+ | 4 | 客户端 `ctx.get("connection")?.rpc.call(channel, endpoint, payload, signal)` | dsh-client-connection | ✅ `createWebConnectionRpc.call`(client.js 4606-4628) | 传输失败 reject;返回 `result`(即 envelope)。经 `rpcCallOf` 反射读取,服务缺席/敌意时降级为 `undefined` |
41
+ | 5 | `dsh.client.inject` 声明 `@deepseek-ai/dsh-client-connection` | DSH web shell | ✅ | bundle 依赖声明,确保连接服务先于本插件可用;**不放进模块级 `inject`**,以便服务缺席时面板降级而非整卡不加载 |
42
+ | 6 | `dsh.compatibility.dshReleases` 声明 | DSH 打包约定 | ✅ 对齐 `dsh-context` | 成熟插件物料约定 |
43
+
44
+ > 证据:dsh-context v0.50.0 使用同一组 API(`watchDetailChannel`,lib/index.js 1943-2012),其 compatibility 矩阵覆盖 DSH `0.1.2-rc.1` → `0.1.5-rc.1`,说明 `connection.rpc` 是**持久公开契约**而非临时接口。
45
+
33
46
  ### Schema 依赖 — 1 个包
34
47
 
35
48
  | 包 | 本地版本 | 用到的 API |
@@ -130,4 +143,8 @@
130
143
  - [ ] `settings.register` 返回值结构是否变化
131
144
  - [ ] `settingsScope.set/mutate` 接口是否变化
132
145
  - [ ] `settings.plugin.item` slot 是否仍可注入
146
+ - [ ] **状态通道(v0.2.0)**:`connection.rpc.handle` 仍为公开 API 且通道名规则未变
147
+ - [ ] **状态通道(v0.2.0)**:`snapshot` / `reset` 端点往返成功(面板显示"● 实时"而非"状态通道不可用")
148
+ - [ ] **状态通道(v0.2.0)**:折叠卡片后 DevTools Network 无 `/llm-rate-limiter/snapshot` 轮询
149
+ - [ ] **状态通道(v0.2.0)**:无 connection 服务的组合下,卡片配置区仍可用且面板显示降级文案
133
150
  4. **如果 cordis 升级到 5.0+**:整个 `apply(ctx)` 接口、`ctx.effect()`、`ctx.on()` 签名可能重写,需要全面适配
package/README.md CHANGED
@@ -17,9 +17,39 @@ Per-model LLM call rate limiter for [DeepSeek Harness](https://github.com/deepse
17
17
  - **Queue mode** — throttled requests wait in queue and are released when a slot opens
18
18
  - **Reject mode** — throttled requests fail immediately (integrates with `dsh-llm-retry` for auto-backoff)
19
19
  - **Interactive GUI** — collapsible card in DSH Settings → Plugins → Configurable
20
+ - **Live status panel** — real-time counters, per-model progress bars and an event log (v0.2.0)
20
21
  - **Hot-reload** — settings changes take effect immediately, no restart needed
21
22
  - **Every request checked** — intercepts `llm/stream` waterfall, covering every LLM call in every agent turn
22
23
 
24
+ ---
25
+
26
+ ## Status Panel (v0.2.0)
27
+
28
+ Expanding the card shows a live panel at the top of its body:
29
+
30
+ ```
31
+ 📊 实时状态 ● 实时 [清零]
32
+ 请求 42 · 通过 38 · 拒绝 2 · 超时 1 · 中止 1 · 平均等待 214ms
33
+ deepseek/deepseek-chat [令牌桶] ▓▓▓▓▓▓▓░░░ 7.5/10 并发 2/5 排队 1
34
+ openai/gpt-4o [滑动窗口] ▓▓▓▓▓▓▓▓▓▓ 3/3 rpm 并发 1/5
35
+ 12:00:03 timeout openai/gpt-4o 等待 1m
36
+ 12:00:01 rejected openai/gpt-4o
37
+ 12:00:00 granted deepseek/deepseek-chat 等待 4.2s
38
+ ```
39
+
40
+ | Aspect | Behaviour |
41
+ |--------|-----------|
42
+ | Data channel | Framework `connection.rpc` channel `/llm-rate-limiter` — authenticated (401/403 fence), POST+JSON, auto-cleaned with the plugin fiber |
43
+ | Cadence | 1 s polling while the card is expanded; backs off 2 s → 4 s → 8 s after failures |
44
+ | Collapsed card | The panel unmounts, so **no polling runs at all** |
45
+ | Endpoints | `snapshot` (live counters) and `reset` (zero the statistics) |
46
+ | Without a channel | Shows "状态通道不可用" and leaves the rest of the card fully functional |
47
+ | Counters | `requests` / `granted` / `rejected` / `timeouts` / `aborted` / `totalWaitMs`, plus the last 8 events (ring buffer of 64) |
48
+ | Progress bars | Token bucket shows `tokens/burstSize`; sliding window shows `countInWindow/maxRpm`; both turn amber as the limit approaches |
49
+
50
+ > 🧠 **From Hindsight memory (dsh-context-host-client)** — the channel idiom is dsh-context's: `ctx.inject(["connection"])` → `conn.rpc.handle(channel, handler)`, with the browser side resolving `ctx.get("connection")?.rpc.call` defensively so a missing service degrades instead of throwing.
51
+
52
+
23
53
  ---
24
54
 
25
55
  ## Installation
package/lib/client.js CHANGED
@@ -53,6 +53,34 @@ window.__ModuleLoader__.load({
53
53
  ".rlr-modelMeta{display:flex;gap:12px;font-size:12px;color:var(--dsw-alias-label-tertiary)}",
54
54
  ".rlr-section{display:flex;flex-direction:column;gap:4px;margin-top:2px}",
55
55
  ".rlr-sectHead{margin:10px 0 0;font-size:14px;font-weight:500}",
56
+ // ── status panel ──
57
+ ".rlr-statusHead{display:flex;align-items:center;gap:8px;margin:10px 0 0}",
58
+ ".rlr-statusTitle{font-size:14px;font-weight:500;flex:1}",
59
+ ".rlr-chips{display:flex;flex-wrap:wrap;gap:6px;margin:8px 0}",
60
+ ".rlr-chip{font-size:11px;padding:2px 8px;border-radius:10px;background:var(--dsw-alias-surface-s2);color:var(--dsw-alias-label-secondary);white-space:nowrap;font-variant-numeric:tabular-nums}",
61
+ ".rlr-chipOk{background:var(--dsw-alias-state-success-primary,#16a34a);color:#fff}",
62
+ ".rlr-chipWarn{background:var(--dsw-alias-state-warning-primary,#d97706);color:#fff}",
63
+ ".rlr-chipErr{background:var(--dsw-alias-state-error-primary,#dc2626);color:#fff}",
64
+ ".rlr-phase{font-size:11px;color:var(--dsw-alias-label-tertiary)}",
65
+ ".rlr-phaseLive{color:var(--dsw-alias-state-success-primary,#16a34a)}",
66
+ ".rlr-phaseFailed{color:var(--dsw-alias-state-error-primary,#dc2626)}",
67
+ ".rlr-models{display:flex;flex-direction:column;gap:4px;margin:6px 0}",
68
+ ".rlr-mRow{display:flex;align-items:center;gap:10px;padding:6px 10px;border-radius:10px;border:1px solid var(--dsw-alias-border-l4)}",
69
+ ".rlr-mName{font-size:12px;font-family:monospace;min-width:0;flex:1;overflow:hidden;text-overflow:ellipsis;white-space:nowrap}",
70
+ ".rlr-mKind{font-size:10px;padding:1px 6px;border-radius:8px;background:var(--dsw-alias-surface-s2);color:var(--dsw-alias-label-tertiary);white-space:nowrap}",
71
+ ".rlr-mMeta{font-size:11px;color:var(--dsw-alias-label-tertiary);white-space:nowrap;font-variant-numeric:tabular-nums}",
72
+ ".rlr-mIdle{opacity:.55}",
73
+ ".rlr-bar{width:90px;height:6px;border-radius:3px;background:var(--dsw-alias-surface-s2);overflow:hidden;flex:none}",
74
+ ".rlr-barFill{height:100%;background:var(--dsw-alias-brand-primary,#4f46e5);transition:width .3s}",
75
+ ".rlr-barFillWarn{background:var(--dsw-alias-state-warning-primary,#d97706)}",
76
+ ".rlr-log{display:flex;flex-direction:column;gap:1px;margin:6px 0;font-family:monospace;font-size:11px}",
77
+ ".rlr-logRow{display:flex;gap:8px;padding:1px 4px;border-radius:4px;color:var(--dsw-alias-label-tertiary)}",
78
+ ".rlr-logRowNew{background:var(--dsw-alias-surface-s2)}",
79
+ ".rlr-logEv{min-width:66px}",
80
+ ".rlr-logEvOk{color:var(--dsw-alias-state-success-primary,#16a34a)}",
81
+ ".rlr-logEvWarn{color:var(--dsw-alias-state-warning-primary,#d97706)}",
82
+ ".rlr-logEvErr{color:var(--dsw-alias-state-error-primary,#dc2626)}",
83
+ ".rlr-logMs{min-width:56px;text-align:right}",
56
84
  ].join("\n");
57
85
  const tagId = "@dsh-llm-rate-limiter/rate-limiter-card.css";
58
86
  if (typeof document !== "undefined" && document.querySelector("style[data-plugin-css=" + JSON.stringify(tagId) + "]") === null) {
@@ -75,8 +103,209 @@ window.__ModuleLoader__.load({
75
103
  scope.mutate([{ op: "unset", path }]);
76
104
  }
77
105
 
106
+ /* ── status channel access (dsh-context defensive idiom) ─────────── */
107
+ const STATUS_CHANNEL = "/llm-rate-limiter";
108
+
109
+ /**
110
+ * The connection's bound generic-RPC caller, or undefined when the
111
+ * service is absent or hostile. Every read is guarded, so this can
112
+ * never throw — a missing channel degrades the panel, not the card.
113
+ */
114
+ function rpcCallOf(ctx) {
115
+ try {
116
+ const rpc = ctx.get("connection")?.rpc;
117
+ const fn = rpc?.call;
118
+ if (rpc && typeof fn === "function") return fn.bind(rpc);
119
+ } catch {}
120
+ return undefined;
121
+ }
122
+
123
+ /** Format a duration in ms for the chips/log (compact). */
124
+ function fmtMs(ms) {
125
+ if (!Number.isFinite(ms) || ms <= 0) return "0";
126
+ if (ms < 1000) return Math.round(ms) + "ms";
127
+ if (ms < 60_000) return (ms / 1000).toFixed(1) + "s";
128
+ return Math.round(ms / 60_000) + "m";
129
+ }
130
+
131
+ /** Human label for a strategy id. */
132
+ function strategyLabel(id) {
133
+ return id === "sliding-window" ? "滑动窗口" : "令牌桶";
134
+ }
135
+
136
+ /** Tunable polling cadence: first retry delay doubles up to 8s. */
137
+ const POLL_MS = 1000;
138
+ const POLL_MAX_MULTIPLIER = 8;
139
+
140
+ /* ── React component: live status panel ──────────────────────────── */
141
+ function RateLimiterStatus({ statusCall, cfg }) {
142
+ const [snap, setSnap] = react.useState(null);
143
+ const [phase, setPhase] = react.useState("connecting");
144
+ const prevRev = react.useRef(-1);
145
+ const [highlight, setHighlight] = react.useState(false);
146
+
147
+ react.useEffect(() => {
148
+ const call = statusCall();
149
+ if (call === undefined) {
150
+ setPhase("no-channel");
151
+ return undefined;
152
+ }
153
+ let stop = false;
154
+ let timer = null;
155
+ let failures = 0;
156
+
157
+ const tick = async () => {
158
+ try {
159
+ const r = await call(STATUS_CHANNEL, "snapshot", {});
160
+ if (stop) return;
161
+ if (r?.ok !== true || !r.value) throw new Error("bad envelope");
162
+ setSnap(r.value);
163
+ setPhase("live");
164
+ failures = 0;
165
+ } catch {
166
+ if (stop) return;
167
+ failures += 1;
168
+ setPhase("failed");
169
+ }
170
+ if (stop) return;
171
+ const multiplier = Math.min(2 ** Math.min(failures, 3), POLL_MAX_MULTIPLIER);
172
+ timer = setTimeout(tick, POLL_MS * multiplier);
173
+ };
174
+
175
+ tick();
176
+ return () => {
177
+ stop = true;
178
+ if (timer) clearTimeout(timer);
179
+ };
180
+ }, [statusCall]);
181
+
182
+ // Flash newly arrived events when `rev` advances between polls.
183
+ react.useEffect(() => {
184
+ if (!snap) return undefined;
185
+ if (prevRev.current === -1) { prevRev.current = snap.rev; return undefined; }
186
+ if (snap.rev === prevRev.current) return undefined;
187
+ prevRev.current = snap.rev;
188
+ setHighlight(true);
189
+ const t = setTimeout(() => setHighlight(false), 700);
190
+ return () => clearTimeout(t);
191
+ }, [snap]);
192
+
193
+ // ── non-live phases ──
194
+ if (phase === "no-channel") {
195
+ return jsx("div", { className: "rlr-status", children: "状态通道不可用(需要 DSH web 组合)" });
196
+ }
197
+ if (phase === "connecting" && !snap) {
198
+ return jsx("div", { className: "rlr-status", children: "正在连接状态通道…" });
199
+ }
200
+ if (!snap) {
201
+ return jsx("div", { className: "rlr-status", children: "连接中断,重试中…" });
202
+ }
203
+
204
+ const t = snap.totals ?? {};
205
+ const avgWait = t.granted > 0 ? (t.totalWaitMs ?? 0) / t.granted : 0;
206
+ const live = phase === "live";
207
+
208
+ function onReset() {
209
+ const call = statusCall();
210
+ if (call === undefined) return;
211
+ Promise.resolve(call(STATUS_CHANNEL, "reset", {})).catch(() => undefined);
212
+ }
213
+
214
+ // Union of configured models and models that have actually been used.
215
+ const snapModels = snap.models ?? {};
216
+ const keys = Array.from(new Set([...Object.keys(cfg.models ?? {}), ...Object.keys(snapModels)]));
217
+
218
+ const chips = [
219
+ { label: "请求 " + (t.requests ?? 0) },
220
+ { label: "通过 " + (t.granted ?? 0), tone: "ok" },
221
+ (t.rejected ?? 0) > 0 && { label: "拒绝 " + t.rejected, tone: "err" },
222
+ (t.timeouts ?? 0) > 0 && { label: "超时 " + t.timeouts, tone: "warn" },
223
+ (t.aborted ?? 0) > 0 && { label: "中止 " + t.aborted, tone: "warn" },
224
+ { label: "平均等待 " + fmtMs(avgWait) },
225
+ ].filter(Boolean);
226
+
227
+ const events = Array.isArray(snap.events) ? snap.events.slice().reverse() : [];
228
+
229
+ return jsxs("div", { className: "rlr-section", children: [
230
+ jsxs("div", { className: "rlr-statusHead", children: [
231
+ jsx("span", { className: "rlr-statusTitle", children: "📊 实时状态" }),
232
+ jsx("span", {
233
+ className: "rlr-phase" + (live ? " rlr-phaseLive" : " rlr-phaseFailed"),
234
+ children: live ? "● 实时" : "○ 重试中",
235
+ }),
236
+ jsx("button", { type: "button", className: "rlr-btnSec", onClick: onReset, children: "清零" }),
237
+ ] }),
238
+
239
+ snap.enabled === false
240
+ ? jsx("div", { className: "rlr-status", children: "限速已停用" })
241
+ : null,
242
+
243
+ jsx("div", { className: "rlr-chips", children: chips.map((c, i) =>
244
+ jsx("span", {
245
+ key: i,
246
+ className: "rlr-chip" + (c.tone === "ok" ? " rlr-chipOk" : c.tone === "warn" ? " rlr-chipWarn" : c.tone === "err" ? " rlr-chipErr" : ""),
247
+ children: c.label,
248
+ }),
249
+ ) }),
250
+
251
+ keys.length === 0
252
+ ? jsx("div", { className: "rlr-status", children: "尚无模型调用记录" })
253
+ : jsx("div", { className: "rlr-models", children: keys.map((key) => {
254
+ const m = snapModels[key];
255
+ if (!m) {
256
+ return jsxs("div", { key, className: "rlr-mRow rlr-mIdle", children: [
257
+ jsx("span", { className: "rlr-mName", children: key }),
258
+ jsx("span", { className: "rlr-mMeta", children: "未调用" }),
259
+ ] });
260
+ }
261
+ const isBucket = m.strategy !== "sliding-window";
262
+ const numerator = isBucket ? (m.tokens ?? 0) : (m.countInWindow ?? 0);
263
+ const denominator = isBucket ? (m.burstSize ?? 0) : (m.maxRpm ?? 0);
264
+ const pct = denominator > 0 ? Math.min(100, Math.round((numerator / denominator) * 100)) : 0;
265
+ // A nearly empty bucket / a nearly full window both mean "close to throttling".
266
+ const nearLimit = isBucket ? pct <= 20 : pct >= 80;
267
+ return jsxs("div", { key, className: "rlr-mRow", children: [
268
+ jsx("span", { className: "rlr-mName", title: key, children: key }),
269
+ jsx("span", { className: "rlr-mKind", children: strategyLabel(m.strategy) }),
270
+ jsx("span", { className: "rlr-bar", children:
271
+ jsx("span", { className: "rlr-barFill" + (nearLimit ? " rlr-barFillWarn" : ""), style: { width: pct + "%" } }),
272
+ }),
273
+ jsx("span", { className: "rlr-mMeta", children:
274
+ (isBucket ? numerator + "/" + denominator : numerator + "/" + denominator + " rpm"),
275
+ }),
276
+ jsx("span", { className: "rlr-mMeta", children: "并发 " + (m.concurrent ?? 0) + "/" + (m.maxConcurrent ?? 0) }),
277
+ (m.queued ?? 0) > 0
278
+ ? jsx("span", { className: "rlr-chip rlr-chipWarn", children: "排队 " + m.queued })
279
+ : null,
280
+ ] });
281
+ }) }),
282
+
283
+ events.length > 0
284
+ ? jsx("div", { className: "rlr-log", children: events.map((e, i) => {
285
+ const good = e.event === "granted" || e.event === "completed";
286
+ const bad = e.event === "rejected";
287
+ const d = new Date(e.ts ?? Date.now());
288
+ const hh = String(d.getHours()).padStart(2, "0");
289
+ const mm = String(d.getMinutes()).padStart(2, "0");
290
+ const ss = String(d.getSeconds()).padStart(2, "0");
291
+ return jsxs("div", {
292
+ className: "rlr-logRow" + (highlight && i === 0 ? " rlr-logRowNew" : ""),
293
+ children: [
294
+ jsx("span", { children: hh + ":" + mm + ":" + ss }),
295
+ jsx("span", { className: "rlr-logEv " + (good ? "rlr-logEvOk" : bad ? "rlr-logEvErr" : "rlr-logEvWarn"), children: e.event }),
296
+ jsx("span", { className: "rlr-mName", title: e.model, children: e.model }),
297
+ (e.waitMs ?? 0) > 0
298
+ ? jsx("span", { className: "rlr-logMs", children: "等待 " + fmtMs(e.waitMs) })
299
+ : null,
300
+ ],
301
+ });
302
+ }) })
303
+ : null,
304
+ ] });
305
+ }
306
+
78
307
  /* ── React component: collapsible card ───────────────────────────── */
79
- function RateLimiterCard({ settingsScope }) {
308
+ function RateLimiterCard({ settingsScope, statusCall }) {
80
309
  const [cfg, setCfg] = react.useState(() => {
81
310
  const snap = settingsScope.getSnapshot?.();
82
311
  return snap?.status === "ready" && snap.value ? { ...snap.value } : {};
@@ -129,6 +358,8 @@ window.__ModuleLoader__.load({
129
358
  ]
130
359
  }),
131
360
  open && jsxs("div", { className: "rlr-body", children: [
361
+ /* ── 实时状态面板(折叠即卸载 → 停止轮询) ── */
362
+ jsx(RateLimiterStatus, { statusCall, cfg }),
132
363
  /* ── 启用开关 ── */
133
364
  jsxs("div", { className: "rlr-row", children: [
134
365
  jsx("label", { children: [
@@ -220,7 +451,10 @@ window.__ModuleLoader__.load({
220
451
  name: "settings.plugin.item",
221
452
  key: name,
222
453
  locale: name,
223
- inject: () => ({ settingsScope: scope }),
454
+ // `statusCall` is resolved per render (never cached) so a
455
+ // late-arriving or hot-reloaded connection service is picked
456
+ // up without re-registering the card.
457
+ inject: () => ({ settingsScope: scope, statusCall: () => rpcCallOf(ctx) }),
224
458
  }, RateLimiterCard);
225
459
  });
226
460
  }
package/lib/index.js CHANGED
@@ -21,10 +21,16 @@
21
21
  import { TokenBucketStrategy } from "./strategies/token-bucket.js";
22
22
  import { SlidingWindowStrategy } from "./strategies/sliding-window.js";
23
23
  import { RateLimiterConfig } from "./types/config.js";
24
+ import { CHANNEL, createStatusChannel } from "./status-rpc.js";
24
25
 
25
26
  const name = "llm-rate-limiter";
26
27
  const SETTINGS_NS = "llm-rate-limiter";
27
28
 
29
+ /** Maximum retained events in the status ring (bounded memory). */
30
+ const EVENT_RING_LIMIT = 64;
31
+ /** Events carried by one snapshot (the panel shows these). */
32
+ const SNAPSHOT_EVENT_COUNT = 8;
33
+
28
34
  /* ── Helpers ─────────────────────────────────────────────────────────── */
29
35
 
30
36
  /**
@@ -106,6 +112,56 @@ function apply(ctx) {
106
112
  /** @type {Map<string, TokenBucketStrategy | SlidingWindowStrategy>} */
107
113
  const limiters = new Map();
108
114
 
115
+ // ── Live statistics (read by the status RPC channel) ───────
116
+ // Counters are plain numbers so a snapshot is always JSON-safe.
117
+ const totals = {
118
+ requests: 0,
119
+ granted: 0,
120
+ rejected: 0,
121
+ timeouts: 0,
122
+ aborted: 0,
123
+ totalWaitMs: 0,
124
+ };
125
+ /** Bumped on every recorded event and on reset — the panel's change cursor. */
126
+ let rev = 0;
127
+ /** Bounded ring of recent events (newest last). */
128
+ const eventRing = [];
129
+
130
+ function recordEvent(event) {
131
+ eventRing.push({ ...event, ts: Date.now() });
132
+ if (eventRing.length > EVENT_RING_LIMIT) eventRing.shift();
133
+ rev += 1;
134
+ }
135
+
136
+ /** Build the JSON-safe snapshot served on the "snapshot" endpoint. */
137
+ function buildSnapshot() {
138
+ const models = {};
139
+ for (const [key, limiter] of limiters) {
140
+ try {
141
+ models[key] = limiter.getStatus();
142
+ } catch {
143
+ // A strategy that cannot report status must not break the whole panel.
144
+ }
145
+ }
146
+ return {
147
+ rev,
148
+ ts: Date.now(),
149
+ enabled: cfg.enabled !== false,
150
+ strategy: cfg.strategy,
151
+ onThrottled: cfg.onThrottled,
152
+ totals: { ...totals },
153
+ models,
154
+ events: eventRing.slice(-SNAPSHOT_EVENT_COUNT),
155
+ };
156
+ }
157
+
158
+ /** Zero every counter and drop retained events (the panel's clear button). */
159
+ function resetTotals() {
160
+ for (const key of Object.keys(totals)) totals[key] = 0;
161
+ eventRing.length = 0;
162
+ rev += 1;
163
+ }
164
+
109
165
  // ── Settings wiring (dynamic inject) ───────────────────────
110
166
  // Wrapped in ctx.inject so the plugin works on older DSH versions that
111
167
  // don't have a settings service at all — the callback simply never runs.
@@ -128,6 +184,30 @@ function apply(ctx) {
128
184
  }, "rate-limiter: settings scope teardown");
129
185
  });
130
186
 
187
+ // ── Status RPC channel (dynamic inject) ────────────────────
188
+ // Same defensive shape as the settings wiring: on a host without a
189
+ // connection service the callback never runs and the plugin stays silent.
190
+ // The registration is owned by this fiber, so unloading withdraws it.
191
+ ctx.inject(["connection"], (scopedCtx) => {
192
+ const connection = scopedCtx.get("connection");
193
+ const handle = typeof connection?.rpc?.handle === "function"
194
+ ? connection.rpc.handle.bind(connection.rpc)
195
+ : undefined;
196
+ if (handle === undefined) return; // older host: degrade silently
197
+
198
+ try {
199
+ scopedCtx.effect(() => {
200
+ const unregister = handle(CHANNEL, createStatusChannel({ buildSnapshot, resetTotals }));
201
+ return () => { unregister(); };
202
+ }, "rate-limiter: status rpc channel");
203
+ } catch (err) {
204
+ logger?.warn("status channel registration failed: %s", err instanceof Error ? err.message : String(err));
205
+ return;
206
+ }
207
+
208
+ logger?.info("rate limiter status channel mounted (%s)", CHANNEL);
209
+ });
210
+
131
211
  // ── LLM stream interceptor ─────────────────────────────────
132
212
  // ctx.on('llm/stream') doesn't need the settings service — it's a plain
133
213
  // waterfall listener that reads the shared `cfg` variable (initialised
@@ -147,6 +227,7 @@ function apply(ctx) {
147
227
  }
148
228
 
149
229
  const limiter = ensureLimiter(limiters, modelKey, cfg);
230
+ totals.requests += 1;
150
231
 
151
232
  // ── Phase 1: acquire ────────────────────────────────────────
152
233
  let slotTaken = false;
@@ -156,12 +237,16 @@ function apply(ctx) {
156
237
  // reject mode still enforces the frequency limit, not just concurrency.
157
238
  const permit = limiter.acquireNonBlocking();
158
239
  if (!permit.granted) {
240
+ totals.rejected += 1;
241
+ recordEvent({ event: "rejected", model: modelKey });
159
242
  logger?.warn("model %C rejected — rate limit reached", modelKey);
160
243
  yield terminalChunk(modelKey, options.signal);
161
244
  return;
162
245
  }
163
246
  // permit granted → slot already taken by acquireNonBlocking; skip acquireSlot.
164
247
  slotTaken = true;
248
+ totals.granted += 1;
249
+ recordEvent({ event: "granted", model: modelKey, waitMs: 0 });
165
250
  } else {
166
251
  // Queue mode — park until a slot opens.
167
252
  const permit = await limiter.acquire({
@@ -169,9 +254,17 @@ function apply(ctx) {
169
254
  timeoutMs: cfg.maxQueueWaitMs ?? 60_000,
170
255
  });
171
256
  if (!permit.granted) {
257
+ const aborted = options.signal?.aborted === true;
258
+ if (aborted) {
259
+ totals.aborted += 1;
260
+ recordEvent({ event: "aborted", model: modelKey, waitMs: permit.waitMs ?? 0 });
261
+ } else {
262
+ totals.timeouts += 1;
263
+ recordEvent({ event: "timeout", model: modelKey, waitMs: permit.waitMs ?? 0 });
264
+ }
172
265
  logger?.warn(
173
266
  "model %C %s after %d ms",
174
- modelKey, options.signal?.aborted ? "aborted" : "queue timeout", permit.waitMs,
267
+ modelKey, aborted ? "aborted" : "queue timeout", permit.waitMs,
175
268
  );
176
269
  yield terminalChunk(modelKey, options.signal);
177
270
  return;
@@ -179,6 +272,9 @@ function apply(ctx) {
179
272
  // Queue mode: acquire() does NOT take a concurrent slot.
180
273
  // We must call acquireSlot() before the actual call.
181
274
  slotTaken = false;
275
+ totals.granted += 1;
276
+ totals.totalWaitMs += permit.waitMs ?? 0;
277
+ recordEvent({ event: "granted", model: modelKey, waitMs: permit.waitMs ?? 0 });
182
278
  }
183
279
 
184
280
  // ── Phase 2: run the real LLM call ─────────────────────────
@@ -187,6 +283,7 @@ function apply(ctx) {
187
283
  yield* next();
188
284
  } finally {
189
285
  limiter.releaseSlot();
286
+ recordEvent({ event: "completed", model: modelKey });
190
287
  }
191
288
  });
192
289
 
@@ -0,0 +1,67 @@
1
+ /**
2
+ * dsh-llm-rate-limiter — status RPC channel (Host half).
3
+ *
4
+ * Exposes live rate-limiter statistics to the browser over the framework's
5
+ * generic Connection RPC channel — the same idiom dsh-context uses for its
6
+ * detail channel. The framework supplies POST+JSON transport, the Host/Origin
7
+ * fence (403) and browser authentication (401); this module only answers
8
+ * endpoints and shapes envelopes.
9
+ *
10
+ * Registration is owned by the calling fiber (`register()` wraps
11
+ * `owner.webServer.register` in `owner.effect`), so unloading the plugin
12
+ * withdraws the channel automatically — there is no connection bookkeeping
13
+ * here.
14
+ *
15
+ * Envelope contract (mirrors the Connection wire schema):
16
+ * success: { ok: true, value: <JSON-safe> }
17
+ * failure: { ok: false, error: { code, message, details } }
18
+ *
19
+ * @module dsh-llm-rate-limiter/status-rpc
20
+ */
21
+
22
+ /**
23
+ * Channel name. Must satisfy the framework's `assertChannel` pattern
24
+ * `/^\/[A-Za-z0-9._~-]+$/` and must not be the reserved `/api` channel.
25
+ */
26
+ const CHANNEL = "/llm-rate-limiter";
27
+
28
+ /**
29
+ * Build a failure envelope. Field-for-field the same shape the Connection
30
+ * host accepts, so callers can rely on `ok` discrimination alone.
31
+ * @param {string} code - stable machine-readable error code.
32
+ * @param {string} message - human-readable detail.
33
+ * @param {Record<string, unknown>} [details] - optional structured context.
34
+ * @returns {{ ok: false, error: { code: string, message: string, details: Record<string, unknown> } }}
35
+ */
36
+ function failure(code, message, details = {}) {
37
+ return { ok: false, error: { code, message, details } };
38
+ }
39
+
40
+ /** Build a success envelope. */
41
+ function success(value) {
42
+ return { ok: true, value };
43
+ }
44
+
45
+ /**
46
+ * Create the channel handler.
47
+ *
48
+ * @param {object} deps
49
+ * @param {() => object} deps.buildSnapshot - live snapshot builder.
50
+ * @param {() => void} deps.resetTotals - zero the counters (and bump `rev`).
51
+ * @returns {(endpoint: string, payload: unknown, signal?: AbortSignal) => Promise<object>}
52
+ */
53
+ function createStatusChannel({ buildSnapshot, resetTotals }) {
54
+ return async function statusChannel(endpoint) {
55
+ if (endpoint === "snapshot") return success(buildSnapshot());
56
+ if (endpoint === "reset") {
57
+ resetTotals();
58
+ return success({});
59
+ }
60
+ return failure(
61
+ "llm-rate-limiter/unknown-endpoint",
62
+ `unknown endpoint: ${String(endpoint)}`,
63
+ );
64
+ };
65
+ }
66
+
67
+ export { CHANNEL, createStatusChannel, failure, success };
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "@leaf233/dsh-llm-rate-limiter",
3
- "version": "0.1.1",
4
- "description": "Per-model LLM call rate limiter for DeepSeek Harness with queue support",
3
+ "version": "0.2.0",
4
+ "description": "Per-model LLM call rate limiter for DeepSeek Harness with queue support and a live status panel",
5
5
  "type": "module",
6
6
  "main": "lib/index.js",
7
7
  "exports": {
@@ -14,9 +14,15 @@
14
14
  },
15
15
  "client": {
16
16
  "inject": [
17
+ "@deepseek-ai/dsh-client-connection",
17
18
  "@deepseek-ai/dsh-client-ui-settings"
18
19
  ],
19
20
  "platform": "web"
21
+ },
22
+ "compatibility": {
23
+ "dshReleases": {
24
+ "0.1.2-rc.1": "compatible"
25
+ }
20
26
  }
21
27
  },
22
28
  "scripts": {