@jxgame2020/dsh-token-quota 0.1.4 → 0.1.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +66 -63
- package/lib/client.js +68 -45
- package/lib/index.js +95 -2
- package/lib/types/client/index.js +34 -3
- package/lib/types/client/locales.d.ts +2 -0
- package/lib/types/client/locales.js +4 -2
- package/lib/types/index.d.ts +17 -0
- package/lib/types/index.js +129 -4
- package/package.json +1 -1
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 jxgame2020
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,34 +1,38 @@
|
|
|
1
1
|
# @jxgame2020/dsh-token-quota
|
|
2
2
|
|
|
3
|
-
DeepSeek Harness 的**每日按模型 Token 限额**插件:后端按模型累计当日 token
|
|
4
|
-
达到上限时拦截请求;前端在 Web UI 右侧提供浮窗,实时显示每个被监控模型的
|
|
5
|
-
「实际/上限」,可对每个模型单独设置限额,满额时一键或自动切换模型。
|
|
3
|
+
DeepSeek Harness 的**每日按模型 Token 限额**插件:后端按模型累计当日 token 用量并持久化、达到上限时拦截请求;前端在 Web UI 右侧提供浮窗,实时显示每个被监控模型的「实际/上限」,可对每个模型单独设置限额,满额时一键或自动切换模型。
|
|
6
4
|
|
|
7
|
-
|
|
8
|
-
白名单;数据通过插件自有的 HTTP 路由(`GET /token-quota`)轮询读取,配置通过
|
|
9
|
-
settings 的 `token-quota` 命名空间读写。
|
|
5
|
+
A daily per-model token quota plugin for the DeepSeek Harness: the host counts each model's token usage for the day (persisted), blocks requests once a cap is reached, and a floating panel on the right side of the Web UI shows live usage/limit per monitored model, lets you set per-model caps, and switches models in one click or automatically when a model is full.
|
|
10
6
|
|
|
11
|
-
|
|
7
|
+
对 DSH 核心仓库**零改动**:不修改任何核心文件、不挂载官方 bundle、不依赖转发事件白名单;数据通过插件自有的 HTTP 路由(`GET /token-quota`)轮询读取,配置通过 settings 的 `token-quota` 命名空间读写。
|
|
12
8
|
|
|
13
|
-
-
|
|
14
|
-
|
|
9
|
+
Zero changes to the DSH core repo: no core file is modified, no official bundle is mounted, and no forwarded-event allowlist is required. The panel polls a plugin-owned HTTP route (`GET /token-quota`), and configuration lives in the `token-quota` settings namespace.
|
|
10
|
+
|
|
11
|
+
## 功能 / Features
|
|
12
|
+
|
|
13
|
+
- **按模型独立计数**:输入 + 输出 + 缓存 token 全部计入当日用量;重启不丢(持久化到 `$DSH_HOME/token-quota.json`)。
|
|
14
|
+
**Per-model counting**: input + output + cache tokens all count toward the daily usage, persisted to `$DSH_HOME/token-quota.json` across restarts.
|
|
15
15
|
- **硬拦截**:达到上限后请求直接停止并提示切换,不会继续消耗额度。
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
-
|
|
19
|
-
-
|
|
20
|
-
-
|
|
21
|
-
-
|
|
22
|
-
|
|
16
|
+
**Hard enforcement**: once a cap is hit, requests are stopped and you are prompted to switch — no extra tokens are spent.
|
|
17
|
+
- **满额策略**(四选一,面板设置里切换,即时生效)/ **Full-quota strategy** (pick one in the panel settings; applies immediately):
|
|
18
|
+
- 停止请求并提示 / Stop and prompt
|
|
19
|
+
- 自动切换到其它已监控且未满的限额模型 / Auto-switch to another monitored, capped-but-available model
|
|
20
|
+
- 自动切换到其它任意可用模型(含非限额)/ Auto-switch to any other available model (incl. uncapped)
|
|
21
|
+
- 自动切换(优先非限额,其次未监控)/ Prefer uncapped models, then unmonitored ones
|
|
22
|
+
- **每日自动重置**:默认本机时区午夜;可在设置里改为任意时区(UTC−12 ~ +14)与时刻(0:00–23:55),到点清零当日计数并归档用量日志。
|
|
23
|
+
**Daily auto-reset**: defaults to local midnight; pick any timezone (UTC−12 ~ +14) and time (0:00–23:55) in settings — counters reset and the finished day is archived into the usage log.
|
|
23
24
|
- **用量日志**:每模型每天一条(日期 / 模型 / 用量),与监控勾选无关。
|
|
24
|
-
|
|
25
|
-
|
|
25
|
+
**Usage log**: one row per model per day (date / model / usage), independent of the monitored set.
|
|
26
|
+
- **浮窗面板**:实时显示各模型「今日已用 / 上限」,行内一键「选择」切换模型、`⚙` 折叠设置单模型限额。
|
|
27
|
+
**Floating panel**: live used/limit per model, a one-click Select button per row, and a folded `⚙` per-model limit editor.
|
|
28
|
+
|
|
29
|
+
## 安装 / Installation
|
|
26
30
|
|
|
27
|
-
|
|
31
|
+
### 方式一:npm 安装(推荐)/ Option A: npm install (recommended)
|
|
28
32
|
|
|
29
|
-
|
|
33
|
+
在你的 web profile(`~/.dsh/profiles/web`)下安装依赖:
|
|
30
34
|
|
|
31
|
-
|
|
35
|
+
Install the dependency in your web profile (`~/.dsh/profiles/web`):
|
|
32
36
|
|
|
33
37
|
```bash
|
|
34
38
|
cd ~/.dsh/profiles/web
|
|
@@ -36,9 +40,12 @@ npm install @jxgame2020/dsh-token-quota
|
|
|
36
40
|
```
|
|
37
41
|
|
|
38
42
|
> 用 pnpm 管理 profile 的话:`pnpm add @jxgame2020/dsh-token-quota`。
|
|
43
|
+
> With pnpm: `pnpm add @jxgame2020/dsh-token-quota`.
|
|
39
44
|
|
|
40
45
|
在 `cordis.patch.yml` 中挂载插件:
|
|
41
46
|
|
|
47
|
+
Mount the plugin in `cordis.patch.yml`:
|
|
48
|
+
|
|
42
49
|
```yaml
|
|
43
50
|
- insert:
|
|
44
51
|
- id: token-quota
|
|
@@ -47,87 +54,83 @@ npm install @jxgame2020/dsh-token-quota
|
|
|
47
54
|
|
|
48
55
|
重启 `dsh web`,浏览器右侧出现「每日 Token 限额」浮窗即安装成功。
|
|
49
56
|
|
|
50
|
-
|
|
57
|
+
Restart `dsh web`; the floating panel appears on the right side once installed.
|
|
58
|
+
|
|
59
|
+
### 方式二:从 GitHub 源码安装 / Option B: install from source (GitHub)
|
|
60
|
+
|
|
61
|
+
适合想改源码、离线分发或审阅代码的情况。源码仓库不提交构建产物(`lib/`),clone 后需在本机构建(工具链来自 DSH 仓库的 workspace)。
|
|
51
62
|
|
|
52
|
-
|
|
53
|
-
clone 后需在本机构建(工具链来自 DSH 仓库的 workspace)。
|
|
63
|
+
For modifying the source, offline distribution, or code review. The source repo does not ship build output (`lib/`); clone and build locally (the toolchain resolves from the DSH repo workspace).
|
|
54
64
|
|
|
55
|
-
1. 克隆到与 `deepseek-harness`
|
|
65
|
+
1. 克隆到与 `deepseek-harness` 同级的开发目录 / Clone next to your `deepseek-harness` checkout:
|
|
56
66
|
|
|
57
67
|
```bash
|
|
58
68
|
git clone https://github.com/jxgame/dsh-token-quota.git deepseek-harness-package/dsh-token-quota
|
|
59
69
|
```
|
|
60
70
|
|
|
61
|
-
2. 在 `deepseek-harness/pnpm-workspace.yaml` 的 `packages:`
|
|
62
|
-
peer 依赖从仓库 workspace 解析):
|
|
71
|
+
2. 在 `deepseek-harness/pnpm-workspace.yaml` 的 `packages:` 下注册该包(让 peer 依赖从仓库 workspace 解析)/ Register the package under `packages:` in `deepseek-harness/pnpm-workspace.yaml` so peer deps resolve from the repo workspace:
|
|
63
72
|
|
|
64
73
|
```yaml
|
|
65
74
|
packages:
|
|
66
75
|
- ../deepseek-harness-package/dsh-token-quota/packages/token-quota
|
|
67
76
|
```
|
|
68
77
|
|
|
69
|
-
然后在仓库根执行 `cd deepseek-harness && pnpm install
|
|
78
|
+
然后在仓库根执行 `cd deepseek-harness && pnpm install`。/ Then run `cd deepseek-harness && pnpm install`.
|
|
70
79
|
|
|
71
|
-
3.
|
|
80
|
+
3. 构建 / Build:
|
|
72
81
|
|
|
73
82
|
```bash
|
|
74
83
|
cd deepseek-harness-package/dsh-token-quota/packages/token-quota
|
|
75
|
-
pnpm exec tsc -p tsconfig.json # 类型检查 + 产出 lib/types
|
|
84
|
+
pnpm exec tsc -p tsconfig.json # 类型检查 + 产出 lib/types / type-check + emit lib/types
|
|
76
85
|
pnpm exec tsdown # 产出 lib/index.js + lib/client.js
|
|
77
86
|
```
|
|
78
87
|
|
|
79
|
-
4. 挂载到 profile:`~/.dsh/profiles/web/package.json` 的 dependencies
|
|
88
|
+
4. 挂载到 profile:`~/.dsh/profiles/web/package.json` 的 dependencies 使用本地路径 / Mount into the profile with a local path dependency:
|
|
80
89
|
|
|
81
90
|
```json
|
|
82
|
-
"@jxgame2020/dsh-token-quota": "link
|
|
91
|
+
"@jxgame2020/dsh-token-quota": "link:/<absolute-path>/deepseek-harness-package/dsh-token-quota/packages/token-quota"
|
|
83
92
|
```
|
|
84
93
|
|
|
85
|
-
然后 `cd ~/.dsh/profiles/web && pnpm install`;`cordis.patch.yml`
|
|
86
|
-
方式一相同;重启 `dsh web`。
|
|
94
|
+
然后 `cd ~/.dsh/profiles/web && pnpm install`;`cordis.patch.yml` 挂载与方式一相同;重启 `dsh web`。/ Then `cd ~/.dsh/profiles/web && pnpm install`; the `cordis.patch.yml` mount is the same as Option A; restart `dsh web`.
|
|
87
95
|
|
|
88
|
-
## 使用
|
|
96
|
+
## 使用 / Usage
|
|
89
97
|
|
|
90
|
-
- 头部「设置」弹窗(改动即时自动保存,右上角 ×
|
|
91
|
-
-
|
|
92
|
-
|
|
93
|
-
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
-
|
|
97
|
-
|
|
98
|
-
-
|
|
99
|
-
- 名称后的「选择」按钮:一键切换当前会话到该模型。
|
|
100
|
-
- 行尾 `⚙`:展开该行限额输入框(数字,`0`=不限),保存后收起。
|
|
101
|
-
- 进度条:绿 <80%,黄 80–100%,红 = 已满额。
|
|
98
|
+
- 头部「设置」弹窗(改动即时自动保存,右上角 × 关闭)/ The header Settings dialog (changes auto-save instantly; close with × at the top-right):
|
|
99
|
+
- **监控模型** / **Monitored models**:勾选要监控的模型(默认全部)。未勾选的模型不显示、不计入当日用量、不受限额拦截(切换时仍可选)。/ Check the models to monitor (all by default). Unchecked models are hidden, not metered, and never capped (they remain selectable).
|
|
100
|
+
- **满额后处理** / **When a model is full**:四选一(见上文「功能」)。/ one of four strategies (see Features above).
|
|
101
|
+
- **每日重置时间** / **Daily reset**:时区(UTC−12 ~ UTC+14)与时刻(0:00–23:55,5 分钟步进),到点自动清零;不设置则沿用「本机时区午夜」。/ timezone (UTC−12 ~ UTC+14) and time (0:00–23:55, 5-min steps); defaults to machine-local midnight.
|
|
102
|
+
- 头部「日志」按钮(设置左侧):用量日志弹窗,表格列出日期 / 模型 / 用量,每模型每天一条。/ The Logs button (left of Settings) opens the usage-log dialog: date / model / usage, one row per model per day.
|
|
103
|
+
- 模型行 / Model rows:
|
|
104
|
+
- 名称后的「选择」按钮:一键切换当前会话到该模型。/ The Select button after the name switches the current session to that model.
|
|
105
|
+
- 行尾 `⚙`:展开该行限额输入框(数字,`0`=不限),保存后收起。/ The `⚙` at the row end unfolds the limit editor (number, `0` = unlimited); it folds back after saving.
|
|
106
|
+
- 进度条:绿 <80%,黄 80–100%,红 = 已满额。/ Progress bar: green <80%, yellow 80–100%, red = full.
|
|
102
107
|
|
|
103
|
-
## 数据与重置
|
|
108
|
+
## 数据与重置 / Data & Reset
|
|
104
109
|
|
|
105
|
-
- 计数文件:`$DSH_HOME/token-quota.json`(默认 `~/.dsh/token-quota.json
|
|
106
|
-
-
|
|
107
|
-
|
|
108
|
-
-
|
|
109
|
-
(`limits` / `monitored` / `onFull` / `reset`)。
|
|
110
|
-
- 面板数据读取:`GET /token-quota`(当前快照)、`GET /token-quota/log`(历史用量)。
|
|
110
|
+
- 计数文件:`$DSH_HOME/token-quota.json`(默认 `~/.dsh/token-quota.json`)。/ Counter file: `$DSH_HOME/token-quota.json` (default `~/.dsh/token-quota.json`).
|
|
111
|
+
- 重置周期默认「本机时区午夜」;可改为任意时区与时刻。跨过重置时刻时当日计数清零并归档进用量日志。/ The reset cycle defaults to machine-local midnight and can be changed to any timezone/time; crossing the reset moment zeroes the counters and archives the finished day into the usage log.
|
|
112
|
+
- 限额与面板配置保存在 `~/.dsh/settings.yaml` 的 `token-quota` 命名空间(`limits` / `monitored` / `onFull` / `reset`)。/ Limits and panel preferences live in the `token-quota` namespace of `~/.dsh/settings.yaml` (`limits` / `monitored` / `onFull` / `reset`).
|
|
113
|
+
- 面板数据读取:`GET /token-quota`(当前快照)、`GET /token-quota/log`(历史用量)。/ Panel data endpoints: `GET /token-quota` (live snapshot), `GET /token-quota/log` (usage history).
|
|
111
114
|
|
|
112
|
-
## 目录结构
|
|
115
|
+
## 目录结构 / Project Structure
|
|
113
116
|
|
|
114
117
|
```
|
|
115
118
|
dsh-token-quota/
|
|
116
119
|
└── packages/token-quota/
|
|
117
120
|
├── src/
|
|
118
|
-
│ ├── index.ts # Host 服务:计量/持久化/拦截/HTTP 快照路由
|
|
119
|
-
│ ├── types.ts # 共享类型与 wire 常量
|
|
120
|
-
│ ├── invariant.ts # 运行时守卫
|
|
121
|
-
│ └── client/ # 浏览器面板(shell.overlay
|
|
122
|
-
│ ├── index.ts # apply:轮询 + 满额策略 + 面板注册
|
|
121
|
+
│ ├── index.ts # Host 服务:计量/持久化/拦截/HTTP 快照路由 / host service: metering, persistence, enforcement, snapshot route
|
|
122
|
+
│ ├── types.ts # 共享类型与 wire 常量 / shared types and wire constants
|
|
123
|
+
│ ├── invariant.ts # 运行时守卫 / runtime guards
|
|
124
|
+
│ └── client/ # 浏览器面板(shell.overlay 浮窗)/ browser panel (shell.overlay)
|
|
125
|
+
│ ├── index.ts # apply:轮询 + 满额策略 + 面板注册 / polling + full-quota strategy + registration
|
|
123
126
|
│ ├── TokenQuotaPanel.tsx
|
|
124
127
|
│ ├── store.ts
|
|
125
128
|
│ ├── locales.ts
|
|
126
129
|
│ └── TokenQuotaPanel.module.css
|
|
127
|
-
├── tsconfig.json # host + client 一起编译
|
|
128
|
-
└── tsdown.config.ts # host 库 + 浏览器 bundle(closure-factory
|
|
130
|
+
├── tsconfig.json # host + client 一起编译 / compiles host + client
|
|
131
|
+
└── tsdown.config.ts # host 库 + 浏览器 bundle(closure-factory)/ host lib + browser bundle
|
|
129
132
|
```
|
|
130
133
|
|
|
131
|
-
## License
|
|
134
|
+
## License / 许可证
|
|
132
135
|
|
|
133
136
|
MIT
|
package/lib/client.js
CHANGED
|
@@ -129,59 +129,59 @@ window.__ModuleLoader__.load({
|
|
|
129
129
|
document.head.appendChild(tag);
|
|
130
130
|
}
|
|
131
131
|
var TokenQuotaPanel_module_css_default = {
|
|
132
|
-
"
|
|
133
|
-
"
|
|
132
|
+
"dialogBtnPrimary": "NPLLIq_dialogBtnPrimary",
|
|
133
|
+
"tabLabel": "NPLLIq_tabLabel",
|
|
134
134
|
"monitorHint": "NPLLIq_monitorHint",
|
|
135
|
-
"
|
|
136
|
-
"title": "NPLLIq_title",
|
|
137
|
-
"body": "NPLLIq_body",
|
|
138
|
-
"row": "NPLLIq_row",
|
|
139
|
-
"gearBtn": "NPLLIq_gearBtn",
|
|
140
|
-
"monitorName": "NPLLIq_monitorName",
|
|
141
|
-
"radioRow": "NPLLIq_radioRow",
|
|
142
|
-
"header": "NPLLIq_header",
|
|
135
|
+
"monitorRow": "NPLLIq_monitorRow",
|
|
143
136
|
"fillOver": "NPLLIq_fillOver",
|
|
144
|
-
"
|
|
145
|
-
"
|
|
137
|
+
"bar": "NPLLIq_bar",
|
|
138
|
+
"panel": "NPLLIq_panel",
|
|
139
|
+
"header": "NPLLIq_header",
|
|
140
|
+
"controls": "NPLLIq_controls",
|
|
141
|
+
"switch": "NPLLIq_switch",
|
|
142
|
+
"resetSelect": "NPLLIq_resetSelect",
|
|
146
143
|
"input": "NPLLIq_input",
|
|
147
|
-
"
|
|
148
|
-
"
|
|
149
|
-
"dialogLabel": "NPLLIq_dialogLabel",
|
|
144
|
+
"save": "NPLLIq_save",
|
|
145
|
+
"selectBtn": "NPLLIq_selectBtn",
|
|
150
146
|
"dialogActions": "NPLLIq_dialogActions",
|
|
147
|
+
"row": "NPLLIq_row",
|
|
148
|
+
"fillWarn": "NPLLIq_fillWarn",
|
|
149
|
+
"settingsBtn": "NPLLIq_settingsBtn",
|
|
150
|
+
"dialogSection": "NPLLIq_dialogSection",
|
|
151
|
+
"dialogTitle": "NPLLIq_dialogTitle",
|
|
152
|
+
"dialogClose": "NPLLIq_dialogClose",
|
|
153
|
+
"logDialog": "NPLLIq_logDialog",
|
|
154
|
+
"tab": "NPLLIq_tab",
|
|
155
|
+
"logScroll": "NPLLIq_logScroll",
|
|
156
|
+
"fillIdle": "NPLLIq_fillIdle",
|
|
157
|
+
"logTable": "NPLLIq_logTable",
|
|
158
|
+
"body": "NPLLIq_body",
|
|
159
|
+
"logUsedCol": "NPLLIq_logUsedCol",
|
|
160
|
+
"resetRow": "NPLLIq_resetRow",
|
|
161
|
+
"headerText": "NPLLIq_headerText",
|
|
162
|
+
"notice": "NPLLIq_notice",
|
|
151
163
|
"headerActions": "NPLLIq_headerActions",
|
|
164
|
+
"monitorName": "NPLLIq_monitorName",
|
|
152
165
|
"rowName": "NPLLIq_rowName",
|
|
153
166
|
"rowMeta": "NPLLIq_rowMeta",
|
|
154
|
-
"collapse": "NPLLIq_collapse",
|
|
155
|
-
"save": "NPLLIq_save",
|
|
156
|
-
"resetSelect": "NPLLIq_resetSelect",
|
|
157
|
-
"logDialog": "NPLLIq_logDialog",
|
|
158
|
-
"dialogClose": "NPLLIq_dialogClose",
|
|
159
167
|
"dialog": "NPLLIq_dialog",
|
|
160
|
-
"dialogBtnPrimary": "NPLLIq_dialogBtnPrimary",
|
|
161
|
-
"rowHeader": "NPLLIq_rowHeader",
|
|
162
|
-
"controls": "NPLLIq_controls",
|
|
163
168
|
"currentBadge": "NPLLIq_currentBadge",
|
|
164
|
-
"
|
|
165
|
-
"dialogHeader": "NPLLIq_dialogHeader",
|
|
166
|
-
"tab": "NPLLIq_tab",
|
|
167
|
-
"dialogTitle": "NPLLIq_dialogTitle",
|
|
168
|
-
"logDayCol": "NPLLIq_logDayCol",
|
|
169
|
-
"settingsBtn": "NPLLIq_settingsBtn",
|
|
170
|
-
"fillWarn": "NPLLIq_fillWarn",
|
|
171
|
-
"fullNotice": "NPLLIq_fullNotice",
|
|
172
|
-
"bar": "NPLLIq_bar",
|
|
169
|
+
"subtitle": "NPLLIq_subtitle",
|
|
173
170
|
"dialogBtn": "NPLLIq_dialogBtn",
|
|
174
|
-
"
|
|
175
|
-
"
|
|
176
|
-
"
|
|
177
|
-
"
|
|
171
|
+
"logDayCol": "NPLLIq_logDayCol",
|
|
172
|
+
"dialogHeader": "NPLLIq_dialogHeader",
|
|
173
|
+
"dialogLabel": "NPLLIq_dialogLabel",
|
|
174
|
+
"gearBtn": "NPLLIq_gearBtn",
|
|
175
|
+
"logModelCol": "NPLLIq_logModelCol",
|
|
176
|
+
"noticeError": "NPLLIq_noticeError",
|
|
177
|
+
"tabIcon": "NPLLIq_tabIcon",
|
|
178
178
|
"logHeader": "NPLLIq_logHeader",
|
|
179
|
-
"
|
|
179
|
+
"fullNotice": "NPLLIq_fullNotice",
|
|
180
180
|
"monitorList": "NPLLIq_monitorList",
|
|
181
|
-
"
|
|
182
|
-
"
|
|
183
|
-
"
|
|
184
|
-
"
|
|
181
|
+
"radioRow": "NPLLIq_radioRow",
|
|
182
|
+
"title": "NPLLIq_title",
|
|
183
|
+
"rowHeader": "NPLLIq_rowHeader",
|
|
184
|
+
"collapse": "NPLLIq_collapse"
|
|
185
185
|
};
|
|
186
186
|
//#endregion
|
|
187
187
|
//#region lib/types/client/TokenQuotaPanel.js
|
|
@@ -768,7 +768,8 @@ window.__ModuleLoader__.load({
|
|
|
768
768
|
fullSwitchAll: "自动切换到其它可用模型(含非限额)",
|
|
769
769
|
fullSwitchPriority: "自动切换(优先非限额,其次未监控)",
|
|
770
770
|
fullNotice: "当前模型今日额度已用尽,请求已停止。请选择其它模型或调整限额。",
|
|
771
|
-
fullSwitchFailed: "
|
|
771
|
+
fullSwitchFailed: "没有可切换的模型,请手动选择其它模型或调整限额",
|
|
772
|
+
fullSwitchTo: "已自动切换到 {model}",
|
|
772
773
|
close: "关闭",
|
|
773
774
|
logs: "日志",
|
|
774
775
|
logsTitle: "用量日志",
|
|
@@ -813,7 +814,8 @@ window.__ModuleLoader__.load({
|
|
|
813
814
|
fullSwitchAll: "Switch to any other model (incl. uncapped)",
|
|
814
815
|
fullSwitchPriority: "Prefer uncapped, then unmonitored",
|
|
815
816
|
fullNotice: "Today’s quota for the current model is exhausted; the request was stopped. Pick another model or raise its limit.",
|
|
816
|
-
fullSwitchFailed: "No switchable model
|
|
817
|
+
fullSwitchFailed: "No switchable model — pick one manually or raise a limit",
|
|
818
|
+
fullSwitchTo: "Auto-switched to {model}",
|
|
817
819
|
close: "Close",
|
|
818
820
|
logs: "Logs",
|
|
819
821
|
logsTitle: "Usage Log",
|
|
@@ -868,6 +870,19 @@ window.__ModuleLoader__.load({
|
|
|
868
870
|
return `${selection.provider}/${selection.model}`;
|
|
869
871
|
};
|
|
870
872
|
const isMonitoredKey = (key) => lastMonitored === null || lastMonitored.includes(key);
|
|
873
|
+
/** Transient auto-switch confirmation: shows for a few seconds, then clears. */
|
|
874
|
+
let fullNoticeTimer;
|
|
875
|
+
const flashFullNotice = (message) => {
|
|
876
|
+
if (fullNoticeTimer !== void 0) {
|
|
877
|
+
clearTimeout(fullNoticeTimer);
|
|
878
|
+
fullNoticeTimer = void 0;
|
|
879
|
+
}
|
|
880
|
+
bound?.setFullNotice(message);
|
|
881
|
+
fullNoticeTimer = setTimeout(() => {
|
|
882
|
+
bound?.setFullNotice(null);
|
|
883
|
+
fullNoticeTimer = void 0;
|
|
884
|
+
}, 6e3);
|
|
885
|
+
};
|
|
871
886
|
/** Pick an auto-switch target for the configured strategy. */
|
|
872
887
|
const pickSwitchTarget = (snapshot, currentKey) => {
|
|
873
888
|
const entryByKey = new Map(snapshot.entries.map((entry) => [entry.key, entry]));
|
|
@@ -916,7 +931,7 @@ window.__ModuleLoader__.load({
|
|
|
916
931
|
}).then(({ result }) => {
|
|
917
932
|
if (result.ok) {
|
|
918
933
|
lastCurrent = result.value.selected;
|
|
919
|
-
|
|
934
|
+
flashFullNotice(t("fullSwitchTo").replace("{model}", target.model));
|
|
920
935
|
pull();
|
|
921
936
|
} else bound?.setFullNotice(t("fullSwitchFailed"));
|
|
922
937
|
}, () => {
|
|
@@ -928,9 +943,16 @@ window.__ModuleLoader__.load({
|
|
|
928
943
|
if (lastSessionId === void 0) return;
|
|
929
944
|
connection.api.sessions.models({ sessionId: lastSessionId }).then(({ result }) => {
|
|
930
945
|
if (result.ok) {
|
|
946
|
+
const currentChanged = lastCurrent?.provider !== result.value.current?.provider || lastCurrent?.model !== result.value.current?.model;
|
|
931
947
|
lastGroups = result.value.groups;
|
|
932
948
|
lastCurrent = result.value.current;
|
|
933
949
|
bound?.setDirectory(result.value.groups, result.value.current);
|
|
950
|
+
if (currentChanged) fetch("/token-quota", { headers: { accept: "application/json" } }).then((response) => response.ok ? response.json() : void 0, () => void 0).then((snapshot) => {
|
|
951
|
+
if (snapshot !== void 0) {
|
|
952
|
+
bound?.setSnapshot(snapshot);
|
|
953
|
+
actOnFull(snapshot);
|
|
954
|
+
}
|
|
955
|
+
});
|
|
934
956
|
}
|
|
935
957
|
}, () => {});
|
|
936
958
|
};
|
|
@@ -955,7 +977,7 @@ window.__ModuleLoader__.load({
|
|
|
955
977
|
bound?.setSnapshot(snapshot);
|
|
956
978
|
actOnFull(snapshot);
|
|
957
979
|
});
|
|
958
|
-
if (pullCount %
|
|
980
|
+
if (pullCount % 2 === 0) refreshCurrent();
|
|
959
981
|
};
|
|
960
982
|
ctx.effect(() => {
|
|
961
983
|
pull();
|
|
@@ -1013,6 +1035,7 @@ window.__ModuleLoader__.load({
|
|
|
1013
1035
|
}).then(({ result }) => {
|
|
1014
1036
|
if (result.ok) {
|
|
1015
1037
|
lastCurrent = result.value.selected;
|
|
1038
|
+
bound?.setFullNotice(null);
|
|
1016
1039
|
bound?.setDirectory(lastGroups, result.value.selected);
|
|
1017
1040
|
pull();
|
|
1018
1041
|
} else bound?.setError(`${result.error.code}: ${result.error.message}`);
|
package/lib/index.js
CHANGED
|
@@ -110,7 +110,7 @@ function usageTokens(usage) {
|
|
|
110
110
|
* Mount it beside the other rows (`@jxgame2020/dsh-token-quota`) and write
|
|
111
111
|
* per-model limits through the `token-quota` settings namespace.
|
|
112
112
|
*/
|
|
113
|
-
var TokenQuotaService = class extends Service {
|
|
113
|
+
var TokenQuotaService = class TokenQuotaService extends Service {
|
|
114
114
|
static Config = z.object({ storagePath: z.string().default("") });
|
|
115
115
|
storagePath;
|
|
116
116
|
reset = {
|
|
@@ -131,6 +131,11 @@ var TokenQuotaService = class extends Service {
|
|
|
131
131
|
/** Per-session folded model key from the latest `request/header`. */
|
|
132
132
|
headerKeys = /* @__PURE__ */ new WeakMap();
|
|
133
133
|
writeTimer;
|
|
134
|
+
/** Cached model directory: list of providers and their models, refreshed lazily. */
|
|
135
|
+
cachedModels = [];
|
|
136
|
+
modelsCachedAt = 0;
|
|
137
|
+
/** How long to reuse the cached model directory before refreshing (30s). */
|
|
138
|
+
static MODEL_CACHE_TTL_MS = 3e4;
|
|
134
139
|
/** Disposer for the optional HTTP snapshot route (`GET /token-quota`). */
|
|
135
140
|
disposeRoute;
|
|
136
141
|
/** Disposer for the optional usage-history route (`GET /token-quota/log`). */
|
|
@@ -192,7 +197,9 @@ var TokenQuotaService = class extends Service {
|
|
|
192
197
|
ctx.on("session/event", (session, event) => {
|
|
193
198
|
this.onSessionEvent(session, event);
|
|
194
199
|
});
|
|
195
|
-
ctx.on("agent/
|
|
200
|
+
ctx.on("agent/created", ({ agent }) => {
|
|
201
|
+
agent.ctx.on("agent/request", async (payload, next) => this.onRequest(payload, next));
|
|
202
|
+
});
|
|
196
203
|
ctx.effect(() => () => {
|
|
197
204
|
this.disposeLocal();
|
|
198
205
|
}, "token-quota: flush on unload");
|
|
@@ -308,6 +315,79 @@ var TokenQuotaService = class extends Service {
|
|
|
308
315
|
this.scheduleWrite();
|
|
309
316
|
}
|
|
310
317
|
}
|
|
318
|
+
/**
|
|
319
|
+
* Refresh the cached list of all registered providers and their advertised
|
|
320
|
+
* models. Re-uses a still-fresh cache; any provider discovery failure is
|
|
321
|
+
* swallowed — we'd rather miss a candidate than crash the request waterfall.
|
|
322
|
+
*/
|
|
323
|
+
async refreshModels() {
|
|
324
|
+
const now = Date.now();
|
|
325
|
+
if (this.cachedModels.length > 0 && now - this.modelsCachedAt < TokenQuotaService.MODEL_CACHE_TTL_MS) return;
|
|
326
|
+
const llm = this.ctx.get("llm");
|
|
327
|
+
if (llm === void 0) {
|
|
328
|
+
this.cachedModels = [];
|
|
329
|
+
this.modelsCachedAt = now;
|
|
330
|
+
return;
|
|
331
|
+
}
|
|
332
|
+
const next = [];
|
|
333
|
+
for (const provider of llm.listProviders()) try {
|
|
334
|
+
const models = await llm.listModels(provider.id);
|
|
335
|
+
for (const model of models) next.push({
|
|
336
|
+
provider: provider.id,
|
|
337
|
+
model: model.id
|
|
338
|
+
});
|
|
339
|
+
} catch (error) {
|
|
340
|
+
this.ctx.logger.debug?.("token-quota: failed to list models for provider \"%s\": %o", provider.id, error);
|
|
341
|
+
}
|
|
342
|
+
this.cachedModels = next;
|
|
343
|
+
this.modelsCachedAt = now;
|
|
344
|
+
}
|
|
345
|
+
/**
|
|
346
|
+
* Choose a replacement model for the exhausted current model according to
|
|
347
|
+
* the configured `onFull` strategy. Returns `undefined` when no eligible
|
|
348
|
+
* candidate exists (caller falls back to the historical stop-and-throw).
|
|
349
|
+
*/
|
|
350
|
+
pickReplacementModel(currentKey) {
|
|
351
|
+
const onFull = this.settingsSource().onFull ?? "stop";
|
|
352
|
+
if (onFull === "stop") return void 0;
|
|
353
|
+
if (this.cachedModels.length === 0) return void 0;
|
|
354
|
+
const availability = (key) => {
|
|
355
|
+
return {
|
|
356
|
+
limit: this.limitOf(key),
|
|
357
|
+
used: this.usage[key] ?? 0
|
|
358
|
+
};
|
|
359
|
+
};
|
|
360
|
+
const isMonitored = (key) => this.isMonitored(key);
|
|
361
|
+
const isAvailable = (key) => {
|
|
362
|
+
if (!isMonitored(key)) return false;
|
|
363
|
+
const { limit, used } = availability(key);
|
|
364
|
+
return limit <= 0 || used < limit;
|
|
365
|
+
};
|
|
366
|
+
const candidates = this.cachedModels.filter(({ provider, model }) => tokenQuotaKey(provider, model) !== currentKey).map(({ provider, model }) => {
|
|
367
|
+
const key = tokenQuotaKey(provider, model);
|
|
368
|
+
const { limit, used } = availability(key);
|
|
369
|
+
return {
|
|
370
|
+
provider,
|
|
371
|
+
model,
|
|
372
|
+
key,
|
|
373
|
+
limit,
|
|
374
|
+
used,
|
|
375
|
+
monitored: isMonitored(key)
|
|
376
|
+
};
|
|
377
|
+
});
|
|
378
|
+
switch (onFull) {
|
|
379
|
+
case "switchQuota": return candidates.filter((c) => c.monitored && c.limit > 0 && c.used < c.limit).sort((a, b) => a.used / a.limit - b.used / b.limit)[0];
|
|
380
|
+
case "switchAll": return candidates.filter((c) => isAvailable(c.key))[0];
|
|
381
|
+
case "switchPriority": {
|
|
382
|
+
const uncapped = candidates.find((c) => c.monitored && c.limit <= 0);
|
|
383
|
+
if (uncapped !== void 0) return uncapped;
|
|
384
|
+
const unmonitored = candidates.find((c) => !c.monitored);
|
|
385
|
+
if (unmonitored !== void 0) return unmonitored;
|
|
386
|
+
return candidates.find((c) => isAvailable(c.key));
|
|
387
|
+
}
|
|
388
|
+
default: return;
|
|
389
|
+
}
|
|
390
|
+
}
|
|
311
391
|
async onRequest(payload, next) {
|
|
312
392
|
const config = await next();
|
|
313
393
|
const { provider, model } = config;
|
|
@@ -319,6 +399,19 @@ var TokenQuotaService = class extends Service {
|
|
|
319
399
|
if (limit <= 0) return config;
|
|
320
400
|
const used = this.usage[key] ?? 0;
|
|
321
401
|
if (used < limit) return config;
|
|
402
|
+
await this.refreshModels();
|
|
403
|
+
const replacement = this.pickReplacementModel(key);
|
|
404
|
+
if (replacement !== void 0) {
|
|
405
|
+
this.ctx.logger.info("token-quota: \"%s\" is full (%d/%d); auto-switching to \"%s\" (%s strategy).", key, used, limit, replacement.key, this.settingsSource().onFull);
|
|
406
|
+
this.headerKeys.set(payload.agent.session, replacement.key);
|
|
407
|
+
this.modelsCachedAt = 0;
|
|
408
|
+
this.cachedModels = [];
|
|
409
|
+
return {
|
|
410
|
+
...config,
|
|
411
|
+
provider: replacement.provider,
|
|
412
|
+
model: replacement.model
|
|
413
|
+
};
|
|
414
|
+
}
|
|
322
415
|
throw new LlmError(`Daily token limit reached for "${provider}/${model}": ${used}/${limit} tokens used today. Switch model in the quota panel or raise its limit.`, TOKEN_QUOTA_EXCEEDED_CODE);
|
|
323
416
|
}
|
|
324
417
|
/**
|
|
@@ -38,6 +38,19 @@ export function apply(ctx) {
|
|
|
38
38
|
return `${selection.provider}/${selection.model}`;
|
|
39
39
|
};
|
|
40
40
|
const isMonitoredKey = (key) => lastMonitored === null || lastMonitored.includes(key);
|
|
41
|
+
/** Transient auto-switch confirmation: shows for a few seconds, then clears. */
|
|
42
|
+
let fullNoticeTimer;
|
|
43
|
+
const flashFullNotice = (message) => {
|
|
44
|
+
if (fullNoticeTimer !== undefined) {
|
|
45
|
+
clearTimeout(fullNoticeTimer);
|
|
46
|
+
fullNoticeTimer = undefined;
|
|
47
|
+
}
|
|
48
|
+
bound?.setFullNotice(message);
|
|
49
|
+
fullNoticeTimer = setTimeout(() => {
|
|
50
|
+
bound?.setFullNotice(null);
|
|
51
|
+
fullNoticeTimer = undefined;
|
|
52
|
+
}, 6000);
|
|
53
|
+
};
|
|
41
54
|
/** Pick an auto-switch target for the configured strategy. */
|
|
42
55
|
const pickSwitchTarget = (snapshot, currentKey) => {
|
|
43
56
|
const entryByKey = new Map(snapshot.entries.map(entry => [entry.key, entry]));
|
|
@@ -101,7 +114,7 @@ export function apply(ctx) {
|
|
|
101
114
|
}).then(({ result }) => {
|
|
102
115
|
if (result.ok) {
|
|
103
116
|
lastCurrent = result.value.selected;
|
|
104
|
-
|
|
117
|
+
flashFullNotice(t('fullSwitchTo').replace('{model}', target.model));
|
|
105
118
|
pull();
|
|
106
119
|
}
|
|
107
120
|
else {
|
|
@@ -113,16 +126,32 @@ export function apply(ctx) {
|
|
|
113
126
|
// single fact source for usage AND the resolved per-model caps; the panel
|
|
114
127
|
// converges within one poll interval (plus an immediate first pull). Every
|
|
115
128
|
// few pulls we re-read the model directory so a model switch made OUTSIDE
|
|
116
|
-
// this panel (the official selector, /model
|
|
129
|
+
// this panel (the official selector, /model, or an automatic server-side
|
|
130
|
+
// switch on full quota) moves the 「当前」 badge without waiting the full
|
|
131
|
+
// directory-refresh cadence.
|
|
117
132
|
let pullCount = 0;
|
|
118
133
|
const refreshCurrent = () => {
|
|
119
134
|
if (lastSessionId === undefined)
|
|
120
135
|
return;
|
|
121
136
|
void connection.api.sessions.models({ sessionId: lastSessionId }).then(({ result }) => {
|
|
122
137
|
if (result.ok) {
|
|
138
|
+
const currentChanged = lastCurrent?.provider !== result.value.current?.provider
|
|
139
|
+
|| lastCurrent?.model !== result.value.current?.model;
|
|
123
140
|
lastGroups = result.value.groups;
|
|
124
141
|
lastCurrent = result.value.current;
|
|
125
142
|
bound?.setDirectory(result.value.groups, result.value.current);
|
|
143
|
+
// If the server silently switched to a different model (e.g. the
|
|
144
|
+
// auto-switch on quota exhaustion), re-run the full-quota strategy
|
|
145
|
+
// immediately against the new current model so any further fallback
|
|
146
|
+
// happens without waiting the next poll.
|
|
147
|
+
if (currentChanged) {
|
|
148
|
+
void fetch('/token-quota', { headers: { accept: 'application/json' } }).then(response => response.ok ? response.json() : undefined, () => undefined).then(snapshot => {
|
|
149
|
+
if (snapshot !== undefined) {
|
|
150
|
+
bound?.setSnapshot(snapshot);
|
|
151
|
+
actOnFull(snapshot);
|
|
152
|
+
}
|
|
153
|
+
});
|
|
154
|
+
}
|
|
126
155
|
}
|
|
127
156
|
}, () => { });
|
|
128
157
|
};
|
|
@@ -152,7 +181,7 @@ export function apply(ctx) {
|
|
|
152
181
|
bound?.setSnapshot(snapshot);
|
|
153
182
|
actOnFull(snapshot);
|
|
154
183
|
});
|
|
155
|
-
if (pullCount %
|
|
184
|
+
if (pullCount % 2 === 0)
|
|
156
185
|
refreshCurrent();
|
|
157
186
|
};
|
|
158
187
|
ctx.effect(() => {
|
|
@@ -203,6 +232,8 @@ export function apply(ctx) {
|
|
|
203
232
|
void connection.api.sessions.selectModel({ sessionId, provider, model }).then(({ result }) => {
|
|
204
233
|
if (result.ok) {
|
|
205
234
|
lastCurrent = result.value.selected;
|
|
235
|
+
// A manual pick clears any leftover auto-switch / exhausted notice.
|
|
236
|
+
bound?.setFullNotice(null);
|
|
206
237
|
bound?.setDirectory(lastGroups, result.value.selected);
|
|
207
238
|
// Immediately re-pull the snapshot so the previous model's final
|
|
208
239
|
// usage (credited around the switch) shows without waiting 3 s.
|
|
@@ -37,6 +37,7 @@ export declare const zh: {
|
|
|
37
37
|
fullSwitchPriority: string;
|
|
38
38
|
fullNotice: string;
|
|
39
39
|
fullSwitchFailed: string;
|
|
40
|
+
fullSwitchTo: string;
|
|
40
41
|
close: string;
|
|
41
42
|
logs: string;
|
|
42
43
|
logsTitle: string;
|
|
@@ -84,6 +85,7 @@ export declare const en: {
|
|
|
84
85
|
fullSwitchPriority: string;
|
|
85
86
|
fullNotice: string;
|
|
86
87
|
fullSwitchFailed: string;
|
|
88
|
+
fullSwitchTo: string;
|
|
87
89
|
close: string;
|
|
88
90
|
logs: string;
|
|
89
91
|
logsTitle: string;
|
|
@@ -36,7 +36,8 @@ export const zh = {
|
|
|
36
36
|
fullSwitchAll: '自动切换到其它可用模型(含非限额)',
|
|
37
37
|
fullSwitchPriority: '自动切换(优先非限额,其次未监控)',
|
|
38
38
|
fullNotice: '当前模型今日额度已用尽,请求已停止。请选择其它模型或调整限额。',
|
|
39
|
-
fullSwitchFailed: '
|
|
39
|
+
fullSwitchFailed: '没有可切换的模型,请手动选择其它模型或调整限额',
|
|
40
|
+
fullSwitchTo: '已自动切换到 {model}',
|
|
40
41
|
close: '关闭',
|
|
41
42
|
logs: '日志',
|
|
42
43
|
logsTitle: '用量日志',
|
|
@@ -81,7 +82,8 @@ export const en = {
|
|
|
81
82
|
fullSwitchAll: 'Switch to any other model (incl. uncapped)',
|
|
82
83
|
fullSwitchPriority: 'Prefer uncapped, then unmonitored',
|
|
83
84
|
fullNotice: 'Today\u2019s quota for the current model is exhausted; the request was stopped. Pick another model or raise its limit.',
|
|
84
|
-
fullSwitchFailed: 'No switchable model
|
|
85
|
+
fullSwitchFailed: 'No switchable model \u2014 pick one manually or raise a limit',
|
|
86
|
+
fullSwitchTo: 'Auto-switched to {model}',
|
|
85
87
|
close: 'Close',
|
|
86
88
|
logs: 'Logs',
|
|
87
89
|
logsTitle: 'Usage Log',
|
package/lib/types/index.d.ts
CHANGED
|
@@ -58,6 +58,11 @@ export declare class TokenQuotaService extends Service {
|
|
|
58
58
|
/** Per-session folded model key from the latest `request/header`. */
|
|
59
59
|
private readonly headerKeys;
|
|
60
60
|
private writeTimer;
|
|
61
|
+
/** Cached model directory: list of providers and their models, refreshed lazily. */
|
|
62
|
+
private cachedModels;
|
|
63
|
+
private modelsCachedAt;
|
|
64
|
+
/** How long to reuse the cached model directory before refreshing (30s). */
|
|
65
|
+
private static readonly MODEL_CACHE_TTL_MS;
|
|
61
66
|
/** Disposer for the optional HTTP snapshot route (`GET /token-quota`). */
|
|
62
67
|
private disposeRoute;
|
|
63
68
|
/** Disposer for the optional usage-history route (`GET /token-quota/log`). */
|
|
@@ -82,6 +87,18 @@ export declare class TokenQuotaService extends Service {
|
|
|
82
87
|
/** Resolve one model's daily cap: positive = capped, `0` = unlimited. */
|
|
83
88
|
limitOf(key: string): number;
|
|
84
89
|
private onSessionEvent;
|
|
90
|
+
/**
|
|
91
|
+
* Refresh the cached list of all registered providers and their advertised
|
|
92
|
+
* models. Re-uses a still-fresh cache; any provider discovery failure is
|
|
93
|
+
* swallowed — we'd rather miss a candidate than crash the request waterfall.
|
|
94
|
+
*/
|
|
95
|
+
private refreshModels;
|
|
96
|
+
/**
|
|
97
|
+
* Choose a replacement model for the exhausted current model according to
|
|
98
|
+
* the configured `onFull` strategy. Returns `undefined` when no eligible
|
|
99
|
+
* candidate exists (caller falls back to the historical stop-and-throw).
|
|
100
|
+
*/
|
|
101
|
+
private pickReplacementModel;
|
|
85
102
|
private onRequest;
|
|
86
103
|
/**
|
|
87
104
|
* Roll over to a new reset cycle, archiving the finished cycle's counters
|
package/lib/types/index.js
CHANGED
|
@@ -114,6 +114,11 @@ export class TokenQuotaService extends Service {
|
|
|
114
114
|
/** Per-session folded model key from the latest `request/header`. */
|
|
115
115
|
headerKeys = new WeakMap();
|
|
116
116
|
writeTimer;
|
|
117
|
+
/** Cached model directory: list of providers and their models, refreshed lazily. */
|
|
118
|
+
cachedModels = [];
|
|
119
|
+
modelsCachedAt = 0;
|
|
120
|
+
/** How long to reuse the cached model directory before refreshing (30s). */
|
|
121
|
+
static MODEL_CACHE_TTL_MS = 30_000;
|
|
117
122
|
/** Disposer for the optional HTTP snapshot route (`GET /token-quota`). */
|
|
118
123
|
disposeRoute;
|
|
119
124
|
/** Disposer for the optional usage-history route (`GET /token-quota/log`). */
|
|
@@ -194,10 +199,16 @@ export class TokenQuotaService extends Service {
|
|
|
194
199
|
ctx.on('session/event', (session, event) => {
|
|
195
200
|
this.onSessionEvent(session, event);
|
|
196
201
|
});
|
|
197
|
-
//
|
|
198
|
-
//
|
|
199
|
-
//
|
|
200
|
-
|
|
202
|
+
// For every newly created agent, install an agent-scoped enforcement
|
|
203
|
+
// listener that runs AFTER the api-proxy's model-selection listener (so
|
|
204
|
+
// user selections always apply first), and is therefore the last word on
|
|
205
|
+
// the chosen provider/model. When the resolved model is at its cap and
|
|
206
|
+
// the configured strategy allows an automatic switch, it rewrites the
|
|
207
|
+
// call config to a suitable replacement model instead of throwing, so
|
|
208
|
+
// the turn proceeds uninterrupted.
|
|
209
|
+
ctx.on('agent/created', ({ agent }) => {
|
|
210
|
+
agent.ctx.on('agent/request', async (payload, next) => this.onRequest(payload, next));
|
|
211
|
+
});
|
|
201
212
|
ctx.effect(() => () => { this.disposeLocal(); }, 'token-quota: flush on unload');
|
|
202
213
|
}
|
|
203
214
|
/** Serve the current snapshot over the plugin-owned HTTP route. */
|
|
@@ -300,6 +311,101 @@ export class TokenQuotaService extends Service {
|
|
|
300
311
|
this.scheduleWrite();
|
|
301
312
|
}
|
|
302
313
|
}
|
|
314
|
+
/**
|
|
315
|
+
* Refresh the cached list of all registered providers and their advertised
|
|
316
|
+
* models. Re-uses a still-fresh cache; any provider discovery failure is
|
|
317
|
+
* swallowed — we'd rather miss a candidate than crash the request waterfall.
|
|
318
|
+
*/
|
|
319
|
+
async refreshModels() {
|
|
320
|
+
const now = Date.now();
|
|
321
|
+
if (this.cachedModels.length > 0 && now - this.modelsCachedAt < TokenQuotaService.MODEL_CACHE_TTL_MS) {
|
|
322
|
+
return;
|
|
323
|
+
}
|
|
324
|
+
const llm = this.ctx.get('llm');
|
|
325
|
+
if (llm === undefined) {
|
|
326
|
+
this.cachedModels = [];
|
|
327
|
+
this.modelsCachedAt = now;
|
|
328
|
+
return;
|
|
329
|
+
}
|
|
330
|
+
const next = [];
|
|
331
|
+
for (const provider of llm.listProviders()) {
|
|
332
|
+
try {
|
|
333
|
+
const models = await llm.listModels(provider.id);
|
|
334
|
+
for (const model of models) {
|
|
335
|
+
next.push({ provider: provider.id, model: model.id });
|
|
336
|
+
}
|
|
337
|
+
}
|
|
338
|
+
catch (error) {
|
|
339
|
+
// Transient provider failure — keep whatever we already have for that
|
|
340
|
+
// provider and continue with the others.
|
|
341
|
+
this.ctx.logger.debug?.('token-quota: failed to list models for provider "%s": %o', provider.id, error);
|
|
342
|
+
}
|
|
343
|
+
}
|
|
344
|
+
this.cachedModels = next;
|
|
345
|
+
this.modelsCachedAt = now;
|
|
346
|
+
}
|
|
347
|
+
/**
|
|
348
|
+
* Choose a replacement model for the exhausted current model according to
|
|
349
|
+
* the configured `onFull` strategy. Returns `undefined` when no eligible
|
|
350
|
+
* candidate exists (caller falls back to the historical stop-and-throw).
|
|
351
|
+
*/
|
|
352
|
+
pickReplacementModel(currentKey) {
|
|
353
|
+
const doc = this.settingsSource();
|
|
354
|
+
const onFull = doc.onFull ?? 'stop';
|
|
355
|
+
if (onFull === 'stop')
|
|
356
|
+
return undefined;
|
|
357
|
+
if (this.cachedModels.length === 0)
|
|
358
|
+
return undefined;
|
|
359
|
+
const availability = (key) => {
|
|
360
|
+
const limit = this.limitOf(key);
|
|
361
|
+
const used = this.usage[key] ?? 0;
|
|
362
|
+
return { limit, used };
|
|
363
|
+
};
|
|
364
|
+
const isMonitored = (key) => this.isMonitored(key);
|
|
365
|
+
const isAvailable = (key) => {
|
|
366
|
+
if (!isMonitored(key))
|
|
367
|
+
return false;
|
|
368
|
+
const { limit, used } = availability(key);
|
|
369
|
+
return limit <= 0 || used < limit;
|
|
370
|
+
};
|
|
371
|
+
// Build candidate list excluding the currently exhausted model.
|
|
372
|
+
const candidates = this.cachedModels
|
|
373
|
+
.filter(({ provider, model }) => tokenQuotaKey(provider, model) !== currentKey)
|
|
374
|
+
.map(({ provider, model }) => {
|
|
375
|
+
const key = tokenQuotaKey(provider, model);
|
|
376
|
+
const { limit, used } = availability(key);
|
|
377
|
+
return { provider, model, key, limit, used, monitored: isMonitored(key) };
|
|
378
|
+
});
|
|
379
|
+
switch (onFull) {
|
|
380
|
+
case 'switchQuota': {
|
|
381
|
+
// Another monitored, capped model that still has headroom, sorted by
|
|
382
|
+
// lowest fill ratio so we spread load across capped models evenly.
|
|
383
|
+
const eligible = candidates
|
|
384
|
+
.filter(c => c.monitored && c.limit > 0 && c.used < c.limit)
|
|
385
|
+
.sort((a, b) => (a.used / a.limit) - (b.used / b.limit));
|
|
386
|
+
return eligible[0];
|
|
387
|
+
}
|
|
388
|
+
case 'switchAll': {
|
|
389
|
+
// Any monitored model that has headroom (capped or unlimited).
|
|
390
|
+
const eligible = candidates.filter(c => isAvailable(c.key));
|
|
391
|
+
return eligible[0];
|
|
392
|
+
}
|
|
393
|
+
case 'switchPriority': {
|
|
394
|
+
// 1. Unlimited monitored models. 2. Unmonitored models (never capped).
|
|
395
|
+
// 3. Capped-but-free monitored models.
|
|
396
|
+
const uncapped = candidates.find(c => c.monitored && c.limit <= 0);
|
|
397
|
+
if (uncapped !== undefined)
|
|
398
|
+
return uncapped;
|
|
399
|
+
const unmonitored = candidates.find(c => !c.monitored);
|
|
400
|
+
if (unmonitored !== undefined)
|
|
401
|
+
return unmonitored;
|
|
402
|
+
const free = candidates.find(c => isAvailable(c.key));
|
|
403
|
+
return free;
|
|
404
|
+
}
|
|
405
|
+
default:
|
|
406
|
+
return undefined;
|
|
407
|
+
}
|
|
408
|
+
}
|
|
303
409
|
async onRequest(payload, next) {
|
|
304
410
|
const config = await next();
|
|
305
411
|
const { provider, model } = config;
|
|
@@ -318,6 +424,25 @@ export class TokenQuotaService extends Service {
|
|
|
318
424
|
const used = this.usage[key] ?? 0;
|
|
319
425
|
if (used < limit)
|
|
320
426
|
return config;
|
|
427
|
+
// Current model is at or over cap. Try an automatic switch when the
|
|
428
|
+
// configured strategy allows it and a candidate exists; otherwise fall
|
|
429
|
+
// through to the historical hard stop.
|
|
430
|
+
await this.refreshModels();
|
|
431
|
+
const replacement = this.pickReplacementModel(key);
|
|
432
|
+
if (replacement !== undefined) {
|
|
433
|
+
this.ctx.logger.info('token-quota: "%s" is full (%d/%d); auto-switching to "%s" (%s strategy).', key, used, limit, replacement.key, this.settingsSource().onFull);
|
|
434
|
+
this.headerKeys.set(payload.agent.session, replacement.key);
|
|
435
|
+
// Invalidate the cached directory so the next request re-discovers any
|
|
436
|
+
// newly-registered models and so the next client-side poll sees the
|
|
437
|
+
// switch reflected without waiting a full TTL.
|
|
438
|
+
this.modelsCachedAt = 0;
|
|
439
|
+
this.cachedModels = [];
|
|
440
|
+
return {
|
|
441
|
+
...config,
|
|
442
|
+
provider: replacement.provider,
|
|
443
|
+
model: replacement.model,
|
|
444
|
+
};
|
|
445
|
+
}
|
|
321
446
|
throw new LlmError(`Daily token limit reached for "${provider}/${model}": ${used}/${limit} tokens used today. `
|
|
322
447
|
+ 'Switch model in the quota panel or raise its limit.', TOKEN_QUOTA_EXCEEDED_CODE);
|
|
323
448
|
}
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@jxgame2020/dsh-token-quota",
|
|
3
3
|
"description": "Daily per-model token quota accounting, enforcement, and a floating quota panel for the DeepSeek Harness — a fully self-contained plugin (no core-Harness wiring, no generated RPC contract)",
|
|
4
|
-
"version": "0.1.
|
|
4
|
+
"version": "0.1.6",
|
|
5
5
|
"publishConfig": {
|
|
6
6
|
"access": "public"
|
|
7
7
|
},
|