@goodandready/dsh-key-rotation 0.8.37 → 0.8.39
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +30 -1
- package/README.md +3 -1
- package/README.ru.md +3 -1
- package/README.zh.md +3 -1
- package/lib/bucket.js +61 -0
- package/lib/client.js +23 -7
- package/lib/index.js +25 -5
- package/lib/ops-status.js +3 -1
- package/lib/persistence.js +72 -5
- package/lib/pool-builder.js +39 -8
- package/lib/pool-state.js +1 -1
- package/lib/pool.js +2 -0
- package/lib/resolver.js +19 -3
- package/lib/rotate.js +13 -4
- package/lib/safe-fetch.js +106 -17
- package/lib/sanitize-snapshot.js +8 -0
- package/package.json +6 -3
package/CHANGELOG.md
CHANGED
|
@@ -1,12 +1,41 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.8.39 - 2026-09-30
|
|
4
|
+
|
|
5
|
+
### Fixed
|
|
6
|
+
- **SSRF guard: IPv4-mapped IPv6 loopbacks & connect-time rebinding (#411)**:
|
|
7
|
+
- Blocked hex (`::ffff:7f00:1`, `::ffff:7f00:0001`), dotted (`::ffff:127.0.0.1`), IPv4-compatible (`::/96`), and NAT64 (`64:ff9b::/96`) loopback addresses via full 8-word IPv6 unpacking in `lib/safe-fetch.js`.
|
|
8
|
+
- Added connect-time DNS resolution enforcement using `undici.Agent` lookup hook to prevent TOCTOU DNS rebinding before establishing socket connections.
|
|
9
|
+
- **Fail-closed credential resolver on pool exhaustion (#406)**:
|
|
10
|
+
- Return fail-closed error `LOCAL_POOL_EXHAUSTED` instead of leaking unmanaged `original(ref)` when all credentials in a managed pool are exhausted or blocked by cooldown, expiry, RPM, or TPM.
|
|
11
|
+
- **Credential status inheritance & permanent auth revocation (#408)**:
|
|
12
|
+
- Model sub-pools inherit `paused`, `revoked`, and `expiresAt` credentials from their parent base provider pool.
|
|
13
|
+
- Runtime 401 permanent auth failure marks credential as revoked (`revokedRefs.add` and `failedUntil = Infinity`) across all associated pools sharing the ref.
|
|
14
|
+
- **Persistence of revocation state and spend counters (#409)**:
|
|
15
|
+
- `StatePersistence` serializes and restores `revokedRefs` (as a Set), `Infinity` failedUntil timestamps, and daily usage/cost aggregation maps (`costDays`, `costPerKey`, `usageDays`, `usageCounts`).
|
|
16
|
+
- **UI credential alignment and model weights on reordering (#412)**:
|
|
17
|
+
- Preserved `paused` and `revoked` flag alignment with credential references during reordering, deletion, and undo actions in the settings card.
|
|
18
|
+
- Model pool key reordering preserves model-specific weights and expiry mappings.
|
|
19
|
+
|
|
20
|
+
## 0.8.38 - 2026-09-30
|
|
21
|
+
|
|
22
|
+
### Added
|
|
23
|
+
- **Sliding-Window TPM (Tokens Per Minute) Rate Limiting (#402, GH #20)**:
|
|
24
|
+
- Added 60-second sliding-window TPM rate-limiting alongside existing RPM (`tpmAllow`, `tpmRecord`, `tpmRetryMs`, `tpmSweep`, `tpmInfo` in `lib/bucket.js`).
|
|
25
|
+
- Added `tpmLimit` configuration at global, provider, and per-model scopes in `lib/index.js` and `lib/pool-builder.js`.
|
|
26
|
+
- Automatically records consumed tokens on streaming and non-streaming responses in `lib/rotate.js`.
|
|
27
|
+
- Enforced fail-over and cooldown in `lib/resolver.js` when candidate key exceeds its configured TPM budget.
|
|
28
|
+
- Exposed live TPM telemetry (`used`, `remaining`, `limit`, `resetMs`) in `GET /dsh-key-rotation/status` with sanitization in `lib/sanitize-snapshot.js`.
|
|
29
|
+
- Added visual UI badge and editable TPM limit input in `lib/client-src/50-card-main.js` with localization (`en`, `zh`).
|
|
30
|
+
- Added automated test suite `test/tpm-rate-limit.test.mjs` verifying bucket math, sweep cleanup, resolver fail-over, and snapshot sanitization.
|
|
31
|
+
|
|
3
32
|
## 0.8.37
|
|
4
33
|
|
|
5
34
|
### Fixed
|
|
6
35
|
- **Peer gate on DSH 0.2.0-rc.1** (#58): the bundle was skipped at profile startup because its `peerDependencies` excluded the running version.
|
|
7
36
|
- **Settings card served no form**: `NS` was the package name rather than the profile entry id, `Config` declared no `.volatile()` field, and `getConfig` handed out `Volatile` boxes. Both 0.1.7-rc.2 and 0.2.0 now serve and read the form.
|
|
8
37
|
- **A saved setting never took effect**: the host applied changes through `settings.register` and `scope.watch`, neither of which exists in either release. Changes are applied on `loader/volatile-update` now.
|
|
9
|
-
- **`settings.plugin.item` registration removed**: retired before DSH 0.1.7-rc.2,
|
|
38
|
+
- **`settings.plugin.item` registration removed**: retired before DSH 0.1.7-rc.2, it only registered the card a second time on a seat that no longer exists.
|
|
10
39
|
|
|
11
40
|
## 0.8.36
|
|
12
41
|
|
package/README.md
CHANGED
|
@@ -207,7 +207,9 @@ Access full visual management under **Settings → Key Rotation** or via the Hea
|
|
|
207
207
|
* **Secure Vault Storage**: Actual secret values reside securely in `$DSH_HOME/.credentials.yaml` managed by the DSH `Credentials` service.
|
|
208
208
|
* **5-Character Masking (`keyTail`)**: Full secret values are never sent to the client browser; only the trailing 5 characters are exposed for visual identification. Short keys (<= 5 characters) return a fixed masked placeholder (`***`) to prevent credential disclosure.
|
|
209
209
|
* **Fail-Closed Loopback & Same-Origin Fencing**: Administrative endpoints strictly enforce loopback origin checks (`isTrustedBridgeRequest`), requiring valid `Origin` headers, matching `Host` headers, and rejecting `cross-site` or non-loopback requests without fallbacks.
|
|
210
|
-
* **
|
|
210
|
+
* **Fail-Closed Resolver on Pool Exhaustion**: When all credentials in a managed pool are exhausted, paused, expired, or blocked by RPM/TPM limits, the resolver fails closed with `LOCAL_POOL_EXHAUSTED` error rather than falling back to leaking unmanaged credentials.
|
|
211
|
+
* **SSRF Protection & Rebinding Guard**: Remote provider pool import strictly enforces HTTPS-only URLs, validates all resolved IP addresses including IPv4-mapped and IPv4-compatible IPv6 addresses (`::ffff:127.0.0.1`, `::ffff:7f00:1`, `64:ff9b::/96`), and enforces connect-time DNS validation via undici agent dispatchers to prevent TOCTOU DNS rebinding.
|
|
212
|
+
* **Cross-Pool Revocation & Inheritance**: Model pools automatically inherit pause, revoke, and expiry states from their base provider; runtime 401 permanent authentication failures revoke the credential across all shared pools immediately.
|
|
211
213
|
|
|
212
214
|
---
|
|
213
215
|
|
package/README.ru.md
CHANGED
|
@@ -194,7 +194,9 @@ graph LR
|
|
|
194
194
|
* **Защищённое хранилище хоста**: Реальные значения ключей сохраняются в `$DSH_HOME/.credentials.yaml` сервисом `Credentials`.
|
|
195
195
|
* **Маскировка в браузере (5 символов)**: Браузер получает только последние 5 символов ключа для визуального отличия. Для коротких ключей (длиной <= 5 символов) возвращается безопасная маскировочная заглушка `***`, исключающая раскрытие ключа целиком.
|
|
196
196
|
* **Fail-Closed изоляция Loopback и Same-Origin**: Все управляющие эндпоинты строго проверяют происхождение через `isTrustedBridgeRequest`, требуя обязательный заголовок `Origin`, соответствие `Host` и отклоняя запросы `cross-site` и не-loopback без каких-либо исключений.
|
|
197
|
-
* **
|
|
197
|
+
* **Fail-Closed резолвер при исчерпании пула**: Если все ключи в пуле исчерпаны, приостановлены, истекли или заблокированы по RPM/TPM, резолвер завершается с ошибкой `LOCAL_POOL_EXHAUSTED`, исключая скрытую утечку исходного неконтролируемого ключа.
|
|
198
|
+
* **SSRF-защита и блокировка DNS Rebinding**: Маршрут импорта пулов провайдеров разрешает исключительно HTTPS, валидирует все IP-адреса, включая IPv4-mapped и IPv4-compatible IPv6 представления (`::ffff:127.0.0.1`, `::ffff:7f00:1`, `64:ff9b::/96`), и выполняет проверку адреса в момент соединения (connect-time DNS validation) через диспетчер undici для защиты от TOCTOU DNS rebinding.
|
|
199
|
+
* **Сквозной отзыв и наследование статусов**: Пул конкретной модели автоматически наследует флаги паузы, отзыва и срок действия базового провайдера; перманентный сбой аутентификации 401 немедленно отзывает ключ во всех связанных пулах.
|
|
198
200
|
|
|
199
201
|
---
|
|
200
202
|
|
package/README.zh.md
CHANGED
|
@@ -186,7 +186,9 @@ graph LR
|
|
|
186
186
|
* **宿主安全存储**:真实密钥持久化保存在 `$DSH_HOME/.credentials.yaml`。
|
|
187
187
|
* **前台 5 字符脱敏**:前端仅展示密钥后 5 位字符进行视觉区分。对于长度 <= 5 的短密钥,返回统一脱敏占位符 (`***`),防止凭证完整泄露。
|
|
188
188
|
* **Fail-Closed 环回同源安全隔离**:管理接口严格通过 `isTrustedBridgeRequest` 验证本地同源请求,强制要求有效 `Origin` 和 `Host` 匹配,无条件拒绝跨站 (`cross-site`) 与非环回调用。
|
|
189
|
-
*
|
|
189
|
+
* **凭证池耗尽 Fail-Closed 熔断**:当受管凭证池中所有密钥均被耗尽、暂停、过期或触发 RPM/TPM 限制时,解析器立即返回 `LOCAL_POOL_EXHAUSTED` 错误熔断,杜绝静默回退泄漏未受控原始密钥。
|
|
190
|
+
* **SSRF 与 DNS 重绑定防护**:远程配置导入严格仅支持 HTTPS,全面校验解析 IP 并拦截 IPv4-mapped/IPv4-compatible 等各类 IPv6 环回变体(如 `::ffff:127.0.0.1`、`::ffff:7f00:1`、`64:ff9b::/96`),并通过 undici 调度器执行连接时(connect-time)DNS 校验,从根本上防御 TOCTOU DNS 重绑定。
|
|
191
|
+
* **跨池吊销与状态继承**:模型子池自动继承基础提供商的暂停、吊销与过期时间;运行时 401 永久认证失败将即时在所有关联共享池中统一标记吊销。
|
|
190
192
|
|
|
191
193
|
---
|
|
192
194
|
|
package/lib/bucket.js
CHANGED
|
@@ -41,3 +41,64 @@ export function bucketInfo(windows, ref, limit, now = Date.now()) {
|
|
|
41
41
|
};
|
|
42
42
|
}
|
|
43
43
|
|
|
44
|
+
|
|
45
|
+
/** Sliding-window check: true if `ref` is under `limit` tokens/min. */
|
|
46
|
+
export function tpmAllow(windows, ref, limit, now = Date.now()) {
|
|
47
|
+
if (!limit || limit <= 0) return true;
|
|
48
|
+
const cut = now - WINDOW_MS;
|
|
49
|
+
const entries = (windows?.get(ref) ?? []).filter((e) => e.time > cut);
|
|
50
|
+
let total = 0;
|
|
51
|
+
for (const e of entries) total += e.tokens;
|
|
52
|
+
return total < limit;
|
|
53
|
+
}
|
|
54
|
+
|
|
55
|
+
/** Record actual consumed tokens for `ref` in sliding 60s window. */
|
|
56
|
+
export function tpmRecord(windows, ref, tokens, now = Date.now()) {
|
|
57
|
+
if (!tokens || tokens <= 0 || !windows) return;
|
|
58
|
+
const cut = now - WINDOW_MS;
|
|
59
|
+
const validTokens = Math.max(1, Math.floor(tokens));
|
|
60
|
+
const entries = (windows.get(ref) ?? []).filter((e) => e.time > cut);
|
|
61
|
+
entries.push({ time: now, tokens: validTokens });
|
|
62
|
+
windows.set(ref, entries);
|
|
63
|
+
}
|
|
64
|
+
|
|
65
|
+
/** ms until `ref` may retry again under TPM limit (0 = now). */
|
|
66
|
+
export function tpmRetryMs(windows, ref, limit, now = Date.now()) {
|
|
67
|
+
if (!limit || limit <= 0) return 0;
|
|
68
|
+
const cut = now - WINDOW_MS;
|
|
69
|
+
const entries = (windows?.get(ref) ?? []).filter((e) => e.time > cut);
|
|
70
|
+
let total = 0;
|
|
71
|
+
for (const e of entries) total += e.tokens;
|
|
72
|
+
if (total < limit) return 0;
|
|
73
|
+
let rem = total;
|
|
74
|
+
for (const e of entries) {
|
|
75
|
+
rem -= e.tokens;
|
|
76
|
+
if (rem < limit) {
|
|
77
|
+
return Math.max(0, e.time + WINDOW_MS - now);
|
|
78
|
+
}
|
|
79
|
+
}
|
|
80
|
+
return WINDOW_MS;
|
|
81
|
+
}
|
|
82
|
+
|
|
83
|
+
/** Drop TPM state for refs that no longer exist. */
|
|
84
|
+
export function tpmSweep(windows, liveRefs) {
|
|
85
|
+
if (!windows) return;
|
|
86
|
+
for (const ref of [...windows.keys()]) {
|
|
87
|
+
if (!liveRefs.has(ref)) windows.delete(ref);
|
|
88
|
+
}
|
|
89
|
+
}
|
|
90
|
+
|
|
91
|
+
/** Snapshot for /status - used/remaining/limit/resetMs for one ref. */
|
|
92
|
+
export function tpmInfo(windows, ref, limit, now = Date.now()) {
|
|
93
|
+
if (!limit || limit <= 0) return null;
|
|
94
|
+
const cut = now - WINDOW_MS;
|
|
95
|
+
const entries = (windows?.get(ref) ?? []).filter((e) => e.time > cut);
|
|
96
|
+
let used = 0;
|
|
97
|
+
for (const e of entries) used += e.tokens;
|
|
98
|
+
return {
|
|
99
|
+
used,
|
|
100
|
+
remaining: Math.max(0, limit - used),
|
|
101
|
+
limit,
|
|
102
|
+
resetMs: used >= limit ? tpmRetryMs(windows, ref, limit, now) : 0,
|
|
103
|
+
};
|
|
104
|
+
}
|
package/lib/client.js
CHANGED
|
@@ -117,6 +117,9 @@ window.__ModuleLoader__.load({
|
|
|
117
117
|
keyWriteFailed: 'could not store the key: {msg}',
|
|
118
118
|
notSecretShape: 'saved, but the value does not look like an API key - check for a typo',
|
|
119
119
|
rpmTitle: 'requests/min: {u} used, {r} remaining',
|
|
120
|
+
tpmTitle: 'tokens/min: {u} used, {r} remaining',
|
|
121
|
+
rpmLimitLabel: 'Rate Limit (RPM)',
|
|
122
|
+
tpmLimitLabel: 'Rate Limit (TPM)',
|
|
120
123
|
budgetLabel: 'budget:',
|
|
121
124
|
exportCsv: 'Export CSV',
|
|
122
125
|
weightHint: 'round-robin weight: how many times this key joins the cycle (1 = equal share)',
|
|
@@ -299,6 +302,9 @@ window.__ModuleLoader__.load({
|
|
|
299
302
|
keyWriteFailed: '无法存储密钥: {msg}',
|
|
300
303
|
notSecretShape: '已保存,但该值看起来不像 API 密钥 - 请检查是否有拼写错误',
|
|
301
304
|
rpmTitle: '请求/分钟: 已用 {u}, 剩余 {r}',
|
|
305
|
+
tpmTitle: 'Token/分钟: 已用 {u}, 剩余 {r}',
|
|
306
|
+
rpmLimitLabel: '速率限制 (RPM)',
|
|
307
|
+
tpmLimitLabel: '速率限制 (TPM)',
|
|
302
308
|
budgetLabel: '预算:',
|
|
303
309
|
exportCsv: '导出 CSV',
|
|
304
310
|
weightHint: '轮换权重:该密钥参与循环的次数 (1 = 等分)',
|
|
@@ -932,7 +938,7 @@ window.__ModuleLoader__.load({
|
|
|
932
938
|
|
|
933
939
|
function reorderKeys(entry, order) {
|
|
934
940
|
const next = { ...entry, keys: order.map(index => entry.keys[index]) };
|
|
935
|
-
for (const [field, fallback] of [['weights', 1], ['expiresAt', 0]]) {
|
|
941
|
+
for (const [field, fallback] of [['weights', 1], ['expiresAt', 0], ['paused', false], ['revoked', false]]) {
|
|
936
942
|
if (Array.isArray(entry[field]) && entry[field].length) next[field] = order.map(index => entry[field][index] ?? fallback);
|
|
937
943
|
}
|
|
938
944
|
return next;
|
|
@@ -1207,7 +1213,7 @@ window.__ModuleLoader__.load({
|
|
|
1207
1213
|
const entry = { ...providers[index] };
|
|
1208
1214
|
const at = Math.min(u.kIndex, entry.keys.length);
|
|
1209
1215
|
entry.keys = [...entry.keys]; entry.keys.splice(at, 0, u.key);
|
|
1210
|
-
for (const [field, fallback] of [['weights', 1], ['expiresAt', 0]]) {
|
|
1216
|
+
for (const [field, fallback] of [['weights', 1], ['expiresAt', 0], ['paused', false], ['revoked', false]]) {
|
|
1211
1217
|
if (u[field] !== undefined || entry[field]?.length) {
|
|
1212
1218
|
const values = Array.from({ length: entry.keys.length - 1 }, (_, i) => entry[field]?.[i] ?? fallback);
|
|
1213
1219
|
values.splice(at, 0, u[field] ?? fallback); entry[field] = values;
|
|
@@ -1557,7 +1563,7 @@ window.__ModuleLoader__.load({
|
|
|
1557
1563
|
const allRefs = providers.flatMap((prov) => prov?.keys ?? []);
|
|
1558
1564
|
entry.keys = [...(entry.keys ?? []), nextKeyRef(entry.provider, entry.keys, allRefs)];
|
|
1559
1565
|
// keep weights aligned with keys (#215): new key gets default weight 1
|
|
1560
|
-
for (const [field, fallback] of [['weights', 1], ['expiresAt', 0]]) {
|
|
1566
|
+
for (const [field, fallback] of [['weights', 1], ['expiresAt', 0], ['paused', false], ['revoked', false]]) {
|
|
1561
1567
|
if (Array.isArray(entry[field]) && entry[field].length) entry[field] = Array.from({ length: entry.keys.length }, (_, i) => entry[field][i] ?? fallback);
|
|
1562
1568
|
}
|
|
1563
1569
|
providers[pIndex] = entry;
|
|
@@ -1568,7 +1574,8 @@ window.__ModuleLoader__.load({
|
|
|
1568
1574
|
const entry = next[pIndex];
|
|
1569
1575
|
if (!entry || !entry.keys[kIndex]) return cur;
|
|
1570
1576
|
stashUndo({ type: 'key', provider: entry.provider, kIndex, key: entry.keys[kIndex],
|
|
1571
|
-
weights: entry.weights?.[kIndex], expiresAt: entry.expiresAt?.[kIndex]
|
|
1577
|
+
weights: entry.weights?.[kIndex], expiresAt: entry.expiresAt?.[kIndex],
|
|
1578
|
+
paused: entry.paused?.[kIndex], revoked: entry.revoked?.[kIndex] });
|
|
1572
1579
|
next[pIndex] = reorderKeys(entry, entry.keys.map((_, index) => index).filter(index => index !== kIndex));
|
|
1573
1580
|
return { ...cur, providers: next };
|
|
1574
1581
|
});
|
|
@@ -1685,9 +1692,9 @@ window.__ModuleLoader__.load({
|
|
|
1685
1692
|
const keys = Array.isArray(mp.keys) ? [...mp.keys] : [];
|
|
1686
1693
|
const target = kIndex + delta;
|
|
1687
1694
|
if (target < 0 || target >= keys.length) return;
|
|
1688
|
-
|
|
1689
|
-
|
|
1690
|
-
models[model] = mp;
|
|
1695
|
+
const order = keys.map((_, i) => i);
|
|
1696
|
+
[order[kIndex], order[target]] = [order[target], order[kIndex]];
|
|
1697
|
+
models[model] = reorderKeys(mp, order);
|
|
1691
1698
|
});
|
|
1692
1699
|
|
|
1693
1700
|
const save = async () => {
|
|
@@ -1918,6 +1925,11 @@ window.__ModuleLoader__.load({
|
|
|
1918
1925
|
title: t('rpmTitle').replace('{u}', String(info.rpm.used)).replace('{r}', String(info.rpm.remaining)),
|
|
1919
1926
|
style: info.rpm.remaining === 0 ? { color: 'var(--dsw-alias-state-error-primary)', fontWeight: 700 } : undefined },
|
|
1920
1927
|
'⏱' + info.rpm.remaining));
|
|
1928
|
+
// TPM capacity indicator (only when tpmLimit is active)
|
|
1929
|
+
if (info && info.tpm) meta.push(h('span', { key: 'tpm', className: 'krot-tail',
|
|
1930
|
+
title: t('tpmTitle').replace('{u}', String(info.tpm.used)).replace('{r}', String(info.tpm.remaining)),
|
|
1931
|
+
style: info.tpm.remaining === 0 ? { color: 'var(--dsw-alias-state-error-primary)', fontWeight: 700 } : undefined },
|
|
1932
|
+
'⚡' + info.tpm.remaining));
|
|
1921
1933
|
if (info && typeof info.cost === 'number' && info.cost > 0) meta.push(h('span', { key: 'c', className: 'krot-tail', title: 'cost' }, '$' + info.cost.toFixed(2)));
|
|
1922
1934
|
const tr = testResult[key];
|
|
1923
1935
|
if (tr) meta.push(h('span', { key: 'tr', className: 'krot-tail',
|
|
@@ -2393,6 +2405,10 @@ window.__ModuleLoader__.load({
|
|
|
2393
2405
|
field(t('cooldown'), textInput(String(val.cooldownMs ?? 60000), (v) => setField((cur) => ({ ...cur, cooldownMs: Number(v) || 0 })))),
|
|
2394
2406
|
field(t('scheduleDays'), textInput(String(val.rotationScheduleDays ?? 0), (v) => setField((cur) => ({ ...cur, rotationScheduleDays: Number(v) || 0 }))))
|
|
2395
2407
|
),
|
|
2408
|
+
h('div', { className: 'krot-grid-2', style: { marginTop: '10px' } },
|
|
2409
|
+
field(t('rpmLimitLabel'), textInput(String(val.rpmLimit ?? 0), (v) => setField((cur) => ({ ...cur, rpmLimit: Number(v) || 0 })))),
|
|
2410
|
+
field(t('tpmLimitLabel'), textInput(String(val.tpmLimit ?? 0), (v) => setField((cur) => ({ ...cur, tpmLimit: Number(v) || 0 }))))
|
|
2411
|
+
),
|
|
2396
2412
|
h('div', { className: 'krot-grid-3', style: { marginTop: '10px' } },
|
|
2397
2413
|
field(t('routingStrategyLabel'), h('select', {
|
|
2398
2414
|
className: 'krot-in',
|
package/lib/index.js
CHANGED
|
@@ -52,6 +52,17 @@ export { keyTail, KEY_TAIL_CHARS, KEY_TAIL_PLACEHOLDER, isLoopbackAddress, isTru
|
|
|
52
52
|
export { notifySwitch, notifyExhaustion };
|
|
53
53
|
|
|
54
54
|
const NS = 'dsh-key-rotation';
|
|
55
|
+
|
|
56
|
+
// DSH serves a namespace's settings form from the volatile fields of its profile
|
|
57
|
+
// entry schema, and a volatile field holds a Volatile box rather than its value.
|
|
58
|
+
// Unwrap before any caller reads one, and read lazily: the Loader mutates the
|
|
59
|
+
// boxes in place and re-announces them with loader/volatile-update.
|
|
60
|
+
export function plainConfig(value) {
|
|
61
|
+
if (value === null || typeof value !== 'object') return value
|
|
62
|
+
if (Array.isArray(value)) return value.map(plainConfig)
|
|
63
|
+
if (typeof value.get === 'function') return plainConfig(value.get())
|
|
64
|
+
return Object.fromEntries(Object.entries(value).map(([k, v]) => [k, plainConfig(v)]))
|
|
65
|
+
}
|
|
55
66
|
const CONFIG_PATH = '/dsh-key-rotation/config';
|
|
56
67
|
const PIAI_NS = 'llm-pi-ai';
|
|
57
68
|
const MARKER = '__dshKeyRotation';
|
|
@@ -107,6 +118,7 @@ const SettingsSchema = Schema.object({
|
|
|
107
118
|
selfHealingIntervalMinutes: Schema.number().default(30),
|
|
108
119
|
routingStrategy: Schema.union(['round-robin', 'least-loaded', 'lowest-latency']).default('round-robin'),
|
|
109
120
|
rpmLimit: Schema.number().default(0),
|
|
121
|
+
tpmLimit: Schema.number().default(0),
|
|
110
122
|
webhookActionToken: Schema.string().role('secret').default(''),
|
|
111
123
|
expiryWarnDays: Schema.number().default(7),
|
|
112
124
|
switchNotify: Schema.boolean().default(false),
|
|
@@ -137,11 +149,15 @@ const SettingsSchema = Schema.object({
|
|
|
137
149
|
quotas: Schema.dict(Schema.object({
|
|
138
150
|
tokenLimit: Schema.number(),
|
|
139
151
|
})).default({}),
|
|
152
|
+
rpmLimit: Schema.number(),
|
|
153
|
+
tpmLimit: Schema.number(),
|
|
140
154
|
})).default({}),
|
|
141
155
|
cooldownMs: Schema.number(),
|
|
142
156
|
maxCooldownMs: Schema.number(),
|
|
143
157
|
routingStrategy: Schema.union(['round-robin', 'least-loaded', 'lowest-latency']),
|
|
144
158
|
proactiveRateLimitGuard: Schema.boolean(),
|
|
159
|
+
rpmLimit: Schema.number(),
|
|
160
|
+
tpmLimit: Schema.number(),
|
|
145
161
|
})).default([...DEFAULT_PROVIDERS]),
|
|
146
162
|
});
|
|
147
163
|
|
|
@@ -233,17 +249,20 @@ export function apply(ctx, config = {}) {
|
|
|
233
249
|
let lastProfilesRef = null;
|
|
234
250
|
|
|
235
251
|
function buildRuntime() {
|
|
236
|
-
const
|
|
252
|
+
const rawConfig = getConfig();
|
|
253
|
+
// Identity check below stays on the raw object: plainConfig allocates a new
|
|
254
|
+
// object each call, which would defeat the runtime memo cache.
|
|
255
|
+
const cfg = plainConfig(rawConfig ?? {});
|
|
237
256
|
let profilesSnapshot = null;
|
|
238
257
|
try {
|
|
239
258
|
profilesSnapshot = readProfiles();
|
|
240
259
|
} catch {
|
|
241
260
|
profilesSnapshot = null;
|
|
242
261
|
}
|
|
243
|
-
if (cachedRuntime &&
|
|
262
|
+
if (cachedRuntime && rawConfig === lastConfigRef && profilesSnapshot === lastProfilesRef) {
|
|
244
263
|
return cachedRuntime;
|
|
245
264
|
}
|
|
246
|
-
lastConfigRef =
|
|
265
|
+
lastConfigRef = rawConfig;
|
|
247
266
|
lastProfilesRef = profilesSnapshot;
|
|
248
267
|
|
|
249
268
|
moduleBreaker.configure({
|
|
@@ -261,6 +280,7 @@ export function apply(ctx, config = {}) {
|
|
|
261
280
|
const quotaResetWindow = cfg.quotaResetWindow ?? { type: 'midnight_utc', hour: 0 };
|
|
262
281
|
const rateLimitThreshold = cfg.rateLimitThreshold ?? 0.1;
|
|
263
282
|
const rpmLimit = cfg.rpmLimit ?? 0;
|
|
283
|
+
const tpmLimit = cfg.tpmLimit ?? 0;
|
|
264
284
|
const webhookActionToken = cfg.webhookActionToken ?? '';
|
|
265
285
|
|
|
266
286
|
const built = buildPools({ cfg, poolState, defaultState: initializePoolState });
|
|
@@ -306,7 +326,7 @@ export function apply(ctx, config = {}) {
|
|
|
306
326
|
|
|
307
327
|
cachedRuntime = {
|
|
308
328
|
switchCodes, cooldownMs, maxCooldownMs, notifyWebhook, notifyThreshold,
|
|
309
|
-
concurrencyLimit, cascade, quotaResetWindow, rateLimitThreshold, rpmLimit,
|
|
329
|
+
concurrencyLimit, cascade, quotaResetWindow, rateLimitThreshold, rpmLimit, tpmLimit,
|
|
310
330
|
webhookActionToken, expiryWarnDays: cfg.expiryWarnDays ?? 7,
|
|
311
331
|
switchNotify: cfg.switchNotify ?? false, verboseLogging: cfg.verboseLogging ?? false,
|
|
312
332
|
proactiveRateLimitGuard: cfg.proactiveRateLimitGuard ?? true,
|
|
@@ -362,7 +382,7 @@ export function apply(ctx, config = {}) {
|
|
|
362
382
|
let _rotateStartMs = Date.now();
|
|
363
383
|
function recordLatency(pool, reqStore) {
|
|
364
384
|
try {
|
|
365
|
-
const cfg = getConfig();
|
|
385
|
+
const cfg = plainConfig(getConfig());
|
|
366
386
|
if (!cfg || cfg.latencyEnabled === false) return;
|
|
367
387
|
const ref = reqStore?.pickedRef ?? pool?.state?.lastUsed;
|
|
368
388
|
if (!ref) return;
|
package/lib/ops-status.js
CHANGED
|
@@ -13,7 +13,7 @@ import {
|
|
|
13
13
|
costForWeek,
|
|
14
14
|
isLoopbackAddress,
|
|
15
15
|
} from './pool.js';
|
|
16
|
-
import { bucketInfo } from './bucket.js';
|
|
16
|
+
import { bucketInfo, tpmInfo } from './bucket.js';
|
|
17
17
|
import { sanitizeSnapshot } from './sanitize-snapshot.js';
|
|
18
18
|
import {
|
|
19
19
|
STATUS_PATH,
|
|
@@ -101,6 +101,8 @@ ctx.effect(() => ctx.webServer.register({
|
|
|
101
101
|
cooldownMsLeft: until !== undefined && until > now ? until - now : 0,
|
|
102
102
|
// #210: RPM capacity snapshot (null when rpmLimit is off)
|
|
103
103
|
rpm: bucketInfo(pool.state.rpmWindows, ref, pool.rpmLimit, now),
|
|
104
|
+
// TPM capacity snapshot (null when tpmLimit is off)
|
|
105
|
+
tpm: tpmInfo(pool.state.tpmWindows, ref, pool.tpmLimit, now),
|
|
104
106
|
// #215: effective round-robin weight of this key
|
|
105
107
|
weight: pool.weights?.[pool.refs.indexOf(ref)] ?? 1,
|
|
106
108
|
usage: pool.state.usageCounts?.get(ref) ?? 0,
|
package/lib/persistence.js
CHANGED
|
@@ -33,8 +33,17 @@ export class StatePersistence {
|
|
|
33
33
|
if (st && st.failedUntil && typeof st.failedUntil.forEach === 'function') {
|
|
34
34
|
st.failedUntil.forEach((until, ref) => {
|
|
35
35
|
if (Number.isFinite(until)) failedUntil[ref] = until;
|
|
36
|
+
else if (until === Infinity) failedUntil[ref] = 'Infinity';
|
|
36
37
|
});
|
|
37
38
|
}
|
|
39
|
+
const revokedRefs = [];
|
|
40
|
+
if (st && st.revokedRefs) {
|
|
41
|
+
if (typeof st.revokedRefs.forEach === 'function') {
|
|
42
|
+
st.revokedRefs.forEach((r) => revokedRefs.push(r));
|
|
43
|
+
} else if (Array.isArray(st.revokedRefs)) {
|
|
44
|
+
revokedRefs.push(...st.revokedRefs);
|
|
45
|
+
}
|
|
46
|
+
}
|
|
38
47
|
const tokenUsage = {};
|
|
39
48
|
if (st && st.tokenUsage && typeof st.tokenUsage.forEach === 'function') {
|
|
40
49
|
st.tokenUsage.forEach((entry, ref) => {
|
|
@@ -45,14 +54,38 @@ export class StatePersistence {
|
|
|
45
54
|
tokenUsage[ref] = { used: Math.floor(used), resetAt: Math.floor(resetAt) };
|
|
46
55
|
});
|
|
47
56
|
}
|
|
57
|
+
const costDays = {};
|
|
58
|
+
if (st?.costDays) {
|
|
59
|
+
const src = st.costDays instanceof Map ? Object.fromEntries(st.costDays) : st.costDays;
|
|
60
|
+
if (typeof src === 'object' && src !== null) Object.assign(costDays, src);
|
|
61
|
+
}
|
|
62
|
+
const costPerKey = {};
|
|
63
|
+
if (st?.costPerKey) {
|
|
64
|
+
const src = st.costPerKey instanceof Map ? Object.fromEntries(st.costPerKey) : st.costPerKey;
|
|
65
|
+
if (typeof src === 'object' && src !== null) Object.assign(costPerKey, src);
|
|
66
|
+
}
|
|
67
|
+
const usageDays = {};
|
|
68
|
+
if (st?.usageDays) {
|
|
69
|
+
const src = st.usageDays instanceof Map ? Object.fromEntries(st.usageDays) : st.usageDays;
|
|
70
|
+
if (typeof src === 'object' && src !== null) Object.assign(usageDays, src);
|
|
71
|
+
}
|
|
72
|
+
const usageCounts = {};
|
|
73
|
+
if (st?.usageCounts instanceof Map) {
|
|
74
|
+
st.usageCounts.forEach((cnt, ref) => {
|
|
75
|
+
if (Number.isFinite(cnt)) usageCounts[ref] = cnt;
|
|
76
|
+
});
|
|
77
|
+
}
|
|
48
78
|
const outPool = {
|
|
49
79
|
failedUntil,
|
|
50
80
|
pointer: Number.isFinite(st?.pointer) ? st.pointer : 0,
|
|
51
81
|
lastUsed: typeof st?.lastUsed === 'string' ? st.lastUsed : null,
|
|
52
82
|
};
|
|
53
|
-
if (
|
|
54
|
-
|
|
55
|
-
|
|
83
|
+
if (revokedRefs.length > 0) outPool.revokedRefs = revokedRefs;
|
|
84
|
+
if (Object.keys(tokenUsage).length > 0) outPool.tokenUsage = tokenUsage;
|
|
85
|
+
if (Object.keys(costDays).length > 0) outPool.costDays = costDays;
|
|
86
|
+
if (Object.keys(costPerKey).length > 0) outPool.costPerKey = costPerKey;
|
|
87
|
+
if (Object.keys(usageDays).length > 0) outPool.usageDays = usageDays;
|
|
88
|
+
if (Object.keys(usageCounts).length > 0) outPool.usageCounts = usageCounts;
|
|
56
89
|
pools[base] = outPool;
|
|
57
90
|
});
|
|
58
91
|
}
|
|
@@ -109,7 +142,7 @@ export class StatePersistence {
|
|
|
109
142
|
async load() {
|
|
110
143
|
const raw = await safeReadJson(this.filePath, null);
|
|
111
144
|
if (!raw || typeof raw !== 'object' || Array.isArray(raw)) return null;
|
|
112
|
-
if (raw.version !== 1) return null;
|
|
145
|
+
if (raw.version !== 1 && raw.version !== 2) return null;
|
|
113
146
|
return raw;
|
|
114
147
|
}
|
|
115
148
|
|
|
@@ -126,7 +159,17 @@ export class StatePersistence {
|
|
|
126
159
|
if (!existing || !hasPoolActivity(state)) {
|
|
127
160
|
if (saved.failedUntil && typeof saved.failedUntil === 'object' && !Array.isArray(saved.failedUntil)) {
|
|
128
161
|
for (const [ref, until] of Object.entries(saved.failedUntil)) {
|
|
129
|
-
if (
|
|
162
|
+
if (until === 'Infinity' || until === Infinity) {
|
|
163
|
+
state.failedUntil.set(ref, Infinity);
|
|
164
|
+
} else if (Number.isFinite(until)) {
|
|
165
|
+
state.failedUntil.set(ref, until);
|
|
166
|
+
}
|
|
167
|
+
}
|
|
168
|
+
}
|
|
169
|
+
if (Array.isArray(saved.revokedRefs)) {
|
|
170
|
+
if (!state.revokedRefs) state.revokedRefs = new Set();
|
|
171
|
+
for (const r of saved.revokedRefs) {
|
|
172
|
+
if (typeof r === 'string') state.revokedRefs.add(r);
|
|
130
173
|
}
|
|
131
174
|
}
|
|
132
175
|
if (saved.tokenUsage && typeof saved.tokenUsage === 'object' && !Array.isArray(saved.tokenUsage)) {
|
|
@@ -139,6 +182,30 @@ export class StatePersistence {
|
|
|
139
182
|
state.tokenUsage.set(ref, { used: Math.floor(used), resetAt: Math.floor(resetAt) });
|
|
140
183
|
}
|
|
141
184
|
}
|
|
185
|
+
if (saved.costDays && typeof saved.costDays === 'object') {
|
|
186
|
+
if (!state.costDays) state.costDays = new Map();
|
|
187
|
+
for (const [day, cost] of Object.entries(saved.costDays)) {
|
|
188
|
+
if (Number.isFinite(cost)) state.costDays.set(day, cost);
|
|
189
|
+
}
|
|
190
|
+
}
|
|
191
|
+
if (saved.costPerKey && typeof saved.costPerKey === 'object') {
|
|
192
|
+
if (!state.costPerKey) state.costPerKey = new Map();
|
|
193
|
+
for (const [k, c] of Object.entries(saved.costPerKey)) {
|
|
194
|
+
if (Number.isFinite(c)) state.costPerKey.set(k, c);
|
|
195
|
+
}
|
|
196
|
+
}
|
|
197
|
+
if (saved.usageDays && typeof saved.usageDays === 'object') {
|
|
198
|
+
if (!state.usageDays) state.usageDays = new Map();
|
|
199
|
+
for (const [d, u] of Object.entries(saved.usageDays)) {
|
|
200
|
+
if (Number.isFinite(u)) state.usageDays.set(d, u);
|
|
201
|
+
}
|
|
202
|
+
}
|
|
203
|
+
if (saved.usageCounts && typeof saved.usageCounts === 'object') {
|
|
204
|
+
if (!state.usageCounts) state.usageCounts = new Map();
|
|
205
|
+
for (const [k, cnt] of Object.entries(saved.usageCounts)) {
|
|
206
|
+
if (Number.isFinite(cnt)) state.usageCounts.set(k, cnt);
|
|
207
|
+
}
|
|
208
|
+
}
|
|
142
209
|
state.pointer = Number.isSafeInteger(saved.pointer) && saved.pointer >= 0 ? saved.pointer : 0;
|
|
143
210
|
state.lastUsed = typeof saved.lastUsed === 'string' ? saved.lastUsed : null;
|
|
144
211
|
}
|
package/lib/pool-builder.js
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
// lib/pool-builder.js — provider & per-model pool assembly and cleanup
|
|
2
|
-
import { bucketSweep } from './bucket.js';
|
|
2
|
+
import { bucketSweep, tpmSweep } from './bucket.js';
|
|
3
3
|
import { isValidRef } from './pool.js';
|
|
4
4
|
import { normalizeModelQuotas } from './model-quota.js';
|
|
5
5
|
import { createPoolIndex, addProviderPools } from './pool-index.js';
|
|
@@ -26,6 +26,7 @@ export function buildPoolItem({
|
|
|
26
26
|
poolStrategy,
|
|
27
27
|
poolGuard,
|
|
28
28
|
rpmLimit,
|
|
29
|
+
tpmLimit = 0,
|
|
29
30
|
makeState,
|
|
30
31
|
provider = null,
|
|
31
32
|
model = null,
|
|
@@ -52,17 +53,30 @@ export function buildPoolItem({
|
|
|
52
53
|
const exp = parseExpiry(expiresAt[indexed[i].index]);
|
|
53
54
|
if (exp !== undefined) parsedExpiry[refs[i]] = exp;
|
|
54
55
|
}
|
|
56
|
+
} else if (expiresAt && typeof expiresAt === 'object') {
|
|
57
|
+
for (let i = 0; i < refs.length; i++) {
|
|
58
|
+
const exp = parseExpiry(expiresAt[refs[i]]);
|
|
59
|
+
if (exp !== undefined) parsedExpiry[refs[i]] = exp;
|
|
60
|
+
}
|
|
55
61
|
}
|
|
56
62
|
const parsedPaused = new Set();
|
|
57
63
|
if (Array.isArray(paused)) {
|
|
58
64
|
for (let i = 0; i < refs.length; i++) {
|
|
59
|
-
if (paused[indexed[i].index] === true) parsedPaused.add(refs[i]);
|
|
65
|
+
if (paused[indexed[i].index] === true || paused.includes(refs[i])) parsedPaused.add(refs[i]);
|
|
66
|
+
}
|
|
67
|
+
} else if (paused instanceof Set) {
|
|
68
|
+
for (let i = 0; i < refs.length; i++) {
|
|
69
|
+
if (paused.has(refs[i])) parsedPaused.add(refs[i]);
|
|
60
70
|
}
|
|
61
71
|
}
|
|
62
72
|
const parsedRevoked = new Set();
|
|
63
73
|
if (Array.isArray(revoked)) {
|
|
64
74
|
for (let i = 0; i < refs.length; i++) {
|
|
65
|
-
if (revoked[indexed[i].index] === true) parsedRevoked.add(refs[i]);
|
|
75
|
+
if (revoked[indexed[i].index] === true || revoked.includes(refs[i])) parsedRevoked.add(refs[i]);
|
|
76
|
+
}
|
|
77
|
+
} else if (revoked instanceof Set) {
|
|
78
|
+
for (let i = 0; i < refs.length; i++) {
|
|
79
|
+
if (revoked.has(refs[i])) parsedRevoked.add(refs[i]);
|
|
66
80
|
}
|
|
67
81
|
}
|
|
68
82
|
const weightsMap = {};
|
|
@@ -88,6 +102,7 @@ export function buildPoolItem({
|
|
|
88
102
|
maxCooldownMs: poolMax,
|
|
89
103
|
expiresAt: parsedExpiry,
|
|
90
104
|
rpmLimit,
|
|
105
|
+
tpmLimit: typeof tpmLimit === 'number' && Number.isFinite(tpmLimit) && tpmLimit > 0 ? Math.floor(tpmLimit) : 0,
|
|
91
106
|
routingStrategy: poolStrategy,
|
|
92
107
|
proactiveRateLimitGuard: poolGuard,
|
|
93
108
|
};
|
|
@@ -95,6 +110,7 @@ export function buildPoolItem({
|
|
|
95
110
|
|
|
96
111
|
export function buildPools({ cfg = {}, poolState = new Map(), defaultState = initializePoolState } = {}) {
|
|
97
112
|
const rpmLimit = cfg.rpmLimit ?? 0;
|
|
113
|
+
const tpmLimit = cfg.tpmLimit ?? 0;
|
|
98
114
|
const quotaResetWindow = cfg.quotaResetWindow ?? null;
|
|
99
115
|
const poolByRef = new Map();
|
|
100
116
|
const providerToPool = new Map();
|
|
@@ -111,19 +127,21 @@ export function buildPools({ cfg = {}, poolState = new Map(), defaultState = ini
|
|
|
111
127
|
return initializePoolState(st);
|
|
112
128
|
};
|
|
113
129
|
|
|
114
|
-
const buildPool = (base, keys, weights, paused, revoked, poolCooldown, poolMax, expiresAt, poolStrategy, poolGuard, meta) =>
|
|
130
|
+
const buildPool = (base, keys, weights, paused, revoked, poolCooldown, poolMax, expiresAt, poolStrategy, poolGuard, meta, pRpm = rpmLimit, pTpm = tpmLimit) =>
|
|
115
131
|
buildPoolItem({
|
|
116
132
|
base, keys, weights, paused, revoked, poolCooldown, poolMax, expiresAt, poolStrategy, poolGuard,
|
|
117
|
-
rpmLimit, makeState, quotaResetWindow, ...meta,
|
|
133
|
+
rpmLimit: pRpm, tpmLimit: pTpm, makeState, quotaResetWindow, ...meta,
|
|
118
134
|
});
|
|
119
135
|
|
|
120
136
|
for (const p of cfg.providers ?? []) {
|
|
121
137
|
const poolCooldown = typeof p.cooldownMs === 'number' ? p.cooldownMs : (cfg.cooldownMs ?? 60000);
|
|
122
138
|
const poolMax = typeof p.maxCooldownMs === 'number' ? p.maxCooldownMs : (cfg.maxCooldownMs ?? undefined);
|
|
139
|
+
const provRpm = typeof p.rpmLimit === 'number' ? p.rpmLimit : rpmLimit;
|
|
140
|
+
const provTpm = typeof p.tpmLimit === 'number' ? p.tpmLimit : tpmLimit;
|
|
123
141
|
const pool = buildPool(p.provider, p.keys, p.weights, p.paused, p.revoked, poolCooldown, poolMax, p.expiresAt, p.routingStrategy, p.proactiveRateLimitGuard, {
|
|
124
142
|
provider: p.provider,
|
|
125
143
|
model: null,
|
|
126
|
-
});
|
|
144
|
+
}, provRpm, provTpm);
|
|
127
145
|
if (pool) {
|
|
128
146
|
for (const ref of pool.refs) {
|
|
129
147
|
if (!poolByRef.has(ref)) poolByRef.set(ref, pool);
|
|
@@ -133,12 +151,24 @@ export function buildPools({ cfg = {}, poolState = new Map(), defaultState = ini
|
|
|
133
151
|
}
|
|
134
152
|
const byModel = new Map();
|
|
135
153
|
for (const [model, mp] of Object.entries(p.models ?? {})) {
|
|
136
|
-
const
|
|
154
|
+
const modelRpm = typeof mp.rpmLimit === 'number' ? mp.rpmLimit : provRpm;
|
|
155
|
+
const modelTpm = typeof mp.tpmLimit === 'number' ? mp.tpmLimit : provTpm;
|
|
156
|
+
const mergedPaused = new Set([
|
|
157
|
+
...(pool?.pausedRefs ?? []),
|
|
158
|
+
...(Array.isArray(mp.paused) ? mp.paused : (mp.paused instanceof Set ? [...mp.paused] : [])),
|
|
159
|
+
]);
|
|
160
|
+
const mergedRevoked = new Set([
|
|
161
|
+
...(pool?.revokedRefs ?? []),
|
|
162
|
+
...(Array.isArray(mp.revoked) ? mp.revoked : (mp.revoked instanceof Set ? [...mp.revoked] : [])),
|
|
163
|
+
]);
|
|
164
|
+
const modelExpiresAt = { ...(pool?.expiresAt ?? {}), ...(typeof mp.expiresAt === 'object' ? mp.expiresAt : {}) };
|
|
165
|
+
const mpool = buildPool(`${p.provider}::${model}`, mp.keys, mp.weights, mergedPaused, mergedRevoked, poolCooldown, poolMax, modelExpiresAt, p.routingStrategy, p.proactiveRateLimitGuard, {
|
|
137
166
|
provider: p.provider,
|
|
138
167
|
model,
|
|
139
168
|
quotas: mp.quotas,
|
|
140
|
-
});
|
|
169
|
+
}, modelRpm, modelTpm);
|
|
141
170
|
if (!mpool) continue;
|
|
171
|
+
mpool.basePool = pool;
|
|
142
172
|
byModel.set(model, mpool);
|
|
143
173
|
for (const ref of mpool.refs) {
|
|
144
174
|
if (!poolByRef.has(ref)) poolByRef.set(ref, mpool);
|
|
@@ -187,6 +217,7 @@ export function cleanupRemovedProviders({
|
|
|
187
217
|
}
|
|
188
218
|
for (const st of poolState.values()) {
|
|
189
219
|
if (st.rpmWindows) bucketSweep(st.rpmWindows, liveRefs);
|
|
220
|
+
if (st.tpmWindows) tpmSweep(st.tpmWindows, liveRefs);
|
|
190
221
|
if (st.tokenUsage instanceof Map) {
|
|
191
222
|
for (const ref of [...st.tokenUsage.keys()]) {
|
|
192
223
|
if (!liveRefs.has(ref)) st.tokenUsage.delete(ref);
|
package/lib/pool-state.js
CHANGED
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
const MAP_FIELDS = [
|
|
4
4
|
'failedUntil', 'failCounts', 'authFailCounts', 'brokenUntil', 'costPerKey',
|
|
5
5
|
'lastUsedAt', 'usageCounts', 'byModel', 'usageDays', 'quotaWindows', 'costDays',
|
|
6
|
-
'tokenUsage',
|
|
6
|
+
'tokenUsage', 'rpmWindows', 'tpmWindows',
|
|
7
7
|
];
|
|
8
8
|
|
|
9
9
|
export function initializePoolState(state = {}) {
|
package/lib/pool.js
CHANGED
|
@@ -316,6 +316,7 @@ export function isKeyPaused(pool, ref) {
|
|
|
316
316
|
if (Array.isArray(pool.pausedRefs) && pool.pausedRefs.includes(ref)) return true;
|
|
317
317
|
if (pool.state && pool.state.pausedRefs instanceof Set && pool.state.pausedRefs.has(ref)) return true;
|
|
318
318
|
if (pool.state && Array.isArray(pool.state.pausedRefs) && pool.state.pausedRefs.includes(ref)) return true;
|
|
319
|
+
if (pool.basePool) return isKeyPaused(pool.basePool, ref);
|
|
319
320
|
return false;
|
|
320
321
|
}
|
|
321
322
|
|
|
@@ -560,5 +561,6 @@ export function isKeyRevoked(pool, ref) {
|
|
|
560
561
|
if (Array.isArray(pool.revokedRefs) && pool.revokedRefs.includes(ref)) return true;
|
|
561
562
|
if (pool.state && pool.state.revokedRefs instanceof Set && pool.state.revokedRefs.has(ref)) return true;
|
|
562
563
|
if (pool.state && Array.isArray(pool.state.revokedRefs) && pool.state.revokedRefs.includes(ref)) return true;
|
|
564
|
+
if (pool.basePool) return isKeyRevoked(pool.basePool, ref);
|
|
563
565
|
return false;
|
|
564
566
|
}
|
package/lib/resolver.js
CHANGED
|
@@ -9,7 +9,7 @@
|
|
|
9
9
|
// credentials service. It never sees or returns a secret value itself; it
|
|
10
10
|
// delegates to `original(ref)`.
|
|
11
11
|
|
|
12
|
-
import { bucketAllow, bucketRetryMs } from './bucket.js';
|
|
12
|
+
import { bucketAllow, bucketRetryMs, tpmAllow, tpmRetryMs } from './bucket.js';
|
|
13
13
|
import { envValue, sortAttemptList, isKeyPaused, isKeyRevoked } from './pool.js';
|
|
14
14
|
import { decryptSecret } from './crypto-storage.js';
|
|
15
15
|
import { resolveRefPool, isQuotaManagedRequest } from './pool-index.js';
|
|
@@ -22,6 +22,7 @@ import {
|
|
|
22
22
|
|
|
23
23
|
/** Internal failure code for "local budgets forbid this dispatch". */
|
|
24
24
|
export const LOCAL_QUOTA_CODE = 'LOCAL_MODEL_QUOTA_EXHAUSTED';
|
|
25
|
+
export const LOCAL_POOL_EXHAUSTED_CODE = 'LOCAL_POOL_EXHAUSTED';
|
|
25
26
|
|
|
26
27
|
/**
|
|
27
28
|
* @param {object} deps
|
|
@@ -78,7 +79,8 @@ export function createResolver({
|
|
|
78
79
|
if (isKeyPaused(pool, candidate)) continue;
|
|
79
80
|
const until = pool.state.failedUntil.get(candidate);
|
|
80
81
|
if (until !== undefined && until > at) continue;
|
|
81
|
-
|
|
82
|
+
const candidateExp = pool.expiresAt?.[candidate] ?? pool.basePool?.expiresAt?.[candidate];
|
|
83
|
+
if (candidateExp !== undefined && at >= candidateExp) continue;
|
|
82
84
|
// Local model token budget: a spent credential is simply not a candidate.
|
|
83
85
|
// This is deliberately NOT recorded as a failure — the key is healthy and
|
|
84
86
|
// merely out of budget — so it must never touch failedUntil / failCounts /
|
|
@@ -95,6 +97,17 @@ export function createResolver({
|
|
|
95
97
|
continue;
|
|
96
98
|
}
|
|
97
99
|
}
|
|
100
|
+
const tpmLimit = pool.tpmLimit ?? 0;
|
|
101
|
+
if (tpmLimit > 0) {
|
|
102
|
+
if (!pool.state.tpmWindows) pool.state.tpmWindows = new Map();
|
|
103
|
+
if (!tpmAllow(pool.state.tpmWindows, candidate, tpmLimit, at)) {
|
|
104
|
+
const waitMs = tpmRetryMs(pool.state.tpmWindows, candidate, tpmLimit, at);
|
|
105
|
+
if ((pool.state.failedUntil.get(candidate) ?? 0) < at + waitMs) {
|
|
106
|
+
pool.state.failedUntil.set(candidate, at + waitMs);
|
|
107
|
+
}
|
|
108
|
+
continue;
|
|
109
|
+
}
|
|
110
|
+
}
|
|
98
111
|
if (pool.perHour) {
|
|
99
112
|
if (!pool.state.quotaWindows) pool.state.quotaWindows = new Map();
|
|
100
113
|
let win = pool.state.quotaWindows.get(candidate);
|
|
@@ -133,7 +146,10 @@ export function createResolver({
|
|
|
133
146
|
{ code: LOCAL_QUOTA_CODE, localQuota: true },
|
|
134
147
|
);
|
|
135
148
|
}
|
|
136
|
-
|
|
149
|
+
throw Object.assign(
|
|
150
|
+
new Error(`dsh-key-rotation: all credentials in pool "${pool.base ?? pool.provider}" are currently blocked, paused, expired, or exhausted`),
|
|
151
|
+
{ code: LOCAL_POOL_EXHAUSTED_CODE, localExhausted: true },
|
|
152
|
+
);
|
|
137
153
|
};
|
|
138
154
|
}
|
|
139
155
|
|
package/lib/rotate.js
CHANGED
|
@@ -16,6 +16,7 @@ import {
|
|
|
16
16
|
} from './pool.js';
|
|
17
17
|
import { pickCascadeFallback, CASCADE_MAX_DEPTH } from './cascade.js';
|
|
18
18
|
import { nextQuotaReset } from './quota-window.js';
|
|
19
|
+
import { tpmRecord } from './bucket.js';
|
|
19
20
|
import {
|
|
20
21
|
isModelQuotaAvailable,
|
|
21
22
|
hasModelQuotaConfig,
|
|
@@ -209,9 +210,10 @@ export function createRotate(deps) {
|
|
|
209
210
|
return llm.stream({ ...options, [MARKER]: true });
|
|
210
211
|
});
|
|
211
212
|
} catch (e) {
|
|
212
|
-
if (e?.localQuota || e?.code === LOCAL_QUOTA_CODE) {
|
|
213
|
-
|
|
214
|
-
|
|
213
|
+
if (e?.localQuota || e?.localExhausted || e?.code === LOCAL_QUOTA_CODE || e?.code === LOCAL_POOL_EXHAUSTED_CODE) {
|
|
214
|
+
const code = e?.code ?? (e?.localQuota ? LOCAL_QUOTA_CODE : LOCAL_POOL_EXHAUSTED_CODE);
|
|
215
|
+
lastFailure = finishError(code, String(e?.message ?? ''));
|
|
216
|
+
logWarn(`[dsh-key-rotation] ${options.provider}/${options.model ?? '-'}: local rejection (${code}) — no upstream request sent`);
|
|
215
217
|
break;
|
|
216
218
|
}
|
|
217
219
|
const curRef = reqStore.pickedRef ?? pool.state.lastUsed;
|
|
@@ -302,9 +304,16 @@ export function createRotate(deps) {
|
|
|
302
304
|
if (activeRef) {
|
|
303
305
|
recordSuccess(pool, activeRef, now());
|
|
304
306
|
}
|
|
307
|
+
if (activeRef) {
|
|
308
|
+
const usedTokens = extractUsageTokens(chunk?.usage ?? chunk?.metadata?.usage);
|
|
309
|
+
if (usedTokens && usedTokens > 0) {
|
|
310
|
+
if (!pool.state.tpmWindows) pool.state.tpmWindows = new Map();
|
|
311
|
+
tpmRecord(pool.state.tpmWindows, activeRef, usedTokens, Date.now());
|
|
312
|
+
}
|
|
313
|
+
}
|
|
305
314
|
if (activeRef && quotaManaged) {
|
|
306
315
|
const charged = consumeModelTokens(
|
|
307
|
-
pool, activeRef, extractUsageTokens(chunk
|
|
316
|
+
pool, activeRef, extractUsageTokens(chunk?.usage ?? chunk?.metadata?.usage),
|
|
308
317
|
runtime0.quotaResetWindow, Date.now(),
|
|
309
318
|
);
|
|
310
319
|
if (charged) {
|
package/lib/safe-fetch.js
CHANGED
|
@@ -1,20 +1,57 @@
|
|
|
1
|
-
// lib/safe-fetch.js — SSRF-protected HTTP fetch with DNS and IP validation (#354).
|
|
1
|
+
// lib/safe-fetch.js — SSRF-protected HTTP fetch with DNS and IP validation (#354, #411).
|
|
2
2
|
import net from 'node:net';
|
|
3
3
|
import dns from 'node:dns/promises';
|
|
4
|
+
import nodeDns from 'node:dns';
|
|
5
|
+
import { Agent } from 'undici';
|
|
4
6
|
|
|
5
7
|
export const MAX_IMPORT_BYTES = 1024 * 1024; // 1 MB limit
|
|
6
8
|
export const MAX_REDIRECTS = 5;
|
|
7
9
|
|
|
10
|
+
/**
|
|
11
|
+
* Parses an IPv6 string into an array of 8 16-bit integers (words).
|
|
12
|
+
* Supports standard hex, compressed `::`, and embedded dotted IPv4 (e.g. `::ffff:127.0.0.1`).
|
|
13
|
+
* @param {string} ip
|
|
14
|
+
* @returns {number[]|null}
|
|
15
|
+
*/
|
|
16
|
+
export function parseIpv6Words(ip) {
|
|
17
|
+
if (typeof ip !== 'string') return null;
|
|
18
|
+
let s = ip.toLowerCase().trim().replace(/^\[|\]$/g, '');
|
|
19
|
+
const lastColon = s.lastIndexOf(':');
|
|
20
|
+
if (lastColon !== -1) {
|
|
21
|
+
const tail = s.slice(lastColon + 1);
|
|
22
|
+
if (net.isIP(tail) === 4) {
|
|
23
|
+
const parts = tail.split('.').map(Number);
|
|
24
|
+
if (parts.length === 4 && !parts.some(n => isNaN(n) || n < 0 || n > 255)) {
|
|
25
|
+
const w6 = ((parts[0] << 8) | parts[1]) & 0xffff;
|
|
26
|
+
const w7 = ((parts[2] << 8) | parts[3]) & 0xffff;
|
|
27
|
+
s = s.slice(0, lastColon) + ':' + w6.toString(16) + ':' + w7.toString(16);
|
|
28
|
+
}
|
|
29
|
+
}
|
|
30
|
+
}
|
|
31
|
+
const halves = s.split('::');
|
|
32
|
+
if (halves.length > 2) return null;
|
|
33
|
+
const left = halves[0] ? halves[0].split(':').map(x => parseInt(x, 16)) : [];
|
|
34
|
+
const right = halves.length === 2 && halves[1] ? halves[1].split(':').map(x => parseInt(x, 16)) : [];
|
|
35
|
+
if (left.some(isNaN) || right.some(isNaN)) return null;
|
|
36
|
+
const missing = 8 - (left.length + right.length);
|
|
37
|
+
if (halves.length === 1 && missing !== 0) return null;
|
|
38
|
+
if (missing < 0) return null;
|
|
39
|
+
const middle = new Array(missing).fill(0);
|
|
40
|
+
return [...left, ...middle, ...right];
|
|
41
|
+
}
|
|
42
|
+
|
|
8
43
|
/**
|
|
9
44
|
* Checks if an IPv4 or IPv6 address is private, loopback, link-local, multicast, or reserved.
|
|
45
|
+
* Canonicalizes IPv6 addresses including hex and dotted-quad IPv4-mapped forms (#411).
|
|
10
46
|
* @param {string} ip
|
|
11
47
|
* @returns {boolean} true if private/reserved/loopback
|
|
12
48
|
*/
|
|
13
49
|
export function isPrivateOrReservedIp(ip) {
|
|
14
50
|
if (!ip || typeof ip !== 'string') return true;
|
|
15
|
-
const
|
|
51
|
+
const cleaned = ip.trim().replace(/^\[|\]$/g, '');
|
|
52
|
+
const kind = net.isIP(cleaned);
|
|
16
53
|
if (kind === 4) {
|
|
17
|
-
const parts =
|
|
54
|
+
const parts = cleaned.split('.').map(Number);
|
|
18
55
|
if (parts.length !== 4 || parts.some(n => isNaN(n) || n < 0 || n > 255)) return true;
|
|
19
56
|
const [a, b, c, d] = parts;
|
|
20
57
|
if (a === 0) return true; // 0.0.0.0/8 current network
|
|
@@ -33,22 +70,70 @@ export function isPrivateOrReservedIp(ip) {
|
|
|
33
70
|
return false;
|
|
34
71
|
}
|
|
35
72
|
if (kind === 6) {
|
|
36
|
-
const
|
|
37
|
-
if (
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
73
|
+
const words = parseIpv6Words(cleaned);
|
|
74
|
+
if (!words || words.length !== 8) return true;
|
|
75
|
+
const [w0, w1, w2, w3, w4, w5, w6, w7] = words;
|
|
76
|
+
// Unspecified :: or loopback ::1
|
|
77
|
+
if (w0 === 0 && w1 === 0 && w2 === 0 && w3 === 0 && w4 === 0 && w5 === 0 && w6 === 0) {
|
|
78
|
+
if (w7 === 0 || w7 === 1) return true;
|
|
79
|
+
}
|
|
80
|
+
// IPv4-mapped (::ffff:0:0/96), IPv4-compatible (::/96), or NAT64 (64:ff9b::/96)
|
|
81
|
+
const isMapped = (w0 === 0 && w1 === 0 && w2 === 0 && w3 === 0 && w4 === 0 && w5 === 0xffff);
|
|
82
|
+
const isCompat = (w0 === 0 && w1 === 0 && w2 === 0 && w3 === 0 && w4 === 0 && w5 === 0);
|
|
83
|
+
const isNat64 = (w0 === 0x0064 && w1 === 0xff9b && w2 === 0 && w3 === 0 && w4 === 0 && w5 === 0);
|
|
84
|
+
if (isMapped || isCompat || isNat64) {
|
|
85
|
+
const a = (w6 >> 8) & 0xff;
|
|
86
|
+
const b = w6 & 0xff;
|
|
87
|
+
const c = (w7 >> 8) & 0xff;
|
|
88
|
+
const d = w7 & 0xff;
|
|
89
|
+
return isPrivateOrReservedIp(`${a}.${b}.${c}.${d}`);
|
|
41
90
|
}
|
|
42
|
-
if (
|
|
43
|
-
if (
|
|
44
|
-
if (
|
|
45
|
-
if (
|
|
46
|
-
if (
|
|
91
|
+
if ((w0 & 0xffc0) === 0xfe80) return true; // fe80::/10 link-local
|
|
92
|
+
if ((w0 & 0xfe00) === 0xfc00) return true; // fc00::/7 unique local
|
|
93
|
+
if ((w0 & 0xff00) === 0xff00) return true; // ff00::/8 multicast
|
|
94
|
+
if (w0 === 0x2001 && w1 === 0x0db8) return true; // 2001:db8::/32 documentation
|
|
95
|
+
if (w0 === 0x0100 && w1 === 0) return true; // 100::/64 discard
|
|
47
96
|
return false;
|
|
48
97
|
}
|
|
49
98
|
return true;
|
|
50
99
|
}
|
|
51
100
|
|
|
101
|
+
/**
|
|
102
|
+
* Creates an undici Agent with connect-time lookup validation against DNS rebinding / TOCTOU (#411).
|
|
103
|
+
* @param {object} [opts]
|
|
104
|
+
* @param {Function} [opts.lookupImpl]
|
|
105
|
+
* @returns {Agent}
|
|
106
|
+
*/
|
|
107
|
+
export function createSafeDispatcher({ lookupImpl = nodeDns.lookup } = {}) {
|
|
108
|
+
return new Agent({
|
|
109
|
+
connect: {
|
|
110
|
+
lookup: (hostname, options, callback) => {
|
|
111
|
+
const cb = typeof options === 'function' ? options : callback;
|
|
112
|
+
const opts = typeof options === 'function' ? {} : options;
|
|
113
|
+
lookupImpl(hostname, { ...opts, all: true }, (err, addresses) => {
|
|
114
|
+
if (err) return cb(err);
|
|
115
|
+
const list = Array.isArray(addresses) ? addresses : [addresses];
|
|
116
|
+
for (const item of list) {
|
|
117
|
+
const addr = typeof item === 'string' ? item : item.address;
|
|
118
|
+
if (isPrivateOrReservedIp(addr)) {
|
|
119
|
+
const ssrfErr = new Error(`dsh-key-rotation: connect blocked: host ${hostname} resolves to private or reserved IP ${addr}`);
|
|
120
|
+
ssrfErr.code = 'SSRF_BLOCKED';
|
|
121
|
+
return cb(ssrfErr);
|
|
122
|
+
}
|
|
123
|
+
}
|
|
124
|
+
if (typeof options === 'function') {
|
|
125
|
+
const first = list[0];
|
|
126
|
+
const addr = typeof first === 'string' ? first : first.address;
|
|
127
|
+
const fam = typeof first === 'object' && first.family ? first.family : (net.isIP(addr) || 4);
|
|
128
|
+
return cb(null, addr, fam);
|
|
129
|
+
}
|
|
130
|
+
return cb(null, addresses);
|
|
131
|
+
});
|
|
132
|
+
},
|
|
133
|
+
},
|
|
134
|
+
});
|
|
135
|
+
}
|
|
136
|
+
|
|
52
137
|
/**
|
|
53
138
|
* Validates a target URL and its hostname/IP against SSRF.
|
|
54
139
|
* @param {string} urlString
|
|
@@ -173,7 +258,7 @@ export async function readBoundedJson(resp, maxBytes = MAX_IMPORT_BYTES) {
|
|
|
173
258
|
}
|
|
174
259
|
|
|
175
260
|
/**
|
|
176
|
-
* Fetches JSON from a URL with SSRF protection, redirect verification, and size bounding.
|
|
261
|
+
* Fetches JSON from a URL with SSRF protection, connect-time rebinding guard, redirect verification, and size bounding.
|
|
177
262
|
* @param {string} url
|
|
178
263
|
* @param {object} options
|
|
179
264
|
*/
|
|
@@ -183,14 +268,18 @@ export async function safeFetchJson(url, {
|
|
|
183
268
|
lookupImpl = dns.lookup,
|
|
184
269
|
maxBytes = MAX_IMPORT_BYTES,
|
|
185
270
|
redirectsLeft = MAX_REDIRECTS,
|
|
271
|
+
dispatcher,
|
|
186
272
|
} = {}) {
|
|
273
|
+
const safeDispatcher = dispatcher ?? (fetchImpl === globalThis.fetch ? createSafeDispatcher() : undefined);
|
|
187
274
|
let currentUrl = url;
|
|
188
275
|
while (true) {
|
|
189
276
|
await assertSafeUrl(currentUrl, { lookupImpl });
|
|
190
|
-
const
|
|
277
|
+
const fetchOpts = {
|
|
191
278
|
signal,
|
|
192
279
|
redirect: 'manual',
|
|
193
|
-
}
|
|
280
|
+
};
|
|
281
|
+
if (safeDispatcher) fetchOpts.dispatcher = safeDispatcher;
|
|
282
|
+
const resp = await fetchImpl(currentUrl, fetchOpts);
|
|
194
283
|
|
|
195
284
|
if ([301, 302, 303, 307, 308].includes(resp.status)) {
|
|
196
285
|
if (redirectsLeft <= 0) {
|
|
@@ -218,4 +307,4 @@ export async function safeFetchJson(url, {
|
|
|
218
307
|
|
|
219
308
|
return await readBoundedJson(resp, maxBytes);
|
|
220
309
|
}
|
|
221
|
-
}
|
|
310
|
+
}
|
package/lib/sanitize-snapshot.js
CHANGED
|
@@ -45,6 +45,14 @@ export function sanitizeKeyStatus(key, now = Date.now()) {
|
|
|
45
45
|
limit: clampNonNegative(key.rpm.limit),
|
|
46
46
|
}
|
|
47
47
|
: key.rpm,
|
|
48
|
+
tpm: key.tpm && typeof key.tpm === 'object'
|
|
49
|
+
? {
|
|
50
|
+
...key.tpm,
|
|
51
|
+
used: clampNonNegative(key.tpm.used),
|
|
52
|
+
remaining: clampNonNegative(key.tpm.remaining),
|
|
53
|
+
limit: clampNonNegative(key.tpm.limit),
|
|
54
|
+
}
|
|
55
|
+
: key.tpm,
|
|
48
56
|
};
|
|
49
57
|
}
|
|
50
58
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@goodandready/dsh-key-rotation",
|
|
3
|
-
"version": "0.8.
|
|
3
|
+
"version": "0.8.39",
|
|
4
4
|
"packageManager": "pnpm@10.33.2",
|
|
5
5
|
"description": "Per-provider API key rotation for DeepSeek Harness: a key pool per provider, auto-created clone routes, and switching to the next key on quota/rate-limit errors. Includes a Settings section (Key Rotation) to edit the key pools, cooldown and switch codes.",
|
|
6
6
|
"keywords": [
|
|
@@ -57,8 +57,8 @@
|
|
|
57
57
|
},
|
|
58
58
|
"peerDependencies": {
|
|
59
59
|
"@deepseek-ai/cordis": "^4.0.1",
|
|
60
|
-
"@deepseek-ai/
|
|
61
|
-
"@deepseek-ai/
|
|
60
|
+
"@deepseek-ai/dsh-llm": "^0.1.7-rc.2 || ^0.2.0-rc.1",
|
|
61
|
+
"@deepseek-ai/schemastery": "^3.18.4"
|
|
62
62
|
},
|
|
63
63
|
"scripts": {
|
|
64
64
|
"test": "node tools/test.mjs",
|
|
@@ -70,5 +70,8 @@
|
|
|
70
70
|
"devDependencies": {
|
|
71
71
|
"react": "18.3.1",
|
|
72
72
|
"react-test-renderer": "18.3.1"
|
|
73
|
+
},
|
|
74
|
+
"dependencies": {
|
|
75
|
+
"undici": "^6.29.0"
|
|
73
76
|
}
|
|
74
77
|
}
|