@goodandready/dsh-key-rotation 0.8.64 → 0.8.65

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,19 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.8.65 - 2026-10-18
4
+
5
+ ### Features
6
+ - **Traffic Smoothing Micro-Queue (#496)**:
7
+ - Leaky bucket traffic smoother (`lib/traffic-smoother.js`) absorbs short micro-bursts ($\le 1$s) across multi-agent workflows without tripping upstream per-second or per-minute rate limits or triggering cooldown churning.
8
+ - Configurable `trafficSmoothingBurst` (default 5 or `ceil(rpmLimit/12)`) and `maxQueueWaitMs` (default 800ms).
9
+ - If scheduled micro-queue delay exceeds `maxQueueWaitMs`, the smoother flags burst overflow, immediately rotating to the next candidate key in the pool.
10
+ - Live metric `smoothedRequests` exposed on provider status, with `TRAFFIC_SMOOTHED` audit events.
11
+ - **Accurate Dynamic Provider Limits & Canonical Presets (#503)**:
12
+ - Header limits detector (`lib/provider-limits-detector.js`): extracts dynamic rate limit headers (`x-ratelimit-limit-*`, `x-ratelimit-remaining-*`, `x-ratelimit-reset-*`, `anthropic-ratelimit-*`, `retry-after`) from upstream provider HTTP responses into `pool.state.detectedLimits`.
13
+ - Canonical provider presets catalog for DeepSeek, Groq, OpenCode, Anthropic, OpenAI, SiliconFlow, Mistral, Gemini, and OpenRouter with standard RPM, TPM, burst capacity, and calendar reset window specifications.
14
+ - Endpoint `GET/POST /dsh-key-rotation/preset-limits` to query and auto-apply canonical limits to configured providers.
15
+ - WebUI button `⚡ Preset Limits` for 1-click rate limit configuration and live upstream limits indicator badge (`⚡ 60 RPM · 100k TPM`).
16
+
3
17
  ## 0.8.64 - 2026-10-18
4
18
 
5
19
  ### Features
package/README.md CHANGED
@@ -67,6 +67,10 @@
67
67
  - **📡 Status extras**: per-provider `circuit` plus `meta.expectedClones` / `meta.notifyQueue`.
68
68
  - **🧪 Smoke harness**: scripted 429 → next-key → success path in `test/smoke-rotation-080.test.mjs`.
69
69
 
70
+ ### 🚀 What's New in v0.8.65
71
+ - **⚡ Leaky Bucket Traffic Smoothing (#496)**: Micro-queue smoothing absorbs rapid multi-agent burst spikes ($\le 1$s) without triggering unnecessary 429 errors or cooldown churning, with zero-delay burst capacity and early failover rotation on queue overflow.
72
+ - **📊 Dynamic Upstream Rate Limits & Presets (#503)**: Dynamic parsing of provider rate limit headers (`x-ratelimit-*`, `anthropic-ratelimit-*`, `retry-after`), live badge display on keys, and 1-click `⚡ Preset Limits` button applying canonical RPM/TPM presets for DeepSeek, Groq, Anthropic, OpenAI, Gemini, and more.
73
+
70
74
  ### 🛠️ What's New in v0.7.33 (Stability & Bugfix Release)
71
75
  - **🔍 Resolved Key Probing BaseURL**: Fixed `resolveBaseUrl` to map key credential refs to owning provider pools, restoring live `probeModels` testing.
72
76
  - **🛡️ Guarded Cascade Recursion**: Prevented call stack overflow in cross-provider failover when circular cascade chains occur.
package/README.ru.md CHANGED
@@ -36,6 +36,10 @@
36
36
 
37
37
  ---
38
38
 
39
+ ### 🚀 Что нового в v0.8.65
40
+ - **⚡ Сглаживание трафика (Traffic Smoothing Micro-Queue) (#496)**: Микро-очередь leaky bucket сглаживает кратковременные пики нагрузки (<=1с) от мультиагентных вызовов без лишних ошибок 429 и лишних кулдаунов. При превышении таймаута ожидания микро-очереди запрос автоматически переключается на следующий ключ в пуле.
41
+ - **📊 Динамическое определение лимитов провайдеров и пресеты (#503)**: Автоматический парсинг заголовков `x-ratelimit-*`, `anthropic-ratelimit-*`, `retry-after` с отображением актуальных лимитов прямо на карточке ключа, а также кнопка в 1 клик `⚡ Preset Limits` с каноническими пресетами для DeepSeek, Groq, Anthropic, OpenAI, Gemini и др.
42
+
39
43
  ### 🚀 Что нового в v0.8.11 (обновление в один клик и quality gate)
40
44
 
41
45
  - **Обновление плагина в карточке настроек**: текущая/последняя версия и кнопка установки без выхода из DSH (#307).
package/README.zh.md CHANGED
@@ -38,6 +38,10 @@
38
38
 
39
39
  ## ⚡ 概述与核心痛点
40
40
 
41
+ ### 🚀 v0.8.65 新特性
42
+ - **⚡ 流量平滑微队列 (Traffic Smoothing Micro-Queue) (#496)**: 漏桶微队列平滑多智能体调用中的毫秒级突发并发,避免触发不必要的429限流与冷却流转。队列等待超限时自动轮转至池中下一个候选密钥。
43
+ - **📊 动态上游限制检测与预设规则 (#503)**: 自动提取上游响应头中的限流数据 (`x-ratelimit-*`, `anthropic-ratelimit-*`, `retry-after`),并在前端展示实时配额状态;支持一键应用 DeepSeek、Groq、Anthropic、OpenAI、Gemini 等官方规范预设限制。
44
+
41
45
  ### 🚀 v0.8.11 新特性(一键更新与质量门禁)
42
46
 
43
47
  - **设置卡片中的插件更新器**:查看当前/最新版本并一键安装(#307)。
package/lib/client.js CHANGED
@@ -99,6 +99,11 @@ window.__ModuleLoader__.load({
99
99
  keyPaused: 'Paused',
100
100
  keyRevoked: 'Revoked (401)',
101
101
  keyAuthPaused: 'Auth Paused (401/403)',
102
+ presetLimitsBtn: '⚡ Preset Limits',
103
+ presetApplied: 'Applied {prov} preset limits ({rpm} RPM / {tpm} TPM)',
104
+ presetNone: 'No canonical preset for this provider',
105
+ detectedLimitsLabel: 'Upstream: {rpm} RPM · {tpm} TPM',
106
+ trafficSmoothingLabel: 'Traffic Smoothing (Burst)',
102
107
  pauseKey: 'Pause key from rotation',
103
108
  resumeKey: 'Resume key rotation',
104
109
  sparklineTraffic: 'Traffic Trend',
@@ -301,6 +306,11 @@ window.__ModuleLoader__.load({
301
306
  keyPaused: '已暂停',
302
307
  keyRevoked: '已撤销 (401)',
303
308
  keyAuthPaused: '身份验证失败已暂停 (401/403)',
309
+ presetLimitsBtn: '⚡ 预设限制',
310
+ presetApplied: '已应用 {prov} 预设限制 ({rpm} RPM / {tpm} TPM)',
311
+ presetNone: '未找到该提供商预设规则',
312
+ detectedLimitsLabel: '上游限制:{rpm} RPM · {tpm} TPM',
313
+ trafficSmoothingLabel: '流量平滑 (突发并发)',
304
314
  pauseKey: '暂停此密钥轮换',
305
315
  resumeKey: '恢复此密钥轮换',
306
316
  sparklineTraffic: '流量趋势',
@@ -1556,6 +1566,31 @@ window.__ModuleLoader__.load({
1556
1566
  };
1557
1567
 
1558
1568
  const [validating, setValidating] = React.useState('');
1569
+ const [presetNotice, setPresetNotice] = React.useState('');
1570
+ const applyPreset = async (pIndex, provName) => {
1571
+ try {
1572
+ const r = await fetch('/dsh-key-rotation/preset-limits?provider=' + encodeURIComponent(provName));
1573
+ const data = await r.json();
1574
+ if (data && data.preset) {
1575
+ setField((cur) => {
1576
+ const providers = [...(cur.providers ?? [])];
1577
+ const ent = { ...providers[pIndex] };
1578
+ if (data.preset.rpmLimit) ent.rpmLimit = data.preset.rpmLimit;
1579
+ if (data.preset.tpmLimit) ent.tpmLimit = data.preset.tpmLimit;
1580
+ providers[pIndex] = ent;
1581
+ return { ...cur, providers };
1582
+ });
1583
+ setPresetNotice(t('presetApplied').replace('{prov}', provName).replace('{rpm}', String(data.preset.rpmLimit)).replace('{tpm}', String(data.preset.tpmLimit)));
1584
+ setTimeout(() => setPresetNotice(''), 4000);
1585
+ } else {
1586
+ setPresetNotice(t('presetNone'));
1587
+ setTimeout(() => setPresetNotice(''), 3000);
1588
+ }
1589
+ } catch (err) {
1590
+ setPresetNotice(String(err?.message || err));
1591
+ setTimeout(() => setPresetNotice(''), 3000);
1592
+ }
1593
+ };
1559
1594
  const [validationResult, setValidationResult] = React.useState({});
1560
1595
  const validateBeforeSave = (ref, value) => {
1561
1596
  setValidating(ref);
@@ -2127,6 +2162,20 @@ window.__ModuleLoader__.load({
2127
2162
  if (info && info.broken) {
2128
2163
  meta.push(h('button', { key: 'rt', className: 'krot-btn', onClick: () => retestBroken(key), title: t('retestBroken') }, t('retestBroken')));
2129
2164
  }
2165
+ if (info && info.detectedLimits) {
2166
+ const dl = info.detectedLimits;
2167
+ const parts = [];
2168
+ if (dl.rpmLimit) parts.push(dl.rpmLimit + ' RPM');
2169
+ if (dl.tpmLimit) parts.push(dl.tpmLimit + ' TPM');
2170
+ if (parts.length > 0) {
2171
+ meta.push(h('span', {
2172
+ key: 'dl',
2173
+ className: 'krot-tail',
2174
+ title: 'Live upstream limits:\n' + JSON.stringify(dl, null, 2),
2175
+ style: { fontSize: '11px', opacity: 0.8, color: 'var(--dsw-alias-brand-primary)' }
2176
+ }, '⚡ ' + parts.join(' · ')));
2177
+ }
2178
+ }
2130
2179
  if (info && info.modelCapabilities && info.modelCapabilities.length > 0) {
2131
2180
  meta.push(h('span', {
2132
2181
  key: 'caps',
@@ -2368,6 +2417,8 @@ window.__ModuleLoader__.load({
2368
2417
  });
2369
2418
  }, { title: t('sortUsage') }),
2370
2419
  h('select', { className: 'krot-in', value: entry.provider, onChange: (e) => setProvider(pIndex, e.target.value) }, options),
2420
+ btn(t('presetLimitsBtn') || '⚡ Preset Limits', () => applyPreset(pIndex, entry.provider), { title: t('presetLimitsBtn') || '⚡ Preset Limits', disabled: !entry.provider.trim() || !stateRef.current.writable }),
2421
+ presetNotice ? h('span', { className: 'krot-badge krot-badge-ok', style: { marginLeft: '4px' } }, presetNotice) : null,
2371
2422
  (() => {
2372
2423
  const ps = status[entry.provider];
2373
2424
  const score = ps && typeof ps.healthScore === 'number' ? ps.healthScore : null;
package/lib/index.js CHANGED
@@ -33,6 +33,7 @@ import { registerOpsRoutes } from './routes-ops.js';
33
33
  import { registerPluginUpdater } from './plugin-updater.js';
34
34
  import { notifySwitch, notifyExhaustion, notifyEarlyWarning, pushEvent } from './notify-events.js';
35
35
  import { QUOTA_WINDOW_TYPES } from './quota-window.js';
36
+ import { TrafficSmoother } from './traffic-smoother.js';
36
37
  import { getLogger } from './logger.js';
37
38
  import { createSandboxService } from './sandbox-service.js';
38
39
  import { checkBudgetAndHealthAlerts } from './budget-monitor.js';
@@ -198,6 +199,7 @@ export function apply(ctx, config = {}) {
198
199
  const webhookSender = new WebhookSender({ fetchImpl: globalThis.fetch });
199
200
  const alertDebouncer = new AlertDebouncer({ sender: webhookSender, debounceMs: 2000 });
200
201
  const moduleNotifyQueue = new NotifyQueue({ send: (url, payload) => alertDebouncer.enqueue(url, payload) });
202
+ const trafficSmoother = new TrafficSmoother({ maxQueueWaitMs: 800 });
201
203
 
202
204
  const sandboxService = createSandboxService({ getRuntime: () => buildRuntime() });
203
205
  const { ensureSandboxRunner, probeRef, lastTestCache } = sandboxService;
@@ -355,7 +357,7 @@ export function apply(ctx, config = {}) {
355
357
  switchNotifyThrottleMs: cfg.switchNotifyThrottleMs ?? 60000,
356
358
  warnBelowHealthy: cfg.warnBelowHealthy ?? 0, latencySloMs: cfg.latencySloMs ?? 0,
357
359
  index, pools,
358
- breaker: moduleBreaker, notifyQueue: moduleNotifyQueue, concurrencyTracker, quotaStore, latencyHistogram,
360
+ breaker: moduleBreaker, notifyQueue: moduleNotifyQueue, concurrencyTracker, quotaStore, latencyHistogram, trafficSmoother,
359
361
  providerTags, providerBudgets, poolByRef, providerToPool, modelPoolByProvider,
360
362
  cloneIds, expectedClones,
361
363
  circuitBreakerEnabled: cfg.circuitBreakerEnabled ?? true,
@@ -434,6 +436,7 @@ export function apply(ctx, config = {}) {
434
436
  MARKER,
435
437
  finishError,
436
438
  setRotateStartMs: (v) => { _rotateStartMs = v; },
439
+ trafficSmoother,
437
440
  quotaStore,
438
441
  circuitBreaker: moduleBreaker,
439
442
  now: nowMono,
package/lib/ops-keys.js CHANGED
@@ -22,7 +22,9 @@ import {
22
22
  DRAIN_PATH,
23
23
  DISCOVER_PATH,
24
24
  AUTO_IMPORT_PATH,
25
+ PRESET_LIMITS_PATH,
25
26
  } from './ops-paths.js';
27
+ import { lookupPreset, CANONICAL_PROVIDER_PRESETS } from './provider-limits-detector.js';
26
28
  import fs from 'node:fs';
27
29
  import os from 'node:os';
28
30
  import path from 'node:path';
@@ -539,4 +541,40 @@ ctx.effect(() => ctx.webServer.register({
539
541
  },
540
542
  }), 'dsh-key-rotation: auto-import route');
541
543
 
544
+ ctx.effect(() => ctx.webServer.register({
545
+ kind: 'exact',
546
+ path: PRESET_LIMITS_PATH,
547
+ handler: async (req, res) => {
548
+ if (!isTrustedBridgeRequest(req)) {
549
+ res.writeHead(403, { 'Content-Type': 'application/json' });
550
+ res.end(JSON.stringify({ ok: false, error: 'Forbidden' }));
551
+ return;
552
+ }
553
+ if (req.method === 'GET') {
554
+ const urlObj = new URL(req.url, 'http://127.0.0.1');
555
+ const prov = urlObj.searchParams.get('provider');
556
+ if (prov) {
557
+ const preset = lookupPreset(prov);
558
+ json(res, 200, { ok: true, provider: prov, preset });
559
+ return;
560
+ }
561
+ json(res, 200, { ok: true, presets: CANONICAL_PROVIDER_PRESETS });
562
+ return;
563
+ }
564
+ if (req.method === 'POST') {
565
+ const body = await readJson(req);
566
+ const prov = body?.provider;
567
+ const preset = lookupPreset(prov);
568
+ if (!preset) {
569
+ json(res, 404, { ok: false, error: `No canonical preset found for provider '${prov}'` });
570
+ return;
571
+ }
572
+ json(res, 200, { ok: true, provider: prov, preset });
573
+ return;
574
+ }
575
+ res.writeHead(405, { 'Content-Type': 'application/json' });
576
+ res.end(JSON.stringify({ ok: false, error: 'Method Not Allowed' }));
577
+ },
578
+ }), 'dsh-key-rotation: preset limits route');
579
+
542
580
  }
package/lib/ops-paths.js CHANGED
@@ -17,3 +17,4 @@ export const DRAIN_PATH = '/dsh-key-rotation/drain';
17
17
  export const TEST_MATRIX_PATH = '/dsh-key-rotation/test-matrix';
18
18
  export const DISCOVER_PATH = '/dsh-key-rotation/discover';
19
19
  export const AUTO_IMPORT_PATH = '/dsh-key-rotation/auto-import';
20
+ export const PRESET_LIMITS_PATH = '/dsh-key-rotation/preset-limits';
package/lib/ops-status.js CHANGED
@@ -123,6 +123,7 @@ ctx.effect(() => ctx.webServer.register({
123
123
  pauseReason: pool.state.pauseReasons?.get?.(ref) ?? null,
124
124
  revoked: isKeyRevoked(pool, ref),
125
125
  modelCapabilities: pool.state.modelCapabilities?.get?.(ref) ? [...pool.state.modelCapabilities.get(ref)] : null,
126
+ detectedLimits: pool.state.detectedLimits?.get?.(ref) ?? null,
126
127
  modelQuota,
127
128
  modelPools: activeModelPools.map(mp => mp.base),
128
129
  });
@@ -172,6 +173,7 @@ ctx.effect(() => ctx.webServer.register({
172
173
  pauseOnBudget: (providerBudgets?.get ? providerBudgets.get(pool.base) : providerBudgets?.[pool.base])?.pauseOnBudget ?? false,
173
174
  routingStrategy: pool.routingStrategy ?? runtime.routingStrategy ?? 'round-robin',
174
175
  proactiveRateLimitGuard: pool.proactiveRateLimitGuard ?? runtime.proactiveRateLimitGuard ?? true,
176
+ smoothedRequests: pool.state.smoothedRequests ?? 0,
175
177
  });
176
178
  } catch (e) {
177
179
  (ctx?.logger ? ctx.logger('dsh-key-rotation') : null)?.warn?.(`[dsh-key-rotation] status: pool ${pool.base} failed: ${String(e?.message ?? e)} ${e?.stack ?? ''}`);
@@ -37,6 +37,8 @@ export function buildPoolItem({
37
37
  labels = null,
38
38
  pinnedRef = null,
39
39
  useKeyRef = null,
40
+ trafficSmoothingBurst = null,
41
+ maxQueueWaitMs = null,
40
42
  }) {
41
43
  const seen = new Set();
42
44
  const indexed = (Array.isArray(keys) ? keys : []).map((key, index) => ({ ref: typeof key === 'string' ? key.trim() : '', index }))
@@ -135,6 +137,12 @@ const weightsMap = {};
135
137
  proactiveRateLimitGuard: poolGuard,
136
138
  labels: labelsMap,
137
139
  pinnedRef: cleanPinned,
140
+ trafficSmoothingBurst: typeof trafficSmoothingBurst === 'number' && Number.isFinite(trafficSmoothingBurst) && trafficSmoothingBurst > 0
141
+ ? Math.floor(trafficSmoothingBurst)
142
+ : (typeof rpmLimit === 'number' && rpmLimit > 0 ? Math.max(2, Math.ceil(rpmLimit / 12)) : 5),
143
+ maxQueueWaitMs: typeof maxQueueWaitMs === 'number' && Number.isFinite(maxQueueWaitMs) && maxQueueWaitMs >= 0
144
+ ? Math.floor(maxQueueWaitMs)
145
+ : 800,
138
146
  };
139
147
  }
140
148
 
@@ -175,6 +183,8 @@ export function buildPools({ cfg = {}, poolState = new Map(), defaultState = ini
175
183
  model: null,
176
184
  labels: p.labels,
177
185
  pinnedRef: p.pinnedRef ?? p.useKeyRef,
186
+ trafficSmoothingBurst: p.trafficSmoothingBurst ?? cfg.trafficSmoothingBurst,
187
+ maxQueueWaitMs: p.maxQueueWaitMs ?? cfg.maxQueueWaitMs,
178
188
  }, provRpm, provTpm, provConcurrency);
179
189
  if (pool) {
180
190
  for (const ref of pool.refs) {
@@ -203,6 +213,8 @@ export function buildPools({ cfg = {}, poolState = new Map(), defaultState = ini
203
213
  quotas: mp.quotas,
204
214
  labels: mp.labels ?? p.labels,
205
215
  pinnedRef: mp.pinnedRef ?? mp.useKeyRef ?? p.pinnedRef ?? p.useKeyRef,
216
+ trafficSmoothingBurst: mp.trafficSmoothingBurst ?? p.trafficSmoothingBurst ?? cfg.trafficSmoothingBurst,
217
+ maxQueueWaitMs: mp.maxQueueWaitMs ?? p.maxQueueWaitMs ?? cfg.maxQueueWaitMs,
206
218
  }, modelRpm, modelTpm, modelConcurrency);
207
219
  if (!mpool) continue;
208
220
  mpool.basePool = pool;
package/lib/pool-state.js CHANGED
@@ -3,7 +3,7 @@
3
3
  const MAP_FIELDS = [
4
4
  'failedUntil', 'failCounts', 'authFailCounts', 'brokenUntil', 'costPerKey',
5
5
  'lastUsedAt', 'usageCounts', 'byModel', 'usageDays', 'quotaWindows', 'costDays',
6
- 'tokenUsage', 'rpmWindows', 'tpmWindows', 'sessionSticky', 'modelCapabilities', 'pauseReasons',
6
+ 'tokenUsage', 'rpmWindows', 'tpmWindows', 'sessionSticky', 'modelCapabilities', 'pauseReasons', 'detectedLimits',
7
7
  ];
8
8
 
9
9
  const SET_FIELDS = ['pausedRefs', 'revokedRefs'];
@@ -0,0 +1,173 @@
1
+ // lib/provider-limits-detector.js — accurate dynamic provider limits and presets (#503).
2
+
3
+ export const CANONICAL_PROVIDER_PRESETS = {
4
+ deepseek: {
5
+ rpmLimit: 60,
6
+ tpmLimit: 100000,
7
+ trafficSmoothingBurst: 5,
8
+ quotaResetWindow: 'midnight_utc',
9
+ displayName: 'DeepSeek Official',
10
+ },
11
+ groq: {
12
+ rpmLimit: 30,
13
+ tpmLimit: 30000,
14
+ trafficSmoothingBurst: 5,
15
+ quotaResetWindow: 'per-minute',
16
+ displayName: 'Groq Cloud',
17
+ },
18
+ opencode: {
19
+ rpmLimit: 120,
20
+ tpmLimit: 200000,
21
+ trafficSmoothingBurst: 10,
22
+ quotaResetWindow: 'midnight_utc',
23
+ displayName: 'OpenCode Zen',
24
+ },
25
+ anthropic: {
26
+ rpmLimit: 50,
27
+ tpmLimit: 100000,
28
+ trafficSmoothingBurst: 5,
29
+ quotaResetWindow: 'per-minute',
30
+ displayName: 'Anthropic Claude',
31
+ },
32
+ openai: {
33
+ rpmLimit: 60,
34
+ tpmLimit: 150000,
35
+ trafficSmoothingBurst: 10,
36
+ quotaResetWindow: 'per-minute',
37
+ displayName: 'OpenAI API',
38
+ },
39
+ siliconflow: {
40
+ rpmLimit: 60,
41
+ tpmLimit: 100000,
42
+ trafficSmoothingBurst: 5,
43
+ quotaResetWindow: 'per-minute',
44
+ displayName: 'SiliconFlow',
45
+ },
46
+ mistral: {
47
+ rpmLimit: 60,
48
+ tpmLimit: 100000,
49
+ trafficSmoothingBurst: 5,
50
+ quotaResetWindow: 'per-minute',
51
+ displayName: 'Mistral AI',
52
+ },
53
+ gemini: {
54
+ rpmLimit: 60,
55
+ tpmLimit: 1000000,
56
+ trafficSmoothingBurst: 10,
57
+ quotaResetWindow: 'midnight_pst',
58
+ displayName: 'Google Gemini',
59
+ },
60
+ openrouter: {
61
+ rpmLimit: 200,
62
+ tpmLimit: 500000,
63
+ trafficSmoothingBurst: 15,
64
+ quotaResetWindow: 'per-minute',
65
+ displayName: 'OpenRouter',
66
+ },
67
+ };
68
+
69
+ /**
70
+ * Extract rate limit and quota information from HTTP response headers.
71
+ * @param {Headers|Record<string, string>} rawHeaders
72
+ * @returns {object|null}
73
+ */
74
+ export function extractProviderLimits(rawHeaders) {
75
+ if (!rawHeaders) return null;
76
+ const headers = {};
77
+ if (typeof rawHeaders.forEach === 'function') {
78
+ rawHeaders.forEach((val, key) => { headers[key.toLowerCase()] = String(val); });
79
+ } else if (typeof rawHeaders === 'object') {
80
+ for (const [k, v] of Object.entries(rawHeaders)) {
81
+ if (typeof k === 'string') headers[k.toLowerCase()] = String(v ?? '');
82
+ }
83
+ }
84
+
85
+ const getNum = (...names) => {
86
+ for (const name of names) {
87
+ const val = headers[name];
88
+ if (val !== undefined && val !== null && val !== '') {
89
+ const n = Number(val);
90
+ if (Number.isFinite(n) && n >= 0) return n;
91
+ }
92
+ }
93
+ return undefined;
94
+ };
95
+
96
+ const getSeconds = (...names) => {
97
+ for (const name of names) {
98
+ const val = headers[name];
99
+ if (!val) continue;
100
+ const str = String(val).trim();
101
+ // Duration string format "6m0s", "1s", "250ms"
102
+ const match = /^(\d+(?:\.\d+)?)(s|ms|m|h)?$/i.exec(str);
103
+ if (match) {
104
+ const num = parseFloat(match[1]);
105
+ const unit = (match[2] || 's').toLowerCase();
106
+ if (unit === 'ms') return Math.max(0.001, num / 1000);
107
+ if (unit === 's') return num;
108
+ if (unit === 'm') return num * 60;
109
+ if (unit === 'h') return num * 3600;
110
+ }
111
+ // Check for ISO Date string (e.g., Anthropic reset timestamp)
112
+ const parsedDate = Date.parse(str);
113
+ if (Number.isFinite(parsedDate)) {
114
+ const diffSec = (parsedDate - Date.now()) / 1000;
115
+ return Math.max(0, diffSec);
116
+ }
117
+ const n = Number(str);
118
+ if (Number.isFinite(n) && n >= 0) return n;
119
+ }
120
+ return undefined;
121
+ };
122
+
123
+ const rpmLimit = getNum('x-ratelimit-limit-requests', 'x-ratelimit-limit-rpm', 'ratelimit-limit', 'anthropic-ratelimit-requests-limit');
124
+ const tpmLimit = getNum('x-ratelimit-limit-tokens', 'x-ratelimit-limit-tpm', 'anthropic-ratelimit-tokens-limit');
125
+ const remainingRequests = getNum('x-ratelimit-remaining-requests', 'x-ratelimit-remaining', 'anthropic-ratelimit-requests-remaining');
126
+ const remainingTokens = getNum('x-ratelimit-remaining-tokens', 'anthropic-ratelimit-tokens-remaining');
127
+ const resetRequests = getSeconds('x-ratelimit-reset-requests', 'x-ratelimit-reset', 'anthropic-ratelimit-requests-reset');
128
+ const resetTokens = getSeconds('x-ratelimit-reset-tokens', 'anthropic-ratelimit-tokens-reset');
129
+ const retryAfter = getSeconds('retry-after');
130
+
131
+ if (rpmLimit === undefined && tpmLimit === undefined && remainingRequests === undefined && remainingTokens === undefined && resetRequests === undefined && retryAfter === undefined) {
132
+ return null;
133
+ }
134
+
135
+ const out = {};
136
+ if (rpmLimit !== undefined) out.rpmLimit = Math.floor(rpmLimit);
137
+ if (tpmLimit !== undefined) out.tpmLimit = Math.floor(tpmLimit);
138
+ if (remainingRequests !== undefined) {
139
+ out.remainingRequests = Math.floor(remainingRequests);
140
+ out.rpmRemaining = out.remainingRequests;
141
+ }
142
+ if (remainingTokens !== undefined) {
143
+ out.remainingTokens = Math.floor(remainingTokens);
144
+ out.tpmRemaining = out.remainingTokens;
145
+ }
146
+ if (resetRequests !== undefined) {
147
+ out.resetRequests = resetRequests;
148
+ out.rpmResetMs = Math.round(resetRequests * 1000);
149
+ }
150
+ if (resetTokens !== undefined) {
151
+ out.resetTokens = resetTokens;
152
+ out.tpmResetMs = Math.round(resetTokens * 1000);
153
+ }
154
+ if (retryAfter !== undefined) {
155
+ out.retryAfterMs = Math.round(retryAfter * 1000);
156
+ }
157
+
158
+ return out;
159
+ }
160
+
161
+ /**
162
+ * Match a configured provider name to canonical preset.
163
+ * @param {string} providerName
164
+ * @returns {object|null}
165
+ */
166
+ export function lookupPreset(providerName) {
167
+ if (!providerName || typeof providerName !== 'string') return null;
168
+ const p = providerName.toLowerCase().replace(/[^a-z0-9]/g, '');
169
+ for (const [key, val] of Object.entries(CANONICAL_PROVIDER_PRESETS)) {
170
+ if (p.includes(key)) return { ...val, id: key };
171
+ }
172
+ return null;
173
+ }
package/lib/rotate.js CHANGED
@@ -35,6 +35,7 @@ import {
35
35
  } from './model-quota.js';
36
36
 
37
37
  import { LOCAL_POOL_EXHAUSTED_CODE } from './resolver.js';
38
+ import { extractProviderLimits } from './provider-limits-detector.js';
38
39
 
39
40
  export const LOCAL_QUOTA_CODE = 'LOCAL_MODEL_QUOTA_EXHAUSTED';
40
41
 
@@ -79,6 +80,7 @@ export function createRotate(deps) {
79
80
  circuitBreaker,
80
81
  logger,
81
82
  schedulePersist,
83
+ trafficSmoother,
82
84
  now = nowMono,
83
85
  } = deps;
84
86
 
@@ -318,6 +320,25 @@ export function createRotate(deps) {
318
320
  };
319
321
 
320
322
  for (let attempt = 0; attempt < attemptList.length; attempt++) {
323
+ const candidateRef = attemptList[attempt];
324
+ // #496: Traffic smoothing micro-queue
325
+ const smoother = pool.trafficSmoother ?? runtime0.trafficSmoother ?? trafficSmoother;
326
+ if (smoother && candidateRef) {
327
+ const smoothRes = await smoother.acquire(candidateRef, {
328
+ rpmLimit: pool.rpmLimit,
329
+ burstLimit: pool.trafficSmoothingBurst,
330
+ maxQueueWaitMs: pool.maxQueueWaitMs,
331
+ signal: options.signal,
332
+ });
333
+ if (smoothRes.overflow) {
334
+ logWarn(`[dsh-key-rotation] ${options.provider}: key ${candidateRef} micro-queue burst overflow (${smoothRes.delayMs}ms) — rotating to next candidate`);
335
+ continue;
336
+ }
337
+ if (smoothRes.delayed) {
338
+ pool.state.smoothedRequests = (pool.state.smoothedRequests ?? 0) + 1;
339
+ pushEvent(pool, candidateRef, 'TRAFFIC_SMOOTHED', smoothRes.delayMs);
340
+ }
341
+ }
321
342
  // #464: Check deadline & attempt budget before dispatch
322
343
  if (!budget.canAttempt()) {
323
344
  const isTimeout = budget.isExpired();
@@ -506,6 +527,15 @@ export function createRotate(deps) {
506
527
  // Proactive rate-limit (#115, #303): if response headers say this key is near
507
528
  // its quota or has retry-after, cool it down so the NEXT request starts on a different key.
508
529
  // We do NOT re-run this (already successful) request — that would double-send.
530
+ // #503: Extract provider limits and reset headers
531
+ const rawHdrs = chunk?.metadata?.headers ?? chunk?.headers;
532
+ if (rawHdrs && activeRef) {
533
+ const detected = extractProviderLimits(rawHdrs);
534
+ if (detected) {
535
+ if (!pool.state.detectedLimits) pool.state.detectedLimits = new Map();
536
+ pool.state.detectedLimits.set(activeRef, { ...detected, at: now() });
537
+ }
538
+ }
509
539
  const guardEnabled = pool.proactiveRateLimitGuard ?? runtime0.proactiveRateLimitGuard ?? true;
510
540
  const rate = extractRateLimit(chunk?.metadata?.headers ?? chunk?.headers);
511
541
  if (guardEnabled && rate && activeRef) {
@@ -0,0 +1,96 @@
1
+ // lib/traffic-smoother.js — leaky bucket micro-queue traffic smoother (#496).
2
+ // Absorbs momentary request bursts without tripping upstream per-second/per-minute rate limits.
3
+
4
+ export class TrafficSmoother {
5
+ constructor({ maxQueueWaitMs = 800, now = Date.now, sleep = null } = {}) {
6
+ this.maxQueueWaitMs = maxQueueWaitMs;
7
+ this.now = now;
8
+ this.sleep = sleep || ((ms, signal) => new Promise((resolve, reject) => {
9
+ if (signal?.aborted) return reject(signal.reason ?? new Error('aborted'));
10
+ const timer = setTimeout(resolve, ms);
11
+ if (signal) {
12
+ signal.addEventListener('abort', () => {
13
+ clearTimeout(timer);
14
+ reject(signal.reason ?? new Error('aborted'));
15
+ }, { once: true });
16
+ }
17
+ }));
18
+ // Map<ref, { lastTime: number, tokens: number, lastUpdate: number }>
19
+ this._buckets = new Map();
20
+ }
21
+
22
+ /**
23
+ * Acquire a slot for `ref` given effective RPM or burst limits.
24
+ * @param {string} ref
25
+ * @param {object} opts { rpmLimit, burstLimit, maxQueueWaitMs, signal }
26
+ * @returns {Promise<{ allowed: boolean, delayed: boolean, delayMs: number, overflow: boolean }>}
27
+ */
28
+ async acquire(ref, { rpmLimit, burstLimit, maxQueueWaitMs, signal } = {}) {
29
+ if (!ref) {
30
+ return { allowed: true, delayed: false, delayMs: 0, overflow: false };
31
+ }
32
+ const maxWait = typeof maxQueueWaitMs === 'number' && maxQueueWaitMs >= 0 ? maxQueueWaitMs : this.maxQueueWaitMs;
33
+ const effectiveRpm = Number(rpmLimit) > 0 ? Number(rpmLimit) : 0;
34
+ if (effectiveRpm <= 0) {
35
+ return { allowed: true, delayed: false, delayMs: 0, overflow: false };
36
+ }
37
+
38
+ const capacity = Number(burstLimit) > 0
39
+ ? Math.floor(burstLimit)
40
+ : Math.max(2, Math.ceil(effectiveRpm / 12));
41
+
42
+ const ratePerSec = effectiveRpm / 60; // tokens per second
43
+ const intervalMs = Math.max(1, 1000 / ratePerSec); // ms between tokens
44
+ const curNow = this.now();
45
+
46
+ let bucket = this._buckets.get(ref);
47
+ if (!bucket) {
48
+ bucket = { tokens: capacity, lastUpdate: curNow, nextSlot: curNow };
49
+ this._buckets.set(ref, bucket);
50
+ }
51
+
52
+ // Replenish tokens based on elapsed time
53
+ const elapsed = Math.max(0, curNow - bucket.lastUpdate);
54
+ const addedTokens = elapsed * (ratePerSec / 1000);
55
+ bucket.tokens = Math.min(capacity, bucket.tokens + addedTokens);
56
+ bucket.lastUpdate = curNow;
57
+
58
+ if (bucket.tokens >= 1) {
59
+ bucket.tokens -= 1;
60
+ bucket.nextSlot = Math.max(curNow, bucket.nextSlot);
61
+ return { allowed: true, delayed: false, delayMs: 0, overflow: false };
62
+ }
63
+
64
+ // Burst is exhausted: calculate delay required for next available slot
65
+ const targetSlot = Math.max(curNow, bucket.nextSlot) + intervalMs;
66
+ const delayMs = Math.max(0, targetSlot - curNow);
67
+
68
+ if (delayMs > maxWait) {
69
+ // Exceeds max micro-queue wait: signal overflow so caller rotates to next candidate
70
+ return { allowed: false, delayed: false, delayMs, overflow: true };
71
+ }
72
+
73
+ bucket.nextSlot = targetSlot;
74
+
75
+ // Wait for the scheduled slot
76
+ await this.sleep(delayMs, signal);
77
+
78
+ return { allowed: true, delayed: true, delayMs, overflow: false };
79
+ }
80
+
81
+ getQueueDepth(ref) {
82
+ if (!ref) return this._buckets.size;
83
+ const b = this._buckets.get(ref);
84
+ if (!b) return 0;
85
+ const curNow = this.now();
86
+ const wait = Math.max(0, b.nextSlot - curNow);
87
+ return wait > 0 ? Math.ceil(wait / 100) : 0;
88
+ }
89
+
90
+ clear(ref) {
91
+ if (ref) this._buckets.delete(ref);
92
+ else this._buckets.clear();
93
+ }
94
+ }
95
+
96
+ export default TrafficSmoother;
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@goodandready/dsh-key-rotation",
3
- "version": "0.8.64",
3
+ "version": "0.8.65",
4
4
  "packageManager": "pnpm@10.33.2",
5
5
  "description": "Per-provider API key rotation for DeepSeek Harness: a key pool per provider, auto-created clone routes, and switching to the next key on quota/rate-limit errors. Includes a Settings section (Key Rotation) to edit the key pools, cooldown and switch codes.",
6
6
  "keywords": [