@goodandready/dsh-key-rotation 0.7.37 → 0.7.39

package/README.md

@@ -253,6 +253,18 @@ All management routes require loopback authentication (`127.0.0.1` / `::1`) with
 
 MIT © [GooDAnDReaDY](https://github.com/GooDAnDReaDY)
 
+### v0.7.39
+- **Self-Healing & lastUsedAt Accuracy**: Fixed key timestamp lookup in `healIdleCooldowns` to read from the modern `pool.state.lastUsedAt` map (with backwards-compatible fallback). `credentials.resolve` now properly records each key invocation timestamp in `lastUsedAt`, surfacing accurate "last used" indicators in the status dashboard and enabling background idle cooldown restoration.
+- **Hotpath & Runtime Memoization**: Eliminated redundant `buildRuntime()` calls across periodic sweeps, provider exhaustion handling, and `/status` query processing.
+- **Test Suite & CI Hardening**: Periodic interval sweep timers and debounce timers unref'ed to allow Node.js event loop natural exit without stalling CI runners. Purged obsolete incident test artifacts.
+
+### v0.7.38
+- **Hot-Path Stream Optimization**: Eliminated 4 redundant `buildRuntime()` calls inside the `rotate()` finish chunk handler by reusing the request-scoped `runtime0` snapshot.
+- **Allocation-Free Rate Limit Header Parsing**: Optimized `extractRateLimit()` with single-pass header inspection and length guards, completely removing dynamic lowercase/uppercase string allocations on every response chunk.
+- **Date ISO String Memoization**: Memoized `todayIso` string calculation once per finishing request instead of creating multiple `Date` instances for `costDays` and `usageDays`.
+- **Zero-Allocation Metrics Aggregation**: Replaced intermediate array allocation `[...values()].reduce()` with iterative summation for `totalUsage` in the `/dsh-key-rotation/status` endpoint.
+- **Stale Notification Cleanup**: Automatically purge stale entries in notification throttling maps (`budgetNotifiedAt`, `lowHealthNotifiedAt`) when provider pools are deleted.
+
 ### v0.7.37
 - **Request Context Isolation via AsyncLocalStorage**: Scoped active key resolution (`pickedRef`), start timestamps, and retry counts strictly to each async dispatch context using Node's `node:async_hooks`. Eliminates race conditions where concurrent streaming requests could penalize healthy keys.
 - **Failover on Pre-Yield Stream Exceptions**: Fixed fatal stream termination where transport errors (e.g. HTTP 429 thrown before headers, socket hang-up) aborted the generator. The catch block now inspects `isSwitchableError` and cascades seamlessly to the next key if no content tokens have been yielded.

package/README.ru.md

@@ -253,6 +253,18 @@ dsh-key-rotation:
 
 MIT © [GooDAnDReaDY](https://github.com/GooDAnDReaDY)
 
+### v0.7.39
+- **Самоисцеление ключей и фиксация lastUsedAt**: Исправлено чтение меток времени в `healIdleCooldowns`, использующее теперь карту `pool.state.lastUsedAt`. В `credentials.resolve` добавлено сохранение точного времени каждого обращения к ключу, что активирует вывод времени использования в панели мониторинга и корректное самоисцеление ключей после простоя.
+- **Оптимизация обращений к buildRuntime**: Устранены повторные вызовы `buildRuntime()` в процедурах периодической очистки, маршруте `/status` и блоке обработки исчерпания пула.
+- **Стабилизация CI и устранение утечек таймеров**: Фоновые таймеры дебаунса и очистки теперь отвязываются (`unref`) от цикла событий Node.js, предотвращая зависание раннера. Удалены тесты устаревшего модуля инцидентов.
+
+### v0.7.38
+- **Оптимизация горячего пути стриминга**: Устранены 4 избыточных вызова `buildRuntime()` при обработке завершающего чанка в `rotate()` за счёт повторного использования снимка `runtime0`.
+- **Парсинг заголовков лимитов без лишних аллокаций**: Оптимизирована функция `extractRateLimit()` — однократный проход по объекту с проверкой длины ключей без постоянных аллокаций строк `.toLowerCase()` и `.toUpperCase()`.
+- **Мемоизация строки даты ISO**: Дата текущего дня (`todayIso`) теперь вычисляется ровно один раз на запрос, исключая создание нескольких дублирующих объектов `Date` для `costDays` и `usageDays`.
+- **Итеративный подсчет totalUsage**: В эндпоинте статуса `/dsh-key-rotation/status` расчёт суммарного потребления переведён на прямой цикл без аллокации промежуточного массива `[...values()]`.
+- **Очистка устаревших записей уведомлений**: При удалении пулов провайдеров из конфигурации автоматически вычищаются связанные ключи из карт троттлинга уведомлений (`budgetNotifiedAt`, `lowHealthNotifiedAt`).
+
 ### v0.7.37
 - **Изоляция контекста запросов через AsyncLocalStorage**: Полная привязка активного ключа (`pickedRef`), времени старта и попыток к асинхронному контексту вызова через `node:async_hooks`. Устранена гонка, при которой параллельные запросы могли ошибочно штрафовать здоровый ключ соседа.
 - **Failover при исключениях в потоке до первого чанка**: Устранено аварийное прерывание потока при транспортных сбоях (например, выброс HTTP 429 до отправки заголовков). Теперь блок перехвата проверяет `isSwitchableError` и прозрачно переключает поток на запасной ключ, если клиенту ещё не было отдано полезных данных.

package/README.zh.md

@@ -214,6 +214,18 @@ dsh-key-rotation:
 
 MIT © [GooDAnDReaDY](https://github.com/GooDAnDReaDY)
 
+### v0.7.39
+- **自动恢复与 lastUsedAt 修复**：修复了 `healIdleCooldowns` 中对密钥调用时间戳的读取逻辑，直接从 `lastUsedAt` 映射读取。在 `credentials.resolve` 中补充记录每次密钥调用的时间戳，使状态面板的最近使用时间生效并正确支持空闲密钥恢复。
+- **运行期快照缓存优化**：消除了定时维护清理、`/status` 路由及密钥池耗尽处理中重复调用 `buildRuntime()` 的开销。
+- **CI 测试稳定性提升**：对维护定时器与通知防抖定时器执行 `unref`，确保 Node.js 事件循环在测试完成后干净退出，彻底解决 CI 运行器假死问题。
+
+### v0.7.38
+- **热路径流处理优化**：在 `rotate()` 结束块处理中消除 4 次多余的 `buildRuntime()` 重复调用，直接复用请求作用域内的 `runtime0` 快照。
+- **零额外字符串分配的限流头解析**：重构 `extractRateLimit()`，采用单次遍历结合键长度检查，彻底消除对每个响应头执行 `.toLowerCase()` / `.toUpperCase()` 的内存碎片分配。
+- **日期 ISO 字符串记忆化**：在统计 `costDays` 与 `usageDays` 时将 `todayIso` 计算收敛为单次，杜绝重复创建 `Date` 实例。
+- **状态统计零数组分配**：在 `/dsh-key-rotation/status` 中将 `totalUsage` 的计算由 `[...values()].reduce()` 改为直接迭代累加，避免频繁轮询引发的垃圾回收波动。
+- **孤立通知记录自动清理**：在配置移除提供商模型池时，自动清理关联的通知限频缓存。
+
 ### v0.7.37
 - **通过 AsyncLocalStorage 隔离请求上下文**：使用 Node.js 的 `node:async_hooks` 将解析后的密钥 (`pickedRef`)、启动时间和重试严格限定在单个请求上下文内，彻底消除并发请求间的竞态条件与误罚。
 - **流异常自动故障转移**：修复流在首个 token 返回前抛出传输异常（如 HTTP 429）直接终止的问题。若未发送内容块，现在会自动触发 `isSwitchableError` 并顺畅切换到备用密钥。

package/lib/heal.js

@@ -11,13 +11,13 @@ export function healIdleCooldowns(pools, idleMs, now = Date.now()) {
   for (const pool of pools) {
     if (!pool || !pool.state || !pool.base) continue;
     const fu = pool.state.failedUntil;
-    const lu = pool.state.lastUsed;
+    const lua = pool.state.lastUsedAt ?? pool.state.lastUsed;
     if (!fu || fu.size === 0) continue;
     const expiredRefs = [];
     for (const [ref, until] of fu.entries()) {
       if (!Number.isFinite(until)) continue;
       if (until > now) continue; // cooldown still active
-      const last = lu ? lu.get(ref) : undefined;
+      const last = typeof lua?.get === 'function' ? lua.get(ref) : undefined;
       if (!Number.isFinite(last)) continue; // never used → no signal, skip
       if (now - last < idleMs) continue; // used recently → don't heal
       expiredRefs.push(ref);

package/lib/index.js

@@ -38,7 +38,7 @@ import { keyTail, isLoopbackAddress, isTrustedBridgeRequest, SWITCHABLE_MESSAGE_
 
 export const name = 'dsh-key-rotation';
 export const inject = ['llm', 'webServer', 'settings', 'credentials'];
-export { keyTail, isLoopbackAddress, isTrustedBridgeRequest, DEFAULT_SWITCH_CODES, isSwitchableError, formatExhaustionMessage };
+export { keyTail, isLoopbackAddress, isTrustedBridgeRequest, DEFAULT_SWITCH_CODES, isSwitchableError, formatExhaustionMessage, getRuntime };
 
 /** Settings namespace owning the GUI-editable section (settingsNamespace-valid). */
 const NS = 'dsh-key-rotation';
@@ -452,13 +452,13 @@ export function apply(ctx, config = {}) {
       }
       const n = sweepExpired(poolState, now);
       if (n > 0) console.warn(`[dsh-key-rotation] sweep: cleared ${n} expired cooldown(s)`);
-      for (const pool of buildRuntime().poolByRef.values()) {
+      const runtime = buildRuntime();
+      for (const pool of runtime.poolByRef.values()) {
         compactUsage(pool, 30, now);
       }
       // #207 expiry pre-warning + #208 cost budget - piggybacked on this timer,
       // deduped to one notification per key/window per day (shouldNotifyDaily).
       try {
-        const runtime = buildRuntime();
         const seen = new Set();
         for (const pool of runtime.poolByRef.values()) {
           if (seen.has(pool.base)) continue;
@@ -563,6 +563,7 @@ export function apply(ctx, config = {}) {
         }
       } catch (_) { /* maintenance must never crash the sweep */ }
     }, 30000);
+    if (typeof id.unref === 'function') id.unref();
     return () => clearInterval(id);
   }, 'dsh-key-rotation: sweep expired cooldowns');
 
@@ -695,7 +696,11 @@ export function apply(ctx, config = {}) {
 
     // auto-cleanup: remove poolState for providers that are now empty or removed
     for (const key of [...poolState.keys()]) {
-      if (![...poolByRef.values()].some((p) => p.base === key)) poolState.delete(key);
+      if (![...poolByRef.values()].some((p) => p.base === key)) {
+        poolState.delete(key);
+        lowHealthNotifiedAt.delete(key);
+        budgetNotifiedAt.delete(key + ':budget');
+      }
     }
     // #192: drop RPM windows for refs that no longer belong to any pool
     for (const st of poolState.values()) {
@@ -767,6 +772,8 @@ export function apply(ctx, config = {}) {
         let hit = await original(candidate);
         if (hit && typeof hit.value === 'string' && hit.value.length > 0) {
           pool.state.lastUsed = candidate;
+          if (!pool.state.lastUsedAt) pool.state.lastUsedAt = new Map();
+          pool.state.lastUsedAt.set(candidate, now);
           const store = dispatchStorage.getStore();
           if (store && store.pool === pool) store.pickedRef = candidate;
           if (pool.state.failCounts) pool.state.failCounts.delete(candidate);
@@ -788,6 +795,8 @@ export function apply(ctx, config = {}) {
         const envVal = envValue(candidate);
         if (envVal !== undefined) {
           pool.state.lastUsed = candidate;
+          if (!pool.state.lastUsedAt) pool.state.lastUsedAt = new Map();
+          pool.state.lastUsedAt.set(candidate, now);
           const store = dispatchStorage.getStore();
           if (store && store.pool === pool) store.pickedRef = candidate;
           if (pool.state.failCounts) pool.state.failCounts.delete(candidate);
@@ -845,12 +854,11 @@ export function apply(ctx, config = {}) {
   // the resolve patch hands out the next key on each dispatch.
   function rotate(options, pool) {
     return (async function* () {
-      const { switchCodes, cooldownMs, maxCooldownMs } = buildRuntime();
+      const runtime0 = buildRuntime();
+      const { switchCodes, cooldownMs, maxCooldownMs, switchNotify, rateLimitThreshold } = runtime0;
       let lastFailure = null;
       const reqStore = { pool, pickedRef: undefined, startMs: Date.now() };
       _rotateStartMs = reqStore.startMs;
-
-      const runtime0 = buildRuntime();
       let attemptList = (pool.weightedRefs ?? pool.refs).slice();
       if (runtime0.concurrencyLimit > 0 && concurrencyTracker.isEnabled()) {
         // #193: prefer least-loaded key within limit
@@ -936,8 +944,8 @@ export function apply(ctx, config = {}) {
                 lastFailure = chunk;
                 console.warn(`[dsh-key-rotation] ${options.provider}: key ${String(activeRef ?? '?')} failed (${String(code)} ${String(message).slice(0, 100)}) - next key`);
                 // #216: per-switch webhook (opt-in switchNotify), deduped per provider
-                if (buildRuntime().switchNotify && activeRef) {
-                  notifySwitch(buildRuntime(), pool, {
+                if (switchNotify && activeRef) {
+                  notifySwitch(runtime0, pool, {
                     provider: options.provider,
                     from: activeRef,
                     code: String(code ?? 'UNKNOWN'),
@@ -948,6 +956,7 @@ export function apply(ctx, config = {}) {
                 break;
               }
               // cost tracking if provider returns usage.cost
+              const todayIso = activeRef ? new Date().toISOString().slice(0, 10) : undefined;
               if (chunk.usage?.cost != null && activeRef) {
                 const c = Number(chunk.usage.cost);
                 if (!isNaN(c)) {
@@ -955,18 +964,16 @@ export function apply(ctx, config = {}) {
                   pool.state.costPerKey.set(activeRef, (pool.state.costPerKey.get(activeRef) ?? 0) + c);
                   // #208: cost per day per key (mirrors usageDays) for budget checks
                   if (!pool.state.costDays) pool.state.costDays = new Map();
-                  const cday = new Date().toISOString().slice(0, 10);
                   const cMap = pool.state.costDays.get(activeRef) || new Map();
-                  cMap.set(cday, (cMap.get(cday) ?? 0) + c);
+                  cMap.set(todayIso, (cMap.get(todayIso) ?? 0) + c);
                   pool.state.costDays.set(activeRef, cMap);
                 }
               }
               // Usage by day (#119)
               if (activeRef) {
                 if (!pool.state.usageDays) pool.state.usageDays = new Map();
-                const day = new Date().toISOString().slice(0, 10);
                 const dayMap = pool.state.usageDays.get(activeRef) || new Map();
-                dayMap.set(day, (dayMap.get(day) ?? 0) + 1);
+                dayMap.set(todayIso, (dayMap.get(todayIso) ?? 0) + 1);
                 pool.state.usageDays.set(activeRef, dayMap);
               }
               // Per-model request detail (#121)
@@ -981,7 +988,6 @@ export function apply(ctx, config = {}) {
               // We do NOT re-run this (already successful) request — that would double-send.
               const rate = extractRateLimit(chunk?.metadata?.headers ?? chunk?.headers);
               if (rate && activeRef) {
-                const { rateLimitThreshold } = buildRuntime();
                 if (isRateLimited(rate, rateLimitThreshold ?? 0.1)) {
                   const cool = rate.reset && rate.reset > Date.now() ? (rate.reset - Date.now()) : pool.cooldownMs;
                   recordFailure(pool, activeRef, Date.now(), cool, pool.maxCooldownMs);
@@ -1010,8 +1016,8 @@ export function apply(ctx, config = {}) {
             pool.state.lastSwitchAt = Date.now();
             lastFailure = finishError(e?.code ?? 'TRANSPORT', String(e?.message ?? e));
             console.warn(`[dsh-key-rotation] ${options.provider}: key ${String(activeRef ?? '?')} stream threw ${String(e?.code ?? e?.message ?? e)} - failover to next key`);
-            if (buildRuntime().switchNotify && activeRef) {
-              notifySwitch(buildRuntime(), pool, {
+            if (switchNotify && activeRef) {
+              notifySwitch(runtime0, pool, {
                 provider: options.provider,
                 from: activeRef,
                 code: String(e?.code ?? 'TRANSPORT'),
@@ -1033,11 +1039,11 @@ export function apply(ctx, config = {}) {
       pool.state.lastExhaustionAt = Date.now();
       pool.state.exhaustionCount = (pool.state.exhaustionCount ?? 0) + 1;
       console.warn(`[dsh-key-rotation] ${options.provider}: pool exhausted — all ${pool.refs.length} keys cooling`);
+      const runtime = buildRuntime();
       // notify via extracted helper (see notifyExhaustion above)
-      notifyExhaustion(buildRuntime(), pool, { provider: options.provider });
+      notifyExhaustion(runtime, pool, { provider: options.provider });
 
       // #194: cross-provider cascade failover (guarded against infinite recursion)
-      const runtime = buildRuntime();
       if (!options.__isCascade && Array.isArray(runtime.cascade) && runtime.cascade.length > 0) {
         const pools = runtime.providerToPool;
         const fb = pickCascadeFallback(options.provider, runtime, pools);
@@ -1078,10 +1084,10 @@ export function apply(ctx, config = {}) {
         json(res, 403, { error: { code: 'forbidden', message: 'dsh-key-rotation: status is local-only' } });
         return;
       }
-      const { poolByRef, providerTags, providerBudgets } = buildRuntime();
+      const runtime = buildRuntime();
+      const { poolByRef, providerTags, providerBudgets, latencySloMs } = runtime;
       const base = ctx.get('credentials');
       const now = Date.now();
-      const latencySloMs = buildRuntime().latencySloMs;
       const seen = new Set();
       const providers = [];
       for (const pool of poolByRef.values()) {
@@ -1148,7 +1154,7 @@ export function apply(ctx, config = {}) {
           lastSwitchAt: pool.state.lastSwitchAt ?? null,
           lastExhaustionAt: pool.state.lastExhaustionAt ?? null,
           exhaustionCount: pool.state.exhaustionCount ?? 0,
-          totalUsage: [...(pool.state.usageCounts?.values() ?? [])].reduce((a, b) => a + b, 0),
+          totalUsage: (() => { let s = 0; if (pool.state.usageCounts) for (const v of pool.state.usageCounts.values()) s += v; return s; })(),
           // #225: aggregate p95 across the pool's keys
           p95: (() => {
             const vals = (pool.refs ?? []).map((r) => latencyHistogram.snapshot(r)).filter((s) => s && s.p95 != null).map((s) => s.p95);

package/lib/pool.js

@@ -248,14 +248,23 @@ export function computeHealthScore(state) {
 /** Extract rate-limit info from an object that may carry response headers. */
 export function extractRateLimit(headers) {
   if (!headers || typeof headers !== 'object') return null;
-  const get = (name) => {
-    const v = headers[name] ?? headers[name.toLowerCase()] ?? headers[name.toUpperCase()];
-    if (v === undefined || v === null) return undefined;
-    return Number(String(v));
-  };
-  const remaining = get('X-RateLimit-Remaining');
-  const limit = get('X-RateLimit-Limit');
-  const reset = get('X-RateLimit-Reset');
+  let remaining, limit, reset;
+  for (const k in headers) {
+    const lk = k.length;
+    if (lk === 21 || lk === 16 || lk === 17) {
+      const lower = k.toLowerCase();
+      if (lower === 'x-ratelimit-remaining') {
+        const v = headers[k];
+        if (v != null) remaining = Number(String(v));
+      } else if (lower === 'x-ratelimit-limit') {
+        const v = headers[k];
+        if (v != null) limit = Number(String(v));
+      } else if (lower === 'x-ratelimit-reset') {
+        const v = headers[k];
+        if (v != null) reset = Number(String(v));
+      }
+    }
+  }
   if (remaining === undefined && limit === undefined) return null;
   return { remaining, limit, reset };
 }

package/lib/webhook.js

@@ -157,6 +157,7 @@ export class AlertDebouncer {
     }
     if (!entry.timer) {
       entry.timer = setTimeout(() => this.flush(url), this._debounceMs);
+      if (typeof entry.timer.unref === 'function') entry.timer.unref();
     }
   }
 

package/package.json

@@ -1,7 +1,6 @@
 {
   "name": "@goodandready/dsh-key-rotation",
-  "version": "0.7.37",
-  "packageManager": "pnpm@10.33.2",
+  "version": "0.7.39",
   "description": "Per-provider API key rotation for DeepSeek Harness: a key pool per provider, auto-created clone routes, and switching to the next key on quota/rate-limit errors. Includes a Settings section (Key Rotation) to edit the key pools, cooldown and switch codes.",
   "keywords": [
     "deepseek-harness",
@@ -54,6 +53,6 @@
     "@deepseek-ai/dsh-llm": "^0.1.0-rc.6"
   },
   "scripts": {
-    "test": "node --test test/*.test.js test/*.test.mjs"
+    "test": "node --test --test-timeout=10000 test/*.test.js test/*.test.mjs"
   }
-}
+}