@goodandready/dsh-key-rotation 0.7.40 → 0.8.1

package/README.md

@@ -28,6 +28,17 @@
 
 ## ⚡ Overview & The Problem
 
+### 🛠️ What's New in v0.8.0 (Stability)
+- **🔌 Circuit breaker**: after N consecutive provider failures the circuit opens and requests fail fast (`CIRCUIT_OPEN`) until a cool-down; half-open probes recover automatically.
+- **🕒 Monotonic clock**: cooldown/breaker durations use process monotonic time so NTP steps cannot invert remaining times.
+- **📮 Non-blocking webhooks**: alerts go through a bounded queue with backoff — stream rotation never waits on webhook HTTP.
+- **🧱 Atomic I/O helpers**: crash-safe writes; corrupt JSON never overwrites previous in-memory state.
+- **🧹 Clone-route GC**: orphaned auto-created clone routes are dropped from the runtime set.
+- **🧭 Error taxonomy**: explicit switch/surface/soft classification for 408/425/429/5xx, sockets and gRPC codes.
+- **📡 Status extras**: per-provider `circuit` plus `meta.expectedClones` / `meta.notifyQueue`.
+- **🧪 Smoke harness**: scripted 429 → next-key → success path in `test/smoke-rotation-080.test.mjs`.
+
+
 ### 🛠️ What's New in v0.7.33 (Stability & Bugfix Release)
 - **🔍 Resolved Key Probing BaseURL**: Fixed `resolveBaseUrl` to map key credential refs to owning provider pools, restoring live `probeModels` testing.
 - **🛡️ Guarded Cascade Recursion**: Prevented call stack overflow in cross-provider failover when circular cascade chains occur.
@@ -192,6 +203,11 @@ dsh-key-rotation:
     - UNKNOWN_MODEL
     - AUTH
   cooldownMs: 60000
+  # v0.8.0 circuit breaker
+  circuitBreakerEnabled: true
+  circuitBreakerThreshold: 5
+  circuitBreakerOpenMs: 30000
+  circuitBreakerHalfOpenProbes: 1
   concurrencyLimit: 5
   quotaResetWindow:
     type: midnight_utc
@@ -220,6 +236,11 @@ dsh-key-rotation:
 |---|---|---|---|
 | `switchCodes` | `string[]` | `[QUOTA, RATE_LIMIT, ...]` | List of error codes that immediately trigger failover. |
 | `cooldownMs` | `number` | `60000` (1 min) | Base penalty duration (in ms) for quarantined keys. |
+| `circuitBreakerEnabled` | `boolean` | `true` | Enable per-provider circuit breaker (v0.8.0). |
+| `circuitBreakerThreshold` | `number` | `5` | Consecutive failures before opening the circuit. |
+| `circuitBreakerOpenMs` | `number` | `30000` | How long the circuit stays open (ms). |
+| `circuitBreakerHalfOpenProbes` | `number` | `1` | Probe requests allowed in half-open state. |
+| `verboseLogging` | `boolean` | `false` | Per-request rotation logs (noisy; off by default). |
 | `concurrencyLimit` | `number` | `0` (disabled) | Max concurrent in-flight streams per key (0 = unlimited). |
 | `quotaResetWindow` | `object` | `null` | Calendar reset alignment (`midnight_utc`, `midnight_pst`, `rolling_24h`). |
 | `cascade` | `array` | `[]` | Fallback provider chain when primary pool is completely exhausted. |
@@ -234,7 +255,7 @@ All management routes require loopback authentication (`127.0.0.1` / `::1`) with
 
 | Route | Method | Description |
 |---|---|---|
-| `/dsh-key-rotation/status` | `GET` | Returns real-time health snapshots, active keys, and cooldown states. |
+| `/dsh-key-rotation/status` | `GET` | Real-time health, keys, cooldowns. Since v0.8.0 also `providers[].circuit` and `meta` (`expectedClones`, `notifyQueue`). |
 | `/dsh-key-rotation/config` | `GET` / `PUT` | Read and update active key rotation settings and provider pools. |
 | `/dsh-key-rotation/key` | `PUT` / `DELETE` | Add, update, or remove credentials in host storage and pool. |
 | `/dsh-key-rotation/reset` | `POST` | Instantly resets all cooldowns and restores all keys to `ready`. |

package/README.ru.md

@@ -28,6 +28,17 @@
 
 ## ⚡ Обзор и решаемая проблема
 
+### 🛠️ Что нового в версии 0.8.0 (Стабильность)
+- **🔌 Автомат выключения провайдера**: после N подряд ошибок контур открывается, запросы быстро завершаются (`CIRCUIT_OPEN`); half-open пробы восстанавливают сервис.
+- **🕒 Монотонные часы**: cooldown/breaker считаются по процессным часам, NTP не ломает оставшееся время.
+- **📮 Неблокирующие webhook**: очередь с backoff — `rotate()` не ждёт HTTP.
+- **🧱 Атомарная запись файлов**: повреждённый JSON не затирает прежнее состояние.
+- **🧹 GC clone-маршрутов**: осиротевшие авто-роуты убираются из runtime-набора.
+- **🧭 Таксономия ошибок**: явная классификация switch/surface/soft (408/425/429/5xx, сокеты, gRPC).
+- **📡 Status**: `circuit` у провайдера + `meta.expectedClones` / `meta.notifyQueue`.
+- **🧪 Smoke**: сценарий 429 → следующий ключ → успех.
+
+
 ### 🛠️ Что нового в версии 0.7.33 (Хотфикс и повышение стабильности)
 - **🔍 Исправление проверки ключей**: Функция `resolveBaseUrl` теперь сопоставляет ref ключа с его пулом, восстанавливая работу живого тестирования моделей.
 - **🛡️ Защита от зацикливания каскада**: Устранена возможность переполнения стека при взаимных кольцевых цепочках failover.
@@ -192,6 +203,10 @@ dsh-key-rotation:
     - UNKNOWN_MODEL
     - AUTH
   cooldownMs: 60000
+  circuitBreakerEnabled: true
+  circuitBreakerThreshold: 5
+  circuitBreakerOpenMs: 30000
+  circuitBreakerHalfOpenProbes: 1
   concurrencyLimit: 5
   quotaResetWindow:
     type: midnight_utc
@@ -282,3 +297,16 @@ MIT © [GooDAnDReaDY](https://github.com/GooDAnDReaDY)
 - **Архитектура настроек**: Добавлена нативная интеграция со снимками `settingsScope` в карточке настроек с безопасным фоллбеком на HTTP-мост (#235).
 - **Локализация**: Фоллбек секции настроек `settings.section` переведён на локализованную метку `t('title')` со слотом `locale: NS`, а китайский словарь `zh` зарегистрирован в `ctx.locale` наряду с `en` и `ru` (#236).
 - **Удаление мёртвого кода**: Удалена неиспользуемая функция `mountDashboard` после перехода на header chip (#240).
+
+### Параметры circuit breaker (Changed in v0.8.0)
+
+| Параметр | Тип | По умолчанию | Описание |
+|---|---|---|---|
+| `circuitBreakerEnabled` | boolean | `true` | Включить/выключить breaker |
+| `circuitBreakerThreshold` | number | `5` | Число подряд ошибок до открытия |
+| `circuitBreakerOpenMs` | number | `30000` | Длительность open, мс |
+| `circuitBreakerHalfOpenProbes` | number | `1` | Проб в half-open |
+| `verboseLogging` | boolean | `false` | Подробные логи rotation (шумно) |
+
+Status API (v0.8.0): у каждого провайдера `circuit`, в корне ответа `meta.expectedClones` и `meta.notifyQueue`.
+

package/README.zh.md

@@ -28,6 +28,17 @@
 
 ## ⚡ 概述与核心痛点
 
+### 🛠️ v0.8.0 版本新特性（稳定性）
+- **🔌 熔断器**：连续失败后快速失败（`CIRCUIT_OPEN`），半开探测自动恢复。
+- **🕒 单调时钟**：冷却/熔断使用进程单调时间，NTP 校时不会颠倒剩余时间。
+- **📮 非阻塞 Webhook**：有界队列 + backoff，轮换不等待 HTTP。
+- **🧱 原子写文件**：损坏 JSON 不会覆盖既有状态。
+- **🧹 克隆路由 GC**：清理孤儿自动路由。
+- **🧭 错误分类**：408/425/429/5xx、套接字与 gRPC 的 switch/surface/soft。
+- **📡 Status**：提供商 `circuit` + `meta`。
+- **🧪 Smoke**：429 → 切换密钥 → 成功。
+
+
 ### 🛠️ v0.7.33 版本新特性 (稳定性与问题修复)
 - **🔍 修复密钥探测 BaseURL 解析**：`resolveBaseUrl` 现已支持从密钥 ref 反查归属提供商池，恢复在线模型连通性探测。
 - **🛡️ 防御级联无限递归**：在跨提供商故障转移中增加递归深度防护，彻底杜绝循环级联导致的堆栈溢出。
@@ -184,6 +195,10 @@ dsh-key-rotation:
     - UNKNOWN_MODEL
     - AUTH
   cooldownMs: 60000
+  circuitBreakerEnabled: true
+  circuitBreakerThreshold: 5
+  circuitBreakerOpenMs: 30000
+  circuitBreakerHalfOpenProbes: 1
   concurrencyLimit: 5
   quotaResetWindow:
     type: midnight_utc
@@ -246,3 +261,13 @@ MIT © [GooDAnDReaDY](https://github.com/GooDAnDReaDY)
 - **设置架构与状态**: 在设置卡片中增加原生 `settingsScope` 绑定支持，保留 HTTP 桥接安全回退机制 (#235)。
 - **本地化与文案**: 侧边栏备用项 `settings.section` 标签支持本地化 `t('title')` 并配置 `locale: NS`，并在 `ctx.locale` 中注册 `zh` 中文字典 (#236)。
 - **死代码清理**: 移除 header-chip 迁移后残留的废弃 `mountDashboard` 函数 (#240)。
+
+
+### 熔断器参数（v0.8.0）
+
+| 参数 | 类型 | 默认 | 说明 |
+|---|---|---|---|
+| `circuitBreakerEnabled` | boolean | `true` | 启用熔断 |
+| `circuitBreakerThreshold` | number | `5` | 连续失败阈值 |
+| `circuitBreakerOpenMs` | number | `30000` | 打开时长 ms |
+| `circuitBreakerHalfOpenProbes` | number | `1` | 半开探测次数 |

package/lib/atomic-io.js

@@ -0,0 +1,60 @@
+// lib/atomic-io.js — crash-safe file writes and safe JSON loads (#264).
+// Never overwrite a non-empty destination with empty/corrupt parse results.
+
+import { promises as fs } from 'node:fs';
+import path from 'node:path';
+import os from 'node:os';
+import crypto from 'node:crypto';
+
+/**
+ * Atomically write data to filePath (temp + rename in same directory).
+ * @param {string} filePath
+ * @param {string|Buffer} data
+ */
+export async function atomicWriteFile(filePath, data) {
+  const dir = path.dirname(filePath);
+  await fs.mkdir(dir, { recursive: true });
+  const tmp = path.join(dir, `.${path.basename(filePath)}.${process.pid}.${crypto.randomBytes(4).toString('hex')}.tmp`);
+  const fh = await fs.open(tmp, 'w');
+  try {
+    await fh.writeFile(data);
+    await fh.sync();
+  } finally {
+    await fh.close();
+  }
+  await fs.rename(tmp, filePath);
+}
+
+/**
+ * Parse JSON safely. On failure returns fallback — never null-by-default wipe.
+ * @param {string|Buffer|null|undefined} text
+ * @param {any} fallback
+ */
+export function safeParseJson(text, fallback = null) {
+  if (text == null || text === '') return fallback;
+  try {
+    const v = JSON.parse(typeof text === 'string' ? text : String(text));
+    return v == null ? fallback : v;
+  } catch {
+    return fallback;
+  }
+}
+
+/**
+ * Read JSON file; corrupt/missing → fallback (previous in-memory value).
+ * @param {string} filePath
+ * @param {any} fallback
+ */
+export async function safeReadJson(filePath, fallback = null) {
+  try {
+    const raw = await fs.readFile(filePath, 'utf8');
+    return safeParseJson(raw, fallback);
+  } catch {
+    return fallback;
+  }
+}
+
+/** Write JSON atomically. */
+export async function atomicWriteJson(filePath, value) {
+  await atomicWriteFile(filePath, JSON.stringify(value, null, 2));
+}

package/lib/bounded-map.js

@@ -0,0 +1,65 @@
+// lib/bounded-map.js — Map with max size + optional TTL (#262).
+
+export class BoundedMap {
+  /** @param {{ max?: number, ttlMs?: number|null }} opts */
+  constructor({ max = 1000, ttlMs = null } = {}) {
+    if (!Number.isFinite(max) || max < 1) throw new Error('BoundedMap: max must be >= 1');
+    this.max = max;
+    this.ttlMs = Number.isFinite(ttlMs) && ttlMs > 0 ? ttlMs : null;
+    this._m = new Map(); // insertion order = LRU order when we re-insert on get/set
+  }
+
+  get size() { return this._m.size; }
+
+  _expired(entry, now) {
+    if (!this.ttlMs || !entry) return false;
+    return now - entry.at >= this.ttlMs;
+  }
+
+  get(key, now = Date.now()) {
+    const e = this._m.get(key);
+    if (!e) return undefined;
+    if (this._expired(e, now)) {
+      this._m.delete(key);
+      return undefined;
+    }
+    // refresh LRU
+    this._m.delete(key);
+    this._m.set(key, e);
+    return e.v;
+  }
+
+  set(key, value, now = Date.now()) {
+    if (this._m.has(key)) this._m.delete(key);
+    this._m.set(key, { v: value, at: now });
+    while (this._m.size > this.max) {
+      const oldest = this._m.keys().next().value;
+      this._m.delete(oldest);
+    }
+    return this;
+  }
+
+  has(key, now = Date.now()) {
+    return this.get(key, now) !== undefined;
+  }
+
+  delete(key) { return this._m.delete(key); }
+  clear() { this._m.clear(); }
+
+  *entries(now = Date.now()) {
+    for (const [k, e] of [...this._m.entries()]) {
+      if (this._expired(e, now)) { this._m.delete(k); continue; }
+      yield [k, e.v];
+    }
+  }
+
+  *keys(now = Date.now()) {
+    for (const [k] of this.entries(now)) yield k;
+  }
+
+  toObject(now = Date.now()) {
+    const out = {};
+    for (const [k, v] of this.entries(now)) out[k] = v;
+    return out;
+  }
+}

package/lib/circuit-breaker.js

@@ -0,0 +1,93 @@
+// lib/circuit-breaker.js — per-provider circuit breaker (#260).
+// closed --(threshold consecutive failures)--> open --(openMs)--> half-open
+// half-open: allow limited probes; success => closed, failure => open again.
+
+export const BREAKER_CLOSED = 'closed';
+export const BREAKER_OPEN = 'open';
+export const BREAKER_HALF_OPEN = 'half_open';
+
+export class CircuitBreaker {
+  /**
+   * @param {{ threshold?: number, openMs?: number, halfOpenProbes?: number, now?: () => number }} opts
+   */
+  constructor({ threshold = 5, openMs = 30_000, halfOpenProbes = 1, now = Date.now } = {}) {
+    this.threshold = Math.max(1, threshold | 0);
+    this.openMs = Math.max(100, openMs | 0);
+    this.halfOpenProbes = Math.max(1, halfOpenProbes | 0);
+    this._now = now;
+    /** @type {Map<string, {state:string, fails:number, openedAt:number, probes:number}>} */
+    this._st = new Map();
+  }
+
+  _entry(provider) {
+    let e = this._st.get(provider);
+    if (!e) {
+      e = { state: BREAKER_CLOSED, fails: 0, openedAt: 0, probes: 0 };
+      this._st.set(provider, e);
+    }
+    return e;
+  }
+
+  /** @returns {boolean} whether a request may be dispatched to this provider */
+  canRequest(provider) {
+    const e = this._entry(provider);
+    const now = this._now();
+    if (e.state === BREAKER_OPEN) {
+      if (now - e.openedAt >= this.openMs) {
+        e.state = BREAKER_HALF_OPEN;
+        e.probes = 0;
+        return true;
+      }
+      return false;
+    }
+    if (e.state === BREAKER_HALF_OPEN) {
+      if (e.probes < this.halfOpenProbes) {
+        e.probes += 1;
+        return true;
+      }
+      return false;
+    }
+    return true;
+  }
+
+  onSuccess(provider) {
+    const e = this._entry(provider);
+    e.fails = 0;
+    e.probes = 0;
+    e.state = BREAKER_CLOSED;
+  }
+
+  onFailure(provider) {
+    const e = this._entry(provider);
+    const now = this._now();
+    if (e.state === BREAKER_HALF_OPEN) {
+      e.state = BREAKER_OPEN;
+      e.openedAt = now;
+      e.fails = this.threshold;
+      return e.state;
+    }
+    e.fails += 1;
+    if (e.fails >= this.threshold) {
+      e.state = BREAKER_OPEN;
+      e.openedAt = now;
+    }
+    return e.state;
+  }
+
+  state(provider) {
+    return this._entry(provider).state;
+  }
+
+  snapshot() {
+    const out = {};
+    for (const [k, e] of this._st) {
+      out[k] = { state: e.state, fails: e.fails, openedAt: e.openedAt || null };
+    }
+    return out;
+  }
+
+  reset(provider) {
+    if (provider) this._st.delete(provider);
+    else this._st.clear();
+  }
+}

package/lib/client.js

@@ -240,6 +240,33 @@ window.__ModuleLoader__.load({
     // отдельными отмеченными галочками, чтобы правило нельзя было потерять.
     const KNOWN_CODES = ['QUOTA', 'RATE_LIMIT', 'SERVER', 'TIMEOUT', 'TRANSPORT', 'EMPTY_RESPONSE', 'UNKNOWN_MODEL', 'AUTH'];
 
+    // #273: keep the settings list item mounted if the section throws.
+    class KeyRotationErrorBoundary extends React.Component {
+      constructor(props) {
+        super(props);
+        this.state = { error: null };
+      }
+      static getDerivedStateFromError(error) {
+        return { error };
+      }
+      componentDidCatch(error, info) {
+        try { console.error('[dsh-key-rotation] settings card crashed', error, info); } catch (_) {}
+      }
+      render() {
+        if (this.state.error) {
+          return React.createElement('div', { className: 'krot krot-err', style: { padding: 12 } },
+            React.createElement('p', null, 'Key Rotation settings failed to render.'),
+            React.createElement('p', { style: { fontSize: 11, opacity: 0.8 } }, String(this.state.error?.message || this.state.error)),
+            React.createElement('button', {
+              type: 'button',
+              className: 'krot-btn',
+              onClick: () => this.setState({ error: null }),
+            }, 'Retry'));
+        }
+        return this.props.children;
+      }
+    }
+
     /** Опрос статуса ротации, пока раздел настроек открыт (smart polling). */
     function useRotationStatus() {
       const [byProvider, setByProvider] = React.useState({});
@@ -414,9 +441,31 @@ window.__ModuleLoader__.load({
         }
         return null;
       }, [props.ctx]);
+      // #273: getSnapshot must be referentially stable or React 18 unmounts the tree
+      const scopeCacheRef = React.useRef({ has: false, value: null });
+      const getScopeSnapshot = React.useCallback(() => {
+        if (!settingsScope || typeof settingsScope.getSnapshot !== 'function') return null;
+        let next = null;
+        try {
+          next = settingsScope.getSnapshot();
+        } catch (_) {
+          return null;
+        }
+        const prev = scopeCacheRef.current;
+        if (prev.has && Object.is(prev.value, next)) return prev.value;
+        // shallow-compare common snapshot shape to avoid identity thrash
+        if (prev.has && prev.value && next
+          && prev.value.status === next.status
+          && prev.value.revision === next.revision
+          && prev.value.value === next.value) {
+          return prev.value;
+        }
+        scopeCacheRef.current = { has: true, value: next };
+        return next;
+      }, [settingsScope]);
       const scopeSnapshot = React.useSyncExternalStore(
-        React.useMemo(() => (cb) => (settingsScope ? settingsScope.subscribe(cb) : () => {}), [settingsScope]),
-        React.useCallback(() => (settingsScope ? settingsScope.getSnapshot() : null), [settingsScope])
+        React.useMemo(() => (cb) => (settingsScope && typeof settingsScope.subscribe === 'function' ? settingsScope.subscribe(cb) : () => {}), [settingsScope]),
+        getScopeSnapshot
       );
       const [state, setState] = React.useState({ status: 'loading', value: null, revision: 0, error: '', providers: [] });
       const [draft, setDraft] = React.useState(null);
@@ -1185,7 +1234,7 @@ window.__ModuleLoader__.load({
             h('span', { className: 'krot-card-chevron' + (open ? ' krot-card-chevron-open' : ''), 'aria-hidden': 'true' },
               h('svg', { width: 14, height: 14, viewBox: '0 0 14 14', fill: 'none', stroke: 'currentColor', strokeWidth: 1.5, style: { display: 'block' } },
                 h('path', { d: 'M3.5 5.25L7 8.75L10.5 5.25' })))),
-          open ? h('div', { className: 'krot-card-body' }, h(KeyRotationSection, { ...props, locale })) : null);
+          open ? h('div', { className: 'krot-card-body' }, h(KeyRotationErrorBoundary, null, h(KeyRotationSection, { ...props, locale }))) : null);
       }
       const tryPluginItem = () => {
         try {
@@ -1209,7 +1258,7 @@ window.__ModuleLoader__.load({
             label: () => (t ? t('title') : 'Key Rotation') || 'Key Rotation',
             inject: () => ({ ctx }),
           },
-          (props) => h(KeyRotationSection, { ...props, locale: useLocale() }),
+          (props) => h(KeyRotationErrorBoundary, null, h(KeyRotationSection, { ...props, locale: useLocale() })),
         ));
       }
     }

package/lib/clock.js

@@ -0,0 +1,24 @@
+// lib/clock.js — monotonic process clock for durations (#261).
+// performance.timeOrigin + performance.now() tracks process time and does not
+// jump when NTP adjusts the wall clock. Wall clock remains for display/cron.
+
+const hasPerformance = typeof performance !== 'undefined'
+  && typeof performance.now === 'function'
+  && Number.isFinite(performance.timeOrigin);
+
+/** Wall-clock epoch ms (Date.now). Use only for display / calendar buckets. */
+export function nowWall() {
+  return Date.now();
+}
+
+/**
+ * Monotonic-in-process epoch ms. Stable under NTP steps for the process lifetime.
+ * Falls back to Date.now() if performance is unavailable.
+ */
+export function nowMono() {
+  if (!hasPerformance) return Date.now();
+  return performance.timeOrigin + performance.now();
+}
+
+/** Default clock injectable into pure helpers. */
+export const defaultClock = { nowWall, nowMono };

package/lib/error-taxonomy.js

@@ -0,0 +1,66 @@
+// lib/error-taxonomy.js — complete failure classification (#267).
+// action: 'switch' | 'surface' | 'cooldown'
+
+const SWITCH_STATUS = new Set([408, 425, 429, 500, 502, 503, 504]);
+const SURFACE_STATUS = new Set([400, 404, 422]);
+const AUTH_STATUS = new Set([401, 403]);
+
+const SWITCH_CODES = new Set([
+  'QUOTA', 'RATE_LIMIT', 'SERVER', 'TIMEOUT', 'TRANSPORT', 'EMPTY_RESPONSE',
+  'UNKNOWN_MODEL', 'AUTH', 'RESOURCE_EXHAUSTED', 'UNAVAILABLE', 'INTERNAL',
+  'DEADLINE_EXCEEDED', 'UNAUTHENTICATED', 'PERMISSION_DENIED', 'ABORTED',
+]);
+const SOCKET_SWITCH = new Set([
+  'ECONNRESET', 'ECONNREFUSED', 'ETIMEDOUT', 'EPIPE', 'EAI_AGAIN', 'ENOTFOUND',
+  'ECONNABORTED', 'UND_ERR_CONNECT_TIMEOUT', 'UND_ERR_SOCKET', 'UND_ERR_HEADERS_TIMEOUT',
+]);
+
+/**
+ * Classify a failure for rotation.
+ * @param {any} failureOrPayload
+ * @returns {{ action: 'switch'|'surface'|'cooldown', code: string, soft: boolean, reason: string }}
+ */
+export function classifyFailure(failureOrPayload) {
+  if (!failureOrPayload) {
+    return { action: 'surface', code: 'UNKNOWN', soft: false, reason: 'empty' };
+  }
+  const failure = failureOrPayload.failure ?? failureOrPayload;
+  const status = Number(failure.status ?? failure.statusCode ?? failure.httpStatus ?? 0);
+  const code = String(failure.code ?? failure.reason ?? failure.name ?? '').toUpperCase();
+  const message = String(failure.message ?? failureOrPayload.message ?? '');
+
+  if (AUTH_STATUS.has(status) || code === 'AUTH' || code === 'UNAUTHENTICATED' || code === 'PERMISSION_DENIED') {
+    return { action: 'switch', code: status ? String(status) : (code || 'AUTH'), soft: false, reason: 'auth' };
+  }
+  if (status === 429 || code === 'RATE_LIMIT' || code === 'RESOURCE_EXHAUSTED' || code === 'QUOTA') {
+    return { action: 'switch', code: status ? '429' : (code || 'RATE_LIMIT'), soft: false, reason: 'quota' };
+  }
+  if (SWITCH_STATUS.has(status)) {
+    const soft = status === 500 || status === 502 || status === 503 || status === 504;
+    return { action: 'switch', code: String(status), soft, reason: 'http' };
+  }
+  if (SURFACE_STATUS.has(status)) {
+    return { action: 'surface', code: String(status), soft: false, reason: 'client' };
+  }
+  if (code === 'TIMEOUT' || code === 'DEADLINE_EXCEEDED' || /timeout/i.test(message)) {
+    return { action: 'switch', code: 'TIMEOUT', soft: true, reason: 'timeout' };
+  }
+  if (SOCKET_SWITCH.has(code) || /ECONNRESET|ECONNREFUSED|ETIMEDOUT|socket hang up|premature close/i.test(message)) {
+    return { action: 'switch', code: 'TRANSPORT', soft: true, reason: 'socket' };
+  }
+  if (code === 'UNAVAILABLE' || code === 'INTERNAL' || code === 'SERVER') {
+    return { action: 'switch', code: code || 'SERVER', soft: true, reason: 'grpc' };
+  }
+  if (code === 'TRANSPORT' || code === 'EMPTY_RESPONSE' || code === 'UNKNOWN_MODEL') {
+    return { action: 'switch', code, soft: code !== 'UNKNOWN_MODEL', reason: 'named' };
+  }
+  if (SWITCH_CODES.has(code)) {
+    return { action: 'switch', code, soft: false, reason: 'named' };
+  }
+  return { action: 'surface', code: code || 'UNKNOWN', soft: false, reason: 'unclassified' };
+}
+
+/** True when classification says switch to next key. */
+export function shouldSwitch(failureOrPayload) {
+  return classifyFailure(failureOrPayload).action === 'switch';
+}

package/lib/index.js

@@ -42,6 +42,12 @@ const dispatchStorage = new AsyncLocalStorage();
 import { findSecrets, looksLikeApiSecret } from './keycheck.js';
 import { json, readJson, descriptorOf, viewOf, writeSection, providerCatalog, handleConfigBridge, NS as BRIDGE_NS } from './http-bridge.js';
 import { createRotate } from './rotate.js';
+import { nowMono } from './clock.js';
+import { CircuitBreaker } from './circuit-breaker.js';
+import { NotifyQueue } from './notify-queue.js';
+import { BoundedMap } from './bounded-map.js';
+import { classifyFailure } from './error-taxonomy.js';
+import { safeParseJson } from './atomic-io.js';
 import { registerOpsRoutes } from './routes-ops.js';
 
 /** The llm-pi-ai namespace whose provider profiles map providers to pools. */
@@ -67,7 +73,11 @@ export function notifySwitch(runtime, pool, info, hooks = { webhookSender, now:
   const now = hooks.now();
   if (now - last < throttle) return;
   switchNotifiedAt.set(info.provider, now);
-  hooks.webhookSender.send(runtime.notifyWebhook, {
+  // #263: non-blocking — enqueue never awaits webhook I/O
+  const send = hooks.notifyQueue
+    ? (url, payload) => { hooks.notifyQueue.enqueue(url, payload); return { sent: true, queued: true }; }
+    : (url, payload) => hooks.webhookSender.send(url, payload);
+  send(runtime.notifyWebhook, {
     title: `Key switched: ${info.provider}`,
     text: `${info.from} failed (${info.code}) - next key in pool`,
     provider: info.provider,
@@ -98,11 +108,18 @@ let lastTestCacheRunnerCtx = null;
 const lastTestCache = new LastTestCache();
 const latencyHistogram = new LatencyHistogram();
 const quotaStore = new QuotaStore();
+// #260/#263 module-scope infra (process-wide, reset on reload via buildRuntime)
+let moduleBreaker = null;
+let moduleNotifyQueue = null;
 // Global config accessor safe against early initialization
 let getConfig = () => null;
 let getRuntime = () => null;
 let sandboxRunner = null;
 const webhookSender = new WebhookSender({ fetchImpl: globalThis.fetch });
+// #263: queue webhook I/O off the hot path
+moduleNotifyQueue = new NotifyQueue({ send: (url, payload) => webhookSender.send(url, payload) });
+// #260: process-wide breaker; thresholds re-read from runtime when dispatching
+moduleBreaker = new CircuitBreaker({ threshold: 5, openMs: 30000, halfOpenProbes: 1, now: nowMono });
 const concurrencyTracker = new ConcurrencyTracker();
 function ensureSandboxRunner(ctx) {
   if (sandboxRunner) return sandboxRunner;
@@ -182,6 +199,11 @@ export const Config = Schema.object({
   expiryWarnDays: Schema.number().default(7),
   switchNotify: Schema.boolean().default(false),
   verboseLogging: Schema.boolean().default(false),
+  // #260 circuit breaker
+  circuitBreakerEnabled: Schema.boolean().default(true),
+  circuitBreakerThreshold: Schema.number().default(5),
+  circuitBreakerOpenMs: Schema.number().default(30000),
+  circuitBreakerHalfOpenProbes: Schema.number().default(1),
   switchNotifyThrottleMs: Schema.number().default(60000),
   warnBelowHealthy: Schema.number().default(0),
   latencySloMs: Schema.number().default(0),
@@ -443,6 +465,8 @@ export function apply(ctx, config = {}) {
           byModel: new Map(),
           usageDays: new Map(),
           quotaWindows: new Map(),
+          // #260 per-provider circuit breaker (lazy)
+          breaker: null,
           pointer: 0,
           lastUsed: undefined,
           switches: 0,
@@ -537,7 +561,30 @@ export function apply(ctx, config = {}) {
       const weekly = typeof p.costBudgetWeekly === 'number' ? p.costBudgetWeekly : 0;
       if (daily > 0 || weekly > 0) providerBudgets.set(p.provider, { costBudgetDaily: daily, costBudgetWeekly: weekly, pauseOnBudget: p.pauseOnBudget ?? false });
     }
-    cachedRuntime = { switchCodes, cooldownMs, maxCooldownMs, notifyWebhook, notifyThreshold, concurrencyLimit, cascade, quotaResetWindow, rateLimitThreshold, rpmLimit, webhookActionToken, expiryWarnDays: cfg.expiryWarnDays ?? 7, switchNotify: cfg.switchNotify ?? false, verboseLogging: cfg.verboseLogging ?? false, switchNotifyThrottleMs: cfg.switchNotifyThrottleMs ?? 60000, warnBelowHealthy: cfg.warnBelowHealthy ?? 0, latencySloMs: cfg.latencySloMs ?? 0, providerTags, providerBudgets, poolByRef, providerToPool, modelPoolByProvider, cloneIds };
+    // #266: orphan clone-route GC — expected clones only for live multi-key providers
+    const expectedClones = new Set();
+    for (const p of cfg.providers ?? []) {
+      const n = (p.keys ?? []).filter((k) => typeof k === 'string' && k.length > 0).length;
+      for (let i = 1; i < n; i++) expectedClones.add(`${p.provider}-${i + 1}`);
+    }
+    // drop breaker entries for removed providers
+    if (moduleBreaker) {
+      for (const key of Object.keys(moduleBreaker.snapshot())) {
+        if (![...providerToPool.keys()].includes(key) && !expectedClones.has(key)) {
+          // keep until TTL; only reset if provider gone from config entirely
+          const still = (cfg.providers ?? []).some((p) => p.provider === key);
+          if (!still) moduleBreaker.reset(key);
+        }
+      }
+    }
+    cachedRuntime = { switchCodes, cooldownMs, maxCooldownMs, notifyWebhook, notifyThreshold, concurrencyLimit, cascade, quotaResetWindow, rateLimitThreshold, rpmLimit, webhookActionToken, expiryWarnDays: cfg.expiryWarnDays ?? 7, switchNotify: cfg.switchNotify ?? false, verboseLogging: cfg.verboseLogging ?? false, switchNotifyThrottleMs: cfg.switchNotifyThrottleMs ?? 60000, warnBelowHealthy: cfg.warnBelowHealthy ?? 0, latencySloMs: cfg.latencySloMs ?? 0, providerTags, providerBudgets, poolByRef, providerToPool, modelPoolByProvider, cloneIds, expectedClones,
+      circuitBreakerEnabled: cfg.circuitBreakerEnabled ?? true,
+      circuitBreakerThreshold: cfg.circuitBreakerThreshold ?? 5,
+      circuitBreakerOpenMs: cfg.circuitBreakerOpenMs ?? 30000,
+      circuitBreakerHalfOpenProbes: cfg.circuitBreakerHalfOpenProbes ?? 1,
+      breaker: moduleBreaker,
+      notifyQueue: moduleNotifyQueue,
+    };
     lastConfigRef = rawConfig;
     lastProfilesRef = currentProfiles;
     return cachedRuntime;
@@ -671,18 +718,21 @@ export function apply(ctx, config = {}) {
   }
 
   // rotate() factory (#253): dependencies injected for testability
-  const rotate = createRotate({
+    const rotate = createRotate({
     ctx,
     dispatchStorage,
     buildRuntime,
     pushEvent,
-    notifySwitch,
+    notifySwitch: (runtime, pool, info) => notifySwitch(runtime, pool, info, { webhookSender, notifyQueue: moduleNotifyQueue, now: () => Date.now() }),
     notifyExhaustion,
     recordLatency,
     concurrencyTracker,
     MARKER,
     finishError,
     setRotateStartMs: (v) => { _rotateStartMs = v; },
+    quotaStore,
+    circuitBreaker: moduleBreaker,
+    now: nowMono,
   });
 
   // Retry one request on the next pool key when the current key fails with a
@@ -727,11 +777,13 @@ export function apply(ctx, config = {}) {
     const code = String(payload?.failure?.code ?? payload?.code ?? '');
     const message = String(payload?.failure?.message ?? payload?.message ?? '');
     const effectiveSwitchCodes = pool.switchCodes ?? switchCodes;
-    const switchable = isSwitchableError(payload, effectiveSwitchCodes);
+    const cls = classifyFailure(payload);
+    const switchable = isSwitchableError(payload, effectiveSwitchCodes) || cls.action === 'switch';
     if (!switchable) return next();
+    if (moduleBreaker) moduleBreaker.onFailure(provider);
     const ref = pool.state.lastUsed;
     if (ref) {
-      const backoff = recordFailure(pool, ref, Date.now(), pool.cooldownMs ?? 60000);
+      const backoff = recordFailure(pool, ref, Date.now(), pool.cooldownMs ?? 60000, undefined, cls.soft);
       pushEvent(pool, ref, code || 'UNKNOWN', backoff);
       pool.state.switches = (pool.state.switches ?? 0) + 1;
       pool.state.lastReason = code || 'UNKNOWN';
@@ -773,7 +825,8 @@ export function notifyExhaustion(runtime, pool, options, hooks = { webhookSender
           { id: `pause-${options.provider}`, label: 'Pause 1h' },
         ] : undefined,
       };
-      hooks.webhookSender.send(runtime.notifyWebhook, payload);
+      if (hooks.notifyQueue) hooks.notifyQueue.enqueue(runtime.notifyWebhook, payload);
+      else hooks.webhookSender.send(runtime.notifyWebhook, payload);
     }
   } catch (_) { /* ponytail: never crash rotate() */ }
 }

package/lib/notify-queue.js

@@ -0,0 +1,75 @@
+// lib/notify-queue.js — non-blocking webhook dispatch with backoff (#263).
+// rotate() must never await webhook I/O.
+
+export class NotifyQueue {
+  /**
+   * @param {{ send: (url:string, payload:object)=>Promise<any>, maxDepth?: number, baseBackoffMs?: number, maxBackoffMs?: number }} opts
+   */
+  constructor({ send, maxDepth = 50, baseBackoffMs = 2000, maxBackoffMs = 60000 } = {}) {
+    if (typeof send !== 'function') throw new Error('NotifyQueue: send required');
+    this._send = send;
+    this.maxDepth = maxDepth;
+    this.baseBackoffMs = baseBackoffMs;
+    this.maxBackoffMs = maxBackoffMs;
+    this._q = [];
+    this._busy = false;
+    this._backoffByUrl = new Map();
+    this._dropped = 0;
+    this._sent = 0;
+    this._failed = 0;
+  }
+
+  enqueue(url, payload) {
+    if (!url) return { queued: false, reason: 'no-url' };
+    if (this._q.length >= this.maxDepth) {
+      this._dropped += 1;
+      return { queued: false, reason: 'full', dropped: this._dropped };
+    }
+    this._q.push({ url, payload });
+    this._pump();
+    return { queued: true };
+  }
+
+  _pump() {
+    if (this._busy) return;
+    const job = this._q.shift();
+    if (!job) return;
+    this._busy = true;
+    const delay = this._backoffByUrl.get(job.url) ?? 0;
+    const run = async () => {
+      try {
+        const res = await this._send(job.url, job.payload);
+        if (res && res.sent === false) {
+          this._failed += 1;
+          this._bumpBackoff(job.url);
+        } else {
+          this._sent += 1;
+          this._backoffByUrl.delete(job.url);
+        }
+      } catch {
+        this._failed += 1;
+        this._bumpBackoff(job.url);
+      } finally {
+        this._busy = false;
+        if (this._q.length > 0) this._pump();
+      }
+    };
+    if (delay > 0) {
+      const t = setTimeout(run, delay);
+      if (typeof t.unref === 'function') t.unref();
+    } else {
+      // fire and forget — do not return a promise to callers
+      run();
+    }
+  }
+
+  _bumpBackoff(url) {
+    const cur = this._backoffByUrl.get(url) ?? 0;
+    const next = cur === 0 ? this.baseBackoffMs : Math.min(cur * 2, this.maxBackoffMs);
+    this._backoffByUrl.set(url, next);
+  }
+
+  stats() {
+    return { depth: this._q.length, dropped: this._dropped, sent: this._sent, failed: this._failed, busy: this._busy };
+  }
+}

package/lib/pool.js

@@ -288,7 +288,8 @@ export function isSwitchableError(failureOrPayload, switchCodes = new Set(DEFAUL
   const code = String(failure.code ?? failure.reason ?? '').toUpperCase();
   const message = String(failure.message ?? failureOrPayload.message ?? '');
 
-  // 1. Direct HTTP status codes
+  // 1. Direct HTTP status codes (#267: also 408/425 transient)
+  if ((status === 429 || status === 408 || status === 425) && (switchCodes.has('RATE_LIMIT') || switchCodes.has('QUOTA') || switchCodes.has('429') || switchCodes.has('TIMEOUT') || switchCodes.has(String(status)))) return true;
   if (status === 429 && (switchCodes.has('RATE_LIMIT') || switchCodes.has('QUOTA') || switchCodes.has('429'))) return true;
   if ((status === 401 || status === 403) && (switchCodes.has('AUTH') || switchCodes.has('401'))) return true;
   if ((status >= 500 && status <= 504) && (switchCodes.has('SERVER') || switchCodes.has(String(status)))) return true;

package/lib/rotate.js

@@ -1,5 +1,16 @@
 // lib/rotate.js — stream rotation / failover generator (#253).
 // Pure wiring helper: dependencies are injected so unit tests can supply mocks.
+import { nowMono } from './clock.js';
+import { classifyFailure } from './error-taxonomy.js';
+import {
+  isSwitchableError,
+  recordFailure,
+  parseRetryAfter,
+  extractRateLimit,
+  isRateLimited,
+  formatExhaustionMessage,
+} from './pool.js';
+import { pickCascadeFallback } from './cascade.js';
 
 /**
  * Create the rotate(options, pool) async generator used by llm/stream.
@@ -18,6 +29,9 @@ export function createRotate(deps) {
     MARKER,
     finishError,
     setRotateStartMs,
+    quotaStore,
+    circuitBreaker,
+    now = nowMono,
   } = deps;
 
   function rotate(options, pool) {
@@ -25,16 +39,16 @@ export function createRotate(deps) {
       const runtime0 = buildRuntime();
       const { switchCodes, cooldownMs, maxCooldownMs, switchNotify, rateLimitThreshold } = runtime0;
       let lastFailure = null;
-      const reqStore = { pool, pickedRef: undefined, startMs: Date.now() };
+      const reqStore = { pool, pickedRef: undefined, startMs: now() };
       setRotateStartMs(reqStore.startMs);
       let attemptList = (pool.weightedRefs ?? pool.refs).slice();
       if (runtime0.concurrencyLimit > 0 && concurrencyTracker.isEnabled()) {
         // #193: prefer least-loaded key within limit
         const available = attemptList.filter((r) => {
           const fu = pool.state.failedUntil.get(r) ?? 0;
-          if (fu > Date.now()) return false;
+          if (fu > now()) return false;
           const exp = pool.expiresAt ? pool.expiresAt[r] : undefined;
-          if (exp !== undefined && Date.now() >= exp) return false;
+          if (exp !== undefined && now() >= exp) return false;
           return true;
         });
         const preferred = concurrencyTracker.pickLeastLoaded(available);
@@ -52,7 +66,9 @@ export function createRotate(deps) {
         const _base = pool.cooldownMs ?? cooldownMs;
         const _max = pool.maxCooldownMs ?? maxCooldownMs;
         const _effBase = _retry !== undefined ? Math.max(_base, Math.min(_retry, _max ?? _base * 8)) : _base;
-        const _b = recordFailure(pool, targetRef, Date.now(), _effBase, _max);
+        const cls = classifyFailure({ code: errCode, message: errMsg });
+        const _b = recordFailure(pool, targetRef, now(), _effBase, _max, cls.soft);
+        if (circuitBreaker) circuitBreaker.onFailure(pool.base ?? options?.provider);
         pushEvent(pool, targetRef, errCode ?? 'UNKNOWN', _b);
         if (!pool.state.authFailCounts) pool.state.authFailCounts = new Map();
         if (!pool.state.brokenUntil) pool.state.brokenUntil = new Map();
@@ -61,14 +77,21 @@ export function createRotate(deps) {
           const _c2 = (pool.state.authFailCounts.get(targetRef) ?? 0) + 1;
           pool.state.authFailCounts.set(targetRef, _c2);
           if (_c2 >= 3) {
-            pool.state.brokenUntil.set(targetRef, Date.now() + 86400000 * 30);
-            pool.state.failedUntil.set(targetRef, Date.now() + 86400000 * 30);
+            pool.state.brokenUntil.set(targetRef, now() + 86400000 * 30);
+            pool.state.failedUntil.set(targetRef, now() + 86400000 * 30);
           }
         } else {
           pool.state.authFailCounts.delete(targetRef);
         }
       };
 
+      // #260: fail fast when provider circuit is open
+      if (circuitBreaker && !circuitBreaker.canRequest(options.provider)) {
+        console.warn(`[dsh-key-rotation] ${options.provider}: circuit open — skipping dispatch`);
+        yield finishError('CIRCUIT_OPEN', `[dsh-key-rotation] provider '${options.provider}' circuit is open`);
+        return;
+      }
+
       for (let attempt = 0; attempt < attemptList.length; attempt++) {
         let yielded = false;
         let switching = false;
@@ -108,7 +131,7 @@ export function createRotate(deps) {
                 penalizeRef(activeRef, code ?? 'UNKNOWN', message);
                 pool.state.switches = (pool.state.switches ?? 0) + 1;
                 pool.state.lastReason = String(code ?? 'UNKNOWN');
-                pool.state.lastSwitchAt = Date.now();
+                pool.state.lastSwitchAt = now();
                 lastFailure = chunk;
                 console.warn(`[dsh-key-rotation] ${options.provider}: key ${String(activeRef ?? '?')} failed (${String(code)} ${String(message).slice(0, 100)}) - next key`);
                 // #216: per-switch webhook (opt-in switchNotify), deduped per provider
@@ -157,18 +180,19 @@ export function createRotate(deps) {
               const rate = extractRateLimit(chunk?.metadata?.headers ?? chunk?.headers);
               if (rate && activeRef) {
                 if (isRateLimited(rate, rateLimitThreshold ?? 0.1)) {
-                  const cool = rate.reset && rate.reset > Date.now() ? (rate.reset - Date.now()) : pool.cooldownMs;
-                  recordFailure(pool, activeRef, Date.now(), cool, pool.maxCooldownMs);
+                  const cool = rate.reset && rate.reset > now() ? (rate.reset - now()) : pool.cooldownMs;
+                  recordFailure(pool, activeRef, now(), cool, pool.maxCooldownMs);
                   pushEvent(pool, activeRef, 'RATE_LIMIT', cool);
                   console.warn(`[dsh-key-rotation] ${options.provider}: key ${activeRef} near quota (remaining ${String(rate.remaining)}/${String(rate.limit)}) — next request will rotate`);
                 }
               }
               // #7: persist quota snapshot regardless of threshold (so dashboard widget can show it).
-              if (rate && activeRef && Number.isFinite(rate.remaining)) {
-                quotaStore.set(activeRef, { remaining: rate.remaining, limit: rate.limit, reset: rate.reset, at: Date.now() });
+              if (rate && activeRef && Number.isFinite(rate.remaining) && quotaStore) {
+                quotaStore.set(activeRef, { remaining: rate.remaining, limit: rate.limit, reset: rate.reset, at: now() });
               }
               yield chunk;
               recordLatency(pool, reqStore);
+              if (circuitBreaker) circuitBreaker.onSuccess(options.provider);
               return;
             }
             yield chunk;
@@ -181,7 +205,7 @@ export function createRotate(deps) {
             penalizeRef(activeRef, e?.code ?? 'TRANSPORT', String(e?.message ?? e));
             pool.state.switches = (pool.state.switches ?? 0) + 1;
             pool.state.lastReason = String(e?.code ?? 'TRANSPORT');
-            pool.state.lastSwitchAt = Date.now();
+            pool.state.lastSwitchAt = now();
             lastFailure = finishError(e?.code ?? 'TRANSPORT', String(e?.message ?? e));
             console.warn(`[dsh-key-rotation] ${options.provider}: key ${String(activeRef ?? '?')} stream threw ${String(e?.code ?? e?.message ?? e)} - failover to next key`);
             if (switchNotify && activeRef) {
@@ -204,7 +228,7 @@ export function createRotate(deps) {
       }
 
       // pool exhausted — all keys cooling or missing
-      pool.state.lastExhaustionAt = Date.now();
+      pool.state.lastExhaustionAt = now();
       pool.state.exhaustionCount = (pool.state.exhaustionCount ?? 0) + 1;
       console.warn(`[dsh-key-rotation] ${options.provider}: pool exhausted — all ${pool.refs.length} keys cooling`);
       const runtime = buildRuntime();
@@ -218,7 +242,7 @@ export function createRotate(deps) {
         if (fb && fb.pool && fb.pool !== pool) {
           console.warn(`[dsh-key-rotation] ${options.provider}: pool exhausted — cascading to ${fb.provider}`);
           pool.state.lastReason = 'CASCADE';
-          pool.state.lastSwitchAt = Date.now();
+          pool.state.lastSwitchAt = now();
           // Re-dispatch on the fallback pool (depth-1 via __isCascade guard)
           const innerCascade = rotate({ ...options, provider: fb.provider, __isCascade: true }, fb.pool);
           for await (const chunk of innerCascade) {

package/lib/routes-ops.js

@@ -21,6 +21,7 @@ import { bucketInfo } from './bucket.js';
 import { usageRows, usageCsv } from './usage-report.js';
 import { findSecrets, looksLikeApiSecret } from './keycheck.js';
 import { nextQuotaReset } from './quota-window.js';
+import { classifyFailure } from './error-taxonomy.js';
 
 const STATUS_PATH = '/dsh-key-rotation/status';
 const SNAPSHOT_PATH = '/dsh-key-rotation/snapshot';
@@ -131,6 +132,13 @@ export function registerOpsRoutes(ctx, deps) {
           provider: pool.base,
           keys,
           tags: providerTags.get(pool.base) ?? [],
+          // #260 circuit breaker state (may be null if not yet tripped)
+          circuit: (() => {
+            const br = runtime.breaker;
+            if (!br) return null;
+            const st = br.state(pool.base);
+            return { state: st, threshold: br.threshold, openMs: br.openMs };
+          })(),
           switches: pool.state.switches ?? 0,
           lastReason: pool.state.lastReason ?? null,
           lastSwitchAt: pool.state.lastSwitchAt ?? null,
@@ -157,7 +165,16 @@ export function registerOpsRoutes(ctx, deps) {
           providers.push({ provider: pool.base, keys: [], statusError: String(e?.message ?? e) });
         }
       }
-      json(res, 200, { providers });
+      json(res, 200, {
+        providers,
+        // #266/#263 operational extras (additive)
+        meta: {
+          expectedClones: [...(runtime.expectedClones ?? [])],
+          notifyQueue: runtime.notifyQueue?.stats?.() ?? null,
+          breakerEnabled: runtime.circuitBreakerEnabled !== false,
+          at: now,
+        },
+      });
     },
   }), 'dsh-key-rotation: status route');
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@goodandready/dsh-key-rotation",
-  "version": "0.7.40",
+  "version": "0.8.1",
   "packageManager": "pnpm@10.33.2",
   "description": "Per-provider API key rotation for DeepSeek Harness: a key pool per provider, auto-created clone routes, and switching to the next key on quota/rate-limit errors. Includes a Settings section (Key Rotation) to edit the key pools, cooldown and switch codes.",
   "keywords": [