@goodandready/dsh-key-rotation 0.8.38 → 0.8.40

package/CHANGELOG.md

@@ -1,5 +1,40 @@
 # Changelog
 
+## 0.8.40 - 2026-09-30
+
+### Fixed
+- **Dynamic concurrency limits & least-loaded connection tracking (#407)**:
+  - Added `configure({ limit, staleMs })` to `ConcurrencyTracker` so runtime limit updates from configuration or settings take immediate effect.
+  - Track in-flight request counts across all keys regardless of whether global limit is 0, ensuring accurate `least-loaded` routing metrics and status telemetry.
+  - Atomically acquire in-flight permit at credential selection time in `lib/resolver.js` and roll back immediately if the credential resolver rejects.
+  - Safely release concurrency permits in the outer `finally` block of `lib/rotate.js` on stream completion, client abort, error, or unhandled rejection.
+  - Skip saturated keys in `pickLeastLoaded` and fail closed when all candidate keys exceed their configured concurrency limits.
+- **Model-aware cross-provider cascade failover (#414)**:
+  - `pickCascadeFallback` now maps fallback models to target model sub-pools (`modelPoolByProvider`) rather than falling back only to base provider pools.
+  - Supported model-only providers (providers where credentials are defined exclusively under `models`) as valid cascade fallback targets.
+  - Validated local model token budgets (`isModelQuotaAvailable`) during cascade selection to bypass exhausted model pools.
+- **Aggregated provider monetary budget & model pool enforcement (#422)**:
+  - Added `getProviderCost(provider, runtime, now)` to aggregate daily and weekly spend across provider base pools and all model sub-pools without double-counting shared state.
+  - `checkBudgetAndHealthAlerts` now consolidates provider spend across all pools, sends unified webhook alerts with correct provider names, and pauses all pools (base and model sub-pools) for 24h when `pauseOnBudget: true`.
+  - Enforced monetary budget refusal (`LOCAL_POOL_EXHAUSTED`) at credential resolution time in `lib/resolver.js` when provider budget is exceeded and `pauseOnBudget` is active.
+
+## 0.8.39 - 2026-09-30
+
+### Fixed
+- **SSRF guard: IPv4-mapped IPv6 loopbacks & connect-time rebinding (#411)**:
+  - Blocked hex (`::ffff:7f00:1`, `::ffff:7f00:0001`), dotted (`::ffff:127.0.0.1`), IPv4-compatible (`::/96`), and NAT64 (`64:ff9b::/96`) loopback addresses via full 8-word IPv6 unpacking in `lib/safe-fetch.js`.
+  - Added connect-time DNS resolution enforcement using `undici.Agent` lookup hook to prevent TOCTOU DNS rebinding before establishing socket connections.
+- **Fail-closed credential resolver on pool exhaustion (#406)**:
+  - Return fail-closed error `LOCAL_POOL_EXHAUSTED` instead of leaking unmanaged `original(ref)` when all credentials in a managed pool are exhausted or blocked by cooldown, expiry, RPM, or TPM.
+- **Credential status inheritance & permanent auth revocation (#408)**:
+  - Model sub-pools inherit `paused`, `revoked`, and `expiresAt` credentials from their parent base provider pool.
+  - Runtime 401 permanent auth failure marks credential as revoked (`revokedRefs.add` and `failedUntil = Infinity`) across all associated pools sharing the ref.
+- **Persistence of revocation state and spend counters (#409)**:
+  - `StatePersistence` serializes and restores `revokedRefs` (as a Set), `Infinity` failedUntil timestamps, and daily usage/cost aggregation maps (`costDays`, `costPerKey`, `usageDays`, `usageCounts`).
+- **UI credential alignment and model weights on reordering (#412)**:
+  - Preserved `paused` and `revoked` flag alignment with credential references during reordering, deletion, and undo actions in the settings card.
+  - Model pool key reordering preserves model-specific weights and expiry mappings.
+
 ## 0.8.38 - 2026-09-30
 
 ### Added

package/README.md

@@ -207,7 +207,9 @@ Access full visual management under **Settings → Key Rotation** or via the Hea
 * **Secure Vault Storage**: Actual secret values reside securely in `$DSH_HOME/.credentials.yaml` managed by the DSH `Credentials` service.
 * **5-Character Masking (`keyTail`)**: Full secret values are never sent to the client browser; only the trailing 5 characters are exposed for visual identification. Short keys (<= 5 characters) return a fixed masked placeholder (`***`) to prevent credential disclosure.
 * **Fail-Closed Loopback & Same-Origin Fencing**: Administrative endpoints strictly enforce loopback origin checks (`isTrustedBridgeRequest`), requiring valid `Origin` headers, matching `Host` headers, and rejecting `cross-site` or non-loopback requests without fallbacks.
-* **SSRF Guard on Import Route**: Remote provider pool import strictly enforces HTTPS-only URLs, validates all resolved IP addresses against private and reserved ranges (RFC 1918, loopback, link-local, multicast, CGNAT), verifies redirect targets, and caps response payload size to 1 MB.
+* **Fail-Closed Resolver on Pool Exhaustion**: When all credentials in a managed pool are exhausted, paused, expired, or blocked by RPM/TPM limits, the resolver fails closed with `LOCAL_POOL_EXHAUSTED` error rather than falling back to leaking unmanaged credentials.
+* **SSRF Protection & Rebinding Guard**: Remote provider pool import strictly enforces HTTPS-only URLs, validates all resolved IP addresses including IPv4-mapped and IPv4-compatible IPv6 addresses (`::ffff:127.0.0.1`, `::ffff:7f00:1`, `64:ff9b::/96`), and enforces connect-time DNS validation via undici agent dispatchers to prevent TOCTOU DNS rebinding.
+* **Cross-Pool Revocation & Inheritance**: Model pools automatically inherit pause, revoke, and expiry states from their base provider; runtime 401 permanent authentication failures revoke the credential across all shared pools immediately.
 
 ---
 

package/README.ru.md

@@ -194,7 +194,9 @@ graph LR
 * **Защищённое хранилище хоста**: Реальные значения ключей сохраняются в `$DSH_HOME/.credentials.yaml` сервисом `Credentials`.
 * **Маскировка в браузере (5 символов)**: Браузер получает только последние 5 символов ключа для визуального отличия. Для коротких ключей (длиной <= 5 символов) возвращается безопасная маскировочная заглушка `***`, исключающая раскрытие ключа целиком.
 * **Fail-Closed изоляция Loopback и Same-Origin**: Все управляющие эндпоинты строго проверяют происхождение через `isTrustedBridgeRequest`, требуя обязательный заголовок `Origin`, соответствие `Host` и отклоняя запросы `cross-site` и не-loopback без каких-либо исключений.
-* **SSRF-защита маршрута импорта**: Маршрут импорта пулов провайдеров разрешает исключительно HTTPS, валидирует все резолвящиеся IP-адреса с блокировкой приватных и зарезервированных диапазонов (RFC 1918, loopback, link-local, multicast, CGNAT), проверяет адреса редиректов и ограничивает размер ответа 1 МБ.
+* **Fail-Closed резолвер при исчерпании пула**: Если все ключи в пуле исчерпаны, приостановлены, истекли или заблокированы по RPM/TPM, резолвер завершается с ошибкой `LOCAL_POOL_EXHAUSTED`, исключая скрытую утечку исходного неконтролируемого ключа.
+* **SSRF-защита и блокировка DNS Rebinding**: Маршрут импорта пулов провайдеров разрешает исключительно HTTPS, валидирует все IP-адреса, включая IPv4-mapped и IPv4-compatible IPv6 представления (`::ffff:127.0.0.1`, `::ffff:7f00:1`, `64:ff9b::/96`), и выполняет проверку адреса в момент соединения (connect-time DNS validation) через диспетчер undici для защиты от TOCTOU DNS rebinding.
+* **Сквозной отзыв и наследование статусов**: Пул конкретной модели автоматически наследует флаги паузы, отзыва и срок действия базового провайдера; перманентный сбой аутентификации 401 немедленно отзывает ключ во всех связанных пулах.
 
 ---
 

package/README.zh.md

@@ -186,7 +186,9 @@ graph LR
 * **宿主安全存储**：真实密钥持久化保存在 `$DSH_HOME/.credentials.yaml`。
 * **前台 5 字符脱敏**：前端仅展示密钥后 5 位字符进行视觉区分。对于长度 <= 5 的短密钥，返回统一脱敏占位符 (`***`)，防止凭证完整泄露。
 * **Fail-Closed 环回同源安全隔离**：管理接口严格通过 `isTrustedBridgeRequest` 验证本地同源请求，强制要求有效 `Origin` 和 `Host` 匹配，无条件拒绝跨站 (`cross-site`) 与非环回调用。
-* **导入接口 SSRF 防护**：远程提供商配置导入严格仅支持 HTTPS，解析并拦截所有私有/保留 IP 地址（RFC 1918、环回、链路本地、多播、CGNAT），对重定向目标进行全程验证，并将响应体上限严格控制在 1 MB 以内。
+* **凭证池耗尽 Fail-Closed 熔断**：当受管凭证池中所有密钥均被耗尽、暂停、过期或触发 RPM/TPM 限制时，解析器立即返回 `LOCAL_POOL_EXHAUSTED` 错误熔断，杜绝静默回退泄漏未受控原始密钥。
+* **SSRF 与 DNS 重绑定防护**：远程配置导入严格仅支持 HTTPS，全面校验解析 IP 并拦截 IPv4-mapped/IPv4-compatible 等各类 IPv6 环回变体（如 `::ffff:127.0.0.1`、`::ffff:7f00:1`、`64:ff9b::/96`），并通过 undici 调度器执行连接时（connect-time）DNS 校验，从根本上防御 TOCTOU DNS 重绑定。
+* **跨池吊销与状态继承**：模型子池自动继承基础提供商的暂停、吊销与过期时间；运行时 401 永久认证失败将即时在所有关联共享池中统一标记吊销。
 
 ---
 

package/lib/budget-monitor.js

@@ -1,6 +1,7 @@
 // lib/budget-monitor.js — periodic budget, expiry, low health and SLO alerts
 import { expiringSoon, shouldNotifyDaily, costForDay, costForWeek, budgetVerdict, isKeyPaused, isKeyRevoked } from './pool.js';
 import { isModelQuotaAvailable, hasModelQuotaConfig } from './model-quota.js';
+import { getProviderCost } from './resolver.js';
 
 const DAY_MS = 86400000;
 
@@ -17,58 +18,61 @@ export function checkBudgetAndHealthAlerts({
   logger,
 }) {
   try {
-    const seen = new Set();
-    for (const pool of runtime.poolByRef.values()) {
-      if (seen.has(pool.base)) continue;
-      seen.add(pool.base);
-      // #207: keys expiring within expiryWarnDays -> one webhook per key/day
-      for (const { ref, expiresInDays } of expiringSoon(pool, runtime.expiryWarnDays, now)) {
-        if (!shouldNotifyDaily(expiryNotifiedAt, pool.base + ':' + ref, now)) continue;
-        logger?.warn?.(`[dsh-key-rotation] ${pool.base}: key ${ref} expires in ~${expiresInDays}d`);
-        if (runtime.notifyWebhook) {
-          webhookSender.send(runtime.notifyWebhook, {
-            title: `Key expiring soon: ${pool.base}`,
-            text: `${ref} expires in ~${expiresInDays} day(s)`,
-            provider: pool.base,
-            kind: 'expiry',
-            keys: [ref],
-          });
-        }
-      }
-      // #208: daily/weekly budget -> warn webhook, optional 1-day pause at 100%
-      const budget = runtime.providerBudgets?.get?.(pool.base);
-      if (budget) {
-        const daily = costForDay(pool.state.costDays);
-        const weekly = costForWeek(pool.state.costDays, now);
+    // #208 / #422: provider monetary budget alerts & pause across base and model pools
+    if (runtime.providerBudgets && runtime.providerBudgets.size > 0) {
+      for (const [provider, budget] of runtime.providerBudgets) {
+        const { daily, weekly, pools } = getProviderCost(provider, runtime, now);
         const verdict = budgetVerdict(daily, budget.costBudgetDaily);
         const wVerdict = budgetVerdict(weekly, budget.costBudgetWeekly);
         const hit = verdict.warn || wVerdict.warn;
-        if (hit && shouldNotifyDaily(budgetNotifiedAt, pool.base + ':budget', now)) {
-          logger?.warn?.(`[dsh-key-rotation] ${pool.base}: cost budget - day $${daily.toFixed(2)}/$${budget.costBudgetDaily} week $${weekly.toFixed(2)}/$${budget.costBudgetWeekly}`);
+        if (hit && shouldNotifyDaily(budgetNotifiedAt, provider + ':budget', now)) {
+          logger?.warn?.(`[dsh-key-rotation] ${provider}: cost budget - day $${daily.toFixed(2)}/$${budget.costBudgetDaily} week $${weekly.toFixed(2)}/$${budget.costBudgetWeekly}`);
           if (runtime.notifyWebhook) {
-            // #217: budget webhook gains action buttons when a callback token is configured
             const token = runtime.webhookActionToken ?? '';
             webhookSender.send(runtime.notifyWebhook, {
-              title: `Cost budget: ${pool.base}`,
+              title: `Cost budget: ${provider}`,
               text: `day $${daily.toFixed(2)} of $${budget.costBudgetDaily} · week $${weekly.toFixed(2)} of $${budget.costBudgetWeekly}` + (verdict.exceeded || wVerdict.exceeded ? ' · EXCEEDED' : ''),
-              provider: pool.base,
+              provider,
               kind: 'budget',
               spend: { daily, weekly },
               actionToken: token || undefined,
               actions: token ? [
-                { id: `pause-${pool.base}`, label: 'Pause 1h' },
-                { id: `reset-${pool.base}`, label: 'Reset cooldown' },
+                { id: `pause-${provider}`, label: 'Pause 1h' },
+                { id: `reset-${provider}`, label: 'Reset cooldown' },
               ] : undefined,
             });
           }
         }
         if ((verdict.exceeded || wVerdict.exceeded) && budget.pauseOnBudget) {
           const until = now + DAY_MS;
-          for (const ref of pool.refs) {
-            if ((pool.state.failedUntil.get(ref) ?? 0) < until) pool.state.failedUntil.set(ref, until);
+          for (const p of pools) {
+            for (const ref of p.refs) {
+              if ((p.state.failedUntil.get(ref) ?? 0) < until) p.state.failedUntil.set(ref, until);
+            }
           }
         }
       }
+    }
+
+    const seen = new Set();
+    for (const pool of runtime.poolByRef.values()) {
+      if (seen.has(pool.base)) continue;
+      seen.add(pool.base);
+      // #207: keys expiring within expiryWarnDays -> one webhook per key/day
+      for (const { ref, expiresInDays } of expiringSoon(pool, runtime.expiryWarnDays, now)) {
+        if (!shouldNotifyDaily(expiryNotifiedAt, pool.base + ':' + ref, now)) continue;
+        logger?.warn?.(`[dsh-key-rotation] ${pool.base}: key ${ref} expires in ~${expiresInDays}d`);
+        if (runtime.notifyWebhook) {
+          webhookSender.send(runtime.notifyWebhook, {
+            title: `Key expiring soon: ${pool.base}`,
+            text: `${ref} expires in ~${expiresInDays} day(s)`,
+            provider: pool.base,
+            kind: 'expiry',
+            keys: [ref],
+          });
+        }
+      }
+      // #208 / #422: handled provider-wide before pool loop
       // #221: pool running low - webhook while healthy < warnBelowHealthy
       const warnBelow = runtime.warnBelowHealthy ?? 0;
       if (warnBelow > 0) {

package/lib/cascade.js

@@ -1,28 +1,41 @@
-// cascade.js — cross-provider failover cascade (issues #194, #364, #376).
+// cascade.js — cross-provider failover cascade (issues #194, #364, #376, #414).
 // ponytail: minimal — pick fallback provider with model mapping from config.
-import { isKeyPaused, isKeyRevoked } from './pool.js';
+import { isKeyPaused, isKeyRevoked, selectPool } from './pool.js';
+import { isModelQuotaAvailable, hasModelQuotaConfig } from './model-quota.js';
 
 export const CASCADE_MAX_DEPTH = 3;
 
-export function pickCascadeFallback(provider, cfg, pools, currentModel) {
+export function pickCascadeFallback(provider, cfg, pools, currentModel, modelPoolByProvider) {
   const list = Array.isArray(cfg && cfg.cascade) ? cfg.cascade : [];
+  const modelPools = modelPoolByProvider || cfg?.modelPoolByProvider || cfg?.index?.modelPoolByProvider || null;
+  const provPools = pools || cfg?.providerToPool || cfg?.index?.providerToPool || null;
+
   for (const entry of list) {
     const fb = typeof entry === 'string' ? { provider: entry } : entry;
     if (!fb || !fb.provider || fb.provider === provider) continue;
-    const pool = pools instanceof Map ? pools.get(fb.provider) : (pools ? pools[fb.provider] : null);
+
+    const mappedModel = (fb.modelMapping && currentModel && fb.modelMapping[currentModel]) || fb.model || currentModel || null;
+    let pool = null;
+    if (modelPools && mappedModel) {
+      pool = selectPool(modelPools, provPools, fb.provider, mappedModel);
+    }
+    if (!pool && provPools) {
+      pool = provPools instanceof Map ? provPools.get(fb.provider) : (provPools ? provPools[fb.provider] : null);
+    }
     if (!pool) continue;
+
     const now = Date.now();
     let healthy = 0;
     for (const ref of pool.refs) {
       if (isKeyPaused(pool, ref) || isKeyRevoked(pool, ref)) continue;
       const failedUntil = (pool.state && pool.state.failedUntil && pool.state.failedUntil.get(ref)) || 0;
       if (failedUntil > now) continue;
-      const exp = pool.expiresAt ? pool.expiresAt[ref] : undefined;
+      const exp = pool.expiresAt ? pool.expiresAt[ref] : (pool.basePool?.expiresAt ? pool.basePool.expiresAt[ref] : undefined);
       if (exp !== undefined && now >= exp) continue;
+      if (hasModelQuotaConfig(pool) && !isModelQuotaAvailable(pool, ref, now)) continue;
       healthy += 1;
     }
     if (healthy === 0) continue;
-    const mappedModel = (fb.modelMapping && currentModel && fb.modelMapping[currentModel]) || fb.model || currentModel || null;
     return { provider: fb.provider, pool, model: mappedModel };
   }
   return null;

package/lib/client.js

@@ -938,7 +938,7 @@ window.__ModuleLoader__.load({
 
     function reorderKeys(entry, order) {
       const next = { ...entry, keys: order.map(index => entry.keys[index]) };
-      for (const [field, fallback] of [['weights', 1], ['expiresAt', 0]]) {
+      for (const [field, fallback] of [['weights', 1], ['expiresAt', 0], ['paused', false], ['revoked', false]]) {
         if (Array.isArray(entry[field]) && entry[field].length) next[field] = order.map(index => entry[field][index] ?? fallback);
       }
       return next;
@@ -1213,7 +1213,7 @@ window.__ModuleLoader__.load({
             const entry = { ...providers[index] };
             const at = Math.min(u.kIndex, entry.keys.length);
             entry.keys = [...entry.keys]; entry.keys.splice(at, 0, u.key);
-            for (const [field, fallback] of [['weights', 1], ['expiresAt', 0]]) {
+            for (const [field, fallback] of [['weights', 1], ['expiresAt', 0], ['paused', false], ['revoked', false]]) {
               if (u[field] !== undefined || entry[field]?.length) {
                 const values = Array.from({ length: entry.keys.length - 1 }, (_, i) => entry[field]?.[i] ?? fallback);
                 values.splice(at, 0, u[field] ?? fallback); entry[field] = values;
@@ -1563,7 +1563,7 @@ window.__ModuleLoader__.load({
         const allRefs = providers.flatMap((prov) => prov?.keys ?? []);
         entry.keys = [...(entry.keys ?? []), nextKeyRef(entry.provider, entry.keys, allRefs)];
         // keep weights aligned with keys (#215): new key gets default weight 1
-        for (const [field, fallback] of [['weights', 1], ['expiresAt', 0]]) {
+        for (const [field, fallback] of [['weights', 1], ['expiresAt', 0], ['paused', false], ['revoked', false]]) {
           if (Array.isArray(entry[field]) && entry[field].length) entry[field] = Array.from({ length: entry.keys.length }, (_, i) => entry[field][i] ?? fallback);
         }
         providers[pIndex] = entry;
@@ -1574,7 +1574,8 @@ window.__ModuleLoader__.load({
         const entry = next[pIndex];
         if (!entry || !entry.keys[kIndex]) return cur;
         stashUndo({ type: 'key', provider: entry.provider, kIndex, key: entry.keys[kIndex],
-          weights: entry.weights?.[kIndex], expiresAt: entry.expiresAt?.[kIndex] });
+          weights: entry.weights?.[kIndex], expiresAt: entry.expiresAt?.[kIndex],
+          paused: entry.paused?.[kIndex], revoked: entry.revoked?.[kIndex] });
         next[pIndex] = reorderKeys(entry, entry.keys.map((_, index) => index).filter(index => index !== kIndex));
         return { ...cur, providers: next };
       });
@@ -1691,9 +1692,9 @@ window.__ModuleLoader__.load({
         const keys = Array.isArray(mp.keys) ? [...mp.keys] : [];
         const target = kIndex + delta;
         if (target < 0 || target >= keys.length) return;
-        [keys[kIndex], keys[target]] = [keys[target], keys[kIndex]];
-        mp.keys = keys;
-        models[model] = mp;
+        const order = keys.map((_, i) => i);
+        [order[kIndex], order[target]] = [order[target], order[kIndex]];
+        models[model] = reorderKeys(mp, order);
       });
 
       const save = async () => {

package/lib/concurrency.js

@@ -1,4 +1,4 @@
-// concurrency.js — per-key in-flight counter + least-connections picking (issue #193).
+// concurrency.js — per-key in-flight counter + least-connections picking (issues #193, #407).
 
 const CONCURRENCY_DEFAULT_LIMIT = 0;
 const CONCURRENCY_STALE_LOCK_MS = 5 * 60 * 1000;
@@ -12,12 +12,22 @@ export class ConcurrencyTracker {
     this._inFlight = new Map();
   }
 
+  configure(opts) {
+    if (!opts) return;
+    if (opts.limit !== undefined) {
+      this._limit = (Number.isFinite(opts.limit) && opts.limit >= 0) ? Math.floor(opts.limit) : 0;
+    }
+    if (opts.staleMs !== undefined && Number.isFinite(opts.staleMs) && opts.staleMs > 0) {
+      this._staleMs = opts.staleMs;
+    }
+  }
+
   isEnabled() { return this._limit > 0; }
   get limit() { return this._limit; }
 
-  acquire(ref, now) {
+  acquire(ref, now, limit) {
     now = now || Date.now();
-    if (!this.isEnabled()) return true;
+    const effectiveLimit = (typeof limit === 'number' && limit >= 0) ? Math.floor(limit) : this._limit;
     let e = this._inFlight.get(ref);
     if (!e) {
       e = { count: 0, lastAcquired: now };
@@ -26,7 +36,7 @@ export class ConcurrencyTracker {
     if (now - e.lastAcquired > this._staleMs) {
       e.count = 0;
     }
-    if (e.count >= this._limit) return false;
+    if (effectiveLimit > 0 && e.count >= effectiveLimit) return false;
     e.count += 1;
     e.lastAcquired = now;
     return true;
@@ -55,8 +65,9 @@ export class ConcurrencyTracker {
     return out;
   }
 
-  pickLeastLoaded(candidates, now, weights) {
+  pickLeastLoaded(candidates, now, weights, limit) {
     now = now || Date.now();
+    const effectiveLimit = (typeof limit === 'number' && limit >= 0) ? Math.floor(limit) : this._limit;
     if (!Array.isArray(candidates) || candidates.length === 0) return null;
     let best = null;
     let bestScore = Infinity;
@@ -64,7 +75,7 @@ export class ConcurrencyTracker {
       const e = this._inFlight.get(ref);
       let count = e ? e.count : 0;
       if (e && now - e.lastAcquired > this._staleMs) count = 0;
-      if (this.isEnabled() && count >= this._limit) continue;
+      if (effectiveLimit > 0 && count >= effectiveLimit) continue;
       const w = (weights && Number(weights[ref]) > 0) ? Number(weights[ref]) : 1;
       const score = count / w;
       if (score < bestScore) {

package/lib/index.js

@@ -324,6 +324,7 @@ export function apply(ctx, config = {}) {
       budgetNotifiedAt,
     });
 
+    concurrencyTracker.configure({ limit: concurrencyLimit });
     cachedRuntime = {
       switchCodes, cooldownMs, maxCooldownMs, notifyWebhook, notifyThreshold,
       concurrencyLimit, cascade, quotaResetWindow, rateLimitThreshold, rpmLimit, tpmLimit,
@@ -355,9 +356,12 @@ export function apply(ctx, config = {}) {
       const resolve = createResolver({
         buildRuntime,
         currentPool: () => dispatchStorage.getStore()?.pool,
-        onPicked: (pool, candidate) => {
+        onPicked: (pool, candidate, meta) => {
           const store = dispatchStorage.getStore();
-          if (store && store.pool === pool) store.pickedRef = candidate;
+          if (store && store.pool === pool) {
+            store.pickedRef = candidate;
+            if (meta?.concurrencyRef) store.concurrencyRef = meta.concurrencyRef;
+          }
         },
         latencyHistogram,
         concurrencyTracker,

package/lib/persistence.js

@@ -33,8 +33,17 @@ export class StatePersistence {
         if (st && st.failedUntil && typeof st.failedUntil.forEach === 'function') {
           st.failedUntil.forEach((until, ref) => {
             if (Number.isFinite(until)) failedUntil[ref] = until;
+            else if (until === Infinity) failedUntil[ref] = 'Infinity';
           });
         }
+        const revokedRefs = [];
+        if (st && st.revokedRefs) {
+          if (typeof st.revokedRefs.forEach === 'function') {
+            st.revokedRefs.forEach((r) => revokedRefs.push(r));
+          } else if (Array.isArray(st.revokedRefs)) {
+            revokedRefs.push(...st.revokedRefs);
+          }
+        }
         const tokenUsage = {};
         if (st && st.tokenUsage && typeof st.tokenUsage.forEach === 'function') {
           st.tokenUsage.forEach((entry, ref) => {
@@ -45,14 +54,38 @@ export class StatePersistence {
             tokenUsage[ref] = { used: Math.floor(used), resetAt: Math.floor(resetAt) };
           });
         }
+        const costDays = {};
+        if (st?.costDays) {
+          const src = st.costDays instanceof Map ? Object.fromEntries(st.costDays) : st.costDays;
+          if (typeof src === 'object' && src !== null) Object.assign(costDays, src);
+        }
+        const costPerKey = {};
+        if (st?.costPerKey) {
+          const src = st.costPerKey instanceof Map ? Object.fromEntries(st.costPerKey) : st.costPerKey;
+          if (typeof src === 'object' && src !== null) Object.assign(costPerKey, src);
+        }
+        const usageDays = {};
+        if (st?.usageDays) {
+          const src = st.usageDays instanceof Map ? Object.fromEntries(st.usageDays) : st.usageDays;
+          if (typeof src === 'object' && src !== null) Object.assign(usageDays, src);
+        }
+        const usageCounts = {};
+        if (st?.usageCounts instanceof Map) {
+          st.usageCounts.forEach((cnt, ref) => {
+            if (Number.isFinite(cnt)) usageCounts[ref] = cnt;
+          });
+        }
         const outPool = {
           failedUntil,
           pointer: Number.isFinite(st?.pointer) ? st.pointer : 0,
           lastUsed: typeof st?.lastUsed === 'string' ? st.lastUsed : null,
         };
-        if (Object.keys(tokenUsage).length > 0) {
-          outPool.tokenUsage = tokenUsage;
-        }
+        if (revokedRefs.length > 0) outPool.revokedRefs = revokedRefs;
+        if (Object.keys(tokenUsage).length > 0) outPool.tokenUsage = tokenUsage;
+        if (Object.keys(costDays).length > 0) outPool.costDays = costDays;
+        if (Object.keys(costPerKey).length > 0) outPool.costPerKey = costPerKey;
+        if (Object.keys(usageDays).length > 0) outPool.usageDays = usageDays;
+        if (Object.keys(usageCounts).length > 0) outPool.usageCounts = usageCounts;
         pools[base] = outPool;
       });
     }
@@ -109,7 +142,7 @@ export class StatePersistence {
   async load() {
     const raw = await safeReadJson(this.filePath, null);
     if (!raw || typeof raw !== 'object' || Array.isArray(raw)) return null;
-    if (raw.version !== 1) return null;
+    if (raw.version !== 1 && raw.version !== 2) return null;
     return raw;
   }
 
@@ -126,7 +159,17 @@ export class StatePersistence {
       if (!existing || !hasPoolActivity(state)) {
         if (saved.failedUntil && typeof saved.failedUntil === 'object' && !Array.isArray(saved.failedUntil)) {
           for (const [ref, until] of Object.entries(saved.failedUntil)) {
-            if (Number.isFinite(until)) state.failedUntil.set(ref, until);
+            if (until === 'Infinity' || until === Infinity) {
+              state.failedUntil.set(ref, Infinity);
+            } else if (Number.isFinite(until)) {
+              state.failedUntil.set(ref, until);
+            }
+          }
+        }
+        if (Array.isArray(saved.revokedRefs)) {
+          if (!state.revokedRefs) state.revokedRefs = new Set();
+          for (const r of saved.revokedRefs) {
+            if (typeof r === 'string') state.revokedRefs.add(r);
           }
         }
         if (saved.tokenUsage && typeof saved.tokenUsage === 'object' && !Array.isArray(saved.tokenUsage)) {
@@ -139,6 +182,30 @@ export class StatePersistence {
             state.tokenUsage.set(ref, { used: Math.floor(used), resetAt: Math.floor(resetAt) });
           }
         }
+        if (saved.costDays && typeof saved.costDays === 'object') {
+          if (!state.costDays) state.costDays = new Map();
+          for (const [day, cost] of Object.entries(saved.costDays)) {
+            if (Number.isFinite(cost)) state.costDays.set(day, cost);
+          }
+        }
+        if (saved.costPerKey && typeof saved.costPerKey === 'object') {
+          if (!state.costPerKey) state.costPerKey = new Map();
+          for (const [k, c] of Object.entries(saved.costPerKey)) {
+            if (Number.isFinite(c)) state.costPerKey.set(k, c);
+          }
+        }
+        if (saved.usageDays && typeof saved.usageDays === 'object') {
+          if (!state.usageDays) state.usageDays = new Map();
+          for (const [d, u] of Object.entries(saved.usageDays)) {
+            if (Number.isFinite(u)) state.usageDays.set(d, u);
+          }
+        }
+        if (saved.usageCounts && typeof saved.usageCounts === 'object') {
+          if (!state.usageCounts) state.usageCounts = new Map();
+          for (const [k, cnt] of Object.entries(saved.usageCounts)) {
+            if (Number.isFinite(cnt)) state.usageCounts.set(k, cnt);
+          }
+        }
         state.pointer = Number.isSafeInteger(saved.pointer) && saved.pointer >= 0 ? saved.pointer : 0;
         state.lastUsed = typeof saved.lastUsed === 'string' ? saved.lastUsed : null;
       }

package/lib/pool-builder.js

@@ -27,6 +27,7 @@ export function buildPoolItem({
   poolGuard,
   rpmLimit,
   tpmLimit = 0,
+  concurrencyLimit = 0,
   makeState,
   provider = null,
   model = null,
@@ -53,17 +54,30 @@ export function buildPoolItem({
       const exp = parseExpiry(expiresAt[indexed[i].index]);
       if (exp !== undefined) parsedExpiry[refs[i]] = exp;
     }
+  } else if (expiresAt && typeof expiresAt === 'object') {
+    for (let i = 0; i < refs.length; i++) {
+      const exp = parseExpiry(expiresAt[refs[i]]);
+      if (exp !== undefined) parsedExpiry[refs[i]] = exp;
+    }
   }
   const parsedPaused = new Set();
   if (Array.isArray(paused)) {
     for (let i = 0; i < refs.length; i++) {
-      if (paused[indexed[i].index] === true) parsedPaused.add(refs[i]);
+      if (paused[indexed[i].index] === true || paused.includes(refs[i])) parsedPaused.add(refs[i]);
+    }
+  } else if (paused instanceof Set) {
+    for (let i = 0; i < refs.length; i++) {
+      if (paused.has(refs[i])) parsedPaused.add(refs[i]);
     }
   }
   const parsedRevoked = new Set();
   if (Array.isArray(revoked)) {
     for (let i = 0; i < refs.length; i++) {
-      if (revoked[indexed[i].index] === true) parsedRevoked.add(refs[i]);
+      if (revoked[indexed[i].index] === true || revoked.includes(refs[i])) parsedRevoked.add(refs[i]);
+    }
+  } else if (revoked instanceof Set) {
+    for (let i = 0; i < refs.length; i++) {
+      if (revoked.has(refs[i])) parsedRevoked.add(refs[i]);
     }
   }
   const weightsMap = {};
@@ -90,6 +104,7 @@ export function buildPoolItem({
     expiresAt: parsedExpiry,
     rpmLimit,
     tpmLimit: typeof tpmLimit === 'number' && Number.isFinite(tpmLimit) && tpmLimit > 0 ? Math.floor(tpmLimit) : 0,
+    concurrencyLimit: typeof concurrencyLimit === 'number' && Number.isFinite(concurrencyLimit) && concurrencyLimit > 0 ? Math.floor(concurrencyLimit) : 0,
     routingStrategy: poolStrategy,
     proactiveRateLimitGuard: poolGuard,
   };
@@ -98,6 +113,7 @@ export function buildPoolItem({
 export function buildPools({ cfg = {}, poolState = new Map(), defaultState = initializePoolState } = {}) {
   const rpmLimit = cfg.rpmLimit ?? 0;
   const tpmLimit = cfg.tpmLimit ?? 0;
+  const concurrencyLimit = cfg.concurrencyLimit ?? 0;
   const quotaResetWindow = cfg.quotaResetWindow ?? null;
   const poolByRef = new Map();
   const providerToPool = new Map();
@@ -114,10 +130,10 @@ export function buildPools({ cfg = {}, poolState = new Map(), defaultState = ini
     return initializePoolState(st);
   };
 
-  const buildPool = (base, keys, weights, paused, revoked, poolCooldown, poolMax, expiresAt, poolStrategy, poolGuard, meta, pRpm = rpmLimit, pTpm = tpmLimit) =>
+  const buildPool = (base, keys, weights, paused, revoked, poolCooldown, poolMax, expiresAt, poolStrategy, poolGuard, meta, pRpm = rpmLimit, pTpm = tpmLimit, pConcurrency = concurrencyLimit) =>
     buildPoolItem({
       base, keys, weights, paused, revoked, poolCooldown, poolMax, expiresAt, poolStrategy, poolGuard,
-      rpmLimit: pRpm, tpmLimit: pTpm, makeState, quotaResetWindow, ...meta,
+      rpmLimit: pRpm, tpmLimit: pTpm, concurrencyLimit: pConcurrency, makeState, quotaResetWindow, ...meta,
     });
 
   for (const p of cfg.providers ?? []) {
@@ -125,10 +141,11 @@ export function buildPools({ cfg = {}, poolState = new Map(), defaultState = ini
     const poolMax = typeof p.maxCooldownMs === 'number' ? p.maxCooldownMs : (cfg.maxCooldownMs ?? undefined);
     const provRpm = typeof p.rpmLimit === 'number' ? p.rpmLimit : rpmLimit;
     const provTpm = typeof p.tpmLimit === 'number' ? p.tpmLimit : tpmLimit;
+    const provConcurrency = typeof p.concurrencyLimit === 'number' ? p.concurrencyLimit : concurrencyLimit;
     const pool = buildPool(p.provider, p.keys, p.weights, p.paused, p.revoked, poolCooldown, poolMax, p.expiresAt, p.routingStrategy, p.proactiveRateLimitGuard, {
       provider: p.provider,
       model: null,
-    }, provRpm, provTpm);
+    }, provRpm, provTpm, provConcurrency);
     if (pool) {
       for (const ref of pool.refs) {
         if (!poolByRef.has(ref)) poolByRef.set(ref, pool);
@@ -140,12 +157,23 @@ export function buildPools({ cfg = {}, poolState = new Map(), defaultState = ini
     for (const [model, mp] of Object.entries(p.models ?? {})) {
       const modelRpm = typeof mp.rpmLimit === 'number' ? mp.rpmLimit : provRpm;
       const modelTpm = typeof mp.tpmLimit === 'number' ? mp.tpmLimit : provTpm;
-      const mpool = buildPool(`${p.provider}::${model}`, mp.keys, mp.weights, mp.paused, mp.revoked, poolCooldown, poolMax, undefined, p.routingStrategy, p.proactiveRateLimitGuard, {
+      const modelConcurrency = typeof mp.concurrencyLimit === 'number' ? mp.concurrencyLimit : provConcurrency;
+      const mergedPaused = new Set([
+        ...(pool?.pausedRefs ?? []),
+        ...(Array.isArray(mp.paused) ? mp.paused : (mp.paused instanceof Set ? [...mp.paused] : [])),
+      ]);
+      const mergedRevoked = new Set([
+        ...(pool?.revokedRefs ?? []),
+        ...(Array.isArray(mp.revoked) ? mp.revoked : (mp.revoked instanceof Set ? [...mp.revoked] : [])),
+      ]);
+      const modelExpiresAt = { ...(pool?.expiresAt ?? {}), ...(typeof mp.expiresAt === 'object' ? mp.expiresAt : {}) };
+      const mpool = buildPool(`${p.provider}::${model}`, mp.keys, mp.weights, mergedPaused, mergedRevoked, poolCooldown, poolMax, modelExpiresAt, p.routingStrategy, p.proactiveRateLimitGuard, {
         provider: p.provider,
         model,
         quotas: mp.quotas,
-      }, modelRpm, modelTpm);
+      }, modelRpm, modelTpm, modelConcurrency);
       if (!mpool) continue;
+      mpool.basePool = pool;
       byModel.set(model, mpool);
       for (const ref of mpool.refs) {
         if (!poolByRef.has(ref)) poolByRef.set(ref, mpool);

package/lib/pool.js

@@ -316,6 +316,7 @@ export function isKeyPaused(pool, ref) {
   if (Array.isArray(pool.pausedRefs) && pool.pausedRefs.includes(ref)) return true;
   if (pool.state && pool.state.pausedRefs instanceof Set && pool.state.pausedRefs.has(ref)) return true;
   if (pool.state && Array.isArray(pool.state.pausedRefs) && pool.state.pausedRefs.includes(ref)) return true;
+  if (pool.basePool) return isKeyPaused(pool.basePool, ref);
   return false;
 }
 
@@ -560,5 +561,6 @@ export function isKeyRevoked(pool, ref) {
   if (Array.isArray(pool.revokedRefs) && pool.revokedRefs.includes(ref)) return true;
   if (pool.state && pool.state.revokedRefs instanceof Set && pool.state.revokedRefs.has(ref)) return true;
   if (pool.state && Array.isArray(pool.state.revokedRefs) && pool.state.revokedRefs.includes(ref)) return true;
+  if (pool.basePool) return isKeyRevoked(pool.basePool, ref);
   return false;
 }

package/lib/resolver.js

@@ -1,4 +1,3 @@
-// lib/resolver.js — the credential selector behind credentials.resolve (#381).
 //
 // Extracted from lib/index.js so the routing decisions that this feature
 // depends on — request-scoped pool ownership, model token quota enforcement,
@@ -10,7 +9,7 @@
 // delegates to `original(ref)`.
 
 import { bucketAllow, bucketRetryMs, tpmAllow, tpmRetryMs } from './bucket.js';
-import { envValue, sortAttemptList, isKeyPaused, isKeyRevoked } from './pool.js';
+import { envValue, sortAttemptList, isKeyPaused, isKeyRevoked, costForDay, costForWeek, budgetVerdict } from './pool.js';
 import { decryptSecret } from './crypto-storage.js';
 import { resolveRefPool, isQuotaManagedRequest } from './pool-index.js';
 import {
@@ -22,12 +21,53 @@ import {
 
 /** Internal failure code for "local budgets forbid this dispatch". */
 export const LOCAL_QUOTA_CODE = 'LOCAL_MODEL_QUOTA_EXHAUSTED';
+export const LOCAL_POOL_EXHAUSTED_CODE = 'LOCAL_POOL_EXHAUSTED';
+
+/**
+ * Calculates aggregated daily and weekly spend across base pool and model pools for a provider without double-counting (#422).
+ */
+export function getProviderCost(provider, runtime, now = Date.now()) {
+  const seenStates = new Set();
+  let daily = 0;
+  let weekly = 0;
+  const pools = [];
+  const basePool = runtime?.providerToPool?.get?.(provider);
+  if (basePool) pools.push(basePool);
+  const modelPools = runtime?.modelPoolByProvider?.get?.(provider);
+  if (modelPools) {
+    for (const mp of modelPools.values()) {
+      if (!pools.includes(mp)) pools.push(mp);
+    }
+  }
+  if (runtime?.index?.allPools) {
+    for (const p of runtime.index.allPools) {
+      if ((p.provider === provider || p.base === provider || p.base?.startsWith(provider + '::')) && !pools.includes(p)) {
+        pools.push(p);
+      }
+    }
+  }
+  if (runtime?.poolByRef) {
+    for (const p of runtime.poolByRef.values()) {
+      if ((p.provider === provider || p.base === provider || p.base?.startsWith(provider + '::')) && !pools.includes(p)) {
+        pools.push(p);
+      }
+    }
+  }
+  for (const p of pools) {
+    if (p.state && !seenStates.has(p.state)) {
+      seenStates.add(p.state);
+      daily += costForDay(p.state.costDays);
+      weekly += costForWeek(p.state.costDays, now);
+    }
+  }
+  return { daily, weekly, pools };
+}
 
 /**
  * @param {object} deps
  * @param {() => object} deps.buildRuntime         current runtime snapshot
  * @param {() => object|undefined} deps.currentPool request-scoped pool (ALS)
- * @param {(pool: object, candidate: string) => void} [deps.onPicked] picked-ref hook
+ * @param {(pool: object, candidate: string, meta?: object) => void} [deps.onPicked] picked-ref hook
  * @param {object} [deps.latencyHistogram]
  * @param {object} [deps.concurrencyTracker]
  * @param {() => number} [deps.now] wall clock; injectable for tests
@@ -59,6 +99,22 @@ export function createResolver({
     }
 
     const at = now();
+
+    // Provider monetary budget refusal (#422):
+    const providerName = pool.provider ?? pool.base?.split?.('::')?.[0];
+    const budget = runtime.providerBudgets?.get?.(providerName);
+    if (budget && budget.pauseOnBudget) {
+      const { daily, weekly } = getProviderCost(providerName, runtime, at);
+      const vDaily = budgetVerdict(daily, budget.costBudgetDaily);
+      const vWeekly = budgetVerdict(weekly, budget.costBudgetWeekly);
+      if (vDaily.exceeded || vWeekly.exceeded) {
+        throw Object.assign(
+          new Error(`dsh-key-rotation: provider "${providerName}" cost budget exceeded (daily: $${daily.toFixed(2)}/${budget.costBudgetDaily}, weekly: $${weekly.toFixed(2)}/${budget.costBudgetWeekly})`),
+          { code: LOCAL_POOL_EXHAUSTED_CODE, localExhausted: true },
+        );
+      }
+    }
+
     const strategy = pool.routingStrategy ?? runtime.routingStrategy ?? 'round-robin';
     let list = pool.weightedRefs ?? pool.refs;
     if (strategy === 'lowest-latency' || strategy === 'least-loaded') {
@@ -78,11 +134,9 @@ export function createResolver({
       if (isKeyPaused(pool, candidate)) continue;
       const until = pool.state.failedUntil.get(candidate);
       if (until !== undefined && until > at) continue;
-      if (pool.expiresAt?.[candidate] !== undefined && at >= pool.expiresAt[candidate]) continue;
+      const candidateExp = pool.expiresAt?.[candidate] ?? pool.basePool?.expiresAt?.[candidate];
+      if (candidateExp !== undefined && at >= candidateExp) continue;
       // Local model token budget: a spent credential is simply not a candidate.
-      // This is deliberately NOT recorded as a failure — the key is healthy and
-      // merely out of budget — so it must never touch failedUntil / failCounts /
-      // authFailCounts / brokenUntil.
       if (quotaManaged && !isModelQuotaAvailable(pool, candidate, at)) continue;
       const rpmLimit = pool.rpmLimit ?? 0;
       if (rpmLimit > 0) {
@@ -117,20 +171,46 @@ export function createResolver({
         }
       }
 
+      // Concurrency limit per-key check (#407):
+      const concurrencyLimit = pool.concurrencyLimit ?? runtime.concurrencyLimit ?? 0;
+      if (concurrencyLimit > 0 && concurrencyTracker && typeof concurrencyTracker.getActive === 'function') {
+        if (concurrencyTracker.getActive(candidate, at) >= concurrencyLimit) {
+          continue;
+        }
+      }
+
+      // Atomically acquire concurrency permit upon candidate selection (#407):
+      let acquired = false;
+      if (concurrencyTracker && typeof concurrencyTracker.acquire === 'function') {
+        const ok = concurrencyTracker.acquire(candidate, at, concurrencyLimit);
+        if (!ok) continue;
+        acquired = true;
+      }
+
       // Advance the shared cursor at selection time: concurrent requests must
       // not all pick the same slot.
       pool.state.pointer = (index + 1) % list.length;
-      const hit = await original(candidate);
-      if (hit && typeof hit.value === 'string' && hit.value.length > 0) {
-        settle(pool, candidate, at, onPicked);
-        return { ...hit, value: decryptSecret(hit.value) };
+      try {
+        const hit = await original(candidate);
+        if (hit && typeof hit.value === 'string' && hit.value.length > 0) {
+          settle(pool, candidate, at, onPicked, acquired ? candidate : undefined);
+          return { ...hit, value: decryptSecret(hit.value) };
+        }
+        // No stored credential: fall back to the launching environment, which is
+        // how env-bootstrapped pools work (issue #7).
+        const envVal = envValue(candidate);
+        if (envVal !== undefined) {
+          settle(pool, candidate, at, onPicked, acquired ? candidate : undefined);
+          return { value: decryptSecret(envVal), source: 'env' };
+        }
+      } catch (err) {
+        if (acquired && typeof concurrencyTracker.release === 'function') {
+          concurrencyTracker.release(candidate, at);
+        }
+        throw err;
       }
-      // No stored credential: fall back to the launching environment, which is
-      // how env-bootstrapped pools work (issue #7).
-      const envVal = envValue(candidate);
-      if (envVal !== undefined) {
-        settle(pool, candidate, at, onPicked);
-        return { value: decryptSecret(envVal), source: 'env' };
+      if (acquired && typeof concurrencyTracker.release === 'function') {
+        concurrencyTracker.release(candidate, at);
       }
     }
 
@@ -144,16 +224,19 @@ export function createResolver({
         { code: LOCAL_QUOTA_CODE, localQuota: true },
       );
     }
-    return original(ref);
+    throw Object.assign(
+      new Error(`dsh-key-rotation: all credentials in pool "${pool.base ?? pool.provider}" are currently blocked, paused, expired, or exhausted`),
+      { code: LOCAL_POOL_EXHAUSTED_CODE, localExhausted: true },
+    );
   };
 }
 
 /** Record a successful credential selection on the pool and request scope. */
-function settle(pool, candidate, now, onPicked) {
+function settle(pool, candidate, now, onPicked, concurrencyRef) {
   pool.state.lastUsed = candidate;
   if (!pool.state.lastUsedAt) pool.state.lastUsedAt = new Map();
   pool.state.lastUsedAt.set(candidate, now);
-  onPicked?.(pool, candidate);
+  onPicked?.(pool, candidate, { concurrencyRef });
   pool.state.failCounts?.delete(candidate);
   pool.state.failedUntil.delete(candidate);
   pool.state.authFailCounts?.delete(candidate);

package/lib/rotate.js

@@ -80,7 +80,7 @@ export function createRotate(deps) {
       let lastFailure = null;
       const breaker = runtime0.circuitBreakerEnabled === false ? null : circuitBreaker;
       let activePermit = null;
-      const reqStore = { pool, pickedRef: undefined, startMs: now() };
+      const reqStore = { pool, pickedRef: undefined, concurrencyRef: undefined, startMs: now() };
       setRotateStartMs(reqStore.startMs);
       const wallNow = Date.now();
       const quotaManaged = hasModelQuotaConfig(pool);
@@ -97,7 +97,7 @@ export function createRotate(deps) {
           const maxDepth = runtimeQ.cascadeMaxDepth ?? CASCADE_MAX_DEPTH;
           const currentDepth = options.__cascadeDepth ?? (options.__isCascade ? 1 : 0);
           if (currentDepth < maxDepth && Array.isArray(runtimeQ.cascade) && runtimeQ.cascade.length > 0) {
-            const fb = pickCascadeFallback(options.provider, runtimeQ, runtimeQ.providerToPool, options.model);
+            const fb = pickCascadeFallback(options.provider, runtimeQ, runtimeQ.providerToPool, options.model, runtimeQ.modelPoolByProvider);
             if (fb && fb.pool && fb.pool !== pool) {
               pool.state.lastReason = 'CASCADE';
               pool.state.lastSwitchAt = now();
@@ -210,9 +210,10 @@ export function createRotate(deps) {
               return llm.stream({ ...options, [MARKER]: true });
             });
           } catch (e) {
-            if (e?.localQuota || e?.code === LOCAL_QUOTA_CODE) {
-              lastFailure = finishError(LOCAL_QUOTA_CODE, String(e?.message ?? ''));
-              logWarn(`[dsh-key-rotation] ${options.provider}/${options.model ?? '-'}: local model token budget blocks dispatch — no upstream request sent`);
+            if (e?.localQuota || e?.localExhausted || e?.code === LOCAL_QUOTA_CODE || e?.code === LOCAL_POOL_EXHAUSTED_CODE) {
+              const code = e?.code ?? (e?.localQuota ? LOCAL_QUOTA_CODE : LOCAL_POOL_EXHAUSTED_CODE);
+              lastFailure = finishError(code, String(e?.message ?? ''));
+              logWarn(`[dsh-key-rotation] ${options.provider}/${options.model ?? '-'}: local rejection (${code}) — no upstream request sent`);
               break;
             }
             const curRef = reqStore.pickedRef ?? pool.state.lastUsed;
@@ -224,7 +225,10 @@ export function createRotate(deps) {
           }
 
           const _pickedRef = reqStore.pickedRef ?? pool.state.lastUsed;
-          const acquired = (_pickedRef && runtime0.concurrencyLimit > 0) ? concurrencyTracker.acquire(_pickedRef) : false;
+          if (_pickedRef && !reqStore.concurrencyRef && typeof concurrencyTracker?.acquire === 'function') {
+            const ok = concurrencyTracker.acquire(_pickedRef, now(), pool.concurrencyLimit ?? runtime0.concurrencyLimit ?? 0);
+            if (ok) reqStore.concurrencyRef = _pickedRef;
+          }
           try {
             try {
               for await (const chunk of preserveDispatchContext(inner, dispatchStorage, reqStore)) {
@@ -397,11 +401,12 @@ export function createRotate(deps) {
             }
             return; // clean end — served
           } finally {
-            if (acquired && _pickedRef) {
-              concurrencyTracker.release(_pickedRef);
-            }
           }
         } finally {
+          if (reqStore.concurrencyRef) {
+            concurrencyTracker?.release(reqStore.concurrencyRef);
+            reqStore.concurrencyRef = null;
+          }
           // Also runs on consumer return(), abort, empty EOF and local errors.
           activePermit?.release();
         }
@@ -420,7 +425,7 @@ export function createRotate(deps) {
       const currentDepth = options.__cascadeDepth ?? (options.__isCascade ? 1 : 0);
       if (currentDepth < maxDepth && Array.isArray(runtime.cascade) && runtime.cascade.length > 0) {
         const pools = runtime.providerToPool;
-        const fb = pickCascadeFallback(options.provider, runtime, pools, options.model);
+        const fb = pickCascadeFallback(options.provider, runtime, pools, options.model, runtime.modelPoolByProvider);
         if (fb && fb.pool && fb.pool !== pool) {
           logWarn(`[dsh-key-rotation] ${options.provider}: pool exhausted — cascading to ${fb.provider} (model: ${fb.model || options.model || 'default'})`);
           pool.state.lastReason = 'CASCADE';

package/lib/safe-fetch.js

@@ -1,20 +1,57 @@
-// lib/safe-fetch.js — SSRF-protected HTTP fetch with DNS and IP validation (#354).
+// lib/safe-fetch.js — SSRF-protected HTTP fetch with DNS and IP validation (#354, #411).
 import net from 'node:net';
 import dns from 'node:dns/promises';
+import nodeDns from 'node:dns';
+import { Agent } from 'undici';
 
 export const MAX_IMPORT_BYTES = 1024 * 1024; // 1 MB limit
 export const MAX_REDIRECTS = 5;
 
+/**
+ * Parses an IPv6 string into an array of 8 16-bit integers (words).
+ * Supports standard hex, compressed `::`, and embedded dotted IPv4 (e.g. `::ffff:127.0.0.1`).
+ * @param {string} ip
+ * @returns {number[]|null}
+ */
+export function parseIpv6Words(ip) {
+  if (typeof ip !== 'string') return null;
+  let s = ip.toLowerCase().trim().replace(/^\[|\]$/g, '');
+  const lastColon = s.lastIndexOf(':');
+  if (lastColon !== -1) {
+    const tail = s.slice(lastColon + 1);
+    if (net.isIP(tail) === 4) {
+      const parts = tail.split('.').map(Number);
+      if (parts.length === 4 && !parts.some(n => isNaN(n) || n < 0 || n > 255)) {
+        const w6 = ((parts[0] << 8) | parts[1]) & 0xffff;
+        const w7 = ((parts[2] << 8) | parts[3]) & 0xffff;
+        s = s.slice(0, lastColon) + ':' + w6.toString(16) + ':' + w7.toString(16);
+      }
+    }
+  }
+  const halves = s.split('::');
+  if (halves.length > 2) return null;
+  const left = halves[0] ? halves[0].split(':').map(x => parseInt(x, 16)) : [];
+  const right = halves.length === 2 && halves[1] ? halves[1].split(':').map(x => parseInt(x, 16)) : [];
+  if (left.some(isNaN) || right.some(isNaN)) return null;
+  const missing = 8 - (left.length + right.length);
+  if (halves.length === 1 && missing !== 0) return null;
+  if (missing < 0) return null;
+  const middle = new Array(missing).fill(0);
+  return [...left, ...middle, ...right];
+}
+
 /**
  * Checks if an IPv4 or IPv6 address is private, loopback, link-local, multicast, or reserved.
+ * Canonicalizes IPv6 addresses including hex and dotted-quad IPv4-mapped forms (#411).
  * @param {string} ip
  * @returns {boolean} true if private/reserved/loopback
  */
 export function isPrivateOrReservedIp(ip) {
   if (!ip || typeof ip !== 'string') return true;
-  const kind = net.isIP(ip);
+  const cleaned = ip.trim().replace(/^\[|\]$/g, '');
+  const kind = net.isIP(cleaned);
   if (kind === 4) {
-    const parts = ip.split('.').map(Number);
+    const parts = cleaned.split('.').map(Number);
     if (parts.length !== 4 || parts.some(n => isNaN(n) || n < 0 || n > 255)) return true;
     const [a, b, c, d] = parts;
     if (a === 0) return true; // 0.0.0.0/8 current network
@@ -33,22 +70,70 @@ export function isPrivateOrReservedIp(ip) {
     return false;
   }
   if (kind === 6) {
-    const norm = ip.toLowerCase();
-    if (norm === '::' || norm === '::1' || norm === '0:0:0:0:0:0:0:0' || norm === '0:0:0:0:0:0:0:1') return true;
-    if (norm.startsWith('::ffff:') || norm.startsWith('0:0:0:0:0:ffff:')) {
-      const last = norm.split(':').pop();
-      if (net.isIP(last) === 4) return isPrivateOrReservedIp(last);
+    const words = parseIpv6Words(cleaned);
+    if (!words || words.length !== 8) return true;
+    const [w0, w1, w2, w3, w4, w5, w6, w7] = words;
+    // Unspecified :: or loopback ::1
+    if (w0 === 0 && w1 === 0 && w2 === 0 && w3 === 0 && w4 === 0 && w5 === 0 && w6 === 0) {
+      if (w7 === 0 || w7 === 1) return true;
+    }
+    // IPv4-mapped (::ffff:0:0/96), IPv4-compatible (::/96), or NAT64 (64:ff9b::/96)
+    const isMapped = (w0 === 0 && w1 === 0 && w2 === 0 && w3 === 0 && w4 === 0 && w5 === 0xffff);
+    const isCompat = (w0 === 0 && w1 === 0 && w2 === 0 && w3 === 0 && w4 === 0 && w5 === 0);
+    const isNat64 = (w0 === 0x0064 && w1 === 0xff9b && w2 === 0 && w3 === 0 && w4 === 0 && w5 === 0);
+    if (isMapped || isCompat || isNat64) {
+      const a = (w6 >> 8) & 0xff;
+      const b = w6 & 0xff;
+      const c = (w7 >> 8) & 0xff;
+      const d = w7 & 0xff;
+      return isPrivateOrReservedIp(`${a}.${b}.${c}.${d}`);
     }
-    if (/^fe[89ab][0-9a-f]:/i.test(norm)) return true; // fe80::/10 link-local
-    if (/^f[cd][0-9a-f]{2}:/i.test(norm)) return true; // fc00::/7 unique local
-    if (/^ff[0-9a-f]{2}:/i.test(norm)) return true; // ff00::/8 multicast
-    if (/^(?:00)?2001:(?:0)?db8:/i.test(norm)) return true; // 2001:db8::/32 documentation
-    if (/^(?:00)?100:/i.test(norm)) return true; // 100::/64 discard
+    if ((w0 & 0xffc0) === 0xfe80) return true; // fe80::/10 link-local
+    if ((w0 & 0xfe00) === 0xfc00) return true; // fc00::/7 unique local
+    if ((w0 & 0xff00) === 0xff00) return true; // ff00::/8 multicast
+    if (w0 === 0x2001 && w1 === 0x0db8) return true; // 2001:db8::/32 documentation
+    if (w0 === 0x0100 && w1 === 0) return true; // 100::/64 discard
     return false;
   }
   return true;
 }
 
+/**
+ * Creates an undici Agent with connect-time lookup validation against DNS rebinding / TOCTOU (#411).
+ * @param {object} [opts]
+ * @param {Function} [opts.lookupImpl]
+ * @returns {Agent}
+ */
+export function createSafeDispatcher({ lookupImpl = nodeDns.lookup } = {}) {
+  return new Agent({
+    connect: {
+      lookup: (hostname, options, callback) => {
+        const cb = typeof options === 'function' ? options : callback;
+        const opts = typeof options === 'function' ? {} : options;
+        lookupImpl(hostname, { ...opts, all: true }, (err, addresses) => {
+          if (err) return cb(err);
+          const list = Array.isArray(addresses) ? addresses : [addresses];
+          for (const item of list) {
+            const addr = typeof item === 'string' ? item : item.address;
+            if (isPrivateOrReservedIp(addr)) {
+              const ssrfErr = new Error(`dsh-key-rotation: connect blocked: host ${hostname} resolves to private or reserved IP ${addr}`);
+              ssrfErr.code = 'SSRF_BLOCKED';
+              return cb(ssrfErr);
+            }
+          }
+          if (typeof options === 'function') {
+            const first = list[0];
+            const addr = typeof first === 'string' ? first : first.address;
+            const fam = typeof first === 'object' && first.family ? first.family : (net.isIP(addr) || 4);
+            return cb(null, addr, fam);
+          }
+          return cb(null, addresses);
+        });
+      },
+    },
+  });
+}
+
 /**
  * Validates a target URL and its hostname/IP against SSRF.
  * @param {string} urlString
@@ -173,7 +258,7 @@ export async function readBoundedJson(resp, maxBytes = MAX_IMPORT_BYTES) {
 }
 
 /**
- * Fetches JSON from a URL with SSRF protection, redirect verification, and size bounding.
+ * Fetches JSON from a URL with SSRF protection, connect-time rebinding guard, redirect verification, and size bounding.
  * @param {string} url
  * @param {object} options
  */
@@ -183,14 +268,18 @@ export async function safeFetchJson(url, {
   lookupImpl = dns.lookup,
   maxBytes = MAX_IMPORT_BYTES,
   redirectsLeft = MAX_REDIRECTS,
+  dispatcher,
 } = {}) {
+  const safeDispatcher = dispatcher ?? (fetchImpl === globalThis.fetch ? createSafeDispatcher() : undefined);
   let currentUrl = url;
   while (true) {
     await assertSafeUrl(currentUrl, { lookupImpl });
-    const resp = await fetchImpl(currentUrl, {
+    const fetchOpts = {
       signal,
       redirect: 'manual',
-    });
+    };
+    if (safeDispatcher) fetchOpts.dispatcher = safeDispatcher;
+    const resp = await fetchImpl(currentUrl, fetchOpts);
 
     if ([301, 302, 303, 307, 308].includes(resp.status)) {
       if (redirectsLeft <= 0) {
@@ -218,4 +307,4 @@ export async function safeFetchJson(url, {
 
     return await readBoundedJson(resp, maxBytes);
   }
-}
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@goodandready/dsh-key-rotation",
-  "version": "0.8.38",
+  "version": "0.8.40",
   "packageManager": "pnpm@10.33.2",
   "description": "Per-provider API key rotation for DeepSeek Harness: a key pool per provider, auto-created clone routes, and switching to the next key on quota/rate-limit errors. Includes a Settings section (Key Rotation) to edit the key pools, cooldown and switch codes.",
   "keywords": [
@@ -57,8 +57,8 @@
   },
   "peerDependencies": {
     "@deepseek-ai/cordis": "^4.0.1",
-    "@deepseek-ai/schemastery": "^3.18.4",
-    "@deepseek-ai/dsh-llm": "^0.1.7-rc.2 || ^0.2.0-rc.1"
+    "@deepseek-ai/dsh-llm": "^0.1.7-rc.2 || ^0.2.0-rc.1",
+    "@deepseek-ai/schemastery": "^3.18.4"
   },
   "scripts": {
     "test": "node tools/test.mjs",
@@ -70,5 +70,8 @@
   "devDependencies": {
     "react": "18.3.1",
     "react-test-renderer": "18.3.1"
+  },
+  "dependencies": {
+    "undici": "^6.29.0"
   }
 }