@goodandready/dsh-subscriptions 0.6.69 → 0.6.71

package/README.md

@@ -207,6 +207,13 @@ const res = await ctx.subscriptions.request('codex', '/backend-api/codex/images/
 * **In-App Toast Alerts (`enableToasts`)**: Lightweight floating toast alerts in the DSH chat interface notifying you when accounts enter quarantine, recover from cooldowns, or exceed quota safety thresholds.
 * **Automated Webhook Integration**: Instant event notifications dispatched via `notifyQuarantineEntered` and `notifyQuarantineReleased`.
 
+### 24. 🛡️ Resilience, Diagnostics & Fallback Enhancements (v0.6.71)
+* **Cross-Vendor Cascading Fallback & Provider Priority Ordering (#536)**: Configure fallback ordering across subscription providers with interactive reordering in WebUI.
+* **1-Click Connection Smoke Test & Latency Ping (#536)**: Test individual account connections on the spot with real-time latency badges.
+* **Live Cooldown Countdown & Availability Toasts (#536)**: Exact countdown timer for cooled-down accounts and automated toast alerts upon recovery.
+* **Proactive OAuth Token Heartbeat (#536)**: Pre-emptive background token refresh before expiry to prevent first-query latency spikes.
+* **Recent Errors Ring Buffer Inspector (#536)**: In-memory sanitized log of the last 20 runtime failures in WebUI Diagnostics.
+
 ### 23. 🧠 Antigravity Tiered Models, Tool Sentinel & Stream Resilience (v0.6.69)
 * **Gemini 3 Tool Replay Sentinel (#534)**: Injects `thoughtSignature: 'skip_thought_signature_validator'` on reconstructed tool calls, enabling reliable multi-turn tool calling without Gemini 3 validation failures.
 * **Tiered Flash Models & Thinking Levels (#534)**: Exposes Gemini 3.8 and 3.7 Flash High, Medium, and Low thinking levels with upstream `*-flash-tiered` mapping.

package/README.ru.md

@@ -183,6 +183,13 @@ const res = await ctx.subscriptions.request('codex', '/backend-api/codex/images/
 * **Всплывающие уведомления (Toast Alerts)**: Легковесные всплывающие тосты в веб-интерфейсе DSH при событиях карантина (с отображением причины), восстановлении доступа и критическом расходе квоты. Включение управляется параметром `enableToasts` (по умолчанию `true`).
 * **Интеграция с вебхуками**: Автоматическая отправка уведомлений через `notifyQuarantineEntered` и `notifyQuarantineReleased`.
 
+### 20. 🛡️ Отказоустойчивость, диагностика и порядок фолбэка (v0.6.71)
+* **Межвендорный каскадный фолбэк с настройкой очередности (#536)**: Настройка порядка переключения провайдеров при исчерпании квот с интерактивным изменением приоритета (▲/▼) в WebUI.
+* **1-Click проверка подключения и пинг задержки (#536)**: Проверка конкретного аккаунта в один клик с отображением задержки (`✓ 240ms`).
+* **Живой таймер кулдауна и уведомления о восстановлении (#536)**: Точный обратный отсчет кулдауна и всплывающие in-app тосты при возвращении аккаунта в строй.
+* **Превентивный OAuth Heartbeat (#536)**: Упреждающее фоновое обновление токенов до истечения срока действия.
+* **Журнал последних ошибок (Ring Buffer) (#536)**: Встроенный инспектор последних 20 ошибок в WebUI Diagnostics с кнопкой очистки.
+
 ### 19. 🧠 Antigravity Tiered-модели, сентинел тулов и устойчивость стримов (v0.6.69)
 * **Сентинел реплея тулов Gemini 3 (#534)**: Автоматическая вставка `thoughtSignature: 'skip_thought_signature_validator'` для реплея истории вызовов тулов без сбоев валидации Google API.
 * **Tiered-модели и уровни мышления (#534)**: Выбор уровней High/Medium/Low для Gemini 3.8 и 3.7 Flash с маппингом в upstream `*-flash-tiered` и `thinkingLevel`.

package/README.zh.md

@@ -132,6 +132,13 @@ graph LR
   queueTimeoutMs: 30000     # v0.6.62: 队列最大等待超时时间（毫秒）
 * **自动化 Webhook 告警集成**：通过 `notifyQuarantineEntered` 与 `notifyQuarantineReleased` 自动向外部通知系统派发隔离生命周期事件。
 
+### 🛡️ 弹性容灾、故障诊断与跨服务商回退 (v0.6.71)
+* **跨服务商级联故障转移与优先级排序 (#536)**: 支持自定义跨服务商回退顺序，可在 WebUI 中通过 ▲/▼ 交互调整优先级。
+* **一键链路测试与延迟检测 (#536)**: 在账户列表或状态弹窗中一键发起探活并显示往返延迟。
+* **冷却倒计时与就绪通知 (#536)**: 实时展示受限账户恢复倒计时，解除冷却时自动弹出就绪通知。
+* **主动 OAuth 令牌心跳 (#536)**: 提前预刷即将过期的访问令牌，避免用户请求时产生额外等待。
+* **最近错误环形日志审查 (#536)**: 内存保存最近 20 条已脱敏的运行错误，支持在 Diagnostics 中查看与清空。
+
 ### Antigravity 分档思考模型、工具回放哨兵与流式自愈 (v0.6.69)
 * **Gemini 3 工具回放哨兵 (#534)**：在重建的工具调用中注入 `thoughtSignature: 'skip_thought_signature_validator'`，彻底解决多轮工具调用历史重放时 Gemini 3 的校验失败问题。
 * **分档 Flash 模型与思考档位 (#534)**：将 `gemini-3.8-flash-tiered` 与 `3.7` 自动解析展开为 High/Medium/Low 档位选择，并在调用时自动组装 `thinkingConfig.thinkingLevel`。

package/lib/adapter-manager.js

@@ -45,6 +45,8 @@ export function createAdapterManager({
     drainBeforeReset: () => live().drainBeforeReset,
     drainSafetyPercent: () => live().drainSafetyPercent,
     switchAtRemaining: () => live().switchAtRemaining,
+    rotationStrategy: () => live().rotationStrategy,
+    enableSoftFailureSentinel: () => live().enableSoftFailureSentinel,
     rememberCooldown: (ref, until, families) => store.rememberCooldown(ref, until, families),
     rememberQuarantine: (ref, reason, until, attempts) => store.rememberQuarantine(ref, reason, until, attempts),
     getQuarantine: (ref) => store.getQuarantine(ref),
@@ -75,7 +77,7 @@ export function createAdapterManager({
     const loggedList = Array.isArray(logged) ? logged : Object.keys(logged || {}).filter((p) => logged[p])
     const available = loggedList.filter((p) => p !== provider)
     visited.add(provider)
-    const nextVendor = resolveFallbackVendor(provider, available, cfg.cascadingChain, visited)
+    const nextVendor = resolveFallbackVendor(provider, available, cfg.cascadingChain, visited, cfg.cascadingFallbackOrder)
     if (!nextVendor) throw err
     visited.add(nextVendor)
     const targetModel = mapFallbackModel(provider, options && options.model, nextVendor)

package/lib/adapter.js

@@ -142,6 +142,12 @@ export class SubscriptionAdapter extends LlmAdapter {
       const effectiveDrainSafetyPercent = options.drainSafetyPercent !== undefined
         ? options.drainSafetyPercent
         : (typeof deps.drainSafetyPercent === 'function' ? deps.drainSafetyPercent() : (deps.drainSafetyPercent ?? 5))
+      const effectiveRotationStrategy = options.rotationStrategy !== undefined
+        ? options.rotationStrategy
+        : (typeof deps.rotationStrategy === 'function' ? deps.rotationStrategy() : (deps.rotationStrategy ?? 'round-robin'))
+      const effectiveSoftFailureSentinel = options.enableSoftFailureSentinel !== undefined
+        ? options.enableSoftFailureSentinel
+        : (typeof deps.enableSoftFailureSentinel === 'function' ? deps.enableSoftFailureSentinel() : (deps.enableSoftFailureSentinel ?? true))
       const sessionId = options.sessionId || options.session_id || options.conversationId
       const effectiveReasoning = resolveEffectiveReasoning(sessionId, options)
       const decisionTrace = {}
@@ -153,6 +159,8 @@ export class SubscriptionAdapter extends LlmAdapter {
         autoPacing: effectiveAutoPacing,
         drainBeforeReset: effectiveDrainBeforeReset,
         drainSafetyPercent: effectiveDrainSafetyPercent,
+        rotationStrategy: effectiveRotationStrategy,
+        enableSoftFailureSentinel: effectiveSoftFailureSentinel,
       }
       for await (const event of streamWithRotation({
         accounts: await deps.listAccounts(provider),

package/lib/cascade.js

@@ -29,10 +29,35 @@ export const DEFAULT_CASCADE_CHAINS = {
  * @param {Set<string>} [visited] - set of already attempted providers in this request cascade
  * @returns {string|null} - the next candidate provider or null if none available
  */
-export function resolveFallbackVendor(currentProvider, availableProviders, customChains = {}, visited = new Set()) {
+/**
+ * Resolves the next candidate fallback provider from the cascade chain.
+ * @param {string} currentProvider - currently failing provider
+ * @param {Set<string>|Array<string>} availableProviders - providers that currently have logged-in accounts
+ * @param {Record<string, string[]>} [customChains] - user-configured custom chains
+ * @param {Set<string>} [visited] - set of already attempted providers in this request cascade
+ * @param {string[]} [globalOrder] - configured global fallback priority sequence
+ * @returns {string|null} - the next candidate provider or null if none available
+ */
+export function resolveFallbackVendor(currentProvider, availableProviders, customChains = {}, visited = new Set(), globalOrder = []) {
   const availSet = new Set(availableProviders || [])
-  const chain = (customChains && customChains[currentProvider]) || DEFAULT_CASCADE_CHAINS[currentProvider] || []
+  const custom = customChains && customChains[currentProvider]
+  if (Array.isArray(custom) && custom.length > 0) {
+    for (const candidate of custom) {
+      if (candidate !== currentProvider && !visited.has(candidate) && availSet.has(candidate)) {
+        return candidate
+      }
+    }
+  }
+
+  if (Array.isArray(globalOrder) && globalOrder.length > 0) {
+    for (const candidate of globalOrder) {
+      if (candidate !== currentProvider && !visited.has(candidate) && availSet.has(candidate)) {
+        return candidate
+      }
+    }
+  }
 
+  const chain = DEFAULT_CASCADE_CHAINS[currentProvider] || []
   for (const candidate of chain) {
     if (candidate !== currentProvider && !visited.has(candidate) && availSet.has(candidate)) {
       return candidate

package/lib/client.js

@@ -430,6 +430,24 @@ const cssId = 'dsh-subscriptions/settings.module.css'
       "smokeTesting": "Pinging…",
       "smokeOk": "Smoke test passed",
       "smokeFail": "Smoke test failed",
+      "testConnection": "Test",
+      "testConnectionSuccess": "Connection verified",
+      "cascadingFallback": "Cross-vendor fallback",
+      "cascadingFallbackHint": "Automatically route queries to alternative providers when all accounts of a primary provider are exhausted.",
+      "cascadingFallbackOrder": "Fallback Provider Order",
+      "cascadingFallbackOrderHint": "Sequence of alternative providers to try when primary accounts are exhausted.",
+      "rotationStrategy": "Account Balancing Strategy",
+      "rotationStrategyHint": "Method used to distribute requests among available accounts of a provider.",
+      "rotationRoundRobin": "Round-Robin (Balanced rotation)",
+      "rotationLeastLoaded": "Least-Loaded (Fair-share quota)",
+      "rotationPriority": "Priority (Sequential by index/weight)",
+      "softFailureSentinel": "Soft-Failure & Refusal Sentinel",
+      "softFailureSentinelHint": "Detects HTTP 200 capacity apologies and auto-cascades to the next healthy account before streaming.",
+      "recentErrors": "Recent Runtime Errors",
+      "recentErrorsDesc": "Last 20 runtime failures across model streams and providers (sanitized).",
+      "recentErrorsClear": "Clear Log",
+      "recentErrorsEmpty": "No recent errors recorded",
+      "toastCooldownRecovered": "is now available (cooldown expired)",
       "modelsCatalogTitle": "Curated Subscription Models",
       "modelsCatalogDesc": "Available model catalog across active subscriptions in DeepSeek Harness.",
       "rotationDecisionsTitle": "Rotation & Routing Decisions",
@@ -702,6 +720,24 @@ const cssId = 'dsh-subscriptions/settings.module.css'
       "smokeTesting": "正在探活…",
       "smokeOk": "冒烟测试成功",
       "smokeFail": "冒烟测试失败",
+      "testConnection": "测试",
+      "testConnectionSuccess": "连接已验证",
+      "cascadingFallback": "跨服务商自动故障转移",
+      "cascadingFallbackHint": "当首选服务商所有账户配额耗尽或限流时，自动无缝转接至备选服务商。",
+      "cascadingFallbackOrder": "故障转移供应商顺序",
+      "cascadingFallbackOrderHint": "首选服务商耗尽时按此顺序依次尝试备选服务商。",
+      "rotationStrategy": "账户负载均衡策略",
+      "rotationStrategyHint": "在同一服务商的多个可用账户之间分发请求的方式。",
+      "rotationRoundRobin": "轮询均衡（标准交替）",
+      "rotationLeastLoaded": "最少负载优先（配额公平共享）",
+      "rotationPriority": "优先级顺序（按序号/权重顺序）",
+      "softFailureSentinel": "软性拒绝与容量哨兵",
+      "softFailureSentinelHint": "自动识别 HTTP 200 状态下的服务商容量道歉并无缝转接至下一个正常账户。",
+      "recentErrors": "最近运行错误记录",
+      "recentErrorsDesc": "最近 20 条流式传输与 API 调用失败记录（已脱敏）。",
+      "recentErrorsClear": "清空记录",
+      "recentErrorsEmpty": "暂无错误记录",
+      "toastCooldownRecovered": "已解除冷却恢复可用",
       "modelsCatalogTitle": "精选订阅模型",
       "modelsCatalogDesc": "当前 DeepSeek Harness 活跃订阅下可用模型一览。",
       "rotationDecisionsTitle": "轮询调度与路由决策",
@@ -987,7 +1023,13 @@ const cssId = 'dsh-subscriptions/settings.module.css'
       if (account.quarantineReason && account.quarantineUntil && account.quarantineUntil > Date.now()) {
         return account.quarantineReason === 'VERIFY' ? t('verifyAccount') : String(account.quarantineReason)
       }
-      if (account.cooldownUntil && account.cooldownUntil > Date.now()) return t('coolingDown')
+      if (account.cooldownUntil && account.cooldownUntil > Date.now()) {
+        const leftSec = Math.max(0, Math.ceil((account.cooldownUntil - Date.now()) / 1000))
+        const m = Math.floor(leftSec / 60)
+        const s = leftSec % 60
+        const pad = (n) => String(n).padStart(2, "0")
+        return t("coolingDown") + " (" + pad(m) + ":" + pad(s) + ")"
+      }
       if (account.usagePercent != null && account.usagePercent >= 100) return t('usageFull')
       return t('connected')
     }
@@ -1114,12 +1156,14 @@ const cssId = 'dsh-subscriptions/settings.module.css'
         fetch('/dsh-subscriptions/diagnostics', { cache: 'no-store' })
           .then((r) => r.json())
           .then((d) => {
+            if (d && d.ok && d.report) setDiagReport(d.report)
             const txt = d && d.ok ? JSON.stringify(d.report, null, 2) : ('error: ' + ((d && d.error && d.error.message) || 'unknown'))
             setDiag(txt)
             try { navigator.clipboard.writeText(txt) } catch { /* clipboard write disallowed */ }
           })
           .catch((e) => setDiag('error: ' + String(e && e.message || e)))
       }
+      const [diagReport, setDiagReport] = React.useState(null)
       const [checkRes, setCheckRes] = React.useState({})
       const [hostPing, setHostPing] = React.useState({ ok: true, latencyMs: null })
       const [telemetry, setTelemetry] = React.useState(null)
@@ -1259,6 +1303,10 @@ const cssId = 'dsh-subscriptions/settings.module.css'
           .then((r) => r.json())
           .then((d) => { if (d && d.ok && d.queue) setQueueStats(d.queue) })
           .catch(() => {})
+        fetch('/dsh-subscriptions/diagnostics', { cache: 'no-store' })
+          .then((r) => r.json())
+          .then((d) => { if (d && d.ok && d.report) setDiagReport(d.report) })
+          .catch(() => {})
       }
 
       const runSmokeTest = async (provider, index) => {
@@ -2081,6 +2129,75 @@ const cssId = 'dsh-subscriptions/settings.module.css'
                 React.createElement('span', null, t('notifyLimits')),
               ),
               React.createElement('div', { className: 'dsub-sub' }, t('notifyLimitsHint')),
+              React.createElement('label', { className: 'dsub-row', style: { marginTop: 10 } },
+                React.createElement('input', {
+                  type: 'checkbox',
+                  checked: Boolean(draft.cascadingFallback),
+                  onChange: (e) => setDraft((d) => Object.assign({}, d, { cascadingFallback: e.target.checked })),
+                }),
+                React.createElement('span', null, t('cascadingFallback')),
+              ),
+              React.createElement('div', { className: 'dsub-sub' }, t('cascadingFallbackHint')),
+              Boolean(draft.cascadingFallback) ? React.createElement('div', { style: { marginTop: 8, padding: '8px 12px', background: 'var(--dsw-alias-bg-layer-2)', borderRadius: 8 } },
+                React.createElement('div', { style: { fontSize: 12, fontWeight: 600, marginBottom: 4 } }, t('cascadingFallbackOrder')),
+                React.createElement('div', { className: 'dsub-sub', style: { marginBottom: 8 } }, t('cascadingFallbackOrderHint')),
+                React.createElement('div', { style: { display: 'flex', flexDirection: 'column', gap: 4 } },
+                  (Array.isArray(draft.cascadingFallbackOrder) && draft.cascadingFallbackOrder.length ? draft.cascadingFallbackOrder : (providers.length ? providers : ['antigravity', 'claude', 'codex', 'grok', 'copilot', 'cursor'])).map((p, idx, arr) => {
+                    return React.createElement('div', { key: p, style: { display: 'flex', alignItems: 'center', justifyContent: 'space-between', padding: '4px 8px', background: 'var(--dsw-alias-bg-layer-1)', borderRadius: 6, fontSize: 12 } },
+                      React.createElement('span', null, (idx + 1) + '. ' + p.toUpperCase()),
+                      React.createElement('div', { style: { display: 'flex', gap: 4 } },
+                        idx > 0 ? React.createElement('button', {
+                          type: 'button', className: 'dsub-mini', style: { padding: '1px 6px', fontSize: 10 },
+                          onClick: () => {
+                            const next = arr.slice()
+                            const tmp = next[idx - 1]
+                            next[idx - 1] = next[idx]
+                            next[idx] = tmp
+                            setDraft((d) => Object.assign({}, d, { cascadingFallbackOrder: next }))
+                          }
+                        }, '▲') : null,
+                        idx < arr.length - 1 ? React.createElement('button', {
+                          type: 'button', className: 'dsub-mini', style: { padding: '1px 6px', fontSize: 10 },
+                          onClick: () => {
+                            const next = arr.slice()
+                            const tmp = next[idx + 1]
+                            next[idx + 1] = next[idx]
+                            next[idx] = tmp
+                            setDraft((d) => Object.assign({}, d, { cascadingFallbackOrder: next }))
+                          }
+                        }, '▼') : null,
+                      )
+                    )
+                  })
+                )
+              ) : null,
+            ),
+            React.createElement('div', { className: 'dsub-row', style: { marginTop: 12 } },
+              React.createElement('label', { style: { display: 'flex', flexDirection: 'column', gap: 4 } },
+                React.createElement('span', { style: { fontWeight: 600 } }, t('rotationStrategy')),
+                React.createElement('span', { className: 'dsub-sub' }, t('rotationStrategyHint')),
+                React.createElement('select', {
+                  className: 'dsub-select',
+                  value: draft.rotationStrategy || 'round-robin',
+                  onChange: (e) => setDraft((d) => Object.assign({}, d, { rotationStrategy: e.target.value })),
+                  style: { maxWidth: 320, marginTop: 4 }
+                },
+                  React.createElement('option', { value: 'round-robin' }, t('rotationRoundRobin')),
+                  React.createElement('option', { value: 'least-loaded' }, t('rotationLeastLoaded')),
+                  React.createElement('option', { value: 'priority' }, t('rotationPriority')),
+                ),
+              ),
+            ),
+            React.createElement('div', { className: 'dsub-row', style: { marginTop: 12 } },
+              React.createElement('label', { style: { display: 'flex', alignItems: 'center', gap: 8, cursor: 'pointer' } },
+                React.createElement('input', {
+                  type: 'checkbox',
+                  checked: draft.enableSoftFailureSentinel !== false,
+                  onChange: (e) => setDraft((d) => Object.assign({}, d, { enableSoftFailureSentinel: e.target.checked })),
+                }),
+                React.createElement('span', null, t('softFailureSentinel')),
+              ),
+              React.createElement('div', { className: 'dsub-sub' }, t('softFailureSentinelHint')),
             ),
           ),
         ),
@@ -2105,6 +2222,39 @@ const cssId = 'dsh-subscriptions/settings.module.css'
             }, t('diagCopy')),
             React.createElement('pre', { className: 'dsub-diag' }, diag),
           ) : null,
+          React.createElement('details', { className: 'dsub-details', style: { marginTop: 12 } },
+            React.createElement('summary', { className: 'dsub-summary' }, '⚠️ ' + t('recentErrors')),
+            React.createElement('div', { className: 'dsub-block' },
+              React.createElement('div', { style: { display: 'flex', justifyContent: 'space-between', alignItems: 'center', marginBottom: 8 } },
+                React.createElement('span', { className: 'dsub-sub' }, t('recentErrorsDesc')),
+                React.createElement('button', {
+                  type: 'button', className: 'dsub-mini',
+                  onClick: async () => {
+                    await fetch('/dsh-subscriptions/diagnostics/errors/clear', { method: 'POST' }).catch(() => {})
+                    genDiag()
+                  }
+                }, t('recentErrorsClear'))
+              ),
+              (diagReport && Array.isArray(diagReport.recentErrors) && diagReport.recentErrors.length ? diagReport.recentErrors : []).length === 0
+                ? React.createElement('div', { className: 'dsub-dim', style: { padding: 8 } }, t('recentErrorsEmpty'))
+                : React.createElement('div', { style: { display: 'flex', flexDirection: 'column', gap: 6 } },
+                    (diagReport.recentErrors || []).map((err, i) => React.createElement('div', {
+                      key: i,
+                      style: { padding: '6px 10px', background: 'var(--dsw-alias-bg-layer-2)', borderRadius: 6, fontSize: 11.5, borderLeft: '3px solid var(--dsw-alias-state-error-primary)' }
+                    },
+                      React.createElement('div', { style: { display: 'flex', justifyContent: 'space-between', marginBottom: 2 } },
+                        React.createElement('span', { style: { fontWeight: 600 } }, (err.provider || 'unknown').toUpperCase() + (err.model ? ' · ' + err.model : '')),
+                        React.createElement('span', { className: 'dsub-dim' }, (err.ts || '').slice(11, 19))
+                      ),
+                      React.createElement('div', { style: { display: 'flex', gap: 6, alignItems: 'center' } },
+                        err.code ? React.createElement('span', { className: 'dsub-pillTag', style: { background: 'color-mix(in srgb,var(--dsw-alias-state-error-primary) 15%,transparent)', color: 'var(--dsw-alias-state-error-primary)' } }, err.code) : null,
+                        err.status ? React.createElement('span', { className: 'dsub-dim' }, 'HTTP ' + err.status) : null,
+                        React.createElement('span', { style: { opacity: 0.9, wordBreak: 'break-word' } }, err.message)
+                      )
+                    ))
+                  )
+            )
+          ),
           wizardModal ? React.createElement('div', { className: 'dsub-modalWrap', onClick: (e) => { if (e.target === e.currentTarget) setWizardModal(null) } },
             React.createElement('div', { className: 'dsub-modal' },
               React.createElement('div', { className: 'dsub-modalHead' },
@@ -2269,8 +2419,16 @@ const cssId = 'dsh-subscriptions/settings.module.css'
                   React.createElement('button', {
                     type: 'button', className: 'dsub-mini',
                     disabled: checking[key],
+                    title: t('smokeDesc'),
                     onClick: () => doCheck(row.slot.provider, row.slot.index).catch((e) => setErr(cleanErrorMessage(e.message || e))),
-                  }, checking[key] ? t('checking') : t('check')),
+                  }, checking[key] ? t('checking') : ('⚡ ' + t('testConnection'))),
+                  (function(){
+                    var r = checkRes[key];
+                    if (!r) return null;
+                    var cls = r.ok ? 'dsub-badge-ok' : 'dsub-badge-bad';
+                    var txt = r.ok ? ('✓ ' + (r.latencyMs != null ? r.latencyMs + 'ms' : 'OK')) : ('✗ ' + ((r.error && (r.error.code || r.error.message)) || 'FAIL'));
+                    return React.createElement('span', { className: 'dsub-badge ' + cls, style: { marginLeft: 4 }, title: r.ok ? t('testConnectionSuccess') : ((r.error && r.error.message) || 'fail') }, txt);
+                  })(),
                   React.createElement('button', {
                     type: 'button', className: 'dsub-mini',
                     title: t('wizardTitle'),
@@ -3141,6 +3299,24 @@ const cssId = 'dsh-subscriptions/settings.module.css'
         )
       }
 
+      const prevCooldownsRef = React.useRef(new Map())
+      React.useEffect(() => {
+        for (const a of (state.accounts || [])) {
+          const isCooled = Boolean(a.cooldownUntil && a.cooldownUntil > now)
+          const wasCooled = prevCooldownsRef.current.get(a.ref)
+          if (wasCooled && !isCooled) {
+            const toastId = 't_cool_' + a.ref + '_' + Date.now()
+            setToasts((prev) => [{
+              id: toastId,
+              message: (a.label || a.ref) + ' ' + t('toastCooldownRecovered'),
+              cls: 'dsub-toast-ok',
+            }, ...prev].slice(0, 3))
+            setTimeout(() => dismissToast(toastId), 5000)
+          }
+          prevCooldownsRef.current.set(a.ref, isCooled)
+        }
+      }, [state.accounts, now])
+
       // Accounts list
       const rows = state.accounts.map((a, i) => {
         const isLogged = state.logged.includes(a.provider)
@@ -3157,8 +3333,12 @@ const cssId = 'dsh-subscriptions/settings.module.css'
           statusText = a.quarantineReason === 'VERIFY' ? t('verifyAccount') : String(a.quarantineReason)
         } else if (cooled) {
           statusClass = 'dsub-statusWarn'
-          // Only some model families may be cooled down - say which ones.
-          statusText = t('subsCooldown') + (Array.isArray(a.cooldownFamilies) && a.cooldownFamilies.length ? ' · ' + a.cooldownFamilies.join('/') : '')
+          const leftSec = Math.max(0, Math.ceil((a.cooldownUntil - now) / 1000))
+          const m = Math.floor(leftSec / 60)
+          const s = leftSec % 60
+          const pad = (n) => String(n).padStart(2, '0')
+          const fams = Array.isArray(a.cooldownFamilies) && a.cooldownFamilies.length ? ' · ' + a.cooldownFamilies.join('/') : ''
+          statusText = t('subsCooldown') + ' (' + pad(m) + ':' + pad(s) + ')' + fams
         } else if (isLogged) {
           if (pct >= 90) { statusClass = 'dsub-statusBad'; statusText = '90%+ ' + t('subsUsed') }
           else if (pct >= 50) { statusClass = 'dsub-statusWarn'; statusText = pct + '% ' + t('subsUsed') }

package/lib/config-schema.js

@@ -108,6 +108,12 @@ export const Config = z.object({
     .description('Custom model catalog for Grok.'),
   cascadingFallback: z.boolean().volatile().default(false)
     .description('#349 Opt-in cross-vendor fallback when all accounts of a provider are exhausted or rate limited.'),
+  cascadingFallbackOrder: z.array(z.string()).volatile().default(PROVIDERS)
+    .description('Configurable global priority sequence for cross-vendor fallback.'),
+  rotationStrategy: z.string().volatile().default('round-robin')
+    .description('Load balancing strategy among available accounts: round-robin (balanced), least-loaded (fair-share quota), or priority (sequential).'),
+  enableSoftFailureSentinel: z.boolean().volatile().default(true)
+    .description('Intercept HTTP 200 soft-failure capacity apologies and automatically cascade to next healthy account.'),
   cascadingChain: z.any().volatile().default({})
     .description('#349 Custom cross-vendor fallback chains (e.g. { claude: ["copilot", "cursor"] }).'),
   webhookUrl: z.string().volatile().default('')

package/lib/diagnostics.js

@@ -6,6 +6,7 @@ import { getSessionPinStats } from './session-pin.js'
 import { maskText } from './mask.js'
 import { normalizeSlots } from './accounts.js'
 import { PROVIDERS } from './refs.js'
+import { defaultErrorBuffer } from './error-buffer.js'
 
 function scrubReport(v) {
   if (typeof v === 'string') return maskText(v)
@@ -18,7 +19,7 @@ function scrubReport(v) {
   return v
 }
 
-export async function createDiagnosticsReport({ live, store, history, NS, pkgVersion }) {
+export async function createDiagnosticsReport({ live, store, history, NS, pkgVersion, heartbeatStats }) {
   const cfg = live()
   const slots = normalizeSlots(cfg.slots)
   const mk = () => ({ loggedIn: false, slots: 0, configured: 0, cooldown: 0, proxy: 0, maxUsagePercent: null })
@@ -51,6 +52,8 @@ export async function createDiagnosticsReport({ live, store, history, NS, pkgVer
     runtime: { node: process.version, platform: process.platform, arch: process.arch, uptimeSec: Math.round(process.uptime()) },
     providers,
     requests: { total: rows.length, byStatus, lastErrors },
+    recentErrors: defaultErrorBuffer.getEntries(),
+    heartbeat: heartbeatStats || { lastRunAt: null, proactiveRefreshCount: 0 },
     cache: calculateCacheMetrics(rows),
     queue: defaultConcurrencyQueue.stats(),
     pins: getSessionPinStats(),
@@ -78,6 +81,10 @@ export async function createDiagnosticsReport({ live, store, history, NS, pkgVer
       privacyMask: !!cfg.privacyMask,
       customVendors: Array.isArray(cfg.customVendors) ? cfg.customVendors.length : 0,
       proxySlots: slots.filter((s) => s.proxyUrl).length,
+      cascadingFallback: !!cfg.cascadingFallback,
+      cascadingFallbackOrder: cfg.cascadingFallbackOrder || PROVIDERS,
+      rotationStrategy: cfg.rotationStrategy || 'round-robin',
+      enableSoftFailureSentinel: cfg.enableSoftFailureSentinel !== false,
     },
   })
 }

package/lib/error-buffer.js

@@ -0,0 +1,49 @@
+import { cleanErrorMessage, maskText } from './mask.js'
+
+/**
+ * Ring buffer holding the most recent runtime errors across vendors and streams.
+ * Sanitizes all text to prevent credential or PII leaks in diagnostics.
+ */
+export class ErrorRingBuffer {
+  /**
+   * @param {number} [limit=20]
+   */
+  constructor(limit = 20) {
+    this.limit = limit
+    /** @type {Array<{ id: string, ts: string, provider: string, ref: string, model: string, status: number|null, code: string, message: string }>} */
+    this.entries = []
+    this._nextId = 1
+  }
+
+  /**
+   * @param {{ provider?: string, ref?: string, model?: string, status?: number|null, code?: string, message?: string }} [param0]
+   */
+  record({ provider = 'unknown', ref = '', model = '', status = null, code = '', message = '' } = {}) {
+    const rawMsg = cleanErrorMessage(maskText(String(message || '')))
+    const entry = {
+      id: String(this._nextId++),
+      ts: new Date().toISOString(),
+      provider: String(provider || ''),
+      ref: String(ref || ''),
+      model: String(model || ''),
+      status: typeof status === 'number' ? status : null,
+      code: String(code || ''),
+      message: rawMsg.slice(0, 250),
+    }
+    this.entries.unshift(entry)
+    if (this.entries.length > this.limit) {
+      this.entries.length = this.limit
+    }
+    return entry
+  }
+
+  getEntries() {
+    return this.entries.slice()
+  }
+
+  clear() {
+    this.entries = []
+  }
+}
+
+export const defaultErrorBuffer = new ErrorRingBuffer(20)

package/lib/index.js

@@ -416,37 +416,45 @@ export function apply(ctx, rawConfig) {
   const pmE = (s) => privacyOn() ? maskEmail(s) : String(s || '')
   const pmL = (s) => privacyOn() ? maskLabel(s) : String(s || '')
 
-  const diagnosticsReport = () => createDiagnosticsReport({ live, store, history, NS, pkgVersion })
+  const heartbeatStats = { lastRunAt: 0, proactiveRefreshCount: 0 }
+  const runProactiveHeartbeat = async () => {
+    heartbeatStats.lastRunAt = Date.now()
+    const cfg = live()
+    const ahead = Number(cfg.refreshAheadMs) || 5 * 60 * 1000
+    const retryMs = Number(cfg.refreshRetryMs) || 10 * 60 * 1000
+    const now = Date.now()
+    let refreshed = 0
+    for (const slot of normalizeSlots(cfg.slots)) {
+      const ref = slot.ref
+      try {
+        const info = await store.describeRef(ref)
+        if (!info.configured) continue
+        if (info.cooldownUntil && info.cooldownUntil > now) continue
+        const raw = await store.resolveRaw(ref)
+        if (!raw) continue
+        const blob = await store.loadBlob(ref).catch(() => null)
+        if (!blob || !blob.refreshToken) continue
+        if (!blob.expiresAt) continue
+        if (blob.expiresAt - now > ahead) continue
+        if (typeof store.shouldSkipRefresh === "function" && store.shouldSkipRefresh(ref, now, retryMs)) continue
+        await store.ensureFresh(slot.provider, blob, ref).then(() => {
+          refreshed++
+          heartbeatStats.proactiveRefreshCount++
+        }).catch((e) => {
+          if (logger && logger.warn) logger.warn("[dsh-subscriptions] background refresh failed for " + ref + ": " + String(e && e.message || e))
+        })
+      } catch { /* best-effort */ }
+    }
+    return refreshed
+  }
+
+  const diagnosticsReport = () => createDiagnosticsReport({ live, store, history, NS, pkgVersion, heartbeatStats })
   const accountsView = () => createAccountsView({ live, store, fetch, pmL })
 
   // Ponytail: background refresh ahead of expiry, single timer
   ctx.effect(() => {
-    const tick = async () => {
-      const cfg = live()
-      const ahead = Number(cfg.refreshAheadMs) || 5 * 60 * 1000
-      const retryMs = Number(cfg.refreshRetryMs) || 10 * 60 * 1000
-      const now = Date.now()
-      for (const slot of normalizeSlots(cfg.slots)) {
-        const ref = slot.ref
-        try {
-          const info = await store.describeRef(ref)
-          if (!info.configured) continue
-          if (info.cooldownUntil && info.cooldownUntil > now) continue
-          const raw = await store.resolveRaw(ref)
-          if (!raw) continue
-          const blob = await store.loadBlob(ref).catch(() => null)
-          if (!blob || !blob.refreshToken) continue
-          if (!blob.expiresAt) continue
-          if (blob.expiresAt - now > ahead) continue
-          if (typeof store.shouldSkipRefresh === 'function' && store.shouldSkipRefresh(ref, now, retryMs)) continue
-          await store.ensureFresh(slot.provider, blob, ref).catch((e) => {
-            if (logger && logger.warn) logger.warn('[dsh-subscriptions] background refresh failed for ' + ref + ': ' + String(e && e.message || e))
-          })
-        } catch { /* best-effort */ }
-      }
-    }
-    tick().catch(() => {})
-    const id = setInterval(() => { tick().catch(() => {}) }, 60 * 1000)
+    runProactiveHeartbeat().catch(() => {})
+    const id = setInterval(() => { runProactiveHeartbeat().catch(() => {}) }, 60 * 1000)
     return () => clearInterval(id)
   }, 'dsh-subscriptions: refresh ahead')
 
@@ -509,6 +517,7 @@ export function apply(ctx, rawConfig) {
     refForSlot,
     resetCredits,
     diagnosticsReport,
+    triggerHeartbeat: runProactiveHeartbeat,
     pending,
     completeOAuth,
     fetchForRef,

package/lib/quarantine.js

@@ -3,6 +3,7 @@ export const REASON_RATE_LIMIT = 'RATE_LIMIT'
 export const REASON_HARD_LIMIT = 'HARD_LIMIT'
 export const REASON_REVOKED = 'TOKEN_REVOKED'
 export const REASON_MANUAL_PARK = 'MANUAL_PARK'
+export const REASON_DEGRADED = 'DEGRADED_CAPACITY'
 
 export const STATUS_ACTIVE = 'active'
 export const STATUS_QUARANTINE = 'quarantine'
@@ -16,18 +17,21 @@ export function quarantineDuration(reason) {
       return 7 * 24 * 60 * 60 * 1000 // 7 days (requires re-auth)
     case REASON_MANUAL_PARK:
       return 60 * 60 * 1000 // 1 hour (manual park)
+    case REASON_DEGRADED:
+      return 5 * 60 * 1000 // 5 minutes degraded cooldown
     case REASON_RATE_LIMIT:
     default:
       return 60 * 60 * 1000 // 1 hour
   }
 }
 
-export function putInQuarantine(account, reason, nowMs = Date.now()) {
+export function putInQuarantine(account, reason, nowMs = Date.now(), customUntil = null) {
   const duration = quarantineDuration(reason)
+  const until = (customUntil && Number(customUntil) > Number(nowMs)) ? Number(customUntil) : (Number(nowMs) + duration)
   return {
     ...account,
     quarantineReason: reason,
-    quarantineUntil: nowMs + duration,
+    quarantineUntil: until,
     status: STATUS_QUARANTINE,
   }
 }

package/lib/reset-clock.js

@@ -0,0 +1,87 @@
+import { parseHeaders } from './ratelimit.js'
+
+export const VENDOR_WINDOW_MS = {
+  claude: 5 * 3600 * 1000,
+  codex: 3 * 3600 * 1000,
+  grok: 2 * 3600 * 1000,
+}
+
+export function parseResetFromText(message, nowMs = Date.now()) {
+  if (!message || typeof message !== 'string') return null
+  const s = message.trim()
+
+  const secMatch = s.match(/(?:try again in|retry after|wait|resets? in)\s+(\d+)\s*(?:s|sec|seconds?)/i)
+  if (secMatch) {
+    const sec = Number(secMatch[1])
+    if (Number.isFinite(sec) && sec > 0) return nowMs + sec * 1000
+  }
+
+  const minMatch = s.match(/(?:try again in|retry after|wait|resets? in)\s+(\d+)\s*(?:m|min|minutes?)/i)
+  if (minMatch) {
+    const min = Number(minMatch[1])
+    if (Number.isFinite(min) && min > 0) return nowMs + min * 60 * 1000
+  }
+
+  const hrMatch = s.match(/(?:try again in|retry after|wait|resets? in)\s+(\d+)\s*(?:h|hr|hours?)/i)
+  if (hrMatch) {
+    const hr = Number(hrMatch[1])
+    if (Number.isFinite(hr) && hr > 0) return nowMs + hr * 3600 * 1000
+  }
+
+  return null
+}
+
+export function resolveAccurateReset(err, account, provider, nowMs = Date.now(), safetyBufferMs = 10000) {
+  const now = Number(nowMs) || Date.now()
+  let targetReset = null
+
+  if (err && err.resetAt) {
+    const val = Number(err.resetAt)
+    if (Number.isFinite(val) && val > now) {
+      targetReset = val
+    }
+  }
+
+  if (!targetReset && err) {
+    const headers = err.headers || (err.response && err.response.headers)
+    if (headers) {
+      const parsed = parseHeaders(headers, now)
+      if (parsed && parsed.resetAt && parsed.resetAt > now) {
+        targetReset = parsed.resetAt
+      }
+    }
+  }
+
+  if (!targetReset && err && err.retryAfter) {
+    const n = Number(err.retryAfter)
+    if (Number.isFinite(n) && n > 0) {
+      targetReset = n > 1e9 ? (n > 1e12 ? n : n * 1000) : now + n * 1000
+    }
+  }
+
+  if (!targetReset && err && err.message) {
+    const textReset = parseResetFromText(err.message, now)
+    if (textReset && textReset > now) {
+      targetReset = textReset
+    }
+  }
+
+  if (!targetReset && account && account.quota && account.quota.resetAt) {
+    const qReset = Number(account.quota.resetAt)
+    if (Number.isFinite(qReset) && qReset > now) {
+      targetReset = qReset
+    }
+  }
+
+  if (targetReset && targetReset > now && targetReset <= now + 7 * 86400 * 1000) {
+    return targetReset + (Number(safetyBufferMs) || 0)
+  }
+
+  const p = String(provider || (account && account.provider) || '').toLowerCase()
+  const windowDefault = VENDOR_WINDOW_MS[p]
+  if (windowDefault && err && (err.status === 429 || err.code === 'RATE_LIMIT' || err.code === 'QUOTA')) {
+    return now + windowDefault + (Number(safetyBufferMs) || 0)
+  }
+
+  return null
+}

package/lib/rotate.js

@@ -7,6 +7,7 @@ const SWITCH_CODES = new Set([
   'QUOTA',
   'QUOTA_EXCEEDED',
   'LICENSE_REQUIRED',
+  'SOFT_FAILURE',
 ])
 
 export function isRegionError(err) {
@@ -18,7 +19,7 @@ export function isRegionError(err) {
 
 export function isSwitchableError(err) {
   if (!err || typeof err !== 'object') return false
-  if (err.name === 'TimeoutError' || err.code === 'TIMEOUT' || err.code === 'EMPTY_RESPONSE') return true
+  if (err.name === 'TimeoutError' || err.code === 'TIMEOUT' || err.code === 'EMPTY_RESPONSE' || err.code === 'SOFT_FAILURE' || err.isSoftFailure) return true
   if (err.name === 'AbortError' || err.code === 'ABORT_ERR' || err.code === 'ERR_ABORTED' || (err.message && /aborted/i.test(err.message))) return false
   const code = err.code || (err.failure && err.failure.code)
   if (SWITCH_CODES.has(code)) return true
@@ -263,6 +264,33 @@ export function pickAccountWithTrace(accounts, nowMs, opts) {
           }
         }
 
+        const strategy = (opts && opts.rotationStrategy) || 'round-robin'
+        if (strategy === 'least-loaded') {
+          const aRem = getAccountRemainingPercent(a)
+          const bRem = getAccountRemainingPercent(b)
+          if (aRem != null && bRem != null) {
+            if (Math.abs(aRem - bRem) >= 5) {
+              return bRem - aRem
+            }
+          } else if (aRem != null && bRem == null) {
+            return -1
+          } else if (aRem == null && bRem != null) {
+            return 1
+          }
+          if (a.usagePercent != null && b.usagePercent != null) {
+            if (Math.abs(Number(a.usagePercent) - Number(b.usagePercent)) >= 5) {
+              return Number(a.usagePercent) - Number(b.usagePercent)
+            }
+          }
+        } else if (strategy === 'priority') {
+          const aWeight = a.weight || (a.isPro ? 5 : 1)
+          const bWeight = b.weight || (b.isPro ? 5 : 1)
+          if (aWeight !== bWeight) return bWeight - aWeight
+          const aIdx = list.indexOf(a)
+          const bIdx = list.indexOf(b)
+          if (aIdx !== -1 && bIdx !== -1 && aIdx !== bIdx) return aIdx - bIdx
+        }
+
         // 0. Auto-Pacing: lower pacing risk preferred (#352)
         const aRisk = Number(a.pacingRisk) || 0
         const bRisk = Number(b.pacingRisk) || 0
@@ -333,14 +361,17 @@ export function pickAccount(accounts, nowMs, opts) {
   return res ? res.account : null
 }
 
-export function markCooldown(account, nowMs, cooldownMs, family) {
+export function markCooldown(account, nowMs, cooldownMs, family, customUntil = null) {
   const wait = Number(cooldownMs)
   const ms = Number.isFinite(wait) && wait > 0 ? wait : 30 * 60 * 1000
+  const until = (customUntil && Number(customUntil) > (Number(nowMs) || 0))
+    ? Number(customUntil)
+    : ((Number(nowMs) || 0) + ms)
   const prev = Array.isArray(account.cooldownFamilies) ? account.cooldownFamilies.slice() : []
   const fams = family ? (prev.includes(family) ? prev : prev.concat(family)) : prev
   return {
     ...account,
-    cooldownUntil: (Number(nowMs) || 0) + ms,
+    cooldownUntil: until,
     ...(fams.length ? { cooldownFamilies: fams } : {}),
   }
 }

package/lib/routes/status.js

@@ -19,6 +19,7 @@ import { isTrustedSettingsRequest, queryOf, readBody, safeJsonHandler, writeJson
 import { quotaSnapshot } from '../ratelimit.js'
 import { PROVIDERS, displayName, droppedCredentialRefs, isProvider, oauthRef } from '../refs.js'
 import { getVendor } from '../vendors/index.js'
+import { defaultErrorBuffer } from '../error-buffer.js'
 
 export function registerStatusRoutes(ctx, state) {
   const {
@@ -298,6 +299,40 @@ export function registerStatusRoutes(ctx, state) {
     }),
   }), 'dsh-subscriptions: /diagnostics')
 
+  ctx.effect(() => ctx.webServer.register({
+    kind: 'exact',
+    path: '/dsh-subscriptions/diagnostics/errors/clear',
+    handler: safeJsonHandler(async (req, res) => {
+      if (req.method !== 'POST') {
+        writeJson(res, 405, { ok: false, error: { code: 'method', message: 'POST only' } })
+        return
+      }
+      if (!isTrustedSettingsRequest(req, ctx)) {
+        writeJson(res, 403, { ok: false, error: { code: 'forbidden', message: 'same-origin only' } })
+        return
+      }
+      defaultErrorBuffer.clear()
+      writeJson(res, 200, { ok: true, cleared: true })
+    }),
+  }), 'dsh-subscriptions: /diagnostics/errors/clear')
+
+  ctx.effect(() => ctx.webServer.register({
+    kind: 'exact',
+    path: '/dsh-subscriptions/heartbeat/run',
+    handler: safeJsonHandler(async (req, res) => {
+      if (req.method !== 'POST') {
+        writeJson(res, 405, { ok: false, error: { code: 'method', message: 'POST only' } })
+        return
+      }
+      if (!isTrustedSettingsRequest(req, ctx)) {
+        writeJson(res, 403, { ok: false, error: { code: 'forbidden', message: 'same-origin only' } })
+        return
+      }
+      const refreshedCount = typeof state.triggerHeartbeat === 'function' ? await state.triggerHeartbeat() : 0
+      writeJson(res, 200, { ok: true, refreshed: refreshedCount })
+    }),
+  }), 'dsh-subscriptions: /heartbeat/run')
+
 
   // ponytail: cheap per-vendor probe; never sets cooldown, never returns tokens
   ctx.effect(() => ctx.webServer.register({
@@ -345,10 +380,6 @@ export function registerStatusRoutes(ctx, state) {
         return
       }
       const vendor = getVendor(provider)
-      if (typeof vendor.check !== 'function') {
-        writeJson(res, 200, { ok: true, provider, index: index, ref, email: pmE(blob.email), label: pmL(blob.label), expiresAt: blob.expiresAt || null, quota: info.quota || null })
-        return
-      }
       let capturedQuota = null
       const probeFetch = async (url, init = {}) => {
         const doFetch = (typeof fetchForRef === "function" && fetchForRef(ref)) || fetch
@@ -360,8 +391,14 @@ export function registerStatusRoutes(ctx, state) {
         return res2
       }
       try {
-        await vendor.check(blob, vendorConfig(provider, live()), probeFetch)
-        const latencyMs = Date.now() - t0;
+        if (typeof vendor.check === 'function') {
+          await vendor.check(blob, vendorConfig(provider, live()), probeFetch)
+        } else if (typeof vendor.listModels === 'function') {
+          await vendor.listModels(blob, vendorConfig(provider, live()), probeFetch)
+        } else if (typeof vendor.usage === 'function') {
+          await vendor.usage(blob, vendorConfig(provider, live()), probeFetch)
+        }
+        const latencyMs = Date.now() - t0
         writeJson(res, 200, { ok: true, provider, index: index, ref, latencyMs, email: pmE(blob.email), label: pmL(blob.label), expiresAt: blob.expiresAt || null, quota: capturedQuota || info.quota || null, usagePercent: info.usagePercent ?? null })
       } catch (e) {
         const isTimeout = isTimeoutError(e)

package/lib/sentinel.js

@@ -0,0 +1,136 @@
+export const SOFT_FAILURE_PATTERNS = [
+  /currently (?:unable to process|unavailable|overloaded)/i,
+  /experiencing (?:exceptionally )?(?:high|heavy) (?:traffic|demand|load)/i,
+  /capacity (?:is )?temporarily (?:unavailable|limited|exceeded)/i,
+  /model is (?:currently )?(?:overloaded|busy)/i,
+  /due to high traffic/i,
+  /usage cap reached, switching to lower/i,
+  /you have reached your (?:current )?usage limit/i,
+  /server is currently under heavy load/i,
+  /i apologize, but (?:i am unable to assist|i cannot fulfill this request due to (?:high demand|capacity|server load))/i,
+  /our systems are experiencing heavy load/i,
+]
+
+const SUSPICIOUS_ROOTS = [
+  'i am',
+  'i apologize',
+  'we are',
+  'capacity',
+  'the model',
+  'due to',
+  'usage cap',
+  'you have reached',
+  'server is',
+  'our systems',
+]
+
+export function extractChunkText(chunk) {
+  if (!chunk) return ''
+  if (typeof chunk === 'string') return chunk
+  if (typeof chunk.delta === 'string') return chunk.delta
+  if (typeof chunk.content === 'string') return chunk.content
+  if (typeof chunk.text === 'string') return chunk.text
+  if (chunk.choices && Array.isArray(chunk.choices) && chunk.choices[0]) {
+    const c0 = chunk.choices[0]
+    if (c0.delta && typeof c0.delta.content === 'string') return c0.delta.content
+    if (typeof c0.text === 'string') return c0.text
+  }
+  if (chunk.candidates && Array.isArray(chunk.candidates) && chunk.candidates[0]) {
+    const cand = chunk.candidates[0]
+    if (cand.content && Array.isArray(cand.content.parts)) {
+      return cand.content.parts.map((p) => (p && typeof p.text === 'string' ? p.text : '')).join('')
+    }
+  }
+  return ''
+}
+
+export function detectSoftFailure(text) {
+  if (!text || typeof text !== 'string') return null
+  const cleaned = text.trim()
+  if (!cleaned) return null
+  for (const pattern of SOFT_FAILURE_PATTERNS) {
+    if (pattern.test(cleaned)) {
+      return pattern.toString()
+    }
+  }
+  return null
+}
+
+export function isCandidatePrefix(text) {
+  if (!text) return true
+  const low = text.toLowerCase().trim()
+  if (low.length < 3) return true
+  for (const root of SUSPICIOUS_ROOTS) {
+    if (root.startsWith(low) || low.startsWith(root)) {
+      return true
+    }
+  }
+  return false
+}
+
+export function createSoftFailureError(pattern, text) {
+  const preview = String(text || '').slice(0, 120).replace(/\s+/g, ' ')
+  const err = new Error(`Soft-failure detected: ${preview}`)
+  return Object.assign(err, {
+    name: 'SoftFailureError',
+    code: 'SOFT_FAILURE',
+    status: 429,
+    isSoftFailure: true,
+    degraded: true,
+    pattern,
+  })
+}
+
+export async function* withSoftFailureSentinel(stream, options = {}) {
+  const enabled = options.enabled !== false
+  if (!enabled || !stream) {
+    if (stream) {
+      for await (const chunk of stream) {
+        yield chunk
+      }
+    }
+    return
+  }
+
+  const bufferLimitChars = Number(options.bufferLimitChars) || 250
+  const maxBufferChunks = Number(options.maxBufferChunks) || 4
+
+  const bufferedChunks = []
+  let accumulatedText = ''
+  let flushed = false
+
+  for await (const chunk of stream) {
+    if (flushed) {
+      yield chunk
+      continue
+    }
+
+    bufferedChunks.push(chunk)
+    accumulatedText += extractChunkText(chunk)
+
+    const matched = detectSoftFailure(accumulatedText)
+    if (matched) {
+      throw createSoftFailureError(matched, accumulatedText)
+    }
+
+    const candidate = isCandidatePrefix(accumulatedText)
+    if (!candidate || accumulatedText.length >= bufferLimitChars || bufferedChunks.length >= maxBufferChunks) {
+      flushed = true
+      for (const b of bufferedChunks) {
+        yield b
+      }
+      bufferedChunks.length = 0
+    }
+  }
+
+  if (!flushed) {
+    const matched = detectSoftFailure(accumulatedText)
+    if (matched) {
+      throw createSoftFailureError(matched, accumulatedText)
+    }
+    for (const b of bufferedChunks) {
+      yield b
+    }
+    bufferedChunks.length = 0
+  }
+}

package/lib/stream-rotate.js

@@ -1,9 +1,12 @@
 import { defaultConcurrencyQueue } from './concurrency-queue.js'
 import { recordSessionCacheUsage } from './session-pin.js'
 import { pickAccountWithTrace, markCooldown, isSwitchableError, isRegionError, modelFamily } from './rotate.js'
-import { putInQuarantine, REASON_RATE_LIMIT, REASON_HARD_LIMIT, REASON_REVOKED } from './quarantine.js'
+import { putInQuarantine, REASON_RATE_LIMIT, REASON_HARD_LIMIT, REASON_REVOKED, REASON_DEGRADED } from './quarantine.js'
 import { notifySessionExpired, notifyQuarantineEntered } from './alerts.js'
 import { withRunawayGuard } from './runaway-guard.js'
+import { defaultErrorBuffer } from './error-buffer.js'
+import { withSoftFailureSentinel } from './sentinel.js'
+import { resolveAccurateReset } from './reset-clock.js'
 
 export async function* streamWithRotation({
   accounts,
@@ -54,6 +57,7 @@ export async function* streamWithRotation({
       autoPacing: options && (options.autoPacing !== undefined ? options.autoPacing : (options.config && options.config.autoPacing)),
       drainBeforeReset: options && (options.drainBeforeReset !== undefined ? options.drainBeforeReset : (options.config && options.config.drainBeforeReset)),
       drainSafetyPercent: options && (options.drainSafetyPercent !== undefined ? options.drainSafetyPercent : (options.config && options.config.drainSafetyPercent)),
+      rotationStrategy: options && (options.rotationStrategy !== undefined ? options.rotationStrategy : (options.config && options.config.rotationStrategy)),
     })
     const account = pickRes.account
 
@@ -146,7 +150,10 @@ export async function* streamWithRotation({
           streamOnce(account, { ...options, _runawayState: runawayState }),
           runawayOpts
         )
-        for await (const chunk of guardedStream) {
+        const sentineledStream = withSoftFailureSentinel(guardedStream, {
+          enabled: (options?.enableSoftFailureSentinel !== false) && (options?.config?.enableSoftFailureSentinel !== false),
+        })
+        for await (const chunk of sentineledStream) {
           firstChunkDelivered = true
           const cachedTokens = Number(
             (chunk && chunk.usage && (chunk.usage.cacheReadTokens || chunk.usage.cachedTokens)) ||
@@ -163,6 +170,14 @@ export async function* streamWithRotation({
         return
       } catch (err) {
         lastError = err
+        defaultErrorBuffer.record({
+          provider: options && options.provider,
+          ref: account.ref || account.id,
+          model: options && options.model,
+          status: err && (err.status || err.statusCode),
+          code: err && (err.code || err.name),
+          message: err && (err.message || String(err)),
+        })
         // If chunks were already yielded to the caller, never rotate mid-stream
         // as that would repeat or scramble generated tokens.
         if (firstChunkDelivered) throw err
@@ -186,18 +201,24 @@ export async function* streamWithRotation({
         currentAttempt.status = 'switched'
         currentAttempt.switchError = err ? (err.code || err.status || err.message) : 'switchable'
 
-        // Move slot to cooldown and quarantine (#172)
-        const cooled = markCooldown(account, nowMs(), cooldownMs, modelFamily(options && options.provider, options && options.model))
+        const accurateUntil = resolveAccurateReset(err, account, options && options.provider, nowMs())
+        const isSoft = Boolean(err && (err.code === 'SOFT_FAILURE' || err.isSoftFailure))
+
+        // Move slot to cooldown and quarantine (#172, #538)
+        const effectiveCooldownMs = isSoft ? Math.min(cooldownMs, 5 * 60 * 1000) : cooldownMs
+        const cooled = markCooldown(account, nowMs(), effectiveCooldownMs, modelFamily(options && options.provider, options && options.model), accurateUntil)
         account.cooldownUntil = cooled.cooldownUntil
         if (cooled.cooldownFamilies) account.cooldownFamilies = cooled.cooldownFamilies
 
         const status = Number(err && (err.status || err.statusCode) || 0)
         const code = String(err && err.code || '')
-        const reason = (status === 401 || code === 'AUTH' || code === 'TOKEN_REVOKED')
-          ? REASON_REVOKED
-          : (status === 403 || code === 'HARD_LIMIT' || code === 'LICENSE_REQUIRED')
-            ? REASON_HARD_LIMIT
-            : REASON_RATE_LIMIT
+        const reason = isSoft
+          ? REASON_DEGRADED
+          : (status === 401 || code === 'AUTH' || code === 'TOKEN_REVOKED')
+            ? REASON_REVOKED
+            : (status === 403 || code === 'HARD_LIMIT' || code === 'LICENSE_REQUIRED')
+              ? REASON_HARD_LIMIT
+              : REASON_RATE_LIMIT
 
         if (err && err.isWarmupProbeFailure && err.warmupAccount) {
           account.quarantineUntil = err.warmupAccount.quarantineUntil
@@ -205,7 +226,7 @@ export async function* streamWithRotation({
           account.quarantineAttempts = err.warmupAccount.quarantineAttempts
           account.status = err.warmupAccount.status
         } else {
-          const quarantined = putInQuarantine(account, reason, nowMs())
+          const quarantined = putInQuarantine(account, reason, nowMs(), accurateUntil)
           account.quarantineUntil = quarantined.quarantineUntil
           account.quarantineReason = quarantined.quarantineReason
           account.quarantineAttempts = (account.quarantineAttempts || 0) + 1
@@ -213,6 +234,10 @@ export async function* streamWithRotation({
 
         currentAttempt.quarantineReason = account.quarantineReason || reason
         currentAttempt.quarantineUntil = account.quarantineUntil
+        if (accurateUntil) {
+          currentAttempt.accurateReset = true
+          currentAttempt.resetAt = accurateUntil
+        }
 
         try {
           notifyQuarantineEntered({

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@goodandready/dsh-subscriptions",
-  "version": "0.6.69",
+  "version": "0.6.71",
   "description": "Use ChatGPT Codex, Claude, Grok, Antigravity, Kimi, GLM, Cursor, Kiro, Copilot, Qwen, ERNIE, Spark, JetBrains, Perplexity, Replit, and Cody subscriptions as DeepSeek Harness LLM providers via OAuth.",
   "license": "MIT",
   "type": "module",