oira666_pi-limits-wait 0.6.11 → 0.6.13

package/README.md

@@ -20,13 +20,13 @@ Tested against Pi 1.5.0.
 - Intercepts the current runtime without registering synthetic providers; every retry and fallback re-enters Pi's normal provider/auth/base URL/configured-header composition, including OAuth refresh checks.
 - Stops background cache warming via `cache_warming_decision` while retry interception is enabled, preventing background refreshes from triggering foreground retries or fallbacks. Normal request caching is unchanged. With `PI_LIMITS_WAIT_DISABLE_ALL_WAITING=true`, Pi's cache-warming policy is left untouched. As with any handler for this event, a later extension can override the decision.
 - Reload-safe ownership ensures `/reload` transfers interception cleanly and aborts waits still owned by the previous extension instance. The interception has one process-global active extension owner, matching the interactive Pi CLI's single active session runtime lifecycle; SDK processes running multiple concurrent Pi sessions are not supported.
-- On rate-limit errors (`429`, `rate_limit`, `too many requests`, quota/reset messages), waits and retries in a loop.
+- On rate-limit errors (`429`, `rate_limit`, `too many requests`, quota/reset messages), waits and retries in a loop. With `PI_LIMITS_WAIT_DEFAULT_WAITING=false` it never waits: it only switches between fallback models (see [Environment variables](#environment-variables)).
 - Remembers provider retry timing exposed through response callbacks or provider errors (`retry-after`, `retry-after-ms`, `retry in ...`, reset messages). Explicit provider retry/reset deadlines are always honored: no early probes occur during that period (unless a wait is manually skipped).
 - If no retry timing is available for a rate limit, waits 30 minutes before retrying.
 - On `server_is_overloaded`, waits 5 minutes, then retries. If the provider is still overloaded after Pi's normal retries, it waits another 5 minutes and repeats.
 - On transient network/transport failures — including undici idle-timeout aborts (`UND_ERR_HEADERS_TIMEOUT` / `UND_ERR_BODY_TIMEOUT`), `fetch failed`, `terminated`, `ECONNRESET`, `ETIMEDOUT`, etc. — treats the error as retryable with a short backoff (default 15s, or the provider's `retry-after`) instead of giving up. This prevents a stalled streaming request from turning into a silent hang.
 - Leaves context-overflow recovery entirely to Pi. The extension uses Pi's own `isContextOverflow()` detector and immediately surfaces matching failures without waiting, retrying, or selecting another fallback. If the overflow came from a fallback already attempted for the request, the extension makes that attempted model current when Pi is still on the request's original model and accepts the switch; it never overrides a newer user model selection. Pi intentionally ignores overflow errors from non-current models. Successful over-window responses and zero-output length stops also pass through unchanged, allowing Pi to compact and perform its built-in recovery.
-- For any other (unclassified) provider error — including an API `Internal server error` and HTTP `400`/`403` failures — waits and retries the same model. There is deliberately no permanent-HTTP-status allowlist. This retry-all behavior is enabled by default and uses a configurable retry limit and interval; it does not alter model-limit wait intervals. ChatGPT Business workspace-admin usage limits and Zhipu/Z.ai plan-exhaustion code `1308` use this bounded policy, even when accompanied by HTTP `429` or `Retry-After`, rather than an endless timed-limit loop. Disable `PI_LIMITS_WAIT_DEFAULT_WAITING` to surface them immediately, matching Pi's non-retryable classification; the provider's error guidance is preserved.
+- For any other (unclassified) provider error — including an API `Internal server error` and HTTP `400`/`403` failures — waits and retries the same model. There is deliberately no permanent-HTTP-status allowlist. This retry-all behavior is enabled by default and uses a configurable retry limit and interval; it does not alter model-limit wait intervals. ChatGPT Business workspace-admin usage limits and Zhipu/Z.ai plan-exhaustion code `1308` use this bounded policy, even when accompanied by HTTP `429` or `Retry-After`, rather than an endless timed-limit loop. Disable `PI_LIMITS_WAIT_DEFAULT_WAITING` to surface them immediately (after trying any fallback models), matching Pi's non-retryable classification; the provider's error guidance is preserved.
 - Reports retry/fallback waits in chat notifications without adding persistent TUI status lines.
 - In non-interactive hosts (`rpc` and `json` modes) the extension never reports a planned retry as an error. While it is waiting to retry, it publishes `info`-level liveliness notifications (plus a `setStatus` entry) describing what it is doing, how long it will wait, and why. An `error`-level notification is emitted only when the extension stops retrying and surfaces the terminal failure. See [Non-interactive (RPC) reporting](#non-interactive-rpc-reporting).
 - During a wait, pressing Enter in an empty editor retries immediately. Typed commands, autocomplete, and model-picker navigation still work normally. Choosing another model with `/model` immediately ends the pending wait and tries that exact model, even if it was previously marked unavailable.
@@ -238,7 +238,7 @@ Example:
 }
 ```
 
-`fallback-models` is an ordered priority list: the entry closest to the top is preferred whenever it is usable.
+The model the chat starts with is always tried first. `fallback-models` is an ordered list of alternatives used only when that model is unavailable; listing the startup model lower does not demote it or try it twice.
 
 Each `fallback-models` entry supports:
 
@@ -252,7 +252,7 @@ Top-level settings keys:
 
 Fallback priority and manual selection:
 
-1. Normally, `fallback-models` sets priority from top to bottom. An unlisted startup model is tried before the list. Unrelated model calls and internal synthetic models are not rerouted.
+1. The startup model comes first, followed by `fallback-models` from top to bottom with duplicates removed. A healthy startup model is never replaced just because another model appears earlier in the configured list. Unrelated model calls and internal synthetic models are not rerouted.
 2. A manual model change overrides that priority, whether the chosen model is listed or not. Calls stay on that model without probing higher-priority models while it works. If it fails, the override ends and normal retry/fallback rules apply. After failover, the configured list controls priority again; recovery of the manually chosen model does not restore its override. Abort and context-overflow recovery do not cancel the override. Without an override, each LLM call starts at the top of the order and uses the highest-priority available model.
 3. If the active model becomes rate-limited, the extension immediately moves down to the next usable model in the order.
 4. As soon as a higher-priority model works again, the next call goes back to it and Pi's active model is switched back automatically. That happens in three ways:
@@ -272,7 +272,7 @@ The extension rereads these files before every LLM call, so changes made during
 | Variable | Default | Description |
 |----------|---------|-------------|
 | `PI_LIMITS_WAIT_DISABLE_ALL_WAITING` | `false` | Master kill switch for this extension's error handling, retries, fallbacks, freezing, waiting, notifications, telemetry, and retry summaries. Set to `true` (also accepts `1`, `yes`, `on`) to leave only Anthropic OAuth system-prompt cleaning enabled; requests continue through Pi's normal runtime, including its normal Anthropic headers. This takes precedence over every setting below. |
-| `PI_LIMITS_WAIT_DEFAULT_WAITING` | `true` | Whether unclassified/unknown provider errors enter the wait-and-retry cycle. Set to `false` (also accepts `0`, `no`, `off`) to surface the error immediately. Rate-limit, overload, authentication, and network retry behavior is unchanged. |
+| `PI_LIMITS_WAIT_DEFAULT_WAITING` | `true` | Whether the extension ever waits. Set to `false` (also accepts `0`, `no`, `off`) to never wait or retry the same model: on a rate limit, overload, authentication, network or unknown error it immediately moves to the next usable `fallback-models` entry, and surfaces the last error once none is left (or at once without fallback models). Model switching itself is controlled by `fallback-models`. Useful for hosts such as proxies that must answer promptly. |
 | `PI_LIMITS_WAIT_MAX_RETRY` | `999999` | Maximum extension-created retries for an unclassified/unknown provider error. The initial failed request and provider-internal retries do not count. HTTP 400/403 remain eligible. Must be a non-negative integer. |
 | `PI_LIMITS_WAIT_LIVELINESS_INTERVAL` | `15` | Seconds between liveliness notifications published while waiting in non-interactive (`rpc`/`json`) modes. Must be a non-negative integer; values below 1 second are clamped to 1 second. Ignored in TUI mode. Also drives the structured status channel and is reported inside it as `livelinessIntervalMs`. |
 | `PI_LIMITS_WAIT_STATUS_JSON` | `true` | Whether the machine-readable status channel (`statusKey` `oira666.pi-limits-wait.json`) is published in non-interactive modes. Set to `false` (also accepts `0`, `no`, `off`) to restore exact 0.5.6 wire behavior. The prose channel is never affected. |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "oira666_pi-limits-wait",
-  "version": "0.6.11",
+  "version": "0.6.13",
   "description": "Extension for PI that allows to automatically wait for limits to reset and retry the request",
   "type": "module",
   "exports": {

package/src/constants.ts

@@ -43,7 +43,7 @@ export const DEFAULT_LIVELINESS_INTERVAL_MS = 15 * 1_000; // 15 seconds
 export const LIVELINESS_JSON_STATUS_KEY = "oira666.pi-limits-wait.json";
 /** Payload schema version for LIVELINESS_JSON_STATUS_KEY. Bump only on a breaking change. */
 export const LIVELINESS_JSON_VERSION = 1;
-export const EXTENSION_VERSION = "0.6.8";
+export const EXTENSION_VERSION = "0.6.13";
 
 export const SETTINGS_FILE_NAME = "limits-wait.json";
 export const FALLBACK_MODELS_KEY = "fallback-models";

package/src/models.ts

@@ -39,16 +39,15 @@ export function configuredAttempt(model: Model<Api>): FallbackModel {
 }
 
 /**
- * A manual choice stays first until it fails. Otherwise use configured priority.
- * An unlisted startup model stays first until an explicit selection has failed.
+ * The startup model stays ahead of configured fallbacks, even if listed lower.
+ * A manual choice stays first until it fails, then releases priority to the list.
  * Unrelated calls are excluded by isFallbackEligibleModel.
  */
 export function candidateOrder(current: Model<Api>): FallbackModel[] {
   const primary = getPrimaryModel(current);
-  const listed = state.fallbackModels.some((entry) => modelKey(entry.model) === modelKey(primary));
   const order: FallbackModel[] = state.manualModelOverride
     ? [configuredAttempt(state.manualModelOverride)]
-    : listed || state.manualModelOverrideReleased ? [] : [configuredAttempt(primary)];
+    : state.manualModelOverrideReleased ? [] : [configuredAttempt(primary)];
   const seen = new Set(order.map((entry) => modelKey(entry.model)));
   for (const entry of state.fallbackModels) {
     const key = modelKey(entry.model);

package/src/state.ts

@@ -13,7 +13,7 @@ export const state = {
   ambientStatusCleanup: undefined as (() => void) | undefined,
   modelStatusCleanup: undefined as (() => void) | undefined,
   fallbackModels: [] as FallbackModel[],
-  unknownErrorWaitingEnabled: true,
+  waitingEnabled: true,
   // Includes the initial request, so the default is one more than max retries.
   nonRetryableMaxAttempts: DEFAULT_UNKNOWN_ERROR_MAX_RETRIES + 1,
   nonRetryableRetryDelayMs: DEFAULT_UNKNOWN_ERROR_RETRY_INTERVAL_MS,

package/src/stream.ts

@@ -217,9 +217,10 @@ export function streamWithLimitsRetry(
     const nonRetryableAttempts = new Map<string, number>();
     const triedModels = new Set<string>();
     let lastExhaustedFailure: string | undefined;
-    // Exhaustion is request-local when freezing is disabled. Every selection,
-    // including invocation-time re-selection, must use the same exclusions.
-    const exclusions = () => freezingEnabled() ? undefined : triedModels;
+    let lastFailure: string | undefined;
+    // Exhaustion is request-local when freezing or waiting is disabled. Every
+    // selection, including invocation-time re-selection, must use the same exclusions.
+    const exclusions = () => freezingEnabled() && state.waitingEnabled ? undefined : triedModels;
     const nextAttemptCandidate = (current: Model<Api>) => findAttemptCandidate(current, exclusions());
     const nextAvailableCandidate = (current: Model<Api>) => nextAttemptCandidate(current)?.entry;
     const earliestCandidateWakeup = (current: Model<Api>) => findCandidateWakeup(current, exclusions());
@@ -317,6 +318,7 @@ export function streamWithLimitsRetry(
         const next = nextAttemptCandidate(attempt.model);
         if (!next) {
           if (!freezingEnabled() && lastExhaustedFailure) throw new Error(lastExhaustedFailure);
+          if (!state.waitingEnabled) throw new Error(lastFailure ?? "All configured models are unavailable.");
           const wakeup = earliestCandidateWakeup(attempt.model)!;
           const wait = await managedWait("model-frozen", Math.max(0, wakeup - Date.now()), "All configured models are unavailable.");
           if (wait === "aborted" || signal?.aborted) {
@@ -334,6 +336,7 @@ export function streamWithLimitsRetry(
       const buffer: AssistantMessageEvent[] = [];
       let retryable: RetryableError | undefined;
       let retryableErrorMessage = "";
+      let retryableEvent: AssistantMessageEvent | undefined;
       let nonRetryableError: { message: string; event?: AssistantMessageEvent } | undefined;
       let observedHttpError: string | undefined;
       let providerPayloadReady = false;
@@ -478,8 +481,10 @@ export function streamWithLimitsRetry(
                   return;
                 }
                 retryable = getRetryableError(errMsg);
-                if (retryable) retryableErrorMessage = errMsg;
-                else nonRetryableError = { message: errMsg, event: surfacedEvent };
+                if (retryable) {
+                  retryableErrorMessage = errMsg;
+                  retryableEvent = surfacedEvent;
+                } else nonRetryableError = { message: errMsg, event: surfacedEvent };
                 break;
               }
 
@@ -608,6 +613,8 @@ export function streamWithLimitsRetry(
         return;
       }
 
+      lastFailure = retryableErrorMessage || nonRetryableError?.message || lastFailure;
+
       // A superseded attempt cannot freeze or limit the user's newer choice.
       if (observedUserModelSelection !== state.userModelSelectionGeneration) continue;
       releaseManualOverride();
@@ -628,40 +635,40 @@ export function streamWithLimitsRetry(
         postponeProbe(attempt.model);
       }
 
-      if (!retryable) {
-        const failureEvent = nonRetryableError?.event;
-        const failureMessage = nonRetryableError?.message ?? "Provider stream ended without a terminal event.";
-        const emitFailure = () => {
-          retryPeriod?.recordReason(failureMessage);
-          retryPeriod?.finalize();
-          // No further wait/retry is planned: this is the only error surfaced
-          // to non-interactive hosts for this request.
-          notifyFinalFailure(
-            `${formatModel(attempt.model)} failed and pi-limits-wait is not retrying: ${formatErrorDetail(failureMessage, Number.MAX_SAFE_INTEGER)}`,
-            failureMessage,
-          );
-          if (!committed) {
-            if (buffer.length > 0) flush(buffer);
-            else {
-              output.push({ type: "start", partial: freshMessage(attempt.model) });
-              committed = true;
-            }
-          }
-          if (failureEvent) output.push(failureEvent);
+      const emitFailure = (failureMessage: string, failureEvent?: AssistantMessageEvent) => {
+        retryPeriod?.recordReason(failureMessage);
+        retryPeriod?.finalize();
+        // No further wait/retry is planned: this is the only error surfaced
+        // to non-interactive hosts for this request.
+        notifyFinalFailure(
+          `${formatModel(attempt.model)} failed and pi-limits-wait is not retrying: ${formatErrorDetail(failureMessage, Number.MAX_SAFE_INTEGER)}`,
+          failureMessage,
+        );
+        if (!committed) {
+          if (buffer.length > 0) flush(buffer);
           else {
-            const error = freshMessage(attempt.model);
-            error.stopReason = "error";
-            error.errorMessage = failureMessage;
-            output.push({ type: "error", reason: "error", error });
+            output.push({ type: "start", partial: freshMessage(attempt.model) });
+            committed = true;
           }
-          output.end();
-        };
+        }
+        if (failureEvent) output.push(failureEvent);
+        else {
+          const error = freshMessage(attempt.model);
+          error.stopReason = "error";
+          error.errorMessage = failureMessage;
+          output.push({ type: "error", reason: "error", error });
+        }
+        output.end();
+      };
+
+      if (!retryable) {
+        const failureMessage = nonRetryableError?.message ?? "Provider stream ended without a terminal event.";
 
-        if (!committed && (state.unknownErrorWaitingEnabled || probe)) {
+        if (!committed) {
           const key = modelKey(attempt.model);
           const attempts = (nonRetryableAttempts.get(key) ?? 0) + 1;
           nonRetryableAttempts.set(key, attempts);
-          if (!probe && attempts < state.nonRetryableMaxAttempts) {
+          if (!probe && state.waitingEnabled && attempts < state.nonRetryableMaxAttempts) {
             notifyRetryingAfterError(attempt.model, state.nonRetryableRetryDelayMs, failureMessage);
             retryPeriod?.beginRetry(failureMessage);
             const wait = await managedWait("retry", state.nonRetryableRetryDelayMs, failureMessage, {
@@ -687,7 +694,7 @@ export function streamWithLimitsRetry(
                 attempt = next;
                 continue;
               }
-              const wakeup = earliestCandidateWakeup(attempt.model);
+              const wakeup = state.waitingEnabled ? earliestCandidateWakeup(attempt.model) : undefined;
               if (wakeup) {
                 retryPeriod?.beginRetry(failureMessage);
                 const wait = await managedWait("model-frozen", Math.max(0, wakeup - Date.now()), failureMessage);
@@ -710,7 +717,23 @@ export function streamWithLimitsRetry(
             }
           }
         }
-        emitFailure();
+        emitFailure(failureMessage, nonRetryableError?.event);
+        return;
+      }
+
+      if (!state.waitingEnabled) {
+        // Waiting is off: move on to the next usable model now, or surface the error.
+        if (allowFallback) {
+          if (retryable.reason !== "network") rememberRateLimit(attempt.model, retryable, retryableErrorMessage, true);
+          triedModels.add(modelKey(attempt.model));
+          const next = nextAvailableCandidate(attempt.model);
+          if (next) {
+            retryPeriod?.recordImmediateRetry(retryableErrorMessage);
+            attempt = next;
+            continue;
+          }
+        }
+        emitFailure(retryableErrorMessage, retryableEvent);
         return;
       }
 

package/src/ui.ts

@@ -126,7 +126,7 @@ export function freezingEnabled(): boolean {
 
 /** Load unknown-error retry settings once per extension instance/reload. */
 export function loadUnknownErrorRetrySettings(): void {
-  state.unknownErrorWaitingEnabled = envBoolean(process.env[DEFAULT_WAITING_ENV_VAR], true);
+  state.waitingEnabled = envBoolean(process.env[DEFAULT_WAITING_ENV_VAR], true);
   const maxRetries = envNonNegativeInteger(
     process.env[MAX_RETRY_ENV_VAR],
     DEFAULT_UNKNOWN_ERROR_MAX_RETRIES,