nodmix 2026.5.25

package/docs/concepts/messages.md

@@ -0,0 +1,214 @@
+---
+summary: "Message flow, sessions, queueing, and reasoning visibility"
+read_when:
+  - Explaining how inbound messages become replies
+  - Clarifying sessions, queueing modes, or streaming behavior
+  - Documenting reasoning visibility and usage implications
+title: "Messages"
+---
+
+Nodmix handles inbound messages through a pipeline of session resolution, queueing, streaming, tool execution, and reasoning visibility. This page maps the path from inbound message to reply.
+
+## Message flow (high level)
+
+```
+Inbound message
+  -> routing/bindings -> session key
+  -> queue (if a run is active)
+  -> agent run (streaming + tools)
+  -> outbound replies (channel limits + chunking)
+```
+
+Key knobs live in configuration:
+
+- `messages.*` for prefixes, queueing, and group behavior.
+- `agents.defaults.*` for block streaming and chunking defaults.
+- Channel overrides (`channels.whatsapp.*`, `channels.telegram.*`, etc.) for caps and streaming toggles.
+
+See [Configuration](/gateway/configuration) for full schema.
+
+## Inbound dedupe
+
+Channels can redeliver the same message after reconnects. Nodmix keeps a
+short-lived cache keyed by channel/account/peer/session/message id so duplicate
+deliveries do not trigger another agent run.
+
+## Inbound debouncing
+
+Rapid consecutive messages from the **same sender** can be batched into a single
+agent turn via `messages.inbound`. Debouncing is scoped per channel + conversation
+and uses the most recent message for reply threading/IDs.
+
+Config (global default + per-channel overrides):
+
+```json5
+{
+  messages: {
+    inbound: {
+      debounceMs: 2000,
+      byChannel: {
+        whatsapp: 5000,
+        slack: 1500,
+        discord: 1500,
+      },
+    },
+  },
+}
+```
+
+Notes:
+
+- Debounce applies to **text-only** messages; media/attachments flush immediately.
+- Control commands bypass debouncing so they remain standalone. Channels that explicitly opt in to same-sender DM coalescing can keep DM commands inside the debounce window so a split-send payload can join the same agent turn.
+
+## Sessions and devices
+
+Sessions are owned by the gateway, not by clients.
+
+- Direct chats collapse into the agent main session key.
+- Groups/channels get their own session keys.
+- The session store and transcripts live on the gateway host.
+
+Multiple devices/channels can map to the same session, but history is not fully
+synced back to every client. Recommendation: use one primary device for long
+conversations to avoid divergent context. The Control UI and TUI always show the
+gateway-backed session transcript, so they are the source of truth.
+
+Details: [Session management](/concepts/session).
+
+## Tool result metadata
+
+Tool result `content` is the model-visible result. Tool result `details` is
+runtime metadata for UI rendering, diagnostics, media delivery, and plugins.
+
+Nodmix keeps that boundary explicit:
+
+- `toolResult.details` is stripped before provider replay and compaction input.
+- Persisted session transcripts keep only bounded `details`; oversized metadata
+  is replaced with a compact summary marked `persistedDetailsTruncated: true`.
+- Plugins and tools should put text the model must read in `content`, not only
+  in `details`.
+
+## Inbound bodies and history context
+
+Nodmix separates the **prompt body** from the **command body**:
+
+- `BodyForAgent`: primary model-facing text for the current message. Channel
+  plugins should keep this focused on the sender's current prompt-bearing text.
+- `Body`: legacy prompt fallback. This may include channel envelopes and
+  optional history wrappers, but current channels should not rely on it as the
+  primary model input when `BodyForAgent` is available.
+- `CommandBody`: raw user text for directive/command parsing.
+- `RawBody`: legacy alias for `CommandBody` (kept for compatibility).
+
+When a channel supplies history, it uses a shared wrapper:
+
+- `[Chat messages since your last reply - for context]`
+- `[Current message - respond to this]`
+
+For **non-direct chats** (groups/channels/rooms), the **current message body** is prefixed with the
+sender label (same style used for history entries). This keeps real-time and queued/history
+messages consistent in the agent prompt.
+
+History buffers are **pending-only**: they include group messages that did _not_
+trigger a run (for example, mention-gated messages) and **exclude** messages
+already in the session transcript.
+
+Directive stripping only applies to the **current message** section so history
+remains intact. Channels that wrap history should set `CommandBody` (or
+`RawBody`) to the original message text and keep `Body` as the combined prompt.
+Structured history, reply, forwarded, and channel metadata are rendered as
+user-role untrusted context blocks during prompt assembly.
+History buffers are configurable via `messages.groupChat.historyLimit` (global
+default) and per-channel overrides like `channels.slack.historyLimit` or
+`channels.telegram.accounts.<id>.historyLimit` (set `0` to disable).
+
+## Queueing and followups
+
+If a run is already active, inbound messages are steered into the current run by
+default. `messages.queue` selects whether active-run messages steer, queue for
+later, collect into one later turn, or interrupt the active run.
+
+- Configure via `messages.queue` (and `messages.queue.byChannel`).
+- Default mode is `steer`, with a 500ms debounce for Codex steering batches and
+  followup/collect queues.
+- Modes: `steer`, `followup`, `collect`, and `interrupt`.
+
+Details: [Command queue](/concepts/queue) and [Steering queue](/concepts/queue-steering).
+
+## Channel run ownership
+
+Channel plugins may preserve ordering, debounce input, and apply transport
+backpressure before a message enters the session queue. They should not impose a
+separate timeout around the agent turn itself. Once a message is routed to a
+session, long-running work is governed by the session, tool, and runtime
+lifecycle so all channels report and recover from slow turns consistently.
+
+## Streaming, chunking, and batching
+
+Block streaming sends partial replies as the model produces text blocks.
+Chunking respects channel text limits and avoids splitting fenced code.
+
+Key settings:
+
+- `agents.defaults.blockStreamingDefault` (`on|off`, default off)
+- `agents.defaults.blockStreamingBreak` (`text_end|message_end`)
+- `agents.defaults.blockStreamingChunk` (`minChars|maxChars|breakPreference`)
+- `agents.defaults.blockStreamingCoalesce` (idle-based batching)
+- `agents.defaults.humanDelay` (human-like pause between block replies)
+- Channel overrides: `*.blockStreaming` and `*.blockStreamingCoalesce` (non-Telegram channels require explicit `*.blockStreaming: true`)
+
+Details: [Streaming + chunking](/concepts/streaming).
+
+## Reasoning visibility and tokens
+
+Nodmix can expose or hide model reasoning:
+
+- `/reasoning on|off|stream` controls visibility.
+- Reasoning content still counts toward token usage when produced by the model.
+- Telegram supports reasoning stream into a transient draft bubble that is deleted after final delivery; use `/reasoning on` for persistent reasoning output.
+
+Details: [Thinking + reasoning directives](/tools/thinking) and [Token use](/reference/token-use).
+
+## Prefixes, threading, and replies
+
+Outbound message formatting is centralized in `messages`:
+
+- `messages.responsePrefix`, `channels.<channel>.responsePrefix`, and `channels.<channel>.accounts.<id>.responsePrefix` (outbound prefix cascade), plus `channels.whatsapp.messagePrefix` (WhatsApp inbound prefix)
+- Reply threading via `replyToMode` and per-channel defaults
+
+Details: [Configuration](/gateway/config-agents#messages) and channel docs.
+
+## Silent replies
+
+The exact silent token `NO_REPLY` / `no_reply` means "do not deliver a user-visible reply".
+When a turn also has pending tool media, such as generated TTS audio, Nodmix
+strips the silent text but still delivers the media attachment.
+Nodmix resolves that behavior by conversation type:
+
+- Direct conversations never receive `NO_REPLY` prompt guidance. If a direct
+  run accidentally returns a bare silent token, Nodmix suppresses it instead
+  of rewriting or delivering it.
+- Groups/channels allow silence by default only for automatic group replies.
+  In `message_tool` visible-reply mode, silence means the model does not call
+  `message(action=send)`.
+- Internal orchestration allows silence by default.
+
+Nodmix also uses silent replies for internal runner failures that happen
+before any assistant reply in non-direct chats, so groups/channels do not see
+gateway error boilerplate. Direct chats show compact failure copy by default;
+raw runner details are shown only when `/verbose full` is enabled.
+
+Defaults live under `agents.defaults.silentReply`; `surfaces.<id>.silentReply`
+can override group/internal policy per surface.
+
+Bare silent replies are dropped on all surfaces, so parent sessions stay quiet
+instead of rewriting sentinel text into fallback chatter.
+
+## Related
+
+- [Message lifecycle refactor](/concepts/message-lifecycle-refactor) - target durable send and receive design
+- [Streaming](/concepts/streaming) — real-time message delivery
+- [Retry](/concepts/retry) — message delivery retry behavior
+- [Queue](/concepts/queue) — message processing queue
+- [Channels](/channels) — messaging platform integrations

package/docs/concepts/model-failover.md

@@ -0,0 +1,385 @@
+---
+summary: "How Nodmix rotates auth profiles and falls back across models"
+read_when:
+  - Diagnosing auth profile rotation, cooldowns, or model fallback behavior
+  - Updating failover rules for auth profiles or models
+  - Understanding how session model overrides interact with fallback retries
+title: "Model failover"
+sidebarTitle: "Model failover"
+---
+
+Nodmix handles failures in two stages:
+
+1. **Auth profile rotation** within the current provider.
+2. **Model fallback** to the next model in `agents.defaults.model.fallbacks`.
+
+This doc explains the runtime rules and the data that backs them.
+
+## Runtime flow
+
+For a normal text run, Nodmix evaluates candidates in this order:
+
+<Steps>
+  <Step title="Resolve session state">
+    Resolve the active session model and auth-profile preference.
+  </Step>
+  <Step title="Build candidate chain">
+    Build the model candidate chain from the current model selection and the fallback policy for that selection source. Configured defaults, cron job primaries, and auto-selected fallback models can use configured fallbacks; explicit user session selections are strict.
+  </Step>
+  <Step title="Try the current provider">
+    Try the current provider with auth-profile rotation/cooldown rules.
+  </Step>
+  <Step title="Advance on failover-worthy errors">
+    If that provider is exhausted with a failover-worthy error, move to the next model candidate.
+  </Step>
+  <Step title="Persist fallback override">
+    Persist the selected fallback override before the retry starts so other session readers see the same provider/model the runner is about to use. The persisted model override is marked `modelOverrideSource: "auto"`.
+  </Step>
+  <Step title="Roll back narrowly on failure">
+    If the fallback candidate fails, roll back only the fallback-owned session override fields when they still match that failed candidate.
+  </Step>
+  <Step title="Throw FallbackSummaryError if exhausted">
+    If every candidate fails, throw a `FallbackSummaryError` with per-attempt detail and the soonest cooldown expiry when one is known.
+  </Step>
+</Steps>
+
+This is intentionally narrower than "save and restore the whole session". The reply runner only persists the model-selection fields it owns for fallback:
+
+- `providerOverride`
+- `modelOverride`
+- `modelOverrideSource`
+- `authProfileOverride`
+- `authProfileOverrideSource`
+- `authProfileOverrideCompactionCount`
+
+That prevents a failed fallback retry from overwriting newer unrelated session mutations such as manual `/model` changes or session rotation updates that happened while the attempt was running.
+
+## Selection source policy
+
+Nodmix separates the selected provider/model from why it was selected. That source controls whether the fallback chain is allowed:
+
+- **Configured default**: `agents.defaults.model.primary` uses `agents.defaults.model.fallbacks`.
+- **Agent primary**: `agents.list[].model` is strict unless that agent model object includes its own `fallbacks`. Use `fallbacks: []` to make the strict behavior explicit, or provide a non-empty list to opt that agent into model fallback.
+- **Auto fallback override**: a runtime fallback writes `providerOverride`, `modelOverride`, `modelOverrideSource: "auto"`, and the selected origin model before retrying. That auto override can keep walking the configured fallback chain without probing the primary on every message, but Nodmix periodically probes the configured origin again and clears the auto override when it recovers. `/new`, `/reset`, and `sessions.reset` also clear auto-sourced overrides. Heartbeat runs without an explicit `heartbeat.model` clear direct auto overrides when their origin no longer matches the current configured default.
+- **User session override**: `/model`, the model picker, `session_status(model=...)`, and `sessions.patch` write `modelOverrideSource: "user"`. That is an exact session selection. If the selected provider/model fails before producing a reply, Nodmix reports the failure instead of answering from an unrelated configured fallback.
+- **Legacy session override**: older session entries may have `modelOverride` without `modelOverrideSource`. Nodmix treats those as user overrides so an explicit old selection is not silently converted into fallback behavior.
+- **Cron payload model**: a cron job `payload.model` / `--model` is a job primary, not a user session override. It uses configured fallbacks unless the job provides `payload.fallbacks`; `payload.fallbacks: []` makes the cron run strict.
+
+The auto fallback primary-probe interval is five minutes and is not configurable. Nodmix remembers recent probes per session and primary model so a failing primary is not retried on every turn. Nodmix sends a visible notice when a session moves onto fallback and another notice when it returns to the selected primary; it does not repeat the notice on every sticky fallback turn.
+
+## User-visible fallback notices
+
+When a session moves onto an auto-selected fallback, Nodmix sends a status notice in the same reply surface:
+
+```text
+↪️ Model Fallback: <fallback> (selected <primary>; <reason>)
+```
+
+When a later probe succeeds and the session returns to the selected primary, Nodmix sends:
+
+```text
+↪️ Model Fallback cleared: <primary> (was <fallback>)
+```
+
+These notices are operational messages, not assistant content. They are delivered once per state change, including side-effect-only turns when feasible, but sticky fallback turns do not repeat them. Delivery bypasses normal source-reply suppression, the notice does not consume the first assistant reply slot for threaded channels, and it is excluded from text-to-speech and commitment extraction.
+
+## Auth storage (keys + OAuth)
+
+Nodmix uses **auth profiles** for both API keys and OAuth tokens.
+
+- Secrets live in `~/.nodmix/agents/<agentId>/agent/auth-profiles.json` (legacy: `~/.nodmix/agent/auth-profiles.json`).
+- Runtime auth-routing state lives in `~/.nodmix/agents/<agentId>/agent/auth-state.json`.
+- Config `auth.profiles` / `auth.order` are **metadata + routing only** (no secrets).
+- Legacy import-only OAuth file: `~/.nodmix/credentials/oauth.json` (imported into `auth-profiles.json` on first use).
+
+More detail: [OAuth](/concepts/oauth)
+
+Credential types:
+
+- `type: "api_key"` → `{ provider, key }`
+- `type: "oauth"` → `{ provider, access, refresh, expires, email? }` (+ `projectId`/`enterpriseUrl` for some providers)
+
+## Profile IDs
+
+OAuth logins create distinct profiles so multiple accounts can coexist.
+
+- Default: `provider:default` when no email is available.
+- OAuth with email: `provider:<email>` (for example `google-antigravity:user@gmail.com`).
+
+Profiles live in `~/.nodmix/agents/<agentId>/agent/auth-profiles.json` under `profiles`.
+
+## Rotation order
+
+When a provider has multiple profiles, Nodmix chooses an order like this:
+
+<Steps>
+  <Step title="Explicit config">
+    `auth.order[provider]` (if set).
+  </Step>
+  <Step title="Configured profiles">
+    `auth.profiles` filtered by provider.
+  </Step>
+  <Step title="Stored profiles">
+    Entries in `auth-profiles.json` for the provider.
+  </Step>
+</Steps>
+
+If no explicit order is configured, Nodmix uses a round-robin order:
+
+- **Primary key:** profile type (**OAuth before API keys**).
+- **Secondary key:** `usageStats.lastUsed` (oldest first, within each type).
+- **Cooldown/disabled profiles** are moved to the end, ordered by soonest expiry.
+
+### Session stickiness (cache-friendly)
+
+Nodmix **pins the chosen auth profile per session** to keep provider caches warm. It does **not** rotate on every request. The pinned profile is reused until:
+
+- the session is reset (`/new` / `/reset`)
+- a compaction completes (compaction count increments)
+- the profile is in cooldown/disabled
+
+Manual selection via `/model …@<profileId>` sets a **user override** for that session and is not auto-rotated until a new session starts.
+
+<Note>
+Auto-pinned profiles (selected by the session router) are treated as a **preference**: they are tried first, but Nodmix may rotate to another profile on rate limits/timeouts. When the original profile becomes available again, new runs can prefer it again without changing the selected model or runtime. User-pinned profiles stay locked to that profile; if it fails and model fallbacks are configured, Nodmix moves to the next model instead of switching profiles.
+</Note>
+
+### OpenAI Codex subscription plus API-key backup
+
+For OpenAI agent models, auth and runtime are separate. `openai/gpt-*` stays on
+the Codex harness while auth can rotate between a Codex subscription profile and
+an OpenAI API-key backup.
+
+Use `auth.order.openai` for the user-facing order:
+
+```json5
+{
+  auth: {
+    order: {
+      openai: ["openai-codex:user@example.com", "openai:api-key-backup"],
+    },
+  },
+}
+```
+
+Existing Codex subscription profiles may still use the legacy
+`openai-codex:*` profile id. The ordered API-key backup can be a normal
+`openai:*` API-key profile. When the subscription hits a Codex usage limit,
+Nodmix records the exact reset time when Codex provides one, tries the next
+ordered auth profile, and keeps the run inside the Codex harness. Once the reset
+time passes, the subscription profile is eligible again and the next automatic
+selection can return to it.
+
+Use a user-pinned profile only when you want to force one account/key for that
+session. User-pinned profiles are intentionally strict and do not silently jump
+to another profile.
+
+## Cooldowns
+
+When a profile fails due to auth/rate-limit errors (or a timeout that looks like rate limiting), Nodmix marks it in cooldown and moves to the next profile.
+
+<AccordionGroup>
+  <Accordion title="What lands in the rate-limit / timeout bucket">
+    That rate-limit bucket is broader than plain `429`: it also includes provider messages such as `Too many concurrent requests`, `ThrottlingException`, `concurrency limit reached`, `workers_ai ... quota limit exceeded`, `throttled`, `resource exhausted`, and periodic usage-window limits such as `weekly/monthly limit reached`.
+
+    Format/invalid-request errors are usually terminal because retrying the same payload would fail the same way, so Nodmix surfaces them instead of rotating auth profiles. Known retry-repair paths can opt in explicitly: for example Cloud Code Assist tool call ID validation failures are sanitized and retried once through the `allowFormatRetry` policy. OpenAI-compatible stop-reason errors such as `Unhandled stop reason: error`, `stop reason: error`, and `reason: error` are classified as timeout/failover signals.
+
+    Generic server text can also land in that timeout bucket when the source matches a known transient pattern. For example, the bare pi-ai stream-wrapper message `An unknown error occurred` is treated as failover-worthy for every provider because pi-ai emits it when provider streams end with `stopReason: "aborted"` or `stopReason: "error"` without specific details. JSON `api_error` payloads with transient server text such as `internal server error`, `unknown error, 520`, `upstream error`, or `backend error` are also treated as failover-worthy timeouts.
+
+    OpenRouter-specific generic upstream text such as bare `Provider returned error` is treated as timeout only when the provider context is actually OpenRouter. Generic internal fallback text such as `LLM request failed with an unknown error.` stays conservative and does not trigger failover by itself.
+
+  </Accordion>
+  <Accordion title="SDK retry-after caps">
+    Some provider SDKs may otherwise sleep for a long `Retry-After` window before returning control to Nodmix. For Stainless-based SDKs such as Anthropic and OpenAI, Nodmix caps SDK-internal `retry-after-ms` / `retry-after` waits at 60 seconds by default and surfaces longer retryable responses immediately so this failover path can run. Tune or disable the cap with `NODMIX_SDK_RETRY_MAX_WAIT_SECONDS`; see [Retry behavior](/concepts/retry).
+  </Accordion>
+  <Accordion title="Model-scoped cooldowns">
+    Rate-limit cooldowns can also be model-scoped:
+
+    - Nodmix records `cooldownModel` for rate-limit failures when the failing model id is known.
+    - A sibling model on the same provider can still be tried when the cooldown is scoped to a different model.
+    - Billing/disabled windows still block the whole profile across models.
+
+  </Accordion>
+</AccordionGroup>
+
+Cooldowns use exponential backoff:
+
+- 1 minute
+- 5 minutes
+- 25 minutes
+- 1 hour (cap)
+
+State is stored in `auth-state.json` under `usageStats`:
+
+```json
+{
+  "usageStats": {
+    "provider:profile": {
+      "lastUsed": 1736160000000,
+      "cooldownUntil": 1736160600000,
+      "errorCount": 2
+    }
+  }
+}
+```
+
+## Billing disables
+
+Billing/credit failures (for example "insufficient credits" / "credit balance too low") are treated as failover-worthy, but they're usually not transient. Instead of a short cooldown, Nodmix marks the profile as **disabled** (with a longer backoff) and rotates to the next profile/provider.
+
+<Note>
+Not every billing-shaped response is `402`, and not every HTTP `402` lands here. Nodmix keeps explicit billing text in the billing lane even when a provider returns `401` or `403` instead, but provider-specific matchers stay scoped to the provider that owns them (for example OpenRouter `403 Key limit exceeded`).
+
+Meanwhile temporary `402` usage-window and organization/workspace spend-limit errors are classified as `rate_limit` when the message looks retryable (for example `weekly usage limit exhausted`, `daily limit reached, resets tomorrow`, or `organization spending limit exceeded`). Those stay on the short cooldown/failover path instead of the long billing-disable path.
+</Note>
+
+State is stored in `auth-state.json`:
+
+```json
+{
+  "usageStats": {
+    "provider:profile": {
+      "disabledUntil": 1736178000000,
+      "disabledReason": "billing"
+    }
+  }
+}
+```
+
+Defaults:
+
+- Billing backoff starts at **5 hours**, doubles per billing failure, and caps at **24 hours**.
+- Backoff counters reset if the profile hasn't failed for **24 hours** (configurable).
+- Overloaded retries allow **1 same-provider profile rotation** before model fallback.
+- Overloaded retries use **0 ms backoff** by default.
+
+## Model fallback
+
+If all profiles for a provider fail, Nodmix moves to the next model in `agents.defaults.model.fallbacks`. This applies to auth failures, rate limits, and timeouts that exhausted profile rotation (other errors do not advance fallback). Provider errors that do not expose enough detail are still labeled precisely in fallback state: `empty_response` means the provider returned no usable message or status, `no_error_details` means the provider explicitly returned `Unknown error (no error details in response)`, and `unclassified` means Nodmix preserved the raw preview but no classifier matched it yet.
+
+Overloaded and rate-limit errors are handled more aggressively than billing cooldowns. By default, Nodmix allows one same-provider auth-profile retry, then switches to the next configured model fallback without waiting. Provider-busy signals such as `ModelNotReadyException` land in that overloaded bucket. Tune this with `auth.cooldowns.overloadedProfileRotations`, `auth.cooldowns.overloadedBackoffMs`, and `auth.cooldowns.rateLimitedProfileRotations`.
+
+When a run starts from the configured default primary, a cron job primary, an agent primary with explicit fallbacks, or an auto-selected fallback override, Nodmix can walk the matching configured fallback chain. Agent primaries without explicit fallbacks and explicit user selections (for example `/model ollama/qwen3.5:27b`, the model picker, `sessions.patch`, or one-off CLI provider/model overrides) are strict: if that provider/model is unreachable or fails before producing a reply, Nodmix reports the failure instead of answering from an unrelated fallback.
+
+### Candidate chain rules
+
+Nodmix builds the candidate list from the currently requested `provider/model` plus configured fallbacks.
+
+<AccordionGroup>
+  <Accordion title="Rules">
+    - The requested model is always first.
+    - Explicit configured fallbacks are deduplicated but not filtered by the model allowlist. They are treated as explicit operator intent.
+    - If the current run is already on a configured fallback in the same provider family, Nodmix keeps using the full configured chain.
+    - When no explicit fallback override is supplied, configured fallbacks are tried before the configured primary even if the requested model uses a different provider.
+    - When no explicit fallback override is supplied to the fallback runner, the configured primary is appended at the end so the chain can settle back onto the normal default once earlier candidates are exhausted.
+    - When a caller supplies `fallbacksOverride`, the runner uses exactly the requested model plus that override list. An empty list disables model fallback and prevents the configured primary from being appended as a hidden retry target.
+
+  </Accordion>
+</AccordionGroup>
+
+### Which errors advance fallback
+
+<Tabs>
+  <Tab title="Continues on">
+    - auth failures
+    - rate limits and cooldown exhaustion
+    - overloaded/provider-busy errors
+    - timeout-shaped failover errors
+    - billing disables
+    - `LiveSessionModelSwitchError`, which is normalized into a failover path so a stale persisted model does not create an outer retry loop
+    - other unrecognized errors when there are still remaining candidates
+
+  </Tab>
+  <Tab title="Does not continue on">
+    - explicit aborts that are not timeout/failover-shaped
+    - context overflow errors that should stay inside compaction/retry logic (for example `request_too_large`, `INVALID_ARGUMENT: input exceeds the maximum number of tokens`, `input token count exceeds the maximum number of input tokens`, `The input is too long for the model`, or `ollama error: context length exceeded`)
+    - a final unknown error when there are no candidates left
+
+  </Tab>
+</Tabs>
+
+### Cooldown skip vs probe behavior
+
+When every auth profile for a provider is already in cooldown, Nodmix does not automatically skip that provider forever. It makes a per-candidate decision:
+
+<AccordionGroup>
+  <Accordion title="Per-candidate decisions">
+    - Persistent auth failures skip the whole provider immediately.
+    - Billing disables usually skip, but the primary candidate can still be probed on a throttle so recovery is possible without restarting.
+    - The primary candidate may be probed near cooldown expiry, with a per-provider throttle.
+    - Same-provider fallback siblings can be attempted despite cooldown when the failure looks transient (`rate_limit`, `overloaded`, or unknown). This is especially relevant when a rate limit is model-scoped and a sibling model may still recover immediately.
+    - Transient cooldown probes are limited to one per provider per fallback run so a single provider does not stall cross-provider fallback.
+
+  </Accordion>
+</AccordionGroup>
+
+## Session overrides and live model switching
+
+Session model changes are shared state. The active runner, `/model` command, compaction/session updates, and live-session reconciliation all read or write parts of the same session entry.
+
+That means fallback retries have to coordinate with live model switching:
+
+- Only explicit user-driven model changes mark a pending live switch. That includes `/model`, `session_status(model=...)`, and `sessions.patch`.
+- System-driven model changes such as fallback rotation, heartbeat overrides, or compaction never mark a pending live switch on their own.
+- User-driven model overrides are treated as exact selections for fallback policy, so an unreachable selected provider surfaces as a failure instead of being masked by `agents.defaults.model.fallbacks`.
+- Before a fallback retry starts, the reply runner persists the selected fallback override fields to the session entry.
+- Auto fallback overrides remain selected on subsequent turns so Nodmix does not probe a known-bad primary on every message. Nodmix periodically probes the configured origin again and clears the auto override when it recovers; `/new`, `/reset`, and `sessions.reset` clear auto-sourced overrides immediately.
+- User replies announce fallback transitions and fallback-cleared recovery once per state change. Sticky fallback turns do not repeat the notice.
+- `/status` shows the selected model and, when fallback state differs, the active fallback model and reason.
+- Live-session reconciliation prefers persisted session overrides over stale runtime model fields.
+- If a live-switch error points at a later candidate in the active fallback chain, Nodmix jumps directly to that selected model instead of walking unrelated candidates first.
+- If the fallback attempt fails, the runner rolls back only the override fields it wrote, and only if they still match that failed candidate.
+
+This prevents the classic race:
+
+<Steps>
+  <Step title="Primary fails">
+    The selected primary model fails.
+  </Step>
+  <Step title="Fallback chosen in memory">
+    Fallback candidate is chosen in memory.
+  </Step>
+  <Step title="Session store still says old primary">
+    Session store still reflects the old primary.
+  </Step>
+  <Step title="Live reconciliation reads stale state">
+    Live-session reconciliation reads the stale session state.
+  </Step>
+  <Step title="Retry snapped back">
+    The retry gets snapped back to the old model before the fallback attempt starts.
+  </Step>
+</Steps>
+
+The persisted fallback override closes that window, and the narrow rollback keeps newer manual or runtime session changes intact.
+
+## Observability and failure summaries
+
+`runWithModelFallback(...)` records per-attempt details that feed logs and user-facing cooldown messaging:
+
+- provider/model attempted
+- reason (`rate_limit`, `overloaded`, `billing`, `auth`, `model_not_found`, and similar failover reasons)
+- optional status/code
+- human-readable error summary
+
+Structured `model_fallback_decision` logs also include flat `fallbackStep*` fields when a candidate fails, is skipped, or a later fallback succeeds. These fields make the attempted transition explicit (`fallbackStepFromModel`, `fallbackStepToModel`, `fallbackStepFromFailureReason`, `fallbackStepFromFailureDetail`, `fallbackStepFinalOutcome`) so log and diagnostic exporters can reconstruct the primary failure even when the terminal fallback also fails.
+
+When every candidate fails, Nodmix throws `FallbackSummaryError`. The outer reply runner can use that to build a more specific message such as "all models are temporarily rate-limited" and include the soonest cooldown expiry when one is known.
+
+That cooldown summary is model-aware:
+
+- unrelated model-scoped rate limits are ignored for the attempted provider/model chain
+- if the remaining block is a matching model-scoped rate limit, Nodmix reports the last matching expiry that still blocks that model
+
+## Related config
+
+See [Gateway configuration](/gateway/configuration) for:
+
+- `auth.profiles` / `auth.order`
+- `auth.cooldowns.billingBackoffHours` / `auth.cooldowns.billingBackoffHoursByProvider`
+- `auth.cooldowns.billingMaxHours` / `auth.cooldowns.failureWindowHours`
+- `auth.cooldowns.overloadedProfileRotations` / `auth.cooldowns.overloadedBackoffMs`
+- `auth.cooldowns.rateLimitedProfileRotations`
+- `agents.defaults.model.primary` / `agents.defaults.model.fallbacks`
+- `agents.defaults.imageModel` routing
+
+See [Models](/concepts/models) for the broader model selection and fallback overview.