better-dsh 0.2.5 → 0.3.0

package/docs/80_upstream-docs-and-skills/hermes-dev-skill/src/docs/developer-guide/provider-runtime.md

@@ -1,222 +0,0 @@
----
-sidebar_position: 4
-title: "Provider Runtime Resolution"
-description: "How Hermes resolves providers, credentials, API modes, and auxiliary models at runtime"
----
-
-# Provider Runtime Resolution
-
-Hermes has a shared provider runtime resolver used across:
-
-- CLI
-- gateway
-- cron jobs
-- ACP
-- auxiliary model calls
-
-Primary implementation:
-
-- `hermes_cli/runtime_provider.py` — credential resolution, custom-endpoint runtime resolution
-- `hermes_cli/auth.py` — provider registry, `resolve_provider()`
-- `hermes_cli/model_switch.py` — shared `/model` switch pipeline (CLI + gateway)
-- `agent/auxiliary_client.py` — auxiliary model routing
-- `providers/` — ABC + registry entry points (`ProviderProfile`, `register_provider`, `get_provider_profile`, `list_providers`)
-- `plugins/model-providers/<name>/` — per-provider plugins (bundled) that declare `api_mode`, `base_url`, `env_vars`, `fallback_models` and register themselves into the registry on first access. User plugins at `$HERMES_HOME/plugins/model-providers/<name>/` override bundled ones of the same name.
-
-`get_provider_profile()` in `providers/` returns a `ProviderProfile` for a given provider id. `runtime_provider.py` calls this at resolution time to get the canonical `base_url`, `env_vars` priority list, `api_mode`, and `fallback_models` without needing to duplicate that data in multiple files. Adding a new plugin under `plugins/model-providers/<your-provider>/` (or `$HERMES_HOME/plugins/model-providers/<your-provider>/`) that calls `register_provider()` is enough for `runtime_provider.py` to pick it up — no branch needed in the resolver itself.
-
-If you are trying to add a new first-class inference provider, read [Adding Providers](./adding-providers.md) and the [Model Provider Plugin guide](./model-provider-plugin.md) alongside this page.
-
-## Chat-completions reasoning shapes
-
-OpenAI-compatible relays can return `reasoning` or `reasoning_content` as strings,
-text-part dictionaries, or lists of text parts and string fragments. Hermes flattens
-these fields before string operations in the main stream, Relay recording, synchronous
-and asynchronous auxiliary streams, and completed-response reasoning extraction.
-Fragments retain their explicit whitespace; normalization adds no intra-field separator.
-Main-stream and Relay recording retain the existing paragraph breaks between complete
-bold reasoning headings. Reasoning stays separate from the visible answer.
-
-## Resolution precedence
-
-At a high level, provider resolution uses:
-
-1. explicit CLI/runtime request
-2. `config.yaml` model/provider config
-3. environment variables
-4. provider-specific defaults or auto resolution
-
-That ordering matters because Hermes treats the saved model/provider choice as the source of truth for normal runs. This prevents a stale shell export from silently overriding the endpoint a user last selected in `hermes model`.
-
-## Providers
-
-Current provider families include (see `plugins/model-providers/` for the complete bundled set):
-
-- AI Gateway (Vercel)
-- OpenRouter
-- Nous Portal
-- OpenAI Codex
-- Copilot / Copilot ACP
-- Anthropic (native)
-- Google / Gemini (`gemini`)
-- Alibaba / DashScope (`alibaba`, `alibaba-coding-plan`)
-- DeepSeek
-- Z.AI
-- Kimi / Moonshot (`kimi-coding`, `kimi-coding-cn`)
-- MiniMax (`minimax`, `minimax-cn`, `minimax-oauth`)
-- Kilo Code
-- Hugging Face
-- OpenCode Zen / OpenCode Go
-- AWS Bedrock
-- Azure Foundry
-- NVIDIA NIM
-- xAI (Grok)
-- Arcee
-- GMI Cloud
-- StepFun
-- Qwen OAuth
-- Xiaomi
-- Ollama Cloud
-- LM Studio
-- Tencent TokenHub
-- Custom (`provider: custom`) — first-class provider for any OpenAI-compatible endpoint
-- Named custom providers (`providers:` dict in config.yaml; the legacy `custom_providers` list is still read for backward compatibility)
-
-## Output of runtime resolution
-
-The runtime resolver returns data such as:
-
-- `provider`
-- `api_mode`
-- `base_url`
-- `api_key`
-- `source`
-- provider-specific metadata like expiry/refresh info
-
-## Why this matters
-
-This resolver is the main reason Hermes can share auth/runtime logic between:
-
-- `hermes chat`
-- gateway message handling
-- cron jobs running in fresh sessions
-- ACP editor sessions
-- auxiliary model tasks
-
-## AI Gateway
-
-Set `AI_GATEWAY_API_KEY` in `~/.hermes/.env` and run with `--provider ai-gateway`. Hermes fetches available models from the gateway's `/models` endpoint, filtering to language models with tool-use support.
-
-## OpenRouter, AI Gateway, and custom OpenAI-compatible base URLs
-
-Hermes contains logic to avoid leaking the wrong API key to a custom endpoint when multiple provider keys exist (e.g. `OPENROUTER_API_KEY`, `AI_GATEWAY_API_KEY`, and `OPENAI_API_KEY`).
-
-Each provider's API key is scoped to its own base URL:
-
-- `OPENROUTER_API_KEY` is only sent to `openrouter.ai` endpoints
-- `AI_GATEWAY_API_KEY` is only sent to `ai-gateway.vercel.sh` endpoints
-- `OPENAI_API_KEY` is used for custom endpoints and as a fallback
-
-Hermes also distinguishes between:
-
-- a real custom endpoint selected by the user
-- the OpenRouter fallback path used when no custom endpoint is configured
-
-That distinction is especially important for:
-
-- local model servers
-- non-OpenRouter/non-AI Gateway OpenAI-compatible APIs
-- switching providers without re-running setup
-- config-saved custom endpoints that should keep working even when `OPENAI_BASE_URL` is not exported in the current shell
-
-## Native Anthropic path
-
-Anthropic is not just "via OpenRouter" anymore.
-
-When provider resolution selects `anthropic`, Hermes uses:
-
-- `api_mode = anthropic_messages`
-- the native Anthropic Messages API
-- `agent/anthropic_adapter.py` for translation
-
-Credential resolution for native Anthropic now prefers refreshable Claude Code credentials over copied env tokens when both are present. In practice that means:
-
-- Claude Code credential files are treated as the preferred source when they include refreshable auth
-- manual `ANTHROPIC_TOKEN` / `CLAUDE_CODE_OAUTH_TOKEN` values still work as explicit overrides
-- Hermes preflights Anthropic credential refresh before native Messages API calls
-- Hermes still retries once on a 401 after rebuilding the Anthropic client, as a fallback path
-
-## OpenAI Codex path
-
-Codex uses a separate Responses API path:
-
-- `api_mode = codex_responses`
-- dedicated credential resolution and auth store support
-
-## Auxiliary model routing
-
-Auxiliary tasks such as:
-
-- vision
-- web extraction summarization
-- context compression summaries
-- skills hub operations
-- MCP helper operations
-- memory flushes
-
-can use their own provider/model routing rather than the main conversational model.
-
-When an auxiliary task is configured with provider `main`, Hermes resolves that through the same shared runtime path as normal chat. In practice that means:
-
-- env-driven custom endpoints still work
-- custom endpoints saved via `hermes model` / `config.yaml` also work
-- auxiliary routing can tell the difference between a real saved custom endpoint and the OpenRouter fallback
-
-## Fallback models
-
-Hermes supports a configured fallback provider chain — a list of `(provider, model)` entries tried in order when the primary model encounters errors. The legacy single-pair `fallback_model` dict is still accepted for back-compat (and migrated on first write).
-
-### How it works internally
-
-1. **Storage**: `AIAgent.__init__` stores the `fallback_model` dict and sets `_fallback_activated = False`.
-
-2. **Trigger points**: `_try_activate_fallback()` (forwarded to `try_activate_fallback()` in `agent/chat_completion_helpers.py`) is called from three places in the turn phases (`agent/turn_api_error.py`, `agent/turn_response_check.py`, `agent/turn_recovery.py`):
-   - After max retries on invalid API responses (None choices, missing content)
-   - On non-retryable client errors (HTTP 401, 403, 404)
-   - After max retries on transient errors (HTTP 429, 500, 502, 503)
-
-3. **Activation flow** (`_try_activate_fallback`):
-   - Returns `False` immediately if already activated or not configured
-   - Calls `resolve_provider_client()` from `auxiliary_client.py` to build a new client with proper auth
-   - Determines `api_mode`: `codex_responses` for openai-codex, `anthropic_messages` for anthropic, `chat_completions` for everything else
-   - Swaps in-place: `self.model`, `self.provider`, `self.base_url`, `self.api_mode`, `self.client`, `self._client_kwargs`
-   - For anthropic fallback: builds a native Anthropic client instead of OpenAI-compatible
-   - Re-evaluates prompt caching (enabled for Claude models on OpenRouter)
-   - Sets `_fallback_activated = True` — prevents firing again
-   - Resets retry count to 0 and continues the loop
-
-4. **Config flow**:
-   - CLI: reads the fallback chain via `hermes_cli/fallback_config.get_fallback_chain()` → passes to `AIAgent(fallback_model=...)`
-   - Gateway: `gateway/run_config_loaders.py._load_fallback_model()` reads `config.yaml` → passes to `AIAgent`
-   - Validation: both `provider` and `model` keys must be non-empty, or fallback is disabled
-
-### What does NOT support fallback
-
-- **Subagent delegation** (`tools/delegate_tool.py`): subagents inherit the parent's provider but not the fallback config
-- **Auxiliary tasks**: use their own independent provider auto-detection chain (see Auxiliary model routing above)
-
-Cron jobs **do** support fallback: `run_job()` reads `fallback_providers` (or legacy `fallback_model`) from `config.yaml` and passes it to `AIAgent(fallback_model=...)`, matching the gateway's `_load_fallback_model()` pattern. See [Cron Internals](./cron-internals.md).
-
-### Test coverage
-
-Fallback behavior is exercised across several suites:
-
-- `tests/agent/test_fallback_credential_isolation.py` — credential isolation between primary and fallback
-- `tests/hermes_cli/test_fallback_cmd.py` — the `/fallback` CLI command
-- `tests/gateway/test_fallback_eviction.py` — gateway eviction of failed providers
-
-## Related docs
-
-- [Agent Loop Internals](./agent-loop.md)
-- [ACP Internals](./acp-internals.md)
-- [Context Compression & Prompt Caching](./context-compression-and-caching.md)