@maxmo74/pi-coding-agent 0.0.0-stage → 1.4.0-max.1

package/docs/models-configuration.md

@@ -0,0 +1,196 @@
+# Models Configuration
+
+<!-- This file is generated from TypeScript source. Do not edit it manually; run npm run generate:references. -->
+
+This reference lists the fields of `models.json`. See [Configure a compatible endpoint](models.md#configure-a-compatible-endpoint) for examples and [Configuration](configuration.md#agent-directory) for the file location.
+
+Add `"$schema": "https://pi.dev/schemas/models.schema.json"` to `models.json` for editor validation and completion.
+
+## Providers
+
+Each key under `providers` is a provider ID. A built-in provider ID changes that provider. Any other ID adds a provider, which needs `baseUrl` and an `api` at provider or model level.
+
+| Field | Type | Default | Description |
+|---|---|---|---|
+| `name` | string |  | Display name. |
+| `baseUrl` | string |  | API base URL. Also applies to the provider's built-in models. |
+| `apiKey` | string |  | Literal key, "$NAME" environment interpolation, or "!command" output. |
+| `api` | string |  | API for the provider's models, such as "openai-completions", "openai-responses", "anthropic-messages", or "google-generative-ai". |
+| `oauth` | `"radius"` |  | OAuth login flow. Requires baseUrl. |
+| `headers` | object |  | Request headers. Values resolve like apiKey. |
+| `compat` | [object](#compatibility) |  | Provider and model compatibility overrides. |
+| `authHeader` | boolean | `false` | Whether to send the resolved API key as Authorization: Bearer. |
+| `models` | [`object[]`](#models) |  |  |
+| `modelOverrides` | [object](#model-overrides) |  |  |
+
+## Models
+
+Each `models` entry adds a model to the provider. An entry whose `id` matches an existing model replaces it.
+
+| Field | Type | Default | Description |
+|---|---|---|---|
+| `id` | string |  | Required. Model ID sent to the API. |
+| `name` | string |  | Display name, also matched by --model patterns. |
+| `api` | string |  | Overrides the provider api. |
+| `baseUrl` | string |  | Overrides the provider baseUrl. |
+| `reasoning` | boolean | `false` | Whether the model supports thinking. |
+| `thinkingLevelMap` | object |  | Provider value sent for each thinking level. null marks a level unsupported. Levels the model does not map use the provider's default value through high; xhigh and max are unsupported. |
+| `input` | `("text" \| "image")[]` | `["text"]` |  |
+| `inputLimits` | [object](#input-limits) |  |  |
+| `cost` | object |  |  |
+| `cost.input` | number |  | Required. Input cost in USD per million tokens. |
+| `cost.output` | number |  | Required. Output cost in USD per million tokens. |
+| `cost.cacheRead` | number |  | Required. Cache-read cost in USD per million tokens. |
+| `cost.cacheWrite` | number |  | Required. Cache-write cost in USD per million tokens. |
+| `cost.tiers` | `object[]` |  | Request-wide pricing tiers. The highest matching input threshold applies to the full request. |
+| `cost.tiers[].inputTokensAbove` | number |  | Required. Use this tier when total request input exceeds this token count. |
+| `cost.tiers[].input` | number |  | Required. Input cost in USD per million tokens. |
+| `cost.tiers[].output` | number |  | Required. Output cost in USD per million tokens. |
+| `cost.tiers[].cacheRead` | number |  | Required. Cache-read cost in USD per million tokens. |
+| `cost.tiers[].cacheWrite` | number |  | Required. Cache-write cost in USD per million tokens. |
+| `promptCache` | object |  | Best-effort prompt cache lifetime in seconds for the short and long retention tiers. Pi warms only tiers with a known lifetime. |
+| `contextWindow` | number | `128000` | Context window in tokens. |
+| `maxTokens` | number | `16384` | Maximum output tokens. |
+| `samplingParams` | object |  | Fields merged into every request body. Applies only to OpenAI-compatible APIs. |
+| `samplingParamsByThinkingLevel` | object |  | samplingParams per thinking level, merged over samplingParams. |
+| `headers` | object |  | Request headers. Values resolve like apiKey. |
+| `compat` | [object](#compatibility) |  | Provider and model compatibility overrides. |
+
+## Model overrides
+
+Each `modelOverrides` key is the ID of a built-in or extension model. Unknown IDs are ignored. Object fields, such as `cost`, merge with the model's value field by field; other fields replace it.
+
+| Field | Type | Default | Description |
+|---|---|---|---|
+| `name` | string |  | Display name, also matched by --model patterns. |
+| `reasoning` | boolean |  | Whether the model supports thinking. |
+| `thinkingLevelMap` | object |  | Provider value sent for each thinking level. null marks a level unsupported. Levels the model does not map use the provider's default value through high; xhigh and max are unsupported. |
+| `input` | `("text" \| "image")[]` |  |  |
+| `inputLimits` | [object](#input-limits) |  |  |
+| `cost` | object |  |  |
+| `cost.input` | number |  | Input cost in USD per million tokens. |
+| `cost.output` | number |  | Output cost in USD per million tokens. |
+| `cost.cacheRead` | number |  | Cache-read cost in USD per million tokens. |
+| `cost.cacheWrite` | number |  | Cache-write cost in USD per million tokens. |
+| `cost.tiers` | `object[]` |  | Request-wide pricing tiers. The highest matching input threshold applies to the full request. |
+| `cost.tiers[].inputTokensAbove` | number |  | Required. Use this tier when total request input exceeds this token count. |
+| `cost.tiers[].input` | number |  | Required. Input cost in USD per million tokens. |
+| `cost.tiers[].output` | number |  | Required. Output cost in USD per million tokens. |
+| `cost.tiers[].cacheRead` | number |  | Required. Cache-read cost in USD per million tokens. |
+| `cost.tiers[].cacheWrite` | number |  | Required. Cache-write cost in USD per million tokens. |
+| `promptCache` | object |  | Best-effort prompt cache lifetime in seconds for the short and long retention tiers. Pi warms only tiers with a known lifetime. |
+| `contextWindow` | number |  | Context window in tokens. |
+| `maxTokens` | number |  | Maximum output tokens. |
+| `samplingParams` | object |  | Fields merged into every request body. Applies only to OpenAI-compatible APIs. |
+| `samplingParamsByThinkingLevel` | object |  | samplingParams per thinking level, merged over samplingParams. |
+| `headers` | object |  | Request headers. Values resolve like apiKey. |
+| `compat` | [object](#compatibility) |  | Provider and model compatibility overrides. |
+
+## Input limits
+
+See [Describe model input and caching](models.md#describe-model-input-and-caching).
+
+| Field | Type | Default | Description |
+|---|---|---|---|
+| `maxRequestBytes` | integer |  | Maximum serialized provider request size in bytes. |
+| `images` | object |  |  |
+| `images.resize` | object |  | Cache-safe resize profile applied before a new image enters conversation history. |
+| `images.resize.maxWidth` | integer |  |  |
+| `images.resize.maxHeight` | integer |  |  |
+| `images.resize.maxBytes` | integer |  | Maximum base64-encoded payload size in bytes. |
+| `images.resize.jpegQuality` | integer |  |  |
+| `images.maxPerMessage` | integer |  | Maximum images accepted in one provider message. |
+| `images.maxPerRequest` | integer |  | Maximum images accepted across one provider request. |
+
+## Compatibility
+
+`compat` describes verified differences from an API's standard request and response behavior. Model values override provider values per field. The fields that apply depend on the model's `api`.
+
+### `openai-completions`
+
+| Field | Type | Default | Description |
+|---|---|---|---|
+| `supportsStore` | boolean |  | Whether the provider supports the store field. Default: auto-detected from URL. |
+| `supportsDeveloperRole` | boolean |  | Whether the provider supports the developer role instead of system. Default: auto-detected from URL. |
+| `supportsReasoningEffort` | boolean |  | Whether the provider supports reasoning_effort. Default: auto-detected from URL. |
+| `supportsUsageInStreaming` | boolean | `true` | Whether the provider supports stream_options.include_usage for token usage in streaming responses. |
+| `supportsFinishReason` | boolean | `true` | Whether streamed responses include finish_reason. When false, pi infers stop or toolUse when the stream ends. |
+| `maxTokensField` | `"max_completion_tokens" \| "max_tokens"` |  | Which field to use for max tokens. Default: auto-detected from URL. |
+| `requiresToolResultName` | boolean |  | Whether tool results require the name field. Default: auto-detected from URL. |
+| `requiresAssistantAfterToolResult` | boolean |  | Whether a user message after tool results requires an assistant message in between. Default: auto-detected from URL. |
+| `requiresThinkingAsText` | boolean |  | Whether thinking blocks must be converted to text blocks with &lt;thinking> delimiters. Default: auto-detected from URL. |
+| `requiresReasoningContentOnAssistantMessages` | boolean |  | Whether all replayed assistant messages must include an empty reasoning_content field when reasoning is enabled. Default: auto-detected from URL. |
+| `thinkingFormat` | `"openai" \| "openrouter" \| "deepseek" \| "together" \| "baseten" \| "zai" \| "qwen" \| "chat-template" \| "qwen-chat-template" \| "string-thinking" \| "ant-ling"` |  | Format for reasoning or thinking parameters. When omitted, Pi auto-detects the format from the provider URL. "openai" uses reasoning_effort, "openrouter" uses reasoning.effort, "deepseek" uses thinking.type plus reasoning_effort when supported, "together" uses reasoning.enabled plus reasoning_effort when supported, "baseten" uses configurable chat_template_args plus reasoning_effort when supported, "zai" uses thinking.type, "qwen" uses top-level enable_thinking, "qwen-chat-template" uses chat_template_kwargs.enable_thinking and preserve_thinking, "chat-template" uses configurable chat_template_kwargs, "string-thinking" uses top-level thinking, and "ant-ling" uses reasoning.effort only when the mapped effort is non-null. |
+| `chatTemplateKwargs` | object |  | Kwargs sent as chat_template_kwargs when thinkingFormat is "chat-template". Use $var with "thinking.enabled", "thinking.effort", or "thinking.budget" for pi-controlled values. |
+| `chatTemplateArgs` | object |  | Arguments sent as chat_template_args when thinkingFormat is "baseten". Use $var with "thinking.enabled", "thinking.effort", or "thinking.budget" for pi-controlled values. |
+| `openRouterRouting` | object |  | OpenRouter provider routing preferences. Controls which upstream providers OpenRouter routes requests to. Sent as the provider field in the OpenRouter API request body. See https://openrouter.ai/docs/guides/routing/provider-selection. |
+| `vercelGatewayRouting` | object |  | Vercel AI Gateway routing preferences. Controls which upstream providers the gateway routes requests to. See https://vercel.com/docs/ai-gateway/models-and-providers/provider-options. |
+| `zaiToolStream` | boolean | `false` | Whether z.ai supports top-level tool_stream for streaming tool call deltas. |
+| `thinkingTokenBudgetField` | `"thinking_token_budget" \| "thinking_budget" \| "thinking_budget_tokens"` |  | Top-level request field used to cap reasoning tokens from thinkingBudgets. Reasoning and the answer share max_tokens on these endpoints. "thinking_token_budget" is vLLM, "thinking_budget" is Qwen, DashScope, or SGLang, and "thinking_budget_tokens" is llama.cpp. Off by default and not set on the generated catalog. |
+| `supportsThinkingTokenBudget` | boolean | `false` | Alias for thinkingTokenBudgetField: "thinking_token_budget" (vLLM). Prefer thinkingTokenBudgetField. |
+| `supportsOpenAIGrammarTools` | boolean | `false` | Whether the provider supports OpenAI custom tools with Lark or regex grammar formats. When false, grammar-constrained tools fall back to normal function tools. The generated catalog enables this for capable models. |
+| `supportsMidConvoSystemMessages` | boolean | `false` | Whether the exact model accepts system or developer messages after the conversation has started. When false, later system messages are folded into the leading system message. The generated catalog enables this for verified models. |
+| `supportsMidConvoToolAdditions` | boolean | `false` | Whether system messages can introduce additional tools mid-conversation. Requires supportsMidConvoSystemMessages. The generated catalog enables this for capable models. |
+| `supportsStrictMode` | boolean | `false` | Whether the provider supports the strict field in tool definitions. Generated capable models enable it explicitly. |
+| `cacheControlFormat` | `"anthropic"` |  | Cache control convention for prompt caching. Anthropic applies cache_control markers to the system prompt, last tool definition, and last user, assistant, or tool-result text content. |
+| `sendSessionAffinityHeaders` | boolean |  | Whether to send session-affinity data from options.sessionId. Default: true for OpenRouter endpoints, false otherwise. |
+| `sessionAffinityFormat` | `"openai" \| "openai-nosession" \| "openrouter"` |  | Session-affinity header format. openai sends session_id, x-client-request-id, and x-session-affinity; openai-nosession sends x-client-request-id and x-session-affinity; openrouter sends x-session-id. Does not affect prompt_cache_key. Default: auto-detected. |
+| `supportsLongCacheRetention` | boolean |  | Whether the provider supports long prompt cache retention (prompt_cache_retention: "24h" or Anthropic-style cache_control.ttl: "1h", depending on format). Default: auto-detected from provider and URL. |
+| `vllmPriority` | number |  | vLLM scheduler priority sent as the top-level priority request field. Lower values are handled earlier and the server default is 0. Only meaningful with --scheduling-policy priority. Off by default and not set on the generated catalog. |
+
+### `openai-responses`
+
+Also applies to `azure-openai-responses` and `openai-codex-responses`.
+
+| Field | Type | Default | Description |
+|---|---|---|---|
+| `supportsDeveloperRole` | boolean | `true` | Whether the provider supports the developer role instead of system. |
+| `supportsMidConvoSystemMessages` | boolean | `false` | Whether the exact model accepts developer or system messages after the conversation has started. When false, later system messages are folded into the leading system message. The generated catalog enables this for verified models. |
+| `sessionAffinityFormat` | `"openai" \| "openai-nosession" \| "openrouter"` |  | Session-affinity header format. openai sends session_id and x-client-request-id; openai-nosession sends x-client-request-id; openrouter sends x-session-id. Does not affect prompt_cache_key. Default: auto-detected. |
+| `supportsLongCacheRetention` | boolean | `true` | Whether the provider supports long prompt cache retention. This uses prompt_cache_options.ttl: "30m" on GPT-5.6+ and prompt_cache_retention: "24h" on earlier models. |
+| `supportsStrictMode` | boolean |  | Whether the provider supports strict JSON-schema function tools. Defaults are API-specific; generated OpenAI models enable it explicitly. |
+| `supportsOpenAIGrammarTools` | boolean | `false` | Whether to emit OpenAI custom tools with Lark or regex grammar formats. When false, grammar-constrained tools fall back to normal function tools. The generated catalog enables this for capable models. |
+| `supportsAdditionalTools` | boolean | `false` | Whether the model supports message-anchored additional_tools input items. |
+| `supportsToolSearch` | boolean | `false` | Whether the model supports client-executed tool search for transcript-anchored additions. |
+| `supportsExplicitPromptCacheMode` | boolean | `false` | Whether the model accepts prompt_cache_options. Older OpenAI models reject the parameter. |
+| `supportsMidConvoEffort` | boolean | `false` | Whether thinking level changes are sent as configuration_update input items instead of changing reasoning.effort, so a change keeps the prompt cache. Only used by the openai-responses API, and ignored with Sign in with ChatGPT on the openai provider. |
+| `supportsMaxOutputTokens` | boolean | `true` | Whether the provider accepts max_output_tokens. Some Codex-protocol gateways reject it. |
+
+### `anthropic-messages`
+
+| Field | Type | Default | Description |
+|---|---|---|---|
+| `supportsEagerToolInputStreaming` | boolean | `true` | Whether the provider accepts per-tool eager_input_streaming. When false, the Anthropic provider omits tools[].eager_input_streaming and sends the legacy fine-grained-tool-streaming-2025-05-14 beta header for tool-enabled requests. |
+| `supportsLongCacheRetention` | boolean | `true` | Whether the provider supports Anthropic long cache retention through cache_control.ttl. |
+| `sendSessionAffinityHeaders` | boolean |  | Whether to send x-session-affinity from options.sessionId when caching is enabled. Required for providers like Fireworks that use session affinity for prompt cache routing; requests to the same replica maximize cache hits. Default: true for OpenRouter endpoints, false otherwise. |
+| `sessionAffinityFormat` | `"openrouter"` |  | Session-affinity format. openrouter sends x-session-id; when unset, sends x-session-affinity. |
+| `supportsCacheControlOnTools` | boolean | `true` | Whether the provider supports Anthropic-style cache_control markers on tool definitions. When false, cache_control is omitted from tool parameters. Some Anthropic-compatible providers, such as Fireworks, do not support this field on tools and may reject or ignore it. |
+| `supportsTemperature` | boolean | `true` | Whether the model accepts the Anthropic temperature request field. Claude Opus 4.7+ rejects non-default values. |
+| `forceAdaptiveThinking` | boolean | `false` | Whether to force adaptive thinking (thinking.type: adaptive plus output_config.effort) regardless of model ID. Built-in models that require adaptive thinking set this in generated metadata. Custom Anthropic-compatible providers can set this to true for any model whose upstream requires the adaptive format. Set false to opt out on overridden built-in models. |
+| `allowEmptySignature` | boolean | `false` | Whether to replay empty thinking signatures instead of converting thinking to text. |
+| `supportsStrictTools` | boolean | `false` | Whether the provider supports Anthropic strict tool schemas. Generated Anthropic models enable it explicitly. |
+| `supportsMidConvoEffort` | boolean | `false` | Whether the exact model transport supports effort-only system messages and thinking binding controls. |
+| `supportsMidConvoSystemMessages` | boolean | `false` | Whether the exact model accepts system-role messages inside the conversation. When false, later system messages are folded into the top-level system prompt. |
+| `supportsMidConvoToolChanges` | boolean | `false` | Whether the exact model accepts mid-conversation tool_addition and tool_removal blocks. Requires supportsMidConvoSystemMessages. |
+| `allowedFallbackModels` | `object[]` |  | Models Anthropic accepts for server-side refusal fallback, with local pricing metadata for returned fallback responses. When absent or empty, callers must omit fallbacks; Anthropic rejects the field for models with no permitted fallback targets. |
+
+### `bedrock-converse-stream`
+
+| Field | Type | Default | Description |
+|---|---|---|---|
+| `supportsStrictMode` | boolean | `false` | Whether the model supports Bedrock strict tool schemas. |
+
+### `mistral-conversations`
+
+| Field | Type | Default | Description |
+|---|---|---|---|
+| `supportsMidConvoSystemMessages` | boolean | `false` | Whether the exact model accepts system messages after the conversation has started. When false, later system messages are folded into the leading system message. |
+
+### `typesafe-system-one`
+
+Also applies to `cloudflare-workers-ai-system-one`.
+
+| Field | Type | Default | Description |
+|---|---|---|---|
+| `imageInput` | `"assets" \| "images" \| "state-image-urls"` |  | How the service takes classifier images. "assets" sends TypeSafe's assets map (PNG and JPEG) and references the images from the state, as TypeSafe and Jev on Workers AI expect. "images" sends Clef's images list of data URLs (PNG, JPEG, and WebP, at most four), as Clef on Workers AI expects. "state-image-urls" places image_url parts before the state, as OpenRouter expects; OpenRouter drops assets without an error. Auto-detected from the provider, baseUrl, and model ID. Classifier models read it from the provider-level compat; models and modelOverrides entries apply to chat models only. |
+| `sendImageOptInHeader` | boolean |  | Whether requests with images send the TypeSafe-Danger-Zone-Images: true header, which TypeSafe's own API requires for its experimental image input. Defaults to true for the typesafe-system-one API unless the model is detected as OpenRouter. |

package/docs/models.md

@@ -0,0 +1,225 @@
+# Choose a Model
+
+For a built-in provider, start with `/login`, then choose a model with `/model`. Use custom model configuration only when Pi does not already include the provider or endpoint you need.
+
+## Choose a connection
+
+| What you have | Recommended setup |
+|---|---|
+| A supported subscription | Sign in through `/login` |
+| A provider API key | Store it through `/login` or set its environment variable |
+| A local GGUF model | Connect Pi to the llama.cpp router |
+| An OpenAI-, Anthropic-, or Google-compatible endpoint | Add it to `models.json` |
+| A provider with a custom protocol or authentication flow | Build or install a provider extension |
+
+Browse the [model catalog](https://pi.dev/models) for current providers, model IDs, capabilities, context limits, and pricing. Pi starts with its bundled catalog and can overlay newer catalog data from pi.dev. Cached catalog data remains available offline; run `pi update --models` to force a refresh.
+
+## Authenticate
+
+Run `/login` and select a provider. Pi stores credentials in [`auth.json`](configuration.md#agent-directory). Run `/logout` to remove stored credentials for a provider.
+
+You can instead provide an API key through the provider's environment variable. This is useful in CI and other environments where Pi should not write credentials. [Providers](providers.md) lists the variables and provider-specific setup.
+
+When several credential sources are configured, Pi uses a runtime `--api-key` first, then a stored `auth.json` credential, an `apiKey` from `models.json`, and finally the provider's environment variables or ambient cloud credentials. Provider extensions can define their own authentication behavior.
+
+Keep `auth.json` and any credential commands private. Project settings and extensions can execute inside the Pi process after you trust a project. Review [Security](security.md) before loading configuration from an untrusted directory.
+
+## Select a model
+
+Run `/model` to search available models. The picker shows models whose providers have usable authentication. Press `Ctrl+S` on a model to save it as the default for new sessions.
+
+Run `/thinking` to select the thinking level for the current model. Press `Ctrl+S` there to save the startup level. Pi limits the choices to levels supported by the selected model.
+
+`Ctrl+P` cycles through available models. Use `/scoped-models` to control that cycle and save the selection, or configure model patterns through [Settings](settings.md#model-cycling).
+
+A session records model and thinking-level changes. Resuming the session restores them without changing defaults for new sessions.
+
+## Connect local models
+
+Pi integrates directly with the llama.cpp router. The router discovers GGUF files and loads models on demand. Pi's `/llama` command manages the router, while `/model` selects one of its loaded models.
+
+Follow [Local Models with llama.cpp](llama-cpp.md) for server startup, model layout, downloads, and connection troubleshooting.
+
+For Ollama, LM Studio, vLLM, SGLang, and other compatible servers, [configure a compatible endpoint](#configure-a-compatible-endpoint) in `models.json`.
+
+## Configure a compatible endpoint
+
+Use [`models.json`](configuration.md#agent-directory) when an endpoint speaks an API Pi already supports. This includes most Ollama, LM Studio, vLLM, SGLang, and proxy deployments.
+
+```json
+{
+  "$schema": "https://pi.dev/schemas/models.schema.json",
+  "providers": {
+    "ollama": {
+      "baseUrl": "http://localhost:11434/v1",
+      "api": "openai-completions",
+      "apiKey": "ollama",
+      "models": [
+        { "id": "qwen2.5-coder:7b" }
+      ]
+    }
+  }
+}
+```
+
+The dummy key makes the model available to Pi; Ollama ignores it. For an authenticated endpoint, `apiKey` and header values can use `$NAME` or `${NAME}` environment interpolation, a literal value, or a leading `!command`. `$$` and `$!` produce a literal `$` and `!`. Commands in `models.json` run at request time and are not cached by Pi.
+
+Opening `/model` reloads the file. A `models` entry adds or replaces a model with the same ID on that provider. Use `modelOverrides` to change metadata for an existing built-in or extension-provided model without replacing the provider's model list. Unknown override IDs are ignored.
+
+To route a built-in provider through a proxy, set only its `baseUrl`. Its models and authentication stay unchanged:
+
+```json
+{
+  "providers": {
+    "anthropic": { "baseUrl": "https://proxy.example.com/v1" }
+  }
+}
+```
+
+[Models Configuration](models-configuration.md) lists every field.
+
+### Describe model input and caching
+
+Use `inputLimits.images.resize` to control how Pi encodes new image attachments, `read` results, and tool-result images before storing them in conversation history:
+
+```json
+{
+  "id": "vision-model",
+  "input": ["text", "image"],
+  "inputLimits": {
+    "images": {
+      "resize": {
+        "maxWidth": 1568,
+        "maxHeight": 1568,
+        "maxBytes": 524288,
+        "jpegQuality": 75
+      }
+    }
+  }
+}
+```
+
+`maxBytes` limits the base64-encoded payload. Omitted resize fields use conservative defaults of 2000 by 2000 pixels, 4.5 MiB encoded, and JPEG quality 80. Images are encoded once; changing models does not rewrite historical images. The catalog can also describe hard request limits with `inputLimits.maxRequestBytes`, `images.maxPerMessage`, and `images.maxPerRequest`, but Pi does not yet rewrite or reject history based on them.
+
+<a id="prompt-cache-lifetimes"></a>
+
+Use `promptCache` to declare the provider's best-effort cache lifetime in seconds for the `short` or `long` retention tier:
+
+```json
+{ "id": "claude-sonnet-5", "promptCache": { "short": 300, "long": 3600 } }
+```
+
+Choose the conservative end of any published range. A model without a lifetime for the active tier is not eligible for cache warming. A `modelOverrides` entry can set `inputLimits` or `promptCache` for a built-in or extension model, including a model accessed through a validated proxy. See [`cacheWarming`](settings.md#model-and-thinking).
+
+### Configure sampling by thinking level
+
+OpenAI-compatible APIs support free-form `samplingParams` model defaults and `samplingParamsByThinkingLevel` overrides. The latter uses Pi thinking-level keys (`off`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`), not provider values from `thinkingLevelMap`:
+
+```json
+{
+  "id": "qwen-thinking-model",
+  "reasoning": true,
+  "samplingParams": {
+    "temperature": 1.0,
+    "top_p": 0.95
+  },
+  "samplingParamsByThinkingLevel": {
+    "off": {
+      "temperature": 0.7,
+      "top_p": 0.8
+    },
+    "high": {
+      "top_k": 20
+    }
+  }
+}
+```
+
+Pi first clamps unsupported thinking levels, then merges model `samplingParams`, the effective level's override, and request-level `samplingParams` in that order. Later values win per key. Missing levels inherit the model defaults. `modelOverrides` merges per-level entries per key with the base model. These fields apply only to `openai-completions`, `openai-responses`, and `azure-openai-responses`; other APIs ignore them.
+
+Compatibility settings should describe verified differences in the endpoint's request or response behavior. Do not enable them based only on an endpoint advertising OpenAI or Anthropic compatibility.
+
+## Use classifier models
+
+Classifier models do not chat. They answer typed questions about JSON state: pick one of several choices, answer yes or no, or give a score, each with probabilities. Pi includes TypeSafe's Jev model from these providers, Cloudflare's Clef, Clef Flash, and Clef Omni models from Workers AI and OpenRouter, and OpenAI's GPT-6 Luna through the [Decisions API](https://developers.openai.com/api/docs/guides/decisions):
+
+| Provider | Model IDs | Authentication |
+|---|---|---|
+| `typesafe` | `jev-latest` | `TYPESAFE_API_KEY` |
+| `openrouter` | `typesafe/jev-1.13`, `typesafe/jev-1.14`, `~typesafe/jev-latest`, `cloudflare/clef`, `cloudflare/clef-flash`, `cloudflare/clef-omni` | `OPENROUTER_API_KEY` or `/login` |
+| `cloudflare-workers-ai` | `typesafe/jev`, `typesafe/jev-1.14.0`, `@cf/cloudflare/clef`, `@cf/cloudflare/clef-flash`, `@cf/cloudflare/clef-omni` | `CLOUDFLARE_API_KEY` and `CLOUDFLARE_ACCOUNT_ID` |
+| `vercel-ai-gateway` | `typesafe-ai/jev` | `AI_GATEWAY_API_KEY` |
+| `opencode` | `jev-1.13`, `jev-1.13-free` | `OPENCODE_API_KEY` |
+| `openai` | `gpt-6-luna` | `OPENAI_API_KEY` |
+
+Chat models on a [llama.cpp router](llama-cpp.md#classification) are also listed as classifier models.
+
+OpenAI's Decisions API needs an API key. Sign in with ChatGPT credentials do not work with it, so `gpt-6-luna` is not listed as available while `openai` is logged in through `/login`, even when `OPENAI_API_KEY` is set; log out of `openai` to use the key. The API rejects inputs above 922K tokens, but requests that run longer than about five seconds, currently above roughly 600K input tokens, fail with a gateway timeout.
+
+Classifier models do not appear in `/model`. The model reaches them through the [`codemode`](cli.md#enable-codemode) tool, which is off unless an MCP server turned it on. Enable it with `"defaultTools": ["+codemode"]` in [settings](settings.md#tools). Scripts then list classifier models with `models.getAvailableOfType("classifier")` and call `models.classify(model, { state, questions })`:
+
+```js
+const jev = await models.getModelOfType("classifier", "typesafe", "jev-latest");
+const result = await models.classify(jev, {
+  state: { message: "The change works, thanks." },
+  questions: {
+    approved: {
+      type: "bool",
+      instructions: "Does the user approve of the result?",
+      criteria: { true: "Approval", false: "No approval" },
+    },
+  },
+});
+return result.answers;
+```
+
+[Codemode](codemode.md#classify) describes the question and answer types.
+
+Classifier models whose `input` includes `"image"` also judge images passed in `images`; other classifier models return an error for them. Image input works with GPT-6 Luna, with Jev from `typesafe` and `cloudflare-workers-ai`, with Clef from `cloudflare-workers-ai`, and with the OpenRouter models whose catalog lists image input. Jev takes PNG and JPEG images, and image input is experimental on TypeSafe's own API. Clef takes PNG, JPEG, and WebP images, at most four per request. Jev from `vercel-ai-gateway` and `opencode` takes text only.
+
+Each System One service takes images in its own request format, set by the model's `compat.imageInput`. Built-in models set it, so overriding a provider's `baseUrl` keeps it. A custom provider that proxies another service's classifiers sets it in its provider-level `compat` in `models.json` (see [System One compatibility](models-configuration.md#typesafe-system-one)). OpenRouter in particular answers requests in TypeSafe's format without an error but drops the images.
+
+When the service reports token counts, as all System One services do, `result.usage` carries them with their cost. Pi adds the usage of a script's classifier calls to the `codemode` tool result, so it counts toward the session cost in the footer and `/session`. The cost uses the model's catalog price; models without one, such as TypeSafe's direct `jev-latest`, report tokens at no cost.
+
+Extensions call classifiers through `ctx.modelRegistry.classify()`, without codemode. [Virtual models](virtual-models.md#route-requests) can use them to route requests; see the `jev-router.ts` example.
+
+## Use image models
+
+Image models generate images from a prompt and optional input images. Pi lists OpenRouter's image models, such as `google/gemini-2.5-flash-image` and `black-forest-labs/flux.2-pro`, under the `openrouter` provider; they use the same `OPENROUTER_API_KEY` or `/login` credential as its chat models.
+
+Like classifier models, image models do not appear in `/model`; the model reaches them through the [`codemode`](cli.md#enable-codemode) tool. Scripts list them with `models.getAvailableOfType("image")` and call `models.generateImages(model, { input })`. The result's `output` holds base64 image blocks, which `image()` attaches to the `codemode` result so the model sees them:
+
+```js
+const painter = await models.getModelOfType("image", "openrouter", "google/gemini-2.5-flash-image");
+const result = await models.generateImages(painter, {
+  input: [{ type: "text", text: "A red fox in the snow, watercolor" }],
+});
+if (result.stopReason !== "stop") return result.errorMessage;
+for (const block of result.output) if (block.type === "image") image(block);
+```
+
+`input` can also contain `{ type: "image", data, mimeType }` blocks to edit or use as references. Pi adds the usage of a script's image calls to the `codemode` tool result, like classifier calls. Generated images are not saved to disk. [Codemode](codemode.md#generate-images) describes the full API.
+
+Extensions generate images through `ctx.modelRegistry.generateImages()`, without codemode.
+
+## Add a custom provider
+
+Use an extension when the provider needs custom streaming, model discovery, or authentication behavior. See [Custom Providers](custom-provider.md) for the extension workflow.
+
+## Troubleshooting
+
+### A model does not appear
+
+Confirm that its provider has usable authentication. Custom models can load from `models.json` but remain unavailable in `/model` until Pi can resolve credentials. For llama.cpp, only models currently loaded by the router appear.
+
+### Authentication works in one shell only
+
+Check whether the key came from an environment variable rather than `auth.json`. Environment variables must be present in the process that starts Pi.
+
+### Sign-in opens a browser on a remote machine
+
+Complete the provider's headless authentication flow when available. Some providers let you paste the final redirect URL or authorization code back into Pi. See [Authenticate interactively](providers.md#authenticate-interactively).
+
+### A compatible endpoint rejects requests
+
+Check its API type and compatibility settings in `models.json`. The upstream server must support the corresponding request fields and behavior.

package/docs/nvidia-rtx-llama-cpp.md

@@ -0,0 +1,96 @@
+# Run Pi with NVIDIA RTX and llama.cpp
+
+[NVIDIA Local AI Inference Recipes](https://nvidia.github.io/local-ai-inference-recipes/index.html) provides llama.cpp settings tuned for specific NVIDIA hardware and models. Each recipe gives a single-model `llama-server` command. Pi works with the llama.cpp router, which can serve and manage several models, so this guide converts the recipe command into a router preset.
+
+The examples below use Windows PowerShell. See [Local Models with llama.cpp](llama-cpp.md) for Pi's complete router and model-management reference.
+
+## Before you begin
+
+1. Open the [recipe catalog](https://nvidia.github.io/local-ai-inference-recipes/index.html), select your hardware, operating system, and model, then choose **llama.cpp**. Check the prerequisites and memory requirements. Keep the recipe's launch command for the conversion below.
+2. Install llama.cpp with the recipe's setup instructions. The recipe's settings depend on its build, for example CUDA support or speculative decoding options.
+3. Run `llama-server --help` and check that it lists `--models-preset`. Without it, the build cannot load the recipe as a router preset. Use a newer build that still meets the recipe's requirements.
+4. Install Pi with the [Quickstart](quickstart.md). On native Windows, also see [Run Pi on Windows](windows.md).
+
+## Convert the recipe to a preset
+
+A preset lists the settings for one or more models. Each section name becomes the model ID shown in Pi. Save the preset next to `llama-server`, or pass its full path when starting the router.
+
+Convert each option from the recipe command into one preset line:
+
+- Remove the leading dashes: `--ctx-size 65536` becomes `ctx-size = 65536`.
+- Write options without a value as `true`: `--jinja` becomes `jinja = true`.
+- Remove quotes and PowerShell line-continuation backticks around values.
+- Leave out `--host` and `--port`. They belong on the router command line.
+- Add `load-on-startup = true` so the router loads the model when it starts.
+- Replace short model-file options with their long preset keys:
+
+| Recipe option | Preset key |
+|---|---|
+| `-hf` | `hf-repo` |
+| `-hff` | `hf-file` |
+| `-hfrd` | `hf-repo-draft` |
+| `--spec-draft-model` | `spec-draft-model` |
+
+For example, this recipe command:
+
+```powershell
+llama-server.exe -hf "unsloth/Qwen3.8-27B-GGUF" `
+  -hff "Qwen3.8-27B-UD-Q4_K_M.gguf" `
+  -hfrd "z-lab/Qwen3.8-27B-DFlash2-GGUF" `
+  --spec-draft-model "Qwen3.8-27B-DFlash2-Q4_K_M.gguf" `
+  --no-mmproj `
+  --host 127.0.0.1 `
+  --port 9931 `
+  --batch-size 4096 `
+  --ubatch-size 1024 `
+  --ctx-size 65536 `
+  --parallel 1 `
+  -ngl 999 `
+  --jinja `
+  --chat-template-kwargs '{"preserve_thinking":true}' `
+  --spec-type draft-dflash `
+  --spec-draft-n-max 7 `
+  --spec-draft-backend-sampling
+```
+
+becomes `qwen3.8-27b.ini`:
+
+```ini
+version = 1
+
+[qwen3.8-27b]
+load-on-startup = true
+hf-repo = unsloth/Qwen3.8-27B-GGUF
+hf-file = Qwen3.8-27B-UD-Q4_K_M.gguf
+hf-repo-draft = z-lab/Qwen3.8-27B-DFlash2-GGUF
+spec-draft-model = Qwen3.8-27B-DFlash2-Q4_K_M.gguf
+no-mmproj = true
+batch-size = 4096
+ubatch-size = 1024
+ctx-size = 65536
+parallel = 1
+ngl = 999
+jinja = true
+chat-template-kwargs = {"preserve_thinking":true}
+spec-type = draft-dflash
+spec-draft-n-max = 7
+spec-draft-backend-sampling = true
+```
+
+## Start the router
+
+Open PowerShell in the llama.cpp folder and run:
+
+```powershell
+.\llama-server.exe --models-preset qwen3.8-27b.ini --host 127.0.0.1 --port 8080
+```
+
+If the recipe requires a Hugging Face token, set `$env:HF_TOKEN` in the same PowerShell window before starting the router. On its first start, the router downloads the model files.
+
+Use the [router health checks](llama-cpp.md#troubleshooting) to confirm that the server is ready. In Windows PowerShell, use `curl.exe` rather than `curl`, which PowerShell maps to a different command. The model is ready when `/models` lists `qwen3.8-27b` with the status `loaded`.
+
+## Use the model in Pi
+
+Follow [Configure Pi](llama-cpp.md#configure-pi) and connect to `http://127.0.0.1:8080`. The preset section name is the model ID, so select `qwen3.8-27b` in `/model` for this example.
+
+NVIDIA recipe memory settings assume that only the selected model uses the GPU. If `/llama` shows other loaded models, unload them before using the recipe model.