claude-autorouter 0.2.0

package/docs/ollama-evaluation.md

@@ -0,0 +1,100 @@
+# Local evaluator measurements
+
+Jev remains the default evaluator. Ollama is an optional local classifier: Claude still generates the answer, and the router's capability and continuation guards still apply. This evaluation uses synthetic prompts only and does not measure the quality of Claude's completed work.
+
+## Candidates and method
+
+Measurements were recorded on September 29, 2026, on an Apple M4 Mac with 16 GiB of unified memory, running Ollama 0.33.3 alongside other applications. Candidates were loaded one at a time with a 4,096-token context. Download size is not the same as resident memory.
+
+| Candidate | Published download size | Purpose |
+| --- | ---: | --- |
+| `qwen3.5:0.8b` | Approximately 1.0 GB | Smallest candidate |
+| `qwen3:1.7b` | Approximately 1.4 GB | Compact speed baseline |
+| `qwen3.5:2b` | Approximately 2.7 GB | Additional compact candidate |
+| `qwen3.5:4b` | Approximately 3.4 GB | Measured larger candidate; rejected for preset latency |
+| `qwen3:4b` | Approximately 2.5 GB | Larger candidate selected for the `quality` option |
+
+Sizes and quantization vary by tag. Use explicit tags: untagged `qwen3.5` currently selects the much larger 9B model. See the official [Qwen3.5 model catalog](https://ollama.com/library/qwen3.5) and [Qwen3 1.7B listing](https://ollama.com/library/qwen3:1.7b).
+
+The `compact` option selects `qwen3:1.7b` for its smaller allocation and faster measured classification. The `quality` option selects `qwen3:4b`, which agreed more often with the held-out labels while using more memory and time. The latter's official listing specifies a roughly 2.5 GB download and Q4_K_M quantization. Additional memory headroom is useful when other applications are open, but more RAM alone does not guarantee better accuracy or meeting the evaluator deadline. See the [official Qwen3 4B listing](https://ollama.com/library/qwen3:4b).
+
+The checked-in fixture contains 36 independently authored, balanced routing cases: 12 tuning cases and 24 held-out cases, with equal numbers of Haiku, Sonnet, and Opus labels. Cases cover mechanical edits, ordinary implementation, difficult correctness and security work, topic changes, tool results, short follow-ups, and misleading routing instructions. Expected labels are judgments under the routing rubric, not independently verified claims about which Claude model would succeed.
+
+The rubric was refined using the tuning set. The final held-out run uses the frozen rubric and is reported separately. Repeating each held-out case three times gives 72 decisions per model, but still only 24 distinct workloads.
+
+The harness calls the production `buildOllamaState` and `evaluateOllama` functions. It includes local model metadata checks in wall-clock latency and bypasses AutoRouter's decision cache. Ollama's normal shared-prefix caching remains enabled. Input excerpts have a 3,000-character/UTF-8-byte ceiling. Requests disable thinking, use temperature 0, seed 0, a 32-token output cap, and a JSON schema containing only the tier. A returned tier is not a calibrated confidence probability. See Ollama's [thinking controls](https://docs.ollama.com/capabilities/thinking) and [structured-output guidance](https://docs.ollama.com/capabilities/structured-outputs).
+
+A cold measurement starts with the model unloaded from Ollama; operating-system file caches and compiled kernels may already be warm. Cold calls have a separate 60-second measurement deadline. Warm measurements use the production 1,500 ms deadline. Model allocation is read from `/api/ps`, not inferred from the download size. On unified-memory hardware, its GPU allocation is not additional independent RAM. See [Ollama's running-model API](https://docs.ollama.com/api/ps) and [context-memory guidance](https://docs.ollama.com/context-length).
+
+## Results
+
+The frozen-rubric tuning results below document model selection. They are not held-out performance. GB values use decimal bytes; allocation is what the local running-model API reported, rather than total process or system memory.
+
+| Model | Tuning agreement | Timeouts | Warm p50 / p95 | Cold wall time | Disk size | Model allocation |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: |
+| `qwen3:1.7b` | 20 / 24 (83.3%) | 0 / 24 | 245 / 280 ms | 2.50 s | 1.36 GB | 1.70 GB |
+| `qwen3:4b` | 24 / 24 (100%) | 0 / 24 | 743 / 889 ms | 9.62 s | 2.50 GB | 3.18 GB |
+| `qwen3.5:0.8b` | 10 / 24 (41.7%) | 0 / 24 | 409 / 437 ms | 2.78 s | 1.04 GB | 1.09 GB |
+| `qwen3.5:2b` | 18 / 24 (75.0%) | 0 / 24 | 918 / 1,009 ms | 6.49 s | 2.74 GB | 2.36 GB |
+| `qwen3.5:4b` | No valid warm results | 24 / 24 | Deadline reached | 7.26 s | 3.39 GB | 3.14 GB |
+
+The smallest model was highly sensitive to rubric wording. It should not be selected solely because its download is small. The 2B candidate was slower and agreed less often than 1.7B on this tuning set. A separate diagnostic gave `qwen3.5:4b` a 5,000 ms deadline: all 12 tuning decisions agreed, but warm p50/p95 was 3,615/4,872 ms. **Qwen3.5 4B is rejected as a preset because it missed the production deadline on every warm tuning request.** Its diagnostic result demonstrates a latency tradeoff, not held-out quality. Extra RAM alone does not establish that this model will meet a 1,500 ms deadline on another machine.
+
+An initial 4B run also exposed a metadata-response limit: its local `/api/show` response exceeded 64 KiB. That issue was fixed before the results above by bounding metadata separately at 1 MiB while retaining a 64 KiB classifier-output limit.
+
+### Held-out results
+
+Both selected candidates completed all 72 held-out decisions without a timeout. Qwen3 4B agreed more often than 1.7B, including every Opus-labeled workload. These measurements were taken later than tuning while other applications remained open; they do not represent an isolated hardware benchmark.
+
+| Model | Agreement | Timeouts | Warm p50 / p95 | Cold wall time | Under-routes / over-routes |
+| --- | ---: | ---: | ---: | ---: | ---: |
+| `qwen3:1.7b` | 42 / 72 (58.3%) | 0 / 72 | 602 / 834 ms | 4.80 s | 18 / 12 |
+| `qwen3:4b` | 66 / 72 (91.7%) | 0 / 72 | 889 / 1,242 ms | 5.92 s | 0 / 6 |
+
+The 1.7B confusion matrix:
+
+| Expected tier | Returned Haiku | Returned Sonnet | Returned Opus |
+| --- | ---: | ---: | ---: |
+| Haiku | 12 | 6 | 6 |
+| Sonnet | 0 | 24 | 0 |
+| Opus | 0 | 18 | 6 |
+
+The 4B confusion matrix:
+
+| Expected tier | Returned Haiku | Returned Sonnet | Returned Opus |
+| --- | ---: | ---: | ---: |
+| Haiku | 24 | 0 | 0 |
+| Sonnet | 0 | 18 | 6 |
+| Opus | 0 | 0 | 24 |
+
+Each row represents eight unique workloads repeated three times. The wrong labels were consistent across repetitions. For 1.7B, six of eight Opus workloads were sent to Sonnet, and four of eight Haiku workloads were sent to a higher tier. For 4B, two of eight Sonnet workloads were sent to Opus. The compact model's gap between tuning and held-out agreement limits what can be claimed about it. Both local options remain experimental; these results do not establish parity with Jev, which was not evaluated on this fixture.
+
+### Full-excerpt performance
+
+An additional eight synthetic requests per model filled the entire 3,000-byte state budget. They kept a clearly mechanical current task and included long synthetic tool output. A different nonce at the beginning of each serialized state prevented reuse of the previous full user-state prefix while preserving the common rubric prefix. Both models exceeded the 1,500 ms deadline on all eight requests. These are separate performance checks, not additional held-out accuracy cases. Short-prompt latency must not be treated as a bound for full excerpts.
+
+| Model | Timeouts at 1,500 ms | Wall p50 / p95 before cancellation |
+| --- | ---: | ---: |
+| `qwen3:1.7b` | 8 / 8 | 1,502 / 1,503 ms |
+| `qwen3:4b` | 8 / 8 | 1,502 / 1,505 ms |
+
+A diagnostic rerun of 1.7B with a 5,000 ms deadline completed all eight and returned Haiku, with p50/p95 of 3,753/3,991 ms. Qwen3 4B still timed out on all eight at that longer deadline, with p50/p95 cancellation times of 5,002/5,007 ms. Its actual completion times for these full excerpts were not measured. A larger model and startup priming do not remove the need for a bounded timeout and fallback during longer requests.
+
+All final comparisons use fixture SHA-256 `1ef5111a6a36f0f4bc8d111d54c6985cac4c4a8b16357023a0958d285f7438ad` and rubric SHA-256 `42c3e18ddcf7c9d8756100b740f686b3c2f1bbdfbcbaf3cbc3021a9c9b4c6ee6`.
+
+## Reproducing the evaluation
+
+Use a source checkout; benchmark scripts and fixtures are development files and are not bundled in the npm package. Start local Ollama and explicitly download the models you intend to test. The harness never downloads or deletes models, and refuses to begin while another model is resident. It unloads each tested model after its measurements.
+
+```sh
+node scripts/evaluate-ollama.mjs --models qwen3:1.7b,qwen3:4b --split tuning --rounds 2 --output artifacts/ollama-tuning.json
+node scripts/evaluate-ollama.mjs --models qwen3:1.7b,qwen3:4b --split heldout --rounds 3 --stress-rounds 8 --output artifacts/ollama-heldout.json
+```
+
+JSON reports contain fixture/rubric hashes, model digests, quantization, per-case labels, timing counters, failures, confusion matrices, and allocation measurements. They contain no private repository prompts or API credentials. Raw reports are written only when `--output` is supplied; `artifacts/` is ignored by Git.
+
+## Limits
+
+This is a small synthetic rubric-agreement benchmark, not a downstream task-quality, cost-savings, or security evaluation. Its prompts cannot represent every repository or long conversation. A model can agree with the labels and still miss important context outside the excerpt. The tests do not establish robust resistance to prompt injection.
+
+Latency depends on hardware, current application load, model residency, and prompt length. Cold loading exceeds the normal evaluator deadline, which is why the launcher primes an installed local model with a synthetic classification using the production request settings before starting Claude. The warm benchmark measurements above already followed a full classification, so this startup improvement does not change those measurements. Timeouts and invalid responses use the router's existing conservative fallback policy. Re-run the evaluation before adopting different tags or changing the rubric; no result here guarantees accuracy on your workload.

package/docs/reference.md

@@ -0,0 +1,227 @@
+# Reference
+
+## Commands
+
+| Command | Purpose |
+| --- | --- |
+| `claude-autorouter setup` | Save subscription-mode configuration and a Jev key |
+| `claude-autorouter setup --auth-mode api-key` | Configure Jev and Anthropic API-key billing |
+| `claude-autorouter setup --evaluator ollama --ollama-preset compact --pull` | Configure a local evaluator and download its selected model if missing |
+| `claude-autorouter setup --force` | Replace an existing user config |
+| `claude-autorouter doctor` | Check config, Claude executable/login, and the selected local Ollama model without paid calls |
+| `claude-autorouter claude [arguments]` | Start a local router and pass arguments through to Claude Code |
+| `claude-autorouter serve` | Run the router for separately configured clients |
+| `claude-autorouter --help` | Show command help |
+| `claude-autorouter --version` | Print the package version |
+
+The launcher binds an ephemeral port on `127.0.0.1`, creates a temporary local credential, starts Claude with the gateway address, and shuts down when Claude exits. It works from any project directory on macOS or Linux, including WSL. Native Windows is unsupported in this release; the status-line command uses a POSIX shell. Standalone `serve` uses the configured port and requires a local token.
+
+## Configuration
+
+Setup defaults to subscription mode unless `--auth-mode` or `AUTOROUTER_AUTH_MODE` selects another mode. Jev remains the default evaluator; `--evaluator ollama` selects local classification. Setup prompts for required secrets without echoing them and writes a private JSON file. Stored keys are plaintext; keep the file private and out of source control. Supply keys through the environment when interactive input is unavailable. Subscription mode with Ollama requires no API keys. API-key authentication always requires `ANTHROPIC_API_KEY`, regardless of evaluator.
+
+The config path is selected in this order:
+
+1. `AUTOROUTER_CONFIG`, when set.
+2. `$XDG_CONFIG_HOME/claude-autorouter/config.json`, when `XDG_CONFIG_HOME` is a nonempty absolute path.
+3. `~/.config/claude-autorouter/config.json`.
+
+The JSON file uses flat environment-style string keys, such as `AUTOROUTER_AUTH_MODE` and `TYPESAFE_API_KEY`. Environment values take precedence over the saved config. Use `setup --force` to replace existing configuration. The launcher does not discover or load a project's `.env` file. From a source checkout, explicitly loading one still works:
+
+```sh
+node --env-file=.env bin/autorouter.mjs claude
+```
+
+For an environment-only subscription launch, set `AUTOROUTER_AUTH_MODE=subscription` and either supply `TYPESAFE_API_KEY` or select `AUTOROUTER_EVALUATOR=ollama` with a running local model. For API-key mode, also supply `ANTHROPIC_API_KEY`. The shell variables are read by the router; Jev's key is removed from the Claude child environment.
+
+| Variable | Default | Purpose |
+| --- | --- | --- |
+| `TYPESAFE_API_KEY` | required for Jev | Jev credential; unused by Ollama |
+| `ANTHROPIC_API_KEY` | required in API-key mode | Upstream Anthropic credential |
+| `AUTOROUTER_CONFIG` | see path order above | Explicit user config path |
+| `AUTOROUTER_AUTH_MODE` | `api-key` without saved config; setup selects `subscription` | Authentication mode |
+| `AUTOROUTER_EVALUATOR` | `jev` | `jev` or local `ollama` classification |
+| `AUTOROUTER_CLIENT_PROFILE` | `compatible` | `native` retains Claude's own model and thinking settings |
+| `AUTOROUTER_STATUSLINE` | enabled | `0` retains your existing status line |
+| `AUTOROUTER_DEBUG` | off | `1` enables launcher metadata logs on stderr |
+| `ENABLE_TOOL_SEARCH` | `true` in launcher when unset | Load MCP tool definitions on demand; explicit values are preserved |
+| `AUTOROUTER_HAIKU_MODEL` | `claude-haiku-4-5-20251001` | Routine tier |
+| `AUTOROUTER_SONNET_MODEL` | `claude-sonnet-5` | Standard tier |
+| `AUTOROUTER_OPUS_MODEL` | `claude-opus-5-5` | Demanding tier and savings baseline |
+| `AUTOROUTER_JEV_MODEL` | `jev-latest` | Classifier version |
+| `AUTOROUTER_JEV_TIMEOUT_MS` | `1500` | Classifier deadline in milliseconds |
+| `AUTOROUTER_OLLAMA_URL` | `http://127.0.0.1:11434` | Loopback Ollama base URL |
+| `AUTOROUTER_OLLAMA_MODEL` | `qwen3:1.7b` | Installed local model tag; setup can choose a preset |
+| `AUTOROUTER_OLLAMA_TIMEOUT_MS` | `1500` | Whole local classification deadline in milliseconds |
+| `AUTOROUTER_OLLAMA_KEEP_ALIVE` | `5m` | How long Ollama retains the evaluator in memory |
+| `AUTOROUTER_TOKEN_COUNT_TIMEOUT_MS` | `1500` | Context-check deadline; runs alongside classification |
+| `AUTOROUTER_MIN_CONFIDENCE` | `0.75` | Jev confidence threshold; does not apply to Ollama |
+| `AUTOROUTER_PORT` | `8787` | Standalone server port |
+| `AUTOROUTER_TOKEN` | none | Standalone local credential, at least 16 characters |
+| `AUTOROUTER_UPSTREAM_URL` | `https://api.anthropic.com` | Anthropic-compatible origin; fixed in subscription mode |
+| `AUTOROUTER_JEV_URL` | `https://api.typesafe.ai/v1/systemone` | Jev endpoint |
+
+Model access depends on your account. The policy recognizes specific Claude model versions; arbitrary gateway aliases do not automatically inherit their capabilities or context windows. Compare overrides with the [Anthropic model catalog](https://platform.claude.com/docs/en/models/overview).
+
+## Ollama evaluator
+
+Ollama is an experimental local classifier. It chooses a Claude tier; Haiku, Sonnet, or Opus still completes the task through Anthropic. Jev remains the default, and selecting Ollama never silently switches back to Jev.
+
+[Install Ollama](https://docs.ollama.com/quickstart) and start its local service first. Open the Ollama app on macOS, or use `ollama serve` if a server is not already running. Then:
+
+```sh
+claude-autorouter setup --evaluator ollama --ollama-preset compact --pull
+claude-autorouter doctor
+claude-autorouter claude
+```
+
+Use `--force` to replace existing configuration. Setup detects the running local API. `--pull` authorizes downloading the chosen model when it is missing; without it, install the model yourself before setup. AutoRouter does not install Ollama, start its daemon, or download models during ordinary launches or `doctor` checks.
+
+| Preset | Model | Selection |
+| --- | --- | --- |
+| `compact` | `qwen3:1.7b` | Default prioritizes lower memory use; lower held-out rubric agreement |
+| `quality` | `qwen3:4b` | Better measured rubric agreement, with higher memory use and latency |
+| `auto` | One of the above | `quality` only when reported total system memory exceeds 24 GiB; otherwise `compact` |
+
+The memory heuristic estimates headroom using total system RAM, not currently free memory or a CPU/GPU benchmark. Both presets were tested on a 16 GiB Mac; `quality` can be selected explicitly on that capacity when memory permits. The automatic threshold is a conservative headroom choice, not a speed or accuracy guarantee. The `quality` model has an approximately 2.5 GB download; download size differs from resident memory, which includes runtime and context allocations. Concurrent applications also need memory. The classifier uses a 4,096-token context to bound that allocation. See [Ollama's context-memory guidance](https://docs.ollama.com/context-length). Downloaded models have their own licenses and are not included in this package.
+
+The 16 GiB M4 comparison used 24 distinct held-out synthetic workloads, eight per tier, repeated three times for each model:
+
+| Model | Rubric agreement | Warm p50 / p95 | Cold call | Model allocation |
+| --- | ---: | ---: | ---: | ---: |
+| `qwen3:1.7b` | 42 / 72 (58.3%) | 602 / 834 ms | 4.80 s | 1.70 GB |
+| `qwen3:4b` | 66 / 72 (91.7%) | 889 / 1,242 ms | 5.92 s | 3.18 GB |
+
+Both completed every short held-out request without a timeout. Compact routed six of eight distinct Opus-labeled workloads to Sonnet. Quality had no under-routing in this fixture; its six errors were two distinct Sonnet workloads routed to Opus on each repetition. Each model exceeded the 1,500 ms deadline on all eight requests in a separate full-excerpt stress test. The [local evaluation report](ollama-evaluation.md) records the test conditions and rejected candidates. There was no Jev comparison, so these results do not establish parity with Jev. Rubric agreement on synthetic cases does not establish the quality or cost of completed Claude tasks.
+
+Use `--ollama-model LOCAL_TAG` to override the preset, for example with an already installed local model. The endpoint must be loopback (`127.0.0.1`, `localhost`, or `::1`), without a path, credentials, query, or fragment. Cloud model tags and metadata identifying a remote model are rejected before sending task text. Claude and Jev credentials are never attached to Ollama requests.
+
+Local classification caps serialized evaluator state at both 3,000 characters and 3,000 UTF-8 bytes, including for non-ASCII prompts. It sends that state to `/api/chat`, requests a strict JSON tier, disables thinking, and caps output at 32 tokens. It does not invent a confidence probability; `AUTOROUTER_MIN_CONFIDENCE` applies only to Jev. All capability, tool-continuation, thinking, and context guards still apply.
+
+Before opening Claude's UI, the launcher loads an installed model and primes the actual classifier rubric with a synthetic task, using a separate deadline of up to 60 seconds. Each normal evaluation has a 1,500 ms deadline covering local model metadata checks and classification. Priming reduces first-request overhead but does not guarantee that longer excerpts finish in time. `AUTOROUTER_OLLAMA_KEEP_ALIVE` defaults to `5m`. After five idle minutes, the next request may need to reload the model, exceed that deadline, and use the fallback. A longer positive keep-alive can reduce reloads while retaining memory longer; `0` unloads immediately and can make every evaluation cold. Supported values are `0` or a positive duration such as `30s`, `5m`, or `1h`, following the [Ollama API's keep-alive setting](https://docs.ollama.com/api/chat).
+
+If startup priming fails, the launcher warns and continues. A missing model, unavailable service, malformed answer, or evaluation timeout falls back to Sonnet or retains an incoming Opus, subject to the usual compatibility policy. No Jev request is made. The status line identifies `Ollama fallback` and its error category. Run `doctor` to inspect local API availability and installed model metadata; it does not download or generate.
+
+## Data flow and authentication
+
+```text
+Claude Code → authenticated local gateway → Jev or local Ollama classification
+                                        → routing policy and optional token check
+                                        → selected Claude model → streamed response
+```
+
+AutoRouter uses Claude Code's [gateway integration](https://code.claude.com/docs/en/llm-gateway-protocol), so it sees inference requests and tool continuations. It does not rely on a user-prompt hook.
+
+The selected evaluator receives a bounded state containing the latest human request and excerpts of the original task, system text, and recent messages: up to 12,000 serialized characters sent to TypeSafe for Jev, or 3,000 UTF-8 bytes sent to the local Ollama service. These excerpts can include private source code and tool results. Images, document payloads, and signed thinking are omitted. Full tool schemas and full conversation history are not sent to either classifier. Anthropic receives the complete request, including its tools and attachments. Large or multimodal requests may also go to Anthropic's token-count endpoint before inference, including when classification is local.
+
+In subscription mode, Claude Code owns login and OAuth refresh. AutoRouter forwards the current request's authorization and beta headers to Anthropic. It does not read keychain or saved login files, persist subscription tokens, or send them to Jev. A separate temporary `X-Autorouter-Token` authenticates the local connection and is stripped upstream. Subscription forwarding is restricted to `https://api.anthropic.com`. See [subscriptions and gateways](https://code.claude.com/docs/en/llm-gateway#subscriptions-and-gateways).
+
+In API-key mode, the upstream key stays in the proxy and Claude receives a temporary local credential. Requests are billed to the supplied API key. Subscription requests remain subject to the subscription's model access and usage limits. AutoRouter never falls back from subscription authentication to API billing.
+
+Routine logs contain route, model, timing, usage, and error-category metadata, not prompts, raw responses, or credentials. Status snapshots contain routing metadata and token counts in a private temporary directory and are deleted on normal launcher exit. Classification, turn, and token-count caches are held in memory. Claude Code and the external providers have their own storage and logging behavior.
+
+## Routing policy
+
+Each `/v1/messages` request is evaluated. Exact repeated bodies reuse a classification for five minutes. Both evaluators use a starting rubric choosing Haiku for routine work, Sonnet for ordinary engineering, and Opus for demanding reasoning. These choices require evaluation on your tasks; they are not quality guarantees.
+
+The evaluator prioritizes the actual human request before startup metadata. Complete Claude reminder and tool-list blocks are excluded from that task excerpt, and long text retains its beginning and end. The outbound Anthropic request remains complete. Complexity outside the bounded excerpt can still be missed.
+
+The following policy applies after classification:
+
+- Jev's 1,500 ms deadline covers the response body and has no retry. Successful calls return immediately. Timeouts, HTTP errors, and invalid responses fall back to Sonnet or retain an existing stronger model.
+- Jev confidence below 0.75 prevents a downgrade below Sonnet or the requested tier. Ollama returns a tier without calibrated confidence; its failure handling and compatibility guards still apply.
+- Tool continuations retain the model chosen at the start of the human turn. Session, agent, and prompt headers identify turns; normalized conversation content provides a fallback. Moving prompt-cache markers does not create a new turn.
+- Thinking history, fixed-budget thinking, server tools, context management, and other recognized model-specific features preserve the current model. Adaptive thinking, effort, and output above 64K prevent a Haiku choice. Fields are never stripped to force a downgrade.
+- Mid-conversation `system` messages preserve the requested model and pass through unchanged. They do not count as a tool continuation by themselves.
+- Recognized compaction and auxiliary requests otherwise preserve their requested model. Token counting and model discovery pass through without classification.
+
+The default `compatible` profile starts Claude with Haiku-compatible requests and client-requested thinking disabled. When an upgrade to Opus 5/5.5 requires adaptive thinking, AutoRouter enables it. `AUTOROUTER_CLIENT_PROFILE=native` preserves normal client settings, which can constrain routing. An explicit Claude `--model` argument overrides the starting model, but `/model` and `--model` are requested models, not locks on the routed result.
+
+The launcher enables `ENABLE_TOOL_SEARCH=true` when unset. Claude can otherwise disable on-demand MCP discovery when using a custom API address, loading connected-tool schemas into even a fresh conversation. Explicit values, including `false` or `auto:5`, are preserved. Managed settings and always-loaded tools can still affect deferral. See [Claude Code tool search](https://code.claude.com/docs/en/mcp#configure-tool-search).
+
+### Context capacity
+
+Context-relevant JSON above 150KB, or image/document blocks including those inside tool results, trigger a token check for otherwise compatible small models. This byte threshold is a trigger, not a token estimate. The check includes system instructions and active tool schemas. Unused deferred schemas are excluded from the trigger; discovered references and historical tool calls add them back. Unknown shapes are counted conservatively.
+
+The check uses [Anthropic's token-count endpoint](https://platform.claude.com/docs/en/build-with-claude/token-counting), the current request's authentication, and the target model's tokenizer. It runs alongside the selected evaluator with a separate 1,500 ms deadline. Counts are cached for five minutes with at most 100 hash-and-number entries. Input at or below 190K can remain on a 200K model, allowing a margin for estimation differences. Unsupported input, timeouts, and failures fall back to the conservative capacity guard.
+
+If the small model cannot fit, the default mapping upgrades it to Sonnet 5's native 1M window. A known capable Opus can be used when a configured Sonnet lacks that capacity. Compatible Haiku tool turns can upgrade as they grow; subsequent calls retain the upgrade. Thinking history and feature constraints remain pinned. Unknown capacities are not guessed. See [Anthropic context windows](https://platform.claude.com/docs/en/build-with-claude/context-windows).
+
+Requests exceeding the selected model's capacity still receive the provider's error. AutoRouter does not truncate context or retry a generation on another model after an error or partial stream.
+
+## Status line and savings
+
+The `claude` launcher automatically adds a temporary [status-line command](https://code.claude.com/docs/en/statusline). Its fields show:
+
+| Field | Meaning |
+| --- | --- |
+| `Sonnet 5 selected` | Routing chose this model; Anthropic has not confirmed it yet |
+| `Opus 5.5` / `last Opus 5.5` | Provider-confirmed streaming or most recent model |
+| `Jev` / `Ollama`, with `cache` or `fallback` when applicable | Classification source; timing includes concurrent context checks |
+| `Jev→Haiku` or `Ollama→Haiku` beside Sonnet | A policy guard overrode the evaluator's Haiku choice |
+| `large context` | Token count exceeded the small-model input budget |
+| `size unverified` | Token checking failed or was unavailable; conservative guard applied |
+| `API ctx` | Latest main request's input divided by the actual model's known window |
+| `CLI ctx` | Claude's client accounting, shown when its window differs or API capacity is unknown |
+| `est saved … vs Opus` | Cumulative API-equivalent token-cost estimate |
+
+Background agents and auxiliary requests cannot replace the foreground model. Errors, fallback, cancellation, and stale/offline state remain visible. The command reads a local snapshot and makes no network requests. It respects terminal width and `NO_COLOR`; lower-priority fields disappear on narrow terminals.
+
+Context includes uncached input, cache reads, and cache writes, excluding output to match [Claude's percentage formula](https://code.claude.com/docs/en/statusline#context-window-fields). It is current context rather than cumulative usage. Historical usage is marked `last`; compaction resets stale readings. The compatible client's 200K window can reach 100% while a routed Sonnet request uses only part of its 1M window. Displaying both does not change Claude's compaction threshold.
+
+Savings compare the actual models' API token prices with the configured Opus model's prices for the **same reported counts and cache profile**. The percentage is `(Opus cost − routed cost) / Opus cost`. Input, output, cache reads, and 5-minute/1-hour cache writes are priced separately using the bundled table based on [Anthropic's published USD pricing](https://platform.claude.com/docs/en/about-claude/pricing).
+
+Totals include completed main, agent, and auxiliary calls for the current session observed by this router process. Streaming usage is counted once, and totals reset when the router or session restarts. Unrecognized prices or unsupported usage produce `partial` or `savings unavailable`. Higher routed costs show `est extra`. The arithmetic runs locally.
+
+This estimate does not measure subscription bill savings or quota credits. It excludes Jev charges, local compute costs, tool fees, negotiated discounts, and unpriced requests. A real Opus run can produce different tokens and cache hits. Incomplete streams, unknown cache-write TTLs, unsupported pricing modifiers, and unrecognized model versions are excluded rather than guessed. The rate table requires updates when prices change.
+
+Set `AUTOROUTER_STATUSLINE=0` to retain an existing status line. Other `--settings` values are retained in the temporary overlay; source-relative Read/Edit rules keep their anchors. Ambiguous relative sandbox paths cause the launcher to skip the overlay and pass original settings through with a notice. Safe mode disables custom status lines; print mode has no status-line UI. Standalone `serve` does not install one.
+
+## Troubleshooting
+
+Run `claude-autorouter doctor` first. It performs local checks without Claude generations or Jev calls, including local HTTP checks for the selected Ollama model. It cannot establish Anthropic/TypeSafe availability, current quota, or whether a key will be accepted remotely.
+
+For metadata logs without terminal noise:
+
+```sh
+AUTOROUTER_DEBUG=1 claude-autorouter claude 2>autorouter-debug.log
+```
+
+The launcher is quiet by default. Standalone `serve` logs to stderr by default. Claude still displays API errors normally.
+
+**Authentication conflicts:** subscription launches remove inherited `ANTHROPIC_API_KEY`, `ANTHROPIC_AUTH_TOKEN`, and `CLAUDE_CODE_OAUTH_TOKEN` from the child environment, plus authentication-related custom headers. Claude settings can still supply conflicting `apiKeyHelper`, `env`, base-URL, or custom-header overrides. Remove those conflicts from the relevant settings, run `claude auth login` if needed, and relaunch. Parent-shell and saved Claude settings remain unchanged. `--bare` is incompatible with subscription OAuth.
+
+**A simple prompt selects Sonnet:** inspect the reason and evaluator choice in the status line. Background context can be large even in a new session. Model-specific settings, tool continuity, Jev confidence, classifier failures, and context limits can override the classified tier. A fresh conversation picks up tool-loading changes; resumed history can retain already-loaded schemas.
+
+**Claude says Haiku while AutoRouter says Opus:** the built-in model label is Claude's starting/requested model. The AutoRouter confirmed-model label comes from Anthropic. Claude's own token-cost estimate can likewise be attributed to the requested model.
+
+**After restarting mid-conversation:** turn state is in memory and expires after 30 minutes. Unknown continuations preserve the incoming model. Start a fresh conversation when restarting around signed thinking; AutoRouter cannot reconstruct the prior actual model from lost turn state.
+
+Switching models can lose prompt-cache reuse. A cheaper price per token does not guarantee a cheaper or faster task. Only requests using Claude's configured base URL are visible to this proxy. Alternate provider modes such as Bedrock, Vertex, Foundry, Mantle, and `ANTHROPIC_AWS` are unsupported; unset their enable flags before launching. Use ordinary `claude` to bypass routing.
+
+## Standalone server
+
+The launcher handles local credentials and environment settings automatically. For separate clients, configure `AUTOROUTER_TOKEN` with a random value of at least 16 characters and start:
+
+```sh
+claude-autorouter serve
+```
+
+For subscription mode, configure the client's environment in another terminal:
+
+```sh
+unset ANTHROPIC_API_KEY ANTHROPIC_AUTH_TOKEN CLAUDE_CODE_OAUTH_TOKEN
+export ANTHROPIC_BASE_URL=http://127.0.0.1:8787
+export ANTHROPIC_CUSTOM_HEADERS='X-Autorouter-Token: YOUR_LOCAL_ROUTER_TOKEN'
+export CLAUDE_CODE_GATEWAY_HINT_HEADERS=1
+export ANTHROPIC_MODEL=claude-haiku-4-5-20251001
+export MAX_THINKING_TOKENS=0
+export ENABLE_TOOL_SEARCH=true
+claude
+```
+
+Replace the placeholder with the server's local token, never a subscription credential. Include other custom headers on separate lines if needed. Omit model/thinking variables to retain native client settings.
+
+For API-key mode, set the base URL and gateway-hint variable above, and set both client `ANTHROPIC_API_KEY` and `ANTHROPIC_AUTH_TOKEN` to the local router token. Keep the real upstream API key in the server environment/config.
+
+The server binds only to `127.0.0.1` and requires local authentication. `/health` checks the local process, not upstream provider availability.

package/docs/releasing.md

@@ -0,0 +1,152 @@
+# CI and npm releases
+
+The package is `claude-autorouter`, licensed under [Apache-2.0](../LICENSE). **npm publication is pending.** The first release needs an interactive npm login; subsequent releases use GitHub Actions with npm trusted publishing. Preparing a tarball or merging a pull request does not publish it.
+
+The GitHub repository is private. Publishing to npm makes the tarball's runtime source, README, configuration example, license, and shipped documentation public. Model weights, user configuration, credentials, transcripts, local artifacts, and test fixtures are excluded. Review the archive before the first publication and whenever the package allowlist changes.
+
+## What runs automatically
+
+| Workflow | Trigger | Behavior |
+| --- | --- | --- |
+| [ci.yml](https://github.com/frapposelli/claude-autorouter/blob/main/.github/workflows/ci.yml) | Pull requests, pushes to `main`, manual runs, and calls from the release workflow | Syntax checks, tests, and package smoke tests on Ubuntu/macOS with Node 22/24 |
+| [publish.yml](https://github.com/frapposelli/claude-autorouter/blob/main/.github/workflows/publish.yml) | Push of a tag matching `v*` | Validate release, run CI, pack and test the candidate, then publish the verified archive |
+
+A release tag must exactly equal `v` plus the version in `package.json`, and its commit must be reachable from `origin/main`. Package name and repository metadata must match `claude-autorouter` and `frapposelli/claude-autorouter`. Stable versions use npm's `latest` tag; prereleases such as `0.3.0-beta.1` use `next`.
+
+The release workflow packs its candidate once and smoke-tests that exact `.tgz`. It uploads the archive and SHA-256 checksum as an Actions artifact. A separate publishing job downloads that artifact by its immutable ID, checks the checksum and every packaged file against the release checkout, then runs `npm publish` with scripts disabled. The publish job uses a GitHub-hosted Ubuntu runner, Node 24, and npm 11.19.1. Only that job has `id-token: write`; there is no `NPM_TOKEN` secret or required GitHub environment. Failed checks prevent publication.
+
+The project has no package dependencies or lockfile, so CI runs its scripts directly without `npm ci`. Live Claude/Jev calls, Ollama downloads, and private repository probes are not CI checks.
+
+## 1. Publish the first version interactively
+
+Merge the release workflows and package metadata to `main`, push to GitHub, and ensure GitHub Actions is enabled for the repository. Its Actions policy must permit the pinned official GitHub actions and the reusable CI workflow in this repository. Use a clean checkout of that commit, with Node 24 and npm 11.19.1 to match the publisher. No release tag is needed for this bootstrap. First check the registry and account:
+
+```sh
+npm ping --registry https://registry.npmjs.org/
+npm view claude-autorouter name version --registry https://registry.npmjs.org/
+npm whoami --registry https://registry.npmjs.org/
+```
+
+The initial registry lookup returned HTTP 404; recheck immediately before release. If the name now exists, verify that your npm account owns it before continuing. A network or authentication failure is not evidence that a name is available. If another owner has claimed the name, choose an available name and update package metadata, release validation, documentation, and trust settings together.
+
+If `whoami` reports `ENEEDAUTH`, sign in interactively and complete npm's browser/2FA prompts:
+
+```sh
+npm login --registry https://registry.npmjs.org/
+npm whoami --registry https://registry.npmjs.org/
+```
+
+Use your own npm account with publishing permission, and keep credentials out of the repository and workflow secrets. See [npm login](https://docs.npmjs.com/cli/v11/commands/npm-login/) and [publishing a public package](https://docs.npmjs.com/creating-and-publishing-unscoped-public-packages/) for account requirements.
+
+For the initial `0.2.0` release, run:
+
+```sh
+npm run check
+npm test
+npm run release:pack
+npm run test:package -- --archive ./dist/claude-autorouter-0.2.0.tgz
+tar -tzf ./dist/claude-autorouter-0.2.0.tgz
+```
+
+`release:pack` checks the allowed files and writes `dist/claude-autorouter-0.2.0.tgz` and its `.sha256` file. The `--archive` smoke test installs and exercises those bytes without repacking. Review the listed contents and retain both files. If anything changes, rebuild and repeat the exact-archive smoke test.
+
+Publish that reviewed candidate, completing npm's interactive authentication challenge when requested:
+
+```sh
+npm publish ./dist/claude-autorouter-0.2.0.tgz --ignore-scripts --access public --tag latest --registry https://registry.npmjs.org/ --provenance=false
+npm view claude-autorouter@0.2.0 version dist.integrity --registry https://registry.npmjs.org/
+```
+
+On a separate machine or disposable environment, verify the registry installation:
+
+```sh
+npm install -g claude-autorouter@0.2.0
+claude-autorouter --version
+claude-autorouter --help
+```
+
+Then run `setup`, `doctor`, and a launch from outside the source checkout as appropriate for that machine. `doctor` is local-only; a live prompt separately verifies provider access. Remove the README's pending-publication notice only after registry publication succeeds.
+
+Do not push `v0.2.0` to test automation after this bootstrap: it would attempt to publish an existing version. npm name/version pairs cannot be reused, including after unpublishing. See the [npm publish reference](https://docs.npmjs.com/cli/v11/commands/npm-publish/).
+
+## 2. Authorize this workflow on npm
+
+Trusted publishing requires the package to exist first, which is why the initial version is published manually. The account configuring trust needs package write access and 2FA. See [npm trust prerequisites](https://docs.npmjs.com/cli/v11/commands/npm-trust/#prerequisites).
+
+On npmjs.com, open the `claude-autorouter` package's **Settings → Trusted Publisher**, select **GitHub Actions**, and enter:
+
+| Setting | Value |
+| --- | --- |
+| Organization or user | `frapposelli` |
+| Repository | `claude-autorouter` |
+| Workflow filename | `publish.yml` |
+| Environment name | Leave blank |
+| Allowed actions | Enable direct `npm publish` |
+
+Use the filename only, not `.github/workflows/publish.yml`. No GitHub environment or npm token secret needs to be created. The owner, repository, and workflow must match exactly. New trust configurations default to permitting staged publication; **enable direct `npm publish`** for this workflow. See [npm trusted publishers](https://docs.npmjs.com/trusted-publishers/) and [staged publishing](https://docs.npmjs.com/staged-publishing/).
+
+The workflow explicitly disables provenance because npm provenance is unsupported for private source repositories, even when the npm package is public. OIDC authentication still works. If the repository becomes public, review the workflow and metadata before enabling provenance. See [npm provenance requirements](https://docs.npmjs.com/generating-provenance-statements/).
+
+After a successful trusted release, npm recommends the optional **Publishing access → Require two-factor authentication and disallow tokens** setting. It does not disable OIDC publishing. See [restricting token access](https://docs.npmjs.com/trusted-publishers/#recommended-restrict-token-access-when-using-trusted-publishers).
+
+## 3. Release subsequent versions by tag
+
+For the next patch after the bootstrap, prepare `0.2.1` on `main` or through a pull request:
+
+```sh
+git switch main
+git pull --ff-only origin main
+npm version 0.2.1 --no-git-tag-version
+```
+
+Review the version change and update any version-specific install examples or release notes. Check the candidate using the new filename:
+
+```sh
+npm run check
+npm test
+npm run release:pack
+npm run test:package -- --archive ./dist/claude-autorouter-0.2.1.tgz
+git diff --check
+```
+
+Commit the intended release changes and get that commit onto `main`, either through a pull request or a direct push allowed by the repository's branch rules. For a direct push with only the version changed:
+
+```sh
+git add package.json
+git commit -m "Release 0.2.1"
+git push origin main
+```
+
+Include any intentional documentation or release-note edits in that commit too. There is no publication from a branch push or PR merge. Wait for CI to pass, then tag that exact release commit:
+
+```sh
+git switch main
+git pull --ff-only origin main
+git tag -a v0.2.1 -m "Release 0.2.1"
+git push origin v0.2.1
+```
+
+Before pushing, confirm `package.json` contains `0.2.1` and the tag points to the intended commit. For a prerelease, use a matching version/tag such as `0.3.0-beta.1` / `v0.3.0-beta.1`; it will publish under `next`, leaving `latest` unchanged.
+
+Release stable versions in increasing version order, one tag at a time, and wait for each run to finish before pushing the next stable tag. The workflow queues releases without canceling an active run, but queue order does not sort semantic versions. Publishing an older stable version afterward could move `latest` backward; there is no registry version-order gate.
+
+Open the tag's run under [GitHub Actions](https://github.com/frapposelli/claude-autorouter/actions). Under **Artifacts**, download `npm-package-<run-id>-<run-attempt>`, which contains the `.tgz` and checksum used for publication. Artifacts expire after 30 days, so retain them with the release record. After the publish job succeeds, verify the registry version and tags:
+
+```sh
+npm view claude-autorouter@0.2.1 version dist.integrity --registry https://registry.npmjs.org/
+npm view claude-autorouter dist-tags --json --registry https://registry.npmjs.org/
+```
+
+Repeat the independent installation check for the released version. A GitHub Release page is optional; pushing the version tag is the publication trigger.
+
+For local release diagnostics after the tag exists, `node scripts/release-check.mjs source v0.2.1` checks the tag, clean checkout, metadata, and ancestry. `node scripts/release-check.mjs archive v0.2.1` checks the candidate checksum and contents. These helpers are run automatically in the release workflow; the first untagged bootstrap uses the checks in step 1 instead.
+
+## Recovering a failed release
+
+- **Checks or archive validation failed:** nothing is published. Fix the cause and repeat validation before making a new release tag. Do not move a tag that already identifies a published version.
+- **npm rejected OIDC authentication:** verify the npm trust fields, direct-publish permission, GitHub-hosted runner, and the publish job's OIDC permission. After correcting npm configuration, rerun the failed job if that version is still unpublished.
+- **A publish timed out or the run was interrupted:** check `npm view` for the exact version before retrying. The registry may have accepted it before the connection failed.
+- **The version already exists:** inspect the registry release; do not overwrite or unpublish to reuse it. Code or documentation corrections need a new version.
+- **Local bootstrap authentication failed:** complete `npm login` and the account's 2FA flow in your terminal. CI trust cannot create the first package or substitute for that account step.
+
+If release code or workflow changes are needed, commit the fix to `main` and prepare a new version/tag. Authentication-only corrections on npm can be retried against the unchanged, unpublished candidate.

package/package.json

@@ -0,0 +1,25 @@
+{
+  "name": "claude-autorouter",
+  "version": "0.2.0",
+  "license": "Apache-2.0",
+  "type": "module",
+  "description": "A local Claude Code model router with Jev and Ollama evaluators",
+  "repository": { "type": "git", "url": "git+https://github.com/frapposelli/claude-autorouter.git" },
+  "bin": { "claude-autorouter": "bin/autorouter.mjs" },
+  "engines": { "node": ">=22" },
+  "os": ["darwin", "linux"],
+  "keywords": ["claude", "claude-code", "model-routing", "jev", "ollama", "cli"],
+  "files": ["bin/*.mjs", "src/*.mjs", "docs/reference.md", "docs/development.md", "docs/releasing.md", "docs/ollama-evaluation.md", ".env.example", "LICENSE"],
+  "publishConfig": { "access": "public", "registry": "https://registry.npmjs.org/" },
+  "scripts": {
+    "start": "node bin/autorouter.mjs serve",
+    "claude": "node bin/autorouter.mjs claude",
+    "test": "node --test test/*.test.mjs",
+    "test:package": "node scripts/package-smoke.mjs",
+    "release:pack": "node scripts/release-pack.mjs",
+    "eval": "node --env-file=.env scripts/evaluate.mjs",
+    "eval:ollama": "node scripts/evaluate-ollama.mjs",
+    "test:live": "node --env-file=.env scripts/live-validation.mjs",
+    "check": "node scripts/check.mjs"
+  }
+}

package/src/auth.mjs

@@ -0,0 +1,51 @@
+export const LOCAL_AUTH_HEADER = 'x-autorouter-token';
+
+export function conflictingProviders(env = process.env) {
+  return ['CLAUDE_CODE_USE_BEDROCK', 'CLAUDE_CODE_USE_VERTEX', 'CLAUDE_CODE_USE_FOUNDRY',
+    'CLAUDE_CODE_USE_MANTLE', 'CLAUDE_CODE_USE_ANTHROPIC_AWS']
+    .filter(key => ['1', 'true'].includes(String(env[key]).toLowerCase()));
+}
+
+// This recognizes the subscription wire format, not the validity of the
+// credential. Anthropic validates the bearer token on every forwarded request.
+export function isSubscriptionRequest(headers) {
+  return !headers['x-api-key']
+    && /^Bearer \S+$/i.test(headers.authorization ?? '')
+    && String(headers['anthropic-beta'] ?? '').split(',').some(value => /^oauth-/.test(value.trim()));
+}
+
+export function buildClaudeEnv(config, baseUrl, parent = process.env) {
+  const env = { ...parent, ANTHROPIC_BASE_URL: baseUrl, CLAUDE_CODE_GATEWAY_HINT_HEADERS: '1' };
+  // Claude Code otherwise disables MCP tool search for a non-first-party
+  // base URL and loads every schema into context. This proxy preserves both
+  // tool_reference blocks and their beta headers. Respect explicit choices.
+  if (env.ENABLE_TOOL_SEARCH === undefined) env.ENABLE_TOOL_SEARCH = 'true';
+  // Start with a request format all three tiers accept. Jev still chooses the
+  // upstream model. Native mode preserves the client's full feature selection.
+  if (config.clientProfile === 'compatible') {
+    env.ANTHROPIC_MODEL = config.models.haiku;
+    env.MAX_THINKING_TOKENS = '0';
+  }
+  const subscription = config.authMode === 'subscription';
+  // Keep unrelated custom headers. Remove stale router credentials and, in
+  // subscription mode, inherited custom auth that could override the login.
+  const headers = String(env.ANTHROPIC_CUSTOM_HEADERS ?? '').split(/\r?\n/).filter(line => {
+    const name = line.split(':', 1)[0].trim().toLowerCase();
+    return line.trim() && name !== LOCAL_AUTH_HEADER
+      && !(subscription && ['authorization', 'x-api-key'].includes(name));
+  });
+  if (subscription) {
+    delete env.ANTHROPIC_API_KEY;
+    delete env.ANTHROPIC_AUTH_TOKEN;
+    delete env.CLAUDE_CODE_OAUTH_TOKEN;
+    headers.push(`X-Autorouter-Token: ${config.localToken}`);
+  } else {
+    env.ANTHROPIC_API_KEY = config.localToken;
+    env.ANTHROPIC_AUTH_TOKEN = config.localToken;
+  }
+  if (headers.length) env.ANTHROPIC_CUSTOM_HEADERS = headers.join('\n');
+  else delete env.ANTHROPIC_CUSTOM_HEADERS;
+  delete env.TYPESAFE_API_KEY;
+  delete env.AUTOROUTER_TOKEN;
+  return env;
+}