claude-autorouter 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/.env.example ADDED
@@ -0,0 +1,45 @@
1
+ # Copy to .env and load with: node --env-file=.env bin/autorouter.mjs claude
2
+ AUTOROUTER_AUTH_MODE=subscription
3
+ AUTOROUTER_CLIENT_PROFILE=compatible
4
+ # Jev remains the default evaluator. Ollama is experimental; see settings below.
5
+ AUTOROUTER_EVALUATOR=jev
6
+ # The launcher enables the router status line for this session. Set 0 to keep your own.
7
+ AUTOROUTER_STATUSLINE=1
8
+ # Optional metadata logs on stderr. Redirect stderr to a file when using the UI.
9
+ # AUTOROUTER_DEBUG=1
10
+ # Required for Jev only; subscription + Ollama needs no API keys.
11
+ TYPESAFE_API_KEY=
12
+ # For API billing instead, set AUTOROUTER_AUTH_MODE=api-key and fill this in.
13
+ # ANTHROPIC_API_KEY=
14
+
15
+ # Model IDs are configurable independently of the tier policy.
16
+ AUTOROUTER_HAIKU_MODEL=claude-haiku-4-5-20251001
17
+ AUTOROUTER_SONNET_MODEL=claude-sonnet-5
18
+ AUTOROUTER_OPUS_MODEL=claude-opus-5-5
19
+ AUTOROUTER_JEV_MODEL=jev-latest
20
+ AUTOROUTER_JEV_TIMEOUT_MS=1500
21
+ # Only suspicious context sizes need counting; this overlaps the evaluator.
22
+ AUTOROUTER_TOKEN_COUNT_TIMEOUT_MS=1500
23
+ AUTOROUTER_MIN_CONFIDENCE=0.75
24
+
25
+ # Experimental local evaluator: install/start Ollama, then configure with:
26
+ # claude-autorouter setup --evaluator ollama --ollama-preset compact --pull
27
+ # compact uses less memory; quality (qwen3:4b) had better synthetic rubric agreement.
28
+ # Both were tested on a 16 GiB M4. Choose quality explicitly when memory permits.
29
+ # auto selects quality above 24 GiB total RAM; this is only a headroom heuristic.
30
+ # Add --force to replace an existing user config. Environment values win over it.
31
+ # Or set AUTOROUTER_EVALUATOR=ollama above and configure an installed local model:
32
+ # AUTOROUTER_OLLAMA_URL=http://127.0.0.1:11434
33
+ # AUTOROUTER_OLLAMA_MODEL=qwen3:1.7b
34
+ # AUTOROUTER_OLLAMA_TIMEOUT_MS=1500
35
+ # AUTOROUTER_OLLAMA_KEEP_ALIVE=5m
36
+ # The launcher primes the classifier before opening Claude, allowing up to 60s.
37
+ # Both presets timed out on full-excerpt stress tests at the runtime deadline.
38
+ # See docs/ollama-evaluation.md for measured latency, memory, and accuracy limits.
39
+ # After the idle period, cold reloading may exceed the deadline and use fallback.
40
+ # A longer keep-alive holds the model in memory longer but avoids some reloads.
41
+ # The confidence threshold above applies only to Jev. Ollama returns a tier.
42
+
43
+ AUTOROUTER_PORT=8787
44
+ # Required only for standalone `serve`; the `claude` launcher generates one.
45
+ AUTOROUTER_TOKEN=
package/LICENSE ADDED
@@ -0,0 +1,202 @@
1
+
2
+ Apache License
3
+ Version 2.0, January 2004
4
+ http://www.apache.org/licenses/
5
+
6
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
7
+
8
+ 1. Definitions.
9
+
10
+ "License" shall mean the terms and conditions for use, reproduction,
11
+ and distribution as defined by Sections 1 through 9 of this document.
12
+
13
+ "Licensor" shall mean the copyright owner or entity authorized by
14
+ the copyright owner that is granting the License.
15
+
16
+ "Legal Entity" shall mean the union of the acting entity and all
17
+ other entities that control, are controlled by, or are under common
18
+ control with that entity. For the purposes of this definition,
19
+ "control" means (i) the power, direct or indirect, to cause the
20
+ direction or management of such entity, whether by contract or
21
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
22
+ outstanding shares, or (iii) beneficial ownership of such entity.
23
+
24
+ "You" (or "Your") shall mean an individual or Legal Entity
25
+ exercising permissions granted by this License.
26
+
27
+ "Source" form shall mean the preferred form for making modifications,
28
+ including but not limited to software source code, documentation
29
+ source, and configuration files.
30
+
31
+ "Object" form shall mean any form resulting from mechanical
32
+ transformation or translation of a Source form, including but
33
+ not limited to compiled object code, generated documentation,
34
+ and conversions to other media types.
35
+
36
+ "Work" shall mean the work of authorship, whether in Source or
37
+ Object form, made available under the License, as indicated by a
38
+ copyright notice that is included in or attached to the work
39
+ (an example is provided in the Appendix below).
40
+
41
+ "Derivative Works" shall mean any work, whether in Source or Object
42
+ form, that is based on (or derived from) the Work and for which the
43
+ editorial revisions, annotations, elaborations, or other modifications
44
+ represent, as a whole, an original work of authorship. For the purposes
45
+ of this License, Derivative Works shall not include works that remain
46
+ separable from, or merely link (or bind by name) to the interfaces of,
47
+ the Work and Derivative Works thereof.
48
+
49
+ "Contribution" shall mean any work of authorship, including
50
+ the original version of the Work and any modifications or additions
51
+ to that Work or Derivative Works thereof, that is intentionally
52
+ submitted to Licensor for inclusion in the Work by the copyright owner
53
+ or by an individual or Legal Entity authorized to submit on behalf of
54
+ the copyright owner. For the purposes of this definition, "submitted"
55
+ means any form of electronic, verbal, or written communication sent
56
+ to the Licensor or its representatives, including but not limited to
57
+ communication on electronic mailing lists, source code control systems,
58
+ and issue tracking systems that are managed by, or on behalf of, the
59
+ Licensor for the purpose of discussing and improving the Work, but
60
+ excluding communication that is conspicuously marked or otherwise
61
+ designated in writing by the copyright owner as "Not a Contribution."
62
+
63
+ "Contributor" shall mean Licensor and any individual or Legal Entity
64
+ on behalf of whom a Contribution has been received by Licensor and
65
+ subsequently incorporated within the Work.
66
+
67
+ 2. Grant of Copyright License. Subject to the terms and conditions of
68
+ this License, each Contributor hereby grants to You a perpetual,
69
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
70
+ copyright license to reproduce, prepare Derivative Works of,
71
+ publicly display, publicly perform, sublicense, and distribute the
72
+ Work and such Derivative Works in Source or Object form.
73
+
74
+ 3. Grant of Patent License. Subject to the terms and conditions of
75
+ this License, each Contributor hereby grants to You a perpetual,
76
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
77
+ (except as stated in this section) patent license to make, have made,
78
+ use, offer to sell, sell, import, and otherwise transfer the Work,
79
+ where such license applies only to those patent claims licensable
80
+ by such Contributor that are necessarily infringed by their
81
+ Contribution(s) alone or by combination of their Contribution(s)
82
+ with the Work to which such Contribution(s) was submitted. If You
83
+ institute patent litigation against any entity (including a
84
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
85
+ or a Contribution incorporated within the Work constitutes direct
86
+ or contributory patent infringement, then any patent licenses
87
+ granted to You under this License for that Work shall terminate
88
+ as of the date such litigation is filed.
89
+
90
+ 4. Redistribution. You may reproduce and distribute copies of the
91
+ Work or Derivative Works thereof in any medium, with or without
92
+ modifications, and in Source or Object form, provided that You
93
+ meet the following conditions:
94
+
95
+ (a) You must give any other recipients of the Work or
96
+ Derivative Works a copy of this License; and
97
+
98
+ (b) You must cause any modified files to carry prominent notices
99
+ stating that You changed the files; and
100
+
101
+ (c) You must retain, in the Source form of any Derivative Works
102
+ that You distribute, all copyright, patent, trademark, and
103
+ attribution notices from the Source form of the Work,
104
+ excluding those notices that do not pertain to any part of
105
+ the Derivative Works; and
106
+
107
+ (d) If the Work includes a "NOTICE" text file as part of its
108
+ distribution, then any Derivative Works that You distribute must
109
+ include a readable copy of the attribution notices contained
110
+ within such NOTICE file, excluding those notices that do not
111
+ pertain to any part of the Derivative Works, in at least one
112
+ of the following places: within a NOTICE text file distributed
113
+ as part of the Derivative Works; within the Source form or
114
+ documentation, if provided along with the Derivative Works; or,
115
+ within a display generated by the Derivative Works, if and
116
+ wherever such third-party notices normally appear. The contents
117
+ of the NOTICE file are for informational purposes only and
118
+ do not modify the License. You may add Your own attribution
119
+ notices within Derivative Works that You distribute, alongside
120
+ or as an addendum to the NOTICE text from the Work, provided
121
+ that such additional attribution notices cannot be construed
122
+ as modifying the License.
123
+
124
+ You may add Your own copyright statement to Your modifications and
125
+ may provide additional or different license terms and conditions
126
+ for use, reproduction, or distribution of Your modifications, or
127
+ for any such Derivative Works as a whole, provided Your use,
128
+ reproduction, and distribution of the Work otherwise complies with
129
+ the conditions stated in this License.
130
+
131
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
132
+ any Contribution intentionally submitted for inclusion in the Work
133
+ by You to the Licensor shall be under the terms and conditions of
134
+ this License, without any additional terms or conditions.
135
+ Notwithstanding the above, nothing herein shall supersede or modify
136
+ the terms of any separate license agreement you may have executed
137
+ with Licensor regarding such Contributions.
138
+
139
+ 6. Trademarks. This License does not grant permission to use the trade
140
+ names, trademarks, service marks, or product names of the Licensor,
141
+ except as required for reasonable and customary use in describing the
142
+ origin of the Work and reproducing the content of the NOTICE file.
143
+
144
+ 7. Disclaimer of Warranty. Unless required by applicable law or
145
+ agreed to in writing, Licensor provides the Work (and each
146
+ Contributor provides its Contributions) on an "AS IS" BASIS,
147
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
148
+ implied, including, without limitation, any warranties or conditions
149
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
150
+ PARTICULAR PURPOSE. You are solely responsible for determining the
151
+ appropriateness of using or redistributing the Work and assume any
152
+ risks associated with Your exercise of permissions under this License.
153
+
154
+ 8. Limitation of Liability. In no event and under no legal theory,
155
+ whether in tort (including negligence), contract, or otherwise,
156
+ unless required by applicable law (such as deliberate and grossly
157
+ negligent acts) or agreed to in writing, shall any Contributor be
158
+ liable to You for damages, including any direct, indirect, special,
159
+ incidental, or consequential damages of any character arising as a
160
+ result of this License or out of the use or inability to use the
161
+ Work (including but not limited to damages for loss of goodwill,
162
+ work stoppage, computer failure or malfunction, or any and all
163
+ other commercial damages or losses), even if such Contributor
164
+ has been advised of the possibility of such damages.
165
+
166
+ 9. Accepting Warranty or Additional Liability. While redistributing
167
+ the Work or Derivative Works thereof, You may choose to offer,
168
+ and charge a fee for, acceptance of support, warranty, indemnity,
169
+ or other liability obligations and/or rights consistent with this
170
+ License. However, in accepting such obligations, You may act only
171
+ on Your own behalf and on Your sole responsibility, not on behalf
172
+ of any other Contributor, and only if You agree to indemnify,
173
+ defend, and hold each Contributor harmless for any liability
174
+ incurred by, or claims asserted against, such Contributor by reason
175
+ of your accepting any such warranty or additional liability.
176
+
177
+ END OF TERMS AND CONDITIONS
178
+
179
+ APPENDIX: How to apply the Apache License to your work.
180
+
181
+ To apply the Apache License to your work, attach the following
182
+ boilerplate notice, with the fields enclosed by brackets "[]"
183
+ replaced with your own identifying information. (Don't include
184
+ the brackets!) The text should be enclosed in the appropriate
185
+ comment syntax for the file format. We also recommend that a
186
+ file or class name and description of purpose be included on the
187
+ same "printed page" as the copyright notice for easier
188
+ identification within third-party archives.
189
+
190
+ Copyright [yyyy] [name of copyright owner]
191
+
192
+ Licensed under the Apache License, Version 2.0 (the "License");
193
+ you may not use this file except in compliance with the License.
194
+ You may obtain a copy of the License at
195
+
196
+ http://www.apache.org/licenses/LICENSE-2.0
197
+
198
+ Unless required by applicable law or agreed to in writing, software
199
+ distributed under the License is distributed on an "AS IS" BASIS,
200
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
201
+ See the License for the specific language governing permissions and
202
+ limitations under the License.
package/README.md ADDED
@@ -0,0 +1,87 @@
1
+ # Claude AutoRouter
2
+
3
+ Use Haiku, Sonnet, and Opus in one Claude Code session. A local gateway classifies each inference request with the selected evaluator, applies compatibility and context checks, and streams the selected model's response back to Claude Code. [TypeSafe Jev](https://typesafe.ai/blog/introducing-system-one-models-and-jev) is the default; an experimental Ollama backend evaluates requests locally.
4
+
5
+ Requires Node.js 22+, macOS or Linux (including WSL), an installed `claude` command, and a Claude subscription login or Anthropic API key. The default evaluator also requires a [TypeSafe API key](https://console.typesafe.ai). There are no runtime package dependencies. Native Windows is not supported in this release.
6
+
7
+ ## Install and start
8
+
9
+ **npm publication is pending.** Install the release tarball now:
10
+
11
+ ```sh
12
+ npm install -g ./claude-autorouter-0.2.0.tgz
13
+ ```
14
+
15
+ Once the package is published, install it from the registry with:
16
+
17
+ ```sh
18
+ npm install -g claude-autorouter
19
+ ```
20
+
21
+ Set up once, then launch from any project directory:
22
+
23
+ ```sh
24
+ claude-autorouter setup
25
+ claude-autorouter doctor
26
+ cd /path/to/project
27
+ claude-autorouter claude
28
+ ```
29
+
30
+ Setup defaults to your Claude subscription and prompts for your Jev key without echoing it. If Claude is not already signed in, run `claude auth login`. No Anthropic API key or exported subscription token is needed for subscription mode. Jev has separate credentials and billing.
31
+
32
+ Setup saves a private JSON config at `~/.config/claude-autorouter/config.json`; `XDG_CONFIG_HOME` and `AUTOROUTER_CONFIG` can change its location. Environment variables override saved configuration. Project `.env` files are not loaded automatically.
33
+
34
+ Claude Code arguments pass through:
35
+
36
+ ```sh
37
+ claude-autorouter claude -p "Fix the typo in README.md"
38
+ claude-autorouter --help
39
+ claude-autorouter --version
40
+ ```
41
+
42
+ For API billing, use `claude-autorouter setup --auth-mode api-key`. Use `--force` to replace an existing config. Automation can supply `TYPESAFE_API_KEY` and, in API-key mode, `ANTHROPIC_API_KEY` through the environment; keys are never command-line arguments. `doctor` checks local configuration and Claude installation/login state without paid requests. See the [configuration reference](docs/reference.md#configuration).
43
+
44
+ ## What you see
45
+
46
+ The launcher adds a temporary status line and leaves saved Claude Code settings unchanged:
47
+
48
+ ```text
49
+ ● AutoRouter · last Haiku 4.5 · ready · Jev 210ms · est saved $0.04 (75%) vs Opus
50
+ ● AutoRouter · Sonnet 5 selected · connecting · Jev→Haiku 290ms · large context
51
+ ```
52
+
53
+ The confirmed model comes from Anthropic's response. Claude's own model label can still show its Haiku starting model. `API ctx` measures input against the actual model's known window; a different client limit remains visible as `CLI ctx`.
54
+
55
+ Savings are an **API-equivalent estimate for the same token counts**, using Opus as the baseline. They do not measure subscription bill reductions or quota credits and exclude Jev and local compute costs. [Status line and savings details](docs/reference.md#status-line-and-savings).
56
+
57
+ ## Experimental local evaluator
58
+
59
+ [Install and start Ollama](https://docs.ollama.com/quickstart), then select a preset. The default `compact` uses less memory; `quality` showed better agreement with the routing rubric:
60
+
61
+ ```sh
62
+ claude-autorouter setup --evaluator ollama --ollama-preset compact --pull
63
+ claude-autorouter doctor
64
+ claude-autorouter claude
65
+ ```
66
+
67
+ Add `--force` when replacing an existing config. Setup downloads a missing selected model only with `--pull`; it does not install or start Ollama. No Jev key is needed. Claude still answers through Anthropic, with the same routing guards and subscription limits.
68
+
69
+ The launcher primes the local classifier before opening Claude's UI. Failed evaluations fall back to Sonnet or retain Opus without contacting Jev. See the [Ollama reference](docs/reference.md#ollama-evaluator) for setup options.
70
+
71
+ Both presets were tested on a 16 GiB M4 Mac using 24 distinct held-out synthetic workloads repeated three times:
72
+
73
+ | Preset | Rubric agreement | Warm p50 / p95 | Model allocation |
74
+ | --- | ---: | ---: | ---: |
75
+ | `compact` (`qwen3:1.7b`) | 58.3% | 602 / 834 ms | 1.70 GB |
76
+ | `quality` (`qwen3:4b`) | 91.7% | 889 / 1,242 ms | 3.18 GB |
77
+
78
+ Use `--ollama-preset quality` to choose the larger model when memory permits, including on a 16 GiB machine. Each model timed out on all eight full-excerpt stress requests at the 1,500 ms deadline. These results do not establish parity with Jev or completed-task quality. Read the [measurements and limits](docs/ollama-evaluation.md) before choosing local classification.
79
+
80
+ ## Behavior and data
81
+
82
+ - The default client profile permits all three routing tiers. Tool continuations, thinking history, model-specific features, and context size can keep or upgrade a model even when the evaluator chooses a cheaper tier. [Routing policy](docs/reference.md#routing-policy).
83
+ - The selected evaluator receives bounded excerpts that can contain source code, system instructions, and tool results: TypeSafe with Jev, or the local service with Ollama. Anthropic receives the complete request. Images, document payloads, and private thinking are omitted from classifier input. [Data flow and authentication](docs/reference.md#data-flow-and-authentication).
84
+ - Subscription access and usage limits still apply. Model switches can reduce cache reuse; cheaper token prices do not guarantee cheaper completed tasks. Run ordinary `claude` to bypass routing.
85
+ - The launcher is quiet by default. Use `AUTOROUTER_DEBUG=1` for metadata diagnostics or `AUTOROUTER_STATUSLINE=0` to retain your existing status line. [Troubleshooting](docs/reference.md#troubleshooting).
86
+
87
+ [Reference](docs/reference.md) · [Development and validation](docs/development.md) · [CI and npm release setup](docs/releasing.md) · [Apache-2.0 license](LICENSE)
@@ -0,0 +1,136 @@
1
+ #!/usr/bin/env node
2
+ import { randomBytes } from 'node:crypto';
3
+ import { spawn } from 'node:child_process';
4
+ import { readFileSync } from 'node:fs';
5
+ import { readConfig, requireKeys } from '../src/config.mjs';
6
+ import { createRouterServer, listen } from '../src/server.mjs';
7
+ import { buildClaudeEnv, conflictingProviders } from '../src/auth.mjs';
8
+ import { dirname } from 'node:path';
9
+ import { createStatusState } from '../src/status-state.mjs';
10
+ import { addStatusLineSettings } from '../src/status-settings.mjs';
11
+ import { loadUserConfig } from '../src/user-config.mjs';
12
+ import { setup, doctor } from '../src/onboarding.mjs';
13
+ import { setupOllama } from '../src/ollama-setup.mjs';
14
+
15
+ const [command = 'help', ...args] = process.argv.slice(2);
16
+ if (['--version', '-v', 'version'].includes(command)) {
17
+ console.log(JSON.parse(readFileSync(new URL('../package.json', import.meta.url), 'utf8')).version);
18
+ } else if (['help', '--help', '-h'].includes(command)
19
+ || (['setup', 'doctor', 'serve'].includes(command) && args.some(arg => ['--help', '-h'].includes(arg)))) {
20
+ console.log(`Claude AutoRouter — routing for Haiku, Sonnet, and Opus
21
+
22
+ Usage:
23
+ claude-autorouter setup [--auth-mode subscription|api-key] [--force]
24
+ [--evaluator jev|ollama] [--ollama-preset compact|quality|auto]
25
+ [--ollama-model MODEL] [--pull]
26
+ claude-autorouter doctor
27
+ claude-autorouter claude [Claude Code arguments]
28
+ claude-autorouter serve
29
+ claude-autorouter --version
30
+
31
+ Setup defaults to subscription authentication and prompts for keys without echoing.
32
+ For noninteractive setup, supply keys through environment variables.
33
+ User config: ~/.config/claude-autorouter/config.json (or XDG_CONFIG_HOME).
34
+ AUTOROUTER_CONFIG selects a different file; environment variables take precedence.
35
+ Project .env files are never loaded automatically.
36
+
37
+ Jev is the default evaluator and requires TYPESAFE_API_KEY.
38
+ Ollama evaluates locally and requires a running local Ollama service.
39
+ Use setup --evaluator ollama --pull to detect Ollama and download a missing model.
40
+ Compact uses Qwen3 1.7B; quality offers Qwen3 4B for machines over 24 GB.
41
+ Local routing is experimental; see docs/ollama-evaluation.md for measured limits.
42
+ The auto preset selects using total RAM; compact is the default.
43
+ AUTOROUTER_AUTH_MODE=subscription uses your saved Claude Code login.
44
+ Without setup, AUTOROUTER_AUTH_MODE defaults to api-key and also requires ANTHROPIC_API_KEY.
45
+ AUTOROUTER_CLIENT_PROFILE=compatible (default) enables all three routing tiers.
46
+ Use AUTOROUTER_CLIENT_PROFILE=native to retain Claude Code's own model/thinking settings.
47
+ Standalone serve also requires AUTOROUTER_TOKEN (at least 16 characters).
48
+ The claude launcher creates a temporary credential and an ephemeral port.
49
+ It enables an AutoRouter status line for this session (AUTOROUTER_STATUSLINE=0 to opt out).
50
+ Launcher logs are quiet by default; AUTOROUTER_DEBUG=1 enables diagnostic logs on stderr.
51
+ Jev sends prompt excerpts to TypeSafe; Ollama keeps classification on this machine.
52
+ Complete inference requests still go to Anthropic. See README.md.`);
53
+ } else if (command === 'setup' || command === 'doctor') {
54
+ try {
55
+ if (command === 'setup') await setup(args);
56
+ else {
57
+ if (args.length) throw new Error('Usage: claude-autorouter doctor');
58
+ if (!await doctor()) process.exitCode = 1;
59
+ }
60
+ } catch (error) { console.error(error.message); process.exitCode = 1; }
61
+ } else if (!['claude', 'serve'].includes(command)) {
62
+ console.error('Unknown command. Run claude-autorouter --help.');
63
+ process.exitCode = 1;
64
+ } else {
65
+ let server;
66
+ let status;
67
+ const stop = () => {
68
+ if (server) { server.close(); server.closeAllConnections(); }
69
+ status?.close();
70
+ };
71
+ try {
72
+ if (command === 'serve' && args.length) throw new Error('Usage: claude-autorouter serve');
73
+ const runtimeEnv = loadUserConfig().env;
74
+ const config = readConfig(runtimeEnv);
75
+ requireKeys(config);
76
+ const diagnosticLogs = command === 'serve' || runtimeEnv.AUTOROUTER_DEBUG === '1';
77
+ if (command === 'claude') {
78
+ if (config.authMode === 'subscription' && args.includes('--bare')) {
79
+ throw new Error('--bare disables Claude Code OAuth; omit it in subscription mode');
80
+ }
81
+ for (const key of conflictingProviders(runtimeEnv)) {
82
+ throw new Error(`Unset ${key}; this router supports the Anthropic Messages API`);
83
+ }
84
+ config.localToken = randomBytes(32).toString('hex');
85
+ }
86
+ if (config.evaluator === 'ollama') {
87
+ console.error(`Preparing local Ollama evaluator (${config.ollamaModel})…`);
88
+ try { await setupOllama(config, { pull: false, warm: true, write: () => {} }); }
89
+ catch {
90
+ console.error('Ollama could not be prepared. Requests will use the conservative fallback while it is unavailable; run claude-autorouter doctor.');
91
+ }
92
+ }
93
+ const statusEnabled = command === 'claude' && runtimeEnv.AUTOROUTER_STATUSLINE !== '0';
94
+ let claudeArgs = args;
95
+ if (statusEnabled) {
96
+ status = createStatusState({ baselineModel: config.models.opus });
97
+ if (status.path) {
98
+ try { claudeArgs = addStatusLineSettings(args, dirname(status.path)); }
99
+ catch {
100
+ status.close(); status = undefined;
101
+ console.error('AutoRouter status line unavailable: could not safely prepare session settings. Passing your original settings to Claude.');
102
+ }
103
+ } else console.error('AutoRouter status line unavailable: could not create local status storage.');
104
+ }
105
+ // Claude owns the terminal while its UI is running. Status updates use the
106
+ // local snapshot independently; proxy JSON must not write over the UI.
107
+ server = createRouterServer(config, {
108
+ log: diagnosticLogs ? undefined : () => {},
109
+ onStatus: event => status?.update(event),
110
+ });
111
+ const address = await listen(server, command === 'claude' ? 0 : config.port);
112
+ const baseUrl = `http://127.0.0.1:${address.port}`;
113
+ if (diagnosticLogs) console.error(`AutoRouter listening on ${baseUrl} (${config.authMode} authentication)`);
114
+ if (diagnosticLogs && command === 'claude' && config.clientProfile === 'compatible') {
115
+ console.error(`AutoRouter uses Haiku-compatible requests with client thinking disabled; ${config.evaluator === 'ollama' ? 'Ollama' : 'Jev'} selects the upstream model.`);
116
+ }
117
+ if (command === 'serve') {
118
+ for (const signal of ['SIGINT', 'SIGTERM']) process.once(signal, () => {
119
+ stop();
120
+ });
121
+ } else {
122
+ const env = buildClaudeEnv(config, baseUrl, runtimeEnv);
123
+ delete env.AUTOROUTER_CONFIG;
124
+ delete env.AUTOROUTER_STATUS_FILE;
125
+ if (status?.path) env.AUTOROUTER_STATUS_FILE = status.path;
126
+ const child = spawn('claude', claudeArgs, { stdio: 'inherit', env });
127
+ child.once('error', () => { console.error('Could not launch Claude Code. Ensure `claude` is installed and on PATH.'); process.exitCode = 1; stop(); });
128
+ child.once('exit', (code, signal) => { process.exitCode = code ?? (signal === 'SIGINT' ? 130 : 1); stop(); });
129
+ for (const signal of ['SIGINT', 'SIGTERM']) process.on(signal, () => child.kill(signal));
130
+ }
131
+ } catch (error) {
132
+ console.error(error.message);
133
+ stop();
134
+ process.exitCode = 1;
135
+ }
136
+ }
@@ -0,0 +1,32 @@
1
+ #!/usr/bin/env node
2
+ import { createReadStream } from 'node:fs';
3
+ import { stat } from 'node:fs/promises';
4
+ import { renderStatusLine } from '../src/statusline.mjs';
5
+
6
+ const LIMIT = 1024 * 1024;
7
+ async function readJson(stream) {
8
+ let size = 0;
9
+ const chunks = [];
10
+ try {
11
+ for await (const chunk of stream) {
12
+ size += chunk.length;
13
+ if (size <= LIMIT) chunks.push(chunk);
14
+ else chunks.length = 0;
15
+ }
16
+ return size <= LIMIT ? JSON.parse(Buffer.concat(chunks).toString('utf8')) : undefined;
17
+ } catch { return undefined; }
18
+ }
19
+ async function readSnapshot(path) {
20
+ if (!path) return undefined;
21
+ try {
22
+ const info = await stat(path);
23
+ if (!info.isFile() || info.size > LIMIT) return undefined;
24
+ // Bound the actual read as well: the file can grow after stat().
25
+ return await readJson(createReadStream(path, { start: 0, end: LIMIT }));
26
+ } catch { return undefined; }
27
+ }
28
+ const [input, snapshot] = await Promise.all([readJson(process.stdin), readSnapshot(process.env.AUTOROUTER_STATUS_FILE)]);
29
+ const alive = pid => { try { process.kill(pid, 0); return true; } catch (error) { return error.code === 'EPERM'; } };
30
+ process.stdout.write(renderStatusLine(input, snapshot, {
31
+ color: !Object.hasOwn(process.env, 'NO_COLOR') && process.env.TERM !== 'dumb', alive,
32
+ }) + '\n');
@@ -0,0 +1,84 @@
1
+ # Development and validation
2
+
3
+ Use Node.js 22+ from a source checkout on macOS or Linux (including WSL). The project has no runtime package dependencies. Development scripts and tests are separate from the installed CLI; user setup is covered in the [README](../README.md).
4
+
5
+ ## Local checks
6
+
7
+ ```sh
8
+ npm run check
9
+ npm test
10
+ npm run test:package
11
+ ```
12
+
13
+ The test suite uses local mocks and fake credentials. It covers Jev and Ollama routing, bounded prompt extraction, confidence and timeout fallback, token checks, model continuity, authentication forwarding, streaming, cancellation, status state, savings, and launcher behavior. Tests that start HTTP services require loopback binding. Local tests make no paid provider calls or model downloads.
14
+
15
+ Package validation checks the distributable and installed command rather than relying on the source checkout's paths. Review the [release procedure](releasing.md) before distributing a tarball.
16
+
17
+ ## Run from source
18
+
19
+ ```sh
20
+ cp .env.example .env
21
+ # Set your Jev key, or select an installed local Ollama evaluator.
22
+ # Choose subscription or API-key authentication.
23
+ node --env-file=.env bin/autorouter.mjs doctor
24
+ node --env-file=.env bin/autorouter.mjs claude
25
+ ```
26
+
27
+ The explicit Node flag loads `.env`; the CLI itself does not auto-load project files. Environment values override the user config. Keep keys out of source control and command arguments.
28
+
29
+ ## Live integration tests
30
+
31
+ These tests make real Claude calls and invoke the configured evaluator, consuming Claude usage and, with Jev, TypeSafe usage. They use temporary synthetic fixtures and disable unrelated customizations and MCP servers. With Ollama, start the local service and install the chosen model first; the harness does not install or download it.
32
+
33
+ ```sh
34
+ npm run test:live
35
+ npm run test:live -- --case coding
36
+ npm run test:live -- --case example_haiku,example_sonnet,example_opus
37
+ npm run test:live -- --case large_context
38
+ node scripts/live-validation.mjs --help
39
+ ```
40
+
41
+ The harness checks response models, HTTP status, answers, tool use, continuation behavior, and independent tests for the coding fixture. The opt-in `large_context` case deliberately exceeds 200K input tokens. Reports contain metadata rather than request bodies or credentials; fixtures are removed afterward. Save any local reports outside the package allowlist.
42
+
43
+ For a classifier-only rubric evaluation:
44
+
45
+ ```sh
46
+ npm run eval
47
+ ```
48
+
49
+ The bundled evaluation makes 12 classifier calls and no Claude generations. Jev is the default and incurs TypeSafe usage; set `AUTOROUTER_EVALUATOR=ollama` to evaluate an installed local model. It reports agreement with the starting rubric, fallback count, and p50/p95 routing latency. Edit `test/fixtures/routing.json` to represent the tasks you want to measure. Rubric agreement alone does not establish answer quality or net savings; compare completed tasks against fixed-model baselines.
50
+
51
+ For local evaluator measurements, distinguish cold model loading from warmed classification, and record the model tag, hardware, Ollama version, context size, prompt length, and resident memory. The launcher primes the classifier rubric with a synthetic task before opening the UI, with a separate deadline of up to 60 seconds. Runtime and benchmark share the 3,000-character/3,000-UTF-8-byte state limit, so include non-ASCII cases and excerpts that fill the budget. Also measure the first request after keep-alive expiration: its reload can hit the normal deadline even when warm requests pass. Repeat on realistic prompt distributions instead of selecting a model from a single easy request. Disk download size is not resident RAM, and the larger preset is not a speed guarantee. Keep model downloads opt-in and respect each model's license.
52
+
53
+ The dedicated Ollama benchmark uses synthetic tuning/held-out fixtures and reports cold latency separately from repeated warm requests:
54
+
55
+ ```sh
56
+ npm run eval:ollama -- --models qwen3:1.7b,qwen3:4b --split heldout --rounds 3 --stress-rounds 8
57
+ node scripts/evaluate-ollama.mjs --help
58
+ ```
59
+
60
+ Install each selected model first and use an idle Ollama instance with no resident models. The benchmark loads one candidate at a time and unloads it afterward. It does not download models or contact Claude or Jev. It reports classification errors and under/over-routing as well as latency; fixture labels are subjective rubric judgments, not measurements of completed task quality.
61
+
62
+ On the 16 GiB M4 test Mac, compact matched 58.3% of held-out labels at warm p50/p95 latency of 602/834 ms and 1.70 GB model allocation. Quality (`qwen3:4b`) matched 91.7% at 889/1,242 ms and 3.18 GB. Each was tested on the same 24 distinct synthetic workloads repeated three times. Quality had no under-routing on this fixture, but routed two distinct Sonnet workloads to Opus on all repetitions. Both models exceeded the 1,500 ms deadline on all eight full-excerpt stress requests. No Jev comparison was run; local classification remains experimental. See the [local evaluator measurements](ollama-evaluation.md) for the hardware, candidate comparisons, and limits. A successful launcher/Enterprise integration request establishes connectivity and model routing, not classifier accuracy or parity with Jev.
63
+
64
+ ## Startup context diagnostics
65
+
66
+ The source-only probe launches Claude in a chosen repository and reports request sizes, tool counts, selected feature flags, and whether the entered prompt reaches the classifier excerpt. By default, a local stub answers and no request context goes to Jev or Anthropic:
67
+
68
+ ```sh
69
+ node scripts/context-probe.mjs --cwd /path/to/project --tool-search unset
70
+ node scripts/context-probe.mjs --cwd /path/to/project --tool-search true
71
+ node scripts/context-probe.mjs --cwd /path/to/project --tool-search true --example haiku
72
+ ```
73
+
74
+ Configured MCP servers still connect for discovery, and ordinary Claude startup customizations can run. The stub emits no tool calls. `--example` also accepts `sonnet` and `opus`.
75
+
76
+ On macOS, `--interactive` uses `/usr/bin/script` and requires a terminal on stdin. This measures the interactive tool set rather than print mode's tool set. The probe records terminal byte counts, not terminal contents, then stops after its response. Claude may save its normal transcript in interactive mode because its no-session-persistence option is limited to print mode.
77
+
78
+ `--classify` invokes the configured evaluator while Claude responses remain stubbed. With the default Jev backend, use it only when the selected repository's excerpts may be sent to TypeSafe; Ollama sends them to the local service. `--live` additionally sends the complete startup request to Anthropic, using `tool_choice: none` to prohibit model tool execution. Load configuration explicitly when using these modes:
79
+
80
+ ```sh
81
+ node --env-file=.env scripts/context-probe.mjs --cwd /path/to/synthetic-fixture --example haiku --classify
82
+ ```
83
+
84
+ Private repository or connected-tool context may be present even when the typed prompt is harmless. Keep private-payload investigations local unless external processing is authorized. For shareable live regressions, prefer the isolated synthetic fixtures above. The probe report itself persists only metadata.