pi-typesafe-router 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +46 -34
- package/docs/0001-route-before-generation.md +58 -0
- package/docs/0002-verify-routing-with-doctor.md +56 -0
- package/examples/cloudflare.json +1 -0
- package/examples/openrouter.json +40 -0
- package/package.json +18 -10
- package/src/classifier.ts +46 -15
- package/src/config.ts +8 -0
- package/src/index.ts +37 -7
- package/src/types.ts +17 -2
- package/docs/architecture.md +0 -59
- package/docs/configuration.md +0 -135
- package/docs/privacy.md +0 -35
- package/docs/releasing.md +0 -26
package/README.md
CHANGED
|
@@ -1,64 +1,56 @@
|
|
|
1
1
|
# pi-typesafe-router
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
Classification supports **TypeSafe direct, Cloudflare Workers AI, and Vercel AI Gateway**. Model fallback happens only before generation. A failed task is never automatically replayed by this extension.
|
|
3
|
+
Use Jev to classify requests and route them to the right model for the task.
|
|
6
4
|
|
|
7
5
|
## Setup
|
|
8
6
|
|
|
9
|
-
|
|
7
|
+
Do not enable this alongside another automatic router.
|
|
10
8
|
|
|
11
|
-
|
|
9
|
+
Install via:
|
|
12
10
|
|
|
13
|
-
```
|
|
14
|
-
npm
|
|
15
|
-
pi -e ./src/index.ts
|
|
11
|
+
```bash
|
|
12
|
+
`pi install npm:pi-typesafe-router`
|
|
16
13
|
```
|
|
17
14
|
|
|
18
|
-
|
|
15
|
+
Choose a real provider/model in Pi first, then run:
|
|
19
16
|
|
|
20
17
|
```text
|
|
21
18
|
/typesafe-router setup typesafe
|
|
22
19
|
```
|
|
23
20
|
|
|
24
|
-
This creates `~/.pi/agent/typesafe-router.json`, initially **off**, using your selected model for all three routes.
|
|
21
|
+
This creates `~/.pi/agent/typesafe-router.json`, initially **off**, using your selected model for all three routes. Edit the mappings, set `TYPESAFE_API_KEY` in the environment that launches Pi, and run `/reload`.
|
|
25
22
|
|
|
26
|
-
|
|
23
|
+
`pi install npm:pi-typesafe-router`.
|
|
27
24
|
|
|
28
25
|
## Usage
|
|
29
26
|
|
|
30
|
-
|
|
27
|
+
Verify your configuration, then enable routing:
|
|
31
28
|
|
|
32
29
|
```text
|
|
33
30
|
/typesafe-router doctor
|
|
34
31
|
/typesafe-router on
|
|
35
32
|
```
|
|
36
33
|
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
Doctor must complete successfully, including the classifier check, with at least one locally eligible, successfully probed target in **each** of `quick`, `standard`, and `deep`. Until then, `on`, `shadow`, and automatic input are blocked; `off` and manual Pi use remain available. Subsequent preflight skips failed targets, even the first configured target, and preserves the configured order of remaining successful candidates. A probe is a health snapshot, not a guarantee that a later task will succeed.
|
|
40
|
-
|
|
41
|
-
Verification is session-only and never persisted. Restart or reload requires doctor again. Changes to routes, credential references, backend configuration, or model metadata—even under the same provider/model ID—invalidate verification. Each doctor refresh discards old proofs; cancellation or incomplete checks never partially unlock routing.
|
|
42
|
-
|
|
43
|
-
`/typesafe-router` (or `status`) is read-only: it reports the applied configuration, runtime mode, current model, and on-disk differences without network calls. Reports include next steps only where relevant. Pi's built-in `/reload` loads updated extension code; it is not needed for configuration changes—use `/typesafe-router doctor` instead.
|
|
34
|
+
Doctor tests the classifier and configured models with synthetic requests, without sending your conversation, using tools, or changing the selected model. These checks may incur charges.
|
|
44
35
|
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
Escape, Ctrl+C, or `/typesafe-router off` cancels preflight and stops the original prompt. If Pi is already resolving generation credentials, cancellation waits for its non-cancellable model setter; verify the selected model before resubmitting. Prompts rejected or cancelled during preflight are not automatically queued or replayed.
|
|
48
|
-
|
|
49
|
-
After a failed generation, use Pi's `/model` to choose a model, inspect completed tool effects, and manually continue when safe. The router never selects a fallback after generation or replays a task. Pi's own retry and compaction settings remain unchanged.
|
|
36
|
+
Run `/typesafe-router help` for a table explaining every command.
|
|
50
37
|
|
|
51
38
|
## Configuration
|
|
52
39
|
|
|
53
|
-
|
|
40
|
+
Configuration examples are provided for [TypeSafe](examples/typesafe.json), [Cloudflare](examples/cloudflare.json), [Vercel](examples/vercel.json), and [OpenRouter](examples/openrouter.json).
|
|
54
41
|
|
|
55
|
-
|
|
56
|
-
| --------------------- | -------------------------------------------- | ----------------- |
|
|
57
|
-
| TypeSafe | `TYPESAFE_API_KEY` | `jev-1.13.0` |
|
|
58
|
-
| Cloudflare Workers AI | `CLOUDFLARE_API_TOKEN`, account ID in config | `typesafe/jev` |
|
|
59
|
-
| Vercel AI Gateway | `AI_GATEWAY_API_KEY` | `typesafe-ai/jev` |
|
|
42
|
+
To use one:
|
|
60
43
|
|
|
61
|
-
|
|
44
|
+
```text
|
|
45
|
+
/typesafe-router setup [typesafe|cloudflare|vercel|openrouter]
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
| Backend | Credential | Model |
|
|
49
|
+
| --------------------- | --------------------------------------------------------- | ------------------- |
|
|
50
|
+
| TypeSafe | `TYPESAFE_API_KEY` | `jev-1.13.0` |
|
|
51
|
+
| Cloudflare AI Gateway | `CLOUDFLARE_API_TOKEN`, account and gateway IDs in config | `typesafe/jev` |
|
|
52
|
+
| Vercel AI Gateway | `AI_GATEWAY_API_KEY` | `typesafe-ai/jev` |
|
|
53
|
+
| OpenRouter | `OPENROUTER_API_KEY` | `typesafe/jev-1.13` |
|
|
62
54
|
|
|
63
55
|
## Project structure
|
|
64
56
|
|
|
@@ -66,11 +58,31 @@ Backends are explicit; there is no cross-backend failover. Vercel uses the exper
|
|
|
66
58
|
- `src/config.ts`, `context.ts`, `routing.ts`: schema, bounded text projection, and deterministic policy.
|
|
67
59
|
- `src/index.ts`, `settings.ts`: Pi lifecycle, commands, cancellation, and global settings.
|
|
68
60
|
- `tests/`: synthetic transport, policy, lifecycle, and real Pi SDK tests. No paid API calls.
|
|
69
|
-
- `
|
|
61
|
+
- `infra/github/`: [Pulumi-managed repository settings](infra/github/README.md) and `main` ruleset.
|
|
62
|
+
- `docs/`: architecture decision records (ADRs), recording implemented decisions and their tradeoffs.
|
|
63
|
+
|
|
64
|
+
## Testing
|
|
65
|
+
|
|
66
|
+
Run `npm run check` for offline tests, lint, formatting, and typechecking.
|
|
67
|
+
|
|
68
|
+
`npm install` also installs the Lefthook pre-commit hook. Each commit formats and lints its
|
|
69
|
+
staged files, applying safe fixes before the commit is created. CI runs the same formatting and
|
|
70
|
+
lint checks as a required job in parallel with the test matrix.
|
|
70
71
|
|
|
71
|
-
|
|
72
|
+
E2E tests require the following variables to be set:
|
|
72
73
|
|
|
73
|
-
|
|
74
|
+
- `OPENROUTER_API_KEY`
|
|
75
|
+
- `TYPESAFE_API_KEY`
|
|
76
|
+
- `CLOUDFLARE_API_TOKEN`
|
|
77
|
+
- `CLOUDFLARE_ACCOUNT_ID`
|
|
78
|
+
- `CLOUDFLARE_GATEWAY_ID`
|
|
79
|
+
- `AI_GATEWAY_API_KEY`
|
|
80
|
+
|
|
81
|
+
Then:
|
|
82
|
+
|
|
83
|
+
```sh
|
|
84
|
+
TYPESAFE_ROUTER_LIVE_E2E=1 npm run test:e2e
|
|
85
|
+
```
|
|
74
86
|
|
|
75
87
|
## License
|
|
76
88
|
|
|
@@ -0,0 +1,58 @@
|
|
|
1
|
+
# ADR 0001: Classify tasks, select locally, and route only before generation
|
|
2
|
+
|
|
3
|
+
Status: Accepted — implemented against Pi 0.85.1
|
|
4
|
+
|
|
5
|
+
## Context
|
|
6
|
+
|
|
7
|
+
We want task-aware model selection without replacing Pi's providers, credentials, conversation handling, or tools. Jev can classify task demand, but its confidence does not measure whether a generation model will complete the task. Untrusted conversation text can also influence classification.
|
|
8
|
+
|
|
9
|
+
The routing boundary matters. In Pi 0.85.1, `before_agent_start` runs after initial model authentication and compaction. A virtual generation provider would hide the real selected model and require us to preserve downstream streaming, limits, identity, and recovery behavior. Switching after generation begins risks repeating tool side effects.
|
|
10
|
+
|
|
11
|
+
## Decision
|
|
12
|
+
|
|
13
|
+
We use Jev as an advisory classifier and keep selection policy local. We route once at an eligible idle `input` boundary, before generation authentication and compaction. We select real Pi models rather than registering a synthetic provider.
|
|
14
|
+
|
|
15
|
+
```text
|
|
16
|
+
idle input → session verification → bounded text classification
|
|
17
|
+
→ local route policy → ordered eligibility checks
|
|
18
|
+
→ serialized model selection → original input continues once
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
### Classification does not grant authority
|
|
22
|
+
|
|
23
|
+
- We use a fixed `quick` / `standard` / `deep` taxonomy plus `uncertain`. Configuration maps routes to flat, ordered lists of exact provider/model identities. This avoids inferred model names, route graphs, and implicit destinations.
|
|
24
|
+
- We validate labels, probability distributions, and confidence at runtime. Errors and timeouts use `defaultRoute`; uncertainty or missing/low confidence uses `uncertainRoute`. Both default to `deep`. Confidence is not a downstream success probability.
|
|
25
|
+
- We check current catalogue availability, model scope, image support, context/output budgets, and the generation proofs from [ADR 0002](0002-verify-routing-with-doctor.md). Failed selection advances only within the configured chain. An exhausted chain stops the prompt rather than silently using a stale model.
|
|
26
|
+
- We use Pi's context accounting and token estimates rather than treating bytes as tokens. Unknown post-compaction usage leaves compaction to Pi; we do not trim generation history to fit a cheaper target.
|
|
27
|
+
|
|
28
|
+
### Disclosure and transport are explicit
|
|
29
|
+
|
|
30
|
+
We read only global configuration under Pi's agent directory. Repository-local configuration could redirect private content without a deliberate global choice. Configuration stores credential references, not keys; setup creates an off-mode file without overwriting existing settings.
|
|
31
|
+
|
|
32
|
+
We send the intact current request and bounded recent user/assistant text, not system prompts, raw tool results, reasoning, images, or files. Oversized, empty, or unexpanded slash input takes the conservative route without classification. This limits disclosure and irrelevant context, but ordinary text can still contain secrets. Shadow mode also sends text and can incur charges.
|
|
33
|
+
|
|
34
|
+
We support one explicitly selected backend: TypeSafe direct, Cloudflare AI Gateway's account-scoped universal REST API with an explicit gateway ID, Vercel AI Gateway's experimental evaluation API, or OpenRouter's alpha Decisions API. Small HTTP adapters serve TypeSafe, Cloudflare, and OpenRouter; Vercel uses pinned AI SDK `7.0.105` because evaluation is not a chat-completions protocol. All adapters share the rubric and normalized validation boundary.
|
|
35
|
+
|
|
36
|
+
We do not retry classification, follow redirects, accept arbitrary classifier endpoints, or fail over between backends. Credentials and classification share a deadline. This bounds optional overhead and avoids silently changing recipients or billing. Pi-managed classifier credentials are opt-in API-key reuse, not proof of endpoint compatibility. Cloudflare requests disabled logging/cache and one attempt; Vercel requests zero data retention by default. Neither control establishes a universal retention guarantee.
|
|
37
|
+
|
|
38
|
+
### Pi owns generation; we do not replay tasks
|
|
39
|
+
|
|
40
|
+
We skip extension-generated input, steering, and queued follow-ups. The router keeps selection sticky through the tool loop rather than chasing per-step prices or sacrificing cache reuse. Manual model selection disables routing. Pi's native retry and compaction behavior remains unchanged; failure guidance waits for `agent_settled` and directs manual recovery through `/model`. There is no router recovery command that selects a post-generation fallback or resubmits work.
|
|
41
|
+
|
|
42
|
+
We serialize operations and invalidate stale work with epochs and abort signals. Cancellation stops the original submission; it does not authorize fallback generation. We return `handled` to stop input because Pi can catch an input-hook exception and continue.
|
|
43
|
+
|
|
44
|
+
Pi's asynchronous `setModel()` has no cancellation argument. We await the actual setter even after cancellation, keeping the selection lock held. Racing it against a timeout and selecting another candidate could let the abandoned setter overwrite the newer choice. Concurrent submissions are rejected; navigation is refused during preflight; teardown cancels and awaits completion. We do not roll back to a potentially stale model.
|
|
45
|
+
|
|
46
|
+
## Consequences and alternatives
|
|
47
|
+
|
|
48
|
+
- We retain Pi's real model identity and generation machinery instead of maintaining a proxy provider. We cannot offer transparent runtime outage failover or exclusive ownership against competing extensions. Do not enable competing routers.
|
|
49
|
+
- A cancelled setter can still change the model. A credential plugin that never resolves can block selection and reload; restarting Pi is the recovery. TUI Escape/Ctrl+C and `off` cancel router preflight; headless callers must use `off` because normal agent abort is not active yet.
|
|
50
|
+
- Context estimates are conservative, not exact candidate-specific tokenization. Later prompt expansion and other extensions can change the eventual request.
|
|
51
|
+
- We defer learned cost optimization, arbitrary classifier rubrics, automatic thinking-level routing, and per-tool-step rerouting. These need outcome evidence, not merely confident task labels.
|
|
52
|
+
- Synthetic tests establish local behavior, not live backend parity, savings, retention, or task quality. Vercel confidence metadata remains account-dependent evidence to verify; missing confidence stays conservative. OpenRouter's Decisions API is alpha and may change. Public performance claims require review of TypeSafe's terms.
|
|
53
|
+
|
|
54
|
+
## Implementation and verification
|
|
55
|
+
|
|
56
|
+
- [Configuration](../src/config.ts), [projection](../src/context.ts), [classifier adapters](../src/classifier.ts), and [pure routing policy](../src/routing.ts).
|
|
57
|
+
- [Lifecycle and cancellation](../src/index.ts); regressions in [core](../tests/core.test.ts), [classifier](../tests/classifier.test.ts), [runtime](../tests/runtime.test.ts), and [real Pi integration](../tests/pi-integration.test.ts) tests.
|
|
58
|
+
- Setup and configuration examples: [README](../README.md#configuration).
|
|
@@ -0,0 +1,56 @@
|
|
|
1
|
+
# ADR 0002: Gate routing on session-local doctor verification
|
|
2
|
+
|
|
3
|
+
Status: Accepted — implemented
|
|
4
|
+
|
|
5
|
+
## Context
|
|
6
|
+
|
|
7
|
+
A catalogue entry and configured credentials do not prove remote model access. They cannot establish permissions, quota, endpoint compatibility, or whether authentication succeeds. Enabling routing from these signals alone can send the first real task into a broken configuration.
|
|
8
|
+
|
|
9
|
+
Separate reload, validation, and network-check commands also make it easy to inspect one configuration while routing with another. We need a single explicit operation that applies configuration and tests access, without enabling routing or running the actual task.
|
|
10
|
+
|
|
11
|
+
## Decision
|
|
12
|
+
|
|
13
|
+
We consolidate configuration refresh, local eligibility checks, classifier testing, and generation probes into `/typesafe-router doctor`. We keep `status` and `help` read-only. The command surface is `setup`, `doctor`, `status`, `on`, `shadow`, `off`, and `help`; omitted subcommands show status. Help prints a command table without reading configuration, making network calls, or interrupting active work. Old check/validate/reload/recover/cancel commands are not part of the contract.
|
|
14
|
+
|
|
15
|
+
### Verification is required, not inferred
|
|
16
|
+
|
|
17
|
+
Every doctor refresh discards previous proofs before checking anything. Readiness requires a completed run with a successful classifier check and at least one locally eligible, successfully probed model in **each** route. We do not require every fallback target to pass: a broken primary should not defeat a working authorized alternative. Later preflight skips failed targets and preserves the configured order of successful candidates.
|
|
18
|
+
|
|
19
|
+
`on`, `shadow`, and automatic input require current verification. Shadow shares the gate so it evaluates a configuration we can actually use, even though it does not select a model. After the gate passes, shadow records its proposal without blocking generation for an empty proposed candidate chain. Off/manual Pi use remains available without doctor.
|
|
20
|
+
|
|
21
|
+
Doctor applies valid configuration while preserving the current session mode. It never enables an off session; invalid or missing configuration disables routing. Status reports applied configuration, disk differences, current activity/model/mode, and a historical last decision. It neither applies changes nor renews proofs. This separates observation from operations with network and billing effects.
|
|
22
|
+
|
|
23
|
+
### Probes exercise the actual generation path without the task
|
|
24
|
+
|
|
25
|
+
Doctor performs local checks, then concurrently runs one synthetic classifier request and one isolated generation probe per distinct configured provider/model. Targets shared across routes are probed once. Generation probes use Pi's `modelRegistry.complete`, not `setModel`, so they exercise configured providers, OAuth, headers, and endpoints without changing the selected model.
|
|
26
|
+
|
|
27
|
+
Probes contain no conversation transcript and use `tools: []`, `maxTokens: 128` where supported, and `maxRetries: 0`. Each generation probe has its own `generationProbeTimeoutMs` deadline, separate from classification. We validate response identity, stop reason, absence of tool calls, and nonempty text rather than trusting the model to declare its own health.
|
|
28
|
+
|
|
29
|
+
Explicit doctor invocation authorizes potentially billable diagnostics without a second confirmation, including while off or headless with `allowHeadless: false`. That flag still gates automatic routing. One command gives a complete access report rather than another optional step that can be mistaken for verification. The cost is real: token and time limits are not a strict monetary cap, and credentials may refresh.
|
|
30
|
+
|
|
31
|
+
### Proofs belong to the current session and configuration
|
|
32
|
+
|
|
33
|
+
We retain passed target identities and a verification fingerprint only in memory. The fingerprint covers configuration, resolved model metadata, provider object identities, auth status, and registered provider configuration, including classifier-only Pi credential providers. We never persist the fingerprint or credential material as verification evidence.
|
|
34
|
+
|
|
35
|
+
Restart/reload or an identity change requires a fresh doctor run. Current local eligibility is rechecked before enabling or routing, so changed scope or context can invalidate readiness. Provider refreshes can conservatively require another run even with unchanged provider/model IDs. The fingerprint does not detect every secret rotation; changing an environment variable's value alone does not change it. Persisted proofs would imply validity across changes we cannot reliably observe.
|
|
36
|
+
|
|
37
|
+
Doctor serializes with preflight and model selection. Cancellation aborts the batch; incomplete work never publishes partial readiness. Probes check cancellation again after authentication and before transport, so late credential resolution cannot start an abandoned request or publish a proof. A credential plugin can still finish authentication or refresh credentials after cancellation.
|
|
38
|
+
|
|
39
|
+
### Reports distinguish progress, access, and readiness
|
|
40
|
+
|
|
41
|
+
We count completed checks, including failures, while preserving configured route order in the final report. TUI progress uses one temporary widget above the editor, independent of routing mode and the footer, and clears on completion or cancellation. Other interfaces receive textual progress; headless diagnostics use stderr. This makes parallel work visible without a stream of persistent TUI notifications.
|
|
42
|
+
|
|
43
|
+
Reports show the applied configuration result, runtime mode, current model, credential source, local eligibility, classifier result, generation outcomes, and readiness. Next steps depend on the actual state. We report normalized reasons rather than raw provider errors or response bodies. Shutdown suppresses stale output.
|
|
44
|
+
|
|
45
|
+
## Consequences and alternatives
|
|
46
|
+
|
|
47
|
+
- We prefer active access checks over catalogue-only validation, but doctor is only a health snapshot. It does not guarantee later availability, quota, task quality, or classifier accuracy.
|
|
48
|
+
- Requiring fresh session proofs adds startup friction and charges, including for shadow mode. We accept this rather than silently treating old success or partial checks as readiness.
|
|
49
|
+
- Deduplication and concurrency reduce duplicate charges and elapsed time, but a large target set still creates a burst of requests. Per-model deadlines are not a total doctor deadline or spending limit.
|
|
50
|
+
- We keep diagnostics separate from task recovery. Doctor never selects a generation model, executes tools, or retries the original task. Generation failure remains manual recovery as described in [ADR 0001](0001-route-before-generation.md).
|
|
51
|
+
|
|
52
|
+
## Implementation and verification
|
|
53
|
+
|
|
54
|
+
- [Command lifecycle and readiness gate](../src/index.ts), [verification identity](../src/verification.ts), [isolated probes](../src/generation-probe.ts), and [report formatting](../src/diagnostics.ts).
|
|
55
|
+
- Regressions in [verification](../tests/verification.test.ts), [probe](../tests/generation-probe.test.ts), [diagnostics](../tests/diagnostics.test.ts), and [runtime](../tests/runtime.test.ts) tests.
|
|
56
|
+
- Run `npm run check` for offline checks and `npm run smoke:package` for packed-install loading. Neither replaces account-specific live compatibility checks of response shape, confidence extraction, permissions, billing controls, and cancellation. Run live checks only with explicit approval for charges.
|
package/examples/cloudflare.json
CHANGED
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
{
|
|
2
|
+
"version": 1,
|
|
3
|
+
"mode": "off",
|
|
4
|
+
"allowHeadless": false,
|
|
5
|
+
"backend": {
|
|
6
|
+
"type": "openrouter",
|
|
7
|
+
"model": "typesafe/jev-1.13",
|
|
8
|
+
"auth": {
|
|
9
|
+
"source": "env",
|
|
10
|
+
"variable": "OPENROUTER_API_KEY"
|
|
11
|
+
}
|
|
12
|
+
},
|
|
13
|
+
"timeoutMs": 1500,
|
|
14
|
+
"minConfidence": 0.8,
|
|
15
|
+
"maxContextChars": 12000,
|
|
16
|
+
"historyMessages": 4,
|
|
17
|
+
"outputReserveTokens": 8192,
|
|
18
|
+
"routes": {
|
|
19
|
+
"quick": [
|
|
20
|
+
{
|
|
21
|
+
"provider": "REPLACE_WITH_PI_PROVIDER",
|
|
22
|
+
"model": "REPLACE_WITH_QUICK_MODEL_ID"
|
|
23
|
+
}
|
|
24
|
+
],
|
|
25
|
+
"standard": [
|
|
26
|
+
{
|
|
27
|
+
"provider": "REPLACE_WITH_PI_PROVIDER",
|
|
28
|
+
"model": "REPLACE_WITH_STANDARD_MODEL_ID"
|
|
29
|
+
}
|
|
30
|
+
],
|
|
31
|
+
"deep": [
|
|
32
|
+
{
|
|
33
|
+
"provider": "REPLACE_WITH_PI_PROVIDER",
|
|
34
|
+
"model": "REPLACE_WITH_DEEP_MODEL_ID"
|
|
35
|
+
}
|
|
36
|
+
]
|
|
37
|
+
},
|
|
38
|
+
"defaultRoute": "deep",
|
|
39
|
+
"uncertainRoute": "deep"
|
|
40
|
+
}
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pi-typesafe-router",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"description": "Opt-in Jev task routing for Pi with TypeSafe, Cloudflare and
|
|
3
|
+
"version": "0.2.0",
|
|
4
|
+
"description": "Opt-in Jev task routing for Pi with TypeSafe, Cloudflare, Vercel, and OpenRouter backends and ordered preflight model fallbacks",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"jev",
|
|
7
7
|
"model-routing",
|
|
@@ -10,38 +10,46 @@
|
|
|
10
10
|
"typesafe"
|
|
11
11
|
],
|
|
12
12
|
"license": "MIT",
|
|
13
|
+
"repository": {
|
|
14
|
+
"type": "git",
|
|
15
|
+
"url": "git+https://github.com/jekozyra/pi-typesafe-router.git"
|
|
16
|
+
},
|
|
13
17
|
"files": [
|
|
14
18
|
"src",
|
|
15
19
|
"examples",
|
|
16
20
|
"README.md",
|
|
17
21
|
"LICENSE",
|
|
18
|
-
"docs/
|
|
19
|
-
"docs/
|
|
20
|
-
"docs/architecture.md",
|
|
21
|
-
"docs/releasing.md"
|
|
22
|
+
"docs/0001-route-before-generation.md",
|
|
23
|
+
"docs/0002-verify-routing-with-doctor.md"
|
|
22
24
|
],
|
|
23
25
|
"type": "module",
|
|
24
26
|
"scripts": {
|
|
25
27
|
"test": "tsx --test tests/*.test.ts",
|
|
28
|
+
"test:e2e": "tsx --test --test-concurrency=1 tests/e2e/*.test.ts",
|
|
26
29
|
"typecheck": "tsc --noEmit",
|
|
27
30
|
"fmt": "oxfmt --write .",
|
|
28
31
|
"fmt:check": "oxfmt --check .",
|
|
29
|
-
"lint": "oxlint --deny-warnings src tests scripts",
|
|
30
|
-
"lint:fix": "oxlint --fix --deny-warnings src tests scripts",
|
|
31
|
-
"check": "npm run fmt:check && npm run lint
|
|
32
|
+
"lint": "oxlint --deny-warnings src tests scripts infra/github",
|
|
33
|
+
"lint:fix": "oxlint --fix --deny-warnings src tests scripts infra/github",
|
|
34
|
+
"check:quality": "lefthook validate && npm run fmt:check && npm run lint",
|
|
35
|
+
"check": "npm run check:quality && npm run typecheck && npm test",
|
|
32
36
|
"smoke:package": "node scripts/package-smoke.mjs",
|
|
33
|
-
"prepack": "npm run typecheck"
|
|
37
|
+
"prepack": "npm run typecheck",
|
|
38
|
+
"changeset": "changeset",
|
|
39
|
+
"release:version": "changeset version && npm install --package-lock-only --ignore-scripts"
|
|
34
40
|
},
|
|
35
41
|
"dependencies": {
|
|
36
42
|
"ai": "7.0.105",
|
|
37
43
|
"zod": "^4.4.3"
|
|
38
44
|
},
|
|
39
45
|
"devDependencies": {
|
|
46
|
+
"@changesets/cli": "3.0.3",
|
|
40
47
|
"@earendil-works/pi-ai": "0.85.1",
|
|
41
48
|
"@earendil-works/pi-coding-agent": "0.85.1",
|
|
42
49
|
"@earendil-works/pi-tui": "0.85.1",
|
|
43
50
|
"@oxlint/plugins": "1.83.0",
|
|
44
51
|
"@types/node": "^22.0.0",
|
|
52
|
+
"lefthook": "2.1.14",
|
|
45
53
|
"oxfmt": "0.68.0",
|
|
46
54
|
"oxlint": "1.83.0",
|
|
47
55
|
"tsx": "^4.20.0",
|
package/src/classifier.ts
CHANGED
|
@@ -72,12 +72,25 @@ const gatewayResponseSchema = z.object({
|
|
|
72
72
|
.catch(undefined),
|
|
73
73
|
});
|
|
74
74
|
|
|
75
|
+
// Run markers must not fall through to legacy answers when a run is incomplete or malformed.
|
|
76
|
+
const cloudflareAnswerSchema = directResponseSchema.extend({
|
|
77
|
+
state: z.never().optional(),
|
|
78
|
+
result: z.never().optional(),
|
|
79
|
+
});
|
|
80
|
+
|
|
81
|
+
const cloudflareResultSchema = z.union([
|
|
82
|
+
z
|
|
83
|
+
.object({ state: z.literal("Completed"), result: directResponseSchema })
|
|
84
|
+
.transform((value) => value.result),
|
|
85
|
+
cloudflareAnswerSchema,
|
|
86
|
+
]);
|
|
87
|
+
|
|
75
88
|
// A malformed envelope must not fall through to the bare-result alternative.
|
|
76
89
|
const cloudflareResponseSchema = z.union([
|
|
77
90
|
z
|
|
78
|
-
.object({ success: z.literal(true), result:
|
|
91
|
+
.object({ success: z.literal(true), result: cloudflareResultSchema })
|
|
79
92
|
.transform((value) => value.result),
|
|
80
|
-
|
|
93
|
+
cloudflareAnswerSchema.extend({ success: z.never().optional() }),
|
|
81
94
|
]);
|
|
82
95
|
|
|
83
96
|
type Answer = z.output<typeof answerSchema>;
|
|
@@ -227,25 +240,43 @@ export function createClassifier(
|
|
|
227
240
|
);
|
|
228
241
|
}
|
|
229
242
|
|
|
230
|
-
const
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
|
|
243
|
+
const headers = new Headers({
|
|
244
|
+
Authorization: `Bearer ${apiKey}`,
|
|
245
|
+
"Content-Type": "application/json",
|
|
246
|
+
});
|
|
234
247
|
|
|
235
|
-
|
|
236
|
-
|
|
237
|
-
|
|
238
|
-
|
|
248
|
+
let url: string;
|
|
249
|
+
let body: string;
|
|
250
|
+
let schema: typeof directResponseSchema | typeof cloudflareResponseSchema;
|
|
251
|
+
|
|
252
|
+
switch (backend.type) {
|
|
253
|
+
case "typesafe":
|
|
254
|
+
url = "https://api.typesafe.ai/v1/systemone";
|
|
255
|
+
body = JSON.stringify({ model: backend.model, state, questions });
|
|
256
|
+
schema = directResponseSchema;
|
|
257
|
+
break;
|
|
258
|
+
case "openrouter":
|
|
259
|
+
url = "https://openrouter.ai/api/alpha/decisions";
|
|
260
|
+
body = JSON.stringify({ model: backend.model, state, questions });
|
|
261
|
+
schema = directResponseSchema;
|
|
262
|
+
break;
|
|
263
|
+
case "cloudflare":
|
|
264
|
+
url = `https://api.cloudflare.com/client/v4/accounts/${encodeURIComponent(backend.accountId)}/ai/run`;
|
|
265
|
+
body = JSON.stringify({ model: backend.model, input: { state, questions } });
|
|
266
|
+
schema = cloudflareResponseSchema;
|
|
267
|
+
headers.set("cf-aig-gateway-id", backend.gatewayId);
|
|
268
|
+
headers.set("cf-aig-collect-log", "false");
|
|
269
|
+
headers.set("cf-aig-skip-cache", "true");
|
|
270
|
+
headers.set("cf-aig-max-attempts", "1");
|
|
271
|
+
break;
|
|
272
|
+
}
|
|
239
273
|
|
|
240
274
|
const response = await guardedFetch(url, {
|
|
241
275
|
method: "POST",
|
|
242
|
-
headers
|
|
243
|
-
body
|
|
276
|
+
headers,
|
|
277
|
+
body,
|
|
244
278
|
});
|
|
245
279
|
|
|
246
|
-
const schema =
|
|
247
|
-
backend.type === "cloudflare" ? cloudflareResponseSchema : directResponseSchema;
|
|
248
|
-
|
|
249
280
|
const parsed = schema.parse(await response.json());
|
|
250
281
|
signal.throwIfAborted();
|
|
251
282
|
|
package/src/config.ts
CHANGED
|
@@ -45,6 +45,7 @@ const backend = z.discriminatedUnion("type", [
|
|
|
45
45
|
.string()
|
|
46
46
|
.length(32)
|
|
47
47
|
.regex(/^[a-fA-F0-9]{32}$/u),
|
|
48
|
+
gatewayId: identifier,
|
|
48
49
|
auth: auth.default(envAuth("CLOUDFLARE_API_TOKEN")),
|
|
49
50
|
})
|
|
50
51
|
.strict(),
|
|
@@ -56,6 +57,13 @@ const backend = z.discriminatedUnion("type", [
|
|
|
56
57
|
zeroDataRetention: z.boolean().default(true),
|
|
57
58
|
})
|
|
58
59
|
.strict(),
|
|
60
|
+
z
|
|
61
|
+
.object({
|
|
62
|
+
type: z.literal("openrouter"),
|
|
63
|
+
model: z.literal("typesafe/jev-1.13").default("typesafe/jev-1.13"),
|
|
64
|
+
auth: auth.default(envAuth("OPENROUTER_API_KEY")),
|
|
65
|
+
})
|
|
66
|
+
.strict(),
|
|
59
67
|
]);
|
|
60
68
|
|
|
61
69
|
const target = z.object({ provider, model: identifier }).strict();
|
package/src/index.ts
CHANGED
|
@@ -29,7 +29,9 @@ import {
|
|
|
29
29
|
import { classifierLines, routeLines, runtimeLines, type EvaluationResult } from "./diagnostics.ts";
|
|
30
30
|
import { abortable, createConfig, loadConfig } from "./settings.ts";
|
|
31
31
|
import {
|
|
32
|
+
BACKEND_TYPES,
|
|
32
33
|
ClassifierError,
|
|
34
|
+
isBackendType,
|
|
33
35
|
targetKey,
|
|
34
36
|
type Classification,
|
|
35
37
|
type Classify,
|
|
@@ -47,8 +49,21 @@ const sessionModeSchema = z.object({ mode: z.enum(["off", "auto", "shadow"]) });
|
|
|
47
49
|
const DISCLOSURE =
|
|
48
50
|
"Classification sends your request and bounded recent user/assistant text to the configured backend. Text can contain private code or secrets. Shadow mode also sends data and may incur charges. No automatic generation replay or classifier-backend failover.";
|
|
49
51
|
|
|
50
|
-
const
|
|
51
|
-
|
|
52
|
+
const BACKEND_HELP = BACKEND_TYPES.join("|");
|
|
53
|
+
|
|
54
|
+
const HELP = `/typesafe-router setup [${BACKEND_HELP}] | doctor | status | on | shadow | off | help`;
|
|
55
|
+
|
|
56
|
+
const COMMAND_HELP = `Usage: /typesafe-router <command>
|
|
57
|
+
|
|
58
|
+
Command Description
|
|
59
|
+
--------------- ------------------------------------
|
|
60
|
+
setup [${BACKEND_HELP}] Create a config interactively.
|
|
61
|
+
doctor Apply and validate config..
|
|
62
|
+
status Show current settings and activity.
|
|
63
|
+
on Enable automatic routing.
|
|
64
|
+
shadow Classify without switching models.
|
|
65
|
+
off Disable routing.
|
|
66
|
+
help Show this table.`;
|
|
52
67
|
|
|
53
68
|
interface Decision {
|
|
54
69
|
route: Route;
|
|
@@ -685,7 +700,7 @@ export function registerRouter(pi: RouterAPI, dependencies: Dependencies = {}):
|
|
|
685
700
|
"classifier check: skipped; no valid configuration",
|
|
686
701
|
invalidConfig
|
|
687
702
|
? "next: fix the configuration JSON, fields, or file permissions, then run /typesafe-router doctor"
|
|
688
|
-
:
|
|
703
|
+
: `next: /typesafe-router setup typesafe (or ${BACKEND_TYPES.slice(1).join("/")}), then /typesafe-router doctor`,
|
|
689
704
|
].join("\n"),
|
|
690
705
|
"error",
|
|
691
706
|
);
|
|
@@ -940,6 +955,12 @@ export function registerRouter(pi: RouterAPI, dependencies: Dependencies = {}):
|
|
|
940
955
|
return;
|
|
941
956
|
}
|
|
942
957
|
|
|
958
|
+
if (command === "help") {
|
|
959
|
+
notify(ctx, COMMAND_HELP);
|
|
960
|
+
|
|
961
|
+
return;
|
|
962
|
+
}
|
|
963
|
+
|
|
943
964
|
if (command === "off") {
|
|
944
965
|
configError = false;
|
|
945
966
|
setMode("off", ctx);
|
|
@@ -1002,12 +1023,11 @@ export function registerRouter(pi: RouterAPI, dependencies: Dependencies = {}):
|
|
|
1002
1023
|
}
|
|
1003
1024
|
|
|
1004
1025
|
const backend =
|
|
1005
|
-
option ??
|
|
1006
|
-
(await ctx.ui.select("Classification backend", ["typesafe", "cloudflare", "vercel"]));
|
|
1026
|
+
option ?? (await ctx.ui.select("Classification backend", [...BACKEND_TYPES]));
|
|
1007
1027
|
|
|
1008
1028
|
if (!backend || !permitted()) return;
|
|
1009
1029
|
|
|
1010
|
-
if (!
|
|
1030
|
+
if (!isBackendType(backend)) {
|
|
1011
1031
|
notify(ctx, HELP, "warning");
|
|
1012
1032
|
|
|
1013
1033
|
return;
|
|
@@ -1020,6 +1040,13 @@ export function registerRouter(pi: RouterAPI, dependencies: Dependencies = {}):
|
|
|
1020
1040
|
|
|
1021
1041
|
if (!permitted() || (backend === "cloudflare" && !accountId)) return;
|
|
1022
1042
|
|
|
1043
|
+
const gatewayId =
|
|
1044
|
+
backend === "cloudflare"
|
|
1045
|
+
? await ctx.ui.input("Cloudflare AI Gateway ID (gateway slug, not an API token)")
|
|
1046
|
+
: undefined;
|
|
1047
|
+
|
|
1048
|
+
if (!permitted() || (backend === "cloudflare" && !gatewayId)) return;
|
|
1049
|
+
|
|
1023
1050
|
const models = ctx.modelRegistry
|
|
1024
1051
|
.getAvailable()
|
|
1025
1052
|
.filter((model) => !["auto", "smart-router", "typesafe-router"].includes(model.provider));
|
|
@@ -1041,7 +1068,10 @@ export function registerRouter(pi: RouterAPI, dependencies: Dependencies = {}):
|
|
|
1041
1068
|
|
|
1042
1069
|
const initial = parseConfig({
|
|
1043
1070
|
version: 1,
|
|
1044
|
-
backend:
|
|
1071
|
+
backend:
|
|
1072
|
+
backend === "cloudflare"
|
|
1073
|
+
? { type: backend, accountId, gatewayId }
|
|
1074
|
+
: { type: backend },
|
|
1045
1075
|
routes: { quick: [target], standard: [target], deep: [target] },
|
|
1046
1076
|
});
|
|
1047
1077
|
|
package/src/types.ts
CHANGED
|
@@ -10,15 +10,30 @@ export type CredentialSource =
|
|
|
10
10
|
| { source: "env"; variable: string }
|
|
11
11
|
| { source: "pi"; provider: string };
|
|
12
12
|
|
|
13
|
+
export const BACKEND_TYPES = ["typesafe", "cloudflare", "vercel", "openrouter"] as const;
|
|
14
|
+
|
|
15
|
+
export type BackendType = (typeof BACKEND_TYPES)[number];
|
|
16
|
+
|
|
17
|
+
export function isBackendType(value: string): value is BackendType {
|
|
18
|
+
return BACKEND_TYPES.some((backend) => backend === value);
|
|
19
|
+
}
|
|
20
|
+
|
|
13
21
|
export type Backend =
|
|
14
22
|
| { type: "typesafe"; model: string; auth: CredentialSource }
|
|
15
|
-
| {
|
|
23
|
+
| {
|
|
24
|
+
type: "cloudflare";
|
|
25
|
+
model: "typesafe/jev";
|
|
26
|
+
accountId: string;
|
|
27
|
+
gatewayId: string;
|
|
28
|
+
auth: CredentialSource;
|
|
29
|
+
}
|
|
16
30
|
| {
|
|
17
31
|
type: "vercel";
|
|
18
32
|
model: "typesafe-ai/jev";
|
|
19
33
|
auth: CredentialSource;
|
|
20
34
|
zeroDataRetention: boolean;
|
|
21
|
-
}
|
|
35
|
+
}
|
|
36
|
+
| { type: "openrouter"; model: "typesafe/jev-1.13"; auth: CredentialSource };
|
|
22
37
|
|
|
23
38
|
export interface Target {
|
|
24
39
|
provider: string;
|
package/docs/architecture.md
DELETED
|
@@ -1,59 +0,0 @@
|
|
|
1
|
-
# Routing before generation
|
|
2
|
-
|
|
3
|
-
Jev classifies task demand. A local policy chooses eligible generation targets. Neither classifier output nor task text can supply a provider URL, invent a model target, or bypass the configured chains.
|
|
4
|
-
|
|
5
|
-
```text
|
|
6
|
-
idle user input
|
|
7
|
-
→ current session doctor verification gate
|
|
8
|
-
→ bounded text projection
|
|
9
|
-
→ one explicitly selected classifier adapter
|
|
10
|
-
→ validated quick / standard / deep / uncertain answer
|
|
11
|
-
→ local confidence policy
|
|
12
|
-
→ ordered candidate eligibility checks
|
|
13
|
-
→ serialized Pi model selection
|
|
14
|
-
→ original input continues once
|
|
15
|
-
```
|
|
16
|
-
|
|
17
|
-
`classify` normalizes three protocols. `chooseRoute` and `candidateChecks` are pure. `registerRouter` owns lifecycle and session-local state. This is not an LLM proxy and does not register a synthetic generation model.
|
|
18
|
-
|
|
19
|
-
## Lifecycle contract
|
|
20
|
-
|
|
21
|
-
The implementation targets Pi **0.85.1**. Its idle `input` hook runs before initial generation authentication and compaction. `before_agent_start` is too late for a router that must rescue a missing initial model/auth configuration. Selection happens once at the idle input boundary; streaming steering/follow-up and extension-generated input pass through without classification. Tool-loop selection is sticky by router policy, not by an immutable host guarantee.
|
|
22
|
-
|
|
23
|
-
The extension returns `handled` when preflight must stop a prompt. Throwing would be unsafe: Pi catches input-hook exceptions and can continue generation. No path uses `sendUserMessage`, resends the original input, or chooses a new model after a generation error. Native Pi retry/overflow recovery is not disabled or replaced. Failure guidance waits for `agent_settled`, after native recovery finishes.
|
|
24
|
-
|
|
25
|
-
Model selection requires a current successful doctor generation proof, then checks exact catalogue identity, configured auth presence, nonempty scoped-model restrictions, image support, and conservative context/output budgets. A setter returning false or throwing skips to the next candidate before generation. Failed doctor targets are skipped, even when first in a chain; remaining successful candidates keep their configured order. Catalogue eligibility alone does not prove remote credentials, quotas, availability, or task quality. A successful probe is only a health snapshot, not a guarantee of future generation. Context sizing uses Pi's `ctx.getContextUsage()` count, which incorporates provider-reported usage and Pi's trailing-message estimates. Unsent input uses Pi's exported `estimateTokens`; the router does not count serialized metadata or treat bytes as tokens. If Pi reports unknown usage immediately after compaction, the router skips only its context-size check and leaves compaction/accounting to Pi. If no usage API result is available, it sums Pi's message estimates. Counts are not exact candidate-specific tokenization; Pi's current accounting is the shared basis. Unknown post-input changes from templates, context hooks, tools, or other extensions cannot be predicted.
|
|
26
|
-
|
|
27
|
-
## Cancellation and concurrency
|
|
28
|
-
|
|
29
|
-
Each operation has an epoch and an AbortController. Classifier credentials and HTTP share a deadline. Late classification/credential results are ignored. Escape/Ctrl+C are intercepted during TUI preflight, and `/typesafe-router off` also invalidates the operation. Headless callers must use `off` during preflight; Pi's normal agent abort signal is not yet active there.
|
|
30
|
-
|
|
31
|
-
Pi's `setModel()` is asynchronous and has **no cancellation argument**. A timeout race cannot prevent it from later changing the model. The router therefore awaits the actual setter, never starts another selection in parallel, and consumes the cancelled original input afterward. Concurrent input is rejected, even if routing was turned off while authentication was pending. Model selection may finish after cancellation; users must verify it before resubmitting. There is no unsafe rollback to a stale model.
|
|
32
|
-
|
|
33
|
-
Session navigation is refused while preflight is active, cancelling the operation first. Shutdown/reload cancels and awaits its completion before allowing teardown. If a credential plugin never resolves its model-auth promise, selection and reload can remain blocked; restarting Pi is the recovery. This is an explicit limitation of the host setter, not something a timer can safely fix.
|
|
34
|
-
|
|
35
|
-
## Failure and recovery
|
|
36
|
-
|
|
37
|
-
Classification errors, timeouts, invalid responses, or missing keys use `defaultRoute`. Low/missing confidence and explicit uncertainty use `uncertainRoute`. Oversized, empty, and unexpanded slash input skip classification and use the conservative route. No eligible candidate consumes the prompt with an error instead of silently using a stale model. After the doctor gate passes, shadow mode records the proposed route but neither changes the model nor blocks generation for an empty candidate chain. Both on and shadow modes, including automatic input, are blocked without current doctor verification; off/manual Pi use remains available.
|
|
38
|
-
|
|
39
|
-
After generation fails, guidance points to Pi's `/model`, inspection of completed tool effects, and manual continuation when safe. The router does not select a post-generation fallback or replay a task. Manual model selection disables routing. Do not run multiple routers: Pi does not provide exclusive model-selection ownership between extensions.
|
|
40
|
-
|
|
41
|
-
## Diagnostics and configuration
|
|
42
|
-
|
|
43
|
-
`/typesafe-router doctor` serializes with preflight and model selection. It applies valid configuration while retaining the current session's on/off/shadow mode; it never enables an off session. Missing or invalid configuration disables routing. Every refresh invalidates old proofs before checking anything. Readiness requires a completed, successful doctor run: the classifier check must succeed and every quick/standard/deep route must contain at least one locally eligible target with a successful generation probe. Cancellation or incomplete work cannot publish partial readiness.
|
|
44
|
-
|
|
45
|
-
Doctor checks local eligibility, then runs the synthetic classifier request and one isolated synthetic generation request per distinct configured provider/model concurrently, deduplicated across routes. A shared progress counter counts completed checks (including failures), while the final report preserves configured route order. TUI progress updates one widget above the editor, independent of the footer and routing mode, and clears on completion, failure, or cancellation. Other interfaces receive textual progress notifications. Cancelling doctor aborts the whole batch; partial results cannot grant verification. Generation probes use Pi's `modelRegistry.complete` with the actual credential providers, including OAuth, custom headers, and configured endpoints. Credentials may refresh. Probes contain no actual conversation transcript and use `tools: []`, `maxTokens: 128` where supported, and `maxRetries: 0`. `generationProbeTimeoutMs` bounds each model probe (default 15000 ms), separately from the classifier's `timeoutMs`. These limits are not a strict monetary cap. Explicit invocation authorizes potentially billable diagnostics even while off or headless with `allowHeadless: false`. Doctor never calls the generation model setter, changes the selected model, or runs or replays the original task.
|
|
46
|
-
|
|
47
|
-
Generation proofs and readiness live only in memory, not session records or config. Restart/reload requires doctor again. The verification identity covers routes, credential references, backend configuration, and model metadata, so changes under the same provider/model IDs still invalidate it. Native auth status and provider references contribute to an in-memory hash; no secrets are persisted. A changed identity requires a fresh successful doctor run, not reuse of old proofs. Effective Pi provider object identity covers native registrations and applied `models.json` changes; a provider-registry refresh can conservatively require doctor again. Classifier-only Pi credential providers are included. Before enabling or routing, every route must still have an eligible verified candidate, including current scope and context limits.
|
|
48
|
-
|
|
49
|
-
Probe cancellation releases doctor promptly and checks the abort signal again after Pi resolves authentication, before invoking provider transport. A credential plugin that ignores cancellation may still finish authentication or refresh credentials later; its late result cannot start the probe transport or publish verification.
|
|
50
|
-
|
|
51
|
-
Reports reflect runtime, config path/application result, actual mode, current model, backend/model, credential source, local eligibility, classifier result/latency, generation probe outcomes, and readiness, with conditional next steps. Cancellation and incomplete work are explicit; shutdown suppresses stale output. Off/Escape cancels pending work but cannot release a pending model setter's lock early.
|
|
52
|
-
|
|
53
|
-
`status`, also the default subcommand, is read-only and makes no network calls. It reports applied configuration and on-disk differences (or inability to check), current activity/model/mode, and a historical last decision. It does not apply configuration or renew verification. Classifier success alone does not prove generation-provider health; successful generation probes do not guarantee later availability, quota, or task quality.
|
|
54
|
-
|
|
55
|
-
## Verification boundaries
|
|
56
|
-
|
|
57
|
-
Offline tests validate schema/policy, request serialization, normalized responses, cancellation, no replay, and real SDK extension loading. Packed-install smoke tests exercise Pi's actual extension loader using an isolated profile and installed production dependencies. Synthetic transports do not establish live service contracts or quality. In particular, Gateway confidence metadata needs a live account check; absent confidence stays conservative.
|
|
58
|
-
|
|
59
|
-
Before claiming savings or quality, evaluate privately on representative coding tasks with pinned versions and per-model success/cost/latency outcomes. Classifier confidence measures concentration, not downstream success probability. TypeSafe's public benchmarking restrictions require permission/clarification before publishing results.
|
package/docs/configuration.md
DELETED
|
@@ -1,135 +0,0 @@
|
|
|
1
|
-
# Configure task routing
|
|
2
|
-
|
|
3
|
-
The router reads only `typesafe-router.json` under Pi's agent directory (normally `~/.pi/agent`). `PI_CODING_AGENT_DIR` changes that directory. It does not read project files or discover credentials in a repository.
|
|
4
|
-
|
|
5
|
-
## Map classes to model chains
|
|
6
|
-
|
|
7
|
-
Copy an [example](../examples/typesafe.json), or run `/typesafe-router setup typesafe` in Pi. Replace every example target with exact IDs from Pi's model catalogue. A provider ID cannot contain a slash; a model ID can.
|
|
8
|
-
|
|
9
|
-
```json
|
|
10
|
-
{
|
|
11
|
-
"version": 1,
|
|
12
|
-
"mode": "off",
|
|
13
|
-
"allowHeadless": false,
|
|
14
|
-
"backend": {
|
|
15
|
-
"type": "typesafe",
|
|
16
|
-
"model": "jev-1.13.0",
|
|
17
|
-
"auth": { "source": "env", "variable": "TYPESAFE_API_KEY" }
|
|
18
|
-
},
|
|
19
|
-
"timeoutMs": 1500,
|
|
20
|
-
"generationProbeTimeoutMs": 15000,
|
|
21
|
-
"minConfidence": 0.8,
|
|
22
|
-
"maxContextChars": 12000,
|
|
23
|
-
"historyMessages": 4,
|
|
24
|
-
"outputReserveTokens": 8192,
|
|
25
|
-
"defaultRoute": "deep",
|
|
26
|
-
"uncertainRoute": "deep",
|
|
27
|
-
"routes": {
|
|
28
|
-
"quick": [{ "provider": "your-provider", "model": "your-fast-model" }],
|
|
29
|
-
"standard": [{ "provider": "your-provider", "model": "your-balanced-model" }],
|
|
30
|
-
"deep": [
|
|
31
|
-
{ "provider": "your-provider", "model": "your-strong-model" },
|
|
32
|
-
{ "provider": "another-provider", "model": "your-backup-model" }
|
|
33
|
-
]
|
|
34
|
-
}
|
|
35
|
-
}
|
|
36
|
-
```
|
|
37
|
-
|
|
38
|
-
Each chain contains one to eight distinct targets. Order is authoritative: targets without a current successful doctor generation proof, or failing local eligibility, are skipped; selection stops at the first successful `pi.setModel`. A failed first target does not prevent selection of a later successful one. Remaining successful candidates keep their configured order. Doctor deduplicates probes across chains by exact provider/model identity. There is no random sampling, cost lookup, or per-task remote health probe. `auto`, `smart-router`, and `typesafe-router` virtual providers are rejected.
|
|
39
|
-
|
|
40
|
-
Unknown fields, duplicate targets, invalid types, and out-of-range settings reject the entire file. Missing, unreadable, or invalid configuration disables routing. Repair the file and run `/typesafe-router doctor` to apply it; explicitly enable routing afterward if desired.
|
|
41
|
-
|
|
42
|
-
| Setting | Default | Accepted values |
|
|
43
|
-
| -------------------------- | ------- | -------------------------------------------------------------------------------- |
|
|
44
|
-
| `version` | `1` | `1` |
|
|
45
|
-
| `mode` | `off` | `off`, `shadow`, `auto` |
|
|
46
|
-
| `allowHeadless` | `false` | Boolean; explicitly permit automatic routing outside TUI |
|
|
47
|
-
| `timeoutMs` | `1500` | Integer, 100–30000; credentials plus classification |
|
|
48
|
-
| `generationProbeTimeoutMs` | `15000` | Integer, 100–60000; timeout for each model’s doctor generation probe |
|
|
49
|
-
| `minConfidence` | `0.8` | Number, 0–1 |
|
|
50
|
-
| `maxContextChars` | `12000` | Integer, 256–32000; combined request and history text |
|
|
51
|
-
| `historyMessages` | `4` | Integer, 0–20 |
|
|
52
|
-
| `outputReserveTokens` | `8192` | Integer, 256–131072; capped at candidate's maximum output |
|
|
53
|
-
| `defaultRoute` | `deep` | `quick`, `standard`, `deep`; classifier failure/unavailability |
|
|
54
|
-
| `uncertainRoute` | `deep` | Same values; uncertainty, absent/low confidence, oversized or unexpanded request |
|
|
55
|
-
|
|
56
|
-
`generationProbeTimeoutMs` is a top-level setting applied separately to each distinct model probe, not a total doctor deadline. Probes request `maxTokens: 128` where supported and `maxRetries: 0`; these are fixed probe options, not configuration fields. They do not establish a strict monetary cap.
|
|
57
|
-
|
|
58
|
-
Character limits use JavaScript string length. The current request is never truncated. If it cannot fit, classification is skipped and the conservative route is used. Only contiguous newest fitting conversation text is included. Set `historyMessages: 0` for request-only classification.
|
|
59
|
-
|
|
60
|
-
## Select one classification backend
|
|
61
|
-
|
|
62
|
-
### TypeSafe direct
|
|
63
|
-
|
|
64
|
-
```json
|
|
65
|
-
{
|
|
66
|
-
"type": "typesafe",
|
|
67
|
-
"model": "jev-1.13.0",
|
|
68
|
-
"auth": { "source": "env", "variable": "TYPESAFE_API_KEY" }
|
|
69
|
-
}
|
|
70
|
-
```
|
|
71
|
-
|
|
72
|
-
Requests go to `https://api.typesafe.ai/v1/systemone`. The versioned model ID is the default; explicitly configured aliases may change behavior over time. No automatic retries occur.
|
|
73
|
-
|
|
74
|
-
### Cloudflare Workers AI
|
|
75
|
-
|
|
76
|
-
```json
|
|
77
|
-
{
|
|
78
|
-
"type": "cloudflare",
|
|
79
|
-
"accountId": "0123456789abcdef0123456789abcdef",
|
|
80
|
-
"model": "typesafe/jev",
|
|
81
|
-
"auth": { "source": "env", "variable": "CLOUDFLARE_API_TOKEN" }
|
|
82
|
-
}
|
|
83
|
-
```
|
|
84
|
-
|
|
85
|
-
Use your own 32-hex-character account ID and a token authorized for Workers AI. The fixed account-scoped REST endpoint is `https://api.cloudflare.com/client/v4/accounts/{accountId}/ai/run`. This is Workers AI evaluation, not arbitrary Cloudflare AI Gateway chat compatibility.
|
|
86
|
-
|
|
87
|
-
### Vercel AI Gateway
|
|
88
|
-
|
|
89
|
-
```json
|
|
90
|
-
{
|
|
91
|
-
"type": "vercel",
|
|
92
|
-
"model": "typesafe-ai/jev",
|
|
93
|
-
"zeroDataRetention": true,
|
|
94
|
-
"auth": { "source": "env", "variable": "AI_GATEWAY_API_KEY" }
|
|
95
|
-
}
|
|
96
|
-
```
|
|
97
|
-
|
|
98
|
-
This uses AI SDK `experimental_evaluate`, pinned to SDK `7.0.105`, and the Gateway evaluation model interface. No OpenAI-compatible chat endpoint is involved. `zeroDataRetention` defaults to true; the provider may reject unsupported retention options instead of silently weakening them.
|
|
99
|
-
|
|
100
|
-
Confidence is read from the per-question `providerMetadata.typesafe.confidence.task_class` map. Public documentation identifies the enclosing confidence metadata but does not demonstrate its full shape. Missing confidence remains missing and selects `uncertainRoute`; malformed confidence fails validation and selects `defaultRoute`. Run `/typesafe-router doctor` with your account before relying on this experimental integration. No live service parity is claimed.
|
|
101
|
-
|
|
102
|
-
### Optional Pi-managed credentials
|
|
103
|
-
|
|
104
|
-
All backends also accept `{"source":"pi","provider":"your-provider-id"}` as `auth`. This calls Pi 0.85.1's `getProviderAuth` at request time. It is opt-in, never an implicit fallback from a missing environment key. Only the resolved API key is forwarded; custom auth headers are not. A provider credential does not establish that the target evaluation endpoint accepts it. Verify audience, permissions, and backend compatibility yourself. Environment credentials are the recommended initial setup.
|
|
105
|
-
|
|
106
|
-
## Commands
|
|
107
|
-
|
|
108
|
-
| Command | Effect |
|
|
109
|
-
| -------------------------------------- | ---------------------------------------------------------------------------------------------------------- |
|
|
110
|
-
| `setup [typesafe\|cloudflare\|vercel]` | Interactive creation of an off-mode config; never overwrites |
|
|
111
|
-
| `doctor` | Apply config, test the classifier and distinct generation targets, and refresh session verification |
|
|
112
|
-
| `status` (default) | Read-only report of applied config, runtime mode, current model/activity, and disk differences; no network |
|
|
113
|
-
| `on` / `shadow` / `off` | Change session mode; on/shadow require successful doctor; off cancels preflight |
|
|
114
|
-
|
|
115
|
-
Prefix each with `/typesafe-router`. With no subcommand, the command shows status.
|
|
116
|
-
|
|
117
|
-
`doctor` reloads valid configuration while preserving the current session's on/off/shadow mode, rather than adopting the file's mode. It never enables an off session. Missing or invalid configuration disables routing. It never changes the selected generation model or runs or replays a user task.
|
|
118
|
-
|
|
119
|
-
Doctor automatically sends one synthetic classifier request and one isolated synthetic generation request per distinct configured provider/model, without confirmation. No actual conversation transcript is sent, and generation probes have `tools: []`. **These requests may incur charges**; there is no strict monetary cap. Explicit invocation authorizes them even with routing off or in print/RPC mode with `allowHeadless: false`. Automatic routing in those modes still requires `allowHeadless: true` and an enabled mode. Headless diagnostics go to stderr, not the assistant's output.
|
|
120
|
-
|
|
121
|
-
Generation probes run through Pi's `modelRegistry.complete`, using the actual configured credential providers, including OAuth, custom headers, and custom endpoints. Credentials may refresh during diagnostics. This differs from the classifier's optional API-key-only Pi credential reuse described above.
|
|
122
|
-
|
|
123
|
-
Readiness requires doctor to complete successfully, including the classifier check, and at least one locally eligible, successfully probed target in each of `quick`, `standard`, and `deep`. Not every target must pass. Failed targets are skipped in subsequent preflight, preserving the configured order of remaining successful candidates. Until ready, `on`, `shadow`, and automatic input are blocked; `off` and manual Pi use remain available. A configured startup mode does not bypass verification.
|
|
124
|
-
|
|
125
|
-
Verification is session-only and never persisted. Restart/reload requires doctor again. Changes to routes, credential references, backend configuration, or model metadata invalidate it even if provider/model IDs stay the same. Native auth status and provider references contribute to an in-memory identity hash; no secrets are persisted. Each doctor refresh invalidates previous proofs. Cancellation or incomplete checks never partially unlock routing; run doctor again to recover.
|
|
126
|
-
|
|
127
|
-
Doctor reports runtime, config path/application result, actual session mode, current generation model, backend/model, credential source, local eligibility, classifier result/latency, generation probe outcomes, and readiness. Reports give conditional next steps for the actual state. Local eligibility alone does not prove generation-provider health; a successful probe is a health snapshot, not a guarantee of later availability, quota, or task quality.
|
|
128
|
-
|
|
129
|
-
Status reports the applied configuration, not merely the file on disk. It identifies disk differences or inability to check the file, and labels the last routing decision as historical. It neither applies changes nor tests services. Use doctor to apply and test changes.
|
|
130
|
-
|
|
131
|
-
Escape/Ctrl+C in the TUI or `off` cancels preflight. While Pi's non-cancellable model setter is pending, the selection lock remains held; verify the selected model before resubmitting. Doctor is serialized with preflight/model selection.
|
|
132
|
-
|
|
133
|
-
After generation fails, use `/model`, inspect completed tool effects, and manually continue when safe. The router performs no post-generation fallback and never replays a task.
|
|
134
|
-
|
|
135
|
-
Manual `/model` selection turns routing off. Ordinary tool loops and queued steering/follow-up messages do not reroute. Session-tree navigation turns routing off. Active preflight blocks concurrent submissions and session navigation; after cancellation settles, repeat the desired submission/navigation yourself.
|
package/docs/privacy.md
DELETED
|
@@ -1,35 +0,0 @@
|
|
|
1
|
-
# What leaves your machine
|
|
2
|
-
|
|
3
|
-
Enabling `auto` or `shadow` sends the current request and bounded recent user/assistant text to the configured classification backend. That text can contain private code, credentials, or personal data. **This extension is not a secret scanner or redaction system.** Do not enable it for material you cannot share with that backend.
|
|
4
|
-
|
|
5
|
-
The default projection excludes system prompts, raw tool calls/results, reasoning blocks, image bytes, attachments, compaction summaries, and arbitrary files. If an assistant quotes a tool result or secret in ordinary text, that text can still be included. Slash-command/template input is not classified before expansion; the conservative route is used instead. There is no background classification while the router is off.
|
|
6
|
-
|
|
7
|
-
## Explicit diagnostics
|
|
8
|
-
|
|
9
|
-
`/typesafe-router doctor` applies valid configuration, checks local eligibility, and automatically sends one synthetic classifier request plus one isolated synthetic generation request to each distinct configured provider/model. It sends no actual conversation transcript, uses no tools (`tools: []`), asks for no confirmation, and **may incur charges**. Generation probes request `maxTokens: 128` where supported and `maxRetries: 0`, with a per-model `generationProbeTimeoutMs` deadline (default 15000 ms). These bounds are not a strict monetary cap.
|
|
10
|
-
|
|
11
|
-
Invoking doctor explicitly authorizes these requests even while routing is off or headless with `allowHeadless: false`; that setting still gates automatic routing. Doctor preserves the current session mode and never enables an off session. Missing or invalid configuration disables routing. Doctor never changes the selected model or runs or replays a user task. Isolated synthetic generation probes are not retries of the actual task.
|
|
12
|
-
|
|
13
|
-
Doctor must complete successfully, including the classifier check, and establish a locally eligible, successful generation candidate for each route before `on`, `shadow`, or automatic input is allowed. Off/manual Pi use remains available. Verification is session-only: restart/reload and identity changes require doctor again. Refresh discards old proofs; cancellation never partially unlocks routing.
|
|
14
|
-
|
|
15
|
-
`/typesafe-router status` (also the default) is read-only and makes no network calls. It reports applied configuration, runtime mode/current model, and on-disk differences. Local eligibility is not remote generation verification, and successful probes are only health snapshots, not future guarantees.
|
|
16
|
-
|
|
17
|
-
## Recipients and retention
|
|
18
|
-
|
|
19
|
-
- Direct: TypeSafe's API.
|
|
20
|
-
- Cloudflare: your account's Workers AI endpoint and its upstream service arrangement.
|
|
21
|
-
- Vercel: AI Gateway and its upstream service arrangement. The request explicitly enables `zeroDataRetention` by default.
|
|
22
|
-
|
|
23
|
-
The router never silently changes classification backends, follows HTTP redirects, or retries classification. A failed classifier uses a local configured generation route instead. Generation content goes to the model provider selected in Pi, under that provider's normal settings. Doctor also sends synthetic probe content to every distinct configured generation target through Pi's `modelRegistry.complete`, including custom endpoints and their normal provider arrangements.
|
|
24
|
-
|
|
25
|
-
No-training promises do not establish zero data retention. TypeSafe's enterprise ZDR offering, gateway logging, account controls, and contract terms need separate review. The extension cannot verify a service's retention behavior. Review [TypeSafe's terms](https://typesafe.ai/legal/mca), [Cloudflare's model documentation](https://developers.cloudflare.com/ai/models/typesafe/jev/), and [Vercel's evaluation documentation](https://vercel.com/docs/ai-gateway/modalities/evaluation) before sharing sensitive content.
|
|
26
|
-
|
|
27
|
-
## Local data and credentials
|
|
28
|
-
|
|
29
|
-
Configuration is global and contains credential references, not keys. Setup creates it with mode `0600` where supported. Existing files and symlinks are never overwritten. Credentials are resolved at request time and are not placed in session records. For classification, Pi-managed credential reuse is explicit and forwards only an API key, not provider-specific headers. Generation probes instead use Pi's actual credential providers, including OAuth and custom headers. Credentials may refresh during doctor, subject to Pi's normal credential storage behavior.
|
|
30
|
-
|
|
31
|
-
Generation proofs and readiness are held only in memory; the router never persists them. The verification identity covers routes, credential references, backend configuration, and model metadata, including changes under unchanged provider/model IDs. Native auth status and provider references contribute to an in-memory hash. No secrets are persisted by the router for verification.
|
|
32
|
-
|
|
33
|
-
Session custom entries contain route/model IDs, classifier labels/probabilities/confidence, normalized usage, elapsed time, safe failure reasons, and skipped candidates. They do not contain request text, raw responses, HTTP error bodies, or API keys. Pi itself already records ordinary conversation content independently of this extension. There is no extension telemetry or external analytics.
|
|
34
|
-
|
|
35
|
-
Diagnostics report configuration errors without printing Zod values or filesystem error bodies. Model IDs and config paths are shown locally; do not use secrets as identifiers. Gateway/provider warnings are suppressed rather than printed verbatim.
|
package/docs/releasing.md
DELETED
|
@@ -1,26 +0,0 @@
|
|
|
1
|
-
# Release checklist
|
|
2
|
-
|
|
3
|
-
Nothing in this repository publishes automatically. The package name was available during research, but is not reserved. There is no configured remote or maintainer identity yet.
|
|
4
|
-
|
|
5
|
-
## Before the first release
|
|
6
|
-
|
|
7
|
-
1. Confirm ownership of the npm name and choose the public Git repository. Add accurate `repository`, `homepage`, and `bugs` metadata to `package.json`.
|
|
8
|
-
2. Confirm the MIT license choice and maintainer identity. Review the public artifact list; research, local paths, tests, and development scripts are not shipped.
|
|
9
|
-
3. Run `npm ci`, `npm run check`, and `npm run smoke:package` on the supported Node/Pi versions. Inspect `npm pack --dry-run`.
|
|
10
|
-
4. With explicit approval for network charges, run `/typesafe-router doctor` for each backend and configured generation target set with least-privilege credentials. Doctor preserves session mode and sends a synthetic classifier request plus one isolated generation probe per distinct provider/model, without confirmation, actual conversation transcript, or tools. It may incur charges even while off or headless with `allowHeadless: false`; token/deadline limits are not a strict monetary cap. Confirm response shape, confidence extraction, permissions, and cancellation. Do not capture keys or raw private state in fixtures. Check the following:
|
|
11
|
-
- Probes use Pi's `modelRegistry.complete` and actual credential providers, including OAuth refresh, custom headers, and custom endpoints. Options are `tools: []`, `maxTokens: 128` where supported, and `maxRetries: 0`. The top-level `generationProbeTimeoutMs` defaults to 15000 and accepts only integers from 100 through 60000, applied per model.
|
|
12
|
-
- Readiness requires completed doctor success, including the classifier, and at least one locally eligible, successfully probed candidate in every quick/standard/deep route. Failed targets, including the first configured target, are skipped during later preflight; successful targets retain their configured order.
|
|
13
|
-
- `on`, `shadow`, and automatic input are blocked without readiness; off/manual Pi use remains available. Doctor does not change the selected model or replay a user task.
|
|
14
|
-
- Proofs are session-only and never persisted. Restart/reload requires doctor again. Route, credential-reference, backend, and model-metadata changes invalidate verification even under unchanged provider/model IDs. Native auth status/provider references contribute to an in-memory hash without persisting secrets.
|
|
15
|
-
- Refresh invalidates old proofs, and cancellation or incomplete work never partially unlocks routing. Treat successful probes as health snapshots, not guarantees of future generation success.
|
|
16
|
-
5. Privately evaluate representative tasks. Seek clarification of TypeSafe MCA §2.3(f) before public benchmarks, performance numbers, or comparative claims.
|
|
17
|
-
6. Review privacy disclosures and experimental Vercel limitations. Choose the package version and record changes.
|
|
18
|
-
7. Configure npm trusted publishing for the chosen repository/workflow and an approved release environment. Use OIDC provenance rather than storing a long-lived npm token. This step depends on the final repository identity and is intentionally not preconfigured.
|
|
19
|
-
8. Only after release approval, publish the reviewed artifact with public access and provenance. Verify installation into a clean Pi profile.
|
|
20
|
-
9. Verify discovery in Pi's package gallery. The manifest already has the `pi-package` keyword and `pi.extensions` entry; gallery indexing is external and is not guaranteed by a local test.
|
|
21
|
-
|
|
22
|
-
The local smoke script installs the packed tarball and Pi into a temporary directory, disables resource discovery and network during extension loading, and checks the registered command through the actual loader. It does not alter the user's Pi settings or installed extensions.
|
|
23
|
-
|
|
24
|
-
## Ongoing releases
|
|
25
|
-
|
|
26
|
-
Keep host compatibility explicit. The tested host is Pi 0.85.1; wildcard optional peers avoid bundling a competing runtime but do not imply every Pi version is compatible. Re-characterize lifecycle/auth changes on a new host version. AI SDK evaluation is experimental and pinned; review serialization/metadata tests before upgrading it.
|