ytdb-slate 0.3.0 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +36 -4
- package/docs/context-budget.md +125 -0
- package/docs/design-principles.md +156 -7
- package/docs/model-routing.md +553 -0
- package/extension/base-model.ts +545 -0
- package/extension/episodes.ts +387 -31
- package/extension/failover.ts +27 -24
- package/extension/handoff.ts +44 -12
- package/extension/index.ts +140 -13
- package/extension/mode.ts +338 -3
- package/extension/model-default.ts +6 -2
- package/extension/model-profiles.ts +812 -0
- package/extension/model-router.ts +858 -0
- package/extension/notify.ts +20 -0
- package/extension/paths.ts +1 -0
- package/extension/render.ts +20 -1
- package/extension/route.ts +1185 -0
- package/extension/state.ts +451 -5
- package/extension/threads.ts +815 -45
- package/extension/tools.ts +80 -16
- package/extension/worker-extensions.ts +392 -0
- package/extension/worker.ts +115 -11
- package/package.json +8 -1
|
@@ -0,0 +1,553 @@
|
|
|
1
|
+
# Action-level model routing
|
|
2
|
+
|
|
3
|
+
Opt-in, per-action model and effort selection for worker
|
|
4
|
+
dispatches. `router` in the project's `slate.json` names a closed
|
|
5
|
+
list of models an action may run on; every dispatch then resolves to
|
|
6
|
+
one model from that list and one effort level from that model's own
|
|
7
|
+
ladder — chosen to be up to the task and no more, so the bill is
|
|
8
|
+
bounded by the action rather than by the session's current model.
|
|
9
|
+
The list is empty by default: with no entries the router is OFF, no
|
|
10
|
+
model list is enforced, no thread gets a seeded base, and a
|
|
11
|
+
dispatch that names no model resolves exactly as it did before the
|
|
12
|
+
router existed — the thread's pre-router pin, else the host
|
|
13
|
+
session's model. (The per-action `model` and `effort` arguments
|
|
14
|
+
themselves are honoured on both router states; see the accepted
|
|
15
|
+
limitations.) This document is reference documentation, not
|
|
16
|
+
workflow doctrine.
|
|
17
|
+
|
|
18
|
+
A model can only be routed to if Slate ships a benchmark profile for
|
|
19
|
+
it. An entry with no profile is named in a warning and excluded —
|
|
20
|
+
the router will not invent a tier, a price or an effort ladder for a
|
|
21
|
+
model it has no traced evidence about. Project-supplied profiles are
|
|
22
|
+
not implemented, so adding a model means a new shipped profile
|
|
23
|
+
rather than a config entry. The nine specs Slate profiles today are
|
|
24
|
+
listed under [Where the numbers come
|
|
25
|
+
from](#where-the-numbers-come-from-and-how-stale-they-can-be).
|
|
26
|
+
|
|
27
|
+
## What routing decides, per action
|
|
28
|
+
|
|
29
|
+
Two levers, both per dispatch — on a new thread and on a
|
|
30
|
+
continuation alike:
|
|
31
|
+
|
|
32
|
+
- **the model** — the `model` argument (`"provider/id"`), else the
|
|
33
|
+
thread's base model (router ON), else the thread's pre-router pin
|
|
34
|
+
or the host session's model (router OFF);
|
|
35
|
+
- **the effort level** — the `effort` argument (one of pi's thinking
|
|
36
|
+
levels), else a level Slate resolves for the model that actually
|
|
37
|
+
runs (router ON) or the level the worker session opened on (router
|
|
38
|
+
OFF). See [Effort levels](#effort-levels).
|
|
39
|
+
|
|
40
|
+
Neither is sticky, and that is the guarantee to hold on to: an
|
|
41
|
+
explicit `model` or `effort` governs its own action and no later
|
|
42
|
+
one. An omitted `model` resolves to the thread's base. An omitted
|
|
43
|
+
`effort` resolves differently on the two router states, and neither
|
|
44
|
+
of them is "whatever the last action asked for":
|
|
45
|
+
|
|
46
|
+
- **router on** — the thread's base effort, re-validated, or a level
|
|
47
|
+
derived for the model that runs (see [Effort
|
|
48
|
+
levels](#effort-levels)). A thread based on `luna@medium` whose
|
|
49
|
+
previous action ran at `max` runs the next one at `medium`.
|
|
50
|
+
- **router off** — nothing is resolved, so the worker session is put
|
|
51
|
+
back to the level it opened on, which is pi's own settings default
|
|
52
|
+
(`defaultThinkingLevel`, else pi's built-in default). Slate passes
|
|
53
|
+
that level explicitly at every open, so it is re-clamped against
|
|
54
|
+
the session's current model rather than inherited from the session
|
|
55
|
+
file.
|
|
56
|
+
|
|
57
|
+
A thread's base model and base effort are separate state, visible in
|
|
58
|
+
the `threads` listing, and are never set from an action's own route.
|
|
59
|
+
|
|
60
|
+
Two things can come back instead of a normal episode:
|
|
61
|
+
|
|
62
|
+
- a **tool error** — the pair was refused (an unlisted model, a
|
|
63
|
+
level the model does not offer, a level the provider rejects
|
|
64
|
+
outright, or an unevidenced level in a project that forbids one).
|
|
65
|
+
Nothing ran, nothing was billed and no episode was written; the
|
|
66
|
+
message names what is allowed.
|
|
67
|
+
- **⚠ notice lines** prefixed above the episode text in the tool
|
|
68
|
+
result — advisory: an evidence gap, a window substitution, a
|
|
69
|
+
long-context billing cliff, a re-seeded base. The action ran.
|
|
70
|
+
|
|
71
|
+
## Configuration
|
|
72
|
+
|
|
73
|
+
`router` lives in `.pi/slate.json` and, like the rest of that file,
|
|
74
|
+
is honored **only in trusted projects** — an untrusted project loads
|
|
75
|
+
no project config, so the router stays off. There are exactly two
|
|
76
|
+
keys:
|
|
77
|
+
|
|
78
|
+
```json
|
|
79
|
+
{
|
|
80
|
+
"router": {
|
|
81
|
+
"models": [
|
|
82
|
+
"openai/gpt-5.6-luna",
|
|
83
|
+
"anthropic/claude-sonnet-5",
|
|
84
|
+
"openai/gpt-5.6-terra",
|
|
85
|
+
"openai/gpt-5.6-sol",
|
|
86
|
+
"anthropic/claude-opus-5",
|
|
87
|
+
"anthropic/claude-fable-5"
|
|
88
|
+
],
|
|
89
|
+
"allowUnmeasuredEffort": true
|
|
90
|
+
}
|
|
91
|
+
}
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
| key | type | default | meaning |
|
|
95
|
+
| --- | --- | --- | --- |
|
|
96
|
+
| `models` | array of `"provider/id"` strings | `[]` | the closed candidate list; empty or absent = router OFF |
|
|
97
|
+
| `allowUnmeasuredEffort` | boolean | `true` | `false` turns an evidence gap from a warning into a refusal |
|
|
98
|
+
|
|
99
|
+
Validation, all of it at session start so a broken list surfaces
|
|
100
|
+
before a dispatch is refused by it:
|
|
101
|
+
|
|
102
|
+
- A `router` value that is not an object is ignored with a warning
|
|
103
|
+
and both defaults apply.
|
|
104
|
+
- An unknown key under `router` is reported by name, with the known
|
|
105
|
+
keys listed — a typo'd `"model"` must not read as an empty list.
|
|
106
|
+
- A `models` value that is not an array is ignored with a warning
|
|
107
|
+
(the router stays off).
|
|
108
|
+
- An individual entry that is not a canonical `provider/id` spec is
|
|
109
|
+
dropped with a warning naming the entry and the defect; the valid
|
|
110
|
+
entries in the same list still apply.
|
|
111
|
+
- A non-boolean `allowUnmeasuredEffort` is warned about and treated
|
|
112
|
+
as `true`.
|
|
113
|
+
|
|
114
|
+
The config is read once, at session start. Editing `slate.json`
|
|
115
|
+
mid-session takes effect only in a new pi session.
|
|
116
|
+
|
|
117
|
+
## How a model becomes routable
|
|
118
|
+
|
|
119
|
+
A configured model is routable only where four things intersect:
|
|
120
|
+
your list, Slate's shipped profile table, pi's model registry, and
|
|
121
|
+
the credentials pi actually has configured. Resolution happens once
|
|
122
|
+
per session, lazily at the first consultation, and the answer is
|
|
123
|
+
then frozen for that session.
|
|
124
|
+
|
|
125
|
+
| what happens | outcome | warning |
|
|
126
|
+
| --- | --- | --- |
|
|
127
|
+
| entry is not a canonical `provider/id` | dropped | names the entry and the defect |
|
|
128
|
+
| no profile in Slate's model profiles | dropped | "has no benchmark data … excluding it from routing" |
|
|
129
|
+
| two entries resolve to the same profile (an alias or a case variant) | first kept, second dropped | names both spellings and the profile |
|
|
130
|
+
| the same spec listed twice | first kept | none (silent) |
|
|
131
|
+
| pi's model registry does not know it | dropped | "routing there could only produce billed failures" |
|
|
132
|
+
| pi has no usable credentials configured for it | dropped | same wording, credentials variant |
|
|
133
|
+
| every entry dropped | router turns OFF | one summary line naming the count |
|
|
134
|
+
| no usable input price for today's date | kept, ordered last | "its cost cannot be compared" |
|
|
135
|
+
| no usable effort ladder in the profile | kept | "every explicit effort level for it will read as off-ladder" |
|
|
136
|
+
| profile's context window differs from the registry's | kept (registry value used) | see [Expected first-session warnings](#expected-first-session-warnings) |
|
|
137
|
+
| profile names unknown routing-critical fields | kept | "routing decisions for it are provisional", naming the fields |
|
|
138
|
+
| no `modelFailover` entry for a candidate | kept | one aggregate line naming every uncovered candidate |
|
|
139
|
+
|
|
140
|
+
Every warning is emitted at most once per session and reaches the
|
|
141
|
+
same channel as the other config warnings: a UI notification when a
|
|
142
|
+
UI is attached, the console otherwise.
|
|
143
|
+
|
|
144
|
+
Half a list is still a routing policy, so partial drops leave the
|
|
145
|
+
router ON. Nothing surviving is not a policy: the router turns OFF
|
|
146
|
+
rather than silently routing to whatever the session happens to be
|
|
147
|
+
on, which would hide the real problem.
|
|
148
|
+
|
|
149
|
+
The registry and credential reads are a snapshot taken at that first
|
|
150
|
+
consultation. Adding an API key later in the session does not revive
|
|
151
|
+
a model dropped as unauthenticated — start a new pi session.
|
|
152
|
+
|
|
153
|
+
The credential check is pi's synchronous configured-auth test, not a
|
|
154
|
+
live call: a key that is configured but expired or invalid survives
|
|
155
|
+
resolution and fails at dispatch instead, which is failover's
|
|
156
|
+
territory rather than the router's.
|
|
157
|
+
|
|
158
|
+
### Ordering and the thread base model
|
|
159
|
+
|
|
160
|
+
Candidates are ordered by five keys, in this order:
|
|
161
|
+
|
|
162
|
+
1. **preference** — a profile carrying a `nonPreferred` reason sorts
|
|
163
|
+
after every preferred candidate, absolutely, whatever the tier or
|
|
164
|
+
price says. The marker means "never a default pick";
|
|
165
|
+
2. **tier sourcing** — within a preference class, candidates whose
|
|
166
|
+
tier is a sourced ordinal come before those whose tier is only a
|
|
167
|
+
cost class read off the price;
|
|
168
|
+
3. **tier**, ascending (1 = cheapest class);
|
|
169
|
+
4. **current effective input price**, ascending;
|
|
170
|
+
5. **spec**, only so the order is total and reproducible.
|
|
171
|
+
|
|
172
|
+
The **base model of a new thread is the cheapest preferred
|
|
173
|
+
candidate**. That guarantees a dispatch which omits `model` can
|
|
174
|
+
never be rejected by the list guard. If every configured model is
|
|
175
|
+
marked non-preferred, the cheapest one is used anyway — the base
|
|
176
|
+
model has to exist — and that fallback is warned about.
|
|
177
|
+
|
|
178
|
+
Prices come from the profile's dated schedule, and the row in force
|
|
179
|
+
on today's date is the one used; a schedule with a dated step change
|
|
180
|
+
therefore re-orders candidates by itself on the day it takes effect.
|
|
181
|
+
Long-context multipliers are deliberately NOT folded into the
|
|
182
|
+
ordering price: they describe what happens above a token threshold,
|
|
183
|
+
not the base rate models are compared on.
|
|
184
|
+
|
|
185
|
+
An existing thread whose base is absent, or has fallen off the list
|
|
186
|
+
after a config change, is re-seeded to what a new thread would get,
|
|
187
|
+
with a warning, and the new base is persisted so it costs one
|
|
188
|
+
warning per thread rather than one per dispatch. An explicit
|
|
189
|
+
off-list `model` is still refused — that call has something to
|
|
190
|
+
correct.
|
|
191
|
+
|
|
192
|
+
## Effort levels
|
|
193
|
+
|
|
194
|
+
The vocabulary is pi's ladder and nothing else: `off`, `minimal`,
|
|
195
|
+
`low`, `medium`, `high`, `xhigh`, `max`. Which of them a model
|
|
196
|
+
actually offers is per model — the shipped table records a ladder
|
|
197
|
+
per model id, not a family rule.
|
|
198
|
+
|
|
199
|
+
**An omitted `effort` never resolves to a fixed default.** In order:
|
|
200
|
+
|
|
201
|
+
1. the thread's stored base effort — but only when the action runs
|
|
202
|
+
on the thread's base model, and only while that stored level
|
|
203
|
+
still reads as measured against today's profile table. A level
|
|
204
|
+
that no longer holds (a refreshed table moved it onto a gap, off
|
|
205
|
+
the ladder, or onto the provider's rejection list) is silently
|
|
206
|
+
re-derived; nobody asked for it, so a stale cache is Slate's
|
|
207
|
+
problem to correct rather than news to report;
|
|
208
|
+
2. otherwise the **lowest measured level of the model that actually
|
|
209
|
+
runs** — the lowest level on its ladder that carries a traced
|
|
210
|
+
capability measurement. A level derived for one model is never
|
|
211
|
+
carried onto another, so an explicit `model`, or a window
|
|
212
|
+
substitution, re-derives it;
|
|
213
|
+
3. and if that model has no measured level at all, nothing is set:
|
|
214
|
+
the level the worker session opened on applies, which is pi's own
|
|
215
|
+
settings default — never a level another action left behind.
|
|
216
|
+
|
|
217
|
+
So a higher level is only ever reached by naming it: pass `effort`
|
|
218
|
+
explicitly on the dispatch. A derived level is measured by
|
|
219
|
+
construction and therefore cannot trip the effort guards; only an
|
|
220
|
+
explicit one can.
|
|
221
|
+
|
|
222
|
+
An explicit level is judged against the model the action will
|
|
223
|
+
actually run on, and the difference between a warning and a refusal
|
|
224
|
+
is:
|
|
225
|
+
|
|
226
|
+
- **evidence gap** (the level is on that model's ladder but no
|
|
227
|
+
traced source reports a capability result there) — advisory. The
|
|
228
|
+
action runs, a ⚠ notice says the result is unevidenced for that
|
|
229
|
+
level, and the episode is marked. Setting
|
|
230
|
+
`router.allowUnmeasuredEffort: false` turns this one case into a
|
|
231
|
+
hard refusal for the whole project;
|
|
232
|
+
- **off the model's ladder** — refused, whenever the ladder is
|
|
233
|
+
known. pi would otherwise clamp the level silently and the
|
|
234
|
+
orchestrator would believe the action ran at a level the model
|
|
235
|
+
never offered;
|
|
236
|
+
- **rejected outright by the provider** — refused, always. Such a
|
|
237
|
+
level is still on pi's ladder for the model (the table records the
|
|
238
|
+
hard rejection separately), so dispatching it would be a
|
|
239
|
+
guaranteed API failure rather than an evidence gap.
|
|
240
|
+
`allowUnmeasuredEffort` does NOT cover it;
|
|
241
|
+
- **no ladder data at all** — nothing is refused and nothing is
|
|
242
|
+
marked. A failure to read evidence is not evidence of a problem;
|
|
243
|
+
the level goes to pi, which clamps it.
|
|
244
|
+
|
|
245
|
+
## The seven dispatch guards
|
|
246
|
+
|
|
247
|
+
Two planning passes run per dispatch: an early one before any state
|
|
248
|
+
is touched (a rejection there is a tool error and costs nothing) and
|
|
249
|
+
an apply-time one once the thread's real context size is knowable (a
|
|
250
|
+
rejection there aborts the dispatch before any billed work, with no
|
|
251
|
+
episode). Both run the same guards, in the same order — and the
|
|
252
|
+
order is load-bearing: the model is settled before any effort
|
|
253
|
+
judgement is made about it, so no level is judged against a model
|
|
254
|
+
that is about to be substituted away.
|
|
255
|
+
|
|
256
|
+
The seven are preceded by an argument check the code numbers 0,
|
|
257
|
+
listed first here because a caller meets it first:
|
|
258
|
+
|
|
259
|
+
| guard | what it protects against |
|
|
260
|
+
| --- | --- |
|
|
261
|
+
| effort vocabulary (0) | a level pi has no name for — and an `effort` argument that is not a string at all, which read as "omitted" would quietly run the action at a different level than the caller asked for |
|
|
262
|
+
| list membership | an explicit model outside the configured list, i.e. the cost bound silently escaped. A thread's base is repaired by re-seeding, never refused, so only an explicit off-list model is ever rejected |
|
|
263
|
+
| context window | a thread whose context no longer fits the routed model. Judged with pi's own compaction predicate against the REGISTRY window; NEVER a hard block. The action moves to the widest listed candidate only when that one is strictly wider, with a warning; if nothing wider exists it runs anyway and says so |
|
|
264
|
+
| API-rejected level | a guaranteed provider failure. Checked before the ladder, because such a level is on the model's pi ladder — the provider refuses it, the ladder does not |
|
|
265
|
+
| ladder validity | pi's silent clamp — the orchestrator believing an action ran at a level the model never offered. Fires only on a KNOWN ladder, and per model, never as a union over models |
|
|
266
|
+
| evidence gap | an unevidenced capability claim reported as a normal result. Advisory unless the project set `allowUnmeasuredEffort: false` |
|
|
267
|
+
| long-context billing | an unnoticed cost cliff: above a model's threshold the price multipliers apply. Warned once per thread and model; a cost event, never a capacity limit |
|
|
268
|
+
| failover carve-out | the router vetoing a rescue. On a **failover switch** the list and effort guards are bypassed entirely and the window check only warns: a model that just failed is worse than an unlisted one that works. The one rule failover still obeys is that the mapping may not resolve to the model that just failed — see `model-failover.md` |
|
|
269
|
+
|
|
270
|
+
Every guard goes inert when the data it needs cannot be read (a
|
|
271
|
+
missing profile, a throwing lookup, an unreadable ladder, an unknown
|
|
272
|
+
window) rather than refusing: the guards exist to stop dispatches
|
|
273
|
+
that are known to be wrong, not ones that could not be checked. The
|
|
274
|
+
one exception is a positive fact that is still readable — a
|
|
275
|
+
provider's hard rejection of a level still refuses it even when that
|
|
276
|
+
model's ladder is unreadable.
|
|
277
|
+
|
|
278
|
+
## What you'll see
|
|
279
|
+
|
|
280
|
+
- **On the call line (TUI):** what the action asked for, e.g.
|
|
281
|
+
`thread t1 [openai/gpt-5.6-sol @medium]`, or just `[@medium]` when
|
|
282
|
+
only the level was named.
|
|
283
|
+
- **On the collapsed result line (TUI):** what it actually ran on,
|
|
284
|
+
labelled so it cannot be read as the request:
|
|
285
|
+
`[ran openai/gpt-5.6-sol @medium]`, with a trailing `unmeasured`
|
|
286
|
+
when the level had no capability measurement. Request and result
|
|
287
|
+
can legitimately differ — pi clamps a level the model cannot do,
|
|
288
|
+
the window guard can substitute a wider model, and a failover
|
|
289
|
+
switches it again.
|
|
290
|
+
- **In the episode header:** a `ran:` segment on the existing
|
|
291
|
+
date/compressor line —
|
|
292
|
+
`> date: … | ran: openai/gpt-5.6-sol @ medium (unmeasured level) | compressor: …`.
|
|
293
|
+
This is the durable copy: it travels into every later prompt that
|
|
294
|
+
cites the episode. It names the model the session ended the action
|
|
295
|
+
on, and it is absent when the action produced no assistant message
|
|
296
|
+
at all. `compressor:` beside it is a different fact — the model
|
|
297
|
+
that wrote the episode body.
|
|
298
|
+
- **In the tool result:** ⚠ notice lines above the episode text (so
|
|
299
|
+
the orchestrator reads them — a cost cliff or an evidence gap is
|
|
300
|
+
its decision to make), the same lines in the live progress output,
|
|
301
|
+
and `details.ranModel` / `details.ranEffort` /
|
|
302
|
+
`details.ranEffortUnmeasured` / `details.warnings` for a renderer.
|
|
303
|
+
- **In the `threads` listing:** `base=<model>@<level>?` — what a
|
|
304
|
+
dispatch that omits `model` will run on. The trailing `?` marks
|
|
305
|
+
the level, not the model, as provisional: it is a stored default,
|
|
306
|
+
re-checked on every dispatch. Then `last=<model>@<level>` — what the thread's
|
|
307
|
+
last action actually ran on, with `(unmeasured)` where that
|
|
308
|
+
applies — and `live=<model> (failover)` while a failover holds the
|
|
309
|
+
live session.
|
|
310
|
+
- **At the session level:** the resolution's own warnings, as UI
|
|
311
|
+
notifications (console when no UI is attached).
|
|
312
|
+
- **In the orchestrator's own system prompt, every turn:** the
|
|
313
|
+
doctrine gains a routing rule — a table with one row per routable
|
|
314
|
+
model, plus the rules for reading it. This is the surface you do
|
|
315
|
+
not see, and it is the router's standing cost: 1,941 characters /
|
|
316
|
+
20 lines for six configured models, 2,519 / 23 for all nine — a
|
|
317
|
+
model row costs 154–186 characters, plus a one-off legend clause
|
|
318
|
+
for each marker it introduces (listing the first cheap-tier model
|
|
319
|
+
costs ~50 characters beyond its row, which is why six→nine
|
|
320
|
+
averages 193 per model against the first six's 171). Configuring
|
|
321
|
+
the router roughly doubles Slate's always-loaded block (1,929
|
|
322
|
+
characters with routing off). Those are PORTABLE characters — the
|
|
323
|
+
doctrine with each occurrence of the installed `docs/` directory
|
|
324
|
+
removed, filenames kept — because the doctrine embeds absolute doc
|
|
325
|
+
paths and its raw size therefore depends on where the package is
|
|
326
|
+
installed. `context-budget.md` states that convention, tabulates
|
|
327
|
+
every configuration with its full basis, and gives the arithmetic
|
|
328
|
+
for your own install.
|
|
329
|
+
|
|
330
|
+
## Expected first-session warnings
|
|
331
|
+
|
|
332
|
+
**A batch of warnings on the first routed session is expected UX,
|
|
333
|
+
not a bug.** With a six-model list on a stock pi install, resolution
|
|
334
|
+
emits eleven lines, broken down below. They are not repeated: every
|
|
335
|
+
warning fires at most once per session, and the resolution is frozen
|
|
336
|
+
after the first consultation.
|
|
337
|
+
|
|
338
|
+
In orchestrator mode that first consultation is the DOCTRINE BUILD,
|
|
339
|
+
not a dispatch: the orchestrator's system prompt carries the
|
|
340
|
+
routable model table, and it is assembled when your first message
|
|
341
|
+
starts a turn. So the whole batch can arrive in a session where you
|
|
342
|
+
typed nothing but "hi" and dispatched nothing at all — that is the
|
|
343
|
+
router reading its inputs for the session, not a dispatch
|
|
344
|
+
misfiring. (Outside orchestrator mode no doctrine is built, so the
|
|
345
|
+
first dispatch triggers it instead. The config-shape warnings above
|
|
346
|
+
land earlier still, at session start.)
|
|
347
|
+
|
|
348
|
+
The breakdown:
|
|
349
|
+
|
|
350
|
+
| lines | class | why |
|
|
351
|
+
| --- | --- | --- |
|
|
352
|
+
| 6 | unknown routing data | each of the six profiles names routing-critical fields with no traced value, or with two published values and no adjudication, so routing decisions for them are provisional |
|
|
353
|
+
| 3 | context-window divergence | the three `openai/gpt-5.6-*` models — see below |
|
|
354
|
+
| 1 | billing-pattern note | one aggregate line naming those three models |
|
|
355
|
+
| 1 | failover coverage | one aggregate line, when `modelFailover` covers none of them |
|
|
356
|
+
|
|
357
|
+
The window divergence is real, and both sides are stated without
|
|
358
|
+
adjudication. On a stock install pi's registry reports **272,000**
|
|
359
|
+
tokens for `openai/gpt-5.6-luna`, `-terra` and `-sol`, while their
|
|
360
|
+
profiles record **1,050,000** (with the profile's asOf date in the
|
|
361
|
+
same line). Routing uses the REGISTRY figure — that is a fact about
|
|
362
|
+
this code, and it is stated as such — but which figure is factually
|
|
363
|
+
correct is not established by the router: it sees two numbers and no
|
|
364
|
+
provenance for either.
|
|
365
|
+
|
|
366
|
+
The registry's 272,000 is also exactly those models' long-context
|
|
367
|
+
BILLING threshold, so one further line says so once, naming all
|
|
368
|
+
three: a window equal to its own threshold would leave the
|
|
369
|
+
long-context price tier unreachable, which is the
|
|
370
|
+
billing-row-restated-as-a-capacity pattern. That is suggestive, not
|
|
371
|
+
proof, which is why it is a pointer and not a verdict.
|
|
372
|
+
|
|
373
|
+
**Why Slate does not "fix" this by changing its own number.** The
|
|
374
|
+
research pass that produced this table explicitly withdrew 272,000
|
|
375
|
+
as a capacity figure: in the source it appears only inside a pricing
|
|
376
|
+
row, as a boundary on input tokens. Restating it as a window is the
|
|
377
|
+
fabrication that pass caught and corrected. Editing the profile to
|
|
378
|
+
match the registry would silence the warning by asserting something
|
|
379
|
+
the evidence does not support, and would suppress a divergence that
|
|
380
|
+
is currently true. The honest options are the two you already have:
|
|
381
|
+
leave it (routing uses the registry number, which is the
|
|
382
|
+
conservative one) or raise the registry window deliberately in pi's
|
|
383
|
+
custom-models file, after which the warning stops on its own —
|
|
384
|
+
`context-budget.md` documents that opt-in and the billing
|
|
385
|
+
consequences of running above the threshold.
|
|
386
|
+
|
|
387
|
+
## What the router does NOT enforce
|
|
388
|
+
|
|
389
|
+
Two obligations carried by the shipped profile data and stated in
|
|
390
|
+
the injected doctrine are **doctrine obligations, not code-enforced
|
|
391
|
+
guards**. Both are PREVENTIVE — they decide what may be dispatched,
|
|
392
|
+
not what must be checked afterwards — and nothing in the dispatch
|
|
393
|
+
path implements either. A warning must not be read as an interlock:
|
|
394
|
+
|
|
395
|
+
- **The compliance refusal.** `anthropic/claude-fable-5` has no
|
|
396
|
+
zero-data-retention option (mandatory 30-day retention, first-
|
|
397
|
+
and third-party), and its profile says a ZDR-obligated action
|
|
398
|
+
must be REFUSED there at every effort level. Slate's dispatch
|
|
399
|
+
path has no concept of a ZDR-obligated action: list that model
|
|
400
|
+
and an action routed to it will run. The refusal is the
|
|
401
|
+
orchestrator's to make, under the doctrine and under your own
|
|
402
|
+
compliance rules.
|
|
403
|
+
- **The measured-level rule for review and gate actions.** Keep
|
|
404
|
+
review and gate actions ON MEASURED LEVELS — dispatch them at a
|
|
405
|
+
level the target model has a capability measurement at. This is a
|
|
406
|
+
rule about what to dispatch, not a licence to run a review at an
|
|
407
|
+
unevidenced level and treat the result carefully afterwards: an
|
|
408
|
+
unmeasured review result is not evidence that can be repaired by
|
|
409
|
+
reading it sceptically, which is why the doctrine states the rule
|
|
410
|
+
up front. Slate's code does not enforce it. It contributes exactly
|
|
411
|
+
two things — the ⚠ notice at dispatch and the `(unmeasured level)`
|
|
412
|
+
marker in the episode header — and neither refuses the dispatch,
|
|
413
|
+
distinguishes a review action from any other, or checks anything
|
|
414
|
+
after the fact. `allowUnmeasuredEffort: false` is not this rule
|
|
415
|
+
either: it is a blanket refusal of unevidenced levels for every
|
|
416
|
+
action in the project.
|
|
417
|
+
|
|
418
|
+
The same is true of a model whose profile puts a verification gate
|
|
419
|
+
on its high effort levels: the gate is advice the orchestrator must
|
|
420
|
+
act on, and the dispatch path neither enforces nor tracks it.
|
|
421
|
+
|
|
422
|
+
The guards in the table above are the complete list of what the
|
|
423
|
+
routing code refuses. Everything else in the profile data — hazards,
|
|
424
|
+
`routeFor` / `avoidFor` clauses, verification gates — is advice the
|
|
425
|
+
orchestrator is expected to follow, not a mechanism.
|
|
426
|
+
|
|
427
|
+
## Where the numbers come from, and how stale they can be
|
|
428
|
+
|
|
429
|
+
The routing data is a static, deep-frozen table in
|
|
430
|
+
`extension/model-profiles.ts` that ships with the package. There is
|
|
431
|
+
no network access and no runtime refresh; the table changes only
|
|
432
|
+
when the package is republished.
|
|
433
|
+
|
|
434
|
+
It profiles nine models, and because the list is closed these are
|
|
435
|
+
the only specs `router.models` can be built from. Copy the spelling
|
|
436
|
+
exactly — a spec that differs is dropped as unprofiled:
|
|
437
|
+
|
|
438
|
+
| canonical spec | measured levels | notes |
|
|
439
|
+
| --- | --- | --- |
|
|
440
|
+
| `openai/gpt-5.6-luna` | medium, max | |
|
|
441
|
+
| `openai/gpt-5.6-terra` | xhigh, max | non-preferred: configured-only, never auto-selected |
|
|
442
|
+
| `openai/gpt-5.6-sol` | medium, high, xhigh, max | |
|
|
443
|
+
| `anthropic/claude-sonnet-5` | high, xhigh, max | non-preferred |
|
|
444
|
+
| `anthropic/claude-opus-5` | low, medium, high, xhigh, max | |
|
|
445
|
+
| `anthropic/claude-fable-5` | high, xhigh, max | non-preferred; no zero-data-retention option (see [What the router does NOT enforce](#what-the-router-does-not-enforce)) |
|
|
446
|
+
| `openai/gpt-5.4-nano` | none | cheap tier, out of scope (below) |
|
|
447
|
+
| `openai/gpt-5.4-mini` | none | cheap tier, out of scope |
|
|
448
|
+
| `anthropic/claude-haiku-4-5` | none | cheap tier, out of scope |
|
|
449
|
+
|
|
450
|
+
"Measured levels" are the levels an omitted `effort` can resolve to
|
|
451
|
+
and the levels an explicit one passes the evidence-gap guard at; the
|
|
452
|
+
first of each list is what a thread based on that model starts at.
|
|
453
|
+
|
|
454
|
+
**Use the canonical spelling.** The table also carries alias
|
|
455
|
+
spellings — the research corpus's dated snapshot ids, and
|
|
456
|
+
`openai/gpt-5.6` for sol — but an alias is only a lookup key for the
|
|
457
|
+
profile, not a routable spec. Checked against a stock pi install:
|
|
458
|
+
`openai/gpt-5.6`, `openai/gpt-5.4-nano-2026-03-17` and
|
|
459
|
+
`openai/gpt-5.4-mini-2026-03-17` are NOT in pi's registry, so listing
|
|
460
|
+
one of them drops it with the "not in pi's model registry" warning
|
|
461
|
+
even though the profile was found. `anthropic/claude-haiku-4-5-20251001`
|
|
462
|
+
happens to be a real registry id and does route. Registry contents
|
|
463
|
+
change, so treat that as a dated observation and prefer the canonical
|
|
464
|
+
column above.
|
|
465
|
+
|
|
466
|
+
The last three are profiled but OUT OF SCOPE for routing — they are
|
|
467
|
+
there so that naming one gets you data instead of a spurious "no
|
|
468
|
+
profile" warning, and all three are marked non-preferred, carry
|
|
469
|
+
assumed rather than traced effort ladders, and have no
|
|
470
|
+
effort-labelled capability results at all. Routing to them is a
|
|
471
|
+
deliberate scope decision, not a default.
|
|
472
|
+
|
|
473
|
+
`PROFILES_AS_OF` is **2026-07-29** — the date of the research behind
|
|
474
|
+
the table — and every profile carries the same date in its own
|
|
475
|
+
`asOf`, which is what the divergence warning quotes. The only
|
|
476
|
+
time-varying part of the data is price-row selection: a schedule
|
|
477
|
+
with a dated step change switches rows by itself on that date.
|
|
478
|
+
|
|
479
|
+
The table's own provenance rules, which the warnings above depend
|
|
480
|
+
on:
|
|
481
|
+
|
|
482
|
+
- a value is transcribed from the research corpus unless the file
|
|
483
|
+
marks it otherwise at its own site (pi's registry decides the id
|
|
484
|
+
spelling; an unsourced tier is a cost class, not a ranking; an
|
|
485
|
+
assumed ladder is a provider-family shape, not a traced fact;
|
|
486
|
+
aliases are resolution spellings, not data);
|
|
487
|
+
- a figure that cannot be traced is NOT carried: the field is `null`
|
|
488
|
+
and its name appears in the profile's unknown-routing-critical
|
|
489
|
+
list, which is exactly what the "routing decisions for it are
|
|
490
|
+
provisional" warning reports;
|
|
491
|
+
- context window and max output are DOCUMENTATION-ONLY and
|
|
492
|
+
non-authoritative — pi's registry is the single runtime authority,
|
|
493
|
+
and the profile figures exist only so a cross-check can warn;
|
|
494
|
+
- long-context threshold and multipliers are BILLING, never
|
|
495
|
+
capacity: crossing the threshold costs money, it does not fail;
|
|
496
|
+
- prices are the provider's first-party standard tier only. Batch,
|
|
497
|
+
flex, priority and fast-mode tiers and every regional or
|
|
498
|
+
geographic uplift are NOT carried, so a dispatch on any of those
|
|
499
|
+
surfaces bills above these numbers.
|
|
500
|
+
|
|
501
|
+
**Standing limitation: the automated checks are provably blind to
|
|
502
|
+
wrong research data.** Slate's automated checks (a development
|
|
503
|
+
harness in the source repository — it is not part of the published
|
|
504
|
+
package) assert **structure only** for this table — ids and aliases
|
|
505
|
+
resolve, ladders are
|
|
506
|
+
duplicate-free subsets of pi's vocabulary, the measured and gap
|
|
507
|
+
lists are disjoint and cover the ladder, price rows are well formed
|
|
508
|
+
and tiers do not price-invert, the table is frozen. It asserts no
|
|
509
|
+
research number, and cannot: scaling every price by the same factor
|
|
510
|
+
passes green, a tier moved so that it does not invert prices passes
|
|
511
|
+
green, and an invented hazard clause or evidence sentence passes
|
|
512
|
+
green. Numeric and evidential fidelity to the research is a review
|
|
513
|
+
concern; a green suite says nothing about it.
|
|
514
|
+
|
|
515
|
+
## Accepted limitations
|
|
516
|
+
|
|
517
|
+
- **Frozen per session.** The candidate list, the registry lookup
|
|
518
|
+
and the credential check are resolved once and cached, so a
|
|
519
|
+
dispatch guard and anything else that consults the router agree.
|
|
520
|
+
A key or provider added mid-session, and an edit to
|
|
521
|
+
`slate.json`, both need a new pi session.
|
|
522
|
+
- **Router OFF is not "no effort checking".** With an empty list
|
|
523
|
+
there is no model list, no window guard, no billing notice, no
|
|
524
|
+
seeded base and no substitution — but an explicit `effort` is
|
|
525
|
+
still judged against the shipped profile for the model that runs,
|
|
526
|
+
when pi's registry can serve that model. The `model` argument
|
|
527
|
+
itself is passed through byte-for-byte, so a malformed spec still
|
|
528
|
+
produces pi's own error.
|
|
529
|
+
- **A mid-thread model switch costs the prompt cache.** A long
|
|
530
|
+
thread runs overwhelmingly on cache reads; routing a small action
|
|
531
|
+
to a cheaper model on a cold cache can cost more than staying on
|
|
532
|
+
the warm one. The router compares base rates, not the cache state
|
|
533
|
+
of your thread.
|
|
534
|
+
- **The window guard can move an action to a model you did not
|
|
535
|
+
choose** (it warns, and names what it replaced), and it never
|
|
536
|
+
blocks: if nothing listed is wider, the action runs and pi
|
|
537
|
+
compacts.
|
|
538
|
+
- **A `base=` level in the `threads` listing is provisional** — it
|
|
539
|
+
is re-checked on every dispatch and may be re-derived silently.
|
|
540
|
+
The `last=` value is fact.
|
|
541
|
+
- **The router bounds the price of an action, not a budget.** There
|
|
542
|
+
is no spend cap, no cross-action cost target and no per-action
|
|
543
|
+
cost attribution: episodes and the `threads` listing account for
|
|
544
|
+
spend per thread and per session, not per route.
|
|
545
|
+
- **A longer model list is not free.** Every routable model adds a
|
|
546
|
+
row to the doctrine table that sits in the orchestrator's system
|
|
547
|
+
prompt for the whole session (see [What you'll
|
|
548
|
+
see](#what-youll-see)), so listing models you will not route to
|
|
549
|
+
costs context on every turn. List the models you actually want
|
|
550
|
+
actions to land on.
|
|
551
|
+
- **Unprofiled models cannot be routed at all.** A model pi can
|
|
552
|
+
serve but the shipped table does not profile is excluded by
|
|
553
|
+
design; adding one today requires a table refresh in the package.
|