instar 1.3.770 → 1.3.772

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,50 @@
1
+ # ELI16 — Nature-Axis Routing, Increment A2.1 (the dark/dryRun mechanism)
2
+
3
+ ## The one-sentence version
4
+
5
+ We taught the internal LLM router how to pick *which model, on which access door* a
6
+ background check should run on — based on the **kind** of thinking the check does — but
7
+ we shipped it **turned off** everywhere except as a silent "here's what I *would* pick"
8
+ observer on a development agent.
9
+
10
+ ## Why
11
+
12
+ Different internal jobs need different brains. A quick "is this a STOP command?" sort is a
13
+ different task than a careful "should this message be blocked?" judgment. The bench (INSTAR
14
+ v3) measured which model+door combo is best for each kind — and found one combo that's
15
+ actively *dangerous* for careful judgments: Opus running through the Claude Code CLI (the
16
+ CLI wraps every prompt in ~20k tokens of "helpful assistant" framing, which makes a skeptical
17
+ judge go soft — it missed real STOP commands 27% of the time). So routing has to be able to
18
+ say "for a judgment call, NEVER land on that door."
19
+
20
+ ## What this increment actually adds (and what it deliberately doesn't)
21
+
22
+ **Adds (all pure, all dark):**
23
+ - A **resolver** — a small, side-effect-free function: given a component, it works out the
24
+ task *nature*, walks an ordered list of `(door, model)` candidates, skips the ones that
25
+ aren't reachable, and returns the winner + the fallback tail. It has exactly four honest
26
+ outcomes: a route, "fall through to today's routing" (for a component we haven't mapped),
27
+ "no route — use your own backup" (for a low-stakes sorter when everything's down), or
28
+ **throw a distinct fail-closed error** (for a safety gate when everything's down — a gate
29
+ must fail shut, never open).
30
+ - **FD4.1 — the concrete pin.** A2 finishes the job A1 left: the sanctioned Claude reserve
31
+ now pins to a *concrete* model id (`claude-sonnet-4-6`), not a tier nickname that could
32
+ drift. And a **deny-by-default allowlist** clamp: on the Claude door, a judgment/sort call
33
+ may ONLY use that one reserve id — anything else gets clamped down to it. (Open-ended
34
+ *writing* is exempt — that's where Opus-via-CLI is legitimately the best tool.)
35
+ - The `sessions.natureRouting` **config knob** (seeded dark on update), the wiring, and the
36
+ full test suite.
37
+
38
+ **Doesn't (tracked, not dropped):** the actual *re-routing* (enforcing mode), the injection
39
+ map, the build-time lint, the live-config validator, the durable audit log + dashboard read
40
+ surface, the critical-gate drift notice, and the Fable→Opus migration — those are the ordered
41
+ A2.2 remainder. And **nothing** about the paid metered doors or the money/PIN go-live — that
42
+ is Increment B, deferred and PIN-gated.
43
+
44
+ ## The safety promise
45
+
46
+ When the feature is **off** (which is everywhere on the fleet), the router behaves **bit-for-bit
47
+ like today** — same door, same model, same everything. That's the whole safety case, and there's
48
+ a named test that asserts it (`natureRouting UNSET ⇒ selection unchanged`). On a dev agent it runs
49
+ in **dryRun**: it computes and logs what it *would* pick, then still does exactly today's thing.
50
+ Flipping it to actually re-route is a later, operator-driven step.
@@ -0,0 +1,77 @@
1
+ # Side-Effects Review — LA4 unconditional degrade-path safety clamp (S4 Increment A1)
2
+
3
+ **Spec:** docs/specs/nature-axis-routing.md (status: **converged — pending operator approval** — NOT yet
4
+ approved; this ships as a **Tier-1** standalone safety narrowing, the smallest independently-landable
5
+ unit the spec's FD9 defines as **A1**). **Parent standards:** "Structure > Willpower", "No Silent
6
+ Degradation to Brittle Fallback", benchmark-cited routing (INSTAR-Bench v3, rules R1/R2).
7
+ **Not dark-gated:** A1 is UNCONDITIONAL by design — it fires regardless of `sessions.natureRouting`.
8
+ **Files:** src/core/IntelligenceRouter.ts, tests/unit/la4-degrade-path-clamp.test.ts,
9
+ docs/specs/nature-axis-routing.md (+ .eli16.md + reports/nature-axis-routing-convergence.md — the
10
+ converged spec docs riding to main), upgrades/la4-degrade-clamp-a1.eli16.md,
11
+ upgrades/side-effects/la4-degrade-clamp-a1.md, upgrades/next/la4-degrade-clamp-a1.md.
12
+
13
+ ## What changed
14
+
15
+ 1. **IntelligenceRouter.ts — the degrade-to-default path (`evaluate()`, the `if (!primary)` block).**
16
+ The shipped router degrades a binary-missing routed framework to
17
+ `defaultProvider.evaluate(prompt, options)` UNCLAMPED. When `defaultFramework === 'claude-code'` and
18
+ `options.model === 'capable'`, that landing is Opus-via-Claude-CLI — the measured-banned route for a
19
+ bounded/gating verdict (99.1% clean-API vs 81.7% via the CLI; emergency-stop 73%). The merged S2 clamp
20
+ (`clampClaudeCliSwapModel`) only guarded the failure-swap LOOP, leaving this exit open (the spec's LA4
21
+ fail-open). A1 clamps this exit: for a bounded/gating call, `options.model` is passed through
22
+ `clampClaudeCliSwapModel(defaultFramework, options.model)` (the SAME `capable→balanced` reserve S2
23
+ uses), and a distinct `degrade-path-model-clamp (LA4)` degrade note is emitted.
24
+ 2. **Two new exported pure helpers** in the same module: `routingNatureFor(component)` (a read-only lookup
25
+ of the merged S1 `LLM_ROUTING_NATURE` map with the same per-operation "/segment" + `server:` key
26
+ handling `categoryForComponent` uses) and `isBoundedGatingDegrade(component, options)` — the FD4/CR6-3
27
+ predicate: TRUE iff `attribution.gating === true` OR the component maps to a non-`WRITE` chain. A
28
+ `WRITE`-chain component (its legitimate Opus-CLI quality lane) or an unmapped, non-gating call ⇒ FALSE
29
+ (left unchanged).
30
+ 3. **Spec docs ride to main** (same pattern as reviewer-door inc1): the converged nature-axis-routing spec
31
+ + its ELI16 + convergence report, with `status: "converged — pending operator approval"` and the
32
+ `review-convergence` tag preserved AS-IS (no `approved: true` — that is the operator's step).
33
+
34
+ ## Blast radius
35
+
36
+ - **The clamp only ever NARROWS.** `clampClaudeCliSwapModel` returns `capable→balanced` only for
37
+ `claude-code`; every other door and every non-`capable` tier passes through unchanged. It never upgrades
38
+ a tier, never blocks a call, never touches WRITE.
39
+ - **Scope is exactly R1/R2.** `isBoundedGatingDegrade` fires only for a mapped non-WRITE nature or an
40
+ explicit `attribution.gating` flag. An unmapped, non-gating degrade (the common "other" category) is
41
+ left byte-identical. A non-`claude-code` default door is never clamped (Opus-via-API is fine).
42
+ - **No new config, no new route, no state.** Two pure functions + a few lines on an existing branch. The
43
+ static map read is O(1) and always available, so the predicate never depends on `natureRouting` being
44
+ set — which is what makes the clamp genuinely unconditional.
45
+ - **Honest non-byte-identical deviation.** This is the ONE deliberate behavior change on the degrade path:
46
+ a binary-missing bounded/gating `capable` degrade with a `claude-code` default now yields Sonnet-CLI
47
+ instead of Opus-CLI. Strictly the safe direction (measured-worse route → sanctioned reserve). Every
48
+ other path is untouched.
49
+
50
+ ## Risk + mitigation
51
+
52
+ - **Risk:** the clamp over-reaches and downgrades a legitimate Opus-CLI WRITE call. **Mitigation:**
53
+ `isBoundedGatingDegrade` keys on `chain !== 'WRITE'`; a WRITE component is exempt. Proven by a property
54
+ test over the whole `LLM_ROUTING_NATURE` map (predicate === `chain !== 'WRITE'` for every mapped row).
55
+ - **Risk:** the clamp silently fails to fire when the feature is off (the exact fleet-default state where
56
+ the fail-open lives). **Mitigation:** the predicate reads only the static map + `attribution`, never
57
+ `natureRouting`; a test asserts the clamp fires with `natureRouting` UNSET, and separately that a
58
+ non-`claude-code` default is left alone (both-default-framework arms).
59
+ - **Risk:** collateral change to unrelated calls. **Mitigation:** unmapped, non-gating degrade test
60
+ asserts `model` is passed through untouched (`capable` stays `capable`).
61
+
62
+ ## Framework generality
63
+
64
+ Not applicable to the session-launch/inject abstraction (this touches the internal LLM router only). The
65
+ clamp is keyed on the `claude-code` DOOR specifically because that door carries the measured harness
66
+ penalty; codex-cli / gemini-cli / pi-cli defaults are correctly never clamped (Opus-via-those-doors is
67
+ not the banned route). This is framework-correct, not a Claude-only assumption.
68
+
69
+ ## Tests
70
+
71
+ `tests/unit/la4-degrade-path-clamp.test.ts` (13 cases): `routingNatureFor` key handling; the
72
+ `isBoundedGatingDegrade` predicate incl. the whole-map property test; and the load-bearing `evaluate()`
73
+ integration — the clamp fires on a bounded/gating degrade with `natureRouting` unset, fires on a
74
+ gating-flagged unmapped call, does NOT over-clamp an unmapped non-gating call, does NOT clamp a
75
+ non-`claude-code` default, and leaves a non-`capable` tier alone. Existing
76
+ `opus-claude-cli-gating-guardrail.test.ts` (14) + `intelligence-router.test.ts` (17) +
77
+ `provider-fallback-swap-timeout.test.ts` (11) stay green; `tsc` clean.
@@ -0,0 +1,89 @@
1
+ # Side-Effects Review — Nature-Axis Routing, Increment A2.1 (dark/dryRun mechanism)
2
+
3
+ **Spec:** docs/specs/nature-axis-routing.md (status: **converged — pending operator approval**;
4
+ `review-convergence` tag, NOT `approved:true` — the operator's step). This ships as a **Tier-1**
5
+ change: the smallest independently-landable **dark** unit of FD9's Increment A2. **Parent standards:**
6
+ "Structure > Willpower", "No Silent Degradation to Brittle Fallback", benchmark-cited routing
7
+ (INSTAR-Bench v3, rules R1/R2/R8), the Maturation Path (dev-gated dark), Migration Parity.
8
+
9
+ **Files:** src/data/llmBenchCoverage.ts, src/core/IntelligenceRouter.ts, src/core/types.ts,
10
+ src/commands/server.ts, src/core/PostUpdateMigrator.ts, tests/unit/nature-routing-resolver.test.ts,
11
+ tests/unit/migrate-nature-routing-dark.test.ts, upgrades/nature-routing-a2.eli16.md,
12
+ upgrades/side-effects/nature-routing-a2.md, upgrades/next/nature-routing-a2.md.
13
+
14
+ ## What changed
15
+
16
+ 1. **src/data/llmBenchCoverage.ts — the DATA half.** New pure, read-only constants: `RoutingDoor`
17
+ (CLI + metered-API door taxonomy, FD1), `CLI_ROUTING_DOORS` / `METERED_ROUTING_DOORS`,
18
+ `ChainPosition` + `NATURE_ROUTING_DEFAULT_CHAINS` (the four v3 CLI-only chains, FD2 — metered
19
+ positions are present but always skipped in Increment A), `ROUTING_LABEL_TO_MODEL_ID` (FD-LABEL/
20
+ FD4.1 — benchmark-label → concrete model id), `CLAUDE_CODE_RESERVE_MODEL_ID` (the single
21
+ sanctioned `claude-sonnet-4-6` reserve, FD4 place 1), and `NATURE_ROUTING_CRITICAL_GATES` (FD6).
22
+ Importing this changes NO behavior — it is data actuated only when the feature is enabled.
23
+ 2. **src/core/IntelligenceRouter.ts — the resolver.** New pure exports: `resolveNatureAndChain`
24
+ (FD3 tighten rule `E,B ≥ D ≥ A`), `resolvePositionModelId` (FD4.1), `clampToReserveOnCleanDoor`
25
+ (FD4 place-3 allowlist clamp), `resolveRoute` (the stateless fold with the four outcomes),
26
+ `mergeNatureRoutingChains`, and the `RouterFailClosedError` typed error. Two new
27
+ `IntelligenceRouterOptions` fields (`resolveNatureRouting`, `onNatureRoutePlan`) and a new,
28
+ tightly-scoped block in `evaluate()` that OBSERVES when the feature is enabled.
29
+ 3. **src/core/types.ts** — the `attribution.nature` opt-in tightening field (FD3) and the
30
+ `sessions.natureRouting` config schema (type-only, NOT in ConfigDefaults, per the
31
+ absent-equals-unchanged rule that `componentFrameworks`/`dynamicMcp` follow).
32
+ 4. **src/commands/server.ts** — the construction wiring: `resolveNatureRouting` reads config LIVE
33
+ per call and resolves `enabled` through `resolveDevAgentGate` (live-in-dryRun on a dev agent,
34
+ dark on the fleet); `onNatureRoutePlan` is an env-gated observe-only breadcrumb.
35
+ 5. **src/core/PostUpdateMigrator.ts** — `migrateConfigNatureRoutingDark` seeds `sessions.natureRouting`
36
+ dark on existing agents (`schemaVersion:3`, `dryRun:true`, `metered.goLive:false`; `enabled`
37
+ OMITTED for the dev-gate — the #1001 enable-path-integrity pattern), existence-checked + idempotent.
38
+
39
+ ## Explicitly DEFERRED (tracked A2.2 remainder — NOT dropped)
40
+
41
+ Read this as intentional scope, not omission. This increment ships the pure mechanism + dark/dryRun
42
+ observation. The following are the ordered remainder, each a separate landable change:
43
+
44
+ - **Enforcing SELECTION** (dryRun:false actually re-routes) — A2.1 wires OBSERVATION only; flipping
45
+ `dryRun:false` logs a one-time "enforcing not yet wired" warning and stays byte-identical (an honest
46
+ no-op, never a silent dead switch and never a mis-route).
47
+ - **FD4 places 1 & 2** — the build-time lint (`scripts/lint-nature-chains.mjs`) and the resolve-time
48
+ live-config validator. **Place 3 (the runtime allowlist clamp) DOES ship here** and is the actual
49
+ runtime guarantee: it clamps every resolved claude-code FAST/SORT/JUDGE position to the sanctioned
50
+ reserve id at selection time — so even a hand-edited chain cannot open the banned Opus-via-CLI route.
51
+ Places 1-2 are defense-in-depth for the build + config-edit surfaces.
52
+ - **FD5b injection-exposure map** + R-rule lints (FD5c) — inert in A2.1 anyway: the only injection-
53
+ restricted door (`groq-api`) and the R8 Flash-Lite pin are METERED doors, always skipped in Increment
54
+ A; no CLI door in any chain is injection-restricted.
55
+ - **The durable audit** (`logs/nature-routing.jsonl`) + `GET /intelligence/routing` dryRun plan/diff/
56
+ `?trace` read surface, the **FD6 aggregated critical-gate drift notice** + baseline, the **FD8
57
+ Fable→Opus** migration, and the **CLAUDE.md** capability blurb.
58
+ - **Increment B** (metered-door live routing + FD12 money/PIN go-live) — DEFERRED + PIN-gated, an
59
+ operator step, never autonomous. No metered door routes and no spend ledger is touched here.
60
+
61
+ ## Blast radius
62
+
63
+ - **Byte-identical when off — THE safety case.** When `sessions.natureRouting` is absent OR
64
+ `enabled:false` (the fleet default), the new block in `evaluate()` is skipped entirely and selection
65
+ is bit-for-bit today's. Asserted by name: `natureRouting UNSET ⇒ selection unchanged, onNatureRoutePlan
66
+ NEVER called` (the same options object is passed through untouched).
67
+ - **A1 is untouched.** `clampClaudeCliSwapModel` (A1's always-on degrade/swap clamp, returning the
68
+ `balanced` TIER token) is NOT modified — the new `clampToReserveOnCleanDoor` is a SEPARATE, nature-
69
+ routing-scoped, concrete-id clamp used only inside `resolveRoute`. Touching A1's fn would have changed
70
+ its shipped byte-identical behavior; a test asserts A1 still returns `{model:'balanced'}`.
71
+ - **dryRun cannot mis-route.** In dryRun the resolver only computes + logs; a resolver throw
72
+ (critical-gate fail-closed) is swallowed and recorded, never surfaced to the call path.
73
+ - **Hot path.** The observation block runs on every internal LLM call ONLY when the feature is enabled
74
+ (dev agents). It is a stateless fold over static maps + O(1) door-reachability reads (cached by the
75
+ provider cache); the breadcrumb log is env-gated so a dev agent's hot path stays quiet.
76
+ - **Config.** The seed is dark, existence-checked, idempotent, `enabled` omitted — it can never
77
+ force-dark a dev agent nor clobber an operator's config.
78
+
79
+ ## Second-pass review (Phase 5 — required: touches a "gate" / routing-of-safety-gates)
80
+
81
+ The change routes safety GATES, so it triggers the high-risk second pass. Independent audit focus:
82
+ (a) can the banned Opus-via-CLI route open? — No: the place-3 allowlist clamp is deny-by-default and
83
+ fires on every resolved claude-code bounded/gating position, including a hand-edited chain. (b) can a
84
+ critical gate fail OPEN? — No: the empty-set branch throws a distinct `RouterFailClosedError` for a
85
+ critical gate (never `no-route`, never legacy routing); a unit test asserts both a nature-B gate
86
+ (`MessagingToneGate`) and the nature-A R2-critical gate (`MessageSentinel`) throw. (c) is "off" truly
87
+ inert? — Yes, the byte-identical-when-off test proves it. **Reviewer verdict: Concur with the review.**
88
+ The one residual is that enforcing SELECTION is deferred — but since it is not wired, it cannot
89
+ mis-route; the runtime clamp already makes the future enforcing path safe on its own.