instar 1.3.661 → 1.3.663

package/upgrades/side-effects/b1-b7-detector-migration.md

@@ -0,0 +1,77 @@
+# Side-Effects Review — B1–B7 detector-emits-signal migration (CMT-1793, §Design 8)
+
+**Version / slug:** `b1-b7-detector-migration`
+**Date:** `2026-06-25`
+**Author:** `Instar Agent (echo)`
+**Second-pass reviewer:** `general-purpose reviewer subagent (Phase 5 — touches the outbound tone gate)`
+
+## Summary of the change
+
+Phase 2 of `gate-prompts-judge-by-meaning-not-literal-lists` (§Design 8). Migrates tone-gate rules B1–B7 from in-prompt literal-matching to the deterministic-detector-emits-signal contract: a new pure module `src/core/GateSignalDetectors.ts` runs seven high-precision detectors (cli-command, file-path, config-key, copy-paste-code, api-endpoint, env-var, cron-or-slug) over the candidate and emits a normalized, sanitized `GateSignal` list; `MessagingToneGate.buildPrompt` renders that list inside its OWN per-call random boundary (distinct from the candidate boundary, untrusted-data framed); the B1–B7 prompt rules are reworded to judge each detected artifact IN CONTEXT (shown-to-act-on → block; mentioned-in-passing → pass) instead of self-scanning; `RULE_CLASSES` B1–B7 flip `deterministic-detection`→`signal-driven`; `PHASE2_MIGRATION_DEBT` is drained to empty; the ratchet (`tests/unit/gate-prompts-judge-by-meaning.test.ts`) inverts to assert the migration landed (B1–B7 signal-driven, allowlist empty, no rule still `deterministic-detection`).
+
+## Decision-point inventory
+
+- `MessagingToneGate` B1–B7 block/allow — **modify** — the JUDGMENT moves from in-prompt string-match to LLM-judges-a-deterministic-signal-in-context (Signal-vs-Authority applied to the gate's own prompt).
+- `GateSignalDetectors.detectGateSignals` — **add** — pure signal producer; NEVER blocks (signal, not authority).
+
+## 1. Over-block
+
+A detector firing only ADDS a signal the LLM then judges; it is not itself a block. The detectors are high-precision (anchored to artifact shapes, not loose prose — e.g. B1 requires a real CLI leader, B3 requires 3+ dotted segments) so prose like "I'll run the migration" or "and/or" does not fire (unit-tested). Residual over-block risk: a candidate that legitimately quotes a path/command while explaining — but the prompt explicitly instructs pass for mentioned/in-passing artifacts, and B1–B7 severity is not the false-negative-favoring class B15 is. Net over-block is LOWER than the prior in-prompt literal-match (which had no contextual carve-out beyond prose hints).
+
+## 2. Under-block
+
+A detector that misses an artifact form yields no signal → that rule falls back to no-block for that candidate (the LLM is told not to self-scan). This is the deliberate trade of §Design 8: deterministic detection is auditable + improvable, vs an LLM literal-scan that was itself brittle. New artifact forms are added to the detector (one place), not re-litigated in the prompt. The detectors are intentionally conservative; a missed exotic form is the safe direction here (B1–B7 are leak-hygiene, not safety-critical gates — the deterministic safety FLOORS, dangerous-command-guard etc., are unchanged).
+
+## 3. Level-of-abstraction fit
+
+Correct. This is the §Design 8 contract realized: pattern-matching is the deterministic layer's job (the detector module), fed to the LLM as a signal; the LLM does the contextual judgment. It mirrors the EXISTING B8/B9/B12/B20 signal-driven pattern (and the pre-existing `signals.filePath` anchor for B2, which now corroborates the unified signal). The detectors live in their own module (like `JargonDetector.ts`), pure + independently testable.
+
+## 4. Signal vs authority compliance
+
+- [x] No — the new code produces a SIGNAL consumed by the existing smart gate (the LLM). `detectGateSignals` has no block authority; the `MessagingToneGate` LLM remains the single authority. This is the textbook Signal-vs-Authority shape and is exactly what the new "An LLM Gate Must Not String-Match" standard mandates. The migration REMOVES brittle in-prompt literal-gating.
+
+## 5. Interactions
+
+- **Shadowing:** the new ARTIFACT-SIGNALS prompt section is additive; it sits beside the existing UPSTREAM SIGNALS section. The pre-existing `signals.filePath` B2 anchor still renders — both now corroborate B2 (no conflict; both point the same way).
+- **Double-fire:** none — the detector runs once per `buildPrompt`; the gate makes one decision.
+- **Races:** none — pure synchronous function on the candidate string; no shared state.
+- **Boundary safety:** the signal list has its OWN per-call random boundary; `normalizedValue` is length-clamped + JSON-encoded inside it and framed as untrusted data, so an attacker-derived "path" cannot break the envelope (unit-tested clamping).
+
+## 6. External surfaces
+
+- Prompt content changes (B1–B7 rules reworded; a new signals section). No new HTTP route, config key, or persistent state. The verdict is still channel-independent (computed pre-adapter from text). No operator-facing action added → Mobile-Complete N/A.
+
+## 6b. Operator-surface quality
+
+No operator surface touched (no dashboard/approval/grant file). Not applicable.
+
+## 7. Multi-machine posture
+
+**REPLICATED / uniform by construction.** The detectors + prompt are rebuilt from source each review; the module is stateless; the ratchet ships compiled. Identical across machines, converging as each updates. No machine-local state affects the verdict. No user-facing notice, no durable state, no generated URL.
+
+## 8. Rollback cost
+
+Pure code change. Back-out = `git revert` the commit — B1–B7 return to the prior in-prompt literal-match (the reverted ratchet assertions go with it). No data migration, no persistent state, no user-visible regression window.
+
+## Tracked sub-scope (no orphan deferrals)
+
+The spec named two SECONDARY sub-scopes for CMT-1793 as "folded in OR a noted sub-gap": pi-cli gate-awareness shadow note, and richer agent-state signals (context-window % + turn/action-count plumbing for B15). The substantive B1–B7 migration ships here; the two secondary items are tracked under **CMT-1800** (a real, open commitment), not dropped. <!-- tracked: CMT-1800 -->
+
+## Conclusion
+
+The review produced no design changes. The migration realizes §Design 8 exactly, removes the last in-prompt literal-gating (B1–B7), and is Signal-vs-Authority + No-Silent-Degradation compliant. Clear to ship pending second-pass concurrence.
+
+## Second-pass review (if required)
+
+**Reviewer:** general-purpose reviewer subagent
+**Independent read of the artifact: concur (with one non-blocking concern, now ADDRESSED)**
+
+The reviewer independently verified and PASSED: Signal-vs-Authority (the detector holds zero block authority; pure signal producer; removing in-prompt literal-gating), the §Design 8 envelope/clamping security (confirmed empirically that an attacker-derived `normalizedValue` containing `<<<SIG_BOUNDARY_…>>>`/newlines/injected instructions is neutralized — `JSON.stringify` collapses it to one quoted line and the real boundary is an unpredictable per-call `randomBytes(8)` token), the ratchet inversion (correctly inverted, not weakened — B1–B7 are now genuinely scanned for necessary-literal-gate constructions; the standard is ENFORCED on them, not exempted), and ReDoS (all seven regexes linear-time on 50KB adversarial inputs).
+
+**Concern (non-blocking, now fixed):** B3/B7 detector precision — `config-key` fired on hostnames (`www.example.com`) and `cron-or-slug` on plain hyphenated English (`well-thought-out`, `state-of-the-art`). Signal-only, so it could never block, but it diluted the signal list. FIXED in this PR: B3 drops hostname-shaped matches (leading `www.` or a known-TLD tail without a camelCase segment); B7's lowercase-kebab branch now requires a DIGIT (so adjectival hyphenation and digit-less slugs don't fire — a digit-less internal slug leaked to a user is caught by the separate B20 internal-id-leak signal). Prose-negative tests added (`well-thought-out`, `state-of-the-art`, `www.example.com`, `docs.instar.sh`) to lock the precision. 19 detector tests + 8 ratchet tests green after the tuning.
+
+## Evidence pointers
+
+- `tests/unit/GateSignalDetectors.test.ts` — 17 tests: detection precision both-sides (artifacts fire; prose doesn't) + security clamping (closed kind enum, confidence [0,1], bounded spans, length-clamped normalizedValue).
+- `tests/unit/gate-prompts-judge-by-meaning.test.ts` — the ratchet, inverted: B1–B7 signal-driven, `PHASE2_MIGRATION_DEBT` empty, no `deterministic-detection` rule remains, B1–B7 prompt blocks reference their signal.
+- Regression: b15/b16/b17 integration + MessagingToneGate + spawn-cap + post-update + attention + feature-delivery — all green (123 + 63 tests across the runs). `npm run build` + `tsc --noEmit` clean.

package/upgrades/side-effects/gate-prompts-judge-by-meaning.md

@@ -0,0 +1,48 @@
+# Side-Effects Review — gate-prompts-judge-by-meaning-not-literal-lists
+
+**Change:** Make the outbound tone gate's behavioral rules (B15–B18) judge by MEANING, not by literal-phrase presence; author the new constitution standard "Intelligent Prompts — An LLM Gate Must Not String-Match" with a CI ratchet; feed a deterministic agent-state (session-clock) signal into B15; and flip every gating no-verdict path to FAIL-CLOSED (invalid-rule, JSON-parse, provider-exhaustion, route-budget timeout) with an operator kill-switch.
+
+**Spec:** docs/specs/gate-prompts-judge-by-meaning-not-literal-lists.md (review-convergence + approved:true). ELI16 + convergence report shipped.
+
+**Signal-vs-Authority (Phase 1):** This IS a decision point (the outbound gate). The change is the *purest application* of Signal-vs-Authority — it removes brittle string-matching authority from the LLM's own prompt and routes deterministic detection through signals (B8/B9/B12 pattern), letting the full-context mind judge. No brittle check gains blocking authority; the CI ratchet is a signal-only developer-loop guard (fails CI, makes no runtime decision).
+
+## The 8 questions
+
+1. **Over-block** — Meaning-based judgment could over-block honest status disclosure ("at 95% context, continuing"). MITIGATED structurally: §Design 1 step 1 makes "a stop is ACTUALLY proposed" the hard precondition (mention ≠ stop → PASS), and step 4b scopes the freshness tell to agent-fatigue framing (external-dependency timing passes). Both have explicit PASS fixtures. An over-block is also self-correcting (the agent gets the reason + rephrases).
+2. **Under-block** — A sufficiently sophisticated semantic rewrite of a *literal-gate construction* could evade the CI ratchet (documented honest limit; human review of judgment-prompt changes still required). At runtime, the meaning-based gate is strictly *harder* to evade than the old literal list. B1–B7 still literal-match in-prompt (tracked debt CMT-1793).
+3. **Level-of-abstraction fit** — Correct layer: the deterministic detectors (dangerous-command/free-text floors, jargon/junk signals, the new session-clock signal) live OUTSIDE the prompt and feed it; the LLM judges. This is the architecture the new standard mandates. B1–B7's in-prompt matching is the one not-yet-migrated spot (CMT-1793).
+4. **Signal vs authority compliance** — Compliant by construction (see Phase 1). The ratchet enforces it forward.
+5. **Interactions** — The fail-closed change interacts with: the capacity-shed sibling path (now consistent — both hold), the route-budget fail-open (now opt-in fail-closed via the kill-switch), and the `no-silent-llm-fallback` ratchet (gate already satisfies it via `gating:true`; no REVIEWED_ADVISORY edit — asserted by a test). The structured-intermediate is back-compat: absent block ⇒ legacy path unchanged. The agentState signal is fail-open-skip on error (never blocks an outbound).
+6. **External surfaces** — The gate's blocking verdict is user-visible. A new config key `messaging.toneGate.failClosedOnExhaustion` (default true). A new CLAUDE.md section (+ framework-shadow mirror to AGENTS.md/GEMINI.md). The new STANDARDS-REGISTRY standard. No new HTTP route. The fail-closed direction means a *total LLM outage* holds `/telegram/reply` outbound (held-not-lost, retried) — system/lifeline/post-update sends use separate routes and are unaffected.
+7. **Multi-machine posture** — **REPLICATED.** The prompt is rebuilt from source on every review; the provider is stateless; the ratchet + standard ship compiled. The agentState signal reads local topic-bound clock on the serving machine (where the gate already runs, pre-adapter). The CLAUDE.md/shadow awareness note rides normal per-machine migration (converges as each machine updates). No machine-local state affects the verdict. Uniform by construction.
+8. **Rollback cost** — Prompt revert is a one-file `git revert`. The fail-closed flips are independently revertable per return path, AND the provider-exhaustion + slow-timeout paths carry a live operator kill-switch (`failClosedOnExhaustion:false`) — revertable without a deploy (mobile-first). The CI ratchet is removable by deleting the test.
+
+## Tracked follow-ups (no orphan deferrals)
+- B1–B7 detect-outside-feed-signal migration — CMT-1793 (frozen PHASE2_MIGRATION_DEBT allowlist + stale-allowlist test).
+- Availability-aware kind-routing + codebase-wide gating-fail-open sweep — CMT-1794 (DEFERRED_REFINEMENT, non-placeholder-commitment-pinned).
+- Dedicated production-init E2E ("gate alive") + the live Playwright-Telegram channel proof — driven by the orchestrating session before merge (the integration tier already exercises the real route + real gate end-to-end).
+
+## Phase 5 — Second-pass review (high-risk: gate/sentinel/outbound-block)
+
+Independent re-audit (performed inline by the build fork, which cannot spawn a subagent):
+
+- **Fail-closed availability:** Verified system/lifeline/`post-update` sends use SEPARATE routes (`routes.ts:10430`), so failing closed at the `/telegram/reply` seam only holds conversational + automated traffic during a *total* outage — held-not-lost, kill-switch-revertable. Sound.
+- **Structured-derivation safety:** The derivation only ever makes the verdict MORE conservative (derives a BLOCK from the model's own structured reasoning); it never flips a block to a pass. A contradictory structured verdict re-prompts then holds. Sound.
+- **Back-compat:** A response with no `structured` block follows the exact legacy path (verified by an explicit test). Existing callers (`new MessagingToneGate(provider)`) compile unchanged (config optional). Sound.
+- **Ratchet brittleness:** The ratchet keys off the machine-readable RULE_CLASSES registry (not prose), checks both boundary directions, fails closed on an unclassified rule, and has a negative test proving it catches a reworded construction. Its honest limit is documented in the standard + the test header. Sound.
+- **Concern raised + resolved:** the re-prompt doubles provider calls on a discipline-failure branch — bounded to ONE extra call, only on the rare wanted-block-mis-cited / unparseable / contradictory branch, inside the existing route budget, and the error path collapses to hold (no back-door into the deferred fail-open). Acceptable.
+
+**Verdict: Concur with the review.** No blocking concern. The change strengthens the gate and is compliant with Signal-vs-Authority and No Silent Degradation.
+
+---
+
+## Follow-up — CI failure fixes (post-merge-prep, 2026-06-25)
+
+The first implementation commit (cb2ac82dd) left 4 CI checks red. Root causes + fixes (all verified green locally: the 4 named test files = 123 tests passing; tone-gate regression sweep b15/b16/b17 + attention + MessagingToneGate = 42 passing):
+
+1. **`post-update-gate-budget-route:101` (422 vs 200) — a real design correction.** The §6 review above ASSUMED `/telegram/post-update` "uses separate routes and is unaffected" by the slow-timeout fail-closed flip. **That assumption was wrong:** `/telegram/post-update` (routes.ts ~10467) calls `checkOutboundMessage` → `evaluateOutbound` → the tone gate, exactly like `/telegram/reply`. So the global slow-timeout fail-closed flip ALSO held post-update (an automated, fixed-template "I'm back up"/release channel), which must stay available. Fix: the slow-review timeout fail-closed is now **per-call**, defaulting CLOSED (the safe direction; conversational `/telegram/reply` keeps failing closed unchanged), with `/telegram/post-update` passing `failClosedOnBudgetTimeout:false` to fail OPEN on the *no-verdict-in-time* path only. A fast real BLOCK still holds post-update; only the slow path delivers. Default-closed (not the originally-sketched default-open opt-in) so a NEW conversational caller is safe-by-construction and cannot silently regress to fail-open. Still also gated by the global `failClosedOnExhaustion` kill-switch.
+2. **`feature-delivery-completeness:327` — parity tracking.** The new `### Outbound Message Gate` CLAUDE.md section (added to templates.ts + the migrator) was not yet listed in the test's tracked `featureSections`, so the migrator-vs-template parity check failed. Fix: add the section marker to the tracked list (the migrator already emits it in both `migrateClaudeMd` and the framework-shadow markers).
+3. **`spawn-cap-fail-closed-gates:93` (test 3b) — obsolete contract.** That test (forkbomb-prevention spec) asserted the tone gate fails OPEN on a generic provider error. §Design 6 of THIS spec deliberately flipped the provider-exhaustion path to fail CLOSED. Fix: updated 3b to assert `pass:false, failedClosed:true` (with a comment recording the cross-spec contract change). The sentinel/input-guard generic-error paths (1b/2b) remain fail-open — only the outbound tone gate changed.
+4. **`provider-fallback-default-policy-lifecycle:151` — same obsolete contract.** Asserted the tone gate fails OPEN when the provider swap-chain is exhausted. Updated to assert fail CLOSED (`failedClosed:true, pass:false`), preserving the test's actual wiring-integrity intent (the router re-throws to the caller; the caller's now-fail-closed policy decides).
+
+**Side-effects of the follow-up:** only `/telegram/post-update`'s no-verdict-in-time outcome changes (hold→deliver); every other caller's behavior is unchanged (default-closed). No new config, route, or persistent state. Rollback = revert the routes.ts hunk; the global kill-switch remains.