@event4u/agent-config 8.9.0 → 8.11.0

package/docs/decisions/ADR-118-loop-engineering-boundaries.md

@@ -0,0 +1,162 @@
+---
+adr: 118
+status: accepted
+date: 2026-07-10
+decision: loop-engineering-boundaries
+supersedes: —
+superseded_by: —
+phase: loop-engineering
+type: structural
+---
+
+# ADR-118 — Loop-engineering boundaries: one closure, four rejections, three deferrals, zero new loop surfaces
+
+## Status
+
+**Accepted** · 2026-07-10. Resolved by web research (five external articles + an
+independent sweep) ground-truthed against the repo's loop inventory, plus an
+AI-council debate (anthropic/claude-sonnet-4-5 + openai/gpt-4o, 2 rounds,
+2026-07-10, converged). Executed by `road-to-loop-engineering`.
+
+## Context
+
+The June-2026 "loop engineering" wave (design the system that prompts the
+agent: verification signal, budget shape, maker/checker topology, on-disk
+state) is mostly a rebranding of ReAct + eval-driven development + bounded
+autonomous agents. Its two durable contributions are the design vocabulary and
+a precise failure-mode taxonomy: verification collapse ("done" is a claim, not
+proof), comprehension debt, cognitive surrender, reward hacking (measured at
+~15% of SE-task runs in one external evaluation), cost blowup on stuck retry
+loops, context rot on long runs.
+
+This package already implements the serious end of that canon: bounded
+verify-fix loops (`verify-repair-loop` max_attempts=3, the N=3 validation
+budget, context-hygiene 15/25 aborts), maker/checker separation (`judge-*`
+cluster, `verify-budget`), human-gated self-improvement (5-stage pipeline,
+≥2 independent evidence), and measurement infrastructure (bench:ab, trigger
+evals, golden sets, orchestration telemetry, local analytics).
+
+An inventory identified 8 places where a measure exists but no adjust-step is
+wired. The question this ADR settles: which of those loops to close, and which
+loop-shaped proposals to reject so they are not relitigated per hype cycle.
+
+Constraint frame: single maintainer, token-frugality canon, no runtime daemon
+(ADR-088 / no-runtime-boundary), human gates on anything irreversible. Under
+these constraints, when human review + `git revert` is cheaper than designing,
+maintaining, and debugging a closed loop, the loop is net-negative
+infrastructure — false positives consume the binding resource (maintainer
+attention), not compute.
+
+## Decision
+
+### 1. Automation threshold (the core principle)
+
+Automate a measure→adjust loop **only** when all three hold:
+
+1. the metric is a **direct** measure of the failure mode (not a proxy or a
+   correlation),
+2. the expected false-positive rate is low (order <5%), and
+3. human judgment adds no unique information to the adjust decision.
+
+"A number exists" is not "the number is trustworthy" (metric-existence
+fallacy). Reversibility of the adjust-step alone never justifies automation:
+investigating one false-positive automated action costs 20–45 minutes of
+maintainer time against a 10-second revert.
+
+### 2. Disposition of the 8 open loops
+
+| # | Open loop | Disposition |
+|---|---|---|
+| 1 | Orchestration telemetry → `subagents.auto` demotion | **Manual by decision.** token_delta spikes for legitimate reasons; demotion needs qualitative judgment of the failure mode. The demotion path in `orchestration-benchmark-gate.md` stays a human one-line edit. |
+| 2 | Artifact-engagement counts → skill pruning | **Never automated.** Selection bias both ways (low engagement ≠ low value: security / error-recovery / niche skills; high engagement ≠ high value). Engagement stays ONE input under `evidence-based-pruning`'s ≥2-independent-evidence floor; the pruning decision is always human. |
+| 3 | Trigger-eval pass rate on cadence | **Closed** — the only closure. Structure/freshness/presence gates already ran in `task ci`; this ADR's roadmap adds a rotating live pass-rate measurement to the existing weekly canary (see §4). Direct metric, low false-positive, adjust-step = surfaced report. |
+| 4 | bench-drift → blocking gate | **Stays advisory.** High false-positive proxy (timing variance, intentional behaviour changes); the manual `bench:baseline-ready` flip remains the enforcement gate. |
+| 5 | Golden-set coverage ratchet (14/91 rules) | **Deferred; growth stays opportunistic.** A hard per-rule ratchet pressures toward mechanically-testable rules over useful rules; most uncovered rules are judgment-shaped. `road-to-golden-set-coverage` remains the deliberate path. |
+| 6 | Live-app verdict source | **Deferred, unchanged** — owned by `road-to-live-app-verdict`; the package ships config, not apps. |
+| 7 | Measured rule-adherence signal | **Rejected.** Annotated-corpus + threshold-tuning build with ongoing maintenance for marginally earlier restatement; ADR-054's coarse heuristics are intentional. |
+| 8 | Success-signal re-evaluation at eval date | **Not automated.** Retroactive evaluation of sunk promotion decisions has low marginal information; active use surfaces bad promotions. The field stays authored documentation. |
+
+### 3. Five written rejections (loop anti-patterns for this package)
+
+1. **Open-ended hill-climbing on the config.** No loop may "optimize"
+   rules/skills via iterated mutation + metric without a human checkpoint per
+   iteration; the bench:ab soak + human decision is the floor for config
+   change evaluation.
+2. **Autonomous config self-editing.** The self-improvement pipeline stays
+   human-gated at every stage; no loop commits changes to `src/skills/` or
+   `src/rules/` without human approval. (Restates the existing pipeline
+   contract as a loop boundary.)
+3. **Unattended multi-hour runs without checkpoints.** Loops must remain
+   reviewable/haltable on the maintainer's schedule; the existing halt
+   conditions and N=3 budgets are the ceiling, not a starting point.
+4. **Metric-optimizing loops without golden-set anchoring.** No loop whose
+   success criterion is "metric goes up" without a golden-set eval anchoring
+   what "good" means (Goodhart guard).
+5. **Cross-session loop state in external services.** No vector DBs, job
+   queues, or external orchestrators as loop state (restates ADR-088 in loop
+   context): state lives in git-tracked files + the conversation.
+
+Also rejected, same reasoning tier: a **loop-design contract/checklist
+artifact** (premature formalization — the caps/judges/state patterns are
+embedded in the existing loop surfaces and enforced by existing rules) and
+**consumer-facing loop templates** (`/loop:design` et al. — scope creep;
+agents answer loop-design questions in conversation using the package's
+existing vocabulary). **No new loop surfaces**: no run-until-condition goal
+command, no fresh-context re-loop mode for roadmap processing (bounded-scope
+by design), no separate nightly self-check workflow (folded into the existing
+weekly canary).
+
+### 4. The one closure — periodic live trigger-eval pass rate
+
+The existing weekly cross-model canary's secrets-gated live tier gains a
+deterministic rotation over the skills that carry `evals/triggers.json`
+(ISO-week keyed, ~5 suites/week), enforcing the per-domain precision/recall
+floors from `eval:record`. A floor breach fails the **scheduled** job — the
+failure is the maintainer notification; PRs are never blocked by live results.
+The local interactive confirmation gate on `skill_trigger_eval` is untouched;
+CI authorization derives exclusively from workflow-provided key secrets, and
+the no-secrets path stays a logged no-op.
+
+## Revisit-if
+
+- **Any rejection above:** reopen only on evidence that closing the loop saves
+  >2 h/month maintainer time in a structurally similar single-maintainer
+  project (logged hours, not vibes) — via `decision-revisit-gate`
+  mechanism-match, then council.
+- **The §4 closure:** demote the canary pass-rate job to advisory
+  (non-failing) if breaches are >~10% spurious over 50+ suite-runs.
+- **§2 #5 coverage ratchet:** reconsider if ≥3 real regressions ship in rules
+  that a golden task would have caught (counted, cited incidents).
+
+## Consequences
+
+- Future "should we automate X?" loop proposals test against §1 before any
+  build; the disposition table is the precedent record.
+- The weekly canary becomes the single periodic live-eval surface — additions
+  to periodic self-checking extend it rather than adding workflows.
+- Skill-pruning, default-demotion, and drift-blocking proposals cite this ADR
+  instead of re-deriving the argument.
+
+## Alternatives considered
+
+- **Close more loops (coverage ratchet, engagement pruning, auto-demotion):**
+  rejected — each fails at least one §1 criterion; the council's round-2
+  convergence was explicit that "metric exists + reversible" is insufficient.
+- **Loop-design contract:** rejected as compliance theater for a
+  single-maintainer project; revisit on a second maintainer with real handoff
+  friction.
+- **Do nothing (pure documentation):** rejected — the trigger-eval pass-rate
+  gap is real, direct-metric, and closable inside an existing workflow at
+  near-zero maintenance cost.
+
+## References
+
+- `road-to-loop-engineering` roadmap (execution + provenance; external source
+  links retained encrypted there per `source-confidentiality`).
+- `docs/contracts/evidence-based-pruning.md` — §2 #2 anchor.
+- `src/agent-src/contexts/execution/orchestration-benchmark-gate.md` — §2 #1 anchor.
+- `docs/contracts/measurement-baseline.md` — §2 #4 anchor.
+- ADR-088 (no external runtime), ADR-054 (adherence restate), ADR-106/-109/-117
+  (orchestration gates), `docs/contracts/no-runtime-boundary.md`.
+- Prior honest-nulls this ADR does not reopen: recursive-verification
+  (2026-06-24), enforcement-projection, reminder-injection.

package/docs/decisions/ADR-119-global-knowledge-default-on.md

@@ -0,0 +1,104 @@
+---
+adr: 119
+status: accepted
+date: 2026-07-12
+decision: global-knowledge-default-on
+supersedes: 103
+superseded_by: —
+phase: opt-decision-flips
+type: structural
+---
+
+# ADR-119 — Global knowledge sharing defaults ON (validated bounded-downside flip)
+
+## Status
+
+Accepted (2026-07-12). Supersedes ADR-103.
+
+## Context
+
+ADR-103 (2026-06-16) kept `knowledge.global_sharing.enabled: false` until
+cross-project reuse was measured across ≥ 2 projects. Verified state at
+re-evaluation (2026-07-11): the measurement instrument
+(`src/scripts/_lib/knowledge_global_promote.ts`) shipped, but no sightings
+data, no global store, and no reuse record existed — reuse can only accrue
+while the layer is ON, and ON was withheld pending reuse. The gate could
+never fire by construction: the same self-locking-measurement deadlock
+ADR-117 broke for `subagents.auto` on 2026-07-09.
+
+A two-round AI-council re-evaluation ran on 2026-07-11 (members:
+anthropic/claude-sonnet-4-5, openai/gpt-4o; cost $0.10). Round 1: both
+members stanced **Option A — flip ON now** (deadlock + ADR-117 precedent +
+shipped guards). Round 2 dissent (gpt-4o): the redaction floor was
+unvalidated for cross-project adversarial scenarios — composite inference,
+homoglyph/encoding smuggling, temporal context collapse. The named
+evidence-that-would-change-my-mind: demonstrating the redaction mechanism
+handles those classes. Convergence: **flip WITH validation** — the dissent
+becomes a pre-flip gate, not a blocker.
+
+## Decision
+
+1. **Pre-flip validation executed (2026-07-12).** Adversarial spot-check in
+   `tests/scripts/_lib_knowledge_global_redaction_adversarial.test.ts` +
+   `tests/fixtures/global-knowledge-redaction/` covering the three named
+   classes. Honest result: the zero-width-smuggling probe FAILED on first
+   run — the dissent's predicted gap was real. Closed by an additive
+   hardening in `knowledge_global_redaction.ts` (`hidden_unicode` violation
+   class + strip-then-rescan); homoglyph emails were already caught
+   (Unicode-aware pattern); temporal collapse is bounded by the freshness
+   state machine (fresh → hypothesis → stale). All 20 tests green,
+   including the 11 pre-existing parity tests.
+2. **`knowledge.global_sharing.enabled` defaults to `true`** in the shipped
+   template.
+3. **Narrowest tier default:** `allowed_tiers: [public]` (down from
+   `[public, vendor]`); `vendor` becomes a deliberate opt-in widening.
+   `proprietary` stays hard-coded manual-only in the gate.
+4. **Pre-registered demotion trigger:** any cross-project PII/secret
+   sighting in a shared card — a confirmed identifier that crossed a
+   project boundary despite the gate — auto-reverts the shipped default to
+   `false` and lands an incident note under `agents/settings/contexts/`.
+   This is the ADR-117 demotion-gate pattern applied to knowledge sharing.
+5. **Measurement window:** 60–90 days from release. The reuse data the
+   ADR-103 gate wanted now accrues; at window end the default is either
+   confirmed (reuse observed, no incidents) or reverted (no reuse — the
+   layer is dead weight — or a demotion trigger fired).
+6. **Accepted residual, pinned by test:** k-anonymity over pure
+   quasi-identifier combinations is out of scope for a write-time text
+   gate. The council accepted this on the single-install trust boundary
+   (an actor with filesystem access to the global store already reads the
+   raw projects, which are strictly more sensitive). The boundary is
+   asserted by a dedicated test so any future change to it is a deliberate
+   decision, not drift.
+
+## Consequences
+
+- Cross-project knowledge cards flow by default on the narrowest tier,
+  through the unchanged suggest-then-confirm promotion flow
+  (`auto_promote_threshold` gates a suggestion, never a silent write) and
+  the hardened redaction gate (halt-and-surface, never silent-rewrite).
+- Single-key revert: `knowledge.global_sharing.enabled: false` fully
+  no-ops the layer; v1 project-local cards are unaffected either way.
+- No new infrastructure: file-first store, no vectors, no services — the
+  ADR-094 Layer-2 sunset is untouched.
+- The hidden-unicode hardening also protects every other consumer of
+  `redaction_scan` (the Phase-4 CI linter over committed global cards).
+
+## Alternatives considered
+
+- **Keep OFF + opt-in campaign (Option B):** rejected — self-selection
+  bias, slow, and it retains the deadlock for everyone who does not
+  opt in.
+- **Maintainer-org pilot first (Option C):** rejected as the primary path —
+  the council judged the guarded flip's downside bounded enough that a
+  pilot's extra latency buys little; the demotion trigger plus the
+  measurement window deliver the same safety net in production shape.
+
+## References
+
+- ADR-103 (superseded), ADR-117 (deadlock-flip precedent), ADR-100/-094
+  (store shape + Layer-2 sunset).
+- Council session 2026-07-11 (claude-sonnet-4-5 + gpt-4o, 2 rounds,
+  convergence inlined above per `no-roadmap-references`).
+- `tests/scripts/_lib_knowledge_global_redaction_adversarial.test.ts` —
+  the validation gate.
+- `agents/roadmaps/road-to-opt-decision-flips.md` Phase 1 (execution).

package/docs/decisions/ADR-200-python-to-typescript-migration.md

@@ -5,12 +5,22 @@ date: 2026-06-11
 decision: python-to-typescript-migration
 supersedes: —
 superseded_by: —
-phase: py2ts · migration-infrastructure
+phase: py2ts · migration-infrastructure (historical record — migration landed on main)
 type: structural
 ---
 
 # ADR-200 — Full Python → TypeScript migration: execution model, branch strategy, dispatcher, parity gates
 
+> **Number retained by decision (2026-07-12):** the provisional-number
+> clause ("final number assigned at the python2ts → main merge") has
+> fired — the migration is complete on `main` (0 tracked `.py` under
+> `src/`). Renumbering was evaluated and declined: 521 files reference
+> "ADR-200" as a stable identifier; a rename would be a 521-file
+> mechanical churn (projections, condensation hashes, docs) for zero
+> behavioral gain. ADR-200 stays the permanent number; the record is
+> historical (migration architecture, landed).
+
+
 > Provisional number 200 (reserved, out-of-band). main ADRs grow sequentially; the migration ADR is renumbered to a high reserved slot to stop recurring collisions on the long-lived `python2ts` branch. Final number is assigned at the `python2ts`→`main` merge.
 
 ## Status

package/docs/decisions/ADR-rule-kernel-and-router.md

@@ -1,3 +1,14 @@
+---
+adr: —
+status: accepted
+date: 2026-05-06
+decision: rule-kernel-and-router-roadmap-closure
+supersedes: —
+superseded_by: —
+phase: road-to-kernel-and-router (closure record)
+type: structural
+---
+
 # ADR — Rule Kernel and Router (Roadmap Closure)
 
 - **Status:** Accepted (2026-05-06)

package/docs/decisions/INDEX.md

@@ -5,8 +5,8 @@ _Auto-generated by `scripts/adr/regenerate_index.py`. Do not edit._
 | # | Title | Status | Date | Supersedes |
 |---|---|---|---|---|
 | [ADR-001](ADR-001-kernel-swap-deferred.md) | Kernel Swap Deferred | accepted | 2026-05-06 | — |
-| [ADR-002](ADR-002-kernel-bucket-overrides.md) | Kernel Bucket Overrides | — | — | — |
-| [ADR-003](ADR-003-flat-cluster-subs-and-colon-syntax.md) | Flat Cluster Subs And Colon Syntax | — | — | — |
+| [ADR-002](ADR-002-kernel-bucket-overrides.md) | Kernel Bucket Overrides | accepted | 2026-05-06 | — |
+| [ADR-003](ADR-003-flat-cluster-subs-and-colon-syntax.md) | Flat Cluster Subs And Colon Syntax | accepted | 2026-05-07 | — |
 | [ADR-004](ADR-004-rule-governance-pruning.md) | Rule Governance Pruning | accepted | 2026-05-08 | — |
 | [ADR-005](ADR-005-subagent-worktrees.md) | Subagent Worktrees No Auto Merge | accepted | 2026-05-09 | — |
 | [ADR-006](ADR-006-skill-tools-python-pilot.md) | Skill Tools Python Pilot Pass | accepted | 2026-05-09 | — |
@@ -31,7 +31,7 @@ _Auto-generated by `scripts/adr/regenerate_index.py`. Do not edit._
 | [ADR-025](ADR-025-workspace-chrome.md) | Workspace Chrome | accepted | 2026-05-24 | — |
 | [ADR-026](ADR-026-explain-mode-translation.md) | Explain Mode Translation | accepted | 2026-05-24 | — |
 | [ADR-027](ADR-027-changelog-machine-vs-manual.md) | Changelog Machine Vs Manual | accepted | 2026-05-25 | — |
-| [ADR-028](ADR-028-root-layout.md) | Root Layout | accepted | 2026-05-25 | — |
+| [ADR-028](ADR-028-root-layout.md) | Root Layout | superseded | 2026-05-25 | — |
 | [ADR-029](ADR-029-multi-workspace-deferred.md) | Multi Workspace Deferred | accepted | 2026-05-25 | — |
 | [ADR-030](ADR-030-claude-code-command-projection.md) | Claude Code Command Projection | accepted | 2026-05-28 | — |
 | [ADR-031](ADR-031-validation-severity-tiers-and-projection-roundtrip.md) | Validation Severity Tiers And Projection Roundtrip | accepted | 2026-05-29 | — |
@@ -41,7 +41,7 @@ _Auto-generated by `scripts/adr/regenerate_index.py`. Do not edit._
 | [ADR-035](ADR-035-model-capability-tiers.md) | Model Capability Tiers | accepted | 2026-05-30 | 034 |
 | [ADR-036](ADR-036-global-install-browser-wizard-handoff.md) | Global Install Browser Wizard Handoff | accepted | 2026-06-01 | — |
 | [ADR-037](ADR-037-cost-profile-untangle.md) | Cost Profile Untangle | accepted | 2026-06-01 | — |
-| [ADR-038](ADR-038-canonical-settings-path.md) | Canonical Settings Path | — | — | — |
+| [ADR-038](ADR-038-canonical-settings-path.md) | Canonical Settings Path | accepted | 2026-06-01 | — |
 | [ADR-039](ADR-039-claude-skills-untracked.md) | Claude Skills Untracked | accepted | 2026-06-01 | — |
 | [ADR-040](ADR-040-execution-model-projection-time-filtering.md) | Execution Model Projection Time Filtering | accepted | 2026-06-02 | — |
 | [ADR-041](ADR-041-controlled-command-verbs.md) | Controlled Command Verbs | accepted | 2026-06-03 | — |
@@ -96,7 +96,7 @@ _Auto-generated by `scripts/adr/regenerate_index.py`. Do not edit._
 | [ADR-090](ADR-090-visibility-command-frontmatter-field.md) | Visibility Command Frontmatter Field | accepted | 2026-06-13 | — |
 | [ADR-091](ADR-091-split-meta-capability-packs.md) | Split Meta Capability Packs | accepted | 2026-06-13 | — |
 | [ADR-092](ADR-092-defer-command-tier-alias-removal.md) | Defer Command Tier Alias Removal | accepted | 2026-06-13 | — |
-| [ADR-093](ADR-093-ai-council-config-user-global.md) | Ai Council Config User Global | accepted | 2026-06-13 | — |
+| [ADR-093](ADR-093-ai-council-config-user-global.md) | Ai Council Config User Global | superseded | 2026-06-13 | — |
 | [ADR-094](ADR-094-agent-memory-layer-removal.md) | Agent Memory Layer Removal | accepted | 2026-06-14 | — |
 | [ADR-095](ADR-095-workspace-boundary-contract.md) | Workspace Boundary Contract | accepted | 2026-06-14 | — |
 | [ADR-096](ADR-096-analysis-workbench.md) | Analysis Workbench | accepted | 2026-06-15 | — |
@@ -106,7 +106,7 @@ _Auto-generated by `scripts/adr/regenerate_index.py`. Do not edit._
 | [ADR-100](ADR-100-global-knowledge-card-sharing.md) | Global Knowledge Card Sharing | accepted | 2026-06-15 | ADR-098 |
 | [ADR-101](ADR-101-ticket-bundle-emission.md) | Ticket Bundle Emission | proposed | 2026-06-15 | — |
 | [ADR-102](ADR-102-ticket-handoff-paste-and-mcp.md) | Ticket Handoff Paste And Mcp | accepted | 2026-06-16 | — |
-| [ADR-103](ADR-103-global-knowledge-default-off-until-measured.md) | Global Knowledge Default Off Until Measured | accepted | 2026-06-16 | — |
+| [ADR-103](ADR-103-global-knowledge-default-off-until-measured.md) | Global Knowledge Default Off Until Measured | superseded | 2026-06-16 | — |
 | [ADR-104](ADR-104-ai-council-config-global-only.md) | Ai Council Config Global Only | accepted | 2026-06-22 | 093 |
 | [ADR-105](ADR-105-automatic-subagent-orchestration.md) | Automatic Subagent Orchestration | accepted | 2026-06-23 | — |
 | [ADR-106](ADR-106-recursive-verification-benchmark-gate.md) | Recursive Verification Benchmark Gate | accepted | 2026-06-23 | — |
@@ -117,14 +117,16 @@ _Auto-generated by `scripts/adr/regenerate_index.py`. Do not edit._
 | [ADR-111](ADR-111-canonical-glama-listing.md) | Canonical Glama Listing | accepted | 2026-07-07 | — |
 | [ADR-112](ADR-112-stdio-lite-stays-read-only.md) | Stdio Lite Stays Read Only | accepted | 2026-07-07 | — |
 | [ADR-113](ADR-113-ci-native-release-label-trigger.md) | Ci Native Release Label Trigger | accepted | 2026-07-08 | — |
-| [ADR-114](ADR-114-commit-policy-iron-law-override.md) | Commit Policy Iron Law Override | — | — | — |
+| [ADR-114](ADR-114-commit-policy-iron-law-override.md) | Commit Policy Iron Law Override | accepted | 2026-07-08 | — |
 | [ADR-115](ADR-115-command-cluster-consolidation-phase-4.md) | Command Cluster Consolidation Phase 4 | accepted | 2026-07-08 | — |
 | [ADR-116](ADR-116-memory-tripwire-activation-path.md) | Memory Tripwire Activation Path | accepted | 2026-07-08 | — |
-| [ADR-117](ADR-117-subagents-auto-default-on.md) | Subagents Auto Default On | accepted | 2026-07-09 | 105 |
+| [ADR-117](ADR-117-subagents-auto-default-on.md) | Subagents Auto Default On | accepted | 2026-07-09 | — |
+| [ADR-118](ADR-118-loop-engineering-boundaries.md) | Loop Engineering Boundaries | accepted | 2026-07-10 | — |
+| [ADR-119](ADR-119-global-knowledge-default-on.md) | Global Knowledge Default On | accepted | 2026-07-12 | 103 |
 | [ADR-200](ADR-200-python-to-typescript-migration.md) | Python To Typescript Migration | accepted | 2026-06-11 | — |
 
 ## Unnumbered (legacy)
 
 | # | Title | Status | Date | Supersedes |
 |---|---|---|---|---|
-| [ADR-rule-kernel-and-router](ADR-rule-kernel-and-router.md) | — | — | — | — |
+| [ADR-rule-kernel-and-router](ADR-rule-kernel-and-router.md) | Rule Kernel And Router Roadmap Closure | accepted | 2026-05-06 | — |

package/docs/design/cross-model-parity-eval.md

@@ -0,0 +1,82 @@
+# Cross-model parity eval — re-scoped design
+
+> Design artifact for `road-to-opt-measurement-unblock` Phase 3 step 1. It
+> re-scopes the archive's most-deferred item (deferred across 4+ roadmaps) to the
+> **smallest capability** that produces a real cross-vendor parity signal, so the
+> build-vs-defer decision (step 2) rests on a concrete design, not the generic
+> "the harness doesn't exist". Writing this design is autonomous; **building and
+> running it is a maintainer cost decision** (step 2/3), disclosed and gated.
+
+## What changed since the last deferral
+
+| Original blocker | Status now |
+|---|---|
+| No delegable-task corpus | **RESOLVED** — `internal/bench/orchestration/corpus/` exists (`orch-*` orchestration tasks + `pv-*` production-validator tasks) |
+| Harness cannot execute model-emitted subagent calls | **STILL TRUE** — do not assume it away; re-scope around it |
+
+The corpus blocker no longer holds, so the item is re-openable — but only if we
+stop trying to build the thing that is still blocked (a full in-host harness that
+runs arbitrary model-emitted subagent calls).
+
+## The re-scope: council-transport execution, not an in-host harness
+
+**Don't** build a harness that executes model-emitted subagent calls end-to-end
+(the still-open blocker). **Do** reuse the capability that already exists: the AI
+council transport dispatches the *same* prompt to *multiple vendors* through
+`consult(members: ExternalAIClient[], …)` in
+[`src/scripts/ai_council/orchestrator.ts`](../../src/scripts/ai_council/orchestrator.ts).
+
+The smallest capability that yields a cross-model parity signal:
+
+1. **Task adapter.** Render each corpus task (`internal/bench/orchestration/corpus/*.md`)
+   into a single self-contained council prompt: the task + its rubric, asking the
+   model to *produce the findings it would surface* (not to actually spawn
+   subagents). This sidesteps the subagent-execution blocker — we measure the
+   model's **finding output on the delegable task**, not its ability to drive a
+   live subagent runtime.
+2. **≥ 2 vendors, same corpus.** Dispatch the identical prompt set to two distinct
+   vendors via the existing `ExternalAIClient` array — no new transport, no new
+   auth surface beyond what the council already has.
+3. **Per-host finding-count distribution.** For each (host × task), record the
+   count and the findings; aggregate to a per-host distribution over the corpus.
+4. **Blind judged quality (optional, gated).** If a quality signal beyond raw
+   counts is wanted, reuse the blind second-judge + Cohen's κ from
+   [`check_quality_regression.ts`](../../src/scripts/check_quality_regression.ts)
+   — the same discipline the Phase-1 rerun uses. This is the billable half; the
+   count distributions (step 3) are cheap and already informative.
+
+**Explicitly out of scope:** running the models' emitted subagent calls, a
+persistent multi-host runtime, any new vendor integration. If the signal needs
+more than council-transport can give, that is a *separate, later* build — named,
+not assumed.
+
+## What the signal feeds — `finding_floor` calibration
+
+`finding_floor` is an **inert mechanism today** (referenced in
+`run_skill_evals.ts` / `skill_linter.ts` / `evals.schema.json`); its recorded
+deferral reason was exactly the missing calibration input this eval produces. The
+per-host finding-count distributions calibrate the floor:
+
+- Derive the floor from the **cross-host lower envelope** (e.g. a low percentile of
+  the per-task finding counts across vendors) so the gate fires on a host/task that
+  under-produces relative to the cross-vendor norm, not relative to one vendor.
+- Once calibrated, promote `finding_floor` from inert to an **enforcing CI gate**
+  (its intended end state).
+
+## Build-vs-defer decision inputs (for step 2, maintainer)
+
+- **Cost:** the count-distribution pass (steps 1–3) is `N_tasks × N_vendors`
+  council calls — small and cheap; render the `council:estimate`-class disclosure
+  before the first call. The judged-quality half (step 4) roughly doubles it.
+- **Value:** unblocks `finding_floor` (arms a real CI gate) **and** produces the
+  first honest cross-vendor parity number the portfolio has ever had.
+- **Defer condition:** if built later, park with the concrete missing capability
+  named — here it would be "council-transport task-adapter for the orchestration
+  corpus", never the generic "harness doesn't exist".
+
+## Exit criteria mapping
+
+- This design doc **exists** (step 1 — done by writing this). ✓
+- Explicit **build/defer decision recorded** (step 2 — maintainer, cost attached). ⏳
+- If built: `finding_floor` calibration data exists and the gate is armed
+  (step 3 — gated on the decision + spend). ⏳

package/docs/design/length-neutral-judge-rerun.md

@@ -0,0 +1,92 @@
+# Length-neutral judge rerun — design
+
+> Design artifact for `road-to-opt-measurement-unblock` Phase 1. It fixes the
+> methodology **before** the first billable call; the run itself is a maintainer
+> `/dev/tty`-gated, money-moving execution (disclose the estimate, confirm the
+> budget in-session, then run). This document is the "design the rerun" step —
+> not the run.
+
+## Why a rerun
+
+The 2026-07-09 discipline-lift judge run was **inconclusive** and now gates ~40%
+of the active token-program portfolio. Three recorded failure modes made it
+un-trustworthy; each has a concrete fix below. Nothing ships from the prior run —
+its verdict is void, not "weakly positive".
+
+| # | Failure mode (2026-07-09) | Magnitude | Fix in this design |
+|---|---|---|---|
+| a | **Length confound** — the preferred arm was systematically longer, so the judge rewarded length, not discipline | 69% of variance | Length-matched pairs **and** a length-partialed rubric (both, belt-and-braces) |
+| b | **Judge inconsistency** — a single judge disagreed with itself / a second grader | 33% inconsistency | Stronger judge tier + a blind second judge, reported as Cohen's κ |
+| c | **Underpowered comparison** | p = 0.196 (n too small) | Pre-registered sample size from the golden corpus, fixed before the first call |
+
+## (a) Length confound — neutralise it two ways
+
+1. **Length-matched pairing.** For each decision task, constrain the two arms
+   (vanilla vs `essential` discipline) to within a **±15% token band** of each
+   other before judging. Pairs that cannot be matched within the band are
+   dropped from the primary analysis and reported separately (never silently
+   discarded). This removes length as a *between-arm* signal.
+2. **Length-partialed rubric.** The judge prompt scores **task quality only** and
+   is explicitly instructed that response length is **not** a quality signal;
+   the harness additionally records each response's token count and runs the
+   verdict through a length-partialed check — if the win-rate correlates with the
+   length delta (Spearman ρ over the surviving pairs), the run is flagged
+   `length-confounded` and the verdict is withheld, exactly as the exit criterion
+   demands ("no length confound flag").
+
+Both, not either: matching removes the gross effect, partialing catches residual
+leakage the band still admits.
+
+## (b) Judge inconsistency — stronger tier + a second judge, reported as κ
+
+- **Judge tier.** Use the strongest available judge model for the primary grade
+  (not the cheapest); the grading cost is a fraction of the generation cost and
+  judge quality is the binding constraint here.
+- **Blind second judge + Cohen's κ — reuse, do not rebuild.** The second-judge
+  agreement machinery already exists: `cohensKappa()` + `judgeKappa()` in
+  [`src/scripts/check_quality_regression.ts`](../../src/scripts/check_quality_regression.ts)
+  align two judges' per-pair winner labels by task id and return chance-corrected
+  κ. Run both judges **blind to arm labels** and to each other; report κ
+  **alongside every verdict**.
+- **κ floor.** The verdict is admissible only when **κ ≥ 0.60** (substantial
+  agreement). Below the floor, the win-rate rests on an unreliable grader — record
+  the κ, do **not** report the win-rate as a finding, and stop (see disposition).
+
+## (c) Underpowered — pre-register the sample size
+
+- **Corpus.** Draw decision tasks from the council-labelled golden set landed by
+  PR #885 (labels are final per the 2026-07-11 amendment — **no re-labelling**).
+- **Power target.** Pre-register n for **80% power** at the smallest
+  effect worth shipping (target: detect a **≥ 10pp** win-rate difference from 50%
+  at α = 0.05). Fix n **before** the first billable call; do not grow n after
+  seeing results (that is the p-hacking the prior run's `p = 0.196` invites).
+- **Statistic.** Paired win/loss over matched pairs → the exact binomial
+  sign-test (`signTestP` already in `second_brain_retrieval.ts`) or McNemar
+  (`bench_ab_v2_stats.ts`); report the two-sided p and the effect size, never a
+  bare "significant".
+
+## Report schema (verdict artifact)
+
+Written under `internal/bench/reports/` with, inline and non-optional:
+
+- per-arm win/loss/tie counts over the **surviving matched pairs** (+ the dropped-pair count);
+- **κ** (both judges) and the κ floor used;
+- the length-confound diagnostic (Spearman ρ of win-rate vs length delta) and the
+  pass/`length-confounded` flag;
+- pre-registered n, achieved power, and the sign-test / McNemar p + effect size;
+- a one-line **verdict**: `ships-lift` / `honest-null` / `inconclusive-<reason>`.
+
+## Disposition (Phase 1 step 4)
+
+- **Trustworthy verdict, either direction** (κ ≥ floor, no length flag, powered):
+  update the token-program tracking table in `road-to-token-proof-and-story.md`
+  and unblock/close the dependent gates.
+- **Second inconclusive:** record *why* with the diagnostics above and **stop** —
+  no third run without a further design change. The anti-pattern this roadmap
+  kills is "gate pending" with no diagnosis.
+
+## What this design does NOT decide
+
+The **execution** — the cost estimate, the in-session budget confirmation, and the
+paid paired run — is the maintainer's, disclosed and confirmed per the acceptance
+criteria. This document only makes the run, when authorized, trustworthy.

package/docs/distribution/consumer-matrix.md

@@ -0,0 +1,96 @@
+# Consumer Matrix — pack-based release E2E
+
+> **Status:** active · **Owner:** maintainer · **Opened:** 2026-07-10
+>
+> Workflow: [`.github/workflows/consumer-matrix.yml`](../../.github/workflows/consumer-matrix.yml)
+> · Runner: [`src/scripts/consumer_matrix.ts`](../../src/scripts/consumer_matrix.ts)
+> · Tripwire: [`.github/workflows/release-adjacent-health.yml`](../../.github/workflows/release-adjacent-health.yml)
+> · Source roadmap: road-to-proof-under-real-conditions (Phase 1)
+
+The consumer matrix exercises the **published tarball** the way a consumer
+does — pack → fresh global install into an isolated prefix → `init` into a
+fresh project → `doctor` → `conformance` → MCP stdio handshake →
+`hooks:doctor` → projection presence (`validate`) → uninstall — plus an
+upgrade leg (last published minor from the registry → packed tarball →
+`doctor` stays green) and pre-tag dry-runs of every release-adjacent
+workflow.
+
+## Why it exists — the tarball window
+
+Ordinary PRs run source-level matrices; release PRs run shape checks and
+(correctly) skip the source matrices per
+[`release-pr-gating`](../contracts/release-pr-gating.md). That skip is sound
+for the *diff* but blind to the *tarball*: every packaging incident below
+entered `main` on an ordinary PR and manifested only at publish time —
+exactly the window where nothing pack-based ran. The consumer matrix is the
+documented exemption from the release-PR cut surface and runs **on** release
+PRs, plus weekly against registry drift.
+
+## Counterfactual map — five historical failures → the leg that catches each
+
+Recorded per the roadmap's acceptance criterion 1. "Leg" names either a
+`consumer_matrix.ts` leg or a `consumer-matrix.yml` job.
+
+| # | Historical failure | Caught by |
+|---|---|---|
+| 1 | Published tarball missing `src/install/` while published code imported it (two minors; `ERR_MODULE_NOT_FOUND` in global installs of `doctor`/`conformance`) | `fresh-install` leg (`--version` + every subsequent CLI leg runs from the tarball) **and** `publish-dry-run` job's `prepack-check.mjs` import-completeness step |
+| 2 | `tsx` absent from the published package — hooks and TS commands failed for consumers | `fresh-install` + `hooks` legs (installed binary + `hooks:doctor` run from the isolated prefix, no repo `node_modules` to lean on) |
+| 3 | Release workflow failed on missing `go-task` in the CI environment | `publish-dry-run` job replicates the publish path pre-tag; any missing binary in a release-adjacent workflow surfaces on the release PR, not post-tag (see also the runbook note in [`release-pr-gating`](../contracts/release-pr-gating.md)) |
+| 4 | npm publish failed on `npm@latest` → npm 12 requiring Node 22 while the job ran Node 20 | `publish-dry-run` job: asserts `publish-npm.yml` pins an npm **major** (fails on `npm@latest`), then installs that exact pin on both Node 20 and 22 and runs the publish steps sans `npm publish` |
+| 5 | MCP worker deploy red across five consecutive releases, undetected (worker deps installed, root `tsx` missing) | `mcp-worker-dry-run` job (same pack + `wrangler deploy --dry-run` path, pre-tag, including the root `npm ci`) **and** the daily `release-adjacent-health` tripwire (fails loudly when any watched workflow's last completed run is non-success for > 48h) |
+
+**Rule for future failures:** a release incident with no leg in this table
+gets a new leg in the same fix PR — or a recorded reason why it is out of
+scope. The table is the audit trail.
+
+## Triggers
+
+| Trigger | What runs | Why |
+|---|---|---|
+| `pull_request` with `release/*` head | all four jobs | the pre-tag gate — release PRs are the primary trigger |
+| weekly cron (Tue 06:00 UTC) | all four jobs | registry/upstream drift between releases — the class the 48h tripwire alone cannot see pre-release |
+| `workflow_dispatch` | all four jobs | on-demand verification (e.g. after a packaging change) |
+| daily cron 06:30 UTC | `release-adjacent-health` | the > 48h durably-red detector |
+
+## Headless default-scope trap (matrix finding, 2026-07-10)
+
+Building this matrix surfaced a live consumer trap: on a fresh machine,
+headless `agent-config init --tools=… --yes` (no `--scope`) picks "project
+default for backward compatibility" and then hard-fails with
+`--scope=project is reserved for maintainers (ADR-020)` — the bare README
+invocation is red in any non-TTY context (CI, scripts), while the TTY path
+escapes into the browser wizard. `--scope=auto` honors detection and
+completes the documented global-only consumer install (`~/.claude`,
+`~/.cursor`, `~/.codeium/windsurf`, `~/.event4u` lockfile). The matrix
+therefore runs `init --scope=auto --yes` and pins that behaviour; whether
+the backward-compatibility default should flip to `auto` is a product
+decision tracked outside this contract (it changes installer behaviour, not
+the gate). Two adjacent facts the matrix also pins: standalone `doctor` is
+project-manifest-scoped (exit 2 without a project lockfile — expected on a
+global-only install; `conformance` embeds `doctor --ci` and is the health
+probe the matrix asserts), and consumer uninstall is `uninstall --global`.
+
+## Branch-protection note
+
+Marking the four jobs as **required** checks for `release/*` PRs is a
+repository setting (Settings → Branches), not something this repo's files
+can enforce. Until flipped, the jobs still run and fail visibly on the
+release PR; flipping them to required is the intended end state.
+
+## Local run (carve-out verification)
+
+```bash
+npm ci && npm run build
+npx tsx src/scripts/consumer_matrix.ts --json           # all legs
+npx tsx src/scripts/consumer_matrix.ts --skip-registry  # offline: skips the upgrade leg
+```
+
+## See also
+
+- [`release-pr-gating`](../contracts/release-pr-gating.md) — the cut surface
+  this matrix is exempt from, and why.
+- [`public-install-smoke`](public-install-smoke.md) — OS × Node entrypoint
+  matrix (installer correctness); the consumer matrix owns tarball shape +
+  runtime surface instead.
+- [`release-sizing`](../contracts/release-sizing.md) — names this matrix as
+  the floor for every release.