@event4u/agent-config 8.10.0 → 8.11.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (192) hide show
  1. package/.claude-plugin/marketplace.json +1 -1
  2. package/CHANGELOG.md +148 -0
  3. package/README.md +5 -5
  4. package/dist/agent-src/commands/challenge-me/vision.md +121 -32
  5. package/dist/agent-src/commands/challenge-me/with-docs.md +21 -8
  6. package/dist/agent-src/commands/commit/in-chunks.md +1 -1
  7. package/dist/agent-src/commands/commit.md +2 -1
  8. package/dist/agent-src/commands/cost/report.md +2 -1
  9. package/dist/agent-src/commands/feature/dev.md +4 -4
  10. package/dist/agent-src/commands/humanize.md +101 -0
  11. package/dist/agent-src/commands/jira-ticket.md +2 -2
  12. package/dist/agent-src/commands/memory/load.md +15 -0
  13. package/dist/agent-src/commands/optimize/project.md +12 -0
  14. package/dist/agent-src/commands/roadmap/create.md +12 -11
  15. package/dist/agent-src/commands/tests/create.md +1 -1
  16. package/dist/agent-src/commands/worktree/cleanup.md +22 -17
  17. package/dist/agent-src/commands/worktree/create.md +3 -1
  18. package/dist/agent-src/commands/worktree/status.md +12 -0
  19. package/dist/agent-src/contexts/execution/autonomy-mechanics.md +66 -0
  20. package/dist/agent-src/contexts/execution/orchestration-benchmark-gate.md +1 -0
  21. package/dist/agent-src/contexts/execution/orchestration-telemetry.md +47 -3
  22. package/dist/agent-src/contexts/execution/roadmap-process-loop.md +14 -0
  23. package/dist/agent-src/contexts/execution/roadmap-writing-source-derived.md +65 -0
  24. package/dist/agent-src/contexts/execution/subagent-modes-detail.md +84 -0
  25. package/dist/agent-src/rules/autonomous-execution.md +7 -37
  26. package/dist/agent-src/rules/broken-access-control.md +7 -45
  27. package/dist/agent-src/rules/code-comment-discipline.md +148 -0
  28. package/dist/agent-src/rules/context-hygiene.md +14 -0
  29. package/dist/agent-src/rules/delegation-policy.md +10 -8
  30. package/dist/agent-src/rules/design-fidelity.md +4 -0
  31. package/dist/agent-src/rules/git-history-discipline.md +4 -79
  32. package/dist/agent-src/rules/legal-safety-floor.md +11 -55
  33. package/dist/agent-src/rules/roadmap-progress-sync.md +5 -56
  34. package/dist/agent-src/rules/think-before-action.md +10 -0
  35. package/dist/agent-src/rules/verify-before-complete.md +12 -0
  36. package/dist/agent-src/skills/agent-docs-writing/SKILL.md +13 -12
  37. package/dist/agent-src/skills/ai-council/SKILL.md +19 -9
  38. package/dist/agent-src/skills/api-design/SKILL.md +7 -5
  39. package/dist/agent-src/skills/api-testing/SKILL.md +1 -1
  40. package/dist/agent-src/skills/authz-review/SKILL.md +59 -2
  41. package/dist/agent-src/skills/blast-radius-analyzer/SKILL.md +1 -1
  42. package/dist/agent-src/skills/bug-analyzer/SKILL.md +19 -16
  43. package/dist/agent-src/skills/code-refactoring/SKILL.md +9 -9
  44. package/dist/agent-src/skills/content-funnel-design/SKILL.md +2 -0
  45. package/dist/agent-src/skills/data-flow-mapper/SKILL.md +1 -1
  46. package/dist/agent-src/skills/defense-in-depth/SKILL.md +2 -2
  47. package/dist/agent-src/skills/dependency-upgrade/SKILL.md +7 -7
  48. package/dist/agent-src/skills/design-intelligence/SKILL.md +8 -8
  49. package/dist/agent-src/skills/design-review/SKILL.md +23 -2
  50. package/dist/agent-src/skills/doc-coauthoring/SKILL.md +2 -0
  51. package/dist/agent-src/skills/editorial-calendar/SKILL.md +2 -0
  52. package/dist/agent-src/skills/fe-design/SKILL.md +7 -9
  53. package/dist/agent-src/skills/git-workflow/SKILL.md +86 -0
  54. package/dist/agent-src/skills/humanizer/SKILL.md +147 -0
  55. package/dist/agent-src/skills/humanizer/data/patterns.md +190 -0
  56. package/dist/agent-src/skills/humanizer/evals/triggers.json +17 -0
  57. package/dist/agent-src/skills/laravel/SKILL.md +14 -1
  58. package/dist/agent-src/skills/launch-readiness/SKILL.md +20 -19
  59. package/dist/agent-src/skills/legal-practice-profile/SKILL.md +126 -0
  60. package/dist/agent-src/skills/logging-monitoring/SKILL.md +1 -1
  61. package/dist/agent-src/skills/mcp/SKILL.md +1 -1
  62. package/dist/agent-src/skills/memory-consolidation/SKILL.md +14 -1
  63. package/dist/agent-src/skills/migration-architect/SKILL.md +2 -1
  64. package/dist/agent-src/skills/performance/SKILL.md +1 -1
  65. package/dist/agent-src/skills/php-debugging/SKILL.md +17 -0
  66. package/dist/agent-src/skills/readme-writing/SKILL.md +2 -0
  67. package/dist/agent-src/skills/readme-writing-package/SKILL.md +2 -0
  68. package/dist/agent-src/skills/release-comms/SKILL.md +2 -0
  69. package/dist/agent-src/skills/roadmap-writing/SKILL.md +22 -51
  70. package/dist/agent-src/skills/security-audit/SKILL.md +6 -9
  71. package/dist/agent-src/skills/sentry-integration/SKILL.md +2 -2
  72. package/dist/agent-src/skills/skill-improvement-pipeline/SKILL.md +8 -0
  73. package/dist/agent-src/skills/source-discovery/SKILL.md +26 -24
  74. package/dist/agent-src/skills/subagent-orchestration/SKILL.md +55 -46
  75. package/dist/agent-src/skills/subagent-orchestration/evals/evals.json +42 -0
  76. package/dist/agent-src/skills/systematic-debugging/SKILL.md +10 -0
  77. package/dist/agent-src/skills/using-git-worktrees/SKILL.md +10 -0
  78. package/dist/agent-src/skills/worktree-lifecycle/SKILL.md +27 -20
  79. package/dist/agent-src/skills/worktree-lifecycle/evals/evals.json +38 -0
  80. package/dist/agent-src/templates/agent-settings.md +3 -2
  81. package/dist/agent-src/templates/agents/agent-project-settings.example.yml +15 -1
  82. package/dist/agent-src/templates/roadmaps.md +14 -0
  83. package/dist/cli/commands/recordTriggerEval.js +4 -14
  84. package/dist/cli/commands/recordTriggerEval.js.map +1 -1
  85. package/dist/discovery/deprecation-report.md +1 -1
  86. package/dist/discovery/discovery-manifest.json +147 -78
  87. package/dist/discovery/discovery-manifest.json.sha256 +1 -1
  88. package/dist/discovery/discovery-manifest.summary.md +6 -6
  89. package/dist/discovery/orphan-report.md +1 -1
  90. package/dist/discovery/packs.json +13 -10
  91. package/dist/discovery/trust-report.md +3 -3
  92. package/dist/discovery/workspaces.json +14 -9
  93. package/dist/install/install.mjs +12 -7
  94. package/dist/mcp/registry-manifest.json +4 -4
  95. package/dist/router.json +1 -1
  96. package/dist/scripts/_lib/claude_settings_hooks.js +8 -0
  97. package/dist/scripts/_lib/claude_settings_hooks.js.map +1 -1
  98. package/dist/scripts/_lib/trigger_eval_floors.js +24 -0
  99. package/dist/scripts/_lib/trigger_eval_floors.js.map +1 -0
  100. package/dist/scripts/condense.js +12 -2
  101. package/dist/scripts/condense.js.map +1 -1
  102. package/dist/server/schemas/settings.js +5 -2
  103. package/dist/server/schemas/settings.js.map +1 -1
  104. package/docs/CLAIMS.md +24 -3
  105. package/docs/architecture.md +3 -3
  106. package/docs/benchmark.md +51 -0
  107. package/docs/catalog.md +8 -5
  108. package/docs/command-flows.md +2 -1
  109. package/docs/contracts/ai-council-config.md +92 -0
  110. package/docs/contracts/command-clusters.md +1 -0
  111. package/docs/contracts/evidence-based-pruning.md +1 -0
  112. package/docs/contracts/hook-architecture-v1.md +25 -0
  113. package/docs/contracts/kernel-membership.md +35 -0
  114. package/docs/contracts/measurement-baseline.md +1 -0
  115. package/docs/contracts/release-sizing.md +22 -0
  116. package/docs/contracts/write-engine.md +38 -1
  117. package/docs/customization.md +12 -0
  118. package/docs/decisions/ADR-002-kernel-bucket-overrides.md +11 -0
  119. package/docs/decisions/ADR-003-flat-cluster-subs-and-colon-syntax.md +11 -0
  120. package/docs/decisions/ADR-006-skill-tools-python-pilot.md +1 -1
  121. package/docs/decisions/ADR-028-root-layout.md +1 -1
  122. package/docs/decisions/ADR-038-canonical-settings-path.md +10 -0
  123. package/docs/decisions/ADR-092-defer-command-tier-alias-removal.md +29 -5
  124. package/docs/decisions/ADR-093-ai-council-config-user-global.md +1 -1
  125. package/docs/decisions/ADR-100-global-knowledge-card-sharing.md +7 -0
  126. package/docs/decisions/ADR-103-global-knowledge-default-off-until-measured.md +2 -2
  127. package/docs/decisions/ADR-105-automatic-subagent-orchestration.md +1 -1
  128. package/docs/decisions/ADR-114-commit-policy-iron-law-override.md +11 -0
  129. package/docs/decisions/ADR-116-memory-tripwire-activation-path.md +11 -1
  130. package/docs/decisions/ADR-117-subagents-auto-default-on.md +2 -1
  131. package/docs/decisions/ADR-118-loop-engineering-boundaries.md +162 -0
  132. package/docs/decisions/ADR-119-global-knowledge-default-on.md +104 -0
  133. package/docs/decisions/ADR-200-python-to-typescript-migration.md +11 -1
  134. package/docs/decisions/ADR-rule-kernel-and-router.md +11 -0
  135. package/docs/decisions/INDEX.md +11 -9
  136. package/docs/design/cross-model-parity-eval.md +82 -0
  137. package/docs/design/length-neutral-judge-rerun.md +92 -0
  138. package/docs/featured-skills.md +2 -2
  139. package/docs/getting-started-by-role.md +1 -1
  140. package/docs/getting-started.md +2 -2
  141. package/docs/governance-advantage.md +1 -1
  142. package/docs/guidelines/agent-infra/roadmap-progress-mechanics.md +80 -5
  143. package/docs/guidelines/code-clarity.md +106 -1
  144. package/docs/guidelines/design-antipatterns.md +20 -0
  145. package/docs/guidelines/php/php-coding-patterns.md +12 -2
  146. package/docs/proof.md +20 -14
  147. package/docs/wedge/production-validator/production-validator.md +27 -0
  148. package/package.json +1 -1
  149. package/src/config/agent-settings.template.yml +29 -9
  150. package/src/scripts/_cli/cmd_doctor.ts +34 -1
  151. package/src/scripts/_lib/auto_dispatch.corpus.test.ts +87 -0
  152. package/src/scripts/_lib/claude_settings_hooks.ts +8 -0
  153. package/src/scripts/_lib/knowledge_global.ts +6 -2
  154. package/src/scripts/_lib/knowledge_global_redaction.ts +42 -4
  155. package/src/scripts/_lib/lexical_index.ts +15 -2
  156. package/src/scripts/_lib/orchestration_record.ts +14 -0
  157. package/src/scripts/_lib/trigger_eval_floors.ts +31 -0
  158. package/src/scripts/ai_council/chairman.ts +90 -0
  159. package/src/scripts/ai_council/clients.ts +18 -3
  160. package/src/scripts/ai_council/config.ts +157 -0
  161. package/src/scripts/ai_council/debate_gates.ts +89 -0
  162. package/src/scripts/ai_council/orchestrator.ts +65 -2
  163. package/src/scripts/ai_council/prompts.ts +121 -11
  164. package/src/scripts/ai_council/replay.ts +17 -0
  165. package/src/scripts/ai_council/stance_tally.ts +184 -0
  166. package/src/scripts/ai_tells_rules.ts +433 -0
  167. package/src/scripts/bench_humanizer_eval.ts +240 -0
  168. package/src/scripts/build_discovery_manifest.ts +6 -0
  169. package/src/scripts/build_proof.ts +21 -9
  170. package/src/scripts/check_claims.ts +40 -0
  171. package/src/scripts/check_rule_invariants.ts +10 -3
  172. package/src/scripts/condense.ts +16 -2
  173. package/src/scripts/consumer_matrix.ts +41 -2
  174. package/src/scripts/council_cli.ts +147 -0
  175. package/src/scripts/detect_ai_tells.ts +312 -0
  176. package/src/scripts/install.ts +12 -0
  177. package/src/scripts/knowledge_global_cli.ts +1 -1
  178. package/src/scripts/lint_framework_leakage.ts +88 -7
  179. package/src/scripts/lint_framework_leakage_allowlist.json +59 -572
  180. package/src/scripts/lint_knowledge_scale.ts +30 -0
  181. package/src/scripts/lint_originality_shingles.ts +210 -0
  182. package/src/scripts/orchestration_record.ts +1 -0
  183. package/src/scripts/schemas/agent-settings.schema.json +11 -0
  184. package/src/scripts/self_review_gate.ts +41 -15
  185. package/src/scripts/skill_eval_coverage.ts +71 -3
  186. package/src/scripts/skill_linter.ts +1 -0
  187. package/src/scripts/trigger_eval_rotation.ts +344 -0
  188. package/src/scripts/update_counts.ts +5 -17
  189. package/src/scripts/worktree_cleanup_check.ts +263 -0
  190. package/src/server/schemas/settings.ts +8 -3
  191. package/src/scripts/analysis_freshness.ts +0 -362
  192. package/src/scripts/export_replay_corpus.ts +0 -208
@@ -29,13 +29,16 @@ question with the Phase-3 investigation evidence.
29
29
  Investigation findings (`road-to-metadata-and-command-surface-leanness`, Phase 3):
30
30
 
31
31
  - **Integer-`tier` readers are internal:** `src/cli/commands/commands.ts`
32
- (fallback after `visibility`), `src/scripts/audit_command_surface.py`
33
- (report column + budget fallback), `src/scripts/build_discovery_manifest.py`
34
- (**dual-emits `tier`** into the manifest). `src/cli/python/workspace_hosts.py`
32
+ (fallback after `visibility`), `src/scripts/audit_command_surface.ts`
33
+ (report column + budget fallback), `src/scripts/build_discovery_manifest.ts`
34
+ (**dual-emits `tier`** into the manifest). `src/cli/python/workspace_hosts.ts`
35
35
  uses a semantically unrelated host-inventory tier.
36
+ <!-- paths updated 2026-07-12: every reader migrated to TS during the
37
+ completed ADR-200 py2ts migration; the original .py list is preserved in
38
+ git history. -->
36
39
  - **The discovery manifest is a published npm artifact** — `package.json`
37
40
  `files` + `build:discovery` in `prepack` + `prepublishOnly:
38
- check_release_includes_discovery.py`. It dual-emits the integer `tier`.
41
+ check_release_includes_discovery.ts`. It dual-emits the integer `tier`.
39
42
  **External npm consumers that read the integer key are unknown.**
40
43
  - The defer rests on a **Runtime Risk that cannot be ruled out** (a consumer
41
44
  may register/branch on the integer `tier`), not on evidence that such a
@@ -75,7 +78,7 @@ forcing function so it cannot calcify into permanent debt:
75
78
  ## Consequences
76
79
 
77
80
  - The `tier:` ⇄ `visibility:` dual-field persists during the deferral;
78
- drift is mitigated by the `lint_command_tiers.py` consistency check (when
81
+ drift is mitigated by the `lint_command_tiers.ts` consistency check (when
79
82
  both present, they must agree).
80
83
  - The removal is **cheaply reversible** if ever executed: restoring the `tier`
81
84
  field is a manifest schema patch (< 1h to publish), far below the rollback
@@ -84,6 +87,27 @@ forcing function so it cannot calcify into permanent debt:
84
87
  - `road-to-tier-removal.md` institutionalises the trigger so the defer stays
85
88
  visible in planning rather than becoming folklore.
86
89
 
90
+ ## Addendum (2026-07-12) — re-open mechanism mid-flight
91
+
92
+ Recorded during `road-to-opt-decision-flips` Phase 2; not a re-litigation —
93
+ this ADR's own forcing function is executing:
94
+
95
+ - **All integer-`tier` readers are now TypeScript** (ADR-200 migration landed
96
+ on `main`; the reader list above updated in place).
97
+ - **Re-open mechanism #1 shipped:** `build_discovery_manifest.ts` emits the
98
+ machine-readable `deprecations` block + `sunset` field announcing the
99
+ `tier` deprecation (road-to-tier-removal Phase 1). Verified in the
100
+ **released** artifact: npm `8.10.0` (published 2026-07-10) ships
101
+ `dist/discovery/discovery-manifest.json` with
102
+ `deprecations[0] = {key: tier, replacement: visibility, since: ADR-092,
103
+ sunset: null}`.
104
+ - **Soak status:** the soak clock runs since publish; zero tier-related
105
+ breakage reports (issue sweep 2026-07-12). The unknown-external-consumer
106
+ hard stop is NOT lifted by the notice alone — the soak must elapse.
107
+ - **Execution continues in `road-to-tier-removal.md`**, revived from
108
+ `later/` to active on 2026-07-12 with the soak evidence cited in its
109
+ banner (portfolio-consolidation Phase 2).
110
+
87
111
  ## References
88
112
 
89
113
  - [ADR-090](ADR-090-visibility-command-frontmatter-field.md) — the deferred
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  adr: 093
3
- status: accepted
3
+ status: superseded
4
4
  date: 2026-06-13
5
5
  decision: ai-council-config-user-global
6
6
  supersedes: —
@@ -11,6 +11,13 @@ type: structural
11
11
 
12
12
  # ADR-100 — Global cross-project knowledge-card sharing
13
13
 
14
+ > **Partial supersession (clarified 2026-07-12):** this ADR supersedes
15
+ > ADR-098 **Decision-10 only** (the measure-then-decide gate for the
16
+ > global layer); every other ADR-098 decision stands. The frontmatter
17
+ > `supersedes:` field carries the bare number because the index
18
+ > link-checker validates it; this note is the qualifier's home.
19
+
20
+
14
21
  ## Status
15
22
 
16
23
  Accepted (2026-06-15). Council-designed (claude-sonnet-4-5 + gpt-4o, design
@@ -1,10 +1,10 @@
1
1
  ---
2
2
  adr: 103
3
- status: accepted
3
+ status: superseded
4
4
  date: 2026-06-16
5
5
  decision: global-knowledge-default-off-until-measured
6
6
  supersedes: —
7
- superseded_by: —
7
+ superseded_by: 119
8
8
  phase: evidence-v2
9
9
  type: structural
10
10
  ---
@@ -4,7 +4,7 @@ status: accepted
4
4
  date: 2026-06-23
5
5
  decision: automatic-subagent-orchestration
6
6
  supersedes: —
7
- superseded_by: —
7
+ superseded_by: — (Decision 2 amended by ADR-117: shipped default ask → on)
8
8
  phase: auto-subagent-orchestration
9
9
  type: structural
10
10
  ---
@@ -1,3 +1,14 @@
1
+ ---
2
+ adr: 114
3
+ status: accepted
4
+ date: 2026-07-08
5
+ decision: commit-policy-iron-law-override
6
+ supersedes: —
7
+ superseded_by: —
8
+ phase: kernel-budget overrides (extends ADR-002)
9
+ type: structural
10
+ ---
11
+
1
12
  # ADR-114 — commit-policy Iron-Law Override (per-rule cap 2.5k → ceiling 4.0k)
2
13
 
3
14
  - **Status:** Accepted (2026-07-08)
@@ -5,11 +5,21 @@ date: 2026-07-08
5
5
  decision: memory-tripwire-activation-path
6
6
  supersedes: —
7
7
  superseded_by: —
8
- phase: road-to-memory-retrieval-economy · Phase 6
8
+ phase: road-to-memory-retrieval-economy · Phase 6 (amended — see 2026-07-12 note)
9
9
  ---
10
10
 
11
11
  # ADR-116 — Memory scale-tripwire activation path: one engine, measured at fire time
12
12
 
13
+ > **Amended in practice (recorded 2026-07-12):** the SQLite-FTS5 engine
14
+ > this ADR pre-decided was never built. The activation path was
15
+ > re-resolved by road-to-retrieval-substrate-hardening B2 (done
16
+ > 2026-07-09): a hand-rolled stdlib BM25 + trigram prefilter in
17
+ > `src/scripts/_lib/lexical_index.ts` (no SQLite, ADR-061 honoured),
18
+ > dormant until the `lint_knowledge_scale` tripwire fires (200/type,
19
+ > 500 total). The tripwire thresholds and measurement-at-activation
20
+ > discipline of this ADR stand unchanged; only the engine choice moved.
21
+
22
+
13
23
  - **Deciders:** maintainer + AI council (anthropic/claude-sonnet-4-5 +
14
24
  openai/gpt-4o; disposition debate 2 rounds + engine tie-break 2 rounds,
15
25
  2026-07-08 — tie-break converged after the in-repo `node:sqlite`
@@ -3,10 +3,11 @@ adr: 117
3
3
  status: accepted
4
4
  date: 2026-07-09
5
5
  decision: subagents-auto-default-on
6
- supersedes: 105
6
+ supersedes: —
7
7
  superseded_by: —
8
8
  phase: auto-subagent-orchestration
9
9
  type: structural
10
+ amends: 105 (Decision 2 only — the shipped default; the contract itself stands)
10
11
  ---
11
12
 
12
13
  # ADR-117 — Flip `subagents.auto` shipped default to `on` (host-gated) on bounded-downside re-evaluation
@@ -0,0 +1,162 @@
1
+ ---
2
+ adr: 118
3
+ status: accepted
4
+ date: 2026-07-10
5
+ decision: loop-engineering-boundaries
6
+ supersedes: —
7
+ superseded_by: —
8
+ phase: loop-engineering
9
+ type: structural
10
+ ---
11
+
12
+ # ADR-118 — Loop-engineering boundaries: one closure, four rejections, three deferrals, zero new loop surfaces
13
+
14
+ ## Status
15
+
16
+ **Accepted** · 2026-07-10. Resolved by web research (five external articles + an
17
+ independent sweep) ground-truthed against the repo's loop inventory, plus an
18
+ AI-council debate (anthropic/claude-sonnet-4-5 + openai/gpt-4o, 2 rounds,
19
+ 2026-07-10, converged). Executed by `road-to-loop-engineering`.
20
+
21
+ ## Context
22
+
23
+ The June-2026 "loop engineering" wave (design the system that prompts the
24
+ agent: verification signal, budget shape, maker/checker topology, on-disk
25
+ state) is mostly a rebranding of ReAct + eval-driven development + bounded
26
+ autonomous agents. Its two durable contributions are the design vocabulary and
27
+ a precise failure-mode taxonomy: verification collapse ("done" is a claim, not
28
+ proof), comprehension debt, cognitive surrender, reward hacking (measured at
29
+ ~15% of SE-task runs in one external evaluation), cost blowup on stuck retry
30
+ loops, context rot on long runs.
31
+
32
+ This package already implements the serious end of that canon: bounded
33
+ verify-fix loops (`verify-repair-loop` max_attempts=3, the N=3 validation
34
+ budget, context-hygiene 15/25 aborts), maker/checker separation (`judge-*`
35
+ cluster, `verify-budget`), human-gated self-improvement (5-stage pipeline,
36
+ ≥2 independent evidence), and measurement infrastructure (bench:ab, trigger
37
+ evals, golden sets, orchestration telemetry, local analytics).
38
+
39
+ An inventory identified 8 places where a measure exists but no adjust-step is
40
+ wired. The question this ADR settles: which of those loops to close, and which
41
+ loop-shaped proposals to reject so they are not relitigated per hype cycle.
42
+
43
+ Constraint frame: single maintainer, token-frugality canon, no runtime daemon
44
+ (ADR-088 / no-runtime-boundary), human gates on anything irreversible. Under
45
+ these constraints, when human review + `git revert` is cheaper than designing,
46
+ maintaining, and debugging a closed loop, the loop is net-negative
47
+ infrastructure — false positives consume the binding resource (maintainer
48
+ attention), not compute.
49
+
50
+ ## Decision
51
+
52
+ ### 1. Automation threshold (the core principle)
53
+
54
+ Automate a measure→adjust loop **only** when all three hold:
55
+
56
+ 1. the metric is a **direct** measure of the failure mode (not a proxy or a
57
+ correlation),
58
+ 2. the expected false-positive rate is low (order <5%), and
59
+ 3. human judgment adds no unique information to the adjust decision.
60
+
61
+ "A number exists" is not "the number is trustworthy" (metric-existence
62
+ fallacy). Reversibility of the adjust-step alone never justifies automation:
63
+ investigating one false-positive automated action costs 20–45 minutes of
64
+ maintainer time against a 10-second revert.
65
+
66
+ ### 2. Disposition of the 8 open loops
67
+
68
+ | # | Open loop | Disposition |
69
+ |---|---|---|
70
+ | 1 | Orchestration telemetry → `subagents.auto` demotion | **Manual by decision.** token_delta spikes for legitimate reasons; demotion needs qualitative judgment of the failure mode. The demotion path in `orchestration-benchmark-gate.md` stays a human one-line edit. |
71
+ | 2 | Artifact-engagement counts → skill pruning | **Never automated.** Selection bias both ways (low engagement ≠ low value: security / error-recovery / niche skills; high engagement ≠ high value). Engagement stays ONE input under `evidence-based-pruning`'s ≥2-independent-evidence floor; the pruning decision is always human. |
72
+ | 3 | Trigger-eval pass rate on cadence | **Closed** — the only closure. Structure/freshness/presence gates already ran in `task ci`; this ADR's roadmap adds a rotating live pass-rate measurement to the existing weekly canary (see §4). Direct metric, low false-positive, adjust-step = surfaced report. |
73
+ | 4 | bench-drift → blocking gate | **Stays advisory.** High false-positive proxy (timing variance, intentional behaviour changes); the manual `bench:baseline-ready` flip remains the enforcement gate. |
74
+ | 5 | Golden-set coverage ratchet (14/91 rules) | **Deferred; growth stays opportunistic.** A hard per-rule ratchet pressures toward mechanically-testable rules over useful rules; most uncovered rules are judgment-shaped. `road-to-golden-set-coverage` remains the deliberate path. |
75
+ | 6 | Live-app verdict source | **Deferred, unchanged** — owned by `road-to-live-app-verdict`; the package ships config, not apps. |
76
+ | 7 | Measured rule-adherence signal | **Rejected.** Annotated-corpus + threshold-tuning build with ongoing maintenance for marginally earlier restatement; ADR-054's coarse heuristics are intentional. |
77
+ | 8 | Success-signal re-evaluation at eval date | **Not automated.** Retroactive evaluation of sunk promotion decisions has low marginal information; active use surfaces bad promotions. The field stays authored documentation. |
78
+
79
+ ### 3. Five written rejections (loop anti-patterns for this package)
80
+
81
+ 1. **Open-ended hill-climbing on the config.** No loop may "optimize"
82
+ rules/skills via iterated mutation + metric without a human checkpoint per
83
+ iteration; the bench:ab soak + human decision is the floor for config
84
+ change evaluation.
85
+ 2. **Autonomous config self-editing.** The self-improvement pipeline stays
86
+ human-gated at every stage; no loop commits changes to `src/skills/` or
87
+ `src/rules/` without human approval. (Restates the existing pipeline
88
+ contract as a loop boundary.)
89
+ 3. **Unattended multi-hour runs without checkpoints.** Loops must remain
90
+ reviewable/haltable on the maintainer's schedule; the existing halt
91
+ conditions and N=3 budgets are the ceiling, not a starting point.
92
+ 4. **Metric-optimizing loops without golden-set anchoring.** No loop whose
93
+ success criterion is "metric goes up" without a golden-set eval anchoring
94
+ what "good" means (Goodhart guard).
95
+ 5. **Cross-session loop state in external services.** No vector DBs, job
96
+ queues, or external orchestrators as loop state (restates ADR-088 in loop
97
+ context): state lives in git-tracked files + the conversation.
98
+
99
+ Also rejected, same reasoning tier: a **loop-design contract/checklist
100
+ artifact** (premature formalization — the caps/judges/state patterns are
101
+ embedded in the existing loop surfaces and enforced by existing rules) and
102
+ **consumer-facing loop templates** (`/loop:design` et al. — scope creep;
103
+ agents answer loop-design questions in conversation using the package's
104
+ existing vocabulary). **No new loop surfaces**: no run-until-condition goal
105
+ command, no fresh-context re-loop mode for roadmap processing (bounded-scope
106
+ by design), no separate nightly self-check workflow (folded into the existing
107
+ weekly canary).
108
+
109
+ ### 4. The one closure — periodic live trigger-eval pass rate
110
+
111
+ The existing weekly cross-model canary's secrets-gated live tier gains a
112
+ deterministic rotation over the skills that carry `evals/triggers.json`
113
+ (ISO-week keyed, ~5 suites/week), enforcing the per-domain precision/recall
114
+ floors from `eval:record`. A floor breach fails the **scheduled** job — the
115
+ failure is the maintainer notification; PRs are never blocked by live results.
116
+ The local interactive confirmation gate on `skill_trigger_eval` is untouched;
117
+ CI authorization derives exclusively from workflow-provided key secrets, and
118
+ the no-secrets path stays a logged no-op.
119
+
120
+ ## Revisit-if
121
+
122
+ - **Any rejection above:** reopen only on evidence that closing the loop saves
123
+ >2 h/month maintainer time in a structurally similar single-maintainer
124
+ project (logged hours, not vibes) — via `decision-revisit-gate`
125
+ mechanism-match, then council.
126
+ - **The §4 closure:** demote the canary pass-rate job to advisory
127
+ (non-failing) if breaches are >~10% spurious over 50+ suite-runs.
128
+ - **§2 #5 coverage ratchet:** reconsider if ≥3 real regressions ship in rules
129
+ that a golden task would have caught (counted, cited incidents).
130
+
131
+ ## Consequences
132
+
133
+ - Future "should we automate X?" loop proposals test against §1 before any
134
+ build; the disposition table is the precedent record.
135
+ - The weekly canary becomes the single periodic live-eval surface — additions
136
+ to periodic self-checking extend it rather than adding workflows.
137
+ - Skill-pruning, default-demotion, and drift-blocking proposals cite this ADR
138
+ instead of re-deriving the argument.
139
+
140
+ ## Alternatives considered
141
+
142
+ - **Close more loops (coverage ratchet, engagement pruning, auto-demotion):**
143
+ rejected — each fails at least one §1 criterion; the council's round-2
144
+ convergence was explicit that "metric exists + reversible" is insufficient.
145
+ - **Loop-design contract:** rejected as compliance theater for a
146
+ single-maintainer project; revisit on a second maintainer with real handoff
147
+ friction.
148
+ - **Do nothing (pure documentation):** rejected — the trigger-eval pass-rate
149
+ gap is real, direct-metric, and closable inside an existing workflow at
150
+ near-zero maintenance cost.
151
+
152
+ ## References
153
+
154
+ - `road-to-loop-engineering` roadmap (execution + provenance; external source
155
+ links retained encrypted there per `source-confidentiality`).
156
+ - `docs/contracts/evidence-based-pruning.md` — §2 #2 anchor.
157
+ - `src/agent-src/contexts/execution/orchestration-benchmark-gate.md` — §2 #1 anchor.
158
+ - `docs/contracts/measurement-baseline.md` — §2 #4 anchor.
159
+ - ADR-088 (no external runtime), ADR-054 (adherence restate), ADR-106/-109/-117
160
+ (orchestration gates), `docs/contracts/no-runtime-boundary.md`.
161
+ - Prior honest-nulls this ADR does not reopen: recursive-verification
162
+ (2026-06-24), enforcement-projection, reminder-injection.
@@ -0,0 +1,104 @@
1
+ ---
2
+ adr: 119
3
+ status: accepted
4
+ date: 2026-07-12
5
+ decision: global-knowledge-default-on
6
+ supersedes: 103
7
+ superseded_by: —
8
+ phase: opt-decision-flips
9
+ type: structural
10
+ ---
11
+
12
+ # ADR-119 — Global knowledge sharing defaults ON (validated bounded-downside flip)
13
+
14
+ ## Status
15
+
16
+ Accepted (2026-07-12). Supersedes ADR-103.
17
+
18
+ ## Context
19
+
20
+ ADR-103 (2026-06-16) kept `knowledge.global_sharing.enabled: false` until
21
+ cross-project reuse was measured across ≥ 2 projects. Verified state at
22
+ re-evaluation (2026-07-11): the measurement instrument
23
+ (`src/scripts/_lib/knowledge_global_promote.ts`) shipped, but no sightings
24
+ data, no global store, and no reuse record existed — reuse can only accrue
25
+ while the layer is ON, and ON was withheld pending reuse. The gate could
26
+ never fire by construction: the same self-locking-measurement deadlock
27
+ ADR-117 broke for `subagents.auto` on 2026-07-09.
28
+
29
+ A two-round AI-council re-evaluation ran on 2026-07-11 (members:
30
+ anthropic/claude-sonnet-4-5, openai/gpt-4o; cost $0.10). Round 1: both
31
+ members stanced **Option A — flip ON now** (deadlock + ADR-117 precedent +
32
+ shipped guards). Round 2 dissent (gpt-4o): the redaction floor was
33
+ unvalidated for cross-project adversarial scenarios — composite inference,
34
+ homoglyph/encoding smuggling, temporal context collapse. The named
35
+ evidence-that-would-change-my-mind: demonstrating the redaction mechanism
36
+ handles those classes. Convergence: **flip WITH validation** — the dissent
37
+ becomes a pre-flip gate, not a blocker.
38
+
39
+ ## Decision
40
+
41
+ 1. **Pre-flip validation executed (2026-07-12).** Adversarial spot-check in
42
+ `tests/scripts/_lib_knowledge_global_redaction_adversarial.test.ts` +
43
+ `tests/fixtures/global-knowledge-redaction/` covering the three named
44
+ classes. Honest result: the zero-width-smuggling probe FAILED on first
45
+ run — the dissent's predicted gap was real. Closed by an additive
46
+ hardening in `knowledge_global_redaction.ts` (`hidden_unicode` violation
47
+ class + strip-then-rescan); homoglyph emails were already caught
48
+ (Unicode-aware pattern); temporal collapse is bounded by the freshness
49
+ state machine (fresh → hypothesis → stale). All 20 tests green,
50
+ including the 11 pre-existing parity tests.
51
+ 2. **`knowledge.global_sharing.enabled` defaults to `true`** in the shipped
52
+ template.
53
+ 3. **Narrowest tier default:** `allowed_tiers: [public]` (down from
54
+ `[public, vendor]`); `vendor` becomes a deliberate opt-in widening.
55
+ `proprietary` stays hard-coded manual-only in the gate.
56
+ 4. **Pre-registered demotion trigger:** any cross-project PII/secret
57
+ sighting in a shared card — a confirmed identifier that crossed a
58
+ project boundary despite the gate — auto-reverts the shipped default to
59
+ `false` and lands an incident note under `agents/settings/contexts/`.
60
+ This is the ADR-117 demotion-gate pattern applied to knowledge sharing.
61
+ 5. **Measurement window:** 60–90 days from release. The reuse data the
62
+ ADR-103 gate wanted now accrues; at window end the default is either
63
+ confirmed (reuse observed, no incidents) or reverted (no reuse — the
64
+ layer is dead weight — or a demotion trigger fired).
65
+ 6. **Accepted residual, pinned by test:** k-anonymity over pure
66
+ quasi-identifier combinations is out of scope for a write-time text
67
+ gate. The council accepted this on the single-install trust boundary
68
+ (an actor with filesystem access to the global store already reads the
69
+ raw projects, which are strictly more sensitive). The boundary is
70
+ asserted by a dedicated test so any future change to it is a deliberate
71
+ decision, not drift.
72
+
73
+ ## Consequences
74
+
75
+ - Cross-project knowledge cards flow by default on the narrowest tier,
76
+ through the unchanged suggest-then-confirm promotion flow
77
+ (`auto_promote_threshold` gates a suggestion, never a silent write) and
78
+ the hardened redaction gate (halt-and-surface, never silent-rewrite).
79
+ - Single-key revert: `knowledge.global_sharing.enabled: false` fully
80
+ no-ops the layer; v1 project-local cards are unaffected either way.
81
+ - No new infrastructure: file-first store, no vectors, no services — the
82
+ ADR-094 Layer-2 sunset is untouched.
83
+ - The hidden-unicode hardening also protects every other consumer of
84
+ `redaction_scan` (the Phase-4 CI linter over committed global cards).
85
+
86
+ ## Alternatives considered
87
+
88
+ - **Keep OFF + opt-in campaign (Option B):** rejected — self-selection
89
+ bias, slow, and it retains the deadlock for everyone who does not
90
+ opt in.
91
+ - **Maintainer-org pilot first (Option C):** rejected as the primary path —
92
+ the council judged the guarded flip's downside bounded enough that a
93
+ pilot's extra latency buys little; the demotion trigger plus the
94
+ measurement window deliver the same safety net in production shape.
95
+
96
+ ## References
97
+
98
+ - ADR-103 (superseded), ADR-117 (deadlock-flip precedent), ADR-100/-094
99
+ (store shape + Layer-2 sunset).
100
+ - Council session 2026-07-11 (claude-sonnet-4-5 + gpt-4o, 2 rounds,
101
+ convergence inlined above per `no-roadmap-references`).
102
+ - `tests/scripts/_lib_knowledge_global_redaction_adversarial.test.ts` —
103
+ the validation gate.
104
+ - `agents/roadmaps/road-to-opt-decision-flips.md` Phase 1 (execution).
@@ -5,12 +5,22 @@ date: 2026-06-11
5
5
  decision: python-to-typescript-migration
6
6
  supersedes: —
7
7
  superseded_by: —
8
- phase: py2ts · migration-infrastructure
8
+ phase: py2ts · migration-infrastructure (historical record — migration landed on main)
9
9
  type: structural
10
10
  ---
11
11
 
12
12
  # ADR-200 — Full Python → TypeScript migration: execution model, branch strategy, dispatcher, parity gates
13
13
 
14
+ > **Number retained by decision (2026-07-12):** the provisional-number
15
+ > clause ("final number assigned at the python2ts → main merge") has
16
+ > fired — the migration is complete on `main` (0 tracked `.py` under
17
+ > `src/`). Renumbering was evaluated and declined: 521 files reference
18
+ > "ADR-200" as a stable identifier; a rename would be a 521-file
19
+ > mechanical churn (projections, condensation hashes, docs) for zero
20
+ > behavioral gain. ADR-200 stays the permanent number; the record is
21
+ > historical (migration architecture, landed).
22
+
23
+
14
24
  > Provisional number 200 (reserved, out-of-band). main ADRs grow sequentially; the migration ADR is renumbered to a high reserved slot to stop recurring collisions on the long-lived `python2ts` branch. Final number is assigned at the `python2ts`→`main` merge.
15
25
 
16
26
  ## Status
@@ -1,3 +1,14 @@
1
+ ---
2
+ adr: —
3
+ status: accepted
4
+ date: 2026-05-06
5
+ decision: rule-kernel-and-router-roadmap-closure
6
+ supersedes: —
7
+ superseded_by: —
8
+ phase: road-to-kernel-and-router (closure record)
9
+ type: structural
10
+ ---
11
+
1
12
  # ADR — Rule Kernel and Router (Roadmap Closure)
2
13
 
3
14
  - **Status:** Accepted (2026-05-06)
@@ -5,8 +5,8 @@ _Auto-generated by `scripts/adr/regenerate_index.py`. Do not edit._
5
5
  | # | Title | Status | Date | Supersedes |
6
6
  |---|---|---|---|---|
7
7
  | [ADR-001](ADR-001-kernel-swap-deferred.md) | Kernel Swap Deferred | accepted | 2026-05-06 | — |
8
- | [ADR-002](ADR-002-kernel-bucket-overrides.md) | Kernel Bucket Overrides | — | — | — |
9
- | [ADR-003](ADR-003-flat-cluster-subs-and-colon-syntax.md) | Flat Cluster Subs And Colon Syntax | — | — | — |
8
+ | [ADR-002](ADR-002-kernel-bucket-overrides.md) | Kernel Bucket Overrides | accepted | 2026-05-06 | — |
9
+ | [ADR-003](ADR-003-flat-cluster-subs-and-colon-syntax.md) | Flat Cluster Subs And Colon Syntax | accepted | 2026-05-07 | — |
10
10
  | [ADR-004](ADR-004-rule-governance-pruning.md) | Rule Governance Pruning | accepted | 2026-05-08 | — |
11
11
  | [ADR-005](ADR-005-subagent-worktrees.md) | Subagent Worktrees No Auto Merge | accepted | 2026-05-09 | — |
12
12
  | [ADR-006](ADR-006-skill-tools-python-pilot.md) | Skill Tools Python Pilot Pass | accepted | 2026-05-09 | — |
@@ -31,7 +31,7 @@ _Auto-generated by `scripts/adr/regenerate_index.py`. Do not edit._
31
31
  | [ADR-025](ADR-025-workspace-chrome.md) | Workspace Chrome | accepted | 2026-05-24 | — |
32
32
  | [ADR-026](ADR-026-explain-mode-translation.md) | Explain Mode Translation | accepted | 2026-05-24 | — |
33
33
  | [ADR-027](ADR-027-changelog-machine-vs-manual.md) | Changelog Machine Vs Manual | accepted | 2026-05-25 | — |
34
- | [ADR-028](ADR-028-root-layout.md) | Root Layout | accepted | 2026-05-25 | — |
34
+ | [ADR-028](ADR-028-root-layout.md) | Root Layout | superseded | 2026-05-25 | — |
35
35
  | [ADR-029](ADR-029-multi-workspace-deferred.md) | Multi Workspace Deferred | accepted | 2026-05-25 | — |
36
36
  | [ADR-030](ADR-030-claude-code-command-projection.md) | Claude Code Command Projection | accepted | 2026-05-28 | — |
37
37
  | [ADR-031](ADR-031-validation-severity-tiers-and-projection-roundtrip.md) | Validation Severity Tiers And Projection Roundtrip | accepted | 2026-05-29 | — |
@@ -41,7 +41,7 @@ _Auto-generated by `scripts/adr/regenerate_index.py`. Do not edit._
41
41
  | [ADR-035](ADR-035-model-capability-tiers.md) | Model Capability Tiers | accepted | 2026-05-30 | 034 |
42
42
  | [ADR-036](ADR-036-global-install-browser-wizard-handoff.md) | Global Install Browser Wizard Handoff | accepted | 2026-06-01 | — |
43
43
  | [ADR-037](ADR-037-cost-profile-untangle.md) | Cost Profile Untangle | accepted | 2026-06-01 | — |
44
- | [ADR-038](ADR-038-canonical-settings-path.md) | Canonical Settings Path | — | — | — |
44
+ | [ADR-038](ADR-038-canonical-settings-path.md) | Canonical Settings Path | accepted | 2026-06-01 | — |
45
45
  | [ADR-039](ADR-039-claude-skills-untracked.md) | Claude Skills Untracked | accepted | 2026-06-01 | — |
46
46
  | [ADR-040](ADR-040-execution-model-projection-time-filtering.md) | Execution Model Projection Time Filtering | accepted | 2026-06-02 | — |
47
47
  | [ADR-041](ADR-041-controlled-command-verbs.md) | Controlled Command Verbs | accepted | 2026-06-03 | — |
@@ -96,7 +96,7 @@ _Auto-generated by `scripts/adr/regenerate_index.py`. Do not edit._
96
96
  | [ADR-090](ADR-090-visibility-command-frontmatter-field.md) | Visibility Command Frontmatter Field | accepted | 2026-06-13 | — |
97
97
  | [ADR-091](ADR-091-split-meta-capability-packs.md) | Split Meta Capability Packs | accepted | 2026-06-13 | — |
98
98
  | [ADR-092](ADR-092-defer-command-tier-alias-removal.md) | Defer Command Tier Alias Removal | accepted | 2026-06-13 | — |
99
- | [ADR-093](ADR-093-ai-council-config-user-global.md) | Ai Council Config User Global | accepted | 2026-06-13 | — |
99
+ | [ADR-093](ADR-093-ai-council-config-user-global.md) | Ai Council Config User Global | superseded | 2026-06-13 | — |
100
100
  | [ADR-094](ADR-094-agent-memory-layer-removal.md) | Agent Memory Layer Removal | accepted | 2026-06-14 | — |
101
101
  | [ADR-095](ADR-095-workspace-boundary-contract.md) | Workspace Boundary Contract | accepted | 2026-06-14 | — |
102
102
  | [ADR-096](ADR-096-analysis-workbench.md) | Analysis Workbench | accepted | 2026-06-15 | — |
@@ -106,7 +106,7 @@ _Auto-generated by `scripts/adr/regenerate_index.py`. Do not edit._
106
106
  | [ADR-100](ADR-100-global-knowledge-card-sharing.md) | Global Knowledge Card Sharing | accepted | 2026-06-15 | ADR-098 |
107
107
  | [ADR-101](ADR-101-ticket-bundle-emission.md) | Ticket Bundle Emission | proposed | 2026-06-15 | — |
108
108
  | [ADR-102](ADR-102-ticket-handoff-paste-and-mcp.md) | Ticket Handoff Paste And Mcp | accepted | 2026-06-16 | — |
109
- | [ADR-103](ADR-103-global-knowledge-default-off-until-measured.md) | Global Knowledge Default Off Until Measured | accepted | 2026-06-16 | — |
109
+ | [ADR-103](ADR-103-global-knowledge-default-off-until-measured.md) | Global Knowledge Default Off Until Measured | superseded | 2026-06-16 | — |
110
110
  | [ADR-104](ADR-104-ai-council-config-global-only.md) | Ai Council Config Global Only | accepted | 2026-06-22 | 093 |
111
111
  | [ADR-105](ADR-105-automatic-subagent-orchestration.md) | Automatic Subagent Orchestration | accepted | 2026-06-23 | — |
112
112
  | [ADR-106](ADR-106-recursive-verification-benchmark-gate.md) | Recursive Verification Benchmark Gate | accepted | 2026-06-23 | — |
@@ -117,14 +117,16 @@ _Auto-generated by `scripts/adr/regenerate_index.py`. Do not edit._
117
117
  | [ADR-111](ADR-111-canonical-glama-listing.md) | Canonical Glama Listing | accepted | 2026-07-07 | — |
118
118
  | [ADR-112](ADR-112-stdio-lite-stays-read-only.md) | Stdio Lite Stays Read Only | accepted | 2026-07-07 | — |
119
119
  | [ADR-113](ADR-113-ci-native-release-label-trigger.md) | Ci Native Release Label Trigger | accepted | 2026-07-08 | — |
120
- | [ADR-114](ADR-114-commit-policy-iron-law-override.md) | Commit Policy Iron Law Override | — | — | — |
120
+ | [ADR-114](ADR-114-commit-policy-iron-law-override.md) | Commit Policy Iron Law Override | accepted | 2026-07-08 | — |
121
121
  | [ADR-115](ADR-115-command-cluster-consolidation-phase-4.md) | Command Cluster Consolidation Phase 4 | accepted | 2026-07-08 | — |
122
122
  | [ADR-116](ADR-116-memory-tripwire-activation-path.md) | Memory Tripwire Activation Path | accepted | 2026-07-08 | — |
123
- | [ADR-117](ADR-117-subagents-auto-default-on.md) | Subagents Auto Default On | accepted | 2026-07-09 | 105 |
123
+ | [ADR-117](ADR-117-subagents-auto-default-on.md) | Subagents Auto Default On | accepted | 2026-07-09 | — |
124
+ | [ADR-118](ADR-118-loop-engineering-boundaries.md) | Loop Engineering Boundaries | accepted | 2026-07-10 | — |
125
+ | [ADR-119](ADR-119-global-knowledge-default-on.md) | Global Knowledge Default On | accepted | 2026-07-12 | 103 |
124
126
  | [ADR-200](ADR-200-python-to-typescript-migration.md) | Python To Typescript Migration | accepted | 2026-06-11 | — |
125
127
 
126
128
  ## Unnumbered (legacy)
127
129
 
128
130
  | # | Title | Status | Date | Supersedes |
129
131
  |---|---|---|---|---|
130
- | [ADR-rule-kernel-and-router](ADR-rule-kernel-and-router.md) | — | — | — | — |
132
+ | [ADR-rule-kernel-and-router](ADR-rule-kernel-and-router.md) | Rule Kernel And Router Roadmap Closure | accepted | 2026-05-06 | — |
@@ -0,0 +1,82 @@
1
+ # Cross-model parity eval — re-scoped design
2
+
3
+ > Design artifact for `road-to-opt-measurement-unblock` Phase 3 step 1. It
4
+ > re-scopes the archive's most-deferred item (deferred across 4+ roadmaps) to the
5
+ > **smallest capability** that produces a real cross-vendor parity signal, so the
6
+ > build-vs-defer decision (step 2) rests on a concrete design, not the generic
7
+ > "the harness doesn't exist". Writing this design is autonomous; **building and
8
+ > running it is a maintainer cost decision** (step 2/3), disclosed and gated.
9
+
10
+ ## What changed since the last deferral
11
+
12
+ | Original blocker | Status now |
13
+ |---|---|
14
+ | No delegable-task corpus | **RESOLVED** — `internal/bench/orchestration/corpus/` exists (`orch-*` orchestration tasks + `pv-*` production-validator tasks) |
15
+ | Harness cannot execute model-emitted subagent calls | **STILL TRUE** — do not assume it away; re-scope around it |
16
+
17
+ The corpus blocker no longer holds, so the item is re-openable — but only if we
18
+ stop trying to build the thing that is still blocked (a full in-host harness that
19
+ runs arbitrary model-emitted subagent calls).
20
+
21
+ ## The re-scope: council-transport execution, not an in-host harness
22
+
23
+ **Don't** build a harness that executes model-emitted subagent calls end-to-end
24
+ (the still-open blocker). **Do** reuse the capability that already exists: the AI
25
+ council transport dispatches the *same* prompt to *multiple vendors* through
26
+ `consult(members: ExternalAIClient[], …)` in
27
+ [`src/scripts/ai_council/orchestrator.ts`](../../src/scripts/ai_council/orchestrator.ts).
28
+
29
+ The smallest capability that yields a cross-model parity signal:
30
+
31
+ 1. **Task adapter.** Render each corpus task (`internal/bench/orchestration/corpus/*.md`)
32
+ into a single self-contained council prompt: the task + its rubric, asking the
33
+ model to *produce the findings it would surface* (not to actually spawn
34
+ subagents). This sidesteps the subagent-execution blocker — we measure the
35
+ model's **finding output on the delegable task**, not its ability to drive a
36
+ live subagent runtime.
37
+ 2. **≥ 2 vendors, same corpus.** Dispatch the identical prompt set to two distinct
38
+ vendors via the existing `ExternalAIClient` array — no new transport, no new
39
+ auth surface beyond what the council already has.
40
+ 3. **Per-host finding-count distribution.** For each (host × task), record the
41
+ count and the findings; aggregate to a per-host distribution over the corpus.
42
+ 4. **Blind judged quality (optional, gated).** If a quality signal beyond raw
43
+ counts is wanted, reuse the blind second-judge + Cohen's κ from
44
+ [`check_quality_regression.ts`](../../src/scripts/check_quality_regression.ts)
45
+ — the same discipline the Phase-1 rerun uses. This is the billable half; the
46
+ count distributions (step 3) are cheap and already informative.
47
+
48
+ **Explicitly out of scope:** running the models' emitted subagent calls, a
49
+ persistent multi-host runtime, any new vendor integration. If the signal needs
50
+ more than council-transport can give, that is a *separate, later* build — named,
51
+ not assumed.
52
+
53
+ ## What the signal feeds — `finding_floor` calibration
54
+
55
+ `finding_floor` is an **inert mechanism today** (referenced in
56
+ `run_skill_evals.ts` / `skill_linter.ts` / `evals.schema.json`); its recorded
57
+ deferral reason was exactly the missing calibration input this eval produces. The
58
+ per-host finding-count distributions calibrate the floor:
59
+
60
+ - Derive the floor from the **cross-host lower envelope** (e.g. a low percentile of
61
+ the per-task finding counts across vendors) so the gate fires on a host/task that
62
+ under-produces relative to the cross-vendor norm, not relative to one vendor.
63
+ - Once calibrated, promote `finding_floor` from inert to an **enforcing CI gate**
64
+ (its intended end state).
65
+
66
+ ## Build-vs-defer decision inputs (for step 2, maintainer)
67
+
68
+ - **Cost:** the count-distribution pass (steps 1–3) is `N_tasks × N_vendors`
69
+ council calls — small and cheap; render the `council:estimate`-class disclosure
70
+ before the first call. The judged-quality half (step 4) roughly doubles it.
71
+ - **Value:** unblocks `finding_floor` (arms a real CI gate) **and** produces the
72
+ first honest cross-vendor parity number the portfolio has ever had.
73
+ - **Defer condition:** if built later, park with the concrete missing capability
74
+ named — here it would be "council-transport task-adapter for the orchestration
75
+ corpus", never the generic "harness doesn't exist".
76
+
77
+ ## Exit criteria mapping
78
+
79
+ - This design doc **exists** (step 1 — done by writing this). ✓
80
+ - Explicit **build/defer decision recorded** (step 2 — maintainer, cost attached). ⏳
81
+ - If built: `finding_floor` calibration data exists and the gate is armed
82
+ (step 3 — gated on the decision + spend). ⏳