@event4u/agent-config 8.10.0 → 8.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +1 -1
- package/CHANGELOG.md +148 -0
- package/README.md +5 -5
- package/dist/agent-src/commands/challenge-me/vision.md +121 -32
- package/dist/agent-src/commands/challenge-me/with-docs.md +21 -8
- package/dist/agent-src/commands/commit/in-chunks.md +1 -1
- package/dist/agent-src/commands/commit.md +2 -1
- package/dist/agent-src/commands/cost/report.md +2 -1
- package/dist/agent-src/commands/feature/dev.md +4 -4
- package/dist/agent-src/commands/humanize.md +101 -0
- package/dist/agent-src/commands/jira-ticket.md +2 -2
- package/dist/agent-src/commands/memory/load.md +15 -0
- package/dist/agent-src/commands/optimize/project.md +12 -0
- package/dist/agent-src/commands/roadmap/create.md +12 -11
- package/dist/agent-src/commands/tests/create.md +1 -1
- package/dist/agent-src/commands/worktree/cleanup.md +22 -17
- package/dist/agent-src/commands/worktree/create.md +3 -1
- package/dist/agent-src/commands/worktree/status.md +12 -0
- package/dist/agent-src/contexts/execution/autonomy-mechanics.md +66 -0
- package/dist/agent-src/contexts/execution/orchestration-benchmark-gate.md +1 -0
- package/dist/agent-src/contexts/execution/orchestration-telemetry.md +47 -3
- package/dist/agent-src/contexts/execution/roadmap-process-loop.md +14 -0
- package/dist/agent-src/contexts/execution/roadmap-writing-source-derived.md +65 -0
- package/dist/agent-src/contexts/execution/subagent-modes-detail.md +84 -0
- package/dist/agent-src/rules/autonomous-execution.md +7 -37
- package/dist/agent-src/rules/broken-access-control.md +7 -45
- package/dist/agent-src/rules/code-comment-discipline.md +148 -0
- package/dist/agent-src/rules/context-hygiene.md +14 -0
- package/dist/agent-src/rules/delegation-policy.md +10 -8
- package/dist/agent-src/rules/design-fidelity.md +4 -0
- package/dist/agent-src/rules/git-history-discipline.md +4 -79
- package/dist/agent-src/rules/legal-safety-floor.md +11 -55
- package/dist/agent-src/rules/roadmap-progress-sync.md +5 -56
- package/dist/agent-src/rules/think-before-action.md +10 -0
- package/dist/agent-src/rules/verify-before-complete.md +12 -0
- package/dist/agent-src/skills/agent-docs-writing/SKILL.md +13 -12
- package/dist/agent-src/skills/ai-council/SKILL.md +19 -9
- package/dist/agent-src/skills/api-design/SKILL.md +7 -5
- package/dist/agent-src/skills/api-testing/SKILL.md +1 -1
- package/dist/agent-src/skills/authz-review/SKILL.md +59 -2
- package/dist/agent-src/skills/blast-radius-analyzer/SKILL.md +1 -1
- package/dist/agent-src/skills/bug-analyzer/SKILL.md +19 -16
- package/dist/agent-src/skills/code-refactoring/SKILL.md +9 -9
- package/dist/agent-src/skills/content-funnel-design/SKILL.md +2 -0
- package/dist/agent-src/skills/data-flow-mapper/SKILL.md +1 -1
- package/dist/agent-src/skills/defense-in-depth/SKILL.md +2 -2
- package/dist/agent-src/skills/dependency-upgrade/SKILL.md +7 -7
- package/dist/agent-src/skills/design-intelligence/SKILL.md +8 -8
- package/dist/agent-src/skills/design-review/SKILL.md +23 -2
- package/dist/agent-src/skills/doc-coauthoring/SKILL.md +2 -0
- package/dist/agent-src/skills/editorial-calendar/SKILL.md +2 -0
- package/dist/agent-src/skills/fe-design/SKILL.md +7 -9
- package/dist/agent-src/skills/git-workflow/SKILL.md +86 -0
- package/dist/agent-src/skills/humanizer/SKILL.md +147 -0
- package/dist/agent-src/skills/humanizer/data/patterns.md +190 -0
- package/dist/agent-src/skills/humanizer/evals/triggers.json +17 -0
- package/dist/agent-src/skills/laravel/SKILL.md +14 -1
- package/dist/agent-src/skills/launch-readiness/SKILL.md +20 -19
- package/dist/agent-src/skills/legal-practice-profile/SKILL.md +126 -0
- package/dist/agent-src/skills/logging-monitoring/SKILL.md +1 -1
- package/dist/agent-src/skills/mcp/SKILL.md +1 -1
- package/dist/agent-src/skills/memory-consolidation/SKILL.md +14 -1
- package/dist/agent-src/skills/migration-architect/SKILL.md +2 -1
- package/dist/agent-src/skills/performance/SKILL.md +1 -1
- package/dist/agent-src/skills/php-debugging/SKILL.md +17 -0
- package/dist/agent-src/skills/readme-writing/SKILL.md +2 -0
- package/dist/agent-src/skills/readme-writing-package/SKILL.md +2 -0
- package/dist/agent-src/skills/release-comms/SKILL.md +2 -0
- package/dist/agent-src/skills/roadmap-writing/SKILL.md +22 -51
- package/dist/agent-src/skills/security-audit/SKILL.md +6 -9
- package/dist/agent-src/skills/sentry-integration/SKILL.md +2 -2
- package/dist/agent-src/skills/skill-improvement-pipeline/SKILL.md +8 -0
- package/dist/agent-src/skills/source-discovery/SKILL.md +26 -24
- package/dist/agent-src/skills/subagent-orchestration/SKILL.md +55 -46
- package/dist/agent-src/skills/subagent-orchestration/evals/evals.json +42 -0
- package/dist/agent-src/skills/systematic-debugging/SKILL.md +10 -0
- package/dist/agent-src/skills/using-git-worktrees/SKILL.md +10 -0
- package/dist/agent-src/skills/worktree-lifecycle/SKILL.md +27 -20
- package/dist/agent-src/skills/worktree-lifecycle/evals/evals.json +38 -0
- package/dist/agent-src/templates/agent-settings.md +3 -2
- package/dist/agent-src/templates/agents/agent-project-settings.example.yml +15 -1
- package/dist/agent-src/templates/roadmaps.md +14 -0
- package/dist/cli/commands/recordTriggerEval.js +4 -14
- package/dist/cli/commands/recordTriggerEval.js.map +1 -1
- package/dist/discovery/deprecation-report.md +1 -1
- package/dist/discovery/discovery-manifest.json +147 -78
- package/dist/discovery/discovery-manifest.json.sha256 +1 -1
- package/dist/discovery/discovery-manifest.summary.md +6 -6
- package/dist/discovery/orphan-report.md +1 -1
- package/dist/discovery/packs.json +13 -10
- package/dist/discovery/trust-report.md +3 -3
- package/dist/discovery/workspaces.json +14 -9
- package/dist/install/install.mjs +12 -7
- package/dist/mcp/registry-manifest.json +4 -4
- package/dist/router.json +1 -1
- package/dist/scripts/_lib/claude_settings_hooks.js +8 -0
- package/dist/scripts/_lib/claude_settings_hooks.js.map +1 -1
- package/dist/scripts/_lib/trigger_eval_floors.js +24 -0
- package/dist/scripts/_lib/trigger_eval_floors.js.map +1 -0
- package/dist/scripts/condense.js +12 -2
- package/dist/scripts/condense.js.map +1 -1
- package/dist/server/schemas/settings.js +5 -2
- package/dist/server/schemas/settings.js.map +1 -1
- package/docs/CLAIMS.md +24 -3
- package/docs/architecture.md +3 -3
- package/docs/benchmark.md +51 -0
- package/docs/catalog.md +8 -5
- package/docs/command-flows.md +2 -1
- package/docs/contracts/ai-council-config.md +92 -0
- package/docs/contracts/command-clusters.md +1 -0
- package/docs/contracts/evidence-based-pruning.md +1 -0
- package/docs/contracts/hook-architecture-v1.md +25 -0
- package/docs/contracts/kernel-membership.md +35 -0
- package/docs/contracts/measurement-baseline.md +1 -0
- package/docs/contracts/release-sizing.md +22 -0
- package/docs/contracts/write-engine.md +38 -1
- package/docs/customization.md +12 -0
- package/docs/decisions/ADR-002-kernel-bucket-overrides.md +11 -0
- package/docs/decisions/ADR-003-flat-cluster-subs-and-colon-syntax.md +11 -0
- package/docs/decisions/ADR-006-skill-tools-python-pilot.md +1 -1
- package/docs/decisions/ADR-028-root-layout.md +1 -1
- package/docs/decisions/ADR-038-canonical-settings-path.md +10 -0
- package/docs/decisions/ADR-092-defer-command-tier-alias-removal.md +29 -5
- package/docs/decisions/ADR-093-ai-council-config-user-global.md +1 -1
- package/docs/decisions/ADR-100-global-knowledge-card-sharing.md +7 -0
- package/docs/decisions/ADR-103-global-knowledge-default-off-until-measured.md +2 -2
- package/docs/decisions/ADR-105-automatic-subagent-orchestration.md +1 -1
- package/docs/decisions/ADR-114-commit-policy-iron-law-override.md +11 -0
- package/docs/decisions/ADR-116-memory-tripwire-activation-path.md +11 -1
- package/docs/decisions/ADR-117-subagents-auto-default-on.md +2 -1
- package/docs/decisions/ADR-118-loop-engineering-boundaries.md +162 -0
- package/docs/decisions/ADR-119-global-knowledge-default-on.md +104 -0
- package/docs/decisions/ADR-200-python-to-typescript-migration.md +11 -1
- package/docs/decisions/ADR-rule-kernel-and-router.md +11 -0
- package/docs/decisions/INDEX.md +11 -9
- package/docs/design/cross-model-parity-eval.md +82 -0
- package/docs/design/length-neutral-judge-rerun.md +92 -0
- package/docs/featured-skills.md +2 -2
- package/docs/getting-started-by-role.md +1 -1
- package/docs/getting-started.md +2 -2
- package/docs/governance-advantage.md +1 -1
- package/docs/guidelines/agent-infra/roadmap-progress-mechanics.md +80 -5
- package/docs/guidelines/code-clarity.md +106 -1
- package/docs/guidelines/design-antipatterns.md +20 -0
- package/docs/guidelines/php/php-coding-patterns.md +12 -2
- package/docs/proof.md +20 -14
- package/docs/wedge/production-validator/production-validator.md +27 -0
- package/package.json +1 -1
- package/src/config/agent-settings.template.yml +29 -9
- package/src/scripts/_cli/cmd_doctor.ts +34 -1
- package/src/scripts/_lib/auto_dispatch.corpus.test.ts +87 -0
- package/src/scripts/_lib/claude_settings_hooks.ts +8 -0
- package/src/scripts/_lib/knowledge_global.ts +6 -2
- package/src/scripts/_lib/knowledge_global_redaction.ts +42 -4
- package/src/scripts/_lib/lexical_index.ts +15 -2
- package/src/scripts/_lib/orchestration_record.ts +14 -0
- package/src/scripts/_lib/trigger_eval_floors.ts +31 -0
- package/src/scripts/ai_council/chairman.ts +90 -0
- package/src/scripts/ai_council/clients.ts +18 -3
- package/src/scripts/ai_council/config.ts +157 -0
- package/src/scripts/ai_council/debate_gates.ts +89 -0
- package/src/scripts/ai_council/orchestrator.ts +65 -2
- package/src/scripts/ai_council/prompts.ts +121 -11
- package/src/scripts/ai_council/replay.ts +17 -0
- package/src/scripts/ai_council/stance_tally.ts +184 -0
- package/src/scripts/ai_tells_rules.ts +433 -0
- package/src/scripts/bench_humanizer_eval.ts +240 -0
- package/src/scripts/build_discovery_manifest.ts +6 -0
- package/src/scripts/build_proof.ts +21 -9
- package/src/scripts/check_claims.ts +40 -0
- package/src/scripts/check_rule_invariants.ts +10 -3
- package/src/scripts/condense.ts +16 -2
- package/src/scripts/consumer_matrix.ts +41 -2
- package/src/scripts/council_cli.ts +147 -0
- package/src/scripts/detect_ai_tells.ts +312 -0
- package/src/scripts/install.ts +12 -0
- package/src/scripts/knowledge_global_cli.ts +1 -1
- package/src/scripts/lint_framework_leakage.ts +88 -7
- package/src/scripts/lint_framework_leakage_allowlist.json +59 -572
- package/src/scripts/lint_knowledge_scale.ts +30 -0
- package/src/scripts/lint_originality_shingles.ts +210 -0
- package/src/scripts/orchestration_record.ts +1 -0
- package/src/scripts/schemas/agent-settings.schema.json +11 -0
- package/src/scripts/self_review_gate.ts +41 -15
- package/src/scripts/skill_eval_coverage.ts +71 -3
- package/src/scripts/skill_linter.ts +1 -0
- package/src/scripts/trigger_eval_rotation.ts +344 -0
- package/src/scripts/update_counts.ts +5 -17
- package/src/scripts/worktree_cleanup_check.ts +263 -0
- package/src/server/schemas/settings.ts +8 -3
- package/src/scripts/analysis_freshness.ts +0 -362
- package/src/scripts/export_replay_corpus.ts +0 -208
|
@@ -29,13 +29,16 @@ question with the Phase-3 investigation evidence.
|
|
|
29
29
|
Investigation findings (`road-to-metadata-and-command-surface-leanness`, Phase 3):
|
|
30
30
|
|
|
31
31
|
- **Integer-`tier` readers are internal:** `src/cli/commands/commands.ts`
|
|
32
|
-
(fallback after `visibility`), `src/scripts/audit_command_surface.
|
|
33
|
-
(report column + budget fallback), `src/scripts/build_discovery_manifest.
|
|
34
|
-
(**dual-emits `tier`** into the manifest). `src/cli/python/workspace_hosts.
|
|
32
|
+
(fallback after `visibility`), `src/scripts/audit_command_surface.ts`
|
|
33
|
+
(report column + budget fallback), `src/scripts/build_discovery_manifest.ts`
|
|
34
|
+
(**dual-emits `tier`** into the manifest). `src/cli/python/workspace_hosts.ts`
|
|
35
35
|
uses a semantically unrelated host-inventory tier.
|
|
36
|
+
<!-- paths updated 2026-07-12: every reader migrated to TS during the
|
|
37
|
+
completed ADR-200 py2ts migration; the original .py list is preserved in
|
|
38
|
+
git history. -->
|
|
36
39
|
- **The discovery manifest is a published npm artifact** — `package.json`
|
|
37
40
|
`files` + `build:discovery` in `prepack` + `prepublishOnly:
|
|
38
|
-
check_release_includes_discovery.
|
|
41
|
+
check_release_includes_discovery.ts`. It dual-emits the integer `tier`.
|
|
39
42
|
**External npm consumers that read the integer key are unknown.**
|
|
40
43
|
- The defer rests on a **Runtime Risk that cannot be ruled out** (a consumer
|
|
41
44
|
may register/branch on the integer `tier`), not on evidence that such a
|
|
@@ -75,7 +78,7 @@ forcing function so it cannot calcify into permanent debt:
|
|
|
75
78
|
## Consequences
|
|
76
79
|
|
|
77
80
|
- The `tier:` ⇄ `visibility:` dual-field persists during the deferral;
|
|
78
|
-
drift is mitigated by the `lint_command_tiers.
|
|
81
|
+
drift is mitigated by the `lint_command_tiers.ts` consistency check (when
|
|
79
82
|
both present, they must agree).
|
|
80
83
|
- The removal is **cheaply reversible** if ever executed: restoring the `tier`
|
|
81
84
|
field is a manifest schema patch (< 1h to publish), far below the rollback
|
|
@@ -84,6 +87,27 @@ forcing function so it cannot calcify into permanent debt:
|
|
|
84
87
|
- `road-to-tier-removal.md` institutionalises the trigger so the defer stays
|
|
85
88
|
visible in planning rather than becoming folklore.
|
|
86
89
|
|
|
90
|
+
## Addendum (2026-07-12) — re-open mechanism mid-flight
|
|
91
|
+
|
|
92
|
+
Recorded during `road-to-opt-decision-flips` Phase 2; not a re-litigation —
|
|
93
|
+
this ADR's own forcing function is executing:
|
|
94
|
+
|
|
95
|
+
- **All integer-`tier` readers are now TypeScript** (ADR-200 migration landed
|
|
96
|
+
on `main`; the reader list above updated in place).
|
|
97
|
+
- **Re-open mechanism #1 shipped:** `build_discovery_manifest.ts` emits the
|
|
98
|
+
machine-readable `deprecations` block + `sunset` field announcing the
|
|
99
|
+
`tier` deprecation (road-to-tier-removal Phase 1). Verified in the
|
|
100
|
+
**released** artifact: npm `8.10.0` (published 2026-07-10) ships
|
|
101
|
+
`dist/discovery/discovery-manifest.json` with
|
|
102
|
+
`deprecations[0] = {key: tier, replacement: visibility, since: ADR-092,
|
|
103
|
+
sunset: null}`.
|
|
104
|
+
- **Soak status:** the soak clock runs since publish; zero tier-related
|
|
105
|
+
breakage reports (issue sweep 2026-07-12). The unknown-external-consumer
|
|
106
|
+
hard stop is NOT lifted by the notice alone — the soak must elapse.
|
|
107
|
+
- **Execution continues in `road-to-tier-removal.md`**, revived from
|
|
108
|
+
`later/` to active on 2026-07-12 with the soak evidence cited in its
|
|
109
|
+
banner (portfolio-consolidation Phase 2).
|
|
110
|
+
|
|
87
111
|
## References
|
|
88
112
|
|
|
89
113
|
- [ADR-090](ADR-090-visibility-command-frontmatter-field.md) — the deferred
|
|
@@ -11,6 +11,13 @@ type: structural
|
|
|
11
11
|
|
|
12
12
|
# ADR-100 — Global cross-project knowledge-card sharing
|
|
13
13
|
|
|
14
|
+
> **Partial supersession (clarified 2026-07-12):** this ADR supersedes
|
|
15
|
+
> ADR-098 **Decision-10 only** (the measure-then-decide gate for the
|
|
16
|
+
> global layer); every other ADR-098 decision stands. The frontmatter
|
|
17
|
+
> `supersedes:` field carries the bare number because the index
|
|
18
|
+
> link-checker validates it; this note is the qualifier's home.
|
|
19
|
+
|
|
20
|
+
|
|
14
21
|
## Status
|
|
15
22
|
|
|
16
23
|
Accepted (2026-06-15). Council-designed (claude-sonnet-4-5 + gpt-4o, design
|
|
@@ -1,3 +1,14 @@
|
|
|
1
|
+
---
|
|
2
|
+
adr: 114
|
|
3
|
+
status: accepted
|
|
4
|
+
date: 2026-07-08
|
|
5
|
+
decision: commit-policy-iron-law-override
|
|
6
|
+
supersedes: —
|
|
7
|
+
superseded_by: —
|
|
8
|
+
phase: kernel-budget overrides (extends ADR-002)
|
|
9
|
+
type: structural
|
|
10
|
+
---
|
|
11
|
+
|
|
1
12
|
# ADR-114 — commit-policy Iron-Law Override (per-rule cap 2.5k → ceiling 4.0k)
|
|
2
13
|
|
|
3
14
|
- **Status:** Accepted (2026-07-08)
|
|
@@ -5,11 +5,21 @@ date: 2026-07-08
|
|
|
5
5
|
decision: memory-tripwire-activation-path
|
|
6
6
|
supersedes: —
|
|
7
7
|
superseded_by: —
|
|
8
|
-
phase: road-to-memory-retrieval-economy · Phase 6
|
|
8
|
+
phase: road-to-memory-retrieval-economy · Phase 6 (amended — see 2026-07-12 note)
|
|
9
9
|
---
|
|
10
10
|
|
|
11
11
|
# ADR-116 — Memory scale-tripwire activation path: one engine, measured at fire time
|
|
12
12
|
|
|
13
|
+
> **Amended in practice (recorded 2026-07-12):** the SQLite-FTS5 engine
|
|
14
|
+
> this ADR pre-decided was never built. The activation path was
|
|
15
|
+
> re-resolved by road-to-retrieval-substrate-hardening B2 (done
|
|
16
|
+
> 2026-07-09): a hand-rolled stdlib BM25 + trigram prefilter in
|
|
17
|
+
> `src/scripts/_lib/lexical_index.ts` (no SQLite, ADR-061 honoured),
|
|
18
|
+
> dormant until the `lint_knowledge_scale` tripwire fires (200/type,
|
|
19
|
+
> 500 total). The tripwire thresholds and measurement-at-activation
|
|
20
|
+
> discipline of this ADR stand unchanged; only the engine choice moved.
|
|
21
|
+
|
|
22
|
+
|
|
13
23
|
- **Deciders:** maintainer + AI council (anthropic/claude-sonnet-4-5 +
|
|
14
24
|
openai/gpt-4o; disposition debate 2 rounds + engine tie-break 2 rounds,
|
|
15
25
|
2026-07-08 — tie-break converged after the in-repo `node:sqlite`
|
|
@@ -3,10 +3,11 @@ adr: 117
|
|
|
3
3
|
status: accepted
|
|
4
4
|
date: 2026-07-09
|
|
5
5
|
decision: subagents-auto-default-on
|
|
6
|
-
supersedes:
|
|
6
|
+
supersedes: —
|
|
7
7
|
superseded_by: —
|
|
8
8
|
phase: auto-subagent-orchestration
|
|
9
9
|
type: structural
|
|
10
|
+
amends: 105 (Decision 2 only — the shipped default; the contract itself stands)
|
|
10
11
|
---
|
|
11
12
|
|
|
12
13
|
# ADR-117 — Flip `subagents.auto` shipped default to `on` (host-gated) on bounded-downside re-evaluation
|
|
@@ -0,0 +1,162 @@
|
|
|
1
|
+
---
|
|
2
|
+
adr: 118
|
|
3
|
+
status: accepted
|
|
4
|
+
date: 2026-07-10
|
|
5
|
+
decision: loop-engineering-boundaries
|
|
6
|
+
supersedes: —
|
|
7
|
+
superseded_by: —
|
|
8
|
+
phase: loop-engineering
|
|
9
|
+
type: structural
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
# ADR-118 — Loop-engineering boundaries: one closure, four rejections, three deferrals, zero new loop surfaces
|
|
13
|
+
|
|
14
|
+
## Status
|
|
15
|
+
|
|
16
|
+
**Accepted** · 2026-07-10. Resolved by web research (five external articles + an
|
|
17
|
+
independent sweep) ground-truthed against the repo's loop inventory, plus an
|
|
18
|
+
AI-council debate (anthropic/claude-sonnet-4-5 + openai/gpt-4o, 2 rounds,
|
|
19
|
+
2026-07-10, converged). Executed by `road-to-loop-engineering`.
|
|
20
|
+
|
|
21
|
+
## Context
|
|
22
|
+
|
|
23
|
+
The June-2026 "loop engineering" wave (design the system that prompts the
|
|
24
|
+
agent: verification signal, budget shape, maker/checker topology, on-disk
|
|
25
|
+
state) is mostly a rebranding of ReAct + eval-driven development + bounded
|
|
26
|
+
autonomous agents. Its two durable contributions are the design vocabulary and
|
|
27
|
+
a precise failure-mode taxonomy: verification collapse ("done" is a claim, not
|
|
28
|
+
proof), comprehension debt, cognitive surrender, reward hacking (measured at
|
|
29
|
+
~15% of SE-task runs in one external evaluation), cost blowup on stuck retry
|
|
30
|
+
loops, context rot on long runs.
|
|
31
|
+
|
|
32
|
+
This package already implements the serious end of that canon: bounded
|
|
33
|
+
verify-fix loops (`verify-repair-loop` max_attempts=3, the N=3 validation
|
|
34
|
+
budget, context-hygiene 15/25 aborts), maker/checker separation (`judge-*`
|
|
35
|
+
cluster, `verify-budget`), human-gated self-improvement (5-stage pipeline,
|
|
36
|
+
≥2 independent evidence), and measurement infrastructure (bench:ab, trigger
|
|
37
|
+
evals, golden sets, orchestration telemetry, local analytics).
|
|
38
|
+
|
|
39
|
+
An inventory identified 8 places where a measure exists but no adjust-step is
|
|
40
|
+
wired. The question this ADR settles: which of those loops to close, and which
|
|
41
|
+
loop-shaped proposals to reject so they are not relitigated per hype cycle.
|
|
42
|
+
|
|
43
|
+
Constraint frame: single maintainer, token-frugality canon, no runtime daemon
|
|
44
|
+
(ADR-088 / no-runtime-boundary), human gates on anything irreversible. Under
|
|
45
|
+
these constraints, when human review + `git revert` is cheaper than designing,
|
|
46
|
+
maintaining, and debugging a closed loop, the loop is net-negative
|
|
47
|
+
infrastructure — false positives consume the binding resource (maintainer
|
|
48
|
+
attention), not compute.
|
|
49
|
+
|
|
50
|
+
## Decision
|
|
51
|
+
|
|
52
|
+
### 1. Automation threshold (the core principle)
|
|
53
|
+
|
|
54
|
+
Automate a measure→adjust loop **only** when all three hold:
|
|
55
|
+
|
|
56
|
+
1. the metric is a **direct** measure of the failure mode (not a proxy or a
|
|
57
|
+
correlation),
|
|
58
|
+
2. the expected false-positive rate is low (order <5%), and
|
|
59
|
+
3. human judgment adds no unique information to the adjust decision.
|
|
60
|
+
|
|
61
|
+
"A number exists" is not "the number is trustworthy" (metric-existence
|
|
62
|
+
fallacy). Reversibility of the adjust-step alone never justifies automation:
|
|
63
|
+
investigating one false-positive automated action costs 20–45 minutes of
|
|
64
|
+
maintainer time against a 10-second revert.
|
|
65
|
+
|
|
66
|
+
### 2. Disposition of the 8 open loops
|
|
67
|
+
|
|
68
|
+
| # | Open loop | Disposition |
|
|
69
|
+
|---|---|---|
|
|
70
|
+
| 1 | Orchestration telemetry → `subagents.auto` demotion | **Manual by decision.** token_delta spikes for legitimate reasons; demotion needs qualitative judgment of the failure mode. The demotion path in `orchestration-benchmark-gate.md` stays a human one-line edit. |
|
|
71
|
+
| 2 | Artifact-engagement counts → skill pruning | **Never automated.** Selection bias both ways (low engagement ≠ low value: security / error-recovery / niche skills; high engagement ≠ high value). Engagement stays ONE input under `evidence-based-pruning`'s ≥2-independent-evidence floor; the pruning decision is always human. |
|
|
72
|
+
| 3 | Trigger-eval pass rate on cadence | **Closed** — the only closure. Structure/freshness/presence gates already ran in `task ci`; this ADR's roadmap adds a rotating live pass-rate measurement to the existing weekly canary (see §4). Direct metric, low false-positive, adjust-step = surfaced report. |
|
|
73
|
+
| 4 | bench-drift → blocking gate | **Stays advisory.** High false-positive proxy (timing variance, intentional behaviour changes); the manual `bench:baseline-ready` flip remains the enforcement gate. |
|
|
74
|
+
| 5 | Golden-set coverage ratchet (14/91 rules) | **Deferred; growth stays opportunistic.** A hard per-rule ratchet pressures toward mechanically-testable rules over useful rules; most uncovered rules are judgment-shaped. `road-to-golden-set-coverage` remains the deliberate path. |
|
|
75
|
+
| 6 | Live-app verdict source | **Deferred, unchanged** — owned by `road-to-live-app-verdict`; the package ships config, not apps. |
|
|
76
|
+
| 7 | Measured rule-adherence signal | **Rejected.** Annotated-corpus + threshold-tuning build with ongoing maintenance for marginally earlier restatement; ADR-054's coarse heuristics are intentional. |
|
|
77
|
+
| 8 | Success-signal re-evaluation at eval date | **Not automated.** Retroactive evaluation of sunk promotion decisions has low marginal information; active use surfaces bad promotions. The field stays authored documentation. |
|
|
78
|
+
|
|
79
|
+
### 3. Five written rejections (loop anti-patterns for this package)
|
|
80
|
+
|
|
81
|
+
1. **Open-ended hill-climbing on the config.** No loop may "optimize"
|
|
82
|
+
rules/skills via iterated mutation + metric without a human checkpoint per
|
|
83
|
+
iteration; the bench:ab soak + human decision is the floor for config
|
|
84
|
+
change evaluation.
|
|
85
|
+
2. **Autonomous config self-editing.** The self-improvement pipeline stays
|
|
86
|
+
human-gated at every stage; no loop commits changes to `src/skills/` or
|
|
87
|
+
`src/rules/` without human approval. (Restates the existing pipeline
|
|
88
|
+
contract as a loop boundary.)
|
|
89
|
+
3. **Unattended multi-hour runs without checkpoints.** Loops must remain
|
|
90
|
+
reviewable/haltable on the maintainer's schedule; the existing halt
|
|
91
|
+
conditions and N=3 budgets are the ceiling, not a starting point.
|
|
92
|
+
4. **Metric-optimizing loops without golden-set anchoring.** No loop whose
|
|
93
|
+
success criterion is "metric goes up" without a golden-set eval anchoring
|
|
94
|
+
what "good" means (Goodhart guard).
|
|
95
|
+
5. **Cross-session loop state in external services.** No vector DBs, job
|
|
96
|
+
queues, or external orchestrators as loop state (restates ADR-088 in loop
|
|
97
|
+
context): state lives in git-tracked files + the conversation.
|
|
98
|
+
|
|
99
|
+
Also rejected, same reasoning tier: a **loop-design contract/checklist
|
|
100
|
+
artifact** (premature formalization — the caps/judges/state patterns are
|
|
101
|
+
embedded in the existing loop surfaces and enforced by existing rules) and
|
|
102
|
+
**consumer-facing loop templates** (`/loop:design` et al. — scope creep;
|
|
103
|
+
agents answer loop-design questions in conversation using the package's
|
|
104
|
+
existing vocabulary). **No new loop surfaces**: no run-until-condition goal
|
|
105
|
+
command, no fresh-context re-loop mode for roadmap processing (bounded-scope
|
|
106
|
+
by design), no separate nightly self-check workflow (folded into the existing
|
|
107
|
+
weekly canary).
|
|
108
|
+
|
|
109
|
+
### 4. The one closure — periodic live trigger-eval pass rate
|
|
110
|
+
|
|
111
|
+
The existing weekly cross-model canary's secrets-gated live tier gains a
|
|
112
|
+
deterministic rotation over the skills that carry `evals/triggers.json`
|
|
113
|
+
(ISO-week keyed, ~5 suites/week), enforcing the per-domain precision/recall
|
|
114
|
+
floors from `eval:record`. A floor breach fails the **scheduled** job — the
|
|
115
|
+
failure is the maintainer notification; PRs are never blocked by live results.
|
|
116
|
+
The local interactive confirmation gate on `skill_trigger_eval` is untouched;
|
|
117
|
+
CI authorization derives exclusively from workflow-provided key secrets, and
|
|
118
|
+
the no-secrets path stays a logged no-op.
|
|
119
|
+
|
|
120
|
+
## Revisit-if
|
|
121
|
+
|
|
122
|
+
- **Any rejection above:** reopen only on evidence that closing the loop saves
|
|
123
|
+
>2 h/month maintainer time in a structurally similar single-maintainer
|
|
124
|
+
project (logged hours, not vibes) — via `decision-revisit-gate`
|
|
125
|
+
mechanism-match, then council.
|
|
126
|
+
- **The §4 closure:** demote the canary pass-rate job to advisory
|
|
127
|
+
(non-failing) if breaches are >~10% spurious over 50+ suite-runs.
|
|
128
|
+
- **§2 #5 coverage ratchet:** reconsider if ≥3 real regressions ship in rules
|
|
129
|
+
that a golden task would have caught (counted, cited incidents).
|
|
130
|
+
|
|
131
|
+
## Consequences
|
|
132
|
+
|
|
133
|
+
- Future "should we automate X?" loop proposals test against §1 before any
|
|
134
|
+
build; the disposition table is the precedent record.
|
|
135
|
+
- The weekly canary becomes the single periodic live-eval surface — additions
|
|
136
|
+
to periodic self-checking extend it rather than adding workflows.
|
|
137
|
+
- Skill-pruning, default-demotion, and drift-blocking proposals cite this ADR
|
|
138
|
+
instead of re-deriving the argument.
|
|
139
|
+
|
|
140
|
+
## Alternatives considered
|
|
141
|
+
|
|
142
|
+
- **Close more loops (coverage ratchet, engagement pruning, auto-demotion):**
|
|
143
|
+
rejected — each fails at least one §1 criterion; the council's round-2
|
|
144
|
+
convergence was explicit that "metric exists + reversible" is insufficient.
|
|
145
|
+
- **Loop-design contract:** rejected as compliance theater for a
|
|
146
|
+
single-maintainer project; revisit on a second maintainer with real handoff
|
|
147
|
+
friction.
|
|
148
|
+
- **Do nothing (pure documentation):** rejected — the trigger-eval pass-rate
|
|
149
|
+
gap is real, direct-metric, and closable inside an existing workflow at
|
|
150
|
+
near-zero maintenance cost.
|
|
151
|
+
|
|
152
|
+
## References
|
|
153
|
+
|
|
154
|
+
- `road-to-loop-engineering` roadmap (execution + provenance; external source
|
|
155
|
+
links retained encrypted there per `source-confidentiality`).
|
|
156
|
+
- `docs/contracts/evidence-based-pruning.md` — §2 #2 anchor.
|
|
157
|
+
- `src/agent-src/contexts/execution/orchestration-benchmark-gate.md` — §2 #1 anchor.
|
|
158
|
+
- `docs/contracts/measurement-baseline.md` — §2 #4 anchor.
|
|
159
|
+
- ADR-088 (no external runtime), ADR-054 (adherence restate), ADR-106/-109/-117
|
|
160
|
+
(orchestration gates), `docs/contracts/no-runtime-boundary.md`.
|
|
161
|
+
- Prior honest-nulls this ADR does not reopen: recursive-verification
|
|
162
|
+
(2026-06-24), enforcement-projection, reminder-injection.
|
|
@@ -0,0 +1,104 @@
|
|
|
1
|
+
---
|
|
2
|
+
adr: 119
|
|
3
|
+
status: accepted
|
|
4
|
+
date: 2026-07-12
|
|
5
|
+
decision: global-knowledge-default-on
|
|
6
|
+
supersedes: 103
|
|
7
|
+
superseded_by: —
|
|
8
|
+
phase: opt-decision-flips
|
|
9
|
+
type: structural
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
# ADR-119 — Global knowledge sharing defaults ON (validated bounded-downside flip)
|
|
13
|
+
|
|
14
|
+
## Status
|
|
15
|
+
|
|
16
|
+
Accepted (2026-07-12). Supersedes ADR-103.
|
|
17
|
+
|
|
18
|
+
## Context
|
|
19
|
+
|
|
20
|
+
ADR-103 (2026-06-16) kept `knowledge.global_sharing.enabled: false` until
|
|
21
|
+
cross-project reuse was measured across ≥ 2 projects. Verified state at
|
|
22
|
+
re-evaluation (2026-07-11): the measurement instrument
|
|
23
|
+
(`src/scripts/_lib/knowledge_global_promote.ts`) shipped, but no sightings
|
|
24
|
+
data, no global store, and no reuse record existed — reuse can only accrue
|
|
25
|
+
while the layer is ON, and ON was withheld pending reuse. The gate could
|
|
26
|
+
never fire by construction: the same self-locking-measurement deadlock
|
|
27
|
+
ADR-117 broke for `subagents.auto` on 2026-07-09.
|
|
28
|
+
|
|
29
|
+
A two-round AI-council re-evaluation ran on 2026-07-11 (members:
|
|
30
|
+
anthropic/claude-sonnet-4-5, openai/gpt-4o; cost $0.10). Round 1: both
|
|
31
|
+
members stanced **Option A — flip ON now** (deadlock + ADR-117 precedent +
|
|
32
|
+
shipped guards). Round 2 dissent (gpt-4o): the redaction floor was
|
|
33
|
+
unvalidated for cross-project adversarial scenarios — composite inference,
|
|
34
|
+
homoglyph/encoding smuggling, temporal context collapse. The named
|
|
35
|
+
evidence-that-would-change-my-mind: demonstrating the redaction mechanism
|
|
36
|
+
handles those classes. Convergence: **flip WITH validation** — the dissent
|
|
37
|
+
becomes a pre-flip gate, not a blocker.
|
|
38
|
+
|
|
39
|
+
## Decision
|
|
40
|
+
|
|
41
|
+
1. **Pre-flip validation executed (2026-07-12).** Adversarial spot-check in
|
|
42
|
+
`tests/scripts/_lib_knowledge_global_redaction_adversarial.test.ts` +
|
|
43
|
+
`tests/fixtures/global-knowledge-redaction/` covering the three named
|
|
44
|
+
classes. Honest result: the zero-width-smuggling probe FAILED on first
|
|
45
|
+
run — the dissent's predicted gap was real. Closed by an additive
|
|
46
|
+
hardening in `knowledge_global_redaction.ts` (`hidden_unicode` violation
|
|
47
|
+
class + strip-then-rescan); homoglyph emails were already caught
|
|
48
|
+
(Unicode-aware pattern); temporal collapse is bounded by the freshness
|
|
49
|
+
state machine (fresh → hypothesis → stale). All 20 tests green,
|
|
50
|
+
including the 11 pre-existing parity tests.
|
|
51
|
+
2. **`knowledge.global_sharing.enabled` defaults to `true`** in the shipped
|
|
52
|
+
template.
|
|
53
|
+
3. **Narrowest tier default:** `allowed_tiers: [public]` (down from
|
|
54
|
+
`[public, vendor]`); `vendor` becomes a deliberate opt-in widening.
|
|
55
|
+
`proprietary` stays hard-coded manual-only in the gate.
|
|
56
|
+
4. **Pre-registered demotion trigger:** any cross-project PII/secret
|
|
57
|
+
sighting in a shared card — a confirmed identifier that crossed a
|
|
58
|
+
project boundary despite the gate — auto-reverts the shipped default to
|
|
59
|
+
`false` and lands an incident note under `agents/settings/contexts/`.
|
|
60
|
+
This is the ADR-117 demotion-gate pattern applied to knowledge sharing.
|
|
61
|
+
5. **Measurement window:** 60–90 days from release. The reuse data the
|
|
62
|
+
ADR-103 gate wanted now accrues; at window end the default is either
|
|
63
|
+
confirmed (reuse observed, no incidents) or reverted (no reuse — the
|
|
64
|
+
layer is dead weight — or a demotion trigger fired).
|
|
65
|
+
6. **Accepted residual, pinned by test:** k-anonymity over pure
|
|
66
|
+
quasi-identifier combinations is out of scope for a write-time text
|
|
67
|
+
gate. The council accepted this on the single-install trust boundary
|
|
68
|
+
(an actor with filesystem access to the global store already reads the
|
|
69
|
+
raw projects, which are strictly more sensitive). The boundary is
|
|
70
|
+
asserted by a dedicated test so any future change to it is a deliberate
|
|
71
|
+
decision, not drift.
|
|
72
|
+
|
|
73
|
+
## Consequences
|
|
74
|
+
|
|
75
|
+
- Cross-project knowledge cards flow by default on the narrowest tier,
|
|
76
|
+
through the unchanged suggest-then-confirm promotion flow
|
|
77
|
+
(`auto_promote_threshold` gates a suggestion, never a silent write) and
|
|
78
|
+
the hardened redaction gate (halt-and-surface, never silent-rewrite).
|
|
79
|
+
- Single-key revert: `knowledge.global_sharing.enabled: false` fully
|
|
80
|
+
no-ops the layer; v1 project-local cards are unaffected either way.
|
|
81
|
+
- No new infrastructure: file-first store, no vectors, no services — the
|
|
82
|
+
ADR-094 Layer-2 sunset is untouched.
|
|
83
|
+
- The hidden-unicode hardening also protects every other consumer of
|
|
84
|
+
`redaction_scan` (the Phase-4 CI linter over committed global cards).
|
|
85
|
+
|
|
86
|
+
## Alternatives considered
|
|
87
|
+
|
|
88
|
+
- **Keep OFF + opt-in campaign (Option B):** rejected — self-selection
|
|
89
|
+
bias, slow, and it retains the deadlock for everyone who does not
|
|
90
|
+
opt in.
|
|
91
|
+
- **Maintainer-org pilot first (Option C):** rejected as the primary path —
|
|
92
|
+
the council judged the guarded flip's downside bounded enough that a
|
|
93
|
+
pilot's extra latency buys little; the demotion trigger plus the
|
|
94
|
+
measurement window deliver the same safety net in production shape.
|
|
95
|
+
|
|
96
|
+
## References
|
|
97
|
+
|
|
98
|
+
- ADR-103 (superseded), ADR-117 (deadlock-flip precedent), ADR-100/-094
|
|
99
|
+
(store shape + Layer-2 sunset).
|
|
100
|
+
- Council session 2026-07-11 (claude-sonnet-4-5 + gpt-4o, 2 rounds,
|
|
101
|
+
convergence inlined above per `no-roadmap-references`).
|
|
102
|
+
- `tests/scripts/_lib_knowledge_global_redaction_adversarial.test.ts` —
|
|
103
|
+
the validation gate.
|
|
104
|
+
- `agents/roadmaps/road-to-opt-decision-flips.md` Phase 1 (execution).
|
|
@@ -5,12 +5,22 @@ date: 2026-06-11
|
|
|
5
5
|
decision: python-to-typescript-migration
|
|
6
6
|
supersedes: —
|
|
7
7
|
superseded_by: —
|
|
8
|
-
phase: py2ts · migration-infrastructure
|
|
8
|
+
phase: py2ts · migration-infrastructure (historical record — migration landed on main)
|
|
9
9
|
type: structural
|
|
10
10
|
---
|
|
11
11
|
|
|
12
12
|
# ADR-200 — Full Python → TypeScript migration: execution model, branch strategy, dispatcher, parity gates
|
|
13
13
|
|
|
14
|
+
> **Number retained by decision (2026-07-12):** the provisional-number
|
|
15
|
+
> clause ("final number assigned at the python2ts → main merge") has
|
|
16
|
+
> fired — the migration is complete on `main` (0 tracked `.py` under
|
|
17
|
+
> `src/`). Renumbering was evaluated and declined: 521 files reference
|
|
18
|
+
> "ADR-200" as a stable identifier; a rename would be a 521-file
|
|
19
|
+
> mechanical churn (projections, condensation hashes, docs) for zero
|
|
20
|
+
> behavioral gain. ADR-200 stays the permanent number; the record is
|
|
21
|
+
> historical (migration architecture, landed).
|
|
22
|
+
|
|
23
|
+
|
|
14
24
|
> Provisional number 200 (reserved, out-of-band). main ADRs grow sequentially; the migration ADR is renumbered to a high reserved slot to stop recurring collisions on the long-lived `python2ts` branch. Final number is assigned at the `python2ts`→`main` merge.
|
|
15
25
|
|
|
16
26
|
## Status
|
|
@@ -1,3 +1,14 @@
|
|
|
1
|
+
---
|
|
2
|
+
adr: —
|
|
3
|
+
status: accepted
|
|
4
|
+
date: 2026-05-06
|
|
5
|
+
decision: rule-kernel-and-router-roadmap-closure
|
|
6
|
+
supersedes: —
|
|
7
|
+
superseded_by: —
|
|
8
|
+
phase: road-to-kernel-and-router (closure record)
|
|
9
|
+
type: structural
|
|
10
|
+
---
|
|
11
|
+
|
|
1
12
|
# ADR — Rule Kernel and Router (Roadmap Closure)
|
|
2
13
|
|
|
3
14
|
- **Status:** Accepted (2026-05-06)
|
package/docs/decisions/INDEX.md
CHANGED
|
@@ -5,8 +5,8 @@ _Auto-generated by `scripts/adr/regenerate_index.py`. Do not edit._
|
|
|
5
5
|
| # | Title | Status | Date | Supersedes |
|
|
6
6
|
|---|---|---|---|---|
|
|
7
7
|
| [ADR-001](ADR-001-kernel-swap-deferred.md) | Kernel Swap Deferred | accepted | 2026-05-06 | — |
|
|
8
|
-
| [ADR-002](ADR-002-kernel-bucket-overrides.md) | Kernel Bucket Overrides |
|
|
9
|
-
| [ADR-003](ADR-003-flat-cluster-subs-and-colon-syntax.md) | Flat Cluster Subs And Colon Syntax |
|
|
8
|
+
| [ADR-002](ADR-002-kernel-bucket-overrides.md) | Kernel Bucket Overrides | accepted | 2026-05-06 | — |
|
|
9
|
+
| [ADR-003](ADR-003-flat-cluster-subs-and-colon-syntax.md) | Flat Cluster Subs And Colon Syntax | accepted | 2026-05-07 | — |
|
|
10
10
|
| [ADR-004](ADR-004-rule-governance-pruning.md) | Rule Governance Pruning | accepted | 2026-05-08 | — |
|
|
11
11
|
| [ADR-005](ADR-005-subagent-worktrees.md) | Subagent Worktrees No Auto Merge | accepted | 2026-05-09 | — |
|
|
12
12
|
| [ADR-006](ADR-006-skill-tools-python-pilot.md) | Skill Tools Python Pilot Pass | accepted | 2026-05-09 | — |
|
|
@@ -31,7 +31,7 @@ _Auto-generated by `scripts/adr/regenerate_index.py`. Do not edit._
|
|
|
31
31
|
| [ADR-025](ADR-025-workspace-chrome.md) | Workspace Chrome | accepted | 2026-05-24 | — |
|
|
32
32
|
| [ADR-026](ADR-026-explain-mode-translation.md) | Explain Mode Translation | accepted | 2026-05-24 | — |
|
|
33
33
|
| [ADR-027](ADR-027-changelog-machine-vs-manual.md) | Changelog Machine Vs Manual | accepted | 2026-05-25 | — |
|
|
34
|
-
| [ADR-028](ADR-028-root-layout.md) | Root Layout |
|
|
34
|
+
| [ADR-028](ADR-028-root-layout.md) | Root Layout | superseded | 2026-05-25 | — |
|
|
35
35
|
| [ADR-029](ADR-029-multi-workspace-deferred.md) | Multi Workspace Deferred | accepted | 2026-05-25 | — |
|
|
36
36
|
| [ADR-030](ADR-030-claude-code-command-projection.md) | Claude Code Command Projection | accepted | 2026-05-28 | — |
|
|
37
37
|
| [ADR-031](ADR-031-validation-severity-tiers-and-projection-roundtrip.md) | Validation Severity Tiers And Projection Roundtrip | accepted | 2026-05-29 | — |
|
|
@@ -41,7 +41,7 @@ _Auto-generated by `scripts/adr/regenerate_index.py`. Do not edit._
|
|
|
41
41
|
| [ADR-035](ADR-035-model-capability-tiers.md) | Model Capability Tiers | accepted | 2026-05-30 | 034 |
|
|
42
42
|
| [ADR-036](ADR-036-global-install-browser-wizard-handoff.md) | Global Install Browser Wizard Handoff | accepted | 2026-06-01 | — |
|
|
43
43
|
| [ADR-037](ADR-037-cost-profile-untangle.md) | Cost Profile Untangle | accepted | 2026-06-01 | — |
|
|
44
|
-
| [ADR-038](ADR-038-canonical-settings-path.md) | Canonical Settings Path |
|
|
44
|
+
| [ADR-038](ADR-038-canonical-settings-path.md) | Canonical Settings Path | accepted | 2026-06-01 | — |
|
|
45
45
|
| [ADR-039](ADR-039-claude-skills-untracked.md) | Claude Skills Untracked | accepted | 2026-06-01 | — |
|
|
46
46
|
| [ADR-040](ADR-040-execution-model-projection-time-filtering.md) | Execution Model Projection Time Filtering | accepted | 2026-06-02 | — |
|
|
47
47
|
| [ADR-041](ADR-041-controlled-command-verbs.md) | Controlled Command Verbs | accepted | 2026-06-03 | — |
|
|
@@ -96,7 +96,7 @@ _Auto-generated by `scripts/adr/regenerate_index.py`. Do not edit._
|
|
|
96
96
|
| [ADR-090](ADR-090-visibility-command-frontmatter-field.md) | Visibility Command Frontmatter Field | accepted | 2026-06-13 | — |
|
|
97
97
|
| [ADR-091](ADR-091-split-meta-capability-packs.md) | Split Meta Capability Packs | accepted | 2026-06-13 | — |
|
|
98
98
|
| [ADR-092](ADR-092-defer-command-tier-alias-removal.md) | Defer Command Tier Alias Removal | accepted | 2026-06-13 | — |
|
|
99
|
-
| [ADR-093](ADR-093-ai-council-config-user-global.md) | Ai Council Config User Global |
|
|
99
|
+
| [ADR-093](ADR-093-ai-council-config-user-global.md) | Ai Council Config User Global | superseded | 2026-06-13 | — |
|
|
100
100
|
| [ADR-094](ADR-094-agent-memory-layer-removal.md) | Agent Memory Layer Removal | accepted | 2026-06-14 | — |
|
|
101
101
|
| [ADR-095](ADR-095-workspace-boundary-contract.md) | Workspace Boundary Contract | accepted | 2026-06-14 | — |
|
|
102
102
|
| [ADR-096](ADR-096-analysis-workbench.md) | Analysis Workbench | accepted | 2026-06-15 | — |
|
|
@@ -106,7 +106,7 @@ _Auto-generated by `scripts/adr/regenerate_index.py`. Do not edit._
|
|
|
106
106
|
| [ADR-100](ADR-100-global-knowledge-card-sharing.md) | Global Knowledge Card Sharing | accepted | 2026-06-15 | ADR-098 |
|
|
107
107
|
| [ADR-101](ADR-101-ticket-bundle-emission.md) | Ticket Bundle Emission | proposed | 2026-06-15 | — |
|
|
108
108
|
| [ADR-102](ADR-102-ticket-handoff-paste-and-mcp.md) | Ticket Handoff Paste And Mcp | accepted | 2026-06-16 | — |
|
|
109
|
-
| [ADR-103](ADR-103-global-knowledge-default-off-until-measured.md) | Global Knowledge Default Off Until Measured |
|
|
109
|
+
| [ADR-103](ADR-103-global-knowledge-default-off-until-measured.md) | Global Knowledge Default Off Until Measured | superseded | 2026-06-16 | — |
|
|
110
110
|
| [ADR-104](ADR-104-ai-council-config-global-only.md) | Ai Council Config Global Only | accepted | 2026-06-22 | 093 |
|
|
111
111
|
| [ADR-105](ADR-105-automatic-subagent-orchestration.md) | Automatic Subagent Orchestration | accepted | 2026-06-23 | — |
|
|
112
112
|
| [ADR-106](ADR-106-recursive-verification-benchmark-gate.md) | Recursive Verification Benchmark Gate | accepted | 2026-06-23 | — |
|
|
@@ -117,14 +117,16 @@ _Auto-generated by `scripts/adr/regenerate_index.py`. Do not edit._
|
|
|
117
117
|
| [ADR-111](ADR-111-canonical-glama-listing.md) | Canonical Glama Listing | accepted | 2026-07-07 | — |
|
|
118
118
|
| [ADR-112](ADR-112-stdio-lite-stays-read-only.md) | Stdio Lite Stays Read Only | accepted | 2026-07-07 | — |
|
|
119
119
|
| [ADR-113](ADR-113-ci-native-release-label-trigger.md) | Ci Native Release Label Trigger | accepted | 2026-07-08 | — |
|
|
120
|
-
| [ADR-114](ADR-114-commit-policy-iron-law-override.md) | Commit Policy Iron Law Override |
|
|
120
|
+
| [ADR-114](ADR-114-commit-policy-iron-law-override.md) | Commit Policy Iron Law Override | accepted | 2026-07-08 | — |
|
|
121
121
|
| [ADR-115](ADR-115-command-cluster-consolidation-phase-4.md) | Command Cluster Consolidation Phase 4 | accepted | 2026-07-08 | — |
|
|
122
122
|
| [ADR-116](ADR-116-memory-tripwire-activation-path.md) | Memory Tripwire Activation Path | accepted | 2026-07-08 | — |
|
|
123
|
-
| [ADR-117](ADR-117-subagents-auto-default-on.md) | Subagents Auto Default On | accepted | 2026-07-09 |
|
|
123
|
+
| [ADR-117](ADR-117-subagents-auto-default-on.md) | Subagents Auto Default On | accepted | 2026-07-09 | — |
|
|
124
|
+
| [ADR-118](ADR-118-loop-engineering-boundaries.md) | Loop Engineering Boundaries | accepted | 2026-07-10 | — |
|
|
125
|
+
| [ADR-119](ADR-119-global-knowledge-default-on.md) | Global Knowledge Default On | accepted | 2026-07-12 | 103 |
|
|
124
126
|
| [ADR-200](ADR-200-python-to-typescript-migration.md) | Python To Typescript Migration | accepted | 2026-06-11 | — |
|
|
125
127
|
|
|
126
128
|
## Unnumbered (legacy)
|
|
127
129
|
|
|
128
130
|
| # | Title | Status | Date | Supersedes |
|
|
129
131
|
|---|---|---|---|---|
|
|
130
|
-
| [ADR-rule-kernel-and-router](ADR-rule-kernel-and-router.md) |
|
|
132
|
+
| [ADR-rule-kernel-and-router](ADR-rule-kernel-and-router.md) | Rule Kernel And Router Roadmap Closure | accepted | 2026-05-06 | — |
|
|
@@ -0,0 +1,82 @@
|
|
|
1
|
+
# Cross-model parity eval — re-scoped design
|
|
2
|
+
|
|
3
|
+
> Design artifact for `road-to-opt-measurement-unblock` Phase 3 step 1. It
|
|
4
|
+
> re-scopes the archive's most-deferred item (deferred across 4+ roadmaps) to the
|
|
5
|
+
> **smallest capability** that produces a real cross-vendor parity signal, so the
|
|
6
|
+
> build-vs-defer decision (step 2) rests on a concrete design, not the generic
|
|
7
|
+
> "the harness doesn't exist". Writing this design is autonomous; **building and
|
|
8
|
+
> running it is a maintainer cost decision** (step 2/3), disclosed and gated.
|
|
9
|
+
|
|
10
|
+
## What changed since the last deferral
|
|
11
|
+
|
|
12
|
+
| Original blocker | Status now |
|
|
13
|
+
|---|---|
|
|
14
|
+
| No delegable-task corpus | **RESOLVED** — `internal/bench/orchestration/corpus/` exists (`orch-*` orchestration tasks + `pv-*` production-validator tasks) |
|
|
15
|
+
| Harness cannot execute model-emitted subagent calls | **STILL TRUE** — do not assume it away; re-scope around it |
|
|
16
|
+
|
|
17
|
+
The corpus blocker no longer holds, so the item is re-openable — but only if we
|
|
18
|
+
stop trying to build the thing that is still blocked (a full in-host harness that
|
|
19
|
+
runs arbitrary model-emitted subagent calls).
|
|
20
|
+
|
|
21
|
+
## The re-scope: council-transport execution, not an in-host harness
|
|
22
|
+
|
|
23
|
+
**Don't** build a harness that executes model-emitted subagent calls end-to-end
|
|
24
|
+
(the still-open blocker). **Do** reuse the capability that already exists: the AI
|
|
25
|
+
council transport dispatches the *same* prompt to *multiple vendors* through
|
|
26
|
+
`consult(members: ExternalAIClient[], …)` in
|
|
27
|
+
[`src/scripts/ai_council/orchestrator.ts`](../../src/scripts/ai_council/orchestrator.ts).
|
|
28
|
+
|
|
29
|
+
The smallest capability that yields a cross-model parity signal:
|
|
30
|
+
|
|
31
|
+
1. **Task adapter.** Render each corpus task (`internal/bench/orchestration/corpus/*.md`)
|
|
32
|
+
into a single self-contained council prompt: the task + its rubric, asking the
|
|
33
|
+
model to *produce the findings it would surface* (not to actually spawn
|
|
34
|
+
subagents). This sidesteps the subagent-execution blocker — we measure the
|
|
35
|
+
model's **finding output on the delegable task**, not its ability to drive a
|
|
36
|
+
live subagent runtime.
|
|
37
|
+
2. **≥ 2 vendors, same corpus.** Dispatch the identical prompt set to two distinct
|
|
38
|
+
vendors via the existing `ExternalAIClient` array — no new transport, no new
|
|
39
|
+
auth surface beyond what the council already has.
|
|
40
|
+
3. **Per-host finding-count distribution.** For each (host × task), record the
|
|
41
|
+
count and the findings; aggregate to a per-host distribution over the corpus.
|
|
42
|
+
4. **Blind judged quality (optional, gated).** If a quality signal beyond raw
|
|
43
|
+
counts is wanted, reuse the blind second-judge + Cohen's κ from
|
|
44
|
+
[`check_quality_regression.ts`](../../src/scripts/check_quality_regression.ts)
|
|
45
|
+
— the same discipline the Phase-1 rerun uses. This is the billable half; the
|
|
46
|
+
count distributions (step 3) are cheap and already informative.
|
|
47
|
+
|
|
48
|
+
**Explicitly out of scope:** running the models' emitted subagent calls, a
|
|
49
|
+
persistent multi-host runtime, any new vendor integration. If the signal needs
|
|
50
|
+
more than council-transport can give, that is a *separate, later* build — named,
|
|
51
|
+
not assumed.
|
|
52
|
+
|
|
53
|
+
## What the signal feeds — `finding_floor` calibration
|
|
54
|
+
|
|
55
|
+
`finding_floor` is an **inert mechanism today** (referenced in
|
|
56
|
+
`run_skill_evals.ts` / `skill_linter.ts` / `evals.schema.json`); its recorded
|
|
57
|
+
deferral reason was exactly the missing calibration input this eval produces. The
|
|
58
|
+
per-host finding-count distributions calibrate the floor:
|
|
59
|
+
|
|
60
|
+
- Derive the floor from the **cross-host lower envelope** (e.g. a low percentile of
|
|
61
|
+
the per-task finding counts across vendors) so the gate fires on a host/task that
|
|
62
|
+
under-produces relative to the cross-vendor norm, not relative to one vendor.
|
|
63
|
+
- Once calibrated, promote `finding_floor` from inert to an **enforcing CI gate**
|
|
64
|
+
(its intended end state).
|
|
65
|
+
|
|
66
|
+
## Build-vs-defer decision inputs (for step 2, maintainer)
|
|
67
|
+
|
|
68
|
+
- **Cost:** the count-distribution pass (steps 1–3) is `N_tasks × N_vendors`
|
|
69
|
+
council calls — small and cheap; render the `council:estimate`-class disclosure
|
|
70
|
+
before the first call. The judged-quality half (step 4) roughly doubles it.
|
|
71
|
+
- **Value:** unblocks `finding_floor` (arms a real CI gate) **and** produces the
|
|
72
|
+
first honest cross-vendor parity number the portfolio has ever had.
|
|
73
|
+
- **Defer condition:** if built later, park with the concrete missing capability
|
|
74
|
+
named — here it would be "council-transport task-adapter for the orchestration
|
|
75
|
+
corpus", never the generic "harness doesn't exist".
|
|
76
|
+
|
|
77
|
+
## Exit criteria mapping
|
|
78
|
+
|
|
79
|
+
- This design doc **exists** (step 1 — done by writing this). ✓
|
|
80
|
+
- Explicit **build/defer decision recorded** (step 2 — maintainer, cost attached). ⏳
|
|
81
|
+
- If built: `finding_floor` calibration data exists and the gate is armed
|
|
82
|
+
(step 3 — gated on the decision + spend). ⏳
|