@event4u/agent-config 7.0.2 → 7.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +9 -1
- package/CHANGELOG.md +117 -0
- package/README.md +4 -2
- package/dist/agent-src/commands/council/default.md +28 -20
- package/dist/agent-src/commands/evals/fix.json +5 -1
- package/dist/agent-src/commands/fix/comments.md +101 -0
- package/dist/agent-src/commands/fix.md +5 -2
- package/dist/agent-src/contexts/execution/auto-dispatch-classification.md +65 -0
- package/dist/agent-src/contexts/execution/auto-orchestration-activation.md +68 -0
- package/dist/agent-src/contexts/execution/host-capability-manifest.md +68 -0
- package/dist/agent-src/contexts/execution/orchestration-benchmark-gate.md +62 -0
- package/dist/agent-src/contexts/execution/orchestration-telemetry.md +59 -0
- package/dist/agent-src/contexts/execution/roadmap-process-loop.md +8 -3
- package/dist/agent-src/contexts/execution/subagent-response-contract.md +56 -0
- package/dist/agent-src/contexts/execution/subagent-routing.md +57 -0
- package/dist/agent-src/contexts/execution/subagent-spawn-contract.md +54 -0
- package/dist/agent-src/contexts/execution/subagent-steering.md +71 -0
- package/dist/agent-src/contexts/execution/verify-budget.md +50 -0
- package/dist/agent-src/rules/brand-source-of-truth.md +10 -0
- package/dist/agent-src/rules/delegation-policy.md +85 -0
- package/dist/agent-src/rules/design-fidelity.md +98 -0
- package/dist/agent-src/rules/domain-safety-pii.md +22 -0
- package/dist/agent-src/rules/legal-safety-floor.md +182 -0
- package/dist/agent-src/rules/lethal-trifecta-guard.md +25 -22
- package/dist/agent-src/rules/output-discipline.md +85 -0
- package/dist/agent-src/rules/telegraph-speak.md +4 -1
- package/dist/agent-src/rules/token-budget-discipline.md +94 -0
- package/dist/agent-src/scripts/archive_completed_roadmaps.ts +102 -15
- package/dist/agent-src/scripts/update_roadmap_progress.ts +30 -3
- package/dist/agent-src/skills/accessibility-auditor/SKILL.md +20 -5
- package/dist/agent-src/skills/ai-council/SKILL.md +28 -1
- package/dist/agent-src/skills/blade-ui/SKILL.md +9 -0
- package/dist/agent-src/skills/contract-review/SKILL.md +125 -0
- package/dist/agent-src/skills/contract-review/evals/triggers.json +17 -0
- package/dist/agent-src/skills/dashboard-design/SKILL.md +21 -5
- package/dist/agent-src/skills/design-intelligence/SKILL.md +96 -0
- package/dist/agent-src/skills/design-review/SKILL.md +83 -0
- package/dist/agent-src/skills/design-system-capture/SKILL.md +234 -0
- package/dist/agent-src/skills/design-tokens/SKILL.md +1 -0
- package/dist/agent-src/skills/dpa-review/SKILL.md +130 -0
- package/dist/agent-src/skills/dpa-review/evals/triggers.json +17 -0
- package/dist/agent-src/skills/existing-ui-audit/SKILL.md +15 -4
- package/dist/agent-src/skills/fe-design/SKILL.md +98 -16
- package/dist/agent-src/skills/flux/SKILL.md +8 -0
- package/dist/agent-src/skills/iconography/SKILL.md +1 -0
- package/dist/agent-src/skills/laravel/SKILL.md +1 -0
- package/dist/agent-src/skills/legal-intake-triage/SKILL.md +128 -0
- package/dist/agent-src/skills/legal-intake-triage/evals/triggers.json +17 -0
- package/dist/agent-src/skills/legal-practice-profile/SKILL.md +101 -0
- package/dist/agent-src/skills/legal-practice-profile/evals/triggers.json +17 -0
- package/dist/agent-src/skills/livewire/SKILL.md +8 -0
- package/dist/agent-src/skills/nda-triage/SKILL.md +114 -0
- package/dist/agent-src/skills/nda-triage/evals/triggers.json +17 -0
- package/dist/agent-src/skills/react-shadcn-ui/SKILL.md +21 -9
- package/dist/agent-src/skills/reasoning-orchestrator/SKILL.md +3 -1
- package/dist/agent-src/skills/recursive-verification/SKILL.md +186 -0
- package/dist/agent-src/skills/roadmap-management/SKILL.md +17 -0
- package/dist/agent-src/skills/skill-writing/SKILL.md +18 -0
- package/dist/agent-src/skills/subagent-orchestration/SKILL.md +11 -0
- package/dist/agent-src/skills/tailwind-engineer/SKILL.md +17 -7
- package/dist/agent-src/skills/typography-system/SKILL.md +14 -0
- package/dist/agent-src/skills/ui-component-architect/SKILL.md +9 -0
- package/dist/agent-src/templates/agent-settings.md +64 -19
- package/dist/discovery/deprecation-report.md +1 -1
- package/dist/discovery/discovery-manifest.json +362 -43
- package/dist/discovery/discovery-manifest.json.sha256 +1 -1
- package/dist/discovery/discovery-manifest.summary.md +13 -5
- package/dist/discovery/orphan-report.md +1 -1
- package/dist/discovery/packs.json +56 -11
- package/dist/discovery/trust-report.md +7 -5
- package/dist/discovery/workspaces.json +38 -7
- package/dist/mcp/registry-manifest.json +3 -3
- package/dist/router.json +1 -1
- package/dist/server/routes/wizard.js +4 -3
- package/dist/server/routes/wizard.js.map +1 -1
- package/dist/server/schemas/settings.js +23 -0
- package/dist/server/schemas/settings.js.map +1 -1
- package/dist/ui/assets/{index-CoQo90sz.js → index-UjcTG_9O.js} +2 -2
- package/dist/ui/assets/index-UjcTG_9O.js.map +1 -0
- package/dist/ui/index.html +1 -1
- package/docs/RELEASE_STORY_TEMPLATE.md +37 -0
- package/docs/architecture.md +4 -4
- package/docs/benchmark.md +46 -0
- package/docs/catalog.md +18 -6
- package/docs/command-flows.md +7 -1
- package/docs/contracts/ai-council-config.md +17 -10
- package/docs/contracts/audit-log-v1.md +7 -0
- package/docs/contracts/auto-orchestration-v1.md +70 -0
- package/docs/contracts/capability-boundary.md +64 -0
- package/docs/contracts/command-clusters.md +1 -1
- package/docs/contracts/discovery-manifest.schema.json +2 -2
- package/docs/contracts/package-self-orientation.md +3 -0
- package/docs/contracts/subagent-boundary.md +54 -0
- package/docs/decisions/ADR-013-discovery-frontmatter-contract.md +32 -0
- package/docs/decisions/ADR-020-global-only-consumer-scope.md +13 -6
- package/docs/decisions/ADR-049-configuration-trust-boundary.md +11 -1
- package/docs/decisions/ADR-093-ai-council-config-user-global.md +10 -4
- package/docs/decisions/ADR-104-ai-council-config-global-only.md +119 -0
- package/docs/decisions/ADR-105-automatic-subagent-orchestration.md +102 -0
- package/docs/decisions/ADR-106-recursive-verification-benchmark-gate.md +126 -0
- package/docs/decisions/ADR-107-legal-domain-pack-adoption.md +70 -0
- package/docs/decisions/ADR-108-open-source-forever-no-commercial-tier.md +46 -0
- package/docs/decisions/INDEX.md +5 -0
- package/docs/getting-started.md +2 -2
- package/docs/guidelines/agent-infra/domain-eval-anti-pattern.md +53 -0
- package/docs/guidelines/agent-infra/domain-pack-architecture.md +56 -0
- package/docs/guidelines/design-antipatterns-triggers.json +17 -0
- package/docs/guidelines/design-antipatterns.md +170 -0
- package/docs/guidelines/design-modes.md +97 -0
- package/package.json +1 -1
- package/src/config/agent-settings.template.yml +103 -3
- package/src/config/discovery/packs.yml +24 -0
- package/src/config/discovery/workspaces.yml +7 -0
- package/src/scripts/_lib/auto_dispatch.ts +95 -0
- package/src/scripts/_lib/host_capability.ts +59 -0
- package/src/scripts/_lib/orchestration_gate.ts +98 -0
- package/src/scripts/_lib/subagent_bundle.ts +154 -0
- package/src/scripts/_lib/subagent_response.ts +82 -0
- package/src/scripts/_lib/subagent_routing.ts +81 -0
- package/src/scripts/_lib/subagent_spawn.ts +97 -0
- package/src/scripts/_lib/subagent_steering.ts +64 -0
- package/src/scripts/_lib/verify_budget.ts +54 -0
- package/src/scripts/ai_council/advisors.ts +4 -2
- package/src/scripts/ai_council/clients.ts +7 -1
- package/src/scripts/ai_council/config.ts +22 -17
- package/src/scripts/bench_ab_v2_run.ts +127 -0
- package/src/scripts/bench_ab_v2_stats.ts +39 -0
- package/src/scripts/check_council_config_location.ts +6 -5
- package/src/scripts/check_trunk_drift.ts +67 -0
- package/src/scripts/lint_design_quality.ts +321 -0
- package/src/scripts/lint_discovery_vocabulary.ts +6 -0
- package/src/scripts/lint_framework_leakage_allowlist.json +21 -0
- package/src/scripts/lint_legal_pack.ts +197 -0
- package/src/scripts/lint_output_slop.ts +223 -0
- package/src/scripts/lint_pack_risk_class.ts +79 -0
- package/src/scripts/probe_projection_fidelity.ts +7 -1
- package/src/scripts/schemas/agent-settings.schema.json +23 -0
- package/src/scripts/schemas/skill.schema.json +5 -0
- package/src/scripts/skill_linter.ts +3 -0
- package/dist/ui/assets/index-CoQo90sz.js.map +0 -1
|
@@ -108,12 +108,19 @@ corrects this:
|
|
|
108
108
|
- The global surface is **secrets** (provider keys, already global) plus the
|
|
109
109
|
curated identity **whitelist** (`MERGEABLE_KEYS`) — not all of `personal.*`,
|
|
110
110
|
not council config, not personas.
|
|
111
|
-
- **Council
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
-
|
|
116
|
-
|
|
111
|
+
- **Council config is NOT on the `MERGEABLE_KEYS` whitelist.** _(Placement
|
|
112
|
+
amended: per [ADR-093](ADR-093-ai-council-config-user-global.md) →
|
|
113
|
+
[ADR-104](ADR-104-ai-council-config-global-only.md) the council config
|
|
114
|
+
is now **user-global** at `~/.event4u/agent-config/settings/.ai-council.yml`,
|
|
115
|
+
never project-local — but it is its own dedicated `.ai-council.yml`, not a
|
|
116
|
+
merged `.agent-settings.yml` key, so it stays off this whitelist.)_ Personas
|
|
117
|
+
stay project-local (version-controlled). The `MERGEABLE_KEYS` whitelist is a
|
|
118
|
+
**security trust boundary**: an untrusted project repo must not be able to
|
|
119
|
+
escalate arbitrary `personal.*` into user scope. Widening the whitelist now
|
|
120
|
+
requires a threat model.
|
|
121
|
+
- The council config key name is **`ai_council`** (canonical), never
|
|
122
|
+
`agent_council`. It lives in the user-global `.ai-council.yml` (ADR-104),
|
|
123
|
+
not under `.agent-settings.yml`.
|
|
117
124
|
|
|
118
125
|
This amendment narrows ADR-020's wording; the global-only **consumer install**
|
|
119
126
|
decision (distributed content writes only to `~/.event4u/agent-config/`) is
|
|
@@ -13,7 +13,17 @@ type: decision
|
|
|
13
13
|
|
|
14
14
|
## Status
|
|
15
15
|
|
|
16
|
-
**Accepted** · 2026-06-03.
|
|
16
|
+
**Accepted** · 2026-06-03. **Council-config *placement* superseded by
|
|
17
|
+
[ADR-093](ADR-093-ai-council-config-user-global.md) → [ADR-104](ADR-104-ai-council-config-global-only.md)**
|
|
18
|
+
(council config now lives user-global at
|
|
19
|
+
`~/.event4u/agent-config/settings/.ai-council.yml`, never project-local).
|
|
20
|
+
The **trust-boundary decision below still stands**: `ai_council` is not on
|
|
21
|
+
the user-global *settings merge whitelist* (`.agent-settings.yml` keys) —
|
|
22
|
+
the council reads its own dedicated `.ai-council.yml`, not a merged
|
|
23
|
+
settings key. Wherever this ADR says council config "lives project-local",
|
|
24
|
+
read "lives user-global per ADR-104".
|
|
25
|
+
|
|
26
|
+
Locks the AI-council convergence
|
|
17
27
|
(anthropic/claude-sonnet-4-5 + openai/gpt-4o, design mode + peer-review,
|
|
18
28
|
2026-06-03) on where the source of truth for personal preferences and AI-council
|
|
19
29
|
configuration lives. Refines [`ADR-020`](ADR-020-global-only-consumer-scope.md)
|
|
@@ -4,7 +4,7 @@ status: accepted
|
|
|
4
4
|
date: 2026-06-13
|
|
5
5
|
decision: ai-council-config-user-global
|
|
6
6
|
supersedes: —
|
|
7
|
-
superseded_by:
|
|
7
|
+
superseded_by: 104
|
|
8
8
|
phase: ai-council-config-relocation
|
|
9
9
|
type: structural
|
|
10
10
|
---
|
|
@@ -13,9 +13,15 @@ type: structural
|
|
|
13
13
|
|
|
14
14
|
## Status
|
|
15
15
|
|
|
16
|
-
**Accepted** · 2026-06-13.
|
|
17
|
-
council
|
|
18
|
-
|
|
16
|
+
**Accepted** · 2026-06-13. **Partially superseded by
|
|
17
|
+
[ADR-104](ADR-104-ai-council-config-global-only.md)** (2026-06-22): the
|
|
18
|
+
user-global relocation stands, but the project-local override this ADR
|
|
19
|
+
*kept* (precedence rule 2 in § Decision) is removed — the council is now
|
|
20
|
+
**global-only** and never searches the project tree.
|
|
21
|
+
|
|
22
|
+
Maintainer directive (standing, restated): the council is a per-developer
|
|
23
|
+
facility and must be configured once, globally, not re-declared per
|
|
24
|
+
project.
|
|
19
25
|
|
|
20
26
|
## Context
|
|
21
27
|
|
|
@@ -0,0 +1,119 @@
|
|
|
1
|
+
---
|
|
2
|
+
adr: 104
|
|
3
|
+
status: accepted
|
|
4
|
+
date: 2026-06-22
|
|
5
|
+
decision: ai-council-config-global-only
|
|
6
|
+
supersedes: 093
|
|
7
|
+
superseded_by: —
|
|
8
|
+
phase: ai-council-config-relocation
|
|
9
|
+
type: structural
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
# ADR-104 — The AI-council config is global-only (drop the project-local override)
|
|
13
|
+
|
|
14
|
+
## Status
|
|
15
|
+
|
|
16
|
+
**Accepted** · 2026-06-22. Maintainer directive (standing, restated and
|
|
17
|
+
sharpened): the council is configured **once, globally** — and the agent
|
|
18
|
+
must never search a project for council config, never infer "council not
|
|
19
|
+
configured" from a project's missing files.
|
|
20
|
+
|
|
21
|
+
Supersedes the project-local override that
|
|
22
|
+
[ADR-093](ADR-093-ai-council-config-user-global.md) deliberately kept.
|
|
23
|
+
|
|
24
|
+
## Context
|
|
25
|
+
|
|
26
|
+
ADR-093 relocated the council config to the user-global namespace
|
|
27
|
+
(`~/.event4u/agent-config/settings/.ai-council.yml`) but **kept** a
|
|
28
|
+
project-local override as precedence rule 2:
|
|
29
|
+
`<project_root>/agents/settings/.ai-council.yml` would win over the
|
|
30
|
+
user-global file when present. ADR-093 also rejected the "global always
|
|
31
|
+
wins" alternative, reasoning it would break a consumer team's
|
|
32
|
+
deliberately-checked-in shared council config.
|
|
33
|
+
|
|
34
|
+
In practice that hedge produced a recurring, infuriating failure mode that
|
|
35
|
+
the loader's own correctness did *not* prevent — because the failure was
|
|
36
|
+
in the **agent's mental model**, not the resolver:
|
|
37
|
+
|
|
38
|
+
> *"Council war in diesem Projekt nicht konfiguriert (kein <!-- md-language-check: ignore -->
|
|
39
|
+
> `.agent-settings.yml`/`scripts/ai_council`), daher habe ich die eine <!-- md-language-check: ignore -->
|
|
40
|
+
> Architektur-Gabel selbst entschieden."* <!-- md-language-check: ignore -->
|
|
41
|
+
|
|
42
|
+
An agent working in a consumer project, asked to consult the council on an
|
|
43
|
+
architecture fork, looked **in the project** for council artefacts — the
|
|
44
|
+
package-internal `scripts/ai_council` directory and the long-removed
|
|
45
|
+
`.agent-settings.yml` `ai_council` block — found neither (they only exist
|
|
46
|
+
inside the agent-config package), and concluded the council was
|
|
47
|
+
unconfigured. It then decided a structural fork solo. The user's council
|
|
48
|
+
*was* configured — globally — and would have answered from any project.
|
|
49
|
+
|
|
50
|
+
The project-local override is the root of this confusion: as long as "the
|
|
51
|
+
project might carry its own council config" is a documented truth, the
|
|
52
|
+
agent treats the project tree as a place to look, and absence-in-project
|
|
53
|
+
reads as not-configured. The override's intended beneficiary (a consumer
|
|
54
|
+
team pinning a shared config in their repo) has never materialised and is
|
|
55
|
+
not worth the recurring solo-decision failure on real work.
|
|
56
|
+
|
|
57
|
+
## Decision
|
|
58
|
+
|
|
59
|
+
**The council config is global-only.** `resolve_config_path` (in
|
|
60
|
+
`scripts/ai_council/config.ts`) resolves with this precedence, first match
|
|
61
|
+
wins:
|
|
62
|
+
|
|
63
|
+
1. **`$AI_COUNCIL_CONFIG`** — an explicit absolute path (tests / power
|
|
64
|
+
users). Honoured even when absent. This is an explicit path, not a
|
|
65
|
+
project search.
|
|
66
|
+
2. **User-global** `~/.event4u/agent-config/settings/.ai-council.yml`
|
|
67
|
+
(legacy `~/.config/agent-config/` read-fallback).
|
|
68
|
+
|
|
69
|
+
The project-local branch is **removed**. The loader never builds or stats
|
|
70
|
+
`<project_root>/agents/settings/.ai-council.yml`. `project_root` stays in
|
|
71
|
+
the signature for caller stability but is not consulted for resolution.
|
|
72
|
+
|
|
73
|
+
The corollary, binding on every agent-facing surface (the `ai-council`
|
|
74
|
+
skill, the `/council` command family, the config contract): **the absence
|
|
75
|
+
of a council file in a project says nothing about whether the council is
|
|
76
|
+
configured.** An agent must never claim "council not configured" from
|
|
77
|
+
missing project files (`.agent-settings.yml`, `scripts/ai_council`, a
|
|
78
|
+
project `.ai-council.yml`). The CLI/resolver is the single authority — it
|
|
79
|
+
reads the global file and works from every project, worktree, and CWD.
|
|
80
|
+
|
|
81
|
+
## Consequences
|
|
82
|
+
|
|
83
|
+
- A user-global `.ai-council.yml` with `enabled: true` works from **every**
|
|
84
|
+
project, including consumer repos and fresh worktrees, with zero
|
|
85
|
+
per-project setup — exactly the "configure once" intent of ADR-093, now
|
|
86
|
+
without the hedge that undermined it.
|
|
87
|
+
- The "council not configured in this project" solo-decision failure mode
|
|
88
|
+
is structurally impossible: there is no project location to be absent.
|
|
89
|
+
- `council_cli.ts` and `cmd_doctor.ts` (`doctor council-cli`) inherit the
|
|
90
|
+
change automatically — both already route through `resolve_config_path`.
|
|
91
|
+
- A consumer team that genuinely wants a repo-pinned shared council config
|
|
92
|
+
uses `$AI_COUNCIL_CONFIG` (e.g. exported in their dev shell / CI) — an
|
|
93
|
+
explicit opt-in, never an implicit project-tree search.
|
|
94
|
+
- `check_council_config_location.ts` keeps guarding against
|
|
95
|
+
`.agent-settings.yml` council references; its docstring is updated to
|
|
96
|
+
drop the "project-local first" description.
|
|
97
|
+
|
|
98
|
+
## Alternatives
|
|
99
|
+
|
|
100
|
+
- **Keep ADR-093 as-is (project-local override stays).** Rejected — this is
|
|
101
|
+
the exact configuration that produced the recurring failure; the loader
|
|
102
|
+
being correct did not save the agent's mental model.
|
|
103
|
+
- **Fix only the agent-facing docs, leave the override in the loader.**
|
|
104
|
+
Rejected — as long as a project-local lookup exists, "look in the
|
|
105
|
+
project" remains a documented truth and the absence-reads-as-unconfigured
|
|
106
|
+
confusion recurs. Removing the lookup makes the instruction unambiguous.
|
|
107
|
+
- **Add a project-local override behind an explicit opt-in flag.** Rejected
|
|
108
|
+
as unneeded complexity — `$AI_COUNCIL_CONFIG` already covers the rare
|
|
109
|
+
repo-pinned case explicitly.
|
|
110
|
+
|
|
111
|
+
## References
|
|
112
|
+
|
|
113
|
+
- [ADR-093](ADR-093-ai-council-config-user-global.md) — the user-global
|
|
114
|
+
relocation this ADR sharpens to global-only.
|
|
115
|
+
- `docs/contracts/ai-council-config.md` § File location — precedence rules.
|
|
116
|
+
- `src/scripts/ai_council/config.ts` — `resolve_config_path`.
|
|
117
|
+
- `src/skills/ai-council/SKILL.md`, `src/domains/meta/council/default/command.md`
|
|
118
|
+
— agent-facing surfaces carrying the "never infer not-configured from
|
|
119
|
+
missing project files" corollary.
|
|
@@ -0,0 +1,102 @@
|
|
|
1
|
+
---
|
|
2
|
+
adr: 105
|
|
3
|
+
status: accepted
|
|
4
|
+
date: 2026-06-23
|
|
5
|
+
decision: automatic-subagent-orchestration
|
|
6
|
+
supersedes: —
|
|
7
|
+
superseded_by: —
|
|
8
|
+
phase: auto-subagent-orchestration
|
|
9
|
+
type: structural
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
# ADR-105 — Automatic subagent orchestration: activate-don't-fragment, prove before default-on
|
|
13
|
+
|
|
14
|
+
## Status
|
|
15
|
+
|
|
16
|
+
**Accepted** · 2026-06-23. Resolved by a deep AI-council round
|
|
17
|
+
(claude-sonnet-4-5 + gpt-4o, design mode) plus host critical-evaluation, and
|
|
18
|
+
implemented by `road-to-auto-subagent-orchestration.md`.
|
|
19
|
+
|
|
20
|
+
## Context
|
|
21
|
+
|
|
22
|
+
The package shipped three latent subagent surfaces — the
|
|
23
|
+
`subagent-orchestration` skill (7 modes), `/orchestrate` (YAML pipelines), and
|
|
24
|
+
the RDP `reasoning-orchestrator` ("dispatch by default") — all **opt-in**,
|
|
25
|
+
none firing automatically. The goal: make subagent use automatic and
|
|
26
|
+
settings-gated, so an orchestrating model delegates delegable sub-tasks to
|
|
27
|
+
cheaper / faster / quota-separate subagents, each configured for its task,
|
|
28
|
+
globally toggleable in `.agent-settings.yml`.
|
|
29
|
+
|
|
30
|
+
The user's stated default was **on**. The council pushed back hard: a "default
|
|
31
|
+
on" whose own design concedes it *may increase* token spend inverts risk in a
|
|
32
|
+
package whose canon includes token-efficiency as an Iron Law, and whose
|
|
33
|
+
multi-tool portability constraint means not every host even has a subagent
|
|
34
|
+
primitive.
|
|
35
|
+
|
|
36
|
+
## Decision
|
|
37
|
+
|
|
38
|
+
1. **Activate, don't fragment.** Auto-dispatch is a settings + host-manifest
|
|
39
|
+
gate over the **existing** `subagent-orchestration` skill and RDP
|
|
40
|
+
"dispatch by default" — not a new parallel rule. Smallest surface.
|
|
41
|
+
2. **Conservative-until-proven shipped default.** The full toggle ships now
|
|
42
|
+
(`subagents.enabled`, `subagents.auto: off|ask|on`), but the **shipped**
|
|
43
|
+
default is `ask` on hosts whose capability manifest reports
|
|
44
|
+
`subagent_spawn: true`, `off` elsewhere. The default flips toward `on`
|
|
45
|
+
(host-gated) **only** behind a real benchmark proving a net token-or-time
|
|
46
|
+
win at held quality (Phase 6). `on` is the destination, reached by
|
|
47
|
+
evidence — the reconciliation of the user's goal with the council's
|
|
48
|
+
prove-first.
|
|
49
|
+
3. **Quota arbitrage is a runtime-detected bonus, never load-bearing.** The
|
|
50
|
+
"separate quota pool" (e.g. a subscription's distinct Sonnet allowance) is a
|
|
51
|
+
`host-capability-manifest` flag; routing is identical (minus the bonus)
|
|
52
|
+
where unsupported. No `.md` names a vendor's billing rule.
|
|
53
|
+
4. **Deterministic classification v1.** Delegability is decided by enumerated
|
|
54
|
+
structural signals (frontmatter `parallelizable:`, ordered-plan,
|
|
55
|
+
independent-slices, a size floor); ambiguity defaults to ask/no-op, never
|
|
56
|
+
speculative spawn. No per-turn LLM meta-call in v1. LLM classification is
|
|
57
|
+
deferred to v2+, gated on the benchmark.
|
|
58
|
+
5. **Verification budget preserves the floors.** Trivial/read-only sub-tasks
|
|
59
|
+
verify deterministically; non-trivial mutating sub-tasks get a full
|
|
60
|
+
cross-model judge (the `subagent-orchestration` Iron Law). A required
|
|
61
|
+
verification recorded as `none` is a surfaced safety gap.
|
|
62
|
+
6. **Instrumentation first; kill-switch + surfaced guardrails.** Orchestration
|
|
63
|
+
telemetry rides additively on `audit-log-v1`. The kill-switch is a
|
|
64
|
+
single settings flip (`enabled: false` / `auto: off`). Rollback thresholds
|
|
65
|
+
(token blowup, spawn-failure, verify-skip, override rate) are surfaced from
|
|
66
|
+
the audit log — there is no automatic cohort-disable (a config package runs
|
|
67
|
+
no daemon).
|
|
68
|
+
7. **Graceful host degradation.** Where the manifest reports
|
|
69
|
+
`subagent_spawn: false`, the whole layer is a clean no-op (single-agent),
|
|
70
|
+
surfaced explicitly. `.md` artefacts stay project-agnostic.
|
|
71
|
+
|
|
72
|
+
## Consequences
|
|
73
|
+
|
|
74
|
+
- Auto-dispatch is the default **where the host and settings allow it**, and a
|
|
75
|
+
no-op everywhere else. No safety floor is lifted.
|
|
76
|
+
- The user's "default on" is an evidence-gated milestone, not assumed — the
|
|
77
|
+
one decision left for the user is to authorise/run the Phase-6 benchmark and
|
|
78
|
+
approve the default flip.
|
|
79
|
+
- The implementation is mostly contexts + small pure helpers + a settings
|
|
80
|
+
block + two activated skills, keeping the diff minimal.
|
|
81
|
+
|
|
82
|
+
## Alternatives considered
|
|
83
|
+
|
|
84
|
+
- **Default `on` immediately** (the literal ask) — rejected: inverts risk
|
|
85
|
+
before measurement, against the frugality canon.
|
|
86
|
+
- **New parallel auto-dispatch rule** — rejected: fragments a surface that
|
|
87
|
+
already exists; activation is the smaller change.
|
|
88
|
+
- **Quota arbitrage as a core motivation** — rejected: vendor-specific,
|
|
89
|
+
fragile, violates portability; demoted to a runtime bonus.
|
|
90
|
+
- **Council-strict default `off` forever** — rejected: forecloses the user's
|
|
91
|
+
goal; the benchmark gate is the middle path.
|
|
92
|
+
|
|
93
|
+
## References
|
|
94
|
+
|
|
95
|
+
- The auto-subagent-orchestration roadmap (now archived) + the
|
|
96
|
+
full council Convergence / Host verdict.
|
|
97
|
+
- [`docs/contracts/auto-orchestration-v1.md`](../contracts/auto-orchestration-v1.md) — the manifest + spawn + telemetry contract.
|
|
98
|
+
- Context docs under `src/agent-src/contexts/execution/`:
|
|
99
|
+
`host-capability-manifest`, `orchestration-telemetry`,
|
|
100
|
+
`auto-orchestration-activation`, `auto-dispatch-classification`,
|
|
101
|
+
`subagent-routing`, `subagent-spawn-contract`, `verify-budget`,
|
|
102
|
+
`subagent-steering`, `orchestration-benchmark-gate`.
|
|
@@ -0,0 +1,126 @@
|
|
|
1
|
+
---
|
|
2
|
+
adr: 106
|
|
3
|
+
status: accepted
|
|
4
|
+
date: 2026-06-23
|
|
5
|
+
decision: recursive-verification-benchmark-gate
|
|
6
|
+
supersedes: —
|
|
7
|
+
superseded_by: —
|
|
8
|
+
phase: recursive-verification · Phase 0
|
|
9
|
+
type: structural
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
# ADR-106 — Recursive-verification benchmark gate: capability-OR-measured-novel-discipline, not capability-only
|
|
13
|
+
|
|
14
|
+
## Status
|
|
15
|
+
|
|
16
|
+
Accepted (2026-06-23). Locks the Phase 0 gate definition for
|
|
17
|
+
[`road-to-recursive-verification.md`](../../agents/roadmaps/archive/road-to-recursive-verification.md)
|
|
18
|
+
before any live `bench:ab` spend.
|
|
19
|
+
|
|
20
|
+
## Context
|
|
21
|
+
|
|
22
|
+
The `recursive-verification` skill (depth-bounded `attempt → critic verdict →
|
|
23
|
+
re-attempt`, default `off`) needs a benchmark gate that decides whether its
|
|
24
|
+
shipped default flips to `on`/`ask` for a given host. The roadmap's initial draft
|
|
25
|
+
proposed **capability-axis-first**: ship only on a capability-axis lift; a
|
|
26
|
+
discipline-axis-only lift does **not** pass, on the assumption that the package's
|
|
27
|
+
existing always-on rules already deliver the discipline lift, so recursion would
|
|
28
|
+
merely duplicate them at ~11× cost.
|
|
29
|
+
|
|
30
|
+
This gate definition was put to the AI council (manual transport disabled; live
|
|
31
|
+
`council run --depth deep`, 3 rounds; members **anthropic/claude-sonnet-4-5** +
|
|
32
|
+
**openai/gpt-4o**, 2026-06-23, actual cost $0.0762). Both members independently
|
|
33
|
+
converged that **capability-only is the wrong gate**, for a sound methodological
|
|
34
|
+
reason; one member (anthropic) also raised a misreading that the host rejected.
|
|
35
|
+
|
|
36
|
+
### What the council got right (accepted)
|
|
37
|
+
|
|
38
|
+
- **The redundancy assumption is unmeasured.** The gate assumed recursion's
|
|
39
|
+
discipline lift is redundant with the existing rules *without measuring it*.
|
|
40
|
+
The fix is a **three-baseline** design and an explicit **marginal** lift:
|
|
41
|
+
- `D₀` = bare host (no rules, no recursion)
|
|
42
|
+
- `D₁` = rules only (always-on rules, no recursion)
|
|
43
|
+
- `D₂` = rules + recursion
|
|
44
|
+
- **novel discipline lift = `D₂ − D₁`** — recursion's contribution *over*
|
|
45
|
+
rules-only. Redundancy is now a measured fact, not an assumption.
|
|
46
|
+
- **Cheap human-preference pre-test first.** Before any expensive benchmark, show
|
|
47
|
+
~10 `(attempt₀, attempt_final)` pairs to humans and ask "which would you pay
|
|
48
|
+
for?". If preference < 60 %, the discipline lift is economically irrelevant →
|
|
49
|
+
capability-only gate stands and the expensive discipline arm is skipped (~$15
|
|
50
|
+
to settle the whole dispute). This mirrors the package's cheap-probe-first,
|
|
51
|
+
measure-first culture.
|
|
52
|
+
- **Concrete cost ceiling, not "tolerable".** The original "at a tolerable cost"
|
|
53
|
+
is circular. Replace with a measurable ceiling (token multiplier / $-per-task).
|
|
54
|
+
- **Per-(host, family) cell**, not host-global — recursion may pass for a weak
|
|
55
|
+
host on one family and fail for a strong host with no headroom.
|
|
56
|
+
|
|
57
|
+
### What the council got wrong (rejected by the host)
|
|
58
|
+
|
|
59
|
+
- anthropic framed the gate as a *"fatal flaw because the 0.333 → 1.000 lift is
|
|
60
|
+
already recursion."* This is a **misread** of `docs/benchmark.md`: that lift is
|
|
61
|
+
the *rules'* effect (vanilla → package), measured **without** recursion;
|
|
62
|
+
recursion's marginal lift is unmeasured (that is exactly what Phase 3 runs). The
|
|
63
|
+
valid extraction is the three-baseline design above, not "capability-first is
|
|
64
|
+
fatally wrong."
|
|
65
|
+
- A weighted composite metric `w_cap·Δcap + w_disc·Δdisc` (openai) is **rejected**:
|
|
66
|
+
the weights are precisely the unknown the gate exists to discover (anthropic's
|
|
67
|
+
own counter, which the host endorses).
|
|
68
|
+
|
|
69
|
+
## Decision
|
|
70
|
+
|
|
71
|
+
The shipped default for `verification.recursive` flips to `on`/`ask` for a
|
|
72
|
+
**(host, family) cell** if and only if:
|
|
73
|
+
|
|
74
|
+
```
|
|
75
|
+
capability_lift(host, family) significant (McNemar on pass-rate, or scorer
|
|
76
|
+
Wilcoxon p < 0.05, ≥6 discordant
|
|
77
|
+
pairs, effect ≥ 0.5)
|
|
78
|
+
OR
|
|
79
|
+
( novel_discipline_lift = D₂ − D₁ > ε_disc (recursion over rules-only —
|
|
80
|
+
MEASURED, never assumed)
|
|
81
|
+
AND cost_per_task ≤ ceiling (concrete token-multiplier / $)
|
|
82
|
+
AND human_preference_rate > 0.6 )
|
|
83
|
+
ELSE off.
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
**Cheap prerequisite (runs first, gates the rest):** the ~$15 human-preference
|
|
87
|
+
test. Preference < 60 % ⇒ capability-only gate stands; skip the discipline arm.
|
|
88
|
+
|
|
89
|
+
**The benchmark must report, per (host, family):** `pass_rate(D₀/D₂)`,
|
|
90
|
+
`discipline_score(D₀ / D₁ / D₂)`, `cost(D₂)/cost(D₀)`, and
|
|
91
|
+
`human_preference(recursion vs baseline)`. Without the `D₁` (rules-only) middle
|
|
92
|
+
term, redundancy cannot be detected; without human-preference, a discipline lift
|
|
93
|
+
cannot be validated as economically real.
|
|
94
|
+
|
|
95
|
+
The first measured cell is `(claude-haiku-4-5, scope-discipline)`; multi-family
|
|
96
|
+
expansion waits until data justifies it.
|
|
97
|
+
|
|
98
|
+
## Consequences
|
|
99
|
+
|
|
100
|
+
- Phase 3 of the roadmap gains a **rules-only (`D₁`) arm** and a **Phase 3a-pre
|
|
101
|
+
human-preference test**; the gate language changes from "capability-first;
|
|
102
|
+
discipline-only ⇒ STOP" to the two-branch rule above.
|
|
103
|
+
- An honest-null is still first-class: capability flat **and** (human-preference
|
|
104
|
+
< 60 % OR `D₂ − D₁` ≤ ε OR cost over ceiling) ⇒ ship nothing, record the null.
|
|
105
|
+
- The default stays `off` until a cell passes; no global flip without its own cell.
|
|
106
|
+
- Slightly more measurement up front (three baselines + a human pre-test), bought
|
|
107
|
+
cheaply ($15 pre-test gates the expensive arm) — net a more honest gate.
|
|
108
|
+
|
|
109
|
+
## Alternatives considered
|
|
110
|
+
|
|
111
|
+
- **Capability-only (original draft).** Rejected: assumes, without measuring,
|
|
112
|
+
that recursion's discipline lift is redundant with the rules.
|
|
113
|
+
- **Weighted composite metric.** Rejected: presupposes the weights the gate
|
|
114
|
+
exists to discover.
|
|
115
|
+
- **Holistic / qualitative gate (openai round 1).** Rejected: too vague to
|
|
116
|
+
operationalise into a ship/no-ship decision.
|
|
117
|
+
|
|
118
|
+
## References
|
|
119
|
+
|
|
120
|
+
- [`road-to-recursive-verification.md`](../../agents/roadmaps/archive/road-to-recursive-verification.md)
|
|
121
|
+
- [`docs/benchmark.md`](../benchmark.md) — the existing wrapper-lift A/B evidence.
|
|
122
|
+
- Council round: anthropic/claude-sonnet-4-5 + openai/gpt-4o, 2026-06-23, deep
|
|
123
|
+
tier (3 rounds), $0.0762 actual.
|
|
124
|
+
- [`subagent-orchestration`](../../src/skills/subagent-orchestration/SKILL.md),
|
|
125
|
+
[`verify-budget`](../../src/agent-src/contexts/execution/verify-budget.md),
|
|
126
|
+
`orchestration-benchmark-gate` (`gateVerdict` / `resolveShippedDefault`).
|
|
@@ -0,0 +1,70 @@
|
|
|
1
|
+
---
|
|
2
|
+
adr: 107
|
|
3
|
+
status: accepted
|
|
4
|
+
date: 2026-06-24
|
|
5
|
+
decision: legal-domain-pack-adoption
|
|
6
|
+
supersedes: —
|
|
7
|
+
superseded_by: —
|
|
8
|
+
phase: road-to-legal-pack
|
|
9
|
+
type: prospective
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
# ADR-107 — Legal domain-pack adoption (EU/DE-scoped, procedure-only, evaluated honestly)
|
|
13
|
+
|
|
14
|
+
## Status
|
|
15
|
+
|
|
16
|
+
**Accepted** · 2026-06-24 · three AI-council rounds (anthropic/claude-sonnet-4-5 + openai/gpt-4o; the curl-timeout fix landed mid-series so rounds 2–3 ran two-voice). Implementing roadmap: road-to-legal-pack (archived on completion).
|
|
17
|
+
|
|
18
|
+
## Context
|
|
19
|
+
|
|
20
|
+
`event4u/agent-config` had legal-adjacent anchors (`contracts-cognition`, `privacy-review`, `data-handling-judgment`, `data-flow-mapper`, `domain-safety-disclaimer` legal sector) but no legal **domain pack**. Opening one is a `domain-adoption-policy` decision (three gates: demand signal, named maintenance owner, CI-tooling). External legal-plugin references (a first-party suite + community packs) showed reusable patterns but enforce everything via prompt instructions only (empty hooks).
|
|
21
|
+
|
|
22
|
+
## Decision
|
|
23
|
+
|
|
24
|
+
> **Amendment (2026-06-24, deep-council).** The pack id was renamed `legal` →
|
|
25
|
+
> `legal-review-prep` after this ADR, to stop the name itself reading as "legal
|
|
26
|
+
> advice" (deep AI-council + 7.1.0 reviewers; see `road-to-legal-review-prep`).
|
|
27
|
+
> Read every `legal` pack-id reference below as `legal-review-prep`.
|
|
28
|
+
|
|
29
|
+
Open a single, narrow `legal` pack — **not** a multi-plugin / multi-agent parity clone.
|
|
30
|
+
|
|
31
|
+
1. **Scope: EU/DE-only, hard refusal.** Every selectable jurisdiction is an implicit currency promise; for a single maintainer the smallest correct cut is EU/DE with "consult local counsel" for everything else. Encoded in the `legal-safety-floor` rule + the pack description; surfaced via a machine-checkable `Jurisdiction:` tag.
|
|
32
|
+
2. **Safety-floor first.** `rule:legal-safety-floor` + its deterministic linters land before any drafting skill (a drafting skill before its floor is the exact liability the floor prevents).
|
|
33
|
+
3. **Skills ship procedure + output template only — no default legal positions.** Positions live in a practice profile (`legal-practice-profile`).
|
|
34
|
+
4. **Deterministic hardening is the differentiator.** Disclaimer-presence, jurisdiction-tag, and freshness linters; privilege markers extend `domain-safety-pii` + `lethal-trifecta-guard` (no new hook unless the extension demonstrably fails).
|
|
35
|
+
5. **Eval = regression/consistency harness, not a correctness oracle.** Self-labeled fixtures narrowed to the maintainer's genuine-competence domain (EU DPA / GDPR Art. 28) with spot-validation; a multi-attorney gold set (inter-annotator ≥0.7) is a **demand+funding-gated** track, NOT parity-gated (legal matching is classification, not the finding-count distributions the cross-model-parity keystone calibrates). The eval achieves *measurement* objectivity (reliable regression detection), never *ground-truth* objectivity — correctness stays attorney review.
|
|
36
|
+
6. **No personas** — procedure-not-personality; role focus lives in skill tags + the practice profile.
|
|
37
|
+
7. **Conditional product-liability gate** — commercial (Pro-tier) ship would require licensed-attorney review of the pack itself. **Resolved N/A 2026-06-24:** the suite is open-source forever, no commercial tier (ADR-108) — the gate never fires; the non-removable disclaimer + per-output attorney-review line cover end-user reliance.
|
|
38
|
+
|
|
39
|
+
## Amendment (2026-06-24, deep-council) — attorney-framing-review promotion tripwire
|
|
40
|
+
|
|
41
|
+
Promotion of `legal-review-prep` out of `surface_tier: lab` / `trust_level_default:
|
|
42
|
+
experimental`, flipping `default_install` to anything but `false`, or any
|
|
43
|
+
commercial / hosted surface, is **blocked** until a licensed German attorney has
|
|
44
|
+
reviewed the *framing* (the `legal-safety-floor` rule, `LEGAL_NOTICE.md`, and the
|
|
45
|
+
RDG individual-case wording — **not** skill correctness) and the review is recorded
|
|
46
|
+
in `LEGAL_NOTICE.md` as `Framing reviewed by <name>, Rechtsanwalt/Rechtsanwältin,
|
|
47
|
+
<date>`. The deep AI-council (2026-06-24) scoped this as a bounded (~€300–800,
|
|
48
|
+
1–2h) one-time gate, **not** a blocker for the current lab release (lab is
|
|
49
|
+
explicitly unvetted; users opt in knowingly via the consent gate). Enforced by
|
|
50
|
+
`lint_legal_pack` (`promotion-gate` rule): a promoted tier without the recorded
|
|
51
|
+
marker fails CI.
|
|
52
|
+
|
|
53
|
+
## Rejections (recorded)
|
|
54
|
+
|
|
55
|
+
matter-workspace (too heavy), scheduled watchers, community-skill installer hub, US-state employment tests, full litigation depth, bar-prep, Swiss cantonal depth, non-EU/DE jurisdictions, the cross-model-parity dependency for the eval (a false blocker, corrected in the full critique round).
|
|
56
|
+
|
|
57
|
+
## Consequences
|
|
58
|
+
|
|
59
|
+
- The pack ships `trust_level_default: experimental` / `surface_tier: lab` as the **conservative default for a high-risk legal domain**. Gate 2 was **resolved 2026-06-24 by maintainer decision: no named-owner / external-qualification gate** — the protection is the always-present, **test-enforced** not-legal-advice notice (every skill · chat output · README · dedicated legal-pack notice; `tests/scripts/legal-disclaimer-presence.test.ts`) plus the RDG individual-case guardrail. Tier promotion is a separate future decision.
|
|
60
|
+
- A reusable domain-pack sequence (Safety-Floor → Enforcement → Evals → Skills → Practice-Profile) is **documented retrospectively**, validated against a second domain before promotion to a rule (per `domain-pack-extraction-when-triggered`, N=2) — it does not drive legal's design.
|
|
61
|
+
|
|
62
|
+
## Alternatives considered
|
|
63
|
+
|
|
64
|
+
Jurisdiction-neutral-with-EU-default (rejected — maximizes the maintenance surface the binding constraint exists to cut); objective eval pack at N=1 (rejected — manufactures false objectivity without a funded attorney gold set); ≤2 legal personas (rejected — decoration with maintenance cost, no incremental safety).
|
|
65
|
+
|
|
66
|
+
## References
|
|
67
|
+
|
|
68
|
+
- road-to-legal-pack — the implementing roadmap (archived on completion)
|
|
69
|
+
- [ADR-013 — Discovery Frontmatter Contract](ADR-013-discovery-frontmatter-contract.md) (vocabulary amendment, same series)
|
|
70
|
+
- `rule:legal-safety-floor`, `rule:domain-adoption-policy`, `rule:domain-safety-disclaimer`
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
---
|
|
2
|
+
adr: 108
|
|
3
|
+
status: accepted
|
|
4
|
+
date: 2026-06-24
|
|
5
|
+
decision: open-source-forever-no-commercial-tier
|
|
6
|
+
supersedes: —
|
|
7
|
+
superseded_by: —
|
|
8
|
+
phase: road-to-legal-pack
|
|
9
|
+
type: standing
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
# ADR-108 — The suite is open-source forever; no commercial / Pro tier
|
|
13
|
+
|
|
14
|
+
## Status
|
|
15
|
+
|
|
16
|
+
**Accepted** · 2026-06-24 · maintainer decision. Standing commitment for the whole `event4u/agent-config` suite, surfaced while resolving the legal pack's conditional product-liability gate (ADR-107 § 0.6).
|
|
17
|
+
|
|
18
|
+
## Context
|
|
19
|
+
|
|
20
|
+
The legal pack (ADR-107) carried a *conditional* product-liability gate: "if shipped commercially (Pro tier), a licensed attorney must review the pack itself before ship." That conditional only matters if a commercial/Pro tier is on the table. The maintainer's standing position is that it is not — and should be made durable so the question is not relitigated and so no future change silently introduces provider liability.
|
|
21
|
+
|
|
22
|
+
## Decision
|
|
23
|
+
|
|
24
|
+
**The entire `event4u/agent-config` suite is and remains open-source. There is no commercial tier, no Pro tier, no paid distribution — now or in the future.**
|
|
25
|
+
|
|
26
|
+
Consequences for governance:
|
|
27
|
+
|
|
28
|
+
1. **The legal pack's conditional product-liability gate (ADR-107 § 0.6) never fires.** There is no provider selling the pack; the only liability surface is end-user reliance, which is addressed by the non-removable disclaimer and the per-output attorney-review line in `legal-safety-floor`.
|
|
29
|
+
2. **Liability disclaimer is permanent.** Every legal-pack output is a research-and-drafting aid, not legal advice, not a substitute for a licensed attorney, and not something anyone may rely on as definitive. This holds independent of distribution model.
|
|
30
|
+
3. **Reopening requires a superseding ADR.** Introducing any paid/commercial tier would be a new, explicit decision that supersedes this ADR — and would re-activate the ADR-107 § 0.6 attorney-review-of-the-pack gate before any paid distribution.
|
|
31
|
+
|
|
32
|
+
## Consequences
|
|
33
|
+
|
|
34
|
+
- `legal-safety-floor` § Distribution states the open-source-forever stance and that the conditional gate is closed by decision.
|
|
35
|
+
- ADR-107 § 0.6 is resolved as **N/A** (not applicable under this standing decision).
|
|
36
|
+
|
|
37
|
+
## Alternatives considered
|
|
38
|
+
|
|
39
|
+
- **Keep 0.6 purely conditional, no standing decision** (rejected — leaves the door open to silently introducing provider liability; the maintainer wanted it hardened for the future).
|
|
40
|
+
- **A CI lint enforcing "no commercial tier"** (rejected — over-engineering; an ADR + the floor statement are the right durable surface, and there is no machine signal of "commercial intent" to lint).
|
|
41
|
+
|
|
42
|
+
## References
|
|
43
|
+
|
|
44
|
+
- [ADR-107 — Legal domain-pack adoption](ADR-107-legal-domain-pack-adoption.md) (§ 0.6 conditional gate, now resolved N/A)
|
|
45
|
+
- `rule:legal-safety-floor` § Distribution + § What this pack is — and is not
|
|
46
|
+
- road-to-legal-pack § 0.6 (archived on completion)
|
package/docs/decisions/INDEX.md
CHANGED
|
@@ -107,6 +107,11 @@ _Auto-generated by `scripts/adr/regenerate_index.py`. Do not edit._
|
|
|
107
107
|
| [ADR-101](ADR-101-ticket-bundle-emission.md) | Ticket Bundle Emission | proposed | 2026-06-15 | — |
|
|
108
108
|
| [ADR-102](ADR-102-ticket-handoff-paste-and-mcp.md) | Ticket Handoff Paste And Mcp | accepted | 2026-06-16 | — |
|
|
109
109
|
| [ADR-103](ADR-103-global-knowledge-default-off-until-measured.md) | Global Knowledge Default Off Until Measured | accepted | 2026-06-16 | — |
|
|
110
|
+
| [ADR-104](ADR-104-ai-council-config-global-only.md) | Ai Council Config Global Only | accepted | 2026-06-22 | 093 |
|
|
111
|
+
| [ADR-105](ADR-105-automatic-subagent-orchestration.md) | Automatic Subagent Orchestration | accepted | 2026-06-23 | — |
|
|
112
|
+
| [ADR-106](ADR-106-recursive-verification-benchmark-gate.md) | Recursive Verification Benchmark Gate | accepted | 2026-06-23 | — |
|
|
113
|
+
| [ADR-107](ADR-107-legal-domain-pack-adoption.md) | Legal Domain Pack Adoption | accepted | 2026-06-24 | — |
|
|
114
|
+
| [ADR-108](ADR-108-open-source-forever-no-commercial-tier.md) | Open Source Forever No Commercial Tier | accepted | 2026-06-24 | — |
|
|
110
115
|
| [ADR-200](ADR-200-python-to-typescript-migration.md) | Python To Typescript Migration | accepted | 2026-06-11 | — |
|
|
111
116
|
|
|
112
117
|
## Unnumbered (legacy)
|
package/docs/getting-started.md
CHANGED
|
@@ -129,7 +129,7 @@ Your agent is now:
|
|
|
129
129
|
- **Respecting your codebase** — no conflicting patterns
|
|
130
130
|
- **Following standards** — consistent code quality
|
|
131
131
|
|
|
132
|
-
This is enforced automatically by
|
|
132
|
+
This is enforced automatically by 93 rules. No configuration needed.
|
|
133
133
|
|
|
134
134
|
---
|
|
135
135
|
|
|
@@ -169,7 +169,7 @@ Your agent now understands slash commands:
|
|
|
169
169
|
| `/quality-fix` | Run and fix all quality checks |
|
|
170
170
|
| `/chat-history import` | Pull a prior session into the current chat (resume) |
|
|
171
171
|
|
|
172
|
-
→ [Browse all
|
|
172
|
+
→ [Browse all 162 active commands](../dist/agent-src/commands/)
|
|
173
173
|
|
|
174
174
|
---
|
|
175
175
|
|
|
@@ -0,0 +1,53 @@
|
|
|
1
|
+
# Domain-eval anti-pattern — manufactured false objectivity at N=1
|
|
2
|
+
|
|
3
|
+
> A retrospective from the legal pack (council round 3). It exists so the next
|
|
4
|
+
> domain does not repeat the mistake. The short version: **do not call a
|
|
5
|
+
> self-labeled regression harness an "objective" correctness gate.**
|
|
6
|
+
|
|
7
|
+
## The anti-pattern
|
|
8
|
+
|
|
9
|
+
Gating skill ship on *"≥X% match to expert ground-truth"* and calling it "the
|
|
10
|
+
objective replacement for reviewer sign-off" — when:
|
|
11
|
+
|
|
12
|
+
1. **The ground-truth is unsourced or self-generated.** If the maintainer guesses
|
|
13
|
+
what an expert would flag, the eval measures "does the skill match the
|
|
14
|
+
maintainer's guess," i.e. reviewer subjectivity wearing a quantitative costume.
|
|
15
|
+
2. **Expert agreement is not truth.** Even an attorney-validated gold set at
|
|
16
|
+
inter-annotator agreement ≥0.7 is *inter-subjective expert consensus*, not
|
|
17
|
+
ground truth — `0.7 ≠ 1.0` admits the domain is contested ~30% of the time.
|
|
18
|
+
The honest goal is **measurement objectivity** (reliably detect change), not
|
|
19
|
+
**ground-truth objectivity** (the labels are correct).
|
|
20
|
+
3. **Model capability caps below the threshold.** Clause-extraction caps at
|
|
21
|
+
~F1 0.62; any passable threshold therefore measures "not worse than a mediocre
|
|
22
|
+
baseline," i.e. regression detection — never correctness.
|
|
23
|
+
4. **The statistical language is borrowed from infra that does not exist.**
|
|
24
|
+
Calibrating a threshold "from cross-model distributions" requires a built
|
|
25
|
+
cross-model-parity layer. Borrowing the vocabulary before the layer exists is
|
|
26
|
+
false precision.
|
|
27
|
+
|
|
28
|
+
## The honest alternative
|
|
29
|
+
|
|
30
|
+
- **Ship a regression / consistency harness.** Self-labeled fixtures, explicitly
|
|
31
|
+
marked "regression-only, not correctness, not objective." It catches "did a
|
|
32
|
+
change make a skill worse than its own baseline."
|
|
33
|
+
- **Keep correctness with the human.** Licensed-expert review on material use,
|
|
34
|
+
baked into the skill's output, not a footnote.
|
|
35
|
+
- **Make the objective eval a separate, gated track.** Attorney/expert-validated
|
|
36
|
+
gold set + inter-annotator ≥0.7 is real but expensive; gate it on **validated
|
|
37
|
+
demand + funding**, not on a phase timeline — and note it is classification, so
|
|
38
|
+
it does **not** depend on a cross-model-parity keystone.
|
|
39
|
+
|
|
40
|
+
## The false-blocker corollary
|
|
41
|
+
|
|
42
|
+
When you scope an eval, check what it *actually* depends on. The legal eval was
|
|
43
|
+
briefly gated on the cross-model-parity keystone; the full critique round showed
|
|
44
|
+
that was wrong — legal fixture matching is **classification** (clause
|
|
45
|
+
present/absent/ambiguous), not the finding-count distributions that keystone
|
|
46
|
+
calibrates. The real dependency was labeled-gold-set tooling, buildable in-pack.
|
|
47
|
+
A false structural blocker is as much an error as false objectivity.
|
|
48
|
+
|
|
49
|
+
## See also
|
|
50
|
+
|
|
51
|
+
- [ADR-107 — Legal domain-pack adoption](../../decisions/ADR-107-legal-domain-pack-adoption.md)
|
|
52
|
+
- `domain-pack-architecture.md` — the retrospective sequence this is part of.
|
|
53
|
+
- `src/domains/legal/evals/README.md` — the regression harness in practice.
|