@event4u/agent-config 12.1.0 → 14.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (185) hide show
  1. package/.claude-plugin/marketplace.json +1 -1
  2. package/CHANGELOG.md +164 -186
  3. package/README.md +1 -1
  4. package/dist/agent-src/commands/cost/report.md +23 -0
  5. package/dist/agent-src/commands/evals/roadmap.json +8 -0
  6. package/dist/agent-src/commands/fix/ci.md +39 -1
  7. package/dist/agent-src/commands/pr/create.md +65 -4
  8. package/dist/agent-src/commands/roadmap/next.md +28 -5
  9. package/dist/agent-src/commands/work.md +16 -0
  10. package/dist/agent-src/contexts/contracts/command-suggestion-flow.md +47 -0
  11. package/dist/agent-src/contexts/execution/contract-decision-sheet.md +94 -0
  12. package/dist/agent-src/contexts/execution/plan-confidence-gate.md +30 -0
  13. package/dist/agent-src/contexts/execution/roadmap-execution-contract.md +21 -8
  14. package/dist/agent-src/contexts/execution/roadmap-process-loop.md +55 -8
  15. package/dist/agent-src/contexts/execution/subagent-response-contract.md +42 -0
  16. package/dist/agent-src/contexts/execution/subagent-routing.md +148 -33
  17. package/dist/agent-src/contexts/execution/subagent-spawn-contract.md +29 -0
  18. package/dist/agent-src/contexts/subagent-configuration.md +69 -7
  19. package/dist/agent-src/rules/autonomous-execution.md +1 -1
  20. package/dist/agent-src/rules/command-suggestion-policy.md +33 -0
  21. package/dist/agent-src/rules/design-fidelity.md +1 -1
  22. package/dist/agent-src/rules/design-review-after-ui-write.md +36 -14
  23. package/dist/agent-src/rules/evaluator-independence.md +27 -5
  24. package/dist/agent-src/rules/git-history-discipline.md +5 -1
  25. package/dist/agent-src/rules/missing-skill-recovery.md +89 -0
  26. package/dist/agent-src/rules/ui-audit-gate.md +34 -12
  27. package/dist/agent-src/scripts/archive_completed_roadmaps.ts +33 -0
  28. package/dist/agent-src/scripts/gate_execute.ts +395 -0
  29. package/dist/agent-src/scripts/resume_probe.ts +422 -0
  30. package/dist/agent-src/scripts/roadmap_gates.ts +138 -4
  31. package/dist/agent-src/scripts/update_roadmap_progress.ts +70 -4
  32. package/dist/agent-src/skills/ai-council/SKILL.md +61 -0
  33. package/dist/agent-src/skills/authz-review/SKILL.md +4 -0
  34. package/dist/agent-src/skills/design-system-capture/SKILL.md +4 -0
  35. package/dist/agent-src/skills/design-system-capture/references/snapshot-preference-order.md +77 -0
  36. package/dist/agent-src/skills/experiment-loop/SKILL.md +116 -0
  37. package/dist/agent-src/skills/experiment-loop/evals/output-schema.yml +27 -0
  38. package/dist/agent-src/skills/experiment-loop/evals/triggers.json +17 -0
  39. package/dist/agent-src/skills/experiment-loop/references/pivot-ladder.md +54 -0
  40. package/dist/agent-src/skills/experiment-loop/references/protocol.md +61 -0
  41. package/dist/agent-src/skills/experiment-loop/references/register-format.md +66 -0
  42. package/dist/agent-src/skills/merge-conflicts/SKILL.md +4 -0
  43. package/dist/agent-src/skills/prompt-optimizer/SKILL.md +17 -1
  44. package/dist/agent-src/skills/roadmap-management/SKILL.md +2 -2
  45. package/dist/agent-src/skills/roadmap-writing/SKILL.md +5 -5
  46. package/dist/agent-src/skills/subagent-orchestration/schemas/team-review-status.json +8 -3
  47. package/dist/agent-src/skills/systematic-debugging/SKILL.md +4 -0
  48. package/dist/agent-src/skills/threat-modeling/SKILL.md +3 -0
  49. package/dist/agent-src/templates/agents/agent-project-settings.example.yml +1 -1
  50. package/dist/agent-src/templates/roadmaps.md +43 -3
  51. package/dist/cli/registry.js +1 -0
  52. package/dist/cli/registry.js.map +1 -1
  53. package/dist/cli-delegate/{chunk-K34QTSD5.js → chunk-7PZ4FSZB.js} +368 -191
  54. package/dist/cli-delegate/{chunk-55WZN5VF.js → chunk-EVTY7ISR.js} +2 -2
  55. package/dist/cli-delegate/{chunk-O52B4F5S.js → chunk-F3YUF7SA.js} +1 -1
  56. package/dist/cli-delegate/{chunk-LKZ5DOYW.js → chunk-XURJ4AQC.js} +11 -3
  57. package/dist/cli-delegate/cmd_conformance.js +224 -104
  58. package/dist/cli-delegate/cmd_doctor.js +2 -2
  59. package/dist/cli-delegate/cmd_export.js +2 -2
  60. package/dist/cli-delegate/cmd_handoff.js +1 -1
  61. package/dist/cli-delegate/cmd_prune.js +2 -2
  62. package/dist/cli-delegate/cmd_settings_migrate.js +2 -2
  63. package/dist/cli-delegate/cmd_settings_set.js +1 -1
  64. package/dist/cli-delegate/cmd_sync.js +2 -2
  65. package/dist/cli-delegate/cmd_uninstall.js +2 -2
  66. package/dist/cli-delegate/cmd_validate.js +2 -2
  67. package/dist/discovery/deprecation-report.md +1 -1
  68. package/dist/discovery/discovery-manifest.json +73 -30
  69. package/dist/discovery/discovery-manifest.json.sha256 +1 -1
  70. package/dist/discovery/discovery-manifest.summary.md +3 -3
  71. package/dist/discovery/dormancy-report.md +1 -1
  72. package/dist/discovery/orphan-report.md +1 -1
  73. package/dist/discovery/packs.json +7 -5
  74. package/dist/discovery/trust-report.md +3 -3
  75. package/dist/discovery/workspaces.json +6 -4
  76. package/dist/hooks/dispatch.js +2653 -1365
  77. package/dist/install/install.mjs +12 -4
  78. package/dist/mcp/registry-manifest.json +3 -3
  79. package/dist/mcp/server.json +2 -2
  80. package/dist/mcp/server.mjs +2679 -887
  81. package/dist/router.json +1 -1
  82. package/dist/scripts/condense.js +1 -1
  83. package/dist/scripts/condense.js.map +1 -1
  84. package/dist/server/schemas/settings.js +2 -2
  85. package/dist/server/schemas/settings.js.map +1 -1
  86. package/dist/ui/assets/{index-B0aMJnuz.js → index-C1pBRm1f.js} +2 -2
  87. package/dist/ui/assets/index-C1pBRm1f.js.map +1 -0
  88. package/dist/ui/index.html +1 -1
  89. package/docs/guidelines/design-fidelity-mechanics.md +21 -0
  90. package/docs/guidelines/design-handover-extraction.md +87 -0
  91. package/docs/guidelines/prompt-templates.md +24 -1
  92. package/package.json +1 -1
  93. package/src/agent-src/scripts/archive_completed_roadmaps.ts +33 -0
  94. package/src/agent-src/scripts/gate_execute.ts +395 -0
  95. package/src/agent-src/scripts/resume_probe.ts +422 -0
  96. package/src/agent-src/scripts/roadmap_gates.ts +138 -4
  97. package/src/agent-src/scripts/update_roadmap_progress.ts +70 -4
  98. package/src/config/ci-local-parity.yml +13 -0
  99. package/src/config/evaluator-budgets.json +9 -8
  100. package/src/config/gate-coverage.yml +140 -12
  101. package/src/config/gate-violation-baselines.json +9 -3
  102. package/src/config/hook-token-budget.json +8 -1
  103. package/src/config/rule-activation-census.json +41 -0
  104. package/src/config/rule-stub-ceilings.json +52 -0
  105. package/src/scripts/_dispatch.bash +4 -0
  106. package/src/scripts/_lib/bench_ab_complexity.ts +439 -0
  107. package/src/scripts/_lib/bench_ab_safety_tier.ts +142 -0
  108. package/src/scripts/_lib/bench_ab_scoring_v2.ts +28 -3
  109. package/src/scripts/_lib/bench_ab_search_adherence.ts +423 -0
  110. package/src/scripts/_lib/bench_ab_size_claim.ts +206 -0
  111. package/src/scripts/_lib/bench_ab_workspace.ts +111 -0
  112. package/src/scripts/_lib/evaluator_contract.ts +159 -0
  113. package/src/scripts/_lib/gh_transient.ts +80 -0
  114. package/src/scripts/_lib/release_material.ts +16 -5
  115. package/src/scripts/_lib/review_independence.ts +111 -0
  116. package/src/scripts/_lib/session_register.ts +14 -0
  117. package/src/scripts/_lib/skill_catalogue.ts +278 -5
  118. package/src/scripts/_lib/skill_catalogue_series.ts +411 -0
  119. package/src/scripts/_lib/tier_budget_routing.ts +31 -331
  120. package/src/scripts/_lib/turn_end_refusals.ts +482 -0
  121. package/src/scripts/ai_council/_py_json.ts +96 -0
  122. package/src/scripts/ai_council/cli_call_budget.ts +357 -0
  123. package/src/scripts/ai_council/clients.ts +101 -164
  124. package/src/scripts/ai_council/config.ts +25 -11
  125. package/src/scripts/ai_council/probe_store.ts +196 -0
  126. package/src/scripts/ai_council/qualification.ts +425 -0
  127. package/src/scripts/ai_council/qualification_wiring.ts +243 -0
  128. package/src/scripts/ai_council/quorum_wiring.ts +211 -0
  129. package/src/scripts/ai_team/team_dispatch.ts +40 -6
  130. package/src/scripts/bench_ab_v2_complexity.ts +252 -0
  131. package/src/scripts/bench_ab_v2_run.ts +3 -42
  132. package/src/scripts/bench_ab_v2_safety.ts +296 -0
  133. package/src/scripts/bench_ab_v2_search.ts +343 -0
  134. package/src/scripts/bench_ab_v2_stats.ts +191 -0
  135. package/src/scripts/build_archive_index.ts +383 -0
  136. package/src/scripts/capture_skill_catalogue.ts +369 -4
  137. package/src/scripts/check_evaluator_schema.ts +249 -0
  138. package/src/scripts/check_pr_ci_current.ts +409 -0
  139. package/src/scripts/check_review_schema.ts +195 -0
  140. package/src/scripts/check_rule_activation_census.ts +451 -0
  141. package/src/scripts/check_rule_stub_ceiling.ts +1128 -0
  142. package/src/scripts/check_source_size_budget.ts +428 -0
  143. package/src/scripts/check_suppression_hygiene.ts +5 -1
  144. package/src/scripts/condense.ts +1 -1
  145. package/src/scripts/cost/backfill_rates.mjs +298 -0
  146. package/src/scripts/cost/budget.mjs +12 -32
  147. package/src/scripts/cost_summary.ts +25 -0
  148. package/src/scripts/council_cli.ts +141 -191
  149. package/src/scripts/dispatch_r2_reviewer.ts +8 -1
  150. package/src/scripts/external_sources_denylist.json +1 -0
  151. package/src/scripts/hook_manifest.yaml +99 -6
  152. package/src/scripts/hooks/concern_registry.ts +6 -0
  153. package/src/scripts/hooks/dispatch_hook.ts +43 -1
  154. package/src/scripts/hooks/interruption_ledger_hook.ts +256 -0
  155. package/src/scripts/hooks/ship_diff_volume_hook.ts +180 -0
  156. package/src/scripts/hooks/skill_route_hook.ts +226 -0
  157. package/src/scripts/hooks/turn_end_gate_hook.ts +119 -26
  158. package/src/scripts/hooks/ui_route_nudge_hook.ts +6 -3
  159. package/src/scripts/hooks_doctor.ts +112 -0
  160. package/src/scripts/interruption_report.ts +443 -0
  161. package/src/scripts/lint_design_antipattern_parity.ts +40 -6
  162. package/src/scripts/lint_evidence_artifacts.ts +359 -0
  163. package/src/scripts/lint_harvest_provenance.ts +35 -1
  164. package/src/scripts/lint_roadmap_blockers.ts +65 -0
  165. package/src/scripts/lint_roadmap_complexity.ts +7 -1
  166. package/src/scripts/lint_rule_skill_pack_reach.ts +43 -0
  167. package/src/scripts/lint_skill_router_head.ts +21 -0
  168. package/src/scripts/mcp_server/consumer_tool_catalog.json +26 -3
  169. package/src/scripts/mcp_server/tool_catalog_source.ts +6 -24
  170. package/src/scripts/mcp_server/tools.ts +87 -0
  171. package/src/scripts/release.ts +43 -58
  172. package/src/scripts/routing_doctor.ts +50 -13
  173. package/src/scripts/rule_activation_census.ts +292 -0
  174. package/src/scripts/schemas/command.schema.json +1 -1
  175. package/src/scripts/schemas/evaluator-output.schema.json +60 -0
  176. package/src/scripts/schemas/review-findings.schema.json +31 -0
  177. package/src/scripts/schemas/skill.schema.json +18 -1
  178. package/src/scripts/self_review_gate.ts +6 -0
  179. package/src/scripts/session_register_hook.ts +30 -1
  180. package/src/scripts/sync_pr_branch.ts +271 -0
  181. package/src/scripts/test_council_qualification.ts +213 -0
  182. package/src/scripts/trigger_coverage.ts +224 -3
  183. package/src/server/schemas/settings.ts +2 -2
  184. package/dist/ui/assets/index-B0aMJnuz.js.map +0 -1
  185. package/src/config/budget-routing.json +0 -7
@@ -180,11 +180,11 @@ class RoadmapStats {
180
180
  }
181
181
 
182
182
  get open_blockers(): Blocker[] {
183
- return this.blockers.filter((b) => b.status !== 'resolved');
183
+ return this.blockers.filter((b) => !blocker_is_resolved(b));
184
184
  }
185
185
 
186
186
  get resolved_blockers(): Blocker[] {
187
- return this.blockers.filter((b) => b.status === 'resolved');
187
+ return this.blockers.filter((b) => blocker_is_resolved(b));
188
188
  }
189
189
 
190
190
  get done(): number {
@@ -365,6 +365,49 @@ interface Blocker {
365
365
  * costs one enum entry and turns a silent swallow into a rendered field.
366
366
  */
367
367
  question: string;
368
+ /**
369
+ * Gate taxonomy: `0` auto-run · `1` budget-preauthorized · `2`
370
+ * consent-once · `3` human-only. Empty string when the entry declares
371
+ * none, and an absent class READS AS 3 at every consumer.
372
+ *
373
+ * The default is the safe end on purpose. An agent that inferred the class
374
+ * at runtime could promote a Hard-Floor gate into something it executes;
375
+ * authoring it into the file means a misclassification is a reviewed edit,
376
+ * and `lint_roadmap_blockers` refuses a runnable class that names no
377
+ * command.
378
+ */
379
+ blockerClass: string;
380
+ /** The command a class-0/1 entry is resolved by running. */
381
+ run: string;
382
+ /** Spend estimate for class 1 — recorded, never enforced by the lint. */
383
+ budget: string;
384
+ }
385
+
386
+ /**
387
+ * Is this blocker settled?
388
+ *
389
+ * A PREFIX test, not equality, and the difference was measured: authors write
390
+ * the resolution into the status line — `**Status:** RESOLVED 2026-08-17 —
391
+ * **option (b)**, narrowed to two rules` — and an equality check matches
392
+ * neither `open` nor `resolved`, so the entry falls through to open. Found
393
+ * 2026-08-17 in `road-to-mixed-trigger-activation-cost`: a decision taken two
394
+ * days earlier was still being rendered by `agent-config gates` as a live one
395
+ * the reader owed an answer to, and it inflated every blocker count in the
396
+ * dashboard by one.
397
+ *
398
+ * `lint_roadmap_blockers` already read the same field as a prefix
399
+ * (`/^-\s*\*\*Status:\*\*\s*resolved/i`), so the two surfaces disagreed about
400
+ * what "resolved" means — the lint exempted the entry from its decidability
401
+ * ratchet while the dashboard counted it open.
402
+ */
403
+ function blocker_is_resolved(b: Blocker): boolean {
404
+ return /^resolved\b/.test(_strip(b.status).toLowerCase());
405
+ }
406
+
407
+ /** Class of a blocker, with the absent-field default applied. */
408
+ function blocker_class(b: Blocker): '0' | '1' | '2' | '3' {
409
+ const c = _strip(b.blockerClass).split(/[\s—–-]/)[0] as string;
410
+ return c === '0' || c === '1' || c === '2' ? c : '3';
368
411
  }
369
412
 
370
413
  // Strip fenced code blocks before blocker detection — a roadmap that shows the
@@ -380,8 +423,23 @@ function _stripFencedCode(text: string): string {
380
423
  return text.replace(FENCED_CODE_RE, (m) => '\n'.repeat(_splitlines(m).length));
381
424
  }
382
425
 
383
- const BLOCKER_FIELD_RE =
384
- /^-[ \t]*\*\*(Status|Owner|Blocks|Resolved when|What to do|Recommendation|If you do nothing|Question):\*\*/i;
426
+ /**
427
+ * Terminator for a blocker field's continuation lines — ANY `- **Label:**`
428
+ * bullet, not a closed list of known labels.
429
+ *
430
+ * It used to enumerate eight names, which silently diverged from the contract
431
+ * the two call sites below already document ("up to the next `- **Field:**`
432
+ * marker"): a bullet whose label was not on the list did not terminate, so the
433
+ * PREVIOUS field kept absorbing it and the dashboard rendered the two as one
434
+ * run-on sentence. Measured when `- **Options:**` and `- **Side finding:**`
435
+ * first appeared in a blocker — a reader saw the side finding as part of
436
+ * `Resolved when`, i.e. the generator changed what a field meant.
437
+ *
438
+ * A closed list is the wrong shape for a terminator: every future field name
439
+ * would have to be added here to avoid corrupting the field above it, and the
440
+ * failure is silent in generated output nobody edits by hand.
441
+ */
442
+ const BLOCKER_FIELD_RE = /^-[ \t]*\*\*[^*]+:\*\*/;
385
443
 
386
444
  /** Strip an inline `<!-- comment -->` and trim. */
387
445
  function _stripComment(s: string): string {
@@ -493,6 +551,9 @@ function parse_blockers(raw_text: string): Blocker[] {
493
551
  recommendation: _blockerField(body, 'Recommendation') ?? '',
494
552
  ifNothing: _blockerField(body, 'If you do nothing') ?? '',
495
553
  question: _blockerField(body, 'Question') ?? '',
554
+ blockerClass: _blockerField(body, 'Class') ?? '',
555
+ run: _blockerField(body, 'Run') ?? '',
556
+ budget: _blockerField(body, 'Budget') ?? '',
496
557
  });
497
558
  }
498
559
  }
@@ -527,6 +588,9 @@ function parse_blockers(raw_text: string): Blocker[] {
527
588
  recommendation: '',
528
589
  ifNothing: '',
529
590
  question: '',
591
+ blockerClass: '',
592
+ run: '',
593
+ budget: '',
530
594
  });
531
595
  }
532
596
  return blockers;
@@ -1172,6 +1236,8 @@ export {
1172
1236
  is_roadmap_candidate,
1173
1237
  count_checkboxes,
1174
1238
  parse_blockers,
1239
+ blocker_class,
1240
+ blocker_is_resolved,
1175
1241
  blocker_needs_user,
1176
1242
  parse_roadmap,
1177
1243
  bar,
@@ -17,6 +17,67 @@ packs:
17
17
 
18
18
  # ai-council
19
19
 
20
+ ## Council-first — try the council BEFORE the user, on these classes
21
+
22
+ ```
23
+ A DESIGN DECISION THE AGENT CANNOT SETTLE FROM THE TREE GOES TO THE COUNCIL
24
+ FIRST, NOT TO THE USER FIRST. THE USER IS THE ESCALATION, NOT THE DEFAULT.
25
+ NEVER PRESENT A COUNCIL VERDICT THAT WAS NOT REACHED. AN INCONCLUSIVE RUN
26
+ IS REPORTED AS INCONCLUSIVE, AND WHATEVER YOU DECIDE INSTEAD IS LABELLED
27
+ OWN ANALYSIS.
28
+ ```
29
+
30
+ Interrupting the user is the expensive path: it costs a round trip and their
31
+ attention, and on a question two models could have settled it buys nothing they
32
+ wanted to be asked. So the ordering is **tree → council → user**, and the
33
+ council leg is the one that gets skipped.
34
+
35
+ **Classes that go to the council first** — each is a judgement with a real
36
+ trade-off and no answer in the tree:
37
+
38
+ | Class | Example |
39
+ |---|---|
40
+ | Competing implementations of one feature | two sessions shipped the same phase; discard, overwrite, or merge |
41
+ | Which of N designs, all defensible | four declared types with a derived fifth, or five declared |
42
+ | Reopening a recorded decision | an ADR or honest-null blocks a change that now looks net-positive |
43
+ | Scope re-cut inside an authorized task | this phase splits into two, or absorbs the next |
44
+ | Is the evidence sufficient to conclude | n=1 measurement, a proxy metric, an unreplicated finding |
45
+
46
+ **Classes that still go straight to the user, and the council never substitutes
47
+ for them:** anything behind the Hard Floor
48
+ ([`non-destructive-by-default`](../../rules/non-destructive-by-default.md)) ·
49
+ spend · a scope EXPANSION beyond what was authorized · a decision whose inputs
50
+ only the user holds (a preference, a deadline, a business constraint) · the
51
+ `high_impact` and `user_required` classes, which
52
+ [`ask-when-uncertain`](../../rules/ask-when-uncertain.md) routes to the user by
53
+ Iron Law. A council cannot consent on the user's behalf, and asking it to is not
54
+ autonomy, it is laundering.
55
+
56
+ ### When the council cannot answer
57
+
58
+ Measured 2026-08-17: a run returned **0 of 2 seats**, both `cli_quota_exhausted`,
59
+ with the counters at 72/50 and 99/50. The CLI reported it correctly as
60
+ `INCONCLUSIVE — DEGRADED` and did not print a quorum it had not reached. Three
61
+ things follow, and the first is the one under pressure:
62
+
63
+ 1. **Never dress your own verdict as the council's.** A fan-out, a solo read, or
64
+ your own judgement is a legitimate substitute **only when named as such** —
65
+ the boundary [`council-availability`](../../rules/council-availability.md)
66
+ states.
67
+ 2. **Do not bounce to the user just because the council failed.** An
68
+ unreachable council does not upgrade a decidable question into a
69
+ user-required one. Decide it, label it own analysis, state the reason the
70
+ council was unavailable, and name what would change the answer.
71
+ 3. **Escalate only if it is genuinely undecidable without them** — the two
72
+ options are close AND the cost of being wrong is high AND nothing in the tree
73
+ separates them. Then say the council was attempted and why it failed, so the
74
+ user is not asked to arbitrate something they will assume was already tried.
75
+
76
+ `agent-config council:status` is free and answers availability; `council_cli
77
+ estimate <file>` is free and answers parse-and-cost. Run both before concluding
78
+ the council is not an option — and note the question goes in a **file**, since
79
+ prose passed as an argument overflows the argument limit.
80
+
20
81
  ## When to use
21
82
 
22
83
  * The host agent has drafted a roadmap, plan, or design and wants an
@@ -11,6 +11,10 @@ workspaces:
11
11
  - engineering
12
12
  packs:
13
13
  - engineering-base
14
+ triggers:
15
+ - phrase: "authorization check"
16
+ - phrase: "permission check"
17
+ - keyword: "authz"
14
18
  ---
15
19
 
16
20
  # authz-review
@@ -306,6 +306,10 @@ handoff shape emitted alongside `design-system.json`
306
306
  — when the artifact is a mock — an explicit "prototype, not production code"
307
307
  warning so a consumer never ships the reference verbatim.
308
308
 
309
+ **What to capture FROM** a running project: sources first, SSR fetch second,
310
+ rendered snapshot only with consumer tooling; record the git SHA. Rationale:
311
+ [snapshot preference order](references/snapshot-preference-order.md).
312
+
309
313
  ## Gotcha
310
314
 
311
315
  - **Over-writing instead of diff-updating** — the most common failure.
@@ -0,0 +1,77 @@
1
+ # Snapshot preference order — what to capture a design system FROM
2
+
3
+ When a design system has to be captured out of a running project — for a
4
+ handoff, an external design pass, or a reference bundle — there is more than one
5
+ thing you could point at, and they are not equally good. This is the preference
6
+ order, highest fidelity per unit of effort first.
7
+
8
+ **Take the first level the project supports. Do not skip down the list for
9
+ convenience, and do not skip UP it for thoroughness** — Level C is additional
10
+ evidence, never a replacement for Level A.
11
+
12
+ ## Level A — source-level (always available, the default)
13
+
14
+ The involved component sources, plus the token layer (DTCG / CSS custom
15
+ properties), plus the page composition that assembles them.
16
+
17
+ **Why this is the default and not the fallback**, which is the part that reads
18
+ backwards until you have seen the alternative: in a component project, a
19
+ rendered DOM dump is *worse* input than the sources it came from. Hydrated
20
+ markup is utility-class soup with the component boundaries erased — the very
21
+ structure a design system is made of is the structure rendering destroys. A
22
+ reader (human or tool) reworks clean sources far better than expanded markup.
23
+
24
+ Level A needs no dev server, no browser, and no network. It is available in a
25
+ bare checkout, which is why it is also the only level that always works.
26
+
27
+ ## Level B — SSR fetch (no browser)
28
+
29
+ When the stack renders server-side and a dev server is running, an ordinary
30
+ HTTP fetch of the rendered page.
31
+
32
+ Use it to answer what Level A cannot: what the composition actually resolves to
33
+ for a given route — conditional branches taken, data-dependent states, the real
34
+ class strings after server-side composition. Zero browser dependency, so it
35
+ stays cheap.
36
+
37
+ Level B **supplements** Level A. Capturing B alone leaves you with the soup
38
+ problem above and no component boundaries.
39
+
40
+ ## Level C — rendered snapshot (consumer tooling only)
41
+
42
+ If the project already carries a browser-automation tool, or a DevTools bridge
43
+ is available in the session, additionally capture a screenshot plus a
44
+ computed-style extract as visual ground truth.
45
+
46
+ **Capability-detect, never hard-depend.** Level C is available when the consumer
47
+ already has the tooling; it is never a reason to install a browser stack, add a
48
+ dependency, or fail a capture. A project without it captures at A (and B where
49
+ applicable) and says so.
50
+
51
+ Of the two artefacts, **the screenshot is the valuable part** — it is the only
52
+ one that records what a person actually sees. The DOM dump is supplementary and
53
+ carries the erasure problem Level A exists to avoid.
54
+
55
+ ## Every snapshot records the source revision
56
+
57
+ Whatever level was captured, record the git SHA the capture was taken at.
58
+
59
+ This is the anchor that makes a return leg possible: without it, reworked output
60
+ cannot be diffed against what was sent, and a capture from an unknown state is
61
+ evidence about nothing in particular. It costs one line and it is the difference
62
+ between a snapshot and a screenshot.
63
+
64
+ ## State the level in the output
65
+
66
+ A capture says which level it reached and why it stopped there — "Level A + B;
67
+ no browser tooling detected, so no visual ground truth". A consumer reading the
68
+ bundle can then tell *absent because unavailable* from *absent because
69
+ overlooked*, which is the same distinction the extraction floor draws for
70
+ not-extractable values.
71
+
72
+ ## See also
73
+
74
+ - [`design-system-json.md`](design-system-json.md) — the import contract the
75
+ captured artefact is emitted against.
76
+ - [`../SKILL.md`](../SKILL.md) § Design-system extraction floor — what a
77
+ complete extraction carries once the source has been chosen.
@@ -0,0 +1,116 @@
1
+ ---
2
+ model_tier: inherit
3
+ name: experiment-loop
4
+ description: "Use to drive a scalar metric down or up across bounded iterations — keep on strict improvement, revert otherwise, state on disk. Triggers 'minimize X', 'optimize until it stops improving'."
5
+ domain: process
6
+ workspaces:
7
+ - agent-config-maintainer
8
+ packs:
9
+ - meta
10
+ ---
11
+
12
+ # experiment-loop
13
+
14
+ > A bounded **change → commit → evaluate → keep-or-revert** cycle against a
15
+ > **scalar metric**, where the decision input is an
16
+ > [evaluator-output](../../../docs/contracts/evaluator-output.md) verdict and the
17
+ > loop's entire state lives in an append-only register on disk.
18
+
19
+ This file **routes**. The protocol, the register format, and the pivot ladder are
20
+ reference files loaded on demand — a loop doctrine read in full on every session
21
+ that merely mentions optimization is a payload nobody asked for.
22
+
23
+ | Need | Load |
24
+ |---|---|
25
+ | The iteration protocol and its two exit conditions | [`references/protocol.md`](references/protocol.md) |
26
+ | The register's row shape and write ordering | [`references/register-format.md`](references/register-format.md) |
27
+ | What to do when the metric stops moving | [`references/pivot-ladder.md`](references/pivot-ladder.md) |
28
+
29
+ ## When to use
30
+
31
+ * The goal is expressible as **"minimize X"** or **"maximize X"** against a
32
+ mechanical verifier — bundle size, violation count, query count, runtime.
33
+ * A verifier already emits, or can be wrapped to emit, the evaluator-output
34
+ contract. Wrapping is usually free: three of this repo's verifiers were wrapped
35
+ with zero changes to any of them.
36
+
37
+ Do **NOT** use when:
38
+
39
+ * The target is a **known end state**, not a number — "make the tests pass" has a
40
+ destination, so use [`verify-repair-loop`](../verify-repair-loop/SKILL.md),
41
+ which converges toward it. This skill has no destination; it has a direction.
42
+ * The verdict is **subjective craft** — no scalar, nothing to optimize.
43
+ * The metric is a **proxy nobody agreed on**. Optimizing an unowned proxy is how
44
+ a loop makes a system worse while its number improves.
45
+
46
+ ## The Iron Law
47
+
48
+ ```
49
+ THE METRIC IS NEVER SOVEREIGN. `pass` IS.
50
+ KEEP ONLY ON `pass && score > baseline`. A CHANGE THAT IMPROVES THE NUMBER
51
+ AND BREAKS BEHAVIOUR IS REVERTED — MEASURED, NOT ASSUMED.
52
+ COMMIT BEFORE YOU EVALUATE, SO EVERY REVERT IS A GIT OPERATION.
53
+ THE REGISTER IS WRITTEN BEFORE THE ACTION IT RECORDS, NEVER AFTER.
54
+ STATE LIVES ON DISK AND IS RE-READ EACH CYCLE — NEVER IN THE CONVERSATION.
55
+ ```
56
+
57
+ Each line is a measured failure, not a preference. Spike s01 ran a change that
58
+ improved its metric by 67 % and broke the behaviour it measured; only `pass`
59
+ reverted it. The same run crashed between its commit and its register append,
60
+ twice, leaving the branch one iteration ahead of its own record.
61
+
62
+ ## Procedure
63
+
64
+ 1. **Inspect the verifier before trusting it.** Run the evaluator once on the
65
+ unchanged tree and check its verdict: `pass` must be true and `metric_state`
66
+ must be `present`. A baseline taken from an evaluator that is already red, or
67
+ whose metric is unreadable, is not a baseline — every later comparison
68
+ inherits the defect silently.
69
+ 2. **Fix the metric and the verifier** before the first change. Name the
70
+ evaluator, its direction, and the baseline score.
71
+ 3. **Declare the bound** — a maximum iteration count, written into the register's
72
+ first row. An unbounded loop is not this skill.
73
+ 4. **Iterate** per [`references/protocol.md`](references/protocol.md): one focused
74
+ change, commit, evaluate, keep or revert, append the outcome.
75
+ 5. **Exit** on either condition (both machine-checkable, never prose): the bound
76
+ is reached, or the explicit exit signal fires.
77
+ 6. **Report** from the register, not from memory. If the register cannot
78
+ reconstruct the run without the transcript, the run is not reportable.
79
+
80
+ ## Validation
81
+
82
+ * The register reconstructs the run with the transcript discarded — every kept
83
+ change traceable to a row, every row to a commit.
84
+ * Both branches ran at least once. **A loop whose revert branch never executed
85
+ is a one-branch loop with a comment**; spike s01 found its revert broken on
86
+ first use after four green iterations.
87
+ * The final tree passes the evaluator with `pass: true`.
88
+
89
+ ## Output format
90
+
91
+ 1. **The register path**, and the iteration count actually run against the bound.
92
+ 2. **A metric line**: baseline → final, and the number of keeps versus reverts.
93
+ 3. **The exit condition that fired**, named — bound reached, or exit signal.
94
+ 4. **Every reverted iteration listed** with its reason. A revert is a result the
95
+ next run needs, not noise to summarise away.
96
+
97
+ ## Gotcha
98
+
99
+ * **A metric that improves while the system degrades.** The reason `pass` is
100
+ sovereign. Never widen a verifier to make an iteration keep.
101
+ * **A revert path that has never run.** Exercise it deliberately once.
102
+ * **A register appended after the mutation.** A crash in between leaves the
103
+ branch ahead of the record; write the row first, then update its outcome.
104
+ * **Loop state in the conversation.** It decays silently and the loop keeps
105
+ comparing against a baseline it no longer holds.
106
+ * **Iterating past the bound because the number is still moving.** The bound is
107
+ the contract; extending it is a new run with a new baseline.
108
+
109
+ ## Do NOT
110
+
111
+ * Do NOT keep a change on `score` alone.
112
+ * Do NOT edit the verifier to move the metric.
113
+ * Do NOT batch several changes into one iteration — a keep you cannot attribute
114
+ is a keep you cannot revert.
115
+ * Do NOT delete a register row. Rows are append-only, including the embarrassing
116
+ ones.
@@ -0,0 +1,27 @@
1
+ # Behavior-schema contract for `experiment-loop`.
2
+ #
3
+ # Locks the sections a refactor could quietly drop while the skill still
4
+ # reads plausibly. Each one is a measured failure from spike s01
5
+ # (agents/evidence/eval-findings/metric-loop-s01.md), not a style
6
+ # preference:
7
+ #
8
+ # The Iron Law — `pass` sovereign over `score`; a change that improved
9
+ # the metric 67% and broke behaviour was reverted only
10
+ # because of it.
11
+ # Validation — the both-branches-ran check; s01's revert path was
12
+ # broken and four green iterations never noticed.
13
+ # Output format — the register-derived report; a report from memory is
14
+ # the in-conversation state the register exists to replace.
15
+ # Do NOT — the four keeps that must not happen.
16
+ #
17
+ # Enforced by `skill_linter` (`lint_output_schema`, matches `^## <header>$`) —
18
+ # a missing section is a hard error.
19
+
20
+ version: 1
21
+
22
+ required_headers:
23
+ - "The Iron Law"
24
+ - "Procedure"
25
+ - "Validation"
26
+ - "Output format"
27
+ - "Do NOT"
@@ -0,0 +1,17 @@
1
+ {
2
+ "skill": "experiment-loop",
3
+ "last_eval": "2026-08-17",
4
+ "description": "5 should-trigger + 5 should-not-trigger. Should-trigger covers the bounded keep-or-revert loop against a SCALAR metric (direction, no destination). Should-not-trigger covers the near-miss neighbours the roadmap's gap-table names: iterate-to-a-known-end-state (verify-repair-loop), subjective craft with no scalar, a one-shot measurement, a one-shot review, and asking what a metric currently is.",
5
+ "queries": [
6
+ {"q": "minimize the bundle size, keep changes that help and revert the rest", "trigger": true},
7
+ {"q": "optimize this until it stops improving, bounded to ten tries", "trigger": true},
8
+ {"q": "drive the query count down across iterations and record what you kept", "trigger": true},
9
+ {"q": "run a keep-or-revert loop against the lint violation count", "trigger": true},
10
+ {"q": "reduce this metric as far as it goes without breaking the tests", "trigger": true},
11
+ {"q": "keep fixing until the tests pass", "trigger": false, "note": "a known end state, not a direction → verify-repair-loop"},
12
+ {"q": "iterate this change to green, run the suite each time", "trigger": false, "note": "converges toward a destination → verify-repair-loop"},
13
+ {"q": "is this diff clean — naming, structure, conventions?", "trigger": false, "note": "subjective craft, no scalar to optimize"},
14
+ {"q": "how big is the bundle right now?", "trigger": false, "note": "a single measurement, no loop"},
15
+ {"q": "review my changes before I open the PR", "trigger": false, "note": "one-shot review → /review-changes"}
16
+ ]
17
+ }
@@ -0,0 +1,54 @@
1
+ # experiment-loop — the pivot ladder
2
+
3
+ Loaded on demand by [`experiment-loop`](../SKILL.md), when the exit signal
4
+ `consecutive_reverts >= N` fires. The metric has stopped moving. The question is
5
+ whether that is a fact about the system or about the hypotheses being tried.
6
+
7
+ Climb one rung at a time. Each rung is a **new run** with a fresh register and a
8
+ re-declared baseline — never a quiet continuation of the old one, because a run
9
+ whose hypothesis class changed mid-flight cannot be read afterwards.
10
+
11
+ ## Rung 1 — a different hypothesis class, same metric
12
+
13
+ The reverts share a shape: every attempt tried to shave the same structure. Pick
14
+ a class the run has not touched — a different module, a different layer, a
15
+ different kind of change — and rerun with the current score as the baseline.
16
+
17
+ Stop climbing here if the first iteration of the new class keeps.
18
+
19
+ ## Rung 2 — check the metric measures what you meant
20
+
21
+ Before concluding the system is optimal, confirm the number is the number. The
22
+ measured failure mode: a diff-volume metric that looked like shipping volume was
23
+ counting the repository's own review bookkeeping — a committed copy of the diff
24
+ being measured — so the ranking it produced was of the wrong thing entirely.
25
+ Excluding the bookkeeping moved the p90 by more than half and reordered the top
26
+ five.
27
+
28
+ Ask: does the metric include artefacts the change did not author? Generated
29
+ files, snapshots, vendored trees, its own output? Correcting that is a
30
+ **measurement fix** and legitimate.
31
+
32
+ ## Rung 3 — decide whether tuning is the honest move
33
+
34
+ Rung 2's correction has a twin that is not legitimate: adjusting the metric until
35
+ the answer is pleasant. The discriminator is stated once, plainly:
36
+
37
+ > **Removing an input that was never part of the quantity is a correction.
38
+ > Moving a threshold or growing an exclusion set until the result improves is
39
+ > tuning.** A correction can be justified without reference to its effect on the
40
+ > outcome. Tuning cannot.
41
+
42
+ If the only available move needs the outcome to justify it, stop. The run's
43
+ answer is that the metric does not move under changes of this kind, and that is
44
+ a real result worth recording.
45
+
46
+ ## Rung 4 — publish the null
47
+
48
+ No rung produced a keep. Write the run up: the metric, the bound, the hypothesis
49
+ classes tried, the register path, and the conclusion that the metric did not
50
+ move. A published null is why the next person does not re-run this.
51
+
52
+ **Do not climb back down.** Re-running rung 1 with slightly different wording is
53
+ the shape that turns a bounded loop into an unbounded one, and it burns budget
54
+ producing rows that all say the same thing.
@@ -0,0 +1,61 @@
1
+ # experiment-loop — the iteration protocol
2
+
3
+ Loaded on demand by [`experiment-loop`](../SKILL.md). One iteration, in order.
4
+
5
+ ## The cycle
6
+
7
+ 1. **Read the register from disk.** Not from the conversation, not from a
8
+ variable held since the last cycle. The baseline for this iteration is the
9
+ `score` of the most recent `keep` row, or the declared baseline if there is
10
+ none.
11
+ 2. **Make ONE focused change.** One hypothesis per iteration. A batched
12
+ iteration produces a keep you cannot attribute and therefore cannot revert.
13
+ 3. **Append the register row** with `decision: "pending"` — before the commit,
14
+ before the evaluation. See [`register-format.md`](register-format.md) for why
15
+ the ordering is this way round.
16
+ 4. **Commit.** The revert is then a git operation rather than a manual undo, and
17
+ the branch carries one commit per iteration whether it survives or not.
18
+ 5. **Evaluate** — run the evaluator, read its
19
+ [evaluator-output](../../../../../docs/contracts/evaluator-output.md) verdict.
20
+ 6. **Decide**, and update the row:
21
+
22
+ | Verdict | Decision |
23
+ |---|---|
24
+ | `pass: true` and `score > baseline` | **keep** |
25
+ | `pass: true` and `score <= baseline` | **revert** — no strict improvement |
26
+ | `pass: false` | **revert** — the tree is red |
27
+ | non-zero exit · absent JSON · timeout | **revert** — the experiment failed, the measurement did not happen |
28
+ | `metric_state: "unreadable"` | **revert** — an unread metric is not a zero |
29
+
30
+ 7. **Revert if the decision says so**, then loop.
31
+
32
+ ## The two exit conditions
33
+
34
+ Both are **machine-checkable predicates**. Neither is prose, and that is
35
+ deliberate: a stop condition written loosely rots, and the rot is invisible
36
+ because the loop keeps running.
37
+
38
+ **Condition A — the bound.** `iterations_run >= max_iterations`, where
39
+ `max_iterations` was written into the register's first row before the first
40
+ change. Reaching it is a normal exit, not a failure.
41
+
42
+ **Condition B — the exit signal.** A predicate declared at the start, evaluated
43
+ against the register each cycle. Two usable shapes:
44
+
45
+ - `consecutive_reverts >= N` — the metric has stopped moving under this
46
+ hypothesis class. Route to [`pivot-ladder.md`](pivot-ladder.md).
47
+ - `score >= target` — the run had a destination after all, and reached it.
48
+
49
+ **Why two and not one.** A bound alone burns the whole budget on a metric that
50
+ stopped moving at iteration three. An exit signal alone is an unbounded loop
51
+ wearing a condition: if the signal never fires — and a subtly wrong predicate
52
+ never fires — nothing stops it. Each guards a different failure direction, so
53
+ neither is the redundant one.
54
+
55
+ ## What ends a run that is not an exit condition
56
+
57
+ Nothing. Specifically **not**: the number looking good enough, the change
58
+ starting to feel risky, or the register getting long. Those are judgements, and
59
+ a loop that stops on a judgement produces a result nobody can reproduce. If the
60
+ run should stop for a reason the predicates do not cover, that is a finding
61
+ about the predicates — record it and fix them for the next run.
@@ -0,0 +1,66 @@
1
+ # experiment-loop — the register
2
+
3
+ Loaded on demand by [`experiment-loop`](../SKILL.md). The register is the run:
4
+ if it cannot reconstruct what happened with the transcript discarded, the run
5
+ was not recorded.
6
+
7
+ ## Shape
8
+
9
+ An **append-only JSONL** file, one row per iteration, **untracked** (the run is
10
+ evidence, not source). Re-read from disk at the top of every cycle.
11
+
12
+ Row 0 declares the run:
13
+
14
+ ```json
15
+ {"row": 0, "evaluator": "bundle-size", "direction": "minimize",
16
+ "baseline_score": -142000, "max_iterations": 12,
17
+ "exit_signal": "consecutive_reverts >= 3"}
18
+ ```
19
+
20
+ Each iteration:
21
+
22
+ ```json
23
+ {"row": 1, "label": "drop the polyfill for evergreen targets", "head": "a1b2c3d",
24
+ "baseline_score": -142000, "score": -138400, "metric": 138400,
25
+ "metric_state": "present", "pass": true, "decision": "keep",
26
+ "reason": "strict improvement"}
27
+ ```
28
+
29
+ `decision` is one of `pending` · `keep` · `revert`. `reason` is one of
30
+ `strict improvement` · `no strict improvement` · `test red` ·
31
+ `evaluator failed` · `metric unreadable`.
32
+
33
+ ## Write the row BEFORE the action it records
34
+
35
+ This is the ordering that costs a run to learn. The obvious sequence — commit,
36
+ evaluate, revert, then append — leaves a window in which the branch has advanced
37
+ and the register has not. Spike s01 hit it twice: the revert threw, the process
38
+ died, and the row was never written, so the register described a run one
39
+ iteration shorter than the branch.
40
+
41
+ So: append the row with `decision: "pending"` first, then act, then rewrite that
42
+ row's decision. A crash then leaves a visible `pending` — a row that says "an
43
+ iteration started here and we do not know how it ended", which is recoverable —
44
+ instead of silence, which is not.
45
+
46
+ **"Append-only" and "rewrite the last row" are not in conflict.** Append-only
47
+ governs *history*: no row is ever deleted and no earlier row is ever edited. The
48
+ pending row is the write frontier, not history, until its outcome lands.
49
+
50
+ ## Why on disk and not in the conversation
51
+
52
+ Two independent references converged on files-as-truth after documenting silent
53
+ degradation of in-conversation loop state, and the mechanism is not exotic: a
54
+ baseline held in context drifts as the context is summarised, and the loop keeps
55
+ comparing against a number that is no longer the one it kept. Nothing errors.
56
+ The run just quietly stops meaning anything.
57
+
58
+ Re-reading from disk each cycle costs one file read and removes the entire
59
+ class.
60
+
61
+ ## Reporting from the register
62
+
63
+ The final report is derived, never remembered: iterations run against the bound,
64
+ baseline → final score, keeps versus reverts, the exit condition that fired, and
65
+ every reverted iteration with its reason. A revert is the most informative row in
66
+ the file — it is the hypothesis the metric rejected, and the next run needs it.
@@ -7,6 +7,10 @@ workspaces:
7
7
  - engineering
8
8
  packs:
9
9
  - engineering-base
10
+ triggers:
11
+ - phrase: "merge conflict"
12
+ - phrase: "rebase conflict"
13
+ - phrase: "resolve conflicts"
10
14
  ---
11
15
 
12
16
  # merge-conflicts