@event4u/agent-config 12.1.0 → 14.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +1 -1
- package/CHANGELOG.md +164 -186
- package/README.md +1 -1
- package/dist/agent-src/commands/cost/report.md +23 -0
- package/dist/agent-src/commands/evals/roadmap.json +8 -0
- package/dist/agent-src/commands/fix/ci.md +39 -1
- package/dist/agent-src/commands/pr/create.md +65 -4
- package/dist/agent-src/commands/roadmap/next.md +28 -5
- package/dist/agent-src/commands/work.md +16 -0
- package/dist/agent-src/contexts/contracts/command-suggestion-flow.md +47 -0
- package/dist/agent-src/contexts/execution/contract-decision-sheet.md +94 -0
- package/dist/agent-src/contexts/execution/plan-confidence-gate.md +30 -0
- package/dist/agent-src/contexts/execution/roadmap-execution-contract.md +21 -8
- package/dist/agent-src/contexts/execution/roadmap-process-loop.md +55 -8
- package/dist/agent-src/contexts/execution/subagent-response-contract.md +42 -0
- package/dist/agent-src/contexts/execution/subagent-routing.md +148 -33
- package/dist/agent-src/contexts/execution/subagent-spawn-contract.md +29 -0
- package/dist/agent-src/contexts/subagent-configuration.md +69 -7
- package/dist/agent-src/rules/autonomous-execution.md +1 -1
- package/dist/agent-src/rules/command-suggestion-policy.md +33 -0
- package/dist/agent-src/rules/design-fidelity.md +1 -1
- package/dist/agent-src/rules/design-review-after-ui-write.md +36 -14
- package/dist/agent-src/rules/evaluator-independence.md +27 -5
- package/dist/agent-src/rules/git-history-discipline.md +5 -1
- package/dist/agent-src/rules/missing-skill-recovery.md +89 -0
- package/dist/agent-src/rules/ui-audit-gate.md +34 -12
- package/dist/agent-src/scripts/archive_completed_roadmaps.ts +33 -0
- package/dist/agent-src/scripts/gate_execute.ts +395 -0
- package/dist/agent-src/scripts/resume_probe.ts +422 -0
- package/dist/agent-src/scripts/roadmap_gates.ts +138 -4
- package/dist/agent-src/scripts/update_roadmap_progress.ts +70 -4
- package/dist/agent-src/skills/ai-council/SKILL.md +61 -0
- package/dist/agent-src/skills/authz-review/SKILL.md +4 -0
- package/dist/agent-src/skills/design-system-capture/SKILL.md +4 -0
- package/dist/agent-src/skills/design-system-capture/references/snapshot-preference-order.md +77 -0
- package/dist/agent-src/skills/experiment-loop/SKILL.md +116 -0
- package/dist/agent-src/skills/experiment-loop/evals/output-schema.yml +27 -0
- package/dist/agent-src/skills/experiment-loop/evals/triggers.json +17 -0
- package/dist/agent-src/skills/experiment-loop/references/pivot-ladder.md +54 -0
- package/dist/agent-src/skills/experiment-loop/references/protocol.md +61 -0
- package/dist/agent-src/skills/experiment-loop/references/register-format.md +66 -0
- package/dist/agent-src/skills/merge-conflicts/SKILL.md +4 -0
- package/dist/agent-src/skills/prompt-optimizer/SKILL.md +17 -1
- package/dist/agent-src/skills/roadmap-management/SKILL.md +2 -2
- package/dist/agent-src/skills/roadmap-writing/SKILL.md +5 -5
- package/dist/agent-src/skills/subagent-orchestration/schemas/team-review-status.json +8 -3
- package/dist/agent-src/skills/systematic-debugging/SKILL.md +4 -0
- package/dist/agent-src/skills/threat-modeling/SKILL.md +3 -0
- package/dist/agent-src/templates/agents/agent-project-settings.example.yml +1 -1
- package/dist/agent-src/templates/roadmaps.md +43 -3
- package/dist/cli/registry.js +1 -0
- package/dist/cli/registry.js.map +1 -1
- package/dist/cli-delegate/{chunk-K34QTSD5.js → chunk-7PZ4FSZB.js} +368 -191
- package/dist/cli-delegate/{chunk-55WZN5VF.js → chunk-EVTY7ISR.js} +2 -2
- package/dist/cli-delegate/{chunk-O52B4F5S.js → chunk-F3YUF7SA.js} +1 -1
- package/dist/cli-delegate/{chunk-LKZ5DOYW.js → chunk-XURJ4AQC.js} +11 -3
- package/dist/cli-delegate/cmd_conformance.js +224 -104
- package/dist/cli-delegate/cmd_doctor.js +2 -2
- package/dist/cli-delegate/cmd_export.js +2 -2
- package/dist/cli-delegate/cmd_handoff.js +1 -1
- package/dist/cli-delegate/cmd_prune.js +2 -2
- package/dist/cli-delegate/cmd_settings_migrate.js +2 -2
- package/dist/cli-delegate/cmd_settings_set.js +1 -1
- package/dist/cli-delegate/cmd_sync.js +2 -2
- package/dist/cli-delegate/cmd_uninstall.js +2 -2
- package/dist/cli-delegate/cmd_validate.js +2 -2
- package/dist/discovery/deprecation-report.md +1 -1
- package/dist/discovery/discovery-manifest.json +73 -30
- package/dist/discovery/discovery-manifest.json.sha256 +1 -1
- package/dist/discovery/discovery-manifest.summary.md +3 -3
- package/dist/discovery/dormancy-report.md +1 -1
- package/dist/discovery/orphan-report.md +1 -1
- package/dist/discovery/packs.json +7 -5
- package/dist/discovery/trust-report.md +3 -3
- package/dist/discovery/workspaces.json +6 -4
- package/dist/hooks/dispatch.js +2653 -1365
- package/dist/install/install.mjs +12 -4
- package/dist/mcp/registry-manifest.json +3 -3
- package/dist/mcp/server.json +2 -2
- package/dist/mcp/server.mjs +2679 -887
- package/dist/router.json +1 -1
- package/dist/scripts/condense.js +1 -1
- package/dist/scripts/condense.js.map +1 -1
- package/dist/server/schemas/settings.js +2 -2
- package/dist/server/schemas/settings.js.map +1 -1
- package/dist/ui/assets/{index-B0aMJnuz.js → index-C1pBRm1f.js} +2 -2
- package/dist/ui/assets/index-C1pBRm1f.js.map +1 -0
- package/dist/ui/index.html +1 -1
- package/docs/guidelines/design-fidelity-mechanics.md +21 -0
- package/docs/guidelines/design-handover-extraction.md +87 -0
- package/docs/guidelines/prompt-templates.md +24 -1
- package/package.json +1 -1
- package/src/agent-src/scripts/archive_completed_roadmaps.ts +33 -0
- package/src/agent-src/scripts/gate_execute.ts +395 -0
- package/src/agent-src/scripts/resume_probe.ts +422 -0
- package/src/agent-src/scripts/roadmap_gates.ts +138 -4
- package/src/agent-src/scripts/update_roadmap_progress.ts +70 -4
- package/src/config/ci-local-parity.yml +13 -0
- package/src/config/evaluator-budgets.json +9 -8
- package/src/config/gate-coverage.yml +140 -12
- package/src/config/gate-violation-baselines.json +9 -3
- package/src/config/hook-token-budget.json +8 -1
- package/src/config/rule-activation-census.json +41 -0
- package/src/config/rule-stub-ceilings.json +52 -0
- package/src/scripts/_dispatch.bash +4 -0
- package/src/scripts/_lib/bench_ab_complexity.ts +439 -0
- package/src/scripts/_lib/bench_ab_safety_tier.ts +142 -0
- package/src/scripts/_lib/bench_ab_scoring_v2.ts +28 -3
- package/src/scripts/_lib/bench_ab_search_adherence.ts +423 -0
- package/src/scripts/_lib/bench_ab_size_claim.ts +206 -0
- package/src/scripts/_lib/bench_ab_workspace.ts +111 -0
- package/src/scripts/_lib/evaluator_contract.ts +159 -0
- package/src/scripts/_lib/gh_transient.ts +80 -0
- package/src/scripts/_lib/release_material.ts +16 -5
- package/src/scripts/_lib/review_independence.ts +111 -0
- package/src/scripts/_lib/session_register.ts +14 -0
- package/src/scripts/_lib/skill_catalogue.ts +278 -5
- package/src/scripts/_lib/skill_catalogue_series.ts +411 -0
- package/src/scripts/_lib/tier_budget_routing.ts +31 -331
- package/src/scripts/_lib/turn_end_refusals.ts +482 -0
- package/src/scripts/ai_council/_py_json.ts +96 -0
- package/src/scripts/ai_council/cli_call_budget.ts +357 -0
- package/src/scripts/ai_council/clients.ts +101 -164
- package/src/scripts/ai_council/config.ts +25 -11
- package/src/scripts/ai_council/probe_store.ts +196 -0
- package/src/scripts/ai_council/qualification.ts +425 -0
- package/src/scripts/ai_council/qualification_wiring.ts +243 -0
- package/src/scripts/ai_council/quorum_wiring.ts +211 -0
- package/src/scripts/ai_team/team_dispatch.ts +40 -6
- package/src/scripts/bench_ab_v2_complexity.ts +252 -0
- package/src/scripts/bench_ab_v2_run.ts +3 -42
- package/src/scripts/bench_ab_v2_safety.ts +296 -0
- package/src/scripts/bench_ab_v2_search.ts +343 -0
- package/src/scripts/bench_ab_v2_stats.ts +191 -0
- package/src/scripts/build_archive_index.ts +383 -0
- package/src/scripts/capture_skill_catalogue.ts +369 -4
- package/src/scripts/check_evaluator_schema.ts +249 -0
- package/src/scripts/check_pr_ci_current.ts +409 -0
- package/src/scripts/check_review_schema.ts +195 -0
- package/src/scripts/check_rule_activation_census.ts +451 -0
- package/src/scripts/check_rule_stub_ceiling.ts +1128 -0
- package/src/scripts/check_source_size_budget.ts +428 -0
- package/src/scripts/check_suppression_hygiene.ts +5 -1
- package/src/scripts/condense.ts +1 -1
- package/src/scripts/cost/backfill_rates.mjs +298 -0
- package/src/scripts/cost/budget.mjs +12 -32
- package/src/scripts/cost_summary.ts +25 -0
- package/src/scripts/council_cli.ts +141 -191
- package/src/scripts/dispatch_r2_reviewer.ts +8 -1
- package/src/scripts/external_sources_denylist.json +1 -0
- package/src/scripts/hook_manifest.yaml +99 -6
- package/src/scripts/hooks/concern_registry.ts +6 -0
- package/src/scripts/hooks/dispatch_hook.ts +43 -1
- package/src/scripts/hooks/interruption_ledger_hook.ts +256 -0
- package/src/scripts/hooks/ship_diff_volume_hook.ts +180 -0
- package/src/scripts/hooks/skill_route_hook.ts +226 -0
- package/src/scripts/hooks/turn_end_gate_hook.ts +119 -26
- package/src/scripts/hooks/ui_route_nudge_hook.ts +6 -3
- package/src/scripts/hooks_doctor.ts +112 -0
- package/src/scripts/interruption_report.ts +443 -0
- package/src/scripts/lint_design_antipattern_parity.ts +40 -6
- package/src/scripts/lint_evidence_artifacts.ts +359 -0
- package/src/scripts/lint_harvest_provenance.ts +35 -1
- package/src/scripts/lint_roadmap_blockers.ts +65 -0
- package/src/scripts/lint_roadmap_complexity.ts +7 -1
- package/src/scripts/lint_rule_skill_pack_reach.ts +43 -0
- package/src/scripts/lint_skill_router_head.ts +21 -0
- package/src/scripts/mcp_server/consumer_tool_catalog.json +26 -3
- package/src/scripts/mcp_server/tool_catalog_source.ts +6 -24
- package/src/scripts/mcp_server/tools.ts +87 -0
- package/src/scripts/release.ts +43 -58
- package/src/scripts/routing_doctor.ts +50 -13
- package/src/scripts/rule_activation_census.ts +292 -0
- package/src/scripts/schemas/command.schema.json +1 -1
- package/src/scripts/schemas/evaluator-output.schema.json +60 -0
- package/src/scripts/schemas/review-findings.schema.json +31 -0
- package/src/scripts/schemas/skill.schema.json +18 -1
- package/src/scripts/self_review_gate.ts +6 -0
- package/src/scripts/session_register_hook.ts +30 -1
- package/src/scripts/sync_pr_branch.ts +271 -0
- package/src/scripts/test_council_qualification.ts +213 -0
- package/src/scripts/trigger_coverage.ts +224 -3
- package/src/server/schemas/settings.ts +2 -2
- package/dist/ui/assets/index-B0aMJnuz.js.map +0 -1
- package/src/config/budget-routing.json +0 -7
|
@@ -180,11 +180,11 @@ class RoadmapStats {
|
|
|
180
180
|
}
|
|
181
181
|
|
|
182
182
|
get open_blockers(): Blocker[] {
|
|
183
|
-
return this.blockers.filter((b) => b
|
|
183
|
+
return this.blockers.filter((b) => !blocker_is_resolved(b));
|
|
184
184
|
}
|
|
185
185
|
|
|
186
186
|
get resolved_blockers(): Blocker[] {
|
|
187
|
-
return this.blockers.filter((b) => b
|
|
187
|
+
return this.blockers.filter((b) => blocker_is_resolved(b));
|
|
188
188
|
}
|
|
189
189
|
|
|
190
190
|
get done(): number {
|
|
@@ -365,6 +365,49 @@ interface Blocker {
|
|
|
365
365
|
* costs one enum entry and turns a silent swallow into a rendered field.
|
|
366
366
|
*/
|
|
367
367
|
question: string;
|
|
368
|
+
/**
|
|
369
|
+
* Gate taxonomy: `0` auto-run · `1` budget-preauthorized · `2`
|
|
370
|
+
* consent-once · `3` human-only. Empty string when the entry declares
|
|
371
|
+
* none, and an absent class READS AS 3 at every consumer.
|
|
372
|
+
*
|
|
373
|
+
* The default is the safe end on purpose. An agent that inferred the class
|
|
374
|
+
* at runtime could promote a Hard-Floor gate into something it executes;
|
|
375
|
+
* authoring it into the file means a misclassification is a reviewed edit,
|
|
376
|
+
* and `lint_roadmap_blockers` refuses a runnable class that names no
|
|
377
|
+
* command.
|
|
378
|
+
*/
|
|
379
|
+
blockerClass: string;
|
|
380
|
+
/** The command a class-0/1 entry is resolved by running. */
|
|
381
|
+
run: string;
|
|
382
|
+
/** Spend estimate for class 1 — recorded, never enforced by the lint. */
|
|
383
|
+
budget: string;
|
|
384
|
+
}
|
|
385
|
+
|
|
386
|
+
/**
|
|
387
|
+
* Is this blocker settled?
|
|
388
|
+
*
|
|
389
|
+
* A PREFIX test, not equality, and the difference was measured: authors write
|
|
390
|
+
* the resolution into the status line — `**Status:** RESOLVED 2026-08-17 —
|
|
391
|
+
* **option (b)**, narrowed to two rules` — and an equality check matches
|
|
392
|
+
* neither `open` nor `resolved`, so the entry falls through to open. Found
|
|
393
|
+
* 2026-08-17 in `road-to-mixed-trigger-activation-cost`: a decision taken two
|
|
394
|
+
* days earlier was still being rendered by `agent-config gates` as a live one
|
|
395
|
+
* the reader owed an answer to, and it inflated every blocker count in the
|
|
396
|
+
* dashboard by one.
|
|
397
|
+
*
|
|
398
|
+
* `lint_roadmap_blockers` already read the same field as a prefix
|
|
399
|
+
* (`/^-\s*\*\*Status:\*\*\s*resolved/i`), so the two surfaces disagreed about
|
|
400
|
+
* what "resolved" means — the lint exempted the entry from its decidability
|
|
401
|
+
* ratchet while the dashboard counted it open.
|
|
402
|
+
*/
|
|
403
|
+
function blocker_is_resolved(b: Blocker): boolean {
|
|
404
|
+
return /^resolved\b/.test(_strip(b.status).toLowerCase());
|
|
405
|
+
}
|
|
406
|
+
|
|
407
|
+
/** Class of a blocker, with the absent-field default applied. */
|
|
408
|
+
function blocker_class(b: Blocker): '0' | '1' | '2' | '3' {
|
|
409
|
+
const c = _strip(b.blockerClass).split(/[\s—–-]/)[0] as string;
|
|
410
|
+
return c === '0' || c === '1' || c === '2' ? c : '3';
|
|
368
411
|
}
|
|
369
412
|
|
|
370
413
|
// Strip fenced code blocks before blocker detection — a roadmap that shows the
|
|
@@ -380,8 +423,23 @@ function _stripFencedCode(text: string): string {
|
|
|
380
423
|
return text.replace(FENCED_CODE_RE, (m) => '\n'.repeat(_splitlines(m).length));
|
|
381
424
|
}
|
|
382
425
|
|
|
383
|
-
|
|
384
|
-
|
|
426
|
+
/**
|
|
427
|
+
* Terminator for a blocker field's continuation lines — ANY `- **Label:**`
|
|
428
|
+
* bullet, not a closed list of known labels.
|
|
429
|
+
*
|
|
430
|
+
* It used to enumerate eight names, which silently diverged from the contract
|
|
431
|
+
* the two call sites below already document ("up to the next `- **Field:**`
|
|
432
|
+
* marker"): a bullet whose label was not on the list did not terminate, so the
|
|
433
|
+
* PREVIOUS field kept absorbing it and the dashboard rendered the two as one
|
|
434
|
+
* run-on sentence. Measured when `- **Options:**` and `- **Side finding:**`
|
|
435
|
+
* first appeared in a blocker — a reader saw the side finding as part of
|
|
436
|
+
* `Resolved when`, i.e. the generator changed what a field meant.
|
|
437
|
+
*
|
|
438
|
+
* A closed list is the wrong shape for a terminator: every future field name
|
|
439
|
+
* would have to be added here to avoid corrupting the field above it, and the
|
|
440
|
+
* failure is silent in generated output nobody edits by hand.
|
|
441
|
+
*/
|
|
442
|
+
const BLOCKER_FIELD_RE = /^-[ \t]*\*\*[^*]+:\*\*/;
|
|
385
443
|
|
|
386
444
|
/** Strip an inline `<!-- comment -->` and trim. */
|
|
387
445
|
function _stripComment(s: string): string {
|
|
@@ -493,6 +551,9 @@ function parse_blockers(raw_text: string): Blocker[] {
|
|
|
493
551
|
recommendation: _blockerField(body, 'Recommendation') ?? '',
|
|
494
552
|
ifNothing: _blockerField(body, 'If you do nothing') ?? '',
|
|
495
553
|
question: _blockerField(body, 'Question') ?? '',
|
|
554
|
+
blockerClass: _blockerField(body, 'Class') ?? '',
|
|
555
|
+
run: _blockerField(body, 'Run') ?? '',
|
|
556
|
+
budget: _blockerField(body, 'Budget') ?? '',
|
|
496
557
|
});
|
|
497
558
|
}
|
|
498
559
|
}
|
|
@@ -527,6 +588,9 @@ function parse_blockers(raw_text: string): Blocker[] {
|
|
|
527
588
|
recommendation: '',
|
|
528
589
|
ifNothing: '',
|
|
529
590
|
question: '',
|
|
591
|
+
blockerClass: '',
|
|
592
|
+
run: '',
|
|
593
|
+
budget: '',
|
|
530
594
|
});
|
|
531
595
|
}
|
|
532
596
|
return blockers;
|
|
@@ -1172,6 +1236,8 @@ export {
|
|
|
1172
1236
|
is_roadmap_candidate,
|
|
1173
1237
|
count_checkboxes,
|
|
1174
1238
|
parse_blockers,
|
|
1239
|
+
blocker_class,
|
|
1240
|
+
blocker_is_resolved,
|
|
1175
1241
|
blocker_needs_user,
|
|
1176
1242
|
parse_roadmap,
|
|
1177
1243
|
bar,
|
|
@@ -17,6 +17,67 @@ packs:
|
|
|
17
17
|
|
|
18
18
|
# ai-council
|
|
19
19
|
|
|
20
|
+
## Council-first — try the council BEFORE the user, on these classes
|
|
21
|
+
|
|
22
|
+
```
|
|
23
|
+
A DESIGN DECISION THE AGENT CANNOT SETTLE FROM THE TREE GOES TO THE COUNCIL
|
|
24
|
+
FIRST, NOT TO THE USER FIRST. THE USER IS THE ESCALATION, NOT THE DEFAULT.
|
|
25
|
+
NEVER PRESENT A COUNCIL VERDICT THAT WAS NOT REACHED. AN INCONCLUSIVE RUN
|
|
26
|
+
IS REPORTED AS INCONCLUSIVE, AND WHATEVER YOU DECIDE INSTEAD IS LABELLED
|
|
27
|
+
OWN ANALYSIS.
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
Interrupting the user is the expensive path: it costs a round trip and their
|
|
31
|
+
attention, and on a question two models could have settled it buys nothing they
|
|
32
|
+
wanted to be asked. So the ordering is **tree → council → user**, and the
|
|
33
|
+
council leg is the one that gets skipped.
|
|
34
|
+
|
|
35
|
+
**Classes that go to the council first** — each is a judgement with a real
|
|
36
|
+
trade-off and no answer in the tree:
|
|
37
|
+
|
|
38
|
+
| Class | Example |
|
|
39
|
+
|---|---|
|
|
40
|
+
| Competing implementations of one feature | two sessions shipped the same phase; discard, overwrite, or merge |
|
|
41
|
+
| Which of N designs, all defensible | four declared types with a derived fifth, or five declared |
|
|
42
|
+
| Reopening a recorded decision | an ADR or honest-null blocks a change that now looks net-positive |
|
|
43
|
+
| Scope re-cut inside an authorized task | this phase splits into two, or absorbs the next |
|
|
44
|
+
| Is the evidence sufficient to conclude | n=1 measurement, a proxy metric, an unreplicated finding |
|
|
45
|
+
|
|
46
|
+
**Classes that still go straight to the user, and the council never substitutes
|
|
47
|
+
for them:** anything behind the Hard Floor
|
|
48
|
+
([`non-destructive-by-default`](../../rules/non-destructive-by-default.md)) ·
|
|
49
|
+
spend · a scope EXPANSION beyond what was authorized · a decision whose inputs
|
|
50
|
+
only the user holds (a preference, a deadline, a business constraint) · the
|
|
51
|
+
`high_impact` and `user_required` classes, which
|
|
52
|
+
[`ask-when-uncertain`](../../rules/ask-when-uncertain.md) routes to the user by
|
|
53
|
+
Iron Law. A council cannot consent on the user's behalf, and asking it to is not
|
|
54
|
+
autonomy, it is laundering.
|
|
55
|
+
|
|
56
|
+
### When the council cannot answer
|
|
57
|
+
|
|
58
|
+
Measured 2026-08-17: a run returned **0 of 2 seats**, both `cli_quota_exhausted`,
|
|
59
|
+
with the counters at 72/50 and 99/50. The CLI reported it correctly as
|
|
60
|
+
`INCONCLUSIVE — DEGRADED` and did not print a quorum it had not reached. Three
|
|
61
|
+
things follow, and the first is the one under pressure:
|
|
62
|
+
|
|
63
|
+
1. **Never dress your own verdict as the council's.** A fan-out, a solo read, or
|
|
64
|
+
your own judgement is a legitimate substitute **only when named as such** —
|
|
65
|
+
the boundary [`council-availability`](../../rules/council-availability.md)
|
|
66
|
+
states.
|
|
67
|
+
2. **Do not bounce to the user just because the council failed.** An
|
|
68
|
+
unreachable council does not upgrade a decidable question into a
|
|
69
|
+
user-required one. Decide it, label it own analysis, state the reason the
|
|
70
|
+
council was unavailable, and name what would change the answer.
|
|
71
|
+
3. **Escalate only if it is genuinely undecidable without them** — the two
|
|
72
|
+
options are close AND the cost of being wrong is high AND nothing in the tree
|
|
73
|
+
separates them. Then say the council was attempted and why it failed, so the
|
|
74
|
+
user is not asked to arbitrate something they will assume was already tried.
|
|
75
|
+
|
|
76
|
+
`agent-config council:status` is free and answers availability; `council_cli
|
|
77
|
+
estimate <file>` is free and answers parse-and-cost. Run both before concluding
|
|
78
|
+
the council is not an option — and note the question goes in a **file**, since
|
|
79
|
+
prose passed as an argument overflows the argument limit.
|
|
80
|
+
|
|
20
81
|
## When to use
|
|
21
82
|
|
|
22
83
|
* The host agent has drafted a roadmap, plan, or design and wants an
|
|
@@ -306,6 +306,10 @@ handoff shape emitted alongside `design-system.json`
|
|
|
306
306
|
— when the artifact is a mock — an explicit "prototype, not production code"
|
|
307
307
|
warning so a consumer never ships the reference verbatim.
|
|
308
308
|
|
|
309
|
+
**What to capture FROM** a running project: sources first, SSR fetch second,
|
|
310
|
+
rendered snapshot only with consumer tooling; record the git SHA. Rationale:
|
|
311
|
+
[snapshot preference order](references/snapshot-preference-order.md).
|
|
312
|
+
|
|
309
313
|
## Gotcha
|
|
310
314
|
|
|
311
315
|
- **Over-writing instead of diff-updating** — the most common failure.
|
|
@@ -0,0 +1,77 @@
|
|
|
1
|
+
# Snapshot preference order — what to capture a design system FROM
|
|
2
|
+
|
|
3
|
+
When a design system has to be captured out of a running project — for a
|
|
4
|
+
handoff, an external design pass, or a reference bundle — there is more than one
|
|
5
|
+
thing you could point at, and they are not equally good. This is the preference
|
|
6
|
+
order, highest fidelity per unit of effort first.
|
|
7
|
+
|
|
8
|
+
**Take the first level the project supports. Do not skip down the list for
|
|
9
|
+
convenience, and do not skip UP it for thoroughness** — Level C is additional
|
|
10
|
+
evidence, never a replacement for Level A.
|
|
11
|
+
|
|
12
|
+
## Level A — source-level (always available, the default)
|
|
13
|
+
|
|
14
|
+
The involved component sources, plus the token layer (DTCG / CSS custom
|
|
15
|
+
properties), plus the page composition that assembles them.
|
|
16
|
+
|
|
17
|
+
**Why this is the default and not the fallback**, which is the part that reads
|
|
18
|
+
backwards until you have seen the alternative: in a component project, a
|
|
19
|
+
rendered DOM dump is *worse* input than the sources it came from. Hydrated
|
|
20
|
+
markup is utility-class soup with the component boundaries erased — the very
|
|
21
|
+
structure a design system is made of is the structure rendering destroys. A
|
|
22
|
+
reader (human or tool) reworks clean sources far better than expanded markup.
|
|
23
|
+
|
|
24
|
+
Level A needs no dev server, no browser, and no network. It is available in a
|
|
25
|
+
bare checkout, which is why it is also the only level that always works.
|
|
26
|
+
|
|
27
|
+
## Level B — SSR fetch (no browser)
|
|
28
|
+
|
|
29
|
+
When the stack renders server-side and a dev server is running, an ordinary
|
|
30
|
+
HTTP fetch of the rendered page.
|
|
31
|
+
|
|
32
|
+
Use it to answer what Level A cannot: what the composition actually resolves to
|
|
33
|
+
for a given route — conditional branches taken, data-dependent states, the real
|
|
34
|
+
class strings after server-side composition. Zero browser dependency, so it
|
|
35
|
+
stays cheap.
|
|
36
|
+
|
|
37
|
+
Level B **supplements** Level A. Capturing B alone leaves you with the soup
|
|
38
|
+
problem above and no component boundaries.
|
|
39
|
+
|
|
40
|
+
## Level C — rendered snapshot (consumer tooling only)
|
|
41
|
+
|
|
42
|
+
If the project already carries a browser-automation tool, or a DevTools bridge
|
|
43
|
+
is available in the session, additionally capture a screenshot plus a
|
|
44
|
+
computed-style extract as visual ground truth.
|
|
45
|
+
|
|
46
|
+
**Capability-detect, never hard-depend.** Level C is available when the consumer
|
|
47
|
+
already has the tooling; it is never a reason to install a browser stack, add a
|
|
48
|
+
dependency, or fail a capture. A project without it captures at A (and B where
|
|
49
|
+
applicable) and says so.
|
|
50
|
+
|
|
51
|
+
Of the two artefacts, **the screenshot is the valuable part** — it is the only
|
|
52
|
+
one that records what a person actually sees. The DOM dump is supplementary and
|
|
53
|
+
carries the erasure problem Level A exists to avoid.
|
|
54
|
+
|
|
55
|
+
## Every snapshot records the source revision
|
|
56
|
+
|
|
57
|
+
Whatever level was captured, record the git SHA the capture was taken at.
|
|
58
|
+
|
|
59
|
+
This is the anchor that makes a return leg possible: without it, reworked output
|
|
60
|
+
cannot be diffed against what was sent, and a capture from an unknown state is
|
|
61
|
+
evidence about nothing in particular. It costs one line and it is the difference
|
|
62
|
+
between a snapshot and a screenshot.
|
|
63
|
+
|
|
64
|
+
## State the level in the output
|
|
65
|
+
|
|
66
|
+
A capture says which level it reached and why it stopped there — "Level A + B;
|
|
67
|
+
no browser tooling detected, so no visual ground truth". A consumer reading the
|
|
68
|
+
bundle can then tell *absent because unavailable* from *absent because
|
|
69
|
+
overlooked*, which is the same distinction the extraction floor draws for
|
|
70
|
+
not-extractable values.
|
|
71
|
+
|
|
72
|
+
## See also
|
|
73
|
+
|
|
74
|
+
- [`design-system-json.md`](design-system-json.md) — the import contract the
|
|
75
|
+
captured artefact is emitted against.
|
|
76
|
+
- [`../SKILL.md`](../SKILL.md) § Design-system extraction floor — what a
|
|
77
|
+
complete extraction carries once the source has been chosen.
|
|
@@ -0,0 +1,116 @@
|
|
|
1
|
+
---
|
|
2
|
+
model_tier: inherit
|
|
3
|
+
name: experiment-loop
|
|
4
|
+
description: "Use to drive a scalar metric down or up across bounded iterations — keep on strict improvement, revert otherwise, state on disk. Triggers 'minimize X', 'optimize until it stops improving'."
|
|
5
|
+
domain: process
|
|
6
|
+
workspaces:
|
|
7
|
+
- agent-config-maintainer
|
|
8
|
+
packs:
|
|
9
|
+
- meta
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
# experiment-loop
|
|
13
|
+
|
|
14
|
+
> A bounded **change → commit → evaluate → keep-or-revert** cycle against a
|
|
15
|
+
> **scalar metric**, where the decision input is an
|
|
16
|
+
> [evaluator-output](../../../docs/contracts/evaluator-output.md) verdict and the
|
|
17
|
+
> loop's entire state lives in an append-only register on disk.
|
|
18
|
+
|
|
19
|
+
This file **routes**. The protocol, the register format, and the pivot ladder are
|
|
20
|
+
reference files loaded on demand — a loop doctrine read in full on every session
|
|
21
|
+
that merely mentions optimization is a payload nobody asked for.
|
|
22
|
+
|
|
23
|
+
| Need | Load |
|
|
24
|
+
|---|---|
|
|
25
|
+
| The iteration protocol and its two exit conditions | [`references/protocol.md`](references/protocol.md) |
|
|
26
|
+
| The register's row shape and write ordering | [`references/register-format.md`](references/register-format.md) |
|
|
27
|
+
| What to do when the metric stops moving | [`references/pivot-ladder.md`](references/pivot-ladder.md) |
|
|
28
|
+
|
|
29
|
+
## When to use
|
|
30
|
+
|
|
31
|
+
* The goal is expressible as **"minimize X"** or **"maximize X"** against a
|
|
32
|
+
mechanical verifier — bundle size, violation count, query count, runtime.
|
|
33
|
+
* A verifier already emits, or can be wrapped to emit, the evaluator-output
|
|
34
|
+
contract. Wrapping is usually free: three of this repo's verifiers were wrapped
|
|
35
|
+
with zero changes to any of them.
|
|
36
|
+
|
|
37
|
+
Do **NOT** use when:
|
|
38
|
+
|
|
39
|
+
* The target is a **known end state**, not a number — "make the tests pass" has a
|
|
40
|
+
destination, so use [`verify-repair-loop`](../verify-repair-loop/SKILL.md),
|
|
41
|
+
which converges toward it. This skill has no destination; it has a direction.
|
|
42
|
+
* The verdict is **subjective craft** — no scalar, nothing to optimize.
|
|
43
|
+
* The metric is a **proxy nobody agreed on**. Optimizing an unowned proxy is how
|
|
44
|
+
a loop makes a system worse while its number improves.
|
|
45
|
+
|
|
46
|
+
## The Iron Law
|
|
47
|
+
|
|
48
|
+
```
|
|
49
|
+
THE METRIC IS NEVER SOVEREIGN. `pass` IS.
|
|
50
|
+
KEEP ONLY ON `pass && score > baseline`. A CHANGE THAT IMPROVES THE NUMBER
|
|
51
|
+
AND BREAKS BEHAVIOUR IS REVERTED — MEASURED, NOT ASSUMED.
|
|
52
|
+
COMMIT BEFORE YOU EVALUATE, SO EVERY REVERT IS A GIT OPERATION.
|
|
53
|
+
THE REGISTER IS WRITTEN BEFORE THE ACTION IT RECORDS, NEVER AFTER.
|
|
54
|
+
STATE LIVES ON DISK AND IS RE-READ EACH CYCLE — NEVER IN THE CONVERSATION.
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
Each line is a measured failure, not a preference. Spike s01 ran a change that
|
|
58
|
+
improved its metric by 67 % and broke the behaviour it measured; only `pass`
|
|
59
|
+
reverted it. The same run crashed between its commit and its register append,
|
|
60
|
+
twice, leaving the branch one iteration ahead of its own record.
|
|
61
|
+
|
|
62
|
+
## Procedure
|
|
63
|
+
|
|
64
|
+
1. **Inspect the verifier before trusting it.** Run the evaluator once on the
|
|
65
|
+
unchanged tree and check its verdict: `pass` must be true and `metric_state`
|
|
66
|
+
must be `present`. A baseline taken from an evaluator that is already red, or
|
|
67
|
+
whose metric is unreadable, is not a baseline — every later comparison
|
|
68
|
+
inherits the defect silently.
|
|
69
|
+
2. **Fix the metric and the verifier** before the first change. Name the
|
|
70
|
+
evaluator, its direction, and the baseline score.
|
|
71
|
+
3. **Declare the bound** — a maximum iteration count, written into the register's
|
|
72
|
+
first row. An unbounded loop is not this skill.
|
|
73
|
+
4. **Iterate** per [`references/protocol.md`](references/protocol.md): one focused
|
|
74
|
+
change, commit, evaluate, keep or revert, append the outcome.
|
|
75
|
+
5. **Exit** on either condition (both machine-checkable, never prose): the bound
|
|
76
|
+
is reached, or the explicit exit signal fires.
|
|
77
|
+
6. **Report** from the register, not from memory. If the register cannot
|
|
78
|
+
reconstruct the run without the transcript, the run is not reportable.
|
|
79
|
+
|
|
80
|
+
## Validation
|
|
81
|
+
|
|
82
|
+
* The register reconstructs the run with the transcript discarded — every kept
|
|
83
|
+
change traceable to a row, every row to a commit.
|
|
84
|
+
* Both branches ran at least once. **A loop whose revert branch never executed
|
|
85
|
+
is a one-branch loop with a comment**; spike s01 found its revert broken on
|
|
86
|
+
first use after four green iterations.
|
|
87
|
+
* The final tree passes the evaluator with `pass: true`.
|
|
88
|
+
|
|
89
|
+
## Output format
|
|
90
|
+
|
|
91
|
+
1. **The register path**, and the iteration count actually run against the bound.
|
|
92
|
+
2. **A metric line**: baseline → final, and the number of keeps versus reverts.
|
|
93
|
+
3. **The exit condition that fired**, named — bound reached, or exit signal.
|
|
94
|
+
4. **Every reverted iteration listed** with its reason. A revert is a result the
|
|
95
|
+
next run needs, not noise to summarise away.
|
|
96
|
+
|
|
97
|
+
## Gotcha
|
|
98
|
+
|
|
99
|
+
* **A metric that improves while the system degrades.** The reason `pass` is
|
|
100
|
+
sovereign. Never widen a verifier to make an iteration keep.
|
|
101
|
+
* **A revert path that has never run.** Exercise it deliberately once.
|
|
102
|
+
* **A register appended after the mutation.** A crash in between leaves the
|
|
103
|
+
branch ahead of the record; write the row first, then update its outcome.
|
|
104
|
+
* **Loop state in the conversation.** It decays silently and the loop keeps
|
|
105
|
+
comparing against a baseline it no longer holds.
|
|
106
|
+
* **Iterating past the bound because the number is still moving.** The bound is
|
|
107
|
+
the contract; extending it is a new run with a new baseline.
|
|
108
|
+
|
|
109
|
+
## Do NOT
|
|
110
|
+
|
|
111
|
+
* Do NOT keep a change on `score` alone.
|
|
112
|
+
* Do NOT edit the verifier to move the metric.
|
|
113
|
+
* Do NOT batch several changes into one iteration — a keep you cannot attribute
|
|
114
|
+
is a keep you cannot revert.
|
|
115
|
+
* Do NOT delete a register row. Rows are append-only, including the embarrassing
|
|
116
|
+
ones.
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
# Behavior-schema contract for `experiment-loop`.
|
|
2
|
+
#
|
|
3
|
+
# Locks the sections a refactor could quietly drop while the skill still
|
|
4
|
+
# reads plausibly. Each one is a measured failure from spike s01
|
|
5
|
+
# (agents/evidence/eval-findings/metric-loop-s01.md), not a style
|
|
6
|
+
# preference:
|
|
7
|
+
#
|
|
8
|
+
# The Iron Law — `pass` sovereign over `score`; a change that improved
|
|
9
|
+
# the metric 67% and broke behaviour was reverted only
|
|
10
|
+
# because of it.
|
|
11
|
+
# Validation — the both-branches-ran check; s01's revert path was
|
|
12
|
+
# broken and four green iterations never noticed.
|
|
13
|
+
# Output format — the register-derived report; a report from memory is
|
|
14
|
+
# the in-conversation state the register exists to replace.
|
|
15
|
+
# Do NOT — the four keeps that must not happen.
|
|
16
|
+
#
|
|
17
|
+
# Enforced by `skill_linter` (`lint_output_schema`, matches `^## <header>$`) —
|
|
18
|
+
# a missing section is a hard error.
|
|
19
|
+
|
|
20
|
+
version: 1
|
|
21
|
+
|
|
22
|
+
required_headers:
|
|
23
|
+
- "The Iron Law"
|
|
24
|
+
- "Procedure"
|
|
25
|
+
- "Validation"
|
|
26
|
+
- "Output format"
|
|
27
|
+
- "Do NOT"
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
{
|
|
2
|
+
"skill": "experiment-loop",
|
|
3
|
+
"last_eval": "2026-08-17",
|
|
4
|
+
"description": "5 should-trigger + 5 should-not-trigger. Should-trigger covers the bounded keep-or-revert loop against a SCALAR metric (direction, no destination). Should-not-trigger covers the near-miss neighbours the roadmap's gap-table names: iterate-to-a-known-end-state (verify-repair-loop), subjective craft with no scalar, a one-shot measurement, a one-shot review, and asking what a metric currently is.",
|
|
5
|
+
"queries": [
|
|
6
|
+
{"q": "minimize the bundle size, keep changes that help and revert the rest", "trigger": true},
|
|
7
|
+
{"q": "optimize this until it stops improving, bounded to ten tries", "trigger": true},
|
|
8
|
+
{"q": "drive the query count down across iterations and record what you kept", "trigger": true},
|
|
9
|
+
{"q": "run a keep-or-revert loop against the lint violation count", "trigger": true},
|
|
10
|
+
{"q": "reduce this metric as far as it goes without breaking the tests", "trigger": true},
|
|
11
|
+
{"q": "keep fixing until the tests pass", "trigger": false, "note": "a known end state, not a direction → verify-repair-loop"},
|
|
12
|
+
{"q": "iterate this change to green, run the suite each time", "trigger": false, "note": "converges toward a destination → verify-repair-loop"},
|
|
13
|
+
{"q": "is this diff clean — naming, structure, conventions?", "trigger": false, "note": "subjective craft, no scalar to optimize"},
|
|
14
|
+
{"q": "how big is the bundle right now?", "trigger": false, "note": "a single measurement, no loop"},
|
|
15
|
+
{"q": "review my changes before I open the PR", "trigger": false, "note": "one-shot review → /review-changes"}
|
|
16
|
+
]
|
|
17
|
+
}
|
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
# experiment-loop — the pivot ladder
|
|
2
|
+
|
|
3
|
+
Loaded on demand by [`experiment-loop`](../SKILL.md), when the exit signal
|
|
4
|
+
`consecutive_reverts >= N` fires. The metric has stopped moving. The question is
|
|
5
|
+
whether that is a fact about the system or about the hypotheses being tried.
|
|
6
|
+
|
|
7
|
+
Climb one rung at a time. Each rung is a **new run** with a fresh register and a
|
|
8
|
+
re-declared baseline — never a quiet continuation of the old one, because a run
|
|
9
|
+
whose hypothesis class changed mid-flight cannot be read afterwards.
|
|
10
|
+
|
|
11
|
+
## Rung 1 — a different hypothesis class, same metric
|
|
12
|
+
|
|
13
|
+
The reverts share a shape: every attempt tried to shave the same structure. Pick
|
|
14
|
+
a class the run has not touched — a different module, a different layer, a
|
|
15
|
+
different kind of change — and rerun with the current score as the baseline.
|
|
16
|
+
|
|
17
|
+
Stop climbing here if the first iteration of the new class keeps.
|
|
18
|
+
|
|
19
|
+
## Rung 2 — check the metric measures what you meant
|
|
20
|
+
|
|
21
|
+
Before concluding the system is optimal, confirm the number is the number. The
|
|
22
|
+
measured failure mode: a diff-volume metric that looked like shipping volume was
|
|
23
|
+
counting the repository's own review bookkeeping — a committed copy of the diff
|
|
24
|
+
being measured — so the ranking it produced was of the wrong thing entirely.
|
|
25
|
+
Excluding the bookkeeping moved the p90 by more than half and reordered the top
|
|
26
|
+
five.
|
|
27
|
+
|
|
28
|
+
Ask: does the metric include artefacts the change did not author? Generated
|
|
29
|
+
files, snapshots, vendored trees, its own output? Correcting that is a
|
|
30
|
+
**measurement fix** and legitimate.
|
|
31
|
+
|
|
32
|
+
## Rung 3 — decide whether tuning is the honest move
|
|
33
|
+
|
|
34
|
+
Rung 2's correction has a twin that is not legitimate: adjusting the metric until
|
|
35
|
+
the answer is pleasant. The discriminator is stated once, plainly:
|
|
36
|
+
|
|
37
|
+
> **Removing an input that was never part of the quantity is a correction.
|
|
38
|
+
> Moving a threshold or growing an exclusion set until the result improves is
|
|
39
|
+
> tuning.** A correction can be justified without reference to its effect on the
|
|
40
|
+
> outcome. Tuning cannot.
|
|
41
|
+
|
|
42
|
+
If the only available move needs the outcome to justify it, stop. The run's
|
|
43
|
+
answer is that the metric does not move under changes of this kind, and that is
|
|
44
|
+
a real result worth recording.
|
|
45
|
+
|
|
46
|
+
## Rung 4 — publish the null
|
|
47
|
+
|
|
48
|
+
No rung produced a keep. Write the run up: the metric, the bound, the hypothesis
|
|
49
|
+
classes tried, the register path, and the conclusion that the metric did not
|
|
50
|
+
move. A published null is why the next person does not re-run this.
|
|
51
|
+
|
|
52
|
+
**Do not climb back down.** Re-running rung 1 with slightly different wording is
|
|
53
|
+
the shape that turns a bounded loop into an unbounded one, and it burns budget
|
|
54
|
+
producing rows that all say the same thing.
|
|
@@ -0,0 +1,61 @@
|
|
|
1
|
+
# experiment-loop — the iteration protocol
|
|
2
|
+
|
|
3
|
+
Loaded on demand by [`experiment-loop`](../SKILL.md). One iteration, in order.
|
|
4
|
+
|
|
5
|
+
## The cycle
|
|
6
|
+
|
|
7
|
+
1. **Read the register from disk.** Not from the conversation, not from a
|
|
8
|
+
variable held since the last cycle. The baseline for this iteration is the
|
|
9
|
+
`score` of the most recent `keep` row, or the declared baseline if there is
|
|
10
|
+
none.
|
|
11
|
+
2. **Make ONE focused change.** One hypothesis per iteration. A batched
|
|
12
|
+
iteration produces a keep you cannot attribute and therefore cannot revert.
|
|
13
|
+
3. **Append the register row** with `decision: "pending"` — before the commit,
|
|
14
|
+
before the evaluation. See [`register-format.md`](register-format.md) for why
|
|
15
|
+
the ordering is this way round.
|
|
16
|
+
4. **Commit.** The revert is then a git operation rather than a manual undo, and
|
|
17
|
+
the branch carries one commit per iteration whether it survives or not.
|
|
18
|
+
5. **Evaluate** — run the evaluator, read its
|
|
19
|
+
[evaluator-output](../../../../../docs/contracts/evaluator-output.md) verdict.
|
|
20
|
+
6. **Decide**, and update the row:
|
|
21
|
+
|
|
22
|
+
| Verdict | Decision |
|
|
23
|
+
|---|---|
|
|
24
|
+
| `pass: true` and `score > baseline` | **keep** |
|
|
25
|
+
| `pass: true` and `score <= baseline` | **revert** — no strict improvement |
|
|
26
|
+
| `pass: false` | **revert** — the tree is red |
|
|
27
|
+
| non-zero exit · absent JSON · timeout | **revert** — the experiment failed, the measurement did not happen |
|
|
28
|
+
| `metric_state: "unreadable"` | **revert** — an unread metric is not a zero |
|
|
29
|
+
|
|
30
|
+
7. **Revert if the decision says so**, then loop.
|
|
31
|
+
|
|
32
|
+
## The two exit conditions
|
|
33
|
+
|
|
34
|
+
Both are **machine-checkable predicates**. Neither is prose, and that is
|
|
35
|
+
deliberate: a stop condition written loosely rots, and the rot is invisible
|
|
36
|
+
because the loop keeps running.
|
|
37
|
+
|
|
38
|
+
**Condition A — the bound.** `iterations_run >= max_iterations`, where
|
|
39
|
+
`max_iterations` was written into the register's first row before the first
|
|
40
|
+
change. Reaching it is a normal exit, not a failure.
|
|
41
|
+
|
|
42
|
+
**Condition B — the exit signal.** A predicate declared at the start, evaluated
|
|
43
|
+
against the register each cycle. Two usable shapes:
|
|
44
|
+
|
|
45
|
+
- `consecutive_reverts >= N` — the metric has stopped moving under this
|
|
46
|
+
hypothesis class. Route to [`pivot-ladder.md`](pivot-ladder.md).
|
|
47
|
+
- `score >= target` — the run had a destination after all, and reached it.
|
|
48
|
+
|
|
49
|
+
**Why two and not one.** A bound alone burns the whole budget on a metric that
|
|
50
|
+
stopped moving at iteration three. An exit signal alone is an unbounded loop
|
|
51
|
+
wearing a condition: if the signal never fires — and a subtly wrong predicate
|
|
52
|
+
never fires — nothing stops it. Each guards a different failure direction, so
|
|
53
|
+
neither is the redundant one.
|
|
54
|
+
|
|
55
|
+
## What ends a run that is not an exit condition
|
|
56
|
+
|
|
57
|
+
Nothing. Specifically **not**: the number looking good enough, the change
|
|
58
|
+
starting to feel risky, or the register getting long. Those are judgements, and
|
|
59
|
+
a loop that stops on a judgement produces a result nobody can reproduce. If the
|
|
60
|
+
run should stop for a reason the predicates do not cover, that is a finding
|
|
61
|
+
about the predicates — record it and fix them for the next run.
|
|
@@ -0,0 +1,66 @@
|
|
|
1
|
+
# experiment-loop — the register
|
|
2
|
+
|
|
3
|
+
Loaded on demand by [`experiment-loop`](../SKILL.md). The register is the run:
|
|
4
|
+
if it cannot reconstruct what happened with the transcript discarded, the run
|
|
5
|
+
was not recorded.
|
|
6
|
+
|
|
7
|
+
## Shape
|
|
8
|
+
|
|
9
|
+
An **append-only JSONL** file, one row per iteration, **untracked** (the run is
|
|
10
|
+
evidence, not source). Re-read from disk at the top of every cycle.
|
|
11
|
+
|
|
12
|
+
Row 0 declares the run:
|
|
13
|
+
|
|
14
|
+
```json
|
|
15
|
+
{"row": 0, "evaluator": "bundle-size", "direction": "minimize",
|
|
16
|
+
"baseline_score": -142000, "max_iterations": 12,
|
|
17
|
+
"exit_signal": "consecutive_reverts >= 3"}
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
Each iteration:
|
|
21
|
+
|
|
22
|
+
```json
|
|
23
|
+
{"row": 1, "label": "drop the polyfill for evergreen targets", "head": "a1b2c3d",
|
|
24
|
+
"baseline_score": -142000, "score": -138400, "metric": 138400,
|
|
25
|
+
"metric_state": "present", "pass": true, "decision": "keep",
|
|
26
|
+
"reason": "strict improvement"}
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
`decision` is one of `pending` · `keep` · `revert`. `reason` is one of
|
|
30
|
+
`strict improvement` · `no strict improvement` · `test red` ·
|
|
31
|
+
`evaluator failed` · `metric unreadable`.
|
|
32
|
+
|
|
33
|
+
## Write the row BEFORE the action it records
|
|
34
|
+
|
|
35
|
+
This is the ordering that costs a run to learn. The obvious sequence — commit,
|
|
36
|
+
evaluate, revert, then append — leaves a window in which the branch has advanced
|
|
37
|
+
and the register has not. Spike s01 hit it twice: the revert threw, the process
|
|
38
|
+
died, and the row was never written, so the register described a run one
|
|
39
|
+
iteration shorter than the branch.
|
|
40
|
+
|
|
41
|
+
So: append the row with `decision: "pending"` first, then act, then rewrite that
|
|
42
|
+
row's decision. A crash then leaves a visible `pending` — a row that says "an
|
|
43
|
+
iteration started here and we do not know how it ended", which is recoverable —
|
|
44
|
+
instead of silence, which is not.
|
|
45
|
+
|
|
46
|
+
**"Append-only" and "rewrite the last row" are not in conflict.** Append-only
|
|
47
|
+
governs *history*: no row is ever deleted and no earlier row is ever edited. The
|
|
48
|
+
pending row is the write frontier, not history, until its outcome lands.
|
|
49
|
+
|
|
50
|
+
## Why on disk and not in the conversation
|
|
51
|
+
|
|
52
|
+
Two independent references converged on files-as-truth after documenting silent
|
|
53
|
+
degradation of in-conversation loop state, and the mechanism is not exotic: a
|
|
54
|
+
baseline held in context drifts as the context is summarised, and the loop keeps
|
|
55
|
+
comparing against a number that is no longer the one it kept. Nothing errors.
|
|
56
|
+
The run just quietly stops meaning anything.
|
|
57
|
+
|
|
58
|
+
Re-reading from disk each cycle costs one file read and removes the entire
|
|
59
|
+
class.
|
|
60
|
+
|
|
61
|
+
## Reporting from the register
|
|
62
|
+
|
|
63
|
+
The final report is derived, never remembered: iterations run against the bound,
|
|
64
|
+
baseline → final score, keeps versus reverts, the exit condition that fired, and
|
|
65
|
+
every reverted iteration with its reason. A revert is the most informative row in
|
|
66
|
+
the file — it is the hypothesis the metric rejected, and the next run needs it.
|