ai-engineering-loop 2.10.0 → 2.11.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -11,6 +11,7 @@
11
11
 
12
12
  Run these when the user asks how the loop is doing, before planning work, or when a gate points at them. All read local Run files; none starts a Run.
13
13
  - `stats [--last <n>] [--json]`: failure causes by harness layer (`task_spec`, `context`, `environment`, `verification`, `state`, `model`), false-victory rate per tier, iterations to PASS, revert rate of delivered Runs (from git) and the verification share of tokens. Fix the bottleneck layer first.
14
+ - `learn promote ... --prove <commit>`: the check must fail on the commit that introduced the mistake. `measurement record --ac <id> --runs <n> --spread <pct> --effect <pct> --limiter <text>` for `measure: true` ACs (`docs verification-loop` 3e).
14
15
  - `recap [--days N] [--for me|lead|client] [--projects <p,...>|all]`: one HTML page of what shipped, with evidence (`docs recap`). Client recaps take one project.
15
16
  - `outcome record --run <id> --source mr-comment|revert|bug|incident --location <file[:line]> --summary <text> --by <name>`: a problem people found after delivery. Only record what the human reported; never invent one. Then `calibrate` lists DA misses and wrong dismissals per reviewer prompt version.
16
17
  - `debt [--file <path>]` lists trade-offs the Judge accepted and L3 waivers; `debt close <id> --by <name>` only on the human's word.
@@ -6,5 +6,6 @@ Applies to HIGH_RISK_PATH changes (auth, payments, migrations, secrets, infra),
6
6
  - Business-logic changes need a blast radius: lifecycle sketch of what reads and writes the changed state; on `adapter_type: dot` load `task-impact-inquiry`. Skip it for docs, config, or UI-copy changes.
7
7
  - Full DA (not the T1 lean shortcut) and Judge on STRONG_MODEL resolved through the host model catalog; if unavailable, escalate to the user. Never run the deterministic Judge for T2.
8
8
  - UNATTENDED: sandbox worktree only, and `budget pause` is the kill switch (`docs runtime-safety`).
9
+ - Before Judge: `safety record --fact "<the one fact it is safe because of>" --cmd "<proof that passes only if it holds>"` (PASS needs level 4+).
9
10
  - The tier never falls back once raised.
10
11
  - New feature or architecture: suggest a reviewed design first (`docs spec-pack`), then `goal draft --from-design <doc>`.
@@ -67,6 +67,7 @@ Return a Finding Ledger as a fenced JSON block and stop:
67
67
  "acceptanceCriteria": "AC-1",
68
68
  "failureScenario": "Concrete failing case",
69
69
  "reproduction": "Steps to reproduce",
70
+ "proof": { "command": "node --test test/cancel.test.js" },
70
71
  "evidence": "Hunk or line you read",
71
72
  "concreteAlternativeDiff": "diff snippet"
72
73
  }
@@ -74,7 +75,7 @@ Return a Finding Ledger as a fenced JSON block and stop:
74
75
  }
75
76
  ```
76
77
 
77
- Rules: axis is spec or standards (default spec). hardConvention is boolean, default false. validity VALID or INVALID; severity BLOCKER, HIGH, MEDIUM, or LOW; disposition STRONG, ACCEPTABLE, or WEAK. VALID BLOCKER or HIGH must include concreteAlternativeDiff. Empty findings is allowed.
78
+ Rules: axis is spec or standards (default spec). hardConvention is boolean, default false. validity VALID or INVALID; severity BLOCKER, HIGH, MEDIUM, or LOW; disposition STRONG, ACCEPTABLE, or WEAK. VALID BLOCKER or HIGH must include concreteAlternativeDiff. For VALID BLOCKER or HIGH, add `proof.command` when a run can show the bug: a test or script that exits non-zero while the bug is present. The CLI runs it at `gate review`; a proof that exits 0 marks the finding `not_reproduced` and it stops blocking, so only claim what a run shows. Never use a destructive command. Empty findings is allowed.
78
79
 
79
80
  Lean output caps:
80
81
 
@@ -23,6 +23,8 @@ ITERATE if verification is missing, vague, non-zero, proves a different artifact
23
23
 
24
24
  On ITERATE or ESCALATE, set `cause` to the harness layer the failure points at: `task_spec` (AC unclear or wrong), `context` (missing project knowledge), `environment` (setup/tooling), `verification` (weak or missing proof), `state` (lost continuity), or `model` (plain implementation defect). The gate infers one when you omit it.
25
25
 
26
+ A finding with `proofStatus: not_reproduced` ran its proof and the bug did not show; it cannot justify ITERATE. If your `cause` repeats the previous ITERATE's cause, add `premise: {statement, test}`: the one assumption both fixes shared and what testing it showed (`PREMISE_REQUIRED`). T2 PASS needs a safety fact at level 4+ (`safety record --cmd`); an AC with `measure: true` needs `measurement record` with 3+ runs, a limiter, and spread below the effect.
27
+
26
28
  The gate recomputes the verdict from the ledger. To block on a problem the ledger lacks (wrong-artifact or happy-path-only evidence, an AC the tests do not prove), add it to `judgeFindings` in the Finding Ledger shape: VALID, BLOCKER or HIGH, with `location`, `failureScenario`, `evidence`, and `concreteAlternativeDiff`. A verdict that blocks without such a finding is rejected. You cannot PASS what the ledger blocks.
27
29
 
28
30
  ## Axes
@@ -68,6 +68,7 @@ Return a Finding Ledger as a fenced JSON block and stop:
68
68
  "acceptanceCriteria": "AC-1",
69
69
  "failureScenario": "Concrete failing case",
70
70
  "reproduction": "Steps to reproduce",
71
+ "proof": { "command": "node --test test/cancel.test.js" },
71
72
  "evidence": "Hunk or line you read",
72
73
  "concreteAlternativeDiff": "diff snippet"
73
74
  }
@@ -75,7 +76,7 @@ Return a Finding Ledger as a fenced JSON block and stop:
75
76
  }
76
77
  ```
77
78
 
78
- Rules: axis is spec or standards (default spec). hardConvention is boolean, default false. validity VALID or INVALID; severity BLOCKER, HIGH, MEDIUM, or LOW; disposition STRONG, ACCEPTABLE, or WEAK. VALID BLOCKER or HIGH must include concreteAlternativeDiff. Empty findings is allowed.
79
+ Rules: axis is spec or standards (default spec). hardConvention is boolean, default false. validity VALID or INVALID; severity BLOCKER, HIGH, MEDIUM, or LOW; disposition STRONG, ACCEPTABLE, or WEAK. VALID BLOCKER or HIGH must include concreteAlternativeDiff. For VALID BLOCKER or HIGH, add `proof.command` when a run can show the bug: a test or script that exits non-zero while the bug is present. The CLI runs it at `gate review`; a proof that exits 0 marks the finding `not_reproduced` and it stops blocking, so only claim what a run shows. Never use a destructive command. Empty findings is allowed.
79
80
 
80
81
  Lean output caps:
81
82
 
@@ -24,6 +24,8 @@ ITERATE if verification is missing, vague, non-zero, proves a different artifact
24
24
 
25
25
  On ITERATE or ESCALATE, set `cause` to the harness layer the failure points at: `task_spec` (AC unclear or wrong), `context` (missing project knowledge), `environment` (setup/tooling), `verification` (weak or missing proof), `state` (lost continuity), or `model` (plain implementation defect). The gate infers one when you omit it.
26
26
 
27
+ A finding with `proofStatus: not_reproduced` ran its proof and the bug did not show; it cannot justify ITERATE. If your `cause` repeats the previous ITERATE's cause, add `premise: {statement, test}`: the one assumption both fixes shared and what testing it showed (`PREMISE_REQUIRED`). T2 PASS needs a safety fact at level 4+ (`safety record --cmd`); an AC with `measure: true` needs `measurement record` with 3+ runs, a limiter, and spread below the effect.
28
+
27
29
  The gate recomputes the verdict from the ledger. To block on a problem the ledger lacks (wrong-artifact or happy-path-only evidence, an AC the tests do not prove), add it to `judgeFindings` in the Finding Ledger shape: VALID, BLOCKER or HIGH, with `location`, `failureScenario`, `evidence`, and `concreteAlternativeDiff`. A verdict that blocks without such a finding is rejected. You cannot PASS what the ledger blocks.
28
30
 
29
31
  ## Axes
package/CHANGELOG.md CHANGED
@@ -6,6 +6,23 @@ All notable changes to this project are documented here. Versions follow [Semant
6
6
 
7
7
  _No changes yet._
8
8
 
9
+ ## 2.11.0 - 2026-11-18
10
+
11
+ From pstack (cards C-29..C-36 in `docs/cards/proof-not-prose-2026-11-18.md`): less human re-checking, because the CLI runs the proof instead of trusting prose.
12
+
13
+ - **Added:** finding proofs (C-29): a VALID BLOCKER/HIGH finding may carry `proof.command`; `gate review` runs it and records `proofStatus`. A proof that passes (`not_reproduced`) stops the finding from blocking; destructive or timed-out proofs keep it blocking. Results in `runs/<id>/proofs.json`; ablation component `proofs`.
14
+ - **Added:** `safety record` (C-30): the one fact a change is safe because of, with a proof level 1–5. T2 Judge PASS needs level 4+ (a passing proof command).
15
+ - **Added:** `learn promote --prove <commit>` (C-31): the new check must fail on the commit that introduced the mistake; `doctor --harness` lists unproven rules.
16
+ - **Added:** premise check (C-32): an ITERATE that repeats the previous explicit `cause` needs `premise: {statement, test}`.
17
+ - **Added:** `measurement record` and AC `measure: true` (C-33): PASS needs 3+ runs, a limiter, and spread below the effect.
18
+ - **Added:** `doctor --harness` warns on feature-map paths that match no file (C-34); `formal_covers` in verification.md adds the `formal` key to default `verify` when covered code changes (C-35).
19
+ - **Added:** `stats` and `recap` report human interventions per Run and the share of finding proofs that reproduced (C-36).
20
+ - **Changed:** T2 runs now need a recorded safety fact before Judge PASS.
21
+
22
+ ## 2.10.1 - 2026-11-17
23
+
24
+ - **Fixed:** after a Judge `ITERATE`, `verify` refused to record evidence for the new Maker diff ("Verification has already passed its gate") and `gate verification` rejected the old evidence's diff hash, so the Run could not continue. The iteration's verification files now move to `runs/<id>/iterations/<n>/` and the next iteration starts with a fresh bundle.
25
+
9
26
  ## 2.10.0 - 2026-11-17
10
27
 
11
28
  Cards C-27 and C-28 in `docs/cards/slop-to-gold-2026-11-16.md`.
package/README.md CHANGED
@@ -1,45 +1,35 @@
1
1
  <div align="center">
2
2
 
3
+ <a href="https://ael.egagofur.studio"><img src="docs/images/readme-banner.png" alt="AI Engineering Loop: From idea to trusted code. npx ai-engineering-loop init" width="100%"></a>
4
+
3
5
  # AI Engineering Loop
4
6
 
5
- ### From idea to trusted code—without babysitting every command.
7
+ ### Give your coding agent a goal and stop babysitting every command.
6
8
 
7
- Give your AI agent a goal. It plans the work, builds it, checks itself, challenges the result, and prepares a safe delivery. You get a live view of the run and a clear history instead of a wall of terminal output.
9
+ It plans, builds, tests its own claims, lets a Devil’s Advocate attack the result, and waits for your approval before anything ships. Your project and its evidence stay on your machine.
8
10
 
9
11
  [![npm](https://img.shields.io/npm/v/ai-engineering-loop?style=flat-square&color=28AD00)](https://www.npmjs.com/package/ai-engineering-loop)
10
12
  [![GitHub release](https://img.shields.io/github/v/release/egagofur/ai-engineering-loop?style=flat-square&color=28AD00)](https://github.com/egagofur/ai-engineering-loop/releases)
11
13
  [![license](https://img.shields.io/badge/license-MIT-181c19?style=flat-square)](LICENSE)
14
+ [![website](https://img.shields.io/badge/site-ael.egagofur.studio-fc6404?style=flat-square)](https://ael.egagofur.studio)
12
15
 
13
- [Get started](#get-started-in-a-minute) · [See what it does](#one-workspace-from-idea-to-delivery) · [Open Workflow Studio](#meet-workflow-studio) · [Read the docs](core/)
16
+ **[Watch a Run play out →](https://ael.egagofur.studio)** · [Get started](#get-started-in-a-minute) · [How a Run works](#how-a-run-works) · [Workflow Studio](#meet-workflow-studio) · [Docs](core/)
14
17
 
15
18
  </div>
16
19
 
17
- [![AI Engineering Loop Workflow Studio](docs/images/studio-hero.svg)](docs/images/studio-hero.svg)
18
-
19
- ## Software delivery that feels simple
20
+ ## Fast agents lose the details
20
21
 
21
- AI coding tools are fast, but important details can disappear between the prompt, the code, and the final answer. AI Engineering Loop gives every task one calm, repeatable path:
22
+ Coding agents write a feature in minutes, then hand you the result with no trail behind it:
22
23
 
23
- | You want | The Loop handles |
24
- |---|---|
25
- | **A clear outcome** | Turns the request into an agreed Goal before code changes |
26
- | **Less supervision** | Guides your agent through planning, building, and checking |
27
- | **Honest progress** | Shows what is ready, running, blocked, or waiting for approval |
28
- | **Fewer surprises** | Tests happy paths, edge cases, and failure cases |
29
- | **A trustworthy result** | Compares every claim with real evidence before delivery |
30
- | **A useful history** | Keeps searchable Runs, decisions, artifacts, and proof |
24
+ - **“Done” without proof.** The agent reports success. Nothing shows which command ran or what it returned.
25
+ - **Plans that vanish.** Scope and edge cases lived in a chat you already closed.
26
+ - **Shipping on trust.** Code reaches a pull request because the agent said it was ready.
31
27
 
32
- You stay in control of the product decisions. Your AI agent takes care of the repetitive engineering ceremony.
28
+ AI Engineering Loop puts every task on one gated path and keeps the receipts. You stay in charge of the product decisions; your agent handles the repetitive engineering ceremony.
33
29
 
34
30
  ## Get started in a minute
35
31
 
36
- Install it in any project:
37
-
38
- ```bash
39
- npx ai-engineering-loop init
40
- ```
41
-
42
- Then tell your coding agent:
32
+ Run the command above in any project. Then tell your coding agent:
43
33
 
44
34
  > Use AI Engineering Loop to build this feature: **your idea here**
45
35
 
@@ -55,50 +45,30 @@ Workflow Studio opens locally in your browser and follows the active Run live. Y
55
45
 
56
46
  On first setup, `.ai-engineering-loop/` is added to your project’s `.gitignore` so plans and Run evidence stay private by default. Want to share that context with your team? Simply remove that line from `.gitignore`.
57
47
 
58
- ### Less artifact guesswork for your AI agent
48
+ <a id="one-workspace-from-idea-to-delivery"></a>
59
49
 
60
- Agents can record test commands directly so verification evidence is tied to the current Run and gated diff:
50
+ ## How a Run works
61
51
 
62
- ```bash
63
- npx ai-engineering-loop verification record --run <run-id> -- npm test
64
- ```
52
+ Every Run passes the same gates, and the Judge can send work back as many times as the recipe allows:
65
53
 
66
- The command captures bounded, redacted output without invoking a shell implicitly, records failures honestly, and generates a compact test summary. For other stage artifacts, use `scaffold delivery --run <run-id>` (or `verification` / `findings`) to create a private, deliberately incomplete template. Copy or rename it to the gate's real artifact filename, fill it with observed evidence, and remove `_templateOnly`; then `validate <repository-relative-file>` reports schema and Run-binding issues without advancing a gate. Templates are never accepted as evidence. `context claimed-vs-reality --run <run-id>` creates the Goal checklist while leaving claim and evidence cells blank.
67
-
68
- Common stage sequences also have one-call forms that gate exactly like the long forms:
69
-
70
- ```bash
71
- npx ai-engineering-loop verify --run <run-id> --cmd "npm test" # record (parallel), gate, compact, claimed-vs-reality
72
- npx ai-engineering-loop review-pack --run <run-id> --json # fast-path + diff-hunk pack + escalation floor
73
- npx ai-engineering-loop deliver --run <run-id> --destination pr --summary "..." --approve --by <name>
54
+ ```mermaid
55
+ flowchart LR
56
+ G["Goal Contract<br/>frozen before code"] --> M["Maker<br/>plan and build"]
57
+ M --> V["Verify<br/>real command, recorded"]
58
+ V --> R["Review<br/>Devil’s Advocate challenges claims"]
59
+ R --> J{"Judge"}
60
+ J -- "ITERATE" --> M
61
+ J -- "PASS" --> D["Deliver<br/>waits for --approve"]
74
62
  ```
75
63
 
76
- `deliver` refuses without `--approve` in ASSISTED mode; pass it only after a human approved delivery.
77
-
78
- Contributors: the host `ael` skills are generated. Edit `skill-src/ael/`, then run `npm run build:skills` (CI checks for drift with `npm run check:skills`). `npm run measure` reports per-tier context and tool calls.
64
+ The experience is organized into four moments:
79
65
 
80
- ### Staying up to date
66
+ 1. **Specify**: agree on what success and failure look like.
67
+ 2. **Make**: let the agent investigate, plan, and build.
68
+ 3. **Review**: verify the result and challenge unsupported claims.
69
+ 4. **Deliver**: ask for human approval before publishing.
81
70
 
82
- `run`, `status`, `init`, and `refresh` check npm in the background at most once a day; the command never waits. What happens next depends on your policy:
83
-
84
- ```bash
85
- npx ai-engineering-loop update policy minor # off | notify | patch | minor | all (majors are always notice-only)
86
- npx ai-engineering-loop projects list # projects refreshed after an update (scan: projects scan --root ~/Development)
87
- npx ai-engineering-loop update rollback --yes
88
- ```
89
-
90
- Global installs apply patch and minor releases automatically. They then re-sync host skills and refresh every registered project, keeping your edits (new templates arrive as `*.ael-new`). Project installs only show a short notice (current -> latest, a few changelog highlights, the exact `npm`/`pnpm`/`yarn`/`bun` command, how to silence it), at most once a day per project, because AEL never edits your `package.json`. Projects from versions before hash tracking are overwritten on their first update, with the old files kept in `.ai-engineering-loop/.ael-backup/` and restored by `update rollback`. Major updates run `migrate --from <old>` after install. A major release such as 2.0.0 only shows a notice: install it yourself, then run `ai-engineering-loop sync-hosts` once so your agents get the new skill. Opt out with `AEL_NO_AUTO_UPDATE=1`; CI is always off. There is no daemon, so other machines pick up an update the next time the CLI runs there.
91
-
92
- Projects with an unfinished run are skipped until that run ends. If a run was simply left behind, close it (the run is kept, marked `CANCELLED`, and `current.json` is released):
93
-
94
- ```bash
95
- npx ai-engineering-loop run --abandon --reason "left behind after delivery" --by ega # asks to confirm; add --yes to skip
96
- npx ai-engineering-loop run --abandon --run <run-id> --reason "superseded" --by ega --yes # a non-current run
97
- ```
98
-
99
- `status`, `state`, and `projects list` show this hint when an active run has not moved for 7 days. UNATTENDED runs need `--yes` from a human.
100
-
101
- When context packs exceed their budget, the CLI now reports the limit and a per-file size/token estimate so an agent can choose smaller files without trial and error. For a delivery gate blocked by a recipe approval node, inspect `node status --run <run-id>` and explicitly approve the READY prerequisite; the gate error points to the actionable command.
71
+ Under the hood, these moments use eight evidence-backed stages: `Specify (stages 0-1)` → `Make (stages 2-4)` → `Review (stages 5-7)` → `Deliver (stage 8)`. The rigor is there when you need it, but it does not have to dominate the experience.
102
72
 
103
73
  ## Meet Workflow Studio
104
74
 
@@ -133,25 +103,13 @@ If the agent needs clarification, its questions can appear inside Workflow Studi
133
103
 
134
104
  Recipes can declare independent Loop Groups with a maximum number of passes. Studio shows a separate `Iteration 2 / 3` counter for each group, and a loop that reaches its limit pauses for a human decision instead of retrying forever.
135
105
 
136
- > **New in v2.0.0:** real-time Studio, wiring-only canvas, `ael bench`, and personal workflows. Studio node creation and import were removed. [See the release](https://github.com/egagofur/ai-engineering-loop/releases/tag/v2.0.0).
137
-
138
- ## One workspace from idea to delivery
139
-
140
- The experience is organized into four simple moments:
141
-
142
- 1. **Specify** — agree on what success and failure look like.
143
- 2. **Make** — let the agent investigate, plan, and build.
144
- 3. **Review** — verify the result and challenge unsupported claims.
145
- 4. **Deliver** — ask for human approval before publishing.
146
-
147
- Under the hood, these moments use eight evidence-backed stages. The rigor is there when you need it, but it does not have to dominate the experience.
148
-
149
- `Specify (stages 0-1)` → `Make (stages 2-4)` → `Review (stages 5-7)` → `Deliver (stage 8)`
150
-
151
106
  ## Bring your own AI agent
152
107
 
153
108
  AI Engineering Loop is designed to work with the tools developers already use:
154
109
 
110
+ <details>
111
+ <summary>Works with Claude Code, Codex, Cursor, Kiro, and 16 more.</summary>
112
+
155
113
  <!-- hosts-table:start (generated from lib/hosts/registry.js; run npm run build:hosts-table) -->
156
114
  | Host | id | Certification | Guardrails | Skill folder | Reviewer agents | Detected by |
157
115
  |---|---|---|---|---|---|---|
@@ -183,6 +141,8 @@ Guardrails: **enforced** = `npx ai-engineering-loop hooks install` adds native p
183
141
  Not supported (no native skills): Aider, Continue.
184
142
  <!-- hosts-table:end -->
185
143
 
144
+ </details>
145
+
186
146
  Every host gets the **same** `ael` skill, invoked as `/ael` (byte-identical, so an agent that scans several skill folders never sees two different versions). The few host-specific spawn rules live in its `references/loop-hosts.md`. Earlier releases named the skill `ai-engineering-loop`; `sync-hosts` removes that old folder when AEL wrote it and `hosts doctor` flags any copy it had to keep. Each host follows the same Goal, verification, review, and approval rules.
187
147
 
188
148
  Install once, then let AEL find your agents:
@@ -299,6 +259,49 @@ By default, smart context skips high-volume review inputs such as CSS, generated
299
259
 
300
260
  The friendly workflow sits on top of a complete local CLI. You can inspect recipes, budgets, evidence, nodes, gates, policies, and handoffs whenever you need to.
301
261
 
262
+ ### Record evidence by hand
263
+
264
+ Agents can record test commands directly so verification evidence is tied to the current Run and gated diff:
265
+
266
+ ```bash
267
+ npx ai-engineering-loop verification record --run <run-id> -- npm test
268
+ ```
269
+
270
+ The command captures bounded, redacted output without invoking a shell implicitly, records failures honestly, and generates a compact test summary. For other stage artifacts, use `scaffold delivery --run <run-id>` (or `verification` / `findings`) to create a private, deliberately incomplete template. Copy or rename it to the gate's real artifact filename, fill it with observed evidence, and remove `_templateOnly`; then `validate <repository-relative-file>` reports schema and Run-binding issues without advancing a gate. Templates are never accepted as evidence. `context claimed-vs-reality --run <run-id>` creates the Goal checklist while leaving claim and evidence cells blank.
271
+
272
+ Common stage sequences also have one-call forms that gate exactly like the long forms:
273
+
274
+ ```bash
275
+ npx ai-engineering-loop verify --run <run-id> --cmd "npm test" # record (parallel), gate, compact, claimed-vs-reality
276
+ npx ai-engineering-loop review-pack --run <run-id> --json # fast-path + diff-hunk pack + escalation floor
277
+ npx ai-engineering-loop deliver --run <run-id> --destination pr --summary "..." --approve --by <name>
278
+ ```
279
+
280
+ `deliver` refuses without `--approve` in ASSISTED mode; pass it only after a human approved delivery.
281
+
282
+ ### Staying up to date
283
+
284
+ `run`, `status`, `init`, and `refresh` check npm in the background at most once a day; the command never waits. What happens next depends on your policy:
285
+
286
+ ```bash
287
+ npx ai-engineering-loop update policy minor # off | notify | patch | minor | all (majors are always notice-only)
288
+ npx ai-engineering-loop projects list # projects refreshed after an update (scan: projects scan --root ~/Development)
289
+ npx ai-engineering-loop update rollback --yes
290
+ ```
291
+
292
+ Global installs apply patch and minor releases automatically. They then re-sync host skills and refresh every registered project, keeping your edits (new templates arrive as `*.ael-new`). Project installs only show a short notice (current -> latest, a few changelog highlights, the exact `npm`/`pnpm`/`yarn`/`bun` command, how to silence it), at most once a day per project, because AEL never edits your `package.json`. Projects from versions before hash tracking are overwritten on their first update, with the old files kept in `.ai-engineering-loop/.ael-backup/` and restored by `update rollback`. Major updates run `migrate --from <old>` after install. A major release such as 2.0.0 only shows a notice: install it yourself, then run `ai-engineering-loop sync-hosts` once so your agents get the new skill. Opt out with `AEL_NO_AUTO_UPDATE=1`; CI is always off. There is no daemon, so other machines pick up an update the next time the CLI runs there.
293
+
294
+ Projects with an unfinished run are skipped until that run ends. If a run was simply left behind, close it (the run is kept, marked `CANCELLED`, and `current.json` is released):
295
+
296
+ ```bash
297
+ npx ai-engineering-loop run --abandon --reason "left behind after delivery" --by ega # asks to confirm; add --yes to skip
298
+ npx ai-engineering-loop run --abandon --run <run-id> --reason "superseded" --by ega --yes # a non-current run
299
+ ```
300
+
301
+ `status`, `state`, and `projects list` show this hint when an active run has not moved for 7 days. UNATTENDED runs need `--yes` from a human.
302
+
303
+ When context packs exceed their budget, the CLI now reports the limit and a per-file size/token estimate so an agent can choose smaller files without trial and error. For a delivery gate blocked by a recipe approval node, inspect `node status --run <run-id>` and explicitly approve the READY prerequisite; the gate error points to the actionable command.
304
+
302
305
  <details>
303
306
  <summary><strong>Show common CLI commands</strong></summary>
304
307
 
@@ -377,6 +380,8 @@ No hidden cloud control plane is required. No project source is uploaded by Work
377
380
 
378
381
  Ideas, workflow recipes, bug reports, and pull requests are welcome. Start with an issue or open a focused PR with a clear explanation of the user outcome.
379
382
 
383
+ Contributors: the host `ael` skills are generated. Edit `skill-src/ael/`, then run `npm run build:skills` (CI checks for drift with `npm run check:skills`). `npm run measure` reports per-tier context and tool calls.
384
+
380
385
  ```bash
381
386
  npm test
382
387
  npm run doctor
@@ -60,6 +60,7 @@ Return a Finding Ledger as a fenced JSON block and stop:
60
60
  "acceptanceCriteria": "AC-1",
61
61
  "failureScenario": "Concrete failing case",
62
62
  "reproduction": "Steps to reproduce",
63
+ "proof": { "command": "node --test test/cancel.test.js" },
63
64
  "evidence": "Hunk or line you read",
64
65
  "concreteAlternativeDiff": "diff snippet"
65
66
  }
@@ -67,7 +68,7 @@ Return a Finding Ledger as a fenced JSON block and stop:
67
68
  }
68
69
  ```
69
70
 
70
- Rules: axis is spec or standards (default spec). hardConvention is boolean, default false. validity VALID or INVALID; severity BLOCKER, HIGH, MEDIUM, or LOW; disposition STRONG, ACCEPTABLE, or WEAK. VALID BLOCKER or HIGH must include concreteAlternativeDiff. Empty findings is allowed.
71
+ Rules: axis is spec or standards (default spec). hardConvention is boolean, default false. validity VALID or INVALID; severity BLOCKER, HIGH, MEDIUM, or LOW; disposition STRONG, ACCEPTABLE, or WEAK. VALID BLOCKER or HIGH must include concreteAlternativeDiff. For VALID BLOCKER or HIGH, add `proof.command` when a run can show the bug: a test or script that exits non-zero while the bug is present. The CLI runs it at `gate review`; a proof that exits 0 marks the finding `not_reproduced` and it stops blocking, so only claim what a run shows. Never use a destructive command. Empty findings is allowed.
71
72
 
72
73
  Lean output caps:
73
74
 
@@ -16,6 +16,8 @@ ITERATE if verification is missing, vague, non-zero, proves a different artifact
16
16
 
17
17
  On ITERATE or ESCALATE, set `cause` to the harness layer the failure points at: `task_spec` (AC unclear or wrong), `context` (missing project knowledge), `environment` (setup/tooling), `verification` (weak or missing proof), `state` (lost continuity), or `model` (plain implementation defect). The gate infers one when you omit it.
18
18
 
19
+ A finding with `proofStatus: not_reproduced` ran its proof and the bug did not show; it cannot justify ITERATE. If your `cause` repeats the previous ITERATE's cause, add `premise: {statement, test}`: the one assumption both fixes shared and what testing it showed (`PREMISE_REQUIRED`). T2 PASS needs a safety fact at level 4+ (`safety record --cmd`); an AC with `measure: true` needs `measurement record` with 3+ runs, a limiter, and spread below the effect.
20
+
19
21
  The gate recomputes the verdict from the ledger. To block on a problem the ledger lacks (wrong-artifact or happy-path-only evidence, an AC the tests do not prove), add it to `judgeFindings` in the Finding Ledger shape: VALID, BLOCKER or HIGH, with `location`, `failureScenario`, `evidence`, and `concreteAlternativeDiff`. A verdict that blocks without such a finding is rejected. You cannot PASS what the ledger blocks.
20
22
 
21
23
  ## Axes
@@ -3530,6 +3530,10 @@ async function handleVerify() {
3530
3530
  if (smokeText && !baselineMode && lanes.startupRiskFiles(CWD, loadRun(CWD, runId)).length) {
3531
3531
  plan.push({ key: 'smoke', layer: 'L2', text: smokeText });
3532
3532
  }
3533
+ const formalText = lanes.verificationCommands(CWD).formal;
3534
+ if (formalText && !baselineMode && !plan.some((step) => step.key === 'formal') && lanes.formalCoveredFiles(CWD, loadRun(CWD, runId)).length) {
3535
+ plan.push({ key: 'formal', layer: 'LF', text: formalText });
3536
+ }
3533
3537
  if (!plan.length) throw Object.assign(new Error('No --cmd given and no build/typecheck/lint/test_unit command in .ai-engineering-loop/verification.md. FIX: add them to verification.md or pass --cmd.'), { code: 'INVALID_COMMAND' });
3534
3538
  commandsToRun = plan.map((step) => ({ ...splitCommand(step.text, shell || Boolean(step.shell)), key: step.key, layer: step.layer, ...(step.env ? { env: step.env } : {}) }));
3535
3539
  defaultPlan = true;
@@ -3766,6 +3770,43 @@ function handleRules() {
3766
3770
  }
3767
3771
  }
3768
3772
 
3773
+ function handleSafety() {
3774
+ try {
3775
+ if (args[1] !== 'record') throw Object.assign(new Error('Use: safety record --run <id> --fact <text> [--cmd <command>] [--live] [--level 1-3]'), { code: 'INVALID_USAGE' });
3776
+ const runId = requiredRunId('safety record');
3777
+ const value = require('../lib/safety-fact.js').recordSafetyFact(CWD, runId, {
3778
+ fact: argValue('--fact'),
3779
+ command: argValue('--cmd'),
3780
+ live: process.argv.includes('--live'),
3781
+ level: argValue('--level')
3782
+ });
3783
+ if (process.argv.includes('--json')) console.log(JSON.stringify({ ok: true, runId, safetyFact: value }, null, 2));
3784
+ else log.success(`✓ Safety fact recorded at level ${value.level} (${value.levelName}): ${value.fact}`);
3785
+ } catch (error) {
3786
+ log.error(`${error.code || 'SAFETY_FAILED'}: ${error.message}`);
3787
+ process.exitCode = 1;
3788
+ }
3789
+ }
3790
+
3791
+ function handleMeasurement() {
3792
+ try {
3793
+ if (args[1] !== 'record') throw Object.assign(new Error('Use: measurement record --run <id> --ac <AC-n> --runs <n> --spread <pct> --effect <pct> --limiter <text>'), { code: 'INVALID_USAGE' });
3794
+ const runId = requiredRunId('measurement record');
3795
+ const value = require('../lib/measurements.js').recordMeasurement(CWD, runId, {
3796
+ ac: argValue('--ac'),
3797
+ runs: argValue('--runs'),
3798
+ spread: argValue('--spread'),
3799
+ effect: argValue('--effect'),
3800
+ limiter: argValue('--limiter')
3801
+ });
3802
+ if (process.argv.includes('--json')) console.log(JSON.stringify({ ok: true, runId, measurement: value }, null, 2));
3803
+ else log.success(`✓ ${value.ac}: ${value.runs} runs, spread ${value.spread}%, effect ${value.effect}%, limited by ${value.limiter}`);
3804
+ } catch (error) {
3805
+ log.error(`${error.code || 'MEASUREMENT_FAILED'}: ${error.message}`);
3806
+ process.exitCode = 1;
3807
+ }
3808
+ }
3809
+
3769
3810
  function handleRecap() {
3770
3811
  try {
3771
3812
  const days = argValue('--days') == null ? 7 : Number(argValue('--days'));
@@ -4291,6 +4332,7 @@ function handleLearn() {
4291
4332
  forbidImport: argValue('--forbid-import'),
4292
4333
  forbidApi: argValue('--forbid-api'),
4293
4334
  except: argValue('--except'),
4335
+ prove: argValue('--prove'),
4294
4336
  shell: !process.argv.includes('--no-shell')
4295
4337
  });
4296
4338
  if (json) console.log(JSON.stringify({ ok: true, rule }));
@@ -4463,6 +4505,8 @@ Commands:
4463
4505
  calibrate Compare recorded outcomes with each Run's Finding Ledger: DA misses, wrong dismissals, per reviewer prompt
4464
4506
  debt Accepted trade-offs and L3 waivers recorded at delivery: list [--file <path>] | close <id> --by <name> [--note <text>]
4465
4507
  next Next unblocked feature from .ai-engineering-loop/features.json, VCR (verified/activated), evidence audit [--json]
4508
+ safety Record the one fact a change is safe because of [record --fact <text> --cmd <proof>] (T2 Judge PASS needs level 4+)
4509
+ measurement Record how a performance number was measured [record --ac <id> --runs <n> --spread <pct> --effect <pct> --limiter <text>]
4466
4510
  recap Evidence-backed recap of the last N days as one HTML page [--days 7] [--for me|lead|client] [--projects <p,...>|all] [--out <file>] [--lang <tag>] [--open] [--json]
4467
4511
  stats Failure causes by harness layer, false-victory rate, iterations to PASS [--last <n>] [--json]
4468
4512
  escalation Compute the minimum Judge model tier from deterministic risk signals
@@ -4687,6 +4731,12 @@ switch (command) {
4687
4731
  case 'recap':
4688
4732
  handleRecap();
4689
4733
  break;
4734
+ case 'safety':
4735
+ handleSafety();
4736
+ break;
4737
+ case 'measurement':
4738
+ handleMeasurement();
4739
+ break;
4690
4740
  case 'stats':
4691
4741
  handleStats();
4692
4742
  break;
@@ -26,6 +26,7 @@ Additional outcomes are `FAILED`, `SKIPPED`, and `BLOCKED`.
26
26
  - Structured conditions read only allowlisted run metadata.
27
27
  - Legacy gates execute atomically under the workflow lock and mark their corresponding node passed only after existing artifact validation succeeds.
28
28
  - Judge `ITERATE` resets Maker and its descendants while preserving upstream analysis.
29
+ - On `ITERATE` the CLI moves that iteration's `verification.json`, `verification-summary.json` and `final-verification.json` to `runs/<id>/iterations/<n>/` and clears the gated-verification marker, so `verify` records fresh evidence for the next Maker diff. Within one iteration, gated evidence stays immutable.
29
30
  - Judge `ESCALATE` blocks unfinished nodes.
30
31
 
31
32
  ## Commands
@@ -162,6 +162,10 @@ For UI or multi-step work, add the flow the user should experience as transition
162
162
 
163
163
  `goal draft` returns it as a `From | Event | To` table and a Mermaid `stateDiagram-v2` (`flow.mermaid`), with warnings (a state nothing reaches, a state with no way out that is not Done/Error), so the human checks the intent before freeze. A malformed `flow` fails the Goal gate. The PR receipt shows the same diagram (GitHub renders it) and table under "Intended flow", after "Verification artifacts" and before the acceptance criteria.
164
164
 
165
+ ## Performance ACs (C-33)
166
+
167
+ Mark an AC that claims a number (faster, smaller, more throughput) with `"measure": true`. Judge PASS then needs `measurement record` for it with at least 3 runs, a limiter and spread below the effect (`docs verification-loop`, section 3e).
168
+
165
169
  ## Look before you freeze: diagrams (C-27)
166
170
 
167
171
  Steps may carry more than `from`, `event`, `to`, so the human can see what else a change touches before freezing:
@@ -23,6 +23,10 @@ MAX_ALLOWED_CEILING = 5
23
23
 
24
24
  ---
25
25
 
26
+ ### Attack the premise (C-32)
27
+
28
+ When the Judge issues ITERATE with an explicit `cause` that repeats the previous ITERATE's cause, verdict.json must carry `premise: {statement, test}`: the one sentence both failed fixes assumed, and the observation that would differ if it were false, with what it showed. Without it `gate judge` refuses (`PREMISE_REQUIRED`). Inferred causes do not trigger this.
29
+
26
30
  ## 3. The Autonomous Iteration Lifecycle
27
31
 
28
32
  ```mermaid
@@ -116,6 +116,31 @@ Detection follows the file's comment syntax: `//`, `/* */` and `*` lines in C-fa
116
116
 
117
117
  A reviewer can downgrade a finding (for example public API docs that a doc generator reads) only with `dismissal: {by, evidence}`. A project that needs comments (a library shipping JSDoc, for example) opts out for every Run by having a human add `"comments": "allow"` to `.ai-engineering-loop/rules.json`; C-18 findings still apply.
118
118
 
119
+ ## 3e. Proof, Not Prose (C-29 to C-35)
120
+
121
+ Adapted from pstack (`interrogate`, `blast-radius`, `correct`, `principle-explain-the-number`; MIT, poteto). Each check moves a claim from the agent's word to something the CLI runs. No extra subagents: agents write the command, the CLI runs it.
122
+
123
+ **Findings must reproduce (C-29).** A VALID BLOCKER or HIGH finding may carry `proof: {command}`: a test or script that exits non-zero while the bug is present. At `gate review` the CLI runs every such proof in the repository (120 s timeout; destructive commands are refused by the guard) and sets `proofStatus`:
124
+
125
+ | proofStatus | Meaning | Blocks? |
126
+ |---|---|---|
127
+ | `reproduced` | the command failed: the bug shows | yes |
128
+ | `not_reproduced` | the command passed: the bug did not show | **no** |
129
+ | `refused` | empty or destructive command | yes |
130
+ | `timed_out` | no answer within the timeout | yes |
131
+
132
+ Results go to `runs/<id>/proofs.json`. Only the CLI writes `proofStatus`; a value written by hand is cleared. A finding without `proof` behaves as before.
133
+
134
+ **One safety fact (C-30).** A risky change is usually safe because of one fact. Record it with `ael safety record --run <id> --fact "<fact>" --cmd "<script or test that passes only if the fact holds>"`. A passing command records level 4 (5 with `--live`, when it ran against the running app); without `--cmd`, `--level 1|2|3` records stated, pointed at the line, or walked the failure path. A T2 Judge PASS needs level 4 or more (`SAFETY_FACT_UNPROVEN`).
135
+
136
+ **Rules must catch a real past mistake (C-31).** `learn promote ... --prove <commit>` runs the new check against the commit that introduced the mistake (its added lines for `--scope diff` and forbid rules; its whole tree, in a temporary worktree, for repo rules). The check must fail there, or the promotion is refused (`RULE_NOT_PROVEN`). The rule records `provenBy`; `doctor --harness` lists rules without it.
137
+
138
+ **Explain the number (C-33).** An AC with `"measure": true` (a speedup, a latency, a throughput) needs `ael measurement record --run <id> --ac AC-n --runs <n> --spread <pct> --effect <pct> --limiter "<what bounds the result>"`. Judge PASS needs at least 3 runs, a limiter, and run-to-run spread smaller than the effect (`MEASUREMENT_UNEXPLAINED`).
139
+
140
+ **Drift (C-34, C-35).** `doctor --harness` warns when a feature-map path matches no tracked file. `- **formal_covers**: src/payments/**` in `verification.md` makes default `verify` add the `formal` key whenever the gated diff touches covered files.
141
+
142
+ **Measure it (C-36).** `stats` reports human interventions per Run (answered questions, escalations, problems recorded after delivery) and how many finding proofs reproduced; `recap` shows interventions per Run for me and the lead.
143
+
119
144
  ## 4. The 4 Verification Gates
120
145
 
121
146
  1. **Gate 1: Unit & Integration Tests**: All unit and edge-case tests must pass with 0 failures.
@@ -0,0 +1,19 @@
1
+ <svg width="960" height="380" viewBox="0 0 960 380" fill="none" xmlns="http://www.w3.org/2000/svg">
2
+ <rect width="960" height="380" rx="12" fill="#FDFDFB"/>
3
+ <rect x="1" y="1" width="958" height="378" rx="11" stroke="#D9D9D5" stroke-width="2"/>
4
+ <text x="48" y="78" fill="#777777" font-family="monospace" font-size="15" letter-spacing="1.5">AI ENGINEERING LOOP</text>
5
+ <text x="48" y="148" fill="#111111" font-family="monospace" font-size="48" letter-spacing="-3">From idea to</text>
6
+ <text x="48" y="204" fill="#FC6404" font-family="monospace" font-size="48" letter-spacing="-3">trusted code.</text>
7
+ <rect x="48" y="258" width="430" height="56" stroke="#111111" stroke-width="2"/>
8
+ <text x="72" y="294" fill="#FC6404" font-family="monospace" font-size="20">$</text>
9
+ <text x="100" y="294" fill="#111111" font-family="monospace" font-size="19">npx ai-engineering-loop init</text>
10
+ <g>
11
+ <circle cx="748" cy="188" r="100" fill="#FC6404">
12
+ <animate attributeName="r" values="100;96;100" dur="3.2s" repeatCount="indefinite"/>
13
+ </circle>
14
+ <ellipse cx="712" cy="190" rx="11" ry="24" fill="#111111" transform="rotate(-8 712 190)"><animate attributeName="opacity" values="1;1;0.12;1" dur="4.8s" repeatCount="indefinite"/></ellipse>
15
+ <ellipse cx="782" cy="190" rx="11" ry="24" fill="#111111" transform="rotate(8 782 190)"><animate attributeName="opacity" values="1;1;0.12;1" dur="4.8s" repeatCount="indefinite"/></ellipse>
16
+ <path d="M602 298H894" stroke="#111111" stroke-width="2"/>
17
+ <ellipse cx="748" cy="304" rx="78" ry="8" fill="#111111" fill-opacity="0.14"/>
18
+ </g>
19
+ </svg>
Binary file
Binary file