ai-engineering-loop 2.5.0 → 2.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,20 +1,28 @@
1
1
  # Stage 5: Verification
2
2
 
3
- - `npx ai-engineering-loop verify --run <runId> [--cmd "<command>"]...` (default `test_unit` from verification.md) records commands in parallel (`--serial`, `--shell`), binds redacted evidence to the gated diff, and only when all pass gates verification, compacts, and scaffolds claimed-vs-reality.
4
- - Iterating (T1/T2): `verify --related` runs `test_related` on the diff's changed files. After Judge PASS, `verify --final` runs the full suite once; delivery requires it after related-only runs. Unchanged diff: passes are reused (`--fresh` re-runs).
5
- - A failed command is recorded honestly and returns non-zero; it cannot pass the gate. Vague "seems green" is invalid.
6
- - Fill `.ai-engineering-loop/tasks/claimed-vs-reality.md`: Claimed from the actual change, Reality from command output. Do not spawn DA while the file is missing or any Claimed row lacks Reality.
7
- - Hand-authored artifacts: `scaffold verification|findings|delivery --run <runId>`, fill it, remove `_templateOnly`, then `validate <file>`. Templates are never evidence.
3
+ - `npx ai-engineering-loop verify --run <runId> [--cmd "<command>"]...` (default `test_unit` from verification.md) runs commands in parallel (`--serial`, `--shell`), binds redacted evidence to the gated diff, and on pass compacts and scaffolds claimed-vs-reality.
4
+ - Iterating (T1/T2): `verify --related` runs `test_related` on changed files. After Judge PASS, `verify --final` runs the full suite once before delivery. Unchanged diff reuses passes (`--fresh` re-runs).
5
+ - Failed commands return non-zero and block the gate. Vague "seems green" is invalid.
6
+ - Fill `.ai-engineering-loop/tasks/claimed-vs-reality.md`: Claimed from change, Reality from command output, optional Layer (`L1`, `L2`, `L3`). Multi-component ACs need L3 evidence or justification if `e2e` is unconfigured. Never spawn DA if file is missing or any row lacks Reality.
7
+ - Hand-authored artifacts: `scaffold verification|findings|delivery --run <runId>`, fill, remove `_templateOnly`, then `validate <file>`.
8
8
  - Pass run-summary.md and verification-summary.json to reviewers instead of raw logs.
9
9
 
10
+ ## Verification Layers & Command Mapping
11
+
12
+ | Layer | Scope | Command (`verification.md`) | Focus |
13
+ |---|---|---|---|
14
+ | **L1** | Static | `typecheck`, `lint`, `build` | Types, lint, build |
15
+ | **L2** | Runtime | `test_unit`, `test_related`, `test_all` | Unit and regression tests |
16
+ | **L3** | End-to-End | `e2e` (optional) | Integrated boundary proof |
17
+
10
18
  ## No claim without fresh evidence
11
19
 
12
- Before saying "done", "fixed", "passing", or "ready", run the proving command in this iteration and cite its output; earlier runs and "should work" are not evidence.
20
+ Before claiming "done", "fixed", "passing", or "ready", run the proving command this iteration; earlier runs and "should work" are not evidence.
13
21
 
14
22
  | Claim | Required evidence |
15
23
  |---|---|
16
- | Tests pass | the test command's output this iteration, exit 0 |
17
- | Bug fixed | the red repro now passes, with the command shown |
18
- | AC met | its claimed-vs-reality row cites that output |
24
+ | Tests pass | test output this iteration, exit 0 |
25
+ | Bug fixed | passing red repro, command shown |
26
+ | AC met | claimed-vs-reality cites output |
19
27
 
20
- After `verify`, continue from run-summary.md and verification-summary.json, not earlier logs or chat.
28
+ After `verify`, continue from run-summary.md and verification-summary.json.
package/CHANGELOG.md CHANGED
@@ -6,6 +6,12 @@ All notable changes to this project are documented here. Versions follow [Semant
6
6
 
7
7
  _No changes yet._
8
8
 
9
+ ## 2.6.0 - 2026-11-10
10
+
11
+ - **Added:** three-layer verification naming in `loop-verify` (#110): L1 unit, L2 related/integration, L3 end-to-end through the existing optional `e2e` verification key, with a manual flow check plus written evidence when `e2e` is not configured. T0 needs L1-L2, T1 adds L3 when an AC crosses component boundaries, T2 always needs L1-L3.
12
+ - **Added:** `context claimed-vs-reality` scaffolds a 4-column table (`AC | Claimed | Reality from command log | Layer`). Existing 3-column files stay valid for `readyForDa`; empty or "seems green" Reality still fails.
13
+ - **Fixed:** the measure script counts characters the same on Windows (CRLF) as on macOS and Linux.
14
+
9
15
  ## 2.5.0 - 2026-11-02
10
16
 
11
17
  - **Changed:** the core host skill is now `ael` (#107), so the slash command is `/ael` on every host (`~/.claude/skills/ael`, `~/.agents/skills/ael`, and each host's own folder). `sync-hosts` and `update` remove the old `ai-engineering-loop` skill files AEL wrote, so the command no longer appears twice; files you edited or added there are kept, and `hosts doctor` reports the leftover folder. Skill sources moved to `skill-src/ael/`.
@@ -43,9 +43,21 @@ A verification `PASS` is **strictly invalid** without concrete execution evidenc
43
43
  }
44
44
  ```
45
45
 
46
+ ## 3. Verification Layers & Command Mapping
47
+
48
+ Verification terminates through three explicit layers of evidence. Unit tests alone have systematic blind spots for cross-component and integration defects. An acceptance criterion crossing component boundaries requires Layer 3 verification or written justification if Layer 3 is unavailable.
49
+
50
+ | Layer | Scope | Configured Command (`verification.md`) | Purpose |
51
+ |---|---|---|---|
52
+ | **L1: Static / Syntax** | Syntax, static types, style, compilation | `typecheck`, `lint`, `build` | Fast static catch before runtime. Rejects type errors, lint regressions, and compilation breakage. |
53
+ | **L2: Behavioral / Runtime** | Functions, units, modules, regressions | `test_unit`, `test_related`, `test_all` | Fast behavioral feedback at named seams and regression verification. |
54
+ | **L3: End-to-End** | Cross-component flows, integrations, browser/API | `e2e` (optional) | Realistic integrated proof across boundary seams. |
55
+
56
+ When `e2e` is configured in `verification.md`, cross-component acceptance criteria must cite L3 command evidence. If `e2e` is empty, Maker must document in Reality why L3 is not configured.
57
+
46
58
  ---
47
59
 
48
- ## 3. The 4 Verification Gates
60
+ ## 4. The 4 Verification Gates
49
61
 
50
62
  1. **Gate 1: Unit & Integration Tests**: All unit and edge-case tests must pass with 0 failures.
51
63
  2. **Gate 2: Static Type Analysis**: `tsc --noEmit` or compiler type checking must exit with code `0` (zero type errors).
@@ -54,7 +66,7 @@ A verification `PASS` is **strictly invalid** without concrete execution evidenc
54
66
 
55
67
  ---
56
68
 
57
- ## 4. Rejection Triggers
69
+ ## 5. Rejection Triggers
58
70
 
59
71
  The Verification Gate immediately halts and returns to the Maker if:
60
72
  - Any test suite fails or times out.
@@ -66,7 +78,7 @@ The Verification Gate immediately halts and returns to the Maker if:
66
78
 
67
79
  ---
68
80
 
69
- ## 5. Claimed vs Reality (before Devil's Advocate)
81
+ ## 6. Claimed vs Reality (before Devil's Advocate)
70
82
 
71
83
  Write `.ai-engineering-loop/tasks/claimed-vs-reality.md` after verification logs exist:
72
84
 
@@ -74,14 +86,14 @@ Write `.ai-engineering-loop/tasks/claimed-vs-reality.md` after verification logs
74
86
  npx ai-engineering-loop context claimed-vs-reality --run <run-id>
75
87
  ```
76
88
 
77
- This scaffolds one row per frozen acceptance criterion but leaves Claimed and Reality empty. Fill them from the implemented change and actual command output; the scaffold is not verification evidence.
89
+ This scaffolds one row per frozen acceptance criterion but leaves Claimed, Reality, and Layer empty. Fill them from the implemented change and actual command output; the scaffold is not verification evidence.
78
90
 
79
91
  ```markdown
80
92
  # Claimed vs Reality
81
93
 
82
- | AC | Claimed | Reality |
83
- |---|---|---|
84
- | AC-1 | <what Maker said is done> | <command, exit code, assertion or log line> |
94
+ | AC | Claimed | Reality from command log | Layer |
95
+ |---|---|---|---|
96
+ | AC-1 | <what Maker said is done> | <command, exit code, assertion or log line> | L2 |
85
97
  ```
86
98
 
87
99
  Do **not** spawn Devil's Advocate if:
@@ -76,8 +76,8 @@ function renderClaimedVsRealityTemplate(goal) {
76
76
  '',
77
77
  '> Generated from the frozen Goal Contract. Fill Claimed and Reality only after implementation and verification. Reality must cite observed command output; this template is not verification evidence.',
78
78
  '',
79
- '| AC | Claimed | Reality from command log |',
80
- '|---|---|---|'
79
+ '| AC | Claimed | Reality from command log | Layer |',
80
+ '|---|---|---|---|'
81
81
  ];
82
82
  for (const criterion of goal.acceptanceCriteria) {
83
83
  const evidence = markdownCell(criterion.evidenceRequired);
@@ -87,7 +87,7 @@ function renderClaimedVsRealityTemplate(goal) {
87
87
  evidence ? `Evidence: ${evidence}` : '',
88
88
  failures ? `Failure cases: ${failures}` : ''
89
89
  ].filter(Boolean).join(' — ');
90
- lines.push(`| ${markdownCell(description)} | | |`);
90
+ lines.push(`| ${markdownCell(description)} | | | |`);
91
91
  }
92
92
  return `${lines.join('\n')}\n`;
93
93
  }
@@ -143,11 +143,15 @@ function parseClaimedVsReality(markdown) {
143
143
  if (/^\|\s*-+\s*\|/.test(trimmed) || /^\|\s*AC\s*\|/i.test(trimmed)) continue;
144
144
  const cells = trimmed.split('|').map((c) => c.trim()).filter((_, i, arr) => i > 0 && i < arr.length - 1);
145
145
  if (cells.length < 3) continue;
146
- rows.push({
146
+ const row = {
147
147
  ac: cells[0],
148
148
  claimed: cells[1],
149
149
  reality: cells[2]
150
- });
150
+ };
151
+ if (cells.length >= 4 && cells[3]) {
152
+ row.layer = cells[3];
153
+ }
154
+ rows.push(row);
151
155
  }
152
156
  return { rows };
153
157
  }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "ai-engineering-loop",
3
- "version": "2.5.0",
3
+ "version": "2.6.0",
4
4
  "description": "A visual workspace that helps AI coding agents plan, build, review, and ship reliable software.",
5
5
  "main": "bin/ai-engineering-loop.js",
6
6
  "bin": {