ai-engineering-loop 1.0.12 → 1.0.14

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -28,7 +28,7 @@ Do not ask for Maker rationale.
28
28
 
29
29
  Report Spec and Standards as separate findings. Do not rerank one axis with the other. A change can pass Spec and fail Standards, or the reverse. Do not spawn children to split axes.
30
30
 
31
- **Spec** (`axis: "spec"`): Goal Contract acceptance criteria, correctness, error handling, security, concurrency, tests that fail to prove an AC. BLOCKER or HIGH only for a real AC breach or runtime defect.
31
+ **Spec** (`axis: "spec"`): Goal Contract acceptance criteria, correctness, error handling, security, concurrency, tests that fail to prove an AC, happy-path-only suite vs written failure table. BLOCKER or HIGH only for a real AC breach or runtime defect.
32
32
 
33
33
  **Standards** (`axis: "standards"`): `.ai-engineering-loop/conventions.md` plus the smell baseline below. Judgement calls: severity MEDIUM or LOW. Set `hardConvention: true` only when conventions.md states a hard rule that this hunk violates. A smell that hides an AC defect is Spec, not Standards.
34
34
 
package/.agents/judge.md CHANGED
@@ -14,6 +14,10 @@ Finish in at most 4 tool calls, then emit the verdict. Read the Finding Ledger a
14
14
 
15
15
  Use only paths in the spawn prompt: Goal Contract, verification evidence, Finding Ledger. Ignore Maker optimism and reviewer tone. Disposition never overrides Validity plus Severity.
16
16
 
17
+ AC source is the Goal Contract **file** only. Chat `setuju` and the parent transcript are not AC. Do not PASS because the parent agreed in chat. `CONTEXT_ISOLATION_ONLY` does not relax this.
18
+
19
+ ITERATE if verification is missing, vague, non-zero, proves a different artifact than the AC names, is only a source grep, or is happy-path-only while the Goal Contract lists empty, boundary, sibling, or error rows.
20
+
17
21
  ## Axes
18
22
 
19
23
  Do not merge Spec and Standards into one ranking.
@@ -5,7 +5,7 @@ description: Run the AI Engineering Loop on Antigravity (init, status, refresh,
5
5
 
6
6
  Follow `policies/review-budget.md`. Parent is Maker plus orchestrator. Do not use `browser_subagent`.
7
7
 
8
- Stage 1: grill if the task is ambiguous (`core/grill-policy.md`), then freeze the Goal Contract. On `adapter_type: dot`, that grill includes `task-impact-inquiry` (Antigravity skill under `~/.gemini/config/skills/`); do not run a second interview. Use `.ai-engineering-loop/glossary.md`. Name test seams. Maker: TDD at those seams (`policies/tdd-policy.md`). Bugs: red repro first (`core/root-cause-analysis.md`). Mid-loop stop: `core/handoff-policy.md`.
8
+ Stage 1: if the user asks for ideas, list a short menu and wait; do not implement. Grill if the task is ambiguous (`core/grill-policy.md`), then freeze the Goal Contract file. Chat agreement is not freeze. On `adapter_type: dot`, that grill includes `task-impact-inquiry` (Antigravity skill under `~/.gemini/config/skills/`); do not run a second interview. Use `.ai-engineering-loop/glossary.md`. Name test seams. AC is a failure table (happy, empty/omit, boundary, sibling, error). One red test per AC row. Do not freeze sunny-path-only. Maker: TDD at those seams (`policies/tdd-policy.md`): one red test per AC row. Bugs: red repro first (`core/root-cause-analysis.md`). Mid-loop stop: `core/handoff-policy.md`.
9
9
 
10
10
  If `invoke_subagent` (or Task) exists, spawn `devil-advocate` then `judge` as siblings. Wait for each child. Do not run them in the background. Use `general-purpose` only if the named type is rejected.
11
11
 
@@ -29,7 +29,7 @@ Do not ask for Maker rationale.
29
29
 
30
30
  Report Spec and Standards as separate findings. Do not rerank one axis with the other. A change can pass Spec and fail Standards, or the reverse. Do not spawn children to split axes.
31
31
 
32
- **Spec** (`axis: "spec"`): Goal Contract acceptance criteria, correctness, error handling, security, concurrency, tests that fail to prove an AC. BLOCKER or HIGH only for a real AC breach or runtime defect.
32
+ **Spec** (`axis: "spec"`): Goal Contract acceptance criteria, correctness, error handling, security, concurrency, tests that fail to prove an AC, happy-path-only suite vs written failure table. BLOCKER or HIGH only for a real AC breach or runtime defect.
33
33
 
34
34
  **Standards** (`axis: "standards"`): `.ai-engineering-loop/conventions.md` plus the smell baseline below. Judgement calls: severity MEDIUM or LOW. Set `hardConvention: true` only when conventions.md states a hard rule that this hunk violates. A smell that hides an AC defect is Spec, not Standards.
35
35
 
@@ -15,6 +15,10 @@ Finish in at most 4 tool calls, then emit the verdict. Read the Finding Ledger a
15
15
 
16
16
  Use only paths in the spawn prompt: Goal Contract, verification evidence, Finding Ledger. Ignore Maker optimism and reviewer tone. Disposition never overrides Validity plus Severity.
17
17
 
18
+ AC source is the Goal Contract **file** only. Chat `setuju` and the parent transcript are not AC. Do not PASS because the parent agreed in chat. `CONTEXT_ISOLATION_ONLY` does not relax this.
19
+
20
+ ITERATE if verification is missing, vague, non-zero, proves a different artifact than the AC names, is only a source grep, or is happy-path-only while the Goal Contract lists empty, boundary, sibling, or error rows.
21
+
18
22
  ## Axes
19
23
 
20
24
  Do not merge Spec and Standards into one ranking.
@@ -32,8 +32,8 @@ Claude Code talks to strict proxies (including Kiro). Follow this exactly:
32
32
  Parent session is Maker plus orchestrator. Spawn Devil's Advocate and Judge as **siblings**, not nested.
33
33
 
34
34
  1. Stage 0: `npx ai-engineering-loop sync-hosts` then `npx ai-engineering-loop status` (init or refresh if missing or stale). Read `.ai-engineering-loop/glossary.md`. If sync-hosts copied files, tell the user a new session is needed for updated skill text; keep going with this session.
35
- 2. Stage 1: Goal Contract (`core/goal-contract.md`). If the task is ambiguous and the user can answer, grill first (`core/grill-policy.md`): design tree, recommended answers, do not ask look-up facts. Skip grill if the contract is already frozen or the user waived it. On `adapter_type: dot`, that grill includes the four-pillar blast radius (state, sibling, approval, queues). Do not run a second interview. Freeze before any production edit. Name test seams. Use glossary terms.
36
- 3. Stages 2-4: Maker in the parent. Bugs: red repro first (`core/root-cause-analysis.md`). Features: TDD at named seams (`policies/tdd-policy.md`). Surgical diff.
35
+ 2. Stage 1: Goal Contract (`core/goal-contract.md`). If the user asks for ideas, list a short menu and wait; do not implement. If the task is ambiguous and the user can answer, grill first (`core/grill-policy.md`): design tree, recommended answers, do not ask look-up facts. Skip grill if the contract is already frozen or the user waived it. On `adapter_type: dot`, that grill includes the four-pillar blast radius (state, sibling, approval, queues). Do not run a second interview. Chat agreement is not freeze: every user-visible decision must be a numbered AC in the Goal Contract file. Freeze before any production edit. Name test seams. Use glossary terms. AC is a failure table (happy, empty/omit, boundary, sibling, error). One red test per AC row. Do not freeze sunny-path-only.
36
+ 3. Stages 2-4: Maker in the parent. Bugs: red repro first (`core/root-cause-analysis.md`). Features: TDD at named seams (`policies/tdd-policy.md`): one red test per AC row. Surgical diff.
37
37
  4. Stage 5: run commands from `.ai-engineering-loop/verification.md`. Keep command, exit code, stdout, test counts. Vague "seems green" is invalid.
38
38
  5. Write artifacts, then spawn. Before Devil's Advocate:
39
39
  - Write `git diff` to a file (for example `.ai-engineering-loop/tasks/current.diff`).
@@ -25,8 +25,8 @@ Canonical mode ids: `TRUE_INDEPENDENT_AGENT`, `ISOLATED_AGENT_INSTANCE`, `FRESH_
25
25
  ## Loop
26
26
 
27
27
  1. Stage 0: `npx ai-engineering-loop sync-hosts` then `npx ai-engineering-loop status` (init or refresh if missing or stale). Read `.ai-engineering-loop/glossary.md`. If sync-hosts copied files, tell the user a new session is needed for updated skill text; keep going with this session.
28
- 2. Stage 1: Goal Contract (`core/goal-contract.md`). If the task is ambiguous and the user can answer, grill first (`core/grill-policy.md`): design tree, recommended answers, do not ask look-up facts. Skip grill if the contract is already frozen or the user waived it. On `adapter_type: dot`, that grill includes `task-impact-inquiry` (this host: `~/.gemini/config/skills/task-impact-inquiry`); do not run a second interview. Freeze before any production edit. Name test seams. Use glossary terms.
29
- 3. Stages 2-4: Maker in the parent. Bugs: red repro first (`core/root-cause-analysis.md`). Features: TDD at named seams (`policies/tdd-policy.md`). Surgical diff.
28
+ 2. Stage 1: Goal Contract (`core/goal-contract.md`). If the user asks for ideas, list a short menu and wait; do not implement. If the task is ambiguous and the user can answer, grill first (`core/grill-policy.md`): design tree, recommended answers, do not ask look-up facts. Skip grill if the contract is already frozen or the user waived it. On `adapter_type: dot`, that grill includes `task-impact-inquiry` (this host: `~/.gemini/config/skills/task-impact-inquiry`); do not run a second interview. Chat agreement is not freeze: every user-visible decision must be a numbered AC in the Goal Contract file. Freeze before any production edit. Name test seams. Use glossary terms. AC is a failure table (happy, empty/omit, boundary, sibling, error). One red test per AC row. Do not freeze sunny-path-only.
29
+ 3. Stages 2-4: Maker in the parent. Bugs: red repro first (`core/root-cause-analysis.md`). Features: TDD at named seams (`policies/tdd-policy.md`): one red test per AC row. Surgical diff.
30
30
  4. Stage 5: run commands from `.ai-engineering-loop/verification.md`. Keep command, exit code, stdout, test counts. Vague "seems green" is invalid.
31
31
  5. Write artifacts, then spawn. Before Devil's Advocate:
32
32
  - Write `git diff` to a file (for example `.ai-engineering-loop/tasks/current.diff`).
@@ -30,7 +30,7 @@ Do not ask for Maker rationale.
30
30
 
31
31
  Report Spec and Standards as separate findings. Do not rerank one axis with the other. A change can pass Spec and fail Standards, or the reverse. Do not spawn children to split axes.
32
32
 
33
- **Spec** (`axis: "spec"`): Goal Contract acceptance criteria, correctness, error handling, security, concurrency, tests that fail to prove an AC. BLOCKER or HIGH only for a real AC breach or runtime defect.
33
+ **Spec** (`axis: "spec"`): Goal Contract acceptance criteria, correctness, error handling, security, concurrency, tests that fail to prove an AC, happy-path-only suite vs written failure table. BLOCKER or HIGH only for a real AC breach or runtime defect.
34
34
 
35
35
  **Standards** (`axis: "standards"`): `.ai-engineering-loop/conventions.md` plus the smell baseline below. Judgement calls: severity MEDIUM or LOW. Set `hardConvention: true` only when conventions.md states a hard rule that this hunk violates. A smell that hides an AC defect is Spec, not Standards.
36
36
 
@@ -16,6 +16,10 @@ Finish in at most 4 tool calls, then emit the verdict. Read the Finding Ledger a
16
16
 
17
17
  Use only paths in the spawn prompt: Goal Contract, verification evidence, Finding Ledger. Ignore Maker optimism and reviewer tone. Disposition never overrides Validity plus Severity.
18
18
 
19
+ AC source is the Goal Contract **file** only. Chat `setuju` and the parent transcript are not AC. Do not PASS because the parent agreed in chat. `CONTEXT_ISOLATION_ONLY` does not relax this.
20
+
21
+ ITERATE if verification is missing, vague, non-zero, proves a different artifact than the AC names, is only a source grep, or is happy-path-only while the Goal Contract lists empty, boundary, sibling, or error rows.
22
+
19
23
  ## Axes
20
24
 
21
25
  Do not merge Spec and Standards into one ranking.
@@ -45,8 +45,8 @@ Run `npx ai-engineering-loop <command>` in the target repo. Do not commit unless
45
45
  ### `/ai-engineering-loop [task]`
46
46
 
47
47
  1. Stage 0: `npx ai-engineering-loop sync-hosts` then `npx ai-engineering-loop status` (init/refresh if missing or stale). Read `.ai-engineering-loop/glossary.md`. If sync-hosts copied files, tell the user a new session is needed for updated skill text; keep going with this session.
48
- 2. Stage 1: Goal Contract (`core/goal-contract.md`). If the task is ambiguous and the user can answer, grill first (`core/grill-policy.md`): design tree, recommended answers, do not ask look-up facts. Skip grill if the contract is already frozen or the user waived it. On `adapter_type: dot`, that grill includes the four-pillar blast radius (state, sibling, approval, queues). Do not run a second interview. Freeze before any production edit. Name test seams. Use glossary terms.
49
- 3. Stages 2–4: Maker work in the **parent**. Bugs: red repro first (`core/root-cause-analysis.md`). Features: TDD at named seams (`policies/tdd-policy.md`). Surgical diff. Parent may be the Maker; do not spawn Maker as a child if you still need to spawn DA/Judge afterward from the same parent.
48
+ 2. Stage 1: Goal Contract (`core/goal-contract.md`). If the user asks for ideas, list a short menu and wait; do not implement. If the task is ambiguous and the user can answer, grill first (`core/grill-policy.md`): design tree, recommended answers, do not ask look-up facts. Skip grill if the contract is already frozen or the user waived it. On `adapter_type: dot`, that grill includes the four-pillar blast radius (state, sibling, approval, queues). Do not run a second interview. Chat agreement is not freeze: every user-visible decision must be a numbered AC in the Goal Contract file. Freeze before any production edit. Name test seams. Use glossary terms. AC is a failure table (happy, empty/omit, boundary, sibling, error). One red test per AC row. Do not freeze sunny-path-only.
49
+ 3. Stages 2–4: Maker work in the **parent**. Bugs: red repro first (`core/root-cause-analysis.md`). Features: TDD at named seams (`policies/tdd-policy.md`): one red test per AC row. Surgical diff. Parent may be the Maker; do not spawn Maker as a child if you still need to spawn DA/Judge afterward from the same parent.
50
50
  4. Stage 5: run verification from `.ai-engineering-loop/verification.md`. Keep command, exit code, stdout, test counts. Vague "seems green" is invalid.
51
51
  5. Write artifacts to disk so children do not need parent chat:
52
52
  - Goal Contract path
package/README.md CHANGED
@@ -6,7 +6,7 @@
6
6
  [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
7
7
  [![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](https://github.com/egagofur/ai-engineering-loop/pulls)
8
8
  [![AI Engineering](https://img.shields.io/badge/AI-Engineering%20Loop-orange.svg)](https://github.com/egagofur/ai-engineering-loop)
9
- [![Release](https://img.shields.io/badge/release-v1.0.12-purple.svg)](https://github.com/egagofur/ai-engineering-loop/releases)
9
+ [![Release](https://img.shields.io/badge/release-v1.0.14-purple.svg)](https://github.com/egagofur/ai-engineering-loop/releases)
10
10
 
11
11
  **A Reusable, Framework-Agnostic AI Engineering Operating System for Autonomous Coding Agents**
12
12
 
@@ -64,10 +64,10 @@ The 8-stage loop stays one OS. These techniques sit **inside** existing stages (
64
64
 
65
65
  | Stage | Technique | Spec |
66
66
  |---|---|---|
67
- | 1 | Grill until the design-tree frontier is empty, then freeze the Goal Contract. Skip if already frozen, waived, or headless with testable AC. | `core/grill-policy.md` |
67
+ | 1 | Grill until the design-tree frontier is empty, then freeze the Goal Contract. Idea requests: menu, wait, then grill the pick. Chat `setuju` is not freeze; user-visible decisions must be numbered AC in the contract file. AC is a **failure table** (happy, empty/omit, boundary, sibling, error), not sunny path only. | `core/grill-policy.md` |
68
68
  | 1 | Ubiquitous language in `.ai-engineering-loop/glossary.md`; load-bearing choices in `.ai-engineering-loop/adrs/` | `core/repo-config-schema.md` |
69
69
  | 2 | Bugs: red repro → minimise → hypothesise → instrument → fix | `core/root-cause-analysis.md` |
70
- | 4–5 | Red-green at named **seams**; no implementation-coupled or tautological tests | `policies/tdd-policy.md` |
70
+ | 4–5 | Red-green at named **seams**; failure table (not happy path only); coverage is a map; no grep/tautology | `policies/tdd-policy.md` |
71
71
  | 6–7 | Spec vs Standards reported separately. Standards BLOCKER/HIGH iterate only when `hardConvention` is true | `policies/finding-policy.md` |
72
72
  | any | Mid-loop stop writes `.ai-engineering-loop/tasks/handoff.md` | `core/handoff-policy.md` |
73
73
 
package/agents/judge.md CHANGED
@@ -49,7 +49,8 @@ flowchart LR
49
49
 
50
50
  ### Step 2: Goal Contract Compliance Audit
51
51
  - Cross-reference every Acceptance Criterion (AC-1 through AC-N) from the [Goal Contract](file:///Users/egagofur/Development/work/ai-engineering-loop/core/goal-contract.md).
52
- - Verify that automated tests exist that explicitly exercise and prove each criterion.
52
+ - Verify that automated tests exist that explicitly exercise and prove each criterion, including non-happy-path rows in the failure table.
53
+ - A happy-path-only suite while the contract lists empty, boundary, sibling, or error rows is `ITERATE`.
53
54
  - Verify that no out-of-scope files were touched and technical constraints were respected.
54
55
 
55
56
  ### Step 3: Finding Triage & Evidence Verification
package/agents/maker.md CHANGED
@@ -28,7 +28,7 @@ flowchart LR
28
28
  3. **Test-Driven Engineering** ([TDD Policy](file:///Users/egagofur/Development/work/ai-engineering-loop/policies/tdd-policy.md)):
29
29
  - Red before green at the Goal Contract's named seams only.
30
30
  - Tests observe public behavior, use glossary terms, and never assert implementation details.
31
- - Cover the AC slice in play: happy path, negative edge, null/empty, boundary. Vertical slices, not all-tests-then-all-code.
31
+ - Cover the AC slice in play: happy path, empty/omit, boundary, sibling/isolation, error/denied. One red test per AC row. Vertical slices, not all-tests-then-all-code.
32
32
  4. **Addressing Reviewer Findings**:
33
33
  - In subsequent iterations, ingest findings from the [Devil's Advocate](file:///Users/egagofur/Development/work/ai-engineering-loop/agents/devil-advocate.md) and directives from the [Judge](file:///Users/egagofur/Development/work/ai-engineering-loop/agents/judge.md).
34
34
  - Fix validated issues surgically.
@@ -22,7 +22,7 @@ Do not ask for Maker rationale.
22
22
 
23
23
  Report Spec and Standards as separate findings. Do not rerank one axis with the other. A change can pass Spec and fail Standards, or the reverse. Do not spawn children to split axes.
24
24
 
25
- **Spec** (`axis: "spec"`): Goal Contract acceptance criteria, correctness, error handling, security, concurrency, tests that fail to prove an AC. BLOCKER or HIGH only for a real AC breach or runtime defect.
25
+ **Spec** (`axis: "spec"`): Goal Contract acceptance criteria, correctness, error handling, security, concurrency, tests that fail to prove an AC, happy-path-only suite vs written failure table. BLOCKER or HIGH only for a real AC breach or runtime defect.
26
26
 
27
27
  **Standards** (`axis: "standards"`): `.ai-engineering-loop/conventions.md` plus the smell baseline below. Judgement calls: severity MEDIUM or LOW. Set `hardConvention: true` only when conventions.md states a hard rule that this hunk violates. A smell that hides an AC defect is Spec, not Standards.
28
28
 
@@ -8,6 +8,10 @@ Finish in at most 4 tool calls, then emit the verdict. Read the Finding Ledger a
8
8
 
9
9
  Use only paths in the spawn prompt: Goal Contract, verification evidence, Finding Ledger. Ignore Maker optimism and reviewer tone. Disposition never overrides Validity plus Severity.
10
10
 
11
+ AC source is the Goal Contract **file** only. Chat `setuju` and the parent transcript are not AC. Do not PASS because the parent agreed in chat. `CONTEXT_ISOLATION_ONLY` does not relax this.
12
+
13
+ ITERATE if verification is missing, vague, non-zero, proves a different artifact than the AC names, is only a source grep, or is happy-path-only while the Goal Contract lists empty, boundary, sibling, or error rows.
14
+
11
15
  ## Axes
12
16
 
13
17
  Do not merge Spec and Standards into one ranking.
@@ -21,7 +21,7 @@ const {
21
21
  formatHostSyncReport
22
22
  } = require('../lib/sync-hosts.js');
23
23
 
24
- const VERSION = '1.0.12';
24
+ const VERSION = '1.0.14';
25
25
  const CWD = process.cwd();
26
26
  const CONTEXT_DIR = path.join(CWD, '.ai-engineering-loop');
27
27
 
@@ -275,7 +275,7 @@ function generateContextFiles(rootDir, discovery, trigger = 'init', impact = 'IN
275
275
 
276
276
  // 0. metadata.json (Baseline)
277
277
  const metadataJson = {
278
- contextVersion: '1.0.12',
278
+ contextVersion: '1.0.14',
279
279
  generatedAt: new Date().toISOString(),
280
280
  repositoryRevision: currentRevision,
281
281
  projectProfile: discovery.profile,
@@ -29,9 +29,15 @@ Every Goal Contract MUST adhere to the following schema in Markdown or structure
29
29
  [Explain what changes for the real-world actor (e.g. Employee, Admin, Customer, System). Describe the before/after lifecycle state transition.]
30
30
 
31
31
  ## 3. Acceptance Criteria (AC)
32
- - [ ] AC-1: [Exact, testable statement with expected outcome]
33
- - [ ] AC-2: [Exact, testable statement with expected outcome]
34
- - [ ] AC-3: [Edge case or boundary behavior explicitly specified]
32
+ Each AC is one row that can **fail**. Happy path alone is not a contract (`policies/tdd-policy.md`).
33
+
34
+ | AC | Input / actor | Expected at seam | Must fail if missing |
35
+ |---|---|---|---|
36
+ | AC-1 | Happy path | [observable] | [what would still look green] |
37
+ | AC-2 | Empty / omitted field | [observable] | [what would still look green] |
38
+ | AC-3 | Boundary | [observable] | [what would still look green] |
39
+ | AC-4 | Sibling / isolation (if domain has siblings) | [observable] | [what would still look green] |
40
+ | AC-5 | Error / denied / locked | [observable] | [what would still look green] |
35
41
 
36
42
  ## 4. Technical Constraints
37
43
  - [Architecture]: [Preserve existing patterns, layer boundaries, dependency conventions]
@@ -77,6 +83,9 @@ Every Goal Contract MUST adhere to the following schema in Markdown or structure
77
83
  - Neither the Maker Agent nor the Devil's Advocate Agent may alter Acceptance Criteria during an iteration loop to make tests pass or bypass critique.
78
84
  3. **Contract Amendments**:
79
85
  - If during implementation a fundamental contradiction or impossible requirement is discovered, the agent must trigger **Human Escalation**. Only a human user may amend the Goal Contract.
86
+ 4. **Chat is not the contract**:
87
+ - Parent-transcript `setuju` does not freeze AC.
88
+ - Every user-visible grill decision (order of pages, enable flags, layout) MUST appear as AC-1..N in the contract file before Maker starts.
80
89
 
81
90
  ---
82
91
 
@@ -100,3 +109,6 @@ Every single item listed under `Acceptance Criteria` must map to at least one co
100
109
  - **The Parallel Glossary**: Using 20 words for a concept that already has a term in `glossary.md`.
101
110
  - **The Missing Constraint**: Failing to declare out-of-scope files, leading to arbitrary refactoring of adjacent legacy modules.
102
111
  - **The Self-Serving Goal**: Modifying acceptance criteria post-hoc when tests fail rather than fixing the underlying implementation.
112
+ - **The Chat Contract**: Treating "setuju" in the parent transcript as AC. The file is the contract.
113
+ - **The Wrong Artifact**: Tests pass on a `/tmp` build while the sample the user opens omits the feature.
114
+ - **The Happy-Path Contract**: Only the sunny case is an AC. Coverage % is not a substitute for a failure table.
@@ -21,6 +21,30 @@ Skip grill (write the Goal Contract from what is already known) when any of thes
21
21
 
22
22
  Do not interview the user for **facts** you can look up (files, scripts, types, git). Look them up. Grill only **decisions**.
23
23
 
24
+ ## Idea and menu requests
25
+
26
+ If the user asks for ideas, a feature catalog, or "what should we add" without naming one task:
27
+
28
+ 1. Do not implement.
29
+ 2. Do not freeze a Goal Contract.
30
+ 3. List at most five options. Mark one recommended. Wait for a pick.
31
+ 4. After they pick, grill that one task, then freeze.
32
+
33
+ `/ai-engineering-loop` on an idea request still stays this loop: menu, then grill. Do not switch to a parallel brainstorm product.
34
+
35
+ ## Freeze gate
36
+
37
+ Chat `setuju`, `lanjut`, `ok`, or "looks good" is **not** a freeze.
38
+
39
+ Before Maker starts:
40
+
41
+ 1. Every accepted grill decision that changes user-visible output is a numbered AC in the Goal Contract **file** (page order, enable flags, layout, copy, numbering).
42
+ 2. Each of those ACs names the artifact to inspect (sample path, command, seam).
43
+ 3. Write `.ai-engineering-loop/tasks/goal-contract.md` (or the path this repo uses). Show the AC list.
44
+ 4. Freeze only after that file exists and contains those ACs.
45
+
46
+ If a decision was agreed in chat and is missing from the file, the contract is not frozen. Do not start Maker.
47
+
24
48
  ## DOT adapter (`adapter_type: dot`)
25
49
 
26
50
  On DOT repositories, grill is **one** session that includes the four-pillar blast radius from Antigravity skill `task-impact-inquiry` (`~/.gemini/config/skills/task-impact-inquiry/SKILL.md`). Do not run that skill as a second interview after grill.
@@ -56,12 +80,14 @@ The grill is done when the frontier is empty: every branch visited, nothing sile
56
80
  ## What the grill must settle
57
81
 
58
82
  - Objective and business outcome
59
- - Acceptance criteria that can fail a test
83
+ - Acceptance criteria that can fail a test, as a **failure table** (not happy path only)
60
84
  - Out of scope
61
85
  - Test **seams** (public interfaces to observe; prefer existing seams; fewer is better)
62
86
  - Glossary terms to use (read and update `.ai-engineering-loop/glossary.md`)
63
87
  - Hard decisions that belong in an ADR under `.ai-engineering-loop/adrs/`
64
88
 
89
+ A Goal Contract with only the sunny path is not frozen. Include at least: happy path, empty/omitted input, one boundary, one isolation/sibling (when the domain has siblings), one error/denied path. DOT four-pillar rows count toward this table.
90
+
65
91
  ## Invariants
66
92
 
67
93
  - No production code edits during grill.
@@ -52,13 +52,19 @@ The Judge renders decisions based strictly on **Validity + Severity**, then **re
52
52
  3. Zero open `VALID + BLOCKER` or `VALID + HIGH` findings.
53
53
  4. Any `INVALID` findings are formally dismissed with counter-evidence.
54
54
  5. Any `VALID + MEDIUM/LOW` findings are documented as tradeoffs.
55
+ 6. AC source is **only** the Goal Contract file given to the Judge. Chat agreement is not an AC. `CONTEXT_ISOLATION_ONLY` does not relax this.
56
+ 7. Evidence is for the artifact the AC names (sample path, seam, verification command). A `/tmp` build is not proof of a different user-facing file.
57
+ 8. Source `grep` of implementation files is not AC proof (`policies/tdd-policy.md`).
55
58
 
56
59
  ---
57
60
 
58
61
  ### Verdict 2: `ITERATE`
59
62
  - **Conditions**:
60
- 1. One or more `VALID + BLOCKER` or `VALID + HIGH` findings exist.
61
- 2. Active iteration count $< \text{MAX\_ITERATIONS}$ (default 3).
63
+ 1. One or more `VALID + BLOCKER` or `VALID + HIGH` findings exist (Spec, or Standards with `hardConvention: true`).
64
+ 2. Active iteration count is below MAX_ITERATIONS (default 3).
65
+ 3. Maker or verification cites chat agreement for behavior that is not an AC in the Goal Contract file.
66
+ 4. Evidence is for the wrong artifact, is source-grep only, or does not map to written AC ids.
67
+ 5. The suite is happy-path-only while the Goal Contract lists empty, boundary, sibling, or error AC rows.
62
68
  - **Maker Directive**: The Maker must adopt the concrete alternative diff or provide an equivalent verified architectural resolution, authoring regression unit tests.
63
69
 
64
70
  ---
@@ -22,7 +22,7 @@ A verification `PASS` is **strictly invalid** without concrete execution evidenc
22
22
  5. **`stdout` & `stderr`**: Raw machine logs captured from execution.
23
23
  6. **`timeoutStatus`**: Must be `"COMPLETED"` (not timed out or backgrounded without completion).
24
24
  7. **`testCounts`**: Explicit counts of passed, failed, and skipped tests.
25
- 8. **`assertionEvidence`**: Specific assertion proof matching the active Goal Contract's Acceptance Criteria, observed at a named test seam.
25
+ 8. **`assertionEvidence`**: Specific assertion proof matching the active Goal Contract's Acceptance Criteria, observed at a named test seam. Must include a non-happy-path AC when the contract has a failure table. Coverage percent is not assertionEvidence.
26
26
 
27
27
  ```json
28
28
  {
@@ -61,3 +61,5 @@ The Verification Gate immediately halts and returns to the Maker if:
61
61
  - Exit code is non-zero.
62
62
  - The command was sent to the background and not verified to completion.
63
63
  - Test logs contain zero passing assertions for new acceptance criteria.
64
+ - assertionEvidence maps only to a happy path while the Goal Contract lists empty, boundary, sibling, or error AC rows.
65
+ - The only new "test" is a source grep or a coverage percentage.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "ai-engineering-loop",
3
- "version": "1.0.12",
3
+ "version": "1.0.14",
4
4
  "description": "A reusable, framework-agnostic AI Engineering Operating System for autonomous coding agents.",
5
5
  "main": "bin/ai-engineering-loop.js",
6
6
  "bin": {
@@ -14,19 +14,39 @@ No test is written at an unconfirmed seam. If the contract omitted seams and gri
14
14
 
15
15
  The test reads like a specification of behavior at the seam. Names use `.ai-engineering-loop/glossary.md`. The test survives an internal rewrite. Expected values come from the Goal Contract or a known-good literal, not from re-running the implementation.
16
16
 
17
+ ## Failure table
18
+
19
+ The Goal Contract's AC-1..N **are** the test list. Each row is one red test at the named seam. Do not invent a parallel "comprehensive suite" of only positive cases.
20
+
21
+ Default rows unless the contract marked them N/A with one sentence:
22
+
23
+ 1. Happy path
24
+ 2. Empty / omitted / null
25
+ 3. Boundary (min, max, off-by-one, locked vs open)
26
+ 4. Sibling / isolation (other entities in the same parent must not change)
27
+ 5. Error / denied / unauthorized / invalid input
28
+
29
+ Do not stop after the first green test. A suite that never asserts a failure mode does not cover the feature.
30
+
17
31
  ## Loop
18
32
 
19
- 1. **Red.** Write one failing test for one AC slice. Confirm it fails for the right reason.
33
+ 1. **Red.** Write one failing test for one AC row. Confirm it fails for the right reason.
20
34
  2. **Green.** Write the smallest production change that passes that test.
21
- 3. Repeat one slice at a time (vertical). Do not write the whole suite first.
22
- 4. After slices covering AC-1..N, run the full verification commands in `.ai-engineering-loop/verification.md` and keep the Evidence Contract.
35
+ 3. Repeat one row at a time (vertical). Do not write the whole suite first.
36
+ 4. After AC-1..N are green, run the full verification commands in `.ai-engineering-loop/verification.md` and keep the Evidence Contract.
37
+ 5. **Coverage as a map, not a score.** If a coverage command exists, use the report to find branches that map to a written AC and are still untested. Add tests only for those. Do not chase 90% with tautological asserts. Line coverage without a failure table is invalid evidence.
38
+ 6. **Mutation (optional).** If `.ai-engineering-loop/verification.md` names a mutation tool (Stryker, mutmut, PIT, cargo-mutants), run it on the files this task touched. Surviving mutants on an AC path are ITERATE. Do not add mutation as a new stage or a global 90% gate.
39
+ 7. **Property-based (optional).** When the input space is broad (ids, dates, strings, amounts), one property per invariant beats a pile of copied positives. Not required for every task.
23
40
 
24
41
  ## Forbidden tests
25
42
 
26
43
  - **Implementation-coupled:** mocks internal collaborators, tests private methods, or asserts through a side channel (raw DB) instead of the seam.
27
44
  - **Tautological:** expected value is computed the same way as the code (`expect(add(a,b)).toBe(a+b)`).
45
+ - **Source grep:** `grep -q 'featureFlag' src/file` is not a test of the AC. Prove behavior at the named seam or on the named artifact.
46
+ - **Happy-path only:** every test is a success case while the contract listed empty, boundary, sibling, or error rows.
47
+ - **Coverage theater:** raising percent with `toBeDefined()`, snapshot-without-oracle, or tests that cannot fail.
28
48
  - **Horizontal slicing:** all tests first, then all implementation.
29
49
 
30
50
  ## Evidence
31
51
 
32
- Stage 5 still requires command, exit code 0, stdout, test counts, and assertionEvidence mapped to AC ids. "We did TDD" is not evidence. A red-then-green story without logs is invalid.
52
+ Stage 5 still requires command, exit code 0, stdout, test counts, and assertionEvidence mapped to AC ids. "We did TDD" is not evidence. A red-then-green story without logs is invalid. assertionEvidence must cite at least one non-happy-path AC when the failure table has one.
@@ -206,6 +206,63 @@ test('Host skills Stage 0 runs sync-hosts before status', () => {
206
206
  }
207
207
  });
208
208
 
209
+ test('Grill freeze gate and idea menu are in the loop, not a parallel product', () => {
210
+ const grill = readRepo('core/grill-policy.md');
211
+ assert.match(grill, /Idea and menu requests/);
212
+ assert.match(grill, /Do not implement/);
213
+ assert.match(grill, /is \*\*not\*\* a freeze/);
214
+ assert.match(grill, /numbered AC/);
215
+ const contract = readRepo('core/goal-contract.md');
216
+ assert.match(contract, /Chat is not the contract/);
217
+ assert.match(contract, /The Chat Contract/);
218
+ assert.match(contract, /The Wrong Artifact/);
219
+ const judge = readRepo('agents/shared/judge.body.md');
220
+ assert.match(judge, /Goal Contract \*\*file\*\* only/);
221
+ assert.match(judge, /source grep/);
222
+ const tdd = readRepo('policies/tdd-policy.md');
223
+ assert.match(tdd, /Source grep/);
224
+ for (const rel of [
225
+ '.claude/skills/ai-engineering-loop/SKILL.md',
226
+ '.grok/skills/ai-engineering-loop/SKILL.md',
227
+ '.gemini/skills/ai-engineering-loop/SKILL.md',
228
+ '.agents/workflows/ai-engineering-loop.md'
229
+ ]) {
230
+ const text = readRepo(rel);
231
+ assert.match(text, /list a short menu and wait/, rel);
232
+ assert.match(text, /Chat agreement is not freeze/, rel);
233
+ }
234
+ });
235
+
236
+ test('Failure table is required to freeze, TDD, verify, and Judge', () => {
237
+ const grill = readRepo('core/grill-policy.md');
238
+ assert.match(grill, /failure table/);
239
+ assert.match(grill, /sunny path is not frozen/);
240
+ const contract = readRepo('core/goal-contract.md');
241
+ assert.match(contract, /The Happy-Path Contract/);
242
+ assert.match(contract, /Empty \/ omitted field/);
243
+ const tdd = readRepo('policies/tdd-policy.md');
244
+ assert.match(tdd, /Coverage as a map, not a score/);
245
+ assert.match(tdd, /Happy-path only/);
246
+ assert.match(tdd, /Coverage theater/);
247
+ const verif = readRepo('core/verification-loop.md');
248
+ assert.match(verif, /non-happy-path AC/);
249
+ const judge = readRepo('agents/shared/judge.body.md');
250
+ assert.match(judge, /happy-path-only/);
251
+ const da = readRepo('agents/shared/devil-advocate.body.md');
252
+ assert.match(da, /happy-path-only suite vs written failure table/);
253
+ for (const rel of [
254
+ '.claude/skills/ai-engineering-loop/SKILL.md',
255
+ '.grok/skills/ai-engineering-loop/SKILL.md',
256
+ '.gemini/skills/ai-engineering-loop/SKILL.md',
257
+ '.agents/workflows/ai-engineering-loop.md'
258
+ ]) {
259
+ const text = readRepo(rel);
260
+ assert.match(text, /failure table/, rel);
261
+ assert.match(text, /One red test per AC row/, rel);
262
+ assert.match(text, /Do not freeze sunny-path-only/, rel);
263
+ }
264
+ });
265
+
209
266
  test('Host skills absorb grill, TDD, glossary, and two-axis review without splitting the loop', () => {
210
267
  for (const rel of [
211
268
  '.claude/skills/ai-engineering-loop/SKILL.md',