ai-engineering-loop 1.0.4 → 1.0.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,17 +1,152 @@
1
1
  # AI Engineering Loop
2
2
 
3
+ <div align="center">
4
+
3
5
  [![NPM Version](https://img.shields.io/npm/v/ai-engineering-loop.svg?color=cb3837)](https://www.npmjs.com/package/ai-engineering-loop)
4
6
  [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
7
+ [![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](https://github.com/egagofur/ai-engineering-loop/pulls)
8
+ [![AI Engineering](https://img.shields.io/badge/AI-Engineering%20Loop-orange.svg)](https://github.com/egagofur/ai-engineering-loop)
9
+ [![Release](https://img.shields.io/badge/release-v1.0.6-purple.svg)](https://github.com/egagofur/ai-engineering-loop/releases)
10
+
11
+ **A Reusable, Framework-Agnostic AI Engineering Operating System for Autonomous Coding Agents**
12
+
13
+ *Featuring living project context, strict verification evidence contracts, 3-stage capability lifecycle registry, and dual-axis Judge evaluation.*
14
+
15
+ [Overview](#overview--philosophy) • [Runtime Capability Registry](#runtime-capability-registry--execution-modes) • [Verification Evidence](#verification-evidence-contract) • [CLI Commands](#cli-interface--commands) • [Grok CLI](#grok-cli-integration) • [Claude Code](#claude-code-integration) • [Antigravity](#antigravity-agent-integration) • [Lifecycle](#lifecycle-stages) • [Architecture](#architecture--5-layer-configuration) • [Project Profiles](#project-profiles) • [Repository Structure](#repository-structure) • [Reference Examples](#reference-examples) • [Contributing](#contributing)
16
+
17
+ </div>
18
+
19
+ ---
20
+
21
+ ## Overview & Philosophy
5
22
 
6
- **A Reusable, Framework-Agnostic AI Engineering Operating System for Autonomous Coding Agents.**
23
+ The AI Engineering Loop enforces clean architectural separation across three core layers:
7
24
 
8
- Features living project context, strict verification evidence contracts, 3-stage capability lifecycle registry, and dual-axis Judge evaluation.
25
+ ```mermaid
26
+ flowchart TD
27
+ Start([User Task in Workspace]) --> PreCheck{Pre-Task Drift Check: metadata.json}
28
+
29
+ PreCheck -->|Context Missing| AutoInit[Stage 0: Bootstrap .ai-engineering-loop/]
30
+ PreCheck -->|Drift Detected| Reconcile[Stage 0: Reconcile Drifted Context]
31
+ PreCheck -->|Context Fresh| GC[Stage 1: Goal Contract: Explicit Acceptance Criteria]
32
+
33
+ AutoInit --> GC
34
+ Reconcile --> GC
35
+
36
+ subgraph CoreEngine [AI ENGINEERING OPERATING SYSTEM]
37
+ GC --> RCA[Stage 2: Root Cause Analysis]
38
+ RCA --> Plan[Stage 3: Implementation Plan]
39
+ Plan --> MA[Stage 4: Maker Agent: Surgical Diff & Tests]
40
+ MA --> DV{Stage 5: Deterministic Verification<br>Evidence Contract: Exit Code 0 & Full Logs}
41
+
42
+ DV -->|Fail| MA
43
+ DV -->|Pass| DA[Stage 6: Devil's Advocate Review<br>Capability Registry & Artifact Barrier]
44
+
45
+ DA --> JD[Stage 7: Judge Agent: Impartial Magistrate<br>Validity + Severity Decision Matrix]
46
+ end
47
+
48
+ JD -->|VALID BLOCKER / HIGH: ITERATE| MA
49
+ JD -->|INVALID: Dismissed / VALID LOW: Tradeoff| CheckDoD{All ACs Verified?}
50
+
51
+ CheckDoD -->|Yes: PASS| ImpactEval{Post-Task Context Impact Assessment}
52
+ ImpactEval -->|NONE: Typo, UI tweak| Adapter[Stage 8: Delivery Adapter: GitLab / GitHub]
53
+ ImpactEval -->|TARGETED: Dep/route changed| PartialRefresh[Surgical Context Update] --> Adapter
54
+ ImpactEval -->|MAJOR: Framework migration| FullRefresh[Full Context Reconciliation] --> Adapter
55
+
56
+ Adapter --> TargetRepo[(Target Repository)]
57
+ ```
9
58
 
10
59
  ---
11
60
 
12
- ## Quick Start
61
+ ## Runtime Capability Registry & Execution Modes
62
+
63
+ The system maintains a strict distinction between **Configuration Support**, **Invocation Availability**, and **Execution Proof**:
64
+
65
+ ```text
66
+ ┌───────────────────────────┐ ┌───────────────────────────┐ ┌───────────────────────────┐
67
+ │ CONFIGURATION_SUPPORTED │ ──> │ INVOCATION_AVAILABLE │ ──> │ EXECUTION_PROVEN │
68
+ │ (Config is recognized) │ │ (Callable tool is active) │ │ (Child LLM response seen) │
69
+ └───────────────────────────┘ └───────────────────────────┘ └───────────────────────────┘
70
+ ```
71
+
72
+ ### 5 Standard Execution Modes (Deterministic Priority):
73
+
74
+ | Priority | Mode Name | Requires Independent LLM Execution? | Condition for Selection |
75
+ |:---:|---|:---:|---|
76
+ | **1** | **`TRUE_INDEPENDENT_AGENT`** | **YES** | Child session exists **AND** actual model response is captured **AND** context is independent. |
77
+ | **2** | **`ISOLATED_AGENT_INSTANCE`** | **YES** | Programmatic SDK agent instance with verified independent model execution. |
78
+ | **3** | **`FRESH_PROCESS_AGENT`** | **YES** | Separate OS process successfully executes an LLM agent with fresh context. |
79
+ | **4** | **`CONTEXT_ISOLATION_ONLY`** | **NO** | Clean-Slate Artifact Isolation Barrier in same session (100% prompt history excluded on disk). |
80
+ | **5** | **`UNAVAILABLE`** | **NO** | No review execution mechanism is available. |
81
+
82
+ ### Truthful Reporting Disclosure:
83
+ When `CONTEXT_ISOLATION_ONLY` is selected, the report strictly produces:
84
+ ```text
85
+ Execution Mode: CONTEXT_ISOLATION_ONLY
86
+ Independent LLM Execution: NOT PROVEN
87
+ Native Subagent Invocation: UNAVAILABLE
88
+ Review Method: Clean-Slate Artifact Isolation Barrier
89
+ ```
90
+
91
+ ---
92
+
93
+ ## Verification Evidence Contract
94
+
95
+ A verification `PASS` is strictly invalid without concrete execution evidence. The system categorically rejects vague statements such as *"command was launched"* or *"test appears to have passed"*.
96
+
97
+ ### Mandatory Execution Proof:
98
+ - **`command`**: Exact CLI string executed.
99
+ - **`executionIdentity`**: PID, execution hash, or system execution identifier.
100
+ - **`startTime` & `endTime`**: Documented execution duration.
101
+ - **`exitCode`**: Must be `0`.
102
+ - **`stdout` & `stderr`**: Raw machine logs captured.
103
+ - **`timeoutStatus`**: Must be `"COMPLETED"`.
104
+ - **`testCounts`**: Explicit counts of passed, failed, and skipped tests.
105
+ - **`assertionEvidence`**: Specific assertion proof matching the active Goal Contract's Acceptance Criteria.
106
+
107
+ ---
108
+
109
+ ## Dual-Axis Finding Model & Judge Decision Matrix
110
+
111
+ The Devil's Advocate categorizes findings along separate **Validity**, **Severity**, and **Disposition** axes:
112
+
113
+ ```json
114
+ {
115
+ "id": "DA-01",
116
+ "topic": "correctness",
117
+ "validity": "VALID",
118
+ "severity": "BLOCKER",
119
+ "disposition": "STRONG",
120
+ "location": "src/services/payment.ts#L42-L58",
121
+ "acceptanceCriteria": "AC-2",
122
+ "failureScenario": "Under concurrent traffic, duplicate rows are inserted before the lock is acquired.",
123
+ "evidence": "Missing SELECT FOR UPDATE in findByPaymentKey query.",
124
+ "concreteAlternativeDiff": "```diff\n- const tx = await findByKey(key);\n+ const tx = await findByKeyWithLock(key, { mode: 'FOR UPDATE' });\n```"
125
+ }
126
+ ```
127
+
128
+ ### Judge Decision Matrix:
129
+ - **`VALID + BLOCKER / HIGH`** $\rightarrow$ **`ITERATE`** (Maker must apply concrete fix diff and add regression tests).
130
+ - **`VALID + MEDIUM / LOW`** $\rightarrow$ **`ACCEPT / TRADEOFF`** (Merged; documented as acceptable tradeoff in MR notes).
131
+ - **`INVALID`** $\rightarrow$ **`DISMISS`** (Reviewer hallucination disproven by code; cannot block delivery; signature recorded).
132
+
133
+ *Reviewer disposition (`STRONG`, `ACCEPTABLE`, `WEAK`) never overrides factual evidence.*
134
+
135
+ ---
13
136
 
14
- You can initialize and manage `.ai-engineering-loop/` context directly using `npx`:
137
+ ## Living Project Context
138
+
139
+ The `.ai-engineering-loop/` directory is **Living Context**, not a static wiki generated once.
140
+
141
+ 1. **Post-Task Context Impact Assessment**: Evaluates completed tasks (`NONE`, `TARGETED`, `MAJOR`) to keep project context fresh without expensive whole-repo re-analysis.
142
+ 2. **Context Baseline (`metadata.json`)**: Tracks `repositoryRevision` (git commit SHA) and `manifestChecksums` for instant Level 0 (0ms) drift verification.
143
+ 3. **Strict Context Isolation**: Decouples living project context from ephemeral task logs and loop execution states.
144
+
145
+ ---
146
+
147
+ ## CLI Interface & Commands
148
+
149
+ The CLI package is published on NPM as [`ai-engineering-loop`](https://www.npmjs.com/package/ai-engineering-loop) and operates against the current working directory.
15
150
 
16
151
  ```bash
17
152
  # Bootstrap .ai-engineering-loop/ context from repository discovery
@@ -29,28 +164,153 @@ npx ai-engineering-loop run
29
164
 
30
165
  ---
31
166
 
32
- ## 🚀 Key Architectural Features
167
+ ## Grok CLI Integration
168
+
169
+ Grok CLI is a first-class host. `spawn_subagent` is a real independent child session (own context, no parent transcript unless `resume_from` is set). After a child id and model response are captured, the registry selects **`TRUE_INDEPENDENT_AGENT`**.
170
+
171
+ | Loop role | Grok `subagent_type` | Spawn rules |
172
+ |---|---|---|
173
+ | Orchestrator / Maker | parent session | Parent stays the orchestrator (Grok nesting depth is 1) |
174
+ | Devil's Advocate | `devil-advocate` (fallback `general-purpose`) | `capability_mode: execute`, omit `resume_from` |
175
+ | Judge | `judge` (fallback `general-purpose`) | Sibling of DA, never nested under DA |
176
+
177
+ Do **not** use `caveman:cavecrew-reviewer` as Devil's Advocate or Judge — its output schema is not the Finding Ledger.
178
+
179
+ Repo-local Grok files:
180
+
181
+ - `.grok/agents/devil-advocate.md` / `.grok/agents/judge.md`
182
+ - `.grok/skills/ai-engineering-loop/SKILL.md`
183
+ - `.grok/commands/ai-engineering-loop.md` → `/ai-engineering-loop`
184
+
185
+ Fallback: `GROK_SUBAGENTS=0` or `--disallowed-tools Agent` → `CONTEXT_ISOLATION_ONLY`, disclosed as such. Optional process fallback: `grok -p` → `FRESH_PROCESS_AGENT` only after a model response is captured.
186
+
187
+ See [docs/grok-cli-feasibility.md](docs/grok-cli-feasibility.md).
188
+
189
+ ---
190
+
191
+ ## Claude Code Integration
192
+
193
+ Claude Code is a first-class host. Use the **Task** (or **Agent**) tool with **only** `subagent_type`, `description`, and `prompt`.
194
+
195
+ Do **not** pass Grok keys (`spawn_subagent`, `capability_mode`, `isolation`, `resume_from`). Extra keys are the usual cause of:
196
+
197
+ ```
198
+ API Error: 400 [kiro/claude-sonnet-5] REQUEST_BODY_INVALID
199
+ ```
200
+
201
+ | Loop role | Claude Code `subagent_type` | Task keys |
202
+ |---|---|---|
203
+ | Orchestrator / Maker | parent session | n/a |
204
+ | Devil's Advocate | `devil-advocate` (fallback `general-purpose`) | `subagent_type`, `description`, `prompt` |
205
+ | Judge | `judge` (fallback `general-purpose`) | `subagent_type`, `description`, `prompt` |
206
+
207
+ Repo-local Claude Code files:
33
208
 
34
- 1. **Living Project Context**: Tracks baseline git SHA and manifest checksums in `metadata.json` for 0ms drift verification and post-task impact assessment (`NONE`, `TARGETED`, `MAJOR`).
35
- 2. **Deterministic Verification Evidence Contract**: Rejects vague assertions ("command was launched"); strictly requires CLI exit code 0, machine logs, and assertion proofs.
36
- 3. **Runtime Capability Registry (3-Stage Lifecycle)**:
37
- $$\text{CONFIGURATION\_SUPPORTED} \longrightarrow \text{INVOCATION\_AVAILABLE} \longrightarrow \text{EXECUTION\_PROVEN}$$
38
- Prevents misleading claims of multi-agent execution by requiring proven child session model execution.
39
- 4. **Dual-Axis Finding Model & Judge Matrix**:
40
- - `VALID + BLOCKER/HIGH` → `ITERATE`
41
- - `VALID + MEDIUM/LOW` → `ACCEPT / TRADEOFF`
42
- - `INVALID` → `DISMISS`
209
+ - `.claude/agents/devil-advocate.md` / `.claude/agents/judge.md`
210
+ - `.claude/skills/ai-engineering-loop/SKILL.md`
211
+ - `.claude/commands/ai-engineering-loop.md` `/ai-engineering-loop`
212
+
213
+ See [docs/claude-code-feasibility.md](docs/claude-code-feasibility.md).
43
214
 
44
215
  ---
45
216
 
46
- ## 📖 Full Documentation & Specifications
217
+ ## Antigravity Agent Integration
218
+
219
+ When working inside the Antigravity IDE or compatible agentic platforms, you can invoke the loop via slash commands:
47
220
 
48
- For complete specifications, agent role definitions, project profiles, and reference walkthroughs, visit the GitHub repository:
221
+ - **`/ai-engineering-loop init`**: Initialize project context only (non-destructive bootstrap).
222
+ - **`/ai-engineering-loop status`**: Check repository context health & baseline freshness.
223
+ - **`/ai-engineering-loop refresh`**: Reconcile drifted context files non-destructively.
224
+ - **`/ai-engineering-loop [task description]`**: Execute the full 8-stage engineering lifecycle with pre-task drift gate and post-task impact assessment.
49
225
 
50
- 👉 **[https://github.com/egagofur/ai-engineering-loop](https://github.com/egagofur/ai-engineering-loop)**
226
+ On Grok CLI the same slash command is provided by `.grok/commands/ai-engineering-loop.md` and runs Devil's Advocate / Judge as native subagents.
227
+
228
+ ---
229
+
230
+ ## Repository Structure
231
+
232
+ ```text
233
+ ai-engineering-loop/
234
+
235
+ ├── README.md # Operating system overview & architecture
236
+ ├── LICENSE # MIT Open Source License
237
+ ├── package.json # CLI package manifest
238
+
239
+ ├── bin/ # CLI execution entrypoints
240
+ │ └── ai-engineering-loop.js # npx executable CLI (init, status, refresh, run)
241
+
242
+ ├── lib/ # Core orchestration & decision engine
243
+ │ └── orchestration.js # 3-stage capability registry, barrier builder, Judge engine
244
+
245
+ ├── tests/ # Deterministic test suites
246
+ │ ├── capability-selection.test.js # Unit tests for capability lifecycle & truthful selection
247
+ │ ├── orchestration.test.js # Tests for isolation, Finding schema, Judge matrix
248
+ │ └── grok-runtime.test.js # Grok spawn_subagent mapping, aliases, forbidden types
249
+
250
+ ├── .grok/ # Grok CLI host adapter
251
+ │ ├── agents/devil-advocate.md # Native DA subagent type
252
+ │ ├── agents/judge.md # Native Judge subagent type
253
+ │ ├── skills/ai-engineering-loop/ # Grok skill (spawn protocol)
254
+ │ └── commands/ai-engineering-loop.md # /ai-engineering-loop slash command
255
+
256
+ ├── .claude/ # Claude Code host adapter (Kiro-safe)
257
+ │ ├── agents/devil-advocate.md # Task subagent type
258
+ │ ├── agents/judge.md # Task subagent type
259
+ │ ├── skills/ai-engineering-loop/ # Claude skill (Task keys only)
260
+ │ └── commands/ai-engineering-loop.md # /ai-engineering-loop slash command
261
+
262
+ ├── core/ # Generic engineering loop specifications
263
+ │ ├── orchestration-model.md # 3-stage capability lifecycle & execution priority
264
+ │ ├── project-initialization.md # Auto-discovery & initialization lifecycle
265
+ │ ├── context-refresh-policy.md # Progressive drift hierarchy & living baseline
266
+ │ ├── context-impact-assessment.md # Post-task impact assessment (NONE, TARGETED, MAJOR)
267
+ │ ├── goal-contract.md # Task contract schema & acceptance criteria
268
+ │ ├── verification-loop.md # Dual-layer verification & Evidence Contract
269
+ │ ├── definition-of-done.md # 5 pillars of Done & rejection triggers
270
+ │ ├── iteration-policy.md # Bounded autonomous loop (MAX_ITERATIONS = 3)
271
+ │ ├── escalation-policy.md # Deterministic human escalation triggers
272
+ │ ├── judge-policy.md # Evaluation rules, triage audit, & verdicts
273
+ │ ├── configuration-precedence.md # 5-layer precedence & conflict resolution
274
+ │ └── repo-config-schema.md # Schema for target repo .ai-engineering-loop/
275
+
276
+ ├── profiles/ # Project archetype profiles
277
+ │ ├── README.md # Profile catalog & auto-detection rules
278
+ │ ├── web-app.md # Frontend web applications
279
+ │ ├── backend-api.md # Backend APIs & microservices
280
+ │ ├── mobile-app.md # Native & cross-platform mobile apps
281
+ │ ├── library.md # Reusable SDKs & shared packages
282
+ │ └── monorepo.md # Multi-package monorepo workspaces
283
+
284
+ ├── agents/ # Triad agent role specifications
285
+ │ ├── maker.md # Maker agent: surgical diffs & unit tests
286
+ │ ├── devil-advocate.md # Adversarial reviewer: dual-axis finding ledger & diffs
287
+ │ └── judge.md # Judge agent: impartial magistrate on Validity + Severity
288
+
289
+ ├── policies/ # Operational schemas & algorithms
290
+ │ ├── discovery-safety-policy.md # Secret protection & non-destructive discovery rules
291
+ │ ├── finding-policy.md # Dual-axis finding schema & severity matrix
292
+ │ ├── evidence-policy.md # 5-level evidence hierarchy & Verification Evidence Contract
293
+ │ └── no-progress-policy.md # Finding signature hashing & stagnation detection
294
+
295
+ ├── adapters/ # Pluggable delivery pipelines
296
+ │ └── dot/ # DOT Indonesia delivery adapter
297
+ │ ├── README.md # DOT adapter overview
298
+ │ ├── gitlab.md # glab CLI, issue cards, & MR generation
299
+ │ ├── multi-branch.md # main / staging / develop cherry-pick propagation
300
+ │ ├── coreview.md # @coreview-bot external review triage (Valid vs Halu)
301
+ │ └── mattermost.md # Channel mapping & MCP dispatch (from: "AI Agent")
302
+
303
+ └── templates/ # Starter templates for target repositories
304
+ └── repo-config/ # Ready-to-copy .ai-engineering-loop/ files
305
+ ├── config.md # Project identity & profile binding
306
+ ├── architecture.md # Layers & boundary invariants
307
+ ├── conventions.md # Code standards & forbidden patterns
308
+ ├── verification.md # CLI test/lint/build commands
309
+ └── adapter.md # Configured release pipeline
310
+ ```
51
311
 
52
312
  ---
53
313
 
54
- ## 📄 License
314
+ ## License
55
315
 
56
- MIT © [Ega Gofur](https://github.com/egagofur)
316
+ This project is licensed under the **MIT License** — see the [LICENSE](LICENSE) file for details.
package/README.npm.md CHANGED
@@ -36,6 +36,17 @@ npx ai-engineering-loop run
36
36
  2. **Living Project Context**: Tracks repository revisions and manifest checksums in `metadata.json` for instant drift detection.
37
37
  3. **Multi-Agent Triad**: Coordinates **Maker** (surgical diffs & tests), **Devil's Advocate** (independent adversarial review), and **Judge** (impartial evaluation & PASS certification).
38
38
  4. **Context Impact Assessment**: Evaluates completed tasks (`NONE`, `TARGETED`, `MAJOR`) to keep project context fresh without expensive whole-repo re-analysis.
39
+ 5. **Grok CLI host**: Native `spawn_subagent` for Devil's Advocate and Judge (`TRUE_INDEPENDENT_AGENT`). Disabled by `GROK_SUBAGENTS=0`.
40
+
41
+ ---
42
+
43
+ ## Grok CLI
44
+
45
+ Inside a Grok TUI session, `/ai-engineering-loop` uses repo-local `.grok/agents/` types. Devil's Advocate and Judge spawn as sibling children (`capability_mode: execute`, no `resume_from`). Do not use `caveman:cavecrew-reviewer` as the loop reviewer.
46
+
47
+ ## Claude Code
48
+
49
+ Inside Claude Code, `/ai-engineering-loop` uses `.claude/agents/` types via the **Task** tool. Pass only `subagent_type`, `description`, and `prompt`. Do not pass Grok keys (`spawn_subagent`, `capability_mode`, `resume_from`) — extra keys cause Kiro `REQUEST_BODY_INVALID`.
39
50
 
40
51
  ---
41
52
 
@@ -12,12 +12,16 @@ The **Devil's Advocate** is an independent adversarial reviewer. Its sole purpos
12
12
 
13
13
  ## 2. Review Execution Modes
14
14
 
15
- The Devil's Advocate executes under one of 4 runtime modes depending on platform capabilities:
15
+ The Devil's Advocate executes under one of 4 runtime modes depending on platform capabilities.
16
16
 
17
- 1. **`NATIVE_SUBAGENT`**: Genuine independent sub-agent session spawned by host runtime.
18
- 2. **`SDK_AGENT`**: Programmatic Python SDK agent instance with isolated memory.
19
- 3. **`HEADLESS_SUBPROCESS`**: Fresh subprocess agent spawned via CLI.
20
- 4. **`ARTIFACT_ISOLATED_REVIEW`**: Clean-Slate Artifact Barrier in single-agent session (*strictly labeled: isolated review context, not independent agent execution*).
17
+ Canonical ids are listed first; skill aliases are in parentheses.
18
+
19
+ 1. **`TRUE_INDEPENDENT_AGENT`** (`NATIVE_SUBAGENT`): Genuine independent child session.
20
+ - **Grok CLI**: `spawn_subagent` with `subagent_type: "devil-advocate"` (fallback `"general-purpose"`), `capability_mode: "execute"`, `isolation: "none"`, **omit** `resume_from`. Agent definition: `.grok/agents/devil-advocate.md`.
21
+ - Do **not** use `caveman:cavecrew-reviewer` (compressed review schema, not a Finding Ledger).
22
+ 2. **`ISOLATED_AGENT_INSTANCE`** (`SDK_AGENT`): Programmatic SDK agent instance with isolated memory.
23
+ 3. **`FRESH_PROCESS_AGENT`** (`HEADLESS_SUBPROCESS`): Fresh OS process such as `grok -p` after a model response is captured.
24
+ 4. **`CONTEXT_ISOLATION_ONLY`** (`ARTIFACT_ISOLATED_REVIEW`): Clean-Slate Artifact Barrier in single-agent session (*strictly labeled: isolated review context, not independent agent execution*).
21
25
 
22
26
  ---
23
27
 
package/agents/judge.md CHANGED
@@ -4,6 +4,8 @@
4
4
 
5
5
  The **Judge Agent** is the final decision-maker of the AI Engineering Loop. It serves as an impartial magistrate that evaluates the complete evidence pipeline (Contract, Diff, Deterministic Verification Logs, and Devil's Advocate findings) to determine whether the iteration should **PASS**, **ITERATE**, or **ESCALATE**.
6
6
 
7
+ On **Grok CLI**, the parent orchestrator spawns the Judge with `spawn_subagent` (`subagent_type: "judge"`, fallback `"general-purpose"`), `capability_mode: "execute"`, and **no** `resume_from`. Agent definition: `.grok/agents/judge.md`. The Judge is a sibling of the Devil's Advocate, never its child (Grok nesting depth is 1).
8
+
7
9
  ```mermaid
8
10
  flowchart LR
9
11
  Inputs[Goal Contract + Diff + Test Logs + Review Findings] --> Judge[Judge Agent]
@@ -15,7 +15,7 @@ const path = require('path');
15
15
  const crypto = require('crypto');
16
16
  const { execSync } = require('child_process');
17
17
 
18
- const VERSION = '1.0.2';
18
+ const VERSION = '1.0.6';
19
19
  const CWD = process.cwd();
20
20
  const CONTEXT_DIR = path.join(CWD, '.ai-engineering-loop');
21
21
 
@@ -553,6 +553,15 @@ function handleRefresh() {
553
553
  handleStatus();
554
554
  }
555
555
 
556
+ function detectGrokHost() {
557
+ try {
558
+ const { detectGrokRuntime } = require('../lib/orchestration.js');
559
+ return detectGrokRuntime(process.env, fs);
560
+ } catch (e) {
561
+ return null;
562
+ }
563
+ }
564
+
556
565
  // Command: run
557
566
  function handleRun() {
558
567
  log.info('AI Engineering Loop — Task Execution Entrypoint (run)');
@@ -564,6 +573,8 @@ function handleRun() {
564
573
  handleStatus();
565
574
  }
566
575
 
576
+ const grok = detectGrokHost();
577
+
567
578
  console.log('\n------------------------------------------------------------');
568
579
  log.bold('AI Agent Ready:');
569
580
  console.log('1. Formulate Goal Contract (core/goal-contract.md)');
@@ -574,6 +585,34 @@ function handleRun() {
574
585
  console.log('6. Judge Agent evaluates DoD and issues PASS verdict');
575
586
  console.log('7. Context Impact Assessment (NONE / TARGETED / MAJOR)');
576
587
  console.log('8. Delivery Adapter creates MR/PR');
588
+
589
+ if (grok && grok.host === 'grok-cli') {
590
+ console.log('------------------------------------------------------------');
591
+ log.bold('Grok CLI host:');
592
+ console.log(`- Binary: ${grok.grokBin || 'detected'}`);
593
+ console.log(`- spawn_subagent: ${grok.invocationAvailable ? 'INVOCATION_AVAILABLE' : 'UNAVAILABLE'}`);
594
+ console.log(`- Execution proven: no (requires child subagent_id + model response)`);
595
+ if (grok.invocationAvailable) {
596
+ console.log('- Devil\'s Advocate: spawn_subagent type=devil-advocate capability_mode=execute (no resume_from)');
597
+ console.log('- Judge: spawn_subagent type=judge capability_mode=execute (sibling, not nested)');
598
+ console.log('- Forbidden types: caveman:cavecrew-reviewer, explore, plan');
599
+ } else {
600
+ console.log(`- Fallback: CONTEXT_ISOLATION_ONLY (${grok.reason})`);
601
+ }
602
+ console.log('- Skill: .grok/skills/ai-engineering-loop/SKILL.md');
603
+ }
604
+
605
+ const claudeSkill = path.join(CWD, '.claude', 'skills', 'ai-engineering-loop', 'SKILL.md');
606
+ const claudeAgent = path.join(CWD, '.claude', 'agents', 'devil-advocate.md');
607
+ if (fs.existsSync(claudeSkill) || fs.existsSync(claudeAgent)) {
608
+ console.log('------------------------------------------------------------');
609
+ log.bold('Claude Code host:');
610
+ console.log('- Subagent tool: Task (or Agent). Keys allowed: subagent_type, description, prompt');
611
+ console.log('- Devil\'s Advocate: Task subagent_type=devil-advocate');
612
+ console.log('- Judge: Task subagent_type=judge (sibling, not nested)');
613
+ console.log('- Do not pass spawn_subagent, capability_mode, isolation, resume_from (Kiro 400 REQUEST_BODY_INVALID)');
614
+ console.log('- Skill: .claude/skills/ai-engineering-loop/SKILL.md');
615
+ }
577
616
  console.log('------------------------------------------------------------\n');
578
617
  }
579
618
 
@@ -69,6 +69,26 @@ The system categorically rejects the following as proof of independent LLM execu
69
69
 
70
70
  ---
71
71
 
72
+ ## 4b. Grok CLI Empirical Discovery Record
73
+
74
+ | Investigated Surface | Tested Command / API | Classification | Status & Result |
75
+ |---|---|---|---|
76
+ | **Grok CLI binary** | `~/.grok/bin/grok` | `CONFIGURATION_SUPPORTED` | Installed (`grok 1.0.5+`). Not execution proof. |
77
+ | **Native subagent tool** | `spawn_subagent` | `INVOCATION_AVAILABLE` | Enabled by default. Disabled by `GROK_SUBAGENTS=0` or `--disallowed-tools Agent`. |
78
+ | **Child session** | `subagent_id` + child model response, `resume_from` omitted | `EXECUTION_PROVEN` → `TRUE_INDEPENDENT_AGENT` | Own context window; parent transcript is not inherited. |
79
+ | **Project agent types** | `.grok/agents/devil-advocate.md`, `.grok/agents/judge.md` | `CONFIGURATION_SUPPORTED` | Prefer these types; fall back to `general-purpose` with the spec prepended. |
80
+ | **Headless process** | `grok -p` | `FRESH_PROCESS_AGENT` | Only after the process prints a model response. |
81
+ | **Cavecrew reviewer** | `caveman:cavecrew-reviewer` | `GROK_COMPRESSED_REVIEW_PRESET` | **Rejected** as Devil's Advocate / Judge (wrong output schema). |
82
+ | **Maker resume** | `spawn_subagent(resume_from=maker)` | tainted history | **Rejected**. DA and Judge must spawn fresh. |
83
+ | **Artifact barrier** | `buildReviewContextBarrier()` | `CONTEXT_ISOLATION_ONLY` | Fallback when spawn is disabled. |
84
+
85
+ ### Architectural Conclusion:
86
+ > *"Grok CLI spawn_subagent is a true independent child session. Select TRUE_INDEPENDENT_AGENT only after a child id and model response are captured without resume_from. Binary presence and agent markdown are never sufficient."*
87
+
88
+ See [docs/grok-cli-feasibility.md](../docs/grok-cli-feasibility.md).
89
+
90
+ ---
91
+
72
92
  ## 5. Truthful Reporting Output
73
93
 
74
94
  When `CONTEXT_ISOLATION_ONLY` is selected, the report generator strictly produces:
@@ -0,0 +1,45 @@
1
+ # Claude Code Feasibility and Kiro Request Safety
2
+
3
+ ## 1. Problem
4
+
5
+ Claude Code users (including Kiro-backed models such as `kiro/claude-sonnet-5`) hit:
6
+
7
+ ```
8
+ API Error: 400 [kiro/claude-sonnet-5] [400]:
9
+ {"message":"Improperly formed request.","reason":"REQUEST_BODY_INVALID"}
10
+ ```
11
+
12
+ Kiro's `generateAssistantResponse` rejects request bodies that Claude Code will happily send. Two skill-side causes were confirmed:
13
+
14
+ 1. **Extra tool keys.** The shared skill told the model to call `spawn_subagent` with `capability_mode`, `isolation`, and `resume_from`. Claude Code's Task/Agent schema does not have those fields. Proxies that validate additionalProperties fail with `REQUEST_BODY_INVALID`.
15
+ 2. **Hostile markup in the injected skill.** Mermaid (`<br>`, nested `{braces}`), LaTeX (`$\rightarrow$`), and folded YAML descriptions inflate and corrupt the system/skill payload Kiro receives.
16
+
17
+ This is separate from 9router bugs that reject any Anthropic `system` field. Those need a router fix. The skill must still not add extra invalid keys or markup.
18
+
19
+ ## 2. Claude Code mapping
20
+
21
+ | Loop role | Claude Code type | Task keys allowed |
22
+ |---|---|---|
23
+ | Orchestrator / Maker | parent session | n/a |
24
+ | Devil's Advocate | `devil-advocate` (fallback `general-purpose`) | `subagent_type`, `description`, `prompt` |
25
+ | Judge | `judge` (fallback `general-purpose`) | `subagent_type`, `description`, `prompt` |
26
+
27
+ Do not pass Grok keys (`spawn_subagent`, `capability_mode`, `resume_from`, `isolation`, `background`) on Claude Code.
28
+
29
+ Repo files:
30
+
31
+ - `.claude/skills/ai-engineering-loop/SKILL.md`
32
+ - `.claude/agents/devil-advocate.md`
33
+ - `.claude/agents/judge.md`
34
+ - `.claude/commands/ai-engineering-loop.md`
35
+
36
+ ## 3. Mode selection
37
+
38
+ | Condition | Mode |
39
+ |---|---|
40
+ | Task/Agent tool present, child result returned | `TRUE_INDEPENDENT_AGENT` |
41
+ | No subagent tool | `CONTEXT_ISOLATION_ONLY` |
42
+
43
+ ## 4. What stays in Grok-only files
44
+
45
+ Grok spawn details live in `.grok/skills/ai-engineering-loop/SKILL.md`. Claude Code must not load that file as its skill. Claude Code discovers `.claude/` first.
@@ -0,0 +1,99 @@
1
+ # Grok CLI Feasibility & Execution Strategy
2
+
3
+ ## 1. Executive Summary
4
+
5
+ Grok CLI (verified `grok 1.0.5`) is a **first-class host** for the AI Engineering Loop. Unlike Antigravity standalone mode, Grok exposes `spawn_subagent` as a real independent child session: own context window, no parent transcript unless `resume_from` is set, and a captured `subagent_id` plus model response.
6
+
7
+ When those evidence fields are present, the capability registry **must** select `TRUE_INDEPENDENT_AGENT` (skill alias: `NATIVE_SUBAGENT`).
8
+
9
+ ```text
10
+ Parent (Maker + orchestrator)
11
+ ├─ spawn_subagent type=devil-advocate → Finding Ledger
12
+ └─ spawn_subagent type=judge → PASS | ITERATE | ESCALATE
13
+ ```
14
+
15
+ Children cannot spawn children (depth 1). The parent therefore remains the orchestrator.
16
+
17
+ ---
18
+
19
+ ## 2. Empirical Discovery Record
20
+
21
+ | Surface | Tested command / API | Classification | Result |
22
+ |---|---|---|---|
23
+ | Grok CLI binary | `~/.grok/bin/grok` (`grok 1.0.5`) | `CONFIGURATION_SUPPORTED` | Installed and authenticated in TUI sessions |
24
+ | Native child session | `spawn_subagent` | `INVOCATION_AVAILABLE` | Enabled by default; disabled only when `GROK_SUBAGENTS=0` or `--disallowed-tools Agent` |
25
+ | Child execution | child `subagent_id` + model response | `EXECUTION_PROVEN` | Proven only after the child returns; binary presence is not proof |
26
+ | Project agents | `.grok/agents/devil-advocate.md`, `.grok/agents/judge.md` | `CONFIGURATION_SUPPORTED` | Registered as `subagent_type` values |
27
+ | Skill + slash command | `.grok/skills/ai-engineering-loop/`, `.grok/commands/ai-engineering-loop.md` | `CONFIGURATION_SUPPORTED` | Grok discovers repo-local skills and command markdown |
28
+ | Headless child | `grok -p "..."` | `FRESH_PROCESS_AGENT` | Valid fallback **after** stdout contains a model response |
29
+ | Compressed reviewer | `caveman:cavecrew-reviewer` | `GROK_COMPRESSED_REVIEW_PRESET` | **Rejected** — output schema is `path:line: emoji severity`, not the Finding Ledger |
30
+ | Browser tools | `browser_*` | `BROWSER_AUTOMATION_TOOL` | Not an LLM reviewer |
31
+ | `resume_from` Maker | `spawn_subagent(resume_from=makerId)` | tainted history | **Rejected** — inherits Maker transcript |
32
+
33
+ ### What is NOT execution proof on Grok
34
+
35
+ 1. `~/.grok/bin/grok` existing on disk
36
+ 2. `.grok/agents/*.md` being discoverable
37
+ 3. Subagents being enabled in `config.toml`
38
+ 4. A persona named "reviewer" without a child response
39
+ 5. Cavecrew reviewer output that looks like a review
40
+
41
+ ---
42
+
43
+ ## 3. Mapping Loop Roles → Grok Types
44
+
45
+ | Loop role | Grok `subagent_type` | `capability_mode` | `isolation` | `resume_from` |
46
+ |---|---|---|---|---|
47
+ | Orchestrator / Maker | parent session (or `general-purpose` if parent is orchestrator-only) | `all` | `none` | n/a |
48
+ | Devil's Advocate | `devil-advocate` (fallback `general-purpose`) | `execute` | `none` | **omit** |
49
+ | Judge | `judge` (fallback `general-purpose`) | `execute` | `none` | **omit** |
50
+
51
+ `execute` allows git/read/search and forbids source edits. Do not use `read-only` if the child needs `git diff`. Do not use `worktree` isolation: DA and Judge must see the Maker working tree.
52
+
53
+ `description` prefixes `[devil-advocate]` and `[judge]` so the Grok TUI labels the child correctly when falling back to `general-purpose`.
54
+
55
+ ---
56
+
57
+ ## 4. Artifact Barrier (still required)
58
+
59
+ Grok children do not inherit parent chat, but they do receive compacted `AGENTS.md` / project instructions. That is acceptable (project context, not Maker thoughts).
60
+
61
+ Still write artifacts to disk and pass **paths**, not Maker narration:
62
+
63
+ - Goal Contract
64
+ - git diff file
65
+ - verification log (command, exit code, stdout, test counts)
66
+ - `.ai-engineering-loop/`
67
+
68
+ ---
69
+
70
+ ## 5. Disable / fallback matrix
71
+
72
+ | Condition | Selected mode |
73
+ |---|---|
74
+ | `spawn_subagent` available, child response captured, no `resume_from` | `TRUE_INDEPENDENT_AGENT` |
75
+ | `spawn_subagent` missing, `grok -p` returns a model response | `FRESH_PROCESS_AGENT` |
76
+ | `GROK_SUBAGENTS=0` or `--disallowed-tools Agent` | `CONTEXT_ISOLATION_ONLY` |
77
+ | No grok, no spawn, no headless | `UNAVAILABLE` or artifact barrier if the same session can still read the diff |
78
+
79
+ ---
80
+
81
+ ## 6. Answers to the eight host questions
82
+
83
+ 1. **Independent sub-agents?** Yes — `spawn_subagent`, own context window.
84
+ 2. **Pass context?** Yes — prompt + on-disk artifacts. Do not pass Maker chat.
85
+ 3. **Same repository?** Yes — `isolation: none` shares the workspace.
86
+ 4. **Structured findings?** Yes — DA agent contract is Finding Ledger JSON.
87
+ 5. **Parent consume findings?** Yes — child summary returns to parent; Judge spawn gets the ledger.
88
+ 6. **Repeat the loop?** Yes — parent iterates; each DA/Judge spawn is fresh (no `resume_from`).
89
+ 7. **Persist state?** Yes — files under `.ai-engineering-loop/` and task artifact paths. Grok also keeps session logs under `~/.grok/sessions/`.
90
+ 8. **Token limits?** Pass artifact paths, not full transcripts. Do not dump Maker reasoning into the DA prompt.
91
+
92
+ ---
93
+
94
+ ## 7. Practical limits on Grok CLI
95
+
96
+ - **Depth 1**: a DA child cannot spawn a Judge. Parent must spawn both.
97
+ - **Plan mode**: a write-capable child is not gated by the parent's plan-mode lock. Keep DA/Judge on `capability_mode: execute` (no writes) so they cannot edit source while the parent is in plan mode.
98
+ - **Headless `--disallowed-tools Agent`**: kills native review. Disclose `CONTEXT_ISOLATION_ONLY`.
99
+ - **Cavecrew**: useful for cheap locate/review, **not** for this loop's Judge input.