ai-engineering-loop 1.0.3 → 1.0.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.grok/agents/devil-advocate.md +67 -0
- package/.grok/agents/judge.md +53 -0
- package/.grok/commands/ai-engineering-loop.md +11 -0
- package/.grok/skills/ai-engineering-loop/SKILL.md +79 -0
- package/README.md +247 -19
- package/README.npm.md +7 -0
- package/agents/devil-advocate.md +9 -5
- package/agents/judge.md +2 -0
- package/bin/ai-engineering-loop.js +28 -1
- package/core/orchestration-model.md +20 -0
- package/docs/grok-cli-feasibility.md +99 -0
- package/lib/orchestration.js +182 -2
- package/package.json +22 -2
- package/tests/grok-runtime.test.js +201 -0
- package/README.full.md +0 -251
package/README.full.md
DELETED
|
@@ -1,251 +0,0 @@
|
|
|
1
|
-
# AI Engineering Loop
|
|
2
|
-
|
|
3
|
-
<div align="center">
|
|
4
|
-
|
|
5
|
-
[](https://www.npmjs.com/package/ai-engineering-loop)
|
|
6
|
-
[](https://opensource.org/licenses/MIT)
|
|
7
|
-
[](https://github.com/egagofur/ai-engineering-loop/pulls)
|
|
8
|
-
[](https://github.com/egagofur/ai-engineering-loop)
|
|
9
|
-
[](https://github.com/egagofur/ai-engineering-loop/releases)
|
|
10
|
-
|
|
11
|
-
**A Reusable, Framework-Agnostic AI Engineering Operating System for Autonomous Coding Agents**
|
|
12
|
-
|
|
13
|
-
*Featuring living project context, strict verification evidence contracts, 3-stage capability lifecycle registry, and dual-axis Judge evaluation.*
|
|
14
|
-
|
|
15
|
-
[Overview](#overview--philosophy) • [Runtime Capability Registry](#runtime-capability-registry--execution-modes) • [Verification Evidence](#verification-evidence-contract) • [CLI Commands](#cli-interface--commands) • [Agent Integration](#antigravity-agent-integration) • [Lifecycle](#lifecycle-stages) • [Architecture](#architecture--5-layer-configuration) • [Project Profiles](#project-profiles) • [Repository Structure](#repository-structure) • [Reference Examples](#reference-examples) • [Contributing](#contributing)
|
|
16
|
-
|
|
17
|
-
</div>
|
|
18
|
-
|
|
19
|
-
---
|
|
20
|
-
|
|
21
|
-
## Overview & Philosophy
|
|
22
|
-
|
|
23
|
-
The AI Engineering Loop enforces clean architectural separation across three core layers:
|
|
24
|
-
|
|
25
|
-
```mermaid
|
|
26
|
-
flowchart TD
|
|
27
|
-
Start([User Task in Workspace]) --> PreCheck{Pre-Task Drift Check: metadata.json}
|
|
28
|
-
|
|
29
|
-
PreCheck -->|Context Missing| AutoInit[Stage 0: Bootstrap .ai-engineering-loop/]
|
|
30
|
-
PreCheck -->|Drift Detected| Reconcile[Stage 0: Reconcile Drifted Context]
|
|
31
|
-
PreCheck -->|Context Fresh| GC[Stage 1: Goal Contract: Explicit Acceptance Criteria]
|
|
32
|
-
|
|
33
|
-
AutoInit --> GC
|
|
34
|
-
Reconcile --> GC
|
|
35
|
-
|
|
36
|
-
subgraph CoreEngine [AI ENGINEERING OPERATING SYSTEM]
|
|
37
|
-
GC --> RCA[Stage 2: Root Cause Analysis]
|
|
38
|
-
RCA --> Plan[Stage 3: Implementation Plan]
|
|
39
|
-
Plan --> MA[Stage 4: Maker Agent: Surgical Diff & Tests]
|
|
40
|
-
MA --> DV{Stage 5: Deterministic Verification<br>Evidence Contract: Exit Code 0 & Full Logs}
|
|
41
|
-
|
|
42
|
-
DV -->|Fail| MA
|
|
43
|
-
DV -->|Pass| DA[Stage 6: Devil's Advocate Review<br>Capability Registry & Artifact Barrier]
|
|
44
|
-
|
|
45
|
-
DA --> JD[Stage 7: Judge Agent: Impartial Magistrate<br>Validity + Severity Decision Matrix]
|
|
46
|
-
end
|
|
47
|
-
|
|
48
|
-
JD -->|VALID BLOCKER / HIGH: ITERATE| MA
|
|
49
|
-
JD -->|INVALID: Dismissed / VALID LOW: Tradeoff| CheckDoD{All ACs Verified?}
|
|
50
|
-
|
|
51
|
-
CheckDoD -->|Yes: PASS| ImpactEval{Post-Task Context Impact Assessment}
|
|
52
|
-
ImpactEval -->|NONE: Typo, UI tweak| Adapter[Stage 8: Delivery Adapter: GitLab / GitHub]
|
|
53
|
-
ImpactEval -->|TARGETED: Dep/route changed| PartialRefresh[Surgical Context Update] --> Adapter
|
|
54
|
-
ImpactEval -->|MAJOR: Framework migration| FullRefresh[Full Context Reconciliation] --> Adapter
|
|
55
|
-
|
|
56
|
-
Adapter --> TargetRepo[(Target Repository)]
|
|
57
|
-
```
|
|
58
|
-
|
|
59
|
-
---
|
|
60
|
-
|
|
61
|
-
## Runtime Capability Registry & Execution Modes
|
|
62
|
-
|
|
63
|
-
The system maintains a strict distinction between **Configuration Support**, **Invocation Availability**, and **Execution Proof**:
|
|
64
|
-
|
|
65
|
-
```text
|
|
66
|
-
┌───────────────────────────┐ ┌───────────────────────────┐ ┌───────────────────────────┐
|
|
67
|
-
│ CONFIGURATION_SUPPORTED │ ──> │ INVOCATION_AVAILABLE │ ──> │ EXECUTION_PROVEN │
|
|
68
|
-
│ (Config is recognized) │ │ (Callable tool is active) │ │ (Child LLM response seen) │
|
|
69
|
-
└───────────────────────────┘ └───────────────────────────┘ └───────────────────────────┘
|
|
70
|
-
```
|
|
71
|
-
|
|
72
|
-
### 5 Standard Execution Modes (Deterministic Priority):
|
|
73
|
-
|
|
74
|
-
| Priority | Mode Name | Requires Independent LLM Execution? | Condition for Selection |
|
|
75
|
-
|:---:|---|:---:|---|
|
|
76
|
-
| **1** | **`TRUE_INDEPENDENT_AGENT`** | **YES** | Child session exists **AND** actual model response is captured **AND** context is independent. |
|
|
77
|
-
| **2** | **`ISOLATED_AGENT_INSTANCE`** | **YES** | Programmatic SDK agent instance with verified independent model execution. |
|
|
78
|
-
| **3** | **`FRESH_PROCESS_AGENT`** | **YES** | Separate OS process successfully executes an LLM agent with fresh context. |
|
|
79
|
-
| **4** | **`CONTEXT_ISOLATION_ONLY`** | **NO** | Clean-Slate Artifact Isolation Barrier in same session (100% prompt history excluded on disk). |
|
|
80
|
-
| **5** | **`UNAVAILABLE`** | **NO** | No review execution mechanism is available. |
|
|
81
|
-
|
|
82
|
-
### Truthful Reporting Disclosure:
|
|
83
|
-
When `CONTEXT_ISOLATION_ONLY` is selected, the report strictly produces:
|
|
84
|
-
```text
|
|
85
|
-
Execution Mode: CONTEXT_ISOLATION_ONLY
|
|
86
|
-
Independent LLM Execution: NOT PROVEN
|
|
87
|
-
Native Subagent Invocation: UNAVAILABLE
|
|
88
|
-
Review Method: Clean-Slate Artifact Isolation Barrier
|
|
89
|
-
```
|
|
90
|
-
|
|
91
|
-
---
|
|
92
|
-
|
|
93
|
-
## Verification Evidence Contract
|
|
94
|
-
|
|
95
|
-
A verification `PASS` is strictly invalid without concrete execution evidence. The system categorically rejects vague statements such as *"command was launched"* or *"test appears to have passed"*.
|
|
96
|
-
|
|
97
|
-
### Mandatory Execution Proof:
|
|
98
|
-
- **`command`**: Exact CLI string executed.
|
|
99
|
-
- **`executionIdentity`**: PID, execution hash, or system execution identifier.
|
|
100
|
-
- **`startTime` & `endTime`**: Documented execution duration.
|
|
101
|
-
- **`exitCode`**: Must be `0`.
|
|
102
|
-
- **`stdout` & `stderr`**: Raw machine logs captured.
|
|
103
|
-
- **`timeoutStatus`**: Must be `"COMPLETED"`.
|
|
104
|
-
- **`testCounts`**: Explicit counts of passed, failed, and skipped tests.
|
|
105
|
-
- **`assertionEvidence`**: Specific assertion proof matching the active Goal Contract's Acceptance Criteria.
|
|
106
|
-
|
|
107
|
-
---
|
|
108
|
-
|
|
109
|
-
## Dual-Axis Finding Model & Judge Decision Matrix
|
|
110
|
-
|
|
111
|
-
The Devil's Advocate categorizes findings along separate **Validity**, **Severity**, and **Disposition** axes:
|
|
112
|
-
|
|
113
|
-
```json
|
|
114
|
-
{
|
|
115
|
-
"id": "DA-01",
|
|
116
|
-
"topic": "correctness",
|
|
117
|
-
"validity": "VALID",
|
|
118
|
-
"severity": "BLOCKER",
|
|
119
|
-
"disposition": "STRONG",
|
|
120
|
-
"location": "src/services/payment.ts#L42-L58",
|
|
121
|
-
"acceptanceCriteria": "AC-2",
|
|
122
|
-
"failureScenario": "Under concurrent traffic, duplicate rows are inserted before the lock is acquired.",
|
|
123
|
-
"evidence": "Missing SELECT FOR UPDATE in findByPaymentKey query.",
|
|
124
|
-
"concreteAlternativeDiff": "```diff\n- const tx = await findByKey(key);\n+ const tx = await findByKeyWithLock(key, { mode: 'FOR UPDATE' });\n```"
|
|
125
|
-
}
|
|
126
|
-
```
|
|
127
|
-
|
|
128
|
-
### Judge Decision Matrix:
|
|
129
|
-
- **`VALID + BLOCKER / HIGH`** $\rightarrow$ **`ITERATE`** (Maker must apply concrete fix diff and add regression tests).
|
|
130
|
-
- **`VALID + MEDIUM / LOW`** $\rightarrow$ **`ACCEPT / TRADEOFF`** (Merged; documented as acceptable tradeoff in MR notes).
|
|
131
|
-
- **`INVALID`** $\rightarrow$ **`DISMISS`** (Reviewer hallucination disproven by code; cannot block delivery; signature recorded).
|
|
132
|
-
|
|
133
|
-
*Reviewer disposition (`STRONG`, `ACCEPTABLE`, `WEAK`) never overrides factual evidence.*
|
|
134
|
-
|
|
135
|
-
---
|
|
136
|
-
|
|
137
|
-
## Living Project Context
|
|
138
|
-
|
|
139
|
-
The `.ai-engineering-loop/` directory is **Living Context**, not a static wiki generated once.
|
|
140
|
-
|
|
141
|
-
1. **Post-Task Context Impact Assessment**: Evaluates completed tasks (`NONE`, `TARGETED`, `MAJOR`) to keep project context fresh without expensive whole-repo re-analysis.
|
|
142
|
-
2. **Context Baseline (`metadata.json`)**: Tracks `repositoryRevision` (git commit SHA) and `manifestChecksums` for instant Level 0 (0ms) drift verification.
|
|
143
|
-
3. **Strict Context Isolation**: Decouples living project context from ephemeral task logs and loop execution states.
|
|
144
|
-
|
|
145
|
-
---
|
|
146
|
-
|
|
147
|
-
## CLI Interface & Commands
|
|
148
|
-
|
|
149
|
-
The CLI package is published on NPM as [`ai-engineering-loop`](https://www.npmjs.com/package/ai-engineering-loop) and operates against the current working directory.
|
|
150
|
-
|
|
151
|
-
```bash
|
|
152
|
-
# Bootstrap .ai-engineering-loop/ context from repository discovery
|
|
153
|
-
npx ai-engineering-loop init
|
|
154
|
-
|
|
155
|
-
# Check the validity, readiness, and baseline freshness of context
|
|
156
|
-
npx ai-engineering-loop status
|
|
157
|
-
|
|
158
|
-
# Reconcile drifted context against repository non-destructively
|
|
159
|
-
npx ai-engineering-loop refresh
|
|
160
|
-
|
|
161
|
-
# Verify context readiness and begin engineering loop
|
|
162
|
-
npx ai-engineering-loop run
|
|
163
|
-
```
|
|
164
|
-
|
|
165
|
-
---
|
|
166
|
-
|
|
167
|
-
## Antigravity Agent Integration
|
|
168
|
-
|
|
169
|
-
When working inside the Antigravity IDE or compatible agentic platforms, you can invoke the loop via slash commands:
|
|
170
|
-
|
|
171
|
-
- **`/ai-engineering-loop init`**: Initialize project context only (non-destructive bootstrap).
|
|
172
|
-
- **`/ai-engineering-loop status`**: Check repository context health & baseline freshness.
|
|
173
|
-
- **`/ai-engineering-loop refresh`**: Reconcile drifted context files non-destructively.
|
|
174
|
-
- **`/ai-engineering-loop [task description]`**: Execute the full 8-stage engineering lifecycle with pre-task drift gate and post-task impact assessment.
|
|
175
|
-
|
|
176
|
-
---
|
|
177
|
-
|
|
178
|
-
## Repository Structure
|
|
179
|
-
|
|
180
|
-
```text
|
|
181
|
-
ai-engineering-loop/
|
|
182
|
-
│
|
|
183
|
-
├── README.md # Operating system overview & architecture
|
|
184
|
-
├── LICENSE # MIT Open Source License
|
|
185
|
-
├── package.json # CLI package manifest
|
|
186
|
-
│
|
|
187
|
-
├── bin/ # CLI execution entrypoints
|
|
188
|
-
│ └── ai-engineering-loop.js # npx executable CLI (init, status, refresh, run)
|
|
189
|
-
│
|
|
190
|
-
├── lib/ # Core orchestration & decision engine
|
|
191
|
-
│ └── orchestration.js # 3-stage capability registry, barrier builder, Judge engine
|
|
192
|
-
│
|
|
193
|
-
├── tests/ # Deterministic test suites
|
|
194
|
-
│ ├── capability-selection.test.js # Unit tests for capability lifecycle & truthful selection
|
|
195
|
-
│ └── orchestration.test.js # Tests for isolation, Finding schema, Judge matrix
|
|
196
|
-
│
|
|
197
|
-
├── core/ # Generic engineering loop specifications
|
|
198
|
-
│ ├── orchestration-model.md # 3-stage capability lifecycle & execution priority
|
|
199
|
-
│ ├── project-initialization.md # Auto-discovery & initialization lifecycle
|
|
200
|
-
│ ├── context-refresh-policy.md # Progressive drift hierarchy & living baseline
|
|
201
|
-
│ ├── context-impact-assessment.md # Post-task impact assessment (NONE, TARGETED, MAJOR)
|
|
202
|
-
│ ├── goal-contract.md # Task contract schema & acceptance criteria
|
|
203
|
-
│ ├── verification-loop.md # Dual-layer verification & Evidence Contract
|
|
204
|
-
│ ├── definition-of-done.md # 5 pillars of Done & rejection triggers
|
|
205
|
-
│ ├── iteration-policy.md # Bounded autonomous loop (MAX_ITERATIONS = 3)
|
|
206
|
-
│ ├── escalation-policy.md # Deterministic human escalation triggers
|
|
207
|
-
│ ├── judge-policy.md # Evaluation rules, triage audit, & verdicts
|
|
208
|
-
│ ├── configuration-precedence.md # 5-layer precedence & conflict resolution
|
|
209
|
-
│ └── repo-config-schema.md # Schema for target repo .ai-engineering-loop/
|
|
210
|
-
│
|
|
211
|
-
├── profiles/ # Project archetype profiles
|
|
212
|
-
│ ├── README.md # Profile catalog & auto-detection rules
|
|
213
|
-
│ ├── web-app.md # Frontend web applications
|
|
214
|
-
│ ├── backend-api.md # Backend APIs & microservices
|
|
215
|
-
│ ├── mobile-app.md # Native & cross-platform mobile apps
|
|
216
|
-
│ ├── library.md # Reusable SDKs & shared packages
|
|
217
|
-
│ └── monorepo.md # Multi-package monorepo workspaces
|
|
218
|
-
│
|
|
219
|
-
├── agents/ # Triad agent role specifications
|
|
220
|
-
│ ├── maker.md # Maker agent: surgical diffs & unit tests
|
|
221
|
-
│ ├── devil-advocate.md # Adversarial reviewer: dual-axis finding ledger & diffs
|
|
222
|
-
│ └── judge.md # Judge agent: impartial magistrate on Validity + Severity
|
|
223
|
-
│
|
|
224
|
-
├── policies/ # Operational schemas & algorithms
|
|
225
|
-
│ ├── discovery-safety-policy.md # Secret protection & non-destructive discovery rules
|
|
226
|
-
│ ├── finding-policy.md # Dual-axis finding schema & severity matrix
|
|
227
|
-
│ ├── evidence-policy.md # 5-level evidence hierarchy & Verification Evidence Contract
|
|
228
|
-
│ └── no-progress-policy.md # Finding signature hashing & stagnation detection
|
|
229
|
-
│
|
|
230
|
-
├── adapters/ # Pluggable delivery pipelines
|
|
231
|
-
│ └── dot/ # DOT Indonesia delivery adapter
|
|
232
|
-
│ ├── README.md # DOT adapter overview
|
|
233
|
-
│ ├── gitlab.md # glab CLI, issue cards, & MR generation
|
|
234
|
-
│ ├── multi-branch.md # main / staging / develop cherry-pick propagation
|
|
235
|
-
│ ├── coreview.md # @coreview-bot external review triage (Valid vs Halu)
|
|
236
|
-
│ └── mattermost.md # Channel mapping & MCP dispatch (from: "AI Agent")
|
|
237
|
-
│
|
|
238
|
-
└── templates/ # Starter templates for target repositories
|
|
239
|
-
└── repo-config/ # Ready-to-copy .ai-engineering-loop/ files
|
|
240
|
-
├── config.md # Project identity & profile binding
|
|
241
|
-
├── architecture.md # Layers & boundary invariants
|
|
242
|
-
├── conventions.md # Code standards & forbidden patterns
|
|
243
|
-
├── verification.md # CLI test/lint/build commands
|
|
244
|
-
└── adapter.md # Configured release pipeline
|
|
245
|
-
```
|
|
246
|
-
|
|
247
|
-
---
|
|
248
|
-
|
|
249
|
-
## License
|
|
250
|
-
|
|
251
|
-
This project is licensed under the **MIT License** — see the [LICENSE](LICENSE) file for details.
|