ai-engineering-loop 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +272 -0
- package/adapters/dot/README.md +55 -0
- package/adapters/dot/coreview.md +88 -0
- package/adapters/dot/gitlab.md +128 -0
- package/adapters/dot/mattermost.md +102 -0
- package/adapters/dot/multi-branch.md +89 -0
- package/agents/devil-advocate.md +111 -0
- package/agents/judge.md +69 -0
- package/agents/maker.md +66 -0
- package/bin/ai-engineering-loop.js +633 -0
- package/core/configuration-precedence.md +102 -0
- package/core/context-impact-assessment.md +127 -0
- package/core/context-refresh-policy.md +116 -0
- package/core/definition-of-done.md +79 -0
- package/core/escalation-policy.md +99 -0
- package/core/goal-contract.md +88 -0
- package/core/iteration-policy.md +108 -0
- package/core/judge-policy.md +123 -0
- package/core/project-initialization.md +97 -0
- package/core/repo-config-schema.md +72 -0
- package/core/verification-loop.md +97 -0
- package/docs/antigravity-feasibility.md +90 -0
- package/docs/migration-plan.md +70 -0
- package/examples/backend-api/payment-idempotency/README.md +33 -0
- package/examples/backend-api/payment-idempotency/goal-contract.md +33 -0
- package/examples/backend-api/payment-idempotency/judge-verdict.md +28 -0
- package/examples/backend-api/payment-idempotency/review-findings.md +50 -0
- package/examples/dot/attendance-confirmation/README.md +22 -0
- package/examples/dot/attendance-confirmation/delivery-report.md +51 -0
- package/examples/dot/attendance-confirmation/goal-contract.md +39 -0
- package/examples/dot/attendance-confirmation/judge-verdict.md +43 -0
- package/examples/dot/attendance-confirmation/review-findings.md +57 -0
- package/examples/initialization/README.md +19 -0
- package/examples/initialization/discovery-trace.md +63 -0
- package/examples/initialization/generated-context.md +110 -0
- package/examples/mobile-app/offline-sync-queue/README.md +33 -0
- package/examples/mobile-app/offline-sync-queue/goal-contract.md +32 -0
- package/examples/mobile-app/offline-sync-queue/judge-verdict.md +27 -0
- package/examples/mobile-app/offline-sync-queue/review-findings.md +30 -0
- package/package.json +31 -0
- package/policies/discovery-safety-policy.md +51 -0
- package/policies/evidence-policy.md +70 -0
- package/policies/finding-policy.md +102 -0
- package/policies/no-progress-policy.md +92 -0
- package/profiles/README.md +42 -0
- package/profiles/backend-api.md +64 -0
- package/profiles/library.md +51 -0
- package/profiles/mobile-app.md +59 -0
- package/profiles/monorepo.md +46 -0
- package/profiles/web-app.md +65 -0
- package/scripts/init.sh +85 -0
- package/templates/repo-config/adapter.md +11 -0
- package/templates/repo-config/architecture.md +15 -0
- package/templates/repo-config/config.md +11 -0
- package/templates/repo-config/conventions.md +16 -0
- package/templates/repo-config/verification.md +13 -0
|
@@ -0,0 +1,102 @@
|
|
|
1
|
+
# Configuration Precedence & Resolution Engine
|
|
2
|
+
|
|
3
|
+
## 1. Overview & Core Principle
|
|
4
|
+
|
|
5
|
+
To support multiple repositories, varied tech stacks, and disparate team conventions without fragmenting or forking the generic core, the AI Engineering Loop implements a **5-Layer Configuration Hierarchy**.
|
|
6
|
+
|
|
7
|
+
The core principle of configuration resolution is:
|
|
8
|
+
|
|
9
|
+
> **The more specific layer refines or overrides the broader layer, provided it does not violate immutable safety boundaries.**
|
|
10
|
+
|
|
11
|
+
```mermaid
|
|
12
|
+
flowchart TD
|
|
13
|
+
Init[LIFECYCLE STAGE 0: Project Initialization & Discovery] --> Res[Dynamic Configuration Hierarchy]
|
|
14
|
+
|
|
15
|
+
subgraph Layers [5-LAYER HIERARCHY]
|
|
16
|
+
L1[Layer 1: GLOBAL ENVIRONMENT<br>Host safety ceilings, execution timeouts, max iteration cap]
|
|
17
|
+
L2[Layer 2: ENGINEERING CORE<br>Contract-driven loop, deterministic precedence, DoD, triad roles]
|
|
18
|
+
L3[Layer 3: PROJECT TYPE PROFILE<br>Archetype defaults: web-app, backend-api, mobile-app, library, monorepo]
|
|
19
|
+
L4[Layer 4: REPOSITORY LOCAL CONFIG<br>.ai-engineering-loop/ auto-discovered or loaded]
|
|
20
|
+
L5[Layer 5: TASK CONTRACT<br>Goal contract for current execution run]
|
|
21
|
+
|
|
22
|
+
L1 --> L2 --> L3 --> L4 --> L5
|
|
23
|
+
end
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
---
|
|
27
|
+
|
|
28
|
+
## 2. The 5 Hierarchy Layers
|
|
29
|
+
|
|
30
|
+
### Layer 1: Global Environment
|
|
31
|
+
- **Authority**: System / Host Platform (Antigravity IDE, CLI environment).
|
|
32
|
+
- **Scope**: Hard safety constraints (e.g. `ABSOLUTE_MAX_ITERATIONS = 5`, maximum command timeout = 10m, safe file path boundaries).
|
|
33
|
+
- **Overridability**: **Immutable**. Cannot be overridden by downstream layers.
|
|
34
|
+
|
|
35
|
+
### Layer 2: Engineering Core (`core/`)
|
|
36
|
+
- **Authority**: AI Engineering Operating System specification.
|
|
37
|
+
- **Scope**: Mandatory engineering invariants:
|
|
38
|
+
- No code changes without a formalized [Goal Contract](file:///Users/egagofur/Development/work/ai-engineering-loop/core/goal-contract.md).
|
|
39
|
+
- Deterministic checks must pass 100% before adversarial review.
|
|
40
|
+
- Review findings require reproducible evidence (Level 1–3) to be valid.
|
|
41
|
+
- Only the [Judge Agent](file:///Users/egagofur/Development/work/ai-engineering-loop/agents/judge.md) can issue a `PASS` verdict.
|
|
42
|
+
- **Overridability**: Invariant. Downstream layers cannot skip verification or eliminate agent roles.
|
|
43
|
+
|
|
44
|
+
### Layer 3: Project Type Profile (`profiles/`)
|
|
45
|
+
- **Authority**: Archetype profiles (`web-app`, `backend-api`, `mobile-app`, `library`, `monorepo`).
|
|
46
|
+
- **Scope**: Tech stack defaults:
|
|
47
|
+
- Relevant review domains for Devil's Advocate (e.g. activating database transaction reviews for APIs, responsive UI checks for web apps).
|
|
48
|
+
- Typical testing patterns (e.g. Jest/Vitest for web, pytest/go test for backend, XCTest/Espresso for mobile).
|
|
49
|
+
- **Overridability**: Overridden by explicit repository-local configuration.
|
|
50
|
+
|
|
51
|
+
### Layer 4: Repository-Local Configuration (`.ai-engineering-loop/`)
|
|
52
|
+
- **Authority**: Target repository checked-in or auto-initialized files.
|
|
53
|
+
- **Scope**: Repository-specific ground truth:
|
|
54
|
+
- Exact test/lint/typecheck commands in `verification.md`.
|
|
55
|
+
- Layer definitions in `architecture.md`.
|
|
56
|
+
- Architectural constraints and forbidden patterns in `conventions.md`.
|
|
57
|
+
- Configured delivery adapter (DOT, GitHub, GitLab) in `adapter.md`.
|
|
58
|
+
- **Auto-Discovery**: Automatically generated if missing via [Project Initialization](file:///Users/egagofur/Development/work/ai-engineering-loop/core/project-initialization.md).
|
|
59
|
+
- **Overridability**: Overrides Layer 3 defaults for this specific codebase.
|
|
60
|
+
|
|
61
|
+
### Layer 5: Task Contract
|
|
62
|
+
- **Authority**: Current execution request / prompt.
|
|
63
|
+
- **Scope**: Task-specific Acceptance Criteria (AC-1..N), out-of-scope boundaries, and temporary overrides.
|
|
64
|
+
- **Overridability**: Scoped strictly to the active task run.
|
|
65
|
+
|
|
66
|
+
---
|
|
67
|
+
|
|
68
|
+
## 3. Precedence Resolution Algorithm
|
|
69
|
+
|
|
70
|
+
When the engine executes a task, it resolves configuration keys using the following lookup order:
|
|
71
|
+
|
|
72
|
+
```python
|
|
73
|
+
def resolve_config_key(key: str, context: ExecutionContext) -> Any:
|
|
74
|
+
# 1. Task Contract (highest priority for task-scoped values)
|
|
75
|
+
if context.task_contract.has(key):
|
|
76
|
+
return context.task_contract.get(key)
|
|
77
|
+
|
|
78
|
+
# 2. Repository Local Configuration (.ai-engineering-loop/)
|
|
79
|
+
if context.repo_config.has(key):
|
|
80
|
+
return context.repo_config.get(key)
|
|
81
|
+
|
|
82
|
+
# 3. Project Type Profile (profiles/<type>.md)
|
|
83
|
+
if context.profile.has(key):
|
|
84
|
+
return context.profile.get(key)
|
|
85
|
+
|
|
86
|
+
# 4. Engineering Core Defaults
|
|
87
|
+
if context.core_defaults.has(key):
|
|
88
|
+
return context.core_defaults.get(key)
|
|
89
|
+
|
|
90
|
+
# 5. Global Environment Safety
|
|
91
|
+
return context.global_env.get(key)
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
---
|
|
95
|
+
|
|
96
|
+
## 4. Conflict Resolution & Escalation Rules
|
|
97
|
+
|
|
98
|
+
1. **Refinement vs Contradiction**:
|
|
99
|
+
- *Refinement (Allowed)*: Profile says "Run unit tests"; Repo config specifies `pnpm run test:unit`. $\rightarrow$ Valid refinement.
|
|
100
|
+
- *Contradiction (Escalate)*: Core requires deterministic verification; Task contract requests skipping tests to merge faster. $\rightarrow$ **Forbidden**. The engine rejects the override and flags a policy violation.
|
|
101
|
+
2. **Unresolvable Contradictions**:
|
|
102
|
+
- If repository configuration directly contradicts an established platform invariant (e.g. a command requires root sudo or drops protected tables), the system halts and triggers [Human Escalation](file:///Users/egagofur/Development/work/ai-engineering-loop/core/escalation-policy.md).
|
|
@@ -0,0 +1,127 @@
|
|
|
1
|
+
# Context Impact Assessment Specification
|
|
2
|
+
|
|
3
|
+
## 1. Overview & Core Philosophy
|
|
4
|
+
|
|
5
|
+
In the AI Engineering Loop, project context (`.ai-engineering-loop/`) is a **Living Knowledge Base** that reflects the current engineering reality of the repository.
|
|
6
|
+
|
|
7
|
+
However, performing an expensive full-repository re-analysis after every completed task introduces unnecessary overhead and latency. Most software changes (e.g. fixing a UI bug, tweaking validation logic, or adding a test) do **not** alter the project's architecture, conventions, or build commands.
|
|
8
|
+
|
|
9
|
+
The **Context Impact Assessment** is a lightweight, evidence-based evaluation executed immediately after a task reaches a **Judge `PASS`** verdict, determining whether and how `.ai-engineering-loop/` should be updated before closing the session.
|
|
10
|
+
|
|
11
|
+
```mermaid
|
|
12
|
+
flowchart TD
|
|
13
|
+
JudgePass([Judge Agent: PASS Verdict]) --> Assess[Context Impact Assessment]
|
|
14
|
+
|
|
15
|
+
Assess --> Level{Determine Impact Level}
|
|
16
|
+
|
|
17
|
+
Level -->|NONE| Skip[No Context Update Required<br>Task Finished]
|
|
18
|
+
Level -->|TARGETED| Partial[Surgical Update of Affected Files<br>Update metadata.json Baseline]
|
|
19
|
+
Level -->|MAJOR| Full[Full Reconciliation Pass<br>Update metadata.json Baseline]
|
|
20
|
+
|
|
21
|
+
Skip --> End([Session Complete])
|
|
22
|
+
Partial --> End
|
|
23
|
+
Full --> End
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
---
|
|
27
|
+
|
|
28
|
+
## 2. The 3 Context Impact Levels
|
|
29
|
+
|
|
30
|
+
```text
|
|
31
|
+
┌─────────────────┬──────────────────────────────────────────┬─────────────────────────────┐
|
|
32
|
+
│ Impact Level │ Task Characteristics │ System Action │
|
|
33
|
+
├─────────────────┼──────────────────────────────────────────┼─────────────────────────────┤
|
|
34
|
+
│ NONE │ Isolated bugfix, UI tweak, test-only, │ NO-OP (Skip refresh) │
|
|
35
|
+
│ (80-90% tasks) │ small refactor preserving architecture. │ │
|
|
36
|
+
├─────────────────┼──────────────────────────────────────────┼─────────────────────────────┤
|
|
37
|
+
│ TARGETED │ Manifest scripts changed, new module, │ Surgically reconcile only │
|
|
38
|
+
│ (10-15% tasks) │ new team invariant, new CI config. │ affected files in context. │
|
|
39
|
+
├─────────────────┼──────────────────────────────────────────┼─────────────────────────────┤
|
|
40
|
+
│ MAJOR │ Framework migration, database redesign, │ Full repository context │
|
|
41
|
+
│ (< 5% tasks) │ monorepo restructuring, auth overhaul. │ reconciliation pass. │
|
|
42
|
+
└─────────────────┴──────────────────────────────────────────┴─────────────────────────────┘
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
---
|
|
46
|
+
|
|
47
|
+
## 3. Impact Level Classification Matrix
|
|
48
|
+
|
|
49
|
+
### Level 1: `NONE` (No Action)
|
|
50
|
+
- **Criteria**:
|
|
51
|
+
- The task diff only touches existing function bodies, templates, styles, or test files.
|
|
52
|
+
- Zero new external dependencies or build scripts introduced.
|
|
53
|
+
- Directory hierarchy and layer boundaries remain unchanged.
|
|
54
|
+
- **Examples**:
|
|
55
|
+
- Fixing a typo or styling bug in `src/components/Button.tsx`.
|
|
56
|
+
- Fixing an off-by-one error in `src/utils/date.ts`.
|
|
57
|
+
- Adding edge-case unit tests to `user.service.spec.ts`.
|
|
58
|
+
- **Action**: Output `Context Impact: NONE`. Do not modify `.ai-engineering-loop/`.
|
|
59
|
+
|
|
60
|
+
---
|
|
61
|
+
|
|
62
|
+
### Level 2: `TARGETED` (Surgical Reconciliation)
|
|
63
|
+
- **Criteria**:
|
|
64
|
+
- A specific dimension of project knowledge has become stale due to the diff.
|
|
65
|
+
- **Mapping by File / Diff Signal**:
|
|
66
|
+
|
|
67
|
+
| Changed Area in Task Diff | Stale Context File | Targeted Action |
|
|
68
|
+
|---|---|---|
|
|
69
|
+
| `package.json` / `go.mod` scripts modified | `verification.md`, `config.md` | Update test/build/lint command entries |
|
|
70
|
+
| New folder in `src/modules/` or `apps/` | `architecture.md` | Add module summary and boundary notes |
|
|
71
|
+
| New global error class or lint rule added | `conventions.md` | Document new pattern or forbidden rule |
|
|
72
|
+
| `.gitlab-ci.yml` / `.github/workflows` edited | `adapter.md` | Update CI/CD workflow references |
|
|
73
|
+
|
|
74
|
+
- **Action**: Reconcile *only* the affected markdown file(s) and update `metadata.json` baseline.
|
|
75
|
+
|
|
76
|
+
---
|
|
77
|
+
|
|
78
|
+
### Level 3: `MAJOR` (Full Context Reconciliation)
|
|
79
|
+
- **Criteria**:
|
|
80
|
+
- The diff introduces a paradigm shift in how the codebase compiles, runs, or organizes architecture.
|
|
81
|
+
- **Examples**:
|
|
82
|
+
- Migrating from Next.js Pages Router to App Router.
|
|
83
|
+
- Converting a single repository into a Turborepo/pnpm monorepo.
|
|
84
|
+
- Migrating ORM from TypeORM to Prisma or replacing REST with tRPC.
|
|
85
|
+
- Introducing a complete OAuth2 / JWT authentication architecture.
|
|
86
|
+
- **Action**: Execute full 5-pass reconciliation, update all relevant `.ai-engineering-loop/` files, and establish a new baseline in `metadata.json`.
|
|
87
|
+
|
|
88
|
+
---
|
|
89
|
+
|
|
90
|
+
## 4. Devil's Advocate & Judge Drift Interaction
|
|
91
|
+
|
|
92
|
+
During task review, the [Devil's Advocate](file:///Users/egagofur/Development/work/ai-engineering-loop/agents/devil-advocate.md) may discover that the codebase has drifted from the documented architecture:
|
|
93
|
+
|
|
94
|
+
> *Example Review Finding*: "The implementation assumes `user-service` connects directly to Postgres, but the codebase has migrated to an asynchronous event queue in `src/events/`."
|
|
95
|
+
|
|
96
|
+
### Triage Protocol:
|
|
97
|
+
1. The **Devil's Advocate** flags the discrepancy in its Finding Ledger.
|
|
98
|
+
2. The **Judge Agent** evaluates the finding:
|
|
99
|
+
- If `VALID`: The Judge flags an active architectural drift.
|
|
100
|
+
- Upon task completion, the Context Impact Assessment automatically triggers a `TARGETED` or `MAJOR` refresh of `architecture.md` to capture the new reality.
|
|
101
|
+
- If `INVALID` (e.g. reviewer hallucination), no context update occurs.
|
|
102
|
+
|
|
103
|
+
---
|
|
104
|
+
|
|
105
|
+
## 5. Output Reporting Protocol
|
|
106
|
+
|
|
107
|
+
Every completed engineering run concludes with a **Context Impact Summary**:
|
|
108
|
+
|
|
109
|
+
### Example: Impact Level NONE
|
|
110
|
+
```text
|
|
111
|
+
Project Context Assessment
|
|
112
|
+
- Impact Level: NONE
|
|
113
|
+
- Reason: Surgical bugfix in src/services/attendance.ts preserved existing architecture and commands.
|
|
114
|
+
- Action: Context refresh SKIPPED (context is CURRENT).
|
|
115
|
+
```
|
|
116
|
+
|
|
117
|
+
### Example: Impact Level TARGETED
|
|
118
|
+
```text
|
|
119
|
+
Project Context Assessment
|
|
120
|
+
- Impact Level: TARGETED
|
|
121
|
+
- Reason: Added Redis client and caching layer in src/infrastructure/cache.ts.
|
|
122
|
+
- Affected Context Files:
|
|
123
|
+
- architecture.md (added Cache Layer boundary)
|
|
124
|
+
- config.md (added Redis dependency)
|
|
125
|
+
- Action: Surgically updated architecture.md and config.md.
|
|
126
|
+
- Baseline: Updated to commit a1b2c3d.
|
|
127
|
+
```
|
|
@@ -0,0 +1,116 @@
|
|
|
1
|
+
# Context Refresh & Living Baseline Policy
|
|
2
|
+
|
|
3
|
+
## 1. Overview & Core Laws
|
|
4
|
+
|
|
5
|
+
The **Context Refresh Policy** defines how the AI Engineering Loop maintains living synchronization between `.ai-engineering-loop/` and the underlying repository without unnecessary computation.
|
|
6
|
+
|
|
7
|
+
### Three Fundamental Invariants:
|
|
8
|
+
1. **Reconciliation, Not Destruction**: Refresh never deletes `.ai-engineering-loop/` or wipes manual developer notes. It reconciles existing context against current repository facts.
|
|
9
|
+
2. **Progressive Cost**: Drift detection uses cheap signals (Level 0) before performing deeper inspections (Levels 1–3).
|
|
10
|
+
3. **Strict Context Isolation**:
|
|
11
|
+
- **Project Context (`.ai-engineering-loop/`)**: Shared, living architectural rules and verification commands.
|
|
12
|
+
- **Task Context**: Ephemeral task contracts, specific diff scopes, and temporary session data.
|
|
13
|
+
- **Loop State**: Transient retry counters, test outputs, and finding signatures.
|
|
14
|
+
*Task execution histories and conversational chat transcripts are NEVER dumped into `.ai-engineering-loop/`.*
|
|
15
|
+
|
|
16
|
+
---
|
|
17
|
+
|
|
18
|
+
## 2. Context Baseline Metadata (`metadata.json`)
|
|
19
|
+
|
|
20
|
+
To enable deterministic, instant drift checks without requiring a database, the `.ai-engineering-loop/` directory maintains a lightweight `metadata.json` baseline:
|
|
21
|
+
|
|
22
|
+
```json
|
|
23
|
+
{
|
|
24
|
+
"contextVersion": "1.0.0",
|
|
25
|
+
"generatedAt": "2026-08-25T10:00:00Z",
|
|
26
|
+
"repositoryRevision": "7f8b9a1c2d3e...",
|
|
27
|
+
"projectProfile": "backend-api",
|
|
28
|
+
"manifestChecksums": {
|
|
29
|
+
"package.json": "a3f5b2c1...",
|
|
30
|
+
"go.mod": "e9b8c7d6..."
|
|
31
|
+
},
|
|
32
|
+
"lastReconciliation": {
|
|
33
|
+
"timestamp": "2026-08-25T10:00:00Z",
|
|
34
|
+
"trigger": "init",
|
|
35
|
+
"impact": "INITIAL_BOOTSTRAP"
|
|
36
|
+
}
|
|
37
|
+
}
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
---
|
|
41
|
+
|
|
42
|
+
## 3. Progressive Multi-Level Drift Detection Hierarchy
|
|
43
|
+
|
|
44
|
+
Drift detection ascends through four progressively detailed levels:
|
|
45
|
+
|
|
46
|
+
```mermaid
|
|
47
|
+
flowchart TD
|
|
48
|
+
Start([Check Context Freshness]) --> L0[Level 0: Cheap Signal Comparison<br>git rev-parse HEAD & manifest checksums]
|
|
49
|
+
|
|
50
|
+
L0 -->|Match: Zero Diff| Current([Status: CURRENT<br>Cost: 0ms])
|
|
51
|
+
L0 -->|Mismatch: Change Detected| L1[Level 1: Inspect Touched Files<br>git diff --name-only baseline..HEAD]
|
|
52
|
+
|
|
53
|
+
L1 -->|Only Non-Architectural Files| Current
|
|
54
|
+
L1 -->|Manifest or Boundary Files Touched| L2[Level 2: Targeted Context Analysis]
|
|
55
|
+
|
|
56
|
+
L2 -->|Surgical Change| SurgReconcile[Surgically Update Affected Files]
|
|
57
|
+
L2 -->|Structural / Paradigm Shift| L3[Level 3: Full Context Reconciliation]
|
|
58
|
+
|
|
59
|
+
SurgReconcile --> UpdateMeta[Update metadata.json Baseline] --> Done([Context Synchronized])
|
|
60
|
+
L3 --> UpdateMeta --> Done
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
### Level 0: Instant Cheap Signal Comparison (Cost: ~0ms)
|
|
64
|
+
- Compare current `git rev-parse HEAD` and manifest hashes (`package.json`, `go.mod`, etc.) against `metadata.json`.
|
|
65
|
+
- If identical $\rightarrow$ status is **`CURRENT`**. Stop immediately.
|
|
66
|
+
|
|
67
|
+
### Level 1: Touched File Scope Inspection (Cost: ~50ms)
|
|
68
|
+
- If HEAD has advanced (e.g. other developers merged commits), inspect touched file paths via `git diff --name-only <baseline>..HEAD`.
|
|
69
|
+
- If all changed files are isolated application logic, styles, or tests (e.g. `src/views/Profile.vue`, `src/utils/math.ts`) $\rightarrow$ status is **`CURRENT`**.
|
|
70
|
+
|
|
71
|
+
### Level 2: Targeted Context Analysis (Cost: ~200ms)
|
|
72
|
+
- If manifests, top-level directory layouts, or CI workflows were touched, re-read those specific files and reconcile only the corresponding markdown files (`verification.md`, `architecture.md`, `adapter.md`).
|
|
73
|
+
|
|
74
|
+
### Level 3: Full Repository Context Reconciliation (Cost: ~1-2s)
|
|
75
|
+
- Triggered only when major structural shifts occur (e.g. monorepo workspace addition, framework migration). Re-runs full 5-pass discovery.
|
|
76
|
+
|
|
77
|
+
---
|
|
78
|
+
|
|
79
|
+
## 4. Pre-Task Drift Gate
|
|
80
|
+
|
|
81
|
+
Before starting any task, the engine executes the Pre-Task Drift Gate:
|
|
82
|
+
|
|
83
|
+
```text
|
|
84
|
+
Load Project Context
|
|
85
|
+
↓
|
|
86
|
+
Level 0 / Level 1 Drift Check
|
|
87
|
+
↓
|
|
88
|
+
┌────────────────────────────────────────┐
|
|
89
|
+
│ CURRENT │
|
|
90
|
+
│ → Proceed directly to Goal Contract │
|
|
91
|
+
│ │
|
|
92
|
+
│ STALE (Drift Detected) │
|
|
93
|
+
│ → Reconcile affected context files │
|
|
94
|
+
│ → Update baseline metadata │
|
|
95
|
+
│ → Proceed to Goal Contract │
|
|
96
|
+
└────────────────────────────────────────┘
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
This guarantees that an AI agent never writes code using outdated build commands or obsolete architectural assumptions.
|
|
100
|
+
|
|
101
|
+
---
|
|
102
|
+
|
|
103
|
+
## 5. Scheduled Maintenance / Heartbeat
|
|
104
|
+
|
|
105
|
+
- **Role**: A secondary safety net catching external repository modifications (e.g. external PRs merged without the loop).
|
|
106
|
+
- **Protocol**: Executes Level 0/Level 1 drift detection periodically (e.g. every 12h or 24h).
|
|
107
|
+
- **Rule**: If Level 0/1 detects no material changes, it **STOPS immediately**. It NEVER performs a full re-analysis simply because a timer fired.
|
|
108
|
+
|
|
109
|
+
---
|
|
110
|
+
|
|
111
|
+
## 6. Anti-Infinite-Loop Safeguard
|
|
112
|
+
|
|
113
|
+
To prevent circular update loops (`Task -> Refresh -> Context Changes -> Refresh -> ...`):
|
|
114
|
+
1. Every refresh operation concludes by updating `metadata.json` to the current repository revision (`repositoryRevision = git rev-parse HEAD`).
|
|
115
|
+
2. Setting the new baseline guarantees that subsequent Level 0 checks evaluate to `CURRENT`.
|
|
116
|
+
3. The refresh engine is strictly **idempotent**: running `refresh` multiple times on unchanged code produces 0 file modifications and terminates with *"Context is already current"*.
|
|
@@ -0,0 +1,79 @@
|
|
|
1
|
+
# Definition of Done (DoD) Specification
|
|
2
|
+
|
|
3
|
+
## 1. Principle & Definition
|
|
4
|
+
|
|
5
|
+
The **Definition of Done (DoD)** is the objective, non-negotiable threshold that must be satisfied before any task can be declared complete, merged, or handed off to downstream delivery pipelines.
|
|
6
|
+
|
|
7
|
+
The core question answered by the DoD is not:
|
|
8
|
+
> *"Does the agent believe it solved the problem?"*
|
|
9
|
+
|
|
10
|
+
The DoD answers:
|
|
11
|
+
> **"Has the system provided verifiable proof satisfying all technical and business criteria?"**
|
|
12
|
+
|
|
13
|
+
---
|
|
14
|
+
|
|
15
|
+
## 2. The 5 Pillars of Done
|
|
16
|
+
|
|
17
|
+
To achieve a `DONE` status, an engineering task must satisfy all five pillars without exception:
|
|
18
|
+
|
|
19
|
+
```mermaid
|
|
20
|
+
flowchart LR
|
|
21
|
+
P1[1. Contract Satisfaction] --> DoD{DEFINITION OF DONE}
|
|
22
|
+
P2[2. Deterministic Verification] --> DoD
|
|
23
|
+
P3[3. Code & Diff Quality] --> DoD
|
|
24
|
+
P4[4. Adversarial Consensus] --> DoD
|
|
25
|
+
P5[5. Judge Certified PASS] --> DoD
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
### Pillar 1: Contract Satisfaction
|
|
29
|
+
- 100% of Acceptance Criteria defined in the [Goal Contract](file:///Users/egagofur/Development/work/ai-engineering-loop/core/goal-contract.md) are demonstrably met.
|
|
30
|
+
- Zero out-of-scope files or unauthorized modules were modified.
|
|
31
|
+
- All technical constraints (e.g. backward compatibility, no unapproved dependencies) are preserved.
|
|
32
|
+
|
|
33
|
+
### Pillar 2: Deterministic Verification
|
|
34
|
+
- **Unit & Regression Tests**: 100% passing tests (0 failures, 0 errors, 0 unresolved broken suites).
|
|
35
|
+
- **Static Typing / Compilation**: 0 type errors (e.g. `tsc --noEmit` exits with `0`).
|
|
36
|
+
- **Linting & Code Formatting**: 0 lint errors on modified files.
|
|
37
|
+
- **Build / Packaging**: Project builds successfully without warnings treated as errors.
|
|
38
|
+
|
|
39
|
+
### Pillar 3: Code & Diff Quality
|
|
40
|
+
- **Surgical Diff**: Smallest coherent diff that completely resolves the issue.
|
|
41
|
+
- **Architecture Preservation**: Adheres to existing repository patterns, naming conventions, and layer boundaries.
|
|
42
|
+
- **Zero Placeholders**: No stubbed functions, empty `catch` blocks, speculative `TODO` comments, or orphaned dead code.
|
|
43
|
+
- **Null & Boundary Safety**: Explicit handling of `null`, `undefined`, empty collections, and error paths.
|
|
44
|
+
|
|
45
|
+
### Pillar 4: Adversarial Consensus
|
|
46
|
+
- Independent [Devil's Advocate Review](file:///Users/egagofur/Development/work/ai-engineering-loop/agents/devil-advocate.md) has been executed across all 6 core review domains.
|
|
47
|
+
- **Zero Unresolved Blocking Findings**: No open `SEV-1 (Critical)` or `SEV-2 (High)` findings.
|
|
48
|
+
- **Evidence-Based Triage**: Every raised finding has been formally triaged with reproducible evidence as `VALID` (and resolved in code) or `INVALID` (with technical proof of why it is a false positive).
|
|
49
|
+
|
|
50
|
+
### Pillar 5: Judge Certified PASS
|
|
51
|
+
- The [Judge Agent](file:///Users/egagofur/Development/work/ai-engineering-loop/agents/judge.md) has evaluated the complete execution trace, verified the evidence, confirmed no-progress limits were not violated, and issued a signed `PASS` verdict.
|
|
52
|
+
|
|
53
|
+
---
|
|
54
|
+
|
|
55
|
+
## 3. DoD Verification Checklist
|
|
56
|
+
|
|
57
|
+
Every iteration concluding in a `PASS` verdict must produce a DoD verification table:
|
|
58
|
+
|
|
59
|
+
| Checklist Item | Required Standard | Status | Evidence Reference |
|
|
60
|
+
|---|---|:---:|---|
|
|
61
|
+
| Acceptance Criteria AC-1..N | 100% satisfied | ✅ | Unit test file & assertion links |
|
|
62
|
+
| Test Suite Execution | 0 failures, 0 errors | ✅ | Test command output log |
|
|
63
|
+
| Typecheck / Compiler | Exit code 0 | ✅ | Typecheck log |
|
|
64
|
+
| Linter | 0 errors on diff | ✅ | Linter log |
|
|
65
|
+
| Build Check | Exit code 0 | ✅ | Build command log |
|
|
66
|
+
| Adversarial Review | Completed across 6 topics | ✅ | Review findings artifact |
|
|
67
|
+
| Blocking Findings (Sev 1/2)| 0 unresolved | ✅ | Triage summary table |
|
|
68
|
+
| Judge Verdict | PASS | ✅ | Judge evaluation report |
|
|
69
|
+
|
|
70
|
+
---
|
|
71
|
+
|
|
72
|
+
## 4. Rejection Criteria
|
|
73
|
+
|
|
74
|
+
An engineering run MUST be rejected and flagged as `FAIL` or `BLOCKED` if any of the following occur:
|
|
75
|
+
|
|
76
|
+
1. **Unverified Claims**: The agent states that a test passed or feature works without providing command outputs or test code.
|
|
77
|
+
2. **Post-Hoc Goal Shifting**: Modifying Acceptance Criteria to match buggy behavior instead of fixing the bug.
|
|
78
|
+
3. **Suppressed Errors**: Adding `@ts-ignore`, `eslint-disable`, empty catch blocks, or skipping tests to force a green build.
|
|
79
|
+
4. **Unresolved Critical Findings**: Attempting to declare completion while a `SEV-1` or `SEV-2` finding from the Devil's Advocate remains open.
|
|
@@ -0,0 +1,99 @@
|
|
|
1
|
+
# Escalation Policy Specification
|
|
2
|
+
|
|
3
|
+
## 1. Principle & Core Philosophy
|
|
4
|
+
|
|
5
|
+
The primary objective of the AI Engineering Loop is:
|
|
6
|
+
|
|
7
|
+
> **Reliable Autonomy, NOT Maximum Autonomy at All Costs.**
|
|
8
|
+
|
|
9
|
+
An autonomous agent that attempts to push through deep architectural ambiguities, unresolvable test regressions, or unverified business rules creates significant technical debt and risk.
|
|
10
|
+
|
|
11
|
+
The **Escalation Policy** defines deterministic thresholds where autonomous iteration MUST immediately halt and hand off control to a human engineer with an actionable, evidence-backed report.
|
|
12
|
+
|
|
13
|
+
---
|
|
14
|
+
|
|
15
|
+
## 2. Mandatory Escalation Triggers
|
|
16
|
+
|
|
17
|
+
An agent MUST trigger Human Escalation when any of the following conditions are met:
|
|
18
|
+
|
|
19
|
+
```mermaid
|
|
20
|
+
flowchart TD
|
|
21
|
+
E1[1. Max Iterations Reached] --> Escalate([TRIGGER HUMAN ESCALATION])
|
|
22
|
+
E2[2. No-Progress / Stalled Loop Detected] --> Escalate
|
|
23
|
+
E3[3. Contradictory / Impossible Requirements] --> Escalate
|
|
24
|
+
E4[4. Destructive / High-Risk Operations] --> Escalate
|
|
25
|
+
E5[5. Unresolvable Agent Disagreement] --> Escalate
|
|
26
|
+
E6[6. Irreproducible Failure] --> Escalate
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
### Trigger 1: Maximum Iterations Reached
|
|
30
|
+
- **Condition**: The system has completed `MAX_ITERATIONS` (default: 3) and unresolved blocking findings (`SEV-1` or `SEV-2`) still persist.
|
|
31
|
+
- **Rationale**: Prolonged retries indicate either an incorrect fundamental strategy or a hidden architectural obstacle.
|
|
32
|
+
|
|
33
|
+
### Trigger 2: No-Progress / Stagnation Detected
|
|
34
|
+
- **Condition**: Two consecutive iterations produce identical or semantically equivalent finding signatures without demonstrable code convergence (see [No-Progress Policy](file:///Users/egagofur/Development/work/ai-engineering-loop/policies/no-progress-policy.md)).
|
|
35
|
+
- **Rationale**: Prevents thrashing where the agent modifies code without resolving the underlying flaw.
|
|
36
|
+
|
|
37
|
+
### Trigger 3: Contradictory or Inconsistent Acceptance Criteria
|
|
38
|
+
- **Condition**: Satisfying Criterion A mathematically or logically violates Criterion B, or the Goal Contract contradicts an established database or platform invariant.
|
|
39
|
+
- **Rationale**: Agents must not invent business compromises without stakeholder input.
|
|
40
|
+
|
|
41
|
+
### Trigger 4: High-Risk / Destructive Operations
|
|
42
|
+
- **Condition**: The fix requires dropping production database columns, bypassing authentication/authorization layers, modifying global build pipelines, or upgrading major core dependencies.
|
|
43
|
+
- **Rationale**: High blast-radius architectural changes require human authorization.
|
|
44
|
+
|
|
45
|
+
### Trigger 5: Irreproducible Failure
|
|
46
|
+
- **Condition**: The agent is unable to reproduce the reported bug deterministically after exhaustive environment inspection and logging.
|
|
47
|
+
- **Rationale**: Prevents speculative fixes for phantom issues.
|
|
48
|
+
|
|
49
|
+
### Trigger 6: Unresolvable Triad Disagreement
|
|
50
|
+
- **Condition**: The Maker Agent and Devil's Advocate Agent present mutually incompatible, evidence-backed arguments regarding architecture or domain rules, and the Judge cannot definitively resolve it from existing codebase artifacts.
|
|
51
|
+
- **Rationale**: Domain-level business tradeoffs belong to product owners and human engineers.
|
|
52
|
+
|
|
53
|
+
---
|
|
54
|
+
|
|
55
|
+
## 3. Human Escalation Report Protocol
|
|
56
|
+
|
|
57
|
+
When escalating, the agent must NEVER output a generic message like *"I am stuck."*
|
|
58
|
+
|
|
59
|
+
Instead, it must render an **Actionable Escalation Report** following this structured template:
|
|
60
|
+
|
|
61
|
+
```markdown
|
|
62
|
+
# ⚠️ Human Escalation Triggered
|
|
63
|
+
|
|
64
|
+
## 1. Escalation Reason
|
|
65
|
+
**Trigger**: [e.g. Trigger 2: No-Progress / Stalled Loop Detected]
|
|
66
|
+
**Iteration**: [e.g. Iteration 3 of 3]
|
|
67
|
+
|
|
68
|
+
## 2. Summary of Attempted Solutions
|
|
69
|
+
- **Iteration 1**: [What was tried, what failed]
|
|
70
|
+
- **Iteration 2**: [What was modified, what remained unresolved]
|
|
71
|
+
- **Iteration 3**: [Current diff and exact blocker]
|
|
72
|
+
|
|
73
|
+
## 3. Core Technical Blocker
|
|
74
|
+
[Precise explanation of why the agent cannot proceed autonomously. Include code locations, conflicting constraints, or missing domain knowledge.]
|
|
75
|
+
|
|
76
|
+
## 4. Active Findings Ledger
|
|
77
|
+
| Finding ID | Severity | Category | Location | State | Summary |
|
|
78
|
+
|---|---|---|---|---|---|
|
|
79
|
+
| SEC-001 | HIGH | Security | `src/auth/guard.ts:42` | TRIAGED_VALID | IDOR risk when tenant ID is omitted |
|
|
80
|
+
|
|
81
|
+
## 5. Specific Decision Required from Human
|
|
82
|
+
[State 2-3 concrete options for the human to choose from, or a focused question]:
|
|
83
|
+
- **Option A**: [Description of architectural option A and tradeoffs]
|
|
84
|
+
- **Option B**: [Description of architectural option B and tradeoffs]
|
|
85
|
+
|
|
86
|
+
## 6. Current Workspace State
|
|
87
|
+
- **Branch**: `<current-working-branch>`
|
|
88
|
+
- **Uncommitted Changes**: [Clean / Stashed / In-flight diff]
|
|
89
|
+
- **Deterministic Test Status**: [Passing / Failing with command logs]
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
---
|
|
93
|
+
|
|
94
|
+
## 4. Resumption Protocol
|
|
95
|
+
|
|
96
|
+
Once the human engineer provides guidance or amends the [Goal Contract](file:///Users/egagofur/Development/work/ai-engineering-loop/core/goal-contract.md):
|
|
97
|
+
1. The iteration counter is reset: $K \leftarrow 1$.
|
|
98
|
+
2. The agent incorporates the human's decision into the Goal Contract constraints.
|
|
99
|
+
3. The loop resumes at the **Maker Agent** phase with clean validation.
|
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
# Goal Contract Specification
|
|
2
|
+
|
|
3
|
+
## 1. Purpose & Core Philosophy
|
|
4
|
+
|
|
5
|
+
The **Goal Contract** is the immutable anchor of the AI Engineering Loop. An autonomous coding agent must never begin implementation on ambiguous prompts, loose descriptions, or conversational requests without first formalizing an explicit contract.
|
|
6
|
+
|
|
7
|
+
The Goal Contract establishes:
|
|
8
|
+
- **What** problem is being solved.
|
|
9
|
+
- **Why** it matters to the business or user lifecycle.
|
|
10
|
+
- **How** success is measured deterministically.
|
|
11
|
+
- **Where** the boundaries are set (preventing scope creep).
|
|
12
|
+
- **When** the work is strictly considered complete.
|
|
13
|
+
|
|
14
|
+
---
|
|
15
|
+
|
|
16
|
+
## 2. Mandatory Contract Schema
|
|
17
|
+
|
|
18
|
+
Every Goal Contract MUST adhere to the following schema in Markdown or structured YAML:
|
|
19
|
+
|
|
20
|
+
```markdown
|
|
21
|
+
# Goal Contract: [Short Title / Feature / Bugfix ID]
|
|
22
|
+
|
|
23
|
+
## 1. Objective
|
|
24
|
+
[Concise 1-2 sentence description of the technical deliverable.]
|
|
25
|
+
|
|
26
|
+
## 2. Business Outcome & User Lifecycle Impact
|
|
27
|
+
[Explain what changes for the real-world actor (e.g. Employee, Admin, Customer, System). Describe the before/after lifecycle state transition.]
|
|
28
|
+
|
|
29
|
+
## 3. Acceptance Criteria (AC)
|
|
30
|
+
- [ ] AC-1: [Exact, testable statement with expected outcome]
|
|
31
|
+
- [ ] AC-2: [Exact, testable statement with expected outcome]
|
|
32
|
+
- [ ] AC-3: [Edge case or boundary behavior explicitly specified]
|
|
33
|
+
|
|
34
|
+
## 4. Technical Constraints
|
|
35
|
+
- [Architecture]: [Preserve existing patterns, layer boundaries, dependency conventions]
|
|
36
|
+
- [API / Schema]: [No breaking changes to existing contracts or database schemas]
|
|
37
|
+
- [Scope of Diff]: [Smallest coherent change; zero speculative abstractions; zero dead code]
|
|
38
|
+
- [Dependencies]: [Do not introduce external packages without explicit justification]
|
|
39
|
+
|
|
40
|
+
## 5. Out of Scope
|
|
41
|
+
- [Explicitly list what the agent MUST NOT touch or refactor during this task]
|
|
42
|
+
|
|
43
|
+
## 6. Verification Requirements
|
|
44
|
+
- **Unit Tests**: [Target files, boundary cases, and minimum expected coverage]
|
|
45
|
+
- **Static Analysis**: [Typecheck command, linter command, schema validation command]
|
|
46
|
+
- **Build / Packaging**: [Build command or bundling check]
|
|
47
|
+
- **Runtime / Integration**: [Manual smoke test steps or integration test command]
|
|
48
|
+
|
|
49
|
+
## 7. Definition of Done (DoD)
|
|
50
|
+
- [ ] All Acceptance Criteria (AC-1 through AC-N) verified with automated tests.
|
|
51
|
+
- [ ] 100% pass on all deterministic verification commands (0 errors, 0 warnings where enforced).
|
|
52
|
+
- [ ] Independent Devil's Advocate review completed with 0 unresolved blocking findings (SEV-1 / SEV-2).
|
|
53
|
+
- [ ] All review findings triaged with evidence (VALID, INVALID, UNCERTAIN).
|
|
54
|
+
- [ ] Judge Agent issues a formal PASS verdict.
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
---
|
|
58
|
+
|
|
59
|
+
## 3. Contract Lifecycle & Immutability Rules
|
|
60
|
+
|
|
61
|
+
1. **Pre-Implementation Freezing**:
|
|
62
|
+
- The Goal Contract is authored and frozen *before* any production code edits.
|
|
63
|
+
- If the task is ambiguous, the agent must refine the contract with the user before touching code.
|
|
64
|
+
2. **Immutability During Iteration**:
|
|
65
|
+
- Neither the Maker Agent nor the Devil's Advocate Agent may alter Acceptance Criteria during an iteration loop to make tests pass or bypass critique.
|
|
66
|
+
3. **Contract Amendments**:
|
|
67
|
+
- If during implementation a fundamental contradiction or impossible requirement is discovered, the agent must trigger **Human Escalation**. Only a human user may amend the Goal Contract.
|
|
68
|
+
|
|
69
|
+
---
|
|
70
|
+
|
|
71
|
+
## 4. Verification Mapping
|
|
72
|
+
|
|
73
|
+
Every single item listed under `Acceptance Criteria` must map to at least one concrete verification method:
|
|
74
|
+
|
|
75
|
+
| Acceptance Criterion | Primary Verification | Fallback Verification |
|
|
76
|
+
|---|---|---|
|
|
77
|
+
| Logic / Computation / Parsing | Automated Unit Test | Deterministic script execution |
|
|
78
|
+
| Type Safety / Schema Integrity | Compiler / Typechecker | Schema validator (`tsc`, `zod`, etc.) |
|
|
79
|
+
| Regression Protection | Existing Test Suite | End-to-end integration test |
|
|
80
|
+
| Visual / Interface State | Component / Snapshot / E2E Test | Exact DOM / State inspection |
|
|
81
|
+
|
|
82
|
+
---
|
|
83
|
+
|
|
84
|
+
## 5. Anti-Patterns to Avoid
|
|
85
|
+
|
|
86
|
+
- **The Vague Contract**: "Make authentication work better." (Invalid: lacks testable acceptance criteria).
|
|
87
|
+
- **The Missing Constraint**: Failing to declare out-of-scope files, leading to arbitrary refactoring of adjacent legacy modules.
|
|
88
|
+
- **The Self-Serving Goal**: Modifying acceptance criteria post-hoc when tests fail rather than fixing the underlying implementation.
|