ai-engineering-loop 1.0.2 → 1.0.4

package/core/judge-policy.md

@@ -1,123 +1,70 @@
-# Judge Policy Specification
+# Judge Agent & Evaluation Policy
 
-## 1. Overview & Role Definition
+## 1. Overview & Core Mission
 
-The **Judge Agent** is the impartial arbiter of the AI Engineering Loop. In traditional workflows, either the author approves its own code (optimistic bias) or a reviewer blocks changes arbitrarily (nitpicking/hallucination).
-
-The Judge Agent evaluates the complete matrix of:
-1. The original [Goal Contract](file:///Users/egagofur/Development/work/ai-engineering-loop/core/goal-contract.md).
-2. The surgical code implementation & diff.
-3. The raw [Deterministic Verification](file:///Users/egagofur/Development/work/ai-engineering-loop/core/verification-loop.md) outputs (tests, typecheck, lint, build).
-4. The [Devil's Advocate Review Findings](file:///Users/egagofur/Development/work/ai-engineering-loop/agents/devil-advocate.md) and Author's evidence/triage.
-5. The [Iteration & No-Progress State](file:///Users/egagofur/Development/work/ai-engineering-loop/core/iteration-policy.md).
+The **Judge Agent** is the impartial magistrate of the AI Engineering Loop. It does not write application code and does not guess at intent. It evaluates objective evidence to render one of three verdicts: **`PASS`**, **`ITERATE`**, or **`ESCALATE`**.
 
 ```mermaid
 flowchart TD
-    GC[Goal Contract] --> Judge[Judge Agent]
-    Diff[Implementation Diff] --> Judge
-    DV[Deterministic Logs] --> Judge
-    DA[Devil's Advocate Findings] --> Judge
+    Start([Evaluation Input]) --> Gate1{Verification Evidence Contract<br>Exit Code 0 & Complete Logs?}
+    
+    Gate1 -->|Failed / Vague| IterateVerif[Verdict: ITERATE<br>Reason: Incomplete Verification Evidence]
+    Gate1 -->|Passed| Gate2{Evaluate Finding Ledger<br>Validity + Severity}
+    
+    Gate2 -->|VALID + BLOCKER / HIGH| IterateFindings[Verdict: ITERATE<br>Reason: Blocking Substantive Issues]
+    Gate2 -->|INVALID Findings| Dismiss[DISMISS Invalid Findings<br>Record Signatures]
+    Gate2 -->|VALID + MEDIUM / LOW| Accept[ACCEPT & Document Tradeoffs in MR]
     
-    Judge --> Decision{Evaluate Rules}
+    Dismiss --> FinalCheck{0 Blocking Findings & 100% ACs Verified?}
+    Accept --> FinalCheck
     
-    Decision -->|All ACs Met, Tests Green, Findings Resolved| PASS[Verdict: PASS]
-    Decision -->|Valid Blocking Findings & Iterations Remaining| ITERATE[Verdict: ITERATE]
-    Decision -->|Stalled / Max Iterations / High Risk| ESCALATE[Verdict: ESCALATE]
+    FinalCheck -->|Yes: PASS| PassVerdict([Verdict: PASS<br>Proceed to Impact Assessment & Delivery])
+    FinalCheck -->|No| IterateFindings
+    
+    IterateFindings --> IterCheck{Active Iterations >= MAX_ITERATIONS?}
+    IterCheck -->|Yes: Escalate| EscalateVerdict([Verdict: ESCALATE<br>Human Escalation Report])
+    IterCheck -->|No| MakerFix([Maker Applies Fix & Adds Tests])
 ```
 
 ---
 
-## 2. Evidence Evaluation Rules
-
-The Judge does NOT blindly accept claims from either the Maker or the Devil's Advocate. Every statement is scrutinized according to the following evidentiary rules:
+## 2. Evidence-Based Decision Matrix
 
-### Rule 1: Deterministic Precedence
-If any deterministic check (unit tests, TypeScript compilation, linter, build) failed or was not executed, the Judge **CANNOT** issue a `PASS` verdict.
+The Judge renders decisions based strictly on **Validity + Severity**. The reviewer's subjective disposition (`STRONG`, `ACCEPTABLE`, `WEAK`) **never overrides factual evidence**:
 
-### Rule 2: Rejection of Unsubstantiated Nitpicks
-A Devil's Advocate finding that:
-- Relies on personal aesthetic preference (e.g. *"variable names could be shorter"*),
-- Suggests speculative future-proofing abstractions not requested in the Goal Contract, or
-- Misunderstands existing repository conventions,
-must be declared **`INVALID`** by the Judge and discarded from blocking the build.
-
-### Rule 3: Strict Severity Thresholds
-- **SEV-1 (Critical)** and **SEV-2 (High)** findings: Must be resolved in code with passing tests before a `PASS` verdict can be granted.
-- **SEV-3 (Medium)** and **SEV-4 (Low)** findings: May be accepted into an action items backlog if they do not violate any Acceptance Criteria in the Goal Contract.
-
-### Rule 4: Verification of Author Triage
-If the Maker Agent claims a finding is `INVALID` (e.g. *"The suggested API does not exist"*), the Judge verifies this claim against the repository before accepting the dismissal.
+| Finding Validity | Finding Severity | Disposition | Judge Action | Impact on Final Verdict |
+|---|---|---|---|---|
+| **`VALID`** | **`BLOCKER`** | Any | **UPHELD (Blocking)** | **`ITERATE`** — Maker must apply alternative diff & tests. |
+| **`VALID`** | **`HIGH`** | Any | **UPHELD (Blocking)** | **`ITERATE`** — Maker must apply alternative diff & tests. |
+| **`VALID`** | **`MEDIUM`** | `ACCEPTABLE` | **UPHELD (Tradeoff)** | **`PASS`** (if ACs met) — Logged as known tradeoff in MR. |
+| **`VALID`** | **`LOW`** | `ACCEPTABLE` | **UPHELD (Tradeoff)** | **`PASS`** (if ACs met) — Logged as known tradeoff in MR. |
+| **`INVALID`** | Any | `WEAK` | **DISMISSED (Hallucination)** | **`PASS`** (if ACs met) — Discarded with evidence proof. |
 
 ---
 
-## 3. Verdict Computation Logic
-
-```text
-FUNCTION ComputeVerdict(Contract, Diff, DeterministicLogs, Findings, IterationState):
-    IF DeterministicLogs.HasFailures() THEN
-        RETURN ITERATE(reason="Deterministic verification gates failed")
-    END IF
+## 3. The 3 Verdict Rules
 
-    IF IterationState.IsStalled() OR IterationState.CurrentIteration >= IterationState.MaxIterations THEN
-        IF Findings.HasUnresolvedBlocking() THEN
-            RETURN ESCALATE(reason="Max iterations reached with unresolved blocking findings")
-        END IF
-    END IF
-
-    IF Contract.ViolatesConstraints(Diff) THEN
-        RETURN ESCALATE(reason="Implementation violated technical constraints or modified out-of-scope files")
-    END IF
-
-    UnresolvedBlocking = Findings.GetUnresolved(severity IN [SEV_1, SEV_2])
-    
-    IF UnresolvedBlocking.Count == 0 THEN
-        IF Contract.AllAcceptanceCriteriaVerified(DeterministicLogs) THEN
-            RETURN PASS(summary="All criteria satisfied and verified with proof")
-        ELSE
-            RETURN ITERATE(reason="Missing deterministic proof for specific Acceptance Criteria")
-        END IF
-    ELSE
-        RETURN ITERATE(reason="Unresolved blocking findings exist", findings=UnresolvedBlocking)
-    END IF
-```
+### Verdict 1: `PASS`
+- **Conditions**:
+  1. 100% of Goal Contract Acceptance Criteria (AC-1..N) proven via deterministic verification.
+  2. Verification Evidence Contract 100% satisfied (exit code 0, complete logs, assertion proof).
+  3. Zero open `VALID + BLOCKER` or `VALID + HIGH` findings.
+  4. Any `INVALID` findings are formally dismissed with counter-evidence.
+  5. Any `VALID + MEDIUM/LOW` findings are documented as tradeoffs.
 
 ---
 
-## 4. Standardized Judge Verdict Report
-
-Every evaluation concludes with a formal **Judge Verdict Artifact**:
-
-```markdown
-# ⚖️ Judge Evaluation Report
-
-## 1. Executive Verdict
-- **Verdict**: `[PASS | ITERATE | ESCALATE]`
-- **Iteration**: `[K of N]`
-- **Confidence**: `[HIGH | MEDIUM | LOW]`
+### Verdict 2: `ITERATE`
+- **Conditions**:
+  1. One or more `VALID + BLOCKER` or `VALID + HIGH` findings exist.
+  2. Active iteration count $< \text{MAX\_ITERATIONS}$ (default 3).
+- **Maker Directive**: The Maker must adopt the concrete alternative diff or provide an equivalent verified architectural resolution, authoring regression unit tests.
 
-## 2. Deterministic Verification Audit
-| Gate | Command | Result | Status |
-|---|---|---|:---:|
-| Unit Tests | `npx jest --testPathIgnorePatterns="integration"` | 14 passed, 0 failed | ✅ PASS |
-| Static Types | `npx tsc --noEmit` | Exit code 0 | ✅ PASS |
-| Linter | `npx eslint --fix <files>` | 0 errors, 0 warnings | ✅ PASS |
-| Build | `npm run build` | Exit code 0 | ✅ PASS |
-
-## 3. Goal Contract Compliance Audit
-| Acceptance Criterion | Verification Method | Status |
-|---|---|:---:|
-| AC-1: Correct attendance status calculation | `resolve-display-status.test.ts` | ✅ PASS |
-| AC-2: Weekend & non-normal hours edge cases | `resolve-display-status.test.ts:L45` | ✅ PASS |
-| AC-3: Backward compatibility preserved | Integration test suite | ✅ PASS |
-
-## 4. Finding Triage & Resolution Matrix
-| ID | Severity | Category | Reviewer Finding | Triage Status | Resolution Proof |
-|---|---|---|---|:---:|---|
-| COR-001 | HIGH | Correctness | Missing null check on overtimeNote | `VALID` | Fixed in `service.ts:L32`; tested in `service.test.ts` |
-| SEC-001 | LOW | Security | Suggest sanitizing display status | `INVALID` | Display status is an enum internally generated, not user input |
+---
 
-## 5. Directives for Next Step
-[If PASS: Hand off to Delivery Adapter.]
-[If ITERATE: Explicit, numbered instructions for the Maker Agent.]
-[If ESCALATE: Actionable human decision points.]
-```
+### Verdict 3: `ESCALATE`
+- **Conditions**:
+  1. Active iteration count $\ge \text{MAX\_ITERATIONS}$ (3).
+  2. Stagnation detected (identical finding signatures repeated across 2 iterations).
+  3. Contradictory architectural invariants that cannot be resolved within task scope.
+- **Action**: Halt the loop immediately and generate an actionable Human Escalation Report.

package/core/orchestration-model.md

@@ -0,0 +1,87 @@
+# Runtime Capability Registry & Execution-Mode Selection Specification
+
+## 1. Overview & Foundational Laws
+
+The AI Engineering Loop enforces strict honesty and evidence-based rigor regarding agent execution:
+
+> **Core Law**:
+> AI Engineering Loop must **NEVER** claim independent agent execution without runtime evidence of an actual separate LLM execution.
+
+### The 3-Stage Capability Lifecycle:
+The architecture explicitly distinguishes three separate stages:
+1. **`CONFIGURATION_SUPPORTED`**: The platform understands subagent configuration (e.g. `.agents/plugins/.../AGENT.md` with `subagent: true`, `mainAgent: false` is discoverable).
+2. **`INVOCATION_AVAILABLE`**: An invocation tool (e.g. `invoke_subagent`) or authenticated agent CLI is actively exposed and callable in the current runtime.
+3. **`EXECUTION_PROVEN`**: An actual separate child session was created, captured a separate execution identity, produced a real child model response, and operated with independent context.
+
+```text
+┌───────────────────────────┐     ┌───────────────────────────┐     ┌───────────────────────────┐
+│  CONFIGURATION_SUPPORTED  │ ──> │   INVOCATION_AVAILABLE    │ ──> │     EXECUTION_PROVEN      │
+│  (Config is recognized)   │     │ (Callable tool is active) │     │ (Child LLM response seen) │
+└───────────────────────────┘     └───────────────────────────┘     └───────────────────────────┘
+```
+
+> [!IMPORTANT]
+> **Semantic Invariant**:
+> If `CONFIGURATION_SUPPORTED = true`, `INVOCATION_AVAILABLE = false`, and `EXECUTION_PROVEN = false`, the engine **must select `CONTEXT_ISOLATION_ONLY`**.
+> Configuration discovery is **never** conflated with execution capability.
+
+---
+
+## 2. Non-Negotiable Invariants: What is NOT Execution Proof
+
+The system categorically rejects the following as proof of independent LLM execution:
+1. **Configuration files**: `AGENT.md`, `subagent: true`, `mainAgent: false`.
+2. **Documentation & plugin manifests**: Markdown descriptions or plugin registration.
+3. **IPC existence**: The existence of `agentapi` on disk or `ANTIGRAVITY_AGENTAPI_EXE`.
+4. **IPC message dispatch**: Successful `agentapi send-message` is classified strictly as `IPC_MESSAGE_DISPATCH`, **NOT** an agent execution capability.
+5. **Metadata fields**: Presence of `subagentSpec: null` or conversation metadata in `get-conversation-metadata`.
+6. **Browser automation tools**: `browser_subagent` is Playwright browser DOM/navigation automation and must **NEVER** be classified as an LLM subagent.
+7. **Persona simulation**: Role-playing as a reviewer in the same session is self-review, not an agent.
+
+---
+
+## 3. Standard 5 Execution Modes (Deterministic Priority)
+
+| Priority | Mode Name | Requires Independent LLM Execution? | Condition for Selection |
+|:---:|---|:---:|---|
+| **1** | **`TRUE_INDEPENDENT_AGENT`** | **YES** | Separate child conversation/process exists **AND** actual LLM model response is produced **AND** conversational history is not inherited. |
+| **2** | **`ISOLATED_AGENT_INSTANCE`** | **YES** | Separate conversation/agent instance exists with verified independent model execution. |
+| **3** | **`FRESH_PROCESS_AGENT`** | **YES** | A separate OS process successfully executes an LLM agent with fresh context and returns a verified model response. |
+| **4** | **`CONTEXT_ISOLATION_ONLY`** | **NO** | Clean-Slate Artifact Isolation Barrier in same session. Strips 100% of prompt history on disk. Guaranteed fallback. |
+| **5** | **`UNAVAILABLE`** | **NO** | No review execution mechanism is available. |
+
+---
+
+## 4. Antigravity Environment Empirical Discovery Record
+
+| Investigated Surface | Tested Command / API | Classification | Status & Result |
+|---|---|---|---|
+| **Custom Agent Config** | `.agents/plugins/.../AGENT.md` | `CONFIGURATION_SUPPORTED` | **Supported**: Antigravity recognizes plugin/agent schemas. |
+| **In-Chat Subagent Tool** | Toolset declarations | `INVOCATION_UNAVAILABLE` | `browser_subagent` present (DOM only); no general code subagent tool. |
+| **Python SDK** | `import google.antigravity` | `UNAVAILABLE` | `ModuleNotFoundError: No module named 'google.antigravity'`. |
+| **External Agent CLI** | `/Users/.../.local/bin/claude -p` | `UNAVAILABLE` | Unauthenticated: `Not logged in · Please run /login`. |
+| **Antigravity `new-conversation`** | `agentapi new-conversation` | `UNAVAILABLE` | Blocked by Language Server `project_id` authorization. |
+| **Antigravity `send-message`** | `agentapi send-message` | `IPC_MESSAGE_DISPATCH` | Succeeded (IPC message dispatch works, but is not an agent). |
+| **Artifact Barrier** | `buildReviewContextBarrier()` | `CONTEXT_ISOLATION_ONLY` | **Available & Verified**: 0% prompt bleed on disk. |
+
+### Architectural Conclusion:
+> *"Antigravity custom subagent configuration is supported/discoverable, but native subagent invocation is not exposed or executable from the current standalone agent runtime."*
+
+---
+
+## 5. Truthful Reporting Output
+
+When `CONTEXT_ISOLATION_ONLY` is selected, the report generator strictly produces:
+
+```text
+Execution Mode: CONTEXT_ISOLATION_ONLY
+Independent LLM Execution: NOT PROVEN
+Native Subagent Invocation: UNAVAILABLE
+Review Method: Clean-Slate Artifact Isolation Barrier
+```
+
+### Strictly Forbidden Phrases during `CONTEXT_ISOLATION_ONLY`:
+- *"independent agent review"*
+- *"subagent"*
+- *"multi-agent review"*
+- *"independent reviewer"*

package/core/verification-loop.md

@@ -1,97 +1,61 @@
-# Verification Loop Specification
+# Deterministic Verification Loop & Evidence Contract
 
-## 1. Overview
+## 1. Overview & Core Laws
 
-The **Verification Loop** is the dual-layer validation engine of the AI Engineering Loop. It guarantees that code is not merely claimed to be functional by its author, but is rigorously tested via **machine-checkable deterministic gates** and scrutinized by **independent adversarial inspection**.
+The Verification Loop is the deterministic machine gate of the AI Engineering Loop:
 
-```mermaid
-flowchart TD
-    Start([Implementation Diff]) --> D1[Deterministic Gate 1: Unit & Integration Tests]
-    D1 -->|Fail| MakerFix[Return to Maker: Fix Logic]
-    D1 -->|Pass| D2[Deterministic Gate 2: Typecheck & Compiler]
-    
-    D2 -->|Fail| MakerFix
-    D2 -->|Pass| D3[Deterministic Gate 3: Linter & Format]
-    
-    D3 -->|Fail| MakerFix
-    D3 -->|Pass| D4[Deterministic Gate 4: Build / Packaging]
-    
-    D4 -->|Fail| MakerFix
-    D4 -->|Pass| Adversarial[Adversarial Gate: Devil's Advocate Layered Review]
-    
-    Adversarial --> JudgeEval[Judge Agent: Evidence Evaluation]
-    JudgeEval -->|ITERATE| MakerFix
-    JudgeEval -->|ESCALATE| Escalate([Human Escalation])
-    JudgeEval -->|PASS| Done([Definition of Done Satisfied])
-```
-
----
-
-## 2. Dynamic Command Discovery
-
-The exact commands executed for each deterministic gate are resolved dynamically according to [Configuration Precedence](file:///Users/egagofur/Development/work/ai-engineering-loop/core/configuration-precedence.md):
-
-1. Read `.ai-engineering-loop/verification.md` from the target repository if present.
-2. If missing, fall back to the defaults of the bound [Project Profile](file:///Users/egagofur/Development/work/ai-engineering-loop/profiles/README.md) (`web-app`, `backend-api`, `mobile-app`, `library`, `monorepo`).
-3. If profile is unspecified, infer commands from repository manifests (`package.json`, `go.mod`, `Cargo.toml`, `pyproject.toml`).
+> **"Code cannot enter Devil's Advocate review until it achieves 100% green machine verification backed by explicit, verifiable execution evidence."**
 
 ---
 
-## 3. Layer 1: Deterministic Verification
-
-Deterministic verification consists of machine-executed commands producing binary (`PASS` / `FAIL`) or structured outputs. 
-
-An agent is **strictly prohibited** from proceeding to adversarial review if any deterministic gate fails.
-
-### The 4 Deterministic Gates
-
-#### Gate 1: Unit & Regression Tests
-- **Objective**: Prove correctness of new logic and ensure zero regressions.
-- **Criteria**: 100% exit code `0`, zero test failures, zero unexpected skipped tests.
-- **Rules**:
-  - Tests must cover happy paths, null/undefined safety, empty inputs, boundary conditions, and error branches.
-  - Test assertions must be strict (e.g. checking specific return values, error types, and state mutations, not merely `toBeDefined()`).
-
-#### Gate 2: Static Typing & Compilation
-- **Objective**: Prove mathematical type safety and schema conformance.
-- **Criteria**: Zero compiler / typechecker errors across the codebase or affected workspaces (e.g. `tsc --noEmit`, `mypy`, `cargo check`).
-
-#### Gate 3: Code Standards & Linting
-- **Objective**: Guarantee zero static analysis rule violations and formatting hygiene.
-- **Criteria**: Zero linter errors on touched files (e.g. `eslint`, `ruff`, `golangci-lint`, `dart analyze`).
-
-#### Gate 4: Build & Packaging
-- **Objective**: Ensure the project compiles, bundles, or packages without missing assets or circular dependencies.
-- **Criteria**: Build command completes with exit code `0`.
+## 2. Verification Evidence Contract
+
+A verification `PASS` is **strictly invalid** without concrete execution evidence. The system categorically rejects vague statements such as *"command was launched"* or *"test appears to have passed"*.
+
+### Required Evidence Properties:
+1. **`command`**: Exact shell string executed (e.g. `npm test -- --runInBand`).
+2. **`executionIdentity`**: Process ID (PID), execution hash, or system execution identifier.
+3. **`startTime` & `endTime`**: ISO timestamps documenting execution duration.
+4. **`exitCode`**: Must be `0`. Any non-zero exit code immediately halts the gate.
+5. **`stdout` & `stderr`**: Raw machine logs captured from execution.
+6. **`timeoutStatus`**: Must be `"COMPLETED"` (not timed out or backgrounded without completion).
+7. **`testCounts`**: Explicit counts of passed, failed, and skipped tests.
+8. **`assertionEvidence`**: Specific assertion proof matching the active Goal Contract's Acceptance Criteria.
+
+```json
+{
+  "command": "npm test",
+  "executionIdentity": "exec-d4f1a2",
+  "startTime": "2026-08-25T10:00:00.000Z",
+  "endTime": "2026-08-25T10:00:04.500Z",
+  "exitCode": 0,
+  "stdout": "PASS src/services/payment.test.ts (14 tests passed, 0 failed)",
+  "stderr": "",
+  "timeoutStatus": "COMPLETED",
+  "testCounts": {
+    "passed": 14,
+    "failed": 0,
+    "skipped": 0
+  },
+  "assertionEvidence": "AC-1: Lock payment row before update verified in test_concurrent_payment_locking"
+}
+```
 
 ---
 
-## 4. Layer 2: Adversarial Verification (Devil's Advocate)
-
-Once all deterministic gates pass, the code diff is submitted to the **Devil's Advocate Agent**.
+## 3. The 4 Verification Gates
 
-### Purpose
-Deterministic tests only test what the author *thought* to test. The Devil's Advocate exists to discover what the author *forgot*, *assumed*, or *misunderstood*.
-
-### Scope of Review
-The Devil's Advocate reviews strictly the git diff against the target base branch:
-```bash
-git diff <base-branch>...HEAD
-```
-
-### Layered Review Rules
-Review domains are dynamically assembled based on the active **Project Profile** and **Repository Invariants** (see [Devil's Advocate Specification](file:///Users/egagofur/Development/work/ai-engineering-loop/agents/devil-advocate.md)), ensuring only substantive, relevant topics are evaluated.
+1. **Gate 1: Unit & Integration Tests**: All unit and edge-case tests must pass with 0 failures.
+2. **Gate 2: Static Type Analysis**: `tsc --noEmit` or compiler type checking must exit with code `0` (zero type errors).
+3. **Gate 3: Linter Analysis**: Linter rules must pass cleanly with 0 errors on modified files.
+4. **Gate 4: Production Build**: The build command (`npm run build`, `go build`, `cargo build`) must produce a clean compile.
 
 ---
 
-## 5. Verification Evidence Protocol
-
-All claims of verification must be accompanied by **reproducible evidence**:
-
-1. **Command Executed**: Full CLI command line string.
-2. **Exit Code**: Exact status code returned.
-3. **Execution Output**: Raw relevant output snippet showing test pass counts, typecheck logs, or lint results.
-4. **Acceptance Criteria Mapping**: Explicit mapping showing which test corresponds to which AC from the [Goal Contract](file:///Users/egagofur/Development/work/ai-engineering-loop/core/goal-contract.md).
+## 4. Rejection Triggers
 
-> [!CAUTION]
-> Statements such as *"I have manually verified that this works"* or *"The tests should pass"* without command outputs are invalid and will be rejected by the Judge Agent.
+The Verification Gate immediately halts and returns to the Maker if:
+- Any test suite fails or times out.
+- Exit code is non-zero.
+- The command was sent to the background and not verified to completion.
+- Test logs contain zero passing assertions for new acceptance criteria.