ai-engineering-loop 1.0.21 → 1.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (89) hide show
  1. package/.agents/workflows/ai-engineering-loop.md +7 -3
  2. package/.claude/skills/ai-engineering-loop/SKILL.md +13 -7
  3. package/.gemini/skills/ai-engineering-loop/SKILL.md +13 -7
  4. package/.grok/skills/ai-engineering-loop/SKILL.md +13 -7
  5. package/README.md +26 -3
  6. package/SECURITY.md +23 -0
  7. package/SUPPORT.md +28 -0
  8. package/bin/ai-engineering-loop.js +255 -3
  9. package/core/artifact-gates.md +49 -0
  10. package/evals/README.md +35 -0
  11. package/evals/cases/01-stale-verification.json +17 -0
  12. package/evals/cases/02-vague-verification.json +16 -0
  13. package/evals/cases/03-missing-ac-evidence.json +17 -0
  14. package/evals/cases/04-hallucinated-citation.json +17 -0
  15. package/evals/cases/05-prompt-injection.json +17 -0
  16. package/evals/cases/06-source-secret.json +16 -0
  17. package/evals/cases/07-log-secret.json +16 -0
  18. package/evals/cases/08-symlink-escape.json +16 -0
  19. package/evals/cases/09-authorization-bypass.json +16 -0
  20. package/evals/cases/10-payment-idempotency.json +16 -0
  21. package/evals/cases/11-concurrent-update.json +16 -0
  22. package/evals/cases/12-swallowed-error.json +16 -0
  23. package/evals/cases/13-happy-path-only.json +17 -0
  24. package/evals/cases/14-invalid-finding-axis.json +16 -0
  25. package/evals/cases/15-blocker-without-alternative.json +16 -0
  26. package/evals/cases/16-unsupported-pass.json +17 -0
  27. package/evals/cases/17-soft-standard.json +16 -0
  28. package/evals/cases/18-iteration-ceiling.json +17 -0
  29. package/evals/cases/19-safe-documentation-change.json +17 -0
  30. package/evals/cases/20-valid-boundary-feature.json +18 -0
  31. package/lib/doctor.js +98 -0
  32. package/lib/escalation.js +192 -0
  33. package/lib/evaluation.js +137 -0
  34. package/lib/gates.js +315 -0
  35. package/lib/orchestration.js +5 -1
  36. package/lib/run-state.js +221 -0
  37. package/lib/safe-context.js +191 -0
  38. package/lib/sync-hosts.js +6 -5
  39. package/package.json +12 -9
  40. package/policies/model-escalation.md +41 -0
  41. package/schemas/delivery.schema.json +14 -0
  42. package/schemas/finding-ledger.schema.json +27 -0
  43. package/schemas/goal-contract.schema.json +32 -0
  44. package/schemas/verdict.schema.json +17 -0
  45. package/schemas/verification.schema.json +32 -0
  46. package/docs/antigravity-feasibility.md +0 -92
  47. package/docs/claude-code-feasibility.md +0 -80
  48. package/docs/grok-cli-feasibility.md +0 -99
  49. package/docs/migration-plan.md +0 -70
  50. package/examples/backend-api/payment-idempotency/README.md +0 -33
  51. package/examples/backend-api/payment-idempotency/goal-contract.md +0 -33
  52. package/examples/backend-api/payment-idempotency/judge-verdict.md +0 -28
  53. package/examples/backend-api/payment-idempotency/review-findings.md +0 -50
  54. package/examples/dot/status-display/README.md +0 -23
  55. package/examples/dot/status-display/delivery-report.md +0 -41
  56. package/examples/dot/status-display/goal-contract.md +0 -46
  57. package/examples/dot/status-display/judge-verdict.md +0 -44
  58. package/examples/dot/status-display/review-findings.md +0 -63
  59. package/examples/initialization/README.md +0 -19
  60. package/examples/initialization/discovery-trace.md +0 -63
  61. package/examples/initialization/generated-context.md +0 -110
  62. package/examples/mobile-app/offline-sync-queue/README.md +0 -33
  63. package/examples/mobile-app/offline-sync-queue/goal-contract.md +0 -32
  64. package/examples/mobile-app/offline-sync-queue/judge-verdict.md +0 -27
  65. package/examples/mobile-app/offline-sync-queue/review-findings.md +0 -30
  66. package/scripts/init.sh +0 -106
  67. package/templates/repo-config/adapter.md +0 -11
  68. package/templates/repo-config/adr-readme.md +0 -33
  69. package/templates/repo-config/architecture.md +0 -15
  70. package/templates/repo-config/claude-permissions.json +0 -10
  71. package/templates/repo-config/config.md +0 -11
  72. package/templates/repo-config/conventions.md +0 -16
  73. package/templates/repo-config/glossary.md +0 -18
  74. package/templates/repo-config/lessons.md +0 -8
  75. package/templates/repo-config/verification.md +0 -13
  76. package/templates/repo-config/workflow.md +0 -25
  77. package/tests/capability-selection.test.js +0 -207
  78. package/tests/claimed-vs-reality.test.js +0 -27
  79. package/tests/dot-branch-policy.test.js +0 -65
  80. package/tests/generate-adapter.test.js +0 -80
  81. package/tests/generate-workflow.test.js +0 -180
  82. package/tests/grok-runtime.test.js +0 -201
  83. package/tests/intern-catalog.test.js +0 -72
  84. package/tests/living-context.test.js +0 -61
  85. package/tests/no-company-leak.test.js +0 -82
  86. package/tests/orchestration.test.js +0 -368
  87. package/tests/qa-delivery-template.test.js +0 -90
  88. package/tests/skill-host-compat.test.js +0 -406
  89. package/tests/sync-hosts.test.js +0 -168
@@ -1,92 +0,0 @@
1
- # Antigravity Sub-Agent Feasibility & Execution Strategy
2
-
3
- ## 1. Executive Summary
4
-
5
- This document evaluates the practical feasibility of executing the **AI Engineering Loop** (Maker, Devil's Advocate, Judge) inside the **Google Antigravity IDE / Agentic Platform**, addressing runtime capabilities, context segregation, token economics, and persistence.
6
-
7
- ---
8
-
9
- ## 2. Analysis of the 8 Core Questions
10
-
11
- ```mermaid
12
- flowchart TD
13
- Main[Antigravity Primary Agent / Planner] -->|Spawn / Invoke| SubDA[Sub-Agent / Persona: Devil's Advocate]
14
- Main -->|Spawn / Invoke| SubJD[Sub-Agent / Persona: Judge]
15
-
16
- SubDA -->|Reads Same Workspace & Git Diff| Repo[(Workspace Files & Git)]
17
- SubDA -->|Returns Finding Ledger| Main
18
-
19
- Main -->|Evaluates with Judge Policy| SubJD
20
- SubJD -->|Returns PASS / ITERATE / ESCALATE| Main
21
-
22
- Main -->|Persists State Artifacts| BrainDir[(brain/<conversation_id>/)]
23
- ```
24
-
25
- ### 1. Can Antigravity summon independent sub-agents?
26
- - **Status**: **YES**.
27
- - **Mechanism**: Antigravity natively supports agentic delegation through sub-agents (e.g. `invoke_subagent` / `browser_subagent` / background tasks). Sub-agents execute with their own independent message loops, instructions, and tool sets.
28
- - **Alternative / Fallback**: If subagent spawning is constrained in a single-agent CLI mode, Antigravity agents can execute **Context-Switched Multi-Turn Personas** by clearing operational context, adopting strict role instructions, and evaluating the isolated `git diff`.
29
-
30
- ### 2. Can the main agent pass context to a sub-agent?
31
- - **Status**: **YES**.
32
- - **Mechanism**: The spawning task payload allows passing:
33
- - The [Goal Contract](file:///Users/egagofur/Development/work/ai-engineering-loop/core/goal-contract.md).
34
- - Target base branch name (`main`, `develop`).
35
- - Active iteration index and finding ledger history.
36
-
37
- ### 3. Can the sub-agent inspect the same repository?
38
- - **Status**: **YES**.
39
- - **Mechanism**: Sub-agents share the active workspace root (`/Users/.../ai-engineering-loop`). They have access to `view_file`, `grep_search`, `list_dir`, and `run_command` (`git diff`, `git log`, `npx jest`, etc.).
40
-
41
- ### 4. Can the sub-agent return structured findings?
42
- - **Status**: **YES**.
43
- - **Mechanism**: The sub-agent is instructed to output strictly formatted YAML/Markdown compliant with the [Finding Policy](file:///Users/egagofur/Development/work/ai-engineering-loop/policies/finding-policy.md). Upon task completion, the structured output is delivered directly back into the primary agent's context.
44
-
45
- ### 5. Can the main agent consume those findings?
46
- - **Status**: **YES**.
47
- - **Mechanism**: The primary agent parses the returned Finding Ledger, updates the iteration state artifact, and feeds the findings into the [Judge Agent](file:///Users/egagofur/Development/work/ai-engineering-loop/agents/judge.md).
48
-
49
- ### 6. Can the loop be repeated automatically?
50
- - **Status**: **YES**.
51
- - **Mechanism**: Antigravity's autonomous loop executes sequentially until the Judge outputs `PASS` or triggers `ESCALATE`. The loop does not require human intervention between iterations unless an escalation trigger is activated.
52
-
53
- ### 7. Can iteration state be persisted?
54
- - **Status**: **YES**.
55
- - **Mechanism**: Antigravity provides a dedicated per-conversation artifact directory:
56
- `<appDataDir>/brain/<conversation-id>/`
57
- The loop persists:
58
- - `goal_contract.md`
59
- - `iteration_ledger.json` (finding signatures, iteration index, diff hashes)
60
- - `verification_logs.txt`
61
- - `judge_verdict.md`
62
-
63
- ### 8. What are the practical token/context limitations?
64
- - **Analysis**:
65
- - Full codebase context in every prompt leads to context bloat and degraded attention.
66
- - **Optimization Strategy**:
67
- - Write `git diff` to `.ai-engineering-loop/tasks/current.diff` and pass that path.
68
- - Devil's Advocate: at most 8 tool calls; skip css/generated; no git log; wait (no background).
69
- - Judge: at most 4 tool calls; ledger + Goal Contract only; fact-check cited locations; wait.
70
- - Do not use `browser_subagent` as a reviewer.
71
- - If `invoke_subagent` is missing, use CONTEXT_ISOLATION_ONLY with the same budgets. See `policies/review-budget.md` and `.agents/`.
72
-
73
- ---
74
-
75
- ## 3. Recommended Execution Modes in Antigravity
76
-
77
- | Dimension | Mode A: Native Sub-Agents (Recommended) | Mode B: Persona Context Switching |
78
- |---|---|---|
79
- | **Context Isolation** | 100% Isolated context window | Sequential turns within single context |
80
- | **Cognitive Bias** | Zero author bias (clean slate) | Low bias if context is scoped to diff |
81
- | **Token Overhead** | Minimal (sub-agent starts fresh) | Moderate (context grows across turns) |
82
- | **Implementation Complexity** | Requires subagent invocation tool | Works natively in standard chat turn |
83
- | **Antigravity Fit** | Ideal for complex multi-file features | Ideal for quick, surgical fixes |
84
-
85
- ---
86
-
87
- ## 4. Implementation Roadmap for Antigravity Skill
88
-
89
- To deploy this workflow as an Antigravity Custom Skill (e.g. `skills/ai-engineering-loop/SKILL.md`):
90
- 1. Package the core loop into an executable skill definition with YAML frontmatter.
91
- 2. Provide slash command shortcuts (e.g. `/loop:start`, `/loop:review`).
92
- 3. Bind the DOT Adapter as an opt-in release plugin when working inside DOT repositories.
@@ -1,80 +0,0 @@
1
- # Claude Code Feasibility and Kiro Request Safety
2
-
3
- ## 1. Problem
4
-
5
- Claude Code users (including Kiro-backed models such as `kiro/claude-sonnet-5`) hit:
6
-
7
- ```
8
- API Error: 400 [kiro/claude-sonnet-5] [400]:
9
- {"message":"Improperly formed request.","reason":"REQUEST_BODY_INVALID"}
10
- ```
11
-
12
- Kiro's `generateAssistantResponse` rejects request bodies that Claude Code will happily send. Two skill-side causes were confirmed:
13
-
14
- 1. **Extra tool keys.** The shared skill told the model to call `spawn_subagent` with `capability_mode`, `isolation`, and `resume_from`. Claude Code's Task/Agent schema does not have those fields. Proxies that validate additionalProperties fail with `REQUEST_BODY_INVALID`.
15
- 2. **Hostile markup in the injected skill.** Mermaid (`<br>`, nested `{braces}`), LaTeX (`$\rightarrow$`), and folded YAML descriptions inflate and corrupt the system/skill payload Kiro receives.
16
-
17
- This is separate from 9router bugs that reject any Anthropic `system` field. Those need a router fix. The skill must still not add extra invalid keys or markup.
18
-
19
- ## 2. Claude Code mapping
20
-
21
- | Loop role | Claude Code type | Task keys allowed |
22
- |---|---|---|
23
- | Orchestrator / Maker | parent session | n/a |
24
- | Devil's Advocate | `devil-advocate` (fallback `general-purpose`) | `subagent_type`, `description`, `prompt`, `run_in_background: false` |
25
- | Judge | `judge` (fallback `general-purpose`) | same; wait for DA first |
26
-
27
- Parent must write `git diff` to a file and pass that path. DA budget: 8 tool calls, skip css/generated, no git log. Do not leave DA running in the background.
28
-
29
- Do not pass Grok keys (`spawn_subagent`, `capability_mode`, `resume_from`, `isolation`, `background`) on Claude Code.
30
-
31
- Repo files:
32
-
33
- - `.claude/skills/ai-engineering-loop/SKILL.md`
34
- - `.claude/agents/devil-advocate.md`
35
- - `.claude/agents/judge.md`
36
- - `.claude/commands/ai-engineering-loop.md`
37
-
38
- ## 3. Mode selection
39
-
40
- | Condition | Mode |
41
- |---|---|
42
- | Task/Agent tool present, child result returned | `TRUE_INDEPENDENT_AGENT` |
43
- | No subagent tool | `CONTEXT_ISOLATION_ONLY` |
44
-
45
- ## 4. Second 400 path: auto-mode Bash classifier
46
-
47
- A later failure looks like this:
48
-
49
- ```
50
- Error: kr/claude-sonnet-5 is temporarily unavailable, so auto mode
51
- cannot determine the safety of Bash right now.
52
- API Error: 400 [kiro/claude-sonnet-5] REQUEST_BODY_INVALID
53
- ```
54
-
55
- Read/Grep still work. The first Bash that needs the auto-mode safety classifier (`npm run typecheck`, `npm test`) is sent to Kiro. When that classifier request is invalid or the model is down, Claude Code retries and Kiro returns 400.
56
-
57
- Mitigations shipped in the skill:
58
-
59
- 1. `allowed-tools` pre-approves `Bash(npm run *)`, `Bash(npm test *)`, `Bash(npx *)`, `Bash(git *)` so those commands skip the classifier when the skill or slash command is active.
60
- 2. The skill tells the model not to retry Bash/Write after a classifier or 400 failure.
61
- 3. Project allow rules (copy into `.claude/settings.local.json`):
62
-
63
- ```json
64
- {
65
- "permissions": {
66
- "allow": [
67
- "Bash(npm run *)",
68
- "Bash(npm test *)",
69
- "Bash(npx *)",
70
- "Bash(git *)"
71
- ]
72
- }
73
- }
74
- ```
75
-
76
- If 400 still happens on a fresh session with no Bash yet, it is the 9router `system` field bug, not this package.
77
-
78
- ## 5. What stays in Grok-only files
79
-
80
- Grok spawn details live in `.grok/skills/ai-engineering-loop/SKILL.md`. Claude Code must not load that file as its skill. Claude Code discovers `.claude/` first.
@@ -1,99 +0,0 @@
1
- # Grok CLI Feasibility & Execution Strategy
2
-
3
- ## 1. Executive Summary
4
-
5
- Grok CLI (verified `grok 1.0.5`) is a **first-class host** for the AI Engineering Loop. Unlike Antigravity standalone mode, Grok exposes `spawn_subagent` as a real independent child session: own context window, no parent transcript unless `resume_from` is set, and a captured `subagent_id` plus model response.
6
-
7
- When those evidence fields are present, the capability registry **must** select `TRUE_INDEPENDENT_AGENT` (skill alias: `NATIVE_SUBAGENT`).
8
-
9
- ```text
10
- Parent (Maker + orchestrator)
11
- ├─ spawn_subagent type=devil-advocate → Finding Ledger
12
- └─ spawn_subagent type=judge → PASS | ITERATE | ESCALATE
13
- ```
14
-
15
- Children cannot spawn children (depth 1). The parent therefore remains the orchestrator.
16
-
17
- ---
18
-
19
- ## 2. Empirical Discovery Record
20
-
21
- | Surface | Tested command / API | Classification | Result |
22
- |---|---|---|---|
23
- | Grok CLI binary | `~/.grok/bin/grok` (`grok 1.0.5`) | `CONFIGURATION_SUPPORTED` | Installed and authenticated in TUI sessions |
24
- | Native child session | `spawn_subagent` | `INVOCATION_AVAILABLE` | Enabled by default; disabled only when `GROK_SUBAGENTS=0` or `--disallowed-tools Agent` |
25
- | Child execution | child `subagent_id` + model response | `EXECUTION_PROVEN` | Proven only after the child returns; binary presence is not proof |
26
- | Project agents | `.grok/agents/devil-advocate.md`, `.grok/agents/judge.md` | `CONFIGURATION_SUPPORTED` | Registered as `subagent_type` values |
27
- | Skill + slash command | `.grok/skills/ai-engineering-loop/`, `.grok/commands/ai-engineering-loop.md` | `CONFIGURATION_SUPPORTED` | Grok discovers repo-local skills and command markdown |
28
- | Headless child | `grok -p "..."` | `FRESH_PROCESS_AGENT` | Valid fallback **after** stdout contains a model response |
29
- | Compressed reviewer | `caveman:cavecrew-reviewer` | `GROK_COMPRESSED_REVIEW_PRESET` | **Rejected** — output schema is `path:line: emoji severity`, not the Finding Ledger |
30
- | Browser tools | `browser_*` | `BROWSER_AUTOMATION_TOOL` | Not an LLM reviewer |
31
- | `resume_from` Maker | `spawn_subagent(resume_from=makerId)` | tainted history | **Rejected** — inherits Maker transcript |
32
-
33
- ### What is NOT execution proof on Grok
34
-
35
- 1. `~/.grok/bin/grok` existing on disk
36
- 2. `.grok/agents/*.md` being discoverable
37
- 3. Subagents being enabled in `config.toml`
38
- 4. A persona named "reviewer" without a child response
39
- 5. Cavecrew reviewer output that looks like a review
40
-
41
- ---
42
-
43
- ## 3. Mapping Loop Roles → Grok Types
44
-
45
- | Loop role | Grok `subagent_type` | `capability_mode` | `isolation` | `resume_from` |
46
- |---|---|---|---|---|
47
- | Orchestrator / Maker | parent session (or `general-purpose` if parent is orchestrator-only) | `all` | `none` | n/a |
48
- | Devil's Advocate | `devil-advocate` (fallback `general-purpose`) | `execute` | `none` | **omit** |
49
- | Judge | `judge` (fallback `general-purpose`) | `execute` | `none` | **omit** |
50
-
51
- `execute` allows git/read/search and forbids source edits. Do not use `read-only` if the child needs `git diff`. Do not use `worktree` isolation: DA and Judge must see the Maker working tree.
52
-
53
- `description` prefixes `[devil-advocate]` and `[judge]` so the Grok TUI labels the child correctly when falling back to `general-purpose`.
54
-
55
- ---
56
-
57
- ## 4. Artifact Barrier (still required)
58
-
59
- Grok children do not inherit parent chat, but they do receive compacted `AGENTS.md` / project instructions. That is acceptable (project context, not Maker thoughts).
60
-
61
- Still write artifacts to disk and pass **paths**, not Maker narration:
62
-
63
- - Goal Contract
64
- - git diff file
65
- - verification log (command, exit code, stdout, test counts)
66
- - `.ai-engineering-loop/`
67
-
68
- ---
69
-
70
- ## 5. Disable / fallback matrix
71
-
72
- | Condition | Selected mode |
73
- |---|---|
74
- | `spawn_subagent` available, child response captured, no `resume_from` | `TRUE_INDEPENDENT_AGENT` |
75
- | `spawn_subagent` missing, `grok -p` returns a model response | `FRESH_PROCESS_AGENT` |
76
- | `GROK_SUBAGENTS=0` or `--disallowed-tools Agent` | `CONTEXT_ISOLATION_ONLY` |
77
- | No grok, no spawn, no headless | `UNAVAILABLE` or artifact barrier if the same session can still read the diff |
78
-
79
- ---
80
-
81
- ## 6. Answers to the eight host questions
82
-
83
- 1. **Independent sub-agents?** Yes — `spawn_subagent`, own context window.
84
- 2. **Pass context?** Yes — prompt + on-disk artifacts. Do not pass Maker chat.
85
- 3. **Same repository?** Yes — `isolation: none` shares the workspace.
86
- 4. **Structured findings?** Yes — DA agent contract is Finding Ledger JSON.
87
- 5. **Parent consume findings?** Yes — child summary returns to parent; Judge spawn gets the ledger.
88
- 6. **Repeat the loop?** Yes — parent iterates; each DA/Judge spawn is fresh (no `resume_from`).
89
- 7. **Persist state?** Yes — files under `.ai-engineering-loop/` and task artifact paths. Grok also keeps session logs under `~/.grok/sessions/`.
90
- 8. **Token limits?** Pass artifact paths, not full transcripts. Do not dump Maker reasoning into the DA prompt.
91
-
92
- ---
93
-
94
- ## 7. Practical limits on Grok CLI
95
-
96
- - **Depth 1**: a DA child cannot spawn a Judge. Parent must spawn both.
97
- - **Plan mode**: a write-capable child is not gated by the parent's plan-mode lock. Keep DA/Judge on `capability_mode: execute` (no writes) so they cannot edit source while the parent is in plan mode.
98
- - **Headless `--disallowed-tools Agent`**: kills native review. Disclose `CONTEXT_ISOLATION_ONLY`.
99
- - **Cavecrew**: useful for cheap locate/review, **not** for this loop's Judge input.
@@ -1,70 +0,0 @@
1
- # Migration Plan: Legacy 8-Phase Workflow to AI Engineering Loop
2
-
3
- ## 1. Overview & Migration Objectives
4
-
5
- The legacy `dot-dev-workflow` established a strong 8-phase engineering foundation across DOT codebases. However, it tightly coupled generic engineering principles (root cause analysis, surgical diffs, unit testing, static typing) with company-specific delivery tooling (GitLab, `glab`, `@coreview-bot`, Mattermost).
6
-
7
- This migration plan maps every single phase of the legacy workflow into the new modular architecture, ensuring that **zero engineering rigor is lost** while achieving complete platform independence.
8
-
9
- ---
10
-
11
- ## 2. Comprehensive Phase-by-Phase Mapping
12
-
13
- ```mermaid
14
- flowchart TD
15
- subgraph LegacyWorkflow [Legacy 8-Phase Workflow]
16
- L1[Phase 1: Root Cause Analysis]
17
- L2[Phase 2: Collaborative Alignment]
18
- L3[Phase 3: Surgical Code Fix]
19
- L4[Phase 4: Unit Testing]
20
- L5[Phase 5: Typecheck & Build]
21
- L6[Phase 6: Multi-Branch glab MRs]
22
- L7[Phase 7: @coreview-bot Triage]
23
- L8[Phase 8: Mattermost Report]
24
- end
25
-
26
- subgraph NewArchitecture [AI Engineering Loop + DOT Adapter]
27
- GC[core/goal-contract.md]
28
- MA[agents/maker.md]
29
- VL[core/verification-loop.md]
30
- DA[agents/devil-advocate.md]
31
- JD[agents/judge.md]
32
- AD_GL[adapters/dot/gitlab.md]
33
- AD_MB[adapters/dot/multi-branch.md]
34
- AD_CV[adapters/dot/coreview.md]
35
- AD_MM[adapters/dot/mattermost.md]
36
- end
37
-
38
- L1 --> MA
39
- L2 --> GC
40
- L3 --> MA
41
- L4 --> VL
42
- L5 --> VL
43
- L6 --> AD_GL & AD_MB
44
- L7 --> DA & AD_CV
45
- L8 --> AD_MM
46
- ```
47
-
48
- ---
49
-
50
- ## 3. Detailed Mapping Matrix
51
-
52
- | Legacy Phase | New Architecture Placement | Action (`Keep / Modify / Move / Remove`) | Detailed Rationale |
53
- |---|---|:---:|---|
54
- | **Phase 1: Comprehensive Code & Root Cause Analysis** | `agents/maker.md` & `core/goal-contract.md` | **MODIFY & KEEP** | End-to-end data tracing, git history examination (`git log -n 5 -p`), and blind spot identification are preserved in the Maker Agent. The output feeds directly into formalizing the Goal Contract. |
55
- | **Phase 2: Collaborative Alignment & Brainstorming** | `core/goal-contract.md` | **MODIFY & KEEP** | Upgraded from informal conversational alignment to a binding, machine-verifiable [Goal Contract](file:///Users/egagofur/Development/work/ai-engineering-loop/core/goal-contract.md) containing explicit Acceptance Criteria, constraints, and out-of-scope declarations. |
56
- | **Phase 3: Surgical & Clean Code Fixing** | `agents/maker.md` & `core/definition-of-done.md` | **KEEP** | Preserves core quality rules: smallest coherent diff, zero dead code, zero speculative abstractions, adhering to existing architectural conventions and design tokens. |
57
- | **Phase 4: Unit Testing & Boundary Coverage** | `core/verification-loop.md` (Gate 1) & `agents/maker.md` | **KEEP** | Retains 100% test pass requirement, boundary condition testing, null/undefined safety, and edge-case permutations as Deterministic Gate 1. |
58
- | **Phase 5: Build & Type Safety Verification** | `core/verification-loop.md` (Gates 2, 3, 4) | **KEEP** | Retains `tsc --noEmit` and `eslint --fix` as mandatory machine-checked gates that must pass prior to review. |
59
- | **Phase 6: Multi-Branch MR & Issue Card via `glab` CLI** | `adapters/dot/gitlab.md` & `adapters/dot/multi-branch.md` | **MOVE TO ADAPTER** | Moved out of generic core into the DOT Adapter. The core loop remains platform-agnostic, while DOT adapter encapsulates `glab issue create`, `glab mr create`, and cherry-picking to `staging` and `develop`. |
60
- | **Phase 7: Automated Bot Review Triage (`@coreview-bot`)** | `agents/devil-advocate.md` (Internal) + `adapters/dot/coreview.md` (External) | **SPLIT & MOVE** | The concept of adversarial review is promoted to the generic core as the internal [Devil's Advocate Agent](file:///Users/egagofur/Development/work/ai-engineering-loop/agents/devil-advocate.md) (runs *before* opening MR). The external `@coreview-bot` triage (`VALID` vs `HALU`) is moved to `adapters/dot/coreview.md` (runs *after* opening MR). |
61
- | **Phase 8: Mattermost Markdown Report & Auto-Send via MCP** | `adapters/dot/mattermost.md` | **MOVE TO ADAPTER** | Moved out of generic core into the DOT Adapter. Preserves channel mapping lookup (`mattermost-channel-mapping.json`), fallback prompt, and MCP `mattermost_send_message` dispatch with `from: "AI Agent"`. |
62
-
63
- ---
64
-
65
- ## 4. Key Improvements over Legacy Workflow
66
-
67
- 1. **Elimination of Self-Certification**: In the legacy workflow, the same agent that wrote code decided it was done. In the new architecture, only the independent [Judge Agent](file:///Users/egagofur/Development/work/ai-engineering-loop/agents/judge.md) can grant a `PASS` verdict.
68
- 2. **Pre-MR Adversarial Inspection**: Flaws are caught and fixed locally via the [Devil's Advocate](file:///Users/egagofur/Development/work/ai-engineering-loop/agents/devil-advocate.md) *before* publishing PRs/MRs to remote repositories.
69
- 3. **No-Progress & Escalation Circuit Breakers**: Bounded iterations (`MAX_ITERATIONS = 3`) and [No-Progress Detection](file:///Users/egagofur/Development/work/ai-engineering-loop/policies/no-progress-policy.md) prevent runaway loops and thrashing.
70
- 4. **Pluggable Architecture**: The same generic core loop can now be used with GitHub, Bitbucket, Jira, Linear, or Slack simply by swapping adapters.
@@ -1,33 +0,0 @@
1
- # Reference Example: Backend API Payment Idempotency & Race Condition Fix
2
-
3
- ## 1. Scenario Overview
4
-
5
- This walkthrough demonstrates the **AI Engineering Loop** operating in a **Backend API** repository (`backend-api` profile) with standard GitHub adapter integration.
6
-
7
- ### The Problem
8
- A Go/PostgreSQL payment processing microservice experienced double-spend vulnerabilities when users clicked the payment confirmation button multiple times in rapid succession. The database updated account balances without transactional row-level locks or idempotency key uniqueness checks.
9
-
10
- ---
11
-
12
- ## 2. Dynamic Context Resolution
13
-
14
- ```text
15
- Engine: AI Engineering Loop Core
16
- Profile: profiles/backend-api.md
17
- Repo Config: payment-service/.ai-engineering-loop/ (Go 1.22, pgx, GitHub Adapter)
18
- Task: Implement atomic idempotency key validation and SELECT FOR UPDATE row locking
19
- ```
20
-
21
- ---
22
-
23
- ## 3. Walkthrough Artifacts
24
-
25
- 1. **[Goal Contract (`goal-contract.md`)](./goal-contract.md)**:
26
- - AC-1: Idempotency-Key header mandatory on POST `/api/v1/payments/charge`.
27
- - AC-2: Atomic reservation using PostgreSQL `INSERT ... ON CONFLICT DO NOTHING`.
28
- - AC-3: Concurrent requests for same user wallet serialized with `SELECT ... FOR UPDATE`.
29
- 2. **[Adversarial Review Findings (`review-findings.md`)](./review-findings.md)**:
30
- - Devil's Advocate activates `backend-api` rules (concurrency, database atomicity).
31
- - Flags missing rollback on external gateway timeout (`ERR-001`).
32
- 3. **[Judge Verdict (`judge-verdict.md`)](./judge-verdict.md)**:
33
- - Evaluates parallel test execution (`go test -race ./...`), verifies concurrency safety, and issues `PASS` verdict.
@@ -1,33 +0,0 @@
1
- # Goal Contract: Payment Idempotency & Concurrency Lock
2
-
3
- ## 1. Objective
4
- Eliminate double-spending and balance corruption in the `payment-service` by enforcing mandatory `Idempotency-Key` headers, database transaction isolation, and row-level wallet locks.
5
-
6
- ## 2. Business Outcome & User Lifecycle Impact
7
- - **Customers**: Guaranteed never to be charged twice for identical checkout operations regardless of network retries or multiple clicks.
8
- - **Finance**: 100% auditability and balance ledger integrity.
9
-
10
- ## 3. Acceptance Criteria (AC)
11
- - [ ] **AC-1**: Reject `POST /api/v1/payments/charge` with HTTP 400 if `Idempotency-Key` header is missing or empty.
12
- - [ ] **AC-2**: Idempotency records stored atomically with 24-hour expiration window. Duplicate requests return cached original response with `X-Cache: HIT`.
13
- - [ ] **AC-3**: User wallet balance updates serialized via `SELECT balance, version FROM wallets WHERE id = $1 FOR UPDATE`.
14
- - [ ] **AC-4**: Zero race conditions detected under 50 simultaneous parallel requests.
15
-
16
- ## 4. Technical Constraints
17
- - Maintain Go 1.22 standard library and `github.com/jackc/pgx/v5`.
18
- - No distributed Redis locks unless DB transactions prove insufficient.
19
- - Zero breaking changes to response JSON schemas.
20
-
21
- ## 5. Out of Scope
22
- - Modifying refund or settlement batch jobs.
23
-
24
- ## 6. Verification Requirements
25
- - **Unit & Race Tests**: `go test -race -v -count=1 ./internal/payment/...`
26
- - **Lint**: `golangci-lint run ./...`
27
- - **Build**: `go build -o /dev/null ./cmd/server`
28
-
29
- ## 7. Definition of Done (DoD)
30
- - [ ] AC-1 through AC-4 verified via automated race test suites.
31
- - [ ] Deterministic verification 100% green.
32
- - [ ] Devil's Advocate review conducted with 0 open SEV-1/2 findings.
33
- - [ ] Judge issues PASS verdict.
@@ -1,28 +0,0 @@
1
- # Judge Evaluation Report: Payment Idempotency
2
-
3
- ## 1. Executive Verdict
4
- - **Verdict**: `PASS`
5
- - **Iteration**: `Iteration 2 of 3`
6
- - **Confidence**: `HIGH`
7
-
8
- ---
9
-
10
- ## 2. Deterministic Verification Audit
11
-
12
- | Check | Command Executed | Raw Result | Status |
13
- |---|---|---|:---:|
14
- | **Race Tests** | `go test -race -v ./internal/payment/...` | `PASS: TestConcurrentPaymentDeductions (50 goroutines). 0 race warnings. ok 2.18s` | ✅ PASS |
15
- | **Idempotency Suite**| `go test -v ./internal/payment/idempotency_test.go` | `PASS: TestIdempotencyDuplicateRequestReturnsCached. ok 0.42s` | ✅ PASS |
16
- | **Linter** | `golangci-lint run ./...` | `0 issues found.` | ✅ PASS |
17
- | **Build** | `go build -o /dev/null ./cmd/server` | `Exit code 0.` | ✅ PASS |
18
-
19
- ---
20
-
21
- ## 3. Finding Triage Audit
22
- - **CONC-001 (Critical - Dangling DB Transaction)**: Resolved with `defer tx.Rollback(ctx)`. Verified by timeout test suite. **Status: RESOLVED & VERIFIED**.
23
-
24
- ---
25
-
26
- ## 4. Hand-off
27
- The Definition of Done has been completely satisfied.
28
- **Authorized next action**: Hand off to configured GitHub Adapter for Pull Request generation.
@@ -1,50 +0,0 @@
1
- # Adversarial Review Findings: Payment Idempotency
2
-
3
- ## 1. Review Summary
4
- - **Reviewer**: Devil's Advocate Agent (Profile: `backend-api`)
5
- - **Active Review Domains**: Auth, Database Transactions, Concurrency/Race, N+1, API Contracts
6
- - **Total Findings**: 1
7
- - **Blocking (SEV-1/2)**: 1
8
-
9
- ---
10
-
11
- ## 2. Findings Ledger
12
-
13
- ### Finding CONC-001: Missing transaction rollback when external payment gateway times out
14
- - **Severity**: `CRITICAL` (SEV-1)
15
- - **Category**: `Concurrency & Error Handling`
16
- - **Location**: `internal/payment/service.go:88-102`
17
- - **Evidence**:
18
- ```go
19
- tx, _ := s.db.Begin(ctx)
20
- wallet.Deduct(amount)
21
- resp, err := s.gatewayClient.Charge(ctx, req)
22
- if err != nil {
23
- return nil, err // Exits without tx.Rollback(ctx)
24
- }
25
- tx.Commit(ctx)
26
- ```
27
- - **Problem**:
28
- If the external payment gateway call returns a timeout or network error, the function exits early without invoking `tx.Rollback(ctx)`. In Go `pgx`, the connection remains locked in a dangling transaction until connection pool cleanup, and the user's wallet balance stays locked.
29
- - **Impact**:
30
- Connection pool exhaustion and locked wallet accounts during network blips.
31
- - **Recommendation**:
32
- ```diff
33
- tx, err := s.db.Begin(ctx)
34
- if err != nil {
35
- return nil, err
36
- }
37
- + defer tx.Rollback(ctx)
38
-
39
- wallet.Deduct(amount)
40
- resp, err := s.gatewayClient.Charge(ctx, req)
41
- if err != nil {
42
- return nil, err
43
- }
44
- - tx.Commit(ctx)
45
- + return resp, tx.Commit(ctx)
46
- ```
47
- - **Confidence**: `HIGH`
48
- - **Status**: `TRIAGED_VALID`
49
- - **Resolution**:
50
- Maker Agent implemented `defer tx.Rollback(ctx)` and verified with a unit test simulating gateway timeout.
@@ -1,23 +0,0 @@
1
- # Reference Example: Request Display Status Fix
2
-
3
- ## 1. Context & Scenario
4
-
5
- This walkthrough shows how a status-display bug moves through the **AI Engineering Loop** and the **DOT Delivery Adapter**. Names, URLs, and schema below are **fictional**. Do not copy real client tickets, GitLab links, or field names into this package.
6
-
7
- ### The Problem
8
-
9
- On an employee portal, request cards on the list and pagination endpoints showed `"PENDING"` after every line item was already approved or rejected. Non-standard-window records were also scored with the standard-day rule.
10
-
11
- ---
12
-
13
- ## 2. Walkthrough Stages & Artifacts
14
-
15
- 1. **[Stage 1: Goal Contract (`goal-contract.md`)](./goal-contract.md)**:
16
- - Objective, acceptance criteria (standard window, holiday, non-standard window, null values), and verification plan.
17
- 2. **[Stage 2: Adversarial Review & Triage (`review-findings.md`)](./review-findings.md)**:
18
- - Devil's Advocate flags a null-safety bug (`COR-001`) and an invalid nitpick (`MAINT-001`).
19
- - Maker triage and surgical fix.
20
- 3. **[Stage 3: Judge Evaluation & Verdict (`judge-verdict.md`)](./judge-verdict.md)**:
21
- - Audit of tests, typecheck, finding resolutions, and `PASS`.
22
- 4. **[Stage 4: DOT Delivery Pipeline (`delivery-report.md`)](./delivery-report.md)**:
23
- - Fictional issue, base MR, cherry-picks to `staging` and `develop`, Coreview triage, Mattermost notice.
@@ -1,41 +0,0 @@
1
- # DOT Delivery Report: Request Display Status Fix
2
-
3
- Fictional sample. Replace host, group, repo, issue, and MR ids from `glab` output. Never paste real client URLs into this package.
4
-
5
- ## 1. Summary of Delivery Actions
6
-
7
- 1. **GitLab Issue Created**: [Issue #42 - [BE] [Requests] Fix display status](https://gitlab.example.com/example-org/work-portal/-/issues/42).
8
- 2. **Primary MR Created**: [MR !10 targeting `main`](https://gitlab.example.com/example-org/work-portal/-/merge_requests/10).
9
- 3. **Multi-Branch Cherry-Pick**:
10
- - [MR !11 targeting `staging`](https://gitlab.example.com/example-org/work-portal/-/merge_requests/11).
11
- - [MR !12 targeting `develop`](https://gitlab.example.com/example-org/work-portal/-/merge_requests/12).
12
- 4. **Coreview Bot Triage**:
13
- - Comments fetched on MR !12. Zero blocking comments.
14
- 5. **Mattermost Notification**:
15
- - Resolved repository `example-org/work-portal` to channel `"team-work-portal"`.
16
- - Dispatched via MCP `mattermost_send_message` with `from: "AI Agent"`.
17
-
18
- ---
19
-
20
- ## 2. GitLab Links
21
-
22
- | Artifact | Branch | Link |
23
- |---|---|---|
24
- | **GitLab Issue** | — | [Issue #42](https://gitlab.example.com/example-org/work-portal/-/issues/42) |
25
- | **Merge Request DEV** | `develop` | [MR !12](https://gitlab.example.com/example-org/work-portal/-/merge_requests/12) |
26
- | **Merge Request STAGING** | `staging` | [MR !11](https://gitlab.example.com/example-org/work-portal/-/merge_requests/11) |
27
- | **Merge Request MAIN** | `main` | [MR !10](https://gitlab.example.com/example-org/work-portal/-/merge_requests/10) |
28
-
29
- ---
30
-
31
- ## 3. Dispatched Mattermost Notification
32
-
33
- ```text
34
- [MR DEV] https://gitlab.example.com/example-org/work-portal/-/merge_requests/12
35
- Changes log
36
- - Include standard-window and rush-note fields when loading request list pages.
37
- - Resolve display status from all line items, including holiday and null collections.
38
- - Add unit tests for standard, holiday, non-standard-window, and null line-item cases.
39
-
40
- cc: @owner
41
- ```
@@ -1,46 +0,0 @@
1
- # Goal Contract: Request Display Status Fix
2
-
3
- ## 1. Objective
4
-
5
- Fix incorrect display status on list and pagination endpoints when a request has mixed line items, holiday logs, non-standard windows, or null collections.
6
-
7
- ## 2. Business Outcome & User Lifecycle Impact
8
-
9
- - **Employees**: See APPROVED, REJECTED, or NEED_APPROVAL matching the real record.
10
- - **Reviewers**: Stop getting false pending notices for already-settled holiday logs.
11
- - **Finance**: Totals used by the downstream aggregate job stay consistent.
12
-
13
- ## 3. Acceptance Criteria (AC)
14
-
15
- - [ ] **AC-1**: If `hasStandardWindow = true` and `duration` is within the standard day limit on a weekday without a rush note, display status is `APPROVED` or `NEED_APPROVAL` from clock-in/out presence only.
16
- - [ ] **AC-2**: If a rush note exists, or duration exceeds the standard limit, or `isHoliday = true`, evaluate pending status across all associated `lineItems`.
17
- - [ ] **AC-3**: Non-standard-window records (`user.type.hasStandardWindow = false`) must not use the weekday duration threshold.
18
- - [ ] **AC-4**: Null or missing `lineItems` default to `APPROVED` without throwing `TypeError`.
19
- - [ ] **AC-5**: The existing API response contract is unchanged.
20
-
21
- ## 4. Technical Constraints
22
-
23
- - Keep the existing list/pagination query shape.
24
- - Centralize logic in a pure function: `resolveRequestDisplayStatus`.
25
- - No new runtime dependencies.
26
- - `tsc --noEmit` exits 0.
27
-
28
- ## 5. Out of Scope
29
-
30
- - Frontend layout changes.
31
- - Database migrations.
32
- - Changing the downstream aggregate/export job.
33
-
34
- ## 6. Verification Requirements
35
-
36
- - **Unit Tests**: Jest suite at `src/server/requests/utils/resolve-display-status.test.ts`.
37
- - **Typecheck**: `npx tsc --noEmit` exits 0.
38
- - **Linter**: `npx eslint --fix` on modified files, 0 errors.
39
- - **Regression**: commands in `.ai-engineering-loop/verification.md` (example: `npx jest && npx tsc --noEmit`).
40
-
41
- ## 7. Definition of Done (DoD)
42
-
43
- - [ ] AC-1 through AC-5 proven by unit tests.
44
- - [ ] Deterministic verification passes.
45
- - [ ] Devil's Advocate review with 0 unresolved blocking findings.
46
- - [ ] Judge issues PASS.
@@ -1,44 +0,0 @@
1
- # Judge Evaluation Report: Request Display Status Fix
2
-
3
- ## 1. Executive Verdict
4
-
5
- - **Verdict**: `PASS`
6
- - **Iteration**: `Iteration 2 of 3`
7
- - **Confidence**: `HIGH`
8
-
9
- ---
10
-
11
- ## 2. Deterministic Verification Audit
12
-
13
- | Check | Command Executed | Raw Result | Status |
14
- |---|---|---|:---:|
15
- | **Unit Tests** | `npx jest src/server/requests/utils/resolve-display-status.test.ts` | `Tests: 12 passed, 12 total.` | PASS |
16
- | **Full Suite** | `npx jest && npx tsc --noEmit` | `Test Suites: 12 passed. tsc exit 0.` | PASS |
17
- | **TypeScript** | `npx tsc --noEmit` | `Exit code 0.` | PASS |
18
- | **Linter** | `npx eslint --fix src/server/requests/**` | `0 errors.` | PASS |
19
-
20
- ---
21
-
22
- ## 3. Goal Contract Compliance Audit
23
-
24
- | Criterion | Verified By Test / Artifact | Result |
25
- |---|---|:---:|
26
- | **AC-1**: Standard-window weekday | `resolve-display-status.test.ts > standard window` | PASS |
27
- | **AC-2**: Holiday and over-limit handling | `resolve-display-status.test.ts > holiday rush` | PASS |
28
- | **AC-3**: Non-standard-window records | `resolve-display-status.test.ts > non-standard window` | PASS |
29
- | **AC-4**: Null `lineItems` | `resolve-display-status.test.ts > null safety` | PASS |
30
- | **AC-5**: API contract | router typecheck | PASS |
31
-
32
- ---
33
-
34
- ## 4. Finding Triage Audit
35
-
36
- - **COR-001 (High - Null Safety)**: Maker applied null coalescing in Iteration 2. **Status: RESOLVED & VERIFIED**.
37
- - **MAINT-001 (Low - Factory Suggestion)**: Invalid speculative nitpick. **Status: DISMISSED / INVALID**.
38
-
39
- ---
40
-
41
- ## 5. Formal Conclusion & Hand-off
42
-
43
- Definition of Done is met with reproducible evidence.
44
- **Authorized next action**: DOT Delivery Adapter (`adapters/dot/README.md`) for GitLab MR, multi-branch cherry-pick, Coreview triage, and Mattermost.