ai-engineering-loop 1.0.21 → 1.2.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (115) hide show
  1. package/.agents/workflows/ai-engineering-loop.md +8 -4
  2. package/.claude/skills/ai-engineering-loop/SKILL.md +15 -7
  3. package/.gemini/skills/ai-engineering-loop/SKILL.md +15 -7
  4. package/.grok/skills/ai-engineering-loop/SKILL.md +15 -7
  5. package/README.md +43 -3
  6. package/README.npm.md +9 -1
  7. package/SECURITY.md +29 -0
  8. package/SUPPORT.md +28 -0
  9. package/agents/judge.md +5 -5
  10. package/agents/maker.md +6 -6
  11. package/bin/ai-engineering-loop.js +472 -10
  12. package/core/artifact-gates.md +49 -0
  13. package/core/configuration-precedence.md +4 -4
  14. package/core/context-impact-assessment.md +1 -1
  15. package/core/definition-of-done.md +3 -3
  16. package/core/escalation-policy.md +2 -2
  17. package/core/goal-contract.md +2 -2
  18. package/core/iteration-policy.md +4 -4
  19. package/core/project-initialization.md +2 -2
  20. package/core/runtime-safety.md +75 -0
  21. package/core/verification-loop.md +1 -1
  22. package/evals/README.md +35 -0
  23. package/evals/cases/01-stale-verification.json +17 -0
  24. package/evals/cases/02-vague-verification.json +16 -0
  25. package/evals/cases/03-missing-ac-evidence.json +17 -0
  26. package/evals/cases/04-hallucinated-citation.json +17 -0
  27. package/evals/cases/05-prompt-injection.json +17 -0
  28. package/evals/cases/06-source-secret.json +16 -0
  29. package/evals/cases/07-log-secret.json +16 -0
  30. package/evals/cases/08-symlink-escape.json +16 -0
  31. package/evals/cases/09-authorization-bypass.json +16 -0
  32. package/evals/cases/10-payment-idempotency.json +16 -0
  33. package/evals/cases/11-concurrent-update.json +16 -0
  34. package/evals/cases/12-swallowed-error.json +16 -0
  35. package/evals/cases/13-happy-path-only.json +17 -0
  36. package/evals/cases/14-invalid-finding-axis.json +16 -0
  37. package/evals/cases/15-blocker-without-alternative.json +16 -0
  38. package/evals/cases/16-unsupported-pass.json +17 -0
  39. package/evals/cases/17-soft-standard.json +16 -0
  40. package/evals/cases/18-iteration-ceiling.json +17 -0
  41. package/evals/cases/19-safe-documentation-change.json +17 -0
  42. package/evals/cases/20-valid-boundary-feature.json +18 -0
  43. package/lib/budget.js +220 -0
  44. package/lib/doctor.js +111 -0
  45. package/lib/escalation.js +192 -0
  46. package/lib/evaluation.js +137 -0
  47. package/lib/gates.js +369 -0
  48. package/lib/orchestration.js +5 -1
  49. package/lib/package-audit.js +78 -0
  50. package/lib/run-state.js +260 -0
  51. package/lib/runtime-files.js +53 -0
  52. package/lib/runtime-policy.js +118 -0
  53. package/lib/safe-context.js +194 -0
  54. package/lib/sandbox.js +305 -0
  55. package/lib/sync-hosts.js +6 -5
  56. package/package.json +16 -10
  57. package/policies/finding-policy.md +2 -2
  58. package/policies/model-escalation.md +41 -0
  59. package/policies/no-progress-policy.md +2 -2
  60. package/profiles/README.md +5 -5
  61. package/profiles/backend-api.md +1 -1
  62. package/profiles/library.md +1 -1
  63. package/profiles/mobile-app.md +1 -1
  64. package/profiles/monorepo.md +1 -1
  65. package/profiles/web-app.md +1 -1
  66. package/schemas/delivery.schema.json +15 -0
  67. package/schemas/finding-ledger.schema.json +27 -0
  68. package/schemas/goal-contract.schema.json +32 -0
  69. package/schemas/report.schema.json +17 -0
  70. package/schemas/verdict.schema.json +17 -0
  71. package/schemas/verification.schema.json +32 -0
  72. package/docs/antigravity-feasibility.md +0 -92
  73. package/docs/claude-code-feasibility.md +0 -80
  74. package/docs/grok-cli-feasibility.md +0 -99
  75. package/docs/migration-plan.md +0 -70
  76. package/examples/backend-api/payment-idempotency/README.md +0 -33
  77. package/examples/backend-api/payment-idempotency/goal-contract.md +0 -33
  78. package/examples/backend-api/payment-idempotency/judge-verdict.md +0 -28
  79. package/examples/backend-api/payment-idempotency/review-findings.md +0 -50
  80. package/examples/dot/status-display/README.md +0 -23
  81. package/examples/dot/status-display/delivery-report.md +0 -41
  82. package/examples/dot/status-display/goal-contract.md +0 -46
  83. package/examples/dot/status-display/judge-verdict.md +0 -44
  84. package/examples/dot/status-display/review-findings.md +0 -63
  85. package/examples/initialization/README.md +0 -19
  86. package/examples/initialization/discovery-trace.md +0 -63
  87. package/examples/initialization/generated-context.md +0 -110
  88. package/examples/mobile-app/offline-sync-queue/README.md +0 -33
  89. package/examples/mobile-app/offline-sync-queue/goal-contract.md +0 -32
  90. package/examples/mobile-app/offline-sync-queue/judge-verdict.md +0 -27
  91. package/examples/mobile-app/offline-sync-queue/review-findings.md +0 -30
  92. package/scripts/init.sh +0 -106
  93. package/templates/repo-config/adapter.md +0 -11
  94. package/templates/repo-config/adr-readme.md +0 -33
  95. package/templates/repo-config/architecture.md +0 -15
  96. package/templates/repo-config/claude-permissions.json +0 -10
  97. package/templates/repo-config/config.md +0 -11
  98. package/templates/repo-config/conventions.md +0 -16
  99. package/templates/repo-config/glossary.md +0 -18
  100. package/templates/repo-config/lessons.md +0 -8
  101. package/templates/repo-config/verification.md +0 -13
  102. package/templates/repo-config/workflow.md +0 -25
  103. package/tests/capability-selection.test.js +0 -207
  104. package/tests/claimed-vs-reality.test.js +0 -27
  105. package/tests/dot-branch-policy.test.js +0 -65
  106. package/tests/generate-adapter.test.js +0 -80
  107. package/tests/generate-workflow.test.js +0 -180
  108. package/tests/grok-runtime.test.js +0 -201
  109. package/tests/intern-catalog.test.js +0 -72
  110. package/tests/living-context.test.js +0 -61
  111. package/tests/no-company-leak.test.js +0 -82
  112. package/tests/orchestration.test.js +0 -368
  113. package/tests/qa-delivery-template.test.js +0 -90
  114. package/tests/skill-host-compat.test.js +0 -406
  115. package/tests/sync-hosts.test.js +0 -168
@@ -7,14 +7,18 @@ Follow `policies/review-budget.md`. Parent is Maker plus orchestrator. Do not us
7
7
 
8
8
  Compact map (not a second OS): Specify (stages 0-1), Make (stages 2-4), Review (stages 5-7), Deliver (stage 8). Keep the 8-stage numbers. Do not skip Goal Contract, verification, Devil's Advocate, or Judge.
9
9
 
10
- Stage 1: if the user asks for ideas, list a short menu and wait; do not implement. Grill if the task is ambiguous (`core/grill-policy.md`), then freeze the Goal Contract file. Chat agreement is not freeze. If the user prompt has no numbered AC, draft `.ai-engineering-loop/tasks/goal-contract.md` with a failure table, show it, wait for freeze. Do not start Maker. On business-logic change, and always on `adapter_type: dot`, grill includes blast radius: lifecycle sketch, four pillars (state, sibling, approval, queues), ASCII picture. Load `task-impact-inquiry` if present. Do not run a second interview. Passing unit tests are not isolation proof. Use `.ai-engineering-loop/glossary.md`. Name test seams. AC is a failure table (happy, empty/omit, boundary, sibling, error). One red test per AC row. Do not freeze sunny-path-only. Stages 2-4: Read `maker_intern` from workflow.md (missing or none = parent is Maker). Do not type a model name. If intern is set, confirm it came from this host's list (`/models` if that command exists). If the list command is missing or the id is not on it, parent is Maker; say intern INVOCATION_UNAVAILABLE. Intern cannot skip verification, Devil's Advocate, or Judge. Maker: TDD at those seams (`policies/tdd-policy.md`): one red test per AC row. Bugs: red repro first (`core/root-cause-analysis.md`). Mid-loop stop: `core/handoff-policy.md`.
10
+ Deterministic gates are mandatory. Read `core/runtime-safety.md`. Stage 0 runs sync-hosts, status, then `npx ai-engineering-loop run [--mode report-only|assisted|unattended]`; `state --json` gives runId, immutable mode, and `.ai-engineering-loop/runs/<runId>/` (`RUN_DIR`). Never edit state.json. Run `budget status --run <runId>` before every model/subagent call and `budget record --input <actual> --output <actual> --model <id> --run <runId>` immediately after provider usage is returned. Never invent counts. Missing accounting stops UNATTENDED and requires human direction in ASSISTED. `budget pause` is the kill switch. Stage 1 keeps Goal Contract Markdown, writes goal-contract.json with failure cases, then runs `gate goal`.
11
+
12
+ Inherited invariants: read `core/grill-policy.md`, `policies/tdd-policy.md`, and `.ai-engineering-loop/glossary.md`. On `adapter_type: dot`, grill includes blast radius: lifecycle sketch and load `task-impact-inquiry`. If the user prompt has no numbered AC, draft the contract, list a short menu and wait. Chat agreement is not freeze. AC is a failure table. One red test per AC row. Do not freeze sunny-path-only. Write claimed-vs-reality.md. Do not spawn Devil's Advocate if the file is missing or any Claimed row lacks Reality. Report Spec and Standards separately. DA prompt: "at most 8 tool calls; read the diff file; skip css and generated files". Judge prompt: "at most 4 tool calls; ledger and contract only; skip css; do not re-review the whole diff". Never use `browser_subagent` as reviewer. `generate-workflow` reads workflow.md and lessons.md; missing maker_intern means INVOCATION_UNAVAILABLE. Do not type a model name.
13
+
14
+ Stage 1: if the user asks for ideas, list a short menu and wait; do not implement. Grill if the task is ambiguous (`core/grill-policy.md`), then freeze the Goal Contract file. Chat agreement is not freeze. If the user prompt has no numbered AC, draft `.ai-engineering-loop/tasks/goal-contract.md` with a failure table, show it, wait for freeze. Do not start Maker. On business-logic change, and always on `adapter_type: dot`, grill includes blast radius: lifecycle sketch, four pillars (state, sibling, approval, queues), ASCII picture. Load `task-impact-inquiry` if present. Do not run a second interview. Passing unit tests are not isolation proof. Use `.ai-engineering-loop/glossary.md`. Name test seams. AC is a failure table (happy, empty/omit, boundary, sibling, error). One red test per AC row. Do not freeze sunny-path-only. Stages 2-4 branch on mode: REPORT_ONLY never edits repository files, writes RUN_DIR/report.json, runs `gate report`, and stops. ASSISTED uses normal Maker. UNATTENDED runs `sandbox create --run <runId>`, Maker only in the returned worktreePath, never edits the parent checkout, and runs `sandbox capture --run <runId>`. Read `maker_intern` from workflow.md (missing or none = parent is Maker). Do not type a model name. If intern is set, confirm it came from this host's list (`/models` if that command exists). If the list command is missing or the id is not on it, parent is Maker; say intern INVOCATION_UNAVAILABLE. Intern cannot skip verification, Devil's Advocate, or Judge. Maker: TDD at those seams (`policies/tdd-policy.md`): one red test per AC row. ASSISTED writes RUN_DIR/diff.patch; UNATTENDED capture writes it; then `gate maker`. Stage 5 writes current RUN_DIR/verification.json and runs `gate verification`. Bugs: red repro first (`core/root-cause-analysis.md`). Mid-loop stop: `core/handoff-policy.md`.
11
15
 
12
16
  If `invoke_subagent` (or Task) exists, spawn `devil-advocate` then `judge` as siblings. Wait for each child. Do not run them in the background. Use `general-purpose` only if the named type is rejected.
13
17
 
14
18
  5. Before Devil's Advocate: write `git diff` to `.ai-engineering-loop/tasks/current.diff` and put `git diff --name-only` in the child prompt. Do not paste Maker rationale. Write `.ai-engineering-loop/tasks/claimed-vs-reality.md` (AC, Claimed, Reality from command log). Do not spawn Devil's Advocate if the file is missing or any Claimed row lacks Reality.
15
- 6. Devil's Advocate prompt: diff file path, name-only list, Goal Contract path, verification log path, conventions.md path, and "at most 8 tool calls; read the diff file; skip css and generated files". Report Spec and Standards axes separately.
16
- 7. Judge prompt: Goal Contract path, verification evidence path, Finding Ledger, and "at most 4 tool calls; ledger and contract only; skip css; do not re-review the whole diff".
19
+ 6. Build `context devil-advocate`, check `budget status`, spawn DA with the pack path, wait, `budget record` actual usage, normalize RUN_DIR/findings.json with runId and diffHash, then `gate review`.
20
+ 7. Run `escalation --json`, resolve the required tier from the host catalog, build Judge context from only goal-contract.json, verification.json, and findings.json, check `budget status`, spawn Judge, wait, `budget record` actual usage, write verdict.json with modelTier, then `gate judge`.
17
21
 
18
22
  If no subagent tool exists, run the same budgets in this session as CONTEXT_ISOLATION_ONLY. Do not claim independent agent execution.
19
23
 
20
- Init/status/refresh/sync-hosts/generate-adapter/generate-workflow: `npx ai-engineering-loop <command>`. generate-workflow grills Q1-Q6. Q6 Maker intern: pick from the host catalog (`grok models` or `/models` if that command exists); do not type a model name. Stage 0: `npx ai-engineering-loop sync-hosts` then `npx ai-engineering-loop status`. Read `.ai-engineering-loop/glossary.md`. Read `.ai-engineering-loop/lessons.md` and `.ai-engineering-loop/workflow.md` if they exist. Follow workflow.md hooks (`before_grill`, `after_freeze`, `after_pass`). Do not skip Goal Contract, verification, Devil's Advocate, or Judge even if workflow.md asks. If workflow.md is missing, run the default 8-stage loop (no extra hooks). If sync-hosts copied files, tell the user a new session is needed for updated skill text; keep going with this session. After Judge PASS: run the `after_pass` hook if it is not `none`. Stage 8: load `adapters/<adapter_type>/`. If adapter.md is missing, run `generate-adapter` (grill). Do not invent a company pipeline. PR/MR body must include `Steps to Reproduce & Testing (QA)` from the adapter (reproduce + how to test). On a bugfix, do not leave those placeholders empty. Confirmed process lessons go to `lessons.md`, not a chat dump.
24
+ Init/status/refresh/sync-hosts/generate-adapter/generate-workflow: `npx ai-engineering-loop <command>`. generate-workflow grills Q1-Q6. Q6 Maker intern: pick from the host catalog; do not type a model name. After Judge PASS, PR/MR body includes `Steps to Reproduce & Testing (QA)` and bugfix placeholders must not stay empty. Preview Stage 8 remote payloads and never include context packs, usage ledgers, raw logs, or credentials. In ASSISTED, wait for explicit human approval and write `humanApproved: true` in RUN_DIR/delivery.json; never infer it. Run `gate delivery`; only that gate completes the run. Confirmed process lessons go to lessons.md, not a chat dump.
@@ -35,20 +35,28 @@ Compact map (not a second OS): Specify (stages 0-1), Make (stages 2-4), Review (
35
35
 
36
36
  Parent session is Maker plus orchestrator. Spawn Devil's Advocate and Judge as **siblings**, not nested.
37
37
 
38
- 1. Stage 0: `npx ai-engineering-loop sync-hosts` then `npx ai-engineering-loop status` (init or refresh if missing or stale). Read `.ai-engineering-loop/glossary.md`. Read `.ai-engineering-loop/lessons.md` and `.ai-engineering-loop/workflow.md` if they exist. Follow workflow.md hooks (`before_grill`, `after_freeze`, `after_pass`). Do not skip Goal Contract, verification, Devil's Advocate, or Judge even if workflow.md asks. If workflow.md is missing, run the default 8-stage loop (no extra hooks). If sync-hosts copied files, tell the user a new session is needed for updated skill text; keep going with this session.
39
- 2. Stage 1: Goal Contract (`core/goal-contract.md`). If the user asks for ideas, list a short menu and wait; do not implement. If the task is ambiguous and the user can answer, grill first (`core/grill-policy.md`): design tree, recommended answers, do not ask look-up facts. Skip grill if the contract is already frozen or the user waived it. On business-logic change, and always on `adapter_type: dot`, grill includes blast radius: lifecycle sketch, four pillars (state, sibling, approval, queues), ASCII picture. Load `task-impact-inquiry` if present. Do not run a second interview. Passing unit tests are not isolation proof. Chat agreement is not freeze: every user-visible decision must be a numbered AC in the Goal Contract file. Freeze before any production edit. Name test seams. Use glossary terms. AC is a failure table (happy, empty/omit, boundary, sibling, error). One red test per AC row. Do not freeze sunny-path-only. If the user prompt has no numbered AC, draft `.ai-engineering-loop/tasks/goal-contract.md` with a failure table, show it, wait for freeze. Do not start Maker.
40
- 3. Stages 2-4: Read `maker_intern` from workflow.md (missing or none = parent is Maker). Do not type a model name. If intern is set, confirm it came from this host's list (`/models` if that command exists). If the list command is missing or the id is not on it, parent is Maker; say intern INVOCATION_UNAVAILABLE. Do not add a model key or HTTP client. Intern cannot skip verification, Devil's Advocate, or Judge. Bugs: red repro first (`core/root-cause-analysis.md`). Features: TDD at named seams (`policies/tdd-policy.md`): one red test per AC row. Surgical diff.
41
- 4. Stage 5: run commands from `.ai-engineering-loop/verification.md`. Keep command, exit code, stdout, test counts. Vague "seems green" is invalid. Write `.ai-engineering-loop/tasks/claimed-vs-reality.md` (AC, Claimed, Reality from command log). Do not spawn Devil's Advocate if the file is missing or any Claimed row lacks Reality.
38
+ ## Deterministic artifact gates (mandatory)
39
+
40
+ Run `npx ai-engineering-loop state --json` to get the current `runId` and use `.ai-engineering-loop/runs/<runId>/` as `RUN_DIR`. Never edit `state.json` or `current.json`.
41
+
42
+ Read `core/runtime-safety.md`. The ledger mode is immutable. Before every model or Task call, run `npx ai-engineering-loop budget status --run <runId>`; after it returns, record actual provider-reported usage with `budget record --input <n> --output <n> --model <id> --run <runId>`. Never invent actual token counts. If usage metadata is unavailable, UNATTENDED stops and escalates; ASSISTED discloses the gap and asks the human whether to continue. `budget pause` is an emergency kill switch.
43
+
44
+ Inherited invariants remain mandatory: read `core/grill-policy.md`, `policies/tdd-policy.md`, and `.ai-engineering-loop/glossary.md`. On `adapter_type: dot`, grill includes blast radius: lifecycle sketch and load `task-impact-inquiry`. If the user prompt has no numbered AC, draft the contract, list a short menu and wait. Chat agreement is not freeze. AC is a failure table. One red test per AC row. Do not freeze sunny-path-only. Write claimed-vs-reality.md. Do not spawn Devil's Advocate if the file is missing or any Claimed row lacks Reality. Report Spec and Standards separately. DA prompt includes "at most 8 tool calls; read the diff file; skip css and generated files". Judge prompt includes "at most 4 tool calls; ledger and contract only; skip css; do not re-review the whole diff".
45
+
46
+ 1. Stage 0: `npx ai-engineering-loop sync-hosts`, `status`, then `run [--mode report-only|assisted|unattended]` to create or resume the ledger. Run `state --json` and `budget status`. Read `.ai-engineering-loop/glossary.md`. Read `.ai-engineering-loop/lessons.md` and `.ai-engineering-loop/workflow.md` if they exist. Follow workflow.md hooks (`before_grill`, `after_freeze`, `after_pass`). Do not skip required gates even if workflow.md asks. If workflow.md is missing, run the default 8-stage loop. If sync-hosts copied files, tell the user a new session is needed; keep going with this session.
47
+ 2. Stage 1: Goal Contract (`core/goal-contract.md`). If the user asks for ideas, list a short menu and wait; do not implement. If the task is ambiguous and the user can answer, grill first (`core/grill-policy.md`): design tree, recommended answers, do not ask look-up facts. On business-logic change, and always on `adapter_type: dot`, include blast radius. Chat agreement is not freeze. Keep the human Markdown contract and write `RUN_DIR/goal-contract.json` with `schemaVersion`, `runId`, objective, numbered AC, evidenceRequired, and non-empty failureCases. Run `npx ai-engineering-loop gate goal`; do not edit production code unless it passes.
48
+ 3. After Goal freeze, branch on mode. REPORT_ONLY: do not edit repository files; write `RUN_DIR/report.json` with summary and recommendations, then run `gate report` and stop. ASSISTED: run the normal Maker and later require human approval. UNATTENDED: run `sandbox create --run <runId>`, set Maker's working directory to the returned worktreePath, never edit the parent checkout, then run `sandbox capture --run <runId>`. For Maker, read `maker_intern` from workflow.md (missing or none = parent is Maker). Do not type a model name. Confirm it through `/models` when available; otherwise parent is Maker and report INVOCATION_UNAVAILABLE. Intern cannot skip later gates. Bugs: red repro first. Features: one red test per AC row. In ASSISTED write the exact review diff to `RUN_DIR/diff.patch`; UNATTENDED capture writes it. Then run `npx ai-engineering-loop gate maker`.
49
+ 4. Stage 5: run commands from `.ai-engineering-loop/verification.md`. Write `RUN_DIR/verification.json` with current git revision, gated diff hash, command, execution identity, times, exit code, stdout, timeout status, and test counts. Vague "seems green" is invalid. Keep `.ai-engineering-loop/tasks/claimed-vs-reality.md`. Run `npx ai-engineering-loop gate verification`; do not spawn Devil's Advocate unless it passes.
42
50
  5. Write artifacts, then spawn. Before Devil's Advocate:
43
51
  - Write `git diff` to a file (for example `.ai-engineering-loop/tasks/current.diff`).
44
52
  - Write changed paths (`git diff --name-only`) into the Task prompt as a short list.
45
53
  - Put those paths in the child prompt. Do not paste Maker rationale.
46
54
  - Write `.ai-engineering-loop/tasks/claimed-vs-reality.md` if missing. Do not spawn DA without it.
47
55
  - If stopping mid-loop, write `.ai-engineering-loop/tasks/handoff.md` (`core/handoff-policy.md`).
48
- 6. Stage 6: Task `subagent_type: devil-advocate`. Use `general-purpose` only if that type is rejected. If the Task schema includes `run_in_background`, set it false. Then wait for Task to return. Do not start Judge or more Maker work until the Finding Ledger is back. Prompt: diff file path, name-only list, Goal Contract path, verification log path, conventions.md path, and "at most 8 tool calls; read the diff file; skip css and generated files". Report Spec and Standards axes separately.
49
- 7. Stage 7: Task `subagent_type: judge` the same way (wait, no background). Use `general-purpose` only if `judge` is rejected. Prompt: Goal Contract path, verification evidence path, Finding Ledger, and "at most 4 tool calls; ledger and contract only; skip css; do not re-review the whole diff".
56
+ 6. Stage 6: Build `npx ai-engineering-loop context devil-advocate <files...>` from contract, diff, verification, conventions, and changed source paths. Recheck `budget status`, then Task `subagent_type: devil-advocate`. Use `general-purpose` only if rejected. Wait and immediately `budget record` its actual usage. Normalize its ledger into `RUN_DIR/findings.json` with runId and diffHash, then run `npx ai-engineering-loop gate review`.
57
+ 7. Stage 7: Run `npx ai-engineering-loop escalation --json` before selecting Judge. Resolve the required tier through `/models`; if unavailable, escalate to the user. Build Judge context from only goal-contract.json, verification.json, and findings.json. Recheck `budget status`, spawn Judge as a sibling and wait, then immediately `budget record` actual usage. Normalize its output into `RUN_DIR/verdict.json`, include `modelTier`, then run `npx ai-engineering-loop gate judge`. A gate failure overrides optimistic prose.
50
58
  8. ITERATE with iteration under 3: fix in the parent, re-verify, spawn a **new** Devil's Advocate (do not resume the previous child).
51
- 9. After Judge PASS: run the `after_pass` hook from workflow.md if it is not `none`. Then Stage 8: delivery from `.ai-engineering-loop/adapter.md`. Load shipped spec `adapters/<adapter_type>/` (`standard`, `github`, `gitlab`, `dot`). If adapter.md is missing or the type is unknown, run `generate-adapter` (grill). Do not invent a company pipeline. PR/MR body must include `Steps to Reproduce & Testing (QA)` from the adapter (reproduce + how to test). On a bugfix, do not leave those placeholders empty. When the user confirms a process lesson, append one row to `lessons.md` (not a chat dump). Terms go to glossary.md.
59
+ 9. After Judge PASS: run the `after_pass` hook, then Stage 8 from `.ai-engineering-loop/adapter.md`. PR/MR body includes `Steps to Reproduce & Testing (QA)`; bugfix placeholders must not stay empty. Preview remote payloads; do not expose context packs, usage ledgers, raw logs, or credentials. In ASSISTED, wait for explicit human approval and write `humanApproved: true`; never infer approval. Write `RUN_DIR/delivery.json` with destination and summary, then run `npx ai-engineering-loop gate delivery`. Only this gate completes the run.
52
60
 
53
61
  ## Report header
54
62
 
@@ -28,20 +28,28 @@ Canonical mode ids: `TRUE_INDEPENDENT_AGENT`, `ISOLATED_AGENT_INSTANCE`, `FRESH_
28
28
 
29
29
  Compact map (not a second OS): Specify (stages 0-1), Make (stages 2-4), Review (stages 5-7), Deliver (stage 8). Keep the 8-stage numbers. Do not skip Goal Contract, verification, Devil's Advocate, or Judge.
30
30
 
31
- 1. Stage 0: `npx ai-engineering-loop sync-hosts` then `npx ai-engineering-loop status` (init or refresh if missing or stale). Read `.ai-engineering-loop/glossary.md`. Read `.ai-engineering-loop/lessons.md` and `.ai-engineering-loop/workflow.md` if they exist. Follow workflow.md hooks (`before_grill`, `after_freeze`, `after_pass`). Do not skip Goal Contract, verification, Devil's Advocate, or Judge even if workflow.md asks. If workflow.md is missing, run the default 8-stage loop (no extra hooks). If sync-hosts copied files, tell the user a new session is needed for updated skill text; keep going with this session.
32
- 2. Stage 1: Goal Contract (`core/goal-contract.md`). If the user asks for ideas, list a short menu and wait; do not implement. If the task is ambiguous and the user can answer, grill first (`core/grill-policy.md`): design tree, recommended answers, do not ask look-up facts. Skip grill if the contract is already frozen or the user waived it. On business-logic change, and always on `adapter_type: dot`, grill includes blast radius: lifecycle sketch, four pillars (state, sibling, approval, queues), ASCII picture. Load `task-impact-inquiry` if present. Do not run a second interview. Passing unit tests are not isolation proof. Chat agreement is not freeze: every user-visible decision must be a numbered AC in the Goal Contract file. Freeze before any production edit. Name test seams. Use glossary terms. AC is a failure table (happy, empty/omit, boundary, sibling, error). One red test per AC row. Do not freeze sunny-path-only. If the user prompt has no numbered AC, draft `.ai-engineering-loop/tasks/goal-contract.md` with a failure table, show it, wait for freeze. Do not start Maker.
33
- 3. Stages 2-4: Read `maker_intern` from workflow.md (missing or none = parent is Maker). Do not type a model name. If intern is set, confirm it came from this host's list (`/models` if that command exists). If the list command is missing or the id is not on it, parent is Maker; say intern INVOCATION_UNAVAILABLE. Do not add HTTP clients or API keys. Intern cannot skip verification, Devil's Advocate, or Judge. After intern returns, parent runs Stage 5. Bugs: red repro first (`core/root-cause-analysis.md`). Features: TDD at named seams (`policies/tdd-policy.md`): one red test per AC row. Surgical diff.
34
- 4. Stage 5: run commands from `.ai-engineering-loop/verification.md`. Keep command, exit code, stdout, test counts. Vague "seems green" is invalid. Write `.ai-engineering-loop/tasks/claimed-vs-reality.md` (AC, Claimed, Reality from command log). Do not spawn Devil's Advocate if the file is missing or any Claimed row lacks Reality.
31
+ ## Deterministic artifact gates (mandatory)
32
+
33
+ Run `npx ai-engineering-loop state --json` to get the current runId and use `.ai-engineering-loop/runs/<runId>/` as `RUN_DIR`. Never edit state.json or current.json.
34
+
35
+ Read `core/runtime-safety.md`. Before every model or subagent call, run `budget status --run <runId>` and afterward run `budget record --input <actual> --output <actual> --model <id> --run <runId>` using provider-reported usage. Never invent usage. Missing usage stops UNATTENDED and requires human direction in ASSISTED. `budget pause` is the kill switch.
36
+
37
+ Inherited invariants remain mandatory: read `core/grill-policy.md`, `policies/tdd-policy.md`, and `.ai-engineering-loop/glossary.md`. On `adapter_type: dot`, grill includes blast radius: lifecycle sketch and load `task-impact-inquiry`. If the user prompt has no numbered AC, draft the contract, list a short menu and wait. Chat agreement is not freeze. AC is a failure table. One red test per AC row. Do not freeze sunny-path-only. Write claimed-vs-reality.md. Do not spawn Devil's Advocate if the file is missing or any Claimed row lacks Reality. Report Spec and Standards separately. DA prompt includes "at most 8 tool calls; read the diff file; skip css and generated files". Judge prompt includes "at most 4 tool calls; ledger and contract only; skip css; do not re-review the whole diff". `generate-workflow` still reads workflow.md and lessons.md; missing maker_intern means INVOCATION_UNAVAILABLE. Do not type a model name.
38
+
39
+ 1. Stage 0: sync hosts, check status, then run `npx ai-engineering-loop run [--mode report-only|assisted|unattended]` to create or resume the ledger. Run `state --json` and `budget status`. Read glossary, lessons, and workflow overlays. Required gates cannot be skipped.
40
+ 2. Stage 1: keep the human Goal Contract Markdown and write `RUN_DIR/goal-contract.json` with schemaVersion, runId, objective, numbered AC, evidenceRequired, and non-empty failureCases. Grill ambiguous decisions and business blast radius first. Run `npx ai-engineering-loop gate goal`; do not edit production code unless it passes.
41
+ 3. After Goal freeze, branch on mode. REPORT_ONLY: never edit repository files; write `RUN_DIR/report.json`, run `gate report`, and stop. ASSISTED: use the normal Maker. UNATTENDED: run `sandbox create --run <runId>`, run Maker only in the returned worktreePath, never edit the parent checkout, then run `sandbox capture --run <runId>`. Resolve maker_intern only through the host catalog; otherwise parent is Maker. Intern cannot skip gates. Use red repro or TDD. ASSISTED writes `RUN_DIR/diff.patch`; UNATTENDED capture writes it. Then run `npx ai-engineering-loop gate maker`.
42
+ 4. Stage 5: write `RUN_DIR/verification.json` with current git revision, gated diff hash, command, execution identity, times, exit code, stdout, timeout status, and test counts. Keep claimed-vs-reality.md. Run `npx ai-engineering-loop gate verification`; do not spawn DA unless it passes.
35
43
  5. Write artifacts, then spawn. Before Devil's Advocate:
36
44
  - Write `git diff` to a file (for example `.ai-engineering-loop/tasks/current.diff`).
37
45
  - Write changed paths (`git diff --name-only`) into the child prompt as a short list.
38
46
  - Put those paths in the child prompt. Do not paste Maker rationale.
39
47
  - Write `.ai-engineering-loop/tasks/claimed-vs-reality.md` if missing. Do not spawn DA without it.
40
48
  - If stopping mid-loop, write `.ai-engineering-loop/tasks/handoff.md` (`core/handoff-policy.md`).
41
- 6. Stage 6: spawn `devil-advocate`. Wait. Prompt: diff file path, name-only list, Goal Contract path, verification log path, conventions.md path, and "at most 8 tool calls; read the diff file; skip css and generated files". Report Spec and Standards axes separately.
42
- 7. Stage 7: spawn `judge` the same way (wait, no background). Prompt: Goal Contract path, verification evidence path, Finding Ledger, and "at most 4 tool calls; ledger and contract only; skip css; do not re-review the whole diff".
49
+ 6. Stage 6: Build `npx ai-engineering-loop context devil-advocate <files...>`, check `budget status`, spawn DA with the pack path, wait, and `budget record` actual usage. Normalize its output into `RUN_DIR/findings.json` with runId and diffHash, then run `npx ai-engineering-loop gate review`.
50
+ 7. Stage 7: Run `npx ai-engineering-loop escalation --json`, resolve the required tier through the host catalog, or escalate to the user. Build Judge context from goal-contract.json, verification.json, and findings.json only. Check `budget status`, spawn Judge, wait, and `budget record` actual usage. Write `RUN_DIR/verdict.json` with modelTier, then run `npx ai-engineering-loop gate judge`.
43
51
  8. ITERATE with iteration under 3: fix in the parent, re-verify, spawn a **new** Devil's Advocate.
44
- 9. After Judge PASS: run the `after_pass` hook from workflow.md if it is not `none`. Then Stage 8: delivery from `.ai-engineering-loop/adapter.md`. Load shipped spec `adapters/<adapter_type>/` (`standard`, `github`, `gitlab`, `dot`). If adapter.md is missing or the type is unknown, run `generate-adapter` (grill). Do not invent a company pipeline. PR/MR body must include `Steps to Reproduce & Testing (QA)` from the adapter (reproduce + how to test). On a bugfix, do not leave those placeholders empty. When the user confirms a process lesson, append one row to `lessons.md` (not a chat dump). Terms go to glossary.md.
52
+ 9. After Judge PASS: run after_pass, then the configured Stage 8 adapter. PR/MR body includes `Steps to Reproduce & Testing (QA)`; bugfix placeholders must not stay empty. Preview remote payloads and never include context packs, usage ledgers, raw logs, or credentials. In ASSISTED, wait for explicit human approval and put `humanApproved: true` in `RUN_DIR/delivery.json`; never infer it. Then run `npx ai-engineering-loop gate delivery`.
45
53
 
46
54
  ## Report header
47
55
 
@@ -46,20 +46,28 @@ Run `npx ai-engineering-loop <command>` in the target repo. Do not commit unless
46
46
 
47
47
  Compact map (not a second OS): Specify (stages 0-1), Make (stages 2-4), Review (stages 5-7), Deliver (stage 8). Keep the 8-stage numbers. Do not skip Goal Contract, verification, Devil's Advocate, or Judge.
48
48
 
49
- 1. Stage 0: `npx ai-engineering-loop sync-hosts` then `npx ai-engineering-loop status` (init/refresh if missing or stale). Read `.ai-engineering-loop/glossary.md`. Read `.ai-engineering-loop/lessons.md` and `.ai-engineering-loop/workflow.md` if they exist. Follow workflow.md hooks (`before_grill`, `after_freeze`, `after_pass`). Do not skip Goal Contract, verification, Devil's Advocate, or Judge even if workflow.md asks. If workflow.md is missing, run the default 8-stage loop (no extra hooks). If sync-hosts copied files, tell the user a new session is needed for updated skill text; keep going with this session.
50
- 2. Stage 1: Goal Contract (`core/goal-contract.md`). If the user asks for ideas, list a short menu and wait; do not implement. If the task is ambiguous and the user can answer, grill first (`core/grill-policy.md`): design tree, recommended answers, do not ask look-up facts. Skip grill if the contract is already frozen or the user waived it. On business-logic change, and always on `adapter_type: dot`, grill includes blast radius: lifecycle sketch, four pillars (state, sibling, approval, queues), ASCII picture. Load `task-impact-inquiry` if present. Do not run a second interview. Passing unit tests are not isolation proof. Chat agreement is not freeze: every user-visible decision must be a numbered AC in the Goal Contract file. Freeze before any production edit. Name test seams. Use glossary terms. AC is a failure table (happy, empty/omit, boundary, sibling, error). One red test per AC row. Do not freeze sunny-path-only. If the user prompt has no numbered AC, draft `.ai-engineering-loop/tasks/goal-contract.md` with a failure table, show it, wait for freeze. Do not start Maker.
51
- 3. Stages 2–4: Read `maker_intern` from workflow.md (missing or none = parent is Maker). Do not type a model name. If a label is set, run `grok models` and confirm the id is in the coding list (not grok-imagine-*). If missing, parent is Maker; say intern INVOCATION_UNAVAILABLE. Otherwise write `.ai-engineering-loop/tasks/maker-brief.md` and spawn that intern as a sibling (`background: false`, wait). Pass a model key only if that key exists on the spawn schema. Do not invent keys, HTTP clients, or custom endpoints. Intern must not spawn DA or Judge (depth 1). After intern returns, parent runs Stage 5 then DA/Judge. Intern cannot skip verification, Devil's Advocate, or Judge. Bugs: red repro first (`core/root-cause-analysis.md`). Features: TDD at named seams (`policies/tdd-policy.md`): one red test per AC row. Surgical diff.
52
- 4. Stage 5: run verification from `.ai-engineering-loop/verification.md`. Keep command, exit code, stdout, test counts. Vague "seems green" is invalid. Write `.ai-engineering-loop/tasks/claimed-vs-reality.md` (AC, Claimed, Reality from command log). Do not spawn Devil's Advocate if the file is missing or any Claimed row lacks Reality.
49
+ ## Deterministic artifact gates (mandatory)
50
+
51
+ Run `npx ai-engineering-loop state --json` to get the current `runId` and use `.ai-engineering-loop/runs/<runId>/` as `RUN_DIR`. Never edit `state.json` or `current.json`.
52
+
53
+ Read `core/runtime-safety.md`. Before every model or subagent call, run `budget status --run <runId>`; immediately afterward run `budget record --input <actual> --output <actual> --model <id> --run <runId>` from provider-reported metadata. Never invent usage. Missing usage stops UNATTENDED and requires human direction in ASSISTED. `budget pause` is the kill switch.
54
+
55
+ Inherited invariants remain mandatory: read `core/grill-policy.md`, `policies/tdd-policy.md`, and `.ai-engineering-loop/glossary.md`. On `adapter_type: dot`, grill includes blast radius: lifecycle sketch and load `task-impact-inquiry`. If the user prompt has no numbered AC, draft the contract, list a short menu and wait. Chat agreement is not freeze. AC is a failure table. One red test per AC row. Do not freeze sunny-path-only. Write claimed-vs-reality.md. Do not spawn Devil's Advocate if the file is missing or any Claimed row lacks Reality. Report Spec and Standards separately. DA prompt includes "at most 8 tool calls; read the diff file; skip css and generated files". Judge prompt includes "at most 4 tool calls; ledger and contract only; skip css; do not re-review the whole diff". `generate-workflow` still reads workflow.md and lessons.md; missing maker_intern means INVOCATION_UNAVAILABLE. Do not type a model name.
56
+
57
+ 1. Stage 0: `npx ai-engineering-loop sync-hosts`, `status`, then `run [--mode report-only|assisted|unattended]` to create or resume the ledger. Run `state --json` and `budget status`. Read glossary, lessons, and workflow overlays. Required gates cannot be skipped.
58
+ 2. Stage 1: keep the human Goal Contract Markdown and write `RUN_DIR/goal-contract.json` with schemaVersion, runId, objective, numbered AC, evidenceRequired, and non-empty failureCases. Grill ambiguous decisions and business blast radius first. Run `npx ai-engineering-loop gate goal`; do not edit production code unless it passes.
59
+ 3. After Goal freeze, branch on mode. REPORT_ONLY: never edit repository files; write `RUN_DIR/report.json`, run `gate report`, and stop. ASSISTED: use the normal Maker. UNATTENDED: run `sandbox create --run <runId>`, run Maker only in the returned worktreePath, never edit the parent checkout, then run `sandbox capture --run <runId>`. Resolve `maker_intern` only through `grok models`; unknown means parent Maker. Intern cannot skip gates or spawn DA/Judge. Use red repro or TDD. ASSISTED writes `RUN_DIR/diff.patch`; UNATTENDED capture writes it. Then run `npx ai-engineering-loop gate maker`.
60
+ 4. Stage 5: run verification and write `RUN_DIR/verification.json` with current git revision, gated diff hash, command, execution identity, times, exit code, stdout, timeout status, and test counts. Keep claimed-vs-reality.md. Run `npx ai-engineering-loop gate verification`; do not spawn DA unless it passes.
53
61
  5. Write artifacts to disk so children do not need parent chat:
54
62
  - Goal Contract path
55
63
  - `git diff` written to `.ai-engineering-loop/tasks/current.diff`
56
64
  - verification log (file)
57
65
  - `.ai-engineering-loop/tasks/claimed-vs-reality.md` (do not spawn DA without it)
58
66
  - If stopping mid-loop, `.ai-engineering-loop/tasks/handoff.md` (`core/handoff-policy.md`)
59
- 6. Stage 6: `spawn_subagent` Devil's Advocate. `background: false`. `capability_mode: "execute"`. Do **not** pass `resume_from`. Wait for the child. Prompt: diff file path, name-only file list, Goal Contract path, verification log path, conventions.md path, plus "at most 8 tool calls; read the diff file; skip css and generated files". Report Spec and Standards axes separately.
60
- 7. Stage 7: `spawn_subagent` Judge the same way (`background: false`, wait). Use `general-purpose` only if `judge` is rejected. Prompt: Goal Contract path, verification evidence path, Finding Ledger, and "at most 4 tool calls; ledger and contract only; skip css; do not re-review the whole diff".
67
+ 6. Stage 6: Build `npx ai-engineering-loop context devil-advocate <files...>`. Check `budget status`, spawn Devil's Advocate with the pack path, `background: false`, `capability_mode: "execute"`, and no `resume_from`; wait and `budget record` actual usage. Normalize its output into `RUN_DIR/findings.json` with runId and diffHash, then run `npx ai-engineering-loop gate review`.
68
+ 7. Stage 7: Run `npx ai-engineering-loop escalation --json`; resolve `STRONG_MODEL` through `grok models`, or escalate to the user if unavailable. Build Judge context from only goal-contract.json, verification.json, and findings.json. Check `budget status`, spawn a fresh sibling Judge and wait, then `budget record` actual usage. Normalize `RUN_DIR/verdict.json` with modelTier, then run `npx ai-engineering-loop gate judge`.
61
69
  8. If Judge says `ITERATE` and iteration < 3, Maker fixes in the parent, re-verify, spawn a **fresh** DA (new spawn, no resume).
62
- 9. After Judge PASS: run the `after_pass` hook from workflow.md if it is not `none`. Then Stage 8: delivery from `.ai-engineering-loop/adapter.md`. Load shipped spec `adapters/<adapter_type>/` (`standard`, `github`, `gitlab`, `dot`). If adapter.md is missing or the type is unknown, run `generate-adapter` (grill). Do not invent a company pipeline. PR/MR body must include `Steps to Reproduce & Testing (QA)` from the adapter (reproduce + how to test). On a bugfix, do not leave those placeholders empty. When the user confirms a process lesson, append one row to `lessons.md` (not a chat dump). Terms go to glossary.md.
70
+ 9. After Judge PASS: run `after_pass`, then the configured Stage 8 adapter. PR/MR body includes `Steps to Reproduce & Testing (QA)`; bugfix placeholders must not stay empty. Preview remote payloads and never include context packs, usage ledgers, raw logs, or credentials. In ASSISTED, wait for explicit human approval and put `humanApproved: true` in `RUN_DIR/delivery.json`; never infer it. Then run `npx ai-engineering-loop gate delivery`.
63
71
 
64
72
  After a proven Grok DA spawn, the report header must be:
65
73
 
package/README.md CHANGED
@@ -6,11 +6,11 @@
6
6
  [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
7
7
  [![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](https://github.com/egagofur/ai-engineering-loop/pulls)
8
8
  [![AI Engineering](https://img.shields.io/badge/AI-Engineering%20Loop-orange.svg)](https://github.com/egagofur/ai-engineering-loop)
9
- [![Release](https://img.shields.io/badge/release-v1.0.21-purple.svg)](https://github.com/egagofur/ai-engineering-loop/releases)
9
+ [![Release](https://img.shields.io/badge/release-v1.2.1-purple.svg)](https://github.com/egagofur/ai-engineering-loop/releases)
10
10
 
11
11
  **A Reusable, Framework-Agnostic AI Engineering Operating System for Autonomous Coding Agents**
12
12
 
13
- *Featuring living project context, strict verification evidence contracts, 3-stage capability lifecycle registry, and dual-axis Judge evaluation.*
13
+ *Featuring graduated safety modes, token budgets, isolated Maker worktrees, deterministic artifact gates, bounded secret-redacted context, and cheap-model escalation.*
14
14
 
15
15
  [Overview](#overview--philosophy) • [Stage techniques](#stage-techniques) • [Runtime Capability Registry](#runtime-capability-registry--execution-modes) • [Verification Evidence](#verification-evidence-contract) • [CLI Commands](#cli-interface--commands) • [Grok CLI](#grok-cli-integration) • [Claude Code](#claude-code-integration) • [Antigravity](#antigravity-agent-integration) • [Lifecycle](#lifecycle-stages) • [Architecture](#architecture--5-layer-configuration) • [Project Profiles](#project-profiles) • [Repository Structure](#repository-structure) • [Reference Examples](#reference-examples) • [Contributing](#contributing)
16
16
 
@@ -71,6 +71,8 @@ The 8-stage loop stays one OS. These techniques sit **inside** existing stages (
71
71
  | 4–5 | Red-green at named **seams**; failure table (not happy path only); coverage is a map; no grep/tautology | `policies/tdd-policy.md` |
72
72
  | 5 | **Claimed vs Reality** table before DA. Missing file or empty Reality blocks Devil's Advocate. "Seems green" is not Reality. | `core/verification-loop.md` |
73
73
  | 6–7 | Spec vs Standards reported separately. Standards BLOCKER/HIGH iterate only when `hardConvention` is true | `policies/finding-policy.md` |
74
+ | 0–8 | Stateful run ledger and deterministic artifact transitions; model claims cannot advance a stage | `core/artifact-gates.md` |
75
+ | 0–8 | `REPORT_ONLY`, human-approved `ASSISTED`, opt-in `UNATTENDED`; actual token budgets, kill switch, and locked Maker worktrees | `core/runtime-safety.md` |
74
76
  | any | Mid-loop stop writes `.ai-engineering-loop/tasks/handoff.md` | `core/handoff-policy.md` |
75
77
  | any | **Compact map** (host guidance, not a second OS): Specify (stages 0-1), Make (stages 2-4), Review (stages 5-7), Deliver (stage 8). Keep the 8-stage numbers. Do not skip Goal Contract, verification, Devil's Advocate, or Judge. | host skills |
76
78
 
@@ -178,8 +180,44 @@ npx ai-engineering-loop status
178
180
  # Reconcile drifted context against repository non-destructively
179
181
  npx ai-engineering-loop refresh
180
182
 
181
- # Verify context readiness and begin engineering loop
183
+ # Start or resume a stateful engineering run
182
184
  npx ai-engineering-loop run
185
+ npx ai-engineering-loop run --mode report-only "audit this repository"
186
+
187
+ # Inspect the run ledger
188
+ npx ai-engineering-loop state --json
189
+
190
+ # Inspect policy, opt into unattended, and operate the token kill switch
191
+ npx ai-engineering-loop policy show
192
+ npx ai-engineering-loop policy set --allow-unattended true
193
+ npx ai-engineering-loop budget status
194
+ npx ai-engineering-loop budget pause
195
+ npx ai-engineering-loop budget resume
196
+
197
+ # Record actual provider-reported usage after each model response
198
+ npx ai-engineering-loop budget record --input 1200 --output 300 --model provider/model-id
199
+
200
+ # Isolate unattended Maker, then capture only its diff
201
+ npx ai-engineering-loop sandbox create
202
+ npx ai-engineering-loop sandbox capture
203
+
204
+ # Validate artifacts and advance stages
205
+ npx ai-engineering-loop gate goal
206
+ npx ai-engineering-loop gate report
207
+ npx ai-engineering-loop gate maker
208
+ npx ai-engineering-loop gate verification
209
+ npx ai-engineering-loop gate review
210
+ npx ai-engineering-loop escalation --json
211
+ npx ai-engineering-loop gate judge
212
+ npx ai-engineering-loop gate delivery
213
+
214
+ # Build bounded model-visible context
215
+ npx ai-engineering-loop context devil-advocate <files...>
216
+ npx ai-engineering-loop context judge <files...>
217
+
218
+ # Validate installation and the production evaluation catalog
219
+ npx ai-engineering-loop doctor
220
+ npx ai-engineering-loop eval
183
221
 
184
222
  # Copy package skills/agents/commands into ~/.claude ~/.grok ~/.gemini ~/.agents
185
223
  npx ai-engineering-loop sync-hosts
@@ -193,6 +231,8 @@ npx ai-engineering-loop generate-workflow
193
231
  npx ai-engineering-loop generate-workflow --write
194
232
  ```
195
233
 
234
+ Every full run stores private local artifacts under `.ai-engineering-loop/runs/<run-id>/`. Usage, worktrees, locks, context packs, and logs are Git-ignored. Markdown remains the human contract; JSON sidecars follow the shipped `schemas/` and make stage claims deterministic. `ASSISTED` is the default and requires explicit human delivery approval. `UNATTENDED` is disabled until a human enables it. See [`core/runtime-safety.md`](core/runtime-safety.md), [`core/artifact-gates.md`](core/artifact-gates.md), [`SECURITY.md`](SECURITY.md), and [`SUPPORT.md`](SUPPORT.md).
235
+
196
236
  Shipped adapters (Stage 8 only, after Judge PASS): `standard`, `github`, `gitlab`, `dot`. Catalog: `adapters/README.md`. Each team generates its own; do not copy a neighbour's pipeline.
197
237
 
198
238
  `sync-hosts` updates only hosts that already exist on the machine. DOT skills (`dot-dev-skill-router`, `dot-dev-workflow`) are updated only if they are already installed. `task-impact-inquiry`, `generate-adapter`, and `generate-workflow` are upserted onto Claude, Grok, and Gemini. After a copy, start a new session so the host reloads skill text. `/ai-engineering-loop` Stage 0 and `run` call `sync-hosts` so a published package bump reaches global host files without a manual copy.
package/README.npm.md CHANGED
@@ -4,7 +4,7 @@
4
4
  [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
5
5
  [![GitHub](https://img.shields.io/badge/GitHub-Repository-black.svg)](https://github.com/egagofur/ai-engineering-loop)
6
6
 
7
- **A Reusable, Framework-Agnostic AI Engineering Operating System for Autonomous Coding Agents**
7
+ **A Reusable, Framework-Agnostic AI Engineering Operating System for Safe Coding Agents**
8
8
 
9
9
  Move beyond linear AI pipelines and optimistic self-evaluation toward contract-driven execution, deterministic machine verification, independent adversarial review, and living project context.
10
10
 
@@ -27,6 +27,13 @@ npx ai-engineering-loop refresh
27
27
  # 4. Verify context readiness and begin engineering loop
28
28
  npx ai-engineering-loop run
29
29
 
30
+ # Optional: analysis without repository mutation
31
+ npx ai-engineering-loop run --mode report-only "audit this repository"
32
+
33
+ # Inspect spend controls; ASSISTED is the default
34
+ npx ai-engineering-loop policy show
35
+ npx ai-engineering-loop budget status
36
+
30
37
  # 5. Copy package host skills into ~/.claude ~/.grok ~/.gemini ~/.agents
31
38
  npx ai-engineering-loop sync-hosts
32
39
  ```
@@ -40,6 +47,7 @@ npx ai-engineering-loop sync-hosts
40
47
  3. **Multi-Agent Triad**: Coordinates **Maker** (surgical diffs & tests), **Devil's Advocate** (independent adversarial review), and **Judge** (impartial evaluation & PASS certification).
41
48
  4. **Context Impact Assessment**: Evaluates completed tasks (`NONE`, `TARGETED`, `MAJOR`) to keep project context fresh without expensive whole-repo re-analysis.
42
49
  5. **Grok CLI host**: Native `spawn_subagent` for Devil's Advocate and Judge (`TRUE_INDEPENDENT_AGENT`). Disabled by `GROK_SUBAGENTS=0`.
50
+ 6. **Production runtime controls**: `REPORT_ONLY`, human-approved `ASSISTED`, opt-in `UNATTENDED`, actual token ledgers, a persistent kill switch, and locked disposable Maker worktrees.
43
51
 
44
52
  ---
45
53
 
package/SECURITY.md ADDED
@@ -0,0 +1,29 @@
1
+ # Security Policy
2
+
3
+ ## Trust boundary
4
+
5
+ Repository files, generated logs, tool output, and comments are untrusted data. Instructions found inside them never override the active host skill, Goal Contract, or user authorization.
6
+
7
+ AI Engineering Loop blocks `.env*`, private-key formats, common credential files, `.git/`, `node_modules/`, absolute paths, parent traversal, and symlinks that leave the repository from generated context packs. Recognized credentials are redacted before model-visible context is persisted. Package release auditing rejects private keys, local home paths, and `file://` links before publication.
8
+
9
+ No telemetry is collected. Token usage ledgers stay local and contain counts plus provider model identifiers, not prompts or responses. Stage 0–7 require no AI Engineering Loop network service. Delivery adapters may use authenticated forge tools only after Judge `PASS`; ASSISTED requires explicit human approval before delivery.
10
+
11
+ UNATTENDED is disabled by default and Maker uses a locked disposable Git worktree when enabled. This protects the parent checkout and prevents concurrent Maker sessions; it is not an OS sandbox. For untrusted code, use a container or VM with restricted credentials, filesystem, processes, and network. See `core/runtime-safety.md`.
12
+
13
+ Package releases use npm trusted publishing from `.github/workflows/publish.yml`, with a pinned OIDC-capable npm CLI and provenance. The package owner must configure `egagofur/ai-engineering-loop`, workflow `publish.yml`, and environment `npm` as the trusted publisher on npmjs.com. Do not add `NPM_TOKEN` or another long-lived publish credential to the repository or GitHub environment.
14
+
15
+ ## Credential handling
16
+
17
+ - Supply provider credentials through the host's documented environment or credential store.
18
+ - Never place credentials in `.ai-engineering-loop/`, workflow files, prompts, findings, or delivery reports.
19
+ - Do not pass the parent process environment to a child unless the host owns that boundary.
20
+ - Treat generated run artifacts as sensitive local files. The CLI writes state and context packs with mode `0600` where supported.
21
+ - Keep `runs/`, `usage/`, `worktrees/`, `locks/`, and task logs under the generated `.ai-engineering-loop/.gitignore`.
22
+ - Use `budget pause` to stop new model boundaries if spend, behavior, or credentials are in doubt. Recording an already completed call remains possible while paused.
23
+ - Run a dedicated secret scanner such as `gitleaks` before delivery when available.
24
+
25
+ Pattern-based redaction is defense in depth, not proof that arbitrary secrets are absent.
26
+
27
+ ## Reporting a vulnerability
28
+
29
+ Do not open a public issue containing credentials, exploit details, private source, or model transcripts. Use GitHub private vulnerability reporting for this repository. Include the package version, host, minimal reproduction, and whether any remote payload was sent.
package/SUPPORT.md ADDED
@@ -0,0 +1,28 @@
1
+ # Public Beta Support Matrix
2
+
3
+ AI Engineering Loop is in public-beta readiness. The workflow contract is stable enough for evaluation, but host tool schemas can change independently.
4
+
5
+ | Host | Installation source | Independent reviewer path | Fallback |
6
+ |---|---|---|---|
7
+ | Claude Code / compatible Task hosts | `.claude/` | `Task` or `Agent` siblings | `CONTEXT_ISOLATION_ONLY` |
8
+ | Grok CLI | `.grok/` | `spawn_subagent` siblings | `grok -p` or `CONTEXT_ISOLATION_ONLY` |
9
+ | Gemini / Antigravity skill hosts | `.gemini/` | `invoke_subagent` or `Task` when present | `CONTEXT_ISOLATION_ONLY` |
10
+ | Antigravity workflows | `.agents/` | `invoke_subagent` or `Task` when present | `CONTEXT_ISOLATION_ONLY` |
11
+
12
+ Independent execution is reported as proven only after a child ID and model response are observed without inherited Maker history.
13
+
14
+ ## Runtime
15
+
16
+ - Node.js 18 or newer.
17
+ - Git is recommended for revision and diff evidence.
18
+ - Linux, macOS, and Windows are covered by the Node test matrix.
19
+ - No runtime npm dependencies.
20
+
21
+ Run diagnostics after installation:
22
+
23
+ ```bash
24
+ npx ai-engineering-loop doctor
25
+ npx ai-engineering-loop eval
26
+ ```
27
+
28
+ New hosts need a documented tool schema, context-isolation behavior, secret boundary, independent-execution proof, and conformance tests before being listed as supported.
package/agents/judge.md CHANGED
@@ -29,7 +29,7 @@ flowchart LR
29
29
  ## 2. Core Authority & Rules of Engagement
30
30
 
31
31
  1. **Sole Authority on Completion**:
32
- - Only the Judge Agent can issue a `PASS` verdict certifying that the [Definition of Done](file:///Users/egagofur/Development/work/ai-engineering-loop/core/definition-of-done.md) has been met.
32
+ - Only the Judge Agent can issue a `PASS` verdict certifying that the [Definition of Done](../core/definition-of-done.md) has been met.
33
33
  2. **Neutral Evaluation**:
34
34
  - The Judge holds neither author bias (optimism) nor reviewer bias (pedantry).
35
35
  - The Judge independently verifies reviewer claims against actual repository facts before treating a finding as blocking.
@@ -48,13 +48,13 @@ flowchart LR
48
48
  - *If any deterministic check failed $\rightarrow$ immediately issue `ITERATE` with instructions to fix failing tests/types.*
49
49
 
50
50
  ### Step 2: Goal Contract Compliance Audit
51
- - Cross-reference every Acceptance Criterion (AC-1 through AC-N) from the [Goal Contract](file:///Users/egagofur/Development/work/ai-engineering-loop/core/goal-contract.md).
51
+ - Cross-reference every Acceptance Criterion (AC-1 through AC-N) from the [Goal Contract](../core/goal-contract.md).
52
52
  - Verify that automated tests exist that explicitly exercise and prove each criterion, including non-happy-path rows in the failure table.
53
53
  - A happy-path-only suite while the contract lists empty, boundary, sibling, or error rows is `ITERATE`.
54
54
  - Verify that no out-of-scope files were touched and technical constraints were respected.
55
55
 
56
56
  ### Step 3: Finding Triage & Evidence Verification
57
- - Review all findings produced by the [Devil's Advocate](file:///Users/egagofur/Development/work/ai-engineering-loop/agents/devil-advocate.md).
57
+ - Review all findings produced by the [Devil's Advocate](./devil-advocate.md).
58
58
  - Validate the Author's triage:
59
59
  - If a finding is `VALID` and unresolved $\rightarrow$ cannot issue `PASS`.
60
60
  - If a finding is triaged as `INVALID`, confirm whether the Maker's defense is factually supported by codebase evidence.
@@ -62,13 +62,13 @@ flowchart LR
62
62
 
63
63
  ### Step 4: Iteration & Stagnation Check
64
64
  - Check current iteration count ($K$). If $K \ge \text{MAX\_ITERATIONS}$ and blocking issues remain $\rightarrow$ issue `ESCALATE`.
65
- - Check finding signatures across iterations using the [No-Progress Policy](file:///Users/egagofur/Development/work/ai-engineering-loop/policies/no-progress-policy.md). If progress is stalled $\rightarrow$ issue `ESCALATE`.
65
+ - Check finding signatures across iterations using the [No-Progress Policy](../policies/no-progress-policy.md). If progress is stalled $\rightarrow$ issue `ESCALATE`.
66
66
 
67
67
  ---
68
68
 
69
69
  ## 4. Output Contract
70
70
 
71
- The Judge Agent generates the official **Judge Evaluation Report** conforming to [Judge Policy](file:///Users/egagofur/Development/work/ai-engineering-loop/core/judge-policy.md).
71
+ The Judge Agent generates the official **Judge Evaluation Report** conforming to [Judge Policy](../core/judge-policy.md).
72
72
 
73
73
  The verdict must be one of:
74
74
  - **`PASS`**: All acceptance criteria proven, deterministic checks 100% green, 0 blocking findings, DoD fully satisfied. Safe to hand off to delivery adapters.
package/agents/maker.md CHANGED
@@ -2,7 +2,7 @@
2
2
 
3
3
  ## 1. Role & Identity
4
4
 
5
- The **Maker Agent** is the builder and execution engine of the AI Engineering Loop. It is responsible for translating the [Goal Contract](file:///Users/egagofur/Development/work/ai-engineering-loop/core/goal-contract.md) into clean, minimal, robust, and verifiable software implementations.
5
+ The **Maker Agent** is the builder and execution engine of the AI Engineering Loop. It is responsible for translating the [Goal Contract](../core/goal-contract.md) into clean, minimal, robust, and verifiable software implementations.
6
6
 
7
7
  ```mermaid
8
8
  flowchart LR
@@ -25,12 +25,12 @@ flowchart LR
25
25
  - Maintain strict adherence to existing codebase architecture, design system tokens, naming conventions, and file organization.
26
26
  - Avoid speculative abstractions, unsolicited refactoring, or touching out-of-scope files.
27
27
  - Leave zero dead code, zero stubs, zero empty catch blocks, and zero speculative TODOs.
28
- 3. **Test-Driven Engineering** ([TDD Policy](file:///Users/egagofur/Development/work/ai-engineering-loop/policies/tdd-policy.md)):
28
+ 3. **Test-Driven Engineering** ([TDD Policy](../policies/tdd-policy.md)):
29
29
  - Red before green at the Goal Contract's named seams only.
30
30
  - Tests observe public behavior, use glossary terms, and never assert implementation details.
31
31
  - Cover the AC slice in play: happy path, empty/omit, boundary, sibling/isolation, error/denied. One red test per AC row. Vertical slices, not all-tests-then-all-code.
32
32
  4. **Addressing Reviewer Findings**:
33
- - In subsequent iterations, ingest findings from the [Devil's Advocate](file:///Users/egagofur/Development/work/ai-engineering-loop/agents/devil-advocate.md) and directives from the [Judge](file:///Users/egagofur/Development/work/ai-engineering-loop/agents/judge.md).
33
+ - In subsequent iterations, ingest findings from the [Devil's Advocate](./devil-advocate.md) and directives from the [Judge](./judge.md).
34
34
  - Fix validated issues surgically.
35
35
  - Provide concrete evidence when pushing back against invalid reviewer findings.
36
36
 
@@ -38,7 +38,7 @@ flowchart LR
38
38
 
39
39
  ## 3. Strict Boundary Rules (What the Maker MUST NOT Do)
40
40
 
41
- 1. **CANNOT Declare Completion**: The Maker Agent must NEVER declare the overall task finished or ready to merge. Completion can only be certified by the [Judge Agent](file:///Users/egagofur/Development/work/ai-engineering-loop/agents/judge.md).
41
+ 1. **CANNOT Declare Completion**: The Maker Agent must NEVER declare the overall task finished or ready to merge. Completion can only be certified by the [Judge Agent](./judge.md).
42
42
  2. **CANNOT Alter Acceptance Criteria**: The Maker Agent must never modify the Goal Contract's acceptance criteria to bypass failing tests.
43
43
  3. **CANNOT Suppress Verification Failures**: The Maker Agent is strictly prohibited from adding `@ts-ignore`, `eslint-disable`, mock overrides that mask bugs, or skipping broken tests to achieve green results.
44
44
  4. **CANNOT Make Unsolicited Renovations**: Even if surrounding legacy code is poorly written, the Maker must not refactor untouched modules unless explicitly included in the Goal Contract.
@@ -49,7 +49,7 @@ flowchart LR
49
49
  ## 4. Execution Workflow
50
50
 
51
51
  ### Step 1: Pre-Code Analysis
52
- - Follow [Root Cause Analysis](file:///Users/egagofur/Development/work/ai-engineering-loop/core/root-cause-analysis.md). For bugs: red repro before a production edit.
52
+ - Follow [Root Cause Analysis](../core/root-cause-analysis.md). For bugs: red repro before a production edit.
53
53
  - Map the data flow across layers.
54
54
  - Confirm seams from the Goal Contract. Use `.ai-engineering-loop/glossary.md` for names.
55
55
  - Check git history for recent changes to the affected files.
@@ -66,4 +66,4 @@ flowchart LR
66
66
  - Run build if applicable.
67
67
 
68
68
  ### Step 4: Submit to Adversarial Gate
69
- - Once all deterministic checks pass 100%, freeze the diff and hand off to the [Devil's Advocate Agent](file:///Users/egagofur/Development/work/ai-engineering-loop/agents/devil-advocate.md).
69
+ - Once all deterministic checks pass 100%, freeze the diff and hand off to the [Devil's Advocate Agent](./devil-advocate.md).