ai-engineering-loop 1.0.21 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/workflows/ai-engineering-loop.md +7 -3
- package/.claude/skills/ai-engineering-loop/SKILL.md +13 -7
- package/.gemini/skills/ai-engineering-loop/SKILL.md +13 -7
- package/.grok/skills/ai-engineering-loop/SKILL.md +13 -7
- package/README.md +26 -3
- package/SECURITY.md +23 -0
- package/SUPPORT.md +28 -0
- package/bin/ai-engineering-loop.js +255 -3
- package/core/artifact-gates.md +49 -0
- package/evals/README.md +35 -0
- package/evals/cases/01-stale-verification.json +17 -0
- package/evals/cases/02-vague-verification.json +16 -0
- package/evals/cases/03-missing-ac-evidence.json +17 -0
- package/evals/cases/04-hallucinated-citation.json +17 -0
- package/evals/cases/05-prompt-injection.json +17 -0
- package/evals/cases/06-source-secret.json +16 -0
- package/evals/cases/07-log-secret.json +16 -0
- package/evals/cases/08-symlink-escape.json +16 -0
- package/evals/cases/09-authorization-bypass.json +16 -0
- package/evals/cases/10-payment-idempotency.json +16 -0
- package/evals/cases/11-concurrent-update.json +16 -0
- package/evals/cases/12-swallowed-error.json +16 -0
- package/evals/cases/13-happy-path-only.json +17 -0
- package/evals/cases/14-invalid-finding-axis.json +16 -0
- package/evals/cases/15-blocker-without-alternative.json +16 -0
- package/evals/cases/16-unsupported-pass.json +17 -0
- package/evals/cases/17-soft-standard.json +16 -0
- package/evals/cases/18-iteration-ceiling.json +17 -0
- package/evals/cases/19-safe-documentation-change.json +17 -0
- package/evals/cases/20-valid-boundary-feature.json +18 -0
- package/lib/doctor.js +98 -0
- package/lib/escalation.js +192 -0
- package/lib/evaluation.js +137 -0
- package/lib/gates.js +315 -0
- package/lib/orchestration.js +5 -1
- package/lib/run-state.js +221 -0
- package/lib/safe-context.js +191 -0
- package/lib/sync-hosts.js +6 -5
- package/package.json +12 -9
- package/policies/model-escalation.md +41 -0
- package/schemas/delivery.schema.json +14 -0
- package/schemas/finding-ledger.schema.json +27 -0
- package/schemas/goal-contract.schema.json +32 -0
- package/schemas/verdict.schema.json +17 -0
- package/schemas/verification.schema.json +32 -0
- package/docs/antigravity-feasibility.md +0 -92
- package/docs/claude-code-feasibility.md +0 -80
- package/docs/grok-cli-feasibility.md +0 -99
- package/docs/migration-plan.md +0 -70
- package/examples/backend-api/payment-idempotency/README.md +0 -33
- package/examples/backend-api/payment-idempotency/goal-contract.md +0 -33
- package/examples/backend-api/payment-idempotency/judge-verdict.md +0 -28
- package/examples/backend-api/payment-idempotency/review-findings.md +0 -50
- package/examples/dot/status-display/README.md +0 -23
- package/examples/dot/status-display/delivery-report.md +0 -41
- package/examples/dot/status-display/goal-contract.md +0 -46
- package/examples/dot/status-display/judge-verdict.md +0 -44
- package/examples/dot/status-display/review-findings.md +0 -63
- package/examples/initialization/README.md +0 -19
- package/examples/initialization/discovery-trace.md +0 -63
- package/examples/initialization/generated-context.md +0 -110
- package/examples/mobile-app/offline-sync-queue/README.md +0 -33
- package/examples/mobile-app/offline-sync-queue/goal-contract.md +0 -32
- package/examples/mobile-app/offline-sync-queue/judge-verdict.md +0 -27
- package/examples/mobile-app/offline-sync-queue/review-findings.md +0 -30
- package/scripts/init.sh +0 -106
- package/templates/repo-config/adapter.md +0 -11
- package/templates/repo-config/adr-readme.md +0 -33
- package/templates/repo-config/architecture.md +0 -15
- package/templates/repo-config/claude-permissions.json +0 -10
- package/templates/repo-config/config.md +0 -11
- package/templates/repo-config/conventions.md +0 -16
- package/templates/repo-config/glossary.md +0 -18
- package/templates/repo-config/lessons.md +0 -8
- package/templates/repo-config/verification.md +0 -13
- package/templates/repo-config/workflow.md +0 -25
- package/tests/capability-selection.test.js +0 -207
- package/tests/claimed-vs-reality.test.js +0 -27
- package/tests/dot-branch-policy.test.js +0 -65
- package/tests/generate-adapter.test.js +0 -80
- package/tests/generate-workflow.test.js +0 -180
- package/tests/grok-runtime.test.js +0 -201
- package/tests/intern-catalog.test.js +0 -72
- package/tests/living-context.test.js +0 -61
- package/tests/no-company-leak.test.js +0 -82
- package/tests/orchestration.test.js +0 -368
- package/tests/qa-delivery-template.test.js +0 -90
- package/tests/skill-host-compat.test.js +0 -406
- package/tests/sync-hosts.test.js +0 -168
|
@@ -7,14 +7,18 @@ Follow `policies/review-budget.md`. Parent is Maker plus orchestrator. Do not us
|
|
|
7
7
|
|
|
8
8
|
Compact map (not a second OS): Specify (stages 0-1), Make (stages 2-4), Review (stages 5-7), Deliver (stage 8). Keep the 8-stage numbers. Do not skip Goal Contract, verification, Devil's Advocate, or Judge.
|
|
9
9
|
|
|
10
|
+
Deterministic gates are mandatory. Stage 0 runs sync-hosts, status, then `npx ai-engineering-loop run`; `state --json` gives runId and `.ai-engineering-loop/runs/<runId>/` (`RUN_DIR`). Never edit state.json. Stage 1 keeps Goal Contract Markdown, writes goal-contract.json with failure cases, then runs `gate goal`. Maker writes RUN_DIR/diff.patch then `gate maker`. Stage 5 writes current verification.json then `gate verification`.
|
|
11
|
+
|
|
12
|
+
Inherited invariants: read `core/grill-policy.md`, `policies/tdd-policy.md`, and `.ai-engineering-loop/glossary.md`. On `adapter_type: dot`, grill includes blast radius: lifecycle sketch and load `task-impact-inquiry`. If the user prompt has no numbered AC, draft the contract, list a short menu and wait. Chat agreement is not freeze. AC is a failure table. One red test per AC row. Do not freeze sunny-path-only. Write claimed-vs-reality.md. Do not spawn Devil's Advocate if the file is missing or any Claimed row lacks Reality. Report Spec and Standards separately. DA prompt: "at most 8 tool calls; read the diff file; skip css and generated files". Judge prompt: "at most 4 tool calls; ledger and contract only; skip css; do not re-review the whole diff". Never use `browser_subagent` as reviewer. `generate-workflow` reads workflow.md and lessons.md; missing maker_intern means INVOCATION_UNAVAILABLE. Do not type a model name.
|
|
13
|
+
|
|
10
14
|
Stage 1: if the user asks for ideas, list a short menu and wait; do not implement. Grill if the task is ambiguous (`core/grill-policy.md`), then freeze the Goal Contract file. Chat agreement is not freeze. If the user prompt has no numbered AC, draft `.ai-engineering-loop/tasks/goal-contract.md` with a failure table, show it, wait for freeze. Do not start Maker. On business-logic change, and always on `adapter_type: dot`, grill includes blast radius: lifecycle sketch, four pillars (state, sibling, approval, queues), ASCII picture. Load `task-impact-inquiry` if present. Do not run a second interview. Passing unit tests are not isolation proof. Use `.ai-engineering-loop/glossary.md`. Name test seams. AC is a failure table (happy, empty/omit, boundary, sibling, error). One red test per AC row. Do not freeze sunny-path-only. Stages 2-4: Read `maker_intern` from workflow.md (missing or none = parent is Maker). Do not type a model name. If intern is set, confirm it came from this host's list (`/models` if that command exists). If the list command is missing or the id is not on it, parent is Maker; say intern INVOCATION_UNAVAILABLE. Intern cannot skip verification, Devil's Advocate, or Judge. Maker: TDD at those seams (`policies/tdd-policy.md`): one red test per AC row. Bugs: red repro first (`core/root-cause-analysis.md`). Mid-loop stop: `core/handoff-policy.md`.
|
|
11
15
|
|
|
12
16
|
If `invoke_subagent` (or Task) exists, spawn `devil-advocate` then `judge` as siblings. Wait for each child. Do not run them in the background. Use `general-purpose` only if the named type is rejected.
|
|
13
17
|
|
|
14
18
|
5. Before Devil's Advocate: write `git diff` to `.ai-engineering-loop/tasks/current.diff` and put `git diff --name-only` in the child prompt. Do not paste Maker rationale. Write `.ai-engineering-loop/tasks/claimed-vs-reality.md` (AC, Claimed, Reality from command log). Do not spawn Devil's Advocate if the file is missing or any Claimed row lacks Reality.
|
|
15
|
-
6.
|
|
16
|
-
7.
|
|
19
|
+
6. Build `context devil-advocate`, spawn DA with the pack path, normalize RUN_DIR/findings.json with runId and diffHash, then `gate review`.
|
|
20
|
+
7. Run `escalation --json`, resolve the required tier from the host catalog, build Judge context from only goal-contract.json, verification.json, and findings.json, spawn Judge, write verdict.json with modelTier, then `gate judge`.
|
|
17
21
|
|
|
18
22
|
If no subagent tool exists, run the same budgets in this session as CONTEXT_ISOLATION_ONLY. Do not claim independent agent execution.
|
|
19
23
|
|
|
20
|
-
Init/status/refresh/sync-hosts/generate-adapter/generate-workflow: `npx ai-engineering-loop <command>`. generate-workflow grills Q1-Q6. Q6 Maker intern: pick from the host catalog
|
|
24
|
+
Init/status/refresh/sync-hosts/generate-adapter/generate-workflow: `npx ai-engineering-loop <command>`. generate-workflow grills Q1-Q6. Q6 Maker intern: pick from the host catalog; do not type a model name. After Judge PASS, PR/MR body includes `Steps to Reproduce & Testing (QA)` and bugfix placeholders must not stay empty. Preview Stage 8 remote payloads and never include context packs, raw logs, or credentials. Write RUN_DIR/delivery.json and run `gate delivery`; only that gate completes the run. Confirmed process lessons go to lessons.md, not a chat dump.
|
|
@@ -35,20 +35,26 @@ Compact map (not a second OS): Specify (stages 0-1), Make (stages 2-4), Review (
|
|
|
35
35
|
|
|
36
36
|
Parent session is Maker plus orchestrator. Spawn Devil's Advocate and Judge as **siblings**, not nested.
|
|
37
37
|
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
38
|
+
## Deterministic artifact gates (mandatory)
|
|
39
|
+
|
|
40
|
+
Run `npx ai-engineering-loop state --json` to get the current `runId` and use `.ai-engineering-loop/runs/<runId>/` as `RUN_DIR`. Never edit `state.json` or `current.json`.
|
|
41
|
+
|
|
42
|
+
Inherited invariants remain mandatory: read `core/grill-policy.md`, `policies/tdd-policy.md`, and `.ai-engineering-loop/glossary.md`. On `adapter_type: dot`, grill includes blast radius: lifecycle sketch and load `task-impact-inquiry`. If the user prompt has no numbered AC, draft the contract, list a short menu and wait. Chat agreement is not freeze. AC is a failure table. One red test per AC row. Do not freeze sunny-path-only. Write claimed-vs-reality.md. Do not spawn Devil's Advocate if the file is missing or any Claimed row lacks Reality. Report Spec and Standards separately. DA prompt includes "at most 8 tool calls; read the diff file; skip css and generated files". Judge prompt includes "at most 4 tool calls; ledger and contract only; skip css; do not re-review the whole diff".
|
|
43
|
+
|
|
44
|
+
1. Stage 0: `npx ai-engineering-loop sync-hosts`, `status`, then `run` to create or resume the ledger. Read `.ai-engineering-loop/glossary.md`. Read `.ai-engineering-loop/lessons.md` and `.ai-engineering-loop/workflow.md` if they exist. Follow workflow.md hooks (`before_grill`, `after_freeze`, `after_pass`). Do not skip Goal Contract, verification, Devil's Advocate, or Judge even if workflow.md asks. If workflow.md is missing, run the default 8-stage loop. If sync-hosts copied files, tell the user a new session is needed; keep going with this session.
|
|
45
|
+
2. Stage 1: Goal Contract (`core/goal-contract.md`). If the user asks for ideas, list a short menu and wait; do not implement. If the task is ambiguous and the user can answer, grill first (`core/grill-policy.md`): design tree, recommended answers, do not ask look-up facts. On business-logic change, and always on `adapter_type: dot`, include blast radius. Chat agreement is not freeze. Keep the human Markdown contract and write `RUN_DIR/goal-contract.json` with `schemaVersion`, `runId`, objective, numbered AC, evidenceRequired, and non-empty failureCases. Run `npx ai-engineering-loop gate goal`; do not edit production code unless it passes.
|
|
46
|
+
3. Stages 2-4: Read `maker_intern` from workflow.md (missing or none = parent is Maker). Do not type a model name. Confirm a configured intern through `/models` when available; otherwise parent is Maker and report INVOCATION_UNAVAILABLE. Intern cannot skip later gates. Bugs: red repro first. Features: one red test per AC row. Write the exact review diff to `RUN_DIR/diff.patch`, then run `npx ai-engineering-loop gate maker`.
|
|
47
|
+
4. Stage 5: run commands from `.ai-engineering-loop/verification.md`. Write `RUN_DIR/verification.json` with current git revision, gated diff hash, command, execution identity, times, exit code, stdout, timeout status, and test counts. Vague "seems green" is invalid. Keep `.ai-engineering-loop/tasks/claimed-vs-reality.md`. Run `npx ai-engineering-loop gate verification`; do not spawn Devil's Advocate unless it passes.
|
|
42
48
|
5. Write artifacts, then spawn. Before Devil's Advocate:
|
|
43
49
|
- Write `git diff` to a file (for example `.ai-engineering-loop/tasks/current.diff`).
|
|
44
50
|
- Write changed paths (`git diff --name-only`) into the Task prompt as a short list.
|
|
45
51
|
- Put those paths in the child prompt. Do not paste Maker rationale.
|
|
46
52
|
- Write `.ai-engineering-loop/tasks/claimed-vs-reality.md` if missing. Do not spawn DA without it.
|
|
47
53
|
- If stopping mid-loop, write `.ai-engineering-loop/tasks/handoff.md` (`core/handoff-policy.md`).
|
|
48
|
-
6. Stage 6:
|
|
49
|
-
7. Stage 7:
|
|
54
|
+
6. Stage 6: Build `npx ai-engineering-loop context devil-advocate <files...>` from contract, diff, verification, conventions, and changed source paths. Task `subagent_type: devil-advocate`. Use `general-purpose` only if rejected. Wait. Normalize its ledger into `RUN_DIR/findings.json` with runId and diffHash, then run `npx ai-engineering-loop gate review`.
|
|
55
|
+
7. Stage 7: Run `npx ai-engineering-loop escalation --json` before selecting Judge. Resolve the required tier through `/models`; if unavailable, escalate to the user. Build Judge context from only goal-contract.json, verification.json, and findings.json. Spawn Judge as a sibling and wait. Normalize its output into `RUN_DIR/verdict.json`, include `modelTier`, then run `npx ai-engineering-loop gate judge`. A gate failure overrides optimistic prose.
|
|
50
56
|
8. ITERATE with iteration under 3: fix in the parent, re-verify, spawn a **new** Devil's Advocate (do not resume the previous child).
|
|
51
|
-
9. After Judge PASS: run the `after_pass` hook
|
|
57
|
+
9. After Judge PASS: run the `after_pass` hook, then Stage 8 from `.ai-engineering-loop/adapter.md`. PR/MR body includes `Steps to Reproduce & Testing (QA)`; bugfix placeholders must not stay empty. Preview remote payloads; do not expose context packs, raw logs, or credentials. Write `RUN_DIR/delivery.json` with destination and summary, then run `npx ai-engineering-loop gate delivery`. Only this gate completes the run.
|
|
52
58
|
|
|
53
59
|
## Report header
|
|
54
60
|
|
|
@@ -28,20 +28,26 @@ Canonical mode ids: `TRUE_INDEPENDENT_AGENT`, `ISOLATED_AGENT_INSTANCE`, `FRESH_
|
|
|
28
28
|
|
|
29
29
|
Compact map (not a second OS): Specify (stages 0-1), Make (stages 2-4), Review (stages 5-7), Deliver (stage 8). Keep the 8-stage numbers. Do not skip Goal Contract, verification, Devil's Advocate, or Judge.
|
|
30
30
|
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
31
|
+
## Deterministic artifact gates (mandatory)
|
|
32
|
+
|
|
33
|
+
Run `npx ai-engineering-loop state --json` to get the current runId and use `.ai-engineering-loop/runs/<runId>/` as `RUN_DIR`. Never edit state.json or current.json.
|
|
34
|
+
|
|
35
|
+
Inherited invariants remain mandatory: read `core/grill-policy.md`, `policies/tdd-policy.md`, and `.ai-engineering-loop/glossary.md`. On `adapter_type: dot`, grill includes blast radius: lifecycle sketch and load `task-impact-inquiry`. If the user prompt has no numbered AC, draft the contract, list a short menu and wait. Chat agreement is not freeze. AC is a failure table. One red test per AC row. Do not freeze sunny-path-only. Write claimed-vs-reality.md. Do not spawn Devil's Advocate if the file is missing or any Claimed row lacks Reality. Report Spec and Standards separately. DA prompt includes "at most 8 tool calls; read the diff file; skip css and generated files". Judge prompt includes "at most 4 tool calls; ledger and contract only; skip css; do not re-review the whole diff". `generate-workflow` still reads workflow.md and lessons.md; missing maker_intern means INVOCATION_UNAVAILABLE. Do not type a model name.
|
|
36
|
+
|
|
37
|
+
1. Stage 0: sync hosts, check status, then run `npx ai-engineering-loop run` to create or resume the ledger. Read glossary, lessons, and workflow overlays. Required stages cannot be skipped.
|
|
38
|
+
2. Stage 1: keep the human Goal Contract Markdown and write `RUN_DIR/goal-contract.json` with schemaVersion, runId, objective, numbered AC, evidenceRequired, and non-empty failureCases. Grill ambiguous decisions and business blast radius first. Run `npx ai-engineering-loop gate goal`; do not edit production code unless it passes.
|
|
39
|
+
3. Stages 2–4: Resolve maker_intern only through the host catalog; otherwise parent is Maker. Intern cannot skip gates. Use red repro or TDD, write `RUN_DIR/diff.patch`, then run `npx ai-engineering-loop gate maker`.
|
|
40
|
+
4. Stage 5: write `RUN_DIR/verification.json` with current git revision, gated diff hash, command, execution identity, times, exit code, stdout, timeout status, and test counts. Keep claimed-vs-reality.md. Run `npx ai-engineering-loop gate verification`; do not spawn DA unless it passes.
|
|
35
41
|
5. Write artifacts, then spawn. Before Devil's Advocate:
|
|
36
42
|
- Write `git diff` to a file (for example `.ai-engineering-loop/tasks/current.diff`).
|
|
37
43
|
- Write changed paths (`git diff --name-only`) into the child prompt as a short list.
|
|
38
44
|
- Put those paths in the child prompt. Do not paste Maker rationale.
|
|
39
45
|
- Write `.ai-engineering-loop/tasks/claimed-vs-reality.md` if missing. Do not spawn DA without it.
|
|
40
46
|
- If stopping mid-loop, write `.ai-engineering-loop/tasks/handoff.md` (`core/handoff-policy.md`).
|
|
41
|
-
6. Stage 6:
|
|
42
|
-
7. Stage 7:
|
|
47
|
+
6. Stage 6: Build `npx ai-engineering-loop context devil-advocate <files...>`, spawn DA with the pack path, and wait. Normalize its output into `RUN_DIR/findings.json` with runId and diffHash, then run `npx ai-engineering-loop gate review`.
|
|
48
|
+
7. Stage 7: Run `npx ai-engineering-loop escalation --json`, resolve the required tier through the host catalog, or escalate to the user. Build Judge context from goal-contract.json, verification.json, and findings.json only. Spawn Judge and wait. Write `RUN_DIR/verdict.json` with modelTier, then run `npx ai-engineering-loop gate judge`.
|
|
43
49
|
8. ITERATE with iteration under 3: fix in the parent, re-verify, spawn a **new** Devil's Advocate.
|
|
44
|
-
9. After Judge PASS: run
|
|
50
|
+
9. After Judge PASS: run after_pass, then the configured Stage 8 adapter. PR/MR body includes `Steps to Reproduce & Testing (QA)`; bugfix placeholders must not stay empty. Preview remote payloads and never include context packs, raw logs, or credentials. Write `RUN_DIR/delivery.json`, then run `npx ai-engineering-loop gate delivery`.
|
|
45
51
|
|
|
46
52
|
## Report header
|
|
47
53
|
|
|
@@ -46,20 +46,26 @@ Run `npx ai-engineering-loop <command>` in the target repo. Do not commit unless
|
|
|
46
46
|
|
|
47
47
|
Compact map (not a second OS): Specify (stages 0-1), Make (stages 2-4), Review (stages 5-7), Deliver (stage 8). Keep the 8-stage numbers. Do not skip Goal Contract, verification, Devil's Advocate, or Judge.
|
|
48
48
|
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
49
|
+
## Deterministic artifact gates (mandatory)
|
|
50
|
+
|
|
51
|
+
Run `npx ai-engineering-loop state --json` to get the current `runId` and use `.ai-engineering-loop/runs/<runId>/` as `RUN_DIR`. Never edit `state.json` or `current.json`.
|
|
52
|
+
|
|
53
|
+
Inherited invariants remain mandatory: read `core/grill-policy.md`, `policies/tdd-policy.md`, and `.ai-engineering-loop/glossary.md`. On `adapter_type: dot`, grill includes blast radius: lifecycle sketch and load `task-impact-inquiry`. If the user prompt has no numbered AC, draft the contract, list a short menu and wait. Chat agreement is not freeze. AC is a failure table. One red test per AC row. Do not freeze sunny-path-only. Write claimed-vs-reality.md. Do not spawn Devil's Advocate if the file is missing or any Claimed row lacks Reality. Report Spec and Standards separately. DA prompt includes "at most 8 tool calls; read the diff file; skip css and generated files". Judge prompt includes "at most 4 tool calls; ledger and contract only; skip css; do not re-review the whole diff". `generate-workflow` still reads workflow.md and lessons.md; missing maker_intern means INVOCATION_UNAVAILABLE. Do not type a model name.
|
|
54
|
+
|
|
55
|
+
1. Stage 0: `npx ai-engineering-loop sync-hosts`, `status`, then `run` to create or resume the ledger. Read glossary, lessons, and workflow overlays. Required stages cannot be skipped.
|
|
56
|
+
2. Stage 1: keep the human Goal Contract Markdown and write `RUN_DIR/goal-contract.json` with schemaVersion, runId, objective, numbered AC, evidenceRequired, and non-empty failureCases. Grill ambiguous decisions and business blast radius first. Run `npx ai-engineering-loop gate goal`; do not edit production code unless it passes.
|
|
57
|
+
3. Stages 2–4: Resolve `maker_intern` only through `grok models`; unknown means parent Maker. Intern cannot skip gates or spawn DA/Judge. Use red repro or TDD, write the exact review diff to `RUN_DIR/diff.patch`, then run `npx ai-engineering-loop gate maker`.
|
|
58
|
+
4. Stage 5: run verification and write `RUN_DIR/verification.json` with current git revision, gated diff hash, command, execution identity, times, exit code, stdout, timeout status, and test counts. Keep claimed-vs-reality.md. Run `npx ai-engineering-loop gate verification`; do not spawn DA unless it passes.
|
|
53
59
|
5. Write artifacts to disk so children do not need parent chat:
|
|
54
60
|
- Goal Contract path
|
|
55
61
|
- `git diff` written to `.ai-engineering-loop/tasks/current.diff`
|
|
56
62
|
- verification log (file)
|
|
57
63
|
- `.ai-engineering-loop/tasks/claimed-vs-reality.md` (do not spawn DA without it)
|
|
58
64
|
- If stopping mid-loop, `.ai-engineering-loop/tasks/handoff.md` (`core/handoff-policy.md`)
|
|
59
|
-
6. Stage 6: `
|
|
60
|
-
7. Stage 7: `
|
|
65
|
+
6. Stage 6: Build `npx ai-engineering-loop context devil-advocate <files...>`. Spawn Devil's Advocate with the pack path, `background: false`, `capability_mode: "execute"`, and no `resume_from`. Normalize its output into `RUN_DIR/findings.json` with runId and diffHash, then run `npx ai-engineering-loop gate review`.
|
|
66
|
+
7. Stage 7: Run `npx ai-engineering-loop escalation --json`; resolve `STRONG_MODEL` through `grok models`, or escalate to the user if unavailable. Build Judge context from only goal-contract.json, verification.json, and findings.json. Spawn a fresh sibling Judge and wait. Normalize `RUN_DIR/verdict.json` with modelTier, then run `npx ai-engineering-loop gate judge`.
|
|
61
67
|
8. If Judge says `ITERATE` and iteration < 3, Maker fixes in the parent, re-verify, spawn a **fresh** DA (new spawn, no resume).
|
|
62
|
-
9. After Judge PASS: run
|
|
68
|
+
9. After Judge PASS: run `after_pass`, then the configured Stage 8 adapter. PR/MR body includes `Steps to Reproduce & Testing (QA)`; bugfix placeholders must not stay empty. Preview remote payloads and never include context packs, raw logs, or credentials. Write `RUN_DIR/delivery.json`, then run `npx ai-engineering-loop gate delivery`.
|
|
63
69
|
|
|
64
70
|
After a proven Grok DA spawn, the report header must be:
|
|
65
71
|
|
package/README.md
CHANGED
|
@@ -6,11 +6,11 @@
|
|
|
6
6
|
[](https://opensource.org/licenses/MIT)
|
|
7
7
|
[](https://github.com/egagofur/ai-engineering-loop/pulls)
|
|
8
8
|
[](https://github.com/egagofur/ai-engineering-loop)
|
|
9
|
-
[](https://github.com/egagofur/ai-engineering-loop/releases)
|
|
10
10
|
|
|
11
11
|
**A Reusable, Framework-Agnostic AI Engineering Operating System for Autonomous Coding Agents**
|
|
12
12
|
|
|
13
|
-
*Featuring living project context,
|
|
13
|
+
*Featuring living project context, deterministic artifact gates, bounded secret-redacted context, cheap-model escalation, and dual-axis Judge evaluation.*
|
|
14
14
|
|
|
15
15
|
[Overview](#overview--philosophy) • [Stage techniques](#stage-techniques) • [Runtime Capability Registry](#runtime-capability-registry--execution-modes) • [Verification Evidence](#verification-evidence-contract) • [CLI Commands](#cli-interface--commands) • [Grok CLI](#grok-cli-integration) • [Claude Code](#claude-code-integration) • [Antigravity](#antigravity-agent-integration) • [Lifecycle](#lifecycle-stages) • [Architecture](#architecture--5-layer-configuration) • [Project Profiles](#project-profiles) • [Repository Structure](#repository-structure) • [Reference Examples](#reference-examples) • [Contributing](#contributing)
|
|
16
16
|
|
|
@@ -71,6 +71,7 @@ The 8-stage loop stays one OS. These techniques sit **inside** existing stages (
|
|
|
71
71
|
| 4–5 | Red-green at named **seams**; failure table (not happy path only); coverage is a map; no grep/tautology | `policies/tdd-policy.md` |
|
|
72
72
|
| 5 | **Claimed vs Reality** table before DA. Missing file or empty Reality blocks Devil's Advocate. "Seems green" is not Reality. | `core/verification-loop.md` |
|
|
73
73
|
| 6–7 | Spec vs Standards reported separately. Standards BLOCKER/HIGH iterate only when `hardConvention` is true | `policies/finding-policy.md` |
|
|
74
|
+
| 0–8 | Stateful run ledger and deterministic artifact transitions; model claims cannot advance a stage | `core/artifact-gates.md` |
|
|
74
75
|
| any | Mid-loop stop writes `.ai-engineering-loop/tasks/handoff.md` | `core/handoff-policy.md` |
|
|
75
76
|
| any | **Compact map** (host guidance, not a second OS): Specify (stages 0-1), Make (stages 2-4), Review (stages 5-7), Deliver (stage 8). Keep the 8-stage numbers. Do not skip Goal Contract, verification, Devil's Advocate, or Judge. | host skills |
|
|
76
77
|
|
|
@@ -178,9 +179,29 @@ npx ai-engineering-loop status
|
|
|
178
179
|
# Reconcile drifted context against repository non-destructively
|
|
179
180
|
npx ai-engineering-loop refresh
|
|
180
181
|
|
|
181
|
-
#
|
|
182
|
+
# Start or resume a stateful engineering run
|
|
182
183
|
npx ai-engineering-loop run
|
|
183
184
|
|
|
185
|
+
# Inspect the run ledger
|
|
186
|
+
npx ai-engineering-loop state --json
|
|
187
|
+
|
|
188
|
+
# Validate artifacts and advance stages
|
|
189
|
+
npx ai-engineering-loop gate goal
|
|
190
|
+
npx ai-engineering-loop gate maker
|
|
191
|
+
npx ai-engineering-loop gate verification
|
|
192
|
+
npx ai-engineering-loop gate review
|
|
193
|
+
npx ai-engineering-loop escalation --json
|
|
194
|
+
npx ai-engineering-loop gate judge
|
|
195
|
+
npx ai-engineering-loop gate delivery
|
|
196
|
+
|
|
197
|
+
# Build bounded model-visible context
|
|
198
|
+
npx ai-engineering-loop context devil-advocate <files...>
|
|
199
|
+
npx ai-engineering-loop context judge <files...>
|
|
200
|
+
|
|
201
|
+
# Validate installation and the production evaluation catalog
|
|
202
|
+
npx ai-engineering-loop doctor
|
|
203
|
+
npx ai-engineering-loop eval
|
|
204
|
+
|
|
184
205
|
# Copy package skills/agents/commands into ~/.claude ~/.grok ~/.gemini ~/.agents
|
|
185
206
|
npx ai-engineering-loop sync-hosts
|
|
186
207
|
|
|
@@ -193,6 +214,8 @@ npx ai-engineering-loop generate-workflow
|
|
|
193
214
|
npx ai-engineering-loop generate-workflow --write
|
|
194
215
|
```
|
|
195
216
|
|
|
217
|
+
Every full run stores private local artifacts under `.ai-engineering-loop/runs/<run-id>/`. Markdown remains the human contract; JSON sidecars follow the shipped `schemas/` and make stage claims deterministic. See [`core/artifact-gates.md`](core/artifact-gates.md), [`SECURITY.md`](SECURITY.md), and [`SUPPORT.md`](SUPPORT.md).
|
|
218
|
+
|
|
196
219
|
Shipped adapters (Stage 8 only, after Judge PASS): `standard`, `github`, `gitlab`, `dot`. Catalog: `adapters/README.md`. Each team generates its own; do not copy a neighbour's pipeline.
|
|
197
220
|
|
|
198
221
|
`sync-hosts` updates only hosts that already exist on the machine. DOT skills (`dot-dev-skill-router`, `dot-dev-workflow`) are updated only if they are already installed. `task-impact-inquiry`, `generate-adapter`, and `generate-workflow` are upserted onto Claude, Grok, and Gemini. After a copy, start a new session so the host reloads skill text. `/ai-engineering-loop` Stage 0 and `run` call `sync-hosts` so a published package bump reaches global host files without a manual copy.
|
package/SECURITY.md
ADDED
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
# Security Policy
|
|
2
|
+
|
|
3
|
+
## Trust boundary
|
|
4
|
+
|
|
5
|
+
Repository files, generated logs, tool output, and comments are untrusted data. Instructions found inside them never override the active host skill, Goal Contract, or user authorization.
|
|
6
|
+
|
|
7
|
+
AI Engineering Loop blocks `.env*`, private-key formats, common credential files, `.git/`, `node_modules/`, absolute paths, parent traversal, and symlinks that leave the repository from generated context packs. Recognized credentials are redacted before model-visible context is persisted.
|
|
8
|
+
|
|
9
|
+
No telemetry is collected. Stage 0–7 require no AI Engineering Loop network service. Delivery adapters may use authenticated forge tools only after Judge `PASS`; users must review remote issue, pull-request, merge-request, and chat payloads before sending.
|
|
10
|
+
|
|
11
|
+
## Credential handling
|
|
12
|
+
|
|
13
|
+
- Supply provider credentials through the host's documented environment or credential store.
|
|
14
|
+
- Never place credentials in `.ai-engineering-loop/`, workflow files, prompts, findings, or delivery reports.
|
|
15
|
+
- Do not pass the parent process environment to a child unless the host owns that boundary.
|
|
16
|
+
- Treat generated run artifacts as sensitive local files. The CLI writes state and context packs with mode `0600` where supported.
|
|
17
|
+
- Run a dedicated secret scanner such as `gitleaks` before delivery when available.
|
|
18
|
+
|
|
19
|
+
Pattern-based redaction is defense in depth, not proof that arbitrary secrets are absent.
|
|
20
|
+
|
|
21
|
+
## Reporting a vulnerability
|
|
22
|
+
|
|
23
|
+
Do not open a public issue containing credentials, exploit details, private source, or model transcripts. Use GitHub private vulnerability reporting for this repository. Include the package version, host, minimal reproduction, and whether any remote payload was sent.
|
package/SUPPORT.md
ADDED
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
# Public Beta Support Matrix
|
|
2
|
+
|
|
3
|
+
AI Engineering Loop is in public-beta readiness. The workflow contract is stable enough for evaluation, but host tool schemas can change independently.
|
|
4
|
+
|
|
5
|
+
| Host | Installation source | Independent reviewer path | Fallback |
|
|
6
|
+
|---|---|---|---|
|
|
7
|
+
| Claude Code / compatible Task hosts | `.claude/` | `Task` or `Agent` siblings | `CONTEXT_ISOLATION_ONLY` |
|
|
8
|
+
| Grok CLI | `.grok/` | `spawn_subagent` siblings | `grok -p` or `CONTEXT_ISOLATION_ONLY` |
|
|
9
|
+
| Gemini / Antigravity skill hosts | `.gemini/` | `invoke_subagent` or `Task` when present | `CONTEXT_ISOLATION_ONLY` |
|
|
10
|
+
| Antigravity workflows | `.agents/` | `invoke_subagent` or `Task` when present | `CONTEXT_ISOLATION_ONLY` |
|
|
11
|
+
|
|
12
|
+
Independent execution is reported as proven only after a child ID and model response are observed without inherited Maker history.
|
|
13
|
+
|
|
14
|
+
## Runtime
|
|
15
|
+
|
|
16
|
+
- Node.js 18 or newer.
|
|
17
|
+
- Git is recommended for revision and diff evidence.
|
|
18
|
+
- Linux, macOS, and Windows are covered by the Node test matrix.
|
|
19
|
+
- No runtime npm dependencies.
|
|
20
|
+
|
|
21
|
+
Run diagnostics after installation:
|
|
22
|
+
|
|
23
|
+
```bash
|
|
24
|
+
npx ai-engineering-loop doctor
|
|
25
|
+
npx ai-engineering-loop eval
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
New hosts need a documented tool schema, context-isolation behavior, secret boundary, independent-execution proof, and conformance tests before being listed as supported.
|
|
@@ -33,8 +33,33 @@ const {
|
|
|
33
33
|
workflowMarkdown,
|
|
34
34
|
lessonsMarkdown
|
|
35
35
|
} = require('../lib/generate-workflow.js');
|
|
36
|
+
const {
|
|
37
|
+
createOrResumeRun,
|
|
38
|
+
getCurrentRun,
|
|
39
|
+
loadRun
|
|
40
|
+
} = require('../lib/run-state.js');
|
|
41
|
+
const {
|
|
42
|
+
applyGate
|
|
43
|
+
} = require('../lib/gates.js');
|
|
44
|
+
const {
|
|
45
|
+
createContextPack,
|
|
46
|
+
contextPackSummary
|
|
47
|
+
} = require('../lib/safe-context.js');
|
|
48
|
+
const {
|
|
49
|
+
defaultCasesDir,
|
|
50
|
+
loadEvaluationCases,
|
|
51
|
+
loadResults,
|
|
52
|
+
scoreEvaluationResults,
|
|
53
|
+
catalogSummary
|
|
54
|
+
} = require('../lib/evaluation.js');
|
|
55
|
+
const {
|
|
56
|
+
assessRunEscalation
|
|
57
|
+
} = require('../lib/escalation.js');
|
|
58
|
+
const {
|
|
59
|
+
runDoctor
|
|
60
|
+
} = require('../lib/doctor.js');
|
|
36
61
|
|
|
37
|
-
const VERSION = '1.0
|
|
62
|
+
const VERSION = '1.1.0';
|
|
38
63
|
const CWD = process.cwd();
|
|
39
64
|
const CONTEXT_DIR = path.join(CWD, '.ai-engineering-loop');
|
|
40
65
|
|
|
@@ -288,7 +313,7 @@ function generateContextFiles(rootDir, discovery, trigger = 'init', impact = 'IN
|
|
|
288
313
|
|
|
289
314
|
// 0. metadata.json (Baseline)
|
|
290
315
|
const metadataJson = {
|
|
291
|
-
contextVersion: '1.0
|
|
316
|
+
contextVersion: '1.1.0',
|
|
292
317
|
generatedAt: new Date().toISOString(),
|
|
293
318
|
repositoryRevision: currentRevision,
|
|
294
319
|
projectProfile: discovery.profile,
|
|
@@ -428,6 +453,11 @@ What becomes easier, harder, or forbidden.
|
|
|
428
453
|
|
|
429
454
|
writeContextFile(path.join(targetDir, 'workflow.md'), workflowMarkdown(), { overwrite: false });
|
|
430
455
|
writeContextFile(path.join(targetDir, 'lessons.md'), lessonsMarkdown(), { overwrite: false });
|
|
456
|
+
writeContextFile(
|
|
457
|
+
path.join(targetDir, '.gitignore'),
|
|
458
|
+
'# Runtime artifacts may contain source excerpts and verification logs.\nruns/\ntasks/current.diff\ntasks/*.log\n',
|
|
459
|
+
{ overwrite: false }
|
|
460
|
+
);
|
|
431
461
|
}
|
|
432
462
|
|
|
433
463
|
/**
|
|
@@ -725,6 +755,19 @@ function handleRun() {
|
|
|
725
755
|
|
|
726
756
|
syncHostsQuiet();
|
|
727
757
|
|
|
758
|
+
const runArgs = process.argv.slice(3);
|
|
759
|
+
const forceNew = runArgs.includes('--new');
|
|
760
|
+
const task = runArgs.filter((arg) => arg !== '--new').join(' ').trim();
|
|
761
|
+
let runResult;
|
|
762
|
+
try {
|
|
763
|
+
runResult = createOrResumeRun(CWD, { task, forceNew });
|
|
764
|
+
} catch (err) {
|
|
765
|
+
log.error(err.message);
|
|
766
|
+
process.exit(1);
|
|
767
|
+
}
|
|
768
|
+
console.log(`- Run: ${runResult.run.runId} (${runResult.resumed ? 'resumed' : 'created'})`);
|
|
769
|
+
console.log(`- State: ${runResult.run.state}, iteration ${runResult.run.iteration}`);
|
|
770
|
+
|
|
728
771
|
const grok = detectGrokHost();
|
|
729
772
|
|
|
730
773
|
console.log('\n------------------------------------------------------------');
|
|
@@ -778,6 +821,176 @@ function handleRun() {
|
|
|
778
821
|
console.log('------------------------------------------------------------\n');
|
|
779
822
|
}
|
|
780
823
|
|
|
824
|
+
function argValue(name) {
|
|
825
|
+
const index = process.argv.indexOf(name);
|
|
826
|
+
return index === -1 ? null : process.argv[index + 1] || null;
|
|
827
|
+
}
|
|
828
|
+
|
|
829
|
+
function handleGate() {
|
|
830
|
+
const gate = process.argv[3];
|
|
831
|
+
const runId = argValue('--run');
|
|
832
|
+
const json = process.argv.includes('--json');
|
|
833
|
+
try {
|
|
834
|
+
const run = applyGate(CWD, gate, { runId });
|
|
835
|
+
if (json) {
|
|
836
|
+
console.log(JSON.stringify({
|
|
837
|
+
ok: true,
|
|
838
|
+
runId: run.runId,
|
|
839
|
+
state: run.state,
|
|
840
|
+
iteration: run.iteration,
|
|
841
|
+
gate
|
|
842
|
+
}));
|
|
843
|
+
} else {
|
|
844
|
+
log.success(`✓ ${gate} gate passed`);
|
|
845
|
+
console.log(`- Run: ${run.runId}`);
|
|
846
|
+
console.log(`- State: ${run.state}, iteration ${run.iteration}`);
|
|
847
|
+
}
|
|
848
|
+
} catch (err) {
|
|
849
|
+
if (json) {
|
|
850
|
+
console.log(JSON.stringify({
|
|
851
|
+
ok: false,
|
|
852
|
+
code: err.code || 'GATE_FAILED',
|
|
853
|
+
error: err.message,
|
|
854
|
+
details: err.details || []
|
|
855
|
+
}));
|
|
856
|
+
} else {
|
|
857
|
+
log.error(err.message);
|
|
858
|
+
for (const detail of err.details || []) console.error(`- ${detail}`);
|
|
859
|
+
}
|
|
860
|
+
process.exit(1);
|
|
861
|
+
}
|
|
862
|
+
}
|
|
863
|
+
|
|
864
|
+
function commandFiles(startIndex, optionsWithValues = []) {
|
|
865
|
+
const files = [];
|
|
866
|
+
for (let index = startIndex; index < process.argv.length; index++) {
|
|
867
|
+
const value = process.argv[index];
|
|
868
|
+
if (optionsWithValues.includes(value)) {
|
|
869
|
+
index += 1;
|
|
870
|
+
continue;
|
|
871
|
+
}
|
|
872
|
+
if (!value.startsWith('--')) files.push(value);
|
|
873
|
+
}
|
|
874
|
+
return files;
|
|
875
|
+
}
|
|
876
|
+
|
|
877
|
+
function handleContext() {
|
|
878
|
+
const stage = process.argv[3];
|
|
879
|
+
const runId = argValue('--run');
|
|
880
|
+
const json = process.argv.includes('--json');
|
|
881
|
+
const files = commandFiles(4, ['--run']);
|
|
882
|
+
try {
|
|
883
|
+
const result = createContextPack(CWD, { runId, stage, files });
|
|
884
|
+
const summary = contextPackSummary(result);
|
|
885
|
+
if (json) {
|
|
886
|
+
console.log(JSON.stringify({ ok: true, ...summary }));
|
|
887
|
+
} else {
|
|
888
|
+
log.success(`✓ ${stage} context pack created`);
|
|
889
|
+
console.log(`- Path: ${summary.path}`);
|
|
890
|
+
console.log(`- Files: ${summary.files}, estimated tokens: ${summary.estimatedTokens}`);
|
|
891
|
+
console.log(`- Redactions: ${summary.redactions}, truncated files: ${summary.truncatedFiles}`);
|
|
892
|
+
}
|
|
893
|
+
} catch (err) {
|
|
894
|
+
if (json) {
|
|
895
|
+
console.log(JSON.stringify({ ok: false, code: err.code || 'UNSAFE_CONTEXT', error: err.message }));
|
|
896
|
+
} else {
|
|
897
|
+
log.error(err.message);
|
|
898
|
+
}
|
|
899
|
+
process.exit(1);
|
|
900
|
+
}
|
|
901
|
+
}
|
|
902
|
+
|
|
903
|
+
function handleEval() {
|
|
904
|
+
const casesDir = argValue('--cases') || defaultCasesDir();
|
|
905
|
+
const resultsDir = argValue('--results');
|
|
906
|
+
const json = process.argv.includes('--json');
|
|
907
|
+
try {
|
|
908
|
+
const fixtures = loadEvaluationCases(casesDir);
|
|
909
|
+
if (!resultsDir) {
|
|
910
|
+
const summary = catalogSummary(fixtures);
|
|
911
|
+
if (json) console.log(JSON.stringify({ ok: true, mode: 'catalog', ...summary }));
|
|
912
|
+
else {
|
|
913
|
+
log.success(`✓ Evaluation catalog valid (${summary.total} cases)`);
|
|
914
|
+
console.log(`- Categories: ${Object.keys(summary.categories).join(', ')}`);
|
|
915
|
+
console.log(`- Oracle verdicts: ${JSON.stringify(summary.verdicts)}`);
|
|
916
|
+
}
|
|
917
|
+
return;
|
|
918
|
+
}
|
|
919
|
+
|
|
920
|
+
const score = scoreEvaluationResults(fixtures, loadResults(resultsDir));
|
|
921
|
+
const ok = score.failed === 0 && score.incorrectPasses === 0 && score.secretLeaks === 0;
|
|
922
|
+
if (json) console.log(JSON.stringify({ ok, ...score }));
|
|
923
|
+
else {
|
|
924
|
+
console.log(`Evaluation: ${score.passed}/${score.total} passed`);
|
|
925
|
+
console.log(`- Incorrect PASS: ${score.incorrectPasses}`);
|
|
926
|
+
console.log(`- Secret leaks: ${score.secretLeaks}`);
|
|
927
|
+
for (const item of score.cases.filter((result) => !result.passed)) {
|
|
928
|
+
console.log(` ${item.caseId}: expected=${item.expected} actual=${item.actual || 'MISSING'}`);
|
|
929
|
+
}
|
|
930
|
+
}
|
|
931
|
+
if (!ok) process.exit(1);
|
|
932
|
+
} catch (err) {
|
|
933
|
+
if (json) console.log(JSON.stringify({ ok: false, code: err.code || 'EVALUATION_FAILED', error: err.message, details: err.details || [] }));
|
|
934
|
+
else {
|
|
935
|
+
log.error(err.message);
|
|
936
|
+
for (const detail of err.details || []) console.error(`- ${detail}`);
|
|
937
|
+
}
|
|
938
|
+
process.exit(1);
|
|
939
|
+
}
|
|
940
|
+
}
|
|
941
|
+
|
|
942
|
+
function handleEscalation() {
|
|
943
|
+
const runId = argValue('--run');
|
|
944
|
+
const json = process.argv.includes('--json');
|
|
945
|
+
try {
|
|
946
|
+
const assessment = assessRunEscalation(CWD, { runId });
|
|
947
|
+
if (json) console.log(JSON.stringify({ ok: true, ...assessment }));
|
|
948
|
+
else {
|
|
949
|
+
console.log(`Required model tier: ${assessment.requiredTier}`);
|
|
950
|
+
for (const reason of assessment.reasons) {
|
|
951
|
+
console.log(`- ${reason.code}: ${reason.detail}`);
|
|
952
|
+
}
|
|
953
|
+
}
|
|
954
|
+
} catch (err) {
|
|
955
|
+
if (json) console.log(JSON.stringify({ ok: false, code: err.code || 'ESCALATION_FAILED', error: err.message }));
|
|
956
|
+
else log.error(err.message);
|
|
957
|
+
process.exit(1);
|
|
958
|
+
}
|
|
959
|
+
}
|
|
960
|
+
|
|
961
|
+
function handleState() {
|
|
962
|
+
const runId = argValue('--run');
|
|
963
|
+
const json = process.argv.includes('--json');
|
|
964
|
+
try {
|
|
965
|
+
const run = runId ? loadRun(CWD, runId) : getCurrentRun(CWD);
|
|
966
|
+
if (!run) throw new Error('No current run. Start one with `ai-engineering-loop run`.');
|
|
967
|
+
if (json) console.log(JSON.stringify({ ok: true, ...run }));
|
|
968
|
+
else {
|
|
969
|
+
console.log(`Run: ${run.runId}`);
|
|
970
|
+
console.log(`State: ${run.state}`);
|
|
971
|
+
console.log(`Iteration: ${run.iteration}`);
|
|
972
|
+
console.log(`Artifacts: ${Object.keys(run.artifacts).join(', ') || 'none'}`);
|
|
973
|
+
}
|
|
974
|
+
} catch (err) {
|
|
975
|
+
if (json) console.log(JSON.stringify({ ok: false, code: err.code || 'STATE_FAILED', error: err.message }));
|
|
976
|
+
else log.error(err.message);
|
|
977
|
+
process.exit(1);
|
|
978
|
+
}
|
|
979
|
+
}
|
|
980
|
+
|
|
981
|
+
function handleDoctor() {
|
|
982
|
+
const json = process.argv.includes('--json');
|
|
983
|
+
const result = runDoctor(path.join(__dirname, '..'));
|
|
984
|
+
if (json) console.log(JSON.stringify(result));
|
|
985
|
+
else {
|
|
986
|
+
console.log(`AI Engineering Loop doctor: ${result.ok ? 'READY' : 'NOT READY'}`);
|
|
987
|
+
for (const check of result.checks) {
|
|
988
|
+
console.log(`${check.passed ? '✓' : '✗'} ${check.id}: ${check.detail}`);
|
|
989
|
+
}
|
|
990
|
+
}
|
|
991
|
+
if (!result.ok) process.exit(1);
|
|
992
|
+
}
|
|
993
|
+
|
|
781
994
|
// Help Menu
|
|
782
995
|
function printHelp() {
|
|
783
996
|
console.log(`
|
|
@@ -791,7 +1004,28 @@ Commands:
|
|
|
791
1004
|
init Bootstrap .ai-engineering-loop/ context from repository discovery
|
|
792
1005
|
status Check the validity, readiness, and baseline freshness of context
|
|
793
1006
|
refresh Reconcile drifted context against repository non-destructively
|
|
794
|
-
run
|
|
1007
|
+
run [task] Start or resume a stateful engineering run, sync hosts, and instruct the agent
|
|
1008
|
+
--new start a new run even when another run is active
|
|
1009
|
+
state Inspect the current run ledger
|
|
1010
|
+
--run <id> target a non-current run
|
|
1011
|
+
--json print the complete machine-readable ledger
|
|
1012
|
+
gate <name> Validate an artifact and advance the current run
|
|
1013
|
+
goal | maker | verification | review | judge | delivery
|
|
1014
|
+
--run <id> target a non-current run
|
|
1015
|
+
--json print machine-readable gate output
|
|
1016
|
+
context <stage> <files...>
|
|
1017
|
+
Build a bounded, redacted context pack for maker, devil-advocate, or judge
|
|
1018
|
+
--run <id> target a non-current run
|
|
1019
|
+
--json print only pack metadata; never print packed content
|
|
1020
|
+
eval Validate the 20-case production evaluation catalog
|
|
1021
|
+
--results <dir> score host-generated JSON results
|
|
1022
|
+
--cases <dir> use another compatible fixture catalog
|
|
1023
|
+
--json print machine-readable metrics
|
|
1024
|
+
escalation Compute the minimum Judge model tier from deterministic risk signals
|
|
1025
|
+
--run <id> target a non-current run
|
|
1026
|
+
--json print machine-readable reasons
|
|
1027
|
+
doctor Check runtime, schemas, host assets, package contents, and eval catalog
|
|
1028
|
+
--json print machine-readable diagnostics
|
|
795
1029
|
sync-hosts Copy package skills/agents/commands into ~/.claude ~/.grok ~/.gemini ~/.agents
|
|
796
1030
|
(only hosts that already exist; DOT skills only if already installed)
|
|
797
1031
|
--dry-run print the plan without writing
|
|
@@ -826,6 +1060,24 @@ switch (command) {
|
|
|
826
1060
|
case 'run':
|
|
827
1061
|
handleRun();
|
|
828
1062
|
break;
|
|
1063
|
+
case 'state':
|
|
1064
|
+
handleState();
|
|
1065
|
+
break;
|
|
1066
|
+
case 'gate':
|
|
1067
|
+
handleGate();
|
|
1068
|
+
break;
|
|
1069
|
+
case 'context':
|
|
1070
|
+
handleContext();
|
|
1071
|
+
break;
|
|
1072
|
+
case 'eval':
|
|
1073
|
+
handleEval();
|
|
1074
|
+
break;
|
|
1075
|
+
case 'escalation':
|
|
1076
|
+
handleEscalation();
|
|
1077
|
+
break;
|
|
1078
|
+
case 'doctor':
|
|
1079
|
+
handleDoctor();
|
|
1080
|
+
break;
|
|
829
1081
|
case 'sync-hosts':
|
|
830
1082
|
handleSyncHosts();
|
|
831
1083
|
break;
|