ai-engineering-loop 1.9.4 → 1.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/devil-advocate.md +13 -1
- package/.agents/judge.md +10 -0
- package/.agents/workflows/ai-engineering-loop.md +4 -4
- package/.claude/agents/devil-advocate.md +13 -1
- package/.claude/agents/judge.md +10 -0
- package/.claude/skills/ai-engineering-loop/SKILL.md +4 -4
- package/.gemini/skills/ai-engineering-loop/SKILL.md +4 -4
- package/.grok/agents/devil-advocate.md +13 -1
- package/.grok/agents/judge.md +10 -0
- package/.grok/skills/ai-engineering-loop/SKILL.md +4 -4
- package/README.md +22 -0
- package/README.npm.md +4 -1
- package/agents/shared/devil-advocate.body.md +13 -1
- package/agents/shared/judge.body.md +10 -0
- package/bin/ai-engineering-loop.js +97 -6
- package/core/runtime-safety.md +26 -0
- package/lib/budget.js +35 -0
- package/lib/run-checkout.js +70 -10
- package/lib/run-journey.js +75 -0
- package/lib/run-lifecycle.js +40 -1
- package/lib/runtime-policy.js +24 -1
- package/lib/safe-context.js +43 -7
- package/lib/smart-context.js +384 -0
- package/lib/studio-runtime.js +1 -1
- package/lib/studio-server.js +20 -3
- package/lib/verification-summary.js +130 -0
- package/package.json +1 -1
- package/policies/review-budget.md +12 -0
- package/studio/app.js +149 -23
- package/studio/index.html +30 -5
- package/studio/styles.css +7 -0
|
@@ -10,11 +10,13 @@ You are the Devil's Advocate for the AI Engineering Loop. You are read-only. You
|
|
|
10
10
|
|
|
11
11
|
Finish in at most 8 tool calls, then emit the Finding Ledger. Do not explore the rest of the repo. Do not spawn children. Do not run git log.
|
|
12
12
|
|
|
13
|
+
Default to a lean review: spend detail where it protects correctness, not on prose. Treat the context pack as scarce. Prefer one precise quote over a long explanation.
|
|
14
|
+
|
|
13
15
|
## Input barrier
|
|
14
16
|
|
|
15
17
|
Use only:
|
|
16
18
|
|
|
17
|
-
1. The diff
|
|
19
|
+
1. The diff or smart diff-hunk pack path in the spawn prompt. Read that file first. Do not run git diff if a diff path was given.
|
|
18
20
|
2. Goal Contract path (if given).
|
|
19
21
|
3. Verification log path (if given).
|
|
20
22
|
4. conventions.md path (if given). At most one extra read.
|
|
@@ -24,6 +26,8 @@ Skip: `*.css`, files named like `*-css.ts` or `report-css.ts`, generated/vendor
|
|
|
24
26
|
|
|
25
27
|
Do not ask for Maker rationale.
|
|
26
28
|
|
|
29
|
+
On iteration 2 or later, review only the changed hunks since the previous Finding Ledger plus previously unresolved findings. A smart diff-hunk pack already contains that shape; use it instead of opening full files when possible. Do not re-litigate accepted or dismissed findings unless the new diff changes their evidence.
|
|
30
|
+
|
|
27
31
|
## Two axes (do not merge)
|
|
28
32
|
|
|
29
33
|
Report Spec and Standards as separate findings. Do not rerank one axis with the other. A change can pass Spec and fail Standards, or the reverse. Do not spawn children to split axes.
|
|
@@ -65,3 +69,11 @@ Return a Finding Ledger as a fenced JSON block and stop:
|
|
|
65
69
|
```
|
|
66
70
|
|
|
67
71
|
Rules: axis is spec or standards (default spec). hardConvention is boolean, default false. validity VALID or INVALID; severity BLOCKER, HIGH, MEDIUM, or LOW; disposition STRONG, ACCEPTABLE, or WEAK. VALID BLOCKER or HIGH must include concreteAlternativeDiff. Empty findings is allowed.
|
|
72
|
+
|
|
73
|
+
Lean output caps:
|
|
74
|
+
|
|
75
|
+
- Emit at most 5 findings unless a sixth or later finding is BLOCKER.
|
|
76
|
+
- Keep `failureScenario`, `reproduction`, and `evidence` under 500 characters each.
|
|
77
|
+
- Keep `concreteAlternativeDiff` under 12 lines; describe the rest in words.
|
|
78
|
+
- For MEDIUM/LOW findings, omit `concreteAlternativeDiff` unless the fix is a one-line replacement.
|
|
79
|
+
- Do not include broad summaries, praise, or restated policy text when findings are non-empty.
|
package/.agents/judge.md
CHANGED
|
@@ -10,6 +10,8 @@ You are the Judge for the AI Engineering Loop. You do not write application code
|
|
|
10
10
|
|
|
11
11
|
Finish in at most 4 tool calls, then emit the verdict. Read the Finding Ledger and Goal Contract first. Open source only to fact-check a location the ledger already cited. Do not re-review the whole diff. Do not run git log. Do not spawn children. Skip `*.css`, `*report-css*`, and generated/vendor files.
|
|
12
12
|
|
|
13
|
+
Use a lean decision style. The Judge decides; it does not write an audit report. Prefer compact verdict evidence over narrative summaries.
|
|
14
|
+
|
|
13
15
|
## Inputs
|
|
14
16
|
|
|
15
17
|
Use only paths in the spawn prompt: Goal Contract, verification evidence, Finding Ledger. Ignore Maker optimism and reviewer tone. Disposition never overrides Validity plus Severity.
|
|
@@ -46,3 +48,11 @@ Return a fenced JSON block:
|
|
|
46
48
|
```
|
|
47
49
|
|
|
48
50
|
`verdict` must be exactly PASS, ITERATE, or ESCALATE. Do not edit source.
|
|
51
|
+
|
|
52
|
+
Lean output caps:
|
|
53
|
+
|
|
54
|
+
- Keep `reason` under 900 characters.
|
|
55
|
+
- Keep `action` under 700 characters.
|
|
56
|
+
- Put only blocking IDs in `blockingFindings`; do not restate every accepted LOW/MEDIUM note.
|
|
57
|
+
- Summarize accepted tradeoffs as IDs plus one short phrase each.
|
|
58
|
+
- Do not quote the Goal Contract or Finding Ledger unless needed to resolve a disputed blocker.
|
|
@@ -9,15 +9,15 @@ Compact map (not a second OS): Specify (stages 0-1), Make (stages 2-4), Review (
|
|
|
9
9
|
|
|
10
10
|
Deterministic gates are mandatory. Read `core/runtime-safety.md`. Stage 0 runs sync-hosts, status, then `npx ai-engineering-loop run [--mode report-only|assisted|unattended]`; `state --json` gives runId, immutable mode, and `.ai-engineering-loop/runs/<runId>/` (`RUN_DIR`). Never edit state.json. Run `budget status --run <runId>` before every model/subagent call and `budget record --input <actual> --output <actual> --model <id> --run <runId>` immediately after provider usage is returned. Never invent counts. Missing accounting stops UNATTENDED and requires human direction in ASSISTED. `budget pause` is the kill switch. Keep Workflow Studio alive without exposing chain-of-thought: before each stage or long operation emit `npx ai-engineering-loop activity heartbeat --run <runId> --phase <phase> --message <short factual summary>`. Existing node and gate commands publish their lifecycle automatically. Persist human questions with `interaction question` so they appear once in Studio.
|
|
11
11
|
|
|
12
|
-
Inherited invariants: read `core/grill-policy.md`, `policies/tdd-policy.md`, and `.ai-engineering-loop/glossary.md`. On `adapter_type: dot`, grill includes blast radius: lifecycle sketch and load `task-impact-inquiry`. If the user prompt has no numbered AC, draft the contract, list a short menu and wait. Chat agreement is not freeze. AC is a failure table. One red test per AC row. Do not freeze sunny-path-only. Write claimed-vs-reality.md. Do not spawn Devil's Advocate if the file is missing or any Claimed row lacks Reality. Report Spec and Standards separately. DA prompt: "at most 8 tool calls; read the diff file; skip css and generated files". Judge prompt: "at most 4 tool calls; ledger and contract only; skip css; do not re-review the whole diff". Never use `browser_subagent` as reviewer. `generate-workflow` reads workflow.md and lessons.md; missing maker_intern means INVOCATION_UNAVAILABLE. Do not type a model name.
|
|
12
|
+
Inherited invariants: read `core/grill-policy.md`, `policies/tdd-policy.md`, and `.ai-engineering-loop/glossary.md`. On `adapter_type: dot`, grill includes blast radius: lifecycle sketch and load `task-impact-inquiry`. If the user prompt has no numbered AC, draft the contract, list a short menu and wait. Chat agreement is not freeze. AC is a failure table. One red test per AC row. Do not freeze sunny-path-only. Write claimed-vs-reality.md. Do not spawn Devil's Advocate if the file is missing or any Claimed row lacks Reality. Report Spec and Standards separately. Use `reviewProfile` from runtime policy (default `lean`) for context packs; use iteration-delta review in iteration 2+. DA prompt: "at most 8 tool calls; read the diff file; skip css and generated files". Judge prompt: "at most 4 tool calls; ledger and contract only; skip css; do not re-review the whole diff". Never use `browser_subagent` as reviewer. `generate-workflow` reads workflow.md and lessons.md; missing maker_intern means INVOCATION_UNAVAILABLE. Do not type a model name.
|
|
13
13
|
|
|
14
|
-
Stage 1: if the user asks for ideas, list a short menu and wait; do not implement. Grill if the task is ambiguous (`core/grill-policy.md`). Chat agreement is not freeze. Publish the complete failure-table JSON through `npx ai-engineering-loop goal draft --run <runId> --file <repository-local-json> --revision <currentRevision>` so Studio shows the exact draft, then wait. After explicit human freeze, run `goal freeze --run <runId> --revision <reviewedRevision> --hash <reviewedHash> --by human`; `gate goal` remains legacy compatibility only. Do not start Maker before freeze. On business-logic change, and always on `adapter_type: dot`, grill includes blast radius: lifecycle sketch, four pillars (state, sibling, approval, queues), ASCII picture. Load `task-impact-inquiry` if present. Do not run a second interview. Passing unit tests are not isolation proof. Use `.ai-engineering-loop/glossary.md`. Name test seams. AC is a failure table (happy, empty/omit, boundary, sibling, error). One red test per AC row. Do not freeze sunny-path-only. Stages 2-4 branch on mode: REPORT_ONLY never edits repository files, writes RUN_DIR/report.json, runs `gate report`, and stops. ASSISTED uses normal Maker. UNATTENDED runs `sandbox create --run <runId>`, Maker only in the returned worktreePath, never edits the parent checkout, and runs `sandbox capture --run <runId>`. Read `maker_intern` from workflow.md (missing or none = parent is Maker). Do not type a model name. If intern is set, confirm it came from this host's list (`/models` if that command exists). If the list command is missing or the id is not on it, parent is Maker; say intern INVOCATION_UNAVAILABLE. Intern cannot skip verification, Devil's Advocate, or Judge. Maker: TDD at those seams (`policies/tdd-policy.md`): one red test per AC row. ASSISTED writes RUN_DIR/diff.patch; UNATTENDED capture writes it; then `gate maker`. Stage 5 writes current RUN_DIR/verification.json
|
|
14
|
+
Stage 1: if the user asks for ideas, list a short menu and wait; do not implement. Grill if the task is ambiguous (`core/grill-policy.md`). Chat agreement is not freeze. Publish the complete failure-table JSON through `npx ai-engineering-loop goal draft --run <runId> --file <repository-local-json> --revision <currentRevision>` so Studio shows the exact draft, then wait. After explicit human freeze, run `goal freeze --run <runId> --revision <reviewedRevision> --hash <reviewedHash> --by human`; `gate goal` remains legacy compatibility only. Do not start Maker before freeze. On business-logic change, and always on `adapter_type: dot`, grill includes blast radius: lifecycle sketch, four pillars (state, sibling, approval, queues), ASCII picture. Load `task-impact-inquiry` if present. Do not run a second interview. Passing unit tests are not isolation proof. Use `.ai-engineering-loop/glossary.md`. Name test seams. AC is a failure table (happy, empty/omit, boundary, sibling, error). One red test per AC row. Do not freeze sunny-path-only. Stages 2-4 branch on mode: REPORT_ONLY never edits repository files, writes RUN_DIR/report.json, runs `gate report`, and stops. ASSISTED uses normal Maker. UNATTENDED runs `sandbox create --run <runId>`, Maker only in the returned worktreePath, never edits the parent checkout, and runs `sandbox capture --run <runId>`. Read `maker_intern` from workflow.md (missing or none = parent is Maker). Do not type a model name. If intern is set, confirm it came from this host's list (`/models` if that command exists). If the list command is missing or the id is not on it, parent is Maker; say intern INVOCATION_UNAVAILABLE. Intern cannot skip verification, Devil's Advocate, or Judge. Maker: TDD at those seams (`policies/tdd-policy.md`): one red test per AC row. ASSISTED writes RUN_DIR/diff.patch; UNATTENDED capture writes it; then `gate maker`. Stage 5 writes current RUN_DIR/verification.json, runs `verification summarize --run <runId>`, then runs `gate verification`. Bugs: red repro first (`core/root-cause-analysis.md`). Mid-loop stop: `core/handoff-policy.md`.
|
|
15
15
|
|
|
16
16
|
If `invoke_subagent` (or Task) exists, spawn `devil-advocate` then `judge` as siblings. Wait for each child. Do not run them in the background. Use `general-purpose` only if the named type is rejected.
|
|
17
17
|
|
|
18
18
|
5. Before Devil's Advocate: write `git diff` to `.ai-engineering-loop/tasks/current.diff` and put `git diff --name-only` in the child prompt. Do not paste Maker rationale. Write `.ai-engineering-loop/tasks/claimed-vs-reality.md` (AC, Claimed, Reality from command log). Do not spawn Devil's Advocate if the file is missing or any Claimed row lacks Reality.
|
|
19
|
-
6. Build `context devil-advocate
|
|
20
|
-
7. Run `escalation --json`, resolve the required tier from the host catalog, build Judge context from only goal-contract.json, verification.json, and findings.json, check `budget status`, spawn Judge, wait, `budget record` actual usage, write verdict.json with modelTier, then `gate judge`.
|
|
19
|
+
6. Build `context diff-hunks --run <runId>` first, add `context devil-advocate` only for hunks that need full file context, check `budget status`, spawn DA with the pack path, wait, `budget record` actual usage, normalize RUN_DIR/findings.json with runId and diffHash, then `gate review`.
|
|
20
|
+
7. Run `escalation --json`, resolve the required tier from the host catalog, build Judge context from only goal-contract.json, verification-summary.json (or verification.json), and findings.json, check `budget status`, spawn Judge, wait, `budget record` actual usage, write verdict.json with modelTier, then `gate judge`.
|
|
21
21
|
|
|
22
22
|
If no subagent tool exists, run the same budgets in this session as CONTEXT_ISOLATION_ONLY. Do not claim independent agent execution.
|
|
23
23
|
|
|
@@ -11,11 +11,13 @@ You are the Devil's Advocate for the AI Engineering Loop. You are read-only. You
|
|
|
11
11
|
|
|
12
12
|
Finish in at most 8 tool calls, then emit the Finding Ledger. Do not explore the rest of the repo. Do not spawn children. Do not run git log.
|
|
13
13
|
|
|
14
|
+
Default to a lean review: spend detail where it protects correctness, not on prose. Treat the context pack as scarce. Prefer one precise quote over a long explanation.
|
|
15
|
+
|
|
14
16
|
## Input barrier
|
|
15
17
|
|
|
16
18
|
Use only:
|
|
17
19
|
|
|
18
|
-
1. The diff
|
|
20
|
+
1. The diff or smart diff-hunk pack path in the spawn prompt. Read that file first. Do not run git diff if a diff path was given.
|
|
19
21
|
2. Goal Contract path (if given).
|
|
20
22
|
3. Verification log path (if given).
|
|
21
23
|
4. conventions.md path (if given). At most one extra read.
|
|
@@ -25,6 +27,8 @@ Skip: `*.css`, files named like `*-css.ts` or `report-css.ts`, generated/vendor
|
|
|
25
27
|
|
|
26
28
|
Do not ask for Maker rationale.
|
|
27
29
|
|
|
30
|
+
On iteration 2 or later, review only the changed hunks since the previous Finding Ledger plus previously unresolved findings. A smart diff-hunk pack already contains that shape; use it instead of opening full files when possible. Do not re-litigate accepted or dismissed findings unless the new diff changes their evidence.
|
|
31
|
+
|
|
28
32
|
## Two axes (do not merge)
|
|
29
33
|
|
|
30
34
|
Report Spec and Standards as separate findings. Do not rerank one axis with the other. A change can pass Spec and fail Standards, or the reverse. Do not spawn children to split axes.
|
|
@@ -66,3 +70,11 @@ Return a Finding Ledger as a fenced JSON block and stop:
|
|
|
66
70
|
```
|
|
67
71
|
|
|
68
72
|
Rules: axis is spec or standards (default spec). hardConvention is boolean, default false. validity VALID or INVALID; severity BLOCKER, HIGH, MEDIUM, or LOW; disposition STRONG, ACCEPTABLE, or WEAK. VALID BLOCKER or HIGH must include concreteAlternativeDiff. Empty findings is allowed.
|
|
73
|
+
|
|
74
|
+
Lean output caps:
|
|
75
|
+
|
|
76
|
+
- Emit at most 5 findings unless a sixth or later finding is BLOCKER.
|
|
77
|
+
- Keep `failureScenario`, `reproduction`, and `evidence` under 500 characters each.
|
|
78
|
+
- Keep `concreteAlternativeDiff` under 12 lines; describe the rest in words.
|
|
79
|
+
- For MEDIUM/LOW findings, omit `concreteAlternativeDiff` unless the fix is a one-line replacement.
|
|
80
|
+
- Do not include broad summaries, praise, or restated policy text when findings are non-empty.
|
package/.claude/agents/judge.md
CHANGED
|
@@ -11,6 +11,8 @@ You are the Judge for the AI Engineering Loop. You do not write application code
|
|
|
11
11
|
|
|
12
12
|
Finish in at most 4 tool calls, then emit the verdict. Read the Finding Ledger and Goal Contract first. Open source only to fact-check a location the ledger already cited. Do not re-review the whole diff. Do not run git log. Do not spawn children. Skip `*.css`, `*report-css*`, and generated/vendor files.
|
|
13
13
|
|
|
14
|
+
Use a lean decision style. The Judge decides; it does not write an audit report. Prefer compact verdict evidence over narrative summaries.
|
|
15
|
+
|
|
14
16
|
## Inputs
|
|
15
17
|
|
|
16
18
|
Use only paths in the spawn prompt: Goal Contract, verification evidence, Finding Ledger. Ignore Maker optimism and reviewer tone. Disposition never overrides Validity plus Severity.
|
|
@@ -47,3 +49,11 @@ Return a fenced JSON block:
|
|
|
47
49
|
```
|
|
48
50
|
|
|
49
51
|
`verdict` must be exactly PASS, ITERATE, or ESCALATE. Do not edit source.
|
|
52
|
+
|
|
53
|
+
Lean output caps:
|
|
54
|
+
|
|
55
|
+
- Keep `reason` under 900 characters.
|
|
56
|
+
- Keep `action` under 700 characters.
|
|
57
|
+
- Put only blocking IDs in `blockingFindings`; do not restate every accepted LOW/MEDIUM note.
|
|
58
|
+
- Summarize accepted tradeoffs as IDs plus one short phrase each.
|
|
59
|
+
- Do not quote the Goal Contract or Finding Ledger unless needed to resolve a disputed blocker.
|
|
@@ -43,20 +43,20 @@ Run `npx ai-engineering-loop state --json` to get the current `runId` and use `.
|
|
|
43
43
|
|
|
44
44
|
Read `core/runtime-safety.md`. The ledger mode is immutable. Before every model or Task call, run `npx ai-engineering-loop budget status --run <runId>`; after it returns, record actual provider-reported usage with `budget record --input <n> --output <n> --model <id> --run <runId>`. Never invent actual token counts. If usage metadata is unavailable, UNATTENDED stops and escalates; ASSISTED discloses the gap and asks the human whether to continue. `budget pause` is an emergency kill switch.
|
|
45
45
|
|
|
46
|
-
Inherited invariants remain mandatory: read `core/grill-policy.md`, `policies/tdd-policy.md`, and `.ai-engineering-loop/glossary.md`. On `adapter_type: dot`, grill includes blast radius: lifecycle sketch and load `task-impact-inquiry`. If the user prompt has no numbered AC, draft the contract, list a short menu and wait. Chat agreement is not freeze. AC is a failure table. One red test per AC row. Do not freeze sunny-path-only. Write claimed-vs-reality.md. Do not spawn Devil's Advocate if the file is missing or any Claimed row lacks Reality. Report Spec and Standards separately. DA prompt includes "at most 8 tool calls; read the diff file; skip css and generated files". Judge prompt includes "at most 4 tool calls; ledger and contract only; skip css; do not re-review the whole diff".
|
|
46
|
+
Inherited invariants remain mandatory: read `core/grill-policy.md`, `policies/tdd-policy.md`, and `.ai-engineering-loop/glossary.md`. On `adapter_type: dot`, grill includes blast radius: lifecycle sketch and load `task-impact-inquiry`. If the user prompt has no numbered AC, draft the contract, list a short menu and wait. Chat agreement is not freeze. AC is a failure table. One red test per AC row. Do not freeze sunny-path-only. Write claimed-vs-reality.md. Do not spawn Devil's Advocate if the file is missing or any Claimed row lacks Reality. Report Spec and Standards separately. Use the runtime policy `reviewProfile` (default `lean`) for context packs; do not upgrade to `standard` or `thorough` unless the human asks or the budget gate requires escalation. DA prompt includes "at most 8 tool calls; read the diff file; skip css and generated files". Judge prompt includes "at most 4 tool calls; ledger and contract only; skip css; do not re-review the whole diff".
|
|
47
47
|
|
|
48
48
|
1. Stage 0: `npx ai-engineering-loop sync-hosts`, `status`, then `run [--mode report-only|assisted|unattended]` to create or resume the ledger. Run `state --json` and `budget status`. Read `.ai-engineering-loop/glossary.md`. Read `.ai-engineering-loop/lessons.md` and `.ai-engineering-loop/workflow.md` if they exist. Follow workflow.md hooks (`before_grill`, `after_freeze`, `after_pass`). Do not skip required gates even if workflow.md asks. If workflow.md is missing, run the default 8-stage loop. If sync-hosts copied files, tell the user a new session is needed; keep going with this session.
|
|
49
49
|
2. Stage 1: Goal Contract (`core/goal-contract.md`). If the user asks for ideas, list a short menu and wait; do not implement. If the task is ambiguous and the user can answer, grill first (`core/grill-policy.md`): design tree, recommended answers, do not ask look-up facts. On business-logic change, and always on `adapter_type: dot`, include blast radius. Chat agreement is not freeze. Emit `npx ai-engineering-loop activity goal-drafting --run <runId> --message <summary>` before drafting. Write the contract JSON to a repository-local temporary file, then publish it with `npx ai-engineering-loop goal draft --run <runId> --file <file> --revision <currentRevision>` so Workflow Studio sees the exact draft and its ready state. Wait. After the user explicitly freezes, use `npx ai-engineering-loop goal freeze --run <runId> --revision <reviewedRevision> --hash <reviewedHash> --by human`; do not edit production code unless it passes. `gate goal` remains the legacy compatibility command, not the preferred revision-bound path.
|
|
50
50
|
3. After Goal freeze, branch on mode. REPORT_ONLY: do not edit repository files; write `RUN_DIR/report.json` with summary and recommendations, then run `gate report` and stop. ASSISTED: run the normal Maker and later require human approval. UNATTENDED: run `sandbox create --run <runId>`, set Maker's working directory to the returned worktreePath, never edit the parent checkout, then run `sandbox capture --run <runId>`. For Maker, read `maker_intern` from workflow.md (missing or none = parent is Maker). Do not type a model name. Confirm it through `/models` when available; otherwise parent is Maker and report INVOCATION_UNAVAILABLE. Intern cannot skip later gates. Bugs: red repro first. Features: one red test per AC row. In ASSISTED write the exact review diff to `RUN_DIR/diff.patch`; UNATTENDED capture writes it. Then run `npx ai-engineering-loop gate maker`.
|
|
51
|
-
4. Stage 5: run commands from `.ai-engineering-loop/verification.md`. Write `RUN_DIR/verification.json` with current git revision, gated diff hash, command, execution identity, times, exit code, stdout, timeout status, and test counts. Vague "seems green" is invalid. Keep `.ai-engineering-loop/tasks/claimed-vs-reality.md`. Run `npx ai-engineering-loop gate verification`; do not spawn Devil's Advocate unless it passes.
|
|
51
|
+
4. Stage 5: run commands from `.ai-engineering-loop/verification.md`. Write `RUN_DIR/verification.json` with current git revision, gated diff hash, command, execution identity, times, exit code, stdout, timeout status, and test counts. Then run `npx ai-engineering-loop verification summarize --run <runId>` and pass the summary path to reviewers instead of raw logs whenever possible. Vague "seems green" is invalid. Keep `.ai-engineering-loop/tasks/claimed-vs-reality.md`. Run `npx ai-engineering-loop gate verification`; do not spawn Devil's Advocate unless it passes.
|
|
52
52
|
5. Write artifacts, then spawn. Before Devil's Advocate:
|
|
53
53
|
- Write `git diff` to a file (for example `.ai-engineering-loop/tasks/current.diff`).
|
|
54
54
|
- Write changed paths (`git diff --name-only`) into the Task prompt as a short list.
|
|
55
55
|
- Put those paths in the child prompt. Do not paste Maker rationale.
|
|
56
56
|
- Write `.ai-engineering-loop/tasks/claimed-vs-reality.md` if missing. Do not spawn DA without it.
|
|
57
57
|
- If stopping mid-loop, write `.ai-engineering-loop/tasks/handoff.md` (`core/handoff-policy.md`).
|
|
58
|
-
6. Stage 6: Build `npx ai-engineering-loop context devil-advocate <files...>`
|
|
59
|
-
7. Stage 7: Run `npx ai-engineering-loop escalation --json` before selecting Judge. Resolve the required tier through `/models`; if unavailable, escalate to the user. Build Judge context from only goal-contract.json, verification.json, and findings.json. Recheck `budget status`, spawn Judge as a sibling and wait, then immediately `budget record` actual usage. Normalize its output into `RUN_DIR/verdict.json`, include `modelTier`, then run `npx ai-engineering-loop gate judge`. A gate failure overrides optimistic prose.
|
|
58
|
+
6. Stage 6: Build `npx ai-engineering-loop context diff-hunks --run <runId>` plus `npx ai-engineering-loop context devil-advocate <files...>` only for files whose hunks are insufficient. In iteration 2+, prefer the diff-hunk pack's unresolved findings and new hunks instead of replaying the whole diff. Recheck `budget status`, then Task `subagent_type: devil-advocate`. Use `general-purpose` only if rejected. Wait and immediately `budget record` its actual usage. Normalize its ledger into `RUN_DIR/findings.json` with runId and diffHash, then run `npx ai-engineering-loop gate review`.
|
|
59
|
+
7. Stage 7: Run `npx ai-engineering-loop escalation --json` before selecting Judge. Resolve the required tier through `/models`; if unavailable, escalate to the user. Build Judge context from only goal-contract.json, verification-summary.json (or verification.json if the summary is missing), and findings.json. Recheck `budget status`, spawn Judge as a sibling and wait, then immediately `budget record` actual usage. Normalize its output into `RUN_DIR/verdict.json`, include `modelTier`, then run `npx ai-engineering-loop gate judge`. A gate failure overrides optimistic prose.
|
|
60
60
|
8. ITERATE with iteration under 3: fix in the parent, re-verify, spawn a **new** Devil's Advocate (do not resume the previous child).
|
|
61
61
|
9. After Judge PASS: run the `after_pass` hook, then Stage 8 from `.ai-engineering-loop/adapter.md`. PR/MR body includes `Steps to Reproduce & Testing (QA)`; bugfix placeholders must not stay empty. Preview remote payloads; do not expose context packs, usage ledgers, raw logs, or credentials. In ASSISTED, wait for explicit human approval and write `humanApproved: true`; never infer approval. Write `RUN_DIR/delivery.json` with destination and summary, then run `npx ai-engineering-loop gate delivery`. Only this gate completes the run.
|
|
62
62
|
|
|
@@ -36,20 +36,20 @@ Run `npx ai-engineering-loop state --json` to get the current runId and use `.ai
|
|
|
36
36
|
|
|
37
37
|
Read `core/runtime-safety.md`. Before every model or subagent call, run `budget status --run <runId>` and afterward run `budget record --input <actual> --output <actual> --model <id> --run <runId>` using provider-reported usage. Never invent usage. Missing usage stops UNATTENDED and requires human direction in ASSISTED. `budget pause` is the kill switch.
|
|
38
38
|
|
|
39
|
-
Inherited invariants remain mandatory: read `core/grill-policy.md`, `policies/tdd-policy.md`, and `.ai-engineering-loop/glossary.md`. On `adapter_type: dot`, grill includes blast radius: lifecycle sketch and load `task-impact-inquiry`. If the user prompt has no numbered AC, draft the contract, list a short menu and wait. Chat agreement is not freeze. AC is a failure table. One red test per AC row. Do not freeze sunny-path-only. Write claimed-vs-reality.md. Do not spawn Devil's Advocate if the file is missing or any Claimed row lacks Reality. Report Spec and Standards separately. DA prompt includes "at most 8 tool calls; read the diff file; skip css and generated files". Judge prompt includes "at most 4 tool calls; ledger and contract only; skip css; do not re-review the whole diff". `generate-workflow` still reads workflow.md and lessons.md; missing maker_intern means INVOCATION_UNAVAILABLE. Do not type a model name.
|
|
39
|
+
Inherited invariants remain mandatory: read `core/grill-policy.md`, `policies/tdd-policy.md`, and `.ai-engineering-loop/glossary.md`. On `adapter_type: dot`, grill includes blast radius: lifecycle sketch and load `task-impact-inquiry`. If the user prompt has no numbered AC, draft the contract, list a short menu and wait. Chat agreement is not freeze. AC is a failure table. One red test per AC row. Do not freeze sunny-path-only. Write claimed-vs-reality.md. Do not spawn Devil's Advocate if the file is missing or any Claimed row lacks Reality. Report Spec and Standards separately. Use the runtime policy `reviewProfile` (default `lean`) for context packs; do not upgrade to `standard` or `thorough` unless the human asks or the budget gate requires escalation. DA prompt includes "at most 8 tool calls; read the diff file; skip css and generated files". Judge prompt includes "at most 4 tool calls; ledger and contract only; skip css; do not re-review the whole diff". `generate-workflow` still reads workflow.md and lessons.md; missing maker_intern means INVOCATION_UNAVAILABLE. Do not type a model name.
|
|
40
40
|
|
|
41
41
|
1. Stage 0: sync hosts, check status, then run `npx ai-engineering-loop run [--mode report-only|assisted|unattended]` to create or resume the ledger. Run `state --json` and `budget status`. Read glossary, lessons, and workflow overlays. Required gates cannot be skipped.
|
|
42
42
|
2. Stage 1: grill ambiguous decisions and business blast radius first. Emit `npx ai-engineering-loop activity goal-drafting --run <runId> --message <summary>`. Write the Goal Contract JSON to a repository-local temporary file, then publish it with `npx ai-engineering-loop goal draft --run <runId> --file <file> --revision <currentRevision>` so Workflow Studio sees it and its ready state. Wait. Chat agreement is not freeze. After explicit human freeze, use `npx ai-engineering-loop goal freeze --run <runId> --revision <reviewedRevision> --hash <reviewedHash> --by human`; do not edit production code unless it passes. `gate goal` remains the legacy compatibility command, not the preferred revision-bound path.
|
|
43
43
|
3. After Goal freeze, branch on mode. REPORT_ONLY: never edit repository files; write `RUN_DIR/report.json`, run `gate report`, and stop. ASSISTED: use the normal Maker. UNATTENDED: run `sandbox create --run <runId>`, run Maker only in the returned worktreePath, never edit the parent checkout, then run `sandbox capture --run <runId>`. Resolve maker_intern only through the host catalog; otherwise parent is Maker. Intern cannot skip gates. Use red repro or TDD. ASSISTED writes `RUN_DIR/diff.patch`; UNATTENDED capture writes it. Then run `npx ai-engineering-loop gate maker`.
|
|
44
|
-
4. Stage 5: write `RUN_DIR/verification.json` with current git revision, gated diff hash, command, execution identity, times, exit code, stdout, timeout status, and test counts. Keep claimed-vs-reality.md. Run `npx ai-engineering-loop gate verification`; do not spawn DA unless it passes.
|
|
44
|
+
4. Stage 5: write `RUN_DIR/verification.json` with current git revision, gated diff hash, command, execution identity, times, exit code, stdout, timeout status, and test counts. Run `npx ai-engineering-loop verification summarize --run <runId>` and pass that summary path to reviewers instead of raw logs whenever possible. Keep claimed-vs-reality.md. Run `npx ai-engineering-loop gate verification`; do not spawn DA unless it passes.
|
|
45
45
|
5. Write artifacts, then spawn. Before Devil's Advocate:
|
|
46
46
|
- Write `git diff` to a file (for example `.ai-engineering-loop/tasks/current.diff`).
|
|
47
47
|
- Write changed paths (`git diff --name-only`) into the child prompt as a short list.
|
|
48
48
|
- Put those paths in the child prompt. Do not paste Maker rationale.
|
|
49
49
|
- Write `.ai-engineering-loop/tasks/claimed-vs-reality.md` if missing. Do not spawn DA without it.
|
|
50
50
|
- If stopping mid-loop, write `.ai-engineering-loop/tasks/handoff.md` (`core/handoff-policy.md`).
|
|
51
|
-
6. Stage 6: Build `npx ai-engineering-loop context devil-advocate <files
|
|
52
|
-
7. Stage 7: Run `npx ai-engineering-loop escalation --json`, resolve the required tier through the host catalog, or escalate to the user. Build Judge context from goal-contract.json, verification.json, and findings.json only. Check `budget status`, spawn Judge, wait, and `budget record` actual usage. Write `RUN_DIR/verdict.json` with modelTier, then run `npx ai-engineering-loop gate judge`.
|
|
51
|
+
6. Stage 6: Build `npx ai-engineering-loop context diff-hunks --run <runId>`, then `context devil-advocate <files...>` only when hunks are insufficient. Use unresolved findings and new hunks instead of replaying accepted old findings. Check `budget status`, spawn DA with the pack path, wait, and `budget record` actual usage. Normalize its output into `RUN_DIR/findings.json` with runId and diffHash, then run `npx ai-engineering-loop gate review`.
|
|
52
|
+
7. Stage 7: Run `npx ai-engineering-loop escalation --json`, resolve the required tier through the host catalog, or escalate to the user. Build Judge context from goal-contract.json, verification-summary.json (or verification.json if the summary is missing), and findings.json only. Check `budget status`, spawn Judge, wait, and `budget record` actual usage. Write `RUN_DIR/verdict.json` with modelTier, then run `npx ai-engineering-loop gate judge`.
|
|
53
53
|
8. ITERATE with iteration under 3: fix in the parent, re-verify, spawn a **new** Devil's Advocate.
|
|
54
54
|
9. After Judge PASS: run after_pass, then the configured Stage 8 adapter. PR/MR body includes `Steps to Reproduce & Testing (QA)`; bugfix placeholders must not stay empty. Preview remote payloads and never include context packs, usage ledgers, raw logs, or credentials. In ASSISTED, wait for explicit human approval and put `humanApproved: true` in `RUN_DIR/delivery.json`; never infer it. Then run `npx ai-engineering-loop gate delivery`.
|
|
55
55
|
|
|
@@ -12,11 +12,13 @@ You are the Devil's Advocate for the AI Engineering Loop. You are read-only. You
|
|
|
12
12
|
|
|
13
13
|
Finish in at most 8 tool calls, then emit the Finding Ledger. Do not explore the rest of the repo. Do not spawn children. Do not run git log.
|
|
14
14
|
|
|
15
|
+
Default to a lean review: spend detail where it protects correctness, not on prose. Treat the context pack as scarce. Prefer one precise quote over a long explanation.
|
|
16
|
+
|
|
15
17
|
## Input barrier
|
|
16
18
|
|
|
17
19
|
Use only:
|
|
18
20
|
|
|
19
|
-
1. The diff
|
|
21
|
+
1. The diff or smart diff-hunk pack path in the spawn prompt. Read that file first. Do not run git diff if a diff path was given.
|
|
20
22
|
2. Goal Contract path (if given).
|
|
21
23
|
3. Verification log path (if given).
|
|
22
24
|
4. conventions.md path (if given). At most one extra read.
|
|
@@ -26,6 +28,8 @@ Skip: `*.css`, files named like `*-css.ts` or `report-css.ts`, generated/vendor
|
|
|
26
28
|
|
|
27
29
|
Do not ask for Maker rationale.
|
|
28
30
|
|
|
31
|
+
On iteration 2 or later, review only the changed hunks since the previous Finding Ledger plus previously unresolved findings. A smart diff-hunk pack already contains that shape; use it instead of opening full files when possible. Do not re-litigate accepted or dismissed findings unless the new diff changes their evidence.
|
|
32
|
+
|
|
29
33
|
## Two axes (do not merge)
|
|
30
34
|
|
|
31
35
|
Report Spec and Standards as separate findings. Do not rerank one axis with the other. A change can pass Spec and fail Standards, or the reverse. Do not spawn children to split axes.
|
|
@@ -67,3 +71,11 @@ Return a Finding Ledger as a fenced JSON block and stop:
|
|
|
67
71
|
```
|
|
68
72
|
|
|
69
73
|
Rules: axis is spec or standards (default spec). hardConvention is boolean, default false. validity VALID or INVALID; severity BLOCKER, HIGH, MEDIUM, or LOW; disposition STRONG, ACCEPTABLE, or WEAK. VALID BLOCKER or HIGH must include concreteAlternativeDiff. Empty findings is allowed.
|
|
74
|
+
|
|
75
|
+
Lean output caps:
|
|
76
|
+
|
|
77
|
+
- Emit at most 5 findings unless a sixth or later finding is BLOCKER.
|
|
78
|
+
- Keep `failureScenario`, `reproduction`, and `evidence` under 500 characters each.
|
|
79
|
+
- Keep `concreteAlternativeDiff` under 12 lines; describe the rest in words.
|
|
80
|
+
- For MEDIUM/LOW findings, omit `concreteAlternativeDiff` unless the fix is a one-line replacement.
|
|
81
|
+
- Do not include broad summaries, praise, or restated policy text when findings are non-empty.
|
package/.grok/agents/judge.md
CHANGED
|
@@ -12,6 +12,8 @@ You are the Judge for the AI Engineering Loop. You do not write application code
|
|
|
12
12
|
|
|
13
13
|
Finish in at most 4 tool calls, then emit the verdict. Read the Finding Ledger and Goal Contract first. Open source only to fact-check a location the ledger already cited. Do not re-review the whole diff. Do not run git log. Do not spawn children. Skip `*.css`, `*report-css*`, and generated/vendor files.
|
|
14
14
|
|
|
15
|
+
Use a lean decision style. The Judge decides; it does not write an audit report. Prefer compact verdict evidence over narrative summaries.
|
|
16
|
+
|
|
15
17
|
## Inputs
|
|
16
18
|
|
|
17
19
|
Use only paths in the spawn prompt: Goal Contract, verification evidence, Finding Ledger. Ignore Maker optimism and reviewer tone. Disposition never overrides Validity plus Severity.
|
|
@@ -48,3 +50,11 @@ Return a fenced JSON block:
|
|
|
48
50
|
```
|
|
49
51
|
|
|
50
52
|
`verdict` must be exactly PASS, ITERATE, or ESCALATE. Do not edit source.
|
|
53
|
+
|
|
54
|
+
Lean output caps:
|
|
55
|
+
|
|
56
|
+
- Keep `reason` under 900 characters.
|
|
57
|
+
- Keep `action` under 700 characters.
|
|
58
|
+
- Put only blocking IDs in `blockingFindings`; do not restate every accepted LOW/MEDIUM note.
|
|
59
|
+
- Summarize accepted tradeoffs as IDs plus one short phrase each.
|
|
60
|
+
- Do not quote the Goal Contract or Finding Ledger unless needed to resolve a disputed blocker.
|
|
@@ -54,20 +54,20 @@ Run `npx ai-engineering-loop state --json` to get the current `runId` and use `.
|
|
|
54
54
|
|
|
55
55
|
Read `core/runtime-safety.md`. Before every model or subagent call, run `budget status --run <runId>`; immediately afterward run `budget record --input <actual> --output <actual> --model <id> --run <runId>` from provider-reported metadata. Never invent usage. Missing usage stops UNATTENDED and requires human direction in ASSISTED. `budget pause` is the kill switch.
|
|
56
56
|
|
|
57
|
-
Inherited invariants remain mandatory: read `core/grill-policy.md`, `policies/tdd-policy.md`, and `.ai-engineering-loop/glossary.md`. On `adapter_type: dot`, grill includes blast radius: lifecycle sketch and load `task-impact-inquiry`. If the user prompt has no numbered AC, draft the contract, list a short menu and wait. Chat agreement is not freeze. AC is a failure table. One red test per AC row. Do not freeze sunny-path-only. Write claimed-vs-reality.md. Do not spawn Devil's Advocate if the file is missing or any Claimed row lacks Reality. Report Spec and Standards separately. DA prompt includes "at most 8 tool calls; read the diff file; skip css and generated files". Judge prompt includes "at most 4 tool calls; ledger and contract only; skip css; do not re-review the whole diff". `generate-workflow` still reads workflow.md and lessons.md; missing maker_intern means INVOCATION_UNAVAILABLE. Do not type a model name.
|
|
57
|
+
Inherited invariants remain mandatory: read `core/grill-policy.md`, `policies/tdd-policy.md`, and `.ai-engineering-loop/glossary.md`. On `adapter_type: dot`, grill includes blast radius: lifecycle sketch and load `task-impact-inquiry`. If the user prompt has no numbered AC, draft the contract, list a short menu and wait. Chat agreement is not freeze. AC is a failure table. One red test per AC row. Do not freeze sunny-path-only. Write claimed-vs-reality.md. Do not spawn Devil's Advocate if the file is missing or any Claimed row lacks Reality. Report Spec and Standards separately. Use the runtime policy `reviewProfile` (default `lean`) for context packs; do not upgrade to `standard` or `thorough` unless the human asks or the budget gate requires escalation. DA prompt includes "at most 8 tool calls; read the diff file; skip css and generated files". Judge prompt includes "at most 4 tool calls; ledger and contract only; skip css; do not re-review the whole diff". `generate-workflow` still reads workflow.md and lessons.md; missing maker_intern means INVOCATION_UNAVAILABLE. Do not type a model name.
|
|
58
58
|
|
|
59
59
|
1. Stage 0: `npx ai-engineering-loop sync-hosts`, `status`, then `run [--mode report-only|assisted|unattended]` to create or resume the ledger. Run `state --json` and `budget status`. Read glossary, lessons, and workflow overlays. Required gates cannot be skipped.
|
|
60
60
|
2. Stage 1: grill ambiguous decisions and business blast radius first. Emit `npx ai-engineering-loop activity goal-drafting --run <runId> --message <summary>`. Write the Goal Contract JSON to a repository-local temporary file, then publish it with `npx ai-engineering-loop goal draft --run <runId> --file <file> --revision <currentRevision>` so Workflow Studio sees it and its ready state. Wait. Chat agreement is not freeze. After explicit human freeze, use `npx ai-engineering-loop goal freeze --run <runId> --revision <reviewedRevision> --hash <reviewedHash> --by human`; do not edit production code unless it passes. `gate goal` remains the legacy compatibility command, not the preferred revision-bound path.
|
|
61
61
|
3. After Goal freeze, branch on mode. REPORT_ONLY: never edit repository files; write `RUN_DIR/report.json`, run `gate report`, and stop. ASSISTED: use the normal Maker. UNATTENDED: run `sandbox create --run <runId>`, run Maker only in the returned worktreePath, never edit the parent checkout, then run `sandbox capture --run <runId>`. Resolve `maker_intern` only through `grok models`; unknown means parent Maker. Intern cannot skip gates or spawn DA/Judge. Use red repro or TDD. ASSISTED writes `RUN_DIR/diff.patch`; UNATTENDED capture writes it. Then run `npx ai-engineering-loop gate maker`.
|
|
62
|
-
4. Stage 5: run verification and write `RUN_DIR/verification.json` with current git revision, gated diff hash, command, execution identity, times, exit code, stdout, timeout status, and test counts. Keep claimed-vs-reality.md. Run `npx ai-engineering-loop gate verification`; do not spawn DA unless it passes.
|
|
62
|
+
4. Stage 5: run verification and write `RUN_DIR/verification.json` with current git revision, gated diff hash, command, execution identity, times, exit code, stdout, timeout status, and test counts. Run `npx ai-engineering-loop verification summarize --run <runId>` and pass that summary path to reviewers instead of raw logs whenever possible. Keep claimed-vs-reality.md. Run `npx ai-engineering-loop gate verification`; do not spawn DA unless it passes.
|
|
63
63
|
5. Write artifacts to disk so children do not need parent chat:
|
|
64
64
|
- Goal Contract path
|
|
65
65
|
- `git diff` written to `.ai-engineering-loop/tasks/current.diff`
|
|
66
66
|
- verification log (file)
|
|
67
67
|
- `.ai-engineering-loop/tasks/claimed-vs-reality.md` (do not spawn DA without it)
|
|
68
68
|
- If stopping mid-loop, `.ai-engineering-loop/tasks/handoff.md` (`core/handoff-policy.md`)
|
|
69
|
-
6. Stage 6: Build `npx ai-engineering-loop context devil-advocate <files
|
|
70
|
-
7. Stage 7: Run `npx ai-engineering-loop escalation --json`; resolve `STRONG_MODEL` through `grok models`, or escalate to the user if unavailable. Build Judge context from only goal-contract.json, verification.json, and findings.json. Check `budget status`, spawn a fresh sibling Judge and wait, then `budget record` actual usage. Normalize `RUN_DIR/verdict.json` with modelTier, then run `npx ai-engineering-loop gate judge`.
|
|
69
|
+
6. Stage 6: Build `npx ai-engineering-loop context diff-hunks --run <runId>`, then `context devil-advocate <files...>` only when hunks are insufficient. Use unresolved findings and new hunks instead of replaying accepted old findings. Check `budget status`, spawn Devil's Advocate with the pack path, `background: false`, `capability_mode: "execute"`, and no `resume_from`; wait and `budget record` actual usage. Normalize its output into `RUN_DIR/findings.json` with runId and diffHash, then run `npx ai-engineering-loop gate review`.
|
|
70
|
+
7. Stage 7: Run `npx ai-engineering-loop escalation --json`; resolve `STRONG_MODEL` through `grok models`, or escalate to the user if unavailable. Build Judge context from only goal-contract.json, verification-summary.json (or verification.json if the summary is missing), and findings.json. Check `budget status`, spawn a fresh sibling Judge and wait, then `budget record` actual usage. Normalize `RUN_DIR/verdict.json` with modelTier, then run `npx ai-engineering-loop gate judge`.
|
|
71
71
|
8. If Judge says `ITERATE` and iteration < 3, Maker fixes in the parent, re-verify, spawn a **fresh** DA (new spawn, no resume).
|
|
72
72
|
9. After Judge PASS: run `after_pass`, then the configured Stage 8 adapter. PR/MR body includes `Steps to Reproduce & Testing (QA)`; bugfix placeholders must not stay empty. Preview remote payloads and never include context packs, usage ledgers, raw logs, or credentials. In ASSISTED, wait for explicit human approval and put `humanApproved: true` in `RUN_DIR/delivery.json`; never infer it. Then run `npx ai-engineering-loop gate delivery`.
|
|
73
73
|
|
package/README.md
CHANGED
|
@@ -163,6 +163,21 @@ Choose the level of autonomy that feels right:
|
|
|
163
163
|
|
|
164
164
|
Assisted mode is the default. Unattended mode stays off until you deliberately enable it.
|
|
165
165
|
|
|
166
|
+
Token-heavy hosts are protected by a lean review profile by default. Lean mode keeps the same Goal, verification, Devil’s Advocate, Judge, and delivery gates, but sends smaller context packs and asks reviewers for compact, finding-first output instead of long audit prose. Upgrade only when the task needs it:
|
|
167
|
+
|
|
168
|
+
```bash
|
|
169
|
+
npx ai-engineering-loop policy set --review-profile standard
|
|
170
|
+
npx ai-engineering-loop policy set --review-profile thorough
|
|
171
|
+
```
|
|
172
|
+
|
|
173
|
+
Smart context keeps review path-first instead of transcript-heavy. It can build a private source summary index and a hunk-only diff pack that includes unresolved findings, so reviewers start from changed hunks and open full files only when necessary:
|
|
174
|
+
|
|
175
|
+
```bash
|
|
176
|
+
npx ai-engineering-loop context index
|
|
177
|
+
npx ai-engineering-loop context diff-hunks --run <run-id>
|
|
178
|
+
npx ai-engineering-loop verification summarize --run <run-id>
|
|
179
|
+
```
|
|
180
|
+
|
|
166
181
|
## For people who want the controls
|
|
167
182
|
|
|
168
183
|
The friendly workflow sits on top of a complete local CLI. You can inspect recipes, budgets, evidence, nodes, gates, policies, and handoffs whenever you need to.
|
|
@@ -187,6 +202,13 @@ npx ai-engineering-loop recipe simulate default --mode assisted
|
|
|
187
202
|
# Inspect the active Run
|
|
188
203
|
npx ai-engineering-loop state --json
|
|
189
204
|
npx ai-engineering-loop node status
|
|
205
|
+
npx ai-engineering-loop budget status --run <run-id>
|
|
206
|
+
|
|
207
|
+
# Token profile
|
|
208
|
+
npx ai-engineering-loop policy set --review-profile lean
|
|
209
|
+
npx ai-engineering-loop context diff-hunks --run <run-id>
|
|
210
|
+
npx ai-engineering-loop verification summarize --run <run-id>
|
|
211
|
+
npx ai-engineering-loop context devil-advocate --profile thorough <files...>
|
|
190
212
|
|
|
191
213
|
# Emergency stop and resume
|
|
192
214
|
npx ai-engineering-loop budget pause
|
package/README.npm.md
CHANGED
|
@@ -33,6 +33,9 @@ npx ai-engineering-loop run --mode report-only "audit this repository"
|
|
|
33
33
|
# Inspect spend controls; ASSISTED is the default
|
|
34
34
|
npx ai-engineering-loop policy show
|
|
35
35
|
npx ai-engineering-loop budget status
|
|
36
|
+
npx ai-engineering-loop policy set --review-profile lean
|
|
37
|
+
npx ai-engineering-loop context diff-hunks --run <run-id>
|
|
38
|
+
npx ai-engineering-loop verification summarize --run <run-id>
|
|
36
39
|
|
|
37
40
|
# Author and observe deterministic workflow recipes locally
|
|
38
41
|
npx ai-engineering-loop studio
|
|
@@ -53,7 +56,7 @@ npx ai-engineering-loop sync-hosts
|
|
|
53
56
|
3. **Multi-Agent Triad**: Coordinates **Maker** (surgical diffs & tests), **Devil's Advocate** (independent adversarial review), and **Judge** (impartial evaluation & PASS certification).
|
|
54
57
|
4. **Context Impact Assessment**: Evaluates completed tasks (`NONE`, `TARGETED`, `MAJOR`) to keep project context fresh without expensive whole-repo re-analysis.
|
|
55
58
|
5. **Grok CLI host**: Native `spawn_subagent` for Devil's Advocate and Judge (`TRUE_INDEPENDENT_AGENT`). Disabled by `GROK_SUBAGENTS=0`.
|
|
56
|
-
6. **Production runtime controls**: `REPORT_ONLY`, human-approved `ASSISTED`, opt-in `UNATTENDED`, actual token ledgers, a persistent kill switch, and locked disposable Maker worktrees.
|
|
59
|
+
6. **Production runtime controls**: `REPORT_ONLY`, human-approved `ASSISTED`, opt-in `UNATTENDED`, actual token ledgers, lean/standard/thorough review profiles, smart hunk-only context packs, a persistent kill switch, and locked disposable Maker worktrees.
|
|
57
60
|
7. **Local Workflow Studio**: A localhost-only authenticated DAG canvas with animated live stage state, token telemetry, explicit approvals, decision memory, atomic recipe installation, and redacted handoff export. It does not execute models or shell commands in the browser.
|
|
58
61
|
|
|
59
62
|
---
|
|
@@ -4,11 +4,13 @@ You are the Devil's Advocate for the AI Engineering Loop. You are read-only. You
|
|
|
4
4
|
|
|
5
5
|
Finish in at most 8 tool calls, then emit the Finding Ledger. Do not explore the rest of the repo. Do not spawn children. Do not run git log.
|
|
6
6
|
|
|
7
|
+
Default to a lean review: spend detail where it protects correctness, not on prose. Treat the context pack as scarce. Prefer one precise quote over a long explanation.
|
|
8
|
+
|
|
7
9
|
## Input barrier
|
|
8
10
|
|
|
9
11
|
Use only:
|
|
10
12
|
|
|
11
|
-
1. The diff
|
|
13
|
+
1. The diff or smart diff-hunk pack path in the spawn prompt. Read that file first. Do not run git diff if a diff path was given.
|
|
12
14
|
2. Goal Contract path (if given).
|
|
13
15
|
3. Verification log path (if given).
|
|
14
16
|
4. conventions.md path (if given). At most one extra read.
|
|
@@ -18,6 +20,8 @@ Skip: `*.css`, files named like `*-css.ts` or `report-css.ts`, generated/vendor
|
|
|
18
20
|
|
|
19
21
|
Do not ask for Maker rationale.
|
|
20
22
|
|
|
23
|
+
On iteration 2 or later, review only the changed hunks since the previous Finding Ledger plus previously unresolved findings. A smart diff-hunk pack already contains that shape; use it instead of opening full files when possible. Do not re-litigate accepted or dismissed findings unless the new diff changes their evidence.
|
|
24
|
+
|
|
21
25
|
## Two axes (do not merge)
|
|
22
26
|
|
|
23
27
|
Report Spec and Standards as separate findings. Do not rerank one axis with the other. A change can pass Spec and fail Standards, or the reverse. Do not spawn children to split axes.
|
|
@@ -59,3 +63,11 @@ Return a Finding Ledger as a fenced JSON block and stop:
|
|
|
59
63
|
```
|
|
60
64
|
|
|
61
65
|
Rules: axis is spec or standards (default spec). hardConvention is boolean, default false. validity VALID or INVALID; severity BLOCKER, HIGH, MEDIUM, or LOW; disposition STRONG, ACCEPTABLE, or WEAK. VALID BLOCKER or HIGH must include concreteAlternativeDiff. Empty findings is allowed.
|
|
66
|
+
|
|
67
|
+
Lean output caps:
|
|
68
|
+
|
|
69
|
+
- Emit at most 5 findings unless a sixth or later finding is BLOCKER.
|
|
70
|
+
- Keep `failureScenario`, `reproduction`, and `evidence` under 500 characters each.
|
|
71
|
+
- Keep `concreteAlternativeDiff` under 12 lines; describe the rest in words.
|
|
72
|
+
- For MEDIUM/LOW findings, omit `concreteAlternativeDiff` unless the fix is a one-line replacement.
|
|
73
|
+
- Do not include broad summaries, praise, or restated policy text when findings are non-empty.
|
|
@@ -4,6 +4,8 @@ You are the Judge for the AI Engineering Loop. You do not write application code
|
|
|
4
4
|
|
|
5
5
|
Finish in at most 4 tool calls, then emit the verdict. Read the Finding Ledger and Goal Contract first. Open source only to fact-check a location the ledger already cited. Do not re-review the whole diff. Do not run git log. Do not spawn children. Skip `*.css`, `*report-css*`, and generated/vendor files.
|
|
6
6
|
|
|
7
|
+
Use a lean decision style. The Judge decides; it does not write an audit report. Prefer compact verdict evidence over narrative summaries.
|
|
8
|
+
|
|
7
9
|
## Inputs
|
|
8
10
|
|
|
9
11
|
Use only paths in the spawn prompt: Goal Contract, verification evidence, Finding Ledger. Ignore Maker optimism and reviewer tone. Disposition never overrides Validity plus Severity.
|
|
@@ -40,3 +42,11 @@ Return a fenced JSON block:
|
|
|
40
42
|
```
|
|
41
43
|
|
|
42
44
|
`verdict` must be exactly PASS, ITERATE, or ESCALATE. Do not edit source.
|
|
45
|
+
|
|
46
|
+
Lean output caps:
|
|
47
|
+
|
|
48
|
+
- Keep `reason` under 900 characters.
|
|
49
|
+
- Keep `action` under 700 characters.
|
|
50
|
+
- Put only blocking IDs in `blockingFindings`; do not restate every accepted LOW/MEDIUM note.
|
|
51
|
+
- Summarize accepted tradeoffs as IDs plus one short phrase each.
|
|
52
|
+
- Do not quote the Goal Contract or Finding Ledger unless needed to resolve a disputed blocker.
|