ai-engineering-loop 1.0.17 → 1.0.19
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/judge.md +1 -1
- package/.agents/workflows/ai-engineering-loop.md +5 -3
- package/.claude/agents/judge.md +1 -1
- package/.claude/skills/ai-engineering-loop/SKILL.md +7 -4
- package/.gemini/skills/ai-engineering-loop/SKILL.md +7 -4
- package/.grok/agents/judge.md +1 -1
- package/.grok/skills/ai-engineering-loop/SKILL.md +7 -4
- package/README.md +6 -3
- package/adapters/generate-workflow/SKILL.md +17 -4
- package/agents/maker.md +1 -0
- package/agents/shared/judge.body.md +1 -1
- package/bin/ai-engineering-loop.js +3 -3
- package/core/goal-contract.md +1 -0
- package/core/grill-policy.md +11 -0
- package/core/handoff-policy.md +1 -0
- package/core/repo-config-schema.md +4 -3
- package/core/verification-loop.md +23 -0
- package/lib/claimed-vs-reality.js +37 -0
- package/lib/generate-workflow.js +35 -4
- package/lib/intern-catalog.js +53 -0
- package/package.json +1 -1
- package/templates/repo-config/workflow.md +5 -0
- package/tests/claimed-vs-reality.test.js +27 -0
- package/tests/generate-workflow.test.js +73 -2
- package/tests/intern-catalog.test.js +72 -0
- package/tests/skill-host-compat.test.js +58 -0
package/.agents/judge.md
CHANGED
|
@@ -16,7 +16,7 @@ Use only paths in the spawn prompt: Goal Contract, verification evidence, Findin
|
|
|
16
16
|
|
|
17
17
|
AC source is the Goal Contract **file** only. Chat `setuju` and the parent transcript are not AC. Do not PASS because the parent agreed in chat. `CONTEXT_ISOLATION_ONLY` does not relax this.
|
|
18
18
|
|
|
19
|
-
ITERATE if verification is missing, vague, non-zero, proves a different artifact than the AC names, is only a source grep, or is happy-path-only while the Goal Contract lists empty, boundary, sibling, or error rows.
|
|
19
|
+
ITERATE if verification is missing, vague, non-zero, proves a different artifact than the AC names, is only a source grep, or is happy-path-only while the Goal Contract lists empty, boundary, sibling, or error rows. ITERATE if claimed-vs-reality.md is missing or a Claimed row has empty Reality.
|
|
20
20
|
|
|
21
21
|
## Axes
|
|
22
22
|
|
|
@@ -5,14 +5,16 @@ description: Run the AI Engineering Loop on Antigravity (init, status, refresh,
|
|
|
5
5
|
|
|
6
6
|
Follow `policies/review-budget.md`. Parent is Maker plus orchestrator. Do not use `browser_subagent`.
|
|
7
7
|
|
|
8
|
-
|
|
8
|
+
Compact map (not a second OS): Specify (stages 0-1), Make (stages 2-4), Review (stages 5-7), Deliver (stage 8). Keep the 8-stage numbers. Do not skip Goal Contract, verification, Devil's Advocate, or Judge.
|
|
9
|
+
|
|
10
|
+
Stage 1: if the user asks for ideas, list a short menu and wait; do not implement. Grill if the task is ambiguous (`core/grill-policy.md`), then freeze the Goal Contract file. Chat agreement is not freeze. If the user prompt has no numbered AC, draft `.ai-engineering-loop/tasks/goal-contract.md` with a failure table, show it, wait for freeze. Do not start Maker. On business-logic change, and always on `adapter_type: dot`, grill includes blast radius: lifecycle sketch, four pillars (state, sibling, approval, queues), ASCII picture. Load `task-impact-inquiry` if present. Do not run a second interview. Passing unit tests are not isolation proof. Use `.ai-engineering-loop/glossary.md`. Name test seams. AC is a failure table (happy, empty/omit, boundary, sibling, error). One red test per AC row. Do not freeze sunny-path-only. Stages 2-4: Read `maker_intern` from workflow.md (missing or none = parent is Maker). Do not type a model name. If intern is set, confirm it came from this host's list (`/models` if that command exists). If the list command is missing or the id is not on it, parent is Maker; say intern INVOCATION_UNAVAILABLE. Intern cannot skip verification, Devil's Advocate, or Judge. Maker: TDD at those seams (`policies/tdd-policy.md`): one red test per AC row. Bugs: red repro first (`core/root-cause-analysis.md`). Mid-loop stop: `core/handoff-policy.md`.
|
|
9
11
|
|
|
10
12
|
If `invoke_subagent` (or Task) exists, spawn `devil-advocate` then `judge` as siblings. Wait for each child. Do not run them in the background. Use `general-purpose` only if the named type is rejected.
|
|
11
13
|
|
|
12
|
-
5. Before Devil's Advocate: write `git diff` to `.ai-engineering-loop/tasks/current.diff` and put `git diff --name-only` in the child prompt. Do not paste Maker rationale.
|
|
14
|
+
5. Before Devil's Advocate: write `git diff` to `.ai-engineering-loop/tasks/current.diff` and put `git diff --name-only` in the child prompt. Do not paste Maker rationale. Write `.ai-engineering-loop/tasks/claimed-vs-reality.md` (AC, Claimed, Reality from command log). Do not spawn Devil's Advocate if the file is missing or any Claimed row lacks Reality.
|
|
13
15
|
6. Devil's Advocate prompt: diff file path, name-only list, Goal Contract path, verification log path, conventions.md path, and "at most 8 tool calls; read the diff file; skip css and generated files". Report Spec and Standards axes separately.
|
|
14
16
|
7. Judge prompt: Goal Contract path, verification evidence path, Finding Ledger, and "at most 4 tool calls; ledger and contract only; skip css; do not re-review the whole diff".
|
|
15
17
|
|
|
16
18
|
If no subagent tool exists, run the same budgets in this session as CONTEXT_ISOLATION_ONLY. Do not claim independent agent execution.
|
|
17
19
|
|
|
18
|
-
Init/status/refresh/sync-hosts/generate-adapter/generate-workflow: `npx ai-engineering-loop <command>`. Stage 0: `npx ai-engineering-loop sync-hosts` then `npx ai-engineering-loop status`. Read `.ai-engineering-loop/glossary.md`. Read `.ai-engineering-loop/lessons.md` and `.ai-engineering-loop/workflow.md` if they exist. Follow workflow.md hooks (`before_grill`, `after_freeze`, `after_pass`). Do not skip Goal Contract, verification, Devil's Advocate, or Judge even if workflow.md asks. If workflow.md is missing, run the default 8-stage loop (no extra hooks). If sync-hosts copied files, tell the user a new session is needed for updated skill text; keep going with this session. After Judge PASS: run the `after_pass` hook if it is not `none`. Stage 8: load `adapters/<adapter_type>/`. If adapter.md is missing, run `generate-adapter` (grill). Do not invent a company pipeline. Confirmed process lessons go to `lessons.md`, not a chat dump.
|
|
20
|
+
Init/status/refresh/sync-hosts/generate-adapter/generate-workflow: `npx ai-engineering-loop <command>`. generate-workflow grills Q1-Q6. Q6 Maker intern: pick from the host catalog (`grok models` or `/models` if that command exists); do not type a model name. Stage 0: `npx ai-engineering-loop sync-hosts` then `npx ai-engineering-loop status`. Read `.ai-engineering-loop/glossary.md`. Read `.ai-engineering-loop/lessons.md` and `.ai-engineering-loop/workflow.md` if they exist. Follow workflow.md hooks (`before_grill`, `after_freeze`, `after_pass`). Do not skip Goal Contract, verification, Devil's Advocate, or Judge even if workflow.md asks. If workflow.md is missing, run the default 8-stage loop (no extra hooks). If sync-hosts copied files, tell the user a new session is needed for updated skill text; keep going with this session. After Judge PASS: run the `after_pass` hook if it is not `none`. Stage 8: load `adapters/<adapter_type>/`. If adapter.md is missing, run `generate-adapter` (grill). Do not invent a company pipeline. Confirmed process lessons go to `lessons.md`, not a chat dump.
|
package/.claude/agents/judge.md
CHANGED
|
@@ -17,7 +17,7 @@ Use only paths in the spawn prompt: Goal Contract, verification evidence, Findin
|
|
|
17
17
|
|
|
18
18
|
AC source is the Goal Contract **file** only. Chat `setuju` and the parent transcript are not AC. Do not PASS because the parent agreed in chat. `CONTEXT_ISOLATION_ONLY` does not relax this.
|
|
19
19
|
|
|
20
|
-
ITERATE if verification is missing, vague, non-zero, proves a different artifact than the AC names, is only a source grep, or is happy-path-only while the Goal Contract lists empty, boundary, sibling, or error rows.
|
|
20
|
+
ITERATE if verification is missing, vague, non-zero, proves a different artifact than the AC names, is only a source grep, or is happy-path-only while the Goal Contract lists empty, boundary, sibling, or error rows. ITERATE if claimed-vs-reality.md is missing or a Claimed row has empty Reality.
|
|
21
21
|
|
|
22
22
|
## Axes
|
|
23
23
|
|
|
@@ -26,21 +26,24 @@ Claude Code talks to strict proxies (including Kiro). Follow this exactly:
|
|
|
26
26
|
|
|
27
27
|
- `init` / `status` / `refresh` / `sync-hosts` / `generate-adapter` / `generate-workflow`: run `npx ai-engineering-loop <command>` in the repo. Do not commit unless asked.
|
|
28
28
|
- `generate-adapter`: load skill `generate-adapter` if present. Grill Q1-Q5. Then write `.ai-engineering-loop/adapter.md`. Do not start Maker.
|
|
29
|
-
- `generate-workflow`: load skill `generate-workflow` if present. Grill Q1-
|
|
29
|
+
- `generate-workflow`: load skill `generate-workflow` if present. Grill Q1-Q6. Q6 Maker intern: if `/models` exists in this session, use it and show numbered options; else only `none`. Do not type a model name. Then write `.ai-engineering-loop/workflow.md` and empty `lessons.md` if missing. Do not start Maker.
|
|
30
30
|
- Any other argument: full loop for that task.
|
|
31
31
|
|
|
32
32
|
## Loop
|
|
33
33
|
|
|
34
|
+
Compact map (not a second OS): Specify (stages 0-1), Make (stages 2-4), Review (stages 5-7), Deliver (stage 8). Keep the 8-stage numbers. Do not skip Goal Contract, verification, Devil's Advocate, or Judge.
|
|
35
|
+
|
|
34
36
|
Parent session is Maker plus orchestrator. Spawn Devil's Advocate and Judge as **siblings**, not nested.
|
|
35
37
|
|
|
36
38
|
1. Stage 0: `npx ai-engineering-loop sync-hosts` then `npx ai-engineering-loop status` (init or refresh if missing or stale). Read `.ai-engineering-loop/glossary.md`. Read `.ai-engineering-loop/lessons.md` and `.ai-engineering-loop/workflow.md` if they exist. Follow workflow.md hooks (`before_grill`, `after_freeze`, `after_pass`). Do not skip Goal Contract, verification, Devil's Advocate, or Judge even if workflow.md asks. If workflow.md is missing, run the default 8-stage loop (no extra hooks). If sync-hosts copied files, tell the user a new session is needed for updated skill text; keep going with this session.
|
|
37
|
-
2. Stage 1: Goal Contract (`core/goal-contract.md`). If the user asks for ideas, list a short menu and wait; do not implement. If the task is ambiguous and the user can answer, grill first (`core/grill-policy.md`): design tree, recommended answers, do not ask look-up facts. Skip grill if the contract is already frozen or the user waived it. On business-logic change, and always on `adapter_type: dot`, grill includes blast radius: lifecycle sketch, four pillars (state, sibling, approval, queues), ASCII picture. Load `task-impact-inquiry` if present. Do not run a second interview. Passing unit tests are not isolation proof. Chat agreement is not freeze: every user-visible decision must be a numbered AC in the Goal Contract file. Freeze before any production edit. Name test seams. Use glossary terms. AC is a failure table (happy, empty/omit, boundary, sibling, error). One red test per AC row. Do not freeze sunny-path-only.
|
|
38
|
-
3. Stages 2-4: Maker
|
|
39
|
-
4. Stage 5: run commands from `.ai-engineering-loop/verification.md`. Keep command, exit code, stdout, test counts. Vague "seems green" is invalid.
|
|
39
|
+
2. Stage 1: Goal Contract (`core/goal-contract.md`). If the user asks for ideas, list a short menu and wait; do not implement. If the task is ambiguous and the user can answer, grill first (`core/grill-policy.md`): design tree, recommended answers, do not ask look-up facts. Skip grill if the contract is already frozen or the user waived it. On business-logic change, and always on `adapter_type: dot`, grill includes blast radius: lifecycle sketch, four pillars (state, sibling, approval, queues), ASCII picture. Load `task-impact-inquiry` if present. Do not run a second interview. Passing unit tests are not isolation proof. Chat agreement is not freeze: every user-visible decision must be a numbered AC in the Goal Contract file. Freeze before any production edit. Name test seams. Use glossary terms. AC is a failure table (happy, empty/omit, boundary, sibling, error). One red test per AC row. Do not freeze sunny-path-only. If the user prompt has no numbered AC, draft `.ai-engineering-loop/tasks/goal-contract.md` with a failure table, show it, wait for freeze. Do not start Maker.
|
|
40
|
+
3. Stages 2-4: Read `maker_intern` from workflow.md (missing or none = parent is Maker). Do not type a model name. If intern is set, confirm it came from this host's list (`/models` if that command exists). If the list command is missing or the id is not on it, parent is Maker; say intern INVOCATION_UNAVAILABLE. Do not add a model key or HTTP client. Intern cannot skip verification, Devil's Advocate, or Judge. Bugs: red repro first (`core/root-cause-analysis.md`). Features: TDD at named seams (`policies/tdd-policy.md`): one red test per AC row. Surgical diff.
|
|
41
|
+
4. Stage 5: run commands from `.ai-engineering-loop/verification.md`. Keep command, exit code, stdout, test counts. Vague "seems green" is invalid. Write `.ai-engineering-loop/tasks/claimed-vs-reality.md` (AC, Claimed, Reality from command log). Do not spawn Devil's Advocate if the file is missing or any Claimed row lacks Reality.
|
|
40
42
|
5. Write artifacts, then spawn. Before Devil's Advocate:
|
|
41
43
|
- Write `git diff` to a file (for example `.ai-engineering-loop/tasks/current.diff`).
|
|
42
44
|
- Write changed paths (`git diff --name-only`) into the Task prompt as a short list.
|
|
43
45
|
- Put those paths in the child prompt. Do not paste Maker rationale.
|
|
46
|
+
- Write `.ai-engineering-loop/tasks/claimed-vs-reality.md` if missing. Do not spawn DA without it.
|
|
44
47
|
- If stopping mid-loop, write `.ai-engineering-loop/tasks/handoff.md` (`core/handoff-policy.md`).
|
|
45
48
|
6. Stage 6: Task `subagent_type: devil-advocate`. Use `general-purpose` only if that type is rejected. If the Task schema includes `run_in_background`, set it false. Then wait for Task to return. Do not start Judge or more Maker work until the Finding Ledger is back. Prompt: diff file path, name-only list, Goal Contract path, verification log path, conventions.md path, and "at most 8 tool calls; read the diff file; skip css and generated files". Report Spec and Standards axes separately.
|
|
46
49
|
7. Stage 7: Task `subagent_type: judge` the same way (wait, no background). Use `general-purpose` only if `judge` is rejected. Prompt: Goal Contract path, verification evidence path, Finding Ledger, and "at most 4 tool calls; ledger and contract only; skip css; do not re-review the whole diff".
|
|
@@ -21,19 +21,22 @@ Canonical mode ids: `TRUE_INDEPENDENT_AGENT`, `ISOLATED_AGENT_INSTANCE`, `FRESH_
|
|
|
21
21
|
|
|
22
22
|
- `init` / `status` / `refresh` / `sync-hosts` / `generate-adapter` / `generate-workflow`: run `npx ai-engineering-loop <command>` in the repo. Do not commit unless asked.
|
|
23
23
|
- `generate-adapter`: load skill `generate-adapter` if present. Grill Q1-Q5. Then write `.ai-engineering-loop/adapter.md`. Do not start Maker.
|
|
24
|
-
- `generate-workflow`: load skill `generate-workflow` if present. Grill Q1-
|
|
24
|
+
- `generate-workflow`: load skill `generate-workflow` if present. Grill Q1-Q6. Q6 Maker intern: if `/models` exists in this session, use it and show numbered options; else only `none`. Do not type a model name. Then write `.ai-engineering-loop/workflow.md` and empty `lessons.md` if missing. Do not start Maker.
|
|
25
25
|
- Any other argument: full loop for that task.
|
|
26
26
|
|
|
27
27
|
## Loop
|
|
28
28
|
|
|
29
|
+
Compact map (not a second OS): Specify (stages 0-1), Make (stages 2-4), Review (stages 5-7), Deliver (stage 8). Keep the 8-stage numbers. Do not skip Goal Contract, verification, Devil's Advocate, or Judge.
|
|
30
|
+
|
|
29
31
|
1. Stage 0: `npx ai-engineering-loop sync-hosts` then `npx ai-engineering-loop status` (init or refresh if missing or stale). Read `.ai-engineering-loop/glossary.md`. Read `.ai-engineering-loop/lessons.md` and `.ai-engineering-loop/workflow.md` if they exist. Follow workflow.md hooks (`before_grill`, `after_freeze`, `after_pass`). Do not skip Goal Contract, verification, Devil's Advocate, or Judge even if workflow.md asks. If workflow.md is missing, run the default 8-stage loop (no extra hooks). If sync-hosts copied files, tell the user a new session is needed for updated skill text; keep going with this session.
|
|
30
|
-
2. Stage 1: Goal Contract (`core/goal-contract.md`). If the user asks for ideas, list a short menu and wait; do not implement. If the task is ambiguous and the user can answer, grill first (`core/grill-policy.md`): design tree, recommended answers, do not ask look-up facts. Skip grill if the contract is already frozen or the user waived it. On business-logic change, and always on `adapter_type: dot`, grill includes blast radius: lifecycle sketch, four pillars (state, sibling, approval, queues), ASCII picture. Load `task-impact-inquiry` if present. Do not run a second interview. Passing unit tests are not isolation proof. Chat agreement is not freeze: every user-visible decision must be a numbered AC in the Goal Contract file. Freeze before any production edit. Name test seams. Use glossary terms. AC is a failure table (happy, empty/omit, boundary, sibling, error). One red test per AC row. Do not freeze sunny-path-only.
|
|
31
|
-
3. Stages 2-4: Maker
|
|
32
|
-
4. Stage 5: run commands from `.ai-engineering-loop/verification.md`. Keep command, exit code, stdout, test counts. Vague "seems green" is invalid.
|
|
32
|
+
2. Stage 1: Goal Contract (`core/goal-contract.md`). If the user asks for ideas, list a short menu and wait; do not implement. If the task is ambiguous and the user can answer, grill first (`core/grill-policy.md`): design tree, recommended answers, do not ask look-up facts. Skip grill if the contract is already frozen or the user waived it. On business-logic change, and always on `adapter_type: dot`, grill includes blast radius: lifecycle sketch, four pillars (state, sibling, approval, queues), ASCII picture. Load `task-impact-inquiry` if present. Do not run a second interview. Passing unit tests are not isolation proof. Chat agreement is not freeze: every user-visible decision must be a numbered AC in the Goal Contract file. Freeze before any production edit. Name test seams. Use glossary terms. AC is a failure table (happy, empty/omit, boundary, sibling, error). One red test per AC row. Do not freeze sunny-path-only. If the user prompt has no numbered AC, draft `.ai-engineering-loop/tasks/goal-contract.md` with a failure table, show it, wait for freeze. Do not start Maker.
|
|
33
|
+
3. Stages 2-4: Read `maker_intern` from workflow.md (missing or none = parent is Maker). Do not type a model name. If intern is set, confirm it came from this host's list (`/models` if that command exists). If the list command is missing or the id is not on it, parent is Maker; say intern INVOCATION_UNAVAILABLE. Do not add HTTP clients or API keys. Intern cannot skip verification, Devil's Advocate, or Judge. After intern returns, parent runs Stage 5. Bugs: red repro first (`core/root-cause-analysis.md`). Features: TDD at named seams (`policies/tdd-policy.md`): one red test per AC row. Surgical diff.
|
|
34
|
+
4. Stage 5: run commands from `.ai-engineering-loop/verification.md`. Keep command, exit code, stdout, test counts. Vague "seems green" is invalid. Write `.ai-engineering-loop/tasks/claimed-vs-reality.md` (AC, Claimed, Reality from command log). Do not spawn Devil's Advocate if the file is missing or any Claimed row lacks Reality.
|
|
33
35
|
5. Write artifacts, then spawn. Before Devil's Advocate:
|
|
34
36
|
- Write `git diff` to a file (for example `.ai-engineering-loop/tasks/current.diff`).
|
|
35
37
|
- Write changed paths (`git diff --name-only`) into the child prompt as a short list.
|
|
36
38
|
- Put those paths in the child prompt. Do not paste Maker rationale.
|
|
39
|
+
- Write `.ai-engineering-loop/tasks/claimed-vs-reality.md` if missing. Do not spawn DA without it.
|
|
37
40
|
- If stopping mid-loop, write `.ai-engineering-loop/tasks/handoff.md` (`core/handoff-policy.md`).
|
|
38
41
|
6. Stage 6: spawn `devil-advocate`. Wait. Prompt: diff file path, name-only list, Goal Contract path, verification log path, conventions.md path, and "at most 8 tool calls; read the diff file; skip css and generated files". Report Spec and Standards axes separately.
|
|
39
42
|
7. Stage 7: spawn `judge` the same way (wait, no background). Prompt: Goal Contract path, verification evidence path, Finding Ledger, and "at most 4 tool calls; ledger and contract only; skip css; do not re-review the whole diff".
|
package/.grok/agents/judge.md
CHANGED
|
@@ -18,7 +18,7 @@ Use only paths in the spawn prompt: Goal Contract, verification evidence, Findin
|
|
|
18
18
|
|
|
19
19
|
AC source is the Goal Contract **file** only. Chat `setuju` and the parent transcript are not AC. Do not PASS because the parent agreed in chat. `CONTEXT_ISOLATION_ONLY` does not relax this.
|
|
20
20
|
|
|
21
|
-
ITERATE if verification is missing, vague, non-zero, proves a different artifact than the AC names, is only a source grep, or is happy-path-only while the Goal Contract lists empty, boundary, sibling, or error rows.
|
|
21
|
+
ITERATE if verification is missing, vague, non-zero, proves a different artifact than the AC names, is only a source grep, or is happy-path-only while the Goal Contract lists empty, boundary, sibling, or error rows. ITERATE if claimed-vs-reality.md is missing or a Claimed row has empty Reality.
|
|
22
22
|
|
|
23
23
|
## Axes
|
|
24
24
|
|
|
@@ -40,18 +40,21 @@ Optional fallback if `devil-advocate` / `judge` types are not registered: `subag
|
|
|
40
40
|
|
|
41
41
|
### `/ai-engineering-loop init|status|refresh|sync-hosts|generate-adapter|generate-workflow`
|
|
42
42
|
|
|
43
|
-
Run `npx ai-engineering-loop <command>` in the target repo. Do not commit unless the user asks. `sync-hosts` copies this package's skills/agents/commands into `~/.claude`, `~/.grok`, `~/.gemini`, and `~/.agents` for hosts that already exist. `generate-adapter`: load skill `generate-adapter` if present. Grill Q1-Q5. Then write `.ai-engineering-loop/adapter.md`. Do not start Maker. `generate-workflow`: load skill `generate-workflow` if present. Grill Q1-
|
|
43
|
+
Run `npx ai-engineering-loop <command>` in the target repo. Do not commit unless the user asks. `sync-hosts` copies this package's skills/agents/commands into `~/.claude`, `~/.grok`, `~/.gemini`, and `~/.agents` for hosts that already exist. `generate-adapter`: load skill `generate-adapter` if present. Grill Q1-Q5. Then write `.ai-engineering-loop/adapter.md`. Do not start Maker. `generate-workflow`: load skill `generate-workflow` if present. Grill Q1-Q6. Q6 Maker intern: run `grok models` (or `/models` if that slash command exists here), drop grok-imagine-* and video, show numbered options plus none. Do not type a model name. Then write `.ai-engineering-loop/workflow.md` and empty `lessons.md` if missing. Do not start Maker.
|
|
44
44
|
|
|
45
45
|
### `/ai-engineering-loop [task]`
|
|
46
46
|
|
|
47
|
+
Compact map (not a second OS): Specify (stages 0-1), Make (stages 2-4), Review (stages 5-7), Deliver (stage 8). Keep the 8-stage numbers. Do not skip Goal Contract, verification, Devil's Advocate, or Judge.
|
|
48
|
+
|
|
47
49
|
1. Stage 0: `npx ai-engineering-loop sync-hosts` then `npx ai-engineering-loop status` (init/refresh if missing or stale). Read `.ai-engineering-loop/glossary.md`. Read `.ai-engineering-loop/lessons.md` and `.ai-engineering-loop/workflow.md` if they exist. Follow workflow.md hooks (`before_grill`, `after_freeze`, `after_pass`). Do not skip Goal Contract, verification, Devil's Advocate, or Judge even if workflow.md asks. If workflow.md is missing, run the default 8-stage loop (no extra hooks). If sync-hosts copied files, tell the user a new session is needed for updated skill text; keep going with this session.
|
|
48
|
-
2. Stage 1: Goal Contract (`core/goal-contract.md`). If the user asks for ideas, list a short menu and wait; do not implement. If the task is ambiguous and the user can answer, grill first (`core/grill-policy.md`): design tree, recommended answers, do not ask look-up facts. Skip grill if the contract is already frozen or the user waived it. On business-logic change, and always on `adapter_type: dot`, grill includes blast radius: lifecycle sketch, four pillars (state, sibling, approval, queues), ASCII picture. Load `task-impact-inquiry` if present. Do not run a second interview. Passing unit tests are not isolation proof. Chat agreement is not freeze: every user-visible decision must be a numbered AC in the Goal Contract file. Freeze before any production edit. Name test seams. Use glossary terms. AC is a failure table (happy, empty/omit, boundary, sibling, error). One red test per AC row. Do not freeze sunny-path-only.
|
|
49
|
-
3. Stages 2–4: Maker
|
|
50
|
-
4. Stage 5: run verification from `.ai-engineering-loop/verification.md`. Keep command, exit code, stdout, test counts. Vague "seems green" is invalid.
|
|
50
|
+
2. Stage 1: Goal Contract (`core/goal-contract.md`). If the user asks for ideas, list a short menu and wait; do not implement. If the task is ambiguous and the user can answer, grill first (`core/grill-policy.md`): design tree, recommended answers, do not ask look-up facts. Skip grill if the contract is already frozen or the user waived it. On business-logic change, and always on `adapter_type: dot`, grill includes blast radius: lifecycle sketch, four pillars (state, sibling, approval, queues), ASCII picture. Load `task-impact-inquiry` if present. Do not run a second interview. Passing unit tests are not isolation proof. Chat agreement is not freeze: every user-visible decision must be a numbered AC in the Goal Contract file. Freeze before any production edit. Name test seams. Use glossary terms. AC is a failure table (happy, empty/omit, boundary, sibling, error). One red test per AC row. Do not freeze sunny-path-only. If the user prompt has no numbered AC, draft `.ai-engineering-loop/tasks/goal-contract.md` with a failure table, show it, wait for freeze. Do not start Maker.
|
|
51
|
+
3. Stages 2–4: Read `maker_intern` from workflow.md (missing or none = parent is Maker). Do not type a model name. If a label is set, run `grok models` and confirm the id is in the coding list (not grok-imagine-*). If missing, parent is Maker; say intern INVOCATION_UNAVAILABLE. Otherwise write `.ai-engineering-loop/tasks/maker-brief.md` and spawn that intern as a sibling (`background: false`, wait). Pass a model key only if that key exists on the spawn schema. Do not invent keys, HTTP clients, or custom endpoints. Intern must not spawn DA or Judge (depth 1). After intern returns, parent runs Stage 5 then DA/Judge. Intern cannot skip verification, Devil's Advocate, or Judge. Bugs: red repro first (`core/root-cause-analysis.md`). Features: TDD at named seams (`policies/tdd-policy.md`): one red test per AC row. Surgical diff.
|
|
52
|
+
4. Stage 5: run verification from `.ai-engineering-loop/verification.md`. Keep command, exit code, stdout, test counts. Vague "seems green" is invalid. Write `.ai-engineering-loop/tasks/claimed-vs-reality.md` (AC, Claimed, Reality from command log). Do not spawn Devil's Advocate if the file is missing or any Claimed row lacks Reality.
|
|
51
53
|
5. Write artifacts to disk so children do not need parent chat:
|
|
52
54
|
- Goal Contract path
|
|
53
55
|
- `git diff` written to `.ai-engineering-loop/tasks/current.diff`
|
|
54
56
|
- verification log (file)
|
|
57
|
+
- `.ai-engineering-loop/tasks/claimed-vs-reality.md` (do not spawn DA without it)
|
|
55
58
|
- If stopping mid-loop, `.ai-engineering-loop/tasks/handoff.md` (`core/handoff-policy.md`)
|
|
56
59
|
6. Stage 6: `spawn_subagent` Devil's Advocate. `background: false`. `capability_mode: "execute"`. Do **not** pass `resume_from`. Wait for the child. Prompt: diff file path, name-only file list, Goal Contract path, verification log path, conventions.md path, plus "at most 8 tool calls; read the diff file; skip css and generated files". Report Spec and Standards axes separately.
|
|
57
60
|
7. Stage 7: `spawn_subagent` Judge the same way (`background: false`, wait). Use `general-purpose` only if `judge` is rejected. Prompt: Goal Contract path, verification evidence path, Finding Ledger, and "at most 4 tool calls; ledger and contract only; skip css; do not re-review the whole diff".
|
package/README.md
CHANGED
|
@@ -6,7 +6,7 @@
|
|
|
6
6
|
[](https://opensource.org/licenses/MIT)
|
|
7
7
|
[](https://github.com/egagofur/ai-engineering-loop/pulls)
|
|
8
8
|
[](https://github.com/egagofur/ai-engineering-loop)
|
|
9
|
-
[](https://github.com/egagofur/ai-engineering-loop/releases)
|
|
10
10
|
|
|
11
11
|
**A Reusable, Framework-Agnostic AI Engineering Operating System for Autonomous Coding Agents**
|
|
12
12
|
|
|
@@ -64,12 +64,15 @@ The 8-stage loop stays one OS. These techniques sit **inside** existing stages (
|
|
|
64
64
|
|
|
65
65
|
| Stage | Technique | Spec |
|
|
66
66
|
|---|---|---|
|
|
67
|
-
| 1 | Grill until the design-tree frontier is empty, then freeze the Goal Contract. Idea requests: menu, wait, then grill the pick. Chat `setuju` is not freeze; user-visible decisions must be numbered AC in the contract file. AC is a **failure table** (happy, empty/omit, boundary, sibling, error), not sunny path only. Business-logic change: **blast radius** (lifecycle sketch, four pillars, ASCII picture) via `task-impact-inquiry` inside grill — not a second loop. Passing unit tests are not isolation proof. | `core/grill-policy.md` |
|
|
67
|
+
| 1 | Grill until the design-tree frontier is empty, then freeze the Goal Contract. Idea requests: menu, wait, then grill the pick. Chat `setuju` is not freeze; user-visible decisions must be numbered AC in the contract file. AC is a **failure table** (happy, empty/omit, boundary, sibling, error), not sunny path only. **Sloppy prompt** (no numbered AC): draft the Goal Contract file, show it, wait for freeze — do not Maker. Business-logic change: **blast radius** (lifecycle sketch, four pillars, ASCII picture) via `task-impact-inquiry` inside grill — not a second loop. Passing unit tests are not isolation proof. | `core/grill-policy.md` |
|
|
68
68
|
| 1 | Ubiquitous language in `.ai-engineering-loop/glossary.md`; load-bearing choices in `.ai-engineering-loop/adrs/` | `core/repo-config-schema.md` |
|
|
69
69
|
| 2 | Bugs: red repro → minimise → hypothesise → instrument → fix | `core/root-cause-analysis.md` |
|
|
70
|
+
| 4 | Optional **Maker intern**: overlay `maker_intern`. Default `none`. Pick from the host catalog (`grok models` or `/models` if that command exists; numbered options). Do not type a model name. No custom endpoint. Imagine/video models are not intern picks. Unknown id → parent Maker (`INVOCATION_UNAVAILABLE`). Devil's Advocate and Judge stay on the parent. | `.ai-engineering-loop/workflow.md` |
|
|
70
71
|
| 4–5 | Red-green at named **seams**; failure table (not happy path only); coverage is a map; no grep/tautology | `policies/tdd-policy.md` |
|
|
72
|
+
| 5 | **Claimed vs Reality** table before DA. Missing file or empty Reality blocks Devil's Advocate. "Seems green" is not Reality. | `core/verification-loop.md` |
|
|
71
73
|
| 6–7 | Spec vs Standards reported separately. Standards BLOCKER/HIGH iterate only when `hardConvention` is true | `policies/finding-policy.md` |
|
|
72
74
|
| any | Mid-loop stop writes `.ai-engineering-loop/tasks/handoff.md` | `core/handoff-policy.md` |
|
|
75
|
+
| any | **Compact map** (host guidance, not a second OS): Specify (stages 0-1), Make (stages 2-4), Review (stages 5-7), Deliver (stage 8). Keep the 8-stage numbers. Do not skip Goal Contract, verification, Devil's Advocate, or Judge. | host skills |
|
|
73
76
|
|
|
74
77
|
`init` now requires `glossary.md` and `adrs/README.md`. Repair fills missing files only; a filled glossary is never overwritten.
|
|
75
78
|
|
|
@@ -342,7 +345,7 @@ ai-engineering-loop/
|
|
|
342
345
|
├── adapters/ # Pluggable Stage 8 delivery (not a second OS)
|
|
343
346
|
│ ├── README.md # Catalog: standard, github, gitlab, dot
|
|
344
347
|
│ ├── generate-adapter/SKILL.md # Grill Q1-Q5, then write adapter.md
|
|
345
|
-
│ ├── generate-workflow/SKILL.md # Grill Q1-
|
|
348
|
+
│ ├── generate-workflow/SKILL.md # Grill Q1-Q6, then write workflow.md + lessons.md
|
|
346
349
|
│ ├── standard/ # Generic git delivery (any forge)
|
|
347
350
|
│ ├── github/ # gh pr / GitHub Issues
|
|
348
351
|
│ ├── gitlab/ # glab mr (not the DOT pipeline)
|
|
@@ -16,7 +16,7 @@ Do not edit production code. Do not create a feature branch for the product.
|
|
|
16
16
|
Do not write `.ai-engineering-loop/workflow.md` until:
|
|
17
17
|
|
|
18
18
|
1. Look-up is done (existing overlay, lessons, adapter).
|
|
19
|
-
2. Q1–
|
|
19
|
+
2. Q1–Q6 are on screen with a recommended answer.
|
|
20
20
|
3. The user has answered or waived.
|
|
21
21
|
|
|
22
22
|
Never skip Goal Contract, verification, Devil's Advocate, or Judge. If the user asks to skip those, refuse and keep them required.
|
|
@@ -53,7 +53,7 @@ Cite what already exists. Do not ask whether those files are present.
|
|
|
53
53
|
|
|
54
54
|
### 2. Grill (wait)
|
|
55
55
|
|
|
56
|
-
Ask all
|
|
56
|
+
Ask all six in one round. Number them. Give Recommended. Wait.
|
|
57
57
|
|
|
58
58
|
```text
|
|
59
59
|
Q1 - Extra step before grill: none | name it
|
|
@@ -70,16 +70,27 @@ Recommended: keep AEL default
|
|
|
70
70
|
|
|
71
71
|
Q5 - Skip generate-adapter when adapter.md already exists: yes | no
|
|
72
72
|
Recommended: yes
|
|
73
|
+
|
|
74
|
+
Q6 - Maker intern: pick from the host catalog. Do not type a model name.
|
|
75
|
+
Look up models with this host's list command:
|
|
76
|
+
- Grok CLI: `grok models`
|
|
77
|
+
- Many agent CLIs (Codex and others): `/models` if that slash command exists in this session
|
|
78
|
+
Drop image/video/imagine ids. Show numbered options plus 0) none.
|
|
79
|
+
If no list command exists here: only 0) none.
|
|
80
|
+
Recommended: none
|
|
73
81
|
```
|
|
74
82
|
|
|
75
83
|
If they name a tool or step, write that name into the matching hook. Do not invent Figma/Jira/Slack unless they said so.
|
|
76
84
|
|
|
85
|
+
Q6 writes `maker_intern` from the numbered pick only. Do not type a model name. Do not use a custom endpoint. Do not paste API keys or URLs. Default none = parent is Maker.
|
|
86
|
+
|
|
77
87
|
### 3. Write
|
|
78
88
|
|
|
79
89
|
Write `.ai-engineering-loop/workflow.md` (create `.ai-engineering-loop/` if needed).
|
|
80
90
|
|
|
81
91
|
- Required stages listed and not skippable
|
|
82
92
|
- Hooks from Q1–Q3 (`none` if they picked none)
|
|
93
|
+
- **maker_intern** from Q6 (`none` if they picked none)
|
|
83
94
|
- Optional skips only `blast_radius` and/or `generate_adapter`
|
|
84
95
|
- No tokens, no coworker handles, no internal ticket URLs
|
|
85
96
|
|
|
@@ -91,7 +102,7 @@ Headless alternative the user can run:
|
|
|
91
102
|
npx ai-engineering-loop generate-workflow --write
|
|
92
103
|
```
|
|
93
104
|
|
|
94
|
-
`--write` skips this interview and writes default hooks (`none`)
|
|
105
|
+
`--write` skips this interview and writes default hooks (`none`) and `maker_intern: none`. Do not pass `--write` yourself unless they waived the grill.
|
|
95
106
|
|
|
96
107
|
### 4. Lessons during later product work
|
|
97
108
|
|
|
@@ -112,7 +123,7 @@ Print the paths written. Do not start Maker. Do not open a PR.
|
|
|
112
123
|
```text
|
|
113
124
|
Generate workflow overlay
|
|
114
125
|
Existing: <workflow yes/no> <lessons yes/no>
|
|
115
|
-
Questions: Q1..
|
|
126
|
+
Questions: Q1..Q6 with Recommended
|
|
116
127
|
Wrote: .ai-engineering-loop/workflow.md
|
|
117
128
|
Lessons: .ai-engineering-loop/lessons.md (created or left)
|
|
118
129
|
Next: /ai-engineering-loop for product work. Stage 0 reads these files.
|
|
@@ -126,3 +137,5 @@ Next: /ai-engineering-loop for product work. Stage 0 reads these files.
|
|
|
126
137
|
- Dumping chat into lessons
|
|
127
138
|
- Mermaid or LaTeX
|
|
128
139
|
- Running the 8-stage product loop inside this skill
|
|
140
|
+
- Typing a model name for Q6
|
|
141
|
+
- Custom intern endpoints
|
package/agents/maker.md
CHANGED
|
@@ -42,6 +42,7 @@ flowchart LR
|
|
|
42
42
|
2. **CANNOT Alter Acceptance Criteria**: The Maker Agent must never modify the Goal Contract's acceptance criteria to bypass failing tests.
|
|
43
43
|
3. **CANNOT Suppress Verification Failures**: The Maker Agent is strictly prohibited from adding `@ts-ignore`, `eslint-disable`, mock overrides that mask bugs, or skipping broken tests to achieve green results.
|
|
44
44
|
4. **CANNOT Make Unsolicited Renovations**: Even if surrounding legacy code is poorly written, the Maker must not refactor untouched modules unless explicitly included in the Goal Contract.
|
|
45
|
+
5. **CANNOT Skip Review**: A Maker intern (optional `maker_intern` overlay label) cannot skip verification, Devil's Advocate, or Judge. Intern is a cheaper Stage 4 builder only. Default `none` means the parent is Maker. The label is host-native and not locked to one vendor. AEL does not store intern API keys.
|
|
45
46
|
|
|
46
47
|
---
|
|
47
48
|
|
|
@@ -10,7 +10,7 @@ Use only paths in the spawn prompt: Goal Contract, verification evidence, Findin
|
|
|
10
10
|
|
|
11
11
|
AC source is the Goal Contract **file** only. Chat `setuju` and the parent transcript are not AC. Do not PASS because the parent agreed in chat. `CONTEXT_ISOLATION_ONLY` does not relax this.
|
|
12
12
|
|
|
13
|
-
ITERATE if verification is missing, vague, non-zero, proves a different artifact than the AC names, is only a source grep, or is happy-path-only while the Goal Contract lists empty, boundary, sibling, or error rows.
|
|
13
|
+
ITERATE if verification is missing, vague, non-zero, proves a different artifact than the AC names, is only a source grep, or is happy-path-only while the Goal Contract lists empty, boundary, sibling, or error rows. ITERATE if claimed-vs-reality.md is missing or a Claimed row has empty Reality.
|
|
14
14
|
|
|
15
15
|
## Axes
|
|
16
16
|
|
|
@@ -34,7 +34,7 @@ const {
|
|
|
34
34
|
lessonsMarkdown
|
|
35
35
|
} = require('../lib/generate-workflow.js');
|
|
36
36
|
|
|
37
|
-
const VERSION = '1.0.
|
|
37
|
+
const VERSION = '1.0.19';
|
|
38
38
|
const CWD = process.cwd();
|
|
39
39
|
const CONTEXT_DIR = path.join(CWD, '.ai-engineering-loop');
|
|
40
40
|
|
|
@@ -288,7 +288,7 @@ function generateContextFiles(rootDir, discovery, trigger = 'init', impact = 'IN
|
|
|
288
288
|
|
|
289
289
|
// 0. metadata.json (Baseline)
|
|
290
290
|
const metadataJson = {
|
|
291
|
-
contextVersion: '1.0.
|
|
291
|
+
contextVersion: '1.0.19',
|
|
292
292
|
generatedAt: new Date().toISOString(),
|
|
293
293
|
repositoryRevision: currentRevision,
|
|
294
294
|
projectProfile: discovery.profile,
|
|
@@ -797,7 +797,7 @@ Commands:
|
|
|
797
797
|
--dry-run print the plan without writing
|
|
798
798
|
generate-adapter Print detected forge and grill protocol for Stage 8
|
|
799
799
|
--type standard|github|gitlab|dot write .ai-engineering-loop/adapter.md (skip grill)
|
|
800
|
-
generate-workflow Print grill protocol for a loop overlay (hooks + lessons.md)
|
|
800
|
+
generate-workflow Print grill protocol for a loop overlay (hooks + intern + lessons.md)
|
|
801
801
|
--write write .ai-engineering-loop/workflow.md and empty lessons.md (skip grill)
|
|
802
802
|
|
|
803
803
|
Options:
|
package/core/goal-contract.md
CHANGED
|
@@ -112,3 +112,4 @@ Every single item listed under `Acceptance Criteria` must map to at least one co
|
|
|
112
112
|
- **The Chat Contract**: Treating "setuju" in the parent transcript as AC. The file is the contract.
|
|
113
113
|
- **The Wrong Artifact**: Tests pass on a `/tmp` build while the sample the user opens omits the feature.
|
|
114
114
|
- **The Happy-Path Contract**: Only the sunny case is an AC. Coverage % is not a substitute for a failure table.
|
|
115
|
+
- **The Sloppy Ship**: Implementing from a vague prompt without a drafted failure table in the Goal Contract file.
|
package/core/grill-policy.md
CHANGED
|
@@ -45,6 +45,17 @@ Before Maker starts:
|
|
|
45
45
|
|
|
46
46
|
If a decision was agreed in chat and is missing from the file, the contract is not frozen. Do not start Maker.
|
|
47
47
|
|
|
48
|
+
## Sloppy / underspecified prompts
|
|
49
|
+
|
|
50
|
+
A prompt with no numbered AC (for example "fix login", "bikin aja bagus") is not permission to Maker.
|
|
51
|
+
|
|
52
|
+
1. Look up facts. Draft `.ai-engineering-loop/tasks/goal-contract.md` with a **failure table** (happy, empty/omit, boundary, sibling, error).
|
|
53
|
+
2. Show the AC list. Recommended answers already filled.
|
|
54
|
+
3. Wait for an explicit freeze of that file. Chat `setuju` is **not** a freeze.
|
|
55
|
+
4. Do not start Maker. Do not treat the draft as frozen.
|
|
56
|
+
|
|
57
|
+
If the prompt already has numbered AC, or a Goal Contract for this task is already frozen on disk, Do not invent a second draft.
|
|
58
|
+
|
|
48
59
|
## Business blast radius
|
|
49
60
|
|
|
50
61
|
Green unit tests are not proof that sibling rows, approvals, or downstream jobs survived.
|
package/core/handoff-policy.md
CHANGED
|
@@ -79,13 +79,14 @@ Maintains lightweight state for instant drift detection:
|
|
|
79
79
|
- Durable why for load-bearing choices settled in grill. `refresh` does not overwrite existing ADRs.
|
|
80
80
|
|
|
81
81
|
### 9. `tasks/` — Run artifacts (optional)
|
|
82
|
-
- `current.diff`, verification logs, `handoff.md`. Not required for `init` validity.
|
|
82
|
+
- `current.diff`, verification logs, `handoff.md`, `claimed-vs-reality.md`, `maker-brief.md`. Not required for `init` validity.
|
|
83
83
|
|
|
84
84
|
### 10. `workflow.md` — Loop overlay (optional)
|
|
85
85
|
- Hooks around the 8-stage OS: `before_grill`, `after_freeze`, `after_pass`.
|
|
86
|
+
- Optional `maker_intern` (default `none`). Pick from this host's catalog (`grok models` or `/models`). Do not type a guessed slug. Not a key or URL.
|
|
86
87
|
- Cannot skip Goal Contract, verification, Devil's Advocate, or Judge.
|
|
87
|
-
- Missing file means default 8-stage loop (no extra hooks).
|
|
88
|
-
- Generate with `npx ai-engineering-loop generate-workflow` (grill) or `--write`.
|
|
88
|
+
- Missing file means default 8-stage loop (no extra hooks, parent is Maker).
|
|
89
|
+
- Generate with `npx ai-engineering-loop generate-workflow` (grill Q1–Q6) or `--write`.
|
|
89
90
|
|
|
90
91
|
### 11. `lessons.md` — Confirmed process lessons (optional)
|
|
91
92
|
- One human-confirmed row per lesson. Not a chat transcript. Not `eval.md`.
|
|
@@ -63,3 +63,26 @@ The Verification Gate immediately halts and returns to the Maker if:
|
|
|
63
63
|
- Test logs contain zero passing assertions for new acceptance criteria.
|
|
64
64
|
- assertionEvidence maps only to a happy path while the Goal Contract lists empty, boundary, sibling, or error AC rows.
|
|
65
65
|
- The only new "test" is a source grep or a coverage percentage.
|
|
66
|
+
|
|
67
|
+
---
|
|
68
|
+
|
|
69
|
+
## 5. Claimed vs Reality (before Devil's Advocate)
|
|
70
|
+
|
|
71
|
+
Write `.ai-engineering-loop/tasks/claimed-vs-reality.md` after verification logs exist:
|
|
72
|
+
|
|
73
|
+
```markdown
|
|
74
|
+
# Claimed vs Reality
|
|
75
|
+
|
|
76
|
+
| AC | Claimed | Reality |
|
|
77
|
+
|---|---|---|
|
|
78
|
+
| AC-1 | <what Maker said is done> | <command, exit code, assertion or log line> |
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
Do **not** spawn Devil's Advocate if:
|
|
82
|
+
|
|
83
|
+
- the file is missing
|
|
84
|
+
- the table has zero rows
|
|
85
|
+
- any Claimed row has empty Reality
|
|
86
|
+
- Reality is only `seems green`
|
|
87
|
+
|
|
88
|
+
Parser: `lib/claimed-vs-reality.js` (`readyForDa`). Pass that path to DA/Judge with the verification log.
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
'use strict';
|
|
2
|
+
|
|
3
|
+
function parseClaimedVsReality(markdown) {
|
|
4
|
+
const rows = [];
|
|
5
|
+
if (!markdown || !String(markdown).trim()) return { rows };
|
|
6
|
+
const lines = String(markdown).split('\n');
|
|
7
|
+
for (const line of lines) {
|
|
8
|
+
const trimmed = line.trim();
|
|
9
|
+
if (!trimmed.startsWith('|')) continue;
|
|
10
|
+
if (/^\|\s*-+\s*\|/.test(trimmed) || /^\|\s*AC\s*\|/i.test(trimmed)) continue;
|
|
11
|
+
const cells = trimmed.split('|').map((c) => c.trim()).filter((_, i, arr) => i > 0 && i < arr.length - 1);
|
|
12
|
+
if (cells.length < 3) continue;
|
|
13
|
+
rows.push({
|
|
14
|
+
ac: cells[0],
|
|
15
|
+
claimed: cells[1],
|
|
16
|
+
reality: cells[2]
|
|
17
|
+
});
|
|
18
|
+
}
|
|
19
|
+
return { rows };
|
|
20
|
+
}
|
|
21
|
+
|
|
22
|
+
function readyForDa(markdown) {
|
|
23
|
+
const { rows } = parseClaimedVsReality(markdown);
|
|
24
|
+
if (rows.length === 0) return false;
|
|
25
|
+
return rows.every((row) => {
|
|
26
|
+
const reality = (row.reality || '').trim();
|
|
27
|
+
if (!reality) return false;
|
|
28
|
+
if (/^seems green$/i.test(reality)) return false;
|
|
29
|
+
if (!(row.claimed || '').trim()) return false;
|
|
30
|
+
return true;
|
|
31
|
+
});
|
|
32
|
+
}
|
|
33
|
+
|
|
34
|
+
module.exports = {
|
|
35
|
+
parseClaimedVsReality,
|
|
36
|
+
readyForDa
|
|
37
|
+
};
|
package/lib/generate-workflow.js
CHANGED
|
@@ -10,6 +10,7 @@ const DEFAULT_HOOKS = {
|
|
|
10
10
|
after_freeze: 'none',
|
|
11
11
|
after_pass: 'none'
|
|
12
12
|
};
|
|
13
|
+
const DEFAULT_MAKER_INTERN = 'none';
|
|
13
14
|
|
|
14
15
|
const REQUIRED_ALIASES = {
|
|
15
16
|
goal_contract: 'goal_contract',
|
|
@@ -51,8 +52,26 @@ function routeDurableNote(kind) {
|
|
|
51
52
|
return map[kind] || 'lessons.md';
|
|
52
53
|
}
|
|
53
54
|
|
|
54
|
-
function
|
|
55
|
+
function normalizeMakerIntern(raw) {
|
|
56
|
+
const s = String(raw || '').trim();
|
|
57
|
+
if (!s || /^none$/i.test(s)) return DEFAULT_MAKER_INTERN;
|
|
58
|
+
if (/api[_-]?key/i.test(s) || /^sk-/i.test(s) || /https?:\/\//i.test(s)) {
|
|
59
|
+
const err = new Error('maker_intern is a host-native label, not a key or URL');
|
|
60
|
+
err.code = 'INTERN_SECRET';
|
|
61
|
+
throw err;
|
|
62
|
+
}
|
|
63
|
+
return s;
|
|
64
|
+
}
|
|
65
|
+
|
|
66
|
+
function parseMakerInternLine(text) {
|
|
67
|
+
const match = String(text || '').match(/\*\*maker_intern\*\*:\s*(.*)/i);
|
|
68
|
+
if (!match) return DEFAULT_MAKER_INTERN;
|
|
69
|
+
return normalizeMakerIntern(match[1]);
|
|
70
|
+
}
|
|
71
|
+
|
|
72
|
+
function workflowMarkdown({ hooks = DEFAULT_HOOKS, optionalSkips = [], makerIntern = DEFAULT_MAKER_INTERN } = {}) {
|
|
55
73
|
const skipLine = optionalSkips.length ? optionalSkips.join(', ') : 'none';
|
|
74
|
+
const intern = normalizeMakerIntern(makerIntern);
|
|
56
75
|
return `# Loop Overlay
|
|
57
76
|
|
|
58
77
|
This file adds hooks around the AI Engineering Loop. It does not replace the 8-stage OS.
|
|
@@ -68,6 +87,11 @@ This file adds hooks around the AI Engineering Loop. It does not replace the 8-s
|
|
|
68
87
|
- **after_freeze**: ${hooks.after_freeze || 'none'}
|
|
69
88
|
- **after_pass**: ${hooks.after_pass || 'none'}
|
|
70
89
|
|
|
90
|
+
## Maker intern
|
|
91
|
+
- **maker_intern**: ${intern}
|
|
92
|
+
|
|
93
|
+
Default \`none\` = parent is Maker. Pick \`maker_intern\` from this host's catalog (\`grok models\` or \`/models\`). Do not type a guessed slug. AEL does not store intern API keys. Devil's Advocate and Judge stay on the parent. Intern cannot skip them.
|
|
94
|
+
|
|
71
95
|
## Optional skips
|
|
72
96
|
${skipLine}
|
|
73
97
|
|
|
@@ -114,6 +138,7 @@ function parseWorkflowMarkdown(text) {
|
|
|
114
138
|
after_freeze: parseHookLine(text, 'after_freeze'),
|
|
115
139
|
after_pass: parseHookLine(text, 'after_pass')
|
|
116
140
|
},
|
|
141
|
+
makerIntern: parseMakerInternLine(text),
|
|
117
142
|
optionalSkips: parseOptionalSkips(text)
|
|
118
143
|
};
|
|
119
144
|
}
|
|
@@ -128,6 +153,7 @@ function parseWorkflowFile(rootDir) {
|
|
|
128
153
|
missing: true,
|
|
129
154
|
required: REQUIRED_STAGES.slice(),
|
|
130
155
|
hooks: { ...DEFAULT_HOOKS },
|
|
156
|
+
makerIntern: DEFAULT_MAKER_INTERN,
|
|
131
157
|
optionalSkips: []
|
|
132
158
|
};
|
|
133
159
|
}
|
|
@@ -138,7 +164,7 @@ function lessonsAreFilled(text) {
|
|
|
138
164
|
return /\| 20\d{2}-/.test(text);
|
|
139
165
|
}
|
|
140
166
|
|
|
141
|
-
function writeWorkflowOverlay(rootDir, { hooks, optionalSkips } = {}) {
|
|
167
|
+
function writeWorkflowOverlay(rootDir, { hooks, optionalSkips, makerIntern } = {}) {
|
|
142
168
|
const skips = optionalSkips || [];
|
|
143
169
|
assertNoRequiredSkip(skips);
|
|
144
170
|
for (const skip of skips) {
|
|
@@ -149,6 +175,7 @@ function writeWorkflowOverlay(rootDir, { hooks, optionalSkips } = {}) {
|
|
|
149
175
|
throw err;
|
|
150
176
|
}
|
|
151
177
|
}
|
|
178
|
+
const intern = normalizeMakerIntern(makerIntern);
|
|
152
179
|
const dir = path.join(rootDir, '.ai-engineering-loop');
|
|
153
180
|
fs.mkdirSync(dir, { recursive: true });
|
|
154
181
|
const workflowPath = path.join(dir, 'workflow.md');
|
|
@@ -157,7 +184,8 @@ function writeWorkflowOverlay(rootDir, { hooks, optionalSkips } = {}) {
|
|
|
157
184
|
workflowPath,
|
|
158
185
|
workflowMarkdown({
|
|
159
186
|
hooks: { ...DEFAULT_HOOKS, ...(hooks || {}) },
|
|
160
|
-
optionalSkips: skips
|
|
187
|
+
optionalSkips: skips,
|
|
188
|
+
makerIntern: intern
|
|
161
189
|
})
|
|
162
190
|
);
|
|
163
191
|
if (!fs.existsSync(lessonsPath) || !lessonsAreFilled(fs.readFileSync(lessonsPath, 'utf8'))) {
|
|
@@ -173,7 +201,8 @@ function formatGenerateWorkflowReport({ version, wrote } = {}) {
|
|
|
173
201
|
`Workflow overlay generate (v${version || 'unknown'})`,
|
|
174
202
|
'This is not a second OS. Required: Goal Contract, verification, Devil\'s Advocate, Judge.',
|
|
175
203
|
'Files: .ai-engineering-loop/workflow.md and lessons.md',
|
|
176
|
-
'Next: load skill generate-workflow. Ask Q1-
|
|
204
|
+
'Next: load skill generate-workflow. Ask Q1-Q6. Wait.',
|
|
205
|
+
'Q6: pick from host catalog (grok models or /models). Do not type a model name. Recommended: none',
|
|
177
206
|
'Headless: npx ai-engineering-loop generate-workflow --write'
|
|
178
207
|
];
|
|
179
208
|
if (wrote) {
|
|
@@ -191,7 +220,9 @@ module.exports = {
|
|
|
191
220
|
REQUIRED_STAGES,
|
|
192
221
|
ALLOWED_OPTIONAL_SKIPS,
|
|
193
222
|
DEFAULT_HOOKS,
|
|
223
|
+
DEFAULT_MAKER_INTERN,
|
|
194
224
|
assertNoRequiredSkip,
|
|
225
|
+
normalizeMakerIntern,
|
|
195
226
|
routeDurableNote,
|
|
196
227
|
workflowMarkdown,
|
|
197
228
|
lessonsMarkdown,
|
|
@@ -0,0 +1,53 @@
|
|
|
1
|
+
'use strict';
|
|
2
|
+
|
|
3
|
+
const HOST_MODEL_LIST_COMMANDS = ['grok models', '/models'];
|
|
4
|
+
|
|
5
|
+
function parseGrokModelsOutput(text) {
|
|
6
|
+
const models = [];
|
|
7
|
+
let defaultModel = '';
|
|
8
|
+
const raw = String(text || '');
|
|
9
|
+
const def = raw.match(/Default model:\s*(\S+)/i);
|
|
10
|
+
if (def) defaultModel = def[1];
|
|
11
|
+
for (const line of raw.split('\n')) {
|
|
12
|
+
const match = line.match(/^\s*[-*]\s+(\S+)/);
|
|
13
|
+
if (!match) continue;
|
|
14
|
+
const id = match[1].replace(/\(default\)/i, '').trim();
|
|
15
|
+
if (id && !models.includes(id)) models.push(id);
|
|
16
|
+
}
|
|
17
|
+
return { defaultModel, models };
|
|
18
|
+
}
|
|
19
|
+
|
|
20
|
+
function isCodingInternModel(id) {
|
|
21
|
+
const s = String(id || '').toLowerCase();
|
|
22
|
+
if (!s) return false;
|
|
23
|
+
if (/imagine/.test(s)) return false;
|
|
24
|
+
if (/(^|-)video($|-)/.test(s)) return false;
|
|
25
|
+
return true;
|
|
26
|
+
}
|
|
27
|
+
|
|
28
|
+
function codingInternModels(models) {
|
|
29
|
+
return (models || []).filter(isCodingInternModel);
|
|
30
|
+
}
|
|
31
|
+
|
|
32
|
+
function formatInternPickList(models) {
|
|
33
|
+
const lines = ['0) none (Recommended)'];
|
|
34
|
+
codingInternModels(models).forEach((id, i) => {
|
|
35
|
+
lines.push(`${i + 1}) ${id}`);
|
|
36
|
+
});
|
|
37
|
+
return lines.join('\n');
|
|
38
|
+
}
|
|
39
|
+
|
|
40
|
+
function internInCatalog(id, models) {
|
|
41
|
+
const s = String(id || '').trim();
|
|
42
|
+
if (!s || /^none$/i.test(s)) return true;
|
|
43
|
+
return codingInternModels(models).includes(s);
|
|
44
|
+
}
|
|
45
|
+
|
|
46
|
+
module.exports = {
|
|
47
|
+
HOST_MODEL_LIST_COMMANDS,
|
|
48
|
+
parseGrokModelsOutput,
|
|
49
|
+
isCodingInternModel,
|
|
50
|
+
codingInternModels,
|
|
51
|
+
formatInternPickList,
|
|
52
|
+
internInCatalog
|
|
53
|
+
};
|
package/package.json
CHANGED
|
@@ -13,6 +13,11 @@ This file adds hooks around the AI Engineering Loop. It does not replace the 8-s
|
|
|
13
13
|
- **after_freeze**: none
|
|
14
14
|
- **after_pass**: none
|
|
15
15
|
|
|
16
|
+
## Maker intern
|
|
17
|
+
- **maker_intern**: none
|
|
18
|
+
|
|
19
|
+
Default `none` = parent is Maker. Pick `maker_intern` from this host's catalog (`grok models` or `/models`). Do not type a guessed slug. AEL does not store intern API keys. Devil's Advocate and Judge stay on the parent. Intern cannot skip them.
|
|
20
|
+
|
|
16
21
|
## Optional skips
|
|
17
22
|
none
|
|
18
23
|
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
'use strict';
|
|
2
|
+
|
|
3
|
+
const test = require('node:test');
|
|
4
|
+
const assert = require('node:assert');
|
|
5
|
+
const { parseClaimedVsReality, readyForDa } = require('../lib/claimed-vs-reality.js');
|
|
6
|
+
|
|
7
|
+
test('AC-4 filled table is ready for DA', () => {
|
|
8
|
+
const md = `# Claimed vs Reality
|
|
9
|
+
|
|
10
|
+
| AC | Claimed | Reality |
|
|
11
|
+
|---|---|---|
|
|
12
|
+
| AC-1 | overlay writes workflow.md | npm test exit 0; AC-1 test passed |
|
|
13
|
+
| AC-2 | missing overlay is default | npm test exit 0; AC-2 test passed |
|
|
14
|
+
`;
|
|
15
|
+
const { rows } = parseClaimedVsReality(md);
|
|
16
|
+
assert.strictEqual(rows.length, 2);
|
|
17
|
+
assert.strictEqual(readyForDa(md), true);
|
|
18
|
+
});
|
|
19
|
+
|
|
20
|
+
test('AC-5 empty Reality or seems-green is not ready for DA', () => {
|
|
21
|
+
assert.strictEqual(readyForDa(''), false);
|
|
22
|
+
assert.strictEqual(readyForDa('# Claimed vs Reality\n'), false);
|
|
23
|
+
const emptyReality = `| AC | Claimed | Reality |\n|---|---|---|\n| AC-1 | tests pass | |\n`;
|
|
24
|
+
assert.strictEqual(readyForDa(emptyReality), false);
|
|
25
|
+
const seems = `| AC | Claimed | Reality |\n|---|---|---|\n| AC-1 | tests pass | seems green |\n`;
|
|
26
|
+
assert.strictEqual(readyForDa(seems), false);
|
|
27
|
+
});
|
|
@@ -9,10 +9,13 @@ const { execFileSync } = require('child_process');
|
|
|
9
9
|
|
|
10
10
|
const {
|
|
11
11
|
parseWorkflowFile,
|
|
12
|
+
parseWorkflowMarkdown,
|
|
12
13
|
writeWorkflowOverlay,
|
|
13
14
|
assertNoRequiredSkip,
|
|
14
15
|
routeDurableNote,
|
|
15
|
-
parseWriteArg
|
|
16
|
+
parseWriteArg,
|
|
17
|
+
normalizeMakerIntern,
|
|
18
|
+
workflowMarkdown
|
|
16
19
|
} = require('../lib/generate-workflow.js');
|
|
17
20
|
|
|
18
21
|
const CLI = path.join(__dirname, '..', 'bin', 'ai-engineering-loop.js');
|
|
@@ -93,7 +96,8 @@ test('CLI generate-workflow without --write prints grill protocol', () => {
|
|
|
93
96
|
const dir = tmpRepo();
|
|
94
97
|
const out = execFileSync('node', [CLI, 'generate-workflow'], { cwd: dir, encoding: 'utf8' });
|
|
95
98
|
assert.match(out, /generate-workflow/);
|
|
96
|
-
assert.match(out, /Q1-
|
|
99
|
+
assert.match(out, /Q1-Q6/);
|
|
100
|
+
assert.match(out, /Do not type a model name/);
|
|
97
101
|
assert.ok(!fs.existsSync(path.join(dir, '.ai-engineering-loop', 'workflow.md')));
|
|
98
102
|
});
|
|
99
103
|
|
|
@@ -107,3 +111,70 @@ test('CLI generate-workflow --write writes overlay files', () => {
|
|
|
107
111
|
assert.ok(fs.existsSync(path.join(dir, '.ai-engineering-loop', 'workflow.md')));
|
|
108
112
|
assert.ok(fs.existsSync(path.join(dir, '.ai-engineering-loop', 'lessons.md')));
|
|
109
113
|
});
|
|
114
|
+
|
|
115
|
+
test('compact pipeline does not add overlay pipeline or Q7', () => {
|
|
116
|
+
const text = workflowMarkdown();
|
|
117
|
+
assert.doesNotMatch(text, /\*\*pipeline\*\*/);
|
|
118
|
+
assert.doesNotMatch(text, /## Pipeline/);
|
|
119
|
+
const skill = fs.readFileSync(
|
|
120
|
+
path.join(__dirname, '..', 'adapters', 'generate-workflow', 'SKILL.md'),
|
|
121
|
+
'utf8'
|
|
122
|
+
);
|
|
123
|
+
assert.doesNotMatch(skill, /Q7/);
|
|
124
|
+
assert.match(skill, /Q1–Q6|Q1-Q6|Q1\.\.Q6|Q6/);
|
|
125
|
+
});
|
|
126
|
+
|
|
127
|
+
test('AC-1 default overlay maker_intern is none', () => {
|
|
128
|
+
const dir = tmpRepo();
|
|
129
|
+
const wrote = writeWorkflowOverlay(dir, {});
|
|
130
|
+
const workflow = fs.readFileSync(wrote.workflow, 'utf8');
|
|
131
|
+
const parsed = parseWorkflowFile(dir);
|
|
132
|
+
assert.match(workflow, /\*\*maker_intern\*\*: none/);
|
|
133
|
+
assert.strictEqual(parsed.makerIntern, 'none');
|
|
134
|
+
assert.match(workflowMarkdown(), /host's catalog/);
|
|
135
|
+
assert.match(workflowMarkdown(), /Do not type a guessed slug/);
|
|
136
|
+
});
|
|
137
|
+
|
|
138
|
+
test('AC-2 missing workflow.md makerIntern is none', () => {
|
|
139
|
+
const dir = tmpRepo();
|
|
140
|
+
const parsed = parseWorkflowFile(dir);
|
|
141
|
+
assert.strictEqual(parsed.missing, true);
|
|
142
|
+
assert.strictEqual(parsed.makerIntern, 'none');
|
|
143
|
+
});
|
|
144
|
+
|
|
145
|
+
test('AC-3 empty or None intern normalizes to none', () => {
|
|
146
|
+
assert.strictEqual(normalizeMakerIntern(''), 'none');
|
|
147
|
+
assert.strictEqual(normalizeMakerIntern('none'), 'none');
|
|
148
|
+
assert.strictEqual(normalizeMakerIntern('None'), 'none');
|
|
149
|
+
const parsed = parseWorkflowMarkdown('# Loop Overlay\n\n- **maker_intern**:\n');
|
|
150
|
+
assert.strictEqual(parsed.makerIntern, 'none');
|
|
151
|
+
});
|
|
152
|
+
|
|
153
|
+
test('AC-4 gemini-flash and haiku are valid intern labels', () => {
|
|
154
|
+
const dir = tmpRepo();
|
|
155
|
+
writeWorkflowOverlay(dir, { makerIntern: 'gemini-flash' });
|
|
156
|
+
assert.strictEqual(parseWorkflowFile(dir).makerIntern, 'gemini-flash');
|
|
157
|
+
writeWorkflowOverlay(dir, { makerIntern: 'haiku' });
|
|
158
|
+
assert.strictEqual(parseWorkflowFile(dir).makerIntern, 'haiku');
|
|
159
|
+
const text = workflowMarkdown();
|
|
160
|
+
assert.doesNotMatch(text, /must use qwen/i);
|
|
161
|
+
assert.doesNotMatch(text, /must use deepseek/i);
|
|
162
|
+
assert.doesNotMatch(text, /required: qwen/i);
|
|
163
|
+
});
|
|
164
|
+
|
|
165
|
+
test('AC-5 intern label does not allow skipping Devil\'s Advocate', () => {
|
|
166
|
+
const dir = tmpRepo();
|
|
167
|
+
assert.throws(
|
|
168
|
+
() => writeWorkflowOverlay(dir, { makerIntern: 'qwen', optionalSkips: ['devil_advocate'] }),
|
|
169
|
+
/cannot skip/
|
|
170
|
+
);
|
|
171
|
+
assert.ok(!fs.existsSync(path.join(dir, '.ai-engineering-loop', 'workflow.md')));
|
|
172
|
+
});
|
|
173
|
+
|
|
174
|
+
test('AC-6 intern rejects API keys and URLs', () => {
|
|
175
|
+
const dir = tmpRepo();
|
|
176
|
+
assert.throws(() => writeWorkflowOverlay(dir, { makerIntern: 'sk-abc' }), /not a key or URL/);
|
|
177
|
+
assert.throws(() => writeWorkflowOverlay(dir, { makerIntern: 'https://api.example.com' }), /not a key or URL/);
|
|
178
|
+
assert.throws(() => normalizeMakerIntern('openai_api_key'), /not a key or URL/);
|
|
179
|
+
assert.ok(!fs.existsSync(path.join(dir, '.ai-engineering-loop', 'workflow.md')));
|
|
180
|
+
});
|
|
@@ -0,0 +1,72 @@
|
|
|
1
|
+
'use strict';
|
|
2
|
+
|
|
3
|
+
const test = require('node:test');
|
|
4
|
+
const assert = require('node:assert');
|
|
5
|
+
const {
|
|
6
|
+
HOST_MODEL_LIST_COMMANDS,
|
|
7
|
+
parseGrokModelsOutput,
|
|
8
|
+
codingInternModels,
|
|
9
|
+
formatInternPickList,
|
|
10
|
+
internInCatalog
|
|
11
|
+
} = require('../lib/intern-catalog.js');
|
|
12
|
+
|
|
13
|
+
const GROK_MODELS_FIXTURE = `You are logged in with grok.com.
|
|
14
|
+
|
|
15
|
+
Default model: grok-4.6
|
|
16
|
+
|
|
17
|
+
Available models:
|
|
18
|
+
- grok-4.20-0309-non-reasoning
|
|
19
|
+
- grok-4.20-0309-reasoning
|
|
20
|
+
- grok-4.20-multi-agent-0309
|
|
21
|
+
- grok-4.3
|
|
22
|
+
- grok-4.5
|
|
23
|
+
* grok-4.6 (default)
|
|
24
|
+
- grok-build-0.1
|
|
25
|
+
- grok-imagine-image
|
|
26
|
+
- grok-imagine-image-2.0
|
|
27
|
+
- grok-imagine-image-quality
|
|
28
|
+
- grok-imagine-video
|
|
29
|
+
- grok-imagine-video-1.5
|
|
30
|
+
`;
|
|
31
|
+
|
|
32
|
+
test('host list commands include grok models and /models', () => {
|
|
33
|
+
assert.ok(HOST_MODEL_LIST_COMMANDS.includes('grok models'));
|
|
34
|
+
assert.ok(HOST_MODEL_LIST_COMMANDS.includes('/models'));
|
|
35
|
+
});
|
|
36
|
+
|
|
37
|
+
test('AC-1 parse grok models into ids plus default', () => {
|
|
38
|
+
const parsed = parseGrokModelsOutput(GROK_MODELS_FIXTURE);
|
|
39
|
+
assert.strictEqual(parsed.defaultModel, 'grok-4.6');
|
|
40
|
+
assert.ok(parsed.models.includes('grok-build-0.1'));
|
|
41
|
+
assert.ok(parsed.models.includes('grok-4.6'));
|
|
42
|
+
});
|
|
43
|
+
|
|
44
|
+
test('AC-2 empty catalog pick list is only none', () => {
|
|
45
|
+
assert.strictEqual(formatInternPickList([]), '0) none (Recommended)');
|
|
46
|
+
assert.strictEqual(internInCatalog('none', []), true);
|
|
47
|
+
assert.strictEqual(internInCatalog('', []), true);
|
|
48
|
+
});
|
|
49
|
+
|
|
50
|
+
test('AC-3 Claude has no grok catalog so only none is in-catalog', () => {
|
|
51
|
+
assert.strictEqual(internInCatalog('gemini-flash', []), false);
|
|
52
|
+
assert.strictEqual(internInCatalog('grok-build-0.1', []), false);
|
|
53
|
+
});
|
|
54
|
+
|
|
55
|
+
test('AC-4 imagine and video models are not coding intern picks', () => {
|
|
56
|
+
const { models } = parseGrokModelsOutput(GROK_MODELS_FIXTURE);
|
|
57
|
+
const coding = codingInternModels(models);
|
|
58
|
+
assert.ok(coding.includes('grok-build-0.1'));
|
|
59
|
+
assert.ok(!coding.includes('grok-imagine-image'));
|
|
60
|
+
assert.ok(!coding.includes('grok-imagine-video'));
|
|
61
|
+
const list = formatInternPickList(models);
|
|
62
|
+
assert.match(list, /0\) none \(Recommended\)/);
|
|
63
|
+
assert.match(list, /grok-build-0.1/);
|
|
64
|
+
assert.doesNotMatch(list, /imagine/);
|
|
65
|
+
});
|
|
66
|
+
|
|
67
|
+
test('AC-5 unknown slug is not in catalog', () => {
|
|
68
|
+
const { models } = parseGrokModelsOutput(GROK_MODELS_FIXTURE);
|
|
69
|
+
assert.strictEqual(internInCatalog('grok-build-0.1', models), true);
|
|
70
|
+
assert.strictEqual(internInCatalog('not-a-real-model', models), false);
|
|
71
|
+
assert.strictEqual(internInCatalog('sk-abc', models), false);
|
|
72
|
+
});
|
|
@@ -218,8 +218,14 @@ test('generate-workflow skill is Claude-safe and host skills read the overlay',
|
|
|
218
218
|
assert.doesNotMatch(skill, /\$\\/);
|
|
219
219
|
assert.doesNotMatch(skill, /spawn_subagent/);
|
|
220
220
|
assert.match(skill, /Q1/);
|
|
221
|
+
assert.match(skill, /Q6/);
|
|
222
|
+
assert.match(skill, /maker_intern/);
|
|
221
223
|
assert.match(skill, /Do not start Maker/);
|
|
222
224
|
assert.match(skill, /lessons\.md/);
|
|
225
|
+
assert.match(skill, /grok models/);
|
|
226
|
+
assert.match(skill, /\/models/);
|
|
227
|
+
assert.match(skill, /Do not type a model name/);
|
|
228
|
+
assert.doesNotMatch(skill, /any host-native label \(gemini-flash/);
|
|
223
229
|
for (const rel of [
|
|
224
230
|
'.claude/skills/ai-engineering-loop/SKILL.md',
|
|
225
231
|
'.grok/skills/ai-engineering-loop/SKILL.md',
|
|
@@ -231,6 +237,9 @@ test('generate-workflow skill is Claude-safe and host skills read the overlay',
|
|
|
231
237
|
assert.match(text, /workflow\.md/, rel);
|
|
232
238
|
assert.match(text, /Do not skip Goal Contract, verification, Devil's Advocate, or Judge/, rel);
|
|
233
239
|
assert.match(text, /generate-workflow/, rel);
|
|
240
|
+
assert.match(text, /maker_intern/, rel);
|
|
241
|
+
assert.match(text, /INVOCATION_UNAVAILABLE/, rel);
|
|
242
|
+
assert.match(text, /Do not type a model name/, rel);
|
|
234
243
|
}
|
|
235
244
|
});
|
|
236
245
|
|
|
@@ -255,6 +264,32 @@ test('generate-adapter skill is Claude-safe and shipped adapters stay generic',
|
|
|
255
264
|
assert.match(standard, /Judge `PASS`/);
|
|
256
265
|
});
|
|
257
266
|
|
|
267
|
+
test('Host compact map names four buckets and keeps 8-stage numbers', () => {
|
|
268
|
+
const readme = readRepo('README.md');
|
|
269
|
+
assert.match(readme, /Specify \(stages 0-1\)/);
|
|
270
|
+
assert.match(readme, /Make \(stages 2-4\)/);
|
|
271
|
+
assert.match(readme, /Review \(stages 5-7\)/);
|
|
272
|
+
assert.match(readme, /Deliver \(stage 8\)/);
|
|
273
|
+
assert.doesNotMatch(readme, /Later phase \(not this change\): host compact pipeline/);
|
|
274
|
+
const grill = readRepo('core/grill-policy.md');
|
|
275
|
+
assert.doesNotMatch(grill, /Later phase \(not this loop\): host compact pipeline/);
|
|
276
|
+
for (const rel of [
|
|
277
|
+
'.claude/skills/ai-engineering-loop/SKILL.md',
|
|
278
|
+
'.grok/skills/ai-engineering-loop/SKILL.md',
|
|
279
|
+
'.gemini/skills/ai-engineering-loop/SKILL.md',
|
|
280
|
+
'.agents/workflows/ai-engineering-loop.md'
|
|
281
|
+
]) {
|
|
282
|
+
const text = readRepo(rel);
|
|
283
|
+
assert.match(text, /Specify \(stages 0-1\)/, rel);
|
|
284
|
+
assert.match(text, /Make \(stages 2-4\)/, rel);
|
|
285
|
+
assert.match(text, /Review \(stages 5-7\)/, rel);
|
|
286
|
+
assert.match(text, /Deliver \(stage 8\)/, rel);
|
|
287
|
+
assert.match(text, /8-stage/, rel);
|
|
288
|
+
assert.match(text, /not a second OS/, rel);
|
|
289
|
+
assert.match(text, /Do not skip Goal Contract, verification, Devil's Advocate, or Judge/, rel);
|
|
290
|
+
}
|
|
291
|
+
});
|
|
292
|
+
|
|
258
293
|
test('Host skills Stage 0 runs sync-hosts before status', () => {
|
|
259
294
|
for (const rel of [
|
|
260
295
|
'.claude/skills/ai-engineering-loop/SKILL.md',
|
|
@@ -267,6 +302,29 @@ test('Host skills Stage 0 runs sync-hosts before status', () => {
|
|
|
267
302
|
}
|
|
268
303
|
});
|
|
269
304
|
|
|
305
|
+
test('Sloppy prompt drafts a Goal Contract; claimed-vs-reality gates DA', () => {
|
|
306
|
+
const grill = readRepo('core/grill-policy.md');
|
|
307
|
+
assert.match(grill, /Sloppy \/ underspecified prompts/);
|
|
308
|
+
assert.match(grill, /Do not start Maker/);
|
|
309
|
+
assert.match(grill, /Do not invent a second draft/);
|
|
310
|
+
const verif = readRepo('core/verification-loop.md');
|
|
311
|
+
assert.match(verif, /claimed-vs-reality\.md/);
|
|
312
|
+
assert.match(verif, /seems green/);
|
|
313
|
+
const judge = readRepo('agents/shared/judge.body.md');
|
|
314
|
+
assert.match(judge, /claimed-vs-reality\.md/);
|
|
315
|
+
for (const rel of [
|
|
316
|
+
'.claude/skills/ai-engineering-loop/SKILL.md',
|
|
317
|
+
'.grok/skills/ai-engineering-loop/SKILL.md',
|
|
318
|
+
'.gemini/skills/ai-engineering-loop/SKILL.md',
|
|
319
|
+
'.agents/workflows/ai-engineering-loop.md'
|
|
320
|
+
]) {
|
|
321
|
+
const text = readRepo(rel);
|
|
322
|
+
assert.match(text, /If the user prompt has no numbered AC/, rel);
|
|
323
|
+
assert.match(text, /claimed-vs-reality\.md/, rel);
|
|
324
|
+
assert.match(text, /Do not spawn Devil's Advocate if the file is missing/, rel);
|
|
325
|
+
}
|
|
326
|
+
});
|
|
327
|
+
|
|
270
328
|
test('Grill freeze gate and idea menu are in the loop, not a parallel product', () => {
|
|
271
329
|
const grill = readRepo('core/grill-policy.md');
|
|
272
330
|
assert.match(grill, /Idea and menu requests/);
|