ai-engineering-loop 1.0.3 → 1.0.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.grok/agents/devil-advocate.md +67 -0
- package/.grok/agents/judge.md +53 -0
- package/.grok/commands/ai-engineering-loop.md +11 -0
- package/.grok/skills/ai-engineering-loop/SKILL.md +79 -0
- package/README.md +247 -19
- package/README.npm.md +7 -0
- package/agents/devil-advocate.md +9 -5
- package/agents/judge.md +2 -0
- package/bin/ai-engineering-loop.js +28 -1
- package/core/orchestration-model.md +20 -0
- package/docs/grok-cli-feasibility.md +99 -0
- package/lib/orchestration.js +182 -2
- package/package.json +22 -2
- package/tests/grok-runtime.test.js +201 -0
- package/README.full.md +0 -251
|
@@ -0,0 +1,99 @@
|
|
|
1
|
+
# Grok CLI Feasibility & Execution Strategy
|
|
2
|
+
|
|
3
|
+
## 1. Executive Summary
|
|
4
|
+
|
|
5
|
+
Grok CLI (verified `grok 1.0.5`) is a **first-class host** for the AI Engineering Loop. Unlike Antigravity standalone mode, Grok exposes `spawn_subagent` as a real independent child session: own context window, no parent transcript unless `resume_from` is set, and a captured `subagent_id` plus model response.
|
|
6
|
+
|
|
7
|
+
When those evidence fields are present, the capability registry **must** select `TRUE_INDEPENDENT_AGENT` (skill alias: `NATIVE_SUBAGENT`).
|
|
8
|
+
|
|
9
|
+
```text
|
|
10
|
+
Parent (Maker + orchestrator)
|
|
11
|
+
├─ spawn_subagent type=devil-advocate → Finding Ledger
|
|
12
|
+
└─ spawn_subagent type=judge → PASS | ITERATE | ESCALATE
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
Children cannot spawn children (depth 1). The parent therefore remains the orchestrator.
|
|
16
|
+
|
|
17
|
+
---
|
|
18
|
+
|
|
19
|
+
## 2. Empirical Discovery Record
|
|
20
|
+
|
|
21
|
+
| Surface | Tested command / API | Classification | Result |
|
|
22
|
+
|---|---|---|---|
|
|
23
|
+
| Grok CLI binary | `~/.grok/bin/grok` (`grok 1.0.5`) | `CONFIGURATION_SUPPORTED` | Installed and authenticated in TUI sessions |
|
|
24
|
+
| Native child session | `spawn_subagent` | `INVOCATION_AVAILABLE` | Enabled by default; disabled only when `GROK_SUBAGENTS=0` or `--disallowed-tools Agent` |
|
|
25
|
+
| Child execution | child `subagent_id` + model response | `EXECUTION_PROVEN` | Proven only after the child returns; binary presence is not proof |
|
|
26
|
+
| Project agents | `.grok/agents/devil-advocate.md`, `.grok/agents/judge.md` | `CONFIGURATION_SUPPORTED` | Registered as `subagent_type` values |
|
|
27
|
+
| Skill + slash command | `.grok/skills/ai-engineering-loop/`, `.grok/commands/ai-engineering-loop.md` | `CONFIGURATION_SUPPORTED` | Grok discovers repo-local skills and command markdown |
|
|
28
|
+
| Headless child | `grok -p "..."` | `FRESH_PROCESS_AGENT` | Valid fallback **after** stdout contains a model response |
|
|
29
|
+
| Compressed reviewer | `caveman:cavecrew-reviewer` | `GROK_COMPRESSED_REVIEW_PRESET` | **Rejected** — output schema is `path:line: emoji severity`, not the Finding Ledger |
|
|
30
|
+
| Browser tools | `browser_*` | `BROWSER_AUTOMATION_TOOL` | Not an LLM reviewer |
|
|
31
|
+
| `resume_from` Maker | `spawn_subagent(resume_from=makerId)` | tainted history | **Rejected** — inherits Maker transcript |
|
|
32
|
+
|
|
33
|
+
### What is NOT execution proof on Grok
|
|
34
|
+
|
|
35
|
+
1. `~/.grok/bin/grok` existing on disk
|
|
36
|
+
2. `.grok/agents/*.md` being discoverable
|
|
37
|
+
3. Subagents being enabled in `config.toml`
|
|
38
|
+
4. A persona named "reviewer" without a child response
|
|
39
|
+
5. Cavecrew reviewer output that looks like a review
|
|
40
|
+
|
|
41
|
+
---
|
|
42
|
+
|
|
43
|
+
## 3. Mapping Loop Roles → Grok Types
|
|
44
|
+
|
|
45
|
+
| Loop role | Grok `subagent_type` | `capability_mode` | `isolation` | `resume_from` |
|
|
46
|
+
|---|---|---|---|---|
|
|
47
|
+
| Orchestrator / Maker | parent session (or `general-purpose` if parent is orchestrator-only) | `all` | `none` | n/a |
|
|
48
|
+
| Devil's Advocate | `devil-advocate` (fallback `general-purpose`) | `execute` | `none` | **omit** |
|
|
49
|
+
| Judge | `judge` (fallback `general-purpose`) | `execute` | `none` | **omit** |
|
|
50
|
+
|
|
51
|
+
`execute` allows git/read/search and forbids source edits. Do not use `read-only` if the child needs `git diff`. Do not use `worktree` isolation: DA and Judge must see the Maker working tree.
|
|
52
|
+
|
|
53
|
+
`description` prefixes `[devil-advocate]` and `[judge]` so the Grok TUI labels the child correctly when falling back to `general-purpose`.
|
|
54
|
+
|
|
55
|
+
---
|
|
56
|
+
|
|
57
|
+
## 4. Artifact Barrier (still required)
|
|
58
|
+
|
|
59
|
+
Grok children do not inherit parent chat, but they do receive compacted `AGENTS.md` / project instructions. That is acceptable (project context, not Maker thoughts).
|
|
60
|
+
|
|
61
|
+
Still write artifacts to disk and pass **paths**, not Maker narration:
|
|
62
|
+
|
|
63
|
+
- Goal Contract
|
|
64
|
+
- git diff file
|
|
65
|
+
- verification log (command, exit code, stdout, test counts)
|
|
66
|
+
- `.ai-engineering-loop/`
|
|
67
|
+
|
|
68
|
+
---
|
|
69
|
+
|
|
70
|
+
## 5. Disable / fallback matrix
|
|
71
|
+
|
|
72
|
+
| Condition | Selected mode |
|
|
73
|
+
|---|---|
|
|
74
|
+
| `spawn_subagent` available, child response captured, no `resume_from` | `TRUE_INDEPENDENT_AGENT` |
|
|
75
|
+
| `spawn_subagent` missing, `grok -p` returns a model response | `FRESH_PROCESS_AGENT` |
|
|
76
|
+
| `GROK_SUBAGENTS=0` or `--disallowed-tools Agent` | `CONTEXT_ISOLATION_ONLY` |
|
|
77
|
+
| No grok, no spawn, no headless | `UNAVAILABLE` or artifact barrier if the same session can still read the diff |
|
|
78
|
+
|
|
79
|
+
---
|
|
80
|
+
|
|
81
|
+
## 6. Answers to the eight host questions
|
|
82
|
+
|
|
83
|
+
1. **Independent sub-agents?** Yes — `spawn_subagent`, own context window.
|
|
84
|
+
2. **Pass context?** Yes — prompt + on-disk artifacts. Do not pass Maker chat.
|
|
85
|
+
3. **Same repository?** Yes — `isolation: none` shares the workspace.
|
|
86
|
+
4. **Structured findings?** Yes — DA agent contract is Finding Ledger JSON.
|
|
87
|
+
5. **Parent consume findings?** Yes — child summary returns to parent; Judge spawn gets the ledger.
|
|
88
|
+
6. **Repeat the loop?** Yes — parent iterates; each DA/Judge spawn is fresh (no `resume_from`).
|
|
89
|
+
7. **Persist state?** Yes — files under `.ai-engineering-loop/` and task artifact paths. Grok also keeps session logs under `~/.grok/sessions/`.
|
|
90
|
+
8. **Token limits?** Pass artifact paths, not full transcripts. Do not dump Maker reasoning into the DA prompt.
|
|
91
|
+
|
|
92
|
+
---
|
|
93
|
+
|
|
94
|
+
## 7. Practical limits on Grok CLI
|
|
95
|
+
|
|
96
|
+
- **Depth 1**: a DA child cannot spawn a Judge. Parent must spawn both.
|
|
97
|
+
- **Plan mode**: a write-capable child is not gated by the parent's plan-mode lock. Keep DA/Judge on `capability_mode: execute` (no writes) so they cannot edit source while the parent is in plan mode.
|
|
98
|
+
- **Headless `--disallowed-tools Agent`**: kills native review. Disclose `CONTEXT_ISOLATION_ONLY`.
|
|
99
|
+
- **Cavecrew**: useful for cheap locate/review, **not** for this loop's Judge input.
|
package/lib/orchestration.js
CHANGED
|
@@ -11,9 +11,15 @@
|
|
|
11
11
|
* 3. Artifact isolation is strictly reported as CONTEXT_ISOLATION_ONLY (Independent LLM execution: NOT PROVEN).
|
|
12
12
|
* 4. browser_subagent is browser automation and must NOT be classified as an LLM subagent.
|
|
13
13
|
* 5. agentapi send-message is an IPC communication capability and must NEVER activate agent execution.
|
|
14
|
+
* 6. grok spawn_subagent is a true independent child session (own context, no parent transcript)
|
|
15
|
+
* when a child id and model response are captured. GROK_SUBAGENTS=0 disables invocation.
|
|
16
|
+
* 7. caveman:cavecrew-reviewer is a compressed code-review preset with a different output schema
|
|
17
|
+
* and must NEVER be used as the Devil's Advocate or Judge.
|
|
14
18
|
*/
|
|
15
19
|
|
|
16
20
|
const crypto = require('crypto');
|
|
21
|
+
const fs = require('fs');
|
|
22
|
+
const path = require('path');
|
|
17
23
|
|
|
18
24
|
// 1. Standard 5 Execution Modes
|
|
19
25
|
const EXECUTION_MODES = {
|
|
@@ -49,6 +55,30 @@ const EXECUTION_MODES = {
|
|
|
49
55
|
}
|
|
50
56
|
};
|
|
51
57
|
|
|
58
|
+
/**
|
|
59
|
+
* Skill / agent-spec aliases. The skill historically used NATIVE_SUBAGENT etc.
|
|
60
|
+
* Canonical runtime IDs remain the 5 EXECUTION_MODES keys.
|
|
61
|
+
*/
|
|
62
|
+
const EXECUTION_MODE_ALIASES = {
|
|
63
|
+
NATIVE_SUBAGENT: 'TRUE_INDEPENDENT_AGENT',
|
|
64
|
+
SDK_AGENT: 'ISOLATED_AGENT_INSTANCE',
|
|
65
|
+
HEADLESS_SUBPROCESS: 'FRESH_PROCESS_AGENT',
|
|
66
|
+
ARTIFACT_ISOLATED_REVIEW: 'CONTEXT_ISOLATION_ONLY'
|
|
67
|
+
};
|
|
68
|
+
|
|
69
|
+
const GROK_FORBIDDEN_REVIEW_TYPES = [
|
|
70
|
+
'caveman:cavecrew-reviewer',
|
|
71
|
+
'caveman:cavecrew-builder',
|
|
72
|
+
'caveman:cavecrew-investigator',
|
|
73
|
+
'explore',
|
|
74
|
+
'plan'
|
|
75
|
+
];
|
|
76
|
+
|
|
77
|
+
function resolveExecutionModeId(modeId) {
|
|
78
|
+
if (!modeId) return 'UNAVAILABLE';
|
|
79
|
+
return EXECUTION_MODE_ALIASES[modeId] || modeId;
|
|
80
|
+
}
|
|
81
|
+
|
|
52
82
|
/**
|
|
53
83
|
* 2. Standardized Capability Evidence Factory with 3-Stage Lifecycle
|
|
54
84
|
*/
|
|
@@ -126,6 +156,15 @@ function isCapabilityProvenForMode(evidence, targetModeId) {
|
|
|
126
156
|
return false;
|
|
127
157
|
}
|
|
128
158
|
|
|
159
|
+
// Reject compressed review presets that do not emit the Finding Ledger schema
|
|
160
|
+
if (
|
|
161
|
+
evidence.classification === 'GROK_COMPRESSED_REVIEW_PRESET' ||
|
|
162
|
+
GROK_FORBIDDEN_REVIEW_TYPES.includes(evidence.mechanism) ||
|
|
163
|
+
GROK_FORBIDDEN_REVIEW_TYPES.includes(evidence.commandOrApi)
|
|
164
|
+
) {
|
|
165
|
+
return false;
|
|
166
|
+
}
|
|
167
|
+
|
|
129
168
|
// Reject documentation-only or configuration-only claims
|
|
130
169
|
if (evidence.isDocumentationOnly || (evidence.configurationSupported && !evidence.executionProven)) {
|
|
131
170
|
if (targetModeId !== 'CONTEXT_ISOLATION_ONLY') {
|
|
@@ -376,7 +415,142 @@ function computeJudgeVerdict({
|
|
|
376
415
|
}
|
|
377
416
|
|
|
378
417
|
/**
|
|
379
|
-
* 9.
|
|
418
|
+
* 9. Detect Grok CLI as a host runtime.
|
|
419
|
+
* Presence of the grok binary is CONFIGURATION_SUPPORTED.
|
|
420
|
+
* GROK_SUBAGENTS=0 means invocation is disabled even if grok is installed.
|
|
421
|
+
* Execution is never proven from detection alone.
|
|
422
|
+
*/
|
|
423
|
+
function detectGrokRuntime(env = process.env, fsApi = fs) {
|
|
424
|
+
const grokHome = env.GROK_HOME || path.join(env.HOME || env.USERPROFILE || '', '.grok');
|
|
425
|
+
const grokBinCandidates = [
|
|
426
|
+
env.GROK_BIN,
|
|
427
|
+
path.join(grokHome, 'bin', 'grok'),
|
|
428
|
+
path.join(grokHome, 'bin', 'agent')
|
|
429
|
+
].filter(Boolean);
|
|
430
|
+
|
|
431
|
+
const grokBin = grokBinCandidates.find((candidate) => {
|
|
432
|
+
try {
|
|
433
|
+
return fsApi.existsSync(candidate);
|
|
434
|
+
} catch (e) {
|
|
435
|
+
return false;
|
|
436
|
+
}
|
|
437
|
+
}) || null;
|
|
438
|
+
|
|
439
|
+
const grokPresent = Boolean(grokBin);
|
|
440
|
+
const subagentsDisabled = env.GROK_SUBAGENTS === '0';
|
|
441
|
+
const invocationAvailable = grokPresent && !subagentsDisabled;
|
|
442
|
+
|
|
443
|
+
let reason;
|
|
444
|
+
if (!grokPresent) {
|
|
445
|
+
reason = 'Grok CLI binary not found under GROK_HOME/bin';
|
|
446
|
+
} else if (subagentsDisabled) {
|
|
447
|
+
reason = 'GROK_SUBAGENTS=0 disables spawn_subagent; fall back to CONTEXT_ISOLATION_ONLY or grok -p';
|
|
448
|
+
} else {
|
|
449
|
+
reason = 'Grok CLI present; spawn_subagent is the native independent-agent tool (enabled by default)';
|
|
450
|
+
}
|
|
451
|
+
|
|
452
|
+
return {
|
|
453
|
+
host: grokPresent ? 'grok-cli' : 'unknown',
|
|
454
|
+
grokHome,
|
|
455
|
+
grokBin,
|
|
456
|
+
configurationSupported: grokPresent,
|
|
457
|
+
invocationAvailable,
|
|
458
|
+
executionProven: false,
|
|
459
|
+
subagentsDisabled,
|
|
460
|
+
commandOrApi: 'spawn_subagent',
|
|
461
|
+
headlessCommand: 'grok -p',
|
|
462
|
+
reason
|
|
463
|
+
};
|
|
464
|
+
}
|
|
465
|
+
|
|
466
|
+
/**
|
|
467
|
+
* 10. Build capability evidence for Grok spawn_subagent.
|
|
468
|
+
* Independent context is proven only when the child did not resume a Maker transcript.
|
|
469
|
+
*/
|
|
470
|
+
function createGrokCapabilityEvidence({
|
|
471
|
+
invocationAvailable = false,
|
|
472
|
+
executionProven = false,
|
|
473
|
+
childConversationId = null,
|
|
474
|
+
childModelResponse = null,
|
|
475
|
+
executionIdentity = null,
|
|
476
|
+
independentContextProven = false,
|
|
477
|
+
historyInherited = null,
|
|
478
|
+
resumeFrom = null,
|
|
479
|
+
reason = null
|
|
480
|
+
} = {}) {
|
|
481
|
+
const resumedMaker = Boolean(resumeFrom);
|
|
482
|
+
const inherited = historyInherited === true || resumedMaker;
|
|
483
|
+
const modelExecuted = Boolean(executionProven || childModelResponse);
|
|
484
|
+
const fullyProven = Boolean(
|
|
485
|
+
modelExecuted &&
|
|
486
|
+
childConversationId &&
|
|
487
|
+
executionIdentity &&
|
|
488
|
+
independentContextProven &&
|
|
489
|
+
!inherited
|
|
490
|
+
);
|
|
491
|
+
|
|
492
|
+
return createCapabilityEvidence({
|
|
493
|
+
mechanism: 'grok-spawn_subagent',
|
|
494
|
+
classification: fullyProven
|
|
495
|
+
? 'TRUE_INDEPENDENT_AGENT'
|
|
496
|
+
: invocationAvailable
|
|
497
|
+
? 'INVOCATION_AVAILABLE'
|
|
498
|
+
: 'CONFIGURATION_SUPPORTED_WITHOUT_INVOCATION_TOOL',
|
|
499
|
+
configurationSupported: true,
|
|
500
|
+
invocationAvailable: Boolean(invocationAvailable),
|
|
501
|
+
executionProven: fullyProven,
|
|
502
|
+
available: Boolean(invocationAvailable || fullyProven),
|
|
503
|
+
commandOrApi: 'spawn_subagent',
|
|
504
|
+
childConversationId,
|
|
505
|
+
childModelResponse,
|
|
506
|
+
executionIdentity,
|
|
507
|
+
modelExecutionProven: modelExecuted,
|
|
508
|
+
independentContextProven: Boolean(independentContextProven && !inherited),
|
|
509
|
+
historyInherited: inherited ? true : historyInherited,
|
|
510
|
+
reason: reason || (resumedMaker
|
|
511
|
+
? 'resume_from inherits the source transcript; Devil\'s Advocate and Judge must spawn fresh'
|
|
512
|
+
: null)
|
|
513
|
+
});
|
|
514
|
+
}
|
|
515
|
+
|
|
516
|
+
/**
|
|
517
|
+
* 11. Grok spawn plan for Devil's Advocate / Judge.
|
|
518
|
+
* Parent orchestrator spawns children; children must not spawn children (depth 1).
|
|
519
|
+
* Never pass resume_from. Never use cavecrew-reviewer (wrong output schema).
|
|
520
|
+
*/
|
|
521
|
+
function buildGrokReviewSpawnPlan({
|
|
522
|
+
role,
|
|
523
|
+
iteration = 1,
|
|
524
|
+
artifactPaths = {}
|
|
525
|
+
} = {}) {
|
|
526
|
+
if (role !== 'devil-advocate' && role !== 'judge') {
|
|
527
|
+
throw new Error('Grok review spawn role must be devil-advocate or judge');
|
|
528
|
+
}
|
|
529
|
+
|
|
530
|
+
return {
|
|
531
|
+
tool: 'spawn_subagent',
|
|
532
|
+
subagent_type: role,
|
|
533
|
+
fallback_subagent_type: 'general-purpose',
|
|
534
|
+
description: role === 'judge'
|
|
535
|
+
? `[judge] evaluate iteration ${iteration}`
|
|
536
|
+
: `[devil-advocate] review iteration ${iteration}`,
|
|
537
|
+
background: false,
|
|
538
|
+
capability_mode: 'execute',
|
|
539
|
+
isolation: 'none',
|
|
540
|
+
resume_from: null,
|
|
541
|
+
forbidden_types: [...GROK_FORBIDDEN_REVIEW_TYPES],
|
|
542
|
+
artifactPaths: {
|
|
543
|
+
goalContract: artifactPaths.goalContract || null,
|
|
544
|
+
gitDiff: artifactPaths.gitDiff || null,
|
|
545
|
+
verificationLogs: artifactPaths.verificationLogs || null,
|
|
546
|
+
projectContext: artifactPaths.projectContext || '.ai-engineering-loop/',
|
|
547
|
+
findingLedger: artifactPaths.findingLedger || null
|
|
548
|
+
}
|
|
549
|
+
};
|
|
550
|
+
}
|
|
551
|
+
|
|
552
|
+
/**
|
|
553
|
+
* 12. Truthful Review Report Generator
|
|
380
554
|
* Enforces mandatory reporting headers and strictly prevents misleading phrasing.
|
|
381
555
|
*/
|
|
382
556
|
function formatExecutionReport({ selectedMode, capabilityRegistry = {} }) {
|
|
@@ -398,6 +572,9 @@ function formatExecutionReport({ selectedMode, capabilityRegistry = {} }) {
|
|
|
398
572
|
|
|
399
573
|
module.exports = {
|
|
400
574
|
EXECUTION_MODES,
|
|
575
|
+
EXECUTION_MODE_ALIASES,
|
|
576
|
+
GROK_FORBIDDEN_REVIEW_TYPES,
|
|
577
|
+
resolveExecutionModeId,
|
|
401
578
|
createCapabilityEvidence,
|
|
402
579
|
isCapabilityProvenForMode,
|
|
403
580
|
selectExecutionMode,
|
|
@@ -405,5 +582,8 @@ module.exports = {
|
|
|
405
582
|
validateVerificationEvidence,
|
|
406
583
|
validateFindingLedger,
|
|
407
584
|
computeJudgeVerdict,
|
|
408
|
-
formatExecutionReport
|
|
585
|
+
formatExecutionReport,
|
|
586
|
+
detectGrokRuntime,
|
|
587
|
+
createGrokCapabilityEvidence,
|
|
588
|
+
buildGrokReviewSpawnPlan
|
|
409
589
|
};
|
package/package.json
CHANGED
|
@@ -1,11 +1,29 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "ai-engineering-loop",
|
|
3
|
-
"version": "1.0.
|
|
3
|
+
"version": "1.0.5",
|
|
4
4
|
"description": "A reusable, framework-agnostic AI Engineering Operating System for autonomous coding agents.",
|
|
5
5
|
"main": "bin/ai-engineering-loop.js",
|
|
6
6
|
"bin": {
|
|
7
7
|
"ai-engineering-loop": "bin/ai-engineering-loop.js"
|
|
8
8
|
},
|
|
9
|
+
"files": [
|
|
10
|
+
"bin/",
|
|
11
|
+
"lib/",
|
|
12
|
+
"core/",
|
|
13
|
+
"agents/",
|
|
14
|
+
"policies/",
|
|
15
|
+
"profiles/",
|
|
16
|
+
"adapters/",
|
|
17
|
+
"templates/",
|
|
18
|
+
"examples/",
|
|
19
|
+
"docs/",
|
|
20
|
+
"tests/",
|
|
21
|
+
"scripts/",
|
|
22
|
+
".grok/",
|
|
23
|
+
"LICENSE",
|
|
24
|
+
"README.md",
|
|
25
|
+
"README.npm.md"
|
|
26
|
+
],
|
|
9
27
|
"scripts": {
|
|
10
28
|
"init": "node bin/ai-engineering-loop.js",
|
|
11
29
|
"test": "node --test tests/*.test.js"
|
|
@@ -21,7 +39,9 @@
|
|
|
21
39
|
"devils-advocate",
|
|
22
40
|
"verification",
|
|
23
41
|
"autonomous-agents",
|
|
24
|
-
"antigravity"
|
|
42
|
+
"antigravity",
|
|
43
|
+
"grok",
|
|
44
|
+
"subagents"
|
|
25
45
|
],
|
|
26
46
|
"author": "Ega Gofur",
|
|
27
47
|
"license": "MIT",
|
|
@@ -0,0 +1,201 @@
|
|
|
1
|
+
const test = require('node:test');
|
|
2
|
+
const assert = require('node:assert');
|
|
3
|
+
const {
|
|
4
|
+
EXECUTION_MODES,
|
|
5
|
+
EXECUTION_MODE_ALIASES,
|
|
6
|
+
GROK_FORBIDDEN_REVIEW_TYPES,
|
|
7
|
+
resolveExecutionModeId,
|
|
8
|
+
createCapabilityEvidence,
|
|
9
|
+
selectExecutionMode,
|
|
10
|
+
detectGrokRuntime,
|
|
11
|
+
createGrokCapabilityEvidence,
|
|
12
|
+
buildGrokReviewSpawnPlan,
|
|
13
|
+
formatExecutionReport
|
|
14
|
+
} = require('../lib/orchestration.js');
|
|
15
|
+
|
|
16
|
+
function fakeFs(existingPaths) {
|
|
17
|
+
return {
|
|
18
|
+
existsSync: (p) => existingPaths.includes(p)
|
|
19
|
+
};
|
|
20
|
+
}
|
|
21
|
+
|
|
22
|
+
test('Grok spawn_subagent with full execution proof selects TRUE_INDEPENDENT_AGENT', () => {
|
|
23
|
+
const nativeSubagent = createGrokCapabilityEvidence({
|
|
24
|
+
invocationAvailable: true,
|
|
25
|
+
childConversationId: 'grok-child-da-01',
|
|
26
|
+
childModelResponse: 'Finding ledger JSON emitted',
|
|
27
|
+
executionIdentity: 'spawn_subagent:grok-child-da-01',
|
|
28
|
+
independentContextProven: true,
|
|
29
|
+
historyInherited: false
|
|
30
|
+
});
|
|
31
|
+
|
|
32
|
+
const selected = selectExecutionMode({
|
|
33
|
+
nativeSubagent,
|
|
34
|
+
artifactIsolation: createCapabilityEvidence({
|
|
35
|
+
mechanism: 'clean-slate-artifact-barrier',
|
|
36
|
+
classification: 'CONTEXT_ISOLATION_ONLY',
|
|
37
|
+
available: true
|
|
38
|
+
})
|
|
39
|
+
});
|
|
40
|
+
|
|
41
|
+
assert.strictEqual(selected.id, EXECUTION_MODES.TRUE_INDEPENDENT_AGENT.id);
|
|
42
|
+
assert.strictEqual(selected.isIndependentExecutionProven, true);
|
|
43
|
+
assert.strictEqual(nativeSubagent.commandOrApi, 'spawn_subagent');
|
|
44
|
+
});
|
|
45
|
+
|
|
46
|
+
test('GROK_SUBAGENTS=0 disables invocation even when grok binary exists', () => {
|
|
47
|
+
const grokBin = '/tmp/fake-grok-home/bin/grok';
|
|
48
|
+
const runtime = detectGrokRuntime(
|
|
49
|
+
{ GROK_HOME: '/tmp/fake-grok-home', GROK_SUBAGENTS: '0', HOME: '/tmp' },
|
|
50
|
+
fakeFs([grokBin])
|
|
51
|
+
);
|
|
52
|
+
|
|
53
|
+
assert.strictEqual(runtime.host, 'grok-cli');
|
|
54
|
+
assert.strictEqual(runtime.configurationSupported, true);
|
|
55
|
+
assert.strictEqual(runtime.invocationAvailable, false);
|
|
56
|
+
assert.strictEqual(runtime.executionProven, false);
|
|
57
|
+
assert.match(runtime.reason, /GROK_SUBAGENTS=0/);
|
|
58
|
+
|
|
59
|
+
const selected = selectExecutionMode({
|
|
60
|
+
nativeSubagent: createGrokCapabilityEvidence({
|
|
61
|
+
invocationAvailable: runtime.invocationAvailable,
|
|
62
|
+
reason: runtime.reason
|
|
63
|
+
}),
|
|
64
|
+
artifactIsolation: createCapabilityEvidence({
|
|
65
|
+
mechanism: 'clean-slate-artifact-barrier',
|
|
66
|
+
classification: 'CONTEXT_ISOLATION_ONLY',
|
|
67
|
+
available: true
|
|
68
|
+
})
|
|
69
|
+
});
|
|
70
|
+
|
|
71
|
+
assert.strictEqual(selected.id, EXECUTION_MODES.CONTEXT_ISOLATION_ONLY.id);
|
|
72
|
+
});
|
|
73
|
+
|
|
74
|
+
test('Grok binary present with default subagents maps to INVOCATION_AVAILABLE, not execution proven', () => {
|
|
75
|
+
const grokBin = '/tmp/fake-grok-home/bin/grok';
|
|
76
|
+
const runtime = detectGrokRuntime(
|
|
77
|
+
{ GROK_HOME: '/tmp/fake-grok-home', HOME: '/tmp' },
|
|
78
|
+
fakeFs([grokBin])
|
|
79
|
+
);
|
|
80
|
+
|
|
81
|
+
assert.strictEqual(runtime.invocationAvailable, true);
|
|
82
|
+
assert.strictEqual(runtime.executionProven, false);
|
|
83
|
+
|
|
84
|
+
const evidence = createGrokCapabilityEvidence({
|
|
85
|
+
invocationAvailable: true
|
|
86
|
+
});
|
|
87
|
+
|
|
88
|
+
const selected = selectExecutionMode({
|
|
89
|
+
nativeSubagent: evidence,
|
|
90
|
+
artifactIsolation: createCapabilityEvidence({
|
|
91
|
+
mechanism: 'clean-slate-artifact-barrier',
|
|
92
|
+
classification: 'CONTEXT_ISOLATION_ONLY',
|
|
93
|
+
available: true
|
|
94
|
+
})
|
|
95
|
+
});
|
|
96
|
+
|
|
97
|
+
assert.strictEqual(evidence.classification, 'INVOCATION_AVAILABLE');
|
|
98
|
+
assert.strictEqual(selected.id, EXECUTION_MODES.CONTEXT_ISOLATION_ONLY.id);
|
|
99
|
+
});
|
|
100
|
+
|
|
101
|
+
test('resume_from Maker transcript cannot activate TRUE_INDEPENDENT_AGENT', () => {
|
|
102
|
+
const tainted = createGrokCapabilityEvidence({
|
|
103
|
+
invocationAvailable: true,
|
|
104
|
+
childConversationId: 'grok-child-resume',
|
|
105
|
+
childModelResponse: 'I remember the Maker rationale',
|
|
106
|
+
executionIdentity: 'spawn_subagent:resume',
|
|
107
|
+
independentContextProven: true,
|
|
108
|
+
resumeFrom: 'maker-subagent-id'
|
|
109
|
+
});
|
|
110
|
+
|
|
111
|
+
const selected = selectExecutionMode({
|
|
112
|
+
nativeSubagent: tainted,
|
|
113
|
+
artifactIsolation: createCapabilityEvidence({
|
|
114
|
+
mechanism: 'clean-slate-artifact-barrier',
|
|
115
|
+
classification: 'CONTEXT_ISOLATION_ONLY',
|
|
116
|
+
available: true
|
|
117
|
+
})
|
|
118
|
+
});
|
|
119
|
+
|
|
120
|
+
assert.strictEqual(tainted.historyInherited, true);
|
|
121
|
+
assert.strictEqual(selected.id, EXECUTION_MODES.CONTEXT_ISOLATION_ONLY.id);
|
|
122
|
+
});
|
|
123
|
+
|
|
124
|
+
test('caveman:cavecrew-reviewer cannot serve as Devil\'s Advocate', () => {
|
|
125
|
+
const compressed = createCapabilityEvidence({
|
|
126
|
+
mechanism: 'caveman:cavecrew-reviewer',
|
|
127
|
+
classification: 'GROK_COMPRESSED_REVIEW_PRESET',
|
|
128
|
+
available: true,
|
|
129
|
+
invocationAvailable: true,
|
|
130
|
+
executionProven: true,
|
|
131
|
+
childConversationId: 'cave-1',
|
|
132
|
+
executionIdentity: 'cave-1',
|
|
133
|
+
childModelResponse: 'path:line: 🔴 BLOCKER: bug. fix it.',
|
|
134
|
+
modelExecutionProven: true,
|
|
135
|
+
independentContextProven: true
|
|
136
|
+
});
|
|
137
|
+
|
|
138
|
+
const selected = selectExecutionMode({
|
|
139
|
+
nativeSubagent: compressed,
|
|
140
|
+
artifactIsolation: createCapabilityEvidence({
|
|
141
|
+
mechanism: 'clean-slate-artifact-barrier',
|
|
142
|
+
classification: 'CONTEXT_ISOLATION_ONLY',
|
|
143
|
+
available: true
|
|
144
|
+
})
|
|
145
|
+
});
|
|
146
|
+
|
|
147
|
+
assert.strictEqual(selected.id, EXECUTION_MODES.CONTEXT_ISOLATION_ONLY.id);
|
|
148
|
+
assert.ok(GROK_FORBIDDEN_REVIEW_TYPES.includes('caveman:cavecrew-reviewer'));
|
|
149
|
+
});
|
|
150
|
+
|
|
151
|
+
test('Skill aliases resolve to canonical execution mode ids', () => {
|
|
152
|
+
assert.strictEqual(resolveExecutionModeId('NATIVE_SUBAGENT'), 'TRUE_INDEPENDENT_AGENT');
|
|
153
|
+
assert.strictEqual(resolveExecutionModeId('SDK_AGENT'), 'ISOLATED_AGENT_INSTANCE');
|
|
154
|
+
assert.strictEqual(resolveExecutionModeId('HEADLESS_SUBPROCESS'), 'FRESH_PROCESS_AGENT');
|
|
155
|
+
assert.strictEqual(resolveExecutionModeId('ARTIFACT_ISOLATED_REVIEW'), 'CONTEXT_ISOLATION_ONLY');
|
|
156
|
+
assert.strictEqual(EXECUTION_MODE_ALIASES.NATIVE_SUBAGENT, EXECUTION_MODES.TRUE_INDEPENDENT_AGENT.id);
|
|
157
|
+
});
|
|
158
|
+
|
|
159
|
+
test('Grok DA/Judge spawn plan is fresh, execute-only, and forbids compressed review types', () => {
|
|
160
|
+
const da = buildGrokReviewSpawnPlan({
|
|
161
|
+
role: 'devil-advocate',
|
|
162
|
+
iteration: 2,
|
|
163
|
+
artifactPaths: { goalContract: 'goal.md', gitDiff: 'diff.patch' }
|
|
164
|
+
});
|
|
165
|
+
|
|
166
|
+
assert.strictEqual(da.tool, 'spawn_subagent');
|
|
167
|
+
assert.strictEqual(da.subagent_type, 'devil-advocate');
|
|
168
|
+
assert.strictEqual(da.fallback_subagent_type, 'general-purpose');
|
|
169
|
+
assert.strictEqual(da.description, '[devil-advocate] review iteration 2');
|
|
170
|
+
assert.strictEqual(da.background, false);
|
|
171
|
+
assert.strictEqual(da.capability_mode, 'execute');
|
|
172
|
+
assert.strictEqual(da.isolation, 'none');
|
|
173
|
+
assert.strictEqual(da.resume_from, null);
|
|
174
|
+
assert.ok(da.forbidden_types.includes('caveman:cavecrew-reviewer'));
|
|
175
|
+
assert.strictEqual(da.artifactPaths.goalContract, 'goal.md');
|
|
176
|
+
|
|
177
|
+
const judge = buildGrokReviewSpawnPlan({ role: 'judge', iteration: 2 });
|
|
178
|
+
assert.strictEqual(judge.subagent_type, 'judge');
|
|
179
|
+
assert.strictEqual(judge.description, '[judge] evaluate iteration 2');
|
|
180
|
+
assert.strictEqual(judge.resume_from, null);
|
|
181
|
+
});
|
|
182
|
+
|
|
183
|
+
test('buildGrokReviewSpawnPlan rejects unknown roles', () => {
|
|
184
|
+
assert.throws(
|
|
185
|
+
() => buildGrokReviewSpawnPlan({ role: 'caveman:cavecrew-reviewer' }),
|
|
186
|
+
/must be devil-advocate or judge/
|
|
187
|
+
);
|
|
188
|
+
});
|
|
189
|
+
|
|
190
|
+
test('Grok proven spawn report discloses independent execution', () => {
|
|
191
|
+
const report = formatExecutionReport({
|
|
192
|
+
selectedMode: EXECUTION_MODES.TRUE_INDEPENDENT_AGENT,
|
|
193
|
+
capabilityRegistry: {
|
|
194
|
+
nativeSubagent: { invocationAvailable: true }
|
|
195
|
+
}
|
|
196
|
+
});
|
|
197
|
+
|
|
198
|
+
assert.match(report, /Execution Mode: TRUE_INDEPENDENT_AGENT/);
|
|
199
|
+
assert.match(report, /Independent LLM Execution: PROVEN/);
|
|
200
|
+
assert.match(report, /Native Subagent Invocation: AVAILABLE/);
|
|
201
|
+
});
|