devlyn-cli 2.9.3 → 2.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +3 -3
- package/CLAUDE.md +4 -4
- package/README.md +4 -4
- package/benchmark/auto-resolve/fixtures/SCHEMA.md +1 -1
- package/benchmark/auto-resolve/scripts/pair_evidence_contract.py +1 -0
- package/bin/devlyn.js +55 -7
- package/config/skills/_shared/adapters/README.md +2 -2
- package/config/skills/_shared/adapters/grok.md +77 -0
- package/config/skills/_shared/archive_run.py +35 -1
- package/config/skills/_shared/collect-codex-findings.py +15 -10
- package/config/skills/_shared/engine-doctor.sh +6 -3
- package/config/skills/_shared/engine-preflight.md +12 -5
- package/config/skills/_shared/resolve-bootstrap.py +630 -0
- package/config/skills/_shared/resolve-stop-hook.py +325 -0
- package/config/skills/_shared/state-phase-write.py +2264 -13
- package/config/skills/_shared/terminal-claim-check.py +537 -0
- package/config/skills/_shared/verify-merge-findings.py +363 -16
- package/config/skills/devlyn:engines/SKILL.md +1 -1
- package/config/skills/devlyn:queue/SKILL.md +12 -0
- package/config/skills/devlyn:resolve/SKILL.md +54 -73
- package/config/skills/devlyn:resolve/references/phases/surface-close.md +34 -0
- package/config/skills/devlyn:resolve/references/phases/verify.md +65 -96
- package/config/skills/devlyn:resolve/references/state-schema.md +25 -18
- package/optional-skills/devlyn:reap/SKILL.md +1 -1
- package/optional-skills/devlyn:reap/scripts/reap.sh +25 -13
- package/optional-skills/devlyn:reap/scripts/scan.sh +15 -1
- package/package.json +1 -1
- package/scripts/lint-skills.sh +131 -67
package/AGENTS.md
CHANGED
|
@@ -10,7 +10,7 @@ This contract serves one goal: any capable engine — Claude, GPT/Codex, or a fu
|
|
|
10
10
|
|
|
11
11
|
Seven rules govern every change. Cite them by name when a decision touches one.
|
|
12
12
|
|
|
13
|
-
1. **No workaround** — fix the root cause, never the symptom. No `any`, no `@ts-ignore`, no silent `catch`, no hardcoded fallback that hides a broken contract.
|
|
13
|
+
1. **No workaround** — fix the root cause, never the symptom. No `any`, no `@ts-ignore`, no silent `catch`, no hardcoded fallback that hides a broken contract. Explicit `--engine`, `--risk-probes`, and `--pair-verify` routes fail closed; automatic VERIFY pair is capability-gated and reports an unavailable-engine solo route.
|
|
14
14
|
2. **No overengineering** — smallest change that closes the goal. New abstractions require an observed failure mode they prevent. Subtractive-first: ask "what can I delete instead?" before writing anything new.
|
|
15
15
|
3. **No guesswork** — verify with the actual files, logs, diffs, and run output before forming conclusions. State the falsifiable prediction BEFORE the experiment; record raw results AFTER.
|
|
16
16
|
4. **Worldclass** — code that survives review at a non-trivial codebase. Zero CRITICAL, zero HIGH security/design findings on the shippable path.
|
|
@@ -36,7 +36,7 @@ ideate (optional) -> resolve -> ship
|
|
|
36
36
|
|
|
37
37
|
Each skill's `SKILL.md` is the source of truth for flags and workflow. Do not duplicate.
|
|
38
38
|
|
|
39
|
-
Engine roles: orchestrator = whichever CLI the user opened (this contract is symmetric with CLAUDE.md; measured status 2026-07-05: Claude Code + omp run the full phase-gated pipeline; in the iter-0061 minimal-repo trivial-add shape on this machine, Codex CLI also ran it when this AGENTS.md was present in the project (ordinary invocation, 4/4); without project AGENTS.md, the same shape silently skipped the pipeline (iter-0040 F6, 4/4). If you are Codex orchestrating `/devlyn:resolve`, the phase machinery is mandatory regardless of task size; if you will not run it, say so explicitly and stop — do not silently degrade to ad-hoc execution); executor (PLAN/IMPLEMENT/CLEANUP) defaults to `claude`, overridable per run with `--engine <name>` or durably via machine-local `.devlyn/engines.json` `{"executor": "<name>", "pair_judge_priority": ["<name>", ...]}`; pair judge = first available OTHER engine. Pins fail closed (`BLOCKED:<engine>-unavailable` / `BLOCKED:invalid-engine-config`); new engines plug in by shipping `_shared/adapters/<name>.md`. `/devlyn:engines` (no args) shows the role table + detected engines; `executor <name>` / `pair <name>,...` / `clear` manage the pins. The executor pin binds the orchestrator in plain conversation too, not only inside a skill run: when you would implement directly and executor is pinned to a non-default engine, route that work through the pin — via `/devlyn:resolve` (reads it at PHASE 0) or by delegating to that engine — instead of editing as `claude`; no pin / no file → unchanged (`claude`). The pair-judge pin stays pipeline-scoped.
|
|
39
|
+
Engine roles: orchestrator = whichever CLI the user opened (this contract is symmetric with CLAUDE.md; measured status 2026-07-05: Claude Code + omp run the full phase-gated pipeline; in the iter-0061 minimal-repo trivial-add shape on this machine, Codex CLI also ran it when this AGENTS.md was present in the project (ordinary invocation, 4/4); without project AGENTS.md, the same shape silently skipped the pipeline (iter-0040 F6, 4/4). If you are Codex orchestrating `/devlyn:resolve`, the phase machinery is mandatory regardless of task size; if you will not run it, say so explicitly and stop — do not silently degrade to ad-hoc execution); executor (PLAN/IMPLEMENT/CLEANUP) defaults to `claude`, overridable per run with `--engine <name>` or durably via machine-local `.devlyn/engines.json` `{"executor": "<name>", "pair_judge_priority": ["<name>", ...]}`; pair judge = first available OTHER engine, default for VERIFY whenever available; risk probes remain conditional and --no-pair opts out. Pins fail closed (`BLOCKED:<engine>-unavailable` / `BLOCKED:invalid-engine-config`); new engines plug in by shipping `_shared/adapters/<name>.md`. `/devlyn:engines` (no args) shows the role table + detected engines; `executor <name>` / `pair <name>,...` / `clear` manage the pins. The executor pin binds the orchestrator in plain conversation too, not only inside a skill run: when you would implement directly and executor is pinned to a non-default engine, route that work through the pin — via `/devlyn:resolve` (reads it at PHASE 0) or by delegating to that engine — instead of editing as `claude`; no pin / no file → unchanged (`claude`). The pair-judge pin stays pipeline-scoped.
|
|
40
40
|
|
|
41
41
|
Conversational handoff + loop engineering (default entry): the orchestrating model — not the user — invokes the skills. `/devlyn:queue` fronts the intent queue (`docs/specs/queue.md`): no args = status, `add <intent>`, `drain` = serial unattended drain per the contract below. Large agreed work is written to `docs/specs/<id>/spec.md` (always a spec file — the user's reviewed contract; free-form large assumes-and-logs instead of halting, zero-scope-signal goals still halt to ideate), summarized once for user review, then run hands-free via `--spec`. Outer loop per task: findings-backed verdicts (NEEDS_WORK, verify/build-gate exhaustion) get spec amendment + re-run, max 3 iterations; infrastructure / invalid-input / engine-availability / implement-empty BLOCKED verdicts surface immediately. Unattended queue drain (`docs/specs/queue.md`, strictly serial): assumptions may only take scope-narrowing, reversible, non-user-visible defaults — material ambiguity marks the item `[F] needs-review` and the drain continues.
|
|
42
42
|
|
|
@@ -81,7 +81,7 @@ No silent fallbacks.
|
|
|
81
81
|
- Fallbacks allowed only when widely accepted and harmless (CSS fallback fonts, CDN failover, image placeholders).
|
|
82
82
|
- Silent `catch` blocks are bugs.
|
|
83
83
|
- Logging is not user-visible error handling.
|
|
84
|
-
-
|
|
84
|
+
- Engine availability follows **No workaround** principle 1: explicit routes fail closed; automatic routes are capability-gated and report a solo skip when OTHER is unavailable. No fallback.
|
|
85
85
|
|
|
86
86
|
## Evidence Over Claim
|
|
87
87
|
|
package/CLAUDE.md
CHANGED
|
@@ -10,7 +10,7 @@ This contract serves one goal: any capable engine — Claude, GPT/Codex, or a fu
|
|
|
10
10
|
|
|
11
11
|
Seven rules govern every change. Cite them by name when a decision touches one.
|
|
12
12
|
|
|
13
|
-
1. **No workaround** — fix the root cause, never the symptom.
|
|
13
|
+
1. **No workaround** — fix the root cause, never the symptom. No `any`, no `@ts-ignore`, no silent `catch`, no hardcoded fallback that hides a broken contract. No config bypasses. Explicit `--engine`, `--risk-probes`, and `--pair-verify` routes fail closed when a required engine is unavailable. Automatic risk probes remain high-risk-gated; VERIFY pair is default-when-available. An unavailable OTHER engine routes automatic VERIFY solo with a reported skip. `--no-pair` and `--no-risk-probes` remain explicit opt-outs.
|
|
14
14
|
2. **No overengineering** — smallest change that closes the goal. New abstractions require an observed failure mode they prevent. Subtractive-first: ask "what can I delete instead?" before writing anything new.
|
|
15
15
|
3. **No guesswork** — verify with the actual files, logs, diffs, and run output before forming conclusions. State the falsifiable prediction BEFORE the experiment; record raw results AFTER. Retroactive prediction edits are dishonest.
|
|
16
16
|
4. **Worldclass** — code that survives review at a non-trivial codebase. Zero CRITICAL, zero HIGH security/design findings on the shippable path.
|
|
@@ -41,7 +41,7 @@ Each skill's `SKILL.md` is the source of truth for its flags and workflow — do
|
|
|
41
41
|
|---|---|---|
|
|
42
42
|
| Orchestrator — conversation, handoff, loop driving | whichever CLI you open (contract is symmetric: CLAUDE.md ↔ AGENTS.md). Measured status 2026-07-05: Claude Code + omp run the full phase-gated pipeline; in the iter-0061 minimal-repo trivial-add shape on this machine, Codex CLI also ran it when the project carried the devlyn AGENTS.md (ordinary invocation, 4/4); without project AGENTS.md, the same shape silently skipped the pipeline (iter-0040 F6, 4/4) | switch CLIs; the file artifacts (spec/queue/state) carry over |
|
|
43
43
|
| Executor — PLAN/IMPLEMENT/CLEANUP + primary VERIFY judge | `claude` | `--engine <name>` per run, or `/devlyn:engines executor <name>` (durable pin) |
|
|
44
|
-
| Pair judge — VERIFY
|
|
44
|
+
| Pair judge — default for VERIFY; conditional for risk probes | first available OTHER engine (claude↔codex) | `/devlyn:engines pair <name>,...`; `--no-pair` opts out |
|
|
45
45
|
|
|
46
46
|
`/devlyn:engines` with no args shows the current role table, detected engines, and how to pin or clear — the pins live in `.devlyn/engines.json`.
|
|
47
47
|
|
|
@@ -158,14 +158,14 @@ A finding without one of these forms is excluded. Vague findings produce vague f
|
|
|
158
158
|
|
|
159
159
|
## Codex invocation
|
|
160
160
|
|
|
161
|
-
When `/devlyn:resolve` or `/devlyn:ideate` route a phase to Codex (`--engine codex
|
|
161
|
+
When `/devlyn:resolve` or `/devlyn:ideate` route a phase to Codex (`--engine codex`, default VERIFY pair, or conditional risk-probe routing), the wrapper-form contract lives in `config/skills/_shared/codex-config.md` (or `.claude/skills/_shared/codex-config.md` once installed). Omit `-m <model>` — the CLI's current flagship is used automatically. MCP is not in the loop. If Codex is required and unavailable, stop with `BLOCKED:codex-unavailable` and setup guidance.
|
|
162
162
|
|
|
163
163
|
## Working Mode
|
|
164
164
|
|
|
165
165
|
- **Checkpoint with TaskCreate / TaskUpdate.** Long investigations or multi-phase work: create tasks at start, mark completed as each one closes — don't batch.
|
|
166
166
|
- **Don't stop early on token-budget concerns.** Context auto-compacts; the model resumes after compaction. Run the work to a real stopping point.
|
|
167
167
|
- **Persist across context windows via disk.** `/devlyn:resolve` writes `.devlyn/pipeline.state.json` plus per-phase log/findings under `.devlyn/runs/<run_id>/`; for ad-hoc long work use `HANDOFF.md` and resume with `@HANDOFF.md continue`.
|
|
168
|
-
- **Parallelize independent tool calls; reserve `Agent` subagents for independent fan-out or isolated-context verification** — a single perspective is the default
|
|
168
|
+
- **Parallelize independent tool calls; reserve `Agent` subagents for independent fan-out or isolated-context verification** — a single perspective is the default outside VERIFY's dual-judge path.
|
|
169
169
|
|
|
170
170
|
## Skill Boundary Policy
|
|
171
171
|
|
package/README.md
CHANGED
|
@@ -107,9 +107,9 @@ devlyn separates three engine roles:
|
|
|
107
107
|
|
|
108
108
|
- **Orchestrator** — the CLI you opened (Claude Code, Codex, or omp) that drives the conversation and loop. The contract is symmetric (`CLAUDE.md` ↔ `AGENTS.md`), so the same phase-gated pipeline runs whichever you launch; the file artifacts (spec, queue, state) carry over if you switch.
|
|
109
109
|
- **Executor** — PLAN / IMPLEMENT / CLEANUP plus the primary VERIFY judge. Defaults to `claude`.
|
|
110
|
-
- **Pair judge** — the first available *other* engine, for
|
|
110
|
+
- **Pair judge** — the first available *other* engine, default for VERIFY and conditional for risk probes.
|
|
111
111
|
|
|
112
|
-
`--engine claude` (default) is the canonical implementation surface for PLAN, IMPLEMENT, BUILD_GATE, and CLEANUP. VERIFY/JUDGE
|
|
112
|
+
`--engine claude` (default) is the canonical implementation surface for PLAN, IMPLEMENT, BUILD_GATE, and CLEANUP. VERIFY/JUDGE runs pair mode by default when the OTHER engine is available.
|
|
113
113
|
|
|
114
114
|
Pin roles durably with `/devlyn:engines` (no args shows the role table + detected engines; `executor <name>` / `pair <name>,...` / `clear` manage the pins, stored machine-local in `.devlyn/engines.json`). Pins fail closed: an unavailable pinned engine stops with `BLOCKED:<engine>-unavailable`, and a name with no `_shared/adapters/<name>.md` adapter stops with `BLOCKED:invalid-engine-config`. New engines plug in by shipping an adapter file — no skill changes.
|
|
115
115
|
|
|
@@ -119,7 +119,7 @@ Pin roles durably with `/devlyn:engines` (no args shows the role table + detecte
|
|
|
119
119
|
/devlyn:resolve "fix the auth bug" --engine codex # research-only
|
|
120
120
|
```
|
|
121
121
|
|
|
122
|
-
If Codex or Claude is absent when explicitly selected or
|
|
122
|
+
If Codex or Claude is absent when explicitly selected, or OTHER engine is absent under `--pair-verify`, the harness stops with `BLOCKED:<engine>-unavailable` and prints setup guidance. Automatic VERIFY absence is a reported solo route. Use `--no-pair` only when intentionally accepting solo VERIFY; use `--no-risk-probes` only when intentionally disabling automatic high-risk probes.
|
|
123
123
|
|
|
124
124
|
### Benchmark score runs
|
|
125
125
|
|
|
@@ -259,7 +259,7 @@ Selected during install. Run `npx devlyn-cli` again to add more.
|
|
|
259
259
|
|---|---|
|
|
260
260
|
| `playwright` | Playwright MCP — powers `/devlyn:resolve` BUILD_GATE browser tier (Chrome MCP → Playwright → curl fallback) |
|
|
261
261
|
|
|
262
|
-
> `--engine codex` and
|
|
262
|
+
> `--engine codex` and default-when-available VERIFY pair mode use the local `codex` CLI binary, not MCP. Install from https://platform.openai.com/docs/codex, run the current Codex auth/login flow, verify `codex --version`, then rerun.
|
|
263
263
|
|
|
264
264
|
</details>
|
|
265
265
|
|
|
@@ -123,7 +123,7 @@ Machine-readable acceptance criteria used by both `run-fixture.sh` verification
|
|
|
123
123
|
|
|
124
124
|
Fixtures with `metadata.json` `category: "high-risk"` must include at least
|
|
125
125
|
one resolve risk-trigger term in `metadata.intent` or `spec.md`, matching the
|
|
126
|
-
conditional
|
|
126
|
+
conditional risk-probe triggers used by `/devlyn:resolve`: security/auth,
|
|
127
127
|
money/pricing/tax/ledger, persistence/data mutation, idempotency/replay,
|
|
128
128
|
API/webhook/signature, allocation/scheduling/inventory/rollback/transaction,
|
|
129
129
|
priority, or output/response-shape contracts. This keeps future pair-evidence
|
package/bin/devlyn.js
CHANGED
|
@@ -770,14 +770,37 @@ function installClaudeCore() {
|
|
|
770
770
|
if (fs.existsSync(settingsPath)) {
|
|
771
771
|
try {
|
|
772
772
|
settings = JSON.parse(fs.readFileSync(settingsPath, 'utf8'));
|
|
773
|
-
} catch {
|
|
774
|
-
settings
|
|
773
|
+
} catch (error) {
|
|
774
|
+
throw new Error(`Cannot merge .claude/settings.json: ${error.message}`);
|
|
775
775
|
}
|
|
776
776
|
}
|
|
777
|
-
if (!settings
|
|
777
|
+
if (!settings || typeof settings !== 'object' || Array.isArray(settings)) {
|
|
778
|
+
throw new Error('Cannot merge .claude/settings.json: root must be a JSON object');
|
|
779
|
+
}
|
|
780
|
+
const hasOwnSetting = (key) => Object.prototype.hasOwnProperty.call(settings, key);
|
|
781
|
+
let settingsChanged = false;
|
|
782
|
+
if (!hasOwnSetting('env')) {
|
|
783
|
+
settings.env = {};
|
|
784
|
+
settingsChanged = true;
|
|
785
|
+
}
|
|
786
|
+
if (!settings.env || typeof settings.env !== 'object' || Array.isArray(settings.env)) {
|
|
787
|
+
throw new Error('Cannot merge .claude/settings.json: env must be a JSON object');
|
|
788
|
+
}
|
|
778
789
|
// Auto-allow pipeline state directory and common git commands so resolve doesn't prompt
|
|
779
|
-
if (!
|
|
780
|
-
|
|
790
|
+
if (!hasOwnSetting('permissions')) {
|
|
791
|
+
settings.permissions = {};
|
|
792
|
+
settingsChanged = true;
|
|
793
|
+
}
|
|
794
|
+
if (!settings.permissions || typeof settings.permissions !== 'object' || Array.isArray(settings.permissions)) {
|
|
795
|
+
throw new Error('Cannot merge .claude/settings.json: permissions must be a JSON object');
|
|
796
|
+
}
|
|
797
|
+
if (!Object.prototype.hasOwnProperty.call(settings.permissions, 'allow')) {
|
|
798
|
+
settings.permissions.allow = [];
|
|
799
|
+
settingsChanged = true;
|
|
800
|
+
}
|
|
801
|
+
if (!Array.isArray(settings.permissions.allow)) {
|
|
802
|
+
throw new Error('Cannot merge .claude/settings.json: permissions.allow must be an array');
|
|
803
|
+
}
|
|
781
804
|
const pipelinePermissions = [
|
|
782
805
|
'Write(.devlyn/**)',
|
|
783
806
|
'Edit(.devlyn/**)',
|
|
@@ -787,7 +810,6 @@ function installClaudeCore() {
|
|
|
787
810
|
'Bash(git status *)',
|
|
788
811
|
'Bash(git log *)',
|
|
789
812
|
];
|
|
790
|
-
let settingsChanged = false;
|
|
791
813
|
for (const perm of pipelinePermissions) {
|
|
792
814
|
if (!settings.permissions.allow.includes(perm)) {
|
|
793
815
|
settings.permissions.allow.push(perm);
|
|
@@ -798,9 +820,35 @@ function installClaudeCore() {
|
|
|
798
820
|
settings.env.CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS = '1';
|
|
799
821
|
settingsChanged = true;
|
|
800
822
|
}
|
|
823
|
+
if (!hasOwnSetting('hooks')) {
|
|
824
|
+
settings.hooks = {};
|
|
825
|
+
settingsChanged = true;
|
|
826
|
+
}
|
|
827
|
+
if (!settings.hooks || typeof settings.hooks !== 'object' || Array.isArray(settings.hooks)) {
|
|
828
|
+
throw new Error('Cannot merge .claude/settings.json: hooks must be a JSON object');
|
|
829
|
+
}
|
|
830
|
+
if (!Object.prototype.hasOwnProperty.call(settings.hooks, 'Stop')) {
|
|
831
|
+
settings.hooks.Stop = [];
|
|
832
|
+
settingsChanged = true;
|
|
833
|
+
}
|
|
834
|
+
if (!Array.isArray(settings.hooks.Stop)) {
|
|
835
|
+
throw new Error('Cannot merge .claude/settings.json: hooks.Stop must be an array');
|
|
836
|
+
}
|
|
837
|
+
const stopHookCommand = 'python3 "$CLAUDE_PROJECT_DIR/.claude/skills/_shared/resolve-stop-hook.py"';
|
|
838
|
+
const stopHookInstalled = settings.hooks.Stop.some((entry) => (
|
|
839
|
+
entry && Array.isArray(entry.hooks) && entry.hooks.some((hook) => (
|
|
840
|
+
hook && hook.type === 'command' && hook.command === stopHookCommand
|
|
841
|
+
))
|
|
842
|
+
));
|
|
843
|
+
if (!stopHookInstalled) {
|
|
844
|
+
settings.hooks.Stop.push({
|
|
845
|
+
hooks: [{ type: 'command', command: stopHookCommand, timeout: 30 }],
|
|
846
|
+
});
|
|
847
|
+
settingsChanged = true;
|
|
848
|
+
}
|
|
801
849
|
if (settingsChanged) {
|
|
802
850
|
fs.writeFileSync(settingsPath, JSON.stringify(settings, null, 2) + '\n');
|
|
803
|
-
log(' → settings.json (agent teams + pipeline permissions)', 'dim');
|
|
851
|
+
log(' → settings.json (agent teams + pipeline permissions + Stop hook)', 'dim');
|
|
804
852
|
}
|
|
805
853
|
|
|
806
854
|
// Configure global Claude Code settings (~/.claude/settings.json)
|
|
@@ -46,8 +46,8 @@ Keep each section to ≤ 8 lines. Adapters are deltas, not full prompts. If an a
|
|
|
46
46
|
|
|
47
47
|
Adapters may add two more sections beyond the base format:
|
|
48
48
|
|
|
49
|
-
- **`## Role eligibility`** — fixed ASCII fields, one per line: `executor: yes|no`, `pair_judge: yes|no`. Absent section = eligible for both roles (every adapter shipped before iter-0051). Present only when a backend is structurally unable
|
|
50
|
-
- **`## Invocation`** — for backends called over an API
|
|
49
|
+
- **`## Role eligibility`** — fixed ASCII fields, one per line: `executor: yes|no`, `pair_judge: yes|no`. Absent section = eligible for both roles (every adapter shipped before iter-0051). Present only when a backend is structurally unable or not certified for a role — e.g. a completion-only local model has no tool/file-edit access, so it declares `executor: no`. `_shared/engine-preflight.md` and `/devlyn:engines` read this section as a structured marker, not prose, so it validates mechanically.
|
|
50
|
+
- **`## Invocation`** — for backends called over an API or CLI backends with non-default invocation contracts: the concrete request shape (endpoint, JSON body, timeout) and the adapter-declared availability probe (`_shared/engine-preflight.md#role-resolution`'s "an adapter may declare a different probe" hook).
|
|
51
51
|
|
|
52
52
|
## When to add a new adapter
|
|
53
53
|
|
|
@@ -0,0 +1,77 @@
|
|
|
1
|
+
# Grok adapter
|
|
2
|
+
|
|
3
|
+
## Role eligibility
|
|
4
|
+
|
|
5
|
+
executor: no
|
|
6
|
+
pair_judge: yes
|
|
7
|
+
|
|
8
|
+
Certification status: the pair-judge seat is wired, isolated, reachable, and probe-capable, but not emission-certified; P-0079-B measured 4/6, and P-0079-F failed emission because a narration preamble was welded to output line 1; a durable `pair grok` pin remains gated on follow-up emission registration and the standing seat-fitness rule.
|
|
9
|
+
|
|
10
|
+
## Invocation
|
|
11
|
+
|
|
12
|
+
**Availability probe**: `command -v grok >/dev/null 2>&1`; record
|
|
13
|
+
`grok --version` as evidence. The probe is necessary, not sufficient: auth or
|
|
14
|
+
wrapper failures after it passes are the same fail-closed availability class as
|
|
15
|
+
a failed probe (`BLOCKED:grok-unavailable` for an explicit route; an automatic
|
|
16
|
+
escalation remains a reported solo skip).
|
|
17
|
+
|
|
18
|
+
```bash
|
|
19
|
+
# $ISO_HOME = ephemeral dir seeded with a copied auth.json + agent_id, and a
|
|
20
|
+
# config.toml containing:
|
|
21
|
+
# [plugins]
|
|
22
|
+
# disabled = ["telegram", "vercel"]
|
|
23
|
+
# enabled = []
|
|
24
|
+
#
|
|
25
|
+
# [skills]
|
|
26
|
+
# ignore = ["~/.agents", "~/.claude", "~/.cursor"]
|
|
27
|
+
# $NEUTRAL = empty dir outside any git repo
|
|
28
|
+
# $REPO = absolute repo root (redirects MUST be absolute — cwd is $NEUTRAL)
|
|
29
|
+
cd "$NEUTRAL"
|
|
30
|
+
GROK_HOME="$ISO_HOME" \
|
|
31
|
+
GROK_CLAUDE_MCPS_ENABLED=false GROK_CURSOR_MCPS_ENABLED=false \
|
|
32
|
+
GROK_CLAUDE_SKILLS_ENABLED=false GROK_CURSOR_SKILLS_ENABLED=false \
|
|
33
|
+
GROK_CLAUDE_HOOKS_ENABLED=false GROK_CURSOR_HOOKS_ENABLED=false \
|
|
34
|
+
GROK_CLAUDE_AGENTS_ENABLED=false GROK_CLAUDE_RULES_ENABLED=false \
|
|
35
|
+
GROK_MEMORY=0 \
|
|
36
|
+
python3 "$DEVLYN_SHARED_DIR/run-bounded.py" 600 -- grok -p "<judge prompt>" \
|
|
37
|
+
--permission-mode dontAsk --no-memory \
|
|
38
|
+
--tools "read_file,grep,list_dir,run_terminal_cmd" \
|
|
39
|
+
--disallowed-tools "Agent,use_tool,search_tool" \
|
|
40
|
+
--allow 'Bash(<repo probe command family>)' \
|
|
41
|
+
--reasoning-effort medium \
|
|
42
|
+
> "$REPO/.devlyn/grok-judge.stdout" 2> "$REPO/.devlyn/grok-judge.stderr"
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
Copy `auth.json` into `$ISO_HOME` for every run and seed its `config.toml` with
|
|
46
|
+
the shown `[plugins]` and `[skills] ignore` entries.
|
|
47
|
+
`$NEUTRAL` and `$ISO_HOME` must be on paths that do not embed a project or repo identifier.
|
|
48
|
+
`--permission-mode plan` is forbidden headless because it silently blocks
|
|
49
|
+
all tools and returns an empty review; `--always-approve` is banned because it
|
|
50
|
+
auto-approves writes. `--tools` and `--disallowed-tools` are headless-only (the
|
|
51
|
+
TUI warns and ignores them); when both are present, the disallow list wins and
|
|
52
|
+
removes tools entirely rather than merely gating execution.
|
|
53
|
+
|
|
54
|
+
The `--allow` rule must cover the full probe-command family the judge may need,
|
|
55
|
+
and the judge prompt must name the exact permitted command(s). Derive the prompt
|
|
56
|
+
anchor and allow-list string identically from one source, in this precedence:
|
|
57
|
+
the solo-headroom hypothesis's backticked observable command when present;
|
|
58
|
+
otherwise the backticked commands in the spec's `## Verification` bullets;
|
|
59
|
+
otherwise the repo's existing test/CLI runner. If none exists, the probe
|
|
60
|
+
obligation does not arise and a static review is correct for that spec.
|
|
61
|
+
The documented `Bash(<command>)` form matches the exact command or its prefix;
|
|
62
|
+
the trailing ` *` form fails a no-argument command.
|
|
63
|
+
|
|
64
|
+
Grok's deny semantics are not Claude-parity: an out-of-allowlist command silently
|
|
65
|
+
truncates the review at exit 0. Treat the resulting collector rejection as
|
|
66
|
+
`BLOCKED` for `verify.pair.emission-contract`, never PASS. After a non-timeout
|
|
67
|
+
spawn, run:
|
|
68
|
+
|
|
69
|
+
```bash
|
|
70
|
+
python3 "$DEVLYN_SHARED_DIR/collect-codex-findings.py" \
|
|
71
|
+
--devlyn-dir "$REPO/.devlyn" \
|
|
72
|
+
--stdout-file grok-judge.stdout \
|
|
73
|
+
--summary-out grok-judge.summary.json
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
On exit 124, write `.devlyn/verify.pair.timeout.json` with
|
|
77
|
+
`{"engine": "grok", "budget_seconds": 600}` before merge.
|
|
@@ -27,7 +27,19 @@ PER_RUN_PATTERNS = (
|
|
|
27
27
|
"*.findings.jsonl",
|
|
28
28
|
"*.log.md",
|
|
29
29
|
"fix-batch.round-*.json",
|
|
30
|
+
"closure-durability.round-*.json",
|
|
31
|
+
"resolve-stop-hook.*.json",
|
|
30
32
|
"criteria.generated.md",
|
|
33
|
+
"goal.raw.txt",
|
|
34
|
+
"surface-close.input.patch",
|
|
35
|
+
"surface-close.prompt",
|
|
36
|
+
"surface-close.output.json",
|
|
37
|
+
"surface-close.stdout",
|
|
38
|
+
# Mutation workers retain the exact session JSONL identified by their
|
|
39
|
+
# dispatch receipt. Round-scoped root files avoid engine-global scans.
|
|
40
|
+
"implement.worker-session.*.jsonl",
|
|
41
|
+
"surface-close.worker-session.*.jsonl",
|
|
42
|
+
"cleanup.worker-session.*.jsonl",
|
|
31
43
|
"risk-probes.jsonl",
|
|
32
44
|
# Probe scripts referenced by risk-probes.jsonl are archived by
|
|
33
45
|
# move_probe_scripts() to preserve probes/<file> layout.
|
|
@@ -159,6 +171,15 @@ def self_test() -> int:
|
|
|
159
171
|
)
|
|
160
172
|
for name in (
|
|
161
173
|
"risk-probes.jsonl",
|
|
174
|
+
"goal.raw.txt",
|
|
175
|
+
"surface-close.input.patch",
|
|
176
|
+
"surface-close.prompt",
|
|
177
|
+
"surface-close.output.json",
|
|
178
|
+
"surface-close.stdout",
|
|
179
|
+
"implement.worker-session.0.jsonl",
|
|
180
|
+
"surface-close.worker-session.0.jsonl",
|
|
181
|
+
"cleanup.worker-session.1.jsonl",
|
|
182
|
+
"resolve-stop-hook.run-1.123.456.json",
|
|
162
183
|
"verify.pair.findings.jsonl",
|
|
163
184
|
"verify-merge.summary.json",
|
|
164
185
|
"codex-judge.stdout",
|
|
@@ -169,13 +190,24 @@ def self_test() -> int:
|
|
|
169
190
|
(devlyn / name).write_text("{}\n", encoding="utf-8")
|
|
170
191
|
(devlyn / "probes").mkdir()
|
|
171
192
|
(devlyn / "probes" / "P1.py").write_text("print('probe')\n", encoding="utf-8")
|
|
193
|
+
global_rollout = devlyn / "engine-sessions" / "rollout-global.jsonl"
|
|
194
|
+
global_rollout.parent.mkdir()
|
|
195
|
+
global_rollout.write_text("{}\n", encoding="utf-8")
|
|
172
196
|
run_id = read_run_id(devlyn)
|
|
173
197
|
assert run_id == "run-1", run_id
|
|
174
198
|
moved = move_artifacts(devlyn, devlyn / "runs" / run_id)
|
|
175
|
-
assert moved >=
|
|
199
|
+
assert moved >= 12, moved
|
|
176
200
|
for name in (
|
|
177
201
|
"pipeline.state.json",
|
|
178
202
|
"risk-probes.jsonl",
|
|
203
|
+
"goal.raw.txt",
|
|
204
|
+
"surface-close.input.patch",
|
|
205
|
+
"surface-close.prompt",
|
|
206
|
+
"surface-close.output.json",
|
|
207
|
+
"surface-close.stdout",
|
|
208
|
+
"implement.worker-session.0.jsonl",
|
|
209
|
+
"surface-close.worker-session.0.jsonl",
|
|
210
|
+
"cleanup.worker-session.1.jsonl",
|
|
179
211
|
"probes/P1.py",
|
|
180
212
|
"verify.pair.findings.jsonl",
|
|
181
213
|
"verify-merge.summary.json",
|
|
@@ -185,6 +217,8 @@ def self_test() -> int:
|
|
|
185
217
|
"claude-judge.stderr",
|
|
186
218
|
):
|
|
187
219
|
assert (devlyn / "runs" / run_id / name).is_file(), name
|
|
220
|
+
assert global_rollout.is_file(), "engine-global session files must stay untouched"
|
|
221
|
+
assert not (devlyn / "runs" / run_id / global_rollout.name).exists()
|
|
188
222
|
|
|
189
223
|
bad = pathlib.Path(tmp) / "bad"
|
|
190
224
|
bad.mkdir()
|
|
@@ -61,9 +61,7 @@ def collect(stdout_path: pathlib.Path) -> tuple[list[dict[str, Any]], dict[str,
|
|
|
61
61
|
if severity not in FINDING_SEVERITIES:
|
|
62
62
|
raise SystemExit(f"error: finding missing valid severity at {stdout_path}:{line_no}")
|
|
63
63
|
findings.append(item)
|
|
64
|
-
if not findings and summary is None:
|
|
65
|
-
raise SystemExit("error: Codex pair-JUDGE stdout contained no JSONL findings or PASS line")
|
|
66
|
-
if summary and summary.get("verdict") in {"NEEDS_WORK", "FAIL", "BLOCKED"} and not findings:
|
|
64
|
+
if not findings and (summary is None or summary.get("verdict") != "PASS"):
|
|
67
65
|
raise SystemExit("error: non-PASS SUMMARY without JSONL findings")
|
|
68
66
|
return findings, summary
|
|
69
67
|
|
|
@@ -90,13 +88,20 @@ def self_test() -> int:
|
|
|
90
88
|
assert "invalid JSON numeric constant: NaN" in str(exc)
|
|
91
89
|
else:
|
|
92
90
|
raise AssertionError("NaN Codex stdout finding must not normalize")
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
91
|
+
rejection_cases = (
|
|
92
|
+
("", "no SUMMARY line"),
|
|
93
|
+
('# SUMMARY {}\n', "empty SUMMARY object"),
|
|
94
|
+
('# SUMMARY {"verdict":"pass"}\n', "lowercase pass"),
|
|
95
|
+
('# SUMMARY {"verdict":"UNKNOWN"}\n', "unknown verdict"),
|
|
96
|
+
)
|
|
97
|
+
for text, label in rejection_cases:
|
|
98
|
+
stdout_path.write_text(text, encoding="utf-8")
|
|
99
|
+
try:
|
|
100
|
+
collect(stdout_path)
|
|
101
|
+
except SystemExit as exc:
|
|
102
|
+
assert str(exc) == "error: non-PASS SUMMARY without JSONL findings"
|
|
103
|
+
else:
|
|
104
|
+
raise AssertionError(f"{label} must not normalize to PASS")
|
|
100
105
|
return 0
|
|
101
106
|
|
|
102
107
|
|
|
@@ -17,12 +17,13 @@ set -uo pipefail
|
|
|
17
17
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]:-$0}")" && pwd)"
|
|
18
18
|
ADAPTERS_DIR="$SCRIPT_DIR/adapters"
|
|
19
19
|
|
|
20
|
-
TARGETS=(claude codex omp pi)
|
|
21
|
-
KINDS=(cli-engine cli-engine cli-engine orchestrator-only)
|
|
22
|
-
BINARIES=(claude codex omp "")
|
|
20
|
+
TARGETS=(claude codex grok omp pi)
|
|
21
|
+
KINDS=(cli-engine cli-engine cli-engine cli-engine orchestrator-only)
|
|
22
|
+
BINARIES=(claude codex grok omp "")
|
|
23
23
|
INSTALL_HINTS=(
|
|
24
24
|
"see https://docs.anthropic.com/en/docs/claude-code"
|
|
25
25
|
"npm install -g @openai/codex"
|
|
26
|
+
"see https://docs.x.ai/build/overview (install + browser auth)"
|
|
26
27
|
"brew install can1357/tap/omp"
|
|
27
28
|
""
|
|
28
29
|
)
|
|
@@ -88,6 +89,8 @@ for i in "${!TARGETS[@]}"; do
|
|
|
88
89
|
;;
|
|
89
90
|
esac
|
|
90
91
|
|
|
92
|
+
[ "$target" = 'grok' ] && note='not emission-certified; BLOCKED pair source'
|
|
93
|
+
|
|
91
94
|
printf '%-8s %-17s %-8s %-8s %-14s %-12s %s\n' \
|
|
92
95
|
"$target" "$kind" "$binary" "$adapter" "$role" "$pin_eligible" "$note"
|
|
93
96
|
done
|
|
@@ -4,12 +4,12 @@ Used by `/devlyn:resolve` and `/devlyn:ideate`. One shared availability rule so
|
|
|
4
4
|
|
|
5
5
|
## Rule
|
|
6
6
|
|
|
7
|
-
Each skill resolves the effective engine from its own SKILL.md default plus any explicit `--engine` flag passed by the user. `/devlyn:resolve` also computes conditional
|
|
7
|
+
Each skill resolves the effective engine from its own SKILL.md default plus any explicit `--engine` flag passed by the user. `/devlyn:resolve` also computes the default VERIFY-pair route and conditional risk-probe requirements before the phase that needs the OTHER engine.
|
|
8
8
|
|
|
9
9
|
Engine requirements have two classes:
|
|
10
10
|
|
|
11
11
|
- **Explicit routes** — `--engine`, `--risk-probes`, `--pair-verify`. These are promises. If the required engine is unavailable, fail closed with `BLOCKED:<engine>-unavailable` and never downgrade to solo.
|
|
12
|
-
- **Automatic
|
|
12
|
+
- **Automatic routes** — high-risk risk-probes remain conditional; VERIFY pair-JUDGE is selected by default unless `--no-pair` was passed. Both require an available OTHER engine. If it is absent, proceed solo and report the skipped route and reason. This is route selection, not a fallback.
|
|
13
13
|
|
|
14
14
|
## Role resolution
|
|
15
15
|
|
|
@@ -32,11 +32,11 @@ When a run or phase requires engine `<name>` (claude, codex, omp, or any adapter
|
|
|
32
32
|
|
|
33
33
|
Never prompt the user mid-pipeline. Missing engines for explicit routes are BLOCKED states, not silent fallbacks. Missing OTHER engines for automatic escalations are reported solo-skips, not fallbacks — the auto route was never selected.
|
|
34
34
|
|
|
35
|
-
Per-skill defaults: `/devlyn:resolve` uses Claude for PLAN/IMPLEMENT when the orchestrator has Claude Code's native Agent primitive; Codex CLI and oh-my-pi orchestrators use their own fresh worker as the default route (Codex child process, omp native `task`) so the pipeline still has structural phase isolation. VERIFY
|
|
35
|
+
Per-skill defaults: `/devlyn:resolve` uses Claude for PLAN/IMPLEMENT when the orchestrator has Claude Code's native Agent primitive; Codex CLI and oh-my-pi orchestrators use their own fresh worker as the default route (Codex child process, omp native `task`) so the pipeline still has structural phase isolation. VERIFY invokes the OTHER engine by default when available unless `--no-pair` was passed. `/devlyn:ideate` defaults to Claude; `--engine` selects the elicitation/normalization adapter, not an automatic cross-model challenge phase. Any future ideate read-only critique must follow `_shared/codex-config.md` isolation rules. Each SKILL.md flag block is source of truth for that skill's default.
|
|
36
36
|
|
|
37
37
|
## What a skill must report after a BLOCKED engine check
|
|
38
38
|
|
|
39
|
-
When an engine required by the selected route or
|
|
39
|
+
When an engine required by the selected route or default VERIFY pair route is absent, the final user-facing report/summary shows the requested route, the missing engine, and setup steps:
|
|
40
40
|
|
|
41
41
|
```
|
|
42
42
|
Engine: claude + codex pair required
|
|
@@ -48,4 +48,11 @@ Do not report a downgraded successful run when a required engine is missing.
|
|
|
48
48
|
|
|
49
49
|
## Canonical cross-engine invocations
|
|
50
50
|
|
|
51
|
-
After the availability check passes
|
|
51
|
+
After the availability check passes, Codex keeps the monitored `codex-monitored.sh`
|
|
52
|
+
path; every other resolved OTHER engine follows `_shared/adapters/<name>.md`
|
|
53
|
+
`## Invocation`. Capture `.devlyn/<name>-judge.stdout`, then normalize it with
|
|
54
|
+
`collect-codex-findings.py --devlyn-dir "<abs repo>/.devlyn" --stdout-file
|
|
55
|
+
<name>-judge.stdout`. A non-zero collector exit writes no canonical findings:
|
|
56
|
+
the pair source is `BLOCKED` for `verify.pair.emission-contract`, and unparsed
|
|
57
|
+
stdout remains diagnostic-only rather than being merged. On exit 124, the
|
|
58
|
+
timeout marker's `engine` field is the resolved `<name>`.
|