devlyn-cli 2.9.3 → 2.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +3 -3
- package/CLAUDE.md +4 -4
- package/README.md +4 -4
- package/benchmark/auto-resolve/fixtures/SCHEMA.md +1 -1
- package/benchmark/auto-resolve/scripts/pair_evidence_contract.py +1 -0
- package/bin/devlyn.js +55 -7
- package/config/skills/_shared/adapters/README.md +2 -2
- package/config/skills/_shared/adapters/grok.md +77 -0
- package/config/skills/_shared/archive_run.py +35 -1
- package/config/skills/_shared/collect-codex-findings.py +15 -10
- package/config/skills/_shared/engine-doctor.sh +6 -3
- package/config/skills/_shared/engine-preflight.md +12 -5
- package/config/skills/_shared/resolve-bootstrap.py +630 -0
- package/config/skills/_shared/resolve-stop-hook.py +325 -0
- package/config/skills/_shared/state-phase-write.py +2264 -13
- package/config/skills/_shared/terminal-claim-check.py +537 -0
- package/config/skills/_shared/verify-merge-findings.py +363 -16
- package/config/skills/devlyn:engines/SKILL.md +1 -1
- package/config/skills/devlyn:queue/SKILL.md +12 -0
- package/config/skills/devlyn:resolve/SKILL.md +54 -73
- package/config/skills/devlyn:resolve/references/phases/surface-close.md +34 -0
- package/config/skills/devlyn:resolve/references/phases/verify.md +65 -96
- package/config/skills/devlyn:resolve/references/state-schema.md +25 -18
- package/optional-skills/devlyn:reap/SKILL.md +1 -1
- package/optional-skills/devlyn:reap/scripts/reap.sh +25 -13
- package/optional-skills/devlyn:reap/scripts/scan.sh +15 -1
- package/package.json +1 -1
- package/scripts/lint-skills.sh +131 -67
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: devlyn:resolve
|
|
3
|
-
description: Hands-free pipeline for any coding task — bug fix, feature, refactor, debug, modify, PR review. Free-form goal or formal spec input. Plan → Implement → Build-gate → Cleanup → Verify (fresh subagent, findings-only). Mechanical-first verification;
|
|
3
|
+
description: Hands-free pipeline for any coding task — bug fix, feature, refactor, debug, modify, PR review. Free-form goal or formal spec input. Plan → Implement → Build-gate → Cleanup → Verify (fresh subagent, findings-only). Mechanical-first verification; Verify dual-judge is default-when-available. Use when the user says "resolve this", "fix this", "implement this", "refactor this", "debug this", "review this PR", or wants hands-off completion.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
Orchestrator for the 2-skill harness pipeline. One fresh worker per phase; file-based handoff via `.devlyn/pipeline.state.json`. VERIFY spawns a fresh-context worker so independence is structural — not advisory.
|
|
@@ -18,13 +18,13 @@ Hands-free. Measured by how far we get without human intervention.
|
|
|
18
18
|
|
|
19
19
|
1. Do not prompt the user mid-pipeline. When tempted to ask, pick the safe default, proceed, and log it in the final report.
|
|
20
20
|
2. Engine availability: follow `_shared/engine-preflight.md` for the explicit-route-vs-automatic-escalation distinction and BLOCKED-vs-skip behavior. Explicit routes (`--engine`, `--risk-probes`, `--pair-verify`) never downgrade to solo; unavailable auto-escalations proceed solo and report the skip.
|
|
21
|
-
3.
|
|
21
|
+
3. Order: PLAN → RISK_PROBES? → IMPLEMENT → SURFACE_CLOSE? → BUILD_GATE → CLEANUP → VERIFY → FINAL_REPORT. No others.
|
|
22
22
|
4. Orchestrator does not write code. It parses input, spawns phases, reads state, branches on verdicts, emits the report.
|
|
23
|
-
5.
|
|
23
|
+
5. Halt only on unrecoverable worker failure, empty IMPLEMENT, exhausted BUILD_GATE/VERIFY `max_rounds`, or SURFACE_CLOSE timeout / input-or-prompt mismatch / attestation failure / execution-audit violation / out-of-surface delta / rollback failure; adjudication-malformed SURFACE_CLOSE continues after successful rollback plus transcript write-audit; otherwise continue.
|
|
24
24
|
</autonomy_contract>
|
|
25
25
|
|
|
26
26
|
<harness_principles>
|
|
27
|
-
Every phase
|
|
27
|
+
Every phase applies Subtractive-first / Goal-locked / No-workaround / Evidence, loaded by file or inline; Codex routes receive it inline.
|
|
28
28
|
</harness_principles>
|
|
29
29
|
|
|
30
30
|
<runtime_paths>
|
|
@@ -54,8 +54,9 @@ Each phase routes to an engine and prepends the per-engine adapter header from `
|
|
|
54
54
|
- Claude Code phases: spawn `Agent` (`mode: "bypassPermissions"`); prompt = adapter-header + canonical-body + task-context.
|
|
55
55
|
- Codex CLI phases: shell out via `bash "$CODEX_MONITORED_PATH"` with the same compounded prompt. Each `codex exec` child is a new session/fresh context. The wrapper closes stdin and emits a heartbeat. No MCP. The wrapper call is foreground-blocking — never a background shell (`run_in_background`, `&`, `nohup`), never end your message while it runs: headless print-mode wind-down kills backgrounded children (0-byte delivery); the heartbeat is the observability channel.
|
|
56
56
|
- oh-my-pi phases: spawn the native `task` tool with a fresh `context` containing adapter-header + canonical-body + task-context. Capture the task result into `.devlyn/<phase>.stdout` and any tool error into `.devlyn/<phase>.stderr` before updating state. If the `task` tool is unavailable for an omp-routed phase, write the current phase verdict as `"BLOCKED"` and report `BLOCKED:fresh-context-unavailable`; do not fall back to same-context execution or a nested `omp -p` subprocess.
|
|
57
|
-
- Default engine: Claude for PLAN / IMPLEMENT / BUILD_GATE / CLEANUP when the orchestrator has Claude Code's native `Agent` primitive; otherwise the default route is the current CLI's fresh worker (`codex` child process for Codex CLI, native `task` for oh-my-pi). Role resolution (`_shared/engine-preflight.md#role-resolution`): explicit `--engine` flag > machine-local `.devlyn/engines.json` `executor` pin > orchestrator-supported default above. Explicit routes and pins still fail closed when their required engine/spawn channel is unavailable.
|
|
58
|
-
-
|
|
57
|
+
- Default engine: Claude for PLAN / IMPLEMENT / BUILD_GATE / CLEANUP when the orchestrator has Claude Code's native `Agent` primitive; otherwise the default route is the current CLI's fresh worker (`codex` child process for Codex CLI, native `task` for oh-my-pi). Role resolution (`_shared/engine-preflight.md#role-resolution`): explicit `--engine` flag > machine-local `.devlyn/engines.json` `executor` pin > orchestrator-supported default above. Explicit routes and pins still fail closed when their required engine/spawn channel is unavailable. VERIFY/JUDGE selects the OTHER engine by default when available — OTHER = first adapter-valid, non-primary, available entry of `engines.json` `pair_judge_priority`, or the binary claude↔codex complement when unset.
|
|
58
|
+
- The `--engine` flag does not disable default pairing: the second judge uses the OTHER engine by default when available.
|
|
59
|
+
- Multi-LLM evolution: when a new model adapter ships in `_shared/adapters/`, that engine becomes selectable for the roles its adapter declares eligible without further skill changes (NORTH-STAR.md "Multi-LLM evolution direction").
|
|
59
60
|
</engine_routing>
|
|
60
61
|
|
|
61
62
|
<modes>
|
|
@@ -67,43 +68,34 @@ Three input shapes:
|
|
|
67
68
|
</modes>
|
|
68
69
|
|
|
69
70
|
<post_implement_invariant>
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
- CLEANUP may only mutate files in the cleanup allowlist (tooling artifacts, dead code added by this diff, doc references this diff invalidated). Other paths trigger revert.
|
|
73
|
-
- VERIFY runs in a fresh worker context with no code-mutation tools. Findings only — never edits files. The fresh-context spawn is the structural guarantee; the prompt body reinforces it but the spawn is what makes independence real. If no fresh worker can be spawned, VERIFY is `BLOCKED:fresh-context-unavailable`, never a same-context review.
|
|
71
|
+
After `state.implement_passed_sha` is set: SURFACE_CLOSE and CLEANUP are limited to their allowlists (empty SURFACE_CLOSE PASS is valid; violations revert); VERIFY is fresh-context, findings-only, and mutation-free. No fresh VERIFY worker → `BLOCKED:fresh-context-unavailable`, never same-context review.
|
|
74
72
|
</post_implement_invariant>
|
|
75
73
|
|
|
74
|
+
<transition_protocol>
|
|
75
|
+
For every direct complete→spawn handoff, call `state-phase-write.py ... --phase <current> transition` with the current phase's normal completion/attestation arguments plus caller-specified `--next-phase`, `--next-round`, `--next-triggered-by`, `--next-engine`, `--next-model`, and any next-phase metadata. The verb validates a legal edge and commits both lifecycle writes atomically; it returns JSON state facts only. It never selects a phase/engine, renders a prompt, or spawns an agent. Use standalone `spawn` only for the initial post-bootstrap dispatch; use standalone `complete` when no next phase opens (including a halt).
|
|
76
|
+
</transition_protocol>
|
|
77
|
+
|
|
76
78
|
## PHASE 0: PARSE + CLASSIFY + ROUTE
|
|
77
79
|
|
|
78
|
-
1.
|
|
79
|
-
- `--max-rounds N` (default 4) — fix-loop budget shared across BUILD_GATE and VERIFY.
|
|
80
|
-
- `--engine MODE` (default: `engines.json` `executor` pin when present, else `claude`) — picks the adapter for IMPLEMENT, CLEANUP, and the primary VERIFY judge. It does not disable VERIFY pair-mode; when a VERIFY pair trigger fires, the second judge uses the OTHER engine per role resolution.
|
|
81
|
-
- `--spec <path>` — switches to spec mode.
|
|
82
|
-
- `--verify-only <ref>` — switches to verify-only mode. Requires `--spec`.
|
|
83
|
-
- `--goal-file <path>` — free-form mode: read the goal text from `<path>` (relative to cwd) instead of an inline positional goal. Used by the devlynd `ResolveAdapter` launcher (goal written to `.devlyn/goal.txt`, referenced by path — never inlined as argv). The file content is opaque goal text fed to the complexity classifier, **never re-parsed as flags**. Mutually exclusive with `--spec`, `--verify-only`, and an inline positional goal.
|
|
84
|
-
- `--pair-verify` — force pair-mode JUDGE in PHASE 5 even when not auto-triggered.
|
|
85
|
-
- `--no-pair` — disable conditional VERIFY pair-JUDGE for this run. Record `pair_trigger.skipped_reason: "user_no_pair"` whenever a trigger would otherwise fire.
|
|
86
|
-
- `--risk-probes` — insert PHASE 1.5 cross-engine probe derivation. The OTHER engine converts visible `## Verification` bullets into bounded executable probes before IMPLEMENT; BUILD_GATE and VERIFY replay them mechanically.
|
|
87
|
-
- `--no-risk-probes` — disable automatic high-risk risk probes. Explicit `--risk-probes` wins over `--no-risk-probes`.
|
|
88
|
-
- `--bypass <phase>[,...]` — skip specific phases. Valid: `build-gate`, `cleanup`. PLAN, IMPLEMENT, VERIFY are non-bypassable.
|
|
89
|
-
- `--perf` — opt in to per-phase timing.
|
|
80
|
+
1. Run the bootstrap once with the exact tokenized `<pipeline_config>` and this orchestrator's default engine from `<engine_routing>`:
|
|
90
81
|
|
|
91
|
-
|
|
82
|
+
```bash
|
|
83
|
+
DEVLYN_DEFAULT_ENGINE="<current-cli-default>" python3 "$DEVLYN_SHARED_DIR/resolve-bootstrap.py" <pipeline_config tokens>
|
|
84
|
+
```
|
|
92
85
|
|
|
93
|
-
|
|
86
|
+
Read its sole JSON result. On `ok:false`, halt on its exact report-level `blocked` string and show `detail`; init failures leave no partial run artifacts and create no phase verdict. On success, the script has atomically initialized the schema-v3 skeleton (`pair_verify: true` only when `--pair-verify` was passed), stamped the null-safe Claude session id, persisted exact-byte Goal/spec identity, staged spec verification inputs, and captured the verify-only external diff. It validates only flags needed for those init fields, including mode exclusivity, `--max-rounds`, `--bypass`, and `--pair-verify`/`--no-pair`; `--pair-verify` and `--no-pair` are mutually exclusive. Free-form init sets `state.source.type = "generated"`. Spec staging validates supported `complexity` frontmatter (including sibling spec `complexity` frontmatter) and any present actionable solo-headroom hypothesis, matching its command to `spec.expected.json.verification_commands[].cmd` or the inline `## Verification` JSON carrier. It does not validate engines, write the untracked baseline, classify complexity/risk, choose a route, or announce. `state.engine` is the raw `--engine` value with `engine_source: "flag"`, otherwise the passed `DEVLYN_DEFAULT_ENGINE` with `engine_source: "default"`; step 2 resolves and replaces both fields.
|
|
94
87
|
|
|
95
|
-
|
|
88
|
+
2. Engine pre-flight: follow `_shared/engine-preflight.md`, including role resolution. If `.devlyn/engines.json` exists: malformed JSON, any named engine without a `_shared/adapters/<name>.md` adapter, or an adapter that declares itself ineligible for the requested role, halts with report-level `BLOCKED:invalid-engine-config` naming the offending entry. A pin is an explicit route — a pinned but unavailable engine halts with `BLOCKED:<engine>-unavailable`. If a required engine is unavailable, halt with a BLOCKED verdict and setup instructions instead of downgrading. After successful role resolution, replace `state.engine` and `state.engine_source` with the resolved executor and its provenance (`"flag" | "engines.json" | "default"`). (These and all other PHASE-0 halts are report-level `BLOCKED:<reason>` strings only; the JSON `phases.*.verdict` carrier in `pipeline.state.json` always stays the bare enum `"BLOCKED"`.)
|
|
96
89
|
|
|
97
|
-
|
|
90
|
+
3. Write `.devlyn/untracked.baseline`: `python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --write-untracked-baseline`.
|
|
98
91
|
|
|
99
|
-
|
|
100
|
-
- **Free-form**: first resolve `goal_text` — the inline positional goal, OR (when `--goal-file <path>` was passed) the raw UTF-8 content of `<path>`, read as opaque goal text (never re-parsed as flags). Fail closed before classifying: empty/whitespace-only file → `BLOCKED:goal-file-empty`; missing/unreadable/not-a-regular-file/invalid-UTF-8 → `BLOCKED:goal-file-unreadable`; an absolute path, a `..` segment, or a path that canonicalizes outside cwd → `BLOCKED:goal-file-invalid-path`. (Like all PHASE-0 halts these are report-level `BLOCKED:<reason>` strings — no `phases.*.verdict` is written, so the bare-enum carrier rule is never violated.) Then read `references/free-form-mode.md` and run the complexity classifier deterministically on `goal_text` (rules over keyword density / file count / spec-shape signals, plus pair-evidence intent). Set `state.complexity ∈ {trivial, medium, large}`. Trivial: write internal mini-spec to `.devlyn/criteria.generated.md` and proceed. Medium: synthesize a minimal spec from the goal + add 1-2 context anchors from the codebase, write to `.devlyn/criteria.generated.md`, proceed. Every free-form branch that writes criteria must set `state.source.type = "generated"`, `state.source.criteria_path = ".devlyn/criteria.generated.md"`, and `state.source.criteria_sha256` from the raw file bytes. Large: follow the Large action in `references/free-form-mode.md` — synthesize a best-effort spec with an `## Assumptions` block, log `recommend: /devlyn:ideate first`, proceed, and flag every assumption in the final report; zero-scope-signal goals halt `BLOCKED:large-needs-ideation`; pair-evidence intent without an actionable solo-headroom hypothesis must halt with `BLOCKED:solo-headroom-hypothesis-required`, and unmeasured pair-candidate intent without solo ceiling avoidance must halt with `BLOCKED:solo-ceiling-avoidance-required`.
|
|
101
|
-
- **Spec**: validate spec exists. If sibling `spec.expected.json` exists, run `--check-expected <expected-path>` to validate both the expected contract, sibling spec `complexity` frontmatter, and any present actionable solo-headroom hypothesis; if the spec has a solo-headroom hypothesis, its observable command must match `spec.expected.json.verification_commands[].cmd`. Then stage `.devlyn/spec-verify.json` from `verification_commands`. Otherwise run `--check <spec-path>` to validate the legacy inline carrier plus supported `complexity` frontmatter and any present actionable solo-headroom hypothesis; if the spec uses an inline `## Verification` JSON carrier, any solo-headroom hypothesis command must match that carrier's `verification_commands[].cmd`. Then stage from the legacy inline carrier. Compute `state.source.spec_sha256`.
|
|
102
|
-
- **Verify-only**: skip to PHASE 5 with `state.source.spec_path` set, the supplied diff captured at `.devlyn/external-diff.patch`.
|
|
92
|
+
4. Read the Goal/spec through `state.source`. For free-form mode, run the deterministic classifier in `references/free-form-mode.md`. Zero-scope-signal goals halt with `BLOCKED:large-needs-ideation`; pair-evidence intent without an actionable solo-headroom hypothesis must halt with `BLOCKED:solo-headroom-hypothesis-required`; unmeasured pair-candidate intent without solo ceiling avoidance must halt with `BLOCKED:solo-ceiling-avoidance-required`. Follow the selected branch to write the trivial/medium/large body of `.devlyn/criteria.generated.md`, then set `state.complexity` and its raw-byte `criteria_sha256`. The Large `## Assumptions`/recommendation/final-report obligations remain unchanged.
|
|
103
93
|
|
|
104
|
-
|
|
94
|
+
Compute `state.risk_profile` from the user goal plus spec/criteria text. Mark `high_risk: true` for auth/authz, permissions, security, token/session, payment/money/billing/invoice/pricing/tax/ledger, persistence/data mutation/deletion/migration, idempotency/replay/duplicate, API/webhook/raw-body/signature, allocation/scheduling/inventory/rollback/transaction, or explicit error-priority/output-shape contracts. Explicit `--risk-probes` sets both probe booleans true. Otherwise an automatic high-risk route enables probes only when the resolved OTHER engine is available and `--no-risk-probes` is absent; if unavailable, keep probes disabled and append `auto-risk-probes skipped: <engine>-unavailable`. `--no-pair` sets `pair_default_enabled: false`. Preserve strict boolean/list types and concise string reasons.
|
|
105
95
|
|
|
106
|
-
|
|
96
|
+
5. Announce one line: `resolve starting — run <run_id> — engine <engine> — mode <mode> — complexity <complexity-or-na> — pair <on|solo:auto_pair_other_engine_unavailable|disabled> — risk_probes <on|off>`.
|
|
97
|
+
|
|
98
|
+
6. Immediately before the first worker dispatch, open the caller-selected `plan` (normal run) or `verify` (verify-only) span with standalone `state-phase-write.py ... spawn`; then dispatch that same phase. The bootstrap never chooses a phase, engine, branch, prompt, or agent.
|
|
107
99
|
|
|
108
100
|
## PHASE 1: PLAN
|
|
109
101
|
|
|
@@ -125,7 +117,7 @@ After return:
|
|
|
125
117
|
Skip unless `--risk-probes` is set OR `state.risk_profile.risk_probes_enabled`
|
|
126
118
|
is true. This phase is findings-as-executable-checks, not a second plan and not
|
|
127
119
|
debate. When it runs, the OTHER engine is required: if unavailable, halt with
|
|
128
|
-
`BLOCKED
|
|
120
|
+
`BLOCKED:<engine>-unavailable` plus setup guidance;
|
|
129
121
|
do not silently continue without probes. Reaching this halt means the route was
|
|
130
122
|
explicitly requested (`--risk-probes`) — an auto high-risk escalation toward an
|
|
131
123
|
unavailable OTHER engine was already gated off in PHASE 0 by leaving
|
|
@@ -239,6 +231,14 @@ State write: `phases.implement.{started_at, verdict, completed_at, duration_ms}`
|
|
|
239
231
|
|
|
240
232
|
After the final phase's gate PASS: `git diff <base_ref.sha>...HEAD --stat` — empty → halt with `BLOCKED:implement-empty`; otherwise set `state.implement_passed_sha = git rev-parse HEAD` (phase commits already checkpoint the work — no extra commit).
|
|
241
233
|
|
|
234
|
+
## PHASE 2.5: SURFACE_CLOSE
|
|
235
|
+
|
|
236
|
+
Run once iff `state.source.type == "generated"` and complexity is trivial/medium. Engine is Claude always, pair-judge-routed; executor flag/pin ignored. Per `_shared/engine-preflight.md`, unavailability skips via `surface-skip`, reports `auto_surface_close_claude_unavailable`, never BLOCKs/reroutes.
|
|
237
|
+
|
|
238
|
+
Freeze `.devlyn/surface-close.input.patch`; assemble Claude adapter + `references/phases/surface-close.md` canonical body VERBATIM + data-only Goal/patch/hashes/surface/commands; hash the bytes and spawn with `--tools "Read,Grep,Glob,Edit,Write" --dangerously-skip-permissions --model claude-sonnet-5 --output-format json --strict-mcp-config --mcp-config '{"mcpServers":{}}'`, recording the same model at SPW spawn. Bound at 600s (native or `run-bounded.py 600 -- claude -p`), else block pre-spawn. Save raw stdout as `.devlyn/surface-close.output.json`, extract its `result` string to `.devlyn/surface-close.stdout`, retain `.devlyn/surface-close.worker-session.<round>.jsonl`, and complete with the wrapper as `--engine-session-log`; non-JSON/missing result follows the existing failure path. Workers execute nothing; `surface-check` gates rows, citations, scope, and execution. Empty PASS completes; authorized delta is scoped-staged and committed. Only `BLOCKED:surface-close-adjudication-malformed` runs `surface-adjudication-recover --authorized-surface-json <array>`, whose successful rollback and transcript Edit/Write audit complete the nonterminal skip carrier and continue to BUILD_GATE. Timeout, input/prompt mismatch, attestation failure, execution-audit violation, and out-of-surface delta run `surface-rollback`, complete bare `BLOCKED`, and halt; rollback failure also halts. One shot; no `max_rounds`.
|
|
239
|
+
|
|
240
|
+
**Common post-fix checkpoint (BUILD_GATE and VERIFY):** After fix IMPLEMENT returns, increment `state.rounds.global`; scoped-stage the authorized surface and commit `chore(pipeline): implement fix round <n>`; run `python3 "$DEVLYN_SHARED_DIR/state-phase-write.py" --devlyn-dir .devlyn --phase implement durability-enforce --round <n> --origin-phase <build_gate|verify>`. Its receipt must PASS before re-entry, which enforces it again before VERIFY artifact clearing or phase spawn.
|
|
241
|
+
|
|
242
242
|
## PHASE 3: BUILD_GATE
|
|
243
243
|
|
|
244
244
|
Skip in verify-only mode OR when `build-gate` in `state.bypasses`. Deterministic — same commands CI / Docker / production run.
|
|
@@ -254,7 +254,7 @@ State write: `phases.build_gate.{started_at, verdict, completed_at, duration_ms,
|
|
|
254
254
|
|
|
255
255
|
Branch:
|
|
256
256
|
- `PASS` → PHASE 4.
|
|
257
|
-
- `FAIL` → fix loop. Spawn IMPLEMENT-engine agent with the build_gate findings as input
|
|
257
|
+
- `FAIL` → fix loop. Spawn IMPLEMENT-engine agent with the build_gate findings as input, then run the common post-fix checkpoint with origin `build_gate`. On second FAIL with `state.rounds.global >= state.rounds.max_rounds` → halt with verdict `BLOCKED:build-gate-exhausted`.
|
|
258
258
|
|
|
259
259
|
## PHASE 4: CLEANUP
|
|
260
260
|
|
|
@@ -282,46 +282,27 @@ Two sub-phases:
|
|
|
282
282
|
|
|
283
283
|
1. **MECHANICAL** (deterministic): re-run `SPEC_VERIFY_PHASE=verify_mechanical SPEC_VERIFY_FINDINGS_FILE=verify-mechanical.findings.jsonl SPEC_VERIFY_FINDING_PREFIX=VERIFY-MECH python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --include-risk-probes` against the post-CLEANUP code (independent of BUILD_GATE's earlier run). If `state.risk_profile.risk_probes_enabled == true`, missing `.devlyn/risk-probes.jsonl` is a CRITICAL mechanical blocker. This emits `.devlyn/verify-mechanical.findings.jsonl` for `verify-merge-findings.py`.
|
|
284
284
|
|
|
285
|
-
2. **JUDGE** (fresh-context Agent): grade the diff against the spec on rubric axes (spec compliance, scope, quality, consistency). Split each Requirement into binding clauses and trace code-order counterexamples; a passing verifier proves only the case it exercises, not neighboring `once` / `regardless` / `duplicate` / auth-order / rollback invariants. Respect scope qualifiers such as `inside a warehouse`, `per resource`, `for this line`, and `after validation`; do not widen a scoped clause into a global invariant, and compose multiple ordering rules in the stated order. For stateful flows, explicitly trace failed-operation rollback and the next entity's state before hunting broader edge cases. For high-complexity specs, construct at least one interaction counterexample that combines ordering/priority with failure handling and state mutation, then execute at least one such scenario through the repo's existing CLI/API/test runner without leaving tracked files behind; one-axis examples and pure mental tracing are insufficient. Default engine = same as IMPLEMENT
|
|
286
|
-
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
|
|
294
|
-
|
|
295
|
-
|
|
296
|
-
|
|
297
|
-
|
|
298
|
-
|
|
299
|
-
|
|
300
|
-
|
|
301
|
-
every reason is automatic) — the OTHER engine being unavailable. In that
|
|
302
|
-
auto-only-unavailable case, set `eligible: false`, leave `reasons` empty, set
|
|
303
|
-
`skipped_reason: "auto_pair_other_engine_unavailable"`, and run solo VERIFY.
|
|
304
|
-
Explicit `--pair-verify` (`mode.pair-verify`) with the OTHER engine absent is a
|
|
305
|
-
promise: it stays `BLOCKED:<engine>-unavailable` and is never skipped.
|
|
306
|
-
|
|
307
|
-
Pair-mode JUDGE: spawn a second Agent with the OTHER engine's adapter; the second judge is a bounded adversarial complement, not a duplicate broad audit. The primary judge owns broad coverage; pair-JUDGE targets the two highest-risk explicit `## Verification` bullets that cross state mutation, all-or-nothing rollback, ordering, idempotency, auth, or error-priority clauses. If the spec includes a solo-headroom hypothesis, one of those targeted probes must exercise that hypothesis with the visible command/input shape and full externally visible result, using the hypothesis's backticked observable command as its command anchor before adding bounded input variations. It must not read `.claude/skills`, `.codex/skills`, `CLAUDE.md`, `AGENTS.md`, or other harness docs unless the orchestrator pasted a specific excerpt into the prompt. It may use only the spec, diff, implementation files, tests, and the repo's existing CLI/API/test runner. It may execute at most two targeted probes before first output, and each probe must compare the full externally visible result (exit/stdout/stderr plus full parsed output object, including accepted/scheduled rows, rejected rows, and remaining state when present), not just a single property. When the spec names exact keys, row shapes, JSON object shape, or an exact error body, pair-JUDGE must compare parsed key sets/deep equality so aliased keys, missing keys, and extra keys are verdict-binding failures, and it must construct inputs with the spec's visible key names. For priority/stateful specs, at least one probe must include an earlier input entity that would succeed under input-order processing, a later higher-priority entity that consumes or blocks the critical resource, and a failure/blocked/rollback edge that determines a later entity's state. For cart/pricing specs where visible verification combines duplicate items, line promotions, tax, coupon, and shipping, the success-path probe must include interleaved duplicates plus taxable and non-taxable items and assert full output rows. Scope qualifiers are binding: pair-JUDGE must not reinterpret `inside a warehouse`, `per resource`, or line-scoped rules as global rules. When both priority ordering and rollback/blocked-interval behavior appear in the spec, this dominance-loss probe is mandatory and comes before any other probe: an earlier lower-priority entity that would succeed alone or under input-order processing must lose because a later higher-priority entity is processed first; a failed/blocked middle entity must not corrupt later state; and the assertion must cover complete accepted/scheduled and rejected output ordering. It must stop and emit JSONL immediately on the first verdict-binding finding, and must emit PASS immediately if both probes plus static scope/dependency checks pass. Both judgments merge with the rule "any HIGH/CRITICAL finding either model surfaces is verdict-binding; high-confidence MEDIUM findings are also verdict-binding when they identify a concrete behavioral regression against the spec, public contract, or existing test contract." Cross-model disagreement on advisory lower-severity findings is logged but does not change the verdict. If MECHANICAL or the primary JUDGE has a verdict-binding finding, skip the second judge and record `pair_judge: null`; the fix loop needs the blocker, not duplicate review.
|
|
308
|
-
|
|
309
|
-
If pair-mode is triggered and the OTHER engine is unavailable, do not downgrade
|
|
310
|
-
or skip the required judge. Set VERIFY to `BLOCKED:<engine>-unavailable`, preserve the
|
|
311
|
-
failed availability check evidence, and print setup guidance:
|
|
312
|
-
|
|
313
|
-
- Codex: install/configure the Codex CLI, run `codex auth` or the current login
|
|
314
|
-
flow, verify `codex --version`, then rerun. Use `--no-pair` only when the user
|
|
315
|
-
intentionally accepts solo VERIFY for this run.
|
|
316
|
-
- Claude: install/configure Claude Code, run `claude --version` when available,
|
|
317
|
-
confirm the host can spawn Claude agents, then rerun.
|
|
318
|
-
|
|
319
|
-
Findings written to `.devlyn/verify.findings.jsonl`. **VERIFY agents have no code-mutation tools.** Codex pair-JUDGE is read-only: invoke `codex-monitored.sh` with `CODEX_MONITORED_ISOLATED=1 CODEX_MONITORED_TIMEOUT_SEC=600` and `-c model_reasoning_effort=medium`, no `tail`/`head`/`grep` pipes, direct stdout/stderr capture, JSONL findings on stdout, and orchestrator-written `.devlyn/verify.pair.findings.jsonl`. Isolation blocks user config, AGENTS.md, hooks, and project rules from hidden context/tool/transcript side effects. If stdout is captured as `.devlyn/codex-judge.stdout`, run `python3 "$DEVLYN_SHARED_DIR/collect-codex-findings.py"` before merge; raw stdout is diagnostic only. If stdout contains findings or a non-PASS summary while `.devlyn/verify.pair.findings.jsonl` is empty, `verify-merge-findings.py` blocks VERIFY for `verify.pair.emission-contract`. Do not ask Codex to `apply_patch` or edit `.devlyn`. When the OTHER engine is Claude (codex/omp orchestrator), the judge call is wrapped as `python3 "$DEVLYN_SHARED_DIR/run-bounded.py" 600 -- claude -p ...` with the same flags as `_shared/adapters/claude.md` `## Invocation`, stdout captured as `.devlyn/claude-judge.stdout`, and the same emission contract applied to that file. On judge subprocess exit 124, the orchestrator writes `.devlyn/verify.pair.timeout.json` (`{"engine": "<codex|claude>", "budget_seconds": 600}`) before running `verify-merge-findings.py`. After primary and pair findings are written, run `python3 "$DEVLYN_SHARED_DIR/verify-merge-findings.py" --write-state`. Branch only on the merged `state.phases.verify.verdict`; a HIGH/CRITICAL finding from either judge must mechanically become `NEEDS_WORK`. Never write `.devlyn/verify-merged.findings.jsonl` or `.devlyn/verify-merge.summary.json` by hand; `verify-merge-findings.py` is their only writer. State write: `phases.verify.{started_at, verdict, completed_at, duration_ms, sub_verdicts: {mechanical, judge, pair_judge?}, artifacts}`.
|
|
285
|
+
2. **JUDGE** (fresh-context Agent): grade the diff against the spec on rubric axes (spec compliance, scope, quality, consistency). Split each Requirement into binding clauses and trace code-order counterexamples; a passing verifier proves only the case it exercises, not neighboring `once` / `regardless` / `duplicate` / auth-order / rollback invariants. Respect scope qualifiers such as `inside a warehouse`, `per resource`, `for this line`, and `after validation`; do not widen a scoped clause into a global invariant, and compose multiple ordering rules in the stated order. For stateful flows, explicitly trace failed-operation rollback and the next entity's state before hunting broader edge cases. For high-complexity specs, construct at least one interaction counterexample that combines ordering/priority with failure handling and state mutation, then execute at least one such scenario through the repo's existing CLI/API/test runner without leaving tracked files behind; one-axis examples and pure mental tracing are insufficient. Default engine = same as IMPLEMENT. MECHANICAL always runs first; a verdict-binding blocker routes to the fix loop without either judge. After MECHANICAL passes, `--no-pair` continues solo with `user_no_pair`. Otherwise resolve the OTHER engine. An unavailable OTHER engine blocks an explicit `--pair-verify` promise; an automatic route records `auto_pair_other_engine_unavailable` and continues solo.
|
|
286
|
+
|
|
287
|
+
When the OTHER engine is available, persist `pair_trigger` before spawn with `pair.default` plus every applicable outcome-independent reason, then dispatch both judges concurrently against the same frozen diff via foreground parallel dispatch, never background shells. A primary blocker never cancels the pair judge. If foreground parallel dispatch is unavailable, run the same two required judges sequentially; this changes dispatch shape only and never permits outcome-dependent escalation or a pair skip. Prefer read-only checks and serialize probes that need exclusive shared state.
|
|
288
|
+
|
|
289
|
+
Keep the existing reasons as telemetry: `mode.verify-only`, `mode.pair-verify`, `complexity.high`, `complexity.large`, `spec.complexity.high`, `spec.complexity.large`, `spec.solo_headroom_hypothesis`, `risk.high`, `risk_probes.enabled`, `risk_probes.present`, `coverage.failed`, `mechanical.warning`, and `judge.warning`. Append the outcome-dependent reasons only after both judges return. Eligible schema-v3 state must include `pair.default`; all other applicable reasons remain completeness-checked. Malformed risk or trigger state BLOCKs VERIFY.
|
|
290
|
+
|
|
291
|
+
Pair-mode JUDGE: spawn a second Agent with the OTHER engine's adapter; the second judge is a bounded adversarial complement, not a duplicate broad audit. The primary judge owns broad coverage; pair-JUDGE targets the two highest-risk explicit `## Verification` bullets that cross state mutation, all-or-nothing rollback, ordering, idempotency, auth, or error-priority clauses. If the spec includes a solo-headroom hypothesis, one of those targeted probes must exercise that hypothesis with the visible command/input shape and full externally visible result, using the hypothesis's backticked observable command as its command anchor before adding bounded input variations. It must not read `.claude/skills`, `.codex/skills`, `CLAUDE.md`, `AGENTS.md`, or other harness docs unless the orchestrator pasted a specific excerpt into the prompt. It may use only the spec, diff, implementation files, tests, and the repo's existing CLI/API/test runner. It may execute at most two targeted probes before first output, and each probe must compare the full externally visible result (exit/stdout/stderr plus full parsed output object, including accepted/scheduled rows, rejected rows, and remaining state when present), not just a single property. When the spec names exact keys, row shapes, JSON object shape, or an exact error body, pair-JUDGE must compare parsed key sets/deep equality so aliased keys, missing keys, and extra keys are verdict-binding failures, and it must construct inputs with the spec's visible key names. For priority/stateful specs, at least one probe must include an earlier input entity that would succeed under input-order processing, a later higher-priority entity that consumes or blocks the critical resource, and a failure/blocked/rollback edge that determines a later entity's state. For cart/pricing specs where visible verification combines duplicate items, line promotions, tax, coupon, and shipping, the success-path probe must include interleaved duplicates plus taxable and non-taxable items and assert full output rows. Scope qualifiers are binding: pair-JUDGE must not reinterpret `inside a warehouse`, `per resource`, or line-scoped rules as global rules. When both priority ordering and rollback/blocked-interval behavior appear in the spec, this dominance-loss probe is mandatory and comes before any other probe: an earlier lower-priority entity that would succeed alone or under input-order processing must lose because a later higher-priority entity is processed first; a failed/blocked middle entity must not corrupt later state; and the assertion must cover complete accepted/scheduled and rejected output ordering. It must stop and emit JSONL immediately on the first verdict-binding finding, and must emit PASS immediately if both probes plus static scope/dependency checks pass. Both judgments merge with the rule "any HIGH/CRITICAL finding either model surfaces is verdict-binding; high-confidence MEDIUM findings are also verdict-binding when they identify a concrete behavioral regression against the spec, public contract, or existing test contract." Cross-model disagreement on advisory lower-severity findings is logged but does not change the verdict. A primary JUDGE verdict-binding finding on a concurrent run does not cancel or discard the pair-JUDGE; both finding sets join the same fix round, and the merged verdict remains the worst source verdict without vote counting. `primary_judge_blocker` is parser-recognized only for archived v2.0 replay; new runs never write it. A verdict-binding MECHANICAL blocker still skips both judges and records `pair_judge: null`.
|
|
292
|
+
|
|
293
|
+
If the OTHER engine is unavailable, automatic VERIFY records
|
|
294
|
+
`auto_pair_other_engine_unavailable` and continues solo. Explicit `--pair-verify`
|
|
295
|
+
instead sets VERIFY to `BLOCKED:<engine>-unavailable`, preserves the failed
|
|
296
|
+
check, and prints setup guidance: install/configure the missing CLI, complete
|
|
297
|
+
its auth/login flow, verify `<engine> --version`, then rerun. `--no-pair` is only
|
|
298
|
+
for an intentional solo run.
|
|
299
|
+
|
|
300
|
+
Findings written to `.devlyn/verify.findings.jsonl`. **VERIFY agents have no code-mutation tools.** For Codex pair-JUDGE, keep the read-only `codex-monitored.sh` route with `CODEX_MONITORED_ISOLATED=1 CODEX_MONITORED_TIMEOUT_SEC=600` and `-c model_reasoning_effort=medium`, no `tail`/`head`/`grep` pipes, and direct stdout/stderr capture; every other resolved OTHER engine follows `_shared/adapters/<name>.md` `## Invocation`. Capture `.devlyn/<name>-judge.stdout`, then run `python3 "$DEVLYN_SHARED_DIR/collect-codex-findings.py" --devlyn-dir "<abs repo>/.devlyn" --stdout-file <name>-judge.stdout` before merge. Raw stdout is diagnostic-only. A non-zero collector exit must write no canonical `.devlyn/verify.pair.findings.jsonl`; set the pair source to `BLOCKED` for `verify.pair.emission-contract` and never merge as though unparsed stdout were only a diagnostic. VERIFY prompts remain bounded, read-only, and must not ask the judge to edit `.devlyn`. On judge subprocess exit 124, write `.devlyn/verify.pair.timeout.json` with `{"engine": "<resolved name>", "budget_seconds": 600}` before merge. The orchestrator records `phases.verify.judge_durations_ms: {judge: <int>, pair_judge: <int|null>}` as it collects each judge result; durations never nest inside `sub_verdicts`. After primary and pair findings are written, run `python3 "$DEVLYN_SHARED_DIR/verify-merge-findings.py" --write-state`. Branch only on the merged `state.phases.verify.verdict`; a HIGH/CRITICAL finding from either judge must mechanically become `NEEDS_WORK`. Never write `.devlyn/verify-merged.findings.jsonl` or `.devlyn/verify-merge.summary.json` by hand; `verify-merge-findings.py` is their only writer. State write: `phases.verify.{started_at, verdict, completed_at, duration_ms, judge_durations_ms: {judge, pair_judge}, sub_verdicts: {mechanical, judge, pair_judge?}, artifacts}`.
|
|
320
301
|
|
|
321
302
|
Branch:
|
|
322
303
|
- `PASS` → PHASE 6.
|
|
323
304
|
- `PASS_WITH_ISSUES` (LOW severity only) → PHASE 6 with banner.
|
|
324
|
-
- `NEEDS_WORK` / `BLOCKED` → fix loop with `triggered_by: "verify"`. Spawn IMPLEMENT-engine agent with the verify findings
|
|
305
|
+
- `NEEDS_WORK` / `BLOCKED` → fix loop with `triggered_by: "verify"`. Spawn IMPLEMENT-engine agent with the verify findings, then run the common post-fix checkpoint with origin `verify`. Second `NEEDS_WORK` → halt with verdict `BLOCKED:verify-exhausted`.
|
|
325
306
|
|
|
326
307
|
## PHASE 6: FINAL REPORT + ARCHIVE
|
|
327
308
|
|
|
@@ -331,9 +312,9 @@ State write: `phases.final_report.started_at` at the top of this phase.
|
|
|
331
312
|
|
|
332
313
|
2. **FINISH GATE** — run `python3 "$DEVLYN_SHARED_DIR/finish-gate.py"`; branch only on exit code plus `.devlyn/finish-gate.findings.jsonl` existence: 0 without the file → clean; 0 with it → terminal verdict floors at `PASS_WITH_ISSUES` (report lists the reverted paths); 1 or 2 → `BLOCKED:finish-gate-unclean`.
|
|
333
314
|
|
|
334
|
-
3. **Terminal verdict** — derive from `state.phases.{plan, implement, build_gate, cleanup, verify}.verdict` per the precedence rules in `references/state-schema.md#terminal-verdict`. Verify-only mode short-circuits to `state.phases.verify.verdict`.
|
|
315
|
+
3. **Terminal verdict** — derive from `state.phases.{plan, implement, surface_close, build_gate, cleanup, verify}.verdict` per the precedence rules in `references/state-schema.md#terminal-verdict`. Verify-only mode short-circuits to `state.phases.verify.verdict`.
|
|
335
316
|
|
|
336
|
-
4. **Render report** — sections: header (run_id, engine, mode, verdict, wall-time), per-phase summary, pair/risk-probe status, findings table (verify + finish-gate findings), follow-up notes (any large-mode `## Assumptions` block, any pair-judge TIMEOUT (headline: solo verdict after pair TIMEOUT), any `--no-pair` / `--no-risk-probes` opt-out, any engine setup guidance after BLOCKED, `/devlyn:ideate` guidance after `BLOCKED:solo-headroom-hypothesis-required` that asks for the visible behavior `solo_claude` is expected to miss, and `/devlyn:ideate` guidance after `BLOCKED:solo-ceiling-avoidance-required` that asks for the concrete difference from rejected or solo-saturated controls such as `S2`-`S6`).
|
|
317
|
+
4. **Render report** — sections: header (run_id, engine, mode, verdict, wall-time), per-phase summary (including SURFACE_CLOSE run or skip), pair/risk-probe status, findings table (verify + finish-gate findings), follow-up notes (the explicit line `pipeline continued to BUILD_GATE — surface_close_rolled_back_adjudication_malformed` when `continued_after_block` is set, any large-mode `## Assumptions` block, any pair-judge TIMEOUT (headline: solo verdict after pair TIMEOUT), any `--no-pair` / `--no-risk-probes` opt-out, any engine setup guidance after BLOCKED, `/devlyn:ideate` guidance after `BLOCKED:solo-headroom-hypothesis-required` that asks for the visible behavior `solo_claude` is expected to miss, and `/devlyn:ideate` guidance after `BLOCKED:solo-ceiling-avoidance-required` that asks for the concrete difference from rejected or solo-saturated controls such as `S2`-`S6`).
|
|
337
318
|
|
|
338
319
|
5. State write: `phases.final_report.{verdict, completed_at, duration_ms}` BEFORE archive runs (archive prune logic skips runs whose `final_report.verdict` is null).
|
|
339
320
|
|
|
@@ -341,4 +322,4 @@ State write: `phases.final_report.started_at` at the top of this phase.
|
|
|
341
322
|
|
|
342
323
|
## State management
|
|
343
324
|
|
|
344
|
-
`.devlyn/pipeline.state.json` is the single authoritative verdict source. Branch on `state.phases.<name>.verdict` directly; never parse `.devlyn/*.findings.jsonl` for routing decisions. Schema and write protocol: `references/state-schema.md`. Every "State write: `phases.X.{...}`" note above describes the resulting shape only — the orchestrator
|
|
325
|
+
`.devlyn/pipeline.state.json` is the single authoritative verdict source. Branch on `state.phases.<name>.verdict` directly; never parse `.devlyn/*.findings.jsonl` for routing decisions. Schema and write protocol: `references/state-schema.md`. Every "State write: `phases.X.{...}`" note above describes the resulting shape only — the orchestrator writes it via initial `spawn`, atomic `transition` for direct handoffs, or terminal `complete`; re-entry preserves the prior lifecycle in `history[]` (`references/state-schema.md#write-protocol`). Phase workers never edit `pipeline.state.json` directly.
|
|
@@ -0,0 +1,34 @@
|
|
|
1
|
+
# PHASE 2.5 — SURFACE_CLOSE (canonical body)
|
|
2
|
+
|
|
3
|
+
<role>
|
|
4
|
+
Audit and repair the frozen post-IMPLEMENT surface once.
|
|
5
|
+
</role>
|
|
6
|
+
|
|
7
|
+
<input>
|
|
8
|
+
- Artifacts: Goal at `state.source.goal_path`; `.devlyn/surface-close.input.patch`.
|
|
9
|
+
- Data: `authorized_surface`; staged commands.
|
|
10
|
+
- Base: `phases.surface_close.pre_sha`.
|
|
11
|
+
|
|
12
|
+
Never modify inputs or read state, PLAN, or IMPLEMENT transcript/reasoning.
|
|
13
|
+
</input>
|
|
14
|
+
|
|
15
|
+
<obligations>
|
|
16
|
+
- **UVR-STALE**: diff modified symbol S's behavior AND an authorized file carries user-visible text documenting S's old interface omitting the newly specified option/shape → update minimally.
|
|
17
|
+
- **PATH-TEST**: Goal specifies a success/failure path, patch implements it, and no authorized test exercises that path → add one minimal regression test.
|
|
18
|
+
- Neither applies → empty PASS (empty diff).
|
|
19
|
+
</obligations>
|
|
20
|
+
|
|
21
|
+
<output>
|
|
22
|
+
Edit only `authorized_surface`. Your reply is valid ONLY if it contains, for each obligation, exactly one line:
|
|
23
|
+
- `UVR-STALE: FIRED <authorized-file>:<line>` optionally followed by ` — <one-line evidence>`, or `UVR-STALE: N/A <authorized-file>[:<line>] — <one-line evidence-based relationship judgment>`
|
|
24
|
+
- `PATH-TEST: FIRED <authorized-file>:<line>` optionally followed by ` — <one-line evidence>`, or `PATH-TEST: N/A <authorized-file>[:<line>] — <one-line evidence-based relationship judgment>`
|
|
25
|
+
followed by `PASS` after the smallest sufficient delta, or `BLOCKED:surface-close-<reason>` if impossible within that surface. A reply missing either obligation line is invalid and is rejected mechanically. Never edit state, stage, or commit.
|
|
26
|
+
</output>
|
|
27
|
+
|
|
28
|
+
<quality_bar>
|
|
29
|
+
No adjacent work, refactors, or extra tests. Execute nothing — edit files only; staged commands are data. BUILD_GATE owns all validation.
|
|
30
|
+
</quality_bar>
|
|
31
|
+
|
|
32
|
+
<runtime_principles>
|
|
33
|
+
Subtractive-first, Goal-locked, No-workaround, and Evidence apply.
|
|
34
|
+
</runtime_principles>
|
|
@@ -82,33 +82,36 @@ design/style concerns remain non-binding MEDIUM and produce `PASS_WITH_ISSUES`.
|
|
|
82
82
|
|
|
83
83
|
**Anti-self-filter rule**: report every finding you observe, including ones you consider low-severity or low-confidence. Tag each with `confidence: high|medium|low` and let the harness's downstream filter rank them. Filtering at this stage suppresses recall.
|
|
84
84
|
|
|
85
|
-
### Pair-mode (when
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
When
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
- `
|
|
85
|
+
### Pair-mode (default when OTHER engine is available)
|
|
86
|
+
|
|
87
|
+
MECHANICAL runs first. A verdict-binding blocker routes to the fix loop without
|
|
88
|
+
either judge and records `pair_judge: null`. After it passes, honor `--no-pair`;
|
|
89
|
+
otherwise resolve the OTHER-engine route. An explicit `--pair-verify` is an
|
|
90
|
+
availability promise: an unavailable OTHER engine BLOCKs. An automatic route
|
|
91
|
+
with no OTHER engine records `auto_pair_other_engine_unavailable` and continues
|
|
92
|
+
solo.
|
|
93
|
+
|
|
94
|
+
When the OTHER engine is available, write `pair_trigger` at judge-spawn time
|
|
95
|
+
with `pair.default` plus every applicable outcome-independent canonical reason,
|
|
96
|
+
then dispatch the primary and pair JUDGEs concurrently against the same frozen
|
|
97
|
+
diff via foreground parallel dispatch, never background shells. A primary blocker does not cancel the pair-JUDGE; both finding sets join the same fix
|
|
98
|
+
round and merge by worst source verdict without vote counting. An orchestrator
|
|
99
|
+
without foreground parallel dispatch runs the same two required judges
|
|
100
|
+
sequentially, with no flag or extra state marker. This is a dispatch-shape
|
|
101
|
+
fallback, not outcome-dependent escalation; the primary cannot skip the pair.
|
|
102
|
+
Prefer read-only checks and serialize probes requiring exclusive shared state.
|
|
103
|
+
|
|
104
|
+
Keep these reasons as telemetry; they no longer gate the second spawn:
|
|
105
|
+
`mode.pair-verify`, `mode.verify-only`, `complexity.high`,
|
|
106
|
+
`complexity.large`, `spec.complexity.high`, `spec.complexity.large`,
|
|
107
|
+
`spec.solo_headroom_hypothesis`, `risk.high`, `risk_probes.enabled`,
|
|
108
|
+
`risk_probes.present`, `coverage.failed`, `mechanical.warning`, and
|
|
109
|
+
`judge.warning`.
|
|
106
110
|
|
|
107
111
|
Malformed `state.risk_profile` is a VERIFY contract violation: it must be an
|
|
108
112
|
object, `high_risk` / `risk_probes_enabled` / `pair_default_enabled` must be
|
|
109
|
-
JSON booleans when present, and `reasons` must be a string array.
|
|
110
|
-
|
|
111
|
-
it because it can hide `risk.high` or `risk_probes.enabled` pair triggers.
|
|
113
|
+
JSON booleans when present, and `reasons` must be a string array. The merge
|
|
114
|
+
blocks malformed state because it can hide a required route.
|
|
112
115
|
|
|
113
116
|
If `--no-pair` was set, do not spawn the OTHER-engine judge. Record
|
|
114
117
|
`pair_trigger: { eligible: false, reasons: [], skipped_reason: "user_no_pair" }`
|
|
@@ -116,58 +119,32 @@ and continue with solo VERIFY. This is an explicit user opt-out, not an engine
|
|
|
116
119
|
availability fallback. `--pair-verify` and `--no-pair` are mutually exclusive;
|
|
117
120
|
if both are present, stop with `BLOCKED:invalid-flags`.
|
|
118
121
|
|
|
119
|
-
|
|
120
|
-
spawning the OTHER-engine pair judge:
|
|
122
|
+
When the OTHER engine is available, persist this before either judge starts:
|
|
121
123
|
|
|
122
124
|
```json
|
|
123
125
|
"pair_trigger": {
|
|
124
126
|
"eligible": true,
|
|
125
|
-
"reasons": ["complexity.high"],
|
|
127
|
+
"reasons": ["pair.default", "complexity.high"],
|
|
126
128
|
"skipped_reason": null
|
|
127
129
|
}
|
|
128
130
|
```
|
|
129
131
|
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
on malformed trigger state. Eligible triggers must contain only canonical
|
|
142
|
-
reasons and at least one reason: `mode.verify-only`, `complexity.high`, `complexity.large`,
|
|
143
|
-
`mode.pair-verify`, `spec.complexity.high`, `spec.complexity.large`,
|
|
144
|
-
`spec.solo_headroom_hypothesis`, `risk.high`, `risk_probes.enabled`,
|
|
145
|
-
`risk_probes.present`, `coverage.failed`, `mechanical.warning`, or
|
|
146
|
-
`judge.warning`.
|
|
132
|
+
After both judges return, append every applicable outcome-dependent reason
|
|
133
|
+
(`coverage.failed`, `mechanical.warning`, `judge.warning`) before merge. Eligible schema-v3 state must contain `pair.default` and every other applicable canonical
|
|
134
|
+
reason. The OTHER-engine judge is mandatory; missing output is a contract
|
|
135
|
+
violation.
|
|
136
|
+
|
|
137
|
+
`pair_trigger` remains strict: eligible state has canonical reasons and no skip;
|
|
138
|
+
ineligible state has empty reasons and one of `user_no_pair`,
|
|
139
|
+
`mechanical_blocker`, `auto_pair_other_engine_unavailable`, or null.
|
|
140
|
+
`primary_judge_blocker` is parser-recognized only for archived v2.0 replay and
|
|
141
|
+
retains the existing rejection when a pre-known reason applied. New runs never
|
|
142
|
+
write it. The `--engine` flag does not disable default pairing.
|
|
147
143
|
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
skip reason.
|
|
153
|
-
The valid skip reasons after a non-empty eligible trigger are deterministic
|
|
154
|
-
MECHANICAL HIGH/CRITICAL blockers, an explicit `--no-pair`, or — when none of the
|
|
155
|
-
applicable reasons is `mode.pair-verify` (every reason is automatic) — the OTHER
|
|
156
|
-
engine being unavailable. Explicit `--pair-verify` with the OTHER engine absent
|
|
157
|
-
is a promise and stays `BLOCKED:<engine>-unavailable`, never a skip.
|
|
158
|
-
|
|
159
|
-
Before invoking the OTHER-engine judge, run the shared availability pre-flight
|
|
160
|
-
for that engine. If the route is explicit (`--pair-verify`) and Codex is required
|
|
161
|
-
and unavailable, set VERIFY to `BLOCKED:codex-unavailable` and tell the user to
|
|
162
|
-
install/configure the Codex CLI, run the current Codex auth/login flow, verify
|
|
163
|
-
`codex --version`, and rerun. If Claude is required and the host cannot spawn a
|
|
164
|
-
Claude agent, set VERIFY to `BLOCKED:claude-unavailable` and tell the user to
|
|
165
|
-
install/configure Claude Code, verify `claude --version`, and rerun. For an
|
|
166
|
-
explicit route, do not convert this to a solo pass and do not synthesize pair
|
|
167
|
-
findings. When the trigger was AUTO-only and the OTHER engine is unavailable, set
|
|
168
|
-
`eligible: false`, `reasons: []`, `skipped_reason:
|
|
169
|
-
"auto_pair_other_engine_unavailable"`, run solo VERIFY, and report the skipped
|
|
170
|
-
auto escalation — route selection, not a fallback.
|
|
144
|
+
Run the shared availability pre-flight before spawn. Explicit `--pair-verify`
|
|
145
|
+
unavailability stays `BLOCKED:<engine>-unavailable` with setup guidance.
|
|
146
|
+
Automatic unavailability records `auto_pair_other_engine_unavailable` and runs
|
|
147
|
+
solo. Never synthesize pair findings.
|
|
171
148
|
|
|
172
149
|
When eligible and the orchestrator spawns a second VERIFY agent with the OTHER engine's adapter, both judgments are merged:
|
|
173
150
|
- Any HIGH/CRITICAL finding either model surfaces is verdict-binding.
|
|
@@ -221,39 +198,31 @@ When eligible and the orchestrator spawns a second VERIFY agent with the OTHER e
|
|
|
221
198
|
and the assertion must cover the complete output ordering for both accepted
|
|
222
199
|
(or scheduled) and rejected rows.
|
|
223
200
|
|
|
224
|
-
|
|
201
|
+
The resolved pair-JUDGE is read-only. Codex keeps the monitored
|
|
202
|
+
`codex-monitored.sh` route with
|
|
225
203
|
`CODEX_MONITORED_ISOLATED=1 CODEX_MONITORED_TIMEOUT_SEC=600` and
|
|
226
|
-
`-c model_reasoning_effort=medium`;
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
Capture
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
as `python3 "$DEVLYN_SHARED_DIR/run-bounded.py" 600 -- claude -p ...` with
|
|
236
|
-
`--permission-mode dontAsk`, an allowlist of `Read,Grep,Glob` plus the repo test
|
|
237
|
-
command, hermetic settings, stdout captured to
|
|
238
|
-
`.devlyn/claude-judge.stdout`. The same bounded-output contract and emission
|
|
239
|
-
rule apply to that stdout file, and the orchestrator writes the same
|
|
240
|
-
canonical `.devlyn/verify.pair.findings.jsonl`.
|
|
241
|
-
The Codex prompt must include a bounded-output contract: no harness-doc reads,
|
|
204
|
+
`-c model_reasoning_effort=medium`; isolation blocks user config, AGENTS.md,
|
|
205
|
+
hooks, and project rules from hidden context/tool side effects. Do not pipe it
|
|
206
|
+
to `tail`, `head`, `grep`, `sed`, or `awk`; capture stdout/stderr directly.
|
|
207
|
+
Every other resolved OTHER engine follows `_shared/adapters/<name>.md`
|
|
208
|
+
`## Invocation`. Capture the result as `.devlyn/<name>-judge.stdout`, then run
|
|
209
|
+
`python3 "$DEVLYN_SHARED_DIR/collect-codex-findings.py" --devlyn-dir
|
|
210
|
+
"<abs repo>/.devlyn" --stdout-file <name>-judge.stdout` before merge.
|
|
211
|
+
The orchestrator writes the canonical `.devlyn/verify.pair.findings.jsonl`.
|
|
212
|
+
The pair prompt must include a bounded-output contract: no harness-doc reads,
|
|
242
213
|
maximum two targeted probes before first output, stop on the first
|
|
243
214
|
verdict-binding finding, and emit PASS immediately after the bounded checks pass.
|
|
244
|
-
|
|
245
|
-
|
|
246
|
-
|
|
247
|
-
`.devlyn
|
|
248
|
-
|
|
249
|
-
|
|
250
|
-
|
|
251
|
-
|
|
252
|
-
|
|
253
|
-
|
|
254
|
-
Both pair-judge directions are wall-budgeted at 600s. A judge subprocess exit
|
|
215
|
+
Raw stdout is diagnostic-only. A non-zero collector exit must write no
|
|
216
|
+
canonical findings file: set the pair source to `BLOCKED` for
|
|
217
|
+
`verify.pair.emission-contract`, and do not merge as though unparsed stdout were
|
|
218
|
+
only a diagnostic. Do not ask the judge to edit `.devlyn`.
|
|
219
|
+
|
|
220
|
+
Both pair-judge directions are wall-budgeted at 600s. The orchestrator records
|
|
221
|
+
`state.phases.verify.judge_durations_ms: {"judge": <int>, "pair_judge": <int|null>}`
|
|
222
|
+
as it collects each judge result; these wall durations are siblings of, never
|
|
223
|
+
nested inside, normalized string `sub_verdicts`. A judge subprocess exit
|
|
255
224
|
124 is a budget abort: before merge, the orchestrator writes
|
|
256
|
-
`.devlyn/verify.pair.timeout.json` with `{"engine": "<
|
|
225
|
+
`.devlyn/verify.pair.timeout.json` with `{"engine": "<resolved name>",
|
|
257
226
|
"budget_seconds": 600}`. Three cases are binding: marker plus no canonical pair
|
|
258
227
|
findings and no parseable stdout findings records `pair_judge: "TIMEOUT"`,
|
|
259
228
|
computes the merged verdict from mechanical plus primary judge, and surfaces
|