devlyn-cli 2.9.3 → 2.10.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +3 -3
- package/CLAUDE.md +5 -5
- package/README.md +5 -5
- package/benchmark/auto-resolve/fixtures/SCHEMA.md +1 -1
- package/benchmark/auto-resolve/scripts/pair_evidence_contract.py +1 -0
- package/benchmark/auto-resolve/scripts/run-fixture.sh +11 -1
- package/benchmark/auto-resolve/scripts/test-benchmark-arg-parsing.sh +83 -1
- package/bin/devlyn.js +108 -15
- package/config/skills/_shared/adapters/README.md +2 -2
- package/config/skills/_shared/adapters/grok.md +99 -0
- package/config/skills/_shared/archive_run.py +35 -1
- package/config/skills/_shared/collect-codex-findings.py +231 -50
- package/config/skills/_shared/engine-doctor.sh +6 -3
- package/config/skills/_shared/engine-preflight.md +12 -5
- package/config/skills/_shared/expected.schema.json +7 -0
- package/config/skills/_shared/grok-anchor-guard.py +61 -0
- package/config/skills/_shared/phase-prompt-render.py +181 -0
- package/config/skills/_shared/resolve-bootstrap.py +630 -0
- package/config/skills/_shared/resolve-stop-hook.py +325 -0
- package/config/skills/_shared/spec-verify-check.py +308 -31
- package/config/skills/_shared/state-phase-write.py +2966 -27
- package/config/skills/_shared/terminal-claim-check.py +537 -0
- package/config/skills/_shared/verify-merge-findings.py +533 -16
- package/config/skills/devlyn:engines/SKILL.md +2 -2
- package/config/skills/devlyn:ideate/SKILL.md +1 -1
- package/config/skills/devlyn:queue/SKILL.md +12 -0
- package/config/skills/devlyn:resolve/SKILL.md +60 -78
- package/config/skills/devlyn:resolve/references/phases/build-gate.md +2 -2
- package/config/skills/devlyn:resolve/references/phases/implement.md +1 -1
- package/config/skills/devlyn:resolve/references/phases/plan.md +1 -5
- package/config/skills/devlyn:resolve/references/phases/probe-derive.md +4 -2
- package/config/skills/devlyn:resolve/references/phases/surface-close.md +34 -0
- package/config/skills/devlyn:resolve/references/phases/verify.md +70 -97
- package/config/skills/devlyn:resolve/references/state-schema.md +26 -19
- package/optional-skills/devlyn:reap/SKILL.md +1 -1
- package/optional-skills/devlyn:reap/scripts/reap.sh +25 -13
- package/optional-skills/devlyn:reap/scripts/scan.sh +15 -1
- package/package.json +1 -1
- package/scripts/lint-fixtures.sh +4 -1
- package/scripts/lint-skills.sh +269 -77
|
@@ -25,7 +25,7 @@ DEVLYN_SHARED_DIR="$(cd "$DEVLYN_SKILL_DIR/../_shared" && pwd)"
|
|
|
25
25
|
## No args — status + how to choose
|
|
26
26
|
|
|
27
27
|
1. Read `cwd/.devlyn/engines.json`. Absent → report "no pins — auto-detection active".
|
|
28
|
-
2. Run `bash "$DEVLYN_SHARED_DIR/engine-doctor.sh"` — read-only; never installs anything or writes `.devlyn/engines.json`. It detects, per target
|
|
28
|
+
2. Run `bash "$DEVLYN_SHARED_DIR/engine-doctor.sh"` — read-only; never installs anything or writes `.devlyn/engines.json`. It detects, per target in the doctor's catalog, whether its binary is present, whether `_shared/adapters/<name>.md` exists, and whether that makes it `pin_eligible` today. Its trailing line is either a pair-judge-diversity confirmation or a recommendation to add a second adapter-valid engine (`autoresearch/iterations/0045-model-arm-drift.md`: different model tiers hit different failure-mode blind spots, so VERIFY pair-judge/risk-probes need ≥2 to ever fire).
|
|
29
29
|
3. Print the role table. Executor's "Available engines" = rows where `pin_eligible: yes` and `role` includes `executor` (i.e. not `judge-only`). Pair judge's "Available engines" = rows where `pin_eligible: yes` and `role` includes `judge` (`executor+judge` or `judge-only`):
|
|
30
30
|
|
|
31
31
|
```
|
|
@@ -40,7 +40,7 @@ pair judge codex binary rule (pin order with: pair <name>,...)
|
|
|
40
40
|
|
|
41
41
|
## Subcommands
|
|
42
42
|
|
|
43
|
-
- `executor <name>` — pin the executor (
|
|
43
|
+
- `executor <name>` — pin the executor (IMPLEMENT/CLEANUP + primary VERIFY judge; PLAN is orchestrator-fixed and never follows the pin). Refuse names with no `_shared/adapters/<name>.md`, or whose adapter's `## Role eligibility` section declares `executor: no` (judge-only backends), listing the valid names either way. If the engine's CLI is not currently available, still write the pin but warn: pins are promises — the run will stop with `BLOCKED:<name>-unavailable` until it is installed.
|
|
44
44
|
- `pair <name>[,<name>...]` — set `pair_judge_priority` (ordered; first adapter-valid, pair-judge-eligible, non-primary, available entry wins at VERIFY/risk-probe time). Refuse names with no `_shared/adapters/<name>.md`, or whose adapter declares `pair_judge: no`.
|
|
45
45
|
- `clear` — remove both pins; delete `.devlyn/engines.json` when nothing else remains in it.
|
|
46
46
|
|
|
@@ -24,7 +24,7 @@ The user does not know context engineering. They will under-specify and over-ass
|
|
|
24
24
|
</elicitation_contract>
|
|
25
25
|
|
|
26
26
|
<harness_principles>
|
|
27
|
-
Read `_shared/runtime-principles.md` (Subtractive-first / Goal-locked /
|
|
27
|
+
Read `_shared/runtime-principles.md` (Subtractive-first / Goal-locked / Evidence-over-claim). The principles bind the spec content as well as your conversation. A spec that says "for future flexibility" is a Subtractive-first violation. A spec that asks for `try { ... } catch { return null }` is a No-workaround violation. AI flags these in elicitation, not after `/devlyn:resolve` has built them.
|
|
28
28
|
</harness_principles>
|
|
29
29
|
|
|
30
30
|
<runtime_paths>
|
|
@@ -9,6 +9,17 @@ Utility front-end for the intent-queue contract in the project instructions ("In
|
|
|
9
9
|
$ARGUMENTS
|
|
10
10
|
</args>
|
|
11
11
|
|
|
12
|
+
For `drain`, resolve the installed shared scripts before the first item:
|
|
13
|
+
|
|
14
|
+
```bash
|
|
15
|
+
DEVLYN_SKILL_DIR="${CLAUDE_SKILL_DIR:-__DEVLYN_SKILL_DIR__}"
|
|
16
|
+
if [ "$DEVLYN_SKILL_DIR" = "__DEVLYN_SKILL_DIR__" ] || [ ! -d "$DEVLYN_SKILL_DIR/../_shared" ]; then
|
|
17
|
+
echo "BLOCKED:shared-dir-unresolved: $DEVLYN_SKILL_DIR/../_shared" >&2
|
|
18
|
+
exit 1
|
|
19
|
+
fi
|
|
20
|
+
DEVLYN_SHARED_DIR="$(cd "$DEVLYN_SKILL_DIR/../_shared" && pwd)"
|
|
21
|
+
```
|
|
22
|
+
|
|
12
23
|
## No args — status
|
|
13
24
|
|
|
14
25
|
Read `docs/specs/queue.md` (absent → report "queue empty — nothing staged" and how to add). Print pending `[ ]`, done `[x]`, and blocked `[F]` counts, the next item up, and one usage line per subcommand.
|
|
@@ -19,6 +30,7 @@ Read `docs/specs/queue.md` (absent → report "queue empty — nothing staged" a
|
|
|
19
30
|
- `drain` — serial drain per the project-instructions contract. For each pending item, in order:
|
|
20
31
|
1. Spec it if unspecced (the queue entry is the user's go-ahead). Unattended assumptions may only take scope-narrowing, reversible, non-user-visible defaults; material ambiguity (user-visible behavior, data/state semantics, new files/scripts/flags, implementation surface) → mark `[F] needs-review: <question>` and continue to the next item.
|
|
21
32
|
2. Run `/devlyn:resolve --spec <path>` hands-free.
|
|
33
|
+
After every resolve invocation, run `python3 "$DEVLYN_SHARED_DIR/terminal-claim-check.py" .`; exit 79 marks `[F] FAILED-INCOMPLETE` from the predicate, never from the session self-report.
|
|
22
34
|
3. Outer loop on the terminal verdict: PASS → mark `[x]`. Findings-backed verdicts (NEEDS_WORK, verify/build-gate exhaustion) → amend the spec (recorded in the spec file), re-run — at most 3 outer iterations. Infrastructure / invalid-input / engine-availability / implement-empty BLOCKED verdicts are not spec-amendable → mark `[F] <verdict>` immediately.
|
|
23
35
|
4. A blocked item never halts the queue; continue.
|
|
24
36
|
5. When the queue is drained (or the session must stop), emit the drain report: per-item verdict, every logged assumption, and the commit range produced.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: devlyn:resolve
|
|
3
|
-
description: Hands-free pipeline for any coding task — bug fix, feature, refactor, debug, modify, PR review. Free-form goal or formal spec input. Plan → Implement → Build-gate → Cleanup → Verify (fresh subagent, findings-only). Mechanical-first verification;
|
|
3
|
+
description: Hands-free pipeline for any coding task — bug fix, feature, refactor, debug, modify, PR review. Free-form goal or formal spec input. Plan → Implement → Build-gate → Cleanup → Verify (fresh subagent, findings-only). Mechanical-first verification; Verify dual-judge is default-when-available. Use when the user says "resolve this", "fix this", "implement this", "refactor this", "debug this", "review this PR", or wants hands-off completion.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
Orchestrator for the 2-skill harness pipeline. One fresh worker per phase; file-based handoff via `.devlyn/pipeline.state.json`. VERIFY spawns a fresh-context worker so independence is structural — not advisory.
|
|
@@ -18,13 +18,13 @@ Hands-free. Measured by how far we get without human intervention.
|
|
|
18
18
|
|
|
19
19
|
1. Do not prompt the user mid-pipeline. When tempted to ask, pick the safe default, proceed, and log it in the final report.
|
|
20
20
|
2. Engine availability: follow `_shared/engine-preflight.md` for the explicit-route-vs-automatic-escalation distinction and BLOCKED-vs-skip behavior. Explicit routes (`--engine`, `--risk-probes`, `--pair-verify`) never downgrade to solo; unavailable auto-escalations proceed solo and report the skip.
|
|
21
|
-
3.
|
|
21
|
+
3. Order: PLAN → RISK_PROBES? → IMPLEMENT → SURFACE_CLOSE? → BUILD_GATE → CLEANUP → VERIFY → FINAL_REPORT. No others.
|
|
22
22
|
4. Orchestrator does not write code. It parses input, spawns phases, reads state, branches on verdicts, emits the report.
|
|
23
|
-
5.
|
|
23
|
+
5. Halt only on unrecoverable worker failure, empty IMPLEMENT, exhausted BUILD_GATE/VERIFY `max_rounds`, or SURFACE_CLOSE timeout / input-or-prompt mismatch / attestation failure / execution-audit violation / out-of-surface delta / rollback failure; adjudication-malformed SURFACE_CLOSE continues after successful rollback plus transcript write-audit; otherwise continue.
|
|
24
24
|
</autonomy_contract>
|
|
25
25
|
|
|
26
26
|
<harness_principles>
|
|
27
|
-
Every phase
|
|
27
|
+
Every phase applies Subtractive-first / Goal-locked / No-workaround / Evidence, loaded by file or inline; Codex routes receive it inline.
|
|
28
28
|
</harness_principles>
|
|
29
29
|
|
|
30
30
|
<runtime_paths>
|
|
@@ -51,11 +51,12 @@ fi
|
|
|
51
51
|
Each phase routes to an engine and prepends the per-engine adapter header from `_shared/adapters/<engine>.md` (e.g. `claude.md`, `codex.md`) to the canonical phase body. Adapter is the per-model delta (Anthropic's prompt-engineering guide for Claude, OpenAI's prompt guidance for Codex). Canonical body is engine-agnostic.
|
|
52
52
|
|
|
53
53
|
- Phase spawning is mandatory for every orchestrator. Same-context PLAN / IMPLEMENT / BUILD_GATE / CLEANUP / VERIFY is a contract violation. If the current CLI cannot spawn a fresh worker, write the current phase verdict as `"BLOCKED"` and report `BLOCKED:fresh-context-unavailable` with the failed spawn command; do not continue with ad-hoc same-context execution.
|
|
54
|
-
- Claude Code phases: spawn `Agent` (`mode: "bypassPermissions"`); prompt = adapter-header + canonical-body + task-context.
|
|
54
|
+
- Claude Code phases other than PLAN: spawn `Agent` (`mode: "bypassPermissions"`); prompt = adapter-header + canonical-body + task-context. PLAN uses the native foreground call shape in PHASE 1.
|
|
55
55
|
- Codex CLI phases: shell out via `bash "$CODEX_MONITORED_PATH"` with the same compounded prompt. Each `codex exec` child is a new session/fresh context. The wrapper closes stdin and emits a heartbeat. No MCP. The wrapper call is foreground-blocking — never a background shell (`run_in_background`, `&`, `nohup`), never end your message while it runs: headless print-mode wind-down kills backgrounded children (0-byte delivery); the heartbeat is the observability channel.
|
|
56
56
|
- oh-my-pi phases: spawn the native `task` tool with a fresh `context` containing adapter-header + canonical-body + task-context. Capture the task result into `.devlyn/<phase>.stdout` and any tool error into `.devlyn/<phase>.stderr` before updating state. If the `task` tool is unavailable for an omp-routed phase, write the current phase verdict as `"BLOCKED"` and report `BLOCKED:fresh-context-unavailable`; do not fall back to same-context execution or a nested `omp -p` subprocess.
|
|
57
|
-
- Default engine: Claude for
|
|
58
|
-
-
|
|
57
|
+
- Default engine: Claude for IMPLEMENT / BUILD_GATE / CLEANUP when the orchestrator has Claude Code's native `Agent` primitive; otherwise the default route is the current CLI's fresh worker (`codex` child process for Codex CLI, native `task` for oh-my-pi). PLAN is orchestrator-fixed and never inherits `--engine`, an executor pin, or `state.engine`. Role resolution (`_shared/engine-preflight.md#role-resolution`): explicit `--engine` flag > machine-local `.devlyn/engines.json` `executor` pin > orchestrator-supported default above. Explicit routes and pins still fail closed when their required engine/spawn channel is unavailable. VERIFY/JUDGE selects the OTHER engine by default when available — OTHER = first adapter-valid, non-primary, available entry of `engines.json` `pair_judge_priority`, or the binary claude↔codex complement when unset.
|
|
58
|
+
- The `--engine` flag does not disable default pairing: the second judge uses the OTHER engine by default when available.
|
|
59
|
+
- Multi-LLM evolution: when a new model adapter ships in `_shared/adapters/`, that engine becomes selectable for the roles its adapter declares eligible without further skill changes (NORTH-STAR.md "Multi-LLM evolution direction").
|
|
59
60
|
</engine_routing>
|
|
60
61
|
|
|
61
62
|
<modes>
|
|
@@ -67,57 +68,48 @@ Three input shapes:
|
|
|
67
68
|
</modes>
|
|
68
69
|
|
|
69
70
|
<post_implement_invariant>
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
- CLEANUP may only mutate files in the cleanup allowlist (tooling artifacts, dead code added by this diff, doc references this diff invalidated). Other paths trigger revert.
|
|
73
|
-
- VERIFY runs in a fresh worker context with no code-mutation tools. Findings only — never edits files. The fresh-context spawn is the structural guarantee; the prompt body reinforces it but the spawn is what makes independence real. If no fresh worker can be spawned, VERIFY is `BLOCKED:fresh-context-unavailable`, never a same-context review.
|
|
71
|
+
After `state.implement_passed_sha` is set: SURFACE_CLOSE and CLEANUP are limited to their allowlists (empty SURFACE_CLOSE PASS is valid; violations revert); VERIFY is fresh-context, findings-only, and mutation-free. No fresh VERIFY worker → `BLOCKED:fresh-context-unavailable`, never same-context review.
|
|
74
72
|
</post_implement_invariant>
|
|
75
73
|
|
|
74
|
+
<transition_protocol>
|
|
75
|
+
For every direct complete→spawn handoff, call `state-phase-write.py ... --phase <current> transition` with the current phase's normal completion/attestation arguments plus caller-specified `--next-phase`, `--next-round`, `--next-triggered-by`, `--next-engine`, `--next-model`, and any next-phase metadata. The verb validates a legal edge and commits both lifecycle writes atomically; it returns JSON state facts only. It never selects a phase/engine, renders a prompt, or spawns an agent. Use standalone `spawn` only for the initial post-bootstrap dispatch; use standalone `complete` when no next phase opens (including a halt).
|
|
76
|
+
</transition_protocol>
|
|
77
|
+
|
|
76
78
|
## PHASE 0: PARSE + CLASSIFY + ROUTE
|
|
77
79
|
|
|
78
|
-
1.
|
|
79
|
-
- `--max-rounds N` (default 4) — fix-loop budget shared across BUILD_GATE and VERIFY.
|
|
80
|
-
- `--engine MODE` (default: `engines.json` `executor` pin when present, else `claude`) — picks the adapter for IMPLEMENT, CLEANUP, and the primary VERIFY judge. It does not disable VERIFY pair-mode; when a VERIFY pair trigger fires, the second judge uses the OTHER engine per role resolution.
|
|
81
|
-
- `--spec <path>` — switches to spec mode.
|
|
82
|
-
- `--verify-only <ref>` — switches to verify-only mode. Requires `--spec`.
|
|
83
|
-
- `--goal-file <path>` — free-form mode: read the goal text from `<path>` (relative to cwd) instead of an inline positional goal. Used by the devlynd `ResolveAdapter` launcher (goal written to `.devlyn/goal.txt`, referenced by path — never inlined as argv). The file content is opaque goal text fed to the complexity classifier, **never re-parsed as flags**. Mutually exclusive with `--spec`, `--verify-only`, and an inline positional goal.
|
|
84
|
-
- `--pair-verify` — force pair-mode JUDGE in PHASE 5 even when not auto-triggered.
|
|
85
|
-
- `--no-pair` — disable conditional VERIFY pair-JUDGE for this run. Record `pair_trigger.skipped_reason: "user_no_pair"` whenever a trigger would otherwise fire.
|
|
86
|
-
- `--risk-probes` — insert PHASE 1.5 cross-engine probe derivation. The OTHER engine converts visible `## Verification` bullets into bounded executable probes before IMPLEMENT; BUILD_GATE and VERIFY replay them mechanically.
|
|
87
|
-
- `--no-risk-probes` — disable automatic high-risk risk probes. Explicit `--risk-probes` wins over `--no-risk-probes`.
|
|
88
|
-
- `--bypass <phase>[,...]` — skip specific phases. Valid: `build-gate`, `cleanup`. PLAN, IMPLEMENT, VERIFY are non-bypassable.
|
|
89
|
-
- `--perf` — opt in to per-phase timing.
|
|
80
|
+
1. Run the bootstrap once with the exact tokenized `<pipeline_config>` and this orchestrator's default engine from `<engine_routing>`:
|
|
90
81
|
|
|
91
|
-
|
|
82
|
+
```bash
|
|
83
|
+
DEVLYN_DEFAULT_ENGINE="<current-cli-default>" python3 "$DEVLYN_SHARED_DIR/resolve-bootstrap.py" <pipeline_config tokens>
|
|
84
|
+
```
|
|
92
85
|
|
|
93
|
-
|
|
86
|
+
Read its sole JSON result. On `ok:false`, halt on its exact report-level `blocked` string and show `detail`; init failures leave no partial run artifacts and create no phase verdict. On success, the script has atomically initialized the schema-v3 skeleton (`pair_verify: true` only when `--pair-verify` was passed), stamped the null-safe Claude session id, persisted exact-byte Goal/spec identity, staged spec verification inputs, and captured the verify-only external diff. It validates only flags needed for those init fields, including mode exclusivity, `--max-rounds`, `--bypass`, and `--pair-verify`/`--no-pair`; `--pair-verify` and `--no-pair` are mutually exclusive. Free-form init sets `state.source.type = "generated"`. Spec staging validates supported `complexity` frontmatter (including sibling spec `complexity` frontmatter) and any present actionable solo-headroom hypothesis, matching its command to `spec.expected.json.verification_commands[].cmd` or the inline `## Verification` JSON carrier. It does not validate engines, write the untracked baseline, classify complexity/risk, choose a route, or announce. `state.engine` is the raw `--engine` value with `engine_source: "flag"`, otherwise the passed `DEVLYN_DEFAULT_ENGINE` with `engine_source: "default"`; step 2 resolves and replaces both fields.
|
|
94
87
|
|
|
95
|
-
|
|
88
|
+
2. Engine pre-flight: follow `_shared/engine-preflight.md`, including role resolution. If `.devlyn/engines.json` exists: malformed JSON, any named engine without a `_shared/adapters/<name>.md` adapter, or an adapter that declares itself ineligible for the requested role, halts with report-level `BLOCKED:invalid-engine-config` naming the offending entry. A pin is an explicit route — a pinned but unavailable engine halts with `BLOCKED:<engine>-unavailable`. If a required engine is unavailable, halt with a BLOCKED verdict and setup instructions instead of downgrading. After successful role resolution, replace `state.engine` and `state.engine_source` with the resolved executor and its provenance (`"flag" | "engines.json" | "default"`). (These and all other PHASE-0 halts are report-level `BLOCKED:<reason>` strings only; the JSON `phases.*.verdict` carrier in `pipeline.state.json` always stays the bare enum `"BLOCKED"`.)
|
|
96
89
|
|
|
97
|
-
|
|
90
|
+
3. Write `.devlyn/untracked.baseline`: `python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --write-untracked-baseline`.
|
|
98
91
|
|
|
99
|
-
|
|
100
|
-
- **Free-form**: first resolve `goal_text` — the inline positional goal, OR (when `--goal-file <path>` was passed) the raw UTF-8 content of `<path>`, read as opaque goal text (never re-parsed as flags). Fail closed before classifying: empty/whitespace-only file → `BLOCKED:goal-file-empty`; missing/unreadable/not-a-regular-file/invalid-UTF-8 → `BLOCKED:goal-file-unreadable`; an absolute path, a `..` segment, or a path that canonicalizes outside cwd → `BLOCKED:goal-file-invalid-path`. (Like all PHASE-0 halts these are report-level `BLOCKED:<reason>` strings — no `phases.*.verdict` is written, so the bare-enum carrier rule is never violated.) Then read `references/free-form-mode.md` and run the complexity classifier deterministically on `goal_text` (rules over keyword density / file count / spec-shape signals, plus pair-evidence intent). Set `state.complexity ∈ {trivial, medium, large}`. Trivial: write internal mini-spec to `.devlyn/criteria.generated.md` and proceed. Medium: synthesize a minimal spec from the goal + add 1-2 context anchors from the codebase, write to `.devlyn/criteria.generated.md`, proceed. Every free-form branch that writes criteria must set `state.source.type = "generated"`, `state.source.criteria_path = ".devlyn/criteria.generated.md"`, and `state.source.criteria_sha256` from the raw file bytes. Large: follow the Large action in `references/free-form-mode.md` — synthesize a best-effort spec with an `## Assumptions` block, log `recommend: /devlyn:ideate first`, proceed, and flag every assumption in the final report; zero-scope-signal goals halt `BLOCKED:large-needs-ideation`; pair-evidence intent without an actionable solo-headroom hypothesis must halt with `BLOCKED:solo-headroom-hypothesis-required`, and unmeasured pair-candidate intent without solo ceiling avoidance must halt with `BLOCKED:solo-ceiling-avoidance-required`.
|
|
101
|
-
- **Spec**: validate spec exists. If sibling `spec.expected.json` exists, run `--check-expected <expected-path>` to validate both the expected contract, sibling spec `complexity` frontmatter, and any present actionable solo-headroom hypothesis; if the spec has a solo-headroom hypothesis, its observable command must match `spec.expected.json.verification_commands[].cmd`. Then stage `.devlyn/spec-verify.json` from `verification_commands`. Otherwise run `--check <spec-path>` to validate the legacy inline carrier plus supported `complexity` frontmatter and any present actionable solo-headroom hypothesis; if the spec uses an inline `## Verification` JSON carrier, any solo-headroom hypothesis command must match that carrier's `verification_commands[].cmd`. Then stage from the legacy inline carrier. Compute `state.source.spec_sha256`.
|
|
102
|
-
- **Verify-only**: skip to PHASE 5 with `state.source.spec_path` set, the supplied diff captured at `.devlyn/external-diff.patch`.
|
|
92
|
+
4. Read the Goal/spec through `state.source`. For free-form mode, run the deterministic classifier in `references/free-form-mode.md`. Zero-scope-signal goals halt with `BLOCKED:large-needs-ideation`; pair-evidence intent without an actionable solo-headroom hypothesis must halt with `BLOCKED:solo-headroom-hypothesis-required`; unmeasured pair-candidate intent without solo ceiling avoidance must halt with `BLOCKED:solo-ceiling-avoidance-required`. Follow the selected branch to write the trivial/medium/large body of `.devlyn/criteria.generated.md`, then set `state.complexity` and its raw-byte `criteria_sha256`. The Large `## Assumptions`/recommendation/final-report obligations remain unchanged.
|
|
103
93
|
|
|
104
|
-
|
|
94
|
+
Compute `state.risk_profile` from the user goal plus spec/criteria text. Mark `high_risk: true` for auth/authz, permissions, security, token/session, payment/money/billing/invoice/pricing/tax/ledger, persistence/data mutation/deletion/migration, idempotency/replay/duplicate, API/webhook/raw-body/signature, allocation/scheduling/inventory/rollback/transaction, or explicit error-priority/output-shape contracts. Explicit `--risk-probes` sets both probe booleans true. Otherwise an automatic high-risk route enables probes only when the resolved OTHER engine is available and `--no-risk-probes` is absent; if unavailable, keep probes disabled and append `auto-risk-probes skipped: <engine>-unavailable`. `--no-pair` sets `pair_default_enabled: false`. Preserve strict boolean/list types and concise string reasons.
|
|
105
95
|
|
|
106
|
-
|
|
96
|
+
5. Announce one line: `resolve starting — run <run_id> — engine <engine> — mode <mode> — complexity <complexity-or-na> — pair <on|solo:auto_pair_other_engine_unavailable|disabled> — risk_probes <on|off>`.
|
|
97
|
+
|
|
98
|
+
6. Immediately before the first worker dispatch, open the caller-selected `plan` (normal run) or `verify` (verify-only) span with standalone `state-phase-write.py ... spawn`; then dispatch that same phase. The bootstrap never chooses a phase, engine, branch, prompt, or agent.
|
|
107
99
|
|
|
108
100
|
## PHASE 1: PLAN
|
|
109
101
|
|
|
110
102
|
Skip in verify-only mode. The heaviest phase by design — spec/criteria define non-negotiable invariants; plan formalizes how the implementation hits them.
|
|
111
103
|
|
|
112
|
-
Engine: Claude (PLAN-pair is **research-only** — iter-0033d/f/g; unblock conditions in PHASE 1 / iter-0033g §H; iter-0020 falsified Codex-BUILD/IMPLEMENT, NOT PLAN-pair).
|
|
104
|
+
Engine: PLAN is orchestrator-fixed and never inherits `--engine`, an executor pin, or `state.engine`; Claude Code uses a native Claude `Agent` worker, while Codex CLI / oh-my-pi use their own fresh worker per `_shared/engine-preflight.md` (PLAN-pair is **research-only** — iter-0033d/f/g; unblock conditions in PHASE 1 / iter-0033g §H; iter-0020 falsified Codex-BUILD/IMPLEMENT, NOT PLAN-pair). Before rendering, assign the current orchestrator's literal adapter name (`PLAN_ENGINE=claude`, `PLAN_ENGINE=codex`, or `PLAN_ENGINE=omp`) and assign `PLAN_MODEL=<exact model id passed to that fresh worker>`; replace the metavalue and fail closed if either assignment is empty. Write the data-only task context to `.devlyn/plan.task-context` with `PLAN_WORKING_DIR="$(pwd -P)"`: its first two lines are exactly `Working directory: $PLAN_WORKING_DIR` and `Plan output: $PLAN_WORKING_DIR/.devlyn/plan.md`, followed by the task data. Then render adapter + `references/phases/plan.md` + task context without byte normalization: `PLAN_PROMPT_SHA256="$(python3 "$DEVLYN_SHARED_DIR/phase-prompt-render.py" --adapter "$DEVLYN_SHARED_DIR/adapters/$PLAN_ENGINE.md" --canonical-body "$DEVLYN_SKILL_DIR/references/phases/plan.md" --task-context .devlyn/plan.task-context --output .devlyn/plan.prompt)"`. The renderer fails closed before writing `.devlyn/plan.prompt` if either fixed header field is missing, non-absolute, or mismatches the active repository paths. Open round 0 with `state-phase-write.py ... --phase plan spawn --round 0 --engine "$PLAN_ENGINE" --model "$PLAN_MODEL" --prompt-sha256 "$PLAN_PROMPT_SHA256"`. In Claude Code, invoke the native `Agent` with its `prompt` field set to the exact `.devlyn/plan.prompt` bytes, explicitly set `run_in_background: false`, and omit `mode`; this is a data field, so a path, `$(cat ...)`, or any variable-reference substitution is not the rendered prompt. If native input validation rejects this call, surface the failure without retrying a different `mode` or background shape and fail closed through the existing `BLOCKED:fresh-context-unavailable` path. The PLAN worker inherits the parent's permission context: the hands-free parent already uses native `bypassPermissions`, while a non-bypass parent's permission prompt remains the user's native boundary and is never silently escalated. Codex CLI and oh-my-pi deliver the same prompt through their unchanged routes. Product code records rendered intent; it never parses a transcript or intercepts, rewrites, wraps, or forces native delivery.
|
|
113
105
|
|
|
114
106
|
Subagent output (writes `.devlyn/plan.md`): file list to touch, risk list (out-of-scope expansions, ambiguous spec sections), acceptance restatement (what `## Verification` actually requires verbatim). For large work only, PLAN may add a fourth section `## Execution phases` under the conditions in `references/phases/plan.md`; a phase block missing a runnable `gate:` line disqualifies the whole section — treat the run as single-phase and log the disqualification.
|
|
115
107
|
|
|
116
|
-
State write: `
|
|
108
|
+
State write: each PLAN receipt carries spawn-known `round`, `started_at`, `triggered_by`, `engine`, `model_requested`, and `prompt_sha256`; completion enriches it with `completed_at`, `duration_ms`, `verdict`, and nullable `model_effective`.
|
|
117
109
|
|
|
118
110
|
After return:
|
|
119
111
|
1. If `.devlyn/plan.md` lists zero files → halt with verdict `BLOCKED:plan-empty`.
|
|
120
|
-
2. If risk list flags an out-of-scope expansion the user did not authorize → re-spawn
|
|
112
|
+
2. If risk list flags an out-of-scope expansion the user did not authorize → the sole legal corrective re-spawn is round 1: update the data-only task context, repeat render + digest, open PLAN with `--round 1 --triggered-by plan` and the actual engine/model/digest, then deliver the new `.devlyn/plan.prompt` using the same native call shape as round 0. A second failed plan halts; no further PLAN dispatch is authorized.
|
|
121
113
|
3. After any re-spawn above, if `state.risk_profile.risk_probes_enabled == true` and `state.risk_profile.risk_probes_explicit == false`, parse the `authorized_surface` array from the JSON block under `<!-- devlyn:authorized-surface -->`. For a well-formed string array, compute `probe_scale_small := len(authorized_surface) <= 2 AND no entry ends in "/**"`. If true, set `risk_probes_enabled = false`, leave `high_risk` unchanged, and append `auto-risk-probes demoted: plan surface small (<n> paths)` to `reasons` using the actual length. A missing or malformed block leaves probe state unchanged; BUILD_GATE owns its malformed-block failure.
|
|
122
114
|
|
|
123
115
|
## PHASE 1.5: RISK_PROBES
|
|
@@ -125,7 +117,7 @@ After return:
|
|
|
125
117
|
Skip unless `--risk-probes` is set OR `state.risk_profile.risk_probes_enabled`
|
|
126
118
|
is true. This phase is findings-as-executable-checks, not a second plan and not
|
|
127
119
|
debate. When it runs, the OTHER engine is required: if unavailable, halt with
|
|
128
|
-
`BLOCKED
|
|
120
|
+
`BLOCKED:<engine>-unavailable` plus setup guidance;
|
|
129
121
|
do not silently continue without probes. Reaching this halt means the route was
|
|
130
122
|
explicitly requested (`--risk-probes`) — an auto high-risk escalation toward an
|
|
131
123
|
unavailable OTHER engine was already gated off in PHASE 0 by leaving
|
|
@@ -164,7 +156,8 @@ When visible text names exact keys, fields, row shapes, JSON objects, response
|
|
|
164
156
|
bodies, stdout/stderr objects, or exact error bodies, `shape_contract` must
|
|
165
157
|
include `uses_visible_input_key_names`, `asserts_visible_output_key_names`, and
|
|
166
158
|
`asserts_no_unexpected_output_keys`; exact JSON error objects/bodies must also
|
|
167
|
-
include `
|
|
159
|
+
include `visible_text_names_exact_json_error_object` and
|
|
160
|
+
`asserts_exact_error_object`. Cart/pricing success probes should use
|
|
168
161
|
`shape_contract` unless they satisfy the `ordering_inversion` markers. The probe
|
|
169
162
|
command must not reference external network URLs; use only worktree-local or
|
|
170
163
|
localhost resources.
|
|
@@ -239,6 +232,14 @@ State write: `phases.implement.{started_at, verdict, completed_at, duration_ms}`
|
|
|
239
232
|
|
|
240
233
|
After the final phase's gate PASS: `git diff <base_ref.sha>...HEAD --stat` — empty → halt with `BLOCKED:implement-empty`; otherwise set `state.implement_passed_sha = git rev-parse HEAD` (phase commits already checkpoint the work — no extra commit).
|
|
241
234
|
|
|
235
|
+
## PHASE 2.5: SURFACE_CLOSE
|
|
236
|
+
|
|
237
|
+
Run once iff `state.source.type == "generated"` and complexity is trivial/medium. Engine is Claude always, pair-judge-routed; executor flag/pin ignored. Per `_shared/engine-preflight.md`, unavailability skips via `surface-skip`, reports `auto_surface_close_claude_unavailable`, never BLOCKs/reroutes.
|
|
238
|
+
|
|
239
|
+
Freeze `.devlyn/surface-close.input.patch`; assemble Claude adapter + `references/phases/surface-close.md` canonical body VERBATIM + data-only Goal/patch/hashes/surface/commands; hash the bytes and spawn with `--tools "Read,Grep,Glob,Edit,Write" --dangerously-skip-permissions --model claude-sonnet-5 --output-format json --strict-mcp-config --mcp-config '{"mcpServers":{}}'`, recording the same model at SPW spawn. Bound at 600s (native or `run-bounded.py 600 -- claude -p`), else block pre-spawn. Save raw stdout as `.devlyn/surface-close.output.json`, extract its `result` string to `.devlyn/surface-close.stdout`, retain `.devlyn/surface-close.worker-session.<round>.jsonl`, and complete with the wrapper as `--engine-session-log`; non-JSON/missing result follows the existing failure path. Workers execute nothing; `surface-check` gates rows, citations, scope, and execution. Empty PASS completes; authorized delta is scoped-staged and committed. Only `BLOCKED:surface-close-adjudication-malformed` runs `surface-adjudication-recover --authorized-surface-json <array>`, whose successful rollback and transcript Edit/Write audit complete the nonterminal skip carrier and continue to BUILD_GATE. Timeout, input/prompt mismatch, attestation failure, execution-audit violation, and out-of-surface delta run `surface-rollback`, complete bare `BLOCKED`, and halt; rollback failure also halts. One shot; no `max_rounds`.
|
|
240
|
+
|
|
241
|
+
**Common post-fix checkpoint (BUILD_GATE and VERIFY):** After fix IMPLEMENT returns, increment `state.rounds.global`; scoped-stage the authorized surface and commit `chore(pipeline): implement fix round <n>`; run `python3 "$DEVLYN_SHARED_DIR/state-phase-write.py" --devlyn-dir .devlyn --phase implement durability-enforce --round <n> --origin-phase <build_gate|verify>`. Its receipt must PASS before re-entry, which enforces it again before VERIFY artifact clearing or phase spawn.
|
|
242
|
+
|
|
242
243
|
## PHASE 3: BUILD_GATE
|
|
243
244
|
|
|
244
245
|
Skip in verify-only mode OR when `build-gate` in `state.bypasses`. Deterministic — same commands CI / Docker / production run.
|
|
@@ -254,7 +255,7 @@ State write: `phases.build_gate.{started_at, verdict, completed_at, duration_ms,
|
|
|
254
255
|
|
|
255
256
|
Branch:
|
|
256
257
|
- `PASS` → PHASE 4.
|
|
257
|
-
- `FAIL` → fix loop. Spawn IMPLEMENT-engine agent with the build_gate findings as input
|
|
258
|
+
- `FAIL` → fix loop. Spawn IMPLEMENT-engine agent with the build_gate findings as input, then run the common post-fix checkpoint with origin `build_gate`. On second FAIL with `state.rounds.global >= state.rounds.max_rounds` → halt with verdict `BLOCKED:build-gate-exhausted`.
|
|
258
259
|
|
|
259
260
|
## PHASE 4: CLEANUP
|
|
260
261
|
|
|
@@ -282,46 +283,27 @@ Two sub-phases:
|
|
|
282
283
|
|
|
283
284
|
1. **MECHANICAL** (deterministic): re-run `SPEC_VERIFY_PHASE=verify_mechanical SPEC_VERIFY_FINDINGS_FILE=verify-mechanical.findings.jsonl SPEC_VERIFY_FINDING_PREFIX=VERIFY-MECH python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --include-risk-probes` against the post-CLEANUP code (independent of BUILD_GATE's earlier run). If `state.risk_profile.risk_probes_enabled == true`, missing `.devlyn/risk-probes.jsonl` is a CRITICAL mechanical blocker. This emits `.devlyn/verify-mechanical.findings.jsonl` for `verify-merge-findings.py`.
|
|
284
285
|
|
|
285
|
-
2. **JUDGE** (fresh-context Agent): grade the diff against the spec on rubric axes (spec compliance, scope, quality, consistency). Split each Requirement into binding clauses and trace code-order counterexamples; a passing verifier proves only the case it exercises, not neighboring `once` / `regardless` / `duplicate` / auth-order / rollback invariants. Respect scope qualifiers such as `inside a warehouse`, `per resource`, `for this line`, and `after validation`; do not widen a scoped clause into a global invariant, and compose multiple ordering rules in the stated order. For stateful flows, explicitly trace failed-operation rollback and the next entity's state before hunting broader edge cases. For high-complexity specs, construct at least one interaction counterexample that combines ordering/priority with failure handling and state mutation, then execute at least one such scenario through the repo's existing CLI/API/test runner without leaving tracked files behind; one-axis examples and pure mental tracing are insufficient. Default engine = same as IMPLEMENT
|
|
286
|
-
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
|
|
294
|
-
|
|
295
|
-
|
|
296
|
-
|
|
297
|
-
|
|
298
|
-
|
|
299
|
-
|
|
300
|
-
|
|
301
|
-
every reason is automatic) — the OTHER engine being unavailable. In that
|
|
302
|
-
auto-only-unavailable case, set `eligible: false`, leave `reasons` empty, set
|
|
303
|
-
`skipped_reason: "auto_pair_other_engine_unavailable"`, and run solo VERIFY.
|
|
304
|
-
Explicit `--pair-verify` (`mode.pair-verify`) with the OTHER engine absent is a
|
|
305
|
-
promise: it stays `BLOCKED:<engine>-unavailable` and is never skipped.
|
|
306
|
-
|
|
307
|
-
Pair-mode JUDGE: spawn a second Agent with the OTHER engine's adapter; the second judge is a bounded adversarial complement, not a duplicate broad audit. The primary judge owns broad coverage; pair-JUDGE targets the two highest-risk explicit `## Verification` bullets that cross state mutation, all-or-nothing rollback, ordering, idempotency, auth, or error-priority clauses. If the spec includes a solo-headroom hypothesis, one of those targeted probes must exercise that hypothesis with the visible command/input shape and full externally visible result, using the hypothesis's backticked observable command as its command anchor before adding bounded input variations. It must not read `.claude/skills`, `.codex/skills`, `CLAUDE.md`, `AGENTS.md`, or other harness docs unless the orchestrator pasted a specific excerpt into the prompt. It may use only the spec, diff, implementation files, tests, and the repo's existing CLI/API/test runner. It may execute at most two targeted probes before first output, and each probe must compare the full externally visible result (exit/stdout/stderr plus full parsed output object, including accepted/scheduled rows, rejected rows, and remaining state when present), not just a single property. When the spec names exact keys, row shapes, JSON object shape, or an exact error body, pair-JUDGE must compare parsed key sets/deep equality so aliased keys, missing keys, and extra keys are verdict-binding failures, and it must construct inputs with the spec's visible key names. For priority/stateful specs, at least one probe must include an earlier input entity that would succeed under input-order processing, a later higher-priority entity that consumes or blocks the critical resource, and a failure/blocked/rollback edge that determines a later entity's state. For cart/pricing specs where visible verification combines duplicate items, line promotions, tax, coupon, and shipping, the success-path probe must include interleaved duplicates plus taxable and non-taxable items and assert full output rows. Scope qualifiers are binding: pair-JUDGE must not reinterpret `inside a warehouse`, `per resource`, or line-scoped rules as global rules. When both priority ordering and rollback/blocked-interval behavior appear in the spec, this dominance-loss probe is mandatory and comes before any other probe: an earlier lower-priority entity that would succeed alone or under input-order processing must lose because a later higher-priority entity is processed first; a failed/blocked middle entity must not corrupt later state; and the assertion must cover complete accepted/scheduled and rejected output ordering. It must stop and emit JSONL immediately on the first verdict-binding finding, and must emit PASS immediately if both probes plus static scope/dependency checks pass. Both judgments merge with the rule "any HIGH/CRITICAL finding either model surfaces is verdict-binding; high-confidence MEDIUM findings are also verdict-binding when they identify a concrete behavioral regression against the spec, public contract, or existing test contract." Cross-model disagreement on advisory lower-severity findings is logged but does not change the verdict. If MECHANICAL or the primary JUDGE has a verdict-binding finding, skip the second judge and record `pair_judge: null`; the fix loop needs the blocker, not duplicate review.
|
|
308
|
-
|
|
309
|
-
If pair-mode is triggered and the OTHER engine is unavailable, do not downgrade
|
|
310
|
-
or skip the required judge. Set VERIFY to `BLOCKED:<engine>-unavailable`, preserve the
|
|
311
|
-
failed availability check evidence, and print setup guidance:
|
|
312
|
-
|
|
313
|
-
- Codex: install/configure the Codex CLI, run `codex auth` or the current login
|
|
314
|
-
flow, verify `codex --version`, then rerun. Use `--no-pair` only when the user
|
|
315
|
-
intentionally accepts solo VERIFY for this run.
|
|
316
|
-
- Claude: install/configure Claude Code, run `claude --version` when available,
|
|
317
|
-
confirm the host can spawn Claude agents, then rerun.
|
|
318
|
-
|
|
319
|
-
Findings written to `.devlyn/verify.findings.jsonl`. **VERIFY agents have no code-mutation tools.** Codex pair-JUDGE is read-only: invoke `codex-monitored.sh` with `CODEX_MONITORED_ISOLATED=1 CODEX_MONITORED_TIMEOUT_SEC=600` and `-c model_reasoning_effort=medium`, no `tail`/`head`/`grep` pipes, direct stdout/stderr capture, JSONL findings on stdout, and orchestrator-written `.devlyn/verify.pair.findings.jsonl`. Isolation blocks user config, AGENTS.md, hooks, and project rules from hidden context/tool/transcript side effects. If stdout is captured as `.devlyn/codex-judge.stdout`, run `python3 "$DEVLYN_SHARED_DIR/collect-codex-findings.py"` before merge; raw stdout is diagnostic only. If stdout contains findings or a non-PASS summary while `.devlyn/verify.pair.findings.jsonl` is empty, `verify-merge-findings.py` blocks VERIFY for `verify.pair.emission-contract`. Do not ask Codex to `apply_patch` or edit `.devlyn`. When the OTHER engine is Claude (codex/omp orchestrator), the judge call is wrapped as `python3 "$DEVLYN_SHARED_DIR/run-bounded.py" 600 -- claude -p ...` with the same flags as `_shared/adapters/claude.md` `## Invocation`, stdout captured as `.devlyn/claude-judge.stdout`, and the same emission contract applied to that file. On judge subprocess exit 124, the orchestrator writes `.devlyn/verify.pair.timeout.json` (`{"engine": "<codex|claude>", "budget_seconds": 600}`) before running `verify-merge-findings.py`. After primary and pair findings are written, run `python3 "$DEVLYN_SHARED_DIR/verify-merge-findings.py" --write-state`. Branch only on the merged `state.phases.verify.verdict`; a HIGH/CRITICAL finding from either judge must mechanically become `NEEDS_WORK`. Never write `.devlyn/verify-merged.findings.jsonl` or `.devlyn/verify-merge.summary.json` by hand; `verify-merge-findings.py` is their only writer. State write: `phases.verify.{started_at, verdict, completed_at, duration_ms, sub_verdicts: {mechanical, judge, pair_judge?}, artifacts}`.
|
|
286
|
+
2. **JUDGE** (fresh-context Agent): grade the diff against the spec on rubric axes (spec compliance, scope, quality, consistency). Split each Requirement into binding clauses and trace code-order counterexamples; a passing verifier proves only the case it exercises, not neighboring `once` / `regardless` / `duplicate` / auth-order / rollback invariants. Respect scope qualifiers such as `inside a warehouse`, `per resource`, `for this line`, and `after validation`; do not widen a scoped clause into a global invariant, and compose multiple ordering rules in the stated order. For stateful flows, explicitly trace failed-operation rollback and the next entity's state before hunting broader edge cases. For high-complexity specs, construct at least one interaction counterexample that combines ordering/priority with failure handling and state mutation, then execute at least one such scenario through the repo's existing CLI/API/test runner without leaving tracked files behind; one-axis examples and pure mental tracing are insufficient. Default engine = same as IMPLEMENT. MECHANICAL always runs first; a verdict-binding blocker routes to the fix loop without either judge. After MECHANICAL passes, `--no-pair` continues solo with `user_no_pair`. Otherwise resolve the OTHER engine. An unavailable OTHER engine blocks an explicit `--pair-verify` promise; an automatic route records `auto_pair_other_engine_unavailable` and continues solo.
|
|
287
|
+
|
|
288
|
+
When the OTHER engine is available, persist `pair_trigger` before spawn with `pair.default` plus every applicable outcome-independent reason, then dispatch both judges concurrently against the same frozen diff via foreground parallel dispatch, never background shells. A primary blocker never cancels the pair judge. If foreground parallel dispatch is unavailable, run the same two required judges sequentially; this changes dispatch shape only and never permits outcome-dependent escalation or a pair skip. Prefer read-only checks and serialize probes that need exclusive shared state.
|
|
289
|
+
|
|
290
|
+
Keep the existing reasons as telemetry: `mode.verify-only`, `mode.pair-verify`, `complexity.high`, `complexity.large`, `spec.complexity.high`, `spec.complexity.large`, `spec.solo_headroom_hypothesis`, `risk.high`, `risk_probes.enabled`, `risk_probes.present`, `coverage.failed`, `mechanical.warning`, and `judge.warning`. Append the outcome-dependent reasons only after both judges return. Eligible schema-v3 state must include `pair.default`; all other applicable reasons remain completeness-checked. Malformed risk or trigger state BLOCKs VERIFY.
|
|
291
|
+
|
|
292
|
+
Pair-mode JUDGE: spawn a second Agent with the OTHER engine's adapter; the second judge is a bounded adversarial complement, not a duplicate broad audit. The primary judge owns broad coverage; pair-JUDGE targets the two highest-risk explicit `## Verification` bullets that cross state mutation, all-or-nothing rollback, ordering, idempotency, auth, or error-priority clauses. If the spec includes a solo-headroom hypothesis, one of those targeted probes must exercise that hypothesis with the visible command/input shape and full externally visible result, using the hypothesis's backticked observable command as its command anchor before adding bounded input variations. It must not read `.claude/skills`, `.codex/skills`, `CLAUDE.md`, `AGENTS.md`, or other harness docs unless the orchestrator pasted a specific excerpt into the prompt. It may use only the spec, diff, implementation files, tests, and the repo's existing CLI/API/test runner. It may execute at most two targeted probes before first output, and each probe must compare the full externally visible result (exit/stdout/stderr plus full parsed output object, including accepted/scheduled rows, rejected rows, and remaining state when present), not just a single property. When the spec names exact keys, row shapes, JSON object shape, or an exact error body, pair-JUDGE must compare parsed key sets/deep equality so aliased keys, missing keys, and extra keys are verdict-binding failures, and it must construct inputs with the spec's visible key names. For priority/stateful specs, at least one probe must include an earlier input entity that would succeed under input-order processing, a later higher-priority entity that consumes or blocks the critical resource, and a failure/blocked/rollback edge that determines a later entity's state. For cart/pricing specs where visible verification combines duplicate items, line promotions, tax, coupon, and shipping, the success-path probe must include interleaved duplicates plus taxable and non-taxable items and assert full output rows. Scope qualifiers are binding: pair-JUDGE must not reinterpret `inside a warehouse`, `per resource`, or line-scoped rules as global rules. When both priority ordering and rollback/blocked-interval behavior appear in the spec, this dominance-loss probe is mandatory and comes before any other probe: an earlier lower-priority entity that would succeed alone or under input-order processing must lose because a later higher-priority entity is processed first; a failed/blocked middle entity must not corrupt later state; and the assertion must cover complete accepted/scheduled and rejected output ordering. It must stop and emit JSONL immediately on the first verdict-binding finding, and must emit PASS immediately if both probes plus static scope/dependency checks pass. Both judgments merge with the rule "any HIGH/CRITICAL finding either model surfaces is verdict-binding; high-confidence MEDIUM findings are also verdict-binding when they identify a concrete behavioral regression against the spec, public contract, or existing test contract." Cross-model disagreement on advisory lower-severity findings is logged but does not change the verdict. A primary JUDGE verdict-binding finding on a concurrent run does not cancel or discard the pair-JUDGE; both finding sets join the same fix round, and the merged verdict remains the worst source verdict without vote counting. `primary_judge_blocker` is parser-recognized only for archived v2.0 replay; new runs never write it. A verdict-binding MECHANICAL blocker still skips both judges and records `pair_judge: null`.
|
|
293
|
+
|
|
294
|
+
If the OTHER engine is unavailable, automatic VERIFY records
|
|
295
|
+
`auto_pair_other_engine_unavailable` and continues solo. Explicit `--pair-verify`
|
|
296
|
+
instead sets VERIFY to `BLOCKED:<engine>-unavailable`, preserves the failed
|
|
297
|
+
check, and prints setup guidance: install/configure the missing CLI, complete
|
|
298
|
+
its auth/login flow, verify `<engine> --version`, then rerun. `--no-pair` is only
|
|
299
|
+
for an intentional solo run.
|
|
300
|
+
|
|
301
|
+
Findings written to `.devlyn/verify.findings.jsonl`. **VERIFY agents have no code-mutation tools.** For Codex pair-JUDGE, keep the read-only `codex-monitored.sh` route with `CODEX_MONITORED_ISOLATED=1 CODEX_MONITORED_TIMEOUT_SEC=600` and `-c model_reasoning_effort=medium`, no `tail`/`head`/`grep` pipes, and direct stdout/stderr capture; every other resolved OTHER engine follows `_shared/adapters/<name>.md` `## Invocation`. Capture `.devlyn/<name>-judge.stdout`, then run `python3 "$DEVLYN_SHARED_DIR/collect-codex-findings.py" --devlyn-dir "<abs repo>/.devlyn" --stdout-file <name>-judge.stdout` before merge. Raw stdout is diagnostic-only. A non-zero collector exit must write no canonical `.devlyn/verify.pair.findings.jsonl`; set the pair source to `BLOCKED` for `verify.pair.emission-contract` and never merge as though unparsed stdout were only a diagnostic. VERIFY prompts remain bounded, read-only, and must not ask the judge to edit `.devlyn`. On judge subprocess exit 124, write `.devlyn/verify.pair.timeout.json` with `{"engine": "<resolved name>", "budget_seconds": 600}` before merge. The orchestrator records `phases.verify.judge_durations_ms: {judge: <int>, pair_judge: <int|null>}` as it collects each judge result; durations never nest inside `sub_verdicts`. After primary and pair findings are written, run `python3 "$DEVLYN_SHARED_DIR/verify-merge-findings.py" --write-state`. Branch only on the merged `state.phases.verify.verdict`; a HIGH/CRITICAL finding from either judge must mechanically become `NEEDS_WORK`. Never write `.devlyn/verify-merged.findings.jsonl` or `.devlyn/verify-merge.summary.json` by hand; `verify-merge-findings.py` is their only writer. State write: `phases.verify.{started_at, verdict, completed_at, duration_ms, judge_durations_ms: {judge, pair_judge}, sub_verdicts: {mechanical, judge, pair_judge?}, artifacts}`.
|
|
320
302
|
|
|
321
303
|
Branch:
|
|
322
304
|
- `PASS` → PHASE 6.
|
|
323
305
|
- `PASS_WITH_ISSUES` (LOW severity only) → PHASE 6 with banner.
|
|
324
|
-
- `NEEDS_WORK` / `BLOCKED` → fix loop with `triggered_by: "verify"`. Spawn IMPLEMENT-engine agent with the verify findings
|
|
306
|
+
- `NEEDS_WORK` / `BLOCKED` → fix loop with `triggered_by: "verify"`. Spawn IMPLEMENT-engine agent with the verify findings, then run the common post-fix checkpoint with origin `verify`. Second `NEEDS_WORK` → halt with verdict `BLOCKED:verify-exhausted`.
|
|
325
307
|
|
|
326
308
|
## PHASE 6: FINAL REPORT + ARCHIVE
|
|
327
309
|
|
|
@@ -331,9 +313,9 @@ State write: `phases.final_report.started_at` at the top of this phase.
|
|
|
331
313
|
|
|
332
314
|
2. **FINISH GATE** — run `python3 "$DEVLYN_SHARED_DIR/finish-gate.py"`; branch only on exit code plus `.devlyn/finish-gate.findings.jsonl` existence: 0 without the file → clean; 0 with it → terminal verdict floors at `PASS_WITH_ISSUES` (report lists the reverted paths); 1 or 2 → `BLOCKED:finish-gate-unclean`.
|
|
333
315
|
|
|
334
|
-
3. **Terminal verdict** — derive from `state.phases.{plan, implement, build_gate, cleanup, verify}.verdict` per the precedence rules in `references/state-schema.md#terminal-verdict`. Verify-only mode short-circuits to `state.phases.verify.verdict`.
|
|
316
|
+
3. **Terminal verdict** — derive from `state.phases.{plan, implement, surface_close, build_gate, cleanup, verify}.verdict` per the precedence rules in `references/state-schema.md#terminal-verdict`. Verify-only mode short-circuits to `state.phases.verify.verdict`.
|
|
335
317
|
|
|
336
|
-
4. **Render report** — sections: header (run_id, engine, mode, verdict, wall-time), per-phase summary, pair/risk-probe status, findings table (verify + finish-gate findings), follow-up notes (any large-mode `## Assumptions` block, any pair-judge TIMEOUT (headline: solo verdict after pair TIMEOUT), any `--no-pair` / `--no-risk-probes` opt-out, any engine setup guidance after BLOCKED, `/devlyn:ideate` guidance after `BLOCKED:solo-headroom-hypothesis-required` that asks for the visible behavior `solo_claude` is expected to miss, and `/devlyn:ideate` guidance after `BLOCKED:solo-ceiling-avoidance-required` that asks for the concrete difference from rejected or solo-saturated controls such as `S2`-`S6`).
|
|
318
|
+
4. **Render report** — sections: header (run_id, engine, mode, verdict, wall-time), per-phase summary (including SURFACE_CLOSE run or skip), pair/risk-probe status, findings table (verify + finish-gate findings), follow-up notes (the explicit line `pipeline continued to BUILD_GATE — surface_close_rolled_back_adjudication_malformed` when `continued_after_block` is set, any large-mode `## Assumptions` block, any pair-judge TIMEOUT (headline: solo verdict after pair TIMEOUT), any `--no-pair` / `--no-risk-probes` opt-out, any engine setup guidance after BLOCKED, `/devlyn:ideate` guidance after `BLOCKED:solo-headroom-hypothesis-required` that asks for the visible behavior `solo_claude` is expected to miss, and `/devlyn:ideate` guidance after `BLOCKED:solo-ceiling-avoidance-required` that asks for the concrete difference from rejected or solo-saturated controls such as `S2`-`S6`).
|
|
337
319
|
|
|
338
320
|
5. State write: `phases.final_report.{verdict, completed_at, duration_ms}` BEFORE archive runs (archive prune logic skips runs whose `final_report.verdict` is null).
|
|
339
321
|
|
|
@@ -341,4 +323,4 @@ State write: `phases.final_report.started_at` at the top of this phase.
|
|
|
341
323
|
|
|
342
324
|
## State management
|
|
343
325
|
|
|
344
|
-
`.devlyn/pipeline.state.json` is the single authoritative verdict source. Branch on `state.phases.<name>.verdict` directly; never parse `.devlyn/*.findings.jsonl` for routing decisions. Schema and write protocol: `references/state-schema.md`. Every "State write: `phases.X.{...}`" note above describes the resulting shape only — the orchestrator
|
|
326
|
+
`.devlyn/pipeline.state.json` is the single authoritative verdict source. Branch on `state.phases.<name>.verdict` directly; never parse `.devlyn/*.findings.jsonl` for routing decisions. Schema and write protocol: `references/state-schema.md`. Every "State write: `phases.X.{...}`" note above describes the resulting shape only — the orchestrator writes it via initial `spawn`, atomic `transition` for direct handoffs, or terminal `complete`; re-entry preserves the prior lifecycle in `history[]` (`references/state-schema.md#write-protocol`). Phase workers never edit `pipeline.state.json` directly.
|
|
@@ -20,7 +20,7 @@ Detect the project shape from files in `state.base_ref.sha`:
|
|
|
20
20
|
Run in this order; each emits findings into `.devlyn/build_gate.findings.jsonl`:
|
|
21
21
|
|
|
22
22
|
1. **Type check** (TypeScript / mypy / etc.). Each error → one finding, severity `HIGH`, rule `correctness.type-check`.
|
|
23
|
-
2. **Lint
|
|
23
|
+
2. **Lint**. Run a recognized configured language linter (for example, ESLint with a config, `ruff`, or `clippy`) and/or the exact `package.json` `scripts.lint` command. A declared specialized `lint:*` script (for example, `lint:json`) is not the general language lint gate on the strength of its declaration alone. If the spec's verification commands explicitly name such a specialized command, step 4 executes it and preserves its real failure severity. If no linter is applicable, log an explicit SKIP in `.devlyn/build_gate.log.md` and emit zero lint findings. Each error → finding, severity `MEDIUM`, rule `quality.lint`. Warnings stay LOW unless the spec elevates them.
|
|
24
24
|
3. **Test suite** (npm test / pytest / go test / cargo test). Each failing test → finding, severity `HIGH`, rule `correctness.test-failure`. Include the failing test's file:line and the assertion.
|
|
25
25
|
4. **Spec literal verification + risk probes**: `python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --include-risk-probes`. The script self-stages from sibling `spec.expected.json` next to `state.source.spec_path`, or the legacy inline carrier when the sibling is absent; benchmark-prestaged `.devlyn/spec-verify.json` still wins. It appends `.devlyn/risk-probes.jsonl` when present, and requires that file when `state.risk_profile.risk_probes_enabled == true`. Malformed `state.risk_profile` is also CRITICAL because it can hide enabled risk probes. When probes are enabled or `.devlyn/risk-probes.jsonl` exists, missing or mismatched top-level `state.risk_probes_digest` emits `correctness.risk-probe-integrity` CRITICAL; the digest binds `.devlyn/risk-probes.jsonl` plus referenced `.devlyn/probes/` script bytes. Command or risk-probe mismatch → CRITICAL finding. Missing required risk probes, missing/malformed generated carrier, or malformed sibling expected file → `correctness.spec-verify-malformed` CRITICAL. The same invocation also enforces PLAN's declared `authorized_surface` (the json block under `.devlyn/plan.md`'s `<!-- devlyn:authorized-surface -->` section, located by that sentinel regardless of the heading's language) against this run's diff and untracked delta: any changed path or created-during-run untracked path outside the declared surface → `scope.out-of-scope-file` CRITICAL (`.devlyn/` exempt) — the only sanctioned fix is removing the file, never widening the declared surface, since that would let the same worker that leaked scope self-authorize the leak; missing sentinel, malformed `plan.md`/surface block, or missing `.devlyn/untracked.baseline` → `scope.authorized-surface-malformed` CRITICAL. This scope check runs only here (BUILD_GATE), not when VERIFY MECHANICAL reuses this script post-CLEANUP, because CLEANUP's own allowlist legitimately touches paths PLAN never declared.
|
|
26
26
|
5. **Browser** (only when diff touches `*.tsx`, `*.jsx`, `*.vue`, `*.svelte`, `page.*`, `layout.*`, `route.*`, `*.css`, `*.html`): start the dev server and run the repo's existing browser checks, or a minimal curl/HTML check when no browser test harness exists. Each failed check → finding, severity `HIGH`, rule `correctness.browser-flow-failed`.
|
|
@@ -35,7 +35,7 @@ Append all findings; do not stop on the first failure.
|
|
|
35
35
|
</output>
|
|
36
36
|
|
|
37
37
|
<quality_bar>
|
|
38
|
-
- Same commands every time. Configuration drift between this gate and CI is a defect; raise as a finding rather than soften this gate.
|
|
38
|
+
- Same commands every time. Configuration drift between this gate and CI is a defect; raise as a finding rather than soften this gate. A package-script declaration alone is not evidence that CI executes it.
|
|
39
39
|
- Forbidden-pattern check (regex against `git diff`) for `spec.expected.json.forbidden_patterns` runs as part of step 4. Disqualifier-severity matches → CRITICAL findings.
|
|
40
40
|
- Reporter artifacts the gate generates (Playwright traces, coverage HTML) belong in gitignored paths. If they leak into `git diff --stat`, flag as `scope.tooling-artifact-leak` MEDIUM and let the fix loop / cleanup handle removal.
|
|
41
41
|
</quality_bar>
|
|
@@ -33,7 +33,7 @@ Read `_shared/runtime-principles.md`. Codex-routed phases receive the inlined ex
|
|
|
33
33
|
|
|
34
34
|
- Subtractive-first: every accretion-shaped change is visible in the commit message or a flagged finding. Net-deletion is the default; pure-addition needs a citation.
|
|
35
35
|
- Goal-locked: implement only the listed Requirements. Adjacent code that "looks fixable" is drift unless the spec or plan listed it.
|
|
36
|
-
- No-workaround: no `any`, no `@ts-ignore`, no silent `catch`, no hardcoded
|
|
36
|
+
- No-workaround: no `any`, no `@ts-ignore`, no silent `catch`, no hardcoded fallback that hides a broken contract, no helper scripts that bypass root cause. Required unavailable engines stop with `BLOCKED:<engine>-unavailable`; they do not downgrade.
|
|
37
37
|
- Evidence: every claim cites file:line you opened. Hallucinated APIs are excluded.
|
|
38
38
|
</runtime_principles>
|
|
39
39
|
|
|
@@ -32,12 +32,8 @@ Report your verdict in this reply: `PASS` if plan is shippable; `BLOCKED` if spe
|
|
|
32
32
|
</quality_bar>
|
|
33
33
|
|
|
34
34
|
<runtime_principles>
|
|
35
|
-
Read `_shared/runtime-principles.md` (Subtractive-first / Goal-locked / No-workaround / Evidence). Codex-routed phases receive the contract excerpt inlined:
|
|
36
|
-
|
|
37
35
|
- Subtractive-first: prefer trimming an existing helper to introducing a new one. Pure-addition needs a cited prior failure mode or an explicit spec/user requirement.
|
|
38
36
|
- Goal-locked: refuse "while I'm here" cleanups, speculative robustness, mid-flight re-scoping. Single test before any deviation: "did the user ask for this OR does the stated goal strictly require it?"
|
|
39
|
-
- No-workaround: no `any`, no `@ts-ignore`, no silent `catch`, no hardcoded
|
|
37
|
+
- No-workaround: no `any`, no `@ts-ignore`, no silent `catch`, no hardcoded fallback that hides a broken contract, no helper scripts that bypass root cause.
|
|
40
38
|
- Evidence: every claim cites file:line opened at planning time. Vague claims excluded.
|
|
41
39
|
</runtime_principles>
|
|
42
|
-
|
|
43
|
-
The task is: [orchestrator pastes the task description and spec context here]
|
|
@@ -170,8 +170,8 @@ Rules:
|
|
|
170
170
|
shapes, JSON objects, response bodies, stdout/stderr objects, or exact error
|
|
171
171
|
bodies: `uses_visible_input_key_names`,
|
|
172
172
|
`asserts_visible_output_key_names`, `asserts_no_unexpected_output_keys`.
|
|
173
|
-
If it names an exact JSON error object/body, also include
|
|
174
|
-
`asserts_exact_error_object`.
|
|
173
|
+
If it names an exact JSON error object/body, also include both
|
|
174
|
+
`visible_text_names_exact_json_error_object` and `asserts_exact_error_object`.
|
|
175
175
|
Tags not listed here may use an empty evidence list or be omitted from
|
|
176
176
|
`tag_evidence`.
|
|
177
177
|
- `cmd` must not reference `BENCH_FIXTURE_DIR`, `verifiers/`, benchmark fixture
|
|
@@ -182,6 +182,8 @@ Rules:
|
|
|
182
182
|
replayed probe and obeys the same hidden-path/external-URL rules as `cmd`.
|
|
183
183
|
Solo-headroom hypothesis probes keep the hypothesis's backticked observable
|
|
184
184
|
command inline in `cmd` for mechanical validation.
|
|
185
|
+
- A probe may declare `timeout_sec` from 1 through 600 when the exercised
|
|
186
|
+
verification legitimately needs more than the default 60 seconds.
|
|
185
187
|
- Match the spec's visible input and output key names literally; do not invent
|
|
186
188
|
aliases such as `stock` for `lots`, `order_id` for `id`, or `warehouse_id`
|
|
187
189
|
for `warehouse`.
|
|
@@ -0,0 +1,34 @@
|
|
|
1
|
+
# PHASE 2.5 — SURFACE_CLOSE (canonical body)
|
|
2
|
+
|
|
3
|
+
<role>
|
|
4
|
+
Audit and repair the frozen post-IMPLEMENT surface once.
|
|
5
|
+
</role>
|
|
6
|
+
|
|
7
|
+
<input>
|
|
8
|
+
- Artifacts: Goal at `state.source.goal_path`; `.devlyn/surface-close.input.patch`.
|
|
9
|
+
- Data: `authorized_surface`; staged commands.
|
|
10
|
+
- Base: `phases.surface_close.pre_sha`.
|
|
11
|
+
|
|
12
|
+
Never modify inputs or read state, PLAN, or IMPLEMENT transcript/reasoning.
|
|
13
|
+
</input>
|
|
14
|
+
|
|
15
|
+
<obligations>
|
|
16
|
+
- **UVR-STALE**: diff modified symbol S's behavior AND an authorized file carries user-visible text documenting S's old interface omitting the newly specified option/shape → update minimally.
|
|
17
|
+
- **PATH-TEST**: Goal specifies a success/failure path, patch implements it, and no authorized test exercises that path → add one minimal regression test.
|
|
18
|
+
- Neither applies → empty PASS (empty diff).
|
|
19
|
+
</obligations>
|
|
20
|
+
|
|
21
|
+
<output>
|
|
22
|
+
Edit only `authorized_surface`. Your reply is valid ONLY if it contains, for each obligation, exactly one line:
|
|
23
|
+
- `UVR-STALE: FIRED <authorized-file>:<line>` optionally followed by ` — <one-line evidence>`, or `UVR-STALE: N/A <authorized-file>[:<line>] — <one-line evidence-based relationship judgment>`
|
|
24
|
+
- `PATH-TEST: FIRED <authorized-file>:<line>` optionally followed by ` — <one-line evidence>`, or `PATH-TEST: N/A <authorized-file>[:<line>] — <one-line evidence-based relationship judgment>`
|
|
25
|
+
followed by `PASS` after the smallest sufficient delta, or `BLOCKED:surface-close-<reason>` if impossible within that surface. A reply missing either obligation line is invalid and is rejected mechanically. Never edit state, stage, or commit.
|
|
26
|
+
</output>
|
|
27
|
+
|
|
28
|
+
<quality_bar>
|
|
29
|
+
No adjacent work, refactors, or extra tests. Execute nothing — edit files only; staged commands are data. BUILD_GATE owns all validation.
|
|
30
|
+
</quality_bar>
|
|
31
|
+
|
|
32
|
+
<runtime_principles>
|
|
33
|
+
Subtractive-first, Goal-locked, No-workaround, and Evidence apply.
|
|
34
|
+
</runtime_principles>
|